* [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames
@ 2018-05-09 16:55 Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions Max Reitz
` (12 more replies)
0 siblings, 13 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
The “Try” in the subject of this series means more or less the same as
the “Try” in the subject line of
http://lists.nongnu.org/archive/html/qemu-block/2018-05/msg00061.html .
The difference is that it’s not so important here, because the worst
outcome is that in very narrow edge cases[1] (e.g. you trying to use
password-secret for rbd) we fail to generate a properly typed json:{}
filename -- but that is no regression over what we currently have. In
the vast majority of cases[1], this series will result in proper
typing.
Or, well, I should add that I’m not quite sure about all the edge
cases[1], solely because the current bdrv_refresh_filename() code is a
mess, especially when it comes to gathering the blockdev options. But
I do know that after my series to address that (“block: Fix some
filename generation issues”), it only comes down to a very small number
of edge cases (the only hard edge case is rbd’s =keyvalue-pairs, but
that is kaput anyway).[1]
But since this series does not have any hard requirement on that other
one, I decided to send it independently.
[1] You might have noticed a number of [1]s above. This is because the
above text is as it was in a version of this series with only five
patches in it. Then I noticed a not-so-edge case where converting to
QAPI broke down: qcow(2) encryption. Since qcow and qcow2 now support
both the old AES and luks encryption, and every encryption type may
offer its own runtime options, you now have a flat union there (under
“encrypt”) which is discriminated based on encrypt.format.
Now the qcow2 driver can of course detect the format from the image
header, so you don’t really have to specify it. In fact, when your
options don’t go through QAPI, you actually don’t have to. So this is
exactly the not-so-edge case I was talking about.
QAPI cannot parse encrypt before it knows encrypt.format. So it’s much
too late when the qcow2 driver detects encrypt.format in its
.bdrv_open() plementation.
I’m more than happy to take alternative suggestions, but this is an
issue we really cannot ignore (unlike =keyvalue-pairs), so the two
solutions I came up with are:
(1) Say auto-detection was a bad idea and make encrypt.format mandatory,
even without QAPI. This is always a possibility, but it does seem
kind of cumbersome to users, so I wanted to explore the other option
first, not knowing how deep into the rabbit hole that would lead.
(2) Allow flat union discriminators to be optional. Since both AES and
luks encryption in practice only allow a key-secret option, we can
add a new runtime encryption pseudo-type, which is “from-image” that
only allows such a key-secret option. Now, when the user does not
specify encrypt.format, the format is assumed to be “from-image”
which means that you can specify a key-secret and qcow/qcow2 will
figure out what encryption to use.
This allows to retain compatibility with the current interface, but
it also means this series exploded, and maybe the idea is
universally hated.
One issue I can see with this approach is how it would fare with new
encryption types added; specifically, what if we add some encryption
type that does not have a key-secret but something else? Do we add
that something else to from-image and make everything optional? Or
do we require users to explicitly specify the encryption type for
such a new type when it is not compatible with from-image?
In any case, one thing we have to watch out for from now on is that
drivers must not allow options to be optional that are mandatory in
the QAPI schema.
More generally, we have to make sure that the driver’s interface on
-drive is compatible to QAPI. But I guess everybody knows that, so
it’s good to be a bit more specific and say that this doesn’t just
mean that options must match. Mandatoriness must match also.
So, OK, obviously I chose solution (2) for now. What this means is that
in this series:
- The QAPI code generator is modified to allow optional discriminators
for flat unions. In such a case, a default-variant must be supplied
to choose when the discriminator value is not present on the wire.
- Accordingly, documentation, tests, and introspection are adjusted.
- This is used to make qcow’s and qcow2’s encrypt.format parameter
optional. It now defaults to “from-image” which is a new
pseudo-format that allows a key-secret to be given, and otherwise
leaves it to the format driver to determine the encryption format.
- qdict_stringify_for_keyval() is added, as already proposed/hinted at
in the rbd QAPI discussion. This includes movement of many
block-specific QDict function declarations into an own header, as
suggested by Markus.
- json:{} filenames are attempted to be typed correctly when they are
generated, by running bs->full_open_options through a healthy mix of
qdict_flatten(), qdict_stringify_for_keyval(), qdict_crumple(), the
keyval input visitor for BlockdevOptions, and the output visitor.
This may not always work but I hope it usually will. Fingers crossed.
(At least it won’t make things worse.)
- Tests, tests, tests.
(Yes, I know that “In this series tests, tests, tests.” is not a
sentence.)
Max Reitz (13):
qapi: Add default-variant for flat unions
docs/qapi: Document optional discriminators
tests: Add QAPI optional discriminator tests
qapi: Formalize qcow2 encryption probing
qapi: Formalize qcow encryption probing
block: Add block-specific QDict header
qdict: Add qdict_stringify_for_keyval()
tests: Add qdict_stringify_for_keyval() test
qdict: Make qdict_flatten() shallow-clone-friendly
tests: Add QDict clone-flatten test
block: Try to create well typed json:{} filenames
iotests: Test internal option typing
iotests: qcow2's encrypt.format is now optional
docs/devel/qapi-code-gen.txt | 21 +++++-
tests/Makefile.include | 3 +
qapi/block-core.json | 72 ++++++++++++++++--
qapi/introspect.json | 8 ++
...ion-optional-discriminator-invalid-default.json | 12 +++
...at-union-optional-discriminator-no-default.json | 11 +++
.../flat-union-optional-discriminator.json | 4 +-
.../flat-union-superfluous-default-variant.json | 11 +++
include/block/qdict.h | 37 +++++++++
include/qapi/qmp/qdict.h | 17 -----
block.c | 70 ++++++++++++++++-
block/gluster.c | 1 +
block/iscsi.c | 1 +
block/nbd.c | 1 +
block/nfs.c | 1 +
block/parallels.c | 1 +
block/qcow.c | 1 +
block/qcow2.c | 1 +
block/qed.c | 1 +
block/quorum.c | 1 +
block/rbd.c | 1 +
block/sheepdog.c | 1 +
block/snapshot.c | 1 +
block/ssh.c | 1 +
block/vhdx.c | 1 +
block/vpc.c | 1 +
block/vvfat.c | 1 +
block/vxhs.c | 1 +
blockdev.c | 1 +
qobject/qdict.c | 81 ++++++++++++++++++--
tests/check-qdict.c | 88 ++++++++++++++++++++++
tests/check-qobject.c | 1 +
tests/test-replication.c | 1 +
util/qemu-config.c | 1 +
scripts/qapi/common.py | 57 +++++++++++---
scripts/qapi/doc.py | 8 +-
scripts/qapi/introspect.py | 10 ++-
scripts/qapi/visit.py | 33 +++++++-
...nion-optional-discriminator-invalid-default.err | 1 +
...ion-optional-discriminator-invalid-default.exit | 1 +
...nion-optional-discriminator-invalid-default.out | 0
...lat-union-optional-discriminator-no-default.err | 1 +
...at-union-optional-discriminator-no-default.exit | 1 +
...lat-union-optional-discriminator-no-default.out | 0
.../flat-union-optional-discriminator.err | 1 -
.../flat-union-optional-discriminator.exit | 2 +-
.../flat-union-optional-discriminator.out | 15 ++++
.../flat-union-superfluous-default-variant.err | 1 +
.../flat-union-superfluous-default-variant.exit | 1 +
.../flat-union-superfluous-default-variant.out | 0
tests/qapi-schema/test-qapi.py | 2 +
tests/qemu-iotests/059.out | 2 +-
tests/qemu-iotests/087 | 2 -
tests/qemu-iotests/089 | 25 ++++++
tests/qemu-iotests/089.out | 9 +++
tests/qemu-iotests/099.out | 4 +-
tests/qemu-iotests/110.out | 2 +-
tests/qemu-iotests/198.out | 4 +-
tests/qemu-iotests/207.out | 10 +--
59 files changed, 580 insertions(+), 68 deletions(-)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.json
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.json
create mode 100644 include/block/qdict.h
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.out
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.exit
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.out
--
2.14.3
^ permalink raw reply [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 13:12 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators Max Reitz
` (11 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
This patch allows specifying a discriminator that is an optional member
of the base struct. In such a case, a default value must be provided
that is used when no value is given.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/introspect.json | 8 ++++++
scripts/qapi/common.py | 57 ++++++++++++++++++++++++++++++++++--------
scripts/qapi/doc.py | 8 ++++--
scripts/qapi/introspect.py | 10 +++++---
scripts/qapi/visit.py | 33 ++++++++++++++++++++++--
tests/qapi-schema/test-qapi.py | 2 ++
6 files changed, 101 insertions(+), 17 deletions(-)
diff --git a/qapi/introspect.json b/qapi/introspect.json
index c7f67b7d78..2d7b1e3745 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -168,6 +168,13 @@
# @tag: the name of the member serving as type tag.
# An element of @members with this name must exist.
#
+# @default-variant: if the @tag element of @members is optional, this
+# is the default value for choosing a variant. Its
+# value must be a valid value for @tag.
+# Present exactly when @tag is present and the
+# associated element of @members is optional.
+# (Since: 2.13)
+#
# @variants: variant members, i.e. additional members that
# depend on the type tag's value. Present exactly when
# @tag is present. The variants are in no particular order,
@@ -181,6 +188,7 @@
{ 'struct': 'SchemaInfoObject',
'data': { 'members': [ 'SchemaInfoObjectMember' ],
'*tag': 'str',
+ '*default-variant': 'str',
'*variants': [ 'SchemaInfoObjectVariant' ] } }
##
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index a032cec375..fbf0244f73 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -721,6 +721,7 @@ def check_union(expr, info):
name = expr['union']
base = expr.get('base')
discriminator = expr.get('discriminator')
+ default_variant = expr.get('default-variant')
members = expr['data']
# Two types of unions, determined by discriminator.
@@ -745,16 +746,37 @@ def check_union(expr, info):
base_members = find_base_members(base)
assert base_members is not None
- # The value of member 'discriminator' must name a non-optional
- # member of the base struct.
+ # The value of member 'discriminator' must name a member of
+ # the base struct.
check_name(info, "Discriminator of flat union '%s'" % name,
discriminator)
- discriminator_type = base_members.get(discriminator)
- if not discriminator_type:
- raise QAPISemError(info,
- "Discriminator '%s' is not a member of base "
- "struct '%s'"
- % (discriminator, base))
+ if default_variant is None:
+ discriminator_type = base_members.get(discriminator)
+ if not discriminator_type:
+ if base_members.get('*' + discriminator) is None:
+ raise QAPISemError(info,
+ "Discriminator '%s' is not a member of "
+ "base struct '%s'"
+ % (discriminator, base))
+ else:
+ raise QAPISemError(info,
+ "Default variant must be specified for "
+ "optional discriminator '%s'"
+ % discriminator)
+ else:
+ discriminator_type = base_members.get('*' + discriminator)
+ if not discriminator_type:
+ if base_members.get(discriminator) is None:
+ raise QAPISemError(info,
+ "Discriminator '%s' is not a member of "
+ "base struct '%s'"
+ % (discriminator, base))
+ else:
+ raise QAPISemError(info,
+ "Must not specify a default variant for "
+ "non-optional discriminator '%s'"
+ % discriminator)
+
enum_define = enum_types.get(discriminator_type)
allow_metas = ['struct']
# Do not allow string discriminator
@@ -763,6 +785,15 @@ def check_union(expr, info):
"Discriminator '%s' must be of enumeration "
"type" % discriminator)
+ if default_variant is not None:
+ # Must be a value of the enumeration
+ if default_variant not in enum_define['data']:
+ raise QAPISemError(info,
+ "Default variant '%s' of flat union '%s' is "
+ "not part of '%s'"
+ % (default_variant, name,
+ discriminator_type))
+
# Check every branch; don't allow an empty union
if len(members) == 0:
raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
@@ -909,7 +940,7 @@ def check_exprs(exprs):
elif 'union' in expr:
meta = 'union'
check_keys(expr_elem, 'union', ['data'],
- ['base', 'discriminator'])
+ ['base', 'discriminator', 'default-variant'])
union_types[expr[meta]] = expr
elif 'alternate' in expr:
meta = 'alternate'
@@ -1335,12 +1366,14 @@ class QAPISchemaObjectTypeMember(QAPISchemaMember):
class QAPISchemaObjectTypeVariants(object):
- def __init__(self, tag_name, tag_member, variants):
+ def __init__(self, tag_name, tag_member, default_tag_value, variants):
# Flat unions pass tag_name but not tag_member.
# Simple unions and alternates pass tag_member but not tag_name.
# After check(), tag_member is always set, and tag_name remains
# a reliable witness of being used by a flat union.
assert bool(tag_member) != bool(tag_name)
+ # default_tag_value is only passed for flat unions.
+ assert bool(tag_name) or not bool(default_tag_value)
assert (isinstance(tag_name, str) or
isinstance(tag_member, QAPISchemaObjectTypeMember))
assert len(variants) > 0
@@ -1348,6 +1381,7 @@ class QAPISchemaObjectTypeVariants(object):
assert isinstance(v, QAPISchemaObjectTypeVariant)
self._tag_name = tag_name
self.tag_member = tag_member
+ self.default_tag_value = default_tag_value
self.variants = variants
def set_owner(self, name):
@@ -1637,6 +1671,7 @@ class QAPISchema(object):
data = expr['data']
base = expr.get('base')
tag_name = expr.get('discriminator')
+ default_tag_value = expr.get('default-variant')
tag_member = None
if isinstance(base, dict):
base = (self._make_implicit_object_type(
@@ -1656,6 +1691,7 @@ class QAPISchema(object):
QAPISchemaObjectType(name, info, doc, base, members,
QAPISchemaObjectTypeVariants(tag_name,
tag_member,
+ default_tag_value,
variants)))
def _def_alternate_type(self, expr, info, doc):
@@ -1668,6 +1704,7 @@ class QAPISchema(object):
QAPISchemaAlternateType(name, info, doc,
QAPISchemaObjectTypeVariants(None,
tag_member,
+ None,
variants)))
def _def_command(self, expr, info, doc):
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 9b312b2c51..91204dc4c6 100644
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -160,8 +160,12 @@ def texi_members(doc, what, base, variants, member_func):
items += '@item The members of @code{%s}\n' % base.doc_type()
if variants:
for v in variants.variants:
- when = ' when @code{%s} is @t{"%s"}' % (
- variants.tag_member.name, v.name)
+ if v.name == variants.default_tag_value:
+ when = ' when @code{%s} is @t{"%s"} or not given' % (
+ variants.tag_member.name, v.name)
+ else:
+ when = ' when @code{%s} is @t{"%s"}' % (
+ variants.tag_member.name, v.name)
if v.type.is_implicit():
assert not v.type.base and not v.type.variants
for m in v.type.local_members:
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index f9e67e8227..2d1d4e320a 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -142,9 +142,12 @@ const QLitObject %(c_name)s = %(c_string)s;
ret['default'] = None
return ret
- def _gen_variants(self, tag_name, variants):
- return {'tag': tag_name,
- 'variants': [self._gen_variant(v) for v in variants]}
+ def _gen_variants(self, tag_name, default_variant, variants):
+ ret = {'tag': tag_name,
+ 'variants': [self._gen_variant(v) for v in variants]}
+ if default_variant:
+ ret['default-variant'] = default_variant
+ return ret
def _gen_variant(self, variant):
return {'case': variant.name, 'type': self._use_type(variant.type)}
@@ -163,6 +166,7 @@ const QLitObject %(c_name)s = %(c_string)s;
obj = {'members': [self._gen_member(m) for m in members]}
if variants:
obj.update(self._gen_variants(variants.tag_member.name,
+ variants.default_tag_value,
variants.variants))
self._gen_qlit(name, 'object', obj)
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 5d72d8936c..ecffc46bd3 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members, variants):
void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
{
Error *err = NULL;
-
''',
c_name=c_name(name))
+ if variants:
+ ret += mcgen('''
+ %(c_type)s %(c_name)s;
+''',
+ c_type=variants.tag_member.type.c_name(),
+ c_name=c_name(variants.tag_member.name))
+
+ ret += mcgen('''
+
+''')
+
if base:
ret += mcgen('''
visit_type_%(c_type)s_members(v, (%(c_type)s *)obj, &err);
@@ -75,8 +85,27 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
''')
if variants:
+ if variants.default_tag_value is None:
+ ret += mcgen('''
+ %(c_name)s = obj->%(c_name)s;
+''',
+ c_name=c_name(variants.tag_member.name))
+ else:
+ ret += mcgen('''
+ if (obj->has_%(c_name)s) {
+ %(c_name)s = obj->%(c_name)s;
+ } else {
+ %(c_name)s = %(enum_const)s;
+ }
+''',
+ c_name=c_name(variants.tag_member.name),
+ enum_const=c_enum_const(
+ variants.tag_member.type.name,
+ variants.default_tag_value,
+ variants.tag_member.type.prefix))
+
ret += mcgen('''
- switch (obj->%(c_name)s) {
+ switch (%(c_name)s) {
''',
c_name=c_name(variants.tag_member.name))
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index c1a144ba29..f2a072b92e 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -56,6 +56,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
def _print_variants(variants):
if variants:
print(' tag %s' % variants.tag_member.name)
+ if variants.default_tag_value:
+ print(' default variant: %s' % variants.default_tag_value)
for v in variants.variants:
print(' case %s: %s' % (v.name, v.type.name))
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 13:14 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests Max Reitz
` (10 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
docs/devel/qapi-code-gen.txt | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index b9b6eabd08..35b9f0d99d 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -472,8 +472,8 @@ All branches of the union must be complex types, and the top-level
members of the union dictionary on the wire will be combination of
members from both the base type and the appropriate branch type (when
merging two dictionaries, there must be no keys in common). The
-'discriminator' member must be the name of a non-optional enum-typed
-member of the base struct.
+'discriminator' member must be the name of an enum-typed member of the
+base struct.
The following example enhances the above simple union example by
adding an optional common member 'read-only', renaming the
@@ -502,6 +502,23 @@ the enum). In the resulting generated C data types, a flat union is
represented as a struct with the base members included directly, and
then a union of structures for each branch of the struct.
+If the discriminator points to an optional member of the base struct,
+its default value must be specified as a 'default-variant'. In the
+following example, the above BlockDriver struct is changed so it
+defaults to the 'file' driver if that field is omitted on the wire:
+
+ { 'union': 'BlockdevOptions',
+ 'base': { '*driver': 'BlockdevDriver', '*read-only': 'bool' },
+ 'discriminator': 'driver',
+ 'default-variant': 'file',
+ 'data': { 'file': 'BlockdevOptionsFile',
+ 'qcow2': 'BlockdevOptionsQcow2' } }
+
+Now the 'file' JSON object can be abbreviated to:
+
+ { "read-only": "true",
+ "filename": "/some/place/my-image" }
+
A simple union can always be re-written as a flat union where the base
class has a single member named 'type', and where each branch of the
union has a struct with a single member named 'data'. That is,
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 14:08 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing Max Reitz
` (9 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
There already is an optional discriminator test, although it also noted
the discriminator name itself as optional. Changing that, and adding a
default-variant field, makes that test pass instead of failing.
Besides this one, a number of tests adding various error cases are
added.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/Makefile.include | 3 +++
...flat-union-optional-discriminator-invalid-default.json | 12 ++++++++++++
.../flat-union-optional-discriminator-no-default.json | 11 +++++++++++
tests/qapi-schema/flat-union-optional-discriminator.json | 4 ++--
.../flat-union-superfluous-default-variant.json | 11 +++++++++++
.../flat-union-optional-discriminator-invalid-default.err | 1 +
...flat-union-optional-discriminator-invalid-default.exit | 1 +
.../flat-union-optional-discriminator-invalid-default.out | 0
.../flat-union-optional-discriminator-no-default.err | 1 +
.../flat-union-optional-discriminator-no-default.exit | 1 +
.../flat-union-optional-discriminator-no-default.out | 0
tests/qapi-schema/flat-union-optional-discriminator.err | 1 -
tests/qapi-schema/flat-union-optional-discriminator.exit | 2 +-
tests/qapi-schema/flat-union-optional-discriminator.out | 15 +++++++++++++++
.../flat-union-superfluous-default-variant.err | 1 +
.../flat-union-superfluous-default-variant.exit | 1 +
.../flat-union-superfluous-default-variant.out | 0
17 files changed, 61 insertions(+), 4 deletions(-)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.json
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.out
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.exit
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.out
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 3b9a5e31a2..d01df12b09 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -500,7 +500,10 @@ qapi-schema += flat-union-invalid-branch-key.json
qapi-schema += flat-union-invalid-discriminator.json
qapi-schema += flat-union-no-base.json
qapi-schema += flat-union-optional-discriminator.json
+qapi-schema += flat-union-optional-discriminator-no-default.json
+qapi-schema += flat-union-optional-discriminator-invalid-default.json
qapi-schema += flat-union-string-discriminator.json
+qapi-schema += flat-union-superfluous-default-variant.json
qapi-schema += funny-char.json
qapi-schema += ident-with-escape.json
qapi-schema += include-before-err.json
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
new file mode 100644
index 0000000000..015a47ba52
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
@@ -0,0 +1,12 @@
+# default-variant must refer to an actual value of the discriminator's
+# enum
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'struct': 'Base',
+ 'data': { '*switch': 'Enum' } }
+{ 'struct': 'Branch', 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': 'switch',
+ 'default-variant': 'three',
+ 'data': { 'one': 'Branch',
+ 'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-no-default.json b/tests/qapi-schema/flat-union-optional-discriminator-no-default.json
new file mode 100644
index 0000000000..0d612f5a62
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-no-default.json
@@ -0,0 +1,11 @@
+# Using an optional discriminator requires specifying a default
+# variant
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'struct': 'Base',
+ 'data': { '*switch': 'Enum' } }
+{ 'struct': 'Branch', 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': 'switch',
+ 'data': { 'one': 'Branch',
+ 'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.json b/tests/qapi-schema/flat-union-optional-discriminator.json
index 08a8f7ef8b..68e87083a1 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.json
+++ b/tests/qapi-schema/flat-union-optional-discriminator.json
@@ -1,10 +1,10 @@
-# we require the discriminator to be non-optional
{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
{ 'struct': 'Base',
'data': { '*switch': 'Enum' } }
{ 'struct': 'Branch', 'data': { 'name': 'str' } }
{ 'union': 'MyUnion',
'base': 'Base',
- 'discriminator': '*switch',
+ 'discriminator': 'switch',
+ 'default-variant': 'one',
'data': { 'one': 'Branch',
'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-superfluous-default-variant.json b/tests/qapi-schema/flat-union-superfluous-default-variant.json
new file mode 100644
index 0000000000..8558165868
--- /dev/null
+++ b/tests/qapi-schema/flat-union-superfluous-default-variant.json
@@ -0,0 +1,11 @@
+# default-variant only makes sense with an optional discriminator
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'struct': 'Base',
+ 'data': { 'switch': 'Enum' } }
+{ 'struct': 'Branch', 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': 'switch',
+ 'default-variant': 'one',
+ 'data': { 'one': 'Branch',
+ 'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
new file mode 100644
index 0000000000..b6bd3423d6
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json:7: Default variant 'three' of flat union 'MyUnion' is not part of 'Enum'
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out b/tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-no-default.err b/tests/qapi-schema/flat-union-optional-discriminator-no-default.err
new file mode 100644
index 0000000000..1342efd9e8
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-no-default.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-optional-discriminator-no-default.json:7: Default variant must be specified for optional discriminator 'switch'
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-no-default.exit b/tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-optional-discriminator-no-default.out b/tests/qapi-schema/flat-union-optional-discriminator-no-default.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.err b/tests/qapi-schema/flat-union-optional-discriminator.err
index aaabedb3bd..e69de29bb2 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.err
+++ b/tests/qapi-schema/flat-union-optional-discriminator.err
@@ -1 +0,0 @@
-tests/qapi-schema/flat-union-optional-discriminator.json:6: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.exit b/tests/qapi-schema/flat-union-optional-discriminator.exit
index d00491fd7e..573541ac97 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.exit
+++ b/tests/qapi-schema/flat-union-optional-discriminator.exit
@@ -1 +1 @@
-1
+0
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.out b/tests/qapi-schema/flat-union-optional-discriminator.out
index e69de29bb2..d3a72f63c2 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.out
+++ b/tests/qapi-schema/flat-union-optional-discriminator.out
@@ -0,0 +1,15 @@
+object q_empty
+enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+ prefix QTYPE
+module flat-union-optional-discriminator.json
+enum Enum ['one', 'two']
+object Base
+ member switch: Enum optional=True
+object Branch
+ member name: str optional=False
+object MyUnion
+ base Base
+ tag switch
+ default variant: one
+ case one: Branch
+ case two: Branch
diff --git a/tests/qapi-schema/flat-union-superfluous-default-variant.err b/tests/qapi-schema/flat-union-superfluous-default-variant.err
new file mode 100644
index 0000000000..5230bbf645
--- /dev/null
+++ b/tests/qapi-schema/flat-union-superfluous-default-variant.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-superfluous-default-variant.json:6: Must not specify a default variant for non-optional discriminator 'switch'
diff --git a/tests/qapi-schema/flat-union-superfluous-default-variant.exit b/tests/qapi-schema/flat-union-superfluous-default-variant.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/flat-union-superfluous-default-variant.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-superfluous-default-variant.out b/tests/qapi-schema/flat-union-superfluous-default-variant.out
new file mode 100644
index 0000000000..e69de29bb2
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (2 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 7:58 ` Daniel P. Berrangé
2018-05-09 16:55 ` [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow " Max Reitz
` (8 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
Currently, you can give no encryption format for a qcow2 file while
still passing a key-secret. That does not conform to the schema, so
this patch changes the schema to allow it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 44 ++++++++++++++++++++++++++++++++++++++++----
1 file changed, 40 insertions(+), 4 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 71c9ab8538..092a1aba2d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -43,6 +43,19 @@
{ 'struct': 'ImageInfoSpecificQCow2EncryptionBase',
'data': { 'format': 'BlockdevQcow2EncryptionFormat'}}
+##
+# @ImageInfoSpecificQCow2EncryptionNoInfo:
+#
+# Only used for the qcow2 encryption format "from-image" in which the
+# actual encryption format is determined from the image header.
+# Therefore, this encryption format will never be reported in
+# ImageInfoSpecificQCow2Encryption.
+#
+# Since: 2.13
+##
+{ 'struct': 'ImageInfoSpecificQCow2EncryptionNoInfo',
+ 'data': { } }
+
##
# @ImageInfoSpecificQCow2Encryption:
#
@@ -52,7 +65,8 @@
'base': 'ImageInfoSpecificQCow2EncryptionBase',
'discriminator': 'format',
'data': { 'aes': 'QCryptoBlockInfoQCow',
- 'luks': 'QCryptoBlockInfoLUKS' } }
+ 'luks': 'QCryptoBlockInfoLUKS',
+ 'from-image': 'ImageInfoSpecificQCow2EncryptionNoInfo' } }
##
# @ImageInfoSpecificQCow2:
@@ -2739,10 +2753,30 @@
# @BlockdevQcow2EncryptionFormat:
# @aes: AES-CBC with plain64 initialization venctors
#
+# @from-image: Determine the encryption format from the image
+# header. This only allows the use of the
+# key-secret option. (Since: 2.13)
+#
# Since: 2.10
##
{ 'enum': 'BlockdevQcow2EncryptionFormat',
- 'data': [ 'aes', 'luks' ] }
+ 'data': [ 'aes', 'luks', 'from-image' ] }
+
+##
+# @BlockdevQcow2EncryptionSecret:
+#
+# Allows specifying a key-secret without specifying the exact
+# encryption format, which is determined automatically from the image
+# header.
+#
+# @key-secret: The ID of a QCryptoSecret object providing the
+# decryption key. Mandatory except when probing
+# image for metadata only.
+#
+# Since: 2.13
+##
+{ 'struct': 'BlockdevQcow2EncryptionSecret',
+ 'data': { '*key-secret': 'str' } }
##
# @BlockdevQcow2Encryption:
@@ -2750,10 +2784,12 @@
# Since: 2.10
##
{ 'union': 'BlockdevQcow2Encryption',
- 'base': { 'format': 'BlockdevQcow2EncryptionFormat' },
+ 'base': { '*format': 'BlockdevQcow2EncryptionFormat' },
'discriminator': 'format',
+ 'default-variant': 'from-image',
'data': { 'aes': 'QCryptoBlockOptionsQCow',
- 'luks': 'QCryptoBlockOptionsLUKS'} }
+ 'luks': 'QCryptoBlockOptionsLUKS',
+ 'from-image': 'BlockdevQcow2EncryptionSecret' } }
##
# @BlockdevOptionsQcow2:
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow encryption probing
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (3 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 14:24 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header Max Reitz
` (7 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
Currently, you can give no encryption format for a qcow file while still
passing a key-secret. That does not conform to the schema, so this
patch changes the schema to allow it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qapi/block-core.json | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 092a1aba2d..d98af0a71d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2712,15 +2712,35 @@
'data': { 'flags': 'Qcow2OverlapCheckFlags',
'mode': 'Qcow2OverlapCheckMode' } }
+##
+# @BlockdevQcowEncryptionSecret:
+#
+# Allows specifying a key-secret without specifying the exact
+# encryption format, which is determined automatically from the image
+# header.
+#
+# @key-secret: The ID of a QCryptoSecret object providing the
+# decryption key. Mandatory except when probing
+# image for metadata only.
+#
+# Since: 2.13
+##
+{ 'struct': 'BlockdevQcowEncryptionSecret',
+ 'data': { '*key-secret': 'str' } }
+
##
# @BlockdevQcowEncryptionFormat:
#
# @aes: AES-CBC with plain64 initialization vectors
#
+# @from-image: Determine the encryption format from the image
+# header. This only allows the use of the
+# key-secret option. (Since: 2.13)
+#
# Since: 2.10
##
{ 'enum': 'BlockdevQcowEncryptionFormat',
- 'data': [ 'aes' ] }
+ 'data': [ 'aes', 'from-image' ] }
##
# @BlockdevQcowEncryption:
@@ -2728,9 +2748,11 @@
# Since: 2.10
##
{ 'union': 'BlockdevQcowEncryption',
- 'base': { 'format': 'BlockdevQcowEncryptionFormat' },
+ 'base': { '*format': 'BlockdevQcowEncryptionFormat' },
'discriminator': 'format',
- 'data': { 'aes': 'QCryptoBlockOptionsQCow' } }
+ 'default-variant': 'from-image',
+ 'data': { 'aes': 'QCryptoBlockOptionsQCow',
+ 'from-image': 'BlockdevQcowEncryptionSecret' } }
##
# @BlockdevOptionsQcow:
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (4 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow " Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 14:54 ` Eric Blake
2018-06-06 13:17 ` Markus Armbruster
2018-05-09 16:55 ` [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval() Max Reitz
` (6 subsequent siblings)
12 siblings, 2 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
There are numerous QDict functions that have been introduced for and are
used only by the block layer. Move their declarations into an own
header file to reflect that.
While qdict_extract_subqdict() is in fact used outside of the block
layer (in util/qemu-config.c), it is still a function related very
closely to how the block layer works with nested QDicts, namely by
sometimes flattening them. Therefore, its declaration is put into this
header as well and util/qemu-config.c includes it with a comment stating
exactly which function it needs.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/qdict.h | 35 +++++++++++++++++++++++++++++++++++
include/qapi/qmp/qdict.h | 17 -----------------
block.c | 1 +
block/gluster.c | 1 +
block/iscsi.c | 1 +
block/nbd.c | 1 +
block/nfs.c | 1 +
block/parallels.c | 1 +
block/qcow.c | 1 +
block/qcow2.c | 1 +
block/qed.c | 1 +
block/quorum.c | 1 +
block/rbd.c | 1 +
block/sheepdog.c | 1 +
block/snapshot.c | 1 +
block/ssh.c | 1 +
block/vhdx.c | 1 +
block/vpc.c | 1 +
block/vvfat.c | 1 +
block/vxhs.c | 1 +
blockdev.c | 1 +
qobject/qdict.c | 1 +
tests/check-qdict.c | 1 +
tests/check-qobject.c | 1 +
tests/test-replication.c | 1 +
util/qemu-config.c | 1 +
26 files changed, 59 insertions(+), 17 deletions(-)
create mode 100644 include/block/qdict.h
diff --git a/include/block/qdict.h b/include/block/qdict.h
new file mode 100644
index 0000000000..825e096a72
--- /dev/null
+++ b/include/block/qdict.h
@@ -0,0 +1,35 @@
+/*
+ * Special QDict functions used by the block layer
+ *
+ * Copyright (c) 2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#ifndef BLOCK_QDICT_H
+#define BLOCK_QDICT_H
+
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qobject.h"
+#include "qapi/error.h"
+
+
+void qdict_copy_default(QDict *dst, QDict *src, const char *key);
+void qdict_set_default_str(QDict *dst, const char *key, const char *val);
+
+void qdict_join(QDict *dest, QDict *src, bool overwrite);
+
+void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
+void qdict_array_split(QDict *src, QList **dst);
+int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(const QDict *src, Error **errp);
+void qdict_flatten(QDict *qdict);
+
+typedef struct QDictRenames {
+ const char *from;
+ const char *to;
+} QDictRenames;
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
+
+#endif
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 921a28d2d3..7f3ec10a10 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -67,23 +67,6 @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key,
bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value);
const char *qdict_get_try_str(const QDict *qdict, const char *key);
-void qdict_copy_default(QDict *dst, QDict *src, const char *key);
-void qdict_set_default_str(QDict *dst, const char *key, const char *val);
-
QDict *qdict_clone_shallow(const QDict *src);
-void qdict_flatten(QDict *qdict);
-
-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
-void qdict_array_split(QDict *src, QList **dst);
-int qdict_array_entries(QDict *src, const char *subqdict);
-QObject *qdict_crumple(const QDict *src, Error **errp);
-
-void qdict_join(QDict *dest, QDict *src, bool overwrite);
-
-typedef struct QDictRenames {
- const char *from;
- const char *to;
-} QDictRenames;
-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
#endif /* QDICT_H */
diff --git a/block.c b/block.c
index 676e57f562..3c3e8fd11d 100644
--- a/block.c
+++ b/block.c
@@ -27,6 +27,7 @@
#include "block/block_int.h"
#include "block/blockjob.h"
#include "block/nbd.h"
+#include "block/qdict.h"
#include "qemu/error-report.h"
#include "module_block.h"
#include "qemu/module.h"
diff --git a/block/gluster.c b/block/gluster.c
index 9900b6420c..b5fe7f3e87 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -11,6 +11,7 @@
#include "qemu/osdep.h"
#include <glusterfs/api/glfs.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
diff --git a/block/iscsi.c b/block/iscsi.c
index 3fd7203916..d70e8aea84 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -33,6 +33,7 @@
#include "qemu/bitops.h"
#include "qemu/bitmap.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "scsi/constants.h"
#include "qemu/iov.h"
#include "qemu/option.h"
diff --git a/block/nbd.c b/block/nbd.c
index 3e1693cc55..f499830410 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -28,6 +28,7 @@
#include "qemu/osdep.h"
#include "block/nbd-client.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/uri.h"
#include "block/block_int.h"
diff --git a/block/nfs.c b/block/nfs.c
index 66fddf12d4..5159ef0aa7 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -29,6 +29,7 @@
#include "qemu/error-report.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "trace.h"
#include "qemu/iov.h"
#include "qemu/option.h"
diff --git a/block/parallels.c b/block/parallels.c
index 6e9c37f44e..c1d9643498 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -31,6 +31,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/qcow.c b/block/qcow.c
index 3ba2ca25ea..e34eedc903 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -26,6 +26,7 @@
#include "qapi/error.h"
#include "qemu/error-report.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/qcow2.c b/block/qcow2.c
index 6d532470a8..416ab54298 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -24,6 +24,7 @@
#include "qemu/osdep.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include <zlib.h>
diff --git a/block/qed.c b/block/qed.c
index 65cfe92393..324a953cbc 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -13,6 +13,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/timer.h"
#include "qemu/bswap.h"
diff --git a/block/quorum.c b/block/quorum.c
index e448d7e384..0d90a02583 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -17,6 +17,7 @@
#include "qemu/cutils.h"
#include "qemu/option.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qapi-events-block.h"
#include "qapi/qmp/qdict.h"
diff --git a/block/rbd.c b/block/rbd.c
index a16431e267..ad5fd30b43 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -18,6 +18,7 @@
#include "qemu/error-report.h"
#include "qemu/option.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "crypto/secret.h"
#include "qemu/cutils.h"
#include "qapi/qmp/qstring.h"
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 4237132419..bf7ab896d2 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -24,6 +24,7 @@
#include "qemu/option.h"
#include "qemu/sockets.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/bitops.h"
#include "qemu/cutils.h"
diff --git a/block/snapshot.c b/block/snapshot.c
index 2953d96c06..f9903bc94e 100644
--- a/block/snapshot.c
+++ b/block/snapshot.c
@@ -25,6 +25,7 @@
#include "qemu/osdep.h"
#include "block/snapshot.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
diff --git a/block/ssh.c b/block/ssh.c
index 4c4fa3ccfc..eec37dd27c 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -28,6 +28,7 @@
#include <libssh2_sftp.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/error-report.h"
#include "qemu/option.h"
diff --git a/block/vhdx.c b/block/vhdx.c
index 0b1e21c750..b409acfeb2 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -18,6 +18,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/vpc.c b/block/vpc.c
index 0ebfcd3cc8..41c8c980f1 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -26,6 +26,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/vvfat.c b/block/vvfat.c
index 662dca0114..4595f335b8 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -27,6 +27,7 @@
#include <dirent.h>
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qemu/module.h"
#include "qemu/option.h"
#include "qemu/bswap.h"
diff --git a/block/vxhs.c b/block/vxhs.c
index 339e23218d..0cb0a007e9 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -12,6 +12,7 @@
#include <qnio/qnio_api.h>
#include <sys/param.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
diff --git a/blockdev.c b/blockdev.c
index 3808b1fc00..19c04d9276 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -35,6 +35,7 @@
#include "sysemu/blockdev.h"
#include "hw/block/block.h"
#include "block/blockjob.h"
+#include "block/qdict.h"
#include "block/throttle-groups.h"
#include "monitor/monitor.h"
#include "qemu/error-report.h"
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 22800eeceb..0554c64553 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -11,6 +11,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qbool.h"
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index eba5d3528e..93e2112b6d 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -11,6 +11,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnum.h"
diff --git a/tests/check-qobject.c b/tests/check-qobject.c
index 5cb08fcb63..16ccbde82c 100644
--- a/tests/check-qobject.c
+++ b/tests/check-qobject.c
@@ -8,6 +8,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
diff --git a/tests/test-replication.c b/tests/test-replication.c
index 68c0d04f2a..c8165ae954 100644
--- a/tests/test-replication.c
+++ b/tests/test-replication.c
@@ -15,6 +15,7 @@
#include "qemu/option.h"
#include "replication.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#define IMG_SIZE (64 * 1024 * 1024)
diff --git a/util/qemu-config.c b/util/qemu-config.c
index 14d84022dc..9d2e278e29 100644
--- a/util/qemu-config.c
+++ b/util/qemu-config.c
@@ -1,4 +1,5 @@
#include "qemu/osdep.h"
+#include "block/qdict.h" /* for qdict_extract_subqdict() */
#include "qapi/error.h"
#include "qapi/qapi-commands-misc.h"
#include "qapi/qmp/qdict.h"
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval()
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (5 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-11 18:39 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test Max Reitz
` (5 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
The purpose of this function is to prepare a QDict for consumption by
the keyval visitor, which only accepts strings and QNull.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/block/qdict.h | 2 ++
qobject/qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 59 insertions(+)
diff --git a/include/block/qdict.h b/include/block/qdict.h
index 825e096a72..05a24677b5 100644
--- a/include/block/qdict.h
+++ b/include/block/qdict.h
@@ -32,4 +32,6 @@ typedef struct QDictRenames {
} QDictRenames;
bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
+void qdict_stringify_for_keyval(QDict *qdict);
+
#endif
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 0554c64553..83678c4951 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -1087,3 +1087,60 @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
}
return true;
}
+
+/**
+ * Convert all values in a QDict so it can be consumed by the keyval
+ * input visitor. QNull values are left as-is, all other values are
+ * converted to strings.
+ *
+ * @qdict must be flattened, i.e. it may not contain any nested QDicts
+ * or QLists.
+ */
+void qdict_stringify_for_keyval(QDict *qdict)
+{
+ const QDictEntry *e;
+
+ for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+ QString *new_value = NULL;
+
+ switch (qobject_type(e->value)) {
+ case QTYPE_QNULL:
+ /* leave as-is */
+ break;
+
+ case QTYPE_QNUM: {
+ char *str = qnum_to_string(qobject_to(QNum, e->value));
+ new_value = qstring_from_str(str);
+ g_free(str);
+ break;
+ }
+
+ case QTYPE_QSTRING:
+ /* leave as-is */
+ break;
+
+ case QTYPE_QDICT:
+ case QTYPE_QLIST:
+ /* @qdict must be flattened */
+ abort();
+
+ case QTYPE_QBOOL:
+ if (qbool_get_bool(qobject_to(QBool, e->value))) {
+ new_value = qstring_from_str("on");
+ } else {
+ new_value = qstring_from_str("off");
+ }
+ break;
+
+ case QTYPE_NONE:
+ case QTYPE__MAX:
+ abort();
+ }
+
+ if (new_value) {
+ QDictEntry *mut_e = (QDictEntry *)e;
+ qobject_unref(mut_e->value);
+ mut_e->value = QOBJECT(new_value);
+ }
+ }
+}
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (6 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval() Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-10 16:02 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly Max Reitz
` (4 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/check-qdict.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 93e2112b6d..ef5d17f815 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -973,6 +973,57 @@ static void qdict_stress_test(void)
qobject_unref(qdict);
}
+static void qdict_stringify_for_keyval_test(void)
+{
+ QDict *dict = qdict_new();
+
+ /*
+ * Test stringification of:
+ *
+ * {
+ * "a": "null",
+ * "b": 42,
+ * "c": -23,
+ * "d": false,
+ * "e": null,
+ * "f": "",
+ * "g": 0.5,
+ * "h": 0xffffffffffffffff,
+ * "i": true,
+ * "j": 0
+ * }
+ *
+ * Note that null is not transformed into a string and remains a
+ * QNull.
+ */
+
+ qdict_put_str(dict, "a", "null");
+ qdict_put_int(dict, "b", 42);
+ qdict_put_int(dict, "c", -23);
+ qdict_put_bool(dict, "d", false);
+ qdict_put_null(dict, "e");
+ qdict_put_str(dict, "f", "");
+ qdict_put(dict, "g", qnum_from_double(0.5));
+ qdict_put(dict, "h", qnum_from_uint(0xffffffffffffffffull));
+ qdict_put_bool(dict, "i", true);
+ qdict_put_int(dict, "j", 0);
+
+ qdict_stringify_for_keyval(dict);
+
+ g_assert(!strcmp(qdict_get_str(dict, "a"), "null"));
+ g_assert(!strcmp(qdict_get_str(dict, "b"), "42"));
+ g_assert(!strcmp(qdict_get_str(dict, "c"), "-23"));
+ g_assert(!strcmp(qdict_get_str(dict, "d"), "off"));
+ g_assert(qobject_type(qdict_get(dict, "e")) == QTYPE_QNULL);
+ g_assert(!strcmp(qdict_get_str(dict, "f"), ""));
+ g_assert(!strcmp(qdict_get_str(dict, "g"), "0.5"));
+ g_assert(!strcmp(qdict_get_str(dict, "h"), "18446744073709551615"));
+ g_assert(!strcmp(qdict_get_str(dict, "i"), "on"));
+ g_assert(!strcmp(qdict_get_str(dict, "j"), "0"));
+
+ qobject_unref(dict);
+}
+
int main(int argc, char **argv)
{
g_test_init(&argc, &argv, NULL);
@@ -1010,6 +1061,9 @@ int main(int argc, char **argv)
g_test_add_func("/public/rename_keys", qdict_rename_keys_test);
+ g_test_add_func("/public/stringify_for_keyval",
+ qdict_stringify_for_keyval_test);
+
/* The Big one */
if (g_test_slow()) {
g_test_add_func("/stress/test", qdict_stress_test);
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (7 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-11 18:44 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test Max Reitz
` (3 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
In its current form, qdict_flatten() removes all entries from nested
QDicts that are moved to the root QDict. It is completely sufficient to
remove all old entries from the root QDict, however. If the nested
dicts have a refcount of 1, this will automatically delete them, too.
And if they have a greater refcount, we probably do not want to modify
them in the first place.
The latter observation means that it was currently (in general)
impossible to qdict_flatten() a shallowly cloned dict because that would
empty nested QDicts in the original dict as well. This patch changes
this, so you can now use qdict_flatten(qdict_shallow_clone(dict)) to get
a flattened copy without disturbing the original.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
qobject/qdict.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 83678c4951..63dcb43e73 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -537,7 +537,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
QObject *value;
const QDictEntry *entry, *next;
char *new_key;
- bool delete;
+ bool copied_value;
entry = qdict_first(qdict);
@@ -546,7 +546,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
next = qdict_next(qdict, entry);
value = qdict_entry_value(entry);
new_key = NULL;
- delete = false;
+ copied_value = false;
if (prefix) {
new_key = g_strdup_printf("%s.%s", prefix, entry->key);
@@ -557,23 +557,32 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
* itself disappears. */
qdict_flatten_qdict(qobject_to(QDict, value), target,
new_key ? new_key : entry->key);
- delete = true;
+ copied_value = true;
} else if (qobject_type(value) == QTYPE_QLIST) {
qdict_flatten_qlist(qobject_to(QList, value), target,
new_key ? new_key : entry->key);
- delete = true;
+ copied_value = true;
} else if (prefix) {
/* All other objects are moved to the target unchanged. */
qdict_put_obj(target, new_key, qobject_ref(value));
- delete = true;
+ copied_value = true;
}
g_free(new_key);
- if (delete) {
+ if (copied_value && qdict == target) {
+ /* If we have copied a value, and if we are on the root
+ * level, we need to remove the old entry. Otherwise, we
+ * do not, because by removing these entries on the root
+ * level, the reference counts of nested dicts and listed
+ * will be reduced automatically. In fact, we probably do
+ * not want to modify nested dicts and lists with
+ * refcounts greater than 1 anyway. */
qdict_del(qdict, entry->key);
- /* Restart loop after modifying the iterated QDict */
+ /* Restart loop after modifying the iterated QDict. We
+ * only need to do this if qdict == target, because
+ * otherwise copying the value did not affect qdict. */
entry = qdict_first(qdict);
continue;
}
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (8 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-11 18:46 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 11/13] block: Try to create well typed json:{} filenames Max Reitz
` (2 subsequent siblings)
12 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
This new test verifies that qdict_flatten() does not modify a shallow
clone of the given QDict.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/check-qdict.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index ef5d17f815..7960b4a385 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -329,6 +329,38 @@ static void qdict_flatten_test(void)
qobject_unref(dict3);
}
+static void qdict_clone_flatten_test(void)
+{
+ QDict *dict1 = qdict_new();
+ QDict *dict2 = qdict_new();
+ QDict *cloned_dict1;
+
+ /*
+ * Test that we can clone and flatten
+ * { "a": { "b": 42 } }
+ * without modifying the clone.
+ */
+
+ qdict_put_int(dict2, "b", 42);
+ qdict_put(dict1, "a", dict2);
+
+ cloned_dict1 = qdict_clone_shallow(dict1);
+
+ qdict_flatten(dict1);
+
+ g_assert(qdict_size(dict1) == 1);
+ g_assert(qdict_get_int(dict1, "a.b") == 42);
+
+ g_assert(qdict_size(cloned_dict1) == 1);
+ g_assert(qdict_get_qdict(cloned_dict1, "a") == dict2);
+
+ g_assert(qdict_size(dict2) == 1);
+ g_assert(qdict_get_int(dict2, "b") == 42);
+
+ qobject_unref(dict1);
+ qobject_unref(cloned_dict1);
+}
+
static void qdict_array_split_test(void)
{
QDict *test_dict = qdict_new();
@@ -1045,6 +1077,7 @@ int main(int argc, char **argv)
g_test_add_func("/public/to_qdict", qobject_to_qdict_test);
g_test_add_func("/public/iterapi", qdict_iterapi_test);
g_test_add_func("/public/flatten", qdict_flatten_test);
+ g_test_add_func("/public/clone-flatten", qdict_clone_flatten_test);
g_test_add_func("/public/array_split", qdict_array_split_test);
g_test_add_func("/public/array_entries", qdict_array_entries_test);
g_test_add_func("/public/join", qdict_join_test);
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 11/13] block: Try to create well typed json:{} filenames
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (9 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 12/13] iotests: Test internal option typing Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 13/13] iotests: qcow2's encrypt.format is now optional Max Reitz
12 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
By applying a health mix of qdict_flatten(), qdict_crumple(),
qdict_stringify_for_keyval(), the keyval input visitor, and the QObject
output visitor (not necessarily in that order), we can at least try to
bring bs->full_open_options into accordance with the QAPI schema. This
may not always work (there are some options left that have not been
QAPI-fied yet), but in practice it usually will.
In any case, sometimes emitting wrongly typed json:{} filenames is
better than doing it effectively half the time.
This affects some iotests because json:{} filenames are now usually
crumpled.
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1534396
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block.c | 69 +++++++++++++++++++++++++++++++++++++++++++++-
tests/qemu-iotests/059.out | 2 +-
tests/qemu-iotests/099.out | 4 +--
tests/qemu-iotests/110.out | 2 +-
tests/qemu-iotests/198.out | 4 +--
tests/qemu-iotests/207.out | 10 +++----
6 files changed, 79 insertions(+), 12 deletions(-)
diff --git a/block.c b/block.c
index 3c3e8fd11d..9e4a6c0d30 100644
--- a/block.c
+++ b/block.c
@@ -36,6 +36,7 @@
#include "qapi/qmp/qjson.h"
#include "qapi/qmp/qnull.h"
#include "qapi/qmp/qstring.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qobject-output-visitor.h"
#include "qapi/qapi-visit-block-core.h"
#include "sysemu/block-backend.h"
@@ -5143,6 +5144,59 @@ static bool append_open_options(QDict *d, BlockDriverState *bs)
return found_any;
}
+/**
+ * Take a blockdev @options QDict and convert its values to the
+ * correct type.
+ *
+ * Fail if @options does not match the QAPI schema of BlockdevOptions.
+ *
+ * In case of failure, return NULL and set @errp.
+ *
+ * In case of success, return a correctly typed new QDict.
+ */
+static QDict *bdrv_type_blockdev_opts(const QDict *options, Error **errp)
+{
+ Visitor *v;
+ BlockdevOptions *blockdev_options;
+ QObject *typed_opts, *crumpled_opts;
+ QDict *string_options;
+ Error *local_err = NULL;
+
+ string_options = qdict_clone_shallow(options);
+
+ qdict_flatten(string_options);
+ qdict_stringify_for_keyval(string_options);
+ crumpled_opts = qdict_crumple(string_options, errp);
+ qobject_unref(string_options);
+ if (!crumpled_opts) {
+ error_prepend(errp, "Failed to crumple options: ");
+ return NULL;
+ }
+
+ v = qobject_input_visitor_new_keyval(crumpled_opts);
+ visit_type_BlockdevOptions(v, NULL, &blockdev_options, &local_err);
+ visit_free(v);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ error_prepend(errp, "Not a valid BlockdevOptions object: ");
+ return NULL;
+ }
+
+ v = qobject_output_visitor_new(&typed_opts);
+ visit_type_BlockdevOptions(v, NULL, &blockdev_options, &local_err);
+ if (!local_err) {
+ visit_complete(v, &typed_opts);
+ }
+ visit_free(v);
+ qapi_free_BlockdevOptions(blockdev_options);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return NULL;
+ }
+
+ return qobject_to(QDict, typed_opts);
+}
+
/* Updates the following BDS fields:
* - exact_filename: A filename which may be used for opening a block device
* which (mostly) equals the given BDS (even without any
@@ -5244,10 +5298,23 @@ void bdrv_refresh_filename(BlockDriverState *bs)
if (bs->exact_filename[0]) {
pstrcpy(bs->filename, sizeof(bs->filename), bs->exact_filename);
} else if (bs->full_open_options) {
- QString *json = qobject_to_json(QOBJECT(bs->full_open_options));
+ QString *json;
+ QDict *typed_opts, *json_opts;
+
+ typed_opts = bdrv_type_blockdev_opts(bs->full_open_options, NULL);
+
+ /* We cannot be certain that bs->full_open_options matches
+ * BlockdevOptions, so bdrv_type_blockdev_opts() may fail.
+ * That is not fatal, we can just emit bs->full_open_options
+ * directly -- qemu will accept that, even if it does not
+ * match the schema. */
+ json_opts = typed_opts ?: bs->full_open_options;
+
+ json = qobject_to_json(QOBJECT(json_opts));
snprintf(bs->filename, sizeof(bs->filename), "json:%s",
qstring_get_str(json));
qobject_unref(json);
+ qobject_unref(typed_opts);
}
}
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index f6dce7947c..0238b9e9a8 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -2050,7 +2050,7 @@ wrote 512/512 bytes at offset 10240
=== Testing monolithicFlat with internally generated JSON file name ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
-can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"inject-error": [{"event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}'
=== Testing version 3 ===
image: TEST_DIR/iotest-version3.IMGFMT
diff --git a/tests/qemu-iotests/099.out b/tests/qemu-iotests/099.out
index 8cce627529..0a9c434148 100644
--- a/tests/qemu-iotests/099.out
+++ b/tests/qemu-iotests/099.out
@@ -12,11 +12,11 @@ blkverify:TEST_DIR/t.IMGFMT.compare:TEST_DIR/t.IMGFMT
=== Testing JSON filename for blkdebug ===
-json:{"driver": "IMGFMT", "file": {"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "l1_update"}}
+json:{"driver": "IMGFMT", "file": {"inject-error": [{"event": "l1_update"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}}
=== Testing indirectly enforced JSON filename ===
-json:{"driver": "raw", "file": {"test": {"driver": "IMGFMT", "file": {"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "l1_update"}}, "driver": "blkverify", "raw": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.compare"}}}
+json:{"driver": "raw", "file": {"test": {"driver": "IMGFMT", "file": {"inject-error": [{"event": "l1_update"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}}, "driver": "blkverify", "raw": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.compare"}}}
=== Testing plain filename for blkdebug ===
diff --git a/tests/qemu-iotests/110.out b/tests/qemu-iotests/110.out
index b3584ff87f..fa19315dfb 100644
--- a/tests/qemu-iotests/110.out
+++ b/tests/qemu-iotests/110.out
@@ -11,7 +11,7 @@ backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
=== Non-reconstructable filename ===
-image: json:{"driver": "IMGFMT", "file": {"set-state.0.event": "read_aio", "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "set-state.0.new_state": 42}}
+image: json:{"driver": "IMGFMT", "file": {"set-state": [{"new_state": 42, "event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}}
file format: IMGFMT
virtual size: 64M (67108864 bytes)
backing file: t.IMGFMT.base (cannot determine actual path)
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index adb805cce9..2a02077d0b 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -32,7 +32,7 @@ read 16777216/16777216 bytes at offset 0
16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== checking image base ==
-image: json:{"encrypt.key-secret": "sec0", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.base"}}
+image: json:{"driver": "IMGFMT", "encrypt": {"key-secret": "sec0"}, "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.base"}}
file format: IMGFMT
virtual size: 16M (16777216 bytes)
Format specific information:
@@ -74,7 +74,7 @@ Format specific information:
master key iters: 1024
== checking image layer ==
-image: json:{"encrypt.key-secret": "sec1", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "encrypt": {"key-secret": "sec1"}, "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}}
file format: IMGFMT
virtual size: 16M (16777216 bytes)
backing file: TEST_DIR/t.IMGFMT.base
diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out
index 417deee970..effb74bf02 100644
--- a/tests/qemu-iotests/207.out
+++ b/tests/qemu-iotests/207.out
@@ -9,7 +9,7 @@ QMP_VERSION
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_DIR/t.IMGFMT", "server": { "port": "22", "host": "127.0.0.1" }}}
file format: IMGFMT
virtual size: 4.0M (4194304 bytes)
@@ -26,7 +26,7 @@ QMP_VERSION
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_DIR/t.IMGFMT", "server": { "port": "22", "host": "127.0.0.1" }}}
file format: IMGFMT
virtual size: 8.0M (8388608 bytes)
Testing:
@@ -36,7 +36,7 @@ QMP_VERSION
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_DIR/t.IMGFMT", "server": { "port": "22", "host": "127.0.0.1" }}}
file format: IMGFMT
virtual size: 4.0M (4194304 bytes)
Testing:
@@ -47,7 +47,7 @@ QMP_VERSION
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_DIR/t.IMGFMT", "server": { "port": "22", "host": "127.0.0.1" }}}
file format: IMGFMT
virtual size: 8.0M (8388608 bytes)
Testing:
@@ -58,7 +58,7 @@ QMP_VERSION
{"return": {}}
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}}
+image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_DIR/t.IMGFMT", "server": { "port": "22", "host": "127.0.0.1" }}}
file format: IMGFMT
virtual size: 4.0M (4194304 bytes)
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 12/13] iotests: Test internal option typing
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (10 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 11/13] block: Try to create well typed json:{} filenames Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 13/13] iotests: qcow2's encrypt.format is now optional Max Reitz
12 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
It would be nice if qemu used the correct types for blockdev options
internally, even if the user specified string values (either through
-drive or by being not so nice and using json:{} with string values).
This patch adds a test verifying that fact.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/089 | 25 +++++++++++++++++++++++++
tests/qemu-iotests/089.out | 9 +++++++++
2 files changed, 34 insertions(+)
diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089
index aa1ba4a98e..0682d08f39 100755
--- a/tests/qemu-iotests/089
+++ b/tests/qemu-iotests/089
@@ -36,6 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
# get standard environment, filters and checks
. ./common.rc
. ./common.filter
+. ./common.qemu
_supported_fmt qcow2
_supported_proto file
@@ -145,6 +146,30 @@ $QEMU_IO -c "open -o driver=qcow2 json:{\"file.filename\":\"$TEST_IMG\"}" \
$QEMU_IO -c "open -o driver=qcow2 json:{\"driver\":\"raw\",\"file.filename\":\"$TEST_IMG\"}" \
-c "info" 2>&1 | _filter_img_info
+echo
+echo "=== Testing option typing ==="
+echo
+
+# json:{} accepts both strings and correctly typed values (even mixed,
+# although we probably do not want to support that...), but when
+# creating a json:{} filename, it should be correctly typed.
+# Therefore, both of these should make the "size" value an integer.
+
+TEST_IMG="json:{'driver': 'null-co', 'size': 42 }" _img_info | grep '^image'
+TEST_IMG="json:{'driver': 'null-co', 'size': '42'}" _img_info | grep '^image'
+
+echo
+
+# This should even work when some driver abuses bs->options to store
+# non-QAPI options (and the given -drive options are not complete)
+# (Watch for whether file.align appears as an int or a string)
+qemu_comm_method=monitor _launch_qemu \
+ -drive if=none,id=drv0,node-name=node0,format=raw,file=blkdebug::null-co://,file.align=512
+
+_send_qemu_cmd $QEMU_HANDLE 'info block' 'json:'
+
+_cleanup_qemu
+
# success, all done
echo "*** done"
diff --git a/tests/qemu-iotests/089.out b/tests/qemu-iotests/089.out
index 89e3e4340a..35ffbabe77 100644
--- a/tests/qemu-iotests/089.out
+++ b/tests/qemu-iotests/089.out
@@ -49,4 +49,13 @@ vm state offset: 512 MiB
format name: IMGFMT
cluster size: 64 KiB
vm state offset: 512 MiB
+
+=== Testing option typing ===
+
+image: json:{"driver": "null-co", "size": 42}
+image: json:{"driver": "null-co", "size": 42}
+
+QEMU X.Y.Z monitor - type 'help' for more information
+(qemu) info block
+drv0 (node0): json:{"driver": "raw", "file": {"image": {"driver": "null-co"}, "driver": "blkdebug", "align": 512}} (raw)
*** done
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [Qemu-devel] [PATCH 13/13] iotests: qcow2's encrypt.format is now optional
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
` (11 preceding siblings ...)
2018-05-09 16:55 ` [Qemu-devel] [PATCH 12/13] iotests: Test internal option typing Max Reitz
@ 2018-05-09 16:55 ` Max Reitz
12 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-09 16:55 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Markus Armbruster, Kevin Wolf, Eric Blake,
Michael Roth
Remove the encrypt.format option from the two blockdev-add test cases
for encrypted qcow2 images in 087 to explicitly test that this parameter
is now optional.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/087 | 2 --
1 file changed, 2 deletions(-)
diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index 2561a14456..1eba457985 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -150,7 +150,6 @@ run_qemu <<EOF
"filename": "$TEST_IMG"
},
"encrypt": {
- "format": "aes",
"key-secret": "sec0"
}
}
@@ -183,7 +182,6 @@ run_qemu <<EOF
"filename": "$TEST_IMG"
},
"encrypt": {
- "format": "luks",
"key-secret": "sec0"
}
}
--
2.14.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing
2018-05-09 16:55 ` [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing Max Reitz
@ 2018-05-10 7:58 ` Daniel P. Berrangé
2018-05-10 14:22 ` Eric Blake
2018-05-11 17:32 ` Max Reitz
0 siblings, 2 replies; 42+ messages in thread
From: Daniel P. Berrangé @ 2018-05-10 7:58 UTC (permalink / raw)
To: Max Reitz
Cc: qemu-block, Kevin Wolf, qemu-devel, Markus Armbruster, Michael Roth
On Wed, May 09, 2018 at 06:55:21PM +0200, Max Reitz wrote:
> Currently, you can give no encryption format for a qcow2 file while
> still passing a key-secret. That does not conform to the schema, so
> this patch changes the schema to allow it.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> qapi/block-core.json | 44 ++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 40 insertions(+), 4 deletions(-)
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 71c9ab8538..092a1aba2d 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -43,6 +43,19 @@
> { 'struct': 'ImageInfoSpecificQCow2EncryptionBase',
> 'data': { 'format': 'BlockdevQcow2EncryptionFormat'}}
>
> +##
> +# @ImageInfoSpecificQCow2EncryptionNoInfo:
> +#
> +# Only used for the qcow2 encryption format "from-image" in which the
> +# actual encryption format is determined from the image header.
> +# Therefore, this encryption format will never be reported in
> +# ImageInfoSpecificQCow2Encryption.
> +#
> +# Since: 2.13
> +##
> +{ 'struct': 'ImageInfoSpecificQCow2EncryptionNoInfo',
> + 'data': { } }
> +
> ##
> # @ImageInfoSpecificQCow2Encryption:
> #
> @@ -52,7 +65,8 @@
> 'base': 'ImageInfoSpecificQCow2EncryptionBase',
> 'discriminator': 'format',
> 'data': { 'aes': 'QCryptoBlockInfoQCow',
> - 'luks': 'QCryptoBlockInfoLUKS' } }
> + 'luks': 'QCryptoBlockInfoLUKS',
> + 'from-image': 'ImageInfoSpecificQCow2EncryptionNoInfo' } }
>
> ##
> # @ImageInfoSpecificQCow2:
> @@ -2739,10 +2753,30 @@
> # @BlockdevQcow2EncryptionFormat:
> # @aes: AES-CBC with plain64 initialization venctors
> #
> +# @from-image: Determine the encryption format from the image
> +# header. This only allows the use of the
> +# key-secret option. (Since: 2.13)
> +#
> # Since: 2.10
> ##
> { 'enum': 'BlockdevQcow2EncryptionFormat',
> - 'data': [ 'aes', 'luks' ] }
> + 'data': [ 'aes', 'luks', 'from-image' ] }
> +
> +##
> +# @BlockdevQcow2EncryptionSecret:
> +#
> +# Allows specifying a key-secret without specifying the exact
> +# encryption format, which is determined automatically from the image
> +# header.
> +#
> +# @key-secret: The ID of a QCryptoSecret object providing the
> +# decryption key. Mandatory except when probing
> +# image for metadata only.
> +#
> +# Since: 2.13
> +##
> +{ 'struct': 'BlockdevQcow2EncryptionSecret',
> + 'data': { '*key-secret': 'str' } }
>
> ##
> # @BlockdevQcow2Encryption:
> @@ -2750,10 +2784,12 @@
> # Since: 2.10
> ##
> { 'union': 'BlockdevQcow2Encryption',
> - 'base': { 'format': 'BlockdevQcow2EncryptionFormat' },
> + 'base': { '*format': 'BlockdevQcow2EncryptionFormat' },
> 'discriminator': 'format',
> + 'default-variant': 'from-image',
> 'data': { 'aes': 'QCryptoBlockOptionsQCow',
> - 'luks': 'QCryptoBlockOptionsLUKS'} }
> + 'luks': 'QCryptoBlockOptionsLUKS',
> + 'from-image': 'BlockdevQcow2EncryptionSecret' } }
Bike-shedding on name, how about "auto" or "probe" ?
IIUC, this schema addition means the QAPI parser now allows
encrypt.format=from-image,encrypt.key-secret=sec0,...other opts...
but the code will not accept "from-image" as a valid string.
eg qcow2_update_options_prepare() will do
case QCOW_CRYPT_AES:
if (encryptfmt && !g_str_equal(encryptfmt, "aes")) {
error_setg(errp,
"Header reported 'aes' encryption format but "
"options specify '%s'", encryptfmt);
ret = -EINVAL;
goto fail;
}
...snip....
case QCOW_CRYPT_LUKS:
if (encryptfmt && !g_str_equal(encryptfmt, "luks")) {
error_setg(errp,
"Header reported 'luks' encryption format but "
"options specify '%s'", encryptfmt);
ret = -EINVAL;
goto fail;
}
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-09 16:55 ` [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions Max Reitz
@ 2018-05-10 13:12 ` Eric Blake
2018-05-10 13:18 ` Eric Blake
2018-05-11 17:38 ` Max Reitz
0 siblings, 2 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-10 13:12 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> This patch allows specifying a discriminator that is an optional member
> of the base struct. In such a case, a default value must be provided
> that is used when no value is given.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> qapi/introspect.json | 8 ++++++
> scripts/qapi/common.py | 57 ++++++++++++++++++++++++++++++++++--------
> scripts/qapi/doc.py | 8 ++++--
> scripts/qapi/introspect.py | 10 +++++---
> scripts/qapi/visit.py | 33 ++++++++++++++++++++++--
> tests/qapi-schema/test-qapi.py | 2 ++
> 6 files changed, 101 insertions(+), 17 deletions(-)
I've been threatening that we might need this for some time, so I'm glad
to see it being implemented. We'll see if the tests in 2 and 3 cover
the code added here.
>
> diff --git a/qapi/introspect.json b/qapi/introspect.json
> index c7f67b7d78..2d7b1e3745 100644
> --- a/qapi/introspect.json
> +++ b/qapi/introspect.json
> @@ -168,6 +168,13 @@
> # @tag: the name of the member serving as type tag.
> # An element of @members with this name must exist.
> #
> +# @default-variant: if the @tag element of @members is optional, this
> +# is the default value for choosing a variant. Its
> +# value must be a valid value for @tag.
Perhaps s/must/will/ as this struct is used for output (and therefore we
always meet the condition, rather than the user having to do something
correctly).
Nice that you remembered introspection.
> +# Present exactly when @tag is present and the
> +# associated element of @members is optional.
> +# (Since: 2.13)
> +#
> # @variants: variant members, i.e. additional members that
> # depend on the type tag's value. Present exactly when
> # @tag is present. The variants are in no particular order,
> @@ -181,6 +188,7 @@
> { 'struct': 'SchemaInfoObject',
> 'data': { 'members': [ 'SchemaInfoObjectMember' ],
> '*tag': 'str',
> + '*default-variant': 'str',
> '*variants': [ 'SchemaInfoObjectVariant' ] } }
>
> ##
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index a032cec375..fbf0244f73 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -721,6 +721,7 @@ def check_union(expr, info):
> name = expr['union']
> base = expr.get('base')
> discriminator = expr.get('discriminator')
> + default_variant = expr.get('default-variant')
> members = expr['data']
>
> # Two types of unions, determined by discriminator.
> @@ -745,16 +746,37 @@ def check_union(expr, info):
> base_members = find_base_members(base)
> assert base_members is not None
>
> - # The value of member 'discriminator' must name a non-optional
> - # member of the base struct.
> + # The value of member 'discriminator' must name a member of
> + # the base struct.
> check_name(info, "Discriminator of flat union '%s'" % name,
> discriminator)
> - discriminator_type = base_members.get(discriminator)
> - if not discriminator_type:
> - raise QAPISemError(info,
> - "Discriminator '%s' is not a member of base "
> - "struct '%s'"
> - % (discriminator, base))
> + if default_variant is None:
> + discriminator_type = base_members.get(discriminator)
> + if not discriminator_type:
> + if base_members.get('*' + discriminator) is None:
Unrelated to your patch, but this reminds me - we already had a question
on list about whether we should allow ANY member of the base struct,
rather than the members directly declared in the struct. (The use case
was that if you have a grandparent struct, then an intermediate base
struct, the code as written does not permit the discriminator to come
from the grandparent struct, which is awkward - but if I also recall
correctly, the question came up in relation to the query-cpus-fast
command, where we ended up not refactoring things after all due to
deprecation of query-cpus).
> + raise QAPISemError(info,
> + "Discriminator '%s' is not a member of "
> + "base struct '%s'"
> + % (discriminator, base))
> + else:
> + raise QAPISemError(info,
> + "Default variant must be specified for "
> + "optional discriminator '%s'"
> + % discriminator)
> + else:
> + discriminator_type = base_members.get('*' + discriminator)
> + if not discriminator_type:
> + if base_members.get(discriminator) is None:
> + raise QAPISemError(info,
> + "Discriminator '%s' is not a member of "
> + "base struct '%s'"
> + % (discriminator, base))
> + else:
> + raise QAPISemError(info,
> + "Must not specify a default variant for "
> + "non-optional discriminator '%s'"
> + % discriminator)
> +
Looks like you've got good error messages for all the cases of enforcing
the rule that default-variant and optional member must be used together.
> enum_define = enum_types.get(discriminator_type)
> allow_metas = ['struct']
> # Do not allow string discriminator
> @@ -763,6 +785,15 @@ def check_union(expr, info):
> "Discriminator '%s' must be of enumeration "
> "type" % discriminator)
>
> + if default_variant is not None:
> + # Must be a value of the enumeration
> + if default_variant not in enum_define['data']:
> + raise QAPISemError(info,
> + "Default variant '%s' of flat union '%s' is "
> + "not part of '%s'"
> + % (default_variant, name,
> + discriminator_type))
> +
> # Check every branch; don't allow an empty union
> if len(members) == 0:
> raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
> @@ -909,7 +940,7 @@ def check_exprs(exprs):
> elif 'union' in expr:
> meta = 'union'
> check_keys(expr_elem, 'union', ['data'],
> - ['base', 'discriminator'])
> + ['base', 'discriminator', 'default-variant'])
> union_types[expr[meta]] = expr
> elif 'alternate' in expr:
> meta = 'alternate'
> @@ -1335,12 +1366,14 @@ class QAPISchemaObjectTypeMember(QAPISchemaMember):
>
>
> class QAPISchemaObjectTypeVariants(object):
> - def __init__(self, tag_name, tag_member, variants):
> + def __init__(self, tag_name, tag_member, default_tag_value, variants):
> # Flat unions pass tag_name but not tag_member.
> # Simple unions and alternates pass tag_member but not tag_name.
> # After check(), tag_member is always set, and tag_name remains
> # a reliable witness of being used by a flat union.
> assert bool(tag_member) != bool(tag_name)
> + # default_tag_value is only passed for flat unions.
> + assert bool(tag_name) or not bool(default_tag_value)
Looks correct.
> assert (isinstance(tag_name, str) or
> isinstance(tag_member, QAPISchemaObjectTypeMember))
> assert len(variants) > 0
> @@ -1348,6 +1381,7 @@ class QAPISchemaObjectTypeVariants(object):
> assert isinstance(v, QAPISchemaObjectTypeVariant)
> self._tag_name = tag_name
> self.tag_member = tag_member
> + self.default_tag_value = default_tag_value
> self.variants = variants
>
> def set_owner(self, name):
> @@ -1637,6 +1671,7 @@ class QAPISchema(object):
> data = expr['data']
> base = expr.get('base')
> tag_name = expr.get('discriminator')
> + default_tag_value = expr.get('default-variant')
> tag_member = None
> if isinstance(base, dict):
> base = (self._make_implicit_object_type(
> @@ -1656,6 +1691,7 @@ class QAPISchema(object):
> QAPISchemaObjectType(name, info, doc, base, members,
> QAPISchemaObjectTypeVariants(tag_name,
> tag_member,
> + default_tag_value,
> variants)))
>
> def _def_alternate_type(self, expr, info, doc):
> @@ -1668,6 +1704,7 @@ class QAPISchema(object):
> QAPISchemaAlternateType(name, info, doc,
> QAPISchemaObjectTypeVariants(None,
> tag_member,
> + None,
> variants)))
>
> def _def_command(self, expr, info, doc):
> diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
> index 9b312b2c51..91204dc4c6 100644
> --- a/scripts/qapi/doc.py
> +++ b/scripts/qapi/doc.py
> @@ -160,8 +160,12 @@ def texi_members(doc, what, base, variants, member_func):
> items += '@item The members of @code{%s}\n' % base.doc_type()
> if variants:
> for v in variants.variants:
> - when = ' when @code{%s} is @t{"%s"}' % (
> - variants.tag_member.name, v.name)
> + if v.name == variants.default_tag_value:
> + when = ' when @code{%s} is @t{"%s"} or not given' % (
> + variants.tag_member.name, v.name)
Nice, you got the generated docs as well as introspection.
> + else:
> + when = ' when @code{%s} is @t{"%s"}' % (
> + variants.tag_member.name, v.name)
> if v.type.is_implicit():
> assert not v.type.base and not v.type.variants
> for m in v.type.local_members:
> diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
> index f9e67e8227..2d1d4e320a 100644
> --- a/scripts/qapi/introspect.py
> +++ b/scripts/qapi/introspect.py
> @@ -142,9 +142,12 @@ const QLitObject %(c_name)s = %(c_string)s;
> ret['default'] = None
> return ret
>
> - def _gen_variants(self, tag_name, variants):
> - return {'tag': tag_name,
> - 'variants': [self._gen_variant(v) for v in variants]}
> + def _gen_variants(self, tag_name, default_variant, variants):
> + ret = {'tag': tag_name,
> + 'variants': [self._gen_variant(v) for v in variants]}
> + if default_variant:
> + ret['default-variant'] = default_variant
> + return ret
>
> def _gen_variant(self, variant):
> return {'case': variant.name, 'type': self._use_type(variant.type)}
> @@ -163,6 +166,7 @@ const QLitObject %(c_name)s = %(c_string)s;
> obj = {'members': [self._gen_member(m) for m in members]}
> if variants:
> obj.update(self._gen_variants(variants.tag_member.name,
> + variants.default_tag_value,
> variants.variants))
> self._gen_qlit(name, 'object', obj)
>
> diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
> index 5d72d8936c..ecffc46bd3 100644
> --- a/scripts/qapi/visit.py
> +++ b/scripts/qapi/visit.py
> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members, variants):
> void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> {
> Error *err = NULL;
> -
> ''',
> c_name=c_name(name))
>
> + if variants:
> + ret += mcgen('''
> + %(c_type)s %(c_name)s;
> +''',
> + c_type=variants.tag_member.type.c_name(),
> + c_name=c_name(variants.tag_member.name))
> +
> + ret += mcgen('''
> +
> +''')
> +
Creating a temp variable makes it easier to handle the default...
> if base:
> ret += mcgen('''
> visit_type_%(c_type)s_members(v, (%(c_type)s *)obj, &err);
> @@ -75,8 +85,27 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> ''')
>
> if variants:
> + if variants.default_tag_value is None:
> + ret += mcgen('''
> + %(c_name)s = obj->%(c_name)s;
> +''',
> + c_name=c_name(variants.tag_member.name))
> + else:
> + ret += mcgen('''
> + if (obj->has_%(c_name)s) {
> + %(c_name)s = obj->%(c_name)s;
> + } else {
> + %(c_name)s = %(enum_const)s;
> + }
> +''',
> + c_name=c_name(variants.tag_member.name),
> + enum_const=c_enum_const(
> + variants.tag_member.type.name,
> + variants.default_tag_value,
> + variants.tag_member.type.prefix))
> +
> ret += mcgen('''
> - switch (obj->%(c_name)s) {
> + switch (%(c_name)s) {
...compared to the old code that just inlined the one thing that used to
be assigned to what is now the temporary variable.
It might be possible to inline things so that the generated code reads
either:
switch (obj->discriminator) {
switch (!obj->has_discriminator ? DEFAULT : obj->discriminator) {
but I don't think it's worth the effort; and the temporary variable is
fine even though it makes the generated file bigger.
> ''',
> c_name=c_name(variants.tag_member.name))
>
> diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> index c1a144ba29..f2a072b92e 100644
> --- a/tests/qapi-schema/test-qapi.py
> +++ b/tests/qapi-schema/test-qapi.py
> @@ -56,6 +56,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
> def _print_variants(variants):
> if variants:
> print(' tag %s' % variants.tag_member.name)
> + if variants.default_tag_value:
> + print(' default variant: %s' % variants.default_tag_value)
> for v in variants.variants:
> print(' case %s: %s' % (v.name, v.type.name))
>
>
Looks good!
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators
2018-05-09 16:55 ` [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators Max Reitz
@ 2018-05-10 13:14 ` Eric Blake
0 siblings, 0 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-10 13:14 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> docs/devel/qapi-code-gen.txt | 21 +++++++++++++++++++--
> 1 file changed, 19 insertions(+), 2 deletions(-)
>
I might have squashed this in with patch 1, but I can live with it separate.
> @@ -502,6 +502,23 @@ the enum). In the resulting generated C data types, a flat union is
> represented as a struct with the base members included directly, and
> then a union of structures for each branch of the struct.
>
> +If the discriminator points to an optional member of the base struct,
> +its default value must be specified as a 'default-variant'. In the
> +following example, the above BlockDriver struct is changed so it
> +defaults to the 'file' driver if that field is omitted on the wire:
> +
> + { 'union': 'BlockdevOptions',
> + 'base': { '*driver': 'BlockdevDriver', '*read-only': 'bool' },
> + 'discriminator': 'driver',
> + 'default-variant': 'file',
> + 'data': { 'file': 'BlockdevOptionsFile',
> + 'qcow2': 'BlockdevOptionsQcow2' } }
> +
> +Now the 'file' JSON object can be abbreviated to:
> +
> + { "read-only": "true",
> + "filename": "/some/place/my-image" }
> +
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-10 13:12 ` Eric Blake
@ 2018-05-10 13:18 ` Eric Blake
2018-05-11 17:59 ` Max Reitz
2018-05-11 17:38 ` Max Reitz
1 sibling, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-10 13:18 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/10/2018 08:12 AM, Eric Blake wrote:
Oh, I just had a thought:
>> +++ b/scripts/qapi/visit.py
>> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members,
>> if variants:
>> + if variants.default_tag_value is None:
>> + ret += mcgen('''
>> + %(c_name)s = obj->%(c_name)s;
>> +''',
>> + c_name=c_name(variants.tag_member.name))
>> + else:
>> + ret += mcgen('''
>> + if (obj->has_%(c_name)s) {
>> + %(c_name)s = obj->%(c_name)s;
>> + } else {
>> + %(c_name)s = %(enum_const)s;
In this branch of code, is it worth also generating:
%has_(c_name)s = true;
That way, the rest of the C code does not have to check
has_discriminator, because the process of assigning the default will
ensure that has_discriminator is always true later on. It does have the
effect that output would never omit the discriminator - but that might
be a good thing: if we ever have an output union that used to have a
mandatory discriminator and want to now make it optional, we don't want
to break older clients that expected the discriminator to be present.
It does obscure whether input relied on the default, but I don't think
that hurts.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests
2018-05-09 16:55 ` [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests Max Reitz
@ 2018-05-10 14:08 ` Eric Blake
2018-05-11 18:06 ` Max Reitz
0 siblings, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-10 14:08 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> There already is an optional discriminator test, although it also noted
> the discriminator name itself as optional. Changing that, and adding a
> default-variant field, makes that test pass instead of failing.
I'm not sure I agree with that one. I think that you should instead add
a positive test of a default variant into qapi-schema-test.json,
especially since that is the one positive test where we also ensure that
the C compiler is happy with the generated code (other positive tests
prove that the generator produced something without error, but not that
what it produced could be compiled).
> 17 files changed, 61 insertions(+), 4 deletions(-)
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.json
> create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.json
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.err
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
> create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.out
> create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.err
> create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.exit
> create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.out
Patch 1 converted 2 error messages into 6, where 2 of them look identical:
>> - # The value of member 'discriminator' must name a non-optional
>> - # member of the base struct.
>> + # The value of member 'discriminator' must name a member of
>> + # the base struct.
>> check_name(info, "Discriminator of flat union '%s'" % name,
>> discriminator)
[0] check_name() checks that 'discriminator':'*name' is rejected - this
check is unchanged
>> - discriminator_type = base_members.get(discriminator)
>> - if not discriminator_type:
>> - raise QAPISemError(info,
>> - "Discriminator '%s' is not a member of base "
>> - "struct '%s'"
>> - % (discriminator, base))
The old code ensured that 'discriminator':'name' finds 'name' as a
mandatory field in the base type; which changed into:
>> + if default_variant is None:
>> + discriminator_type = base_members.get(discriminator)
>> + if not discriminator_type:
>> + if base_members.get('*' + discriminator) is None:
>> + raise QAPISemError(info,
>> + "Discriminator '%s' is not a member of "
>> + "base struct '%s'"
>> + % (discriminator, base))
[1] the discriminator type must be a member field (we didn't find either
a mandatory or an optional field)
>> + else:
>> + raise QAPISemError(info,
>> + "Default variant must be specified for "
>> + "optional discriminator '%s'"
>> + % discriminator)
[2] without default_variant, the member field must be mandatory
>> + else:
>> + discriminator_type = base_members.get('*' + discriminator)
>> + if not discriminator_type:
>> + if base_members.get(discriminator) is None:
>> + raise QAPISemError(info,
>> + "Discriminator '%s' is not a member of "
>> + "base struct '%s'"
>> + % (discriminator, base))
[3] the discriminator type must be a member field (we didn't find either
a mandatory or an optional field), identical message to [1]
>> + else:
>> + raise QAPISemError(info,
>> + "Must not specify a default variant for "
>> + "non-optional discriminator '%s'"
>> + % discriminator)
[4] with default_variant, the member field must be optional
>> +
>> enum_define = enum_types.get(discriminator_type)
>> allow_metas = ['struct']
>> # Do not allow string discriminator
>> @@ -763,6 +785,15 @@ def check_union(expr, info):
>> "Discriminator '%s' must be of enumeration "
>> "type" % discriminator)
>>
>> + if default_variant is not None:
>> + # Must be a value of the enumeration
>> + if default_variant not in enum_define['data']:
>> + raise QAPISemError(info,
>> + "Default variant '%s' of flat union '%s' is "
>> + "not part of '%s'"
>> + % (default_variant, name,
>> + discriminator_type))
>> +
[5] the default discriminator value must belong to the discriminator's type
Of those 6 messages, you now have coverage of:
>
> +++ b/tests/Makefile.include
> @@ -500,7 +500,10 @@ qapi-schema += flat-union-invalid-branch-key.json
> qapi-schema += flat-union-invalid-discriminator.json
[1] no change to that test
> qapi-schema += flat-union-no-base.json
> qapi-schema += flat-union-optional-discriminator.json
the old test for [0] - you turned it into success. See more below...
> +qapi-schema += flat-union-optional-discriminator-no-default.json
new test for [2]
> +qapi-schema += flat-union-optional-discriminator-invalid-default.json
new test for [5]
> qapi-schema += flat-union-string-discriminator.json
> +qapi-schema += flat-union-superfluous-default-variant.json
new test for [4]
I don't know if it is worth trying to explicitly test for [3], given
that it is identical to [1], but the loss of the test for [0] bothers me.
> +++ b/tests/qapi-schema/flat-union-optional-discriminator.err
> @@ -1 +0,0 @@
> -tests/qapi-schema/flat-union-optional-discriminator.json:6: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
This used to be covered by [0] - but you rewrote the test by changing
'discriminator':'*switch' to 'discriminator':'switch'.
I'd argue that you WANT to keep 'discriminator':'*switch' in this test,
to keep the testing of error [0] (even when the discriminator type is
optional, we spell it without leading '*' in the 'union' description),
and then only qapi-schema-test.json gets a positive test, which also
gives us the additional coverage that the C compiler likes our generated
code.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing
2018-05-10 7:58 ` Daniel P. Berrangé
@ 2018-05-10 14:22 ` Eric Blake
2018-05-11 17:32 ` Max Reitz
1 sibling, 0 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-10 14:22 UTC (permalink / raw)
To: Daniel P. Berrangé, Max Reitz
Cc: Kevin Wolf, Michael Roth, qemu-devel, qemu-block, Markus Armbruster
On 05/10/2018 02:58 AM, Daniel P. Berrangé wrote:
> On Wed, May 09, 2018 at 06:55:21PM +0200, Max Reitz wrote:
>> Currently, you can give no encryption format for a qcow2 file while
>> still passing a key-secret. That does not conform to the schema, so
>> this patch changes the schema to allow it.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> qapi/block-core.json | 44 ++++++++++++++++++++++++++++++++++++++++----
>> 1 file changed, 40 insertions(+), 4 deletions(-)
>> { 'union': 'BlockdevQcow2Encryption',
>> - 'base': { 'format': 'BlockdevQcow2EncryptionFormat' },
>> + 'base': { '*format': 'BlockdevQcow2EncryptionFormat' },
>> 'discriminator': 'format',
>> + 'default-variant': 'from-image',
>> 'data': { 'aes': 'QCryptoBlockOptionsQCow',
>> - 'luks': 'QCryptoBlockOptionsLUKS'} }
>> + 'luks': 'QCryptoBlockOptionsLUKS',
>> + 'from-image': 'BlockdevQcow2EncryptionSecret' } }
>
> Bike-shedding on name, how about "auto" or "probe" ?
Either of those sounds nicer to me; 'auto' might be better in the
context of creation (that way, we can state that creating a NEW image
with x-blockdev-create maps 'auto' to 'luks'; while connecting to an
EXISTING image maps 'auto' to either 'aes' or 'luks' as appropriate).
>
> IIUC, this schema addition means the QAPI parser now allows
>
> encrypt.format=from-image,encrypt.key-secret=sec0,...other opts...
Yes. You could, perhaps, add a special case on the command line parsing
code to reject an explicit use of format=from-image, but the QMP should
not reject an explicit discriminator.
Hmm, it plays in with my comment on 1/13 - should the QMP parser
automatically set has_discriminator to true when it supplies the
default? If it does, you lose the ability to see whether the user
supplied an explicit encrypt.format=from-image (or the equivalent when
using QMP instead of the command line), if you wanted to enforce that
the user MUST omit format when relying on the from-image variant.
I don't see a problem in allowing the user to explicitly specify the
name of the default branch, but I _do_ think the patch is incomplete for
not handling the new QCOW_CRYPT_FROM_IMAGE case and converting it as
soon as possible back into one of the other two preferred enum values.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow encryption probing
2018-05-09 16:55 ` [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow " Max Reitz
@ 2018-05-10 14:24 ` Eric Blake
2018-05-10 14:32 ` Daniel P. Berrangé
2018-05-11 18:07 ` Max Reitz
0 siblings, 2 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-10 14:24 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> Currently, you can give no encryption format for a qcow file while still
> passing a key-secret. That does not conform to the schema, so this
> patch changes the schema to allow it.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> ##
> # @BlockdevQcowEncryptionFormat:
> #
> # @aes: AES-CBC with plain64 initialization vectors
> #
> +# @from-image: Determine the encryption format from the image
> +# header. This only allows the use of the
> +# key-secret option. (Since: 2.13)
> +#
> # Since: 2.10
> ##
> { 'enum': 'BlockdevQcowEncryptionFormat',
> - 'data': [ 'aes' ] }
> + 'data': [ 'aes', 'from-image' ] }
Overkill. Why not just:
>
> ##
> # @BlockdevQcowEncryption:
> @@ -2728,9 +2748,11 @@
> # Since: 2.10
> ##
> { 'union': 'BlockdevQcowEncryption',
> - 'base': { 'format': 'BlockdevQcowEncryptionFormat' },
> + 'base': { '*format': 'BlockdevQcowEncryptionFormat' },
> 'discriminator': 'format',
> - 'data': { 'aes': 'QCryptoBlockOptionsQCow' } }
> + 'default-variant': 'from-image',
'default-variant': 'aes'
> + 'data': { 'aes': 'QCryptoBlockOptionsQCow',
and call it good, because there are no other options to pick from, so
'from-image' would always resolve to 'aes' anyway.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow encryption probing
2018-05-10 14:24 ` Eric Blake
@ 2018-05-10 14:32 ` Daniel P. Berrangé
2018-05-11 18:07 ` Max Reitz
1 sibling, 0 replies; 42+ messages in thread
From: Daniel P. Berrangé @ 2018-05-10 14:32 UTC (permalink / raw)
To: Eric Blake
Cc: Max Reitz, qemu-block, Kevin Wolf, Michael Roth, qemu-devel,
Markus Armbruster
On Thu, May 10, 2018 at 09:24:24AM -0500, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
> > Currently, you can give no encryption format for a qcow file while still
> > passing a key-secret. That does not conform to the schema, so this
> > patch changes the schema to allow it.
> >
> > Signed-off-by: Max Reitz <mreitz@redhat.com>
> > ---
>
> > ##
> > # @BlockdevQcowEncryptionFormat:
> > #
> > # @aes: AES-CBC with plain64 initialization vectors
> > #
> > +# @from-image: Determine the encryption format from the image
> > +# header. This only allows the use of the
> > +# key-secret option. (Since: 2.13)
> > +#
> > # Since: 2.10
> > ##
> > { 'enum': 'BlockdevQcowEncryptionFormat',
> > - 'data': [ 'aes' ] }
> > + 'data': [ 'aes', 'from-image' ] }
>
> Overkill. Why not just:
>
> > ##
> > # @BlockdevQcowEncryption:
> > @@ -2728,9 +2748,11 @@
> > # Since: 2.10
> > ##
> > { 'union': 'BlockdevQcowEncryption',
> > - 'base': { 'format': 'BlockdevQcowEncryptionFormat' },
> > + 'base': { '*format': 'BlockdevQcowEncryptionFormat' },
> > 'discriminator': 'format',
> > - 'data': { 'aes': 'QCryptoBlockOptionsQCow' } }
> > + 'default-variant': 'from-image',
>
> 'default-variant': 'aes'
>
> > + 'data': { 'aes': 'QCryptoBlockOptionsQCow',
>
> and call it good, because there are no other options to pick from, so
> 'from-image' would always resolve to 'aes' anyway.
Sounds reasonable because qcowv1 is a dead format we don't intend to
add more features to.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-05-09 16:55 ` [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header Max Reitz
@ 2018-05-10 14:54 ` Eric Blake
2018-05-11 18:11 ` Max Reitz
2018-06-06 13:17 ` Markus Armbruster
1 sibling, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-10 14:54 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> There are numerous QDict functions that have been introduced for and are
> used only by the block layer. Move their declarations into an own
s/an own/their own/
> header file to reflect that.
>
> While qdict_extract_subqdict() is in fact used outside of the block
> layer (in util/qemu-config.c), it is still a function related very
> closely to how the block layer works with nested QDicts, namely by
> sometimes flattening them. Therefore, its declaration is put into this
> header as well and util/qemu-config.c includes it with a comment stating
> exactly which function it needs.
>
> Suggested-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> +++ b/include/block/qdict.h
> @@ -0,0 +1,35 @@
> +/*
> + * Special QDict functions used by the block layer
> + *
> + * Copyright (c) 2018 Red Hat, Inc.
You are extracting this from qdict.h which has:
* Copyright (C) 2009 Red Hat Inc.
Is it worth listing 2009-2018, instead of just this year?
> +
> +void qdict_copy_default(QDict *dst, QDict *src, const char *key);
> +void qdict_set_default_str(QDict *dst, const char *key, const char *val);
These two might be useful outside of the block layer; we can move them
back in a later patch if so. But for now, I agree with your choice of
moving them.
> +
> +void qdict_join(QDict *dest, QDict *src, bool overwrite);
This is borderline whether it would be useful outside of the block layer.
> +
> +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> +void qdict_array_split(QDict *src, QList **dst);
> +int qdict_array_entries(QDict *src, const char *subqdict);
> +QObject *qdict_crumple(const QDict *src, Error **errp);
> +void qdict_flatten(QDict *qdict);
And these are definitely hacks that the block layer relies on, where
hopefully someday long term we can rewrite the block layer to use QAPI
types directly instead of constant reconversion between QemuOpts and
QDict and QAPI types.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test
2018-05-09 16:55 ` [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test Max Reitz
@ 2018-05-10 16:02 ` Eric Blake
2018-05-11 18:13 ` Max Reitz
0 siblings, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-10 16:02 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/check-qdict.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 54 insertions(+)
>
> +static void qdict_stringify_for_keyval_test(void)
> +{
> + QDict *dict = qdict_new();
> +
> + /*
> + * Test stringification of:
> + *
> + * {
> + * "a": "null",
> + * "b": 42,
> + * "c": -23,
> + * "d": false,
> + * "e": null,
> + * "f": "",
> + * "g": 0.5,
> + * "h": 0xffffffffffffffff,
> + * "i": true,
> + * "j": 0
Is it worth testing fun things like '-0.0'?
> + g_assert(!strcmp(qdict_get_str(dict, "a"), "null"));
> + g_assert(!strcmp(qdict_get_str(dict, "b"), "42"));
> + g_assert(!strcmp(qdict_get_str(dict, "c"), "-23"));
> + g_assert(!strcmp(qdict_get_str(dict, "d"), "off"));
> + g_assert(qobject_type(qdict_get(dict, "e")) == QTYPE_QNULL);
Is it worth shortening this line to:
g_assert(qobject_to(QNull, qdict_get(dict, "e")));
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing
2018-05-10 7:58 ` Daniel P. Berrangé
2018-05-10 14:22 ` Eric Blake
@ 2018-05-11 17:32 ` Max Reitz
1 sibling, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 17:32 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-block, Kevin Wolf, qemu-devel, Markus Armbruster, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 4554 bytes --]
On 2018-05-10 09:58, Daniel P. Berrangé wrote:
> On Wed, May 09, 2018 at 06:55:21PM +0200, Max Reitz wrote:
>> Currently, you can give no encryption format for a qcow2 file while
>> still passing a key-secret. That does not conform to the schema, so
>> this patch changes the schema to allow it.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> qapi/block-core.json | 44 ++++++++++++++++++++++++++++++++++++++++----
>> 1 file changed, 40 insertions(+), 4 deletions(-)
>>
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 71c9ab8538..092a1aba2d 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -43,6 +43,19 @@
>> { 'struct': 'ImageInfoSpecificQCow2EncryptionBase',
>> 'data': { 'format': 'BlockdevQcow2EncryptionFormat'}}
>>
>> +##
>> +# @ImageInfoSpecificQCow2EncryptionNoInfo:
>> +#
>> +# Only used for the qcow2 encryption format "from-image" in which the
>> +# actual encryption format is determined from the image header.
>> +# Therefore, this encryption format will never be reported in
>> +# ImageInfoSpecificQCow2Encryption.
>> +#
>> +# Since: 2.13
>> +##
>> +{ 'struct': 'ImageInfoSpecificQCow2EncryptionNoInfo',
>> + 'data': { } }
>> +
>> ##
>> # @ImageInfoSpecificQCow2Encryption:
>> #
>> @@ -52,7 +65,8 @@
>> 'base': 'ImageInfoSpecificQCow2EncryptionBase',
>> 'discriminator': 'format',
>> 'data': { 'aes': 'QCryptoBlockInfoQCow',
>> - 'luks': 'QCryptoBlockInfoLUKS' } }
>> + 'luks': 'QCryptoBlockInfoLUKS',
>> + 'from-image': 'ImageInfoSpecificQCow2EncryptionNoInfo' } }
>>
>> ##
>> # @ImageInfoSpecificQCow2:
>> @@ -2739,10 +2753,30 @@
>> # @BlockdevQcow2EncryptionFormat:
>> # @aes: AES-CBC with plain64 initialization venctors
>> #
>> +# @from-image: Determine the encryption format from the image
>> +# header. This only allows the use of the
>> +# key-secret option. (Since: 2.13)
>> +#
>> # Since: 2.10
>> ##
>> { 'enum': 'BlockdevQcow2EncryptionFormat',
>> - 'data': [ 'aes', 'luks' ] }
>> + 'data': [ 'aes', 'luks', 'from-image' ] }
>> +
>> +##
>> +# @BlockdevQcow2EncryptionSecret:
>> +#
>> +# Allows specifying a key-secret without specifying the exact
>> +# encryption format, which is determined automatically from the image
>> +# header.
>> +#
>> +# @key-secret: The ID of a QCryptoSecret object providing the
>> +# decryption key. Mandatory except when probing
>> +# image for metadata only.
>> +#
>> +# Since: 2.13
>> +##
>> +{ 'struct': 'BlockdevQcow2EncryptionSecret',
>> + 'data': { '*key-secret': 'str' } }
>>
>> ##
>> # @BlockdevQcow2Encryption:
>> @@ -2750,10 +2784,12 @@
>> # Since: 2.10
>> ##
>> { 'union': 'BlockdevQcow2Encryption',
>> - 'base': { 'format': 'BlockdevQcow2EncryptionFormat' },
>> + 'base': { '*format': 'BlockdevQcow2EncryptionFormat' },
>> 'discriminator': 'format',
>> + 'default-variant': 'from-image',
>> 'data': { 'aes': 'QCryptoBlockOptionsQCow',
>> - 'luks': 'QCryptoBlockOptionsLUKS'} }
>> + 'luks': 'QCryptoBlockOptionsLUKS',
>> + 'from-image': 'BlockdevQcow2EncryptionSecret' } }
>
> Bike-shedding on name, how about "auto" or "probe" ?
Sure. I like "probe" a bit better than "auto", although "auto" is what
we usually have... But I think "probe" is still a bit better.
>
> IIUC, this schema addition means the QAPI parser now allows
>
> encrypt.format=from-image,encrypt.key-secret=sec0,...other opts...
>
> but the code will not accept "from-image" as a valid string.
Ah, right, I forgot that. Will fix.
Thanks for reviewing!
Max
> eg qcow2_update_options_prepare() will do
>
> case QCOW_CRYPT_AES:
> if (encryptfmt && !g_str_equal(encryptfmt, "aes")) {
> error_setg(errp,
> "Header reported 'aes' encryption format but "
> "options specify '%s'", encryptfmt);
> ret = -EINVAL;
> goto fail;
> }
>
> ...snip....
>
> case QCOW_CRYPT_LUKS:
> if (encryptfmt && !g_str_equal(encryptfmt, "luks")) {
> error_setg(errp,
> "Header reported 'luks' encryption format but "
> "options specify '%s'", encryptfmt);
> ret = -EINVAL;
> goto fail;
> }
>
>
> Regards,
> Daniel
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-10 13:12 ` Eric Blake
2018-05-10 13:18 ` Eric Blake
@ 2018-05-11 17:38 ` Max Reitz
1 sibling, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 17:38 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 6368 bytes --]
On 2018-05-10 15:12, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> This patch allows specifying a discriminator that is an optional member
>> of the base struct. In such a case, a default value must be provided
>> that is used when no value is given.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> qapi/introspect.json | 8 ++++++
>> scripts/qapi/common.py | 57
>> ++++++++++++++++++++++++++++++++++--------
>> scripts/qapi/doc.py | 8 ++++--
>> scripts/qapi/introspect.py | 10 +++++---
>> scripts/qapi/visit.py | 33 ++++++++++++++++++++++--
>> tests/qapi-schema/test-qapi.py | 2 ++
>> 6 files changed, 101 insertions(+), 17 deletions(-)
>
> I've been threatening that we might need this for some time, so I'm glad
> to see it being implemented. We'll see if the tests in 2 and 3 cover
> the code added here.
>
>>
>> diff --git a/qapi/introspect.json b/qapi/introspect.json
>> index c7f67b7d78..2d7b1e3745 100644
>> --- a/qapi/introspect.json
>> +++ b/qapi/introspect.json
>> @@ -168,6 +168,13 @@
>> # @tag: the name of the member serving as type tag.
>> # An element of @members with this name must exist.
>> #
>> +# @default-variant: if the @tag element of @members is optional, this
>> +# is the default value for choosing a variant. Its
>> +# value must be a valid value for @tag.
>
> Perhaps s/must/will/ as this struct is used for output (and therefore we
> always meet the condition, rather than the user having to do something
> correctly).
I mostly copied from the other descriptions which seemed to prefer
"must", but I'm happy with either.
> Nice that you remembered introspection.
I didn't, because I had no idea how introspection works exactly before
this series. But one of the tests broke, thus telling me I might have
forgotten something. :-)
>> +# Present exactly when @tag is present and the
>> +# associated element of @members is optional.
>> +# (Since: 2.13)
>> +#
>> # @variants: variant members, i.e. additional members that
>> # depend on the type tag's value. Present exactly when
>> # @tag is present. The variants are in no particular order,
[...]
>> diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
>> index 5d72d8936c..ecffc46bd3 100644
>> --- a/scripts/qapi/visit.py
>> +++ b/scripts/qapi/visit.py
>> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members,
>> variants):
>> void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj,
>> Error **errp)
>> {
>> Error *err = NULL;
>> -
>> ''',
>> c_name=c_name(name))
>> + if variants:
>> + ret += mcgen('''
>> + %(c_type)s %(c_name)s;
>> +''',
>> + c_type=variants.tag_member.type.c_name(),
>> + c_name=c_name(variants.tag_member.name))
>> +
>> + ret += mcgen('''
>> +
>> +''')
>> +
>
> Creating a temp variable makes it easier to handle the default...
>
>> if base:
>> ret += mcgen('''
>> visit_type_%(c_type)s_members(v, (%(c_type)s *)obj, &err);
>> @@ -75,8 +85,27 @@ void visit_type_%(c_name)s_members(Visitor *v,
>> %(c_name)s *obj, Error **errp)
>> ''')
>> if variants:
>> + if variants.default_tag_value is None:
>> + ret += mcgen('''
>> + %(c_name)s = obj->%(c_name)s;
>> +''',
>> + c_name=c_name(variants.tag_member.name))
>> + else:
>> + ret += mcgen('''
>> + if (obj->has_%(c_name)s) {
>> + %(c_name)s = obj->%(c_name)s;
>> + } else {
>> + %(c_name)s = %(enum_const)s;
>> + }
>> +''',
>> + c_name=c_name(variants.tag_member.name),
>> + enum_const=c_enum_const(
>> + variants.tag_member.type.name,
>> + variants.default_tag_value,
>> + variants.tag_member.type.prefix))
>> +
>> ret += mcgen('''
>> - switch (obj->%(c_name)s) {
>> + switch (%(c_name)s) {
>
> ...compared to the old code that just inlined the one thing that used to
> be assigned to what is now the temporary variable.
>
> It might be possible to inline things so that the generated code reads
> either:
>
> switch (obj->discriminator) {
> switch (!obj->has_discriminator ? DEFAULT : obj->discriminator) {
>
> but I don't think it's worth the effort; and the temporary variable is
> fine even though it makes the generated file bigger.
I don't really mind either way, but depending on the default value and
the discriminator name, using ?: may lead to a rather long line.
>> ''',
>> c_name=c_name(variants.tag_member.name))
>> diff --git a/tests/qapi-schema/test-qapi.py
>> b/tests/qapi-schema/test-qapi.py
>> index c1a144ba29..f2a072b92e 100644
>> --- a/tests/qapi-schema/test-qapi.py
>> +++ b/tests/qapi-schema/test-qapi.py
>> @@ -56,6 +56,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
>> def _print_variants(variants):
>> if variants:
>> print(' tag %s' % variants.tag_member.name)
>> + if variants.default_tag_value:
>> + print(' default variant: %s' %
>> variants.default_tag_value)
>> for v in variants.variants:
>> print(' case %s: %s' % (v.name, v.type.name))
>>
>
> Looks good!
> Reviewed-by: Eric Blake <eblake@redhat.com>
Phew. :-)
Thanks!
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-10 13:18 ` Eric Blake
@ 2018-05-11 17:59 ` Max Reitz
2018-05-11 18:13 ` Eric Blake
0 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-11 17:59 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2176 bytes --]
On 2018-05-10 15:18, Eric Blake wrote:
> On 05/10/2018 08:12 AM, Eric Blake wrote:
>
> Oh, I just had a thought:
>
>>> +++ b/scripts/qapi/visit.py
>>> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members,
>
>>> if variants:
>>> + if variants.default_tag_value is None:
>>> + ret += mcgen('''
>>> + %(c_name)s = obj->%(c_name)s;
>>> +''',
>>> + c_name=c_name(variants.tag_member.name))
>>> + else:
>>> + ret += mcgen('''
>>> + if (obj->has_%(c_name)s) {
>>> + %(c_name)s = obj->%(c_name)s;
>>> + } else {
>>> + %(c_name)s = %(enum_const)s;
>
> In this branch of code, is it worth also generating:
>
> %has_(c_name)s = true;
You mean obj->has_%(c_name)s, and then also set obj->%(c_name)s?
> That way, the rest of the C code does not have to check
> has_discriminator, because the process of assigning the default will
> ensure that has_discriminator is always true later on. It does have the
> effect that output would never omit the discriminator - but that might
> be a good thing: if we ever have an output union that used to have a
> mandatory discriminator and want to now make it optional, we don't want
> to break older clients that expected the discriminator to be present. It
> does obscure whether input relied on the default, but I don't think that
> hurts.
Also, it would make code a bit simpler because it can cover the !has_X
case under X == default_X.
But OTOH, you could make that case for any optional value -- except
where "is missing" has special value.
And that's the tricky part: I think it's hard to say that a missing
discriminator never has special meaning. We only need the
default-variant so we know which variant of the union to pick; but we
don't know that the fact that the discriminator value was missing does
not have special meaning.
Of course, we can simply foreclose that by setting it here.
I don't have money in this game, so I suppose it's yours and Markus's
call. :-)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests
2018-05-10 14:08 ` Eric Blake
@ 2018-05-11 18:06 ` Max Reitz
0 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 18:06 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 8813 bytes --]
On 2018-05-10 16:08, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> There already is an optional discriminator test, although it also noted
>> the discriminator name itself as optional. Changing that, and adding a
>> default-variant field, makes that test pass instead of failing.
>
> I'm not sure I agree with that one. I think that you should instead add
> a positive test of a default variant into qapi-schema-test.json,
> especially since that is the one positive test where we also ensure that
> the C compiler is happy with the generated code (other positive tests
> prove that the generator produced something without error, but not that
> what it produced could be compiled).
Oh, sure.
>> 17 files changed, 61 insertions(+), 4 deletions(-)
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-no-default.json
>> create mode 100644
>> tests/qapi-schema/flat-union-superfluous-default-variant.json
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-no-default.err
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
>> create mode 100644
>> tests/qapi-schema/flat-union-optional-discriminator-no-default.out
>> create mode 100644
>> tests/qapi-schema/flat-union-superfluous-default-variant.err
>> create mode 100644
>> tests/qapi-schema/flat-union-superfluous-default-variant.exit
>> create mode 100644
>> tests/qapi-schema/flat-union-superfluous-default-variant.out
>
> Patch 1 converted 2 error messages into 6, where 2 of them look identical:
>
>>> - # The value of member 'discriminator' must name a non-optional
>>> - # member of the base struct.
>>> + # The value of member 'discriminator' must name a member of
>>> + # the base struct.
>>> check_name(info, "Discriminator of flat union '%s'" % name,
>>> discriminator)
>
> [0] check_name() checks that 'discriminator':'*name' is rejected - this
> check is unchanged
>
>>> - discriminator_type = base_members.get(discriminator)
>>> - if not discriminator_type:
>>> - raise QAPISemError(info,
>>> - "Discriminator '%s' is not a member
>>> of base "
>>> - "struct '%s'"
>>> - % (discriminator, base))
>
> The old code ensured that 'discriminator':'name' finds 'name' as a
> mandatory field in the base type; which changed into:
>
>>> + if default_variant is None:
>>> + discriminator_type = base_members.get(discriminator)
>>> + if not discriminator_type:
>>> + if base_members.get('*' + discriminator) is None:
>>> + raise QAPISemError(info,
>>> + "Discriminator '%s' is not a
>>> member of "
>>> + "base struct '%s'"
>>> + % (discriminator, base))
>
> [1] the discriminator type must be a member field (we didn't find either
> a mandatory or an optional field)
>
>>> + else:
>>> + raise QAPISemError(info,
>>> + "Default variant must be
>>> specified for "
>>> + "optional discriminator '%s'"
>>> + % discriminator)
>
> [2] without default_variant, the member field must be mandatory
>
>>> + else:
>>> + discriminator_type = base_members.get('*' + discriminator)
>>> + if not discriminator_type:
>>> + if base_members.get(discriminator) is None:
>>> + raise QAPISemError(info,
>>> + "Discriminator '%s' is not a
>>> member of "
>>> + "base struct '%s'"
>>> + % (discriminator, base))
>
> [3] the discriminator type must be a member field (we didn't find either
> a mandatory or an optional field), identical message to [1]
>
>>> + else:
>>> + raise QAPISemError(info,
>>> + "Must not specify a default
>>> variant for "
>>> + "non-optional discriminator
>>> '%s'"
>>> + % discriminator)
>
> [4] with default_variant, the member field must be optional
>
>>> +
>>> enum_define = enum_types.get(discriminator_type)
>>> allow_metas = ['struct']
>>> # Do not allow string discriminator
>>> @@ -763,6 +785,15 @@ def check_union(expr, info):
>>> "Discriminator '%s' must be of
>>> enumeration "
>>> "type" % discriminator)
>>>
>>> + if default_variant is not None:
>>> + # Must be a value of the enumeration
>>> + if default_variant not in enum_define['data']:
>>> + raise QAPISemError(info,
>>> + "Default variant '%s' of flat
>>> union '%s' is "
>>> + "not part of '%s'"
>>> + % (default_variant, name,
>>> + discriminator_type))
>>> +
>
> [5] the default discriminator value must belong to the discriminator's type
>
> Of those 6 messages, you now have coverage of:
>
>>
>> +++ b/tests/Makefile.include
>> @@ -500,7 +500,10 @@ qapi-schema += flat-union-invalid-branch-key.json
>> qapi-schema += flat-union-invalid-discriminator.json
>
> [1] no change to that test
>
>> qapi-schema += flat-union-no-base.json
>> qapi-schema += flat-union-optional-discriminator.json
>
> the old test for [0] - you turned it into success. See more below...
>
>> +qapi-schema += flat-union-optional-discriminator-no-default.json
>
> new test for [2]
>
>> +qapi-schema += flat-union-optional-discriminator-invalid-default.json
>
> new test for [5]
>
>> qapi-schema += flat-union-string-discriminator.json
>> +qapi-schema += flat-union-superfluous-default-variant.json
>
> new test for [4]
>
> I don't know if it is worth trying to explicitly test for [3], given
> that it is identical to [1], but the loss of the test for [0] bothers me.
>
>> +++ b/tests/qapi-schema/flat-union-optional-discriminator.err
>> @@ -1 +0,0 @@
>> -tests/qapi-schema/flat-union-optional-discriminator.json:6:
>> Discriminator of flat union 'MyUnion' does not allow optional name
>> '*switch'
>
> This used to be covered by [0] - but you rewrote the test by changing
> 'discriminator':'*switch' to 'discriminator':'switch'.
>
> I'd argue that you WANT to keep 'discriminator':'*switch' in this test,
> to keep the testing of error [0] (even when the discriminator type is
> optional, we spell it without leading '*' in the 'union' description),
> and then only qapi-schema-test.json gets a positive test, which also
> gives us the additional coverage that the C compiler likes our generated
> code.
OK, but I'd like to rename the test, then.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow encryption probing
2018-05-10 14:24 ` Eric Blake
2018-05-10 14:32 ` Daniel P. Berrangé
@ 2018-05-11 18:07 ` Max Reitz
1 sibling, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 18:07 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]
On 2018-05-10 16:24, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> Currently, you can give no encryption format for a qcow file while still
>> passing a key-secret. That does not conform to the schema, so this
>> patch changes the schema to allow it.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>
>> ##
>> # @BlockdevQcowEncryptionFormat:
>> #
>> # @aes: AES-CBC with plain64 initialization vectors
>> #
>> +# @from-image: Determine the encryption format from the image
>> +# header. This only allows the use of the
>> +# key-secret option. (Since: 2.13)
>> +#
>> # Since: 2.10
>> ##
>> { 'enum': 'BlockdevQcowEncryptionFormat',
>> - 'data': [ 'aes' ] }
>> + 'data': [ 'aes', 'from-image' ] }
>
> Overkill. Why not just:
>
>> ##
>> # @BlockdevQcowEncryption:
>> @@ -2728,9 +2748,11 @@
>> # Since: 2.10
>> ##
>> { 'union': 'BlockdevQcowEncryption',
>> - 'base': { 'format': 'BlockdevQcowEncryptionFormat' },
>> + 'base': { '*format': 'BlockdevQcowEncryptionFormat' },
>> 'discriminator': 'format',
>> - 'data': { 'aes': 'QCryptoBlockOptionsQCow' } }
>> + 'default-variant': 'from-image',
>
> 'default-variant': 'aes'
>
>> + 'data': { 'aes': 'QCryptoBlockOptionsQCow',
>
> and call it good, because there are no other options to pick from, so
> 'from-image' would always resolve to 'aes' anyway.
Hmmmm. Yes. :-)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-05-10 14:54 ` Eric Blake
@ 2018-05-11 18:11 ` Max Reitz
2018-06-06 13:10 ` Markus Armbruster
0 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-11 18:11 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2793 bytes --]
On 2018-05-10 16:54, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> There are numerous QDict functions that have been introduced for and are
>> used only by the block layer. Move their declarations into an own
>
> s/an own/their own/
>
>> header file to reflect that.
>>
>> While qdict_extract_subqdict() is in fact used outside of the block
>> layer (in util/qemu-config.c), it is still a function related very
>> closely to how the block layer works with nested QDicts, namely by
>> sometimes flattening them. Therefore, its declaration is put into this
>> header as well and util/qemu-config.c includes it with a comment stating
>> exactly which function it needs.
>>
>> Suggested-by: Markus Armbruster <armbru@redhat.com>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>
>> +++ b/include/block/qdict.h
>> @@ -0,0 +1,35 @@
>> +/*
>> + * Special QDict functions used by the block layer
>> + *
>> + * Copyright (c) 2018 Red Hat, Inc.
>
> You are extracting this from qdict.h which has:
> * Copyright (C) 2009 Red Hat Inc.
>
> Is it worth listing 2009-2018, instead of just this year?
I don't know, is it? I don't know whether it makes a real difference.
>> +
>> +void qdict_copy_default(QDict *dst, QDict *src, const char *key);
>> +void qdict_set_default_str(QDict *dst, const char *key, const char
>> *val);
>
> These two might be useful outside of the block layer; we can move them
> back in a later patch if so. But for now, I agree with your choice of
> moving them.
>
>> +
>> +void qdict_join(QDict *dest, QDict *src, bool overwrite);
>
> This is borderline whether it would be useful outside of the block layer.
I decided I wanted to move the *_default* functions, and if I did that,
I would have to move this one as well. I decided I can't be biased just
because I wrote this one. :-)
But in general I'd say if anyone wants to use any of these functions
outside of the block layer, they are welcome to move them back to the
main header, provided they have a good reason to do so. I suppose a
good reason for using qdict_join() may indeed turn up sooner or later.
Max
>> +
>> +void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>> +void qdict_array_split(QDict *src, QList **dst);
>> +int qdict_array_entries(QDict *src, const char *subqdict);
>> +QObject *qdict_crumple(const QDict *src, Error **errp);
>> +void qdict_flatten(QDict *qdict);
>
> And these are definitely hacks that the block layer relies on, where
> hopefully someday long term we can rewrite the block layer to use QAPI
> types directly instead of constant reconversion between QemuOpts and
> QDict and QAPI types.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions
2018-05-11 17:59 ` Max Reitz
@ 2018-05-11 18:13 ` Eric Blake
0 siblings, 0 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-11 18:13 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/11/2018 12:59 PM, Max Reitz wrote:
> On 2018-05-10 15:18, Eric Blake wrote:
>> On 05/10/2018 08:12 AM, Eric Blake wrote:
>>
>> Oh, I just had a thought:
>>
>>>> +++ b/scripts/qapi/visit.py
>>>> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members,
>>
>>>> if variants:
>>>> + if variants.default_tag_value is None:
>>>> + ret += mcgen('''
>>>> + %(c_name)s = obj->%(c_name)s;
>>>> +''',
>>>> + c_name=c_name(variants.tag_member.name))
>>>> + else:
>>>> + ret += mcgen('''
>>>> + if (obj->has_%(c_name)s) {
>>>> + %(c_name)s = obj->%(c_name)s;
>>>> + } else {
>>>> + %(c_name)s = %(enum_const)s;
>>
>> In this branch of code, is it worth also generating:
>>
>> %has_(c_name)s = true;
>
> You mean obj->has_%(c_name)s, and then also set obj->%(c_name)s?
Umm, yeah ;)
In fact, I think it is as simple as:
if variants:
if variants.default_tag_value:
ret += mcgen('''
if (!obj->has_%(c_name)s) {
obj->has_%(c_name)s = true;
obj->%(c_name)s = %(enum_const)s;
}
''')
ret += mcgen('''
switch (obj->%(c_name)s) {
...
and you are back to not needing a temporary variable.
>
>> That way, the rest of the C code does not have to check
>> has_discriminator, because the process of assigning the default will
>> ensure that has_discriminator is always true later on. It does have the
>> effect that output would never omit the discriminator - but that might
>> be a good thing: if we ever have an output union that used to have a
>> mandatory discriminator and want to now make it optional, we don't want
>> to break older clients that expected the discriminator to be present. It
>> does obscure whether input relied on the default, but I don't think that
>> hurts.
>
> Also, it would make code a bit simpler because it can cover the !has_X
> case under X == default_X.
>
> But OTOH, you could make that case for any optional value -- except
> where "is missing" has special value.
My preference - special-casing "is missing" is prone to abuse. I don't
want to support that if we can at all avoid it. The sane semantics is
that the default is populated as soon as we detect that something is
missing, and whether the user relies on the default by leaving the
discriminator absent, or explicitly supplies the discriminator set to
the default, the behavior should always be the same.
>
> And that's the tricky part: I think it's hard to say that a missing
> discriminator never has special meaning.
It won't, if we decide right now that we don't want to let it :)
> We only need the
> default-variant so we know which variant of the union to pick; but we
> don't know that the fact that the discriminator value was missing does
> not have special meaning.
>
> Of course, we can simply foreclose that by setting it here.
And that's the way I'm leaning.
>
> I don't have money in this game, so I suppose it's yours and Markus's
> call. :-)
Markus, what's your preference?
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test
2018-05-10 16:02 ` Eric Blake
@ 2018-05-11 18:13 ` Max Reitz
2018-05-11 18:33 ` Eric Blake
0 siblings, 1 reply; 42+ messages in thread
From: Max Reitz @ 2018-05-11 18:13 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1539 bytes --]
On 2018-05-10 18:02, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> tests/check-qdict.c | 54
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 54 insertions(+)
>>
>
>> +static void qdict_stringify_for_keyval_test(void)
>> +{
>> + QDict *dict = qdict_new();
>> +
>> + /*
>> + * Test stringification of:
>> + *
>> + * {
>> + * "a": "null",
>> + * "b": 42,
>> + * "c": -23,
>> + * "d": false,
>> + * "e": null,
>> + * "f": "",
>> + * "g": 0.5,
>> + * "h": 0xffffffffffffffff,
>> + * "i": true,
>> + * "j": 0
>
> Is it worth testing fun things like '-0.0'?
Sure, why not. Maybe even infinity, although I'm not quite sure the
input visitor can handle it...
>> + g_assert(!strcmp(qdict_get_str(dict, "a"), "null"));
>> + g_assert(!strcmp(qdict_get_str(dict, "b"), "42"));
>> + g_assert(!strcmp(qdict_get_str(dict, "c"), "-23"));
>> + g_assert(!strcmp(qdict_get_str(dict, "d"), "off"));
>> + g_assert(qobject_type(qdict_get(dict, "e")) == QTYPE_QNULL);
>
> Is it worth shortening this line to:
> g_assert(qobject_to(QNull, qdict_get(dict, "e")));
I think explicitly checking the type is a bit more expressive.
Max
> Reviewed-by: Eric Blake <eblake@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test
2018-05-11 18:13 ` Max Reitz
@ 2018-05-11 18:33 ` Eric Blake
0 siblings, 0 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-11 18:33 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/11/2018 01:13 PM, Max Reitz wrote:
>>> + * "h": 0xffffffffffffffff,
>>> + * "i": true,
>>> + * "j": 0
>>
>> Is it worth testing fun things like '-0.0'?
>
> Sure, why not. Maybe even infinity, although I'm not quite sure the
> input visitor can handle it...
JSON can't handle Inf or NaN, even if the input visitor can. So
probably best to not worry about those.
>>> + g_assert(!strcmp(qdict_get_str(dict, "a"), "null"));
>>> + g_assert(!strcmp(qdict_get_str(dict, "b"), "42"));
>>> + g_assert(!strcmp(qdict_get_str(dict, "c"), "-23"));
>>> + g_assert(!strcmp(qdict_get_str(dict, "d"), "off"));
>>> + g_assert(qobject_type(qdict_get(dict, "e")) == QTYPE_QNULL);
>>
>> Is it worth shortening this line to:
>> g_assert(qobject_to(QNull, qdict_get(dict, "e")));
>
> I think explicitly checking the type is a bit more expressive.
Okay (qobject_to() checks the type under the hood, and returns non-NULL
only when it was the right type - but I see your point about that being
a bit more magic)
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval()
2018-05-09 16:55 ` [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval() Max Reitz
@ 2018-05-11 18:39 ` Eric Blake
2018-05-11 21:42 ` Max Reitz
0 siblings, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-11 18:39 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> The purpose of this function is to prepare a QDict for consumption by
> the keyval visitor, which only accepts strings and QNull.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> include/block/qdict.h | 2 ++
> qobject/qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 59 insertions(+)
>
> +/**
> + * Convert all values in a QDict so it can be consumed by the keyval
> + * input visitor. QNull values are left as-is, all other values are
> + * converted to strings.
> + *
> + * @qdict must be flattened, i.e. it may not contain any nested QDicts
> + * or QLists.
Maybe s/flattened/flattened already/
or would it be worth letting this function PERFORM the flattening step
automatically?
> + */
> +void qdict_stringify_for_keyval(QDict *qdict)
> +{
> + const QDictEntry *e;
> +
> + for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
> + QString *new_value = NULL;
> +
> + switch (qobject_type(e->value)) {
> + case QTYPE_QNULL:
> + /* leave as-is */
> + break;
> +
> + case QTYPE_QNUM: {
> + char *str = qnum_to_string(qobject_to(QNum, e->value));
> + new_value = qstring_from_str(str);
> + g_free(str);
> + break;
> + }
> +
> + case QTYPE_QSTRING:
> + /* leave as-is */
> + break;
> +
> + case QTYPE_QDICT:
> + case QTYPE_QLIST:
> + /* @qdict must be flattened */
> + abort();
Matches your documentation about requiring it to be already flattened.
> +
> + case QTYPE_QBOOL:
> + if (qbool_get_bool(qobject_to(QBool, e->value))) {
> + new_value = qstring_from_str("on");
> + } else {
> + new_value = qstring_from_str("off");
> + }
> + break;
> +
> + case QTYPE_NONE:
> + case QTYPE__MAX:
> + abort();
> + }
> +
> + if (new_value) {
> + QDictEntry *mut_e = (QDictEntry *)e;
A bit of a shame that you have to cast away const. It took me a couple
reads to figure out this meant 'mutable_e'. But in the end, the code
looks correct.
> + qobject_unref(mut_e->value);
> + mut_e->value = QOBJECT(new_value);
If we're okay requiring the caller to pre-flatten before calling this,
instead of offering to do it here,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly
2018-05-09 16:55 ` [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly Max Reitz
@ 2018-05-11 18:44 ` Eric Blake
0 siblings, 0 replies; 42+ messages in thread
From: Eric Blake @ 2018-05-11 18:44 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> In its current form, qdict_flatten() removes all entries from nested
> QDicts that are moved to the root QDict. It is completely sufficient to
> remove all old entries from the root QDict, however. If the nested
> dicts have a refcount of 1, this will automatically delete them, too.
> And if they have a greater refcount, we probably do not want to modify
> them in the first place.
>
> The latter observation means that it was currently (in general)
> impossible to qdict_flatten() a shallowly cloned dict because that would
> empty nested QDicts in the original dict as well. This patch changes
> this, so you can now use qdict_flatten(qdict_shallow_clone(dict)) to get
> a flattened copy without disturbing the original.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> qobject/qdict.c | 23 ++++++++++++++++-------
> 1 file changed, 16 insertions(+), 7 deletions(-)
>
>
> g_free(new_key);
>
> - if (delete) {
> + if (copied_value && qdict == target) {
> + /* If we have copied a value, and if we are on the root
> + * level, we need to remove the old entry. Otherwise, we
> + * do not, because by removing these entries on the root
> + * level, the reference counts of nested dicts and listed
s/listed/lists/
> + * will be reduced automatically. In fact, we probably do
> + * not want to modify nested dicts and lists with
> + * refcounts greater than 1 anyway. */
> qdict_del(qdict, entry->key);
>
> - /* Restart loop after modifying the iterated QDict */
> + /* Restart loop after modifying the iterated QDict. We
> + * only need to do this if qdict == target, because
> + * otherwise copying the value did not affect qdict. */
> entry = qdict_first(qdict);
> continue;
> }
>
With the typo fix,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test
2018-05-09 16:55 ` [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test Max Reitz
@ 2018-05-11 18:46 ` Eric Blake
2018-05-11 21:41 ` Max Reitz
0 siblings, 1 reply; 42+ messages in thread
From: Eric Blake @ 2018-05-11 18:46 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
On 05/09/2018 11:55 AM, Max Reitz wrote:
> This new test verifies that qdict_flatten() does not modify a shallow
> clone of the given QDict.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/check-qdict.c | 33 +++++++++++++++++++++++++++++++++
> 1 file changed, 33 insertions(+)
>
I'm not sure I even want to know how long it took you to debug the crash
that you obviously hit before adding the fix in 9/13 plus this test ;)
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test
2018-05-11 18:46 ` Eric Blake
@ 2018-05-11 21:41 ` Max Reitz
0 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 21:41 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]
On 2018-05-11 20:46, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> This new test verifies that qdict_flatten() does not modify a shallow
>> clone of the given QDict.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> tests/check-qdict.c | 33 +++++++++++++++++++++++++++++++++
>> 1 file changed, 33 insertions(+)
>>
>
> I'm not sure I even want to know how long it took you to debug the crash
> that you obviously hit before adding the fix in 9/13 plus this test ;)
Thank you very much, I made myself forget about that trauma already.
In short, I wondered why the whole thing worked for null-co directly:
driver=null-co,size=512
=> {"driver": "null-co", "size": 512}
But not for null-co through raw:
driver=raw,file.driver=null-co,file.size=512
=> {"file": {}}
(Or something like that, I don't remember exactly.)
With some debugging sprinkled into block.c, I could see that the correct
options were there on the null-co level... But for some reason they
disappeared one level above.
Then I recalled that dict cloning is just a shallow cloning and looked
for the culprit...
Of course, in reality, much more cursing was involved.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval()
2018-05-11 18:39 ` Eric Blake
@ 2018-05-11 21:42 ` Max Reitz
0 siblings, 0 replies; 42+ messages in thread
From: Max Reitz @ 2018-05-11 21:42 UTC (permalink / raw)
To: Eric Blake, qemu-block
Cc: qemu-devel, Markus Armbruster, Kevin Wolf, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 3339 bytes --]
On 2018-05-11 20:39, Eric Blake wrote:
> On 05/09/2018 11:55 AM, Max Reitz wrote:
>> The purpose of this function is to prepare a QDict for consumption by
>> the keyval visitor, which only accepts strings and QNull.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> include/block/qdict.h | 2 ++
>> qobject/qdict.c | 57
>> +++++++++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 59 insertions(+)
>>
>
>> +/**
>> + * Convert all values in a QDict so it can be consumed by the keyval
>> + * input visitor. QNull values are left as-is, all other values are
>> + * converted to strings.
>> + *
>> + * @qdict must be flattened, i.e. it may not contain any nested QDicts
>> + * or QLists.
>
> Maybe s/flattened/flattened already/
>
> or would it be worth letting this function PERFORM the flattening step
> automatically?
Possibly, but I don't think it would be any more efficient, so I'd
rather leave it up to the caller to do it explicitly than to do it here
and maybe surprise someone.
(Indeed I personally prefer to be surprised by an abort() than by some
unintended data change.)
Max
>> + */
>> +void qdict_stringify_for_keyval(QDict *qdict)
>> +{
>> + const QDictEntry *e;
>> +
>> + for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
>> + QString *new_value = NULL;
>> +
>> + switch (qobject_type(e->value)) {
>> + case QTYPE_QNULL:
>> + /* leave as-is */
>> + break;
>> +
>> + case QTYPE_QNUM: {
>> + char *str = qnum_to_string(qobject_to(QNum, e->value));
>> + new_value = qstring_from_str(str);
>> + g_free(str);
>> + break;
>> + }
>> +
>> + case QTYPE_QSTRING:
>> + /* leave as-is */
>> + break;
>> +
>> + case QTYPE_QDICT:
>> + case QTYPE_QLIST:
>> + /* @qdict must be flattened */
>> + abort();
>
> Matches your documentation about requiring it to be already flattened.
>
>> +
>> + case QTYPE_QBOOL:
>> + if (qbool_get_bool(qobject_to(QBool, e->value))) {
>> + new_value = qstring_from_str("on");
>> + } else {
>> + new_value = qstring_from_str("off");
>> + }
>> + break;
>> +
>> + case QTYPE_NONE:
>> + case QTYPE__MAX:
>> + abort();
>> + }
>> +
>> + if (new_value) {
>> + QDictEntry *mut_e = (QDictEntry *)e;
>
> A bit of a shame that you have to cast away const. It took me a couple
> reads to figure out this meant 'mutable_e'. But in the end, the code
> looks correct.
>
>> + qobject_unref(mut_e->value);
>> + mut_e->value = QOBJECT(new_value);
>
> If we're okay requiring the caller to pre-flatten before calling this,
> instead of offering to do it here,
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-05-11 18:11 ` Max Reitz
@ 2018-06-06 13:10 ` Markus Armbruster
0 siblings, 0 replies; 42+ messages in thread
From: Markus Armbruster @ 2018-06-06 13:10 UTC (permalink / raw)
To: Max Reitz; +Cc: Eric Blake, qemu-block, Kevin Wolf, Michael Roth, qemu-devel
Max Reitz <mreitz@redhat.com> writes:
> On 2018-05-10 16:54, Eric Blake wrote:
>> On 05/09/2018 11:55 AM, Max Reitz wrote:
>>> There are numerous QDict functions that have been introduced for and are
>>> used only by the block layer. Move their declarations into an own
>>
>> s/an own/their own/
>>
>>> header file to reflect that.
>>>
>>> While qdict_extract_subqdict() is in fact used outside of the block
>>> layer (in util/qemu-config.c), it is still a function related very
>>> closely to how the block layer works with nested QDicts, namely by
>>> sometimes flattening them. Therefore, its declaration is put into this
>>> header as well and util/qemu-config.c includes it with a comment stating
>>> exactly which function it needs.
>>>
>>> Suggested-by: Markus Armbruster <armbru@redhat.com>
>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>>> ---
>>
>>> +++ b/include/block/qdict.h
>>> @@ -0,0 +1,35 @@
>>> +/*
>>> + * Special QDict functions used by the block layer
>>> + *
>>> + * Copyright (c) 2018 Red Hat, Inc.
>>
>> You are extracting this from qdict.h which has:
>> * Copyright (C) 2009 Red Hat Inc.
>>
>> Is it worth listing 2009-2018, instead of just this year?
>
> I don't know, is it? I don't know whether it makes a real difference.
Where's your taste for pedantry? Here's mine: please make it 2013-2018,
because the oldest code moved is from 2013.
>>> + *
>>> + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
>>> + * See the COPYING.LIB file in the top-level directory.
>>> + */
>>> +
>>> +#ifndef BLOCK_QDICT_H
>>> +#define BLOCK_QDICT_H
>>> +
>>> +#include "qapi/qmp/qdict.h"
>>> +#include "qapi/qmp/qobject.h"
>>> +#include "qapi/error.h"
Only the first #include is necessary, due to qemu/typedefs.h. Please
drop the other two.
>>> +
>>> +
>>> +void qdict_copy_default(QDict *dst, QDict *src, const char *key);
>>> +void qdict_set_default_str(QDict *dst, const char *key, const char
>>> *val);
>>
>> These two might be useful outside of the block layer; we can move them
>> back in a later patch if so. But for now, I agree with your choice of
>> moving them.
>>
>>> +
>>> +void qdict_join(QDict *dest, QDict *src, bool overwrite);
>>
>> This is borderline whether it would be useful outside of the block layer.
>
> I decided I wanted to move the *_default* functions, and if I did that,
> I would have to move this one as well. I decided I can't be biased just
> because I wrote this one. :-)
>
> But in general I'd say if anyone wants to use any of these functions
> outside of the block layer, they are welcome to move them back to the
> main header, provided they have a good reason to do so. I suppose a
> good reason for using qdict_join() may indeed turn up sooner or later.
I like it this way.
With the trivial changes I asked for:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-05-09 16:55 ` [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header Max Reitz
2018-05-10 14:54 ` Eric Blake
@ 2018-06-06 13:17 ` Markus Armbruster
2018-06-06 14:19 ` Markus Armbruster
1 sibling, 1 reply; 42+ messages in thread
From: Markus Armbruster @ 2018-06-06 13:17 UTC (permalink / raw)
To: Max Reitz; +Cc: qemu-block, Kevin Wolf, qemu-devel, Michael Roth
MAINTAINERS files include/block/qdict.h under "Block layer core". Let's
split qobject/qdict.c as well, so it's there, too. Obvious patch
appended. Feel free to squash it into yours without giving me credit
for it.
PS: I tried to move block-qdict.c to block/, but totally failed at
persuading Make to link it into all programs that need it. Oh well.
>From 4614533375729c84c0599e1fed44420b29a180a6 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Wed, 6 Jun 2018 15:11:54 +0200
Subject: [PATCH] qobject: Move block-specific qdict code to block-qdict.c
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
MAINTAINERS | 1 +
qobject/Makefile.objs | 1 +
qobject/block-qdict.c | 637 ++++++++++++++++++++++++++++++++++++++++++
qobject/qdict.c | 629 -----------------------------------------
4 files changed, 639 insertions(+), 629 deletions(-)
create mode 100644 qobject/block-qdict.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 41cd3736a9..9f9835e052 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1365,6 +1365,7 @@ F: qemu-img*
F: qemu-io*
F: tests/qemu-iotests/
F: util/qemu-progress.c
+F: qobject/block-qdict.c
T: git git://repo.or.cz/qemu/kevin.git block
Block I/O path
diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs
index 002d25873a..7b12c9cacf 100644
--- a/qobject/Makefile.objs
+++ b/qobject/Makefile.objs
@@ -1,2 +1,3 @@
util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o
util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
+util-obj-y += block-qdict.o
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
new file mode 100644
index 0000000000..01c453a8c0
--- /dev/null
+++ b/qobject/block-qdict.c
@@ -0,0 +1,637 @@
+/*
+ * Special QDict functions used by the block layer
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "block/qdict.h"
+#include "qapi/qmp/qlist.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+
+/**
+ * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
+ * value of 'key' in 'src' is copied there (and the refcount increased
+ * accordingly).
+ */
+void qdict_copy_default(QDict *dst, QDict *src, const char *key)
+{
+ QObject *val;
+
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ val = qdict_get(src, key);
+ if (val) {
+ qdict_put_obj(dst, key, qobject_ref(val));
+ }
+}
+
+/**
+ * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
+ * new QString initialised by 'val' is put there.
+ */
+void qdict_set_default_str(QDict *dst, const char *key, const char *val)
+{
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ qdict_put_str(dst, key, val);
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target,
+ const char *prefix);
+
+static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QListEntry *entry;
+ char *new_key;
+ int i;
+
+ /* This function is never called with prefix == NULL, i.e., it is always
+ * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
+ * need to remove list entries during the iteration (the whole list will be
+ * deleted eventually anyway from qdict_flatten_qdict()). */
+ assert(prefix);
+
+ entry = qlist_first(qlist);
+
+ for (i = 0; entry; entry = qlist_next(entry), i++) {
+ value = qlist_entry_obj(entry);
+ new_key = g_strdup_printf("%s.%i", prefix, i);
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
+ } else {
+ /* All other types are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ }
+
+ g_free(new_key);
+ }
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QDictEntry *entry, *next;
+ char *new_key;
+ bool delete;
+
+ entry = qdict_first(qdict);
+
+ while (entry != NULL) {
+
+ next = qdict_next(qdict, entry);
+ value = qdict_entry_value(entry);
+ new_key = NULL;
+ delete = false;
+
+ if (prefix) {
+ new_key = g_strdup_printf("%s.%s", prefix, entry->key);
+ }
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ /* Entries of QDicts are processed recursively, the QDict object
+ * itself disappears. */
+ qdict_flatten_qdict(qobject_to(QDict, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (prefix) {
+ /* All other objects are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ delete = true;
+ }
+
+ g_free(new_key);
+
+ if (delete) {
+ qdict_del(qdict, entry->key);
+
+ /* Restart loop after modifying the iterated QDict */
+ entry = qdict_first(qdict);
+ continue;
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_flatten(): For each nested QDict with key x, all fields with key y
+ * are moved to this QDict and their key is renamed to "x.y". For each nested
+ * QList with key x, the field at index y is moved to this QDict with the key
+ * "x.y" (i.e., the reverse of what qdict_array_split() does).
+ * This operation is applied recursively for nested QDicts and QLists.
+ */
+void qdict_flatten(QDict *qdict)
+{
+ qdict_flatten_qdict(qdict, qdict, NULL);
+}
+
+/* extract all the src QDict entries starting by start into dst */
+void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
+
+{
+ const QDictEntry *entry, *next;
+ const char *p;
+
+ *dst = qdict_new();
+ entry = qdict_first(src);
+
+ while (entry != NULL) {
+ next = qdict_next(src, entry);
+ if (strstart(entry->key, start, &p)) {
+ qdict_put_obj(*dst, p, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+ entry = next;
+ }
+}
+
+static int qdict_count_prefixed_entries(const QDict *src, const char *start)
+{
+ const QDictEntry *entry;
+ int count = 0;
+
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (strstart(entry->key, start, NULL)) {
+ if (count == INT_MAX) {
+ return -ERANGE;
+ }
+ count++;
+ }
+ }
+
+ return count;
+}
+
+/**
+ * qdict_array_split(): This function moves array-like elements of a QDict into
+ * a new QList. Every entry in the original QDict with a key "%u" or one
+ * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
+ * incrementally counting up, will be moved to a new QDict at index %u in the
+ * output QList with the key prefix removed, if that prefix is "%u.". If the
+ * whole key is just "%u", the whole QObject will be moved unchanged without
+ * creating a new QDict. The function terminates when there is no entry in the
+ * QDict with a prefix directly (incrementally) following the last one; it also
+ * returns if there are both entries with "%u" and "%u." for the same index %u.
+ * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
+ * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
+ * => [{"a": 42, "b": 23}, {"x": 0}, 66]
+ * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
+ */
+void qdict_array_split(QDict *src, QList **dst)
+{
+ unsigned i;
+
+ *dst = qlist_new();
+
+ for (i = 0; i < UINT_MAX; i++) {
+ QObject *subqobj;
+ bool is_subqdict;
+ QDict *subqdict;
+ char indexstr[32], prefix[32];
+ size_t snprintf_ret;
+
+ snprintf_ret = snprintf(indexstr, 32, "%u", i);
+ assert(snprintf_ret < 32);
+
+ subqobj = qdict_get(src, indexstr);
+
+ snprintf_ret = snprintf(prefix, 32, "%u.", i);
+ assert(snprintf_ret < 32);
+
+ /* Overflow is the same as positive non-zero results */
+ is_subqdict = qdict_count_prefixed_entries(src, prefix);
+
+ // There may be either a single subordinate object (named "%u") or
+ // multiple objects (each with a key prefixed "%u."), but not both.
+ if (!subqobj == !is_subqdict) {
+ break;
+ }
+
+ if (is_subqdict) {
+ qdict_extract_subqdict(src, &subqdict, prefix);
+ assert(qdict_size(subqdict) > 0);
+ } else {
+ qobject_ref(subqobj);
+ qdict_del(src, indexstr);
+ }
+
+ qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
+ }
+}
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * e.g.
+ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned 'prefix'
+ * string. The 'suffix' string will be left in escaped format, so it
+ * can be fed back into the qdict_split_flat_key() key as the input
+ * later.
+ *
+ * The caller is responsible for freeing the string returned in @prefix
+ * using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix,
+ const char **suffix)
+{
+ const char *separator;
+ size_t i, j;
+
+ /* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+ separator = NULL;
+ do {
+ if (separator) {
+ separator += 2;
+ } else {
+ separator = key;
+ }
+ separator = strchr(separator, '.');
+ } while (separator && separator[1] == '.');
+
+ if (separator) {
+ *prefix = g_strndup(key, separator - key);
+ *suffix = separator + 1;
+ } else {
+ *prefix = g_strdup(key);
+ *suffix = NULL;
+ }
+
+ /* Unescape the '..' sequence into '.' */
+ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+ if ((*prefix)[i] == '.') {
+ assert((*prefix)[i + 1] == '.');
+ i++;
+ }
+ (*prefix)[j] = (*prefix)[i];
+ }
+ (*prefix)[j] = '\0';
+}
+
+/**
+ * qdict_is_list:
+ * @maybe_list: dict to check if keys represent list elements.
+ *
+ * Determine whether all keys in @maybe_list are valid list elements.
+ * If @maybe_list is non-zero in length and all the keys look like
+ * valid list indexes, this will return 1. If @maybe_list is zero
+ * length or all keys are non-numeric then it will return 0 to indicate
+ * it is a normal qdict. If there is a mix of numeric and non-numeric
+ * keys, or the list indexes are non-contiguous, an error is reported.
+ *
+ * Returns: 1 if a valid list, 0 if a dict, -1 on error
+ */
+static int qdict_is_list(QDict *maybe_list, Error **errp)
+{
+ const QDictEntry *ent;
+ ssize_t len = 0;
+ ssize_t max = -1;
+ int is_list = -1;
+ int64_t val;
+
+ for (ent = qdict_first(maybe_list); ent != NULL;
+ ent = qdict_next(maybe_list, ent)) {
+
+ if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
+ if (is_list == -1) {
+ is_list = 1;
+ } else if (!is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ len++;
+ if (val > max) {
+ max = val;
+ }
+ } else {
+ if (is_list == -1) {
+ is_list = 0;
+ } else if (is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ }
+ }
+
+ if (is_list == -1) {
+ assert(!qdict_size(maybe_list));
+ is_list = 0;
+ }
+
+ /* NB this isn't a perfect check - e.g. it won't catch
+ * a list containing '1', '+1', '01', '3', but that
+ * does not matter - we've still proved that the
+ * input is a list. It is up the caller to do a
+ * stricter check if desired */
+ if (len != (max + 1)) {
+ error_setg(errp, "List indices are not contiguous, "
+ "saw %zd elements but %zd largest index",
+ len, max);
+ return -1;
+ }
+
+ return is_list;
+}
+
+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary (only scalar values) to crumple
+ *
+ * Takes a flat dictionary whose keys use '.' separator to indicate
+ * nesting, and values are scalars, and crumples it into a nested
+ * structure.
+ *
+ * To include a literal '.' in a key name, it must be escaped as '..'
+ *
+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in an output of:
+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *
+ * The following scenarios in the input dict will result in an
+ * error being returned:
+ *
+ * - Any values in @src are non-scalar types
+ * - If keys in @src imply that a particular level is both a
+ * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
+ * - If keys in @src imply that a particular level is a list,
+ * but the indices are non-contiguous. e.g. "foo.0.bar" and
+ * "foo.2.bar" without any "foo.1.bar" present.
+ * - If keys in @src represent list indexes, but are not in
+ * the "%zu" format. e.g. "foo.+0.bar"
+ *
+ * Returns: either a QDict or QList for the nested data structure, or NULL
+ * on error
+ */
+QObject *qdict_crumple(const QDict *src, Error **errp)
+{
+ const QDictEntry *ent;
+ QDict *two_level, *multi_level = NULL;
+ QObject *dst = NULL, *child;
+ size_t i;
+ char *prefix = NULL;
+ const char *suffix = NULL;
+ int is_list;
+
+ two_level = qdict_new();
+
+ /* Step 1: split our totally flat dict into a two level dict */
+ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
+ if (qobject_type(ent->value) == QTYPE_QDICT ||
+ qobject_type(ent->value) == QTYPE_QLIST) {
+ error_setg(errp, "Value %s is not a scalar",
+ ent->key);
+ goto error;
+ }
+
+ qdict_split_flat_key(ent->key, &prefix, &suffix);
+
+ child = qdict_get(two_level, prefix);
+ if (suffix) {
+ QDict *child_dict = qobject_to(QDict, child);
+ if (!child_dict) {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a scalar",
+ prefix);
+ goto error;
+ }
+
+ child_dict = qdict_new();
+ qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
+ }
+
+ qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
+ } else {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a dict",
+ prefix);
+ goto error;
+ }
+ qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
+ }
+
+ g_free(prefix);
+ prefix = NULL;
+ }
+
+ /* Step 2: optionally process the two level dict recursively
+ * into a multi-level dict */
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
+ QDict *dict = qobject_to(QDict, ent->value);
+ if (dict) {
+ child = qdict_crumple(dict, errp);
+ if (!child) {
+ goto error;
+ }
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
+ }
+ }
+ qobject_unref(two_level);
+ two_level = NULL;
+
+ /* Step 3: detect if we need to turn our dict into list */
+ is_list = qdict_is_list(multi_level, errp);
+ if (is_list < 0) {
+ goto error;
+ }
+
+ if (is_list) {
+ dst = QOBJECT(qlist_new());
+
+ for (i = 0; i < qdict_size(multi_level); i++) {
+ char *key = g_strdup_printf("%zu", i);
+
+ child = qdict_get(multi_level, key);
+ g_free(key);
+
+ if (!child) {
+ error_setg(errp, "Missing list index %zu", i);
+ goto error;
+ }
+
+ qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
+ }
+ qobject_unref(multi_level);
+ multi_level = NULL;
+ } else {
+ dst = QOBJECT(multi_level);
+ }
+
+ return dst;
+
+ error:
+ g_free(prefix);
+ qobject_unref(multi_level);
+ qobject_unref(two_level);
+ qobject_unref(dst);
+ return NULL;
+}
+
+/**
+ * qdict_array_entries(): Returns the number of direct array entries if the
+ * sub-QDict of src specified by the prefix in subqdict (or src itself for
+ * prefix == "") is valid as an array, i.e. the length of the created list if
+ * the sub-QDict would become empty after calling qdict_array_split() on it. If
+ * the array is not valid, -EINVAL is returned.
+ */
+int qdict_array_entries(QDict *src, const char *subqdict)
+{
+ const QDictEntry *entry;
+ unsigned i;
+ unsigned entries = 0;
+ size_t subqdict_len = strlen(subqdict);
+
+ assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
+
+ /* qdict_array_split() loops until UINT_MAX, but as we want to return
+ * negative errors, we only have a signed return value here. Any additional
+ * entries will lead to -EINVAL. */
+ for (i = 0; i < INT_MAX; i++) {
+ QObject *subqobj;
+ int subqdict_entries;
+ char *prefix = g_strdup_printf("%s%u.", subqdict, i);
+
+ subqdict_entries = qdict_count_prefixed_entries(src, prefix);
+
+ /* Remove ending "." */
+ prefix[strlen(prefix) - 1] = 0;
+ subqobj = qdict_get(src, prefix);
+
+ g_free(prefix);
+
+ if (subqdict_entries < 0) {
+ return subqdict_entries;
+ }
+
+ /* There may be either a single subordinate object (named "%u") or
+ * multiple objects (each with a key prefixed "%u."), but not both. */
+ if (subqobj && subqdict_entries) {
+ return -EINVAL;
+ } else if (!subqobj && !subqdict_entries) {
+ break;
+ }
+
+ entries += subqdict_entries ? subqdict_entries : 1;
+ }
+
+ /* Consider everything handled that isn't part of the given sub-QDict */
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
+ entries++;
+ }
+ }
+
+ /* Anything left in the sub-QDict that wasn't handled? */
+ if (qdict_size(src) != entries) {
+ return -EINVAL;
+ }
+
+ return i;
+}
+
+/**
+ * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
+ * elements from src to dest.
+ *
+ * If an element from src has a key already present in dest, it will not be
+ * moved unless overwrite is true.
+ *
+ * If overwrite is true, the conflicting values in dest will be discarded and
+ * replaced by the corresponding values from src.
+ *
+ * Therefore, with overwrite being true, the src QDict will always be empty when
+ * this function returns. If overwrite is false, the src QDict will be empty
+ * iff there were no conflicts.
+ */
+void qdict_join(QDict *dest, QDict *src, bool overwrite)
+{
+ const QDictEntry *entry, *next;
+
+ entry = qdict_first(src);
+ while (entry) {
+ next = qdict_next(src, entry);
+
+ if (overwrite || !qdict_haskey(dest, entry->key)) {
+ qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_rename_keys(): Rename keys in qdict according to the replacements
+ * specified in the array renames. The array must be terminated by an entry
+ * with from = NULL.
+ *
+ * The renames are performed individually in the order of the array, so entries
+ * may be renamed multiple times and may or may not conflict depending on the
+ * order of the renames array.
+ *
+ * Returns true for success, false in error cases.
+ */
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
+{
+ QObject *qobj;
+
+ while (renames->from) {
+ if (qdict_haskey(qdict, renames->from)) {
+ if (qdict_haskey(qdict, renames->to)) {
+ error_setg(errp, "'%s' and its alias '%s' can't be used at the "
+ "same time", renames->to, renames->from);
+ return false;
+ }
+
+ qobj = qdict_get(qdict, renames->from);
+ qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
+ qdict_del(qdict, renames->from);
+ }
+
+ renames++;
+ }
+ return true;
+}
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 0554c64553..3d8c2f7bbc 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -11,17 +11,11 @@
*/
#include "qemu/osdep.h"
-#include "block/qdict.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnull.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/error.h"
-#include "qemu/queue.h"
-#include "qemu-common.h"
-#include "qemu/cutils.h"
/**
* qdict_new(): Create a new QDict
@@ -464,626 +458,3 @@ void qdict_destroy_obj(QObject *obj)
g_free(qdict);
}
-
-/**
- * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
- * value of 'key' in 'src' is copied there (and the refcount increased
- * accordingly).
- */
-void qdict_copy_default(QDict *dst, QDict *src, const char *key)
-{
- QObject *val;
-
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- val = qdict_get(src, key);
- if (val) {
- qdict_put_obj(dst, key, qobject_ref(val));
- }
-}
-
-/**
- * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
- * new QString initialised by 'val' is put there.
- */
-void qdict_set_default_str(QDict *dst, const char *key, const char *val)
-{
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- qdict_put_str(dst, key, val);
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target,
- const char *prefix);
-
-static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
-{
- QObject *value;
- const QListEntry *entry;
- char *new_key;
- int i;
-
- /* This function is never called with prefix == NULL, i.e., it is always
- * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
- * need to remove list entries during the iteration (the whole list will be
- * deleted eventually anyway from qdict_flatten_qdict()). */
- assert(prefix);
-
- entry = qlist_first(qlist);
-
- for (i = 0; entry; entry = qlist_next(entry), i++) {
- value = qlist_entry_obj(entry);
- new_key = g_strdup_printf("%s.%i", prefix, i);
-
- if (qobject_type(value) == QTYPE_QDICT) {
- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
- } else {
- /* All other types are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- }
-
- g_free(new_key);
- }
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
-{
- QObject *value;
- const QDictEntry *entry, *next;
- char *new_key;
- bool delete;
-
- entry = qdict_first(qdict);
-
- while (entry != NULL) {
-
- next = qdict_next(qdict, entry);
- value = qdict_entry_value(entry);
- new_key = NULL;
- delete = false;
-
- if (prefix) {
- new_key = g_strdup_printf("%s.%s", prefix, entry->key);
- }
-
- if (qobject_type(value) == QTYPE_QDICT) {
- /* Entries of QDicts are processed recursively, the QDict object
- * itself disappears. */
- qdict_flatten_qdict(qobject_to(QDict, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (prefix) {
- /* All other objects are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- delete = true;
- }
-
- g_free(new_key);
-
- if (delete) {
- qdict_del(qdict, entry->key);
-
- /* Restart loop after modifying the iterated QDict */
- entry = qdict_first(qdict);
- continue;
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_flatten(): For each nested QDict with key x, all fields with key y
- * are moved to this QDict and their key is renamed to "x.y". For each nested
- * QList with key x, the field at index y is moved to this QDict with the key
- * "x.y" (i.e., the reverse of what qdict_array_split() does).
- * This operation is applied recursively for nested QDicts and QLists.
- */
-void qdict_flatten(QDict *qdict)
-{
- qdict_flatten_qdict(qdict, qdict, NULL);
-}
-
-/* extract all the src QDict entries starting by start into dst */
-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
-
-{
- const QDictEntry *entry, *next;
- const char *p;
-
- *dst = qdict_new();
- entry = qdict_first(src);
-
- while (entry != NULL) {
- next = qdict_next(src, entry);
- if (strstart(entry->key, start, &p)) {
- qdict_put_obj(*dst, p, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
- entry = next;
- }
-}
-
-static int qdict_count_prefixed_entries(const QDict *src, const char *start)
-{
- const QDictEntry *entry;
- int count = 0;
-
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (strstart(entry->key, start, NULL)) {
- if (count == INT_MAX) {
- return -ERANGE;
- }
- count++;
- }
- }
-
- return count;
-}
-
-/**
- * qdict_array_split(): This function moves array-like elements of a QDict into
- * a new QList. Every entry in the original QDict with a key "%u" or one
- * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
- * incrementally counting up, will be moved to a new QDict at index %u in the
- * output QList with the key prefix removed, if that prefix is "%u.". If the
- * whole key is just "%u", the whole QObject will be moved unchanged without
- * creating a new QDict. The function terminates when there is no entry in the
- * QDict with a prefix directly (incrementally) following the last one; it also
- * returns if there are both entries with "%u" and "%u." for the same index %u.
- * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
- * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
- * => [{"a": 42, "b": 23}, {"x": 0}, 66]
- * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
- */
-void qdict_array_split(QDict *src, QList **dst)
-{
- unsigned i;
-
- *dst = qlist_new();
-
- for (i = 0; i < UINT_MAX; i++) {
- QObject *subqobj;
- bool is_subqdict;
- QDict *subqdict;
- char indexstr[32], prefix[32];
- size_t snprintf_ret;
-
- snprintf_ret = snprintf(indexstr, 32, "%u", i);
- assert(snprintf_ret < 32);
-
- subqobj = qdict_get(src, indexstr);
-
- snprintf_ret = snprintf(prefix, 32, "%u.", i);
- assert(snprintf_ret < 32);
-
- /* Overflow is the same as positive non-zero results */
- is_subqdict = qdict_count_prefixed_entries(src, prefix);
-
- // There may be either a single subordinate object (named "%u") or
- // multiple objects (each with a key prefixed "%u."), but not both.
- if (!subqobj == !is_subqdict) {
- break;
- }
-
- if (is_subqdict) {
- qdict_extract_subqdict(src, &subqdict, prefix);
- assert(qdict_size(subqdict) > 0);
- } else {
- qobject_ref(subqobj);
- qdict_del(src, indexstr);
- }
-
- qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
- }
-}
-
-/**
- * qdict_split_flat_key:
- * @key: the key string to split
- * @prefix: non-NULL pointer to hold extracted prefix
- * @suffix: non-NULL pointer to remaining suffix
- *
- * Given a flattened key such as 'foo.0.bar', split it into two parts
- * at the first '.' separator. Allows double dot ('..') to escape the
- * normal separator.
- *
- * e.g.
- * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
- * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
- *
- * The '..' sequence will be unescaped in the returned 'prefix'
- * string. The 'suffix' string will be left in escaped format, so it
- * can be fed back into the qdict_split_flat_key() key as the input
- * later.
- *
- * The caller is responsible for freeing the string returned in @prefix
- * using g_free().
- */
-static void qdict_split_flat_key(const char *key, char **prefix,
- const char **suffix)
-{
- const char *separator;
- size_t i, j;
-
- /* Find first '.' separator, but if there is a pair '..'
- * that acts as an escape, so skip over '..' */
- separator = NULL;
- do {
- if (separator) {
- separator += 2;
- } else {
- separator = key;
- }
- separator = strchr(separator, '.');
- } while (separator && separator[1] == '.');
-
- if (separator) {
- *prefix = g_strndup(key, separator - key);
- *suffix = separator + 1;
- } else {
- *prefix = g_strdup(key);
- *suffix = NULL;
- }
-
- /* Unescape the '..' sequence into '.' */
- for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
- if ((*prefix)[i] == '.') {
- assert((*prefix)[i + 1] == '.');
- i++;
- }
- (*prefix)[j] = (*prefix)[i];
- }
- (*prefix)[j] = '\0';
-}
-
-/**
- * qdict_is_list:
- * @maybe_list: dict to check if keys represent list elements.
- *
- * Determine whether all keys in @maybe_list are valid list elements.
- * If @maybe_list is non-zero in length and all the keys look like
- * valid list indexes, this will return 1. If @maybe_list is zero
- * length or all keys are non-numeric then it will return 0 to indicate
- * it is a normal qdict. If there is a mix of numeric and non-numeric
- * keys, or the list indexes are non-contiguous, an error is reported.
- *
- * Returns: 1 if a valid list, 0 if a dict, -1 on error
- */
-static int qdict_is_list(QDict *maybe_list, Error **errp)
-{
- const QDictEntry *ent;
- ssize_t len = 0;
- ssize_t max = -1;
- int is_list = -1;
- int64_t val;
-
- for (ent = qdict_first(maybe_list); ent != NULL;
- ent = qdict_next(maybe_list, ent)) {
-
- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
- if (is_list == -1) {
- is_list = 1;
- } else if (!is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- len++;
- if (val > max) {
- max = val;
- }
- } else {
- if (is_list == -1) {
- is_list = 0;
- } else if (is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- }
- }
-
- if (is_list == -1) {
- assert(!qdict_size(maybe_list));
- is_list = 0;
- }
-
- /* NB this isn't a perfect check - e.g. it won't catch
- * a list containing '1', '+1', '01', '3', but that
- * does not matter - we've still proved that the
- * input is a list. It is up the caller to do a
- * stricter check if desired */
- if (len != (max + 1)) {
- error_setg(errp, "List indices are not contiguous, "
- "saw %zd elements but %zd largest index",
- len, max);
- return -1;
- }
-
- return is_list;
-}
-
-/**
- * qdict_crumple:
- * @src: the original flat dictionary (only scalar values) to crumple
- *
- * Takes a flat dictionary whose keys use '.' separator to indicate
- * nesting, and values are scalars, and crumples it into a nested
- * structure.
- *
- * To include a literal '.' in a key name, it must be escaped as '..'
- *
- * For example, an input of:
- *
- * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
- * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
- *
- * will result in an output of:
- *
- * {
- * 'foo': [
- * { 'bar': 'one', 'wizz': '1' },
- * { 'bar': 'two', 'wizz': '2' }
- * ],
- * }
- *
- * The following scenarios in the input dict will result in an
- * error being returned:
- *
- * - Any values in @src are non-scalar types
- * - If keys in @src imply that a particular level is both a
- * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
- * - If keys in @src imply that a particular level is a list,
- * but the indices are non-contiguous. e.g. "foo.0.bar" and
- * "foo.2.bar" without any "foo.1.bar" present.
- * - If keys in @src represent list indexes, but are not in
- * the "%zu" format. e.g. "foo.+0.bar"
- *
- * Returns: either a QDict or QList for the nested data structure, or NULL
- * on error
- */
-QObject *qdict_crumple(const QDict *src, Error **errp)
-{
- const QDictEntry *ent;
- QDict *two_level, *multi_level = NULL;
- QObject *dst = NULL, *child;
- size_t i;
- char *prefix = NULL;
- const char *suffix = NULL;
- int is_list;
-
- two_level = qdict_new();
-
- /* Step 1: split our totally flat dict into a two level dict */
- for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT ||
- qobject_type(ent->value) == QTYPE_QLIST) {
- error_setg(errp, "Value %s is not a scalar",
- ent->key);
- goto error;
- }
-
- qdict_split_flat_key(ent->key, &prefix, &suffix);
-
- child = qdict_get(two_level, prefix);
- if (suffix) {
- QDict *child_dict = qobject_to(QDict, child);
- if (!child_dict) {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a scalar",
- prefix);
- goto error;
- }
-
- child_dict = qdict_new();
- qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
- }
-
- qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
- } else {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a dict",
- prefix);
- goto error;
- }
- qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
- }
-
- g_free(prefix);
- prefix = NULL;
- }
-
- /* Step 2: optionally process the two level dict recursively
- * into a multi-level dict */
- multi_level = qdict_new();
- for (ent = qdict_first(two_level); ent != NULL;
- ent = qdict_next(two_level, ent)) {
- QDict *dict = qobject_to(QDict, ent->value);
- if (dict) {
- child = qdict_crumple(dict, errp);
- if (!child) {
- goto error;
- }
-
- qdict_put_obj(multi_level, ent->key, child);
- } else {
- qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
- }
- }
- qobject_unref(two_level);
- two_level = NULL;
-
- /* Step 3: detect if we need to turn our dict into list */
- is_list = qdict_is_list(multi_level, errp);
- if (is_list < 0) {
- goto error;
- }
-
- if (is_list) {
- dst = QOBJECT(qlist_new());
-
- for (i = 0; i < qdict_size(multi_level); i++) {
- char *key = g_strdup_printf("%zu", i);
-
- child = qdict_get(multi_level, key);
- g_free(key);
-
- if (!child) {
- error_setg(errp, "Missing list index %zu", i);
- goto error;
- }
-
- qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
- }
- qobject_unref(multi_level);
- multi_level = NULL;
- } else {
- dst = QOBJECT(multi_level);
- }
-
- return dst;
-
- error:
- g_free(prefix);
- qobject_unref(multi_level);
- qobject_unref(two_level);
- qobject_unref(dst);
- return NULL;
-}
-
-/**
- * qdict_array_entries(): Returns the number of direct array entries if the
- * sub-QDict of src specified by the prefix in subqdict (or src itself for
- * prefix == "") is valid as an array, i.e. the length of the created list if
- * the sub-QDict would become empty after calling qdict_array_split() on it. If
- * the array is not valid, -EINVAL is returned.
- */
-int qdict_array_entries(QDict *src, const char *subqdict)
-{
- const QDictEntry *entry;
- unsigned i;
- unsigned entries = 0;
- size_t subqdict_len = strlen(subqdict);
-
- assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
-
- /* qdict_array_split() loops until UINT_MAX, but as we want to return
- * negative errors, we only have a signed return value here. Any additional
- * entries will lead to -EINVAL. */
- for (i = 0; i < INT_MAX; i++) {
- QObject *subqobj;
- int subqdict_entries;
- char *prefix = g_strdup_printf("%s%u.", subqdict, i);
-
- subqdict_entries = qdict_count_prefixed_entries(src, prefix);
-
- /* Remove ending "." */
- prefix[strlen(prefix) - 1] = 0;
- subqobj = qdict_get(src, prefix);
-
- g_free(prefix);
-
- if (subqdict_entries < 0) {
- return subqdict_entries;
- }
-
- /* There may be either a single subordinate object (named "%u") or
- * multiple objects (each with a key prefixed "%u."), but not both. */
- if (subqobj && subqdict_entries) {
- return -EINVAL;
- } else if (!subqobj && !subqdict_entries) {
- break;
- }
-
- entries += subqdict_entries ? subqdict_entries : 1;
- }
-
- /* Consider everything handled that isn't part of the given sub-QDict */
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
- entries++;
- }
- }
-
- /* Anything left in the sub-QDict that wasn't handled? */
- if (qdict_size(src) != entries) {
- return -EINVAL;
- }
-
- return i;
-}
-
-/**
- * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
- * elements from src to dest.
- *
- * If an element from src has a key already present in dest, it will not be
- * moved unless overwrite is true.
- *
- * If overwrite is true, the conflicting values in dest will be discarded and
- * replaced by the corresponding values from src.
- *
- * Therefore, with overwrite being true, the src QDict will always be empty when
- * this function returns. If overwrite is false, the src QDict will be empty
- * iff there were no conflicts.
- */
-void qdict_join(QDict *dest, QDict *src, bool overwrite)
-{
- const QDictEntry *entry, *next;
-
- entry = qdict_first(src);
- while (entry) {
- next = qdict_next(src, entry);
-
- if (overwrite || !qdict_haskey(dest, entry->key)) {
- qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_rename_keys(): Rename keys in qdict according to the replacements
- * specified in the array renames. The array must be terminated by an entry
- * with from = NULL.
- *
- * The renames are performed individually in the order of the array, so entries
- * may be renamed multiple times and may or may not conflict depending on the
- * order of the renames array.
- *
- * Returns true for success, false in error cases.
- */
-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
-{
- QObject *qobj;
-
- while (renames->from) {
- if (qdict_haskey(qdict, renames->from)) {
- if (qdict_haskey(qdict, renames->to)) {
- error_setg(errp, "'%s' and its alias '%s' can't be used at the "
- "same time", renames->to, renames->from);
- return false;
- }
-
- qobj = qdict_get(qdict, renames->from);
- qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
- qdict_del(qdict, renames->from);
- }
-
- renames++;
- }
- return true;
-}
--
2.17.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-06-06 13:17 ` Markus Armbruster
@ 2018-06-06 14:19 ` Markus Armbruster
2018-06-06 14:35 ` Markus Armbruster
0 siblings, 1 reply; 42+ messages in thread
From: Markus Armbruster @ 2018-06-06 14:19 UTC (permalink / raw)
Cc: qemu-block, Kevin Wolf, qemu-devel, Michael Roth
Markus Armbruster <armbru@redhat.com> writes:
> MAINTAINERS files include/block/qdict.h under "Block layer core". Let's
> split qobject/qdict.c as well, so it's there, too. Obvious patch
> appended. Feel free to squash it into yours without giving me credit
> for it.
Hmm, we should split tests/check-qdict.c for the same reason. Updated
patch appended.
> PS: I tried to move block-qdict.c to block/, but totally failed at
> persuading Make to link it into all programs that need it. Oh well.
>From bdb7f6581a3066db853bca6c81cc2c42f6cec8e9 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Wed, 6 Jun 2018 15:11:54 +0200
Subject: [PATCH] qobject: Move block-specific qdict code to block-qdict.c
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
MAINTAINERS | 1 +
qobject/Makefile.objs | 1 +
qobject/block-qdict.c | 637 ++++++++++++++++++++++++++++++++++++
qobject/qdict.c | 629 ------------------------------------
tests/Makefile.include | 4 +
tests/check-block-qdict.c | 657 ++++++++++++++++++++++++++++++++++++++
tests/check-qdict.c | 642 -------------------------------------
7 files changed, 1300 insertions(+), 1271 deletions(-)
create mode 100644 qobject/block-qdict.c
create mode 100644 tests/check-block-qdict.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 41cd3736a9..9f9835e052 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1365,6 +1365,7 @@ F: qemu-img*
F: qemu-io*
F: tests/qemu-iotests/
F: util/qemu-progress.c
+F: qobject/block-qdict.c
T: git git://repo.or.cz/qemu/kevin.git block
Block I/O path
diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs
index 002d25873a..7b12c9cacf 100644
--- a/qobject/Makefile.objs
+++ b/qobject/Makefile.objs
@@ -1,2 +1,3 @@
util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o
util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
+util-obj-y += block-qdict.o
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
new file mode 100644
index 0000000000..01c453a8c0
--- /dev/null
+++ b/qobject/block-qdict.c
@@ -0,0 +1,637 @@
+/*
+ * Special QDict functions used by the block layer
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "block/qdict.h"
+#include "qapi/qmp/qlist.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+
+/**
+ * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
+ * value of 'key' in 'src' is copied there (and the refcount increased
+ * accordingly).
+ */
+void qdict_copy_default(QDict *dst, QDict *src, const char *key)
+{
+ QObject *val;
+
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ val = qdict_get(src, key);
+ if (val) {
+ qdict_put_obj(dst, key, qobject_ref(val));
+ }
+}
+
+/**
+ * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
+ * new QString initialised by 'val' is put there.
+ */
+void qdict_set_default_str(QDict *dst, const char *key, const char *val)
+{
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ qdict_put_str(dst, key, val);
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target,
+ const char *prefix);
+
+static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QListEntry *entry;
+ char *new_key;
+ int i;
+
+ /* This function is never called with prefix == NULL, i.e., it is always
+ * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
+ * need to remove list entries during the iteration (the whole list will be
+ * deleted eventually anyway from qdict_flatten_qdict()). */
+ assert(prefix);
+
+ entry = qlist_first(qlist);
+
+ for (i = 0; entry; entry = qlist_next(entry), i++) {
+ value = qlist_entry_obj(entry);
+ new_key = g_strdup_printf("%s.%i", prefix, i);
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
+ } else {
+ /* All other types are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ }
+
+ g_free(new_key);
+ }
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QDictEntry *entry, *next;
+ char *new_key;
+ bool delete;
+
+ entry = qdict_first(qdict);
+
+ while (entry != NULL) {
+
+ next = qdict_next(qdict, entry);
+ value = qdict_entry_value(entry);
+ new_key = NULL;
+ delete = false;
+
+ if (prefix) {
+ new_key = g_strdup_printf("%s.%s", prefix, entry->key);
+ }
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ /* Entries of QDicts are processed recursively, the QDict object
+ * itself disappears. */
+ qdict_flatten_qdict(qobject_to(QDict, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (prefix) {
+ /* All other objects are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ delete = true;
+ }
+
+ g_free(new_key);
+
+ if (delete) {
+ qdict_del(qdict, entry->key);
+
+ /* Restart loop after modifying the iterated QDict */
+ entry = qdict_first(qdict);
+ continue;
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_flatten(): For each nested QDict with key x, all fields with key y
+ * are moved to this QDict and their key is renamed to "x.y". For each nested
+ * QList with key x, the field at index y is moved to this QDict with the key
+ * "x.y" (i.e., the reverse of what qdict_array_split() does).
+ * This operation is applied recursively for nested QDicts and QLists.
+ */
+void qdict_flatten(QDict *qdict)
+{
+ qdict_flatten_qdict(qdict, qdict, NULL);
+}
+
+/* extract all the src QDict entries starting by start into dst */
+void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
+
+{
+ const QDictEntry *entry, *next;
+ const char *p;
+
+ *dst = qdict_new();
+ entry = qdict_first(src);
+
+ while (entry != NULL) {
+ next = qdict_next(src, entry);
+ if (strstart(entry->key, start, &p)) {
+ qdict_put_obj(*dst, p, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+ entry = next;
+ }
+}
+
+static int qdict_count_prefixed_entries(const QDict *src, const char *start)
+{
+ const QDictEntry *entry;
+ int count = 0;
+
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (strstart(entry->key, start, NULL)) {
+ if (count == INT_MAX) {
+ return -ERANGE;
+ }
+ count++;
+ }
+ }
+
+ return count;
+}
+
+/**
+ * qdict_array_split(): This function moves array-like elements of a QDict into
+ * a new QList. Every entry in the original QDict with a key "%u" or one
+ * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
+ * incrementally counting up, will be moved to a new QDict at index %u in the
+ * output QList with the key prefix removed, if that prefix is "%u.". If the
+ * whole key is just "%u", the whole QObject will be moved unchanged without
+ * creating a new QDict. The function terminates when there is no entry in the
+ * QDict with a prefix directly (incrementally) following the last one; it also
+ * returns if there are both entries with "%u" and "%u." for the same index %u.
+ * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
+ * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
+ * => [{"a": 42, "b": 23}, {"x": 0}, 66]
+ * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
+ */
+void qdict_array_split(QDict *src, QList **dst)
+{
+ unsigned i;
+
+ *dst = qlist_new();
+
+ for (i = 0; i < UINT_MAX; i++) {
+ QObject *subqobj;
+ bool is_subqdict;
+ QDict *subqdict;
+ char indexstr[32], prefix[32];
+ size_t snprintf_ret;
+
+ snprintf_ret = snprintf(indexstr, 32, "%u", i);
+ assert(snprintf_ret < 32);
+
+ subqobj = qdict_get(src, indexstr);
+
+ snprintf_ret = snprintf(prefix, 32, "%u.", i);
+ assert(snprintf_ret < 32);
+
+ /* Overflow is the same as positive non-zero results */
+ is_subqdict = qdict_count_prefixed_entries(src, prefix);
+
+ // There may be either a single subordinate object (named "%u") or
+ // multiple objects (each with a key prefixed "%u."), but not both.
+ if (!subqobj == !is_subqdict) {
+ break;
+ }
+
+ if (is_subqdict) {
+ qdict_extract_subqdict(src, &subqdict, prefix);
+ assert(qdict_size(subqdict) > 0);
+ } else {
+ qobject_ref(subqobj);
+ qdict_del(src, indexstr);
+ }
+
+ qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
+ }
+}
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * e.g.
+ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned 'prefix'
+ * string. The 'suffix' string will be left in escaped format, so it
+ * can be fed back into the qdict_split_flat_key() key as the input
+ * later.
+ *
+ * The caller is responsible for freeing the string returned in @prefix
+ * using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix,
+ const char **suffix)
+{
+ const char *separator;
+ size_t i, j;
+
+ /* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+ separator = NULL;
+ do {
+ if (separator) {
+ separator += 2;
+ } else {
+ separator = key;
+ }
+ separator = strchr(separator, '.');
+ } while (separator && separator[1] == '.');
+
+ if (separator) {
+ *prefix = g_strndup(key, separator - key);
+ *suffix = separator + 1;
+ } else {
+ *prefix = g_strdup(key);
+ *suffix = NULL;
+ }
+
+ /* Unescape the '..' sequence into '.' */
+ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+ if ((*prefix)[i] == '.') {
+ assert((*prefix)[i + 1] == '.');
+ i++;
+ }
+ (*prefix)[j] = (*prefix)[i];
+ }
+ (*prefix)[j] = '\0';
+}
+
+/**
+ * qdict_is_list:
+ * @maybe_list: dict to check if keys represent list elements.
+ *
+ * Determine whether all keys in @maybe_list are valid list elements.
+ * If @maybe_list is non-zero in length and all the keys look like
+ * valid list indexes, this will return 1. If @maybe_list is zero
+ * length or all keys are non-numeric then it will return 0 to indicate
+ * it is a normal qdict. If there is a mix of numeric and non-numeric
+ * keys, or the list indexes are non-contiguous, an error is reported.
+ *
+ * Returns: 1 if a valid list, 0 if a dict, -1 on error
+ */
+static int qdict_is_list(QDict *maybe_list, Error **errp)
+{
+ const QDictEntry *ent;
+ ssize_t len = 0;
+ ssize_t max = -1;
+ int is_list = -1;
+ int64_t val;
+
+ for (ent = qdict_first(maybe_list); ent != NULL;
+ ent = qdict_next(maybe_list, ent)) {
+
+ if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
+ if (is_list == -1) {
+ is_list = 1;
+ } else if (!is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ len++;
+ if (val > max) {
+ max = val;
+ }
+ } else {
+ if (is_list == -1) {
+ is_list = 0;
+ } else if (is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ }
+ }
+
+ if (is_list == -1) {
+ assert(!qdict_size(maybe_list));
+ is_list = 0;
+ }
+
+ /* NB this isn't a perfect check - e.g. it won't catch
+ * a list containing '1', '+1', '01', '3', but that
+ * does not matter - we've still proved that the
+ * input is a list. It is up the caller to do a
+ * stricter check if desired */
+ if (len != (max + 1)) {
+ error_setg(errp, "List indices are not contiguous, "
+ "saw %zd elements but %zd largest index",
+ len, max);
+ return -1;
+ }
+
+ return is_list;
+}
+
+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary (only scalar values) to crumple
+ *
+ * Takes a flat dictionary whose keys use '.' separator to indicate
+ * nesting, and values are scalars, and crumples it into a nested
+ * structure.
+ *
+ * To include a literal '.' in a key name, it must be escaped as '..'
+ *
+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in an output of:
+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *
+ * The following scenarios in the input dict will result in an
+ * error being returned:
+ *
+ * - Any values in @src are non-scalar types
+ * - If keys in @src imply that a particular level is both a
+ * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
+ * - If keys in @src imply that a particular level is a list,
+ * but the indices are non-contiguous. e.g. "foo.0.bar" and
+ * "foo.2.bar" without any "foo.1.bar" present.
+ * - If keys in @src represent list indexes, but are not in
+ * the "%zu" format. e.g. "foo.+0.bar"
+ *
+ * Returns: either a QDict or QList for the nested data structure, or NULL
+ * on error
+ */
+QObject *qdict_crumple(const QDict *src, Error **errp)
+{
+ const QDictEntry *ent;
+ QDict *two_level, *multi_level = NULL;
+ QObject *dst = NULL, *child;
+ size_t i;
+ char *prefix = NULL;
+ const char *suffix = NULL;
+ int is_list;
+
+ two_level = qdict_new();
+
+ /* Step 1: split our totally flat dict into a two level dict */
+ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
+ if (qobject_type(ent->value) == QTYPE_QDICT ||
+ qobject_type(ent->value) == QTYPE_QLIST) {
+ error_setg(errp, "Value %s is not a scalar",
+ ent->key);
+ goto error;
+ }
+
+ qdict_split_flat_key(ent->key, &prefix, &suffix);
+
+ child = qdict_get(two_level, prefix);
+ if (suffix) {
+ QDict *child_dict = qobject_to(QDict, child);
+ if (!child_dict) {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a scalar",
+ prefix);
+ goto error;
+ }
+
+ child_dict = qdict_new();
+ qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
+ }
+
+ qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
+ } else {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a dict",
+ prefix);
+ goto error;
+ }
+ qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
+ }
+
+ g_free(prefix);
+ prefix = NULL;
+ }
+
+ /* Step 2: optionally process the two level dict recursively
+ * into a multi-level dict */
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
+ QDict *dict = qobject_to(QDict, ent->value);
+ if (dict) {
+ child = qdict_crumple(dict, errp);
+ if (!child) {
+ goto error;
+ }
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
+ }
+ }
+ qobject_unref(two_level);
+ two_level = NULL;
+
+ /* Step 3: detect if we need to turn our dict into list */
+ is_list = qdict_is_list(multi_level, errp);
+ if (is_list < 0) {
+ goto error;
+ }
+
+ if (is_list) {
+ dst = QOBJECT(qlist_new());
+
+ for (i = 0; i < qdict_size(multi_level); i++) {
+ char *key = g_strdup_printf("%zu", i);
+
+ child = qdict_get(multi_level, key);
+ g_free(key);
+
+ if (!child) {
+ error_setg(errp, "Missing list index %zu", i);
+ goto error;
+ }
+
+ qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
+ }
+ qobject_unref(multi_level);
+ multi_level = NULL;
+ } else {
+ dst = QOBJECT(multi_level);
+ }
+
+ return dst;
+
+ error:
+ g_free(prefix);
+ qobject_unref(multi_level);
+ qobject_unref(two_level);
+ qobject_unref(dst);
+ return NULL;
+}
+
+/**
+ * qdict_array_entries(): Returns the number of direct array entries if the
+ * sub-QDict of src specified by the prefix in subqdict (or src itself for
+ * prefix == "") is valid as an array, i.e. the length of the created list if
+ * the sub-QDict would become empty after calling qdict_array_split() on it. If
+ * the array is not valid, -EINVAL is returned.
+ */
+int qdict_array_entries(QDict *src, const char *subqdict)
+{
+ const QDictEntry *entry;
+ unsigned i;
+ unsigned entries = 0;
+ size_t subqdict_len = strlen(subqdict);
+
+ assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
+
+ /* qdict_array_split() loops until UINT_MAX, but as we want to return
+ * negative errors, we only have a signed return value here. Any additional
+ * entries will lead to -EINVAL. */
+ for (i = 0; i < INT_MAX; i++) {
+ QObject *subqobj;
+ int subqdict_entries;
+ char *prefix = g_strdup_printf("%s%u.", subqdict, i);
+
+ subqdict_entries = qdict_count_prefixed_entries(src, prefix);
+
+ /* Remove ending "." */
+ prefix[strlen(prefix) - 1] = 0;
+ subqobj = qdict_get(src, prefix);
+
+ g_free(prefix);
+
+ if (subqdict_entries < 0) {
+ return subqdict_entries;
+ }
+
+ /* There may be either a single subordinate object (named "%u") or
+ * multiple objects (each with a key prefixed "%u."), but not both. */
+ if (subqobj && subqdict_entries) {
+ return -EINVAL;
+ } else if (!subqobj && !subqdict_entries) {
+ break;
+ }
+
+ entries += subqdict_entries ? subqdict_entries : 1;
+ }
+
+ /* Consider everything handled that isn't part of the given sub-QDict */
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
+ entries++;
+ }
+ }
+
+ /* Anything left in the sub-QDict that wasn't handled? */
+ if (qdict_size(src) != entries) {
+ return -EINVAL;
+ }
+
+ return i;
+}
+
+/**
+ * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
+ * elements from src to dest.
+ *
+ * If an element from src has a key already present in dest, it will not be
+ * moved unless overwrite is true.
+ *
+ * If overwrite is true, the conflicting values in dest will be discarded and
+ * replaced by the corresponding values from src.
+ *
+ * Therefore, with overwrite being true, the src QDict will always be empty when
+ * this function returns. If overwrite is false, the src QDict will be empty
+ * iff there were no conflicts.
+ */
+void qdict_join(QDict *dest, QDict *src, bool overwrite)
+{
+ const QDictEntry *entry, *next;
+
+ entry = qdict_first(src);
+ while (entry) {
+ next = qdict_next(src, entry);
+
+ if (overwrite || !qdict_haskey(dest, entry->key)) {
+ qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_rename_keys(): Rename keys in qdict according to the replacements
+ * specified in the array renames. The array must be terminated by an entry
+ * with from = NULL.
+ *
+ * The renames are performed individually in the order of the array, so entries
+ * may be renamed multiple times and may or may not conflict depending on the
+ * order of the renames array.
+ *
+ * Returns true for success, false in error cases.
+ */
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
+{
+ QObject *qobj;
+
+ while (renames->from) {
+ if (qdict_haskey(qdict, renames->from)) {
+ if (qdict_haskey(qdict, renames->to)) {
+ error_setg(errp, "'%s' and its alias '%s' can't be used at the "
+ "same time", renames->to, renames->from);
+ return false;
+ }
+
+ qobj = qdict_get(qdict, renames->from);
+ qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
+ qdict_del(qdict, renames->from);
+ }
+
+ renames++;
+ }
+ return true;
+}
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 0554c64553..3d8c2f7bbc 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -11,17 +11,11 @@
*/
#include "qemu/osdep.h"
-#include "block/qdict.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnull.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/error.h"
-#include "qemu/queue.h"
-#include "qemu-common.h"
-#include "qemu/cutils.h"
/**
* qdict_new(): Create a new QDict
@@ -464,626 +458,3 @@ void qdict_destroy_obj(QObject *obj)
g_free(qdict);
}
-
-/**
- * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
- * value of 'key' in 'src' is copied there (and the refcount increased
- * accordingly).
- */
-void qdict_copy_default(QDict *dst, QDict *src, const char *key)
-{
- QObject *val;
-
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- val = qdict_get(src, key);
- if (val) {
- qdict_put_obj(dst, key, qobject_ref(val));
- }
-}
-
-/**
- * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
- * new QString initialised by 'val' is put there.
- */
-void qdict_set_default_str(QDict *dst, const char *key, const char *val)
-{
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- qdict_put_str(dst, key, val);
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target,
- const char *prefix);
-
-static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
-{
- QObject *value;
- const QListEntry *entry;
- char *new_key;
- int i;
-
- /* This function is never called with prefix == NULL, i.e., it is always
- * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
- * need to remove list entries during the iteration (the whole list will be
- * deleted eventually anyway from qdict_flatten_qdict()). */
- assert(prefix);
-
- entry = qlist_first(qlist);
-
- for (i = 0; entry; entry = qlist_next(entry), i++) {
- value = qlist_entry_obj(entry);
- new_key = g_strdup_printf("%s.%i", prefix, i);
-
- if (qobject_type(value) == QTYPE_QDICT) {
- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
- } else {
- /* All other types are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- }
-
- g_free(new_key);
- }
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
-{
- QObject *value;
- const QDictEntry *entry, *next;
- char *new_key;
- bool delete;
-
- entry = qdict_first(qdict);
-
- while (entry != NULL) {
-
- next = qdict_next(qdict, entry);
- value = qdict_entry_value(entry);
- new_key = NULL;
- delete = false;
-
- if (prefix) {
- new_key = g_strdup_printf("%s.%s", prefix, entry->key);
- }
-
- if (qobject_type(value) == QTYPE_QDICT) {
- /* Entries of QDicts are processed recursively, the QDict object
- * itself disappears. */
- qdict_flatten_qdict(qobject_to(QDict, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (prefix) {
- /* All other objects are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- delete = true;
- }
-
- g_free(new_key);
-
- if (delete) {
- qdict_del(qdict, entry->key);
-
- /* Restart loop after modifying the iterated QDict */
- entry = qdict_first(qdict);
- continue;
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_flatten(): For each nested QDict with key x, all fields with key y
- * are moved to this QDict and their key is renamed to "x.y". For each nested
- * QList with key x, the field at index y is moved to this QDict with the key
- * "x.y" (i.e., the reverse of what qdict_array_split() does).
- * This operation is applied recursively for nested QDicts and QLists.
- */
-void qdict_flatten(QDict *qdict)
-{
- qdict_flatten_qdict(qdict, qdict, NULL);
-}
-
-/* extract all the src QDict entries starting by start into dst */
-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
-
-{
- const QDictEntry *entry, *next;
- const char *p;
-
- *dst = qdict_new();
- entry = qdict_first(src);
-
- while (entry != NULL) {
- next = qdict_next(src, entry);
- if (strstart(entry->key, start, &p)) {
- qdict_put_obj(*dst, p, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
- entry = next;
- }
-}
-
-static int qdict_count_prefixed_entries(const QDict *src, const char *start)
-{
- const QDictEntry *entry;
- int count = 0;
-
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (strstart(entry->key, start, NULL)) {
- if (count == INT_MAX) {
- return -ERANGE;
- }
- count++;
- }
- }
-
- return count;
-}
-
-/**
- * qdict_array_split(): This function moves array-like elements of a QDict into
- * a new QList. Every entry in the original QDict with a key "%u" or one
- * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
- * incrementally counting up, will be moved to a new QDict at index %u in the
- * output QList with the key prefix removed, if that prefix is "%u.". If the
- * whole key is just "%u", the whole QObject will be moved unchanged without
- * creating a new QDict. The function terminates when there is no entry in the
- * QDict with a prefix directly (incrementally) following the last one; it also
- * returns if there are both entries with "%u" and "%u." for the same index %u.
- * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
- * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
- * => [{"a": 42, "b": 23}, {"x": 0}, 66]
- * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
- */
-void qdict_array_split(QDict *src, QList **dst)
-{
- unsigned i;
-
- *dst = qlist_new();
-
- for (i = 0; i < UINT_MAX; i++) {
- QObject *subqobj;
- bool is_subqdict;
- QDict *subqdict;
- char indexstr[32], prefix[32];
- size_t snprintf_ret;
-
- snprintf_ret = snprintf(indexstr, 32, "%u", i);
- assert(snprintf_ret < 32);
-
- subqobj = qdict_get(src, indexstr);
-
- snprintf_ret = snprintf(prefix, 32, "%u.", i);
- assert(snprintf_ret < 32);
-
- /* Overflow is the same as positive non-zero results */
- is_subqdict = qdict_count_prefixed_entries(src, prefix);
-
- // There may be either a single subordinate object (named "%u") or
- // multiple objects (each with a key prefixed "%u."), but not both.
- if (!subqobj == !is_subqdict) {
- break;
- }
-
- if (is_subqdict) {
- qdict_extract_subqdict(src, &subqdict, prefix);
- assert(qdict_size(subqdict) > 0);
- } else {
- qobject_ref(subqobj);
- qdict_del(src, indexstr);
- }
-
- qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
- }
-}
-
-/**
- * qdict_split_flat_key:
- * @key: the key string to split
- * @prefix: non-NULL pointer to hold extracted prefix
- * @suffix: non-NULL pointer to remaining suffix
- *
- * Given a flattened key such as 'foo.0.bar', split it into two parts
- * at the first '.' separator. Allows double dot ('..') to escape the
- * normal separator.
- *
- * e.g.
- * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
- * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
- *
- * The '..' sequence will be unescaped in the returned 'prefix'
- * string. The 'suffix' string will be left in escaped format, so it
- * can be fed back into the qdict_split_flat_key() key as the input
- * later.
- *
- * The caller is responsible for freeing the string returned in @prefix
- * using g_free().
- */
-static void qdict_split_flat_key(const char *key, char **prefix,
- const char **suffix)
-{
- const char *separator;
- size_t i, j;
-
- /* Find first '.' separator, but if there is a pair '..'
- * that acts as an escape, so skip over '..' */
- separator = NULL;
- do {
- if (separator) {
- separator += 2;
- } else {
- separator = key;
- }
- separator = strchr(separator, '.');
- } while (separator && separator[1] == '.');
-
- if (separator) {
- *prefix = g_strndup(key, separator - key);
- *suffix = separator + 1;
- } else {
- *prefix = g_strdup(key);
- *suffix = NULL;
- }
-
- /* Unescape the '..' sequence into '.' */
- for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
- if ((*prefix)[i] == '.') {
- assert((*prefix)[i + 1] == '.');
- i++;
- }
- (*prefix)[j] = (*prefix)[i];
- }
- (*prefix)[j] = '\0';
-}
-
-/**
- * qdict_is_list:
- * @maybe_list: dict to check if keys represent list elements.
- *
- * Determine whether all keys in @maybe_list are valid list elements.
- * If @maybe_list is non-zero in length and all the keys look like
- * valid list indexes, this will return 1. If @maybe_list is zero
- * length or all keys are non-numeric then it will return 0 to indicate
- * it is a normal qdict. If there is a mix of numeric and non-numeric
- * keys, or the list indexes are non-contiguous, an error is reported.
- *
- * Returns: 1 if a valid list, 0 if a dict, -1 on error
- */
-static int qdict_is_list(QDict *maybe_list, Error **errp)
-{
- const QDictEntry *ent;
- ssize_t len = 0;
- ssize_t max = -1;
- int is_list = -1;
- int64_t val;
-
- for (ent = qdict_first(maybe_list); ent != NULL;
- ent = qdict_next(maybe_list, ent)) {
-
- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
- if (is_list == -1) {
- is_list = 1;
- } else if (!is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- len++;
- if (val > max) {
- max = val;
- }
- } else {
- if (is_list == -1) {
- is_list = 0;
- } else if (is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- }
- }
-
- if (is_list == -1) {
- assert(!qdict_size(maybe_list));
- is_list = 0;
- }
-
- /* NB this isn't a perfect check - e.g. it won't catch
- * a list containing '1', '+1', '01', '3', but that
- * does not matter - we've still proved that the
- * input is a list. It is up the caller to do a
- * stricter check if desired */
- if (len != (max + 1)) {
- error_setg(errp, "List indices are not contiguous, "
- "saw %zd elements but %zd largest index",
- len, max);
- return -1;
- }
-
- return is_list;
-}
-
-/**
- * qdict_crumple:
- * @src: the original flat dictionary (only scalar values) to crumple
- *
- * Takes a flat dictionary whose keys use '.' separator to indicate
- * nesting, and values are scalars, and crumples it into a nested
- * structure.
- *
- * To include a literal '.' in a key name, it must be escaped as '..'
- *
- * For example, an input of:
- *
- * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
- * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
- *
- * will result in an output of:
- *
- * {
- * 'foo': [
- * { 'bar': 'one', 'wizz': '1' },
- * { 'bar': 'two', 'wizz': '2' }
- * ],
- * }
- *
- * The following scenarios in the input dict will result in an
- * error being returned:
- *
- * - Any values in @src are non-scalar types
- * - If keys in @src imply that a particular level is both a
- * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
- * - If keys in @src imply that a particular level is a list,
- * but the indices are non-contiguous. e.g. "foo.0.bar" and
- * "foo.2.bar" without any "foo.1.bar" present.
- * - If keys in @src represent list indexes, but are not in
- * the "%zu" format. e.g. "foo.+0.bar"
- *
- * Returns: either a QDict or QList for the nested data structure, or NULL
- * on error
- */
-QObject *qdict_crumple(const QDict *src, Error **errp)
-{
- const QDictEntry *ent;
- QDict *two_level, *multi_level = NULL;
- QObject *dst = NULL, *child;
- size_t i;
- char *prefix = NULL;
- const char *suffix = NULL;
- int is_list;
-
- two_level = qdict_new();
-
- /* Step 1: split our totally flat dict into a two level dict */
- for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT ||
- qobject_type(ent->value) == QTYPE_QLIST) {
- error_setg(errp, "Value %s is not a scalar",
- ent->key);
- goto error;
- }
-
- qdict_split_flat_key(ent->key, &prefix, &suffix);
-
- child = qdict_get(two_level, prefix);
- if (suffix) {
- QDict *child_dict = qobject_to(QDict, child);
- if (!child_dict) {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a scalar",
- prefix);
- goto error;
- }
-
- child_dict = qdict_new();
- qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
- }
-
- qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
- } else {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a dict",
- prefix);
- goto error;
- }
- qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
- }
-
- g_free(prefix);
- prefix = NULL;
- }
-
- /* Step 2: optionally process the two level dict recursively
- * into a multi-level dict */
- multi_level = qdict_new();
- for (ent = qdict_first(two_level); ent != NULL;
- ent = qdict_next(two_level, ent)) {
- QDict *dict = qobject_to(QDict, ent->value);
- if (dict) {
- child = qdict_crumple(dict, errp);
- if (!child) {
- goto error;
- }
-
- qdict_put_obj(multi_level, ent->key, child);
- } else {
- qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
- }
- }
- qobject_unref(two_level);
- two_level = NULL;
-
- /* Step 3: detect if we need to turn our dict into list */
- is_list = qdict_is_list(multi_level, errp);
- if (is_list < 0) {
- goto error;
- }
-
- if (is_list) {
- dst = QOBJECT(qlist_new());
-
- for (i = 0; i < qdict_size(multi_level); i++) {
- char *key = g_strdup_printf("%zu", i);
-
- child = qdict_get(multi_level, key);
- g_free(key);
-
- if (!child) {
- error_setg(errp, "Missing list index %zu", i);
- goto error;
- }
-
- qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
- }
- qobject_unref(multi_level);
- multi_level = NULL;
- } else {
- dst = QOBJECT(multi_level);
- }
-
- return dst;
-
- error:
- g_free(prefix);
- qobject_unref(multi_level);
- qobject_unref(two_level);
- qobject_unref(dst);
- return NULL;
-}
-
-/**
- * qdict_array_entries(): Returns the number of direct array entries if the
- * sub-QDict of src specified by the prefix in subqdict (or src itself for
- * prefix == "") is valid as an array, i.e. the length of the created list if
- * the sub-QDict would become empty after calling qdict_array_split() on it. If
- * the array is not valid, -EINVAL is returned.
- */
-int qdict_array_entries(QDict *src, const char *subqdict)
-{
- const QDictEntry *entry;
- unsigned i;
- unsigned entries = 0;
- size_t subqdict_len = strlen(subqdict);
-
- assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
-
- /* qdict_array_split() loops until UINT_MAX, but as we want to return
- * negative errors, we only have a signed return value here. Any additional
- * entries will lead to -EINVAL. */
- for (i = 0; i < INT_MAX; i++) {
- QObject *subqobj;
- int subqdict_entries;
- char *prefix = g_strdup_printf("%s%u.", subqdict, i);
-
- subqdict_entries = qdict_count_prefixed_entries(src, prefix);
-
- /* Remove ending "." */
- prefix[strlen(prefix) - 1] = 0;
- subqobj = qdict_get(src, prefix);
-
- g_free(prefix);
-
- if (subqdict_entries < 0) {
- return subqdict_entries;
- }
-
- /* There may be either a single subordinate object (named "%u") or
- * multiple objects (each with a key prefixed "%u."), but not both. */
- if (subqobj && subqdict_entries) {
- return -EINVAL;
- } else if (!subqobj && !subqdict_entries) {
- break;
- }
-
- entries += subqdict_entries ? subqdict_entries : 1;
- }
-
- /* Consider everything handled that isn't part of the given sub-QDict */
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
- entries++;
- }
- }
-
- /* Anything left in the sub-QDict that wasn't handled? */
- if (qdict_size(src) != entries) {
- return -EINVAL;
- }
-
- return i;
-}
-
-/**
- * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
- * elements from src to dest.
- *
- * If an element from src has a key already present in dest, it will not be
- * moved unless overwrite is true.
- *
- * If overwrite is true, the conflicting values in dest will be discarded and
- * replaced by the corresponding values from src.
- *
- * Therefore, with overwrite being true, the src QDict will always be empty when
- * this function returns. If overwrite is false, the src QDict will be empty
- * iff there were no conflicts.
- */
-void qdict_join(QDict *dest, QDict *src, bool overwrite)
-{
- const QDictEntry *entry, *next;
-
- entry = qdict_first(src);
- while (entry) {
- next = qdict_next(src, entry);
-
- if (overwrite || !qdict_haskey(dest, entry->key)) {
- qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_rename_keys(): Rename keys in qdict according to the replacements
- * specified in the array renames. The array must be terminated by an entry
- * with from = NULL.
- *
- * The renames are performed individually in the order of the array, so entries
- * may be renamed multiple times and may or may not conflict depending on the
- * order of the renames array.
- *
- * Returns true for success, false in error cases.
- */
-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
-{
- QObject *qobj;
-
- while (renames->from) {
- if (qdict_haskey(qdict, renames->from)) {
- if (qdict_haskey(qdict, renames->to)) {
- error_setg(errp, "'%s' and its alias '%s' can't be used at the "
- "same time", renames->to, renames->from);
- return false;
- }
-
- qobj = qdict_get(qdict, renames->from);
- qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
- qdict_del(qdict, renames->from);
- }
-
- renames++;
- }
- return true;
-}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 86f90c0cb0..fd0274b5dc 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -40,6 +40,8 @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \
check-unit-y = tests/check-qdict$(EXESUF)
gcov-files-check-qdict-y = qobject/qdict.c
+check-unit-y = tests/check-block-qdict$(EXESUF)
+gcov-files-check-block-qdict-y = qobject/block-qdict.c
check-unit-y += tests/test-char$(EXESUF)
gcov-files-check-qdict-y = chardev/char.c
check-unit-y += tests/check-qnum$(EXESUF)
@@ -582,6 +584,7 @@ GENERATED_FILES += tests/test-qapi-types.h tests/test-qapi-visit.h \
test-obj-y = tests/check-qnum.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qnull.o tests/check-qobject.o \
tests/check-qjson.o tests/check-qlit.o \
+ tests/check-block-qtest.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \
tests/test-clone-visitor.o \
@@ -612,6 +615,7 @@ test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o
tests/check-qnum$(EXESUF): tests/check-qnum.o $(test-util-obj-y)
tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y)
tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y)
+tests/check-block-qdict$(EXESUF): tests/check-block-qdict.o $(test-util-obj-y)
tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y)
tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y)
tests/check-qobject$(EXESUF): tests/check-qobject.o $(test-util-obj-y)
diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c
new file mode 100644
index 0000000000..f9845fd1f0
--- /dev/null
+++ b/tests/check-block-qdict.c
@@ -0,0 +1,657 @@
+/*
+ * Unit-tests for Block layer QDict extras
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "block/qdict.h"
+#include "qapi/qmp/qlist.h"
+#include "qapi/qmp/qnum.h"
+#include "qapi/error.h"
+
+static void qdict_defaults_test(void)
+{
+ QDict *dict, *copy;
+
+ dict = qdict_new();
+ copy = qdict_new();
+
+ qdict_set_default_str(dict, "foo", "abc");
+ qdict_set_default_str(dict, "foo", "def");
+ g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc");
+ qdict_set_default_str(dict, "bar", "ghi");
+
+ qdict_copy_default(copy, dict, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc");
+ qdict_set_default_str(copy, "bar", "xyz");
+ qdict_copy_default(copy, dict, "bar");
+ g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz");
+
+ qobject_unref(copy);
+ qobject_unref(dict);
+}
+
+static void qdict_flatten_test(void)
+{
+ QList *list1 = qlist_new();
+ QList *list2 = qlist_new();
+ QDict *dict1 = qdict_new();
+ QDict *dict2 = qdict_new();
+ QDict *dict3 = qdict_new();
+
+ /*
+ * Test the flattening of
+ *
+ * {
+ * "e": [
+ * 42,
+ * [
+ * 23,
+ * 66,
+ * {
+ * "a": 0,
+ * "b": 1
+ * }
+ * ]
+ * ],
+ * "f": {
+ * "c": 2,
+ * "d": 3,
+ * },
+ * "g": 4
+ * }
+ *
+ * to
+ *
+ * {
+ * "e.0": 42,
+ * "e.1.0": 23,
+ * "e.1.1": 66,
+ * "e.1.2.a": 0,
+ * "e.1.2.b": 1,
+ * "f.c": 2,
+ * "f.d": 3,
+ * "g": 4
+ * }
+ */
+
+ qdict_put_int(dict1, "a", 0);
+ qdict_put_int(dict1, "b", 1);
+
+ qlist_append_int(list1, 23);
+ qlist_append_int(list1, 66);
+ qlist_append(list1, dict1);
+ qlist_append_int(list2, 42);
+ qlist_append(list2, list1);
+
+ qdict_put_int(dict2, "c", 2);
+ qdict_put_int(dict2, "d", 3);
+ qdict_put(dict3, "e", list2);
+ qdict_put(dict3, "f", dict2);
+ qdict_put_int(dict3, "g", 4);
+
+ qdict_flatten(dict3);
+
+ g_assert(qdict_get_int(dict3, "e.0") == 42);
+ g_assert(qdict_get_int(dict3, "e.1.0") == 23);
+ g_assert(qdict_get_int(dict3, "e.1.1") == 66);
+ g_assert(qdict_get_int(dict3, "e.1.2.a") == 0);
+ g_assert(qdict_get_int(dict3, "e.1.2.b") == 1);
+ g_assert(qdict_get_int(dict3, "f.c") == 2);
+ g_assert(qdict_get_int(dict3, "f.d") == 3);
+ g_assert(qdict_get_int(dict3, "g") == 4);
+
+ g_assert(qdict_size(dict3) == 8);
+
+ qobject_unref(dict3);
+}
+
+static void qdict_array_split_test(void)
+{
+ QDict *test_dict = qdict_new();
+ QDict *dict1, *dict2;
+ QNum *int1;
+ QList *test_list;
+
+ /*
+ * Test the split of
+ *
+ * {
+ * "1.x": 0,
+ * "4.y": 1,
+ * "0.a": 42,
+ * "o.o": 7,
+ * "0.b": 23,
+ * "2": 66
+ * }
+ *
+ * to
+ *
+ * [
+ * {
+ * "a": 42,
+ * "b": 23
+ * },
+ * {
+ * "x": 0
+ * },
+ * 66
+ * ]
+ *
+ * and
+ *
+ * {
+ * "4.y": 1,
+ * "o.o": 7
+ * }
+ *
+ * (remaining in the old QDict)
+ *
+ * This example is given in the comment of qdict_array_split().
+ */
+
+ qdict_put_int(test_dict, "1.x", 0);
+ qdict_put_int(test_dict, "4.y", 1);
+ qdict_put_int(test_dict, "0.a", 42);
+ qdict_put_int(test_dict, "o.o", 7);
+ qdict_put_int(test_dict, "0.b", 23);
+ qdict_put_int(test_dict, "2", 66);
+
+ qdict_array_split(test_dict, &test_list);
+
+ dict1 = qobject_to(QDict, qlist_pop(test_list));
+ dict2 = qobject_to(QDict, qlist_pop(test_list));
+ int1 = qobject_to(QNum, qlist_pop(test_list));
+
+ g_assert(dict1);
+ g_assert(dict2);
+ g_assert(int1);
+ g_assert(qlist_empty(test_list));
+
+ qobject_unref(test_list);
+
+ g_assert(qdict_get_int(dict1, "a") == 42);
+ g_assert(qdict_get_int(dict1, "b") == 23);
+
+ g_assert(qdict_size(dict1) == 2);
+
+ qobject_unref(dict1);
+
+ g_assert(qdict_get_int(dict2, "x") == 0);
+
+ g_assert(qdict_size(dict2) == 1);
+
+ qobject_unref(dict2);
+
+ g_assert_cmpint(qnum_get_int(int1), ==, 66);
+
+ qobject_unref(int1);
+
+ g_assert(qdict_get_int(test_dict, "4.y") == 1);
+ g_assert(qdict_get_int(test_dict, "o.o") == 7);
+
+ g_assert(qdict_size(test_dict) == 2);
+
+ qobject_unref(test_dict);
+
+ /*
+ * Test the split of
+ *
+ * {
+ * "0": 42,
+ * "1": 23,
+ * "1.x": 84
+ * }
+ *
+ * to
+ *
+ * [
+ * 42
+ * ]
+ *
+ * and
+ *
+ * {
+ * "1": 23,
+ * "1.x": 84
+ * }
+ *
+ * That is, test whether splitting stops if there is both an entry with key
+ * of "%u" and other entries with keys prefixed "%u." for the same index.
+ */
+
+ test_dict = qdict_new();
+
+ qdict_put_int(test_dict, "0", 42);
+ qdict_put_int(test_dict, "1", 23);
+ qdict_put_int(test_dict, "1.x", 84);
+
+ qdict_array_split(test_dict, &test_list);
+
+ int1 = qobject_to(QNum, qlist_pop(test_list));
+
+ g_assert(int1);
+ g_assert(qlist_empty(test_list));
+
+ qobject_unref(test_list);
+
+ g_assert_cmpint(qnum_get_int(int1), ==, 42);
+
+ qobject_unref(int1);
+
+ g_assert(qdict_get_int(test_dict, "1") == 23);
+ g_assert(qdict_get_int(test_dict, "1.x") == 84);
+
+ g_assert(qdict_size(test_dict) == 2);
+
+ qobject_unref(test_dict);
+}
+
+static void qdict_array_entries_test(void)
+{
+ QDict *dict = qdict_new();
+
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
+
+ qdict_put_int(dict, "bar", 0);
+ qdict_put_int(dict, "baz.0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
+
+ qdict_put_int(dict, "foo.1", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
+ qdict_put_int(dict, "foo.0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2);
+ qdict_put_int(dict, "foo.bar", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
+ qdict_del(dict, "foo.bar");
+
+ qdict_put_int(dict, "foo.2.a", 0);
+ qdict_put_int(dict, "foo.2.b", 0);
+ qdict_put_int(dict, "foo.2.c", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+
+ qobject_unref(dict);
+
+ dict = qdict_new();
+ qdict_put_int(dict, "1", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+ qdict_put_int(dict, "0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2);
+ qdict_put_int(dict, "bar", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+ qdict_del(dict, "bar");
+
+ qdict_put_int(dict, "2.a", 0);
+ qdict_put_int(dict, "2.b", 0);
+ qdict_put_int(dict, "2.c", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3);
+
+ qobject_unref(dict);
+}
+
+static void qdict_join_test(void)
+{
+ QDict *dict1, *dict2;
+ bool overwrite = false;
+ int i;
+
+ dict1 = qdict_new();
+ dict2 = qdict_new();
+
+ /* Test everything once without overwrite and once with */
+ do
+ {
+ /* Test empty dicts */
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 0);
+ g_assert(qdict_size(dict2) == 0);
+
+ /* First iteration: Test movement */
+ /* Second iteration: Test empty source and non-empty destination */
+ qdict_put_int(dict2, "foo", 42);
+
+ for (i = 0; i < 2; i++) {
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 1);
+ g_assert(qdict_size(dict2) == 0);
+
+ g_assert(qdict_get_int(dict1, "foo") == 42);
+ }
+
+ /* Test non-empty source and destination without conflict */
+ qdict_put_int(dict2, "bar", 23);
+
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 2);
+ g_assert(qdict_size(dict2) == 0);
+
+ g_assert(qdict_get_int(dict1, "foo") == 42);
+ g_assert(qdict_get_int(dict1, "bar") == 23);
+
+ /* Test conflict */
+ qdict_put_int(dict2, "foo", 84);
+
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 2);
+ g_assert(qdict_size(dict2) == !overwrite);
+
+ g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42));
+ g_assert(qdict_get_int(dict1, "bar") == 23);
+
+ if (!overwrite) {
+ g_assert(qdict_get_int(dict2, "foo") == 84);
+ }
+
+ /* Check the references */
+ g_assert(qdict_get(dict1, "foo")->base.refcnt == 1);
+ g_assert(qdict_get(dict1, "bar")->base.refcnt == 1);
+
+ if (!overwrite) {
+ g_assert(qdict_get(dict2, "foo")->base.refcnt == 1);
+ }
+
+ /* Clean up */
+ qdict_del(dict1, "foo");
+ qdict_del(dict1, "bar");
+
+ if (!overwrite) {
+ qdict_del(dict2, "foo");
+ }
+ }
+ while (overwrite ^= true);
+
+ qobject_unref(dict1);
+ qobject_unref(dict2);
+}
+
+static void qdict_crumple_test_recursive(void)
+{
+ QDict *src, *dst, *rule, *vnc, *acl, *listen;
+ QList *rules;
+
+ src = qdict_new();
+ qdict_put_str(src, "vnc.listen.addr", "127.0.0.1");
+ qdict_put_str(src, "vnc.listen.port", "5901");
+ qdict_put_str(src, "vnc.acl.rules.0.match", "fred");
+ qdict_put_str(src, "vnc.acl.rules.0.policy", "allow");
+ qdict_put_str(src, "vnc.acl.rules.1.match", "bob");
+ qdict_put_str(src, "vnc.acl.rules.1.policy", "deny");
+ qdict_put_str(src, "vnc.acl.default", "deny");
+ qdict_put_str(src, "vnc.acl..name", "acl0");
+ qdict_put_str(src, "vnc.acl.rule..name", "acl0");
+
+ dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
+ g_assert(dst);
+ g_assert_cmpint(qdict_size(dst), ==, 1);
+
+ vnc = qdict_get_qdict(dst, "vnc");
+ g_assert(vnc);
+ g_assert_cmpint(qdict_size(vnc), ==, 3);
+
+ listen = qdict_get_qdict(vnc, "listen");
+ g_assert(listen);
+ g_assert_cmpint(qdict_size(listen), ==, 2);
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
+
+ acl = qdict_get_qdict(vnc, "acl");
+ g_assert(acl);
+ g_assert_cmpint(qdict_size(acl), ==, 3);
+
+ rules = qdict_get_qlist(acl, "rules");
+ g_assert(rules);
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ rule = qobject_to(QDict, qlist_pop(rules));
+ g_assert(rule);
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ qobject_unref(rule);
+
+ rule = qobject_to(QDict, qlist_pop(rules));
+ g_assert(rule);
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ qobject_unref(rule);
+
+ /* With recursive crumpling, we should see all names unescaped */
+ g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
+ g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
+
+ qobject_unref(src);
+ qobject_unref(dst);
+}
+
+static void qdict_crumple_test_empty(void)
+{
+ QDict *src, *dst;
+
+ src = qdict_new();
+
+ dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
+
+ g_assert_cmpint(qdict_size(dst), ==, 0);
+
+ qobject_unref(src);
+ qobject_unref(dst);
+}
+
+static int qdict_count_entries(QDict *dict)
+{
+ const QDictEntry *e;
+ int count = 0;
+
+ for (e = qdict_first(dict); e; e = qdict_next(dict, e)) {
+ count++;
+ }
+
+ return count;
+}
+
+static void qdict_rename_keys_test(void)
+{
+ QDict *dict = qdict_new();
+ QDict *copy;
+ QDictRenames *renames;
+ Error *local_err = NULL;
+
+ qdict_put_str(dict, "abc", "foo");
+ qdict_put_str(dict, "abcdef", "bar");
+ qdict_put_int(dict, "number", 42);
+ qdict_put_bool(dict, "flag", true);
+ qdict_put_null(dict, "nothing");
+
+ /* Empty rename list */
+ renames = (QDictRenames[]) {
+ { NULL, "this can be anything" }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Simple rename of all entries */
+ renames = (QDictRenames[]) {
+ { "abc", "str1" },
+ { "abcdef", "str2" },
+ { "number", "int" },
+ { "flag", "bool" },
+ { "nothing", "null" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert(!qdict_haskey(copy, "abc"));
+ g_assert(!qdict_haskey(copy, "abcdef"));
+ g_assert(!qdict_haskey(copy, "number"));
+ g_assert(!qdict_haskey(copy, "flag"));
+ g_assert(!qdict_haskey(copy, "nothing"));
+
+ g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Renames are processed top to bottom */
+ renames = (QDictRenames[]) {
+ { "abc", "tmp" },
+ { "abcdef", "abc" },
+ { "number", "abcdef" },
+ { "flag", "number" },
+ { "nothing", "flag" },
+ { "tmp", "nothing" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL);
+ g_assert(!qdict_haskey(copy, "tmp"));
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Conflicting rename */
+ renames = (QDictRenames[]) {
+ { "abcdef", "abc" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &local_err);
+
+ g_assert(local_err != NULL);
+ error_free(local_err);
+ local_err = NULL;
+
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Renames in an empty dict */
+ renames = (QDictRenames[]) {
+ { "abcdef", "abc" },
+ { NULL , NULL }
+ };
+
+ qobject_unref(dict);
+ dict = qdict_new();
+
+ qdict_rename_keys(dict, renames, &error_abort);
+ g_assert(qdict_first(dict) == NULL);
+
+ qobject_unref(dict);
+}
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+ QDict *src;
+ Error *error = NULL;
+
+ src = qdict_new();
+ /* rule.0 can't be both a string and a dict */
+ qdict_put_str(src, "rule.0", "fred");
+ qdict_put_str(src, "rule.0.policy", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* rule can't be both a list and a dict */
+ qdict_put_str(src, "rule.0", "fred");
+ qdict_put_str(src, "rule.a", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* The input should be flat, ie no dicts or lists */
+ qdict_put(src, "rule.a", qdict_new());
+ qdict_put_str(src, "rule.b", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* List indexes must not have gaps */
+ qdict_put_str(src, "rule.0", "deny");
+ qdict_put_str(src, "rule.3", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* List indexes must be in %zu format */
+ qdict_put_str(src, "rule.0", "deny");
+ qdict_put_str(src, "rule.+1", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ g_test_add_func("/public/defaults", qdict_defaults_test);
+ g_test_add_func("/public/flatten", qdict_flatten_test);
+ g_test_add_func("/public/array_split", qdict_array_split_test);
+ g_test_add_func("/public/array_entries", qdict_array_entries_test);
+ g_test_add_func("/public/join", qdict_join_test);
+ g_test_add_func("/public/crumple/recursive",
+ qdict_crumple_test_recursive);
+ g_test_add_func("/public/crumple/empty",
+ qdict_crumple_test_empty);
+ g_test_add_func("/public/crumple/bad_inputs",
+ qdict_crumple_test_bad_inputs);
+
+ g_test_add_func("/public/rename_keys", qdict_rename_keys_test);
+
+ return g_test_run();
+}
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 93e2112b6d..86e9fe7dc4 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -11,13 +11,7 @@
*/
#include "qemu/osdep.h"
-#include "block/qdict.h"
#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qnum.h"
-#include "qapi/qmp/qstring.h"
-#include "qapi/error.h"
-#include "qemu-common.h"
/*
* Public Interface test-cases
@@ -157,28 +151,6 @@ static void qdict_get_try_str_test(void)
qobject_unref(tests_dict);
}
-static void qdict_defaults_test(void)
-{
- QDict *dict, *copy;
-
- dict = qdict_new();
- copy = qdict_new();
-
- qdict_set_default_str(dict, "foo", "abc");
- qdict_set_default_str(dict, "foo", "def");
- g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc");
- qdict_set_default_str(dict, "bar", "ghi");
-
- qdict_copy_default(copy, dict, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc");
- qdict_set_default_str(copy, "bar", "xyz");
- qdict_copy_default(copy, dict, "bar");
- g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz");
-
- qobject_unref(copy);
- qobject_unref(dict);
-}
-
static void qdict_haskey_not_test(void)
{
QDict *tests_dict = qdict_new();
@@ -254,606 +226,6 @@ static void qdict_iterapi_test(void)
qobject_unref(tests_dict);
}
-static void qdict_flatten_test(void)
-{
- QList *list1 = qlist_new();
- QList *list2 = qlist_new();
- QDict *dict1 = qdict_new();
- QDict *dict2 = qdict_new();
- QDict *dict3 = qdict_new();
-
- /*
- * Test the flattening of
- *
- * {
- * "e": [
- * 42,
- * [
- * 23,
- * 66,
- * {
- * "a": 0,
- * "b": 1
- * }
- * ]
- * ],
- * "f": {
- * "c": 2,
- * "d": 3,
- * },
- * "g": 4
- * }
- *
- * to
- *
- * {
- * "e.0": 42,
- * "e.1.0": 23,
- * "e.1.1": 66,
- * "e.1.2.a": 0,
- * "e.1.2.b": 1,
- * "f.c": 2,
- * "f.d": 3,
- * "g": 4
- * }
- */
-
- qdict_put_int(dict1, "a", 0);
- qdict_put_int(dict1, "b", 1);
-
- qlist_append_int(list1, 23);
- qlist_append_int(list1, 66);
- qlist_append(list1, dict1);
- qlist_append_int(list2, 42);
- qlist_append(list2, list1);
-
- qdict_put_int(dict2, "c", 2);
- qdict_put_int(dict2, "d", 3);
- qdict_put(dict3, "e", list2);
- qdict_put(dict3, "f", dict2);
- qdict_put_int(dict3, "g", 4);
-
- qdict_flatten(dict3);
-
- g_assert(qdict_get_int(dict3, "e.0") == 42);
- g_assert(qdict_get_int(dict3, "e.1.0") == 23);
- g_assert(qdict_get_int(dict3, "e.1.1") == 66);
- g_assert(qdict_get_int(dict3, "e.1.2.a") == 0);
- g_assert(qdict_get_int(dict3, "e.1.2.b") == 1);
- g_assert(qdict_get_int(dict3, "f.c") == 2);
- g_assert(qdict_get_int(dict3, "f.d") == 3);
- g_assert(qdict_get_int(dict3, "g") == 4);
-
- g_assert(qdict_size(dict3) == 8);
-
- qobject_unref(dict3);
-}
-
-static void qdict_array_split_test(void)
-{
- QDict *test_dict = qdict_new();
- QDict *dict1, *dict2;
- QNum *int1;
- QList *test_list;
-
- /*
- * Test the split of
- *
- * {
- * "1.x": 0,
- * "4.y": 1,
- * "0.a": 42,
- * "o.o": 7,
- * "0.b": 23,
- * "2": 66
- * }
- *
- * to
- *
- * [
- * {
- * "a": 42,
- * "b": 23
- * },
- * {
- * "x": 0
- * },
- * 66
- * ]
- *
- * and
- *
- * {
- * "4.y": 1,
- * "o.o": 7
- * }
- *
- * (remaining in the old QDict)
- *
- * This example is given in the comment of qdict_array_split().
- */
-
- qdict_put_int(test_dict, "1.x", 0);
- qdict_put_int(test_dict, "4.y", 1);
- qdict_put_int(test_dict, "0.a", 42);
- qdict_put_int(test_dict, "o.o", 7);
- qdict_put_int(test_dict, "0.b", 23);
- qdict_put_int(test_dict, "2", 66);
-
- qdict_array_split(test_dict, &test_list);
-
- dict1 = qobject_to(QDict, qlist_pop(test_list));
- dict2 = qobject_to(QDict, qlist_pop(test_list));
- int1 = qobject_to(QNum, qlist_pop(test_list));
-
- g_assert(dict1);
- g_assert(dict2);
- g_assert(int1);
- g_assert(qlist_empty(test_list));
-
- qobject_unref(test_list);
-
- g_assert(qdict_get_int(dict1, "a") == 42);
- g_assert(qdict_get_int(dict1, "b") == 23);
-
- g_assert(qdict_size(dict1) == 2);
-
- qobject_unref(dict1);
-
- g_assert(qdict_get_int(dict2, "x") == 0);
-
- g_assert(qdict_size(dict2) == 1);
-
- qobject_unref(dict2);
-
- g_assert_cmpint(qnum_get_int(int1), ==, 66);
-
- qobject_unref(int1);
-
- g_assert(qdict_get_int(test_dict, "4.y") == 1);
- g_assert(qdict_get_int(test_dict, "o.o") == 7);
-
- g_assert(qdict_size(test_dict) == 2);
-
- qobject_unref(test_dict);
-
- /*
- * Test the split of
- *
- * {
- * "0": 42,
- * "1": 23,
- * "1.x": 84
- * }
- *
- * to
- *
- * [
- * 42
- * ]
- *
- * and
- *
- * {
- * "1": 23,
- * "1.x": 84
- * }
- *
- * That is, test whether splitting stops if there is both an entry with key
- * of "%u" and other entries with keys prefixed "%u." for the same index.
- */
-
- test_dict = qdict_new();
-
- qdict_put_int(test_dict, "0", 42);
- qdict_put_int(test_dict, "1", 23);
- qdict_put_int(test_dict, "1.x", 84);
-
- qdict_array_split(test_dict, &test_list);
-
- int1 = qobject_to(QNum, qlist_pop(test_list));
-
- g_assert(int1);
- g_assert(qlist_empty(test_list));
-
- qobject_unref(test_list);
-
- g_assert_cmpint(qnum_get_int(int1), ==, 42);
-
- qobject_unref(int1);
-
- g_assert(qdict_get_int(test_dict, "1") == 23);
- g_assert(qdict_get_int(test_dict, "1.x") == 84);
-
- g_assert(qdict_size(test_dict) == 2);
-
- qobject_unref(test_dict);
-}
-
-static void qdict_array_entries_test(void)
-{
- QDict *dict = qdict_new();
-
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
-
- qdict_put_int(dict, "bar", 0);
- qdict_put_int(dict, "baz.0", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
-
- qdict_put_int(dict, "foo.1", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
- qdict_put_int(dict, "foo.0", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2);
- qdict_put_int(dict, "foo.bar", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
- qdict_del(dict, "foo.bar");
-
- qdict_put_int(dict, "foo.2.a", 0);
- qdict_put_int(dict, "foo.2.b", 0);
- qdict_put_int(dict, "foo.2.c", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
-
- qobject_unref(dict);
-
- dict = qdict_new();
- qdict_put_int(dict, "1", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
- qdict_put_int(dict, "0", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2);
- qdict_put_int(dict, "bar", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
- qdict_del(dict, "bar");
-
- qdict_put_int(dict, "2.a", 0);
- qdict_put_int(dict, "2.b", 0);
- qdict_put_int(dict, "2.c", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3);
-
- qobject_unref(dict);
-}
-
-static void qdict_join_test(void)
-{
- QDict *dict1, *dict2;
- bool overwrite = false;
- int i;
-
- dict1 = qdict_new();
- dict2 = qdict_new();
-
- /* Test everything once without overwrite and once with */
- do
- {
- /* Test empty dicts */
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 0);
- g_assert(qdict_size(dict2) == 0);
-
- /* First iteration: Test movement */
- /* Second iteration: Test empty source and non-empty destination */
- qdict_put_int(dict2, "foo", 42);
-
- for (i = 0; i < 2; i++) {
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 1);
- g_assert(qdict_size(dict2) == 0);
-
- g_assert(qdict_get_int(dict1, "foo") == 42);
- }
-
- /* Test non-empty source and destination without conflict */
- qdict_put_int(dict2, "bar", 23);
-
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 2);
- g_assert(qdict_size(dict2) == 0);
-
- g_assert(qdict_get_int(dict1, "foo") == 42);
- g_assert(qdict_get_int(dict1, "bar") == 23);
-
- /* Test conflict */
- qdict_put_int(dict2, "foo", 84);
-
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 2);
- g_assert(qdict_size(dict2) == !overwrite);
-
- g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42));
- g_assert(qdict_get_int(dict1, "bar") == 23);
-
- if (!overwrite) {
- g_assert(qdict_get_int(dict2, "foo") == 84);
- }
-
- /* Check the references */
- g_assert(qdict_get(dict1, "foo")->base.refcnt == 1);
- g_assert(qdict_get(dict1, "bar")->base.refcnt == 1);
-
- if (!overwrite) {
- g_assert(qdict_get(dict2, "foo")->base.refcnt == 1);
- }
-
- /* Clean up */
- qdict_del(dict1, "foo");
- qdict_del(dict1, "bar");
-
- if (!overwrite) {
- qdict_del(dict2, "foo");
- }
- }
- while (overwrite ^= true);
-
- qobject_unref(dict1);
- qobject_unref(dict2);
-}
-
-static void qdict_crumple_test_recursive(void)
-{
- QDict *src, *dst, *rule, *vnc, *acl, *listen;
- QList *rules;
-
- src = qdict_new();
- qdict_put_str(src, "vnc.listen.addr", "127.0.0.1");
- qdict_put_str(src, "vnc.listen.port", "5901");
- qdict_put_str(src, "vnc.acl.rules.0.match", "fred");
- qdict_put_str(src, "vnc.acl.rules.0.policy", "allow");
- qdict_put_str(src, "vnc.acl.rules.1.match", "bob");
- qdict_put_str(src, "vnc.acl.rules.1.policy", "deny");
- qdict_put_str(src, "vnc.acl.default", "deny");
- qdict_put_str(src, "vnc.acl..name", "acl0");
- qdict_put_str(src, "vnc.acl.rule..name", "acl0");
-
- dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
- g_assert(dst);
- g_assert_cmpint(qdict_size(dst), ==, 1);
-
- vnc = qdict_get_qdict(dst, "vnc");
- g_assert(vnc);
- g_assert_cmpint(qdict_size(vnc), ==, 3);
-
- listen = qdict_get_qdict(vnc, "listen");
- g_assert(listen);
- g_assert_cmpint(qdict_size(listen), ==, 2);
- g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
- g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
-
- acl = qdict_get_qdict(vnc, "acl");
- g_assert(acl);
- g_assert_cmpint(qdict_size(acl), ==, 3);
-
- rules = qdict_get_qlist(acl, "rules");
- g_assert(rules);
- g_assert_cmpint(qlist_size(rules), ==, 2);
-
- rule = qobject_to(QDict, qlist_pop(rules));
- g_assert(rule);
- g_assert_cmpint(qdict_size(rule), ==, 2);
- g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
- qobject_unref(rule);
-
- rule = qobject_to(QDict, qlist_pop(rules));
- g_assert(rule);
- g_assert_cmpint(qdict_size(rule), ==, 2);
- g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
- qobject_unref(rule);
-
- /* With recursive crumpling, we should see all names unescaped */
- g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
- g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
-
- qobject_unref(src);
- qobject_unref(dst);
-}
-
-static void qdict_crumple_test_empty(void)
-{
- QDict *src, *dst;
-
- src = qdict_new();
-
- dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
-
- g_assert_cmpint(qdict_size(dst), ==, 0);
-
- qobject_unref(src);
- qobject_unref(dst);
-}
-
-static int qdict_count_entries(QDict *dict)
-{
- const QDictEntry *e;
- int count = 0;
-
- for (e = qdict_first(dict); e; e = qdict_next(dict, e)) {
- count++;
- }
-
- return count;
-}
-
-static void qdict_rename_keys_test(void)
-{
- QDict *dict = qdict_new();
- QDict *copy;
- QDictRenames *renames;
- Error *local_err = NULL;
-
- qdict_put_str(dict, "abc", "foo");
- qdict_put_str(dict, "abcdef", "bar");
- qdict_put_int(dict, "number", 42);
- qdict_put_bool(dict, "flag", true);
- qdict_put_null(dict, "nothing");
-
- /* Empty rename list */
- renames = (QDictRenames[]) {
- { NULL, "this can be anything" }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Simple rename of all entries */
- renames = (QDictRenames[]) {
- { "abc", "str1" },
- { "abcdef", "str2" },
- { "number", "int" },
- { "flag", "bool" },
- { "nothing", "null" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert(!qdict_haskey(copy, "abc"));
- g_assert(!qdict_haskey(copy, "abcdef"));
- g_assert(!qdict_haskey(copy, "number"));
- g_assert(!qdict_haskey(copy, "flag"));
- g_assert(!qdict_haskey(copy, "nothing"));
-
- g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Renames are processed top to bottom */
- renames = (QDictRenames[]) {
- { "abc", "tmp" },
- { "abcdef", "abc" },
- { "number", "abcdef" },
- { "flag", "number" },
- { "nothing", "flag" },
- { "tmp", "nothing" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL);
- g_assert(!qdict_haskey(copy, "tmp"));
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Conflicting rename */
- renames = (QDictRenames[]) {
- { "abcdef", "abc" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &local_err);
-
- g_assert(local_err != NULL);
- error_free(local_err);
- local_err = NULL;
-
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Renames in an empty dict */
- renames = (QDictRenames[]) {
- { "abcdef", "abc" },
- { NULL , NULL }
- };
-
- qobject_unref(dict);
- dict = qdict_new();
-
- qdict_rename_keys(dict, renames, &error_abort);
- g_assert(qdict_first(dict) == NULL);
-
- qobject_unref(dict);
-}
-
-static void qdict_crumple_test_bad_inputs(void)
-{
- QDict *src;
- Error *error = NULL;
-
- src = qdict_new();
- /* rule.0 can't be both a string and a dict */
- qdict_put_str(src, "rule.0", "fred");
- qdict_put_str(src, "rule.0.policy", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* rule can't be both a list and a dict */
- qdict_put_str(src, "rule.0", "fred");
- qdict_put_str(src, "rule.a", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* The input should be flat, ie no dicts or lists */
- qdict_put(src, "rule.a", qdict_new());
- qdict_put_str(src, "rule.b", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* List indexes must not have gaps */
- qdict_put_str(src, "rule.0", "deny");
- qdict_put_str(src, "rule.3", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* List indexes must be in %zu format */
- qdict_put_str(src, "rule.0", "deny");
- qdict_put_str(src, "rule.+1", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-}
-
/*
* Errors test-cases
*/
@@ -987,29 +359,15 @@ int main(int argc, char **argv)
g_test_add_func("/public/get_try_int", qdict_get_try_int_test);
g_test_add_func("/public/get_str", qdict_get_str_test);
g_test_add_func("/public/get_try_str", qdict_get_try_str_test);
- g_test_add_func("/public/defaults", qdict_defaults_test);
g_test_add_func("/public/haskey_not", qdict_haskey_not_test);
g_test_add_func("/public/haskey", qdict_haskey_test);
g_test_add_func("/public/del", qdict_del_test);
g_test_add_func("/public/to_qdict", qobject_to_qdict_test);
g_test_add_func("/public/iterapi", qdict_iterapi_test);
- g_test_add_func("/public/flatten", qdict_flatten_test);
- g_test_add_func("/public/array_split", qdict_array_split_test);
- g_test_add_func("/public/array_entries", qdict_array_entries_test);
- g_test_add_func("/public/join", qdict_join_test);
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
- g_test_add_func("/public/crumple/recursive",
- qdict_crumple_test_recursive);
- g_test_add_func("/public/crumple/empty",
- qdict_crumple_test_empty);
- g_test_add_func("/public/crumple/bad_inputs",
- qdict_crumple_test_bad_inputs);
-
- g_test_add_func("/public/rename_keys", qdict_rename_keys_test);
-
/* The Big one */
if (g_test_slow()) {
g_test_add_func("/stress/test", qdict_stress_test);
--
2.17.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header
2018-06-06 14:19 ` Markus Armbruster
@ 2018-06-06 14:35 ` Markus Armbruster
0 siblings, 0 replies; 42+ messages in thread
From: Markus Armbruster @ 2018-06-06 14:35 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Michael Roth
Markus Armbruster <armbru@redhat.com> writes:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> MAINTAINERS files include/block/qdict.h under "Block layer core". Let's
>> split qobject/qdict.c as well, so it's there, too. Obvious patch
>> appended. Feel free to squash it into yours without giving me credit
>> for it.
>
> Hmm, we should split tests/check-qdict.c for the same reason. Updated
> patch appended.
>
>> PS: I tried to move block-qdict.c to block/, but totally failed at
>> persuading Make to link it into all programs that need it. Oh well.
>
>
> From bdb7f6581a3066db853bca6c81cc2c42f6cec8e9 Mon Sep 17 00:00:00 2001
> From: Markus Armbruster <armbru@redhat.com>
> Date: Wed, 6 Jun 2018 15:11:54 +0200
> Subject: [PATCH] qobject: Move block-specific qdict code to block-qdict.c
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> MAINTAINERS | 1 +
> qobject/Makefile.objs | 1 +
> qobject/block-qdict.c | 637 ++++++++++++++++++++++++++++++++++++
> qobject/qdict.c | 629 ------------------------------------
> tests/Makefile.include | 4 +
> tests/check-block-qdict.c | 657 ++++++++++++++++++++++++++++++++++++++
> tests/check-qdict.c | 642 -------------------------------------
> 7 files changed, 1300 insertions(+), 1271 deletions(-)
> create mode 100644 qobject/block-qdict.c
> create mode 100644 tests/check-block-qdict.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 41cd3736a9..9f9835e052 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1365,6 +1365,7 @@ F: qemu-img*
> F: qemu-io*
> F: tests/qemu-iotests/
> F: util/qemu-progress.c
> +F: qobject/block-qdict.c
> T: git git://repo.or.cz/qemu/kevin.git block
>
> Block I/O path
Forgot to update MAINTAINERS for tests/check-block-qdict.c. Fixup:
diff --git a/MAINTAINERS b/MAINTAINERS
index 9f9835e052..21f4bc1806 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1366,6 +1366,7 @@ F: qemu-io*
F: tests/qemu-iotests/
F: util/qemu-progress.c
F: qobject/block-qdict.c
+F: test/check-block-qdict.c
T: git git://repo.or.cz/qemu/kevin.git block
Block I/O path
[...]
^ permalink raw reply related [flat|nested] 42+ messages in thread
end of thread, other threads:[~2018-06-06 14:44 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-09 16:55 [Qemu-devel] [PATCH 00/13] block: Try to create well typed json:{} filenames Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 01/13] qapi: Add default-variant for flat unions Max Reitz
2018-05-10 13:12 ` Eric Blake
2018-05-10 13:18 ` Eric Blake
2018-05-11 17:59 ` Max Reitz
2018-05-11 18:13 ` Eric Blake
2018-05-11 17:38 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 02/13] docs/qapi: Document optional discriminators Max Reitz
2018-05-10 13:14 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 03/13] tests: Add QAPI optional discriminator tests Max Reitz
2018-05-10 14:08 ` Eric Blake
2018-05-11 18:06 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing Max Reitz
2018-05-10 7:58 ` Daniel P. Berrangé
2018-05-10 14:22 ` Eric Blake
2018-05-11 17:32 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 05/13] qapi: Formalize qcow " Max Reitz
2018-05-10 14:24 ` Eric Blake
2018-05-10 14:32 ` Daniel P. Berrangé
2018-05-11 18:07 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 06/13] block: Add block-specific QDict header Max Reitz
2018-05-10 14:54 ` Eric Blake
2018-05-11 18:11 ` Max Reitz
2018-06-06 13:10 ` Markus Armbruster
2018-06-06 13:17 ` Markus Armbruster
2018-06-06 14:19 ` Markus Armbruster
2018-06-06 14:35 ` Markus Armbruster
2018-05-09 16:55 ` [Qemu-devel] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval() Max Reitz
2018-05-11 18:39 ` Eric Blake
2018-05-11 21:42 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test Max Reitz
2018-05-10 16:02 ` Eric Blake
2018-05-11 18:13 ` Max Reitz
2018-05-11 18:33 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly Max Reitz
2018-05-11 18:44 ` Eric Blake
2018-05-09 16:55 ` [Qemu-devel] [PATCH 10/13] tests: Add QDict clone-flatten test Max Reitz
2018-05-11 18:46 ` Eric Blake
2018-05-11 21:41 ` Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 11/13] block: Try to create well typed json:{} filenames Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 12/13] iotests: Test internal option typing Max Reitz
2018-05-09 16:55 ` [Qemu-devel] [PATCH 13/13] iotests: qcow2's encrypt.format is now optional Max Reitz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.