* [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties
@ 2016-09-30 14:45 Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static Daniel P. Berrange
` (22 more replies)
0 siblings, 23 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
An update of a series previously posted
v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
v5: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg00485.html
v6: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg03876.html
v7: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00919.html
v8: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03115.html
v9: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02653.html
v10: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02694.html
v11: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg00652.html
v12: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03559.html
v13: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04212.html
v14: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07037.html
This series provides the infrastructure to allow use of non-scalar
properties with the -object CLI arg, and object_add monitor commands.
eg a property which is a list of structs. The syntax used for this is
intentionally compatible with the syntax used by the block layer. This
will allow the qdict_crumple method to be used by the block layer to
convert from QemuOpts into structured QAPI block layer structs at a
future date. It is already used by one of Max's patch series, by
Kevin's -blockdev series, and by my own ACL series.
This patch series has grown a fair bit larger since previous
postings (19^H^H21 patches instead of 6). This was to avoid trying
to squish too much into one patch, to make it easier to review.
It does not neccessarily need to be merged in one go. There are
three logical blocks of patches here.
Patches 1-8 are the core support for visiting QemuOpts using the
QObjectInputVisitor class. Merging this would unblock at least
Kevin's blockdev series, and probably Max's series too.
The patches 9-14 add enhancements to the code to support various
special cases / edge cases needed to provide fully semantic
replacement of the OptsVisitor
Patches 15-21 deal with converting code over to use the new visitor
logic.
Once this is merged, there is also scope to further enhance the
QObjectInputVisitor to deal with visiting single-properties,
which would let us delete StringInputVisitor too. This would
mean we have a single input visitor class, instead of the
current three.
Changed in v15:
- Re-arranged patch series to allow easier incremental merge
of core functionality
- Extra error checking of some scenarios to be avoided (Eric)
- Misc typos (Eric)
- Rename QEMU_OPTS_REPEAT_POLICY_LIST to
QEMU_OPTS_REPEAT_POLICY_ALL, and add QEMU_OPTS_REPEAT_POLICY_ERROR
(Eric)
Changed in v14:
- Extend qemu_opts_to_qdict to optionally handle case of
repeated keys without dropping them.
- Support parsing of integer ranges for "list of int" visits
for back compat with OptsVisitor
- Support visiting nested structs which are represented as
a flat keyspace for back compat with OptsVisitor
- Add trace events for visitor functions to facilitate
debugging
- Add a QObjectInputVisitor constructor which takes a
QemuOpts object, to avoid repeated code patterns in callers
- Greater test coverage of -object to validate correct handling
of "list of ints" legacy OptsVisitor syntax
- Convert all other use of OptsVisitor to QObjectInputVisitor
- Delete OptsVisitor
Changed in v13:
- Fix typos (Kevin)
- Remove unneeded line breaks (Kevin)
Changed in v12:
- More user friendly error message for mixing dict/list
keys (Kevin)
- Report error instead of assert for non-contiguous list
keys (Kevin)
- Fix tests for non-contiguous list keys (Kevin)
- Add tests for escaping of '.' when crumpling (Kevin)
- Fix remaining references to Qmp(In|Out)putVisitor (Markus)
- Misc typos / whitespace fixes (Eric, Kevin)
- Avoid touching 'ret' when parsing int64 fails (Eric)
- Testing of more edge cases in QObjectInputVisitor (Eric)
- Simplify API doc format (Markus)
- Use parse_option_size instead of qemu_strtosz_suffix
for consistency (Kevin)
- Use safer qobject_to_qdict casts (Eric)
Changed in v11:
- Split QAPI/QOM patches off from the access control patches
Changed in v10:
- Fixed stupid build mistake
Changed in v9:
- Rename QmpInputVisitor -> QObjectInputVisitor (Markus/Eric)
- Rename QmpOutputVisitor -> QObjectOutputVisitor (Markus/Eric)
- Drop "strict" param from qobject_string_visitor_new() (Marus)
- Misc docs typos
- Add a visitor able to use strict or string types (for Eric's
netdev series)
- Add a authorization API implementation that uses PAM
Changed in v8:
- Rebase due to merge of Visitor API changes (Eric)
Changed in v7:
- Misc typos in API docs (Marc-André)
- Fix parsing of properties using type_size visitor (Marc-André)
- Mark based auth class as abstract (Marc-André)
- Fix QAPI version annotations to say 2.7 (Marc-André)
Changed in v6:
- Switch from while() to for() loop for iterating over
dicts (Markus)
- Avoid redundant strdup (Markus)
- Rewrap comments at 70 chars (Markus)
- Change qdict_list_size() to qdict_is_list() (Markus)
- Misc docs changes (Markus)
- Change QmpInputVisitor so the code for handling the
string types is separate from code using native
scalar types (Paolo)
- Centralize code parsing bool strings (Markus)
- Centralize code parsing int strings (Markus)
Changed in v5:
- Resolved conflicts with Eric's visitor refactoring which
made it stricter about struct begin/end calls
- Added support for ACLs to migration code now its TLS
support is merged.
- Fixed typos in example in commit message
Changed in v4:
- Ensure examples use shell escaping for '*' (Eric)
- Add more tests for crumple impl (Eric)
- Raise error if sasl-acl/tls-acl are requested but
sasl/tls auth are not enabled (Eric)
- Document return codes for auth check more clearly (Eric)
- Don't silently turn a glob match into a strcmp
- Other misc small typos/fixes (Eric)
Changed in v3:
- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)
Changed in v2:
- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple
Daniel P. Berrange (21):
option: make parse_option_bool/number non-static
qdict: implement a qdict_crumple method for un-flattening a dict
qapi: add trace events for visitor
qapi: rename QmpInputVisitor to QObjectInputVisitor
qapi: rename QmpOutputVisitor to QObjectOutputVisitor
qapi: don't pass two copies of TestInputVisitorData to tests
qapi: permit scalar type conversions in QObjectInputVisitor
qapi: allow QObjectInputVisitor to be created with QemuOpts
qapi: permit auto-creating single element lists
qapi: permit auto-creating nested structs
qapi: add integer range support for QObjectInputVisitor
option: allow qemu_opts_to_qdict to merge repeated options
qdict: allow qdict_crumple to accept compound types as values
qapi: allow repeated opts with qobject_input_visitor_new_opts
qom: support non-scalar properties with -object
hmp: support non-scalar properties with object_add
numa: convert to use QObjectInputVisitor for -numa
block: convert crypto driver to use QObjectInputVisitor
acpi: convert to QObjectInputVisitor for -acpi parsing
net: convert to QObjectInputVisitor for -net/-netdev parsing
qapi: delete unused OptsVisitor code
Makefile.objs | 1 +
block/crypto.c | 14 +-
block/qapi.c | 4 +-
blockdev.c | 11 +-
docs/qapi-code-gen.txt | 4 +-
hmp.c | 25 +-
hw/acpi/core.c | 14 +-
include/qapi/opts-visitor.h | 40 --
include/qapi/qmp-input-visitor.h | 30 -
include/qapi/qmp/qdict.h | 1 +
include/qapi/qobject-input-visitor.h | 130 ++++
...p-output-visitor.h => qobject-output-visitor.h} | 10 +-
include/qapi/visitor.h | 6 +-
include/qemu/option.h | 17 +-
include/sysemu/numa_int.h | 11 +
monitor.c | 5 +-
net/net.c | 18 +-
numa.c | 36 +-
qapi/Makefile.objs | 6 +-
qapi/opts-visitor.c | 561 ---------------
qapi/qapi-clone-visitor.c | 2 +-
qapi/qapi-visit-core.c | 27 +
qapi/qmp-input-visitor.c | 412 -----------
qapi/qmp-output-visitor.c | 256 -------
qapi/qobject-input-visitor.c | 785 +++++++++++++++++++++
qapi/qobject-output-visitor.c | 254 +++++++
qapi/trace-events | 33 +
qemu-img.c | 12 +-
qemu-io-cmds.c | 3 +-
qemu-io.c | 6 +-
qemu-nbd.c | 3 +-
qmp.c | 4 +-
qobject/qdict.c | 302 ++++++++
qom/object_interfaces.c | 42 +-
qom/qom-qobject.c | 8 +-
scripts/qapi-commands.py | 8 +-
scripts/qapi-event.py | 4 +-
stubs/Makefile.objs | 5 +
stubs/exec.c | 6 +
stubs/hostmem.c | 14 +
stubs/memory.c | 41 ++
stubs/qdev.c | 8 +
stubs/vl.c | 8 +
stubs/vmstate.c | 4 +
target-s390x/cpu_models.c | 4 +-
tests/.gitignore | 6 +-
tests/Makefile.include | 27 +-
tests/check-qdict.c | 436 ++++++++++++
tests/check-qnull.c | 8 +-
tests/check-qom-proplist.c | 367 +++++++++-
tests/qemu-iotests/051.out | 6 +-
tests/qemu-iotests/051.pc.out | 6 +-
tests/qemu-iotests/137.out | 4 +-
tests/test-numa.c | 116 +++
tests/test-opts-visitor.c | 268 -------
tests/test-qemu-opts.c | 132 ++++
tests/test-qmp-commands.c | 4 +-
...-input-strict.c => test-qobject-input-strict.c} | 6 +-
...nput-visitor.c => test-qobject-input-visitor.c} | 637 +++++++++++++++--
...put-visitor.c => test-qobject-output-visitor.c} | 6 +-
tests/test-replication.c | 9 +-
tests/test-string-input-visitor.c | 2 +-
tests/test-string-output-visitor.c | 2 +-
tests/test-visitor-serialization.c | 8 +-
util/qemu-option.c | 91 ++-
util/qemu-sockets.c | 4 +-
vl.c | 1 -
67 files changed, 3566 insertions(+), 1775 deletions(-)
delete mode 100644 include/qapi/opts-visitor.h
delete mode 100644 include/qapi/qmp-input-visitor.h
create mode 100644 include/qapi/qobject-input-visitor.h
rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
create mode 100644 include/sysemu/numa_int.h
delete mode 100644 qapi/opts-visitor.c
delete mode 100644 qapi/qmp-input-visitor.c
delete mode 100644 qapi/qmp-output-visitor.c
create mode 100644 qapi/qobject-input-visitor.c
create mode 100644 qapi/qobject-output-visitor.c
create mode 100644 qapi/trace-events
create mode 100644 stubs/exec.c
create mode 100644 stubs/hostmem.c
create mode 100644 stubs/memory.c
create mode 100644 stubs/qdev.c
create mode 100644 stubs/vl.c
create mode 100644 tests/test-numa.c
delete mode 100644 tests/test-opts-visitor.c
rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (57%)
rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)
--
2.7.4
^ permalink raw reply [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-21 16:55 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
` (21 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The opts-visitor.c opts_type_bool() method has code for
parsing a string to set a bool value, as does the
qemu-option.c parse_option_bool() method, except it
handles fewer cases.
To enable consistency across the codebase, extend
parse_option_bool() to handle "yes", "no", "y" and
"n", and make it non-static. Convert the opts
visitor to call this method directly.
Also make parse_option_number() non-static to allow
for similar reuse later.
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qemu/option.h | 4 ++++
qapi/opts-visitor.c | 19 +------------------
tests/qemu-iotests/051.out | 6 +++---
tests/qemu-iotests/051.pc.out | 6 +++---
tests/qemu-iotests/137.out | 4 ++--
util/qemu-option.c | 27 ++++++++++++++++-----------
6 files changed, 29 insertions(+), 37 deletions(-)
diff --git a/include/qemu/option.h b/include/qemu/option.h
index 1f9e3f9..2a5266f 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -37,6 +37,10 @@ int get_param_value(char *buf, int buf_size,
const char *tag, const char *str);
+void parse_option_bool(const char *name, const char *value, bool *ret,
+ Error **errp);
+void parse_option_number(const char *name, const char *value,
+ uint64_t *ret, Error **errp);
void parse_option_size(const char *name, const char *value,
uint64_t *ret, Error **errp);
bool has_help_option(const char *param);
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index 1048bbc..084f7cc 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -334,7 +334,6 @@ opts_type_str(Visitor *v, const char *name, char **obj, Error **errp)
}
-/* mimics qemu-option.c::parse_option_bool() */
static void
opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
{
@@ -346,23 +345,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
return;
}
- if (opt->str) {
- if (strcmp(opt->str, "on") == 0 ||
- strcmp(opt->str, "yes") == 0 ||
- strcmp(opt->str, "y") == 0) {
- *obj = true;
- } else if (strcmp(opt->str, "off") == 0 ||
- strcmp(opt->str, "no") == 0 ||
- strcmp(opt->str, "n") == 0) {
- *obj = false;
- } else {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
- "on|yes|y|off|no|n");
- return;
- }
- } else {
- *obj = true;
- }
+ parse_option_bool(opt->name, opt->str, obj, errp);
processed(ov, name);
}
diff --git a/tests/qemu-iotests/051.out b/tests/qemu-iotests/051.out
index 408d613..ffdc0b9 100644
--- a/tests/qemu-iotests/051.out
+++ b/tests/qemu-iotests/051.out
@@ -86,13 +86,13 @@ QEMU X.Y.Z monitor - type 'help' for more information
(qemu) q^[[K^[[Dqu^[[K^[[D^[[Dqui^[[K^[[D^[[D^[[Dquit^[[K
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
=== With version 2 images enabling lazy refcounts must fail ===
diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index ec6d222..6abf1ab 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -86,13 +86,13 @@ QEMU X.Y.Z monitor - type 'help' for more information
(qemu) q^[[K^[[Dqu^[[K^[[D^[[Dqui^[[K^[[D^[[D^[[Dquit^[[K
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
=== With version 2 images enabling lazy refcounts must fail ===
diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out
index c0e7534..61c1f90 100644
--- a/tests/qemu-iotests/137.out
+++ b/tests/qemu-iotests/137.out
@@ -15,7 +15,7 @@ read 33554432/33554432 bytes at offset 0
=== Try setting some invalid values ===
-Parameter 'lazy-refcounts' expects 'on' or 'off'
+Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
cache-size, l2-cache-size and refcount-cache-size may not be set the same time
l2-cache-size may not exceed cache-size
refcount-cache-size may not exceed cache-size
@@ -40,7 +40,7 @@ incompatible_features 0x0
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
wrote 65536/65536 bytes at offset 0
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Parameter 'lazy-refcounts' expects 'on' or 'off'
+Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
qcow2: Marking image as corrupt: Preventing invalid write on metadata (overlaps with qcow2_header); further corruption events will be suppressed
write failed: Input/output error
*** done
diff --git a/util/qemu-option.c b/util/qemu-option.c
index 3467dc2..41b356c 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,
return get_next_param_value(buf, buf_size, tag, &str);
}
-static void parse_option_bool(const char *name, const char *value, bool *ret,
- Error **errp)
+void parse_option_bool(const char *name, const char *value, bool *ret,
+ Error **errp)
{
if (value != NULL) {
- if (!strcmp(value, "on")) {
- *ret = 1;
- } else if (!strcmp(value, "off")) {
- *ret = 0;
+ if (strcmp(value, "on") == 0 ||
+ strcmp(value, "yes") == 0 ||
+ strcmp(value, "y") == 0) {
+ *ret = true;
+ } else if (strcmp(value, "off") == 0 ||
+ strcmp(value, "no") == 0 ||
+ strcmp(value, "n") == 0) {
+ *ret = false;
} else {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name, "'on' or 'off'");
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+ "'on', 'yes', 'y', 'off', 'no' or 'n'");
+ return;
}
} else {
- *ret = 1;
+ *ret = true;
}
}
-static void parse_option_number(const char *name, const char *value,
- uint64_t *ret, Error **errp)
+void parse_option_number(const char *name, const char *value,
+ uint64_t *ret, Error **errp)
{
char *postfix;
uint64_t number;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-18 14:32 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
` (20 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The qdict_flatten() method will take a dict whose elements are
further nested dicts/lists and flatten them by concatenating
keys.
The qdict_crumple() method aims to do the reverse, taking a flat
qdict, and turning it into a set of nested dicts/lists. It will
apply nesting based on the key name, with a '.' indicating a
new level in the hierarchy. If the keys in the nested structure
are all numeric, it will create a list, otherwise it will create
a dict.
If the keys are a mixture of numeric and non-numeric, or the
numeric keys are not in strictly ascending order, an error will
be reported.
As an example, a flat dict containing
{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}
will get turned into a dict with one element 'foo' whose
value is a list. The list elements will each in turn be
dicts.
{
'foo': [
{ 'bar': 'one', 'wizz': '1' },
{ 'bar': 'two', 'wizz': '2' }
],
}
If the key is intended to contain a literal '.', then it must
be escaped as '..'. ie a flat dict
{
'foo..bar': 'wizz',
'bar.foo..bar': 'eek',
'bar.hello': 'world'
}
Will end up as
{
'foo.bar': 'wizz',
'bar': {
'foo.bar': 'eek',
'hello': 'world'
}
}
The intent of this function is that it allows a set of QemuOpts
to be turned into a nested data structure that mirrors the nesting
used when the same object is defined over QMP.
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qmp/qdict.h | 1 +
qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 551 insertions(+)
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 71b8eb0..e0d24e1 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
void qdict_join(QDict *dest, QDict *src, bool overwrite);
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 60f158c..c38e90e 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -17,6 +17,7 @@
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qobject.h"
+#include "qapi/error.h"
#include "qemu/queue.h"
#include "qemu-common.h"
#include "qemu/cutils.h"
@@ -683,6 +684,294 @@ void qdict_array_split(QDict *src, QList **dst)
}
}
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * e.g.
+ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned 'prefix'
+ * string. The 'suffix' string will be left in escaped format, so it
+ * can be fed back into the qdict_split_flat_key() key as the input
+ * later.
+ *
+ * The caller is responsible for freeing the string returned in @prefix
+ * using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix,
+ const char **suffix)
+{
+ const char *separator;
+ size_t i, j;
+
+ /* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+ separator = NULL;
+ do {
+ if (separator) {
+ separator += 2;
+ } else {
+ separator = key;
+ }
+ separator = strchr(separator, '.');
+ } while (separator && separator[1] == '.');
+
+ if (separator) {
+ *prefix = g_strndup(key, separator - key);
+ *suffix = separator + 1;
+ } else {
+ *prefix = g_strdup(key);
+ *suffix = NULL;
+ }
+
+ /* Unescape the '..' sequence into '.' */
+ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+ if ((*prefix)[i] == '.') {
+ assert((*prefix)[i + 1] == '.');
+ i++;
+ }
+ (*prefix)[j] = (*prefix)[i];
+ }
+ (*prefix)[j] = '\0';
+}
+
+
+/**
+ * qdict_is_list:
+ * @maybe_list: dict to check if keys represent list elements.
+ *
+ * Determine whether all keys in @maybe_list are valid list elements.
+ * If @maybe_list is non-zero in length and all the keys look like
+ * valid list indexes, this will return 1. If @maybe_list is zero
+ * length or all keys are non-numeric then it will return 0 to indicate
+ * it is a normal qdict. If there is a mix of numeric and non-numeric
+ * keys, or the list indexes are non-contiguous, an error is reported.
+ *
+ * Returns: 1 if a valid list, 0 if a dict, -1 on error
+ */
+static int qdict_is_list(QDict *maybe_list, Error **errp)
+{
+ const QDictEntry *ent;
+ ssize_t len = 0;
+ ssize_t max = -1;
+ int is_list = -1;
+ int64_t val;
+
+ for (ent = qdict_first(maybe_list); ent != NULL;
+ ent = qdict_next(maybe_list, ent)) {
+
+ if (qemu_strtoll(ent->key, NULL, 10, &val) == 0) {
+ if (is_list == -1) {
+ is_list = 1;
+ } else if (!is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ len++;
+ if (val > max) {
+ max = val;
+ }
+ } else {
+ if (is_list == -1) {
+ is_list = 0;
+ } else if (is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ }
+ }
+
+ if (is_list == -1) {
+ assert(!qdict_size(maybe_list));
+ is_list = 0;
+ }
+
+ /* NB this isn't a perfect check - e.g. it won't catch
+ * a list containing '1', '+1', '01', '3', but that
+ * does not matter - we've still proved that the
+ * input is a list. It is up the caller to do a
+ * stricter check if desired */
+ if (len != (max + 1)) {
+ error_setg(errp, "List indices are not contiguous, "
+ "saw %zd elements but %zd largest index",
+ len, max);
+ return -1;
+ }
+
+ return is_list;
+}
+
+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary (only scalar values) to crumple
+ * @recursive: true to recursively crumple nested dictionaries
+ *
+ * Takes a flat dictionary whose keys use '.' separator to indicate
+ * nesting, and values are scalars, and crumples it into a nested
+ * structure. If the @recursive parameter is false, then only the
+ * first level of structure implied by the keys will be crumpled. If
+ * @recursive is true, then the input will be recursively crumpled to
+ * expand all levels of structure in the keys.
+ *
+ * To include a literal '.' in a key name, it must be escaped as '..'
+ *
+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in an output of:
+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *
+ * The following scenarios in the input dict will result in an
+ * error being returned:
+ *
+ * - Any values in @src are non-scalar types
+ * - If keys in @src imply that a particular level is both a
+ * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
+ * - If keys in @src imply that a particular level is a list,
+ * but the indices are non-contiguous. e.g. "foo.0.bar" and
+ * "foo.2.bar" without any "foo.1.bar" present.
+ * - If keys in @src represent list indexes, but are not in
+ * the "%zu" format. e.g. "foo.+0.bar"
+ *
+ * Returns: either a QDict or QList for the nested data structure, or NULL
+ * on error
+ */
+QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
+{
+ const QDictEntry *ent;
+ QDict *two_level, *multi_level = NULL;
+ QObject *dst = NULL, *child;
+ size_t i;
+ char *prefix = NULL;
+ const char *suffix = NULL;
+ int is_list;
+
+ two_level = qdict_new();
+
+ /* Step 1: split our totally flat dict into a two level dict */
+ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
+ if (qobject_type(ent->value) == QTYPE_QDICT ||
+ qobject_type(ent->value) == QTYPE_QLIST) {
+ error_setg(errp, "Value %s is not a scalar",
+ ent->key);
+ goto error;
+ }
+
+ qdict_split_flat_key(ent->key, &prefix, &suffix);
+
+ child = qdict_get(two_level, prefix);
+ if (suffix) {
+ if (child) {
+ if (qobject_type(child) != QTYPE_QDICT) {
+ error_setg(errp, "Key %s prefix is already set as a scalar",
+ prefix);
+ goto error;
+ }
+ } else {
+ child = QOBJECT(qdict_new());
+ qdict_put_obj(two_level, prefix, child);
+ }
+ qobject_incref(ent->value);
+ qdict_put_obj(qobject_to_qdict(child), suffix, ent->value);
+ } else {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a dict",
+ prefix);
+ goto error;
+ }
+ qobject_incref(ent->value);
+ qdict_put_obj(two_level, prefix, ent->value);
+ }
+
+ g_free(prefix);
+ prefix = NULL;
+ }
+
+ /* Step 2: optionally process the two level dict recursively
+ * into a multi-level dict */
+ if (recursive) {
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
+
+ if (qobject_type(ent->value) == QTYPE_QDICT) {
+ child = qdict_crumple(qobject_to_qdict(ent->value),
+ recursive, errp);
+ if (!child) {
+ goto error;
+ }
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qobject_incref(ent->value);
+ qdict_put_obj(multi_level, ent->key, ent->value);
+ }
+ }
+ QDECREF(two_level);
+ } else {
+ multi_level = two_level;
+ }
+ two_level = NULL;
+
+ /* Step 3: detect if we need to turn our dict into list */
+ is_list = qdict_is_list(multi_level, errp);
+ if (is_list < 0) {
+ goto error;
+ }
+
+ if (is_list) {
+ dst = QOBJECT(qlist_new());
+
+ for (i = 0; i < qdict_size(multi_level); i++) {
+ char *key = g_strdup_printf("%zu", i);
+
+ child = qdict_get(multi_level, key);
+ g_free(key);
+
+ if (!child) {
+ error_setg(errp, "Missing list index %zu", i);
+ goto error;
+ }
+
+ qobject_incref(child);
+ qlist_append_obj(qobject_to_qlist(dst), child);
+ }
+ QDECREF(multi_level);
+ multi_level = NULL;
+ } else {
+ dst = QOBJECT(multi_level);
+ }
+
+ return dst;
+
+ error:
+ g_free(prefix);
+ QDECREF(multi_level);
+ QDECREF(two_level);
+ qobject_decref(dst);
+ return NULL;
+}
+
+
/**
* qdict_array_entries(): Returns the number of direct array entries if the
* sub-QDict of src specified by the prefix in subqdict (or src itself for
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 42da1e6..64c33ab 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -14,6 +14,7 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
+#include "qapi/error.h"
#include "qemu-common.h"
/*
@@ -595,6 +596,255 @@ static void qdict_join_test(void)
QDECREF(dict2);
}
+
+static void qdict_crumple_test_nonrecursive(void)
+{
+ QDict *src, *dst, *rules, *vnc, *acl;
+ QObject *child, *res;
+
+ src = qdict_new();
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+ qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+ qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+ qdict_put(src, "acl..name", qstring_from_str("acl0"));
+ qdict_put(src, "acl.rule..name", qstring_from_str("acl0"));
+
+ res = qdict_crumple(src, false, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 4);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qdict_get_qdict(dst, "vnc");
+
+ g_assert_cmpint(qdict_size(vnc), ==, 2);
+
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
+
+ child = qdict_get(dst, "rule");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ rules = qdict_get_qdict(dst, "rule");
+
+ g_assert_cmpint(qdict_size(rules), ==, 4);
+
+ g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
+ g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
+
+ /* With non-recursive crumpling, we should only see the first
+ * level names unescaped */
+ g_assert_cmpstr("acl0", ==, qdict_get_str(dst, "acl.name"));
+ child = qdict_get(dst, "acl");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ acl = qdict_get_qdict(dst, "acl");
+ g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule..name"));
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_listtop(void)
+{
+ QDict *src, *rule;
+ QList *rules;
+ QObject *res;
+
+ src = qdict_new();
+ qdict_put(src, "0.match.name", qstring_from_str("Fred"));
+ qdict_put(src, "0.match.email", qstring_from_str("fred@example.com"));
+ qdict_put(src, "0.policy", qstring_from_str("allow"));
+ qdict_put(src, "1.match.name", qstring_from_str("Bob"));
+ qdict_put(src, "1.match.email", qstring_from_str("bob@example.com"));
+ qdict_put(src, "1.policy", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, false, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QLIST);
+
+ rules = qobject_to_qlist(res);
+
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 3);
+
+ g_assert_cmpstr("Fred", ==, qdict_get_str(rule, "match.name"));
+ g_assert_cmpstr("fred@example.com", ==, qdict_get_str(rule, "match.email"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 3);
+
+ g_assert_cmpstr("Bob", ==, qdict_get_str(rule, "match.name"));
+ g_assert_cmpstr("bob@example.com", ==, qdict_get_str(rule, "match.email"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ QDECREF(src);
+ qobject_decref(res);
+}
+
+
+static void qdict_crumple_test_recursive(void)
+{
+ QDict *src, *dst, *rule, *vnc, *acl, *listen;
+ QObject *child, *res;
+ QList *rules;
+
+ src = qdict_new();
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "vnc.acl.rules.0.match", qstring_from_str("fred"));
+ qdict_put(src, "vnc.acl.rules.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "vnc.acl.rules.1.match", qstring_from_str("bob"));
+ qdict_put(src, "vnc.acl.rules.1.policy", qstring_from_str("deny"));
+ qdict_put(src, "vnc.acl.default", qstring_from_str("deny"));
+ qdict_put(src, "vnc.acl..name", qstring_from_str("acl0"));
+ qdict_put(src, "vnc.acl.rule..name", qstring_from_str("acl0"));
+
+ res = qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 1);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qobject_to_qdict(child);
+
+ child = qdict_get(vnc, "listen");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ listen = qobject_to_qdict(child);
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
+
+ child = qdict_get(vnc, "acl");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ acl = qobject_to_qdict(child);
+
+ child = qdict_get(acl, "rules");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+ rules = qobject_to_qlist(child);
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ /* With recursive crumpling, we should see all names unescaped */
+ g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
+ child = qdict_get(vnc, "acl");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ acl = qdict_get_qdict(vnc, "acl");
+ g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_empty(void)
+{
+ QDict *src, *dst;
+
+ src = qdict_new();
+
+ dst = (QDict *)qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qdict_size(dst), ==, 0);
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+ QDict *src;
+ Error *error = NULL;
+
+ src = qdict_new();
+ /* rule.0 can't be both a string and a dict */
+ qdict_put(src, "rule.0", qstring_from_str("fred"));
+ qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+ src = qdict_new();
+ /* rule can't be both a list and a dict */
+ qdict_put(src, "rule.0", qstring_from_str("fred"));
+ qdict_put(src, "rule.a", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+ src = qdict_new();
+ /* The input should be flat, ie no dicts or lists */
+ qdict_put(src, "rule.a", qdict_new());
+ qdict_put(src, "rule.b", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+
+ src = qdict_new();
+ /* List indexes must not have gaps */
+ qdict_put(src, "rule.0", qstring_from_str("deny"));
+ qdict_put(src, "rule.3", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+
+ src = qdict_new();
+ /* List indexes must be in %zu format */
+ qdict_put(src, "rule.0", qstring_from_str("deny"));
+ qdict_put(src, "rule.+1", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+}
+
+
/*
* Errors test-cases
*/
@@ -742,6 +992,17 @@ int main(int argc, char **argv)
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
+ g_test_add_func("/public/crumple/nonrecursive",
+ qdict_crumple_test_nonrecursive);
+ g_test_add_func("/public/crumple/recursive",
+ qdict_crumple_test_recursive);
+ g_test_add_func("/public/crumple/listtop",
+ qdict_crumple_test_listtop);
+ g_test_add_func("/public/crumple/empty",
+ qdict_crumple_test_empty);
+ g_test_add_func("/public/crumple/bad_inputs",
+ qdict_crumple_test_bad_inputs);
+
/* The Big one */
if (g_test_slow()) {
g_test_add_func("/stress/test", qdict_stress_test);
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 15:49 ` Eric Blake
` (2 more replies)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor Daniel P. Berrange
` (19 subsequent siblings)
22 siblings, 3 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Allow tracing of the operation of visitors
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
Makefile.objs | 1 +
qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
qapi/trace-events | 33 +++++++++++++++++++++++++++++++++
3 files changed, 61 insertions(+)
create mode 100644 qapi/trace-events
diff --git a/Makefile.objs b/Makefile.objs
index a8e0224..b3e8aef 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -160,3 +160,4 @@ trace-events-y += target-s390x/trace-events
trace-events-y += target-ppc/trace-events
trace-events-y += qom/trace-events
trace-events-y += linux-user/trace-events
+trace-events-y += qapi/trace-events
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 55f5876..bfcb276 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -19,10 +19,12 @@
#include "qapi/qmp/qerror.h"
#include "qapi/visitor.h"
#include "qapi/visitor-impl.h"
+#include "trace.h"
void visit_complete(Visitor *v, void *opaque)
{
assert(v->type != VISITOR_OUTPUT || v->complete);
+ trace_visit_complete(v, opaque);
if (v->complete) {
v->complete(v, opaque);
}
@@ -30,6 +32,7 @@ void visit_complete(Visitor *v, void *opaque)
void visit_free(Visitor *v)
{
+ trace_visit_free(v);
if (v) {
v->free(v);
}
@@ -40,6 +43,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
{
Error *err = NULL;
+ trace_visit_start_struct(v, name, obj, size);
if (obj) {
assert(size);
assert(!(v->type & VISITOR_OUTPUT) || *obj);
@@ -53,6 +57,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
void visit_check_struct(Visitor *v, Error **errp)
{
+ trace_visit_check_struct(v);
if (v->check_struct) {
v->check_struct(v, errp);
}
@@ -60,6 +65,7 @@ void visit_check_struct(Visitor *v, Error **errp)
void visit_end_struct(Visitor *v, void **obj)
{
+ trace_visit_end_struct(v, obj);
v->end_struct(v, obj);
}
@@ -69,6 +75,7 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
Error *err = NULL;
assert(!list || size >= sizeof(GenericList));
+ trace_visit_start_list(v, name, list, size);
v->start_list(v, name, list, size, &err);
if (list && (v->type & VISITOR_INPUT)) {
assert(!(err && *list));
@@ -79,11 +86,13 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
{
assert(tail && size >= sizeof(GenericList));
+ trace_visit_next_list(v, tail, size);
return v->next_list(v, tail, size);
}
void visit_end_list(Visitor *v, void **obj)
{
+ trace_visit_end_list(v, obj);
v->end_list(v, obj);
}
@@ -95,6 +104,7 @@ void visit_start_alternate(Visitor *v, const char *name,
assert(obj && size >= sizeof(GenericAlternate));
assert(!(v->type & VISITOR_OUTPUT) || *obj);
+ trace_visit_start_alternate(v, name, obj, size, promote_int);
if (v->start_alternate) {
v->start_alternate(v, name, obj, size, promote_int, &err);
}
@@ -106,6 +116,7 @@ void visit_start_alternate(Visitor *v, const char *name,
void visit_end_alternate(Visitor *v, void **obj)
{
+ trace_visit_end_alternate(v, obj);
if (v->end_alternate) {
v->end_alternate(v, obj);
}
@@ -113,6 +124,7 @@ void visit_end_alternate(Visitor *v, void **obj)
bool visit_optional(Visitor *v, const char *name, bool *present)
{
+ trace_visit_optional(v, name, present);
if (v->optional) {
v->optional(v, name, present);
}
@@ -127,6 +139,7 @@ bool visit_is_input(Visitor *v)
void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
{
assert(obj);
+ trace_visit_type_int(v, name, obj);
v->type_int64(v, name, obj, errp);
}
@@ -151,6 +164,7 @@ void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
Error **errp)
{
uint64_t value = *obj;
+ trace_visit_type_uint8(v, name, obj);
visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
*obj = value;
}
@@ -159,6 +173,7 @@ void visit_type_uint16(Visitor *v, const char *name, uint16_t *obj,
Error **errp)
{
uint64_t value = *obj;
+ trace_visit_type_uint16(v, name, obj);
visit_type_uintN(v, &value, name, UINT16_MAX, "uint16_t", errp);
*obj = value;
}
@@ -167,6 +182,7 @@ void visit_type_uint32(Visitor *v, const char *name, uint32_t *obj,
Error **errp)
{
uint64_t value = *obj;
+ trace_visit_type_uint32(v, name, obj);
visit_type_uintN(v, &value, name, UINT32_MAX, "uint32_t", errp);
*obj = value;
}
@@ -175,6 +191,7 @@ void visit_type_uint64(Visitor *v, const char *name, uint64_t *obj,
Error **errp)
{
assert(obj);
+ trace_visit_type_uint64(v, name, obj);
v->type_uint64(v, name, obj, errp);
}
@@ -199,6 +216,7 @@ static void visit_type_intN(Visitor *v, int64_t *obj, const char *name,
void visit_type_int8(Visitor *v, const char *name, int8_t *obj, Error **errp)
{
int64_t value = *obj;
+ trace_visit_type_int8(v, name, obj);
visit_type_intN(v, &value, name, INT8_MIN, INT8_MAX, "int8_t", errp);
*obj = value;
}
@@ -207,6 +225,7 @@ void visit_type_int16(Visitor *v, const char *name, int16_t *obj,
Error **errp)
{
int64_t value = *obj;
+ trace_visit_type_int16(v, name, obj);
visit_type_intN(v, &value, name, INT16_MIN, INT16_MAX, "int16_t", errp);
*obj = value;
}
@@ -215,6 +234,7 @@ void visit_type_int32(Visitor *v, const char *name, int32_t *obj,
Error **errp)
{
int64_t value = *obj;
+ trace_visit_type_int32(v, name, obj);
visit_type_intN(v, &value, name, INT32_MIN, INT32_MAX, "int32_t", errp);
*obj = value;
}
@@ -223,6 +243,7 @@ void visit_type_int64(Visitor *v, const char *name, int64_t *obj,
Error **errp)
{
assert(obj);
+ trace_visit_type_int64(v, name, obj);
v->type_int64(v, name, obj, errp);
}
@@ -230,6 +251,7 @@ void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
Error **errp)
{
assert(obj);
+ trace_visit_type_size(v, name, obj);
if (v->type_size) {
v->type_size(v, name, obj, errp);
} else {
@@ -240,6 +262,7 @@ void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
void visit_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
{
assert(obj);
+ trace_visit_type_bool(v, name, obj);
v->type_bool(v, name, obj, errp);
}
@@ -252,6 +275,7 @@ void visit_type_str(Visitor *v, const char *name, char **obj, Error **errp)
* can enable:
assert(!(v->type & VISITOR_OUTPUT) || *obj);
*/
+ trace_visit_type_str(v, name, obj);
v->type_str(v, name, obj, &err);
if (v->type & VISITOR_INPUT) {
assert(!err != !*obj);
@@ -263,6 +287,7 @@ void visit_type_number(Visitor *v, const char *name, double *obj,
Error **errp)
{
assert(obj);
+ trace_visit_type_number(v, name, obj);
v->type_number(v, name, obj, errp);
}
@@ -272,6 +297,7 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
assert(obj);
assert(v->type != VISITOR_OUTPUT || *obj);
+ trace_visit_type_any(v, name, obj);
v->type_any(v, name, obj, &err);
if (v->type == VISITOR_INPUT) {
assert(!err != !*obj);
@@ -281,6 +307,7 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
void visit_type_null(Visitor *v, const char *name, Error **errp)
{
+ trace_visit_type_null(v, name);
v->type_null(v, name, errp);
}
diff --git a/qapi/trace-events b/qapi/trace-events
new file mode 100644
index 0000000..2c5d3bc
--- /dev/null
+++ b/qapi/trace-events
@@ -0,0 +1,33 @@
+# qapi-visit-core.c
+visit_free(void *v) "v=%p"
+visit_complete(void *v, void *opaque) "v=%p opaque=%p"
+
+visit_start_struct(void *v, const char *name, void *obj, size_t size) "v=%p name=%s obj=%p size=%zu"
+visit_check_struct(void *v) "v=%p"
+visit_end_struct(void *v, void *obj) "v=%p obj=%p"
+
+visit_start_list(void *v, const char *name, void *obj, size_t size) "v=%p name=%s obj=%p size=%zu"
+visit_next_list(void *v, void *tail, size_t size) "v=%p tail=%p size=%zu"
+visit_end_list(void *v, void *obj) "v=%p obj=%p"
+
+visit_start_alternate(void *v, const char *name, void *obj, size_t size, bool promote_int) "v=%p name=%s obj=%p size=%zu promote_int=%d"
+visit_end_alternate(void *v, void *obj) "v=%p obj=%p"
+
+visit_optional(void *v, const char *name, bool *present) "v=%p name=%s present=%p"
+
+visit_type_enum(void *v, const char *name, int *obj) "v=%p name=%s obj=%p"
+visit_type_int(void *v, const char *name, int64_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint8(void *v, const char *name, uint8_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint16(void *v, const char *name, uint16_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint32(void *v, const char *name, uint32_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint64(void *v, const char *name, uint64_t *obj) "v=%p name=%s obj=%p"
+visit_type_int8(void *v, const char *name, int8_t *obj) "v=%p name=%s obj=%p"
+visit_type_int16(void *v, const char *name, int16_t *obj) "v=%p name=%s obj=%p"
+visit_type_int32(void *v, const char *name, int32_t *obj) "v=%p name=%s obj=%p"
+visit_type_int64(void *v, const char *name, int64_t *obj) "v=%p name=%s obj=%p"
+visit_type_size(void *v, const char *name, uint64_t *obj) "v=%p name=%s obj=%p"
+visit_type_bool(void *v, const char *name, bool *obj) "v=%p name=%s obj=%p"
+visit_type_str(void *v, const char *name, char **obj) "v=%p name=%s obj=%p"
+visit_type_number(void *v, const char *name, double *obj) "v=%p name=%s obj=%p"
+visit_type_any(void *v, const char *name, void *obj) "v=%p name=%s obj=%p"
+visit_type_null(void *v, const char *name) "v=%p name=%s"
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (2 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-25 13:41 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor Daniel P. Berrange
` (18 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The QmpInputVisitor has no direct dependency on QMP. It is
valid to use it anywhere that one has a QObject. Rename it
to better reflect its functionality as a generic QObject
to QAPI converter.
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
docs/qapi-code-gen.txt | 2 +-
...qmp-input-visitor.h => qobject-input-visitor.h} | 10 +-
include/qapi/visitor.h | 6 +-
monitor.c | 2 +-
qapi/Makefile.objs | 2 +-
...qmp-input-visitor.c => qobject-input-visitor.c} | 171 +++++++++++----------
qmp.c | 4 +-
qom/qom-qobject.c | 4 +-
scripts/qapi-commands.py | 4 +-
target-s390x/cpu_models.c | 4 +-
tests/.gitignore | 4 +-
tests/Makefile.include | 12 +-
tests/check-qnull.c | 4 +-
tests/test-qmp-commands.c | 4 +-
...-input-strict.c => test-qobject-input-strict.c} | 6 +-
...nput-visitor.c => test-qobject-input-visitor.c} | 6 +-
tests/test-string-input-visitor.c | 2 +-
tests/test-visitor-serialization.c | 4 +-
util/qemu-sockets.c | 2 +-
19 files changed, 128 insertions(+), 125 deletions(-)
rename include/qapi/{qmp-input-visitor.h => qobject-input-visitor.h} (63%)
rename qapi/{qmp-input-visitor.c => qobject-input-visitor.c} (56%)
rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (99%)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5d4c2cd..d2604b6 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1024,7 +1024,7 @@ Example:
Visitor *v;
UserDefOneList *arg1 = NULL;
- v = qmp_input_visitor_new(QOBJECT(args), true);
+ v = qobject_input_visitor_new(QOBJECT(args), true);
visit_start_struct(v, NULL, NULL, 0, &err);
if (err) {
goto out;
diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qobject-input-visitor.h
similarity index 63%
rename from include/qapi/qmp-input-visitor.h
rename to include/qapi/qobject-input-visitor.h
index f3ff5f3..cde328d 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -11,20 +11,20 @@
*
*/
-#ifndef QMP_INPUT_VISITOR_H
-#define QMP_INPUT_VISITOR_H
+#ifndef QOBJECT_INPUT_VISITOR_H
+#define QOBJECT_INPUT_VISITOR_H
#include "qapi/visitor.h"
#include "qapi/qmp/qobject.h"
-typedef struct QmpInputVisitor QmpInputVisitor;
+typedef struct QObjectInputVisitor QObjectInputVisitor;
/*
- * Return a new input visitor that converts QMP to QAPI.
+ * Return a new input visitor that converts a QObject to a QAPI object.
*
* Set @strict to reject a parse that doesn't consume all keys of a
* dictionary; otherwise excess input is ignored.
*/
-Visitor *qmp_input_visitor_new(QObject *obj, bool strict);
+Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
#endif
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 6c77a91..9bb6cba 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -25,14 +25,14 @@
* for doing work at each node of a QAPI graph; it can also be used
* for a virtual walk, where there is no actual QAPI C struct.
*
- * There are four kinds of visitor classes: input visitors (QMP,
+ * There are four kinds of visitor classes: input visitors (QObject,
* string, and QemuOpts) parse an external representation and build
- * the corresponding QAPI graph, output visitors (QMP and string) take
+ * the corresponding QAPI graph, output visitors (QObject and string) take
* a completed QAPI graph and generate an external representation, the
* dealloc visitor can take a QAPI graph (possibly partially
* constructed) and recursively free its resources, and the clone
* visitor performs a deep clone of one QAPI object to another. While
- * the dealloc and QMP input/output visitors are general, the string,
+ * the dealloc and QObject input/output visitors are general, the string,
* QemuOpts, and clone visitors have some implementation limitations;
* see the documentation for each visitor for more details on what it
* supports. Also, see visitor-impl.h for the callback contracts
diff --git a/monitor.c b/monitor.c
index 83c4edf..14089cc 100644
--- a/monitor.c
+++ b/monitor.c
@@ -957,7 +957,7 @@ EventInfoList *qmp_query_events(Error **errp)
* directly into QObject instead of first parsing it with
* visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it
* to QObject with generated output marshallers, every time. Instead,
- * we do it in test-qmp-input-visitor.c, just to make sure
+ * we do it in test-qobject-input-visitor.c, just to make sure
* qapi-introspect.py's output actually conforms to the schema.
*/
static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 7ea4aeb..6ec7bdc 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,4 +1,4 @@
-util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
+util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
util-obj-y += opts-visitor.o qapi-clone-visitor.o
diff --git a/qapi/qmp-input-visitor.c b/qapi/qobject-input-visitor.c
similarity index 56%
rename from qapi/qmp-input-visitor.c
rename to qapi/qobject-input-visitor.c
index 64dd392..5ff3db3 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -14,7 +14,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/visitor-impl.h"
#include "qemu/queue.h"
#include "qemu-common.h"
@@ -34,7 +34,7 @@ typedef struct StackObject
QSLIST_ENTRY(StackObject) node;
} StackObject;
-struct QmpInputVisitor
+struct QObjectInputVisitor
{
Visitor visitor;
@@ -49,14 +49,14 @@ struct QmpInputVisitor
bool strict;
};
-static QmpInputVisitor *to_qiv(Visitor *v)
+static QObjectInputVisitor *to_qiv(Visitor *v)
{
- return container_of(v, QmpInputVisitor, visitor);
+ return container_of(v, QObjectInputVisitor, visitor);
}
-static QObject *qmp_input_get_object(QmpInputVisitor *qiv,
- const char *name,
- bool consume)
+static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
+ const char *name,
+ bool consume)
{
StackObject *tos;
QObject *qobj;
@@ -97,8 +97,9 @@ static void qdict_add_key(const char *key, QObject *obj, void *opaque)
g_hash_table_insert(h, (gpointer) key, NULL);
}
-static const QListEntry *qmp_input_push(QmpInputVisitor *qiv, QObject *obj,
- void *qapi, Error **errp)
+static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
+ QObject *obj, void *qapi,
+ Error **errp)
{
GHashTable *h;
StackObject *tos = g_new0(StackObject, 1);
@@ -120,9 +121,9 @@ static const QListEntry *qmp_input_push(QmpInputVisitor *qiv, QObject *obj,
}
-static void qmp_input_check_struct(Visitor *v, Error **errp)
+static void qobject_input_check_struct(Visitor *v, Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
+ QObjectInputVisitor *qiv = to_qiv(v);
StackObject *tos = QSLIST_FIRST(&qiv->stack);
assert(tos && !tos->entry);
@@ -140,7 +141,7 @@ static void qmp_input_check_struct(Visitor *v, Error **errp)
}
}
-static void qmp_input_stack_object_free(StackObject *tos)
+static void qobject_input_stack_object_free(StackObject *tos)
{
if (tos->h) {
g_hash_table_unref(tos->h);
@@ -149,21 +150,21 @@ static void qmp_input_stack_object_free(StackObject *tos)
g_free(tos);
}
-static void qmp_input_pop(Visitor *v, void **obj)
+static void qobject_input_pop(Visitor *v, void **obj)
{
- QmpInputVisitor *qiv = to_qiv(v);
+ QObjectInputVisitor *qiv = to_qiv(v);
StackObject *tos = QSLIST_FIRST(&qiv->stack);
assert(tos && tos->qapi == obj);
QSLIST_REMOVE_HEAD(&qiv->stack, node);
- qmp_input_stack_object_free(tos);
+ qobject_input_stack_object_free(tos);
}
-static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
- size_t size, Error **errp)
+static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
+ size_t size, Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, true);
Error *err = NULL;
if (obj) {
@@ -175,7 +176,7 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
return;
}
- qmp_input_push(qiv, qobj, obj, &err);
+ qobject_input_push(qiv, qobj, obj, &err);
if (err) {
error_propagate(errp, err);
return;
@@ -187,11 +188,12 @@ static void qmp_input_start_struct(Visitor *v, const char *name, void **obj,
}
-static void qmp_input_start_list(Visitor *v, const char *name,
- GenericList **list, size_t size, Error **errp)
+static void qobject_input_start_list(Visitor *v, const char *name,
+ GenericList **list, size_t size,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, true);
const QListEntry *entry;
if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
@@ -203,7 +205,7 @@ static void qmp_input_start_list(Visitor *v, const char *name,
return;
}
- entry = qmp_input_push(qiv, qobj, list, errp);
+ entry = qobject_input_push(qiv, qobj, list, errp);
if (list) {
if (entry) {
*list = g_malloc0(size);
@@ -213,10 +215,10 @@ static void qmp_input_start_list(Visitor *v, const char *name,
}
}
-static GenericList *qmp_input_next_list(Visitor *v, GenericList *tail,
- size_t size)
+static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
+ size_t size)
{
- QmpInputVisitor *qiv = to_qiv(v);
+ QObjectInputVisitor *qiv = to_qiv(v);
StackObject *so = QSLIST_FIRST(&qiv->stack);
if (!so->entry) {
@@ -227,12 +229,12 @@ static GenericList *qmp_input_next_list(Visitor *v, GenericList *tail,
}
-static void qmp_input_start_alternate(Visitor *v, const char *name,
- GenericAlternate **obj, size_t size,
- bool promote_int, Error **errp)
+static void qobject_input_start_alternate(Visitor *v, const char *name,
+ GenericAlternate **obj, size_t size,
+ bool promote_int, Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, false);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, false);
if (!qobj) {
*obj = NULL;
@@ -246,11 +248,11 @@ static void qmp_input_start_alternate(Visitor *v, const char *name,
}
}
-static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
- Error **errp)
+static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QInt *qint = qobject_to_qint(qobject_input_get_object(qiv, name, true));
if (!qint) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -261,12 +263,12 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
*obj = qint_get_int(qint);
}
-static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
- Error **errp)
+static void qobject_input_type_uint64(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
{
/* FIXME: qobject_to_qint mishandles values over INT64_MAX */
- QmpInputVisitor *qiv = to_qiv(v);
- QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QInt *qint = qobject_to_qint(qobject_input_get_object(qiv, name, true));
if (!qint) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -277,11 +279,11 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
*obj = qint_get_int(qint);
}
-static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
- Error **errp)
+static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QBool *qbool = qobject_to_qbool(qobject_input_get_object(qiv, name, true));
if (!qbool) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -292,11 +294,12 @@ static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
*obj = qbool_get_bool(qbool);
}
-static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
- Error **errp)
+static void qobject_input_type_str(Visitor *v, const char *name, char **obj,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
if (!qstr) {
*obj = NULL;
@@ -308,11 +311,11 @@ static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
*obj = g_strdup(qstring_get_str(qstr));
}
-static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
- Error **errp)
+static void qobject_input_type_number(Visitor *v, const char *name, double *obj,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, true);
QInt *qint;
QFloat *qfloat;
@@ -332,20 +335,20 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
"number");
}
-static void qmp_input_type_any(Visitor *v, const char *name, QObject **obj,
- Error **errp)
+static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj,
+ Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, true);
qobject_incref(qobj);
*obj = qobj;
}
-static void qmp_input_type_null(Visitor *v, const char *name, Error **errp)
+static void qobject_input_type_null(Visitor *v, const char *name, Error **errp)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, true);
if (qobject_type(qobj) != QTYPE_QNULL) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -353,10 +356,10 @@ static void qmp_input_type_null(Visitor *v, const char *name, Error **errp)
}
}
-static void qmp_input_optional(Visitor *v, const char *name, bool *present)
+static void qobject_input_optional(Visitor *v, const char *name, bool *present)
{
- QmpInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qmp_input_get_object(qiv, name, false);
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qobject_input_get_object(qiv, name, false);
if (!qobj) {
*present = false;
@@ -366,43 +369,43 @@ static void qmp_input_optional(Visitor *v, const char *name, bool *present)
*present = true;
}
-static void qmp_input_free(Visitor *v)
+static void qobject_input_free(Visitor *v)
{
- QmpInputVisitor *qiv = to_qiv(v);
+ QObjectInputVisitor *qiv = to_qiv(v);
while (!QSLIST_EMPTY(&qiv->stack)) {
StackObject *tos = QSLIST_FIRST(&qiv->stack);
QSLIST_REMOVE_HEAD(&qiv->stack, node);
- qmp_input_stack_object_free(tos);
+ qobject_input_stack_object_free(tos);
}
qobject_decref(qiv->root);
g_free(qiv);
}
-Visitor *qmp_input_visitor_new(QObject *obj, bool strict)
+Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
{
- QmpInputVisitor *v;
+ QObjectInputVisitor *v;
v = g_malloc0(sizeof(*v));
v->visitor.type = VISITOR_INPUT;
- v->visitor.start_struct = qmp_input_start_struct;
- v->visitor.check_struct = qmp_input_check_struct;
- v->visitor.end_struct = qmp_input_pop;
- v->visitor.start_list = qmp_input_start_list;
- v->visitor.next_list = qmp_input_next_list;
- v->visitor.end_list = qmp_input_pop;
- v->visitor.start_alternate = qmp_input_start_alternate;
- v->visitor.type_int64 = qmp_input_type_int64;
- v->visitor.type_uint64 = qmp_input_type_uint64;
- v->visitor.type_bool = qmp_input_type_bool;
- v->visitor.type_str = qmp_input_type_str;
- v->visitor.type_number = qmp_input_type_number;
- v->visitor.type_any = qmp_input_type_any;
- v->visitor.type_null = qmp_input_type_null;
- v->visitor.optional = qmp_input_optional;
- v->visitor.free = qmp_input_free;
+ v->visitor.start_struct = qobject_input_start_struct;
+ v->visitor.check_struct = qobject_input_check_struct;
+ v->visitor.end_struct = qobject_input_pop;
+ v->visitor.start_list = qobject_input_start_list;
+ v->visitor.next_list = qobject_input_next_list;
+ v->visitor.end_list = qobject_input_pop;
+ v->visitor.start_alternate = qobject_input_start_alternate;
+ v->visitor.type_int64 = qobject_input_type_int64;
+ v->visitor.type_uint64 = qobject_input_type_uint64;
+ v->visitor.type_bool = qobject_input_type_bool;
+ v->visitor.type_str = qobject_input_type_str;
+ v->visitor.type_number = qobject_input_type_number;
+ v->visitor.type_any = qobject_input_type_any;
+ v->visitor.type_null = qobject_input_type_null;
+ v->visitor.optional = qobject_input_optional;
+ v->visitor.free = qobject_input_free;
v->strict = strict;
v->root = obj;
diff --git a/qmp.c b/qmp.c
index 621f6ae..642bf41 100644
--- a/qmp.c
+++ b/qmp.c
@@ -31,7 +31,7 @@
#include "qom/qom-qobject.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qobject.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "hw/boards.h"
#include "qom/object_interfaces.h"
#include "hw/mem/pc-dimm.h"
@@ -672,7 +672,7 @@ void qmp_object_add(const char *type, const char *id,
}
}
- v = qmp_input_visitor_new(props, true);
+ v = qobject_input_visitor_new(props, true);
obj = user_creatable_add_type(type, id, pdict, v, errp);
visit_free(v);
if (obj) {
diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
index c225abc..81959e0 100644
--- a/qom/qom-qobject.c
+++ b/qom/qom-qobject.c
@@ -15,7 +15,7 @@
#include "qom/object.h"
#include "qom/qom-qobject.h"
#include "qapi/visitor.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
void object_property_set_qobject(Object *obj, QObject *value,
@@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
{
Visitor *v;
/* TODO: Should we reject, rather than ignore, excess input? */
- v = qmp_input_visitor_new(value, false);
+ v = qobject_input_visitor_new(value, false);
object_property_set(obj, v, name, errp);
visit_free(v);
}
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 2f603b0..04158a3 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -130,7 +130,7 @@ def gen_marshal(name, arg_type, boxed, ret_type):
push_indent()
ret += mcgen('''
- v = qmp_input_visitor_new(QOBJECT(args), true);
+ v = qobject_input_visitor_new(QOBJECT(args), true);
visit_start_struct(v, NULL, NULL, 0, &err);
if (err) {
goto out;
@@ -294,7 +294,7 @@ fdef.write(mcgen('''
#include "qapi/qmp/dispatch.h"
#include "qapi/visitor.h"
#include "qapi/qmp-output-visitor.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/dealloc-visitor.h"
#include "%(prefix)sqapi-types.h"
#include "%(prefix)sqapi-visit.h"
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 3ff6a70..c1e729d 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -17,7 +17,7 @@
#include "qapi/visitor.h"
#include "qemu/error-report.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp/qbool.h"
#ifndef CONFIG_USER_ONLY
#include "sysemu/arch_init.h"
@@ -345,7 +345,7 @@ static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info,
}
if (qdict) {
- visitor = qmp_input_visitor_new(info->props, true);
+ visitor = qobject_input_visitor_new(info->props, true);
visit_start_struct(visitor, NULL, NULL, 0, errp);
if (*errp) {
object_unref(obj);
diff --git a/tests/.gitignore b/tests/.gitignore
index 0f0c79b..bc076eb 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -57,8 +57,8 @@ test-qht-par
test-qmp-commands
test-qmp-commands.h
test-qmp-event
-test-qmp-input-strict
-test-qmp-input-visitor
+test-qobject-input-strict
+test-qobject-input-visitor
test-qmp-introspect.[ch]
test-qmp-marshal.c
test-qmp-output-visitor
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8162f6f..eb2c8b0 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -24,9 +24,9 @@ check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
check-unit-y += tests/test-clone-visitor$(EXESUF)
gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
-check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
-gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
-check-unit-y += tests/test-qmp-input-strict$(EXESUF)
+check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
+gcov-files-test-qobject-input-visitor-y = qapi/qobject-input-visitor.c
+check-unit-y += tests/test-qobject-input-strict$(EXESUF)
check-unit-y += tests/test-qmp-commands$(EXESUF)
gcov-files-test-qmp-commands-y = qapi/qmp-dispatch.c
check-unit-y += tests/test-string-input-visitor$(EXESUF)
@@ -441,7 +441,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
tests/test-clone-visitor.o \
- tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
+ tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
tests/test-qmp-commands.o tests/test-visitor-serialization.o \
tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
tests/test-opts-visitor.o tests/test-qmp-event.o \
@@ -544,8 +544,8 @@ tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(te
tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
-tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
-tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
+tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y)
+tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y)
tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
diff --git a/tests/check-qnull.c b/tests/check-qnull.c
index dc906b1..eeb803a 100644
--- a/tests/check-qnull.c
+++ b/tests/check-qnull.c
@@ -10,7 +10,7 @@
#include "qapi/qmp/qobject.h"
#include "qemu-common.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/error.h"
@@ -47,7 +47,7 @@ static void qnull_visit_test(void)
g_assert(qnull_.refcnt == 1);
obj = qnull();
- v = qmp_input_visitor_new(obj, true);
+ v = qobject_input_visitor_new(obj, true);
qobject_decref(obj);
visit_type_null(v, NULL, &error_abort);
visit_free(v);
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 81cbe54..ff94481 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -4,7 +4,7 @@
#include "test-qmp-commands.h"
#include "qapi/qmp/dispatch.h"
#include "qemu/module.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "tests/test-qapi-types.h"
#include "tests/test-qapi-visit.h"
@@ -244,7 +244,7 @@ static void test_dealloc_partial(void)
ud2_dict = qdict_new();
qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
- v = qmp_input_visitor_new(QOBJECT(ud2_dict), true);
+ v = qobject_input_visitor_new(QOBJECT(ud2_dict), true);
visit_type_UserDefTwo(v, NULL, &ud2, &err);
visit_free(v);
QDECREF(ud2_dict);
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qobject-input-strict.c
similarity index 98%
rename from tests/test-qmp-input-strict.c
rename to tests/test-qobject-input-strict.c
index 814550a..8250365 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qobject-input-strict.c
@@ -1,5 +1,5 @@
/*
- * QMP Input Visitor unit-tests (strict mode).
+ * QObject Input Visitor unit-tests (strict mode).
*
* Copyright (C) 2011-2012, 2015 Red Hat Inc.
*
@@ -15,7 +15,7 @@
#include "qemu-common.h"
#include "qapi/error.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
@@ -53,7 +53,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
- data->qiv = qmp_input_visitor_new(data->obj, true);
+ data->qiv = qobject_input_visitor_new(data->obj, true);
g_assert(data->qiv);
return data->qiv;
}
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qobject-input-visitor.c
similarity index 99%
rename from tests/test-qmp-input-visitor.c
rename to tests/test-qobject-input-visitor.c
index f583dce..0e65e63 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -1,5 +1,5 @@
/*
- * QMP Input Visitor unit-tests.
+ * QObject Input Visitor unit-tests.
*
* Copyright (C) 2011-2016 Red Hat Inc.
*
@@ -14,7 +14,7 @@
#include "qemu-common.h"
#include "qapi/error.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
@@ -49,7 +49,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
- data->qiv = qmp_input_visitor_new(data->obj, false);
+ data->qiv = qobject_input_visitor_new(data->obj, false);
g_assert(data->qiv);
return data->qiv;
}
diff --git a/tests/test-string-input-visitor.c b/tests/test-string-input-visitor.c
index a679fbc..7f10e25 100644
--- a/tests/test-string-input-visitor.c
+++ b/tests/test-string-input-visitor.c
@@ -4,7 +4,7 @@
* Copyright (C) 2012 Red Hat Inc.
*
* Authors:
- * Paolo Bonzini <pbonzini@redhat.com> (based on test-qmp-input-visitor)
+ * Paolo Bonzini <pbonzini@redhat.com> (based on test-qobject-input-visitor)
*
* This work is licensed under the terms of the GNU GPL, version 2 or later.
* See the COPYING file in the top-level directory.
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index dba4670..51df428 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -20,7 +20,7 @@
#include "qapi/error.h"
#include "qapi/qmp/types.h"
#include "qapi/qmp/qjson.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/string-input-visitor.h"
#include "qapi/string-output-visitor.h"
@@ -1040,7 +1040,7 @@ static void qmp_deserialize(void **native_out, void *datap,
obj = qobject_from_json(qstring_get_str(output_json));
QDECREF(output_json);
- d->qiv = qmp_input_visitor_new(obj, true);
+ d->qiv = qobject_input_visitor_new(obj, true);
qobject_decref(obj_orig);
qobject_decref(obj);
visit(d->qiv, native_out, errp);
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 6db48b3..9e30a21 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -21,7 +21,7 @@
#include "qapi/error.h"
#include "qemu/sockets.h"
#include "qemu/main-loop.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi-visit.h"
#include "qemu/cutils.h"
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (3 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-25 13:36 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests Daniel P. Berrange
` (17 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The QmpOutputVisitor has no direct dependency on QMP. It is
valid to use it anywhere that one wants a QObject. Rename it
to better reflect its functionality as a generic QAPI
to QObject converter.
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
block/qapi.c | 4 +-
blockdev.c | 4 +-
docs/qapi-code-gen.txt | 2 +-
...p-output-visitor.h => qobject-output-visitor.h} | 10 +-
qapi/Makefile.objs | 2 +-
qapi/qapi-clone-visitor.c | 2 +-
qapi/qmp-output-visitor.c | 256 ---------------------
qapi/qobject-output-visitor.c | 254 ++++++++++++++++++++
qemu-img.c | 8 +-
qom/object_interfaces.c | 2 +-
qom/qom-qobject.c | 4 +-
scripts/qapi-commands.py | 4 +-
scripts/qapi-event.py | 4 +-
tests/.gitignore | 2 +-
tests/Makefile.include | 8 +-
tests/check-qnull.c | 4 +-
...put-visitor.c => test-qobject-output-visitor.c} | 6 +-
tests/test-string-output-visitor.c | 2 +-
tests/test-visitor-serialization.c | 4 +-
util/qemu-sockets.c | 2 +-
20 files changed, 291 insertions(+), 293 deletions(-)
rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
delete mode 100644 qapi/qmp-output-visitor.c
create mode 100644 qapi/qobject-output-visitor.c
rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)
diff --git a/block/qapi.c b/block/qapi.c
index 6f947e3..f35c89f 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -29,7 +29,7 @@
#include "block/write-threshold.h"
#include "qmp-commands.h"
#include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/qmp/types.h"
#include "sysemu/block-backend.h"
#include "qemu/cutils.h"
@@ -691,7 +691,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f,
ImageInfoSpecific *info_spec)
{
QObject *obj, *data;
- Visitor *v = qmp_output_visitor_new(&obj);
+ Visitor *v = qobject_output_visitor_new(&obj);
visit_type_ImageInfoSpecific(v, NULL, &info_spec, &error_abort);
visit_complete(v, &obj);
diff --git a/blockdev.c b/blockdev.c
index 29c6561..814d49f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -43,7 +43,7 @@
#include "qapi/qmp/types.h"
#include "qapi-visit.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/util.h"
#include "sysemu/sysemu.h"
#include "block/block_int.h"
@@ -3828,7 +3828,7 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)
{
BlockDriverState *bs;
QObject *obj;
- Visitor *v = qmp_output_visitor_new(&obj);
+ Visitor *v = qobject_output_visitor_new(&obj);
QDict *qdict;
Error *local_err = NULL;
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d2604b6..2841c51 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1005,7 +1005,7 @@ Example:
Error *err = NULL;
Visitor *v;
- v = qmp_output_visitor_new(ret_out);
+ v = qobject_output_visitor_new(ret_out);
visit_type_UserDefOne(v, "unused", &ret_in, &err);
if (!err) {
visit_complete(v, ret_out);
diff --git a/include/qapi/qmp-output-visitor.h b/include/qapi/qobject-output-visitor.h
similarity index 66%
rename from include/qapi/qmp-output-visitor.h
rename to include/qapi/qobject-output-visitor.h
index 040fdda..358c959 100644
--- a/include/qapi/qmp-output-visitor.h
+++ b/include/qapi/qobject-output-visitor.h
@@ -11,20 +11,20 @@
*
*/
-#ifndef QMP_OUTPUT_VISITOR_H
-#define QMP_OUTPUT_VISITOR_H
+#ifndef QOBJECT_OUTPUT_VISITOR_H
+#define QOBJECT_OUTPUT_VISITOR_H
#include "qapi/visitor.h"
#include "qapi/qmp/qobject.h"
-typedef struct QmpOutputVisitor QmpOutputVisitor;
+typedef struct QObjectOutputVisitor QObjectOutputVisitor;
/*
- * Create a new QMP output visitor.
+ * Create a new QOBJECT output visitor.
*
* If everything else succeeds, pass @result to visit_complete() to
* collect the result of the visit.
*/
-Visitor *qmp_output_visitor_new(QObject **result);
+Visitor *qobject_output_visitor_new(QObject **result);
#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 6ec7bdc..33906ff 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,5 +1,5 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
-util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
+util-obj-y += qobject-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
util-obj-y += opts-visitor.o qapi-clone-visitor.o
util-obj-y += qmp-event.o
diff --git a/qapi/qapi-clone-visitor.c b/qapi/qapi-clone-visitor.c
index 0bb8216..34086cb 100644
--- a/qapi/qapi-clone-visitor.c
+++ b/qapi/qapi-clone-visitor.c
@@ -110,7 +110,7 @@ static void qapi_clone_type_str(Visitor *v, const char *name, char **obj,
assert(qcv->depth);
/*
* Pointer was already cloned by g_memdup; create fresh copy.
- * Note that as long as qmp-output-visitor accepts NULL instead of
+ * Note that as long as qobject-output-visitor accepts NULL instead of
* "", then we must do likewise. However, we want to obey the
* input visitor semantics of never producing NULL when the empty
* string is intended.
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
deleted file mode 100644
index 9e3b67c..0000000
--- a/qapi/qmp-output-visitor.c
+++ /dev/null
@@ -1,256 +0,0 @@
-/*
- * Core Definitions for QAPI/QMP Command Registry
- *
- * Copyright (C) 2012-2016 Red Hat, Inc.
- * Copyright IBM, Corp. 2011
- *
- * Authors:
- * Anthony Liguori <aliguori@us.ibm.com>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#include "qemu/osdep.h"
-#include "qapi/qmp-output-visitor.h"
-#include "qapi/visitor-impl.h"
-#include "qemu/queue.h"
-#include "qemu-common.h"
-#include "qapi/qmp/types.h"
-
-typedef struct QStackEntry
-{
- QObject *value;
- void *qapi; /* sanity check that caller uses same pointer */
- QSLIST_ENTRY(QStackEntry) node;
-} QStackEntry;
-
-struct QmpOutputVisitor
-{
- Visitor visitor;
- QSLIST_HEAD(, QStackEntry) stack; /* Stack of unfinished containers */
- QObject *root; /* Root of the output visit */
- QObject **result; /* User's storage location for result */
-};
-
-#define qmp_output_add(qov, name, value) \
- qmp_output_add_obj(qov, name, QOBJECT(value))
-#define qmp_output_push(qov, value, qapi) \
- qmp_output_push_obj(qov, QOBJECT(value), qapi)
-
-static QmpOutputVisitor *to_qov(Visitor *v)
-{
- return container_of(v, QmpOutputVisitor, visitor);
-}
-
-/* Push @value onto the stack of current QObjects being built */
-static void qmp_output_push_obj(QmpOutputVisitor *qov, QObject *value,
- void *qapi)
-{
- QStackEntry *e = g_malloc0(sizeof(*e));
-
- assert(qov->root);
- assert(value);
- e->value = value;
- e->qapi = qapi;
- QSLIST_INSERT_HEAD(&qov->stack, e, node);
-}
-
-/* Pop a value off the stack of QObjects being built, and return it. */
-static QObject *qmp_output_pop(QmpOutputVisitor *qov, void *qapi)
-{
- QStackEntry *e = QSLIST_FIRST(&qov->stack);
- QObject *value;
-
- assert(e);
- assert(e->qapi == qapi);
- QSLIST_REMOVE_HEAD(&qov->stack, node);
- value = e->value;
- assert(value);
- g_free(e);
- return value;
-}
-
-/* Add @value to the current QObject being built.
- * If the stack is visiting a dictionary or list, @value is now owned
- * by that container. Otherwise, @value is now the root. */
-static void qmp_output_add_obj(QmpOutputVisitor *qov, const char *name,
- QObject *value)
-{
- QStackEntry *e = QSLIST_FIRST(&qov->stack);
- QObject *cur = e ? e->value : NULL;
-
- if (!cur) {
- /* Don't allow reuse of visitor on more than one root */
- assert(!qov->root);
- qov->root = value;
- } else {
- switch (qobject_type(cur)) {
- case QTYPE_QDICT:
- assert(name);
- qdict_put_obj(qobject_to_qdict(cur), name, value);
- break;
- case QTYPE_QLIST:
- assert(!name);
- qlist_append_obj(qobject_to_qlist(cur), value);
- break;
- default:
- g_assert_not_reached();
- }
- }
-}
-
-static void qmp_output_start_struct(Visitor *v, const char *name, void **obj,
- size_t unused, Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- QDict *dict = qdict_new();
-
- qmp_output_add(qov, name, dict);
- qmp_output_push(qov, dict, obj);
-}
-
-static void qmp_output_end_struct(Visitor *v, void **obj)
-{
- QmpOutputVisitor *qov = to_qov(v);
- QObject *value = qmp_output_pop(qov, obj);
- assert(qobject_type(value) == QTYPE_QDICT);
-}
-
-static void qmp_output_start_list(Visitor *v, const char *name,
- GenericList **listp, size_t size,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- QList *list = qlist_new();
-
- qmp_output_add(qov, name, list);
- qmp_output_push(qov, list, listp);
-}
-
-static GenericList *qmp_output_next_list(Visitor *v, GenericList *tail,
- size_t size)
-{
- return tail->next;
-}
-
-static void qmp_output_end_list(Visitor *v, void **obj)
-{
- QmpOutputVisitor *qov = to_qov(v);
- QObject *value = qmp_output_pop(qov, obj);
- assert(qobject_type(value) == QTYPE_QLIST);
-}
-
-static void qmp_output_type_int64(Visitor *v, const char *name, int64_t *obj,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- qmp_output_add(qov, name, qint_from_int(*obj));
-}
-
-static void qmp_output_type_uint64(Visitor *v, const char *name, uint64_t *obj,
- Error **errp)
-{
- /* FIXME: QMP outputs values larger than INT64_MAX as negative */
- QmpOutputVisitor *qov = to_qov(v);
- qmp_output_add(qov, name, qint_from_int(*obj));
-}
-
-static void qmp_output_type_bool(Visitor *v, const char *name, bool *obj,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- qmp_output_add(qov, name, qbool_from_bool(*obj));
-}
-
-static void qmp_output_type_str(Visitor *v, const char *name, char **obj,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- if (*obj) {
- qmp_output_add(qov, name, qstring_from_str(*obj));
- } else {
- qmp_output_add(qov, name, qstring_from_str(""));
- }
-}
-
-static void qmp_output_type_number(Visitor *v, const char *name, double *obj,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- qmp_output_add(qov, name, qfloat_from_double(*obj));
-}
-
-static void qmp_output_type_any(Visitor *v, const char *name, QObject **obj,
- Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- qobject_incref(*obj);
- qmp_output_add_obj(qov, name, *obj);
-}
-
-static void qmp_output_type_null(Visitor *v, const char *name, Error **errp)
-{
- QmpOutputVisitor *qov = to_qov(v);
- qmp_output_add_obj(qov, name, qnull());
-}
-
-/* Finish building, and return the root object.
- * The root object is never null. The caller becomes the object's
- * owner, and should use qobject_decref() when done with it. */
-static void qmp_output_complete(Visitor *v, void *opaque)
-{
- QmpOutputVisitor *qov = to_qov(v);
-
- /* A visit must have occurred, with each start paired with end. */
- assert(qov->root && QSLIST_EMPTY(&qov->stack));
- assert(opaque == qov->result);
-
- qobject_incref(qov->root);
- *qov->result = qov->root;
- qov->result = NULL;
-}
-
-static void qmp_output_free(Visitor *v)
-{
- QmpOutputVisitor *qov = to_qov(v);
- QStackEntry *e;
-
- while (!QSLIST_EMPTY(&qov->stack)) {
- e = QSLIST_FIRST(&qov->stack);
- QSLIST_REMOVE_HEAD(&qov->stack, node);
- g_free(e);
- }
-
- qobject_decref(qov->root);
- g_free(qov);
-}
-
-Visitor *qmp_output_visitor_new(QObject **result)
-{
- QmpOutputVisitor *v;
-
- v = g_malloc0(sizeof(*v));
-
- v->visitor.type = VISITOR_OUTPUT;
- v->visitor.start_struct = qmp_output_start_struct;
- v->visitor.end_struct = qmp_output_end_struct;
- v->visitor.start_list = qmp_output_start_list;
- v->visitor.next_list = qmp_output_next_list;
- v->visitor.end_list = qmp_output_end_list;
- v->visitor.type_int64 = qmp_output_type_int64;
- v->visitor.type_uint64 = qmp_output_type_uint64;
- v->visitor.type_bool = qmp_output_type_bool;
- v->visitor.type_str = qmp_output_type_str;
- v->visitor.type_number = qmp_output_type_number;
- v->visitor.type_any = qmp_output_type_any;
- v->visitor.type_null = qmp_output_type_null;
- v->visitor.complete = qmp_output_complete;
- v->visitor.free = qmp_output_free;
-
- *result = NULL;
- v->result = result;
-
- return &v->visitor;
-}
diff --git a/qapi/qobject-output-visitor.c b/qapi/qobject-output-visitor.c
new file mode 100644
index 0000000..df43cb8
--- /dev/null
+++ b/qapi/qobject-output-visitor.c
@@ -0,0 +1,254 @@
+/*
+ * Core Definitions for QAPI/QMP Command Registry
+ *
+ * Copyright (C) 2012-2016 Red Hat, Inc.
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Anthony Liguori <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/qobject-output-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "qapi/qmp/types.h"
+
+typedef struct QStackEntry {
+ QObject *value;
+ void *qapi; /* sanity check that caller uses same pointer */
+ QSLIST_ENTRY(QStackEntry) node;
+} QStackEntry;
+
+struct QObjectOutputVisitor {
+ Visitor visitor;
+ QSLIST_HEAD(, QStackEntry) stack; /* Stack of unfinished containers */
+ QObject *root; /* Root of the output visit */
+ QObject **result; /* User's storage location for result */
+};
+
+#define qobject_output_add(qov, name, value) \
+ qobject_output_add_obj(qov, name, QOBJECT(value))
+#define qobject_output_push(qov, value, qapi) \
+ qobject_output_push_obj(qov, QOBJECT(value), qapi)
+
+static QObjectOutputVisitor *to_qov(Visitor *v)
+{
+ return container_of(v, QObjectOutputVisitor, visitor);
+}
+
+/* Push @value onto the stack of current QObjects being built */
+static void qobject_output_push_obj(QObjectOutputVisitor *qov, QObject *value,
+ void *qapi)
+{
+ QStackEntry *e = g_malloc0(sizeof(*e));
+
+ assert(qov->root);
+ assert(value);
+ e->value = value;
+ e->qapi = qapi;
+ QSLIST_INSERT_HEAD(&qov->stack, e, node);
+}
+
+/* Pop a value off the stack of QObjects being built, and return it. */
+static QObject *qobject_output_pop(QObjectOutputVisitor *qov, void *qapi)
+{
+ QStackEntry *e = QSLIST_FIRST(&qov->stack);
+ QObject *value;
+
+ assert(e);
+ assert(e->qapi == qapi);
+ QSLIST_REMOVE_HEAD(&qov->stack, node);
+ value = e->value;
+ assert(value);
+ g_free(e);
+ return value;
+}
+
+/* Add @value to the current QObject being built.
+ * If the stack is visiting a dictionary or list, @value is now owned
+ * by that container. Otherwise, @value is now the root. */
+static void qobject_output_add_obj(QObjectOutputVisitor *qov, const char *name,
+ QObject *value)
+{
+ QStackEntry *e = QSLIST_FIRST(&qov->stack);
+ QObject *cur = e ? e->value : NULL;
+
+ if (!cur) {
+ /* Don't allow reuse of visitor on more than one root */
+ assert(!qov->root);
+ qov->root = value;
+ } else {
+ switch (qobject_type(cur)) {
+ case QTYPE_QDICT:
+ assert(name);
+ qdict_put_obj(qobject_to_qdict(cur), name, value);
+ break;
+ case QTYPE_QLIST:
+ assert(!name);
+ qlist_append_obj(qobject_to_qlist(cur), value);
+ break;
+ default:
+ g_assert_not_reached();
+ }
+ }
+}
+
+static void qobject_output_start_struct(Visitor *v, const char *name,
+ void **obj, size_t unused, Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ QDict *dict = qdict_new();
+
+ qobject_output_add(qov, name, dict);
+ qobject_output_push(qov, dict, obj);
+}
+
+static void qobject_output_end_struct(Visitor *v, void **obj)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ QObject *value = qobject_output_pop(qov, obj);
+ assert(qobject_type(value) == QTYPE_QDICT);
+}
+
+static void qobject_output_start_list(Visitor *v, const char *name,
+ GenericList **listp, size_t size,
+ Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ QList *list = qlist_new();
+
+ qobject_output_add(qov, name, list);
+ qobject_output_push(qov, list, listp);
+}
+
+static GenericList *qobject_output_next_list(Visitor *v, GenericList *tail,
+ size_t size)
+{
+ return tail->next;
+}
+
+static void qobject_output_end_list(Visitor *v, void **obj)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ QObject *value = qobject_output_pop(qov, obj);
+ assert(qobject_type(value) == QTYPE_QLIST);
+}
+
+static void qobject_output_type_int64(Visitor *v, const char *name,
+ int64_t *obj, Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_output_add(qov, name, qint_from_int(*obj));
+}
+
+static void qobject_output_type_uint64(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
+{
+ /* FIXME: QOBJECT outputs values larger than INT64_MAX as negative */
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_output_add(qov, name, qint_from_int(*obj));
+}
+
+static void qobject_output_type_bool(Visitor *v, const char *name, bool *obj,
+ Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_output_add(qov, name, qbool_from_bool(*obj));
+}
+
+static void qobject_output_type_str(Visitor *v, const char *name, char **obj,
+ Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ if (*obj) {
+ qobject_output_add(qov, name, qstring_from_str(*obj));
+ } else {
+ qobject_output_add(qov, name, qstring_from_str(""));
+ }
+}
+
+static void qobject_output_type_number(Visitor *v, const char *name,
+ double *obj, Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_output_add(qov, name, qfloat_from_double(*obj));
+}
+
+static void qobject_output_type_any(Visitor *v, const char *name,
+ QObject **obj, Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_incref(*obj);
+ qobject_output_add_obj(qov, name, *obj);
+}
+
+static void qobject_output_type_null(Visitor *v, const char *name, Error **errp)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ qobject_output_add_obj(qov, name, qnull());
+}
+
+/* Finish building, and return the root object.
+ * The root object is never null. The caller becomes the object's
+ * owner, and should use qobject_decref() when done with it. */
+static void qobject_output_complete(Visitor *v, void *opaque)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+
+ /* A visit must have occurred, with each start paired with end. */
+ assert(qov->root && QSLIST_EMPTY(&qov->stack));
+ assert(opaque == qov->result);
+
+ qobject_incref(qov->root);
+ *qov->result = qov->root;
+ qov->result = NULL;
+}
+
+static void qobject_output_free(Visitor *v)
+{
+ QObjectOutputVisitor *qov = to_qov(v);
+ QStackEntry *e;
+
+ while (!QSLIST_EMPTY(&qov->stack)) {
+ e = QSLIST_FIRST(&qov->stack);
+ QSLIST_REMOVE_HEAD(&qov->stack, node);
+ g_free(e);
+ }
+
+ qobject_decref(qov->root);
+ g_free(qov);
+}
+
+Visitor *qobject_output_visitor_new(QObject **result)
+{
+ QObjectOutputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->visitor.type = VISITOR_OUTPUT;
+ v->visitor.start_struct = qobject_output_start_struct;
+ v->visitor.end_struct = qobject_output_end_struct;
+ v->visitor.start_list = qobject_output_start_list;
+ v->visitor.next_list = qobject_output_next_list;
+ v->visitor.end_list = qobject_output_end_list;
+ v->visitor.type_int64 = qobject_output_type_int64;
+ v->visitor.type_uint64 = qobject_output_type_uint64;
+ v->visitor.type_bool = qobject_output_type_bool;
+ v->visitor.type_str = qobject_output_type_str;
+ v->visitor.type_number = qobject_output_type_number;
+ v->visitor.type_any = qobject_output_type_any;
+ v->visitor.type_null = qobject_output_type_null;
+ v->visitor.complete = qobject_output_complete;
+ v->visitor.free = qobject_output_free;
+
+ *result = NULL;
+ v->result = result;
+
+ return &v->visitor;
+}
diff --git a/qemu-img.c b/qemu-img.c
index ceffefe..851422a 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -25,7 +25,7 @@
#include "qemu-version.h"
#include "qapi/error.h"
#include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qjson.h"
#include "qemu/cutils.h"
@@ -500,7 +500,7 @@ static void dump_json_image_check(ImageCheck *check, bool quiet)
{
QString *str;
QObject *obj;
- Visitor *v = qmp_output_visitor_new(&obj);
+ Visitor *v = qobject_output_visitor_new(&obj);
visit_type_ImageCheck(v, NULL, &check, &error_abort);
visit_complete(v, &obj);
@@ -2193,7 +2193,7 @@ static void dump_json_image_info_list(ImageInfoList *list)
{
QString *str;
QObject *obj;
- Visitor *v = qmp_output_visitor_new(&obj);
+ Visitor *v = qobject_output_visitor_new(&obj);
visit_type_ImageInfoList(v, NULL, &list, &error_abort);
visit_complete(v, &obj);
@@ -2209,7 +2209,7 @@ static void dump_json_image_info(ImageInfo *info)
{
QString *str;
QObject *obj;
- Visitor *v = qmp_output_visitor_new(&obj);
+ Visitor *v = qobject_output_visitor_new(&obj);
visit_type_ImageInfo(v, NULL, &info, &error_abort);
visit_complete(v, &obj);
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index bf59846..ded4d84 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -3,7 +3,7 @@
#include "qom/object_interfaces.h"
#include "qemu/module.h"
#include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/opts-visitor.h"
void user_creatable_complete(Object *obj, Error **errp)
diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
index 81959e0..447e4a0 100644
--- a/qom/qom-qobject.c
+++ b/qom/qom-qobject.c
@@ -16,7 +16,7 @@
#include "qom/qom-qobject.h"
#include "qapi/visitor.h"
#include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
void object_property_set_qobject(Object *obj, QObject *value,
const char *name, Error **errp)
@@ -35,7 +35,7 @@ QObject *object_property_get_qobject(Object *obj, const char *name,
Error *local_err = NULL;
Visitor *v;
- v = qmp_output_visitor_new(&ret);
+ v = qobject_output_visitor_new(&ret);
object_property_get(obj, v, name, &local_err);
if (!local_err) {
visit_complete(v, &ret);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 04158a3..09e8467 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -68,7 +68,7 @@ static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out,
Error *err = NULL;
Visitor *v;
- v = qmp_output_visitor_new(ret_out);
+ v = qobject_output_visitor_new(ret_out);
visit_type_%(c_name)s(v, "unused", &ret_in, &err);
if (!err) {
visit_complete(v, ret_out);
@@ -293,7 +293,7 @@ fdef.write(mcgen('''
#include "qapi/qmp/types.h"
#include "qapi/qmp/dispatch.h"
#include "qapi/visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/qobject-input-visitor.h"
#include "qapi/dealloc-visitor.h"
#include "%(prefix)sqapi-types.h"
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 38d8211..f4eb7f8 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -98,7 +98,7 @@ def gen_event_send(name, arg_type, boxed):
if arg_type and not arg_type.is_empty():
ret += mcgen('''
- v = qmp_output_visitor_new(&obj);
+ v = qobject_output_visitor_new(&obj);
''')
if not arg_type.is_implicit():
ret += mcgen('''
@@ -209,7 +209,7 @@ fdef.write(mcgen('''
#include "qemu-common.h"
#include "%(prefix)sqapi-event.h"
#include "%(prefix)sqapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/qmp-event.h"
''',
diff --git a/tests/.gitignore b/tests/.gitignore
index bc076eb..8ee11c8 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -61,7 +61,7 @@ test-qobject-input-strict
test-qobject-input-visitor
test-qmp-introspect.[ch]
test-qmp-marshal.c
-test-qmp-output-visitor
+test-qobject-output-visitor
test-rcu-list
test-replication
test-rfifolock
diff --git a/tests/Makefile.include b/tests/Makefile.include
index eb2c8b0..605f276 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -20,8 +20,8 @@ check-unit-y += tests/check-qnull$(EXESUF)
gcov-files-check-qnull-y = qobject/qnull.c
check-unit-y += tests/check-qjson$(EXESUF)
gcov-files-check-qjson-y = qobject/qjson.c
-check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
-gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
+check-unit-y += tests/test-qobject-output-visitor$(EXESUF)
+gcov-files-test-qobject-output-visitor-y = qapi/qobject-output-visitor.c
check-unit-y += tests/test-clone-visitor$(EXESUF)
gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
@@ -439,7 +439,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \
tests/check-qjson.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
- tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
+ tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \
tests/test-clone-visitor.o \
tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
tests/test-qmp-commands.o tests/test-visitor-serialization.o \
@@ -542,7 +542,7 @@ $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-int
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y)
tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)
tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
-tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
+tests/test-qobject-output-visitor$(EXESUF): tests/test-qobject-output-visitor.o $(test-qapi-obj-y)
tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y)
tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
diff --git a/tests/check-qnull.c b/tests/check-qnull.c
index eeb803a..b50bb8a 100644
--- a/tests/check-qnull.c
+++ b/tests/check-qnull.c
@@ -11,7 +11,7 @@
#include "qapi/qmp/qobject.h"
#include "qemu-common.h"
#include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/error.h"
/*
@@ -52,7 +52,7 @@ static void qnull_visit_test(void)
visit_type_null(v, NULL, &error_abort);
visit_free(v);
- v = qmp_output_visitor_new(&obj);
+ v = qobject_output_visitor_new(&obj);
visit_type_null(v, NULL, &error_abort);
visit_complete(v, &obj);
g_assert(obj == &qnull_);
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qobject-output-visitor.c
similarity index 99%
rename from tests/test-qmp-output-visitor.c
rename to tests/test-qobject-output-visitor.c
index 513d71f..4e2d79c 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qobject-output-visitor.c
@@ -1,5 +1,5 @@
/*
- * QMP Output Visitor unit-tests.
+ * QObject Output Visitor unit-tests.
*
* Copyright (C) 2011-2016 Red Hat Inc.
*
@@ -14,7 +14,7 @@
#include "qemu-common.h"
#include "qapi/error.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
@@ -28,7 +28,7 @@ typedef struct TestOutputVisitorData {
static void visitor_output_setup(TestOutputVisitorData *data,
const void *unused)
{
- data->ov = qmp_output_visitor_new(&data->obj);
+ data->ov = qobject_output_visitor_new(&data->obj);
g_assert(data->ov);
}
diff --git a/tests/test-string-output-visitor.c b/tests/test-string-output-visitor.c
index 444844a..e736db3 100644
--- a/tests/test-string-output-visitor.c
+++ b/tests/test-string-output-visitor.c
@@ -4,7 +4,7 @@
* Copyright (C) 2012 Red Hat Inc.
*
* Authors:
- * Paolo Bonzini <pbonzini@redhat.com> (based on test-qmp-output-visitor)
+ * Paolo Bonzini <pbonzini@redhat.com> (based on test-qobject-output-visitor)
*
* This work is licensed under the terms of the GNU GPL, version 2 or later.
* See the COPYING file in the top-level directory.
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 51df428..66b2b1c 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -21,7 +21,7 @@
#include "qapi/qmp/types.h"
#include "qapi/qmp/qjson.h"
#include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi/string-input-visitor.h"
#include "qapi/string-output-visitor.h"
#include "qapi-types.h"
@@ -1022,7 +1022,7 @@ static void qmp_serialize(void *native_in, void **datap,
{
QmpSerializeData *d = g_malloc0(sizeof(*d));
- d->qov = qmp_output_visitor_new(&d->obj);
+ d->qov = qobject_output_visitor_new(&d->obj);
visit(d->qov, &native_in, errp);
*datap = d;
}
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 9e30a21..4cef549 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -22,7 +22,7 @@
#include "qemu/sockets.h"
#include "qemu/main-loop.h"
#include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
#include "qapi-visit.h"
#include "qemu/cutils.h"
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (4 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 17:43 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor Daniel P. Berrange
` (16 subsequent siblings)
22 siblings, 2 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The input_visitor_test_add() method was accepting an instance
of 'TestInputVisitorData' and passing it as the 'user_data'
parameter to test functions. The main 'TestInputVisitorData'
instance that was actually used, was meanwhile being allocated
automatically by the test framework fixture setup.
The 'user_data' parameter is going to be needed for tests
added in later patches, so getting rid of the current mistaken
usage now allows this.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
1 file changed, 32 insertions(+), 44 deletions(-)
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 0e65e63..26c5012 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -747,10 +747,11 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
}
static void input_visitor_test_add(const char *testpath,
- TestInputVisitorData *data,
- void (*test_func)(TestInputVisitorData *data, const void *user_data))
+ const void *user_data,
+ void (*test_func)(TestInputVisitorData *data,
+ const void *user_data))
{
- g_test_add(testpath, TestInputVisitorData, data, NULL, test_func,
+ g_test_add(testpath, TestInputVisitorData, user_data, NULL, test_func,
visitor_input_teardown);
}
@@ -833,77 +834,64 @@ static void test_visitor_in_wrong_type(TestInputVisitorData *data,
int main(int argc, char **argv)
{
- TestInputVisitorData in_visitor_data;
-
g_test_init(&argc, &argv, NULL);
input_visitor_test_add("/visitor/input/int",
- &in_visitor_data, test_visitor_in_int);
+ NULL, test_visitor_in_int);
input_visitor_test_add("/visitor/input/int_overflow",
- &in_visitor_data, test_visitor_in_int_overflow);
+ NULL, test_visitor_in_int_overflow);
input_visitor_test_add("/visitor/input/bool",
- &in_visitor_data, test_visitor_in_bool);
+ NULL, test_visitor_in_bool);
input_visitor_test_add("/visitor/input/number",
- &in_visitor_data, test_visitor_in_number);
+ NULL, test_visitor_in_number);
input_visitor_test_add("/visitor/input/string",
- &in_visitor_data, test_visitor_in_string);
+ NULL, test_visitor_in_string);
input_visitor_test_add("/visitor/input/enum",
- &in_visitor_data, test_visitor_in_enum);
+ NULL, test_visitor_in_enum);
input_visitor_test_add("/visitor/input/struct",
- &in_visitor_data, test_visitor_in_struct);
+ NULL, test_visitor_in_struct);
input_visitor_test_add("/visitor/input/struct-nested",
- &in_visitor_data, test_visitor_in_struct_nested);
+ NULL, test_visitor_in_struct_nested);
input_visitor_test_add("/visitor/input/list",
- &in_visitor_data, test_visitor_in_list);
+ NULL, test_visitor_in_list);
input_visitor_test_add("/visitor/input/any",
- &in_visitor_data, test_visitor_in_any);
+ NULL, test_visitor_in_any);
input_visitor_test_add("/visitor/input/null",
- &in_visitor_data, test_visitor_in_null);
+ NULL, test_visitor_in_null);
input_visitor_test_add("/visitor/input/union-flat",
- &in_visitor_data, test_visitor_in_union_flat);
+ NULL, test_visitor_in_union_flat);
input_visitor_test_add("/visitor/input/alternate",
- &in_visitor_data, test_visitor_in_alternate);
+ NULL, test_visitor_in_alternate);
input_visitor_test_add("/visitor/input/errors",
- &in_visitor_data, test_visitor_in_errors);
+ NULL, test_visitor_in_errors);
input_visitor_test_add("/visitor/input/wrong-type",
- &in_visitor_data, test_visitor_in_wrong_type);
+ NULL, test_visitor_in_wrong_type);
input_visitor_test_add("/visitor/input/alternate-number",
- &in_visitor_data, test_visitor_in_alternate_number);
+ NULL, test_visitor_in_alternate_number);
input_visitor_test_add("/visitor/input/native_list/int",
- &in_visitor_data,
- test_visitor_in_native_list_int);
+ NULL, test_visitor_in_native_list_int);
input_visitor_test_add("/visitor/input/native_list/int8",
- &in_visitor_data,
- test_visitor_in_native_list_int8);
+ NULL, test_visitor_in_native_list_int8);
input_visitor_test_add("/visitor/input/native_list/int16",
- &in_visitor_data,
- test_visitor_in_native_list_int16);
+ NULL, test_visitor_in_native_list_int16);
input_visitor_test_add("/visitor/input/native_list/int32",
- &in_visitor_data,
- test_visitor_in_native_list_int32);
+ NULL, test_visitor_in_native_list_int32);
input_visitor_test_add("/visitor/input/native_list/int64",
- &in_visitor_data,
- test_visitor_in_native_list_int64);
+ NULL, test_visitor_in_native_list_int64);
input_visitor_test_add("/visitor/input/native_list/uint8",
- &in_visitor_data,
- test_visitor_in_native_list_uint8);
+ NULL, test_visitor_in_native_list_uint8);
input_visitor_test_add("/visitor/input/native_list/uint16",
- &in_visitor_data,
- test_visitor_in_native_list_uint16);
+ NULL, test_visitor_in_native_list_uint16);
input_visitor_test_add("/visitor/input/native_list/uint32",
- &in_visitor_data,
- test_visitor_in_native_list_uint32);
+ NULL, test_visitor_in_native_list_uint32);
input_visitor_test_add("/visitor/input/native_list/uint64",
- &in_visitor_data,
- test_visitor_in_native_list_uint64);
+ NULL, test_visitor_in_native_list_uint64);
input_visitor_test_add("/visitor/input/native_list/bool",
- &in_visitor_data, test_visitor_in_native_list_bool);
+ NULL, test_visitor_in_native_list_bool);
input_visitor_test_add("/visitor/input/native_list/str",
- &in_visitor_data,
- test_visitor_in_native_list_string);
+ NULL, test_visitor_in_native_list_string);
input_visitor_test_add("/visitor/input/native_list/number",
- &in_visitor_data,
- test_visitor_in_native_list_number);
+ NULL, test_visitor_in_native_list_number);
g_test_run();
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (5 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 17:48 ` Eric Blake
2016-10-11 16:20 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
` (15 subsequent siblings)
22 siblings, 2 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Currently the QObjectInputVisitor assumes that all scalar
values are directly represented as the final types declared
by the thing being visited. ie it assumes an 'int' is using
QInt, and a 'bool' is using QBool, etc. This is good when
QObjectInputVisitor is fed a QObject that came from a JSON
document on the QMP monitor, as it will strictly validate
correctness.
To allow QObjectInputVisitor to be reused for visiting
a QObject originating from QemuOpts, an alternative mode
is needed where all the scalars types are represented as
QString and converted on the fly to the final desired
type.
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 32 +++++-
qapi/qobject-input-visitor.c | 132 ++++++++++++++++++++++++
tests/test-qobject-input-visitor.c | 194 ++++++++++++++++++++++++++++++++++-
3 files changed, 350 insertions(+), 8 deletions(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index cde328d..5022297 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -19,12 +19,36 @@
typedef struct QObjectInputVisitor QObjectInputVisitor;
-/*
- * Return a new input visitor that converts a QObject to a QAPI object.
+/**
+ * Create a new input visitor that converts @obj to a QAPI object.
+ *
+ * Any scalar values in the @obj input data structure should be in the
+ * required type already. i.e. if visiting a bool, the value should
+ * already be a QBool instance.
*
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation. If @strict is false
+ * then extra dict keys are silently ignored.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
*/
Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
+/**
+ * Create a new input visitor that converts @obj to a QAPI object.
+ *
+ * Any scalar values in the @obj input data structure should always be
+ * represented as strings. i.e. if visiting a boolean, the value should
+ * be a QString whose contents represent a valid boolean.
+ *
+ * The visitor always operates in strict mode, requiring all dict keys
+ * to be consumed during visitation. An error will be reported if this
+ * does not happen.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
+ */
+Visitor *qobject_input_visitor_new_autocast(QObject *obj);
+
#endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 5ff3db3..cf41df6 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -20,6 +20,7 @@
#include "qemu-common.h"
#include "qapi/qmp/types.h"
#include "qapi/qmp/qerror.h"
+#include "qemu/cutils.h"
#define QIV_STACK_SIZE 1024
@@ -263,6 +264,28 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
*obj = qint_get_int(qint);
}
+
+static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
+ int64_t *obj, Error **errp)
+{
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
+ int64_t ret;
+
+ if (!qstr || !qstr->string) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "string");
+ return;
+ }
+
+ if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
+ return;
+ }
+ *obj = ret;
+}
+
static void qobject_input_type_uint64(Visitor *v, const char *name,
uint64_t *obj, Error **errp)
{
@@ -279,6 +302,27 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
*obj = qint_get_int(qint);
}
+static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
+{
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
+ unsigned long long ret;
+
+ if (!qstr || !qstr->string) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "string");
+ return;
+ }
+
+ if (parse_uint_full(qstr->string, &ret, 0) < 0) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
+ return;
+ }
+ *obj = ret;
+}
+
static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
Error **errp)
{
@@ -294,6 +338,22 @@ static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
*obj = qbool_get_bool(qbool);
}
+static void qobject_input_type_bool_autocast(Visitor *v, const char *name,
+ bool *obj, Error **errp)
+{
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
+
+ if (!qstr || !qstr->string) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "string");
+ return;
+ }
+
+ parse_option_bool(name, qstr->string, obj, errp);
+}
+
static void qobject_input_type_str(Visitor *v, const char *name, char **obj,
Error **errp)
{
@@ -335,6 +395,30 @@ static void qobject_input_type_number(Visitor *v, const char *name, double *obj,
"number");
}
+static void qobject_input_type_number_autocast(Visitor *v, const char *name,
+ double *obj, Error **errp)
+{
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
+ char *endp;
+
+ if (!qstr || !qstr->string) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "string");
+ return;
+ }
+
+ errno = 0;
+ *obj = strtod(qstr->string, &endp);
+ if (errno == 0 && endp != qstr->string && *endp == '\0') {
+ return;
+ }
+
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "number");
+}
+
static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj,
Error **errp)
{
@@ -356,6 +440,22 @@ static void qobject_input_type_null(Visitor *v, const char *name, Error **errp)
}
}
+static void qobject_input_type_size_autocast(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
+{
+ QObjectInputVisitor *qiv = to_qiv(v);
+ QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
+
+ if (!qstr || !qstr->string) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "string");
+ return;
+ }
+
+ parse_option_size(name, qstr->string, obj, errp);
+}
+
static void qobject_input_optional(Visitor *v, const char *name, bool *present)
{
QObjectInputVisitor *qiv = to_qiv(v);
@@ -413,3 +513,35 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
return &v->visitor;
}
+
+Visitor *qobject_input_visitor_new_autocast(QObject *obj)
+{
+ QObjectInputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->visitor.type = VISITOR_INPUT;
+ v->visitor.start_struct = qobject_input_start_struct;
+ v->visitor.check_struct = qobject_input_check_struct;
+ v->visitor.end_struct = qobject_input_pop;
+ v->visitor.start_list = qobject_input_start_list;
+ v->visitor.next_list = qobject_input_next_list;
+ v->visitor.end_list = qobject_input_pop;
+ v->visitor.start_alternate = qobject_input_start_alternate;
+ v->visitor.type_int64 = qobject_input_type_int64_autocast;
+ v->visitor.type_uint64 = qobject_input_type_uint64_autocast;
+ v->visitor.type_bool = qobject_input_type_bool_autocast;
+ v->visitor.type_str = qobject_input_type_str;
+ v->visitor.type_number = qobject_input_type_number_autocast;
+ v->visitor.type_any = qobject_input_type_any;
+ v->visitor.type_null = qobject_input_type_null;
+ v->visitor.type_size = qobject_input_type_size_autocast;
+ v->visitor.optional = qobject_input_optional;
+ v->visitor.free = qobject_input_free;
+ v->strict = true;
+
+ v->root = obj;
+ qobject_incref(obj);
+
+ return &v->visitor;
+}
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 26c5012..d5b8044 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -41,6 +41,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
function so that the JSON string used by the tests are kept in the test
functions (and not in main()). */
static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
+ bool strict, bool autocast,
const char *json_string,
va_list *ap)
{
@@ -49,11 +50,31 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
- data->qiv = qobject_input_visitor_new(data->obj, false);
+ if (autocast) {
+ assert(strict);
+ data->qiv = qobject_input_visitor_new_autocast(data->obj);
+ } else {
+ data->qiv = qobject_input_visitor_new(data->obj, strict);
+ }
g_assert(data->qiv);
return data->qiv;
}
+static GCC_FMT_ATTR(4, 5)
+Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
+ bool strict, bool autocast,
+ const char *json_string, ...)
+{
+ Visitor *v;
+ va_list ap;
+
+ va_start(ap, json_string);
+ v = visitor_input_test_init_internal(data, strict, autocast,
+ json_string, &ap);
+ va_end(ap);
+ return v;
+}
+
static GCC_FMT_ATTR(2, 3)
Visitor *visitor_input_test_init(TestInputVisitorData *data,
const char *json_string, ...)
@@ -62,7 +83,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- v = visitor_input_test_init_internal(data, json_string, &ap);
+ v = visitor_input_test_init_internal(data, true, false,
+ json_string, &ap);
va_end(ap);
return v;
}
@@ -77,7 +99,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- return visitor_input_test_init_internal(data, json_string, NULL);
+ return visitor_input_test_init_internal(data, true, false,
+ json_string, NULL);
}
static void test_visitor_in_int(TestInputVisitorData *data,
@@ -109,6 +132,45 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
error_free_or_abort(&err);
}
+static void test_visitor_in_int_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ int64_t res = 0, value = -42;
+ Error *err = NULL;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true,
+ "%" PRId64, value);
+ visit_type_int(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
+static void test_visitor_in_int_str_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ int64_t res = 0, value = -42;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true,
+ "\"-42\"");
+
+ visit_type_int(v, NULL, &res, &error_abort);
+ g_assert_cmpint(res, ==, value);
+}
+
+static void test_visitor_in_int_str_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ int64_t res = 0;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"-42\"");
+
+ visit_type_int(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
static void test_visitor_in_bool(TestInputVisitorData *data,
const void *unused)
{
@@ -121,6 +183,44 @@ static void test_visitor_in_bool(TestInputVisitorData *data,
g_assert_cmpint(res, ==, true);
}
+static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ bool res = false;
+ Error *err = NULL;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, "true");
+
+ visit_type_bool(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
+static void test_visitor_in_bool_str_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ bool res = false;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, "\"yes\"");
+
+ visit_type_bool(v, NULL, &res, &error_abort);
+ g_assert_cmpint(res, ==, true);
+}
+
+static void test_visitor_in_bool_str_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ bool res = false;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"true\"");
+
+ visit_type_bool(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
static void test_visitor_in_number(TestInputVisitorData *data,
const void *unused)
{
@@ -133,6 +233,69 @@ static void test_visitor_in_number(TestInputVisitorData *data,
g_assert_cmpfloat(res, ==, value);
}
+static void test_visitor_in_number_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ double res = 0, value = 3.14;
+ Error *err = NULL;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, "%f", value);
+
+ visit_type_number(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
+static void test_visitor_in_number_str_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ double res = 0, value = 3.14;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, "\"3.14\"");
+
+ visit_type_number(v, NULL, &res, &error_abort);
+ g_assert_cmpfloat(res, ==, value);
+}
+
+static void test_visitor_in_number_str_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ double res = 0;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"3.14\"");
+
+ visit_type_number(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
+static void test_visitor_in_size_str_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ uint64_t res, value = 500 * 1024 * 1024;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, "\"500M\"");
+
+ visit_type_size(v, NULL, &res, &error_abort);
+ g_assert_cmpfloat(res, ==, value);
+}
+
+static void test_visitor_in_size_str_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ uint64_t res = 0;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"500M\"");
+
+ visit_type_size(v, NULL, &res, &err);
+ error_free_or_abort(&err);
+}
+
static void test_visitor_in_string(TestInputVisitorData *data,
const void *unused)
{
@@ -289,7 +452,8 @@ static void test_visitor_in_null(TestInputVisitorData *data,
* when input is not null.
*/
- v = visitor_input_test_init(data, "{ 'a': null, 'b': '' }");
+ v = visitor_input_test_init_full(data, false, false,
+ "{ 'a': null, 'b': '' }");
visit_start_struct(v, NULL, NULL, 0, &error_abort);
visit_type_null(v, "a", &error_abort);
visit_type_str(v, "a", &tmp, &err);
@@ -840,10 +1004,32 @@ int main(int argc, char **argv)
NULL, test_visitor_in_int);
input_visitor_test_add("/visitor/input/int_overflow",
NULL, test_visitor_in_int_overflow);
+ input_visitor_test_add("/visitor/input/int_autocast",
+ NULL, test_visitor_in_int_autocast);
+ input_visitor_test_add("/visitor/input/int_str_autocast",
+ NULL, test_visitor_in_int_str_autocast);
+ input_visitor_test_add("/visitor/input/int_str_noautocast",
+ NULL, test_visitor_in_int_str_noautocast);
input_visitor_test_add("/visitor/input/bool",
NULL, test_visitor_in_bool);
+ input_visitor_test_add("/visitor/input/bool_autocast",
+ NULL, test_visitor_in_bool_autocast);
+ input_visitor_test_add("/visitor/input/bool_str_autocast",
+ NULL, test_visitor_in_bool_str_autocast);
+ input_visitor_test_add("/visitor/input/bool_str_noautocast",
+ NULL, test_visitor_in_bool_str_noautocast);
input_visitor_test_add("/visitor/input/number",
NULL, test_visitor_in_number);
+ input_visitor_test_add("/visitor/input/number_autocast",
+ NULL, test_visitor_in_number_autocast);
+ input_visitor_test_add("/visitor/input/number_str_autocast",
+ NULL, test_visitor_in_number_str_autocast);
+ input_visitor_test_add("/visitor/input/number_str_noautocast",
+ NULL, test_visitor_in_number_str_noautocast);
+ input_visitor_test_add("/visitor/input/size_str_autocast",
+ NULL, test_visitor_in_size_str_autocast);
+ input_visitor_test_add("/visitor/input/size_str_noautocast",
+ NULL, test_visitor_in_size_str_noautocast);
input_visitor_test_add("/visitor/input/string",
NULL, test_visitor_in_string);
input_visitor_test_add("/visitor/input/enum",
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (6 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 17:55 ` Eric Blake
` (2 more replies)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists Daniel P. Berrange
` (14 subsequent siblings)
22 siblings, 3 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Instead of requiring all callers to go through the mutli-step
process of turning QemuOpts into a suitable QObject for visiting,
add a new constructor that encapsulates this logic. This will
allow QObjectInputVisitor to be a drop-in replacement for the
existing OptsVisitor with minimal code changes for callers.
NB, at this point it is only supporting opts syntax which
explicitly matches the QAPI schema structure, so is not yet
a true drop-in replacement for OptsVisitor. The patches that
follow will add the special cases requird for full backwards
compatibility with OptsVisitor.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 15 +++++++++++++++
include/qemu/option.h | 2 +-
qapi/qobject-input-visitor.c | 25 +++++++++++++++++++++++++
util/qemu-option.c | 2 +-
4 files changed, 42 insertions(+), 2 deletions(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index 5022297..f134d90 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -51,4 +51,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
*/
Visitor *qobject_input_visitor_new_autocast(QObject *obj);
+
+/**
+ * Create a new input visitor that converts @opts to a QAPI object.
+ *
+ * The QemuOpts will be converted into a QObject using the
+ * qdict_crumple() method to automatically create structs
+ * and lists. The resulting QDict will then be passed to the
+ * qobject_input_visitor_new_autocast() method. See the docs
+ * of that method for further details on processing behaviour.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
+ */
+Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts, Error **errp);
+
#endif
diff --git a/include/qemu/option.h b/include/qemu/option.h
index 2a5266f..29f3f18 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -125,7 +125,7 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
int permit_abbrev);
QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
Error **errp);
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index cf41df6..d9269c9 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -545,3 +545,28 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
return &v->visitor;
}
+
+
+Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
+ Error **errp)
+{
+ QDict *pdict;
+ QObject *pobj = NULL;
+ Visitor *v = NULL;
+
+ pdict = qemu_opts_to_qdict(opts, NULL);
+ if (!pdict) {
+ goto cleanup;
+ }
+
+ pobj = qdict_crumple(pdict, true, errp);
+ if (!pobj) {
+ goto cleanup;
+ }
+
+ v = qobject_input_visitor_new_autocast(pobj);
+ cleanup:
+ qobject_decref(pobj);
+ QDECREF(pdict);
+ return v;
+}
diff --git a/util/qemu-option.c b/util/qemu-option.c
index 41b356c..418cbb6 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -1058,7 +1058,7 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
* TODO We'll want to use types appropriate for opt->desc->type, but
* this is enough for now.
*/
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict)
{
QemuOpt *opt;
QObject *val;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (7 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 17:59 ` Eric Blake
2016-10-12 9:18 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
` (13 subsequent siblings)
22 siblings, 2 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
When converting QemuOpts to a QObject, there is no information
about compound types available, so when visiting a list, the
corresponding QObject is not guaranteed to be a QList. We
therefore need to be able to auto-create a single element QList
from whatever type we find.
This mode should only be enabled if you have compatibility
requirements for
-arg foo=hello,foo=world
to be treated as equivalent to the preferred syntax:
-arg foo.0=hello,foo.1=world
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 20 +++++++-
qapi/qobject-input-visitor.c | 27 +++++++++--
tests/test-qobject-input-visitor.c | 88 +++++++++++++++++++++++++++++++-----
3 files changed, 117 insertions(+), 18 deletions(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index f134d90..1809f48 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -42,6 +42,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
* represented as strings. i.e. if visiting a boolean, the value should
* be a QString whose contents represent a valid boolean.
*
+ * If @autocreate_list is true, then as an alternative to a normal QList,
+ * list values can be stored as a QString or QDict instead, which will
+ * be interpreted as representing single element lists. This should only
+ * by used if compatibility is required with the OptsVisitor which allowed
+ * repeated keys, without list indexes, to represent lists. e.g. set this
+ * to true if you have compatibility requirements for
+ *
+ * -arg foo=hello,foo=world
+ *
+ * to be treated as equivalent to the preferred syntax:
+ *
+ * -arg foo.0=hello,foo.1=world
+ *
* The visitor always operates in strict mode, requiring all dict keys
* to be consumed during visitation. An error will be reported if this
* does not happen.
@@ -49,7 +62,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
* The returned input visitor should be released by calling
* visit_free() when no longer required.
*/
-Visitor *qobject_input_visitor_new_autocast(QObject *obj);
+Visitor *qobject_input_visitor_new_autocast(QObject *obj,
+ bool autocreate_list);
/**
@@ -64,6 +78,8 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj);
* The returned input visitor should be released by calling
* visit_free() when no longer required.
*/
-Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts, Error **errp);
+Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
+ bool autocreate_list,
+ Error **errp);
#endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index d9269c9..d88e9f9 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -48,6 +48,10 @@ struct QObjectInputVisitor
/* True to reject parse in visit_end_struct() if unvisited keys remain. */
bool strict;
+
+ /* Whether we can auto-create single element lists when
+ * encountering a non-QList type */
+ bool autocreate_list;
};
static QObjectInputVisitor *to_qiv(Visitor *v)
@@ -108,6 +112,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
assert(obj);
tos->obj = obj;
tos->qapi = qapi;
+ qobject_incref(obj);
if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
h = g_hash_table_new(g_str_hash, g_str_equal);
@@ -147,6 +152,7 @@ static void qobject_input_stack_object_free(StackObject *tos)
if (tos->h) {
g_hash_table_unref(tos->h);
}
+ qobject_decref(tos->obj);
g_free(tos);
}
@@ -197,7 +203,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
QObject *qobj = qobject_input_get_object(qiv, name, true);
const QListEntry *entry;
- if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
+ if (!qobj || (!qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST)) {
if (list) {
*list = NULL;
}
@@ -206,7 +212,16 @@ static void qobject_input_start_list(Visitor *v, const char *name,
return;
}
- entry = qobject_input_push(qiv, qobj, list, errp);
+ if (qobject_type(qobj) != QTYPE_QLIST) {
+ QList *tmplist = qlist_new();
+ qlist_append_obj(tmplist, qobj);
+ qobject_incref(qobj);
+ entry = qobject_input_push(qiv, QOBJECT(tmplist), list, errp);
+ QDECREF(tmplist);
+ } else {
+ entry = qobject_input_push(qiv, qobj, list, errp);
+ }
+
if (list) {
if (entry) {
*list = g_malloc0(size);
@@ -514,7 +529,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
return &v->visitor;
}
-Visitor *qobject_input_visitor_new_autocast(QObject *obj)
+Visitor *qobject_input_visitor_new_autocast(QObject *obj,
+ bool autocreate_list)
{
QObjectInputVisitor *v;
@@ -539,6 +555,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
v->visitor.optional = qobject_input_optional;
v->visitor.free = qobject_input_free;
v->strict = true;
+ v->autocreate_list = autocreate_list;
v->root = obj;
qobject_incref(obj);
@@ -548,6 +565,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
+ bool autocreate_list,
Error **errp)
{
QDict *pdict;
@@ -564,7 +582,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
goto cleanup;
}
- v = qobject_input_visitor_new_autocast(pobj);
+ v = qobject_input_visitor_new_autocast(pobj,
+ autocreate_list);
cleanup:
qobject_decref(pobj);
QDECREF(pdict);
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index d5b8044..c227565 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -42,6 +42,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
functions (and not in main()). */
static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
bool strict, bool autocast,
+ bool autocreate_list,
const char *json_string,
va_list *ap)
{
@@ -52,17 +53,20 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
if (autocast) {
assert(strict);
- data->qiv = qobject_input_visitor_new_autocast(data->obj);
+ data->qiv = qobject_input_visitor_new_autocast(data->obj,
+ autocreate_list);
} else {
+ assert(!autocreate_list);
data->qiv = qobject_input_visitor_new(data->obj, strict);
}
g_assert(data->qiv);
return data->qiv;
}
-static GCC_FMT_ATTR(4, 5)
+static GCC_FMT_ATTR(5, 6)
Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
bool strict, bool autocast,
+ bool autocreate_list,
const char *json_string, ...)
{
Visitor *v;
@@ -70,6 +74,7 @@ Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
va_start(ap, json_string);
v = visitor_input_test_init_internal(data, strict, autocast,
+ autocreate_list,
json_string, &ap);
va_end(ap);
return v;
@@ -83,7 +88,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- v = visitor_input_test_init_internal(data, true, false,
+ v = visitor_input_test_init_internal(data, true, false, false,
json_string, &ap);
va_end(ap);
return v;
@@ -99,7 +104,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- return visitor_input_test_init_internal(data, true, false,
+ return visitor_input_test_init_internal(data, true, false, false,
json_string, NULL);
}
@@ -139,7 +144,7 @@ static void test_visitor_in_int_autocast(TestInputVisitorData *data,
Error *err = NULL;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true,
+ v = visitor_input_test_init_full(data, true, true, false,
"%" PRId64, value);
visit_type_int(v, NULL, &res, &err);
error_free_or_abort(&err);
@@ -151,7 +156,7 @@ static void test_visitor_in_int_str_autocast(TestInputVisitorData *data,
int64_t res = 0, value = -42;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true,
+ v = visitor_input_test_init_full(data, true, true, false,
"\"-42\"");
visit_type_int(v, NULL, &res, &error_abort);
@@ -190,7 +195,7 @@ static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
Error *err = NULL;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true, "true");
+ v = visitor_input_test_init_full(data, true, true, false, "true");
visit_type_bool(v, NULL, &res, &err);
error_free_or_abort(&err);
@@ -202,7 +207,7 @@ static void test_visitor_in_bool_str_autocast(TestInputVisitorData *data,
bool res = false;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true, "\"yes\"");
+ v = visitor_input_test_init_full(data, true, true, false, "\"yes\"");
visit_type_bool(v, NULL, &res, &error_abort);
g_assert_cmpint(res, ==, true);
@@ -240,7 +245,7 @@ static void test_visitor_in_number_autocast(TestInputVisitorData *data,
Error *err = NULL;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true, "%f", value);
+ v = visitor_input_test_init_full(data, true, true, false, "%f", value);
visit_type_number(v, NULL, &res, &err);
error_free_or_abort(&err);
@@ -252,7 +257,7 @@ static void test_visitor_in_number_str_autocast(TestInputVisitorData *data,
double res = 0, value = 3.14;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true, "\"3.14\"");
+ v = visitor_input_test_init_full(data, true, true, false, "\"3.14\"");
visit_type_number(v, NULL, &res, &error_abort);
g_assert_cmpfloat(res, ==, value);
@@ -277,7 +282,7 @@ static void test_visitor_in_size_str_autocast(TestInputVisitorData *data,
uint64_t res, value = 500 * 1024 * 1024;
Visitor *v;
- v = visitor_input_test_init_full(data, true, true, "\"500M\"");
+ v = visitor_input_test_init_full(data, true, true, false, "\"500M\"");
visit_type_size(v, NULL, &res, &error_abort);
g_assert_cmpfloat(res, ==, value);
@@ -396,6 +401,59 @@ static void test_visitor_in_list(TestInputVisitorData *data,
g_assert(!head);
}
+static void test_visitor_in_list_autocreate_none(TestInputVisitorData *data,
+ const void *unused)
+{
+ UserDefOneList *head = NULL;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init_full(data, true, true, false,
+ "{ 'string': 'string0', 'integer': 42 }");
+
+ visit_type_UserDefOneList(v, NULL, &head, &err);
+ error_free_or_abort(&err);
+ g_assert(head == NULL);
+}
+
+static void test_visitor_in_list_autocreate_dict(TestInputVisitorData *data,
+ const void *unused)
+{
+ UserDefOneList *head = NULL;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, true, true, true,
+ "{ 'string': 'string0', 'integer': '42' }");
+
+ visit_type_UserDefOneList(v, NULL, &head, &error_abort);
+ g_assert(head != NULL);
+
+ g_assert_cmpstr(head->value->string, ==, "string0");
+ g_assert_cmpint(head->value->integer, ==, 42);
+ g_assert(head->next == NULL);
+
+ qapi_free_UserDefOneList(head);
+ head = NULL;
+}
+
+static void test_visitor_in_list_autocreate_int(TestInputVisitorData *data,
+ const void *unused)
+{
+ uint32List *head = NULL;
+ Visitor *v;
+
+ /* Verify that we auto-create a single element list from the int */
+ v = visitor_input_test_init_full(data, true, true, true, "'42'");
+
+ visit_type_uint32List(v, NULL, &head, &error_abort);
+ g_assert(head != NULL);
+
+ g_assert_cmpint(head->value, ==, 42);
+
+ qapi_free_uint32List(head);
+ head = NULL;
+}
+
static void test_visitor_in_any(TestInputVisitorData *data,
const void *unused)
{
@@ -452,7 +510,7 @@ static void test_visitor_in_null(TestInputVisitorData *data,
* when input is not null.
*/
- v = visitor_input_test_init_full(data, false, false,
+ v = visitor_input_test_init_full(data, false, false, false,
"{ 'a': null, 'b': '' }");
visit_start_struct(v, NULL, NULL, 0, &error_abort);
visit_type_null(v, "a", &error_abort);
@@ -1040,6 +1098,12 @@ int main(int argc, char **argv)
NULL, test_visitor_in_struct_nested);
input_visitor_test_add("/visitor/input/list",
NULL, test_visitor_in_list);
+ input_visitor_test_add("/visitor/input/list-autocreate-noautocast",
+ NULL, test_visitor_in_list_autocreate_none);
+ input_visitor_test_add("/visitor/input/list-autocreate-autocast",
+ NULL, test_visitor_in_list_autocreate_dict);
+ input_visitor_test_add("/visitor/input/list-autocreate-int",
+ NULL, test_visitor_in_list_autocreate_int);
input_visitor_test_add("/visitor/input/any",
NULL, test_visitor_in_any);
input_visitor_test_add("/visitor/input/null",
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (8 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 18:23 ` Eric Blake
` (2 more replies)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor Daniel P. Berrange
` (12 subsequent siblings)
22 siblings, 3 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Some of the historical command line opts that had their
keys in in a completely flat namespace are now represented
by QAPI schemas that use a nested structs. When converting
the QemuOpts to QObject, there is no information about
compound types available, so the QObject will be completely
flat, even after the qdict_crumple() call. So when starting
a struct, we may not have a QDict available in the input
data, so we auto-create a QDict containing all the currently
unvisited input data keys. Not all historical command line
opts require this, so the behaviour is opt-in, by specifying
how many levels of structs are permitted to be auto-created.
Note that this only works if the child struct is the last
field to the visited in the parent struct. This is always
the case for currently existing legacy command line options.
The example is the NetLegacy type which has 3 levels of
structs. The modern way to represent this in QemuOpts would
be the dot-separated component approach
-net vlan=1,id=foo,name=bar,opts.type=tap,\
opts.data.fd=3,opts.data.script=ifup
The legacy syntax will just be presenting
-net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
So we need to auto-create 3 levels of struct when visiting.
The implementation here will enable visiting in both the
modern and legacy syntax, compared to OptsVisitor which
only allows the legacy syntax.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 22 +++++-
qapi/qobject-input-visitor.c | 59 ++++++++++++++--
tests/test-qobject-input-visitor.c | 130 ++++++++++++++++++++++++++++++++---
3 files changed, 194 insertions(+), 17 deletions(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index 1809f48..94051f3 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -45,7 +45,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
* If @autocreate_list is true, then as an alternative to a normal QList,
* list values can be stored as a QString or QDict instead, which will
* be interpreted as representing single element lists. This should only
- * by used if compatibility is required with the OptsVisitor which allowed
+ * be used if compatibility is required with the OptsVisitor which allowed
* repeated keys, without list indexes, to represent lists. e.g. set this
* to true if you have compatibility requirements for
*
@@ -55,6 +55,22 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
*
* -arg foo.0=hello,foo.1=world
*
+ * If @autocreate_struct_levels is non-zero, then when visiting structs,
+ * if the corresponding QDict is not found, it will automatically create
+ * a QDict containing all remaining unvisited options. This should only
+ * be used if compatibility is required with the OptsVisitor which flatten
+ * structs so that all keys were at the same level. e.g. set this to a
+ * non-zero number if you compatibility requirements for
+ *
+ * -arg type=person,surname=blogs,forename=fred
+ *
+ * to be treated as equivalent to the perferred syntax
+ *
+ * -arg type=person,data.surname=blogs,data.forename=fred
+ *
+ * The value given determines how many levels of structs are allowed to
+ * be flattened in this way.
+ *
* The visitor always operates in strict mode, requiring all dict keys
* to be consumed during visitation. An error will be reported if this
* does not happen.
@@ -63,7 +79,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
* visit_free() when no longer required.
*/
Visitor *qobject_input_visitor_new_autocast(QObject *obj,
- bool autocreate_list);
+ bool autocreate_list,
+ size_t autocreate_struct_levels);
/**
@@ -80,6 +97,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
*/
Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
+ size_t autocreate_struct_levels,
Error **errp);
#endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index d88e9f9..1be4865 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -52,6 +52,14 @@ struct QObjectInputVisitor
/* Whether we can auto-create single element lists when
* encountering a non-QList type */
bool autocreate_list;
+
+ /* Current depth of recursion into structs */
+ size_t struct_level;
+
+ /* Numbers of levels at which we will
+ * consider auto-creating a struct containing
+ * remaining unvisited items */
+ size_t autocreate_struct_levels;
};
static QObjectInputVisitor *to_qiv(Visitor *v)
@@ -79,7 +87,12 @@ static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
if (qobject_type(qobj) == QTYPE_QDICT) {
assert(name);
- ret = qdict_get(qobject_to_qdict(qobj), name);
+ if (qiv->autocreate_struct_levels &&
+ !g_hash_table_contains(tos->h, name)) {
+ ret = NULL;
+ } else {
+ ret = qdict_get(qobject_to_qdict(qobj), name);
+ }
if (tos->h && consume && ret) {
bool removed = g_hash_table_remove(tos->h, name);
assert(removed);
@@ -114,7 +127,8 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
tos->qapi = qapi;
qobject_incref(obj);
- if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
+ if ((qiv->autocreate_struct_levels || qiv->strict) &&
+ qobject_type(obj) == QTYPE_QDICT) {
h = g_hash_table_new(g_str_hash, g_str_equal);
qdict_iter(qobject_to_qdict(obj), qdict_add_key, h);
tos->h = h;
@@ -163,6 +177,9 @@ static void qobject_input_pop(Visitor *v, void **obj)
StackObject *tos = QSLIST_FIRST(&qiv->stack);
assert(tos && tos->qapi == obj);
+ if (tos->h != NULL) {
+ qiv->struct_level--;
+ }
QSLIST_REMOVE_HEAD(&qiv->stack, node);
qobject_input_stack_object_free(tos);
}
@@ -171,12 +188,38 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
size_t size, Error **errp)
{
QObjectInputVisitor *qiv = to_qiv(v);
- QObject *qobj = qobject_input_get_object(qiv, name, true);
+ QObject *qobj;
+ QDict *subopts = NULL;
Error *err = NULL;
+ StackObject *tos = QSLIST_FIRST(&qiv->stack);
+ qobj = qobject_input_get_object(qiv, name, true);
if (obj) {
*obj = NULL;
}
+
+ if (!qobj && (qiv->struct_level < qiv->autocreate_struct_levels)) {
+ /* Create a new dict that contains all the currently
+ * unvisited items */
+ if (tos) {
+ GHashTableIter iter;
+ const char *key;
+
+ subopts = qdict_new();
+
+ g_hash_table_iter_init(&iter, tos->h);
+ while (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
+ QObject *val = qdict_get(qobject_to_qdict(tos->obj), key);
+ qobject_incref(val);
+ qdict_put_obj(subopts, key, val);
+ }
+ g_hash_table_remove_all(tos->h);
+ qobj = QOBJECT(subopts);
+ } else {
+ qobj = qiv->root;
+ }
+ }
+
if (!qobj || qobject_type(qobj) != QTYPE_QDICT) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"QDict");
@@ -184,6 +227,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
}
qobject_input_push(qiv, qobj, obj, &err);
+ qiv->struct_level++;
if (err) {
error_propagate(errp, err);
return;
@@ -192,6 +236,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
if (obj) {
*obj = g_malloc0(size);
}
+ QDECREF(subopts);
}
@@ -530,7 +575,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
}
Visitor *qobject_input_visitor_new_autocast(QObject *obj,
- bool autocreate_list)
+ bool autocreate_list,
+ size_t autocreate_struct_levels)
{
QObjectInputVisitor *v;
@@ -556,6 +602,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
v->visitor.free = qobject_input_free;
v->strict = true;
v->autocreate_list = autocreate_list;
+ v->autocreate_struct_levels = autocreate_struct_levels;
v->root = obj;
qobject_incref(obj);
@@ -566,6 +613,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
+ size_t autocreate_struct_levels,
Error **errp)
{
QDict *pdict;
@@ -583,7 +631,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
}
v = qobject_input_visitor_new_autocast(pobj,
- autocreate_list);
+ autocreate_list,
+ autocreate_struct_levels);
cleanup:
qobject_decref(pobj);
QDECREF(pdict);
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index c227565..ab55f99 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -40,11 +40,13 @@ static void visitor_input_teardown(TestInputVisitorData *data,
/* The various test_init functions are provided instead of a test setup
function so that the JSON string used by the tests are kept in the test
functions (and not in main()). */
-static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
- bool strict, bool autocast,
- bool autocreate_list,
- const char *json_string,
- va_list *ap)
+static Visitor *
+visitor_input_test_init_internal(TestInputVisitorData *data,
+ bool strict, bool autocast,
+ bool autocreate_list,
+ size_t autocreate_struct_levels,
+ const char *json_string,
+ va_list *ap)
{
visitor_input_teardown(data, NULL);
@@ -53,10 +55,11 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
if (autocast) {
assert(strict);
- data->qiv = qobject_input_visitor_new_autocast(data->obj,
- autocreate_list);
+ data->qiv = qobject_input_visitor_new_autocast(
+ data->obj, autocreate_list, autocreate_struct_levels);
} else {
assert(!autocreate_list);
+ assert(!autocreate_struct_levels);
data->qiv = qobject_input_visitor_new(data->obj, strict);
}
g_assert(data->qiv);
@@ -74,7 +77,7 @@ Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
va_start(ap, json_string);
v = visitor_input_test_init_internal(data, strict, autocast,
- autocreate_list,
+ autocreate_list, 0,
json_string, &ap);
va_end(ap);
return v;
@@ -88,7 +91,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- v = visitor_input_test_init_internal(data, true, false, false,
+ v = visitor_input_test_init_internal(data, true, false, false, 0,
json_string, &ap);
va_end(ap);
return v;
@@ -104,7 +107,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- return visitor_input_test_init_internal(data, true, false, false,
+ return visitor_input_test_init_internal(data, true, false, false, 0,
json_string, NULL);
}
@@ -372,6 +375,109 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
qapi_free_UserDefTwo(udp);
}
+static void test_visitor_in_struct_autocreate(TestInputVisitorData *data,
+ const void *unused)
+{
+ Visitor *v;
+ int64_t vlan;
+ char *id = NULL;
+ char *type;
+ int64_t fd;
+ char *script = NULL;
+
+ v = visitor_input_test_init_internal(
+ data, true, true, false, 3,
+ "{ 'vlan': '1', 'id': 'foo', 'type': 'tap', 'fd': '3', "
+ "'script': 'ifup' }", NULL);
+
+ visit_start_struct(v, NULL, NULL, 0, &error_abort);
+
+ visit_type_int64(v, "vlan", &vlan, &error_abort);
+ visit_type_str(v, "id", &id, &error_abort);
+
+ visit_start_struct(v, "opts", NULL, 0, &error_abort);
+ visit_type_str(v, "type", &type, &error_abort);
+
+ visit_start_struct(v, "data", NULL, 0, &error_abort);
+
+ visit_type_int64(v, "fd", &fd, &error_abort);
+ visit_type_str(v, "script", &script, &error_abort);
+
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+
+ g_assert_cmpstr(id, ==, "foo");
+ g_assert_cmpint(vlan, ==, 1);
+ g_assert_cmpstr(type, ==, "tap");
+ g_assert_cmpstr(script, ==, "ifup");
+ g_assert_cmpint(fd, ==, 3);
+
+ g_free(id);
+ g_free(type);
+ g_free(script);
+}
+
+static void test_visitor_in_struct_autocreate_extra(TestInputVisitorData *data,
+ const void *unused)
+{
+ Visitor *v;
+ int64_t vlan;
+ char *id = NULL;
+ char *type;
+ int64_t fd;
+ char *script = NULL;
+ Error *err = NULL;
+
+ v = visitor_input_test_init_internal(
+ data, true, true, false, 3,
+ "{ 'vlan': '1', 'id': 'foo', 'type': 'tap', 'fd': '3', "
+ "'script': 'ifup' }", NULL);
+
+ visit_start_struct(v, NULL, NULL, 0, &error_abort);
+
+ visit_type_int64(v, "vlan", &vlan, &error_abort);
+
+ visit_start_struct(v, "opts", NULL, 0, &error_abort);
+ visit_type_str(v, "type", &type, &error_abort);
+
+ visit_start_struct(v, "data", NULL, 0, &error_abort);
+
+ visit_type_int64(v, "fd", &fd, &error_abort);
+ visit_type_str(v, "script", &script, &error_abort);
+
+ /* We've not visited 'id' so should see a complaint */
+ visit_check_struct(v, &err);
+ error_free_or_abort(&err);
+ visit_end_struct(v, NULL);
+
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+
+ /* We can't visit stuff in the base struct after
+ * we auto-created child structs */
+ visit_type_str(v, "id", &id, &err);
+ error_free_or_abort(&err);
+
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+
+ g_assert_cmpstr(id, ==, NULL);
+ g_assert_cmpint(vlan, ==, 1);
+ g_assert_cmpstr(type, ==, "tap");
+ g_assert_cmpstr(script, ==, "ifup");
+ g_assert_cmpint(fd, ==, 3);
+
+ g_free(id);
+ g_free(type);
+ g_free(script);
+}
+
static void test_visitor_in_list(TestInputVisitorData *data,
const void *unused)
{
@@ -1096,6 +1202,10 @@ int main(int argc, char **argv)
NULL, test_visitor_in_struct);
input_visitor_test_add("/visitor/input/struct-nested",
NULL, test_visitor_in_struct_nested);
+ input_visitor_test_add("/visitor/input/struct-autocreate",
+ NULL, test_visitor_in_struct_autocreate);
+ input_visitor_test_add("/visitor/input/struct-autocreate-extra",
+ NULL, test_visitor_in_struct_autocreate_extra);
input_visitor_test_add("/visitor/input/list",
NULL, test_visitor_in_list);
input_visitor_test_add("/visitor/input/list-autocreate-noautocast",
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (9 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 19:45 ` Eric Blake
2016-10-12 15:50 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
` (11 subsequent siblings)
22 siblings, 2 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The traditional CLI arg syntax allows two ways to specify
integer lists, either one value per key, or a range of
values per key. eg the following are identical:
-arg foo=5,foo=6,foo=7
-arg foo=5-7
This extends the QObjectInputVisitor so that it is able
to parse ranges and turn them into distinct list entries.
This means that
-arg foo=5-7
is treated as equivalent to
-arg foo.0=5,foo.1=6,foo.2=7
Edge case tests are copied from test-opts-visitor to
ensure identical behaviour when parsing.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 23 ++++-
qapi/qobject-input-visitor.c | 158 ++++++++++++++++++++++++++--
tests/test-qobject-input-visitor.c | 195 +++++++++++++++++++++++++++++++++--
3 files changed, 360 insertions(+), 16 deletions(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index 94051f3..242b767 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -19,6 +19,12 @@
typedef struct QObjectInputVisitor QObjectInputVisitor;
+/* Inclusive upper bound on the size of any flattened range. This is a safety
+ * (= anti-annoyance) measure; wrong ranges should not cause long startup
+ * delays nor exhaust virtual memory.
+ */
+#define QIV_RANGE_MAX 65536
+
/**
* Create a new input visitor that converts @obj to a QAPI object.
*
@@ -71,6 +77,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
* The value given determines how many levels of structs are allowed to
* be flattened in this way.
*
+ * If @permit_int_ranges is true, then when visiting a list of integers,
+ * the integer value strings may encode ranges eg a single element
+ * containing "5-7" is treated as if there were three elements "5", "6",
+ * "7". This should only be used if compatibility is required with the
+ * OptsVisitor which would allow integer ranges. e.g. set this to true
+ * if you have compatibility requirements for
+ *
+ * -arg val=5-8
+ *
+ * to be treated as equivalent to the preferred syntax:
+ *
+ * -arg val.0=5,val.1=6,val.2=7,val.3=8
+ *
* The visitor always operates in strict mode, requiring all dict keys
* to be consumed during visitation. An error will be reported if this
* does not happen.
@@ -80,7 +99,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
*/
Visitor *qobject_input_visitor_new_autocast(QObject *obj,
bool autocreate_list,
- size_t autocreate_struct_levels);
+ size_t autocreate_struct_levels,
+ bool permit_int_ranges);
/**
@@ -98,6 +118,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
size_t autocreate_struct_levels,
+ bool permit_int_ranges,
Error **errp);
#endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 1be4865..6b3d0f2 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -31,6 +31,8 @@ typedef struct StackObject
GHashTable *h; /* If obj is dict: unvisited keys */
const QListEntry *entry; /* If obj is list: unvisited tail */
+ uint64_t range_val;
+ uint64_t range_limit;
QSLIST_ENTRY(StackObject) node;
} StackObject;
@@ -60,6 +62,10 @@ struct QObjectInputVisitor
* consider auto-creating a struct containing
* remaining unvisited items */
size_t autocreate_struct_levels;
+
+ /* Whether int lists can have single values representing
+ * ranges of values */
+ bool permit_int_ranges;
};
static QObjectInputVisitor *to_qiv(Visitor *v)
@@ -282,7 +288,7 @@ static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
QObjectInputVisitor *qiv = to_qiv(v);
StackObject *so = QSLIST_FIRST(&qiv->stack);
- if (!so->entry) {
+ if ((so->range_val == so->range_limit) && !so->entry) {
return NULL;
}
tail->next = g_malloc0(size);
@@ -329,21 +335,87 @@ static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
int64_t *obj, Error **errp)
{
QObjectInputVisitor *qiv = to_qiv(v);
- QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
- true));
+ QString *qstr;
int64_t ret;
+ const char *end = NULL;
+ StackObject *tos;
+ bool inlist = false;
+
+ /* Preferentially generate values from a range, before
+ * trying to consume another QList element */
+ tos = QSLIST_FIRST(&qiv->stack);
+ if (tos) {
+ if ((int64_t)tos->range_val < (int64_t)tos->range_limit) {
+ *obj = tos->range_val + 1;
+ tos->range_val++;
+ return;
+ } else {
+ inlist = tos->entry != NULL;
+ }
+ }
+ qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
if (!qstr || !qstr->string) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"string");
return;
}
- if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
+ if (qemu_strtoll(qstr->string, &end, 0, &ret) < 0) {
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
return;
}
*obj = ret;
+
+ /*
+ * If we have string that represents an integer range (5-24),
+ * parse the end of the range and set things up so we'll process
+ * the rest of the range before consuming another element
+ * from the QList.
+ */
+ if (end && *end) {
+ if (!qiv->permit_int_ranges) {
+ error_setg(errp,
+ "Integer ranges are not permitted here");
+ return;
+ }
+ if (!inlist) {
+ error_setg(errp,
+ "Integer ranges are only permitted when "
+ "visiting list parameters");
+ return;
+ }
+ if (*end != '-') {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+ "a number range");
+ return;
+ }
+ end++;
+ if (qemu_strtoll(end, NULL, 0, &ret) < 0) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
+ return;
+ }
+ if (*obj > ret) {
+ error_setg(errp,
+ "Parameter '%s' range start %" PRIu64
+ " must be less than (or equal to) %" PRIu64,
+ name, *obj, ret);
+ return;
+ }
+
+ if ((*obj <= (INT64_MAX - QIV_RANGE_MAX)) &&
+ (ret >= (*obj + QIV_RANGE_MAX))) {
+ error_setg(errp,
+ "Parameter '%s' range must be less than %d",
+ name, QIV_RANGE_MAX);
+ return;
+ }
+ if (*obj != ret) {
+ tos->range_val = *obj;
+ tos->range_limit = ret;
+ }
+ }
}
static void qobject_input_type_uint64(Visitor *v, const char *name,
@@ -366,21 +438,85 @@ static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
uint64_t *obj, Error **errp)
{
QObjectInputVisitor *qiv = to_qiv(v);
- QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
- true));
+ QString *qstr;
unsigned long long ret;
+ char *end = NULL;
+ StackObject *tos;
+ bool inlist = false;
+
+ /* Preferentially generate values from a range, before
+ * trying to consume another QList element */
+ tos = QSLIST_FIRST(&qiv->stack);
+ if (tos) {
+ if (tos->range_val < tos->range_limit) {
+ *obj = tos->range_val + 1;
+ tos->range_val++;
+ return;
+ } else {
+ inlist = tos->entry != NULL;
+ }
+ }
+ qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
+ true));
if (!qstr || !qstr->string) {
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"string");
return;
}
- if (parse_uint_full(qstr->string, &ret, 0) < 0) {
+ if (parse_uint(qstr->string, &ret, &end, 0) < 0) {
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
return;
}
*obj = ret;
+
+ /*
+ * If we have string that represents an integer range (5-24),
+ * parse the end of the range and set things up so we'll process
+ * the rest of the range before consuming another element
+ * from the QList.
+ */
+ if (end && *end) {
+ if (!qiv->permit_int_ranges) {
+ error_setg(errp,
+ "Integer ranges are not permitted here");
+ return;
+ }
+ if (!inlist) {
+ error_setg(errp,
+ "Integer ranges are only permitted when "
+ "visiting list parameters");
+ return;
+ }
+ if (*end != '-') {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+ "a number range");
+ return;
+ }
+ end++;
+ if (parse_uint_full(end, &ret, 0) < 0) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
+ return;
+ }
+ if (*obj > ret) {
+ error_setg(errp,
+ "Parameter '%s' range start %" PRIu64
+ " must be less than (or equal to) %llu",
+ name, *obj, ret);
+ return;
+ }
+ if ((ret - *obj) > (QIV_RANGE_MAX - 1)) {
+ error_setg(errp,
+ "Parameter '%s' range must be less than %d",
+ name, QIV_RANGE_MAX);
+ return;
+ }
+ if (*obj != ret) {
+ tos->range_val = *obj;
+ tos->range_limit = ret;
+ }
+ }
}
static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
@@ -576,7 +712,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
Visitor *qobject_input_visitor_new_autocast(QObject *obj,
bool autocreate_list,
- size_t autocreate_struct_levels)
+ size_t autocreate_struct_levels,
+ bool permit_int_ranges)
{
QObjectInputVisitor *v;
@@ -603,6 +740,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
v->strict = true;
v->autocreate_list = autocreate_list;
v->autocreate_struct_levels = autocreate_struct_levels;
+ v->permit_int_ranges = permit_int_ranges;
v->root = obj;
qobject_incref(obj);
@@ -614,6 +752,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
size_t autocreate_struct_levels,
+ bool permit_int_ranges,
Error **errp)
{
QDict *pdict;
@@ -632,7 +771,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
v = qobject_input_visitor_new_autocast(pobj,
autocreate_list,
- autocreate_struct_levels);
+ autocreate_struct_levels,
+ permit_int_ranges);
cleanup:
qobject_decref(pobj);
QDECREF(pdict);
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index ab55f99..783ecd4 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -45,6 +45,7 @@ visitor_input_test_init_internal(TestInputVisitorData *data,
bool strict, bool autocast,
bool autocreate_list,
size_t autocreate_struct_levels,
+ bool permit_int_ranges,
const char *json_string,
va_list *ap)
{
@@ -56,10 +57,12 @@ visitor_input_test_init_internal(TestInputVisitorData *data,
if (autocast) {
assert(strict);
data->qiv = qobject_input_visitor_new_autocast(
- data->obj, autocreate_list, autocreate_struct_levels);
+ data->obj, autocreate_list, autocreate_struct_levels,
+ permit_int_ranges);
} else {
assert(!autocreate_list);
assert(!autocreate_struct_levels);
+ assert(!permit_int_ranges);
data->qiv = qobject_input_visitor_new(data->obj, strict);
}
g_assert(data->qiv);
@@ -77,7 +80,7 @@ Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
va_start(ap, json_string);
v = visitor_input_test_init_internal(data, strict, autocast,
- autocreate_list, 0,
+ autocreate_list, 0, false,
json_string, &ap);
va_end(ap);
return v;
@@ -91,7 +94,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- v = visitor_input_test_init_internal(data, true, false, false, 0,
+ v = visitor_input_test_init_internal(data, true, false, false, 0, false,
json_string, &ap);
va_end(ap);
return v;
@@ -107,7 +110,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- return visitor_input_test_init_internal(data, true, false, false, 0,
+ return visitor_input_test_init_internal(data, true, false, false, 0, false,
json_string, NULL);
}
@@ -386,7 +389,7 @@ static void test_visitor_in_struct_autocreate(TestInputVisitorData *data,
char *script = NULL;
v = visitor_input_test_init_internal(
- data, true, true, false, 3,
+ data, true, true, false, 3, false,
"{ 'vlan': '1', 'id': 'foo', 'type': 'tap', 'fd': '3', "
"'script': 'ifup' }", NULL);
@@ -435,7 +438,7 @@ static void test_visitor_in_struct_autocreate_extra(TestInputVisitorData *data,
Error *err = NULL;
v = visitor_input_test_init_internal(
- data, true, true, false, 3,
+ data, true, true, false, 3, false,
"{ 'vlan': '1', 'id': 'foo', 'type': 'tap', 'fd': '3', "
"'script': 'ifup' }", NULL);
@@ -560,6 +563,88 @@ static void test_visitor_in_list_autocreate_int(TestInputVisitorData *data,
head = NULL;
}
+typedef struct {
+ const char *range_str;
+ bool expect_fail;
+ int64_t *values;
+ size_t nvalues;
+} TestIntRangeData;
+
+static void test_visitor_in_list_autocast_int_range(TestInputVisitorData *data,
+ const void *opaque)
+{
+ const TestIntRangeData *idata = opaque;
+ int64List *item, *head = NULL;
+ Visitor *v;
+ size_t i;
+ Error *err = NULL;
+
+ /* Verify that we auto-expand signed int ranges */
+ v = visitor_input_test_init_internal(data, true, true, true, 0, true,
+ idata->range_str, NULL);
+
+ visit_type_int64List(v, NULL, &head, &err);
+ if (idata->expect_fail) {
+ error_free_or_abort(&err);
+ g_assert(head == NULL);
+ } else {
+ g_assert(err == NULL);
+ g_assert(head != NULL);
+ for (i = 0, item = head; item != NULL && i < idata->nvalues;
+ item = item->next, i++) {
+ if (idata->values != NULL) {
+ g_assert_cmpint(item->value, ==, idata->values[i]);
+ }
+ }
+ g_assert_cmpint(i, ==, idata->nvalues);
+
+ qapi_free_int64List(head);
+ head = NULL;
+ }
+}
+
+typedef struct {
+ const char *range_str;
+ bool expect_fail;
+ uint64_t *values;
+ size_t nvalues;
+} TestUIntRangeData;
+
+static void test_visitor_in_list_autocast_uint_range(TestInputVisitorData *data,
+ const void *opaque)
+{
+ const TestUIntRangeData *idata = opaque;
+ uint64List *item, *head = NULL;
+ Visitor *v;
+ size_t i;
+ Error *err = NULL;
+
+ /* Verify that we auto-expand signed int ranges */
+ v = visitor_input_test_init_internal(data, true, true, true, 0, true,
+ idata->range_str, NULL);
+
+ visit_type_uint64List(v, NULL, &head, &err);
+ if (idata->expect_fail) {
+ g_assert(err != NULL);
+ g_assert(head == NULL);
+ } else {
+ g_assert(err == NULL);
+ g_assert(head != NULL);
+
+ for (i = 0, item = head; item != NULL && i < idata->nvalues;
+ item = item->next, i++) {
+ if (idata->values != NULL) {
+ g_assert_cmpint(item->value, ==, idata->values[i]);
+ }
+ }
+ g_assert_cmpint(i, ==, idata->nvalues);
+
+ qapi_free_uint64List(head);
+ head = NULL;
+ }
+}
+
+
static void test_visitor_in_any(TestInputVisitorData *data,
const void *unused)
{
@@ -1253,6 +1338,104 @@ int main(int argc, char **argv)
input_visitor_test_add("/visitor/input/native_list/number",
NULL, test_visitor_in_native_list_number);
+#define INT_RANGE_TEST(suffix, str, expect_fail, values, nvalues) \
+ TestIntRangeData idata ## suffix = \
+ { str, expect_fail, values, nvalues }; \
+ input_visitor_test_add("/visitor/input/int-list-autocast-" #suffix, \
+ &idata ## suffix, \
+ test_visitor_in_list_autocast_int_range)
+
+#define UINT_RANGE_TEST(suffix, str, expect_fail, values, nvalues) \
+ TestUIntRangeData udata ## suffix = \
+ { str, expect_fail, values, nvalues }; \
+ input_visitor_test_add("/visitor/input/uint-list-autocast-" #suffix,\
+ &udata ## suffix, \
+ test_visitor_in_list_autocast_uint_range)
+
+
+ int64_t multvals[] = { -1, 0, -11, -10, -9, 5, -16, -15, -14, -13, 14,
+ 15, 7, 2, 3, 9, 10, 11, 12, -7, -6, -5, -4, -3 };
+ INT_RANGE_TEST(multiple,
+ "['-1-0','-11--9','5-5'," \
+ "'-16--13','14-15'," \
+ "'7','2-3','9-12','-7--3']",
+ false, multvals, G_N_ELEMENTS(multvals));
+
+ INT_RANGE_TEST(errno,
+ "'0x7fffffffffffffff-0x8000000000000000'",
+ true, NULL, 0);
+ INT_RANGE_TEST(incomplete, "'5-'",
+ true, NULL, 0);
+ INT_RANGE_TEST(trailing, "'5-6z'",
+ true, NULL, 0);
+ INT_RANGE_TEST(empty, "\"6-5\"",
+ true, NULL, 0);
+ int64_t negvals[] = { -10, -9, -8, -7 };
+ INT_RANGE_TEST(neg, "\"-10--7\"",
+ false, negvals, G_N_ELEMENTS(negvals));
+ INT_RANGE_TEST(minval,
+ "'-0x8000000000000000--0x8000000000000000'",
+ false, (int64_t[]){ -0x8000000000000000 }, 1);
+ INT_RANGE_TEST(maxval,
+ "'0x7fffffffffffffff-0x7fffffffffffffff'",
+ false, (int64_t[]){ 0x7fffffffffffffff }, 1);
+
+ UINT_RANGE_TEST(errno,
+ "'0xffffffffffffffff-0x10000000000000000'",
+ true, NULL, 0);
+ UINT_RANGE_TEST(incomplete, "'5-'",
+ true, NULL, 0);
+ UINT_RANGE_TEST(trailing, "'5-6z'",
+ true, NULL, 0);
+ UINT_RANGE_TEST(empty, "'6-5'",
+ true, NULL, 0);
+ UINT_RANGE_TEST(neg, "\"-10--7\"",
+ true, NULL, 0);
+ UINT_RANGE_TEST(minval, "'0-0'",
+ false, (uint64_t[]){ 0 }, 1);
+ UINT_RANGE_TEST(maxval,
+ "'0xffffffffffffffff-0xffffffffffffffff'",
+ false, (uint64_t[]){ 0xffffffffffffffff }, 1);
+
+ /* Test maximum range sizes. The macro value is open-coded here
+ * *intentionally*; the test case must use concrete values by design. If
+ * QIV_RANGE_MAX is changed, the following values need to be
+ * recalculated as well. The assert and this comment should help with it.
+ */
+ g_assert(QIV_RANGE_MAX == 65536);
+
+ /* The unsigned case is simple, a u64-u64 difference can always be
+ * represented as a u64.
+ */
+ UINT_RANGE_TEST(max,
+ "'0-65535'",
+ false, NULL, QIV_RANGE_MAX);
+ UINT_RANGE_TEST(2big,
+ "'0-65536'",
+ true, NULL, 0);
+
+ INT_RANGE_TEST(max_pos_a,
+ "'0x7fffffffffff0000-0x7fffffffffffffff'",
+ false, NULL, 100);
+ INT_RANGE_TEST(max_pos_b,
+ "'0x7ffffffffffeffff-0x7ffffffffffffffe'",
+ false, NULL, 100);
+ INT_RANGE_TEST(max_neg_a,
+ "'-0x8000000000000000--0x7fffffffffff0001'",
+ false, NULL, 100);
+ INT_RANGE_TEST(max_neg_b,
+ "'-0x7fffffffffffffff--0x7fffffffffff0000'",
+ false, NULL, 100);
+ INT_RANGE_TEST(2big_pos,
+ "'0x7ffffffffffeffff-0x7fffffffffffffff'",
+ true, NULL, 0);
+ INT_RANGE_TEST(2big_neg,
+ "'-0x8000000000000000--0x7fffffffffff0000'",
+ true, NULL, 0);
+ INT_RANGE_TEST(2big_neg_pos,
+ "'-0x8000000000000000-0x7fffffffffffffff'",
+ true, NULL, 0);
+
g_test_run();
return 0;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (10 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 20:08 ` Eric Blake
` (2 more replies)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values Daniel P. Berrange
` (10 subsequent siblings)
22 siblings, 3 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
If given an option string such as
size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
the qemu_opts_to_qdict() method will currently overwrite
the values for repeated option keys, so only the last
value is in the returned dict:
size=QString("1024")
nodes=QString("1-2")
policy=QString("bind")
With this change the caller can optionally ask for all
the repeated values to be stored in a QList. In the
above example that would result in 'nodes' being a
QList, so the returned dict would contain
size=QString("1024")
nodes=QList([QString("10"),
QString("4-5"),
QString("1-2")])
policy=QString("bind")
Note that the conversion has no way of knowing whether
any given key is expected to be a list upfront - it can
only figure that out when seeing the first duplicated
key. Thus the caller has to be prepared to deal with the
fact that if a key 'foo' is a list, then the returned
qdict may contain either a QString or a QList for the
key 'foo'.
In a third mode, it is possible to ask for repeated
options to be reported as an error, rather than silently
dropping all but the last one.
All existing callers are all converted to explicitly
request the historical behaviour of only reporting the
last key. Later patches will make use of the new modes.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
blockdev.c | 7 ++-
include/qemu/option.h | 13 ++++-
monitor.c | 3 +-
qapi/qobject-input-visitor.c | 4 +-
qemu-img.c | 4 +-
qemu-io-cmds.c | 3 +-
qemu-io.c | 6 +-
qemu-nbd.c | 3 +-
qom/object_interfaces.c | 3 +-
tests/test-qemu-opts.c | 132 +++++++++++++++++++++++++++++++++++++++++++
tests/test-replication.c | 9 ++-
util/qemu-option.c | 64 ++++++++++++++++++---
12 files changed, 229 insertions(+), 22 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 814d49f..a999d46 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -914,7 +914,8 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
/* Get a QDict for processing the options */
bs_opts = qdict_new();
- qemu_opts_to_qdict(all_opts, bs_opts);
+ qemu_opts_to_qdict(all_opts, bs_opts, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
legacy_opts = qemu_opts_create(&qemu_legacy_drive_opts, NULL, 0,
&error_abort);
@@ -3804,8 +3805,8 @@ void hmp_drive_add_node(Monitor *mon, const char *optstr)
return;
}
- qdict = qemu_opts_to_qdict(opts, NULL);
-
+ qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
if (!qdict_get_try_str(qdict, "node-name")) {
QDECREF(qdict);
error_report("'node-name' needs to be specified");
diff --git a/include/qemu/option.h b/include/qemu/option.h
index 29f3f18..ffd841d 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -125,7 +125,18 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
int permit_abbrev);
QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
Error **errp);
-QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
+typedef enum {
+ /* Last occurrence of a duplicate option silently replaces earlier */
+ QEMU_OPTS_REPEAT_POLICY_LAST,
+ /* Each occurrence of a duplicate option converts value to a QList */
+ QEMU_OPTS_REPEAT_POLICY_ALL,
+ /* First occurrence of a duplicate option causes an error */
+ QEMU_OPTS_REPEAT_POLICY_ERROR,
+} QemuOptsRepeatPolicy;
+
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
+ QemuOptsRepeatPolicy repeatPolicy,
+ Error **errp);
void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
diff --git a/monitor.c b/monitor.c
index 14089cc..84e79d4 100644
--- a/monitor.c
+++ b/monitor.c
@@ -2642,7 +2642,8 @@ static QDict *monitor_parse_arguments(Monitor *mon,
if (!opts) {
goto fail;
}
- qemu_opts_to_qdict(opts, qdict);
+ qemu_opts_to_qdict(opts, qdict, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
qemu_opts_del(opts);
}
break;
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 6b3d0f2..2287d11 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -759,7 +759,9 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
QObject *pobj = NULL;
Visitor *v = NULL;
- pdict = qemu_opts_to_qdict(opts, NULL);
+ pdict = qemu_opts_to_qdict(opts, NULL,
+ QEMU_OPTS_REPEAT_POLICY_LAST,
+ errp);
if (!pdict) {
goto cleanup;
}
diff --git a/qemu-img.c b/qemu-img.c
index 851422a..f47ea75 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -273,7 +273,9 @@ static BlockBackend *img_open_opts(const char *optstr,
QDict *options;
Error *local_err = NULL;
BlockBackend *blk;
- options = qemu_opts_to_qdict(opts, NULL);
+ options = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
+
blk = blk_new_open(NULL, NULL, options, flags, &local_err);
if (!blk) {
error_reportf_err(local_err, "Could not open '%s': ", optstr);
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index 3a3838a..ce654f4 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -1952,7 +1952,8 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv)
}
qopts = qemu_opts_find(&reopen_opts, NULL);
- opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
+ opts = qopts ? qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort) : NULL;
qemu_opts_reset(&reopen_opts);
brq = bdrv_reopen_queue(NULL, bs, opts, flags);
diff --git a/qemu-io.c b/qemu-io.c
index db129ea..bb374a6 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -207,7 +207,8 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
}
qopts = qemu_opts_find(&empty_opts, NULL);
- opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
+ opts = qopts ? qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort) : NULL;
qemu_opts_reset(&empty_opts);
if (optind == argc - 1) {
@@ -593,7 +594,8 @@ int main(int argc, char **argv)
if (!qopts) {
exit(1);
}
- opts = qemu_opts_to_qdict(qopts, NULL);
+ opts = qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
openfile(NULL, flags, writethrough, opts);
} else {
if (format) {
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 99297a5..73b40b1 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -859,7 +859,8 @@ int main(int argc, char **argv)
qemu_opts_reset(&file_opts);
exit(EXIT_FAILURE);
}
- options = qemu_opts_to_qdict(opts, NULL);
+ options = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
qemu_opts_reset(&file_opts);
blk = blk_new_open(NULL, NULL, options, flags, &local_err);
} else {
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index ded4d84..fdc406b 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -161,7 +161,8 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
Object *obj = NULL;
v = opts_visitor_new(opts);
- pdict = qemu_opts_to_qdict(opts, NULL);
+ pdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
obj = user_creatable_add(pdict, v, errp);
visit_free(v);
diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
index a505a3e..3b59978 100644
--- a/tests/test-qemu-opts.c
+++ b/tests/test-qemu-opts.c
@@ -421,6 +421,130 @@ static void test_qemu_opts_set(void)
g_assert(opts == NULL);
}
+
+static void test_qemu_opts_to_qdict_repeat_last(void)
+{
+ QemuOpts *opts;
+ QDict *dict;
+
+ /* dynamically initialized (parsed) opts */
+ opts = qemu_opts_parse(&opts_list_03,
+ "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
+ false, NULL);
+ g_assert(opts);
+
+ dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
+ g_assert(dict);
+
+
+ g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
+ g_assert_cmpstr(qdict_get_str(dict, "nodes"), ==, "1-2");
+ g_assert(!qdict_haskey(dict, "nodes.0"));
+ g_assert(!qdict_haskey(dict, "nodes.1"));
+ g_assert(!qdict_haskey(dict, "nodes.2"));
+ g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
+ QDECREF(dict);
+
+ qemu_opts_del(opts);
+ qemu_opts_reset(&opts_list_03);
+}
+
+
+static void test_qemu_opts_to_qdict_repeat_all(void)
+{
+ QemuOpts *opts;
+ QDict *dict;
+ QList *list;
+ const QListEntry *ent;
+ QString *str;
+
+ /* dynamically initialized (parsed) opts */
+ opts = qemu_opts_parse(&opts_list_03,
+ "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
+ false, NULL);
+ g_assert(opts);
+
+ dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ALL,
+ &error_abort);
+ g_assert(dict);
+
+ g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
+ g_assert(qdict_haskey(dict, "nodes"));
+ list = qobject_to_qlist(qdict_get(dict, "nodes"));
+ g_assert(list);
+
+ ent = qlist_first(list);
+ g_assert(ent);
+ str = qobject_to_qstring(ent->value);
+ g_assert(str);
+ g_assert_cmpstr(qstring_get_str(str), ==, "10");
+
+ ent = qlist_next(ent);
+ g_assert(ent);
+ str = qobject_to_qstring(ent->value);
+ g_assert(str);
+ g_assert_cmpstr(qstring_get_str(str), ==, "4-5");
+
+ ent = qlist_next(ent);
+ g_assert(ent);
+ str = qobject_to_qstring(ent->value);
+ g_assert(str);
+ g_assert_cmpstr(qstring_get_str(str), ==, "1-2");
+
+ ent = qlist_next(ent);
+ g_assert(!ent);
+
+ g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
+ QDECREF(dict);
+
+ qemu_opts_del(opts);
+ qemu_opts_reset(&opts_list_03);
+}
+
+static void test_qemu_opts_to_qdict_repeat_err_fail(void)
+{
+ QemuOpts *opts;
+ QDict *dict;
+ Error *err = NULL;
+
+ /* dynamically initialized (parsed) opts */
+ opts = qemu_opts_parse(&opts_list_03,
+ "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
+ false, NULL);
+ g_assert(opts);
+
+ dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ERROR,
+ &err);
+ error_free_or_abort(&err);
+ g_assert(!dict);
+
+ qemu_opts_del(opts);
+ qemu_opts_reset(&opts_list_03);
+}
+
+static void test_qemu_opts_to_qdict_repeat_err_ok(void)
+{
+ QemuOpts *opts;
+ QDict *dict;
+
+ /* dynamically initialized (parsed) opts */
+ opts = qemu_opts_parse(&opts_list_03,
+ "size=1024,nodes=10,policy=bind",
+ false, NULL);
+ g_assert(opts);
+
+ dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ERROR,
+ &error_abort);
+ g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
+ g_assert_cmpstr(qdict_get_str(dict, "nodes"), ==, "10");
+ g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
+
+ QDECREF(dict);
+ qemu_opts_del(opts);
+ qemu_opts_reset(&opts_list_03);
+}
+
int main(int argc, char *argv[])
{
register_opts();
@@ -435,6 +559,14 @@ int main(int argc, char *argv[])
g_test_add_func("/qemu-opts/opt_unset", test_qemu_opt_unset);
g_test_add_func("/qemu-opts/opts_reset", test_qemu_opts_reset);
g_test_add_func("/qemu-opts/opts_set", test_qemu_opts_set);
+ g_test_add_func("/qemu-opts/to_qdict/repeat-last",
+ test_qemu_opts_to_qdict_repeat_last);
+ g_test_add_func("/qemu-opts/to_qdict/repeat-all",
+ test_qemu_opts_to_qdict_repeat_all);
+ g_test_add_func("/qemu-opts/to_qdict/repeat-err-fail",
+ test_qemu_opts_to_qdict_repeat_err_fail);
+ g_test_add_func("/qemu-opts/to_qdict/repeat-err-ok",
+ test_qemu_opts_to_qdict_repeat_err_ok);
g_test_run();
return 0;
}
diff --git a/tests/test-replication.c b/tests/test-replication.c
index 0997bd8..e267f9a 100644
--- a/tests/test-replication.c
+++ b/tests/test-replication.c
@@ -181,7 +181,8 @@ static BlockBackend *start_primary(void)
opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
g_free(cmdline);
- qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
@@ -311,7 +312,8 @@ static BlockBackend *start_secondary(void)
opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
g_free(cmdline);
- qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
@@ -336,7 +338,8 @@ static BlockBackend *start_secondary(void)
opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
g_free(cmdline);
- qdict = qemu_opts_to_qdict(opts, NULL);
+ qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ &error_abort);
qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
diff --git a/util/qemu-option.c b/util/qemu-option.c
index 418cbb6..0129ede 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -1057,23 +1057,73 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
* The QDict values are of type QString.
* TODO We'll want to use types appropriate for opt->desc->type, but
* this is enough for now.
+ *
+ * If the @opts contains multiple occurrences of the same key,
+ * then the @repeatPolicy parameter determines how they are to
+ * be handled. Traditional behaviour was to only store the
+ * last occurrence, but if @repeatPolicy is set to
+ * QEMU_OPTS_REPEAT_POLICY_ALL, then a QList will be returned
+ * containing all values, for any key with multiple occurrences.
+ * The QEMU_OPTS_REPEAT_POLICY_ERROR value can be used to fail
+ * conversion with an error if multiple occurrences of a key
+ * are seen.
*/
-QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict)
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
+ QemuOptsRepeatPolicy repeatPolicy,
+ Error **errp)
{
QemuOpt *opt;
- QObject *val;
+ QObject *val, *prevval;
+ QList *list;
+ QDict *ret;
- if (!qdict) {
- qdict = qdict_new();
+ if (qdict) {
+ ret = qdict;
+ } else {
+ ret = qdict_new();
}
if (opts->id) {
- qdict_put(qdict, "id", qstring_from_str(opts->id));
+ qdict_put(ret, "id", qstring_from_str(opts->id));
}
QTAILQ_FOREACH(opt, &opts->head, next) {
val = QOBJECT(qstring_from_str(opt->str));
- qdict_put_obj(qdict, opt->name, val);
+
+ if (qdict_haskey(ret, opt->name)) {
+ switch (repeatPolicy) {
+ case QEMU_OPTS_REPEAT_POLICY_ERROR:
+ error_setg(errp, "Option '%s' appears more than once",
+ opt->name);
+ qobject_decref(val);
+ if (!qdict) {
+ QDECREF(ret);
+ }
+ return NULL;
+
+ case QEMU_OPTS_REPEAT_POLICY_ALL:
+ prevval = qdict_get(ret, opt->name);
+ if (qobject_type(prevval) == QTYPE_QLIST) {
+ /* Already identified this key as a list */
+ list = qobject_to_qlist(prevval);
+ } else {
+ /* Replace current scalar with a list */
+ list = qlist_new();
+ qobject_incref(prevval);
+ qlist_append_obj(list, prevval);
+ qdict_put_obj(ret, opt->name, QOBJECT(list));
+ }
+ qlist_append_obj(list, val);
+ break;
+
+ case QEMU_OPTS_REPEAT_POLICY_LAST:
+ /* Just discard previously set value */
+ qdict_put_obj(ret, opt->name, val);
+ break;
+ }
+ } else {
+ qdict_put_obj(ret, opt->name, val);
+ }
}
- return qdict;
+ return ret;
}
/* Validate parsed opts against descriptions where no
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (11 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-13 12:35 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts Daniel P. Berrange
` (9 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Currently qdict_crumple requires a totally flat QDict as its
input. i.e. all values in the QDict must be scalar types.
In order to have backwards compatibility with the OptsVisitor,
qemu_opt_to_qdict() has a new mode where it may return a QList
for values in the QDict, if there was a repeated key. We thus
need to allow compound types to appear as values in the input
dict given to qdict_crumple().
To avoid confusion, we sanity check that the user has not mixed
the old and new syntax at the same time. e.g. these are allowed
foo=hello,foo=world,foo=wibble
foo.0=hello,foo.1=world,foo.2=wibble
but this is forbidden
foo=hello,foo=world,foo.2=wibble
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
qobject/qdict.c | 45 +++++++++-----
tests/check-qdict.c | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 204 insertions(+), 16 deletions(-)
diff --git a/qobject/qdict.c b/qobject/qdict.c
index c38e90e..83384b2 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -814,15 +814,16 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
/**
* qdict_crumple:
- * @src: the original flat dictionary (only scalar values) to crumple
+ * @src: the original dictionary to crumple
* @recursive: true to recursively crumple nested dictionaries
*
- * Takes a flat dictionary whose keys use '.' separator to indicate
- * nesting, and values are scalars, and crumples it into a nested
- * structure. If the @recursive parameter is false, then only the
- * first level of structure implied by the keys will be crumpled. If
- * @recursive is true, then the input will be recursively crumpled to
- * expand all levels of structure in the keys.
+ * Takes a dictionary whose keys use '.' separator to indicate
+ * nesting, crumples it into a nested structure. The values in
+ * @src are permitted to be scalars or compound typee. If the
+ * @recursive parameter is false, then only the first level of
+ * structure implied by the keys will be crumpled. If @recursive
+ * is true, then the input will be recursively crumpled to expand
+ * all levels of structure in the keys.
*
* To include a literal '.' in a key name, it must be escaped as '..'
*
@@ -843,7 +844,10 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
* The following scenarios in the input dict will result in an
* error being returned:
*
- * - Any values in @src are non-scalar types
+ * - If a key in @src has a value that is non-scalar, and
+ * a later key implies creation of a compound type. e.g.
+ * if 'foo' point to a 'QList' or 'QDict', then it is
+ * not permitted to also have 'foo.NNN' keys later.
* - If keys in @src imply that a particular level is both a
* list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
* - If keys in @src imply that a particular level is a list,
@@ -864,21 +868,26 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
char *prefix = NULL;
const char *suffix = NULL;
int is_list;
+ QDict *compound;
+ compound = qdict_new();
two_level = qdict_new();
/* Step 1: split our totally flat dict into a two level dict */
for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT ||
- qobject_type(ent->value) == QTYPE_QLIST) {
- error_setg(errp, "Value %s is not a scalar",
- ent->key);
- goto error;
- }
-
qdict_split_flat_key(ent->key, &prefix, &suffix);
child = qdict_get(two_level, prefix);
+
+ if (qdict_haskey(compound, prefix)) {
+ error_setg(errp, "Key %s is already set as a list/dict", prefix);
+ goto error;
+ }
+ if (qobject_type(ent->value) == QTYPE_QLIST ||
+ qobject_type(ent->value) == QTYPE_QDICT) {
+ qobject_incref(ent->value);
+ qdict_put_obj(compound, prefix, ent->value);
+ }
if (suffix) {
if (child) {
if (qobject_type(child) != QTYPE_QDICT) {
@@ -913,7 +922,8 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
for (ent = qdict_first(two_level); ent != NULL;
ent = qdict_next(two_level, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT) {
+ if (qobject_type(ent->value) == QTYPE_QDICT &&
+ !qdict_haskey(compound, ent->key)) {
child = qdict_crumple(qobject_to_qdict(ent->value),
recursive, errp);
if (!child) {
@@ -961,10 +971,13 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
dst = QOBJECT(multi_level);
}
+ QDECREF(compound);
+
return dst;
error:
g_free(prefix);
+ QDECREF(compound);
QDECREF(multi_level);
QDECREF(two_level);
qobject_decref(dst);
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 64c33ab..29d7362 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -844,6 +844,173 @@ static void qdict_crumple_test_bad_inputs(void)
QDECREF(src);
}
+static void qdict_crumple_test_non_flat_list_ok(void)
+{
+ QDict *src, *dst, *vnc, *listen;
+ QObject *child, *res;
+ QList *list;
+ QString *match;
+
+ src = qdict_new();
+ list = qlist_new();
+ qlist_append(list, qstring_from_str("fred"));
+ qlist_append(list, qstring_from_str("bob"));
+
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "match", list);
+
+ res = qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 2);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qobject_to_qdict(child);
+
+ child = qdict_get(vnc, "listen");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ listen = qobject_to_qdict(child);
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
+
+ child = qdict_get(dst, "match");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+ list = qobject_to_qlist(child);
+
+ g_assert_cmpint(qlist_size(list), ==, 2);
+
+ child = qlist_pop(list);
+ g_assert(child);
+ match = qobject_to_qstring(child);
+ g_assert_cmpstr("fred", ==, qstring_get_str(match));
+ qobject_decref(child);
+
+ child = qlist_pop(list);
+ g_assert(child);
+ match = qobject_to_qstring(child);
+ g_assert_cmpstr("bob", ==, qstring_get_str(match));
+ qobject_decref(child);
+
+ child = qlist_pop(list);
+ g_assert(!child);
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_non_flat_list_bad(void)
+{
+ QDict *src;
+ QObject *res;
+ QList *list;
+ Error *err = NULL;
+
+ src = qdict_new();
+ list = qlist_new();
+ qlist_append(list, qstring_from_str("fred"));
+ qlist_append(list, qstring_from_str("bob"));
+
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "match.0", qstring_from_str("frank"));
+ qdict_put(src, "match", list);
+
+ res = qdict_crumple(src, false, &err);
+ g_assert(!res);
+ error_free_or_abort(&err);
+
+ QDECREF(src);
+}
+
+
+static void qdict_crumple_test_non_flat_dict_ok(void)
+{
+ QDict *src, *dst, *rule, *vnc, *acl;
+ QObject *child, *res;
+ QList *rules;
+
+ vnc = qdict_new();
+ qdict_put(vnc, "listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(vnc, "listen.port", qstring_from_str("5901"));
+
+ src = qdict_new();
+ qdict_put(src, "vnc", vnc);
+ qdict_put(src, "acl.rules.0.match", qstring_from_str("fred"));
+ qdict_put(src, "acl.rules.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "acl.rules.1.match", qstring_from_str("bob"));
+ qdict_put(src, "acl.rules.1.policy", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 2);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qobject_to_qdict(child);
+
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
+
+ child = qdict_get(dst, "acl");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ acl = qobject_to_qdict(child);
+
+ child = qdict_get(acl, "rules");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+ rules = qobject_to_qlist(child);
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_non_flat_dict_bad(void)
+{
+ QDict *src, *vnc;
+ QObject *res;
+ Error *err = NULL;
+
+ vnc = qdict_new();
+ qdict_put(vnc, "listen.addr", qstring_from_str("127.0.0.1"));
+
+ src = qdict_new();
+ qdict_put(src, "vnc", vnc);
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "acl.rules.0.match", qstring_from_str("fred"));
+ qdict_put(src, "acl.rules.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "acl.rules.1.match", qstring_from_str("bob"));
+ qdict_put(src, "acl.rules.1.policy", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, true, &err);
+ g_assert(!res);
+ error_free_or_abort(&err);
+
+ QDECREF(src);
+}
+
/*
* Errors test-cases
@@ -1002,6 +1169,14 @@ int main(int argc, char **argv)
qdict_crumple_test_empty);
g_test_add_func("/public/crumple/bad_inputs",
qdict_crumple_test_bad_inputs);
+ g_test_add_func("/public/crumple/non-flat-list-ok",
+ qdict_crumple_test_non_flat_list_ok);
+ g_test_add_func("/public/crumple/non-flat-list-bad",
+ qdict_crumple_test_non_flat_list_bad);
+ g_test_add_func("/public/crumple/non-flat-dict-ok",
+ qdict_crumple_test_non_flat_dict_ok);
+ g_test_add_func("/public/crumple/non-flat-dict-bad",
+ qdict_crumple_test_non_flat_dict_bad);
/* The Big one */
if (g_test_slow()) {
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (12 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-18 17:13 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object Daniel P. Berrange
` (8 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The qobject_input_visitor_new_opts() method gains a new
parameter to control whether it allows repeated option
keys in the input QemuOpts or not.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qobject-input-visitor.h | 6 ++++++
qapi/qobject-input-visitor.c | 5 ++++-
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index 242b767..bc5062a 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -112,6 +112,11 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
* qobject_input_visitor_new_autocast() method. See the docs
* of that method for further details on processing behaviour.
*
+ * If the @permit_repeated_opts parameter is true, then the input
+ * @opts is allowed to contain repeated keys and they will be
+ * turned into a QList. If it is false, then repeated keys will
+ * result in an error being reported.
+ *
* The returned input visitor should be released by calling
* visit_free() when no longer required.
*/
@@ -119,6 +124,7 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
size_t autocreate_struct_levels,
bool permit_int_ranges,
+ bool permit_repeated_opts,
Error **errp);
#endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 2287d11..5a3872c 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -753,6 +753,7 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
bool autocreate_list,
size_t autocreate_struct_levels,
bool permit_int_ranges,
+ bool permit_repeated_opts,
Error **errp)
{
QDict *pdict;
@@ -760,7 +761,9 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
Visitor *v = NULL;
pdict = qemu_opts_to_qdict(opts, NULL,
- QEMU_OPTS_REPEAT_POLICY_LAST,
+ permit_repeated_opts ?
+ QEMU_OPTS_REPEAT_POLICY_ALL :
+ QEMU_OPTS_REPEAT_POLICY_ERROR,
errp);
if (!pdict) {
goto cleanup;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (13 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-19 16:54 ` Markus Armbruster
2017-07-10 19:30 ` Manos Pitsidianakis
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add Daniel P. Berrange
` (7 subsequent siblings)
22 siblings, 2 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The current -object command line syntax only allows for
creation of objects with scalar properties, or a list
with a fixed scalar element type. Objects which have
properties that are represented as structs in the QAPI
schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor
is written. It simply iterates over the QemuOpts values
as a flat list. The support for lists is enabled by
allowing the same key to be repeated in the opts string.
The QObjectInputVisitor now has functionality that allows
it to replace OptsVisitor, maintaining full backwards
compatibility for previous CLI syntax, while also allowing
use of new syntax for structs.
Thus -object can now support non-scalar properties,
for example the QMP object:
{
"execute": "object-add",
"arguments": {
"qom-type": "demo",
"id": "demo0",
"parameters": {
"foo": [
{ "bar": "one", "wizz": "1" },
{ "bar": "two", "wizz": "2" }
]
}
}
}
Would be creatable via the CLI now using
$QEMU \
-object demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2
Notice that this syntax is intentionally compatible
with that currently used by block drivers. NB the
indentation seen here after line continuations should
not be used in reality, it is just for clarity in this
commit message.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
qapi/qobject-input-visitor.c | 2 +-
qom/object_interfaces.c | 37 ++++-
tests/check-qom-proplist.c | 367 ++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 397 insertions(+), 9 deletions(-)
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 5a3872c..f1030d5 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -204,7 +204,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
*obj = NULL;
}
- if (!qobj && (qiv->struct_level < qiv->autocreate_struct_levels)) {
+ if (!qobj && (qiv->struct_level <= qiv->autocreate_struct_levels)) {
/* Create a new dict that contains all the currently
* unvisited items */
if (tos) {
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index fdc406b..88a1e88 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -4,7 +4,8 @@
#include "qemu/module.h"
#include "qapi-visit.h"
#include "qapi/qobject-output-visitor.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qemu/option.h"
void user_creatable_complete(Object *obj, Error **errp)
{
@@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
if (local_err) {
goto out_visit;
}
- visit_check_struct(v, &local_err);
+
+ obj = user_creatable_add_type(type, id, pdict, v, &local_err);
if (local_err) {
goto out_visit;
}
- obj = user_creatable_add_type(type, id, pdict, v, &local_err);
+ visit_check_struct(v, &local_err);
+ if (local_err) {
+ goto out_visit;
+ }
out_visit:
visit_end_struct(v, NULL);
@@ -114,7 +119,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
assert(qdict);
obj = object_new(type);
- visit_start_struct(v, NULL, NULL, 0, &local_err);
+ visit_start_struct(v, "props", NULL, 0, &local_err);
if (local_err) {
goto out;
}
@@ -158,14 +163,32 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
{
Visitor *v;
QDict *pdict;
+ QObject *pobj;
Object *obj = NULL;
- v = opts_visitor_new(opts);
- pdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
+ pdict = qemu_opts_to_qdict(opts, NULL,
+ QEMU_OPTS_REPEAT_POLICY_ALL,
&error_abort);
- obj = user_creatable_add(pdict, v, errp);
+ pobj = qdict_crumple(pdict, true, errp);
+ if (!pobj) {
+ goto cleanup;
+ }
+ /*
+ * We need autocreate_list=true + permit_int_ranges=true
+ * in order to maintain compat with OptsVisitor creation
+ * of the 'numa' object which had an int16List property.
+ *
+ * We need autocreate_structs=1, because at the CLI level
+ * we have the object type + properties in the same flat
+ * struct, even though at the QMP level they are nested.
+ */
+ v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
+
+ obj = user_creatable_add(qobject_to_qdict(pobj), v, errp);
visit_free(v);
+ qobject_decref(pobj);
+ cleanup:
QDECREF(pdict);
return obj;
}
diff --git a/tests/check-qom-proplist.c b/tests/check-qom-proplist.c
index a16cefc..20eb1d1 100644
--- a/tests/check-qom-proplist.c
+++ b/tests/check-qom-proplist.c
@@ -22,7 +22,16 @@
#include "qapi/error.h"
#include "qom/object.h"
+#include "qom/object_interfaces.h"
#include "qemu/module.h"
+#include "qapi/visitor.h"
+#include "qom/object_interfaces.h"
+#include "qemu/option.h"
+#include "qemu/config-file.h"
+#include "qapi/qmp/qjson.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qapi-visit.h"
+#include "qapi/dealloc-visitor.h"
#define TYPE_DUMMY "qemu-dummy"
@@ -30,6 +39,11 @@
typedef struct DummyObject DummyObject;
typedef struct DummyObjectClass DummyObjectClass;
+typedef struct DummyPerson DummyPerson;
+typedef struct DummyAddr DummyAddr;
+typedef struct DummyAddrList DummyAddrList;
+typedef struct DummySizeList DummySizeList;
+
#define DUMMY_OBJECT(obj) \
OBJECT_CHECK(DummyObject, (obj), TYPE_DUMMY)
@@ -50,12 +64,35 @@ static const char *const dummy_animal_map[DUMMY_LAST + 1] = {
[DUMMY_LAST] = NULL,
};
+
+struct DummyAddr {
+ char *ip;
+ int64_t prefix;
+ bool ipv6only;
+};
+
+struct DummyAddrList {
+ DummyAddrList *next;
+ struct DummyAddr *value;
+};
+
+struct DummyPerson {
+ char *name;
+ int64_t age;
+};
+
struct DummyObject {
Object parent_obj;
bool bv;
DummyAnimal av;
char *sv;
+
+ intList *sizes;
+
+ DummyPerson *person;
+
+ DummyAddrList *addrs;
};
struct DummyObjectClass {
@@ -117,6 +154,157 @@ static char *dummy_get_sv(Object *obj,
return g_strdup(dobj->sv);
}
+static void visit_type_DummyPerson_fields(Visitor *v, DummyPerson **obj,
+ Error **errp)
+{
+ Error *err = NULL;
+
+ visit_type_str(v, "name", &(*obj)->name, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_int(v, "age", &(*obj)->age, &err);
+ if (err) {
+ goto out;
+ }
+
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyPerson(Visitor *v, const char *name,
+ DummyPerson **obj, Error **errp)
+{
+ Error *err = NULL;
+
+ visit_start_struct(v, name, (void **)obj, sizeof(DummyPerson), &err);
+ if (err) {
+ goto out;
+ }
+ if (!*obj) {
+ goto out_obj;
+ }
+ visit_type_DummyPerson_fields(v, obj, &err);
+ error_propagate(errp, err);
+ err = NULL;
+out_obj:
+ visit_end_struct(v, (void **)obj);
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyAddr_members(Visitor *v, DummyAddr **obj,
+ Error **errp)
+{
+ Error *err = NULL;
+
+ visit_type_str(v, "ip", &(*obj)->ip, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_int(v, "prefix", &(*obj)->prefix, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_bool(v, "ipv6only", &(*obj)->ipv6only, &err);
+ if (err) {
+ goto out;
+ }
+
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyAddr(Visitor *v, const char *name,
+ DummyAddr **obj, Error **errp)
+{
+ Error *err = NULL;
+
+ visit_start_struct(v, name, (void **)obj, sizeof(DummyAddr), &err);
+ if (err) {
+ goto out;
+ }
+ if (!*obj) {
+ goto out_obj;
+ }
+ visit_type_DummyAddr_members(v, obj, &err);
+ error_propagate(errp, err);
+ err = NULL;
+out_obj:
+ visit_end_struct(v, (void **)obj);
+out:
+ error_propagate(errp, err);
+}
+
+static void qapi_free_DummyAddrList(DummyAddrList *obj);
+
+static void visit_type_DummyAddrList(Visitor *v, const char *name,
+ DummyAddrList **obj, Error **errp)
+{
+ Error *err = NULL;
+ DummyAddrList *tail;
+ size_t size = sizeof(**obj);
+
+ visit_start_list(v, name, (GenericList **)obj, size, &err);
+ if (err) {
+ goto out;
+ }
+
+ for (tail = *obj; tail;
+ tail = (DummyAddrList *)visit_next_list(v,
+ (GenericList *)tail,
+ size)) {
+ visit_type_DummyAddr(v, NULL, &tail->value, &err);
+ if (err) {
+ break;
+ }
+ }
+
+ visit_end_list(v, (void **)obj);
+ if (err && visit_is_input(v)) {
+ qapi_free_DummyAddrList(*obj);
+ *obj = NULL;
+ }
+out:
+ error_propagate(errp, err);
+}
+
+static void qapi_free_DummyAddrList(DummyAddrList *obj)
+{
+ Visitor *v;
+
+ if (!obj) {
+ return;
+ }
+
+ v = qapi_dealloc_visitor_new();
+ visit_type_DummyAddrList(v, NULL, &obj, NULL);
+ visit_free(v);
+}
+
+static void dummy_set_sizes(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_intList(v, name, &dobj->sizes, errp);
+}
+
+static void dummy_set_person(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_DummyPerson(v, name, &dobj->person, errp);
+}
+
+static void dummy_set_addrs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_DummyAddrList(v, name, &dobj->addrs, errp);
+}
static void dummy_init(Object *obj)
{
@@ -126,9 +314,16 @@ static void dummy_init(Object *obj)
NULL);
}
+static void
+dummy_complete(UserCreatable *uc, Error **errp)
+{
+}
static void dummy_class_init(ObjectClass *cls, void *data)
{
+ UserCreatableClass *ucc = USER_CREATABLE_CLASS(cls);
+ ucc->complete = dummy_complete;
+
object_class_property_add_bool(cls, "bv",
dummy_get_bv,
dummy_set_bv,
@@ -143,12 +338,34 @@ static void dummy_class_init(ObjectClass *cls, void *data)
dummy_get_av,
dummy_set_av,
NULL);
+ object_class_property_add(cls, "sizes",
+ "int[]",
+ NULL,
+ dummy_set_sizes,
+ NULL, NULL, NULL);
+ object_class_property_add(cls, "person",
+ "DummyPerson",
+ NULL,
+ dummy_set_person,
+ NULL, NULL, NULL);
+ object_class_property_add(cls, "addrs",
+ "DummyAddrList",
+ NULL,
+ dummy_set_addrs,
+ NULL, NULL, NULL);
}
static void dummy_finalize(Object *obj)
{
DummyObject *dobj = DUMMY_OBJECT(obj);
+ Visitor *v;
+
+ v = qapi_dealloc_visitor_new();
+ visit_type_intList(v, NULL, &dobj->sizes, NULL);
+ visit_type_DummyAddrList(v, NULL, &dobj->addrs, NULL);
+ visit_type_DummyPerson(v, NULL, &dobj->person, NULL);
+ visit_free(v);
g_free(dobj->sv);
}
@@ -162,6 +379,10 @@ static const TypeInfo dummy_info = {
.instance_finalize = dummy_finalize,
.class_size = sizeof(DummyObjectClass),
.class_init = dummy_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_USER_CREATABLE },
+ { }
+ }
};
@@ -388,6 +609,128 @@ static void test_dummy_createlist(void)
object_unparent(OBJECT(dobj));
}
+
+static QemuOptsList dummy_opts = {
+ .name = "object",
+ .implied_opt_name = "qom-type",
+ .head = QTAILQ_HEAD_INITIALIZER(dummy_opts.head),
+ .desc = {
+ { }
+ },
+};
+
+static void test_dummy_create_complex(DummyObject *dummy)
+{
+ g_assert(dummy->person != NULL);
+ g_assert_cmpstr(dummy->person->name, ==, "fred");
+ g_assert_cmpint(dummy->person->age, ==, 52);
+
+ g_assert(dummy->sizes != NULL);
+ g_assert_cmpint(dummy->sizes->value, ==, 12);
+ g_assert_cmpint(dummy->sizes->next->value, ==, 13);
+ g_assert_cmpint(dummy->sizes->next->next->value, ==, 8139);
+
+ g_assert(dummy->addrs != NULL);
+ g_assert_cmpstr(dummy->addrs->value->ip, ==, "127.0.0.1");
+ g_assert_cmpint(dummy->addrs->value->prefix, ==, 24);
+ g_assert(dummy->addrs->value->ipv6only);
+ g_assert_cmpstr(dummy->addrs->next->value->ip, ==, "0.0.0.0");
+ g_assert_cmpint(dummy->addrs->next->value->prefix, ==, 16);
+ g_assert(!dummy->addrs->next->value->ipv6only);
+}
+
+
+static void _test_dummy_createopts(const char *optstr)
+{
+ QemuOpts *opts;
+ DummyObject *dummy;
+
+ opts = qemu_opts_parse_noisily(&dummy_opts,
+ optstr, true);
+ g_assert(opts != NULL);
+
+ dummy = DUMMY_OBJECT(user_creatable_add_opts(opts, &error_abort));
+
+ test_dummy_create_complex(dummy);
+
+ object_unparent(OBJECT(dummy));
+ object_unref(OBJECT(dummy));
+ qemu_opts_reset(&dummy_opts);
+}
+
+
+static void test_dummy_createopts(void)
+{
+ const char *optstr = "qemu-dummy,id=dummy0,bv=yes,av=alligator,sv=hiss,"
+ "person.name=fred,person.age=52,sizes.0=12,sizes.1=13,sizes.2=8139,"
+ "addrs.0.ip=127.0.0.1,addrs.0.prefix=24,addrs.0.ipv6only=yes,"
+ "addrs.1.ip=0.0.0.0,addrs.1.prefix=16,addrs.1.ipv6only=no";
+ _test_dummy_createopts(optstr);
+}
+
+static void test_dummy_createopts_repeat(void)
+{
+ const char *optstr = "qemu-dummy,id=dummy0,bv=yes,av=alligator,sv=hiss,"
+ "person.name=fred,person.age=52,sizes=12,sizes=13,sizes=8139,"
+ "addrs.0.ip=127.0.0.1,addrs.0.prefix=24,addrs.0.ipv6only=yes,"
+ "addrs.1.ip=0.0.0.0,addrs.1.prefix=16,addrs.1.ipv6only=no";
+ _test_dummy_createopts(optstr);
+}
+
+static void test_dummy_createopts_range(void)
+{
+ const char *optstr = "qemu-dummy,id=dummy0,bv=yes,av=alligator,sv=hiss,"
+ "person.name=fred,person.age=52,sizes=12-13,sizes=8139,"
+ "addrs.0.ip=127.0.0.1,addrs.0.prefix=24,addrs.0.ipv6only=yes,"
+ "addrs.1.ip=0.0.0.0,addrs.1.prefix=16,addrs.1.ipv6only=no";
+ _test_dummy_createopts(optstr);
+}
+
+
+static void test_dummy_createopts_bad(void)
+{
+ /* Something that tries to create a QList at the top level
+ * should be invalid. */
+ const char *optstr = "qemu-dummy,id=dummy0,1=foo,2=bar,3=wizz";
+ QemuOpts *opts;
+ DummyObject *dummy;
+ Error *err = NULL;
+
+ opts = qemu_opts_parse_noisily(&dummy_opts,
+ optstr, true);
+ g_assert(opts != NULL);
+
+ dummy = DUMMY_OBJECT(user_creatable_add_opts(opts, &err));
+ error_free_or_abort(&err);
+ g_assert(!dummy);
+ qemu_opts_reset(&dummy_opts);
+}
+
+
+static void test_dummy_createqmp(void)
+{
+ const char *jsonstr =
+ "{ 'bv': true, 'av': 'alligator', 'sv': 'hiss', "
+ " 'person': { 'name': 'fred', 'age': 52 }, "
+ " 'sizes': [12, 13, 8139], "
+ " 'addrs': [ { 'ip': '127.0.0.1', 'prefix': 24, 'ipv6only': true }, "
+ " { 'ip': '0.0.0.0', 'prefix': 16, 'ipv6only': false } ] }";
+ QObject *obj = qobject_from_json(jsonstr);
+ Visitor *v = qobject_input_visitor_new(obj, true);
+ DummyObject *dummy;
+ g_assert(obj);
+ dummy = DUMMY_OBJECT(user_creatable_add_type("qemu-dummy", "dummy0",
+ qobject_to_qdict(obj), v,
+ &error_abort));
+
+ test_dummy_create_complex(dummy);
+ visit_free(v);
+ object_unparent(OBJECT(dummy));
+ object_unref(OBJECT(dummy));
+ qobject_decref(obj);
+}
+
+
static void test_dummy_badenum(void)
{
Error *err = NULL;
@@ -473,7 +816,9 @@ static void test_dummy_iterator(void)
ObjectProperty *prop;
ObjectPropertyIterator iter;
- bool seenbv = false, seensv = false, seenav = false, seentype;
+ bool seenbv = false, seensv = false, seenav = false,
+ seentype = false, seenaddrs = false, seenperson = false,
+ seensizes = false;
object_property_iter_init(&iter, OBJECT(dobj));
while ((prop = object_property_iter_next(&iter))) {
@@ -486,6 +831,12 @@ static void test_dummy_iterator(void)
} else if (g_str_equal(prop->name, "type")) {
/* This prop comes from the base Object class */
seentype = true;
+ } else if (g_str_equal(prop->name, "addrs")) {
+ seenaddrs = true;
+ } else if (g_str_equal(prop->name, "person")) {
+ seenperson = true;
+ } else if (g_str_equal(prop->name, "sizes")) {
+ seensizes = true;
} else {
g_printerr("Found prop '%s'\n", prop->name);
g_assert_not_reached();
@@ -495,6 +846,9 @@ static void test_dummy_iterator(void)
g_assert(seenav);
g_assert(seensv);
g_assert(seentype);
+ g_assert(seenaddrs);
+ g_assert(seenperson);
+ g_assert(seensizes);
object_unparent(OBJECT(dobj));
}
@@ -513,11 +867,15 @@ static void test_dummy_delchild(void)
object_unparent(OBJECT(dev));
}
+
int main(int argc, char **argv)
{
g_test_init(&argc, &argv, NULL);
module_call_init(MODULE_INIT_QOM);
+
+ qemu_add_opts(&dummy_opts);
+
type_register_static(&dummy_info);
type_register_static(&dummy_dev_info);
type_register_static(&dummy_bus_info);
@@ -525,6 +883,13 @@ int main(int argc, char **argv)
g_test_add_func("/qom/proplist/createlist", test_dummy_createlist);
g_test_add_func("/qom/proplist/createv", test_dummy_createv);
+ g_test_add_func("/qom/proplist/createopts", test_dummy_createopts);
+ g_test_add_func("/qom/proplist/createoptsrepeat",
+ test_dummy_createopts_repeat);
+ g_test_add_func("/qom/proplist/createoptsrange",
+ test_dummy_createopts_range);
+ g_test_add_func("/qom/proplist/createoptsbad", test_dummy_createopts_bad);
+ g_test_add_func("/qom/proplist/createqmp", test_dummy_createqmp);
g_test_add_func("/qom/proplist/badenum", test_dummy_badenum);
g_test_add_func("/qom/proplist/getenum", test_dummy_getenum);
g_test_add_func("/qom/proplist/iterator", test_dummy_iterator);
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (14 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-20 6:43 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa Daniel P. Berrange
` (6 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The current object_add HMP syntax only allows for
creation of objects with scalar properties, or a list
with a fixed scalar element type. Objects which have
properties that are represented as structs in the QAPI
schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor
is written. It simply iterates over the QemuOpts values
as a flat list. The support for lists is enabled by
allowing the same key to be repeated in the opts string.
The QObjectInputVisitor now has functionality that allows
it to replace OptsVisitor, maintaining full backwards
compatibility for previous CLI syntax, while also allowing
use of new syntax for structs.
Thus -object can now support non-scalar properties,
for example the QMP object:
{
"execute": "object-add",
"arguments": {
"qom-type": "demo",
"id": "demo0",
"parameters": {
"foo": [
{ "bar": "one", "wizz": "1" },
{ "bar": "two", "wizz": "2" }
]
}
}
}
Would be creatable via the HMP now using
object_add demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2
Notice that this syntax is intentionally compatible
with that currently used by block drivers. NB the
indentation seen here after line continuations should
not be used in reality, it is just for clarity in this
commit message.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
hmp.c | 25 +++++++++++++++++--------
1 file changed, 17 insertions(+), 8 deletions(-)
diff --git a/hmp.c b/hmp.c
index 336e7bf..b32d8c8 100644
--- a/hmp.c
+++ b/hmp.c
@@ -25,7 +25,7 @@
#include "qemu/sockets.h"
#include "monitor/monitor.h"
#include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi/qmp/qerror.h"
#include "qapi/string-output-visitor.h"
#include "qapi/util.h"
@@ -1696,21 +1696,30 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
void hmp_object_add(Monitor *mon, const QDict *qdict)
{
Error *err = NULL;
- QemuOpts *opts;
Visitor *v;
Object *obj = NULL;
+ QObject *pobj;
- opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
+ pobj = qdict_crumple(qdict, true, &err);
if (err) {
- hmp_handle_error(mon, &err);
- return;
+ goto cleanup;
}
- v = opts_visitor_new(opts);
- obj = user_creatable_add(qdict, v, &err);
+ /*
+ * We need autocreate_list=true + permit_int_ranges=true
+ * in order to maintain compat with OptsVisitor creation
+ * of the 'numa' object which had an int16List property.
+ *
+ * We need autocreate_structs=1, because at the CLI level
+ * we have the object type + properties in the same flat
+ * struct, even though at the QMP level they are nested.
+ */
+ v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
+ obj = user_creatable_add(qobject_to_qdict(pobj), v, &err);
visit_free(v);
- qemu_opts_del(opts);
+ cleanup:
+ qobject_decref(pobj);
if (err) {
hmp_handle_error(mon, &err);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (15 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-20 6:57 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 18/21] block: convert crypto driver to use QObjectInputVisitor Daniel P. Berrange
` (5 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Switch away from using OptsVisitor to parse the -numa
argument processing. This enables use of the modern
list syntax for specifying CPUs. e.g. the old syntax
-numa node,nodeid=0,cpus=0-3,cpus=8-11,mem=107
is equivalent to
-numa node,nodeid=0,cpus.0=0,cpus.1=1,cpus.2=2,cpus.3=3,\
cpus.4=8,cpus.5=9,cpus.6=10,cpus.7=11,mem=107
Furthermore, the cli arg can now follow the QAPI schema
nesting, so the above is equivalent to the canonical
syntax:
-numa type=node,data.nodeid=0,data.cpus.0=0,data.cpus.1=1,\
data.cpus.2=2,data.cpus.3=3,data.cpus.4=8,data.cpus.5=9,\
data.cpus.6=10,data.cpus.7=11,data.mem=107
A test case is added to cover argument parsing to validate
that both the old and new syntax is correctly handled.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/sysemu/numa_int.h | 11 +++++
numa.c | 36 +++++++++-----
stubs/Makefile.objs | 5 ++
stubs/exec.c | 6 +++
stubs/hostmem.c | 14 ++++++
stubs/memory.c | 41 ++++++++++++++++
stubs/qdev.c | 8 ++++
stubs/vl.c | 8 ++++
stubs/vmstate.c | 4 ++
tests/Makefile.include | 2 +
tests/test-numa.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++
11 files changed, 240 insertions(+), 11 deletions(-)
create mode 100644 include/sysemu/numa_int.h
create mode 100644 stubs/exec.c
create mode 100644 stubs/hostmem.c
create mode 100644 stubs/memory.c
create mode 100644 stubs/qdev.c
create mode 100644 stubs/vl.c
create mode 100644 tests/test-numa.c
diff --git a/include/sysemu/numa_int.h b/include/sysemu/numa_int.h
new file mode 100644
index 0000000..93160da
--- /dev/null
+++ b/include/sysemu/numa_int.h
@@ -0,0 +1,11 @@
+#ifndef SYSEMU_NUMA_INT_H
+#define SYSEMU_NUMA_INT_H
+
+#include "sysemu/numa.h"
+
+extern int have_memdevs;
+extern int max_numa_nodeid;
+
+int parse_numa(void *opaque, QemuOpts *opts, Error **errp);
+
+#endif
diff --git a/numa.c b/numa.c
index 6289f46..653ebf1 100644
--- a/numa.c
+++ b/numa.c
@@ -23,14 +23,14 @@
*/
#include "qemu/osdep.h"
-#include "sysemu/numa.h"
+#include "sysemu/numa_int.h"
#include "exec/cpu-common.h"
#include "qemu/bitmap.h"
#include "qom/cpu.h"
#include "qemu/error-report.h"
#include "include/exec/cpu-common.h" /* for RAM_ADDR_FMT */
#include "qapi-visit.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "hw/boards.h"
#include "sysemu/hostmem.h"
#include "qmp-commands.h"
@@ -45,10 +45,10 @@ QemuOptsList qemu_numa_opts = {
.desc = { { 0 } } /* validated with OptsVisitor */
};
-static int have_memdevs = -1;
-static int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
- * For all nodes, nodeid < max_numa_nodeid
- */
+int have_memdevs = -1;
+int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
+ * For all nodes, nodeid < max_numa_nodeid
+ */
int nb_numa_nodes;
NodeInfo numa_info[MAX_NODES];
@@ -189,6 +189,9 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
if (node->has_mem) {
uint64_t mem_size = node->mem;
const char *mem_str = qemu_opt_get(opts, "mem");
+ if (!mem_str) {
+ mem_str = qemu_opt_get(opts, "data.mem");
+ }
/* Fix up legacy suffix-less format */
if (g_ascii_isdigit(mem_str[strlen(mem_str) - 1])) {
mem_size <<= 20;
@@ -211,16 +214,27 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
max_numa_nodeid = MAX(max_numa_nodeid, nodenr + 1);
}
-static int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
+int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
{
NumaOptions *object = NULL;
Error *err = NULL;
+ Visitor *v;
- {
- Visitor *v = opts_visitor_new(opts);
- visit_type_NumaOptions(v, NULL, &object, &err);
- visit_free(v);
+ /*
+ * Needs autocreate_list=true, permit_int_ranges=true and
+ * permit_repeated_opts=true in order to support existing
+ * syntax for 'cpus' parameter which is an int list.
+ *
+ * Needs autocreate_struct_levels=1, because existing syntax
+ * used a nested struct in the QMP schema with a flat namespace
+ * in the CLI args.
+ */
+ v = qobject_input_visitor_new_opts(opts, true, 1, true, true, &err);
+ if (err) {
+ goto end;
}
+ visit_type_NumaOptions(v, NULL, &object, &err);
+ visit_free(v);
if (err) {
goto end;
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index c5850e8..661b48a 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -48,3 +48,8 @@ stub-obj-y += iohandler.o
stub-obj-y += smbios_type_38.o
stub-obj-y += ipmi.o
stub-obj-y += pc_madt_cpu_entry.o
+stub-obj-y += vl.o
+stub-obj-y += exec.o
+stub-obj-y += memory.o
+stub-obj-y += hostmem.o
+stub-obj-y += qdev.o
diff --git a/stubs/exec.c b/stubs/exec.c
new file mode 100644
index 0000000..e37f002
--- /dev/null
+++ b/stubs/exec.c
@@ -0,0 +1,6 @@
+
+#include "qemu/osdep.h"
+#include "exec/cpu-common.h"
+#include "qom/cpu.h"
+
+struct CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
diff --git a/stubs/hostmem.c b/stubs/hostmem.c
new file mode 100644
index 0000000..301d853
--- /dev/null
+++ b/stubs/hostmem.c
@@ -0,0 +1,14 @@
+
+#include "qemu/osdep.h"
+#include "sysemu/hostmem.h"
+
+void host_memory_backend_set_mapped(HostMemoryBackend *backend, bool mapped)
+{
+}
+
+
+MemoryRegion *host_memory_backend_get_memory(HostMemoryBackend *backend,
+ Error **errp)
+{
+ return NULL;
+}
diff --git a/stubs/memory.c b/stubs/memory.c
new file mode 100644
index 0000000..c849d9d
--- /dev/null
+++ b/stubs/memory.c
@@ -0,0 +1,41 @@
+
+#include "qemu/osdep.h"
+#include "exec/memory.h"
+
+void memory_region_init_ram(MemoryRegion *mr,
+ struct Object *owner,
+ const char *name,
+ uint64_t size,
+ Error **errp)
+{
+}
+
+#ifdef __linux__
+void memory_region_init_ram_from_file(MemoryRegion *mr,
+ struct Object *owner,
+ const char *name,
+ uint64_t size,
+ bool share,
+ const char *path,
+ Error **errp)
+{
+}
+#endif
+
+void memory_region_init(MemoryRegion *mr,
+ struct Object *owner,
+ const char *name,
+ uint64_t size)
+{
+}
+
+void memory_region_add_subregion(MemoryRegion *mr,
+ hwaddr offset,
+ MemoryRegion *subregion)
+{
+}
+
+bool memory_region_is_mapped(MemoryRegion *mr)
+{
+ return false;
+}
diff --git a/stubs/qdev.c b/stubs/qdev.c
new file mode 100644
index 0000000..e28fdf2
--- /dev/null
+++ b/stubs/qdev.c
@@ -0,0 +1,8 @@
+
+#include "qemu/osdep.h"
+#include "hw/qdev-core.h"
+
+Object *qdev_get_machine(void)
+{
+ return NULL;
+}
diff --git a/stubs/vl.c b/stubs/vl.c
new file mode 100644
index 0000000..698fa80
--- /dev/null
+++ b/stubs/vl.c
@@ -0,0 +1,8 @@
+
+#include "qemu/osdep.h"
+#include "exec/cpu-common.h"
+
+int max_cpus = 1;
+ram_addr_t ram_size;
+const char *mem_path;
+int mem_prealloc;
diff --git a/stubs/vmstate.c b/stubs/vmstate.c
index 94b831e..f2708ee 100644
--- a/stubs/vmstate.c
+++ b/stubs/vmstate.c
@@ -23,3 +23,7 @@ void vmstate_unregister(DeviceState *dev,
void *opaque)
{
}
+
+void vmstate_register_ram_global(struct MemoryRegion *memory)
+{
+}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 605f276..72f71ea 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -116,6 +116,7 @@ check-unit-$(CONFIG_REPLICATION) += tests/test-replication$(EXESUF)
check-unit-y += tests/test-bufferiszero$(EXESUF)
gcov-files-check-bufferiszero-y = util/bufferiszero.c
check-unit-y += tests/test-uuid$(EXESUF)
+check-unit-y += tests/test-numa$(EXESUF)
check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
@@ -584,6 +585,7 @@ tests/test-crypto-pbkdf$(EXESUF): tests/test-crypto-pbkdf.o $(test-crypto-obj-y)
tests/test-crypto-ivgen$(EXESUF): tests/test-crypto-ivgen.o $(test-crypto-obj-y)
tests/test-crypto-afsplit$(EXESUF): tests/test-crypto-afsplit.o $(test-crypto-obj-y)
tests/test-crypto-block$(EXESUF): tests/test-crypto-block.o $(test-crypto-obj-y)
+tests/test-numa$(EXESUF): tests/test-numa.o numa.o $(test-qom-obj-y)
libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/test-numa.c b/tests/test-numa.c
new file mode 100644
index 0000000..458681b
--- /dev/null
+++ b/tests/test-numa.c
@@ -0,0 +1,116 @@
+/*
+ * QEMU NUMA testing
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "sysemu/numa_int.h"
+
+static void test_numa_parse(const char **nodestr)
+{
+ QemuOpts *opts;
+ size_t i;
+
+ DECLARE_BITMAP(node0cpus, MAX_CPUMASK_BITS);
+ DECLARE_BITMAP(node5cpus, MAX_CPUMASK_BITS);
+
+ bitmap_zero(node0cpus, MAX_CPUMASK_BITS);
+ bitmap_zero(node5cpus, MAX_CPUMASK_BITS);
+ for (i = 0; i <= 3; i++) {
+ bitmap_set(node0cpus, i, 1);
+ }
+ for (i = 8; i <= 11; i++) {
+ bitmap_set(node0cpus, i, 1);
+ }
+ for (i = 4; i <= 7; i++) {
+ bitmap_set(node5cpus, i, 1);
+ }
+ for (i = 12; i <= 15; i++) {
+ bitmap_set(node5cpus, i, 1);
+ }
+
+ max_cpus = 16;
+
+ opts = qemu_opts_parse_noisily(&qemu_numa_opts,
+ nodestr[0], true);
+ g_assert(opts != NULL);
+
+ opts = qemu_opts_parse_noisily(&qemu_numa_opts,
+ nodestr[1], true);
+ g_assert(opts != NULL);
+
+ qemu_opts_foreach(&qemu_numa_opts, parse_numa, NULL, NULL);
+
+ g_assert_cmpint(max_numa_nodeid, ==, 6);
+ g_assert(!have_memdevs);
+
+ g_assert_cmpint(nb_numa_nodes, ==, 2);
+ for (i = 0; i < MAX_NODES; i++) {
+ if (i == 0 || i == 5) {
+ g_assert(numa_info[i].present);
+ g_assert_cmpint(numa_info[i].node_mem, ==, 107 * 1024 * 1024);
+
+ if (i == 0) {
+ g_assert(bitmap_equal(node0cpus,
+ numa_info[i].node_cpu,
+ MAX_CPUMASK_BITS));
+ } else {
+ g_assert(bitmap_equal(node5cpus,
+ numa_info[i].node_cpu,
+ MAX_CPUMASK_BITS));
+ }
+ } else {
+ g_assert(!numa_info[i].present);
+ }
+ }
+
+ nb_numa_nodes = 0;
+ max_numa_nodeid = 0;
+ memset(&numa_info, 0, sizeof(numa_info));
+ g_assert(!numa_info[0].present);
+ qemu_opts_reset(&qemu_numa_opts);
+}
+
+static void test_numa_parse_legacy(void)
+{
+ const char *nodestr[] = {
+ "node,nodeid=0,cpus=0-3,cpus=8-11,mem=107",
+ "node,nodeid=5,cpus=4-7,cpus=12-15,mem=107"
+ };
+ test_numa_parse(nodestr);
+}
+
+static void test_numa_parse_modern(void)
+{
+ const char *nodestr[] = {
+ "type=node,data.nodeid=0,data.cpus.0=0,data.cpus.1=1,data.cpus.2=2,data.cpus.3=3,"
+ "data.cpus.4=8,data.cpus.5=9,data.cpus.6=10,data.cpus.7=11,data.mem=107",
+ "type=node,data.nodeid=5,data.cpus.0=4,data.cpus.1=5,data.cpus.2=6,data.cpus.3=7,"
+ "data.cpus.4=12,data.cpus.5=13,data.cpus.6=14,data.cpus.7=15,data.mem=107",
+ };
+ test_numa_parse(nodestr);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+ g_test_add_func("/numa/parse/legacy", test_numa_parse_legacy);
+ g_test_add_func("/numa/parse/modern", test_numa_parse_modern);
+ return g_test_run();
+}
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 18/21] block: convert crypto driver to use QObjectInputVisitor
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (16 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 19/21] acpi: convert to QObjectInputVisitor for -acpi parsing Daniel P. Berrange
` (4 subsequent siblings)
22 siblings, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The crypto block driver currently uses OptsVisitor to
convert from the block driver open/create options into
QCryptoBlockOpenOptions/QCryptoBlockCreateOptions. This
is easily replaced by use of QObjectInputVisitor with
no need to enable any compatibility options, since the
structs dealt with contain only scalars.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
block/crypto.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/block/crypto.c b/block/crypto.c
index 7aa7eb5..12b0b9f 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -23,7 +23,7 @@
#include "block/block_int.h"
#include "sysemu/block-backend.h"
#include "crypto/block.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi-visit.h"
#include "qapi/error.h"
@@ -206,7 +206,11 @@ block_crypto_open_opts_init(QCryptoBlockFormat format,
ret = g_new0(QCryptoBlockOpenOptions, 1);
ret->format = format;
- v = opts_visitor_new(opts);
+ v = qobject_input_visitor_new_opts(opts, false, 0, false, false,
+ &local_err);
+ if (local_err) {
+ goto out;
+ }
visit_start_struct(v, NULL, NULL, 0, &local_err);
if (local_err) {
@@ -252,7 +256,11 @@ block_crypto_create_opts_init(QCryptoBlockFormat format,
ret = g_new0(QCryptoBlockCreateOptions, 1);
ret->format = format;
- v = opts_visitor_new(opts);
+ v = qobject_input_visitor_new_opts(opts, false, 0, false, false,
+ &local_err);
+ if (local_err) {
+ goto out;
+ }
visit_start_struct(v, NULL, NULL, 0, &local_err);
if (local_err) {
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 19/21] acpi: convert to QObjectInputVisitor for -acpi parsing
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (17 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 18/21] block: convert crypto driver to use QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing Daniel P. Berrange
` (3 subsequent siblings)
22 siblings, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The -acpi command line option parsing uses the OptsVisitor
currently. This is easily replaced by the QObjectInputVisitor
instead. There is no need to enable any of the compatibility
options, since the AcpiTableOptions QAPI struct only contains
scalar properties.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
hw/acpi/core.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/hw/acpi/core.c b/hw/acpi/core.c
index e890a5d..67c0abe 100644
--- a/hw/acpi/core.c
+++ b/hw/acpi/core.c
@@ -25,7 +25,7 @@
#include "hw/acpi/acpi.h"
#include "hw/nvram/fw_cfg.h"
#include "qemu/config-file.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "qapi-visit.h"
#include "qapi-event.h"
@@ -237,14 +237,14 @@ void acpi_table_add(const QemuOpts *opts, Error **errp)
char **cur;
size_t bloblen = 0;
char unsigned *blob = NULL;
+ Visitor *v;
- {
- Visitor *v;
-
- v = opts_visitor_new(opts);
- visit_type_AcpiTableOptions(v, NULL, &hdrs, &err);
- visit_free(v);
+ v = qobject_input_visitor_new_opts(opts, false, 0, false, false, &err);
+ if (err) {
+ goto out;
}
+ visit_type_AcpiTableOptions(v, NULL, &hdrs, &err);
+ visit_free(v);
if (err) {
goto out;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (18 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 19/21] acpi: convert to QObjectInputVisitor for -acpi parsing Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-10-20 7:38 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 21/21] qapi: delete unused OptsVisitor code Daniel P. Berrange
` (2 subsequent siblings)
22 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
The -net/-netdev command line parsing code uses OptsVisitor
for parsing options to populate NetLegacy or NetDev struct
respectively. Although those structs have nesting, the
OptsVisitor flattens them, so we must enable compatibility
options to auto-create structs. This allows the legacy
syntax
-net tap,id=net0,vlan=3,fd=3,script=/bin/ifup-qemu
to be treated as equivalent to the modern QAPI based
syntax
-net id=net0,vlan=3,opts.type=tap,opts.data.fd=3,opts.data.script=/bin/ifup-qemu
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
net/net.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/net/net.c b/net/net.c
index ec984bf..de6bf8e 100644
--- a/net/net.c
+++ b/net/net.c
@@ -43,7 +43,7 @@
#include "qemu/iov.h"
#include "qemu/main-loop.h"
#include "qapi-visit.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
#include "sysemu/sysemu.h"
#include "net/filter.h"
#include "qapi/string-output-visitor.h"
@@ -1069,7 +1069,21 @@ int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
void *object = NULL;
Error *err = NULL;
int ret = -1;
- Visitor *v = opts_visitor_new(opts);
+ /*
+ * Needs autocreate_lists=true in order support existing
+ * syntax for list options where the bare key is repeated
+ *
+ * Needs autocreate_struct_levels=3 in order to deal with
+ * 3 level nesting in NetLegacy option args, which was
+ * exposed as a flat namespace with OptVisitor
+ */
+ Visitor *v = qobject_input_visitor_new_opts(opts, true, 3, false, true,
+ &err);
+
+ if (err) {
+ error_propagate(errp, err);
+ return -1;
+ }
{
/* Parse convenience option format ip6-net=fec0::0[/64] */
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [Qemu-devel] [PATCH v14 21/21] qapi: delete unused OptsVisitor code
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (19 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing Daniel P. Berrange
@ 2016-09-30 14:45 ` Daniel P. Berrange
2016-09-30 15:45 ` [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties no-reply
2016-10-21 18:30 ` Markus Armbruster
22 siblings, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-09-30 14:45 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber, Eric Blake, Daniel P. Berrange
Now that all code has been converted to QObjectInputVisitor's
QemuOpts compatibility mode, there is no longer any reason
to keep OptsVisitor alive.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/opts-visitor.h | 40 ----
qapi/Makefile.objs | 2 +-
qapi/opts-visitor.c | 544 --------------------------------------------
tests/Makefile.include | 5 +-
tests/test-opts-visitor.c | 268 ----------------------
vl.c | 1 -
6 files changed, 2 insertions(+), 858 deletions(-)
delete mode 100644 include/qapi/opts-visitor.h
delete mode 100644 qapi/opts-visitor.c
delete mode 100644 tests/test-opts-visitor.c
diff --git a/include/qapi/opts-visitor.h b/include/qapi/opts-visitor.h
deleted file mode 100644
index 6462c96..0000000
--- a/include/qapi/opts-visitor.h
+++ /dev/null
@@ -1,40 +0,0 @@
-/*
- * Options Visitor
- *
- * Copyright Red Hat, Inc. 2012
- *
- * Author: Laszlo Ersek <lersek@redhat.com>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#ifndef OPTS_VISITOR_H
-#define OPTS_VISITOR_H
-
-#include "qapi/visitor.h"
-#include "qemu/option.h"
-
-/* Inclusive upper bound on the size of any flattened range. This is a safety
- * (= anti-annoyance) measure; wrong ranges should not cause long startup
- * delays nor exhaust virtual memory.
- */
-#define OPTS_VISITOR_RANGE_MAX 65536
-
-typedef struct OptsVisitor OptsVisitor;
-
-/* Contrarily to qemu-option.c::parse_option_number(), OptsVisitor's "int"
- * parser relies on strtoll() instead of strtoull(). Consequences:
- * - string representations of negative numbers yield negative values,
- * - values below INT64_MIN or LLONG_MIN are rejected,
- * - values above INT64_MAX or LLONG_MAX are rejected.
- *
- * The Opts input visitor does not implement support for visiting QAPI
- * alternates, numbers (other than integers), null, or arbitrary
- * QTypes. It also requires a non-null list argument to
- * visit_start_list().
- */
-Visitor *opts_visitor_new(const QemuOpts *opts);
-
-#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 33906ff..4b820a7 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,6 +1,6 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
util-obj-y += qobject-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
-util-obj-y += opts-visitor.o qapi-clone-visitor.o
+util-obj-y += qapi-clone-visitor.o
util-obj-y += qmp-event.o
util-obj-y += qapi-util.o
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
deleted file mode 100644
index 084f7cc..0000000
--- a/qapi/opts-visitor.c
+++ /dev/null
@@ -1,544 +0,0 @@
-/*
- * Options Visitor
- *
- * Copyright Red Hat, Inc. 2012-2016
- *
- * Author: Laszlo Ersek <lersek@redhat.com>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#include "qemu/osdep.h"
-#include "qapi/error.h"
-#include "qemu/cutils.h"
-#include "qapi/qmp/qerror.h"
-#include "qapi/opts-visitor.h"
-#include "qemu/queue.h"
-#include "qemu/option_int.h"
-#include "qapi/visitor-impl.h"
-
-
-enum ListMode
-{
- LM_NONE, /* not traversing a list of repeated options */
-
- LM_IN_PROGRESS, /* opts_next_list() ready to be called.
- *
- * Generating the next list link will consume the most
- * recently parsed QemuOpt instance of the repeated
- * option.
- *
- * Parsing a value into the list link will examine the
- * next QemuOpt instance of the repeated option, and
- * possibly enter LM_SIGNED_INTERVAL or
- * LM_UNSIGNED_INTERVAL.
- */
-
- LM_SIGNED_INTERVAL, /* opts_next_list() has been called.
- *
- * Generating the next list link will consume the most
- * recently stored element from the signed interval,
- * parsed from the most recent QemuOpt instance of the
- * repeated option. This may consume QemuOpt itself
- * and return to LM_IN_PROGRESS.
- *
- * Parsing a value into the list link will store the
- * next element of the signed interval.
- */
-
- LM_UNSIGNED_INTERVAL /* Same as above, only for an unsigned interval. */
-};
-
-typedef enum ListMode ListMode;
-
-struct OptsVisitor
-{
- Visitor visitor;
-
- /* Ownership remains with opts_visitor_new()'s caller. */
- const QemuOpts *opts_root;
-
- unsigned depth;
-
- /* Non-null iff depth is positive. Each key is a QemuOpt name. Each value
- * is a non-empty GQueue, enumerating all QemuOpt occurrences with that
- * name. */
- GHashTable *unprocessed_opts;
-
- /* The list currently being traversed with opts_start_list() /
- * opts_next_list(). The list must have a struct element type in the
- * schema, with a single mandatory scalar member. */
- ListMode list_mode;
- GQueue *repeated_opts;
-
- /* When parsing a list of repeating options as integers, values of the form
- * "a-b", representing a closed interval, are allowed. Elements in the
- * range are generated individually.
- */
- union {
- int64_t s;
- uint64_t u;
- } range_next, range_limit;
-
- /* If "opts_root->id" is set, reinstantiate it as a fake QemuOpt for
- * uniformity. Only its "name" and "str" fields are set. "fake_id_opt" does
- * not survive or escape the OptsVisitor object.
- */
- QemuOpt *fake_id_opt;
-};
-
-
-static OptsVisitor *to_ov(Visitor *v)
-{
- return container_of(v, OptsVisitor, visitor);
-}
-
-
-static void
-destroy_list(gpointer list)
-{
- g_queue_free(list);
-}
-
-
-static void
-opts_visitor_insert(GHashTable *unprocessed_opts, const QemuOpt *opt)
-{
- GQueue *list;
-
- list = g_hash_table_lookup(unprocessed_opts, opt->name);
- if (list == NULL) {
- list = g_queue_new();
-
- /* GHashTable will never try to free the keys -- we supply NULL as
- * "key_destroy_func" in opts_start_struct(). Thus cast away key
- * const-ness in order to suppress gcc's warning.
- */
- g_hash_table_insert(unprocessed_opts, (gpointer)opt->name, list);
- }
-
- /* Similarly, destroy_list() doesn't call g_queue_free_full(). */
- g_queue_push_tail(list, (gpointer)opt);
-}
-
-
-static void
-opts_start_struct(Visitor *v, const char *name, void **obj,
- size_t size, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
-
- if (obj) {
- *obj = g_malloc0(size);
- }
- if (ov->depth++ > 0) {
- return;
- }
-
- ov->unprocessed_opts = g_hash_table_new_full(&g_str_hash, &g_str_equal,
- NULL, &destroy_list);
- QTAILQ_FOREACH(opt, &ov->opts_root->head, next) {
- /* ensured by qemu-option.c::opts_do_parse() */
- assert(strcmp(opt->name, "id") != 0);
-
- opts_visitor_insert(ov->unprocessed_opts, opt);
- }
-
- if (ov->opts_root->id != NULL) {
- ov->fake_id_opt = g_malloc0(sizeof *ov->fake_id_opt);
-
- ov->fake_id_opt->name = g_strdup("id");
- ov->fake_id_opt->str = g_strdup(ov->opts_root->id);
- opts_visitor_insert(ov->unprocessed_opts, ov->fake_id_opt);
- }
-}
-
-
-static void
-opts_check_struct(Visitor *v, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- GHashTableIter iter;
- GQueue *any;
-
- if (ov->depth > 0) {
- return;
- }
-
- /* we should have processed all (distinct) QemuOpt instances */
- g_hash_table_iter_init(&iter, ov->unprocessed_opts);
- if (g_hash_table_iter_next(&iter, NULL, (void **)&any)) {
- const QemuOpt *first;
-
- first = g_queue_peek_head(any);
- error_setg(errp, QERR_INVALID_PARAMETER, first->name);
- }
-}
-
-
-static void
-opts_end_struct(Visitor *v, void **obj)
-{
- OptsVisitor *ov = to_ov(v);
-
- if (--ov->depth > 0) {
- return;
- }
-
- g_hash_table_destroy(ov->unprocessed_opts);
- ov->unprocessed_opts = NULL;
- if (ov->fake_id_opt) {
- g_free(ov->fake_id_opt->name);
- g_free(ov->fake_id_opt->str);
- g_free(ov->fake_id_opt);
- }
- ov->fake_id_opt = NULL;
-}
-
-
-static GQueue *
-lookup_distinct(const OptsVisitor *ov, const char *name, Error **errp)
-{
- GQueue *list;
-
- list = g_hash_table_lookup(ov->unprocessed_opts, name);
- if (!list) {
- error_setg(errp, QERR_MISSING_PARAMETER, name);
- }
- return list;
-}
-
-
-static void
-opts_start_list(Visitor *v, const char *name, GenericList **list, size_t size,
- Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
-
- /* we can't traverse a list in a list */
- assert(ov->list_mode == LM_NONE);
- /* we don't support visits without a list */
- assert(list);
- ov->repeated_opts = lookup_distinct(ov, name, errp);
- if (ov->repeated_opts) {
- ov->list_mode = LM_IN_PROGRESS;
- *list = g_malloc0(size);
- } else {
- *list = NULL;
- }
-}
-
-
-static GenericList *
-opts_next_list(Visitor *v, GenericList *tail, size_t size)
-{
- OptsVisitor *ov = to_ov(v);
-
- switch (ov->list_mode) {
- case LM_SIGNED_INTERVAL:
- case LM_UNSIGNED_INTERVAL:
- if (ov->list_mode == LM_SIGNED_INTERVAL) {
- if (ov->range_next.s < ov->range_limit.s) {
- ++ov->range_next.s;
- break;
- }
- } else if (ov->range_next.u < ov->range_limit.u) {
- ++ov->range_next.u;
- break;
- }
- ov->list_mode = LM_IN_PROGRESS;
- /* range has been completed, fall through in order to pop option */
-
- case LM_IN_PROGRESS: {
- const QemuOpt *opt;
-
- opt = g_queue_pop_head(ov->repeated_opts);
- if (g_queue_is_empty(ov->repeated_opts)) {
- g_hash_table_remove(ov->unprocessed_opts, opt->name);
- return NULL;
- }
- break;
- }
-
- default:
- abort();
- }
-
- tail->next = g_malloc0(size);
- return tail->next;
-}
-
-
-static void
-opts_end_list(Visitor *v, void **obj)
-{
- OptsVisitor *ov = to_ov(v);
-
- assert(ov->list_mode == LM_IN_PROGRESS ||
- ov->list_mode == LM_SIGNED_INTERVAL ||
- ov->list_mode == LM_UNSIGNED_INTERVAL);
- ov->repeated_opts = NULL;
- ov->list_mode = LM_NONE;
-}
-
-
-static const QemuOpt *
-lookup_scalar(const OptsVisitor *ov, const char *name, Error **errp)
-{
- if (ov->list_mode == LM_NONE) {
- GQueue *list;
-
- /* the last occurrence of any QemuOpt takes effect when queried by name
- */
- list = lookup_distinct(ov, name, errp);
- return list ? g_queue_peek_tail(list) : NULL;
- }
- assert(ov->list_mode == LM_IN_PROGRESS);
- return g_queue_peek_head(ov->repeated_opts);
-}
-
-
-static void
-processed(OptsVisitor *ov, const char *name)
-{
- if (ov->list_mode == LM_NONE) {
- g_hash_table_remove(ov->unprocessed_opts, name);
- return;
- }
- assert(ov->list_mode == LM_IN_PROGRESS);
- /* do nothing */
-}
-
-
-static void
-opts_type_str(Visitor *v, const char *name, char **obj, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
-
- opt = lookup_scalar(ov, name, errp);
- if (!opt) {
- *obj = NULL;
- return;
- }
- *obj = g_strdup(opt->str ? opt->str : "");
- /* Note that we consume a string even if this is called as part of
- * an enum visit that later fails because the string is not a
- * valid enum value; this is harmless because tracking what gets
- * consumed only matters to visit_end_struct() as the final error
- * check if there were no other failures during the visit. */
- processed(ov, name);
-}
-
-
-static void
-opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
-
- opt = lookup_scalar(ov, name, errp);
- if (!opt) {
- return;
- }
-
- parse_option_bool(opt->name, opt->str, obj, errp);
-
- processed(ov, name);
-}
-
-
-static void
-opts_type_int64(Visitor *v, const char *name, int64_t *obj, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
- const char *str;
- long long val;
- char *endptr;
-
- if (ov->list_mode == LM_SIGNED_INTERVAL) {
- *obj = ov->range_next.s;
- return;
- }
-
- opt = lookup_scalar(ov, name, errp);
- if (!opt) {
- return;
- }
- str = opt->str ? opt->str : "";
-
- /* we've gotten past lookup_scalar() */
- assert(ov->list_mode == LM_NONE || ov->list_mode == LM_IN_PROGRESS);
-
- errno = 0;
- val = strtoll(str, &endptr, 0);
- if (errno == 0 && endptr > str && INT64_MIN <= val && val <= INT64_MAX) {
- if (*endptr == '\0') {
- *obj = val;
- processed(ov, name);
- return;
- }
- if (*endptr == '-' && ov->list_mode == LM_IN_PROGRESS) {
- long long val2;
-
- str = endptr + 1;
- val2 = strtoll(str, &endptr, 0);
- if (errno == 0 && endptr > str && *endptr == '\0' &&
- INT64_MIN <= val2 && val2 <= INT64_MAX && val <= val2 &&
- (val > INT64_MAX - OPTS_VISITOR_RANGE_MAX ||
- val2 < val + OPTS_VISITOR_RANGE_MAX)) {
- ov->range_next.s = val;
- ov->range_limit.s = val2;
- ov->list_mode = LM_SIGNED_INTERVAL;
-
- /* as if entering on the top */
- *obj = ov->range_next.s;
- return;
- }
- }
- }
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
- (ov->list_mode == LM_NONE) ? "an int64 value" :
- "an int64 value or range");
-}
-
-
-static void
-opts_type_uint64(Visitor *v, const char *name, uint64_t *obj, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
- const char *str;
- unsigned long long val;
- char *endptr;
-
- if (ov->list_mode == LM_UNSIGNED_INTERVAL) {
- *obj = ov->range_next.u;
- return;
- }
-
- opt = lookup_scalar(ov, name, errp);
- if (!opt) {
- return;
- }
- str = opt->str;
-
- /* we've gotten past lookup_scalar() */
- assert(ov->list_mode == LM_NONE || ov->list_mode == LM_IN_PROGRESS);
-
- if (parse_uint(str, &val, &endptr, 0) == 0 && val <= UINT64_MAX) {
- if (*endptr == '\0') {
- *obj = val;
- processed(ov, name);
- return;
- }
- if (*endptr == '-' && ov->list_mode == LM_IN_PROGRESS) {
- unsigned long long val2;
-
- str = endptr + 1;
- if (parse_uint_full(str, &val2, 0) == 0 &&
- val2 <= UINT64_MAX && val <= val2 &&
- val2 - val < OPTS_VISITOR_RANGE_MAX) {
- ov->range_next.u = val;
- ov->range_limit.u = val2;
- ov->list_mode = LM_UNSIGNED_INTERVAL;
-
- /* as if entering on the top */
- *obj = ov->range_next.u;
- return;
- }
- }
- }
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
- (ov->list_mode == LM_NONE) ? "a uint64 value" :
- "a uint64 value or range");
-}
-
-
-static void
-opts_type_size(Visitor *v, const char *name, uint64_t *obj, Error **errp)
-{
- OptsVisitor *ov = to_ov(v);
- const QemuOpt *opt;
- int64_t val;
- char *endptr;
-
- opt = lookup_scalar(ov, name, errp);
- if (!opt) {
- return;
- }
-
- val = qemu_strtosz_suffix(opt->str ? opt->str : "", &endptr,
- QEMU_STRTOSZ_DEFSUFFIX_B);
- if (val < 0 || *endptr) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
- "a size value representible as a non-negative int64");
- return;
- }
-
- *obj = val;
- processed(ov, name);
-}
-
-
-static void
-opts_optional(Visitor *v, const char *name, bool *present)
-{
- OptsVisitor *ov = to_ov(v);
-
- /* we only support a single mandatory scalar field in a list node */
- assert(ov->list_mode == LM_NONE);
- *present = (lookup_distinct(ov, name, NULL) != NULL);
-}
-
-
-static void
-opts_free(Visitor *v)
-{
- OptsVisitor *ov = to_ov(v);
-
- if (ov->unprocessed_opts != NULL) {
- g_hash_table_destroy(ov->unprocessed_opts);
- }
- g_free(ov->fake_id_opt);
- g_free(ov);
-}
-
-
-Visitor *
-opts_visitor_new(const QemuOpts *opts)
-{
- OptsVisitor *ov;
-
- ov = g_malloc0(sizeof *ov);
-
- ov->visitor.type = VISITOR_INPUT;
-
- ov->visitor.start_struct = &opts_start_struct;
- ov->visitor.check_struct = &opts_check_struct;
- ov->visitor.end_struct = &opts_end_struct;
-
- ov->visitor.start_list = &opts_start_list;
- ov->visitor.next_list = &opts_next_list;
- ov->visitor.end_list = &opts_end_list;
-
- ov->visitor.type_int64 = &opts_type_int64;
- ov->visitor.type_uint64 = &opts_type_uint64;
- ov->visitor.type_size = &opts_type_size;
- ov->visitor.type_bool = &opts_type_bool;
- ov->visitor.type_str = &opts_type_str;
-
- /* type_number() is not filled in, but this is not the first visitor to
- * skip some mandatory methods... */
-
- ov->visitor.optional = &opts_optional;
- ov->visitor.free = opts_free;
-
- ov->opts_root = opts;
-
- return &ov->visitor;
-}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 72f71ea..df39976 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -35,8 +35,6 @@ check-unit-y += tests/test-string-output-visitor$(EXESUF)
gcov-files-test-string-output-visitor-y = qapi/string-output-visitor.c
check-unit-y += tests/test-qmp-event$(EXESUF)
gcov-files-test-qmp-event-y += qapi/qmp-event.c
-check-unit-y += tests/test-opts-visitor$(EXESUF)
-gcov-files-test-opts-visitor-y = qapi/opts-visitor.c
check-unit-y += tests/test-coroutine$(EXESUF)
gcov-files-test-coroutine-y = coroutine-$(CONFIG_COROUTINE_BACKEND).c
check-unit-y += tests/test-visitor-serialization$(EXESUF)
@@ -445,7 +443,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
tests/test-qmp-commands.o tests/test-visitor-serialization.o \
tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
- tests/test-opts-visitor.o tests/test-qmp-event.o \
+ tests/test-qmp-event.o \
tests/rcutorture.o tests/test-rcu-list.o \
tests/test-qdist.o \
tests/test-qht.o tests/qht-bench.o tests/test-qht-par.o
@@ -549,7 +547,6 @@ tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(
tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y)
-tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
tests/test-mul64$(EXESUF): tests/test-mul64.o $(test-util-obj-y)
tests/test-bitops$(EXESUF): tests/test-bitops.o $(test-util-obj-y)
diff --git a/tests/test-opts-visitor.c b/tests/test-opts-visitor.c
deleted file mode 100644
index 0a9e75f..0000000
--- a/tests/test-opts-visitor.c
+++ /dev/null
@@ -1,268 +0,0 @@
-/*
- * Options Visitor unit-tests.
- *
- * Copyright (C) 2013 Red Hat, Inc.
- *
- * Authors:
- * Laszlo Ersek <lersek@redhat.com> (based on test-string-output-visitor)
- *
- * This work is licensed under the terms of the GNU GPL, version 2 or later.
- * See the COPYING file in the top-level directory.
- */
-
-#include "qemu/osdep.h"
-
-#include "qemu/config-file.h" /* qemu_add_opts() */
-#include "qemu/option.h" /* qemu_opts_parse() */
-#include "qapi/error.h"
-#include "qapi/opts-visitor.h" /* opts_visitor_new() */
-#include "test-qapi-visit.h" /* visit_type_UserDefOptions() */
-
-static QemuOptsList userdef_opts = {
- .name = "userdef",
- .head = QTAILQ_HEAD_INITIALIZER(userdef_opts.head),
- .desc = { { 0 } } /* validated with OptsVisitor */
-};
-
-/* fixture (= glib test case context) and test case manipulation */
-
-typedef struct OptsVisitorFixture {
- UserDefOptions *userdef;
- Error *err;
-} OptsVisitorFixture;
-
-
-static void
-setup_fixture(OptsVisitorFixture *f, gconstpointer test_data)
-{
- const char *opts_string = test_data;
- QemuOpts *opts;
- Visitor *v;
-
- opts = qemu_opts_parse(qemu_find_opts("userdef"), opts_string, false,
- NULL);
- g_assert(opts != NULL);
-
- v = opts_visitor_new(opts);
- visit_type_UserDefOptions(v, NULL, &f->userdef, &f->err);
- visit_free(v);
- qemu_opts_del(opts);
-}
-
-
-static void
-teardown_fixture(OptsVisitorFixture *f, gconstpointer test_data)
-{
- qapi_free_UserDefOptions(f->userdef);
- error_free(f->err);
-}
-
-
-static void
-add_test(const char *testpath,
- void (*test_func)(OptsVisitorFixture *f, gconstpointer test_data),
- gconstpointer test_data)
-{
- g_test_add(testpath, OptsVisitorFixture, test_data, setup_fixture,
- test_func, teardown_fixture);
-}
-
-/* test output evaluation */
-
-static void
-expect_ok(OptsVisitorFixture *f, gconstpointer test_data)
-{
- g_assert(f->err == NULL);
- g_assert(f->userdef != NULL);
-}
-
-
-static void
-expect_fail(OptsVisitorFixture *f, gconstpointer test_data)
-{
- g_assert(f->err != NULL);
-
- /* The error message is printed when this test utility is invoked directly
- * (ie. without gtester) and the --verbose flag is passed:
- *
- * tests/test-opts-visitor --verbose
- */
- g_test_message("'%s': %s", (const char *)test_data,
- error_get_pretty(f->err));
-}
-
-
-static void
-test_value(OptsVisitorFixture *f, gconstpointer test_data)
-{
- uint64_t magic, bitval;
- intList *i64;
- uint64List *u64;
- uint16List *u16;
-
- expect_ok(f, test_data);
-
- magic = 0;
- for (i64 = f->userdef->i64; i64 != NULL; i64 = i64->next) {
- g_assert(-16 <= i64->value && i64->value < 64-16);
- bitval = 1ull << (i64->value + 16);
- g_assert((magic & bitval) == 0);
- magic |= bitval;
- }
- g_assert(magic == 0xDEADBEEF);
-
- magic = 0;
- for (u64 = f->userdef->u64; u64 != NULL; u64 = u64->next) {
- g_assert(u64->value < 64);
- bitval = 1ull << u64->value;
- g_assert((magic & bitval) == 0);
- magic |= bitval;
- }
- g_assert(magic == 0xBADC0FFEE0DDF00DULL);
-
- magic = 0;
- for (u16 = f->userdef->u16; u16 != NULL; u16 = u16->next) {
- g_assert(u16->value < 64);
- bitval = 1ull << u16->value;
- g_assert((magic & bitval) == 0);
- magic |= bitval;
- }
- g_assert(magic == 0xD15EA5E);
-}
-
-
-static void
-expect_i64_min(OptsVisitorFixture *f, gconstpointer test_data)
-{
- expect_ok(f, test_data);
- g_assert(f->userdef->has_i64);
- g_assert(f->userdef->i64->next == NULL);
- g_assert(f->userdef->i64->value == INT64_MIN);
-}
-
-
-static void
-expect_i64_max(OptsVisitorFixture *f, gconstpointer test_data)
-{
- expect_ok(f, test_data);
- g_assert(f->userdef->has_i64);
- g_assert(f->userdef->i64->next == NULL);
- g_assert(f->userdef->i64->value == INT64_MAX);
-}
-
-
-static void
-expect_zero(OptsVisitorFixture *f, gconstpointer test_data)
-{
- expect_ok(f, test_data);
- g_assert(f->userdef->has_u64);
- g_assert(f->userdef->u64->next == NULL);
- g_assert(f->userdef->u64->value == 0);
-}
-
-
-static void
-expect_u64_max(OptsVisitorFixture *f, gconstpointer test_data)
-{
- expect_ok(f, test_data);
- g_assert(f->userdef->has_u64);
- g_assert(f->userdef->u64->next == NULL);
- g_assert(f->userdef->u64->value == UINT64_MAX);
-}
-
-/* test cases */
-
-int
-main(int argc, char **argv)
-{
- g_test_init(&argc, &argv, NULL);
-
- qemu_add_opts(&userdef_opts);
-
- /* Three hexadecimal magic numbers, "dead beef", "bad coffee, odd food" and
- * "disease", from
- * <http://en.wikipedia.org/wiki/Magic_number_%28programming%29>, were
- * converted to binary and dissected into bit ranges. Each magic number is
- * going to be recomposed using the lists called "i64", "u64" and "u16",
- * respectively.
- *
- * (Note that these types pertain to the individual bit shift counts, not
- * the magic numbers themselves; the intent is to exercise opts_type_int()
- * and opts_type_uint64().)
- *
- * The "i64" shift counts have been decreased by 16 (decimal) in order to
- * test negative values as well. Finally, the full list of QemuOpt elements
- * has been permuted with "shuf".
- *
- * Both "i64" and "u64" have some (distinct) single-element ranges
- * represented as both "a" and "a-a". "u16" is a special case of "i64" (see
- * visit_type_uint16()), so it wouldn't add a separate test in this regard.
- */
-
- add_test("/visitor/opts/flatten/value", &test_value,
- "i64=-1-0,u64=12-16,u64=2-3,i64=-11--9,u64=57,u16=9,i64=5-5,"
- "u16=1-4,u16=20,u64=63-63,i64=-16--13,u64=50-52,i64=14-15,u16=11,"
- "i64=7,u16=18,i64=2-3,u16=6,u64=54-55,u64=0,u64=18-20,u64=33-43,"
- "i64=9-12,u16=26-27,u64=59-61,u16=13-16,u64=29-31,u64=22-23,"
- "u16=24,i64=-7--3");
-
- add_test("/visitor/opts/i64/val1/errno", &expect_fail,
- "i64=0x8000000000000000");
- add_test("/visitor/opts/i64/val1/empty", &expect_fail, "i64=");
- add_test("/visitor/opts/i64/val1/trailing", &expect_fail, "i64=5z");
- add_test("/visitor/opts/i64/nonlist", &expect_fail, "i64x=5-6");
- add_test("/visitor/opts/i64/val2/errno", &expect_fail,
- "i64=0x7fffffffffffffff-0x8000000000000000");
- add_test("/visitor/opts/i64/val2/empty", &expect_fail, "i64=5-");
- add_test("/visitor/opts/i64/val2/trailing", &expect_fail, "i64=5-6z");
- add_test("/visitor/opts/i64/range/empty", &expect_fail, "i64=6-5");
- add_test("/visitor/opts/i64/range/minval", &expect_i64_min,
- "i64=-0x8000000000000000--0x8000000000000000");
- add_test("/visitor/opts/i64/range/maxval", &expect_i64_max,
- "i64=0x7fffffffffffffff-0x7fffffffffffffff");
-
- add_test("/visitor/opts/u64/val1/errno", &expect_fail, "u64=-1");
- add_test("/visitor/opts/u64/val1/empty", &expect_fail, "u64=");
- add_test("/visitor/opts/u64/val1/trailing", &expect_fail, "u64=5z");
- add_test("/visitor/opts/u64/nonlist", &expect_fail, "u64x=5-6");
- add_test("/visitor/opts/u64/val2/errno", &expect_fail,
- "u64=0xffffffffffffffff-0x10000000000000000");
- add_test("/visitor/opts/u64/val2/empty", &expect_fail, "u64=5-");
- add_test("/visitor/opts/u64/val2/trailing", &expect_fail, "u64=5-6z");
- add_test("/visitor/opts/u64/range/empty", &expect_fail, "u64=6-5");
- add_test("/visitor/opts/u64/range/minval", &expect_zero, "u64=0-0");
- add_test("/visitor/opts/u64/range/maxval", &expect_u64_max,
- "u64=0xffffffffffffffff-0xffffffffffffffff");
-
- /* Test maximum range sizes. The macro value is open-coded here
- * *intentionally*; the test case must use concrete values by design. If
- * OPTS_VISITOR_RANGE_MAX is changed, the following values need to be
- * recalculated as well. The assert and this comment should help with it.
- */
- g_assert(OPTS_VISITOR_RANGE_MAX == 65536);
-
- /* The unsigned case is simple, a u64-u64 difference can always be
- * represented as a u64.
- */
- add_test("/visitor/opts/u64/range/max", &expect_ok, "u64=0-65535");
- add_test("/visitor/opts/u64/range/2big", &expect_fail, "u64=0-65536");
-
- /* The same cannot be said about an i64-i64 difference. */
- add_test("/visitor/opts/i64/range/max/pos/a", &expect_ok,
- "i64=0x7fffffffffff0000-0x7fffffffffffffff");
- add_test("/visitor/opts/i64/range/max/pos/b", &expect_ok,
- "i64=0x7ffffffffffeffff-0x7ffffffffffffffe");
- add_test("/visitor/opts/i64/range/2big/pos", &expect_fail,
- "i64=0x7ffffffffffeffff-0x7fffffffffffffff");
- add_test("/visitor/opts/i64/range/max/neg/a", &expect_ok,
- "i64=-0x8000000000000000--0x7fffffffffff0001");
- add_test("/visitor/opts/i64/range/max/neg/b", &expect_ok,
- "i64=-0x7fffffffffffffff--0x7fffffffffff0000");
- add_test("/visitor/opts/i64/range/2big/neg", &expect_fail,
- "i64=-0x8000000000000000--0x7fffffffffff0000");
- add_test("/visitor/opts/i64/range/2big/full", &expect_fail,
- "i64=-0x8000000000000000-0x7fffffffffffffff");
-
- g_test_run();
- return 0;
-}
diff --git a/vl.c b/vl.c
index ab0349b..3a214c1 100644
--- a/vl.c
+++ b/vl.c
@@ -115,7 +115,6 @@ int main(int argc, char **argv)
#include "ui/qemu-spice.h"
#include "qapi/string-input-visitor.h"
-#include "qapi/opts-visitor.h"
#include "qom/object_interfaces.h"
#include "qapi-event.h"
#include "exec/semihost.h"
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (20 preceding siblings ...)
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 21/21] qapi: delete unused OptsVisitor code Daniel P. Berrange
@ 2016-09-30 15:45 ` no-reply
2016-09-30 18:50 ` Eric Blake
2016-10-21 18:30 ` Markus Armbruster
22 siblings, 1 reply; 109+ messages in thread
From: no-reply @ 2016-09-30 15:45 UTC (permalink / raw)
To: berrange; +Cc: famz, qemu-devel, qemu-block, armbru, mreitz, pbonzini, afaerber
Hi,
Your series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 1475246744-29302-1-git-send-email-berrange@redhat.com
Subject: [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
make J=8 docker-test-quick@centos6
make J=8 docker-test-mingw@fedora
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
- [tag update] patchew/1475202478-20716-1-git-send-email-zhangchen.fnst@cn.fujitsu.com -> patchew/1475202478-20716-1-git-send-email-zhangchen.fnst@cn.fujitsu.com
Switched to a new branch 'test'
9b89b2d qapi: delete unused OptsVisitor code
7d7d027 net: convert to QObjectInputVisitor for -net/-netdev parsing
f7fd6c1 acpi: convert to QObjectInputVisitor for -acpi parsing
fecaabf block: convert crypto driver to use QObjectInputVisitor
7941b3f numa: convert to use QObjectInputVisitor for -numa
3c6762d hmp: support non-scalar properties with object_add
efb01de qom: support non-scalar properties with -object
3c90b78 qapi: allow repeated opts with qobject_input_visitor_new_opts
81f728f qdict: allow qdict_crumple to accept compound types as values
8353c22 option: allow qemu_opts_to_qdict to merge repeated options
0948fcd qapi: add integer range support for QObjectInputVisitor
6da7117 qapi: permit auto-creating nested structs
1170f4b qapi: permit auto-creating single element lists
2ec3eac qapi: allow QObjectInputVisitor to be created with QemuOpts
b56363f qapi: permit scalar type conversions in QObjectInputVisitor
6fa7850 qapi: don't pass two copies of TestInputVisitorData to tests
d1b24d3 qapi: rename QmpOutputVisitor to QObjectOutputVisitor
794fec6 qapi: rename QmpInputVisitor to QObjectInputVisitor
43b5ede qapi: add trace events for visitor
587d218 qdict: implement a qdict_crumple method for un-flattening a dict
67cbb95 option: make parse_option_bool/number non-static
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf'
BUILD centos6
ARCHIVE qemu.tgz
ARCHIVE dtc.tgz
COPY RUNNER
RUN test-quick in centos6
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
gcc-4.4.7-17.el6.x86_64
git-1.7.1-4.el6_7.1.x86_64
glib2-devel-2.28.8-5.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64
Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ zlib-devel glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=9320e4514cd6
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/src/tests/docker/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /tmp/qemu-test/src/tests/docker/install
BIOS directory /tmp/qemu-test/src/tests/docker/install/share/qemu
binary directory /tmp/qemu-test/src/tests/docker/install/bin
library directory /tmp/qemu-test/src/tests/docker/install/lib
module directory /tmp/qemu-test/src/tests/docker/install/lib/qemu
libexec directory /tmp/qemu-test/src/tests/docker/install/libexec
include directory /tmp/qemu-test/src/tests/docker/install/include
config directory /tmp/qemu-test/src/tests/docker/install/etc
local state directory /tmp/qemu-test/src/tests/docker/install/var
Manual directory /tmp/qemu-test/src/tests/docker/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
tcg debug enabled no
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
pixman system
SDL support yes (1.2.14)
GTK support no
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle no
nettle kdf no
libtasn1 no
curses support no
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support no
xen support no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
RDMA support no
TCG interpreter no
fdt support yes
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
libcap-ng support no
vhost-net support yes
vhost-scsi support yes
vhost-vsock support yes
Trace backends log
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
GlusterFS support no
Archipelago support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
QOM debugging yes
lzo support no
snappy support no
bzip2 support no
NUMA host support no
tcmalloc support no
jemalloc support no
avx2 optimization no
replication support yes
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qmp-commands.h
GEN qapi-types.h
GEN qapi-visit.h
GEN qapi-event.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN qmp-introspect.h
GEN module_block.h
GEN tests/test-qapi-types.h
GEN tests/test-qapi-visit.h
GEN tests/test-qmp-commands.h
GEN tests/test-qapi-event.h
GEN tests/test-qmp-introspect.h
GEN config-all-devices.mak
GEN trace/generated-events.h
GEN trace/generated-tracers.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
CC tests/qemu-iotests/socket_scm_helper.o
GEN qga/qapi-generated/qga-qapi-types.h
GEN qga/qapi-generated/qga-qapi-visit.h
GEN qga/qapi-generated/qga-qmp-commands.h
GEN qga/qapi-generated/qga-qapi-types.c
GEN qga/qapi-generated/qga-qapi-visit.c
GEN qga/qapi-generated/qga-qmp-marshal.c
GEN qmp-introspect.c
GEN qapi-types.c
GEN qapi-visit.c
GEN qapi-event.c
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
/tmp/qemu-test/src/qapi/qobject-input-visitor.c: In function ‘qobject_input_get_object’:
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: warning: implicit declaration of function ‘g_hash_table_contains’
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: warning: nested extern declaration of ‘g_hash_table_contains’
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qint.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qfloat.o
CC qobject/qbool.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
GEN trace/generated-events.c
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/compatfd.o
CC util/event_notifier-posix.o
CC util/mmap-alloc.o
CC util/oslib-posix.o
CC util/qemu-openpty.o
CC util/qemu-thread-posix.o
CC util/memfd.o
CC util/path.o
CC util/envlist.o
CC util/module.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rfifolock.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-ucontext.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
/tmp/qemu-test/src/util/qht.c: In function ‘qht_reset_size’:
/tmp/qemu-test/src/util/qht.c:413: warning: ‘new’ may be used uninitialized in this function
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/fdset-add-fd.o
CC stubs/fdset-find-fd.o
CC stubs/fdset-get-fd.o
CC stubs/fdset-remove-fd.o
CC stubs/gdbstub.o
CC stubs/get-fd.o
CC stubs/get-next-serial.o
CC stubs/get-vm-name.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/mon-is-qmp.o
CC stubs/mon-printf.o
CC stubs/monitor-init.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/replay-user.o
CC stubs/reset.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/cpus.o
CC stubs/kvm.o
CC stubs/qmp_pc_dimm_device_list.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/vhost.o
CC stubs/iohandler.o
CC stubs/smbios_type_38.o
CC stubs/ipmi.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vl.o
CC stubs/exec.o
CC stubs/memory.o
CC stubs/hostmem.o
CC stubs/qdev.o
CC contrib/ivshmem-client/ivshmem-client.o
CC contrib/ivshmem-client/main.o
CC contrib/ivshmem-server/ivshmem-server.o
CC contrib/ivshmem-server/main.o
CC qemu-nbd.o
CC async.o
CC thread-pool.o
CC block.o
CC blockjob.o
CC main-loop.o
CC iohandler.o
CC qemu-timer.o
CC aio-posix.o
CC qemu-io-cmds.o
CC block/raw_bsd.o
CC replication.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qed.o
CC block/qed-gencb.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/raw-posix.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC nbd/common.o
CC crypto/init.o
CC crypto/hash.o
CC crypto/hash-glib.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsanon.o
CC crypto/tlscredsx509.o
CC crypto/tlssession.o
CC crypto/secret.o
CC crypto/random-platform.o
CC crypto/pbkdf.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-buffer.o
CC io/channel-command.o
CC io/channel-file.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/channel-util.o
CC io/task.o
CC qom/object.o
CC qom/container.o
CC qom/qom-qobject.o
CC qom/object_interfaces.o
GEN qemu-img-cmds.h
CC qemu-io.o
CC qemu-bridge-helper.o
CC blockdev.o
CC blockdev-nbd.o
CC iothread.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-posix.o
CC qemu-char.o
CC page_cache.o
CC accel.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
GEN qmp-marshal.c
CC qmp.o
CC hmp.o
CC tcg-runtime.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/sdlaudio.o
CC audio/ossaudio.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/rng-random.o
CC backends/msmouse.o
CC backends/testdev.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/hostmem-file.o
CC block/stream.o
CC disas/arm.o
CC disas/i386.o
CC fsdev/qemu-fsdev-dummy.o
CC fsdev/qemu-fsdev-opts.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/pcihp.o
CC hw/acpi/ich9.o
CC hw/acpi/tco.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/memory_hotplug_acpi_table.o
CC hw/acpi/cpu.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/wm8750.o
CC hw/audio/pl041.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/pflash_cfi02.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/cadence_uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/hotplug.o
CC hw/core/ptimer.o
CC hw/core/sysbus.o
CC hw/core/machine.o
CC hw/core/null-machine.o
CC hw/core/loader.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/platform-bus.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/ssd0303.o
CC hw/display/ssd0323.o
CC hw/display/vga-pci.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/blizzard.o
CC hw/display/exynos4210_fimd.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/zaurus.o
CC hw/gpio/gpio_key.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/input/hid.o
CC hw/input/lm832x.o
CC hw/input/pckbd.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/vmmouse.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/input/virtio-input-host.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/imx_avic.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3_redist.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp105.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pc-testdev.o
CC hw/misc/pci-testdev.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/ne2000.o
CC hw/net/eepro100.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/fw_cfg.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
/tmp/qemu-test/src/hw/nvram/fw_cfg.c: In function ‘fw_cfg_dma_transfer’:
/tmp/qemu-test/src/hw/nvram/fw_cfg.c:330: warning: ‘read’ may be used uninitialized in this function
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie.o
CC hw/pci/pcie_aer.o
CC hw/pci/pcie_port.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/timer/arm_timer.o
CC hw/timer/arm_mptimer.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/pl031.o
CC hw/timer/twl92230.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_passthrough.o
CC hw/tpm/tpm_util.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/dev-mtp.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/virtio-mmio.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/vmstate.o
CC migration/qemu-file.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block.o
CC net/net.o
CC net/queue.o
CC net/checksum.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/eth.o
CC net/l2tpv3.o
CC net/tap.o
CC net/vhost-user.o
CC net/tap-linux.o
CC net/slirp.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-internal.o
CC replay/replay-events.o
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
/tmp/qemu-test/src/replay/replay-internal.c: In function ‘replay_put_array’:
/tmp/qemu-test/src/replay/replay-internal.c:65: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_icmp.o
CC slirp/ip6_input.o
CC slirp/ip6_output.o
CC slirp/ip_input.o
CC slirp/ip_output.o
CC slirp/dnssearch.o
CC slirp/dhcpv6.o
CC slirp/slirp.o
CC slirp/mbuf.o
CC slirp/misc.o
CC slirp/sbuf.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_output.o
CC slirp/tcp_subr.o
CC slirp/tcp_timer.o
CC slirp/udp.o
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
/tmp/qemu-test/src/slirp/tcp_input.c: In function ‘tcp_input’:
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_p’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_len’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_tos’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_id’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_off’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_ttl’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_sum’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_src.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_dst.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:220: warning: ‘save_ip6.ip_nh’ may be used uninitialized in this function
CC slirp/ndp_table.o
CC ui/keymaps.o
CC ui/console.o
CC ui/cursor.o
CC ui/qemu-pixman.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-legacy.o
CC ui/input-linux.o
CC ui/sdl.o
CC ui/sdl_zoom.o
CC ui/x_keymap.o
CC ui/vnc.o
CC ui/vnc-enc-zlib.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
AS optionrom/multiboot.o
AS optionrom/linuxboot.o
CC optionrom/linuxboot_dma.o
cc: unrecognized option '-no-integrated-as'
cc: unrecognized option '-no-integrated-as'
AS optionrom/kvmvapic.o
Building optionrom/multiboot.img
Building optionrom/linuxboot.img
Building optionrom/linuxboot_dma.img
Building optionrom/kvmvapic.img
Building optionrom/multiboot.raw
Building optionrom/linuxboot.raw
LINK tests/qemu-iotests/socket_scm_helper
Building optionrom/linuxboot_dma.raw
Building optionrom/kvmvapic.raw
Signing optionrom/multiboot.bin
Signing optionrom/linuxboot.bin
Signing optionrom/linuxboot_dma.bin
CC qga/commands.o
Signing optionrom/kvmvapic.bin
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/commands-posix.o
CC qga/channel-posix.o
CC qga/qapi-generated/qga-qapi-types.o
CC qga/qapi-generated/qga-qapi-visit.o
CC qga/qapi-generated/qga-qmp-marshal.o
CC qmp-introspect.o
CC qapi-types.o
CC qapi-visit.o
CC qapi-event.o
AR libqemustub.a
CC qemu-img.o
CC qmp-marshal.o
CC trace/generated-events.o
AR libqemuutil.a
LINK qemu-ga
LINK ivshmem-client
LINK ivshmem-server
LINK qemu-nbd
LINK qemu-img
LINK qemu-io
LINK qemu-bridge-helper
libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make: *** [qemu-ga] Error 1
make: *** Waiting for unfinished jobs....
libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make: *** [qemu-nbd] Error 1
libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make: *** [qemu-img] Error 1
libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make: *** [qemu-io] Error 1
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/hmp-commands-info.h
GEN x86_64-softmmu/config-target.h
GEN aarch64-softmmu/hmp-commands.h
GEN aarch64-softmmu/hmp-commands-info.h
GEN aarch64-softmmu/config-target.h
CC x86_64-softmmu/exec.o
CC x86_64-softmmu/translate-all.o
CC x86_64-softmmu/cpu-exec.o
CC x86_64-softmmu/translate-common.o
CC x86_64-softmmu/cpu-exec-common.o
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/tcg-op.o
CC aarch64-softmmu/exec.o
CC x86_64-softmmu/tcg/optimize.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/disas.o
CC x86_64-softmmu/arch_init.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC aarch64-softmmu/translate-all.o
CC aarch64-softmmu/cpu-exec.o
CC x86_64-softmmu/gdbstub.o
CC aarch64-softmmu/translate-common.o
CC x86_64-softmmu/balloon.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC aarch64-softmmu/cpu-exec-common.o
CC aarch64-softmmu/tcg/tcg.o
CC aarch64-softmmu/tcg/tcg-op.o
CC aarch64-softmmu/tcg/optimize.o
CC x86_64-softmmu/qtest.o
CC aarch64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/bootdevice.o
CC aarch64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/kvm-all.o
CC aarch64-softmmu/disas.o
CC x86_64-softmmu/memory.o
CC x86_64-softmmu/cputlb.o
CC x86_64-softmmu/memory_mapping.o
GEN aarch64-softmmu/gdbstub-xml.c
CC aarch64-softmmu/kvm-stub.o
CC x86_64-softmmu/dump.o
CC x86_64-softmmu/migration/ram.o
CC x86_64-softmmu/migration/savevm.o
CC aarch64-softmmu/arch_init.o
CC aarch64-softmmu/cpus.o
CC aarch64-softmmu/monitor.o
CC aarch64-softmmu/gdbstub.o
CC x86_64-softmmu/xen-common-stub.o
CC aarch64-softmmu/balloon.o
CC aarch64-softmmu/ioport.o
CC x86_64-softmmu/xen-hvm-stub.o
CC aarch64-softmmu/numa.o
CC x86_64-softmmu/hw/acpi/nvdimm.o
CC x86_64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/qtest.o
CC aarch64-softmmu/bootdevice.o
CC x86_64-softmmu/hw/block/dataplane/virtio-blk.o
CC x86_64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/memory.o
CC aarch64-softmmu/cputlb.o
CC aarch64-softmmu/memory_mapping.o
CC aarch64-softmmu/dump.o
CC aarch64-softmmu/migration/ram.o
CC aarch64-softmmu/migration/savevm.o
CC x86_64-softmmu/hw/core/nmi.o
CC aarch64-softmmu/xen-common-stub.o
CC x86_64-softmmu/hw/cpu/core.o
CC x86_64-softmmu/hw/display/vga.o
CC aarch64-softmmu/xen-hvm-stub.o
CC aarch64-softmmu/hw/block/virtio-blk.o
CC x86_64-softmmu/hw/display/virtio-gpu.o
CC x86_64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/hw/char/exynos4210_uart.o
CC aarch64-softmmu/hw/char/omap_uart.o
CC x86_64-softmmu/hw/display/virtio-gpu-pci.o
CC aarch64-softmmu/hw/char/digic-uart.o
CC x86_64-softmmu/hw/display/virtio-vga.o
CC aarch64-softmmu/hw/char/stm32f2xx_usart.o
CC aarch64-softmmu/hw/char/bcm2835_aux.o
CC aarch64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/hw/core/nmi.o
CC x86_64-softmmu/hw/intc/apic.o
CC x86_64-softmmu/hw/intc/apic_common.o
CC x86_64-softmmu/hw/intc/ioapic.o
CC x86_64-softmmu/hw/isa/lpc_ich9.o
CC x86_64-softmmu/hw/misc/vmport.o
CC aarch64-softmmu/hw/cpu/arm11mpcore.o
CC x86_64-softmmu/hw/misc/ivshmem.o
CC x86_64-softmmu/hw/misc/pvpanic.o
CC aarch64-softmmu/hw/cpu/realview_mpcore.o
CC aarch64-softmmu/hw/cpu/a9mpcore.o
CC aarch64-softmmu/hw/cpu/a15mpcore.o
CC aarch64-softmmu/hw/cpu/core.o
CC x86_64-softmmu/hw/misc/edu.o
CC aarch64-softmmu/hw/display/omap_dss.o
CC x86_64-softmmu/hw/misc/hyperv_testdev.o
CC x86_64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/display/omap_lcdc.o
CC x86_64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/display/pxa2xx_lcd.o
CC aarch64-softmmu/hw/display/bcm2835_fb.o
CC x86_64-softmmu/hw/scsi/virtio-scsi.o
CC x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/scsi/vhost-scsi.o
CC aarch64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/hw/display/virtio-gpu-pci.o
CC x86_64-softmmu/hw/timer/mc146818rtc.o
CC aarch64-softmmu/hw/display/dpcd.o
CC x86_64-softmmu/hw/vfio/common.o
CC x86_64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/display/xlnx_dp.o
CC aarch64-softmmu/hw/dma/xlnx_dpdma.o
CC aarch64-softmmu/hw/dma/omap_dma.o
CC aarch64-softmmu/hw/dma/soc_dma.o
CC aarch64-softmmu/hw/dma/pxa2xx_dma.o
CC aarch64-softmmu/hw/dma/bcm2835_dma.o
CC aarch64-softmmu/hw/gpio/omap_gpio.o
CC x86_64-softmmu/hw/vfio/pci-quirks.o
CC x86_64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/gpio/imx_gpio.o
CC x86_64-softmmu/hw/vfio/calxeda-xgmac.o
CC aarch64-softmmu/hw/i2c/omap_i2c.o
CC x86_64-softmmu/hw/vfio/amd-xgbe.o
CC aarch64-softmmu/hw/input/pxa2xx_keypad.o
CC aarch64-softmmu/hw/input/tsc210x.o
CC x86_64-softmmu/hw/vfio/spapr.o
CC x86_64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/intc/armv7m_nvic.o
CC aarch64-softmmu/hw/intc/exynos4210_gic.o
CC x86_64-softmmu/hw/virtio/virtio-balloon.o
CC x86_64-softmmu/hw/virtio/vhost.o
CC x86_64-softmmu/hw/virtio/vhost-backend.o
CC x86_64-softmmu/hw/virtio/vhost-user.o
CC x86_64-softmmu/hw/virtio/vhost-vsock.o
CC aarch64-softmmu/hw/intc/exynos4210_combiner.o
CC aarch64-softmmu/hw/intc/omap_intc.o
CC x86_64-softmmu/hw/i386/multiboot.o
CC x86_64-softmmu/hw/i386/pc.o
CC x86_64-softmmu/hw/i386/pc_piix.o
CC x86_64-softmmu/hw/i386/pc_q35.o
CC x86_64-softmmu/hw/i386/pc_sysfw.o
CC x86_64-softmmu/hw/i386/x86-iommu.o
CC aarch64-softmmu/hw/intc/bcm2835_ic.o
CC aarch64-softmmu/hw/intc/bcm2836_control.o
CC aarch64-softmmu/hw/intc/allwinner-a10-pic.o
CC aarch64-softmmu/hw/intc/aspeed_vic.o
CC x86_64-softmmu/hw/i386/intel_iommu.o
CC x86_64-softmmu/hw/i386/amd_iommu.o
CC aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
CC x86_64-softmmu/hw/i386/kvmvapic.o
CC x86_64-softmmu/hw/i386/acpi-build.o
/tmp/qemu-test/src/hw/i386/pc_piix.c: In function ‘igd_passthrough_isa_bridge_create’:
/tmp/qemu-test/src/hw/i386/pc_piix.c:1046: warning: ‘pch_rev_id’ may be used uninitialized in this function
CC x86_64-softmmu/hw/i386/pci-assign-load-rom.o
CC x86_64-softmmu/hw/i386/kvm/clock.o
CC aarch64-softmmu/hw/misc/ivshmem.o
CC aarch64-softmmu/hw/misc/arm_sysctl.o
CC x86_64-softmmu/hw/i386/kvm/apic.o
CC x86_64-softmmu/hw/i386/kvm/i8259.o
CC x86_64-softmmu/hw/i386/kvm/ioapic.o
CC aarch64-softmmu/hw/misc/cbus.o
CC aarch64-softmmu/hw/misc/exynos4210_pmu.o
CC aarch64-softmmu/hw/misc/imx_ccm.o
/tmp/qemu-test/src/hw/i386/acpi-build.c: In function ‘build_append_pci_bus_devices’:
/tmp/qemu-test/src/hw/i386/acpi-build.c:472: warning: ‘notify_method’ may be used uninitialized in this function
CC aarch64-softmmu/hw/misc/imx31_ccm.o
CC aarch64-softmmu/hw/misc/imx25_ccm.o
CC aarch64-softmmu/hw/misc/imx6_ccm.o
CC x86_64-softmmu/hw/i386/kvm/i8254.o
CC x86_64-softmmu/hw/i386/kvm/pci-assign.o
CC aarch64-softmmu/hw/misc/imx6_src.o
CC x86_64-softmmu/target-i386/translate.o
CC aarch64-softmmu/hw/misc/mst_fpga.o
CC aarch64-softmmu/hw/misc/omap_clk.o
CC aarch64-softmmu/hw/misc/omap_gpmc.o
CC aarch64-softmmu/hw/misc/omap_l4.o
CC x86_64-softmmu/target-i386/helper.o
CC aarch64-softmmu/hw/misc/omap_sdrc.o
CC aarch64-softmmu/hw/misc/omap_tap.o
CC aarch64-softmmu/hw/misc/bcm2835_mbox.o
CC aarch64-softmmu/hw/misc/bcm2835_property.o
CC x86_64-softmmu/target-i386/cpu.o
CC x86_64-softmmu/target-i386/bpt_helper.o
CC x86_64-softmmu/target-i386/excp_helper.o
CC aarch64-softmmu/hw/misc/zynq_slcr.o
CC aarch64-softmmu/hw/misc/zynq-xadc.o
CC aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
CC aarch64-softmmu/hw/misc/edu.o
CC x86_64-softmmu/target-i386/fpu_helper.o
CC x86_64-softmmu/target-i386/cc_helper.o
CC aarch64-softmmu/hw/misc/auxbus.o
CC aarch64-softmmu/hw/misc/aspeed_scu.o
CC x86_64-softmmu/target-i386/int_helper.o
CC x86_64-softmmu/target-i386/svm_helper.o
CC aarch64-softmmu/hw/misc/aspeed_sdmc.o
CC aarch64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/net/vhost_net.o
CC x86_64-softmmu/target-i386/smm_helper.o
CC aarch64-softmmu/hw/pcmcia/pxa2xx.o
CC x86_64-softmmu/target-i386/misc_helper.o
CC aarch64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC x86_64-softmmu/target-i386/mem_helper.o
CC x86_64-softmmu/target-i386/seg_helper.o
CC aarch64-softmmu/hw/scsi/vhost-scsi.o
CC aarch64-softmmu/hw/sd/omap_mmc.o
CC x86_64-softmmu/target-i386/mpx_helper.o
CC x86_64-softmmu/target-i386/gdbstub.o
CC aarch64-softmmu/hw/sd/pxa2xx_mmci.o
CC aarch64-softmmu/hw/ssi/omap_spi.o
CC aarch64-softmmu/hw/ssi/imx_spi.o
CC x86_64-softmmu/target-i386/machine.o
CC aarch64-softmmu/hw/timer/exynos4210_mct.o
CC x86_64-softmmu/target-i386/arch_memory_mapping.o
CC x86_64-softmmu/target-i386/arch_dump.o
CC x86_64-softmmu/target-i386/monitor.o
CC x86_64-softmmu/target-i386/kvm.o
CC x86_64-softmmu/target-i386/hyperv.o
CC aarch64-softmmu/hw/timer/exynos4210_pwm.o
GEN trace/generated-helpers.c
CC x86_64-softmmu/trace/control-target.o
CC aarch64-softmmu/hw/timer/exynos4210_rtc.o
CC aarch64-softmmu/hw/timer/omap_gptimer.o
CC x86_64-softmmu/trace/generated-helpers.o
CC aarch64-softmmu/hw/timer/omap_synctimer.o
CC aarch64-softmmu/hw/timer/pxa2xx_timer.o
CC aarch64-softmmu/hw/timer/digic-timer.o
CC aarch64-softmmu/hw/timer/allwinner-a10-pit.o
CC aarch64-softmmu/hw/usb/tusb6010.o
CC aarch64-softmmu/hw/vfio/common.o
CC aarch64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/vfio/pci-quirks.o
CC aarch64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/vfio/calxeda-xgmac.o
CC aarch64-softmmu/hw/vfio/amd-xgbe.o
CC aarch64-softmmu/hw/vfio/spapr.o
LINK x86_64-softmmu/qemu-system-x86_64
CC aarch64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/virtio/vhost.o
CC aarch64-softmmu/hw/virtio/vhost-backend.o
CC aarch64-softmmu/hw/virtio/vhost-user.o
CC aarch64-softmmu/hw/virtio/vhost-vsock.o
CC aarch64-softmmu/hw/arm/boot.o
CC aarch64-softmmu/hw/arm/collie.o
CC aarch64-softmmu/hw/arm/exynos4_boards.o
CC aarch64-softmmu/hw/arm/gumstix.o
../libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make[1]: *** [qemu-system-x86_64] Error 1
make: *** [subdir-x86_64-softmmu] Error 2
CC aarch64-softmmu/hw/arm/highbank.o
CC aarch64-softmmu/hw/arm/digic_boards.o
CC aarch64-softmmu/hw/arm/integratorcp.o
CC aarch64-softmmu/hw/arm/mainstone.o
CC aarch64-softmmu/hw/arm/musicpal.o
CC aarch64-softmmu/hw/arm/nseries.o
CC aarch64-softmmu/hw/arm/omap_sx1.o
CC aarch64-softmmu/hw/arm/palm.o
CC aarch64-softmmu/hw/arm/realview.o
CC aarch64-softmmu/hw/arm/spitz.o
CC aarch64-softmmu/hw/arm/stellaris.o
CC aarch64-softmmu/hw/arm/tosa.o
CC aarch64-softmmu/hw/arm/versatilepb.o
CC aarch64-softmmu/hw/arm/vexpress.o
CC aarch64-softmmu/hw/arm/virt.o
CC aarch64-softmmu/hw/arm/xilinx_zynq.o
CC aarch64-softmmu/hw/arm/z2.o
CC aarch64-softmmu/hw/arm/virt-acpi-build.o
CC aarch64-softmmu/hw/arm/netduino2.o
CC aarch64-softmmu/hw/arm/sysbus-fdt.o
CC aarch64-softmmu/hw/arm/armv7m.o
CC aarch64-softmmu/hw/arm/exynos4210.o
CC aarch64-softmmu/hw/arm/pxa2xx.o
CC aarch64-softmmu/hw/arm/pxa2xx_gpio.o
CC aarch64-softmmu/hw/arm/pxa2xx_pic.o
CC aarch64-softmmu/hw/arm/digic.o
CC aarch64-softmmu/hw/arm/omap1.o
CC aarch64-softmmu/hw/arm/omap2.o
CC aarch64-softmmu/hw/arm/strongarm.o
CC aarch64-softmmu/hw/arm/allwinner-a10.o
CC aarch64-softmmu/hw/arm/cubieboard.o
CC aarch64-softmmu/hw/arm/bcm2835_peripherals.o
CC aarch64-softmmu/hw/arm/bcm2836.o
CC aarch64-softmmu/hw/arm/raspi.o
CC aarch64-softmmu/hw/arm/stm32f205_soc.o
CC aarch64-softmmu/hw/arm/xlnx-zynqmp.o
CC aarch64-softmmu/hw/arm/xlnx-ep108.o
CC aarch64-softmmu/hw/arm/fsl-imx25.o
CC aarch64-softmmu/hw/arm/imx25_pdk.o
CC aarch64-softmmu/hw/arm/fsl-imx31.o
CC aarch64-softmmu/hw/arm/kzm.o
CC aarch64-softmmu/hw/arm/fsl-imx6.o
CC aarch64-softmmu/hw/arm/sabrelite.o
CC aarch64-softmmu/hw/arm/aspeed_soc.o
CC aarch64-softmmu/hw/arm/aspeed.o
CC aarch64-softmmu/target-arm/arm-semi.o
CC aarch64-softmmu/target-arm/machine.o
CC aarch64-softmmu/target-arm/psci.o
CC aarch64-softmmu/target-arm/arch_dump.o
CC aarch64-softmmu/target-arm/monitor.o
CC aarch64-softmmu/target-arm/kvm-stub.o
CC aarch64-softmmu/target-arm/translate.o
CC aarch64-softmmu/target-arm/op_helper.o
CC aarch64-softmmu/target-arm/helper.o
CC aarch64-softmmu/target-arm/cpu.o
CC aarch64-softmmu/target-arm/neon_helper.o
CC aarch64-softmmu/target-arm/iwmmxt_helper.o
CC aarch64-softmmu/target-arm/gdbstub.o
CC aarch64-softmmu/target-arm/cpu64.o
CC aarch64-softmmu/target-arm/translate-a64.o
CC aarch64-softmmu/target-arm/helper-a64.o
CC aarch64-softmmu/target-arm/gdbstub64.o
CC aarch64-softmmu/target-arm/crypto_helper.o
CC aarch64-softmmu/target-arm/arm-powerctl.o
GEN trace/generated-helpers.c
CC aarch64-softmmu/trace/control-target.o
CC aarch64-softmmu/gdbstub-xml.o
CC aarch64-softmmu/trace/generated-helpers.o
/tmp/qemu-test/src/target-arm/translate-a64.c: In function ‘handle_shri_with_rndacc’:
/tmp/qemu-test/src/target-arm/translate-a64.c:6333: warning: ‘tcg_src_hi’ may be used uninitialized in this function
/tmp/qemu-test/src/target-arm/translate-a64.c: In function ‘disas_simd_scalar_two_reg_misc’:
/tmp/qemu-test/src/target-arm/translate-a64.c:8060: warning: ‘rmode’ may be used uninitialized in this function
LINK aarch64-softmmu/qemu-system-aarch64
../libqemuutil.a(qobject-input-visitor.o): In function `qobject_input_get_object':
/tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: undefined reference to `g_hash_table_contains'
collect2: ld returned 1 exit status
make[1]: *** [qemu-system-aarch64] Error 1
make: *** [subdir-aarch64-softmmu] Error 2
tests/docker/Makefile.include:107: recipe for target 'docker-run-test-quick@centos6' failed
make: *** [docker-run-test-quick@centos6] Error 2
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
@ 2016-09-30 15:49 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-07 13:59 ` [Qemu-devel] " Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 15:49 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 827 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> Allow tracing of the operation of visitors
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> Makefile.objs | 1 +
> qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
> qapi/trace-events | 33 +++++++++++++++++++++++++++++++++
> 3 files changed, 61 insertions(+)
> create mode 100644 qapi/trace-events
Reviewed-by: Eric Blake <eblake@redhat.com>
> +
> +visit_start_alternate(void *v, const char *name, void *obj, size_t size, bool promote_int) "v=%p name=%s obj=%p size=%zu promote_int=%d"
Unrelated to your patch, but should the trace engines be taught how to
honor line wrapping in the trace files?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests Daniel P. Berrange
@ 2016-09-30 17:43 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 17:43 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 940 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> The input_visitor_test_add() method was accepting an instance
> of 'TestInputVisitorData' and passing it as the 'user_data'
> parameter to test functions. The main 'TestInputVisitorData'
> instance that was actually used, was meanwhile being allocated
> automatically by the test framework fixture setup.
>
> The 'user_data' parameter is going to be needed for tests
> added in later patches, so getting rid of the current mistaken
> usage now allows this.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
> 1 file changed, 32 insertions(+), 44 deletions(-)
Reviewed-by: Eric Blake <eblake@redhat.com>
The improved commit message makes the difference :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 17:48 ` Eric Blake
2016-10-11 16:20 ` Markus Armbruster
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 17:48 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> Currently the QObjectInputVisitor assumes that all scalar
> values are directly represented as the final types declared
> by the thing being visited. ie it assumes an 'int' is using
> QInt, and a 'bool' is using QBool, etc. This is good when
> QObjectInputVisitor is fed a QObject that came from a JSON
> document on the QMP monitor, as it will strictly validate
> correctness.
>
> To allow QObjectInputVisitor to be reused for visiting
> a QObject originating from QemuOpts, an alternative mode
> is needed where all the scalars types are represented as
> QString and converted on the fly to the final desired
> type.
>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 32 +++++-
> qapi/qobject-input-visitor.c | 132 ++++++++++++++++++++++++
> tests/test-qobject-input-visitor.c | 194 ++++++++++++++++++++++++++++++++++-
> 3 files changed, 350 insertions(+), 8 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
@ 2016-09-30 17:55 ` Eric Blake
2016-10-06 14:56 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-12 8:08 ` [Qemu-devel] " Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 17:55 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> Instead of requiring all callers to go through the mutli-step
> process of turning QemuOpts into a suitable QObject for visiting,
> add a new constructor that encapsulates this logic. This will
> allow QObjectInputVisitor to be a drop-in replacement for the
> existing OptsVisitor with minimal code changes for callers.
>
> NB, at this point it is only supporting opts syntax which
> explicitly matches the QAPI schema structure, so is not yet
> a true drop-in replacement for OptsVisitor. The patches that
> follow will add the special cases requird for full backwards
s/requird/required/
> compatibility with OptsVisitor.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> +++ b/include/qemu/option.h
> @@ -125,7 +125,7 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
> int permit_abbrev);
> QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
> Error **errp);
> -QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
> void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
Should the const-correctness of this parameter be hoisted to any earlier
patch? But here is fine, as the first place where the compiler requires it.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists Daniel P. Berrange
@ 2016-09-30 17:59 ` Eric Blake
2016-10-12 9:18 ` Markus Armbruster
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 17:59 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> When converting QemuOpts to a QObject, there is no information
> about compound types available, so when visiting a list, the
> corresponding QObject is not guaranteed to be a QList. We
> therefore need to be able to auto-create a single element QList
> from whatever type we find.
>
> This mode should only be enabled if you have compatibility
> requirements for
>
> -arg foo=hello,foo=world
>
> to be treated as equivalent to the preferred syntax:
>
> -arg foo.0=hello,foo.1=world
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 20 +++++++-
> qapi/qobject-input-visitor.c | 27 +++++++++--
> tests/test-qobject-input-visitor.c | 88 +++++++++++++++++++++++++++++++-----
> 3 files changed, 117 insertions(+), 18 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
@ 2016-09-30 18:23 ` Eric Blake
2016-10-06 15:10 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-12 14:12 ` [Qemu-devel] " Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 18:23 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 4736 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> Some of the historical command line opts that had their
> keys in in a completely flat namespace are now represented
> by QAPI schemas that use a nested structs. When converting
singular/plural mismatch; either s/a // or s/structs/struct/
> the QemuOpts to QObject, there is no information about
> compound types available, so the QObject will be completely
> flat, even after the qdict_crumple() call. So when starting
> a struct, we may not have a QDict available in the input
> data, so we auto-create a QDict containing all the currently
> unvisited input data keys. Not all historical command line
> opts require this, so the behaviour is opt-in, by specifying
> how many levels of structs are permitted to be auto-created.
>
> Note that this only works if the child struct is the last
> field to the visited in the parent struct. This is always
> the case for currently existing legacy command line options.
>
> The example is the NetLegacy type which has 3 levels of
> structs. The modern way to represent this in QemuOpts would
> be the dot-separated component approach
>
> -net vlan=1,id=foo,name=bar,opts.type=tap,\
> opts.data.fd=3,opts.data.script=ifup
>
> The legacy syntax will just be presenting
>
> -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
So basically, any key=values that were not consumed as known keys in the
parent struct are reparsed as the (presumably lone) remaining QDict
child, recursing as needed to follow the QAPI nesting.
Does this still work if the command line is presented in a different order:
-net script=ifup,vlan=1,type=tap,fd=3,id=foo,name=bar
My guess is yes, but if the testsuite doesn't cover it, you may want to
add a test.
>
> So we need to auto-create 3 levels of struct when visiting.
>
> The implementation here will enable visiting in both the
> modern and legacy syntax, compared to OptsVisitor which
> only allows the legacy syntax.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 22 +++++-
> qapi/qobject-input-visitor.c | 59 ++++++++++++++--
> tests/test-qobject-input-visitor.c | 130 ++++++++++++++++++++++++++++++++---
> 3 files changed, 194 insertions(+), 17 deletions(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index 1809f48..94051f3 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -45,7 +45,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * If @autocreate_list is true, then as an alternative to a normal QList,
> * list values can be stored as a QString or QDict instead, which will
> * be interpreted as representing single element lists. This should only
> - * by used if compatibility is required with the OptsVisitor which allowed
> + * be used if compatibility is required with the OptsVisitor which allowed
This typo fix should be hoisted earlier in the series when it was
introduced.
> * repeated keys, without list indexes, to represent lists. e.g. set this
> * to true if you have compatibility requirements for
> *
> @@ -55,6 +55,22 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> *
> * -arg foo.0=hello,foo.1=world
> *
> + * If @autocreate_struct_levels is non-zero, then when visiting structs,
> + * if the corresponding QDict is not found, it will automatically create
> + * a QDict containing all remaining unvisited options. This should only
> + * be used if compatibility is required with the OptsVisitor which flatten
> + * structs so that all keys were at the same level. e.g. set this to a
> + * non-zero number if you compatibility requirements for
> + *
> + * -arg type=person,surname=blogs,forename=fred
> + *
> + * to be treated as equivalent to the perferred syntax
s/perferred/preferred/
> +static void test_visitor_in_struct_autocreate(TestInputVisitorData *data,
> + const void *unused)
> +{
> + Visitor *v;
> + int64_t vlan;
> + char *id = NULL;
> + char *type;
> + int64_t fd;
> + char *script = NULL;
> +
> + v = visitor_input_test_init_internal(
> + data, true, true, false, 3,
> + "{ 'vlan': '1', 'id': 'foo', 'type': 'tap', 'fd': '3', "
> + "'script': 'ifup' }", NULL);
> +
This is where intentionally mixing up the parameter orders might be
worthwhile.
Overall, the idea looks reasonable.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties
2016-09-30 15:45 ` [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties no-reply
@ 2016-09-30 18:50 ` Eric Blake
0 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 18:50 UTC (permalink / raw)
To: qemu-devel, berrange; +Cc: famz, qemu-block, armbru, mreitz, pbonzini, afaerber
[-- Attachment #1: Type: text/plain, Size: 999 bytes --]
On 09/30/2016 10:45 AM,
no-reply@ec2-52-6-146-230.compute-1.amazonaws.com wrote:
> Hi,
>
> Your series failed automatic build test. Please find the testing commands and
> their output below. If you have docker installed, you can probably reproduce it
> locally.
>
> CC qapi/string-output-visitor.o
> /tmp/qemu-test/src/qapi/qobject-input-visitor.c: In function ‘qobject_input_get_object’:
> /tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: warning: implicit declaration of function ‘g_hash_table_contains’
> /tmp/qemu-test/src/qapi/qobject-input-visitor.c:97: warning: nested extern declaration of ‘g_hash_table_contains’
>
> tests/docker/Makefile.include:107: recipe for target 'docker-run-test-quick@centos6' failed
> make: *** [docker-run-test-quick@centos6] Error 2
Cool - the autobuilder is now detecting use of too-new glib functions :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor Daniel P. Berrange
@ 2016-09-30 19:45 ` Eric Blake
2016-10-12 15:50 ` Markus Armbruster
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 19:45 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 4933 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> The traditional CLI arg syntax allows two ways to specify
> integer lists, either one value per key, or a range of
> values per key. eg the following are identical:
>
> -arg foo=5,foo=6,foo=7
> -arg foo=5-7
>
> This extends the QObjectInputVisitor so that it is able
> to parse ranges and turn them into distinct list entries.
>
> This means that
>
> -arg foo=5-7
>
> is treated as equivalent to
>
> -arg foo.0=5,foo.1=6,foo.2=7
>
> Edge case tests are copied from test-opts-visitor to
> ensure identical behaviour when parsing.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> @@ -71,6 +77,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * The value given determines how many levels of structs are allowed to
> * be flattened in this way.
> *
> + * If @permit_int_ranges is true, then when visiting a list of integers,
> + * the integer value strings may encode ranges eg a single element
My usual complaint that 'e.g.' is usually spelled with dots :)
> + * containing "5-7" is treated as if there were three elements "5", "6",
> + * "7". This should only be used if compatibility is required with the
> + * OptsVisitor which would allow integer ranges. e.g. set this to true
Ah. Here you DID use dots, but used the wrong abbreviation.
Substituting "For example" in this location does not make as much sense
as "i.e." would. Or go for simplicity; it reads just fine as:
"...would allow integer ranges; so only set this to true if..."
> + * if you have compatibility requirements for
> + *
> + * -arg val=5-8
> + *
> + * to be treated as equivalent to the preferred syntax:
> + *
> + * -arg val.0=5,val.1=6,val.2=7,val.3=8
> + *
> +
> + /*
> + * If we have string that represents an integer range (5-24),
> + * parse the end of the range and set things up so we'll process
> + * the rest of the range before consuming another element
> + * from the QList.
> + */
> + if (end && *end) {
> + if (!qiv->permit_int_ranges) {
> + error_setg(errp,
> + "Integer ranges are not permitted here");
> + return;
> + }
> + if (!inlist) {
> + error_setg(errp,
> + "Integer ranges are only permitted when "
> + "visiting list parameters");
> + return;
> + }
> + if (*end != '-') {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
> + "a number range");
> + return;
> + }
> + end++;
> + if (qemu_strtoll(end, NULL, 0, &ret) < 0) {
Signed parse...
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> + return;
> + }
> + if (*obj > ret) {
...and signed comparison...
> + error_setg(errp,
> + "Parameter '%s' range start %" PRIu64
> + " must be less than (or equal to) %" PRIu64,
...but unsigned numbers in the error message. Looks a bit odd (but not
necessarily worse than what our OptsVisitor was already doing).
> + name, *obj, ret);
> + return;
> + }
> +
> + if ((*obj <= (INT64_MAX - QIV_RANGE_MAX)) &&
> + (ret >= (*obj + QIV_RANGE_MAX))) {
> + error_setg(errp,
> + "Parameter '%s' range must be less than %d",
> + name, QIV_RANGE_MAX);
> + return;
> + }
> + if (*obj != ret) {
> + tos->range_val = *obj;
> + tos->range_limit = ret;
> + }
> + }
> }
Overall, I found this a bit easier to read than the OptsVisitor state
machine. Which makes me worried that we may have some subtle
translation bugs; but I hope it's the same end result.
My consolation is that if there are any differences, they would be in
the corner cases, where a user is probably going to get surprising
behavior from the command line anyway, and shouldn't have been
requesting ranges in that corner case.
>
> static void qobject_input_type_uint64(Visitor *v, const char *name,
> @@ -366,21 +438,85 @@ static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
> uint64_t *obj, Error **errp)
> {
> QObjectInputVisitor *qiv = to_qiv(v);
Feels like a lot of code duplication; but I don't know if there's any
sane way to share more of the code while handling both signed and
unsigned types.
The testsuite additions are reassuring; I confirmed that they match
test-opts-visitor.c in coverage.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
@ 2016-09-30 20:08 ` Eric Blake
2016-10-12 17:46 ` Markus Armbruster
2016-10-13 8:31 ` Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-09-30 20:08 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: qemu-block, Markus Armbruster, Max Reitz, Paolo Bonzini,
Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 5062 bytes --]
On 09/30/2016 09:45 AM, Daniel P. Berrange wrote:
> If given an option string such as
>
> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>
> the qemu_opts_to_qdict() method will currently overwrite
> the values for repeated option keys, so only the last
> value is in the returned dict:
>
> size=QString("1024")
> nodes=QString("1-2")
> policy=QString("bind")
>
> With this change the caller can optionally ask for all
> the repeated values to be stored in a QList. In the
> above example that would result in 'nodes' being a
> QList, so the returned dict would contain
>
> size=QString("1024")
> nodes=QList([QString("10"),
> QString("4-5"),
> QString("1-2")])
> policy=QString("bind")
I think the last time around you were converting keys (turning "nodes"
into "nodes.0"; I think your idea here of keeping a single key tied to
QList makes more sense.
>
> Note that the conversion has no way of knowing whether
> any given key is expected to be a list upfront - it can
> only figure that out when seeing the first duplicated
> key. Thus the caller has to be prepared to deal with the
> fact that if a key 'foo' is a list, then the returned
> qdict may contain either a QString or a QList for the
> key 'foo'.
>
> In a third mode, it is possible to ask for repeated
> options to be reported as an error, rather than silently
> dropping all but the last one.
>
> All existing callers are all converted to explicitly
> request the historical behaviour of only reporting the
> last key. Later patches will make use of the new modes.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> -QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
> +typedef enum {
> + /* Last occurrence of a duplicate option silently replaces earlier */
> + QEMU_OPTS_REPEAT_POLICY_LAST,
> + /* Each occurrence of a duplicate option converts value to a QList */
> + QEMU_OPTS_REPEAT_POLICY_ALL,
> + /* First occurrence of a duplicate option causes an error */
> + QEMU_OPTS_REPEAT_POLICY_ERROR,
> +} QemuOptsRepeatPolicy;
> +
Nicer. Thanks!
> +++ b/tests/test-qemu-opts.c
> @@ -421,6 +421,130 @@ static void test_qemu_opts_set(void)
> g_assert(opts == NULL);
> }
>
> +static void test_qemu_opts_to_qdict_repeat_all(void)
> +{
> + QemuOpts *opts;
> + QDict *dict;
> + QList *list;
> + const QListEntry *ent;
> + QString *str;
> +
> + /* dynamically initialized (parsed) opts */
> + opts = qemu_opts_parse(&opts_list_03,
> + "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
> + false, NULL);
> + g_assert(opts);
> +
> + dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ALL,
> + &error_abort);
> + g_assert(dict);
> +
> + g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
> + g_assert(qdict_haskey(dict, "nodes"));
> + list = qobject_to_qlist(qdict_get(dict, "nodes"));
> + g_assert(list);
> +
> + ent = qlist_first(list);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "10");
> +
> + ent = qlist_next(ent);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "4-5");
> +
> + ent = qlist_next(ent);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "1-2");
Yes, this is different from v13, but does look nicer with the value
turning into QList instead of the key being rewritten into dotted form.
> +++ b/util/qemu-option.c
> @@ -1057,23 +1057,73 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
> * The QDict values are of type QString.
> * TODO We'll want to use types appropriate for opt->desc->type, but
> * this is enough for now.
> + *
> + * If the @opts contains multiple occurrences of the same key,
> + * then the @repeatPolicy parameter determines how they are to
> + * be handled. Traditional behaviour was to only store the
> + * last occurrence, but if @repeatPolicy is set to
> + * QEMU_OPTS_REPEAT_POLICY_ALL, then a QList will be returned
> + * containing all values, for any key with multiple occurrences.
> + * The QEMU_OPTS_REPEAT_POLICY_ERROR value can be used to fail
> + * conversion with an error if multiple occurrences of a key
> + * are seen.
> */
> -QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict)
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
> + QemuOptsRepeatPolicy repeatPolicy,
> + Error **errp)
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 03/21] qapi: add trace events for visitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
2016-09-30 15:49 ` Eric Blake
@ 2016-10-06 14:39 ` Kevin Wolf
2016-10-07 13:59 ` [Qemu-devel] " Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 14:39 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> Allow tracing of the operation of visitors
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests Daniel P. Berrange
2016-09-30 17:43 ` Eric Blake
@ 2016-10-06 14:39 ` Kevin Wolf
1 sibling, 0 replies; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 14:39 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> The input_visitor_test_add() method was accepting an instance
> of 'TestInputVisitorData' and passing it as the 'user_data'
> parameter to test functions. The main 'TestInputVisitorData'
> instance that was actually used, was meanwhile being allocated
> automatically by the test framework fixture setup.
>
> The 'user_data' parameter is going to be needed for tests
> added in later patches, so getting rid of the current mistaken
> usage now allows this.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
2016-09-30 17:55 ` Eric Blake
@ 2016-10-06 14:56 ` Kevin Wolf
2016-10-12 8:08 ` [Qemu-devel] " Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 14:56 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> Instead of requiring all callers to go through the mutli-step
> process of turning QemuOpts into a suitable QObject for visiting,
> add a new constructor that encapsulates this logic. This will
> allow QObjectInputVisitor to be a drop-in replacement for the
> existing OptsVisitor with minimal code changes for callers.
>
> NB, at this point it is only supporting opts syntax which
> explicitly matches the QAPI schema structure, so is not yet
> a true drop-in replacement for OptsVisitor. The patches that
> follow will add the special cases requird for full backwards
> compatibility with OptsVisitor.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
2016-09-30 18:23 ` Eric Blake
@ 2016-10-06 15:10 ` Kevin Wolf
2016-10-06 15:18 ` Daniel P. Berrange
2016-10-12 14:12 ` [Qemu-devel] " Markus Armbruster
2 siblings, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 15:10 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> Some of the historical command line opts that had their
> keys in in a completely flat namespace are now represented
> by QAPI schemas that use a nested structs. When converting
> the QemuOpts to QObject, there is no information about
> compound types available, so the QObject will be completely
> flat, even after the qdict_crumple() call. So when starting
> a struct, we may not have a QDict available in the input
> data, so we auto-create a QDict containing all the currently
> unvisited input data keys. Not all historical command line
> opts require this, so the behaviour is opt-in, by specifying
> how many levels of structs are permitted to be auto-created.
>
> Note that this only works if the child struct is the last
> field to the visited in the parent struct. This is always
> the case for currently existing legacy command line options.
>
> The example is the NetLegacy type which has 3 levels of
> structs. The modern way to represent this in QemuOpts would
> be the dot-separated component approach
>
> -net vlan=1,id=foo,name=bar,opts.type=tap,\
> opts.data.fd=3,opts.data.script=ifup
>
> The legacy syntax will just be presenting
>
> -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
>
> So we need to auto-create 3 levels of struct when visiting.
>
> The implementation here will enable visiting in both the
> modern and legacy syntax, compared to OptsVisitor which
> only allows the legacy syntax.
So you're actually introducing the modern syntax only now?
I don't think an "opts.data." prefix makes a lot of sense. I suspect the
schema looks this way only because we didn't have flat unions in 1.2.
So, considering that it is a purely internally used type not visible in
QMP, would it make sense to change NetLegacy to be a flat union instead,
with NetLegacyOptions as the common base? Then you get the same flat
namespace that we always had and that makes much more sense as an API.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:10 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-06 15:18 ` Daniel P. Berrange
2016-10-06 15:30 ` Kevin Wolf
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-06 15:18 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
> Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> > Some of the historical command line opts that had their
> > keys in in a completely flat namespace are now represented
> > by QAPI schemas that use a nested structs. When converting
> > the QemuOpts to QObject, there is no information about
> > compound types available, so the QObject will be completely
> > flat, even after the qdict_crumple() call. So when starting
> > a struct, we may not have a QDict available in the input
> > data, so we auto-create a QDict containing all the currently
> > unvisited input data keys. Not all historical command line
> > opts require this, so the behaviour is opt-in, by specifying
> > how many levels of structs are permitted to be auto-created.
> >
> > Note that this only works if the child struct is the last
> > field to the visited in the parent struct. This is always
> > the case for currently existing legacy command line options.
> >
> > The example is the NetLegacy type which has 3 levels of
> > structs. The modern way to represent this in QemuOpts would
> > be the dot-separated component approach
> >
> > -net vlan=1,id=foo,name=bar,opts.type=tap,\
> > opts.data.fd=3,opts.data.script=ifup
> >
> > The legacy syntax will just be presenting
> >
> > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
> >
> > So we need to auto-create 3 levels of struct when visiting.
> >
> > The implementation here will enable visiting in both the
> > modern and legacy syntax, compared to OptsVisitor which
> > only allows the legacy syntax.
>
> So you're actually introducing the modern syntax only now?
No, the modern syntax is fully implemented by patch 8.
This patch is about adding hacks for the legacy syntax used
by the OptsVisitor. The OptsVisitor didn't interpret struct
nesting at all, so everything looked flat - this only works
as long as you don't have the same key used in multiple
structs at different levels, so is not useful as a general
approach - it only works by luck really.
> I don't think an "opts.data." prefix makes a lot of sense. I suspect the
> schema looks this way only because we didn't have flat unions in 1.2.
>
> So, considering that it is a purely internally used type not visible in
> QMP, would it make sense to change NetLegacy to be a flat union instead,
> with NetLegacyOptions as the common base? Then you get the same flat
> namespace that we always had and that makes much more sense as an API.
Changing that will impact on the QMP data structure, so I don't think
we can do that.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:18 ` Daniel P. Berrange
@ 2016-10-06 15:30 ` Kevin Wolf
2016-10-06 15:39 ` Daniel P. Berrange
2016-10-06 17:54 ` Eric Blake
0 siblings, 2 replies; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 15:30 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 06.10.2016 um 17:18 hat Daniel P. Berrange geschrieben:
> On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
> > Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> > > Some of the historical command line opts that had their
> > > keys in in a completely flat namespace are now represented
> > > by QAPI schemas that use a nested structs. When converting
> > > the QemuOpts to QObject, there is no information about
> > > compound types available, so the QObject will be completely
> > > flat, even after the qdict_crumple() call. So when starting
> > > a struct, we may not have a QDict available in the input
> > > data, so we auto-create a QDict containing all the currently
> > > unvisited input data keys. Not all historical command line
> > > opts require this, so the behaviour is opt-in, by specifying
> > > how many levels of structs are permitted to be auto-created.
> > >
> > > Note that this only works if the child struct is the last
> > > field to the visited in the parent struct. This is always
> > > the case for currently existing legacy command line options.
> > >
> > > The example is the NetLegacy type which has 3 levels of
> > > structs. The modern way to represent this in QemuOpts would
> > > be the dot-separated component approach
> > >
> > > -net vlan=1,id=foo,name=bar,opts.type=tap,\
> > > opts.data.fd=3,opts.data.script=ifup
> > >
> > > The legacy syntax will just be presenting
> > >
> > > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
> > >
> > > So we need to auto-create 3 levels of struct when visiting.
> > >
> > > The implementation here will enable visiting in both the
> > > modern and legacy syntax, compared to OptsVisitor which
> > > only allows the legacy syntax.
> >
> > So you're actually introducing the modern syntax only now?
>
> No, the modern syntax is fully implemented by patch 8.
"now" in the sense of "in this series" (because this means that there is
no external API to preserve yet).
> This patch is about adding hacks for the legacy syntax used
> by the OptsVisitor. The OptsVisitor didn't interpret struct
> nesting at all, so everything looked flat - this only works
> as long as you don't have the same key used in multiple
> structs at different levels, so is not useful as a general
> approach - it only works by luck really.
>
> > I don't think an "opts.data." prefix makes a lot of sense. I suspect the
> > schema looks this way only because we didn't have flat unions in 1.2.
> >
> > So, considering that it is a purely internally used type not visible in
> > QMP, would it make sense to change NetLegacy to be a flat union instead,
> > with NetLegacyOptions as the common base? Then you get the same flat
> > namespace that we always had and that makes much more sense as an API.
>
> Changing that will impact on the QMP data structure, so I don't think
> we can do that.
I don't see this type used in QMP at all. It's only used for command
line parsing, and only with the OptsVisitor, so I think we're fine if we
flatten it now.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:30 ` Kevin Wolf
@ 2016-10-06 15:39 ` Daniel P. Berrange
2016-10-06 15:51 ` Kevin Wolf
2016-10-06 17:54 ` Eric Blake
1 sibling, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-06 15:39 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
On Thu, Oct 06, 2016 at 05:30:05PM +0200, Kevin Wolf wrote:
> Am 06.10.2016 um 17:18 hat Daniel P. Berrange geschrieben:
> > On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
> > > Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> > > > Some of the historical command line opts that had their
> > > > keys in in a completely flat namespace are now represented
> > > > by QAPI schemas that use a nested structs. When converting
> > > > the QemuOpts to QObject, there is no information about
> > > > compound types available, so the QObject will be completely
> > > > flat, even after the qdict_crumple() call. So when starting
> > > > a struct, we may not have a QDict available in the input
> > > > data, so we auto-create a QDict containing all the currently
> > > > unvisited input data keys. Not all historical command line
> > > > opts require this, so the behaviour is opt-in, by specifying
> > > > how many levels of structs are permitted to be auto-created.
> > > >
> > > > Note that this only works if the child struct is the last
> > > > field to the visited in the parent struct. This is always
> > > > the case for currently existing legacy command line options.
> > > >
> > > > The example is the NetLegacy type which has 3 levels of
> > > > structs. The modern way to represent this in QemuOpts would
> > > > be the dot-separated component approach
> > > >
> > > > -net vlan=1,id=foo,name=bar,opts.type=tap,\
> > > > opts.data.fd=3,opts.data.script=ifup
> > > >
> > > > The legacy syntax will just be presenting
> > > >
> > > > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
> > > >
> > > > So we need to auto-create 3 levels of struct when visiting.
> > > >
> > > > The implementation here will enable visiting in both the
> > > > modern and legacy syntax, compared to OptsVisitor which
> > > > only allows the legacy syntax.
> > >
> > > So you're actually introducing the modern syntax only now?
> >
> > No, the modern syntax is fully implemented by patch 8.
>
> "now" in the sense of "in this series" (because this means that there is
> no external API to preserve yet).
Well the syntax implemented in patch 8 is designed to
100% mirror the QAPI schema structure nesting. I don't
think we want to change that behaviour.
If there are certain QAPI schemas structs can still
have freedom to change though, its fine to reconsider
them
>
> > This patch is about adding hacks for the legacy syntax used
> > by the OptsVisitor. The OptsVisitor didn't interpret struct
> > nesting at all, so everything looked flat - this only works
> > as long as you don't have the same key used in multiple
> > structs at different levels, so is not useful as a general
> > approach - it only works by luck really.
> >
> > > I don't think an "opts.data." prefix makes a lot of sense. I suspect the
> > > schema looks this way only because we didn't have flat unions in 1.2.
> > >
> > > So, considering that it is a purely internally used type not visible in
> > > QMP, would it make sense to change NetLegacy to be a flat union instead,
> > > with NetLegacyOptions as the common base? Then you get the same flat
> > > namespace that we always had and that makes much more sense as an API.
> >
> > Changing that will impact on the QMP data structure, so I don't think
> > we can do that.
>
> I don't see this type used in QMP at all. It's only used for command
> line parsing, and only with the OptsVisitor, so I think we're fine if we
> flatten it now.
Ok, yes, it seems "NetLegacy" is only used in CLI arg parsing, so we
do have some freedom there.
This patch was also needed for -numa handling too - again we might have
some flexibility to flatten that.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:39 ` Daniel P. Berrange
@ 2016-10-06 15:51 ` Kevin Wolf
2016-10-06 15:57 ` Daniel P. Berrange
0 siblings, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-06 15:51 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 06.10.2016 um 17:39 hat Daniel P. Berrange geschrieben:
> On Thu, Oct 06, 2016 at 05:30:05PM +0200, Kevin Wolf wrote:
> > Am 06.10.2016 um 17:18 hat Daniel P. Berrange geschrieben:
> > > On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
> > > > Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> > > > > The example is the NetLegacy type which has 3 levels of
> > > > > structs. The modern way to represent this in QemuOpts would
> > > > > be the dot-separated component approach
> > > > >
> > > > > -net vlan=1,id=foo,name=bar,opts.type=tap,\
> > > > > opts.data.fd=3,opts.data.script=ifup
> > > > >
> > > > > The legacy syntax will just be presenting
> > > > >
> > > > > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
> > > > >
> > > > > So we need to auto-create 3 levels of struct when visiting.
> > > > >
> > > > > The implementation here will enable visiting in both the
> > > > > modern and legacy syntax, compared to OptsVisitor which
> > > > > only allows the legacy syntax.
> > > >
> > > > So you're actually introducing the modern syntax only now?
> > >
> > > No, the modern syntax is fully implemented by patch 8.
> >
> > "now" in the sense of "in this series" (because this means that there is
> > no external API to preserve yet).
>
> Well the syntax implemented in patch 8 is designed to
> 100% mirror the QAPI schema structure nesting. I don't
> think we want to change that behaviour.
I think patch 8 is fine as it is.
> If there are certain QAPI schemas structs can still
> have freedom to change though, its fine to reconsider
> them
>
> >
> > > This patch is about adding hacks for the legacy syntax used
> > > by the OptsVisitor. The OptsVisitor didn't interpret struct
> > > nesting at all, so everything looked flat - this only works
> > > as long as you don't have the same key used in multiple
> > > structs at different levels, so is not useful as a general
> > > approach - it only works by luck really.
> > >
> > > > I don't think an "opts.data." prefix makes a lot of sense. I suspect the
> > > > schema looks this way only because we didn't have flat unions in 1.2.
> > > >
> > > > So, considering that it is a purely internally used type not visible in
> > > > QMP, would it make sense to change NetLegacy to be a flat union instead,
> > > > with NetLegacyOptions as the common base? Then you get the same flat
> > > > namespace that we always had and that makes much more sense as an API.
> > >
> > > Changing that will impact on the QMP data structure, so I don't think
> > > we can do that.
> >
> > I don't see this type used in QMP at all. It's only used for command
> > line parsing, and only with the OptsVisitor, so I think we're fine if we
> > flatten it now.
>
> Ok, yes, it seems "NetLegacy" is only used in CLI arg parsing, so we
> do have some freedom there.
>
> This patch was also needed for -numa handling too - again we might have
> some flexibility to flatten that.
NumaOptions is also unused in QMP, so it's the same thing.
If these two are the only options that need the behaviour, I would
prefer if we changed the QAPI schema to flatten them, and then we could
save ourselves some complexity by dropping this patch.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:51 ` Kevin Wolf
@ 2016-10-06 15:57 ` Daniel P. Berrange
2016-10-12 14:00 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-06 15:57 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-devel, qemu-block, Markus Armbruster, Max Reitz,
Paolo Bonzini, Andreas Färber
On Thu, Oct 06, 2016 at 05:51:57PM +0200, Kevin Wolf wrote:
> Am 06.10.2016 um 17:39 hat Daniel P. Berrange geschrieben:
> > On Thu, Oct 06, 2016 at 05:30:05PM +0200, Kevin Wolf wrote:
> > > Am 06.10.2016 um 17:18 hat Daniel P. Berrange geschrieben:
> > > > On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
> > > > > Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
> > > > > > The example is the NetLegacy type which has 3 levels of
> > > > > > structs. The modern way to represent this in QemuOpts would
> > > > > > be the dot-separated component approach
> > > > > >
> > > > > > -net vlan=1,id=foo,name=bar,opts.type=tap,\
> > > > > > opts.data.fd=3,opts.data.script=ifup
> > > > > >
> > > > > > The legacy syntax will just be presenting
> > > > > >
> > > > > > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
> > > > > >
> > > > > > So we need to auto-create 3 levels of struct when visiting.
> > > > > >
> > > > > > The implementation here will enable visiting in both the
> > > > > > modern and legacy syntax, compared to OptsVisitor which
> > > > > > only allows the legacy syntax.
> > > > >
> > > > > So you're actually introducing the modern syntax only now?
> > > >
> > > > No, the modern syntax is fully implemented by patch 8.
> > >
> > > "now" in the sense of "in this series" (because this means that there is
> > > no external API to preserve yet).
> >
> > Well the syntax implemented in patch 8 is designed to
> > 100% mirror the QAPI schema structure nesting. I don't
> > think we want to change that behaviour.
>
> I think patch 8 is fine as it is.
>
> > If there are certain QAPI schemas structs can still
> > have freedom to change though, its fine to reconsider
> > them
> >
> > >
> > > > This patch is about adding hacks for the legacy syntax used
> > > > by the OptsVisitor. The OptsVisitor didn't interpret struct
> > > > nesting at all, so everything looked flat - this only works
> > > > as long as you don't have the same key used in multiple
> > > > structs at different levels, so is not useful as a general
> > > > approach - it only works by luck really.
> > > >
> > > > > I don't think an "opts.data." prefix makes a lot of sense. I suspect the
> > > > > schema looks this way only because we didn't have flat unions in 1.2.
> > > > >
> > > > > So, considering that it is a purely internally used type not visible in
> > > > > QMP, would it make sense to change NetLegacy to be a flat union instead,
> > > > > with NetLegacyOptions as the common base? Then you get the same flat
> > > > > namespace that we always had and that makes much more sense as an API.
> > > >
> > > > Changing that will impact on the QMP data structure, so I don't think
> > > > we can do that.
> > >
> > > I don't see this type used in QMP at all. It's only used for command
> > > line parsing, and only with the OptsVisitor, so I think we're fine if we
> > > flatten it now.
> >
> > Ok, yes, it seems "NetLegacy" is only used in CLI arg parsing, so we
> > do have some freedom there.
> >
> > This patch was also needed for -numa handling too - again we might have
> > some flexibility to flatten that.
>
> NumaOptions is also unused in QMP, so it's the same thing.
>
> If these two are the only options that need the behaviour, I would
> prefer if we changed the QAPI schema to flatten them, and then we could
> save ourselves some complexity by dropping this patch.
Agreed, the fewer hacks like this that we need the better.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:30 ` Kevin Wolf
2016-10-06 15:39 ` Daniel P. Berrange
@ 2016-10-06 17:54 ` Eric Blake
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-10-06 17:54 UTC (permalink / raw)
To: Kevin Wolf, Daniel P. Berrange
Cc: qemu-block, Markus Armbruster, qemu-devel, Max Reitz,
Paolo Bonzini, Andreas Färber
[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]
On 10/06/2016 10:30 AM, Kevin Wolf wrote:
>>> So, considering that it is a purely internally used type not visible in
>>> QMP, would it make sense to change NetLegacy to be a flat union instead,
>>> with NetLegacyOptions as the common base? Then you get the same flat
>>> namespace that we always had and that makes much more sense as an API.
>>
>> Changing that will impact on the QMP data structure, so I don't think
>> we can do that.
>
> I don't see this type used in QMP at all. It's only used for command
> line parsing, and only with the OptsVisitor, so I think we're fine if we
> flatten it now.
In fact, in all my work to move netdev_add towards QAPI, I intentionally
special-cased NetLegacy to be unchanged, because it was not being used
by QMP at the time, and I didn't want any QMP changes to netdev to break
command line usage of NetLegacy.
We still have the annoying problem that my last patch for converting
netdev_add to QAPI didn't make 2.7, because we hadn't sorted out whether
we wanted to be able to handle back-compat of a user that requested "1"
vs. 1 (the QemuOpts code accepted either spelling, by virtue of the fact
that QDict to opts conversion rewrote the parsed QMP object into all
strings); and maybe this series solves that issue. But the issue for
netdev_add (which IS visible to QMP) is slightly different than the
issue for NetLegacy (which does not have QMP ties other than using QAPI
to define a struct and glue code in net.c to map it back to normal
netdev code).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
2016-09-30 15:49 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-07 13:59 ` Markus Armbruster
2016-10-07 14:16 ` Daniel P. Berrange
2 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-07 13:59 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Allow tracing of the operation of visitors
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> Makefile.objs | 1 +
> qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
> qapi/trace-events | 33 +++++++++++++++++++++++++++++++++
> 3 files changed, 61 insertions(+)
> create mode 100644 qapi/trace-events
>
> diff --git a/Makefile.objs b/Makefile.objs
> index a8e0224..b3e8aef 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -160,3 +160,4 @@ trace-events-y += target-s390x/trace-events
> trace-events-y += target-ppc/trace-events
> trace-events-y += qom/trace-events
> trace-events-y += linux-user/trace-events
> +trace-events-y += qapi/trace-events
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 55f5876..bfcb276 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -19,10 +19,12 @@
> #include "qapi/qmp/qerror.h"
> #include "qapi/visitor.h"
> #include "qapi/visitor-impl.h"
> +#include "trace.h"
>
> void visit_complete(Visitor *v, void *opaque)
> {
> assert(v->type != VISITOR_OUTPUT || v->complete);
> + trace_visit_complete(v, opaque);
Trace after or before checking the precondition? Preferences, anyone?
> if (v->complete) {
> v->complete(v, opaque);
> }
> @@ -30,6 +32,7 @@ void visit_complete(Visitor *v, void *opaque)
>
> void visit_free(Visitor *v)
> {
> + trace_visit_free(v);
> if (v) {
> v->free(v);
> }
> @@ -40,6 +43,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
> {
> Error *err = NULL;
>
> + trace_visit_start_struct(v, name, obj, size);
> if (obj) {
> assert(size);
> assert(!(v->type & VISITOR_OUTPUT) || *obj);
> @@ -53,6 +57,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
>
> void visit_check_struct(Visitor *v, Error **errp)
> {
> + trace_visit_check_struct(v);
> if (v->check_struct) {
> v->check_struct(v, errp);
> }
> @@ -60,6 +65,7 @@ void visit_check_struct(Visitor *v, Error **errp)
>
> void visit_end_struct(Visitor *v, void **obj)
> {
> + trace_visit_end_struct(v, obj);
> v->end_struct(v, obj);
> }
>
> @@ -69,6 +75,7 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
> Error *err = NULL;
>
> assert(!list || size >= sizeof(GenericList));
> + trace_visit_start_list(v, name, list, size);
> v->start_list(v, name, list, size, &err);
> if (list && (v->type & VISITOR_INPUT)) {
> assert(!(err && *list));
> @@ -79,11 +86,13 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
> GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
> {
> assert(tail && size >= sizeof(GenericList));
> + trace_visit_next_list(v, tail, size);
> return v->next_list(v, tail, size);
> }
>
> void visit_end_list(Visitor *v, void **obj)
> {
> + trace_visit_end_list(v, obj);
> v->end_list(v, obj);
> }
>
> @@ -95,6 +104,7 @@ void visit_start_alternate(Visitor *v, const char *name,
>
> assert(obj && size >= sizeof(GenericAlternate));
> assert(!(v->type & VISITOR_OUTPUT) || *obj);
> + trace_visit_start_alternate(v, name, obj, size, promote_int);
> if (v->start_alternate) {
> v->start_alternate(v, name, obj, size, promote_int, &err);
> }
> @@ -106,6 +116,7 @@ void visit_start_alternate(Visitor *v, const char *name,
>
> void visit_end_alternate(Visitor *v, void **obj)
> {
> + trace_visit_end_alternate(v, obj);
> if (v->end_alternate) {
> v->end_alternate(v, obj);
> }
> @@ -113,6 +124,7 @@ void visit_end_alternate(Visitor *v, void **obj)
>
> bool visit_optional(Visitor *v, const char *name, bool *present)
> {
> + trace_visit_optional(v, name, present);
> if (v->optional) {
> v->optional(v, name, present);
> }
> @@ -127,6 +139,7 @@ bool visit_is_input(Visitor *v)
> void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
> {
> assert(obj);
> + trace_visit_type_int(v, name, obj);
> v->type_int64(v, name, obj, errp);
> }
>
> @@ -151,6 +164,7 @@ void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
> Error **errp)
> {
> uint64_t value = *obj;
> + trace_visit_type_uint8(v, name, obj);
> visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
> *obj = value;
> }
Putting the trace in the middle of the "value = *obj;
visit_type_uintN(v, &value, ...); *obj = value" pattern makes it less
visible.
Preserving the pattern requires replacing the initializer by an
assignment:
uint64_t value;
trace_visit_type_uint8(v, name, obj);
value = *obj;
visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
*obj = value;
Looks slightly more legible to me. What do you think?
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor
2016-10-07 13:59 ` [Qemu-devel] " Markus Armbruster
@ 2016-10-07 14:16 ` Daniel P. Berrange
2016-10-21 10:52 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-07 14:16 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Fri, Oct 07, 2016 at 03:59:07PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > Allow tracing of the operation of visitors
> >
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> > Makefile.objs | 1 +
> > qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
> > qapi/trace-events | 33 +++++++++++++++++++++++++++++++++
> > 3 files changed, 61 insertions(+)
> > create mode 100644 qapi/trace-events
> >
> > diff --git a/Makefile.objs b/Makefile.objs
> > index a8e0224..b3e8aef 100644
> > --- a/Makefile.objs
> > +++ b/Makefile.objs
> > @@ -160,3 +160,4 @@ trace-events-y += target-s390x/trace-events
> > trace-events-y += target-ppc/trace-events
> > trace-events-y += qom/trace-events
> > trace-events-y += linux-user/trace-events
> > +trace-events-y += qapi/trace-events
> > diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> > index 55f5876..bfcb276 100644
> > --- a/qapi/qapi-visit-core.c
> > +++ b/qapi/qapi-visit-core.c
> > @@ -19,10 +19,12 @@
> > #include "qapi/qmp/qerror.h"
> > #include "qapi/visitor.h"
> > #include "qapi/visitor-impl.h"
> > +#include "trace.h"
> >
> > void visit_complete(Visitor *v, void *opaque)
> > {
> > assert(v->type != VISITOR_OUTPUT || v->complete);
> > + trace_visit_complete(v, opaque);
>
> Trace after or before checking the precondition? Preferences, anyone?
I'm ambivalent, as the assert will crash you either way so
whether you get a trace event just before the crash seems
mostly irrelevant to me - the abort stack trace is what
you'll use to diagnose the crash.
>
> > if (v->complete) {
> > v->complete(v, opaque);
> > }
> > @@ -30,6 +32,7 @@ void visit_complete(Visitor *v, void *opaque)
> >
> > void visit_free(Visitor *v)
> > {
> > + trace_visit_free(v);
> > if (v) {
> > v->free(v);
> > }
> > @@ -40,6 +43,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
> > {
> > Error *err = NULL;
> >
> > + trace_visit_start_struct(v, name, obj, size);
> > if (obj) {
> > assert(size);
> > assert(!(v->type & VISITOR_OUTPUT) || *obj);
> > @@ -53,6 +57,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
> >
> > void visit_check_struct(Visitor *v, Error **errp)
> > {
> > + trace_visit_check_struct(v);
> > if (v->check_struct) {
> > v->check_struct(v, errp);
> > }
> > @@ -60,6 +65,7 @@ void visit_check_struct(Visitor *v, Error **errp)
> >
> > void visit_end_struct(Visitor *v, void **obj)
> > {
> > + trace_visit_end_struct(v, obj);
> > v->end_struct(v, obj);
> > }
> >
> > @@ -69,6 +75,7 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
> > Error *err = NULL;
> >
> > assert(!list || size >= sizeof(GenericList));
> > + trace_visit_start_list(v, name, list, size);
> > v->start_list(v, name, list, size, &err);
> > if (list && (v->type & VISITOR_INPUT)) {
> > assert(!(err && *list));
> > @@ -79,11 +86,13 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
> > GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
> > {
> > assert(tail && size >= sizeof(GenericList));
> > + trace_visit_next_list(v, tail, size);
> > return v->next_list(v, tail, size);
> > }
> >
> > void visit_end_list(Visitor *v, void **obj)
> > {
> > + trace_visit_end_list(v, obj);
> > v->end_list(v, obj);
> > }
> >
> > @@ -95,6 +104,7 @@ void visit_start_alternate(Visitor *v, const char *name,
> >
> > assert(obj && size >= sizeof(GenericAlternate));
> > assert(!(v->type & VISITOR_OUTPUT) || *obj);
> > + trace_visit_start_alternate(v, name, obj, size, promote_int);
> > if (v->start_alternate) {
> > v->start_alternate(v, name, obj, size, promote_int, &err);
> > }
> > @@ -106,6 +116,7 @@ void visit_start_alternate(Visitor *v, const char *name,
> >
> > void visit_end_alternate(Visitor *v, void **obj)
> > {
> > + trace_visit_end_alternate(v, obj);
> > if (v->end_alternate) {
> > v->end_alternate(v, obj);
> > }
> > @@ -113,6 +124,7 @@ void visit_end_alternate(Visitor *v, void **obj)
> >
> > bool visit_optional(Visitor *v, const char *name, bool *present)
> > {
> > + trace_visit_optional(v, name, present);
> > if (v->optional) {
> > v->optional(v, name, present);
> > }
> > @@ -127,6 +139,7 @@ bool visit_is_input(Visitor *v)
> > void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
> > {
> > assert(obj);
> > + trace_visit_type_int(v, name, obj);
> > v->type_int64(v, name, obj, errp);
> > }
> >
> > @@ -151,6 +164,7 @@ void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
> > Error **errp)
> > {
> > uint64_t value = *obj;
> > + trace_visit_type_uint8(v, name, obj);
> > visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
> > *obj = value;
> > }
>
> Putting the trace in the middle of the "value = *obj;
> visit_type_uintN(v, &value, ...); *obj = value" pattern makes it less
> visible.
>
> Preserving the pattern requires replacing the initializer by an
> assignment:
>
> uint64_t value;
>
> trace_visit_type_uint8(v, name, obj);
> value = *obj;
> visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
> *obj = value;
>
> Looks slightly more legible to me. What do you think?
I'm fine with whatever you prefer.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor Daniel P. Berrange
2016-09-30 17:48 ` Eric Blake
@ 2016-10-11 16:20 ` Markus Armbruster
2016-10-12 14:51 ` Markus Armbruster
1 sibling, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-11 16:20 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Currently the QObjectInputVisitor assumes that all scalar
> values are directly represented as the final types declared
> by the thing being visited. ie it assumes an 'int' is using
i.e.
> QInt, and a 'bool' is using QBool, etc. This is good when
> QObjectInputVisitor is fed a QObject that came from a JSON
> document on the QMP monitor, as it will strictly validate
> correctness.
>
> To allow QObjectInputVisitor to be reused for visiting
> a QObject originating from QemuOpts, an alternative mode
> is needed where all the scalars types are represented as
> QString and converted on the fly to the final desired
> type.
>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 32 +++++-
> qapi/qobject-input-visitor.c | 132 ++++++++++++++++++++++++
> tests/test-qobject-input-visitor.c | 194 ++++++++++++++++++++++++++++++++++-
> 3 files changed, 350 insertions(+), 8 deletions(-)
Quite some code. Here's how I'm persuading myself to like it.
We need to parse two and a half different languages into QAPI objects:
JSON, command line key=value,... (QemuOpts), and just value (same syntax
as in QemuOpts).
Two language differences stand out:
1. JSON is recursive, key=value,... is flat (but see below), and
2. JSON values have syntactically obvious JSON types, key=value values
are just strings.
To parse JSON into a QAPI object, we first parse it into a QObject, then
use the QObject input visitor to convert the QObject into a QAPI object.
To parse key=value, we first parse it into a QemuOpts, then use either
the options visitor to convert the QemuOpts into a QAPI object. We can
also use the string input visitor to convert just an individual key's
value. Or any string for that matter.
Note that we always use the QemuOpts *string* value, never the integer
or boolean value QemuOpts additionally provides when the key has been
specified by a QemuOptDesc with a non-string type. Such QemuOptDesc are
best avoided when we parse the string again with a visitor. For when it
isn't avoided, the visitors' parsers need to match the QemuOpts parser.
Yes, this is a mess.
The flatness of key=value,... has become overly limiting. We've grown
workarounds for lists in both the options and the string input visitor.
Independent ones *sigh*. More recently, the block layer's dotted key
convention has provided a way to overcome flatness completely.
We could implement this convention in the options visitor. Instead,
you're picking a different route: you reuse existing
qemu_opts_to_qdict() to go from QemuOpts to flat qdict, create
qdict_crumple() to unflatten using dotted key convention, then reuse the
existing QObject input visitor to go to QAPI object. This decomposes
the task into smaller ones, some of them solved already, at a tolerable
cost in efficiency.
Except the QObject input visitor won't do as is, because of language
difference 2. This patch makes it cope.
The need to do this negates much of the code reuse advantage. Doesn't
look so hot by itself. However, you then do more work and replace the
options visitor, with an obvious path to replacing the string input
visitor as well.
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index cde328d..5022297 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -19,12 +19,36 @@
>
> typedef struct QObjectInputVisitor QObjectInputVisitor;
>
> -/*
> - * Return a new input visitor that converts a QObject to a QAPI object.
> +/**
> + * Create a new input visitor that converts @obj to a QAPI object.
> + *
> + * Any scalar values in the @obj input data structure should be in the
> + * required type already. i.e. if visiting a bool, the value should
> + * already be a QBool instance.
> *
> - * Set @strict to reject a parse that doesn't consume all keys of a
> - * dictionary; otherwise excess input is ignored.
> + * If @strict is set to true, then an error will be reported if any
> + * dict keys are not consumed during visitation. If @strict is false
> + * then extra dict keys are silently ignored.
Aside: I want to get rid of non-strict, badly. Quoting commit 240f64b
"qapi: Use strict QMP input visitor in more places":
The only remaining uses of non-strict input visits are:
- QMP 'qom-set' (which eventually executes
object_property_set_qobject()) - mark it as something to revisit
in the future (I didn't want to spend any more time on this patch
auditing if we have any QOM dictionary properties that might be
impacted, and couldn't easily prove whether this code path is
shared with anything else).
- test-qmp-input-visitor: explicit tests of non-strict mode. If
we later get rid of users that don't need strictness, then this
test should be merged with test-qmp-input-strict
> + *
> + * The returned input visitor should be released by calling
> + * visit_free() when no longer required.
> */
> Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
>
> +/**
> + * Create a new input visitor that converts @obj to a QAPI object.
> + *
> + * Any scalar values in the @obj input data structure should always be
> + * represented as strings. i.e. if visiting a boolean, the value should
> + * be a QString whose contents represent a valid boolean.
> + *
> + * The visitor always operates in strict mode, requiring all dict keys
> + * to be consumed during visitation. An error will be reported if this
> + * does not happen.
> + *
> + * The returned input visitor should be released by calling
> + * visit_free() when no longer required.
> + */
> +Visitor *qobject_input_visitor_new_autocast(QObject *obj);
Not thrilled about _autocast. It kind of suggests that scalar values
are converted as necessary. They are, but the full truth is that all
scalar values must be strings. _string_scalars is even longer, though.
> +
> #endif
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index 5ff3db3..cf41df6 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -20,6 +20,7 @@
> #include "qemu-common.h"
> #include "qapi/qmp/types.h"
> #include "qapi/qmp/qerror.h"
> +#include "qemu/cutils.h"
>
> #define QIV_STACK_SIZE 1024
>
> @@ -263,6 +264,28 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
> *obj = qint_get_int(qint);
> }
>
> +
> +static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
> + int64_t *obj, Error **errp)
> +{
> + QObjectInputVisitor *qiv = to_qiv(v);
> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
Needs a rebase for commit 1382d4a. Same elsewhere.
> + int64_t ret;
> +
> + if (!qstr || !qstr->string) {
I don't think !qstr->string can happen. Same elsewhere.
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "string");
> + return;
> + }
So far, this is basically qobject_input_type_str() less the g_strdup().
Worth factoring out?
Now we're entering out parsing swamp:
> +
> + if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> + return;
> + }
To serve as replacement for the options visitor, this needs to parse
exactly like opts_type_int64().
It should also match the JSON parser as far as possible, to minimize
difference between the two QObject input visitor variants, and the
QemuOpts parser, for command line consistency.
opts_type_int64() uses strtoll() directly. It carefully checks for no
conversion (both ways, EINVAL and endptr == str), range, and string not
fully consumed.
Your code uses qemu_strtoll(). Bug#1: qemu_strtoll() assumes long long
is exactly 64 bits. If it's wider, we fail to diagnose overflow. If
it's narrower, we can't parse all values. Bug#2: your code fails to
check the string is fully consumed.
The JSON parser also uses strtoll() directly, in parse_literal(). When
we get there, we know that the string consists only of decimal digits,
possibly prefixed by a minus sign. Therefore, strtoll() can only fail
with ERANGE, and parse_literal() handles that correctly.
QemuOpts doesn't do signed integers.
> + *obj = ret;
> +}
> +
> static void qobject_input_type_uint64(Visitor *v, const char *name,
> uint64_t *obj, Error **errp)
> {
> @@ -279,6 +302,27 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
> *obj = qint_get_int(qint);
> }
>
> +static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
> + uint64_t *obj, Error **errp)
> +{
> + QObjectInputVisitor *qiv = to_qiv(v);
> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> + unsigned long long ret;
> +
> + if (!qstr || !qstr->string) {
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "string");
> + return;
> + }
> +
> + if (parse_uint_full(qstr->string, &ret, 0) < 0) {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> + return;
> + }
> + *obj = ret;
Differently wrong :)
To serve as replacement for the options visitor, this needs to parse
exactly like opts_type_uint64().
Again, this should also match the JSON parser and the QemuOpts parser as
far as possible.
opts_type_uint64() uses parse_uint(). It carefully checks for no
conversion (EINVAL; parse_uint() normalizes), range, and string not
fully consumed.
You use parse_uint_full(). You therefore don't have bug#2 here. You do
have bug#1: you assume unsigned long long is exactly 64 bits. If it's
wider, we fail to diagnose overflow. If it's narrower, we can't parse
all values.
There's a discrepancy with the JSON parser, but it's not your fault: the
JSON parser represents JSON numbers that fit into int64_t to QInt, and
any others as QFloat. In particular, the integers between INT64_MAX+1
and UINT64_MAX are represented as QFloat. qmp_input_type_uint64()
accepts only QInt. You can declare things as uint64 in the schema, but
you're still limited to 0..UINT64_MAX in QMP.
QemuOpts uses strtoull() directly, in parse_option_number(). Bug (not
yours): it happily ignores errors other than "string not fully
consumed".
> +}
> +
> static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
> Error **errp)
> {
> @@ -294,6 +338,22 @@ static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
> *obj = qbool_get_bool(qbool);
> }
>
> +static void qobject_input_type_bool_autocast(Visitor *v, const char *name,
> + bool *obj, Error **errp)
> +{
> + QObjectInputVisitor *qiv = to_qiv(v);
> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> +
> + if (!qstr || !qstr->string) {
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "string");
> + return;
> + }
> +
> + parse_option_bool(name, qstr->string, obj, errp);
To serve as replacement for the options visitor, this needs to parse
exactly like opts_type_bool(). Which parses with parse_option_bool(),
too. Good.
QemuOpts parses the same way.
JSON syntax is different.
> +}
> +
> static void qobject_input_type_str(Visitor *v, const char *name, char **obj,
> Error **errp)
> {
> @@ -335,6 +395,30 @@ static void qobject_input_type_number(Visitor *v, const char *name, double *obj,
> "number");
> }
>
> +static void qobject_input_type_number_autocast(Visitor *v, const char *name,
> + double *obj, Error **errp)
> +{
> + QObjectInputVisitor *qiv = to_qiv(v);
> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> + char *endp;
> +
> + if (!qstr || !qstr->string) {
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "string");
> + return;
> + }
> +
> + errno = 0;
> + *obj = strtod(qstr->string, &endp);
> + if (errno == 0 && endp != qstr->string && *endp == '\0') {
> + return;
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "number");
> +}
> +
The options visitor does not implement this method.
Your error checking looks good.
The JSON parser also uses strtod(), in parse_literal(). When we get
there, we know that the string is a valid JSON number. Therefore,
strtod() can only fail with ERANGE. Bug (not yours): we ignore that.
QemuOpts doesn't do floating-point.
> static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj,
> Error **errp)
> {
> @@ -356,6 +440,22 @@ static void qobject_input_type_null(Visitor *v, const char *name, Error **errp)
> }
> }
>
> +static void qobject_input_type_size_autocast(Visitor *v, const char *name,
> + uint64_t *obj, Error **errp)
> +{
> + QObjectInputVisitor *qiv = to_qiv(v);
> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> +
> + if (!qstr || !qstr->string) {
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "string");
> + return;
> + }
> +
> + parse_option_size(name, qstr->string, obj, errp);
The options visitor uses qemu_strtosz_suffix(). Are you sure
parse_option_size() is a compatible replacement? (I'm getting tired...)
> +}
> +
> static void qobject_input_optional(Visitor *v, const char *name, bool *present)
> {
> QObjectInputVisitor *qiv = to_qiv(v);
> @@ -413,3 +513,35 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
>
> return &v->visitor;
> }
> +
> +Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> +{
> + QObjectInputVisitor *v;
> +
> + v = g_malloc0(sizeof(*v));
> +
> + v->visitor.type = VISITOR_INPUT;
> + v->visitor.start_struct = qobject_input_start_struct;
> + v->visitor.check_struct = qobject_input_check_struct;
> + v->visitor.end_struct = qobject_input_pop;
> + v->visitor.start_list = qobject_input_start_list;
> + v->visitor.next_list = qobject_input_next_list;
> + v->visitor.end_list = qobject_input_pop;
> + v->visitor.start_alternate = qobject_input_start_alternate;
> + v->visitor.type_int64 = qobject_input_type_int64_autocast;
> + v->visitor.type_uint64 = qobject_input_type_uint64_autocast;
> + v->visitor.type_bool = qobject_input_type_bool_autocast;
> + v->visitor.type_str = qobject_input_type_str;
> + v->visitor.type_number = qobject_input_type_number_autocast;
> + v->visitor.type_any = qobject_input_type_any;
> + v->visitor.type_null = qobject_input_type_null;
> + v->visitor.type_size = qobject_input_type_size_autocast;
> + v->visitor.optional = qobject_input_optional;
> + v->visitor.free = qobject_input_free;
> + v->strict = true;
> +
> + v->root = obj;
> + qobject_incref(obj);
> +
> + return &v->visitor;
> +}
> diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
> index 26c5012..d5b8044 100644
> --- a/tests/test-qobject-input-visitor.c
> +++ b/tests/test-qobject-input-visitor.c
> @@ -41,6 +41,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
> function so that the JSON string used by the tests are kept in the test
> functions (and not in main()). */
> static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> + bool strict, bool autocast,
> const char *json_string,
> va_list *ap)
> {
> @@ -49,11 +50,31 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> - data->qiv = qobject_input_visitor_new(data->obj, false);
> + if (autocast) {
> + assert(strict);
> + data->qiv = qobject_input_visitor_new_autocast(data->obj);
> + } else {
> + data->qiv = qobject_input_visitor_new(data->obj, strict);
> + }
> g_assert(data->qiv);
> return data->qiv;
> }
>
> +static GCC_FMT_ATTR(4, 5)
> +Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
> + bool strict, bool autocast,
> + const char *json_string, ...)
> +{
> + Visitor *v;
> + va_list ap;
> +
> + va_start(ap, json_string);
> + v = visitor_input_test_init_internal(data, strict, autocast,
> + json_string, &ap);
> + va_end(ap);
> + return v;
> +}
> +
> static GCC_FMT_ATTR(2, 3)
> Visitor *visitor_input_test_init(TestInputVisitorData *data,
> const char *json_string, ...)
> @@ -62,7 +83,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> va_list ap;
>
> va_start(ap, json_string);
> - v = visitor_input_test_init_internal(data, json_string, &ap);
> + v = visitor_input_test_init_internal(data, true, false,
> + json_string, &ap);
> va_end(ap);
> return v;
> }
> @@ -77,7 +99,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
> const char *json_string)
> {
> - return visitor_input_test_init_internal(data, json_string, NULL);
> + return visitor_input_test_init_internal(data, true, false,
> + json_string, NULL);
> }
>
> static void test_visitor_in_int(TestInputVisitorData *data,
> @@ -109,6 +132,45 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
> error_free_or_abort(&err);
> }
>
> +static void test_visitor_in_int_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0, value = -42;
> + Error *err = NULL;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true,
> + "%" PRId64, value);
> + visit_type_int(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> +static void test_visitor_in_int_str_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0, value = -42;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true,
> + "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, value);
> +}
> +
> +static void test_visitor_in_int_str_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> static void test_visitor_in_bool(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -121,6 +183,44 @@ static void test_visitor_in_bool(TestInputVisitorData *data,
> g_assert_cmpint(res, ==, true);
> }
>
> +static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Error *err = NULL;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true, "true");
> +
> + visit_type_bool(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> +static void test_visitor_in_bool_str_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true, "\"yes\"");
> +
> + visit_type_bool(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, true);
> +}
> +
> +static void test_visitor_in_bool_str_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"true\"");
> +
> + visit_type_bool(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> static void test_visitor_in_number(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -133,6 +233,69 @@ static void test_visitor_in_number(TestInputVisitorData *data,
> g_assert_cmpfloat(res, ==, value);
> }
>
> +static void test_visitor_in_number_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0, value = 3.14;
> + Error *err = NULL;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true, "%f", value);
> +
> + visit_type_number(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> +static void test_visitor_in_number_str_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0, value = 3.14;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &error_abort);
> + g_assert_cmpfloat(res, ==, value);
> +}
> +
> +static void test_visitor_in_number_str_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> +static void test_visitor_in_size_str_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + uint64_t res, value = 500 * 1024 * 1024;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, true, true, "\"500M\"");
> +
> + visit_type_size(v, NULL, &res, &error_abort);
> + g_assert_cmpfloat(res, ==, value);
> +}
> +
> +static void test_visitor_in_size_str_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + uint64_t res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"500M\"");
> +
> + visit_type_size(v, NULL, &res, &err);
> + error_free_or_abort(&err);
> +}
> +
> static void test_visitor_in_string(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -289,7 +452,8 @@ static void test_visitor_in_null(TestInputVisitorData *data,
> * when input is not null.
> */
>
> - v = visitor_input_test_init(data, "{ 'a': null, 'b': '' }");
> + v = visitor_input_test_init_full(data, false, false,
> + "{ 'a': null, 'b': '' }");
> visit_start_struct(v, NULL, NULL, 0, &error_abort);
> visit_type_null(v, "a", &error_abort);
> visit_type_str(v, "a", &tmp, &err);
> @@ -840,10 +1004,32 @@ int main(int argc, char **argv)
> NULL, test_visitor_in_int);
> input_visitor_test_add("/visitor/input/int_overflow",
> NULL, test_visitor_in_int_overflow);
> + input_visitor_test_add("/visitor/input/int_autocast",
> + NULL, test_visitor_in_int_autocast);
> + input_visitor_test_add("/visitor/input/int_str_autocast",
> + NULL, test_visitor_in_int_str_autocast);
> + input_visitor_test_add("/visitor/input/int_str_noautocast",
> + NULL, test_visitor_in_int_str_noautocast);
> input_visitor_test_add("/visitor/input/bool",
> NULL, test_visitor_in_bool);
> + input_visitor_test_add("/visitor/input/bool_autocast",
> + NULL, test_visitor_in_bool_autocast);
> + input_visitor_test_add("/visitor/input/bool_str_autocast",
> + NULL, test_visitor_in_bool_str_autocast);
> + input_visitor_test_add("/visitor/input/bool_str_noautocast",
> + NULL, test_visitor_in_bool_str_noautocast);
> input_visitor_test_add("/visitor/input/number",
> NULL, test_visitor_in_number);
> + input_visitor_test_add("/visitor/input/number_autocast",
> + NULL, test_visitor_in_number_autocast);
> + input_visitor_test_add("/visitor/input/number_str_autocast",
> + NULL, test_visitor_in_number_str_autocast);
> + input_visitor_test_add("/visitor/input/number_str_noautocast",
> + NULL, test_visitor_in_number_str_noautocast);
> + input_visitor_test_add("/visitor/input/size_str_autocast",
> + NULL, test_visitor_in_size_str_autocast);
> + input_visitor_test_add("/visitor/input/size_str_noautocast",
> + NULL, test_visitor_in_size_str_noautocast);
> input_visitor_test_add("/visitor/input/string",
> NULL, test_visitor_in_string);
> input_visitor_test_add("/visitor/input/enum",
I don't like the test code duplication. The alternative would be a
function that rewrites all the scalars in a QObject to strings.
Probably not worthwhile. We'll live with the duplication then.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
2016-09-30 17:55 ` Eric Blake
2016-10-06 14:56 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-12 8:08 ` Markus Armbruster
2016-10-13 7:23 ` Markus Armbruster
2 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 8:08 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Instead of requiring all callers to go through the mutli-step
multi-step
> process of turning QemuOpts into a suitable QObject for visiting,
> add a new constructor that encapsulates this logic. This will
> allow QObjectInputVisitor to be a drop-in replacement for the
> existing OptsVisitor with minimal code changes for callers.
>
> NB, at this point it is only supporting opts syntax which
> explicitly matches the QAPI schema structure, so is not yet
> a true drop-in replacement for OptsVisitor. The patches that
> follow will add the special cases requird for full backwards
> compatibility with OptsVisitor.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 15 +++++++++++++++
> include/qemu/option.h | 2 +-
> qapi/qobject-input-visitor.c | 25 +++++++++++++++++++++++++
> util/qemu-option.c | 2 +-
> 4 files changed, 42 insertions(+), 2 deletions(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index 5022297..f134d90 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -51,4 +51,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> */
> Visitor *qobject_input_visitor_new_autocast(QObject *obj);
>
> +
> +/**
> + * Create a new input visitor that converts @opts to a QAPI object.
> + *
> + * The QemuOpts will be converted into a QObject using the
> + * qdict_crumple() method to automatically create structs
> + * and lists. The resulting QDict will then be passed to the
> + * qobject_input_visitor_new_autocast() method. See the docs
> + * of that method for further details on processing behaviour.
> + *
> + * The returned input visitor should be released by calling
> + * visit_free() when no longer required.
> + */
> +Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts, Error **errp);
> +
> #endif
> diff --git a/include/qemu/option.h b/include/qemu/option.h
> index 2a5266f..29f3f18 100644
> --- a/include/qemu/option.h
> +++ b/include/qemu/option.h
> @@ -125,7 +125,7 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
> int permit_abbrev);
> QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
> Error **errp);
> -QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
> void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
>
> typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index cf41df6..d9269c9 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -545,3 +545,28 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>
> return &v->visitor;
> }
> +
> +
> +Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> + Error **errp)
> +{
> + QDict *pdict;
> + QObject *pobj = NULL;
@pdict and @pobj are unusual names. Let's stick to the more common
@dict and @obj.
> + Visitor *v = NULL;
> +
> + pdict = qemu_opts_to_qdict(opts, NULL);
> + if (!pdict) {
> + goto cleanup;
> + }
> +
> + pobj = qdict_crumple(pdict, true, errp);
> + if (!pobj) {
> + goto cleanup;
> + }
> +
> + v = qobject_input_visitor_new_autocast(pobj);
> + cleanup:
> + qobject_decref(pobj);
> + QDECREF(pdict);
> + return v;
> +}
> diff --git a/util/qemu-option.c b/util/qemu-option.c
> index 41b356c..418cbb6 100644
> --- a/util/qemu-option.c
> +++ b/util/qemu-option.c
> @@ -1058,7 +1058,7 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
> * TODO We'll want to use types appropriate for opt->desc->type, but
> * this is enough for now.
Implementing this TODO could break users that pass it to
qobject_input_visitor_new_autocast(), because despite its name, the
_autocast visitor expects *only* string scalars.
I guess (but have not checked) that these users have null opt->desc
since their opts->list->desc[] is empty.
We should either drop the TODO (needs review of the other users), or
update it to reflect the new situation. Perhaps users of
qobject_input_visitor_new_autocast() should additionally assert
opts->list->desc[] is empty.
> */
> -QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict)
> {
> QemuOpt *opt;
> QObject *val;
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists Daniel P. Berrange
2016-09-30 17:59 ` Eric Blake
@ 2016-10-12 9:18 ` Markus Armbruster
2016-10-20 14:23 ` Daniel P. Berrange
1 sibling, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 9:18 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> When converting QemuOpts to a QObject, there is no information
> about compound types available,
Yes, that's a drawback of splitting the conversion into a QemuOpts ->
QObject part that is oblivious of types, and a QObject -> QAPI object
part that knows the types.
> so when visiting a list, the
> corresponding QObject is not guaranteed to be a QList. We
> therefore need to be able to auto-create a single element QList
> from whatever type we find.
>
> This mode should only be enabled if you have compatibility
> requirements for
>
> -arg foo=hello,foo=world
>
> to be treated as equivalent to the preferred syntax:
>
> -arg foo.0=hello,foo.1=world
Not sure this is "preferred". "More powerfully warty" is probably
closer to the truth ;)
How is "-arg foo=hello,foo=world" treated if this mode isn't enabled?
What would be the drawbacks of doing this always instead of only when we
"have compatibility requirements"?
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 20 +++++++-
> qapi/qobject-input-visitor.c | 27 +++++++++--
> tests/test-qobject-input-visitor.c | 88 +++++++++++++++++++++++++++++++-----
> 3 files changed, 117 insertions(+), 18 deletions(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index f134d90..1809f48 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -42,6 +42,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * represented as strings. i.e. if visiting a boolean, the value should
> * be a QString whose contents represent a valid boolean.
> *
> + * If @autocreate_list is true, then as an alternative to a normal QList,
> + * list values can be stored as a QString or QDict instead, which will
> + * be interpreted as representing single element lists. This should only
> + * by used if compatibility is required with the OptsVisitor which allowed
> + * repeated keys, without list indexes, to represent lists. e.g. set this
> + * to true if you have compatibility requirements for
> + *
> + * -arg foo=hello,foo=world
> + *
> + * to be treated as equivalent to the preferred syntax:
> + *
> + * -arg foo.0=hello,foo.1=world
> + *
> * The visitor always operates in strict mode, requiring all dict keys
> * to be consumed during visitation. An error will be reported if this
> * does not happen.
> @@ -49,7 +62,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * The returned input visitor should be released by calling
> * visit_free() when no longer required.
> */
> -Visitor *qobject_input_visitor_new_autocast(QObject *obj);
> +Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> + bool autocreate_list);
>
>
> /**
> @@ -64,6 +78,8 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj);
> * The returned input visitor should be released by calling
> * visit_free() when no longer required.
> */
> -Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts, Error **errp);
> +Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> + bool autocreate_list,
> + Error **errp);
>
> #endif
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index d9269c9..d88e9f9 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -48,6 +48,10 @@ struct QObjectInputVisitor
>
> /* True to reject parse in visit_end_struct() if unvisited keys remain. */
> bool strict;
> +
> + /* Whether we can auto-create single element lists when
> + * encountering a non-QList type */
> + bool autocreate_list;
> };
>
> static QObjectInputVisitor *to_qiv(Visitor *v)
> @@ -108,6 +112,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
> assert(obj);
> tos->obj = obj;
> tos->qapi = qapi;
> + qobject_incref(obj);
>
> if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
> h = g_hash_table_new(g_str_hash, g_str_equal);
> @@ -147,6 +152,7 @@ static void qobject_input_stack_object_free(StackObject *tos)
> if (tos->h) {
> g_hash_table_unref(tos->h);
> }
> + qobject_decref(tos->obj);
>
> g_free(tos);
> }
Can you explain the reference counting change?
> @@ -197,7 +203,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
> QObject *qobj = qobject_input_get_object(qiv, name, true);
> const QListEntry *entry;
>
> - if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
> + if (!qobj || (!qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST)) {
Long line, but I believe it'll go away when you rebase for commit
1382d4a.
> if (list) {
> *list = NULL;
> }
> @@ -206,7 +212,16 @@ static void qobject_input_start_list(Visitor *v, const char *name,
> return;
> }
>
> - entry = qobject_input_push(qiv, qobj, list, errp);
> + if (qobject_type(qobj) != QTYPE_QLIST) {
> + QList *tmplist = qlist_new();
> + qlist_append_obj(tmplist, qobj);
> + qobject_incref(qobj);
> + entry = qobject_input_push(qiv, QOBJECT(tmplist), list, errp);
> + QDECREF(tmplist);
> + } else {
> + entry = qobject_input_push(qiv, qobj, list, errp);
> + }
> +
> if (list) {
> if (entry) {
> *list = g_malloc0(size);
Buries autolist behavior in the middle of things. What about doing it
first, so it's more separate?
QObjectInputVisitor *qiv = to_qiv(v);
QObject *qobj = qobject_input_get_object_(qiv, name, true, errp);
const QListEntry *entry;
if (!qobj) {
return;
}
+ if (qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST) {
+ QList *auto_list = qlist_new();
+ qlist_append_obj(auto_list, qobj);
+ qobj = auto_list;
+ }
+
if (qobject_type(qobj) != QTYPE_QLIST) {
I ignored reference counting here, because I don't yet understand how
and why you're changing it.
> @@ -514,7 +529,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
> return &v->visitor;
> }
>
> -Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> +Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> + bool autocreate_list)
> {
> QObjectInputVisitor *v;
>
> @@ -539,6 +555,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> v->visitor.optional = qobject_input_optional;
> v->visitor.free = qobject_input_free;
> v->strict = true;
> + v->autocreate_list = autocreate_list;
>
> v->root = obj;
> qobject_incref(obj);
> @@ -548,6 +565,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>
>
> Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> + bool autocreate_list,
> Error **errp)
> {
> QDict *pdict;
> @@ -564,7 +582,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> goto cleanup;
> }
>
> - v = qobject_input_visitor_new_autocast(pobj);
> + v = qobject_input_visitor_new_autocast(pobj,
> + autocreate_list);
> cleanup:
> qobject_decref(pobj);
> QDECREF(pdict);
[Skipping test updates for now...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-10-06 15:57 ` Daniel P. Berrange
@ 2016-10-12 14:00 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 14:00 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Kevin Wolf, qemu-block, qemu-devel, Max Reitz, Paolo Bonzini,
Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Thu, Oct 06, 2016 at 05:51:57PM +0200, Kevin Wolf wrote:
>> Am 06.10.2016 um 17:39 hat Daniel P. Berrange geschrieben:
>> > On Thu, Oct 06, 2016 at 05:30:05PM +0200, Kevin Wolf wrote:
>> > > Am 06.10.2016 um 17:18 hat Daniel P. Berrange geschrieben:
>> > > > On Thu, Oct 06, 2016 at 05:10:42PM +0200, Kevin Wolf wrote:
>> > > > > Am 30.09.2016 um 16:45 hat Daniel P. Berrange geschrieben:
>> > > > > > The example is the NetLegacy type which has 3 levels of
>> > > > > > structs. The modern way to represent this in QemuOpts would
>> > > > > > be the dot-separated component approach
>> > > > > >
>> > > > > > -net vlan=1,id=foo,name=bar,opts.type=tap,\
>> > > > > > opts.data.fd=3,opts.data.script=ifup
>> > > > > >
>> > > > > > The legacy syntax will just be presenting
>> > > > > >
>> > > > > > -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
>> > > > > >
>> > > > > > So we need to auto-create 3 levels of struct when visiting.
>> > > > > >
>> > > > > > The implementation here will enable visiting in both the
>> > > > > > modern and legacy syntax, compared to OptsVisitor which
>> > > > > > only allows the legacy syntax.
>> > > > >
>> > > > > So you're actually introducing the modern syntax only now?
>> > > >
>> > > > No, the modern syntax is fully implemented by patch 8.
>> > >
>> > > "now" in the sense of "in this series" (because this means that there is
>> > > no external API to preserve yet).
>> >
>> > Well the syntax implemented in patch 8 is designed to
>> > 100% mirror the QAPI schema structure nesting. I don't
>> > think we want to change that behaviour.
>>
>> I think patch 8 is fine as it is.
>>
>> > If there are certain QAPI schemas structs can still
>> > have freedom to change though, its fine to reconsider
>> > them
>> >
>> > >
>> > > > This patch is about adding hacks for the legacy syntax used
>> > > > by the OptsVisitor. The OptsVisitor didn't interpret struct
>> > > > nesting at all, so everything looked flat - this only works
>> > > > as long as you don't have the same key used in multiple
>> > > > structs at different levels, so is not useful as a general
>> > > > approach - it only works by luck really.
>> > > >
>> > > > > I don't think an "opts.data." prefix makes a lot of sense. I suspect the
>> > > > > schema looks this way only because we didn't have flat unions in 1.2.
>> > > > >
>> > > > > So, considering that it is a purely internally used type not visible in
>> > > > > QMP, would it make sense to change NetLegacy to be a flat union instead,
>> > > > > with NetLegacyOptions as the common base? Then you get the same flat
>> > > > > namespace that we always had and that makes much more sense as an API.
>> > > >
>> > > > Changing that will impact on the QMP data structure, so I don't think
>> > > > we can do that.
>> > >
>> > > I don't see this type used in QMP at all. It's only used for command
>> > > line parsing, and only with the OptsVisitor, so I think we're fine if we
>> > > flatten it now.
>> >
>> > Ok, yes, it seems "NetLegacy" is only used in CLI arg parsing, so we
>> > do have some freedom there.
>> >
>> > This patch was also needed for -numa handling too - again we might have
>> > some flexibility to flatten that.
>>
>> NumaOptions is also unused in QMP, so it's the same thing.
For better or worse, the QAPI schema doesn't separate externally visible
stuff from purely internal stuff. Useful trick to see what's external:
$ python -B scripts/qapi-introspect.py -cu -p scratch- qapi-schema.json
This generates scratch-qmp-introspect.c describing the external QMP
interface with unmasked type names (-u). Anything not mentioned there
can be changed freely.
I'd like to see patches flattening simple unions that aren't externally
visible. Flat carries a bit of notational overhead in the schema, but I
hope to address that once the QAPI queue is under control.
>> If these two are the only options that need the behaviour, I would
>> prefer if we changed the QAPI schema to flatten them, and then we could
>> save ourselves some complexity by dropping this patch.
>
> Agreed, the fewer hacks like this that we need the better.
Indeed. Please find out which hacks we actually need.
I believe the best way to keep the warts in check is getting rid of the
options visitor (done in this series) and the string visitor (hopefully
enabled by this series).
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
2016-09-30 18:23 ` Eric Blake
2016-10-06 15:10 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-12 14:12 ` Markus Armbruster
2 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 14:12 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Some of the historical command line opts that had their
> keys in in a completely flat namespace are now represented
> by QAPI schemas that use a nested structs. When converting
> the QemuOpts to QObject, there is no information about
> compound types available, so the QObject will be completely
> flat, even after the qdict_crumple() call. So when starting
> a struct, we may not have a QDict available in the input
> data, so we auto-create a QDict containing all the currently
> unvisited input data keys. Not all historical command line
> opts require this, so the behaviour is opt-in, by specifying
> how many levels of structs are permitted to be auto-created.
>
> Note that this only works if the child struct is the last
> field to the visited in the parent struct. This is always
> the case for currently existing legacy command line options.
>
> The example is the NetLegacy type which has 3 levels of
> structs. The modern way to represent this in QemuOpts would
> be the dot-separated component approach
>
> -net vlan=1,id=foo,name=bar,opts.type=tap,\
> opts.data.fd=3,opts.data.script=ifup
>
> The legacy syntax will just be presenting
>
> -net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup
>
> So we need to auto-create 3 levels of struct when visiting.
>
> The implementation here will enable visiting in both the
> modern and legacy syntax, compared to OptsVisitor which
> only allows the legacy syntax.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 22 +++++-
> qapi/qobject-input-visitor.c | 59 ++++++++++++++--
> tests/test-qobject-input-visitor.c | 130 ++++++++++++++++++++++++++++++++---
> 3 files changed, 194 insertions(+), 17 deletions(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index 1809f48..94051f3 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -45,7 +45,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * If @autocreate_list is true, then as an alternative to a normal QList,
> * list values can be stored as a QString or QDict instead, which will
> * be interpreted as representing single element lists. This should only
> - * by used if compatibility is required with the OptsVisitor which allowed
> + * be used if compatibility is required with the OptsVisitor which allowed
> * repeated keys, without list indexes, to represent lists. e.g. set this
> * to true if you have compatibility requirements for
> *
Typo introduced in the previous patch, please fix it there.
Skipping the rest of this patch until it's clear we need it.
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-10-11 16:20 ` Markus Armbruster
@ 2016-10-12 14:51 ` Markus Armbruster
2016-10-12 15:05 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 14:51 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
Markus Armbruster <armbru@redhat.com> writes:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> Currently the QObjectInputVisitor assumes that all scalar
>> values are directly represented as the final types declared
>> by the thing being visited. ie it assumes an 'int' is using
>
> i.e.
>
>> QInt, and a 'bool' is using QBool, etc. This is good when
>> QObjectInputVisitor is fed a QObject that came from a JSON
>> document on the QMP monitor, as it will strictly validate
>> correctness.
>>
>> To allow QObjectInputVisitor to be reused for visiting
>> a QObject originating from QemuOpts, an alternative mode
>> is needed where all the scalars types are represented as
>> QString and converted on the fly to the final desired
>> type.
>>
>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
[...]
>> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
>> index 5ff3db3..cf41df6 100644
>> --- a/qapi/qobject-input-visitor.c
>> +++ b/qapi/qobject-input-visitor.c
>> @@ -20,6 +20,7 @@
>> #include "qemu-common.h"
>> #include "qapi/qmp/types.h"
>> #include "qapi/qmp/qerror.h"
>> +#include "qemu/cutils.h"
>>
>> #define QIV_STACK_SIZE 1024
>>
>> @@ -263,6 +264,28 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
>> *obj = qint_get_int(qint);
>> }
>>
>> +
>> +static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
>> + int64_t *obj, Error **errp)
>> +{
>> + QObjectInputVisitor *qiv = to_qiv(v);
>> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>> + true));
>
> Needs a rebase for commit 1382d4a. Same elsewhere.
>
>> + int64_t ret;
>> +
>> + if (!qstr || !qstr->string) {
>
> I don't think !qstr->string can happen. Same elsewhere.
>
>> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>> + "string");
>> + return;
>> + }
>
> So far, this is basically qobject_input_type_str() less the g_strdup().
> Worth factoring out?
>
> Now we're entering out parsing swamp:
>
>> +
>> + if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
>> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
"integer", actually.
>> + return;
>> + }
>
> To serve as replacement for the options visitor, this needs to parse
> exactly like opts_type_int64().
>
> It should also match the JSON parser as far as possible, to minimize
> difference between the two QObject input visitor variants, and the
> QemuOpts parser, for command line consistency.
>
> opts_type_int64() uses strtoll() directly. It carefully checks for no
> conversion (both ways, EINVAL and endptr == str), range, and string not
> fully consumed.
>
> Your code uses qemu_strtoll(). Bug#1: qemu_strtoll() assumes long long
> is exactly 64 bits. If it's wider, we fail to diagnose overflow. If
> it's narrower, we can't parse all values. Bug#2: your code fails to
> check the string is fully consumed.
>
> The JSON parser also uses strtoll() directly, in parse_literal(). When
> we get there, we know that the string consists only of decimal digits,
> possibly prefixed by a minus sign. Therefore, strtoll() can only fail
> with ERANGE, and parse_literal() handles that correctly.
>
> QemuOpts doesn't do signed integers.
>
>> + *obj = ret;
>> +}
>> +
>> static void qobject_input_type_uint64(Visitor *v, const char *name,
>> uint64_t *obj, Error **errp)
>> {
>> @@ -279,6 +302,27 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
>> *obj = qint_get_int(qint);
>> }
>>
>> +static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
>> + uint64_t *obj, Error **errp)
>> +{
>> + QObjectInputVisitor *qiv = to_qiv(v);
>> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>> + true));
>> + unsigned long long ret;
>> +
>> + if (!qstr || !qstr->string) {
>> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>> + "string");
>> + return;
>> + }
>> +
>> + if (parse_uint_full(qstr->string, &ret, 0) < 0) {
>> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
"integer".
>> + return;
>> + }
>> + *obj = ret;
>
> Differently wrong :)
>
> To serve as replacement for the options visitor, this needs to parse
> exactly like opts_type_uint64().
>
> Again, this should also match the JSON parser and the QemuOpts parser as
> far as possible.
>
> opts_type_uint64() uses parse_uint(). It carefully checks for no
> conversion (EINVAL; parse_uint() normalizes), range, and string not
> fully consumed.
>
> You use parse_uint_full(). You therefore don't have bug#2 here. You do
> have bug#1: you assume unsigned long long is exactly 64 bits. If it's
> wider, we fail to diagnose overflow. If it's narrower, we can't parse
> all values.
>
> There's a discrepancy with the JSON parser, but it's not your fault: the
> JSON parser represents JSON numbers that fit into int64_t to QInt, and
> any others as QFloat. In particular, the integers between INT64_MAX+1
> and UINT64_MAX are represented as QFloat. qmp_input_type_uint64()
> accepts only QInt. You can declare things as uint64 in the schema, but
> you're still limited to 0..UINT64_MAX in QMP.
>
> QemuOpts uses strtoull() directly, in parse_option_number(). Bug (not
> yours): it happily ignores errors other than "string not fully
> consumed".
>
>> +}
>> +
>> static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
>> Error **errp)
>> {
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-10-12 14:51 ` Markus Armbruster
@ 2016-10-12 15:05 ` Markus Armbruster
2016-10-12 15:26 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 15:05 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
Markus Armbruster <armbru@redhat.com> writes:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>>> Currently the QObjectInputVisitor assumes that all scalar
>>> values are directly represented as the final types declared
>>> by the thing being visited. ie it assumes an 'int' is using
>>
>> i.e.
>>
>>> QInt, and a 'bool' is using QBool, etc. This is good when
>>> QObjectInputVisitor is fed a QObject that came from a JSON
>>> document on the QMP monitor, as it will strictly validate
>>> correctness.
>>>
>>> To allow QObjectInputVisitor to be reused for visiting
>>> a QObject originating from QemuOpts, an alternative mode
>>> is needed where all the scalars types are represented as
>>> QString and converted on the fly to the final desired
>>> type.
>>>
>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> [...]
>>> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
>>> index 5ff3db3..cf41df6 100644
>>> --- a/qapi/qobject-input-visitor.c
>>> +++ b/qapi/qobject-input-visitor.c
>>> @@ -20,6 +20,7 @@
>>> #include "qemu-common.h"
>>> #include "qapi/qmp/types.h"
>>> #include "qapi/qmp/qerror.h"
>>> +#include "qemu/cutils.h"
>>>
>>> #define QIV_STACK_SIZE 1024
>>>
>>> @@ -263,6 +264,28 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
>>> *obj = qint_get_int(qint);
>>> }
>>>
>>> +
>>> +static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
>>> + int64_t *obj, Error **errp)
>>> +{
>>> + QObjectInputVisitor *qiv = to_qiv(v);
>>> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>>> + true));
>>
>> Needs a rebase for commit 1382d4a. Same elsewhere.
>>
>>> + int64_t ret;
>>> +
>>> + if (!qstr || !qstr->string) {
>>
>> I don't think !qstr->string can happen. Same elsewhere.
>>
>>> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>>> + "string");
>>> + return;
>>> + }
>>
>> So far, this is basically qobject_input_type_str() less the g_strdup().
>> Worth factoring out?
>>
>> Now we're entering out parsing swamp:
>>
>>> +
>>> + if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
>>> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
>
> "integer", actually.
Wait! You're using QERR_INVALID_PARAMETER_VALUE here, so it's "an
integer".
To be pedantically correct, we'd have to complain about the type when
qemu_strtoll() fails to parse, and about the value when it parses okay,
but the value is out of range.
>>> + return;
>>> + }
>>
>> To serve as replacement for the options visitor, this needs to parse
>> exactly like opts_type_int64().
>>
>> It should also match the JSON parser as far as possible, to minimize
>> difference between the two QObject input visitor variants, and the
>> QemuOpts parser, for command line consistency.
>>
>> opts_type_int64() uses strtoll() directly. It carefully checks for no
>> conversion (both ways, EINVAL and endptr == str), range, and string not
>> fully consumed.
>>
>> Your code uses qemu_strtoll(). Bug#1: qemu_strtoll() assumes long long
>> is exactly 64 bits. If it's wider, we fail to diagnose overflow. If
>> it's narrower, we can't parse all values. Bug#2: your code fails to
>> check the string is fully consumed.
>>
>> The JSON parser also uses strtoll() directly, in parse_literal(). When
>> we get there, we know that the string consists only of decimal digits,
>> possibly prefixed by a minus sign. Therefore, strtoll() can only fail
>> with ERANGE, and parse_literal() handles that correctly.
>>
>> QemuOpts doesn't do signed integers.
>>
>>> + *obj = ret;
>>> +}
>>> +
>>> static void qobject_input_type_uint64(Visitor *v, const char *name,
>>> uint64_t *obj, Error **errp)
>>> {
>>> @@ -279,6 +302,27 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
>>> *obj = qint_get_int(qint);
>>> }
>>>
>>> +static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
>>> + uint64_t *obj, Error **errp)
>>> +{
>>> + QObjectInputVisitor *qiv = to_qiv(v);
>>> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>>> + true));
>>> + unsigned long long ret;
>>> +
>>> + if (!qstr || !qstr->string) {
>>> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>>> + "string");
>>> + return;
>>> + }
>>> +
>>> + if (parse_uint_full(qstr->string, &ret, 0) < 0) {
>>> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
>
> "integer".
Likewise.
>>> + return;
>>> + }
>>> + *obj = ret;
>>
>> Differently wrong :)
>>
>> To serve as replacement for the options visitor, this needs to parse
>> exactly like opts_type_uint64().
>>
>> Again, this should also match the JSON parser and the QemuOpts parser as
>> far as possible.
>>
>> opts_type_uint64() uses parse_uint(). It carefully checks for no
>> conversion (EINVAL; parse_uint() normalizes), range, and string not
>> fully consumed.
>>
>> You use parse_uint_full(). You therefore don't have bug#2 here. You do
>> have bug#1: you assume unsigned long long is exactly 64 bits. If it's
>> wider, we fail to diagnose overflow. If it's narrower, we can't parse
>> all values.
>>
>> There's a discrepancy with the JSON parser, but it's not your fault: the
>> JSON parser represents JSON numbers that fit into int64_t to QInt, and
>> any others as QFloat. In particular, the integers between INT64_MAX+1
>> and UINT64_MAX are represented as QFloat. qmp_input_type_uint64()
>> accepts only QInt. You can declare things as uint64 in the schema, but
>> you're still limited to 0..UINT64_MAX in QMP.
>>
>> QemuOpts uses strtoull() directly, in parse_option_number(). Bug (not
>> yours): it happily ignores errors other than "string not fully
>> consumed".
>>
>>> +}
>>> +
>>> static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
>>> Error **errp)
>>> {
> [...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor
2016-10-12 15:05 ` Markus Armbruster
@ 2016-10-12 15:26 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 15:26 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
Markus Armbruster <armbru@redhat.com> writes:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> Markus Armbruster <armbru@redhat.com> writes:
>>
>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>>
>>>> Currently the QObjectInputVisitor assumes that all scalar
>>>> values are directly represented as the final types declared
>>>> by the thing being visited. ie it assumes an 'int' is using
>>>
>>> i.e.
>>>
>>>> QInt, and a 'bool' is using QBool, etc. This is good when
>>>> QObjectInputVisitor is fed a QObject that came from a JSON
>>>> document on the QMP monitor, as it will strictly validate
>>>> correctness.
>>>>
>>>> To allow QObjectInputVisitor to be reused for visiting
>>>> a QObject originating from QemuOpts, an alternative mode
>>>> is needed where all the scalars types are represented as
>>>> QString and converted on the fly to the final desired
>>>> type.
>>>>
>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> [...]
>>>> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
>>>> index 5ff3db3..cf41df6 100644
>>>> --- a/qapi/qobject-input-visitor.c
>>>> +++ b/qapi/qobject-input-visitor.c
>>>> @@ -20,6 +20,7 @@
>>>> #include "qemu-common.h"
>>>> #include "qapi/qmp/types.h"
>>>> #include "qapi/qmp/qerror.h"
>>>> +#include "qemu/cutils.h"
>>>>
>>>> #define QIV_STACK_SIZE 1024
>>>>
>>>> @@ -263,6 +264,28 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
>>>> *obj = qint_get_int(qint);
>>>> }
>>>>
>>>> +
>>>> +static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
>>>> + int64_t *obj, Error **errp)
>>>> +{
>>>> + QObjectInputVisitor *qiv = to_qiv(v);
>>>> + QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>>>> + true));
>>>
>>> Needs a rebase for commit 1382d4a. Same elsewhere.
>>>
>>>> + int64_t ret;
>>>> +
>>>> + if (!qstr || !qstr->string) {
>>>
>>> I don't think !qstr->string can happen. Same elsewhere.
>>>
>>>> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>>>> + "string");
>>>> + return;
>>>> + }
>>>
>>> So far, this is basically qobject_input_type_str() less the g_strdup().
>>> Worth factoring out?
>>>
>>> Now we're entering out parsing swamp:
>>>
>>>> +
>>>> + if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
>>>> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
>>
>> "integer", actually.
>
> Wait! You're using QERR_INVALID_PARAMETER_VALUE here, so it's "an
> integer".
>
> To be pedantically correct, we'd have to complain about the type when
> qemu_strtoll() fails to parse, and about the value when it parses okay,
> but the value is out of range.
Nevermind, I got confused. The type is actually always string here, so
your use of QERR_INVALID_PARAMETER_VALUE is appropriate.
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor Daniel P. Berrange
2016-09-30 19:45 ` Eric Blake
@ 2016-10-12 15:50 ` Markus Armbruster
2016-10-12 16:03 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-20 14:28 ` [Qemu-devel] " Daniel P. Berrange
1 sibling, 2 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 15:50 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The traditional CLI arg syntax allows two ways to specify
> integer lists, either one value per key, or a range of
> values per key. eg the following are identical:
>
> -arg foo=5,foo=6,foo=7
> -arg foo=5-7
>
> This extends the QObjectInputVisitor so that it is able
> to parse ranges and turn them into distinct list entries.
>
> This means that
>
> -arg foo=5-7
>
> is treated as equivalent to
>
> -arg foo.0=5,foo.1=6,foo.2=7
>
> Edge case tests are copied from test-opts-visitor to
> ensure identical behaviour when parsing.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 23 ++++-
> qapi/qobject-input-visitor.c | 158 ++++++++++++++++++++++++++--
> tests/test-qobject-input-visitor.c | 195 +++++++++++++++++++++++++++++++++--
> 3 files changed, 360 insertions(+), 16 deletions(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index 94051f3..242b767 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -19,6 +19,12 @@
>
> typedef struct QObjectInputVisitor QObjectInputVisitor;
>
> +/* Inclusive upper bound on the size of any flattened range. This is a safety
> + * (= anti-annoyance) measure; wrong ranges should not cause long startup
> + * delays nor exhaust virtual memory.
> + */
> +#define QIV_RANGE_MAX 65536
> +
> /**
> * Create a new input visitor that converts @obj to a QAPI object.
> *
> @@ -71,6 +77,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> * The value given determines how many levels of structs are allowed to
> * be flattened in this way.
> *
> + * If @permit_int_ranges is true, then when visiting a list of integers,
> + * the integer value strings may encode ranges eg a single element
> + * containing "5-7" is treated as if there were three elements "5", "6",
> + * "7". This should only be used if compatibility is required with the
> + * OptsVisitor which would allow integer ranges. e.g. set this to true
> + * if you have compatibility requirements for
> + *
> + * -arg val=5-8
> + *
> + * to be treated as equivalent to the preferred syntax:
> + *
> + * -arg val.0=5,val.1=6,val.2=7,val.3=8
> + *
> * The visitor always operates in strict mode, requiring all dict keys
> * to be consumed during visitation. An error will be reported if this
> * does not happen.
> @@ -80,7 +99,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
> */
> Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> bool autocreate_list,
> - size_t autocreate_struct_levels);
> + size_t autocreate_struct_levels,
> + bool permit_int_ranges);
>
>
> /**
> @@ -98,6 +118,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> bool autocreate_list,
> size_t autocreate_struct_levels,
> + bool permit_int_ranges,
> Error **errp);
>
> #endif
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index 1be4865..6b3d0f2 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -31,6 +31,8 @@ typedef struct StackObject
>
> GHashTable *h; /* If obj is dict: unvisited keys */
> const QListEntry *entry; /* If obj is list: unvisited tail */
> + uint64_t range_val;
> + uint64_t range_limit;
It's either ugly unions or ugly type casts. The options visitor picked
ugly unions, you're picking ugly type casts. Matter of taste, as long
as the type casts are all kosher.
>
> QSLIST_ENTRY(StackObject) node;
> } StackObject;
> @@ -60,6 +62,10 @@ struct QObjectInputVisitor
> * consider auto-creating a struct containing
> * remaining unvisited items */
> size_t autocreate_struct_levels;
> +
> + /* Whether int lists can have single values representing
> + * ranges of values */
> + bool permit_int_ranges;
> };
>
> static QObjectInputVisitor *to_qiv(Visitor *v)
> @@ -282,7 +288,7 @@ static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
> QObjectInputVisitor *qiv = to_qiv(v);
> StackObject *so = QSLIST_FIRST(&qiv->stack);
>
> - if (!so->entry) {
> + if ((so->range_val == so->range_limit) && !so->entry) {
> return NULL;
> }
> tail->next = g_malloc0(size);
> @@ -329,21 +335,87 @@ static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
> int64_t *obj, Error **errp)
> {
> QObjectInputVisitor *qiv = to_qiv(v);
> - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> - true));
> + QString *qstr;
> int64_t ret;
> + const char *end = NULL;
> + StackObject *tos;
> + bool inlist = false;
> +
> + /* Preferentially generate values from a range, before
> + * trying to consume another QList element */
> + tos = QSLIST_FIRST(&qiv->stack);
> + if (tos) {
> + if ((int64_t)tos->range_val < (int64_t)tos->range_limit) {
> + *obj = tos->range_val + 1;
> + tos->range_val++;
Roundabout way to write
*obj = tos->range_val++;
> + return;
> + } else {
> + inlist = tos->entry != NULL;
> + }
> + }
I'd prefer
tos = QSLIST_FIRST(&qiv->stack);
inlist = tos && tos->entry;
if (inlist) {
if ((int64_t)tos->range_val < (int64_t)tos->range_limit) {
*obj = tos->range_val++;
return;
}
}
Note for later:
!tos || (int64_t)tos->range_val >= (int64_t)tos->range_limit
holds here.
>
> + qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> if (!qstr || !qstr->string) {
> error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> "string");
> return;
> }
>
> - if (qemu_strtoll(qstr->string, NULL, 0, &ret) < 0) {
> + if (qemu_strtoll(qstr->string, &end, 0, &ret) < 0) {
> error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> return;
> }
> *obj = ret;
> +
> + /*
> + * If we have string that represents an integer range (5-24),
> + * parse the end of the range and set things up so we'll process
> + * the rest of the range before consuming another element
> + * from the QList.
> + */
> + if (end && *end) {
> + if (!qiv->permit_int_ranges) {
> + error_setg(errp,
> + "Integer ranges are not permitted here");
> + return;
> + }
> + if (!inlist) {
> + error_setg(errp,
> + "Integer ranges are only permitted when "
> + "visiting list parameters");
> + return;
> + }
Note for later: since inlist is tos && tos->entry, we know tos is
non-null here. Recommend assert(tos) to make it more obvious.
> + if (*end != '-') {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
> + "a number range");
"an integer or integer range", actually.
> + return;
> + }
> + end++;
> + if (qemu_strtoll(end, NULL, 0, &ret) < 0) {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
"an integer".
> + return;
> + }
> + if (*obj > ret) {
> + error_setg(errp,
> + "Parameter '%s' range start %" PRIu64
> + " must be less than (or equal to) %" PRIu64,
> + name, *obj, ret);
> + return;
> + }
> +
> + if ((*obj <= (INT64_MAX - QIV_RANGE_MAX)) &&
> + (ret >= (*obj + QIV_RANGE_MAX))) {
> + error_setg(errp,
> + "Parameter '%s' range must be less than %d",
> + name, QIV_RANGE_MAX);
> + return;
> + }
I see you improved the error messages over the option visitor's. Good.
> + if (*obj != ret) {
> + tos->range_val = *obj;
> + tos->range_limit = ret;
> + }
Does this need to be conditional?
Dereferencing tos is safe (see note above).
> + }
No else, thus tos->range_val and tos->range_limit remain unchanged.
Works because we know that !tos || tos->range_val >= tos->range_limit
(see note above). If it wasn't, the next visit wouldn't consume another
object as it should. Considering the distance to the place that ensures
this, I'd prefer to spell this out here with an assertion, or perhaps by
assigning to tos->range_val and tos->range_limit.
> }
Overall, less hairy than the options visitor.
>
> static void qobject_input_type_uint64(Visitor *v, const char *name,
> @@ -366,21 +438,85 @@ static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
> uint64_t *obj, Error **errp)
> {
> QObjectInputVisitor *qiv = to_qiv(v);
> - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> - true));
> + QString *qstr;
> unsigned long long ret;
> + char *end = NULL;
> + StackObject *tos;
> + bool inlist = false;
> +
> + /* Preferentially generate values from a range, before
> + * trying to consume another QList element */
> + tos = QSLIST_FIRST(&qiv->stack);
> + if (tos) {
> + if (tos->range_val < tos->range_limit) {
> + *obj = tos->range_val + 1;
> + tos->range_val++;
> + return;
> + } else {
> + inlist = tos->entry != NULL;
> + }
> + }
>
> + qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> + true));
> if (!qstr || !qstr->string) {
> error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> "string");
> return;
> }
>
> - if (parse_uint_full(qstr->string, &ret, 0) < 0) {
> + if (parse_uint(qstr->string, &ret, &end, 0) < 0) {
> error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> return;
> }
> *obj = ret;
> +
> + /*
> + * If we have string that represents an integer range (5-24),
> + * parse the end of the range and set things up so we'll process
> + * the rest of the range before consuming another element
> + * from the QList.
> + */
> + if (end && *end) {
> + if (!qiv->permit_int_ranges) {
> + error_setg(errp,
> + "Integer ranges are not permitted here");
> + return;
> + }
> + if (!inlist) {
> + error_setg(errp,
> + "Integer ranges are only permitted when "
> + "visiting list parameters");
> + return;
> + }
> + if (*end != '-') {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
> + "a number range");
> + return;
> + }
> + end++;
> + if (parse_uint_full(end, &ret, 0) < 0) {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> + return;
> + }
> + if (*obj > ret) {
> + error_setg(errp,
> + "Parameter '%s' range start %" PRIu64
> + " must be less than (or equal to) %llu",
> + name, *obj, ret);
> + return;
> + }
> + if ((ret - *obj) > (QIV_RANGE_MAX - 1)) {
> + error_setg(errp,
> + "Parameter '%s' range must be less than %d",
> + name, QIV_RANGE_MAX);
> + return;
> + }
> + if (*obj != ret) {
> + tos->range_val = *obj;
> + tos->range_limit = ret;
> + }
> + }
> }
Duplicates the signed code, which is sad, but I don't have better ideas.
Except this one: are we actually using both the signed and the unsigned
case now? If not, can we get rid of the one we don't use?
>
> static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
> @@ -576,7 +712,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
>
> Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> bool autocreate_list,
> - size_t autocreate_struct_levels)
> + size_t autocreate_struct_levels,
> + bool permit_int_ranges)
> {
> QObjectInputVisitor *v;
>
> @@ -603,6 +740,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> v->strict = true;
> v->autocreate_list = autocreate_list;
> v->autocreate_struct_levels = autocreate_struct_levels;
> + v->permit_int_ranges = permit_int_ranges;
>
> v->root = obj;
> qobject_incref(obj);
> @@ -614,6 +752,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> bool autocreate_list,
> size_t autocreate_struct_levels,
> + bool permit_int_ranges,
> Error **errp)
> {
> QDict *pdict;
> @@ -632,7 +771,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
>
> v = qobject_input_visitor_new_autocast(pobj,
> autocreate_list,
> - autocreate_struct_levels);
> + autocreate_struct_levels,
> + permit_int_ranges);
> cleanup:
> qobject_decref(pobj);
> QDECREF(pdict);
[Skipping test updates for now...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-10-12 15:50 ` Markus Armbruster
@ 2016-10-12 16:03 ` Kevin Wolf
2016-10-12 18:14 ` Markus Armbruster
2016-10-20 14:28 ` [Qemu-devel] " Daniel P. Berrange
1 sibling, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-12 16:03 UTC (permalink / raw)
To: Markus Armbruster
Cc: Daniel P. Berrange, Paolo Bonzini, Andreas Färber,
qemu-devel, qemu-block, Max Reitz
Am 12.10.2016 um 17:50 hat Markus Armbruster geschrieben:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The traditional CLI arg syntax allows two ways to specify
> > integer lists, either one value per key, or a range of
> > values per key. eg the following are identical:
> >
> > -arg foo=5,foo=6,foo=7
> > -arg foo=5-7
> >
> > This extends the QObjectInputVisitor so that it is able
> > to parse ranges and turn them into distinct list entries.
> >
> > This means that
> >
> > -arg foo=5-7
> >
> > is treated as equivalent to
> >
> > -arg foo.0=5,foo.1=6,foo.2=7
> >
> > Edge case tests are copied from test-opts-visitor to
> > ensure identical behaviour when parsing.
> >
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > @@ -329,21 +335,87 @@ static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
> > int64_t *obj, Error **errp)
> > {
> > QObjectInputVisitor *qiv = to_qiv(v);
> > - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> > - true));
> > + QString *qstr;
> > int64_t ret;
> > + const char *end = NULL;
> > + StackObject *tos;
> > + bool inlist = false;
> > +
> > + /* Preferentially generate values from a range, before
> > + * trying to consume another QList element */
> > + tos = QSLIST_FIRST(&qiv->stack);
> > + if (tos) {
> > + if ((int64_t)tos->range_val < (int64_t)tos->range_limit) {
> > + *obj = tos->range_val + 1;
> > + tos->range_val++;
>
> Roundabout way to write
>
> *obj = tos->range_val++;
*obj = ++tos->range_val, actually.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
2016-09-30 20:08 ` Eric Blake
@ 2016-10-12 17:46 ` Markus Armbruster
2016-10-13 9:21 ` Markus Armbruster
2016-10-20 14:29 ` Daniel P. Berrange
2016-10-13 8:31 ` Markus Armbruster
2 siblings, 2 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 17:46 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> If given an option string such as
>
> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>
> the qemu_opts_to_qdict() method will currently overwrite
> the values for repeated option keys, so only the last
> value is in the returned dict:
>
> size=QString("1024")
> nodes=QString("1-2")
> policy=QString("bind")
>
> With this change the caller can optionally ask for all
> the repeated values to be stored in a QList. In the
> above example that would result in 'nodes' being a
> QList, so the returned dict would contain
>
> size=QString("1024")
> nodes=QList([QString("10"),
> QString("4-5"),
> QString("1-2")])
> policy=QString("bind")
>
> Note that the conversion has no way of knowing whether
> any given key is expected to be a list upfront - it can
> only figure that out when seeing the first duplicated
> key. Thus the caller has to be prepared to deal with the
> fact that if a key 'foo' is a list, then the returned
> qdict may contain either a QString or a QList for the
> key 'foo'.
>
> In a third mode, it is possible to ask for repeated
> options to be reported as an error, rather than silently
> dropping all but the last one.
To serve as a replacement for the options visitor, this needs to be able
to behave exactly the same together with a suitably hacked up QObject
input visitor. Before I dive into the actual patch, let me summarize
QemuOpts and options visitor behavior.
Warning, this is going to get ugly.
QemuOpts faithfully represents a key=value,... string as a list of
QemuOpt. Each QemuOpt represents one key=value. They are in the same
order. If key occurs multiple times in the string, it occurs just the
same in the list.
*Except* key "id" is special: it's stored outside the list, and all but
the first one are silently ignored.
Most users only ever get the last value of a key. Any non-last
key=value are silently ignored.
We actually exploit this behavior to do defaults, by *prepending* them
to the list. See the use of qemu_opts_set_defaults() in main().
A few users get all values of keys (other than key "id"):
* -device, in qdev_device_add() with callback set_property().
We first get "driver" and "bus" normally (silently ignoring non-last
values, as usual). All other keys are device properties. To set
them, we get all (key, value), ignore keys "driver" and "bus", and set
the rest. If a key occurs multiple times, it gets set multiple times.
This effectively ignores all but the last one, silently.
* -semihosting-config, in main() with callback add_semihosting_arg().
We first get a bunch of keys normally. Key "arg" is special: it may
be repeated to build a list. To implement that, we get all (key,
value), ignore keys other than "arg", and accumulate the values.
* -machine & friends, in main() with callback machine_set_property()
Similar to -device, only for machines, with "type" instead of "driver"
and "bus".
* -spice, in qemu_spice_init() with callback add_channel()
Keys "tls-channel" and "plaintext-channel" may be used repeated to
specify multiple channels. To implement that, we get all (key,
value), ignore keys other than "tls-channel" and "plaintext-channel",
and set up a channel for each of the others.
* -writeconfig, in config_write_opts() with callback config_write_opt()
We write out all keys in order.
* The options visitor, in opts_start_struct()
We convert the list of (key, value) to a hash table of (key, list of
values). Most of the time, the list of values has exactly one
element.
When the visitor's user asks for a scalar, we return the last element
of the list of values, in lookup_scalar().
When the user asks for list elements, we return the elements of the
list of values in order, in opts_next_list(), or if there are none,
the empty list in opts_start_list().
Unlike the options visitor, this patch (judging from your description)
makes a list only when keys are repeated. The QObject visitor will have
to cope with finding both scalars and lists. When it finds a scalar,
but needs a list, it'll have to wrap it in a list (PATCH 09, I think).
When it finds a list, but needs a scalar, it'll have to fish it out of
the list (where is that?).
> All existing callers are all converted to explicitly
> request the historical behaviour of only reporting the
> last key. Later patches will make use of the new modes.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Out of steam for today.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-10-12 16:03 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-12 18:14 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-12 18:14 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Max Reitz, Paolo Bonzini, Andreas Färber
Kevin Wolf <kwolf@redhat.com> writes:
> Am 12.10.2016 um 17:50 hat Markus Armbruster geschrieben:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > The traditional CLI arg syntax allows two ways to specify
>> > integer lists, either one value per key, or a range of
>> > values per key. eg the following are identical:
>> >
>> > -arg foo=5,foo=6,foo=7
>> > -arg foo=5-7
>> >
>> > This extends the QObjectInputVisitor so that it is able
>> > to parse ranges and turn them into distinct list entries.
>> >
>> > This means that
>> >
>> > -arg foo=5-7
>> >
>> > is treated as equivalent to
>> >
>> > -arg foo.0=5,foo.1=6,foo.2=7
>> >
>> > Edge case tests are copied from test-opts-visitor to
>> > ensure identical behaviour when parsing.
>> >
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>
>> > @@ -329,21 +335,87 @@ static void qobject_input_type_int64_autocast(Visitor *v, const char *name,
>> > int64_t *obj, Error **errp)
>> > {
>> > QObjectInputVisitor *qiv = to_qiv(v);
>> > - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>> > - true));
>> > + QString *qstr;
>> > int64_t ret;
>> > + const char *end = NULL;
>> > + StackObject *tos;
>> > + bool inlist = false;
>> > +
>> > + /* Preferentially generate values from a range, before
>> > + * trying to consume another QList element */
>> > + tos = QSLIST_FIRST(&qiv->stack);
>> > + if (tos) {
>> > + if ((int64_t)tos->range_val < (int64_t)tos->range_limit) {
>> > + *obj = tos->range_val + 1;
>> > + tos->range_val++;
>>
>> Roundabout way to write
>>
>> *obj = tos->range_val++;
>
> *obj = ++tos->range_val, actually.
Of course, thanks.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts
2016-10-12 8:08 ` [Qemu-devel] " Markus Armbruster
@ 2016-10-13 7:23 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-13 7:23 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
Markus Armbruster <armbru@redhat.com> writes:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> Instead of requiring all callers to go through the mutli-step
>
> multi-step
>
>> process of turning QemuOpts into a suitable QObject for visiting,
>> add a new constructor that encapsulates this logic. This will
>> allow QObjectInputVisitor to be a drop-in replacement for the
>> existing OptsVisitor with minimal code changes for callers.
>>
>> NB, at this point it is only supporting opts syntax which
>> explicitly matches the QAPI schema structure, so is not yet
>> a true drop-in replacement for OptsVisitor. The patches that
>> follow will add the special cases requird for full backwards
>> compatibility with OptsVisitor.
>>
>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
[...]
>> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
>> index cf41df6..d9269c9 100644
>> --- a/qapi/qobject-input-visitor.c
>> +++ b/qapi/qobject-input-visitor.c
>> @@ -545,3 +545,28 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>>
>> return &v->visitor;
>> }
>> +
>> +
>> +Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
>> + Error **errp)
>> +{
>> + QDict *pdict;
>> + QObject *pobj = NULL;
>
> @pdict and @pobj are unusual names. Let's stick to the more common
> @dict and @obj.
>
>> + Visitor *v = NULL;
>> +
>> + pdict = qemu_opts_to_qdict(opts, NULL);
>> + if (!pdict) {
>> + goto cleanup;
Returns null without setting an error, which is wrong. Fortunately,
qemu_opts_to_qdict() cannot fail. Please drop the broken error
handling.
>> + }
>> +
>> + pobj = qdict_crumple(pdict, true, errp);
>> + if (!pobj) {
>> + goto cleanup;
>> + }
>> +
>> + v = qobject_input_visitor_new_autocast(pobj);
>> + cleanup:
>> + qobject_decref(pobj);
>> + QDECREF(pdict);
>> + return v;
>> +}
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
2016-09-30 20:08 ` Eric Blake
2016-10-12 17:46 ` Markus Armbruster
@ 2016-10-13 8:31 ` Markus Armbruster
2016-10-20 14:37 ` Daniel P. Berrange
2 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-13 8:31 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> If given an option string such as
>
> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>
> the qemu_opts_to_qdict() method will currently overwrite
> the values for repeated option keys, so only the last
> value is in the returned dict:
>
> size=QString("1024")
> nodes=QString("1-2")
> policy=QString("bind")
>
> With this change the caller can optionally ask for all
> the repeated values to be stored in a QList. In the
> above example that would result in 'nodes' being a
> QList, so the returned dict would contain
>
> size=QString("1024")
> nodes=QList([QString("10"),
> QString("4-5"),
> QString("1-2")])
> policy=QString("bind")
>
> Note that the conversion has no way of knowing whether
> any given key is expected to be a list upfront - it can
> only figure that out when seeing the first duplicated
> key. Thus the caller has to be prepared to deal with the
> fact that if a key 'foo' is a list, then the returned
> qdict may contain either a QString or a QList for the
> key 'foo'.
Actually three cases, not two:
0. qdict does not contain the key means empty list.
1. qdict contains the key with a QString value means list of one
element.
2. qdict contains the key with a QList value means list of more than one
element.
I consider this weird. However, it's usefully weird with at least your
QObject input visitor.
> In a third mode, it is possible to ask for repeated
> options to be reported as an error, rather than silently
> dropping all but the last one.
Got users for this policy in the pipeline?
> All existing callers are all converted to explicitly
> request the historical behaviour of only reporting the
> last key. Later patches will make use of the new modes.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> blockdev.c | 7 ++-
> include/qemu/option.h | 13 ++++-
> monitor.c | 3 +-
> qapi/qobject-input-visitor.c | 4 +-
> qemu-img.c | 4 +-
> qemu-io-cmds.c | 3 +-
> qemu-io.c | 6 +-
> qemu-nbd.c | 3 +-
> qom/object_interfaces.c | 3 +-
> tests/test-qemu-opts.c | 132 +++++++++++++++++++++++++++++++++++++++++++
> tests/test-replication.c | 9 ++-
> util/qemu-option.c | 64 ++++++++++++++++++---
> 12 files changed, 229 insertions(+), 22 deletions(-)
>
> diff --git a/blockdev.c b/blockdev.c
> index 814d49f..a999d46 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -914,7 +914,8 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
>
> /* Get a QDict for processing the options */
> bs_opts = qdict_new();
> - qemu_opts_to_qdict(all_opts, bs_opts);
> + qemu_opts_to_qdict(all_opts, bs_opts, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
>
> legacy_opts = qemu_opts_create(&qemu_legacy_drive_opts, NULL, 0,
> &error_abort);
> @@ -3804,8 +3805,8 @@ void hmp_drive_add_node(Monitor *mon, const char *optstr)
> return;
> }
>
> - qdict = qemu_opts_to_qdict(opts, NULL);
> -
> + qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> if (!qdict_get_try_str(qdict, "node-name")) {
> QDECREF(qdict);
> error_report("'node-name' needs to be specified");
> diff --git a/include/qemu/option.h b/include/qemu/option.h
> index 29f3f18..ffd841d 100644
> --- a/include/qemu/option.h
> +++ b/include/qemu/option.h
> @@ -125,7 +125,18 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
> int permit_abbrev);
> QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
> Error **errp);
> -QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict);
Blank line here, please.
> +typedef enum {
> + /* Last occurrence of a duplicate option silently replaces earlier */
> + QEMU_OPTS_REPEAT_POLICY_LAST,
> + /* Each occurrence of a duplicate option converts value to a QList */
> + QEMU_OPTS_REPEAT_POLICY_ALL,
> + /* First occurrence of a duplicate option causes an error */
> + QEMU_OPTS_REPEAT_POLICY_ERROR,
> +} QemuOptsRepeatPolicy;
> +
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
> + QemuOptsRepeatPolicy repeatPolicy,
> + Error **errp);
> void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
>
> typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
> diff --git a/monitor.c b/monitor.c
> index 14089cc..84e79d4 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -2642,7 +2642,8 @@ static QDict *monitor_parse_arguments(Monitor *mon,
> if (!opts) {
> goto fail;
> }
> - qemu_opts_to_qdict(opts, qdict);
> + qemu_opts_to_qdict(opts, qdict, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> qemu_opts_del(opts);
> }
> break;
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index 6b3d0f2..2287d11 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -759,7 +759,9 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> QObject *pobj = NULL;
> Visitor *v = NULL;
>
> - pdict = qemu_opts_to_qdict(opts, NULL);
> + pdict = qemu_opts_to_qdict(opts, NULL,
> + QEMU_OPTS_REPEAT_POLICY_LAST,
Join the previous two lines, please.
> + errp);
Unlike the other callers, this one doesn't pass &error_abort. The error
handling is actually dead code. Let's pass &error_abort and be done
with it.
> if (!pdict) {
> goto cleanup;
> }
> diff --git a/qemu-img.c b/qemu-img.c
> index 851422a..f47ea75 100644
> --- a/qemu-img.c
> +++ b/qemu-img.c
> @@ -273,7 +273,9 @@ static BlockBackend *img_open_opts(const char *optstr,
> QDict *options;
> Error *local_err = NULL;
> BlockBackend *blk;
> - options = qemu_opts_to_qdict(opts, NULL);
> + options = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> +
> blk = blk_new_open(NULL, NULL, options, flags, &local_err);
> if (!blk) {
> error_reportf_err(local_err, "Could not open '%s': ", optstr);
> diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
> index 3a3838a..ce654f4 100644
> --- a/qemu-io-cmds.c
> +++ b/qemu-io-cmds.c
> @@ -1952,7 +1952,8 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv)
> }
>
> qopts = qemu_opts_find(&reopen_opts, NULL);
> - opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
> + opts = qopts ? qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort) : NULL;
Short ternaries are more legible than if statements, but this one is
pushing the limit now.
> qemu_opts_reset(&reopen_opts);
>
> brq = bdrv_reopen_queue(NULL, bs, opts, flags);
> diff --git a/qemu-io.c b/qemu-io.c
> index db129ea..bb374a6 100644
> --- a/qemu-io.c
> +++ b/qemu-io.c
> @@ -207,7 +207,8 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
> }
>
> qopts = qemu_opts_find(&empty_opts, NULL);
> - opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
> + opts = qopts ? qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort) : NULL;
Likewise.
> qemu_opts_reset(&empty_opts);
>
> if (optind == argc - 1) {
> @@ -593,7 +594,8 @@ int main(int argc, char **argv)
> if (!qopts) {
> exit(1);
> }
> - opts = qemu_opts_to_qdict(qopts, NULL);
> + opts = qemu_opts_to_qdict(qopts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
Even this line's getting ugly. Can we find a prefix that's less
loquacious than QEMU_OPTS_REPEAT_POLICY_, but still self-explaining?
> openfile(NULL, flags, writethrough, opts);
> } else {
> if (format) {
> diff --git a/qemu-nbd.c b/qemu-nbd.c
> index 99297a5..73b40b1 100644
> --- a/qemu-nbd.c
> +++ b/qemu-nbd.c
> @@ -859,7 +859,8 @@ int main(int argc, char **argv)
> qemu_opts_reset(&file_opts);
> exit(EXIT_FAILURE);
> }
> - options = qemu_opts_to_qdict(opts, NULL);
> + options = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> qemu_opts_reset(&file_opts);
> blk = blk_new_open(NULL, NULL, options, flags, &local_err);
> } else {
> diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
> index ded4d84..fdc406b 100644
> --- a/qom/object_interfaces.c
> +++ b/qom/object_interfaces.c
> @@ -161,7 +161,8 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
> Object *obj = NULL;
>
> v = opts_visitor_new(opts);
> - pdict = qemu_opts_to_qdict(opts, NULL);
> + pdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
>
> obj = user_creatable_add(pdict, v, errp);
> visit_free(v);
> diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
> index a505a3e..3b59978 100644
> --- a/tests/test-qemu-opts.c
> +++ b/tests/test-qemu-opts.c
> @@ -421,6 +421,130 @@ static void test_qemu_opts_set(void)
> g_assert(opts == NULL);
> }
>
> +
> +static void test_qemu_opts_to_qdict_repeat_last(void)
> +{
> + QemuOpts *opts;
> + QDict *dict;
> +
> + /* dynamically initialized (parsed) opts */
What are you trying to say with this comment? Hmm, you merely copied it
from the existing tests. I'd drop the existing instances of this
comment instead.
> + opts = qemu_opts_parse(&opts_list_03,
> + "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
> + false, NULL);
> + g_assert(opts);
You could pass &error_abort.
> +
> + dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> + g_assert(dict);
> +
> +
One blank line, not two, please.
> + g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
> + g_assert_cmpstr(qdict_get_str(dict, "nodes"), ==, "1-2");
> + g_assert(!qdict_haskey(dict, "nodes.0"));
> + g_assert(!qdict_haskey(dict, "nodes.1"));
> + g_assert(!qdict_haskey(dict, "nodes.2"));
> + g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
Checking for absence of "nodes.0"... feels odd. A better idea is to
check we got exactly the keys we expect, and no more. Here's a simple
way to do that:
g_assert(qdict_size(dict) == 3);
> + QDECREF(dict);
> +
> + qemu_opts_del(opts);
> + qemu_opts_reset(&opts_list_03);
> +}
> +
> +
> +static void test_qemu_opts_to_qdict_repeat_all(void)
> +{
> + QemuOpts *opts;
> + QDict *dict;
> + QList *list;
> + const QListEntry *ent;
> + QString *str;
> +
> + /* dynamically initialized (parsed) opts */
> + opts = qemu_opts_parse(&opts_list_03,
> + "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
> + false, NULL);
> + g_assert(opts);
> +
> + dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ALL,
> + &error_abort);
> + g_assert(dict);
> +
> + g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
> + g_assert(qdict_haskey(dict, "nodes"));
> + list = qobject_to_qlist(qdict_get(dict, "nodes"));
> + g_assert(list);
Works, but I'd do
obj = qdict_get(dict, "nodes");
g_assert(obj);
list = qobject_to_qlist(obj);
g_assert(list);
> +
> + ent = qlist_first(list);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "10");
Since g_assert_cmpstr() does the right thing when @str is null,
g_assert(str) is redundant. More of the same below.
> +
> + ent = qlist_next(ent);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "4-5");
> +
> + ent = qlist_next(ent);
> + g_assert(ent);
> + str = qobject_to_qstring(ent->value);
> + g_assert(str);
> + g_assert_cmpstr(qstring_get_str(str), ==, "1-2");
> +
> + ent = qlist_next(ent);
> + g_assert(!ent);
> +
> + g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
g_assert(qdict_size(dict) == 3);
> + QDECREF(dict);
> +
> + qemu_opts_del(opts);
> + qemu_opts_reset(&opts_list_03);
> +}
> +
> +static void test_qemu_opts_to_qdict_repeat_err_fail(void)
> +{
> + QemuOpts *opts;
> + QDict *dict;
> + Error *err = NULL;
> +
> + /* dynamically initialized (parsed) opts */
> + opts = qemu_opts_parse(&opts_list_03,
> + "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
> + false, NULL);
> + g_assert(opts);
> +
> + dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ERROR,
> + &err);
> + error_free_or_abort(&err);
> + g_assert(!dict);
> +
> + qemu_opts_del(opts);
> + qemu_opts_reset(&opts_list_03);
> +}
> +
> +static void test_qemu_opts_to_qdict_repeat_err_ok(void)
> +{
> + QemuOpts *opts;
> + QDict *dict;
> +
> + /* dynamically initialized (parsed) opts */
> + opts = qemu_opts_parse(&opts_list_03,
> + "size=1024,nodes=10,policy=bind",
> + false, NULL);
> + g_assert(opts);
> +
> + dict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_ERROR,
> + &error_abort);
> + g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
> + g_assert_cmpstr(qdict_get_str(dict, "nodes"), ==, "10");
> + g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
g_assert(qdict_size(dict) == 3);
> +
> + QDECREF(dict);
> + qemu_opts_del(opts);
> + qemu_opts_reset(&opts_list_03);
> +}
> +
> int main(int argc, char *argv[])
> {
> register_opts();
> @@ -435,6 +559,14 @@ int main(int argc, char *argv[])
> g_test_add_func("/qemu-opts/opt_unset", test_qemu_opt_unset);
> g_test_add_func("/qemu-opts/opts_reset", test_qemu_opts_reset);
> g_test_add_func("/qemu-opts/opts_set", test_qemu_opts_set);
> + g_test_add_func("/qemu-opts/to_qdict/repeat-last",
> + test_qemu_opts_to_qdict_repeat_last);
> + g_test_add_func("/qemu-opts/to_qdict/repeat-all",
> + test_qemu_opts_to_qdict_repeat_all);
> + g_test_add_func("/qemu-opts/to_qdict/repeat-err-fail",
> + test_qemu_opts_to_qdict_repeat_err_fail);
> + g_test_add_func("/qemu-opts/to_qdict/repeat-err-ok",
> + test_qemu_opts_to_qdict_repeat_err_ok);
> g_test_run();
> return 0;
> }
> diff --git a/tests/test-replication.c b/tests/test-replication.c
> index 0997bd8..e267f9a 100644
> --- a/tests/test-replication.c
> +++ b/tests/test-replication.c
> @@ -181,7 +181,8 @@ static BlockBackend *start_primary(void)
> opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
> g_free(cmdline);
>
> - qdict = qemu_opts_to_qdict(opts, NULL);
> + qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
>
> @@ -311,7 +312,8 @@ static BlockBackend *start_secondary(void)
> opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
> g_free(cmdline);
>
> - qdict = qemu_opts_to_qdict(opts, NULL);
> + qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
>
> @@ -336,7 +338,8 @@ static BlockBackend *start_secondary(void)
> opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
> g_free(cmdline);
>
> - qdict = qemu_opts_to_qdict(opts, NULL);
> + qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + &error_abort);
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
> qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
>
> diff --git a/util/qemu-option.c b/util/qemu-option.c
> index 418cbb6..0129ede 100644
> --- a/util/qemu-option.c
> +++ b/util/qemu-option.c
> @@ -1057,23 +1057,73 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
> * The QDict values are of type QString.
> * TODO We'll want to use types appropriate for opt->desc->type, but
> * this is enough for now.
While there, we could document @obj and @dict properly. But see below
on @dict.
> + *
> + * If the @opts contains multiple occurrences of the same key,
> + * then the @repeatPolicy parameter determines how they are to
> + * be handled. Traditional behaviour was to only store the
> + * last occurrence, but if @repeatPolicy is set to
> + * QEMU_OPTS_REPEAT_POLICY_ALL, then a QList will be returned
> + * containing all values, for any key with multiple occurrences.
> + * The QEMU_OPTS_REPEAT_POLICY_ERROR value can be used to fail
> + * conversion with an error if multiple occurrences of a key
> + * are seen.
What the traditional behavior was is quite interesting in a commit
message. Less so in a comment. Suggest:
* be handled. With @repeatPolicy QEMU_OPTS_REPEAT_POLICY_LAST,
* all values but the last are silently ignored. With
* QEMU_OPTS_REPEAT_POLICY_ALL, a QList will be returned
* containing all values, for any key with multiple occurrences.
* With QEMU_OPTS_REPEAT_POLICY_ERROR, the conversion fails.
Let's spell out that the function can only fail with
QEMU_OPTS_REPEAT_POLICY_ERROR:
* This is the only way this function can fail.
> */
> -QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict)
> +QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
> + QemuOptsRepeatPolicy repeatPolicy,
> + Error **errp)
> {
> QemuOpt *opt;
> - QObject *val;
> + QObject *val, *prevval;
I'd prefer prev_val.
> + QList *list;
> + QDict *ret;
>
> - if (!qdict) {
> - qdict = qdict_new();
> + if (qdict) {
> + ret = qdict;
> + } else {
> + ret = qdict_new();
> }
You need this change to get the reference counting on failure right.
This makes me question the qdict parameter. I can see two non-null
arguments for it outside tests. The one in drive_new() is basically
stupid:
bs_opts = qdict_new();
qemu_opts_to_qdict(all_opts, bs_opts, QEMU_OPTS_REPEAT_POLICY_LAST,
&error_abort);
Easily replaced by
bs_opts = qemu_opts_to_qdict(all_opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
&error_abort);
The one in monitor_parse_arguments()
qemu_opts_to_qdict(opts, qdict, QEMU_OPTS_REPEAT_POLICY_LAST,
&error_abort);
could be replaced by
QDECREF(qdict);
qdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
&error_abort);
at a negligible loss of efficiency. We could then drop the parameter,
simplifying the function's contract.
Let's insert a blank line here...
> if (opts->id) {
> - qdict_put(qdict, "id", qstring_from_str(opts->id));
> + qdict_put(ret, "id", qstring_from_str(opts->id));
> }
... and here.
> QTAILQ_FOREACH(opt, &opts->head, next) {
> val = QOBJECT(qstring_from_str(opt->str));
> - qdict_put_obj(qdict, opt->name, val);
> +
> + if (qdict_haskey(ret, opt->name)) {
> + switch (repeatPolicy) {
> + case QEMU_OPTS_REPEAT_POLICY_ERROR:
> + error_setg(errp, "Option '%s' appears more than once",
> + opt->name);
> + qobject_decref(val);
> + if (!qdict) {
> + QDECREF(ret);
> + }
> + return NULL;
> +
> + case QEMU_OPTS_REPEAT_POLICY_ALL:
> + prevval = qdict_get(ret, opt->name);
> + if (qobject_type(prevval) == QTYPE_QLIST) {
> + /* Already identified this key as a list */
> + list = qobject_to_qlist(prevval);
> + } else {
> + /* Replace current scalar with a list */
> + list = qlist_new();
> + qobject_incref(prevval);
Where is this reference decremented?
> + qlist_append_obj(list, prevval);
> + qdict_put_obj(ret, opt->name, QOBJECT(list));
> + }
> + qlist_append_obj(list, val);
> + break;
> +
> + case QEMU_OPTS_REPEAT_POLICY_LAST:
> + /* Just discard previously set value */
> + qdict_put_obj(ret, opt->name, val);
> + break;
> + }
> + } else {
> + qdict_put_obj(ret, opt->name, val);
> + }
A possible alternative:
val = QOBJECT(qstring_from_str(opt->str));
if (qdict_haskey(ret, opt->name)) {
switch (repeatPolicy) {
case QEMU_OPTS_REPEAT_POLICY_ERROR:
error_setg(errp, "Option '%s' appears more than once",
opt->name);
qobject_decref(val);
if (!qdict) {
QDECREF(ret);
}
return NULL;
case QEMU_OPTS_REPEAT_POLICY_ALL:
prevval = qdict_get(ret, opt->name);
if (qobject_type(prevval) == QTYPE_QLIST) {
/* Already identified this key as a list */
list = qobject_to_qlist(prevval);
} else {
/* Replace current scalar with a list */
list = qlist_new();
qobject_incref(prevval);
qlist_append_obj(list, prevval);
}
qlist_append_obj(list, val);
val = QOBJECT(list);
break;
case QEMU_OPTS_REPEAT_POLICY_LAST:
break;
}
}
qdict_put_obj(ret, opt->name, val);
This shows the common part of the behavior more clearly. Matter of
taste, you get to use your artistic license.
> }
> - return qdict;
> + return ret;
> }
>
> /* Validate parsed opts against descriptions where no
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-12 17:46 ` Markus Armbruster
@ 2016-10-13 9:21 ` Markus Armbruster
2016-10-20 14:29 ` Daniel P. Berrange
1 sibling, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-13 9:21 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
Markus Armbruster <armbru@redhat.com> writes:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> If given an option string such as
>>
>> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>>
>> the qemu_opts_to_qdict() method will currently overwrite
>> the values for repeated option keys, so only the last
>> value is in the returned dict:
>>
>> size=QString("1024")
>> nodes=QString("1-2")
>> policy=QString("bind")
>>
>> With this change the caller can optionally ask for all
>> the repeated values to be stored in a QList. In the
>> above example that would result in 'nodes' being a
>> QList, so the returned dict would contain
>>
>> size=QString("1024")
>> nodes=QList([QString("10"),
>> QString("4-5"),
>> QString("1-2")])
>> policy=QString("bind")
>>
>> Note that the conversion has no way of knowing whether
>> any given key is expected to be a list upfront - it can
>> only figure that out when seeing the first duplicated
>> key. Thus the caller has to be prepared to deal with the
>> fact that if a key 'foo' is a list, then the returned
>> qdict may contain either a QString or a QList for the
>> key 'foo'.
>>
>> In a third mode, it is possible to ask for repeated
>> options to be reported as an error, rather than silently
>> dropping all but the last one.
>
> To serve as a replacement for the options visitor, this needs to be able
> to behave exactly the same together with a suitably hacked up QObject
> input visitor. Before I dive into the actual patch, let me summarize
> QemuOpts and options visitor behavior.
>
> Warning, this is going to get ugly.
>
> QemuOpts faithfully represents a key=value,... string as a list of
> QemuOpt. Each QemuOpt represents one key=value. They are in the same
> order. If key occurs multiple times in the string, it occurs just the
> same in the list.
>
> *Except* key "id" is special: it's stored outside the list, and all but
> the first one are silently ignored.
>
> Most users only ever get the last value of a key. Any non-last
> key=value are silently ignored.
>
> We actually exploit this behavior to do defaults, by *prepending* them
> to the list. See the use of qemu_opts_set_defaults() in main().
This prepending of defaults assumes all users ignore values other than
the last. It breaks if any user gets non-last values.
> A few users get all values of keys (other than key "id"):
>
> * -device, in qdev_device_add() with callback set_property().
>
> We first get "driver" and "bus" normally (silently ignoring non-last
> values, as usual). All other keys are device properties. To set
> them, we get all (key, value), ignore keys "driver" and "bus", and set
> the rest. If a key occurs multiple times, it gets set multiple times.
> This effectively ignores all but the last one, silently.
>
> * -semihosting-config, in main() with callback add_semihosting_arg().
>
> We first get a bunch of keys normally. Key "arg" is special: it may
> be repeated to build a list. To implement that, we get all (key,
> value), ignore keys other than "arg", and accumulate the values.
>
> * -machine & friends, in main() with callback machine_set_property()
>
> Similar to -device, only for machines, with "type" instead of "driver"
> and "bus".
>
> * -spice, in qemu_spice_init() with callback add_channel()
>
> Keys "tls-channel" and "plaintext-channel" may be used repeated to
> specify multiple channels. To implement that, we get all (key,
> value), ignore keys other than "tls-channel" and "plaintext-channel",
> and set up a channel for each of the others.
>
> * -writeconfig, in config_write_opts() with callback config_write_opt()
>
> We write out all keys in order.
>
> * The options visitor, in opts_start_struct()
>
> We convert the list of (key, value) to a hash table of (key, list of
> values). Most of the time, the list of values has exactly one
> element.
>
> When the visitor's user asks for a scalar, we return the last element
> of the list of values, in lookup_scalar().
>
> When the user asks for list elements, we return the elements of the
> list of values in order, in opts_next_list(), or if there are none,
> the empty list in opts_start_list().
Note that the only way to get non-last values is to iterate over all
(key, value). The combination of "getting a specific key's value gets
the last one" and "iterating over all keys gets all values" is poor
interface design. The latter feature got pressed into service to do
list-valued keys. When qemu_opts_set_defaults() got added (commit
4f6dd9a), the bad interaction with the list-valued keys hack wasn't
considered, probably because the whole thing had become too byzantine to
fully understand.
> Unlike the options visitor, this patch (judging from your description)
> makes a list only when keys are repeated. The QObject visitor will have
> to cope with finding both scalars and lists. When it finds a scalar,
> but needs a list, it'll have to wrap it in a list (PATCH 09, I think).
> When it finds a list, but needs a scalar, it'll have to fish it out of
> the list (where is that?).
>
>> All existing callers are all converted to explicitly
>> request the historical behaviour of only reporting the
>> last key. Later patches will make use of the new modes.
>>
>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>
> Out of steam for today.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values Daniel P. Berrange
@ 2016-10-13 12:35 ` Markus Armbruster
2016-10-13 14:46 ` Kevin Wolf
2016-10-20 14:46 ` [Qemu-devel] " Daniel P. Berrange
0 siblings, 2 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-13 12:35 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini,
Andreas Färber, Kevin Wolf
Cc: Kevin for discussion of QemuOpts dotted key convention
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Currently qdict_crumple requires a totally flat QDict as its
> input. i.e. all values in the QDict must be scalar types.
>
> In order to have backwards compatibility with the OptsVisitor,
> qemu_opt_to_qdict() has a new mode where it may return a QList
> for values in the QDict, if there was a repeated key. We thus
> need to allow compound types to appear as values in the input
> dict given to qdict_crumple().
>
> To avoid confusion, we sanity check that the user has not mixed
> the old and new syntax at the same time. e.g. these are allowed
>
> foo=hello,foo=world,foo=wibble
> foo.0=hello,foo.1=world,foo.2=wibble
>
> but this is forbidden
>
> foo=hello,foo=world,foo.2=wibble
I understand the need for foo.bar=val. It makes it possible to specify
nested dictionaries with QemuOpts.
The case for foo.0=val is less clear. QemuOpts already supports lists,
by repeating keys. Why do we need a second, wordier way to specify
them?
Note that this second way creates entirely new failure modes and
restrictions. Let me show using an example derived from one in
qdict_crumple()'s contract:
foo.0.bar=bla,foo.eek.bar=blubb
Without the dotted key convention, this is perfectly fine: key
"foo.0.bar" has the single value "bla", and key "foo.eek.bar" has
the single value "blubb". Equivalent JSON would be
{ "foo.0.bar": "bla", "foo.eek.bar": "blubb" }
With just the struct convention, it's still fine: it obviously means
the same as JSON
{ "foo": { "0": { "bar": "bla" }, "eek": { "bar": "blubb" } } }
Adding the list convention makes it invalid. It also outlaws a
bunch of keys that would be just fine in JSON, namely any that get
recognized as list index. Raise your hand if you're willing to bet
real money on your predictions of what will be recognized as list
index, without looking at the code. I'm not.
I'm afraid I have growing doubts regarding the QemuOpts dotted key
convention in general.
The convention makes '.' a special character in keys, but only
sometimes. If the key gets consumed by something that uses dotted key
convention, '.' is special, and to get a non-special '.', you need to
escape it by doubling. Else, it's not.
Since the same key can be used differently by different code, the same
'.' could in theory be both special and non-special. In practice, this
would be madness.
Adopting the dotted key convention for an existing QemuOpts option, say
-object [PATCH 15], *breaks* existing command line usage of keys
containing '.', because you now have to escape the '.'. Dan, I'm afraid
you need to show that no such keys exist, or if they exist they don't
matter.
I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
property "ide.0". Our chronic inability to consistently restrict names
in ABI to something sane is beyond foolish.
It's probably too late to back out the dotted key convention completely.
Kevin?
Can we still back out the list part of the convention, and use repeated
keys instead?
If we're stuck with some form of the dotted key convention, can we at
least make it a more integral part of QemuOpts rather than something
bolted on as an afterthought? Here's my thinking on how that might be
done:
* Have a QemuOptsList flag @flat.
* If @flat, QemuOpts behaves as it always has: the special characters
are ',' and '=', and parsing a key=value,... string produces a list
where each element represents one key=value from the string, in the
same order.
* If not @flat, '.' becomes an additional special character, and parsing
a key=value,... string produces a dictionary, similar to the one you
get now by converting with qemu_opts_to_qdict() and filtering through
qdict_crumple().
The difference to now is that you either always crumple, or not at all,
and the meaning of '.' is unambiguous.
I wish we had refrained from saddling QemuOpts with even more magic.
Compared to this swamp, use of JSON on the command line looks rather
appealing to me.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-13 12:35 ` Markus Armbruster
@ 2016-10-13 14:46 ` Kevin Wolf
2016-10-17 14:50 ` Markus Armbruster
2016-10-20 14:46 ` [Qemu-devel] " Daniel P. Berrange
1 sibling, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-13 14:46 UTC (permalink / raw)
To: Markus Armbruster
Cc: Daniel P. Berrange, qemu-devel, qemu-block, Max Reitz,
Paolo Bonzini, Andreas Färber
Am 13.10.2016 um 14:35 hat Markus Armbruster geschrieben:
> Cc: Kevin for discussion of QemuOpts dotted key convention
>
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > Currently qdict_crumple requires a totally flat QDict as its
> > input. i.e. all values in the QDict must be scalar types.
> >
> > In order to have backwards compatibility with the OptsVisitor,
> > qemu_opt_to_qdict() has a new mode where it may return a QList
> > for values in the QDict, if there was a repeated key. We thus
> > need to allow compound types to appear as values in the input
> > dict given to qdict_crumple().
> >
> > To avoid confusion, we sanity check that the user has not mixed
> > the old and new syntax at the same time. e.g. these are allowed
> >
> > foo=hello,foo=world,foo=wibble
> > foo.0=hello,foo.1=world,foo.2=wibble
> >
> > but this is forbidden
> >
> > foo=hello,foo=world,foo.2=wibble
>
> I understand the need for foo.bar=val. It makes it possible to specify
> nested dictionaries with QemuOpts.
>
> The case for foo.0=val is less clear. QemuOpts already supports lists,
> by repeating keys. Why do we need a second, wordier way to specify
> them?
Probably primarily because someone didn't realise this when introducing
the dotted syntax. Also because flat QDicts can't represent this.
But even if I realised that QemuOpts support this syntax, I think we
would still have to use the dotted syntax because it's explicit about
the index and we need that because the list can contains dicts.
Compare this:
driver=quorum,
child.0.driver=file,child.0.filename=disk1.img,
child.1.driver=host_device,child.1.filename=/dev/sdb,
child.2.driver=nbd,child.2.host=localhost
And this:
driver=quorum,
child.driver=file,child.filename=disk1.img,
child.driver=host_device,child.filename=/dev/sdb,
child.driver=nbd,child.host=localhost
For starters, can we really trust the order in QemuOpts so that the
right driver and filename are associated with each other?
We would also have code to associate the third child.driver with the
first child.host (because file and host_device don't have a host
property). And this isn't even touching optional arguments yet. Would
you really want to implement or review this?
> Note that this second way creates entirely new failure modes and
> restrictions. Let me show using an example derived from one in
> qdict_crumple()'s contract:
>
> foo.0.bar=bla,foo.eek.bar=blubb
>
> Without the dotted key convention, this is perfectly fine: key
> "foo.0.bar" has the single value "bla", and key "foo.eek.bar" has
> the single value "blubb". Equivalent JSON would be
>
> { "foo.0.bar": "bla", "foo.eek.bar": "blubb" }
>
> With just the struct convention, it's still fine: it obviously means
> the same as JSON
>
> { "foo": { "0": { "bar": "bla" }, "eek": { "bar": "blubb" } } }
>
> Adding the list convention makes it invalid. It also outlaws a
> bunch of keys that would be just fine in JSON, namely any that get
> recognized as list index. Raise your hand if you're willing to bet
> real money on your predictions of what will be recognized as list
> index, without looking at the code. I'm not.
If you're adding dict keys to the schema that could be reasonably parsed
as a list index, I think you're doing something wrong. I would agree
that this is a problem if we were talking about user-supplied values,
but it's just keys that are defined by qemu.
> I'm afraid I have growing doubts regarding the QemuOpts dotted key
> convention in general.
>
> The convention makes '.' a special character in keys, but only
> sometimes. If the key gets consumed by something that uses dotted key
> convention, '.' is special, and to get a non-special '.', you need to
> escape it by doubling. Else, it's not.
Do we really implement escaping by doubling dots? This would be news to
me.
Do we even have a reason to escape dots, i.e. are there any options in
the schema whose keys contain a dot? I think we took care to avoid such
things.
> Since the same key can be used differently by different code, the same
> '.' could in theory be both special and non-special. In practice, this
> would be madness.
>
> Adopting the dotted key convention for an existing QemuOpts option, say
> -object [PATCH 15], *breaks* existing command line usage of keys
> containing '.', because you now have to escape the '.'. Dan, I'm afraid
> you need to show that no such keys exist, or if they exist they don't
> matter.
>
> I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
> property "ide.0". Our chronic inability to consistently restrict names
> in ABI to something sane is beyond foolish.
I wanted to have a look at this example, but I can only find the string
"ide.0" used as a bus name in the sources, that is, a value rather than
a key.
Do you have a pointer to the property definition that you mean?
> It's probably too late to back out the dotted key convention completely.
> Kevin?
Yes, far too late.
Also, I haven't yet seen even just an idea for an alternative solution.
> Can we still back out the list part of the convention, and use repeated
> keys instead?
See above, repeated keys are messy and can't provide the same thing.
Also it's been in use since quorum was introduced in commit c88a1de51,
February 2014. Pretty sure that this means no anyway.
> If we're stuck with some form of the dotted key convention, can we at
> least make it a more integral part of QemuOpts rather than something
> bolted on as an afterthought? Here's my thinking on how that might be
> done:
>
> * Have a QemuOptsList flag @flat.
>
> * If @flat, QemuOpts behaves as it always has: the special characters
> are ',' and '=', and parsing a key=value,... string produces a list
> where each element represents one key=value from the string, in the
> same order.
>
> * If not @flat, '.' becomes an additional special character, and parsing
> a key=value,... string produces a dictionary, similar to the one you
> get now by converting with qemu_opts_to_qdict() and filtering through
> qdict_crumple().
>
> The difference to now is that you either always crumple, or not at all,
> and the meaning of '.' is unambiguous.
Integrating qdict_crumple() functionality in QemuOpts sounds like a
reasonable next step after Dan's series.
> I wish we had refrained from saddling QemuOpts with even more magic.
> Compared to this swamp, use of JSON on the command line looks rather
> appealing to me.
I'd bet most users would disagree with you because they use only simple
cases.
In more complicated setups, dotted syntax still works (unlike repeated
options), but becomes a bit verbose because of the repetition of long
prefixes. If we wanted to attack this, we would have to introduce two
more special characters (brackets) for a more concise nesting syntax.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-13 14:46 ` Kevin Wolf
@ 2016-10-17 14:50 ` Markus Armbruster
2016-10-17 15:43 ` Paolo Bonzini
` (2 more replies)
0 siblings, 3 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-17 14:50 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Max Reitz, Paolo Bonzini, Andreas Färber
Kevin Wolf <kwolf@redhat.com> writes:
> Am 13.10.2016 um 14:35 hat Markus Armbruster geschrieben:
>> Cc: Kevin for discussion of QemuOpts dotted key convention
>>
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > Currently qdict_crumple requires a totally flat QDict as its
>> > input. i.e. all values in the QDict must be scalar types.
>> >
>> > In order to have backwards compatibility with the OptsVisitor,
>> > qemu_opt_to_qdict() has a new mode where it may return a QList
>> > for values in the QDict, if there was a repeated key. We thus
>> > need to allow compound types to appear as values in the input
>> > dict given to qdict_crumple().
>> >
>> > To avoid confusion, we sanity check that the user has not mixed
>> > the old and new syntax at the same time. e.g. these are allowed
>> >
>> > foo=hello,foo=world,foo=wibble
>> > foo.0=hello,foo.1=world,foo.2=wibble
>> >
>> > but this is forbidden
>> >
>> > foo=hello,foo=world,foo.2=wibble
>>
>> I understand the need for foo.bar=val. It makes it possible to specify
>> nested dictionaries with QemuOpts.
>>
>> The case for foo.0=val is less clear. QemuOpts already supports lists,
>> by repeating keys. Why do we need a second, wordier way to specify
>> them?
>
> Probably primarily because someone didn't realise this when introducing
> the dotted syntax.
Can't even blame "someone" for that; it's an obscure, underdocumented
feature of an interface that's collapsing under its load of obscure,
underdocumented features.
On the other hand, that's not exactly a state that allows for *more*
obscure features.
> Also because flat QDicts can't represent this.
Explain?
> But even if I realised that QemuOpts support this syntax, I think we
> would still have to use the dotted syntax because it's explicit about
> the index and we need that because the list can contains dicts.
>
> Compare this:
>
> driver=quorum,
> child.0.driver=file,child.0.filename=disk1.img,
> child.1.driver=host_device,child.1.filename=/dev/sdb,
> child.2.driver=nbd,child.2.host=localhost
>
> And this:
>
> driver=quorum,
> child.driver=file,child.filename=disk1.img,
> child.driver=host_device,child.filename=/dev/sdb,
> child.driver=nbd,child.host=localhost
Aside: both are about equally illegible, to be honest.
> For starters, can we really trust the order in QemuOpts so that the
> right driver and filename are associated with each other?
The order is trustworthy, but...
> We would also have code to associate the third child.driver with the
> first child.host (because file and host_device don't have a host
> property). And this isn't even touching optional arguments yet. Would
> you really want to implement or review this?
... you're right, doing lists by repeating keys breaks down when
combined with the dotted key convention's use of repetition to do
structured values.
Permit me to digress.
QemuOpts wasn't designed for list-values keys. Doing lists by
repetition was clever.
QemuOpts wasn't designed for structured values. Doing structured values
by a dotted key convention plus repetition was clever.
And there's the problem: too much cleverness, not enough "this is being
pushed way beyond its design limits, time to replace it".
For me, a replacement should do structured values by providing suitable
*value* syntax instead of hacking it into the keys:
{ "driver": "quorum",
"child": [ { "driver": "file", "filename": "disk1.img" },
{ "driver": "host_device", "filename=/dev/sdb" },
{ "driver": "nbd", "host": "localhost" } ] }
Note how the structure is obvious. It isn't with dotted keys, not even
if you order them sensibly (which users inevitably won't).
Also not that the value needs to be parsed by QemuOpts! You can't leave
it to the consumer of the QemuOpts, because if you did, you'd have to
escape the commas.
If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
could try
driver=quorum,
child=[{ driver=file, filename=disk1.img },
{ driver=host_device, filename=/dev/sdb },
{ driver=nbd, host=localhost } ]
I'd go with some existing syntax, though. The one we already use is
JSON.
End of digression.
>> Note that this second way creates entirely new failure modes and
>> restrictions. Let me show using an example derived from one in
>> qdict_crumple()'s contract:
>>
>> foo.0.bar=bla,foo.eek.bar=blubb
>>
>> Without the dotted key convention, this is perfectly fine: key
>> "foo.0.bar" has the single value "bla", and key "foo.eek.bar" has
>> the single value "blubb". Equivalent JSON would be
>>
>> { "foo.0.bar": "bla", "foo.eek.bar": "blubb" }
>>
>> With just the struct convention, it's still fine: it obviously means
>> the same as JSON
>>
>> { "foo": { "0": { "bar": "bla" }, "eek": { "bar": "blubb" } } }
>>
>> Adding the list convention makes it invalid. It also outlaws a
>> bunch of keys that would be just fine in JSON, namely any that get
>> recognized as list index. Raise your hand if you're willing to bet
>> real money on your predictions of what will be recognized as list
>> index, without looking at the code. I'm not.
>
> If you're adding dict keys to the schema that could be reasonably parsed
> as a list index, I think you're doing something wrong. I would agree
> that this is a problem if we were talking about user-supplied values,
> but it's just keys that are defined by qemu.
Your argument would be stronger if we didn't have such a disappointing
track record on sensible naming. We've failed at picking a consistent
rules for names, let alone enforcing them. Instead, we have local
rules, enforced to varying degrees, and of course inconsistent with each
other. We even have code to mangle names, so that they conform to some
other set of rules.
We should not extend an interface in a way that makes it rely on certain
rules for names without writing down and enforcing these rules.
You did it *locally* for block stuff, and got away with it, because you
control the keys (hopefully), and nobody (including probably you and
very much me) wants to think about this particular mess unless it's
absolutely, obviously, embarrassingly necessary.
Which is what he get to do now, because now we're discussing to adopt
your local thing for general use, with keys no single person really
knows, let alone controls.
Your dotted key convention requires two rules: 1. names must not look
like integers, and 2. names must not contain '.'.
We can avoid rule 2 by requiring '.' to be escaped. Dan's
qdict_crumple() currently does that, to your surprise. Adding the
escaping to existing options is a compatibility break, however. So, if
names with '.' already exist, we can either break compatibility by
renaming them, or break it by requiring the '.' to be escaped.
The names in question are the names declared in QemuOptDesc (easy enough
to find) plus any names that the code may use with empty QemuOptDesc
(harder). Empty QemuOptDesc are used with the following option groups:
* "drive", qemu_drive_opts in blockdev.c
Also empty_opts in qemu-io.c and qemu_drive_opts in test-replication.c
* "numa", qemu_numa_opts in numa.c
* "device", qemu_device_opts in qdev-monitor.c
This one pulls in qdev properties. Properties violating rule 2 exist.
* "object", qemu_object_opts in vl.c
Also qemu_object_opts in qemu-img.c, qemu-io.c and qemu-nbd.c (WTF?)
This one pulls in properties of user-creatable objects.
* "source", qemu_soource_opts (sic!) in qemu-img.c
* "reopen", reopen_opts in qemu-io-cmds.c
* "file", file_opts in qemu-io.c and qemu-nbd.c
* "machine", qemu_machine_opts in vl.c
This one pulls in machine properties.
* "tpmdev", qemu_tpmdev_opts in vl.c
* "netdev", qemu_netdev_opts in net.c
* "net", qemu_net_opts in net.c
* "userdef", userdef_opts in tests-opts-visitor.c
* "opts_list_03", opts_list_03 in test-qemu-opts.c
* "acpi", qemu_acpi_opts in hw/acpi/core.c
* "smbios", qemu_smbios_opts in smbios.c
Empty QemuOptDesc is used this widely because QemuOpts is too primitive
to support the needs of its users. Unlike a QAPI schema. Just sayin'.
Finding the names and assessing how they're impacted by the dotted key
convention is going to be a substantial chunk of work.
>> I'm afraid I have growing doubts regarding the QemuOpts dotted key
>> convention in general.
>>
>> The convention makes '.' a special character in keys, but only
>> sometimes. If the key gets consumed by something that uses dotted key
>> convention, '.' is special, and to get a non-special '.', you need to
>> escape it by doubling. Else, it's not.
>
> Do we really implement escaping by doubling dots? This would be news to
> me.
Dan's PATCH 02/21 does.
> Do we even have a reason to escape dots, i.e. are there any options in
> the schema whose keys contain a dot? I think we took care to avoid such
> things.
I'm afraid we did not.
>> Since the same key can be used differently by different code, the same
>> '.' could in theory be both special and non-special. In practice, this
>> would be madness.
>>
>> Adopting the dotted key convention for an existing QemuOpts option, say
>> -object [PATCH 15], *breaks* existing command line usage of keys
>> containing '.', because you now have to escape the '.'. Dan, I'm afraid
>> you need to show that no such keys exist, or if they exist they don't
>> matter.
>>
>> I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
>> property "ide.0". Our chronic inability to consistently restrict names
>> in ABI to something sane is beyond foolish.
>
> I wanted to have a look at this example, but I can only find the string
> "ide.0" used as a bus name in the sources, that is, a value rather than
> a key.
>
> Do you have a pointer to the property definition that you mean?
We've gotten better at hiding property definitions...
"qemu-system-ppc -device macio-ide,help" shows the property:
macio-ide.ide.0=child<IDE>
The device is not marked no-user, but the obvious attempt to use it
fails:
$ qemu-system-ppc -device macio-ide
qemu-system-ppc: Option '-device macio-ide' cannot be handled by this machine
This is the Alex's dynamic system bus thingy, which put most of the
system bus devices in help even when they cannot be used.
Perhaps you can make a case why this property cannot be used via
QemuOpts. If so, great! There are about three dozen more for you to
check, followed by fourteen more option groups with empty QemuOptDesc,
and a bunch of non-empty QemuOptDesc ;-P
>> It's probably too late to back out the dotted key convention completely.
>> Kevin?
>
> Yes, far too late.
If we're stuck with the existing use of dotted key convention, we're
stuck with it. But that's no excuse for adding more uses.
If we decide we want to use dotted key convention more widely, we must
document and enforce the naming conventions it requires at least in all
the places where we use it. As discussed above, this might involve
compatibility breaks, hopefully minor.
> Also, I haven't yet seen even just an idea for an alternative solution.
See digression above.
>> Can we still back out the list part of the convention, and use repeated
>> keys instead?
>
> See above, repeated keys are messy and can't provide the same thing.
>
> Also it's been in use since quorum was introduced in commit c88a1de51,
> February 2014. Pretty sure that this means no anyway.
>
>> If we're stuck with some form of the dotted key convention, can we at
>> least make it a more integral part of QemuOpts rather than something
>> bolted on as an afterthought? Here's my thinking on how that might be
>> done:
>>
>> * Have a QemuOptsList flag @flat.
>>
>> * If @flat, QemuOpts behaves as it always has: the special characters
>> are ',' and '=', and parsing a key=value,... string produces a list
>> where each element represents one key=value from the string, in the
>> same order.
>>
>> * If not @flat, '.' becomes an additional special character, and parsing
>> a key=value,... string produces a dictionary, similar to the one you
>> get now by converting with qemu_opts_to_qdict() and filtering through
>> qdict_crumple().
>>
>> The difference to now is that you either always crumple, or not at all,
>> and the meaning of '.' is unambiguous.
>
> Integrating qdict_crumple() functionality in QemuOpts sounds like a
> reasonable next step after Dan's series.
>
>> I wish we had refrained from saddling QemuOpts with even more magic.
>> Compared to this swamp, use of JSON on the command line looks rather
>> appealing to me.
>
> I'd bet most users would disagree with you because they use only simple
> cases.
Use of JSON on the command line when and where you need nesting, not
everywhere. The simple cases can stay exactly as they are.
> In more complicated setups, dotted syntax still works (unlike repeated
> options), but becomes a bit verbose because of the repetition of long
> prefixes. If we wanted to attack this, we would have to introduce two
> more special characters (brackets) for a more concise nesting syntax.
>
> Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-17 14:50 ` Markus Armbruster
@ 2016-10-17 15:43 ` Paolo Bonzini
2016-10-17 17:48 ` Markus Armbruster
2016-10-17 17:38 ` Eric Blake
2016-10-18 9:34 ` Kevin Wolf
2 siblings, 1 reply; 109+ messages in thread
From: Paolo Bonzini @ 2016-10-17 15:43 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, qemu-block, qemu-devel, Max Reitz, Andreas Färber
> For me, a replacement should do structured values by providing suitable
> *value* syntax instead of hacking it into the keys:
>
> { "driver": "quorum",
> "child": [ { "driver": "file", "filename": "disk1.img" },
> { "driver": "host_device", "filename=/dev/sdb" },
> { "driver": "nbd", "host": "localhost" } ] }
>
> Note how the structure is obvious. It isn't with dotted keys, not even
> if you order them sensibly (which users inevitably won't).
>
> Also not that the value needs to be parsed by QemuOpts! You can't leave
> it to the consumer of the QemuOpts, because if you did, you'd have to
> escape the commas.
>
> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
> could try
>
> driver=quorum,
> child=[{ driver=file, filename=disk1.img },
> { driver=host_device, filename=/dev/sdb },
> { driver=nbd, host=localhost } ]
>
> I'd go with some existing syntax, though. The one we already use is
> JSON.
In fact there is already "filename=json:{...}" support in the block layer.
By the way, abuse of QemuOpts dates back to http://wiki.qemu.org/Features/QCFG.
> Your dotted key convention requires two rules: 1. names must not look
> like integers, and 2. names must not contain '.'.
>
> We can avoid rule 2 by requiring '.' to be escaped. Dan's
> qdict_crumple() currently does that, to your surprise. Adding the
> escaping to existing options is a compatibility break, however. So, if
> names with '.' already exist, we can either break compatibility by
> renaming them, or break it by requiring the '.' to be escaped.
> * "device", qemu_device_opts in qdev-monitor.c
>
> This one pulls in qdev properties. Properties violating rule 2 exist.
Which are they? Only bus names?
> * "object", qemu_object_opts in vl.c
>
> This one pulls in properties of user-creatable objects.
>
> * "machine", qemu_machine_opts in vl.c
>
> This one pulls in machine properties.
> > > I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
> > > property "ide.0". Our chronic inability to consistently restrict names
> > > in ABI to something sane is beyond foolish.
> >
> > I wanted to have a look at this example, but I can only find the string
> > "ide.0" used as a bus name in the sources, that is, a value rather than
> > a key.
> >
> > Do you have a pointer to the property definition that you mean?
>
> We've gotten better at hiding property definitions...
>
> "qemu-system-ppc -device macio-ide,help" shows the property:
>
> macio-ide.ide.0=child<IDE>
It is a bug that this property is shown in the help, because it's not
assignable (same for all other child<> properties). I'd rather declare
other occurrences of "." in user-accessible property names to be bugs,
and break the ABI if there are any.
Paolo
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-17 14:50 ` Markus Armbruster
2016-10-17 15:43 ` Paolo Bonzini
@ 2016-10-17 17:38 ` Eric Blake
2016-10-18 10:59 ` Markus Armbruster
2016-10-18 9:34 ` Kevin Wolf
2 siblings, 1 reply; 109+ messages in thread
From: Eric Blake @ 2016-10-17 17:38 UTC (permalink / raw)
To: Markus Armbruster, Kevin Wolf
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 2895 bytes --]
On 10/17/2016 09:50 AM, Markus Armbruster wrote:
>> But even if I realised that QemuOpts support this syntax, I think we
>> would still have to use the dotted syntax because it's explicit about
>> the index and we need that because the list can contains dicts.
>>
>> Compare this:
>>
>> driver=quorum,
>> child.0.driver=file,child.0.filename=disk1.img,
>> child.1.driver=host_device,child.1.filename=/dev/sdb,
>> child.2.driver=nbd,child.2.host=localhost
>>
>> And this:
>>
>> driver=quorum,
>> child.driver=file,child.filename=disk1.img,
>> child.driver=host_device,child.filename=/dev/sdb,
>> child.driver=nbd,child.host=localhost
>
> Aside: both are about equally illegible, to be honest.
> Permit me to digress.
>
> QemuOpts wasn't designed for list-values keys. Doing lists by
> repetition was clever.
>
> QemuOpts wasn't designed for structured values. Doing structured values
> by a dotted key convention plus repetition was clever.
>
> And there's the problem: too much cleverness, not enough "this is being
> pushed way beyond its design limits, time to replace it".
>
> For me, a replacement should do structured values by providing suitable
> *value* syntax instead of hacking it into the keys:
>
> { "driver": "quorum",
> "child": [ { "driver": "file", "filename": "disk1.img" },
> { "driver": "host_device", "filename=/dev/sdb" },
> { "driver": "nbd", "host": "localhost" } ] }
Possible hack solution:
QemuOpts already special-cases id=. What if we ALSO make it
special-case a leading json=? Shown here with shell quoting, the above
example of creating a Quorum -drive argument could then be:
-drive json='
{ "driver": "quorum",
"child": [ { "driver": "file", "filename": "disk1.img" },
{ "driver": "host_device", "filename=/dev/sdb" },
{ "driver": "nbd", "host": "localhost" } ] }
'
As far as I know, we don't have 'json' as any existing QemuOpts key (do
we? A full audit may be better than my quick git grep '"json"'). Thus,
if QemuOpts sees a leading json=, it hands off the rest of the string to
the same parser as we use for QMP, where we no longer have to escape
commas (even nicer than the drive hack where we support
filename=json:{...} but have to double up all commas to make it through
the QemuOpts layer). Encountering json= as anything other than the
first option would be an error, and you would be unable to combine a
json= option with any other old-style option. In other words, the use
of leading json= would be the switch for whether to do old-style parsing
or to use a saner syntax for everything else that needs structure, on a
per-argument basis.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-17 15:43 ` Paolo Bonzini
@ 2016-10-17 17:48 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-17 17:48 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Kevin Wolf, Andreas Färber, qemu-devel, qemu-block, Max Reitz
Paolo Bonzini <pbonzini@redhat.com> writes:
>> For me, a replacement should do structured values by providing suitable
>> *value* syntax instead of hacking it into the keys:
>>
>> { "driver": "quorum",
>> "child": [ { "driver": "file", "filename": "disk1.img" },
>> { "driver": "host_device", "filename=/dev/sdb" },
>> { "driver": "nbd", "host": "localhost" } ] }
>>
>> Note how the structure is obvious. It isn't with dotted keys, not even
>> if you order them sensibly (which users inevitably won't).
>>
>> Also not that the value needs to be parsed by QemuOpts! You can't leave
>> it to the consumer of the QemuOpts, because if you did, you'd have to
>> escape the commas.
>>
>> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
>> could try
>>
>> driver=quorum,
>> child=[{ driver=file, filename=disk1.img },
>> { driver=host_device, filename=/dev/sdb },
>> { driver=nbd, host=localhost } ]
>>
>> I'd go with some existing syntax, though. The one we already use is
>> JSON.
>
> In fact there is already "filename=json:{...}" support in the block layer.
Yes.
> By the way, abuse of QemuOpts dates back to http://wiki.qemu.org/Features/QCFG.
That page is from 2011, when QAPI was being implemented. The idea to
bring the power of the QAPI schema to bear on the command line is sound,
but QCFG never made it past the ideas stage, I'm afraid. It uses either
dotted keys or braces for nested structs. It doesn't cover lists.
>> Your dotted key convention requires two rules: 1. names must not look
>> like integers, and 2. names must not contain '.'.
>>
>> We can avoid rule 2 by requiring '.' to be escaped. Dan's
>> qdict_crumple() currently does that, to your surprise. Adding the
>> escaping to existing options is a compatibility break, however. So, if
>> names with '.' already exist, we can either break compatibility by
>> renaming them, or break it by requiring the '.' to be escaped.
>
>> * "device", qemu_device_opts in qdev-monitor.c
>>
>> This one pulls in qdev properties. Properties violating rule 2 exist.
>
> Which are they? Only bus names?
Finding properties is difficult, because we (foolishly!) define them in
code rather than data. My testing with "info qdm" for all targets
finds:
= xlnx.axi-dma =
xlnx.axi-dma[0]
= xlnx.xps-ethernetlite =
xlnx.xps-ethernetlite[0]
= xlnx.xps-intc =
xlnx.xps-intc[0]
= xlnx.xps-uartlite =
xlnx.xps-uartlite[0]
= xlnx.axi-dma =
xlnx.axi-dma[0]
= xlnx.xps-ethernetlite =
xlnx.xps-ethernetlite[0]
= xlnx.xps-intc =
xlnx.xps-intc[0]
= xlnx.xps-uartlite =
xlnx.xps-uartlite[0]
= cuda =
adb.0
= raven-pcihost =
pci.0
= macio-ide =
ide.0
= mpc8544-guts =
mpc8544.guts[0]
= xlnx.xps-ethernetlite =
xlnx.xps-ethernetlite[0]
= xlnx.xps-intc =
xlnx.xps-intc[0]
= xlnx.xps-uartlite =
xlnx.xps-uartlite[0]
= cuda =
adb.0
= raven-pcihost =
pci.0
= macio-ide =
ide.0
= mpc8544-guts =
mpc8544.guts[0]
= xlnx.xps-ethernetlite =
xlnx.xps-ethernetlite[0]
= xlnx.xps-intc =
xlnx.xps-intc[0]
= xlnx.xps-uartlite =
xlnx.xps-uartlite[0]
= s390-sclp-event-facility =
s390-sclp-events-bus.0
= cgthree =
cg3.reg[0]
cg3.prom[0]
= SUNW,tcx =
tcx.thc[0]
tcx.rblit[0]
tcx.dac[0]
tcx.alt[0]
tcx.tec[0]
tcx.rstip[0]
tcx.blit[0]
tcx.dhc[0]
tcx.prom[0]
tcx.stip[0]
>> * "object", qemu_object_opts in vl.c
>>
>> This one pulls in properties of user-creatable objects.
>>
>> * "machine", qemu_machine_opts in vl.c
>>
>> This one pulls in machine properties.
>
>> > > I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
>> > > property "ide.0". Our chronic inability to consistently restrict names
>> > > in ABI to something sane is beyond foolish.
>> >
>> > I wanted to have a look at this example, but I can only find the string
>> > "ide.0" used as a bus name in the sources, that is, a value rather than
>> > a key.
>> >
>> > Do you have a pointer to the property definition that you mean?
>>
>> We've gotten better at hiding property definitions...
>>
>> "qemu-system-ppc -device macio-ide,help" shows the property:
>>
>> macio-ide.ide.0=child<IDE>
>
> It is a bug that this property is shown in the help, because it's not
> assignable (same for all other child<> properties).
Let's fix it then.
> I'd rather declare
> other occurrences of "." in user-accessible property names to be bugs,
> and break the ABI if there are any.
I propose to adopt QAPI's rules on permissible names: "All names must
begin with a letter, and contain only ASCII letters, digits, dash, and
underscore" (docs/qapi-code-gen.txt). QAPI's naming rules get adopted
anyway on QAPIfication, so why wait? :)
Note that this may affect names generated by automatic arrayification
(commit 3396590), because those contain [ and ].
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-17 14:50 ` Markus Armbruster
2016-10-17 15:43 ` Paolo Bonzini
2016-10-17 17:38 ` Eric Blake
@ 2016-10-18 9:34 ` Kevin Wolf
2016-10-18 15:35 ` Markus Armbruster
2 siblings, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-18 9:34 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-block, qemu-devel, Max Reitz, Paolo Bonzini, Andreas Färber
Am 17.10.2016 um 16:50 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 13.10.2016 um 14:35 hat Markus Armbruster geschrieben:
> >> Cc: Kevin for discussion of QemuOpts dotted key convention
> >>
> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>
> >> > Currently qdict_crumple requires a totally flat QDict as its
> >> > input. i.e. all values in the QDict must be scalar types.
> >> >
> >> > In order to have backwards compatibility with the OptsVisitor,
> >> > qemu_opt_to_qdict() has a new mode where it may return a QList
> >> > for values in the QDict, if there was a repeated key. We thus
> >> > need to allow compound types to appear as values in the input
> >> > dict given to qdict_crumple().
> >> >
> >> > To avoid confusion, we sanity check that the user has not mixed
> >> > the old and new syntax at the same time. e.g. these are allowed
> >> >
> >> > foo=hello,foo=world,foo=wibble
> >> > foo.0=hello,foo.1=world,foo.2=wibble
> >> >
> >> > but this is forbidden
> >> >
> >> > foo=hello,foo=world,foo.2=wibble
> >>
> >> I understand the need for foo.bar=val. It makes it possible to specify
> >> nested dictionaries with QemuOpts.
> >>
> >> The case for foo.0=val is less clear. QemuOpts already supports lists,
> >> by repeating keys. Why do we need a second, wordier way to specify
> >> them?
> >
> > Probably primarily because someone didn't realise this when introducing
> > the dotted syntax.
>
> Can't even blame "someone" for that; it's an obscure, underdocumented
> feature of an interface that's collapsing under its load of obscure,
> underdocumented features.
>
> On the other hand, that's not exactly a state that allows for *more*
> obscure features.
I don't really think we're introducing more obscure features here, but
rather trying to implement a single, and rather straightforward, way as
the standard.
Dotted syntax for hierarchy has actually plenty of precedence in qemu if
you look a bit closer (the block layer, -global, -device foo,help, even
the bus names you mentioned below are really just flattened lists), so
we're only making things more consistent.
> > Also because flat QDicts can't represent this.
>
> Explain?
Repeated options are parsed into QLists. If you keep it at that without
flattening you have at least a QDict that contains a QList that contains
simple types. This is not flat any more.
Of course, you could argue that flat QDicts are the wrong data structure
in the first place and instead of flatting everything we should have
done the equivalent of qdict_crumple from the beginning, but they were
the natural extension of what was already there and happened to work
good enough, so this is what we're currently using.
> > But even if I realised that QemuOpts support this syntax, I think we
> > would still have to use the dotted syntax because it's explicit about
> > the index and we need that because the list can contains dicts.
> >
> > Compare this:
> >
> > driver=quorum,
> > child.0.driver=file,child.0.filename=disk1.img,
> > child.1.driver=host_device,child.1.filename=/dev/sdb,
> > child.2.driver=nbd,child.2.host=localhost
> >
> > And this:
> >
> > driver=quorum,
> > child.driver=file,child.filename=disk1.img,
> > child.driver=host_device,child.filename=/dev/sdb,
> > child.driver=nbd,child.host=localhost
>
> Aside: both are about equally illegible, to be honest.
Not sure about equally, but let's agree on "both are illegible".
> > For starters, can we really trust the order in QemuOpts so that the
> > right driver and filename are associated with each other?
>
> The order is trustworthy, but...
>
> > We would also have code to associate the third child.driver with the
> > first child.host (because file and host_device don't have a host
> > property). And this isn't even touching optional arguments yet. Would
> > you really want to implement or review this?
>
> ... you're right, doing lists by repeating keys breaks down when
> combined with the dotted key convention's use of repetition to do
> structured values.
>
> Permit me to digress.
>
> QemuOpts wasn't designed for list-values keys. Doing lists by
> repetition was clever.
>
> QemuOpts wasn't designed for structured values. Doing structured values
> by a dotted key convention plus repetition was clever.
>
> And there's the problem: too much cleverness, not enough "this is being
> pushed way beyond its design limits, time to replace it".
>
> For me, a replacement should do structured values by providing suitable
> *value* syntax instead of hacking it into the keys:
>
> { "driver": "quorum",
> "child": [ { "driver": "file", "filename": "disk1.img" },
> { "driver": "host_device", "filename=/dev/sdb" },
> { "driver": "nbd", "host": "localhost" } ] }
>
> Note how the structure is obvious. It isn't with dotted keys, not even
> if you order them sensibly (which users inevitably won't).
>
> Also not that the value needs to be parsed by QemuOpts! You can't leave
> it to the consumer of the QemuOpts, because if you did, you'd have to
> escape the commas.
Okay, so I like JSON. It's a great format for our monitor protocol. We
even have pretty printers so that it's more or less readable as output
for human users. However, there is one thing for which it is horrible:
Getting user input.
Yes, getting rid of the comma escaping is a first step, but typing JSON
on the command line remains a PITA. Mostly because of the quotes (which
you probably have to escape), but also things like the useless outer
brackets.
> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
> could try
>
> driver=quorum,
> child=[{ driver=file, filename=disk1.img },
> { driver=host_device, filename=/dev/sdb },
> { driver=nbd, host=localhost } ]
This looks a lot more agreeable as something that I have to type on the
command line.
What's more, this is a direct extension of the existing syntax. You
complained about how the step from simple configurations with -hda to
more complex ones with -drive completely changes the syntax (and rightly
so). Going from simple QemuOpts syntax to doing JSON as soon as you get
structured values and lists would be another similar step. In contrast,
the new syntax you're suggesting above is a natural extension of what's
there.
> I'd go with some existing syntax, though. The one we already use is
> JSON.
The one we already use in this context is comma separated key/value
pairs as parsed by QemuOpts.
> Your dotted key convention requires two rules: 1. names must not look
> like integers, and 2. names must not contain '.'.
>
> We can avoid rule 2 by requiring '.' to be escaped. Dan's
> qdict_crumple() currently does that, to your surprise. Adding the
> escaping to existing options is a compatibility break, however. So, if
> names with '.' already exist, we can either break compatibility by
> renaming them, or break it by requiring the '.' to be escaped.
I don't think we should support any escaping and rather forbid '.'
completely in names.
> The names in question are the names declared in QemuOptDesc (easy enough
> to find) plus any names that the code may use with empty QemuOptDesc
> (harder). Empty QemuOptDesc are used with the following option groups:
>
> * "drive", qemu_drive_opts in blockdev.c
> Also empty_opts in qemu-io.c and qemu_drive_opts in test-replication.c
>
> * "numa", qemu_numa_opts in numa.c
>
> * "device", qemu_device_opts in qdev-monitor.c
>
> This one pulls in qdev properties. Properties violating rule 2 exist.
You posted a list of them in another email. Apart from not being
user-configurable, I think it actually makes sense to interpret these as
being structured.
A lot of them are really array/list properties with a single entry. It
seems we have some inconsistency there as to whether it the syntax is
'list.index' or 'list[index]'. The former matches the dotted syntax
rule and is used in external interfaces (e.g. in bus=...), and I seem to
remember the latter is a newer QOM thing.
Would it be too late to change the latter into the former? In other
words, is the difference externally visible outside of qom-*, which we
declared a non-ABI, as far as I know?
> * "object", qemu_object_opts in vl.c
> Also qemu_object_opts in qemu-img.c, qemu-io.c and qemu-nbd.c (WTF?)
>
> This one pulls in properties of user-creatable objects.
>
> * "source", qemu_soource_opts (sic!) in qemu-img.c
>
> * "reopen", reopen_opts in qemu-io-cmds.c
>
> * "file", file_opts in qemu-io.c and qemu-nbd.c
>
> * "machine", qemu_machine_opts in vl.c
>
> This one pulls in machine properties.
>
> * "tpmdev", qemu_tpmdev_opts in vl.c
>
> * "netdev", qemu_netdev_opts in net.c
>
> * "net", qemu_net_opts in net.c
>
> * "userdef", userdef_opts in tests-opts-visitor.c
>
> * "opts_list_03", opts_list_03 in test-qemu-opts.c
>
> * "acpi", qemu_acpi_opts in hw/acpi/core.c
>
> * "smbios", qemu_smbios_opts in smbios.c
>
> Empty QemuOptDesc is used this widely because QemuOpts is too primitive
> to support the needs of its users. Unlike a QAPI schema. Just sayin'.
I don't think you have to convince anyone that QemuOpts is too limited.
But today it's still a piece that is needed to get from the command line
to something superior like QAPI. This is the whole reason why we're
doing all of this.
Completely QAPIfication of everything and going directly from the
command line to QAPI objects would be nice, but somehow I feel it's not
a realistic hope to get this as a short term solution.
> Finding the names and assessing how they're impacted by the dotted key
> convention is going to be a substantial chunk of work.
Possibly.
The other option would be to take it step by step and let every user of
QemuOpts specify on creation whether dots are used for structure or
should be taken literally. Then you can convert user by user and assess
them one at a time (which is still a substantial chunk for -object and
-device, but at least you can concentrate on only these for now).
If you're concerned about compatibility issues if we find a dot in a
name only in one of the later users: We can handle it. If you then
stumble across an obscure "weird.name" property where the dot is really
part of the name and not already representing structure, you can still
artificially reinterpret it as a nested structure with a single field
even if logically that's not what it is... Might be a bit ugly, but
keeps it working.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-17 17:38 ` Eric Blake
@ 2016-10-18 10:59 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-18 10:59 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, Paolo Bonzini, Max Reitz, Andreas Färber,
qemu-block, qemu-devel
Eric Blake <eblake@redhat.com> writes:
> On 10/17/2016 09:50 AM, Markus Armbruster wrote:
>>> But even if I realised that QemuOpts support this syntax, I think we
>>> would still have to use the dotted syntax because it's explicit about
>>> the index and we need that because the list can contains dicts.
>>>
>>> Compare this:
>>>
>>> driver=quorum,
>>> child.0.driver=file,child.0.filename=disk1.img,
>>> child.1.driver=host_device,child.1.filename=/dev/sdb,
>>> child.2.driver=nbd,child.2.host=localhost
>>>
>>> And this:
>>>
>>> driver=quorum,
>>> child.driver=file,child.filename=disk1.img,
>>> child.driver=host_device,child.filename=/dev/sdb,
>>> child.driver=nbd,child.host=localhost
>>
>> Aside: both are about equally illegible, to be honest.
>
>> Permit me to digress.
>>
>> QemuOpts wasn't designed for list-values keys. Doing lists by
>> repetition was clever.
>>
>> QemuOpts wasn't designed for structured values. Doing structured values
>> by a dotted key convention plus repetition was clever.
>>
>> And there's the problem: too much cleverness, not enough "this is being
>> pushed way beyond its design limits, time to replace it".
>>
>> For me, a replacement should do structured values by providing suitable
>> *value* syntax instead of hacking it into the keys:
>>
>> { "driver": "quorum",
>> "child": [ { "driver": "file", "filename": "disk1.img" },
>> { "driver": "host_device", "filename=/dev/sdb" },
>> { "driver": "nbd", "host": "localhost" } ] }
>
> Possible hack solution:
>
> QemuOpts already special-cases id=. What if we ALSO make it
> special-case a leading json=? Shown here with shell quoting, the above
> example of creating a Quorum -drive argument could then be:
>
> -drive json='
> { "driver": "quorum",
> "child": [ { "driver": "file", "filename": "disk1.img" },
> { "driver": "host_device", "filename=/dev/sdb" },
> { "driver": "nbd", "host": "localhost" } ] }
> '
>
> As far as I know, we don't have 'json' as any existing QemuOpts key (do
> we? A full audit may be better than my quick git grep '"json"'). Thus,
> if QemuOpts sees a leading json=, it hands off the rest of the string to
> the same parser as we use for QMP, where we no longer have to escape
> commas (even nicer than the drive hack where we support
> filename=json:{...} but have to double up all commas to make it through
> the QemuOpts layer). Encountering json= as anything other than the
> first option would be an error, and you would be unable to combine a
> json= option with any other old-style option. In other words, the use
> of leading json= would be the switch for whether to do old-style parsing
> or to use a saner syntax for everything else that needs structure, on a
> per-argument basis.
Slight variation: omit the 'json=' and recognize the '{':
-drive '{ "driver": "quorum",
"child": [ { "driver": "file", "filename": "disk1.img" },
{ "driver": "host_device", "filename=/dev/sdb" },
{ "driver": "nbd", "host": "localhost" } ] }'
As always when extending haphazardly defined syntax, the question to ask
is whether this makes the syntax (more) ambiguous.
So what is the option argument syntax now? The abstract syntax is
simple enough: "list of (key, value) pairs". For the concrete syntax,
we need to study opts_do_parse().
Each iteration of its loop accepts an abstract (key, value). It first
looks for the leftmost '=' and ','. Cases:
1. There is no '=', or it is to the right of the leftmost ','. In other
words, this (key, value) can only be key with an implied value or a
value with an implied key.
1a. If this is the first iteration, and we accept an implied key, this
is a value for the implied key. We consume everything up to the first
non-escaped ','. This may be more than the leftmost ',' we found above.
The consumed string with escapes processed is the value.
1b. Else, this is a key with an implied value. We consume everything up
to the leftmost ',' (no escaping here).
1b1. If the consumed string starts with "no", the key is everything after
the "no" and the value is "off".
1b2. Else the key is the string and the value is "on".
2. This is a key and a value. We first consume everything up to the
leftmost '=' (no escaping here). The consumed string is the key. We
then consume '='. Finally, we consume everything up to the first
non-escaped ','The consumed string with escapes processed is the value.
Thus, the option argument starts either with a key (case 1b1, 2), "no"
(case 1b2) or a value (case 1a).
Adding JSON object syntax (which always starts with '{') is ambiguous
when a key can start with '{' (case 1b1, 2) or when a value can (case
1a).
Keys starting with '{' are basically foolish. Let's outlaw them by
adopting QAPI's name rules.
Values starting with '{' are possible. The implied keys I can remember
don't have such values, though. If we interpret '{' as JSON, then users
can't omit the key when the value starts with '{'. Not pretty, but I'd
say it's tolerable.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
@ 2016-10-18 14:32 ` Markus Armbruster
2016-10-20 14:11 ` Daniel P. Berrange
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-18 14:32 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The qdict_flatten() method will take a dict whose elements are
> further nested dicts/lists and flatten them by concatenating
> keys.
>
> The qdict_crumple() method aims to do the reverse, taking a flat
> qdict, and turning it into a set of nested dicts/lists. It will
> apply nesting based on the key name, with a '.' indicating a
> new level in the hierarchy. If the keys in the nested structure
> are all numeric, it will create a list, otherwise it will create
> a dict.
>
> If the keys are a mixture of numeric and non-numeric, or the
> numeric keys are not in strictly ascending order, an error will
> be reported.
>
> As an example, a flat dict containing
>
> {
> 'foo.0.bar': 'one',
> 'foo.0.wizz': '1',
> 'foo.1.bar': 'two',
> 'foo.1.wizz': '2'
> }
>
> will get turned into a dict with one element 'foo' whose
> value is a list. The list elements will each in turn be
> dicts.
>
> {
> 'foo': [
> { 'bar': 'one', 'wizz': '1' },
> { 'bar': 'two', 'wizz': '2' }
> ],
> }
>
> If the key is intended to contain a literal '.', then it must
> be escaped as '..'. ie a flat dict
>
> {
> 'foo..bar': 'wizz',
> 'bar.foo..bar': 'eek',
> 'bar.hello': 'world'
> }
>
> Will end up as
>
> {
> 'foo.bar': 'wizz',
> 'bar': {
> 'foo.bar': 'eek',
> 'hello': 'world'
> }
> }
>
> The intent of this function is that it allows a set of QemuOpts
> to be turned into a nested data structure that mirrors the nesting
> used when the same object is defined over QMP.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qmp/qdict.h | 1 +
> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 551 insertions(+)
>
> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> index 71b8eb0..e0d24e1 100644
> --- a/include/qapi/qmp/qdict.h
> +++ b/include/qapi/qmp/qdict.h
> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> void qdict_array_split(QDict *src, QList **dst);
> int qdict_array_entries(QDict *src, const char *subqdict);
> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>
> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>
> diff --git a/qobject/qdict.c b/qobject/qdict.c
> index 60f158c..c38e90e 100644
> --- a/qobject/qdict.c
> +++ b/qobject/qdict.c
[...]
> +/**
> + * qdict_crumple:
> + * @src: the original flat dictionary (only scalar values) to crumple
> + * @recursive: true to recursively crumple nested dictionaries
Is recursive=false used outside tests in this series?
> + *
> + * Takes a flat dictionary whose keys use '.' separator to indicate
> + * nesting, and values are scalars, and crumples it into a nested
> + * structure. If the @recursive parameter is false, then only the
> + * first level of structure implied by the keys will be crumpled. If
> + * @recursive is true, then the input will be recursively crumpled to
> + * expand all levels of structure in the keys.
> + *
> + * To include a literal '.' in a key name, it must be escaped as '..'
> + *
> + * For example, an input of:
> + *
> + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
> + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
> + *
> + * will result in an output of:
> + *
> + * {
> + * 'foo': [
> + * { 'bar': 'one', 'wizz': '1' },
> + * { 'bar': 'two', 'wizz': '2' }
> + * ],
> + * }
> + *
> + * The following scenarios in the input dict will result in an
> + * error being returned:
> + *
> + * - Any values in @src are non-scalar types
> + * - If keys in @src imply that a particular level is both a
> + * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
> + * - If keys in @src imply that a particular level is a list,
> + * but the indices are non-contiguous. e.g. "foo.0.bar" and
> + * "foo.2.bar" without any "foo.1.bar" present.
> + * - If keys in @src represent list indexes, but are not in
> + * the "%zu" format. e.g. "foo.+0.bar"
> + *
> + * Returns: either a QDict or QList for the nested data structure, or NULL
> + * on error
> + */
> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-18 9:34 ` Kevin Wolf
@ 2016-10-18 15:35 ` Markus Armbruster
2016-10-19 9:25 ` Kevin Wolf
2016-10-19 13:31 ` [Qemu-devel] [Qemu-block] " Eric Blake
0 siblings, 2 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-18 15:35 UTC (permalink / raw)
To: Kevin Wolf
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
Kevin Wolf <kwolf@redhat.com> writes:
> Am 17.10.2016 um 16:50 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 13.10.2016 um 14:35 hat Markus Armbruster geschrieben:
>> >> Cc: Kevin for discussion of QemuOpts dotted key convention
>> >>
>> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
>> >>
>> >> > Currently qdict_crumple requires a totally flat QDict as its
>> >> > input. i.e. all values in the QDict must be scalar types.
>> >> >
>> >> > In order to have backwards compatibility with the OptsVisitor,
>> >> > qemu_opt_to_qdict() has a new mode where it may return a QList
>> >> > for values in the QDict, if there was a repeated key. We thus
>> >> > need to allow compound types to appear as values in the input
>> >> > dict given to qdict_crumple().
>> >> >
>> >> > To avoid confusion, we sanity check that the user has not mixed
>> >> > the old and new syntax at the same time. e.g. these are allowed
>> >> >
>> >> > foo=hello,foo=world,foo=wibble
>> >> > foo.0=hello,foo.1=world,foo.2=wibble
>> >> >
>> >> > but this is forbidden
>> >> >
>> >> > foo=hello,foo=world,foo.2=wibble
>> >>
>> >> I understand the need for foo.bar=val. It makes it possible to specify
>> >> nested dictionaries with QemuOpts.
>> >>
>> >> The case for foo.0=val is less clear. QemuOpts already supports lists,
>> >> by repeating keys. Why do we need a second, wordier way to specify
>> >> them?
>> >
>> > Probably primarily because someone didn't realise this when introducing
>> > the dotted syntax.
>>
>> Can't even blame "someone" for that; it's an obscure, underdocumented
>> feature of an interface that's collapsing under its load of obscure,
>> underdocumented features.
>>
>> On the other hand, that's not exactly a state that allows for *more*
>> obscure features.
>
> I don't really think we're introducing more obscure features here, but
> rather trying to implement a single, and rather straightforward, way as
> the standard.
>
> Dotted syntax for hierarchy has actually plenty of precedence in qemu if
> you look a bit closer (the block layer, -global, -device foo,help, even
> the bus names you mentioned below are really just flattened lists), so
> we're only making things more consistent.
>
>> > Also because flat QDicts can't represent this.
>>
>> Explain?
>
> Repeated options are parsed into QLists. If you keep it at that without
> flattening you have at least a QDict that contains a QList that contains
> simple types. This is not flat any more.
*All* options are parsed into a (non-QList) list. That's what is flat.
Only when you start crumpling things go beyond flat, and QDict/QList
come into play.
> Of course, you could argue that flat QDicts are the wrong data structure
> in the first place and instead of flatting everything we should have
> done the equivalent of qdict_crumple from the beginning, but they were
> the natural extension of what was already there and happened to work
> good enough, so this is what we're currently using.
We didn't "flatten everything", because QemuOpts is and has always been
flat to begin with. Rather we've started to crumple a few things, and
in a separate layer.
I now think this is the wrong approach. We have clearly outgrown flat
options. We should redo QemuOpts to support what we need instead of
bolting on an optional crumpling layer.
I guess I reached the place you described, just on a different path :)
>> > But even if I realised that QemuOpts support this syntax, I think we
>> > would still have to use the dotted syntax because it's explicit about
>> > the index and we need that because the list can contains dicts.
>> >
>> > Compare this:
>> >
>> > driver=quorum,
>> > child.0.driver=file,child.0.filename=disk1.img,
>> > child.1.driver=host_device,child.1.filename=/dev/sdb,
>> > child.2.driver=nbd,child.2.host=localhost
>> >
>> > And this:
>> >
>> > driver=quorum,
>> > child.driver=file,child.filename=disk1.img,
>> > child.driver=host_device,child.filename=/dev/sdb,
>> > child.driver=nbd,child.host=localhost
>>
>> Aside: both are about equally illegible, to be honest.
>
> Not sure about equally, but let's agree on "both are illegible".
>
>> > For starters, can we really trust the order in QemuOpts so that the
>> > right driver and filename are associated with each other?
>>
>> The order is trustworthy, but...
>>
>> > We would also have code to associate the third child.driver with the
>> > first child.host (because file and host_device don't have a host
>> > property). And this isn't even touching optional arguments yet. Would
>> > you really want to implement or review this?
>>
>> ... you're right, doing lists by repeating keys breaks down when
>> combined with the dotted key convention's use of repetition to do
>> structured values.
>>
>> Permit me to digress.
>>
>> QemuOpts wasn't designed for list-values keys. Doing lists by
>> repetition was clever.
>>
>> QemuOpts wasn't designed for structured values. Doing structured values
>> by a dotted key convention plus repetition was clever.
>>
>> And there's the problem: too much cleverness, not enough "this is being
>> pushed way beyond its design limits, time to replace it".
>>
>> For me, a replacement should do structured values by providing suitable
>> *value* syntax instead of hacking it into the keys:
>>
>> { "driver": "quorum",
>> "child": [ { "driver": "file", "filename": "disk1.img" },
>> { "driver": "host_device", "filename=/dev/sdb" },
>> { "driver": "nbd", "host": "localhost" } ] }
>>
>> Note how the structure is obvious. It isn't with dotted keys, not even
>> if you order them sensibly (which users inevitably won't).
>>
>> Also not that the value needs to be parsed by QemuOpts! You can't leave
>> it to the consumer of the QemuOpts, because if you did, you'd have to
>> escape the commas.
>
> Okay, so I like JSON. It's a great format for our monitor protocol. We
> even have pretty printers so that it's more or less readable as output
> for human users. However, there is one thing for which it is horrible:
> Getting user input.
>
> Yes, getting rid of the comma escaping is a first step, but typing JSON
> on the command line remains a PITA. Mostly because of the quotes (which
> you probably have to escape), but also things like the useless outer
> brackets.
As long as you don't need "'" in your JSON, you can simply enclose in
"'" and be done. Since "'" can only occur in JSON strings, and the same
strings would be present in any other syntax, any other syntax would
be pretty much the same PITA.
>> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
>> could try
>>
>> driver=quorum,
>> child=[{ driver=file, filename=disk1.img },
>> { driver=host_device, filename=/dev/sdb },
>> { driver=nbd, host=localhost } ]
>
> This looks a lot more agreeable as something that I have to type on the
> command line.
>
> What's more, this is a direct extension of the existing syntax. You
> complained about how the step from simple configurations with -hda to
> more complex ones with -drive completely changes the syntax (and rightly
> so). Going from simple QemuOpts syntax to doing JSON as soon as you get
> structured values and lists would be another similar step. In contrast,
> the new syntax you're suggesting above is a natural extension of what's
> there.
Point taken. I just don't like inventing syntax, because as a rule, way
too much syntax gets invented, and almost always badly.
>> I'd go with some existing syntax, though. The one we already use is
>> JSON.
>
> The one we already use in this context is comma separated key/value
> pairs as parsed by QemuOpts.
Which is flat, and flat doesn't cut it anymore.
If you make it non-flat with dotted key convention, the dotted key
convention becomes part of the syntax. Counts as inventing syntax in my
book.
>> Your dotted key convention requires two rules: 1. names must not look
>> like integers, and 2. names must not contain '.'.
>>
>> We can avoid rule 2 by requiring '.' to be escaped. Dan's
>> qdict_crumple() currently does that, to your surprise. Adding the
>> escaping to existing options is a compatibility break, however. So, if
>> names with '.' already exist, we can either break compatibility by
>> renaming them, or break it by requiring the '.' to be escaped.
>
> I don't think we should support any escaping and rather forbid '.'
> completely in names.
I think we should adopt QAPI's naming rules.
>> The names in question are the names declared in QemuOptDesc (easy enough
>> to find) plus any names that the code may use with empty QemuOptDesc
>> (harder). Empty QemuOptDesc are used with the following option groups:
>>
>> * "drive", qemu_drive_opts in blockdev.c
>> Also empty_opts in qemu-io.c and qemu_drive_opts in test-replication.c
>>
>> * "numa", qemu_numa_opts in numa.c
>>
>> * "device", qemu_device_opts in qdev-monitor.c
>>
>> This one pulls in qdev properties. Properties violating rule 2 exist.
>
> You posted a list of them in another email. Apart from not being
> user-configurable, I think it actually makes sense to interpret these as
> being structured.
>
> A lot of them are really array/list properties with a single entry. It
> seems we have some inconsistency there as to whether it the syntax is
> 'list.index' or 'list[index]'.
Yes.
> The former matches the dotted syntax
> rule and is used in external interfaces (e.g. in bus=...), and I seem to
> remember the latter is a newer QOM thing.
Commit 3396590 "qom: Add automatic arrayification to
object_property_add()". I tried to strangle it in the crib,
unsuccessfully:
Message-Id: <1415812130-2592-1-git-send-email-armbru@redhat.com>
http://lists.gnu.org/archive/html/qemu-devel/2014-11/msg01905.html
> Would it be too late to change the latter into the former? In other
> words, is the difference externally visible outside of qom-*, which we
> declared a non-ABI, as far as I know?
Paolo?
>> * "object", qemu_object_opts in vl.c
>> Also qemu_object_opts in qemu-img.c, qemu-io.c and qemu-nbd.c (WTF?)
>>
>> This one pulls in properties of user-creatable objects.
>>
>> * "source", qemu_soource_opts (sic!) in qemu-img.c
>>
>> * "reopen", reopen_opts in qemu-io-cmds.c
>>
>> * "file", file_opts in qemu-io.c and qemu-nbd.c
>>
>> * "machine", qemu_machine_opts in vl.c
>>
>> This one pulls in machine properties.
>>
>> * "tpmdev", qemu_tpmdev_opts in vl.c
>>
>> * "netdev", qemu_netdev_opts in net.c
>>
>> * "net", qemu_net_opts in net.c
>>
>> * "userdef", userdef_opts in tests-opts-visitor.c
>>
>> * "opts_list_03", opts_list_03 in test-qemu-opts.c
>>
>> * "acpi", qemu_acpi_opts in hw/acpi/core.c
>>
>> * "smbios", qemu_smbios_opts in smbios.c
>>
>> Empty QemuOptDesc is used this widely because QemuOpts is too primitive
>> to support the needs of its users. Unlike a QAPI schema. Just sayin'.
>
> I don't think you have to convince anyone that QemuOpts is too limited.
> But today it's still a piece that is needed to get from the command line
> to something superior like QAPI. This is the whole reason why we're
> doing all of this.
>
> Completely QAPIfication of everything and going directly from the
> command line to QAPI objects would be nice, but somehow I feel it's not
> a realistic hope to get this as a short term solution.
No argument. However, we should try hard to avoid short term hacks that
make a QAPI-based solution harder. I believe the hard part will be
retaining all the special cases that got in for convenience, or by
accident, then were repurposed for whatever. And that's why I'm
resisting additional syntactic/semantic cleverness *especially* when
it's used only sometimes. Stuff that's used only sometimes can easily
conflict with stuff used at other times.
I don't think we understand the existing cleverness sufficiently to
predict interactions with new cleverness with confidence.
>> Finding the names and assessing how they're impacted by the dotted key
>> convention is going to be a substantial chunk of work.
>
> Possibly.
>
> The other option would be to take it step by step and let every user of
> QemuOpts specify on creation whether dots are used for structure or
> should be taken literally. Then you can convert user by user and assess
> them one at a time (which is still a substantial chunk for -object and
> -device, but at least you can concentrate on only these for now).
I'm wary of protracted iterative conversion. We're better at starting
these than finishing them. The started-but-not-finished state is
exactly the state that scares me here: still more special cases
basically nobody fully understands.
> If you're concerned about compatibility issues if we find a dot in a
> name only in one of the later users: We can handle it. If you then
> stumble across an obscure "weird.name" property where the dot is really
> part of the name and not already representing structure, you can still
> artificially reinterpret it as a nested structure with a single field
> even if logically that's not what it is... Might be a bit ugly, but
> keeps it working.
Yes, that's how we'd do backward compatibility, if needed.
So, what can we do? Perhaps something like this:
* Pick a concrete syntax that supports nested stuff. Proposals on the
table so far are
1. Dotted key convention as used in the block layer
2. Array syntax as used in QOM's automatic arrayification (not a
complete solution, mentioned because it conflicts with the
previous)
3. JSON
4. JSON with ':' replaced by '=', double-quotes and the outermost pair
of braces dropped.
* Replace the QemuOpts internal representation: QDict instead of list.
* Replace the QemuOpts parser.
* Add suitable struct- and list-valued accessors.
qdict_crumple(qemu_opt_to_qdict(...), true, &errp) gets replaced by a
trivial "get the whole dictionary" operation.
Other uses of qemu_opt_to_qdict() need to be rewritten.
* "Repetition builds a list" backward compatibility. It's used in only
a few places.
* Possibly "keys with funny characters" backward compatibility.
Feels doable to me. 2.8 would be ambitious, though: soft freeze in in
less than two weeks. However, getting this series into 2.8 has started
to feel ambitious to me, too.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts Daniel P. Berrange
@ 2016-10-18 17:13 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-18 17:13 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The qobject_input_visitor_new_opts() method gains a new
> parameter to control whether it allows repeated option
> keys in the input QemuOpts or not.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/qapi/qobject-input-visitor.h | 6 ++++++
> qapi/qobject-input-visitor.c | 5 ++++-
> 2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
> index 242b767..bc5062a 100644
> --- a/include/qapi/qobject-input-visitor.h
> +++ b/include/qapi/qobject-input-visitor.h
> @@ -112,6 +112,11 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> * qobject_input_visitor_new_autocast() method. See the docs
> * of that method for further details on processing behaviour.
> *
> + * If the @permit_repeated_opts parameter is true, then the input
> + * @opts is allowed to contain repeated keys and they will be
> + * turned into a QList. If it is false, then repeated keys will
> + * result in an error being reported.
> + *
> * The returned input visitor should be released by calling
> * visit_free() when no longer required.
> */
> @@ -119,6 +124,7 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> bool autocreate_list,
> size_t autocreate_struct_levels,
> bool permit_int_ranges,
> + bool permit_repeated_opts,
> Error **errp);
>
> #endif
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index 2287d11..5a3872c 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -753,6 +753,7 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> bool autocreate_list,
> size_t autocreate_struct_levels,
> bool permit_int_ranges,
> + bool permit_repeated_opts,
> Error **errp)
> {
> QDict *pdict;
> @@ -760,7 +761,9 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> Visitor *v = NULL;
>
> pdict = qemu_opts_to_qdict(opts, NULL,
> - QEMU_OPTS_REPEAT_POLICY_LAST,
> + permit_repeated_opts ?
> + QEMU_OPTS_REPEAT_POLICY_ALL :
> + QEMU_OPTS_REPEAT_POLICY_ERROR,
> errp);
> if (!pdict) {
> goto cleanup;
@permit_repeated_opts applies to the whole @opts. However, the "repeat
to build a list" feature is actually used more selectively. For
instance, -spice wants it for "tls-channel" and "plaintext-channel", but
not for its other keys. There, the code uses qemu_opt_get() or similar,
which gets the last value.
The options visitor works the same. Internally, it stores only lists.
When the visit is for a scalar, it gets the last value from the list.
When the visit is for a list, it sets up things so that the list element
visits get the values in order.
As long as no key is list-valued, you can keep @permit_repeated_opts
false, and have duplicate keys rejected, for whatever that's worth.
As soon as you have a list-valued key, you need to set
@permit_repeated_opts to true. But then *no* duplicate keys are
rejected. If consistency with @permit_repeated_opts = false is needed,
then something else needs to detect unwanted lists and reject them.
What use cases for @permit_repeated_opts = false do you envisage?
To serve as replacement for the options visitor, this needs to treat
duplicates the same. As we saw in PATCH 12, QEMU_OPTS_REPEAT_POLICY_ALL
makes a list when the key is repeated, while the options visitor makes a
list always. I guess some other patch hides this difference.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-18 15:35 ` Markus Armbruster
@ 2016-10-19 9:25 ` Kevin Wolf
2016-10-19 9:42 ` Daniel P. Berrange
2016-10-19 13:31 ` [Qemu-devel] [Qemu-block] " Eric Blake
1 sibling, 1 reply; 109+ messages in thread
From: Kevin Wolf @ 2016-10-19 9:25 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
Am 18.10.2016 um 17:35 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
> > Of course, you could argue that flat QDicts are the wrong data structure
> > in the first place and instead of flatting everything we should have
> > done the equivalent of qdict_crumple from the beginning, but they were
> > the natural extension of what was already there and happened to work
> > good enough, so this is what we're currently using.
>
> We didn't "flatten everything", because QemuOpts is and has always been
> flat to begin with. Rather we've started to crumple a few things, and
> in a separate layer.
That's the QemuOpts point of view.
I was talking from a block layer view. There we get data in two
different formats, QMP and the command line. The first is structured,
the second is flat. We need to make both uniform before we can pass them
to the actual block layer functions.
The current code chose to flatten what we get from QMP blockdev-add and
use that throughout the block layer. We could instead have decided that
we leave the blockdev-add input as it is, crumple what we get from
QemuOpts and use nested QObjects throughout the block layer.
> I now think this is the wrong approach. We have clearly outgrown flat
> options. We should redo QemuOpts to support what we need instead of
> bolting on an optional crumpling layer.
>
> I guess I reached the place you described, just on a different path :)
Yes, I think the conclusion is easy to agree on.
> > Okay, so I like JSON. It's a great format for our monitor protocol. We
> > even have pretty printers so that it's more or less readable as output
> > for human users. However, there is one thing for which it is horrible:
> > Getting user input.
> >
> > Yes, getting rid of the comma escaping is a first step, but typing JSON
> > on the command line remains a PITA. Mostly because of the quotes (which
> > you probably have to escape), but also things like the useless outer
> > brackets.
>
> As long as you don't need "'" in your JSON, you can simply enclose in
> "'" and be done. Since "'" can only occur in JSON strings, and the same
> strings would be present in any other syntax, any other syntax would
> be pretty much the same PITA.
I've written enough scripts (qemu-iotests cases) that produce JSON with
shell variables in it, so if anything you can use "" quoting for the
shell and make use of qemu's extension that '' is accepted in JSON, too.
Anyway, the quotes aren't only nasty because of the escaping, but also
just because I have to type them.
> >> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
> >> could try
> >>
> >> driver=quorum,
> >> child=[{ driver=file, filename=disk1.img },
> >> { driver=host_device, filename=/dev/sdb },
> >> { driver=nbd, host=localhost } ]
> >
> > This looks a lot more agreeable as something that I have to type on the
> > command line.
> >
> > What's more, this is a direct extension of the existing syntax. You
> > complained about how the step from simple configurations with -hda to
> > more complex ones with -drive completely changes the syntax (and rightly
> > so). Going from simple QemuOpts syntax to doing JSON as soon as you get
> > structured values and lists would be another similar step. In contrast,
> > the new syntax you're suggesting above is a natural extension of what's
> > there.
>
> Point taken. I just don't like inventing syntax, because as a rule, way
> too much syntax gets invented, and almost always badly.
>
> >> I'd go with some existing syntax, though. The one we already use is
> >> JSON.
> >
> > The one we already use in this context is comma separated key/value
> > pairs as parsed by QemuOpts.
>
> Which is flat, and flat doesn't cut it anymore.
>
> If you make it non-flat with dotted key convention, the dotted key
> convention becomes part of the syntax. Counts as inventing syntax in my
> book.
Yes, it is. Though in the context of command line options, dotted syntax
is an invention already made, whereas JSON would be a new invention.
(Well, not completely, because for block devices we already have json: -
but that works a little different again.)
> >> Your dotted key convention requires two rules: 1. names must not look
> >> like integers, and 2. names must not contain '.'.
> >>
> >> We can avoid rule 2 by requiring '.' to be escaped. Dan's
> >> qdict_crumple() currently does that, to your surprise. Adding the
> >> escaping to existing options is a compatibility break, however. So, if
> >> names with '.' already exist, we can either break compatibility by
> >> renaming them, or break it by requiring the '.' to be escaped.
> >
> > I don't think we should support any escaping and rather forbid '.'
> > completely in names.
>
> I think we should adopt QAPI's naming rules.
Which includes what I said, so fine with me.
> > If you're concerned about compatibility issues if we find a dot in a
> > name only in one of the later users: We can handle it. If you then
> > stumble across an obscure "weird.name" property where the dot is really
> > part of the name and not already representing structure, you can still
> > artificially reinterpret it as a nested structure with a single field
> > even if logically that's not what it is... Might be a bit ugly, but
> > keeps it working.
>
> Yes, that's how we'd do backward compatibility, if needed.
>
> So, what can we do? Perhaps something like this:
>
> * Pick a concrete syntax that supports nested stuff. Proposals on the
> table so far are
>
> 1. Dotted key convention as used in the block layer
>
> 2. Array syntax as used in QOM's automatic arrayification (not a
> complete solution, mentioned because it conflicts with the
> previous)
>
> 3. JSON
>
> 4. JSON with ':' replaced by '=', double-quotes and the outermost pair
> of braces dropped.
Biased block layer view on what we ideally want to have in the end:
Anthony's QCFG proposal plus a [] syntax for lists.
This is more or less what you describe in option 4, except that the
existing dotted syntax is still supported. It is part of the ABI by now,
so we can't get rid of it anyway, and supporting it everywhere is not
only more consistent but probably also easier than making it a special
case.
The tricky part here is the {} and [] syntax for dicts and lists:
Parsing this requires a schema if you don't want to restrict string
values to not contain any of these characters. Such a restriction would
be incompatible, so we probably can't make it.
Essentially this means that we're back to "QAPIfy everything" before we
can make this work. Leaves us with implementing only the dotted syntax
subset for now, which is equally powerful, even if a lot more verbose.
> * Replace the QemuOpts internal representation: QDict instead of list.
>
> * Replace the QemuOpts parser.
>
> * Add suitable struct- and list-valued accessors.
>
> qdict_crumple(qemu_opt_to_qdict(...), true, &errp) gets replaced by a
> trivial "get the whole dictionary" operation.
>
> Other uses of qemu_opt_to_qdict() need to be rewritten.
>
> * "Repetition builds a list" backward compatibility. It's used in only
> a few places.
>
> * Possibly "keys with funny characters" backward compatibility.
>
> Feels doable to me. 2.8 would be ambitious, though: soft freeze in in
> less than two weeks. However, getting this series into 2.8 has started
> to feel ambitious to me, too.
Let's merge qdict_crumple() at least, this would allow us to add the SSH
and NBD blockdev-add support for 2.8, possibly NFS to. No command line
syntax involved at all there. I can take the patch through my tree if
nobody objects.
With -blockdev I'll have to wait for the final decision, but if we
decide to use a syntax that includes dotted syntax (possibly as a
subset), I think this series would provide the right external interface
even if there is still some infrastructure cleanup work to do
internally.
Kevin
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-19 9:25 ` Kevin Wolf
@ 2016-10-19 9:42 ` Daniel P. Berrange
0 siblings, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-19 9:42 UTC (permalink / raw)
To: Kevin Wolf
Cc: Markus Armbruster, Paolo Bonzini, Max Reitz, Andreas Färber,
qemu-block, qemu-devel
On Wed, Oct 19, 2016 at 11:25:27AM +0200, Kevin Wolf wrote:
> Am 18.10.2016 um 17:35 hat Markus Armbruster geschrieben:
> > Kevin Wolf <kwolf@redhat.com> writes:
> > > Of course, you could argue that flat QDicts are the wrong data structure
> > > in the first place and instead of flatting everything we should have
> > > done the equivalent of qdict_crumple from the beginning, but they were
> > > the natural extension of what was already there and happened to work
> > > good enough, so this is what we're currently using.
> >
> > We didn't "flatten everything", because QemuOpts is and has always been
> > flat to begin with. Rather we've started to crumple a few things, and
> > in a separate layer.
>
> That's the QemuOpts point of view.
>
> I was talking from a block layer view. There we get data in two
> different formats, QMP and the command line. The first is structured,
> the second is flat. We need to make both uniform before we can pass them
> to the actual block layer functions.
>
> The current code chose to flatten what we get from QMP blockdev-add and
> use that throughout the block layer. We could instead have decided that
> we leave the blockdev-add input as it is, crumple what we get from
> QemuOpts and use nested QObjects throughout the block layer.
I would very much like to see BlockdevOptions structs passed around
the block drivers, instead of QemuOpts - I think it'd lead to much
clearer code than the QemuOpts stuff we have today IMHO
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-18 15:35 ` Markus Armbruster
2016-10-19 9:25 ` Kevin Wolf
@ 2016-10-19 13:31 ` Eric Blake
1 sibling, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-10-19 13:31 UTC (permalink / raw)
To: Markus Armbruster, Kevin Wolf
Cc: Paolo Bonzini, Max Reitz, Andreas Färber, qemu-block, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]
On 10/18/2016 10:35 AM, Markus Armbruster wrote:
>>>> But even if I realised that QemuOpts support this syntax, I think we
>>>> would still have to use the dotted syntax because it's explicit about
>>>> the index and we need that because the list can contains dicts.
>>>>
>>>> Compare this:
>>>>
>>>> driver=quorum,
>>>> child.0.driver=file,child.0.filename=disk1.img,
>>>> child.1.driver=host_device,child.1.filename=/dev/sdb,
>>>> child.2.driver=nbd,child.2.host=localhost
>>>>
>>>> And this:
>>>>
>>>> driver=quorum,
>>>> child.driver=file,child.filename=disk1.img,
>>>> child.driver=host_device,child.filename=/dev/sdb,
>>>> child.driver=nbd,child.host=localhost
>>>
>>> Aside: both are about equally illegible, to be honest.
Furthermore, both of these currently require really long command lines.
Overnight, I was thinking about whether it would be worth improving
QemuOpts to ignore whitespace after (unquoted) commas, so you can do:
-drive 'driver=quorum,
child.0.driver=file, child.0.filename=disk1.img,
child.1.driver=host_device, child.1.filename=/dev/sdb,
child.2.driver=nbd, child.2.host=localhost
'
I think this one should be just fine - we don't have ANY keys that start
with leading whitespace, so ignoring space will still let us parse out
key names, but allow for much more legibility in the command lines. And
this is true whether we use dotted notation...
>>> If you'd rather invent syntax closer to QemuOpts than reuse JSON, you
>>> could try
>>>
>>> driver=quorum,
>>> child=[{ driver=file, filename=disk1.img },
>>> { driver=host_device, filename=/dev/sdb },
>>> { driver=nbd, host=localhost } ]
>>
>> This looks a lot more agreeable as something that I have to type on the
>> command line.
...or new syntax (where of course the new syntax has already
demonstrated that we want to support strategic ignoring of whitespace).
So I'll probably propose a patch along those lines soon, as it seems
like an orthogonal improvement.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object Daniel P. Berrange
@ 2016-10-19 16:54 ` Markus Armbruster
2017-07-10 19:30 ` Manos Pitsidianakis
1 sibling, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-19 16:54 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The current -object command line syntax only allows for
> creation of objects with scalar properties, or a list
> with a fixed scalar element type. Objects which have
> properties that are represented as structs in the QAPI
> schema cannot be created using -object.
>
> This is a design limitation of the way the OptsVisitor
> is written. It simply iterates over the QemuOpts values
> as a flat list. The support for lists is enabled by
> allowing the same key to be repeated in the opts string.
>
> The QObjectInputVisitor now has functionality that allows
> it to replace OptsVisitor, maintaining full backwards
> compatibility for previous CLI syntax, while also allowing
> use of new syntax for structs.
>
> Thus -object can now support non-scalar properties,
> for example the QMP object:
>
> {
> "execute": "object-add",
> "arguments": {
> "qom-type": "demo",
> "id": "demo0",
> "parameters": {
> "foo": [
> { "bar": "one", "wizz": "1" },
> { "bar": "two", "wizz": "2" }
> ]
> }
> }
> }
>
> Would be creatable via the CLI now using
>
> $QEMU \
> -object demo,id=demo0,\
> foo.0.bar=one,foo.0.wizz=1,\
> foo.1.bar=two,foo.1.wizz=2
>
> Notice that this syntax is intentionally compatible
> with that currently used by block drivers. NB the
> indentation seen here after line continuations should
> not be used in reality, it is just for clarity in this
> commit message.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> qapi/qobject-input-visitor.c | 2 +-
> qom/object_interfaces.c | 37 ++++-
> tests/check-qom-proplist.c | 367 ++++++++++++++++++++++++++++++++++++++++++-
> 3 files changed, 397 insertions(+), 9 deletions(-)
>
> diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> index 5a3872c..f1030d5 100644
> --- a/qapi/qobject-input-visitor.c
> +++ b/qapi/qobject-input-visitor.c
> @@ -204,7 +204,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
> *obj = NULL;
> }
>
> - if (!qobj && (qiv->struct_level < qiv->autocreate_struct_levels)) {
> + if (!qobj && (qiv->struct_level <= qiv->autocreate_struct_levels)) {
> /* Create a new dict that contains all the currently
> * unvisited items */
> if (tos) {
Uh, can you explain this hunk? The < comes from PATCH 10.
> diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
> index fdc406b..88a1e88 100644
> --- a/qom/object_interfaces.c
> +++ b/qom/object_interfaces.c
> @@ -4,7 +4,8 @@
> #include "qemu/module.h"
> #include "qapi-visit.h"
> #include "qapi/qobject-output-visitor.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qobject-input-visitor.h"
> +#include "qemu/option.h"
>
> void user_creatable_complete(Object *obj, Error **errp)
> {
> @@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
> if (local_err) {
> goto out_visit;
> }
> - visit_check_struct(v, &local_err);
> +
> + obj = user_creatable_add_type(type, id, pdict, v, &local_err);
> if (local_err) {
> goto out_visit;
> }
>
> - obj = user_creatable_add_type(type, id, pdict, v, &local_err);
> + visit_check_struct(v, &local_err);
> + if (local_err) {
> + goto out_visit;
> + }
>
> out_visit:
> visit_end_struct(v, NULL);
Can you explain why you swap the order of visit_check_struct() and
user_creatable_add_type()? It smells like a bug fix to me...
Odd: opts_check_struct() does nothing unless depth == 0. But depth is
at least 1 between opts_start_struct() and opts_end_struct(). Bug?
> @@ -114,7 +119,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
>
> assert(qdict);
> obj = object_new(type);
> - visit_start_struct(v, NULL, NULL, 0, &local_err);
> + visit_start_struct(v, "props", NULL, 0, &local_err);
> if (local_err) {
> goto out;
> }
Why this hunk?
> @@ -158,14 +163,32 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
> {
> Visitor *v;
> QDict *pdict;
> + QObject *pobj;
> Object *obj = NULL;
>
> - v = opts_visitor_new(opts);
> - pdict = qemu_opts_to_qdict(opts, NULL, QEMU_OPTS_REPEAT_POLICY_LAST,
> + pdict = qemu_opts_to_qdict(opts, NULL,
> + QEMU_OPTS_REPEAT_POLICY_ALL,
> &error_abort);
>
> - obj = user_creatable_add(pdict, v, errp);
> + pobj = qdict_crumple(pdict, true, errp);
> + if (!pobj) {
> + goto cleanup;
> + }
> + /*
> + * We need autocreate_list=true + permit_int_ranges=true
> + * in order to maintain compat with OptsVisitor creation
> + * of the 'numa' object which had an int16List property.
I can't find a "numa" object in the object-add sense, only a NumaOptions
QAPI object created by -numa. Is that what you mean?
> + *
> + * We need autocreate_structs=1, because at the CLI level
> + * we have the object type + properties in the same flat
> + * struct, even though at the QMP level they are nested.
We screwed up QMP object-add. device_add, netdev_add and object_add all
treat additional arguments as properties *except* for QMP object-add,
which has them wrapped in a JSON object instead.
Aside: if someone foolishly adds a property named "qom-type" or "id",
it'll work in QMP but not in HMP or -object.
The inconsistency complicates the code even before this patch:
* Core: user_creatable_add_type() takes properties in *two* forms: as a
visitor and as a qdict. It visits a struct with the members given by
the qdict's keys. That's its only use of the qdict.
* QMP: qmp_object_add() gets properties as a separate QDict. It creates
a QObject visitor for it, and passes it to user_creatable_add_type()
along with the QDict.
* -object: user_creatable_add_opts() gets the option argument as a
QemuOpts. It creates an options visitor for it *and* converts it to a
QDict, then passes both to user_creatable_add().
* user_creatable_add() visits a struct. It first visits the
non-property arguments "qom-type" and "id". It passes the visitor and
a shallow copy of the QDict with the non-property entries deleted to
user_creatable_add_type() to visit the remaining struct members.
* HMP: hmp_object_add() gets its arguments as a QDict. It converts it
to QemuOpts and creates an options visitor for it *boggle*. It passes
the visitor along with original QDict to user_creatable_add().
I still can't quite see what's requiring autocreate_structs. But I hope
we can get rid of it by cleaning up this mess a bit. Here's how I'd
try:
* Move visit_start_struct() and visit_end_struct() from
user_creatable_add_type() to its two callers qmp_object_add() and
user_creatable_add().
* Make user_creatable_add_type() ignore QDict keys "qom-type" and "id".
* QMP: any attempt to specify property "qom-type" or "id" now fails,
because visit_check_struct() flags them as unexpected.
* user_creatable_add(): ditch the silly QDict copying.
* hmp_object_add(): ditch the silly conversion to QemuOpts, and create a
QObject visitor instead.
> + */
> + v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
> +
> + obj = user_creatable_add(qobject_to_qdict(pobj), v, errp);
> visit_free(v);
> + qobject_decref(pobj);
> + cleanup:
> QDECREF(pdict);
> return obj;
> }
[Skipping test updates for now...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add Daniel P. Berrange
@ 2016-10-20 6:43 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-20 6:43 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The current object_add HMP syntax only allows for
> creation of objects with scalar properties, or a list
> with a fixed scalar element type. Objects which have
> properties that are represented as structs in the QAPI
> schema cannot be created using -object.
>
> This is a design limitation of the way the OptsVisitor
> is written. It simply iterates over the QemuOpts values
> as a flat list. The support for lists is enabled by
> allowing the same key to be repeated in the opts string.
>
> The QObjectInputVisitor now has functionality that allows
> it to replace OptsVisitor, maintaining full backwards
> compatibility for previous CLI syntax, while also allowing
> use of new syntax for structs.
>
> Thus -object can now support non-scalar properties,
> for example the QMP object:
>
> {
> "execute": "object-add",
> "arguments": {
> "qom-type": "demo",
> "id": "demo0",
> "parameters": {
> "foo": [
> { "bar": "one", "wizz": "1" },
> { "bar": "two", "wizz": "2" }
> ]
> }
> }
> }
>
> Would be creatable via the HMP now using
>
> object_add demo,id=demo0,\
> foo.0.bar=one,foo.0.wizz=1,\
> foo.1.bar=two,foo.1.wizz=2
>
> Notice that this syntax is intentionally compatible
> with that currently used by block drivers. NB the
> indentation seen here after line continuations should
> not be used in reality, it is just for clarity in this
> commit message.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> hmp.c | 25 +++++++++++++++++--------
> 1 file changed, 17 insertions(+), 8 deletions(-)
>
> diff --git a/hmp.c b/hmp.c
> index 336e7bf..b32d8c8 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -25,7 +25,7 @@
> #include "qemu/sockets.h"
> #include "monitor/monitor.h"
> #include "monitor/qdev.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qobject-input-visitor.h"
> #include "qapi/qmp/qerror.h"
> #include "qapi/string-output-visitor.h"
> #include "qapi/util.h"
> @@ -1696,21 +1696,30 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
> void hmp_object_add(Monitor *mon, const QDict *qdict)
> {
> Error *err = NULL;
> - QemuOpts *opts;
> Visitor *v;
> Object *obj = NULL;
> + QObject *pobj;
>
> - opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
> + pobj = qdict_crumple(qdict, true, &err);
> if (err) {
> - hmp_handle_error(mon, &err);
> - return;
> + goto cleanup;
> }
>
> - v = opts_visitor_new(opts);
> - obj = user_creatable_add(qdict, v, &err);
> + /*
> + * We need autocreate_list=true + permit_int_ranges=true
> + * in order to maintain compat with OptsVisitor creation
> + * of the 'numa' object which had an int16List property.
> + *
> + * We need autocreate_structs=1, because at the CLI level
autocreate_struct_levels=1
Same in previous patch.
> + * we have the object type + properties in the same flat
> + * struct, even though at the QMP level they are nested.
> + */
> + v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
> + obj = user_creatable_add(qobject_to_qdict(pobj), v, &err);
> visit_free(v);
> - qemu_opts_del(opts);
>
> + cleanup:
> + qobject_decref(pobj);
> if (err) {
> hmp_handle_error(mon, &err);
> }
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa Daniel P. Berrange
@ 2016-10-20 6:57 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-20 6:57 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Switch away from using OptsVisitor to parse the -numa
> argument processing. This enables use of the modern
> list syntax for specifying CPUs. e.g. the old syntax
>
> -numa node,nodeid=0,cpus=0-3,cpus=8-11,mem=107
>
> is equivalent to
>
> -numa node,nodeid=0,cpus.0=0,cpus.1=1,cpus.2=2,cpus.3=3,\
> cpus.4=8,cpus.5=9,cpus.6=10,cpus.7=11,mem=107
>
> Furthermore, the cli arg can now follow the QAPI schema
> nesting, so the above is equivalent to the canonical
> syntax:
>
> -numa type=node,data.nodeid=0,data.cpus.0=0,data.cpus.1=1,\
> data.cpus.2=2,data.cpus.3=3,data.cpus.4=8,data.cpus.5=9,\
> data.cpus.6=10,data.cpus.7=11,data.mem=107
>
> A test case is added to cover argument parsing to validate
> that both the old and new syntax is correctly handled.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> include/sysemu/numa_int.h | 11 +++++
> numa.c | 36 +++++++++-----
> stubs/Makefile.objs | 5 ++
> stubs/exec.c | 6 +++
> stubs/hostmem.c | 14 ++++++
> stubs/memory.c | 41 ++++++++++++++++
> stubs/qdev.c | 8 ++++
> stubs/vl.c | 8 ++++
> stubs/vmstate.c | 4 ++
> tests/Makefile.include | 2 +
> tests/test-numa.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++
> 11 files changed, 240 insertions(+), 11 deletions(-)
> create mode 100644 include/sysemu/numa_int.h
> create mode 100644 stubs/exec.c
> create mode 100644 stubs/hostmem.c
> create mode 100644 stubs/memory.c
> create mode 100644 stubs/qdev.c
> create mode 100644 stubs/vl.c
> create mode 100644 tests/test-numa.c
>
> diff --git a/include/sysemu/numa_int.h b/include/sysemu/numa_int.h
> new file mode 100644
> index 0000000..93160da
> --- /dev/null
> +++ b/include/sysemu/numa_int.h
> @@ -0,0 +1,11 @@
> +#ifndef SYSEMU_NUMA_INT_H
> +#define SYSEMU_NUMA_INT_H
> +
> +#include "sysemu/numa.h"
> +
> +extern int have_memdevs;
> +extern int max_numa_nodeid;
> +
> +int parse_numa(void *opaque, QemuOpts *opts, Error **errp);
> +
> +#endif
> diff --git a/numa.c b/numa.c
> index 6289f46..653ebf1 100644
> --- a/numa.c
> +++ b/numa.c
> @@ -23,14 +23,14 @@
> */
>
> #include "qemu/osdep.h"
> -#include "sysemu/numa.h"
> +#include "sysemu/numa_int.h"
> #include "exec/cpu-common.h"
> #include "qemu/bitmap.h"
> #include "qom/cpu.h"
> #include "qemu/error-report.h"
> #include "include/exec/cpu-common.h" /* for RAM_ADDR_FMT */
> #include "qapi-visit.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qobject-input-visitor.h"
> #include "hw/boards.h"
> #include "sysemu/hostmem.h"
> #include "qmp-commands.h"
> @@ -45,10 +45,10 @@ QemuOptsList qemu_numa_opts = {
> .desc = { { 0 } } /* validated with OptsVisitor */
> };
>
> -static int have_memdevs = -1;
> -static int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
> - * For all nodes, nodeid < max_numa_nodeid
> - */
> +int have_memdevs = -1;
> +int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
> + * For all nodes, nodeid < max_numa_nodeid
> + */
> int nb_numa_nodes;
> NodeInfo numa_info[MAX_NODES];
>
> @@ -189,6 +189,9 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
> if (node->has_mem) {
> uint64_t mem_size = node->mem;
> const char *mem_str = qemu_opt_get(opts, "mem");
> + if (!mem_str) {
> + mem_str = qemu_opt_get(opts, "data.mem");
> + }
> /* Fix up legacy suffix-less format */
> if (g_ascii_isdigit(mem_str[strlen(mem_str) - 1])) {
> mem_size <<= 20;
> @@ -211,16 +214,27 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
> max_numa_nodeid = MAX(max_numa_nodeid, nodenr + 1);
> }
>
> -static int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
> +int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
> {
> NumaOptions *object = NULL;
> Error *err = NULL;
> + Visitor *v;
>
> - {
> - Visitor *v = opts_visitor_new(opts);
> - visit_type_NumaOptions(v, NULL, &object, &err);
> - visit_free(v);
> + /*
> + * Needs autocreate_list=true, permit_int_ranges=true and
> + * permit_repeated_opts=true in order to support existing
> + * syntax for 'cpus' parameter which is an int list.
> + *
> + * Needs autocreate_struct_levels=1, because existing syntax
> + * used a nested struct in the QMP schema with a flat namespace
> + * in the CLI args.
Which QAPI definition(s) do you allude to here?
If it's just union NumaOptions, that's not ABI. I'd simply flatten it.
Sketch appended.
> + */
> + v = qobject_input_visitor_new_opts(opts, true, 1, true, true, &err);
> + if (err) {
> + goto end;
> }
> + visit_type_NumaOptions(v, NULL, &object, &err);
> + visit_free(v);
>
> if (err) {
> goto end;
[Skipping test updates for now...]
diff --git a/numa.c b/numa.c
index 9c09e45..792fa2a 100644
--- a/numa.c
+++ b/numa.c
@@ -227,8 +227,8 @@ static int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
}
switch (object->type) {
- case NUMA_OPTIONS_KIND_NODE:
- numa_node_parse(object->u.node.data, opts, &err);
+ case NUMA_OPTIONS_TYPE_NODE:
+ numa_node_parse(&object->u.node, opts, &err);
if (err) {
goto end;
}
diff --git a/qapi-schema.json b/qapi-schema.json
index ded1179..4838258 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4255,7 +4255,10 @@
#
# Since 2.1
##
+{ 'enum': 'NumaOptionsType', 'data': ['node'] }
{ 'union': 'NumaOptions',
+ 'base': { 'type': 'NumaOptionsType' },
+ 'discriminator': 'type',
'data': {
'node': 'NumaNodeOptions' }}
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing Daniel P. Berrange
@ 2016-10-20 7:38 ` Markus Armbruster
2016-10-20 13:43 ` [Qemu-devel] [Qemu-block] " Eric Blake
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-20 7:38 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The -net/-netdev command line parsing code uses OptsVisitor
> for parsing options to populate NetLegacy or NetDev struct
> respectively. Although those structs have nesting, the
> OptsVisitor flattens them, so we must enable compatibility
> options to auto-create structs. This allows the legacy
> syntax
>
> -net tap,id=net0,vlan=3,fd=3,script=/bin/ifup-qemu
>
> to be treated as equivalent to the modern QAPI based
> syntax
>
> -net id=net0,vlan=3,opts.type=tap,opts.data.fd=3,opts.data.script=/bin/ifup-qemu
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> net/net.c | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/net/net.c b/net/net.c
> index ec984bf..de6bf8e 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -43,7 +43,7 @@
> #include "qemu/iov.h"
> #include "qemu/main-loop.h"
> #include "qapi-visit.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qobject-input-visitor.h"
> #include "sysemu/sysemu.h"
> #include "net/filter.h"
> #include "qapi/string-output-visitor.h"
> @@ -1069,7 +1069,21 @@ int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
> void *object = NULL;
> Error *err = NULL;
> int ret = -1;
> - Visitor *v = opts_visitor_new(opts);
> + /*
> + * Needs autocreate_lists=true in order support existing
> + * syntax for list options where the bare key is repeated
> + *
> + * Needs autocreate_struct_levels=3 in order to deal with
> + * 3 level nesting in NetLegacy option args, which was
> + * exposed as a flat namespace with OptVisitor
> + */
> + Visitor *v = qobject_input_visitor_new_opts(opts, true, 3, false, true,
> + &err);
> +
> + if (err) {
> + error_propagate(errp, err);
> + return -1;
> + }
>
> {
> /* Parse convenience option format ip6-net=fec0::0[/64] */
Neither NetLegacy nor Netdev are ABI, so if I understand the problem,
perhaps I can find a way around it. Let's figure out what exactly
requires levels=3.
Your commit message shows opts.* and opts.data.* in -net. Anything
else?
I can see opts.* in the QAPI schema: it's NetLegacy member @opts, of
union type NetLegacyOptions. It's boxed on the wire, as any member of
complex type is.
I can also see opts.data.*: NetLegacyOptions is a simple union, which
wraps 'data': { ... } around the variant on the wire.
The latter wrapping is easy to avoid: make NetLegacyOptions a flat
union.
The former wrapping isn't so easy. QAPI can't unbox complex members.
The only way to include a type unboxed is making it the base type. But
base types must be struct, and @opts is a union.
However, if you take a step back you can see that the (flat) argument of
-net / -netdev is fundamentally just a (flat) union! The nesting is an
artifact of the clumsy way it's defined in the QAPI schema. Sketch
doing it less clumsily appended; look ma, no nesting! Compile-tested
only.
diff --git a/net/net.c b/net/net.c
index ec984bf..f2cbf28 100644
--- a/net/net.c
+++ b/net/net.c
@@ -984,55 +984,54 @@ static int net_client_init1(const void *object, bool is_netdev, Error **errp)
}
} else {
const NetLegacy *net = object;
- const NetLegacyOptions *opts = net->opts;
legacy.id = net->id;
netdev = &legacy;
/* missing optional values have been initialized to "all bits zero" */
name = net->has_id ? net->id : net->name;
/* Map the old options to the new flat type */
- switch (opts->type) {
- case NET_LEGACY_OPTIONS_KIND_NONE:
+ switch (net->type) {
+ case NET_LEGACY_TYPE_NONE:
return 0; /* nothing to do */
- case NET_LEGACY_OPTIONS_KIND_NIC:
+ case NET_LEGACY_TYPE_NIC:
legacy.type = NET_CLIENT_DRIVER_NIC;
- legacy.u.nic = *opts->u.nic.data;
+ legacy.u.nic = net->u.nic;
break;
- case NET_LEGACY_OPTIONS_KIND_USER:
+ case NET_LEGACY_TYPE_USER:
legacy.type = NET_CLIENT_DRIVER_USER;
- legacy.u.user = *opts->u.user.data;
+ legacy.u.user = net->u.user;
break;
- case NET_LEGACY_OPTIONS_KIND_TAP:
+ case NET_LEGACY_TYPE_TAP:
legacy.type = NET_CLIENT_DRIVER_TAP;
- legacy.u.tap = *opts->u.tap.data;
+ legacy.u.tap = net->u.tap;
break;
- case NET_LEGACY_OPTIONS_KIND_L2TPV3:
+ case NET_LEGACY_TYPE_L2TPV3:
legacy.type = NET_CLIENT_DRIVER_L2TPV3;
- legacy.u.l2tpv3 = *opts->u.l2tpv3.data;
+ legacy.u.l2tpv3 = net->u.l2tpv3;
break;
- case NET_LEGACY_OPTIONS_KIND_SOCKET:
+ case NET_LEGACY_TYPE_SOCKET:
legacy.type = NET_CLIENT_DRIVER_SOCKET;
- legacy.u.socket = *opts->u.socket.data;
+ legacy.u.socket = net->u.socket;
break;
- case NET_LEGACY_OPTIONS_KIND_VDE:
+ case NET_LEGACY_TYPE_VDE:
legacy.type = NET_CLIENT_DRIVER_VDE;
- legacy.u.vde = *opts->u.vde.data;
+ legacy.u.vde = net->u.vde;
break;
- case NET_LEGACY_OPTIONS_KIND_DUMP:
+ case NET_LEGACY_TYPE_DUMP:
legacy.type = NET_CLIENT_DRIVER_DUMP;
- legacy.u.dump = *opts->u.dump.data;
+ legacy.u.dump = net->u.dump;
break;
- case NET_LEGACY_OPTIONS_KIND_BRIDGE:
+ case NET_LEGACY_TYPE_BRIDGE:
legacy.type = NET_CLIENT_DRIVER_BRIDGE;
- legacy.u.bridge = *opts->u.bridge.data;
+ legacy.u.bridge = net->u.bridge;
break;
- case NET_LEGACY_OPTIONS_KIND_NETMAP:
+ case NET_LEGACY_TYPE_NETMAP:
legacy.type = NET_CLIENT_DRIVER_NETMAP;
- legacy.u.netmap = *opts->u.netmap.data;
+ legacy.u.netmap = net->u.netmap;
break;
- case NET_LEGACY_OPTIONS_KIND_VHOST_USER:
+ case NET_LEGACY_TYPE_VHOST_USER:
legacy.type = NET_CLIENT_DRIVER_VHOST_USER;
- legacy.u.vhost_user = *opts->u.vhost_user.data;
+ legacy.u.vhost_user = net->u.vhost_user;
break;
default:
abort();
@@ -1047,7 +1046,7 @@ static int net_client_init1(const void *object, bool is_netdev, Error **errp)
/* Do not add to a vlan if it's a nic with a netdev= parameter. */
if (netdev->type != NET_CLIENT_DRIVER_NIC ||
- !opts->u.nic.data->has_netdev) {
+ !net->u.nic.has_netdev) {
peer = net_hub_add_port(net->has_vlan ? net->vlan : 0, NULL);
}
}
diff --git a/qapi-schema.json b/qapi-schema.json
index 4838258..d863418 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2915,21 +2915,16 @@
#
# Since 1.2
##
-{ 'struct': 'NetLegacy',
- 'data': {
+{ 'enum': 'NetLegacyType',
+ 'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',
+ 'dump', 'bridge', 'netmap', 'vhost-user' ] }
+{ 'union': 'NetLegacy',
+ 'base': {
+ 'type': 'NetLegacyType',
'*vlan': 'int32',
'*id': 'str',
- '*name': 'str',
- 'opts': 'NetLegacyOptions' } }
-
-##
-# @NetLegacyOptions
-#
-# Like Netdev, but for use only by the legacy command line options
-#
-# Since 1.2
-##
-{ 'union': 'NetLegacyOptions',
+ '*name': 'str' },
+ 'discriminator': 'type',
'data': {
'none': 'NetdevNoneOptions',
'nic': 'NetLegacyNicOptions',
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing
2016-10-20 7:38 ` Markus Armbruster
@ 2016-10-20 13:43 ` Eric Blake
0 siblings, 0 replies; 109+ messages in thread
From: Eric Blake @ 2016-10-20 13:43 UTC (permalink / raw)
To: Markus Armbruster, Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 1726 bytes --]
On 10/20/2016 02:38 AM, Markus Armbruster wrote:
>> @@ -1069,7 +1069,21 @@ int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
>> void *object = NULL;
>> Error *err = NULL;
>> int ret = -1;
>> - Visitor *v = opts_visitor_new(opts);
>> + /*
>> + * Needs autocreate_lists=true in order support existing
>> + * syntax for list options where the bare key is repeated
>> + *
>> + * Needs autocreate_struct_levels=3 in order to deal with
>> + * 3 level nesting in NetLegacy option args, which was
>> + * exposed as a flat namespace with OptVisitor
>> + */
>> + Visitor *v = qobject_input_visitor_new_opts(opts, true, 3, false, true,
>> + &err);
>> +
>> + if (err) {
>> + error_propagate(errp, err);
>> + return -1;
>> + }
>>
>> {
>> /* Parse convenience option format ip6-net=fec0::0[/64] */
>
> Neither NetLegacy nor Netdev are ABI, so if I understand the problem,
> perhaps I can find a way around it. Let's figure out what exactly
> requires levels=3.
Netdev is not ABI only because we decided to NOT apply the last patch of
QAPI-fying it in 2.7 while deciding to handle the back-compat
(non-?)issues that existing netdev_add QMP command accepts both 1 and
"1", but using the Netdev type would accept only 1.
While working towards making netdev_add use QAPI, I intentionally left
NetLegacy unchanged; but since NetLegacy is solely used by the command
line, feel free to rearrange that type as long as CLI back-compat is kept.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-18 14:32 ` Markus Armbruster
@ 2016-10-20 14:11 ` Daniel P. Berrange
2016-10-21 9:58 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:11 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The qdict_flatten() method will take a dict whose elements are
> > further nested dicts/lists and flatten them by concatenating
> > keys.
> >
> > The qdict_crumple() method aims to do the reverse, taking a flat
> > qdict, and turning it into a set of nested dicts/lists. It will
> > apply nesting based on the key name, with a '.' indicating a
> > new level in the hierarchy. If the keys in the nested structure
> > are all numeric, it will create a list, otherwise it will create
> > a dict.
> >
> > If the keys are a mixture of numeric and non-numeric, or the
> > numeric keys are not in strictly ascending order, an error will
> > be reported.
> >
> > As an example, a flat dict containing
> >
> > {
> > 'foo.0.bar': 'one',
> > 'foo.0.wizz': '1',
> > 'foo.1.bar': 'two',
> > 'foo.1.wizz': '2'
> > }
> >
> > will get turned into a dict with one element 'foo' whose
> > value is a list. The list elements will each in turn be
> > dicts.
> >
> > {
> > 'foo': [
> > { 'bar': 'one', 'wizz': '1' },
> > { 'bar': 'two', 'wizz': '2' }
> > ],
> > }
> >
> > If the key is intended to contain a literal '.', then it must
> > be escaped as '..'. ie a flat dict
> >
> > {
> > 'foo..bar': 'wizz',
> > 'bar.foo..bar': 'eek',
> > 'bar.hello': 'world'
> > }
> >
> > Will end up as
> >
> > {
> > 'foo.bar': 'wizz',
> > 'bar': {
> > 'foo.bar': 'eek',
> > 'hello': 'world'
> > }
> > }
> >
> > The intent of this function is that it allows a set of QemuOpts
> > to be turned into a nested data structure that mirrors the nesting
> > used when the same object is defined over QMP.
> >
> > Reviewed-by: Eric Blake <eblake@redhat.com>
> > Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> > include/qapi/qmp/qdict.h | 1 +
> > qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
> > tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 551 insertions(+)
> >
> > diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> > index 71b8eb0..e0d24e1 100644
> > --- a/include/qapi/qmp/qdict.h
> > +++ b/include/qapi/qmp/qdict.h
> > @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
> > void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> > void qdict_array_split(QDict *src, QList **dst);
> > int qdict_array_entries(QDict *src, const char *subqdict);
> > +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
> >
> > void qdict_join(QDict *dest, QDict *src, bool overwrite);
> >
> > diff --git a/qobject/qdict.c b/qobject/qdict.c
> > index 60f158c..c38e90e 100644
> > --- a/qobject/qdict.c
> > +++ b/qobject/qdict.c
> [...]
> > +/**
> > + * qdict_crumple:
> > + * @src: the original flat dictionary (only scalar values) to crumple
> > + * @recursive: true to recursively crumple nested dictionaries
>
> Is recursive=false used outside tests in this series?
No, its not used.
It was suggested in a way earlier version by Max, but not sure if his
code uses it or not.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists
2016-10-12 9:18 ` Markus Armbruster
@ 2016-10-20 14:23 ` Daniel P. Berrange
2016-10-21 11:58 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:23 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Wed, Oct 12, 2016 at 11:18:21AM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > When converting QemuOpts to a QObject, there is no information
> > about compound types available,
>
> Yes, that's a drawback of splitting the conversion into a QemuOpts ->
> QObject part that is oblivious of types, and a QObject -> QAPI object
> part that knows the types.
>
> > so when visiting a list, the
> > corresponding QObject is not guaranteed to be a QList. We
> > therefore need to be able to auto-create a single element QList
> > from whatever type we find.
> >
> > This mode should only be enabled if you have compatibility
> > requirements for
> >
> > -arg foo=hello,foo=world
> >
> > to be treated as equivalent to the preferred syntax:
> >
> > -arg foo.0=hello,foo.1=world
>
> Not sure this is "preferred". "More powerfully warty" is probably
> closer to the truth ;)
Well, I call it "preferred" in the sense that that option syntax
directly maps to the QAPI syntax in an unambigous manner. ie
given the arg value alone "foo.0=hello,foo.1=world" you can clearly
determine that "foo" is a list. With the compat syntax you cannot
distinguish list from scalar, without knowing the QAPI schema.
> How is "-arg foo=hello,foo=world" treated if this mode isn't enabled?
The default behaviour would be that only the last key is present in
the dict, eg foo=world, and then if you tried to visit a list, the
visitor would complain that its got a QString instead of QList for
the key 'foo'.
This is related to patch 14
> What would be the drawbacks of doing this always instead of only when we
> "have compatibility requirements"?
Essentially we'd be permanently allowing 2 distinct syntaxes for
dealing with lists, for all options. I felt it desirable that we
have only a single syntax and only allow this alt syntax in the
backcompat cases.
> > diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
> > index d9269c9..d88e9f9 100644
> > --- a/qapi/qobject-input-visitor.c
> > +++ b/qapi/qobject-input-visitor.c
> > @@ -48,6 +48,10 @@ struct QObjectInputVisitor
> >
> > /* True to reject parse in visit_end_struct() if unvisited keys remain. */
> > bool strict;
> > +
> > + /* Whether we can auto-create single element lists when
> > + * encountering a non-QList type */
> > + bool autocreate_list;
> > };
> >
> > static QObjectInputVisitor *to_qiv(Visitor *v)
> > @@ -108,6 +112,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
> > assert(obj);
> > tos->obj = obj;
> > tos->qapi = qapi;
> > + qobject_incref(obj);
> >
> > if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
> > h = g_hash_table_new(g_str_hash, g_str_equal);
> > @@ -147,6 +152,7 @@ static void qobject_input_stack_object_free(StackObject *tos)
> > if (tos->h) {
> > g_hash_table_unref(tos->h);
> > }
> > + qobject_decref(tos->obj);
> >
> > g_free(tos);
> > }
>
> Can you explain the reference counting change?
Previously the stack stored a borrowed reference, since it didn't
ever need responsibility for free'ing the object when popping the
stack. This is no longer the case if you look a few lines later....
>
> > @@ -197,7 +203,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
> > QObject *qobj = qobject_input_get_object(qiv, name, true);
> > const QListEntry *entry;
> >
> > - if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
> > + if (!qobj || (!qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST)) {
>
> Long line, but I believe it'll go away when you rebase for commit
> 1382d4a.
>
> > if (list) {
> > *list = NULL;
> > }
> > @@ -206,7 +212,16 @@ static void qobject_input_start_list(Visitor *v, const char *name,
> > return;
> > }
> >
> > - entry = qobject_input_push(qiv, qobj, list, errp);
> > + if (qobject_type(qobj) != QTYPE_QLIST) {
> > + QList *tmplist = qlist_new();
> > + qlist_append_obj(tmplist, qobj);
> > + qobject_incref(qobj);
> > + entry = qobject_input_push(qiv, QOBJECT(tmplist), list, errp);
> > + QDECREF(tmplist);
... here we are storing the 'qmplist' in the stack, and so when
popping the stack, we must free that object. We thus need
the stack to always hold its own reference, so when popping
it can decref and (potentially) release the last reference.
> > + } else {
> > + entry = qobject_input_push(qiv, qobj, list, errp);
> > + }
> > +
> > if (list) {
> > if (entry) {
> > *list = g_malloc0(size);
>
> Buries autolist behavior in the middle of things. What about doing it
> first, so it's more separate?
I'm not sure I understand what you mean here ?
>
> QObjectInputVisitor *qiv = to_qiv(v);
> QObject *qobj = qobject_input_get_object_(qiv, name, true, errp);
> const QListEntry *entry;
>
> if (!qobj) {
> return;
> }
>
> + if (qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST) {
> + QList *auto_list = qlist_new();
> + qlist_append_obj(auto_list, qobj);
> + qobj = auto_list;
> + }
> +
> if (qobject_type(qobj) != QTYPE_QLIST) {
>
> I ignored reference counting here, because I don't yet understand how
> and why you're changing it.
>
> > @@ -514,7 +529,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
> > return &v->visitor;
> > }
> >
> > -Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> > +Visitor *qobject_input_visitor_new_autocast(QObject *obj,
> > + bool autocreate_list)
> > {
> > QObjectInputVisitor *v;
> >
> > @@ -539,6 +555,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> > v->visitor.optional = qobject_input_optional;
> > v->visitor.free = qobject_input_free;
> > v->strict = true;
> > + v->autocreate_list = autocreate_list;
> >
> > v->root = obj;
> > qobject_incref(obj);
> > @@ -548,6 +565,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
> >
> >
> > Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> > + bool autocreate_list,
> > Error **errp)
> > {
> > QDict *pdict;
> > @@ -564,7 +582,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
> > goto cleanup;
> > }
> >
> > - v = qobject_input_visitor_new_autocast(pobj);
> > + v = qobject_input_visitor_new_autocast(pobj,
> > + autocreate_list);
> > cleanup:
> > qobject_decref(pobj);
> > QDECREF(pdict);
> [Skipping test updates for now...]
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-10-12 15:50 ` Markus Armbruster
2016-10-12 16:03 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
@ 2016-10-20 14:28 ` Daniel P. Berrange
2016-10-21 10:58 ` Markus Armbruster
1 sibling, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:28 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Wed, Oct 12, 2016 at 05:50:41PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The traditional CLI arg syntax allows two ways to specify
> > integer lists, either one value per key, or a range of
> > values per key. eg the following are identical:
> >
> > -arg foo=5,foo=6,foo=7
> > -arg foo=5-7
> >
> > This extends the QObjectInputVisitor so that it is able
> > to parse ranges and turn them into distinct list entries.
> >
> > This means that
> >
> > -arg foo=5-7
> >
> > is treated as equivalent to
> >
> > -arg foo.0=5,foo.1=6,foo.2=7
> >
> > Edge case tests are copied from test-opts-visitor to
> > ensure identical behaviour when parsing.
> >
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> > include/qapi/qobject-input-visitor.h | 23 ++++-
> > qapi/qobject-input-visitor.c | 158 ++++++++++++++++++++++++++--
> > tests/test-qobject-input-visitor.c | 195 +++++++++++++++++++++++++++++++++--
> > 3 files changed, 360 insertions(+), 16 deletions(-)
> >
> > static void qobject_input_type_uint64(Visitor *v, const char *name,
> > @@ -366,21 +438,85 @@ static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
> > uint64_t *obj, Error **errp)
> > {
> > QObjectInputVisitor *qiv = to_qiv(v);
> > - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> > - true));
> > + QString *qstr;
> > unsigned long long ret;
> > + char *end = NULL;
> > + StackObject *tos;
> > + bool inlist = false;
> > +
> > + /* Preferentially generate values from a range, before
> > + * trying to consume another QList element */
> > + tos = QSLIST_FIRST(&qiv->stack);
> > + if (tos) {
> > + if (tos->range_val < tos->range_limit) {
> > + *obj = tos->range_val + 1;
> > + tos->range_val++;
> > + return;
> > + } else {
> > + inlist = tos->entry != NULL;
> > + }
> > + }
> >
> > + qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
> > + true));
> > if (!qstr || !qstr->string) {
> > error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> > "string");
> > return;
> > }
> >
> > - if (parse_uint_full(qstr->string, &ret, 0) < 0) {
> > + if (parse_uint(qstr->string, &ret, &end, 0) < 0) {
> > error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> > return;
> > }
> > *obj = ret;
> > +
> > + /*
> > + * If we have string that represents an integer range (5-24),
> > + * parse the end of the range and set things up so we'll process
> > + * the rest of the range before consuming another element
> > + * from the QList.
> > + */
> > + if (end && *end) {
> > + if (!qiv->permit_int_ranges) {
> > + error_setg(errp,
> > + "Integer ranges are not permitted here");
> > + return;
> > + }
> > + if (!inlist) {
> > + error_setg(errp,
> > + "Integer ranges are only permitted when "
> > + "visiting list parameters");
> > + return;
> > + }
> > + if (*end != '-') {
> > + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
> > + "a number range");
> > + return;
> > + }
> > + end++;
> > + if (parse_uint_full(end, &ret, 0) < 0) {
> > + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
> > + return;
> > + }
> > + if (*obj > ret) {
> > + error_setg(errp,
> > + "Parameter '%s' range start %" PRIu64
> > + " must be less than (or equal to) %llu",
> > + name, *obj, ret);
> > + return;
> > + }
> > + if ((ret - *obj) > (QIV_RANGE_MAX - 1)) {
> > + error_setg(errp,
> > + "Parameter '%s' range must be less than %d",
> > + name, QIV_RANGE_MAX);
> > + return;
> > + }
> > + if (*obj != ret) {
> > + tos->range_val = *obj;
> > + tos->range_limit = ret;
> > + }
> > + }
> > }
>
> Duplicates the signed code, which is sad, but I don't have better ideas.
>
> Except this one: are we actually using both the signed and the unsigned
> case now? If not, can we get rid of the one we don't use?
Out of the args that I converted in this series, I only see uint16List
used. eg for -numa and -object hostmem
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-12 17:46 ` Markus Armbruster
2016-10-13 9:21 ` Markus Armbruster
@ 2016-10-20 14:29 ` Daniel P. Berrange
2016-10-21 11:09 ` Markus Armbruster
1 sibling, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:29 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Wed, Oct 12, 2016 at 07:46:00PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > If given an option string such as
> >
> > size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
> >
> > the qemu_opts_to_qdict() method will currently overwrite
> > the values for repeated option keys, so only the last
> > value is in the returned dict:
> >
> > size=QString("1024")
> > nodes=QString("1-2")
> > policy=QString("bind")
> >
> > With this change the caller can optionally ask for all
> > the repeated values to be stored in a QList. In the
> > above example that would result in 'nodes' being a
> > QList, so the returned dict would contain
> >
> > size=QString("1024")
> > nodes=QList([QString("10"),
> > QString("4-5"),
> > QString("1-2")])
> > policy=QString("bind")
> >
> > Note that the conversion has no way of knowing whether
> > any given key is expected to be a list upfront - it can
> > only figure that out when seeing the first duplicated
> > key. Thus the caller has to be prepared to deal with the
> > fact that if a key 'foo' is a list, then the returned
> > qdict may contain either a QString or a QList for the
> > key 'foo'.
> >
> > In a third mode, it is possible to ask for repeated
> > options to be reported as an error, rather than silently
> > dropping all but the last one.
>
> To serve as a replacement for the options visitor, this needs to be able
> to behave exactly the same together with a suitably hacked up QObject
> input visitor. Before I dive into the actual patch, let me summarize
> QemuOpts and options visitor behavior.
>
> Warning, this is going to get ugly.
>
> QemuOpts faithfully represents a key=value,... string as a list of
> QemuOpt. Each QemuOpt represents one key=value. They are in the same
> order. If key occurs multiple times in the string, it occurs just the
> same in the list.
>
> *Except* key "id" is special: it's stored outside the list, and all but
> the first one are silently ignored.
>
> Most users only ever get the last value of a key. Any non-last
> key=value are silently ignored.
>
> We actually exploit this behavior to do defaults, by *prepending* them
> to the list. See the use of qemu_opts_set_defaults() in main().
>
> A few users get all values of keys (other than key "id"):
>
> * -device, in qdev_device_add() with callback set_property().
>
> We first get "driver" and "bus" normally (silently ignoring non-last
> values, as usual). All other keys are device properties. To set
> them, we get all (key, value), ignore keys "driver" and "bus", and set
> the rest. If a key occurs multiple times, it gets set multiple times.
> This effectively ignores all but the last one, silently.
>
> * -semihosting-config, in main() with callback add_semihosting_arg().
>
> We first get a bunch of keys normally. Key "arg" is special: it may
> be repeated to build a list. To implement that, we get all (key,
> value), ignore keys other than "arg", and accumulate the values.
>
> * -machine & friends, in main() with callback machine_set_property()
>
> Similar to -device, only for machines, with "type" instead of "driver"
> and "bus".
>
> * -spice, in qemu_spice_init() with callback add_channel()
>
> Keys "tls-channel" and "plaintext-channel" may be used repeated to
> specify multiple channels. To implement that, we get all (key,
> value), ignore keys other than "tls-channel" and "plaintext-channel",
> and set up a channel for each of the others.
>
> * -writeconfig, in config_write_opts() with callback config_write_opt()
>
> We write out all keys in order.
>
> * The options visitor, in opts_start_struct()
>
> We convert the list of (key, value) to a hash table of (key, list of
> values). Most of the time, the list of values has exactly one
> element.
>
> When the visitor's user asks for a scalar, we return the last element
> of the list of values, in lookup_scalar().
>
> When the user asks for list elements, we return the elements of the
> list of values in order, in opts_next_list(), or if there are none,
> the empty list in opts_start_list().
>
> Unlike the options visitor, this patch (judging from your description)
> makes a list only when keys are repeated. The QObject visitor will have
> to cope with finding both scalars and lists. When it finds a scalar,
> but needs a list, it'll have to wrap it in a list (PATCH 09, I think).
> When it finds a list, but needs a scalar, it'll have to fish it out of
> the list (where is that?).
If my code finds a list but wants a scalar, it is reporting an
error.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-13 8:31 ` Markus Armbruster
@ 2016-10-20 14:37 ` Daniel P. Berrange
2016-10-21 11:28 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:37 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Thu, Oct 13, 2016 at 10:31:37AM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > If given an option string such as
> >
> > size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
> >
> > the qemu_opts_to_qdict() method will currently overwrite
> > the values for repeated option keys, so only the last
> > value is in the returned dict:
> >
> > size=QString("1024")
> > nodes=QString("1-2")
> > policy=QString("bind")
> >
> > With this change the caller can optionally ask for all
> > the repeated values to be stored in a QList. In the
> > above example that would result in 'nodes' being a
> > QList, so the returned dict would contain
> >
> > size=QString("1024")
> > nodes=QList([QString("10"),
> > QString("4-5"),
> > QString("1-2")])
> > policy=QString("bind")
> >
> > Note that the conversion has no way of knowing whether
> > any given key is expected to be a list upfront - it can
> > only figure that out when seeing the first duplicated
> > key. Thus the caller has to be prepared to deal with the
> > fact that if a key 'foo' is a list, then the returned
> > qdict may contain either a QString or a QList for the
> > key 'foo'.
>
> Actually three cases, not two:
>
> 0. qdict does not contain the key means empty list.
>
> 1. qdict contains the key with a QString value means list of one
> element.
>
> 2. qdict contains the key with a QList value means list of more than one
> element.
>
> I consider this weird. However, it's usefully weird with at least your
> QObject input visitor.
>
> > In a third mode, it is possible to ask for repeated
> > options to be reported as an error, rather than silently
> > dropping all but the last one.
>
> Got users for this policy in the pipeline?
I in fact used it in the QObjectInputVisitor, when the
autocreate_list is not set.
I guess strictly speaking this is not back-compatible
if someone is passing repeated keys, but I judged that
rather than silently ignoring this incorrect usage, it
was acceptable to report an error.
> > QTAILQ_FOREACH(opt, &opts->head, next) {
> > val = QOBJECT(qstring_from_str(opt->str));
> > - qdict_put_obj(qdict, opt->name, val);
> > +
> > + if (qdict_haskey(ret, opt->name)) {
> > + switch (repeatPolicy) {
> > + case QEMU_OPTS_REPEAT_POLICY_ERROR:
> > + error_setg(errp, "Option '%s' appears more than once",
> > + opt->name);
> > + qobject_decref(val);
> > + if (!qdict) {
> > + QDECREF(ret);
> > + }
> > + return NULL;
> > +
> > + case QEMU_OPTS_REPEAT_POLICY_ALL:
> > + prevval = qdict_get(ret, opt->name);
> > + if (qobject_type(prevval) == QTYPE_QLIST) {
> > + /* Already identified this key as a list */
> > + list = qobject_to_qlist(prevval);
> > + } else {
> > + /* Replace current scalar with a list */
> > + list = qlist_new();
> > + qobject_incref(prevval);
>
> Where is this reference decremented?
'prevval' is a borrowed reference from 'ret', against the
key opt->name.
qdict_put_obj decrements the reference we borrowed
from ret against the key opt->name.
qlist_append_obj() takes ownership of the reference
it is passed, so we must qobject_incref() to avoid
qdict_put_obj free'ing prevval.
When we call qdict_put_obj() we're replacing the value
currently associted
>
> > + qlist_append_obj(list, prevval);
> > + qdict_put_obj(ret, opt->name, QOBJECT(list));
> > + }
> > + qlist_append_obj(list, val);
> > + break;
> > +
> > + case QEMU_OPTS_REPEAT_POLICY_LAST:
> > + /* Just discard previously set value */
> > + qdict_put_obj(ret, opt->name, val);
> > + break;
> > + }
> > + } else {
> > + qdict_put_obj(ret, opt->name, val);
> > + }
>
> A possible alternative:
>
> val = QOBJECT(qstring_from_str(opt->str));
>
> if (qdict_haskey(ret, opt->name)) {
> switch (repeatPolicy) {
> case QEMU_OPTS_REPEAT_POLICY_ERROR:
> error_setg(errp, "Option '%s' appears more than once",
> opt->name);
> qobject_decref(val);
> if (!qdict) {
> QDECREF(ret);
> }
> return NULL;
>
> case QEMU_OPTS_REPEAT_POLICY_ALL:
> prevval = qdict_get(ret, opt->name);
> if (qobject_type(prevval) == QTYPE_QLIST) {
> /* Already identified this key as a list */
> list = qobject_to_qlist(prevval);
> } else {
> /* Replace current scalar with a list */
> list = qlist_new();
> qobject_incref(prevval);
> qlist_append_obj(list, prevval);
> }
> qlist_append_obj(list, val);
> val = QOBJECT(list);
> break;
>
> case QEMU_OPTS_REPEAT_POLICY_LAST:
> break;
> }
> }
> qdict_put_obj(ret, opt->name, val);
>
> This shows the common part of the behavior more clearly. Matter of
> taste, you get to use your artistic license.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values
2016-10-13 12:35 ` Markus Armbruster
2016-10-13 14:46 ` Kevin Wolf
@ 2016-10-20 14:46 ` Daniel P. Berrange
1 sibling, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-20 14:46 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini,
Andreas Färber, Kevin Wolf
On Thu, Oct 13, 2016 at 02:35:38PM +0200, Markus Armbruster wrote:
> Cc: Kevin for discussion of QemuOpts dotted key convention
>
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > Currently qdict_crumple requires a totally flat QDict as its
> > input. i.e. all values in the QDict must be scalar types.
> >
> > In order to have backwards compatibility with the OptsVisitor,
> > qemu_opt_to_qdict() has a new mode where it may return a QList
> > for values in the QDict, if there was a repeated key. We thus
> > need to allow compound types to appear as values in the input
> > dict given to qdict_crumple().
> >
> > To avoid confusion, we sanity check that the user has not mixed
> > the old and new syntax at the same time. e.g. these are allowed
> >
> > foo=hello,foo=world,foo=wibble
> > foo.0=hello,foo.1=world,foo.2=wibble
> >
> > but this is forbidden
> >
> > foo=hello,foo=world,foo.2=wibble
>
> I understand the need for foo.bar=val. It makes it possible to specify
> nested dictionaries with QemuOpts.
>
> The case for foo.0=val is less clear. QemuOpts already supports lists,
> by repeating keys. Why do we need a second, wordier way to specify
> them?
Two reasons I did this. First blockdev already uses this foo.0=val
syntax, and I wanted to be compatible with blockdev, so it could be
converted to use this new code.
Second, using foo.0 syntax means that you can unambigously determine
whether a key is going to be a scalar or a list. This lets the
qdict_crumple() method convert the QemuOpts to a QDict without
needing to know anything about the QAPI schema.
Of course I later had to add hacks to the visitor to cope with
the bare repeated key syntax, so I lost some of that benefit.
Personally I still prefer the unambiguous syntax as it lets us
give clear error messages when users do unexpected things, instead
of say, silently ignoring all but the last key.
> Note that this second way creates entirely new failure modes and
> restrictions. Let me show using an example derived from one in
> qdict_crumple()'s contract:
>
> foo.0.bar=bla,foo.eek.bar=blubb
>
> Without the dotted key convention, this is perfectly fine: key
> "foo.0.bar" has the single value "bla", and key "foo.eek.bar" has
> the single value "blubb". Equivalent JSON would be
>
> { "foo.0.bar": "bla", "foo.eek.bar": "blubb" }
>
> With just the struct convention, it's still fine: it obviously means
> the same as JSON
>
> { "foo": { "0": { "bar": "bla" }, "eek": { "bar": "blubb" } } }
>
> Adding the list convention makes it invalid. It also outlaws a
> bunch of keys that would be just fine in JSON, namely any that get
> recognized as list index. Raise your hand if you're willing to bet
> real money on your predictions of what will be recognized as list
> index, without looking at the code. I'm not.
>
> I'm afraid I have growing doubts regarding the QemuOpts dotted key
> convention in general.
>
> The convention makes '.' a special character in keys, but only
> sometimes. If the key gets consumed by something that uses dotted key
> convention, '.' is special, and to get a non-special '.', you need to
> escape it by doubling. Else, it's not.
>
> Since the same key can be used differently by different code, the same
> '.' could in theory be both special and non-special. In practice, this
> would be madness.
>
> Adopting the dotted key convention for an existing QemuOpts option, say
> -object [PATCH 15], *breaks* existing command line usage of keys
> containing '.', because you now have to escape the '.'. Dan, I'm afraid
> you need to show that no such keys exist, or if they exist they don't
> matter.
I checked the things that I converted (eg -net, -object, -numa, etc),
but I didn't check -device since that's processed using different code.
>
> I know we have keys containing '.' elsewhere, e.g. device "macio-ide"
> property "ide.0". Our chronic inability to consistently restrict names
> in ABI to something sane is beyond foolish.
>
> It's probably too late to back out the dotted key convention completely.
> Kevin?
>
> Can we still back out the list part of the convention, and use repeated
> keys instead?
>
> If we're stuck with some form of the dotted key convention, can we at
> least make it a more integral part of QemuOpts rather than something
> bolted on as an afterthought? Here's my thinking on how that might be
> done:
The only issue with dropping the dotted list convention is compat
with the block layer code - we couldn't easily use this new visitor
logic to turn -drive into a QAPI BlockOptions object. Kevin's new
-blockdev arg would potentially be ok with it since its a new arg,
but IIUC, we would have to do some cleanup inside various block
driver impls, since block layer doesn't use the QAPI objects
internally - they all get converted back into QemuOpts :-(
>
> * Have a QemuOptsList flag @flat.
>
> * If @flat, QemuOpts behaves as it always has: the special characters
> are ',' and '=', and parsing a key=value,... string produces a list
> where each element represents one key=value from the string, in the
> same order.
>
> * If not @flat, '.' becomes an additional special character, and parsing
> a key=value,... string produces a dictionary, similar to the one you
> get now by converting with qemu_opts_to_qdict() and filtering through
> qdict_crumple().
>
> The difference to now is that you either always crumple, or not at all,
> and the meaning of '.' is unambiguous.
>
> I wish we had refrained from saddling QemuOpts with even more magic.
> Compared to this swamp, use of JSON on the command line looks rather
> appealing to me.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-20 14:11 ` Daniel P. Berrange
@ 2016-10-21 9:58 ` Markus Armbruster
2016-10-21 18:31 ` Max Reitz
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 9:58 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > The qdict_flatten() method will take a dict whose elements are
>> > further nested dicts/lists and flatten them by concatenating
>> > keys.
>> >
>> > The qdict_crumple() method aims to do the reverse, taking a flat
>> > qdict, and turning it into a set of nested dicts/lists. It will
>> > apply nesting based on the key name, with a '.' indicating a
>> > new level in the hierarchy. If the keys in the nested structure
>> > are all numeric, it will create a list, otherwise it will create
>> > a dict.
>> >
>> > If the keys are a mixture of numeric and non-numeric, or the
>> > numeric keys are not in strictly ascending order, an error will
>> > be reported.
>> >
>> > As an example, a flat dict containing
>> >
>> > {
>> > 'foo.0.bar': 'one',
>> > 'foo.0.wizz': '1',
>> > 'foo.1.bar': 'two',
>> > 'foo.1.wizz': '2'
>> > }
>> >
>> > will get turned into a dict with one element 'foo' whose
>> > value is a list. The list elements will each in turn be
>> > dicts.
>> >
>> > {
>> > 'foo': [
>> > { 'bar': 'one', 'wizz': '1' },
>> > { 'bar': 'two', 'wizz': '2' }
>> > ],
>> > }
>> >
>> > If the key is intended to contain a literal '.', then it must
>> > be escaped as '..'. ie a flat dict
>> >
>> > {
>> > 'foo..bar': 'wizz',
>> > 'bar.foo..bar': 'eek',
>> > 'bar.hello': 'world'
>> > }
>> >
>> > Will end up as
>> >
>> > {
>> > 'foo.bar': 'wizz',
>> > 'bar': {
>> > 'foo.bar': 'eek',
>> > 'hello': 'world'
>> > }
>> > }
>> >
>> > The intent of this function is that it allows a set of QemuOpts
>> > to be turned into a nested data structure that mirrors the nesting
>> > used when the same object is defined over QMP.
>> >
>> > Reviewed-by: Eric Blake <eblake@redhat.com>
>> > Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> > ---
>> > include/qapi/qmp/qdict.h | 1 +
>> > qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>> > tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>> > 3 files changed, 551 insertions(+)
>> >
>> > diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>> > index 71b8eb0..e0d24e1 100644
>> > --- a/include/qapi/qmp/qdict.h
>> > +++ b/include/qapi/qmp/qdict.h
>> > @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>> > void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>> > void qdict_array_split(QDict *src, QList **dst);
>> > int qdict_array_entries(QDict *src, const char *subqdict);
>> > +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>> >
>> > void qdict_join(QDict *dest, QDict *src, bool overwrite);
>> >
>> > diff --git a/qobject/qdict.c b/qobject/qdict.c
>> > index 60f158c..c38e90e 100644
>> > --- a/qobject/qdict.c
>> > +++ b/qobject/qdict.c
>> [...]
>> > +/**
>> > + * qdict_crumple:
>> > + * @src: the original flat dictionary (only scalar values) to crumple
>> > + * @recursive: true to recursively crumple nested dictionaries
>>
>> Is recursive=false used outside tests in this series?
>
> No, its not used.
>
> It was suggested in a way earlier version by Max, but not sure if his
> code uses it or not.
In general, I prefer features to be added right before they're used, and
I really dislike adding features "just in case". YAGNI.
Max, do you actually need this one? If yes, please explain your use
case.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor
2016-10-07 14:16 ` Daniel P. Berrange
@ 2016-10-21 10:52 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 10:52 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Fri, Oct 07, 2016 at 03:59:07PM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > Allow tracing of the operation of visitors
>> >
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> > ---
>> > Makefile.objs | 1 +
>> > qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
>> > qapi/trace-events | 33 +++++++++++++++++++++++++++++++++
>> > 3 files changed, 61 insertions(+)
>> > create mode 100644 qapi/trace-events
>> >
>> > diff --git a/Makefile.objs b/Makefile.objs
>> > index a8e0224..b3e8aef 100644
>> > --- a/Makefile.objs
>> > +++ b/Makefile.objs
>> > @@ -160,3 +160,4 @@ trace-events-y += target-s390x/trace-events
>> > trace-events-y += target-ppc/trace-events
>> > trace-events-y += qom/trace-events
>> > trace-events-y += linux-user/trace-events
>> > +trace-events-y += qapi/trace-events
>> > diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
>> > index 55f5876..bfcb276 100644
>> > --- a/qapi/qapi-visit-core.c
>> > +++ b/qapi/qapi-visit-core.c
>> > @@ -19,10 +19,12 @@
>> > #include "qapi/qmp/qerror.h"
>> > #include "qapi/visitor.h"
>> > #include "qapi/visitor-impl.h"
>> > +#include "trace.h"
>> >
>> > void visit_complete(Visitor *v, void *opaque)
>> > {
>> > assert(v->type != VISITOR_OUTPUT || v->complete);
>> > + trace_visit_complete(v, opaque);
>>
>> Trace after or before checking the precondition? Preferences, anyone?
>
> I'm ambivalent, as the assert will crash you either way so
> whether you get a trace event just before the crash seems
> mostly irrelevant to me - the abort stack trace is what
> you'll use to diagnose the crash.
I have a slight preference for trace first, because trace is one call,
while precondition checking can be several. However, grep shows neither
order is prevalent. Not worth a respin.
>> > if (v->complete) {
>> > v->complete(v, opaque);
>> > }
>> > @@ -30,6 +32,7 @@ void visit_complete(Visitor *v, void *opaque)
>> >
>> > void visit_free(Visitor *v)
>> > {
>> > + trace_visit_free(v);
>> > if (v) {
>> > v->free(v);
>> > }
>> > @@ -40,6 +43,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
>> > {
>> > Error *err = NULL;
>> >
>> > + trace_visit_start_struct(v, name, obj, size);
>> > if (obj) {
>> > assert(size);
>> > assert(!(v->type & VISITOR_OUTPUT) || *obj);
>> > @@ -53,6 +57,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
>> >
>> > void visit_check_struct(Visitor *v, Error **errp)
>> > {
>> > + trace_visit_check_struct(v);
>> > if (v->check_struct) {
>> > v->check_struct(v, errp);
>> > }
>> > @@ -60,6 +65,7 @@ void visit_check_struct(Visitor *v, Error **errp)
>> >
>> > void visit_end_struct(Visitor *v, void **obj)
>> > {
>> > + trace_visit_end_struct(v, obj);
>> > v->end_struct(v, obj);
>> > }
>> >
>> > @@ -69,6 +75,7 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
>> > Error *err = NULL;
>> >
>> > assert(!list || size >= sizeof(GenericList));
>> > + trace_visit_start_list(v, name, list, size);
>> > v->start_list(v, name, list, size, &err);
>> > if (list && (v->type & VISITOR_INPUT)) {
>> > assert(!(err && *list));
>> > @@ -79,11 +86,13 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
>> > GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
>> > {
>> > assert(tail && size >= sizeof(GenericList));
>> > + trace_visit_next_list(v, tail, size);
>> > return v->next_list(v, tail, size);
>> > }
>> >
>> > void visit_end_list(Visitor *v, void **obj)
>> > {
>> > + trace_visit_end_list(v, obj);
>> > v->end_list(v, obj);
>> > }
>> >
>> > @@ -95,6 +104,7 @@ void visit_start_alternate(Visitor *v, const char *name,
>> >
>> > assert(obj && size >= sizeof(GenericAlternate));
>> > assert(!(v->type & VISITOR_OUTPUT) || *obj);
>> > + trace_visit_start_alternate(v, name, obj, size, promote_int);
>> > if (v->start_alternate) {
>> > v->start_alternate(v, name, obj, size, promote_int, &err);
>> > }
>> > @@ -106,6 +116,7 @@ void visit_start_alternate(Visitor *v, const char *name,
>> >
>> > void visit_end_alternate(Visitor *v, void **obj)
>> > {
>> > + trace_visit_end_alternate(v, obj);
>> > if (v->end_alternate) {
>> > v->end_alternate(v, obj);
>> > }
>> > @@ -113,6 +124,7 @@ void visit_end_alternate(Visitor *v, void **obj)
>> >
>> > bool visit_optional(Visitor *v, const char *name, bool *present)
>> > {
>> > + trace_visit_optional(v, name, present);
>> > if (v->optional) {
>> > v->optional(v, name, present);
>> > }
>> > @@ -127,6 +139,7 @@ bool visit_is_input(Visitor *v)
>> > void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
>> > {
>> > assert(obj);
>> > + trace_visit_type_int(v, name, obj);
>> > v->type_int64(v, name, obj, errp);
>> > }
>> >
>> > @@ -151,6 +164,7 @@ void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
>> > Error **errp)
>> > {
>> > uint64_t value = *obj;
>> > + trace_visit_type_uint8(v, name, obj);
>> > visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
>> > *obj = value;
>> > }
>>
>> Putting the trace in the middle of the "value = *obj;
>> visit_type_uintN(v, &value, ...); *obj = value" pattern makes it less
>> visible.
>>
>> Preserving the pattern requires replacing the initializer by an
>> assignment:
>>
>> uint64_t value;
>>
>> trace_visit_type_uint8(v, name, obj);
>> value = *obj;
>> visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
>> *obj = value;
>>
>> Looks slightly more legible to me. What do you think?
>
> I'm fine with whatever you prefer.
I can touch up this one on commit.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor
2016-10-20 14:28 ` [Qemu-devel] " Daniel P. Berrange
@ 2016-10-21 10:58 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 10:58 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Wed, Oct 12, 2016 at 05:50:41PM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > The traditional CLI arg syntax allows two ways to specify
>> > integer lists, either one value per key, or a range of
>> > values per key. eg the following are identical:
>> >
>> > -arg foo=5,foo=6,foo=7
>> > -arg foo=5-7
>> >
>> > This extends the QObjectInputVisitor so that it is able
>> > to parse ranges and turn them into distinct list entries.
>> >
>> > This means that
>> >
>> > -arg foo=5-7
>> >
>> > is treated as equivalent to
>> >
>> > -arg foo.0=5,foo.1=6,foo.2=7
>> >
>> > Edge case tests are copied from test-opts-visitor to
>> > ensure identical behaviour when parsing.
>> >
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> > ---
>> > include/qapi/qobject-input-visitor.h | 23 ++++-
>> > qapi/qobject-input-visitor.c | 158 ++++++++++++++++++++++++++--
>> > tests/test-qobject-input-visitor.c | 195 +++++++++++++++++++++++++++++++++--
>> > 3 files changed, 360 insertions(+), 16 deletions(-)
>> >
>
>> > static void qobject_input_type_uint64(Visitor *v, const char *name,
>> > @@ -366,21 +438,85 @@ static void qobject_input_type_uint64_autocast(Visitor *v, const char *name,
>> > uint64_t *obj, Error **errp)
>> > {
>> > QObjectInputVisitor *qiv = to_qiv(v);
>> > - QString *qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>> > - true));
>> > + QString *qstr;
>> > unsigned long long ret;
>> > + char *end = NULL;
>> > + StackObject *tos;
>> > + bool inlist = false;
>> > +
>> > + /* Preferentially generate values from a range, before
>> > + * trying to consume another QList element */
>> > + tos = QSLIST_FIRST(&qiv->stack);
>> > + if (tos) {
>> > + if (tos->range_val < tos->range_limit) {
>> > + *obj = tos->range_val + 1;
>> > + tos->range_val++;
>> > + return;
>> > + } else {
>> > + inlist = tos->entry != NULL;
>> > + }
>> > + }
>> >
>> > + qstr = qobject_to_qstring(qobject_input_get_object(qiv, name,
>> > + true));
>> > if (!qstr || !qstr->string) {
>> > error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>> > "string");
>> > return;
>> > }
>> >
>> > - if (parse_uint_full(qstr->string, &ret, 0) < 0) {
>> > + if (parse_uint(qstr->string, &ret, &end, 0) < 0) {
>> > error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
>> > return;
>> > }
>> > *obj = ret;
>> > +
>> > + /*
>> > + * If we have string that represents an integer range (5-24),
>> > + * parse the end of the range and set things up so we'll process
>> > + * the rest of the range before consuming another element
>> > + * from the QList.
>> > + */
>> > + if (end && *end) {
>> > + if (!qiv->permit_int_ranges) {
>> > + error_setg(errp,
>> > + "Integer ranges are not permitted here");
>> > + return;
>> > + }
>> > + if (!inlist) {
>> > + error_setg(errp,
>> > + "Integer ranges are only permitted when "
>> > + "visiting list parameters");
>> > + return;
>> > + }
>> > + if (*end != '-') {
>> > + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
>> > + "a number range");
>> > + return;
>> > + }
>> > + end++;
>> > + if (parse_uint_full(end, &ret, 0) < 0) {
>> > + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name, "a number");
>> > + return;
>> > + }
>> > + if (*obj > ret) {
>> > + error_setg(errp,
>> > + "Parameter '%s' range start %" PRIu64
>> > + " must be less than (or equal to) %llu",
>> > + name, *obj, ret);
>> > + return;
>> > + }
>> > + if ((ret - *obj) > (QIV_RANGE_MAX - 1)) {
>> > + error_setg(errp,
>> > + "Parameter '%s' range must be less than %d",
>> > + name, QIV_RANGE_MAX);
>> > + return;
>> > + }
>> > + if (*obj != ret) {
>> > + tos->range_val = *obj;
>> > + tos->range_limit = ret;
>> > + }
>> > + }
>> > }
>>
>> Duplicates the signed code, which is sad, but I don't have better ideas.
>>
>> Except this one: are we actually using both the signed and the unsigned
>> case now? If not, can we get rid of the one we don't use?
>
> Out of the args that I converted in this series, I only see uint16List
> used. eg for -numa and -object hostmem
Let's drop the signed copy then.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-20 14:29 ` Daniel P. Berrange
@ 2016-10-21 11:09 ` Markus Armbruster
2016-10-21 11:14 ` Daniel P. Berrange
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 11:09 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Wed, Oct 12, 2016 at 07:46:00PM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > If given an option string such as
>> >
>> > size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>> >
>> > the qemu_opts_to_qdict() method will currently overwrite
>> > the values for repeated option keys, so only the last
>> > value is in the returned dict:
>> >
>> > size=QString("1024")
>> > nodes=QString("1-2")
>> > policy=QString("bind")
>> >
>> > With this change the caller can optionally ask for all
>> > the repeated values to be stored in a QList. In the
>> > above example that would result in 'nodes' being a
>> > QList, so the returned dict would contain
>> >
>> > size=QString("1024")
>> > nodes=QList([QString("10"),
>> > QString("4-5"),
>> > QString("1-2")])
>> > policy=QString("bind")
>> >
>> > Note that the conversion has no way of knowing whether
>> > any given key is expected to be a list upfront - it can
>> > only figure that out when seeing the first duplicated
>> > key. Thus the caller has to be prepared to deal with the
>> > fact that if a key 'foo' is a list, then the returned
>> > qdict may contain either a QString or a QList for the
>> > key 'foo'.
>> >
>> > In a third mode, it is possible to ask for repeated
>> > options to be reported as an error, rather than silently
>> > dropping all but the last one.
>>
>> To serve as a replacement for the options visitor, this needs to be able
>> to behave exactly the same together with a suitably hacked up QObject
>> input visitor. Before I dive into the actual patch, let me summarize
>> QemuOpts and options visitor behavior.
[...]
>> Unlike the options visitor, this patch (judging from your description)
>> makes a list only when keys are repeated. The QObject visitor will have
>> to cope with finding both scalars and lists. When it finds a scalar,
>> but needs a list, it'll have to wrap it in a list (PATCH 09, I think).
>> When it finds a list, but needs a scalar, it'll have to fish it out of
>> the list (where is that?).
>
> If my code finds a list but wants a scalar, it is reporting an
> error.
I'm afraid that breaks things like
--numa node,cpus=0,cpus=2,mem=512,nodeid=0,nodeid=0
The options visitor inteprets cpus=0,cpus=2 as a list [0,2] (because
NumaNodeOptions member @cpus has a list type) and nodeid=0,nodid=0 as
integer 0 (because member @nodeid has a scalar type).
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-21 11:09 ` Markus Armbruster
@ 2016-10-21 11:14 ` Daniel P. Berrange
0 siblings, 0 replies; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-21 11:14 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
On Fri, Oct 21, 2016 at 01:09:07PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > On Wed, Oct 12, 2016 at 07:46:00PM +0200, Markus Armbruster wrote:
> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>
> >> > If given an option string such as
> >> >
> >> > size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
> >> >
> >> > the qemu_opts_to_qdict() method will currently overwrite
> >> > the values for repeated option keys, so only the last
> >> > value is in the returned dict:
> >> >
> >> > size=QString("1024")
> >> > nodes=QString("1-2")
> >> > policy=QString("bind")
> >> >
> >> > With this change the caller can optionally ask for all
> >> > the repeated values to be stored in a QList. In the
> >> > above example that would result in 'nodes' being a
> >> > QList, so the returned dict would contain
> >> >
> >> > size=QString("1024")
> >> > nodes=QList([QString("10"),
> >> > QString("4-5"),
> >> > QString("1-2")])
> >> > policy=QString("bind")
> >> >
> >> > Note that the conversion has no way of knowing whether
> >> > any given key is expected to be a list upfront - it can
> >> > only figure that out when seeing the first duplicated
> >> > key. Thus the caller has to be prepared to deal with the
> >> > fact that if a key 'foo' is a list, then the returned
> >> > qdict may contain either a QString or a QList for the
> >> > key 'foo'.
> >> >
> >> > In a third mode, it is possible to ask for repeated
> >> > options to be reported as an error, rather than silently
> >> > dropping all but the last one.
> >>
> >> To serve as a replacement for the options visitor, this needs to be able
> >> to behave exactly the same together with a suitably hacked up QObject
> >> input visitor. Before I dive into the actual patch, let me summarize
> >> QemuOpts and options visitor behavior.
> [...]
> >> Unlike the options visitor, this patch (judging from your description)
> >> makes a list only when keys are repeated. The QObject visitor will have
> >> to cope with finding both scalars and lists. When it finds a scalar,
> >> but needs a list, it'll have to wrap it in a list (PATCH 09, I think).
> >> When it finds a list, but needs a scalar, it'll have to fish it out of
> >> the list (where is that?).
> >
> > If my code finds a list but wants a scalar, it is reporting an
> > error.
>
> I'm afraid that breaks things like
>
> --numa node,cpus=0,cpus=2,mem=512,nodeid=0,nodeid=0
>
> The options visitor inteprets cpus=0,cpus=2 as a list [0,2] (because
> NumaNodeOptions member @cpus has a list type) and nodeid=0,nodid=0 as
> integer 0 (because member @nodeid has a scalar type).
Right, but that command line syntax is arguably broken already - the
app/user invoking QEMU is buggy, so I felt it was justifiable to
change from silently ignoring the app error to telling them of their
mistake.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options
2016-10-20 14:37 ` Daniel P. Berrange
@ 2016-10-21 11:28 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 11:28 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Thu, Oct 13, 2016 at 10:31:37AM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > If given an option string such as
>> >
>> > size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>> >
>> > the qemu_opts_to_qdict() method will currently overwrite
>> > the values for repeated option keys, so only the last
>> > value is in the returned dict:
>> >
>> > size=QString("1024")
>> > nodes=QString("1-2")
>> > policy=QString("bind")
>> >
>> > With this change the caller can optionally ask for all
>> > the repeated values to be stored in a QList. In the
>> > above example that would result in 'nodes' being a
>> > QList, so the returned dict would contain
>> >
>> > size=QString("1024")
>> > nodes=QList([QString("10"),
>> > QString("4-5"),
>> > QString("1-2")])
>> > policy=QString("bind")
>> >
>> > Note that the conversion has no way of knowing whether
>> > any given key is expected to be a list upfront - it can
>> > only figure that out when seeing the first duplicated
>> > key. Thus the caller has to be prepared to deal with the
>> > fact that if a key 'foo' is a list, then the returned
>> > qdict may contain either a QString or a QList for the
>> > key 'foo'.
>>
>> Actually three cases, not two:
>>
>> 0. qdict does not contain the key means empty list.
>>
>> 1. qdict contains the key with a QString value means list of one
>> element.
>>
>> 2. qdict contains the key with a QList value means list of more than one
>> element.
>>
>> I consider this weird. However, it's usefully weird with at least your
>> QObject input visitor.
>>
>> > In a third mode, it is possible to ask for repeated
>> > options to be reported as an error, rather than silently
>> > dropping all but the last one.
>>
>> Got users for this policy in the pipeline?
>
> I in fact used it in the QObjectInputVisitor, when the
> autocreate_list is not set.
>
> I guess strictly speaking this is not back-compatible
> if someone is passing repeated keys, but I judged that
> rather than silently ignoring this incorrect usage, it
> was acceptable to report an error.
While usage like
-machine usb=on,usb=off
could perhaps be declared erroneous post hoc, the "last one wins"
feature also has genuinely useful applications. For instance, tack a
correction to a long command line:
-machine usb=on [... lots of other stuff ...] -machine usb=off
Or modify canned configuration:
-readconfig vm1.config -machine usb=off
# vm1.config sets usb=on, -machine overrides
-readconfig vm1.config -set drive.disk0.file=tmp.qcow2
# vm1.config defines drive "disk0", -set overrides its image file
I don't think we can break them all.
[...]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists
2016-10-20 14:23 ` Daniel P. Berrange
@ 2016-10-21 11:58 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 11:58 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Wed, Oct 12, 2016 at 11:18:21AM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > When converting QemuOpts to a QObject, there is no information
>> > about compound types available,
>>
>> Yes, that's a drawback of splitting the conversion into a QemuOpts ->
>> QObject part that is oblivious of types, and a QObject -> QAPI object
>> part that knows the types.
>>
>> > so when visiting a list, the
>> > corresponding QObject is not guaranteed to be a QList. We
>> > therefore need to be able to auto-create a single element QList
>> > from whatever type we find.
>> >
>> > This mode should only be enabled if you have compatibility
>> > requirements for
>> >
>> > -arg foo=hello,foo=world
>> >
>> > to be treated as equivalent to the preferred syntax:
>> >
>> > -arg foo.0=hello,foo.1=world
>>
>> Not sure this is "preferred". "More powerfully warty" is probably
>> closer to the truth ;)
>
> Well, I call it "preferred" in the sense that that option syntax
> directly maps to the QAPI syntax in an unambigous manner. ie
> given the arg value alone "foo.0=hello,foo.1=world" you can clearly
> determine that "foo" is a list. With the compat syntax you cannot
> distinguish list from scalar, without knowing the QAPI schema.
>
>> How is "-arg foo=hello,foo=world" treated if this mode isn't enabled?
>
> The default behaviour would be that only the last key is present in
> the dict, eg foo=world, and then if you tried to visit a list, the
> visitor would complain that its got a QString instead of QList for
> the key 'foo'.
>
> This is related to patch 14
>
>> What would be the drawbacks of doing this always instead of only when we
>> "have compatibility requirements"?
>
> Essentially we'd be permanently allowing 2 distinct syntaxes for
> dealing with lists, for all options. I felt it desirable that we
> have only a single syntax and only allow this alt syntax in the
> backcompat cases.
Fair point.
The bolted-on extensions (options visitor's integer list syntax, block
layer's dotted key convention) have only ever worked in the places that
choose to use them. Even the integrated support for repeated keys is
only used for lists in the places that choose to use it that way.
>> > diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
>> > index d9269c9..d88e9f9 100644
>> > --- a/qapi/qobject-input-visitor.c
>> > +++ b/qapi/qobject-input-visitor.c
>> > @@ -48,6 +48,10 @@ struct QObjectInputVisitor
>> >
>> > /* True to reject parse in visit_end_struct() if unvisited keys remain. */
>> > bool strict;
>> > +
>> > + /* Whether we can auto-create single element lists when
>> > + * encountering a non-QList type */
>> > + bool autocreate_list;
>> > };
>> >
>> > static QObjectInputVisitor *to_qiv(Visitor *v)
>> > @@ -108,6 +112,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
>> > assert(obj);
>> > tos->obj = obj;
>> > tos->qapi = qapi;
>> > + qobject_incref(obj);
>> >
>> > if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
>> > h = g_hash_table_new(g_str_hash, g_str_equal);
>> > @@ -147,6 +152,7 @@ static void qobject_input_stack_object_free(StackObject *tos)
>> > if (tos->h) {
>> > g_hash_table_unref(tos->h);
>> > }
>> > + qobject_decref(tos->obj);
>> >
>> > g_free(tos);
>> > }
>>
>> Can you explain the reference counting change?
>
> Previously the stack stored a borrowed reference, since it didn't
> ever need responsibility for free'ing the object when popping the
> stack. This is no longer the case if you look a few lines later....
>
>
>>
>> > @@ -197,7 +203,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
>> > QObject *qobj = qobject_input_get_object(qiv, name, true);
>> > const QListEntry *entry;
>> >
>> > - if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
>> > + if (!qobj || (!qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST)) {
>>
>> Long line, but I believe it'll go away when you rebase for commit
>> 1382d4a.
>>
>> > if (list) {
>> > *list = NULL;
>> > }
>> > @@ -206,7 +212,16 @@ static void qobject_input_start_list(Visitor *v, const char *name,
>> > return;
>> > }
>> >
>> > - entry = qobject_input_push(qiv, qobj, list, errp);
>> > + if (qobject_type(qobj) != QTYPE_QLIST) {
>> > + QList *tmplist = qlist_new();
>> > + qlist_append_obj(tmplist, qobj);
>> > + qobject_incref(qobj);
>> > + entry = qobject_input_push(qiv, QOBJECT(tmplist), list, errp);
>> > + QDECREF(tmplist);
>
> ... here we are storing the 'qmplist' in the stack, and so when
> popping the stack, we must free that object. We thus need
> the stack to always hold its own reference, so when popping
> it can decref and (potentially) release the last reference.
Aha.
Manual reference counting is a PITA.
>> > + } else {
>> > + entry = qobject_input_push(qiv, qobj, list, errp);
>> > + }
>> > +
>> > if (list) {
>> > if (entry) {
>> > *list = g_malloc0(size);
>>
>> Buries autolist behavior in the middle of things. What about doing it
>> first, so it's more separate?
>
> I'm not sure I understand what you mean here ?
I'm suggesting a structure like this:
1. Map special case to normal case
2. Deal with normal case
Keeping the normal case and the special case separate like that can make
it easier to understand either.
Here's my attempt to do it:
>> QObjectInputVisitor *qiv = to_qiv(v);
>> QObject *qobj = qobject_input_get_object_(qiv, name, true, errp);
>> const QListEntry *entry;
>>
>> if (!qobj) {
>> return;
>> }
>>
>> + if (qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST) {
>> + QList *auto_list = qlist_new();
>> + qlist_append_obj(auto_list, qobj);
>> + qobj = auto_list;
>> + }
>> +
The new code is the mapping. The normal case code below remains
unchanged. It's just a sketch; I didn't spend a single brainwave on the
reference counting.
>> if (qobject_type(qobj) != QTYPE_QLIST) {
>>
>> I ignored reference counting here, because I don't yet understand how
>> and why you're changing it.
>>
>> > @@ -514,7 +529,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
>> > return &v->visitor;
>> > }
>> >
>> > -Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>> > +Visitor *qobject_input_visitor_new_autocast(QObject *obj,
>> > + bool autocreate_list)
>> > {
>> > QObjectInputVisitor *v;
>> >
>> > @@ -539,6 +555,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>> > v->visitor.optional = qobject_input_optional;
>> > v->visitor.free = qobject_input_free;
>> > v->strict = true;
>> > + v->autocreate_list = autocreate_list;
>> >
>> > v->root = obj;
>> > qobject_incref(obj);
>> > @@ -548,6 +565,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
>> >
>> >
>> > Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
>> > + bool autocreate_list,
>> > Error **errp)
>> > {
>> > QDict *pdict;
>> > @@ -564,7 +582,8 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
>> > goto cleanup;
>> > }
>> >
>> > - v = qobject_input_visitor_new_autocast(pobj);
>> > + v = qobject_input_visitor_new_autocast(pobj,
>> > + autocreate_list);
>> > cleanup:
>> > qobject_decref(pobj);
>> > QDECREF(pdict);
>> [Skipping test updates for now...]
>
> Regards,
> Daniel
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static Daniel P. Berrange
@ 2016-10-21 16:55 ` Markus Armbruster
2016-10-21 17:12 ` Eric Blake
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 16:55 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The opts-visitor.c opts_type_bool() method has code for
> parsing a string to set a bool value, as does the
> qemu-option.c parse_option_bool() method, except it
> handles fewer cases.
>
> To enable consistency across the codebase, extend
> parse_option_bool() to handle "yes", "no", "y" and
> "n", and make it non-static. Convert the opts
> visitor to call this method directly.
>
> Also make parse_option_number() non-static to allow
> for similar reuse later.
>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
On first glance, this patch makes the boolean values recognized on the
command line consistent regardless of which part of the code is used to
recognize them, by extending the QemuOpts parser to accept the option
visitor's additional convenience values.
Once we've done that in a release, we can't go back. No problem if it
actually achieved consistency. But it doesn't: there are other parsers
that recognize only "on" and "off". A quick grep finds
select_display(), bdrv_open_inherit(), netfilter_set_status().
bdrv_open_inherit() is particularly instructive: -drive gets parsed by
QemuOpts (evidence: your patch makes read-only=y work), but by the time
the options arrive here, they're a QDict. Curiously,
bdrv_open_inherit() expects the value of key "read-only" to be a
*string*, and it checks for "on". Fortunately, something on the way to
bdrv_open_inherit() replaces the "y" by "on", so this works. My point
is: this patch is trickier than it looks on first glance.
There's also HMP, which continues to accept only "on" and "off".
We might want to treat the option visitor's additional boolean values
like its other syntax extensions: keep for compatibility, but confine
to existing uses.
I think we should decide that when we merge the rest of your option
visitor replacement work, hopefully in 2.9.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static
2016-10-21 16:55 ` Markus Armbruster
@ 2016-10-21 17:12 ` Eric Blake
2016-10-21 17:51 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Eric Blake @ 2016-10-21 17:12 UTC (permalink / raw)
To: Markus Armbruster, Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]
On 10/21/2016 11:55 AM, Markus Armbruster wrote:
>
> On first glance, this patch makes the boolean values recognized on the
> command line consistent regardless of which part of the code is used to
> recognize them, by extending the QemuOpts parser to accept the option
> visitor's additional convenience values.
>
> Once we've done that in a release, we can't go back. No problem if it
> actually achieved consistency. But it doesn't: there are other parsers
> that recognize only "on" and "off". A quick grep finds
> select_display(), bdrv_open_inherit(), netfilter_set_status().
Would it be wrong to convert these to use the same function, to get us
to the point where they are consistent instead of self-limited? Yes, it
means that newer qemu will accept spellings that older doesn't, and that
we can't later prune the set of spellings back down, but trying to
remain backwards-compatible to every different set of inputs on a
command-by-command basis is harder than just making all commands accept
all spellings.
>
> bdrv_open_inherit() is particularly instructive: -drive gets parsed by
> QemuOpts (evidence: your patch makes read-only=y work), but by the time
> the options arrive here, they're a QDict. Curiously,
> bdrv_open_inherit() expects the value of key "read-only" to be a
> *string*, and it checks for "on". Fortunately, something on the way to
> bdrv_open_inherit() replaces the "y" by "on", so this works. My point
> is: this patch is trickier than it looks on first glance.
>
> There's also HMP, which continues to accept only "on" and "off".
Particularly for HMP, where we DON'T have backwards-compatibility
constraints, I'd argue that ease-of-use is the driving factor and that
HMP should allow ALL spellings of boolean arguments.
>
> We might want to treat the option visitor's additional boolean values
> like its other syntax extensions: keep for compatibility, but confine
> to existing uses.
As in - new interfaces ONLY get to pass "true" or "false", and only
existing interfaces get to also pass "y" or "on"?
>
> I think we should decide that when we merge the rest of your option
> visitor replacement work, hopefully in 2.9.
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static
2016-10-21 17:12 ` Eric Blake
@ 2016-10-21 17:51 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 17:51 UTC (permalink / raw)
To: Eric Blake
Cc: Daniel P. Berrange, Paolo Bonzini, Max Reitz,
Andreas Färber, qemu-block, qemu-devel
Eric Blake <eblake@redhat.com> writes:
> On 10/21/2016 11:55 AM, Markus Armbruster wrote:
>
>>
>> On first glance, this patch makes the boolean values recognized on the
>> command line consistent regardless of which part of the code is used to
>> recognize them, by extending the QemuOpts parser to accept the option
>> visitor's additional convenience values.
>>
>> Once we've done that in a release, we can't go back. No problem if it
>> actually achieved consistency. But it doesn't: there are other parsers
>> that recognize only "on" and "off". A quick grep finds
>> select_display(), bdrv_open_inherit(), netfilter_set_status().
>
> Would it be wrong to convert these to use the same function, to get us
> to the point where they are consistent instead of self-limited? Yes, it
> means that newer qemu will accept spellings that older doesn't, and that
> we can't later prune the set of spellings back down, but trying to
> remain backwards-compatible to every different set of inputs on a
> command-by-command basis is harder than just making all commands accept
> all spellings.
I have two concerns:
1. Accepting alternate spellings in just a few places is slightly bad,
accepting them is most, but not all places is worse. Converting all the
parsers as you suggest would take care of this one.
>> bdrv_open_inherit() is particularly instructive: -drive gets parsed by
>> QemuOpts (evidence: your patch makes read-only=y work), but by the time
>> the options arrive here, they're a QDict. Curiously,
>> bdrv_open_inherit() expects the value of key "read-only" to be a
>> *string*, and it checks for "on". Fortunately, something on the way to
>> bdrv_open_inherit() replaces the "y" by "on", so this works. My point
>> is: this patch is trickier than it looks on first glance.
2. I'm scared some user of QemuOpts declares QEMU_OPT_BOOL, but then
checks the *string*. Yes, QemuOpts lets you do that. Making the parser
accept more strings would break such code. Need a careful code review
to put this concern to rest.
>> There's also HMP, which continues to accept only "on" and "off".
>
> Particularly for HMP, where we DON'T have backwards-compatibility
> constraints, I'd argue that ease-of-use is the driving factor and that
> HMP should allow ALL spellings of boolean arguments.
Personally, I find alternate spellings more confusing than helpful, but
I agree that once we adopt them for the command line (consistently, not
just in a few places), we should also adopt them for HMP.
>> We might want to treat the option visitor's additional boolean values
>> like its other syntax extensions: keep for compatibility, but confine
>> to existing uses.
>
> As in - new interfaces ONLY get to pass "true" or "false", and only
> existing interfaces get to also pass "y" or "on"?
Very few places accept anything but "on" and "off", namely the ones that
use the options or the string visitor. Dan's work kills of the former,
and hopefully permits us killing off the latter. The alternate
spellings will then be confined in one place, guarded by a compatibility
flag you're not supposed to set in new code. Looks workable to me.
>> I think we should decide that when we merge the rest of your option
>> visitor replacement work, hopefully in 2.9.
I'm not rejecting this patch, I'm merely punting the ABI-affecting parts
of this series to 2.9.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
` (21 preceding siblings ...)
2016-09-30 15:45 ` [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties no-reply
@ 2016-10-21 18:30 ` Markus Armbruster
22 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-21 18:30 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> An update of a series previously posted
>
> v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
> v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
> v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
> v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
> v5: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg00485.html
> v6: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg03876.html
> v7: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00919.html
> v8: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03115.html
> v9: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02653.html
> v10: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02694.html
> v11: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg00652.html
> v12: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03559.html
> v13: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04212.html
> v14: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07037.html
>
> This series provides the infrastructure to allow use of non-scalar
> properties with the -object CLI arg, and object_add monitor commands.
> eg a property which is a list of structs. The syntax used for this is
> intentionally compatible with the syntax used by the block layer. This
> will allow the qdict_crumple method to be used by the block layer to
> convert from QemuOpts into structured QAPI block layer structs at a
> future date. It is already used by one of Max's patch series, by
> Kevin's -blockdev series, and by my own ACL series.
>
> This patch series has grown a fair bit larger since previous
> postings (19^H^H21 patches instead of 6). This was to avoid trying
> to squish too much into one patch, to make it easier to review.
>
> It does not neccessarily need to be merged in one go. There are
> three logical blocks of patches here.
>
> Patches 1-8 are the core support for visiting QemuOpts using the
> QObjectInputVisitor class. Merging this would unblock at least
> Kevin's blockdev series, and probably Max's series too.
>
> The patches 9-14 add enhancements to the code to support various
> special cases / edge cases needed to provide fully semantic
> replacement of the OptsVisitor
>
> Patches 15-21 deal with converting code over to use the new visitor
> logic.
>
> Once this is merged, there is also scope to further enhance the
> QObjectInputVisitor to deal with visiting single-properties,
> which would let us delete StringInputVisitor too. This would
> mean we have a single input visitor class, instead of the
> current three.
PATCH 03-06 applied to qapi-next. For PATCH 02, the question whether we
need @recursive is still open. I hope Max can help me figure it out, so
I can apply it next week. I think we'll rebase the rest on QemuOpts
work I want for 2.9. Thanks!
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-21 9:58 ` Markus Armbruster
@ 2016-10-21 18:31 ` Max Reitz
2016-10-24 9:18 ` Markus Armbruster
2016-10-25 10:03 ` Markus Armbruster
0 siblings, 2 replies; 109+ messages in thread
From: Max Reitz @ 2016-10-21 18:31 UTC (permalink / raw)
To: Markus Armbruster, Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block
[-- Attachment #1: Type: text/plain, Size: 4930 bytes --]
On 21.10.2016 11:58, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>>
>>>> The qdict_flatten() method will take a dict whose elements are
>>>> further nested dicts/lists and flatten them by concatenating
>>>> keys.
>>>>
>>>> The qdict_crumple() method aims to do the reverse, taking a flat
>>>> qdict, and turning it into a set of nested dicts/lists. It will
>>>> apply nesting based on the key name, with a '.' indicating a
>>>> new level in the hierarchy. If the keys in the nested structure
>>>> are all numeric, it will create a list, otherwise it will create
>>>> a dict.
>>>>
>>>> If the keys are a mixture of numeric and non-numeric, or the
>>>> numeric keys are not in strictly ascending order, an error will
>>>> be reported.
>>>>
>>>> As an example, a flat dict containing
>>>>
>>>> {
>>>> 'foo.0.bar': 'one',
>>>> 'foo.0.wizz': '1',
>>>> 'foo.1.bar': 'two',
>>>> 'foo.1.wizz': '2'
>>>> }
>>>>
>>>> will get turned into a dict with one element 'foo' whose
>>>> value is a list. The list elements will each in turn be
>>>> dicts.
>>>>
>>>> {
>>>> 'foo': [
>>>> { 'bar': 'one', 'wizz': '1' },
>>>> { 'bar': 'two', 'wizz': '2' }
>>>> ],
>>>> }
>>>>
>>>> If the key is intended to contain a literal '.', then it must
>>>> be escaped as '..'. ie a flat dict
>>>>
>>>> {
>>>> 'foo..bar': 'wizz',
>>>> 'bar.foo..bar': 'eek',
>>>> 'bar.hello': 'world'
>>>> }
>>>>
>>>> Will end up as
>>>>
>>>> {
>>>> 'foo.bar': 'wizz',
>>>> 'bar': {
>>>> 'foo.bar': 'eek',
>>>> 'hello': 'world'
>>>> }
>>>> }
>>>>
>>>> The intent of this function is that it allows a set of QemuOpts
>>>> to be turned into a nested data structure that mirrors the nesting
>>>> used when the same object is defined over QMP.
>>>>
>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>>>> ---
>>>> include/qapi/qmp/qdict.h | 1 +
>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>>>> 3 files changed, 551 insertions(+)
>>>>
>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>>>> index 71b8eb0..e0d24e1 100644
>>>> --- a/include/qapi/qmp/qdict.h
>>>> +++ b/include/qapi/qmp/qdict.h
>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>>>> void qdict_array_split(QDict *src, QList **dst);
>>>> int qdict_array_entries(QDict *src, const char *subqdict);
>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>>>>
>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>>>>
>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
>>>> index 60f158c..c38e90e 100644
>>>> --- a/qobject/qdict.c
>>>> +++ b/qobject/qdict.c
>>> [...]
>>>> +/**
>>>> + * qdict_crumple:
>>>> + * @src: the original flat dictionary (only scalar values) to crumple
>>>> + * @recursive: true to recursively crumple nested dictionaries
>>>
>>> Is recursive=false used outside tests in this series?
>>
>> No, its not used.
>>
>> It was suggested in a way earlier version by Max, but not sure if his
>> code uses it or not.
>
> In general, I prefer features to be added right before they're used, and
> I really dislike adding features "just in case". YAGNI.
>
> Max, do you actually need this one? If yes, please explain your use
> case.
As far as I can tell from a quick glance, I made the point for v1 that
qdict_crumple() could be simplified by using qdict_array_split() and
qdict_array_entries().
Dan then (correctly) said that using these functions would worsen
runtime performance of qdict_crumple() and that instead we can replace
qdict_array_split() by qdict_crumple(). However, for that to work, we
need to be able to make qdict_crumple() non-recursive (because
qdict_array_split() is non-recursive).
Dan also said that in the long run we want to keep the tree structure in
the block layer instead of flattening everything down which would rid us
of several other QDict functions (and would make non-recursive behavior
obsolete again). I believe that this is an idea for the (far?) future,
as can be seen from the discussion you and others had on this very topic
in this version here.
However, clearly there are no patches out yet that replace
qdict_array_split() by qdict_crumple(recursive=false), and I certainly
won't write any until qdict_crumple() is merged.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-21 18:31 ` Max Reitz
@ 2016-10-24 9:18 ` Markus Armbruster
2016-10-24 10:28 ` Daniel P. Berrange
2016-10-25 10:03 ` Markus Armbruster
1 sibling, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-24 9:18 UTC (permalink / raw)
To: Max Reitz
Cc: Daniel P. Berrange, Paolo Bonzini, Andreas Färber,
qemu-block, qemu-devel
Max Reitz <mreitz@redhat.com> writes:
> On 21.10.2016 11:58, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>>>
>>>>> The qdict_flatten() method will take a dict whose elements are
>>>>> further nested dicts/lists and flatten them by concatenating
>>>>> keys.
>>>>>
>>>>> The qdict_crumple() method aims to do the reverse, taking a flat
>>>>> qdict, and turning it into a set of nested dicts/lists. It will
>>>>> apply nesting based on the key name, with a '.' indicating a
>>>>> new level in the hierarchy. If the keys in the nested structure
>>>>> are all numeric, it will create a list, otherwise it will create
>>>>> a dict.
>>>>>
>>>>> If the keys are a mixture of numeric and non-numeric, or the
>>>>> numeric keys are not in strictly ascending order, an error will
>>>>> be reported.
>>>>>
>>>>> As an example, a flat dict containing
>>>>>
>>>>> {
>>>>> 'foo.0.bar': 'one',
>>>>> 'foo.0.wizz': '1',
>>>>> 'foo.1.bar': 'two',
>>>>> 'foo.1.wizz': '2'
>>>>> }
>>>>>
>>>>> will get turned into a dict with one element 'foo' whose
>>>>> value is a list. The list elements will each in turn be
>>>>> dicts.
>>>>>
>>>>> {
>>>>> 'foo': [
>>>>> { 'bar': 'one', 'wizz': '1' },
>>>>> { 'bar': 'two', 'wizz': '2' }
>>>>> ],
>>>>> }
>>>>>
>>>>> If the key is intended to contain a literal '.', then it must
>>>>> be escaped as '..'. ie a flat dict
>>>>>
>>>>> {
>>>>> 'foo..bar': 'wizz',
>>>>> 'bar.foo..bar': 'eek',
>>>>> 'bar.hello': 'world'
>>>>> }
>>>>>
>>>>> Will end up as
>>>>>
>>>>> {
>>>>> 'foo.bar': 'wizz',
>>>>> 'bar': {
>>>>> 'foo.bar': 'eek',
>>>>> 'hello': 'world'
>>>>> }
>>>>> }
>>>>>
>>>>> The intent of this function is that it allows a set of QemuOpts
>>>>> to be turned into a nested data structure that mirrors the nesting
>>>>> used when the same object is defined over QMP.
>>>>>
>>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>>>>> ---
>>>>> include/qapi/qmp/qdict.h | 1 +
>>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>>>>> 3 files changed, 551 insertions(+)
>>>>>
>>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>>>>> index 71b8eb0..e0d24e1 100644
>>>>> --- a/include/qapi/qmp/qdict.h
>>>>> +++ b/include/qapi/qmp/qdict.h
>>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>>>>> void qdict_array_split(QDict *src, QList **dst);
>>>>> int qdict_array_entries(QDict *src, const char *subqdict);
>>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>>>>>
>>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>>>>>
>>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
>>>>> index 60f158c..c38e90e 100644
>>>>> --- a/qobject/qdict.c
>>>>> +++ b/qobject/qdict.c
>>>> [...]
>>>>> +/**
>>>>> + * qdict_crumple:
>>>>> + * @src: the original flat dictionary (only scalar values) to crumple
>>>>> + * @recursive: true to recursively crumple nested dictionaries
>>>>
>>>> Is recursive=false used outside tests in this series?
>>>
>>> No, its not used.
>>>
>>> It was suggested in a way earlier version by Max, but not sure if his
>>> code uses it or not.
>>
>> In general, I prefer features to be added right before they're used, and
>> I really dislike adding features "just in case". YAGNI.
>>
>> Max, do you actually need this one? If yes, please explain your use
>> case.
>
> As far as I can tell from a quick glance, I made the point for v1 that
> qdict_crumple() could be simplified by using qdict_array_split() and
> qdict_array_entries().
>
> Dan then (correctly) said that using these functions would worsen
> runtime performance of qdict_crumple() and that instead we can replace
> qdict_array_split() by qdict_crumple(). However, for that to work, we
> need to be able to make qdict_crumple() non-recursive (because
> qdict_array_split() is non-recursive).
>
> Dan also said that in the long run we want to keep the tree structure in
> the block layer instead of flattening everything down which would rid us
> of several other QDict functions (and would make non-recursive behavior
> obsolete again). I believe that this is an idea for the (far?) future,
> as can be seen from the discussion you and others had on this very topic
> in this version here.
In the long run, the block layer should use proper C types instead of
mucking around with QDict. That's what QAPI is for.
> However, clearly there are no patches out yet that replace
> qdict_array_split() by qdict_crumple(recursive=false), and I certainly
> won't write any until qdict_crumple() is merged.
All right, I'll go hunting for prior versions of qdict_crumple() to see
whether I find a suitable one without @recursive.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-24 9:18 ` Markus Armbruster
@ 2016-10-24 10:28 ` Daniel P. Berrange
2016-10-24 12:38 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-24 10:28 UTC (permalink / raw)
To: Markus Armbruster
Cc: Max Reitz, Paolo Bonzini, Andreas Färber, qemu-block, qemu-devel
On Mon, Oct 24, 2016 at 11:18:29AM +0200, Markus Armbruster wrote:
> Max Reitz <mreitz@redhat.com> writes:
>
> > On 21.10.2016 11:58, Markus Armbruster wrote:
> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>
> >>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
> >>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>>>
> >>>>> The qdict_flatten() method will take a dict whose elements are
> >>>>> further nested dicts/lists and flatten them by concatenating
> >>>>> keys.
> >>>>>
> >>>>> The qdict_crumple() method aims to do the reverse, taking a flat
> >>>>> qdict, and turning it into a set of nested dicts/lists. It will
> >>>>> apply nesting based on the key name, with a '.' indicating a
> >>>>> new level in the hierarchy. If the keys in the nested structure
> >>>>> are all numeric, it will create a list, otherwise it will create
> >>>>> a dict.
> >>>>>
> >>>>> If the keys are a mixture of numeric and non-numeric, or the
> >>>>> numeric keys are not in strictly ascending order, an error will
> >>>>> be reported.
> >>>>>
> >>>>> As an example, a flat dict containing
> >>>>>
> >>>>> {
> >>>>> 'foo.0.bar': 'one',
> >>>>> 'foo.0.wizz': '1',
> >>>>> 'foo.1.bar': 'two',
> >>>>> 'foo.1.wizz': '2'
> >>>>> }
> >>>>>
> >>>>> will get turned into a dict with one element 'foo' whose
> >>>>> value is a list. The list elements will each in turn be
> >>>>> dicts.
> >>>>>
> >>>>> {
> >>>>> 'foo': [
> >>>>> { 'bar': 'one', 'wizz': '1' },
> >>>>> { 'bar': 'two', 'wizz': '2' }
> >>>>> ],
> >>>>> }
> >>>>>
> >>>>> If the key is intended to contain a literal '.', then it must
> >>>>> be escaped as '..'. ie a flat dict
> >>>>>
> >>>>> {
> >>>>> 'foo..bar': 'wizz',
> >>>>> 'bar.foo..bar': 'eek',
> >>>>> 'bar.hello': 'world'
> >>>>> }
> >>>>>
> >>>>> Will end up as
> >>>>>
> >>>>> {
> >>>>> 'foo.bar': 'wizz',
> >>>>> 'bar': {
> >>>>> 'foo.bar': 'eek',
> >>>>> 'hello': 'world'
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> The intent of this function is that it allows a set of QemuOpts
> >>>>> to be turned into a nested data structure that mirrors the nesting
> >>>>> used when the same object is defined over QMP.
> >>>>>
> >>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
> >>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> >>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> >>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> >>>>> ---
> >>>>> include/qapi/qmp/qdict.h | 1 +
> >>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
> >>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
> >>>>> 3 files changed, 551 insertions(+)
> >>>>>
> >>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> >>>>> index 71b8eb0..e0d24e1 100644
> >>>>> --- a/include/qapi/qmp/qdict.h
> >>>>> +++ b/include/qapi/qmp/qdict.h
> >>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
> >>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> >>>>> void qdict_array_split(QDict *src, QList **dst);
> >>>>> int qdict_array_entries(QDict *src, const char *subqdict);
> >>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
> >>>>>
> >>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
> >>>>>
> >>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
> >>>>> index 60f158c..c38e90e 100644
> >>>>> --- a/qobject/qdict.c
> >>>>> +++ b/qobject/qdict.c
> >>>> [...]
> >>>>> +/**
> >>>>> + * qdict_crumple:
> >>>>> + * @src: the original flat dictionary (only scalar values) to crumple
> >>>>> + * @recursive: true to recursively crumple nested dictionaries
> >>>>
> >>>> Is recursive=false used outside tests in this series?
> >>>
> >>> No, its not used.
> >>>
> >>> It was suggested in a way earlier version by Max, but not sure if his
> >>> code uses it or not.
> >>
> >> In general, I prefer features to be added right before they're used, and
> >> I really dislike adding features "just in case". YAGNI.
> >>
> >> Max, do you actually need this one? If yes, please explain your use
> >> case.
> >
> > As far as I can tell from a quick glance, I made the point for v1 that
> > qdict_crumple() could be simplified by using qdict_array_split() and
> > qdict_array_entries().
> >
> > Dan then (correctly) said that using these functions would worsen
> > runtime performance of qdict_crumple() and that instead we can replace
> > qdict_array_split() by qdict_crumple(). However, for that to work, we
> > need to be able to make qdict_crumple() non-recursive (because
> > qdict_array_split() is non-recursive).
> >
> > Dan also said that in the long run we want to keep the tree structure in
> > the block layer instead of flattening everything down which would rid us
> > of several other QDict functions (and would make non-recursive behavior
> > obsolete again). I believe that this is an idea for the (far?) future,
> > as can be seen from the discussion you and others had on this very topic
> > in this version here.
>
> In the long run, the block layer should use proper C types instead of
> mucking around with QDict. That's what QAPI is for.
>
> > However, clearly there are no patches out yet that replace
> > qdict_array_split() by qdict_crumple(recursive=false), and I certainly
> > won't write any until qdict_crumple() is merged.
>
> All right, I'll go hunting for prior versions of qdict_crumple() to see
> whether I find a suitable one without @recursive.
Please don't do that - there have been a tonne of other fixes and
improvements since the version that introduced @recursive. We need
to just chop out that parameter from the v15 version instead.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-24 10:28 ` Daniel P. Berrange
@ 2016-10-24 12:38 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-24 12:38 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, Paolo Bonzini, Andreas Färber, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Mon, Oct 24, 2016 at 11:18:29AM +0200, Markus Armbruster wrote:
>> Max Reitz <mreitz@redhat.com> writes:
>>
>> > On 21.10.2016 11:58, Markus Armbruster wrote:
>> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
>> >>
>> >>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>> >>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>> >>>>
>> >>>>> The qdict_flatten() method will take a dict whose elements are
>> >>>>> further nested dicts/lists and flatten them by concatenating
>> >>>>> keys.
>> >>>>>
>> >>>>> The qdict_crumple() method aims to do the reverse, taking a flat
>> >>>>> qdict, and turning it into a set of nested dicts/lists. It will
>> >>>>> apply nesting based on the key name, with a '.' indicating a
>> >>>>> new level in the hierarchy. If the keys in the nested structure
>> >>>>> are all numeric, it will create a list, otherwise it will create
>> >>>>> a dict.
>> >>>>>
>> >>>>> If the keys are a mixture of numeric and non-numeric, or the
>> >>>>> numeric keys are not in strictly ascending order, an error will
>> >>>>> be reported.
>> >>>>>
>> >>>>> As an example, a flat dict containing
>> >>>>>
>> >>>>> {
>> >>>>> 'foo.0.bar': 'one',
>> >>>>> 'foo.0.wizz': '1',
>> >>>>> 'foo.1.bar': 'two',
>> >>>>> 'foo.1.wizz': '2'
>> >>>>> }
>> >>>>>
>> >>>>> will get turned into a dict with one element 'foo' whose
>> >>>>> value is a list. The list elements will each in turn be
>> >>>>> dicts.
>> >>>>>
>> >>>>> {
>> >>>>> 'foo': [
>> >>>>> { 'bar': 'one', 'wizz': '1' },
>> >>>>> { 'bar': 'two', 'wizz': '2' }
>> >>>>> ],
>> >>>>> }
>> >>>>>
>> >>>>> If the key is intended to contain a literal '.', then it must
>> >>>>> be escaped as '..'. ie a flat dict
>> >>>>>
>> >>>>> {
>> >>>>> 'foo..bar': 'wizz',
>> >>>>> 'bar.foo..bar': 'eek',
>> >>>>> 'bar.hello': 'world'
>> >>>>> }
>> >>>>>
>> >>>>> Will end up as
>> >>>>>
>> >>>>> {
>> >>>>> 'foo.bar': 'wizz',
>> >>>>> 'bar': {
>> >>>>> 'foo.bar': 'eek',
>> >>>>> 'hello': 'world'
>> >>>>> }
>> >>>>> }
>> >>>>>
>> >>>>> The intent of this function is that it allows a set of QemuOpts
>> >>>>> to be turned into a nested data structure that mirrors the nesting
>> >>>>> used when the same object is defined over QMP.
>> >>>>>
>> >>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> >>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>> >>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> >>>>> ---
>> >>>>> include/qapi/qmp/qdict.h | 1 +
>> >>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>> >>>>> 3 files changed, 551 insertions(+)
>> >>>>>
>> >>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>> >>>>> index 71b8eb0..e0d24e1 100644
>> >>>>> --- a/include/qapi/qmp/qdict.h
>> >>>>> +++ b/include/qapi/qmp/qdict.h
>> >>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>> >>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>> >>>>> void qdict_array_split(QDict *src, QList **dst);
>> >>>>> int qdict_array_entries(QDict *src, const char *subqdict);
>> >>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>> >>>>>
>> >>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>> >>>>>
>> >>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
>> >>>>> index 60f158c..c38e90e 100644
>> >>>>> --- a/qobject/qdict.c
>> >>>>> +++ b/qobject/qdict.c
>> >>>> [...]
>> >>>>> +/**
>> >>>>> + * qdict_crumple:
>> >>>>> + * @src: the original flat dictionary (only scalar values) to crumple
>> >>>>> + * @recursive: true to recursively crumple nested dictionaries
>> >>>>
>> >>>> Is recursive=false used outside tests in this series?
>> >>>
>> >>> No, its not used.
>> >>>
>> >>> It was suggested in a way earlier version by Max, but not sure if his
>> >>> code uses it or not.
>> >>
>> >> In general, I prefer features to be added right before they're used, and
>> >> I really dislike adding features "just in case". YAGNI.
>> >>
>> >> Max, do you actually need this one? If yes, please explain your use
>> >> case.
>> >
>> > As far as I can tell from a quick glance, I made the point for v1 that
>> > qdict_crumple() could be simplified by using qdict_array_split() and
>> > qdict_array_entries().
>> >
>> > Dan then (correctly) said that using these functions would worsen
>> > runtime performance of qdict_crumple() and that instead we can replace
>> > qdict_array_split() by qdict_crumple(). However, for that to work, we
>> > need to be able to make qdict_crumple() non-recursive (because
>> > qdict_array_split() is non-recursive).
>> >
>> > Dan also said that in the long run we want to keep the tree structure in
>> > the block layer instead of flattening everything down which would rid us
>> > of several other QDict functions (and would make non-recursive behavior
>> > obsolete again). I believe that this is an idea for the (far?) future,
>> > as can be seen from the discussion you and others had on this very topic
>> > in this version here.
>>
>> In the long run, the block layer should use proper C types instead of
>> mucking around with QDict. That's what QAPI is for.
>>
>> > However, clearly there are no patches out yet that replace
>> > qdict_array_split() by qdict_crumple(recursive=false), and I certainly
>> > won't write any until qdict_crumple() is merged.
>>
>> All right, I'll go hunting for prior versions of qdict_crumple() to see
>> whether I find a suitable one without @recursive.
>
> Please don't do that - there have been a tonne of other fixes and
> improvements since the version that introduced @recursive. We need
> to just chop out that parameter from the v15 version instead.
I realized that once I looked at the interdiffs.
Chopping off @recursive is easy enough. So is dropping all tests that
pass false, but I'm not sure what that does to test coverage. Any
advice?
I'm sorry I didn't catch this in my prior reviews (v5+v8).
diff --git a/hmp.c b/hmp.c
index b32d8c8..3845d1b 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1700,7 +1700,7 @@ void hmp_object_add(Monitor *mon, const QDict *qdict)
Object *obj = NULL;
QObject *pobj;
- pobj = qdict_crumple(qdict, true, &err);
+ pobj = qdict_crumple(qdict, &err);
if (err) {
goto cleanup;
}
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index e0d24e1..fe9a4c5 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -73,7 +73,7 @@ void qdict_flatten(QDict *qdict);
void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
-QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
+QObject *qdict_crumple(const QDict *src, Error **errp);
void qdict_join(QDict *dest, QDict *src, bool overwrite);
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index f1030d5..c743a5b 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -769,7 +769,7 @@ Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
goto cleanup;
}
- pobj = qdict_crumple(pdict, true, errp);
+ pobj = qdict_crumple(pdict, errp);
if (!pobj) {
goto cleanup;
}
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 83384b2..bc970bf 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -815,15 +815,11 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
/**
* qdict_crumple:
* @src: the original dictionary to crumple
- * @recursive: true to recursively crumple nested dictionaries
*
* Takes a dictionary whose keys use '.' separator to indicate
* nesting, crumples it into a nested structure. The values in
- * @src are permitted to be scalars or compound typee. If the
- * @recursive parameter is false, then only the first level of
- * structure implied by the keys will be crumpled. If @recursive
- * is true, then the input will be recursively crumpled to expand
- * all levels of structure in the keys.
+ * @src are permitted to be scalars or compound typee. It will be
+ * recursively crumpled to expand all levels of structure in the keys.
*
* To include a literal '.' in a key name, it must be escaped as '..'
*
@@ -859,7 +855,7 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
* Returns: either a QDict or QList for the nested data structure, or NULL
* on error
*/
-QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
+QObject *qdict_crumple(const QDict *src, Error **errp)
{
const QDictEntry *ent;
QDict *two_level, *multi_level = NULL;
@@ -917,29 +913,24 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
/* Step 2: optionally process the two level dict recursively
* into a multi-level dict */
- if (recursive) {
- multi_level = qdict_new();
- for (ent = qdict_first(two_level); ent != NULL;
- ent = qdict_next(two_level, ent)) {
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT &&
- !qdict_haskey(compound, ent->key)) {
- child = qdict_crumple(qobject_to_qdict(ent->value),
- recursive, errp);
- if (!child) {
- goto error;
- }
-
- qdict_put_obj(multi_level, ent->key, child);
- } else {
- qobject_incref(ent->value);
- qdict_put_obj(multi_level, ent->key, ent->value);
+ if (qobject_type(ent->value) == QTYPE_QDICT &&
+ !qdict_haskey(compound, ent->key)) {
+ child = qdict_crumple(qobject_to_qdict(ent->value), errp);
+ if (!child) {
+ goto error;
}
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qobject_incref(ent->value);
+ qdict_put_obj(multi_level, ent->key, ent->value);
}
- QDECREF(two_level);
- } else {
- multi_level = two_level;
}
+ QDECREF(two_level);
two_level = NULL;
/* Step 3: detect if we need to turn our dict into list */
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index 88a1e88..0ea22fd 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -170,7 +170,7 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
QEMU_OPTS_REPEAT_POLICY_ALL,
&error_abort);
- pobj = qdict_crumple(pdict, true, errp);
+ pobj = qdict_crumple(pdict, errp);
if (!pobj) {
goto cleanup;
}
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 29d7362..0d1f996 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -596,108 +596,6 @@ static void qdict_join_test(void)
QDECREF(dict2);
}
-
-static void qdict_crumple_test_nonrecursive(void)
-{
- QDict *src, *dst, *rules, *vnc, *acl;
- QObject *child, *res;
-
- src = qdict_new();
- qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
- qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
- qdict_put(src, "rule.0.match", qstring_from_str("fred"));
- qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
- qdict_put(src, "rule.1.match", qstring_from_str("bob"));
- qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
- qdict_put(src, "acl..name", qstring_from_str("acl0"));
- qdict_put(src, "acl.rule..name", qstring_from_str("acl0"));
-
- res = qdict_crumple(src, false, &error_abort);
-
- g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
-
- dst = qobject_to_qdict(res);
-
- g_assert_cmpint(qdict_size(dst), ==, 4);
-
- child = qdict_get(dst, "vnc");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- vnc = qdict_get_qdict(dst, "vnc");
-
- g_assert_cmpint(qdict_size(vnc), ==, 2);
-
- g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
- g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
-
- child = qdict_get(dst, "rule");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- rules = qdict_get_qdict(dst, "rule");
-
- g_assert_cmpint(qdict_size(rules), ==, 4);
-
- g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
- g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
-
- /* With non-recursive crumpling, we should only see the first
- * level names unescaped */
- g_assert_cmpstr("acl0", ==, qdict_get_str(dst, "acl.name"));
- child = qdict_get(dst, "acl");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- acl = qdict_get_qdict(dst, "acl");
- g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule..name"));
-
- QDECREF(src);
- QDECREF(dst);
-}
-
-
-static void qdict_crumple_test_listtop(void)
-{
- QDict *src, *rule;
- QList *rules;
- QObject *res;
-
- src = qdict_new();
- qdict_put(src, "0.match.name", qstring_from_str("Fred"));
- qdict_put(src, "0.match.email", qstring_from_str("fred@example.com"));
- qdict_put(src, "0.policy", qstring_from_str("allow"));
- qdict_put(src, "1.match.name", qstring_from_str("Bob"));
- qdict_put(src, "1.match.email", qstring_from_str("bob@example.com"));
- qdict_put(src, "1.policy", qstring_from_str("deny"));
-
- res = qdict_crumple(src, false, &error_abort);
-
- g_assert_cmpint(qobject_type(res), ==, QTYPE_QLIST);
-
- rules = qobject_to_qlist(res);
-
- g_assert_cmpint(qlist_size(rules), ==, 2);
-
- g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
- rule = qobject_to_qdict(qlist_pop(rules));
- g_assert_cmpint(qdict_size(rule), ==, 3);
-
- g_assert_cmpstr("Fred", ==, qdict_get_str(rule, "match.name"));
- g_assert_cmpstr("fred@example.com", ==, qdict_get_str(rule, "match.email"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
- QDECREF(rule);
-
- g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
- rule = qobject_to_qdict(qlist_pop(rules));
- g_assert_cmpint(qdict_size(rule), ==, 3);
-
- g_assert_cmpstr("Bob", ==, qdict_get_str(rule, "match.name"));
- g_assert_cmpstr("bob@example.com", ==, qdict_get_str(rule, "match.email"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
- QDECREF(rule);
-
- QDECREF(src);
- qobject_decref(res);
-}
-
-
static void qdict_crumple_test_recursive(void)
{
QDict *src, *dst, *rule, *vnc, *acl, *listen;
@@ -715,7 +613,7 @@ static void qdict_crumple_test_recursive(void)
qdict_put(src, "vnc.acl..name", qstring_from_str("acl0"));
qdict_put(src, "vnc.acl.rule..name", qstring_from_str("acl0"));
- res = qdict_crumple(src, true, &error_abort);
+ res = qdict_crumple(src, &error_abort);
g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
@@ -765,14 +663,13 @@ static void qdict_crumple_test_recursive(void)
QDECREF(dst);
}
-
static void qdict_crumple_test_empty(void)
{
QDict *src, *dst;
src = qdict_new();
- dst = (QDict *)qdict_crumple(src, true, &error_abort);
+ dst = (QDict *)qdict_crumple(src, &error_abort);
g_assert_cmpint(qdict_size(dst), ==, 0);
@@ -780,7 +677,6 @@ static void qdict_crumple_test_empty(void)
QDECREF(dst);
}
-
static void qdict_crumple_test_bad_inputs(void)
{
QDict *src;
@@ -791,7 +687,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("fred"));
qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -802,7 +698,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("fred"));
qdict_put(src, "rule.a", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -813,7 +709,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.a", qdict_new());
qdict_put(src, "rule.b", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -825,7 +721,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("deny"));
qdict_put(src, "rule.3", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -837,7 +733,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("deny"));
qdict_put(src, "rule.+1", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -860,7 +756,7 @@ static void qdict_crumple_test_non_flat_list_ok(void)
qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
qdict_put(src, "match", list);
- res = qdict_crumple(src, true, &error_abort);
+ res = qdict_crumple(src, &error_abort);
g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
@@ -903,32 +799,6 @@ static void qdict_crumple_test_non_flat_list_ok(void)
QDECREF(dst);
}
-
-static void qdict_crumple_test_non_flat_list_bad(void)
-{
- QDict *src;
- QObject *res;
- QList *list;
- Error *err = NULL;
-
- src = qdict_new();
- list = qlist_new();
- qlist_append(list, qstring_from_str("fred"));
- qlist_append(list, qstring_from_str("bob"));
-
- qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
- qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
- qdict_put(src, "match.0", qstring_from_str("frank"));
- qdict_put(src, "match", list);
-
- res = qdict_crumple(src, false, &err);
- g_assert(!res);
- error_free_or_abort(&err);
-
- QDECREF(src);
-}
-
-
static void qdict_crumple_test_non_flat_dict_ok(void)
{
QDict *src, *dst, *rule, *vnc, *acl;
@@ -946,7 +816,7 @@ static void qdict_crumple_test_non_flat_dict_ok(void)
qdict_put(src, "acl.rules.1.match", qstring_from_str("bob"));
qdict_put(src, "acl.rules.1.policy", qstring_from_str("deny"));
- res = qdict_crumple(src, true, &error_abort);
+ res = qdict_crumple(src, &error_abort);
g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
@@ -986,7 +856,6 @@ static void qdict_crumple_test_non_flat_dict_ok(void)
QDECREF(dst);
}
-
static void qdict_crumple_test_non_flat_dict_bad(void)
{
QDict *src, *vnc;
@@ -1004,14 +873,13 @@ static void qdict_crumple_test_non_flat_dict_bad(void)
qdict_put(src, "acl.rules.1.match", qstring_from_str("bob"));
qdict_put(src, "acl.rules.1.policy", qstring_from_str("deny"));
- res = qdict_crumple(src, true, &err);
+ res = qdict_crumple(src, &err);
g_assert(!res);
error_free_or_abort(&err);
QDECREF(src);
}
-
/*
* Errors test-cases
*/
@@ -1159,20 +1027,14 @@ int main(int argc, char **argv)
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
- g_test_add_func("/public/crumple/nonrecursive",
- qdict_crumple_test_nonrecursive);
g_test_add_func("/public/crumple/recursive",
qdict_crumple_test_recursive);
- g_test_add_func("/public/crumple/listtop",
- qdict_crumple_test_listtop);
g_test_add_func("/public/crumple/empty",
qdict_crumple_test_empty);
g_test_add_func("/public/crumple/bad_inputs",
qdict_crumple_test_bad_inputs);
g_test_add_func("/public/crumple/non-flat-list-ok",
qdict_crumple_test_non_flat_list_ok);
- g_test_add_func("/public/crumple/non-flat-list-bad",
- qdict_crumple_test_non_flat_list_bad);
g_test_add_func("/public/crumple/non-flat-dict-ok",
qdict_crumple_test_non_flat_dict_ok);
g_test_add_func("/public/crumple/non-flat-dict-bad",
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-21 18:31 ` Max Reitz
2016-10-24 9:18 ` Markus Armbruster
@ 2016-10-25 10:03 ` Markus Armbruster
2016-10-25 10:20 ` Daniel P. Berrange
1 sibling, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2016-10-25 10:03 UTC (permalink / raw)
To: Max Reitz
Cc: Daniel P. Berrange, Paolo Bonzini, Andreas Färber,
qemu-block, qemu-devel
Max Reitz <mreitz@redhat.com> writes:
> On 21.10.2016 11:58, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>>>
>>>>> The qdict_flatten() method will take a dict whose elements are
>>>>> further nested dicts/lists and flatten them by concatenating
>>>>> keys.
>>>>>
>>>>> The qdict_crumple() method aims to do the reverse, taking a flat
>>>>> qdict, and turning it into a set of nested dicts/lists. It will
>>>>> apply nesting based on the key name, with a '.' indicating a
>>>>> new level in the hierarchy. If the keys in the nested structure
>>>>> are all numeric, it will create a list, otherwise it will create
>>>>> a dict.
>>>>>
>>>>> If the keys are a mixture of numeric and non-numeric, or the
>>>>> numeric keys are not in strictly ascending order, an error will
>>>>> be reported.
>>>>>
>>>>> As an example, a flat dict containing
>>>>>
>>>>> {
>>>>> 'foo.0.bar': 'one',
>>>>> 'foo.0.wizz': '1',
>>>>> 'foo.1.bar': 'two',
>>>>> 'foo.1.wizz': '2'
>>>>> }
>>>>>
>>>>> will get turned into a dict with one element 'foo' whose
>>>>> value is a list. The list elements will each in turn be
>>>>> dicts.
>>>>>
>>>>> {
>>>>> 'foo': [
>>>>> { 'bar': 'one', 'wizz': '1' },
>>>>> { 'bar': 'two', 'wizz': '2' }
>>>>> ],
>>>>> }
>>>>>
>>>>> If the key is intended to contain a literal '.', then it must
>>>>> be escaped as '..'. ie a flat dict
>>>>>
>>>>> {
>>>>> 'foo..bar': 'wizz',
>>>>> 'bar.foo..bar': 'eek',
>>>>> 'bar.hello': 'world'
>>>>> }
>>>>>
>>>>> Will end up as
>>>>>
>>>>> {
>>>>> 'foo.bar': 'wizz',
>>>>> 'bar': {
>>>>> 'foo.bar': 'eek',
>>>>> 'hello': 'world'
>>>>> }
>>>>> }
>>>>>
>>>>> The intent of this function is that it allows a set of QemuOpts
>>>>> to be turned into a nested data structure that mirrors the nesting
>>>>> used when the same object is defined over QMP.
>>>>>
>>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>>>>> ---
>>>>> include/qapi/qmp/qdict.h | 1 +
>>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>>>>> 3 files changed, 551 insertions(+)
>>>>>
>>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>>>>> index 71b8eb0..e0d24e1 100644
>>>>> --- a/include/qapi/qmp/qdict.h
>>>>> +++ b/include/qapi/qmp/qdict.h
>>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>>>>> void qdict_array_split(QDict *src, QList **dst);
>>>>> int qdict_array_entries(QDict *src, const char *subqdict);
>>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>>>>>
>>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>>>>>
>>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
>>>>> index 60f158c..c38e90e 100644
>>>>> --- a/qobject/qdict.c
>>>>> +++ b/qobject/qdict.c
>>>> [...]
>>>>> +/**
>>>>> + * qdict_crumple:
>>>>> + * @src: the original flat dictionary (only scalar values) to crumple
>>>>> + * @recursive: true to recursively crumple nested dictionaries
>>>>
>>>> Is recursive=false used outside tests in this series?
>>>
>>> No, its not used.
>>>
>>> It was suggested in a way earlier version by Max, but not sure if his
>>> code uses it or not.
>>
>> In general, I prefer features to be added right before they're used, and
>> I really dislike adding features "just in case". YAGNI.
>>
>> Max, do you actually need this one? If yes, please explain your use
>> case.
>
> As far as I can tell from a quick glance, I made the point for v1 that
> qdict_crumple() could be simplified by using qdict_array_split() and
> qdict_array_entries().
>
> Dan then (correctly) said that using these functions would worsen
> runtime performance of qdict_crumple() and that instead we can replace
> qdict_array_split() by qdict_crumple(). However, for that to work, we
> need to be able to make qdict_crumple() non-recursive (because
> qdict_array_split() is non-recursive).
>
> Dan also said that in the long run we want to keep the tree structure in
> the block layer instead of flattening everything down which would rid us
> of several other QDict functions (and would make non-recursive behavior
> obsolete again). I believe that this is an idea for the (far?) future,
> as can be seen from the discussion you and others had on this very topic
> in this version here.
>
> However, clearly there are no patches out yet that replace
> qdict_array_split() by qdict_crumple(recursive=false), and I certainly
> won't write any until qdict_crumple() is merged.
I prefer not to maintain code that isn't actually used. Since Dan is
enjoying a few days off, and the soft freeze is closing in, I'm
proposing to squash in the following:
* Drop @recursive, in the most straightforward way possible.
* Drop all tests that pass false to @recursive. This might reduce
coverage, but we can fix that on top.
* While there, collapse some vertical whitespace, for consistency with
spacing elsewhere in the same files.
Resulting fixup patch appended. I'd appreciate a quick look-over before
I do a pull request. Current state at
http://repo.or.cz/w/qemu/armbru.git branch qapi-not-next.
>From df87de10a12bd65d2ddc47514e951712de7c06f0 Mon Sep 17 00:00:00 2001
From: Markus Armbruster <armbru@redhat.com>
Date: Tue, 25 Oct 2016 10:09:46 +0200
Subject: [PATCH] fixup! qdict: implement a qdict_crumple method for
un-flattening a dict
---
include/qapi/qmp/qdict.h | 2 +-
qobject/qdict.c | 42 ++++++---------
tests/check-qdict.c | 133 +++--------------------------------------------
3 files changed, 23 insertions(+), 154 deletions(-)
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index e0d24e1..fe9a4c5 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -73,7 +73,7 @@ void qdict_flatten(QDict *qdict);
void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
-QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
+QObject *qdict_crumple(const QDict *src, Error **errp);
void qdict_join(QDict *dest, QDict *src, bool overwrite);
diff --git a/qobject/qdict.c b/qobject/qdict.c
index c38e90e..197b0fb 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -684,7 +684,6 @@ void qdict_array_split(QDict *src, QList **dst)
}
}
-
/**
* qdict_split_flat_key:
* @key: the key string to split
@@ -744,7 +743,6 @@ static void qdict_split_flat_key(const char *key, char **prefix,
(*prefix)[j] = '\0';
}
-
/**
* qdict_is_list:
* @maybe_list: dict to check if keys represent list elements.
@@ -815,14 +813,10 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
/**
* qdict_crumple:
* @src: the original flat dictionary (only scalar values) to crumple
- * @recursive: true to recursively crumple nested dictionaries
*
* Takes a flat dictionary whose keys use '.' separator to indicate
* nesting, and values are scalars, and crumples it into a nested
- * structure. If the @recursive parameter is false, then only the
- * first level of structure implied by the keys will be crumpled. If
- * @recursive is true, then the input will be recursively crumpled to
- * expand all levels of structure in the keys.
+ * structure.
*
* To include a literal '.' in a key name, it must be escaped as '..'
*
@@ -855,7 +849,7 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
* Returns: either a QDict or QList for the nested data structure, or NULL
* on error
*/
-QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
+QObject *qdict_crumple(const QDict *src, Error **errp)
{
const QDictEntry *ent;
QDict *two_level, *multi_level = NULL;
@@ -908,28 +902,23 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
/* Step 2: optionally process the two level dict recursively
* into a multi-level dict */
- if (recursive) {
- multi_level = qdict_new();
- for (ent = qdict_first(two_level); ent != NULL;
- ent = qdict_next(two_level, ent)) {
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT) {
- child = qdict_crumple(qobject_to_qdict(ent->value),
- recursive, errp);
- if (!child) {
- goto error;
- }
-
- qdict_put_obj(multi_level, ent->key, child);
- } else {
- qobject_incref(ent->value);
- qdict_put_obj(multi_level, ent->key, ent->value);
+ if (qobject_type(ent->value) == QTYPE_QDICT) {
+ child = qdict_crumple(qobject_to_qdict(ent->value), errp);
+ if (!child) {
+ goto error;
}
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qobject_incref(ent->value);
+ qdict_put_obj(multi_level, ent->key, ent->value);
}
- QDECREF(two_level);
- } else {
- multi_level = two_level;
}
+ QDECREF(two_level);
two_level = NULL;
/* Step 3: detect if we need to turn our dict into list */
@@ -971,7 +960,6 @@ QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp)
return NULL;
}
-
/**
* qdict_array_entries(): Returns the number of direct array entries if the
* sub-QDict of src specified by the prefix in subqdict (or src itself for
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 64c33ab..07b1c79 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -413,7 +413,6 @@ static void qdict_array_split_test(void)
QDECREF(test_dict);
-
/*
* Test the split of
*
@@ -519,7 +518,6 @@ static void qdict_join_test(void)
dict1 = qdict_new();
dict2 = qdict_new();
-
/* Test everything once without overwrite and once with */
do
{
@@ -529,7 +527,6 @@ static void qdict_join_test(void)
g_assert(qdict_size(dict1) == 0);
g_assert(qdict_size(dict2) == 0);
-
/* First iteration: Test movement */
/* Second iteration: Test empty source and non-empty destination */
qdict_put(dict2, "foo", qint_from_int(42));
@@ -543,7 +540,6 @@ static void qdict_join_test(void)
g_assert(qdict_get_int(dict1, "foo") == 42);
}
-
/* Test non-empty source and destination without conflict */
qdict_put(dict2, "bar", qint_from_int(23));
@@ -555,7 +551,6 @@ static void qdict_join_test(void)
g_assert(qdict_get_int(dict1, "foo") == 42);
g_assert(qdict_get_int(dict1, "bar") == 23);
-
/* Test conflict */
qdict_put(dict2, "foo", qint_from_int(84));
@@ -571,7 +566,6 @@ static void qdict_join_test(void)
g_assert(qdict_get_int(dict2, "foo") == 84);
}
-
/* Check the references */
g_assert(qdict_get(dict1, "foo")->refcnt == 1);
g_assert(qdict_get(dict1, "bar")->refcnt == 1);
@@ -580,7 +574,6 @@ static void qdict_join_test(void)
g_assert(qdict_get(dict2, "foo")->refcnt == 1);
}
-
/* Clean up */
qdict_del(dict1, "foo");
qdict_del(dict1, "bar");
@@ -591,113 +584,10 @@ static void qdict_join_test(void)
}
while (overwrite ^= true);
-
QDECREF(dict1);
QDECREF(dict2);
}
-
-static void qdict_crumple_test_nonrecursive(void)
-{
- QDict *src, *dst, *rules, *vnc, *acl;
- QObject *child, *res;
-
- src = qdict_new();
- qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
- qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
- qdict_put(src, "rule.0.match", qstring_from_str("fred"));
- qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
- qdict_put(src, "rule.1.match", qstring_from_str("bob"));
- qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
- qdict_put(src, "acl..name", qstring_from_str("acl0"));
- qdict_put(src, "acl.rule..name", qstring_from_str("acl0"));
-
- res = qdict_crumple(src, false, &error_abort);
-
- g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
-
- dst = qobject_to_qdict(res);
-
- g_assert_cmpint(qdict_size(dst), ==, 4);
-
- child = qdict_get(dst, "vnc");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- vnc = qdict_get_qdict(dst, "vnc");
-
- g_assert_cmpint(qdict_size(vnc), ==, 2);
-
- g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
- g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
-
- child = qdict_get(dst, "rule");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- rules = qdict_get_qdict(dst, "rule");
-
- g_assert_cmpint(qdict_size(rules), ==, 4);
-
- g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
- g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
-
- /* With non-recursive crumpling, we should only see the first
- * level names unescaped */
- g_assert_cmpstr("acl0", ==, qdict_get_str(dst, "acl.name"));
- child = qdict_get(dst, "acl");
- g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
- acl = qdict_get_qdict(dst, "acl");
- g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule..name"));
-
- QDECREF(src);
- QDECREF(dst);
-}
-
-
-static void qdict_crumple_test_listtop(void)
-{
- QDict *src, *rule;
- QList *rules;
- QObject *res;
-
- src = qdict_new();
- qdict_put(src, "0.match.name", qstring_from_str("Fred"));
- qdict_put(src, "0.match.email", qstring_from_str("fred@example.com"));
- qdict_put(src, "0.policy", qstring_from_str("allow"));
- qdict_put(src, "1.match.name", qstring_from_str("Bob"));
- qdict_put(src, "1.match.email", qstring_from_str("bob@example.com"));
- qdict_put(src, "1.policy", qstring_from_str("deny"));
-
- res = qdict_crumple(src, false, &error_abort);
-
- g_assert_cmpint(qobject_type(res), ==, QTYPE_QLIST);
-
- rules = qobject_to_qlist(res);
-
- g_assert_cmpint(qlist_size(rules), ==, 2);
-
- g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
- rule = qobject_to_qdict(qlist_pop(rules));
- g_assert_cmpint(qdict_size(rule), ==, 3);
-
- g_assert_cmpstr("Fred", ==, qdict_get_str(rule, "match.name"));
- g_assert_cmpstr("fred@example.com", ==, qdict_get_str(rule, "match.email"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
- QDECREF(rule);
-
- g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
- rule = qobject_to_qdict(qlist_pop(rules));
- g_assert_cmpint(qdict_size(rule), ==, 3);
-
- g_assert_cmpstr("Bob", ==, qdict_get_str(rule, "match.name"));
- g_assert_cmpstr("bob@example.com", ==, qdict_get_str(rule, "match.email"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
- QDECREF(rule);
-
- QDECREF(src);
- qobject_decref(res);
-}
-
-
static void qdict_crumple_test_recursive(void)
{
QDict *src, *dst, *rule, *vnc, *acl, *listen;
@@ -715,7 +605,7 @@ static void qdict_crumple_test_recursive(void)
qdict_put(src, "vnc.acl..name", qstring_from_str("acl0"));
qdict_put(src, "vnc.acl.rule..name", qstring_from_str("acl0"));
- res = qdict_crumple(src, true, &error_abort);
+ res = qdict_crumple(src, &error_abort);
g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
@@ -765,14 +655,13 @@ static void qdict_crumple_test_recursive(void)
QDECREF(dst);
}
-
static void qdict_crumple_test_empty(void)
{
QDict *src, *dst;
src = qdict_new();
- dst = (QDict *)qdict_crumple(src, true, &error_abort);
+ dst = (QDict *)qdict_crumple(src, &error_abort);
g_assert_cmpint(qdict_size(dst), ==, 0);
@@ -780,7 +669,6 @@ static void qdict_crumple_test_empty(void)
QDECREF(dst);
}
-
static void qdict_crumple_test_bad_inputs(void)
{
QDict *src;
@@ -791,7 +679,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("fred"));
qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -802,7 +690,7 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.0", qstring_from_str("fred"));
qdict_put(src, "rule.a", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
@@ -813,38 +701,35 @@ static void qdict_crumple_test_bad_inputs(void)
qdict_put(src, "rule.a", qdict_new());
qdict_put(src, "rule.b", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
QDECREF(src);
-
src = qdict_new();
/* List indexes must not have gaps */
qdict_put(src, "rule.0", qstring_from_str("deny"));
qdict_put(src, "rule.3", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
QDECREF(src);
-
src = qdict_new();
/* List indexes must be in %zu format */
qdict_put(src, "rule.0", qstring_from_str("deny"));
qdict_put(src, "rule.+1", qstring_from_str("allow"));
- g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(qdict_crumple(src, &error) == NULL);
g_assert(error != NULL);
error_free(error);
error = NULL;
QDECREF(src);
}
-
/*
* Errors test-cases
*/
@@ -992,12 +877,8 @@ int main(int argc, char **argv)
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
- g_test_add_func("/public/crumple/nonrecursive",
- qdict_crumple_test_nonrecursive);
g_test_add_func("/public/crumple/recursive",
qdict_crumple_test_recursive);
- g_test_add_func("/public/crumple/listtop",
- qdict_crumple_test_listtop);
g_test_add_func("/public/crumple/empty",
qdict_crumple_test_empty);
g_test_add_func("/public/crumple/bad_inputs",
--
2.5.5
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-25 10:03 ` Markus Armbruster
@ 2016-10-25 10:20 ` Daniel P. Berrange
2016-10-25 12:29 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2016-10-25 10:20 UTC (permalink / raw)
To: Markus Armbruster
Cc: Max Reitz, Paolo Bonzini, Andreas Färber, qemu-block, qemu-devel
On Tue, Oct 25, 2016 at 12:03:33PM +0200, Markus Armbruster wrote:
> Max Reitz <mreitz@redhat.com> writes:
>
> > On 21.10.2016 11:58, Markus Armbruster wrote:
> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>
> >>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
> >>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
> >>>>
> >>>>> The qdict_flatten() method will take a dict whose elements are
> >>>>> further nested dicts/lists and flatten them by concatenating
> >>>>> keys.
> >>>>>
> >>>>> The qdict_crumple() method aims to do the reverse, taking a flat
> >>>>> qdict, and turning it into a set of nested dicts/lists. It will
> >>>>> apply nesting based on the key name, with a '.' indicating a
> >>>>> new level in the hierarchy. If the keys in the nested structure
> >>>>> are all numeric, it will create a list, otherwise it will create
> >>>>> a dict.
> >>>>>
> >>>>> If the keys are a mixture of numeric and non-numeric, or the
> >>>>> numeric keys are not in strictly ascending order, an error will
> >>>>> be reported.
> >>>>>
> >>>>> As an example, a flat dict containing
> >>>>>
> >>>>> {
> >>>>> 'foo.0.bar': 'one',
> >>>>> 'foo.0.wizz': '1',
> >>>>> 'foo.1.bar': 'two',
> >>>>> 'foo.1.wizz': '2'
> >>>>> }
> >>>>>
> >>>>> will get turned into a dict with one element 'foo' whose
> >>>>> value is a list. The list elements will each in turn be
> >>>>> dicts.
> >>>>>
> >>>>> {
> >>>>> 'foo': [
> >>>>> { 'bar': 'one', 'wizz': '1' },
> >>>>> { 'bar': 'two', 'wizz': '2' }
> >>>>> ],
> >>>>> }
> >>>>>
> >>>>> If the key is intended to contain a literal '.', then it must
> >>>>> be escaped as '..'. ie a flat dict
> >>>>>
> >>>>> {
> >>>>> 'foo..bar': 'wizz',
> >>>>> 'bar.foo..bar': 'eek',
> >>>>> 'bar.hello': 'world'
> >>>>> }
> >>>>>
> >>>>> Will end up as
> >>>>>
> >>>>> {
> >>>>> 'foo.bar': 'wizz',
> >>>>> 'bar': {
> >>>>> 'foo.bar': 'eek',
> >>>>> 'hello': 'world'
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> The intent of this function is that it allows a set of QemuOpts
> >>>>> to be turned into a nested data structure that mirrors the nesting
> >>>>> used when the same object is defined over QMP.
> >>>>>
> >>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
> >>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> >>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> >>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> >>>>> ---
> >>>>> include/qapi/qmp/qdict.h | 1 +
> >>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
> >>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
> >>>>> 3 files changed, 551 insertions(+)
> >>>>>
> >>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> >>>>> index 71b8eb0..e0d24e1 100644
> >>>>> --- a/include/qapi/qmp/qdict.h
> >>>>> +++ b/include/qapi/qmp/qdict.h
> >>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
> >>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> >>>>> void qdict_array_split(QDict *src, QList **dst);
> >>>>> int qdict_array_entries(QDict *src, const char *subqdict);
> >>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
> >>>>>
> >>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
> >>>>>
> >>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
> >>>>> index 60f158c..c38e90e 100644
> >>>>> --- a/qobject/qdict.c
> >>>>> +++ b/qobject/qdict.c
> >>>> [...]
> >>>>> +/**
> >>>>> + * qdict_crumple:
> >>>>> + * @src: the original flat dictionary (only scalar values) to crumple
> >>>>> + * @recursive: true to recursively crumple nested dictionaries
> >>>>
> >>>> Is recursive=false used outside tests in this series?
> >>>
> >>> No, its not used.
> >>>
> >>> It was suggested in a way earlier version by Max, but not sure if his
> >>> code uses it or not.
> >>
> >> In general, I prefer features to be added right before they're used, and
> >> I really dislike adding features "just in case". YAGNI.
> >>
> >> Max, do you actually need this one? If yes, please explain your use
> >> case.
> >
> > As far as I can tell from a quick glance, I made the point for v1 that
> > qdict_crumple() could be simplified by using qdict_array_split() and
> > qdict_array_entries().
> >
> > Dan then (correctly) said that using these functions would worsen
> > runtime performance of qdict_crumple() and that instead we can replace
> > qdict_array_split() by qdict_crumple(). However, for that to work, we
> > need to be able to make qdict_crumple() non-recursive (because
> > qdict_array_split() is non-recursive).
> >
> > Dan also said that in the long run we want to keep the tree structure in
> > the block layer instead of flattening everything down which would rid us
> > of several other QDict functions (and would make non-recursive behavior
> > obsolete again). I believe that this is an idea for the (far?) future,
> > as can be seen from the discussion you and others had on this very topic
> > in this version here.
> >
> > However, clearly there are no patches out yet that replace
> > qdict_array_split() by qdict_crumple(recursive=false), and I certainly
> > won't write any until qdict_crumple() is merged.
>
> I prefer not to maintain code that isn't actually used. Since Dan is
> enjoying a few days off, and the soft freeze is closing in, I'm
> proposing to squash in the following:
>
> * Drop @recursive, in the most straightforward way possible.
>
> * Drop all tests that pass false to @recursive. This might reduce
> coverage, but we can fix that on top.
>
> * While there, collapse some vertical whitespace, for consistency with
> spacing elsewhere in the same files.
>
> Resulting fixup patch appended. I'd appreciate a quick look-over before
> I do a pull request. Current state at
> http://repo.or.cz/w/qemu/armbru.git branch qapi-not-next.
Had a quick look, and it seems fine to me.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict
2016-10-25 10:20 ` Daniel P. Berrange
@ 2016-10-25 12:29 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-25 12:29 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, Paolo Bonzini, Andreas Färber, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Tue, Oct 25, 2016 at 12:03:33PM +0200, Markus Armbruster wrote:
>> Max Reitz <mreitz@redhat.com> writes:
>>
>> > On 21.10.2016 11:58, Markus Armbruster wrote:
>> >> "Daniel P. Berrange" <berrange@redhat.com> writes:
>> >>
>> >>> On Tue, Oct 18, 2016 at 04:32:13PM +0200, Markus Armbruster wrote:
>> >>>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>> >>>>
>> >>>>> The qdict_flatten() method will take a dict whose elements are
>> >>>>> further nested dicts/lists and flatten them by concatenating
>> >>>>> keys.
>> >>>>>
>> >>>>> The qdict_crumple() method aims to do the reverse, taking a flat
>> >>>>> qdict, and turning it into a set of nested dicts/lists. It will
>> >>>>> apply nesting based on the key name, with a '.' indicating a
>> >>>>> new level in the hierarchy. If the keys in the nested structure
>> >>>>> are all numeric, it will create a list, otherwise it will create
>> >>>>> a dict.
>> >>>>>
>> >>>>> If the keys are a mixture of numeric and non-numeric, or the
>> >>>>> numeric keys are not in strictly ascending order, an error will
>> >>>>> be reported.
>> >>>>>
>> >>>>> As an example, a flat dict containing
>> >>>>>
>> >>>>> {
>> >>>>> 'foo.0.bar': 'one',
>> >>>>> 'foo.0.wizz': '1',
>> >>>>> 'foo.1.bar': 'two',
>> >>>>> 'foo.1.wizz': '2'
>> >>>>> }
>> >>>>>
>> >>>>> will get turned into a dict with one element 'foo' whose
>> >>>>> value is a list. The list elements will each in turn be
>> >>>>> dicts.
>> >>>>>
>> >>>>> {
>> >>>>> 'foo': [
>> >>>>> { 'bar': 'one', 'wizz': '1' },
>> >>>>> { 'bar': 'two', 'wizz': '2' }
>> >>>>> ],
>> >>>>> }
>> >>>>>
>> >>>>> If the key is intended to contain a literal '.', then it must
>> >>>>> be escaped as '..'. ie a flat dict
>> >>>>>
>> >>>>> {
>> >>>>> 'foo..bar': 'wizz',
>> >>>>> 'bar.foo..bar': 'eek',
>> >>>>> 'bar.hello': 'world'
>> >>>>> }
>> >>>>>
>> >>>>> Will end up as
>> >>>>>
>> >>>>> {
>> >>>>> 'foo.bar': 'wizz',
>> >>>>> 'bar': {
>> >>>>> 'foo.bar': 'eek',
>> >>>>> 'hello': 'world'
>> >>>>> }
>> >>>>> }
>> >>>>>
>> >>>>> The intent of this function is that it allows a set of QemuOpts
>> >>>>> to be turned into a nested data structure that mirrors the nesting
>> >>>>> used when the same object is defined over QMP.
>> >>>>>
>> >>>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> >>>>> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>> >>>>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >>>>> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> >>>>> ---
>> >>>>> include/qapi/qmp/qdict.h | 1 +
>> >>>>> qobject/qdict.c | 289 +++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>> tests/check-qdict.c | 261 ++++++++++++++++++++++++++++++++++++++++++
>> >>>>> 3 files changed, 551 insertions(+)
>> >>>>>
>> >>>>> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
>> >>>>> index 71b8eb0..e0d24e1 100644
>> >>>>> --- a/include/qapi/qmp/qdict.h
>> >>>>> +++ b/include/qapi/qmp/qdict.h
>> >>>>> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
>> >>>>> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
>> >>>>> void qdict_array_split(QDict *src, QList **dst);
>> >>>>> int qdict_array_entries(QDict *src, const char *subqdict);
>> >>>>> +QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
>> >>>>>
>> >>>>> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>> >>>>>
>> >>>>> diff --git a/qobject/qdict.c b/qobject/qdict.c
>> >>>>> index 60f158c..c38e90e 100644
>> >>>>> --- a/qobject/qdict.c
>> >>>>> +++ b/qobject/qdict.c
>> >>>> [...]
>> >>>>> +/**
>> >>>>> + * qdict_crumple:
>> >>>>> + * @src: the original flat dictionary (only scalar values) to crumple
>> >>>>> + * @recursive: true to recursively crumple nested dictionaries
>> >>>>
>> >>>> Is recursive=false used outside tests in this series?
>> >>>
>> >>> No, its not used.
>> >>>
>> >>> It was suggested in a way earlier version by Max, but not sure if his
>> >>> code uses it or not.
>> >>
>> >> In general, I prefer features to be added right before they're used, and
>> >> I really dislike adding features "just in case". YAGNI.
>> >>
>> >> Max, do you actually need this one? If yes, please explain your use
>> >> case.
>> >
>> > As far as I can tell from a quick glance, I made the point for v1 that
>> > qdict_crumple() could be simplified by using qdict_array_split() and
>> > qdict_array_entries().
>> >
>> > Dan then (correctly) said that using these functions would worsen
>> > runtime performance of qdict_crumple() and that instead we can replace
>> > qdict_array_split() by qdict_crumple(). However, for that to work, we
>> > need to be able to make qdict_crumple() non-recursive (because
>> > qdict_array_split() is non-recursive).
>> >
>> > Dan also said that in the long run we want to keep the tree structure in
>> > the block layer instead of flattening everything down which would rid us
>> > of several other QDict functions (and would make non-recursive behavior
>> > obsolete again). I believe that this is an idea for the (far?) future,
>> > as can be seen from the discussion you and others had on this very topic
>> > in this version here.
>> >
>> > However, clearly there are no patches out yet that replace
>> > qdict_array_split() by qdict_crumple(recursive=false), and I certainly
>> > won't write any until qdict_crumple() is merged.
>>
>> I prefer not to maintain code that isn't actually used. Since Dan is
>> enjoying a few days off, and the soft freeze is closing in, I'm
>> proposing to squash in the following:
>>
>> * Drop @recursive, in the most straightforward way possible.
>>
>> * Drop all tests that pass false to @recursive. This might reduce
>> coverage, but we can fix that on top.
>>
>> * While there, collapse some vertical whitespace, for consistency with
>> spacing elsewhere in the same files.
>>
>> Resulting fixup patch appended. I'd appreciate a quick look-over before
>> I do a pull request. Current state at
>> http://repo.or.cz/w/qemu/armbru.git branch qapi-not-next.
>
> Had a quick look, and it seems fine to me.
Applied to qapi-next, thanks!
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor Daniel P. Berrange
@ 2016-10-25 13:36 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-25 13:36 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The QmpOutputVisitor has no direct dependency on QMP. It is
> valid to use it anywhere that one wants a QObject. Rename it
> to better reflect its functionality as a generic QAPI
> to QObject converter.
>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> block/qapi.c | 4 +-
> blockdev.c | 4 +-
> docs/qapi-code-gen.txt | 2 +-
> ...p-output-visitor.h => qobject-output-visitor.h} | 10 +-
> qapi/Makefile.objs | 2 +-
> qapi/qapi-clone-visitor.c | 2 +-
> qapi/qmp-output-visitor.c | 256 ---------------------
> qapi/qobject-output-visitor.c | 254 ++++++++++++++++++++
> qemu-img.c | 8 +-
> qom/object_interfaces.c | 2 +-
> qom/qom-qobject.c | 4 +-
> scripts/qapi-commands.py | 4 +-
> scripts/qapi-event.py | 4 +-
> tests/.gitignore | 2 +-
> tests/Makefile.include | 8 +-
> tests/check-qnull.c | 4 +-
> ...put-visitor.c => test-outqobject-output-visitor.c} | 6 +-
> tests/test-string-output-visitor.c | 2 +-
> tests/test-visitor-serialization.c | 4 +-
> util/qemu-sockets.c | 2 +-
> 20 files changed, 291 insertions(+), 293 deletions(-)
> rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
> delete mode 100644 qapi/qmp-output-visitor.c
> create mode 100644 qapi/qobject-output-visitor.c
> rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)
Renaming the file and the identifiers at the same time defeats rename
detection, and thus git-log --follow. Since this would be quite
annoying, I'm going to split this patch into one that renames files and
one that renames C identifiers.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor Daniel P. Berrange
@ 2016-10-25 13:41 ` Markus Armbruster
0 siblings, 0 replies; 109+ messages in thread
From: Markus Armbruster @ 2016-10-25 13:41 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The QmpInputVisitor has no direct dependency on QMP. It is
> valid to use it anywhere that one has a QObject. Rename it
> to better reflect its functionality as a generic QObject
> to QAPI converter.
>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> docs/qapi-code-gen.txt | 2 +-
> ...qmp-input-visitor.h => qobject-input-visitor.h} | 10 +-
> include/qapi/visitor.h | 6 +-
> monitor.c | 2 +-
> qapi/Makefile.objs | 2 +-
> ...qmp-input-visitor.c => qobject-input-visitor.c} | 171 +++++++++++----------
> qmp.c | 4 +-
> qom/qom-qobject.c | 4 +-
> scripts/qapi-commands.py | 4 +-
> target-s390x/cpu_models.c | 4 +-
> tests/.gitignore | 4 +-
> tests/Makefile.include | 12 +-
> tests/check-qnull.c | 4 +-
> tests/test-qmp-commands.c | 4 +-
> ...-input-strict.c => test-qobject-input-strict.c} | 6 +-
> ...nput-visitor.c => test-qobject-input-visitor.c} | 6 +-
> tests/test-string-input-visitor.c | 2 +-
> tests/test-visitor-serialization.c | 4 +-
> util/qemu-sockets.c | 2 +-
> 19 files changed, 128 insertions(+), 125 deletions(-)
> rename include/qapi/{qmp-input-visitor.h => qobject-input-visitor.h} (63%)
> rename qapi/{qmp-input-visitor.c => qobject-input-visitor.c} (56%)
> rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
> rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (99%)
Here, git detects the rename, barely. I'll split this patch anyway, for
symmetry.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object Daniel P. Berrange
2016-10-19 16:54 ` Markus Armbruster
@ 2017-07-10 19:30 ` Manos Pitsidianakis
2017-07-11 8:10 ` Markus Armbruster
1 sibling, 1 reply; 109+ messages in thread
From: Manos Pitsidianakis @ 2017-07-10 19:30 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-block, armbru, mreitz, pbonzini, afaerber, berrange
[-- Attachment #1: Type: text/plain, Size: 330 bytes --]
Is there a specific reason this patch wasn't finished? If I'm not wrong
using non-scalar properties with -object is still not possible, yet
would be a very useful feature for drivers with UserCreatable objects.
Archive link since this is an old patch:
https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08252.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2017-07-10 19:30 ` Manos Pitsidianakis
@ 2017-07-11 8:10 ` Markus Armbruster
2017-07-11 14:44 ` Markus Armbruster
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2017-07-11 8:10 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, mreitz, pbonzini, afaerber, berrange
Manos Pitsidianakis <el13635@mail.ntua.gr> writes:
> Is there a specific reason this patch wasn't finished? If I'm not
> wrong using non-scalar properties with -object is still not possible,
> yet would be a very useful feature for drivers with UserCreatable
> objects.
>
> Archive link since this is an old patch:
> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08252.html
Yes, we want non-scalar properties with -object.
This series attempts to provide them by extending QemuOpts. The trouble
is that we've already pushed QemuOpts beyond its design limits, and this
series pushes it some more. We need to stop working around the design
limits of QemuOpts and start replacing them by something that serves our
current needs, as I explained in:
Subject: Re: [PATCH v14 00/19] QAPI/QOM work for non-scalar object properties
Message-ID: <87vawnnjpi.fsf@dusky.pond.sub.org>
https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg04895.html
I started the replacement work (merge commit 8746709). It provides
non-scalar properties for -blockdev. I stole Dan's PATCH 07 for it,
which became commit cbd8acf. See also the design thread
Subject: Non-flat command line option argument syntax
Message-ID: <87bmukmlau.fsf@dusky.pond.sub.org>
https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg00555.html
PATCH 02-06 have been merged separately (merge commit ede0cbe).
More work is needed to apply the solution for -blockdev to -object, and
I intend to work on it. A main difficulty is backwards compatibility to
all the ill-designed / accidental QemuOpts warts. We may have to accept
some incompatibility to make progress.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2017-07-11 8:10 ` Markus Armbruster
@ 2017-07-11 14:44 ` Markus Armbruster
2017-07-11 14:49 ` Daniel P. Berrange
0 siblings, 1 reply; 109+ messages in thread
From: Markus Armbruster @ 2017-07-11 14:44 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, mreitz, pbonzini, afaerber, berrange, Kevin Wolf
Markus Armbruster <armbru@redhat.com> writes:
> Manos Pitsidianakis <el13635@mail.ntua.gr> writes:
>
>> Is there a specific reason this patch wasn't finished? If I'm not
>> wrong using non-scalar properties with -object is still not possible,
>> yet would be a very useful feature for drivers with UserCreatable
>> objects.
>>
>> Archive link since this is an old patch:
>> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08252.html
>
> Yes, we want non-scalar properties with -object.
>
> This series attempts to provide them by extending QemuOpts. The trouble
> is that we've already pushed QemuOpts beyond its design limits, and this
> series pushes it some more. We need to stop working around the design
> limits of QemuOpts and start replacing them by something that serves our
> current needs, as I explained in:
>
> Subject: Re: [PATCH v14 00/19] QAPI/QOM work for non-scalar object properties
> Message-ID: <87vawnnjpi.fsf@dusky.pond.sub.org>
> https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg04895.html
>
> I started the replacement work (merge commit 8746709). It provides
> non-scalar properties for -blockdev. I stole Dan's PATCH 07 for it,
> which became commit cbd8acf. See also the design thread
>
> Subject: Non-flat command line option argument syntax
> Message-ID: <87bmukmlau.fsf@dusky.pond.sub.org>
> https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg00555.html
>
> PATCH 02-06 have been merged separately (merge commit ede0cbe).
>
> More work is needed to apply the solution for -blockdev to -object, and
> I intend to work on it. A main difficulty is backwards compatibility to
> all the ill-designed / accidental QemuOpts warts. We may have to accept
> some incompatibility to make progress.
Waiting for that work to be completed may be less than ideal for you.
Perhaps we can crack the simple part of the problem quickly, so you can
play with it while I still work on the harder parts.
Simple part: -object argument in JSON syntax, exactly as in QMP.
Example:
-object '{ "qom-type": "memory-backend-ram", "id": "mem1", "props": { "size": 4096 } }'
Non-scalar members of "props" should just work there.
Hard part: non-scalar properties in the traditional KEY=VALUE syntax,
using dotted key convention.
Sketch appended. If this would be useful for you, I can prepare proper
patches.
diff --git a/qapi-schema.json b/qapi-schema.json
index 37c4b95..82cb48d 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3538,15 +3538,23 @@
{ 'command': 'netdev_del', 'data': {'id': 'str'} }
##
-# @object-add:
+# @ObjectOptions:
#
-# Create a QOM object.
+# Options for creating an object.
#
# @qom-type: the class name for the object to be created
#
# @id: the name of the new object
#
# @props: a dictionary of properties to be passed to the backend
+##
+{ 'struct': 'ObjectOptions',
+ 'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
+
+##
+# @object-add:
+#
+# Create a QOM object.
#
# Returns: Nothing on success
# Error if @qom-type is not a valid class name
@@ -3562,7 +3570,7 @@
#
##
{ 'command': 'object-add',
- 'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
+ 'data': 'ObjectOptions' }
##
# @object-del:
diff --git a/vl.c b/vl.c
index d17c863..1a8a11d 100644
--- a/vl.c
+++ b/vl.c
@@ -2841,7 +2841,6 @@ static bool object_create_initial(const char *type)
return true;
}
-
/*
* The remainder of object creation happens after the
* creation of chardev, fsdev, net clients and device data types.
@@ -2851,6 +2850,37 @@ static bool object_create_delayed(const char *type)
return !object_create_initial(type);
}
+typedef struct ObjectOptionsQueueEntry {
+ ObjectOptions *oo;
+ Location loc;
+ QSIMPLEQ_ENTRY(ObjectOptionsQueueEntry) entry;
+} ObjectOptionsQueueEntry;
+
+typedef QSIMPLEQ_HEAD(ObjectOptionsQueue, ObjectOptionsQueueEntry) ObjectOptionsQueue;
+
+ObjectOptionsQueue oo_queue = QSIMPLEQ_HEAD_INITIALIZER(oo_queue);
+
+
+static void object_create(bool (*type_predicate)(const char *))
+{
+ ObjectOptionsQueueEntry *e;
+
+ QSIMPLEQ_FOREACH(e, &oo_queue, entry) {
+ if (!type_predicate(e->oo->qom_type)) {
+ continue;
+ }
+ loc_push_restore(&e->loc);
+ qmp_object_add(e->oo->qom_type, e->oo->id,
+ e->oo->has_props, e->oo->props, &error_fatal);
+ loc_pop(&e->loc);
+ }
+
+ if (qemu_opts_foreach(qemu_find_opts("object"),
+ user_creatable_add_opts_foreach,
+ type_predicate, NULL)) {
+ exit(1);
+ }
+}
static void set_memory_options(uint64_t *ram_slots, ram_addr_t *maxram_size,
MachineClass *mc)
@@ -4067,6 +4097,29 @@ int main(int argc, char **argv, char **envp)
#endif
break;
case QEMU_OPTION_object:
+ /*
+ * TODO Use qobject_input_visitor_new_str() instead of
+ * QemuOpts, not in addition to. Not done now because
+ * keyval_parse() isn't wart-compatible with QemuOpts.
+ */
+ if (optarg[0] == '{') {
+ Visitor *v;
+ ObjectOptionsQueueEntry *e;
+
+ v = qobject_input_visitor_new_str(optarg, "qom-type",
+ &err);
+ if (!v) {
+ error_report_err(err);
+ exit(1);
+ }
+
+ e = g_new(ObjectOptionsQueueEntry, 1);
+ visit_type_ObjectOptions(v, NULL, &e->oo, &error_fatal);
+ visit_free(v);
+ loc_save(&e->loc);
+ QSIMPLEQ_INSERT_TAIL(&oo_queue, e, entry);
+ break;
+ }
opts = qemu_opts_parse_noisily(qemu_find_opts("object"),
optarg, true);
if (!opts) {
@@ -4386,11 +4439,7 @@ int main(int argc, char **argv, char **envp)
page_size_init();
socket_init();
- if (qemu_opts_foreach(qemu_find_opts("object"),
- user_creatable_add_opts_foreach,
- object_create_initial, NULL)) {
- exit(1);
- }
+ object_create(object_create_initial);
if (qemu_opts_foreach(qemu_find_opts("chardev"),
chardev_init_func, NULL, NULL)) {
@@ -4503,10 +4552,14 @@ int main(int argc, char **argv, char **envp)
exit(1);
}
- if (qemu_opts_foreach(qemu_find_opts("object"),
- user_creatable_add_opts_foreach,
- object_create_delayed, NULL)) {
- exit(1);
+ object_create(object_create_delayed);
+
+ while (!QSIMPLEQ_EMPTY(&oo_queue)) {
+ ObjectOptionsQueueEntry *e = QSIMPLEQ_FIRST(&oo_queue);
+
+ QSIMPLEQ_REMOVE_HEAD(&oo_queue, entry);
+ qapi_free_ObjectOptions(e->oo);
+ g_free(e);
}
#ifdef CONFIG_TPM
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2017-07-11 14:44 ` Markus Armbruster
@ 2017-07-11 14:49 ` Daniel P. Berrange
2017-07-12 17:56 ` Manos Pitsidianakis
0 siblings, 1 reply; 109+ messages in thread
From: Daniel P. Berrange @ 2017-07-11 14:49 UTC (permalink / raw)
To: Markus Armbruster
Cc: Manos Pitsidianakis, qemu-devel, qemu-block, mreitz, pbonzini,
afaerber, Kevin Wolf
On Tue, Jul 11, 2017 at 04:44:46PM +0200, Markus Armbruster wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
> > Manos Pitsidianakis <el13635@mail.ntua.gr> writes:
> >
> >> Is there a specific reason this patch wasn't finished? If I'm not
> >> wrong using non-scalar properties with -object is still not possible,
> >> yet would be a very useful feature for drivers with UserCreatable
> >> objects.
> >>
> >> Archive link since this is an old patch:
> >> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08252.html
> >
> > Yes, we want non-scalar properties with -object.
> >
> > This series attempts to provide them by extending QemuOpts. The trouble
> > is that we've already pushed QemuOpts beyond its design limits, and this
> > series pushes it some more. We need to stop working around the design
> > limits of QemuOpts and start replacing them by something that serves our
> > current needs, as I explained in:
> >
> > Subject: Re: [PATCH v14 00/19] QAPI/QOM work for non-scalar object properties
> > Message-ID: <87vawnnjpi.fsf@dusky.pond.sub.org>
> > https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg04895.html
> >
> > I started the replacement work (merge commit 8746709). It provides
> > non-scalar properties for -blockdev. I stole Dan's PATCH 07 for it,
> > which became commit cbd8acf. See also the design thread
> >
> > Subject: Non-flat command line option argument syntax
> > Message-ID: <87bmukmlau.fsf@dusky.pond.sub.org>
> > https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg00555.html
> >
> > PATCH 02-06 have been merged separately (merge commit ede0cbe).
> >
> > More work is needed to apply the solution for -blockdev to -object, and
> > I intend to work on it. A main difficulty is backwards compatibility to
> > all the ill-designed / accidental QemuOpts warts. We may have to accept
> > some incompatibility to make progress.
>
> Waiting for that work to be completed may be less than ideal for you.
> Perhaps we can crack the simple part of the problem quickly, so you can
> play with it while I still work on the harder parts.
>
> Simple part: -object argument in JSON syntax, exactly as in QMP.
> Example:
>
> -object '{ "qom-type": "memory-backend-ram", "id": "mem1", "props": { "size": 4096 } }'
>
> Non-scalar members of "props" should just work there.
>
> Hard part: non-scalar properties in the traditional KEY=VALUE syntax,
> using dotted key convention.
>
> Sketch appended. If this would be useful for you, I can prepare proper
> patches.
Seems reasonable to enable the JSON syntax right now, as long as we are
confident it'll not need changes once we do the key/val syntax change too.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object
2017-07-11 14:49 ` Daniel P. Berrange
@ 2017-07-12 17:56 ` Manos Pitsidianakis
0 siblings, 0 replies; 109+ messages in thread
From: Manos Pitsidianakis @ 2017-07-12 17:56 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Markus Armbruster, qemu-devel, qemu-block, mreitz, pbonzini,
afaerber, Kevin Wolf
[-- Attachment #1: Type: text/plain, Size: 2998 bytes --]
On Tue, Jul 11, 2017 at 03:49:50PM +0100, Daniel P. Berrange wrote:
>On Tue, Jul 11, 2017 at 04:44:46PM +0200, Markus Armbruster wrote:
>> Markus Armbruster <armbru@redhat.com> writes:
>>
>> > Manos Pitsidianakis <el13635@mail.ntua.gr> writes:
>> >
>> >> Is there a specific reason this patch wasn't finished? If I'm not
>> >> wrong using non-scalar properties with -object is still not possible,
>> >> yet would be a very useful feature for drivers with UserCreatable
>> >> objects.
>> >>
>> >> Archive link since this is an old patch:
>> >> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08252.html
>> >
>> > Yes, we want non-scalar properties with -object.
>> >
>> > This series attempts to provide them by extending QemuOpts. The trouble
>> > is that we've already pushed QemuOpts beyond its design limits, and this
>> > series pushes it some more. We need to stop working around the design
>> > limits of QemuOpts and start replacing them by something that serves our
>> > current needs, as I explained in:
>> >
>> > Subject: Re: [PATCH v14 00/19] QAPI/QOM work for non-scalar object properties
>> > Message-ID: <87vawnnjpi.fsf@dusky.pond.sub.org>
>> > https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg04895.html
>> >
>> > I started the replacement work (merge commit 8746709). It provides
>> > non-scalar properties for -blockdev. I stole Dan's PATCH 07 for it,
>> > which became commit cbd8acf. See also the design thread
>> >
>> > Subject: Non-flat command line option argument syntax
>> > Message-ID: <87bmukmlau.fsf@dusky.pond.sub.org>
>> > https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg00555.html
>> >
>> > PATCH 02-06 have been merged separately (merge commit ede0cbe).
>> >
>> > More work is needed to apply the solution for -blockdev to -object, and
>> > I intend to work on it. A main difficulty is backwards compatibility to
>> > all the ill-designed / accidental QemuOpts warts. We may have to accept
>> > some incompatibility to make progress.
>>
>> Waiting for that work to be completed may be less than ideal for you.
>> Perhaps we can crack the simple part of the problem quickly, so you can
>> play with it while I still work on the harder parts.
>>
>> Simple part: -object argument in JSON syntax, exactly as in QMP.
>> Example:
>>
>> -object '{ "qom-type": "memory-backend-ram", "id": "mem1", "props": { "size": 4096 } }'
>>
>> Non-scalar members of "props" should just work there.
>>
>> Hard part: non-scalar properties in the traditional KEY=VALUE syntax,
>> using dotted key convention.
>>
>> Sketch appended. If this would be useful for you, I can prepare proper
>> patches.
>
>Seems reasonable to enable the JSON syntax right now, as long as we are
>confident it'll not need changes once we do the key/val syntax change too.
>
>Regards,
>Daniel
This would be useful indeed. If you end up sending the json patch, CC
me.
Thanks a lot!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
end of thread, other threads:[~2017-07-12 17:57 UTC | newest]
Thread overview: 109+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-30 14:45 [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 01/21] option: make parse_option_bool/number non-static Daniel P. Berrange
2016-10-21 16:55 ` Markus Armbruster
2016-10-21 17:12 ` Eric Blake
2016-10-21 17:51 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 02/21] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
2016-10-18 14:32 ` Markus Armbruster
2016-10-20 14:11 ` Daniel P. Berrange
2016-10-21 9:58 ` Markus Armbruster
2016-10-21 18:31 ` Max Reitz
2016-10-24 9:18 ` Markus Armbruster
2016-10-24 10:28 ` Daniel P. Berrange
2016-10-24 12:38 ` Markus Armbruster
2016-10-25 10:03 ` Markus Armbruster
2016-10-25 10:20 ` Daniel P. Berrange
2016-10-25 12:29 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 03/21] qapi: add trace events for visitor Daniel P. Berrange
2016-09-30 15:49 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-07 13:59 ` [Qemu-devel] " Markus Armbruster
2016-10-07 14:16 ` Daniel P. Berrange
2016-10-21 10:52 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 04/21] qapi: rename QmpInputVisitor to QObjectInputVisitor Daniel P. Berrange
2016-10-25 13:41 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 05/21] qapi: rename QmpOutputVisitor to QObjectOutputVisitor Daniel P. Berrange
2016-10-25 13:36 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 06/21] qapi: don't pass two copies of TestInputVisitorData to tests Daniel P. Berrange
2016-09-30 17:43 ` Eric Blake
2016-10-06 14:39 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 07/21] qapi: permit scalar type conversions in QObjectInputVisitor Daniel P. Berrange
2016-09-30 17:48 ` Eric Blake
2016-10-11 16:20 ` Markus Armbruster
2016-10-12 14:51 ` Markus Armbruster
2016-10-12 15:05 ` Markus Armbruster
2016-10-12 15:26 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 08/21] qapi: allow QObjectInputVisitor to be created with QemuOpts Daniel P. Berrange
2016-09-30 17:55 ` Eric Blake
2016-10-06 14:56 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-12 8:08 ` [Qemu-devel] " Markus Armbruster
2016-10-13 7:23 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 09/21] qapi: permit auto-creating single element lists Daniel P. Berrange
2016-09-30 17:59 ` Eric Blake
2016-10-12 9:18 ` Markus Armbruster
2016-10-20 14:23 ` Daniel P. Berrange
2016-10-21 11:58 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 10/21] qapi: permit auto-creating nested structs Daniel P. Berrange
2016-09-30 18:23 ` Eric Blake
2016-10-06 15:10 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-06 15:18 ` Daniel P. Berrange
2016-10-06 15:30 ` Kevin Wolf
2016-10-06 15:39 ` Daniel P. Berrange
2016-10-06 15:51 ` Kevin Wolf
2016-10-06 15:57 ` Daniel P. Berrange
2016-10-12 14:00 ` Markus Armbruster
2016-10-06 17:54 ` Eric Blake
2016-10-12 14:12 ` [Qemu-devel] " Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 11/21] qapi: add integer range support for QObjectInputVisitor Daniel P. Berrange
2016-09-30 19:45 ` Eric Blake
2016-10-12 15:50 ` Markus Armbruster
2016-10-12 16:03 ` [Qemu-devel] [Qemu-block] " Kevin Wolf
2016-10-12 18:14 ` Markus Armbruster
2016-10-20 14:28 ` [Qemu-devel] " Daniel P. Berrange
2016-10-21 10:58 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 12/21] option: allow qemu_opts_to_qdict to merge repeated options Daniel P. Berrange
2016-09-30 20:08 ` Eric Blake
2016-10-12 17:46 ` Markus Armbruster
2016-10-13 9:21 ` Markus Armbruster
2016-10-20 14:29 ` Daniel P. Berrange
2016-10-21 11:09 ` Markus Armbruster
2016-10-21 11:14 ` Daniel P. Berrange
2016-10-13 8:31 ` Markus Armbruster
2016-10-20 14:37 ` Daniel P. Berrange
2016-10-21 11:28 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 13/21] qdict: allow qdict_crumple to accept compound types as values Daniel P. Berrange
2016-10-13 12:35 ` Markus Armbruster
2016-10-13 14:46 ` Kevin Wolf
2016-10-17 14:50 ` Markus Armbruster
2016-10-17 15:43 ` Paolo Bonzini
2016-10-17 17:48 ` Markus Armbruster
2016-10-17 17:38 ` Eric Blake
2016-10-18 10:59 ` Markus Armbruster
2016-10-18 9:34 ` Kevin Wolf
2016-10-18 15:35 ` Markus Armbruster
2016-10-19 9:25 ` Kevin Wolf
2016-10-19 9:42 ` Daniel P. Berrange
2016-10-19 13:31 ` [Qemu-devel] [Qemu-block] " Eric Blake
2016-10-20 14:46 ` [Qemu-devel] " Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 14/21] qapi: allow repeated opts with qobject_input_visitor_new_opts Daniel P. Berrange
2016-10-18 17:13 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 15/21] qom: support non-scalar properties with -object Daniel P. Berrange
2016-10-19 16:54 ` Markus Armbruster
2017-07-10 19:30 ` Manos Pitsidianakis
2017-07-11 8:10 ` Markus Armbruster
2017-07-11 14:44 ` Markus Armbruster
2017-07-11 14:49 ` Daniel P. Berrange
2017-07-12 17:56 ` Manos Pitsidianakis
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 16/21] hmp: support non-scalar properties with object_add Daniel P. Berrange
2016-10-20 6:43 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 17/21] numa: convert to use QObjectInputVisitor for -numa Daniel P. Berrange
2016-10-20 6:57 ` Markus Armbruster
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 18/21] block: convert crypto driver to use QObjectInputVisitor Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 19/21] acpi: convert to QObjectInputVisitor for -acpi parsing Daniel P. Berrange
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 20/21] net: convert to QObjectInputVisitor for -net/-netdev parsing Daniel P. Berrange
2016-10-20 7:38 ` Markus Armbruster
2016-10-20 13:43 ` [Qemu-devel] [Qemu-block] " Eric Blake
2016-09-30 14:45 ` [Qemu-devel] [PATCH v14 21/21] qapi: delete unused OptsVisitor code Daniel P. Berrange
2016-09-30 15:45 ` [Qemu-devel] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties no-reply
2016-09-30 18:50 ` Eric Blake
2016-10-21 18:30 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.