[Qemu-devel] [PATCH 00/21] qapi: QMP dispatch and input visitor work

[Qemu-devel] [PATCH 00/21] qapi: QMP dispatch and input visitor work

[Markus Armbruster]

PATCH 10+11 have previously been posed as "[PATCH 0/2] qapi: Improve
qobject input visitor error reporting".

Markus Armbruster (21):
  qga: Fix crash on non-dictionary QMP argument
  libqtest: Work around a "QMP wants a newline" bug
  qmp-test: New, covering basic QMP protocol
  qmp: Dumb down how we run QMP command registration
  qmp: Clean up how we enforce capability negotiation
  qmp: Drop duplicated QMP command object checks
  qmp: Eliminate silly QERR_QMP_* macros
  qmp: Improve QMP dispatch error messages
  qapi: Improve a QObject input visitor error message
  qapi: Clean up after commit 3d344c2
  qapi: Make QObject input visitor set *list reliably
  qapi: Improve qobject input visitor error reporting
  qapi: Drop string input visitor method optional()
  qapi: Make string input and opts visitor require non-null input
  qom: Make object_property_set_qobject()'s input visitor strict
  test-qobject-input-visitor: Use strict visitor
  qapi: Drop unused non-strict qobject input visitor
  tests-qobject-input-strict: Merge into test-qobject-input-visitor
  tests: Cover partial input visit of list
  qapi: Make input visitors detect unvisited list tails
  qapi: Improve qobject visitor documentation

 block/nbd.c                           |   2 +-
 block/nfs.c                           |   2 +-
 block/ssh.c                           |   2 +-
 docs/qapi-code-gen.txt                |   2 +-
 hw/ppc/spapr_drc.c                    |   5 +
 include/monitor/monitor.h             |   1 +
 include/qapi/qmp/qerror.h             |   9 -
 include/qapi/qobject-input-visitor.h  |  39 +++-
 include/qapi/qobject-output-visitor.h |  35 +++-
 include/qapi/visitor-impl.h           |   7 +-
 include/qapi/visitor.h                |  19 +-
 include/qemu/module.h                 |   2 -
 monitor.c                             | 149 +++++--------
 qapi/opts-visitor.c                   |  12 ++
 qapi/qapi-visit-core.c                |   8 +
 qapi/qmp-dispatch.c                   |  22 +-
 qapi/qobject-input-visitor.c          | 209 ++++++++++++-------
 qapi/string-input-visitor.c           |  97 +++++----
 qapi/trace-events                     |   1 +
 qga/main.c                            |   2 +-
 qmp.c                                 |   2 +-
 qom/qom-qobject.c                     |   4 +-
 scripts/qapi-commands.py              |   7 +-
 scripts/qapi-visit.py                 |   3 +
 target/s390x/cpu_models.c             |   2 +-
 tests/Makefile.include                |   9 +-
 tests/check-qnull.c                   |   2 +-
 tests/libqtest.c                      |  29 ++-
 tests/libqtest.h                      |   8 +
 tests/qmp-test.c                      | 139 +++++++++++++
 tests/test-opts-visitor.c             |  41 ++++
 tests/test-qga.c                      |   2 +-
 tests/test-qmp-commands.c             |   4 +-
 tests/test-qobject-input-strict.c     | 381 ----------------------------------
 tests/test-qobject-input-visitor.c    | 218 ++++++++++++++++++-
 tests/test-string-input-visitor.c     |  20 ++
 tests/test-visitor-serialization.c    |   2 +-
 trace-events                          |   1 -
 vl.c                                  |   2 +-
 39 files changed, 823 insertions(+), 678 deletions(-)
 create mode 100644 tests/qmp-test.c
 delete mode 100644 tests/test-qobject-input-strict.c

-- 
2.7.4

[Qemu-devel] [PATCH 01/21] qga: Fix crash on non-dictionary QMP argument

[Markus Armbruster]

The value of key 'arguments' must be a JSON object.  qemu-ga neglects
to check, and crashes.  To reproduce, send

    { 'execute': 'guest-sync', 'arguments': [] }

to qemu-ga.

do_qmp_dispatch() uses qdict_get_qdict() to get the arguments.  When
not a JSON object, this gets a null pointer, which flows through the
generated marshalling function to qobject_input_visitor_new(), where
it fails the assertion.  qmp_dispatch_check_obj() needs to catch this
error.

QEMU isn't affected, because it runs qmp_check_input_obj() first,
which basically duplicates qmp_check_input_obj()'s checks, plus the
missing one.

Fix by copying the missing one from qmp_check_input_obj() to
qmp_dispatch_check_obj().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 qapi/qmp-dispatch.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 48bec20..621922f 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -47,7 +47,13 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
                 return NULL;
             }
             has_exec_key = true;
-        } else if (strcmp(arg_name, "arguments")) {
+        } else if (!strcmp(arg_name, "arguments")) {
+            if (qobject_type(arg_obj) != QTYPE_QDICT) {
+                error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
+                           "arguments", "object");
+                return NULL;
+            }
+        } else {
             error_setg(errp, QERR_QMP_EXTRA_MEMBER, arg_name);
             return NULL;
         }
-- 
2.7.4

[Qemu-devel] [PATCH 02/21] libqtest: Work around a "QMP wants a newline" bug

[Markus Armbruster]

The next commit is going to add a test that calls qmp("null").
Curiously, this hangs.  Here's why.

qmp_fd_sendv() doesn't send newlines.  Not even when @fmt contains
some.  At first glance, the QMP parser seems to be fine with that.
However, it turns out that it fails to react to input until it sees
either a newline, an object or an array.  To reproduce, feed to a QMP
monitor like this:

    $ echo -n 'null' | socat UNIX:/work/armbru/images/test-qmp STDIO
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 8, "major": 2}, "package": " (v2.8.0-1195-gf84141e-dirty)"}, "capabilities": []}}

No output after the greeting.

Add a newline:

    $ echo 'null' | socat UNIX:/work/armbru/images/test-qmp STDIO
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 8, "major": 2}, "package": " (v2.8.0-1195-gf84141e-dirty)"}, "capabilities": []}}
    {"error": {"class": "GenericError", "desc": "Expected 'object' in QMP input"}}

Correct output for input 'null'.

Add an object instead:

    $ echo -n 'null { "execute": "qmp_capabilities" }' | socat UNIX:qmp-socket STDIO
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 8, "major": 2}, "package": " (v2.8.0-1195-gf84141e-dirty)"}, "capabilities": []}}
    {"error": {"class": "GenericError", "desc": "Expected 'object' in QMP input"}}
    {"return": {}}

Also correct output.

Work around this QMP bug by having qmp_fd_sendv() append a newline.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/libqtest.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/tests/libqtest.c b/tests/libqtest.c
index e54354d..ad23ce9 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -442,14 +442,20 @@ void qmp_fd_sendv(int fd, const char *fmt, va_list ap)
     if (qobj) {
         int log = getenv("QTEST_LOG") != NULL;
         QString *qstr = qobject_to_json(qobj);
-        const char *str = qstring_get_str(qstr);
-        size_t size = qstring_get_length(qstr);
+        const char *str;
+
+        /*
+         * BUG: QMP doesn't react to input until it sees a newline, an
+         * object, or an array.  Work-around: give it a newline.
+         */
+        qstring_append_chr(qstr, '\n');
+        str = qstring_get_str(qstr);
 
         if (log) {
             fprintf(stderr, "%s", str);
         }
         /* Send QMP request */
-        socket_send(fd, str, size);
+        socket_send(fd, str, qstring_get_length(qstr));
 
         QDECREF(qstr);
         qobject_decref(qobj);
-- 
2.7.4

[Qemu-devel] [PATCH 03/21] qmp-test: New, covering basic QMP protocol

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/Makefile.include |   5 +-
 tests/libqtest.c       |  17 ++++--
 tests/libqtest.h       |   8 +++
 tests/qmp-test.c       | 139 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 163 insertions(+), 6 deletions(-)
 create mode 100644 tests/qmp-test.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e60bb6c..b212150 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -133,7 +133,9 @@ check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 # All QTests for now are POSIX-only, but the dependencies are
 # really in libqtest, not in the testcases themselves.
 
-check-qtest-generic-y = tests/device-introspect-test$(EXESUF)
+check-qtest-generic-y = tests/qmp-test$(EXESUF)
+gcov-files-generic-y = monitor.c qapi/qmp-dispatch.c
+check-qtest-generic-y += tests/device-introspect-test$(EXESUF)
 gcov-files-generic-y = qdev-monitor.c qmp.c
 
 gcov-files-ipack-y += hw/ipack/ipack.c
@@ -654,6 +656,7 @@ libqos-imx-obj-y = $(libqos-obj-y) tests/libqos/i2c-imx.o
 libqos-usb-obj-y = $(libqos-spapr-obj-y) $(libqos-pc-obj-y) tests/libqos/usb.o
 libqos-virtio-obj-y = $(libqos-spapr-obj-y) $(libqos-pc-obj-y) tests/libqos/virtio.o tests/libqos/virtio-pci.o tests/libqos/virtio-mmio.o tests/libqos/malloc-generic.o
 
+tests/qmp-test$(EXESUF): tests/qmp-test.o
 tests/device-introspect-test$(EXESUF): tests/device-introspect-test.o
 tests/rtc-test$(EXESUF): tests/rtc-test.o
 tests/m48t59-test$(EXESUF): tests/m48t59-test.o
diff --git a/tests/libqtest.c b/tests/libqtest.c
index ad23ce9..cf27afc 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -149,7 +149,7 @@ void qtest_add_abrt_handler(GHookFunc fn, const void *data)
     g_hook_prepend(&abrt_hooks, hook);
 }
 
-QTestState *qtest_init(const char *extra_args)
+QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
 {
     QTestState *s;
     int sock, qmpsock, i;
@@ -204,10 +204,6 @@ QTestState *qtest_init(const char *extra_args)
         s->irq_level[i] = false;
     }
 
-    /* Read the QMP greeting and then do the handshake */
-    qtest_qmp_discard_response(s, "");
-    qtest_qmp_discard_response(s, "{ 'execute': 'qmp_capabilities' }");
-
     if (getenv("QTEST_STOP")) {
         kill(s->qemu_pid, SIGSTOP);
     }
@@ -219,6 +215,17 @@ QTestState *qtest_init(const char *extra_args)
     return s;
 }
 
+QTestState *qtest_init(const char *extra_args)
+{
+    QTestState *s = qtest_init_without_qmp_handshake(extra_args);
+
+    /* Read the QMP greeting and then do the handshake */
+    qtest_qmp_discard_response(s, "");
+    qtest_qmp_discard_response(s, "{ 'execute': 'qmp_capabilities' }");
+
+    return s;
+}
+
 void qtest_quit(QTestState *s)
 {
     qtest_instances = g_list_remove(qtest_instances, s);
diff --git a/tests/libqtest.h b/tests/libqtest.h
index 90f182e..23bd76c 100644
--- a/tests/libqtest.h
+++ b/tests/libqtest.h
@@ -32,6 +32,14 @@ extern QTestState *global_qtest;
 QTestState *qtest_init(const char *extra_args);
 
 /**
+ * qtest_init:
+ * @extra_args: other arguments to pass to QEMU.
+ *
+ * Returns: #QTestState instance.
+ */
+QTestState *qtest_init_without_qmp_handshake(const char *extra_args);
+
+/**
  * qtest_quit:
  * @s: #QTestState instance to operate on.
  *
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
new file mode 100644
index 0000000..405e49e
--- /dev/null
+++ b/tests/qmp-test.c
@@ -0,0 +1,139 @@
+/*
+ * QMP protocol test cases
+ *
+ * Copyright (c) 2017 Red Hat Inc.
+ *
+ * Authors:
+ *  Markus Armbruster <armbru@redhat.com>,
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+#include "qapi-visit.h"
+#include "qapi/error.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qapi/visitor.h"
+
+const char common_args[] = "-nodefaults -machine none";
+
+static const char *get_error_class(QDict *resp)
+{
+    QDict *error = qdict_get_qdict(resp, "error");
+    const char *desc = qdict_get_try_str(error, "desc");
+
+    g_assert(desc);
+    return error ? qdict_get_try_str(error, "class") : NULL;
+}
+
+static void test_version(QObject *version)
+{
+    Visitor *v;
+    VersionInfo *vinfo;
+
+    g_assert(version);
+    v = qobject_input_visitor_new(version, true);
+    visit_type_VersionInfo(v, "version", &vinfo, &error_abort);
+    qapi_free_VersionInfo(vinfo);
+    visit_free(v);
+}
+
+static void test_malformed(void)
+{
+    QDict *resp;
+
+    /* Not even a dictionary */
+    resp = qmp("null");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    QDECREF(resp);
+
+    /* No "execute" key */
+    resp = qmp("{}");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    QDECREF(resp);
+
+    /* "execute" isn't a string */
+    resp = qmp("{ 'execute': true }");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    QDECREF(resp);
+
+    /* "arguments" isn't a dictionary */
+    resp = qmp("{ 'execute': 'no-such-cmd', 'arguments': [] }");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    QDECREF(resp);
+
+    /* extra key */
+    resp = qmp("{ 'execute': 'no-such-cmd', 'extra': true }");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    QDECREF(resp);
+}
+
+static void test_qmp_protocol(void)
+{
+    QDict *resp, *q, *ret;
+    QList *capabilities;
+
+    global_qtest = qtest_init_without_qmp_handshake(common_args);
+
+    /* Test greeting */
+    resp = qmp_receive();
+    q = qdict_get_qdict(resp, "QMP");
+    g_assert(q);
+    test_version(qdict_get(q, "version"));
+    capabilities = qdict_get_qlist(q, "capabilities");
+    g_assert(capabilities && qlist_empty(capabilities));
+    QDECREF(resp);
+
+    /* Test valid command before handshake */
+    resp = qmp("{ 'execute': 'query-version' }");
+    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
+    QDECREF(resp);
+
+    /* Test malformed commands before handshake */
+    test_malformed();
+
+    /* Test handshake */
+    resp = qmp("{ 'execute': 'qmp_capabilities' }");
+    ret = qdict_get_qdict(resp, "return");
+    g_assert(ret && !qdict_size(ret));
+    QDECREF(resp);
+
+    /* Test repeated handshake */
+    resp = qmp("{ 'execute': 'qmp_capabilities' }");
+    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
+    QDECREF(resp);
+
+    /* Test valid command */
+    resp = qmp("{ 'execute': 'query-version' }");
+    test_version(qdict_get(resp, "return"));
+    QDECREF(resp);
+
+    /* Test malformed commands */
+    test_malformed();
+
+    /* Test 'id' */
+    resp = qmp("{ 'execute': 'query-name', 'id': 'cookie#1' }");
+    ret = qdict_get_qdict(resp, "return");
+    g_assert(ret);
+    g_assert_cmpstr(qdict_get_try_str(resp, "id"), ==, "cookie#1");
+    QDECREF(resp);
+
+    /* Test command failure with 'id' */
+    resp = qmp("{ 'execute': 'human-monitor-command', 'id': 2 }");
+    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
+    g_assert_cmpint(qdict_get_int(resp, "id"), ==, 2);
+    QDECREF(resp);
+
+    qtest_end();
+}
+
+int main(int argc, char *argv[])
+{
+    g_test_init(&argc, &argv, NULL);
+
+    qtest_add_func("qmp/protocol", test_qmp_protocol);
+
+    return g_test_run();
+}
-- 
2.7.4

[Qemu-devel] [PATCH 04/21] qmp: Dumb down how we run QMP command registration

[Markus Armbruster]

The way we get QMP commands registered is high tech:

* qapi-commands.py generates qmp_init_marshal() that does the actual work

* it also generates the magic to register it as a MODULE_INIT_QAPI
  function, so it runs when someone calls
  module_call_init(MODULE_INIT_QAPI)

* main() calls module_call_init()

QEMU needs to register a few non-qapified commands.  Same high tech
works: monitor.c has its own qmp_init_marshal() along with the magic
to make it run in module_call_init(MODULE_INIT_QAPI).

QEMU also needs to unregister commands that are not wanted in this
build's configuration (commit 5032a16).  Simple enough:
qmp_unregister_commands_hack().  The difficulty is to make it run
after the generated qmp_init_marshal().  We can't simply run it in
monitor.c's qmp_init_marshal(), because the order in which the
registered functions run is indeterminate.  So qmp_init_marshal()
registers qmp_init_marshal() separately.  Since registering *appends*
to the list of registered functions, this will make it run after all
the functions that have been registered already.

I suspect it takes a long and expensive computer science education to
not find this silly.

Dumb it down as follows:

* Drop MODULE_INIT_QAPI entirely

* Give the generated qmp_init_marshal() external linkage.

* Call it instead of module_call_init(MODULE_INIT_QAPI)

* Except in QEMU proper, call new monitor_init_qmp_commands() that in
  turn calls the generated qmp_init_marshal(), registers the
  additional commands and unregisters the unwanted ones.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/monitor/monitor.h | 1 +
 include/qemu/module.h     | 2 --
 monitor.c                 | 9 ++++-----
 qga/main.c                | 2 +-
 scripts/qapi-commands.py  | 5 ++---
 tests/test-qmp-commands.c | 2 +-
 vl.c                      | 2 +-
 7 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index e64b944..d2b3aaf 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -16,6 +16,7 @@ extern Monitor *cur_mon;
 
 bool monitor_cur_is_qmp(void);
 
+void monitor_init_qmp_commands(void);
 void monitor_init(Chardev *chr, int flags);
 void monitor_cleanup(void);
 
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 877cca7..56dd218 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -42,7 +42,6 @@ static void __attribute__((constructor)) do_qemu_init_ ## function(void)    \
 typedef enum {
     MODULE_INIT_BLOCK,
     MODULE_INIT_OPTS,
-    MODULE_INIT_QAPI,
     MODULE_INIT_QOM,
     MODULE_INIT_TRACE,
     MODULE_INIT_MAX
@@ -50,7 +49,6 @@ typedef enum {
 
 #define block_init(function) module_init(function, MODULE_INIT_BLOCK)
 #define opts_init(function) module_init(function, MODULE_INIT_OPTS)
-#define qapi_init(function) module_init(function, MODULE_INIT_QAPI)
 #define type_init(function) module_init(function, MODULE_INIT_QOM)
 #define trace_init(function) module_init(function, MODULE_INIT_TRACE)
 
diff --git a/monitor.c b/monitor.c
index 56867e8..5be71d6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -995,8 +995,10 @@ static void qmp_unregister_commands_hack(void)
 #endif
 }
 
-static void qmp_init_marshal(void)
+void monitor_init_qmp_commands(void)
 {
+    qmp_init_marshal();
+
     qmp_register_command("query-qmp-schema", qmp_query_qmp_schema,
                          QCO_NO_OPTIONS);
     qmp_register_command("device_add", qmp_device_add,
@@ -1004,12 +1006,9 @@ static void qmp_init_marshal(void)
     qmp_register_command("netdev_add", qmp_netdev_add,
                          QCO_NO_OPTIONS);
 
-    /* call it after the rest of qapi_init() */
-    register_module_init(qmp_unregister_commands_hack, MODULE_INIT_QAPI);
+    qmp_unregister_commands_hack();
 }
 
-qapi_init(qmp_init_marshal);
-
 /* set the current CPU defined by the user */
 int monitor_set_cpu(int cpu_index)
 {
diff --git a/qga/main.c b/qga/main.c
index 538e4ee..6f8c614 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -1321,7 +1321,7 @@ int main(int argc, char **argv)
 
     config->log_level = G_LOG_LEVEL_ERROR | G_LOG_LEVEL_CRITICAL;
 
-    module_call_init(MODULE_INIT_QAPI);
+    qmp_init_marshal();
 
     init_dfl_pathnames();
     config_load(config);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 09e8467..a75946f 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -208,14 +208,12 @@ def gen_register_command(name, success_response):
 def gen_registry(registry):
     ret = mcgen('''
 
-static void qmp_init_marshal(void)
+void qmp_init_marshal(void)
 {
 ''')
     ret += registry
     ret += mcgen('''
 }
-
-qapi_init(qmp_init_marshal);
 ''')
     return ret
 
@@ -308,6 +306,7 @@ fdecl.write(mcgen('''
 #include "qapi/qmp/qdict.h"
 #include "qapi/error.h"
 
+void qmp_init_marshal(void);
 ''',
                   prefix=prefix))
 
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index ff94481..c4e20d1 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -273,7 +273,7 @@ int main(int argc, char **argv)
     g_test_add_func("/0.15/dealloc_types", test_dealloc_types);
     g_test_add_func("/0.15/dealloc_partial", test_dealloc_partial);
 
-    module_call_init(MODULE_INIT_QAPI);
+    qmp_init_marshal();
     g_test_run();
 
     return 0;
diff --git a/vl.c b/vl.c
index b5d0a19..a0634b0 100644
--- a/vl.c
+++ b/vl.c
@@ -2966,7 +2966,7 @@ int main(int argc, char **argv, char **envp)
     qemu_init_exec_dir(argv[0]);
 
     module_call_init(MODULE_INIT_QOM);
-    module_call_init(MODULE_INIT_QAPI);
+    monitor_init_qmp_commands();
 
     qemu_add_opts(&qemu_drive_opts);
     qemu_add_drive_opts(&qemu_legacy_drive_opts);
-- 
2.7.4

[Qemu-devel] [PATCH 05/21] qmp: Clean up how we enforce capability negotiation

[Markus Armbruster]

To enforce capability negotiation before normal operation,
handle_qmp_command() inspects every command before it's handed off to
qmp_dispatch().  This is a bit of a layering violation, and results in
duplicated code.

Before capability negotiation (!cur_mon->in_command_mode), we fail
commands other than "qmp_capabilities".  This is what enforces
capability negotiation.

Afterwards, we fail command "qmp_capabilities".

Clean this up as follows.

The obvious place to fail a command is the command itself, so move the
"afterwards" check to qmp_qmp_capabilities().

We do the "before" check in every other command, but that would be
bothersome.  Instead, start without all the other commands, by
registering only qmp_qmp_capabilities().  Register the others in
qmp_qmp_capabilities().

Additionally, replace the generic human-readable error message for
CommandNotFound by one that reminds the user to run qmp_capabilities.
Without that, we'd regress commit 2d5a834.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c | 70 ++++++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 40 insertions(+), 30 deletions(-)

diff --git a/monitor.c b/monitor.c
index 5be71d6..dcf2de7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -563,11 +563,6 @@ static void monitor_qapi_event_init(void)
     qmp_event_set_func_emit(monitor_qapi_event_queue);
 }
 
-void qmp_qmp_capabilities(Error **errp)
-{
-    cur_mon->qmp.in_command_mode = true;
-}
-
 static void handle_hmp_command(Monitor *mon, const char *cmdline);
 
 static void monitor_data_init(Monitor *mon)
@@ -995,7 +990,7 @@ static void qmp_unregister_commands_hack(void)
 #endif
 }
 
-void monitor_init_qmp_commands(void)
+static void monitor_init_real_qmp_commands(void)
 {
     qmp_init_marshal();
 
@@ -1009,6 +1004,32 @@ void monitor_init_qmp_commands(void)
     qmp_unregister_commands_hack();
 }
 
+void monitor_init_qmp_commands(void)
+{
+    /*
+     * Start with just qmp_capabilities, to enforce capability
+     * negotiation.  qmp_capabilities will register the other
+     * commands.  See also the error message replacement hack in
+     * handle_qmp_command().
+     */
+    qmp_register_command("qmp_capabilities",
+                         qmp_marshal_qmp_capabilities, QCO_NO_OPTIONS);
+}
+
+void qmp_qmp_capabilities(Error **errp)
+{
+    if (cur_mon->qmp.in_command_mode) {
+        error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
+                  "Capabilities negotiation is already complete, command "
+                  "ignored");
+        return;
+    }
+
+    cur_mon->qmp.in_command_mode = true;
+    qmp_unregister_command("qmp_capabilities");
+    monitor_init_real_qmp_commands();
+}
+
 /* set the current CPU defined by the user */
 int monitor_set_cpu(int cpu_index)
 {
@@ -3676,26 +3697,6 @@ static int monitor_can_read(void *opaque)
     return (mon->suspend_cnt == 0) ? 1 : 0;
 }
 
-static bool invalid_qmp_mode(const Monitor *mon, const char *cmd,
-                             Error **errp)
-{
-    bool is_cap = g_str_equal(cmd, "qmp_capabilities");
-
-    if (is_cap && mon->qmp.in_command_mode) {
-        error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
-                  "Capabilities negotiation is already complete, command "
-                  "'%s' ignored", cmd);
-        return true;
-    }
-    if (!is_cap && !mon->qmp.in_command_mode) {
-        error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
-                  "Expecting capabilities negotiation with "
-                  "'qmp_capabilities' before command '%s'", cmd);
-        return true;
-    }
-    return false;
-}
-
 /*
  * Input object checking rules
  *
@@ -3781,12 +3782,21 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     cmd_name = qdict_get_str(qdict, "execute");
     trace_handle_qmp_command(mon, cmd_name);
 
-    if (invalid_qmp_mode(mon, cmd_name, &err)) {
-        goto err_out;
-    }
-
     rsp = qmp_dispatch(req);
 
+    qdict = qdict_get_qdict(qobject_to_qdict(rsp), "error");
+    if (qdict) {
+        if (!mon->qmp.in_command_mode
+            && !strcmp(qdict_get_try_str(qdict, "class"),
+                       QapiErrorClass_lookup[ERROR_CLASS_COMMAND_NOT_FOUND])) {
+            /* Provide a more useful error message */
+            qdict_del(qdict, "desc");
+            qdict_put(qdict, "desc",
+                      qstring_from_str("Expecting capabilities negotiation"
+                                       " with 'qmp_capabilities'"));
+        }
+    }
+
 err_out:
     if (err) {
         qdict = qdict_new();
-- 
2.7.4

[Qemu-devel] [PATCH 06/21] qmp: Drop duplicated QMP command object checks

[Markus Armbruster]

qmp_check_input_obj() duplicates qmp_dispatch_check_obj(), except the
latter screws up an error message.  handle_qmp_command() runs first
the former, then the latter via qmp_dispatch(), masking the screwup.

qemu-ga also masks the screwup, because it also duplicates checks,
just differently.

qmp_check_input_obj() exists because handle_qmp_command() needs to
examine the command before dispatching it.  The previous commit got
rid of this need, except for a tracepoint, and a bit of "id" code that
relies on qdict not being null.

Fix up the error message in qmp_dispatch_check_obj(), drop
qmp_check_input_obj() and the tracepoint.  Protect the "id" code with
a conditional.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c           | 74 +++++------------------------------------------------
 qapi/qmp-dispatch.c |  3 +--
 trace-events        |  1 -
 3 files changed, 7 insertions(+), 71 deletions(-)

diff --git a/monitor.c b/monitor.c
index dcf2de7..d83888d 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3697,67 +3697,10 @@ static int monitor_can_read(void *opaque)
     return (mon->suspend_cnt == 0) ? 1 : 0;
 }
 
-/*
- * Input object checking rules
- *
- * 1. Input object must be a dict
- * 2. The "execute" key must exist
- * 3. The "execute" key must be a string
- * 4. If the "arguments" key exists, it must be a dict
- * 5. If the "id" key exists, it can be anything (ie. json-value)
- * 6. Any argument not listed above is considered invalid
- */
-static QDict *qmp_check_input_obj(QObject *input_obj, Error **errp)
-{
-    const QDictEntry *ent;
-    int has_exec_key = 0;
-    QDict *input_dict;
-
-    input_dict = qobject_to_qdict(input_obj);
-    if (!input_dict) {
-        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT, "object");
-        return NULL;
-    }
-
-
-    for (ent = qdict_first(input_dict); ent; ent = qdict_next(input_dict, ent)){
-        const char *arg_name = qdict_entry_key(ent);
-        const QObject *arg_obj = qdict_entry_value(ent);
-
-        if (!strcmp(arg_name, "execute")) {
-            if (qobject_type(arg_obj) != QTYPE_QSTRING) {
-                error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
-                           "execute", "string");
-                return NULL;
-            }
-            has_exec_key = 1;
-        } else if (!strcmp(arg_name, "arguments")) {
-            if (qobject_type(arg_obj) != QTYPE_QDICT) {
-                error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
-                           "arguments", "object");
-                return NULL;
-            }
-        } else if (!strcmp(arg_name, "id")) {
-            /* Any string is acceptable as "id", so nothing to check */
-        } else {
-            error_setg(errp, QERR_QMP_EXTRA_MEMBER, arg_name);
-            return NULL;
-        }
-    }
-
-    if (!has_exec_key) {
-        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT, "execute");
-        return NULL;
-    }
-
-    return input_dict;
-}
-
 static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
     QObject *req, *rsp = NULL, *id = NULL;
     QDict *qdict = NULL;
-    const char *cmd_name;
     Monitor *mon = cur_mon;
     Error *err = NULL;
 
@@ -3770,17 +3713,12 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err_out;
     }
 
-    qdict = qmp_check_input_obj(req, &err);
-    if (!qdict) {
-        goto err_out;
-    }
-
-    id = qdict_get(qdict, "id");
-    qobject_incref(id);
-    qdict_del(qdict, "id");
-
-    cmd_name = qdict_get_str(qdict, "execute");
-    trace_handle_qmp_command(mon, cmd_name);
+    qdict = qobject_to_qdict(req);
+    if (qdict) {
+        id = qdict_get(qdict, "id");
+        qobject_incref(id);
+        qdict_del(qdict, "id");
+    } /* else will fail qmp_dispatch() */
 
     rsp = qmp_dispatch(req);
 
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 621922f..4fc1122 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -30,8 +30,7 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 
     dict = qobject_to_qdict(request);
     if (!dict) {
-        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT,
-                   "request is not a dictionary");
+        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT, "object");
         return NULL;
     }
 
diff --git a/trace-events b/trace-events
index 7288557..b07a09b 100644
--- a/trace-events
+++ b/trace-events
@@ -65,7 +65,6 @@ xen_remap_bucket(uint64_t index) "index %#"PRIx64
 xen_map_cache_return(void* ptr) "%p"
 
 # monitor.c
-handle_qmp_command(void *mon, const char *cmd_name) "mon %p cmd_name \"%s\""
 monitor_protocol_event_handler(uint32_t event, void *qdict) "event=%d data=%p"
 monitor_protocol_event_emit(uint32_t event, void *data) "event=%d data=%p"
 monitor_protocol_event_queue(uint32_t event, void *qdict, uint64_t rate) "event=%d data=%p rate=%" PRId64
-- 
2.7.4

[Qemu-devel] [PATCH 07/21] qmp: Eliminate silly QERR_QMP_* macros

[Markus Armbruster]

The QERR_ macros are leftovers from the days of "rich" error objects.

QERR_QMP_BAD_INPUT_OBJECT, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
QERR_QMP_EXTRA_MEMBER are used in just one place now, except for one
use that has crept into qobject-input-visitor.c.

Drop these macros, to make the (bad) error messages more visible.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/qerror.h    |  9 ---------
 qapi/qmp-dispatch.c          | 13 +++++++------
 qapi/qobject-input-visitor.c |  3 ++-
 3 files changed, 9 insertions(+), 16 deletions(-)

diff --git a/include/qapi/qmp/qerror.h b/include/qapi/qmp/qerror.h
index 6586c9f..c82360f 100644
--- a/include/qapi/qmp/qerror.h
+++ b/include/qapi/qmp/qerror.h
@@ -82,15 +82,6 @@
 #define QERR_QGA_COMMAND_FAILED \
     "Guest agent command failed, error was '%s'"
 
-#define QERR_QMP_BAD_INPUT_OBJECT \
-    "Expected '%s' in QMP input"
-
-#define QERR_QMP_BAD_INPUT_OBJECT_MEMBER \
-    "QMP input object member '%s' expects '%s'"
-
-#define QERR_QMP_EXTRA_MEMBER \
-    "QMP input object member '%s' is unexpected"
-
 #define QERR_SET_PASSWD_FAILED \
     "Could not set password"
 
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 4fc1122..377ebb5 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -30,7 +30,7 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 
     dict = qobject_to_qdict(request);
     if (!dict) {
-        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT, "object");
+        error_setg(errp, "Expected '%s' in QMP input", "object");
         return NULL;
     }
 
@@ -41,25 +41,26 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 
         if (!strcmp(arg_name, "execute")) {
             if (qobject_type(arg_obj) != QTYPE_QSTRING) {
-                error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT_MEMBER, "execute",
-                           "string");
+                error_setg(errp, "QMP input object member '%s' expects '%s'",
+                           "execute", "string");
                 return NULL;
             }
             has_exec_key = true;
         } else if (!strcmp(arg_name, "arguments")) {
             if (qobject_type(arg_obj) != QTYPE_QDICT) {
-                error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
+                error_setg(errp, "QMP input object member '%s' expects '%s'",
                            "arguments", "object");
                 return NULL;
             }
         } else {
-            error_setg(errp, QERR_QMP_EXTRA_MEMBER, arg_name);
+            error_setg(errp, "QMP input object member '%s' is unexpected",
+                       arg_name);
             return NULL;
         }
     }
 
     if (!has_exec_key) {
-        error_setg(errp, QERR_QMP_BAD_INPUT_OBJECT, "execute");
+        error_setg(errp, "Expected '%s' in QMP input", "execute");
         return NULL;
     }
 
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 0063327..ed6c24c 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -140,7 +140,8 @@ static void qobject_input_check_struct(Visitor *v, Error **errp)
 
             g_hash_table_iter_init(&iter, top_ht);
             if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
-                error_setg(errp, QERR_QMP_EXTRA_MEMBER, key);
+                error_setg(errp, "QMP input object member '%s' is unexpected",
+                           key);
             }
         }
     }
-- 
2.7.4

[Qemu-devel] [PATCH 08/21] qmp: Improve QMP dispatch error messages

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qmp-dispatch.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 377ebb5..4610b6d 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -30,7 +30,7 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 
     dict = qobject_to_qdict(request);
     if (!dict) {
-        error_setg(errp, "Expected '%s' in QMP input", "object");
+        error_setg(errp, "QMP input must be a JSON object");
         return NULL;
     }
 
@@ -41,15 +41,17 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 
         if (!strcmp(arg_name, "execute")) {
             if (qobject_type(arg_obj) != QTYPE_QSTRING) {
-                error_setg(errp, "QMP input object member '%s' expects '%s'",
-                           "execute", "string");
+                error_setg(errp,
+                           "QMP input object member '%s' must be %s",
+                           "execute", "a string");
                 return NULL;
             }
             has_exec_key = true;
         } else if (!strcmp(arg_name, "arguments")) {
             if (qobject_type(arg_obj) != QTYPE_QDICT) {
-                error_setg(errp, "QMP input object member '%s' expects '%s'",
-                           "arguments", "object");
+                error_setg(errp,
+                           "QMP input object member '%s' must be %s",
+                           "arguments", "an object");
                 return NULL;
             }
         } else {
@@ -60,7 +62,7 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
     }
 
     if (!has_exec_key) {
-        error_setg(errp, "Expected '%s' in QMP input", "execute");
+        error_setg(errp, "QMP input object lacks key 'execute'");
         return NULL;
     }
 
-- 
2.7.4

[Qemu-devel] [PATCH 09/21] qapi: Improve a QObject input visitor error message

[Markus Armbruster]

The QObject input visitor has three error message formats:

* Parameter '%s' is missing
* "Invalid parameter type for '%s', expected: %s"
* "QMP input object member '%s' is unexpected"

The '%s' are member names (or "null", but I'll fix that later).

The last error message calls the thing "QMP input object member"
instead of "parameter".  Misleading when the visitor is used on
QObjects that don't come from QMP.  Change it to "Parameter '%s' is
unexpected".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qobject-input-visitor.c | 3 +--
 tests/test-qga.c             | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index ed6c24c..f3b6713 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -140,8 +140,7 @@ static void qobject_input_check_struct(Visitor *v, Error **errp)
 
             g_hash_table_iter_init(&iter, top_ht);
             if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
-                error_setg(errp, "QMP input object member '%s' is unexpected",
-                           key);
+                error_setg(errp, "Parameter '%s' is unexpected", key);
             }
         }
     }
diff --git a/tests/test-qga.c b/tests/test-qga.c
index 868b02a..ae97b57 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -213,7 +213,7 @@ static void test_qga_invalid_args(gconstpointer fix)
     desc = qdict_get_try_str(error, "desc");
 
     g_assert_cmpstr(class, ==, "GenericError");
-    g_assert_cmpstr(desc, ==, "QMP input object member 'foo' is unexpected");
+    g_assert_cmpstr(desc, ==, "Parameter 'foo' is unexpected");
 
     QDECREF(ret);
 }
-- 
2.7.4

[Qemu-devel] [PATCH 10/21] qapi: Clean up after commit 3d344c2

[Markus Armbruster]

Drop unused QIV_STACK_SIZE and unused qobject_input_start_struct()
parameter errp.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qobject-input-visitor.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index f3b6713..2c2f883 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -21,8 +21,6 @@
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qerror.h"
 
-#define QIV_STACK_SIZE 1024
-
 typedef struct StackObject
 {
     QObject *obj; /* Object being visited */
@@ -103,8 +101,7 @@ static void qdict_add_key(const char *key, QObject *obj, void *opaque)
 }
 
 static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
-                                            QObject *obj, void *qapi,
-                                            Error **errp)
+                                            QObject *obj, void *qapi)
 {
     GHashTable *h;
     StackObject *tos = g_new0(StackObject, 1);
@@ -170,7 +167,6 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
 {
     QObjectInputVisitor *qiv = to_qiv(v);
     QObject *qobj = qobject_input_get_object(qiv, name, true, errp);
-    Error *err = NULL;
 
     if (obj) {
         *obj = NULL;
@@ -184,11 +180,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
         return;
     }
 
-    qobject_input_push(qiv, qobj, obj, &err);
-    if (err) {
-        error_propagate(errp, err);
-        return;
-    }
+    qobject_input_push(qiv, qobj, obj);
 
     if (obj) {
         *obj = g_malloc0(size);
@@ -216,7 +208,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
         return;
     }
 
-    entry = qobject_input_push(qiv, qobj, list, errp);
+    entry = qobject_input_push(qiv, qobj, list);
     if (list) {
         if (entry) {
             *list = g_malloc0(size);
-- 
2.7.4

[Qemu-devel] [PATCH 11/21] qapi: Make QObject input visitor set *list reliably

[Markus Armbruster]

qobject_input_start_struct() sets *list, except when it fails because
qobject_input_get_object() fails, i.e. the input object doesn't exist.

All the other input visitor start_struct(), start_list(),
start_alternate() always set *obj / *list.

Change qobject_input_start_struct() to match.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/qobject-input-visitor.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 2c2f883..d58696c 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -196,25 +196,21 @@ static void qobject_input_start_list(Visitor *v, const char *name,
     QObject *qobj = qobject_input_get_object(qiv, name, true, errp);
     const QListEntry *entry;
 
+    if (list) {
+        *list = NULL;
+    }
     if (!qobj) {
         return;
     }
     if (qobject_type(qobj) != QTYPE_QLIST) {
-        if (list) {
-            *list = NULL;
-        }
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "list");
         return;
     }
 
     entry = qobject_input_push(qiv, qobj, list);
-    if (list) {
-        if (entry) {
-            *list = g_malloc0(size);
-        } else {
-            *list = NULL;
-        }
+    if (entry && list) {
+        *list = g_malloc0(size);
     }
 }
 
-- 
2.7.4

[Qemu-devel] [PATCH 12/21] qapi: Improve qobject input visitor error reporting

[Markus Armbruster]

Error messages refer to nodes of the QObject being visited by name.
Trouble is the names are sometimes less than helpful:

* The name of the root QObject is whatever @name argument got passed
  to the visitor, except NULL gets mapped to "null".  We commonly pass
  NULL.  Not good.

  Avoiding errors "at the root" mitigates.  For instance,
  visit_start_struct() can only fail when the visited object is not a
  dictionary, and we commonly ensure it is beforehand.

* The name of a QDict's member is the member key.  Good enough only
  when this happens to be unique.

* The name of a QList's member is "null".  Not good.

Improve error messages by referring to nodes by path instead, as
follows:

* The path of the root QObject is whatever @name argument got passed
  to the visitor, except NULL gets mapped to "<anonymous>".

* The path of a root QDict's member is the member key.

* The path of a root QList's member is "[%zu]", where %zu is the list
  index, starting at zero.

* The path of a non-root QDict's member is the path of the QDict
  concatenated with "." and the member key.

* The path of a non-root QList's member is the path of the QList
  concatenated with "[%zu]", where %zu is the list index.

For example, the incorrect QMP command

    { "execute": "blockdev-add", "arguments": { "node-name": "foo", "driver": "raw", "file": {"driver": "file" } } }

now fails with

    {"error": {"class": "GenericError", "desc": "Parameter 'file.filename' is missing"}}

instead of

    {"error": {"class": "GenericError", "desc": "Parameter 'filename' is missing"}}

and

    { "execute": "input-send-event", "arguments": { "device": "bar", "events": [ [] ] } }

now fails with

    {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'events[0]', expected: object"}}

instead of

    {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'null', expected: QDict"}}

Aside: calling the thing "parameter" is suboptimal for QMP, because
the root object is "arguments" there.

The qobject output visitor doesn't have this problem because it should
not fail.  Same for dealloc and clone visitors.

The string visitors don't have this problem because they visit just
one value, whose name needs to be passed to the visitor as @name.  The
string output visitor shouldn't fail anyway.

The options visitor uses QemuOpts names.  Their name space is flat, so
the use of QDict member keys as names is fine.  NULL names used with
roots and lists could conceivably result in bad error messages.  Left
for another day.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/visitor.h       |   6 ---
 qapi/qobject-input-visitor.c | 121 +++++++++++++++++++++++++++++++------------
 2 files changed, 87 insertions(+), 40 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 9bb6cba..7c91a50 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -66,12 +66,6 @@
  * object, @name is the key associated with the value; and when
  * visiting a member of a list, @name is NULL.
  *
- * FIXME: Clients must pass NULL for @name when visiting a member of a
- * list, but this leads to poor error messages; it might be nicer to
- * require a non-NULL name such as "key.0" for '{ "key": [ "value" ]
- * }' if an error is encountered on "value" (or to have the visitor
- * core auto-generate the nicer name).
- *
  * The visit_type_FOO() functions expect a non-null @obj argument;
  * they allocate *@obj during input visits, leave it unchanged on
  * output visits, and recursively free any resources during a dealloc
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index d58696c..8015a98 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -21,19 +21,19 @@
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qerror.h"
 
-typedef struct StackObject
-{
-    QObject *obj; /* Object being visited */
+typedef struct StackObject {
+    const char *name;            /* Name of @obj in its parent, if any */
+    QObject *obj;                /* QDict or QList being visited */
     void *qapi; /* sanity check that caller uses same pointer */
 
-    GHashTable *h;           /* If obj is dict: unvisited keys */
-    const QListEntry *entry; /* If obj is list: unvisited tail */
+    GHashTable *h;              /* If @obj is QDict: unvisited keys */
+    const QListEntry *entry;    /* If @obj is QList: unvisited tail */
+    unsigned index;             /* If @obj is QList: list index of @entry */
 
-    QSLIST_ENTRY(StackObject) node;
+    QSLIST_ENTRY(StackObject) node; /* parent */
 } StackObject;
 
-struct QObjectInputVisitor
-{
+struct QObjectInputVisitor {
     Visitor visitor;
 
     /* Root of visit at visitor creation. */
@@ -45,6 +45,8 @@ struct QObjectInputVisitor
 
     /* True to reject parse in visit_end_struct() if unvisited keys remain. */
     bool strict;
+
+    GString *errname;           /* Accumulator for full_name() */
 };
 
 static QObjectInputVisitor *to_qiv(Visitor *v)
@@ -52,9 +54,42 @@ static QObjectInputVisitor *to_qiv(Visitor *v)
     return container_of(v, QObjectInputVisitor, visitor);
 }
 
-static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
-                                         const char *name,
-                                         bool consume, Error **errp)
+static const char *full_name(QObjectInputVisitor *qiv, const char *name)
+{
+    StackObject *so;
+    char buf[32];
+
+    if (qiv->errname) {
+        g_string_truncate(qiv->errname, 0);
+    } else {
+        qiv->errname = g_string_new("");
+    }
+
+    QSLIST_FOREACH(so , &qiv->stack, node) {
+        if (qobject_type(so->obj) == QTYPE_QDICT) {
+            g_string_prepend(qiv->errname, name);
+            g_string_prepend_c(qiv->errname, '.');
+        } else {
+            snprintf(buf, sizeof(buf), "[%u]", so->index);
+            g_string_prepend(qiv->errname, buf);
+        }
+        name = so->name;
+    }
+
+    if (name) {
+        g_string_prepend(qiv->errname, name);
+    } else if (qiv->errname->str[0] == '.') {
+        g_string_erase(qiv->errname, 0, 1);
+    } else {
+        return "<anonymous>";
+    }
+
+    return qiv->errname->str;
+}
+
+static QObject *qobject_input_try_get_object(QObjectInputVisitor *qiv,
+                                             const char *name,
+                                             bool consume)
 {
     StackObject *tos;
     QObject *qobj;
@@ -78,9 +113,6 @@ static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
             bool removed = g_hash_table_remove(tos->h, name);
             assert(removed);
         }
-        if (!ret) {
-            error_setg(errp, QERR_MISSING_PARAMETER, name);
-        }
     } else {
         assert(qobject_type(qobj) == QTYPE_QLIST);
         assert(!name);
@@ -88,12 +120,25 @@ static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
         assert(ret);
         if (consume) {
             tos->entry = qlist_next(tos->entry);
+            tos->index++;
         }
     }
 
     return ret;
 }
 
+static QObject *qobject_input_get_object(QObjectInputVisitor *qiv,
+                                         const char *name,
+                                         bool consume, Error **errp)
+{
+    QObject *obj = qobject_input_try_get_object(qiv, name, consume);
+
+    if (!obj) {
+        error_setg(errp, QERR_MISSING_PARAMETER, full_name(qiv, name));
+    }
+    return obj;
+}
+
 static void qdict_add_key(const char *key, QObject *obj, void *opaque)
 {
     GHashTable *h = opaque;
@@ -101,12 +146,14 @@ static void qdict_add_key(const char *key, QObject *obj, void *opaque)
 }
 
 static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
+                                            const char *name,
                                             QObject *obj, void *qapi)
 {
     GHashTable *h;
     StackObject *tos = g_new0(StackObject, 1);
 
     assert(obj);
+    tos->name = name;
     tos->obj = obj;
     tos->qapi = qapi;
 
@@ -116,6 +163,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
         tos->h = h;
     } else if (qobject_type(obj) == QTYPE_QLIST) {
         tos->entry = qlist_first(qobject_to_qlist(obj));
+        tos->index = -1;
     }
 
     QSLIST_INSERT_HEAD(&qiv->stack, tos, node);
@@ -137,7 +185,8 @@ static void qobject_input_check_struct(Visitor *v, Error **errp)
 
             g_hash_table_iter_init(&iter, top_ht);
             if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
-                error_setg(errp, "Parameter '%s' is unexpected", key);
+                error_setg(errp, "Parameter '%s' is unexpected",
+                           full_name(qiv, key));
             }
         }
     }
@@ -175,12 +224,12 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
         return;
     }
     if (qobject_type(qobj) != QTYPE_QDICT) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "QDict");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "object");
         return;
     }
 
-    qobject_input_push(qiv, qobj, obj);
+    qobject_input_push(qiv, name, qobj, obj);
 
     if (obj) {
         *obj = g_malloc0(size);
@@ -203,12 +252,12 @@ static void qobject_input_start_list(Visitor *v, const char *name,
         return;
     }
     if (qobject_type(qobj) != QTYPE_QLIST) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "list");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "array");
         return;
     }
 
-    entry = qobject_input_push(qiv, qobj, list);
+    entry = qobject_input_push(qiv, name, qobj, list);
     if (entry && list) {
         *list = g_malloc0(size);
     }
@@ -258,8 +307,8 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
     }
     qint = qobject_to_qint(qobj);
     if (!qint) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "integer");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "integer");
         return;
     }
 
@@ -279,8 +328,8 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
     }
     qint = qobject_to_qint(qobj);
     if (!qint) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "integer");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "integer");
         return;
     }
 
@@ -299,8 +348,8 @@ static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
     }
     qbool = qobject_to_qbool(qobj);
     if (!qbool) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "boolean");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "boolean");
         return;
     }
 
@@ -320,8 +369,8 @@ static void qobject_input_type_str(Visitor *v, const char *name, char **obj,
     }
     qstr = qobject_to_qstring(qobj);
     if (!qstr) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "string");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "string");
         return;
     }
 
@@ -351,8 +400,8 @@ static void qobject_input_type_number(Visitor *v, const char *name, double *obj,
         return;
     }
 
-    error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-               "number");
+    error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+               full_name(qiv, name), "number");
 }
 
 static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj,
@@ -380,15 +429,15 @@ static void qobject_input_type_null(Visitor *v, const char *name, Error **errp)
     }
 
     if (qobject_type(qobj) != QTYPE_QNULL) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "null");
+        error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
+                   full_name(qiv, name), "null");
     }
 }
 
 static void qobject_input_optional(Visitor *v, const char *name, bool *present)
 {
     QObjectInputVisitor *qiv = to_qiv(v);
-    QObject *qobj = qobject_input_get_object(qiv, name, false, NULL);
+    QObject *qobj = qobject_input_try_get_object(qiv, name, false);
 
     if (!qobj) {
         *present = false;
@@ -401,6 +450,7 @@ static void qobject_input_optional(Visitor *v, const char *name, bool *present)
 static void qobject_input_free(Visitor *v)
 {
     QObjectInputVisitor *qiv = to_qiv(v);
+
     while (!QSLIST_EMPTY(&qiv->stack)) {
         StackObject *tos = QSLIST_FIRST(&qiv->stack);
 
@@ -409,6 +459,9 @@ static void qobject_input_free(Visitor *v)
     }
 
     qobject_decref(qiv->root);
+    if (qiv->errname) {
+        g_string_free(qiv->errname, TRUE);
+    }
     g_free(qiv);
 }
 
-- 
2.7.4

[Qemu-devel] [PATCH 13/21] qapi: Drop string input visitor method optional()

[Markus Armbruster]

visit_optional() is to be called only between visit_start_struct() and
visit_end_struct().  Visitors that don't support struct visits,
i.e. don't implement start_struct(), end_struct(), have no use for it.
Clarify documentation.

The string input visitor doesn't support struct visits.  Its
parse_optional() is therefore useless.  Drop it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/visitor-impl.h |  4 ++--
 qapi/string-input-visitor.c | 13 -------------
 2 files changed, 2 insertions(+), 15 deletions(-)

diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 8bd47ee..962ba1d 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -102,8 +102,8 @@ struct Visitor
     /* Must be set to visit explicit null values.  */
     void (*type_null)(Visitor *v, const char *name, Error **errp);
 
-    /* Must be set for input visitors, optional otherwise.  The core
-     * takes care of the return type in the public interface. */
+    /* Must be set for input visitors to visit structs, optional otherwise.
+       The core takes care of the return type in the public interface. */
     void (*optional)(Visitor *v, const char *name, bool *present);
 
     /* Must be set */
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index 8dfa561..1a855c5 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -314,18 +314,6 @@ static void parse_type_number(Visitor *v, const char *name, double *obj,
     *obj = val;
 }
 
-static void parse_optional(Visitor *v, const char *name, bool *present)
-{
-    StringInputVisitor *siv = to_siv(v);
-
-    if (!siv->string) {
-        *present = false;
-        return;
-    }
-
-    *present = true;
-}
-
 static void string_input_free(Visitor *v)
 {
     StringInputVisitor *siv = to_siv(v);
@@ -351,7 +339,6 @@ Visitor *string_input_visitor_new(const char *str)
     v->visitor.start_list = start_list;
     v->visitor.next_list = next_list;
     v->visitor.end_list = end_list;
-    v->visitor.optional = parse_optional;
     v->visitor.free = string_input_free;
 
     v->string = str;
-- 
2.7.4

[Qemu-devel] [PATCH 14/21] qapi: Make string input and opts visitor require non-null input

[Markus Armbruster]

The string input visitor tries to cope with null input.  Null input
isn't used anywhere, and isn't covered by tests.  Unsurprisingly, it
doesn't fully work: start_list() crashes because it passes the input
via parse_str() to strtoll() unchecked.

Make string_input_visitor_new() assert its argument isn't null, and
drop the code trying to deal with null input.

The opts visitor crashes when you try to actually visit something with
null input.  Make opts_visitor_new() assert its argument isn't null,
mostly for clarity.

qobject_input_visitor_new() already asserts its argument isn't null.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qapi/opts-visitor.c         |  1 +
 qapi/string-input-visitor.c | 54 ++++++++++++++-------------------------------
 2 files changed, 18 insertions(+), 37 deletions(-)

diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index a0a7c0e..c50dc4b 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -528,6 +528,7 @@ opts_visitor_new(const QemuOpts *opts)
 {
     OptsVisitor *ov;
 
+    assert(opts);
     ov = g_malloc0(sizeof *ov);
 
     ov->visitor.type = VISITOR_INPUT;
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index 1a855c5..f126cd9 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -182,12 +182,6 @@ static void parse_type_int64(Visitor *v, const char *name, int64_t *obj,
 {
     StringInputVisitor *siv = to_siv(v);
 
-    if (!siv->string) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "integer");
-        return;
-    }
-
     if (parse_str(siv, name, errp) < 0) {
         return;
     }
@@ -242,13 +236,7 @@ static void parse_type_size(Visitor *v, const char *name, uint64_t *obj,
     Error *err = NULL;
     uint64_t val;
 
-    if (siv->string) {
-        parse_option_size(name, siv->string, &val, &err);
-    } else {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "size");
-        return;
-    }
+    parse_option_size(name, siv->string, &val, &err);
     if (err) {
         error_propagate(errp, err);
         return;
@@ -262,19 +250,17 @@ static void parse_type_bool(Visitor *v, const char *name, bool *obj,
 {
     StringInputVisitor *siv = to_siv(v);
 
-    if (siv->string) {
-        if (!strcasecmp(siv->string, "on") ||
-            !strcasecmp(siv->string, "yes") ||
-            !strcasecmp(siv->string, "true")) {
-            *obj = true;
-            return;
-        }
-        if (!strcasecmp(siv->string, "off") ||
-            !strcasecmp(siv->string, "no") ||
-            !strcasecmp(siv->string, "false")) {
-            *obj = false;
-            return;
-        }
+    if (!strcasecmp(siv->string, "on") ||
+        !strcasecmp(siv->string, "yes") ||
+        !strcasecmp(siv->string, "true")) {
+        *obj = true;
+        return;
+    }
+    if (!strcasecmp(siv->string, "off") ||
+        !strcasecmp(siv->string, "no") ||
+        !strcasecmp(siv->string, "false")) {
+        *obj = false;
+        return;
     }
 
     error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
@@ -285,13 +271,8 @@ static void parse_type_str(Visitor *v, const char *name, char **obj,
                            Error **errp)
 {
     StringInputVisitor *siv = to_siv(v);
-    if (siv->string) {
-        *obj = g_strdup(siv->string);
-    } else {
-        *obj = NULL;
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "string");
-    }
+
+    *obj = g_strdup(siv->string);
 }
 
 static void parse_type_number(Visitor *v, const char *name, double *obj,
@@ -302,10 +283,8 @@ static void parse_type_number(Visitor *v, const char *name, double *obj,
     double val;
 
     errno = 0;
-    if (siv->string) {
-        val = strtod(siv->string, &endp);
-    }
-    if (!siv->string || errno || endp == siv->string || *endp) {
+    val = strtod(siv->string, &endp);
+    if (errno || endp == siv->string || *endp) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
                    "number");
         return;
@@ -327,6 +306,7 @@ Visitor *string_input_visitor_new(const char *str)
 {
     StringInputVisitor *v;
 
+    assert(str);
     v = g_malloc0(sizeof(*v));
 
     v->visitor.type = VISITOR_INPUT;
-- 
2.7.4

[Qemu-devel] [PATCH 15/21] qom: Make object_property_set_qobject()'s input visitor strict

[Markus Armbruster]

Commit 240f64b made all qobject input visitors created outside tests
strict, except for the one in object_property_set_qobject().  That one
was left behind only because Eric couldn't spare the time to figure
out whether making it strict would break anything, with a TODO
comment.  Time to resolve it.

Strict makes a difference only for otherwise successful visits of QAPI
structs or unions.  Let's examine what the callers of
object_property_set_qobject() visit:

* object_property_set_str(), object_property_set_bool(),
  object_property_set_int() visit a QString, QBool, QInt,
  respectively.  Strictness can't matter.

* qmp_qom_set visits its @value argument.  Comes straight from QMP and
  can be anything ('any' in the QAPI schema).  Strictness matters when
  the property's set() method visits a struct or union QAPI type.

  No such methods exist, thus switching to strict can't break
  anything.

  If we acquire such methods in the future, we'll *want* the visitor
  to be strict, so that unexpected members get rejected as they should
  be.

Switch to strict.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 qom/qom-qobject.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
index 447e4a0..bbdedda 100644
--- a/qom/qom-qobject.c
+++ b/qom/qom-qobject.c
@@ -22,8 +22,8 @@ void object_property_set_qobject(Object *obj, QObject *value,
                                  const char *name, Error **errp)
 {
     Visitor *v;
-    /* TODO: Should we reject, rather than ignore, excess input? */
-    v = qobject_input_visitor_new(value, false);
+
+    v = qobject_input_visitor_new(value, true);
     object_property_set(obj, v, name, errp);
     visit_free(v);
 }
-- 
2.7.4

[Qemu-devel] [PATCH 16/21] test-qobject-input-visitor: Use strict visitor

[Markus Armbruster]

The qobject input visitor comes in a strict and a non-strict variant.
This test is the non-strict variant's last user.  Turns out it relies
on non-strict only in test_visitor_in_null(), and just out of
laziness.  We don't actually test the non-strict behavior.

Clean up test_visitor_in_null(), and switch to the strict variant.
The next commit will drop the non-strict variant.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/test-qobject-input-visitor.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 945404a..125e34c 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -49,7 +49,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
     data->obj = qobject_from_jsonv(json_string, ap);
     g_assert(data->obj);
 
-    data->qiv = qobject_input_visitor_new(data->obj, false);
+    data->qiv = qobject_input_visitor_new(data->obj, true);
     g_assert(data->qiv);
     return data->qiv;
 }
@@ -290,14 +290,14 @@ static void test_visitor_in_null(TestInputVisitorData *data,
      * when input is not null.
      */
 
-    v = visitor_input_test_init(data, "{ 'a': null, 'b': '' }");
+    v = visitor_input_test_init(data, "{ 'a': null, 'b': '', 'c': null }");
     visit_start_struct(v, NULL, NULL, 0, &error_abort);
     visit_type_null(v, "a", &error_abort);
-    visit_type_str(v, "a", &tmp, &err);
-    g_assert(!tmp);
-    error_free_or_abort(&err);
     visit_type_null(v, "b", &err);
     error_free_or_abort(&err);
+    visit_type_str(v, "c", &tmp, &err);
+    g_assert(!tmp);
+    error_free_or_abort(&err);
     visit_check_struct(v, &error_abort);
     visit_end_struct(v, NULL);
 }
-- 
2.7.4

[Qemu-devel] [PATCH 17/21] qapi: Drop unused non-strict qobject input visitor

[Markus Armbruster]

The split between tests/test-qobject-input-visitor.c and
tests/test-qobject-input-strict.c now makes less sense than ever.  The
next commit will take care of that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 block/nbd.c                          |  2 +-
 block/nfs.c                          |  2 +-
 block/ssh.c                          |  2 +-
 docs/qapi-code-gen.txt               |  2 +-
 include/qapi/qobject-input-visitor.h |  5 +----
 qapi/qobject-input-visitor.c         | 29 ++++++++++-------------------
 qmp.c                                |  2 +-
 qom/qom-qobject.c                    |  2 +-
 scripts/qapi-commands.py             |  2 +-
 target/s390x/cpu_models.c            |  2 +-
 tests/check-qnull.c                  |  2 +-
 tests/qmp-test.c                     |  2 +-
 tests/test-qmp-commands.c            |  2 +-
 tests/test-qobject-input-strict.c    |  2 +-
 tests/test-qobject-input-visitor.c   |  2 +-
 tests/test-visitor-serialization.c   |  2 +-
 16 files changed, 25 insertions(+), 37 deletions(-)

diff --git a/block/nbd.c b/block/nbd.c
index a7f9108..f478f80 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -278,7 +278,7 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, Error **errp)
         goto done;
     }
 
-    iv = qobject_input_visitor_new(crumpled_addr, true);
+    iv = qobject_input_visitor_new(crumpled_addr);
     visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
diff --git a/block/nfs.c b/block/nfs.c
index 0cf115e..2884ad1 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -455,7 +455,7 @@ static NFSServer *nfs_config(QDict *options, Error **errp)
         goto out;
     }
 
-    iv = qobject_input_visitor_new(crumpled_addr, true);
+    iv = qobject_input_visitor_new(crumpled_addr);
     visit_type_NFSServer(iv, NULL, &server, &local_error);
     if (local_error) {
         error_propagate(errp, local_error);
diff --git a/block/ssh.c b/block/ssh.c
index 835932e..278e66f 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -601,7 +601,7 @@ static InetSocketAddress *ssh_config(QDict *options, Error **errp)
         goto out;
     }
 
-    iv = qobject_input_visitor_new(crumpled_addr, true);
+    iv = qobject_input_visitor_new(crumpled_addr);
     visit_type_InetSocketAddress(iv, NULL, &inet, &local_error);
     if (local_error) {
         error_propagate(errp, local_error);
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 7eb7be1..6746c10 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1138,7 +1138,7 @@ Example:
         Visitor *v;
         UserDefOneList *arg1 = NULL;
 
-        v = qobject_input_visitor_new(QOBJECT(args), true);
+        v = qobject_input_visitor_new(QOBJECT(args));
         visit_start_struct(v, NULL, NULL, 0, &err);
         if (err) {
             goto out;
diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index cde328d..21db9c4 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -21,10 +21,7 @@ typedef struct QObjectInputVisitor QObjectInputVisitor;
 
 /*
  * Return a new input visitor that converts a QObject to a QAPI object.
- *
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
  */
-Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
+Visitor *qobject_input_visitor_new(QObject *obj);
 
 #endif
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 8015a98..97a27e2 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -43,9 +43,6 @@ struct QObjectInputVisitor {
      * QDict or QList). */
     QSLIST_HEAD(, StackObject) stack;
 
-    /* True to reject parse in visit_end_struct() if unvisited keys remain. */
-    bool strict;
-
     GString *errname;           /* Accumulator for full_name() */
 };
 
@@ -157,11 +154,12 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
     tos->obj = obj;
     tos->qapi = qapi;
 
-    if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
+    if (qobject_type(obj) == QTYPE_QDICT) {
         h = g_hash_table_new(g_str_hash, g_str_equal);
         qdict_iter(qobject_to_qdict(obj), qdict_add_key, h);
         tos->h = h;
-    } else if (qobject_type(obj) == QTYPE_QLIST) {
+    } else {
+        assert(qobject_type(obj) == QTYPE_QLIST);
         tos->entry = qlist_first(qobject_to_qlist(obj));
         tos->index = -1;
     }
@@ -175,20 +173,15 @@ static void qobject_input_check_struct(Visitor *v, Error **errp)
 {
     QObjectInputVisitor *qiv = to_qiv(v);
     StackObject *tos = QSLIST_FIRST(&qiv->stack);
+    GHashTableIter iter;
+    const char *key;
 
     assert(tos && !tos->entry);
-    if (qiv->strict) {
-        GHashTable *const top_ht = tos->h;
-        if (top_ht) {
-            GHashTableIter iter;
-            const char *key;
 
-            g_hash_table_iter_init(&iter, top_ht);
-            if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
-                error_setg(errp, "Parameter '%s' is unexpected",
-                           full_name(qiv, key));
-            }
-        }
+    g_hash_table_iter_init(&iter, tos->h);
+    if (g_hash_table_iter_next(&iter, (void **)&key, NULL)) {
+        error_setg(errp, "Parameter '%s' is unexpected",
+                   full_name(qiv, key));
     }
 }
 
@@ -276,7 +269,6 @@ static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
     return tail->next;
 }
 
-
 static void qobject_input_start_alternate(Visitor *v, const char *name,
                                           GenericAlternate **obj, size_t size,
                                           bool promote_int, Error **errp)
@@ -465,7 +457,7 @@ static void qobject_input_free(Visitor *v)
     g_free(qiv);
 }
 
-Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
+Visitor *qobject_input_visitor_new(QObject *obj)
 {
     QObjectInputVisitor *v;
 
@@ -489,7 +481,6 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
     v->visitor.type_null = qobject_input_type_null;
     v->visitor.optional = qobject_input_optional;
     v->visitor.free = qobject_input_free;
-    v->strict = strict;
 
     v->root = obj;
     qobject_incref(obj);
diff --git a/qmp.c b/qmp.c
index dfaabac..fa82b59 100644
--- a/qmp.c
+++ b/qmp.c
@@ -675,7 +675,7 @@ void qmp_object_add(const char *type, const char *id,
         pdict = qdict_new();
     }
 
-    v = qobject_input_visitor_new(QOBJECT(pdict), true);
+    v = qobject_input_visitor_new(QOBJECT(pdict));
     obj = user_creatable_add_type(type, id, pdict, v, errp);
     visit_free(v);
     if (obj) {
diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
index bbdedda..4aec20d 100644
--- a/qom/qom-qobject.c
+++ b/qom/qom-qobject.c
@@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
 {
     Visitor *v;
 
-    v = qobject_input_visitor_new(value, true);
+    v = qobject_input_visitor_new(value);
     object_property_set(obj, v, name, errp);
     visit_free(v);
 }
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a75946f..befac8f 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -130,7 +130,7 @@ def gen_marshal(name, arg_type, boxed, ret_type):
         push_indent()
 
     ret += mcgen('''
-    v = qobject_input_visitor_new(QOBJECT(args), true);
+    v = qobject_input_visitor_new(QOBJECT(args));
     visit_start_struct(v, NULL, NULL, 0, &err);
     if (err) {
         goto out;
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 5b66d33..784d7b5 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -346,7 +346,7 @@ static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info,
     }
 
     if (qdict) {
-        visitor = qobject_input_visitor_new(info->props, true);
+        visitor = qobject_input_visitor_new(info->props);
         visit_start_struct(visitor, NULL, NULL, 0, errp);
         if (*errp) {
             object_unref(obj);
diff --git a/tests/check-qnull.c b/tests/check-qnull.c
index b50bb8a..8dd1c96 100644
--- a/tests/check-qnull.c
+++ b/tests/check-qnull.c
@@ -47,7 +47,7 @@ static void qnull_visit_test(void)
 
     g_assert(qnull_.refcnt == 1);
     obj = qnull();
-    v = qobject_input_visitor_new(obj, true);
+    v = qobject_input_visitor_new(obj);
     qobject_decref(obj);
     visit_type_null(v, NULL, &error_abort);
     visit_free(v);
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index 405e49e..5d0260b 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -34,7 +34,7 @@ static void test_version(QObject *version)
     VersionInfo *vinfo;
 
     g_assert(version);
-    v = qobject_input_visitor_new(version, true);
+    v = qobject_input_visitor_new(version);
     visit_type_VersionInfo(v, "version", &vinfo, &error_abort);
     qapi_free_VersionInfo(vinfo);
     visit_free(v);
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index c4e20d1..a815056 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -244,7 +244,7 @@ static void test_dealloc_partial(void)
         ud2_dict = qdict_new();
         qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
 
-        v = qobject_input_visitor_new(QOBJECT(ud2_dict), true);
+        v = qobject_input_visitor_new(QOBJECT(ud2_dict));
         visit_type_UserDefTwo(v, NULL, &ud2, &err);
         visit_free(v);
         QDECREF(ud2_dict);
diff --git a/tests/test-qobject-input-strict.c b/tests/test-qobject-input-strict.c
index 4087ea3..7d26113 100644
--- a/tests/test-qobject-input-strict.c
+++ b/tests/test-qobject-input-strict.c
@@ -53,7 +53,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
     data->obj = qobject_from_jsonv(json_string, ap);
     g_assert(data->obj);
 
-    data->qiv = qobject_input_visitor_new(data->obj, true);
+    data->qiv = qobject_input_visitor_new(data->obj);
     g_assert(data->qiv);
     return data->qiv;
 }
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 125e34c..658fa2c 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -49,7 +49,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
     data->obj = qobject_from_jsonv(json_string, ap);
     g_assert(data->obj);
 
-    data->qiv = qobject_input_visitor_new(data->obj, true);
+    data->qiv = qobject_input_visitor_new(data->obj);
     g_assert(data->qiv);
     return data->qiv;
 }
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 66b2b1c..c7e64f0 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -1040,7 +1040,7 @@ static void qmp_deserialize(void **native_out, void *datap,
     obj = qobject_from_json(qstring_get_str(output_json));
 
     QDECREF(output_json);
-    d->qiv = qobject_input_visitor_new(obj, true);
+    d->qiv = qobject_input_visitor_new(obj);
     qobject_decref(obj_orig);
     qobject_decref(obj);
     visit(d->qiv, native_out, errp);
-- 
2.7.4

[Qemu-devel] [PATCH 18/21] tests-qobject-input-strict: Merge into test-qobject-input-visitor

[Markus Armbruster]

Much test-qobject-input-strict.c duplicates
test-qobject-input-strict.c less assertions on expected output:

* test_validate_struct() duplicates test_visitor_in_struct()

* test_validate_struct_nested() duplicates
  test_visitor_in_struct_nested()

* test_validate_list() duplicates the first half of
  test_visitor_in_list()

* test_validate_union_native_list() duplicates
  test_visitor_in_native_list_int()

* test_validate_union_flat() duplicates test_visitor_in_union_flat()

* test_validate_alternate() duplicates the first part of
  test_visitor_in_alternate()

Merge the remaining test cases into test-qobject-input-visitor.c, and
drop the now redundant test-qobject-input-strict.c.

Test case "/visitor/input-strict/fail/list" isn't really about lists,
it's about a bad struct nested in a list.  Rename accordingly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/Makefile.include             |   4 +-
 tests/test-qobject-input-strict.c  | 381 -------------------------------------
 tests/test-qobject-input-visitor.c | 187 ++++++++++++++++++
 3 files changed, 188 insertions(+), 384 deletions(-)
 delete mode 100644 tests/test-qobject-input-strict.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index b212150..cb97473 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -28,7 +28,6 @@ check-unit-y += tests/test-clone-visitor$(EXESUF)
 gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
 check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
 gcov-files-test-qobject-input-visitor-y = qapi/qobject-input-visitor.c
-check-unit-y += tests/test-qobject-input-strict$(EXESUF)
 check-unit-y += tests/test-qmp-commands$(EXESUF)
 gcov-files-test-qmp-commands-y = qapi/qmp-dispatch.c
 check-unit-y += tests/test-string-input-visitor$(EXESUF)
@@ -490,7 +489,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/test-coroutine.o tests/test-string-output-visitor.o \
 	tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \
 	tests/test-clone-visitor.o \
-	tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
+	tests/test-qobject-input-visitor.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
 	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
 	tests/test-opts-visitor.o tests/test-qmp-event.o \
@@ -599,7 +598,6 @@ tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
 tests/test-qobject-output-visitor$(EXESUF): tests/test-qobject-output-visitor.o $(test-qapi-obj-y)
 tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
 tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y)
-tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
 tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
 tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y)
 tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
diff --git a/tests/test-qobject-input-strict.c b/tests/test-qobject-input-strict.c
deleted file mode 100644
index 7d26113..0000000
--- a/tests/test-qobject-input-strict.c
+++ /dev/null
@@ -1,381 +0,0 @@
-/*
- * QObject Input Visitor unit-tests (strict mode).
- *
- * Copyright (C) 2011-2012, 2015 Red Hat Inc.
- *
- * Authors:
- *  Luiz Capitulino <lcapitulino@redhat.com>
- *  Paolo Bonzini <pbonzini@redhat.com>
- *
- * This work is licensed under the terms of the GNU GPL, version 2 or later.
- * See the COPYING file in the top-level directory.
- */
-
-#include "qemu/osdep.h"
-
-#include "qemu-common.h"
-#include "qapi/error.h"
-#include "qapi/qobject-input-visitor.h"
-#include "test-qapi-types.h"
-#include "test-qapi-visit.h"
-#include "qapi/qmp/types.h"
-#include "qapi/qmp/qjson.h"
-#include "test-qmp-introspect.h"
-#include "qmp-introspect.h"
-#include "qapi-visit.h"
-
-typedef struct TestInputVisitorData {
-    QObject *obj;
-    Visitor *qiv;
-} TestInputVisitorData;
-
-static void validate_teardown(TestInputVisitorData *data,
-                               const void *unused)
-{
-    qobject_decref(data->obj);
-    data->obj = NULL;
-
-    if (data->qiv) {
-        visit_free(data->qiv);
-        data->qiv = NULL;
-    }
-}
-
-/* The various test_init functions are provided instead of a test setup
-   function so that the JSON string used by the tests are kept in the test
-   functions (and not in main()). */
-static Visitor *validate_test_init_internal(TestInputVisitorData *data,
-                                            const char *json_string,
-                                            va_list *ap)
-{
-    validate_teardown(data, NULL);
-
-    data->obj = qobject_from_jsonv(json_string, ap);
-    g_assert(data->obj);
-
-    data->qiv = qobject_input_visitor_new(data->obj);
-    g_assert(data->qiv);
-    return data->qiv;
-}
-
-static GCC_FMT_ATTR(2, 3)
-Visitor *validate_test_init(TestInputVisitorData *data,
-                             const char *json_string, ...)
-{
-    Visitor *v;
-    va_list ap;
-
-    va_start(ap, json_string);
-    v = validate_test_init_internal(data, json_string, &ap);
-    va_end(ap);
-    return v;
-}
-
-/* similar to validate_test_init(), but does not expect a string
- * literal/format json_string argument and so can be used for
- * programatically generated strings (and we can't pass in programatically
- * generated strings via %s format parameters since qobject_from_jsonv()
- * will wrap those in double-quotes and treat the entire object as a
- * string)
- */
-static Visitor *validate_test_init_raw(TestInputVisitorData *data,
-                                       const char *json_string)
-{
-    return validate_test_init_internal(data, json_string, NULL);
-}
-
-
-static void test_validate_struct(TestInputVisitorData *data,
-                                  const void *unused)
-{
-    TestStruct *p = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
-
-    visit_type_TestStruct(v, NULL, &p, &error_abort);
-    g_free(p->string);
-    g_free(p);
-}
-
-static void test_validate_struct_nested(TestInputVisitorData *data,
-                                         const void *unused)
-{
-    UserDefTwo *udp = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'string0': 'string0', "
-                           "'dict1': { 'string1': 'string1', "
-                           "'dict2': { 'userdef': { 'integer': 42, "
-                           "'string': 'string' }, 'string': 'string2'}}}");
-
-    visit_type_UserDefTwo(v, NULL, &udp, &error_abort);
-    qapi_free_UserDefTwo(udp);
-}
-
-static void test_validate_list(TestInputVisitorData *data,
-                                const void *unused)
-{
-    UserDefOneList *head = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44 } ]");
-
-    visit_type_UserDefOneList(v, NULL, &head, &error_abort);
-    qapi_free_UserDefOneList(head);
-}
-
-static void test_validate_union_native_list(TestInputVisitorData *data,
-                                            const void *unused)
-{
-    UserDefNativeListUnion *tmp = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'type': 'integer', 'data' : [ 1, 2 ] }");
-
-    visit_type_UserDefNativeListUnion(v, NULL, &tmp, &error_abort);
-    qapi_free_UserDefNativeListUnion(tmp);
-}
-
-static void test_validate_union_flat(TestInputVisitorData *data,
-                                     const void *unused)
-{
-    UserDefFlatUnion *tmp = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data,
-                           "{ 'enum1': 'value1', "
-                           "'integer': 41, "
-                           "'string': 'str', "
-                           "'boolean': true }");
-
-    visit_type_UserDefFlatUnion(v, NULL, &tmp, &error_abort);
-    qapi_free_UserDefFlatUnion(tmp);
-}
-
-static void test_validate_alternate(TestInputVisitorData *data,
-                                    const void *unused)
-{
-    UserDefAlternate *tmp = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "42");
-
-    visit_type_UserDefAlternate(v, NULL, &tmp, &error_abort);
-    qapi_free_UserDefAlternate(tmp);
-}
-
-static void test_validate_fail_struct(TestInputVisitorData *data,
-                                       const void *unused)
-{
-    TestStruct *p = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo', 'extra': 42 }");
-
-    visit_type_TestStruct(v, NULL, &p, &err);
-    error_free_or_abort(&err);
-    g_assert(!p);
-}
-
-static void test_validate_fail_struct_nested(TestInputVisitorData *data,
-                                              const void *unused)
-{
-    UserDefTwo *udp = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
-
-    visit_type_UserDefTwo(v, NULL, &udp, &err);
-    error_free_or_abort(&err);
-    g_assert(!udp);
-}
-
-static void test_validate_fail_struct_missing(TestInputVisitorData *data,
-                                              const void *unused)
-{
-    Error *err = NULL;
-    Visitor *v;
-    QObject *any;
-    GenericAlternate *alt;
-    bool present;
-    int en;
-    int64_t i64;
-    uint32_t u32;
-    int8_t i8;
-    char *str;
-    double dbl;
-
-    v = validate_test_init(data, "{}");
-    visit_start_struct(v, NULL, NULL, 0, &error_abort);
-    visit_start_struct(v, "struct", NULL, 0, &err);
-    error_free_or_abort(&err);
-    visit_start_list(v, "list", NULL, 0, &err);
-    error_free_or_abort(&err);
-    visit_start_alternate(v, "alternate", &alt, sizeof(*alt), false, &err);
-    error_free_or_abort(&err);
-    visit_optional(v, "optional", &present);
-    g_assert(!present);
-    visit_type_enum(v, "enum", &en, EnumOne_lookup, &err);
-    error_free_or_abort(&err);
-    visit_type_int(v, "i64", &i64, &err);
-    error_free_or_abort(&err);
-    visit_type_uint32(v, "u32", &u32, &err);
-    error_free_or_abort(&err);
-    visit_type_int8(v, "i8", &i8, &err);
-    error_free_or_abort(&err);
-    visit_type_str(v, "i8", &str, &err);
-    error_free_or_abort(&err);
-    visit_type_number(v, "dbl", &dbl, &err);
-    error_free_or_abort(&err);
-    visit_type_any(v, "any", &any, &err);
-    error_free_or_abort(&err);
-    visit_type_null(v, "null", &err);
-    error_free_or_abort(&err);
-    visit_end_struct(v, NULL);
-}
-
-static void test_validate_fail_list(TestInputVisitorData *data,
-                                     const void *unused)
-{
-    UserDefOneList *head = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44, 'extra': 'ggg' } ]");
-
-    visit_type_UserDefOneList(v, NULL, &head, &err);
-    error_free_or_abort(&err);
-    g_assert(!head);
-}
-
-static void test_validate_fail_union_native_list(TestInputVisitorData *data,
-                                                 const void *unused)
-{
-    UserDefNativeListUnion *tmp = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data,
-                           "{ 'type': 'integer', 'data' : [ 'string' ] }");
-
-    visit_type_UserDefNativeListUnion(v, NULL, &tmp, &err);
-    error_free_or_abort(&err);
-    g_assert(!tmp);
-}
-
-static void test_validate_fail_union_flat(TestInputVisitorData *data,
-                                          const void *unused)
-{
-    UserDefFlatUnion *tmp = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    v = validate_test_init(data, "{ 'string': 'c', 'integer': 41, 'boolean': true }");
-
-    visit_type_UserDefFlatUnion(v, NULL, &tmp, &err);
-    error_free_or_abort(&err);
-    g_assert(!tmp);
-}
-
-static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
-                                                     const void *unused)
-{
-    UserDefFlatUnion2 *tmp = NULL;
-    Error *err = NULL;
-    Visitor *v;
-
-    /* test situation where discriminator field ('enum1' here) is missing */
-    v = validate_test_init(data, "{ 'integer': 42, 'string': 'c', 'string1': 'd', 'string2': 'e' }");
-
-    visit_type_UserDefFlatUnion2(v, NULL, &tmp, &err);
-    error_free_or_abort(&err);
-    g_assert(!tmp);
-}
-
-static void test_validate_fail_alternate(TestInputVisitorData *data,
-                                         const void *unused)
-{
-    UserDefAlternate *tmp;
-    Visitor *v;
-    Error *err = NULL;
-
-    v = validate_test_init(data, "3.14");
-
-    visit_type_UserDefAlternate(v, NULL, &tmp, &err);
-    error_free_or_abort(&err);
-    g_assert(!tmp);
-}
-
-static void do_test_validate_qmp_introspect(TestInputVisitorData *data,
-                                            const char *schema_json)
-{
-    SchemaInfoList *schema = NULL;
-    Visitor *v;
-
-    v = validate_test_init_raw(data, schema_json);
-
-    visit_type_SchemaInfoList(v, NULL, &schema, &error_abort);
-    g_assert(schema);
-
-    qapi_free_SchemaInfoList(schema);
-}
-
-static void test_validate_qmp_introspect(TestInputVisitorData *data,
-                                           const void *unused)
-{
-    do_test_validate_qmp_introspect(data, test_qmp_schema_json);
-    do_test_validate_qmp_introspect(data, qmp_schema_json);
-}
-
-static void validate_test_add(const char *testpath,
-                               TestInputVisitorData *data,
-                               void (*test_func)(TestInputVisitorData *data, const void *user_data))
-{
-    g_test_add(testpath, TestInputVisitorData, data, NULL, test_func,
-               validate_teardown);
-}
-
-int main(int argc, char **argv)
-{
-    TestInputVisitorData testdata;
-
-    g_test_init(&argc, &argv, NULL);
-
-    validate_test_add("/visitor/input-strict/pass/struct",
-                      &testdata, test_validate_struct);
-    validate_test_add("/visitor/input-strict/pass/struct-nested",
-                      &testdata, test_validate_struct_nested);
-    validate_test_add("/visitor/input-strict/pass/list",
-                      &testdata, test_validate_list);
-    validate_test_add("/visitor/input-strict/pass/union-flat",
-                      &testdata, test_validate_union_flat);
-    validate_test_add("/visitor/input-strict/pass/alternate",
-                      &testdata, test_validate_alternate);
-    validate_test_add("/visitor/input-strict/pass/union-native-list",
-                      &testdata, test_validate_union_native_list);
-    validate_test_add("/visitor/input-strict/fail/struct",
-                      &testdata, test_validate_fail_struct);
-    validate_test_add("/visitor/input-strict/fail/struct-nested",
-                      &testdata, test_validate_fail_struct_nested);
-    validate_test_add("/visitor/input-strict/fail/struct-missing",
-                      &testdata, test_validate_fail_struct_missing);
-    validate_test_add("/visitor/input-strict/fail/list",
-                      &testdata, test_validate_fail_list);
-    validate_test_add("/visitor/input-strict/fail/union-flat",
-                      &testdata, test_validate_fail_union_flat);
-    validate_test_add("/visitor/input-strict/fail/union-flat-no-discriminator",
-                      &testdata, test_validate_fail_union_flat_no_discrim);
-    validate_test_add("/visitor/input-strict/fail/alternate",
-                      &testdata, test_validate_fail_alternate);
-    validate_test_add("/visitor/input-strict/fail/union-native-list",
-                      &testdata, test_validate_fail_union_native_list);
-    validate_test_add("/visitor/input-strict/pass/qmp-introspect",
-                      &testdata, test_validate_qmp_introspect);
-
-    g_test_run();
-
-    return 0;
-}
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 658fa2c..32c6b3d 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -5,6 +5,7 @@
  *
  * Authors:
  *  Luiz Capitulino <lcapitulino@redhat.com>
+ *  Paolo Bonzini <pbonzini@redhat.com>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
@@ -19,6 +20,9 @@
 #include "test-qapi-visit.h"
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qjson.h"
+#include "test-qmp-introspect.h"
+#include "qmp-introspect.h"
+#include "qapi-visit.h"
 
 typedef struct TestInputVisitorData {
     QObject *obj;
@@ -833,6 +837,171 @@ static void test_visitor_in_wrong_type(TestInputVisitorData *data,
     error_free_or_abort(&err);
 }
 
+static void test_visitor_in_fail_struct(TestInputVisitorData *data,
+                                        const void *unused)
+{
+    TestStruct *p = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo', 'extra': 42 }");
+
+    visit_type_TestStruct(v, NULL, &p, &err);
+    error_free_or_abort(&err);
+    g_assert(!p);
+}
+
+static void test_visitor_in_fail_struct_nested(TestInputVisitorData *data,
+                                               const void *unused)
+{
+    UserDefTwo *udp = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
+
+    visit_type_UserDefTwo(v, NULL, &udp, &err);
+    error_free_or_abort(&err);
+    g_assert(!udp);
+}
+
+static void test_visitor_in_fail_struct_in_list(TestInputVisitorData *data,
+                                                const void *unused)
+{
+    UserDefOneList *head = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44, 'extra': 'ggg' } ]");
+
+    visit_type_UserDefOneList(v, NULL, &head, &err);
+    error_free_or_abort(&err);
+    g_assert(!head);
+}
+
+static void test_visitor_in_fail_struct_missing(TestInputVisitorData *data,
+                                                const void *unused)
+{
+    Error *err = NULL;
+    Visitor *v;
+    QObject *any;
+    GenericAlternate *alt;
+    bool present;
+    int en;
+    int64_t i64;
+    uint32_t u32;
+    int8_t i8;
+    char *str;
+    double dbl;
+
+    v = visitor_input_test_init(data, "{}");
+    visit_start_struct(v, NULL, NULL, 0, &error_abort);
+    visit_start_struct(v, "struct", NULL, 0, &err);
+    error_free_or_abort(&err);
+    visit_start_list(v, "list", NULL, 0, &err);
+    error_free_or_abort(&err);
+    visit_start_alternate(v, "alternate", &alt, sizeof(*alt), false, &err);
+    error_free_or_abort(&err);
+    visit_optional(v, "optional", &present);
+    g_assert(!present);
+    visit_type_enum(v, "enum", &en, EnumOne_lookup, &err);
+    error_free_or_abort(&err);
+    visit_type_int(v, "i64", &i64, &err);
+    error_free_or_abort(&err);
+    visit_type_uint32(v, "u32", &u32, &err);
+    error_free_or_abort(&err);
+    visit_type_int8(v, "i8", &i8, &err);
+    error_free_or_abort(&err);
+    visit_type_str(v, "i8", &str, &err);
+    error_free_or_abort(&err);
+    visit_type_number(v, "dbl", &dbl, &err);
+    error_free_or_abort(&err);
+    visit_type_any(v, "any", &any, &err);
+    error_free_or_abort(&err);
+    visit_type_null(v, "null", &err);
+    error_free_or_abort(&err);
+    visit_end_struct(v, NULL);
+}
+
+static void test_visitor_in_fail_union_native_list(TestInputVisitorData *data,
+                                                   const void *unused)
+{
+    UserDefNativeListUnion *tmp = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init(data,
+                                "{ 'type': 'integer', 'data' : [ 'string' ] }");
+
+    visit_type_UserDefNativeListUnion(v, NULL, &tmp, &err);
+    error_free_or_abort(&err);
+    g_assert(!tmp);
+}
+
+static void test_visitor_in_fail_union_flat(TestInputVisitorData *data,
+                                            const void *unused)
+{
+    UserDefFlatUnion *tmp = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "{ 'string': 'c', 'integer': 41, 'boolean': true }");
+
+    visit_type_UserDefFlatUnion(v, NULL, &tmp, &err);
+    error_free_or_abort(&err);
+    g_assert(!tmp);
+}
+
+static void test_visitor_in_fail_union_flat_no_discrim(TestInputVisitorData *data,
+                                                       const void *unused)
+{
+    UserDefFlatUnion2 *tmp = NULL;
+    Error *err = NULL;
+    Visitor *v;
+
+    /* test situation where discriminator field ('enum1' here) is missing */
+    v = visitor_input_test_init(data, "{ 'integer': 42, 'string': 'c', 'string1': 'd', 'string2': 'e' }");
+
+    visit_type_UserDefFlatUnion2(v, NULL, &tmp, &err);
+    error_free_or_abort(&err);
+    g_assert(!tmp);
+}
+
+static void test_visitor_in_fail_alternate(TestInputVisitorData *data,
+                                           const void *unused)
+{
+    UserDefAlternate *tmp;
+    Visitor *v;
+    Error *err = NULL;
+
+    v = visitor_input_test_init(data, "3.14");
+
+    visit_type_UserDefAlternate(v, NULL, &tmp, &err);
+    error_free_or_abort(&err);
+    g_assert(!tmp);
+}
+
+static void do_test_visitor_in_qmp_introspect(TestInputVisitorData *data,
+                                              const char *schema_json)
+{
+    SchemaInfoList *schema = NULL;
+    Visitor *v;
+
+    v = visitor_input_test_init_raw(data, schema_json);
+
+    visit_type_SchemaInfoList(v, NULL, &schema, &error_abort);
+    g_assert(schema);
+
+    qapi_free_SchemaInfoList(schema);
+}
+
+static void test_visitor_in_qmp_introspect(TestInputVisitorData *data,
+                                           const void *unused)
+{
+    do_test_visitor_in_qmp_introspect(data, test_qmp_schema_json);
+    do_test_visitor_in_qmp_introspect(data, qmp_schema_json);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
@@ -893,6 +1062,24 @@ int main(int argc, char **argv)
                            NULL, test_visitor_in_native_list_string);
     input_visitor_test_add("/visitor/input/native_list/number",
                            NULL, test_visitor_in_native_list_number);
+    input_visitor_test_add("/visitor/input/fail/struct",
+                           NULL, test_visitor_in_fail_struct);
+    input_visitor_test_add("/visitor/input/fail/struct-nested",
+                           NULL, test_visitor_in_fail_struct_nested);
+    input_visitor_test_add("/visitor/input/fail/struct-in-list",
+                           NULL, test_visitor_in_fail_struct_in_list);
+    input_visitor_test_add("/visitor/input/fail/struct-missing",
+                           NULL, test_visitor_in_fail_struct_missing);
+    input_visitor_test_add("/visitor/input/fail/union-flat",
+                           NULL, test_visitor_in_fail_union_flat);
+    input_visitor_test_add("/visitor/input/fail/union-flat-no-discriminator",
+                           NULL, test_visitor_in_fail_union_flat_no_discrim);
+    input_visitor_test_add("/visitor/input/fail/alternate",
+                           NULL, test_visitor_in_fail_alternate);
+    input_visitor_test_add("/visitor/input/fail/union-native-list",
+                           NULL, test_visitor_in_fail_union_native_list);
+    input_visitor_test_add("/visitor/input/qmp-introspect",
+                           NULL, test_visitor_in_qmp_introspect);
 
     g_test_run();
 
-- 
2.7.4

[Qemu-devel] [PATCH 19/21] tests: Cover partial input visit of list

[Markus Armbruster]

Demonstrates a design flaw: there is no way to for input visitors to
report that a list visit didn't visit the complete input list.  The
generated list visits always do, but manual visits needn't.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/test-opts-visitor.c          | 41 ++++++++++++++++++++++++++++++++++++++
 tests/test-qobject-input-visitor.c | 19 ++++++++++++++++++
 tests/test-string-input-visitor.c  | 19 ++++++++++++++++++
 3 files changed, 79 insertions(+)

diff --git a/tests/test-opts-visitor.c b/tests/test-opts-visitor.c
index 0a9e75f..58d0eb1 100644
--- a/tests/test-opts-visitor.c
+++ b/tests/test-opts-visitor.c
@@ -172,6 +172,44 @@ expect_u64_max(OptsVisitorFixture *f, gconstpointer test_data)
 
 /* test cases */
 
+static void
+test_opts_range_unvisited(void)
+{
+    intList *list = NULL;
+    intList *tail;
+    QemuOpts *opts;
+    Visitor *v;
+
+    opts = qemu_opts_parse(qemu_find_opts("userdef"), "ilist=0-2", false,
+                           &error_abort);
+
+    v = opts_visitor_new(opts);
+
+    visit_start_struct(v, NULL, NULL, 0, &error_abort);
+
+    /* Would be simpler if  opts visitor supported virtual list walks */
+    visit_start_list(v, "ilist", (GenericList **)&list, sizeof(*list),
+                     &error_abort);
+    tail = list;
+    visit_type_int(v, NULL, &tail->value, &error_abort);
+    g_assert_cmpint(tail->value, ==, 0);
+    tail = (intList *)visit_next_list(v, (GenericList *)tail, sizeof(*list));
+    g_assert(tail);
+    visit_type_int(v, NULL, &tail->value, &error_abort);
+    g_assert_cmpint(tail->value, ==, 1);
+    tail = (intList *)visit_next_list(v, (GenericList *)tail, sizeof(*list));
+    g_assert(tail);
+    visit_end_list(v, (void **)&list);
+    /* BUG: unvisited tail not reported; actually not reportable by design */
+
+    visit_check_struct(v, &error_abort);
+    visit_end_struct(v, NULL);
+
+    qapi_free_intList(list);
+    visit_free(v);
+    qemu_opts_del(opts);
+}
+
 int
 main(int argc, char **argv)
 {
@@ -263,6 +301,9 @@ main(int argc, char **argv)
     add_test("/visitor/opts/i64/range/2big/full", &expect_fail,
              "i64=-0x8000000000000000-0x7fffffffffffffff");
 
+    g_test_add_func("/visitor/opts/range/unvisited",
+                    test_opts_range_unvisited);
+
     g_test_run();
     return 0;
 }
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 32c6b3d..ac87e75 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -923,6 +923,23 @@ static void test_visitor_in_fail_struct_missing(TestInputVisitorData *data,
     visit_end_struct(v, NULL);
 }
 
+static void test_visitor_in_fail_list(TestInputVisitorData *data,
+                                      const void *unused)
+{
+    int64_t i64 = -1;
+    Visitor *v;
+
+    v = visitor_input_test_init(data, "[ 1, 2, 3 ]");
+
+    visit_start_list(v, NULL, NULL, 0, &error_abort);
+    visit_type_int(v, NULL, &i64, &error_abort);
+    g_assert_cmpint(i64, ==, 1);
+    visit_type_int(v, NULL, &i64, &error_abort);
+    g_assert_cmpint(i64, ==, 2);
+    visit_end_list(v, NULL);
+    /* BUG: unvisited tail not reported; actually not reportable by design */
+}
+
 static void test_visitor_in_fail_union_native_list(TestInputVisitorData *data,
                                                    const void *unused)
 {
@@ -1070,6 +1087,8 @@ int main(int argc, char **argv)
                            NULL, test_visitor_in_fail_struct_in_list);
     input_visitor_test_add("/visitor/input/fail/struct-missing",
                            NULL, test_visitor_in_fail_struct_missing);
+    input_visitor_test_add("/visitor/input/fail/list",
+                           NULL, test_visitor_in_fail_list);
     input_visitor_test_add("/visitor/input/fail/union-flat",
                            NULL, test_visitor_in_fail_union_flat);
     input_visitor_test_add("/visitor/input/fail/union-flat-no-discriminator",
diff --git a/tests/test-string-input-visitor.c b/tests/test-string-input-visitor.c
index 7f10e25..bb50473 100644
--- a/tests/test-string-input-visitor.c
+++ b/tests/test-string-input-visitor.c
@@ -94,6 +94,25 @@ static void test_visitor_in_intList(TestInputVisitorData *data,
     visit_type_int16List(v, NULL, &res, &err);
     error_free_or_abort(&err);
     g_assert(!res);
+
+    v = visitor_input_test_init(data, "0,2-3");
+
+    /* Would be simpler if  opts visitor supported virtual list walks */
+    visit_start_list(v, NULL, (GenericList **)&res, sizeof(*res),
+                     &error_abort);
+    tmp = res;
+    visit_type_int16(v, NULL, &tmp->value, &error_abort);
+    g_assert_cmpint(tmp->value, ==, 0);
+    tmp = (int16List *)visit_next_list(v, (GenericList *)tmp, sizeof(*res));
+    g_assert(tmp);
+    visit_type_int16(v, NULL, &tmp->value, &error_abort);
+    g_assert_cmpint(tmp->value, ==, 2);
+    tmp = (int16List *)visit_next_list(v, (GenericList *)tmp, sizeof(*res));
+    g_assert(tmp);
+    visit_end_list(v, (void **)&res);
+    /* BUG: unvisited tail not reported; actually not reportable by design */
+
+    qapi_free_int16List(res);
 }
 
 static void test_visitor_in_bool(TestInputVisitorData *data,
-- 
2.7.4

[Qemu-devel] [PATCH 20/21] qapi: Make input visitors detect unvisited list tails

[Markus Armbruster]

Fix the design flaw demonstrated in the previous commit: new method
check_list() lets input visitors report that unvisited input remains
for a list, exactly like check_struct() lets them report that
unvisited input remains for a struct or union.

Implement the method for the qobject input visitor (straightforward),
and the string input visitor (less so, due to the magic list syntax
there).  The opts visitor's list magic is even more impenetrable, and
all I can do there today is a stub with a FIXME comment.  No worse
than before.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 hw/ppc/spapr_drc.c                 |  5 +++++
 include/qapi/visitor-impl.h        |  3 +++
 include/qapi/visitor.h             | 13 +++++++++++++
 qapi/opts-visitor.c                | 11 +++++++++++
 qapi/qapi-visit-core.c             |  8 ++++++++
 qapi/qobject-input-visitor.c       | 35 +++++++++++++++++++++++++++++++----
 qapi/string-input-visitor.c        | 30 ++++++++++++++++++++++++++++++
 qapi/trace-events                  |  1 +
 scripts/qapi-visit.py              |  3 +++
 tests/test-opts-visitor.c          |  2 +-
 tests/test-qobject-input-visitor.c |  4 +++-
 tests/test-string-input-visitor.c  |  3 ++-
 12 files changed, 111 insertions(+), 7 deletions(-)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 2de6377..fe7b988 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -326,6 +326,11 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
                     return;
                 }
             }
+            visit_check_list(v, &err);
+            if (err) {
+                error_propagate(errp, err);
+                return;
+            }
             visit_end_list(v, NULL);
             break;
         }
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 962ba1d..e87709d 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -61,6 +61,9 @@ struct Visitor
     /* Must be set */
     GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);
 
+    /* Optional; intended for input visitors */
+    void (*check_list)(Visitor *v, Error **errp);
+
     /* Must be set */
     void (*end_list)(Visitor *v, void **list);
 
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 7c91a50..1a1b620 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -370,6 +370,19 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
 GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size);
 
 /*
+ * Prepare for completing a list visit.
+ *
+ * @errp obeys typical error usage, and reports failures such as
+ * unvisited list tail remaining in the input stream.
+ *
+ * Should be called prior to visit_end_list() if all other
+ * intermediate visit steps were successful, to allow the visitor one
+ * last chance to report errors.  May be skipped on a cleanup path,
+ * where there is no need to check for further errors.
+ */
+void visit_check_list(Visitor *v, Error **errp);
+
+/*
  * Complete a list visit started earlier.
  *
  * @list must match what was passed to the paired visit_start_list().
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index c50dc4b..026d25b 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -273,6 +273,16 @@ opts_next_list(Visitor *v, GenericList *tail, size_t size)
 
 
 static void
+opts_check_list(Visitor *v, Error **errp)
+{
+    /*
+     * FIXME should set error when unvisited elements remain.  Mostly
+     * harmless, as the generated visits always visit all elements.
+     */
+}
+
+
+static void
 opts_end_list(Visitor *v, void **obj)
 {
     OptsVisitor *ov = to_ov(v);
@@ -539,6 +549,7 @@ opts_visitor_new(const QemuOpts *opts)
 
     ov->visitor.start_list = &opts_start_list;
     ov->visitor.next_list  = &opts_next_list;
+    ov->visitor.check_list = &opts_check_list;
     ov->visitor.end_list   = &opts_end_list;
 
     ov->visitor.type_int64  = &opts_type_int64;
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index e6e93f0..43a09d1 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -90,6 +90,14 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
     return v->next_list(v, tail, size);
 }
 
+void visit_check_list(Visitor *v, Error **errp)
+{
+    trace_visit_check_list(v);
+    if (v->check_list) {
+        v->check_list(v, errp);
+    }
+}
+
 void visit_end_list(Visitor *v, void **obj)
 {
     trace_visit_end_list(v, obj);
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 97a27e2..0229046 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -51,7 +51,8 @@ static QObjectInputVisitor *to_qiv(Visitor *v)
     return container_of(v, QObjectInputVisitor, visitor);
 }
 
-static const char *full_name(QObjectInputVisitor *qiv, const char *name)
+static const char *full_name_nth(QObjectInputVisitor *qiv, const char *name,
+                                 int n)
 {
     StackObject *so;
     char buf[32];
@@ -63,6 +64,10 @@ static const char *full_name(QObjectInputVisitor *qiv, const char *name)
     }
 
     QSLIST_FOREACH(so , &qiv->stack, node) {
+        if (n > 0) {
+            n--;
+            continue;
+        }
         if (qobject_type(so->obj) == QTYPE_QDICT) {
             g_string_prepend(qiv->errname, name);
             g_string_prepend_c(qiv->errname, '.');
@@ -72,18 +77,24 @@ static const char *full_name(QObjectInputVisitor *qiv, const char *name)
         }
         name = so->name;
     }
+    assert(!n);
 
     if (name) {
         g_string_prepend(qiv->errname, name);
     } else if (qiv->errname->str[0] == '.') {
         g_string_erase(qiv->errname, 0, 1);
-    } else {
+    } else if (!qiv->errname->str[0]) {
         return "<anonymous>";
     }
 
     return qiv->errname->str;
 }
 
+static const char *full_name(QObjectInputVisitor *qiv, const char *name)
+{
+    return full_name_nth(qiv, name, 0);
+}
+
 static QObject *qobject_input_try_get_object(QObjectInputVisitor *qiv,
                                              const char *name,
                                              bool consume)
@@ -260,15 +271,30 @@ static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
                                             size_t size)
 {
     QObjectInputVisitor *qiv = to_qiv(v);
-    StackObject *so = QSLIST_FIRST(&qiv->stack);
+    StackObject *tos = QSLIST_FIRST(&qiv->stack);
 
-    if (!so->entry) {
+    assert(tos && tos->obj && qobject_type(tos->obj) == QTYPE_QLIST);
+
+    if (!tos->entry) {
         return NULL;
     }
     tail->next = g_malloc0(size);
     return tail->next;
 }
 
+static void qobject_input_check_list(Visitor *v, Error **errp)
+{
+    QObjectInputVisitor *qiv = to_qiv(v);
+    StackObject *tos = QSLIST_FIRST(&qiv->stack);
+
+    assert(tos && tos->obj && qobject_type(tos->obj) == QTYPE_QLIST);
+
+    if (tos->entry) {
+        error_setg(errp, "Only %u list elements expected in %s",
+                   tos->index + 1, full_name_nth(qiv, NULL, 1));
+    }
+}
+
 static void qobject_input_start_alternate(Visitor *v, const char *name,
                                           GenericAlternate **obj, size_t size,
                                           bool promote_int, Error **errp)
@@ -470,6 +496,7 @@ Visitor *qobject_input_visitor_new(QObject *obj)
     v->visitor.end_struct = qobject_input_pop;
     v->visitor.start_list = qobject_input_start_list;
     v->visitor.next_list = qobject_input_next_list;
+    v->visitor.check_list = qobject_input_check_list;
     v->visitor.end_list = qobject_input_pop;
     v->visitor.start_alternate = qobject_input_start_alternate;
     v->visitor.type_int64 = qobject_input_type_int64;
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index f126cd9..806b01ae 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -170,6 +170,35 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
     return tail->next;
 }
 
+static void check_list(Visitor *v, Error **errp)
+{
+    const StringInputVisitor *siv = to_siv(v);
+    Range *r;
+    GList *cur_range;
+
+    if (!siv->ranges || !siv->cur_range) {
+        return;
+    }
+
+    r = siv->cur_range->data;
+    if (!r) {
+        return;
+    }
+
+    if (!range_contains(r, siv->cur)) {
+        cur_range = g_list_next(siv->cur_range);
+        if (!cur_range) {
+            return;
+        }
+        r = cur_range->data;
+        if (!r) {
+            return;
+        }
+    }
+
+    error_setg(errp, "Range contains too many values");
+}
+
 static void end_list(Visitor *v, void **obj)
 {
     StringInputVisitor *siv = to_siv(v);
@@ -318,6 +347,7 @@ Visitor *string_input_visitor_new(const char *str)
     v->visitor.type_number = parse_type_number;
     v->visitor.start_list = start_list;
     v->visitor.next_list = next_list;
+    v->visitor.check_list = check_list;
     v->visitor.end_list = end_list;
     v->visitor.free = string_input_free;
 
diff --git a/qapi/trace-events b/qapi/trace-events
index 9cbb61b..339cacf 100644
--- a/qapi/trace-events
+++ b/qapi/trace-events
@@ -8,6 +8,7 @@ visit_end_struct(void *v, void *obj) "v=%p obj=%p"
 
 visit_start_list(void *v, const char *name, void *obj, size_t size) "v=%p name=%s obj=%p size=%zu"
 visit_next_list(void *v, void *tail, size_t size) "v=%p tail=%p size=%zu"
+visit_check_list(void *v) "v=%p"
 visit_end_list(void *v, void *obj) "v=%p obj=%p"
 
 visit_start_alternate(void *v, const char *name, void *obj, size_t size, bool promote_int) "v=%p name=%s obj=%p size=%zu promote_int=%d"
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 96f2491..330b9f3 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -133,6 +133,9 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
         }
     }
 
+    if (!err) {
+        visit_check_list(v, &err);
+    }
     visit_end_list(v, (void **)obj);
     if (err && visit_is_input(v)) {
         qapi_free_%(c_name)s(*obj);
diff --git a/tests/test-opts-visitor.c b/tests/test-opts-visitor.c
index 58d0eb1..1efd327 100644
--- a/tests/test-opts-visitor.c
+++ b/tests/test-opts-visitor.c
@@ -199,8 +199,8 @@ test_opts_range_unvisited(void)
     g_assert_cmpint(tail->value, ==, 1);
     tail = (intList *)visit_next_list(v, (GenericList *)tail, sizeof(*list));
     g_assert(tail);
+    visit_check_list(v, &error_abort); /* BUG: unvisited tail not reported */
     visit_end_list(v, (void **)&list);
-    /* BUG: unvisited tail not reported; actually not reportable by design */
 
     visit_check_struct(v, &error_abort);
     visit_end_struct(v, NULL);
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index ac87e75..bb5bdf4 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -927,6 +927,7 @@ static void test_visitor_in_fail_list(TestInputVisitorData *data,
                                       const void *unused)
 {
     int64_t i64 = -1;
+    Error *err = NULL;
     Visitor *v;
 
     v = visitor_input_test_init(data, "[ 1, 2, 3 ]");
@@ -936,8 +937,9 @@ static void test_visitor_in_fail_list(TestInputVisitorData *data,
     g_assert_cmpint(i64, ==, 1);
     visit_type_int(v, NULL, &i64, &error_abort);
     g_assert_cmpint(i64, ==, 2);
+    visit_check_list(v, &err);
+    error_free_or_abort(&err);
     visit_end_list(v, NULL);
-    /* BUG: unvisited tail not reported; actually not reportable by design */
 }
 
 static void test_visitor_in_fail_union_native_list(TestInputVisitorData *data,
diff --git a/tests/test-string-input-visitor.c b/tests/test-string-input-visitor.c
index bb50473..d101cc2 100644
--- a/tests/test-string-input-visitor.c
+++ b/tests/test-string-input-visitor.c
@@ -109,8 +109,9 @@ static void test_visitor_in_intList(TestInputVisitorData *data,
     g_assert_cmpint(tmp->value, ==, 2);
     tmp = (int16List *)visit_next_list(v, (GenericList *)tmp, sizeof(*res));
     g_assert(tmp);
+    visit_check_list(v, &err);
+    error_free_or_abort(&err);
     visit_end_list(v, (void **)&res);
-    /* BUG: unvisited tail not reported; actually not reportable by design */
 
     qapi_free_int16List(res);
 }
-- 
2.7.4

[Qemu-devel] [PATCH 21/21] qapi: Improve qobject visitor documentation

[Markus Armbruster]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qobject-input-visitor.h  | 36 ++++++++++++++++++++++++++++++++++-
 include/qapi/qobject-output-visitor.h | 35 ++++++++++++++++++++++++++++++----
 2 files changed, 66 insertions(+), 5 deletions(-)

diff --git a/include/qapi/qobject-input-visitor.h b/include/qapi/qobject-input-visitor.h
index 21db9c4..3ca233a 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -20,7 +20,41 @@
 typedef struct QObjectInputVisitor QObjectInputVisitor;
 
 /*
- * Return a new input visitor that converts a QObject to a QAPI object.
+ * Create a QObject input visitor for @obj
+ *
+ * A QObject input visitor visit builds a QAPI object from a QObject.
+ * This simultaneously walks the QAPI object being built and the
+ * QObject.  The latter walk starts at @obj.
+ *
+ * visit_type_FOO() creates an instance of QAPI type FOO.  The visited
+ * QObject must match FOO.  QDict matches struct/union types, QList
+ * matches list types, QString matches type 'str' and enumeration
+ * types, QInt matches integer types, QFloat matches type 'number',
+ * QBool matches type 'bool'.  Type 'any' is matched by QObject.  A
+ * QAPI alternate type is matched when one of its member types is.
+ *
+ * visit_start_struct() ... visit_end_struct() visits a QDict and
+ * creates a QAPI struct/union.  Visits in between visit the
+ * dictionary members.  visit_optional() is true when the QDict has
+ * this member.  visit_check_struct() fails if unvisited members
+ * remain.
+ *
+ * visit_start_list() ... visit_end_list() visits a QList and creates
+ * a QAPI list.  Visits in between visit list members, one after the
+ * other.  visit_next_list() returns NULL when all QList members have
+ * been visited.  visit_check_list() fails if unvisited members
+ * remain.
+ *
+ * visit_start_alternate() ... visit_end_alternate() visits a QObject
+ * and creates a QAPI alternate.  The visit in between visits the same
+ * QObject and initializes the alternate member that is in use.
+ *
+ * Error messages refer to parts of @obj in JavaScript/Python syntax.
+ * For example, 'a.b[2]' refers to the second member of the QList
+ * member 'b' of the QDict member 'a' of QDict @obj.
+ *
+ * The caller is responsible for freeing the visitor with
+ * visit_free().
  */
 Visitor *qobject_input_visitor_new(QObject *obj);
 
diff --git a/include/qapi/qobject-output-visitor.h b/include/qapi/qobject-output-visitor.h
index 8241877..9b990c3 100644
--- a/include/qapi/qobject-output-visitor.h
+++ b/include/qapi/qobject-output-visitor.h
@@ -19,11 +19,38 @@
 
 typedef struct QObjectOutputVisitor QObjectOutputVisitor;
 
-/*
- * Create a new QObject output visitor.
+/**
+ * Create a QObject output visitor for @obj
  *
- * If everything else succeeds, pass @result to visit_complete() to
- * collect the result of the visit.
+ * A QObject output visitor visit builds a QObject from QAPI Object.
+ * This simultaneously walks the QAPI object and the QObject being
+ * built.  The latter walk starts at @obj.
+ *
+ * visit_type_FOO() creates a QObject for QAPI type FOO.  It creates a
+ * QDict for struct/union types, a QList for list types, QString for
+ * type 'str' and enumeration types, QInt for integer types, QFloat
+ * for type 'number', QBool for type 'bool'.  For type 'any', it
+ * increments the QObject's reference count.  For QAPI alternate
+ * types, it creates the QObject for the member that is in use.
+ *
+ * visit_start_struct() ... visit_end_struct() visits a QAPI
+ * struct/union and creates a QDict.  Visits in between visit the
+ * members.  visit_optional() is true when the struct/union has this
+ * member.  visit_check_struct() does nothing.
+ *
+ * visit_start_list() ... visit_end_list() visits a QAPI list and
+ * creates a QList.  Visits in between visit list members, one after
+ * the other.  visit_next_list() returns NULL when all QAPI list
+ * members have been visited.  visit_check_list() does nothing.
+ *
+ * visit_start_alternate() ... visit_end_alternate() visits a QAPI
+ * alternate.  The visit in between creates the QObject for the
+ * alternate member that is in use.
+ *
+ * Errors are not expected to happen.
+ *
+ * The caller is responsible for freeing the visitor with
+ * visit_free().
  */
 Visitor *qobject_output_visitor_new(QObject **result);
 
-- 
2.7.4

Re: [Qemu-devel] [PATCH 01/21] qga: Fix crash on non-dictionary QMP argument

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

On 02/23/2017 03:44 PM, Markus Armbruster wrote:
> The value of key 'arguments' must be a JSON object.  qemu-ga neglects
> to check, and crashes.  To reproduce, send
> 
>     { 'execute': 'guest-sync', 'arguments': [] }
> 
> to qemu-ga.
> 
> do_qmp_dispatch() uses qdict_get_qdict() to get the arguments.  When
> not a JSON object, this gets a null pointer, which flows through the
> generated marshalling function to qobject_input_visitor_new(), where
> it fails the assertion.  qmp_dispatch_check_obj() needs to catch this
> error.
> 
> QEMU isn't affected, because it runs qmp_check_input_obj() first,
> which basically duplicates qmp_check_input_obj()'s checks, plus the
> missing one.
> 
> Fix by copying the missing one from qmp_check_input_obj() to
> qmp_dispatch_check_obj().
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
>  qapi/qmp-dispatch.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 01/21] qga: Fix crash on non-dictionary QMP argument

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

On 02/23/2017 04:46 PM, Eric Blake wrote:
> On 02/23/2017 03:44 PM, Markus Armbruster wrote:
>> The value of key 'arguments' must be a JSON object.  qemu-ga neglects
>> to check, and crashes.  To reproduce, send
>>
>>     { 'execute': 'guest-sync', 'arguments': [] }
>>
>> to qemu-ga.
>>
>> do_qmp_dispatch() uses qdict_get_qdict() to get the arguments.  When
>> not a JSON object, this gets a null pointer, which flows through the
>> generated marshalling function to qobject_input_visitor_new(), where
>> it fails the assertion.  qmp_dispatch_check_obj() needs to catch this
>> error.
>>
>> QEMU isn't affected, because it runs qmp_check_input_obj() first,
>> which basically duplicates qmp_check_input_obj()'s checks, plus the

This sentence is weird (func A can't duplicate func A's checks; you're
missing a func B, but I'm not sure which instance is wrong, nor what B
should be).

>> missing one.
>>
>> Fix by copying the missing one from qmp_check_input_obj() to
>> qmp_dispatch_check_obj().
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> ---
>>  qapi/qmp-dispatch.c | 8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> Reviewed-by: Eric Blake <eblake@redhat.com>
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 02/21] libqtest: Work around a "QMP wants a newline" bug

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

On 02/23/2017 03:44 PM, Markus Armbruster wrote:
> The next commit is going to add a test that calls qmp("null").
> Curiously, this hangs.  Here's why.
> 
> qmp_fd_sendv() doesn't send newlines.  Not even when @fmt contains
> some.  At first glance, the QMP parser seems to be fine with that.
> However, it turns out that it fails to react to input until it sees
> either a newline, an object or an array.  To reproduce, feed to a QMP
> monitor like this:
> 

> 
> Work around this QMP bug by having qmp_fd_sendv() append a newline.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  tests/libqtest.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 

> +++ b/tests/libqtest.c
> @@ -442,14 +442,20 @@ void qmp_fd_sendv(int fd, const char *fmt, va_list ap)
>      if (qobj) {
>          int log = getenv("QTEST_LOG") != NULL;
>          QString *qstr = qobject_to_json(qobj);
> -        const char *str = qstring_get_str(qstr);
> -        size_t size = qstring_get_length(qstr);
> +        const char *str;
> +
> +        /*
> +         * BUG: QMP doesn't react to input until it sees a newline, an
> +         * object, or an array.  Work-around: give it a newline.
> +         */
> +        qstring_append_chr(qstr, '\n');
> +        str = qstring_get_str(qstr);
>  
>          if (log) {
>              fprintf(stderr, "%s", str);

Bonus - the log now has newlines, too :)

>          }
>          /* Send QMP request */
> -        socket_send(fd, str, size);
> +        socket_send(fd, str, qstring_get_length(qstr));

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 03/21] qmp-test: New, covering basic QMP protocol

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 4510 bytes --]

On 02/23/2017 03:44 PM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  tests/Makefile.include |   5 +-
>  tests/libqtest.c       |  17 ++++--
>  tests/libqtest.h       |   8 +++
>  tests/qmp-test.c       | 139 +++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 163 insertions(+), 6 deletions(-)
>  create mode 100644 tests/qmp-test.c
> 

> +++ b/tests/libqtest.h
> @@ -32,6 +32,14 @@ extern QTestState *global_qtest;
>  QTestState *qtest_init(const char *extra_args);
>  
>  /**
> + * qtest_init:

Wrong name (too much copy-and-paste)

> + * @extra_args: other arguments to pass to QEMU.
> + *
> + * Returns: #QTestState instance.
> + */
> +QTestState *qtest_init_without_qmp_handshake(const char *extra_args);
> +
> +/**
>   * qtest_quit:
>   * @s: #QTestState instance to operate on.
>   *
> diff --git a/tests/qmp-test.c b/tests/qmp-test.c
> new file mode 100644
> index 0000000..405e49e
> --- /dev/null
> +++ b/tests/qmp-test.c

> +
> +static void test_malformed(void)
> +{
> +    QDict *resp;
> +
> +    /* Not even a dictionary */
> +    resp = qmp("null");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    QDECREF(resp);
> +

Shall we test an array, integer, and/or boolean literal as well?

> +    /* No "execute" key */
> +    resp = qmp("{}");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    QDECREF(resp);
> +
> +    /* "execute" isn't a string */
> +    resp = qmp("{ 'execute': true }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    QDECREF(resp);
> +
> +    /* "arguments" isn't a dictionary */
> +    resp = qmp("{ 'execute': 'no-such-cmd', 'arguments': [] }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    QDECREF(resp);
> +
> +    /* extra key */
> +    resp = qmp("{ 'execute': 'no-such-cmd', 'extra': true }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    QDECREF(resp);

Worth testing "{ 'arguments': {} }"?

> +}
> +
> +static void test_qmp_protocol(void)
> +{
> +    QDict *resp, *q, *ret;
> +    QList *capabilities;
> +
> +    global_qtest = qtest_init_without_qmp_handshake(common_args);
> +
> +    /* Test greeting */
> +    resp = qmp_receive();
> +    q = qdict_get_qdict(resp, "QMP");
> +    g_assert(q);
> +    test_version(qdict_get(q, "version"));
> +    capabilities = qdict_get_qlist(q, "capabilities");
> +    g_assert(capabilities && qlist_empty(capabilities));
> +    QDECREF(resp);
> +
> +    /* Test valid command before handshake */
> +    resp = qmp("{ 'execute': 'query-version' }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
> +    QDECREF(resp);
> +
> +    /* Test malformed commands before handshake */
> +    test_malformed();
> +
> +    /* Test handshake */
> +    resp = qmp("{ 'execute': 'qmp_capabilities' }");
> +    ret = qdict_get_qdict(resp, "return");
> +    g_assert(ret && !qdict_size(ret));
> +    QDECREF(resp);
> +
> +    /* Test repeated handshake */
> +    resp = qmp("{ 'execute': 'qmp_capabilities' }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
> +    QDECREF(resp);
> +
> +    /* Test valid command */
> +    resp = qmp("{ 'execute': 'query-version' }");
> +    test_version(qdict_get(resp, "return"));
> +    QDECREF(resp);
> +
> +    /* Test malformed commands */
> +    test_malformed();
> +
> +    /* Test 'id' */
> +    resp = qmp("{ 'execute': 'query-name', 'id': 'cookie#1' }");
> +    ret = qdict_get_qdict(resp, "return");
> +    g_assert(ret);
> +    g_assert_cmpstr(qdict_get_try_str(resp, "id"), ==, "cookie#1");
> +    QDECREF(resp);
> +
> +    /* Test command failure with 'id' */
> +    resp = qmp("{ 'execute': 'human-monitor-command', 'id': 2 }");
> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
> +    g_assert_cmpint(qdict_get_int(resp, "id"), ==, 2);
> +    QDECREF(resp);

id can be _any JSON_ (at one point, QMP crashed if id included a null
literal); so maybe we should have more id tests such as "'id': null,",
"'id': [ { }, true ]"

But any tests at all is better than our previous state of none, so even
if you don't add tests, but just fix the typo:

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 01/21] qga: Fix crash on non-dictionary QMP argument

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 04:46 PM, Eric Blake wrote:
>> On 02/23/2017 03:44 PM, Markus Armbruster wrote:
>>> The value of key 'arguments' must be a JSON object.  qemu-ga neglects
>>> to check, and crashes.  To reproduce, send
>>>
>>>     { 'execute': 'guest-sync', 'arguments': [] }
>>>
>>> to qemu-ga.
>>>
>>> do_qmp_dispatch() uses qdict_get_qdict() to get the arguments.  When
>>> not a JSON object, this gets a null pointer, which flows through the
>>> generated marshalling function to qobject_input_visitor_new(), where
>>> it fails the assertion.  qmp_dispatch_check_obj() needs to catch this
>>> error.
>>>
>>> QEMU isn't affected, because it runs qmp_check_input_obj() first,
>>> which basically duplicates qmp_check_input_obj()'s checks, plus the
>
> This sentence is weird (func A can't duplicate func A's checks; you're
> missing a func B, but I'm not sure which instance is wrong, nor what B
> should be).

B is qmp_dispatch_check_obj().  I'll fix it.

>>> missing one.
>>>
>>> Fix by copying the missing one from qmp_check_input_obj() to
>>> qmp_dispatch_check_obj().
>>>
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>> ---
>>>  qapi/qmp-dispatch.c | 8 +++++++-
>>>  1 file changed, 7 insertions(+), 1 deletion(-)
>> 
>> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 03/21] qmp-test: New, covering basic QMP protocol

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:44 PM, Markus Armbruster wrote:
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  tests/Makefile.include |   5 +-
>>  tests/libqtest.c       |  17 ++++--
>>  tests/libqtest.h       |   8 +++
>>  tests/qmp-test.c       | 139 +++++++++++++++++++++++++++++++++++++++++++++++++
>>  4 files changed, 163 insertions(+), 6 deletions(-)
>>  create mode 100644 tests/qmp-test.c
>> 
>
>> +++ b/tests/libqtest.h
>> @@ -32,6 +32,14 @@ extern QTestState *global_qtest;
>>  QTestState *qtest_init(const char *extra_args);
>>  
>>  /**
>> + * qtest_init:
>
> Wrong name (too much copy-and-paste)

One of the several reasons why I detest this comment style.  I'll fix
it, of course.

>> + * @extra_args: other arguments to pass to QEMU.
>> + *
>> + * Returns: #QTestState instance.
>> + */
>> +QTestState *qtest_init_without_qmp_handshake(const char *extra_args);
>> +
>> +/**
>>   * qtest_quit:
>>   * @s: #QTestState instance to operate on.
>>   *
>> diff --git a/tests/qmp-test.c b/tests/qmp-test.c
>> new file mode 100644
>> index 0000000..405e49e
>> --- /dev/null
>> +++ b/tests/qmp-test.c
>
>> +
>> +static void test_malformed(void)
>> +{
>> +    QDict *resp;
>> +
>> +    /* Not even a dictionary */
>> +    resp = qmp("null");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    QDECREF(resp);
>> +
>
> Shall we test an array, integer, and/or boolean literal as well?

Not necessary for code coverage.  Wouldn't hurt, of course.

>> +    /* No "execute" key */
>> +    resp = qmp("{}");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    QDECREF(resp);
>> +
>> +    /* "execute" isn't a string */
>> +    resp = qmp("{ 'execute': true }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    QDECREF(resp);
>> +
>> +    /* "arguments" isn't a dictionary */
>> +    resp = qmp("{ 'execute': 'no-such-cmd', 'arguments': [] }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    QDECREF(resp);
>> +
>> +    /* extra key */
>> +    resp = qmp("{ 'execute': 'no-such-cmd', 'extra': true }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    QDECREF(resp);
>
> Worth testing "{ 'arguments': {} }"?

Likewise.

>> +}
>> +
>> +static void test_qmp_protocol(void)
>> +{
>> +    QDict *resp, *q, *ret;
>> +    QList *capabilities;
>> +
>> +    global_qtest = qtest_init_without_qmp_handshake(common_args);
>> +
>> +    /* Test greeting */
>> +    resp = qmp_receive();
>> +    q = qdict_get_qdict(resp, "QMP");
>> +    g_assert(q);
>> +    test_version(qdict_get(q, "version"));
>> +    capabilities = qdict_get_qlist(q, "capabilities");
>> +    g_assert(capabilities && qlist_empty(capabilities));
>> +    QDECREF(resp);
>> +
>> +    /* Test valid command before handshake */
>> +    resp = qmp("{ 'execute': 'query-version' }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
>> +    QDECREF(resp);
>> +
>> +    /* Test malformed commands before handshake */
>> +    test_malformed();
>> +
>> +    /* Test handshake */
>> +    resp = qmp("{ 'execute': 'qmp_capabilities' }");
>> +    ret = qdict_get_qdict(resp, "return");
>> +    g_assert(ret && !qdict_size(ret));
>> +    QDECREF(resp);
>> +
>> +    /* Test repeated handshake */
>> +    resp = qmp("{ 'execute': 'qmp_capabilities' }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound");
>> +    QDECREF(resp);
>> +
>> +    /* Test valid command */
>> +    resp = qmp("{ 'execute': 'query-version' }");
>> +    test_version(qdict_get(resp, "return"));
>> +    QDECREF(resp);
>> +
>> +    /* Test malformed commands */
>> +    test_malformed();
>> +
>> +    /* Test 'id' */
>> +    resp = qmp("{ 'execute': 'query-name', 'id': 'cookie#1' }");
>> +    ret = qdict_get_qdict(resp, "return");
>> +    g_assert(ret);
>> +    g_assert_cmpstr(qdict_get_try_str(resp, "id"), ==, "cookie#1");
>> +    QDECREF(resp);
>> +
>> +    /* Test command failure with 'id' */
>> +    resp = qmp("{ 'execute': 'human-monitor-command', 'id': 2 }");
>> +    g_assert_cmpstr(get_error_class(resp), ==, "GenericError");
>> +    g_assert_cmpint(qdict_get_int(resp, "id"), ==, 2);
>> +    QDECREF(resp);
>
> id can be _any JSON_ (at one point, QMP crashed if id included a null
> literal); so maybe we should have more id tests such as "'id': null,",
> "'id': [ { }, true ]"
>
> But any tests at all is better than our previous state of none, so even
> if you don't add tests, but just fix the typo:
>
> Reviewed-by: Eric Blake <eblake@redhat.com>

I'm in a bit of a time squeeze, so I'll take your offer to just fix the
typo for now.

Thanks!

Re: [Qemu-devel] [PATCH 17/21] qapi: Drop unused non-strict qobject input visitor

[Paolo Bonzini]

On 23/02/2017 22:45, Markus Armbruster wrote:
> The split between tests/test-qobject-input-visitor.c and
> tests/test-qobject-input-strict.c now makes less sense than ever.  The
> next commit will take care of that.

I'm actually adding a use for non-strict visitors (and one that makes
sense IMHO, with comments, testcases and all that :)).  See
https://www.mail-archive.com/qemu-devel@nongnu.org/msg432069.html.

For what it's worth, however, I believe that even non-strict visits
should detect unvisited list tails.

Paolo

Re: [Qemu-devel] [PATCH 17/21] qapi: Drop unused non-strict qobject input visitor

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 23/02/2017 22:45, Markus Armbruster wrote:
>> The split between tests/test-qobject-input-visitor.c and
>> tests/test-qobject-input-strict.c now makes less sense than ever.  The
>> next commit will take care of that.
>
> I'm actually adding a use for non-strict visitors (and one that makes
> sense IMHO, with comments, testcases and all that :)).  See
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg432069.html.

I replied.

> For what it's worth, however, I believe that even non-strict visits
> should detect unvisited list tails.

Bit of an asymmetry.  Not sure it matters, because to not visit list
tails, you have to put in some effort.

Re: [Qemu-devel] [PATCH 04/21] qmp: Dumb down how we run QMP command registration

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3189 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> The way we get QMP commands registered is high tech:
> 
> * qapi-commands.py generates qmp_init_marshal() that does the actual work
> 
> * it also generates the magic to register it as a MODULE_INIT_QAPI
>   function, so it runs when someone calls
>   module_call_init(MODULE_INIT_QAPI)
> 
> * main() calls module_call_init()
> 
> QEMU needs to register a few non-qapified commands.  Same high tech
> works: monitor.c has its own qmp_init_marshal() along with the magic
> to make it run in module_call_init(MODULE_INIT_QAPI).

Eww. Two static functions with the same name, which really messes with
gdb debugging.

> 
> QEMU also needs to unregister commands that are not wanted in this
> build's configuration (commit 5032a16).  Simple enough:
> qmp_unregister_commands_hack().  The difficulty is to make it run
> after the generated qmp_init_marshal().  We can't simply run it in
> monitor.c's qmp_init_marshal(), because the order in which the
> registered functions run is indeterminate.  So qmp_init_marshal()
> registers qmp_init_marshal() separately.  Since registering *appends*

Another case of "A sets up A" when you meant "A sets up B", but this
time, I'm fairly certain you mean qmp_init_marshal() registers
qmp_unregister_commands_hack() separately.

> to the list of registered functions, this will make it run after all
> the functions that have been registered already.
> 
> I suspect it takes a long and expensive computer science education to
> not find this silly.

ROFL
(does that mean I didn't spend enough on my education? I'm sure you
could arrange to sell me an online certificate if I wanted more... :)

> 
> Dumb it down as follows:
> 
> * Drop MODULE_INIT_QAPI entirely
> 
> * Give the generated qmp_init_marshal() external linkage.
> 
> * Call it instead of module_call_init(MODULE_INIT_QAPI)
> 
> * Except in QEMU proper, call new monitor_init_qmp_commands() that in
>   turn calls the generated qmp_init_marshal(), registers the
>   additional commands and unregisters the unwanted ones.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---

> -static void qmp_init_marshal(void)
> +void monitor_init_qmp_commands(void)
>  {
> +    qmp_init_marshal();
> +
>      qmp_register_command("query-qmp-schema", qmp_query_qmp_schema,
>                           QCO_NO_OPTIONS);
>      qmp_register_command("device_add", qmp_device_add,
> @@ -1004,12 +1006,9 @@ static void qmp_init_marshal(void)
>      qmp_register_command("netdev_add", qmp_netdev_add,
>                           QCO_NO_OPTIONS);
>  
> -    /* call it after the rest of qapi_init() */
> -    register_module_init(qmp_unregister_commands_hack, MODULE_INIT_QAPI);
> +    qmp_unregister_commands_hack();

We could inline this function now, but keeping the _hack() name around
reminds us to consider conditional-compilation support in the generator
at a later date, so I'm fine with the approach you took.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 00/21] qapi: QMP dispatch and input visitor work

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 1487886317-27400-1-git-send-email-armbru@redhat.com
Type: series
Subject: [Qemu-devel] [PATCH 00/21] qapi: QMP dispatch and input visitor work

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

# Useful git options
git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/1487886317-27400-1-git-send-email-armbru@redhat.com -> patchew/1487886317-27400-1-git-send-email-armbru@redhat.com
Switched to a new branch 'test'
700a2ac qapi: Improve qobject visitor documentation
175f23f qapi: Make input visitors detect unvisited list tails
f8ca0b1 tests: Cover partial input visit of list
756fdaa tests-qobject-input-strict: Merge into test-qobject-input-visitor
eb9410c qapi: Drop unused non-strict qobject input visitor
172f593 test-qobject-input-visitor: Use strict visitor
5685ece qom: Make object_property_set_qobject()'s input visitor strict
6ea05df qapi: Make string input and opts visitor require non-null input
9112d89 qapi: Drop string input visitor method optional()
dae8b6a qapi: Improve qobject input visitor error reporting
92c4f7b qapi: Make QObject input visitor set *list reliably
015ae02 qapi: Clean up after commit 3d344c2
48848af qapi: Improve a QObject input visitor error message
f531352 qmp: Improve QMP dispatch error messages
0fa4ae4 qmp: Eliminate silly QERR_QMP_* macros
458f762 qmp: Drop duplicated QMP command object checks
0c32703 qmp: Clean up how we enforce capability negotiation
e2a3724 qmp: Dumb down how we run QMP command registration
16e7bcf qmp-test: New, covering basic QMP protocol
173fd91 libqtest: Work around a "QMP wants a newline" bug
7d1c822 qga: Fix crash on non-dictionary QMP argument

=== OUTPUT BEGIN ===
Checking PATCH 1/21: qga: Fix crash on non-dictionary QMP argument...
Checking PATCH 2/21: libqtest: Work around a "QMP wants a newline" bug...
Checking PATCH 3/21: qmp-test: New, covering basic QMP protocol...
Checking PATCH 4/21: qmp: Dumb down how we run QMP command registration...
Checking PATCH 5/21: qmp: Clean up how we enforce capability negotiation...
Checking PATCH 6/21: qmp: Drop duplicated QMP command object checks...
Checking PATCH 7/21: qmp: Eliminate silly QERR_QMP_* macros...
Checking PATCH 8/21: qmp: Improve QMP dispatch error messages...
Checking PATCH 9/21: qapi: Improve a QObject input visitor error message...
Checking PATCH 10/21: qapi: Clean up after commit 3d344c2...
Checking PATCH 11/21: qapi: Make QObject input visitor set *list reliably...
Checking PATCH 12/21: qapi: Improve qobject input visitor error reporting...
Checking PATCH 13/21: qapi: Drop string input visitor method optional()...
Checking PATCH 14/21: qapi: Make string input and opts visitor require non-null input...
Checking PATCH 15/21: qom: Make object_property_set_qobject()'s input visitor strict...
Checking PATCH 16/21: test-qobject-input-visitor: Use strict visitor...
Checking PATCH 17/21: qapi: Drop unused non-strict qobject input visitor...
Checking PATCH 18/21: tests-qobject-input-strict: Merge into test-qobject-input-visitor...
ERROR: line over 90 characters
#484: FILE: tests/test-qobject-input-visitor.c:847:
+    v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo', 'extra': 42 }");

ERROR: line over 90 characters
#498: FILE: tests/test-qobject-input-visitor.c:861:
+    v = visitor_input_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");

ERROR: line over 90 characters
#512: FILE: tests/test-qobject-input-visitor.c:875:
+    v = visitor_input_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44, 'extra': 'ggg' } ]");

ERROR: line over 90 characters
#585: FILE: tests/test-qobject-input-visitor.c:948:
+    v = visitor_input_test_init(data, "{ 'string': 'c', 'integer': 41, 'boolean': true }");

WARNING: line over 80 characters
#592: FILE: tests/test-qobject-input-visitor.c:955:
+static void test_visitor_in_fail_union_flat_no_discrim(TestInputVisitorData *data,

ERROR: line over 90 characters
#600: FILE: tests/test-qobject-input-visitor.c:963:
+    v = visitor_input_test_init(data, "{ 'integer': 42, 'string': 'c', 'string1': 'd', 'string2': 'e' }");

total: 5 errors, 1 warnings, 233 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 19/21: tests: Cover partial input visit of list...
Checking PATCH 20/21: qapi: Make input visitors detect unvisited list tails...
Checking PATCH 21/21: qapi: Improve qobject visitor documentation...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org

Re: [Qemu-devel] [PATCH 04/21] qmp: Dumb down how we run QMP command registration

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> The way we get QMP commands registered is high tech:
>> 
>> * qapi-commands.py generates qmp_init_marshal() that does the actual work
>> 
>> * it also generates the magic to register it as a MODULE_INIT_QAPI
>>   function, so it runs when someone calls
>>   module_call_init(MODULE_INIT_QAPI)
>> 
>> * main() calls module_call_init()
>> 
>> QEMU needs to register a few non-qapified commands.  Same high tech
>> works: monitor.c has its own qmp_init_marshal() along with the magic
>> to make it run in module_call_init(MODULE_INIT_QAPI).
>
> Eww. Two static functions with the same name, which really messes with
> gdb debugging.
>
>> 
>> QEMU also needs to unregister commands that are not wanted in this
>> build's configuration (commit 5032a16).  Simple enough:
>> qmp_unregister_commands_hack().  The difficulty is to make it run
>> after the generated qmp_init_marshal().  We can't simply run it in
>> monitor.c's qmp_init_marshal(), because the order in which the
>> registered functions run is indeterminate.  So qmp_init_marshal()
>> registers qmp_init_marshal() separately.  Since registering *appends*
>
> Another case of "A sets up A" when you meant "A sets up B", but this
> time, I'm fairly certain you mean qmp_init_marshal() registers
> qmp_unregister_commands_hack() separately.

Correct.  I'll fix it.

>> to the list of registered functions, this will make it run after all
>> the functions that have been registered already.
>> 
>> I suspect it takes a long and expensive computer science education to
>> not find this silly.
>
> ROFL
> (does that mean I didn't spend enough on my education? I'm sure you
> could arrange to sell me an online certificate if I wanted more... :)

Plenty of peddlers out there!

>> Dumb it down as follows:
>> 
>> * Drop MODULE_INIT_QAPI entirely
>> 
>> * Give the generated qmp_init_marshal() external linkage.
>> 
>> * Call it instead of module_call_init(MODULE_INIT_QAPI)
>> 
>> * Except in QEMU proper, call new monitor_init_qmp_commands() that in
>>   turn calls the generated qmp_init_marshal(), registers the
>>   additional commands and unregisters the unwanted ones.
>> 
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>
>> -static void qmp_init_marshal(void)
>> +void monitor_init_qmp_commands(void)
>>  {
>> +    qmp_init_marshal();
>> +
>>      qmp_register_command("query-qmp-schema", qmp_query_qmp_schema,
>>                           QCO_NO_OPTIONS);
>>      qmp_register_command("device_add", qmp_device_add,
>> @@ -1004,12 +1006,9 @@ static void qmp_init_marshal(void)
>>      qmp_register_command("netdev_add", qmp_netdev_add,
>>                           QCO_NO_OPTIONS);
>>  
>> -    /* call it after the rest of qapi_init() */
>> -    register_module_init(qmp_unregister_commands_hack, MODULE_INIT_QAPI);
>> +    qmp_unregister_commands_hack();
>
> We could inline this function now, but keeping the _hack() name around
> reminds us to consider conditional-compilation support in the generator
> at a later date, so I'm fine with the approach you took.

Exactly.

> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 05/21] qmp: Clean up how we enforce capability negotiation

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2523 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> To enforce capability negotiation before normal operation,
> handle_qmp_command() inspects every command before it's handed off to
> qmp_dispatch().  This is a bit of a layering violation, and results in
> duplicated code.
> 
> Before capability negotiation (!cur_mon->in_command_mode), we fail
> commands other than "qmp_capabilities".  This is what enforces
> capability negotiation.
> 
> Afterwards, we fail command "qmp_capabilities".
> 
> Clean this up as follows.
> 
> The obvious place to fail a command is the command itself, so move the
> "afterwards" check to qmp_qmp_capabilities().
> 
> We do the "before" check in every other command, but that would be
> bothersome.  Instead, start without all the other commands, by
> registering only qmp_qmp_capabilities().  Register the others in
> qmp_qmp_capabilities().
> 
> Additionally, replace the generic human-readable error message for
> CommandNotFound by one that reminds the user to run qmp_capabilities.
> Without that, we'd regress commit 2d5a834.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  monitor.c | 70 ++++++++++++++++++++++++++++++++++++---------------------------
>  1 file changed, 40 insertions(+), 30 deletions(-)
> 

> @@ -3781,12 +3782,21 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>      cmd_name = qdict_get_str(qdict, "execute");
>      trace_handle_qmp_command(mon, cmd_name);
>  
> -    if (invalid_qmp_mode(mon, cmd_name, &err)) {
> -        goto err_out;
> -    }
> -
>      rsp = qmp_dispatch(req);
>  
> +    qdict = qdict_get_qdict(qobject_to_qdict(rsp), "error");
> +    if (qdict) {
> +        if (!mon->qmp.in_command_mode
> +            && !strcmp(qdict_get_try_str(qdict, "class"),

Ouch - this can call strcmp(NULL, ...) if the error is ill-formed.  Is
that a problem?  Can we assert that "class" will always exist as a string?

> +                       QapiErrorClass_lookup[ERROR_CLASS_COMMAND_NOT_FOUND])) {
> +            /* Provide a more useful error message */
> +            qdict_del(qdict, "desc");
> +            qdict_put(qdict, "desc",
> +                      qstring_from_str("Expecting capabilities negotiation"
> +                                       " with 'qmp_capabilities'"));
> +        }
> +    }
> +

Otherwise, I like it.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 06/21] qmp: Drop duplicated QMP command object checks

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2516 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> qmp_check_input_obj() duplicates qmp_dispatch_check_obj(), except the
> latter screws up an error message.  handle_qmp_command() runs first
> the former, then the latter via qmp_dispatch(), masking the screwup.
> 
> qemu-ga also masks the screwup, because it also duplicates checks,
> just differently.

Cleaning up the duplication is a win in my book.

> 
> qmp_check_input_obj() exists because handle_qmp_command() needs to
> examine the command before dispatching it.  The previous commit got
> rid of this need, except for a tracepoint, and a bit of "id" code that
> relies on qdict not being null.
> 
> Fix up the error message in qmp_dispatch_check_obj(), drop
> qmp_check_input_obj() and the tracepoint.  Protect the "id" code with
> a conditional.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  monitor.c           | 74 +++++------------------------------------------------
>  qapi/qmp-dispatch.c |  3 +--
>  trace-events        |  1 -
>  3 files changed, 7 insertions(+), 71 deletions(-)
> 
> diff --git a/monitor.c b/monitor.c
> index dcf2de7..d83888d 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -3697,67 +3697,10 @@ static int monitor_can_read(void *opaque)
>      return (mon->suspend_cnt == 0) ? 1 : 0;
>  }
>  
> -/*
> - * Input object checking rules
> - *
> - * 1. Input object must be a dict
> - * 2. The "execute" key must exist
> - * 3. The "execute" key must be a string
> - * 4. If the "arguments" key exists, it must be a dict
> - * 5. If the "id" key exists, it can be anything (ie. json-value)
> - * 6. Any argument not listed above is considered invalid

You know, with just a little tweak, we could add something to the .json
file, and let generated code do all the checks of even the top-level
entities:

{ 'struct': 'WireCommand',
  'data': { 'execute': 'str', '*arguments': 'dict', '*id': 'any' } }

if we had a way to force 'dict' as a subset of 'any' but where it must
be a QDict.  (If we had a 'dict' recognized by the QAPI code generators,
we could probably also fix 'object-add' to use '*props':'dict' instead
of its current use of 'any')

But that sounds like a bit much to ask for in the 2.9 timeframe, so it
doesn't hold up this patch.

We lose a trace, but I don't think that's fatal.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 07/21] qmp: Eliminate silly QERR_QMP_* macros

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> The QERR_ macros are leftovers from the days of "rich" error objects.
> 
> QERR_QMP_BAD_INPUT_OBJECT, QERR_QMP_BAD_INPUT_OBJECT_MEMBER,
> QERR_QMP_EXTRA_MEMBER are used in just one place now, except for one
> use that has crept into qobject-input-visitor.c.
> 
> Drop these macros, to make the (bad) error messages more visible.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 08/21] qmp: Improve QMP dispatch error messages

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  qapi/qmp-dispatch.c | 14 ++++++++------
>  1 file changed, 8 insertions(+), 6 deletions(-)
> 

> @@ -41,15 +41,17 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
>  
>          if (!strcmp(arg_name, "execute")) {
>              if (qobject_type(arg_obj) != QTYPE_QSTRING) {
> -                error_setg(errp, "QMP input object member '%s' expects '%s'",
> -                           "execute", "string");
> +                error_setg(errp,
> +                           "QMP input object member '%s' must be %s",
> +                           "execute", "a string");

Any reason we can't inline this to:

"QMP input object member 'execute' must be a string"

? It's not like we're translating the message.

>                  return NULL;
>              }
>              has_exec_key = true;
>          } else if (!strcmp(arg_name, "arguments")) {
>              if (qobject_type(arg_obj) != QTYPE_QDICT) {
> -                error_setg(errp, "QMP input object member '%s' expects '%s'",
> -                           "arguments", "object");
> +                error_setg(errp,
> +                           "QMP input object member '%s' must be %s",
> +                           "arguments", "an object");

and again

Then again, if you use my idea of a QAPI-generated visitor of each input
wire object, you'd get whatever error message qobject-input normally
gives, which may render these changes irrelevant.

At any rate, the new wordings are nicer, whether or not you inline
constant text.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 05/21] qmp: Clean up how we enforce capability negotiation

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> To enforce capability negotiation before normal operation,
>> handle_qmp_command() inspects every command before it's handed off to
>> qmp_dispatch().  This is a bit of a layering violation, and results in
>> duplicated code.
>> 
>> Before capability negotiation (!cur_mon->in_command_mode), we fail
>> commands other than "qmp_capabilities".  This is what enforces
>> capability negotiation.
>> 
>> Afterwards, we fail command "qmp_capabilities".
>> 
>> Clean this up as follows.
>> 
>> The obvious place to fail a command is the command itself, so move the
>> "afterwards" check to qmp_qmp_capabilities().
>> 
>> We do the "before" check in every other command, but that would be
>> bothersome.  Instead, start without all the other commands, by
>> registering only qmp_qmp_capabilities().  Register the others in
>> qmp_qmp_capabilities().
>> 
>> Additionally, replace the generic human-readable error message for
>> CommandNotFound by one that reminds the user to run qmp_capabilities.
>> Without that, we'd regress commit 2d5a834.
>> 
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  monitor.c | 70 ++++++++++++++++++++++++++++++++++++---------------------------
>>  1 file changed, 40 insertions(+), 30 deletions(-)
>> 
>
>> @@ -3781,12 +3782,21 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>>      cmd_name = qdict_get_str(qdict, "execute");
>>      trace_handle_qmp_command(mon, cmd_name);
>>  
>> -    if (invalid_qmp_mode(mon, cmd_name, &err)) {
>> -        goto err_out;
>> -    }
>> -
>>      rsp = qmp_dispatch(req);
>>  
>> +    qdict = qdict_get_qdict(qobject_to_qdict(rsp), "error");
>> +    if (qdict) {
>> +        if (!mon->qmp.in_command_mode
>> +            && !strcmp(qdict_get_try_str(qdict, "class"),
>
> Ouch - this can call strcmp(NULL, ...) if the error is ill-formed.  Is
> that a problem?  Can we assert that "class" will always exist as a string?

It's safe, because it comes from qmp_build_error_object().  But I'll
switch to g_strcmp0() to make it more obviously safe.

>> +                       QapiErrorClass_lookup[ERROR_CLASS_COMMAND_NOT_FOUND])) {
>> +            /* Provide a more useful error message */
>> +            qdict_del(qdict, "desc");
>> +            qdict_put(qdict, "desc",
>> +                      qstring_from_str("Expecting capabilities negotiation"
>> +                                       " with 'qmp_capabilities'"));
>> +        }
>> +    }
>> +
>
> Otherwise, I like it.

Thanks!

Re: [Qemu-devel] [PATCH 06/21] qmp: Drop duplicated QMP command object checks

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> qmp_check_input_obj() duplicates qmp_dispatch_check_obj(), except the
>> latter screws up an error message.  handle_qmp_command() runs first
>> the former, then the latter via qmp_dispatch(), masking the screwup.
>> 
>> qemu-ga also masks the screwup, because it also duplicates checks,
>> just differently.
>
> Cleaning up the duplication is a win in my book.
>
>> 
>> qmp_check_input_obj() exists because handle_qmp_command() needs to
>> examine the command before dispatching it.  The previous commit got
>> rid of this need, except for a tracepoint, and a bit of "id" code that
>> relies on qdict not being null.
>> 
>> Fix up the error message in qmp_dispatch_check_obj(), drop
>> qmp_check_input_obj() and the tracepoint.  Protect the "id" code with
>> a conditional.
>> 
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  monitor.c           | 74 +++++------------------------------------------------
>>  qapi/qmp-dispatch.c |  3 +--
>>  trace-events        |  1 -
>>  3 files changed, 7 insertions(+), 71 deletions(-)
>> 
>> diff --git a/monitor.c b/monitor.c
>> index dcf2de7..d83888d 100644
>> --- a/monitor.c
>> +++ b/monitor.c
>> @@ -3697,67 +3697,10 @@ static int monitor_can_read(void *opaque)
>>      return (mon->suspend_cnt == 0) ? 1 : 0;
>>  }
>>  
>> -/*
>> - * Input object checking rules
>> - *
>> - * 1. Input object must be a dict
>> - * 2. The "execute" key must exist
>> - * 3. The "execute" key must be a string
>> - * 4. If the "arguments" key exists, it must be a dict
>> - * 5. If the "id" key exists, it can be anything (ie. json-value)
>> - * 6. Any argument not listed above is considered invalid
>
> You know, with just a little tweak, we could add something to the .json
> file, and let generated code do all the checks of even the top-level
> entities:
>
> { 'struct': 'WireCommand',
>   'data': { 'execute': 'str', '*arguments': 'dict', '*id': 'any' } }
>
> if we had a way to force 'dict' as a subset of 'any' but where it must
> be a QDict.  (If we had a 'dict' recognized by the QAPI code generators,
> we could probably also fix 'object-add' to use '*props':'dict' instead
> of its current use of 'any')
>
> But that sounds like a bit much to ask for in the 2.9 timeframe, so it
> doesn't hold up this patch.

The thought came to me, too, but I banished it due to time pressure
before it could take presentable form.

Valid point on object-add.  If we find more uses for "generic object",
we can explore a new built-in QAPI type.

> We lose a trace, but I don't think that's fatal.

If we need one, it should be added to qmp_dispatch(), where it's also
useful for qemu-ga.

> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 08/21] qmp: Improve QMP dispatch error messages

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  qapi/qmp-dispatch.c | 14 ++++++++------
>>  1 file changed, 8 insertions(+), 6 deletions(-)
>> 
>
>> @@ -41,15 +41,17 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
>>  
>>          if (!strcmp(arg_name, "execute")) {
>>              if (qobject_type(arg_obj) != QTYPE_QSTRING) {
>> -                error_setg(errp, "QMP input object member '%s' expects '%s'",
>> -                           "execute", "string");
>> +                error_setg(errp,
>> +                           "QMP input object member '%s' must be %s",
>> +                           "execute", "a string");
>
> Any reason we can't inline this to:
>
> "QMP input object member 'execute' must be a string"
>
> ? It's not like we're translating the message.

Saves a few bytes.  Residual damage from my 8 bit days, I guess.

Inlined would probably be better for translating.

>>                  return NULL;
>>              }
>>              has_exec_key = true;
>>          } else if (!strcmp(arg_name, "arguments")) {
>>              if (qobject_type(arg_obj) != QTYPE_QDICT) {
>> -                error_setg(errp, "QMP input object member '%s' expects '%s'",
>> -                           "arguments", "object");
>> +                error_setg(errp,
>> +                           "QMP input object member '%s' must be %s",
>> +                           "arguments", "an object");
>
> and again
>
> Then again, if you use my idea of a QAPI-generated visitor of each input
> wire object, you'd get whatever error message qobject-input normally
> gives, which may render these changes irrelevant.
>
> At any rate, the new wordings are nicer, whether or not you inline
> constant text.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 09/21] qapi: Improve a QObject input visitor error message

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> The QObject input visitor has three error message formats:
> 
> * Parameter '%s' is missing
> * "Invalid parameter type for '%s', expected: %s"
> * "QMP input object member '%s' is unexpected"
> 
> The '%s' are member names (or "null", but I'll fix that later).
> 
> The last error message calls the thing "QMP input object member"
> instead of "parameter".  Misleading when the visitor is used on
> QObjects that don't come from QMP.  Change it to "Parameter '%s' is
> unexpected".
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  qapi/qobject-input-visitor.c | 3 +--
>  tests/test-qga.c             | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 10/21] qapi: Clean up after commit 3d344c2

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> Drop unused QIV_STACK_SIZE and unused qobject_input_start_struct()
> parameter errp.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  qapi/qobject-input-visitor.c | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 11/21] qapi: Make QObject input visitor set *list reliably

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> qobject_input_start_struct() sets *list, except when it fails because
> qobject_input_get_object() fails, i.e. the input object doesn't exist.
> 
> All the other input visitor start_struct(), start_list(),
> start_alternate() always set *obj / *list.
> 
> Change qobject_input_start_struct() to match.

I thought I'd caught all those. Thanks for the cleanup.

> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  qapi/qobject-input-visitor.c | 14 +++++---------
>  1 file changed, 5 insertions(+), 9 deletions(-)

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 12/21] qapi: Improve qobject input visitor error reporting

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3060 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> Error messages refer to nodes of the QObject being visited by name.
> Trouble is the names are sometimes less than helpful:
> 

> Improve error messages by referring to nodes by path instead, as
> follows:
> 
> * The path of the root QObject is whatever @name argument got passed
>   to the visitor, except NULL gets mapped to "<anonymous>".
> 
> * The path of a root QDict's member is the member key.
> 
> * The path of a root QList's member is "[%zu]", where %zu is the list
>   index, starting at zero.
> 
> * The path of a non-root QDict's member is the path of the QDict
>   concatenated with "." and the member key.
> 
> * The path of a non-root QList's member is the path of the QList
>   concatenated with "[%zu]", where %zu is the list index.
> 

> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  include/qapi/visitor.h       |   6 ---
>  qapi/qobject-input-visitor.c | 121 +++++++++++++++++++++++++++++++------------
>  2 files changed, 87 insertions(+), 40 deletions(-)
> 

> +typedef struct StackObject {
> +    const char *name;            /* Name of @obj in its parent, if any */
> +    QObject *obj;                /* QDict or QList being visited */
>      void *qapi; /* sanity check that caller uses same pointer */
>  
> -    GHashTable *h;           /* If obj is dict: unvisited keys */
> -    const QListEntry *entry; /* If obj is list: unvisited tail */
> +    GHashTable *h;              /* If @obj is QDict: unvisited keys */
> +    const QListEntry *entry;    /* If @obj is QList: unvisited tail */
> +    unsigned index;             /* If @obj is QList: list index of @entry */

Could perhaps make the QDict vs. QList fields shared through a union if
we were tight on storage space or cache line pressure. I don't think
that's the case, though, so no need to change it.

> +static const char *full_name(QObjectInputVisitor *qiv, const char *name)
> +{
> +    StackObject *so;
> +    char buf[32];
> +
> +    if (qiv->errname) {
> +        g_string_truncate(qiv->errname, 0);
> +    } else {
> +        qiv->errname = g_string_new("");
> +    }
> +
> +    QSLIST_FOREACH(so , &qiv->stack, node) {
> +        if (qobject_type(so->obj) == QTYPE_QDICT) {
> +            g_string_prepend(qiv->errname, name);
> +            g_string_prepend_c(qiv->errname, '.');
> +        } else {
> +            snprintf(buf, sizeof(buf), "[%u]", so->index);
> +            g_string_prepend(qiv->errname, buf);
> +        }
> +        name = so->name;
> +    }

Building the string from back to front requires quite a bit of memory
reshuffling, as well as the temporary buffer for integer formatting. Is
there any way to build it from front to back? But this code is only
triggered on error paths, so I don't care if it is slow.  I'm fine with
what you have.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 13/21] qapi: Drop string input visitor method optional()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> visit_optional() is to be called only between visit_start_struct() and
> visit_end_struct().  Visitors that don't support struct visits,
> i.e. don't implement start_struct(), end_struct(), have no use for it.
> Clarify documentation.
> 
> The string input visitor doesn't support struct visits.  Its
> parse_optional() is therefore useless.  Drop it.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  include/qapi/visitor-impl.h |  4 ++--
>  qapi/string-input-visitor.c | 13 -------------
>  2 files changed, 2 insertions(+), 15 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 14/21] qapi: Make string input and opts visitor require non-null input

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> The string input visitor tries to cope with null input.  Null input
> isn't used anywhere, and isn't covered by tests.  Unsurprisingly, it
> doesn't fully work: start_list() crashes because it passes the input
> via parse_str() to strtoll() unchecked.
> 
> Make string_input_visitor_new() assert its argument isn't null, and
> drop the code trying to deal with null input.
> 
> The opts visitor crashes when you try to actually visit something with
> null input.  Make opts_visitor_new() assert its argument isn't null,
> mostly for clarity.
> 
> qobject_input_visitor_new() already asserts its argument isn't null.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  qapi/opts-visitor.c         |  1 +
>  qapi/string-input-visitor.c | 54 ++++++++++++++-------------------------------
>  2 files changed, 18 insertions(+), 37 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 15/21] qom: Make object_property_set_qobject()'s input visitor strict

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> Commit 240f64b made all qobject input visitors created outside tests
> strict, except for the one in object_property_set_qobject().  That one
> was left behind only because Eric couldn't spare the time to figure
> out whether making it strict would break anything, with a TODO
> comment.  Time to resolve it.

Yay.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 16/21] test-qobject-input-visitor: Use strict visitor

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

On 02/23/2017 03:45 PM, Markus Armbruster wrote:
> The qobject input visitor comes in a strict and a non-strict variant.
> This test is the non-strict variant's last user.

Well, depending on Paolo's proposed addition.

>  Turns out it relies
> on non-strict only in test_visitor_in_null(), and just out of
> laziness.  We don't actually test the non-strict behavior.
> 
> Clean up test_visitor_in_null(), and switch to the strict variant.
> The next commit will drop the non-strict variant.
> 
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  tests/test-qobject-input-visitor.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)

I'm in favor of unifying the two tests into one and reducing the
duplication (this and the next patch), whether or not we decide that
Paolo's patch means we don't want to eliminate non-strict mode.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 17/21] qapi: Drop unused non-strict qobject input visitor

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]

On 02/24/2017 09:02 AM, Markus Armbruster wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
>> On 23/02/2017 22:45, Markus Armbruster wrote:
>>> The split between tests/test-qobject-input-visitor.c and
>>> tests/test-qobject-input-strict.c now makes less sense than ever.  The
>>> next commit will take care of that.
>>
>> I'm actually adding a use for non-strict visitors (and one that makes
>> sense IMHO, with comments, testcases and all that :)).  See
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg432069.html.
> 
> I replied.

Hmm. Right now, we implement strict checking by populating a hash table
in parallel to the QObject being visited, then checking if it gets
emptied.  I wonder if it would be any simpler to clone the QObject and
then remove keys, rather than tracking a separate hash table.  In fact,
you could then turn strict checking into an after-the-fact item:
creating the visitor passes in a QObject, then after the visit you check
whether the QObject has been emptied.  But that's not something that has
to be solved for 2.9.

> 
>> For what it's worth, however, I believe that even non-strict visits
>> should detect unvisited list tails.
> 
> Bit of an asymmetry.  Not sure it matters, because to not visit list
> tails, you have to put in some effort.
> 
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 12/21] qapi: Improve qobject input visitor error reporting

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> Error messages refer to nodes of the QObject being visited by name.
>> Trouble is the names are sometimes less than helpful:
>> 
>
>> Improve error messages by referring to nodes by path instead, as
>> follows:
>> 
>> * The path of the root QObject is whatever @name argument got passed
>>   to the visitor, except NULL gets mapped to "<anonymous>".
>> 
>> * The path of a root QDict's member is the member key.
>> 
>> * The path of a root QList's member is "[%zu]", where %zu is the list
>>   index, starting at zero.
>> 
>> * The path of a non-root QDict's member is the path of the QDict
>>   concatenated with "." and the member key.
>> 
>> * The path of a non-root QList's member is the path of the QList
>>   concatenated with "[%zu]", where %zu is the list index.
>> 
>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  include/qapi/visitor.h       |   6 ---
>>  qapi/qobject-input-visitor.c | 121 +++++++++++++++++++++++++++++++------------
>>  2 files changed, 87 insertions(+), 40 deletions(-)
>> 
>
>> +typedef struct StackObject {
>> +    const char *name;            /* Name of @obj in its parent, if any */
>> +    QObject *obj;                /* QDict or QList being visited */
>>      void *qapi; /* sanity check that caller uses same pointer */
>>  
>> -    GHashTable *h;           /* If obj is dict: unvisited keys */
>> -    const QListEntry *entry; /* If obj is list: unvisited tail */
>> +    GHashTable *h;              /* If @obj is QDict: unvisited keys */
>> +    const QListEntry *entry;    /* If @obj is QList: unvisited tail */
>> +    unsigned index;             /* If @obj is QList: list index of @entry */
>
> Could perhaps make the QDict vs. QList fields shared through a union if
> we were tight on storage space or cache line pressure. I don't think
> that's the case, though, so no need to change it.
>
>> +static const char *full_name(QObjectInputVisitor *qiv, const char *name)
>> +{
>> +    StackObject *so;
>> +    char buf[32];
>> +
>> +    if (qiv->errname) {
>> +        g_string_truncate(qiv->errname, 0);
>> +    } else {
>> +        qiv->errname = g_string_new("");
>> +    }
>> +
>> +    QSLIST_FOREACH(so , &qiv->stack, node) {
>> +        if (qobject_type(so->obj) == QTYPE_QDICT) {
>> +            g_string_prepend(qiv->errname, name);
>> +            g_string_prepend_c(qiv->errname, '.');
>> +        } else {
>> +            snprintf(buf, sizeof(buf), "[%u]", so->index);
>> +            g_string_prepend(qiv->errname, buf);
>> +        }
>> +        name = so->name;
>> +    }
>
> Building the string from back to front requires quite a bit of memory
> reshuffling, as well as the temporary buffer for integer formatting. Is
> there any way to build it from front to back? But this code is only
> triggered on error paths, so I don't care if it is slow.  I'm fine with
> what you have.

I opted for simplicity over speed here.

The easiest way to support forward iteration would be making the stack
doubly linked.  Fairly cheap, but it's doubtful whether it would be an
amortized win.

An obvious alternative: measure the stack depth, allocate a char *[],
fill it with names back to front, format front to back.  Not too
complicated, but is it worth on an error path?

However, users exist that try one visit, and if it fails, another one.
For instance, set_pci_devfn() tries visit_type_str() first, then
visit_type_int32.  It should arguably use a suitable
visit_start_alternate() instead.

I think we should hunt down such visitor misuses before we complicate
error paths for speed.

In general,

    foo(..., &err);
    if (err) {
        error_free(err);
        ...
    }

is a performance anti-pattern, because it allocates and frees an Error
object, a message string and whatever it takes to build that just to
transmit one bit of information.

It's better to have foo() return something:

    if (foo(..., NULL)) {
        ...
    }

> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 16/21] test-qobject-input-visitor: Use strict visitor

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/23/2017 03:45 PM, Markus Armbruster wrote:
>> The qobject input visitor comes in a strict and a non-strict variant.
>> This test is the non-strict variant's last user.
>
> Well, depending on Paolo's proposed addition.

Needs a commit message tweak if Paolo's goes in first.

>>  Turns out it relies
>> on non-strict only in test_visitor_in_null(), and just out of
>> laziness.  We don't actually test the non-strict behavior.
>> 
>> Clean up test_visitor_in_null(), and switch to the strict variant.
>> The next commit will drop the non-strict variant.
>> 
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  tests/test-qobject-input-visitor.c | 10 +++++-----
>>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> I'm in favor of unifying the two tests into one and reducing the
> duplication (this and the next patch), whether or not we decide that

Patch after next, actually.

> Paolo's patch means we don't want to eliminate non-strict mode.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks!

Re: [Qemu-devel] [PATCH 17/21] qapi: Drop unused non-strict qobject input visitor

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/24/2017 09:02 AM, Markus Armbruster wrote:
>> Paolo Bonzini <pbonzini@redhat.com> writes:
>> 
>>> On 23/02/2017 22:45, Markus Armbruster wrote:
>>>> The split between tests/test-qobject-input-visitor.c and
>>>> tests/test-qobject-input-strict.c now makes less sense than ever.  The
>>>> next commit will take care of that.
>>>
>>> I'm actually adding a use for non-strict visitors (and one that makes
>>> sense IMHO, with comments, testcases and all that :)).  See
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg432069.html.
>> 
>> I replied.
>
> Hmm. Right now, we implement strict checking by populating a hash table
> in parallel to the QObject being visited, then checking if it gets
> emptied.  I wonder if it would be any simpler to clone the QObject and
> then remove keys, rather than tracking a separate hash table.  In fact,
> you could then turn strict checking into an after-the-fact item:
> creating the visitor passes in a QObject, then after the visit you check
> whether the QObject has been emptied.  But that's not something that has
> to be solved for 2.9.

Only dictionaries need to be cloned, everything else can be shared.

But cloning is still slow.

If QDict was an association list rather than an array, we could use a
bit vector to record visits.

Association list could well be better anyway.  I suspect most QDicts
have few keys.  Measurements needed to confirm.

>>> For what it's worth, however, I believe that even non-strict visits
>>> should detect unvisited list tails.
>> 
>> Bit of an asymmetry.  Not sure it matters, because to not visit list
>> tails, you have to put in some effort.