All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
@ 2018-03-12 18:35 Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 01/36] qapi2texi: minor python code simplification Eric Blake
                   ` (36 more replies)
  0 siblings, 37 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel

The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:

  Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12

for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:

  qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)

This builds and passes 'make check', so even though the OOB portion
depends on chardev fixes that are still pending a pull request from
Paolo, that dependence can only be observed at runtime by clients
that use the new oob feature.  Given the timing of soft freeze, and
the fact that the chardev fixes do not form a build dependency, I
think it's okay if this pull request gets processed before Paolo's
(but it's also okay if Paolo's goes in first).

----------------------------------------------------------------
qapi patches for 2018-03-12, 2.12 softfreeze

- Marc-André Lureau: 0/4 qapi: generate a literal qobject for introspection
- Max Reitz: 0/7 block: Handle null backing link
- Peter Xu: 00/23 QMP: out-of-band (OOB) execution support
- Vladimir Sementsov-Ogievskiy: 0/2 block latency histogram

----------------------------------------------------------------
Marc-André Lureau (4):
      qapi2texi: minor python code simplification
      qlit: use QType instead of int
      qlit: add qobject_from_qlit()
      qapi: generate a literal qobject for introspection

Max Reitz (7):
      compiler: Add QEMU_BUILD_BUG_MSG() macro
      qapi: Add qobject_to()
      qapi: Replace qobject_to_X(o) by qobject_to(X, o)
      qapi: Remove qobject_to_X() functions
      qapi: Make more of qobject_to()
      block: Handle null backing link
      block: Deprecate "backing": ""

Peter Xu (23):
      docs: update QMP documents for OOB commands
      qobject: introduce qstring_get_try_str()
      qobject: introduce qobject_get_try_str()
      qobject: let object_property_get_str() use new API
      monitor: move skip_flush into monitor_data_init
      monitor: move the cur_mon hack deeper for QMP
      monitor: unify global init
      monitor: let mon_list be tail queue
      monitor: allow using IO thread for parsing
      qmp: introduce QMPCapability
      monitor: introduce monitor_qmp_respond()
      monitor: let suspend_cnt be thread safe
      monitor: let suspend/resume work even with QMPs
      monitor: separate QMP parser and dispatcher
      qmp: add new event "command-dropped"
      monitor: send event when command queue full
      qapi: introduce new cmd option "allow-oob"
      qmp: support out-of-band (oob) execution
      qmp: isolate responses into io thread
      monitor: enable IO thread for (qmp & !mux) typed
      qmp: add command "x-oob-test"
      tests: qmp-test: verify command batching
      tests: qmp-test: add oob test

Vladimir Sementsov-Ogievskiy (2):
      block/accounting: introduce latency histogram
      qapi: add block latency histogram interface

 docs/devel/qapi-code-gen.txt        |  87 ++++-
 docs/interop/qmp-spec.txt           |  36 +-
 qemu-doc.texi                       |   7 +
 qapi/block-core.json                | 115 +++++-
 qapi/introspect.json                |   6 +-
 qapi/misc.json                      |  87 ++++-
 scripts/qapi/commands.py            |  18 +-
 scripts/qapi/common.py              |  15 +-
 scripts/qapi/doc.py                 |   9 +-
 scripts/qapi/introspect.py          |  83 +++--
 include/block/accounting.h          |  35 ++
 include/monitor/monitor.h           |   2 +-
 include/qapi/qmp/dispatch.h         |   7 +-
 include/qapi/qmp/qbool.h            |   1 -
 include/qapi/qmp/qdict.h            |   1 -
 include/qapi/qmp/qlist.h            |   1 -
 include/qapi/qmp/qlit.h             |   4 +-
 include/qapi/qmp/qnum.h             |   1 -
 include/qapi/qmp/qobject.h          |  30 ++
 include/qapi/qmp/qstring.h          |   3 +-
 include/qemu/compiler.h             |  12 +-
 vl.c                                |   7 +-
 tests/libqtest.c                    |   6 +-
 block.c                             |  13 +-
 block/accounting.c                  |  91 +++++
 block/qapi.c                        |  53 ++-
 block/qcow2.c                       |   2 +-
 block/rbd.c                         |   8 +-
 block/sheepdog.c                    |   2 +-
 blockdev.c                          |  64 +++-
 hw/i386/acpi-build.c                |  14 +-
 monitor.c                           | 701 +++++++++++++++++++++++++++++++-----
 qapi/qmp-dispatch.c                 |  35 +-
 qapi/qobject-input-visitor.c        |  24 +-
 qapi/qobject-output-visitor.c       |   4 +-
 qga/main.c                          |   2 +-
 qmp.c                               |  18 +-
 qobject/json-parser.c               |  13 +-
 qobject/qbool.c                     |  15 +-
 qobject/qdict.c                     |  65 ++--
 qobject/qjson.c                     |  10 +-
 qobject/qlist.c                     |  17 +-
 qobject/qlit.c                      |  47 ++-
 qobject/qnum.c                      |  17 +-
 qobject/qstring.c                   |  38 +-
 qom/object.c                        |  15 +-
 target/i386/cpu.c                   |   2 +-
 target/s390x/cpu_models.c           |   2 +-
 tests/check-qdict.c                 |  20 +-
 tests/check-qjson.c                 |  41 ++-
 tests/check-qlist.c                 |   4 +-
 tests/check-qlit.c                  |  30 +-
 tests/check-qnum.c                  |   4 +-
 tests/check-qobject.c               |   2 +-
 tests/check-qstring.c               |   2 +-
 tests/device-introspect-test.c      |  14 +-
 tests/numa-test.c                   |   8 +-
 tests/qmp-test.c                    |  97 ++++-
 tests/qom-test.c                    |   4 +-
 tests/test-char.c                   |   2 +-
 tests/test-keyval.c                 |   8 +-
 tests/test-qga.c                    |  19 +-
 tests/test-qmp-cmds.c               |  12 +-
 tests/test-qmp-event.c              |  16 +-
 tests/test-qobject-input-visitor.c  |  21 +-
 tests/test-qobject-output-visitor.c |  54 +--
 tests/test-x86-cpuid-compat.c       |  17 +-
 util/keyval.c                       |   4 +-
 util/qemu-config.c                  |   2 +-
 util/qemu-option.c                  |   6 +-
 qemu-options.hx                     |   4 +-
 tests/qapi-schema/test-qapi.py      |   2 +-
 tests/qemu-iotests/089              |  20 +
 tests/qemu-iotests/089.out          |   8 +
 trace-events                        |   3 +
 75 files changed, 1785 insertions(+), 484 deletions(-)

-- 
2.14.3

^ permalink raw reply	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 01/36] qapi2texi: minor python code simplification
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 02/36] qlit: use QType instead of int Eric Blake
                   ` (35 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Marc-André Lureau

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-2-marcandre.lureau@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 scripts/qapi/doc.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 0ea68bf8134..79d11bbe9b5 100644
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -134,10 +134,9 @@ def texi_enum_value(value):
 def texi_member(member, suffix=''):
     """Format a table of members item for an object type member"""
     typ = member.type.doc_type()
-    return '@item @code{%s%s%s}%s%s\n' % (
-        member.name,
-        ': ' if typ else '',
-        typ if typ else '',
+    membertype = ': ' + typ if typ else ''
+    return '@item @code{%s%s}%s%s\n' % (
+        member.name, membertype,
         ' (optional)' if member.optional else '',
         suffix)

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 02/36] qlit: use QType instead of int
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 01/36] qapi2texi: minor python code simplification Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 03/36] qlit: add qobject_from_qlit() Eric Blake
                   ` (34 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Marc-André Lureau, Markus Armbruster

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-3-marcandre.lureau@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qlit.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/qapi/qmp/qlit.h b/include/qapi/qmp/qlit.h
index 56f9d97bd98..f1ed082df81 100644
--- a/include/qapi/qmp/qlit.h
+++ b/include/qapi/qmp/qlit.h
@@ -20,7 +20,7 @@ typedef struct QLitDictEntry QLitDictEntry;
 typedef struct QLitObject QLitObject;

 struct QLitObject {
-    int type;
+    QType type;
     union {
         bool qbool;
         int64_t qnum;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 03/36] qlit: add qobject_from_qlit()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 01/36] qapi2texi: minor python code simplification Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 02/36] qlit: use QType instead of int Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 04/36] qapi: generate a literal qobject for introspection Eric Blake
                   ` (33 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Marc-André Lureau, Markus Armbruster

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Instantiate a QObject* from a literal QLitObject.

LitObject only supports int64_t for now.  uint64_t and double aren't
implemented.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-4-marcandre.lureau@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qlit.h |  2 ++
 qobject/qlit.c          | 37 +++++++++++++++++++++++++++++++++++++
 tests/check-qlit.c      | 28 ++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+)

diff --git a/include/qapi/qmp/qlit.h b/include/qapi/qmp/qlit.h
index f1ed082df81..c0676d5daf2 100644
--- a/include/qapi/qmp/qlit.h
+++ b/include/qapi/qmp/qlit.h
@@ -50,4 +50,6 @@ struct QLitDictEntry {

 bool qlit_equal_qobject(const QLitObject *lhs, const QObject *rhs);

+QObject *qobject_from_qlit(const QLitObject *qlit);
+
 #endif /* QLIT_H */
diff --git a/qobject/qlit.c b/qobject/qlit.c
index 948e0b860cd..d0e1b72fd01 100644
--- a/qobject/qlit.c
+++ b/qobject/qlit.c
@@ -21,6 +21,7 @@
 #include "qapi/qmp/qnum.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/qmp/qnull.h"

 static bool qlit_equal_qdict(const QLitObject *lhs, const QDict *qdict)
 {
@@ -86,3 +87,39 @@ bool qlit_equal_qobject(const QLitObject *lhs, const QObject *rhs)

     return false;
 }
+
+QObject *qobject_from_qlit(const QLitObject *qlit)
+{
+    switch (qlit->type) {
+    case QTYPE_QNULL:
+        return QOBJECT(qnull());
+    case QTYPE_QNUM:
+        return QOBJECT(qnum_from_int(qlit->value.qnum));
+    case QTYPE_QSTRING:
+        return QOBJECT(qstring_from_str(qlit->value.qstr));
+    case QTYPE_QDICT: {
+        QDict *qdict = qdict_new();
+        QLitDictEntry *e;
+
+        for (e = qlit->value.qdict; e->key; e++) {
+            qdict_put_obj(qdict, e->key, qobject_from_qlit(&e->value));
+        }
+        return QOBJECT(qdict);
+    }
+    case QTYPE_QLIST: {
+        QList *qlist = qlist_new();
+        QLitObject *e;
+
+        for (e = qlit->value.qlist; e->type != QTYPE_NONE; e++) {
+            qlist_append_obj(qlist, qobject_from_qlit(e));
+        }
+        return QOBJECT(qlist);
+    }
+    case QTYPE_QBOOL:
+        return QOBJECT(qbool_from_bool(qlit->value.qbool));
+    default:
+        assert(0);
+    }
+
+    return NULL;
+}
diff --git a/tests/check-qlit.c b/tests/check-qlit.c
index 5d0f65b9c77..836f4a3090c 100644
--- a/tests/check-qlit.c
+++ b/tests/check-qlit.c
@@ -9,9 +9,11 @@

 #include "qemu/osdep.h"

+#include "qapi/qmp/qbool.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qlist.h"
 #include "qapi/qmp/qlit.h"
+#include "qapi/qmp/qnum.h"
 #include "qapi/qmp/qstring.h"

 static QLitObject qlit = QLIT_QDICT(((QLitDictEntry[]) {
@@ -63,11 +65,37 @@ static void qlit_equal_qobject_test(void)
     qobject_decref(qobj);
 }

+static void qobject_from_qlit_test(void)
+{
+    QObject *obj, *qobj = qobject_from_qlit(&qlit);
+    QDict *qdict;
+    QList *bee;
+
+    qdict = qobject_to_qdict(qobj);
+    g_assert_cmpint(qdict_get_int(qdict, "foo"), ==, 42);
+    g_assert_cmpstr(qdict_get_str(qdict, "bar"), ==, "hello world");
+    g_assert(qobject_type(qdict_get(qdict, "baz")) == QTYPE_QNULL);
+
+    bee = qdict_get_qlist(qdict, "bee");
+    obj = qlist_pop(bee);
+    g_assert_cmpint(qnum_get_int(qobject_to_qnum(obj)), ==, 43);
+    qobject_decref(obj);
+    obj = qlist_pop(bee);
+    g_assert_cmpint(qnum_get_int(qobject_to_qnum(obj)), ==, 44);
+    qobject_decref(obj);
+    obj = qlist_pop(bee);
+    g_assert(qbool_get_bool(qobject_to_qbool(obj)));
+    qobject_decref(obj);
+
+    qobject_decref(qobj);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);

     g_test_add_func("/qlit/equal_qobject", qlit_equal_qobject_test);
+    g_test_add_func("/qlit/qobject_from_qlit", qobject_from_qlit_test);

     return g_test_run();
 }
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 04/36] qapi: generate a literal qobject for introspection
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (2 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 03/36] qlit: add qobject_from_qlit() Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 05/36] compiler: Add QEMU_BUILD_BUG_MSG() macro Eric Blake
                   ` (32 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel
  Cc: Marc-André Lureau, Markus Armbruster, Michael Roth,
	Dr. David Alan Gilbert

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Replace the generated json string with a literal qobject. The later is
easier to deal with, at run time as well as compile time: adding #if
conditionals will be easier than in a json string.

The output of query-qmp-schema is not changed.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-5-marcandre.lureau@redhat.com>
[eblake: fix python 3 failure]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/devel/qapi-code-gen.txt       | 29 ++++++++++-----
 scripts/qapi/introspect.py         | 76 +++++++++++++++++++++-----------------
 monitor.c                          |  2 +-
 tests/test-qobject-input-visitor.c | 11 ++++--
 4 files changed, 70 insertions(+), 48 deletions(-)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 25b7180a189..99d81230e26 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -1319,18 +1319,27 @@ Example:
     #ifndef EXAMPLE_QMP_INTROSPECT_H
     #define EXAMPLE_QMP_INTROSPECT_H

-    extern const char example_qmp_schema_json[];
+    extern const QLitObject qmp_schema_qlit;

     #endif
     $ cat qapi-generated/example-qapi-introspect.c
 [Uninteresting stuff omitted...]

-    const char example_qmp_schema_json[] = "["
-        "{\"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": \"MY_EVENT\"}, "
-        "{\"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": \"my-command\", \"ret-type\": \"2\"}, "
-        "{\"members\": [], \"meta-type\": \"object\", \"name\": \"0\"}, "
-        "{\"members\": [{\"name\": \"arg1\", \"type\": \"[2]\"}], \"meta-type\": \"object\", \"name\": \"1\"}, "
-        "{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"default\": null, \"name\": \"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": \"2\"}, "
-        "{\"element-type\": \"2\", \"meta-type\": \"array\", \"name\": \"[2]\"}, "
-        "{\"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": \"int\"}, "
-        "{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": \"str\"}]";
+    const QLitObject example_qmp_schema_qlit = QLIT_QLIST(((QLitObject[]) {
+        QLIT_QDICT(((QLitDictEntry[]) {
+            { "arg-type", QLIT_QSTR("0") },
+            { "meta-type", QLIT_QSTR("event") },
+            { "name", QLIT_QSTR("Event") },
+            { }
+        })),
+        QLIT_QDICT(((QLitDictEntry[]) {
+            { "members", QLIT_QLIST(((QLitObject[]) {
+                { }
+            })) },
+            { "meta-type", QLIT_QSTR("object") },
+            { "name", QLIT_QSTR("0") },
+            { }
+        })),
+        ...
+        { }
+    }));
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index f66c397fb0e..1d46a6d6b6f 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -13,26 +13,36 @@ See the COPYING file in the top-level directory.
 from qapi.common import *


-# Caveman's json.dumps() replacement (we're stuck at Python 2.4)
-# TODO try to use json.dumps() once we get unstuck
-def to_json(obj, level=0):
+def to_qlit(obj, level=0, suppress_first_indent=False):
+
+    def indent(level):
+        return level * 4 * ' '
+
+    ret = ''
+    if not suppress_first_indent:
+        ret += indent(level)
     if obj is None:
-        ret = 'null'
+        ret += 'QLIT_QNULL'
     elif isinstance(obj, str):
-        ret = '"' + obj.replace('"', r'\"') + '"'
+        ret += 'QLIT_QSTR(' + to_c_string(obj) + ')'
     elif isinstance(obj, list):
-        elts = [to_json(elt, level + 1)
+        elts = [to_qlit(elt, level + 1)
                 for elt in obj]
-        ret = '[' + ', '.join(elts) + ']'
+        elts.append(indent(level + 1) + "{}")
+        ret += 'QLIT_QLIST(((QLitObject[]) {\n'
+        ret += ',\n'.join(elts) + '\n'
+        ret += indent(level) + '}))'
     elif isinstance(obj, dict):
-        elts = ['"%s": %s' % (key.replace('"', r'\"'),
-                              to_json(obj[key], level + 1))
-                for key in sorted(obj.keys())]
-        ret = '{' + ', '.join(elts) + '}'
+        elts = []
+        for key, value in sorted(obj.items()):
+            elts.append(indent(level + 1) + '{ %s, %s }' %
+                        (to_c_string(key), to_qlit(value, level + 1, True)))
+        elts.append(indent(level + 1) + '{}')
+        ret += 'QLIT_QDICT(((QLitDictEntry[]) {\n'
+        ret += ',\n'.join(elts) + '\n'
+        ret += indent(level) + '}))'
     else:
         assert False                # not implemented
-    if level == 1:
-        ret = '\n' + ret
     return ret


@@ -48,7 +58,7 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaMonolithicCVisitor):
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
         self._schema = None
-        self._jsons = []
+        self._qlits = []
         self._used_types = []
         self._name_map = {}
         self._genc.add(mcgen('''
@@ -63,27 +73,27 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaMonolithicCVisitor):

     def visit_end(self):
         # visit the types that are actually used
-        jsons = self._jsons
-        self._jsons = []
+        qlits = self._qlits
+        self._qlits = []
         for typ in self._used_types:
             typ.visit(self)
         # generate C
         # TODO can generate awfully long lines
-        jsons.extend(self._jsons)
-        name = c_name(self._prefix, protect=False) + 'qmp_schema_json'
+        qlits.extend(self._qlits)
+        name = c_name(self._prefix, protect=False) + 'qmp_schema_qlit'
         self._genh.add(mcgen('''
-extern const char %(c_name)s[];
+#include "qapi/qmp/qlit.h"
+
+extern const QLitObject %(c_name)s;
 ''',
                              c_name=c_name(name)))
-        lines = to_json(jsons).split('\n')
-        c_string = '\n    '.join([to_c_string(line) for line in lines])
         self._genc.add(mcgen('''
-const char %(c_name)s[] = %(c_string)s;
+const QLitObject %(c_name)s = %(c_string)s;
 ''',
                              c_name=c_name(name),
-                             c_string=c_string))
+                             c_string=to_qlit(qlits)))
         self._schema = None
-        self._jsons = []
+        self._qlits = []
         self._used_types = []
         self._name_map = {}

@@ -117,12 +127,12 @@ const char %(c_name)s[] = %(c_string)s;
             return '[' + self._use_type(typ.element_type) + ']'
         return self._name(typ.name)

-    def _gen_json(self, name, mtype, obj):
+    def _gen_qlit(self, name, mtype, obj):
         if mtype not in ('command', 'event', 'builtin', 'array'):
             name = self._name(name)
         obj['name'] = name
         obj['meta-type'] = mtype
-        self._jsons.append(obj)
+        self._qlits.append(obj)

     def _gen_member(self, member):
         ret = {'name': member.name, 'type': self._use_type(member.type)}
@@ -138,24 +148,24 @@ const char %(c_name)s[] = %(c_string)s;
         return {'case': variant.name, 'type': self._use_type(variant.type)}

     def visit_builtin_type(self, name, info, json_type):
-        self._gen_json(name, 'builtin', {'json-type': json_type})
+        self._gen_qlit(name, 'builtin', {'json-type': json_type})

     def visit_enum_type(self, name, info, values, prefix):
-        self._gen_json(name, 'enum', {'values': values})
+        self._gen_qlit(name, 'enum', {'values': values})

     def visit_array_type(self, name, info, element_type):
         element = self._use_type(element_type)
-        self._gen_json('[' + element + ']', 'array', {'element-type': element})
+        self._gen_qlit('[' + element + ']', 'array', {'element-type': element})

     def visit_object_type_flat(self, name, info, members, variants):
         obj = {'members': [self._gen_member(m) for m in members]}
         if variants:
             obj.update(self._gen_variants(variants.tag_member.name,
                                           variants.variants))
-        self._gen_json(name, 'object', obj)
+        self._gen_qlit(name, 'object', obj)

     def visit_alternate_type(self, name, info, variants):
-        self._gen_json(name, 'alternate',
+        self._gen_qlit(name, 'alternate',
                        {'members': [{'type': self._use_type(m.type)}
                                     for m in variants.variants]})

@@ -163,13 +173,13 @@ const char %(c_name)s[] = %(c_string)s;
                       gen, success_response, boxed):
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
-        self._gen_json(name, 'command',
+        self._gen_qlit(name, 'command',
                        {'arg-type': self._use_type(arg_type),
                         'ret-type': self._use_type(ret_type)})

     def visit_event(self, name, info, arg_type, boxed):
         arg_type = arg_type or self._schema.the_empty_object_type
-        self._gen_json(name, 'event', {'arg-type': self._use_type(arg_type)})
+        self._gen_qlit(name, 'event', {'arg-type': self._use_type(arg_type)})


 def gen_introspect(schema, output_dir, prefix, opt_unmask):
diff --git a/monitor.c b/monitor.c
index a4417f26cde..db166fab52f 100644
--- a/monitor.c
+++ b/monitor.c
@@ -956,7 +956,7 @@ EventInfoList *qmp_query_events(Error **errp)
 static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
                                  Error **errp)
 {
-    *ret_data = qobject_from_json(qmp_schema_json, &error_abort);
+    *ret_data = qobject_from_qlit(&qmp_schema_qlit);
 }

 /*
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 79b1a8cb175..55db6bdef13 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -1250,24 +1250,27 @@ static void test_visitor_in_fail_alternate(TestInputVisitorData *data,
 }

 static void do_test_visitor_in_qmp_introspect(TestInputVisitorData *data,
-                                              const char *schema_json)
+                                              const QLitObject *qlit)
 {
     SchemaInfoList *schema = NULL;
+    QObject *obj = qobject_from_qlit(qlit);
     Visitor *v;

-    v = visitor_input_test_init_raw(data, schema_json);
+    v = qobject_input_visitor_new(obj);

     visit_type_SchemaInfoList(v, NULL, &schema, &error_abort);
     g_assert(schema);

     qapi_free_SchemaInfoList(schema);
+    qobject_decref(obj);
+    visit_free(v);
 }

 static void test_visitor_in_qmp_introspect(TestInputVisitorData *data,
                                            const void *unused)
 {
-    do_test_visitor_in_qmp_introspect(data, test_qmp_schema_json);
-    do_test_visitor_in_qmp_introspect(data, qmp_schema_json);
+    do_test_visitor_in_qmp_introspect(data, &test_qmp_schema_qlit);
+    do_test_visitor_in_qmp_introspect(data, &qmp_schema_qlit);
 }

 int main(int argc, char **argv)
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 05/36] compiler: Add QEMU_BUILD_BUG_MSG() macro
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (3 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 04/36] qapi: generate a literal qobject for introspection Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 06/36] qapi: Add qobject_to() Eric Blake
                   ` (31 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Max Reitz

From: Max Reitz <mreitz@redhat.com>

_Static_assert() allows us to specify messages, and that may come in
handy.  Even without _Static_assert(), encouraging developers to put a
helpful message next to the QEMU_BUILD_BUG_* may make debugging easier
whenever it breaks.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20180224154033.29559-2-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qemu/compiler.h | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
index 2cbe6a4f16e..9f762695d11 100644
--- a/include/qemu/compiler.h
+++ b/include/qemu/compiler.h
@@ -82,15 +82,21 @@
         int:(x) ? -1 : 1; \
     }

+/* QEMU_BUILD_BUG_MSG() emits the message given if _Static_assert is
+ * supported; otherwise, it will be omitted from the compiler error
+ * message (but as it remains present in the source code, it can still
+ * be useful when debugging). */
 #if defined(CONFIG_STATIC_ASSERT)
-#define QEMU_BUILD_BUG_ON(x) _Static_assert(!(x), "not expecting: " #x)
+#define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
 #elif defined(__COUNTER__)
-#define QEMU_BUILD_BUG_ON(x) typedef QEMU_BUILD_BUG_ON_STRUCT(x) \
+#define QEMU_BUILD_BUG_MSG(x, msg) typedef QEMU_BUILD_BUG_ON_STRUCT(x) \
     glue(qemu_build_bug_on__, __COUNTER__) __attribute__((unused))
 #else
-#define QEMU_BUILD_BUG_ON(x)
+#define QEMU_BUILD_BUG_MSG(x, msg)
 #endif

+#define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
+
 #define QEMU_BUILD_BUG_ON_ZERO(x) (sizeof(QEMU_BUILD_BUG_ON_STRUCT(x)) - \
                                    sizeof(QEMU_BUILD_BUG_ON_STRUCT(x)))

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 06/36] qapi: Add qobject_to()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (4 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 05/36] compiler: Add QEMU_BUILD_BUG_MSG() macro Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 07/36] qapi: Replace qobject_to_X(o) by qobject_to(X, o) Eric Blake
                   ` (30 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Max Reitz, Markus Armbruster

From: Max Reitz <mreitz@redhat.com>

This is a dynamic casting macro that, given a QObject type, returns an
object as that type or NULL if the object is of a different type (or
NULL itself).

The macro uses lower-case letters because:
1. There does not seem to be a hard rule on whether qemu macros have to
   be upper-cased,
2. The current situation in qapi/qmp is inconsistent (compare e.g.
   QINCREF() vs. qdict_put()),
3. qobject_to() will evaluate its @obj parameter only once, thus it is
   generally not important to the caller whether it is a macro or not,
4. I prefer it aesthetically.

The macro parameter order is chosen with typename first for
consistency with other QAPI macros like QAPI_CLONE(), as well as
for legibility (read it as "qobject to" type "applied to" obj).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20180224154033.29559-3-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
[eblake: swap parameter order to list type first]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qobject.h | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h
index 012439a2e3b..ea9702270e7 100644
--- a/include/qapi/qmp/qobject.h
+++ b/include/qapi/qmp/qobject.h
@@ -50,6 +50,22 @@ struct QObject {
 #define QDECREF(obj)              \
     qobject_decref(obj ? QOBJECT(obj) : NULL)

+/* Required for qobject_to() */
+#define QTYPE_CAST_TO_QNull     QTYPE_QNULL
+#define QTYPE_CAST_TO_QNum      QTYPE_QNUM
+#define QTYPE_CAST_TO_QString   QTYPE_QSTRING
+#define QTYPE_CAST_TO_QDict     QTYPE_QDICT
+#define QTYPE_CAST_TO_QList     QTYPE_QLIST
+#define QTYPE_CAST_TO_QBool     QTYPE_QBOOL
+
+QEMU_BUILD_BUG_MSG(QTYPE__MAX != 7,
+                   "The QTYPE_CAST_TO_* list needs to be extended");
+
+#define qobject_to(type, obj) \
+    container_of(qobject_check_type(obj, glue(QTYPE_CAST_TO_, type)) ?: \
+                     QOBJECT((type *)NULL), \
+                 type, base)
+
 /* Initialize an object to default values */
 static inline void qobject_init(QObject *obj, QType type)
 {
@@ -102,4 +118,18 @@ static inline QType qobject_type(const QObject *obj)
     return obj->type;
 }

+/**
+ * qobject_check_type(): Helper function for the qobject_to() macro.
+ * Return @obj, but only if @obj is not NULL and @type is equal to
+ * @obj's type.  Return NULL otherwise.
+ */
+static inline QObject *qobject_check_type(const QObject *obj, QType type)
+{
+    if (obj && qobject_type(obj) == type) {
+        return (QObject *)obj;
+    } else {
+        return NULL;
+    }
+}
+
 #endif /* QOBJECT_H */
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 07/36] qapi: Replace qobject_to_X(o) by qobject_to(X, o)
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (5 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 06/36] qapi: Add qobject_to() Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-03-12 18:35 ` [Qemu-devel] [PULL 08/36] qapi: Remove qobject_to_X() functions Eric Blake
                   ` (29 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel
  Cc: Max Reitz, Kevin Wolf, Markus Armbruster, Josh Durgin, Jeff Cody,
	Hitoshi Mitake, Liu Yuan, Michael S. Tsirkin, Igor Mammedov,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost,
	Marcel Apfelbaum, Dr. David Alan Gilbert, Michael Roth,
	Andreas Färber, Alexander Graf, David Hildenbrand,
	Cornelia Huck, open list:Block layer core, open list:Sheepdog,
	open list:S390

From: Max Reitz <mreitz@redhat.com>

This patch was generated using the following Coccinelle script:

@@
expression Obj;
@@
(
- qobject_to_qnum(Obj)
+ qobject_to(QNum, Obj)
|
- qobject_to_qstring(Obj)
+ qobject_to(QString, Obj)
|
- qobject_to_qdict(Obj)
+ qobject_to(QDict, Obj)
|
- qobject_to_qlist(Obj)
+ qobject_to(QList, Obj)
|
- qobject_to_qbool(Obj)
+ qobject_to(QBool, Obj)
)

and a bit of manual fix-up for overly long lines and three places in
tests/check-qjson.c that Coccinelle did not find.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-4-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: swap order from qobject_to(o, X), rebase to master]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/libqtest.c                    |  6 ++---
 block.c                             |  4 +--
 block/qapi.c                        | 12 ++++-----
 block/qcow2.c                       |  2 +-
 block/rbd.c                         |  8 +++---
 block/sheepdog.c                    |  2 +-
 blockdev.c                          |  7 ++---
 hw/i386/acpi-build.c                | 14 +++++-----
 monitor.c                           |  8 +++---
 qapi/qmp-dispatch.c                 |  2 +-
 qapi/qobject-input-visitor.c        | 20 +++++++-------
 qapi/qobject-output-visitor.c       |  4 +--
 qga/main.c                          |  2 +-
 qmp.c                               |  2 +-
 qobject/json-parser.c               |  2 +-
 qobject/qbool.c                     |  4 +--
 qobject/qdict.c                     | 38 +++++++++++++-------------
 qobject/qjson.c                     | 10 +++----
 qobject/qlist.c                     |  6 ++---
 qobject/qlit.c                      | 10 +++----
 qobject/qnum.c                      |  6 ++---
 qobject/qstring.c                   |  6 ++---
 qom/object.c                        |  8 +++---
 target/i386/cpu.c                   |  2 +-
 target/s390x/cpu_models.c           |  2 +-
 tests/check-qdict.c                 | 20 +++++++-------
 tests/check-qjson.c                 | 41 ++++++++++++++--------------
 tests/check-qlist.c                 |  4 +--
 tests/check-qlit.c                  | 10 +++----
 tests/check-qnum.c                  |  4 +--
 tests/check-qobject.c               |  2 +-
 tests/check-qstring.c               |  2 +-
 tests/device-introspect-test.c      | 14 +++++-----
 tests/numa-test.c                   |  8 +++---
 tests/qom-test.c                    |  4 +--
 tests/test-char.c                   |  2 +-
 tests/test-keyval.c                 |  8 +++---
 tests/test-qga.c                    | 19 ++++++-------
 tests/test-qmp-cmds.c               | 12 ++++-----
 tests/test-qmp-event.c              | 16 +++++------
 tests/test-qobject-input-visitor.c  | 10 +++----
 tests/test-qobject-output-visitor.c | 54 ++++++++++++++++++-------------------
 tests/test-x86-cpuid-compat.c       | 17 ++++++------
 util/keyval.c                       |  4 +--
 util/qemu-config.c                  |  2 +-
 util/qemu-option.c                  |  6 ++---
 46 files changed, 225 insertions(+), 221 deletions(-)

diff --git a/tests/libqtest.c b/tests/libqtest.c
index 13c910069b5..200b2b9e92a 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -430,7 +430,7 @@ static void qmp_response(JSONMessageParser *parser, GQueue *tokens)
     }

     g_assert(!qmp->response);
-    qmp->response = qobject_to_qdict(obj);
+    qmp->response = qobject_to(QDict, obj);
     g_assert(qmp->response);
 }

@@ -1008,11 +1008,11 @@ void qtest_cb_for_every_machine(void (*cb)(const char *machine))
     g_assert(list);

     for (p = qlist_first(list); p; p = qlist_next(p)) {
-        minfo = qobject_to_qdict(qlist_entry_obj(p));
+        minfo = qobject_to(QDict, qlist_entry_obj(p));
         g_assert(minfo);
         qobj = qdict_get(minfo, "name");
         g_assert(qobj);
-        qstr = qobject_to_qstring(qobj);
+        qstr = qobject_to(QString, qobj);
         g_assert(qstr);
         mname = qstring_get_str(qstr);
         cb(mname);
diff --git a/block.c b/block.c
index 75a9fd49de1..cd74da3a601 100644
--- a/block.c
+++ b/block.c
@@ -1457,7 +1457,7 @@ static QDict *parse_json_filename(const char *filename, Error **errp)
         return NULL;
     }

-    options = qobject_to_qdict(options_obj);
+    options = qobject_to(QDict, options_obj);
     if (!options) {
         qobject_decref(options_obj);
         error_setg(errp, "Invalid JSON object given");
@@ -2433,7 +2433,7 @@ BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp)
         }
         visit_complete(v, &obj);

-        qdict = qobject_to_qdict(obj);
+        qdict = qobject_to(QDict, obj);
         qdict_flatten(qdict);

         /* bdrv_open_inherit() defaults to the values in bdrv_flags (for
diff --git a/block/qapi.c b/block/qapi.c
index 4c9923d262b..f2e0aa2cbe7 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -647,29 +647,29 @@ static void dump_qobject(fprintf_function func_fprintf, void *f,
 {
     switch (qobject_type(obj)) {
         case QTYPE_QNUM: {
-            QNum *value = qobject_to_qnum(obj);
+            QNum *value = qobject_to(QNum, obj);
             char *tmp = qnum_to_string(value);
             func_fprintf(f, "%s", tmp);
             g_free(tmp);
             break;
         }
         case QTYPE_QSTRING: {
-            QString *value = qobject_to_qstring(obj);
+            QString *value = qobject_to(QString, obj);
             func_fprintf(f, "%s", qstring_get_str(value));
             break;
         }
         case QTYPE_QDICT: {
-            QDict *value = qobject_to_qdict(obj);
+            QDict *value = qobject_to(QDict, obj);
             dump_qdict(func_fprintf, f, comp_indent, value);
             break;
         }
         case QTYPE_QLIST: {
-            QList *value = qobject_to_qlist(obj);
+            QList *value = qobject_to(QList, obj);
             dump_qlist(func_fprintf, f, comp_indent, value);
             break;
         }
         case QTYPE_QBOOL: {
-            QBool *value = qobject_to_qbool(obj);
+            QBool *value = qobject_to(QBool, obj);
             func_fprintf(f, "%s", qbool_get_bool(value) ? "true" : "false");
             break;
         }
@@ -730,7 +730,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f,

     visit_type_ImageInfoSpecific(v, NULL, &info_spec, &error_abort);
     visit_complete(v, &obj);
-    data = qdict_get(qobject_to_qdict(obj), "data");
+    data = qdict_get(qobject_to(QDict, obj), "data");
     dump_qobject(func_fprintf, f, 1, data);
     qobject_decref(obj);
     visit_free(v);
diff --git a/block/qcow2.c b/block/qcow2.c
index 7472af69312..cf4f3becae6 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3125,7 +3125,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
     /* Now get the QAPI type BlockdevCreateOptions */
     qobj = qdict_crumple(qdict, errp);
     QDECREF(qdict);
-    qdict = qobject_to_qdict(qobj);
+    qdict = qobject_to(QDict, qobj);
     if (qdict == NULL) {
         ret = -EINVAL;
         goto finish;
diff --git a/block/rbd.c b/block/rbd.c
index 294ed07ac41..5b64849dc67 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -263,14 +263,14 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const char *keypairs_json,
     if (!keypairs_json) {
         return ret;
     }
-    keypairs = qobject_to_qlist(qobject_from_json(keypairs_json,
-                                                  &error_abort));
+    keypairs = qobject_to(QList,
+                          qobject_from_json(keypairs_json, &error_abort));
     remaining = qlist_size(keypairs) / 2;
     assert(remaining);

     while (remaining--) {
-        name = qobject_to_qstring(qlist_pop(keypairs));
-        value = qobject_to_qstring(qlist_pop(keypairs));
+        name = qobject_to(QString, qlist_pop(keypairs));
+        value = qobject_to(QString, qlist_pop(keypairs));
         assert(name && value);
         key = qstring_get_str(name);

diff --git a/block/sheepdog.c b/block/sheepdog.c
index 8680b2926f0..abda8857768 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1887,7 +1887,7 @@ static int sd_create_prealloc(BlockdevOptionsSheepdog *location, int64_t size,
         return -EINVAL;
     }

-    qdict = qobject_to_qdict(obj);
+    qdict = qobject_to(QDict, obj);
     qdict_flatten(qdict);

     qdict_put_str(qdict, "driver", "sheepdog");
diff --git a/blockdev.c b/blockdev.c
index 1fbfd3a2c40..c9a212b3149 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -334,7 +334,8 @@ static bool parse_stats_intervals(BlockAcctStats *stats, QList *intervals,

         case QTYPE_QSTRING: {
             unsigned long long length;
-            const char *str = qstring_get_str(qobject_to_qstring(entry->value));
+            const char *str = qstring_get_str(qobject_to(QString,
+                                                         entry->value));
             if (parse_uint_full(str, &length, 10) == 0 &&
                 length > 0 && length <= UINT_MAX) {
                 block_acct_add_interval(stats, (unsigned) length);
@@ -346,7 +347,7 @@ static bool parse_stats_intervals(BlockAcctStats *stats, QList *intervals,
         }

         case QTYPE_QNUM: {
-            int64_t length = qnum_get_int(qobject_to_qnum(entry->value));
+            int64_t length = qnum_get_int(qobject_to(QNum, entry->value));

             if (length > 0 && length <= UINT_MAX) {
                 block_acct_add_interval(stats, (unsigned) length);
@@ -3982,7 +3983,7 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)
     }

     visit_complete(v, &obj);
-    qdict = qobject_to_qdict(obj);
+    qdict = qobject_to(QDict, obj);

     qdict_flatten(qdict);

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index deb440f2863..2d62499bf12 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -154,21 +154,21 @@ static void acpi_get_pm_info(AcpiPmInfo *pm)
     /* Fill in optional s3/s4 related properties */
     o = object_property_get_qobject(obj, ACPI_PM_PROP_S3_DISABLED, NULL);
     if (o) {
-        pm->s3_disabled = qnum_get_uint(qobject_to_qnum(o));
+        pm->s3_disabled = qnum_get_uint(qobject_to(QNum, o));
     } else {
         pm->s3_disabled = false;
     }
     qobject_decref(o);
     o = object_property_get_qobject(obj, ACPI_PM_PROP_S4_DISABLED, NULL);
     if (o) {
-        pm->s4_disabled = qnum_get_uint(qobject_to_qnum(o));
+        pm->s4_disabled = qnum_get_uint(qobject_to(QNum, o));
     } else {
         pm->s4_disabled = false;
     }
     qobject_decref(o);
     o = object_property_get_qobject(obj, ACPI_PM_PROP_S4_VAL, NULL);
     if (o) {
-        pm->s4_val = qnum_get_uint(qobject_to_qnum(o));
+        pm->s4_val = qnum_get_uint(qobject_to(QNum, o));
     } else {
         pm->s4_val = false;
     }
@@ -514,7 +514,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,

     bsel = object_property_get_qobject(OBJECT(bus), ACPI_PCIHP_PROP_BSEL, NULL);
     if (bsel) {
-        uint64_t bsel_val = qnum_get_uint(qobject_to_qnum(bsel));
+        uint64_t bsel_val = qnum_get_uint(qobject_to(QNum, bsel));

         aml_append(parent_scope, aml_name_decl("BSEL", aml_int(bsel_val)));
         notify_method = aml_method("DVNT", 2, AML_NOTSERIALIZED);
@@ -624,7 +624,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,

     /* If bus supports hotplug select it and notify about local events */
     if (bsel) {
-        uint64_t bsel_val = qnum_get_uint(qobject_to_qnum(bsel));
+        uint64_t bsel_val = qnum_get_uint(qobject_to(QNum, bsel));

         aml_append(method, aml_store(aml_int(bsel_val), aml_name("BNUM")));
         aml_append(method,
@@ -2638,12 +2638,12 @@ static bool acpi_get_mcfg(AcpiMcfgInfo *mcfg)
     if (!o) {
         return false;
     }
-    mcfg->mcfg_base = qnum_get_uint(qobject_to_qnum(o));
+    mcfg->mcfg_base = qnum_get_uint(qobject_to(QNum, o));
     qobject_decref(o);

     o = object_property_get_qobject(pci_host, PCIE_HOST_MCFG_SIZE, NULL);
     assert(o);
-    mcfg->mcfg_size = qnum_get_uint(qobject_to_qnum(o));
+    mcfg->mcfg_size = qnum_get_uint(qobject_to(QNum, o));
     qobject_decref(o);
     return true;
 }
diff --git a/monitor.c b/monitor.c
index db166fab52f..c18a2f2dc98 100644
--- a/monitor.c
+++ b/monitor.c
@@ -447,7 +447,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp)
         /* Unthrottled event */
         monitor_qapi_event_emit(event, qdict);
     } else {
-        QDict *data = qobject_to_qdict(qdict_get(qdict, "data"));
+        QDict *data = qobject_to(QDict, qdict_get(qdict, "data"));
         MonitorQAPIEventState key = { .event = event, .data = data };

         evstate = g_hash_table_lookup(monitor_qapi_event_state, &key);
@@ -3774,7 +3774,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err_out;
     }

-    qdict = qobject_to_qdict(req);
+    qdict = qobject_to(QDict, req);
     if (qdict) {
         id = qdict_get(qdict, "id");
         qobject_incref(id);
@@ -3790,7 +3790,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     rsp = qmp_dispatch(cur_mon->qmp.commands, req);

     if (mon->qmp.commands == &qmp_cap_negotiation_commands) {
-        qdict = qdict_get_qdict(qobject_to_qdict(rsp), "error");
+        qdict = qdict_get_qdict(qobject_to(QDict, rsp), "error");
         if (qdict
             && !g_strcmp0(qdict_get_try_str(qdict, "class"),
                     QapiErrorClass_str(ERROR_CLASS_COMMAND_NOT_FOUND))) {
@@ -3811,7 +3811,7 @@ err_out:

     if (rsp) {
         if (id) {
-            qdict_put_obj(qobject_to_qdict(rsp), "id", id);
+            qdict_put_obj(qobject_to(QDict, rsp), "id", id);
             id = NULL;
         }

diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index e31ac4be1fb..af537f3d7d2 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -26,7 +26,7 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
     bool has_exec_key = false;
     QDict *dict = NULL;

-    dict = qobject_to_qdict(request);
+    dict = qobject_to(QDict, request);
     if (!dict) {
         error_setg(errp, "QMP input must be a JSON object");
         return NULL;
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 023317b05fc..0b5293385e5 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -137,7 +137,7 @@ static QObject *qobject_input_try_get_object(QObjectInputVisitor *qiv,

     if (qobject_type(qobj) == QTYPE_QDICT) {
         assert(name);
-        ret = qdict_get(qobject_to_qdict(qobj), name);
+        ret = qdict_get(qobject_to(QDict, qobj), name);
         if (tos->h && consume && ret) {
             bool removed = g_hash_table_remove(tos->h, name);
             assert(removed);
@@ -185,7 +185,7 @@ static const char *qobject_input_get_keyval(QObjectInputVisitor *qiv,
         return NULL;
     }

-    qstr = qobject_to_qstring(qobj);
+    qstr = qobject_to(QString, qobj);
     if (!qstr) {
         switch (qobject_type(qobj)) {
         case QTYPE_QDICT:
@@ -224,11 +224,11 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,

     if (qobject_type(obj) == QTYPE_QDICT) {
         h = g_hash_table_new(g_str_hash, g_str_equal);
-        qdict_iter(qobject_to_qdict(obj), qdict_add_key, h);
+        qdict_iter(qobject_to(QDict, obj), qdict_add_key, h);
         tos->h = h;
     } else {
         assert(qobject_type(obj) == QTYPE_QLIST);
-        tos->entry = qlist_first(qobject_to_qlist(obj));
+        tos->entry = qlist_first(qobject_to(QList, obj));
         tos->index = -1;
     }

@@ -395,7 +395,7 @@ static void qobject_input_type_int64(Visitor *v, const char *name, int64_t *obj,
     if (!qobj) {
         return;
     }
-    qnum = qobject_to_qnum(qobj);
+    qnum = qobject_to(QNum, qobj);
     if (!qnum || !qnum_get_try_int(qnum, obj)) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
                    full_name(qiv, name), "integer");
@@ -430,7 +430,7 @@ static void qobject_input_type_uint64(Visitor *v, const char *name,
     if (!qobj) {
         return;
     }
-    qnum = qobject_to_qnum(qobj);
+    qnum = qobject_to(QNum, qobj);
     if (!qnum) {
         goto err;
     }
@@ -477,7 +477,7 @@ static void qobject_input_type_bool(Visitor *v, const char *name, bool *obj,
     if (!qobj) {
         return;
     }
-    qbool = qobject_to_qbool(qobj);
+    qbool = qobject_to(QBool, qobj);
     if (!qbool) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
                    full_name(qiv, name), "boolean");
@@ -518,7 +518,7 @@ static void qobject_input_type_str(Visitor *v, const char *name, char **obj,
     if (!qobj) {
         return;
     }
-    qstr = qobject_to_qstring(qobj);
+    qstr = qobject_to(QString, qobj);
     if (!qstr) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
                    full_name(qiv, name), "string");
@@ -547,7 +547,7 @@ static void qobject_input_type_number(Visitor *v, const char *name, double *obj,
     if (!qobj) {
         return;
     }
-    qnum = qobject_to_qnum(qobj);
+    qnum = qobject_to(QNum, qobj);
     if (!qnum) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
                    full_name(qiv, name), "number");
@@ -734,7 +734,7 @@ Visitor *qobject_input_visitor_new_str(const char *str,
             }
             return NULL;
         }
-        args = qobject_to_qdict(obj);
+        args = qobject_to(QDict, obj);
         assert(args);
         v = qobject_input_visitor_new(QOBJECT(args));
     } else {
diff --git a/qapi/qobject-output-visitor.c b/qapi/qobject-output-visitor.c
index 7c3b42cfe2b..877e37eeb88 100644
--- a/qapi/qobject-output-visitor.c
+++ b/qapi/qobject-output-visitor.c
@@ -92,11 +92,11 @@ static void qobject_output_add_obj(QObjectOutputVisitor *qov, const char *name,
         switch (qobject_type(cur)) {
         case QTYPE_QDICT:
             assert(name);
-            qdict_put_obj(qobject_to_qdict(cur), name, value);
+            qdict_put_obj(qobject_to(QDict, cur), name, value);
             break;
         case QTYPE_QLIST:
             assert(!name);
-            qlist_append_obj(qobject_to_qlist(cur), value);
+            qlist_append_obj(qobject_to(QList, cur), value);
             break;
         default:
             g_assert_not_reached();
diff --git a/qga/main.c b/qga/main.c
index f9c83050c5d..7cc1748c04f 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -607,7 +607,7 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens)
     g_assert(s && parser);

     g_debug("process_event: called");
-    qdict = qobject_to_qdict(json_parser_parse_err(tokens, NULL, &err));
+    qdict = qobject_to(QDict, json_parser_parse_err(tokens, NULL, &err));
     if (err || !qdict) {
         QDECREF(qdict);
         qdict = qdict_new();
diff --git a/qmp.c b/qmp.c
index 8c7d1cc4790..7e606d84867 100644
--- a/qmp.c
+++ b/qmp.c
@@ -705,7 +705,7 @@ void qmp_object_add(const char *type, const char *id,
     Object *obj;

     if (props) {
-        pdict = qobject_to_qdict(props);
+        pdict = qobject_to(QDict, props);
         if (!pdict) {
             error_setg(errp, QERR_INVALID_PARAMETER_TYPE, "props", "dict");
             return;
diff --git a/qobject/json-parser.c b/qobject/json-parser.c
index b7245624158..055c7f0272a 100644
--- a/qobject/json-parser.c
+++ b/qobject/json-parser.c
@@ -308,7 +308,7 @@ static int parse_pair(JSONParserContext *ctxt, QDict *dict, va_list *ap)
         goto out;
     }

-    qdict_put_obj(dict, qstring_get_str(qobject_to_qstring(key)), value);
+    qdict_put_obj(dict, qstring_get_str(qobject_to(QString, key)), value);

     qobject_decref(key);

diff --git a/qobject/qbool.c b/qobject/qbool.c
index e5a7a538793..5be6277cca8 100644
--- a/qobject/qbool.c
+++ b/qobject/qbool.c
@@ -55,7 +55,7 @@ QBool *qobject_to_qbool(const QObject *obj)
  */
 bool qbool_is_equal(const QObject *x, const QObject *y)
 {
-    return qobject_to_qbool(x)->value == qobject_to_qbool(y)->value;
+    return qobject_to(QBool, x)->value == qobject_to(QBool, y)->value;
 }

 /**
@@ -65,5 +65,5 @@ bool qbool_is_equal(const QObject *x, const QObject *y)
 void qbool_destroy_obj(QObject *obj)
 {
     assert(obj != NULL);
-    g_free(qobject_to_qbool(obj));
+    g_free(qobject_to(QBool, obj));
 }
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 229b8c840b1..1e588123d00 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -206,7 +206,7 @@ size_t qdict_size(const QDict *qdict)
  */
 double qdict_get_double(const QDict *qdict, const char *key)
 {
-    return qnum_get_double(qobject_to_qnum(qdict_get(qdict, key)));
+    return qnum_get_double(qobject_to(QNum, qdict_get(qdict, key)));
 }

 /**
@@ -219,7 +219,7 @@ double qdict_get_double(const QDict *qdict, const char *key)
  */
 int64_t qdict_get_int(const QDict *qdict, const char *key)
 {
-    return qnum_get_int(qobject_to_qnum(qdict_get(qdict, key)));
+    return qnum_get_int(qobject_to(QNum, qdict_get(qdict, key)));
 }

 /**
@@ -232,7 +232,7 @@ int64_t qdict_get_int(const QDict *qdict, const char *key)
  */
 bool qdict_get_bool(const QDict *qdict, const char *key)
 {
-    return qbool_get_bool(qobject_to_qbool(qdict_get(qdict, key)));
+    return qbool_get_bool(qobject_to(QBool, qdict_get(qdict, key)));
 }

 /**
@@ -240,7 +240,7 @@ bool qdict_get_bool(const QDict *qdict, const char *key)
  */
 QList *qdict_get_qlist(const QDict *qdict, const char *key)
 {
-    return qobject_to_qlist(qdict_get(qdict, key));
+    return qobject_to(QList, qdict_get(qdict, key));
 }

 /**
@@ -248,7 +248,7 @@ QList *qdict_get_qlist(const QDict *qdict, const char *key)
  */
 QDict *qdict_get_qdict(const QDict *qdict, const char *key)
 {
-    return qobject_to_qdict(qdict_get(qdict, key));
+    return qobject_to(QDict, qdict_get(qdict, key));
 }

 /**
@@ -262,7 +262,7 @@ QDict *qdict_get_qdict(const QDict *qdict, const char *key)
  */
 const char *qdict_get_str(const QDict *qdict, const char *key)
 {
-    return qstring_get_str(qobject_to_qstring(qdict_get(qdict, key)));
+    return qstring_get_str(qobject_to(QString, qdict_get(qdict, key)));
 }

 /**
@@ -275,7 +275,7 @@ const char *qdict_get_str(const QDict *qdict, const char *key)
 int64_t qdict_get_try_int(const QDict *qdict, const char *key,
                           int64_t def_value)
 {
-    QNum *qnum = qobject_to_qnum(qdict_get(qdict, key));
+    QNum *qnum = qobject_to(QNum, qdict_get(qdict, key));
     int64_t val;

     if (!qnum || !qnum_get_try_int(qnum, &val)) {
@@ -294,7 +294,7 @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key,
  */
 bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value)
 {
-    QBool *qbool = qobject_to_qbool(qdict_get(qdict, key));
+    QBool *qbool = qobject_to(QBool, qdict_get(qdict, key));

     return qbool ? qbool_get_bool(qbool) : def_value;
 }
@@ -309,7 +309,7 @@ bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value)
  */
 const char *qdict_get_try_str(const QDict *qdict, const char *key)
 {
-    QString *qstr = qobject_to_qstring(qdict_get(qdict, key));
+    QString *qstr = qobject_to(QString, qdict_get(qdict, key));

     return qstr ? qstring_get_str(qstr) : NULL;
 }
@@ -432,8 +432,8 @@ void qdict_del(QDict *qdict, const char *key)
  */
 bool qdict_is_equal(const QObject *x, const QObject *y)
 {
-    const QDict *dict_x = qobject_to_qdict(x);
-    const QDict *dict_y = qobject_to_qdict(y);
+    const QDict *dict_x = qobject_to(QDict, x);
+    const QDict *dict_y = qobject_to(QDict, y);
     const QDictEntry *e;

     if (qdict_size(dict_x) != qdict_size(dict_y)) {
@@ -461,7 +461,7 @@ void qdict_destroy_obj(QObject *obj)
     QDict *qdict;

     assert(obj != NULL);
-    qdict = qobject_to_qdict(obj);
+    qdict = qobject_to(QDict, obj);

     for (i = 0; i < QDICT_BUCKET_MAX; i++) {
         QDictEntry *entry = QLIST_FIRST(&qdict->table[i]);
@@ -532,9 +532,9 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
         new_key = g_strdup_printf("%s.%i", prefix, i);

         if (qobject_type(value) == QTYPE_QDICT) {
-            qdict_flatten_qdict(qobject_to_qdict(value), target, new_key);
+            qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
         } else if (qobject_type(value) == QTYPE_QLIST) {
-            qdict_flatten_qlist(qobject_to_qlist(value), target, new_key);
+            qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
         } else {
             /* All other types are moved to the target unchanged. */
             qobject_incref(value);
@@ -568,11 +568,11 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
         if (qobject_type(value) == QTYPE_QDICT) {
             /* Entries of QDicts are processed recursively, the QDict object
              * itself disappears. */
-            qdict_flatten_qdict(qobject_to_qdict(value), target,
+            qdict_flatten_qdict(qobject_to(QDict, value), target,
                                 new_key ? new_key : entry->key);
             delete = true;
         } else if (qobject_type(value) == QTYPE_QLIST) {
-            qdict_flatten_qlist(qobject_to_qlist(value), target,
+            qdict_flatten_qlist(qobject_to(QList, value), target,
                                 new_key ? new_key : entry->key);
             delete = true;
         } else if (prefix) {
@@ -904,7 +904,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
                 qdict_put_obj(two_level, prefix, child);
             }
             qobject_incref(ent->value);
-            qdict_put_obj(qobject_to_qdict(child), suffix, ent->value);
+            qdict_put_obj(qobject_to(QDict, child), suffix, ent->value);
         } else {
             if (child) {
                 error_setg(errp, "Key %s prefix is already set as a dict",
@@ -926,7 +926,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
          ent = qdict_next(two_level, ent)) {

         if (qobject_type(ent->value) == QTYPE_QDICT) {
-            child = qdict_crumple(qobject_to_qdict(ent->value), errp);
+            child = qdict_crumple(qobject_to(QDict, ent->value), errp);
             if (!child) {
                 goto error;
             }
@@ -961,7 +961,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
             }

             qobject_incref(child);
-            qlist_append_obj(qobject_to_qlist(dst), child);
+            qlist_append_obj(qobject_to(QList, dst), child);
         }
         QDECREF(multi_level);
         multi_level = NULL;
diff --git a/qobject/qjson.c b/qobject/qjson.c
index e1ce75651cc..655d38adf1b 100644
--- a/qobject/qjson.c
+++ b/qobject/qjson.c
@@ -137,14 +137,14 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
         qstring_append(str, "null");
         break;
     case QTYPE_QNUM: {
-        QNum *val = qobject_to_qnum(obj);
+        QNum *val = qobject_to(QNum, obj);
         char *buffer = qnum_to_string(val);
         qstring_append(str, buffer);
         g_free(buffer);
         break;
     }
     case QTYPE_QSTRING: {
-        QString *val = qobject_to_qstring(obj);
+        QString *val = qobject_to(QString, obj);
         const char *ptr;
         int cp;
         char buf[16];
@@ -201,7 +201,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
     }
     case QTYPE_QDICT: {
         ToJsonIterState s;
-        QDict *val = qobject_to_qdict(obj);
+        QDict *val = qobject_to(QDict, obj);

         s.count = 0;
         s.str = str;
@@ -220,7 +220,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
     }
     case QTYPE_QLIST: {
         ToJsonIterState s;
-        QList *val = qobject_to_qlist(obj);
+        QList *val = qobject_to(QList, obj);

         s.count = 0;
         s.str = str;
@@ -238,7 +238,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
         break;
     }
     case QTYPE_QBOOL: {
-        QBool *val = qobject_to_qbool(obj);
+        QBool *val = qobject_to(QBool, obj);

         if (qbool_get_bool(val)) {
             qstring_append(str, "true");
diff --git a/qobject/qlist.c b/qobject/qlist.c
index 613a95c12b9..77f19ffda78 100644
--- a/qobject/qlist.c
+++ b/qobject/qlist.c
@@ -173,8 +173,8 @@ QList *qobject_to_qlist(const QObject *obj)
  */
 bool qlist_is_equal(const QObject *x, const QObject *y)
 {
-    const QList *list_x = qobject_to_qlist(x);
-    const QList *list_y = qobject_to_qlist(y);
+    const QList *list_x = qobject_to(QList, x);
+    const QList *list_y = qobject_to(QList, y);
     const QListEntry *entry_x, *entry_y;

     entry_x = qlist_first(list_x);
@@ -203,7 +203,7 @@ void qlist_destroy_obj(QObject *obj)
     QListEntry *entry, *next_entry;

     assert(obj != NULL);
-    qlist = qobject_to_qlist(obj);
+    qlist = qobject_to(QList, obj);

     QTAILQ_FOREACH_SAFE(entry, &qlist->head, next, next_entry) {
         QTAILQ_REMOVE(&qlist->head, entry, next);
diff --git a/qobject/qlit.c b/qobject/qlit.c
index d0e1b72fd01..be8332136c2 100644
--- a/qobject/qlit.c
+++ b/qobject/qlit.c
@@ -69,16 +69,16 @@ bool qlit_equal_qobject(const QLitObject *lhs, const QObject *rhs)

     switch (lhs->type) {
     case QTYPE_QBOOL:
-        return lhs->value.qbool == qbool_get_bool(qobject_to_qbool(rhs));
+        return lhs->value.qbool == qbool_get_bool(qobject_to(QBool, rhs));
     case QTYPE_QNUM:
-        return lhs->value.qnum ==  qnum_get_int(qobject_to_qnum(rhs));
+        return lhs->value.qnum ==  qnum_get_int(qobject_to(QNum, rhs));
     case QTYPE_QSTRING:
         return (strcmp(lhs->value.qstr,
-                       qstring_get_str(qobject_to_qstring(rhs))) == 0);
+                       qstring_get_str(qobject_to(QString, rhs))) == 0);
     case QTYPE_QDICT:
-        return qlit_equal_qdict(lhs, qobject_to_qdict(rhs));
+        return qlit_equal_qdict(lhs, qobject_to(QDict, rhs));
     case QTYPE_QLIST:
-        return qlit_equal_qlist(lhs, qobject_to_qlist(rhs));
+        return qlit_equal_qlist(lhs, qobject_to(QList, rhs));
     case QTYPE_QNULL:
         return true;
     default:
diff --git a/qobject/qnum.c b/qobject/qnum.c
index 60c395c1bce..ea091cfaa4b 100644
--- a/qobject/qnum.c
+++ b/qobject/qnum.c
@@ -221,8 +221,8 @@ QNum *qobject_to_qnum(const QObject *obj)
  */
 bool qnum_is_equal(const QObject *x, const QObject *y)
 {
-    QNum *num_x = qobject_to_qnum(x);
-    QNum *num_y = qobject_to_qnum(y);
+    QNum *num_x = qobject_to(QNum, x);
+    QNum *num_y = qobject_to(QNum, y);

     switch (num_x->kind) {
     case QNUM_I64:
@@ -271,5 +271,5 @@ bool qnum_is_equal(const QObject *x, const QObject *y)
 void qnum_destroy_obj(QObject *obj)
 {
     assert(obj != NULL);
-    g_free(qobject_to_qnum(obj));
+    g_free(qobject_to(QNum, obj));
 }
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 05b4bbc2d62..2b2153206df 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -132,8 +132,8 @@ const char *qstring_get_str(const QString *qstring)
  */
 bool qstring_is_equal(const QObject *x, const QObject *y)
 {
-    return !strcmp(qobject_to_qstring(x)->string,
-                   qobject_to_qstring(y)->string);
+    return !strcmp(qobject_to(QString, x)->string,
+                   qobject_to(QString, y)->string);
 }

 /**
@@ -145,7 +145,7 @@ void qstring_destroy_obj(QObject *obj)
     QString *qs;

     assert(obj != NULL);
-    qs = qobject_to_qstring(obj);
+    qs = qobject_to(QString, obj);
     g_free(qs->string);
     g_free(qs);
 }
diff --git a/qom/object.c b/qom/object.c
index 755ad038197..7b3a6158fde 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -1129,7 +1129,7 @@ char *object_property_get_str(Object *obj, const char *name,
     if (!ret) {
         return NULL;
     }
-    qstring = qobject_to_qstring(ret);
+    qstring = qobject_to(QString, ret);
     if (!qstring) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "string");
         retval = NULL;
@@ -1190,7 +1190,7 @@ bool object_property_get_bool(Object *obj, const char *name,
     if (!ret) {
         return false;
     }
-    qbool = qobject_to_qbool(ret);
+    qbool = qobject_to(QBool, ret);
     if (!qbool) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "boolean");
         retval = false;
@@ -1222,7 +1222,7 @@ int64_t object_property_get_int(Object *obj, const char *name,
         return -1;
     }

-    qnum = qobject_to_qnum(ret);
+    qnum = qobject_to(QNum, ret);
     if (!qnum || !qnum_get_try_int(qnum, &retval)) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "int");
         retval = -1;
@@ -1251,7 +1251,7 @@ uint64_t object_property_get_uint(Object *obj, const char *name,
     if (!ret) {
         return 0;
     }
-    qnum = qobject_to_qnum(ret);
+    qnum = qobject_to(QNum, ret);
     if (!qnum || !qnum_get_try_uint(qnum, &retval)) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "uint");
         retval = 0;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2c04645ceac..a1234cc780d 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3101,7 +3101,7 @@ arch_query_cpu_model_expansion(CpuModelExpansionType type,

     xc = x86_cpu_from_model(model->name,
                             model->has_props ?
-                                qobject_to_qdict(model->props) :
+                                qobject_to(QDict, model->props) :
                                 NULL, &err);
     if (err) {
         goto out;
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 1d5f0da4fe0..2741b6803f7 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -453,7 +453,7 @@ static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info,
     Object *obj;

     if (info->props) {
-        qdict = qobject_to_qdict(info->props);
+        qdict = qobject_to(QDict, info->props);
         if (!qdict) {
             error_setg(errp, QERR_INVALID_PARAMETER_TYPE, "props", "dict");
             return;
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index a3faea8bfc2..2e73c2f86ea 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -51,7 +51,7 @@ static void qdict_put_obj_test(void)

     g_assert(qdict_size(qdict) == 1);
     ent = QLIST_FIRST(&qdict->table[12345 % QDICT_BUCKET_MAX]);
-    qn = qobject_to_qnum(ent->value);
+    qn = qobject_to(QNum, ent->value);
     g_assert_cmpint(qnum_get_int(qn), ==, num);

     QDECREF(qdict);
@@ -81,7 +81,7 @@ static void qdict_get_test(void)
     obj = qdict_get(tests_dict, key);
     g_assert(obj != NULL);

-    qn = qobject_to_qnum(obj);
+    qn = qobject_to(QNum, obj);
     g_assert_cmpint(qnum_get_int(qn), ==, value);

     QDECREF(tests_dict);
@@ -216,7 +216,7 @@ static void qdict_del_test(void)
 static void qobject_to_qdict_test(void)
 {
     QDict *tests_dict = qdict_new();
-    g_assert(qobject_to_qdict(QOBJECT(tests_dict)) == tests_dict);
+    g_assert(qobject_to(QDict, QOBJECT(tests_dict)) == tests_dict);

     QDECREF(tests_dict);
 }
@@ -381,9 +381,9 @@ static void qdict_array_split_test(void)

     qdict_array_split(test_dict, &test_list);

-    dict1 = qobject_to_qdict(qlist_pop(test_list));
-    dict2 = qobject_to_qdict(qlist_pop(test_list));
-    int1 = qobject_to_qnum(qlist_pop(test_list));
+    dict1 = qobject_to(QDict, qlist_pop(test_list));
+    dict2 = qobject_to(QDict, qlist_pop(test_list));
+    int1 = qobject_to(QNum, qlist_pop(test_list));

     g_assert(dict1);
     g_assert(dict2);
@@ -450,7 +450,7 @@ static void qdict_array_split_test(void)

     qdict_array_split(test_dict, &test_list);

-    int1 = qobject_to_qnum(qlist_pop(test_list));
+    int1 = qobject_to(QNum, qlist_pop(test_list));

     g_assert(int1);
     g_assert(qlist_empty(test_list));
@@ -607,7 +607,7 @@ static void qdict_crumple_test_recursive(void)
     qdict_put_str(src, "vnc.acl..name", "acl0");
     qdict_put_str(src, "vnc.acl.rule..name", "acl0");

-    dst = qobject_to_qdict(qdict_crumple(src, &error_abort));
+    dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
     g_assert(dst);
     g_assert_cmpint(qdict_size(dst), ==, 1);

@@ -629,14 +629,14 @@ static void qdict_crumple_test_recursive(void)
     g_assert(rules);
     g_assert_cmpint(qlist_size(rules), ==, 2);

-    rule = qobject_to_qdict(qlist_pop(rules));
+    rule = qobject_to(QDict, qlist_pop(rules));
     g_assert(rule);
     g_assert_cmpint(qdict_size(rule), ==, 2);
     g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
     g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
     QDECREF(rule);

-    rule = qobject_to_qdict(qlist_pop(rules));
+    rule = qobject_to(QDict, qlist_pop(rules));
     g_assert(rule);
     g_assert_cmpint(qdict_size(rule), ==, 2);
     g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
diff --git a/tests/check-qjson.c b/tests/check-qjson.c
index a18ea47cb74..997f4d3d2c9 100644
--- a/tests/check-qjson.c
+++ b/tests/check-qjson.c
@@ -60,7 +60,7 @@ static void escaped_string(void)
         QString *str;

         obj = qobject_from_json(test_cases[i].encoded, &error_abort);
-        str = qobject_to_qstring(obj);
+        str = qobject_to(QString, obj);
         g_assert(str);
         g_assert_cmpstr(qstring_get_str(str), ==, test_cases[i].decoded);

@@ -92,7 +92,7 @@ static void simple_string(void)
         QString *str;

         obj = qobject_from_json(test_cases[i].encoded, &error_abort);
-        str = qobject_to_qstring(obj);
+        str = qobject_to(QString, obj);
         g_assert(str);
         g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);

@@ -123,7 +123,7 @@ static void single_quote_string(void)
         QString *str;

         obj = qobject_from_json(test_cases[i].encoded, &error_abort);
-        str = qobject_to_qstring(obj);
+        str = qobject_to(QString, obj);
         g_assert(str);
         g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);

@@ -817,7 +817,7 @@ static void utf8_string(void)

         obj = qobject_from_json(json_in, utf8_out ? &error_abort : NULL);
         if (utf8_out) {
-            str = qobject_to_qstring(obj);
+            str = qobject_to(QString, obj);
             g_assert(str);
             g_assert_cmpstr(qstring_get_str(str), ==, utf8_out);
         } else {
@@ -843,7 +843,7 @@ static void utf8_string(void)
          */
         if (0 && json_out != json_in) {
             obj = qobject_from_json(json_out, &error_abort);
-            str = qobject_to_qstring(obj);
+            str = qobject_to(QString, obj);
             g_assert(str);
             g_assert_cmpstr(qstring_get_str(str), ==, utf8_out);
         }
@@ -864,8 +864,8 @@ static void vararg_string(void)
     for (i = 0; test_cases[i].decoded; i++) {
         QString *str;

-        str = qobject_to_qstring(qobject_from_jsonf("%s",
-                                                    test_cases[i].decoded));
+        str = qobject_to(QString,
+                         qobject_from_jsonf("%s", test_cases[i].decoded));
         g_assert(str);
         g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);

@@ -893,8 +893,9 @@ static void simple_number(void)
         QNum *qnum;
         int64_t val;

-        qnum = qobject_to_qnum(qobject_from_json(test_cases[i].encoded,
-                                                 &error_abort));
+        qnum = qobject_to(QNum,
+                          qobject_from_json(test_cases[i].encoded,
+                                            &error_abort));
         g_assert(qnum);
         g_assert(qnum_get_try_int(qnum, &val));
         g_assert_cmpint(val, ==, test_cases[i].decoded);
@@ -920,7 +921,7 @@ static void large_number(void)
     uint64_t val;
     int64_t ival;

-    qnum = qobject_to_qnum(qobject_from_json(maxu64, &error_abort));
+    qnum = qobject_to(QNum, qobject_from_json(maxu64, &error_abort));
     g_assert(qnum);
     g_assert_cmpuint(qnum_get_uint(qnum), ==, 18446744073709551615U);
     g_assert(!qnum_get_try_int(qnum, &ival));
@@ -930,7 +931,7 @@ static void large_number(void)
     QDECREF(str);
     QDECREF(qnum);

-    qnum = qobject_to_qnum(qobject_from_json(gtu64, &error_abort));
+    qnum = qobject_to(QNum, qobject_from_json(gtu64, &error_abort));
     g_assert(qnum);
     g_assert_cmpfloat(qnum_get_double(qnum), ==, 18446744073709552e3);
     g_assert(!qnum_get_try_uint(qnum, &val));
@@ -941,7 +942,7 @@ static void large_number(void)
     QDECREF(str);
     QDECREF(qnum);

-    qnum = qobject_to_qnum(qobject_from_json(lti64, &error_abort));
+    qnum = qobject_to(QNum, qobject_from_json(lti64, &error_abort));
     g_assert(qnum);
     g_assert_cmpfloat(qnum_get_double(qnum), ==, -92233720368547758e2);
     g_assert(!qnum_get_try_uint(qnum, &val));
@@ -973,7 +974,7 @@ static void float_number(void)
         QNum *qnum;

         obj = qobject_from_json(test_cases[i].encoded, &error_abort);
-        qnum = qobject_to_qnum(obj);
+        qnum = qobject_to(QNum, obj);
         g_assert(qnum);
         g_assert(qnum_get_double(qnum) == test_cases[i].decoded);

@@ -997,17 +998,17 @@ static void vararg_number(void)
     double valuef = 2.323423423;
     int64_t val;

-    qnum = qobject_to_qnum(qobject_from_jsonf("%d", value));
+    qnum = qobject_to(QNum, qobject_from_jsonf("%d", value));
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, value);
     QDECREF(qnum);

-    qnum = qobject_to_qnum(qobject_from_jsonf("%lld", value_ll));
+    qnum = qobject_to(QNum, qobject_from_jsonf("%lld", value_ll));
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, value_ll);
     QDECREF(qnum);

-    qnum = qobject_to_qnum(qobject_from_jsonf("%f", valuef));
+    qnum = qobject_to(QNum, qobject_from_jsonf("%f", valuef));
     g_assert(qnum_get_double(qnum) == valuef);
     QDECREF(qnum);
 }
@@ -1020,7 +1021,7 @@ static void keyword_literal(void)
     QString *str;

     obj = qobject_from_json("true", &error_abort);
-    qbool = qobject_to_qbool(obj);
+    qbool = qobject_to(QBool, obj);
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == true);

@@ -1031,7 +1032,7 @@ static void keyword_literal(void)
     QDECREF(qbool);

     obj = qobject_from_json("false", &error_abort);
-    qbool = qobject_to_qbool(obj);
+    qbool = qobject_to(QBool, obj);
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == false);

@@ -1041,13 +1042,13 @@ static void keyword_literal(void)

     QDECREF(qbool);

-    qbool = qobject_to_qbool(qobject_from_jsonf("%i", false));
+    qbool = qobject_to(QBool, qobject_from_jsonf("%i", false));
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == false);
     QDECREF(qbool);

     /* Test that non-zero values other than 1 get collapsed to true */
-    qbool = qobject_to_qbool(qobject_from_jsonf("%i", 2));
+    qbool = qobject_to(QBool, qobject_from_jsonf("%i", 2));
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == true);
     QDECREF(qbool);
diff --git a/tests/check-qlist.c b/tests/check-qlist.c
index 259980d5239..a1c69ed6481 100644
--- a/tests/check-qlist.c
+++ b/tests/check-qlist.c
@@ -56,7 +56,7 @@ static void qobject_to_qlist_test(void)

     qlist = qlist_new();

-    g_assert(qobject_to_qlist(QOBJECT(qlist)) == qlist);
+    g_assert(qobject_to(QList, QOBJECT(qlist)) == qlist);

     QDECREF(qlist);
 }
@@ -71,7 +71,7 @@ static void iter_func(QObject *obj, void *opaque)

     g_assert(opaque == NULL);

-    qi = qobject_to_qnum(obj);
+    qi = qobject_to(QNum, obj);
     g_assert(qi != NULL);

     g_assert(qnum_get_try_int(qi, &val));
diff --git a/tests/check-qlit.c b/tests/check-qlit.c
index 836f4a3090c..96bbb06f2c7 100644
--- a/tests/check-qlit.c
+++ b/tests/check-qlit.c
@@ -59,7 +59,7 @@ static void qlit_equal_qobject_test(void)

     g_assert(!qlit_equal_qobject(&qlit_foo, qobj));

-    qdict_put(qobject_to_qdict(qobj), "bee", qlist_new());
+    qdict_put(qobject_to(QDict, qobj), "bee", qlist_new());
     g_assert(!qlit_equal_qobject(&qlit, qobj));

     qobject_decref(qobj);
@@ -71,20 +71,20 @@ static void qobject_from_qlit_test(void)
     QDict *qdict;
     QList *bee;

-    qdict = qobject_to_qdict(qobj);
+    qdict = qobject_to(QDict, qobj);
     g_assert_cmpint(qdict_get_int(qdict, "foo"), ==, 42);
     g_assert_cmpstr(qdict_get_str(qdict, "bar"), ==, "hello world");
     g_assert(qobject_type(qdict_get(qdict, "baz")) == QTYPE_QNULL);

     bee = qdict_get_qlist(qdict, "bee");
     obj = qlist_pop(bee);
-    g_assert_cmpint(qnum_get_int(qobject_to_qnum(obj)), ==, 43);
+    g_assert_cmpint(qnum_get_int(qobject_to(QNum, obj)), ==, 43);
     qobject_decref(obj);
     obj = qlist_pop(bee);
-    g_assert_cmpint(qnum_get_int(qobject_to_qnum(obj)), ==, 44);
+    g_assert_cmpint(qnum_get_int(qobject_to(QNum, obj)), ==, 44);
     qobject_decref(obj);
     obj = qlist_pop(bee);
-    g_assert(qbool_get_bool(qobject_to_qbool(obj)));
+    g_assert(qbool_get_bool(qobject_to(QBool, obj)));
     qobject_decref(obj);

     qobject_decref(qobj);
diff --git a/tests/check-qnum.c b/tests/check-qnum.c
index 2b667f7ad77..9187da734b0 100644
--- a/tests/check-qnum.c
+++ b/tests/check-qnum.c
@@ -126,11 +126,11 @@ static void qobject_to_qnum_test(void)
     QNum *qn;

     qn = qnum_from_int(0);
-    g_assert(qobject_to_qnum(QOBJECT(qn)) == qn);
+    g_assert(qobject_to(QNum, QOBJECT(qn)) == qn);
     QDECREF(qn);

     qn = qnum_from_double(0);
-    g_assert(qobject_to_qnum(QOBJECT(qn)) == qn);
+    g_assert(qobject_to(QNum, QOBJECT(qn)) == qn);
     QDECREF(qn);
 }

diff --git a/tests/check-qobject.c b/tests/check-qobject.c
index 7a3670643cb..7629b8071b3 100644
--- a/tests/check-qobject.c
+++ b/tests/check-qobject.c
@@ -275,7 +275,7 @@ static void qobject_is_equal_dict_test(void)
                   dict_different_null_key, dict_longer, dict_shorter,
                   dict_nested);

-    dict_crumpled = qobject_to_qdict(qdict_crumple(dict_1, &local_err));
+    dict_crumpled = qobject_to(QDict, qdict_crumple(dict_1, &local_err));
     g_assert(!local_err);
     check_equal(dict_crumpled, dict_nested);

diff --git a/tests/check-qstring.c b/tests/check-qstring.c
index 112ec08967f..9c4dd3f94fc 100644
--- a/tests/check-qstring.c
+++ b/tests/check-qstring.c
@@ -79,7 +79,7 @@ static void qobject_to_qstring_test(void)
     QString *qstring;

     qstring = qstring_from_str("foo");
-    g_assert(qobject_to_qstring(QOBJECT(qstring)) == qstring);
+    g_assert(qobject_to(QString, QOBJECT(qstring)) == qstring);

     QDECREF(qstring);
 }
diff --git a/tests/device-introspect-test.c b/tests/device-introspect-test.c
index b80058fe988..a01321aced7 100644
--- a/tests/device-introspect-test.c
+++ b/tests/device-introspect-test.c
@@ -52,7 +52,7 @@ static QDict *qom_type_index(QList *types)
     QListEntry *e;

     QLIST_FOREACH_ENTRY(types, e) {
-        QDict *d = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *d = qobject_to(QDict, qlist_entry_obj(e));
         const char *name = qdict_get_str(d, "name");
         QINCREF(d);
         qdict_put(index, name, d);
@@ -85,7 +85,7 @@ static QDict *type_list_find(QList *types, const char *name)
     QListEntry *e;

     QLIST_FOREACH_ENTRY(types, e) {
-        QDict *d = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *d = qobject_to(QDict, qlist_entry_obj(e));
         const char *ename = qdict_get_str(d, "name");
         if (!strcmp(ename, name)) {
             return d;
@@ -151,7 +151,7 @@ static void test_qom_list_parents(const char *parent)
     index = qom_type_index(types);

     QLIST_FOREACH_ENTRY(types, e) {
-        QDict *d = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *d = qobject_to(QDict, qlist_entry_obj(e));
         const char *name = qdict_get_str(d, "name");

         g_assert(qom_has_parent(index, name, parent));
@@ -173,7 +173,7 @@ static void test_qom_list_fields(void)
     non_abstract = qom_list_types(NULL, false);

     QLIST_FOREACH_ENTRY(all_types, e) {
-        QDict *d = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *d = qobject_to(QDict, qlist_entry_obj(e));
         const char *name = qdict_get_str(d, "name");
         bool abstract = qdict_haskey(d, "abstract") ?
                         qdict_get_bool(d, "abstract") :
@@ -216,8 +216,8 @@ static void test_device_intro_concrete(void)
     types = device_type_list(false);

     QLIST_FOREACH_ENTRY(types, entry) {
-        type = qdict_get_try_str(qobject_to_qdict(qlist_entry_obj(entry)),
-                                "name");
+        type = qdict_get_try_str(qobject_to(QDict, qlist_entry_obj(entry)),
+                                 "name");
         g_assert(type);
         test_one_device(type);
     }
@@ -238,7 +238,7 @@ static void test_abstract_interfaces(void)
     index = qom_type_index(all_types);

     QLIST_FOREACH_ENTRY(all_types, e) {
-        QDict *d = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *d = qobject_to(QDict, qlist_entry_obj(e));
         const char *name = qdict_get_str(d, "name");

         /*
diff --git a/tests/numa-test.c b/tests/numa-test.c
index 68aca9cb389..0f861d8176d 100644
--- a/tests/numa-test.c
+++ b/tests/numa-test.c
@@ -98,7 +98,7 @@ static void test_query_cpus(const void *data)
         QDict *cpu, *props;
         int64_t cpu_idx, node;

-        cpu = qobject_to_qdict(e);
+        cpu = qobject_to(QDict, e);
         g_assert(qdict_haskey(cpu, "CPU"));
         g_assert(qdict_haskey(cpu, "props"));

@@ -140,7 +140,7 @@ static void pc_numa_cpu(const void *data)
         QDict *cpu, *props;
         int64_t socket, core, thread, node;

-        cpu = qobject_to_qdict(e);
+        cpu = qobject_to(QDict, e);
         g_assert(qdict_haskey(cpu, "props"));
         props = qdict_get_qdict(cpu, "props");

@@ -193,7 +193,7 @@ static void spapr_numa_cpu(const void *data)
         QDict *cpu, *props;
         int64_t core, node;

-        cpu = qobject_to_qdict(e);
+        cpu = qobject_to(QDict, e);
         g_assert(qdict_haskey(cpu, "props"));
         props = qdict_get_qdict(cpu, "props");

@@ -236,7 +236,7 @@ static void aarch64_numa_cpu(const void *data)
         QDict *cpu, *props;
         int64_t thread, node;

-        cpu = qobject_to_qdict(e);
+        cpu = qobject_to(QDict, e);
         g_assert(qdict_haskey(cpu, "props"));
         props = qdict_get_qdict(cpu, "props");

diff --git a/tests/qom-test.c b/tests/qom-test.c
index 9dab7ac61ec..a34ff6ba53f 100644
--- a/tests/qom-test.c
+++ b/tests/qom-test.c
@@ -62,9 +62,9 @@ static void test_properties(const char *path, bool recurse)
     }

     g_assert(qdict_haskey(response, "return"));
-    list = qobject_to_qlist(qdict_get(response, "return"));
+    list = qobject_to(QList, qdict_get(response, "return"));
     QLIST_FOREACH_ENTRY(list, entry) {
-        tuple = qobject_to_qdict(qlist_entry_obj(entry));
+        tuple = qobject_to(QDict, qlist_entry_obj(entry));
         bool is_child = strstart(qdict_get_str(tuple, "type"), "child<", NULL);
         bool is_link = strstart(qdict_get_str(tuple, "type"), "link<", NULL);

diff --git a/tests/test-char.c b/tests/test-char.c
index b3a77af0855..c4178f95e7f 100644
--- a/tests/test-char.c
+++ b/tests/test-char.c
@@ -321,7 +321,7 @@ static void char_socket_test(void)
     g_assert(!object_property_get_bool(OBJECT(chr), "connected", &error_abort));

     addr = object_property_get_qobject(OBJECT(chr), "addr", &error_abort);
-    qdict = qobject_to_qdict(addr);
+    qdict = qobject_to(QDict, addr);
     port = qdict_get_str(qdict, "port");
     tmp = g_strdup_printf("tcp:127.0.0.1:%s", port);
     QDECREF(qdict);
diff --git a/tests/test-keyval.c b/tests/test-keyval.c
index 94eb4df28d4..029f05202a8 100644
--- a/tests/test-keyval.c
+++ b/tests/test-keyval.c
@@ -195,7 +195,7 @@ static void check_list012(QList *qlist)

     g_assert(qlist);
     for (i = 0; i < ARRAY_SIZE(expected); i++) {
-        qstr = qobject_to_qstring(qlist_pop(qlist));
+        qstr = qobject_to(QString, qlist_pop(qlist));
         g_assert(qstr);
         g_assert_cmpstr(qstring_get_str(qstr), ==, expected[i]);
         QDECREF(qstr);
@@ -654,12 +654,12 @@ static void test_keyval_visit_any(void)
     QDECREF(qdict);
     visit_start_struct(v, NULL, NULL, 0, &error_abort);
     visit_type_any(v, "a", &any, &error_abort);
-    qlist = qobject_to_qlist(any);
+    qlist = qobject_to(QList, any);
     g_assert(qlist);
-    qstr = qobject_to_qstring(qlist_pop(qlist));
+    qstr = qobject_to(QString, qlist_pop(qlist));
     g_assert_cmpstr(qstring_get_str(qstr), ==, "null");
     QDECREF(qstr);
-    qstr = qobject_to_qstring(qlist_pop(qlist));
+    qstr = qobject_to(QString, qlist_pop(qlist));
     g_assert_cmpstr(qstring_get_str(qstr), ==, "1");
     g_assert(qlist_empty(qlist));
     QDECREF(qstr);
diff --git a/tests/test-qga.c b/tests/test-qga.c
index 5c5b661f8a3..e6ab788f31a 100644
--- a/tests/test-qga.c
+++ b/tests/test-qga.c
@@ -297,8 +297,8 @@ static void test_qga_get_vcpus(gconstpointer fix)
     /* check there is at least a cpu */
     list = qdict_get_qlist(ret, "return");
     entry = qlist_first(list);
-    g_assert(qdict_haskey(qobject_to_qdict(entry->value), "online"));
-    g_assert(qdict_haskey(qobject_to_qdict(entry->value), "logical-id"));
+    g_assert(qdict_haskey(qobject_to(QDict, entry->value), "online"));
+    g_assert(qdict_haskey(qobject_to(QDict, entry->value), "logical-id"));

     QDECREF(ret);
 }
@@ -318,10 +318,10 @@ static void test_qga_get_fsinfo(gconstpointer fix)
     list = qdict_get_qlist(ret, "return");
     entry = qlist_first(list);
     if (entry) {
-        g_assert(qdict_haskey(qobject_to_qdict(entry->value), "name"));
-        g_assert(qdict_haskey(qobject_to_qdict(entry->value), "mountpoint"));
-        g_assert(qdict_haskey(qobject_to_qdict(entry->value), "type"));
-        g_assert(qdict_haskey(qobject_to_qdict(entry->value), "disk"));
+        g_assert(qdict_haskey(qobject_to(QDict, entry->value), "name"));
+        g_assert(qdict_haskey(qobject_to(QDict, entry->value), "mountpoint"));
+        g_assert(qdict_haskey(qobject_to(QDict, entry->value), "type"));
+        g_assert(qdict_haskey(qobject_to(QDict, entry->value), "disk"));
     }

     QDECREF(ret);
@@ -363,8 +363,9 @@ static void test_qga_get_memory_blocks(gconstpointer fix)
         entry = qlist_first(list);
         /* newer versions of qga may return empty list without error */
         if (entry) {
-            g_assert(qdict_haskey(qobject_to_qdict(entry->value), "phys-index"));
-            g_assert(qdict_haskey(qobject_to_qdict(entry->value), "online"));
+            g_assert(qdict_haskey(qobject_to(QDict, entry->value),
+                                  "phys-index"));
+            g_assert(qdict_haskey(qobject_to(QDict, entry->value), "online"));
         }
     }

@@ -385,7 +386,7 @@ static void test_qga_network_get_interfaces(gconstpointer fix)
     /* check there is at least an interface */
     list = qdict_get_qlist(ret, "return");
     entry = qlist_first(list);
-    g_assert(qdict_haskey(qobject_to_qdict(entry->value), "name"));
+    g_assert(qdict_haskey(qobject_to(QDict, entry->value), "name"));

     QDECREF(ret);
 }
diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index 5b1cee69122..93fbbb1b733 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -100,7 +100,7 @@ static void test_dispatch_cmd(void)

     resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
     assert(resp != NULL);
-    assert(!qdict_haskey(qobject_to_qdict(resp), "error"));
+    assert(!qdict_haskey(qobject_to(QDict, resp), "error"));

     qobject_decref(resp);
     QDECREF(req);
@@ -117,7 +117,7 @@ static void test_dispatch_cmd_failure(void)

     resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
     assert(resp != NULL);
-    assert(qdict_haskey(qobject_to_qdict(resp), "error"));
+    assert(qdict_haskey(qobject_to(QDict, resp), "error"));

     qobject_decref(resp);
     QDECREF(req);
@@ -131,7 +131,7 @@ static void test_dispatch_cmd_failure(void)

     resp = qmp_dispatch(&qmp_commands, QOBJECT(req));
     assert(resp != NULL);
-    assert(qdict_haskey(qobject_to_qdict(resp), "error"));
+    assert(qdict_haskey(qobject_to(QDict, resp), "error"));

     qobject_decref(resp);
     QDECREF(req);
@@ -145,7 +145,7 @@ static QObject *test_qmp_dispatch(QDict *req)

     resp_obj = qmp_dispatch(&qmp_commands, QOBJECT(req));
     assert(resp_obj);
-    resp = qobject_to_qdict(resp_obj);
+    resp = qobject_to(QDict, resp_obj);
     assert(resp && !qdict_haskey(resp, "error"));
     ret = qdict_get(resp, "return");
     assert(ret);
@@ -176,7 +176,7 @@ static void test_dispatch_cmd_io(void)
     qdict_put(req, "arguments", args);
     qdict_put_str(req, "execute", "user_def_cmd2");

-    ret = qobject_to_qdict(test_qmp_dispatch(req));
+    ret = qobject_to(QDict, test_qmp_dispatch(req));

     assert(!strcmp(qdict_get_str(ret, "string0"), "blah1"));
     ret_dict = qdict_get_qdict(ret, "dict1");
@@ -197,7 +197,7 @@ static void test_dispatch_cmd_io(void)
     qdict_put(req, "arguments", args3);
     qdict_put_str(req, "execute", "guest-get-time");

-    ret3 = qobject_to_qnum(test_qmp_dispatch(req));
+    ret3 = qobject_to(QNum, test_qmp_dispatch(req));
     g_assert(qnum_get_try_int(ret3, &val));
     g_assert_cmpint(val, ==, 66);
     QDECREF(ret3);
diff --git a/tests/test-qmp-event.c b/tests/test-qmp-event.c
index 31f35b3e66a..bb1036615fd 100644
--- a/tests/test-qmp-event.c
+++ b/tests/test-qmp-event.c
@@ -60,22 +60,22 @@ void qdict_cmp_do_simple(const char *key, QObject *obj1, void *opaque)

     switch (qobject_type(obj1)) {
     case QTYPE_QBOOL:
-        d->result = (qbool_get_bool(qobject_to_qbool(obj1)) ==
-                     qbool_get_bool(qobject_to_qbool(obj2)));
+        d->result = (qbool_get_bool(qobject_to(QBool, obj1)) ==
+                     qbool_get_bool(qobject_to(QBool, obj2)));
         return;
     case QTYPE_QNUM:
-        g_assert(qnum_get_try_int(qobject_to_qnum(obj1), &val1));
-        g_assert(qnum_get_try_int(qobject_to_qnum(obj2), &val2));
+        g_assert(qnum_get_try_int(qobject_to(QNum, obj1), &val1));
+        g_assert(qnum_get_try_int(qobject_to(QNum, obj2), &val2));
         d->result = val1 == val2;
         return;
     case QTYPE_QSTRING:
-        d->result = g_strcmp0(qstring_get_str(qobject_to_qstring(obj1)),
-                              qstring_get_str(qobject_to_qstring(obj2))) == 0;
+        d->result = g_strcmp0(qstring_get_str(qobject_to(QString, obj1)),
+                              qstring_get_str(qobject_to(QString, obj2))) == 0;
         return;
     case QTYPE_QDICT:
-        d_new.expect = qobject_to_qdict(obj2);
+        d_new.expect = qobject_to(QDict, obj2);
         d_new.result = true;
-        qdict_iter(qobject_to_qdict(obj1), qdict_cmp_do_simple, &d_new);
+        qdict_iter(qobject_to(QDict, obj1), qdict_cmp_do_simple, &d_new);
         d->result = d_new.result;
         return;
     default:
diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c
index 55db6bdef13..6dc59c62116 100644
--- a/tests/test-qobject-input-visitor.c
+++ b/tests/test-qobject-input-visitor.c
@@ -479,7 +479,7 @@ static void test_visitor_in_any(TestInputVisitorData *data,

     v = visitor_input_test_init(data, "-42");
     visit_type_any(v, NULL, &res, &error_abort);
-    qnum = qobject_to_qnum(res);
+    qnum = qobject_to(QNum, res);
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, -42);
@@ -487,22 +487,22 @@ static void test_visitor_in_any(TestInputVisitorData *data,

     v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
     visit_type_any(v, NULL, &res, &error_abort);
-    qdict = qobject_to_qdict(res);
+    qdict = qobject_to(QDict, res);
     g_assert(qdict && qdict_size(qdict) == 3);
     qobj = qdict_get(qdict, "integer");
     g_assert(qobj);
-    qnum = qobject_to_qnum(qobj);
+    qnum = qobject_to(QNum, qobj);
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, -42);
     qobj = qdict_get(qdict, "boolean");
     g_assert(qobj);
-    qbool = qobject_to_qbool(qobj);
+    qbool = qobject_to(QBool, qobj);
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == true);
     qobj = qdict_get(qdict, "string");
     g_assert(qobj);
-    qstring = qobject_to_qstring(qobj);
+    qstring = qobject_to(QString, qobj);
     g_assert(qstring);
     g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
     qobject_decref(res);
diff --git a/tests/test-qobject-output-visitor.c b/tests/test-qobject-output-visitor.c
index 11e8c5aa403..ecf21c0f318 100644
--- a/tests/test-qobject-output-visitor.c
+++ b/tests/test-qobject-output-visitor.c
@@ -66,7 +66,7 @@ static void test_visitor_out_int(TestOutputVisitorData *data,

     visit_type_int(data->ov, NULL, &value, &error_abort);

-    qnum = qobject_to_qnum(visitor_get(data));
+    qnum = qobject_to(QNum, visitor_get(data));
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, value);
@@ -80,7 +80,7 @@ static void test_visitor_out_bool(TestOutputVisitorData *data,

     visit_type_bool(data->ov, NULL, &value, &error_abort);

-    qbool = qobject_to_qbool(visitor_get(data));
+    qbool = qobject_to(QBool, visitor_get(data));
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == value);
 }
@@ -93,7 +93,7 @@ static void test_visitor_out_number(TestOutputVisitorData *data,

     visit_type_number(data->ov, NULL, &value, &error_abort);

-    qnum = qobject_to_qnum(visitor_get(data));
+    qnum = qobject_to(QNum, visitor_get(data));
     g_assert(qnum);
     g_assert(qnum_get_double(qnum) == value);
 }
@@ -106,7 +106,7 @@ static void test_visitor_out_string(TestOutputVisitorData *data,

     visit_type_str(data->ov, NULL, &string, &error_abort);

-    qstr = qobject_to_qstring(visitor_get(data));
+    qstr = qobject_to(QString, visitor_get(data));
     g_assert(qstr);
     g_assert_cmpstr(qstring_get_str(qstr), ==, string);
 }
@@ -120,7 +120,7 @@ static void test_visitor_out_no_string(TestOutputVisitorData *data,
     /* A null string should return "" */
     visit_type_str(data->ov, NULL, &string, &error_abort);

-    qstr = qobject_to_qstring(visitor_get(data));
+    qstr = qobject_to(QString, visitor_get(data));
     g_assert(qstr);
     g_assert_cmpstr(qstring_get_str(qstr), ==, "");
 }
@@ -134,7 +134,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,
     for (i = 0; i < ENUM_ONE__MAX; i++) {
         visit_type_EnumOne(data->ov, "unused", &i, &error_abort);

-        qstr = qobject_to_qstring(visitor_get(data));
+        qstr = qobject_to(QString, visitor_get(data));
         g_assert(qstr);
         g_assert_cmpstr(qstring_get_str(qstr), ==, EnumOne_str(i));
         visitor_reset(data);
@@ -167,7 +167,7 @@ static void test_visitor_out_struct(TestOutputVisitorData *data,

     visit_type_TestStruct(data->ov, NULL, &p, &error_abort);

-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
     g_assert_cmpint(qdict_size(qdict), ==, 3);
     g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 42);
@@ -206,7 +206,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,

     visit_type_UserDefTwo(data->ov, "unused", &ud2, &error_abort);

-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
     g_assert_cmpint(qdict_size(qdict), ==, 2);
     g_assert_cmpstr(qdict_get_str(qdict, "string0"), ==, strings[0]);
@@ -280,7 +280,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,

     visit_type_TestStructList(data->ov, NULL, &head, &error_abort);

-    qlist = qobject_to_qlist(visitor_get(data));
+    qlist = qobject_to(QList, visitor_get(data));
     g_assert(qlist);
     g_assert(!qlist_empty(qlist));

@@ -289,7 +289,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
     QLIST_FOREACH_ENTRY(qlist, entry) {
         QDict *qdict;

-        qdict = qobject_to_qdict(entry->value);
+        qdict = qobject_to(QDict, entry->value);
         g_assert(qdict);
         g_assert_cmpint(qdict_size(qdict), ==, 3);
         g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, value_int + i);
@@ -342,7 +342,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,

     qobj = QOBJECT(qnum_from_int(-42));
     visit_type_any(data->ov, NULL, &qobj, &error_abort);
-    qnum = qobject_to_qnum(visitor_get(data));
+    qnum = qobject_to(QNum, visitor_get(data));
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, -42);
@@ -356,16 +356,16 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
     qobj = QOBJECT(qdict);
     visit_type_any(data->ov, NULL, &qobj, &error_abort);
     qobject_decref(qobj);
-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
-    qnum = qobject_to_qnum(qdict_get(qdict, "integer"));
+    qnum = qobject_to(QNum, qdict_get(qdict, "integer"));
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, -42);
-    qbool = qobject_to_qbool(qdict_get(qdict, "boolean"));
+    qbool = qobject_to(QBool, qdict_get(qdict, "boolean"));
     g_assert(qbool);
     g_assert(qbool_get_bool(qbool) == true);
-    qstring = qobject_to_qstring(qdict_get(qdict, "string"));
+    qstring = qobject_to(QString, qdict_get(qdict, "string"));
     g_assert(qstring);
     g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
 }
@@ -382,7 +382,7 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
     tmp->u.value1.boolean = true;

     visit_type_UserDefFlatUnion(data->ov, NULL, &tmp, &error_abort);
-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
     g_assert_cmpstr(qdict_get_str(qdict, "enum1"), ==, "value1");
     g_assert_cmpstr(qdict_get_str(qdict, "string"), ==, "str");
@@ -406,7 +406,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     tmp->u.i = 42;

     visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
-    qnum = qobject_to_qnum(visitor_get(data));
+    qnum = qobject_to(QNum, visitor_get(data));
     g_assert(qnum);
     g_assert(qnum_get_try_int(qnum, &val));
     g_assert_cmpint(val, ==, 42);
@@ -419,7 +419,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     tmp->u.e = ENUM_ONE_VALUE1;

     visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
-    qstr = qobject_to_qstring(visitor_get(data));
+    qstr = qobject_to(QString, visitor_get(data));
     g_assert(qstr);
     g_assert_cmpstr(qstring_get_str(qstr), ==, "value1");

@@ -444,7 +444,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
     tmp->u.udfu.u.value1.boolean = true;

     visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
     g_assert_cmpint(qdict_size(qdict), ==, 4);
     g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 1);
@@ -466,7 +466,7 @@ static void test_visitor_out_null(TestOutputVisitorData *data,
     visit_type_null(data->ov, "a", &null, &error_abort);
     visit_check_struct(data->ov, &error_abort);
     visit_end_struct(data->ov, NULL);
-    qdict = qobject_to_qdict(visitor_get(data));
+    qdict = qobject_to(QDict, visitor_get(data));
     g_assert(qdict);
     g_assert_cmpint(qdict_size(qdict), ==, 1);
     nil = qdict_get(qdict, "a");
@@ -610,10 +610,10 @@ static void check_native_list(QObject *qobj,
     QList *qlist;
     int i;

-    qdict = qobject_to_qdict(qobj);
+    qdict = qobject_to(QDict, qobj);
     g_assert(qdict);
     g_assert(qdict_haskey(qdict, "data"));
-    qlist = qlist_copy(qobject_to_qlist(qdict_get(qdict, "data")));
+    qlist = qlist_copy(qobject_to(QList, qdict_get(qdict, "data")));

     switch (kind) {
     case USER_DEF_NATIVE_LIST_UNION_KIND_U8:
@@ -627,7 +627,7 @@ static void check_native_list(QObject *qobj,

             tmp = qlist_peek(qlist);
             g_assert(tmp);
-            qvalue = qobject_to_qnum(tmp);
+            qvalue = qobject_to(QNum, tmp);
             g_assert(qnum_get_try_uint(qvalue, &val));
             g_assert_cmpint(val, ==, i);
             qobject_decref(qlist_pop(qlist));
@@ -651,7 +651,7 @@ static void check_native_list(QObject *qobj,

             tmp = qlist_peek(qlist);
             g_assert(tmp);
-            qvalue = qobject_to_qnum(tmp);
+            qvalue = qobject_to(QNum, tmp);
             g_assert(qnum_get_try_int(qvalue, &val));
             g_assert_cmpint(val, ==, i);
             qobject_decref(qlist_pop(qlist));
@@ -663,7 +663,7 @@ static void check_native_list(QObject *qobj,
             QBool *qvalue;
             tmp = qlist_peek(qlist);
             g_assert(tmp);
-            qvalue = qobject_to_qbool(tmp);
+            qvalue = qobject_to(QBool, tmp);
             g_assert_cmpint(qbool_get_bool(qvalue), ==, i % 3 == 0);
             qobject_decref(qlist_pop(qlist));
         }
@@ -675,7 +675,7 @@ static void check_native_list(QObject *qobj,
             gchar str[8];
             tmp = qlist_peek(qlist);
             g_assert(tmp);
-            qvalue = qobject_to_qstring(tmp);
+            qvalue = qobject_to(QString, tmp);
             sprintf(str, "%d", i);
             g_assert_cmpstr(qstring_get_str(qvalue), ==, str);
             qobject_decref(qlist_pop(qlist));
@@ -690,7 +690,7 @@ static void check_native_list(QObject *qobj,

             tmp = qlist_peek(qlist);
             g_assert(tmp);
-            qvalue = qobject_to_qnum(tmp);
+            qvalue = qobject_to(QNum, tmp);
             g_string_printf(double_expected, "%.6f", (double)i / 3);
             g_string_printf(double_actual, "%.6f", qnum_get_double(qvalue));
             g_assert_cmpstr(double_actual->str, ==, double_expected->str);
diff --git a/tests/test-x86-cpuid-compat.c b/tests/test-x86-cpuid-compat.c
index 495dd1e7ef3..02e41843fc9 100644
--- a/tests/test-x86-cpuid-compat.c
+++ b/tests/test-x86-cpuid-compat.c
@@ -17,7 +17,7 @@ static char *get_cpu0_qom_path(void)
     g_assert(qdict_haskey(resp, "return"));
     ret = qdict_get_qlist(resp, "return");

-    cpu0 = qobject_to_qdict(qlist_peek(ret));
+    cpu0 = qobject_to(QDict, qlist_peek(ret));
     path = g_strdup(qdict_get_str(cpu0, "qom_path"));
     QDECREF(resp);
     return path;
@@ -38,7 +38,7 @@ static QObject *qom_get(const char *path, const char *prop)
 #ifdef CONFIG_HAS_GLIB_SUBPROCESS_TESTS
 static bool qom_get_bool(const char *path, const char *prop)
 {
-    QBool *value = qobject_to_qbool(qom_get(path, prop));
+    QBool *value = qobject_to(QBool, qom_get(path, prop));
     bool b = qbool_get_bool(value);

     QDECREF(value);
@@ -61,7 +61,7 @@ static void test_cpuid_prop(const void *data)

     qtest_start(args->cmdline);
     path = get_cpu0_qom_path();
-    value = qobject_to_qnum(qom_get(path, args->property));
+    value = qobject_to(QNum, qom_get(path, args->property));
     g_assert(qnum_get_try_int(value, &val));
     g_assert_cmpint(val, ==, args->expected_value);
     qtest_end();
@@ -105,7 +105,7 @@ static uint32_t get_feature_word(QList *features, uint32_t eax, uint32_t ecx,
     const QListEntry *e;

     for (e = qlist_first(features); e; e = qlist_next(e)) {
-        QDict *w = qobject_to_qdict(qlist_entry_obj(e));
+        QDict *w = qobject_to(QDict, qlist_entry_obj(e));
         const char *rreg = qdict_get_str(w, "cpuid-register");
         uint32_t reax = qdict_get_int(w, "cpuid-input-eax");
         bool has_ecx = qdict_haskey(w, "cpuid-input-ecx");
@@ -116,8 +116,9 @@ static uint32_t get_feature_word(QList *features, uint32_t eax, uint32_t ecx,
             recx = qdict_get_int(w, "cpuid-input-ecx");
         }
         if (eax == reax && (!has_ecx || ecx == recx) && !strcmp(rreg, reg)) {
-            g_assert(qnum_get_try_int(qobject_to_qnum(qdict_get(w, "features")),
-                                  &val));
+            g_assert(qnum_get_try_int(qobject_to(QNum,
+                                                 qdict_get(w, "features")),
+                                      &val));
             return val;
         }
     }
@@ -133,8 +134,8 @@ static void test_feature_flag(const void *data)

     qtest_start(args->cmdline);
     path = get_cpu0_qom_path();
-    present = qobject_to_qlist(qom_get(path, "feature-words"));
-    filtered = qobject_to_qlist(qom_get(path, "filtered-features"));
+    present = qobject_to(QList, qom_get(path, "feature-words"));
+    filtered = qobject_to(QList, qom_get(path, "filtered-features"));
     value = get_feature_word(present, args->in_eax, args->in_ecx, args->reg);
     value |= get_feature_word(filtered, args->in_eax, args->in_ecx, args->reg);
     qtest_end();
diff --git a/util/keyval.c b/util/keyval.c
index 212ae90d003..1c7351a233e 100644
--- a/util/keyval.c
+++ b/util/keyval.c
@@ -221,7 +221,7 @@ static const char *keyval_parse_one(QDict *qdict, const char *params,
             if (!next) {
                 return NULL;
             }
-            cur = qobject_to_qdict(next);
+            cur = qobject_to(QDict, next);
             assert(cur);
         }

@@ -314,7 +314,7 @@ static QObject *keyval_listify(QDict *cur, GSList *key_of_cur, Error **errp)
             has_member = true;
         }

-        qdict = qobject_to_qdict(ent->value);
+        qdict = qobject_to(QDict, ent->value);
         if (!qdict) {
             continue;
         }
diff --git a/util/qemu-config.c b/util/qemu-config.c
index c651c4826e7..20f7d1429d8 100644
--- a/util/qemu-config.c
+++ b/util/qemu-config.c
@@ -528,7 +528,7 @@ static void config_parse_qdict_section(QDict *options, QemuOptsList *opts,
         }

         QLIST_FOREACH_ENTRY(list, list_entry) {
-            QDict *section = qobject_to_qdict(qlist_entry_obj(list_entry));
+            QDict *section = qobject_to(QDict, qlist_entry_obj(list_entry));
             char *opt_name;

             if (!section) {
diff --git a/util/qemu-option.c b/util/qemu-option.c
index 2b412eff5e5..d0756fda58d 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -919,15 +919,15 @@ static void qemu_opts_from_qdict_1(const char *key, QObject *obj, void *opaque)

     switch (qobject_type(obj)) {
     case QTYPE_QSTRING:
-        value = qstring_get_str(qobject_to_qstring(obj));
+        value = qstring_get_str(qobject_to(QString, obj));
         break;
     case QTYPE_QNUM:
-        tmp = qnum_to_string(qobject_to_qnum(obj));
+        tmp = qnum_to_string(qobject_to(QNum, obj));
         value = tmp;
         break;
     case QTYPE_QBOOL:
         pstrcpy(buf, sizeof(buf),
-                qbool_get_bool(qobject_to_qbool(obj)) ? "on" : "off");
+                qbool_get_bool(qobject_to(QBool, obj)) ? "on" : "off");
         value = buf;
         break;
     default:
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 08/36] qapi: Remove qobject_to_X() functions
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (6 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 07/36] qapi: Replace qobject_to_X(o) by qobject_to(X, o) Eric Blake
@ 2018-03-12 18:35 ` Eric Blake
  2018-04-12 17:51   ` [Qemu-devel] [PULL, " Yuval Shaia
  2018-03-12 18:36 ` [Qemu-devel] [PULL 09/36] qapi: Make more of qobject_to() Eric Blake
                   ` (28 subsequent siblings)
  36 siblings, 1 reply; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:35 UTC (permalink / raw)
  To: qemu-devel; +Cc: Max Reitz, Markus Armbruster

From: Max Reitz <mreitz@redhat.com>

They are no longer needed now.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-5-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qbool.h   |  1 -
 include/qapi/qmp/qdict.h   |  1 -
 include/qapi/qmp/qlist.h   |  1 -
 include/qapi/qmp/qnum.h    |  1 -
 include/qapi/qmp/qstring.h |  1 -
 qobject/qbool.c            | 11 -----------
 qobject/qdict.c            | 11 -----------
 qobject/qlist.c            | 11 -----------
 qobject/qnum.c             | 11 -----------
 qobject/qstring.c          | 11 -----------
 10 files changed, 60 deletions(-)

diff --git a/include/qapi/qmp/qbool.h b/include/qapi/qmp/qbool.h
index 629c508d34f..b9a44a1bfe0 100644
--- a/include/qapi/qmp/qbool.h
+++ b/include/qapi/qmp/qbool.h
@@ -23,7 +23,6 @@ struct QBool {

 QBool *qbool_from_bool(bool value);
 bool qbool_get_bool(const QBool *qb);
-QBool *qobject_to_qbool(const QObject *obj);
 bool qbool_is_equal(const QObject *x, const QObject *y);
 void qbool_destroy_obj(QObject *obj);

diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 7c6d8445495..2cc3e906f7b 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -39,7 +39,6 @@ void qdict_put_obj(QDict *qdict, const char *key, QObject *value);
 void qdict_del(QDict *qdict, const char *key);
 int qdict_haskey(const QDict *qdict, const char *key);
 QObject *qdict_get(const QDict *qdict, const char *key);
-QDict *qobject_to_qdict(const QObject *obj);
 bool qdict_is_equal(const QObject *x, const QObject *y);
 void qdict_iter(const QDict *qdict,
                 void (*iter)(const char *key, QObject *obj, void *opaque),
diff --git a/include/qapi/qmp/qlist.h b/include/qapi/qmp/qlist.h
index 5fd976a3981..5c673acb060 100644
--- a/include/qapi/qmp/qlist.h
+++ b/include/qapi/qmp/qlist.h
@@ -53,7 +53,6 @@ QObject *qlist_pop(QList *qlist);
 QObject *qlist_peek(QList *qlist);
 int qlist_empty(const QList *qlist);
 size_t qlist_size(const QList *qlist);
-QList *qobject_to_qlist(const QObject *obj);
 bool qlist_is_equal(const QObject *x, const QObject *y);
 void qlist_destroy_obj(QObject *obj);

diff --git a/include/qapi/qmp/qnum.h b/include/qapi/qmp/qnum.h
index 15e3971c7f1..3e47475b2cb 100644
--- a/include/qapi/qmp/qnum.h
+++ b/include/qapi/qmp/qnum.h
@@ -68,7 +68,6 @@ double qnum_get_double(QNum *qn);

 char *qnum_to_string(QNum *qn);

-QNum *qobject_to_qnum(const QObject *obj);
 bool qnum_is_equal(const QObject *x, const QObject *y);
 void qnum_destroy_obj(QObject *obj);

diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index 98070ef3d6a..b72843fc1b0 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -30,7 +30,6 @@ const char *qstring_get_str(const QString *qstring);
 void qstring_append_int(QString *qstring, int64_t value);
 void qstring_append(QString *qstring, const char *str);
 void qstring_append_chr(QString *qstring, int c);
-QString *qobject_to_qstring(const QObject *obj);
 bool qstring_is_equal(const QObject *x, const QObject *y);
 void qstring_destroy_obj(QObject *obj);

diff --git a/qobject/qbool.c b/qobject/qbool.c
index 5be6277cca8..b58249925c2 100644
--- a/qobject/qbool.c
+++ b/qobject/qbool.c
@@ -39,17 +39,6 @@ bool qbool_get_bool(const QBool *qb)
     return qb->value;
 }

-/**
- * qobject_to_qbool(): Convert a QObject into a QBool
- */
-QBool *qobject_to_qbool(const QObject *obj)
-{
-    if (!obj || qobject_type(obj) != QTYPE_QBOOL) {
-        return NULL;
-    }
-    return container_of(obj, QBool, base);
-}
-
 /**
  * qbool_is_equal(): Test whether the two QBools are equal
  */
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 1e588123d00..45c8b53361f 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -37,17 +37,6 @@ QDict *qdict_new(void)
     return qdict;
 }

-/**
- * qobject_to_qdict(): Convert a QObject into a QDict
- */
-QDict *qobject_to_qdict(const QObject *obj)
-{
-    if (!obj || qobject_type(obj) != QTYPE_QDICT) {
-        return NULL;
-    }
-    return container_of(obj, QDict, base);
-}
-
 /**
  * tdb_hash(): based on the hash agorithm from gdbm, via tdb
  * (from module-init-tools)
diff --git a/qobject/qlist.c b/qobject/qlist.c
index 77f19ffda78..954fe983751 100644
--- a/qobject/qlist.c
+++ b/qobject/qlist.c
@@ -151,17 +151,6 @@ size_t qlist_size(const QList *qlist)
     return count;
 }

-/**
- * qobject_to_qlist(): Convert a QObject into a QList
- */
-QList *qobject_to_qlist(const QObject *obj)
-{
-    if (!obj || qobject_type(obj) != QTYPE_QLIST) {
-        return NULL;
-    }
-    return container_of(obj, QList, base);
-}
-
 /**
  * qlist_is_equal(): Test whether the two QLists are equal
  *
diff --git a/qobject/qnum.c b/qobject/qnum.c
index ea091cfaa4b..1501c828323 100644
--- a/qobject/qnum.c
+++ b/qobject/qnum.c
@@ -199,17 +199,6 @@ char *qnum_to_string(QNum *qn)
     return NULL;
 }

-/**
- * qobject_to_qnum(): Convert a QObject into a QNum
- */
-QNum *qobject_to_qnum(const QObject *obj)
-{
-    if (!obj || qobject_type(obj) != QTYPE_QNUM) {
-        return NULL;
-    }
-    return container_of(obj, QNum, base);
-}
-
 /**
  * qnum_is_equal(): Test whether the two QNums are equal
  *
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 2b2153206df..8bae529ef79 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -105,17 +105,6 @@ void qstring_append_chr(QString *qstring, int c)
     qstring->string[qstring->length] = 0;
 }

-/**
- * qobject_to_qstring(): Convert a QObject to a QString
- */
-QString *qobject_to_qstring(const QObject *obj)
-{
-    if (!obj || qobject_type(obj) != QTYPE_QSTRING) {
-        return NULL;
-    }
-    return container_of(obj, QString, base);
-}
-
 /**
  * qstring_get_str(): Return a pointer to the stored string
  *
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 09/36] qapi: Make more of qobject_to()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (7 preceding siblings ...)
  2018-03-12 18:35 ` [Qemu-devel] [PULL 08/36] qapi: Remove qobject_to_X() functions Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 10/36] block: Handle null backing link Eric Blake
                   ` (27 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Max Reitz, Markus Armbruster, Michael Roth

From: Max Reitz <mreitz@redhat.com>

This patch reworks some places which use either qobject_type() checks
plus qobject_to(), where the latter alone is sufficient, or NULL checks
plus qobject_type() checks where we can simply do a qobject_to() != NULL
check.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-6-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase to qobject_to() parameter ordering]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/qobject-input-visitor.c |  4 ++--
 qobject/json-parser.c        | 13 +++++++------
 qobject/qdict.c              | 20 +++++++++++---------
 3 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 0b5293385e5..a7569d5dcef 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -339,7 +339,7 @@ static GenericList *qobject_input_next_list(Visitor *v, GenericList *tail,
     QObjectInputVisitor *qiv = to_qiv(v);
     StackObject *tos = QSLIST_FIRST(&qiv->stack);

-    assert(tos && tos->obj && qobject_type(tos->obj) == QTYPE_QLIST);
+    assert(tos && qobject_to(QList, tos->obj));

     if (!tos->entry) {
         return NULL;
@@ -353,7 +353,7 @@ static void qobject_input_check_list(Visitor *v, Error **errp)
     QObjectInputVisitor *qiv = to_qiv(v);
     StackObject *tos = QSLIST_FIRST(&qiv->stack);

-    assert(tos && tos->obj && qobject_type(tos->obj) == QTYPE_QLIST);
+    assert(tos && qobject_to(QList, tos->obj));

     if (tos->entry) {
         error_setg(errp, "Only %u list elements expected in %s",
diff --git a/qobject/json-parser.c b/qobject/json-parser.c
index 055c7f0272a..769b960c9fe 100644
--- a/qobject/json-parser.c
+++ b/qobject/json-parser.c
@@ -276,7 +276,8 @@ static void parser_context_free(JSONParserContext *ctxt)
  */
 static int parse_pair(JSONParserContext *ctxt, QDict *dict, va_list *ap)
 {
-    QObject *key = NULL, *value;
+    QObject *value;
+    QString *key = NULL;
     JSONToken *peek, *token;

     peek = parser_context_peek_token(ctxt);
@@ -285,8 +286,8 @@ static int parse_pair(JSONParserContext *ctxt, QDict *dict, va_list *ap)
         goto out;
     }

-    key = parse_value(ctxt, ap);
-    if (!key || qobject_type(key) != QTYPE_QSTRING) {
+    key = qobject_to(QString, parse_value(ctxt, ap));
+    if (!key) {
         parse_error(ctxt, peek, "key is not a string in object");
         goto out;
     }
@@ -308,14 +309,14 @@ static int parse_pair(JSONParserContext *ctxt, QDict *dict, va_list *ap)
         goto out;
     }

-    qdict_put_obj(dict, qstring_get_str(qobject_to(QString, key)), value);
+    qdict_put_obj(dict, qstring_get_str(key), value);

-    qobject_decref(key);
+    QDECREF(key);

     return 0;

 out:
-    qobject_decref(key);
+    QDECREF(key);

     return -1;
 }
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 45c8b53361f..d1997a0d8af 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -882,18 +882,20 @@ QObject *qdict_crumple(const QDict *src, Error **errp)

         child = qdict_get(two_level, prefix);
         if (suffix) {
-            if (child) {
-                if (qobject_type(child) != QTYPE_QDICT) {
+            QDict *child_dict = qobject_to(QDict, child);
+            if (!child_dict) {
+                if (child) {
                     error_setg(errp, "Key %s prefix is already set as a scalar",
                                prefix);
                     goto error;
                 }
-            } else {
-                child = QOBJECT(qdict_new());
-                qdict_put_obj(two_level, prefix, child);
+
+                child_dict = qdict_new();
+                qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
             }
+
             qobject_incref(ent->value);
-            qdict_put_obj(qobject_to(QDict, child), suffix, ent->value);
+            qdict_put_obj(child_dict, suffix, ent->value);
         } else {
             if (child) {
                 error_setg(errp, "Key %s prefix is already set as a dict",
@@ -913,9 +915,9 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
     multi_level = qdict_new();
     for (ent = qdict_first(two_level); ent != NULL;
          ent = qdict_next(two_level, ent)) {
-
-        if (qobject_type(ent->value) == QTYPE_QDICT) {
-            child = qdict_crumple(qobject_to(QDict, ent->value), errp);
+        QDict *dict = qobject_to(QDict, ent->value);
+        if (dict) {
+            child = qdict_crumple(dict, errp);
             if (!child) {
                 goto error;
             }
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 10/36] block: Handle null backing link
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (8 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 09/36] qapi: Make more of qobject_to() Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 11/36] block: Deprecate "backing": "" Eric Blake
                   ` (26 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Max Reitz, Kevin Wolf, Markus Armbruster, open list:Block layer core

From: Max Reitz <mreitz@redhat.com>

Instead of converting all "backing": null instances into "backing": "",
handle a null value directly in bdrv_open_inherit().

This enables explicitly null backing links for json:{} filenames.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-7-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase to qobject_to() parameter order and qapi headers split]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block.c                    |  5 ++++-
 blockdev.c                 | 14 --------------
 tests/qemu-iotests/089     | 20 ++++++++++++++++++++
 tests/qemu-iotests/089.out |  8 ++++++++
 4 files changed, 32 insertions(+), 15 deletions(-)

diff --git a/block.c b/block.c
index cd74da3a601..b673a004daf 100644
--- a/block.c
+++ b/block.c
@@ -33,6 +33,7 @@
 #include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qnull.h"
 #include "qapi/qmp/qstring.h"
 #include "qapi/qobject-output-visitor.h"
 #include "qapi/qapi-visit-block-core.h"
@@ -2645,7 +2646,9 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,

     /* See cautionary note on accessing @options above */
     backing = qdict_get_try_str(options, "backing");
-    if (backing && *backing == '\0') {
+    if (qobject_to(QNull, qdict_get(options, "backing")) != NULL ||
+        (backing && *backing == '\0'))
+    {
         flags |= BDRV_O_NO_BACKING;
         qdict_del(options, "backing");
     }
diff --git a/blockdev.c b/blockdev.c
index c9a212b3149..6a751996022 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3973,7 +3973,6 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)
     QObject *obj;
     Visitor *v = qobject_output_visitor_new(&obj);
     QDict *qdict;
-    const QDictEntry *ent;
     Error *local_err = NULL;

     visit_type_BlockdevOptions(v, NULL, &options, &local_err);
@@ -3987,19 +3986,6 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)

     qdict_flatten(qdict);

-    /*
-     * Rewrite "backing": null to "backing": ""
-     * TODO Rewrite "" to null instead, and perhaps not even here
-     */
-    for (ent = qdict_first(qdict); ent; ent = qdict_next(qdict, ent)) {
-        char *dot = strrchr(ent->key, '.');
-
-        if (!strcmp(dot ? dot + 1 : ent->key, "backing")
-            && qobject_type(ent->value) == QTYPE_QNULL) {
-            qdict_put(qdict, ent->key, qstring_new());
-        }
-    }
-
     if (!qdict_get_try_str(qdict, "node-name")) {
         error_setg(errp, "'node-name' must be specified for the root node");
         goto fail;
diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089
index 0b059aba905..aa1ba4a98e7 100755
--- a/tests/qemu-iotests/089
+++ b/tests/qemu-iotests/089
@@ -82,6 +82,26 @@ $QEMU_IO_PROG --cache $CACHEMODE \
 $QEMU_IO -c 'read -P 42 0 512' "$TEST_IMG" | _filter_qemu_io


+echo
+echo "=== Testing correct handling of 'backing':null ==="
+echo
+
+_make_test_img -b "$TEST_IMG.base" $IMG_SIZE
+
+# This should read 42
+$QEMU_IO -c 'read -P 42 0 512' "$TEST_IMG" | _filter_qemu_io
+
+# This should read 0
+$QEMU_IO -c 'read -P 0 0 512' "json:{\
+    'driver': '$IMGFMT',
+    'file': {
+        'driver': 'file',
+        'filename': '$TEST_IMG'
+    },
+    'backing': null
+}" | _filter_qemu_io
+
+
 # Taken from test 071
 echo
 echo "=== Testing blkdebug ==="
diff --git a/tests/qemu-iotests/089.out b/tests/qemu-iotests/089.out
index 0bf5a13ec18..89e3e4340a4 100644
--- a/tests/qemu-iotests/089.out
+++ b/tests/qemu-iotests/089.out
@@ -19,6 +19,14 @@ Pattern verification failed at offset 0, 512 bytes
 read 512/512 bytes at offset 0
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

+=== Testing correct handling of 'backing':null ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base
+read 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
 === Testing blkdebug ===

 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 11/36] block: Deprecate "backing": ""
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (9 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 10/36] block: Handle null backing link Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 12/36] docs: update QMP documents for OOB commands Eric Blake
                   ` (25 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Max Reitz, Kevin Wolf, Markus Armbruster, open list:Block layer core

From: Max Reitz <mreitz@redhat.com>

We have a clear replacement, so let's deprecate it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-8-mreitz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qemu-doc.texi        | 7 +++++++
 qapi/block-core.json | 4 ++--
 block.c              | 4 ++++
 qemu-options.hx      | 4 ++--
 4 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/qemu-doc.texi b/qemu-doc.texi
index 39e38c87ecf..f81fe40d61f 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -2779,6 +2779,13 @@ support page sizes < 4096 any longer.
 The ``xlnx-ep108'' machine has been replaced by the ``xlnx-zcu102'' machine.
 The ``xlnx-zcu102'' machine has the same features and capabilites in QEMU.

+@section Block device options
+
+@subsection "backing": "" (since 2.12.0)
+
+In order to prevent QEMU from automatically opening an image's backing
+chain, use ``"backing": null'' instead.
+
 @node License
 @appendix License

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 524d51567a5..55dcc8bd85a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1090,7 +1090,7 @@
 # @overlay: reference to the existing block device that will become
 #           the overlay of @node, as part of creating the snapshot.
 #           It must not have a current backing file (this can be
-#           achieved by passing "backing": "" to blockdev-add).
+#           achieved by passing "backing": null to blockdev-add).
 #
 # Since: 2.5
 ##
@@ -1238,7 +1238,7 @@
 #                     "node-name": "node1534",
 #                     "file": { "driver": "file",
 #                               "filename": "hd1.qcow2" },
-#                     "backing": "" } }
+#                     "backing": null } }
 #
 # <- { "return": {} }
 #
diff --git a/block.c b/block.c
index b673a004daf..e11f1355374 100644
--- a/block.c
+++ b/block.c
@@ -2649,6 +2649,10 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
     if (qobject_to(QNull, qdict_get(options, "backing")) != NULL ||
         (backing && *backing == '\0'))
     {
+        if (backing) {
+            warn_report("Use of \"backing\": \"\" is deprecated; "
+                        "use \"backing\": null instead");
+        }
         flags |= BDRV_O_NO_BACKING;
         qdict_del(options, "backing");
     }
diff --git a/qemu-options.hx b/qemu-options.hx
index 6585058c6cd..991ff4016db 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -740,8 +740,8 @@ Reference to or definition of the data source block driver node

 @item backing
 Reference to or definition of the backing file block device (default is taken
-from the image file). It is allowed to pass an empty string here in order to
-disable the default backing file.
+from the image file). It is allowed to pass @code{null} here in order to disable
+the default backing file.

 @item lazy-refcounts
 Whether to enable the lazy refcounts feature (on/off; default is taken from the
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 12/36] docs: update QMP documents for OOB commands
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (10 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 11/36] block: Deprecate "backing": "" Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 13/36] qobject: introduce qstring_get_try_str() Eric Blake
                   ` (24 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Michael Roth

From: Peter Xu <peterx@redhat.com>

Update both the developer and spec for the new QMP OOB (Out-Of-Band)
command.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-2-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/devel/qapi-code-gen.txt | 58 ++++++++++++++++++++++++++++++++++++++++----
 docs/interop/qmp-spec.txt    | 36 +++++++++++++++++++++------
 2 files changed, 82 insertions(+), 12 deletions(-)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 99d81230e26..a569d247455 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -554,9 +554,12 @@ following example objects:

 === Commands ===

+--- General Command Layout ---
+
 Usage: { 'command': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,
          '*returns': TYPE-NAME, '*boxed': true,
-         '*gen': false, '*success-response': false }
+         '*gen': false, '*success-response': false,
+         '*allow-oob': true }

 Commands are defined by using a dictionary containing several members,
 where three members are most common.  The 'command' member is a
@@ -636,6 +639,49 @@ possible, the command expression should include the optional key
 'success-response' with boolean value false.  So far, only QGA makes
 use of this member.

+A command can be declared to support Out-Of-Band (OOB) execution.  By
+default, commands do not support OOB.  To declare a command that
+supports it, the schema includes an extra 'allow-oob' field.  For
+example:
+
+ { 'command': 'migrate_recover',
+   'data': { 'uri': 'str' }, 'allow-oob': true }
+
+To execute a command with out-of-band priority, the client specifies
+the "control" field in the request, with "run-oob" set to
+true. Example:
+
+ => { "execute": "command-support-oob",
+      "arguments": { ... },
+      "control": { "run-oob": true } }
+ <= { "return": { } }
+
+Without it, even the commands that support out-of-band execution will
+still be run in-band.
+
+Under normal QMP command execution, the following apply to each
+command:
+
+- They are executed in order,
+- They run only in main thread of QEMU,
+- They have the BQL taken during execution.
+
+When a command is executed with OOB, the following changes occur:
+
+- They can be completed before a pending in-band command,
+- They run in a dedicated monitor thread,
+- They do not take the BQL during execution.
+
+OOB command handlers must satisfy the following conditions:
+
+- It executes extremely fast,
+- It does not take any lock, or, it can take very small locks if all
+  critical regions also follow the rules for OOB command handler code,
+- It does not invoke system calls that may block,
+- It does not access guest RAM that may block when userfaultfd is
+  enabled for postcopy live migration.
+
+If in doubt, do not implement OOB execution support.

 === Events ===

@@ -739,10 +785,12 @@ references by name.
 QAPI schema definitions not reachable that way are omitted.

 The SchemaInfo for a command has meta-type "command", and variant
-members "arg-type" and "ret-type".  On the wire, the "arguments"
-member of a client's "execute" command must conform to the object type
-named by "arg-type".  The "return" member that the server passes in a
-success response conforms to the type named by "ret-type".
+members "arg-type", "ret-type" and "allow-oob".  On the wire, the
+"arguments" member of a client's "execute" command must conform to the
+object type named by "arg-type".  The "return" member that the server
+passes in a success response conforms to the type named by
+"ret-type".  When "allow-oob" is set, it means the command supports
+out-of-band execution.

 If the command takes no arguments, "arg-type" names an object type
 without members.  Likewise, if the command returns nothing, "ret-type"
diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt
index f8b53560159..6fa193a80bc 100644
--- a/docs/interop/qmp-spec.txt
+++ b/docs/interop/qmp-spec.txt
@@ -83,16 +83,27 @@ The greeting message format is:
 2.2.1 Capabilities
 ------------------

-As of the date this document was last revised, no server or client
-capability strings have been defined.
+Currently supported capabilities are:

+- "oob": the QMP server supports "Out-Of-Band" (OOB) command
+  execution.  For more details, please see the "run-oob" parameter in
+  the "Issuing Commands" section below.  Not all commands allow this
+  "oob" execution.  The "query-qmp-schema" command can be used to
+  inspect which commands support "oob" execution.
+
+QMP clients can get a list of supported QMP capabilities of the QMP
+server in the greeting message mentioned above.  By default, all the
+capabilities are off.  To enable any QMP capabilities, the QMP client
+needs to send the "qmp_capabilities" command with an extra parameter
+for the requested capabilities.

 2.3 Issuing Commands
 --------------------

 The format for command execution is:

-{ "execute": json-string, "arguments": json-object, "id": json-value }
+{ "execute": json-string, "arguments": json-object, "id": json-value,
+  "control": json-object }

  Where,

@@ -102,10 +113,16 @@ The format for command execution is:
   required. Each command documents what contents will be considered
   valid when handling the json-argument
 - The "id" member is a transaction identification associated with the
-  command execution, it is optional and will be part of the response if
-  provided. The "id" member can be any json-value, although most
-  clients merely use a json-number incremented for each successive
-  command
+  command execution.  It is required for all commands if the OOB -
+  capability was enabled at startup, and optional otherwise.  The same
+  "id" field will be part of the response if provided. The "id" member
+  can be any json-value, although most clients merely use a
+  json-number incremented for each successive command
+- The "control" member is optional, and currently only used for
+  out-of-band execution. The handling or response of an "oob" command
+  can overtake prior in-band commands.  To enable "oob" handling of a
+  particular command, just provide a control field with: { "control":
+  { "run-oob": true } }

 2.4 Commands Responses
 ----------------------
@@ -113,6 +130,11 @@ The format for command execution is:
 There are two possible responses which the Server will issue as the result
 of a command execution: success or error.

+As long as the commands were issued with a proper "id" field, then the
+same "id" field will be attached in the corresponding response message
+so that requests and responses can match.  Clients should drop all the
+responses that have an unknown "id" field.
+
 2.4.1 success
 -------------

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 13/36] qobject: introduce qstring_get_try_str()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (11 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 12/36] docs: update QMP documents for OOB commands Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 14/36] qobject: introduce qobject_get_try_str() Eric Blake
                   ` (23 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

The only difference from qstring_get_str() is that it allows the qstring
to be NULL.  If so, NULL is returned.

CC: Eric Blake <eblake@redhat.com>
CC: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-3-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qstring.h |  1 +
 qobject/qstring.c          | 10 ++++++++++
 2 files changed, 11 insertions(+)

diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index b72843fc1b0..38c680b7185 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -27,6 +27,7 @@ QString *qstring_from_str(const char *str);
 QString *qstring_from_substr(const char *str, int start, int end);
 size_t qstring_get_length(const QString *qstring);
 const char *qstring_get_str(const QString *qstring);
+const char *qstring_get_try_str(const QString *qstring);
 void qstring_append_int(QString *qstring, int64_t value);
 void qstring_append(QString *qstring, const char *str);
 void qstring_append_chr(QString *qstring, int c);
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 8bae529ef79..5777f0af231 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -116,6 +116,16 @@ const char *qstring_get_str(const QString *qstring)
     return qstring->string;
 }

+/**
+ * qstring_get_try_str(): Return a pointer to the stored string
+ *
+ * NOTE: will return NULL if qstring is not provided.
+ */
+const char *qstring_get_try_str(const QString *qstring)
+{
+    return qstring ? qstring_get_str(qstring) : NULL;
+}
+
 /**
  * qstring_is_equal(): Test whether the two QStrings are equal
  */
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 14/36] qobject: introduce qobject_get_try_str()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (12 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 13/36] qobject: introduce qstring_get_try_str() Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 15/36] qobject: let object_property_get_str() use new API Eric Blake
                   ` (22 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

A quick way to fetch string from qobject when it's a QString.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-4-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase to qobject_to() macro]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/qstring.h |  1 +
 qobject/qstring.c          | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index 38c680b7185..30ae260a7fc 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -28,6 +28,7 @@ QString *qstring_from_substr(const char *str, int start, int end);
 size_t qstring_get_length(const QString *qstring);
 const char *qstring_get_str(const QString *qstring);
 const char *qstring_get_try_str(const QString *qstring);
+const char *qobject_get_try_str(const QObject *qstring);
 void qstring_append_int(QString *qstring, int64_t value);
 void qstring_append(QString *qstring, const char *str);
 void qstring_append_chr(QString *qstring, int c);
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 5777f0af231..afca54b47ae 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -126,6 +126,17 @@ const char *qstring_get_try_str(const QString *qstring)
     return qstring ? qstring_get_str(qstring) : NULL;
 }

+/**
+ * qobject_get_try_str(): Return a pointer to the corresponding string
+ *
+ * NOTE: the string will only be returned if the object is valid, and
+ * its type is QString, otherwise NULL is returned.
+ */
+const char *qobject_get_try_str(const QObject *qstring)
+{
+    return qstring_get_try_str(qobject_to(QString, qstring));
+}
+
 /**
  * qstring_is_equal(): Test whether the two QStrings are equal
  */
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 15/36] qobject: let object_property_get_str() use new API
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (13 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 14/36] qobject: introduce qobject_get_try_str() Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 16/36] monitor: move skip_flush into monitor_data_init Eric Blake
                   ` (21 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Andreas Färber

From: Peter Xu <peterx@redhat.com>

We can simplify object_property_get_str() using the new
qobject_get_try_str().

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-5-peterx@redhat.com>
[eblake: rebase context of qobject_to() macro]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qom/object.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/qom/object.c b/qom/object.c
index 7b3a6158fde..1c1b8cf8be0 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -1123,18 +1123,15 @@ char *object_property_get_str(Object *obj, const char *name,
                               Error **errp)
 {
     QObject *ret = object_property_get_qobject(obj, name, errp);
-    QString *qstring;
     char *retval;

     if (!ret) {
         return NULL;
     }
-    qstring = qobject_to(QString, ret);
-    if (!qstring) {
+
+    retval = g_strdup(qobject_get_try_str(ret));
+    if (!retval) {
         error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "string");
-        retval = NULL;
-    } else {
-        retval = g_strdup(qstring_get_str(qstring));
     }

     qobject_decref(ret);
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 16/36] monitor: move skip_flush into monitor_data_init
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (14 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 15/36] qobject: let object_property_get_str() use new API Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 17/36] monitor: move the cur_mon hack deeper for QMP Eric Blake
                   ` (20 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

It's part of the data init.  Collect it.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-6-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/monitor.c b/monitor.c
index c18a2f2dc98..b5acc92b735 100644
--- a/monitor.c
+++ b/monitor.c
@@ -570,13 +570,14 @@ static void monitor_qapi_event_init(void)

 static void handle_hmp_command(Monitor *mon, const char *cmdline);

-static void monitor_data_init(Monitor *mon)
+static void monitor_data_init(Monitor *mon, bool skip_flush)
 {
     memset(mon, 0, sizeof(Monitor));
     qemu_mutex_init(&mon->out_lock);
     mon->outbuf = qstring_new();
     /* Use *mon_cmds by default. */
     mon->cmd_table = mon_cmds;
+    mon->skip_flush = skip_flush;
 }

 static void monitor_data_destroy(Monitor *mon)
@@ -597,8 +598,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
     char *output = NULL;
     Monitor *old_mon, hmp;

-    monitor_data_init(&hmp);
-    hmp.skip_flush = true;
+    monitor_data_init(&hmp, true);

     old_mon = cur_mon;
     cur_mon = &hmp;
@@ -4039,7 +4039,7 @@ void monitor_init(Chardev *chr, int flags)
     }

     mon = g_malloc(sizeof(*mon));
-    monitor_data_init(mon);
+    monitor_data_init(mon, false);

     qemu_chr_fe_init(&mon->chr, chr, &error_abort);
     mon->flags = flags;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 17/36] monitor: move the cur_mon hack deeper for QMP
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (15 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 16/36] monitor: move skip_flush into monitor_data_init Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 18/36] monitor: unify global init Eric Blake
                   ` (19 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

In monitor_qmp_read(), we have the hack to temporarily replace the
cur_mon pointer.  Now we move this hack deeper inside the QMP dispatcher
routine since the Monitor pointer can be actually obtained using
container_of() upon the parser object, just like most of the other JSON
parser users do.

This does not make much sense as a single patch.  However, this will be
a big step for the next patch, when the QMP dispatcher routine will be
split from the QMP parser.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-7-peterx@redhat.com>
[eblake: rebase context of qobject_to() macro]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/monitor.c b/monitor.c
index b5acc92b735..83e98741b4c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3762,7 +3762,9 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
     QObject *req, *rsp = NULL, *id = NULL;
     QDict *qdict = NULL;
-    Monitor *mon = cur_mon;
+    MonitorQMP *mon_qmp = container_of(parser, MonitorQMP, parser);
+    Monitor *old_mon, *mon = container_of(mon_qmp, Monitor, qmp);
+
     Error *err = NULL;

     req = json_parser_parse_err(tokens, NULL, &err);
@@ -3787,8 +3789,13 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         QDECREF(req_json);
     }

+    old_mon = cur_mon;
+    cur_mon = mon;
+
     rsp = qmp_dispatch(cur_mon->qmp.commands, req);

+    cur_mon = old_mon;
+
     if (mon->qmp.commands == &qmp_cap_negotiation_commands) {
         qdict = qdict_get_qdict(qobject_to(QDict, rsp), "error");
         if (qdict
@@ -3825,13 +3832,9 @@ err_out:

 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
 {
-    Monitor *old_mon = cur_mon;
+    Monitor *mon = opaque;

-    cur_mon = opaque;
-
-    json_message_parser_feed(&cur_mon->qmp.parser, (const char *) buf, size);
-
-    cur_mon = old_mon;
+    json_message_parser_feed(&mon->qmp.parser, (const char *) buf, size);
 }

 static void monitor_read(void *opaque, const uint8_t *buf, int size)
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 18/36] monitor: unify global init
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (16 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 17/36] monitor: move the cur_mon hack deeper for QMP Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 19/36] monitor: let mon_list be tail queue Eric Blake
                   ` (18 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster, Paolo Bonzini

From: Peter Xu <peterx@redhat.com>

There are many places where the monitor initializes its globals:

- monitor_init_qmp_commands() at the very beginning
- single function to init monitor_lock
- in the first entry of monitor_init() using "is_first_init"

Unify them a bit.

monitor_lock is not used before monitor_init() (as confirmed by code
analysis and gdb watchpoints); so we are safe delaying what was a
constructor-time initialization of the mutex into the later first call
to monitor_init().

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-8-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/monitor/monitor.h |  2 +-
 vl.c                      |  7 ++++++-
 monitor.c                 | 25 ++++++++++---------------
 3 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index d1024d4bdc3..0cb0538a315 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -16,7 +16,7 @@ extern Monitor *cur_mon;

 bool monitor_cur_is_qmp(void);

-void monitor_init_qmp_commands(void);
+void monitor_init_globals(void);
 void monitor_init(Chardev *chr, int flags);
 void monitor_cleanup(void);

diff --git a/vl.c b/vl.c
index 3ef04ce991f..2e4fac7e4a0 100644
--- a/vl.c
+++ b/vl.c
@@ -3065,7 +3065,6 @@ int main(int argc, char **argv, char **envp)
     qemu_init_exec_dir(argv[0]);

     module_call_init(MODULE_INIT_QOM);
-    monitor_init_qmp_commands();

     qemu_add_opts(&qemu_drive_opts);
     qemu_add_drive_opts(&qemu_legacy_drive_opts);
@@ -4535,6 +4534,12 @@ int main(int argc, char **argv, char **envp)
     default_drive(default_floppy, snapshot, IF_FLOPPY, 0, FD_OPTS);
     default_drive(default_sdcard, snapshot, IF_SD, 0, SD_OPTS);

+    /*
+     * Note: qtest_enabled() (which is used in monitor_qapi_event_init())
+     * depends on configure_accelerator() above.
+     */
+    monitor_init_globals();
+
     if (qemu_opts_foreach(qemu_find_opts("mon"),
                           mon_init_func, NULL, NULL)) {
         exit(1);
diff --git a/monitor.c b/monitor.c
index 83e98741b4c..71d53be1ddd 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1003,7 +1003,7 @@ static void qmp_unregister_commands_hack(void)
 #endif
 }

-void monitor_init_qmp_commands(void)
+static void monitor_init_qmp_commands(void)
 {
     /*
      * Two command lists:
@@ -3985,6 +3985,14 @@ static void sortcmdlist(void)
     qsort((void *)info_cmds, array_num, elem_size, compare_mon_cmd);
 }

+void monitor_init_globals(void)
+{
+    monitor_init_qmp_commands();
+    monitor_qapi_event_init();
+    sortcmdlist();
+    qemu_mutex_init(&monitor_lock);
+}
+
 /* These functions just adapt the readline interface in a typesafe way.  We
  * could cast function pointers but that discards compiler checks.
  */
@@ -4025,23 +4033,10 @@ void error_vprintf_unless_qmp(const char *fmt, va_list ap)
     }
 }

-static void __attribute__((constructor)) monitor_lock_init(void)
-{
-    qemu_mutex_init(&monitor_lock);
-}
-
 void monitor_init(Chardev *chr, int flags)
 {
-    static int is_first_init = 1;
-    Monitor *mon;
+    Monitor *mon = g_malloc(sizeof(*mon));

-    if (is_first_init) {
-        monitor_qapi_event_init();
-        sortcmdlist();
-        is_first_init = 0;
-    }
-
-    mon = g_malloc(sizeof(*mon));
     monitor_data_init(mon, false);

     qemu_chr_fe_init(&mon->chr, chr, &error_abort);
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 19/36] monitor: let mon_list be tail queue
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (17 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 18/36] monitor: unify global init Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 20/36] monitor: allow using IO thread for parsing Eric Blake
                   ` (17 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

It was QLIST.  I want to use this list to do monitor priority job later,
which need tail insertion ability.  So switching to a tail queue.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-9-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/monitor.c b/monitor.c
index 71d53be1ddd..63b93f83217 100644
--- a/monitor.c
+++ b/monitor.c
@@ -207,7 +207,7 @@ struct Monitor {
     void *password_opaque;
     mon_cmd_t *cmd_table;
     QLIST_HEAD(,mon_fd_t) fds;
-    QLIST_ENTRY(Monitor) entry;
+    QTAILQ_ENTRY(Monitor) entry;
 };

 /* QMP checker flags */
@@ -216,7 +216,7 @@ struct Monitor {
 /* Protects mon_list, monitor_event_state.  */
 static QemuMutex monitor_lock;

-static QLIST_HEAD(mon_list, Monitor) mon_list;
+static QTAILQ_HEAD(mon_list, Monitor) mon_list;
 static QLIST_HEAD(mon_fdsets, MonFdset) mon_fdsets;
 static int mon_refcount;

@@ -417,7 +417,7 @@ static void monitor_qapi_event_emit(QAPIEvent event, QDict *qdict)
     Monitor *mon;

     trace_monitor_protocol_event_emit(event, qdict);
-    QLIST_FOREACH(mon, &mon_list, entry) {
+    QTAILQ_FOREACH(mon, &mon_list, entry) {
         if (monitor_is_qmp(mon)
             && mon->qmp.commands != &qmp_cap_negotiation_commands) {
             monitor_json_emitter(mon, QOBJECT(qdict));
@@ -4060,7 +4060,7 @@ void monitor_init(Chardev *chr, int flags)
     }

     qemu_mutex_lock(&monitor_lock);
-    QLIST_INSERT_HEAD(&mon_list, mon, entry);
+    QTAILQ_INSERT_HEAD(&mon_list, mon, entry);
     qemu_mutex_unlock(&monitor_lock);
 }

@@ -4069,8 +4069,8 @@ void monitor_cleanup(void)
     Monitor *mon, *next;

     qemu_mutex_lock(&monitor_lock);
-    QLIST_FOREACH_SAFE(mon, &mon_list, entry, next) {
-        QLIST_REMOVE(mon, entry);
+    QTAILQ_FOREACH_SAFE(mon, &mon_list, entry, next) {
+        QTAILQ_REMOVE(&mon_list, mon, entry);
         monitor_data_destroy(mon);
         g_free(mon);
     }
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 20/36] monitor: allow using IO thread for parsing
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (18 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 19/36] monitor: let mon_list be tail queue Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 21/36] qmp: introduce QMPCapability Eric Blake
                   ` (16 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

For each Monitor, add one field "use_io_thr" to show whether it will be
using the dedicated monitor IO thread to handle input/output.  When set,
monitor IO parsing work will be offloaded to the dedicated monitor IO
thread, rather than the original main loop thread.

This only works for QMP.  HMP will always be run on the main loop
thread.

Currently we're still keeping use_io_thr off always.  Will turn it on
later at some point.

One thing to mention is that we cannot set use_io_thr for every QMP
monitor.  The problem is that MUXed typed chardevs may not work well
with it now. When MUX is used, frontend of chardev can be the monitor
plus something else.  The only thing we know would be safe to be run
outside main thread so far is the monitor frontend. All the rest of the
frontends should still be run in main thread only.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-10-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 90 insertions(+), 8 deletions(-)

diff --git a/monitor.c b/monitor.c
index 63b93f83217..ce9f7fc8e8a 100644
--- a/monitor.c
+++ b/monitor.c
@@ -35,6 +35,7 @@
 #include "net/net.h"
 #include "net/slirp.h"
 #include "chardev/char-fe.h"
+#include "chardev/char-io.h"
 #include "ui/qemu-spice.h"
 #include "sysemu/numa.h"
 #include "monitor/monitor.h"
@@ -79,6 +80,7 @@
 #include "qapi/qapi-introspect.h"
 #include "sysemu/qtest.h"
 #include "sysemu/cpus.h"
+#include "sysemu/iothread.h"
 #include "qemu/cutils.h"

 #if defined(TARGET_S390X)
@@ -192,6 +194,7 @@ struct Monitor {
     int flags;
     int suspend_cnt;
     bool skip_flush;
+    bool use_io_thr;

     QemuMutex out_lock;
     QString *outbuf;
@@ -210,6 +213,11 @@ struct Monitor {
     QTAILQ_ENTRY(Monitor) entry;
 };

+/* Let's add monitor global variables to this struct. */
+static struct {
+    IOThread *mon_iothread;
+} mon_global;
+
 /* QMP checker flags */
 #define QMP_ACCEPT_UNKNOWNS 1

@@ -570,7 +578,8 @@ static void monitor_qapi_event_init(void)

 static void handle_hmp_command(Monitor *mon, const char *cmdline);

-static void monitor_data_init(Monitor *mon, bool skip_flush)
+static void monitor_data_init(Monitor *mon, bool skip_flush,
+                              bool use_io_thr)
 {
     memset(mon, 0, sizeof(Monitor));
     qemu_mutex_init(&mon->out_lock);
@@ -578,6 +587,7 @@ static void monitor_data_init(Monitor *mon, bool skip_flush)
     /* Use *mon_cmds by default. */
     mon->cmd_table = mon_cmds;
     mon->skip_flush = skip_flush;
+    mon->use_io_thr = use_io_thr;
 }

 static void monitor_data_destroy(Monitor *mon)
@@ -598,7 +608,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
     char *output = NULL;
     Monitor *old_mon, hmp;

-    monitor_data_init(&hmp, true);
+    monitor_data_init(&hmp, true, false);

     old_mon = cur_mon;
     cur_mon = &hmp;
@@ -3985,12 +3995,29 @@ static void sortcmdlist(void)
     qsort((void *)info_cmds, array_num, elem_size, compare_mon_cmd);
 }

+static GMainContext *monitor_get_io_context(void)
+{
+    return iothread_get_g_main_context(mon_global.mon_iothread);
+}
+
+static AioContext *monitor_get_aio_context(void)
+{
+    return iothread_get_aio_context(mon_global.mon_iothread);
+}
+
+static void monitor_iothread_init(void)
+{
+    mon_global.mon_iothread = iothread_create("mon_iothread",
+                                              &error_abort);
+}
+
 void monitor_init_globals(void)
 {
     monitor_init_qmp_commands();
     monitor_qapi_event_init();
     sortcmdlist();
     qemu_mutex_init(&monitor_lock);
+    monitor_iothread_init();
 }

 /* These functions just adapt the readline interface in a typesafe way.  We
@@ -4033,11 +4060,41 @@ void error_vprintf_unless_qmp(const char *fmt, va_list ap)
     }
 }

+static void monitor_list_append(Monitor *mon)
+{
+    qemu_mutex_lock(&monitor_lock);
+    QTAILQ_INSERT_HEAD(&mon_list, mon, entry);
+    qemu_mutex_unlock(&monitor_lock);
+}
+
+static void monitor_qmp_setup_handlers_bh(void *opaque)
+{
+    Monitor *mon = opaque;
+    GMainContext *context;
+
+    if (mon->use_io_thr) {
+        /*
+         * When use_io_thr is set, we use the global shared dedicated
+         * IO thread for this monitor to handle input/output.
+         */
+        context = monitor_get_io_context();
+        /* We should have inited globals before reaching here. */
+        assert(context);
+    } else {
+        /* The default main loop, which is the main thread */
+        context = NULL;
+    }
+
+    qemu_chr_fe_set_handlers(&mon->chr, monitor_can_read, monitor_qmp_read,
+                             monitor_qmp_event, NULL, mon, context, true);
+    monitor_list_append(mon);
+}
+
 void monitor_init(Chardev *chr, int flags)
 {
     Monitor *mon = g_malloc(sizeof(*mon));

-    monitor_data_init(mon, false);
+    monitor_data_init(mon, false, false);

     qemu_chr_fe_init(&mon->chr, chr, &error_abort);
     mon->flags = flags;
@@ -4050,24 +4107,46 @@ void monitor_init(Chardev *chr, int flags)
     }

     if (monitor_is_qmp(mon)) {
-        qemu_chr_fe_set_handlers(&mon->chr, monitor_can_read, monitor_qmp_read,
-                                 monitor_qmp_event, NULL, mon, NULL, true);
         qemu_chr_fe_set_echo(&mon->chr, true);
         json_message_parser_init(&mon->qmp.parser, handle_qmp_command);
+        if (mon->use_io_thr) {
+            /*
+             * We can't call qemu_chr_fe_set_handlers() directly here
+             * since during the procedure the chardev will be active
+             * and running in monitor iothread, while we'll still do
+             * something before returning from it, which is a possible
+             * race too.  To avoid that, we just create a BH to setup
+             * the handlers.
+             */
+            aio_bh_schedule_oneshot(monitor_get_aio_context(),
+                                    monitor_qmp_setup_handlers_bh, mon);
+            /* We'll add this to mon_list in the BH when setup done */
+            return;
+        } else {
+            qemu_chr_fe_set_handlers(&mon->chr, monitor_can_read,
+                                     monitor_qmp_read, monitor_qmp_event,
+                                     NULL, mon, NULL, true);
+        }
     } else {
         qemu_chr_fe_set_handlers(&mon->chr, monitor_can_read, monitor_read,
                                  monitor_event, NULL, mon, NULL, true);
     }

-    qemu_mutex_lock(&monitor_lock);
-    QTAILQ_INSERT_HEAD(&mon_list, mon, entry);
-    qemu_mutex_unlock(&monitor_lock);
+    monitor_list_append(mon);
 }

 void monitor_cleanup(void)
 {
     Monitor *mon, *next;

+    /*
+     * We need to explicitly stop the iothread (but not destroy it),
+     * cleanup the monitor resources, then destroy the iothread since
+     * we need to unregister from chardev below in
+     * monitor_data_destroy(), and chardev is not thread-safe yet
+     */
+    iothread_stop(mon_global.mon_iothread);
+
     qemu_mutex_lock(&monitor_lock);
     QTAILQ_FOREACH_SAFE(mon, &mon_list, entry, next) {
         QTAILQ_REMOVE(&mon_list, mon, entry);
@@ -4075,6 +4154,9 @@ void monitor_cleanup(void)
         g_free(mon);
     }
     qemu_mutex_unlock(&monitor_lock);
+
+    iothread_destroy(mon_global.mon_iothread);
+    mon_global.mon_iothread = NULL;
 }

 QemuOptsList qemu_mon_opts = {
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 21/36] qmp: introduce QMPCapability
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (19 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 20/36] monitor: allow using IO thread for parsing Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 22/36] monitor: introduce monitor_qmp_respond() Eric Blake
                   ` (15 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

There were no QMP capabilities defined.  Define the first capability,
"oob", to allow out-of-band messages.

After this patch, we will allow QMP clients to enable QMP capabilities
when sending the first "qmp_capabilities" command.  Originally we are
starting QMP session with no arguments like:

  { "execute": "qmp_capabilities" }

Now we can enable some QMP capabilities using (take OOB as example,
which is the only capability that we support):

  { "execute": "qmp_capabilities",
    "arguments": { "enable": [ "oob" ] } }

When the "arguments" key is not provided, no capability is enabled.

For capability "oob", the monitor needs to be run on a dedicated IO
thread, otherwise the command will fail.  For example, trying to enable
OOB on a MUXed typed QMP monitor will fail.

One thing to mention is that QMP capabilities are per-monitor, and also
when the connection is closed due to some reason, the capabilities will
be reset.

Also, touch up qmp-test.c to test the new bits.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-11-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: touch up commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/misc.json   | 32 ++++++++++++++++++++---
 monitor.c        | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++----
 tests/qmp-test.c | 10 +++++++-
 3 files changed, 110 insertions(+), 9 deletions(-)

diff --git a/qapi/misc.json b/qapi/misc.json
index bcd5d10778a..b2f69eb4ece 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -10,21 +10,47 @@
 #
 # Enable QMP capabilities.
 #
-# Arguments: None.
+# Arguments:
+#
+# @enable:   An optional list of QMPCapability values to enable.  The
+#            client must not enable any capability that is not
+#            mentioned in the QMP greeting message.  If the field is not
+#            provided, it means no QMP capabilities will be enabled.
+#            (since 2.12)
 #
 # Example:
 #
-# -> { "execute": "qmp_capabilities" }
+# -> { "execute": "qmp_capabilities",
+#      "arguments": { "enable": [ "oob" ] } }
 # <- { "return": {} }
 #
 # Notes: This command is valid exactly when first connecting: it must be
 # issued before any other command will be accepted, and will fail once the
 # monitor is accepting other commands. (see qemu docs/interop/qmp-spec.txt)
 #
+# The QMP client needs to explicitly enable QMP capabilities, otherwise
+# all the QMP capabilities will be turned off by default.
+#
 # Since: 0.13
 #
 ##
-{ 'command': 'qmp_capabilities' }
+{ 'command': 'qmp_capabilities',
+  'data': { '*enable': [ 'QMPCapability' ] } }
+
+##
+# @QMPCapability:
+#
+# Enumeration of capabilities to be advertised during initial client
+# connection, used for agreeing on particular QMP extension behaviors.
+#
+# @oob:   QMP ability to support Out-Of-Band requests.
+#         (Please refer to qmp-spec.txt for more information on OOB)
+#
+# Since: 2.12
+#
+##
+{ 'enum': 'QMPCapability',
+  'data': [ 'oob' ] }

 ##
 # @VersionTriple:
diff --git a/monitor.c b/monitor.c
index ce9f7fc8e8a..1a95fb372c6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -59,6 +59,7 @@
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/json-streamer.h"
 #include "qapi/qmp/json-parser.h"
+#include "qapi/qmp/qlist.h"
 #include "qom/object_interfaces.h"
 #include "trace-root.h"
 #include "trace/control.h"
@@ -170,6 +171,7 @@ typedef struct {
      * mode.
      */
     QmpCommandList *commands;
+    bool qmp_caps[QMP_CAPABILITY__MAX];
 } MonitorQMP;

 /*
@@ -1039,8 +1041,42 @@ static void monitor_init_qmp_commands(void)
                          qmp_marshal_qmp_capabilities, QCO_NO_OPTIONS);
 }

-void qmp_qmp_capabilities(Error **errp)
+static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
+                           Error **errp)
 {
+    for (; list; list = list->next) {
+        assert(list->value < QMP_CAPABILITY__MAX);
+        switch (list->value) {
+        case QMP_CAPABILITY_OOB:
+            if (!mon->use_io_thr) {
+                /*
+                 * Out-Of-Band only works with monitors that are
+                 * running on dedicated IOThread.
+                 */
+                error_setg(errp, "This monitor does not support "
+                           "Out-Of-Band (OOB)");
+                return;
+            }
+            break;
+        default:
+            break;
+        }
+    }
+}
+
+/* This function should only be called after capabilities are checked. */
+static void qmp_caps_apply(Monitor *mon, QMPCapabilityList *list)
+{
+    for (; list; list = list->next) {
+        mon->qmp.qmp_caps[list->value] = true;
+    }
+}
+
+void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
+                          Error **errp)
+{
+    Error *local_err = NULL;
+
     if (cur_mon->qmp.commands == &qmp_commands) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
                   "Capabilities negotiation is already complete, command "
@@ -1048,6 +1084,21 @@ void qmp_qmp_capabilities(Error **errp)
         return;
     }

+    /* Enable QMP capabilities provided by the client if applicable. */
+    if (has_enable) {
+        qmp_caps_check(cur_mon, enable, &local_err);
+        if (local_err) {
+            /*
+             * Failed check on any of the capabilities will fail the
+             * entire command (and thus not apply any of the other
+             * capabilities that were also requested).
+             */
+            error_propagate(errp, local_err);
+            return;
+        }
+        qmp_caps_apply(cur_mon, enable);
+    }
+
     cur_mon->qmp.commands = &qmp_commands;
 }

@@ -3893,14 +3944,29 @@ void monitor_resume(Monitor *mon)
         readline_show_prompt(mon->rs);
 }

-static QObject *get_qmp_greeting(void)
+static QObject *get_qmp_greeting(Monitor *mon)
 {
+    QList *cap_list = qlist_new();
     QObject *ver = NULL;
+    QMPCapability cap;

     qmp_marshal_query_version(NULL, &ver, NULL);

-    return qobject_from_jsonf("{'QMP': {'version': %p, 'capabilities': []}}",
-                              ver);
+    for (cap = 0; cap < QMP_CAPABILITY__MAX; cap++) {
+        if (!mon->use_io_thr && cap == QMP_CAPABILITY_OOB) {
+            /* Monitors that are not using IOThread won't support OOB */
+            continue;
+        }
+        qlist_append(cap_list, qstring_from_str(QMPCapability_str(cap)));
+    }
+
+    return qobject_from_jsonf("{'QMP': {'version': %p, 'capabilities': %p}}",
+                              ver, cap_list);
+}
+
+static void monitor_qmp_caps_reset(Monitor *mon)
+{
+    memset(mon->qmp.qmp_caps, 0, sizeof(mon->qmp.qmp_caps));
 }

 static void monitor_qmp_event(void *opaque, int event)
@@ -3911,7 +3977,8 @@ static void monitor_qmp_event(void *opaque, int event)
     switch (event) {
     case CHR_EVENT_OPENED:
         mon->qmp.commands = &qmp_cap_negotiation_commands;
-        data = get_qmp_greeting();
+        monitor_qmp_caps_reset(mon);
+        data = get_qmp_greeting(mon);
         monitor_json_emitter(mon, data);
         qobject_decref(data);
         mon_refcount++;
diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index 22445d9ec25..a8384bdda3b 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -20,6 +20,7 @@
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/util.h"
 #include "qapi/visitor.h"
+#include "qapi/qmp/qstring.h"

 const char common_args[] = "-nodefaults -machine none";

@@ -79,6 +80,8 @@ static void test_qmp_protocol(void)
     QDict *resp, *q, *ret;
     QList *capabilities;
     QTestState *qts;
+    const QListEntry *entry;
+    QString *qstr;

     qts = qtest_init_without_qmp_handshake(common_args);

@@ -88,7 +91,12 @@ static void test_qmp_protocol(void)
     g_assert(q);
     test_version(qdict_get(q, "version"));
     capabilities = qdict_get_qlist(q, "capabilities");
-    g_assert(capabilities && qlist_empty(capabilities));
+    g_assert(capabilities);
+    entry = qlist_first(capabilities);
+    g_assert(entry);
+    qstr = qobject_to(QString, entry->value);
+    g_assert(qstr);
+    g_assert_cmpstr(qstring_get_str(qstr), ==, "oob");
     QDECREF(resp);

     /* Test valid command before handshake */
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 22/36] monitor: introduce monitor_qmp_respond()
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (20 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 21/36] qmp: introduce QMPCapability Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 23/36] monitor: let suspend_cnt be thread safe Eric Blake
                   ` (14 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

A tiny refactoring, preparing to split the QMP dispatcher away.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-12-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase to qobject_to() usage]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 50 +++++++++++++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 17 deletions(-)

diff --git a/monitor.c b/monitor.c
index 1a95fb372c6..a3412359ca8 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3819,6 +3819,38 @@ static int monitor_can_read(void *opaque)
     return (mon->suspend_cnt == 0) ? 1 : 0;
 }

+/*
+ * 1. This function takes ownership of rsp, err, and id.
+ * 2. rsp, err, and id may be NULL.
+ * 3. If err != NULL then rsp must be NULL.
+ */
+static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
+                                Error *err, QObject *id)
+{
+    QDict *qdict = NULL;
+
+    if (err) {
+        assert(!rsp);
+        qdict = qdict_new();
+        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
+        error_free(err);
+        rsp = QOBJECT(qdict);
+    }
+
+    if (rsp) {
+        if (id) {
+            /* This is for the qdict below. */
+            qobject_incref(id);
+            qdict_put_obj(qobject_to(QDict, rsp), "id", id);
+        }
+
+        monitor_json_emitter(mon, rsp);
+    }
+
+    qobject_decref(id);
+    qobject_decref(rsp);
+}
+
 static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
     QObject *req, *rsp = NULL, *id = NULL;
@@ -3870,24 +3902,8 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     }

 err_out:
-    if (err) {
-        qdict = qdict_new();
-        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
-        error_free(err);
-        rsp = QOBJECT(qdict);
-    }
+    monitor_qmp_respond(mon, rsp, err, id);

-    if (rsp) {
-        if (id) {
-            qdict_put_obj(qobject_to(QDict, rsp), "id", id);
-            id = NULL;
-        }
-
-        monitor_json_emitter(mon, rsp);
-    }
-
-    qobject_decref(id);
-    qobject_decref(rsp);
     qobject_decref(req);
 }

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 23/36] monitor: let suspend_cnt be thread safe
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (21 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 22/36] monitor: introduce monitor_qmp_respond() Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 24/36] monitor: let suspend/resume work even with QMPs Eric Blake
                   ` (13 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

Monitor code now can be run in more than one thread.  Let it be thread
safe when accessing suspend_cnt counter.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-13-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/monitor.c b/monitor.c
index a3412359ca8..608faf35c25 100644
--- a/monitor.c
+++ b/monitor.c
@@ -194,7 +194,7 @@ struct Monitor {
     CharBackend chr;
     int reset_seen;
     int flags;
-    int suspend_cnt;
+    int suspend_cnt;            /* Needs to be accessed atomically */
     bool skip_flush;
     bool use_io_thr;

@@ -3816,7 +3816,7 @@ static int monitor_can_read(void *opaque)
 {
     Monitor *mon = opaque;

-    return (mon->suspend_cnt == 0) ? 1 : 0;
+    return !atomic_mb_read(&mon->suspend_cnt);
 }

 /*
@@ -3948,7 +3948,7 @@ int monitor_suspend(Monitor *mon)
 {
     if (!mon->rs)
         return -ENOTTY;
-    mon->suspend_cnt++;
+    atomic_inc(&mon->suspend_cnt);
     return 0;
 }

@@ -3956,8 +3956,9 @@ void monitor_resume(Monitor *mon)
 {
     if (!mon->rs)
         return;
-    if (--mon->suspend_cnt == 0)
+    if (atomic_dec_fetch(&mon->suspend_cnt) == 0) {
         readline_show_prompt(mon->rs);
+    }
 }

 static QObject *get_qmp_greeting(Monitor *mon)
@@ -4022,19 +4023,19 @@ static void monitor_event(void *opaque, int event)
             monitor_resume(mon);
             monitor_flush(mon);
         } else {
-            mon->suspend_cnt = 0;
+            atomic_mb_set(&mon->suspend_cnt, 0);
         }
         break;

     case CHR_EVENT_MUX_OUT:
         if (mon->reset_seen) {
-            if (mon->suspend_cnt == 0) {
+            if (atomic_mb_read(&mon->suspend_cnt) == 0) {
                 monitor_printf(mon, "\n");
             }
             monitor_flush(mon);
             monitor_suspend(mon);
         } else {
-            mon->suspend_cnt++;
+            atomic_inc(&mon->suspend_cnt);
         }
         qemu_mutex_lock(&mon->out_lock);
         mon->mux_out = 1;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 24/36] monitor: let suspend/resume work even with QMPs
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (22 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 23/36] monitor: let suspend_cnt be thread safe Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 25/36] monitor: separate QMP parser and dispatcher Eric Blake
                   ` (12 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

This patches allows QMP monitors to be suspended/resumed.

One thing to mention is that for QMPs that are using IOThreads, we need
an explicit kick for the IOThread in case it is sleeping.

Meanwhile, we need to take special care on non-interactive HMPs.
Currently only gdbserver is using that.  For these monitors, we still
don't allow suspend/resume operations.

Since at it, add traces for the operations.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-14-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c    | 47 ++++++++++++++++++++++++++++++++++++++++++++---
 trace-events |  1 +
 2 files changed, 45 insertions(+), 3 deletions(-)

diff --git a/monitor.c b/monitor.c
index 608faf35c25..6b225e9fc5c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -250,6 +250,21 @@ static inline bool monitor_is_qmp(const Monitor *mon)
     return (mon->flags & MONITOR_USE_CONTROL);
 }

+/**
+ * Whether @mon is using readline?  Note: not all HMP monitors use
+ * readline, e.g., gdbserver has a non-interactive HMP monitor, so
+ * readline is not used there.
+ */
+static inline bool monitor_uses_readline(const Monitor *mon)
+{
+    return mon->flags & MONITOR_USE_READLINE;
+}
+
+static inline bool monitor_is_hmp_non_interactive(const Monitor *mon)
+{
+    return !monitor_is_qmp(mon) && !monitor_uses_readline(mon);
+}
+
 /**
  * Is the current monitor, if any, a QMP monitor?
  */
@@ -3946,19 +3961,45 @@ static void monitor_command_cb(void *opaque, const char *cmdline,

 int monitor_suspend(Monitor *mon)
 {
-    if (!mon->rs)
+    if (monitor_is_hmp_non_interactive(mon)) {
         return -ENOTTY;
+    }
+
     atomic_inc(&mon->suspend_cnt);
+
+    if (monitor_is_qmp(mon)) {
+        /*
+         * Kick iothread to make sure this takes effect.  It'll be
+         * evaluated again in prepare() of the watch object.
+         */
+        aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
+    }
+
+    trace_monitor_suspend(mon, 1);
     return 0;
 }

 void monitor_resume(Monitor *mon)
 {
-    if (!mon->rs)
+    if (monitor_is_hmp_non_interactive(mon)) {
         return;
+    }
+
     if (atomic_dec_fetch(&mon->suspend_cnt) == 0) {
-        readline_show_prompt(mon->rs);
+        if (monitor_is_qmp(mon)) {
+            /*
+             * For QMP monitors that are running in IOThread, let's
+             * kick the thread in case it's sleeping.
+             */
+            if (mon->use_io_thr) {
+                aio_notify(iothread_get_aio_context(mon_global.mon_iothread));
+            }
+        } else {
+            assert(mon->rs);
+            readline_show_prompt(mon->rs);
+        }
     }
+    trace_monitor_suspend(mon, -1);
 }

 static QObject *get_qmp_greeting(Monitor *mon)
diff --git a/trace-events b/trace-events
index 89fcad0fd13..fe10c3b487c 100644
--- a/trace-events
+++ b/trace-events
@@ -47,6 +47,7 @@ monitor_protocol_event_emit(uint32_t event, void *data) "event=%d data=%p"
 monitor_protocol_event_queue(uint32_t event, void *qdict, uint64_t rate) "event=%d data=%p rate=%" PRId64
 handle_hmp_command(void *mon, const char *cmdline) "mon %p cmdline: %s"
 handle_qmp_command(void *mon, const char *req) "mon %p req: %s"
+monitor_suspend(void *ptr, int cnt) "mon %p: %d"

 # dma-helpers.c
 dma_blk_io(void *dbs, void *bs, int64_t offset, bool to_dev) "dbs=%p bs=%p offset=%" PRId64 " to_dev=%d"
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 25/36] monitor: separate QMP parser and dispatcher
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (23 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 24/36] monitor: let suspend/resume work even with QMPs Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 26/36] qmp: add new event "command-dropped" Eric Blake
                   ` (11 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

Originally QMP goes through these steps:

  JSON Parser --> QMP Dispatcher --> Respond
      /|\    (2)                (3)     |
   (1) |                               \|/ (4)
       +---------  main thread  --------+

This patch does this:

  JSON Parser     QMP Dispatcher --> Respond
      /|\ |           /|\       (4)     |
       |  | (2)        | (3)            |  (5)
   (1) |  +----->      |               \|/
       +---------  main thread  <-------+

So the parsing job and the dispatching job is isolated now.  It gives us
a chance in follow up patches to totally move the parser outside.

The isolation is done using one QEMUBH. Only one dispatcher QEMUBH is
used for all the monitors.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-15-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweaks, rebase to qobject_to()]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 178 insertions(+), 23 deletions(-)

diff --git a/monitor.c b/monitor.c
index 6b225e9fc5c..07446d6b879 100644
--- a/monitor.c
+++ b/monitor.c
@@ -172,6 +172,13 @@ typedef struct {
      */
     QmpCommandList *commands;
     bool qmp_caps[QMP_CAPABILITY__MAX];
+    /*
+     * Protects qmp request/response queue.  Please take monitor_lock
+     * first when used together.
+     */
+    QemuMutex qmp_queue_lock;
+    /* Input queue that holds all the parsed QMP requests */
+    GQueue *qmp_requests;
 } MonitorQMP;

 /*
@@ -218,6 +225,8 @@ struct Monitor {
 /* Let's add monitor global variables to this struct. */
 static struct {
     IOThread *mon_iothread;
+    /* Bottom half to dispatch the requests received from IO thread */
+    QEMUBH *qmp_dispatcher_bh;
 } mon_global;

 /* QMP checker flags */
@@ -600,11 +609,13 @@ static void monitor_data_init(Monitor *mon, bool skip_flush,
 {
     memset(mon, 0, sizeof(Monitor));
     qemu_mutex_init(&mon->out_lock);
+    qemu_mutex_init(&mon->qmp.qmp_queue_lock);
     mon->outbuf = qstring_new();
     /* Use *mon_cmds by default. */
     mon->cmd_table = mon_cmds;
     mon->skip_flush = skip_flush;
     mon->use_io_thr = use_io_thr;
+    mon->qmp.qmp_requests = g_queue_new();
 }

 static void monitor_data_destroy(Monitor *mon)
@@ -617,6 +628,8 @@ static void monitor_data_destroy(Monitor *mon)
     readline_free(mon->rs);
     QDECREF(mon->outbuf);
     qemu_mutex_destroy(&mon->out_lock);
+    qemu_mutex_destroy(&mon->qmp.qmp_queue_lock);
+    g_queue_free(mon->qmp.qmp_requests);
 }

 char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
@@ -1056,6 +1069,16 @@ static void monitor_init_qmp_commands(void)
                          qmp_marshal_qmp_capabilities, QCO_NO_OPTIONS);
 }

+static bool qmp_cap_enabled(Monitor *mon, QMPCapability cap)
+{
+    return mon->qmp.qmp_caps[cap];
+}
+
+static bool qmp_oob_enabled(Monitor *mon)
+{
+    return qmp_cap_enabled(mon, QMP_CAPABILITY_OOB);
+}
+
 static void qmp_caps_check(Monitor *mon, QMPCapabilityList *list,
                            Error **errp)
 {
@@ -3866,30 +3889,39 @@ static void monitor_qmp_respond(Monitor *mon, QObject *rsp,
     qobject_decref(rsp);
 }

-static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
+struct QMPRequest {
+    /* Owner of the request */
+    Monitor *mon;
+    /* "id" field of the request */
+    QObject *id;
+    /* Request object to be handled */
+    QObject *req;
+    /*
+     * Whether we need to resume the monitor afterward.  This flag is
+     * used to emulate the old QMP server behavior that the current
+     * command must be completed before execution of the next one.
+     */
+    bool need_resume;
+};
+typedef struct QMPRequest QMPRequest;
+
+/*
+ * Dispatch one single QMP request. The function will free the req_obj
+ * and objects inside it before return.
+ */
+static void monitor_qmp_dispatch_one(QMPRequest *req_obj)
 {
-    QObject *req, *rsp = NULL, *id = NULL;
+    Monitor *mon, *old_mon;
+    QObject *req, *rsp = NULL, *id;
     QDict *qdict = NULL;
-    MonitorQMP *mon_qmp = container_of(parser, MonitorQMP, parser);
-    Monitor *old_mon, *mon = container_of(mon_qmp, Monitor, qmp);
+    bool need_resume;

-    Error *err = NULL;
+    req = req_obj->req;
+    mon = req_obj->mon;
+    id = req_obj->id;
+    need_resume = req_obj->need_resume;

-    req = json_parser_parse_err(tokens, NULL, &err);
-    if (!req && !err) {
-        /* json_parser_parse_err() sucks: can fail without setting @err */
-        error_setg(&err, QERR_JSON_PARSING);
-    }
-    if (err) {
-        goto err_out;
-    }
-
-    qdict = qobject_to(QDict, req);
-    if (qdict) {
-        id = qdict_get(qdict, "id");
-        qobject_incref(id);
-        qdict_del(qdict, "id");
-    } /* else will fail qmp_dispatch() */
+    g_free(req_obj);

     if (trace_event_get_state_backends(TRACE_HANDLE_QMP_COMMAND)) {
         QString *req_json = qobject_to_json(req);
@@ -3900,7 +3932,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     old_mon = cur_mon;
     cur_mon = mon;

-    rsp = qmp_dispatch(cur_mon->qmp.commands, req);
+    rsp = qmp_dispatch(mon->qmp.commands, req);

     cur_mon = old_mon;

@@ -3916,12 +3948,122 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         }
     }

-err_out:
-    monitor_qmp_respond(mon, rsp, err, id);
+    /* Respond if necessary */
+    monitor_qmp_respond(mon, rsp, NULL, id);
+
+    /* This pairs with the monitor_suspend() in handle_qmp_command(). */
+    if (need_resume) {
+        monitor_resume(mon);
+    }

     qobject_decref(req);
 }

+/*
+ * Pop one QMP request from monitor queues, return NULL if not found.
+ * We are using round-robin fashion to pop the request, to avoid
+ * processing commands only on a very busy monitor.  To achieve that,
+ * when we process one request on a specific monitor, we put that
+ * monitor to the end of mon_list queue.
+ */
+static QMPRequest *monitor_qmp_requests_pop_one(void)
+{
+    QMPRequest *req_obj = NULL;
+    Monitor *mon;
+
+    qemu_mutex_lock(&monitor_lock);
+
+    QTAILQ_FOREACH(mon, &mon_list, entry) {
+        qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
+        req_obj = g_queue_pop_head(mon->qmp.qmp_requests);
+        qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+        if (req_obj) {
+            break;
+        }
+    }
+
+    if (req_obj) {
+        /*
+         * We found one request on the monitor. Degrade this monitor's
+         * priority to lowest by re-inserting it to end of queue.
+         */
+        QTAILQ_REMOVE(&mon_list, mon, entry);
+        QTAILQ_INSERT_TAIL(&mon_list, mon, entry);
+    }
+
+    qemu_mutex_unlock(&monitor_lock);
+
+    return req_obj;
+}
+
+static void monitor_qmp_bh_dispatcher(void *data)
+{
+    QMPRequest *req_obj = monitor_qmp_requests_pop_one();
+
+    if (req_obj) {
+        monitor_qmp_dispatch_one(req_obj);
+        /* Reschedule instead of looping so the main loop stays responsive */
+        qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+    }
+}
+
+static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
+{
+    QObject *req, *id = NULL;
+    QDict *qdict = NULL;
+    MonitorQMP *mon_qmp = container_of(parser, MonitorQMP, parser);
+    Monitor *mon = container_of(mon_qmp, Monitor, qmp);
+    Error *err = NULL;
+    QMPRequest *req_obj;
+
+    req = json_parser_parse_err(tokens, NULL, &err);
+    if (!req && !err) {
+        /* json_parser_parse_err() sucks: can fail without setting @err */
+        error_setg(&err, QERR_JSON_PARSING);
+    }
+    if (err) {
+        monitor_qmp_respond(mon, NULL, err, NULL);
+        qobject_decref(req);
+        return;
+    }
+
+    qdict = qobject_to(QDict, req);
+    if (qdict) {
+        id = qdict_get(qdict, "id");
+        qobject_incref(id);
+        qdict_del(qdict, "id");
+    } /* else will fail qmp_dispatch() */
+
+    req_obj = g_new0(QMPRequest, 1);
+    req_obj->mon = mon;
+    req_obj->id = id;
+    req_obj->req = req;
+    req_obj->need_resume = false;
+
+    /*
+     * If OOB is not enabled on the current monitor, we'll emulate the
+     * old behavior that we won't process the current monitor any more
+     * until it has responded.  This helps make sure that as long as
+     * OOB is not enabled, the server will never drop any command.
+     */
+    if (!qmp_oob_enabled(mon)) {
+        monitor_suspend(mon);
+        req_obj->need_resume = true;
+    }
+
+    /*
+     * Put the request to the end of queue so that requests will be
+     * handled in time order.  Ownership for req_obj, req, id,
+     * etc. will be delivered to the handler side.
+     */
+    qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
+    g_queue_push_tail(mon->qmp.qmp_requests, req_obj);
+    qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+
+    /* Kick the dispatcher routine */
+    qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+}
+
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
 {
     Monitor *mon = opaque;
@@ -4134,6 +4276,15 @@ static void monitor_iothread_init(void)
 {
     mon_global.mon_iothread = iothread_create("mon_iothread",
                                               &error_abort);
+
+    /*
+     * This MUST be on main loop thread since we have commands that
+     * have assumption to be run on main loop thread.  It would be
+     * nice that one day we can remove this assumption in the future.
+     */
+    mon_global.qmp_dispatcher_bh = aio_bh_new(qemu_get_aio_context(),
+                                              monitor_qmp_bh_dispatcher,
+                                              NULL);
 }

 void monitor_init_globals(void)
@@ -4280,6 +4431,10 @@ void monitor_cleanup(void)
     }
     qemu_mutex_unlock(&monitor_lock);

+    /* QEMUBHs needs to be deleted before destroying the IOThread. */
+    qemu_bh_delete(mon_global.qmp_dispatcher_bh);
+    mon_global.qmp_dispatcher_bh = NULL;
+
     iothread_destroy(mon_global.mon_iothread);
     mon_global.mon_iothread = NULL;
 }
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 26/36] qmp: add new event "command-dropped"
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (24 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 25/36] monitor: separate QMP parser and dispatcher Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 27/36] monitor: send event when command queue full Eric Blake
                   ` (10 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

This event will be emitted if one QMP command is dropped.  Also,
declare an enum for the reasons.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-16-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/misc.json | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/qapi/misc.json b/qapi/misc.json
index b2f69eb4ece..b6ebf210ba8 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3242,3 +3242,40 @@
 # Since: 2.9
 ##
 { 'command': 'query-vm-generation-id', 'returns': 'GuidInfo' }
+
+##
+# @CommandDropReason:
+#
+# Reasons that caused one command to be dropped.
+#
+# @queue-full: the command queue is full. This can only occur when
+#              the client sends a new non-oob command before the
+#              response to the previous non-oob command has been
+#              received.
+#
+# Since: 2.12
+##
+{ 'enum': 'CommandDropReason',
+  'data': [ 'queue-full' ] }
+
+##
+# @COMMAND_DROPPED:
+#
+# Emitted when a command is dropped due to some reason.  Commands can
+# only be dropped when the oob capability is enabled.
+#
+# @id: The dropped command's "id" field.
+#
+# @reason: The reason why the command is dropped.
+#
+# Since: 2.12
+#
+# Example:
+#
+# { "event": "COMMAND_DROPPED",
+#   "data": {"result": {"id": "libvirt-102",
+#                       "reason": "queue-full" } } }
+#
+##
+{ 'event': 'COMMAND_DROPPED' ,
+  'data': { 'id': 'any', 'reason': 'CommandDropReason' } }
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 27/36] monitor: send event when command queue full
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (25 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 26/36] qmp: add new event "command-dropped" Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 28/36] qapi: introduce new cmd option "allow-oob" Eric Blake
                   ` (9 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

Set maximum QMP command queue length to 8.  If the queue is full,
instead of queuing the command, we directly return a "command-dropped"
event, telling the client that a specific command is dropped.

Note that this flow control mechanism is only valid if OOB is enabled.
If it's not, the effective queue length will always be 1, which strictly
follows original behavior of QMP command handling (which never drops
messages).

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-17-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: commit message grammar, abort on failure to send event]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/monitor.c b/monitor.c
index 07446d6b879..d28457d60d7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4007,6 +4007,8 @@ static void monitor_qmp_bh_dispatcher(void *data)
     }
 }

+#define  QMP_REQ_QUEUE_LEN_MAX  (8)
+
 static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
     QObject *req, *id = NULL;
@@ -4040,6 +4042,9 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     req_obj->req = req;
     req_obj->need_resume = false;

+    /* Protect qmp_requests and fetching its length. */
+    qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
+
     /*
      * If OOB is not enabled on the current monitor, we'll emulate the
      * old behavior that we won't process the current monitor any more
@@ -4049,6 +4054,18 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     if (!qmp_oob_enabled(mon)) {
         monitor_suspend(mon);
         req_obj->need_resume = true;
+    } else {
+        /* Drop the request if queue is full. */
+        if (mon->qmp.qmp_requests->length >= QMP_REQ_QUEUE_LEN_MAX) {
+            qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+            qapi_event_send_command_dropped(id,
+                                            COMMAND_DROP_REASON_QUEUE_FULL,
+                                            &error_abort);
+            qobject_decref(id);
+            qobject_decref(req);
+            g_free(req_obj);
+            return;
+        }
     }

     /*
@@ -4056,7 +4073,6 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
      * handled in time order.  Ownership for req_obj, req, id,
      * etc. will be delivered to the handler side.
      */
-    qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
     g_queue_push_tail(mon->qmp.qmp_requests, req_obj);
     qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 28/36] qapi: introduce new cmd option "allow-oob"
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (26 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 27/36] monitor: send event when command queue full Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 29/36] qmp: support out-of-band (oob) execution Eric Blake
                   ` (8 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Michael Roth

From: Peter Xu <peterx@redhat.com>

Here "oob" stands for "Out-Of-Band".  When "allow-oob" is set, it means
the command allows out-of-band execution.

The "oob" idea is proposed by Markus Armbruster in following thread:

  https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg02057.html

This new "allow-oob" boolean will be exposed by "query-qmp-schema" as
well for command entries, so that QMP clients can know which commands
can be used in out-of-band calls. For example the command "migrate"
originally looks like:

  {"name": "migrate", "ret-type": "17", "meta-type": "command",
   "arg-type": "86"}

And it'll be changed into:

  {"name": "migrate", "ret-type": "17", "allow-oob": false,
   "meta-type": "command", "arg-type": "86"}

This patch only provides the QMP interface level changes.  It does not
contain the real out-of-band execution implementation yet.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-18-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase on introspection done by qlit]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/introspect.json           |  6 +++++-
 scripts/qapi/commands.py       | 18 +++++++++++++-----
 scripts/qapi/common.py         | 15 ++++++++++-----
 scripts/qapi/doc.py            |  2 +-
 scripts/qapi/introspect.py     |  7 +++++--
 include/qapi/qmp/dispatch.h    |  5 +++--
 tests/qapi-schema/test-qapi.py |  2 +-
 7 files changed, 38 insertions(+), 17 deletions(-)

diff --git a/qapi/introspect.json b/qapi/introspect.json
index 5b3e6e9d78b..c7f67b7d78b 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -259,12 +259,16 @@
 #
 # @ret-type: the name of the command's result type.
 #
+# @allow-oob: whether the command allows out-of-band execution.
+#             (Since: 2.12)
+#
 # TODO: @success-response (currently irrelevant, because it's QGA, not QMP)
 #
 # Since: 2.5
 ##
 { 'struct': 'SchemaInfoCommand',
-  'data': { 'arg-type': 'str', 'ret-type': 'str' } }
+  'data': { 'arg-type': 'str', 'ret-type': 'str',
+            'allow-oob': 'bool' } }

 ##
 # @SchemaInfoEvent:
diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 21a7e0dbe61..0c5da3a54d6 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -193,10 +193,18 @@ out:
     return ret


-def gen_register_command(name, success_response):
-    options = 'QCO_NO_OPTIONS'
+def gen_register_command(name, success_response, allow_oob):
+    options = []
+
     if not success_response:
-        options = 'QCO_NO_SUCCESS_RESP'
+        options += ['QCO_NO_SUCCESS_RESP']
+    if allow_oob:
+        options += ['QCO_ALLOW_OOB']
+
+    if not options:
+        options = ['QCO_NO_OPTIONS']
+
+    options = " | ".join(options)

     ret = mcgen('''
     qmp_register_command(cmds, "%(name)s",
@@ -268,7 +276,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
         genc.add(gen_registry(self._regy, self._prefix))

     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      gen, success_response, boxed, allow_oob):
         if not gen:
             return
         self._genh.add(gen_command_decl(name, arg_type, boxed, ret_type))
@@ -277,7 +285,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
             self._genc.add(gen_marshal_output(ret_type))
         self._genh.add(gen_marshal_decl(name))
         self._genc.add(gen_marshal(name, arg_type, boxed, ret_type))
-        self._regy += gen_register_command(name, success_response)
+        self._regy += gen_register_command(name, success_response, allow_oob)


 def gen_commands(schema, output_dir, prefix):
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 97e9060b674..2c05e3c2845 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -921,7 +921,8 @@ def check_exprs(exprs):
         elif 'command' in expr:
             meta = 'command'
             check_keys(expr_elem, 'command', [],
-                       ['data', 'returns', 'gen', 'success-response', 'boxed'])
+                       ['data', 'returns', 'gen', 'success-response',
+                        'boxed', 'allow-oob'])
         elif 'event' in expr:
             meta = 'event'
             check_keys(expr_elem, 'event', [], ['data', 'boxed'])
@@ -1044,7 +1045,7 @@ class QAPISchemaVisitor(object):
         pass

     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      gen, success_response, boxed, allow_oob):
         pass

     def visit_event(self, name, info, arg_type, boxed):
@@ -1421,7 +1422,7 @@ class QAPISchemaAlternateType(QAPISchemaType):

 class QAPISchemaCommand(QAPISchemaEntity):
     def __init__(self, name, info, doc, arg_type, ret_type,
-                 gen, success_response, boxed):
+                 gen, success_response, boxed, allow_oob):
         QAPISchemaEntity.__init__(self, name, info, doc)
         assert not arg_type or isinstance(arg_type, str)
         assert not ret_type or isinstance(ret_type, str)
@@ -1432,6 +1433,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         self.gen = gen
         self.success_response = success_response
         self.boxed = boxed
+        self.allow_oob = allow_oob

     def check(self, schema):
         if self._arg_type_name:
@@ -1455,7 +1457,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
     def visit(self, visitor):
         visitor.visit_command(self.name, self.info,
                               self.arg_type, self.ret_type,
-                              self.gen, self.success_response, self.boxed)
+                              self.gen, self.success_response,
+                              self.boxed, self.allow_oob)


 class QAPISchemaEvent(QAPISchemaEntity):
@@ -1674,6 +1677,7 @@ class QAPISchema(object):
         gen = expr.get('gen', True)
         success_response = expr.get('success-response', True)
         boxed = expr.get('boxed', False)
+        allow_oob = expr.get('allow-oob', False)
         if isinstance(data, OrderedDict):
             data = self._make_implicit_object_type(
                 name, info, doc, 'arg', self._make_members(data, info))
@@ -1681,7 +1685,8 @@ class QAPISchema(object):
             assert len(rets) == 1
             rets = self._make_array_type(rets[0], info)
         self._def_entity(QAPISchemaCommand(name, info, doc, data, rets,
-                                           gen, success_response, boxed))
+                                           gen, success_response,
+                                           boxed, allow_oob))

     def _def_event(self, expr, info, doc):
         name = expr['event']
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 79d11bbe9b5..9b312b2c51b 100644
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -227,7 +227,7 @@ class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):
                                body=texi_entity(doc, 'Members')))

     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      gen, success_response, boxed, allow_oob):
         doc = self.cur_doc
         if boxed:
             body = texi_body(doc)
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 1d46a6d6b6f..f9e67e8227b 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -41,6 +41,8 @@ def to_qlit(obj, level=0, suppress_first_indent=False):
         ret += 'QLIT_QDICT(((QLitDictEntry[]) {\n'
         ret += ',\n'.join(elts) + '\n'
         ret += indent(level) + '}))'
+    elif isinstance(obj, bool):
+        ret += 'QLIT_QBOOL(%s)' % ('true' if obj else 'false')
     else:
         assert False                # not implemented
     return ret
@@ -170,12 +172,13 @@ const QLitObject %(c_name)s = %(c_string)s;
                                     for m in variants.variants]})

     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      gen, success_response, boxed, allow_oob):
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
         self._gen_qlit(name, 'command',
                        {'arg-type': self._use_type(arg_type),
-                        'ret-type': self._use_type(ret_type)})
+                        'ret-type': self._use_type(ret_type),
+                        'allow-oob': allow_oob})

     def visit_event(self, name, info, arg_type, boxed):
         arg_type = arg_type or self._schema.the_empty_object_type
diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 1e694b5ecf9..26eb13ff417 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -20,8 +20,9 @@ typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);

 typedef enum QmpCommandOptions
 {
-    QCO_NO_OPTIONS = 0x0,
-    QCO_NO_SUCCESS_RESP = 0x1,
+    QCO_NO_OPTIONS            =  0x0,
+    QCO_NO_SUCCESS_RESP       =  (1U << 0),
+    QCO_ALLOW_OOB             =  (1U << 1),
 } QmpCommandOptions;

 typedef struct QmpCommand
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 67e417e2980..10e68b01d98 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -42,7 +42,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
         self._print_variants(variants)

     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      gen, success_response, boxed, allow_oob):
         print('command %s %s -> %s' % \
               (name, arg_type and arg_type.name, ret_type and ret_type.name))
         print('   gen=%s success_response=%s boxed=%s' % \
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 29/36] qmp: support out-of-band (oob) execution
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (27 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 28/36] qapi: introduce new cmd option "allow-oob" Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 30/36] qmp: isolate responses into io thread Eric Blake
                   ` (7 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert, Michael Roth

From: Peter Xu <peterx@redhat.com>

Having "allow-oob":true for a command does not mean that this command
will always be run in out-of-band mode.  The out-of-band quick path will
only be executed if we specify the extra "run-oob" flag when sending the
QMP request:

    { "execute":   "command-that-allows-oob",
      "arguments": { ... },
      "control":   { "run-oob": true } }

The "control" key is introduced to store this extra flag.  "control"
field is used to store arguments that are shared by all the commands,
rather than command specific arguments.  Let "run-oob" be the first.

Note that in the patch I exported qmp_dispatch_check_obj() to be used to
check the request earlier, and at the same time allowed "id" field to be
there since actually we always allow that.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-19-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase to qobject_to(), spelling fix]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/qapi/qmp/dispatch.h |  2 ++
 monitor.c                   | 84 ++++++++++++++++++++++++++++++++++++++++-----
 qapi/qmp-dispatch.c         | 33 +++++++++++++++++-
 trace-events                |  2 ++
 4 files changed, 111 insertions(+), 10 deletions(-)

diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 26eb13ff417..ffb4652f716 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -48,6 +48,8 @@ bool qmp_command_is_enabled(const QmpCommand *cmd);
 const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QObject *qmp_build_error_object(Error *err);
+QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp);
+bool qmp_is_oob(QDict *dict);

 typedef void (*qmp_cmd_callback_fn)(QmpCommand *cmd, void *opaque);

diff --git a/monitor.c b/monitor.c
index d28457d60d7..3b7a4a5c3a7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1110,6 +1110,44 @@ static void qmp_caps_apply(Monitor *mon, QMPCapabilityList *list)
     }
 }

+/*
+ * Return true if check successful, or false otherwise.  When false is
+ * returned, detailed error will be in errp if provided.
+ */
+static bool qmp_cmd_oob_check(Monitor *mon, QDict *req, Error **errp)
+{
+    const char *command;
+    QmpCommand *cmd;
+
+    command = qdict_get_try_str(req, "execute");
+    if (!command) {
+        error_setg(errp, "Command field 'execute' missing");
+        return false;
+    }
+
+    cmd = qmp_find_command(mon->qmp.commands, command);
+    if (!cmd) {
+        error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
+                  "The command %s has not been found", command);
+        return false;
+    }
+
+    if (qmp_is_oob(req)) {
+        if (!qmp_oob_enabled(mon)) {
+            error_setg(errp, "Please enable Out-Of-Band first "
+                       "for the session during capabilities negotiation");
+            return false;
+        }
+        if (!(cmd->options & QCO_ALLOW_OOB)) {
+            error_setg(errp, "The command %s does not support OOB",
+                       command);
+            return false;
+        }
+    }
+
+    return true;
+}
+
 void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
                           Error **errp)
 {
@@ -4001,6 +4039,7 @@ static void monitor_qmp_bh_dispatcher(void *data)
     QMPRequest *req_obj = monitor_qmp_requests_pop_one();

     if (req_obj) {
+        trace_monitor_qmp_cmd_in_band(qobject_get_try_str(req_obj->id) ?: "");
         monitor_qmp_dispatch_one(req_obj);
         /* Reschedule instead of looping so the main loop stays responsive */
         qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
@@ -4024,17 +4063,31 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         error_setg(&err, QERR_JSON_PARSING);
     }
     if (err) {
-        monitor_qmp_respond(mon, NULL, err, NULL);
-        qobject_decref(req);
-        return;
+        goto err;
     }

-    qdict = qobject_to(QDict, req);
-    if (qdict) {
-        id = qdict_get(qdict, "id");
-        qobject_incref(id);
-        qdict_del(qdict, "id");
-    } /* else will fail qmp_dispatch() */
+    /* Check against the request in general layout */
+    qdict = qmp_dispatch_check_obj(req, &err);
+    if (!qdict) {
+        goto err;
+    }
+
+    /* Check against OOB specific */
+    if (!qmp_cmd_oob_check(mon, qdict, &err)) {
+        goto err;
+    }
+
+    id = qdict_get(qdict, "id");
+
+    /* When OOB is enabled, the "id" field is mandatory. */
+    if (qmp_oob_enabled(mon) && !id) {
+        error_setg(&err, "Out-Of-Band capability requires that "
+                   "every command contains an 'id' field");
+        goto err;
+    }
+
+    qobject_incref(id);
+    qdict_del(qdict, "id");

     req_obj = g_new0(QMPRequest, 1);
     req_obj->mon = mon;
@@ -4042,6 +4095,14 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     req_obj->req = req;
     req_obj->need_resume = false;

+    if (qmp_is_oob(qdict)) {
+        /* Out-Of-Band (OOB) requests are executed directly in parser. */
+        trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(req_obj->id)
+                                          ?: "");
+        monitor_qmp_dispatch_one(req_obj);
+        return;
+    }
+
     /* Protect qmp_requests and fetching its length. */
     qemu_mutex_lock(&mon->qmp.qmp_queue_lock);

@@ -4078,6 +4139,11 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)

     /* Kick the dispatcher routine */
     qemu_bh_schedule(mon_global.qmp_dispatcher_bh);
+    return;
+
+err:
+    monitor_qmp_respond(mon, NULL, err, NULL);
+    qobject_decref(req);
 }

 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index af537f3d7d2..dd059072651 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -17,8 +17,9 @@
 #include "qapi/qmp/json-parser.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qbool.h"

-static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
+QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
 {
     const QDictEntry *ent;
     const char *arg_name;
@@ -50,6 +51,14 @@ static QDict *qmp_dispatch_check_obj(const QObject *request, Error **errp)
                            "QMP input member 'arguments' must be an object");
                 return NULL;
             }
+        } else if (!strcmp(arg_name, "id")) {
+            continue;
+        } else if (!strcmp(arg_name, "control")) {
+            if (qobject_type(arg_obj) != QTYPE_QDICT) {
+                error_setg(errp,
+                           "QMP input member 'control' must be a dict");
+                return NULL;
+            }
         } else {
             error_setg(errp, "QMP input member '%s' is unexpected",
                        arg_name);
@@ -120,6 +129,28 @@ QObject *qmp_build_error_object(Error *err)
                               error_get_pretty(err));
 }

+/*
+ * Detect whether a request should be run out-of-band, by quickly
+ * peeking at whether we have: { "control": { "run-oob": true } }. By
+ * default commands are run in-band.
+ */
+bool qmp_is_oob(QDict *dict)
+{
+    QBool *bool_obj;
+
+    dict = qdict_get_qdict(dict, "control");
+    if (!dict) {
+        return false;
+    }
+
+    bool_obj = qobject_to(QBool, qdict_get(dict, "run-oob"));
+    if (!bool_obj) {
+        return false;
+    }
+
+    return qbool_get_bool(bool_obj);
+}
+
 QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request)
 {
     Error *err = NULL;
diff --git a/trace-events b/trace-events
index fe10c3b487c..bd036da278d 100644
--- a/trace-events
+++ b/trace-events
@@ -48,6 +48,8 @@ monitor_protocol_event_queue(uint32_t event, void *qdict, uint64_t rate) "event=
 handle_hmp_command(void *mon, const char *cmdline) "mon %p cmdline: %s"
 handle_qmp_command(void *mon, const char *req) "mon %p req: %s"
 monitor_suspend(void *ptr, int cnt) "mon %p: %d"
+monitor_qmp_cmd_in_band(const char *id) "%s"
+monitor_qmp_cmd_out_of_band(const char *id) "%s"

 # dma-helpers.c
 dma_blk_io(void *dbs, void *bs, int64_t offset, bool to_dev) "dbs=%p bs=%p offset=%" PRId64 " to_dev=%d"
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 30/36] qmp: isolate responses into io thread
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (28 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 29/36] qmp: support out-of-band (oob) execution Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed Eric Blake
                   ` (6 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster, Dr. David Alan Gilbert

From: Peter Xu <peterx@redhat.com>

For those monitors who have enabled IO thread, we'll offload the
responding procedure into IO thread.  The main reason is that chardev is
not thread safe, and we need to do all the read/write IOs in the same
thread.  For use_io_thr=true monitors, that thread is the IO thread.

We do this isolation in similar pattern as what we have done to the
request queue: we first create one response queue for each monitor, then
instead of replying directly in the main thread, we queue the responses
and kick the IO thread to do the rest of the job for us.

A funny thing after doing this is that, when the QMP clients send "quit"
to QEMU, it's possible that we close the IOThread even earlier than
replying to that "quit".  So another thing we need to do before cleaning
up the monitors is that we need to flush the response queue (we don't
need to do that for command queue; after all we are quitting) to make
sure replies for handled commands are always flushed back to clients.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-20-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 95 insertions(+), 1 deletion(-)

diff --git a/monitor.c b/monitor.c
index 3b7a4a5c3a7..905d5efc6fa 100644
--- a/monitor.c
+++ b/monitor.c
@@ -179,6 +179,8 @@ typedef struct {
     QemuMutex qmp_queue_lock;
     /* Input queue that holds all the parsed QMP requests */
     GQueue *qmp_requests;
+    /* Output queue contains all the QMP responses in order */
+    GQueue *qmp_responses;
 } MonitorQMP;

 /*
@@ -205,6 +207,7 @@ struct Monitor {
     bool skip_flush;
     bool use_io_thr;

+    /* We can't access guest memory when holding the lock */
     QemuMutex out_lock;
     QString *outbuf;
     guint out_watch;
@@ -227,6 +230,8 @@ static struct {
     IOThread *mon_iothread;
     /* Bottom half to dispatch the requests received from IO thread */
     QEMUBH *qmp_dispatcher_bh;
+    /* Bottom half to deliver the responses back to clients */
+    QEMUBH *qmp_respond_bh;
 } mon_global;

 /* QMP checker flags */
@@ -416,7 +421,8 @@ int monitor_fprintf(FILE *stream, const char *fmt, ...)
     return 0;
 }

-static void monitor_json_emitter(Monitor *mon, const QObject *data)
+static void monitor_json_emitter_raw(Monitor *mon,
+                                     QObject *data)
 {
     QString *json;

@@ -430,6 +436,71 @@ static void monitor_json_emitter(Monitor *mon, const QObject *data)
     QDECREF(json);
 }

+static void monitor_json_emitter(Monitor *mon, QObject *data)
+{
+    if (mon->use_io_thr) {
+        /*
+         * If using IO thread, we need to queue the item so that IO
+         * thread will do the rest for us.  Take refcount so that
+         * caller won't free the data (which will be finally freed in
+         * responder thread).
+         */
+        qobject_incref(data);
+        qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
+        g_queue_push_tail(mon->qmp.qmp_responses, (void *)data);
+        qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+        qemu_bh_schedule(mon_global.qmp_respond_bh);
+    } else {
+        /*
+         * If not using monitor IO thread, then we are in main thread.
+         * Do the emission right away.
+         */
+        monitor_json_emitter_raw(mon, data);
+    }
+}
+
+struct QMPResponse {
+    Monitor *mon;
+    QObject *data;
+};
+typedef struct QMPResponse QMPResponse;
+
+/*
+ * Return one QMPResponse.  The response is only valid if
+ * response.data is not NULL.
+ */
+static QMPResponse monitor_qmp_response_pop_one(void)
+{
+    Monitor *mon;
+    QObject *data = NULL;
+
+    qemu_mutex_lock(&monitor_lock);
+    QTAILQ_FOREACH(mon, &mon_list, entry) {
+        qemu_mutex_lock(&mon->qmp.qmp_queue_lock);
+        data = g_queue_pop_head(mon->qmp.qmp_responses);
+        qemu_mutex_unlock(&mon->qmp.qmp_queue_lock);
+        if (data) {
+            break;
+        }
+    }
+    qemu_mutex_unlock(&monitor_lock);
+    return (QMPResponse) { .mon = mon, .data = data };
+}
+
+static void monitor_qmp_bh_responder(void *opaque)
+{
+    QMPResponse response;
+
+    while (true) {
+        response = monitor_qmp_response_pop_one();
+        if (!response.data) {
+            break;
+        }
+        monitor_json_emitter_raw(response.mon, response.data);
+        qobject_decref(response.data);
+    }
+}
+
 static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX] = {
     /* Limit guest-triggerable events to 1 per second */
     [QAPI_EVENT_RTC_CHANGE]        = { 1000 * SCALE_MS },
@@ -616,6 +687,7 @@ static void monitor_data_init(Monitor *mon, bool skip_flush,
     mon->skip_flush = skip_flush;
     mon->use_io_thr = use_io_thr;
     mon->qmp.qmp_requests = g_queue_new();
+    mon->qmp.qmp_responses = g_queue_new();
 }

 static void monitor_data_destroy(Monitor *mon)
@@ -630,6 +702,7 @@ static void monitor_data_destroy(Monitor *mon)
     qemu_mutex_destroy(&mon->out_lock);
     qemu_mutex_destroy(&mon->qmp.qmp_queue_lock);
     g_queue_free(mon->qmp.qmp_requests);
+    g_queue_free(mon->qmp.qmp_responses);
 }

 char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
@@ -4367,6 +4440,15 @@ static void monitor_iothread_init(void)
     mon_global.qmp_dispatcher_bh = aio_bh_new(qemu_get_aio_context(),
                                               monitor_qmp_bh_dispatcher,
                                               NULL);
+
+    /*
+     * Unlike the dispatcher BH, this must be run on the monitor IO
+     * thread, so that monitors that are using IO thread will make
+     * sure read/write operations are all done on the IO thread.
+     */
+    mon_global.qmp_respond_bh = aio_bh_new(monitor_get_aio_context(),
+                                           monitor_qmp_bh_responder,
+                                           NULL);
 }

 void monitor_init_globals(void)
@@ -4505,9 +4587,19 @@ void monitor_cleanup(void)
      */
     iothread_stop(mon_global.mon_iothread);

+    /*
+     * After we have IOThread to send responses, it's possible that
+     * when we stop the IOThread there are still replies queued in the
+     * responder queue.  Flush all of them.  Note that even after this
+     * flush it's still possible that out buffer is not flushed.
+     * It'll be done in below monitor_flush() as the last resort.
+     */
+    monitor_qmp_bh_responder(NULL);
+
     qemu_mutex_lock(&monitor_lock);
     QTAILQ_FOREACH_SAFE(mon, &mon_list, entry, next) {
         QTAILQ_REMOVE(&mon_list, mon, entry);
+        monitor_flush(mon);
         monitor_data_destroy(mon);
         g_free(mon);
     }
@@ -4516,6 +4608,8 @@ void monitor_cleanup(void)
     /* QEMUBHs needs to be deleted before destroying the IOThread. */
     qemu_bh_delete(mon_global.qmp_dispatcher_bh);
     mon_global.qmp_dispatcher_bh = NULL;
+    qemu_bh_delete(mon_global.qmp_respond_bh);
+    mon_global.qmp_respond_bh = NULL;

     iothread_destroy(mon_global.mon_iothread);
     mon_global.mon_iothread = NULL;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (29 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 30/36] qmp: isolate responses into io thread Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-21 12:37   ` Max Reitz
  2018-03-12 18:36 ` [Qemu-devel] [PULL 32/36] qmp: add command "x-oob-test" Eric Blake
                   ` (5 subsequent siblings)
  36 siblings, 1 reply; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Dr. David Alan Gilbert, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

Start to use dedicate IO thread for QMP monitors that are not using
MUXed chardev.

Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-21-peterx@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/monitor.c b/monitor.c
index 905d5efc6fa..dea4111e693 100644
--- a/monitor.c
+++ b/monitor.c
@@ -36,6 +36,7 @@
 #include "net/slirp.h"
 #include "chardev/char-fe.h"
 #include "chardev/char-io.h"
+#include "chardev/char-mux.h"
 #include "ui/qemu-spice.h"
 #include "sysemu/numa.h"
 #include "monitor/monitor.h"
@@ -4533,8 +4534,10 @@ static void monitor_qmp_setup_handlers_bh(void *opaque)
 void monitor_init(Chardev *chr, int flags)
 {
     Monitor *mon = g_malloc(sizeof(*mon));
+    /* Enable IOThread for QMPs that are not using MUX chardev backends. */
+    bool use_io_thr = (!CHARDEV_IS_MUX(chr)) && (flags & MONITOR_USE_CONTROL);

-    monitor_data_init(mon, false, false);
+    monitor_data_init(mon, false, use_io_thr);

     qemu_chr_fe_init(&mon->chr, chr, &error_abort);
     mon->flags = flags;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 32/36] qmp: add command "x-oob-test"
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (30 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 33/36] tests: qmp-test: verify command batching Eric Blake
                   ` (4 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

This command is only used to test OOB functionality.  It should not be
used for any other purposes.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-22-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweak]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/misc.json | 18 ++++++++++++++++++
 qmp.c          | 16 ++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/qapi/misc.json b/qapi/misc.json
index b6ebf210ba8..fc3550f3acc 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -3279,3 +3279,21 @@
 ##
 { 'event': 'COMMAND_DROPPED' ,
   'data': { 'id': 'any', 'reason': 'CommandDropReason' } }
+
+##
+# @x-oob-test:
+#
+# Test OOB functionality.  When sending this command with lock=true,
+# it'll try to hang the dispatcher.  When sending it with lock=false,
+# it'll try to notify the locked thread to continue.  Note: it should
+# only be used by QMP test program rather than anything else.
+#
+# Since: 2.12
+#
+# Example:
+#
+# { "execute": "x-oob-test",
+#   "arguments": { "lock": true } }
+##
+{ 'command': 'x-oob-test', 'data' : { 'lock': 'bool' },
+  'allow-oob': true }
diff --git a/qmp.c b/qmp.c
index 7e606d84867..ea3760acb12 100644
--- a/qmp.c
+++ b/qmp.c
@@ -770,3 +770,19 @@ MemoryInfo *qmp_query_memory_size_summary(Error **errp)

     return mem_info;
 }
+
+static QemuSemaphore x_oob_test_sem;
+
+static void __attribute__((constructor)) x_oob_test_init(void)
+{
+    qemu_sem_init(&x_oob_test_sem, 0);
+}
+
+void qmp_x_oob_test(bool lock, Error **errp)
+{
+    if (lock) {
+        qemu_sem_wait(&x_oob_test_sem);
+    } else {
+        qemu_sem_post(&x_oob_test_sem);
+    }
+}
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 33/36] tests: qmp-test: verify command batching
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (31 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 32/36] qmp: add command "x-oob-test" Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 34/36] tests: qmp-test: add oob test Eric Blake
                   ` (3 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

OOB introduced DROP event for flow control.  This should not affect old
QMP clients.  Add a command batching check to make sure of it.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-23-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qmp-test.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index a8384bdda3b..fd4033a13fd 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -82,6 +82,7 @@ static void test_qmp_protocol(void)
     QTestState *qts;
     const QListEntry *entry;
     QString *qstr;
+    int i;

     qts = qtest_init_without_qmp_handshake(common_args);

@@ -139,6 +140,27 @@ static void test_qmp_protocol(void)
     g_assert_cmpint(qdict_get_int(resp, "id"), ==, 2);
     QDECREF(resp);

+    /*
+     * Test command batching.  In current test OOB is not enabled, we
+     * should be able to run as many commands in batch as we like.
+     * Using 16 (>8, which is OOB queue length) to make sure OOB won't
+     * break existing clients.  Note: this test does not control the
+     * scheduling of QEMU's QMP command processing threads so it may
+     * not really trigger batching inside QEMU.  This is just a
+     * best-effort test.
+     */
+    for (i = 0; i < 16; i++) {
+        qtest_async_qmp(qts, "{ 'execute': 'query-version' }");
+    }
+    /* Verify the replies to make sure no command is dropped. */
+    for (i = 0; i < 16; i++) {
+        resp = qtest_qmp_receive(qts);
+        /* It should never be dropped.  Each of them should be a reply. */
+        g_assert(qdict_haskey(resp, "return"));
+        g_assert(!qdict_haskey(resp, "event"));
+        QDECREF(resp);
+    }
+
     qtest_quit(qts);
 }

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 34/36] tests: qmp-test: add oob test
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (32 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 33/36] tests: qmp-test: verify command batching Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 35/36] block/accounting: introduce latency histogram Eric Blake
                   ` (2 subsequent siblings)
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Xu, Markus Armbruster

From: Peter Xu <peterx@redhat.com>

Test the new OOB capability.  Here we used the new "x-oob-test" command.
First, we send a lock=true and oob=false command to hang the main
thread.  Then send another lock=false and oob=true command (which will
be run inside parser this time) to free that hanged command.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-24-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qmp-test.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/tests/qmp-test.c b/tests/qmp-test.c
index fd4033a13fd..17cf4ccf4c9 100644
--- a/tests/qmp-test.c
+++ b/tests/qmp-test.c
@@ -164,6 +164,70 @@ static void test_qmp_protocol(void)
     qtest_quit(qts);
 }

+/* Tests for Out-Of-Band support. */
+static void test_qmp_oob(void)
+{
+    QDict *resp;
+    int acks = 0;
+    const char *cmd_id;
+
+    global_qtest = qtest_init_without_qmp_handshake(common_args);
+
+    /* Ignore the greeting message. */
+    resp = qmp_receive();
+    g_assert(qdict_get_qdict(resp, "QMP"));
+    QDECREF(resp);
+
+    /* Try a fake capability, it should fail. */
+    resp = qmp("{ 'execute': 'qmp_capabilities', "
+               "  'arguments': { 'enable': [ 'cap-does-not-exist' ] } }");
+    g_assert(qdict_haskey(resp, "error"));
+    QDECREF(resp);
+
+    /* Now, enable OOB in current QMP session, it should succeed. */
+    resp = qmp("{ 'execute': 'qmp_capabilities', "
+               "  'arguments': { 'enable': [ 'oob' ] } }");
+    g_assert(qdict_haskey(resp, "return"));
+    QDECREF(resp);
+
+    /*
+     * Try any command that does not support OOB but with OOB flag. We
+     * should get failure.
+     */
+    resp = qmp("{ 'execute': 'query-cpus',"
+               "  'control': { 'run-oob': true } }");
+    g_assert(qdict_haskey(resp, "error"));
+    QDECREF(resp);
+
+    /*
+     * First send the "x-oob-test" command with lock=true and
+     * oob=false, it should hang the dispatcher and main thread;
+     * later, we send another lock=false with oob=true to continue
+     * that thread processing.  Finally we should receive replies from
+     * both commands.
+     */
+    qmp_async("{ 'execute': 'x-oob-test',"
+              "  'arguments': { 'lock': true }, "
+              "  'id': 'lock-cmd'}");
+    qmp_async("{ 'execute': 'x-oob-test', "
+              "  'arguments': { 'lock': false }, "
+              "  'control': { 'run-oob': true }, "
+              "  'id': 'unlock-cmd' }");
+
+    /* Ignore all events.  Wait for 2 acks */
+    while (acks < 2) {
+        resp = qmp_receive();
+        cmd_id = qdict_get_str(resp, "id");
+        if (!g_strcmp0(cmd_id, "lock-cmd") ||
+            !g_strcmp0(cmd_id, "unlock-cmd")) {
+            acks++;
+        }
+        QDECREF(resp);
+    }
+
+    qtest_end();
+}
+
 static int query_error_class(const char *cmd)
 {
     static struct {
@@ -343,6 +407,7 @@ int main(int argc, char *argv[])
     g_test_init(&argc, &argv, NULL);

     qtest_add_func("qmp/protocol", test_qmp_protocol);
+    qtest_add_func("qmp/oob", test_qmp_oob);
     qmp_schema_init(&schema);
     add_query_tests(&schema);

-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 35/36] block/accounting: introduce latency histogram
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (33 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 34/36] tests: qmp-test: add oob test Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-12 18:36 ` [Qemu-devel] [PULL 36/36] qapi: add block latency histogram interface Eric Blake
  2018-03-13 14:02 ` [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Peter Maydell
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Vladimir Sementsov-Ogievskiy, Kevin Wolf, Max Reitz,
	open list:Block layer core

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Introduce latency histogram statics for block devices.
For each accounted operation type, the latency region [0, +inf) is
divided into subregions by several points. Then, calculate
hits for each subregion.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180309165212.97144-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 include/block/accounting.h | 35 ++++++++++++++++++
 block/accounting.c         | 91 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 126 insertions(+)

diff --git a/include/block/accounting.h b/include/block/accounting.h
index b833d26d6c7..d1f67b10ddf 100644
--- a/include/block/accounting.h
+++ b/include/block/accounting.h
@@ -27,6 +27,7 @@

 #include "qemu/timed-average.h"
 #include "qemu/thread.h"
+#include "qapi/qapi-builtin-types.h"

 typedef struct BlockAcctTimedStats BlockAcctTimedStats;
 typedef struct BlockAcctStats BlockAcctStats;
@@ -45,6 +46,36 @@ struct BlockAcctTimedStats {
     QSLIST_ENTRY(BlockAcctTimedStats) entries;
 };

+typedef struct BlockLatencyHistogram {
+    /* The following histogram is represented like this:
+     *
+     * 5|           *
+     * 4|           *
+     * 3| *         *
+     * 2| *         *    *
+     * 1| *    *    *    *
+     *  +------------------
+     *      10   50   100
+     *
+     * BlockLatencyHistogram histogram = {
+     *     .nbins = 4,
+     *     .boundaries = {10, 50, 100},
+     *     .bins = {3, 1, 5, 2},
+     * };
+     *
+     * @boundaries array define histogram intervals as follows:
+     * [0, boundaries[0]), [boundaries[0], boundaries[1]), ...
+     * [boundaries[nbins-2], +inf)
+     *
+     * So, for example above, histogram intervals are:
+     * [0, 10), [10, 50), [50, 100), [100, +inf)
+     */
+    int nbins;
+    uint64_t *boundaries; /* @nbins-1 numbers here
+                             (all boundaries, except 0 and +inf) */
+    uint64_t *bins;
+} BlockLatencyHistogram;
+
 struct BlockAcctStats {
     QemuMutex lock;
     uint64_t nr_bytes[BLOCK_MAX_IOTYPE];
@@ -57,6 +88,7 @@ struct BlockAcctStats {
     QSLIST_HEAD(, BlockAcctTimedStats) intervals;
     bool account_invalid;
     bool account_failed;
+    BlockLatencyHistogram latency_histogram[BLOCK_MAX_IOTYPE];
 };

 typedef struct BlockAcctCookie {
@@ -82,5 +114,8 @@ void block_acct_merge_done(BlockAcctStats *stats, enum BlockAcctType type,
 int64_t block_acct_idle_time_ns(BlockAcctStats *stats);
 double block_acct_queue_depth(BlockAcctTimedStats *stats,
                               enum BlockAcctType type);
+int block_latency_histogram_set(BlockAcctStats *stats, enum BlockAcctType type,
+                                uint64List *boundaries);
+void block_latency_histograms_clear(BlockAcctStats *stats);

 #endif
diff --git a/block/accounting.c b/block/accounting.c
index 87ef5bbfaa0..70a3d9a426c 100644
--- a/block/accounting.c
+++ b/block/accounting.c
@@ -94,6 +94,94 @@ void block_acct_start(BlockAcctStats *stats, BlockAcctCookie *cookie,
     cookie->type = type;
 }

+/* block_latency_histogram_compare_func:
+ * Compare @key with interval [@it[0], @it[1]).
+ * Return: -1 if @key < @it[0]
+ *          0 if @key in [@it[0], @it[1])
+ *         +1 if @key >= @it[1]
+ */
+static int block_latency_histogram_compare_func(const void *key, const void *it)
+{
+    uint64_t k = *(uint64_t *)key;
+    uint64_t a = ((uint64_t *)it)[0];
+    uint64_t b = ((uint64_t *)it)[1];
+
+    return k < a ? -1 : (k < b ? 0 : 1);
+}
+
+static void block_latency_histogram_account(BlockLatencyHistogram *hist,
+                                            int64_t latency_ns)
+{
+    uint64_t *pos;
+
+    if (hist->bins == NULL) {
+        /* histogram disabled */
+        return;
+    }
+
+
+    if (latency_ns < hist->boundaries[0]) {
+        hist->bins[0]++;
+        return;
+    }
+
+    if (latency_ns >= hist->boundaries[hist->nbins - 2]) {
+        hist->bins[hist->nbins - 1]++;
+        return;
+    }
+
+    pos = bsearch(&latency_ns, hist->boundaries, hist->nbins - 2,
+                  sizeof(hist->boundaries[0]),
+                  block_latency_histogram_compare_func);
+    assert(pos != NULL);
+
+    hist->bins[pos - hist->boundaries + 1]++;
+}
+
+int block_latency_histogram_set(BlockAcctStats *stats, enum BlockAcctType type,
+                                uint64List *boundaries)
+{
+    BlockLatencyHistogram *hist = &stats->latency_histogram[type];
+    uint64List *entry;
+    uint64_t *ptr;
+    uint64_t prev = 0;
+    int new_nbins = 1;
+
+    for (entry = boundaries; entry; entry = entry->next) {
+        if (entry->value <= prev) {
+            return -EINVAL;
+        }
+        new_nbins++;
+        prev = entry->value;
+    }
+
+    hist->nbins = new_nbins;
+    g_free(hist->boundaries);
+    hist->boundaries = g_new(uint64_t, hist->nbins - 1);
+    for (entry = boundaries, ptr = hist->boundaries; entry;
+         entry = entry->next, ptr++)
+    {
+        *ptr = entry->value;
+    }
+
+    g_free(hist->bins);
+    hist->bins = g_new0(uint64_t, hist->nbins);
+
+    return 0;
+}
+
+void block_latency_histograms_clear(BlockAcctStats *stats)
+{
+    int i;
+
+    for (i = 0; i < BLOCK_MAX_IOTYPE; i++) {
+        BlockLatencyHistogram *hist = &stats->latency_histogram[i];
+        g_free(hist->bins);
+        g_free(hist->boundaries);
+        memset(hist, 0, sizeof(*hist));
+    }
+}
+
 static void block_account_one_io(BlockAcctStats *stats, BlockAcctCookie *cookie,
                                  bool failed)
 {
@@ -116,6 +204,9 @@ static void block_account_one_io(BlockAcctStats *stats, BlockAcctCookie *cookie,
         stats->nr_ops[cookie->type]++;
     }

+    block_latency_histogram_account(&stats->latency_histogram[cookie->type],
+                                    latency_ns);
+
     if (!failed || stats->account_failed) {
         stats->total_time_ns[cookie->type] += latency_ns;
         stats->last_access_time_ns = time_ns;
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* [Qemu-devel] [PULL 36/36] qapi: add block latency histogram interface
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (34 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 35/36] block/accounting: introduce latency histogram Eric Blake
@ 2018-03-12 18:36 ` Eric Blake
  2018-03-13 14:02 ` [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Peter Maydell
  36 siblings, 0 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-12 18:36 UTC (permalink / raw)
  To: qemu-devel
  Cc: Vladimir Sementsov-Ogievskiy, Markus Armbruster, Kevin Wolf,
	Max Reitz, open list:Block layer core

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Set (and clear) histograms through new command
block-latency-histogram-set and show new statistics in
query-blockstats results.

For now, the command is marked experimental with prefix 'x-',
to gain experience with the interface without being stuck
with design decisions.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180309165212.97144-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[eblake: fix typos, mention x- prefix in commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/block-core.json | 111 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 block/qapi.c         |  41 +++++++++++++++++++
 blockdev.c           |  43 ++++++++++++++++++++
 3 files changed, 194 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 55dcc8bd85a..233c8719596 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -450,6 +450,106 @@
   'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
            'status': 'DirtyBitmapStatus'} }

+##
+# @BlockLatencyHistogramInfo:
+#
+# Block latency histogram.
+#
+# @boundaries: list of interval boundary values in nanoseconds, all greater
+#              than zero and in ascending order.
+#              For example, the list [10, 50, 100] produces the following
+#              histogram intervals: [0, 10), [10, 50), [50, 100), [100, +inf).
+#
+# @bins: list of io request counts corresponding to histogram intervals.
+#        len(@bins) = len(@boundaries) + 1
+#        For the example above, @bins may be something like [3, 1, 5, 2],
+#        and corresponding histogram looks like:
+#
+#        5|           *
+#        4|           *
+#        3| *         *
+#        2| *         *    *
+#        1| *    *    *    *
+#         +------------------
+#             10   50   100
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockLatencyHistogramInfo',
+  'data': {'boundaries': ['uint64'], 'bins': ['uint64'] } }
+
+##
+# @x-block-latency-histogram-set:
+#
+# Manage read, write and flush latency histograms for the device.
+#
+# If only @device parameter is specified, remove all present latency histograms
+# for the device. Otherwise, add/reset some of (or all) latency histograms.
+#
+# @device: device name to set latency histogram for.
+#
+# @boundaries: list of interval boundary values (see description in
+#              BlockLatencyHistogramInfo definition). If specified, all
+#              latency histograms are removed, and empty ones created for all
+#              io types with intervals corresponding to @boundaries (except for
+#              io types, for which specific boundaries are set through the
+#              following parameters).
+#
+# @boundaries-read: list of interval boundary values for read latency
+#                   histogram. If specified, old read latency histogram is
+#                   removed, and empty one created with intervals
+#                   corresponding to @boundaries-read. The parameter has higher
+#                   priority then @boundaries.
+#
+# @boundaries-write: list of interval boundary values for write latency
+#                    histogram.
+#
+# @boundaries-flush: list of interval boundary values for flush latency
+#                    histogram.
+#
+# Returns: error if device is not found or any boundary arrays are invalid.
+#
+# Since: 2.12
+#
+# Example: set new histograms for all io types with intervals
+# [0, 10), [10, 50), [50, 100), [100, +inf):
+#
+# -> { "execute": "block-latency-histogram-set",
+#      "arguments": { "device": "drive0",
+#                     "boundaries": [10, 50, 100] } }
+# <- { "return": {} }
+#
+# Example: set new histogram only for write, other histograms will remain
+# not changed (or not created):
+#
+# -> { "execute": "block-latency-histogram-set",
+#      "arguments": { "device": "drive0",
+#                     "boundaries-write": [10, 50, 100] } }
+# <- { "return": {} }
+#
+# Example: set new histograms with the following intervals:
+#   read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
+#   write: [0, 1000), [1000, 5000), [5000, +inf)
+#
+# -> { "execute": "block-latency-histogram-set",
+#      "arguments": { "device": "drive0",
+#                     "boundaries": [10, 50, 100],
+#                     "boundaries-write": [1000, 5000] } }
+# <- { "return": {} }
+#
+# Example: remove all latency histograms:
+#
+# -> { "execute": "block-latency-histogram-set",
+#      "arguments": { "device": "drive0" } }
+# <- { "return": {} }
+##
+{ 'command': 'x-block-latency-histogram-set',
+  'data': {'device': 'str',
+           '*boundaries': ['uint64'],
+           '*boundaries-read': ['uint64'],
+           '*boundaries-write': ['uint64'],
+           '*boundaries-flush': ['uint64'] } }
+
 ##
 # @BlockInfo:
 #
@@ -730,6 +830,12 @@
 # @timed_stats: Statistics specific to the set of previously defined
 #               intervals of time (Since 2.5)
 #
+# @x_rd_latency_histogram: @BlockLatencyHistogramInfo. (Since 2.12)
+#
+# @x_wr_latency_histogram: @BlockLatencyHistogramInfo. (Since 2.12)
+#
+# @x_flush_latency_histogram: @BlockLatencyHistogramInfo. (Since 2.12)
+#
 # Since: 0.14.0
 ##
 { 'struct': 'BlockDeviceStats',
@@ -742,7 +848,10 @@
            'failed_flush_operations': 'int', 'invalid_rd_operations': 'int',
            'invalid_wr_operations': 'int', 'invalid_flush_operations': 'int',
            'account_invalid': 'bool', 'account_failed': 'bool',
-           'timed_stats': ['BlockDeviceTimedStats'] } }
+           'timed_stats': ['BlockDeviceTimedStats'],
+           '*x_rd_latency_histogram': 'BlockLatencyHistogramInfo',
+           '*x_wr_latency_histogram': 'BlockLatencyHistogramInfo',
+           '*x_flush_latency_histogram': 'BlockLatencyHistogramInfo' } }

 ##
 # @BlockStats:
diff --git a/block/qapi.c b/block/qapi.c
index f2e0aa2cbe7..04c6fc69b92 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -394,6 +394,37 @@ static void bdrv_query_info(BlockBackend *blk, BlockInfo **p_info,
     qapi_free_BlockInfo(info);
 }

+static uint64List *uint64_list(uint64_t *list, int size)
+{
+    int i;
+    uint64List *out_list = NULL;
+    uint64List **pout_list = &out_list;
+
+    for (i = 0; i < size; i++) {
+        uint64List *entry = g_new(uint64List, 1);
+        entry->value = list[i];
+        *pout_list = entry;
+        pout_list = &entry->next;
+    }
+
+    *pout_list = NULL;
+
+    return out_list;
+}
+
+static void bdrv_latency_histogram_stats(BlockLatencyHistogram *hist,
+                                         bool *not_null,
+                                         BlockLatencyHistogramInfo **info)
+{
+    *not_null = hist->bins != NULL;
+    if (*not_null) {
+        *info = g_new0(BlockLatencyHistogramInfo, 1);
+
+        (*info)->boundaries = uint64_list(hist->boundaries, hist->nbins - 1);
+        (*info)->bins = uint64_list(hist->bins, hist->nbins);
+    }
+}
+
 static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
 {
     BlockAcctStats *stats = blk_get_stats(blk);
@@ -459,6 +490,16 @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
         dev_stats->avg_wr_queue_depth =
             block_acct_queue_depth(ts, BLOCK_ACCT_WRITE);
     }
+
+    bdrv_latency_histogram_stats(&stats->latency_histogram[BLOCK_ACCT_READ],
+                                 &ds->has_x_rd_latency_histogram,
+                                 &ds->x_rd_latency_histogram);
+    bdrv_latency_histogram_stats(&stats->latency_histogram[BLOCK_ACCT_WRITE],
+                                 &ds->has_x_wr_latency_histogram,
+                                 &ds->x_wr_latency_histogram);
+    bdrv_latency_histogram_stats(&stats->latency_histogram[BLOCK_ACCT_FLUSH],
+                                 &ds->has_x_flush_latency_histogram,
+                                 &ds->x_flush_latency_histogram);
 }

 static BlockStats *bdrv_query_bds_stats(BlockDriverState *bs,
diff --git a/blockdev.c b/blockdev.c
index 6a751996022..69cdcc8d59d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -4167,6 +4167,49 @@ void qmp_x_blockdev_set_iothread(const char *node_name, StrOrNull *iothread,
     aio_context_release(old_context);
 }

+void qmp_x_block_latency_histogram_set(
+    const char *device,
+    bool has_boundaries, uint64List *boundaries,
+    bool has_boundaries_read, uint64List *boundaries_read,
+    bool has_boundaries_write, uint64List *boundaries_write,
+    bool has_boundaries_flush, uint64List *boundaries_flush,
+    Error **errp)
+{
+    BlockBackend *blk = blk_by_name(device);
+    BlockAcctStats *stats;
+
+    if (!blk) {
+        error_setg(errp, "Device '%s' not found", device);
+        return;
+    }
+    stats = blk_get_stats(blk);
+
+    if (!has_boundaries && !has_boundaries_read && !has_boundaries_write &&
+        !has_boundaries_flush)
+    {
+        block_latency_histograms_clear(stats);
+        return;
+    }
+
+    if (has_boundaries || has_boundaries_read) {
+        block_latency_histogram_set(
+            stats, BLOCK_ACCT_READ,
+            has_boundaries_read ? boundaries_read : boundaries);
+    }
+
+    if (has_boundaries || has_boundaries_write) {
+        block_latency_histogram_set(
+            stats, BLOCK_ACCT_WRITE,
+            has_boundaries_write ? boundaries_write : boundaries);
+    }
+
+    if (has_boundaries || has_boundaries_flush) {
+        block_latency_histogram_set(
+            stats, BLOCK_ACCT_FLUSH,
+            has_boundaries_flush ? boundaries_flush : boundaries);
+    }
+}
+
 QemuOptsList qemu_common_drive_opts = {
     .name = "drive",
     .head = QTAILQ_HEAD_INITIALIZER(qemu_common_drive_opts.head),
-- 
2.14.3

^ permalink raw reply related	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
  2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
                   ` (35 preceding siblings ...)
  2018-03-12 18:36 ` [Qemu-devel] [PULL 36/36] qapi: add block latency histogram interface Eric Blake
@ 2018-03-13 14:02 ` Peter Maydell
  2018-03-13 14:17   ` Eric Blake
  36 siblings, 1 reply; 50+ messages in thread
From: Peter Maydell @ 2018-03-13 14:02 UTC (permalink / raw)
  To: Eric Blake; +Cc: QEMU Developers

On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
> The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
>
> for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
>
>   qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
>
> This builds and passes 'make check', so even though the OOB portion
> depends on chardev fixes that are still pending a pull request from
> Paolo, that dependence can only be observed at runtime by clients
> that use the new oob feature.  Given the timing of soft freeze, and
> the fact that the chardev fixes do not form a build dependency, I
> think it's okay if this pull request gets processed before Paolo's
> (but it's also okay if Paolo's goes in first).
>
> ----------------------------------------------------------------
> qapi patches for 2018-03-12, 2.12 softfreeze
>
> - Marc-André Lureau: 0/4 qapi: generate a literal qobject for introspection
> - Max Reitz: 0/7 block: Handle null backing link
> - Peter Xu: 00/23 QMP: out-of-band (OOB) execution support
> - Vladimir Sementsov-Ogievskiy: 0/2 block latency histogram
>

Hi; I'm afraid this failed some of my build tests:

x86/Linux and x86 OpenBSD, compile failure (probably gcc-version-dependent):

/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c: In
function ‘build_append_pci_bus_devices’:
/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:617:9:
error: ‘notify_method’ may be used uninitialized in this function
[-Werror=maybe-uninitialized]
         aml_append(parent_scope, notify_method);
         ^
/home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:510:16:
note: ‘notify_method’ was declared here
     Aml *dev, *notify_method, *method;
                ^
cc1: all warnings being treated as errors


on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
looks like the same assert but in different tests:

ppc64:
TEST: tests/qmp-test... (pid=48041)
  /alpha/qmp/protocol:                                                 OK
  /alpha/qmp/oob:                                                      OK
  /alpha/qmp/query-status:                                             OK
  /alpha/qmp/query-block:
qemu-system-alpha: /home/pm215/qemu/
chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
((void *)0)' failed.
Broken pipe
FAIL
GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
(pid=48057)

aarch64:
  /aarch64/qmp/query-qmp-schema:                                       OK
  /aarch64/qmp/query-version:                                          OK
  /aarch64/qmp/query-commands:
Assertion failed: (iwp->src == NULL)
, function io_watch_poll_finalize, file /root/qemu/chardev/char-io.c, line 91.
Broken pipe
FAIL
GTester: last random seed: R02Sbd982a1e1da0c16d183f7725739b58af
(pid=58277)

and the others are the same but in
/mips64/device/introspect/abstract-interfaces
/alpha/qmp/query-named-block-nodes

thanks
-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
  2018-03-13 14:02 ` [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Peter Maydell
@ 2018-03-13 14:17   ` Eric Blake
  2018-03-13 15:21     ` Peter Xu
  2018-03-13 21:55     ` Eric Blake
  0 siblings, 2 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-13 14:17 UTC (permalink / raw)
  To: Peter Maydell; +Cc: QEMU Developers, Paolo Bonzini, Peter Xu

On 03/13/2018 09:02 AM, Peter Maydell wrote:
> On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
>> The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
>>
>>    Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
>>
>> are available in the Git repository at:
>>
>>    git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
>>
>> for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
>>
>>    qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
>>
>> This builds and passes 'make check', so even though the OOB portion
>> depends on chardev fixes that are still pending a pull request from
>> Paolo, that dependence can only be observed at runtime by clients
>> that use the new oob feature.  Given the timing of soft freeze, and
>> the fact that the chardev fixes do not form a build dependency, I
>> think it's okay if this pull request gets processed before Paolo's
>> (but it's also okay if Paolo's goes in first).

Based on the testsuite failures, it looks like Paolo's pull request with 
chardev fixes DOES have to go in first. More at [1] below.

> x86/Linux and x86 OpenBSD, compile failure (probably gcc-version-dependent):
> 
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c: In
> function ‘build_append_pci_bus_devices’:
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:617:9:
> error: ‘notify_method’ may be used uninitialized in this function
> [-Werror=maybe-uninitialized]
>           aml_append(parent_scope, notify_method);
>           ^
> /home/petmay01/linaro/qemu-for-merges/hw/i386/acpi-build.c:510:16:
> note: ‘notify_method’ was declared here
>       Aml *dev, *notify_method, *method;
>                  ^
> cc1: all warnings being treated as errors

Odd - The only mention of notify_method in that file in my series is in 
the context:

$ git diff 6ceb1b51f..pull-qapi-2018-03-12 hw/i386/acpi-build.c \
    |grep notify_method
          notify_method = aml_method("DVNT", 2, AML_NOTSERIALIZED);

and where all the hunks look like:

@@ -154,21 +154,21 @@ static void acpi_get_pm_info(AcpiPmInfo *pm)
      /* Fill in optional s3/s4 related properties */
      o = object_property_get_qobject(obj, ACPI_PM_PROP_S3_DISABLED, NULL);
      if (o) {
-        pm->s3_disabled = qnum_get_uint(qobject_to_qnum(o));
+        pm->s3_disabled = qnum_get_uint(qobject_to(QNum, o));

which doesn't change control flow logic.  So all I can guess is that 
this has been a latent pre-existing problem, but the compiler is now 
flagging it.  At any rate, I can try and shut it up, and will spin a v2 
once the other issue is solved.


> 
> on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
> looks like the same assert but in different tests:
> 
> ppc64:
> TEST: tests/qmp-test... (pid=48041)
>    /alpha/qmp/protocol:                                                 OK
>    /alpha/qmp/oob:                                                      OK
>    /alpha/qmp/query-status:                                             OK
>    /alpha/qmp/query-block:
> qemu-system-alpha: /home/pm215/qemu/
> chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
> ((void *)0)' failed.
> Broken pipe
> FAIL
> GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
> (pid=48057)
> 

[1] this is probably the chardev fixes being tickled by oob.  (Weird 
that the change is not failing the oob test, though - or is the failure 
happening during cleanup of the oob test, AFTER it reported OK?)  Here's 
where I'm hoping Paolo's pull request with chardev fixes is the 
solution, otherwise, I may have to disable Peter's OOB patches.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
  2018-03-13 14:17   ` Eric Blake
@ 2018-03-13 15:21     ` Peter Xu
  2018-03-13 21:55     ` Eric Blake
  1 sibling, 0 replies; 50+ messages in thread
From: Peter Xu @ 2018-03-13 15:21 UTC (permalink / raw)
  To: Eric Blake; +Cc: Peter Maydell, QEMU Developers, Paolo Bonzini

On Tue, Mar 13, 2018 at 09:17:36AM -0500, Eric Blake wrote:
> On 03/13/2018 09:02 AM, Peter Maydell wrote:
> > On 12 March 2018 at 18:35, Eric Blake <eblake@redhat.com> wrote:
> > > The following changes since commit 6ceb1b51f05f9e1892d082960ed602dca7b6696e:
> > > 
> > >    Merge remote-tracking branch 'remotes/kraxel/tags/audio-20180312-pull-request' into staging (2018-03-12 16:14:37 +0000)
> > > 
> > > are available in the Git repository at:
> > > 
> > >    git://repo.or.cz/qemu/ericb.git tags/pull-qapi-2018-03-12
> > > 
> > > for you to fetch changes up to a083c533b5a17c77ef164acdbf30eedfa9681fc6:
> > > 
> > >    qapi: add block latency histogram interface (2018-03-12 13:22:11 -0500)
> > > 
> > > This builds and passes 'make check', so even though the OOB portion
> > > depends on chardev fixes that are still pending a pull request from
> > > Paolo, that dependence can only be observed at runtime by clients
> > > that use the new oob feature.  Given the timing of soft freeze, and
> > > the fact that the chardev fixes do not form a build dependency, I
> > > think it's okay if this pull request gets processed before Paolo's
> > > (but it's also okay if Paolo's goes in first).
> 
> Based on the testsuite failures, it looks like Paolo's pull request with
> chardev fixes DOES have to go in first. More at [1] below.

[...]

> > 
> > on PPC64 Linux, FreeBSD x86, OpenBSD x86, aarch64 Linux hosts, test fails;
> > looks like the same assert but in different tests:
> > 
> > ppc64:
> > TEST: tests/qmp-test... (pid=48041)
> >    /alpha/qmp/protocol:                                                 OK
> >    /alpha/qmp/oob:                                                      OK
> >    /alpha/qmp/query-status:                                             OK
> >    /alpha/qmp/query-block:
> > qemu-system-alpha: /home/pm215/qemu/
> > chardev/char-io.c:91: io_watch_poll_finalize: Assertion `iwp->src ==
> > ((void *)0)' failed.
> > Broken pipe
> > FAIL
> > GTester: last random seed: R02S3e793887202ca8b099adb20531a072e6
> > (pid=48057)
> > 
> 
> [1] this is probably the chardev fixes being tickled by oob.  (Weird that
> the change is not failing the oob test, though - or is the failure happening
> during cleanup of the oob test, AFTER it reported OK?)  Here's where I'm
> hoping Paolo's pull request with chardev fixes is the solution, otherwise, I
> may have to disable Peter's OOB patches.

Yes it is.  The failure can possibly happen randomly on very random
tests if without the whole bunch of chardev patches.

I confirmed with Paolo offlist that all the chardev fixes will be in
Paolo's next chardev pull request for the softfreeze.  So with Paolo's
next pull request, all the tests should pass, with 100%. If it still
fails any, then please feel free to drop the whole OOB series so that
I'll rework after 2.12.

Sorry again for the troublesome.

-- 
Peter Xu

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
  2018-03-13 14:17   ` Eric Blake
  2018-03-13 15:21     ` Peter Xu
@ 2018-03-13 21:55     ` Eric Blake
  2018-03-14 12:31       ` Peter Maydell
  1 sibling, 1 reply; 50+ messages in thread
From: Eric Blake @ 2018-03-13 21:55 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers, Peter Xu

On 03/13/2018 09:17 AM, Eric Blake wrote:
>>> This builds and passes 'make check', so even though the OOB portion
>>> depends on chardev fixes that are still pending a pull request from
>>> Paolo, that dependence can only be observed at runtime by clients
>>> that use the new oob feature.  Given the timing of soft freeze, and
>>> the fact that the chardev fixes do not form a build dependency, I
>>> think it's okay if this pull request gets processed before Paolo's
>>> (but it's also okay if Paolo's goes in first).
> 
> Based on the testsuite failures, it looks like Paolo's pull request with 
> chardev fixes DOES have to go in first.

Nearing the end of my workday; I'm not sure what the status is on 
Paolo's pull request with the chardev fixes, but hope that it doesn't 
invalidate this pull request from still being considered as soft freeze 
material.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze
  2018-03-13 21:55     ` Eric Blake
@ 2018-03-14 12:31       ` Peter Maydell
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Maydell @ 2018-03-14 12:31 UTC (permalink / raw)
  To: Eric Blake; +Cc: Paolo Bonzini, QEMU Developers, Peter Xu

On 13 March 2018 at 21:55, Eric Blake <eblake@redhat.com> wrote:
> On 03/13/2018 09:17 AM, Eric Blake wrote:
>>>>
>>>> This builds and passes 'make check', so even though the OOB portion
>>>> depends on chardev fixes that are still pending a pull request from
>>>> Paolo, that dependence can only be observed at runtime by clients
>>>> that use the new oob feature.  Given the timing of soft freeze, and
>>>> the fact that the chardev fixes do not form a build dependency, I
>>>> think it's okay if this pull request gets processed before Paolo's
>>>> (but it's also okay if Paolo's goes in first).
>>
>>
>> Based on the testsuite failures, it looks like Paolo's pull request with
>> chardev fixes DOES have to go in first.
>
>
> Nearing the end of my workday; I'm not sure what the status is on Paolo's
> pull request with the chardev fixes, but hope that it doesn't invalidate
> this pull request from still being considered as soft freeze material.

The freeze approach is that if you get v1 of the pull on the list
before the date, then the purpose of the following week before rc0
is to get the stragglers in and so that we can do re-spins if there
were minor issues with anything. So it's ok.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-12 18:36 ` [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed Eric Blake
@ 2018-03-21 12:37   ` Max Reitz
  2018-03-21 12:41     ` Max Reitz
  2018-03-21 17:11     ` Eric Blake
  0 siblings, 2 replies; 50+ messages in thread
From: Max Reitz @ 2018-03-21 12:37 UTC (permalink / raw)
  To: Eric Blake, qemu-devel
  Cc: Dr. David Alan Gilbert, Peter Xu, Markus Armbruster

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

On 2018-03-12 19:36, Eric Blake wrote:
> From: Peter Xu <peterx@redhat.com>
> 
> Start to use dedicate IO thread for QMP monitors that are not using
> MUXed chardev.
> 
> Reviewed-by: Fam Zheng <famz@redhat.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> Message-Id: <20180309090006.10018-21-peterx@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  monitor.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)

It seems the iotests aren't quite ready for this change (breakage in
040, 041, 051, 060, 085, 087, 093, 095, 109, 124, 127, 132, 136, 144,
148, 152, 182, 183, 185, 186; and 030 hangs).

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-21 12:37   ` Max Reitz
@ 2018-03-21 12:41     ` Max Reitz
  2018-03-21 17:11     ` Eric Blake
  1 sibling, 0 replies; 50+ messages in thread
From: Max Reitz @ 2018-03-21 12:41 UTC (permalink / raw)
  To: Eric Blake, qemu-devel
  Cc: Dr. David Alan Gilbert, Peter Xu, Markus Armbruster

[-- Attachment #1: Type: text/plain, Size: 860 bytes --]

On 2018-03-21 13:37, Max Reitz wrote:
> On 2018-03-12 19:36, Eric Blake wrote:
>> From: Peter Xu <peterx@redhat.com>
>>
>> Start to use dedicate IO thread for QMP monitors that are not using
>> MUXed chardev.
>>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
>> Signed-off-by: Peter Xu <peterx@redhat.com>
>> Message-Id: <20180309090006.10018-21-peterx@redhat.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>>  monitor.c | 5 ++++-
>>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> It seems the iotests aren't quite ready for this change (breakage in
> 040, 041, 051, 060, 085, 087, 093, 095, 109, 124, 127, 132, 136, 144,
> 148, 152, 182, 183, 185, 186; and 030 hangs).

(On second thought, 051, 185, and 186 have probably broken because of
something else.

Max)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-21 12:37   ` Max Reitz
  2018-03-21 12:41     ` Max Reitz
@ 2018-03-21 17:11     ` Eric Blake
  2018-03-21 17:15       ` Dr. David Alan Gilbert
  2018-03-22  3:09       ` Peter Xu
  1 sibling, 2 replies; 50+ messages in thread
From: Eric Blake @ 2018-03-21 17:11 UTC (permalink / raw)
  To: Max Reitz, qemu-devel; +Cc: Dr. David Alan Gilbert, Peter Xu, Markus Armbruster

On 03/21/2018 07:37 AM, Max Reitz wrote:
> On 2018-03-12 19:36, Eric Blake wrote:
>> From: Peter Xu <peterx@redhat.com>
>>
>> Start to use dedicate IO thread for QMP monitors that are not using
>> MUXed chardev.
>>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
>> Signed-off-by: Peter Xu <peterx@redhat.com>
>> Message-Id: <20180309090006.10018-21-peterx@redhat.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>>   monitor.c | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> It seems the iotests aren't quite ready for this change (breakage in
> 040, 041, 051, 060, 085, 087, 093, 095, 109, 124, 127, 132, 136, 144,
> 148, 152, 182, 183, 185, 186; and 030 hangs).

Serves me right for not testing iotests on my qapi tree as rigoursly as 
I do on my NBD tree.  Fixing this during freeze is acceptable, although 
I'd like it if Peter Xu can take a look instead of just me.

(And still a bummer that iotests aren't being run by 'make check' or CLI 
to have caught it earlier?)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-21 17:11     ` Eric Blake
@ 2018-03-21 17:15       ` Dr. David Alan Gilbert
  2018-03-22  3:09       ` Peter Xu
  1 sibling, 0 replies; 50+ messages in thread
From: Dr. David Alan Gilbert @ 2018-03-21 17:15 UTC (permalink / raw)
  To: Eric Blake; +Cc: Max Reitz, qemu-devel, Peter Xu, Markus Armbruster

* Eric Blake (eblake@redhat.com) wrote:
> On 03/21/2018 07:37 AM, Max Reitz wrote:
> > On 2018-03-12 19:36, Eric Blake wrote:
> > > From: Peter Xu <peterx@redhat.com>
> > > 
> > > Start to use dedicate IO thread for QMP monitors that are not using
> > > MUXed chardev.
> > > 
> > > Reviewed-by: Fam Zheng <famz@redhat.com>
> > > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> > > Signed-off-by: Peter Xu <peterx@redhat.com>
> > > Message-Id: <20180309090006.10018-21-peterx@redhat.com>
> > > Signed-off-by: Eric Blake <eblake@redhat.com>
> > > ---
> > >   monitor.c | 5 ++++-
> > >   1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > It seems the iotests aren't quite ready for this change (breakage in
> > 040, 041, 051, 060, 085, 087, 093, 095, 109, 124, 127, 132, 136, 144,
> > 148, 152, 182, 183, 185, 186; and 030 hangs).
> 
> Serves me right for not testing iotests on my qapi tree as rigoursly as I do
> on my NBD tree.  Fixing this during freeze is acceptable, although I'd like
> it if Peter Xu can take a look instead of just me.
> 
> (And still a bummer that iotests aren't being run by 'make check' or CLI to
> have caught it earlier?)

I'm guessing from those that 030 is the easiest to try and debug because
it hangs and you get a nice backtrace.  The others are a bit mysterious
(I think it's 042 which just has an f in one place in the output rather
than a .)

Dave

> -- 
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3266
> Virtualization:  qemu.org | libvirt.org
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed
  2018-03-21 17:11     ` Eric Blake
  2018-03-21 17:15       ` Dr. David Alan Gilbert
@ 2018-03-22  3:09       ` Peter Xu
  1 sibling, 0 replies; 50+ messages in thread
From: Peter Xu @ 2018-03-22  3:09 UTC (permalink / raw)
  To: Eric Blake
  Cc: Max Reitz, qemu-devel, Dr. David Alan Gilbert, Markus Armbruster

On Wed, Mar 21, 2018 at 12:11:21PM -0500, Eric Blake wrote:
> On 03/21/2018 07:37 AM, Max Reitz wrote:
> > On 2018-03-12 19:36, Eric Blake wrote:
> > > From: Peter Xu <peterx@redhat.com>
> > > 
> > > Start to use dedicate IO thread for QMP monitors that are not using
> > > MUXed chardev.
> > > 
> > > Reviewed-by: Fam Zheng <famz@redhat.com>
> > > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> > > Signed-off-by: Peter Xu <peterx@redhat.com>
> > > Message-Id: <20180309090006.10018-21-peterx@redhat.com>
> > > Signed-off-by: Eric Blake <eblake@redhat.com>
> > > ---
> > >   monitor.c | 5 ++++-
> > >   1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > It seems the iotests aren't quite ready for this change (breakage in
> > 040, 041, 051, 060, 085, 087, 093, 095, 109, 124, 127, 132, 136, 144,
> > 148, 152, 182, 183, 185, 186; and 030 hangs).
> 
> Serves me right for not testing iotests on my qapi tree as rigoursly as I do
> on my NBD tree.  Fixing this during freeze is acceptable, although I'd like
> it if Peter Xu can take a look instead of just me.
> 
> (And still a bummer that iotests aren't being run by 'make check' or CLI to
> have caught it earlier?)

Hello, Max, Eric,

I'll have a look soon, along with the problem that Marc-Andre
reported.  Will keep you updated.

Thanks,
Peter

-- 
Peter Xu

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL, 08/36] qapi: Remove qobject_to_X() functions
  2018-03-12 18:35 ` [Qemu-devel] [PULL 08/36] qapi: Remove qobject_to_X() functions Eric Blake
@ 2018-04-12 17:51   ` Yuval Shaia
  2018-04-12 20:52     ` Eric Blake
  0 siblings, 1 reply; 50+ messages in thread
From: Yuval Shaia @ 2018-04-12 17:51 UTC (permalink / raw)
  To: Eric Blake; +Cc: qemu-devel, Markus Armbruster, Max Reitz, Marcel Apfelbaum

Hi Eric,

On Mon, Mar 12, 2018 at 01:35:59PM -0500, Eric Blake wrote:
> From: Max Reitz <mreitz@redhat.com>
> 
> They are no longer needed now.

I'm doing some off-list development which use these functions.
Unfortunately i'm not subscribed (yet) to qemu-devel so obviously missed
this email.

Is there any alternative to these functions?

My code looks something like this:

QObject *oid;
unsigned long id;

oid = qlist_pop(list);
id = qnum_get_uint(qobject_to_qnum(oid));

Since objects in the list are QObject type.

Yuval

> 
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> Reviewed-by: Alberto Garcia <berto@igalia.com>
> Message-Id: <20180224154033.29559-5-mreitz@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  include/qapi/qmp/qbool.h   |  1 -
>  include/qapi/qmp/qdict.h   |  1 -
>  include/qapi/qmp/qlist.h   |  1 -
>  include/qapi/qmp/qnum.h    |  1 -
>  include/qapi/qmp/qstring.h |  1 -
>  qobject/qbool.c            | 11 -----------
>  qobject/qdict.c            | 11 -----------
>  qobject/qlist.c            | 11 -----------
>  qobject/qnum.c             | 11 -----------
>  qobject/qstring.c          | 11 -----------
>  10 files changed, 60 deletions(-)
> 
> diff --git a/include/qapi/qmp/qbool.h b/include/qapi/qmp/qbool.h
> index 629c508d34f..b9a44a1bfe0 100644
> --- a/include/qapi/qmp/qbool.h
> +++ b/include/qapi/qmp/qbool.h
> @@ -23,7 +23,6 @@ struct QBool {
> 
>  QBool *qbool_from_bool(bool value);
>  bool qbool_get_bool(const QBool *qb);
> -QBool *qobject_to_qbool(const QObject *obj);
>  bool qbool_is_equal(const QObject *x, const QObject *y);
>  void qbool_destroy_obj(QObject *obj);
> 
> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> index 7c6d8445495..2cc3e906f7b 100644
> --- a/include/qapi/qmp/qdict.h
> +++ b/include/qapi/qmp/qdict.h
> @@ -39,7 +39,6 @@ void qdict_put_obj(QDict *qdict, const char *key, QObject *value);
>  void qdict_del(QDict *qdict, const char *key);
>  int qdict_haskey(const QDict *qdict, const char *key);
>  QObject *qdict_get(const QDict *qdict, const char *key);
> -QDict *qobject_to_qdict(const QObject *obj);
>  bool qdict_is_equal(const QObject *x, const QObject *y);
>  void qdict_iter(const QDict *qdict,
>                  void (*iter)(const char *key, QObject *obj, void *opaque),
> diff --git a/include/qapi/qmp/qlist.h b/include/qapi/qmp/qlist.h
> index 5fd976a3981..5c673acb060 100644
> --- a/include/qapi/qmp/qlist.h
> +++ b/include/qapi/qmp/qlist.h
> @@ -53,7 +53,6 @@ QObject *qlist_pop(QList *qlist);
>  QObject *qlist_peek(QList *qlist);
>  int qlist_empty(const QList *qlist);
>  size_t qlist_size(const QList *qlist);
> -QList *qobject_to_qlist(const QObject *obj);
>  bool qlist_is_equal(const QObject *x, const QObject *y);
>  void qlist_destroy_obj(QObject *obj);
> 
> diff --git a/include/qapi/qmp/qnum.h b/include/qapi/qmp/qnum.h
> index 15e3971c7f1..3e47475b2cb 100644
> --- a/include/qapi/qmp/qnum.h
> +++ b/include/qapi/qmp/qnum.h
> @@ -68,7 +68,6 @@ double qnum_get_double(QNum *qn);
> 
>  char *qnum_to_string(QNum *qn);
> 
> -QNum *qobject_to_qnum(const QObject *obj);
>  bool qnum_is_equal(const QObject *x, const QObject *y);
>  void qnum_destroy_obj(QObject *obj);
> 
> diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
> index 98070ef3d6a..b72843fc1b0 100644
> --- a/include/qapi/qmp/qstring.h
> +++ b/include/qapi/qmp/qstring.h
> @@ -30,7 +30,6 @@ const char *qstring_get_str(const QString *qstring);
>  void qstring_append_int(QString *qstring, int64_t value);
>  void qstring_append(QString *qstring, const char *str);
>  void qstring_append_chr(QString *qstring, int c);
> -QString *qobject_to_qstring(const QObject *obj);
>  bool qstring_is_equal(const QObject *x, const QObject *y);
>  void qstring_destroy_obj(QObject *obj);
> 
> diff --git a/qobject/qbool.c b/qobject/qbool.c
> index 5be6277cca8..b58249925c2 100644
> --- a/qobject/qbool.c
> +++ b/qobject/qbool.c
> @@ -39,17 +39,6 @@ bool qbool_get_bool(const QBool *qb)
>      return qb->value;
>  }
> 
> -/**
> - * qobject_to_qbool(): Convert a QObject into a QBool
> - */
> -QBool *qobject_to_qbool(const QObject *obj)
> -{
> -    if (!obj || qobject_type(obj) != QTYPE_QBOOL) {
> -        return NULL;
> -    }
> -    return container_of(obj, QBool, base);
> -}
> -
>  /**
>   * qbool_is_equal(): Test whether the two QBools are equal
>   */
> diff --git a/qobject/qdict.c b/qobject/qdict.c
> index 1e588123d00..45c8b53361f 100644
> --- a/qobject/qdict.c
> +++ b/qobject/qdict.c
> @@ -37,17 +37,6 @@ QDict *qdict_new(void)
>      return qdict;
>  }
> 
> -/**
> - * qobject_to_qdict(): Convert a QObject into a QDict
> - */
> -QDict *qobject_to_qdict(const QObject *obj)
> -{
> -    if (!obj || qobject_type(obj) != QTYPE_QDICT) {
> -        return NULL;
> -    }
> -    return container_of(obj, QDict, base);
> -}
> -
>  /**
>   * tdb_hash(): based on the hash agorithm from gdbm, via tdb
>   * (from module-init-tools)
> diff --git a/qobject/qlist.c b/qobject/qlist.c
> index 77f19ffda78..954fe983751 100644
> --- a/qobject/qlist.c
> +++ b/qobject/qlist.c
> @@ -151,17 +151,6 @@ size_t qlist_size(const QList *qlist)
>      return count;
>  }
> 
> -/**
> - * qobject_to_qlist(): Convert a QObject into a QList
> - */
> -QList *qobject_to_qlist(const QObject *obj)
> -{
> -    if (!obj || qobject_type(obj) != QTYPE_QLIST) {
> -        return NULL;
> -    }
> -    return container_of(obj, QList, base);
> -}
> -
>  /**
>   * qlist_is_equal(): Test whether the two QLists are equal
>   *
> diff --git a/qobject/qnum.c b/qobject/qnum.c
> index ea091cfaa4b..1501c828323 100644
> --- a/qobject/qnum.c
> +++ b/qobject/qnum.c
> @@ -199,17 +199,6 @@ char *qnum_to_string(QNum *qn)
>      return NULL;
>  }
> 
> -/**
> - * qobject_to_qnum(): Convert a QObject into a QNum
> - */
> -QNum *qobject_to_qnum(const QObject *obj)
> -{
> -    if (!obj || qobject_type(obj) != QTYPE_QNUM) {
> -        return NULL;
> -    }
> -    return container_of(obj, QNum, base);
> -}
> -
>  /**
>   * qnum_is_equal(): Test whether the two QNums are equal
>   *
> diff --git a/qobject/qstring.c b/qobject/qstring.c
> index 2b2153206df..8bae529ef79 100644
> --- a/qobject/qstring.c
> +++ b/qobject/qstring.c
> @@ -105,17 +105,6 @@ void qstring_append_chr(QString *qstring, int c)
>      qstring->string[qstring->length] = 0;
>  }
> 
> -/**
> - * qobject_to_qstring(): Convert a QObject to a QString
> - */
> -QString *qobject_to_qstring(const QObject *obj)
> -{
> -    if (!obj || qobject_type(obj) != QTYPE_QSTRING) {
> -        return NULL;
> -    }
> -    return container_of(obj, QString, base);
> -}
> -
>  /**
>   * qstring_get_str(): Return a pointer to the stored string
>   *

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL, 08/36] qapi: Remove qobject_to_X() functions
  2018-04-12 17:51   ` [Qemu-devel] [PULL, " Yuval Shaia
@ 2018-04-12 20:52     ` Eric Blake
  2018-04-13 11:33       ` Yuval Shaia
  0 siblings, 1 reply; 50+ messages in thread
From: Eric Blake @ 2018-04-12 20:52 UTC (permalink / raw)
  To: Yuval Shaia; +Cc: qemu-devel, Markus Armbruster, Max Reitz, Marcel Apfelbaum

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On 04/12/2018 12:51 PM, Yuval Shaia wrote:
> Hi Eric,
> 
> On Mon, Mar 12, 2018 at 01:35:59PM -0500, Eric Blake wrote:
>> From: Max Reitz <mreitz@redhat.com>
>>
>> They are no longer needed now.
> 
> I'm doing some off-list development which use these functions.
> Unfortunately i'm not subscribed (yet) to qemu-devel so obviously missed
> this email.
> 
> Is there any alternative to these functions?

Yes, in the patches right before this one.

> 
> My code looks something like this:
> 
> QObject *oid;
> unsigned long id;
> 
> oid = qlist_pop(list);
> id = qnum_get_uint(qobject_to_qnum(oid));
> 

id = qnum_get_uint(qobject_to(QNum, oid));

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

^ permalink raw reply	[flat|nested] 50+ messages in thread

* Re: [Qemu-devel] [PULL, 08/36] qapi: Remove qobject_to_X() functions
  2018-04-12 20:52     ` Eric Blake
@ 2018-04-13 11:33       ` Yuval Shaia
  0 siblings, 0 replies; 50+ messages in thread
From: Yuval Shaia @ 2018-04-13 11:33 UTC (permalink / raw)
  To: Eric Blake; +Cc: qemu-devel, Markus Armbruster, Max Reitz, Marcel Apfelbaum

On Thu, Apr 12, 2018 at 03:52:50PM -0500, Eric Blake wrote:
> On 04/12/2018 12:51 PM, Yuval Shaia wrote:
> > Hi Eric,
> > 
> > On Mon, Mar 12, 2018 at 01:35:59PM -0500, Eric Blake wrote:
> >> From: Max Reitz <mreitz@redhat.com>
> >>
> >> They are no longer needed now.
> > 
> > I'm doing some off-list development which use these functions.
> > Unfortunately i'm not subscribed (yet) to qemu-devel so obviously missed
> > this email.
> > 
> > Is there any alternative to these functions?
> 
> Yes, in the patches right before this one.
> 
> > 
> > My code looks something like this:
> > 
> > QObject *oid;
> > unsigned long id;
> > 
> > oid = qlist_pop(list);
> > id = qnum_get_uint(qobject_to_qnum(oid));
> > 
> 
> id = qnum_get_uint(qobject_to(QNum, oid));

Thanks,

> 
> -- 
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3266
> Virtualization:  qemu.org | libvirt.org
> 

^ permalink raw reply	[flat|nested] 50+ messages in thread

end of thread, other threads:[~2018-04-13 11:33 UTC | newest]

Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-12 18:35 [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 01/36] qapi2texi: minor python code simplification Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 02/36] qlit: use QType instead of int Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 03/36] qlit: add qobject_from_qlit() Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 04/36] qapi: generate a literal qobject for introspection Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 05/36] compiler: Add QEMU_BUILD_BUG_MSG() macro Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 06/36] qapi: Add qobject_to() Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 07/36] qapi: Replace qobject_to_X(o) by qobject_to(X, o) Eric Blake
2018-03-12 18:35 ` [Qemu-devel] [PULL 08/36] qapi: Remove qobject_to_X() functions Eric Blake
2018-04-12 17:51   ` [Qemu-devel] [PULL, " Yuval Shaia
2018-04-12 20:52     ` Eric Blake
2018-04-13 11:33       ` Yuval Shaia
2018-03-12 18:36 ` [Qemu-devel] [PULL 09/36] qapi: Make more of qobject_to() Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 10/36] block: Handle null backing link Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 11/36] block: Deprecate "backing": "" Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 12/36] docs: update QMP documents for OOB commands Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 13/36] qobject: introduce qstring_get_try_str() Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 14/36] qobject: introduce qobject_get_try_str() Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 15/36] qobject: let object_property_get_str() use new API Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 16/36] monitor: move skip_flush into monitor_data_init Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 17/36] monitor: move the cur_mon hack deeper for QMP Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 18/36] monitor: unify global init Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 19/36] monitor: let mon_list be tail queue Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 20/36] monitor: allow using IO thread for parsing Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 21/36] qmp: introduce QMPCapability Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 22/36] monitor: introduce monitor_qmp_respond() Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 23/36] monitor: let suspend_cnt be thread safe Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 24/36] monitor: let suspend/resume work even with QMPs Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 25/36] monitor: separate QMP parser and dispatcher Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 26/36] qmp: add new event "command-dropped" Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 27/36] monitor: send event when command queue full Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 28/36] qapi: introduce new cmd option "allow-oob" Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 29/36] qmp: support out-of-band (oob) execution Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 30/36] qmp: isolate responses into io thread Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 31/36] monitor: enable IO thread for (qmp & !mux) typed Eric Blake
2018-03-21 12:37   ` Max Reitz
2018-03-21 12:41     ` Max Reitz
2018-03-21 17:11     ` Eric Blake
2018-03-21 17:15       ` Dr. David Alan Gilbert
2018-03-22  3:09       ` Peter Xu
2018-03-12 18:36 ` [Qemu-devel] [PULL 32/36] qmp: add command "x-oob-test" Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 33/36] tests: qmp-test: verify command batching Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 34/36] tests: qmp-test: add oob test Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 35/36] block/accounting: introduce latency histogram Eric Blake
2018-03-12 18:36 ` [Qemu-devel] [PULL 36/36] qapi: add block latency histogram interface Eric Blake
2018-03-13 14:02 ` [Qemu-devel] [PULL 00/36] QAPI patches for 2018-03-12, 2.12 softfreeze Peter Maydell
2018-03-13 14:17   ` Eric Blake
2018-03-13 15:21     ` Peter Xu
2018-03-13 21:55     ` Eric Blake
2018-03-14 12:31       ` Peter Maydell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.