All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode
@ 2016-08-17 16:57 Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 01/20] tests: do qmp introspect validation per target Marc-André Lureau
                   ` (19 more replies)
  0 siblings, 20 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Hi,

Although some QMP commands are still not fully qapi'fied, it is
possible to use more qapi common and generated code by dropping the
'middle' mode and use qmp_dispatch().

v5:
- replace the dynamic unregister commands approach by a few
  preliminary patches to make the json schema #ifdef conditional and
  per-target
- fix "check invalid arguments" patch based on Markus version
- patch reordering

v4:
- export all marshaller functions (so we can keep calling them after
  middle mode is removed), remove 'export-marshal' patch
- get rid of unnecessary lambda in python code (leftover), remove
  second mcgen(), and outdated comment
- remove disabled commands at run-time to avoid any regression. It's
  now on my TODO list to fix qapi generator in 2.8 to have
  conditionals
- move qmp-commands.txt to doc/
- split the last patch, remove trailing ws
- add QEMU_VERSION_{MAJOR,MINOR,MICRO} patch, simplifying
  qmp_query_version() (could be applied outside this series)
- update commit title/messages/order

v3:
- add a reference to docs/qmp-spec.txt in qmp_capabilities doc
- remove 'props' from device_add doc, improve example
- replace a g_strcmp0 with more appropriate g_str_equal
- add 'export-marshal' command generator key patch
- call qmp_marshal_query_version() directly (also get rid of the need
  to do a make clean, since the qapi json is modified)
- add patch to check invalid arguments on no-args (the old dispatch
  code checks that), and a test
- patch reordering to fix intermediate builds
- commit messages improvements
- split some misc doc fixes in last patch
- add some r-b and rebase

v2:
- rebased on master
- add Since: 0.13 to qmp_capabilities and device_add documentation
- fix device_add doc
- add missing spaces after ',' in get_qmp_greeting()
- fix some grammar in monitor.c while touching it

Marc-André Lureau (20):
  tests: do qmp introspect validation per target
  qapi.py: add a simple #ifdef conditional
  build-sys: make qemu qapi per-target
  build-sys: use config headers to generate qapi
  qapi: configure the schema
  build-sys: define QEMU_VERSION_{MAJOR,MINOR,MICRO}
  qapi-schema: use generated marshaller for 'qmp_capabilities'
  qapi-schema: add 'device_add'
  monitor: simplify invalid_qmp_mode()
  monitor: register gen:false commands manually
  qapi: export the marshallers
  monitor: use qmp_find_command() (using generated qapi code)
  monitor: implement 'qmp_query_commands' without qmp_cmds
  monitor: remove mhandler.cmd_new
  qapi: remove the "middle" mode
  qapi: check invalid arguments on no-args commands
  monitor: use qmp_dispatch()
  build-sys: remove qmp-commands-old.h
  qmp-commands.hx: fix some styling
  Replace qmp-commands.hx by doc/qmp-commands.txt

 hmp.c                                    |    2 +
 monitor.c                                |  415 ++--------
 qmp.c                                    |   48 +-
 stubs/qapi-event.c                       |   74 ++
 tests/qmp-introspect-test.c              |   50 ++
 tests/test-qmp-commands.c                |   15 +
 tests/test-qmp-input-strict.c            |    2 -
 vl.c                                     |    1 +
 Makefile                                 |   61 +-
 Makefile.objs                            |    7 +-
 Makefile.target                          |   32 +-
 scripts/create_config                    |    6 +
 scripts/qapi-commands.py                 |   81 +-
 scripts/qapi.py                          |   43 +-
 stubs/Makefile.objs                      |    1 +
 tests/Makefile.include                   |    2 +
 .gitignore                               |    1 -
 MAINTAINERS                              |    1 -
 docs/qapi-code-gen.txt                   |    6 +-
 qmp-commands.hx => docs/qmp-commands.txt | 1278 +-----------------------------
 docs/writing-qmp-commands.txt            |   46 +-
 hmp-commands-info.hx                     |  120 +--
 hmp-commands.hx                          |  208 ++---
 qapi-schema.json                         |   91 +++
 qapi.mak                                 |   14 +
 qapi/event.json                          |    8 +
 trace-events                             |    1 -
 27 files changed, 667 insertions(+), 1947 deletions(-)
 create mode 100644 stubs/qapi-event.c
 create mode 100644 tests/qmp-introspect-test.c
 rename qmp-commands.hx => docs/qmp-commands.txt (82%)
 create mode 100644 qapi.mak

-- 
2.9.0

^ permalink raw reply	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 01/20] tests: do qmp introspect validation per target
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional Marc-André Lureau
                   ` (18 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

The following commits will move the qmp schema to be per target, and
can't be linked. Instead of validating it directly, query it from the
target via qmp.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/qmp-introspect-test.c   | 50 +++++++++++++++++++++++++++++++++++++++++++
 tests/test-qmp-input-strict.c |  2 --
 tests/Makefile.include        |  2 ++
 3 files changed, 52 insertions(+), 2 deletions(-)
 create mode 100644 tests/qmp-introspect-test.c

diff --git a/tests/qmp-introspect-test.c b/tests/qmp-introspect-test.c
new file mode 100644
index 0000000..cafb311
--- /dev/null
+++ b/tests/qmp-introspect-test.c
@@ -0,0 +1,50 @@
+/*
+ * Per-target QAPI introspection test cases
+ *
+ * Copyright (c) 2016 Red Hat Inc.
+ *
+ * Authors:
+ *  Marc-André Lureau <marcandre.lureau@redhat.com>,
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qapi/qmp/qstring.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi/error.h"
+#include "qapi-visit.h"
+#include "libqtest.h"
+
+const char common_args[] = "-nodefaults -machine none";
+
+static void test_qmp_introspect_validate(void)
+{
+    SchemaInfoList *schema;
+    QDict *resp;
+    Visitor *v;
+
+    qtest_start(common_args);
+
+    resp = qmp("{'execute': 'query-qmp-schema'}");
+    v = qmp_input_visitor_new(qdict_get(resp, "return"), true);
+    visit_type_SchemaInfoList(v, NULL, &schema, &error_abort);
+    g_assert(schema);
+
+    qapi_free_SchemaInfoList(schema);
+    visit_free(v);
+    QDECREF(resp);
+
+    qtest_end();
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+
+    qtest_add_func("qmp-introspect/validate", test_qmp_introspect_validate);
+
+    return g_test_run();
+}
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 814550a..003a90e 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -21,7 +21,6 @@
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qjson.h"
 #include "test-qmp-introspect.h"
-#include "qmp-introspect.h"
 #include "qapi-visit.h"
 
 typedef struct TestInputVisitorData {
@@ -283,7 +282,6 @@ static void test_validate_qmp_introspect(TestInputVisitorData *data,
                                            const void *unused)
 {
     do_test_validate_qmp_introspect(data, test_qmp_schema_json);
-    do_test_validate_qmp_introspect(data, qmp_schema_json);
 }
 
 static void validate_test_add(const char *testpath,
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 14be491..983b5b4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -119,6 +119,7 @@ check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 # really in libqtest, not in the testcases themselves.
 
 check-qtest-generic-y = tests/device-introspect-test$(EXESUF)
+check-qtest-generic-y = tests/qmp-introspect-test$(EXESUF)
 gcov-files-generic-y = qdev-monitor.c qmp.c
 
 gcov-files-ipack-y += hw/ipack/ipack.c
@@ -565,6 +566,7 @@ libqos-imx-obj-y = $(libqos-obj-y) tests/libqos/i2c-imx.o
 libqos-usb-obj-y = $(libqos-pc-obj-y) tests/libqos/usb.o
 libqos-virtio-obj-y = $(libqos-pc-obj-y) tests/libqos/virtio.o tests/libqos/virtio-pci.o tests/libqos/virtio-mmio.o tests/libqos/malloc-generic.o
 
+tests/qmp-introspect-test$(EXESUF): tests/qmp-introspect-test.o
 tests/device-introspect-test$(EXESUF): tests/device-introspect-test.o
 tests/rtc-test$(EXESUF): tests/rtc-test.o
 tests/m48t59-test$(EXESUF): tests/m48t59-test.o
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 01/20] tests: do qmp introspect validation per target Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-09-06 12:35   ` Markus Armbruster
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 03/20] build-sys: make qemu qapi per-target Marc-André Lureau
                   ` (17 subsequent siblings)
  19 siblings, 1 reply; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Learn to parse #define files provided with -f option, and skip
undefined #ifdef blocks in the schema.

This is a very simple pre-processing, without stacking support or
evaluation (it could be implemented if needed).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 40 insertions(+), 3 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 21bc32f..d0b8a66 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -76,6 +76,7 @@ struct_types = []
 union_types = []
 events = []
 all_names = {}
+defs = []
 
 #
 # Parsing the schema into expressions
@@ -177,6 +178,7 @@ class QAPISchemaParser(object):
                 self.exprs.append(expr_elem)
 
     def accept(self):
+        ok = True
         while True:
             self.tok = self.src[self.cursor]
             self.pos = self.cursor
@@ -184,7 +186,19 @@ class QAPISchemaParser(object):
             self.val = None
 
             if self.tok == '#':
-                self.cursor = self.src.find('\n', self.cursor)
+                end = self.src.find('\n', self.cursor)
+                line = self.src[self.cursor:end+1]
+                self.cursor = end
+                sline = line.split()
+                if len(defs) and len(sline) >= 1 \
+                   and sline[0] in ['ifdef', 'endif']:
+                    if sline[0] == 'ifdef':
+                        ok = sline[1] in defs
+                    elif sline[0] == 'endif':
+                        ok = True
+                    continue
+            elif not ok:
+                continue
             elif self.tok in "{}:,[]":
                 return
             elif self.tok == "'":
@@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
 #
 # Common command line parsing
 #
+def defile(filename):
+    f = open(filename, 'r')
+    while 1:
+        line = f.readline()
+        if not line:
+            break
+        while line[-2:] == '\\\n':
+            nextline = f.readline()
+            if not nextline:
+                break
+            line = line + nextline
+        tmp = line.strip()
+        if tmp[:1] != '#':
+            continue
+        tmp = tmp[1:]
+        words = tmp.split()
+        if words[0] != "define":
+            continue
+        defs.append(words[1])
+    f.close()
 
 
 def parse_command_line(extra_options="", extra_long_options=[]):
 
     try:
         opts, args = getopt.gnu_getopt(sys.argv[1:],
-                                       "chp:o:" + extra_options,
+                                       "chp:o:f:" + extra_options,
                                        ["source", "header", "prefix=",
-                                        "output-dir="] + extra_long_options)
+                                        "output-dir=", "--defile="] +
+                                       extra_long_options)
     except getopt.GetoptError as err:
         print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
         sys.exit(1)
@@ -1742,6 +1777,8 @@ def parse_command_line(extra_options="", extra_long_options=[]):
             do_c = True
         elif o in ("-h", "--header"):
             do_h = True
+        elif o in ("-f", "--defile"):
+            defile(a)
         else:
             extra_opts.append(oa)
 
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 03/20] build-sys: make qemu qapi per-target
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 01/20] tests: do qmp introspect validation per target Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 04/20] build-sys: use config headers to generate qapi Marc-André Lureau
                   ` (16 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

The qapi schema has per-target definitions. Generate the marshaller,
events and introspection per target. Keep the types, visitors and
headers generic to keep common code unaware. However, having per-target
events requires block events stubs.

The following patch will use config headers to conditionally remove
disabled features from the schema.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 stubs/qapi-event.c  | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 Makefile            | 54 +++++++++++++++-----------------------
 Makefile.objs       |  7 +----
 Makefile.target     | 23 +++++++++++++++++
 stubs/Makefile.objs |  1 +
 qapi.mak            | 14 ++++++++++
 6 files changed, 133 insertions(+), 40 deletions(-)
 create mode 100644 stubs/qapi-event.c
 create mode 100644 qapi.mak

diff --git a/stubs/qapi-event.c b/stubs/qapi-event.c
new file mode 100644
index 0000000..73b5424
--- /dev/null
+++ b/stubs/qapi-event.c
@@ -0,0 +1,74 @@
+#include "qemu/osdep.h"
+#include "qapi-event.h"
+
+void qapi_event_send_device_tray_moved(const char *device, bool tray_open,
+                                       Error **errp)
+{
+}
+
+void qapi_event_send_quorum_report_bad(QuorumOpType type, bool has_error,
+                                       const char *error, const char *node_name,
+                                       int64_t sector_num,
+                                       int64_t sectors_count, Error **errp)
+{
+}
+
+void qapi_event_send_quorum_failure(const char *reference, int64_t sector_num,
+                                    int64_t sectors_count, Error **errp)
+{
+}
+
+void qapi_event_send_block_job_cancelled(BlockJobType type, const char *device,
+                                         int64_t len, int64_t offset,
+                                         int64_t speed, Error **errp)
+{
+}
+
+void qapi_event_send_block_job_completed(BlockJobType type, const char *device,
+                                         int64_t len, int64_t offset,
+                                         int64_t speed, bool has_error,
+                                         const char *error, Error **errp)
+{
+}
+
+void qapi_event_send_block_job_error(const char *device,
+                                     IoOperationType operation,
+                                     BlockErrorAction action, Error **errp)
+{
+}
+
+void qapi_event_send_block_job_ready(BlockJobType type, const char *device,
+                                     int64_t len, int64_t offset, int64_t speed,
+                                     Error **errp)
+{
+}
+
+void qapi_event_send_block_io_error(const char *device,
+                                    IoOperationType operation,
+                                    BlockErrorAction action, bool has_nospace,
+                                    bool nospace, const char *reason,
+                                    Error **errp)
+{
+}
+
+void qapi_event_send_block_image_corrupted(const char *device,
+                                           bool has_node_name,
+                                           const char *node_name,
+                                           const char *msg, bool has_offset,
+                                           int64_t offset, bool has_size,
+                                           int64_t size, bool fatal,
+                                           Error **errp)
+{
+}
+
+void qapi_event_send_block_write_threshold(const char *node_name,
+                                           uint64_t amount_exceeded,
+                                           uint64_t write_threshold,
+                                           Error **errp)
+{
+}
+
+void qapi_event_send_device_deleted(bool has_device, const char *device,
+                                    const char *path, Error **errp)
+{
+}
diff --git a/Makefile b/Makefile
index 50b4b3a..83d27fb 100644
--- a/Makefile
+++ b/Makefile
@@ -49,12 +49,11 @@ endif
 endif
 
 include $(SRC_PATH)/rules.mak
+include $(SRC_PATH)/qapi.mak
 
 GENERATED_HEADERS = qemu-version.h config-host.h qemu-options.def
 GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
-GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
-GENERATED_HEADERS += qmp-introspect.h
-GENERATED_SOURCES += qmp-introspect.c
+GENERATED_SOURCES += qapi-types.c qapi-visit.c
 
 GENERATED_HEADERS += trace/generated-events.h
 GENERATED_SOURCES += trace/generated-events.c
@@ -268,56 +267,43 @@ qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
 qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
 qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
 
-gen-out-type = $(subst .,-,$(suffix $@))
-
-qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
-
 qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
-$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-types-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
-		$(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
-$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-visit-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
-		$(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
-$(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-commands-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 
-qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
-               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
-               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
-               $(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
-               $(SRC_PATH)/qapi/trace.json
-
 qapi-types.c qapi-types.h :\
-$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
+$(qapi-modules) $(qapi-types-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
-		$(gen-out-type) -o "." -b $<, \
+		$(qapi-gen-type) -o "." -b $<, \
 		"  GEN   $@")
 qapi-visit.c qapi-visit.h :\
-$(qapi-modules) $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
+$(qapi-modules) $(qapi-visit-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
-		$(gen-out-type) -o "." -b $<, \
+		$(qapi-gen-type) -o "." -b $<, \
 		"  GEN   $@")
-qapi-event.c qapi-event.h :\
-$(qapi-modules) $(SRC_PATH)/scripts/qapi-event.py $(qapi-py)
+
+qapi-event.h :\
+$(qapi-modules) $(qapi-event-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \
-		$(gen-out-type) -o "." $<, \
+		$(qapi-gen-type) -o "." $<, \
 		"  GEN   $@")
-qmp-commands.h qmp-marshal.c :\
-$(qapi-modules) $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
+
+qmp-commands.h :\
+$(qapi-modules) $(qapi-commands-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(gen-out-type) -o "." -m $<, \
-		"  GEN   $@")
-qmp-introspect.h qmp-introspect.c :\
-$(qapi-modules) $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py)
-	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \
-		$(gen-out-type) -o "." $<, \
+		$(qapi-gen-type) -o "." -m $<, \
 		"  GEN   $@")
 
 QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
diff --git a/Makefile.objs b/Makefile.objs
index 6d5ddcf..7addab8 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -2,7 +2,7 @@
 # Common libraries for tools and emulators
 stub-obj-y = stubs/ crypto/
 util-obj-y = util/ qobject/ qapi/
-util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
+util-obj-y += qapi-types.o qapi-visit.o
 
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
@@ -78,11 +78,6 @@ common-obj-$(CONFIG_SECCOMP) += qemu-seccomp.o
 
 common-obj-$(CONFIG_FDT) += device_tree.o
 
-######################################################################
-# qapi
-
-common-obj-y += qmp-marshal.o
-common-obj-y += qmp-introspect.o
 common-obj-y += qmp.o hmp.o
 endif
 
diff --git a/Makefile.target b/Makefile.target
index a440bcb..6b672eb 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -6,6 +6,7 @@ include ../config-host.mak
 include config-target.mak
 include config-devices.mak
 include $(SRC_PATH)/rules.mak
+include $(SRC_PATH)/qapi.mak
 
 $(call set-vpath, $(SRC_PATH):$(BUILD_DIR))
 ifdef CONFIG_LINUX
@@ -157,6 +158,28 @@ obj-y += hw/$(TARGET_BASE_ARCH)/
 endif
 
 GENERATED_HEADERS += hmp-commands.h hmp-commands-info.h qmp-commands-old.h
+GENERATED_HEADERS += qmp-introspect.h
+GENERATED_SOURCES += qmp-introspect.c qmp-marshal.c qapi-event.c
+
+qmp-marshal.c :\
+$(qapi-modules) $(qapi-commands-py)
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
+		$(qapi-gen-type) -o "." -m $<, \
+		"  GEN   $(TARGET_DIR)$@")
+
+qapi-event.c :\
+$(qapi-modules) $(qapi-event-py)
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \
+		$(qapi-gen-type) -o "." $<, \
+		"  GEN   $@")
+
+qmp-introspect.h qmp-introspect.c :\
+$(qapi-modules) $(qapi-introspect-py)
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \
+		$(qapi-gen-type) -o "." $<, \
+		"  GEN   $(TARGET_DIR)$@")
+
+obj-y += qmp-marshal.o qmp-introspect.o qapi-event.o
 
 endif # CONFIG_SOFTMMU
 
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 55edd15..aa2cbef 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -45,3 +45,4 @@ stub-obj-y += iohandler.o
 stub-obj-y += smbios_type_38.o
 stub-obj-y += ipmi.o
 stub-obj-y += pc_madt_cpu_entry.o
+stub-obj-y += qapi-event.o
diff --git a/qapi.mak b/qapi.mak
new file mode 100644
index 0000000..4e26714
--- /dev/null
+++ b/qapi.mak
@@ -0,0 +1,14 @@
+qapi-gen-type = $(subst .,-,$(suffix $@))
+
+qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
+       $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
+       $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
+       $(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
+       $(SRC_PATH)/qapi/trace.json
+
+qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
+
+qapi-types-py = $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
+qapi-visit-py = $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
+qapi-commands-py = $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
+qapi-introspect-py = $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py)
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 04/20] build-sys: use config headers to generate qapi
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (2 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 03/20] build-sys: make qemu qapi per-target Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 05/20] qapi: configure the schema Marc-André Lureau
                   ` (15 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Configure the marshaller, events and introspection with config host and
target headers. Keep the types, visitors, and command header generic
since it will only use what is actually needed, but needs all
declarations to be compatible with the various targets.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 Makefile        | 15 +++++++++------
 Makefile.target | 12 +++++++-----
 2 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/Makefile b/Makefile
index 83d27fb..f83b208 100644
--- a/Makefile
+++ b/Makefile
@@ -267,20 +267,23 @@ qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
 qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
 qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
 
+qapi-hconfig = config-host.h
+qapi-flags = $(patsubst %,-f %,$(qapi-hconfig))
+
 qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
-$(SRC_PATH)/qga/qapi-schema.json $(qapi-types-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-types-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \
-		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-flags) $(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
-$(SRC_PATH)/qga/qapi-schema.json $(qapi-visit-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-visit-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py \
-		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-flags) $(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
-$(SRC_PATH)/qga/qapi-schema.json $(qapi-commands-py)
+$(SRC_PATH)/qga/qapi-schema.json $(qapi-commands-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
+		$(qapi-flags) $(qapi-gen-type) -o qga/qapi-generated -p "qga-" $<, \
 		"  GEN   $@")
 
 qapi-types.c qapi-types.h :\
diff --git a/Makefile.target b/Makefile.target
index 6b672eb..c6c20bf 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -160,23 +160,25 @@ endif
 GENERATED_HEADERS += hmp-commands.h hmp-commands-info.h qmp-commands-old.h
 GENERATED_HEADERS += qmp-introspect.h
 GENERATED_SOURCES += qmp-introspect.c qmp-marshal.c qapi-event.c
+qapi-hconfig = ../config-host.h config-target.h
+qapi-flags = $(patsubst %,-f %,$(qapi-hconfig))
 
 qmp-marshal.c :\
-$(qapi-modules) $(qapi-commands-py)
+$(qapi-modules) $(qapi-commands-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(qapi-gen-type) -o "." -m $<, \
+		$(qapi-flags) $(qapi-gen-type) -o "." -m $<, \
 		"  GEN   $(TARGET_DIR)$@")
 
 qapi-event.c :\
 $(qapi-modules) $(qapi-event-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \
-		$(qapi-gen-type) -o "." $<, \
+		$(qapi-flags) $(qapi-gen-type) -o "." $<, \
 		"  GEN   $@")
 
 qmp-introspect.h qmp-introspect.c :\
-$(qapi-modules) $(qapi-introspect-py)
+$(qapi-modules) $(qapi-introspect-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \
-		$(qapi-gen-type) -o "." $<, \
+		$(qapi-flags) $(qapi-gen-type) -o "." $<, \
 		"  GEN   $(TARGET_DIR)$@")
 
 obj-y += qmp-marshal.o qmp-introspect.o qapi-event.o
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 05/20] qapi: configure the schema
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (3 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 04/20] build-sys: use config headers to generate qapi Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 06/20] build-sys: define QEMU_VERSION_{MAJOR, MINOR, MICRO} Marc-André Lureau
                   ` (14 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Make parts of the qapi schema conditional based on host and target
config. Remove the no longer needed fallback commands implementations.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 hmp.c                |  2 ++
 monitor.c            | 22 ----------------------
 qmp.c                | 32 --------------------------------
 hmp-commands-info.hx |  2 ++
 qapi-schema.json     | 30 ++++++++++++++++++++++++++++++
 qapi/event.json      |  8 ++++++++
 qmp-commands.hx      |  7 ++++---
 7 files changed, 46 insertions(+), 57 deletions(-)

diff --git a/hmp.c b/hmp.c
index cc2056e..339f5c6 100644
--- a/hmp.c
+++ b/hmp.c
@@ -570,6 +570,7 @@ void hmp_info_blockstats(Monitor *mon, const QDict *qdict)
     qapi_free_BlockStatsList(stats_list);
 }
 
+#ifdef CONFIG_VNC
 void hmp_info_vnc(Monitor *mon, const QDict *qdict)
 {
     VncInfo *info;
@@ -615,6 +616,7 @@ void hmp_info_vnc(Monitor *mon, const QDict *qdict)
 out:
     qapi_free_VncInfo(info);
 }
+#endif
 
 #ifdef CONFIG_SPICE
 void hmp_info_spice(Monitor *mon, const QDict *qdict)
diff --git a/monitor.c b/monitor.c
index 5c00373..289fa5e 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4306,28 +4306,6 @@ QemuOptsList qemu_mon_opts = {
     },
 };
 
-#ifndef TARGET_I386
-void qmp_rtc_reset_reinjection(Error **errp)
-{
-    error_setg(errp, QERR_FEATURE_DISABLED, "rtc-reset-reinjection");
-}
-#endif
-
-#ifndef TARGET_S390X
-void qmp_dump_skeys(const char *filename, Error **errp)
-{
-    error_setg(errp, QERR_FEATURE_DISABLED, "dump-skeys");
-}
-#endif
-
-#ifndef TARGET_ARM
-GICCapabilityList *qmp_query_gic_capabilities(Error **errp)
-{
-    error_setg(errp, QERR_FEATURE_DISABLED, "query-gic-capabilities");
-    return NULL;
-}
-#endif
-
 HotpluggableCPUList *qmp_query_hotpluggable_cpus(Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
diff --git a/qmp.c b/qmp.c
index b6d531e..01671ad 100644
--- a/qmp.c
+++ b/qmp.c
@@ -145,38 +145,6 @@ void qmp_cpu_add(int64_t id, Error **errp)
     }
 }
 
-#ifndef CONFIG_VNC
-/* If VNC support is enabled, the "true" query-vnc command is
-   defined in the VNC subsystem */
-VncInfo *qmp_query_vnc(Error **errp)
-{
-    error_setg(errp, QERR_FEATURE_DISABLED, "vnc");
-    return NULL;
-};
-
-VncInfo2List *qmp_query_vnc_servers(Error **errp)
-{
-    error_setg(errp, QERR_FEATURE_DISABLED, "vnc");
-    return NULL;
-};
-#endif
-
-#ifndef CONFIG_SPICE
-/*
- * qmp-commands.hx ensures that QMP command query-spice exists only
- * #ifdef CONFIG_SPICE.  Necessary for an accurate query-commands
- * result.  However, the QAPI schema is blissfully unaware of that,
- * and the QAPI code generator happily generates a dead
- * qmp_marshal_query_spice() that calls qmp_query_spice().  Provide it
- * one, or else linking fails.  FIXME Educate the QAPI schema on
- * CONFIG_SPICE.
- */
-SpiceInfo *qmp_query_spice(Error **errp)
-{
-    abort();
-};
-#endif
-
 void qmp_cont(Error **errp)
 {
     Error *local_err = NULL;
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 74446c6..be6e13a 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -430,6 +430,7 @@ STEXI
 Show which guest mouse is receiving events.
 ETEXI
 
+#if defined(CONFIG_VNC)
     {
         .name       = "vnc",
         .args_type  = "",
@@ -437,6 +438,7 @@ ETEXI
         .help       = "show the vnc server status",
         .mhandler.cmd = hmp_info_vnc,
     },
+#endif
 
 STEXI
 @item info vnc
diff --git a/qapi-schema.json b/qapi-schema.json
index 5658723..caad127 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -973,6 +973,8 @@
 { 'enum': 'NetworkAddressFamily',
   'data': [ 'ipv4', 'ipv6', 'unix', 'unknown' ] }
 
+#ifdef CONFIG_VNC
+
 ##
 # @VncBasicInfo
 #
@@ -1146,6 +1148,10 @@
 ##
 { 'command': 'query-vnc-servers', 'returns': ['VncInfo2'] }
 
+#endif CONFIG_VNC
+
+#ifdef CONFIG_SPICE
+
 ##
 # @SpiceBasicInfo
 #
@@ -1271,6 +1277,8 @@
 ##
 { 'command': 'query-spice', 'returns': 'SpiceInfo' }
 
+#endif CONFIG_SPICE
+
 ##
 # @BalloonInfo:
 #
@@ -2000,6 +2008,8 @@
 ##
 { 'command': 'expire_password', 'data': {'protocol': 'str', 'time': 'str'} }
 
+#ifdef CONFIG_VNC
+
 ##
 # @change-vnc-password:
 #
@@ -2014,6 +2024,8 @@
 ##
 { 'command': 'change-vnc-password', 'data': {'password': 'str'} }
 
+#endif
+
 ##
 # @change:
 #
@@ -2343,6 +2355,8 @@
 { 'command': 'query-dump-guest-memory-capability',
   'returns': 'DumpGuestMemoryCapability' }
 
+#ifdef TARGET_S390X
+
 ##
 # @dump-skeys
 #
@@ -2357,6 +2371,8 @@
 { 'command': 'dump-skeys',
   'data': { 'filename': 'str' } }
 
+#endif TARGET_S390X
+
 ##
 # @netdev_add:
 #
@@ -3371,6 +3387,8 @@
   'base': 'ChardevCommon' }
 
 
+#ifdef CONFIG_SPICE
+
 ##
 # @ChardevSpiceChannel:
 #
@@ -3395,6 +3413,8 @@
 { 'struct': 'ChardevSpicePort', 'data': { 'fqdn'  : 'str' },
   'base': 'ChardevCommon' }
 
+#endif CONFIG_SPICE
+
 ##
 # @ChardevVC:
 #
@@ -3446,8 +3466,10 @@
                                        'testdev': 'ChardevCommon',
                                        'stdio'  : 'ChardevStdio',
                                        'console': 'ChardevCommon',
+#ifdef CONFIG_SPICE
                                        'spicevmc' : 'ChardevSpiceChannel',
                                        'spiceport' : 'ChardevSpicePort',
+#endif
                                        'vc'     : 'ChardevVC',
                                        'ringbuf': 'ChardevRingbuf',
                                        # next one is just for compatibility
@@ -4200,6 +4222,8 @@
 { 'enum': 'GuestPanicAction',
   'data': [ 'pause' ] }
 
+#ifdef TARGET_I386
+
 ##
 # @rtc-reset-reinjection
 #
@@ -4212,6 +4236,8 @@
 ##
 { 'command': 'rtc-reset-reinjection' }
 
+#endif TARGET_I386
+
 # Rocker ethernet network switch
 { 'include': 'qapi/rocker.json' }
 
@@ -4271,6 +4297,8 @@
             'emulated': 'bool',
             'kernel': 'bool' } }
 
+#ifdef TARGET_ARM
+
 ##
 # @query-gic-capabilities:
 #
@@ -4283,6 +4311,8 @@
 ##
 { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
 
+#endif TARGET_ARM
+
 ##
 # CpuInstanceProperties
 #
diff --git a/qapi/event.json b/qapi/event.json
index 8642052..712045f 100644
--- a/qapi/event.json
+++ b/qapi/event.json
@@ -138,6 +138,8 @@
 { 'event': 'NIC_RX_FILTER_CHANGED',
   'data': { '*name': 'str', 'path': 'str' } }
 
+#ifdef CONFIG_VNC
+
 ##
 # @VNC_CONNECTED
 #
@@ -187,6 +189,10 @@
   'data': { 'server': 'VncServerInfo',
             'client': 'VncClientInfo' } }
 
+#endif CONFIG_VNC
+
+#ifdef CONFIG_SPICE
+
 ##
 # @SPICE_CONNECTED
 #
@@ -242,6 +248,8 @@
 ##
 { 'event': 'SPICE_MIGRATE_COMPLETED' }
 
+#endif
+
 ##
 # @MIGRATION
 #
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 6866264..0eb0d9c 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3347,7 +3347,7 @@ Example:
    }
 
 EQMP
-
+#if defined(CONFIG_VNC)
     {
         .name       = "query-vnc",
         .args_type  = "",
@@ -3358,7 +3358,7 @@ EQMP
         .args_type  = "",
         .mhandler.cmd_new = qmp_marshal_query_vnc_servers,
     },
-
+#endif
 SQMP
 query-spice
 -----------
@@ -3911,12 +3911,13 @@ EQMP
         .args_type  = "",
         .mhandler.cmd_new = qmp_marshal_nbd_server_stop,
     },
-
+#if defined(CONFIG_VNC)
     {
         .name       = "change-vnc-password",
         .args_type  = "password:s",
         .mhandler.cmd_new = qmp_marshal_change_vnc_password,
     },
+#endif
     {
         .name       = "qom-list-types",
         .args_type  = "implements:s?,abstract:b?",
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 06/20] build-sys: define QEMU_VERSION_{MAJOR, MINOR, MICRO}
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (4 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 05/20] qapi: configure the schema Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 07/20] qapi-schema: use generated marshaller for 'qmp_capabilities' Marc-André Lureau
                   ` (13 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

There are better chances to find what went wrong at build time than a
later assert in qmp_query_version

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 qmp.c                 | 16 +++-------------
 scripts/create_config |  6 ++++++
 2 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/qmp.c b/qmp.c
index 01671ad..03d1374 100644
--- a/qmp.c
+++ b/qmp.c
@@ -51,21 +51,11 @@ NameInfo *qmp_query_name(Error **errp)
 VersionInfo *qmp_query_version(Error **errp)
 {
     VersionInfo *info = g_new0(VersionInfo, 1);
-    const char *version = QEMU_VERSION;
-    const char *tmp;
-    int err;
 
     info->qemu = g_new0(VersionTriple, 1);
-    err = qemu_strtoll(version, &tmp, 10, &info->qemu->major);
-    assert(err == 0);
-    tmp++;
-
-    err = qemu_strtoll(tmp, &tmp, 10, &info->qemu->minor);
-    assert(err == 0);
-    tmp++;
-
-    err = qemu_strtoll(tmp, &tmp, 10, &info->qemu->micro);
-    assert(err == 0);
+    info->qemu->major = QEMU_VERSION_MAJOR;
+    info->qemu->minor = QEMU_VERSION_MINOR;
+    info->qemu->micro = QEMU_VERSION_MICRO;
     info->package = g_strdup(QEMU_PKGVERSION);
 
     return info;
diff --git a/scripts/create_config b/scripts/create_config
index 1dd6a35..e6929dd 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -7,7 +7,13 @@ while read line; do
 case $line in
  VERSION=*) # configuration
     version=${line#*=}
+    major=$(echo "$version" | cut -d. -f1)
+    minor=$(echo "$version" | cut -d. -f2)
+    micro=$(echo "$version" | cut -d. -f3)
     echo "#define QEMU_VERSION \"$version\""
+    echo "#define QEMU_VERSION_MAJOR $major"
+    echo "#define QEMU_VERSION_MINOR $minor"
+    echo "#define QEMU_VERSION_MICRO $micro"
     ;;
  qemu_*dir=*) # qemu-specific directory configuration
     name=${line%=*}
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 07/20] qapi-schema: use generated marshaller for 'qmp_capabilities'
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (5 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 06/20] build-sys: define QEMU_VERSION_{MAJOR, MINOR, MICRO} Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 08/20] qapi-schema: add 'device_add' Marc-André Lureau
                   ` (12 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

qapi'fy the 'qmp_capabilities' command, makes the command visible in
query-qmp-schema.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
 monitor.c        |  4 ++--
 qapi-schema.json | 21 +++++++++++++++++++++
 qmp-commands.hx  |  2 +-
 3 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/monitor.c b/monitor.c
index 289fa5e..4a6f97d 100644
--- a/monitor.c
+++ b/monitor.c
@@ -617,7 +617,7 @@ static void monitor_qapi_event_init(void)
     qmp_event_set_func_emit(monitor_qapi_event_queue);
 }
 
-static void qmp_capabilities(QDict *params, QObject **ret_data, Error **errp)
+void qmp_qmp_capabilities(Error **errp)
 {
     cur_mon->qmp.in_command_mode = true;
 }
@@ -3656,7 +3656,7 @@ static int monitor_can_read(void *opaque)
 static bool invalid_qmp_mode(const Monitor *mon, const mon_cmd_t *cmd,
                              Error **errp)
 {
-    bool is_cap = cmd->mhandler.cmd_new == qmp_capabilities;
+    bool is_cap = cmd->mhandler.cmd_new == qmp_marshal_qmp_capabilities;
 
     if (is_cap && mon->qmp.in_command_mode) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
diff --git a/qapi-schema.json b/qapi-schema.json
index caad127..1eb8db5 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -20,6 +20,27 @@
 # QAPI introspection
 { 'include': 'qapi/introspect.json' }
 
+##
+# @qmp_capabilities:
+#
+# Enable QMP capabilities.
+#
+# Arguments: None.
+#
+# Example:
+#
+# -> { "execute": "qmp_capabilities" }
+# <- { "return": {} }
+#
+# Notes: This command is valid exactly when first connecting: it must be
+# issued before any other command will be accepted, and will fail once the
+# monitor is accepting other commands. (see qemu docs/qmp-spec.txt)
+#
+# Since: 0.13
+#
+##
+{ 'command': 'qmp_capabilities' }
+
 ##
 # @LostTickPolicy:
 #
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 0eb0d9c..5336ec2 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -2209,7 +2209,7 @@ EQMP
         .args_type  = "",
         .params     = "",
         .help       = "enable QMP capabilities",
-        .mhandler.cmd_new = qmp_capabilities,
+        .mhandler.cmd_new = qmp_marshal_qmp_capabilities,
     },
 
 SQMP
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 08/20] qapi-schema: add 'device_add'
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (6 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 07/20] qapi-schema: use generated marshaller for 'qmp_capabilities' Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 09/20] monitor: simplify invalid_qmp_mode() Marc-André Lureau
                   ` (11 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Even though device_add is not fully qapi'fied, we may add it to the json
schema with 'gen': false, so registration and documentation can be
generated.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 qapi-schema.json | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/qapi-schema.json b/qapi-schema.json
index 1eb8db5..a2fe8c2 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2212,6 +2212,46 @@
 ##
 { 'command': 'xen-set-global-dirty-log', 'data': { 'enable': 'bool' } }
 
+##
+# @device_add:
+#
+# @driver: the name of the new device's driver
+#
+# @bus: #optional the device's parent bus (device tree path)
+#
+# @id: the device's ID, must be unique
+#
+# Additional arguments depend on the type.
+#
+# Add a device.
+#
+# Notes:
+# 1. For detailed information about this command, please refer to the
+#    'docs/qdev-device-use.txt' file.
+#
+# 2. It's possible to list device properties by running QEMU with the
+#    "-device DEVICE,help" command-line argument, where DEVICE is the
+#    device's name
+#
+# Example:
+#
+# -> { "execute": "device_add",
+#      "arguments": { "driver": "e1000", "id": "net1",
+#                     "bus": "pci.0",
+#                     "mac": "52:54:00:12:34:56" } }
+# <- { "return": {} }
+#
+# TODO This command effectively bypasses QAPI completely due to its
+# "additional arguments" business.  It shouldn't have been added to
+# the schema in this form.  It should be qapified properly, or
+# replaced by a properly qapified command.
+#
+# Since: 0.13
+##
+{ 'command': 'device_add',
+  'data': {'driver': 'str', 'id': 'str'},
+  'gen': false } # so we can get the additional arguments
+
 ##
 # @device_del:
 #
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 09/20] monitor: simplify invalid_qmp_mode()
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (7 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 08/20] qapi-schema: add 'device_add' Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 10/20] monitor: register gen:false commands manually Marc-André Lureau
                   ` (10 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

handle_qmp_command() will switch to use qmp_dispatch().  It won't have a
pointer to the marshaller function anymore, but only the name of the
command to invoke. Simplify invalid_qmp_mode() so it can just be called
with the command name.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/monitor.c b/monitor.c
index 4a6f97d..aad44c7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3653,21 +3653,21 @@ static int monitor_can_read(void *opaque)
     return (mon->suspend_cnt == 0) ? 1 : 0;
 }
 
-static bool invalid_qmp_mode(const Monitor *mon, const mon_cmd_t *cmd,
+static bool invalid_qmp_mode(const Monitor *mon, const char *cmd,
                              Error **errp)
 {
-    bool is_cap = cmd->mhandler.cmd_new == qmp_marshal_qmp_capabilities;
+    bool is_cap = g_str_equal(cmd, "qmp_capabilities");
 
     if (is_cap && mon->qmp.in_command_mode) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
                   "Capabilities negotiation is already complete, command "
-                  "'%s' ignored", cmd->name);
+                  "'%s' ignored", cmd);
         return true;
     }
     if (!is_cap && !mon->qmp.in_command_mode) {
         error_set(errp, ERROR_CLASS_COMMAND_NOT_FOUND,
                   "Expecting capabilities negotiation with "
-                  "'qmp_capabilities' before command '%s'", cmd->name);
+                  "'qmp_capabilities' before command '%s'", cmd);
         return true;
     }
     return false;
@@ -3958,7 +3958,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
                   "The command %s has not been found", cmd_name);
         goto err_out;
     }
-    if (invalid_qmp_mode(mon, cmd, &local_err)) {
+    if (invalid_qmp_mode(mon, cmd_name, &local_err)) {
         goto err_out;
     }
 
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 10/20] monitor: register gen:false commands manually
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (8 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 09/20] monitor: simplify invalid_qmp_mode() Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 11/20] qapi: export the marshallers Marc-André Lureau
                   ` (9 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Since a few commands are using 'gen': false, they are not registered
automatically by the generator. Register manually instead.

This is in preparation for removal of qapi 'middle' mode generation.

Note that qmp_init_marshal() function isn't run yet, so the commands
aren't actually registered, until module_call_init(MODULE_INIT_QAPI) is
added in a later patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/monitor.c b/monitor.c
index aad44c7..598b389 100644
--- a/monitor.c
+++ b/monitor.c
@@ -79,6 +79,7 @@
 #include "sysemu/block-backend.h"
 #include "sysemu/qtest.h"
 #include "qemu/cutils.h"
+#include "qapi/qmp/dispatch.h"
 
 /* for hmp_info_irq/pic */
 #if defined(TARGET_SPARC)
@@ -1007,6 +1008,18 @@ static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
     *ret_data = qobject_from_json(qmp_schema_json);
 }
 
+static void qmp_init_marshal(void)
+{
+    qmp_register_command("query-qmp-schema", qmp_query_qmp_schema,
+                         QCO_NO_OPTIONS);
+    qmp_register_command("device_add", qmp_device_add,
+                         QCO_NO_OPTIONS);
+    qmp_register_command("netdev_add", qmp_netdev_add,
+                         QCO_NO_OPTIONS);
+}
+
+qapi_init(qmp_init_marshal);
+
 /* set the current CPU defined by the user */
 int monitor_set_cpu(int cpu_index)
 {
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 11/20] qapi: export the marshallers
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (9 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 10/20] monitor: register gen:false commands manually Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 12/20] monitor: use qmp_find_command() (using generated qapi code) Marc-André Lureau
                   ` (8 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Make it possible to call marshallers manually, without going through
qmp_dispatch(). (this is currently only possible in middle-mode, but
it's also useful in general)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi-commands.py | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a06a2c4..b150db9 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -84,10 +84,7 @@ static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out,
 
 
 def gen_marshal_proto(name):
-    ret = 'void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
-    if not middle_mode:
-        ret = 'static ' + ret
-    return ret
+    return 'void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
 
 
 def gen_marshal_decl(name):
@@ -222,8 +219,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
         if ret_type and ret_type not in self._visited_ret_types:
             self._visited_ret_types.add(ret_type)
             self.defn += gen_marshal_output(ret_type)
-        if middle_mode:
-            self.decl += gen_marshal_decl(name)
+        self.decl += gen_marshal_decl(name)
         self.defn += gen_marshal(name, arg_type, boxed, ret_type)
         if not middle_mode:
             self._regy += gen_register_command(name, success_response)
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 12/20] monitor: use qmp_find_command() (using generated qapi code)
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (10 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 11/20] qapi: export the marshallers Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 13/20] monitor: implement 'qmp_query_commands' without qmp_cmds Marc-André Lureau
                   ` (7 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Stop using the so-called 'middle' mode. Instead, use qmp_find_command()
from generated qapi commands registry. Update and fix the documentation
too.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 monitor.c                     |  10 ++-
 vl.c                          |   1 +
 Makefile                      |   2 +-
 Makefile.target               |   2 +-
 docs/writing-qmp-commands.txt |   8 +--
 qmp-commands.hx               | 143 ------------------------------------------
 6 files changed, 12 insertions(+), 154 deletions(-)

diff --git a/monitor.c b/monitor.c
index 598b389..e72804a 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3941,6 +3941,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     QObject *obj, *data;
     QDict *input, *args;
     const mon_cmd_t *cmd;
+    QmpCommand *qcmd;
     const char *cmd_name;
     Monitor *mon = cur_mon;
 
@@ -3966,7 +3967,8 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
     cmd_name = qdict_get_str(input, "execute");
     trace_handle_qmp_command(mon, cmd_name);
     cmd = qmp_find_cmd(cmd_name);
-    if (!cmd) {
+    qcmd = qmp_find_command(cmd_name);
+    if (!qcmd || !cmd) {
         error_set(&local_err, ERROR_CLASS_COMMAND_NOT_FOUND,
                   "The command %s has not been found", cmd_name);
         goto err_out;
@@ -3988,7 +3990,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
         goto err_out;
     }
 
-    cmd->mhandler.cmd_new(args, &data, &local_err);
+    qcmd->fn(args, &data, &local_err);
 
 err_out:
     monitor_protocol_emitter(mon, data, local_err);
@@ -4060,7 +4062,9 @@ static QObject *get_qmp_greeting(void)
     QObject *ver = NULL;
 
     qmp_marshal_query_version(NULL, &ver, NULL);
-    return qobject_from_jsonf("{'QMP':{'version': %p,'capabilities': []}}",ver);
+
+    return qobject_from_jsonf("{'QMP': {'version': %p, 'capabilities': []}}",
+                              ver);
 }
 
 static void monitor_qmp_event(void *opaque, int event)
diff --git a/vl.c b/vl.c
index b3c80d5..04dc336 100644
--- a/vl.c
+++ b/vl.c
@@ -2973,6 +2973,7 @@ int main(int argc, char **argv, char **envp)
     qemu_init_exec_dir(argv[0]);
 
     module_call_init(MODULE_INIT_QOM);
+    module_call_init(MODULE_INIT_QAPI);
 
     qemu_add_opts(&qemu_drive_opts);
     qemu_add_drive_opts(&qemu_legacy_drive_opts);
diff --git a/Makefile b/Makefile
index f83b208..fc2e53a 100644
--- a/Makefile
+++ b/Makefile
@@ -306,7 +306,7 @@ $(qapi-modules) $(qapi-event-py)
 qmp-commands.h :\
 $(qapi-modules) $(qapi-commands-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(qapi-gen-type) -o "." -m $<, \
+		$(qapi-gen-type) -o "." $<, \
 		"  GEN   $@")
 
 QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
diff --git a/Makefile.target b/Makefile.target
index c6c20bf..0ba50ae 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -166,7 +166,7 @@ qapi-flags = $(patsubst %,-f %,$(qapi-hconfig))
 qmp-marshal.c :\
 $(qapi-modules) $(qapi-commands-py) $(qapi-hconfig)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
-		$(qapi-flags) $(qapi-gen-type) -o "." -m $<, \
+		$(qapi-flags) $(qapi-gen-type) -o "." $<, \
 		"  GEN   $(TARGET_DIR)$@")
 
 qapi-event.c :\
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index 59aa77a..0a66e0e 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -127,7 +127,6 @@ following at the bottom:
     {
         .name       = "hello-world",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_hello_world,
     },
 
 You're done. Now build qemu, run it as suggested in the "Testing" section,
@@ -179,7 +178,6 @@ The last step is to update the qmp-commands.hx file:
     {
         .name       = "hello-world",
         .args_type  = "message:s?",
-        .mhandler.cmd_new = qmp_marshal_hello_world,
     },
 
 Notice that the "args_type" member got our "message" argument. The character
@@ -454,12 +452,11 @@ There are a number of things to be noticed:
 6. You have to include the "qmp-commands.h" header file in qemu-timer.c,
    otherwise qemu won't build
 
-The last step is to add the correspoding entry in the qmp-commands.hx file:
+The last step is to add the corresponding entry in the qmp-commands.hx file:
 
     {
         .name       = "query-alarm-clock",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_alarm_clock,
     },
 
 Time to test the new command. Build qemu, run it as described in the "Testing"
@@ -518,7 +515,7 @@ in the monitor.c file. The entry for the "info alarmclock" follows:
         .args_type  = "",
         .params     = "",
         .help       = "show information about the alarm clock",
-        .mhandler.info = hmp_info_alarm_clock,
+        .mhandler.cmd = hmp_info_alarm_clock,
     },
 
 To test this, run qemu and type "info alarmclock" in the user monitor.
@@ -605,7 +602,6 @@ To test this you have to add the corresponding qmp-commands.hx entry:
     {
         .name       = "query-alarm-methods",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_alarm_methods,
     },
 
 Now Build qemu, run it as explained in the "Testing" section and try our new
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 5336ec2..f395f47 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -63,7 +63,6 @@ EQMP
     {
         .name       = "quit",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_quit,
     },
 
 SQMP
@@ -84,7 +83,6 @@ EQMP
     {
         .name       = "eject",
         .args_type  = "force:-f,device:B",
-        .mhandler.cmd_new = qmp_marshal_eject,
     },
 
 SQMP
@@ -110,7 +108,6 @@ EQMP
     {
         .name       = "change",
         .args_type  = "device:B,target:F,arg:s?",
-        .mhandler.cmd_new = qmp_marshal_change,
     },
 
 SQMP
@@ -146,7 +143,6 @@ EQMP
     {
         .name       = "screendump",
         .args_type  = "filename:F",
-        .mhandler.cmd_new = qmp_marshal_screendump,
     },
 
 SQMP
@@ -169,7 +165,6 @@ EQMP
     {
         .name       = "stop",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_stop,
     },
 
 SQMP
@@ -190,7 +185,6 @@ EQMP
     {
         .name       = "cont",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_cont,
     },
 
 SQMP
@@ -211,7 +205,6 @@ EQMP
     {
         .name       = "system_wakeup",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_system_wakeup,
     },
 
 SQMP
@@ -232,7 +225,6 @@ EQMP
     {
         .name       = "system_reset",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_system_reset,
     },
 
 SQMP
@@ -253,7 +245,6 @@ EQMP
     {
         .name       = "system_powerdown",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_system_powerdown,
     },
 
 SQMP
@@ -276,7 +267,6 @@ EQMP
         .args_type  = "device:O",
         .params     = "driver[,prop=value][,...]",
         .help       = "add device, like -device on the command line",
-        .mhandler.cmd_new = qmp_device_add,
     },
 
 SQMP
@@ -310,7 +300,6 @@ EQMP
     {
         .name       = "device_del",
         .args_type  = "id:s",
-        .mhandler.cmd_new = qmp_marshal_device_del,
     },
 
 SQMP
@@ -338,7 +327,6 @@ EQMP
     {
         .name       = "send-key",
         .args_type  = "keys:q,hold-time:i?",
-        .mhandler.cmd_new = qmp_marshal_send_key,
     },
 
 SQMP
@@ -369,7 +357,6 @@ EQMP
     {
         .name       = "cpu",
         .args_type  = "index:i",
-        .mhandler.cmd_new = qmp_marshal_cpu,
     },
 
 SQMP
@@ -394,7 +381,6 @@ EQMP
     {
         .name       = "cpu-add",
         .args_type  = "id:i",
-        .mhandler.cmd_new = qmp_marshal_cpu_add,
     },
 
 SQMP
@@ -417,7 +403,6 @@ EQMP
     {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
-        .mhandler.cmd_new = qmp_marshal_memsave,
     },
 
 SQMP
@@ -446,7 +431,6 @@ EQMP
     {
         .name       = "pmemsave",
         .args_type  = "val:l,size:i,filename:s",
-        .mhandler.cmd_new = qmp_marshal_pmemsave,
     },
 
 SQMP
@@ -474,7 +458,6 @@ EQMP
     {
         .name       = "inject-nmi",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_inject_nmi,
     },
 
 SQMP
@@ -497,7 +480,6 @@ EQMP
     {
         .name       = "ringbuf-write",
         .args_type  = "device:s,data:s,format:s?",
-        .mhandler.cmd_new = qmp_marshal_ringbuf_write,
     },
 
 SQMP
@@ -526,7 +508,6 @@ EQMP
     {
         .name       = "ringbuf-read",
         .args_type  = "device:s,size:i,format:s?",
-        .mhandler.cmd_new = qmp_marshal_ringbuf_read,
     },
 
 SQMP
@@ -562,7 +543,6 @@ EQMP
     {
         .name       = "xen-save-devices-state",
         .args_type  = "filename:F",
-    .mhandler.cmd_new = qmp_marshal_xen_save_devices_state,
     },
 
 SQMP
@@ -589,7 +569,6 @@ EQMP
     {
         .name       = "xen-load-devices-state",
         .args_type  = "filename:F",
-        .mhandler.cmd_new = qmp_marshal_xen_load_devices_state,
     },
 
 SQMP
@@ -616,7 +595,6 @@ EQMP
     {
         .name       = "xen-set-global-dirty-log",
         .args_type  = "enable:b",
-        .mhandler.cmd_new = qmp_marshal_xen_set_global_dirty_log,
     },
 
 SQMP
@@ -640,7 +618,6 @@ EQMP
     {
         .name       = "migrate",
         .args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
-        .mhandler.cmd_new = qmp_marshal_migrate,
     },
 
 SQMP
@@ -673,7 +650,6 @@ EQMP
     {
         .name       = "migrate_cancel",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_migrate_cancel,
     },
 
 SQMP
@@ -694,7 +670,6 @@ EQMP
     {
         .name       = "migrate-incoming",
         .args_type  = "uri:s",
-        .mhandler.cmd_new = qmp_marshal_migrate_incoming,
     },
 
 SQMP
@@ -722,7 +697,6 @@ EQMP
     {
         .name       = "migrate-set-cache-size",
         .args_type  = "value:o",
-        .mhandler.cmd_new = qmp_marshal_migrate_set_cache_size,
     },
 
 SQMP
@@ -745,7 +719,6 @@ EQMP
     {
         .name       = "migrate-start-postcopy",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_migrate_start_postcopy,
     },
 
 SQMP
@@ -764,7 +737,6 @@ EQMP
     {
         .name       = "query-migrate-cache-size",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_migrate_cache_size,
     },
 
 SQMP
@@ -786,7 +758,6 @@ EQMP
     {
         .name       = "migrate_set_speed",
         .args_type  = "value:o",
-        .mhandler.cmd_new = qmp_marshal_migrate_set_speed,
     },
 
 SQMP
@@ -809,7 +780,6 @@ EQMP
     {
         .name       = "migrate_set_downtime",
         .args_type  = "value:T",
-        .mhandler.cmd_new = qmp_marshal_migrate_set_downtime,
     },
 
 SQMP
@@ -834,7 +804,6 @@ EQMP
         .args_type  = "protocol:s,hostname:s,port:i?,tls-port:i?,cert-subject:s?",
         .params     = "protocol hostname port tls-port cert-subject",
         .help       = "set migration information for remote display",
-        .mhandler.cmd_new = qmp_marshal_client_migrate_info,
     },
 
 SQMP
@@ -868,7 +837,6 @@ EQMP
         .args_type  = "paging:b,protocol:s,detach:b?,begin:i?,end:i?,format:s?",
         .params     = "-p protocol [-d] [begin] [length] [format]",
         .help       = "dump guest memory to file",
-        .mhandler.cmd_new = qmp_marshal_dump_guest_memory,
     },
 
 SQMP
@@ -907,7 +875,6 @@ EQMP
     {
         .name       = "query-dump-guest-memory-capability",
         .args_type  = "",
-    .mhandler.cmd_new = qmp_marshal_query_dump_guest_memory_capability,
     },
 
 SQMP
@@ -929,7 +896,6 @@ EQMP
         .args_type  = "",
         .params     = "",
         .help       = "query background dump status",
-        .mhandler.cmd_new = qmp_marshal_query_dump,
     },
 
 SQMP
@@ -952,7 +918,6 @@ EQMP
     {
         .name       = "dump-skeys",
         .args_type  = "filename:F",
-        .mhandler.cmd_new = qmp_marshal_dump_skeys,
     },
 #endif
 
@@ -976,7 +941,6 @@ EQMP
     {
         .name       = "netdev_add",
         .args_type  = "netdev:O",
-        .mhandler.cmd_new = qmp_netdev_add,
     },
 
 SQMP
@@ -1007,7 +971,6 @@ EQMP
     {
         .name       = "netdev_del",
         .args_type  = "id:s",
-        .mhandler.cmd_new = qmp_marshal_netdev_del,
     },
 
 SQMP
@@ -1031,7 +994,6 @@ EQMP
     {
         .name       = "object-add",
         .args_type  = "qom-type:s,id:s,props:q?",
-        .mhandler.cmd_new = qmp_marshal_object_add,
     },
 
 SQMP
@@ -1057,7 +1019,6 @@ EQMP
     {
         .name       = "object-del",
         .args_type  = "id:s",
-        .mhandler.cmd_new = qmp_marshal_object_del,
     },
 
 SQMP
@@ -1082,7 +1043,6 @@ EQMP
     {
         .name       = "block_resize",
         .args_type  = "device:s?,node-name:s?,size:o",
-        .mhandler.cmd_new = qmp_marshal_block_resize,
     },
 
 SQMP
@@ -1107,7 +1067,6 @@ EQMP
     {
         .name       = "block-stream",
         .args_type  = "job-id:s?,device:B,base:s?,speed:o?,backing-file:s?,on-error:s?",
-        .mhandler.cmd_new = qmp_marshal_block_stream,
     },
 
 SQMP
@@ -1152,7 +1111,6 @@ EQMP
     {
         .name       = "block-commit",
         .args_type  = "job-id:s?,device:B,base:s?,top:s?,backing-file:s?,speed:o?",
-        .mhandler.cmd_new = qmp_marshal_block_commit,
     },
 
 SQMP
@@ -1218,7 +1176,6 @@ EQMP
         .name       = "drive-backup",
         .args_type  = "job-id:s?,sync:s,device:B,target:s,speed:i?,mode:s?,"
                       "format:s?,bitmap:s?,on-source-error:s?,on-target-error:s?",
-        .mhandler.cmd_new = qmp_marshal_drive_backup,
     },
 
 SQMP
@@ -1274,7 +1231,6 @@ EQMP
         .name       = "blockdev-backup",
         .args_type  = "job-id:s?,sync:s,device:B,target:B,speed:i?,"
                       "on-source-error:s?,on-target-error:s?",
-        .mhandler.cmd_new = qmp_marshal_blockdev_backup,
     },
 
 SQMP
@@ -1316,33 +1272,27 @@ EQMP
     {
         .name       = "block-job-set-speed",
         .args_type  = "device:B,speed:o",
-        .mhandler.cmd_new = qmp_marshal_block_job_set_speed,
     },
 
     {
         .name       = "block-job-cancel",
         .args_type  = "device:B,force:b?",
-        .mhandler.cmd_new = qmp_marshal_block_job_cancel,
     },
     {
         .name       = "block-job-pause",
         .args_type  = "device:B",
-        .mhandler.cmd_new = qmp_marshal_block_job_pause,
     },
     {
         .name       = "block-job-resume",
         .args_type  = "device:B",
-        .mhandler.cmd_new = qmp_marshal_block_job_resume,
     },
     {
         .name       = "block-job-complete",
         .args_type  = "device:B",
-        .mhandler.cmd_new = qmp_marshal_block_job_complete,
     },
     {
         .name       = "transaction",
         .args_type  = "actions:q,properties:q?",
-        .mhandler.cmd_new = qmp_marshal_transaction,
     },
 
 SQMP
@@ -1436,7 +1386,6 @@ EQMP
     {
         .name       = "block-dirty-bitmap-add",
         .args_type  = "node:B,name:s,granularity:i?",
-        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
     },
 
 SQMP
@@ -1464,7 +1413,6 @@ EQMP
     {
         .name       = "block-dirty-bitmap-remove",
         .args_type  = "node:B,name:s",
-        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_remove,
     },
 
 SQMP
@@ -1492,7 +1440,6 @@ EQMP
     {
         .name       = "block-dirty-bitmap-clear",
         .args_type  = "node:B,name:s",
-        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_clear,
     },
 
 SQMP
@@ -1521,7 +1468,6 @@ EQMP
     {
         .name       = "blockdev-snapshot-sync",
         .args_type  = "device:s?,node-name:s?,snapshot-file:s,snapshot-node-name:s?,format:s?,mode:s?",
-        .mhandler.cmd_new = qmp_marshal_blockdev_snapshot_sync,
     },
 
 SQMP
@@ -1557,7 +1503,6 @@ EQMP
     {
         .name       = "blockdev-snapshot",
         .args_type  = "node:s,overlay:s",
-        .mhandler.cmd_new = qmp_marshal_blockdev_snapshot,
     },
 
 SQMP
@@ -1595,7 +1540,6 @@ EQMP
     {
         .name       = "blockdev-snapshot-internal-sync",
         .args_type  = "device:B,name:s",
-        .mhandler.cmd_new = qmp_marshal_blockdev_snapshot_internal_sync,
     },
 
 SQMP
@@ -1624,8 +1568,6 @@ EQMP
     {
         .name       = "blockdev-snapshot-delete-internal-sync",
         .args_type  = "device:B,id:s?,name:s?",
-        .mhandler.cmd_new =
-                      qmp_marshal_blockdev_snapshot_delete_internal_sync,
     },
 
 SQMP
@@ -1669,7 +1611,6 @@ EQMP
                       "on-source-error:s?,on-target-error:s?,"
                       "unmap:b?,"
                       "granularity:i?,buf-size:i?",
-        .mhandler.cmd_new = qmp_marshal_drive_mirror,
     },
 
 SQMP
@@ -1733,7 +1674,6 @@ EQMP
         .args_type  = "job-id:s?,sync:s,device:B,target:B,replaces:s?,speed:i?,"
                       "on-source-error:s?,on-target-error:s?,"
                       "granularity:i?,buf-size:i?",
-        .mhandler.cmd_new = qmp_marshal_blockdev_mirror,
     },
 
 SQMP
@@ -1781,7 +1721,6 @@ EQMP
     {
         .name       = "change-backing-file",
         .args_type  = "device:s,image-node-name:s,backing-file:s",
-        .mhandler.cmd_new = qmp_marshal_change_backing_file,
     },
 
 SQMP
@@ -1820,7 +1759,6 @@ EQMP
     {
         .name       = "balloon",
         .args_type  = "value:M",
-        .mhandler.cmd_new = qmp_marshal_balloon,
     },
 
 SQMP
@@ -1843,7 +1781,6 @@ EQMP
     {
         .name       = "set_link",
         .args_type  = "name:s,up:b",
-        .mhandler.cmd_new = qmp_marshal_set_link,
     },
 
 SQMP
@@ -1869,7 +1806,6 @@ EQMP
         .args_type  = "fdname:s",
         .params     = "getfd name",
         .help       = "receive a file descriptor via SCM rights and assign it a name",
-        .mhandler.cmd_new = qmp_marshal_getfd,
     },
 
 SQMP
@@ -1902,7 +1838,6 @@ EQMP
         .args_type  = "fdname:s",
         .params     = "closefd name",
         .help       = "close a file descriptor previously passed via SCM rights",
-        .mhandler.cmd_new = qmp_marshal_closefd,
     },
 
 SQMP
@@ -1927,7 +1862,6 @@ EQMP
         .args_type  = "fdset-id:i?,opaque:s?",
         .params     = "add-fd fdset-id opaque",
         .help       = "Add a file descriptor, that was passed via SCM rights, to an fd set",
-        .mhandler.cmd_new = qmp_marshal_add_fd,
     },
 
 SQMP
@@ -1966,7 +1900,6 @@ EQMP
         .args_type  = "fdset-id:i,fd:i?",
         .params     = "remove-fd fdset-id fd",
         .help       = "Remove a file descriptor from an fd set",
-        .mhandler.cmd_new = qmp_marshal_remove_fd,
     },
 
 SQMP
@@ -1998,7 +1931,6 @@ EQMP
         .name       = "query-fdsets",
         .args_type  = "",
         .help       = "Return information describing all fd sets",
-        .mhandler.cmd_new = qmp_marshal_query_fdsets,
     },
 
 SQMP
@@ -2047,7 +1979,6 @@ EQMP
     {
         .name       = "block_passwd",
         .args_type  = "device:s?,node-name:s?,password:s",
-        .mhandler.cmd_new = qmp_marshal_block_passwd,
     },
 
 SQMP
@@ -2073,7 +2004,6 @@ EQMP
     {
         .name       = "block_set_io_throttle",
         .args_type  = "device:B,bps:l,bps_rd:l,bps_wr:l,iops:l,iops_rd:l,iops_wr:l,bps_max:l?,bps_rd_max:l?,bps_wr_max:l?,iops_max:l?,iops_rd_max:l?,iops_wr_max:l?,bps_max_length:l?,bps_rd_max_length:l?,bps_wr_max_length:l?,iops_max_length:l?,iops_rd_max_length:l?,iops_wr_max_length:l?,iops_size:l?,group:s?",
-        .mhandler.cmd_new = qmp_marshal_block_set_io_throttle,
     },
 
 SQMP
@@ -2130,7 +2060,6 @@ EQMP
     {
         .name       = "set_password",
         .args_type  = "protocol:s,password:s,connected:s?",
-        .mhandler.cmd_new = qmp_marshal_set_password,
     },
 
 SQMP
@@ -2156,7 +2085,6 @@ EQMP
     {
         .name       = "expire_password",
         .args_type  = "protocol:s,time:s",
-        .mhandler.cmd_new = qmp_marshal_expire_password,
     },
 
 SQMP
@@ -2181,7 +2109,6 @@ EQMP
     {
         .name       = "add_client",
         .args_type  = "protocol:s,fdname:s,skipauth:b?,tls:b?",
-        .mhandler.cmd_new = qmp_marshal_add_client,
     },
 
 SQMP
@@ -2209,7 +2136,6 @@ EQMP
         .args_type  = "",
         .params     = "",
         .help       = "enable QMP capabilities",
-        .mhandler.cmd_new = qmp_marshal_qmp_capabilities,
     },
 
 SQMP
@@ -2232,7 +2158,6 @@ EQMP
     {
         .name       = "human-monitor-command",
         .args_type  = "command-line:s,cpu-index:i?",
-        .mhandler.cmd_new = qmp_marshal_human_monitor_command,
     },
 
 SQMP
@@ -2311,7 +2236,6 @@ EQMP
     {
         .name       = "query-version",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_version,
     },
 
 SQMP
@@ -2348,7 +2272,6 @@ EQMP
     {
         .name       = "query-commands",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_commands,
     },
 
 SQMP
@@ -2385,7 +2308,6 @@ EQMP
     {
         .name       = "query-events",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_events,
     },
 
 SQMP
@@ -2402,7 +2324,6 @@ EQMP
     {
         .name       = "query-qmp-schema",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_query_qmp_schema,
     },
 
 SQMP
@@ -2447,7 +2368,6 @@ EQMP
     {
         .name       = "query-chardev",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_chardev,
     },
 
 SQMP
@@ -2488,7 +2408,6 @@ EQMP
     {
         .name       = "query-chardev-backends",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_chardev_backends,
     },
 
 SQMP
@@ -2672,7 +2591,6 @@ EQMP
     {
         .name       = "query-block",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_block,
     },
 
 SQMP
@@ -2869,7 +2787,6 @@ EQMP
     {
         .name       = "query-blockstats",
         .args_type  = "query-nodes:b?",
-        .mhandler.cmd_new = qmp_marshal_query_blockstats,
     },
 
 SQMP
@@ -2924,7 +2841,6 @@ EQMP
     {
         .name       = "query-cpus",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_cpus,
     },
 
 SQMP
@@ -2963,7 +2879,6 @@ EQMP
     {
         .name       = "query-iothreads",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_iothreads,
     },
 
 SQMP
@@ -3180,7 +3095,6 @@ EQMP
     {
         .name       = "query-pci",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_pci,
     },
 
 SQMP
@@ -3204,7 +3118,6 @@ EQMP
     {
         .name       = "query-kvm",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_kvm,
     },
 
 SQMP
@@ -3244,7 +3157,6 @@ EQMP
     {
         .name       = "query-status",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_status,
     },
 
 SQMP
@@ -3288,7 +3200,6 @@ EQMP
     {
         .name       = "query-mice",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_mice,
     },
 
 SQMP
@@ -3351,12 +3262,10 @@ EQMP
     {
         .name       = "query-vnc",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_vnc,
     },
     {
         .name       = "query-vnc-servers",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_vnc_servers,
     },
 #endif
 SQMP
@@ -3433,7 +3342,6 @@ EQMP
     {
         .name       = "query-spice",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_spice,
     },
 #endif
 
@@ -3457,7 +3365,6 @@ EQMP
     {
         .name       = "query-name",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_name,
     },
 
 SQMP
@@ -3480,7 +3387,6 @@ EQMP
     {
         .name       = "query-uuid",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_uuid,
     },
 
 SQMP
@@ -3529,7 +3435,6 @@ EQMP
     {
         .name       = "query-command-line-options",
         .args_type  = "option:s?",
-        .mhandler.cmd_new = qmp_marshal_query_command_line_options,
     },
 
 SQMP
@@ -3707,7 +3612,6 @@ EQMP
     {
         .name       = "query-migrate",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_migrate,
     },
 
 SQMP
@@ -3737,7 +3641,6 @@ EQMP
         .name       = "migrate-set-capabilities",
         .args_type  = "capabilities:q",
         .params     = "capability:s,state:b",
-        .mhandler.cmd_new = qmp_marshal_migrate_set_capabilities,
     },
 SQMP
 query-migrate-capabilities
@@ -3774,7 +3677,6 @@ EQMP
     {
         .name       = "query-migrate-capabilities",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_migrate_capabilities,
     },
 
 SQMP
@@ -3804,7 +3706,6 @@ EQMP
         .name       = "migrate-set-parameters",
         .args_type  =
             "compress-level:i?,compress-threads:i?,decompress-threads:i?,cpu-throttle-initial:i?,cpu-throttle-increment:i?",
-        .mhandler.cmd_new = qmp_marshal_migrate_set_parameters,
     },
 SQMP
 query-migrate-parameters
@@ -3841,7 +3742,6 @@ EQMP
     {
         .name       = "query-migrate-parameters",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_migrate_parameters,
     },
 
 SQMP
@@ -3869,89 +3769,74 @@ EQMP
     {
         .name       = "query-balloon",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_balloon,
     },
 
     {
         .name       = "query-block-jobs",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_block_jobs,
     },
 
     {
         .name       = "qom-list",
         .args_type  = "path:s",
-        .mhandler.cmd_new = qmp_marshal_qom_list,
     },
 
     {
         .name       = "qom-set",
 	.args_type  = "path:s,property:s,value:q",
-        .mhandler.cmd_new = qmp_marshal_qom_set,
     },
 
     {
         .name       = "qom-get",
 	.args_type  = "path:s,property:s",
-        .mhandler.cmd_new = qmp_marshal_qom_get,
     },
 
     {
         .name       = "nbd-server-start",
         .args_type  = "addr:q,tls-creds:s?",
-        .mhandler.cmd_new = qmp_marshal_nbd_server_start,
     },
     {
         .name       = "nbd-server-add",
         .args_type  = "device:B,writable:b?",
-        .mhandler.cmd_new = qmp_marshal_nbd_server_add,
     },
     {
         .name       = "nbd-server-stop",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_nbd_server_stop,
     },
 #if defined(CONFIG_VNC)
     {
         .name       = "change-vnc-password",
         .args_type  = "password:s",
-        .mhandler.cmd_new = qmp_marshal_change_vnc_password,
     },
 #endif
     {
         .name       = "qom-list-types",
         .args_type  = "implements:s?,abstract:b?",
-        .mhandler.cmd_new = qmp_marshal_qom_list_types,
     },
 
     {
         .name       = "device-list-properties",
         .args_type  = "typename:s",
-        .mhandler.cmd_new = qmp_marshal_device_list_properties,
     },
 
     {
         .name       = "query-machines",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_machines,
     },
 
     {
         .name       = "query-cpu-definitions",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_cpu_definitions,
     },
 
     {
         .name       = "query-target",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_target,
     },
 
     {
         .name       = "query-tpm",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_tpm,
     },
 
 SQMP
@@ -3985,7 +3870,6 @@ EQMP
     {
         .name       = "query-tpm-models",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_tpm_models,
     },
 
 SQMP
@@ -4006,7 +3890,6 @@ EQMP
     {
         .name       = "query-tpm-types",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_tpm_types,
     },
 
 SQMP
@@ -4027,7 +3910,6 @@ EQMP
     {
         .name       = "chardev-add",
         .args_type  = "id:s,backend:q",
-        .mhandler.cmd_new = qmp_marshal_chardev_add,
     },
 
 SQMP
@@ -4064,7 +3946,6 @@ EQMP
     {
         .name       = "chardev-remove",
         .args_type  = "id:s",
-        .mhandler.cmd_new = qmp_marshal_chardev_remove,
     },
 
 
@@ -4087,7 +3968,6 @@ EQMP
     {
         .name       = "query-rx-filter",
         .args_type  = "name:s?",
-        .mhandler.cmd_new = qmp_marshal_query_rx_filter,
     },
 
 SQMP
@@ -4153,7 +4033,6 @@ EQMP
     {
         .name       = "blockdev-add",
         .args_type  = "options:q",
-        .mhandler.cmd_new = qmp_marshal_blockdev_add,
     },
 
 SQMP
@@ -4212,7 +4091,6 @@ EQMP
     {
         .name       = "x-blockdev-del",
         .args_type  = "id:s?,node-name:s?",
-        .mhandler.cmd_new = qmp_marshal_x_blockdev_del,
     },
 
 SQMP
@@ -4269,7 +4147,6 @@ EQMP
     {
         .name       = "blockdev-open-tray",
         .args_type  = "device:s,force:b?",
-        .mhandler.cmd_new = qmp_marshal_blockdev_open_tray,
     },
 
 SQMP
@@ -4317,7 +4194,6 @@ EQMP
     {
         .name       = "blockdev-close-tray",
         .args_type  = "device:s",
-        .mhandler.cmd_new = qmp_marshal_blockdev_close_tray,
     },
 
 SQMP
@@ -4352,7 +4228,6 @@ EQMP
     {
         .name       = "x-blockdev-remove-medium",
         .args_type  = "device:s",
-        .mhandler.cmd_new = qmp_marshal_x_blockdev_remove_medium,
     },
 
 SQMP
@@ -4400,7 +4275,6 @@ EQMP
     {
         .name       = "x-blockdev-insert-medium",
         .args_type  = "device:s,node-name:s",
-        .mhandler.cmd_new = qmp_marshal_x_blockdev_insert_medium,
     },
 
 SQMP
@@ -4440,7 +4314,6 @@ EQMP
     {
         .name       = "x-blockdev-change",
         .args_type  = "parent:B,child:B?,node:B?",
-        .mhandler.cmd_new = qmp_marshal_x_blockdev_change,
     },
 
 SQMP
@@ -4493,7 +4366,6 @@ EQMP
     {
         .name       = "query-named-block-nodes",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_named_block_nodes,
     },
 
 SQMP
@@ -4555,7 +4427,6 @@ EQMP
     {
         .name       = "blockdev-change-medium",
         .args_type  = "device:B,filename:F,format:s?,read-only-mode:s?",
-        .mhandler.cmd_new = qmp_marshal_blockdev_change_medium,
     },
 
 SQMP
@@ -4608,7 +4479,6 @@ EQMP
     {
         .name       = "query-memdev",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_memdev,
     },
 
 SQMP
@@ -4646,7 +4516,6 @@ EQMP
     {
         .name       = "query-memory-devices",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_memory_devices,
     },
 
 SQMP
@@ -4673,7 +4542,6 @@ EQMP
     {
         .name       = "query-acpi-ospm-status",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_acpi_ospm_status,
     },
 
 SQMP
@@ -4696,7 +4564,6 @@ EQMP
     {
         .name       = "rtc-reset-reinjection",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_rtc_reset_reinjection,
     },
 #endif
 
@@ -4717,7 +4584,6 @@ EQMP
     {
         .name       = "trace-event-get-state",
         .args_type  = "name:s,vcpu:i?",
-        .mhandler.cmd_new = qmp_marshal_trace_event_get_state,
     },
 
 SQMP
@@ -4749,7 +4615,6 @@ EQMP
     {
         .name       = "trace-event-set-state",
         .args_type  = "name:s,enable:b,ignore-unavailable:b?,vcpu:i?",
-        .mhandler.cmd_new = qmp_marshal_trace_event_set_state,
     },
 
 SQMP
@@ -4784,7 +4649,6 @@ EQMP
     {
         .name       = "input-send-event",
         .args_type  = "console:i?,events:q",
-        .mhandler.cmd_new = qmp_marshal_input_send_event,
     },
 
 SQMP
@@ -4848,7 +4712,6 @@ EQMP
     {
         .name       = "block-set-write-threshold",
         .args_type  = "node-name:s,write-threshold:l",
-        .mhandler.cmd_new = qmp_marshal_block_set_write_threshold,
     },
 
 SQMP
@@ -4876,7 +4739,6 @@ EQMP
     {
         .name       = "query-rocker",
         .args_type  = "name:s",
-        .mhandler.cmd_new = qmp_marshal_query_rocker,
     },
 
 SQMP
@@ -4897,7 +4759,6 @@ EQMP
     {
         .name       = "query-rocker-ports",
         .args_type  = "name:s",
-        .mhandler.cmd_new = qmp_marshal_query_rocker_ports,
     },
 
 SQMP
@@ -4922,7 +4783,6 @@ EQMP
     {
         .name       = "query-rocker-of-dpa-flows",
         .args_type  = "name:s,tbl-id:i?",
-        .mhandler.cmd_new = qmp_marshal_query_rocker_of_dpa_flows,
     },
 
 SQMP
@@ -4951,7 +4811,6 @@ EQMP
     {
         .name       = "query-rocker-of-dpa-groups",
         .args_type  = "name:s,type:i?",
-        .mhandler.cmd_new = qmp_marshal_query_rocker_of_dpa_groups,
     },
 
 SQMP
@@ -4982,7 +4841,6 @@ EQMP
     {
         .name       = "query-gic-capabilities",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_gic_capabilities,
     },
 #endif
 
@@ -5006,7 +4864,6 @@ EQMP
     {
         .name       = "query-hotpluggable-cpus",
         .args_type  = "",
-        .mhandler.cmd_new = qmp_marshal_query_hotpluggable_cpus,
     },
 
 SQMP
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 13/20] monitor: implement 'qmp_query_commands' without qmp_cmds
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (11 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 12/20] monitor: use qmp_find_command() (using generated qapi code) Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 14/20] monitor: remove mhandler.cmd_new Marc-André Lureau
                   ` (6 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

One step towards getting rid of the static qmp_cmds table.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 monitor.c | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/monitor.c b/monitor.c
index e72804a..dad49b6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -957,21 +957,28 @@ static void hmp_info_help(Monitor *mon, const QDict *qdict)
     help_cmd(mon, "info");
 }
 
-CommandInfoList *qmp_query_commands(Error **errp)
+static void query_commands_cb(QmpCommand *cmd, void *opaque)
 {
-    CommandInfoList *info, *cmd_list = NULL;
-    const mon_cmd_t *cmd;
-
-    for (cmd = qmp_cmds; cmd->name != NULL; cmd++) {
-        info = g_malloc0(sizeof(*info));
-        info->value = g_malloc0(sizeof(*info->value));
-        info->value->name = g_strdup(cmd->name);
+    CommandInfoList *info, **list = opaque;
 
-        info->next = cmd_list;
-        cmd_list = info;
+    if (!cmd->enabled) {
+        return;
     }
 
-    return cmd_list;
+    info = g_malloc0(sizeof(*info));
+    info->value = g_malloc0(sizeof(*info->value));
+    info->value->name = g_strdup(cmd->name);
+    info->next = *list;
+    *list = info;
+}
+
+CommandInfoList *qmp_query_commands(Error **errp)
+{
+    CommandInfoList *list = NULL;
+
+    qmp_for_each_command(query_commands_cb, &list);
+
+    return list;
 }
 
 EventInfoList *qmp_query_events(Error **errp)
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 14/20] monitor: remove mhandler.cmd_new
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (12 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 13/20] monitor: implement 'qmp_query_commands' without qmp_cmds Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 15/20] qapi: remove the "middle" mode Marc-André Lureau
                   ` (5 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

This is no longer necessary now that we aren't using middle mode
anymore.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 monitor.c                     |  13 +--
 docs/writing-qmp-commands.txt |   4 +-
 hmp-commands-info.hx          | 118 ++++++++++++------------
 hmp-commands.hx               | 208 +++++++++++++++++++++---------------------
 4 files changed, 170 insertions(+), 173 deletions(-)

diff --git a/monitor.c b/monitor.c
index dad49b6..f07fc31 100644
--- a/monitor.c
+++ b/monitor.c
@@ -130,13 +130,10 @@ typedef struct mon_cmd_t {
     const char *args_type;
     const char *params;
     const char *help;
-    union {
-        void (*cmd)(Monitor *mon, const QDict *qdict);
-        void (*cmd_new)(QDict *params, QObject **ret_data, Error **errp);
-    } mhandler;
-    /* @sub_table is a list of 2nd level of commands. If it do not exist,
-     * mhandler should be used. If it exist, sub_table[?].mhandler should be
-     * used, and mhandler of 1st level plays the role of help function.
+    void (*cmd)(Monitor *mon, const QDict *qdict);
+    /* @sub_table is a list of 2nd level of commands. If it does not exist,
+     * cmd should be used. If it exists, sub_table[?].cmd should be
+     * used, and cmd of 1st level plays the role of help function.
      */
     struct mon_cmd_t *sub_table;
     void (*command_completion)(ReadLineState *rs, int nb_args, const char *str);
@@ -2974,7 +2971,7 @@ static void handle_hmp_command(Monitor *mon, const char *cmdline)
         return;
     }
 
-    cmd->mhandler.cmd(mon, qdict);
+    cmd->cmd(mon, qdict);
     QDECREF(qdict);
 }
 
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index 0a66e0e..c425393 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -335,7 +335,7 @@ we should add it to the hmp-commands.hx file:
         .args_type  = "message:s?",
         .params     = "hello-world [message]",
         .help       = "Print message to the standard output",
-        .mhandler.cmd = hmp_hello_world,
+        .cmd        = hmp_hello_world,
     },
 
 STEXI
@@ -515,7 +515,7 @@ in the monitor.c file. The entry for the "info alarmclock" follows:
         .args_type  = "",
         .params     = "",
         .help       = "show information about the alarm clock",
-        .mhandler.cmd = hmp_info_alarm_clock,
+        .cmd        = hmp_info_alarm_clock,
     },
 
 To test this, run qemu and type "info alarmclock" in the user monitor.
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index be6e13a..dbb24f0 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -18,7 +18,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the version of QEMU",
-        .mhandler.cmd = hmp_info_version,
+        .cmd        = hmp_info_version,
     },
 
 STEXI
@@ -32,7 +32,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the network state",
-        .mhandler.cmd = hmp_info_network,
+        .cmd        = hmp_info_network,
     },
 
 STEXI
@@ -46,7 +46,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the character devices",
-        .mhandler.cmd = hmp_info_chardev,
+        .cmd        = hmp_info_chardev,
     },
 
 STEXI
@@ -61,7 +61,7 @@ ETEXI
         .params     = "[-n] [-v] [device]",
         .help       = "show info of one block device or all block devices "
                       "(-n: show named nodes; -v: show details)",
-        .mhandler.cmd = hmp_info_block,
+        .cmd        = hmp_info_block,
     },
 
 STEXI
@@ -75,7 +75,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show block device statistics",
-        .mhandler.cmd = hmp_info_blockstats,
+        .cmd        = hmp_info_blockstats,
     },
 
 STEXI
@@ -89,7 +89,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show progress of ongoing block device operations",
-        .mhandler.cmd = hmp_info_block_jobs,
+        .cmd        = hmp_info_block_jobs,
     },
 
 STEXI
@@ -103,7 +103,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the cpu registers",
-        .mhandler.cmd = hmp_info_registers,
+        .cmd        = hmp_info_registers,
     },
 
 STEXI
@@ -118,7 +118,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show local apic state",
-        .mhandler.cmd = hmp_info_local_apic,
+        .cmd        = hmp_info_local_apic,
     },
 #endif
 
@@ -134,7 +134,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show io apic state",
-        .mhandler.cmd = hmp_info_io_apic,
+        .cmd        = hmp_info_io_apic,
     },
 #endif
 
@@ -149,7 +149,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show infos for each CPU",
-        .mhandler.cmd = hmp_info_cpus,
+        .cmd        = hmp_info_cpus,
     },
 
 STEXI
@@ -163,7 +163,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the command line history",
-        .mhandler.cmd = hmp_info_history,
+        .cmd        = hmp_info_history,
     },
 
 STEXI
@@ -180,11 +180,11 @@ ETEXI
         .params     = "",
         .help       = "show the interrupts statistics (if available)",
 #ifdef TARGET_SPARC
-        .mhandler.cmd = sun4m_hmp_info_irq,
+        .cmd        = sun4m_hmp_info_irq,
 #elif defined(TARGET_LM32)
-        .mhandler.cmd = lm32_hmp_info_irq,
+        .cmd        = lm32_hmp_info_irq,
 #else
-        .mhandler.cmd = hmp_info_irq,
+        .cmd        = hmp_info_irq,
 #endif
     },
 
@@ -200,11 +200,11 @@ ETEXI
         .params     = "",
         .help       = "show i8259 (PIC) state",
 #ifdef TARGET_SPARC
-        .mhandler.cmd = sun4m_hmp_info_pic,
+        .cmd        = sun4m_hmp_info_pic,
 #elif defined(TARGET_LM32)
-        .mhandler.cmd = lm32_hmp_info_pic,
+        .cmd        = lm32_hmp_info_pic,
 #else
-        .mhandler.cmd = hmp_info_pic,
+        .cmd        = hmp_info_pic,
 #endif
     },
 #endif
@@ -220,7 +220,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show PCI info",
-        .mhandler.cmd = hmp_info_pci,
+        .cmd        = hmp_info_pci,
     },
 
 STEXI
@@ -236,7 +236,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show virtual to physical memory mappings",
-        .mhandler.cmd = hmp_info_tlb,
+        .cmd        = hmp_info_tlb,
     },
 #endif
 
@@ -252,7 +252,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the active virtual memory mappings",
-        .mhandler.cmd = hmp_info_mem,
+        .cmd        = hmp_info_mem,
     },
 #endif
 
@@ -267,7 +267,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show memory tree",
-        .mhandler.cmd = hmp_info_mtree,
+        .cmd        = hmp_info_mtree,
     },
 
 STEXI
@@ -281,7 +281,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler info",
-        .mhandler.cmd = hmp_info_jit,
+        .cmd        = hmp_info_jit,
     },
 
 STEXI
@@ -295,7 +295,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler opcode counters",
-        .mhandler.cmd = hmp_info_opcount,
+        .cmd        = hmp_info_opcount,
     },
 
 STEXI
@@ -309,7 +309,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show KVM information",
-        .mhandler.cmd = hmp_info_kvm,
+        .cmd        = hmp_info_kvm,
     },
 
 STEXI
@@ -323,7 +323,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show NUMA information",
-        .mhandler.cmd = hmp_info_numa,
+        .cmd        = hmp_info_numa,
     },
 
 STEXI
@@ -337,7 +337,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show guest USB devices",
-        .mhandler.cmd = hmp_info_usb,
+        .cmd        = hmp_info_usb,
     },
 
 STEXI
@@ -351,7 +351,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show host USB devices",
-        .mhandler.cmd = hmp_info_usbhost,
+        .cmd        = hmp_info_usbhost,
     },
 
 STEXI
@@ -365,7 +365,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show profiling information",
-        .mhandler.cmd = hmp_info_profile,
+        .cmd        = hmp_info_profile,
     },
 
 STEXI
@@ -379,7 +379,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show capture information",
-        .mhandler.cmd = hmp_info_capture,
+        .cmd        = hmp_info_capture,
     },
 
 STEXI
@@ -393,7 +393,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the currently saved VM snapshots",
-        .mhandler.cmd = hmp_info_snapshots,
+        .cmd        = hmp_info_snapshots,
     },
 
 STEXI
@@ -407,7 +407,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the current VM status (running|paused)",
-        .mhandler.cmd = hmp_info_status,
+        .cmd        = hmp_info_status,
     },
 
 STEXI
@@ -421,7 +421,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show which guest mouse is receiving events",
-        .mhandler.cmd = hmp_info_mice,
+        .cmd        = hmp_info_mice,
     },
 
 STEXI
@@ -436,7 +436,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the vnc server status",
-        .mhandler.cmd = hmp_info_vnc,
+        .cmd        = hmp_info_vnc,
     },
 #endif
 
@@ -452,7 +452,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the spice server status",
-        .mhandler.cmd = hmp_info_spice,
+        .cmd        = hmp_info_spice,
     },
 #endif
 
@@ -467,7 +467,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the current VM name",
-        .mhandler.cmd = hmp_info_name,
+        .cmd        = hmp_info_name,
     },
 
 STEXI
@@ -481,7 +481,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the current VM UUID",
-        .mhandler.cmd = hmp_info_uuid,
+        .cmd        = hmp_info_uuid,
     },
 
 STEXI
@@ -495,7 +495,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show CPU statistics",
-        .mhandler.cmd = hmp_info_cpustats,
+        .cmd        = hmp_info_cpustats,
     },
 
 STEXI
@@ -510,7 +510,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show user network stack connection states",
-        .mhandler.cmd = hmp_info_usernet,
+        .cmd        = hmp_info_usernet,
     },
 #endif
 
@@ -525,7 +525,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show migration status",
-        .mhandler.cmd = hmp_info_migrate,
+        .cmd        = hmp_info_migrate,
     },
 
 STEXI
@@ -539,7 +539,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show current migration capabilities",
-        .mhandler.cmd = hmp_info_migrate_capabilities,
+        .cmd        = hmp_info_migrate_capabilities,
     },
 
 STEXI
@@ -553,7 +553,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show current migration parameters",
-        .mhandler.cmd = hmp_info_migrate_parameters,
+        .cmd        = hmp_info_migrate_parameters,
     },
 
 STEXI
@@ -567,7 +567,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show current migration xbzrle cache size",
-        .mhandler.cmd = hmp_info_migrate_cache_size,
+        .cmd        = hmp_info_migrate_cache_size,
     },
 
 STEXI
@@ -581,7 +581,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show balloon information",
-        .mhandler.cmd = hmp_info_balloon,
+        .cmd        = hmp_info_balloon,
     },
 
 STEXI
@@ -595,7 +595,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show device tree",
-        .mhandler.cmd = hmp_info_qtree,
+        .cmd        = hmp_info_qtree,
     },
 
 STEXI
@@ -609,7 +609,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show qdev device model list",
-        .mhandler.cmd = hmp_info_qdm,
+        .cmd        = hmp_info_qdm,
     },
 
 STEXI
@@ -623,7 +623,7 @@ ETEXI
         .args_type  = "path:s?",
         .params     = "[path]",
         .help       = "show QOM composition tree",
-        .mhandler.cmd = hmp_info_qom_tree,
+        .cmd        = hmp_info_qom_tree,
     },
 
 STEXI
@@ -637,7 +637,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show roms",
-        .mhandler.cmd = hmp_info_roms,
+        .cmd        = hmp_info_roms,
     },
 
 STEXI
@@ -652,7 +652,7 @@ ETEXI
         .params     = "[name] [vcpu]",
         .help       = "show available trace-events & their state "
                       "(name: event name pattern; vcpu: vCPU to query, default is any)",
-        .mhandler.cmd = hmp_info_trace_events,
+        .cmd = hmp_info_trace_events,
         .command_completion = info_trace_events_completion,
     },
 
@@ -667,7 +667,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show the TPM device",
-        .mhandler.cmd = hmp_info_tpm,
+        .cmd        = hmp_info_tpm,
     },
 
 STEXI
@@ -681,7 +681,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show memory backends",
-        .mhandler.cmd = hmp_info_memdev,
+        .cmd        = hmp_info_memdev,
     },
 
 STEXI
@@ -695,7 +695,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show memory devices",
-        .mhandler.cmd = hmp_info_memory_devices,
+        .cmd        = hmp_info_memory_devices,
     },
 
 STEXI
@@ -709,7 +709,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "show iothreads",
-        .mhandler.cmd = hmp_info_iothreads,
+        .cmd        = hmp_info_iothreads,
     },
 
 STEXI
@@ -723,7 +723,7 @@ ETEXI
         .args_type  = "name:s",
         .params     = "name",
         .help       = "Show rocker switch",
-        .mhandler.cmd = hmp_rocker,
+        .cmd        = hmp_rocker,
     },
 
 STEXI
@@ -737,7 +737,7 @@ ETEXI
         .args_type  = "name:s",
         .params     = "name",
         .help       = "Show rocker ports",
-        .mhandler.cmd = hmp_rocker_ports,
+        .cmd        = hmp_rocker_ports,
     },
 
 STEXI
@@ -751,7 +751,7 @@ ETEXI
         .args_type  = "name:s,tbl_id:i?",
         .params     = "name [tbl_id]",
         .help       = "Show rocker OF-DPA flow tables",
-        .mhandler.cmd = hmp_rocker_of_dpa_flows,
+        .cmd        = hmp_rocker_of_dpa_flows,
     },
 
 STEXI
@@ -765,7 +765,7 @@ ETEXI
         .args_type  = "name:s,type:i?",
         .params     = "name [type]",
         .help       = "Show rocker OF-DPA groups",
-        .mhandler.cmd = hmp_rocker_of_dpa_groups,
+        .cmd        = hmp_rocker_of_dpa_groups,
     },
 
 STEXI
@@ -780,7 +780,7 @@ ETEXI
         .args_type  = "addr:l",
         .params     = "address",
         .help       = "Display the value of a storage key",
-        .mhandler.cmd = hmp_info_skeys,
+        .cmd        = hmp_info_skeys,
     },
 #endif
 
@@ -795,7 +795,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "Display the latest dump status",
-        .mhandler.cmd = hmp_info_dump,
+        .cmd        = hmp_info_dump,
     },
 
 STEXI
@@ -809,7 +809,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "Show information about hotpluggable CPUs",
-        .mhandler.cmd = hmp_hotpluggable_cpus,
+        .cmd        = hmp_hotpluggable_cpus,
     },
 
 STEXI
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 848efee..6b2ae30 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -14,7 +14,7 @@ ETEXI
         .args_type  = "name:S?",
         .params     = "[cmd]",
         .help       = "show the help",
-        .mhandler.cmd = do_help_cmd,
+        .cmd        = do_help_cmd,
     },
 
 STEXI
@@ -28,7 +28,7 @@ ETEXI
         .args_type  = "device:B",
         .params     = "device|all",
         .help       = "commit changes to the disk images (if -snapshot is used) or backing files",
-        .mhandler.cmd = hmp_commit,
+        .cmd        = hmp_commit,
     },
 
 STEXI
@@ -47,7 +47,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "quit the emulator",
-        .mhandler.cmd = hmp_quit,
+        .cmd        = hmp_quit,
     },
 
 STEXI
@@ -61,7 +61,7 @@ ETEXI
         .args_type  = "device:B,size:o",
         .params     = "device size",
         .help       = "resize a block image",
-        .mhandler.cmd = hmp_block_resize,
+        .cmd        = hmp_block_resize,
     },
 
 STEXI
@@ -78,7 +78,7 @@ ETEXI
         .args_type  = "device:B,speed:o?,base:s?",
         .params     = "device [speed [base]]",
         .help       = "copy data from a backing file into a block device",
-        .mhandler.cmd = hmp_block_stream,
+        .cmd        = hmp_block_stream,
     },
 
 STEXI
@@ -92,7 +92,7 @@ ETEXI
         .args_type  = "device:B,speed:o",
         .params     = "device speed",
         .help       = "set maximum speed for a background block operation",
-        .mhandler.cmd = hmp_block_job_set_speed,
+        .cmd        = hmp_block_job_set_speed,
     },
 
 STEXI
@@ -107,7 +107,7 @@ ETEXI
         .params     = "[-f] device",
         .help       = "stop an active background block operation (use -f"
                       "\n\t\t\t if the operation is currently paused)",
-        .mhandler.cmd = hmp_block_job_cancel,
+        .cmd        = hmp_block_job_cancel,
     },
 
 STEXI
@@ -121,7 +121,7 @@ ETEXI
         .args_type  = "device:B",
         .params     = "device",
         .help       = "stop an active background block operation",
-        .mhandler.cmd = hmp_block_job_complete,
+        .cmd        = hmp_block_job_complete,
     },
 
 STEXI
@@ -136,7 +136,7 @@ ETEXI
         .args_type  = "device:B",
         .params     = "device",
         .help       = "pause an active background block operation",
-        .mhandler.cmd = hmp_block_job_pause,
+        .cmd        = hmp_block_job_pause,
     },
 
 STEXI
@@ -150,7 +150,7 @@ ETEXI
         .args_type  = "device:B",
         .params     = "device",
         .help       = "resume a paused background block operation",
-        .mhandler.cmd = hmp_block_job_resume,
+        .cmd        = hmp_block_job_resume,
     },
 
 STEXI
@@ -164,7 +164,7 @@ ETEXI
         .args_type  = "force:-f,device:B",
         .params     = "[-f] device",
         .help       = "eject a removable medium (use -f to force it)",
-        .mhandler.cmd = hmp_eject,
+        .cmd        = hmp_eject,
     },
 
 STEXI
@@ -178,7 +178,7 @@ ETEXI
         .args_type  = "id:B",
         .params     = "device",
         .help       = "remove host block device",
-        .mhandler.cmd = hmp_drive_del,
+        .cmd        = hmp_drive_del,
     },
 
 STEXI
@@ -197,7 +197,7 @@ ETEXI
         .args_type  = "device:B,target:F,arg:s?,read-only-mode:s?",
         .params     = "device filename [format [read-only-mode]]",
         .help       = "change a removable medium, optional format",
-        .mhandler.cmd = hmp_change,
+        .cmd        = hmp_change,
     },
 
 STEXI
@@ -256,7 +256,7 @@ ETEXI
         .args_type  = "filename:F",
         .params     = "filename",
         .help       = "save screen into PPM image 'filename'",
-        .mhandler.cmd = hmp_screendump,
+        .cmd        = hmp_screendump,
     },
 
 STEXI
@@ -270,7 +270,7 @@ ETEXI
         .args_type  = "filename:F",
         .params     = "filename",
         .help       = "output logs to 'filename'",
-        .mhandler.cmd = hmp_logfile,
+        .cmd        = hmp_logfile,
     },
 
 STEXI
@@ -285,7 +285,7 @@ ETEXI
         .params     = "name on|off [vcpu]",
         .help       = "changes status of a specific trace event "
                       "(vcpu: vCPU to set, default is all)",
-        .mhandler.cmd = hmp_trace_event,
+        .cmd = hmp_trace_event,
         .command_completion = trace_event_completion,
     },
 
@@ -301,7 +301,7 @@ ETEXI
         .args_type  = "op:s?,arg:F?",
         .params     = "on|off|flush|set [arg]",
         .help       = "open, close, or flush trace file, or set a new file name",
-        .mhandler.cmd = hmp_trace_file,
+        .cmd        = hmp_trace_file,
     },
 
 STEXI
@@ -316,7 +316,7 @@ ETEXI
         .args_type  = "items:s",
         .params     = "item1[,...]",
         .help       = "activate logging of the specified items",
-        .mhandler.cmd = hmp_log,
+        .cmd        = hmp_log,
     },
 
 STEXI
@@ -330,7 +330,7 @@ ETEXI
         .args_type  = "name:s?",
         .params     = "[tag|id]",
         .help       = "save a VM snapshot. If no tag or id are provided, a new snapshot is created",
-        .mhandler.cmd = hmp_savevm,
+        .cmd        = hmp_savevm,
     },
 
 STEXI
@@ -347,7 +347,7 @@ ETEXI
         .args_type  = "name:s",
         .params     = "tag|id",
         .help       = "restore a VM snapshot from its tag or id",
-        .mhandler.cmd = hmp_loadvm,
+        .cmd        = hmp_loadvm,
         .command_completion = loadvm_completion,
     },
 
@@ -363,7 +363,7 @@ ETEXI
         .args_type  = "name:s",
         .params     = "tag|id",
         .help       = "delete a VM snapshot from its tag or id",
-        .mhandler.cmd = hmp_delvm,
+        .cmd        = hmp_delvm,
         .command_completion = delvm_completion,
     },
 
@@ -378,7 +378,7 @@ ETEXI
         .args_type  = "option:s?",
         .params     = "[on|off]",
         .help       = "run emulation in singlestep mode or switch to normal mode",
-        .mhandler.cmd = hmp_singlestep,
+        .cmd        = hmp_singlestep,
     },
 
 STEXI
@@ -393,7 +393,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "stop emulation",
-        .mhandler.cmd = hmp_stop,
+        .cmd        = hmp_stop,
     },
 
 STEXI
@@ -407,7 +407,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "resume emulation",
-        .mhandler.cmd = hmp_cont,
+        .cmd        = hmp_cont,
     },
 
 STEXI
@@ -421,7 +421,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "wakeup guest from suspend",
-        .mhandler.cmd = hmp_system_wakeup,
+        .cmd        = hmp_system_wakeup,
     },
 
 STEXI
@@ -435,7 +435,7 @@ ETEXI
         .args_type  = "device:s?",
         .params     = "[device]",
         .help       = "start gdbserver on given device (default 'tcp::1234'), stop with 'none'",
-        .mhandler.cmd = hmp_gdbserver,
+        .cmd        = hmp_gdbserver,
     },
 
 STEXI
@@ -449,7 +449,7 @@ ETEXI
         .args_type  = "fmt:/,addr:l",
         .params     = "/fmt addr",
         .help       = "virtual memory dump starting at 'addr'",
-        .mhandler.cmd = hmp_memory_dump,
+        .cmd        = hmp_memory_dump,
     },
 
 STEXI
@@ -463,7 +463,7 @@ ETEXI
         .args_type  = "fmt:/,addr:l",
         .params     = "/fmt addr",
         .help       = "physical memory dump starting at 'addr'",
-        .mhandler.cmd = hmp_physical_memory_dump,
+        .cmd        = hmp_physical_memory_dump,
     },
 
 STEXI
@@ -530,7 +530,7 @@ ETEXI
         .args_type  = "fmt:/,val:l",
         .params     = "/fmt expr",
         .help       = "print expression value (use $reg for CPU register access)",
-        .mhandler.cmd = do_print,
+        .cmd        = do_print,
     },
 
 STEXI
@@ -545,7 +545,7 @@ ETEXI
         .args_type  = "fmt:/,addr:i,index:i.",
         .params     = "/fmt addr",
         .help       = "I/O port read",
-        .mhandler.cmd = hmp_ioport_read,
+        .cmd        = hmp_ioport_read,
     },
 
 STEXI
@@ -559,7 +559,7 @@ ETEXI
         .args_type  = "fmt:/,addr:i,val:i",
         .params     = "/fmt addr value",
         .help       = "I/O port write",
-        .mhandler.cmd = hmp_ioport_write,
+        .cmd        = hmp_ioport_write,
     },
 
 STEXI
@@ -573,7 +573,7 @@ ETEXI
         .args_type  = "keys:s,hold-time:i?",
         .params     = "keys [hold_ms]",
         .help       = "send keys to the VM (e.g. 'sendkey ctrl-alt-f1', default hold time=100 ms)",
-        .mhandler.cmd = hmp_sendkey,
+        .cmd        = hmp_sendkey,
         .command_completion = sendkey_completion,
     },
 
@@ -596,7 +596,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "reset the system",
-        .mhandler.cmd = hmp_system_reset,
+        .cmd        = hmp_system_reset,
     },
 
 STEXI
@@ -610,7 +610,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "send system power down event",
-        .mhandler.cmd = hmp_system_powerdown,
+        .cmd        = hmp_system_powerdown,
     },
 
 STEXI
@@ -624,7 +624,7 @@ ETEXI
         .args_type  = "start:i,size:i",
         .params     = "addr size",
         .help       = "compute the checksum of a memory region",
-        .mhandler.cmd = hmp_sum,
+        .cmd        = hmp_sum,
     },
 
 STEXI
@@ -638,7 +638,7 @@ ETEXI
         .args_type  = "devname:s",
         .params     = "device",
         .help       = "add USB device (e.g. 'host:bus.addr' or 'host:vendor_id:product_id')",
-        .mhandler.cmd = hmp_usb_add,
+        .cmd        = hmp_usb_add,
     },
 
 STEXI
@@ -653,7 +653,7 @@ ETEXI
         .args_type  = "devname:s",
         .params     = "device",
         .help       = "remove USB device 'bus.addr'",
-        .mhandler.cmd = hmp_usb_del,
+        .cmd        = hmp_usb_del,
     },
 
 STEXI
@@ -669,7 +669,7 @@ ETEXI
         .args_type  = "device:O",
         .params     = "driver[,prop=value][,...]",
         .help       = "add device, like -device on the command line",
-        .mhandler.cmd = hmp_device_add,
+        .cmd        = hmp_device_add,
         .command_completion = device_add_completion,
     },
 
@@ -684,7 +684,7 @@ ETEXI
         .args_type  = "id:s",
         .params     = "device",
         .help       = "remove device",
-        .mhandler.cmd = hmp_device_del,
+        .cmd        = hmp_device_del,
         .command_completion = device_del_completion,
     },
 
@@ -700,7 +700,7 @@ ETEXI
         .args_type  = "index:i",
         .params     = "index",
         .help       = "set the default CPU",
-        .mhandler.cmd = hmp_cpu,
+        .cmd        = hmp_cpu,
     },
 
 STEXI
@@ -714,7 +714,7 @@ ETEXI
         .args_type  = "dx_str:s,dy_str:s,dz_str:s?",
         .params     = "dx dy [dz]",
         .help       = "send mouse move events",
-        .mhandler.cmd = hmp_mouse_move,
+        .cmd        = hmp_mouse_move,
     },
 
 STEXI
@@ -729,7 +729,7 @@ ETEXI
         .args_type  = "button_state:i",
         .params     = "state",
         .help       = "change mouse button state (1=L, 2=M, 4=R)",
-        .mhandler.cmd = hmp_mouse_button,
+        .cmd        = hmp_mouse_button,
     },
 
 STEXI
@@ -743,7 +743,7 @@ ETEXI
         .args_type  = "index:i",
         .params     = "index",
         .help       = "set which mouse device receives events",
-        .mhandler.cmd = hmp_mouse_set,
+        .cmd        = hmp_mouse_set,
     },
 
 STEXI
@@ -761,7 +761,7 @@ ETEXI
         .args_type  = "path:F,freq:i?,bits:i?,nchannels:i?",
         .params     = "path [frequency [bits [channels]]]",
         .help       = "capture audio to a wave file (default frequency=44100 bits=16 channels=2)",
-        .mhandler.cmd = hmp_wavcapture,
+        .cmd        = hmp_wavcapture,
     },
 STEXI
 @item wavcapture @var{filename} [@var{frequency} [@var{bits} [@var{channels}]]]
@@ -782,7 +782,7 @@ ETEXI
         .args_type  = "n:i",
         .params     = "capture index",
         .help       = "stop capture",
-        .mhandler.cmd = hmp_stopcapture,
+        .cmd        = hmp_stopcapture,
     },
 STEXI
 @item stopcapture @var{index}
@@ -798,7 +798,7 @@ ETEXI
         .args_type  = "val:l,size:i,filename:s",
         .params     = "addr size file",
         .help       = "save to disk virtual memory dump starting at 'addr' of size 'size'",
-        .mhandler.cmd = hmp_memsave,
+        .cmd        = hmp_memsave,
     },
 
 STEXI
@@ -812,7 +812,7 @@ ETEXI
         .args_type  = "val:l,size:i,filename:s",
         .params     = "addr size file",
         .help       = "save to disk physical memory dump starting at 'addr' of size 'size'",
-        .mhandler.cmd = hmp_pmemsave,
+        .cmd        = hmp_pmemsave,
     },
 
 STEXI
@@ -826,7 +826,7 @@ ETEXI
         .args_type  = "bootdevice:s",
         .params     = "bootdevice",
         .help       = "define new values for the boot device list",
-        .mhandler.cmd = hmp_boot_set,
+        .cmd        = hmp_boot_set,
     },
 
 STEXI
@@ -844,7 +844,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "inject an NMI",
-        .mhandler.cmd = hmp_nmi,
+        .cmd        = hmp_nmi,
     },
 STEXI
 @item nmi @var{cpu}
@@ -858,7 +858,7 @@ ETEXI
         .args_type  = "device:s,data:s",
         .params     = "device data",
         .help       = "Write to a ring buffer character device",
-        .mhandler.cmd = hmp_ringbuf_write,
+        .cmd        = hmp_ringbuf_write,
         .command_completion = ringbuf_write_completion,
     },
 
@@ -875,7 +875,7 @@ ETEXI
         .args_type  = "device:s,size:i",
         .params     = "device size",
         .help       = "Read from a ring buffer character device",
-        .mhandler.cmd = hmp_ringbuf_read,
+        .cmd        = hmp_ringbuf_read,
         .command_completion = ringbuf_write_completion,
     },
 
@@ -901,7 +901,7 @@ ETEXI
 		      " full copy of disk\n\t\t\t -i for migration without "
 		      "shared storage with incremental copy of disk "
 		      "(base image shared between src and destination)",
-        .mhandler.cmd = hmp_migrate,
+        .cmd        = hmp_migrate,
     },
 
 
@@ -918,7 +918,7 @@ ETEXI
         .args_type  = "",
         .params     = "",
         .help       = "cancel the current VM migration",
-        .mhandler.cmd = hmp_migrate_cancel,
+        .cmd        = hmp_migrate_cancel,
     },
 
 STEXI
@@ -933,7 +933,7 @@ ETEXI
         .args_type  = "uri:s",
         .params     = "uri",
         .help       = "Continue an incoming migration from an -incoming defer",
-        .mhandler.cmd = hmp_migrate_incoming,
+        .cmd        = hmp_migrate_incoming,
     },
 
 STEXI
@@ -954,7 +954,7 @@ ETEXI
                       "The cache size affects the number of cache misses."
                       "In case of a high cache miss ratio you need to increase"
                       " the cache size",
-        .mhandler.cmd = hmp_migrate_set_cache_size,
+        .cmd        = hmp_migrate_set_cache_size,
     },
 
 STEXI
@@ -969,7 +969,7 @@ ETEXI
         .params     = "value",
         .help       = "set maximum speed (in bytes) for migrations. "
 	"Defaults to MB if no size suffix is specified, ie. B/K/M/G/T",
-        .mhandler.cmd = hmp_migrate_set_speed,
+        .cmd        = hmp_migrate_set_speed,
     },
 
 STEXI
@@ -983,7 +983,7 @@ ETEXI
         .args_type  = "value:T",
         .params     = "value",
         .help       = "set maximum tolerated downtime (in seconds) for migrations",
-        .mhandler.cmd = hmp_migrate_set_downtime,
+        .cmd        = hmp_migrate_set_downtime,
     },
 
 STEXI
@@ -997,7 +997,7 @@ ETEXI
         .args_type  = "capability:s,state:b",
         .params     = "capability state",
         .help       = "Enable/Disable the usage of a capability for migration",
-        .mhandler.cmd = hmp_migrate_set_capability,
+        .cmd        = hmp_migrate_set_capability,
         .command_completion = migrate_set_capability_completion,
     },
 
@@ -1012,7 +1012,7 @@ ETEXI
         .args_type  = "parameter:s,value:s",
         .params     = "parameter value",
         .help       = "Set the parameter for migration",
-        .mhandler.cmd = hmp_migrate_set_parameter,
+        .cmd        = hmp_migrate_set_parameter,
         .command_completion = migrate_set_parameter_completion,
     },
 
@@ -1029,7 +1029,7 @@ ETEXI
         .help       = "Followup to a migration command to switch the migration"
                       " to postcopy mode. The postcopy-ram capability must "
                       "be set before the original migration command.",
-        .mhandler.cmd = hmp_migrate_start_postcopy,
+        .cmd        = hmp_migrate_start_postcopy,
     },
 
 STEXI
@@ -1044,7 +1044,7 @@ ETEXI
         .args_type  = "protocol:s,hostname:s,port:i?,tls-port:i?,cert-subject:s?",
         .params     = "protocol hostname port tls-port cert-subject",
         .help       = "set migration information for remote display",
-        .mhandler.cmd = hmp_client_migrate_info,
+        .cmd        = hmp_client_migrate_info,
     },
 
 STEXI
@@ -1067,7 +1067,7 @@ ETEXI
                       "-s: dump in kdump-compressed format, with snappy compression.\n\t\t\t"
                       "begin: the starting physical address.\n\t\t\t"
                       "length: the memory size, in bytes.",
-        .mhandler.cmd = hmp_dump_guest_memory,
+        .cmd        = hmp_dump_guest_memory,
     },
 
 
@@ -1094,7 +1094,7 @@ ETEXI
         .args_type  = "filename:F",
         .params     = "",
         .help       = "Save guest storage keys into file 'filename'.\n",
-        .mhandler.cmd = hmp_dump_skeys,
+        .cmd        = hmp_dump_skeys,
     },
 #endif
 
@@ -1116,7 +1116,7 @@ ETEXI
                       "The default format is qcow2.  The -n flag requests QEMU\n\t\t\t"
                       "to reuse the image found in new-image-file, instead of\n\t\t\t"
                       "recreating it from scratch.",
-        .mhandler.cmd = hmp_snapshot_blkdev,
+        .cmd        = hmp_snapshot_blkdev,
     },
 
 STEXI
@@ -1132,7 +1132,7 @@ ETEXI
         .help       = "take an internal snapshot of device.\n\t\t\t"
                       "The format of the image used by device must\n\t\t\t"
                       "support it, such as qcow2.\n\t\t\t",
-        .mhandler.cmd = hmp_snapshot_blkdev_internal,
+        .cmd        = hmp_snapshot_blkdev_internal,
     },
 
 STEXI
@@ -1150,7 +1150,7 @@ ETEXI
                       "the snapshot matching both id and name.\n\t\t\t"
                       "The format of the image used by device must\n\t\t\t"
                       "support it, such as qcow2.\n\t\t\t",
-        .mhandler.cmd = hmp_snapshot_delete_blkdev_internal,
+        .cmd        = hmp_snapshot_delete_blkdev_internal,
     },
 
 STEXI
@@ -1171,7 +1171,7 @@ ETEXI
                       "in new-image-file, instead of recreating it from scratch.\n\t\t\t"
                       "The -f flag requests QEMU to copy the whole disk,\n\t\t\t"
                       "so that the result does not need a backing file.\n\t\t\t",
-        .mhandler.cmd = hmp_drive_mirror,
+        .cmd        = hmp_drive_mirror,
     },
 STEXI
 @item drive_mirror
@@ -1192,7 +1192,7 @@ ETEXI
                       "in new-image-file, instead of recreating it from scratch.\n\t\t\t"
                       "The -f flag requests QEMU to copy the whole disk,\n\t\t\t"
                       "so that the result does not need a backing file.\n\t\t\t",
-        .mhandler.cmd = hmp_drive_backup,
+        .cmd        = hmp_drive_backup,
     },
 STEXI
 @item drive_backup
@@ -1210,7 +1210,7 @@ ETEXI
                       "[,snapshot=on|off][,cache=on|off]\n"
                       "[,readonly=on|off][,copy-on-read=on|off]",
         .help       = "add drive to PCI storage controller",
-        .mhandler.cmd = hmp_drive_add,
+        .cmd        = hmp_drive_add,
     },
 
 STEXI
@@ -1234,7 +1234,7 @@ ETEXI
                       "<error_status> = error string or 32bit\n\t\t\t"
                       "<tlb header> = 32bit x 4\n\t\t\t"
                       "<tlb header prefix> = 32bit x 4",
-        .mhandler.cmd = hmp_pcie_aer_inject_error,
+        .cmd        = hmp_pcie_aer_inject_error,
     },
 
 STEXI
@@ -1248,7 +1248,7 @@ ETEXI
         .args_type  = "device:s,opts:s?",
         .params     = "tap|user|socket|vde|netmap|bridge|vhost-user|dump [options]",
         .help       = "add host VLAN client",
-        .mhandler.cmd = hmp_host_net_add,
+        .cmd        = hmp_host_net_add,
         .command_completion = host_net_add_completion,
     },
 
@@ -1263,7 +1263,7 @@ ETEXI
         .args_type  = "vlan_id:i,device:s",
         .params     = "vlan_id name",
         .help       = "remove host VLAN client",
-        .mhandler.cmd = hmp_host_net_remove,
+        .cmd        = hmp_host_net_remove,
         .command_completion = host_net_remove_completion,
     },
 
@@ -1278,7 +1278,7 @@ ETEXI
         .args_type  = "netdev:O",
         .params     = "[user|tap|socket|vde|bridge|hubport|netmap|vhost-user],id=str[,prop=value][,...]",
         .help       = "add host network device",
-        .mhandler.cmd = hmp_netdev_add,
+        .cmd        = hmp_netdev_add,
         .command_completion = netdev_add_completion,
     },
 
@@ -1293,7 +1293,7 @@ ETEXI
         .args_type  = "id:s",
         .params     = "id",
         .help       = "remove host network device",
-        .mhandler.cmd = hmp_netdev_del,
+        .cmd        = hmp_netdev_del,
         .command_completion = netdev_del_completion,
     },
 
@@ -1308,7 +1308,7 @@ ETEXI
         .args_type  = "object:O",
         .params     = "[qom-type=]type,id=str[,prop=value][,...]",
         .help       = "create QOM object",
-        .mhandler.cmd = hmp_object_add,
+        .cmd        = hmp_object_add,
         .command_completion = object_add_completion,
     },
 
@@ -1323,7 +1323,7 @@ ETEXI
         .args_type  = "id:s",
         .params     = "id",
         .help       = "destroy QOM object",
-        .mhandler.cmd = hmp_object_del,
+        .cmd        = hmp_object_del,
         .command_completion = object_del_completion,
     },
 
@@ -1339,7 +1339,7 @@ ETEXI
         .args_type  = "arg1:s,arg2:s?,arg3:s?",
         .params     = "[vlan_id name] [tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport",
         .help       = "redirect TCP or UDP connections from host to guest (requires -net user)",
-        .mhandler.cmd = hmp_hostfwd_add,
+        .cmd        = hmp_hostfwd_add,
     },
 #endif
 STEXI
@@ -1354,7 +1354,7 @@ ETEXI
         .args_type  = "arg1:s,arg2:s?,arg3:s?",
         .params     = "[vlan_id name] [tcp|udp]:[hostaddr]:hostport",
         .help       = "remove host-to-guest TCP or UDP redirection",
-        .mhandler.cmd = hmp_hostfwd_remove,
+        .cmd        = hmp_hostfwd_remove,
     },
 
 #endif
@@ -1369,7 +1369,7 @@ ETEXI
         .args_type  = "value:M",
         .params     = "target",
         .help       = "request VM to change its memory allocation (in MB)",
-        .mhandler.cmd = hmp_balloon,
+        .cmd        = hmp_balloon,
     },
 
 STEXI
@@ -1383,7 +1383,7 @@ ETEXI
         .args_type  = "name:s,up:b",
         .params     = "name on|off",
         .help       = "change the link status of a network adapter",
-        .mhandler.cmd = hmp_set_link,
+        .cmd        = hmp_set_link,
         .command_completion = set_link_completion,
     },
 
@@ -1398,7 +1398,7 @@ ETEXI
         .args_type  = "action:s",
         .params     = "[reset|shutdown|poweroff|pause|debug|none]",
         .help       = "change watchdog action",
-        .mhandler.cmd = hmp_watchdog_action,
+        .cmd        = hmp_watchdog_action,
         .command_completion = watchdog_action_completion,
     },
 
@@ -1413,7 +1413,7 @@ ETEXI
         .args_type  = "aclname:s",
         .params     = "aclname",
         .help       = "list rules in the access control list",
-        .mhandler.cmd = hmp_acl_show,
+        .cmd        = hmp_acl_show,
     },
 
 STEXI
@@ -1430,7 +1430,7 @@ ETEXI
         .args_type  = "aclname:s,policy:s",
         .params     = "aclname allow|deny",
         .help       = "set default access control list policy",
-        .mhandler.cmd = hmp_acl_policy,
+        .cmd        = hmp_acl_policy,
     },
 
 STEXI
@@ -1446,7 +1446,7 @@ ETEXI
         .args_type  = "aclname:s,match:s,policy:s,index:i?",
         .params     = "aclname match allow|deny [index]",
         .help       = "add a match rule to the access control list",
-        .mhandler.cmd = hmp_acl_add,
+        .cmd        = hmp_acl_add,
     },
 
 STEXI
@@ -1465,7 +1465,7 @@ ETEXI
         .args_type  = "aclname:s,match:s",
         .params     = "aclname match",
         .help       = "remove a match rule from the access control list",
-        .mhandler.cmd = hmp_acl_remove,
+        .cmd        = hmp_acl_remove,
     },
 
 STEXI
@@ -1479,7 +1479,7 @@ ETEXI
         .args_type  = "aclname:s",
         .params     = "aclname",
         .help       = "reset the access control list",
-        .mhandler.cmd = hmp_acl_reset,
+        .cmd        = hmp_acl_reset,
     },
 
 STEXI
@@ -1494,7 +1494,7 @@ ETEXI
         .args_type  = "all:-a,writable:-w,uri:s",
         .params     = "nbd_server_start [-a] [-w] host:port",
         .help       = "serve block devices on the given host and port",
-        .mhandler.cmd = hmp_nbd_server_start,
+        .cmd        = hmp_nbd_server_start,
     },
 STEXI
 @item nbd_server_start @var{host}:@var{port}
@@ -1510,7 +1510,7 @@ ETEXI
         .args_type  = "writable:-w,device:B",
         .params     = "nbd_server_add [-w] device",
         .help       = "export a block device via NBD",
-        .mhandler.cmd = hmp_nbd_server_add,
+        .cmd        = hmp_nbd_server_add,
     },
 STEXI
 @item nbd_server_add @var{device}
@@ -1525,7 +1525,7 @@ ETEXI
         .args_type  = "",
         .params     = "nbd_server_stop",
         .help       = "stop serving block devices using the NBD protocol",
-        .mhandler.cmd = hmp_nbd_server_stop,
+        .cmd        = hmp_nbd_server_stop,
     },
 STEXI
 @item nbd_server_stop
@@ -1541,7 +1541,7 @@ ETEXI
         .args_type  = "broadcast:-b,cpu_index:i,bank:i,status:l,mcg_status:l,addr:l,misc:l",
         .params     = "[-b] cpu bank status mcgstatus addr misc",
         .help       = "inject a MCE on the given CPU [and broadcast to other CPUs with -b option]",
-        .mhandler.cmd = hmp_mce,
+        .cmd        = hmp_mce,
     },
 
 #endif
@@ -1556,7 +1556,7 @@ ETEXI
         .args_type  = "fdname:s",
         .params     = "getfd name",
         .help       = "receive a file descriptor via SCM rights and assign it a name",
-        .mhandler.cmd = hmp_getfd,
+        .cmd        = hmp_getfd,
     },
 
 STEXI
@@ -1572,7 +1572,7 @@ ETEXI
         .args_type  = "fdname:s",
         .params     = "closefd name",
         .help       = "close a file descriptor previously passed via SCM rights",
-        .mhandler.cmd = hmp_closefd,
+        .cmd        = hmp_closefd,
     },
 
 STEXI
@@ -1588,7 +1588,7 @@ ETEXI
         .args_type  = "device:B,password:s",
         .params     = "block_passwd device password",
         .help       = "set the password of encrypted block devices",
-        .mhandler.cmd = hmp_block_passwd,
+        .cmd        = hmp_block_passwd,
     },
 
 STEXI
@@ -1602,7 +1602,7 @@ ETEXI
         .args_type  = "device:B,bps:l,bps_rd:l,bps_wr:l,iops:l,iops_rd:l,iops_wr:l",
         .params     = "device bps bps_rd bps_wr iops iops_rd iops_wr",
         .help       = "change I/O throttle limits for a block drive",
-        .mhandler.cmd = hmp_block_set_io_throttle,
+        .cmd        = hmp_block_set_io_throttle,
     },
 
 STEXI
@@ -1616,7 +1616,7 @@ ETEXI
         .args_type  = "protocol:s,password:s,connected:s?",
         .params     = "protocol password action-if-connected",
         .help       = "set spice/vnc password",
-        .mhandler.cmd = hmp_set_password,
+        .cmd        = hmp_set_password,
     },
 
 STEXI
@@ -1635,7 +1635,7 @@ ETEXI
         .args_type  = "protocol:s,time:s",
         .params     = "protocol time",
         .help       = "set spice/vnc password expire-time",
-        .mhandler.cmd = hmp_expire_password,
+        .cmd        = hmp_expire_password,
     },
 
 STEXI
@@ -1666,7 +1666,7 @@ ETEXI
         .args_type  = "args:s",
         .params     = "args",
         .help       = "add chardev",
-        .mhandler.cmd = hmp_chardev_add,
+        .cmd        = hmp_chardev_add,
         .command_completion = chardev_add_completion,
     },
 
@@ -1682,7 +1682,7 @@ ETEXI
         .args_type  = "id:s",
         .params     = "id",
         .help       = "remove chardev",
-        .mhandler.cmd = hmp_chardev_remove,
+        .cmd        = hmp_chardev_remove,
         .command_completion = chardev_remove_completion,
     },
 
@@ -1698,7 +1698,7 @@ ETEXI
         .args_type  = "device:B,command:s",
         .params     = "[device] \"[command]\"",
         .help       = "run a qemu-io command on a block device",
-        .mhandler.cmd = hmp_qemu_io,
+        .cmd        = hmp_qemu_io,
     },
 
 STEXI
@@ -1713,7 +1713,7 @@ ETEXI
         .args_type  = "id:i",
         .params     = "id",
         .help       = "add cpu",
-        .mhandler.cmd  = hmp_cpu_add,
+        .cmd        = hmp_cpu_add,
     },
 
 STEXI
@@ -1727,7 +1727,7 @@ ETEXI
         .args_type  = "path:s?",
         .params     = "path",
         .help       = "list QOM properties",
-        .mhandler.cmd  = hmp_qom_list,
+        .cmd        = hmp_qom_list,
     },
 
 STEXI
@@ -1740,7 +1740,7 @@ ETEXI
         .args_type  = "path:s,property:s,value:s",
         .params     = "path property value",
         .help       = "set QOM property",
-        .mhandler.cmd  = hmp_qom_set,
+        .cmd        = hmp_qom_set,
     },
 
 STEXI
@@ -1753,8 +1753,8 @@ ETEXI
         .args_type  = "item:s?",
         .params     = "[subcommand]",
         .help       = "show various information about the system state",
-        .mhandler.cmd = hmp_info_help,
-        .sub_table = info_cmds,
+        .cmd        = hmp_info_help,
+        .sub_table  = info_cmds,
     },
 
 STEXI
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 15/20] qapi: remove the "middle" mode
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (13 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 14/20] monitor: remove mhandler.cmd_new Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 16/20] qapi: check invalid arguments on no-args commands Marc-André Lureau
                   ` (4 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Now that the register function is always generated, we can
remove the so-called "middle" mode from the generator script.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi-commands.py | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index b150db9..eac64ce 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -206,8 +206,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
         self._visited_ret_types = set()
 
     def visit_end(self):
-        if not middle_mode:
-            self.defn += gen_registry(self._regy)
+        self.defn += gen_registry(self._regy)
         self._regy = None
         self._visited_ret_types = None
 
@@ -221,18 +220,10 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
             self.defn += gen_marshal_output(ret_type)
         self.decl += gen_marshal_decl(name)
         self.defn += gen_marshal(name, arg_type, boxed, ret_type)
-        if not middle_mode:
-            self._regy += gen_register_command(name, success_response)
+        self._regy += gen_register_command(name, success_response)
 
 
-middle_mode = False
-
-(input_file, output_dir, do_c, do_h, prefix, opts) = \
-    parse_command_line("m", ["middle"])
-
-for o, a in opts:
-    if o in ("-m", "--middle"):
-        middle_mode = True
+(input_file, output_dir, do_c, do_h, prefix, opts) = parse_command_line()
 
 c_comment = '''
 /*
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 16/20] qapi: check invalid arguments on no-args commands
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (14 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 15/20] qapi: remove the "middle" mode Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch() Marc-André Lureau
                   ` (3 subsequent siblings)
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

The generated marshal functions do not visit arguments from commands
that take no arguments. Thus they fail to catch invalid
members. Visit the arguments, if provided, to throw an error in case of
invalid members.

Currently, qmp_check_client_args() checks for invalid arguments and
correctly catches this case. When switching to qmp_dispatch() we want to
keep that behaviour. The commands using 'O' may have arbitrary
arguments, and must have 'gen': false in the qapi schema to skip the
generated checks.

Old/new diff:
 void qmp_marshal_stop(QDict *args, QObject **ret, Error **errp)
 {
     Error *err = NULL;
+    Visitor *v = NULL;

-    (void)args;
+    if (args) {
+        v = qmp_input_visitor_new(QOBJECT(args), true);
+        visit_start_struct(v, NULL, NULL, 0, &err);
+        if (err) {
+            goto out;
+        }
+
+        if (!err) {
+            visit_check_struct(v, &err);
+        }
+        visit_end_struct(v, NULL);
+        if (err) {
+            goto out;
+        }
+    }

     qmp_stop(&err);
+
+out:
     error_propagate(errp, err);
+    visit_free(v);
+    if (args) {
+        v = qapi_dealloc_visitor_new();
+        visit_start_struct(v, NULL, NULL, 0, NULL);
+
+        visit_end_struct(v, NULL);
+        visit_free(v);
+    }
 }

The new code closely resembles code for a command with arguments.
Differences:
- the visit of the argument and its cleanup struct don't visit any
  members (because there are none).
- the visit of the argument struct and its cleanup are conditional.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/test-qmp-commands.c | 15 ++++++++++++
 scripts/qapi-commands.py  | 58 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 58 insertions(+), 15 deletions(-)

diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 261fd9e..81cbe54 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -106,6 +106,7 @@ static void test_dispatch_cmd(void)
 static void test_dispatch_cmd_failure(void)
 {
     QDict *req = qdict_new();
+    QDict *args = qdict_new();
     QObject *resp;
 
     qdict_put_obj(req, "execute", QOBJECT(qstring_from_str("user_def_cmd2")));
@@ -114,6 +115,20 @@ static void test_dispatch_cmd_failure(void)
     assert(resp != NULL);
     assert(qdict_haskey(qobject_to_qdict(resp), "error"));
 
+    qobject_decref(resp);
+    QDECREF(req);
+
+    /* check that with extra arguments it throws an error */
+    req = qdict_new();
+    qdict_put(args, "a", qint_from_int(66));
+    qdict_put(req, "arguments", args);
+
+    qdict_put_obj(req, "execute", QOBJECT(qstring_from_str("user_def_cmd")));
+
+    resp = qmp_dispatch(QOBJECT(req));
+    assert(resp != NULL);
+    assert(qdict_haskey(qobject_to_qdict(resp), "error"));
+
     qobject_decref(resp);
     QDECREF(req);
 }
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index eac64ce..2f603b0 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -95,6 +95,8 @@ def gen_marshal_decl(name):
 
 
 def gen_marshal(name, arg_type, boxed, ret_type):
+    have_args = arg_type and not arg_type.is_empty()
+
     ret = mcgen('''
 
 %(proto)s
@@ -109,17 +111,31 @@ def gen_marshal(name, arg_type, boxed, ret_type):
 ''',
                      c_type=ret_type.c_type())
 
-    if arg_type and not arg_type.is_empty():
+    if have_args:
+        visit_members = ('visit_type_%s_members(v, &arg, &err);'
+                         % arg_type.c_name())
         ret += mcgen('''
     Visitor *v;
     %(c_name)s arg = {0};
 
+''',
+                     c_name=arg_type.c_name())
+    else:
+        visit_members = ''
+        ret += mcgen('''
+    Visitor *v = NULL;
+
+    if (args) {
+''')
+        push_indent()
+
+    ret += mcgen('''
     v = qmp_input_visitor_new(QOBJECT(args), true);
     visit_start_struct(v, NULL, NULL, 0, &err);
     if (err) {
         goto out;
     }
-    visit_type_%(c_name)s_members(v, &arg, &err);
+    %(visit_members)s
     if (!err) {
         visit_check_struct(v, &err);
     }
@@ -128,35 +144,47 @@ def gen_marshal(name, arg_type, boxed, ret_type):
         goto out;
     }
 ''',
-                     c_name=arg_type.c_name())
+                 visit_members=visit_members)
 
-    else:
+    if not have_args:
+        pop_indent()
         ret += mcgen('''
-
-    (void)args;
+    }
 ''')
 
     ret += gen_call(name, arg_type, boxed, ret_type)
 
-    # 'goto out' produced above for arg_type, and by gen_call() for ret_type
-    if (arg_type and not arg_type.is_empty()) or ret_type:
-        ret += mcgen('''
+    ret += mcgen('''
 
 out:
-''')
-    ret += mcgen('''
     error_propagate(errp, err);
+    visit_free(v);
 ''')
-    if arg_type and not arg_type.is_empty():
+
+    if have_args:
+        visit_members = ('visit_type_%s_members(v, &arg, NULL);'
+                         % arg_type.c_name())
+    else:
+        visit_members = ''
         ret += mcgen('''
-    visit_free(v);
+    if (args) {
+''')
+        push_indent()
+
+    ret += mcgen('''
     v = qapi_dealloc_visitor_new();
     visit_start_struct(v, NULL, NULL, 0, NULL);
-    visit_type_%(c_name)s_members(v, &arg, NULL);
+    %(visit_members)s
     visit_end_struct(v, NULL);
     visit_free(v);
 ''',
-                     c_name=arg_type.c_name())
+                 visit_members=visit_members)
+
+    if not have_args:
+        pop_indent()
+        ret += mcgen('''
+    }
+''')
 
     ret += mcgen('''
 }
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch()
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (15 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 16/20] qapi: check invalid arguments on no-args commands Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 17:04   ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 18/20] build-sys: remove qmp-commands-old.h Marc-André Lureau
                   ` (2 subsequent siblings)
  19 siblings, 1 reply; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Replace the old manual dispatch and validation code by the generic one
provided by qapi common code.

Note that it is now possible to call the following commands that used to
be disabled by compile-time conditionals:
- dump-skeys
- query-spice
- rtc-reset-reinjection
- query-gic-capabilities

Their fallback functions return an appropriate "feature disabled" error.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 monitor.c    | 326 +++++++----------------------------------------------------
 trace-events |   1 -
 2 files changed, 34 insertions(+), 293 deletions(-)

diff --git a/monitor.c b/monitor.c
index f07fc31..81926c7 100644
--- a/monitor.c
+++ b/monitor.c
@@ -166,7 +166,6 @@ struct MonFdset {
 };
 
 typedef struct {
-    QObject *id;
     JSONMessageParser parser;
     /*
      * When a client connects, we're in capabilities negotiation mode.
@@ -229,8 +228,6 @@ static int mon_refcount;
 static mon_cmd_t mon_cmds[];
 static mon_cmd_t info_cmds[];
 
-static const mon_cmd_t qmp_cmds[];
-
 Monitor *cur_mon;
 
 static QEMUClockType event_clock_type = QEMU_CLOCK_REALTIME;
@@ -401,49 +398,6 @@ static void monitor_json_emitter(Monitor *mon, const QObject *data)
     QDECREF(json);
 }
 
-static QDict *build_qmp_error_dict(Error *err)
-{
-    QObject *obj;
-
-    obj = qobject_from_jsonf("{ 'error': { 'class': %s, 'desc': %s } }",
-                             QapiErrorClass_lookup[error_get_class(err)],
-                             error_get_pretty(err));
-
-    return qobject_to_qdict(obj);
-}
-
-static void monitor_protocol_emitter(Monitor *mon, QObject *data,
-                                     Error *err)
-{
-    QDict *qmp;
-
-    trace_monitor_protocol_emitter(mon);
-
-    if (!err) {
-        /* success response */
-        qmp = qdict_new();
-        if (data) {
-            qobject_incref(data);
-            qdict_put_obj(qmp, "return", data);
-        } else {
-            /* return an empty QDict by default */
-            qdict_put(qmp, "return", qdict_new());
-        }
-    } else {
-        /* error response */
-        qmp = build_qmp_error_dict(err);
-    }
-
-    if (mon->qmp.id) {
-        qdict_put_obj(qmp, "id", mon->qmp.id);
-        mon->qmp.id = NULL;
-    }
-
-    monitor_json_emitter(mon, QOBJECT(qmp));
-    QDECREF(qmp);
-}
-
-
 static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX] = {
     /* Limit guest-triggerable events to 1 per second */
     [QAPI_EVENT_RTC_CHANGE]        = { 1000 * SCALE_MS },
@@ -2180,11 +2134,6 @@ static mon_cmd_t mon_cmds[] = {
     { NULL, NULL, },
 };
 
-static const mon_cmd_t qmp_cmds[] = {
-#include "qmp-commands-old.h"
-    { /* NULL */ },
-};
-
 /*******************************************************************/
 
 static const char *pch;
@@ -2535,11 +2484,6 @@ static const mon_cmd_t *search_dispatch_table(const mon_cmd_t *disp_table,
     return NULL;
 }
 
-static const mon_cmd_t *qmp_find_cmd(const char *cmdname)
-{
-    return search_dispatch_table(qmp_cmds, cmdname);
-}
-
 /*
  * Parse command name from @cmdp according to command table @table.
  * If blank, return NULL.
@@ -3690,199 +3634,6 @@ static bool invalid_qmp_mode(const Monitor *mon, const char *cmd,
     return false;
 }
 
-/*
- * Argument validation rules:
- *
- * 1. The argument must exist in cmd_args qdict
- * 2. The argument type must be the expected one
- *
- * Special case: If the argument doesn't exist in cmd_args and
- *               the QMP_ACCEPT_UNKNOWNS flag is set, then the
- *               checking is skipped for it.
- */
-static void check_client_args_type(const QDict *client_args,
-                                   const QDict *cmd_args, int flags,
-                                   Error **errp)
-{
-    const QDictEntry *ent;
-
-    for (ent = qdict_first(client_args); ent;ent = qdict_next(client_args,ent)){
-        QObject *obj;
-        QString *arg_type;
-        const QObject *client_arg = qdict_entry_value(ent);
-        const char *client_arg_name = qdict_entry_key(ent);
-
-        obj = qdict_get(cmd_args, client_arg_name);
-        if (!obj) {
-            if (flags & QMP_ACCEPT_UNKNOWNS) {
-                /* handler accepts unknowns */
-                continue;
-            }
-            /* client arg doesn't exist */
-            error_setg(errp, QERR_INVALID_PARAMETER, client_arg_name);
-            return;
-        }
-
-        arg_type = qobject_to_qstring(obj);
-        assert(arg_type != NULL);
-
-        /* check if argument's type is correct */
-        switch (qstring_get_str(arg_type)[0]) {
-        case 'F':
-        case 'B':
-        case 's':
-            if (qobject_type(client_arg) != QTYPE_QSTRING) {
-                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
-                           client_arg_name, "string");
-                return;
-            }
-        break;
-        case 'i':
-        case 'l':
-        case 'M':
-        case 'o':
-            if (qobject_type(client_arg) != QTYPE_QINT) {
-                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
-                           client_arg_name, "int");
-                return;
-            }
-            break;
-        case 'T':
-            if (qobject_type(client_arg) != QTYPE_QINT &&
-                qobject_type(client_arg) != QTYPE_QFLOAT) {
-                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
-                           client_arg_name, "number");
-                return;
-            }
-            break;
-        case 'b':
-        case '-':
-            if (qobject_type(client_arg) != QTYPE_QBOOL) {
-                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
-                           client_arg_name, "bool");
-                return;
-            }
-            break;
-        case 'O':
-            assert(flags & QMP_ACCEPT_UNKNOWNS);
-            break;
-        case 'q':
-            /* Any QObject can be passed.  */
-            break;
-        case '/':
-        case '.':
-            /*
-             * These types are not supported by QMP and thus are not
-             * handled here. Fall through.
-             */
-        default:
-            abort();
-        }
-    }
-}
-
-/*
- * - Check if the client has passed all mandatory args
- * - Set special flags for argument validation
- */
-static void check_mandatory_args(const QDict *cmd_args,
-                                 const QDict *client_args, int *flags,
-                                 Error **errp)
-{
-    const QDictEntry *ent;
-
-    for (ent = qdict_first(cmd_args); ent; ent = qdict_next(cmd_args, ent)) {
-        const char *cmd_arg_name = qdict_entry_key(ent);
-        QString *type = qobject_to_qstring(qdict_entry_value(ent));
-        assert(type != NULL);
-
-        if (qstring_get_str(type)[0] == 'O') {
-            assert((*flags & QMP_ACCEPT_UNKNOWNS) == 0);
-            *flags |= QMP_ACCEPT_UNKNOWNS;
-        } else if (qstring_get_str(type)[0] != '-' &&
-                   qstring_get_str(type)[1] != '?' &&
-                   !qdict_haskey(client_args, cmd_arg_name)) {
-            error_setg(errp, QERR_MISSING_PARAMETER, cmd_arg_name);
-            return;
-        }
-    }
-}
-
-static QDict *qdict_from_args_type(const char *args_type)
-{
-    int i;
-    QDict *qdict;
-    QString *key, *type, *cur_qs;
-
-    assert(args_type != NULL);
-
-    qdict = qdict_new();
-
-    if (args_type == NULL || args_type[0] == '\0') {
-        /* no args, empty qdict */
-        goto out;
-    }
-
-    key = qstring_new();
-    type = qstring_new();
-
-    cur_qs = key;
-
-    for (i = 0;; i++) {
-        switch (args_type[i]) {
-            case ',':
-            case '\0':
-                qdict_put(qdict, qstring_get_str(key), type);
-                QDECREF(key);
-                if (args_type[i] == '\0') {
-                    goto out;
-                }
-                type = qstring_new(); /* qdict has ref */
-                cur_qs = key = qstring_new();
-                break;
-            case ':':
-                cur_qs = type;
-                break;
-            default:
-                qstring_append_chr(cur_qs, args_type[i]);
-                break;
-        }
-    }
-
-out:
-    return qdict;
-}
-
-/*
- * Client argument checking rules:
- *
- * 1. Client must provide all mandatory arguments
- * 2. Each argument provided by the client must be expected
- * 3. Each argument provided by the client must have the type expected
- *    by the command
- */
-static void qmp_check_client_args(const mon_cmd_t *cmd, QDict *client_args,
-                                  Error **errp)
-{
-    Error *err = NULL;
-    int flags;
-    QDict *cmd_args;
-
-    cmd_args = qdict_from_args_type(cmd->args_type);
-
-    flags = 0;
-    check_mandatory_args(cmd_args, client_args, &flags, &err);
-    if (err) {
-        goto out;
-    }
-
-    check_client_args_type(client_args, cmd_args, flags, &err);
-
-out:
-    error_propagate(errp, err);
-    QDECREF(cmd_args);
-}
-
 /*
  * Input object checking rules
  *
@@ -3941,67 +3692,58 @@ static QDict *qmp_check_input_obj(QObject *input_obj, Error **errp)
 
 static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
 {
-    Error *local_err = NULL;
-    QObject *obj, *data;
-    QDict *input, *args;
-    const mon_cmd_t *cmd;
-    QmpCommand *qcmd;
+    QObject *req, *rsp = NULL, *id = NULL;
+    QDict *qdict = NULL;
     const char *cmd_name;
     Monitor *mon = cur_mon;
+    Error *err = NULL;
 
-    args = input = NULL;
-    data = NULL;
-
-    obj = json_parser_parse(tokens, NULL);
-    if (!obj) {
-        // FIXME: should be triggered in json_parser_parse()
-        error_setg(&local_err, QERR_JSON_PARSING);
+    req = json_parser_parse_err(tokens, NULL, &err);
+    if (err || !req || qobject_type(req) != QTYPE_QDICT) {
+        if (!err) {
+            error_setg(&err, QERR_JSON_PARSING);
+        }
         goto err_out;
     }
 
-    input = qmp_check_input_obj(obj, &local_err);
-    if (!input) {
-        qobject_decref(obj);
+    qdict = qmp_check_input_obj(req, &err);
+    if (!qdict) {
         goto err_out;
     }
 
-    mon->qmp.id = qdict_get(input, "id");
-    qobject_incref(mon->qmp.id);
+    id = qdict_get(qdict, "id");
+    qobject_incref(id);
+    qdict_del(qdict, "id");
 
-    cmd_name = qdict_get_str(input, "execute");
+    cmd_name = qdict_get_str(qdict, "execute");
     trace_handle_qmp_command(mon, cmd_name);
-    cmd = qmp_find_cmd(cmd_name);
-    qcmd = qmp_find_command(cmd_name);
-    if (!qcmd || !cmd) {
-        error_set(&local_err, ERROR_CLASS_COMMAND_NOT_FOUND,
-                  "The command %s has not been found", cmd_name);
-        goto err_out;
-    }
-    if (invalid_qmp_mode(mon, cmd_name, &local_err)) {
+
+    if (invalid_qmp_mode(mon, cmd_name, &err)) {
         goto err_out;
     }
 
-    obj = qdict_get(input, "arguments");
-    if (!obj) {
-        args = qdict_new();
-    } else {
-        args = qobject_to_qdict(obj);
-        QINCREF(args);
-    }
+    rsp = qmp_dispatch(req);
 
-    qmp_check_client_args(cmd, args, &local_err);
-    if (local_err) {
-        goto err_out;
+err_out:
+    if (err) {
+        qdict = qdict_new();
+        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
+        error_free(err);
+        rsp = QOBJECT(qdict);
     }
 
-    qcmd->fn(args, &data, &local_err);
+    if (rsp) {
+        if (id) {
+            qdict_put_obj(qobject_to_qdict(rsp), "id", id);
+            id = NULL;
+        }
 
-err_out:
-    monitor_protocol_emitter(mon, data, local_err);
-    qobject_decref(data);
-    error_free(local_err);
-    QDECREF(input);
-    QDECREF(args);
+        monitor_json_emitter(mon, rsp);
+    }
+
+    qobject_decref(id);
+    qobject_decref(rsp);
+    qobject_decref(req);
 }
 
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
diff --git a/trace-events b/trace-events
index 616cc52..8d59631 100644
--- a/trace-events
+++ b/trace-events
@@ -98,7 +98,6 @@ qemu_co_mutex_unlock_return(void *mutex, void *self) "mutex %p self %p"
 
 # monitor.c
 handle_qmp_command(void *mon, const char *cmd_name) "mon %p cmd_name \"%s\""
-monitor_protocol_emitter(void *mon) "mon %p"
 monitor_protocol_event_handler(uint32_t event, void *qdict) "event=%d data=%p"
 monitor_protocol_event_emit(uint32_t event, void *data) "event=%d data=%p"
 monitor_protocol_event_queue(uint32_t event, void *qdict, uint64_t rate) "event=%d data=%p rate=%" PRId64
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 18/20] build-sys: remove qmp-commands-old.h
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (16 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch() Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 19/20] qmp-commands.hx: fix some styling Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt Marc-André Lureau
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 Makefile.target | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index 0ba50ae..3b1f59f 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -157,7 +157,7 @@ else
 obj-y += hw/$(TARGET_BASE_ARCH)/
 endif
 
-GENERATED_HEADERS += hmp-commands.h hmp-commands-info.h qmp-commands-old.h
+GENERATED_HEADERS += hmp-commands.h hmp-commands-info.h
 GENERATED_HEADERS += qmp-introspect.h
 GENERATED_SOURCES += qmp-introspect.c qmp-marshal.c qapi-event.c
 qapi-hconfig = ../config-host.h config-target.h
@@ -234,13 +234,10 @@ hmp-commands.h: $(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/scripts/hxtool
 hmp-commands-info.h: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxtool
 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"  GEN   $(TARGET_DIR)$@")
 
-qmp-commands-old.h: $(SRC_PATH)/qmp-commands.hx $(SRC_PATH)/scripts/hxtool
-	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"  GEN   $(TARGET_DIR)$@")
-
 clean:
 	rm -f *.a *~ $(PROGS)
 	rm -f $(shell find . -name '*.[od]')
-	rm -f hmp-commands.h qmp-commands-old.h gdbstub-xml.c
+	rm -f hmp-commands.h gdbstub-xml.c
 ifdef CONFIG_TRACE_SYSTEMTAP
 	rm -f *.stp
 endif
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 19/20] qmp-commands.hx: fix some styling
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (17 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 18/20] build-sys: remove qmp-commands-old.h Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt Marc-André Lureau
  19 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

Add some missing lines, remove superflous @ in command name, remove
trailing spaces.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 qmp-commands.hx | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)

diff --git a/qmp-commands.hx b/qmp-commands.hx
index f395f47..2e0d1e6 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -91,7 +91,7 @@ eject
 
 Eject a removable medium.
 
-Arguments: 
+Arguments:
 
 - force: force ejection (json-bool, optional)
 - device: device name (json-string)
@@ -1389,7 +1389,6 @@ EQMP
     },
 
 SQMP
-
 block-dirty-bitmap-add
 ----------------------
 Since 2.4
@@ -1416,7 +1415,6 @@ EQMP
     },
 
 SQMP
-
 block-dirty-bitmap-remove
 -------------------------
 Since 2.4
@@ -1443,7 +1441,6 @@ EQMP
     },
 
 SQMP
-
 block-dirty-bitmap-clear
 ------------------------
 Since 2.4
@@ -2166,7 +2163,7 @@ human-monitor-command
 
 Execute a Human Monitor command.
 
-Arguments: 
+Arguments:
 
 - command-line: the command name and its arguments, just like the
                 Human Monitor's shell (json-string)
@@ -2978,7 +2975,7 @@ Example:
                   },
                   "function":0,
                   "regions":[
-   
+
                   ]
                },
                {
@@ -2995,7 +2992,7 @@ Example:
                   },
                   "function":0,
                   "regions":[
-   
+
                   ]
                },
                {
@@ -3457,8 +3454,8 @@ The main json-object contains the following:
 - "setup-time" amount of setup time in milliseconds _before_ the
                iterations begin but _after_ the QMP command is issued.
                This is designed to provide an accounting of any activities
-               (such as RDMA pinning) which may be expensive, but do not 
-               actually occur during the iterative migration rounds 
+               (such as RDMA pinning) which may be expensive, but do not
+               actually occur during the iterative migration rounds
                themselves. (json-int)
 - "downtime": only present when migration has finished correctly
               total amount in ms for downtime that happened (json-int)
@@ -4369,8 +4366,8 @@ EQMP
     },
 
 SQMP
-@query-named-block-nodes
-------------------------
+query-named-block-nodes
+-----------------------
 
 Return a list of BlockDeviceInfo for all the named block driver nodes
 
@@ -4519,7 +4516,7 @@ EQMP
     },
 
 SQMP
-@query-memory-devices
+query-memory-devices
 --------------------
 
 Return a list of memory devices.
@@ -4537,6 +4534,7 @@ Example:
                         "slot": 0},
                    "type": "dimm"
                  } ] }
+
 EQMP
 
     {
@@ -4545,8 +4543,8 @@ EQMP
     },
 
 SQMP
-@query-acpi-ospm-status
---------------------
+query-acpi-ospm-status
+----------------------
 
 Return list of ACPIOSTInfo for devices that support status reporting
 via ACPI _OST method.
@@ -4558,6 +4556,7 @@ Example:
                  { "slot": "2", "slot-type": "DIMM", "source": 0, "status": 0},
                  { "slot": "3", "slot-type": "DIMM", "source": 0, "status": 0}
    ]}
+
 EQMP
 
 #if defined TARGET_I386
@@ -4579,6 +4578,7 @@ Example:
 
 -> { "execute": "rtc-reset-reinjection" }
 <- { "return": {} }
+
 EQMP
 
     {
@@ -4610,6 +4610,7 @@ Example:
 
 -> { "execute": "trace-event-get-state", "arguments": { "name": "qemu_memalign" } }
 <- { "return": [ { "name": "qemu_memalign", "state": "disabled" } ] }
+
 EQMP
 
     {
@@ -4644,6 +4645,7 @@ Example:
 
 -> { "execute": "trace-event-set-state", "arguments": { "name": "qemu_memalign", "enable": "true" } }
 <- { "return": {} }
+
 EQMP
 
     {
@@ -4652,8 +4654,8 @@ EQMP
     },
 
 SQMP
-@input-send-event
------------------
+input-send-event
+----------------
 
 Send input event to guest.
 
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt
  2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
                   ` (18 preceding siblings ...)
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 19/20] qmp-commands.hx: fix some styling Marc-André Lureau
@ 2016-08-17 16:57 ` Marc-André Lureau
  2016-09-09 12:43   ` Markus Armbruster
  19 siblings, 1 reply; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 16:57 UTC (permalink / raw)
  To: qemu-devel; +Cc: eblake, armbru, Marc-André Lureau

The only remaining function of qmp-commands.hx is to let us generate
qmp-commands.txt from it.  Replace qmp-commands.hx by qmp-commands.txt.

We intend to move the documentation into the QAPI schema and generate
qapi-commands.txt from it, but not right now.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 Makefile                                 |    6 +-
 .gitignore                               |    1 -
 MAINTAINERS                              |    1 -
 docs/qapi-code-gen.txt                   |    6 +-
 qmp-commands.hx => docs/qmp-commands.txt | 1112 ------------------------------
 docs/writing-qmp-commands.txt            |   38 -
 6 files changed, 4 insertions(+), 1160 deletions(-)
 rename qmp-commands.hx => docs/qmp-commands.txt (87%)

diff --git a/Makefile b/Makefile
index fc2e53a..dade87c 100644
--- a/Makefile
+++ b/Makefile
@@ -91,7 +91,6 @@ HELPERS-$(CONFIG_LINUX) = qemu-bridge-helper$(EXESUF)
 
 ifdef BUILD_DOCS
 DOCS=qemu-doc.html qemu-tech.html qemu.1 qemu-img.1 qemu-nbd.8 qemu-ga.8
-DOCS+=qmp-commands.txt
 ifdef CONFIG_VIRTFS
 DOCS+=fsdev/virtfs-proxy-helper.1
 endif
@@ -421,7 +420,7 @@ endif
 install-doc: $(DOCS)
 	$(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)"
 	$(INSTALL_DATA) qemu-doc.html  qemu-tech.html "$(DESTDIR)$(qemu_docdir)"
-	$(INSTALL_DATA) qmp-commands.txt "$(DESTDIR)$(qemu_docdir)"
+	$(INSTALL_DATA) docs/qmp-commands.txt "$(DESTDIR)$(qemu_docdir)"
 ifdef CONFIG_POSIX
 	$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man1"
 	$(INSTALL_DATA) qemu.1 "$(DESTDIR)$(mandir)/man1"
@@ -544,9 +543,6 @@ qemu-monitor.texi: $(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/scripts/hxtool
 qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxtool
 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"  GEN   $@")
 
-qmp-commands.txt: $(SRC_PATH)/qmp-commands.hx $(SRC_PATH)/scripts/hxtool
-	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -q < $< > $@,"  GEN   $@")
-
 qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"  GEN   $@")
 
diff --git a/.gitignore b/.gitignore
index 88ec249..0ab8263 100644
--- a/.gitignore
+++ b/.gitignore
@@ -53,7 +53,6 @@
 /qemu-bridge-helper
 /qemu-monitor.texi
 /qemu-monitor-info.texi
-/qmp-commands.txt
 /vscclient
 /fsdev/virtfs-proxy-helper
 *.[1-9]
diff --git a/MAINTAINERS b/MAINTAINERS
index b6fb84e..84bdc46 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1237,7 +1237,6 @@ M: Markus Armbruster <armbru@redhat.com>
 S: Supported
 F: qmp.c
 F: monitor.c
-F: qmp-commands.hx
 F: docs/*qmp-*
 F: scripts/qmp/
 T: git git://repo.or.cz/qemu/armbru.git qapi-next
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index de298dc..5d4c2cd 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -964,9 +964,9 @@ Example:
 
 Used to generate the marshaling/dispatch functions for the commands
 defined in the schema. The generated code implements
-qmp_marshal_COMMAND() (mentioned in qmp-commands.hx, and registered
-automatically), and declares qmp_COMMAND() that the user must
-implement.  The following files are generated:
+qmp_marshal_COMMAND() (registered automatically), and declares
+qmp_COMMAND() that the user must implement.  The following files are
+generated:
 
 $(prefix)qmp-marshal.c: command marshal/dispatch functions for each
                         QMP command defined in the schema. Functions
diff --git a/qmp-commands.hx b/docs/qmp-commands.txt
similarity index 87%
rename from qmp-commands.hx
rename to docs/qmp-commands.txt
index 2e0d1e6..90cf122 100644
--- a/qmp-commands.hx
+++ b/docs/qmp-commands.txt
@@ -1,8 +1,3 @@
-HXCOMM QMP dispatch table and documentation
-HXCOMM Text between SQMP and EQMP is copied to the QMP documentation file and
-HXCOMM does not show up in the other formats.
-
-SQMP
                         QMP Supported Commands
                         ----------------------
 
@@ -58,14 +53,6 @@ If you're planning to adopt QMP, please observe the following:
 Server's responses in the examples below are always a success response, please
 refer to the QMP specification for more details on error responses.
 
-EQMP
-
-    {
-        .name       = "quit",
-        .args_type  = "",
-    },
-
-SQMP
 quit
 ----
 
@@ -78,14 +65,6 @@ Example:
 -> { "execute": "quit" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "eject",
-        .args_type  = "force:-f,device:B",
-    },
-
-SQMP
 eject
 -----
 
@@ -103,14 +82,6 @@ Example:
 
 Note: The "force" argument defaults to false.
 
-EQMP
-
-    {
-        .name       = "change",
-        .args_type  = "device:B,target:F,arg:s?",
-    },
-
-SQMP
 change
 ------
 
@@ -138,14 +109,6 @@ Examples:
                             "arg": "foobar1" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "screendump",
-        .args_type  = "filename:F",
-    },
-
-SQMP
 screendump
 ----------
 
@@ -160,14 +123,6 @@ Example:
 -> { "execute": "screendump", "arguments": { "filename": "/tmp/image" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "stop",
-        .args_type  = "",
-    },
-
-SQMP
 stop
 ----
 
@@ -180,14 +135,6 @@ Example:
 -> { "execute": "stop" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "cont",
-        .args_type  = "",
-    },
-
-SQMP
 cont
 ----
 
@@ -200,14 +147,6 @@ Example:
 -> { "execute": "cont" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "system_wakeup",
-        .args_type  = "",
-    },
-
-SQMP
 system_wakeup
 -------------
 
@@ -220,14 +159,6 @@ Example:
 -> { "execute": "system_wakeup" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "system_reset",
-        .args_type  = "",
-    },
-
-SQMP
 system_reset
 ------------
 
@@ -240,14 +171,6 @@ Example:
 -> { "execute": "system_reset" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "system_powerdown",
-        .args_type  = "",
-    },
-
-SQMP
 system_powerdown
 ----------------
 
@@ -260,16 +183,6 @@ Example:
 -> { "execute": "system_powerdown" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "device_add",
-        .args_type  = "device:O",
-        .params     = "driver[,prop=value][,...]",
-        .help       = "add device, like -device on the command line",
-    },
-
-SQMP
 device_add
 ----------
 
@@ -295,14 +208,6 @@ Notes:
 (2) It's possible to list device properties by running QEMU with the
     "-device DEVICE,\?" command-line argument, where DEVICE is the device's name
 
-EQMP
-
-    {
-        .name       = "device_del",
-        .args_type  = "id:s",
-    },
-
-SQMP
 device_del
 ----------
 
@@ -322,14 +227,6 @@ Example:
 -> { "execute": "device_del", "arguments": { "id": "/machine/peripheral-anon/device[0]" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "send-key",
-        .args_type  = "keys:q,hold-time:i?",
-    },
-
-SQMP
 send-key
 ----------
 
@@ -352,14 +249,6 @@ Example:
                               { "type": "qcode", "data": "delete" } ] } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "cpu",
-        .args_type  = "index:i",
-    },
-
-SQMP
 cpu
 ---
 
@@ -376,14 +265,6 @@ Example:
 
 Note: CPUs' indexes are obtained with the 'query-cpus' command.
 
-EQMP
-
-    {
-        .name       = "cpu-add",
-        .args_type  = "id:i",
-    },
-
-SQMP
 cpu-add
 -------
 
@@ -398,14 +279,6 @@ Example:
 -> { "execute": "cpu-add", "arguments": { "id": 2 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "memsave",
-        .args_type  = "val:l,size:i,filename:s,cpu:i?",
-    },
-
-SQMP
 memsave
 -------
 
@@ -426,14 +299,6 @@ Example:
                             "filename": "/tmp/virtual-mem-dump" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "pmemsave",
-        .args_type  = "val:l,size:i,filename:s",
-    },
-
-SQMP
 pmemsave
 --------
 
@@ -453,14 +318,6 @@ Example:
                             "filename": "/tmp/physical-mem-dump" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "inject-nmi",
-        .args_type  = "",
-    },
-
-SQMP
 inject-nmi
 ----------
 
@@ -475,14 +332,6 @@ Example:
 
 Note: inject-nmi fails when the guest doesn't support injecting.
 
-EQMP
-
-    {
-        .name       = "ringbuf-write",
-        .args_type  = "device:s,data:s,format:s?",
-    },
-
-SQMP
 ringbuf-write
 -------------
 
@@ -503,14 +352,6 @@ Example:
                                "format": "utf8" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "ringbuf-read",
-        .args_type  = "device:s,size:i,format:s?",
-    },
-
-SQMP
 ringbuf-read
 -------------
 
@@ -538,14 +379,6 @@ Example:
                                "format": "utf8" } }
 <- {"return": "abcdefgh"}
 
-EQMP
-
-    {
-        .name       = "xen-save-devices-state",
-        .args_type  = "filename:F",
-    },
-
-SQMP
 xen-save-devices-state
 -------
 
@@ -564,14 +397,6 @@ Example:
      "arguments": { "filename": "/tmp/save" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "xen-load-devices-state",
-        .args_type  = "filename:F",
-    },
-
-SQMP
 xen-load-devices-state
 ----------------------
 
@@ -590,14 +415,6 @@ Example:
      "arguments": { "filename": "/tmp/resume" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "xen-set-global-dirty-log",
-        .args_type  = "enable:b",
-    },
-
-SQMP
 xen-set-global-dirty-log
 -------
 
@@ -613,14 +430,6 @@ Example:
      "arguments": { "enable": true } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "migrate",
-        .args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
-    },
-
-SQMP
 migrate
 -------
 
@@ -645,14 +454,6 @@ Notes:
 (3) The user Monitor's "detach" argument is invalid in QMP and should not
     be used
 
-EQMP
-
-    {
-        .name       = "migrate_cancel",
-        .args_type  = "",
-    },
-
-SQMP
 migrate_cancel
 --------------
 
@@ -665,14 +466,6 @@ Example:
 -> { "execute": "migrate_cancel" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "migrate-incoming",
-        .args_type  = "uri:s",
-    },
-
-SQMP
 migrate-incoming
 ----------------
 
@@ -693,13 +486,6 @@ Notes:
     be used
 (2) The uri format is the same as for -incoming
 
-EQMP
-    {
-        .name       = "migrate-set-cache-size",
-        .args_type  = "value:o",
-    },
-
-SQMP
 migrate-set-cache-size
 ----------------------
 
@@ -715,13 +501,6 @@ Example:
 -> { "execute": "migrate-set-cache-size", "arguments": { "value": 536870912 } }
 <- { "return": {} }
 
-EQMP
-    {
-        .name       = "migrate-start-postcopy",
-        .args_type  = "",
-    },
-
-SQMP
 migrate-start-postcopy
 ----------------------
 
@@ -732,14 +511,6 @@ Example:
 -> { "execute": "migrate-start-postcopy" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "query-migrate-cache-size",
-        .args_type  = "",
-    },
-
-SQMP
 query-migrate-cache-size
 ------------------------
 
@@ -753,14 +524,6 @@ Example:
 -> { "execute": "query-migrate-cache-size" }
 <- { "return": 67108864 }
 
-EQMP
-
-    {
-        .name       = "migrate_set_speed",
-        .args_type  = "value:o",
-    },
-
-SQMP
 migrate_set_speed
 -----------------
 
@@ -775,14 +538,6 @@ Example:
 -> { "execute": "migrate_set_speed", "arguments": { "value": 1024 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "migrate_set_downtime",
-        .args_type  = "value:T",
-    },
-
-SQMP
 migrate_set_downtime
 --------------------
 
@@ -797,16 +552,6 @@ Example:
 -> { "execute": "migrate_set_downtime", "arguments": { "value": 0.1 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "client_migrate_info",
-        .args_type  = "protocol:s,hostname:s,port:i?,tls-port:i?,cert-subject:s?",
-        .params     = "protocol hostname port tls-port cert-subject",
-        .help       = "set migration information for remote display",
-    },
-
-SQMP
 client_migrate_info
 -------------------
 
@@ -830,16 +575,6 @@ Example:
                     "port": 1234 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "dump-guest-memory",
-        .args_type  = "paging:b,protocol:s,detach:b?,begin:i?,end:i?,format:s?",
-        .params     = "-p protocol [-d] [begin] [length] [format]",
-        .help       = "dump guest memory to file",
-    },
-
-SQMP
 dump
 
 
@@ -870,14 +605,6 @@ Notes:
 
 (1) All boolean arguments default to false
 
-EQMP
-
-    {
-        .name       = "query-dump-guest-memory-capability",
-        .args_type  = "",
-    },
-
-SQMP
 query-dump-guest-memory-capability
 ----------
 
@@ -889,16 +616,6 @@ Example:
 <- { "return": { "formats":
                     ["elf", "kdump-zlib", "kdump-lzo", "kdump-snappy"] }
 
-EQMP
-
-    {
-        .name       = "query-dump",
-        .args_type  = "",
-        .params     = "",
-        .help       = "query background dump status",
-    },
-
-SQMP
 query-dump
 ----------
 
@@ -912,16 +629,6 @@ Example:
 <- { "return": { "status": "active", "completed": 1024000,
                  "total": 2048000 } }
 
-EQMP
-
-#if defined TARGET_S390X
-    {
-        .name       = "dump-skeys",
-        .args_type  = "filename:F",
-    },
-#endif
-
-SQMP
 dump-skeys
 ----------
 
@@ -936,14 +643,6 @@ Example:
 -> { "execute": "dump-skeys", "arguments": { "filename": "/tmp/skeys" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "netdev_add",
-        .args_type  = "netdev:O",
-    },
-
-SQMP
 netdev_add
 ----------
 
@@ -966,14 +665,6 @@ Note: The supported device options are the same ones supported by the '-netdev'
       command-line argument, which are listed in the '-help' output or QEMU's
       manual
 
-EQMP
-
-    {
-        .name       = "netdev_del",
-        .args_type  = "id:s",
-    },
-
-SQMP
 netdev_del
 ----------
 
@@ -989,14 +680,6 @@ Example:
 <- { "return": {} }
 
 
-EQMP
-
-    {
-        .name       = "object-add",
-        .args_type  = "qom-type:s,id:s,props:q?",
-    },
-
-SQMP
 object-add
 ----------
 
@@ -1014,14 +697,6 @@ Example:
      "props": { "filename": "/dev/hwrng" } } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "object-del",
-        .args_type  = "id:s",
-    },
-
-SQMP
 object-del
 ----------
 
@@ -1037,15 +712,6 @@ Example:
 <- { "return": {} }
 
 
-EQMP
-
-
-    {
-        .name       = "block_resize",
-        .args_type  = "device:s?,node-name:s?,size:o",
-    },
-
-SQMP
 block_resize
 ------------
 
@@ -1062,14 +728,6 @@ Example:
 -> { "execute": "block_resize", "arguments": { "device": "scratch", "size": 1073741824 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-stream",
-        .args_type  = "job-id:s?,device:B,base:s?,speed:o?,backing-file:s?,on-error:s?",
-    },
-
-SQMP
 block-stream
 ------------
 
@@ -1106,14 +764,6 @@ Example:
                                                "base": "/tmp/master.qcow2" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-commit",
-        .args_type  = "job-id:s?,device:B,base:s?,top:s?,backing-file:s?,speed:o?",
-    },
-
-SQMP
 block-commit
 ------------
 
@@ -1170,15 +820,6 @@ Example:
                                               "top": "/tmp/snap1.qcow2" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "drive-backup",
-        .args_type  = "job-id:s?,sync:s,device:B,target:s,speed:i?,mode:s?,"
-                      "format:s?,bitmap:s?,on-source-error:s?,on-target-error:s?",
-    },
-
-SQMP
 drive-backup
 ------------
 
@@ -1225,15 +866,6 @@ Example:
                                                "target": "backup.img" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-backup",
-        .args_type  = "job-id:s?,sync:s,device:B,target:B,speed:i?,"
-                      "on-source-error:s?,on-target-error:s?",
-    },
-
-SQMP
 blockdev-backup
 ---------------
 
@@ -1267,35 +899,6 @@ Example:
                                                   "target": "tgt-id" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-job-set-speed",
-        .args_type  = "device:B,speed:o",
-    },
-
-    {
-        .name       = "block-job-cancel",
-        .args_type  = "device:B,force:b?",
-    },
-    {
-        .name       = "block-job-pause",
-        .args_type  = "device:B",
-    },
-    {
-        .name       = "block-job-resume",
-        .args_type  = "device:B",
-    },
-    {
-        .name       = "block-job-complete",
-        .args_type  = "device:B",
-    },
-    {
-        .name       = "transaction",
-        .args_type  = "actions:q,properties:q?",
-    },
-
-SQMP
 transaction
 -----------
 
@@ -1381,14 +984,6 @@ Example:
                                          "name": "snapshot0" } } ] } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-dirty-bitmap-add",
-        .args_type  = "node:B,name:s,granularity:i?",
-    },
-
-SQMP
 block-dirty-bitmap-add
 ----------------------
 Since 2.4
@@ -1407,14 +1002,6 @@ Example:
                                                    "name": "bitmap0" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-dirty-bitmap-remove",
-        .args_type  = "node:B,name:s",
-    },
-
-SQMP
 block-dirty-bitmap-remove
 -------------------------
 Since 2.4
@@ -1433,14 +1020,6 @@ Example:
                                                       "name": "bitmap0" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-dirty-bitmap-clear",
-        .args_type  = "node:B,name:s",
-    },
-
-SQMP
 block-dirty-bitmap-clear
 ------------------------
 Since 2.4
@@ -1460,14 +1039,6 @@ Example:
                                                            "name": "bitmap0" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-snapshot-sync",
-        .args_type  = "device:s?,node-name:s?,snapshot-file:s,snapshot-node-name:s?,format:s?,mode:s?",
-    },
-
-SQMP
 blockdev-snapshot-sync
 ----------------------
 
@@ -1495,14 +1066,6 @@ Example:
                                                         "format": "qcow2" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-snapshot",
-        .args_type  = "node:s,overlay:s",
-    },
-
-SQMP
 blockdev-snapshot
 -----------------
 Since 2.5
@@ -1532,14 +1095,6 @@ Example:
                                                     "overlay": "node1534" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-snapshot-internal-sync",
-        .args_type  = "device:B,name:s",
-    },
-
-SQMP
 blockdev-snapshot-internal-sync
 -------------------------------
 
@@ -1560,14 +1115,6 @@ Example:
    }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-snapshot-delete-internal-sync",
-        .args_type  = "device:B,id:s?,name:s?",
-    },
-
-SQMP
 blockdev-snapshot-delete-internal-sync
 --------------------------------------
 
@@ -1599,18 +1146,6 @@ Example:
      }
    }
 
-EQMP
-
-    {
-        .name       = "drive-mirror",
-        .args_type  = "job-id:s?,sync:s,device:B,target:s,speed:i?,mode:s?,"
-                      "format:s?,node-name:s?,replaces:s?,"
-                      "on-source-error:s?,on-target-error:s?,"
-                      "unmap:b?,"
-                      "granularity:i?,buf-size:i?",
-    },
-
-SQMP
 drive-mirror
 ------------
 
@@ -1664,16 +1199,6 @@ Example:
                                                "format": "qcow2" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-mirror",
-        .args_type  = "job-id:s?,sync:s,device:B,target:B,replaces:s?,speed:i?,"
-                      "on-source-error:s?,on-target-error:s?,"
-                      "granularity:i?,buf-size:i?",
-    },
-
-SQMP
 blockdev-mirror
 ------------
 
@@ -1714,13 +1239,6 @@ Example:
                                                   "sync": "full" } }
 <- { "return": {} }
 
-EQMP
-    {
-        .name       = "change-backing-file",
-        .args_type  = "device:s,image-node-name:s,backing-file:s",
-    },
-
-SQMP
 change-backing-file
 -------------------
 Since: 2.1
@@ -1751,14 +1269,6 @@ Arguments:
 Returns: Nothing on success
          If "device" does not exist or cannot be determined, DeviceNotFound
 
-EQMP
-
-    {
-        .name       = "balloon",
-        .args_type  = "value:M",
-    },
-
-SQMP
 balloon
 -------
 
@@ -1773,14 +1283,6 @@ Example:
 -> { "execute": "balloon", "arguments": { "value": 536870912 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "set_link",
-        .args_type  = "name:s,up:b",
-    },
-
-SQMP
 set_link
 --------
 
@@ -1796,16 +1298,6 @@ Example:
 -> { "execute": "set_link", "arguments": { "name": "e1000.0", "up": false } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "getfd",
-        .args_type  = "fdname:s",
-        .params     = "getfd name",
-        .help       = "receive a file descriptor via SCM rights and assign it a name",
-    },
-
-SQMP
 getfd
 -----
 
@@ -1828,16 +1320,6 @@ Notes:
 (2) The 'closefd' command can be used to explicitly close the file
     descriptor when it is no longer needed.
 
-EQMP
-
-    {
-        .name       = "closefd",
-        .args_type  = "fdname:s",
-        .params     = "closefd name",
-        .help       = "close a file descriptor previously passed via SCM rights",
-    },
-
-SQMP
 closefd
 -------
 
@@ -1852,16 +1334,6 @@ Example:
 -> { "execute": "closefd", "arguments": { "fdname": "fd1" } }
 <- { "return": {} }
 
-EQMP
-
-     {
-        .name       = "add-fd",
-        .args_type  = "fdset-id:i?,opaque:s?",
-        .params     = "add-fd fdset-id opaque",
-        .help       = "Add a file descriptor, that was passed via SCM rights, to an fd set",
-    },
-
-SQMP
 add-fd
 -------
 
@@ -1890,16 +1362,6 @@ Notes:
 (1) The list of fd sets is shared by all monitor connections.
 (2) If "fdset-id" is not specified, a new fd set will be created.
 
-EQMP
-
-     {
-        .name       = "remove-fd",
-        .args_type  = "fdset-id:i,fd:i?",
-        .params     = "remove-fd fdset-id fd",
-        .help       = "Remove a file descriptor from an fd set",
-    },
-
-SQMP
 remove-fd
 ---------
 
@@ -1922,15 +1384,6 @@ Notes:
 (2) If "fd" is not specified, all file descriptors in "fdset-id" will be
     removed.
 
-EQMP
-
-    {
-        .name       = "query-fdsets",
-        .args_type  = "",
-        .help       = "Return information describing all fd sets",
-    },
-
-SQMP
 query-fdsets
 -------------
 
@@ -1971,14 +1424,6 @@ Example:
 
 Note: The list of fd sets is shared by all monitor connections.
 
-EQMP
-
-    {
-        .name       = "block_passwd",
-        .args_type  = "device:s?,node-name:s?,password:s",
-    },
-
-SQMP
 block_passwd
 ------------
 
@@ -1996,14 +1441,6 @@ Example:
                                                "password": "12345" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block_set_io_throttle",
-        .args_type  = "device:B,bps:l,bps_rd:l,bps_wr:l,iops:l,iops_rd:l,iops_wr:l,bps_max:l?,bps_rd_max:l?,bps_wr_max:l?,iops_max:l?,iops_rd_max:l?,iops_wr_max:l?,bps_max_length:l?,bps_rd_max_length:l?,bps_wr_max_length:l?,iops_max_length:l?,iops_rd_max_length:l?,iops_wr_max_length:l?,iops_size:l?,group:s?",
-    },
-
-SQMP
 block_set_io_throttle
 ------------
 
@@ -2052,14 +1489,6 @@ Example:
                                                "iops_size": 0 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "set_password",
-        .args_type  = "protocol:s,password:s,connected:s?",
-    },
-
-SQMP
 set_password
 ------------
 
@@ -2077,14 +1506,6 @@ Example:
                                                "password": "secret" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "expire_password",
-        .args_type  = "protocol:s,time:s",
-    },
-
-SQMP
 expire_password
 ---------------
 
@@ -2101,14 +1522,6 @@ Example:
                                                   "time": "+60" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "add_client",
-        .args_type  = "protocol:s,fdname:s,skipauth:b?,tls:b?",
-    },
-
-SQMP
 add_client
 ----------
 
@@ -2127,15 +1540,6 @@ Example:
                                              "fdname": "myclient" } }
 <- { "return": {} }
 
-EQMP
-    {
-        .name       = "qmp_capabilities",
-        .args_type  = "",
-        .params     = "",
-        .help       = "enable QMP capabilities",
-    },
-
-SQMP
 qmp_capabilities
 ----------------
 
@@ -2150,14 +1554,6 @@ Example:
 
 Note: This command must be issued before issuing any other command.
 
-EQMP
-
-    {
-        .name       = "human-monitor-command",
-        .args_type  = "command-line:s,cpu-index:i?",
-    },
-
-SQMP
 human-monitor-command
 ---------------------
 
@@ -2194,13 +1590,7 @@ Notes:
 3. Query Commands
 =================
 
-HXCOMM Each query command below is inside a SQMP/EQMP section, do NOT change
-HXCOMM this! We will possibly move query commands definitions inside those
-HXCOMM sections, just like regular commands.
 
-EQMP
-
-SQMP
 query-version
 -------------
 
@@ -2228,14 +1618,6 @@ Example:
       }
    }
 
-EQMP
-
-    {
-        .name       = "query-version",
-        .args_type  = "",
-    },
-
-SQMP
 query-commands
 --------------
 
@@ -2264,14 +1646,6 @@ Example:
 
 Note: This example has been shortened as the real response is too long.
 
-EQMP
-
-    {
-        .name       = "query-commands",
-        .args_type  = "",
-    },
-
-SQMP
 query-events
 --------------
 
@@ -2300,14 +1674,6 @@ Example:
 
 Note: This example has been shortened as the real response is too long.
 
-EQMP
-
-    {
-        .name       = "query-events",
-        .args_type  = "",
-    },
-
-SQMP
 query-qmp-schema
 ----------------
 
@@ -2316,14 +1682,6 @@ named schema entities.  Entities are commands, events and various
 types.  See docs/qapi-code-gen.txt for information on their structure
 and intended use.
 
-EQMP
-
-    {
-        .name       = "query-qmp-schema",
-        .args_type  = "",
-    },
-
-SQMP
 query-chardev
 -------------
 
@@ -2360,14 +1718,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-chardev",
-        .args_type  = "",
-    },
-
-SQMP
 query-chardev-backends
 -------------
 
@@ -2400,14 +1750,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-chardev-backends",
-        .args_type  = "",
-    },
-
-SQMP
 query-block
 -----------
 
@@ -2583,14 +1925,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-block",
-        .args_type  = "",
-    },
-
-SQMP
 query-blockstats
 ----------------
 
@@ -2779,14 +2113,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-blockstats",
-        .args_type  = "query-nodes:b?",
-    },
-
-SQMP
 query-cpus
 ----------
 
@@ -2833,14 +2159,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-cpus",
-        .args_type  = "",
-    },
-
-SQMP
 query-iothreads
 ---------------
 
@@ -2871,14 +2189,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-iothreads",
-        .args_type  = "",
-    },
-
-SQMP
 query-pci
 ---------
 
@@ -3087,14 +2397,6 @@ Example:
 
 Note: This example has been shortened as the real response is too long.
 
-EQMP
-
-    {
-        .name       = "query-pci",
-        .args_type  = "",
-    },
-
-SQMP
 query-kvm
 ---------
 
@@ -3110,14 +2412,6 @@ Example:
 -> { "execute": "query-kvm" }
 <- { "return": { "enabled": true, "present": true } }
 
-EQMP
-
-    {
-        .name       = "query-kvm",
-        .args_type  = "",
-    },
-
-SQMP
 query-status
 ------------
 
@@ -3149,14 +2443,6 @@ Example:
 -> { "execute": "query-status" }
 <- { "return": { "running": true, "singlestep": false, "status": "running" } }
 
-EQMP
-    
-    {
-        .name       = "query-status",
-        .args_type  = "",
-    },
-
-SQMP
 query-mice
 ----------
 
@@ -3192,14 +2478,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "query-mice",
-        .args_type  = "",
-    },
-
-SQMP
 query-vnc
 ---------
 
@@ -3254,18 +2532,6 @@ Example:
       }
    }
 
-EQMP
-#if defined(CONFIG_VNC)
-    {
-        .name       = "query-vnc",
-        .args_type  = "",
-    },
-    {
-        .name       = "query-vnc-servers",
-        .args_type  = "",
-    },
-#endif
-SQMP
 query-spice
 -----------
 
@@ -3333,16 +2599,6 @@ Example:
       }
    }
 
-EQMP
-
-#if defined(CONFIG_SPICE)
-    {
-        .name       = "query-spice",
-        .args_type  = "",
-    },
-#endif
-
-SQMP
 query-name
 ----------
 
@@ -3357,14 +2613,6 @@ Example:
 -> { "execute": "query-name" }
 <- { "return": { "name": "qemu-name" } }
 
-EQMP
-
-    {
-        .name       = "query-name",
-        .args_type  = "",
-    },
-
-SQMP
 query-uuid
 ----------
 
@@ -3379,14 +2627,6 @@ Example:
 -> { "execute": "query-uuid" }
 <- { "return": { "UUID": "550e8400-e29b-41d4-a716-446655440000" } }
 
-EQMP
-
-    {
-        .name       = "query-uuid",
-        .args_type  = "",
-    },
-
-SQMP
 query-command-line-options
 --------------------------
 
@@ -3427,14 +2667,6 @@ Example:
      ]
    }
 
-EQMP
-
-    {
-        .name       = "query-command-line-options",
-        .args_type  = "option:s?",
-    },
-
-SQMP
 query-migrate
 -------------
 
@@ -3604,14 +2836,6 @@ Examples:
       }
    }
 
-EQMP
-
-    {
-        .name       = "query-migrate",
-        .args_type  = "",
-    },
-
-SQMP
 migrate-set-capabilities
 ------------------------
 
@@ -3632,14 +2856,6 @@ Example:
 -> { "execute": "migrate-set-capabilities" , "arguments":
      { "capabilities": [ { "capability": "xbzrle", "state": true } ] } }
 
-EQMP
-
-    {
-        .name       = "migrate-set-capabilities",
-        .args_type  = "capabilities:q",
-        .params     = "capability:s,state:b",
-    },
-SQMP
 query-migrate-capabilities
 --------------------------
 
@@ -3669,14 +2885,6 @@ Example:
      {"state": false, "capability": "postcopy-ram"}
    ]}
 
-EQMP
-
-    {
-        .name       = "query-migrate-capabilities",
-        .args_type  = "",
-    },
-
-SQMP
 migrate-set-parameters
 ----------------------
 
@@ -3697,14 +2905,6 @@ Example:
 -> { "execute": "migrate-set-parameters" , "arguments":
       { "compress-level": 1 } }
 
-EQMP
-
-    {
-        .name       = "migrate-set-parameters",
-        .args_type  =
-            "compress-level:i?,compress-threads:i?,decompress-threads:i?,cpu-throttle-initial:i?,cpu-throttle-increment:i?",
-    },
-SQMP
 query-migrate-parameters
 ------------------------
 
@@ -3734,14 +2934,6 @@ Example:
       }
    }
 
-EQMP
-
-    {
-        .name       = "query-migrate-parameters",
-        .args_type  = "",
-    },
-
-SQMP
 query-balloon
 -------------
 
@@ -3761,82 +2953,6 @@ Example:
       }
    }
 
-EQMP
-
-    {
-        .name       = "query-balloon",
-        .args_type  = "",
-    },
-
-    {
-        .name       = "query-block-jobs",
-        .args_type  = "",
-    },
-
-    {
-        .name       = "qom-list",
-        .args_type  = "path:s",
-    },
-
-    {
-        .name       = "qom-set",
-	.args_type  = "path:s,property:s,value:q",
-    },
-
-    {
-        .name       = "qom-get",
-	.args_type  = "path:s,property:s",
-    },
-
-    {
-        .name       = "nbd-server-start",
-        .args_type  = "addr:q,tls-creds:s?",
-    },
-    {
-        .name       = "nbd-server-add",
-        .args_type  = "device:B,writable:b?",
-    },
-    {
-        .name       = "nbd-server-stop",
-        .args_type  = "",
-    },
-#if defined(CONFIG_VNC)
-    {
-        .name       = "change-vnc-password",
-        .args_type  = "password:s",
-    },
-#endif
-    {
-        .name       = "qom-list-types",
-        .args_type  = "implements:s?,abstract:b?",
-    },
-
-    {
-        .name       = "device-list-properties",
-        .args_type  = "typename:s",
-    },
-
-    {
-        .name       = "query-machines",
-        .args_type  = "",
-    },
-
-    {
-        .name       = "query-cpu-definitions",
-        .args_type  = "",
-    },
-
-    {
-        .name       = "query-target",
-        .args_type  = "",
-    },
-
-    {
-        .name       = "query-tpm",
-        .args_type  = "",
-    },
-
-SQMP
 query-tpm
 ---------
 
@@ -3862,14 +2978,6 @@ Example:
      ]
    }
 
-EQMP
-
-    {
-        .name       = "query-tpm-models",
-        .args_type  = "",
-    },
-
-SQMP
 query-tpm-models
 ----------------
 
@@ -3882,14 +2990,6 @@ Example:
 -> { "execute": "query-tpm-models" }
 <- { "return": [ "tpm-tis" ] }
 
-EQMP
-
-    {
-        .name       = "query-tpm-types",
-        .args_type  = "",
-    },
-
-SQMP
 query-tpm-types
 ---------------
 
@@ -3902,14 +3002,6 @@ Example:
 -> { "execute": "query-tpm-types" }
 <- { "return": [ "passthrough" ] }
 
-EQMP
-
-    {
-        .name       = "chardev-add",
-        .args_type  = "id:s,backend:q",
-    },
-
-SQMP
 chardev-add
 ----------------
 
@@ -3938,15 +3030,6 @@ Examples:
                      "backend" : { "type" : "pty", "data" : {} } } }
 <- { "return": { "pty" : "/dev/pty/42" } }
 
-EQMP
-
-    {
-        .name       = "chardev-remove",
-        .args_type  = "id:s",
-    },
-
-
-SQMP
 chardev-remove
 --------------
 
@@ -3961,13 +3044,6 @@ Example:
 -> { "execute": "chardev-remove", "arguments": { "id" : "foo" } }
 <- { "return": {} }
 
-EQMP
-    {
-        .name       = "query-rx-filter",
-        .args_type  = "name:s?",
-    },
-
-SQMP
 query-rx-filter
 ---------------
 
@@ -4025,14 +3101,6 @@ Example:
       ]
    }
 
-EQMP
-
-    {
-        .name       = "blockdev-add",
-        .args_type  = "options:q",
-    },
-
-SQMP
 blockdev-add
 ------------
 
@@ -4083,14 +3151,6 @@ Example (2):
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "x-blockdev-del",
-        .args_type  = "id:s?,node-name:s?",
-    },
-
-SQMP
 x-blockdev-del
 ------------
 Since 2.5
@@ -4139,14 +3199,6 @@ Example:
    }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-open-tray",
-        .args_type  = "device:s,force:b?",
-    },
-
-SQMP
 blockdev-open-tray
 ------------------
 
@@ -4186,14 +3238,6 @@ Example:
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "blockdev-close-tray",
-        .args_type  = "device:s",
-    },
-
-SQMP
 blockdev-close-tray
 -------------------
 
@@ -4220,14 +3264,6 @@ Example:
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "x-blockdev-remove-medium",
-        .args_type  = "device:s",
-    },
-
-SQMP
 x-blockdev-remove-medium
 ------------------------
 
@@ -4267,14 +3303,6 @@ Example:
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "x-blockdev-insert-medium",
-        .args_type  = "device:s,node-name:s",
-    },
-
-SQMP
 x-blockdev-insert-medium
 ------------------------
 
@@ -4306,14 +3334,6 @@ Example:
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "x-blockdev-change",
-        .args_type  = "parent:B,child:B?,node:B?",
-    },
-
-SQMP
 x-blockdev-change
 -----------------
 
@@ -4358,14 +3378,6 @@ Delete a quorum's node
                     "child": "children.1" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "query-named-block-nodes",
-        .args_type  = "",
-    },
-
-SQMP
 query-named-block-nodes
 -----------------------
 
@@ -4419,14 +3431,6 @@ Example:
                       }
                    } } ] }
 
-EQMP
-
-    {
-        .name       = "blockdev-change-medium",
-        .args_type  = "device:B,filename:F,format:s?,read-only-mode:s?",
-    },
-
-SQMP
 blockdev-change-medium
 ----------------------
 
@@ -4471,14 +3475,6 @@ Examples:
 
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "query-memdev",
-        .args_type  = "",
-    },
-
-SQMP
 query-memdev
 ------------
 
@@ -4508,14 +3504,6 @@ Example (1):
      ]
    }
 
-EQMP
-
-    {
-        .name       = "query-memory-devices",
-        .args_type  = "",
-    },
-
-SQMP
 query-memory-devices
 --------------------
 
@@ -4535,14 +3523,6 @@ Example:
                    "type": "dimm"
                  } ] }
 
-EQMP
-
-    {
-        .name       = "query-acpi-ospm-status",
-        .args_type  = "",
-    },
-
-SQMP
 query-acpi-ospm-status
 ----------------------
 
@@ -4557,16 +3537,6 @@ Example:
                  { "slot": "3", "slot-type": "DIMM", "source": 0, "status": 0}
    ]}
 
-EQMP
-
-#if defined TARGET_I386
-    {
-        .name       = "rtc-reset-reinjection",
-        .args_type  = "",
-    },
-#endif
-
-SQMP
 rtc-reset-reinjection
 ---------------------
 
@@ -4579,14 +3549,6 @@ Example:
 -> { "execute": "rtc-reset-reinjection" }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "trace-event-get-state",
-        .args_type  = "name:s,vcpu:i?",
-    },
-
-SQMP
 trace-event-get-state
 ---------------------
 
@@ -4611,14 +3573,6 @@ Example:
 -> { "execute": "trace-event-get-state", "arguments": { "name": "qemu_memalign" } }
 <- { "return": [ { "name": "qemu_memalign", "state": "disabled" } ] }
 
-EQMP
-
-    {
-        .name       = "trace-event-set-state",
-        .args_type  = "name:s,enable:b,ignore-unavailable:b?,vcpu:i?",
-    },
-
-SQMP
 trace-event-set-state
 ---------------------
 
@@ -4646,14 +3600,6 @@ Example:
 -> { "execute": "trace-event-set-state", "arguments": { "name": "qemu_memalign", "enable": "true" } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "input-send-event",
-        .args_type  = "console:i?,events:q",
-    },
-
-SQMP
 input-send-event
 ----------------
 
@@ -4709,14 +3655,6 @@ Move mouse pointer to absolute coordinates (20000, 400).
                { "type": "abs", "data" : { "axis": "y", "value" : 400 } } ] } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "block-set-write-threshold",
-        .args_type  = "node-name:s,write-threshold:l",
-    },
-
-SQMP
 block-set-write-threshold
 ------------
 
@@ -4736,14 +3674,6 @@ Example:
                  "write-threshold": 17179869184 } }
 <- { "return": {} }
 
-EQMP
-
-    {
-        .name       = "query-rocker",
-        .args_type  = "name:s",
-    },
-
-SQMP
 Show rocker switch
 ------------------
 
@@ -4756,14 +3686,6 @@ Example:
 -> { "execute": "query-rocker", "arguments": { "name": "sw1" } }
 <- { "return": {"name": "sw1", "ports": 2, "id": 1327446905938}}
 
-EQMP
-
-    {
-        .name       = "query-rocker-ports",
-        .args_type  = "name:s",
-    },
-
-SQMP
 Show rocker switch ports
 ------------------------
 
@@ -4780,14 +3702,6 @@ Example:
                   "autoneg": "off", "link-up": true, "speed": 10000}
    ]}
 
-EQMP
-
-    {
-        .name       = "query-rocker-of-dpa-flows",
-        .args_type  = "name:s,tbl-id:i?",
-    },
-
-SQMP
 Show rocker switch OF-DPA flow tables
 -------------------------------------
 
@@ -4808,14 +3722,6 @@ Example:
                  {...more...},
    ]}
 
-EQMP
-
-    {
-        .name       = "query-rocker-of-dpa-groups",
-        .args_type  = "name:s,type:i?",
-    },
-
-SQMP
 Show rocker OF-DPA group tables
 -------------------------------
 
@@ -4837,16 +3743,6 @@ Example:
                   "pop-vlan": 1, "id": 251658240}
    ]}
 
-EQMP
-
-#if defined TARGET_ARM
-    {
-        .name       = "query-gic-capabilities",
-        .args_type  = "",
-    },
-#endif
-
-SQMP
 query-gic-capabilities
 ---------------
 
@@ -4861,14 +3757,6 @@ Example:
 <- { "return": [{ "version": 2, "emulated": true, "kernel": false },
                 { "version": 3, "emulated": false, "kernel": true } ] }
 
-EQMP
-
-    {
-        .name       = "query-hotpluggable-cpus",
-        .args_type  = "",
-    },
-
-SQMP
 Show existing/possible CPUs
 ---------------------------
 
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index c425393..cfa6fe7 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -119,16 +119,6 @@ There are a few things to be noticed:
 5. Printing to the terminal is discouraged for QMP commands, we do it here
    because it's the easiest way to demonstrate a QMP command
 
-Now a little hack is needed. As we're still using the old QMP server we need
-to add the new command to its internal dispatch table. This step won't be
-required in the near future. Open the qmp-commands.hx file and add the
-following at the bottom:
-
-    {
-        .name       = "hello-world",
-        .args_type  = "",
-    },
-
 You're done. Now build qemu, run it as suggested in the "Testing" section,
 and then type the following QMP command:
 
@@ -173,20 +163,6 @@ There are two important details to be noticed:
 2. The C implementation signature must follow the schema's argument ordering,
    which is defined by the "data" member
 
-The last step is to update the qmp-commands.hx file:
-
-    {
-        .name       = "hello-world",
-        .args_type  = "message:s?",
-    },
-
-Notice that the "args_type" member got our "message" argument. The character
-"s" stands for "string" and "?" means it's optional. This too must be ordered
-according to the C implementation and schema file. You can look for more
-examples in the qmp-commands.hx file if you need to define more arguments.
-
-Again, this step won't be required in the future.
-
 Time to test our new version of the "hello-world" command. Build qemu, run it as
 described in the "Testing" section and then send two commands:
 
@@ -452,13 +428,6 @@ There are a number of things to be noticed:
 6. You have to include the "qmp-commands.h" header file in qemu-timer.c,
    otherwise qemu won't build
 
-The last step is to add the corresponding entry in the qmp-commands.hx file:
-
-    {
-        .name       = "query-alarm-clock",
-        .args_type  = "",
-    },
-
 Time to test the new command. Build qemu, run it as described in the "Testing"
 section and try this:
 
@@ -597,13 +566,6 @@ iteration of the loop. That's because the alarm timer method in use is the
 first element of the alarm_timers array. Also notice that QAPI lists are handled
 by hand and we return the head of the list.
 
-To test this you have to add the corresponding qmp-commands.hx entry:
-
-    {
-        .name       = "query-alarm-methods",
-        .args_type  = "",
-    },
-
 Now Build qemu, run it as explained in the "Testing" section and try our new
 command:
 
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch()
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch() Marc-André Lureau
@ 2016-08-17 17:04   ` Marc-André Lureau
  0 siblings, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-08-17 17:04 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel, eblake, armbru

Hi

----- Original Message -----
> Replace the old manual dispatch and validation code by the generic one
> provided by qapi common code.
> 
> Note that it is now possible to call the following commands that used to
> be disabled by compile-time conditionals:
> - dump-skeys
> - query-spice
> - rtc-reset-reinjection
> - query-gic-capabilities
> 
> Their fallback functions return an appropriate "feature disabled" error.

This is no longer valid, I dropped that comment.

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  monitor.c    | 326
>  +++++++----------------------------------------------------
>  trace-events |   1 -
>  2 files changed, 34 insertions(+), 293 deletions(-)
> 
> diff --git a/monitor.c b/monitor.c
> index f07fc31..81926c7 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -166,7 +166,6 @@ struct MonFdset {
>  };
>  
>  typedef struct {
> -    QObject *id;
>      JSONMessageParser parser;
>      /*
>       * When a client connects, we're in capabilities negotiation mode.
> @@ -229,8 +228,6 @@ static int mon_refcount;
>  static mon_cmd_t mon_cmds[];
>  static mon_cmd_t info_cmds[];
>  
> -static const mon_cmd_t qmp_cmds[];
> -
>  Monitor *cur_mon;
>  
>  static QEMUClockType event_clock_type = QEMU_CLOCK_REALTIME;
> @@ -401,49 +398,6 @@ static void monitor_json_emitter(Monitor *mon, const
> QObject *data)
>      QDECREF(json);
>  }
>  
> -static QDict *build_qmp_error_dict(Error *err)
> -{
> -    QObject *obj;
> -
> -    obj = qobject_from_jsonf("{ 'error': { 'class': %s, 'desc': %s } }",
> -                             QapiErrorClass_lookup[error_get_class(err)],
> -                             error_get_pretty(err));
> -
> -    return qobject_to_qdict(obj);
> -}
> -
> -static void monitor_protocol_emitter(Monitor *mon, QObject *data,
> -                                     Error *err)
> -{
> -    QDict *qmp;
> -
> -    trace_monitor_protocol_emitter(mon);
> -
> -    if (!err) {
> -        /* success response */
> -        qmp = qdict_new();
> -        if (data) {
> -            qobject_incref(data);
> -            qdict_put_obj(qmp, "return", data);
> -        } else {
> -            /* return an empty QDict by default */
> -            qdict_put(qmp, "return", qdict_new());
> -        }
> -    } else {
> -        /* error response */
> -        qmp = build_qmp_error_dict(err);
> -    }
> -
> -    if (mon->qmp.id) {
> -        qdict_put_obj(qmp, "id", mon->qmp.id);
> -        mon->qmp.id = NULL;
> -    }
> -
> -    monitor_json_emitter(mon, QOBJECT(qmp));
> -    QDECREF(qmp);
> -}
> -
> -
>  static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX] = {
>      /* Limit guest-triggerable events to 1 per second */
>      [QAPI_EVENT_RTC_CHANGE]        = { 1000 * SCALE_MS },
> @@ -2180,11 +2134,6 @@ static mon_cmd_t mon_cmds[] = {
>      { NULL, NULL, },
>  };
>  
> -static const mon_cmd_t qmp_cmds[] = {
> -#include "qmp-commands-old.h"
> -    { /* NULL */ },
> -};
> -
>  /*******************************************************************/
>  
>  static const char *pch;
> @@ -2535,11 +2484,6 @@ static const mon_cmd_t *search_dispatch_table(const
> mon_cmd_t *disp_table,
>      return NULL;
>  }
>  
> -static const mon_cmd_t *qmp_find_cmd(const char *cmdname)
> -{
> -    return search_dispatch_table(qmp_cmds, cmdname);
> -}
> -
>  /*
>   * Parse command name from @cmdp according to command table @table.
>   * If blank, return NULL.
> @@ -3690,199 +3634,6 @@ static bool invalid_qmp_mode(const Monitor *mon,
> const char *cmd,
>      return false;
>  }
>  
> -/*
> - * Argument validation rules:
> - *
> - * 1. The argument must exist in cmd_args qdict
> - * 2. The argument type must be the expected one
> - *
> - * Special case: If the argument doesn't exist in cmd_args and
> - *               the QMP_ACCEPT_UNKNOWNS flag is set, then the
> - *               checking is skipped for it.
> - */
> -static void check_client_args_type(const QDict *client_args,
> -                                   const QDict *cmd_args, int flags,
> -                                   Error **errp)
> -{
> -    const QDictEntry *ent;
> -
> -    for (ent = qdict_first(client_args); ent;ent =
> qdict_next(client_args,ent)){
> -        QObject *obj;
> -        QString *arg_type;
> -        const QObject *client_arg = qdict_entry_value(ent);
> -        const char *client_arg_name = qdict_entry_key(ent);
> -
> -        obj = qdict_get(cmd_args, client_arg_name);
> -        if (!obj) {
> -            if (flags & QMP_ACCEPT_UNKNOWNS) {
> -                /* handler accepts unknowns */
> -                continue;
> -            }
> -            /* client arg doesn't exist */
> -            error_setg(errp, QERR_INVALID_PARAMETER, client_arg_name);
> -            return;
> -        }
> -
> -        arg_type = qobject_to_qstring(obj);
> -        assert(arg_type != NULL);
> -
> -        /* check if argument's type is correct */
> -        switch (qstring_get_str(arg_type)[0]) {
> -        case 'F':
> -        case 'B':
> -        case 's':
> -            if (qobject_type(client_arg) != QTYPE_QSTRING) {
> -                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
> -                           client_arg_name, "string");
> -                return;
> -            }
> -        break;
> -        case 'i':
> -        case 'l':
> -        case 'M':
> -        case 'o':
> -            if (qobject_type(client_arg) != QTYPE_QINT) {
> -                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
> -                           client_arg_name, "int");
> -                return;
> -            }
> -            break;
> -        case 'T':
> -            if (qobject_type(client_arg) != QTYPE_QINT &&
> -                qobject_type(client_arg) != QTYPE_QFLOAT) {
> -                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
> -                           client_arg_name, "number");
> -                return;
> -            }
> -            break;
> -        case 'b':
> -        case '-':
> -            if (qobject_type(client_arg) != QTYPE_QBOOL) {
> -                error_setg(errp, QERR_INVALID_PARAMETER_TYPE,
> -                           client_arg_name, "bool");
> -                return;
> -            }
> -            break;
> -        case 'O':
> -            assert(flags & QMP_ACCEPT_UNKNOWNS);
> -            break;
> -        case 'q':
> -            /* Any QObject can be passed.  */
> -            break;
> -        case '/':
> -        case '.':
> -            /*
> -             * These types are not supported by QMP and thus are not
> -             * handled here. Fall through.
> -             */
> -        default:
> -            abort();
> -        }
> -    }
> -}
> -
> -/*
> - * - Check if the client has passed all mandatory args
> - * - Set special flags for argument validation
> - */
> -static void check_mandatory_args(const QDict *cmd_args,
> -                                 const QDict *client_args, int *flags,
> -                                 Error **errp)
> -{
> -    const QDictEntry *ent;
> -
> -    for (ent = qdict_first(cmd_args); ent; ent = qdict_next(cmd_args, ent))
> {
> -        const char *cmd_arg_name = qdict_entry_key(ent);
> -        QString *type = qobject_to_qstring(qdict_entry_value(ent));
> -        assert(type != NULL);
> -
> -        if (qstring_get_str(type)[0] == 'O') {
> -            assert((*flags & QMP_ACCEPT_UNKNOWNS) == 0);
> -            *flags |= QMP_ACCEPT_UNKNOWNS;
> -        } else if (qstring_get_str(type)[0] != '-' &&
> -                   qstring_get_str(type)[1] != '?' &&
> -                   !qdict_haskey(client_args, cmd_arg_name)) {
> -            error_setg(errp, QERR_MISSING_PARAMETER, cmd_arg_name);
> -            return;
> -        }
> -    }
> -}
> -
> -static QDict *qdict_from_args_type(const char *args_type)
> -{
> -    int i;
> -    QDict *qdict;
> -    QString *key, *type, *cur_qs;
> -
> -    assert(args_type != NULL);
> -
> -    qdict = qdict_new();
> -
> -    if (args_type == NULL || args_type[0] == '\0') {
> -        /* no args, empty qdict */
> -        goto out;
> -    }
> -
> -    key = qstring_new();
> -    type = qstring_new();
> -
> -    cur_qs = key;
> -
> -    for (i = 0;; i++) {
> -        switch (args_type[i]) {
> -            case ',':
> -            case '\0':
> -                qdict_put(qdict, qstring_get_str(key), type);
> -                QDECREF(key);
> -                if (args_type[i] == '\0') {
> -                    goto out;
> -                }
> -                type = qstring_new(); /* qdict has ref */
> -                cur_qs = key = qstring_new();
> -                break;
> -            case ':':
> -                cur_qs = type;
> -                break;
> -            default:
> -                qstring_append_chr(cur_qs, args_type[i]);
> -                break;
> -        }
> -    }
> -
> -out:
> -    return qdict;
> -}
> -
> -/*
> - * Client argument checking rules:
> - *
> - * 1. Client must provide all mandatory arguments
> - * 2. Each argument provided by the client must be expected
> - * 3. Each argument provided by the client must have the type expected
> - *    by the command
> - */
> -static void qmp_check_client_args(const mon_cmd_t *cmd, QDict *client_args,
> -                                  Error **errp)
> -{
> -    Error *err = NULL;
> -    int flags;
> -    QDict *cmd_args;
> -
> -    cmd_args = qdict_from_args_type(cmd->args_type);
> -
> -    flags = 0;
> -    check_mandatory_args(cmd_args, client_args, &flags, &err);
> -    if (err) {
> -        goto out;
> -    }
> -
> -    check_client_args_type(client_args, cmd_args, flags, &err);
> -
> -out:
> -    error_propagate(errp, err);
> -    QDECREF(cmd_args);
> -}
> -
>  /*
>   * Input object checking rules
>   *
> @@ -3941,67 +3692,58 @@ static QDict *qmp_check_input_obj(QObject *input_obj,
> Error **errp)
>  
>  static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens)
>  {
> -    Error *local_err = NULL;
> -    QObject *obj, *data;
> -    QDict *input, *args;
> -    const mon_cmd_t *cmd;
> -    QmpCommand *qcmd;
> +    QObject *req, *rsp = NULL, *id = NULL;
> +    QDict *qdict = NULL;
>      const char *cmd_name;
>      Monitor *mon = cur_mon;
> +    Error *err = NULL;
>  
> -    args = input = NULL;
> -    data = NULL;
> -
> -    obj = json_parser_parse(tokens, NULL);
> -    if (!obj) {
> -        // FIXME: should be triggered in json_parser_parse()
> -        error_setg(&local_err, QERR_JSON_PARSING);
> +    req = json_parser_parse_err(tokens, NULL, &err);
> +    if (err || !req || qobject_type(req) != QTYPE_QDICT) {
> +        if (!err) {
> +            error_setg(&err, QERR_JSON_PARSING);
> +        }
>          goto err_out;
>      }
>  
> -    input = qmp_check_input_obj(obj, &local_err);
> -    if (!input) {
> -        qobject_decref(obj);
> +    qdict = qmp_check_input_obj(req, &err);
> +    if (!qdict) {
>          goto err_out;
>      }
>  
> -    mon->qmp.id = qdict_get(input, "id");
> -    qobject_incref(mon->qmp.id);
> +    id = qdict_get(qdict, "id");
> +    qobject_incref(id);
> +    qdict_del(qdict, "id");
>  
> -    cmd_name = qdict_get_str(input, "execute");
> +    cmd_name = qdict_get_str(qdict, "execute");
>      trace_handle_qmp_command(mon, cmd_name);
> -    cmd = qmp_find_cmd(cmd_name);
> -    qcmd = qmp_find_command(cmd_name);
> -    if (!qcmd || !cmd) {
> -        error_set(&local_err, ERROR_CLASS_COMMAND_NOT_FOUND,
> -                  "The command %s has not been found", cmd_name);
> -        goto err_out;
> -    }
> -    if (invalid_qmp_mode(mon, cmd_name, &local_err)) {
> +
> +    if (invalid_qmp_mode(mon, cmd_name, &err)) {
>          goto err_out;
>      }
>  
> -    obj = qdict_get(input, "arguments");
> -    if (!obj) {
> -        args = qdict_new();
> -    } else {
> -        args = qobject_to_qdict(obj);
> -        QINCREF(args);
> -    }
> +    rsp = qmp_dispatch(req);
>  
> -    qmp_check_client_args(cmd, args, &local_err);
> -    if (local_err) {
> -        goto err_out;
> +err_out:
> +    if (err) {
> +        qdict = qdict_new();
> +        qdict_put_obj(qdict, "error", qmp_build_error_object(err));
> +        error_free(err);
> +        rsp = QOBJECT(qdict);
>      }
>  
> -    qcmd->fn(args, &data, &local_err);
> +    if (rsp) {
> +        if (id) {
> +            qdict_put_obj(qobject_to_qdict(rsp), "id", id);
> +            id = NULL;
> +        }
>  
> -err_out:
> -    monitor_protocol_emitter(mon, data, local_err);
> -    qobject_decref(data);
> -    error_free(local_err);
> -    QDECREF(input);
> -    QDECREF(args);
> +        monitor_json_emitter(mon, rsp);
> +    }
> +
> +    qobject_decref(id);
> +    qobject_decref(rsp);
> +    qobject_decref(req);
>  }
>  
>  static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
> diff --git a/trace-events b/trace-events
> index 616cc52..8d59631 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -98,7 +98,6 @@ qemu_co_mutex_unlock_return(void *mutex, void *self) "mutex
> %p self %p"
>  
>  # monitor.c
>  handle_qmp_command(void *mon, const char *cmd_name) "mon %p cmd_name \"%s\""
> -monitor_protocol_emitter(void *mon) "mon %p"
>  monitor_protocol_event_handler(uint32_t event, void *qdict) "event=%d
>  data=%p"
>  monitor_protocol_event_emit(uint32_t event, void *data) "event=%d data=%p"
>  monitor_protocol_event_queue(uint32_t event, void *qdict, uint64_t rate)
>  "event=%d data=%p rate=%" PRId64
> --
> 2.9.0
> 
> 

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional Marc-André Lureau
@ 2016-09-06 12:35   ` Markus Armbruster
  2016-09-06 13:17     ` Marc-André Lureau
  0 siblings, 1 reply; 31+ messages in thread
From: Markus Armbruster @ 2016-09-06 12:35 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Learn to parse #define files provided with -f option, and skip
> undefined #ifdef blocks in the schema.
>
> This is a very simple pre-processing, without stacking support or
> evaluation (it could be implemented if needed).
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---

Missing: update of docs/qapi-code-gen.txt.

>  1 file changed, 40 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 21bc32f..d0b8a66 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -76,6 +76,7 @@ struct_types = []
>  union_types = []
>  events = []
>  all_names = {}
> +defs = []
>  
>  #
>  # Parsing the schema into expressions
> @@ -177,6 +178,7 @@ class QAPISchemaParser(object):
>                  self.exprs.append(expr_elem)
>  
>      def accept(self):
> +        ok = True
>          while True:
>              self.tok = self.src[self.cursor]
>              self.pos = self.cursor
> @@ -184,7 +186,19 @@ class QAPISchemaParser(object):
>              self.val = None
>  
>              if self.tok == '#':
> -                self.cursor = self.src.find('\n', self.cursor)
> +                end = self.src.find('\n', self.cursor)
> +                line = self.src[self.cursor:end+1]
> +                self.cursor = end
> +                sline = line.split()
> +                if len(defs) and len(sline) >= 1 \
> +                   and sline[0] in ['ifdef', 'endif']:
> +                    if sline[0] == 'ifdef':
> +                        ok = sline[1] in defs
> +                    elif sline[0] == 'endif':
> +                        ok = True
> +                    continue
> +            elif not ok:
> +                continue
>              elif self.tok in "{}:,[]":
>                  return
>              elif self.tok == "'":

Oww, you're abusing comments!  That's horrible :)

Can we make this real syntax, like everything else, including 'include'?

Unfortunately, the natural

    { 'ifdef': 'CONFIG_FOO'
      'then': ...   # ignored unless CONFIG_FOO
      'else': ...   # ignored if CONFIG_FOO (optional)
    }

is awkward, because the ... have to be a single JSON value, but a QAPI
schema isn't a single JSON value, it's a *sequence* of JSON values.  A
top-level stanza

    JSON-value1 JSON-value2 ...

would become

    [ JSON-value1, JSON-value2, ... ]

within a conditional.  Blech.

Could sacrifice the nesting and do

    { 'ifdef': 'CONFIG_FOO' }
    ...
    { 'endif' }

Widens the gap between syntax and semantics.  Editors can no longer
easily jump over the conditional (e.g. Emacs forward-sexp).  Nested
conditionals becomes as confusing as in C.  Not exactly elegant, either.

Yet another option is to add 'ifdef' keys to the definitions
themselves, i.e.

    { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
      'ifdef': 'TARGET_ARM' }

> @@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
>  #
>  # Common command line parsing
>  #
> +def defile(filename):

>From The Collaborative International Dictionary of English v.0.48 [gcide]:

Defile \De*file"\ (d[-e]*f[imac]l"), v. t. [OE. defoulen,
   -foilen, to tread down, OF. defouler; de- + fouler to trample
   (see Full, v. t.), and OE. defoulen to foul (influenced in
   form by the older verb defoilen). See File to defile,
   Foul, Defoul.]
   1. To make foul or impure; to make filthy; to dirty; to
      befoul; to pollute.
      [1913 Webster]

            They that touch pitch will be defiled. --Shak.
      [1913 Webster]

   2. To soil or sully; to tarnish, as reputation; to taint.
      [1913 Webster]

            He is . . . among the greatest prelates of this age,
            however his character may be defiled by . . . dirty
            hands.                                --Swift.
      [1913 Webster]

   3. To injure in purity of character; to corrupt.
      [1913 Webster]

            Defile not yourselves with the idols of Egypt.
                                                  --Ezek. xx. 7.
      [1913 Webster]

   4. To corrupt the chastity of; to debauch; to violate; to
      rape.
      [1913 Webster]

            The husband murder'd and the wife defiled. --Prior.
      [1913 Webster]

   5. To make ceremonially unclean; to pollute.
      [1913 Webster]

            That which dieth of itself, or is torn with beasts,
            he shall not eat to defile therewith. --Lev. xxii.
                                                  8.
      [1913 Webster]

Fitting in a way; you're defiling the poor, innocent comment syntax ;)

> +    f = open(filename, 'r')
> +    while 1:

while True:

> +        line = f.readline()
> +        if not line:
> +            break
> +        while line[-2:] == '\\\n':
> +            nextline = f.readline()
> +            if not nextline:
> +                break
> +            line = line + nextline
> +        tmp = line.strip()
> +        if tmp[:1] != '#':
> +            continue
> +        tmp = tmp[1:]
> +        words = tmp.split()
> +        if words[0] != "define":
> +            continue
> +        defs.append(words[1])
> +    f.close()

This parses Yet Another Language.  Worse, Yet Another Undocumented
Language.  Why not JSON?

Hmm, peeking ahead to PATCH 04... aha!  This is for reading
config-host.h and config-target.h.  So, this actually doesn't parse
YAUL, it parses C.  Sloppily, of course.

I guess we could instead parse config-host.mak and config-target.mak
sloppily.  Not sure which idea is more disgusting :)

Could we punt evaluating conditionals to the C compiler?  Instead of
emitting TEXT when CONFIG_FOO is defined, emit

    #ifdef CONFIG_FOO
    TEXT
    #endif

>  
>  
>  def parse_command_line(extra_options="", extra_long_options=[]):
>  
>      try:
>          opts, args = getopt.gnu_getopt(sys.argv[1:],
> -                                       "chp:o:" + extra_options,
> +                                       "chp:o:f:" + extra_options,
>                                         ["source", "header", "prefix=",
> -                                        "output-dir="] + extra_long_options)
> +                                        "output-dir=", "--defile="] +

https://docs.python.org/3/library/getopt.html on the third argument:
"The leading '--' characters should not be included in the option name."

> +                                       extra_long_options)
>      except getopt.GetoptError as err:
>          print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
>          sys.exit(1)
> @@ -1742,6 +1777,8 @@ def parse_command_line(extra_options="", extra_long_options=[]):
>              do_c = True
>          elif o in ("-h", "--header"):
>              do_h = True
> +        elif o in ("-f", "--defile"):
> +            defile(a)
>          else:
>              extra_opts.append(oa)

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-06 12:35   ` Markus Armbruster
@ 2016-09-06 13:17     ` Marc-André Lureau
  2016-09-06 15:58       ` Markus Armbruster
  0 siblings, 1 reply; 31+ messages in thread
From: Marc-André Lureau @ 2016-09-06 13:17 UTC (permalink / raw)
  To: Markus Armbruster; +Cc: qemu-devel

Hi

On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com> wrote:

> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Learn to parse #define files provided with -f option, and skip
> > undefined #ifdef blocks in the schema.
> >
> > This is a very simple pre-processing, without stacking support or
> > evaluation (it could be implemented if needed).
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---
>
> Missing: update of docs/qapi-code-gen.txt.
>
> >  1 file changed, 40 insertions(+), 3 deletions(-)
> >
> > diff --git a/scripts/qapi.py b/scripts/qapi.py
> > index 21bc32f..d0b8a66 100644
> > --- a/scripts/qapi.py
> > +++ b/scripts/qapi.py
> > @@ -76,6 +76,7 @@ struct_types = []
> >  union_types = []
> >  events = []
> >  all_names = {}
> > +defs = []
> >
> >  #
> >  # Parsing the schema into expressions
> > @@ -177,6 +178,7 @@ class QAPISchemaParser(object):
> >                  self.exprs.append(expr_elem)
> >
> >      def accept(self):
> > +        ok = True
> >          while True:
> >              self.tok = self.src[self.cursor]
> >              self.pos = self.cursor
> > @@ -184,7 +186,19 @@ class QAPISchemaParser(object):
> >              self.val = None
> >
> >              if self.tok == '#':
> > -                self.cursor = self.src.find('\n', self.cursor)
> > +                end = self.src.find('\n', self.cursor)
> > +                line = self.src[self.cursor:end+1]
> > +                self.cursor = end
> > +                sline = line.split()
> > +                if len(defs) and len(sline) >= 1 \
> > +                   and sline[0] in ['ifdef', 'endif']:
> > +                    if sline[0] == 'ifdef':
> > +                        ok = sline[1] in defs
> > +                    elif sline[0] == 'endif':
> > +                        ok = True
> > +                    continue
> > +            elif not ok:
> > +                continue
> >              elif self.tok in "{}:,[]":
> >                  return
> >              elif self.tok == "'":
>
> Oww, you're abusing comments!  That's horrible :)
>

Can we make this real syntax, like everything else, including 'include'?
>
>
We already abuse json, which doesn't support comments either. :) Even
without comments, our syntax is wrong and confuse most editors/validators.


> Unfortunately, the natural
>
>     { 'ifdef': 'CONFIG_FOO'
>       'then': ...   # ignored unless CONFIG_FOO
>       'else': ...   # ignored if CONFIG_FOO (optional)
>     }
>
> is awkward, because the ... have to be a single JSON value, but a QAPI
> schema isn't a single JSON value, it's a *sequence* of JSON values.  A
> top-level stanza
>
>     JSON-value1 JSON-value2 ...
>
> would become
>
>     [ JSON-value1, JSON-value2, ... ]
>
> within a conditional.  Blech.
>
> Could sacrifice the nesting and do
>
>     { 'ifdef': 'CONFIG_FOO' }
>     ...
>     { 'endif' }
>
> Widens the gap between syntax and semantics.  Editors can no longer
> easily jump over the conditional (e.g. Emacs forward-sexp).  Nested
> conditionals becomes as confusing as in C.  Not exactly elegant, either.
>
> Yet another option is to add 'ifdef' keys to the definitions
> themselves, i.e.
>
>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
>       'ifdef': 'TARGET_ARM' }
>

That's the worst of all options imho, as it makes it hard to filter out
unions/enums, ex:

 @ -3446,8 +3466,10 @@
                                        'testdev': 'ChardevCommon',
                                        'stdio'  : 'ChardevStdio',
                                        'console': 'ChardevCommon',
+#ifdef CONFIG_SPICE
                                        'spicevmc' : 'ChardevSpiceChannel',
                                        'spiceport' : 'ChardevSpicePort',
+#endif
                                        'vc'     : 'ChardevVC',
                                        'ringbuf': 'ChardevRingbuf',

I think #ifdef is the most straightforward/easy thing to do. My experience
with doc generations tells me that is really not an issue to reserve the
#\w* lines for pre-processing.

And it's a natural fit with the rest of qemu #if conditionals.




> > @@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
> >  #
> >  # Common command line parsing
> >  #
> > +def defile(filename):
>
> From The Collaborative International Dictionary of English v.0.48 [gcide]:
>
> Defile \De*file"\ (d[-e]*f[imac]l"), v. t. [OE. defoulen,
>    -foilen, to tread down, OF. defouler; de- + fouler to trample
>    (see Full, v. t.), and OE. defoulen to foul (influenced in
>    form by the older verb defoilen). See File to defile,
>    Foul, Defoul.]
>    1. To make foul or impure; to make filthy; to dirty; to
>       befoul; to pollute.
>       [1913 Webster]
>
>             They that touch pitch will be defiled. --Shak.
>       [1913 Webster]
>
>    2. To soil or sully; to tarnish, as reputation; to taint.
>       [1913 Webster]
>
>             He is . . . among the greatest prelates of this age,
>             however his character may be defiled by . . . dirty
>             hands.                                --Swift.
>       [1913 Webster]
>
>    3. To injure in purity of character; to corrupt.
>       [1913 Webster]
>
>             Defile not yourselves with the idols of Egypt.
>                                                   --Ezek. xx. 7.
>       [1913 Webster]
>
>    4. To corrupt the chastity of; to debauch; to violate; to
>       rape.
>       [1913 Webster]
>
>             The husband murder'd and the wife defiled. --Prior.
>       [1913 Webster]
>
>    5. To make ceremonially unclean; to pollute.
>       [1913 Webster]
>
>             That which dieth of itself, or is torn with beasts,
>             he shall not eat to defile therewith. --Lev. xxii.
>                                                   8.
>       [1913 Webster]
>
> Fitting in a way; you're defiling the poor, innocent comment syntax ;)
>

ok


>
> > +    f = open(filename, 'r')
> > +    while 1:
>
> while True:
>
> > +        line = f.readline()
> > +        if not line:
> > +            break
> > +        while line[-2:] == '\\\n':
> > +            nextline = f.readline()
> > +            if not nextline:
> > +                break
> > +            line = line + nextline
> > +        tmp = line.strip()
> > +        if tmp[:1] != '#':
> > +            continue
> > +        tmp = tmp[1:]
> > +        words = tmp.split()
> > +        if words[0] != "define":
> > +            continue
> > +        defs.append(words[1])
> > +    f.close()
>
> This parses Yet Another Language.  Worse, Yet Another Undocumented
> Language.  Why not JSON?
>

> Hmm, peeking ahead to PATCH 04... aha!  This is for reading
> config-host.h and config-target.h.  So, this actually doesn't parse
> YAUL, it parses C.  Sloppily, of course.
>

Yes


>
> I guess we could instead parse config-host.mak and config-target.mak
> sloppily.  Not sure which idea is more disgusting :)
>
> Could we punt evaluating conditionals to the C compiler?  Instead of
> emitting TEXT when CONFIG_FOO is defined, emit
>
>     #ifdef CONFIG_FOO
>     TEXT
>     #endif
>
>
I don't follow you


> >
> >
> >  def parse_command_line(extra_options="", extra_long_options=[]):
> >
> >      try:
> >          opts, args = getopt.gnu_getopt(sys.argv[1:],
> > -                                       "chp:o:" + extra_options,
> > +                                       "chp:o:f:" + extra_options,
> >                                         ["source", "header", "prefix=",
> > -                                        "output-dir="] +
> extra_long_options)
> > +                                        "output-dir=", "--defile="] +
>
> https://docs.python.org/3/library/getopt.html on the third argument:
> "The leading '--' characters should not be included in the option name."
>

ok


>
> > +                                       extra_long_options)
> >      except getopt.GetoptError as err:
> >          print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
> >          sys.exit(1)
> > @@ -1742,6 +1777,8 @@ def parse_command_line(extra_options="",
> extra_long_options=[]):
> >              do_c = True
> >          elif o in ("-h", "--header"):
> >              do_h = True
> > +        elif o in ("-f", "--defile"):
> > +            defile(a)
> >          else:
> >              extra_opts.append(oa)
>
> --
Marc-André Lureau

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-06 13:17     ` Marc-André Lureau
@ 2016-09-06 15:58       ` Markus Armbruster
  2016-09-06 16:44         ` Marc-André Lureau
  0 siblings, 1 reply; 31+ messages in thread
From: Markus Armbruster @ 2016-09-06 15:58 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel, Eric Blake

QAPI language design issues, copying Eric.

Marc-André Lureau <marcandre.lureau@gmail.com> writes:

> Hi
>
> On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com> wrote:
>
>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>
>> > Learn to parse #define files provided with -f option, and skip
>> > undefined #ifdef blocks in the schema.
>> >
>> > This is a very simple pre-processing, without stacking support or
>> > evaluation (it could be implemented if needed).
>> >
>> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> > ---
>> >  scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---
>>
>> Missing: update of docs/qapi-code-gen.txt.
>>
>> >  1 file changed, 40 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/scripts/qapi.py b/scripts/qapi.py
>> > index 21bc32f..d0b8a66 100644
>> > --- a/scripts/qapi.py
>> > +++ b/scripts/qapi.py
>> > @@ -76,6 +76,7 @@ struct_types = []
>> >  union_types = []
>> >  events = []
>> >  all_names = {}
>> > +defs = []
>> >
>> >  #
>> >  # Parsing the schema into expressions
>> > @@ -177,6 +178,7 @@ class QAPISchemaParser(object):
>> >                  self.exprs.append(expr_elem)
>> >
>> >      def accept(self):
>> > +        ok = True
>> >          while True:
>> >              self.tok = self.src[self.cursor]
>> >              self.pos = self.cursor
>> > @@ -184,7 +186,19 @@ class QAPISchemaParser(object):
>> >              self.val = None
>> >
>> >              if self.tok == '#':
>> > -                self.cursor = self.src.find('\n', self.cursor)
>> > +                end = self.src.find('\n', self.cursor)
>> > +                line = self.src[self.cursor:end+1]
>> > +                self.cursor = end
>> > +                sline = line.split()
>> > +                if len(defs) and len(sline) >= 1 \
>> > +                   and sline[0] in ['ifdef', 'endif']:
>> > +                    if sline[0] == 'ifdef':
>> > +                        ok = sline[1] in defs
>> > +                    elif sline[0] == 'endif':
>> > +                        ok = True
>> > +                    continue
>> > +            elif not ok:
>> > +                continue
>> >              elif self.tok in "{}:,[]":
>> >                  return
>> >              elif self.tok == "'":
>>
>> Oww, you're abusing comments!  That's horrible :)
>>
>> Can we make this real syntax, like everything else, including 'include'?
>>
>>
> We already abuse json, which doesn't support comments either. :) Even
> without comments, our syntax is wrong and confuse most editors/validators.

True.  Python mode works okay for me, though.

Perhaps we've outgrown the JSON syntax.  Or perhaps we've just messed up
by taking too many liberties with it, with too little thought.  Not
exactly an excuse for taking *more* liberties :)

>> Unfortunately, the natural
>>
>>     { 'ifdef': 'CONFIG_FOO'
>>       'then': ...   # ignored unless CONFIG_FOO
>>       'else': ...   # ignored if CONFIG_FOO (optional)
>>     }
>>
>> is awkward, because the ... have to be a single JSON value, but a QAPI
>> schema isn't a single JSON value, it's a *sequence* of JSON values.  A
>> top-level stanza
>>
>>     JSON-value1 JSON-value2 ...
>>
>> would become
>>
>>     [ JSON-value1, JSON-value2, ... ]
>>
>> within a conditional.  Blech.
>>
>> Could sacrifice the nesting and do
>>
>>     { 'ifdef': 'CONFIG_FOO' }
>>     ...
>>     { 'endif' }
>>
>> Widens the gap between syntax and semantics.  Editors can no longer
>> easily jump over the conditional (e.g. Emacs forward-sexp).  Nested
>> conditionals becomes as confusing as in C.  Not exactly elegant, either.
>>
>> Yet another option is to add 'ifdef' keys to the definitions
>> themselves, i.e.
>>
>>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
>>       'ifdef': 'TARGET_ARM' }
>>
>
> That's the worst of all options imho, as it makes it hard to filter out
> unions/enums, ex:
>
>  @ -3446,8 +3466,10 @@
>                                         'testdev': 'ChardevCommon',
>                                         'stdio'  : 'ChardevStdio',
>                                         'console': 'ChardevCommon',
> +#ifdef CONFIG_SPICE
>                                         'spicevmc' : 'ChardevSpiceChannel',
>                                         'spiceport' : 'ChardevSpicePort',
> +#endif
>                                         'vc'     : 'ChardevVC',
>                                         'ringbuf': 'ChardevRingbuf',

Point taken.

Fixable the same way as always when a definition needs to grow
additional properties, but the syntax only provides a single value: make
that single value an object, and the old non-object value syntactic
sugar for the equivalent object value.  We've previously discussed this
technique in the context of giving command arguments default values.
I'm not saying this is what we should do here, only pointing out it's
possible.

> I think #ifdef is the most straightforward/easy thing to do. My experience
> with doc generations tells me that is really not an issue to reserve the
> #\w* lines for pre-processing.

Two justifications for doc generators recognizing magic comments: 1. If
you create a machine processing comments (e.g. formatting them nicely
for a medium richer than ASCII text, you fundamentally define a comment
language, embedded in a host language's comments, and 2. what else can
you do when you can't change the host language?

Neither applies to adding conditional compilation directives to the QAPI
schema language.  Letting a language grow into its comments is, to be
frank, disgusting.

Besides, if we truly want the C preprocessor, we should use the C
preprocessor.  Not implement a half-assed clone of the half-assed hack
of a macro processor the C preprocessor is.

> And it's a natural fit with the rest of qemu #if conditionals.

It's an awfully unnatural fit with the QAPI schema comment syntax.

##
# @Frobs
#
# Collection of frobs that need to be frobnicated, except when
# ifdef CONFIG_NOFROB
{ 'struct': 'Frobs'
  ...
}

I stand by 'horrible'.

>> > @@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
>> >  #
>> >  # Common command line parsing
>> >  #
>> > +def defile(filename):
>>
>> From The Collaborative International Dictionary of English v.0.48 [gcide]:
>>
>> Defile \De*file"\ (d[-e]*f[imac]l"), v. t. [OE. defoulen,
>>    -foilen, to tread down, OF. defouler; de- + fouler to trample
>>    (see Full, v. t.), and OE. defoulen to foul (influenced in
>>    form by the older verb defoilen). See File to defile,
>>    Foul, Defoul.]
>>    1. To make foul or impure; to make filthy; to dirty; to
>>       befoul; to pollute.
>>       [1913 Webster]
>>
>>             They that touch pitch will be defiled. --Shak.
>>       [1913 Webster]
>>
>>    2. To soil or sully; to tarnish, as reputation; to taint.
>>       [1913 Webster]
>>
>>             He is . . . among the greatest prelates of this age,
>>             however his character may be defiled by . . . dirty
>>             hands.                                --Swift.
>>       [1913 Webster]
>>
>>    3. To injure in purity of character; to corrupt.
>>       [1913 Webster]
>>
>>             Defile not yourselves with the idols of Egypt.
>>                                                   --Ezek. xx. 7.
>>       [1913 Webster]
>>
>>    4. To corrupt the chastity of; to debauch; to violate; to
>>       rape.
>>       [1913 Webster]
>>
>>             The husband murder'd and the wife defiled. --Prior.
>>       [1913 Webster]
>>
>>    5. To make ceremonially unclean; to pollute.
>>       [1913 Webster]
>>
>>             That which dieth of itself, or is torn with beasts,
>>             he shall not eat to defile therewith. --Lev. xxii.
>>                                                   8.
>>       [1913 Webster]
>>
>> Fitting in a way; you're defiling the poor, innocent comment syntax ;)
>>
>
> ok

Seriously: can we give this function a better name?

>>
>> > +    f = open(filename, 'r')
>> > +    while 1:
>>
>> while True:
>>
>> > +        line = f.readline()
>> > +        if not line:
>> > +            break
>> > +        while line[-2:] == '\\\n':
>> > +            nextline = f.readline()
>> > +            if not nextline:
>> > +                break
>> > +            line = line + nextline
>> > +        tmp = line.strip()
>> > +        if tmp[:1] != '#':
>> > +            continue
>> > +        tmp = tmp[1:]
>> > +        words = tmp.split()
>> > +        if words[0] != "define":
>> > +            continue
>> > +        defs.append(words[1])
>> > +    f.close()
>>
>> This parses Yet Another Language.  Worse, Yet Another Undocumented
>> Language.  Why not JSON?
>>
>
>> Hmm, peeking ahead to PATCH 04... aha!  This is for reading
>> config-host.h and config-target.h.  So, this actually doesn't parse
>> YAUL, it parses C.  Sloppily, of course.
>>
>
> Yes
>
>
>>
>> I guess we could instead parse config-host.mak and config-target.mak
>> sloppily.  Not sure which idea is more disgusting :)
>>
>> Could we punt evaluating conditionals to the C compiler?  Instead of
>> emitting TEXT when CONFIG_FOO is defined, emit
>>
>>     #ifdef CONFIG_FOO
>>     TEXT
>>     #endif
>>
>>
> I don't follow you

Let me try to explain with an example.  Say we make
query-gic-capabilities conditional on TARGET_ARM in the schema.  In your
syntax:

    #ifdef TARGET_ARM
    { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
    #endif

Now consider qmp-commands.h.  If we evaluate conditionals at QAPI
generation time, we generate it per target.  For targets with TARGET_ARM
defined, it contains

    GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
    void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret, Error **errp);

For the others, it doesn't contain this text.

If we evaluate conditionals at C compile time, we generate
qmp-commands.h *once*, and it has

    #ifdef TARGET_ARM
    GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
    void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret, Error **errp);
    #endif

It needs to be compiled per target, of course.

The QAPI generator doesn't interpret the conditional text in any way, it
just passes it through to the C compiler.

Makes supporting complex conditions easy (we discussed this before, I
think).  A schema snippet

    #if defined(TARGET_I386) && !defined(TARGET_X86_64)
    ...
    #endif

could generate

    #if defined(TARGET_I386) && !defined(TARGET_X86_64)
    whatever is generated for ...
    #endif

To do this at QAPI generation time, you need to build an expression
evaluator into qapi.py.

Also saves us the trouble of reading config-*.h or config-*.mak.

>> >  def parse_command_line(extra_options="", extra_long_options=[]):
>> >
>> >      try:
>> >          opts, args = getopt.gnu_getopt(sys.argv[1:],
>> > -                                       "chp:o:" + extra_options,
>> > +                                       "chp:o:f:" + extra_options,
>> >                                         ["source", "header", "prefix=",
>> > -                                        "output-dir="] +
>> extra_long_options)
>> > +                                        "output-dir=", "--defile="] +
>>
>> https://docs.python.org/3/library/getopt.html on the third argument:
>> "The leading '--' characters should not be included in the option name."
>>
>
> ok
>
>
>>
>> > +                                       extra_long_options)
>> >      except getopt.GetoptError as err:
>> >          print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
>> >          sys.exit(1)
>> > @@ -1742,6 +1777,8 @@ def parse_command_line(extra_options="", extra_long_options=[]):
>> >              do_c = True
>> >          elif o in ("-h", "--header"):
>> >              do_h = True
>> > +        elif o in ("-f", "--defile"):
>> > +            defile(a)
>> >          else:
>> >              extra_opts.append(oa)
>>
>> --
> Marc-André Lureau

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-06 15:58       ` Markus Armbruster
@ 2016-09-06 16:44         ` Marc-André Lureau
  2016-09-07  8:44           ` Markus Armbruster
  0 siblings, 1 reply; 31+ messages in thread
From: Marc-André Lureau @ 2016-09-06 16:44 UTC (permalink / raw)
  To: Markus Armbruster; +Cc: qemu-devel, Eric Blake

Hi

On Tue, Sep 6, 2016 at 7:58 PM Markus Armbruster <armbru@redhat.com> wrote:

> QAPI language design issues, copying Eric.
>
> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>
> > Hi
> >
> > On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com>
> wrote:
> >
> >> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >>
> >> > Learn to parse #define files provided with -f option, and skip
> >> > undefined #ifdef blocks in the schema.
> >> >
> >> > This is a very simple pre-processing, without stacking support or
> >> > evaluation (it could be implemented if needed).
> >> >
> >> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> >> > ---
> >> >  scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---
> >>
> >> Missing: update of docs/qapi-code-gen.txt.
> >>
> >> >  1 file changed, 40 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/scripts/qapi.py b/scripts/qapi.py
> >> > index 21bc32f..d0b8a66 100644
> >> > --- a/scripts/qapi.py
> >> > +++ b/scripts/qapi.py
> >> > @@ -76,6 +76,7 @@ struct_types = []
> >> >  union_types = []
> >> >  events = []
> >> >  all_names = {}
> >> > +defs = []
> >> >
> >> >  #
> >> >  # Parsing the schema into expressions
> >> > @@ -177,6 +178,7 @@ class QAPISchemaParser(object):
> >> >                  self.exprs.append(expr_elem)
> >> >
> >> >      def accept(self):
> >> > +        ok = True
> >> >          while True:
> >> >              self.tok = self.src[self.cursor]
> >> >              self.pos = self.cursor
> >> > @@ -184,7 +186,19 @@ class QAPISchemaParser(object):
> >> >              self.val = None
> >> >
> >> >              if self.tok == '#':
> >> > -                self.cursor = self.src.find('\n', self.cursor)
> >> > +                end = self.src.find('\n', self.cursor)
> >> > +                line = self.src[self.cursor:end+1]
> >> > +                self.cursor = end
> >> > +                sline = line.split()
> >> > +                if len(defs) and len(sline) >= 1 \
> >> > +                   and sline[0] in ['ifdef', 'endif']:
> >> > +                    if sline[0] == 'ifdef':
> >> > +                        ok = sline[1] in defs
> >> > +                    elif sline[0] == 'endif':
> >> > +                        ok = True
> >> > +                    continue
> >> > +            elif not ok:
> >> > +                continue
> >> >              elif self.tok in "{}:,[]":
> >> >                  return
> >> >              elif self.tok == "'":
> >>
> >> Oww, you're abusing comments!  That's horrible :)
> >>
> >> Can we make this real syntax, like everything else, including 'include'?
> >>
> >>
> > We already abuse json, which doesn't support comments either. :) Even
> > without comments, our syntax is wrong and confuse most
> editors/validators.
>
> True.  Python mode works okay for me, though.
>
> Perhaps we've outgrown the JSON syntax.  Or perhaps we've just messed up
> by taking too many liberties with it, with too little thought.  Not
> exactly an excuse for taking *more* liberties :)
>

It depends, json is very limited. Doing everything is json is not
convenient and not necessary. Why not having preprocessor?


> >> Unfortunately, the natural
> >>
> >>     { 'ifdef': 'CONFIG_FOO'
> >>       'then': ...   # ignored unless CONFIG_FOO
> >>       'else': ...   # ignored if CONFIG_FOO (optional)
> >>     }
> >>
> >> is awkward, because the ... have to be a single JSON value, but a QAPI
> >> schema isn't a single JSON value, it's a *sequence* of JSON values.  A
> >> top-level stanza
> >>
> >>     JSON-value1 JSON-value2 ...
> >>
> >> would become
> >>
> >>     [ JSON-value1, JSON-value2, ... ]
> >>
> >> within a conditional.  Blech.
> >>
> >> Could sacrifice the nesting and do
> >>
> >>     { 'ifdef': 'CONFIG_FOO' }
> >>     ...
> >>     { 'endif' }
> >>
> >> Widens the gap between syntax and semantics.  Editors can no longer
> >> easily jump over the conditional (e.g. Emacs forward-sexp).  Nested
> >> conditionals becomes as confusing as in C.  Not exactly elegant, either.
> >>
> >> Yet another option is to add 'ifdef' keys to the definitions
> >> themselves, i.e.
> >>
> >>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
> >>       'ifdef': 'TARGET_ARM' }
> >>
> >
> > That's the worst of all options imho, as it makes it hard to filter out
> > unions/enums, ex:
> >
> >  @ -3446,8 +3466,10 @@
> >                                         'testdev': 'ChardevCommon',
> >                                         'stdio'  : 'ChardevStdio',
> >                                         'console': 'ChardevCommon',
> > +#ifdef CONFIG_SPICE
> >                                         'spicevmc' :
> 'ChardevSpiceChannel',
> >                                         'spiceport' : 'ChardevSpicePort',
> > +#endif
> >                                         'vc'     : 'ChardevVC',
> >                                         'ringbuf': 'ChardevRingbuf',
>
> Point taken.
>
> Fixable the same way as always when a definition needs to grow
> additional properties, but the syntax only provides a single value: make
> that single value an object, and the old non-object value syntactic
> sugar for the equivalent object value.  We've previously discussed this
> technique in the context of giving command arguments default values.
> I'm not saying this is what we should do here, only pointing out it's
> possible.
>

Ok, but let's find something, if possible simple and convenient, no?


>
> > I think #ifdef is the most straightforward/easy thing to do. My
> experience
> > with doc generations tells me that is really not an issue to reserve the
> > #\w* lines for pre-processing.
>
> Two justifications for doc generators recognizing magic comments: 1. If
> you create a machine processing comments (e.g. formatting them nicely
> for a medium richer than ASCII text, you fundamentally define a comment
> language, embedded in a host language's comments, and 2. what else can
> you do when you can't change the host language?
>
> Neither applies to adding conditional compilation directives to the QAPI
> schema language.  Letting a language grow into its comments is, to be
> frank, disgusting.
>
> Besides, if we truly want the C preprocessor, we should use the C
> preprocessor.  Not implement a half-assed clone of the half-assed hack
> of a macro processor the C preprocessor is.
>

That's possible, although not so easy. First we have to convert all
comments syntax. And then we have to generate files and change the build
sys etc. Doable, but not so great imho. Right now we just need simple
#ifdef conditions that can be easily handled when parsing, far from a full
C preprocessor.


> > And it's a natural fit with the rest of qemu #if conditionals.
>
> It's an awfully unnatural fit with the QAPI schema comment syntax.
>

> ##
> # @Frobs
> #
> # Collection of frobs that need to be frobnicated, except when
> # ifdef CONFIG_NOFROB
> { 'struct': 'Frobs'
>   ...
> }
>
> I stand by 'horrible'.
>

Ok. let's switch the comment syntax to a regular js/c style then?

>
> >> > @@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
> >> >  #
> >> >  # Common command line parsing
> >> >  #
> >> > +def defile(filename):
> >>
> >> From The Collaborative International Dictionary of English v.0.48
> [gcide]:
> >>
> >> Defile \De*file"\ (d[-e]*f[imac]l"), v. t. [OE. defoulen,
> >>    -foilen, to tread down, OF. defouler; de- + fouler to trample
> >>    (see Full, v. t.), and OE. defoulen to foul (influenced in
> >>    form by the older verb defoilen). See File to defile,
> >>    Foul, Defoul.]
> >>    1. To make foul or impure; to make filthy; to dirty; to
> >>       befoul; to pollute.
> >>       [1913 Webster]
> >>
> >>             They that touch pitch will be defiled. --Shak.
> >>       [1913 Webster]
> >>
> >>    2. To soil or sully; to tarnish, as reputation; to taint.
> >>       [1913 Webster]
> >>
> >>             He is . . . among the greatest prelates of this age,
> >>             however his character may be defiled by . . . dirty
> >>             hands.                                --Swift.
> >>       [1913 Webster]
> >>
> >>    3. To injure in purity of character; to corrupt.
> >>       [1913 Webster]
> >>
> >>             Defile not yourselves with the idols of Egypt.
> >>                                                   --Ezek. xx. 7.
> >>       [1913 Webster]
> >>
> >>    4. To corrupt the chastity of; to debauch; to violate; to
> >>       rape.
> >>       [1913 Webster]
> >>
> >>             The husband murder'd and the wife defiled. --Prior.
> >>       [1913 Webster]
> >>
> >>    5. To make ceremonially unclean; to pollute.
> >>       [1913 Webster]
> >>
> >>             That which dieth of itself, or is torn with beasts,
> >>             he shall not eat to defile therewith. --Lev. xxii.
> >>                                                   8.
> >>       [1913 Webster]
> >>
> >> Fitting in a way; you're defiling the poor, innocent comment syntax ;)
> >>
> >
> > ok
>
> Seriously: can we give this function a better name?
>

yes load_define_file?


>
> >>
> >> > +    f = open(filename, 'r')
> >> > +    while 1:
> >>
> >> while True:
> >>
> >> > +        line = f.readline()
> >> > +        if not line:
> >> > +            break
> >> > +        while line[-2:] == '\\\n':
> >> > +            nextline = f.readline()
> >> > +            if not nextline:
> >> > +                break
> >> > +            line = line + nextline
> >> > +        tmp = line.strip()
> >> > +        if tmp[:1] != '#':
> >> > +            continue
> >> > +        tmp = tmp[1:]
> >> > +        words = tmp.split()
> >> > +        if words[0] != "define":
> >> > +            continue
> >> > +        defs.append(words[1])
> >> > +    f.close()
> >>
> >> This parses Yet Another Language.  Worse, Yet Another Undocumented
> >> Language.  Why not JSON?
> >>
> >
> >> Hmm, peeking ahead to PATCH 04... aha!  This is for reading
> >> config-host.h and config-target.h.  So, this actually doesn't parse
> >> YAUL, it parses C.  Sloppily, of course.
> >>
> >
> > Yes
> >
> >
> >>
> >> I guess we could instead parse config-host.mak and config-target.mak
> >> sloppily.  Not sure which idea is more disgusting :)
> >>
> >> Could we punt evaluating conditionals to the C compiler?  Instead of
> >> emitting TEXT when CONFIG_FOO is defined, emit
> >>
> >>     #ifdef CONFIG_FOO
> >>     TEXT
> >>     #endif
> >>
> >>
> > I don't follow you
>
> Let me try to explain with an example.  Say we make
> query-gic-capabilities conditional on TARGET_ARM in the schema.  In your
> syntax:
>
>     #ifdef TARGET_ARM
>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
>     #endif
>
> Now consider qmp-commands.h.  If we evaluate conditionals at QAPI
> generation time, we generate it per target.  For targets with TARGET_ARM
> defined, it contains
>
>     GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
>     void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret,
> Error **errp);
>
> For the others, it doesn't contain this text.
>
> If we evaluate conditionals at C compile time, we generate
> qmp-commands.h *once*, and it has
>
>     #ifdef TARGET_ARM
>     GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
>     void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret,
> Error **errp);
>     #endif
>
> It needs to be compiled per target, of course.
>

Doable for commands, but not easily doable for introspect.c. I tried to
implement that at first, it was really complicated. I think we should also
consider simplicity here.


>
> The QAPI generator doesn't interpret the conditional text in any way, it
> just passes it through to the C compiler.
>
> Makes supporting complex conditions easy (we discussed this before, I
> think).  A schema snippet
>
>     #if defined(TARGET_I386) && !defined(TARGET_X86_64)
>     ...
>     #endif
>
> could generate
>
>     #if defined(TARGET_I386) && !defined(TARGET_X86_64)
>     whatever is generated for ...
>     #endif
>
> To do this at QAPI generation time, you need to build an expression
> evaluator into qapi.py.
>

That's relatively easy, I've done that a couple of time, I can do it here
too. Only it was not needed so far.

>
> Also saves us the trouble of reading config-*.h or config-*.mak.
>
>  If needed we could generate a simpler file, like a python config file, or
generating command line arguments, etc.. with just the values we need for
qapi generation. But reading the config-*.h is quite straightforward.


> >> >  def parse_command_line(extra_options="", extra_long_options=[]):
> >> >
> >> >      try:
> >> >          opts, args = getopt.gnu_getopt(sys.argv[1:],
> >> > -                                       "chp:o:" + extra_options,
> >> > +                                       "chp:o:f:" + extra_options,
> >> >                                         ["source", "header",
> "prefix=",
> >> > -                                        "output-dir="] +
> >> extra_long_options)
> >> > +                                        "output-dir=", "--defile="] +
> >>
> >> https://docs.python.org/3/library/getopt.html on the third argument:
> >> "The leading '--' characters should not be included in the option name."
> >>
> >
> > ok
> >
> >
> >>
> >> > +                                       extra_long_options)
> >> >      except getopt.GetoptError as err:
> >> >          print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
> >> >          sys.exit(1)
> >> > @@ -1742,6 +1777,8 @@ def parse_command_line(extra_options="",
> extra_long_options=[]):
> >> >              do_c = True
> >> >          elif o in ("-h", "--header"):
> >> >              do_h = True
> >> > +        elif o in ("-f", "--defile"):
> >> > +            defile(a)
> >> >          else:
> >> >              extra_opts.append(oa)
> >>
> >> --
> > Marc-André Lureau
>
-- 
Marc-André Lureau

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-06 16:44         ` Marc-André Lureau
@ 2016-09-07  8:44           ` Markus Armbruster
  2016-09-07 13:40             ` Markus Armbruster
  0 siblings, 1 reply; 31+ messages in thread
From: Markus Armbruster @ 2016-09-07  8:44 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel

Marc-André Lureau <marcandre.lureau@gmail.com> writes:

> Hi
>
> On Tue, Sep 6, 2016 at 7:58 PM Markus Armbruster <armbru@redhat.com> wrote:
>
>> QAPI language design issues, copying Eric.
>>
>> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>>
>> > Hi
>> >
>> > On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com>
>> wrote:
>> >
>> >> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>> >>
>> >> > Learn to parse #define files provided with -f option, and skip
>> >> > undefined #ifdef blocks in the schema.
>> >> >
>> >> > This is a very simple pre-processing, without stacking support or
>> >> > evaluation (it could be implemented if needed).
>> >> >
>> >> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >> > ---
>> >> >  scripts/qapi.py | 43 ++++++++++++++++++++++++++++++++++++++++---
>> >>
>> >> Missing: update of docs/qapi-code-gen.txt.
>> >>
>> >> >  1 file changed, 40 insertions(+), 3 deletions(-)
>> >> >
>> >> > diff --git a/scripts/qapi.py b/scripts/qapi.py
>> >> > index 21bc32f..d0b8a66 100644
>> >> > --- a/scripts/qapi.py
>> >> > +++ b/scripts/qapi.py
>> >> > @@ -76,6 +76,7 @@ struct_types = []
>> >> >  union_types = []
>> >> >  events = []
>> >> >  all_names = {}
>> >> > +defs = []
>> >> >
>> >> >  #
>> >> >  # Parsing the schema into expressions
>> >> > @@ -177,6 +178,7 @@ class QAPISchemaParser(object):
>> >> >                  self.exprs.append(expr_elem)
>> >> >
>> >> >      def accept(self):
>> >> > +        ok = True
>> >> >          while True:
>> >> >              self.tok = self.src[self.cursor]
>> >> >              self.pos = self.cursor
>> >> > @@ -184,7 +186,19 @@ class QAPISchemaParser(object):
>> >> >              self.val = None
>> >> >
>> >> >              if self.tok == '#':
>> >> > -                self.cursor = self.src.find('\n', self.cursor)
>> >> > +                end = self.src.find('\n', self.cursor)
>> >> > +                line = self.src[self.cursor:end+1]
>> >> > +                self.cursor = end
>> >> > +                sline = line.split()
>> >> > +                if len(defs) and len(sline) >= 1 \
>> >> > +                   and sline[0] in ['ifdef', 'endif']:
>> >> > +                    if sline[0] == 'ifdef':
>> >> > +                        ok = sline[1] in defs
>> >> > +                    elif sline[0] == 'endif':
>> >> > +                        ok = True
>> >> > +                    continue
>> >> > +            elif not ok:
>> >> > +                continue
>> >> >              elif self.tok in "{}:,[]":
>> >> >                  return
>> >> >              elif self.tok == "'":
>> >>
>> >> Oww, you're abusing comments!  That's horrible :)
>> >>
>> >> Can we make this real syntax, like everything else, including 'include'?
>> >>
>> >>
>> > We already abuse json, which doesn't support comments either. :) Even
>> > without comments, our syntax is wrong and confuse most
>> editors/validators.
>>
>> True.  Python mode works okay for me, though.
>>
>> Perhaps we've outgrown the JSON syntax.  Or perhaps we've just messed up
>> by taking too many liberties with it, with too little thought.  Not
>> exactly an excuse for taking *more* liberties :)
>>
>
> It depends, json is very limited. Doing everything is json is not
> convenient and not necessary. Why not having preprocessor?

In language design, a preprocessor transforming text is a cop out.

With a simple and regular syntax such as JSON, syntax tree
transformations are at least as easy to grasp, and make more sense.
Compare the C preprocessor (text-transforming water pistol, tacked on,
way too complex for what it can do) to Lisp macros (syntax-transforming
auto-cannon, compile-time language identical to run-time language) to
C++ templates (syntax-transforming auto-cannon, completely separate
compile-time language, fully understood by about three people, one has
since died, one has forgotten everything, and one has gone crazy).

I'm not asking for Lisp macros here.  Sometimes, the most practical
solution is to cop out.  I just want us to think before we cop out.

>> >> Unfortunately, the natural
>> >>
>> >>     { 'ifdef': 'CONFIG_FOO'
>> >>       'then': ...   # ignored unless CONFIG_FOO
>> >>       'else': ...   # ignored if CONFIG_FOO (optional)
>> >>     }
>> >>
>> >> is awkward, because the ... have to be a single JSON value, but a QAPI
>> >> schema isn't a single JSON value, it's a *sequence* of JSON values.  A
>> >> top-level stanza
>> >>
>> >>     JSON-value1 JSON-value2 ...
>> >>
>> >> would become
>> >>
>> >>     [ JSON-value1, JSON-value2, ... ]
>> >>
>> >> within a conditional.  Blech.
>> >>
>> >> Could sacrifice the nesting and do
>> >>
>> >>     { 'ifdef': 'CONFIG_FOO' }
>> >>     ...
>> >>     { 'endif' }
>> >>
>> >> Widens the gap between syntax and semantics.  Editors can no longer
>> >> easily jump over the conditional (e.g. Emacs forward-sexp).  Nested
>> >> conditionals becomes as confusing as in C.  Not exactly elegant, either.
>> >>
>> >> Yet another option is to add 'ifdef' keys to the definitions
>> >> themselves, i.e.
>> >>
>> >>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
>> >>       'ifdef': 'TARGET_ARM' }
>> >>
>> >
>> > That's the worst of all options imho, as it makes it hard to filter out
>> > unions/enums, ex:
>> >
>> >  @ -3446,8 +3466,10 @@
>> >                                         'testdev': 'ChardevCommon',
>> >                                         'stdio'  : 'ChardevStdio',
>> >                                         'console': 'ChardevCommon',
>> > +#ifdef CONFIG_SPICE
>> >                                         'spicevmc' :
>> 'ChardevSpiceChannel',
>> >                                         'spiceport' : 'ChardevSpicePort',
>> > +#endif
>> >                                         'vc'     : 'ChardevVC',
>> >                                         'ringbuf': 'ChardevRingbuf',
>>
>> Point taken.
>>
>> Fixable the same way as always when a definition needs to grow
>> additional properties, but the syntax only provides a single value: make
>> that single value an object, and the old non-object value syntactic
>> sugar for the equivalent object value.  We've previously discussed this
>> technique in the context of giving command arguments default values.
>> I'm not saying this is what we should do here, only pointing out it's
>> possible.
>>
>
> Ok, but let's find something, if possible simple and convenient, no?

I agree it needs to be simple, both the interface (QAPI language) and
the implementation.  However, I don't like "first past the post".  I
prefer to explore the design space a bit.

So let me explore the "add 'ifdef' keys to definitions" corner of the
QAPI language design space.

Easily done for top-level definitions, because they're all JSON objects.
Could even add it to the include directive if we wanted to.

Less easily done for enumeration, struct, union and alternate members.
Note that command and event arguments specified inline are a special
case of struct members.

The "can't specify additional stuff for struct members" problem isn't
new.  We hacked around it to specify "optional": encode it into the
member name.  Doesn't scale.  We need to solve the problem to be able to
specify default values, and we already decided how: have an JSON object
instead of a mere JSON string, make the string syntax sugar for {
'type': STRING }.  See commit 6b5abc7 and the discussion in qemu-devel
leading up to it.  For consistency, we'll do it for union and alternate
members, too.

That leaves just enumeration members.  The same technique applies.

If I remember correctly, we only need conditional commands right now, to
avoid regressing query-commands.  The more complicated member work could
be done later.

>> > I think #ifdef is the most straightforward/easy thing to do. My
>> experience
>> > with doc generations tells me that is really not an issue to reserve the
>> > #\w* lines for pre-processing.
>>
>> Two justifications for doc generators recognizing magic comments: 1. If
>> you create a machine processing comments (e.g. formatting them nicely
>> for a medium richer than ASCII text, you fundamentally define a comment
>> language, embedded in a host language's comments, and 2. what else can
>> you do when you can't change the host language?
>>
>> Neither applies to adding conditional compilation directives to the QAPI
>> schema language.  Letting a language grow into its comments is, to be
>> frank, disgusting.
>>
>> Besides, if we truly want the C preprocessor, we should use the C
>> preprocessor.  Not implement a half-assed clone of the half-assed hack
>> of a macro processor the C preprocessor is.
>>
>
> That's possible, although not so easy. First we have to convert all
> comments syntax. And then we have to generate files and change the build
> sys etc. Doable, but not so great imho. Right now we just need simple
> #ifdef conditions that can be easily handled when parsing, far from a full
> C preprocessor.

Things always start that way.  Trouble is they rarely stay that way.

>> > And it's a natural fit with the rest of qemu #if conditionals.
>>
>> It's an awfully unnatural fit with the QAPI schema comment syntax.
>>
>
>> ##
>> # @Frobs
>> #
>> # Collection of frobs that need to be frobnicated, except when
>> # ifdef CONFIG_NOFROB
>> { 'struct': 'Frobs'
>>   ...
>> }
>>
>> I stand by 'horrible'.
>>
>
> Ok. let's switch the comment syntax to a regular js/c style then?

Switching to JavaScript comment syntax is an option.  Painful loss of
git-blame information, though.

>> >> > @@ -1707,15 +1721,36 @@ def gen_params(arg_type, boxed, extra):
>> >> >  #
>> >> >  # Common command line parsing
>> >> >  #
>> >> > +def defile(filename):
>> >>
>> >> From The Collaborative International Dictionary of English v.0.48
>> [gcide]:
>> >>
>> >> Defile \De*file"\ (d[-e]*f[imac]l"), v. t. [OE. defoulen,
>> >>    -foilen, to tread down, OF. defouler; de- + fouler to trample
>> >>    (see Full, v. t.), and OE. defoulen to foul (influenced in
>> >>    form by the older verb defoilen). See File to defile,
>> >>    Foul, Defoul.]
>> >>    1. To make foul or impure; to make filthy; to dirty; to
>> >>       befoul; to pollute.
>> >>       [1913 Webster]
>> >>
>> >>             They that touch pitch will be defiled. --Shak.
>> >>       [1913 Webster]
>> >>
>> >>    2. To soil or sully; to tarnish, as reputation; to taint.
>> >>       [1913 Webster]
>> >>
>> >>             He is . . . among the greatest prelates of this age,
>> >>             however his character may be defiled by . . . dirty
>> >>             hands.                                --Swift.
>> >>       [1913 Webster]
>> >>
>> >>    3. To injure in purity of character; to corrupt.
>> >>       [1913 Webster]
>> >>
>> >>             Defile not yourselves with the idols of Egypt.
>> >>                                                   --Ezek. xx. 7.
>> >>       [1913 Webster]
>> >>
>> >>    4. To corrupt the chastity of; to debauch; to violate; to
>> >>       rape.
>> >>       [1913 Webster]
>> >>
>> >>             The husband murder'd and the wife defiled. --Prior.
>> >>       [1913 Webster]
>> >>
>> >>    5. To make ceremonially unclean; to pollute.
>> >>       [1913 Webster]
>> >>
>> >>             That which dieth of itself, or is torn with beasts,
>> >>             he shall not eat to defile therewith. --Lev. xxii.
>> >>                                                   8.
>> >>       [1913 Webster]
>> >>
>> >> Fitting in a way; you're defiling the poor, innocent comment syntax ;)
>> >>
>> >
>> > ok
>>
>> Seriously: can we give this function a better name?
>>
>
> yes load_define_file?

read_config_h?

>> >> > +    f = open(filename, 'r')
>> >> > +    while 1:
>> >>
>> >> while True:
>> >>
>> >> > +        line = f.readline()
>> >> > +        if not line:
>> >> > +            break
>> >> > +        while line[-2:] == '\\\n':
>> >> > +            nextline = f.readline()
>> >> > +            if not nextline:
>> >> > +                break
>> >> > +            line = line + nextline
>> >> > +        tmp = line.strip()
>> >> > +        if tmp[:1] != '#':
>> >> > +            continue
>> >> > +        tmp = tmp[1:]
>> >> > +        words = tmp.split()
>> >> > +        if words[0] != "define":
>> >> > +            continue
>> >> > +        defs.append(words[1])
>> >> > +    f.close()
>> >>
>> >> This parses Yet Another Language.  Worse, Yet Another Undocumented
>> >> Language.  Why not JSON?
>> >>
>> >
>> >> Hmm, peeking ahead to PATCH 04... aha!  This is for reading
>> >> config-host.h and config-target.h.  So, this actually doesn't parse
>> >> YAUL, it parses C.  Sloppily, of course.
>> >>
>> >
>> > Yes
>> >
>> >
>> >>
>> >> I guess we could instead parse config-host.mak and config-target.mak
>> >> sloppily.  Not sure which idea is more disgusting :)
>> >>
>> >> Could we punt evaluating conditionals to the C compiler?  Instead of
>> >> emitting TEXT when CONFIG_FOO is defined, emit
>> >>
>> >>     #ifdef CONFIG_FOO
>> >>     TEXT
>> >>     #endif
>> >>
>> >>
>> > I don't follow you
>>
>> Let me try to explain with an example.  Say we make
>> query-gic-capabilities conditional on TARGET_ARM in the schema.  In your
>> syntax:
>>
>>     #ifdef TARGET_ARM
>>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
>>     #endif
>>
>> Now consider qmp-commands.h.  If we evaluate conditionals at QAPI
>> generation time, we generate it per target.  For targets with TARGET_ARM
>> defined, it contains
>>
>>     GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
>>     void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret,
>> Error **errp);
>>
>> For the others, it doesn't contain this text.
>>
>> If we evaluate conditionals at C compile time, we generate
>> qmp-commands.h *once*, and it has
>>
>>     #ifdef TARGET_ARM
>>     GICCapabilityList *qmp_query_gic_capabilities(Error **errp);
>>     void qmp_marshal_query_gic_capabilities(QDict *args, QObject **ret,
>> Error **errp);
>>     #endif
>>
>> It needs to be compiled per target, of course.
>>
>
> Doable for commands, but not easily doable for introspect.c. I tried to
> implement that at first, it was really complicated. I think we should also
> consider simplicity here.

Of course.

I'd like to have a look at qapi-introspect.py myself.

>> The QAPI generator doesn't interpret the conditional text in any way, it
>> just passes it through to the C compiler.
>>
>> Makes supporting complex conditions easy (we discussed this before, I
>> think).  A schema snippet
>>
>>     #if defined(TARGET_I386) && !defined(TARGET_X86_64)
>>     ...
>>     #endif
>>
>> could generate
>>
>>     #if defined(TARGET_I386) && !defined(TARGET_X86_64)
>>     whatever is generated for ...
>>     #endif
>>
>> To do this at QAPI generation time, you need to build an expression
>> evaluator into qapi.py.
>>
>
> That's relatively easy, I've done that a couple of time, I can do it here
> too. Only it was not needed so far.

See "Trouble is they rarely stay that way" above.

>> Also saves us the trouble of reading config-*.h or config-*.mak.
>
>  If needed we could generate a simpler file, like a python config file, or
> generating command line arguments, etc.. with just the values we need for
> qapi generation.

Yes.

>                  But reading the config-*.h is quite straightforward.

Until somebody gets clever in configure, and things break.

We can ignore this detail for now.

[...]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-07  8:44           ` Markus Armbruster
@ 2016-09-07 13:40             ` Markus Armbruster
  2016-09-07 14:23               ` Marc-André Lureau
  2016-09-07 18:49               ` Eric Blake
  0 siblings, 2 replies; 31+ messages in thread
From: Markus Armbruster @ 2016-09-07 13:40 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel, Eric Blake

Markus Armbruster <armbru@redhat.com> writes:

> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>
>> Hi
>>
>> On Tue, Sep 6, 2016 at 7:58 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>>> QAPI language design issues, copying Eric.
>>>
>>> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>>>
>>> > Hi
>>> >
>>> > On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com>
>>> wrote:
[...]
>>> >> Yet another option is to add 'ifdef' keys to the definitions
>>> >> themselves, i.e.
>>> >>
>>> >>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
>>> >>       'ifdef': 'TARGET_ARM' }
>>> >>
>>> >
>>> > That's the worst of all options imho, as it makes it hard to filter out
>>> > unions/enums, ex:
>>> >
>>> >  @ -3446,8 +3466,10 @@
>>> >                                         'testdev': 'ChardevCommon',
>>> >                                         'stdio'  : 'ChardevStdio',
>>> >                                         'console': 'ChardevCommon',
>>> > +#ifdef CONFIG_SPICE
>>> >                                         'spicevmc' :
>>> 'ChardevSpiceChannel',
>>> >                                         'spiceport' : 'ChardevSpicePort',
>>> > +#endif
>>> >                                         'vc'     : 'ChardevVC',
>>> >                                         'ringbuf': 'ChardevRingbuf',
>>>
>>> Point taken.
>>>
>>> Fixable the same way as always when a definition needs to grow
>>> additional properties, but the syntax only provides a single value: make
>>> that single value an object, and the old non-object value syntactic
>>> sugar for the equivalent object value.  We've previously discussed this
>>> technique in the context of giving command arguments default values.
>>> I'm not saying this is what we should do here, only pointing out it's
>>> possible.
>>>
>>
>> Ok, but let's find something, if possible simple and convenient, no?
>
> I agree it needs to be simple, both the interface (QAPI language) and
> the implementation.  However, I don't like "first past the post".  I
> prefer to explore the design space a bit.
>
> So let me explore the "add 'ifdef' keys to definitions" corner of the
> QAPI language design space.
>
> Easily done for top-level definitions, because they're all JSON objects.
> Could even add it to the include directive if we wanted to.
>
> Less easily done for enumeration, struct, union and alternate members.
> Note that command and event arguments specified inline are a special
> case of struct members.
>
> The "can't specify additional stuff for struct members" problem isn't
> new.  We hacked around it to specify "optional": encode it into the
> member name.  Doesn't scale.  We need to solve the problem to be able to
> specify default values, and we already decided how: have an JSON object
> instead of a mere JSON string, make the string syntax sugar for {
> 'type': STRING }.  See commit 6b5abc7 and the discussion in qemu-devel
> leading up to it.  For consistency, we'll do it for union and alternate
> members, too.
>
> That leaves just enumeration members.  The same technique applies.
>
> If I remember correctly, we only need conditional commands right now, to
> avoid regressing query-commands.  The more complicated member work could
> be done later.

To gauge whether this idea is practical, I implemented key 'if' for
commands.  It's just a sketch, and has a number of issues, which I
marked FIXME.

I ported qmp-commands.hx's #if to qapi-schema.json.  The TARGET_FOO are
poisoned, so I commented them out.  There's a CONFIG_SPICE left, which
will do for testing.

I also turned key 'gen': false into 'if': false.  Possibly a bad idea.

Anyway, diffstat isn't bad:

 docs/qapi-code-gen.txt                     | 14 ++++++-----
 qapi-schema.json                           | 15 ++++++++---
 qapi/introspect.json                       |  2 +-
 scripts/qapi-commands.py                   | 12 +++++++--
 scripts/qapi-introspect.py                 | 22 ++++++++++------
 scripts/qapi.py                            | 40 ++++++++++++++++++++++--------
 tests/qapi-schema/type-bypass-bad-gen.err  |  2 +-
 tests/qapi-schema/type-bypass-bad-gen.json |  4 +--
 8 files changed, 77 insertions(+), 34 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index de298dc..93e99d8 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -423,7 +423,7 @@ part of a Client JSON Protocol command.  The 'data' member is optional
 and defaults to {} (an empty dictionary).  If present, it must be the
 string name of a complex type, or a dictionary that declares an
 anonymous type with the same semantics as a 'struct' expression, with
-one exception noted below when 'gen' is used.
+one exception noted below when 'if': false is used.
 
 The 'returns' member describes what will appear in the "return" member
 of a Client JSON Protocol reply on successful completion of a command.
@@ -431,8 +431,8 @@ The member is optional from the command declaration; if absent, the
 "return" member will be an empty dictionary.  If 'returns' is present,
 it must be the string name of a complex or built-in type, a
 one-element array containing the name of a complex or built-in type,
-with one exception noted below when 'gen' is used.  Although it is
-permitted to have the 'returns' member name a built-in type or an
+with one exception noted below when 'if':false is used.  Although it
+is permitted to have the 'returns' member name a built-in type or an
 array of built-in types, any command that does this cannot be extended
 to return additional information in the future; thus, new commands
 should strongly consider returning a dictionary-based type or an array
@@ -475,16 +475,18 @@ arguments for the user's function out of an input QDict, calls the
 user's function, and if it succeeded, builds an output QObject from
 its return value.
 
+FIXME document 'if'
+
 In rare cases, QAPI cannot express a type-safe representation of a
 corresponding Client JSON Protocol command.  You then have to suppress
-generation of a marshalling function by including a key 'gen' with
+generation of a marshalling function by including a key 'if' with
 boolean value false, and instead write your own function.  Please try
 to avoid adding new commands that rely on this, and instead use
 type-safe unions.  For an example of this usage:
 
  { 'command': 'netdev_add',
-   'data': {'type': 'str', 'id': 'str'},
-   'gen': false }
+   'if': false,
+   'data': {'type': 'str', 'id': 'str'} }
 
 Normally, the QAPI schema is used to describe synchronous exchanges,
 where a response is expected.  But in some cases, the action of a
diff --git a/qapi-schema.json b/qapi-schema.json
index c4f3674..ad0559e 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1269,7 +1269,9 @@
 #
 # Since: 0.14.0
 ##
-{ 'command': 'query-spice', 'returns': 'SpiceInfo' }
+{ 'command': 'query-spice',
+  'if': 'defined(CONFIG_SPICE)',
+  'returns': 'SpiceInfo' }
 
 ##
 # @BalloonInfo:
@@ -2355,6 +2357,7 @@
 # Since: 2.5
 ##
 { 'command': 'dump-skeys',
+#  'if': 'defined(TARGET_S390X)',
   'data': { 'filename': 'str' } }
 
 ##
@@ -2380,8 +2383,8 @@
 #          If @type is not a valid network backend, DeviceNotFound
 ##
 { 'command': 'netdev_add',
-  'data': {'type': 'str', 'id': 'str'},
-  'gen': false }                # so we can get the additional arguments
+  'if': false,                  # so we can get the additional arguments
+  'data': {'type': 'str', 'id': 'str'} }
 
 ##
 # @netdev_del:
@@ -4455,6 +4458,7 @@
 # Since: 2.1
 ##
 { 'command': 'rtc-reset-reinjection' }
+#  'if': 'defined(TARGET_I386)'
 
 # Rocker ethernet network switch
 { 'include': 'qapi/rocker.json' }
@@ -4525,7 +4529,10 @@
 #
 # Since: 2.6
 ##
-{ 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
+{ 'command': 'query-gic-capabilities',
+#  'if': 'defined(TARGET_ARM)',
+  'returns': ['GICCapability']
+}
 
 ##
 # CpuInstanceProperties
diff --git a/qapi/introspect.json b/qapi/introspect.json
index 3fd81fb..b8f421a 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -46,7 +46,7 @@
 ##
 { 'command': 'query-qmp-schema',
   'returns': [ 'SchemaInfo' ],
-  'gen': false }                # just to simplify qmp_query_json()
+  'if': false }                 # just to simplify qmp_query_json()
 
 ##
 # @SchemaMetaType
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a06a2c4..f34e4cc 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -215,9 +215,13 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
         self._visited_ret_types = None
 
     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
-        if not gen:
+                      genif, success_response, boxed):
+        if genif is False:
             return
+        pp_if = gen_pp_if(genif)
+        pp_endif = gen_pp_endif(genif)
+        self.decl += pp_if
+        self.defn += pp_if      # FIXME blank lines are off
         self.decl += gen_command_decl(name, arg_type, boxed, ret_type)
         if ret_type and ret_type not in self._visited_ret_types:
             self._visited_ret_types.add(ret_type)
@@ -226,7 +230,11 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
             self.decl += gen_marshal_decl(name)
         self.defn += gen_marshal(name, arg_type, boxed, ret_type)
         if not middle_mode:
+            self._regy += pp_if
             self._regy += gen_register_command(name, success_response)
+            self._regy += pp_endif
+        self.decl += pp_endif
+        self.defn += pp_endif
 
 
 middle_mode = False
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index 541644e..0d8cec7 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -30,8 +30,6 @@ def to_json(obj, level=0):
         ret = '{' + ', '.join(elts) + '}'
     else:
         assert False                # not implemented
-    if level == 1:
-        ret = '\n' + ret
     return ret
 
 
@@ -69,8 +67,15 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor):
 extern const char %(c_name)s[];
 ''',
                           c_name=c_name(name))
-        lines = to_json(jsons).split('\n')
-        c_string = '\n    '.join([to_c_string(line) for line in lines])
+        c_string = '"["'
+        for i in jsons:
+            js, genif = i
+            # FIXME blank lines are off
+            c_string += gen_pp_if(genif or True)
+            c_string += '\n    ' + to_c_string(to_json(js) + ', ')
+            c_string += gen_pp_endif(genif or True)
+        # FIXME trailing comma (JSON sucks)
+        c_string += '\n    "]"'
         self.defn = mcgen('''
 const char %(c_name)s[] = %(c_string)s;
 ''',
@@ -111,12 +116,12 @@ const char %(c_name)s[] = %(c_string)s;
             return '[' + self._use_type(typ.element_type) + ']'
         return self._name(typ.name)
 
-    def _gen_json(self, name, mtype, obj):
+    def _gen_json(self, name, mtype, obj, genif=True):
         if mtype not in ('command', 'event', 'builtin', 'array'):
             name = self._name(name)
         obj['name'] = name
         obj['meta-type'] = mtype
-        self._jsons.append(obj)
+        self._jsons.append((obj, genif))
 
     def _gen_member(self, member):
         ret = {'name': member.name, 'type': self._use_type(member.type)}
@@ -154,12 +159,13 @@ const char %(c_name)s[] = %(c_string)s;
                                     for m in variants.variants]})
 
     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      genif, success_response, boxed):
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
         self._gen_json(name, 'command',
                        {'arg-type': self._use_type(arg_type),
-                        'ret-type': self._use_type(ret_type)})
+                        'ret-type': self._use_type(ret_type)},
+                       genif)
 
     def visit_event(self, name, info, arg_type, boxed):
         arg_type = arg_type or self._schema.the_empty_object_type
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 21bc32f..6c0cf9f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -698,7 +698,13 @@ def check_keys(expr_elem, meta, required, optional=[]):
             raise QAPIExprError(info,
                                 "Unknown key '%s' in %s '%s'"
                                 % (key, meta, name))
-        if (key == 'gen' or key == 'success-response') and value is not False:
+        if (key == 'if'
+            and value is not False and not isinstance(value, str)):
+            # FIXME update error message
+            raise QAPIExprError(info,
+                                "'%s' of %s '%s' should only use false value"
+                                % (key, meta, name))
+        if (key == 'success-response') and value is not False:
             raise QAPIExprError(info,
                                 "'%s' of %s '%s' should only use false value"
                                 % (key, meta, name))
@@ -737,7 +743,7 @@ def check_exprs(exprs):
             add_struct(expr, info)
         elif 'command' in expr:
             check_keys(expr_elem, 'command', [],
-                       ['data', 'returns', 'gen', 'success-response', 'boxed'])
+                       ['data', 'returns', 'if', 'success-response', 'boxed'])
             add_name(expr['command'], info, 'command')
         elif 'event' in expr:
             check_keys(expr_elem, 'event', [], ['data', 'boxed'])
@@ -838,7 +844,7 @@ class QAPISchemaVisitor(object):
         pass
 
     def visit_command(self, name, info, arg_type, ret_type,
-                      gen, success_response, boxed):
+                      genif, success_response, boxed):
         pass
 
     def visit_event(self, name, info, arg_type, boxed):
@@ -1180,8 +1186,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
 
 
 class QAPISchemaCommand(QAPISchemaEntity):
-    def __init__(self, name, info, arg_type, ret_type, gen, success_response,
-                 boxed):
+    def __init__(self, name, info, arg_type, ret_type,
+                 genif, success_response, boxed):
         QAPISchemaEntity.__init__(self, name, info)
         assert not arg_type or isinstance(arg_type, str)
         assert not ret_type or isinstance(ret_type, str)
@@ -1189,7 +1195,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         self.arg_type = None
         self._ret_type_name = ret_type
         self.ret_type = None
-        self.gen = gen
+        self.genif = genif
         self.success_response = success_response
         self.boxed = boxed
 
@@ -1216,7 +1222,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
     def visit(self, visitor):
         visitor.visit_command(self.name, self.info,
                               self.arg_type, self.ret_type,
-                              self.gen, self.success_response, self.boxed)
+                              self.genif, self.success_response, self.boxed)
 
 
 class QAPISchemaEvent(QAPISchemaEntity):
@@ -1419,17 +1425,20 @@ class QAPISchema(object):
         name = expr['command']
         data = expr.get('data')
         rets = expr.get('returns')
-        gen = expr.get('gen', True)
+        genif = expr.get('if', True)
         success_response = expr.get('success-response', True)
         boxed = expr.get('boxed', False)
         if isinstance(data, OrderedDict):
+            # TODO apply genif to the implicit object type
             data = self._make_implicit_object_type(
                 name, info, 'arg', self._make_members(data, info))
         if isinstance(rets, list):
+            # TODO apply genif to the implicit array type
+            # TODO disjunction of all the genif
             assert len(rets) == 1
             rets = self._make_array_type(rets[0], info)
-        self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
-                                           success_response, boxed))
+        self._def_entity(QAPISchemaCommand(name, info, data, rets,
+                                           genif, success_response, boxed))
 
     def _def_event(self, expr, info):
         name = expr['event']
@@ -1704,6 +1713,17 @@ def gen_params(arg_type, boxed, extra):
     return ret
 
 
+def gen_pp_if(cond):
+    if cond is True:
+        return ''
+    return '\n#if ' + cond + '\n'
+
+
+def gen_pp_endif(cond):
+    if cond is True:
+        return ''
+    return '\n#endif  /* ' + cond + ' */\n'
+
 #
 # Common command line parsing
 #
diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
index a83c3c6..cca25f1 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.err
+++ b/tests/qapi-schema/type-bypass-bad-gen.err
@@ -1 +1 @@
-tests/qapi-schema/type-bypass-bad-gen.json:2: 'gen' of command 'foo' should only use false value
+tests/qapi-schema/type-bypass-bad-gen.json:2: 'if' of command 'foo' should only use false value
diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
index e8dec34..637b11f 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.json
+++ b/tests/qapi-schema/type-bypass-bad-gen.json
@@ -1,2 +1,2 @@
-# 'gen' should only appear with value false
-{ 'command': 'foo', 'gen': 'whatever' }
+# 'if' should only appear with value false FIXME or str
+{ 'command': 'foo', 'if': null }

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-07 13:40             ` Markus Armbruster
@ 2016-09-07 14:23               ` Marc-André Lureau
  2016-09-07 18:49               ` Eric Blake
  1 sibling, 0 replies; 31+ messages in thread
From: Marc-André Lureau @ 2016-09-07 14:23 UTC (permalink / raw)
  To: Markus Armbruster; +Cc: qemu-devel, Eric Blake

Hi

On Wed, Sep 7, 2016 at 5:40 PM Markus Armbruster <armbru@redhat.com> wrote:

> Markus Armbruster <armbru@redhat.com> writes:
>
> > Marc-André Lureau <marcandre.lureau@gmail.com> writes:
> >
> >> Hi
> >>
> >> On Tue, Sep 6, 2016 at 7:58 PM Markus Armbruster <armbru@redhat.com>
> wrote:
> >>
> >>> QAPI language design issues, copying Eric.
> >>>
> >>> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
> >>>
> >>> > Hi
> >>> >
> >>> > On Tue, Sep 6, 2016 at 5:00 PM Markus Armbruster <armbru@redhat.com>
> >>> wrote:
> [...]
> >>> >> Yet another option is to add 'ifdef' keys to the definitions
> >>> >> themselves, i.e.
> >>> >>
> >>> >>     { 'command': 'query-gic-capabilities', 'returns':
> ['GICCapability'],
> >>> >>       'ifdef': 'TARGET_ARM' }
> >>> >>
> >>> >
> >>> > That's the worst of all options imho, as it makes it hard to filter
> out
> >>> > unions/enums, ex:
> >>> >
> >>> >  @ -3446,8 +3466,10 @@
> >>> >                                         'testdev': 'ChardevCommon',
> >>> >                                         'stdio'  : 'ChardevStdio',
> >>> >                                         'console': 'ChardevCommon',
> >>> > +#ifdef CONFIG_SPICE
> >>> >                                         'spicevmc' :
> >>> 'ChardevSpiceChannel',
> >>> >                                         'spiceport' :
> 'ChardevSpicePort',
> >>> > +#endif
> >>> >                                         'vc'     : 'ChardevVC',
> >>> >                                         'ringbuf': 'ChardevRingbuf',
> >>>
> >>> Point taken.
> >>>
> >>> Fixable the same way as always when a definition needs to grow
> >>> additional properties, but the syntax only provides a single value:
> make
> >>> that single value an object, and the old non-object value syntactic
> >>> sugar for the equivalent object value.  We've previously discussed this
> >>> technique in the context of giving command arguments default values.
> >>> I'm not saying this is what we should do here, only pointing out it's
> >>> possible.
> >>>
> >>
> >> Ok, but let's find something, if possible simple and convenient, no?
> >
> > I agree it needs to be simple, both the interface (QAPI language) and
> > the implementation.  However, I don't like "first past the post".  I
> > prefer to explore the design space a bit.
> >
> > So let me explore the "add 'ifdef' keys to definitions" corner of the
> > QAPI language design space.
> >
> > Easily done for top-level definitions, because they're all JSON objects.
> > Could even add it to the include directive if we wanted to.
> >
> > Less easily done for enumeration, struct, union and alternate members.
> > Note that command and event arguments specified inline are a special
> > case of struct members.
> >
> > The "can't specify additional stuff for struct members" problem isn't
> > new.  We hacked around it to specify "optional": encode it into the
> > member name.  Doesn't scale.  We need to solve the problem to be able to
> > specify default values, and we already decided how: have an JSON object
> > instead of a mere JSON string, make the string syntax sugar for {
> > 'type': STRING }.  See commit 6b5abc7 and the discussion in qemu-devel
> > leading up to it.  For consistency, we'll do it for union and alternate
> > members, too.
> >
> > That leaves just enumeration members.  The same technique applies.
> >
> > If I remember correctly, we only need conditional commands right now, to
> > avoid regressing query-commands.  The more complicated member work could
> > be done later.
>
> To gauge whether this idea is practical, I implemented key 'if' for
> commands.  It's just a sketch, and has a number of issues, which I
> marked FIXME.
>

Tbh, I think it is scratching the surface doing it only for commands.
Furthermore, it's pushing the burdden of conditional generation at a
different level, at the C level, which is probably not so nice if some day
it targets a different language.

If the main issue is using #\w* for preprocessing, we could use something
else like #! (oh no someone else did it, there are many ex where comments
are not just comments), or .\w* (.ifdef / .endif etc).

As you said, the main goal of this patch was to disable commands from the
qapi-dispatch that are currently disabled with the middle mode.

Do you think it would be reasonable to accept this simple approach for now,
until we find a better more suitable solution? After all, this is internal,
we can iterate. Or should we drop this patch for this series? I don't like
seeing it delayed by weeks...


>
> I ported qmp-commands.hx's #if to qapi-schema.json.  The TARGET_FOO are
> poisoned, so I commented them out.  There's a CONFIG_SPICE left, which
> will do for testing.
>
> I also turned key 'gen': false into 'if': false.  Possibly a bad idea.
>
> Anyway, diffstat isn't bad:
>
>  docs/qapi-code-gen.txt                     | 14 ++++++-----
>  qapi-schema.json                           | 15 ++++++++---
>  qapi/introspect.json                       |  2 +-
>  scripts/qapi-commands.py                   | 12 +++++++--
>  scripts/qapi-introspect.py                 | 22 ++++++++++------
>  scripts/qapi.py                            | 40
> ++++++++++++++++++++++--------
>  tests/qapi-schema/type-bypass-bad-gen.err  |  2 +-
>  tests/qapi-schema/type-bypass-bad-gen.json |  4 +--
>  8 files changed, 77 insertions(+), 34 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index de298dc..93e99d8 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -423,7 +423,7 @@ part of a Client JSON Protocol command.  The 'data'
> member is optional
>  and defaults to {} (an empty dictionary).  If present, it must be the
>  string name of a complex type, or a dictionary that declares an
>  anonymous type with the same semantics as a 'struct' expression, with
> -one exception noted below when 'gen' is used.
> +one exception noted below when 'if': false is used.
>
>  The 'returns' member describes what will appear in the "return" member
>  of a Client JSON Protocol reply on successful completion of a command.
> @@ -431,8 +431,8 @@ The member is optional from the command declaration;
> if absent, the
>  "return" member will be an empty dictionary.  If 'returns' is present,
>  it must be the string name of a complex or built-in type, a
>  one-element array containing the name of a complex or built-in type,
> -with one exception noted below when 'gen' is used.  Although it is
> -permitted to have the 'returns' member name a built-in type or an
> +with one exception noted below when 'if':false is used.  Although it
> +is permitted to have the 'returns' member name a built-in type or an
>  array of built-in types, any command that does this cannot be extended
>  to return additional information in the future; thus, new commands
>  should strongly consider returning a dictionary-based type or an array
> @@ -475,16 +475,18 @@ arguments for the user's function out of an input
> QDict, calls the
>  user's function, and if it succeeded, builds an output QObject from
>  its return value.
>
> +FIXME document 'if'
> +
>  In rare cases, QAPI cannot express a type-safe representation of a
>  corresponding Client JSON Protocol command.  You then have to suppress
> -generation of a marshalling function by including a key 'gen' with
> +generation of a marshalling function by including a key 'if' with
>  boolean value false, and instead write your own function.  Please try
>  to avoid adding new commands that rely on this, and instead use
>  type-safe unions.  For an example of this usage:
>
>   { 'command': 'netdev_add',
> -   'data': {'type': 'str', 'id': 'str'},
> -   'gen': false }
> +   'if': false,
> +   'data': {'type': 'str', 'id': 'str'} }
>
>  Normally, the QAPI schema is used to describe synchronous exchanges,
>  where a response is expected.  But in some cases, the action of a
> diff --git a/qapi-schema.json b/qapi-schema.json
> index c4f3674..ad0559e 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -1269,7 +1269,9 @@
>  #
>  # Since: 0.14.0
>  ##
> -{ 'command': 'query-spice', 'returns': 'SpiceInfo' }
> +{ 'command': 'query-spice',
> +  'if': 'defined(CONFIG_SPICE)',
> +  'returns': 'SpiceInfo' }
>
>  ##
>  # @BalloonInfo:
> @@ -2355,6 +2357,7 @@
>  # Since: 2.5
>  ##
>  { 'command': 'dump-skeys',
> +#  'if': 'defined(TARGET_S390X)',
>    'data': { 'filename': 'str' } }
>
>  ##
> @@ -2380,8 +2383,8 @@
>  #          If @type is not a valid network backend, DeviceNotFound
>  ##
>  { 'command': 'netdev_add',
> -  'data': {'type': 'str', 'id': 'str'},
> -  'gen': false }                # so we can get the additional arguments
> +  'if': false,                  # so we can get the additional arguments
> +  'data': {'type': 'str', 'id': 'str'} }
>
>  ##
>  # @netdev_del:
> @@ -4455,6 +4458,7 @@
>  # Since: 2.1
>  ##
>  { 'command': 'rtc-reset-reinjection' }
> +#  'if': 'defined(TARGET_I386)'
>
>  # Rocker ethernet network switch
>  { 'include': 'qapi/rocker.json' }
> @@ -4525,7 +4529,10 @@
>  #
>  # Since: 2.6
>  ##
> -{ 'command': 'query-gic-capabilities', 'returns': ['GICCapability'] }
> +{ 'command': 'query-gic-capabilities',
> +#  'if': 'defined(TARGET_ARM)',
> +  'returns': ['GICCapability']
> +}
>
>  ##
>  # CpuInstanceProperties
> diff --git a/qapi/introspect.json b/qapi/introspect.json
> index 3fd81fb..b8f421a 100644
> --- a/qapi/introspect.json
> +++ b/qapi/introspect.json
> @@ -46,7 +46,7 @@
>  ##
>  { 'command': 'query-qmp-schema',
>    'returns': [ 'SchemaInfo' ],
> -  'gen': false }                # just to simplify qmp_query_json()
> +  'if': false }                 # just to simplify qmp_query_json()
>
>  ##
>  # @SchemaMetaType
> diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
> index a06a2c4..f34e4cc 100644
> --- a/scripts/qapi-commands.py
> +++ b/scripts/qapi-commands.py
> @@ -215,9 +215,13 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
>          self._visited_ret_types = None
>
>      def visit_command(self, name, info, arg_type, ret_type,
> -                      gen, success_response, boxed):
> -        if not gen:
> +                      genif, success_response, boxed):
> +        if genif is False:
>              return
> +        pp_if = gen_pp_if(genif)
> +        pp_endif = gen_pp_endif(genif)
> +        self.decl += pp_if
> +        self.defn += pp_if      # FIXME blank lines are off
>          self.decl += gen_command_decl(name, arg_type, boxed, ret_type)
>          if ret_type and ret_type not in self._visited_ret_types:
>              self._visited_ret_types.add(ret_type)
> @@ -226,7 +230,11 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
>              self.decl += gen_marshal_decl(name)
>          self.defn += gen_marshal(name, arg_type, boxed, ret_type)
>          if not middle_mode:
> +            self._regy += pp_if
>              self._regy += gen_register_command(name, success_response)
> +            self._regy += pp_endif
> +        self.decl += pp_endif
> +        self.defn += pp_endif
>
>
>  middle_mode = False
> diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
> index 541644e..0d8cec7 100644
> --- a/scripts/qapi-introspect.py
> +++ b/scripts/qapi-introspect.py
> @@ -30,8 +30,6 @@ def to_json(obj, level=0):
>          ret = '{' + ', '.join(elts) + '}'
>      else:
>          assert False                # not implemented
> -    if level == 1:
> -        ret = '\n' + ret
>      return ret
>
>
> @@ -69,8 +67,15 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor):
>  extern const char %(c_name)s[];
>  ''',
>                            c_name=c_name(name))
> -        lines = to_json(jsons).split('\n')
> -        c_string = '\n    '.join([to_c_string(line) for line in lines])
> +        c_string = '"["'
> +        for i in jsons:
> +            js, genif = i
> +            # FIXME blank lines are off
> +            c_string += gen_pp_if(genif or True)
> +            c_string += '\n    ' + to_c_string(to_json(js) + ', ')
> +            c_string += gen_pp_endif(genif or True)
> +        # FIXME trailing comma (JSON sucks)
> +        c_string += '\n    "]"'
>          self.defn = mcgen('''
>  const char %(c_name)s[] = %(c_string)s;
>  ''',
> @@ -111,12 +116,12 @@ const char %(c_name)s[] = %(c_string)s;
>              return '[' + self._use_type(typ.element_type) + ']'
>          return self._name(typ.name)
>
> -    def _gen_json(self, name, mtype, obj):
> +    def _gen_json(self, name, mtype, obj, genif=True):
>          if mtype not in ('command', 'event', 'builtin', 'array'):
>              name = self._name(name)
>          obj['name'] = name
>          obj['meta-type'] = mtype
> -        self._jsons.append(obj)
> +        self._jsons.append((obj, genif))
>
>      def _gen_member(self, member):
>          ret = {'name': member.name, 'type': self._use_type(member.type)}
> @@ -154,12 +159,13 @@ const char %(c_name)s[] = %(c_string)s;
>                                      for m in variants.variants]})
>
>      def visit_command(self, name, info, arg_type, ret_type,
> -                      gen, success_response, boxed):
> +                      genif, success_response, boxed):
>          arg_type = arg_type or self._schema.the_empty_object_type
>          ret_type = ret_type or self._schema.the_empty_object_type
>          self._gen_json(name, 'command',
>                         {'arg-type': self._use_type(arg_type),
> -                        'ret-type': self._use_type(ret_type)})
> +                        'ret-type': self._use_type(ret_type)},
> +                       genif)
>
>      def visit_event(self, name, info, arg_type, boxed):
>          arg_type = arg_type or self._schema.the_empty_object_type
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 21bc32f..6c0cf9f 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -698,7 +698,13 @@ def check_keys(expr_elem, meta, required,
> optional=[]):
>              raise QAPIExprError(info,
>                                  "Unknown key '%s' in %s '%s'"
>                                  % (key, meta, name))
> -        if (key == 'gen' or key == 'success-response') and value is not
> False:
> +        if (key == 'if'
> +            and value is not False and not isinstance(value, str)):
> +            # FIXME update error message
> +            raise QAPIExprError(info,
> +                                "'%s' of %s '%s' should only use false
> value"
> +                                % (key, meta, name))
> +        if (key == 'success-response') and value is not False:
>              raise QAPIExprError(info,
>                                  "'%s' of %s '%s' should only use false
> value"
>                                  % (key, meta, name))
> @@ -737,7 +743,7 @@ def check_exprs(exprs):
>              add_struct(expr, info)
>          elif 'command' in expr:
>              check_keys(expr_elem, 'command', [],
> -                       ['data', 'returns', 'gen', 'success-response',
> 'boxed'])
> +                       ['data', 'returns', 'if', 'success-response',
> 'boxed'])
>              add_name(expr['command'], info, 'command')
>          elif 'event' in expr:
>              check_keys(expr_elem, 'event', [], ['data', 'boxed'])
> @@ -838,7 +844,7 @@ class QAPISchemaVisitor(object):
>          pass
>
>      def visit_command(self, name, info, arg_type, ret_type,
> -                      gen, success_response, boxed):
> +                      genif, success_response, boxed):
>          pass
>
>      def visit_event(self, name, info, arg_type, boxed):
> @@ -1180,8 +1186,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
>
>
>  class QAPISchemaCommand(QAPISchemaEntity):
> -    def __init__(self, name, info, arg_type, ret_type, gen,
> success_response,
> -                 boxed):
> +    def __init__(self, name, info, arg_type, ret_type,
> +                 genif, success_response, boxed):
>          QAPISchemaEntity.__init__(self, name, info)
>          assert not arg_type or isinstance(arg_type, str)
>          assert not ret_type or isinstance(ret_type, str)
> @@ -1189,7 +1195,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
>          self.arg_type = None
>          self._ret_type_name = ret_type
>          self.ret_type = None
> -        self.gen = gen
> +        self.genif = genif
>          self.success_response = success_response
>          self.boxed = boxed
>
> @@ -1216,7 +1222,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
>      def visit(self, visitor):
>          visitor.visit_command(self.name, self.info,
>                                self.arg_type, self.ret_type,
> -                              self.gen, self.success_response, self.boxed)
> +                              self.genif, self.success_response,
> self.boxed)
>
>
>  class QAPISchemaEvent(QAPISchemaEntity):
> @@ -1419,17 +1425,20 @@ class QAPISchema(object):
>          name = expr['command']
>          data = expr.get('data')
>          rets = expr.get('returns')
> -        gen = expr.get('gen', True)
> +        genif = expr.get('if', True)
>          success_response = expr.get('success-response', True)
>          boxed = expr.get('boxed', False)
>          if isinstance(data, OrderedDict):
> +            # TODO apply genif to the implicit object type
>              data = self._make_implicit_object_type(
>                  name, info, 'arg', self._make_members(data, info))
>          if isinstance(rets, list):
> +            # TODO apply genif to the implicit array type
> +            # TODO disjunction of all the genif
>              assert len(rets) == 1
>              rets = self._make_array_type(rets[0], info)
> -        self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
> -                                           success_response, boxed))
> +        self._def_entity(QAPISchemaCommand(name, info, data, rets,
> +                                           genif, success_response,
> boxed))
>
>      def _def_event(self, expr, info):
>          name = expr['event']
> @@ -1704,6 +1713,17 @@ def gen_params(arg_type, boxed, extra):
>      return ret
>
>
> +def gen_pp_if(cond):
> +    if cond is True:
> +        return ''
> +    return '\n#if ' + cond + '\n'
> +
> +
> +def gen_pp_endif(cond):
> +    if cond is True:
> +        return ''
> +    return '\n#endif  /* ' + cond + ' */\n'
> +
>  #
>  # Common command line parsing
>  #
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.err
> b/tests/qapi-schema/type-bypass-bad-gen.err
> index a83c3c6..cca25f1 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.err
> +++ b/tests/qapi-schema/type-bypass-bad-gen.err
> @@ -1 +1 @@
> -tests/qapi-schema/type-bypass-bad-gen.json:2: 'gen' of command 'foo'
> should only use false value
> +tests/qapi-schema/type-bypass-bad-gen.json:2: 'if' of command 'foo'
> should only use false value
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.json
> b/tests/qapi-schema/type-bypass-bad-gen.json
> index e8dec34..637b11f 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.json
> +++ b/tests/qapi-schema/type-bypass-bad-gen.json
> @@ -1,2 +1,2 @@
> -# 'gen' should only appear with value false
> -{ 'command': 'foo', 'gen': 'whatever' }
> +# 'if' should only appear with value false FIXME or str
> +{ 'command': 'foo', 'if': null }
>
-- 
Marc-André Lureau

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional
  2016-09-07 13:40             ` Markus Armbruster
  2016-09-07 14:23               ` Marc-André Lureau
@ 2016-09-07 18:49               ` Eric Blake
  1 sibling, 0 replies; 31+ messages in thread
From: Eric Blake @ 2016-09-07 18:49 UTC (permalink / raw)
  To: Markus Armbruster, Marc-André Lureau; +Cc: qemu-devel

[-- Attachment #1: Type: text/plain, Size: 5964 bytes --]

On 09/07/2016 08:40 AM, Markus Armbruster wrote:
>>>>>> Yet another option is to add 'ifdef' keys to the definitions
>>>>>> themselves, i.e.
>>>>>>
>>>>>>     { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
>>>>>>       'ifdef': 'TARGET_ARM' }

Seems like it might be useful; could even be expanded to:

{ 'command': 'query-arm-x86', 'returns': ['Foo'],
  'ifdef': [ 'TARGET_ARM', 'TARGET_X86' ] }

unless you are okay with it instead being:

{ 'command': 'query-arm-x86', 'returns': ['Foo'],
  'ifdef': 'TARGET_ARM || TARGET_X86' }

Either way, we need to make sure that whatever we put here is easy to
translate into the appropriate generated code; and keeping in mind that
while '#if A || B' makes sense to the preprocessor, '#ifdef A || B' does
not.

>>>> Fixable the same way as always when a definition needs to grow
>>>> additional properties, but the syntax only provides a single value: make
>>>> that single value an object, and the old non-object value syntactic
>>>> sugar for the equivalent object value.  We've previously discussed this
>>>> technique in the context of giving command arguments default values.
>>>> I'm not saying this is what we should do here, only pointing out it's
>>>> possible.
>>>>
>>>
>>> Ok, but let's find something, if possible simple and convenient, no?
>>
>> I agree it needs to be simple, both the interface (QAPI language) and
>> the implementation.  However, I don't like "first past the post".  I
>> prefer to explore the design space a bit.

Another nice thing about expanding it by changing 'name':'typestr' to be
sugar for 'name':{'type':'typestr'} is that QMP introspection is already
prepared to expose a dictionary of attributes (type being one, and
ifdef'ness being another) per member, so we already would have a nice
mapping between qapi JSON files (a member name tied to a dictionary) and
the generated output.

>>
>> So let me explore the "add 'ifdef' keys to definitions" corner of the
>> QAPI language design space.
>>
>> Easily done for top-level definitions, because they're all JSON objects.
>> Could even add it to the include directive if we wanted to.
>>
>> Less easily done for enumeration, struct, union and alternate members.
>> Note that command and event arguments specified inline are a special
>> case of struct members.
>>

and ideally, whatever we can do for a named struct, we can also do for
an anonymous inlined struct for the command and event arguments.

> 
> I ported qmp-commands.hx's #if to qapi-schema.json.  The TARGET_FOO are
> poisoned, so I commented them out.  There's a CONFIG_SPICE left, which
> will do for testing.

Obviously, we'd have to find a way to work around the poisoned ifdef
names, but the idea makes some sense to me.

> 
> I also turned key 'gen': false into 'if': false.  Possibly a bad idea.

Maybe, maybe not. We have so few that it's easy to convert them all in
one go, at which point you are just giving directives to the code
generator: if the 'if' key is present, it controls what gets omitted
(either the omitted code is gated by #if (or #ifdef) of the gate string,
or 'if':false means the generator omits the code altogether).

> @@ -475,16 +475,18 @@ arguments for the user's function out of an input QDict, calls the
>  user's function, and if it succeeded, builds an output QObject from
>  its return value.
>  
> +FIXME document 'if'
> +

And in particular, what the string must look like (will it be fed to
generated #if or to generated #ifdef?  What if you have more than one
conditional?)

> +++ b/qapi-schema.json
> @@ -1269,7 +1269,9 @@
>  #
>  # Since: 0.14.0
>  ##
> -{ 'command': 'query-spice', 'returns': 'SpiceInfo' }
> +{ 'command': 'query-spice',
> +  'if': 'defined(CONFIG_SPICE)',
> +  'returns': 'SpiceInfo' }

So the idea here is that qemu built without SPICE support would
completely lack the query-spice command.  This is currently detectable
during 'query-commands' but NOT detectable during 'query-qmp-schema';
but after the patch, we could begin to make the introspection likewise
hide the unused command.  Sounds useful.

>  
>  ##
>  # @BalloonInfo:
> @@ -2355,6 +2357,7 @@
>  # Since: 2.5
>  ##
>  { 'command': 'dump-skeys',
> +#  'if': 'defined(TARGET_S390X)',
>    'data': { 'filename': 'str' } }

And here you ran into the poisoned variable problem.  Obviously
something to be solved if we like the overall idea.

> +++ b/scripts/qapi-introspect.py
> @@ -30,8 +30,6 @@ def to_json(obj, level=0):
>          ret = '{' + ', '.join(elts) + '}'
>      else:
>          assert False                # not implemented
> -    if level == 1:
> -        ret = '\n' + ret
>      return ret
>  
>  
> @@ -69,8 +67,15 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor):
>  extern const char %(c_name)s[];
>  ''',
>                            c_name=c_name(name))
> -        lines = to_json(jsons).split('\n')
> -        c_string = '\n    '.join([to_c_string(line) for line in lines])
> +        c_string = '"["'
> +        for i in jsons:
> +            js, genif = i
> +            # FIXME blank lines are off
> +            c_string += gen_pp_if(genif or True)
> +            c_string += '\n    ' + to_c_string(to_json(js) + ', ')
> +            c_string += gen_pp_endif(genif or True)

And this change to introspection output shows the true power - by having
the conditional expression be part of the .json QAPI file, we can now
reflect the conditions through ALL parts of the generated code, instead
of having discrepancies between query-commands vs. query-qmp-schema.

At any rate, for a quick day's hack, I like this approach for feeling
like it fits in with the QAPI language, rather than being bolted on as
additional non-JSON syntax.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt
  2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt Marc-André Lureau
@ 2016-09-09 12:43   ` Markus Armbruster
  0 siblings, 0 replies; 31+ messages in thread
From: Markus Armbruster @ 2016-09-09 12:43 UTC (permalink / raw)
  To: Marc-André Lureau; +Cc: qemu-devel

s,doc/,docs/ in subject.  Can touch up on commit.

^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2016-09-09 12:43 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-17 16:57 [Qemu-devel] [PATCH v5 00/20] qapi: remove the 'middle' mode Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 01/20] tests: do qmp introspect validation per target Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 02/20] qapi.py: add a simple #ifdef conditional Marc-André Lureau
2016-09-06 12:35   ` Markus Armbruster
2016-09-06 13:17     ` Marc-André Lureau
2016-09-06 15:58       ` Markus Armbruster
2016-09-06 16:44         ` Marc-André Lureau
2016-09-07  8:44           ` Markus Armbruster
2016-09-07 13:40             ` Markus Armbruster
2016-09-07 14:23               ` Marc-André Lureau
2016-09-07 18:49               ` Eric Blake
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 03/20] build-sys: make qemu qapi per-target Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 04/20] build-sys: use config headers to generate qapi Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 05/20] qapi: configure the schema Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 06/20] build-sys: define QEMU_VERSION_{MAJOR, MINOR, MICRO} Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 07/20] qapi-schema: use generated marshaller for 'qmp_capabilities' Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 08/20] qapi-schema: add 'device_add' Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 09/20] monitor: simplify invalid_qmp_mode() Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 10/20] monitor: register gen:false commands manually Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 11/20] qapi: export the marshallers Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 12/20] monitor: use qmp_find_command() (using generated qapi code) Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 13/20] monitor: implement 'qmp_query_commands' without qmp_cmds Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 14/20] monitor: remove mhandler.cmd_new Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 15/20] qapi: remove the "middle" mode Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 16/20] qapi: check invalid arguments on no-args commands Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 17/20] monitor: use qmp_dispatch() Marc-André Lureau
2016-08-17 17:04   ` Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 18/20] build-sys: remove qmp-commands-old.h Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 19/20] qmp-commands.hx: fix some styling Marc-André Lureau
2016-08-17 16:57 ` [Qemu-devel] [PATCH v5 20/20] Replace qmp-commands.hx by doc/qmp-commands.txt Marc-André Lureau
2016-09-09 12:43   ` Markus Armbruster

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.