* [PATCH v3 01/18] qapi: Rename (one) qjson.h to qobject-json.h
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 02/18] qapi: Improve use of qmp/types.h Eric Blake
` (17 subsequent siblings)
18 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel
Cc: armbru, famz, Kevin Wolf, Max Reitz, Chrysostomos Nanakos,
Jeff Cody, Paolo Bonzini, Alberto Garcia, Michael S. Tsirkin,
Jason Wang, Luiz Capitulino, Michael Roth, Christian Borntraeger,
Cornelia Huck, Alexander Graf, Richard Henderson, Gerd Hoffmann,
open list:Block layer core, open list:Overall
We have two different JSON visitors in the tree; and having both
named 'qjson.h' can cause include confusion. Rename the qapi
version.
Why did I pick that one? A later patch plans on deleting the
top-level qjson.c once we have a native JSON output visitor; we
could have renamed that one for less overall churn. On the other
hand, all of the QObject subtypes have their own qFOO.c file, but
qjson.c makes it sound like we have a QTYPE_JSON subclass of
QObject; the new name of qobject-json makes it obvious that the
file is used for conversions between QObject and JSON, and not a
QObject subtype.
Kill trailing whitespace in the renamed tests/check-qobject-json.c
to keep checkpatch.pl happy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
v3: rebase to master
v2: retitle, enhance commit message, rebase to master
---
include/qapi/qmp/{qjson.h => qobject-json.h} | 0
include/qapi/qmp/types.h | 2 +-
balloon.c | 2 +-
block.c | 2 +-
block/archipelago.c | 2 +-
block/nbd.c | 2 +-
block/quorum.c | 2 +-
blockjob.c | 2 +-
hw/core/qdev.c | 2 +-
hw/misc/pvpanic.c | 2 +-
hw/net/virtio-net.c | 2 +-
monitor.c | 2 +-
qapi/qmp-event.c | 2 +-
qemu-img.c | 2 +-
qga/main.c | 2 +-
qobject/{qjson.c => qobject-json.c} | 2 +-
target-s390x/kvm.c | 2 +-
tests/{check-qjson.c => check-qobject-json.c} | 14 +++++++-------
tests/libqtest.c | 2 +-
ui/spice-core.c | 2 +-
vl.c | 2 +-
MAINTAINERS | 2 +-
qobject/Makefile.objs | 3 ++-
tests/.gitignore | 2 +-
tests/Makefile | 8 ++++----
25 files changed, 34 insertions(+), 33 deletions(-)
rename include/qapi/qmp/{qjson.h => qobject-json.h} (100%)
rename qobject/{qjson.c => qobject-json.c} (99%)
rename tests/{check-qjson.c => check-qobject-json.c} (99%)
diff --git a/include/qapi/qmp/qjson.h b/include/qapi/qmp/qobject-json.h
similarity index 100%
rename from include/qapi/qmp/qjson.h
rename to include/qapi/qmp/qobject-json.h
diff --git a/include/qapi/qmp/types.h b/include/qapi/qmp/types.h
index 7782ec5..9109eda 100644
--- a/include/qapi/qmp/types.h
+++ b/include/qapi/qmp/types.h
@@ -20,6 +20,6 @@
#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#endif /* QEMU_OBJECTS_H */
diff --git a/balloon.c b/balloon.c
index f2ef50c..2a11689 100644
--- a/balloon.c
+++ b/balloon.c
@@ -32,7 +32,7 @@
#include "trace.h"
#include "qmp-commands.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
static QEMUBalloonEvent *balloon_event_fn;
static QEMUBalloonStatus *balloon_stat_fn;
diff --git a/block.c b/block.c
index d4939b4..6e86863 100644
--- a/block.c
+++ b/block.c
@@ -29,7 +29,7 @@
#include "qemu/module.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "sysemu/block-backend.h"
#include "sysemu/sysemu.h"
#include "qemu/notify.h"
diff --git a/block/archipelago.c b/block/archipelago.c
index b9f5e69..104f2f9 100644
--- a/block/archipelago.c
+++ b/block/archipelago.c
@@ -57,7 +57,7 @@
#include "qemu/thread.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/atomic.h"
#include <xseg/xseg.h>
diff --git a/block/nbd.c b/block/nbd.c
index f7ea3b3..d8899c4 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -33,7 +33,7 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qstring.h"
#include "qemu/cutils.h"
diff --git a/block/quorum.c b/block/quorum.c
index da15465..a0a053a 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -19,7 +19,7 @@
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qstring.h"
#include "qapi-event.h"
diff --git a/blockjob.c b/blockjob.c
index 9fc37ca..cafa576 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -31,7 +31,7 @@
#include "block/block_int.h"
#include "sysemu/block-backend.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/coroutine.h"
#include "qmp-commands.h"
#include "qemu/timer.h"
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index db41aa1..1806d8b 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -31,7 +31,7 @@
#include "sysemu/sysemu.h"
#include "qapi/qmp/qerror.h"
#include "qapi/visitor.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/error-report.h"
#include "hw/hotplug.h"
#include "hw/boards.h"
diff --git a/hw/misc/pvpanic.c b/hw/misc/pvpanic.c
index 0ac1e6a..0f2f38e 100644
--- a/hw/misc/pvpanic.c
+++ b/hw/misc/pvpanic.c
@@ -14,7 +14,7 @@
#include "qemu/osdep.h"
#include "qapi/qmp/qobject.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "sysemu/sysemu.h"
#include "qemu/log.h"
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 5798f87..a0ace28 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -22,7 +22,7 @@
#include "hw/virtio/virtio-net.h"
#include "net/vhost_net.h"
#include "hw/virtio/virtio-bus.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi-event.h"
#include "hw/virtio/virtio-access.h"
diff --git a/monitor.c b/monitor.c
index d1c1930..3db357c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -57,7 +57,7 @@
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
#include <qom/object_interfaces.h>
diff --git a/qapi/qmp-event.c b/qapi/qmp-event.c
index 8bba165..af96d61 100644
--- a/qapi/qmp-event.c
+++ b/qapi/qmp-event.c
@@ -16,7 +16,7 @@
#include "qemu-common.h"
#include "qapi/qmp-event.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
static QMPEventFuncEmit qmp_emit;
diff --git a/qemu-img.c b/qemu-img.c
index 46f2a6d..e976851 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -26,7 +26,7 @@
#include "qapi-visit.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/cutils.h"
#include "qemu/config-file.h"
#include "qemu/option.h"
diff --git a/qga/main.c b/qga/main.c
index c5527821..1c0315e 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -21,7 +21,7 @@
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qga/guest-agent-core.h"
#include "qemu/module.h"
#include "qapi/qmp/qerror.h"
diff --git a/qobject/qjson.c b/qobject/qobject-json.c
similarity index 99%
rename from qobject/qjson.c
rename to qobject/qobject-json.c
index ef160d2..bcc3f6e 100644
--- a/qobject/qjson.c
+++ b/qobject/qobject-json.c
@@ -15,7 +15,7 @@
#include "qapi/qmp/json-lexer.h"
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qbool.h"
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index e1859ca..361d248 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -36,7 +36,7 @@
#include "hw/hw.h"
#include "cpu.h"
#include "sysemu/device_tree.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "exec/gdbstub.h"
#include "exec/address-spaces.h"
#include "trace.h"
diff --git a/tests/check-qjson.c b/tests/check-qobject-json.c
similarity index 99%
rename from tests/check-qjson.c
rename to tests/check-qobject-json.c
index 99de6f5..dc721a8 100644
--- a/tests/check-qjson.c
+++ b/tests/check-qobject-json.c
@@ -19,7 +19,7 @@
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qfloat.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu-common.h"
@@ -64,7 +64,7 @@ static void escaped_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert_cmpstr(qstring_get_str(str), ==, test_cases[i].decoded);
@@ -99,7 +99,7 @@ static void simple_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -107,7 +107,7 @@ static void simple_string(void)
g_assert(strcmp(qstring_get_str(str), test_cases[i].encoded) == 0);
qobject_decref(obj);
-
+
QDECREF(str);
}
}
@@ -133,7 +133,7 @@ static void single_quote_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -881,7 +881,7 @@ static void vararg_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -1145,7 +1145,7 @@ static int compare_litqobj_to_qobj(LiteralQObject *lhs, QObject *rhs)
helper.index = 0;
helper.objs = lhs->value.qlist;
helper.result = 1;
-
+
qlist_iter(qobject_to_qlist(rhs), compare_helper, &helper);
return helper.result;
diff --git a/tests/libqtest.c b/tests/libqtest.c
index b12a9e4..2f42bc9 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -24,7 +24,7 @@
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#define MAX_IRQ 256
#define SOCKET_TIMEOUT 5
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 61db3c1..9eec388 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -33,7 +33,7 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/notify.h"
#include "migration/migration.h"
#include "hw/hw.h"
diff --git a/vl.c b/vl.c
index 5fd22cb..2d62382 100644
--- a/vl.c
+++ b/vl.c
@@ -88,7 +88,7 @@ int main(int argc, char **argv)
#include "audio/audio.h"
#include "migration/migration.h"
#include "sysemu/kvm.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/option.h"
#include "qemu/config-file.h"
#include "qemu-options.h"
diff --git a/MAINTAINERS b/MAINTAINERS
index 81e7fac..c56b657 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1190,7 +1190,7 @@ X: include/qapi/qmp/dispatch.h
F: tests/check-qdict.c
F: tests/check-qfloat.c
F: tests/check-qint.c
-F: tests/check-qjson.c
+F: tests/check-qobject-json.c
F: tests/check-qlist.c
F: tests/check-qstring.c
T: git git://repo.or.cz/qemu/qmp-unstable.git queue/qmp
diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs
index bed5508..16a48ec 100644
--- a/qobject/Makefile.objs
+++ b/qobject/Makefile.objs
@@ -1,2 +1,3 @@
util-obj-y = qnull.o qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o
-util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
+util-obj-y += qobject-json.o qobject.o
+util-obj-y += json-lexer.o json-streamer.o json-parser.o
diff --git a/tests/.gitignore b/tests/.gitignore
index a06a8ba..2f8c7ea 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -1,9 +1,9 @@
check-qdict
check-qfloat
check-qint
-check-qjson
check-qlist
check-qnull
+check-qobject-json
check-qstring
check-qom-interface
check-qom-proplist
diff --git a/tests/Makefile b/tests/Makefile
index 9dddde6..f71ed1c 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -18,8 +18,8 @@ check-unit-y += tests/check-qlist$(EXESUF)
gcov-files-check-qlist-y = qobject/qlist.c
check-unit-y += tests/check-qnull$(EXESUF)
gcov-files-check-qnull-y = qobject/qnull.c
-check-unit-y += tests/check-qjson$(EXESUF)
-gcov-files-check-qjson-y = qobject/qjson.c
+check-unit-y += tests/check-qobject-json$(EXESUF)
+gcov-files-check-qobject-json-y = qobject/qobject-json.c
check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
@@ -385,7 +385,7 @@ GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \
- tests/check-qjson.o \
+ tests/check-qobject-json.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
@@ -414,7 +414,7 @@ tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y)
tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y)
tests/check-qfloat$(EXESUF): tests/check-qfloat.o $(test-util-obj-y)
tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y)
-tests/check-qjson$(EXESUF): tests/check-qjson.o $(test-util-obj-y)
+tests/check-qobject-json$(EXESUF): tests/check-qobject-json.o $(test-util-obj-y)
tests/check-qom-interface$(EXESUF): tests/check-qom-interface.o $(test-qom-obj-y)
tests/check-qom-proplist$(EXESUF): tests/check-qom-proplist.o $(test-qom-obj-y)
tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(test-block-obj-y)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* [Qemu-devel] [PATCH v3 01/18] qapi: Rename (one) qjson.h to qobject-json.h
@ 2016-04-29 4:23 ` Eric Blake
0 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel
Cc: armbru, famz, Kevin Wolf, Max Reitz, Chrysostomos Nanakos,
Jeff Cody, Paolo Bonzini, Alberto Garcia, Michael S. Tsirkin,
Jason Wang, Luiz Capitulino, Michael Roth, Christian Borntraeger,
Cornelia Huck, Alexander Graf, Richard Henderson, Gerd Hoffmann,
open list:Block layer core, open list:Overall
We have two different JSON visitors in the tree; and having both
named 'qjson.h' can cause include confusion. Rename the qapi
version.
Why did I pick that one? A later patch plans on deleting the
top-level qjson.c once we have a native JSON output visitor; we
could have renamed that one for less overall churn. On the other
hand, all of the QObject subtypes have their own qFOO.c file, but
qjson.c makes it sound like we have a QTYPE_JSON subclass of
QObject; the new name of qobject-json makes it obvious that the
file is used for conversions between QObject and JSON, and not a
QObject subtype.
Kill trailing whitespace in the renamed tests/check-qobject-json.c
to keep checkpatch.pl happy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
v3: rebase to master
v2: retitle, enhance commit message, rebase to master
---
include/qapi/qmp/{qjson.h => qobject-json.h} | 0
include/qapi/qmp/types.h | 2 +-
balloon.c | 2 +-
block.c | 2 +-
block/archipelago.c | 2 +-
block/nbd.c | 2 +-
block/quorum.c | 2 +-
blockjob.c | 2 +-
hw/core/qdev.c | 2 +-
hw/misc/pvpanic.c | 2 +-
hw/net/virtio-net.c | 2 +-
monitor.c | 2 +-
qapi/qmp-event.c | 2 +-
qemu-img.c | 2 +-
qga/main.c | 2 +-
qobject/{qjson.c => qobject-json.c} | 2 +-
target-s390x/kvm.c | 2 +-
tests/{check-qjson.c => check-qobject-json.c} | 14 +++++++-------
tests/libqtest.c | 2 +-
ui/spice-core.c | 2 +-
vl.c | 2 +-
MAINTAINERS | 2 +-
qobject/Makefile.objs | 3 ++-
tests/.gitignore | 2 +-
tests/Makefile | 8 ++++----
25 files changed, 34 insertions(+), 33 deletions(-)
rename include/qapi/qmp/{qjson.h => qobject-json.h} (100%)
rename qobject/{qjson.c => qobject-json.c} (99%)
rename tests/{check-qjson.c => check-qobject-json.c} (99%)
diff --git a/include/qapi/qmp/qjson.h b/include/qapi/qmp/qobject-json.h
similarity index 100%
rename from include/qapi/qmp/qjson.h
rename to include/qapi/qmp/qobject-json.h
diff --git a/include/qapi/qmp/types.h b/include/qapi/qmp/types.h
index 7782ec5..9109eda 100644
--- a/include/qapi/qmp/types.h
+++ b/include/qapi/qmp/types.h
@@ -20,6 +20,6 @@
#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#endif /* QEMU_OBJECTS_H */
diff --git a/balloon.c b/balloon.c
index f2ef50c..2a11689 100644
--- a/balloon.c
+++ b/balloon.c
@@ -32,7 +32,7 @@
#include "trace.h"
#include "qmp-commands.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
static QEMUBalloonEvent *balloon_event_fn;
static QEMUBalloonStatus *balloon_stat_fn;
diff --git a/block.c b/block.c
index d4939b4..6e86863 100644
--- a/block.c
+++ b/block.c
@@ -29,7 +29,7 @@
#include "qemu/module.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "sysemu/block-backend.h"
#include "sysemu/sysemu.h"
#include "qemu/notify.h"
diff --git a/block/archipelago.c b/block/archipelago.c
index b9f5e69..104f2f9 100644
--- a/block/archipelago.c
+++ b/block/archipelago.c
@@ -57,7 +57,7 @@
#include "qemu/thread.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/atomic.h"
#include <xseg/xseg.h>
diff --git a/block/nbd.c b/block/nbd.c
index f7ea3b3..d8899c4 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -33,7 +33,7 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qstring.h"
#include "qemu/cutils.h"
diff --git a/block/quorum.c b/block/quorum.c
index da15465..a0a053a 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -19,7 +19,7 @@
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qstring.h"
#include "qapi-event.h"
diff --git a/blockjob.c b/blockjob.c
index 9fc37ca..cafa576 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -31,7 +31,7 @@
#include "block/block_int.h"
#include "sysemu/block-backend.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/coroutine.h"
#include "qmp-commands.h"
#include "qemu/timer.h"
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index db41aa1..1806d8b 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -31,7 +31,7 @@
#include "sysemu/sysemu.h"
#include "qapi/qmp/qerror.h"
#include "qapi/visitor.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/error-report.h"
#include "hw/hotplug.h"
#include "hw/boards.h"
diff --git a/hw/misc/pvpanic.c b/hw/misc/pvpanic.c
index 0ac1e6a..0f2f38e 100644
--- a/hw/misc/pvpanic.c
+++ b/hw/misc/pvpanic.c
@@ -14,7 +14,7 @@
#include "qemu/osdep.h"
#include "qapi/qmp/qobject.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "sysemu/sysemu.h"
#include "qemu/log.h"
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 5798f87..a0ace28 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -22,7 +22,7 @@
#include "hw/virtio/virtio-net.h"
#include "net/vhost_net.h"
#include "hw/virtio/virtio-bus.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi-event.h"
#include "hw/virtio/virtio-access.h"
diff --git a/monitor.c b/monitor.c
index d1c1930..3db357c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -57,7 +57,7 @@
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
#include <qom/object_interfaces.h>
diff --git a/qapi/qmp-event.c b/qapi/qmp-event.c
index 8bba165..af96d61 100644
--- a/qapi/qmp-event.c
+++ b/qapi/qmp-event.c
@@ -16,7 +16,7 @@
#include "qemu-common.h"
#include "qapi/qmp-event.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
static QMPEventFuncEmit qmp_emit;
diff --git a/qemu-img.c b/qemu-img.c
index 46f2a6d..e976851 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -26,7 +26,7 @@
#include "qapi-visit.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/cutils.h"
#include "qemu/config-file.h"
#include "qemu/option.h"
diff --git a/qga/main.c b/qga/main.c
index c5527821..1c0315e 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -21,7 +21,7 @@
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qga/guest-agent-core.h"
#include "qemu/module.h"
#include "qapi/qmp/qerror.h"
diff --git a/qobject/qjson.c b/qobject/qobject-json.c
similarity index 99%
rename from qobject/qjson.c
rename to qobject/qobject-json.c
index ef160d2..bcc3f6e 100644
--- a/qobject/qjson.c
+++ b/qobject/qobject-json.c
@@ -15,7 +15,7 @@
#include "qapi/qmp/json-lexer.h"
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qbool.h"
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index e1859ca..361d248 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -36,7 +36,7 @@
#include "hw/hw.h"
#include "cpu.h"
#include "sysemu/device_tree.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "exec/gdbstub.h"
#include "exec/address-spaces.h"
#include "trace.h"
diff --git a/tests/check-qjson.c b/tests/check-qobject-json.c
similarity index 99%
rename from tests/check-qjson.c
rename to tests/check-qobject-json.c
index 99de6f5..dc721a8 100644
--- a/tests/check-qjson.c
+++ b/tests/check-qobject-json.c
@@ -19,7 +19,7 @@
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qfloat.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu-common.h"
@@ -64,7 +64,7 @@ static void escaped_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert_cmpstr(qstring_get_str(str), ==, test_cases[i].decoded);
@@ -99,7 +99,7 @@ static void simple_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -107,7 +107,7 @@ static void simple_string(void)
g_assert(strcmp(qstring_get_str(str), test_cases[i].encoded) == 0);
qobject_decref(obj);
-
+
QDECREF(str);
}
}
@@ -133,7 +133,7 @@ static void single_quote_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -881,7 +881,7 @@ static void vararg_string(void)
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QSTRING);
-
+
str = qobject_to_qstring(obj);
g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0);
@@ -1145,7 +1145,7 @@ static int compare_litqobj_to_qobj(LiteralQObject *lhs, QObject *rhs)
helper.index = 0;
helper.objs = lhs->value.qlist;
helper.result = 1;
-
+
qlist_iter(qobject_to_qlist(rhs), compare_helper, &helper);
return helper.result;
diff --git a/tests/libqtest.c b/tests/libqtest.c
index b12a9e4..2f42bc9 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -24,7 +24,7 @@
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#define MAX_IRQ 256
#define SOCKET_TIMEOUT 5
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 61db3c1..9eec388 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -33,7 +33,7 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/notify.h"
#include "migration/migration.h"
#include "hw/hw.h"
diff --git a/vl.c b/vl.c
index 5fd22cb..2d62382 100644
--- a/vl.c
+++ b/vl.c
@@ -88,7 +88,7 @@ int main(int argc, char **argv)
#include "audio/audio.h"
#include "migration/migration.h"
#include "sysemu/kvm.h"
-#include "qapi/qmp/qjson.h"
+#include "qapi/qmp/qobject-json.h"
#include "qemu/option.h"
#include "qemu/config-file.h"
#include "qemu-options.h"
diff --git a/MAINTAINERS b/MAINTAINERS
index 81e7fac..c56b657 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1190,7 +1190,7 @@ X: include/qapi/qmp/dispatch.h
F: tests/check-qdict.c
F: tests/check-qfloat.c
F: tests/check-qint.c
-F: tests/check-qjson.c
+F: tests/check-qobject-json.c
F: tests/check-qlist.c
F: tests/check-qstring.c
T: git git://repo.or.cz/qemu/qmp-unstable.git queue/qmp
diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs
index bed5508..16a48ec 100644
--- a/qobject/Makefile.objs
+++ b/qobject/Makefile.objs
@@ -1,2 +1,3 @@
util-obj-y = qnull.o qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o
-util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
+util-obj-y += qobject-json.o qobject.o
+util-obj-y += json-lexer.o json-streamer.o json-parser.o
diff --git a/tests/.gitignore b/tests/.gitignore
index a06a8ba..2f8c7ea 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -1,9 +1,9 @@
check-qdict
check-qfloat
check-qint
-check-qjson
check-qlist
check-qnull
+check-qobject-json
check-qstring
check-qom-interface
check-qom-proplist
diff --git a/tests/Makefile b/tests/Makefile
index 9dddde6..f71ed1c 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -18,8 +18,8 @@ check-unit-y += tests/check-qlist$(EXESUF)
gcov-files-check-qlist-y = qobject/qlist.c
check-unit-y += tests/check-qnull$(EXESUF)
gcov-files-check-qnull-y = qobject/qnull.c
-check-unit-y += tests/check-qjson$(EXESUF)
-gcov-files-check-qjson-y = qobject/qjson.c
+check-unit-y += tests/check-qobject-json$(EXESUF)
+gcov-files-check-qobject-json-y = qobject/qobject-json.c
check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
@@ -385,7 +385,7 @@ GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \
- tests/check-qjson.o \
+ tests/check-qobject-json.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
@@ -414,7 +414,7 @@ tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y)
tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y)
tests/check-qfloat$(EXESUF): tests/check-qfloat.o $(test-util-obj-y)
tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y)
-tests/check-qjson$(EXESUF): tests/check-qjson.o $(test-util-obj-y)
+tests/check-qobject-json$(EXESUF): tests/check-qobject-json.o $(test-util-obj-y)
tests/check-qom-interface$(EXESUF): tests/check-qom-interface.o $(test-qom-obj-y)
tests/check-qom-proplist$(EXESUF): tests/check-qom-proplist.o $(test-qom-obj-y)
tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(test-block-obj-y)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 02/18] qapi: Improve use of qmp/types.h
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
2016-04-29 4:23 ` [Qemu-devel] " Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 11:46 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping Eric Blake
` (16 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel
Cc: armbru, famz, Michael S. Tsirkin, Luiz Capitulino, Michael Roth
'qobject-json.h' is not a QObject subtype; include this file
directly in .c files that are using it, rather than abusing
qmp/types.h for that purpose.
Meanwhile, for files that include a list of individual QObject
subtypes, it's easier to just use qmp/types.h for that purpose.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
v3: no change
v2: no change
---
include/qapi/qmp/types.h | 1 -
hw/pci/pcie_aer.c | 1 +
monitor.c | 6 +-----
qapi/qmp-dispatch.c | 1 +
qobject/json-parser.c | 7 +------
qobject/qobject-json.c | 6 +-----
qobject/qobject.c | 7 +------
tests/check-qobject-json.c | 7 +------
tests/test-qmp-input-strict.c | 1 +
tests/test-qmp-input-visitor.c | 1 +
tests/test-qmp-output-visitor.c | 1 +
tests/test-visitor-serialization.c | 1 +
12 files changed, 11 insertions(+), 29 deletions(-)
diff --git a/include/qapi/qmp/types.h b/include/qapi/qmp/types.h
index 9109eda..f21ecf4 100644
--- a/include/qapi/qmp/types.h
+++ b/include/qapi/qmp/types.h
@@ -20,6 +20,5 @@
#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qobject-json.h"
#endif /* QEMU_OBJECTS_H */
diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
index e2d4e68..63163a6 100644
--- a/hw/pci/pcie_aer.c
+++ b/hw/pci/pcie_aer.c
@@ -21,6 +21,7 @@
#include "qemu/osdep.h"
#include "sysemu/sysemu.h"
#include "qapi/qmp/types.h"
+#include "qapi/qmp/qobject-json.h"
#include "monitor/monitor.h"
#include "hw/pci/pci_bridge.h"
#include "hw/pci/pcie.h"
diff --git a/monitor.c b/monitor.c
index 3db357c..4393151 100644
--- a/monitor.c
+++ b/monitor.c
@@ -52,12 +52,8 @@
#include "qemu/acl.h"
#include "sysemu/tpm.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qfloat.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qobject-json.h"
+#include "qapi/qmp/types.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
#include <qom/object_interfaces.h>
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 08faf85..1d739cb 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -16,6 +16,7 @@
#include "qapi/qmp/types.h"
#include "qapi/qmp/dispatch.h"
#include "qapi/qmp/json-parser.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi-types.h"
#include "qapi/qmp/qerror.h"
diff --git a/qobject/json-parser.c b/qobject/json-parser.c
index 67ed727..c18e48a 100644
--- a/qobject/json-parser.c
+++ b/qobject/json-parser.c
@@ -14,12 +14,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "qemu-common.h"
-#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qfloat.h"
-#include "qapi/qmp/qbool.h"
+#include "qapi/qmp/types.h"
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-lexer.h"
#include "qapi/qmp/json-streamer.h"
diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
index bcc3f6e..24e7d80 100644
--- a/qobject/qobject-json.c
+++ b/qobject/qobject-json.c
@@ -16,12 +16,8 @@
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/qobject-json.h"
-#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qfloat.h"
-#include "qapi/qmp/qdict.h"
#include "qemu/unicode.h"
+#include "qapi/qmp/types.h"
typedef struct JSONParsingState
{
diff --git a/qobject/qobject.c b/qobject/qobject.c
index cd41fb9..fe4fa10 100644
--- a/qobject/qobject.c
+++ b/qobject/qobject.c
@@ -9,12 +9,7 @@
#include "qemu/osdep.h"
#include "qemu-common.h"
-#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qfloat.h"
-#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qstring.h"
+#include "qapi/qmp/types.h"
static void (*qdestroy[QTYPE__MAX])(QObject *) = {
[QTYPE_NONE] = NULL, /* No such object exists */
diff --git a/tests/check-qobject-json.c b/tests/check-qobject-json.c
index dc721a8..d889501 100644
--- a/tests/check-qobject-json.c
+++ b/tests/check-qobject-json.c
@@ -13,12 +13,7 @@
#include "qemu/osdep.h"
#include <glib.h>
-#include "qapi/qmp/qstring.h"
-#include "qapi/qmp/qint.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qfloat.h"
-#include "qapi/qmp/qbool.h"
+#include "qapi/qmp/types.h"
#include "qapi/qmp/qobject-json.h"
#include "qemu-common.h"
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 4602529..edb4b31 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -20,6 +20,7 @@
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
+#include "qapi/qmp/qobject-json.h"
#include "test-qmp-introspect.h"
#include "qmp-introspect.h"
#include "qapi-visit.h"
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index cee07ce..d02306c 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -19,6 +19,7 @@
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
+#include "qapi/qmp/qobject-json.h"
typedef struct TestInputVisitorData {
QObject *obj;
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 0706fe9..050d65b 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -19,6 +19,7 @@
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
+#include "qapi/qmp/qobject-json.h"
typedef struct TestOutputVisitorData {
QmpOutputVisitor *qov;
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 7b14b5a..d102fa6 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -20,6 +20,7 @@
#include "test-qapi-visit.h"
#include "qapi/error.h"
#include "qapi/qmp/types.h"
+#include "qapi/qmp/qobject-json.h"
#include "qapi/qmp-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/string-input-visitor.h"
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 02/18] qapi: Improve use of qmp/types.h
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 02/18] qapi: Improve use of qmp/types.h Eric Blake
@ 2016-04-29 11:46 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-04-29 11:46 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, Michael Roth, Luiz Capitulino, famz, Michael S. Tsirkin
Eric Blake <eblake@redhat.com> writes:
> 'qobject-json.h' is not a QObject subtype; include this file
> directly in .c files that are using it, rather than abusing
> qmp/types.h for that purpose.
>
> Meanwhile, for files that include a list of individual QObject
> subtypes, it's easier to just use qmp/types.h for that purpose.
I checked that this patch doesn't add any new dependencies to .d files.
In general, I'm wary of headers including headers to save typing,
because it invites including the kitchen sink, resulting in slow
compiles. Not this patch's (or series') problem, though.
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Fam Zheng <famz@redhat.com>
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
2016-04-29 4:23 ` [Qemu-devel] " Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 02/18] qapi: Improve use of qmp/types.h Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 12:09 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting Eric Blake
` (15 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Luiz Capitulino
Pull out a new qstring_append_json_string() helper, so that all
JSON output producers can use the same output escaping rules.
While it appears that vmstate's use of the simpler qjson.c
formatter is not currently encountering any string that needs
escapes to be valid JSON, it is better to be safe than sorry.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
v3: rebase to include cleanups in master
v2: no change
---
include/qapi/qmp/qstring.h | 1 +
qjson.c | 9 +++----
qobject/qobject-json.c | 59 ++-------------------------------------------
qobject/qstring.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 66 insertions(+), 63 deletions(-)
diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index 10076b7..a254ee3 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -30,6 +30,7 @@ const char *qstring_get_str(const QString *qstring);
void qstring_append_int(QString *qstring, int64_t value);
void qstring_append(QString *qstring, const char *str);
void qstring_append_chr(QString *qstring, int c);
+void qstring_append_json_string(QString *qstring, const char *raw);
QString *qobject_to_qstring(const QObject *obj);
void qstring_destroy_obj(QObject *obj);
diff --git a/qjson.c b/qjson.c
index b65ca6e..b9a9a36 100644
--- a/qjson.c
+++ b/qjson.c
@@ -36,9 +36,8 @@ static void json_emit_element(QJSON *json, const char *name)
}
if (name) {
- qstring_append(json->str, "\"");
- qstring_append(json->str, name);
- qstring_append(json->str, "\" : ");
+ qstring_append_json_string(json->str, name);
+ qstring_append(json->str, " : ");
}
}
@@ -77,9 +76,7 @@ void json_prop_int(QJSON *json, const char *name, int64_t val)
void json_prop_str(QJSON *json, const char *name, const char *str)
{
json_emit_element(json, name);
- qstring_append_chr(json->str, '"');
- qstring_append(json->str, str);
- qstring_append_chr(json->str, '"');
+ qstring_append_json_string(json->str, str);
}
const char *qjson_get_str(QJSON *json)
diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
index 24e7d80..6fed1ee 100644
--- a/qobject/qobject-json.c
+++ b/qobject/qobject-json.c
@@ -16,7 +16,6 @@
#include "qapi/qmp/json-parser.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/qobject-json.h"
-#include "qemu/unicode.h"
#include "qapi/qmp/types.h"
typedef struct JSONParsingState
@@ -81,7 +80,6 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent);
static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
{
ToJsonIterState *s = opaque;
- QString *qkey;
int j;
if (s->count) {
@@ -94,9 +92,7 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
qstring_append(s->str, " ");
}
- qkey = qstring_from_str(key);
- to_json(QOBJECT(qkey), s->str, s->pretty, s->indent);
- QDECREF(qkey);
+ qstring_append_json_string(s->str, key);
qstring_append(s->str, ": ");
to_json(obj, s->str, s->pretty, s->indent);
@@ -138,58 +134,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
}
case QTYPE_QSTRING: {
QString *val = qobject_to_qstring(obj);
- const char *ptr;
- int cp;
- char buf[16];
- char *end;
-
- ptr = qstring_get_str(val);
- qstring_append(str, "\"");
-
- for (; *ptr; ptr = end) {
- cp = mod_utf8_codepoint(ptr, 6, &end);
- switch (cp) {
- case '\"':
- qstring_append(str, "\\\"");
- break;
- case '\\':
- qstring_append(str, "\\\\");
- break;
- case '\b':
- qstring_append(str, "\\b");
- break;
- case '\f':
- qstring_append(str, "\\f");
- break;
- case '\n':
- qstring_append(str, "\\n");
- break;
- case '\r':
- qstring_append(str, "\\r");
- break;
- case '\t':
- qstring_append(str, "\\t");
- break;
- default:
- if (cp < 0) {
- cp = 0xFFFD; /* replacement character */
- }
- if (cp > 0xFFFF) {
- /* beyond BMP; need a surrogate pair */
- snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
- 0xD800 + ((cp - 0x10000) >> 10),
- 0xDC00 + ((cp - 0x10000) & 0x3FF));
- } else if (cp < 0x20 || cp >= 0x7F) {
- snprintf(buf, sizeof(buf), "\\u%04X", cp);
- } else {
- buf[0] = cp;
- buf[1] = 0;
- }
- qstring_append(str, buf);
- }
- };
-
- qstring_append(str, "\"");
+ qstring_append_json_string(str, qstring_get_str(val));
break;
}
case QTYPE_QDICT: {
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 5da7b5f..9f0df5b 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -14,6 +14,7 @@
#include "qapi/qmp/qobject.h"
#include "qapi/qmp/qstring.h"
#include "qemu-common.h"
+#include "qemu/unicode.h"
/**
* qstring_new(): Create a new empty QString
@@ -107,6 +108,65 @@ void qstring_append_chr(QString *qstring, int c)
}
/**
+ * qstring_append_json_string(): Append a raw string to a QString, while
+ * adding outer "" and JSON escape sequences.
+ */
+void qstring_append_json_string(QString *qstring, const char *raw)
+{
+ const char *ptr = raw;
+ int cp;
+ char buf[16];
+ char *end;
+
+ qstring_append(qstring, "\"");
+
+ for (; *ptr; ptr = end) {
+ cp = mod_utf8_codepoint(ptr, 6, &end);
+ switch (cp) {
+ case '\"':
+ qstring_append(qstring, "\\\"");
+ break;
+ case '\\':
+ qstring_append(qstring, "\\\\");
+ break;
+ case '\b':
+ qstring_append(qstring, "\\b");
+ break;
+ case '\f':
+ qstring_append(qstring, "\\f");
+ break;
+ case '\n':
+ qstring_append(qstring, "\\n");
+ break;
+ case '\r':
+ qstring_append(qstring, "\\r");
+ break;
+ case '\t':
+ qstring_append(qstring, "\\t");
+ break;
+ default:
+ if (cp < 0) {
+ cp = 0xFFFD; /* replacement character */
+ }
+ if (cp > 0xFFFF) {
+ /* beyond BMP; need a surrogate pair */
+ snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
+ 0xD800 + ((cp - 0x10000) >> 10),
+ 0xDC00 + ((cp - 0x10000) & 0x3FF));
+ } else if (cp < 0x20 || cp >= 0x7F) {
+ snprintf(buf, sizeof(buf), "\\u%04X", cp);
+ } else {
+ buf[0] = cp;
+ buf[1] = 0;
+ }
+ qstring_append(qstring, buf);
+ }
+ };
+
+ qstring_append(qstring, "\"");
+}
+
+/**
* qobject_to_qstring(): Convert a QObject to a QString
*/
QString *qobject_to_qstring(const QObject *obj)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping Eric Blake
@ 2016-04-29 12:09 ` Markus Armbruster
2016-04-29 17:57 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-04-29 12:09 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Pull out a new qstring_append_json_string() helper, so that all
> JSON output producers can use the same output escaping rules.
>
> While it appears that vmstate's use of the simpler qjson.c
> formatter is not currently encountering any string that needs
> escapes to be valid JSON, it is better to be safe than sorry.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Fam Zheng <famz@redhat.com>
>
> ---
> v3: rebase to include cleanups in master
> v2: no change
> ---
> include/qapi/qmp/qstring.h | 1 +
> qjson.c | 9 +++----
> qobject/qobject-json.c | 59 ++-------------------------------------------
> qobject/qstring.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 66 insertions(+), 63 deletions(-)
>
> diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
> index 10076b7..a254ee3 100644
> --- a/include/qapi/qmp/qstring.h
> +++ b/include/qapi/qmp/qstring.h
> @@ -30,6 +30,7 @@ const char *qstring_get_str(const QString *qstring);
> void qstring_append_int(QString *qstring, int64_t value);
> void qstring_append(QString *qstring, const char *str);
> void qstring_append_chr(QString *qstring, int c);
> +void qstring_append_json_string(QString *qstring, const char *raw);
> QString *qobject_to_qstring(const QObject *obj);
> void qstring_destroy_obj(QObject *obj);
>
> diff --git a/qjson.c b/qjson.c
> index b65ca6e..b9a9a36 100644
> --- a/qjson.c
> +++ b/qjson.c
> @@ -36,9 +36,8 @@ static void json_emit_element(QJSON *json, const char *name)
> }
>
> if (name) {
> - qstring_append(json->str, "\"");
> - qstring_append(json->str, name);
> - qstring_append(json->str, "\" : ");
> + qstring_append_json_string(json->str, name);
> + qstring_append(json->str, " : ");
> }
> }
>
> @@ -77,9 +76,7 @@ void json_prop_int(QJSON *json, const char *name, int64_t val)
> void json_prop_str(QJSON *json, const char *name, const char *str)
> {
> json_emit_element(json, name);
> - qstring_append_chr(json->str, '"');
> - qstring_append(json->str, str);
> - qstring_append_chr(json->str, '"');
> + qstring_append_json_string(json->str, str);
> }
>
> const char *qjson_get_str(QJSON *json)
> diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
> index 24e7d80..6fed1ee 100644
> --- a/qobject/qobject-json.c
> +++ b/qobject/qobject-json.c
> @@ -16,7 +16,6 @@
> #include "qapi/qmp/json-parser.h"
> #include "qapi/qmp/json-streamer.h"
> #include "qapi/qmp/qobject-json.h"
> -#include "qemu/unicode.h"
> #include "qapi/qmp/types.h"
>
> typedef struct JSONParsingState
> @@ -81,7 +80,6 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent);
> static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
> {
> ToJsonIterState *s = opaque;
> - QString *qkey;
> int j;
>
> if (s->count) {
> @@ -94,9 +92,7 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
> qstring_append(s->str, " ");
> }
>
> - qkey = qstring_from_str(key);
> - to_json(QOBJECT(qkey), s->str, s->pretty, s->indent);
> - QDECREF(qkey);
> + qstring_append_json_string(s->str, key);
>
> qstring_append(s->str, ": ");
> to_json(obj, s->str, s->pretty, s->indent);
> @@ -138,58 +134,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
> }
> case QTYPE_QSTRING: {
> QString *val = qobject_to_qstring(obj);
> - const char *ptr;
> - int cp;
> - char buf[16];
> - char *end;
> -
> - ptr = qstring_get_str(val);
> - qstring_append(str, "\"");
> -
> - for (; *ptr; ptr = end) {
> - cp = mod_utf8_codepoint(ptr, 6, &end);
> - switch (cp) {
> - case '\"':
> - qstring_append(str, "\\\"");
> - break;
> - case '\\':
> - qstring_append(str, "\\\\");
> - break;
> - case '\b':
> - qstring_append(str, "\\b");
> - break;
> - case '\f':
> - qstring_append(str, "\\f");
> - break;
> - case '\n':
> - qstring_append(str, "\\n");
> - break;
> - case '\r':
> - qstring_append(str, "\\r");
> - break;
> - case '\t':
> - qstring_append(str, "\\t");
> - break;
> - default:
> - if (cp < 0) {
> - cp = 0xFFFD; /* replacement character */
> - }
> - if (cp > 0xFFFF) {
> - /* beyond BMP; need a surrogate pair */
> - snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
> - 0xD800 + ((cp - 0x10000) >> 10),
> - 0xDC00 + ((cp - 0x10000) & 0x3FF));
> - } else if (cp < 0x20 || cp >= 0x7F) {
> - snprintf(buf, sizeof(buf), "\\u%04X", cp);
> - } else {
> - buf[0] = cp;
> - buf[1] = 0;
> - }
> - qstring_append(str, buf);
> - }
> - };
> -
> - qstring_append(str, "\"");
> + qstring_append_json_string(str, qstring_get_str(val));
> break;
> }
> case QTYPE_QDICT: {
> diff --git a/qobject/qstring.c b/qobject/qstring.c
> index 5da7b5f..9f0df5b 100644
> --- a/qobject/qstring.c
> +++ b/qobject/qstring.c
> @@ -14,6 +14,7 @@
> #include "qapi/qmp/qobject.h"
> #include "qapi/qmp/qstring.h"
> #include "qemu-common.h"
> +#include "qemu/unicode.h"
>
> /**
> * qstring_new(): Create a new empty QString
> @@ -107,6 +108,65 @@ void qstring_append_chr(QString *qstring, int c)
> }
>
> /**
> + * qstring_append_json_string(): Append a raw string to a QString, while
> + * adding outer "" and JSON escape sequences.
> + */
> +void qstring_append_json_string(QString *qstring, const char *raw)
> +{
> + const char *ptr = raw;
> + int cp;
> + char buf[16];
> + char *end;
> +
> + qstring_append(qstring, "\"");
> +
> + for (; *ptr; ptr = end) {
Make that
for (ptr = raw; *ptr; ptr = end) {
and drop ptr's initializer.
> + cp = mod_utf8_codepoint(ptr, 6, &end);
> + switch (cp) {
> + case '\"':
> + qstring_append(qstring, "\\\"");
> + break;
> + case '\\':
> + qstring_append(qstring, "\\\\");
> + break;
> + case '\b':
> + qstring_append(qstring, "\\b");
> + break;
> + case '\f':
> + qstring_append(qstring, "\\f");
> + break;
> + case '\n':
> + qstring_append(qstring, "\\n");
> + break;
> + case '\r':
> + qstring_append(qstring, "\\r");
> + break;
> + case '\t':
> + qstring_append(qstring, "\\t");
> + break;
> + default:
> + if (cp < 0) {
> + cp = 0xFFFD; /* replacement character */
> + }
> + if (cp > 0xFFFF) {
> + /* beyond BMP; need a surrogate pair */
> + snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
> + 0xD800 + ((cp - 0x10000) >> 10),
> + 0xDC00 + ((cp - 0x10000) & 0x3FF));
> + } else if (cp < 0x20 || cp >= 0x7F) {
> + snprintf(buf, sizeof(buf), "\\u%04X", cp);
> + } else {
> + buf[0] = cp;
> + buf[1] = 0;
> + }
> + qstring_append(qstring, buf);
> + }
> + };
> +
> + qstring_append(qstring, "\"");
> +}
I think this belongs to qobject-json.c, because it's very much about
JSON (it encapsulates knowledge on JSON string escaping), and a mere
user of QString (it knows nothing about QString's implementation).
Precedence: qobject_from_json() & friends are there, not in qobject.c.
Since qjson.c doesn't already use the space-wasting "start the function
comment with the function's name" style, let's drop that waste of screen
space and simply write
/*
* Append a JSON string to @qstring that encodes the C string @str.
* The JSON string is enclosed in double quotes and has funny
* characters escaped.
*/
and rename raw to str.
> +
> +/**
> * qobject_to_qstring(): Convert a QObject to a QString
> */
> QString *qobject_to_qstring(const QObject *obj)
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping
2016-04-29 12:09 ` Markus Armbruster
@ 2016-04-29 17:57 ` Eric Blake
2016-05-03 7:36 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 17:57 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]
On 04/29/2016 06:09 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Pull out a new qstring_append_json_string() helper, so that all
>> JSON output producers can use the same output escaping rules.
>>
>> While it appears that vmstate's use of the simpler qjson.c
>> formatter is not currently encountering any string that needs
>> escapes to be valid JSON, it is better to be safe than sorry.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>> - qstring_append(str, "\"");
>> + qstring_append_json_string(str, qstring_get_str(val));
>> break;
> I think this belongs to qobject-json.c, because it's very much about
> JSON (it encapsulates knowledge on JSON string escaping), and a mere
> user of QString (it knows nothing about QString's implementation).
>
> Precedence: qobject_from_json() & friends are there, not in qobject.c.
Fair enough. Does the name qstring_append_json_string() still work, or
do I need to adjust the name to something with 'qobject' in there, to
make it easier to know which header and .c file to use for the function?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping
2016-04-29 17:57 ` Eric Blake
@ 2016-05-03 7:36 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 7:36 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 04/29/2016 06:09 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Pull out a new qstring_append_json_string() helper, so that all
>>> JSON output producers can use the same output escaping rules.
>>>
>>> While it appears that vmstate's use of the simpler qjson.c
>>> formatter is not currently encountering any string that needs
>>> escapes to be valid JSON, it is better to be safe than sorry.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> Reviewed-by: Fam Zheng <famz@redhat.com>
>
>>> - qstring_append(str, "\"");
>>> + qstring_append_json_string(str, qstring_get_str(val));
>>> break;
>
>> I think this belongs to qobject-json.c, because it's very much about
>> JSON (it encapsulates knowledge on JSON string escaping), and a mere
>> user of QString (it knows nothing about QString's implementation).
>>
>> Precedence: qobject_from_json() & friends are there, not in qobject.c.
>
> Fair enough. Does the name qstring_append_json_string() still work, or
> do I need to adjust the name to something with 'qobject' in there, to
> make it easier to know which header and .c file to use for the function?
I think the name is fine.
If you strongly prefer to encode the source file in the identifier, you
could use qobject_json_string_append_to_qstring(), but that's even
longer, and less clear.
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (2 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 03/18] qapi: Factor out JSON string escaping Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 13:22 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 05/18] qapi: Use qstring_append_chr() where appropriate Eric Blake
` (14 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Luiz Capitulino
Pull out a new qstring_append_json_number() helper, so that all
JSON output producers can use a consistent style for printing
floating point without duplicating code (since we are doing more
data massaging than a simple printf format can handle).
Address one FIXME by adding an Error parameter and warning the
caller if the requested number cannot be represented in JSON;
but add another FIXME in its place because we have no way to
report the problem higher up the stack.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
v3: rebase to latest; even though the patch differs quite a bit
from v2, the changes are due to prior comment changes that are
just moving between files, so R-b kept
v2: minor wording tweaks
---
include/qapi/qmp/qstring.h | 4 +++-
qobject/qobject-json.c | 27 +++------------------------
qobject/qstring.c | 42 +++++++++++++++++++++++++++++++++++++++++-
3 files changed, 47 insertions(+), 26 deletions(-)
diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index a254ee3..f00e3df 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -1,7 +1,7 @@
/*
* QString Module
*
- * Copyright (C) 2009 Red Hat Inc.
+ * Copyright (C) 2009, 2015 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -14,6 +14,7 @@
#define QSTRING_H
#include "qapi/qmp/qobject.h"
+#include "qapi/error.h"
typedef struct QString {
QObject base;
@@ -31,6 +32,7 @@ void qstring_append_int(QString *qstring, int64_t value);
void qstring_append(QString *qstring, const char *str);
void qstring_append_chr(QString *qstring, int c);
void qstring_append_json_string(QString *qstring, const char *raw);
+void qstring_append_json_number(QString *qstring, double number, Error **errp);
QString *qobject_to_qstring(const QObject *obj);
void qstring_destroy_obj(QObject *obj);
diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
index 6fed1ee..97bccb7 100644
--- a/qobject/qobject-json.c
+++ b/qobject/qobject-json.c
@@ -177,30 +177,9 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
}
case QTYPE_QFLOAT: {
QFloat *val = qobject_to_qfloat(obj);
- char buffer[1024];
- int len;
-
- /* FIXME: snprintf() is locale dependent; but JSON requires
- * numbers to be formatted as if in the C locale. Dependence
- * on C locale is a pervasive issue in QEMU. */
- /* FIXME: This risks printing Inf or NaN, which are not valid
- * JSON values. */
- /* FIXME: the default precision of 6 for %f often causes
- * rounding errors; we should be using DBL_DECIMAL_DIG (17),
- * and only rounding to a shorter number if the result would
- * still produce the same floating point value. */
- len = snprintf(buffer, sizeof(buffer), "%f", qfloat_get_double(val));
- while (len > 0 && buffer[len - 1] == '0') {
- len--;
- }
-
- if (len && buffer[len - 1] == '.') {
- buffer[len - 1] = 0;
- } else {
- buffer[len] = 0;
- }
-
- qstring_append(str, buffer);
+ /* FIXME: no way to report invalid JSON to caller, so for now
+ * we just ignore it */
+ qstring_append_json_number(str, qfloat_get_double(val), NULL);
break;
}
case QTYPE_QBOOL: {
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 9f0df5b..618dd8f 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -1,7 +1,7 @@
/*
* QString Module
*
- * Copyright (C) 2009 Red Hat Inc.
+ * Copyright (C) 2009, 2015-2016 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -15,6 +15,7 @@
#include "qapi/qmp/qstring.h"
#include "qemu-common.h"
#include "qemu/unicode.h"
+#include <math.h>
/**
* qstring_new(): Create a new empty QString
@@ -167,6 +168,45 @@ void qstring_append_json_string(QString *qstring, const char *raw)
}
/**
+ * qstring_append_json_number(): Append a JSON number to a QString.
+ * Set @errp if the number is not representable in JSON, but append the
+ * output anyway (callers can then choose to ignore the warning).
+ */
+void qstring_append_json_number(QString *qstring, double number, Error **errp)
+{
+ char buffer[1024];
+ int len;
+
+ /* JSON does not allow Inf or NaN; append it but set errp */
+ if (!isfinite(number)) {
+ error_setg(errp, "Non-finite number %f is not valid JSON", number);
+ }
+
+ /* FIXME: snprintf() is locale dependent; but JSON requires
+ * numbers to be formatted as if in the C locale. Dependence
+ * on C locale is a pervasive issue in QEMU. */
+ /* FIXME: This risks printing Inf or NaN, which are not valid
+ * JSON values. */
+ /* FIXME: the default precision of 6 for %f often causes
+ * rounding errors; we should be using DBL_DECIMAL_DIG (17),
+ * and only rounding to a shorter number if the result would
+ * still produce the same floating point value. */
+ len = snprintf(buffer, sizeof(buffer), "%f", number);
+ assert(len > 0 && len < sizeof(buffer));
+ while (len > 0 && buffer[len - 1] == '0') {
+ len--;
+ }
+
+ if (len && buffer[len - 1] == '.') {
+ buffer[len - 1] = 0;
+ } else {
+ buffer[len] = 0;
+ }
+
+ qstring_append(qstring, buffer);
+}
+
+/**
* qobject_to_qstring(): Convert a QObject to a QString
*/
QString *qobject_to_qstring(const QObject *obj)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting Eric Blake
@ 2016-04-29 13:22 ` Markus Armbruster
2016-04-29 13:43 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-04-29 13:22 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Pull out a new qstring_append_json_number() helper, so that all
> JSON output producers can use a consistent style for printing
> floating point without duplicating code (since we are doing more
> data massaging than a simple printf format can handle).
>
> Address one FIXME by adding an Error parameter and warning the
> caller if the requested number cannot be represented in JSON;
> but add another FIXME in its place because we have no way to
> report the problem higher up the stack.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Fam Zheng <famz@redhat.com>
>
> ---
> v3: rebase to latest; even though the patch differs quite a bit
> from v2, the changes are due to prior comment changes that are
> just moving between files, so R-b kept
> v2: minor wording tweaks
> ---
> include/qapi/qmp/qstring.h | 4 +++-
> qobject/qobject-json.c | 27 +++------------------------
> qobject/qstring.c | 42 +++++++++++++++++++++++++++++++++++++++++-
> 3 files changed, 47 insertions(+), 26 deletions(-)
>
> diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
> index a254ee3..f00e3df 100644
> --- a/include/qapi/qmp/qstring.h
> +++ b/include/qapi/qmp/qstring.h
> @@ -1,7 +1,7 @@
> /*
> * QString Module
> *
> - * Copyright (C) 2009 Red Hat Inc.
> + * Copyright (C) 2009, 2015 Red Hat Inc.
> *
> * Authors:
> * Luiz Capitulino <lcapitulino@redhat.com>
> @@ -14,6 +14,7 @@
> #define QSTRING_H
>
> #include "qapi/qmp/qobject.h"
> +#include "qapi/error.h"
>
> typedef struct QString {
> QObject base;
> @@ -31,6 +32,7 @@ void qstring_append_int(QString *qstring, int64_t value);
> void qstring_append(QString *qstring, const char *str);
> void qstring_append_chr(QString *qstring, int c);
> void qstring_append_json_string(QString *qstring, const char *raw);
> +void qstring_append_json_number(QString *qstring, double number, Error **errp);
> QString *qobject_to_qstring(const QObject *obj);
> void qstring_destroy_obj(QObject *obj);
>
> diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
> index 6fed1ee..97bccb7 100644
> --- a/qobject/qobject-json.c
> +++ b/qobject/qobject-json.c
> @@ -177,30 +177,9 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
> }
> case QTYPE_QFLOAT: {
> QFloat *val = qobject_to_qfloat(obj);
> - char buffer[1024];
> - int len;
> -
> - /* FIXME: snprintf() is locale dependent; but JSON requires
> - * numbers to be formatted as if in the C locale. Dependence
> - * on C locale is a pervasive issue in QEMU. */
> - /* FIXME: This risks printing Inf or NaN, which are not valid
> - * JSON values. */
> - /* FIXME: the default precision of 6 for %f often causes
> - * rounding errors; we should be using DBL_DECIMAL_DIG (17),
> - * and only rounding to a shorter number if the result would
> - * still produce the same floating point value. */
> - len = snprintf(buffer, sizeof(buffer), "%f", qfloat_get_double(val));
> - while (len > 0 && buffer[len - 1] == '0') {
> - len--;
> - }
> -
> - if (len && buffer[len - 1] == '.') {
> - buffer[len - 1] = 0;
> - } else {
> - buffer[len] = 0;
> - }
> -
> - qstring_append(str, buffer);
> + /* FIXME: no way to report invalid JSON to caller, so for now
> + * we just ignore it */
> + qstring_append_json_number(str, qfloat_get_double(val), NULL);
> break;
> }
> case QTYPE_QBOOL: {
> diff --git a/qobject/qstring.c b/qobject/qstring.c
> index 9f0df5b..618dd8f 100644
> --- a/qobject/qstring.c
> +++ b/qobject/qstring.c
> @@ -1,7 +1,7 @@
> /*
> * QString Module
> *
> - * Copyright (C) 2009 Red Hat Inc.
> + * Copyright (C) 2009, 2015-2016 Red Hat Inc.
> *
> * Authors:
> * Luiz Capitulino <lcapitulino@redhat.com>
> @@ -15,6 +15,7 @@
> #include "qapi/qmp/qstring.h"
> #include "qemu-common.h"
> #include "qemu/unicode.h"
> +#include <math.h>
>
> /**
> * qstring_new(): Create a new empty QString
> @@ -167,6 +168,45 @@ void qstring_append_json_string(QString *qstring, const char *raw)
> }
>
> /**
> + * qstring_append_json_number(): Append a JSON number to a QString.
> + * Set @errp if the number is not representable in JSON, but append the
> + * output anyway (callers can then choose to ignore the warning).
> + */
> +void qstring_append_json_number(QString *qstring, double number, Error **errp)
> +{
> + char buffer[1024];
> + int len;
> +
> + /* JSON does not allow Inf or NaN; append it but set errp */
> + if (!isfinite(number)) {
> + error_setg(errp, "Non-finite number %f is not valid JSON", number);
> + }
Separate patch, please.
"Append it but set errp" feels odd. Normally, returning with an error
set means the function failed to do its job.
> +
> + /* FIXME: snprintf() is locale dependent; but JSON requires
> + * numbers to be formatted as if in the C locale. Dependence
> + * on C locale is a pervasive issue in QEMU. */
> + /* FIXME: This risks printing Inf or NaN, which are not valid
> + * JSON values. */
Your !isfinite() conditional addresses this, doesn't it?
> + /* FIXME: the default precision of 6 for %f often causes
> + * rounding errors; we should be using DBL_DECIMAL_DIG (17),
> + * and only rounding to a shorter number if the result would
> + * still produce the same floating point value. */
> + len = snprintf(buffer, sizeof(buffer), "%f", number);
> + assert(len > 0 && len < sizeof(buffer));
> + while (len > 0 && buffer[len - 1] == '0') {
> + len--;
> + }
> +
> + if (len && buffer[len - 1] == '.') {
> + buffer[len - 1] = 0;
> + } else {
> + buffer[len] = 0;
> + }
> +
> + qstring_append(qstring, buffer);
> +}
I think this belongs into qobject-json.c, like the previous patch's
qstring_append_json_string().
> +
> +/**
> * qobject_to_qstring(): Convert a QObject to a QString
> */
> QString *qobject_to_qstring(const QObject *obj)
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting
2016-04-29 13:22 ` Markus Armbruster
@ 2016-04-29 13:43 ` Eric Blake
2016-05-03 8:02 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 13:43 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 2836 bytes --]
On 04/29/2016 07:22 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Pull out a new qstring_append_json_number() helper, so that all
>> JSON output producers can use a consistent style for printing
>> floating point without duplicating code (since we are doing more
>> data massaging than a simple printf format can handle).
>>
>> Address one FIXME by adding an Error parameter and warning the
>> caller if the requested number cannot be represented in JSON;
>> but add another FIXME in its place because we have no way to
>> report the problem higher up the stack.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>>
>> /**
>> + * qstring_append_json_number(): Append a JSON number to a QString.
>> + * Set @errp if the number is not representable in JSON, but append the
>> + * output anyway (callers can then choose to ignore the warning).
>> + */
>> +void qstring_append_json_number(QString *qstring, double number, Error **errp)
>> +{
>> + char buffer[1024];
>> + int len;
>> +
>> + /* JSON does not allow Inf or NaN; append it but set errp */
>> + if (!isfinite(number)) {
>> + error_setg(errp, "Non-finite number %f is not valid JSON", number);
>> + }
>
> Separate patch, please.
Okay.
>
> "Append it but set errp" feels odd. Normally, returning with an error
> set means the function failed to do its job.
This one's weird because by the end of the series, it will be used by
the new JSON visitor (which wants the error message because that is not
valid JSON, and doesn't care if the QString is slightly longer); as well
as the existing QMP output visitor (where existing behavior ignores that
it is not valid JSON, and we don't really have a convenient way to pass
errors back up the stack). Is it worth trying to plumb in better error
reporting to the QMP output visitor, and/or add assertions that values
are finite, and/or document that QMP has an extension beyond JSON in
that it accepts and also might produce Inf/NaN?
>
>> +
>> + /* FIXME: snprintf() is locale dependent; but JSON requires
>> + * numbers to be formatted as if in the C locale. Dependence
>> + * on C locale is a pervasive issue in QEMU. */
>> + /* FIXME: This risks printing Inf or NaN, which are not valid
>> + * JSON values. */
>
> Your !isfinite() conditional addresses this, doesn't it?
Yep. Looks like I messed up the rebase (I realized I had to re-move
updated code, but didn't scrub the comments after the move).
>
> I think this belongs into qobject-json.c, like the previous patch's
> qstring_append_json_string().
Sounds reasonable.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting
2016-04-29 13:43 ` Eric Blake
@ 2016-05-03 8:02 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 8:02 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 04/29/2016 07:22 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Pull out a new qstring_append_json_number() helper, so that all
>>> JSON output producers can use a consistent style for printing
>>> floating point without duplicating code (since we are doing more
>>> data massaging than a simple printf format can handle).
>>>
>>> Address one FIXME by adding an Error parameter and warning the
>>> caller if the requested number cannot be represented in JSON;
>>> but add another FIXME in its place because we have no way to
>>> report the problem higher up the stack.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> Reviewed-by: Fam Zheng <famz@redhat.com>
>>>
>
>>> /**
>>> + * qstring_append_json_number(): Append a JSON number to a QString.
>>> + * Set @errp if the number is not representable in JSON, but append the
>>> + * output anyway (callers can then choose to ignore the warning).
>>> + */
>>> +void qstring_append_json_number(QString *qstring, double number, Error **errp)
>>> +{
>>> + char buffer[1024];
>>> + int len;
>>> +
>>> + /* JSON does not allow Inf or NaN; append it but set errp */
>>> + if (!isfinite(number)) {
>>> + error_setg(errp, "Non-finite number %f is not valid JSON", number);
>>> + }
>>
>> Separate patch, please.
>
> Okay.
>
>>
>> "Append it but set errp" feels odd. Normally, returning with an error
>> set means the function failed to do its job.
>
> This one's weird because by the end of the series, it will be used by
> the new JSON visitor (which wants the error message because that is not
> valid JSON, and doesn't care if the QString is slightly longer); as well
> as the existing QMP output visitor (where existing behavior ignores that
> it is not valid JSON, and we don't really have a convenient way to pass
> errors back up the stack). Is it worth trying to plumb in better error
> reporting to the QMP output visitor, and/or add assertions that values
> are finite, and/or document that QMP has an extension beyond JSON in
> that it accepts and also might produce Inf/NaN?
QMP is what it is. We can try to get it back closer to pure JSON, and
declaring attempts to emit non-finite numbers bugs would be a step in
that direction.
What kind of bugs would that be?
If it's a programming bug, then assert. How would a user of the QMP
output visitor avoid this programming bug?
If it's not a programming bug, then fail with error set. How would a
user of the QMP output visitor handle this error?
That said, I'm reluctant to mess with QMP now. We really, really need
to control the scope of your ongoing QAPI work, or we'll never get to
Marc-André's.
Instead, I'd prefer to document what we have. If we plan to change it,
say so, and add a suitable TODO or FIXME comment.
With things kept as they are, I can see two ways to avoid
qstring_append_json_number()'s unusual behavior:
1. Add a @must_be_finite parameter. Set the error only when it's true,
and don't do anything else then.
2. Lift the decision whether non-finite is okay into the callers.
Instead of
// non-finite is okay
qstring_append_json_number(qs, num, NULL);
// non-finite is not okay
qstring_append_json_number(qs, num, &err);
if (err) {
// handle error...
// return, goto out, or similar
}
do
// non-finite is okay
qstring_append_json_number(qs, num);
// non-finite is not okay
if (!isfinite(num) {
error_setg(&err, "Non-finite number %f is not valid JSON", num);
// handle error...
// return, goto out, or similar
}
qstring_append_json_number(qs, num);
>>> +
>>> + /* FIXME: snprintf() is locale dependent; but JSON requires
>>> + * numbers to be formatted as if in the C locale. Dependence
>>> + * on C locale is a pervasive issue in QEMU. */
>>> + /* FIXME: This risks printing Inf or NaN, which are not valid
>>> + * JSON values. */
>>
>> Your !isfinite() conditional addresses this, doesn't it?
>
> Yep. Looks like I messed up the rebase (I realized I had to re-move
> updated code, but didn't scrub the comments after the move).
>
>
>>
>> I think this belongs into qobject-json.c, like the previous patch's
>> qstring_append_json_string().
>
> Sounds reasonable.
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 05/18] qapi: Use qstring_append_chr() where appropriate
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (3 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 04/18] qapi: Factor out JSON number formatting Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 13:25 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 06/18] qapi: Add qstring_append_format() Eric Blake
` (13 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Luiz Capitulino
No need to create a temporary buffer, when we already have a
function available for our needs.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
v3: no change
v2: no change
---
qobject/json-parser.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/qobject/json-parser.c b/qobject/json-parser.c
index c18e48a..5fbaf58 100644
--- a/qobject/json-parser.c
+++ b/qobject/json-parser.c
@@ -204,12 +204,7 @@ static QString *qstring_from_escaped_str(JSONParserContext *ctxt,
goto out;
}
} else {
- char dummy[2];
-
- dummy[0] = *ptr++;
- dummy[1] = 0;
-
- qstring_append(str, dummy);
+ qstring_append_chr(str, *ptr++);
}
}
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 05/18] qapi: Use qstring_append_chr() where appropriate
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 05/18] qapi: Use qstring_append_chr() where appropriate Eric Blake
@ 2016-04-29 13:25 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-04-29 13:25 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> No need to create a temporary buffer, when we already have a
> function available for our needs.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Fam Zheng <famz@redhat.com>
>
> ---
> v3: no change
> v2: no change
> ---
> qobject/json-parser.c | 7 +------
> 1 file changed, 1 insertion(+), 6 deletions(-)
>
> diff --git a/qobject/json-parser.c b/qobject/json-parser.c
> index c18e48a..5fbaf58 100644
> --- a/qobject/json-parser.c
> +++ b/qobject/json-parser.c
> @@ -204,12 +204,7 @@ static QString *qstring_from_escaped_str(JSONParserContext *ctxt,
> goto out;
> }
> } else {
> - char dummy[2];
> -
> - dummy[0] = *ptr++;
> - dummy[1] = 0;
> -
> - qstring_append(str, dummy);
> + qstring_append_chr(str, *ptr++);
> }
> }
The function has several qstring_append(str, S) where S consists of a
single character. They could all be converted to qstring_append_chr().
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 06/18] qapi: Add qstring_append_format()
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (4 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 05/18] qapi: Use qstring_append_chr() where appropriate Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 13:40 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor Eric Blake
` (12 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Luiz Capitulino
Back in commit 764c1ca (Nov 2009), we added qstring_append_int().
However, it did not see any use until commit 190c882 (Jan 2015).
Furthermore, it has a rather limited use case - to print anything
else, callers still have to format into a temporary buffer, unless
we want to introduce an explosion of new qstring_append_* methods
for each useful type to print.
A much better approach is to add a wrapper that merges printf
behavior onto qstring_append, via the new qstring_append_format()
(and its vararg counterpart). In fact, with that in place, we
no longer need qstring_append_int().
Other immediate uses for the new function include simplifying
two existing clients of qstring_append() on a just-formatted
buffer, and the fact that we can take advantage of printf width
manipulations for more efficient indentation.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
v3: rebase to master
v2: also simplify qstring_append_json_string(), add assertion that
format is well-formed
---
include/qapi/qmp/qstring.h | 7 +++++--
qjson.c | 2 +-
qobject/qobject-json.c | 25 +++++--------------------
qobject/qstring.c | 37 +++++++++++++++++++++++++------------
4 files changed, 36 insertions(+), 35 deletions(-)
diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
index f00e3df..10afa5d 100644
--- a/include/qapi/qmp/qstring.h
+++ b/include/qapi/qmp/qstring.h
@@ -1,7 +1,7 @@
/*
* QString Module
*
- * Copyright (C) 2009, 2015 Red Hat Inc.
+ * Copyright (C) 2009-2016 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -28,9 +28,12 @@ QString *qstring_from_str(const char *str);
QString *qstring_from_substr(const char *str, int start, int end);
size_t qstring_get_length(const QString *qstring);
const char *qstring_get_str(const QString *qstring);
-void qstring_append_int(QString *qstring, int64_t value);
void qstring_append(QString *qstring, const char *str);
void qstring_append_chr(QString *qstring, int c);
+void qstring_append_format(QString *qstring, const char *fmt, ...)
+ GCC_FMT_ATTR(2, 3);
+void qstring_append_vformat(QString *qstring, const char *fmt, va_list ap)
+ GCC_FMT_ATTR(2, 0);
void qstring_append_json_string(QString *qstring, const char *raw);
void qstring_append_json_number(QString *qstring, double number, Error **errp);
QString *qobject_to_qstring(const QObject *obj);
diff --git a/qjson.c b/qjson.c
index b9a9a36..d172b1f 100644
--- a/qjson.c
+++ b/qjson.c
@@ -70,7 +70,7 @@ void json_end_array(QJSON *json)
void json_prop_int(QJSON *json, const char *name, int64_t val)
{
json_emit_element(json, name);
- qstring_append_int(json->str, val);
+ qstring_append_format(json->str, "%" PRId64, val);
}
void json_prop_str(QJSON *json, const char *name, const char *str)
diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
index 97bccb7..963de07 100644
--- a/qobject/qobject-json.c
+++ b/qobject/qobject-json.c
@@ -80,16 +80,13 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent);
static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
{
ToJsonIterState *s = opaque;
- int j;
if (s->count) {
qstring_append(s->str, s->pretty ? "," : ", ");
}
if (s->pretty) {
- qstring_append(s->str, "\n");
- for (j = 0 ; j < s->indent ; j++)
- qstring_append(s->str, " ");
+ qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
}
qstring_append_json_string(s->str, key);
@@ -102,16 +99,13 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
static void to_json_list_iter(QObject *obj, void *opaque)
{
ToJsonIterState *s = opaque;
- int j;
if (s->count) {
qstring_append(s->str, s->pretty ? "," : ", ");
}
if (s->pretty) {
- qstring_append(s->str, "\n");
- for (j = 0 ; j < s->indent ; j++)
- qstring_append(s->str, " ");
+ qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
}
to_json(obj, s->str, s->pretty, s->indent);
@@ -126,10 +120,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
break;
case QTYPE_QINT: {
QInt *val = qobject_to_qint(obj);
- char buffer[1024];
-
- snprintf(buffer, sizeof(buffer), "%" PRId64, qint_get_int(val));
- qstring_append(str, buffer);
+ qstring_append_format(str, "%" PRId64, qint_get_int(val));
break;
}
case QTYPE_QSTRING: {
@@ -148,10 +139,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
qstring_append(str, "{");
qdict_iter(val, to_json_dict_iter, &s);
if (pretty) {
- int j;
- qstring_append(str, "\n");
- for (j = 0 ; j < indent ; j++)
- qstring_append(str, " ");
+ qstring_append_format(str, "\n%*s", 4 * indent, "");
}
qstring_append(str, "}");
break;
@@ -167,10 +155,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
qstring_append(str, "[");
qlist_iter(val, (void *)to_json_list_iter, &s);
if (pretty) {
- int j;
- qstring_append(str, "\n");
- for (j = 0 ; j < indent ; j++)
- qstring_append(str, " ");
+ qstring_append_format(str, "\n%*s", 4 * indent, "");
}
qstring_append(str, "]");
break;
diff --git a/qobject/qstring.c b/qobject/qstring.c
index 618dd8f..0285d53 100644
--- a/qobject/qstring.c
+++ b/qobject/qstring.c
@@ -90,12 +90,28 @@ void qstring_append(QString *qstring, const char *str)
qstring->string[qstring->length] = 0;
}
-void qstring_append_int(QString *qstring, int64_t value)
+void qstring_append_format(QString *qstring, const char *fmt, ...)
{
- char num[32];
+ va_list ap;
- snprintf(num, sizeof(num), "%" PRId64, value);
- qstring_append(qstring, num);
+ va_start(ap, fmt);
+ qstring_append_vformat(qstring, fmt, ap);
+ va_end(ap);
+}
+
+void qstring_append_vformat(QString *qstring, const char *fmt, va_list ap)
+{
+ va_list ap2;
+ int len;
+
+ va_copy(ap2, ap);
+ len = vsnprintf(qstring->string + qstring->length, 0, fmt, ap2);
+ assert(len >= 0);
+ va_end(ap2);
+
+ capacity_increase(qstring, len);
+ vsnprintf(qstring->string + qstring->length, len + 1, fmt, ap);
+ qstring->length += len;
}
/**
@@ -116,7 +132,6 @@ void qstring_append_json_string(QString *qstring, const char *raw)
{
const char *ptr = raw;
int cp;
- char buf[16];
char *end;
qstring_append(qstring, "\"");
@@ -151,16 +166,14 @@ void qstring_append_json_string(QString *qstring, const char *raw)
}
if (cp > 0xFFFF) {
/* beyond BMP; need a surrogate pair */
- snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
- 0xD800 + ((cp - 0x10000) >> 10),
- 0xDC00 + ((cp - 0x10000) & 0x3FF));
+ qstring_append_format(qstring, "\\u%04X\\u%04X",
+ 0xD800 + ((cp - 0x10000) >> 10),
+ 0xDC00 + ((cp - 0x10000) & 0x3FF));
} else if (cp < 0x20 || cp >= 0x7F) {
- snprintf(buf, sizeof(buf), "\\u%04X", cp);
+ qstring_append_format(qstring, "\\u%04X", cp);
} else {
- buf[0] = cp;
- buf[1] = 0;
+ qstring_append_chr(qstring, cp);
}
- qstring_append(qstring, buf);
}
};
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 06/18] qapi: Add qstring_append_format()
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 06/18] qapi: Add qstring_append_format() Eric Blake
@ 2016-04-29 13:40 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-04-29 13:40 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Back in commit 764c1ca (Nov 2009), we added qstring_append_int().
> However, it did not see any use until commit 190c882 (Jan 2015).
> Furthermore, it has a rather limited use case - to print anything
> else, callers still have to format into a temporary buffer, unless
> we want to introduce an explosion of new qstring_append_* methods
> for each useful type to print.
>
> A much better approach is to add a wrapper that merges printf
> behavior onto qstring_append, via the new qstring_append_format()
> (and its vararg counterpart). In fact, with that in place, we
> no longer need qstring_append_int().
>
> Other immediate uses for the new function include simplifying
> two existing clients of qstring_append() on a just-formatted
> buffer, and the fact that we can take advantage of printf width
> manipulations for more efficient indentation.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Fam Zheng <famz@redhat.com>
>
> ---
> v3: rebase to master
> v2: also simplify qstring_append_json_string(), add assertion that
> format is well-formed
> ---
> include/qapi/qmp/qstring.h | 7 +++++--
> qjson.c | 2 +-
> qobject/qobject-json.c | 25 +++++--------------------
> qobject/qstring.c | 37 +++++++++++++++++++++++++------------
> 4 files changed, 36 insertions(+), 35 deletions(-)
>
> diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h
> index f00e3df..10afa5d 100644
> --- a/include/qapi/qmp/qstring.h
> +++ b/include/qapi/qmp/qstring.h
> @@ -1,7 +1,7 @@
> /*
> * QString Module
> *
> - * Copyright (C) 2009, 2015 Red Hat Inc.
> + * Copyright (C) 2009-2016 Red Hat Inc.
> *
> * Authors:
> * Luiz Capitulino <lcapitulino@redhat.com>
> @@ -28,9 +28,12 @@ QString *qstring_from_str(const char *str);
> QString *qstring_from_substr(const char *str, int start, int end);
> size_t qstring_get_length(const QString *qstring);
> const char *qstring_get_str(const QString *qstring);
> -void qstring_append_int(QString *qstring, int64_t value);
> void qstring_append(QString *qstring, const char *str);
> void qstring_append_chr(QString *qstring, int c);
> +void qstring_append_format(QString *qstring, const char *fmt, ...)
> + GCC_FMT_ATTR(2, 3);
> +void qstring_append_vformat(QString *qstring, const char *fmt, va_list ap)
> + GCC_FMT_ATTR(2, 0);
Let's call these qstring_append_printf() and qstring_append_vprintf(),
to match GLib's g_string_append_printf() and g_string_append_vprintf().
> void qstring_append_json_string(QString *qstring, const char *raw);
> void qstring_append_json_number(QString *qstring, double number, Error **errp);
> QString *qobject_to_qstring(const QObject *obj);
> diff --git a/qjson.c b/qjson.c
> index b9a9a36..d172b1f 100644
> --- a/qjson.c
> +++ b/qjson.c
> @@ -70,7 +70,7 @@ void json_end_array(QJSON *json)
> void json_prop_int(QJSON *json, const char *name, int64_t val)
> {
> json_emit_element(json, name);
> - qstring_append_int(json->str, val);
> + qstring_append_format(json->str, "%" PRId64, val);
> }
>
> void json_prop_str(QJSON *json, const char *name, const char *str)
> diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
> index 97bccb7..963de07 100644
> --- a/qobject/qobject-json.c
> +++ b/qobject/qobject-json.c
> @@ -80,16 +80,13 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent);
> static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
> {
> ToJsonIterState *s = opaque;
> - int j;
>
> if (s->count) {
> qstring_append(s->str, s->pretty ? "," : ", ");
> }
>
> if (s->pretty) {
> - qstring_append(s->str, "\n");
> - for (j = 0 ; j < s->indent ; j++)
> - qstring_append(s->str, " ");
> + qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
> }
>
> qstring_append_json_string(s->str, key);
> @@ -102,16 +99,13 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
> static void to_json_list_iter(QObject *obj, void *opaque)
> {
> ToJsonIterState *s = opaque;
> - int j;
>
> if (s->count) {
> qstring_append(s->str, s->pretty ? "," : ", ");
> }
>
> if (s->pretty) {
> - qstring_append(s->str, "\n");
> - for (j = 0 ; j < s->indent ; j++)
> - qstring_append(s->str, " ");
> + qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
> }
>
> to_json(obj, s->str, s->pretty, s->indent);
> @@ -126,10 +120,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
> break;
> case QTYPE_QINT: {
> QInt *val = qobject_to_qint(obj);
> - char buffer[1024];
> -
> - snprintf(buffer, sizeof(buffer), "%" PRId64, qint_get_int(val));
> - qstring_append(str, buffer);
> + qstring_append_format(str, "%" PRId64, qint_get_int(val));
> break;
> }
> case QTYPE_QSTRING: {
> @@ -148,10 +139,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
> qstring_append(str, "{");
> qdict_iter(val, to_json_dict_iter, &s);
> if (pretty) {
> - int j;
> - qstring_append(str, "\n");
> - for (j = 0 ; j < indent ; j++)
> - qstring_append(str, " ");
> + qstring_append_format(str, "\n%*s", 4 * indent, "");
> }
> qstring_append(str, "}");
> break;
> @@ -167,10 +155,7 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
> qstring_append(str, "[");
> qlist_iter(val, (void *)to_json_list_iter, &s);
> if (pretty) {
> - int j;
> - qstring_append(str, "\n");
> - for (j = 0 ; j < indent ; j++)
> - qstring_append(str, " ");
> + qstring_append_format(str, "\n%*s", 4 * indent, "");
> }
> qstring_append(str, "]");
> break;
> diff --git a/qobject/qstring.c b/qobject/qstring.c
> index 618dd8f..0285d53 100644
> --- a/qobject/qstring.c
> +++ b/qobject/qstring.c
> @@ -90,12 +90,28 @@ void qstring_append(QString *qstring, const char *str)
> qstring->string[qstring->length] = 0;
> }
>
> -void qstring_append_int(QString *qstring, int64_t value)
> +void qstring_append_format(QString *qstring, const char *fmt, ...)
> {
> - char num[32];
> + va_list ap;
>
> - snprintf(num, sizeof(num), "%" PRId64, value);
> - qstring_append(qstring, num);
> + va_start(ap, fmt);
> + qstring_append_vformat(qstring, fmt, ap);
> + va_end(ap);
> +}
> +
> +void qstring_append_vformat(QString *qstring, const char *fmt, va_list ap)
> +{
> + va_list ap2;
> + int len;
> +
> + va_copy(ap2, ap);
> + len = vsnprintf(qstring->string + qstring->length, 0, fmt, ap2);
Why bother with the first argument here? NULL would do :)
> + assert(len >= 0);
> + va_end(ap2);
> +
> + capacity_increase(qstring, len);
> + vsnprintf(qstring->string + qstring->length, len + 1, fmt, ap);
> + qstring->length += len;
> }
>
> /**
> @@ -116,7 +132,6 @@ void qstring_append_json_string(QString *qstring, const char *raw)
> {
> const char *ptr = raw;
> int cp;
> - char buf[16];
> char *end;
>
> qstring_append(qstring, "\"");
> @@ -151,16 +166,14 @@ void qstring_append_json_string(QString *qstring, const char *raw)
> }
> if (cp > 0xFFFF) {
> /* beyond BMP; need a surrogate pair */
> - snprintf(buf, sizeof(buf), "\\u%04X\\u%04X",
> - 0xD800 + ((cp - 0x10000) >> 10),
> - 0xDC00 + ((cp - 0x10000) & 0x3FF));
> + qstring_append_format(qstring, "\\u%04X\\u%04X",
> + 0xD800 + ((cp - 0x10000) >> 10),
> + 0xDC00 + ((cp - 0x10000) & 0x3FF));
> } else if (cp < 0x20 || cp >= 0x7F) {
> - snprintf(buf, sizeof(buf), "\\u%04X", cp);
> + qstring_append_format(qstring, "\\u%04X", cp);
> } else {
> - buf[0] = cp;
> - buf[1] = 0;
> + qstring_append_chr(qstring, cp);
> }
> - qstring_append(qstring, buf);
> }
> };
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (5 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 06/18] qapi: Add qstring_append_format() Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 9:15 ` Markus Armbruster
2016-05-02 15:00 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor Eric Blake
` (11 subsequent siblings)
18 siblings, 2 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Michael Roth
We have several places that want to go from qapi to JSON; right now,
they have to create an intermediate QObject to do the work. That
also has the drawback that the JSON formatting of a QDict will
rearrange keys (according to a deterministic, but unpredictable,
hash), when humans have an easier time if dicts are produced in
the same order as the qapi type.
For these reasons, it is time to add a new JSON output visitor.
This patch just adds the basic visitor and tests that it works;
later patches will add pretty-printing, and convert clients to
use the visitor.
Design choices: Unlike the QMP output visitor, the JSON visitor
refuses to visit a required string with a NULL value (abort), as
well as a non-finite number (raises an error message). Reusing
QString to grow the contents means that we easily share code with
both qobject-json.c and qjson.c; although it might be nice to
enhance things to take an optional output callback function so
that the output can truly be streamed instead of collected in
memory.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: retitle, rebase to master, minor cleanups
v2: rebase to qapi subset E v8; add test of error outputting
infinity; use unsigned depth
---
include/qapi/visitor.h | 20 +-
include/qapi/json-output-visitor.h | 29 +++
qapi/json-output-visitor.c | 202 ++++++++++++++++++
tests/test-json-output-visitor.c | 418 +++++++++++++++++++++++++++++++++++++
qapi/Makefile.objs | 2 +-
tests/.gitignore | 1 +
tests/Makefile | 4 +
7 files changed, 665 insertions(+), 11 deletions(-)
create mode 100644 include/qapi/json-output-visitor.h
create mode 100644 qapi/json-output-visitor.c
create mode 100644 tests/test-json-output-visitor.c
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index a430c19..e8a4403 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -26,16 +26,16 @@
*
* There are three kinds of visitor classes: input visitors (QMP,
* string, and QemuOpts) parse an external representation and build
- * the corresponding QAPI graph, output visitors (QMP and string) take
- * a completed QAPI graph and generate an external representation, and
- * the dealloc visitor can take a QAPI graph (possibly partially
- * constructed) and recursively free its resources. While the dealloc
- * and QMP input/output visitors are general, the string and QemuOpts
- * visitors have some implementation limitations; see the
- * documentation for each visitor for more details on what it
- * supports. Also, see visitor-impl.h for the callback contracts
- * implemented by each visitor, and docs/qapi-code-gen.txt for more
- * about the QAPI code generator.
+ * the corresponding QAPI graph, output visitors (QMP, string, and
+ * JSON) take a completed QAPI graph and generate an external
+ * representation, and the dealloc visitor can take a QAPI graph
+ * (possibly partially constructed) and recursively free its
+ * resources. While the dealloc and QMP input/output visitors are
+ * general, the string and QemuOpts visitors have some implementation
+ * limitations; see the documentation for each visitor for more
+ * details on what it supports. Also, see visitor-impl.h for the
+ * callback contracts implemented by each visitor, and
+ * docs/qapi-code-gen.txt for more about the QAPI code generator.
*
* All QAPI types have a corresponding function with a signature
* roughly compatible with this:
diff --git a/include/qapi/json-output-visitor.h b/include/qapi/json-output-visitor.h
new file mode 100644
index 0000000..ac03da1
--- /dev/null
+++ b/include/qapi/json-output-visitor.h
@@ -0,0 +1,29 @@
+/*
+ * JSON Output Visitor
+ *
+ * Copyright (C) 2015-2016 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef JSON_OUTPUT_VISITOR_H
+#define JSON_OUTPUT_VISITOR_H
+
+#include "qapi/visitor.h"
+
+typedef struct JsonOutputVisitor JsonOutputVisitor;
+
+/*
+ * The JSON output visitor does not accept Infinity or NaN to
+ * visit_type_number().
+ */
+JsonOutputVisitor *json_output_visitor_new(void);
+void json_output_visitor_cleanup(JsonOutputVisitor *v);
+void json_output_visitor_reset(JsonOutputVisitor *v);
+
+char *json_output_get_string(JsonOutputVisitor *v);
+Visitor *json_output_get_visitor(JsonOutputVisitor *v);
+
+#endif
diff --git a/qapi/json-output-visitor.c b/qapi/json-output-visitor.c
new file mode 100644
index 0000000..3539db5
--- /dev/null
+++ b/qapi/json-output-visitor.c
@@ -0,0 +1,202 @@
+/*
+ * Convert QAPI to JSON
+ *
+ * Copyright (C) 2015-2016 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/json-output-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "qapi/qmp/qstring.h"
+#include "qapi/qmp/qobject-json.h"
+
+struct JsonOutputVisitor {
+ Visitor visitor;
+ QString *str;
+ bool comma;
+ unsigned int depth;
+};
+
+static JsonOutputVisitor *to_jov(Visitor *v)
+{
+ return container_of(v, JsonOutputVisitor, visitor);
+}
+
+static void json_output_name(JsonOutputVisitor *jov, const char *name)
+{
+ if (!jov->str) {
+ jov->str = qstring_new();
+ }
+ if (jov->comma) {
+ qstring_append(jov->str, ", ");
+ } else {
+ jov->comma = true;
+ }
+ if (name && jov->depth) {
+ qstring_append_json_string(jov->str, name);
+ qstring_append(jov->str, ": ");
+ }
+}
+
+static void json_output_start_struct(Visitor *v, const char *name, void **obj,
+ size_t unused, Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append(jov->str, "{");
+ jov->comma = false;
+ jov->depth++;
+}
+
+static void json_output_end_struct(Visitor *v)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ assert(jov->depth);
+ jov->depth--;
+ qstring_append(jov->str, "}");
+ jov->comma = true;
+}
+
+static void json_output_start_list(Visitor *v, const char *name,
+ GenericList **listp, size_t size,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append(jov->str, "[");
+ jov->comma = false;
+ jov->depth++;
+}
+
+static GenericList *json_output_next_list(Visitor *v, GenericList *tail,
+ size_t size)
+{
+ return tail->next;
+}
+
+static void json_output_end_list(Visitor *v)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ assert(jov->depth);
+ jov->depth--;
+ qstring_append(jov->str, "]");
+ jov->comma = true;
+}
+
+static void json_output_type_int64(Visitor *v, const char *name, int64_t *obj,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append_format(jov->str, "%" PRId64, *obj);
+}
+
+static void json_output_type_uint64(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append_format(jov->str, "%" PRIu64, *obj);
+}
+
+static void json_output_type_bool(Visitor *v, const char *name, bool *obj,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append(jov->str, *obj ? "true" : "false");
+}
+
+static void json_output_type_str(Visitor *v, const char *name, char **obj,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ assert(*obj);
+ json_output_name(jov, name);
+ qstring_append_json_string(jov->str, *obj);
+}
+
+static void json_output_type_number(Visitor *v, const char *name, double *obj,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append_json_number(jov->str, *obj, errp);
+}
+
+static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
+ Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ QString *str = qobject_to_json(*obj);
+ assert(str);
+ json_output_name(jov, name);
+ qstring_append(jov->str, qstring_get_str(str));
+ QDECREF(str);
+}
+
+static void json_output_type_null(Visitor *v, const char *name, Error **errp)
+{
+ JsonOutputVisitor *jov = to_jov(v);
+ json_output_name(jov, name);
+ qstring_append(jov->str, "null");
+}
+
+/* Finish building, and return the resulting string. Will not be NULL. */
+char *json_output_get_string(JsonOutputVisitor *jov)
+{
+ char *result;
+
+ assert(!jov->depth);
+ result = g_strdup(qstring_get_str(jov->str));
+ json_output_visitor_reset(jov);
+ return result;
+}
+
+Visitor *json_output_get_visitor(JsonOutputVisitor *v)
+{
+ return &v->visitor;
+}
+
+void json_output_visitor_reset(JsonOutputVisitor *v)
+{
+ QDECREF(v->str);
+ v->str = NULL;
+ v->depth = 0;
+ v->comma = false;
+}
+
+void json_output_visitor_cleanup(JsonOutputVisitor *v)
+{
+ json_output_visitor_reset(v);
+ g_free(v);
+}
+
+JsonOutputVisitor *json_output_visitor_new(void)
+{
+ JsonOutputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->visitor.type = VISITOR_OUTPUT;
+ v->visitor.start_struct = json_output_start_struct;
+ v->visitor.end_struct = json_output_end_struct;
+ v->visitor.start_list = json_output_start_list;
+ v->visitor.next_list = json_output_next_list;
+ v->visitor.end_list = json_output_end_list;
+ v->visitor.type_int64 = json_output_type_int64;
+ v->visitor.type_uint64 = json_output_type_uint64;
+ v->visitor.type_bool = json_output_type_bool;
+ v->visitor.type_str = json_output_type_str;
+ v->visitor.type_number = json_output_type_number;
+ v->visitor.type_any = json_output_type_any;
+ v->visitor.type_null = json_output_type_null;
+
+ return v;
+}
diff --git a/tests/test-json-output-visitor.c b/tests/test-json-output-visitor.c
new file mode 100644
index 0000000..57fe1c6
--- /dev/null
+++ b/tests/test-json-output-visitor.c
@@ -0,0 +1,418 @@
+/*
+ * JSON Output Visitor unit-tests.
+ *
+ * Copyright (C) 2015-2016 Red Hat Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include <glib.h>
+#include <math.h>
+
+#include "qemu-common.h"
+#include "qapi/json-output-visitor.h"
+#include "test-qapi-types.h"
+#include "test-qapi-visit.h"
+#include "qapi/qmp/types.h"
+
+typedef struct TestOutputVisitorData {
+ JsonOutputVisitor *jov;
+ Visitor *ov;
+} TestOutputVisitorData;
+
+static void visitor_output_setup(TestOutputVisitorData *data,
+ const void *unused)
+{
+ data->jov = json_output_visitor_new();
+ g_assert(data->jov);
+
+ data->ov = json_output_get_visitor(data->jov);
+ g_assert(data->ov);
+}
+
+static void visitor_output_teardown(TestOutputVisitorData *data,
+ const void *unused)
+{
+ json_output_visitor_cleanup(data->jov);
+ data->jov = NULL;
+ data->ov = NULL;
+}
+
+static void test_visitor_out_int(TestOutputVisitorData *data,
+ const void *unused)
+{
+ int64_t value = -42;
+ char *out;
+
+ visit_type_int(data->ov, NULL, &value, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "-42");
+ g_free(out);
+}
+
+static void test_visitor_out_bool(TestOutputVisitorData *data,
+ const void *unused)
+{
+ bool value = true;
+ char *out;
+
+ visit_type_bool(data->ov, NULL, &value, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "true");
+ g_free(out);
+}
+
+static void test_visitor_out_number(TestOutputVisitorData *data,
+ const void *unused)
+{
+ double value = 3.14;
+ char *out;
+ Error *err = NULL;
+
+ visit_type_number(data->ov, NULL, &value, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "3.14");
+ g_free(out);
+
+ /* JSON requires finite numbers */
+ value = INFINITY;
+ visit_type_number(data->ov, NULL, &value, &err);
+ g_assert(err);
+ error_free(err);
+}
+
+static void test_visitor_out_string(TestOutputVisitorData *data,
+ const void *unused)
+{
+ char *string = (char *) "Q E M U";
+ char *out;
+
+ visit_type_str(data->ov, NULL, &string, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "\"Q E M U\"");
+ g_free(out);
+}
+
+static void test_visitor_out_enum(TestOutputVisitorData *data,
+ const void *unused)
+{
+ char *out;
+ char *tmp;
+ EnumOne i;
+ size_t len;
+
+ for (i = 0; i < ENUM_ONE__MAX; i++) {
+ visit_type_EnumOne(data->ov, "unused", &i, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert(*out == '"');
+ len = strlen(out);
+ g_assert(out[len - 1] == '"');
+ tmp = out + 1;
+ out[len - 1] = 0;
+ g_assert_cmpstr(tmp, ==, EnumOne_lookup[i]);
+ g_free(out);
+ }
+}
+
+static void test_visitor_out_enum_errors(TestOutputVisitorData *data,
+ const void *unused)
+{
+ EnumOne i, bad_values[] = { ENUM_ONE__MAX, -1 };
+ Error *err;
+
+ for (i = 0; i < ARRAY_SIZE(bad_values) ; i++) {
+ err = NULL;
+ visit_type_EnumOne(data->ov, "unused", &bad_values[i], &err);
+ g_assert(err);
+ error_free(err);
+ }
+}
+
+
+static void test_visitor_out_struct(TestOutputVisitorData *data,
+ const void *unused)
+{
+ TestStruct test_struct = { .integer = 42,
+ .boolean = false,
+ .string = (char *) "foo"};
+ TestStruct *p = &test_struct;
+ char *out;
+
+ visit_type_TestStruct(data->ov, NULL, &p, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==,
+ "{"
+ "\"integer\": 42, "
+ "\"boolean\": false, "
+ "\"string\": \"foo\""
+ "}");
+ g_free(out);
+}
+
+static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
+ const void *unused)
+{
+ int64_t value = 42;
+ UserDefTwo *ud2;
+ const char *string = "user def string";
+ const char *strings[] = { "forty two", "forty three", "forty four",
+ "forty five" };
+ char *out;
+
+ ud2 = g_malloc0(sizeof(*ud2));
+ ud2->string0 = g_strdup(strings[0]);
+
+ ud2->dict1 = g_malloc0(sizeof(*ud2->dict1));
+ ud2->dict1->string1 = g_strdup(strings[1]);
+
+ ud2->dict1->dict2 = g_malloc0(sizeof(*ud2->dict1->dict2));
+ ud2->dict1->dict2->userdef = g_new0(UserDefOne, 1);
+ ud2->dict1->dict2->userdef->string = g_strdup(string);
+ ud2->dict1->dict2->userdef->integer = value;
+ ud2->dict1->dict2->string = g_strdup(strings[2]);
+
+ ud2->dict1->dict3 = g_malloc0(sizeof(*ud2->dict1->dict3));
+ ud2->dict1->has_dict3 = true;
+ ud2->dict1->dict3->userdef = g_new0(UserDefOne, 1);
+ ud2->dict1->dict3->userdef->string = g_strdup(string);
+ ud2->dict1->dict3->userdef->integer = value;
+ ud2->dict1->dict3->string = g_strdup(strings[3]);
+
+ visit_type_UserDefTwo(data->ov, "unused", &ud2, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==,
+ "{"
+ "\"string0\": \"forty two\", "
+ "\"dict1\": {"
+ "\"string1\": \"forty three\", "
+ "\"dict2\": {"
+ "\"userdef\": {"
+ "\"integer\": 42, "
+ "\"string\": \"user def string\""
+ "}, "
+ "\"string\": \"forty four\""
+ "}, "
+ "\"dict3\": {"
+ "\"userdef\": {"
+ "\"integer\": 42, "
+ "\"string\": \"user def string\""
+ "}, "
+ "\"string\": \"forty five\""
+ "}"
+ "}"
+ "}");
+ qapi_free_UserDefTwo(ud2);
+ g_free(out);
+}
+
+static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
+ const void *unused)
+{
+ EnumOne bad_values[] = { ENUM_ONE__MAX, -1 };
+ UserDefOne u = { .string = (char *)"" };
+ UserDefOne *pu = &u;
+ Error *err;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(bad_values) ; i++) {
+ err = NULL;
+ u.has_enum1 = true;
+ u.enum1 = bad_values[i];
+ visit_type_UserDefOne(data->ov, "unused", &pu, &err);
+ g_assert(err);
+ error_free(err);
+ json_output_visitor_reset(data->jov);
+ }
+}
+
+
+static void test_visitor_out_list(TestOutputVisitorData *data,
+ const void *unused)
+{
+ const char *value_str = "list value";
+ TestStructList *p, *head = NULL;
+ const int max_items = 10;
+ bool value_bool = true;
+ int value_int = 10;
+ int i;
+ char *out;
+
+ for (i = 0; i < max_items; i++) {
+ p = g_malloc0(sizeof(*p));
+ p->value = g_malloc0(sizeof(*p->value));
+ p->value->integer = value_int + (max_items - i - 1);
+ p->value->boolean = value_bool;
+ p->value->string = g_strdup(value_str);
+
+ p->next = head;
+ head = p;
+ }
+
+ visit_type_TestStructList(data->ov, NULL, &head, &error_abort);
+
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==,
+ "["
+ "{\"integer\": 10, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 11, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 12, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 13, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 14, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 15, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 16, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 17, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 18, \"boolean\": true, "
+ "\"string\": \"list value\"}, "
+ "{\"integer\": 19, \"boolean\": true, "
+ "\"string\": \"list value\"}"
+ "]");
+ qapi_free_TestStructList(head);
+ g_free(out);
+}
+
+static void test_visitor_out_any(TestOutputVisitorData *data,
+ const void *unused)
+{
+ QObject *qobj;
+ QDict *qdict;
+ char *out;
+
+ qobj = QOBJECT(qint_from_int(-42));
+ visit_type_any(data->ov, NULL, &qobj, &error_abort);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "-42");
+ qobject_decref(qobj);
+ g_free(out);
+
+ qdict = qdict_new();
+ qdict_put(qdict, "integer", qint_from_int(-42));
+ qdict_put(qdict, "boolean", qbool_from_bool(true));
+ qdict_put(qdict, "string", qstring_from_str("foo"));
+ qobj = QOBJECT(qdict);
+ visit_type_any(data->ov, NULL, &qobj, &error_abort);
+ qobject_decref(qobj);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==,
+ "{"
+ "\"integer\": -42, "
+ "\"boolean\": true, "
+ "\"string\": \"foo\""
+ "}");
+ g_free(out);
+}
+
+static void test_visitor_out_union_flat(TestOutputVisitorData *data,
+ const void *unused)
+{
+ char *out;
+ UserDefFlatUnion *tmp = g_malloc0(sizeof(UserDefFlatUnion));
+
+ tmp->enum1 = ENUM_ONE_VALUE1;
+ tmp->string = g_strdup("str");
+ tmp->integer = 41;
+ tmp->u.value1.boolean = true;
+
+ visit_type_UserDefFlatUnion(data->ov, NULL, &tmp, &error_abort);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==,
+ "{"
+ "\"integer\": 41, "
+ "\"string\": \"str\", "
+ "\"enum1\": \"value1\", "
+ "\"boolean\": true"
+ "}");
+ g_free(out);
+ qapi_free_UserDefFlatUnion(tmp);
+}
+
+static void test_visitor_out_alternate(TestOutputVisitorData *data,
+ const void *unused)
+{
+ UserDefAlternate *tmp;
+ char *out;
+
+ tmp = g_new0(UserDefAlternate, 1);
+ tmp->type = QTYPE_QINT;
+ tmp->u.i = 42;
+
+ visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "42");
+ g_free(out);
+ qapi_free_UserDefAlternate(tmp);
+
+ tmp = g_new0(UserDefAlternate, 1);
+ tmp->type = QTYPE_QSTRING;
+ tmp->u.s = g_strdup("hello");
+
+ visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "\"hello\"");
+ g_free(out);
+ qapi_free_UserDefAlternate(tmp);
+}
+
+static void test_visitor_out_null(TestOutputVisitorData *data,
+ const void *unused)
+{
+ char *out;
+
+ visit_type_null(data->ov, NULL, &error_abort);
+ out = json_output_get_string(data->jov);
+ g_assert_cmpstr(out, ==, "null");
+ g_free(out);
+}
+
+static void output_visitor_test_add(const char *testpath,
+ void (*test_func)(TestOutputVisitorData *,
+ const void *))
+{
+ g_test_add(testpath, TestOutputVisitorData, NULL, visitor_output_setup,
+ test_func, visitor_output_teardown);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ output_visitor_test_add("/visitor/json/int", test_visitor_out_int);
+ output_visitor_test_add("/visitor/json/bool", test_visitor_out_bool);
+ output_visitor_test_add("/visitor/json/number", test_visitor_out_number);
+ output_visitor_test_add("/visitor/json/string", test_visitor_out_string);
+ output_visitor_test_add("/visitor/json/enum", test_visitor_out_enum);
+ output_visitor_test_add("/visitor/json/enum-errors",
+ test_visitor_out_enum_errors);
+ output_visitor_test_add("/visitor/json/struct", test_visitor_out_struct);
+ output_visitor_test_add("/visitor/json/struct-nested",
+ test_visitor_out_struct_nested);
+ output_visitor_test_add("/visitor/json/struct-errors",
+ test_visitor_out_struct_errors);
+ output_visitor_test_add("/visitor/json/list", test_visitor_out_list);
+ output_visitor_test_add("/visitor/json/any", test_visitor_out_any);
+ output_visitor_test_add("/visitor/json/union-flat",
+ test_visitor_out_union_flat);
+ output_visitor_test_add("/visitor/json/alternate",
+ test_visitor_out_alternate);
+ output_visitor_test_add("/visitor/json/null", test_visitor_out_null);
+
+ g_test_run();
+
+ return 0;
+}
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 2278970..b60e11b 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,6 +1,6 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
-util-obj-y += opts-visitor.o
+util-obj-y += opts-visitor.o json-output-visitor.o
util-obj-y += qmp-event.o
util-obj-y += qapi-util.o
diff --git a/tests/.gitignore b/tests/.gitignore
index 2f8c7ea..c2aad79 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -40,6 +40,7 @@ test-io-channel-file.txt
test-io-channel-socket
test-io-channel-tls
test-io-task
+test-json-output-visitor
test-logging
test-mul64
test-opts-visitor
diff --git a/tests/Makefile b/tests/Makefile
index f71ed1c..0b5a7a8 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -22,6 +22,8 @@ check-unit-y += tests/check-qobject-json$(EXESUF)
gcov-files-check-qobject-json-y = qobject/qobject-json.c
check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
+check-unit-y += tests/test-json-output-visitor$(EXESUF)
+gcov-files-test-json-output-visitor-y = qapi/json-output-visitor.c
check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
check-unit-y += tests/test-qmp-input-strict$(EXESUF)
@@ -388,6 +390,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qobject-json.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
+ tests/test-json-output-visitor.o \
tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
tests/test-qmp-commands.o tests/test-visitor-serialization.o \
tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
@@ -478,6 +481,7 @@ tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(
tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)
tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
+tests/test-json-output-visitor$(EXESUF): tests/test-json-output-visitor.o $(test-qapi-obj-y)
tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor Eric Blake
@ 2016-05-02 9:15 ` Markus Armbruster
2016-05-02 15:11 ` Eric Blake
2016-05-18 15:16 ` Eric Blake
2016-05-02 15:00 ` Markus Armbruster
1 sibling, 2 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 9:15 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Michael Roth
Title: s/json/JSON/
Eric Blake <eblake@redhat.com> writes:
> We have several places that want to go from qapi to JSON; right now,
> they have to create an intermediate QObject to do the work. That
> also has the drawback that the JSON formatting of a QDict will
> rearrange keys (according to a deterministic, but unpredictable,
> hash), when humans have an easier time if dicts are produced in
> the same order as the qapi type.
There's a drawback, though: more code.
Could the JSON output visitor replace the QMP output visitor?
Aside: the QMP visitors are really QObject visitors.
> For these reasons, it is time to add a new JSON output visitor.
> This patch just adds the basic visitor and tests that it works;
> later patches will add pretty-printing, and convert clients to
> use the visitor.
>
> Design choices: Unlike the QMP output visitor, the JSON visitor
> refuses to visit a required string with a NULL value (abort), as
> well as a non-finite number (raises an error message). Reusing
> QString to grow the contents means that we easily share code with
> both qobject-json.c and qjson.c; although it might be nice to
> enhance things to take an optional output callback function so
> that the output can truly be streamed instead of collected in
> memory.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: retitle, rebase to master, minor cleanups
> v2: rebase to qapi subset E v8; add test of error outputting
> infinity; use unsigned depth
> ---
> include/qapi/visitor.h | 20 +-
> include/qapi/json-output-visitor.h | 29 +++
> qapi/json-output-visitor.c | 202 ++++++++++++++++++
> tests/test-json-output-visitor.c | 418 +++++++++++++++++++++++++++++++++++++
> qapi/Makefile.objs | 2 +-
> tests/.gitignore | 1 +
> tests/Makefile | 4 +
> 7 files changed, 665 insertions(+), 11 deletions(-)
> create mode 100644 include/qapi/json-output-visitor.h
> create mode 100644 qapi/json-output-visitor.c
> create mode 100644 tests/test-json-output-visitor.c
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index a430c19..e8a4403 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -26,16 +26,16 @@
> *
> * There are three kinds of visitor classes: input visitors (QMP,
> * string, and QemuOpts) parse an external representation and build
> - * the corresponding QAPI graph, output visitors (QMP and string) take
> - * a completed QAPI graph and generate an external representation, and
> - * the dealloc visitor can take a QAPI graph (possibly partially
> - * constructed) and recursively free its resources. While the dealloc
> - * and QMP input/output visitors are general, the string and QemuOpts
> - * visitors have some implementation limitations; see the
> - * documentation for each visitor for more details on what it
> - * supports. Also, see visitor-impl.h for the callback contracts
> - * implemented by each visitor, and docs/qapi-code-gen.txt for more
> - * about the QAPI code generator.
> + * the corresponding QAPI graph, output visitors (QMP, string, and
> + * JSON) take a completed QAPI graph and generate an external
> + * representation, and the dealloc visitor can take a QAPI graph
> + * (possibly partially constructed) and recursively free its
> + * resources. While the dealloc and QMP input/output visitors are
> + * general, the string and QemuOpts visitors have some implementation
> + * limitations; see the documentation for each visitor for more
> + * details on what it supports. Also, see visitor-impl.h for the
> + * callback contracts implemented by each visitor, and
> + * docs/qapi-code-gen.txt for more about the QAPI code generator.
> *
> * All QAPI types have a corresponding function with a signature
> * roughly compatible with this:
> diff --git a/include/qapi/json-output-visitor.h b/include/qapi/json-output-visitor.h
> new file mode 100644
> index 0000000..ac03da1
> --- /dev/null
> +++ b/include/qapi/json-output-visitor.h
> @@ -0,0 +1,29 @@
> +/*
> + * JSON Output Visitor
> + *
> + * Copyright (C) 2015-2016 Red Hat, Inc.
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
> + * See the COPYING.LIB file in the top-level directory.
> + *
> + */
> +
> +#ifndef JSON_OUTPUT_VISITOR_H
> +#define JSON_OUTPUT_VISITOR_H
> +
> +#include "qapi/visitor.h"
> +
> +typedef struct JsonOutputVisitor JsonOutputVisitor;
> +
> +/*
> + * The JSON output visitor does not accept Infinity or NaN to
> + * visit_type_number().
> + */
> +JsonOutputVisitor *json_output_visitor_new(void);
> +void json_output_visitor_cleanup(JsonOutputVisitor *v);
> +void json_output_visitor_reset(JsonOutputVisitor *v);
Hmm. Why is "reset" not a Visitor method?
I think this would let us put the things enforced by your "qmp: Tighten
output visitor rules" in the Visitor contract.
> +
> +char *json_output_get_string(JsonOutputVisitor *v);
> +Visitor *json_output_get_visitor(JsonOutputVisitor *v);
> +
> +#endif
> diff --git a/qapi/json-output-visitor.c b/qapi/json-output-visitor.c
> new file mode 100644
> index 0000000..3539db5
> --- /dev/null
> +++ b/qapi/json-output-visitor.c
> @@ -0,0 +1,202 @@
> +/*
> + * Convert QAPI to JSON
> + *
> + * Copyright (C) 2015-2016 Red Hat, Inc.
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
> + * See the COPYING.LIB file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/json-output-visitor.h"
> +#include "qapi/visitor-impl.h"
> +#include "qemu/queue.h"
> +#include "qemu-common.h"
qemu/queue.h and qemu-common.h are superfluous.
> +#include "qapi/qmp/qstring.h"
> +#include "qapi/qmp/qobject-json.h"
> +
> +struct JsonOutputVisitor {
> + Visitor visitor;
> + QString *str;
> + bool comma;
> + unsigned int depth;
> +};
> +
> +static JsonOutputVisitor *to_jov(Visitor *v)
> +{
> + return container_of(v, JsonOutputVisitor, visitor);
> +}
> +
> +static void json_output_name(JsonOutputVisitor *jov, const char *name)
This is called for every value, by its visit_start_FOO() or
visit_type_FOO() function. Quote visitor.h:
* The @name parameter of visit_type_FOO() describes the relation
* between this QAPI value and its parent container. When visiting
* the root of a tree, @name is ignored; when visiting a member of an
* object, @name is the key associated with the value; and when
* visiting a member of a list, @name is NULL.
Aside: it should mention visit_start_FOO() in addition to
visit_type_FOO().
> +{
> + if (!jov->str) {
> + jov->str = qstring_new();
This happens on the first call after creation or reset of jov.
If you call json_output_get_string() after an empty visit, it chokes on
null jov->str. Could be declared a feature. The Visitor contract is
silent on it, but the QMP output visitor rejects it since "qmp: Tighten
output visitor rules".
Regardless: why not create jov->str in json_output_visitor_new(), and
truncate it in json_output_visitor_reset()?
To retain the "feature", you'd assert qstring_get_length(jov->str).
> + }
> + if (jov->comma) {
> + qstring_append(jov->str, ", ");
Must be in an array or object.
We get here only when called multiple times without a reset of
jov->comma in between, i.e. a call of json_output_start_struct(),
json_output_start_list(), or json_output_visitor_reset().
In other words, if we get here outside an array or object, the caller
screwed up. Okay, although the Visitor contract is silent on it.
> + } else {
> + jov->comma = true;
> + }
> + if (name && jov->depth) {
> + qstring_append_json_string(jov->str, name);
> + qstring_append(jov->str, ": ");
Must be in an object. Covered by the Visitor contract quoted above.
> + }
> +}
> +
> +static void json_output_start_struct(Visitor *v, const char *name, void **obj,
> + size_t unused, Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
Blank line between declarations and statements, please. More of the
same below.
> + json_output_name(jov, name);
> + qstring_append(jov->str, "{");
> + jov->comma = false;
> + jov->depth++;
> +}
> +
> +static void json_output_end_struct(Visitor *v)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + assert(jov->depth);
> + jov->depth--;
> + qstring_append(jov->str, "}");
> + jov->comma = true;
> +}
> +
> +static void json_output_start_list(Visitor *v, const char *name,
> + GenericList **listp, size_t size,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append(jov->str, "[");
> + jov->comma = false;
> + jov->depth++;
> +}
> +
> +static GenericList *json_output_next_list(Visitor *v, GenericList *tail,
> + size_t size)
> +{
> + return tail->next;
> +}
> +
> +static void json_output_end_list(Visitor *v)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + assert(jov->depth);
> + jov->depth--;
> + qstring_append(jov->str, "]");
> + jov->comma = true;
> +}
The nesting checks are less thorough than the QMP visitor's, because we
don't use a stack. Okay.
> +
> +static void json_output_type_int64(Visitor *v, const char *name, int64_t *obj,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append_format(jov->str, "%" PRId64, *obj);
> +}
> +
> +static void json_output_type_uint64(Visitor *v, const char *name,
> + uint64_t *obj, Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append_format(jov->str, "%" PRIu64, *obj);
> +}
> +
> +static void json_output_type_bool(Visitor *v, const char *name, bool *obj,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append(jov->str, *obj ? "true" : "false");
> +}
> +
> +static void json_output_type_str(Visitor *v, const char *name, char **obj,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + assert(*obj);
> + json_output_name(jov, name);
> + qstring_append_json_string(jov->str, *obj);
> +}
> +
> +static void json_output_type_number(Visitor *v, const char *name, double *obj,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append_json_number(jov->str, *obj, errp);
> +}
> +
> +static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
> + Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + QString *str = qobject_to_json(*obj);
> + assert(str);
Can't happen.
> + json_output_name(jov, name);
> + qstring_append(jov->str, qstring_get_str(str));
> + QDECREF(str);
> +}
> +
> +static void json_output_type_null(Visitor *v, const char *name, Error **errp)
> +{
> + JsonOutputVisitor *jov = to_jov(v);
> + json_output_name(jov, name);
> + qstring_append(jov->str, "null");
> +}
> +
> +/* Finish building, and return the resulting string. Will not be NULL. */
> +char *json_output_get_string(JsonOutputVisitor *jov)
> +{
> + char *result;
> +
> + assert(!jov->depth);
> + result = g_strdup(qstring_get_str(jov->str));
> + json_output_visitor_reset(jov);
Could avoid the strdup() if we wanted to. Needs a way to convert
jov->str to char * destructively, like g_string_free() can do for a
GString. Your choice.
> + return result;
> +}
> +
> +Visitor *json_output_get_visitor(JsonOutputVisitor *v)
> +{
> + return &v->visitor;
> +}
> +
> +void json_output_visitor_reset(JsonOutputVisitor *v)
> +{
> + QDECREF(v->str);
> + v->str = NULL;
> + v->depth = 0;
> + v->comma = false;
> +}
> +
> +void json_output_visitor_cleanup(JsonOutputVisitor *v)
> +{
> + json_output_visitor_reset(v);
> + g_free(v);
> +}
> +
> +JsonOutputVisitor *json_output_visitor_new(void)
> +{
> + JsonOutputVisitor *v;
> +
> + v = g_malloc0(sizeof(*v));
> +
> + v->visitor.type = VISITOR_OUTPUT;
> + v->visitor.start_struct = json_output_start_struct;
> + v->visitor.end_struct = json_output_end_struct;
> + v->visitor.start_list = json_output_start_list;
> + v->visitor.next_list = json_output_next_list;
> + v->visitor.end_list = json_output_end_list;
> + v->visitor.type_int64 = json_output_type_int64;
> + v->visitor.type_uint64 = json_output_type_uint64;
> + v->visitor.type_bool = json_output_type_bool;
> + v->visitor.type_str = json_output_type_str;
> + v->visitor.type_number = json_output_type_number;
> + v->visitor.type_any = json_output_type_any;
> + v->visitor.type_null = json_output_type_null;
> +
> + return v;
> +}
> diff --git a/tests/test-json-output-visitor.c b/tests/test-json-output-visitor.c
> new file mode 100644
> index 0000000..57fe1c6
> --- /dev/null
> +++ b/tests/test-json-output-visitor.c
> @@ -0,0 +1,418 @@
> +/*
> + * JSON Output Visitor unit-tests.
> + *
> + * Copyright (C) 2015-2016 Red Hat Inc.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include <glib.h>
> +#include <math.h>
> +
> +#include "qemu-common.h"
> +#include "qapi/json-output-visitor.h"
> +#include "test-qapi-types.h"
> +#include "test-qapi-visit.h"
> +#include "qapi/qmp/types.h"
> +
> +typedef struct TestOutputVisitorData {
> + JsonOutputVisitor *jov;
> + Visitor *ov;
> +} TestOutputVisitorData;
> +
> +static void visitor_output_setup(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + data->jov = json_output_visitor_new();
> + g_assert(data->jov);
> +
> + data->ov = json_output_get_visitor(data->jov);
> + g_assert(data->ov);
> +}
> +
> +static void visitor_output_teardown(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + json_output_visitor_cleanup(data->jov);
> + data->jov = NULL;
> + data->ov = NULL;
> +}
> +
> +static void test_visitor_out_int(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + int64_t value = -42;
> + char *out;
> +
> + visit_type_int(data->ov, NULL, &value, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "-42");
> + g_free(out);
> +}
> +
> +static void test_visitor_out_bool(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + bool value = true;
> + char *out;
> +
> + visit_type_bool(data->ov, NULL, &value, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "true");
> + g_free(out);
> +}
> +
> +static void test_visitor_out_number(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + double value = 3.14;
> + char *out;
> + Error *err = NULL;
> +
> + visit_type_number(data->ov, NULL, &value, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "3.14");
> + g_free(out);
> +
> + /* JSON requires finite numbers */
> + value = INFINITY;
> + visit_type_number(data->ov, NULL, &value, &err);
> + g_assert(err);
> + error_free(err);
error_free_or_abort()
> +}
> +
> +static void test_visitor_out_string(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + char *string = (char *) "Q E M U";
> + char *out;
> +
> + visit_type_str(data->ov, NULL, &string, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "\"Q E M U\"");
> + g_free(out);
> +}
> +
> +static void test_visitor_out_enum(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + char *out;
> + char *tmp;
> + EnumOne i;
> + size_t len;
> +
> + for (i = 0; i < ENUM_ONE__MAX; i++) {
> + visit_type_EnumOne(data->ov, "unused", &i, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert(*out == '"');
> + len = strlen(out);
> + g_assert(out[len - 1] == '"');
> + tmp = out + 1;
> + out[len - 1] = 0;
> + g_assert_cmpstr(tmp, ==, EnumOne_lookup[i]);
> + g_free(out);
Unlike in test-qmp-output-visitor.c, you don't need
qmp_output_visitor_reset() here, because json_output_get_string() does
it automatically.
Is the difference between json_output_get_string() and
qmp_output_get_qobject() a good idea?
> + }
> +}
> +
> +static void test_visitor_out_enum_errors(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + EnumOne i, bad_values[] = { ENUM_ONE__MAX, -1 };
> + Error *err;
> +
> + for (i = 0; i < ARRAY_SIZE(bad_values) ; i++) {
> + err = NULL;
> + visit_type_EnumOne(data->ov, "unused", &bad_values[i], &err);
> + g_assert(err);
> + error_free(err);
error_free_or_abort()
Why no json_output_visitor_reset() here?
Running out of time, skipping the rest of the test for now.
> + }
> +}
> +
> +
> +static void test_visitor_out_struct(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + TestStruct test_struct = { .integer = 42,
> + .boolean = false,
> + .string = (char *) "foo"};
> + TestStruct *p = &test_struct;
> + char *out;
> +
> + visit_type_TestStruct(data->ov, NULL, &p, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==,
> + "{"
> + "\"integer\": 42, "
> + "\"boolean\": false, "
> + "\"string\": \"foo\""
> + "}");
> + g_free(out);
> +}
> +
> +static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + int64_t value = 42;
> + UserDefTwo *ud2;
> + const char *string = "user def string";
> + const char *strings[] = { "forty two", "forty three", "forty four",
> + "forty five" };
> + char *out;
> +
> + ud2 = g_malloc0(sizeof(*ud2));
> + ud2->string0 = g_strdup(strings[0]);
> +
> + ud2->dict1 = g_malloc0(sizeof(*ud2->dict1));
> + ud2->dict1->string1 = g_strdup(strings[1]);
> +
> + ud2->dict1->dict2 = g_malloc0(sizeof(*ud2->dict1->dict2));
> + ud2->dict1->dict2->userdef = g_new0(UserDefOne, 1);
> + ud2->dict1->dict2->userdef->string = g_strdup(string);
> + ud2->dict1->dict2->userdef->integer = value;
> + ud2->dict1->dict2->string = g_strdup(strings[2]);
> +
> + ud2->dict1->dict3 = g_malloc0(sizeof(*ud2->dict1->dict3));
> + ud2->dict1->has_dict3 = true;
> + ud2->dict1->dict3->userdef = g_new0(UserDefOne, 1);
> + ud2->dict1->dict3->userdef->string = g_strdup(string);
> + ud2->dict1->dict3->userdef->integer = value;
> + ud2->dict1->dict3->string = g_strdup(strings[3]);
> +
> + visit_type_UserDefTwo(data->ov, "unused", &ud2, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==,
> + "{"
> + "\"string0\": \"forty two\", "
> + "\"dict1\": {"
> + "\"string1\": \"forty three\", "
> + "\"dict2\": {"
> + "\"userdef\": {"
> + "\"integer\": 42, "
> + "\"string\": \"user def string\""
> + "}, "
> + "\"string\": \"forty four\""
> + "}, "
> + "\"dict3\": {"
> + "\"userdef\": {"
> + "\"integer\": 42, "
> + "\"string\": \"user def string\""
> + "}, "
> + "\"string\": \"forty five\""
> + "}"
> + "}"
> + "}");
> + qapi_free_UserDefTwo(ud2);
> + g_free(out);
> +}
> +
> +static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + EnumOne bad_values[] = { ENUM_ONE__MAX, -1 };
> + UserDefOne u = { .string = (char *)"" };
> + UserDefOne *pu = &u;
> + Error *err;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(bad_values) ; i++) {
> + err = NULL;
> + u.has_enum1 = true;
> + u.enum1 = bad_values[i];
> + visit_type_UserDefOne(data->ov, "unused", &pu, &err);
> + g_assert(err);
> + error_free(err);
error_free_or_abort()
> + json_output_visitor_reset(data->jov);
> + }
> +}
> +
> +
> +static void test_visitor_out_list(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + const char *value_str = "list value";
> + TestStructList *p, *head = NULL;
> + const int max_items = 10;
> + bool value_bool = true;
> + int value_int = 10;
> + int i;
> + char *out;
> +
> + for (i = 0; i < max_items; i++) {
> + p = g_malloc0(sizeof(*p));
> + p->value = g_malloc0(sizeof(*p->value));
> + p->value->integer = value_int + (max_items - i - 1);
> + p->value->boolean = value_bool;
> + p->value->string = g_strdup(value_str);
> +
> + p->next = head;
> + head = p;
> + }
> +
> + visit_type_TestStructList(data->ov, NULL, &head, &error_abort);
> +
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==,
> + "["
> + "{\"integer\": 10, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 11, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 12, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 13, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 14, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 15, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 16, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 17, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 18, \"boolean\": true, "
> + "\"string\": \"list value\"}, "
> + "{\"integer\": 19, \"boolean\": true, "
> + "\"string\": \"list value\"}"
> + "]");
> + qapi_free_TestStructList(head);
> + g_free(out);
> +}
> +
> +static void test_visitor_out_any(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + QObject *qobj;
> + QDict *qdict;
> + char *out;
> +
> + qobj = QOBJECT(qint_from_int(-42));
> + visit_type_any(data->ov, NULL, &qobj, &error_abort);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "-42");
> + qobject_decref(qobj);
> + g_free(out);
> +
> + qdict = qdict_new();
> + qdict_put(qdict, "integer", qint_from_int(-42));
> + qdict_put(qdict, "boolean", qbool_from_bool(true));
> + qdict_put(qdict, "string", qstring_from_str("foo"));
> + qobj = QOBJECT(qdict);
> + visit_type_any(data->ov, NULL, &qobj, &error_abort);
> + qobject_decref(qobj);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==,
> + "{"
> + "\"integer\": -42, "
> + "\"boolean\": true, "
> + "\"string\": \"foo\""
> + "}");
> + g_free(out);
> +}
> +
> +static void test_visitor_out_union_flat(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + char *out;
> + UserDefFlatUnion *tmp = g_malloc0(sizeof(UserDefFlatUnion));
> +
> + tmp->enum1 = ENUM_ONE_VALUE1;
> + tmp->string = g_strdup("str");
> + tmp->integer = 41;
> + tmp->u.value1.boolean = true;
> +
> + visit_type_UserDefFlatUnion(data->ov, NULL, &tmp, &error_abort);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==,
> + "{"
> + "\"integer\": 41, "
> + "\"string\": \"str\", "
> + "\"enum1\": \"value1\", "
> + "\"boolean\": true"
> + "}");
> + g_free(out);
> + qapi_free_UserDefFlatUnion(tmp);
> +}
> +
> +static void test_visitor_out_alternate(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + UserDefAlternate *tmp;
> + char *out;
> +
> + tmp = g_new0(UserDefAlternate, 1);
> + tmp->type = QTYPE_QINT;
> + tmp->u.i = 42;
> +
> + visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "42");
> + g_free(out);
> + qapi_free_UserDefAlternate(tmp);
> +
> + tmp = g_new0(UserDefAlternate, 1);
> + tmp->type = QTYPE_QSTRING;
> + tmp->u.s = g_strdup("hello");
> +
> + visit_type_UserDefAlternate(data->ov, NULL, &tmp, &error_abort);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "\"hello\"");
> + g_free(out);
> + qapi_free_UserDefAlternate(tmp);
> +}
> +
> +static void test_visitor_out_null(TestOutputVisitorData *data,
> + const void *unused)
> +{
> + char *out;
> +
> + visit_type_null(data->ov, NULL, &error_abort);
> + out = json_output_get_string(data->jov);
> + g_assert_cmpstr(out, ==, "null");
> + g_free(out);
> +}
> +
> +static void output_visitor_test_add(const char *testpath,
> + void (*test_func)(TestOutputVisitorData *,
> + const void *))
> +{
> + g_test_add(testpath, TestOutputVisitorData, NULL, visitor_output_setup,
> + test_func, visitor_output_teardown);
> +}
> +
> +int main(int argc, char **argv)
> +{
> + g_test_init(&argc, &argv, NULL);
> +
> + output_visitor_test_add("/visitor/json/int", test_visitor_out_int);
> + output_visitor_test_add("/visitor/json/bool", test_visitor_out_bool);
> + output_visitor_test_add("/visitor/json/number", test_visitor_out_number);
> + output_visitor_test_add("/visitor/json/string", test_visitor_out_string);
> + output_visitor_test_add("/visitor/json/enum", test_visitor_out_enum);
> + output_visitor_test_add("/visitor/json/enum-errors",
> + test_visitor_out_enum_errors);
> + output_visitor_test_add("/visitor/json/struct", test_visitor_out_struct);
> + output_visitor_test_add("/visitor/json/struct-nested",
> + test_visitor_out_struct_nested);
> + output_visitor_test_add("/visitor/json/struct-errors",
> + test_visitor_out_struct_errors);
> + output_visitor_test_add("/visitor/json/list", test_visitor_out_list);
> + output_visitor_test_add("/visitor/json/any", test_visitor_out_any);
> + output_visitor_test_add("/visitor/json/union-flat",
> + test_visitor_out_union_flat);
> + output_visitor_test_add("/visitor/json/alternate",
> + test_visitor_out_alternate);
> + output_visitor_test_add("/visitor/json/null", test_visitor_out_null);
> +
> + g_test_run();
> +
> + return 0;
> +}
This is obviously patterned after test-qmp-output-visitor.c. Should we
link the two with comments, to improve our chances of them being kept in
sync?
> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
> index 2278970..b60e11b 100644
> --- a/qapi/Makefile.objs
> +++ b/qapi/Makefile.objs
> @@ -1,6 +1,6 @@
> util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
> util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
> util-obj-y += string-input-visitor.o string-output-visitor.o
> -util-obj-y += opts-visitor.o
> +util-obj-y += opts-visitor.o json-output-visitor.o
> util-obj-y += qmp-event.o
> util-obj-y += qapi-util.o
> diff --git a/tests/.gitignore b/tests/.gitignore
> index 2f8c7ea..c2aad79 100644
> --- a/tests/.gitignore
> +++ b/tests/.gitignore
> @@ -40,6 +40,7 @@ test-io-channel-file.txt
> test-io-channel-socket
> test-io-channel-tls
> test-io-task
> +test-json-output-visitor
> test-logging
> test-mul64
> test-opts-visitor
> diff --git a/tests/Makefile b/tests/Makefile
> index f71ed1c..0b5a7a8 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -22,6 +22,8 @@ check-unit-y += tests/check-qobject-json$(EXESUF)
> gcov-files-check-qobject-json-y = qobject/qobject-json.c
> check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
> gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
> +check-unit-y += tests/test-json-output-visitor$(EXESUF)
> +gcov-files-test-json-output-visitor-y = qapi/json-output-visitor.c
> check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
> gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
> check-unit-y += tests/test-qmp-input-strict$(EXESUF)
> @@ -388,6 +390,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
> tests/check-qobject-json.o \
> tests/test-coroutine.o tests/test-string-output-visitor.o \
> tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
> + tests/test-json-output-visitor.o \
> tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
> tests/test-qmp-commands.o tests/test-visitor-serialization.o \
> tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
> @@ -478,6 +481,7 @@ tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(
> tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)
> tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
> tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
> +tests/test-json-output-visitor$(EXESUF): tests/test-json-output-visitor.o $(test-qapi-obj-y)
> tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
> tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
> tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-02 9:15 ` Markus Armbruster
@ 2016-05-02 15:11 ` Eric Blake
2016-05-03 8:22 ` Markus Armbruster
2016-05-18 15:16 ` Eric Blake
1 sibling, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-02 15:11 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 7002 bytes --]
On 05/02/2016 03:15 AM, Markus Armbruster wrote:
> Title: s/json/JSON/
>
> Eric Blake <eblake@redhat.com> writes:
>
>> We have several places that want to go from qapi to JSON; right now,
>> they have to create an intermediate QObject to do the work. That
>> also has the drawback that the JSON formatting of a QDict will
>> rearrange keys (according to a deterministic, but unpredictable,
>> hash), when humans have an easier time if dicts are produced in
>> the same order as the qapi type.
>
> There's a drawback, though: more code.
>
> Could the JSON output visitor replace the QMP output visitor?
Hmm. As written here, the JSON output visitor _reuses_ the QMP output
visitor, for outputting an 'any' object. Maybe the QMP output visitor
could do a virtual visit through the JSON visitor, though (as in rather
than directly outputting to JSON, it instead opens a JSON visitor under
the hood, for some recursion when doing an 'any' visit). I can try it
as followup patches, but don't want to make the original checkin any
more complex than it has to be.
>
> Aside: the QMP visitors are really QObject visitors.
Yeah, particularly since they are also used by QGA. Is it worth renaming
them?
>> +/*
>> + * The JSON output visitor does not accept Infinity or NaN to
>> + * visit_type_number().
>> + */
>> +JsonOutputVisitor *json_output_visitor_new(void);
>> +void json_output_visitor_cleanup(JsonOutputVisitor *v);
>> +void json_output_visitor_reset(JsonOutputVisitor *v);
>
> Hmm. Why is "reset" not a Visitor method?
>
> I think this would let us put the things enforced by your "qmp: Tighten
> output visitor rules" in the Visitor contract.
I thought about that, and now that you've mentioned it, I'll probably
give it a try (that is, make visit_reset() a top-level construct that
ALL visitors must support, rather than just qmp-output and json-output).
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/json-output-visitor.h"
>> +#include "qapi/visitor-impl.h"
>> +#include "qemu/queue.h"
>> +#include "qemu-common.h"
>
> qemu/queue.h and qemu-common.h are superfluous.
Rebase churn, I first wrote the patches before the header cleanups.
Will fix.
>> +
>> +static void json_output_name(JsonOutputVisitor *jov, const char *name)
>
> This is called for every value, by its visit_start_FOO() or
> visit_type_FOO() function. Quote visitor.h:
>
> * The @name parameter of visit_type_FOO() describes the relation
> * between this QAPI value and its parent container. When visiting
> * the root of a tree, @name is ignored; when visiting a member of an
> * object, @name is the key associated with the value; and when
> * visiting a member of a list, @name is NULL.
>
> Aside: it should mention visit_start_FOO() in addition to
> visit_type_FOO().
>
Separate cleanup, but sounds useful. I can add it.
>> +{
>> + if (!jov->str) {
>> + jov->str = qstring_new();
>
> This happens on the first call after creation or reset of jov.
>
> If you call json_output_get_string() after an empty visit, it chokes on
> null jov->str. Could be declared a feature. The Visitor contract is
> silent on it, but the QMP output visitor rejects it since "qmp: Tighten
> output visitor rules".
I think feature, so yes, I should probably make the Visitor contract
explicit that at least something has to be visited via visit_type_FOO()
or visit_start_XXX().
>
> Regardless: why not create jov->str in json_output_visitor_new(), and
> truncate it in json_output_visitor_reset()?
>
> To retain the "feature", you'd assert qstring_get_length(jov->str).
Sounds reasonable.
...
>> +static void json_output_end_list(Visitor *v)
>> +{
>> + JsonOutputVisitor *jov = to_jov(v);
>> + assert(jov->depth);
>> + jov->depth--;
>> + qstring_append(jov->str, "]");
>> + jov->comma = true;
>> +}
>
> The nesting checks are less thorough than the QMP visitor's, because we
> don't use a stack. Okay.
And at times, I've debated about giving qapi-visit-core.c a stack, if
only to centralize some of the sanity checking for all visitors rather
than just the particular visitors that need a stack.
>> +static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
>> + Error **errp)
>> +{
>> + JsonOutputVisitor *jov = to_jov(v);
>> + QString *str = qobject_to_json(*obj);
>> + assert(str);
>
> Can't happen.
Can't happen now, but COULD happen if we teach qobject_to_json() to fail
on an attempt to visit Inf or NaN as a number (since that is not valid
JSON). Then again, if we teach it to fail, we'd want to add an Error
parameter. May also be impacted by how I refactor detection of invalid
numbers in response to your comments on 4/18. Should I keep or drop the
assert?
>> +/* Finish building, and return the resulting string. Will not be NULL. */
>> +char *json_output_get_string(JsonOutputVisitor *jov)
>> +{
>> + char *result;
>> +
>> + assert(!jov->depth);
>> + result = g_strdup(qstring_get_str(jov->str));
>> + json_output_visitor_reset(jov);
>
> Could avoid the strdup() if we wanted to. Needs a way to convert
> jov->str to char * destructively, like g_string_free() can do for a
> GString. Your choice.
May be a nice pre-req patch to add; not sure if there are any other
places already in tree that would benefit from it.
>> + for (i = 0; i < ENUM_ONE__MAX; i++) {
>> + visit_type_EnumOne(data->ov, "unused", &i, &error_abort);
>> +
>> + out = json_output_get_string(data->jov);
>> + g_assert(*out == '"');
>> + len = strlen(out);
>> + g_assert(out[len - 1] == '"');
>> + tmp = out + 1;
>> + out[len - 1] = 0;
>> + g_assert_cmpstr(tmp, ==, EnumOne_lookup[i]);
>> + g_free(out);
>
> Unlike in test-qmp-output-visitor.c, you don't need
> qmp_output_visitor_reset() here, because json_output_get_string() does
> it automatically.
>
> Is the difference between json_output_get_string() and
> qmp_output_get_qobject() a good idea?
No, and it probably means I have a bug for NOT requiring the reset.
>> + output_visitor_test_add("/visitor/json/any", test_visitor_out_any);
>> + output_visitor_test_add("/visitor/json/union-flat",
>> + test_visitor_out_union_flat);
>> + output_visitor_test_add("/visitor/json/alternate",
>> + test_visitor_out_alternate);
>> + output_visitor_test_add("/visitor/json/null", test_visitor_out_null);
>> +
>> + g_test_run();
>> +
>> + return 0;
>> +}
>
> This is obviously patterned after test-qmp-output-visitor.c. Should we
> link the two with comments, to improve our chances of them being kept in
> sync?
Sure.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-02 15:11 ` Eric Blake
@ 2016-05-03 8:22 ` Markus Armbruster
2016-05-04 15:45 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 8:22 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 05/02/2016 03:15 AM, Markus Armbruster wrote:
>> Title: s/json/JSON/
>>
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> We have several places that want to go from qapi to JSON; right now,
>>> they have to create an intermediate QObject to do the work. That
>>> also has the drawback that the JSON formatting of a QDict will
>>> rearrange keys (according to a deterministic, but unpredictable,
>>> hash), when humans have an easier time if dicts are produced in
>>> the same order as the qapi type.
>>
>> There's a drawback, though: more code.
>>
>> Could the JSON output visitor replace the QMP output visitor?
>
> Hmm. As written here, the JSON output visitor _reuses_ the QMP output
> visitor, for outputting an 'any' object. Maybe the QMP output visitor
> could do a virtual visit through the JSON visitor, though (as in rather
> than directly outputting to JSON, it instead opens a JSON visitor under
> the hood, for some recursion when doing an 'any' visit). I can try it
> as followup patches, but don't want to make the original checkin any
> more complex than it has to be.
Explorative followup patches are okay. The decision whether the JSON
visitor's additional code is worthwhile is easier when we know whether
it can replace the QMP output visitor.
>> Aside: the QMP visitors are really QObject visitors.
>
> Yeah, particularly since they are also used by QGA. Is it worth renaming
> them?
Let's consider the rename when the current visitors rework settles.
Perhaps we have less to rename then.
>>> +/*
>>> + * The JSON output visitor does not accept Infinity or NaN to
>>> + * visit_type_number().
>>> + */
>>> +JsonOutputVisitor *json_output_visitor_new(void);
>>> +void json_output_visitor_cleanup(JsonOutputVisitor *v);
>>> +void json_output_visitor_reset(JsonOutputVisitor *v);
>>
>> Hmm. Why is "reset" not a Visitor method?
>>
>> I think this would let us put the things enforced by your "qmp: Tighten
>> output visitor rules" in the Visitor contract.
>
> I thought about that, and now that you've mentioned it, I'll probably
> give it a try (that is, make visit_reset() a top-level construct that
> ALL visitors must support, rather than just qmp-output and json-output).
Yes, please.
>>> +
>>> +#include "qemu/osdep.h"
>>> +#include "qapi/json-output-visitor.h"
>>> +#include "qapi/visitor-impl.h"
>>> +#include "qemu/queue.h"
>>> +#include "qemu-common.h"
>>
>> qemu/queue.h and qemu-common.h are superfluous.
>
> Rebase churn, I first wrote the patches before the header cleanups.
> Will fix.
>
>>> +
>>> +static void json_output_name(JsonOutputVisitor *jov, const char *name)
>>
>> This is called for every value, by its visit_start_FOO() or
>> visit_type_FOO() function. Quote visitor.h:
>>
>> * The @name parameter of visit_type_FOO() describes the relation
>> * between this QAPI value and its parent container. When visiting
>> * the root of a tree, @name is ignored; when visiting a member of an
>> * object, @name is the key associated with the value; and when
>> * visiting a member of a list, @name is NULL.
>>
>> Aside: it should mention visit_start_FOO() in addition to
>> visit_type_FOO().
>>
>
> Separate cleanup, but sounds useful. I can add it.
>
>>> +{
>>> + if (!jov->str) {
>>> + jov->str = qstring_new();
>>
>> This happens on the first call after creation or reset of jov.
>>
>> If you call json_output_get_string() after an empty visit, it chokes on
>> null jov->str. Could be declared a feature. The Visitor contract is
>> silent on it, but the QMP output visitor rejects it since "qmp: Tighten
>> output visitor rules".
>
> I think feature, so yes, I should probably make the Visitor contract
> explicit that at least something has to be visited via visit_type_FOO()
> or visit_start_XXX().
To do that right, you have to make reset a method.
>> Regardless: why not create jov->str in json_output_visitor_new(), and
>> truncate it in json_output_visitor_reset()?
>>
>> To retain the "feature", you'd assert qstring_get_length(jov->str).
>
> Sounds reasonable.
>
> ...
>>> +static void json_output_end_list(Visitor *v)
>>> +{
>>> + JsonOutputVisitor *jov = to_jov(v);
>>> + assert(jov->depth);
>>> + jov->depth--;
>>> + qstring_append(jov->str, "]");
>>> + jov->comma = true;
>>> +}
>>
>> The nesting checks are less thorough than the QMP visitor's, because we
>> don't use a stack. Okay.
>
> And at times, I've debated about giving qapi-visit-core.c a stack, if
> only to centralize some of the sanity checking for all visitors rather
> than just the particular visitors that need a stack.
Doesn't feel worthwhile, unless you can *replace* the visitors' stacks.
Too much code just for catching a kind of bug that seems rather
unlikely.
If the stack is just for checking nesting, it could be a bit vector.
>>> +static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
>>> + Error **errp)
>>> +{
>>> + JsonOutputVisitor *jov = to_jov(v);
>>> + QString *str = qobject_to_json(*obj);
>>> + assert(str);
>>
>> Can't happen.
>
> Can't happen now, but COULD happen if we teach qobject_to_json() to fail
> on an attempt to visit Inf or NaN as a number (since that is not valid
> JSON). Then again, if we teach it to fail, we'd want to add an Error
> parameter. May also be impacted by how I refactor detection of invalid
> numbers in response to your comments on 4/18. Should I keep or drop the
> assert?
I'd drop it.
In general, I consider asserting "pointer not null" right before the
pointer is dereferenced a waste of space, except when it does
double-duty as documentation. Matter of taste, of course.
Here, the assertion does double-duty confusing me: how can this be null?
Are my ideas on qobject_to_json()'s behavior mistaken? Dig, dig, aha,
I'm not mistaken, it can't return null. In short, it wastes programmer
time in addition to space.
>>> +/* Finish building, and return the resulting string. Will not be NULL. */
>>> +char *json_output_get_string(JsonOutputVisitor *jov)
>>> +{
>>> + char *result;
>>> +
>>> + assert(!jov->depth);
>>> + result = g_strdup(qstring_get_str(jov->str));
>>> + json_output_visitor_reset(jov);
>>
>> Could avoid the strdup() if we wanted to. Needs a way to convert
>> jov->str to char * destructively, like g_string_free() can do for a
>> GString. Your choice.
>
> May be a nice pre-req patch to add; not sure if there are any other
> places already in tree that would benefit from it.
Just don't do it with a flag parameter, like g_string_free() does.
Mashing two operations as different as those into one function shows a
deplorable lack of taste.
>>> + for (i = 0; i < ENUM_ONE__MAX; i++) {
>>> + visit_type_EnumOne(data->ov, "unused", &i, &error_abort);
>>> +
>>> + out = json_output_get_string(data->jov);
>>> + g_assert(*out == '"');
>>> + len = strlen(out);
>>> + g_assert(out[len - 1] == '"');
>>> + tmp = out + 1;
>>> + out[len - 1] = 0;
>>> + g_assert_cmpstr(tmp, ==, EnumOne_lookup[i]);
>>> + g_free(out);
>>
>> Unlike in test-qmp-output-visitor.c, you don't need
>> qmp_output_visitor_reset() here, because json_output_get_string() does
>> it automatically.
>>
>> Is the difference between json_output_get_string() and
>> qmp_output_get_qobject() a good idea?
>
> No, and it probably means I have a bug for NOT requiring the reset.
>
>>> + output_visitor_test_add("/visitor/json/any", test_visitor_out_any);
>>> + output_visitor_test_add("/visitor/json/union-flat",
>>> + test_visitor_out_union_flat);
>>> + output_visitor_test_add("/visitor/json/alternate",
>>> + test_visitor_out_alternate);
>>> + output_visitor_test_add("/visitor/json/null", test_visitor_out_null);
>>> +
>>> + g_test_run();
>>> +
>>> + return 0;
>>> +}
>>
>> This is obviously patterned after test-qmp-output-visitor.c. Should we
>> link the two with comments, to improve our chances of them being kept in
>> sync?
>
> Sure.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-03 8:22 ` Markus Armbruster
@ 2016-05-04 15:45 ` Markus Armbruster
2016-05-06 4:16 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 15:45 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Michael Roth
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 05/02/2016 03:15 AM, Markus Armbruster wrote:
[...]
>>>> +/*
>>>> + * The JSON output visitor does not accept Infinity or NaN to
>>>> + * visit_type_number().
>>>> + */
>>>> +JsonOutputVisitor *json_output_visitor_new(void);
>>>> +void json_output_visitor_cleanup(JsonOutputVisitor *v);
>>>> +void json_output_visitor_reset(JsonOutputVisitor *v);
>>>
>>> Hmm. Why is "reset" not a Visitor method?
>>>
>>> I think this would let us put the things enforced by your "qmp: Tighten
>>> output visitor rules" in the Visitor contract.
>>
>> I thought about that, and now that you've mentioned it, I'll probably
>> give it a try (that is, make visit_reset() a top-level construct that
>> ALL visitors must support, rather than just qmp-output and json-output).
>
> Yes, please.
Same question for "cleanup". Stupid name for a destructor, by the way.
[...]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-04 15:45 ` Markus Armbruster
@ 2016-05-06 4:16 ` Eric Blake
2016-05-06 12:31 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-06 4:16 UTC (permalink / raw)
To: Markus Armbruster; +Cc: famz, qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 3309 bytes --]
On 05/04/2016 09:45 AM, Markus Armbruster wrote:
>>>>> +void json_output_visitor_reset(JsonOutputVisitor *v);
>>>>
>>>> Hmm. Why is "reset" not a Visitor method?
>>>>
>>>> I think this would let us put the things enforced by your "qmp: Tighten
>>>> output visitor rules" in the Visitor contract.
>>>
>>> I thought about that, and now that you've mentioned it, I'll probably
>>> give it a try (that is, make visit_reset() a top-level construct that
>>> ALL visitors must support, rather than just qmp-output and json-output).
>>
>> Yes, please.
>
> Same question for "cleanup". Stupid name for a destructor, by the way.
Interface question - all of the FOO_visitor_new() functions return a
subtype pointer Foo*, rather than Visitor*; along with a Visitor
*FOO_get_visitor(FOO*) for up-casting, so that FOO* can be used in the
per-type cleanup function; the FOO* pointers are also useful for two
additional output functions in the two output visitors. We're proposing
hiding the per-type cleanup function behind a simpler visit_free(Visitor
*v). So all that's left are the two output functions. Can we get rid
of those, and make Visitor* the only public interface, rather than
making every caller have to do upcasts?
It looks like outside of the testsuite, all uses of these visitors are
local to a single function; and improving the testsuite is not the end
of the world. Particularly if only the testsuite is using reset, it may
be easier to just patch the testsuite to use a new visitor in the places
where it currently does a reset. How ugly would it be to require that
the caller pass in a pointer to the result as part of creating the
visitor, with the promise that the result is populated at the end of a
successful visit, and left NULL if the visit is reset early? Or maybe a
visit_complete() function that is a no-op on input visitors, but
populates the parameter passed at creation for output visitors. If we
do that, we could rewrite things like the existing:
QObject *object_property_get_qobject(Object *obj, const char *name,
Error **errp)
{
QObject *ret = NULL;
Error *local_err = NULL;
QmpOutputVisitor *qov;
qov = qmp_output_visitor_new();
object_property_get(obj, qmp_output_get_visitor(qov), name, &local_err);
if (!local_err) {
ret = qmp_output_get_qobject(qov);
}
error_propagate(errp, local_err);
qmp_output_visitor_cleanup(qov);
return ret;
}
to instead be:
QObject *object_property_get_qobject(Object *obj, const char *name,
Error **errp)
{
QObject *ret = NULL;
Error *local_err = NULL;
Visitor *v;
v = qmp_output_visitor_new(&ret);
object_property_get(obj, v, name, &local_err);
if (!local_err) {
visit_complete(v); /* populates ret */
}
error_propagate(errp, local_err);
visit_free(v);
return ret;
}
Slightly shorter, but populating 'ret' at a distance feels a bit weird.
Maybe we need to keep the FOO_new() functions returning FOO* rather
than Visitor*, along with the FOO_get_visitor() functions, after all.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-06 4:16 ` Eric Blake
@ 2016-05-06 12:31 ` Markus Armbruster
2016-05-06 14:08 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-06 12:31 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 05/04/2016 09:45 AM, Markus Armbruster wrote:
>>>>>> +void json_output_visitor_reset(JsonOutputVisitor *v);
>>>>>
>>>>> Hmm. Why is "reset" not a Visitor method?
>>>>>
>>>>> I think this would let us put the things enforced by your "qmp: Tighten
>>>>> output visitor rules" in the Visitor contract.
>>>>
>>>> I thought about that, and now that you've mentioned it, I'll probably
>>>> give it a try (that is, make visit_reset() a top-level construct that
>>>> ALL visitors must support, rather than just qmp-output and json-output).
>>>
>>> Yes, please.
>>
>> Same question for "cleanup". Stupid name for a destructor, by the way.
>
> Interface question - all of the FOO_visitor_new() functions return a
> subtype pointer Foo*, rather than Visitor*; along with a Visitor
> *FOO_get_visitor(FOO*) for up-casting, so that FOO* can be used in the
> per-type cleanup function; the FOO* pointers are also useful for two
> additional output functions in the two output visitors. We're proposing
> hiding the per-type cleanup function behind a simpler visit_free(Visitor
> *v). So all that's left are the two output functions. Can we get rid
> of those, and make Visitor* the only public interface, rather than
> making every caller have to do upcasts?
The two output functions are
QObject *qmp_output_get_qobject(QmpOutputVisitor *v);
char *string_output_get_string(StringOutputVisitor *v);
> It looks like outside of the testsuite, all uses of these visitors are
> local to a single function; and improving the testsuite is not the end
> of the world. Particularly if only the testsuite is using reset, it may
> be easier to just patch the testsuite to use a new visitor in the places
> where it currently does a reset.
I'm okay with replacing reset by destroy + new in the test suite.
> How ugly would it be to require that
> the caller pass in a pointer to the result as part of creating the
> visitor, with the promise that the result is populated at the end of a
> successful visit, and left NULL if the visit is reset early? Or maybe a
> visit_complete() function that is a no-op on input visitors, but
> populates the parameter passed at creation for output visitors. If we
> do that, we could rewrite things like the existing:
>
> QObject *object_property_get_qobject(Object *obj, const char *name,
> Error **errp)
> {
> QObject *ret = NULL;
> Error *local_err = NULL;
> QmpOutputVisitor *qov;
>
> qov = qmp_output_visitor_new();
> object_property_get(obj, qmp_output_get_visitor(qov), name, &local_err);
> if (!local_err) {
> ret = qmp_output_get_qobject(qov);
> }
> error_propagate(errp, local_err);
> qmp_output_visitor_cleanup(qov);
> return ret;
> }
>
> to instead be:
>
> QObject *object_property_get_qobject(Object *obj, const char *name,
> Error **errp)
> {
> QObject *ret = NULL;
> Error *local_err = NULL;
> Visitor *v;
>
> v = qmp_output_visitor_new(&ret);
> object_property_get(obj, v, name, &local_err);
> if (!local_err) {
> visit_complete(v); /* populates ret */
> }
> error_propagate(errp, local_err);
> visit_free(v);
> return ret;
> }
>
> Slightly shorter, but populating 'ret' at a distance feels a bit weird.
I like not having to deal with the QmpOutputVisitor type, but like you,
I don't like action at a distance.
> Maybe we need to keep the FOO_new() functions returning FOO* rather
> than Visitor*, along with the FOO_get_visitor() functions, after all.
I can think of a other ways to hide the QmpOutputVisitor type, but they
have drawbacks, too.
You can let the two output functions take Visitor *. Drawback: now the
compiler lets you pass the wrong kind of visitor.
You can let visit_complete() return the output (if any) as void *.
Drawback: now the compiler lets you misinterpret the output's type.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-06 12:31 ` Markus Armbruster
@ 2016-05-06 14:08 ` Eric Blake
2016-05-10 4:22 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-06 14:08 UTC (permalink / raw)
To: Markus Armbruster; +Cc: famz, qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2665 bytes --]
On 05/06/2016 06:31 AM, Markus Armbruster wrote:
>> So all that's left are the two output functions. Can we get rid
>> of those, and make Visitor* the only public interface, rather than
>> making every caller have to do upcasts?
>
> The two output functions are
>
> QObject *qmp_output_get_qobject(QmpOutputVisitor *v);
> char *string_output_get_string(StringOutputVisitor *v);
>
>> It looks like outside of the testsuite, all uses of these visitors are
>> local to a single function; and improving the testsuite is not the end
>> of the world. Particularly if only the testsuite is using reset, it may
>> be easier to just patch the testsuite to use a new visitor in the places
>> where it currently does a reset.
>
> I'm okay with replacing reset by destroy + new in the test suite.
That part's (relatively) easy, so it will be in the next spin.
>> QObject *object_property_get_qobject(Object *obj, const char *name,
>> Error **errp)
>> {
>> QObject *ret = NULL;
>> Error *local_err = NULL;
>> Visitor *v;
>>
>> v = qmp_output_visitor_new(&ret);
>> object_property_get(obj, v, name, &local_err);
>> if (!local_err) {
>> visit_complete(v); /* populates ret */
>> }
>> error_propagate(errp, local_err);
>> visit_free(v);
>> return ret;
>> }
>>
>> Slightly shorter, but populating 'ret' at a distance feels a bit weird.
>
> I like not having to deal with the QmpOutputVisitor type, but like you,
> I don't like action at a distance.
>
>> Maybe we need to keep the FOO_new() functions returning FOO* rather
>> than Visitor*, along with the FOO_get_visitor() functions, after all.
>
> I can think of a other ways to hide the QmpOutputVisitor type, but they
> have drawbacks, too.
>
> You can let the two output functions take Visitor *. Drawback: now the
> compiler lets you pass the wrong kind of visitor.
But at least you can assert that the right visitor was used.
>
> You can let visit_complete() return the output (if any) as void *.
> Drawback: now the compiler lets you misinterpret the output's type.
And you lose any chance to assert things.
Another (hybrid?) option:
void visit_complete(Visitor *v, void *opaque);
Visitor *qmp_output_visitor_new(QObject **ret);
called as:
v = qmp_output_visitor_new(&ret);
...
visit_complete(v, &ret);
where the completion function asserts that opaque matches the same
pointer as passed in with correct type during new().
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-06 14:08 ` Eric Blake
@ 2016-05-10 4:22 ` Eric Blake
0 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-05-10 4:22 UTC (permalink / raw)
To: Markus Armbruster; +Cc: famz, qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]
On 05/06/2016 08:08 AM, Eric Blake wrote:
> On 05/06/2016 06:31 AM, Markus Armbruster wrote:
>>> So all that's left are the two output functions. Can we get rid
>>> of those, and make Visitor* the only public interface, rather than
>>> making every caller have to do upcasts?
>>
>> The two output functions are
>>
>> QObject *qmp_output_get_qobject(QmpOutputVisitor *v);
>> char *string_output_get_string(StringOutputVisitor *v);
>>
>>> It looks like outside of the testsuite, all uses of these visitors are
>>> local to a single function; and improving the testsuite is not the end
>>> of the world. Particularly if only the testsuite is using reset, it may
>>> be easier to just patch the testsuite to use a new visitor in the places
>>> where it currently does a reset.
>>
>> I'm okay with replacing reset by destroy + new in the test suite.
>
> That part's (relatively) easy, so it will be in the next spin.
In fact, it's easier to replace that part in the existing qapi-next
branch than it is to revert the addition of qmp_output_visitor_reset();
see patch posted in the other thread (v16A 17/24).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-02 9:15 ` Markus Armbruster
2016-05-02 15:11 ` Eric Blake
@ 2016-05-18 15:16 ` Eric Blake
2016-05-18 15:24 ` Eric Blake
1 sibling, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-18 15:16 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]
On 05/02/2016 03:15 AM, Markus Armbruster wrote:
> Title: s/json/JSON/
>
> Eric Blake <eblake@redhat.com> writes:
>
>> We have several places that want to go from qapi to JSON; right now,
>> they have to create an intermediate QObject to do the work. That
>> also has the drawback that the JSON formatting of a QDict will
>> rearrange keys (according to a deterministic, but unpredictable,
>> hash), when humans have an easier time if dicts are produced in
>> the same order as the qapi type.
>
> There's a drawback, though: more code.
>
> Could the JSON output visitor replace the QMP output visitor?
Yes, it turned out quite nicely to write qobject_to_json() on top of a
JSON output visitor. And in fact, doing so makes it trivial to write a
QObject deep-cloner - pass the QObject to the qmp-output-visitor instead
of the json-output-visitor!
>> +static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
>> + Error **errp)
>> +{
>> + JsonOutputVisitor *jov = to_jov(v);
>> + QString *str = qobject_to_json(*obj);
>> + assert(str);
>
> Can't happen.
Can too. From tests/check-qobject-json.c:
obj = QOBJECT(qstring_from_str(utf8_in));
str = qobject_to_json(obj);
if (json_out) {
g_assert(str);
g_assert_cmpstr(qstring_get_str(str), ==, json_out);
} else {
g_assert(!str);
}
where the failures occur when it is impossible to output proper UTF-8
due to invalid encoding in a string. Arguably, that case would be nicer
if it could set an Error* (and would make my argument for setting an
error on Inf/NaN for numbers also a bit more tenable), but that is
additional work that I haven't tackled yet. I'm trying to get the series
posted for another round of review, where I've done some major
reshuffling (such as doing the clone visitor first, not second, in the
series), so hopefully later today.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-05-18 15:16 ` Eric Blake
@ 2016-05-18 15:24 ` Eric Blake
0 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-05-18 15:24 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1621 bytes --]
On 05/18/2016 09:16 AM, Eric Blake wrote:
>>> +static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
>>> + Error **errp)
>>> +{
>>> + JsonOutputVisitor *jov = to_jov(v);
>>> + QString *str = qobject_to_json(*obj);
>>> + assert(str);
>>
>> Can't happen.
>
> Can too. From tests/check-qobject-json.c:
>
> obj = QOBJECT(qstring_from_str(utf8_in));
> str = qobject_to_json(obj);
> if (json_out) {
> g_assert(str);
> g_assert_cmpstr(qstring_get_str(str), ==, json_out);
> } else {
> g_assert(!str);
> }
>
> where the failures occur when it is impossible to output proper UTF-8
> due to invalid encoding in a string.
Correction - that test has dead code, because:
json_out = test_cases[i].json_out ?: test_cases[i].json_in;
so you are right after all - we currently cannot fail on a conversion to
JSON (although the result might not be valid JSON). At any rate, my
conclusion remains:
> Arguably, that case would be nicer
> if it could set an Error* (and would make my argument for setting an
> error on Inf/NaN for numbers also a bit more tenable), but that is
> additional work that I haven't tackled yet. I'm trying to get the series
> posted for another round of review, where I've done some major
> reshuffling (such as doing the clone visitor first, not second, in the
> series), so hopefully later today.
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor Eric Blake
2016-05-02 9:15 ` Markus Armbruster
@ 2016-05-02 15:00 ` Markus Armbruster
1 sibling, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 15:00 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> We have several places that want to go from qapi to JSON; right now,
> they have to create an intermediate QObject to do the work. That
> also has the drawback that the JSON formatting of a QDict will
> rearrange keys (according to a deterministic, but unpredictable,
> hash), when humans have an easier time if dicts are produced in
> the same order as the qapi type.
>
> For these reasons, it is time to add a new JSON output visitor.
> This patch just adds the basic visitor and tests that it works;
> later patches will add pretty-printing, and convert clients to
> use the visitor.
>
> Design choices: Unlike the QMP output visitor, the JSON visitor
> refuses to visit a required string with a NULL value (abort), as
> well as a non-finite number (raises an error message). Reusing
> QString to grow the contents means that we easily share code with
> both qobject-json.c and qjson.c; although it might be nice to
> enhance things to take an optional output callback function so
> that the output can truly be streamed instead of collected in
> memory.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: retitle, rebase to master, minor cleanups
> v2: rebase to qapi subset E v8; add test of error outputting
> infinity; use unsigned depth
> ---
> include/qapi/visitor.h | 20 +-
> include/qapi/json-output-visitor.h | 29 +++
> qapi/json-output-visitor.c | 202 ++++++++++++++++++
> tests/test-json-output-visitor.c | 418 +++++++++++++++++++++++++++++++++++++
> qapi/Makefile.objs | 2 +-
> tests/.gitignore | 1 +
> tests/Makefile | 4 +
> 7 files changed, 665 insertions(+), 11 deletions(-)
> create mode 100644 include/qapi/json-output-visitor.h
> create mode 100644 qapi/json-output-visitor.c
> create mode 100644 tests/test-json-output-visitor.c
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index a430c19..e8a4403 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -26,16 +26,16 @@
> *
> * There are three kinds of visitor classes: input visitors (QMP,
> * string, and QemuOpts) parse an external representation and build
> - * the corresponding QAPI graph, output visitors (QMP and string) take
> - * a completed QAPI graph and generate an external representation, and
> - * the dealloc visitor can take a QAPI graph (possibly partially
> - * constructed) and recursively free its resources. While the dealloc
> - * and QMP input/output visitors are general, the string and QemuOpts
> - * visitors have some implementation limitations; see the
> - * documentation for each visitor for more details on what it
> - * supports. Also, see visitor-impl.h for the callback contracts
> - * implemented by each visitor, and docs/qapi-code-gen.txt for more
> - * about the QAPI code generator.
> + * the corresponding QAPI graph, output visitors (QMP, string, and
> + * JSON) take a completed QAPI graph and generate an external
> + * representation, and the dealloc visitor can take a QAPI graph
> + * (possibly partially constructed) and recursively free its
> + * resources. While the dealloc and QMP input/output visitors are
and the JSON output visitor?
> + * general, the string and QemuOpts visitors have some implementation
> + * limitations; see the documentation for each visitor for more
> + * details on what it supports. Also, see visitor-impl.h for the
> + * callback contracts implemented by each visitor, and
> + * docs/qapi-code-gen.txt for more about the QAPI code generator.
> *
> * All QAPI types have a corresponding function with a signature
> * roughly compatible with this:
[...]
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (6 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 07/18] qapi: Add json output visitor Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 12:45 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 09/18] Revert "qjson: Simplify by using json-output-visitor" Eric Blake
` (10 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz
Instead of rolling our own limited JSON outputter, we can just
wrap the more full-featured JSON output Visitor.
This slightly changes the output (different spacing), but the
result is still equivalent JSON contents.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: rebase to master
v2: rebase to earlier changes
---
qjson.c | 61 ++++++++++++++++++++++---------------------------------------
1 file changed, 22 insertions(+), 39 deletions(-)
diff --git a/qjson.c b/qjson.c
index d172b1f..17cc962 100644
--- a/qjson.c
+++ b/qjson.c
@@ -12,102 +12,85 @@
*/
#include "qemu/osdep.h"
-#include <qapi/qmp/qstring.h>
-#include <glib.h>
#include <qjson.h>
#include <qemu/module.h>
#include <qom/object.h>
+#include "qapi/json-output-visitor.h"
+#include "qapi/error.h"
struct QJSON {
Object obj;
- QString *str;
- bool omit_comma;
+ JsonOutputVisitor *jov;
+ char *str;
};
#define QJSON(obj) OBJECT_CHECK(QJSON, (obj), TYPE_QJSON)
-static void json_emit_element(QJSON *json, const char *name)
-{
- /* Check whether we need to print a , before an element */
- if (json->omit_comma) {
- json->omit_comma = false;
- } else {
- qstring_append(json->str, ", ");
- }
-
- if (name) {
- qstring_append_json_string(json->str, name);
- qstring_append(json->str, " : ");
- }
-}
-
void json_start_object(QJSON *json, const char *name)
{
- json_emit_element(json, name);
- qstring_append(json->str, "{ ");
- json->omit_comma = true;
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_start_struct(v, name, NULL, 0, &error_abort);
}
void json_end_object(QJSON *json)
{
- qstring_append(json->str, " }");
- json->omit_comma = false;
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v);
}
void json_start_array(QJSON *json, const char *name)
{
- json_emit_element(json, name);
- qstring_append(json->str, "[ ");
- json->omit_comma = true;
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_start_list(v, name, NULL, 0, &error_abort);
}
void json_end_array(QJSON *json)
{
- qstring_append(json->str, " ]");
- json->omit_comma = false;
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_end_list(v);
}
void json_prop_int(QJSON *json, const char *name, int64_t val)
{
- json_emit_element(json, name);
- qstring_append_format(json->str, "%" PRId64, val);
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_type_int(v, name, &val, &error_abort);
}
void json_prop_str(QJSON *json, const char *name, const char *str)
{
- json_emit_element(json, name);
- qstring_append_json_string(json->str, str);
+ Visitor *v = json_output_get_visitor(json->jov);
+ visit_type_str(v, name, (char **)&str, &error_abort);
}
const char *qjson_get_str(QJSON *json)
{
- return qstring_get_str(json->str);
+ return json->str;
}
QJSON *qjson_new(void)
{
QJSON *json = QJSON(object_new(TYPE_QJSON));
+ json_start_object(json, NULL);
return json;
}
void qjson_finish(QJSON *json)
{
json_end_object(json);
+ json->str = json_output_get_string(json->jov);
}
static void qjson_initfn(Object *obj)
{
QJSON *json = QJSON(obj);
-
- json->str = qstring_from_str("{ ");
- json->omit_comma = true;
+ json->jov = json_output_visitor_new();
}
static void qjson_finalizefn(Object *obj)
{
QJSON *json = QJSON(obj);
-
- qobject_decref(QOBJECT(json->str));
+ g_free(json->str);
}
static const TypeInfo qjson_type_info = {
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor Eric Blake
@ 2016-05-02 12:45 ` Markus Armbruster
2016-05-02 12:49 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 12:45 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz
Eric Blake <eblake@redhat.com> writes:
> Instead of rolling our own limited JSON outputter, we can just
> wrap the more full-featured JSON output Visitor.
>
> This slightly changes the output (different spacing), but the
> result is still equivalent JSON contents.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
The file comment
/*
* QEMU JSON writer
*
becomes misleading. This isn't "the" QEMU JSON writer. It's not even
"a" JSON writer anymore. It's a (QOM) object to accumulate JSON in a
limited way.
Why it's a *QOM* object I can't see.
If I take the QOMness away, all that's left is a thin, trivial wrapper
around the visitor. Why is it useful?
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor
2016-05-02 12:45 ` Markus Armbruster
@ 2016-05-02 12:49 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 12:49 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> Instead of rolling our own limited JSON outputter, we can just
>> wrap the more full-featured JSON output Visitor.
>>
>> This slightly changes the output (different spacing), but the
>> result is still equivalent JSON contents.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> The file comment
>
> /*
> * QEMU JSON writer
> *
>
> becomes misleading. This isn't "the" QEMU JSON writer. It's not even
> "a" JSON writer anymore. It's a (QOM) object to accumulate JSON in a
> limited way.
>
> Why it's a *QOM* object I can't see.
>
> If I take the QOMness away, all that's left is a thin, trivial wrapper
> around the visitor. Why is it useful?
Oh, now I see: you asked yourself the same question, and PATCH 10+11 are
the result.
The weekend deleted the cover letter from my memory %-)
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 09/18] Revert "qjson: Simplify by using json-output-visitor"
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (7 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 08/18] qjson: Simplify by using json-output-visitor Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor Eric Blake
` (9 subsequent siblings)
18 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz
Showing two alternative approaches; revert approach 1 to
go forward with approach 2
This reverts commit 7af47db0b00e982231399a81a3af490d84aee010.
---
qjson.c | 61 +++++++++++++++++++++++++++++++++++++++----------------------
1 file changed, 39 insertions(+), 22 deletions(-)
diff --git a/qjson.c b/qjson.c
index 17cc962..d172b1f 100644
--- a/qjson.c
+++ b/qjson.c
@@ -12,85 +12,102 @@
*/
#include "qemu/osdep.h"
+#include <qapi/qmp/qstring.h>
+#include <glib.h>
#include <qjson.h>
#include <qemu/module.h>
#include <qom/object.h>
-#include "qapi/json-output-visitor.h"
-#include "qapi/error.h"
struct QJSON {
Object obj;
- JsonOutputVisitor *jov;
- char *str;
+ QString *str;
+ bool omit_comma;
};
#define QJSON(obj) OBJECT_CHECK(QJSON, (obj), TYPE_QJSON)
+static void json_emit_element(QJSON *json, const char *name)
+{
+ /* Check whether we need to print a , before an element */
+ if (json->omit_comma) {
+ json->omit_comma = false;
+ } else {
+ qstring_append(json->str, ", ");
+ }
+
+ if (name) {
+ qstring_append_json_string(json->str, name);
+ qstring_append(json->str, " : ");
+ }
+}
+
void json_start_object(QJSON *json, const char *name)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_start_struct(v, name, NULL, 0, &error_abort);
+ json_emit_element(json, name);
+ qstring_append(json->str, "{ ");
+ json->omit_comma = true;
}
void json_end_object(QJSON *json)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_check_struct(v, &error_abort);
- visit_end_struct(v);
+ qstring_append(json->str, " }");
+ json->omit_comma = false;
}
void json_start_array(QJSON *json, const char *name)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_start_list(v, name, NULL, 0, &error_abort);
+ json_emit_element(json, name);
+ qstring_append(json->str, "[ ");
+ json->omit_comma = true;
}
void json_end_array(QJSON *json)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_end_list(v);
+ qstring_append(json->str, " ]");
+ json->omit_comma = false;
}
void json_prop_int(QJSON *json, const char *name, int64_t val)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_type_int(v, name, &val, &error_abort);
+ json_emit_element(json, name);
+ qstring_append_format(json->str, "%" PRId64, val);
}
void json_prop_str(QJSON *json, const char *name, const char *str)
{
- Visitor *v = json_output_get_visitor(json->jov);
- visit_type_str(v, name, (char **)&str, &error_abort);
+ json_emit_element(json, name);
+ qstring_append_json_string(json->str, str);
}
const char *qjson_get_str(QJSON *json)
{
- return json->str;
+ return qstring_get_str(json->str);
}
QJSON *qjson_new(void)
{
QJSON *json = QJSON(object_new(TYPE_QJSON));
- json_start_object(json, NULL);
return json;
}
void qjson_finish(QJSON *json)
{
json_end_object(json);
- json->str = json_output_get_string(json->jov);
}
static void qjson_initfn(Object *obj)
{
QJSON *json = QJSON(obj);
- json->jov = json_output_visitor_new();
+
+ json->str = qstring_from_str("{ ");
+ json->omit_comma = true;
}
static void qjson_finalizefn(Object *obj)
{
QJSON *json = QJSON(obj);
- g_free(json->str);
+
+ qobject_decref(QOBJECT(json->str));
}
static const TypeInfo qjson_type_info = {
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (8 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 09/18] Revert "qjson: Simplify by using json-output-visitor" Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 13:26 ` Markus Armbruster
2016-05-03 9:44 ` Dr. David Alan Gilbert
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 11/18] qjson: Remove unused file Eric Blake
` (8 subsequent siblings)
18 siblings, 2 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Juan Quintela, Amit Shah
Rather than using a QJSON object and converting the QString result
to a char *, we can use the new JSON output visitor and get directly
to a char *.
The conversions are a bit tricky in place (in places, we have to
copy an integer to an int64_t temporary to get the right pointer for
visit_type_int(); and for several strings, we have to copy to a
temporary variable so we can take an address (&char[] is not the
same as &char*) and cast away const), but overall still fairly
mechanical.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: retitle, rebase to master
v2: new patch
---
include/migration/vmstate.h | 4 +--
migration/savevm.c | 68 ++++++++++++++++++++++++++++-----------------
migration/vmstate.c | 64 ++++++++++++++++++++++++++----------------
tests/Makefile | 2 +-
4 files changed, 86 insertions(+), 52 deletions(-)
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 84ee355..2cdfce9 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -29,7 +29,7 @@
#ifndef CONFIG_USER_ONLY
#include <migration/qemu-file.h>
#endif
-#include <qjson.h>
+#include "qapi/json-output-visitor.h"
typedef void SaveStateHandler(QEMUFile *f, void *opaque);
typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
@@ -925,7 +925,7 @@ void loadvm_free_handlers(MigrationIncomingState *mis);
int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque, int version_id);
void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
- void *opaque, QJSON *vmdesc);
+ void *opaque, Visitor *vmdesc);
bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque);
diff --git a/migration/savevm.c b/migration/savevm.c
index 16ba443..c858fe9 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2,7 +2,7 @@
* QEMU System Emulator
*
* Copyright (c) 2003-2008 Fabrice Bellard
- * Copyright (c) 2009-2015 Red Hat Inc
+ * Copyright (c) 2009-2016 Red Hat Inc
*
* Authors:
* Juan Quintela <quintela@redhat.com>
@@ -645,27 +645,32 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
return vmstate_load_state(f, se->vmsd, se->opaque, version_id);
}
-static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
+static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
+ Visitor *vmdesc)
{
int64_t old_offset, size;
+ const char *tmp;
old_offset = qemu_ftell_fast(f);
se->ops->save_state(f, se->opaque);
size = qemu_ftell_fast(f) - old_offset;
if (vmdesc) {
- json_prop_int(vmdesc, "size", size);
- json_start_array(vmdesc, "fields");
- json_start_object(vmdesc, NULL);
- json_prop_str(vmdesc, "name", "data");
- json_prop_int(vmdesc, "size", size);
- json_prop_str(vmdesc, "type", "buffer");
- json_end_object(vmdesc);
- json_end_array(vmdesc);
+ visit_type_int(vmdesc, "size", &size, &error_abort);
+ visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
+ visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
+ tmp = "data";
+ visit_type_str(vmdesc, "name", (char **)&tmp, &error_abort);
+ visit_type_int(vmdesc, "size", &size, &error_abort);
+ tmp = "buffer";
+ visit_type_str(vmdesc, "type", (char **)&tmp, &error_abort);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
+ visit_end_list(vmdesc);
}
}
-static void vmstate_save(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
+static void vmstate_save(QEMUFile *f, SaveStateEntry *se, Visitor *vmdesc)
{
trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)");
if (!se->vmsd) {
@@ -891,7 +896,7 @@ void qemu_savevm_state_header(QEMUFile *f)
if (!savevm_state.skip_configuration || enforce_config_section()) {
qemu_put_byte(f, QEMU_VM_CONFIGURATION);
- vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0);
+ vmstate_save_state(f, &vmstate_configuration, &savevm_state, NULL);
}
}
@@ -1033,11 +1038,15 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f)
void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
{
- QJSON *vmdesc;
+ JsonOutputVisitor *vmdesc_jov;
+ Visitor *vmdesc;
+ char *vmdesc_str;
int vmdesc_len;
SaveStateEntry *se;
int ret;
bool in_postcopy = migration_in_postcopy(migrate_get_current());
+ int64_t tmp_i;
+ char *tmp_s;
trace_savevm_state_complete_precopy();
@@ -1073,9 +1082,12 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
return;
}
- vmdesc = qjson_new();
- json_prop_int(vmdesc, "page_size", TARGET_PAGE_SIZE);
- json_start_array(vmdesc, "devices");
+ vmdesc_jov = json_output_visitor_new();
+ vmdesc = json_output_get_visitor(vmdesc_jov);
+ visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
+ tmp_i = TARGET_PAGE_SIZE;
+ visit_type_int(vmdesc, "page_size", &tmp_i, &error_abort);
+ visit_start_list(vmdesc, "devices", NULL, 0, &error_abort);
QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {
if ((!se->ops || !se->ops->save_state) && !se->vmsd) {
@@ -1088,16 +1100,19 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
trace_savevm_section_start(se->idstr, se->section_id);
- json_start_object(vmdesc, NULL);
- json_prop_str(vmdesc, "name", se->idstr);
- json_prop_int(vmdesc, "instance_id", se->instance_id);
+ visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
+ tmp_s = se->idstr;
+ visit_type_str(vmdesc, "name", &tmp_s, &error_abort);
+ tmp_i = se->instance_id;
+ visit_type_int(vmdesc, "instance_id", &tmp_i, &error_abort);
save_section_header(f, se, QEMU_VM_SECTION_FULL);
vmstate_save(f, se, vmdesc);
trace_savevm_section_end(se->idstr, se->section_id, 0);
save_section_footer(f, se);
- json_end_object(vmdesc);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
}
if (!in_postcopy) {
@@ -1105,16 +1120,19 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
qemu_put_byte(f, QEMU_VM_EOF);
}
- json_end_array(vmdesc);
- qjson_finish(vmdesc);
- vmdesc_len = strlen(qjson_get_str(vmdesc));
+ visit_end_list(vmdesc);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
+ vmdesc_str = json_output_get_string(vmdesc_jov);
+ vmdesc_len = strlen(vmdesc_str);
if (should_send_vmdesc()) {
qemu_put_byte(f, QEMU_VM_VMDESCRIPTION);
qemu_put_be32(f, vmdesc_len);
- qemu_put_buffer(f, (uint8_t *)qjson_get_str(vmdesc), vmdesc_len);
+ qemu_put_buffer(f, (uint8_t *)vmdesc_str, vmdesc_len);
}
- object_unref(OBJECT(vmdesc));
+ g_free(vmdesc_str);
+ json_output_visitor_cleanup(vmdesc_jov);
qemu_fflush(f);
}
diff --git a/migration/vmstate.c b/migration/vmstate.c
index bf3d5db..e7f894c 100644
--- a/migration/vmstate.c
+++ b/migration/vmstate.c
@@ -1,15 +1,15 @@
#include "qemu/osdep.h"
#include "qemu-common.h"
+#include "qapi/error.h"
#include "migration/migration.h"
#include "migration/qemu-file.h"
#include "migration/vmstate.h"
#include "qemu/bitops.h"
#include "qemu/error-report.h"
#include "trace.h"
-#include "qjson.h"
static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
- void *opaque, QJSON *vmdesc);
+ void *opaque, Visitor *vmdesc);
static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque);
@@ -226,12 +226,15 @@ static bool vmsd_can_compress(VMStateField *field)
return true;
}
-static void vmsd_desc_field_start(const VMStateDescription *vmsd, QJSON *vmdesc,
+static void vmsd_desc_field_start(const VMStateDescription *vmsd,
+ Visitor *vmdesc,
VMStateField *field, int i, int max)
{
char *name, *old_name;
bool is_array = max > 1;
bool can_compress = vmsd_can_compress(field);
+ int64_t tmp;
+ const char *type;
if (!vmdesc) {
return;
@@ -247,38 +250,47 @@ static void vmsd_desc_field_start(const VMStateDescription *vmsd, QJSON *vmdesc,
g_free(old_name);
}
- json_start_object(vmdesc, NULL);
- json_prop_str(vmdesc, "name", name);
+ visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
+ visit_type_str(vmdesc, "name", &name, &error_abort);
if (is_array) {
if (can_compress) {
- json_prop_int(vmdesc, "array_len", max);
+ tmp = max;
+ visit_type_int(vmdesc, "array_len", &tmp, &error_abort);
} else {
- json_prop_int(vmdesc, "index", i);
+ tmp = i;
+ visit_type_int(vmdesc, "index", &tmp, &error_abort);
}
}
- json_prop_str(vmdesc, "type", vmfield_get_type_name(field));
+ type = vmfield_get_type_name(field);
+ visit_type_str(vmdesc, "type", (char **)&type, &error_abort);
if (field->flags & VMS_STRUCT) {
- json_start_object(vmdesc, "struct");
+ visit_start_struct(vmdesc, "struct", NULL, 0, &error_abort);
}
g_free(name);
}
-static void vmsd_desc_field_end(const VMStateDescription *vmsd, QJSON *vmdesc,
+static void vmsd_desc_field_end(const VMStateDescription *vmsd,
+ Visitor *vmdesc,
VMStateField *field, size_t size, int i)
{
+ int64_t tmp;
+
if (!vmdesc) {
return;
}
if (field->flags & VMS_STRUCT) {
/* We printed a struct in between, close its child object */
- json_end_object(vmdesc);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
}
- json_prop_int(vmdesc, "size", size);
- json_end_object(vmdesc);
+ tmp = size;
+ visit_type_int(vmdesc, "size", &tmp, &error_abort);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
}
@@ -293,7 +305,7 @@ bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque)
void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
- void *opaque, QJSON *vmdesc)
+ void *opaque, Visitor *vmdesc)
{
VMStateField *field = vmsd->fields;
@@ -302,9 +314,11 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
}
if (vmdesc) {
- json_prop_str(vmdesc, "vmsd_name", vmsd->name);
- json_prop_int(vmdesc, "version", vmsd->version_id);
- json_start_array(vmdesc, "fields");
+ const char *tmp_s = vmsd->name;
+ int64_t tmp_i = vmsd->version_id;
+ visit_type_str(vmdesc, "vmsd_name", (char **)&tmp_s, &error_abort);
+ visit_type_int(vmdesc, "version", &tmp_i, &error_abort);
+ visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
}
while (field->name) {
@@ -314,7 +328,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
int i, n_elems = vmstate_n_elems(opaque, field);
int size = vmstate_size(opaque, field);
int64_t old_offset, written_bytes;
- QJSON *vmdesc_loop = vmdesc;
+ Visitor *vmdesc_loop = vmdesc;
for (i = 0; i < n_elems; i++) {
void *addr = base_addr + size * i;
@@ -350,7 +364,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
}
if (vmdesc) {
- json_end_array(vmdesc);
+ visit_end_list(vmdesc);
}
vmstate_subsection_save(f, vmsd, opaque, vmdesc);
@@ -420,7 +434,7 @@ static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
}
static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
- void *opaque, QJSON *vmdesc)
+ void *opaque, Visitor *vmdesc)
{
const VMStateDescription **sub = vmsd->subsections;
bool subsection_found = false;
@@ -433,11 +447,12 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
if (vmdesc) {
/* Only create subsection array when we have any */
if (!subsection_found) {
- json_start_array(vmdesc, "subsections");
+ visit_start_list(vmdesc, "subsections", NULL, 0,
+ &error_abort);
subsection_found = true;
}
- json_start_object(vmdesc, NULL);
+ visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
}
qemu_put_byte(f, QEMU_VM_SUBSECTION);
@@ -448,14 +463,15 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
vmstate_save_state(f, vmsd, opaque, vmdesc);
if (vmdesc) {
- json_end_object(vmdesc);
+ visit_check_struct(vmdesc, &error_abort);
+ visit_end_struct(vmdesc);
}
}
sub++;
}
if (vmdesc && subsection_found) {
- json_end_array(vmdesc);
+ visit_end_list(vmdesc);
}
}
diff --git a/tests/Makefile b/tests/Makefile
index 0b5a7a8..1f8a39d 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -442,7 +442,7 @@ tests/test-qdev-global-props$(EXESUF): tests/test-qdev-global-props.o \
$(test-qapi-obj-y)
tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
migration/vmstate.o migration/qemu-file.o migration/qemu-file-buf.o \
- migration/qemu-file-unix.o qjson.o \
+ migration/qemu-file-unix.o qapi/json-output-visitor.o \
$(test-qom-obj-y)
tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
$(test-util-obj-y)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor Eric Blake
@ 2016-05-02 13:26 ` Markus Armbruster
2016-05-02 14:23 ` Eric Blake
2016-05-03 9:44 ` Dr. David Alan Gilbert
1 sibling, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 13:26 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Amit Shah, famz, Juan Quintela
Eric Blake <eblake@redhat.com> writes:
> Rather than using a QJSON object and converting the QString result
> to a char *, we can use the new JSON output visitor and get directly
> to a char *.
>
> The conversions are a bit tricky in place (in places, we have to
> copy an integer to an int64_t temporary to get the right pointer for
> visit_type_int(); and for several strings, we have to copy to a
> temporary variable so we can take an address (&char[] is not the
> same as &char*) and cast away const), but overall still fairly
> mechanical.
>
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: retitle, rebase to master
> v2: new patch
> ---
> include/migration/vmstate.h | 4 +--
> migration/savevm.c | 68 ++++++++++++++++++++++++++++-----------------
> migration/vmstate.c | 64 ++++++++++++++++++++++++++----------------
> tests/Makefile | 2 +-
> 4 files changed, 86 insertions(+), 52 deletions(-)
>
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index 84ee355..2cdfce9 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -29,7 +29,7 @@
> #ifndef CONFIG_USER_ONLY
> #include <migration/qemu-file.h>
> #endif
> -#include <qjson.h>
> +#include "qapi/json-output-visitor.h"
>
> typedef void SaveStateHandler(QEMUFile *f, void *opaque);
> typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
> @@ -925,7 +925,7 @@ void loadvm_free_handlers(MigrationIncomingState *mis);
> int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
> void *opaque, int version_id);
> void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> - void *opaque, QJSON *vmdesc);
> + void *opaque, Visitor *vmdesc);
>
> bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque);
>
> diff --git a/migration/savevm.c b/migration/savevm.c
> index 16ba443..c858fe9 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -2,7 +2,7 @@
> * QEMU System Emulator
> *
> * Copyright (c) 2003-2008 Fabrice Bellard
> - * Copyright (c) 2009-2015 Red Hat Inc
> + * Copyright (c) 2009-2016 Red Hat Inc
> *
> * Authors:
> * Juan Quintela <quintela@redhat.com>
> @@ -645,27 +645,32 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
> return vmstate_load_state(f, se->vmsd, se->opaque, version_id);
> }
>
> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
> + Visitor *vmdesc)
> {
> int64_t old_offset, size;
> + const char *tmp;
>
> old_offset = qemu_ftell_fast(f);
> se->ops->save_state(f, se->opaque);
> size = qemu_ftell_fast(f) - old_offset;
>
> if (vmdesc) {
Conditionals could be avoided: use a null visitor. Not sure it's worth
it, though.
> - json_prop_int(vmdesc, "size", size);
> - json_start_array(vmdesc, "fields");
> - json_start_object(vmdesc, NULL);
> - json_prop_str(vmdesc, "name", "data");
> - json_prop_int(vmdesc, "size", size);
> - json_prop_str(vmdesc, "type", "buffer");
> - json_end_object(vmdesc);
> - json_end_array(vmdesc);
> + visit_type_int(vmdesc, "size", &size, &error_abort);
> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> + tmp = "data";
> + visit_type_str(vmdesc, "name", (char **)&tmp, &error_abort);
The Visitor interface is the same for input and for output. Convenient
when the code is direction-agnostic. Inconvenient when it's output: you
have to pass the value by reference even though it's only read. In
particular, literals need a temporary, and types have to be adjusted via
cast or temporary more frequently than for by-value.
If that bothers us, we can add by-value wrappers to the interface.
Are there other output-only visitor uses?
> + visit_type_int(vmdesc, "size", &size, &error_abort);
> + tmp = "buffer";
> + visit_type_str(vmdesc, "type", (char **)&tmp, &error_abort);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> + visit_end_list(vmdesc);
> }
> }
>
> -static void vmstate_save(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
> +static void vmstate_save(QEMUFile *f, SaveStateEntry *se, Visitor *vmdesc)
> {
> trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)");
> if (!se->vmsd) {
> @@ -891,7 +896,7 @@ void qemu_savevm_state_header(QEMUFile *f)
>
> if (!savevm_state.skip_configuration || enforce_config_section()) {
> qemu_put_byte(f, QEMU_VM_CONFIGURATION);
> - vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0);
> + vmstate_save_state(f, &vmstate_configuration, &savevm_state, NULL);
Cleans up use of 0 as pointer literal while there, good.
Note to self: use Coccinelle to find this style bug's buddies.
> }
>
> }
[...]
Well, it doesn't exactly make this code prettier, but having a stupid
wrapper just to hide the ugliness isn't so hot, either.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-02 13:26 ` Markus Armbruster
@ 2016-05-02 14:23 ` Eric Blake
2016-05-03 8:30 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-02 14:23 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Amit Shah, famz, Juan Quintela
[-- Attachment #1: Type: text/plain, Size: 2641 bytes --]
On 05/02/2016 07:26 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Rather than using a QJSON object and converting the QString result
>> to a char *, we can use the new JSON output visitor and get directly
>> to a char *.
>>
>> The conversions are a bit tricky in place (in places, we have to
>> copy an integer to an int64_t temporary to get the right pointer for
>> visit_type_int(); and for several strings, we have to copy to a
>> temporary variable so we can take an address (&char[] is not the
>> same as &char*) and cast away const), but overall still fairly
>> mechanical.
>>
>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
>> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
>> + Visitor *vmdesc)
>> {
>> int64_t old_offset, size;
>> + const char *tmp;
>>
>> old_offset = qemu_ftell_fast(f);
>> se->ops->save_state(f, se->opaque);
>> size = qemu_ftell_fast(f) - old_offset;
>>
>> if (vmdesc) {
>
> Conditionals could be avoided: use a null visitor. Not sure it's worth
> it, though.
We could just teach qapi-visit-core.c to be a no-op for v==NULL (thus
hiding the conditionals in the core code, but that then slows down the
common case for more conditionals on every caller. Maybe a null visitor
is reasonable, after all?
>> + tmp = "data";
>> + visit_type_str(vmdesc, "name", (char **)&tmp, &error_abort);
>
> The Visitor interface is the same for input and for output. Convenient
> when the code is direction-agnostic. Inconvenient when it's output: you
> have to pass the value by reference even though it's only read. In
> particular, literals need a temporary, and types have to be adjusted via
> cast or temporary more frequently than for by-value.
>
> If that bothers us, we can add by-value wrappers to the interface.
>
> Are there other output-only visitor uses?
qom-get is output-only, just as qom-set is input-only. Maybe it's worth
an experiment to see how difficult it would be.
> Well, it doesn't exactly make this code prettier, but having a stupid
> wrapper just to hide the ugliness isn't so hot, either.
And now you see why I posted two alternatives, to see which way we want
to go. Having convenient wrappers for output-only visits may swing the
vote.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-02 14:23 ` Eric Blake
@ 2016-05-03 8:30 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 8:30 UTC (permalink / raw)
To: Eric Blake; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
Eric Blake <eblake@redhat.com> writes:
> On 05/02/2016 07:26 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Rather than using a QJSON object and converting the QString result
>>> to a char *, we can use the new JSON output visitor and get directly
>>> to a char *.
>>>
>>> The conversions are a bit tricky in place (in places, we have to
>>> copy an integer to an int64_t temporary to get the right pointer for
>>> visit_type_int(); and for several strings, we have to copy to a
>>> temporary variable so we can take an address (&char[] is not the
>>> same as &char*) and cast away const), but overall still fairly
>>> mechanical.
>>>
>>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>
>>> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
>>> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
>>> + Visitor *vmdesc)
>>> {
>>> int64_t old_offset, size;
>>> + const char *tmp;
>>>
>>> old_offset = qemu_ftell_fast(f);
>>> se->ops->save_state(f, se->opaque);
>>> size = qemu_ftell_fast(f) - old_offset;
>>>
>>> if (vmdesc) {
>>
>> Conditionals could be avoided: use a null visitor. Not sure it's worth
>> it, though.
>
> We could just teach qapi-visit-core.c to be a no-op for v==NULL (thus
> hiding the conditionals in the core code, but that then slows down the
> common case for more conditionals on every caller. Maybe a null visitor
> is reasonable, after all?
I'd prefer a null visitor to messing up the core with conditionals.
Again: not sure avoiding the conditionals here is worth a null visitor.
If you want to find out, cook up a patch and we'll see.
>>> + tmp = "data";
>>> + visit_type_str(vmdesc, "name", (char **)&tmp, &error_abort);
>>
>> The Visitor interface is the same for input and for output. Convenient
>> when the code is direction-agnostic. Inconvenient when it's output: you
>> have to pass the value by reference even though it's only read. In
>> particular, literals need a temporary, and types have to be adjusted via
>> cast or temporary more frequently than for by-value.
>>
>> If that bothers us, we can add by-value wrappers to the interface.
>>
>> Are there other output-only visitor uses?
>
> qom-get is output-only, just as qom-set is input-only. Maybe it's worth
> an experiment to see how difficult it would be.
>
>
>> Well, it doesn't exactly make this code prettier, but having a stupid
>> wrapper just to hide the ugliness isn't so hot, either.
>
> And now you see why I posted two alternatives, to see which way we want
> to go. Having convenient wrappers for output-only visits may swing the
> vote.
Having read your first alternative, my immediate reaction was "why don't
you do X instead?", where X turned out to be your second alternative. I
really don't like this feature-limited wrapper that is used in just one
place. I also don't like its confusing git-log, courtesy of its
unwisely chosen filename.
But having read the second alternative, I understand why you're offering
the first one: both are ugly.
So which of the uglies do I prefer? The experiment could indeed swing
my vote.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor Eric Blake
2016-05-02 13:26 ` Markus Armbruster
@ 2016-05-03 9:44 ` Dr. David Alan Gilbert
2016-05-03 12:26 ` Markus Armbruster
1 sibling, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-03 9:44 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Amit Shah, famz, armbru, Juan Quintela
* Eric Blake (eblake@redhat.com) wrote:
> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
> + Visitor *vmdesc)
> {
> int64_t old_offset, size;
> + const char *tmp;
>
> old_offset = qemu_ftell_fast(f);
> se->ops->save_state(f, se->opaque);
> size = qemu_ftell_fast(f) - old_offset;
>
> if (vmdesc) {
> - json_prop_int(vmdesc, "size", size);
> - json_start_array(vmdesc, "fields");
> - json_start_object(vmdesc, NULL);
> - json_prop_str(vmdesc, "name", "data");
> - json_prop_int(vmdesc, "size", size);
> - json_prop_str(vmdesc, "type", "buffer");
> - json_end_object(vmdesc);
> - json_end_array(vmdesc);
> + visit_type_int(vmdesc, "size", &size, &error_abort);
> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
Please avoid error_abort in migration code, especially on the source side.
You've got an apparently happily working VM, we must never kill it
while attempting migration.
Dave
> + tmp = "data";
> + visit_type_str(vmdesc, "name", (char **)&tmp, &error_abort);
> + visit_type_int(vmdesc, "size", &size, &error_abort);
> + tmp = "buffer";
> + visit_type_str(vmdesc, "type", (char **)&tmp, &error_abort);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> + visit_end_list(vmdesc);
> }
> }
>
> -static void vmstate_save(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
> +static void vmstate_save(QEMUFile *f, SaveStateEntry *se, Visitor *vmdesc)
> {
> trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)");
> if (!se->vmsd) {
> @@ -891,7 +896,7 @@ void qemu_savevm_state_header(QEMUFile *f)
>
> if (!savevm_state.skip_configuration || enforce_config_section()) {
> qemu_put_byte(f, QEMU_VM_CONFIGURATION);
> - vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0);
> + vmstate_save_state(f, &vmstate_configuration, &savevm_state, NULL);
> }
>
> }
> @@ -1033,11 +1038,15 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f)
>
> void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
> {
> - QJSON *vmdesc;
> + JsonOutputVisitor *vmdesc_jov;
> + Visitor *vmdesc;
> + char *vmdesc_str;
> int vmdesc_len;
> SaveStateEntry *se;
> int ret;
> bool in_postcopy = migration_in_postcopy(migrate_get_current());
> + int64_t tmp_i;
> + char *tmp_s;
>
> trace_savevm_state_complete_precopy();
>
> @@ -1073,9 +1082,12 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
> return;
> }
>
> - vmdesc = qjson_new();
> - json_prop_int(vmdesc, "page_size", TARGET_PAGE_SIZE);
> - json_start_array(vmdesc, "devices");
> + vmdesc_jov = json_output_visitor_new();
> + vmdesc = json_output_get_visitor(vmdesc_jov);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> + tmp_i = TARGET_PAGE_SIZE;
> + visit_type_int(vmdesc, "page_size", &tmp_i, &error_abort);
> + visit_start_list(vmdesc, "devices", NULL, 0, &error_abort);
> QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {
>
> if ((!se->ops || !se->ops->save_state) && !se->vmsd) {
> @@ -1088,16 +1100,19 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
>
> trace_savevm_section_start(se->idstr, se->section_id);
>
> - json_start_object(vmdesc, NULL);
> - json_prop_str(vmdesc, "name", se->idstr);
> - json_prop_int(vmdesc, "instance_id", se->instance_id);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> + tmp_s = se->idstr;
> + visit_type_str(vmdesc, "name", &tmp_s, &error_abort);
> + tmp_i = se->instance_id;
> + visit_type_int(vmdesc, "instance_id", &tmp_i, &error_abort);
>
> save_section_header(f, se, QEMU_VM_SECTION_FULL);
> vmstate_save(f, se, vmdesc);
> trace_savevm_section_end(se->idstr, se->section_id, 0);
> save_section_footer(f, se);
>
> - json_end_object(vmdesc);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> }
>
> if (!in_postcopy) {
> @@ -1105,16 +1120,19 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
> qemu_put_byte(f, QEMU_VM_EOF);
> }
>
> - json_end_array(vmdesc);
> - qjson_finish(vmdesc);
> - vmdesc_len = strlen(qjson_get_str(vmdesc));
> + visit_end_list(vmdesc);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> + vmdesc_str = json_output_get_string(vmdesc_jov);
> + vmdesc_len = strlen(vmdesc_str);
>
> if (should_send_vmdesc()) {
> qemu_put_byte(f, QEMU_VM_VMDESCRIPTION);
> qemu_put_be32(f, vmdesc_len);
> - qemu_put_buffer(f, (uint8_t *)qjson_get_str(vmdesc), vmdesc_len);
> + qemu_put_buffer(f, (uint8_t *)vmdesc_str, vmdesc_len);
> }
> - object_unref(OBJECT(vmdesc));
> + g_free(vmdesc_str);
> + json_output_visitor_cleanup(vmdesc_jov);
>
> qemu_fflush(f);
> }
> diff --git a/migration/vmstate.c b/migration/vmstate.c
> index bf3d5db..e7f894c 100644
> --- a/migration/vmstate.c
> +++ b/migration/vmstate.c
> @@ -1,15 +1,15 @@
> #include "qemu/osdep.h"
> #include "qemu-common.h"
> +#include "qapi/error.h"
> #include "migration/migration.h"
> #include "migration/qemu-file.h"
> #include "migration/vmstate.h"
> #include "qemu/bitops.h"
> #include "qemu/error-report.h"
> #include "trace.h"
> -#include "qjson.h"
>
> static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
> - void *opaque, QJSON *vmdesc);
> + void *opaque, Visitor *vmdesc);
> static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
> void *opaque);
>
> @@ -226,12 +226,15 @@ static bool vmsd_can_compress(VMStateField *field)
> return true;
> }
>
> -static void vmsd_desc_field_start(const VMStateDescription *vmsd, QJSON *vmdesc,
> +static void vmsd_desc_field_start(const VMStateDescription *vmsd,
> + Visitor *vmdesc,
> VMStateField *field, int i, int max)
> {
> char *name, *old_name;
> bool is_array = max > 1;
> bool can_compress = vmsd_can_compress(field);
> + int64_t tmp;
> + const char *type;
>
> if (!vmdesc) {
> return;
> @@ -247,38 +250,47 @@ static void vmsd_desc_field_start(const VMStateDescription *vmsd, QJSON *vmdesc,
> g_free(old_name);
> }
>
> - json_start_object(vmdesc, NULL);
> - json_prop_str(vmdesc, "name", name);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> + visit_type_str(vmdesc, "name", &name, &error_abort);
> if (is_array) {
> if (can_compress) {
> - json_prop_int(vmdesc, "array_len", max);
> + tmp = max;
> + visit_type_int(vmdesc, "array_len", &tmp, &error_abort);
> } else {
> - json_prop_int(vmdesc, "index", i);
> + tmp = i;
> + visit_type_int(vmdesc, "index", &tmp, &error_abort);
> }
> }
> - json_prop_str(vmdesc, "type", vmfield_get_type_name(field));
> + type = vmfield_get_type_name(field);
> + visit_type_str(vmdesc, "type", (char **)&type, &error_abort);
>
> if (field->flags & VMS_STRUCT) {
> - json_start_object(vmdesc, "struct");
> + visit_start_struct(vmdesc, "struct", NULL, 0, &error_abort);
> }
>
> g_free(name);
> }
>
> -static void vmsd_desc_field_end(const VMStateDescription *vmsd, QJSON *vmdesc,
> +static void vmsd_desc_field_end(const VMStateDescription *vmsd,
> + Visitor *vmdesc,
> VMStateField *field, size_t size, int i)
> {
> + int64_t tmp;
> +
> if (!vmdesc) {
> return;
> }
>
> if (field->flags & VMS_STRUCT) {
> /* We printed a struct in between, close its child object */
> - json_end_object(vmdesc);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> }
>
> - json_prop_int(vmdesc, "size", size);
> - json_end_object(vmdesc);
> + tmp = size;
> + visit_type_int(vmdesc, "size", &tmp, &error_abort);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> }
>
>
> @@ -293,7 +305,7 @@ bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque)
>
>
> void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> - void *opaque, QJSON *vmdesc)
> + void *opaque, Visitor *vmdesc)
> {
> VMStateField *field = vmsd->fields;
>
> @@ -302,9 +314,11 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> }
>
> if (vmdesc) {
> - json_prop_str(vmdesc, "vmsd_name", vmsd->name);
> - json_prop_int(vmdesc, "version", vmsd->version_id);
> - json_start_array(vmdesc, "fields");
> + const char *tmp_s = vmsd->name;
> + int64_t tmp_i = vmsd->version_id;
> + visit_type_str(vmdesc, "vmsd_name", (char **)&tmp_s, &error_abort);
> + visit_type_int(vmdesc, "version", &tmp_i, &error_abort);
> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> }
>
> while (field->name) {
> @@ -314,7 +328,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> int i, n_elems = vmstate_n_elems(opaque, field);
> int size = vmstate_size(opaque, field);
> int64_t old_offset, written_bytes;
> - QJSON *vmdesc_loop = vmdesc;
> + Visitor *vmdesc_loop = vmdesc;
>
> for (i = 0; i < n_elems; i++) {
> void *addr = base_addr + size * i;
> @@ -350,7 +364,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> }
>
> if (vmdesc) {
> - json_end_array(vmdesc);
> + visit_end_list(vmdesc);
> }
>
> vmstate_subsection_save(f, vmsd, opaque, vmdesc);
> @@ -420,7 +434,7 @@ static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
> }
>
> static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
> - void *opaque, QJSON *vmdesc)
> + void *opaque, Visitor *vmdesc)
> {
> const VMStateDescription **sub = vmsd->subsections;
> bool subsection_found = false;
> @@ -433,11 +447,12 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
> if (vmdesc) {
> /* Only create subsection array when we have any */
> if (!subsection_found) {
> - json_start_array(vmdesc, "subsections");
> + visit_start_list(vmdesc, "subsections", NULL, 0,
> + &error_abort);
> subsection_found = true;
> }
>
> - json_start_object(vmdesc, NULL);
> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> }
>
> qemu_put_byte(f, QEMU_VM_SUBSECTION);
> @@ -448,14 +463,15 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
> vmstate_save_state(f, vmsd, opaque, vmdesc);
>
> if (vmdesc) {
> - json_end_object(vmdesc);
> + visit_check_struct(vmdesc, &error_abort);
> + visit_end_struct(vmdesc);
> }
> }
> sub++;
> }
>
> if (vmdesc && subsection_found) {
> - json_end_array(vmdesc);
> + visit_end_list(vmdesc);
> }
> }
>
> diff --git a/tests/Makefile b/tests/Makefile
> index 0b5a7a8..1f8a39d 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -442,7 +442,7 @@ tests/test-qdev-global-props$(EXESUF): tests/test-qdev-global-props.o \
> $(test-qapi-obj-y)
> tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
> migration/vmstate.o migration/qemu-file.o migration/qemu-file-buf.o \
> - migration/qemu-file-unix.o qjson.o \
> + migration/qemu-file-unix.o qapi/json-output-visitor.o \
> $(test-qom-obj-y)
> tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
> $(test-util-obj-y)
> --
> 2.5.5
>
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 9:44 ` Dr. David Alan Gilbert
@ 2016-05-03 12:26 ` Markus Armbruster
2016-05-03 12:34 ` Eric Blake
2016-05-03 13:23 ` Dr. David Alan Gilbert
0 siblings, 2 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 12:26 UTC (permalink / raw)
To: Dr. David Alan Gilbert
Cc: Eric Blake, Amit Shah, Juan Quintela, famz, qemu-devel
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Eric Blake (eblake@redhat.com) wrote:
>
>> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
>> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
>> + Visitor *vmdesc)
>> {
>> int64_t old_offset, size;
>> + const char *tmp;
>>
>> old_offset = qemu_ftell_fast(f);
>> se->ops->save_state(f, se->opaque);
>> size = qemu_ftell_fast(f) - old_offset;
>>
>> if (vmdesc) {
>> - json_prop_int(vmdesc, "size", size);
>> - json_start_array(vmdesc, "fields");
>> - json_start_object(vmdesc, NULL);
>> - json_prop_str(vmdesc, "name", "data");
>> - json_prop_int(vmdesc, "size", size);
>> - json_prop_str(vmdesc, "type", "buffer");
>> - json_end_object(vmdesc);
>> - json_end_array(vmdesc);
>> + visit_type_int(vmdesc, "size", &size, &error_abort);
>> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
>> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
>
> Please avoid error_abort in migration code, especially on the source side.
> You've got an apparently happily working VM, we must never kill it
> while attempting migration.
These functions cannot fail, and &error_abort is a concise way to
express that. It's the same as
visit_type_int(vmdesc, "size", &size, &err);
assert(!err);
An alternative would be ignoring errors:
visit_type_int(vmdesc, "size", &size, NULL);
Ignoring violations of design invariants is hardly ever a good idea,
though.
Another alternative would be trying to recover from the violation, like
this:
visit_type_int(vmdesc, "size", &size, &err);
if (err) {
report we're fscked...
do whatever needs to be done to recover...
goto out;
}
Fancy untestable error paths are hardly ever good ideas, either.
Complete list of conditions where the JSON output visitor sets an error:
* Conditions where the visitor core sets an error:
- visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
a value out of bounds. This is a serious programming error in
qapi-visit-core.c. We're almost certainly screwed, and attempting
to continue is unsafe.
- visit_type_int(): likewise.
- output_type_enum() when the numeric value is out of bounds. This is
either a serious programming error in qapi-visit-core.c, or
corrupted state. Either way, we're almost certainly screwed, and
attempting to continue is unsafe.
- input_type_enum() when the string value is unknown. This is either
a serious programming error in qapi-visit-core.c, or bad input.
However, the JSON output visitor isn't supposed to ever call
input_type_enum(), so it's the former. Once again, we're almost
certainly screwed, and attempting to continue is unsafe.
* Conditions where the JSON output visitor itself sets an error:
- None.
Do you still object to &error_abort?
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 12:26 ` Markus Armbruster
@ 2016-05-03 12:34 ` Eric Blake
2016-05-03 13:27 ` Dr. David Alan Gilbert
2016-05-03 13:23 ` Dr. David Alan Gilbert
1 sibling, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-03 12:34 UTC (permalink / raw)
To: Markus Armbruster, Dr. David Alan Gilbert
Cc: Amit Shah, Juan Quintela, famz, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]
On 05/03/2016 06:26 AM, Markus Armbruster wrote:
>>> + visit_type_int(vmdesc, "size", &size, &error_abort);
>>> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
>>> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
>>
>> Please avoid error_abort in migration code, especially on the source side.
>> You've got an apparently happily working VM, we must never kill it
>> while attempting migration.
>
> These functions cannot fail, and &error_abort is a concise way to
> express that. It's the same as
>
> visit_type_int(vmdesc, "size", &size, &err);
> assert(!err);
&error_abort is ONLY supposed to be used to flag programming errors (ie.
they should never be reachable). I'm asserting that the errors don't
happen, and therefore this cannot make the migration fail - in other
words, this is NOT going to kill a VM that attempts migration.
> * Conditions where the JSON output visitor itself sets an error:
>
> - None.
The JSON output visitor itself may be adding an error for an attempt to
output Inf or NaN for a floating point number - but since vmstate
doesn't use visit_type_number(), this is not possible. And if we are
really worried about it, then in my next spin of the patch I may make it
user-configurable whether we stick to strict JSON or whether we relax
things and output Inf/NaN anyways.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 12:34 ` Eric Blake
@ 2016-05-03 13:27 ` Dr. David Alan Gilbert
2016-05-04 8:39 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-03 13:27 UTC (permalink / raw)
To: Eric Blake; +Cc: Markus Armbruster, Amit Shah, Juan Quintela, famz, qemu-devel
* Eric Blake (eblake@redhat.com) wrote:
> On 05/03/2016 06:26 AM, Markus Armbruster wrote:
>
> >>> + visit_type_int(vmdesc, "size", &size, &error_abort);
> >>> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> >>> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> >>
> >> Please avoid error_abort in migration code, especially on the source side.
> >> You've got an apparently happily working VM, we must never kill it
> >> while attempting migration.
> >
> > These functions cannot fail, and &error_abort is a concise way to
> > express that. It's the same as
> >
> > visit_type_int(vmdesc, "size", &size, &err);
> > assert(!err);
>
> &error_abort is ONLY supposed to be used to flag programming errors (ie.
> they should never be reachable). I'm asserting that the errors don't
> happen, and therefore this cannot make the migration fail - in other
> words, this is NOT going to kill a VM that attempts migration.
OK, but remember that I work on the basis that there are programming errors
in both the migration code and the VMState descriptions for devices.
If those break it still shouldn't kill the source.
(Note this isn't just true of migration - we need to be careful about
it in all cases where we're doing stuff to an otherwise happy VM).
> > * Conditions where the JSON output visitor itself sets an error:
> >
> > - None.
>
> The JSON output visitor itself may be adding an error for an attempt to
> output Inf or NaN for a floating point number - but since vmstate
> doesn't use visit_type_number(), this is not possible. And if we are
> really worried about it, then in my next spin of the patch I may make it
> user-configurable whether we stick to strict JSON or whether we relax
> things and output Inf/NaN anyways.
If that's the only case, and you're already saying it doesn't use it, then
I don't see there's a point in making that bit any more configurable.
Dave
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 13:27 ` Dr. David Alan Gilbert
@ 2016-05-04 8:39 ` Markus Armbruster
2016-05-04 8:54 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 8:39 UTC (permalink / raw)
To: Dr. David Alan Gilbert
Cc: Eric Blake, Amit Shah, qemu-devel, famz, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Eric Blake (eblake@redhat.com) wrote:
>> On 05/03/2016 06:26 AM, Markus Armbruster wrote:
>>
>> >>> + visit_type_int(vmdesc, "size", &size, &error_abort);
>> >>> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
>> >>> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
>> >>
>> >> Please avoid error_abort in migration code, especially on the source side.
>> >> You've got an apparently happily working VM, we must never kill it
>> >> while attempting migration.
>> >
>> > These functions cannot fail, and &error_abort is a concise way to
>> > express that. It's the same as
>> >
>> > visit_type_int(vmdesc, "size", &size, &err);
>> > assert(!err);
>>
>> &error_abort is ONLY supposed to be used to flag programming errors (ie.
>> they should never be reachable). I'm asserting that the errors don't
>> happen, and therefore this cannot make the migration fail - in other
>> words, this is NOT going to kill a VM that attempts migration.
>
> OK, but remember that I work on the basis that there are programming errors
> in both the migration code and the VMState descriptions for devices.
> If those break it still shouldn't kill the source.
> (Note this isn't just true of migration - we need to be careful about
> it in all cases where we're doing stuff to an otherwise happy VM).
While you can safely recover from certain programming errors, you can't
do it in general. Worse, deciding whether recovery from a certain
programming error is safe can be intractable.
Example: visit_type_enum(v, name, &enum_val, enum_str, &err), where v is
an output visitor. This can fail when enum_val is not a valid subscript
of enum_str[]. Can we recover safely? Assume that we can cleanly fail
the task at hand at this point of its execution.
Perhaps enum_str[] doesn't match the actual enum. This is a programming
error. Failing the task is graceful degradation, and safe enough.
But what if enum_str[] is fine, but enum_val got corrupted? Then
failing the task is still safe as long as enum_val isn't visible outside
the task. But if it is visible, all bets are off. The corruption can
spread, and do real damage. Can be the difference between a crash that
forces a reboot with a filesystem journal replay, and massive data
corruption.
So, should we try to recover here? Assuming we want to, badly. If
analysis shows the possible causes of this error are safely isolated by
the recovery, yes. Without such analysis, the only prudent answer is
no.
Real world examples typically deal with state more complex than just an
enum (all too often a thicket of pointers), and the safety argument gets
much hairier.
If you want more tractable arguments, try Erlang.
>> > * Conditions where the JSON output visitor itself sets an error:
>> >
>> > - None.
>>
>> The JSON output visitor itself may be adding an error for an attempt to
>> output Inf or NaN for a floating point number - but since vmstate
>> doesn't use visit_type_number(), this is not possible. And if we are
>> really worried about it, then in my next spin of the patch I may make it
>> user-configurable whether we stick to strict JSON or whether we relax
>> things and output Inf/NaN anyways.
>
> If that's the only case, and you're already saying it doesn't use it, then
> I don't see there's a point in making that bit any more configurable.
I listed all possible failures of the JSON output visitor upthread.
This is an additional failure we've considered. I'm wary of adding it
precisely because I do worry about upsetting apple carts like this one.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 8:39 ` Markus Armbruster
@ 2016-05-04 8:54 ` Dr. David Alan Gilbert
2016-05-24 7:15 ` Paolo Bonzini
0 siblings, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-04 8:54 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Eric Blake, Amit Shah, qemu-devel, famz, Juan Quintela
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Eric Blake (eblake@redhat.com) wrote:
> >> On 05/03/2016 06:26 AM, Markus Armbruster wrote:
> >>
> >> >>> + visit_type_int(vmdesc, "size", &size, &error_abort);
> >> >>> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> >> >>> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> >> >>
> >> >> Please avoid error_abort in migration code, especially on the source side.
> >> >> You've got an apparently happily working VM, we must never kill it
> >> >> while attempting migration.
> >> >
> >> > These functions cannot fail, and &error_abort is a concise way to
> >> > express that. It's the same as
> >> >
> >> > visit_type_int(vmdesc, "size", &size, &err);
> >> > assert(!err);
> >>
> >> &error_abort is ONLY supposed to be used to flag programming errors (ie.
> >> they should never be reachable). I'm asserting that the errors don't
> >> happen, and therefore this cannot make the migration fail - in other
> >> words, this is NOT going to kill a VM that attempts migration.
> >
> > OK, but remember that I work on the basis that there are programming errors
> > in both the migration code and the VMState descriptions for devices.
> > If those break it still shouldn't kill the source.
> > (Note this isn't just true of migration - we need to be careful about
> > it in all cases where we're doing stuff to an otherwise happy VM).
>
> While you can safely recover from certain programming errors, you can't
> do it in general. Worse, deciding whether recovery from a certain
> programming error is safe can be intractable.
>
> Example: visit_type_enum(v, name, &enum_val, enum_str, &err), where v is
> an output visitor. This can fail when enum_val is not a valid subscript
> of enum_str[]. Can we recover safely? Assume that we can cleanly fail
> the task at hand at this point of its execution.
>
> Perhaps enum_str[] doesn't match the actual enum. This is a programming
> error. Failing the task is graceful degradation, and safe enough.
>
> But what if enum_str[] is fine, but enum_val got corrupted? Then
> failing the task is still safe as long as enum_val isn't visible outside
> the task. But if it is visible, all bets are off. The corruption can
> spread, and do real damage. Can be the difference between a crash that
> forces a reboot with a filesystem journal replay, and massive data
> corruption.
>
> So, should we try to recover here? Assuming we want to, badly. If
> analysis shows the possible causes of this error are safely isolated by
> the recovery, yes. Without such analysis, the only prudent answer is
> no.
>
> Real world examples typically deal with state more complex than just an
> enum (all too often a thicket of pointers), and the safety argument gets
> much hairier.
>
> If you want more tractable arguments, try Erlang.
And so my argument here is very simple; if we believe we have a corruption
in migration data then we fail migration - I don't try and do anything
clever about trying to bound what's broken.
This isn't about getting formal/tractable arguments, it's about making
a practical system.
> >> > * Conditions where the JSON output visitor itself sets an error:
> >> >
> >> > - None.
> >>
> >> The JSON output visitor itself may be adding an error for an attempt to
> >> output Inf or NaN for a floating point number - but since vmstate
> >> doesn't use visit_type_number(), this is not possible. And if we are
> >> really worried about it, then in my next spin of the patch I may make it
> >> user-configurable whether we stick to strict JSON or whether we relax
> >> things and output Inf/NaN anyways.
> >
> > If that's the only case, and you're already saying it doesn't use it, then
> > I don't see there's a point in making that bit any more configurable.
>
> I listed all possible failures of the JSON output visitor upthread.
> This is an additional failure we've considered. I'm wary of adding it
> precisely because I do worry about upsetting apple carts like this one.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 8:54 ` Dr. David Alan Gilbert
@ 2016-05-24 7:15 ` Paolo Bonzini
0 siblings, 0 replies; 78+ messages in thread
From: Paolo Bonzini @ 2016-05-24 7:15 UTC (permalink / raw)
To: Dr. David Alan Gilbert, Markus Armbruster
Cc: Amit Shah, famz, qemu-devel, Juan Quintela
On 04/05/2016 10:54, Dr. David Alan Gilbert wrote:
> And so my argument here is very simple; if we believe we have a corruption
> in migration data then we fail migration - I don't try and do anything
> clever about trying to bound what's broken.
> This isn't about getting formal/tractable arguments, it's about making
> a practical system.
But this code is not driven by the migration data. It is driven by the
vmstate description which is static. It's not const, but still it's not
supposed to get corrupted at all.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 12:26 ` Markus Armbruster
2016-05-03 12:34 ` Eric Blake
@ 2016-05-03 13:23 ` Dr. David Alan Gilbert
2016-05-04 9:11 ` Markus Armbruster
1 sibling, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-03 13:23 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Eric Blake, Amit Shah, Juan Quintela, famz, qemu-devel
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Eric Blake (eblake@redhat.com) wrote:
> >
> >> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
> >> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
> >> + Visitor *vmdesc)
> >> {
> >> int64_t old_offset, size;
> >> + const char *tmp;
> >>
> >> old_offset = qemu_ftell_fast(f);
> >> se->ops->save_state(f, se->opaque);
> >> size = qemu_ftell_fast(f) - old_offset;
> >>
> >> if (vmdesc) {
> >> - json_prop_int(vmdesc, "size", size);
> >> - json_start_array(vmdesc, "fields");
> >> - json_start_object(vmdesc, NULL);
> >> - json_prop_str(vmdesc, "name", "data");
> >> - json_prop_int(vmdesc, "size", size);
> >> - json_prop_str(vmdesc, "type", "buffer");
> >> - json_end_object(vmdesc);
> >> - json_end_array(vmdesc);
> >> + visit_type_int(vmdesc, "size", &size, &error_abort);
> >> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
> >> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
> >
> > Please avoid error_abort in migration code, especially on the source side.
> > You've got an apparently happily working VM, we must never kill it
> > while attempting migration.
>
> These functions cannot fail,
Hang on though - this takes a Visitor* - that could be any visitor and that
could fail.
and &error_abort is a concise way to
> express that. It's the same as
>
> visit_type_int(vmdesc, "size", &size, &err);
> assert(!err);
>
> An alternative would be ignoring errors:
>
> visit_type_int(vmdesc, "size", &size, NULL);
>
> Ignoring violations of design invariants is hardly ever a good idea,
> though.
>
> Another alternative would be trying to recover from the violation, like
> this:
>
> visit_type_int(vmdesc, "size", &size, &err);
> if (err) {
> report we're fscked...
> do whatever needs to be done to recover...
> goto out;
> }
>
> Fancy untestable error paths are hardly ever good ideas, either.
For an outgoing migration we must never kill the source unless we think the
data structures the source is using are itself corrupt.
We get programming errors both in our migration code and the migration
structures on devices.
If our migration code is broken/failing an invariant that still doesn't mean
you should kill the source - it should kill the migration only.
> Complete list of conditions where the JSON output visitor sets an error:
>
> * Conditions where the visitor core sets an error:
>
> - visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
> a value out of bounds. This is a serious programming error in
> qapi-visit-core.c. We're almost certainly screwed, and attempting
> to continue is unsafe.
>
> - visit_type_int(): likewise.
>
> - output_type_enum() when the numeric value is out of bounds. This is
> either a serious programming error in qapi-visit-core.c, or
> corrupted state. Either way, we're almost certainly screwed, and
> attempting to continue is unsafe.
>
> - input_type_enum() when the string value is unknown. This is either
> a serious programming error in qapi-visit-core.c, or bad input.
> However, the JSON output visitor isn't supposed to ever call
> input_type_enum(), so it's the former. Once again, we're almost
> certainly screwed, and attempting to continue is unsafe.
>
> * Conditions where the JSON output visitor itself sets an error:
>
> - None.
>
> Do you still object to &error_abort?
So at the very least it should be commented as to why it can't happen.
My worry about it is that you've got a fairly long comment about why
it can't happen, and I worry that in 6 months someone adds a feature
to either the visitors or the migration code that means there's now
a case where it can happen.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-03 13:23 ` Dr. David Alan Gilbert
@ 2016-05-04 9:11 ` Markus Armbruster
2016-05-04 9:22 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 9:11 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>>
>> > * Eric Blake (eblake@redhat.com) wrote:
>> >
>> >> -static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc)
>> >> +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
>> >> + Visitor *vmdesc)
>> >> {
>> >> int64_t old_offset, size;
>> >> + const char *tmp;
>> >>
>> >> old_offset = qemu_ftell_fast(f);
>> >> se->ops->save_state(f, se->opaque);
>> >> size = qemu_ftell_fast(f) - old_offset;
>> >>
>> >> if (vmdesc) {
>> >> - json_prop_int(vmdesc, "size", size);
>> >> - json_start_array(vmdesc, "fields");
>> >> - json_start_object(vmdesc, NULL);
>> >> - json_prop_str(vmdesc, "name", "data");
>> >> - json_prop_int(vmdesc, "size", size);
>> >> - json_prop_str(vmdesc, "type", "buffer");
>> >> - json_end_object(vmdesc);
>> >> - json_end_array(vmdesc);
>> >> + visit_type_int(vmdesc, "size", &size, &error_abort);
>> >> + visit_start_list(vmdesc, "fields", NULL, 0, &error_abort);
>> >> + visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
>> >
>> > Please avoid error_abort in migration code, especially on the source side.
>> > You've got an apparently happily working VM, we must never kill it
>> > while attempting migration.
>>
>> These functions cannot fail,
>
> Hang on though - this takes a Visitor* - that could be any visitor and that
> could fail.
vmdesc is either NULL or it's the JSON output visitor created by
qemu_savevm_state_complete_precopy():
vmdesc_jov = json_output_visitor_new(false);
vmdesc = json_output_get_visitor(vmdesc_jov);
This is by design: the purpose of this code is *writing* a *JSON*
description of the migration stream. See commit 8118f09.
> and &error_abort is a concise way to
>> express that. It's the same as
>>
>> visit_type_int(vmdesc, "size", &size, &err);
>> assert(!err);
>>
>> An alternative would be ignoring errors:
>>
>> visit_type_int(vmdesc, "size", &size, NULL);
>>
>> Ignoring violations of design invariants is hardly ever a good idea,
>> though.
>>
>> Another alternative would be trying to recover from the violation, like
>> this:
>>
>> visit_type_int(vmdesc, "size", &size, &err);
>> if (err) {
>> report we're fscked...
>> do whatever needs to be done to recover...
>> goto out;
>> }
>>
>> Fancy untestable error paths are hardly ever good ideas, either.
>
> For an outgoing migration we must never kill the source unless we think the
> data structures the source is using are itself corrupt.
> We get programming errors both in our migration code and the migration
> structures on devices.
> If our migration code is broken/failing an invariant that still doesn't mean
> you should kill the source - it should kill the migration only.
"git-grep assert migration" suggests you do kill the source on certain
programming errors.
I reiterate my point that fancy, untestable error recovery is unlikely
to actually recover. "Fancy" can work, "untestable" might work (but
color me skeptic), but once you got both, you're a dead man walking.
>> Complete list of conditions where the JSON output visitor sets an error:
>>
>> * Conditions where the visitor core sets an error:
>>
>> - visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
>> a value out of bounds. This is a serious programming error in
>> qapi-visit-core.c. We're almost certainly screwed, and attempting
>> to continue is unsafe.
>>
>> - visit_type_int(): likewise.
>>
>> - output_type_enum() when the numeric value is out of bounds. This is
>> either a serious programming error in qapi-visit-core.c, or
>> corrupted state. Either way, we're almost certainly screwed, and
>> attempting to continue is unsafe.
>>
>> - input_type_enum() when the string value is unknown. This is either
>> a serious programming error in qapi-visit-core.c, or bad input.
>> However, the JSON output visitor isn't supposed to ever call
>> input_type_enum(), so it's the former. Once again, we're almost
>> certainly screwed, and attempting to continue is unsafe.
>>
>> * Conditions where the JSON output visitor itself sets an error:
>>
>> - None.
>>
>> Do you still object to &error_abort?
>
> So at the very least it should be commented as to why it can't happen.
> My worry about it is that you've got a fairly long comment about why
> it can't happen, and I worry that in 6 months someone adds a feature
> to either the visitors or the migration code that means there's now
> a case where it can happen.
Here's why I don't think new failure modes are likely.
What does this helper module do, and how could it possibly fail? By
"possibly", I mean any conceivable reasonable implementation, not just
the two we have (this patch gets rid of one).
This helper module builds JSON text and returns it as a string. Its
interface mirrors JSON abstract syntax: start object, end object, start
array, end array, string, ... Additionally, initialize, finalize, get
the result as a string.
Conceivable failure modes:
* Out of memory. We die, like we generally do for smallish allocations.
* Data not representable in JSON. This is basically non-finite numbers,
and we already chose to extend JSON instead of making this an error.
Such a decision will not be revised without a thorough analysis of
impact on existing users.
* Interface misused, e.g. invalid nesting. Clearly a programming error.
We can either silently produce garbage output, fail, or die. Before
the patch: garbage output. After the patch: die by assertion failure
(*not* via &error_abort).
* Anything else?
"Not via &error_abort" leads me to another point. The &error_abort are
the assertions you can see in the patch. The ones you can't see are in
the visitor core and the JSON output visitor. They're all about misuse
of the interface.
The old code is different: it doesn't detect misuse, and produces
invalid JSON instead. "Never check for an error you don't know how to
handle."
With the new code, misuse should be caught in general migration testing,
"make check" if it's any good.
With the old code, it could more easily escape testing, because you have
to parse the resulting JSON to detect it.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 9:11 ` Markus Armbruster
@ 2016-05-04 9:22 ` Dr. David Alan Gilbert
2016-05-04 11:37 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-04 9:22 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> "git-grep assert migration" suggests you do kill the source on certain
> programming errors.
I'm just trying hard to reduce them; I know I'm not there, but I'd rather
we didn't have any - especially on the source side.
> I reiterate my point that fancy, untestable error recovery is unlikely
> to actually recover. "Fancy" can work, "untestable" might work (but
> color me skeptic), but once you got both, you're a dead man walking.
Then we should make the error recovery paths easy; at the moment visitor
error paths are just too painful.
>
> >> Complete list of conditions where the JSON output visitor sets an error:
> >>
> >> * Conditions where the visitor core sets an error:
> >>
> >> - visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
> >> a value out of bounds. This is a serious programming error in
> >> qapi-visit-core.c. We're almost certainly screwed, and attempting
> >> to continue is unsafe.
> >>
> >> - visit_type_int(): likewise.
> >>
> >> - output_type_enum() when the numeric value is out of bounds. This is
> >> either a serious programming error in qapi-visit-core.c, or
> >> corrupted state. Either way, we're almost certainly screwed, and
> >> attempting to continue is unsafe.
> >>
> >> - input_type_enum() when the string value is unknown. This is either
> >> a serious programming error in qapi-visit-core.c, or bad input.
> >> However, the JSON output visitor isn't supposed to ever call
> >> input_type_enum(), so it's the former. Once again, we're almost
> >> certainly screwed, and attempting to continue is unsafe.
> >>
> >> * Conditions where the JSON output visitor itself sets an error:
> >>
> >> - None.
> >>
> >> Do you still object to &error_abort?
> >
> > So at the very least it should be commented as to why it can't happen.
> > My worry about it is that you've got a fairly long comment about why
> > it can't happen, and I worry that in 6 months someone adds a feature
> > to either the visitors or the migration code that means there's now
> > a case where it can happen.
>
> Here's why I don't think new failure modes are likely.
>
> What does this helper module do, and how could it possibly fail? By
> "possibly", I mean any conceivable reasonable implementation, not just
> the two we have (this patch gets rid of one).
>
> This helper module builds JSON text and returns it as a string. Its
> interface mirrors JSON abstract syntax: start object, end object, start
> array, end array, string, ... Additionally, initialize, finalize, get
> the result as a string.
>
> Conceivable failure modes:
>
> * Out of memory. We die, like we generally do for smallish allocations.
>
> * Data not representable in JSON. This is basically non-finite numbers,
> and we already chose to extend JSON instead of making this an error.
> Such a decision will not be revised without a thorough analysis of
> impact on existing users.
>
> * Interface misused, e.g. invalid nesting. Clearly a programming error.
> We can either silently produce garbage output, fail, or die. Before
> the patch: garbage output. After the patch: die by assertion failure
> (*not* via &error_abort).
>
> * Anything else?
>
> "Not via &error_abort" leads me to another point. The &error_abort are
> the assertions you can see in the patch. The ones you can't see are in
> the visitor core and the JSON output visitor. They're all about misuse
> of the interface.
>
> The old code is different: it doesn't detect misuse, and produces
> invalid JSON instead. "Never check for an error you don't know how to
> handle."
>
> With the new code, misuse should be caught in general migration testing,
> "make check" if it's any good.
>
> With the old code, it could more easily escape testing, because you have
> to parse the resulting JSON to detect it.
And what happens to the users VM if that JSON is invalid? *nothing*
The user doesn't see any problem at all; no corruption, no crash, nothing.
That's what I like users to see.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 9:22 ` Dr. David Alan Gilbert
@ 2016-05-04 11:37 ` Markus Armbruster
2016-05-04 11:56 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 11:37 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
>> "git-grep assert migration" suggests you do kill the source on certain
>> programming errors.
>
> I'm just trying hard to reduce them; I know I'm not there, but I'd rather
> we didn't have any - especially on the source side.
>
>> I reiterate my point that fancy, untestable error recovery is unlikely
>> to actually recover. "Fancy" can work, "untestable" might work (but
>> color me skeptic), but once you got both, you're a dead man walking.
>
> Then we should make the error recovery paths easy; at the moment visitor
> error paths are just too painful.
I've never seen error handling in C that wasn't painful and still
correct. Surprise me!
>> >> Complete list of conditions where the JSON output visitor sets an error:
>> >>
>> >> * Conditions where the visitor core sets an error:
>> >>
>> >> - visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
>> >> a value out of bounds. This is a serious programming error in
>> >> qapi-visit-core.c. We're almost certainly screwed, and attempting
>> >> to continue is unsafe.
>> >>
>> >> - visit_type_int(): likewise.
>> >>
>> >> - output_type_enum() when the numeric value is out of bounds. This is
>> >> either a serious programming error in qapi-visit-core.c, or
>> >> corrupted state. Either way, we're almost certainly screwed, and
>> >> attempting to continue is unsafe.
>> >>
>> >> - input_type_enum() when the string value is unknown. This is either
>> >> a serious programming error in qapi-visit-core.c, or bad input.
>> >> However, the JSON output visitor isn't supposed to ever call
>> >> input_type_enum(), so it's the former. Once again, we're almost
>> >> certainly screwed, and attempting to continue is unsafe.
>> >>
>> >> * Conditions where the JSON output visitor itself sets an error:
>> >>
>> >> - None.
>> >>
>> >> Do you still object to &error_abort?
>> >
>> > So at the very least it should be commented as to why it can't happen.
>> > My worry about it is that you've got a fairly long comment about why
>> > it can't happen, and I worry that in 6 months someone adds a feature
>> > to either the visitors or the migration code that means there's now
>> > a case where it can happen.
>>
>> Here's why I don't think new failure modes are likely.
>>
>> What does this helper module do, and how could it possibly fail? By
>> "possibly", I mean any conceivable reasonable implementation, not just
>> the two we have (this patch gets rid of one).
>>
>> This helper module builds JSON text and returns it as a string. Its
>> interface mirrors JSON abstract syntax: start object, end object, start
>> array, end array, string, ... Additionally, initialize, finalize, get
>> the result as a string.
>>
>> Conceivable failure modes:
>>
>> * Out of memory. We die, like we generally do for smallish allocations.
>>
>> * Data not representable in JSON. This is basically non-finite numbers,
>> and we already chose to extend JSON instead of making this an error.
>> Such a decision will not be revised without a thorough analysis of
>> impact on existing users.
>>
>> * Interface misused, e.g. invalid nesting. Clearly a programming error.
>> We can either silently produce garbage output, fail, or die. Before
>> the patch: garbage output. After the patch: die by assertion failure
>> (*not* via &error_abort).
>>
>> * Anything else?
>>
>> "Not via &error_abort" leads me to another point. The &error_abort are
>> the assertions you can see in the patch. The ones you can't see are in
>> the visitor core and the JSON output visitor. They're all about misuse
>> of the interface.
>>
>> The old code is different: it doesn't detect misuse, and produces
>> invalid JSON instead. "Never check for an error you don't know how to
>> handle."
>>
>> With the new code, misuse should be caught in general migration testing,
>> "make check" if it's any good.
>>
>> With the old code, it could more easily escape testing, because you have
>> to parse the resulting JSON to detect it.
>
> And what happens to the users VM if that JSON is invalid? *nothing*
> The user doesn't see any problem at all; no corruption, no crash, nothing.
> That's what I like users to see.
This assumes that the root cause of the assertion failure has no further
ill effects. I call that assumption bold. But to each his own.
I figure we're unlikely to reach consensus on this, so I'd like to
propose we agree to disagree, and do the following:
* We shelve the de-duplication of JSON formatting (this patch)
indefinitely.
* We move qjson.c to migration/, next to its only user, and add a
comment explaining why it migration doesn't want to use general
infrastructure here (JSON output visitor), but needs its own thing.
This gets the file covered in MAINTAINERS, and will help prevent it
growing additional users.
Deal?
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 11:37 ` Markus Armbruster
@ 2016-05-04 11:56 ` Dr. David Alan Gilbert
2016-05-04 13:00 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-04 11:56 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >
> >> "git-grep assert migration" suggests you do kill the source on certain
> >> programming errors.
> >
> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
> > we didn't have any - especially on the source side.
> >
> >> I reiterate my point that fancy, untestable error recovery is unlikely
> >> to actually recover. "Fancy" can work, "untestable" might work (but
> >> color me skeptic), but once you got both, you're a dead man walking.
> >
> > Then we should make the error recovery paths easy; at the moment visitor
> > error paths are just too painful.
>
> I've never seen error handling in C that wasn't painful and still
> correct. Surprise me!
The thing that makes it hard for the visitor code is the need to check
it after every call and the check is complicated.
> >> >> Complete list of conditions where the JSON output visitor sets an error:
> >> >>
> >> >> * Conditions where the visitor core sets an error:
> >> >>
> >> >> - visit_type_uintN() when one of the visit_type_uint{8,16,32}() passes
> >> >> a value out of bounds. This is a serious programming error in
> >> >> qapi-visit-core.c. We're almost certainly screwed, and attempting
> >> >> to continue is unsafe.
> >> >>
> >> >> - visit_type_int(): likewise.
> >> >>
> >> >> - output_type_enum() when the numeric value is out of bounds. This is
> >> >> either a serious programming error in qapi-visit-core.c, or
> >> >> corrupted state. Either way, we're almost certainly screwed, and
> >> >> attempting to continue is unsafe.
> >> >>
> >> >> - input_type_enum() when the string value is unknown. This is either
> >> >> a serious programming error in qapi-visit-core.c, or bad input.
> >> >> However, the JSON output visitor isn't supposed to ever call
> >> >> input_type_enum(), so it's the former. Once again, we're almost
> >> >> certainly screwed, and attempting to continue is unsafe.
> >> >>
> >> >> * Conditions where the JSON output visitor itself sets an error:
> >> >>
> >> >> - None.
> >> >>
> >> >> Do you still object to &error_abort?
> >> >
> >> > So at the very least it should be commented as to why it can't happen.
> >> > My worry about it is that you've got a fairly long comment about why
> >> > it can't happen, and I worry that in 6 months someone adds a feature
> >> > to either the visitors or the migration code that means there's now
> >> > a case where it can happen.
> >>
> >> Here's why I don't think new failure modes are likely.
> >>
> >> What does this helper module do, and how could it possibly fail? By
> >> "possibly", I mean any conceivable reasonable implementation, not just
> >> the two we have (this patch gets rid of one).
> >>
> >> This helper module builds JSON text and returns it as a string. Its
> >> interface mirrors JSON abstract syntax: start object, end object, start
> >> array, end array, string, ... Additionally, initialize, finalize, get
> >> the result as a string.
> >>
> >> Conceivable failure modes:
> >>
> >> * Out of memory. We die, like we generally do for smallish allocations.
> >>
> >> * Data not representable in JSON. This is basically non-finite numbers,
> >> and we already chose to extend JSON instead of making this an error.
> >> Such a decision will not be revised without a thorough analysis of
> >> impact on existing users.
> >>
> >> * Interface misused, e.g. invalid nesting. Clearly a programming error.
> >> We can either silently produce garbage output, fail, or die. Before
> >> the patch: garbage output. After the patch: die by assertion failure
> >> (*not* via &error_abort).
> >>
> >> * Anything else?
> >>
> >> "Not via &error_abort" leads me to another point. The &error_abort are
> >> the assertions you can see in the patch. The ones you can't see are in
> >> the visitor core and the JSON output visitor. They're all about misuse
> >> of the interface.
> >>
> >> The old code is different: it doesn't detect misuse, and produces
> >> invalid JSON instead. "Never check for an error you don't know how to
> >> handle."
> >>
> >> With the new code, misuse should be caught in general migration testing,
> >> "make check" if it's any good.
> >>
> >> With the old code, it could more easily escape testing, because you have
> >> to parse the resulting JSON to detect it.
> >
> > And what happens to the users VM if that JSON is invalid? *nothing*
> > The user doesn't see any problem at all; no corruption, no crash, nothing.
> > That's what I like users to see.
>
> This assumes that the root cause of the assertion failure has no further
> ill effects. I call that assumption bold. But to each his own.
The whole JSON use in migration is just for debug/parsing in external tools -
even if it's complete rubbish it doesn't affect the VM, which is why I don't
want an error producing it to kill the VM.
> I figure we're unlikely to reach consensus on this, so I'd like to
> propose we agree to disagree, and do the following:
>
> * We shelve the de-duplication of JSON formatting (this patch)
> indefinitely.
>
> * We move qjson.c to migration/, next to its only user, and add a
> comment explaining why it migration doesn't want to use general
> infrastructure here (JSON output visitor), but needs its own thing.
> This gets the file covered in MAINTAINERS, and will help prevent it
> growing additional users.
>
> Deal?
No, sorry; the JSON use in the migration is just a debug thing;
we don't want to maintain a separate JSON instance for it.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 11:56 ` Dr. David Alan Gilbert
@ 2016-05-04 13:00 ` Markus Armbruster
2016-05-04 13:19 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 13:00 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>>
>> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >
>> >> "git-grep assert migration" suggests you do kill the source on certain
>> >> programming errors.
>> >
>> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
>> > we didn't have any - especially on the source side.
>> >
>> >> I reiterate my point that fancy, untestable error recovery is unlikely
>> >> to actually recover. "Fancy" can work, "untestable" might work (but
>> >> color me skeptic), but once you got both, you're a dead man walking.
>> >
>> > Then we should make the error recovery paths easy; at the moment visitor
>> > error paths are just too painful.
>>
>> I've never seen error handling in C that wasn't painful and still
>> correct. Surprise me!
>
> The thing that makes it hard for the visitor code is the need to check
> it after every call and the check is complicated.
Having to check every call is certainly painful, but there's no general
and safe way around it. Accumulating errors that need to be checked
only at the end of a job can be less painful, but then the job's code
needs to be very carefully written to be safe even in presence of
errors. Most code isn't, and some code can't.
The check for failure is simple, but annoyingly verbose when the
function's return value is useless:
Error *err = NULL;
foo(..., &err);
if (err) {
...
}
I'm playing with a update to conventions and usage to permit
if (!foo(..., &err)) {
...
}
Just as simple, but more readable.
[...]
>> I figure we're unlikely to reach consensus on this, so I'd like to
>> propose we agree to disagree, and do the following:
>>
>> * We shelve the de-duplication of JSON formatting (this patch)
>> indefinitely.
>>
>> * We move qjson.c to migration/, next to its only user, and add a
>> comment explaining why it migration doesn't want to use general
>> infrastructure here (JSON output visitor), but needs its own thing.
>> This gets the file covered in MAINTAINERS, and will help prevent it
>> growing additional users.
>>
>> Deal?
>
> No, sorry; the JSON use in the migration is just a debug thing;
> we don't want to maintain a separate JSON instance for it.
Well, you already do, except in name. Who else do you think is
maintaining qjson.[ch], created by migration people, for migration's
use? Certainly not me.
If you can't use the general JSON output code I maintain because of
special requirements, you get to continue maintaining your own. All 109
SLOC of it. All I'm asking is to make it official, and to deter
accidental use of migration's JSON writer instead of the general one.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 13:00 ` Markus Armbruster
@ 2016-05-04 13:19 ` Dr. David Alan Gilbert
2016-05-04 14:10 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-04 13:19 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >>
> >> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >> >
> >> >> "git-grep assert migration" suggests you do kill the source on certain
> >> >> programming errors.
> >> >
> >> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
> >> > we didn't have any - especially on the source side.
> >> >
> >> >> I reiterate my point that fancy, untestable error recovery is unlikely
> >> >> to actually recover. "Fancy" can work, "untestable" might work (but
> >> >> color me skeptic), but once you got both, you're a dead man walking.
> >> >
> >> > Then we should make the error recovery paths easy; at the moment visitor
> >> > error paths are just too painful.
> >>
> >> I've never seen error handling in C that wasn't painful and still
> >> correct. Surprise me!
> >
> > The thing that makes it hard for the visitor code is the need to check
> > it after every call and the check is complicated.
>
> Having to check every call is certainly painful, but there's no general
> and safe way around it. Accumulating errors that need to be checked
> only at the end of a job can be less painful, but then the job's code
> needs to be very carefully written to be safe even in presence of
> errors. Most code isn't, and some code can't.
Yes; output visitors would seem to be the easiest case though?
> The check for failure is simple, but annoyingly verbose when the
> function's return value is useless:
>
> Error *err = NULL;
> foo(..., &err);
> if (err) {
> ...
> }
>
> I'm playing with a update to conventions and usage to permit
>
> if (!foo(..., &err)) {
> ...
> }
If that became;
if (!foo(..., &err) ||
!foo(..., &err) ||
!foo(..., &err)) {
...
}
That would be both readable and not verbose.
> Just as simple, but more readable.
>
> [...]
> >> I figure we're unlikely to reach consensus on this, so I'd like to
> >> propose we agree to disagree, and do the following:
> >>
> >> * We shelve the de-duplication of JSON formatting (this patch)
> >> indefinitely.
> >>
> >> * We move qjson.c to migration/, next to its only user, and add a
> >> comment explaining why it migration doesn't want to use general
> >> infrastructure here (JSON output visitor), but needs its own thing.
> >> This gets the file covered in MAINTAINERS, and will help prevent it
> >> growing additional users.
> >>
> >> Deal?
> >
> > No, sorry; the JSON use in the migration is just a debug thing;
> > we don't want to maintain a separate JSON instance for it.
>
> Well, you already do, except in name. Who else do you think is
> maintaining qjson.[ch], created by migration people, for migration's
> use? Certainly not me.
That came from migration? Really? I didn't think we used JSON at
all until last year.
> If you can't use the general JSON output code I maintain because of
> special requirements, you get to continue maintaining your own. All 109
> SLOC of it. All I'm asking is to make it official, and to deter
> accidental use of migration's JSON writer instead of the general one.
Yeh; I'd love to share the JSON code; just lets try and avoid anything that
can kill the source, however broken the migration.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 13:19 ` Dr. David Alan Gilbert
@ 2016-05-04 14:10 ` Markus Armbruster
2016-05-04 14:53 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 14:10 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>>
>> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >>
>> >> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >> >
>> >> >> "git-grep assert migration" suggests you do kill the source on certain
>> >> >> programming errors.
>> >> >
>> >> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
>> >> > we didn't have any - especially on the source side.
>> >> >
>> >> >> I reiterate my point that fancy, untestable error recovery is unlikely
>> >> >> to actually recover. "Fancy" can work, "untestable" might work (but
>> >> >> color me skeptic), but once you got both, you're a dead man walking.
>> >> >
>> >> > Then we should make the error recovery paths easy; at the moment visitor
>> >> > error paths are just too painful.
>> >>
>> >> I've never seen error handling in C that wasn't painful and still
>> >> correct. Surprise me!
>> >
>> > The thing that makes it hard for the visitor code is the need to check
>> > it after every call and the check is complicated.
>>
>> Having to check every call is certainly painful, but there's no general
>> and safe way around it. Accumulating errors that need to be checked
>> only at the end of a job can be less painful, but then the job's code
>> needs to be very carefully written to be safe even in presence of
>> errors. Most code isn't, and some code can't.
>
> Yes; output visitors would seem to be the easiest case though?
Here's the example from visitor.h at the end of this series (with a
small mistake corrected):
Visitor *v;
Error *err = NULL;
int value;
v = ...obtain visitor...
visit_start_struct(v, NULL, NULL, 0, &err);
if (err) {
goto out;
}
visit_start_list(v, "list", NULL, 0, &err);
if (err) {
goto outobj;
}
value = 1;
visit_type_int(v, NULL, &value, &err);
if (err) {
goto outlist;
}
value = 2;
visit_type_int(v, NULL, &value, &err);
if (err) {
goto outlist;
}
outlist:
visit_end_list(v, NULL);
if (!err) {
visit_check_struct(v, &err);
}
outobj:
visit_end_struct(v, NULL);
out:
error_propagate(errp, err);
...clean up v...
With accumulating Errors, we could elide some but not all error checks.
In particular, the ones after visit_start_FOO() are still required,
because visit_end_FOO() may only be called after visit_start_FOO()
succeeded.
If we did anything interesting in addition to calling visitors, we'd
have to additionally consider whether doing it is safe after errors.
Accumulating errors *can* make the code easier on the eyes, but they
also make it easy to screw up behavior after error.
>> The check for failure is simple, but annoyingly verbose when the
>> function's return value is useless:
>>
>> Error *err = NULL;
>> foo(..., &err);
>> if (err) {
>> ...
>> }
>>
>> I'm playing with a update to conventions and usage to permit
>>
>> if (!foo(..., &err)) {
>> ...
>> }
>
> If that became;
> if (!foo(..., &err) ||
> !foo(..., &err) ||
> !foo(..., &err)) {
> ...
> }
>
> That would be both readable and not verbose.
Yes, that could be done then.
>> Just as simple, but more readable.
>>
>> [...]
>> >> I figure we're unlikely to reach consensus on this, so I'd like to
>> >> propose we agree to disagree, and do the following:
>> >>
>> >> * We shelve the de-duplication of JSON formatting (this patch)
>> >> indefinitely.
>> >>
>> >> * We move qjson.c to migration/, next to its only user, and add a
>> >> comment explaining why it migration doesn't want to use general
>> >> infrastructure here (JSON output visitor), but needs its own thing.
>> >> This gets the file covered in MAINTAINERS, and will help prevent it
>> >> growing additional users.
>> >>
>> >> Deal?
>> >
>> > No, sorry; the JSON use in the migration is just a debug thing;
>> > we don't want to maintain a separate JSON instance for it.
>>
>> Well, you already do, except in name. Who else do you think is
>> maintaining qjson.[ch], created by migration people, for migration's
>> use? Certainly not me.
>
> That came from migration? Really? I didn't think we used JSON at
> all until last year.
Commit 0457d07..b174257.
Migration is still the only user of this special JSON writer, and if you
ask me, it better remain the only one.
>> If you can't use the general JSON output code I maintain because of
>> special requirements, you get to continue maintaining your own. All 109
>> SLOC of it. All I'm asking is to make it official, and to deter
>> accidental use of migration's JSON writer instead of the general one.
>
> Yeh; I'd love to share the JSON code; just lets try and avoid anything that
> can kill the source, however broken the migration.
Visitors will abort when their preconditions or invariants are violated.
If that's not okay for migration, I'm afraid migration needs to continue
to roll its own JSON writer. Visitors are pretty heavily used nowadays,
and we very much rely on these assertions to catch mistakes.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 14:10 ` Markus Armbruster
@ 2016-05-04 14:53 ` Dr. David Alan Gilbert
2016-05-04 15:17 ` Eric Blake
2016-05-04 15:42 ` Markus Armbruster
0 siblings, 2 replies; 78+ messages in thread
From: Dr. David Alan Gilbert @ 2016-05-04 14:53 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >>
> >> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >> >>
> >> >> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> >> >> >
> >> >> >> "git-grep assert migration" suggests you do kill the source on certain
> >> >> >> programming errors.
> >> >> >
> >> >> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
> >> >> > we didn't have any - especially on the source side.
> >> >> >
> >> >> >> I reiterate my point that fancy, untestable error recovery is unlikely
> >> >> >> to actually recover. "Fancy" can work, "untestable" might work (but
> >> >> >> color me skeptic), but once you got both, you're a dead man walking.
> >> >> >
> >> >> > Then we should make the error recovery paths easy; at the moment visitor
> >> >> > error paths are just too painful.
> >> >>
> >> >> I've never seen error handling in C that wasn't painful and still
> >> >> correct. Surprise me!
> >> >
> >> > The thing that makes it hard for the visitor code is the need to check
> >> > it after every call and the check is complicated.
> >>
> >> Having to check every call is certainly painful, but there's no general
> >> and safe way around it. Accumulating errors that need to be checked
> >> only at the end of a job can be less painful, but then the job's code
> >> needs to be very carefully written to be safe even in presence of
> >> errors. Most code isn't, and some code can't.
> >
> > Yes; output visitors would seem to be the easiest case though?
>
> Here's the example from visitor.h at the end of this series (with a
> small mistake corrected):
>
> Visitor *v;
> Error *err = NULL;
> int value;
>
> v = ...obtain visitor...
> visit_start_struct(v, NULL, NULL, 0, &err);
> if (err) {
> goto out;
> }
> visit_start_list(v, "list", NULL, 0, &err);
> if (err) {
> goto outobj;
> }
> value = 1;
> visit_type_int(v, NULL, &value, &err);
> if (err) {
> goto outlist;
> }
> value = 2;
> visit_type_int(v, NULL, &value, &err);
> if (err) {
> goto outlist;
> }
> outlist:
> visit_end_list(v, NULL);
> if (!err) {
> visit_check_struct(v, &err);
> }
> outobj:
> visit_end_struct(v, NULL);
> out:
> error_propagate(errp, err);
> ...clean up v...
>
> With accumulating Errors, we could elide some but not all error checks.
> In particular, the ones after visit_start_FOO() are still required,
> because visit_end_FOO() may only be called after visit_start_FOO()
> succeeded.
Hmm the visit_end_* are interesting; I guess we have to be careful
of those, unless that is you could make the visit_end_struct(v, NULL)
to fail nicely in that case.
> If we did anything interesting in addition to calling visitors, we'd
> have to additionally consider whether doing it is safe after errors.
>
> Accumulating errors *can* make the code easier on the eyes, but they
> also make it easy to screw up behavior after error.
>
> >> The check for failure is simple, but annoyingly verbose when the
> >> function's return value is useless:
> >>
> >> Error *err = NULL;
> >> foo(..., &err);
> >> if (err) {
> >> ...
> >> }
> >>
> >> I'm playing with a update to conventions and usage to permit
> >>
> >> if (!foo(..., &err)) {
> >> ...
> >> }
> >
> > If that became;
> > if (!foo(..., &err) ||
> > !foo(..., &err) ||
> > !foo(..., &err)) {
> > ...
> > }
> >
> > That would be both readable and not verbose.
>
> Yes, that could be done then.
How would we deal with all the visit_end_* - if we've decided
there's an error are we required to call all the end's before we
just free the visitor or something like that?
> >> Just as simple, but more readable.
> >>
> >> [...]
> >> >> I figure we're unlikely to reach consensus on this, so I'd like to
> >> >> propose we agree to disagree, and do the following:
> >> >>
> >> >> * We shelve the de-duplication of JSON formatting (this patch)
> >> >> indefinitely.
> >> >>
> >> >> * We move qjson.c to migration/, next to its only user, and add a
> >> >> comment explaining why it migration doesn't want to use general
> >> >> infrastructure here (JSON output visitor), but needs its own thing.
> >> >> This gets the file covered in MAINTAINERS, and will help prevent it
> >> >> growing additional users.
> >> >>
> >> >> Deal?
> >> >
> >> > No, sorry; the JSON use in the migration is just a debug thing;
> >> > we don't want to maintain a separate JSON instance for it.
> >>
> >> Well, you already do, except in name. Who else do you think is
> >> maintaining qjson.[ch], created by migration people, for migration's
> >> use? Certainly not me.
> >
> > That came from migration? Really? I didn't think we used JSON at
> > all until last year.
>
> Commit 0457d07..b174257.
>
> Migration is still the only user of this special JSON writer, and if you
> ask me, it better remain the only one.
>
> >> If you can't use the general JSON output code I maintain because of
> >> special requirements, you get to continue maintaining your own. All 109
> >> SLOC of it. All I'm asking is to make it official, and to deter
> >> accidental use of migration's JSON writer instead of the general one.
> >
> > Yeh; I'd love to share the JSON code; just lets try and avoid anything that
> > can kill the source, however broken the migration.
>
> Visitors will abort when their preconditions or invariants are violated.
> If that's not okay for migration, I'm afraid migration needs to continue
> to roll its own JSON writer. Visitors are pretty heavily used nowadays,
> and we very much rely on these assertions to catch mistakes.
OK, lets keep our own writer; if we can't have more control over visitors
failure paths, we'll have to.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 14:53 ` Dr. David Alan Gilbert
@ 2016-05-04 15:17 ` Eric Blake
2016-05-04 15:42 ` Markus Armbruster
1 sibling, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-05-04 15:17 UTC (permalink / raw)
To: Dr. David Alan Gilbert, Markus Armbruster
Cc: Amit Shah, famz, qemu-devel, Juan Quintela
[-- Attachment #1: Type: text/plain, Size: 2194 bytes --]
On 05/04/2016 08:53 AM, Dr. David Alan Gilbert wrote:
>> Here's the example from visitor.h at the end of this series (with a
>> small mistake corrected):
>>
>> Visitor *v;
>> Error *err = NULL;
>> int value;
>>
>> v = ...obtain visitor...
>> visit_start_struct(v, NULL, NULL, 0, &err);
>> if (err) {
>> goto out;
>> }
>> visit_start_list(v, "list", NULL, 0, &err);
>> if (err) {
>> goto outobj;
>> }
>> value = 1;
>> visit_type_int(v, NULL, &value, &err);
>> if (err) {
>> goto outlist;
>> }
>> value = 2;
>> visit_type_int(v, NULL, &value, &err);
>> if (err) {
>> goto outlist;
>> }
>> outlist:
>> visit_end_list(v, NULL);
>> if (!err) {
>> visit_check_struct(v, &err);
>> }
>> outobj:
>> visit_end_struct(v, NULL);
> Hmm the visit_end_* are interesting; I guess we have to be careful
> of those, unless that is you could make the visit_end_struct(v, NULL)
> to fail nicely in that case.
Here, visit_end_struct() _can't_ fail; you are passing NULL to match the
fact that the earlier visit_start_struct() passed NULL for obj (that is,
at the end of the current qapi series, the parameter changes from
Error** to void **, and is NOT an error indicator).
>>
>> Visitors will abort when their preconditions or invariants are violated.
And it's very easy to avoid violating those preconditions. But if it's
not easy enough for you, then the solution is...
>> If that's not okay for migration, I'm afraid migration needs to continue
>> to roll its own JSON writer. Visitors are pretty heavily used nowadays,
>> and we very much rely on these assertions to catch mistakes.
>
> OK, lets keep our own writer; if we can't have more control over visitors
> failure paths, we'll have to.
...exactly this. So in the next spin of my series, I will drop (both!)
of my attempts to rework qjson.c to reuse the visitors, and instead just
move it into the migration tree and out of the top level.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor
2016-05-04 14:53 ` Dr. David Alan Gilbert
2016-05-04 15:17 ` Eric Blake
@ 2016-05-04 15:42 ` Markus Armbruster
1 sibling, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-04 15:42 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: Amit Shah, famz, qemu-devel, Juan Quintela
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>>
>> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >>
>> >> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >> >>
>> >> >> > * Markus Armbruster (armbru@redhat.com) wrote:
>> >> >> >> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>> >> >> >
>> >> >> >> "git-grep assert migration" suggests you do kill the source on certain
>> >> >> >> programming errors.
>> >> >> >
>> >> >> > I'm just trying hard to reduce them; I know I'm not there, but I'd rather
>> >> >> > we didn't have any - especially on the source side.
>> >> >> >
>> >> >> >> I reiterate my point that fancy, untestable error recovery is unlikely
>> >> >> >> to actually recover. "Fancy" can work, "untestable" might work (but
>> >> >> >> color me skeptic), but once you got both, you're a dead man walking.
>> >> >> >
>> >> >> > Then we should make the error recovery paths easy; at the moment visitor
>> >> >> > error paths are just too painful.
>> >> >>
>> >> >> I've never seen error handling in C that wasn't painful and still
>> >> >> correct. Surprise me!
>> >> >
>> >> > The thing that makes it hard for the visitor code is the need to check
>> >> > it after every call and the check is complicated.
>> >>
>> >> Having to check every call is certainly painful, but there's no general
>> >> and safe way around it. Accumulating errors that need to be checked
>> >> only at the end of a job can be less painful, but then the job's code
>> >> needs to be very carefully written to be safe even in presence of
>> >> errors. Most code isn't, and some code can't.
>> >
>> > Yes; output visitors would seem to be the easiest case though?
>>
>> Here's the example from visitor.h at the end of this series (with a
>> small mistake corrected):
>>
>> Visitor *v;
>> Error *err = NULL;
>> int value;
>>
>> v = ...obtain visitor...
>> visit_start_struct(v, NULL, NULL, 0, &err);
>> if (err) {
>> goto out;
>> }
>> visit_start_list(v, "list", NULL, 0, &err);
>> if (err) {
>> goto outobj;
>> }
>> value = 1;
>> visit_type_int(v, NULL, &value, &err);
>> if (err) {
>> goto outlist;
>> }
>> value = 2;
>> visit_type_int(v, NULL, &value, &err);
>> if (err) {
>> goto outlist;
>> }
>> outlist:
>> visit_end_list(v, NULL);
>> if (!err) {
>> visit_check_struct(v, &err);
>> }
>> outobj:
>> visit_end_struct(v, NULL);
>> out:
>> error_propagate(errp, err);
>> ...clean up v...
>>
>> With accumulating Errors, we could elide some but not all error checks.
>> In particular, the ones after visit_start_FOO() are still required,
>> because visit_end_FOO() may only be called after visit_start_FOO()
>> succeeded.
>
> Hmm the visit_end_* are interesting; I guess we have to be careful
> of those, unless that is you could make the visit_end_struct(v, NULL)
> to fail nicely in that case.
Unfortunately, making functions fail "nicely" on programming errors
creates a new burden for users that do not want to recover from
programming errors (because it's *unsafe*): now they have to somehow
find out whether the error they got is a programming error. That's not
even possible right now, but we could add a "this is a programming
error" predicate to the error type. Anyway, the error boilerplate would
then explode, and the error handling bugs would multiply.
You can't make the same API serve radically different requirements
equally well. "Fail as cleanly as possible even on violations of
preconditions and invariants, and do your best to survive regardless of
how I handle them" is radically different from "rely on preconditions
and invariants, but do check them to catch bugs". An interface doing
the former is bound to have weaker postconditions. It's also harder to
implement, and harder to use.
>> If we did anything interesting in addition to calling visitors, we'd
>> have to additionally consider whether doing it is safe after errors.
>>
>> Accumulating errors *can* make the code easier on the eyes, but they
>> also make it easy to screw up behavior after error.
>>
>> >> The check for failure is simple, but annoyingly verbose when the
>> >> function's return value is useless:
>> >>
>> >> Error *err = NULL;
>> >> foo(..., &err);
>> >> if (err) {
>> >> ...
>> >> }
>> >>
>> >> I'm playing with a update to conventions and usage to permit
>> >>
>> >> if (!foo(..., &err)) {
>> >> ...
>> >> }
>> >
>> > If that became;
>> > if (!foo(..., &err) ||
>> > !foo(..., &err) ||
>> > !foo(..., &err)) {
>> > ...
>> > }
>> >
>> > That would be both readable and not verbose.
>>
>> Yes, that could be done then.
>
> How would we deal with all the visit_end_* - if we've decided
> there's an error are we required to call all the end's before we
> just free the visitor or something like that?
After this series, the visitor contract guarantees (in writing) that you
can safely destroy or reset the visitor at any time.
Existing users call the matching visit_end_FOO() instead, because that's
easy enough for them. But if you find yourself in a situation where you
don't want to, go ahead and destroy the visitor without further ado.
>> >> Just as simple, but more readable.
>> >>
>> >> [...]
>> >> >> I figure we're unlikely to reach consensus on this, so I'd like to
>> >> >> propose we agree to disagree, and do the following:
>> >> >>
>> >> >> * We shelve the de-duplication of JSON formatting (this patch)
>> >> >> indefinitely.
>> >> >>
>> >> >> * We move qjson.c to migration/, next to its only user, and add a
>> >> >> comment explaining why it migration doesn't want to use general
>> >> >> infrastructure here (JSON output visitor), but needs its own thing.
>> >> >> This gets the file covered in MAINTAINERS, and will help prevent it
>> >> >> growing additional users.
>> >> >>
>> >> >> Deal?
>> >> >
>> >> > No, sorry; the JSON use in the migration is just a debug thing;
>> >> > we don't want to maintain a separate JSON instance for it.
>> >>
>> >> Well, you already do, except in name. Who else do you think is
>> >> maintaining qjson.[ch], created by migration people, for migration's
>> >> use? Certainly not me.
>> >
>> > That came from migration? Really? I didn't think we used JSON at
>> > all until last year.
>>
>> Commit 0457d07..b174257.
>>
>> Migration is still the only user of this special JSON writer, and if you
>> ask me, it better remain the only one.
>>
>> >> If you can't use the general JSON output code I maintain because of
>> >> special requirements, you get to continue maintaining your own. All 109
>> >> SLOC of it. All I'm asking is to make it official, and to deter
>> >> accidental use of migration's JSON writer instead of the general one.
>> >
>> > Yeh; I'd love to share the JSON code; just lets try and avoid anything that
>> > can kill the source, however broken the migration.
>>
>> Visitors will abort when their preconditions or invariants are violated.
>> If that's not okay for migration, I'm afraid migration needs to continue
>> to roll its own JSON writer. Visitors are pretty heavily used nowadays,
>> and we very much rely on these assertions to catch mistakes.
>
> OK, lets keep our own writer; if we can't have more control over visitors
> failure paths, we'll have to.
All right, I'll cook up a patch.
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 11/18] qjson: Remove unused file
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (9 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 10/18] vmstate: Use new JSON output visitor Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix() Eric Blake
` (7 subsequent siblings)
18 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz
Now that we have a JSON output visitor, and the previous patch
fixed the only client of vmstate to use it, we no longer need the
simpler QJSON object doing the same thing.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: rebase to master
v2: new patch
---
include/qjson.h | 29 -------------
qjson.c | 126 --------------------------------------------------------
Makefile.objs | 1 -
3 files changed, 156 deletions(-)
delete mode 100644 include/qjson.h
delete mode 100644 qjson.c
diff --git a/include/qjson.h b/include/qjson.h
deleted file mode 100644
index 7c54fdf..0000000
--- a/include/qjson.h
+++ /dev/null
@@ -1,29 +0,0 @@
-/*
- * QEMU JSON writer
- *
- * Copyright Alexander Graf
- *
- * Authors:
- * Alexander Graf <agraf@suse.de>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-#ifndef QEMU_QJSON_H
-#define QEMU_QJSON_H
-
-#define TYPE_QJSON "QJSON"
-typedef struct QJSON QJSON;
-
-QJSON *qjson_new(void);
-void json_prop_str(QJSON *json, const char *name, const char *str);
-void json_prop_int(QJSON *json, const char *name, int64_t val);
-void json_end_array(QJSON *json);
-void json_start_array(QJSON *json, const char *name);
-void json_end_object(QJSON *json);
-void json_start_object(QJSON *json, const char *name);
-const char *qjson_get_str(QJSON *json);
-void qjson_finish(QJSON *json);
-
-#endif /* QEMU_QJSON_H */
diff --git a/qjson.c b/qjson.c
deleted file mode 100644
index d172b1f..0000000
--- a/qjson.c
+++ /dev/null
@@ -1,126 +0,0 @@
-/*
- * QEMU JSON writer
- *
- * Copyright Alexander Graf
- *
- * Authors:
- * Alexander Graf <agraf@suse.de>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#include "qemu/osdep.h"
-#include <qapi/qmp/qstring.h>
-#include <glib.h>
-#include <qjson.h>
-#include <qemu/module.h>
-#include <qom/object.h>
-
-struct QJSON {
- Object obj;
- QString *str;
- bool omit_comma;
-};
-
-#define QJSON(obj) OBJECT_CHECK(QJSON, (obj), TYPE_QJSON)
-
-static void json_emit_element(QJSON *json, const char *name)
-{
- /* Check whether we need to print a , before an element */
- if (json->omit_comma) {
- json->omit_comma = false;
- } else {
- qstring_append(json->str, ", ");
- }
-
- if (name) {
- qstring_append_json_string(json->str, name);
- qstring_append(json->str, " : ");
- }
-}
-
-void json_start_object(QJSON *json, const char *name)
-{
- json_emit_element(json, name);
- qstring_append(json->str, "{ ");
- json->omit_comma = true;
-}
-
-void json_end_object(QJSON *json)
-{
- qstring_append(json->str, " }");
- json->omit_comma = false;
-}
-
-void json_start_array(QJSON *json, const char *name)
-{
- json_emit_element(json, name);
- qstring_append(json->str, "[ ");
- json->omit_comma = true;
-}
-
-void json_end_array(QJSON *json)
-{
- qstring_append(json->str, " ]");
- json->omit_comma = false;
-}
-
-void json_prop_int(QJSON *json, const char *name, int64_t val)
-{
- json_emit_element(json, name);
- qstring_append_format(json->str, "%" PRId64, val);
-}
-
-void json_prop_str(QJSON *json, const char *name, const char *str)
-{
- json_emit_element(json, name);
- qstring_append_json_string(json->str, str);
-}
-
-const char *qjson_get_str(QJSON *json)
-{
- return qstring_get_str(json->str);
-}
-
-QJSON *qjson_new(void)
-{
- QJSON *json = QJSON(object_new(TYPE_QJSON));
- return json;
-}
-
-void qjson_finish(QJSON *json)
-{
- json_end_object(json);
-}
-
-static void qjson_initfn(Object *obj)
-{
- QJSON *json = QJSON(obj);
-
- json->str = qstring_from_str("{ ");
- json->omit_comma = true;
-}
-
-static void qjson_finalizefn(Object *obj)
-{
- QJSON *json = QJSON(obj);
-
- qobject_decref(QOBJECT(json->str));
-}
-
-static const TypeInfo qjson_type_info = {
- .name = TYPE_QJSON,
- .parent = TYPE_OBJECT,
- .instance_size = sizeof(QJSON),
- .instance_init = qjson_initfn,
- .instance_finalize = qjson_finalizefn,
-};
-
-static void qjson_register_types(void)
-{
- type_register_static(&qjson_type_info);
-}
-
-type_init(qjson_register_types)
diff --git a/Makefile.objs b/Makefile.objs
index 8f705f6..da49b71 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -52,7 +52,6 @@ common-obj-$(CONFIG_LINUX) += fsdev/
common-obj-y += migration/
common-obj-y += qemu-char.o #aio.o
common-obj-y += page_cache.o
-common-obj-y += qjson.o
common-obj-$(CONFIG_SPICE) += spice-qemu-char.o
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix()
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (10 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 11/18] qjson: Remove unused file Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 13:56 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 13/18] qapi: Support pretty printing in JSON output visitor Eric Blake
` (6 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Luiz Capitulino
The next patch will add pretty indentation to the JSON visitor.
But in order to support pretty output in the type_any() callback,
we need to prefix every line of the QObject visitor by the current
indentation in the JSON visitor. Hence, a new function
qobject_to_json_pretty_indent(), and the old function becomes a
thin wrapper to the expanded behavior.
While at it, change 'pretty' to be a bool, to match its usage.
Note that the new simple_pretty() test is a bit sensitive to our
current notion of prettiness, as well as to the hash ordering in
QDict (most of the tests in check-qobject-json intentionally do
not compare the original string to the round-trip string, because
we liberally accept more input forms than the canonical form that
we output).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: no change
v2: no change
---
include/qapi/qmp/qobject-json.h | 1 +
qobject/qobject-json.c | 40 ++++++++++++++++++-------
tests/check-qobject-json.c | 65 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 95 insertions(+), 11 deletions(-)
diff --git a/include/qapi/qmp/qobject-json.h b/include/qapi/qmp/qobject-json.h
index 02b1f2c..d420c71 100644
--- a/include/qapi/qmp/qobject-json.h
+++ b/include/qapi/qmp/qobject-json.h
@@ -23,5 +23,6 @@ QObject *qobject_from_jsonv(const char *string, va_list *ap) GCC_FMT_ATTR(1, 0);
QString *qobject_to_json(const QObject *obj);
QString *qobject_to_json_pretty(const QObject *obj);
+QString *qobject_to_json_pretty_prefix(const QObject *obj, const char *prefix);
#endif /* QJSON_H */
diff --git a/qobject/qobject-json.c b/qobject/qobject-json.c
index 963de07..4469835 100644
--- a/qobject/qobject-json.c
+++ b/qobject/qobject-json.c
@@ -70,12 +70,14 @@ QObject *qobject_from_jsonf(const char *string, ...)
typedef struct ToJsonIterState
{
int indent;
- int pretty;
+ bool pretty;
int count;
QString *str;
+ const char *prefix;
} ToJsonIterState;
-static void to_json(const QObject *obj, QString *str, int pretty, int indent);
+static void to_json(const QObject *obj, QString *str, bool pretty, int indent,
+ const char *prefix);
static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
{
@@ -86,13 +88,13 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque)
}
if (s->pretty) {
- qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
+ qstring_append_format(s->str, "\n%s%*s", s->prefix, 4 * s->indent, "");
}
qstring_append_json_string(s->str, key);
qstring_append(s->str, ": ");
- to_json(obj, s->str, s->pretty, s->indent);
+ to_json(obj, s->str, s->pretty, s->indent, s->prefix);
s->count++;
}
@@ -105,14 +107,15 @@ static void to_json_list_iter(QObject *obj, void *opaque)
}
if (s->pretty) {
- qstring_append_format(s->str, "\n%*s", 4 * s->indent, "");
+ qstring_append_format(s->str, "\n%s%*s", s->prefix, 4 * s->indent, "");
}
- to_json(obj, s->str, s->pretty, s->indent);
+ to_json(obj, s->str, s->pretty, s->indent, s->prefix);
s->count++;
}
-static void to_json(const QObject *obj, QString *str, int pretty, int indent)
+static void to_json(const QObject *obj, QString *str, bool pretty, int indent,
+ const char *prefix)
{
switch (qobject_type(obj)) {
case QTYPE_QNULL:
@@ -136,10 +139,11 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
s.str = str;
s.indent = indent + 1;
s.pretty = pretty;
+ s.prefix = prefix;
qstring_append(str, "{");
qdict_iter(val, to_json_dict_iter, &s);
if (pretty) {
- qstring_append_format(str, "\n%*s", 4 * indent, "");
+ qstring_append_format(str, "\n%s%*s", prefix, 4 * indent, "");
}
qstring_append(str, "}");
break;
@@ -152,10 +156,11 @@ static void to_json(const QObject *obj, QString *str, int pretty, int indent)
s.str = str;
s.indent = indent + 1;
s.pretty = pretty;
+ s.prefix = prefix;
qstring_append(str, "[");
qlist_iter(val, (void *)to_json_list_iter, &s);
if (pretty) {
- qstring_append_format(str, "\n%*s", 4 * indent, "");
+ qstring_append_format(str, "\n%s%*s", prefix, 4 * indent, "");
}
qstring_append(str, "]");
break;
@@ -186,7 +191,7 @@ QString *qobject_to_json(const QObject *obj)
{
QString *str = qstring_new();
- to_json(obj, str, 0, 0);
+ to_json(obj, str, false, 0, "");
return str;
}
@@ -195,7 +200,20 @@ QString *qobject_to_json_pretty(const QObject *obj)
{
QString *str = qstring_new();
- to_json(obj, str, 1, 0);
+ to_json(obj, str, true, 0, "");
+
+ return str;
+}
+
+/*
+ * Pretty-print JSON representing obj, inserting prefix after every newline.
+ * Note that prefix is NOT applied before the first character.
+ */
+QString *qobject_to_json_pretty_prefix(const QObject *obj, const char *prefix)
+{
+ QString *str = qstring_new();
+
+ to_json(obj, str, true, 0, prefix);
return str;
}
diff --git a/tests/check-qobject-json.c b/tests/check-qobject-json.c
index d889501..96bb017 100644
--- a/tests/check-qobject-json.c
+++ b/tests/check-qobject-json.c
@@ -1388,6 +1388,70 @@ static void simple_whitespace(void)
}
}
+static void simple_pretty(void)
+{
+ int i;
+ struct {
+ const char *encoded;
+ LiteralQObject decoded;
+ } test_cases[] = {
+ {
+ /* The first line is intentionally not prefixed. */
+ .encoded =
+ "[\n"
+ " 43,\n"
+ " 42\n"
+ " ]",
+ .decoded = QLIT_QLIST(((LiteralQObject[]){
+ QLIT_QINT(43),
+ QLIT_QINT(42),
+ { }
+ })),
+ },
+ {
+ .encoded =
+ "[\n"
+ " 43,\n"
+ " {\n"
+ " \"a\": 32,\n"
+ " \"h\": \"b\"\n"
+ " },\n"
+ " [\n"
+ " ],\n"
+ " 42\n"
+ " ]",
+ .decoded = QLIT_QLIST(((LiteralQObject[]){
+ QLIT_QINT(43),
+ QLIT_QDICT(((LiteralQDictEntry[]){
+ { "a", QLIT_QINT(32) },
+ { "h", QLIT_QSTR("b") },
+ { } })),
+ QLIT_QLIST(((LiteralQObject[]){
+ { } })),
+ QLIT_QINT(42),
+ { }
+ })),
+ },
+ { }
+ };
+
+ for (i = 0; test_cases[i].encoded; i++) {
+ QObject *obj;
+ QString *str;
+
+ obj = qobject_from_json(test_cases[i].encoded);
+ g_assert(obj != NULL);
+ g_assert(qobject_type(obj) == QTYPE_QLIST);
+
+ g_assert(compare_litqobj_to_qobj(&test_cases[i].decoded, obj) == 1);
+
+ str = qobject_to_json_pretty_prefix(obj, " ");
+ qobject_decref(obj);
+ g_assert_cmpstr(qstring_get_str(str), ==, test_cases[i].encoded);
+ QDECREF(str);
+ }
+}
+
static void simple_varargs(void)
{
QObject *embedded_obj;
@@ -1525,6 +1589,7 @@ int main(int argc, char **argv)
g_test_add_func("/lists/simple_list", simple_list);
g_test_add_func("/whitespace/simple_whitespace", simple_whitespace);
+ g_test_add_func("/whitespace/pretty", simple_pretty);
g_test_add_func("/varargs/simple_varargs", simple_varargs);
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix()
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix() Eric Blake
@ 2016-05-02 13:56 ` Markus Armbruster
2016-05-02 15:14 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 13:56 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> The next patch will add pretty indentation to the JSON visitor.
> But in order to support pretty output in the type_any() callback,
> we need to prefix every line of the QObject visitor by the current
> indentation in the JSON visitor. Hence, a new function
> qobject_to_json_pretty_indent(), and the old function becomes a
> thin wrapper to the expanded behavior.
>
> While at it, change 'pretty' to be a bool, to match its usage.
>
> Note that the new simple_pretty() test is a bit sensitive to our
> current notion of prettiness, as well as to the hash ordering in
> QDict (most of the tests in check-qobject-json intentionally do
> not compare the original string to the round-trip string, because
> we liberally accept more input forms than the canonical form that
> we output).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: no change
> v2: no change
> ---
> include/qapi/qmp/qobject-json.h | 1 +
> qobject/qobject-json.c | 40 ++++++++++++++++++-------
> tests/check-qobject-json.c | 65 +++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 95 insertions(+), 11 deletions(-)
>
> diff --git a/include/qapi/qmp/qobject-json.h b/include/qapi/qmp/qobject-json.h
> index 02b1f2c..d420c71 100644
> --- a/include/qapi/qmp/qobject-json.h
> +++ b/include/qapi/qmp/qobject-json.h
> @@ -23,5 +23,6 @@ QObject *qobject_from_jsonv(const char *string, va_list *ap) GCC_FMT_ATTR(1, 0);
>
> QString *qobject_to_json(const QObject *obj);
> QString *qobject_to_json_pretty(const QObject *obj);
> +QString *qobject_to_json_pretty_prefix(const QObject *obj, const char *prefix);
Why a string prefix, and not indentation?
>
> #endif /* QJSON_H */
[...]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix()
2016-05-02 13:56 ` Markus Armbruster
@ 2016-05-02 15:14 ` Eric Blake
2016-05-03 8:32 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-02 15:14 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]
On 05/02/2016 07:56 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> The next patch will add pretty indentation to the JSON visitor.
>> But in order to support pretty output in the type_any() callback,
>> we need to prefix every line of the QObject visitor by the current
>> indentation in the JSON visitor. Hence, a new function
>> qobject_to_json_pretty_indent(), and the old function becomes a
>> thin wrapper to the expanded behavior.
>>
>> QString *qobject_to_json(const QObject *obj);
>> QString *qobject_to_json_pretty(const QObject *obj);
>> +QString *qobject_to_json_pretty_prefix(const QObject *obj, const char *prefix);
>
> Why a string prefix, and not indentation?
Might be possible. Would it be better as an integer for 'number of
spaces' or for 'number of indentation levels (where number of spaces per
indentation-level is not configurable)'?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix()
2016-05-02 15:14 ` Eric Blake
@ 2016-05-03 8:32 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 8:32 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 05/02/2016 07:56 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> The next patch will add pretty indentation to the JSON visitor.
>>> But in order to support pretty output in the type_any() callback,
>>> we need to prefix every line of the QObject visitor by the current
>>> indentation in the JSON visitor. Hence, a new function
>>> qobject_to_json_pretty_indent(), and the old function becomes a
>>> thin wrapper to the expanded behavior.
>>>
>
>>> QString *qobject_to_json(const QObject *obj);
>>> QString *qobject_to_json_pretty(const QObject *obj);
>>> +QString *qobject_to_json_pretty_prefix(const QObject *obj, const char *prefix);
>>
>> Why a string prefix, and not indentation?
>
> Might be possible. Would it be better as an integer for 'number of
> spaces' or for 'number of indentation levels (where number of spaces per
> indentation-level is not configurable)'?
I'd go with indentation levels. The number of spaces per level is
already an implementation detail of the pretty printer. Duplicating it
outside of it is not a good idea.
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 13/18] qapi: Support pretty printing in JSON output visitor
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (11 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 12/18] qapi: Add qobject_to_json_pretty_prefix() Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 14/18] qemu-img: Use new JSON output formatter Eric Blake
` (5 subsequent siblings)
18 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Michael Roth, Juan Quintela, Amit Shah
Similar to pretty printing in the QObject visitor. The trickiest
parts are the fact that during type_any(), we have to coordinate
with QObject to also print pretty; and the fact that the testsuite
now has to honor parameterization on whether pretty printing is
enabled.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: rebase to earlier changes
v2: rebase to earlier changes
---
include/qapi/json-output-visitor.h | 2 +-
migration/savevm.c | 2 +-
qapi/json-output-visitor.c | 25 +++++-
tests/test-json-output-visitor.c | 160 ++++++++++++++++++++++++++-----------
4 files changed, 136 insertions(+), 53 deletions(-)
diff --git a/include/qapi/json-output-visitor.h b/include/qapi/json-output-visitor.h
index ac03da1..53ffdd7 100644
--- a/include/qapi/json-output-visitor.h
+++ b/include/qapi/json-output-visitor.h
@@ -19,7 +19,7 @@ typedef struct JsonOutputVisitor JsonOutputVisitor;
* The JSON output visitor does not accept Infinity or NaN to
* visit_type_number().
*/
-JsonOutputVisitor *json_output_visitor_new(void);
+JsonOutputVisitor *json_output_visitor_new(bool pretty);
void json_output_visitor_cleanup(JsonOutputVisitor *v);
void json_output_visitor_reset(JsonOutputVisitor *v);
diff --git a/migration/savevm.c b/migration/savevm.c
index c858fe9..8cb3af9 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1082,7 +1082,7 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
return;
}
- vmdesc_jov = json_output_visitor_new();
+ vmdesc_jov = json_output_visitor_new(false);
vmdesc = json_output_get_visitor(vmdesc_jov);
visit_start_struct(vmdesc, NULL, NULL, 0, &error_abort);
tmp_i = TARGET_PAGE_SIZE;
diff --git a/qapi/json-output-visitor.c b/qapi/json-output-visitor.c
index 3539db5..f479034 100644
--- a/qapi/json-output-visitor.c
+++ b/qapi/json-output-visitor.c
@@ -19,6 +19,7 @@
struct JsonOutputVisitor {
Visitor visitor;
QString *str;
+ bool pretty;
bool comma;
unsigned int depth;
};
@@ -34,10 +35,13 @@ static void json_output_name(JsonOutputVisitor *jov, const char *name)
jov->str = qstring_new();
}
if (jov->comma) {
- qstring_append(jov->str, ", ");
+ qstring_append(jov->str, jov->pretty ? "," : ", ");
} else {
jov->comma = true;
}
+ if (jov->pretty && jov->depth) {
+ qstring_append_format(jov->str, "\n%*s", 4 * jov->depth, "");
+ }
if (name && jov->depth) {
qstring_append_json_string(jov->str, name);
qstring_append(jov->str, ": ");
@@ -59,6 +63,9 @@ static void json_output_end_struct(Visitor *v)
JsonOutputVisitor *jov = to_jov(v);
assert(jov->depth);
jov->depth--;
+ if (jov->pretty) {
+ qstring_append_format(jov->str, "\n%*s", 4 * jov->depth, "");
+ }
qstring_append(jov->str, "}");
jov->comma = true;
}
@@ -85,6 +92,9 @@ static void json_output_end_list(Visitor *v)
JsonOutputVisitor *jov = to_jov(v);
assert(jov->depth);
jov->depth--;
+ if (jov->pretty) {
+ qstring_append_format(jov->str, "\n%*s", 4 * jov->depth, "");
+ }
qstring_append(jov->str, "]");
jov->comma = true;
}
@@ -134,7 +144,15 @@ static void json_output_type_any(Visitor *v, const char *name, QObject **obj,
Error **errp)
{
JsonOutputVisitor *jov = to_jov(v);
- QString *str = qobject_to_json(*obj);
+ QString *str;
+
+ if (jov->pretty) {
+ char *prefix = g_strdup_printf("%*s", 4 * jov->depth, "");
+ str = qobject_to_json_pretty_prefix(*obj, prefix);
+ g_free(prefix);
+ } else {
+ str = qobject_to_json(*obj);
+ }
assert(str);
json_output_name(jov, name);
qstring_append(jov->str, qstring_get_str(str));
@@ -178,11 +196,12 @@ void json_output_visitor_cleanup(JsonOutputVisitor *v)
g_free(v);
}
-JsonOutputVisitor *json_output_visitor_new(void)
+JsonOutputVisitor *json_output_visitor_new(bool pretty)
{
JsonOutputVisitor *v;
v = g_malloc0(sizeof(*v));
+ v->pretty = pretty;
v->visitor.type = VISITOR_OUTPUT;
v->visitor.start_struct = json_output_start_struct;
diff --git a/tests/test-json-output-visitor.c b/tests/test-json-output-visitor.c
index 57fe1c6..5bf6fc5 100644
--- a/tests/test-json-output-visitor.c
+++ b/tests/test-json-output-visitor.c
@@ -23,9 +23,10 @@ typedef struct TestOutputVisitorData {
} TestOutputVisitorData;
static void visitor_output_setup(TestOutputVisitorData *data,
- const void *unused)
+ const void *arg)
{
- data->jov = json_output_visitor_new();
+ const bool *pretty = arg;
+ data->jov = json_output_visitor_new(*pretty);
g_assert(data->jov);
data->ov = json_output_get_visitor(data->jov);
@@ -158,8 +159,9 @@ static void test_visitor_out_struct(TestOutputVisitorData *data,
}
static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
- const void *unused)
+ const void *arg)
{
+ const bool *pretty = arg;
int64_t value = 42;
UserDefTwo *ud2;
const char *string = "user def string";
@@ -189,27 +191,51 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
visit_type_UserDefTwo(data->ov, "unused", &ud2, &error_abort);
out = json_output_get_string(data->jov);
- g_assert_cmpstr(out, ==,
- "{"
- "\"string0\": \"forty two\", "
- "\"dict1\": {"
- "\"string1\": \"forty three\", "
- "\"dict2\": {"
- "\"userdef\": {"
- "\"integer\": 42, "
- "\"string\": \"user def string\""
- "}, "
- "\"string\": \"forty four\""
- "}, "
- "\"dict3\": {"
- "\"userdef\": {"
- "\"integer\": 42, "
- "\"string\": \"user def string\""
- "}, "
- "\"string\": \"forty five\""
- "}"
- "}"
- "}");
+ if (*pretty) {
+ g_assert_cmpstr(out, ==,
+ "{\n"
+ " \"string0\": \"forty two\",\n"
+ " \"dict1\": {\n"
+ " \"string1\": \"forty three\",\n"
+ " \"dict2\": {\n"
+ " \"userdef\": {\n"
+ " \"integer\": 42,\n"
+ " \"string\": \"user def string\"\n"
+ " },\n"
+ " \"string\": \"forty four\"\n"
+ " },\n"
+ " \"dict3\": {\n"
+ " \"userdef\": {\n"
+ " \"integer\": 42,\n"
+ " \"string\": \"user def string\"\n"
+ " },\n"
+ " \"string\": \"forty five\"\n"
+ " }\n"
+ " }\n"
+ "}");
+ } else {
+ g_assert_cmpstr(out, ==,
+ "{"
+ "\"string0\": \"forty two\", "
+ "\"dict1\": {"
+ "\"string1\": \"forty three\", "
+ "\"dict2\": {"
+ "\"userdef\": {"
+ "\"integer\": 42, "
+ "\"string\": \"user def string\""
+ "}, "
+ "\"string\": \"forty four\""
+ "}, "
+ "\"dict3\": {"
+ "\"userdef\": {"
+ "\"integer\": 42, "
+ "\"string\": \"user def string\""
+ "}, "
+ "\"string\": \"forty five\""
+ "}"
+ "}"
+ "}");
+ }
qapi_free_UserDefTwo(ud2);
g_free(out);
}
@@ -288,16 +314,23 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
}
static void test_visitor_out_any(TestOutputVisitorData *data,
- const void *unused)
+ const void *arg)
{
+ const bool *pretty = arg;
QObject *qobj;
QDict *qdict;
char *out;
qobj = QOBJECT(qint_from_int(-42));
+ visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
visit_type_any(data->ov, NULL, &qobj, &error_abort);
+ visit_end_list(data->ov);
out = json_output_get_string(data->jov);
- g_assert_cmpstr(out, ==, "-42");
+ if (*pretty) {
+ g_assert_cmpstr(out, ==, "[\n -42\n]");
+ } else {
+ g_assert_cmpstr(out, ==, "[-42]");
+ }
qobject_decref(qobj);
g_free(out);
@@ -306,15 +339,30 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qdict_put(qdict, "boolean", qbool_from_bool(true));
qdict_put(qdict, "string", qstring_from_str("foo"));
qobj = QOBJECT(qdict);
+ visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
visit_type_any(data->ov, NULL, &qobj, &error_abort);
+ visit_end_list(data->ov);
qobject_decref(qobj);
out = json_output_get_string(data->jov);
- g_assert_cmpstr(out, ==,
- "{"
- "\"integer\": -42, "
- "\"boolean\": true, "
- "\"string\": \"foo\""
- "}");
+ if (*pretty) {
+ g_assert_cmpstr(out, ==,
+ "[\n"
+ " {\n"
+ " \"integer\": -42,\n"
+ " \"boolean\": true,\n"
+ " \"string\": \"foo\"\n"
+ " }\n"
+ "]");
+ } else {
+ g_assert_cmpstr(out, ==,
+ "["
+ "{"
+ "\"integer\": -42, "
+ "\"boolean\": true, "
+ "\"string\": \"foo\""
+ "}"
+ "]");
+ }
g_free(out);
}
@@ -380,37 +428,53 @@ static void test_visitor_out_null(TestOutputVisitorData *data,
g_free(out);
}
-static void output_visitor_test_add(const char *testpath,
+static void output_visitor_test_add(const char *testpath, bool *data,
void (*test_func)(TestOutputVisitorData *,
const void *))
{
- g_test_add(testpath, TestOutputVisitorData, NULL, visitor_output_setup,
+ g_test_add(testpath, TestOutputVisitorData, data, visitor_output_setup,
test_func, visitor_output_teardown);
}
int main(int argc, char **argv)
{
+ bool plain = false;
+ bool pretty = true;
+
g_test_init(&argc, &argv, NULL);
- output_visitor_test_add("/visitor/json/int", test_visitor_out_int);
- output_visitor_test_add("/visitor/json/bool", test_visitor_out_bool);
- output_visitor_test_add("/visitor/json/number", test_visitor_out_number);
- output_visitor_test_add("/visitor/json/string", test_visitor_out_string);
- output_visitor_test_add("/visitor/json/enum", test_visitor_out_enum);
- output_visitor_test_add("/visitor/json/enum-errors",
+ output_visitor_test_add("/visitor/json/int", &plain,
+ test_visitor_out_int);
+ output_visitor_test_add("/visitor/json/bool", &plain,
+ test_visitor_out_bool);
+ output_visitor_test_add("/visitor/json/number", &plain,
+ test_visitor_out_number);
+ output_visitor_test_add("/visitor/json/string", &plain,
+ test_visitor_out_string);
+ output_visitor_test_add("/visitor/json/enum", &plain,
+ test_visitor_out_enum);
+ output_visitor_test_add("/visitor/json/enum-errors", &plain,
test_visitor_out_enum_errors);
- output_visitor_test_add("/visitor/json/struct", test_visitor_out_struct);
- output_visitor_test_add("/visitor/json/struct-nested",
+ output_visitor_test_add("/visitor/json/struct", &plain,
+ test_visitor_out_struct);
+ output_visitor_test_add("/visitor/json/struct-nested", &plain,
test_visitor_out_struct_nested);
- output_visitor_test_add("/visitor/json/struct-errors",
+ output_visitor_test_add("/visitor/json-pretty/struct-nested", &pretty,
+ test_visitor_out_struct_nested);
+ output_visitor_test_add("/visitor/json/struct-errors", &plain,
test_visitor_out_struct_errors);
- output_visitor_test_add("/visitor/json/list", test_visitor_out_list);
- output_visitor_test_add("/visitor/json/any", test_visitor_out_any);
- output_visitor_test_add("/visitor/json/union-flat",
+ output_visitor_test_add("/visitor/json/list", &plain,
+ test_visitor_out_list);
+ output_visitor_test_add("/visitor/json/any", &plain,
+ test_visitor_out_any);
+ output_visitor_test_add("/visitor/json-pretty/any", &pretty,
+ test_visitor_out_any);
+ output_visitor_test_add("/visitor/json/union-flat", &plain,
test_visitor_out_union_flat);
- output_visitor_test_add("/visitor/json/alternate",
+ output_visitor_test_add("/visitor/json/alternate", &plain,
test_visitor_out_alternate);
- output_visitor_test_add("/visitor/json/null", test_visitor_out_null);
+ output_visitor_test_add("/visitor/json/null", &plain,
+ test_visitor_out_null);
g_test_run();
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* [Qemu-devel] [PATCH v3 14/18] qemu-img: Use new JSON output formatter
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (12 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 13/18] qapi: Support pretty printing in JSON output visitor Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 14:04 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor Eric Blake
` (4 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel
Cc: armbru, famz, Kevin Wolf, Max Reitz, open list:Block layer core
Now that we can pretty-print straight to JSON from a visitor,
we can eliminate the temporary conversion into QObject inside
qemu-img.
The changes to qemu-iotests 043 expected output demonstrates
the fact that output is now done in qapi declaration order,
rather than QDict hash order.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: rebase to master
v2: Drop RFC, adjust expected output of 043; rebase to 'name' motion
Overlaps with Fam's qemu-img edits, although he has expressed
interest in getting this one in first.
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01756.html
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01806.html
---
qemu-img.c | 69 +++++++++++++++++++---------------------------
tests/qemu-iotests/043.out | 22 +++++++--------
2 files changed, 40 insertions(+), 51 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
index e976851..7bc3610 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -24,9 +24,9 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/json-output-visitor.h"
#include "qapi/qmp/qerror.h"
-#include "qapi/qmp/qobject-json.h"
+#include "qapi/qmp/types.h"
#include "qemu/cutils.h"
#include "qemu/config-file.h"
#include "qemu/option.h"
@@ -479,19 +479,15 @@ fail:
static void dump_json_image_check(ImageCheck *check, bool quiet)
{
- Error *local_err = NULL;
- QString *str;
- QmpOutputVisitor *ov = qmp_output_visitor_new();
- QObject *obj;
- visit_type_ImageCheck(qmp_output_get_visitor(ov), NULL, &check,
- &local_err);
- obj = qmp_output_get_qobject(ov);
- str = qobject_to_json_pretty(obj);
- assert(str != NULL);
- qprintf(quiet, "%s\n", qstring_get_str(str));
- qobject_decref(obj);
- qmp_output_visitor_cleanup(ov);
- QDECREF(str);
+ char *str;
+ JsonOutputVisitor *ov = json_output_visitor_new(true);
+ visit_type_ImageCheck(json_output_get_visitor(ov), NULL,
+ &check, &error_abort);
+ str = json_output_get_string(ov);
+ assert(str);
+ qprintf(quiet, "%s\n", str);
+ g_free(str);
+ json_output_visitor_cleanup(ov);
}
static void dump_human_image_check(ImageCheck *check, bool quiet)
@@ -2152,35 +2148,28 @@ static void dump_snapshots(BlockDriverState *bs)
static void dump_json_image_info_list(ImageInfoList *list)
{
- Error *local_err = NULL;
- QString *str;
- QmpOutputVisitor *ov = qmp_output_visitor_new();
- QObject *obj;
- visit_type_ImageInfoList(qmp_output_get_visitor(ov), NULL, &list,
- &local_err);
- obj = qmp_output_get_qobject(ov);
- str = qobject_to_json_pretty(obj);
- assert(str != NULL);
- printf("%s\n", qstring_get_str(str));
- qobject_decref(obj);
- qmp_output_visitor_cleanup(ov);
- QDECREF(str);
+ char *str;
+ JsonOutputVisitor *ov = json_output_visitor_new(true);
+ visit_type_ImageInfoList(json_output_get_visitor(ov), NULL,
+ &list, &error_abort);
+ str = json_output_get_string(ov);
+ assert(str);
+ printf("%s\n", str);
+ json_output_visitor_cleanup(ov);
+ g_free(str);
}
static void dump_json_image_info(ImageInfo *info)
{
- Error *local_err = NULL;
- QString *str;
- QmpOutputVisitor *ov = qmp_output_visitor_new();
- QObject *obj;
- visit_type_ImageInfo(qmp_output_get_visitor(ov), NULL, &info, &local_err);
- obj = qmp_output_get_qobject(ov);
- str = qobject_to_json_pretty(obj);
- assert(str != NULL);
- printf("%s\n", qstring_get_str(str));
- qobject_decref(obj);
- qmp_output_visitor_cleanup(ov);
- QDECREF(str);
+ char *str;
+ JsonOutputVisitor *ov = json_output_visitor_new(true);
+ visit_type_ImageInfo(json_output_get_visitor(ov), NULL, &info,
+ &error_abort);
+ str = json_output_get_string(ov);
+ assert(str);
+ printf("%s\n", str);
+ json_output_visitor_cleanup(ov);
+ g_free(str);
}
static void dump_human_image_info_list(ImageInfoList *list)
diff --git a/tests/qemu-iotests/043.out b/tests/qemu-iotests/043.out
index b37d2a3..41697a2 100644
--- a/tests/qemu-iotests/043.out
+++ b/tests/qemu-iotests/043.out
@@ -40,29 +40,29 @@ cluster_size: 65536
== finite chain of length 3 (json) ==
[
{
- "virtual-size": 134217728,
"filename": "TEST_DIR/t.IMGFMT",
- "cluster-size": 65536,
"format": "IMGFMT",
- "full-backing-filename": "TEST_DIR/t.IMGFMT.2.base",
+ "dirty-flag": false,
+ "virtual-size": 134217728,
+ "cluster-size": 65536,
"backing-filename": "TEST_DIR/t.IMGFMT.2.base",
- "dirty-flag": false
+ "full-backing-filename": "TEST_DIR/t.IMGFMT.2.base",
},
{
- "virtual-size": 134217728,
"filename": "TEST_DIR/t.IMGFMT.2.base",
- "cluster-size": 65536,
"format": "IMGFMT",
- "full-backing-filename": "TEST_DIR/t.IMGFMT.1.base",
+ "dirty-flag": false,
+ "virtual-size": 134217728,
+ "cluster-size": 65536,
"backing-filename": "TEST_DIR/t.IMGFMT.1.base",
- "dirty-flag": false
+ "full-backing-filename": "TEST_DIR/t.IMGFMT.1.base",
},
{
- "virtual-size": 134217728,
"filename": "TEST_DIR/t.IMGFMT.1.base",
- "cluster-size": 65536,
"format": "IMGFMT",
- "dirty-flag": false
+ "dirty-flag": false,
+ "virtual-size": 134217728,
+ "cluster-size": 65536,
}
]
*** done
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 14/18] qemu-img: Use new JSON output formatter
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 14/18] qemu-img: Use new JSON output formatter Eric Blake
@ 2016-05-02 14:04 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 14:04 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, Kevin Wolf, famz, open list:Block layer core, Max Reitz
Eric Blake <eblake@redhat.com> writes:
> Now that we can pretty-print straight to JSON from a visitor,
> we can eliminate the temporary conversion into QObject inside
> qemu-img.
>
> The changes to qemu-iotests 043 expected output demonstrates
> the fact that output is now done in qapi declaration order,
> rather than QDict hash order.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: rebase to master
> v2: Drop RFC, adjust expected output of 043; rebase to 'name' motion
>
> Overlaps with Fam's qemu-img edits, although he has expressed
> interest in getting this one in first.
> https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01756.html
> https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01806.html
> ---
> qemu-img.c | 69 +++++++++++++++++++---------------------------
> tests/qemu-iotests/043.out | 22 +++++++--------
> 2 files changed, 40 insertions(+), 51 deletions(-)
>
> diff --git a/qemu-img.c b/qemu-img.c
> index e976851..7bc3610 100644
> --- a/qemu-img.c
> +++ b/qemu-img.c
> @@ -24,9 +24,9 @@
> #include "qemu/osdep.h"
> #include "qapi/error.h"
> #include "qapi-visit.h"
> -#include "qapi/qmp-output-visitor.h"
> +#include "qapi/json-output-visitor.h"
> #include "qapi/qmp/qerror.h"
> -#include "qapi/qmp/qobject-json.h"
> +#include "qapi/qmp/types.h"
> #include "qemu/cutils.h"
> #include "qemu/config-file.h"
> #include "qemu/option.h"
> @@ -479,19 +479,15 @@ fail:
>
> static void dump_json_image_check(ImageCheck *check, bool quiet)
> {
> - Error *local_err = NULL;
> - QString *str;
> - QmpOutputVisitor *ov = qmp_output_visitor_new();
> - QObject *obj;
> - visit_type_ImageCheck(qmp_output_get_visitor(ov), NULL, &check,
> - &local_err);
> - obj = qmp_output_get_qobject(ov);
> - str = qobject_to_json_pretty(obj);
> - assert(str != NULL);
> - qprintf(quiet, "%s\n", qstring_get_str(str));
> - qobject_decref(obj);
> - qmp_output_visitor_cleanup(ov);
> - QDECREF(str);
> + char *str;
> + JsonOutputVisitor *ov = json_output_visitor_new(true);
Blank line between declarations and statements, please. More of the
same below.
> + visit_type_ImageCheck(json_output_get_visitor(ov), NULL,
> + &check, &error_abort);
Silent bug fix: if visit_type_ImageCheck() fails, we now abort instead
of leaking the error object. Separate patch or mention in the commit
message.
> + str = json_output_get_string(ov);
> + assert(str);
Can json_output_get_string() legitimately return null?
If no, shouldn't the assertion live there rather than here?
> + qprintf(quiet, "%s\n", str);
> + g_free(str);
> + json_output_visitor_cleanup(ov);
> }
>
> static void dump_human_image_check(ImageCheck *check, bool quiet)
> @@ -2152,35 +2148,28 @@ static void dump_snapshots(BlockDriverState *bs)
>
> static void dump_json_image_info_list(ImageInfoList *list)
> {
> - Error *local_err = NULL;
> - QString *str;
> - QmpOutputVisitor *ov = qmp_output_visitor_new();
> - QObject *obj;
> - visit_type_ImageInfoList(qmp_output_get_visitor(ov), NULL, &list,
> - &local_err);
> - obj = qmp_output_get_qobject(ov);
> - str = qobject_to_json_pretty(obj);
> - assert(str != NULL);
> - printf("%s\n", qstring_get_str(str));
> - qobject_decref(obj);
> - qmp_output_visitor_cleanup(ov);
> - QDECREF(str);
> + char *str;
> + JsonOutputVisitor *ov = json_output_visitor_new(true);
> + visit_type_ImageInfoList(json_output_get_visitor(ov), NULL,
> + &list, &error_abort);
> + str = json_output_get_string(ov);
> + assert(str);
> + printf("%s\n", str);
> + json_output_visitor_cleanup(ov);
> + g_free(str);
> }
>
> static void dump_json_image_info(ImageInfo *info)
> {
> - Error *local_err = NULL;
> - QString *str;
> - QmpOutputVisitor *ov = qmp_output_visitor_new();
> - QObject *obj;
> - visit_type_ImageInfo(qmp_output_get_visitor(ov), NULL, &info, &local_err);
> - obj = qmp_output_get_qobject(ov);
> - str = qobject_to_json_pretty(obj);
> - assert(str != NULL);
> - printf("%s\n", qstring_get_str(str));
> - qobject_decref(obj);
> - qmp_output_visitor_cleanup(ov);
> - QDECREF(str);
> + char *str;
> + JsonOutputVisitor *ov = json_output_visitor_new(true);
> + visit_type_ImageInfo(json_output_get_visitor(ov), NULL, &info,
> + &error_abort);
> + str = json_output_get_string(ov);
> + assert(str);
> + printf("%s\n", str);
> + json_output_visitor_cleanup(ov);
> + g_free(str);
> }
>
> static void dump_human_image_info_list(ImageInfoList *list)
> diff --git a/tests/qemu-iotests/043.out b/tests/qemu-iotests/043.out
> index b37d2a3..41697a2 100644
> --- a/tests/qemu-iotests/043.out
> +++ b/tests/qemu-iotests/043.out
> @@ -40,29 +40,29 @@ cluster_size: 65536
> == finite chain of length 3 (json) ==
> [
> {
> - "virtual-size": 134217728,
> "filename": "TEST_DIR/t.IMGFMT",
> - "cluster-size": 65536,
> "format": "IMGFMT",
> - "full-backing-filename": "TEST_DIR/t.IMGFMT.2.base",
> + "dirty-flag": false,
> + "virtual-size": 134217728,
> + "cluster-size": 65536,
> "backing-filename": "TEST_DIR/t.IMGFMT.2.base",
> - "dirty-flag": false
> + "full-backing-filename": "TEST_DIR/t.IMGFMT.2.base",
> },
> {
> - "virtual-size": 134217728,
> "filename": "TEST_DIR/t.IMGFMT.2.base",
> - "cluster-size": 65536,
> "format": "IMGFMT",
> - "full-backing-filename": "TEST_DIR/t.IMGFMT.1.base",
> + "dirty-flag": false,
> + "virtual-size": 134217728,
> + "cluster-size": 65536,
> "backing-filename": "TEST_DIR/t.IMGFMT.1.base",
> - "dirty-flag": false
> + "full-backing-filename": "TEST_DIR/t.IMGFMT.1.base",
> },
> {
> - "virtual-size": 134217728,
> "filename": "TEST_DIR/t.IMGFMT.1.base",
> - "cluster-size": 65536,
> "format": "IMGFMT",
> - "dirty-flag": false
> + "dirty-flag": false,
> + "virtual-size": 134217728,
> + "cluster-size": 65536,
> }
> ]
> *** done
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (13 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 14/18] qemu-img: Use new JSON output formatter Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 17:54 ` Markus Armbruster
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 16/18] sockets: Use new QAPI cloning Eric Blake
` (3 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Michael Roth
We have a couple places in the code base that want to deep-clone
one QAPI object into another, and they were resorting to serializing
the struct out to QObject then reparsing it. A much more efficient
version can be done by adding a new clone visitor.
Note that we can only clone objects (including alternates) and lists,
not built-ins. This is because of things like visit_type_uint8: our
visitor only implements a 64-bit callback, so we have no indication
what size int to read from the source, and cannot blindly assume that
it is safe to read a 64-bit int. As long as a built-in is not the
root of the visit, scalars copy over just fine by virtue of a
g_memdup() each time we push another struct onto the stack.
As such, I tried to document that the clone visitor is for direct
use only by generated code; other code should stick to wrappers.
Add testsuite coverage for several different clone situations, to
ensure that the code is working. I also tested that valgrind was
happy with the test.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
include/qapi/visitor.h | 27 ++---
include/qapi/visitor-impl.h | 1 +
scripts/qapi-types.py | 42 ++++++++
include/qapi/clone-visitor.h | 28 +++++
qapi/qapi-visit-core.c | 1 +
qapi/qapi-clone-visitor.c | 166 ++++++++++++++++++++++++++++++
tests/test-clone-visitor.c | 239 +++++++++++++++++++++++++++++++++++++++++++
docs/qapi-code-gen.txt | 38 +++++++
qapi/Makefile.objs | 2 +-
tests/.gitignore | 1 +
tests/Makefile | 5 +-
11 files changed, 536 insertions(+), 14 deletions(-)
create mode 100644 include/qapi/clone-visitor.h
create mode 100644 qapi/qapi-clone-visitor.c
create mode 100644 tests/test-clone-visitor.c
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index e8a4403..4c122cc 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -24,14 +24,15 @@
* for doing work at each node of a QAPI graph; it can also be used
* for a virtual walk, where there is no actual QAPI C struct.
*
- * There are three kinds of visitor classes: input visitors (QMP,
+ * There are four kinds of visitor classes: input visitors (QMP,
* string, and QemuOpts) parse an external representation and build
* the corresponding QAPI graph, output visitors (QMP, string, and
* JSON) take a completed QAPI graph and generate an external
- * representation, and the dealloc visitor can take a QAPI graph
- * (possibly partially constructed) and recursively free its
- * resources. While the dealloc and QMP input/output visitors are
- * general, the string and QemuOpts visitors have some implementation
+ * representation, the dealloc visitor can take a QAPI graph (possibly
+ * partially constructed) and recursively free its resources, and the
+ * clone visitor performs a deep clone of one QAPI object to another.
+ * While the dealloc and QMP input/output visitors are general, the
+ * clone, string and QemuOpts visitors have some implementation
* limitations; see the documentation for each visitor for more
* details on what it supports. Also, see visitor-impl.h for the
* callback contracts implemented by each visitor, and
@@ -85,16 +86,18 @@
* struct.
*
* Additionally, in qapi-types.h, all QAPI pointer types (structs,
- * unions, alternates, and lists) have a generated function compatible
- * with:
+ * unions, alternates, and lists) have two generated functions
+ * compatible with:
*
+ * FOO *qapi_FOO_clone(const FOO *src);
* void qapi_free_FOO(FOO *obj);
*
- * which behaves like free() in that @obj may be NULL. Because of
- * these functions, the dealloc visitor is seldom used directly
- * outside of generated code. QAPI types can also inherit from a base
- * class; when this happens, a function is generated for easily going
- * from the derived type to the base type:
+ * where the former does a deep clone, and the latter behaves like
+ * free() in that @obj may be NULL. Because of these functions, the
+ * clone and dealloc visitor are seldom used directly outside of
+ * generated code. QAPI types can also inherit from a base class;
+ * when this happens, a function is generated for easily going from
+ * the derived type to the base type:
*
* BASE *qapi_CHILD_base(CHILD *obj);
*
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 145afd0..a5a2dd0 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -35,6 +35,7 @@ typedef enum VisitorType {
VISITOR_INPUT,
VISITOR_OUTPUT,
VISITOR_DEALLOC,
+ VISITOR_CLONE,
} VisitorType;
struct Visitor
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 437cf6c..c5ac493 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -116,6 +116,38 @@ static inline %(base)s *qapi_%(c_name)s_base(const %(c_name)s *obj)
c_name=c_name(name), base=base.c_name())
+def gen_type_clone_decl(name):
+ return mcgen('''
+
+%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src);
+''',
+ c_name=c_name(name))
+
+
+def gen_type_clone(name):
+ ret = mcgen('''
+
+%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src)
+{
+ QapiCloneVisitor *qcv;
+ Visitor *v;
+ %(c_name)s *dst;
+
+ if (!src) {
+ return NULL;
+ }
+
+ qcv = qapi_clone_visitor_new(src);
+ v = qapi_clone_get_visitor(qcv);
+ visit_type_%(c_name)s(v, NULL, &dst, &error_abort);
+ qapi_clone_visitor_cleanup(qcv);
+ return dst;
+}
+''',
+ c_name=c_name(name))
+ return ret
+
+
def gen_variants(variants):
ret = mcgen('''
union { /* union tag is @%(c_name)s */
@@ -212,12 +244,16 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
if isinstance(element_type, QAPISchemaBuiltinType):
self._btin += gen_fwd_object_or_array(name)
self._btin += gen_array(name, element_type)
+ self._btin += gen_type_clone_decl(name)
self._btin += gen_type_cleanup_decl(name)
if do_builtins:
+ self.defn += gen_type_clone(name)
self.defn += gen_type_cleanup(name)
else:
self._fwdecl += gen_fwd_object_or_array(name)
self.decl += gen_array(name, element_type)
+ self.decl += gen_type_clone_decl(name)
+ self.defn += gen_type_clone(name)
self._gen_type_cleanup(name)
def visit_object_type(self, name, info, base, members, variants):
@@ -232,11 +268,15 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
# directly use rather than repeat type.is_implicit()?
if not name.startswith('q_'):
# implicit types won't be directly allocated/freed
+ self.decl += gen_type_clone_decl(name)
+ self.defn += gen_type_clone(name)
self._gen_type_cleanup(name)
def visit_alternate_type(self, name, info, variants):
self._fwdecl += gen_fwd_object_or_array(name)
self.decl += gen_object(name, None, [variants.tag_member], variants)
+ self.decl += gen_type_clone_decl(name)
+ self.defn += gen_type_clone(name)
self._gen_type_cleanup(name)
# If you link code generated from multiple schemata, you want only one
@@ -288,7 +328,9 @@ h_comment = '''
fdef.write(mcgen('''
#include "qemu/osdep.h"
+#include "qapi/clone-visitor.h"
#include "qapi/dealloc-visitor.h"
+#include "qapi/error.h"
#include "%(prefix)sqapi-types.h"
#include "%(prefix)sqapi-visit.h"
''',
diff --git a/include/qapi/clone-visitor.h b/include/qapi/clone-visitor.h
new file mode 100644
index 0000000..8da5d0f
--- /dev/null
+++ b/include/qapi/clone-visitor.h
@@ -0,0 +1,28 @@
+/*
+ * Clone Visitor
+ *
+ * Copyright (C) 2016 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QAPI_CLONE_VISITOR_H
+#define QAPI_CLONE_VISITOR_H
+
+#include "qapi/visitor.h"
+
+typedef struct QapiCloneVisitor QapiCloneVisitor;
+
+/* The clone visitor is for use only by generated qapi_FOO_clone()
+ * functions; it requires that the root visit occur on an object,
+ * list, or alternate, and is directly not usable on built-in QAPI
+ * types.
+ */
+QapiCloneVisitor *qapi_clone_visitor_new(const void *src);
+void qapi_clone_visitor_cleanup(QapiCloneVisitor *v);
+
+Visitor *qapi_clone_get_visitor(QapiCloneVisitor *v);
+
+#endif
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index f5d4b52..838e5d5 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -325,4 +325,5 @@ void visit_type_enum(Visitor *v, const char *name, int *obj,
} else if (v->type == VISITOR_OUTPUT) {
output_type_enum(v, name, obj, strings, errp);
}
+ /* dealloc and clone visitors have nothing to do */
}
diff --git a/qapi/qapi-clone-visitor.c b/qapi/qapi-clone-visitor.c
new file mode 100644
index 0000000..42384d3
--- /dev/null
+++ b/qapi/qapi-clone-visitor.c
@@ -0,0 +1,166 @@
+/*
+ * Copy one QAPI object to another
+ *
+ * Copyright (C) 2016 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/visitor-impl.h"
+
+struct QapiCloneVisitor {
+ Visitor visitor;
+ const void *root; /* Must be object, alternate, or list */
+ size_t depth;
+};
+
+static QapiCloneVisitor *to_qcv(Visitor *v)
+{
+ return container_of(v, QapiCloneVisitor, visitor);
+}
+
+static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
+ size_t size, Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+
+ if (!obj) {
+ /* Nothing to allocate on the virtual walk during an
+ * alternate, but we still have to push depth.
+ * FIXME: passing obj to visit_end_struct would make this easier */
+ assert(qcv->depth);
+ qcv->depth++;
+ return;
+ }
+
+ *obj = g_memdup(qcv->depth ? *obj : qcv->root, size);
+ qcv->depth++;
+}
+
+static void qapi_clone_end(Visitor *v)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ qcv->depth--;
+}
+
+static void qapi_clone_start_list(Visitor *v, const char *name,
+ GenericList **listp, size_t size,
+ Error **errp)
+{
+ qapi_clone_start_struct(v, name, (void **)listp, size, errp);
+}
+
+static GenericList *qapi_clone_next_list(Visitor *v, GenericList *tail,
+ size_t size)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Unshare the tail of the list cloned by g_memdup */
+ tail->next = g_memdup(tail->next, size);
+ return tail->next;
+}
+
+static void qapi_clone_start_alternate(Visitor *v, const char *name,
+ GenericAlternate **obj, size_t size,
+ bool promote_int, Error **errp)
+{
+ qapi_clone_start_struct(v, name, (void **)obj, size, errp);
+}
+
+static void qapi_clone_type_int64(Visitor *v, const char *name, int64_t *obj,
+ Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Value was already cloned by g_memdup */
+}
+
+static void qapi_clone_type_uint64(Visitor *v, const char *name,
+ uint64_t *obj, Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Value was already cloned by g_memdup */
+}
+
+static void qapi_clone_type_bool(Visitor *v, const char *name, bool *obj,
+ Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Value was already cloned by g_memdup */
+}
+
+static void qapi_clone_type_str(Visitor *v, const char *name, char **obj,
+ Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Pointer was already cloned by g_memdup; create fresh copy */
+ *obj = g_strdup(*obj);
+}
+
+static void qapi_clone_type_number(Visitor *v, const char *name, double *obj,
+ Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Value was already cloned by g_memdup */
+}
+
+static void qapi_clone_type_any(Visitor *v, const char *name, QObject **obj,
+ Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Pointer was already copied by g_memdup; fix the refcount */
+ qobject_incref(*obj);
+}
+
+static void qapi_clone_type_null(Visitor *v, const char *name, Error **errp)
+{
+ QapiCloneVisitor *qcv = to_qcv(v);
+ assert(qcv->depth);
+ /* Nothing to do */
+}
+
+Visitor *qapi_clone_get_visitor(QapiCloneVisitor *v)
+{
+ return &v->visitor;
+}
+
+void qapi_clone_visitor_cleanup(QapiCloneVisitor *v)
+{
+ g_free(v);
+}
+
+QapiCloneVisitor *qapi_clone_visitor_new(const void *src)
+{
+ QapiCloneVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+ v->root = src;
+
+ v->visitor.type = VISITOR_CLONE;
+ v->visitor.start_struct = qapi_clone_start_struct;
+ v->visitor.end_struct = qapi_clone_end;
+ v->visitor.start_list = qapi_clone_start_list;
+ v->visitor.next_list = qapi_clone_next_list;
+ v->visitor.end_list = qapi_clone_end;
+ v->visitor.start_alternate = qapi_clone_start_alternate;
+ v->visitor.end_alternate = qapi_clone_end;
+ v->visitor.type_int64 = qapi_clone_type_int64;
+ v->visitor.type_uint64 = qapi_clone_type_uint64;
+ v->visitor.type_bool = qapi_clone_type_bool;
+ v->visitor.type_str = qapi_clone_type_str;
+ v->visitor.type_number = qapi_clone_type_number;
+ v->visitor.type_any = qapi_clone_type_any;
+ v->visitor.type_null = qapi_clone_type_null;
+
+ return v;
+}
diff --git a/tests/test-clone-visitor.c b/tests/test-clone-visitor.c
new file mode 100644
index 0000000..4aeaece
--- /dev/null
+++ b/tests/test-clone-visitor.c
@@ -0,0 +1,239 @@
+/*
+ * QAPI Clone Visitor unit-tests.
+ *
+ * Copyright (C) 2016 Red Hat Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include <glib.h>
+
+#include "qemu-common.h"
+#include "qapi/clone-visitor.h"
+#include "test-qapi-types.h"
+#include "test-qapi-visit.h"
+#include "qapi/qmp/types.h"
+
+static void test_clone_struct(void)
+{
+ UserDefOne *src, *dst;
+
+ src = g_new0(UserDefOne, 1);
+ src->integer = 42;
+ src->string = g_strdup("Hello");
+ src->has_enum1 = false;
+ src->enum1 = ENUM_ONE_VALUE2;
+
+ dst = qapi_UserDefOne_clone(src);
+ g_assert(dst);
+ g_assert_cmpint(dst->integer, ==, 42);
+ g_assert(dst->string != src->string);
+ g_assert_cmpstr(dst->string, ==, "Hello");
+ g_assert_cmpint(dst->has_enum1, ==, false);
+ /* Our implementation does this, but it is not required:
+ g_assert_cmpint(dst->enum1, ==, ENUM_ONE_VALUE2);
+ */
+
+ qapi_free_UserDefOne(src);
+ qapi_free_UserDefOne(dst);
+}
+
+static void test_clone_alternate(void)
+{
+ AltStrBool *b_src, *s_src, *b_dst, *s_dst;
+
+ b_src = g_new0(AltStrBool, 1);
+ b_src->type = QTYPE_QBOOL;
+ b_src->u.b = true;
+ s_src = g_new0(AltStrBool, 1);
+ s_src->type = QTYPE_QSTRING;
+ s_src->u.s = g_strdup("World");
+
+ b_dst = qapi_AltStrBool_clone(b_src);
+ g_assert(b_dst);
+ g_assert_cmpint(b_dst->type, ==, b_src->type);
+ g_assert_cmpint(b_dst->u.b, ==, b_src->u.b);
+ s_dst = qapi_AltStrBool_clone(s_src);
+ g_assert(s_dst);
+ g_assert_cmpint(s_dst->type, ==, s_src->type);
+ g_assert_cmpstr(s_dst->u.s, ==, s_src->u.s);
+ g_assert(s_dst->u.s != s_src->u.s);
+
+ qapi_free_AltStrBool(b_src);
+ qapi_free_AltStrBool(s_src);
+ qapi_free_AltStrBool(b_dst);
+ qapi_free_AltStrBool(s_dst);
+}
+
+static void test_clone_native_list(void)
+{
+ uint8List *src, *dst;
+ uint8List *tmp = NULL;
+ int i;
+
+ /* Build list in reverse */
+ for (i = 10; i; i--) {
+ src = g_new0(uint8List, 1);
+ src->next = tmp;
+ src->value = i;
+ tmp = src;
+ }
+
+ dst = qapi_uint8List_clone(src);
+ for (tmp = dst, i = 1; i <= 10; i++) {
+ g_assert(tmp);
+ g_assert_cmpint(tmp->value, ==, i);
+ tmp = tmp->next;
+ }
+ g_assert(!tmp);
+
+ qapi_free_uint8List(src);
+ qapi_free_uint8List(dst);
+}
+
+static void test_clone_empty(void)
+{
+ Empty2 *src, *dst;
+
+ src = g_new0(Empty2, 1);
+ dst = qapi_Empty2_clone(src);
+ g_assert(dst);
+ qapi_free_Empty2(src);
+ qapi_free_Empty2(dst);
+}
+
+static void test_clone_complex1(void)
+{
+ UserDefNativeListUnion *src, *dst;
+ anyList *tmp;
+ QDict *dict;
+ QList *list;
+
+ src = g_new0(UserDefNativeListUnion, 1);
+ src->type = USER_DEF_NATIVE_LIST_UNION_KIND_STRING;
+
+ dst = qapi_UserDefNativeListUnion_clone(src);
+ g_assert(dst);
+ g_assert_cmpint(dst->type, ==, src->type);
+ g_assert(!dst->u.string.data);
+ qapi_free_UserDefNativeListUnion(dst);
+
+ src->type = USER_DEF_NATIVE_LIST_UNION_KIND_ANY;
+ tmp = src->u.any.data = g_new0(anyList, 1);
+ tmp->value = QOBJECT(qint_from_int(42));
+ tmp = tmp->next = g_new0(anyList, 1);
+ tmp->value = QOBJECT(dict = qdict_new());
+ qdict_put(dict, "key", qstring_from_str("value"));
+ tmp = tmp->next = g_new0(anyList, 1);
+ tmp->value = QOBJECT(qlist_new());
+
+ dst = qapi_UserDefNativeListUnion_clone(src);
+ g_assert(dst);
+ g_assert_cmpint(dst->type, ==, src->type);
+ tmp = dst->u.any.data;
+ g_assert(tmp);
+ g_assert_cmpint(qobject_type(tmp->value), ==, QTYPE_QINT);
+ g_assert_cmpint(qint_get_int(qobject_to_qint(tmp->value)), ==, 42);
+ tmp = tmp->next;
+ g_assert(tmp);
+ g_assert_cmpint(qobject_type(tmp->value), ==, QTYPE_QDICT);
+ dict = qobject_to_qdict(tmp->value);
+ g_assert_cmpint(qdict_size(dict), ==, 1);
+ g_assert_cmpstr(qdict_get_str(dict, "key"), ==, "value");
+ tmp = tmp->next;
+ g_assert(tmp);
+ g_assert_cmpint(qobject_type(tmp->value), ==, QTYPE_QLIST);
+ list = qobject_to_qlist(tmp->value);
+ g_assert_cmpint(qlist_size(list), ==, 0);
+ tmp = tmp->next;
+ g_assert(!tmp);
+
+ qapi_free_UserDefNativeListUnion(src);
+ qapi_free_UserDefNativeListUnion(dst);
+}
+
+static void test_clone_complex2(void)
+{
+ WrapAlternate *src, *dst;
+
+ src = g_new0(WrapAlternate, 1);
+ src->alt = g_new(UserDefAlternate, 1);
+ src->alt->type = QTYPE_QDICT;
+ src->alt->u.udfu.integer = 42;
+ src->alt->u.udfu.string = g_strdup("Hello");
+ src->alt->u.udfu.enum1 = ENUM_ONE_VALUE3;
+ src->alt->u.udfu.u.value3.intb = 99;
+ src->alt->u.udfu.u.value3.has_a_b = true;
+ src->alt->u.udfu.u.value3.a_b = true;
+
+ dst = qapi_WrapAlternate_clone(src);
+ g_assert(dst);
+ g_assert(dst->alt);
+ g_assert_cmpint(dst->alt->type, ==, QTYPE_QDICT);
+ g_assert_cmpint(dst->alt->u.udfu.integer, ==, 42);
+ g_assert_cmpstr(dst->alt->u.udfu.string, ==, "Hello");
+ g_assert_cmpint(dst->alt->u.udfu.enum1, ==, ENUM_ONE_VALUE3);
+ g_assert_cmpint(dst->alt->u.udfu.u.value3.intb, ==, 99);
+ g_assert_cmpint(dst->alt->u.udfu.u.value3.has_a_b, ==, true);
+ g_assert_cmpint(dst->alt->u.udfu.u.value3.a_b, ==, true);
+
+ qapi_free_WrapAlternate(src);
+ qapi_free_WrapAlternate(dst);
+}
+
+static void test_clone_complex3(void)
+{
+ __org_qemu_x_Struct2 *src, *dst;
+ __org_qemu_x_Union1List *tmp;
+
+ src = g_new0(__org_qemu_x_Struct2, 1);
+ tmp = src->array = g_new0(__org_qemu_x_Union1List, 1);
+ tmp->value = g_new0(__org_qemu_x_Union1, 1);
+ tmp->value->type = ORG_QEMU_X_UNION1_KIND___ORG_QEMU_X_BRANCH;
+ tmp->value->u.__org_qemu_x_branch.data = g_strdup("one");
+ tmp = tmp->next = g_new0(__org_qemu_x_Union1List, 1);
+ tmp->value = g_new0(__org_qemu_x_Union1, 1);
+ tmp->value->type = ORG_QEMU_X_UNION1_KIND___ORG_QEMU_X_BRANCH;
+ tmp->value->u.__org_qemu_x_branch.data = g_strdup("two");
+ tmp = tmp->next = g_new0(__org_qemu_x_Union1List, 1);
+ tmp->value = g_new0(__org_qemu_x_Union1, 1);
+ tmp->value->type = ORG_QEMU_X_UNION1_KIND___ORG_QEMU_X_BRANCH;
+ tmp->value->u.__org_qemu_x_branch.data = g_strdup("three");
+
+ dst = qapi___org_qemu_x_Struct2_clone(src);
+ g_assert(dst);
+ tmp = dst->array;
+ g_assert(tmp);
+ g_assert(tmp->value);
+ g_assert_cmpstr(tmp->value->u.__org_qemu_x_branch.data, ==, "one");
+ tmp = tmp->next;
+ g_assert(tmp);
+ g_assert(tmp->value);
+ g_assert_cmpstr(tmp->value->u.__org_qemu_x_branch.data, ==, "two");
+ tmp = tmp->next;
+ g_assert(tmp);
+ g_assert(tmp->value);
+ g_assert_cmpstr(tmp->value->u.__org_qemu_x_branch.data, ==, "three");
+ tmp = tmp->next;
+ g_assert(!tmp);
+
+ qapi_free___org_qemu_x_Struct2(src);
+ qapi_free___org_qemu_x_Struct2(dst);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ g_test_add_func("/visitor/clone/struct", test_clone_struct);
+ g_test_add_func("/visitor/clone/alternate", test_clone_alternate);
+ g_test_add_func("/visitor/clone/native_list", test_clone_native_list);
+ g_test_add_func("/visitor/clone/empty", test_clone_empty);
+ g_test_add_func("/visitor/clone/complex1", test_clone_complex1);
+ g_test_add_func("/visitor/clone/complex2", test_clone_complex2);
+ g_test_add_func("/visitor/clone/complex3", test_clone_complex3);
+
+ return g_test_run();
+}
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d7d6987..92fbb0e 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -787,6 +787,8 @@ Example:
char *string;
};
+ UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src);
+
void qapi_free_UserDefOne(UserDefOne *obj);
struct UserDefOneList {
@@ -794,12 +796,31 @@ Example:
UserDefOne *value;
};
+ UserDefOneList *qapi_UserDefOneList_clone(const UserDefOneList *src);
+
void qapi_free_UserDefOneList(UserDefOneList *obj);
#endif
$ cat qapi-generated/example-qapi-types.c
[Uninteresting stuff omitted...]
+ UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src)
+ {
+ QapiCloneVisitor *qcv;
+ Visitor *v;
+ UserDefOne *dst;
+
+ if (!src) {
+ return;
+ }
+
+ qcv = qapi_clone_visitor_new(src);
+ v = qapi_clone_get_visitor(qcv);
+ visit_type_UserDefOne(v, NULL, &dst, NULL);
+ qapi_clone_visitor_cleanup(qcv);
+ return dst;
+ }
+
void qapi_free_UserDefOne(UserDefOne *obj)
{
QapiDeallocVisitor *qdv;
@@ -815,6 +836,23 @@ Example:
qapi_dealloc_visitor_cleanup(qdv);
}
+ UserDefOneList *qapi_UserDefOneList_clone(const UserDefOneList *src)
+ {
+ QapiCloneVisitor *qcv;
+ Visitor *v;
+ UserDefOneList *dst;
+
+ if (!dst) {
+ return;
+ }
+
+ qcv = qapi_clone_visitor_new(src);
+ v = qapi_clone_get_visitor(qcv);
+ visit_type_UserDefOneList(v, NULL, &dst, NULL);
+ qapi_clone_visitor_cleanup(qcv);
+ return dst;
+ }
+
void qapi_free_UserDefOneList(UserDefOneList *obj)
{
QapiDeallocVisitor *qdv;
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index b60e11b..1406df7 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,6 +1,6 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
-util-obj-y += opts-visitor.o json-output-visitor.o
+util-obj-y += opts-visitor.o json-output-visitor.o qapi-clone-visitor.o
util-obj-y += qmp-event.o
util-obj-y += qapi-util.o
diff --git a/tests/.gitignore b/tests/.gitignore
index c2aad79..60ff7cc 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -12,6 +12,7 @@ test-aio
test-base64
test-bitops
test-blockjob-txn
+test-clone-visitor
test-coroutine
test-crypto-afsplit
test-crypto-block
diff --git a/tests/Makefile b/tests/Makefile
index 1f8a39d..10ed072 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -24,6 +24,8 @@ check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
check-unit-y += tests/test-json-output-visitor$(EXESUF)
gcov-files-test-json-output-visitor-y = qapi/json-output-visitor.c
+check-unit-y += tests/test-clone-visitor$(EXESUF)
+gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
check-unit-y += tests/test-qmp-input-strict$(EXESUF)
@@ -390,7 +392,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qobject-json.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
- tests/test-json-output-visitor.o \
+ tests/test-clone-visitor.o tests/test-json-output-visitor.o \
tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
tests/test-qmp-commands.o tests/test-visitor-serialization.o \
tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
@@ -482,6 +484,7 @@ tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(te
tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
tests/test-json-output-visitor$(EXESUF): tests/test-json-output-visitor.o $(test-qapi-obj-y)
+tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor Eric Blake
@ 2016-05-02 17:54 ` Markus Armbruster
2016-05-02 19:25 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 17:54 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, famz, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> We have a couple places in the code base that want to deep-clone
> one QAPI object into another, and they were resorting to serializing
> the struct out to QObject then reparsing it. A much more efficient
> version can be done by adding a new clone visitor.
>
> Note that we can only clone objects (including alternates) and lists,
> not built-ins.
not built-ins or enums.
> This is because of things like visit_type_uint8: our
> visitor only implements a 64-bit callback,
"Our visitor implements" suggests it's the clone visitor's fault. It's
actually the visitor core's fault, and only since commit 04e070d.
I guess we could clone everything except the integers narrower than 64
bits if we really wanted to. But all that would buy us now is more
generated code we don't use.
Suggest something like:
Note that we can only clone objects (including alternates) and lists,
not built-ins or enums. The visitor core hides integer width from the
actual visitor (since commit 04e070d), and as long as that's the case,
we can't clone integers. Restricting cloning to just objects and
lists is cleaner than restricting it to non-integers.
> so we have no indication
> what size int to read from the source, and cannot blindly assume that
> it is safe to read a 64-bit int. As long as a built-in is not the
> root of the visit, scalars copy over just fine by virtue of a
> g_memdup() each time we push another struct onto the stack.
>
> As such, I tried to document that the clone visitor is for direct
> use only by generated code; other code should stick to wrappers.
>
> Add testsuite coverage for several different clone situations, to
> ensure that the code is working. I also tested that valgrind was
> happy with the test.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
qapi-types.h grows by 492 lines (19KiB, +19%). Roughly one non-blank
line per non-simple type, including list types. It's included all over
the place.
qapi-types.c grows by 4212 lines (92KiB, +90%).
Is it a good idea to generate all clone functions? As far as I can see,
we use only a fraction of them.
A sufficiently smart link-time optimizer can get rid of the ones we
don't use. But I consider the "sufficiently smart optimizer" argument a
cop out until proven otherwise.
We already generate plenty of code we don't use, but that's not exactly
a reason for generating more code we don't use.
>
> ---
> v3: new patch
> ---
> include/qapi/visitor.h | 27 ++---
> include/qapi/visitor-impl.h | 1 +
> scripts/qapi-types.py | 42 ++++++++
> include/qapi/clone-visitor.h | 28 +++++
> qapi/qapi-visit-core.c | 1 +
> qapi/qapi-clone-visitor.c | 166 ++++++++++++++++++++++++++++++
> tests/test-clone-visitor.c | 239 +++++++++++++++++++++++++++++++++++++++++++
> docs/qapi-code-gen.txt | 38 +++++++
> qapi/Makefile.objs | 2 +-
> tests/.gitignore | 1 +
> tests/Makefile | 5 +-
> 11 files changed, 536 insertions(+), 14 deletions(-)
> create mode 100644 include/qapi/clone-visitor.h
> create mode 100644 qapi/qapi-clone-visitor.c
> create mode 100644 tests/test-clone-visitor.c
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index e8a4403..4c122cc 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -24,14 +24,15 @@
> * for doing work at each node of a QAPI graph; it can also be used
> * for a virtual walk, where there is no actual QAPI C struct.
> *
> - * There are three kinds of visitor classes: input visitors (QMP,
> + * There are four kinds of visitor classes: input visitors (QMP,
> * string, and QemuOpts) parse an external representation and build
> * the corresponding QAPI graph, output visitors (QMP, string, and
> * JSON) take a completed QAPI graph and generate an external
> - * representation, and the dealloc visitor can take a QAPI graph
> - * (possibly partially constructed) and recursively free its
> - * resources. While the dealloc and QMP input/output visitors are
> - * general, the string and QemuOpts visitors have some implementation
> + * representation, the dealloc visitor can take a QAPI graph (possibly
> + * partially constructed) and recursively free its resources, and the
> + * clone visitor performs a deep clone of one QAPI object to another.
> + * While the dealloc and QMP input/output visitors are general, the
> + * clone, string and QemuOpts visitors have some implementation
> * limitations; see the documentation for each visitor for more
> * details on what it supports. Also, see visitor-impl.h for the
> * callback contracts implemented by each visitor, and
> @@ -85,16 +86,18 @@
> * struct.
> *
> * Additionally, in qapi-types.h, all QAPI pointer types (structs,
> - * unions, alternates, and lists) have a generated function compatible
> - * with:
> + * unions, alternates, and lists) have two generated functions
> + * compatible with:
> *
> + * FOO *qapi_FOO_clone(const FOO *src);
> * void qapi_free_FOO(FOO *obj);
> *
> - * which behaves like free() in that @obj may be NULL. Because of
> - * these functions, the dealloc visitor is seldom used directly
> - * outside of generated code. QAPI types can also inherit from a base
> - * class; when this happens, a function is generated for easily going
> - * from the derived type to the base type:
> + * where the former does a deep clone, and the latter behaves like
> + * free() in that @obj may be NULL. Because of these functions, the
> + * clone and dealloc visitor are seldom used directly outside of
> + * generated code. QAPI types can also inherit from a base class;
> + * when this happens, a function is generated for easily going from
> + * the derived type to the base type:
> *
> * BASE *qapi_CHILD_base(CHILD *obj);
> *
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index 145afd0..a5a2dd0 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -35,6 +35,7 @@ typedef enum VisitorType {
> VISITOR_INPUT,
> VISITOR_OUTPUT,
> VISITOR_DEALLOC,
> + VISITOR_CLONE,
It's *two* visitors, running in lockstep: an input visitor visiting
@src, and an output visitor visiting the clone.
To which one does the Visitor's type apply?
Let's review how it's used:
* visit_start_struct()
if (obj && v->type == VISITOR_INPUT) {
assert(!err != !*obj);
}
The clone visitor behaves like an input visitor here.
* visit_start_alternate()
Likewise.
* visit_type_str()
Likewise.
* visit_type_any()
Likewise.
* visit_type_enum()
if (v->type == VISITOR_INPUT) {
input_type_enum(v, name, obj, strings, errp);
} else if (v->type == VISITOR_OUTPUT) {
output_type_enum(v, name, obj, strings, errp);
}
The clone visitor wants to override this completely.
Hypothetical input from / output to binary visitors would probably
want the same. Perhaps this part of commit da72ab0 wasn't a good
idea.
Overall, this looks like an input visitor to me so far. But let's have
a look at its code.
> } VisitorType;
>
> struct Visitor
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index 437cf6c..c5ac493 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -116,6 +116,38 @@ static inline %(base)s *qapi_%(c_name)s_base(const %(c_name)s *obj)
> c_name=c_name(name), base=base.c_name())
>
>
> +def gen_type_clone_decl(name):
> + return mcgen('''
> +
> +%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src);
Hmm. It's qapi_free_FOO(), but qapi_FOO_clone().
> +''',
> + c_name=c_name(name))
> +
> +
> +def gen_type_clone(name):
> + ret = mcgen('''
> +
> +%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src)
> +{
> + QapiCloneVisitor *qcv;
> + Visitor *v;
> + %(c_name)s *dst;
> +
> + if (!src) {
> + return NULL;
> + }
> +
> + qcv = qapi_clone_visitor_new(src);
> + v = qapi_clone_get_visitor(qcv);
> + visit_type_%(c_name)s(v, NULL, &dst, &error_abort);
> + qapi_clone_visitor_cleanup(qcv);
> + return dst;
> +}
> +''',
> + c_name=c_name(name))
> + return ret
> +
> +
> def gen_variants(variants):
> ret = mcgen('''
> union { /* union tag is @%(c_name)s */
> @@ -212,12 +244,16 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
> if isinstance(element_type, QAPISchemaBuiltinType):
> self._btin += gen_fwd_object_or_array(name)
> self._btin += gen_array(name, element_type)
> + self._btin += gen_type_clone_decl(name)
> self._btin += gen_type_cleanup_decl(name)
> if do_builtins:
> + self.defn += gen_type_clone(name)
> self.defn += gen_type_cleanup(name)
> else:
> self._fwdecl += gen_fwd_object_or_array(name)
> self.decl += gen_array(name, element_type)
> + self.decl += gen_type_clone_decl(name)
> + self.defn += gen_type_clone(name)
> self._gen_type_cleanup(name)
>
> def visit_object_type(self, name, info, base, members, variants):
> @@ -232,11 +268,15 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
> # directly use rather than repeat type.is_implicit()?
> if not name.startswith('q_'):
> # implicit types won't be directly allocated/freed
> + self.decl += gen_type_clone_decl(name)
> + self.defn += gen_type_clone(name)
> self._gen_type_cleanup(name)
>
> def visit_alternate_type(self, name, info, variants):
> self._fwdecl += gen_fwd_object_or_array(name)
> self.decl += gen_object(name, None, [variants.tag_member], variants)
> + self.decl += gen_type_clone_decl(name)
> + self.defn += gen_type_clone(name)
> self._gen_type_cleanup(name)
>
> # If you link code generated from multiple schemata, you want only one
> @@ -288,7 +328,9 @@ h_comment = '''
>
> fdef.write(mcgen('''
> #include "qemu/osdep.h"
> +#include "qapi/clone-visitor.h"
> #include "qapi/dealloc-visitor.h"
> +#include "qapi/error.h"
> #include "%(prefix)sqapi-types.h"
> #include "%(prefix)sqapi-visit.h"
> ''',
> diff --git a/include/qapi/clone-visitor.h b/include/qapi/clone-visitor.h
> new file mode 100644
> index 0000000..8da5d0f
> --- /dev/null
> +++ b/include/qapi/clone-visitor.h
> @@ -0,0 +1,28 @@
> +/*
> + * Clone Visitor
> + *
> + * Copyright (C) 2016 Red Hat, Inc.
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
> + * See the COPYING.LIB file in the top-level directory.
> + *
> + */
> +
> +#ifndef QAPI_CLONE_VISITOR_H
> +#define QAPI_CLONE_VISITOR_H
> +
> +#include "qapi/visitor.h"
> +
> +typedef struct QapiCloneVisitor QapiCloneVisitor;
> +
> +/* The clone visitor is for use only by generated qapi_FOO_clone()
> + * functions; it requires that the root visit occur on an object,
> + * list, or alternate, and is directly not usable on built-in QAPI
> + * types.
> + */
Wing your winged comments at both ends, please.
> +QapiCloneVisitor *qapi_clone_visitor_new(const void *src);
> +void qapi_clone_visitor_cleanup(QapiCloneVisitor *v);
> +
> +Visitor *qapi_clone_get_visitor(QapiCloneVisitor *v);
> +
> +#endif
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index f5d4b52..838e5d5 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -325,4 +325,5 @@ void visit_type_enum(Visitor *v, const char *name, int *obj,
> } else if (v->type == VISITOR_OUTPUT) {
> output_type_enum(v, name, obj, strings, errp);
> }
> + /* dealloc and clone visitors have nothing to do */
> }
> diff --git a/qapi/qapi-clone-visitor.c b/qapi/qapi-clone-visitor.c
> new file mode 100644
> index 0000000..42384d3
> --- /dev/null
> +++ b/qapi/qapi-clone-visitor.c
> @@ -0,0 +1,166 @@
> +/*
> + * Copy one QAPI object to another
Uh, "copy" suggests it's an assignment. A clone is more. "Deep copy"
would be better, but I'd say something like "Create a clone of a QAPI
object".
> + *
> + * Copyright (C) 2016 Red Hat, Inc.
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
> + * See the COPYING.LIB file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/clone-visitor.h"
> +#include "qapi/visitor-impl.h"
> +
> +struct QapiCloneVisitor {
> + Visitor visitor;
> + const void *root; /* Must be object, alternate, or list */
> + size_t depth;
> +};
> +
> +static QapiCloneVisitor *to_qcv(Visitor *v)
> +{
> + return container_of(v, QapiCloneVisitor, visitor);
> +}
> +
> +static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
> + size_t size, Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> +
> + if (!obj) {
> + /* Nothing to allocate on the virtual walk during an
> + * alternate, but we still have to push depth.
I don't quite get this comment. I understand why we don't memdup() ---
it's pointless without a place to store the duplicate. I think I
understand why we want to increment qcv->depth unconditionally, but I
might be wrong. What exactly is the meaning of member depth?
What's the relation to alternate?
I see this is temporary. Should I not worry and move on?
> + * FIXME: passing obj to visit_end_struct would make this easier */
> + assert(qcv->depth);
> + qcv->depth++;
> + return;
> + }
> +
> + *obj = g_memdup(qcv->depth ? *obj : qcv->root, size);
> + qcv->depth++;
> +}
> +
> +static void qapi_clone_end(Visitor *v)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + qcv->depth--;
> +}
> +
> +static void qapi_clone_start_list(Visitor *v, const char *name,
> + GenericList **listp, size_t size,
> + Error **errp)
> +{
> + qapi_clone_start_struct(v, name, (void **)listp, size, errp);
> +}
> +
> +static GenericList *qapi_clone_next_list(Visitor *v, GenericList *tail,
> + size_t size)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Unshare the tail of the list cloned by g_memdup */
Humor me: g_memdup()
> + tail->next = g_memdup(tail->next, size);
> + return tail->next;
> +}
> +
> +static void qapi_clone_start_alternate(Visitor *v, const char *name,
> + GenericAlternate **obj, size_t size,
> + bool promote_int, Error **errp)
> +{
> + qapi_clone_start_struct(v, name, (void **)obj, size, errp);
> +}
> +
> +static void qapi_clone_type_int64(Visitor *v, const char *name, int64_t *obj,
> + Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Value was already cloned by g_memdup */
> +}
> +
> +static void qapi_clone_type_uint64(Visitor *v, const char *name,
> + uint64_t *obj, Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Value was already cloned by g_memdup */
> +}
> +
> +static void qapi_clone_type_bool(Visitor *v, const char *name, bool *obj,
> + Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Value was already cloned by g_memdup */
> +}
> +
> +static void qapi_clone_type_str(Visitor *v, const char *name, char **obj,
> + Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Pointer was already cloned by g_memdup; create fresh copy */
> + *obj = g_strdup(*obj);
> +}
> +
> +static void qapi_clone_type_number(Visitor *v, const char *name, double *obj,
> + Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Value was already cloned by g_memdup */
> +}
> +
> +static void qapi_clone_type_any(Visitor *v, const char *name, QObject **obj,
> + Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Pointer was already copied by g_memdup; fix the refcount */
> + qobject_incref(*obj);
'any' values are shared?
> +}
> +
> +static void qapi_clone_type_null(Visitor *v, const char *name, Error **errp)
> +{
> + QapiCloneVisitor *qcv = to_qcv(v);
> + assert(qcv->depth);
> + /* Nothing to do */
> +}
> +
> +Visitor *qapi_clone_get_visitor(QapiCloneVisitor *v)
> +{
> + return &v->visitor;
> +}
> +
> +void qapi_clone_visitor_cleanup(QapiCloneVisitor *v)
> +{
> + g_free(v);
> +}
> +
> +QapiCloneVisitor *qapi_clone_visitor_new(const void *src)
> +{
> + QapiCloneVisitor *v;
> +
> + v = g_malloc0(sizeof(*v));
> + v->root = src;
> +
> + v->visitor.type = VISITOR_CLONE;
> + v->visitor.start_struct = qapi_clone_start_struct;
> + v->visitor.end_struct = qapi_clone_end;
> + v->visitor.start_list = qapi_clone_start_list;
> + v->visitor.next_list = qapi_clone_next_list;
> + v->visitor.end_list = qapi_clone_end;
> + v->visitor.start_alternate = qapi_clone_start_alternate;
> + v->visitor.end_alternate = qapi_clone_end;
> + v->visitor.type_int64 = qapi_clone_type_int64;
> + v->visitor.type_uint64 = qapi_clone_type_uint64;
> + v->visitor.type_bool = qapi_clone_type_bool;
> + v->visitor.type_str = qapi_clone_type_str;
> + v->visitor.type_number = qapi_clone_type_number;
> + v->visitor.type_any = qapi_clone_type_any;
> + v->visitor.type_null = qapi_clone_type_null;
> +
> + return v;
> +}
[Getting late, skipping the test for now]
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index d7d6987..92fbb0e 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -787,6 +787,8 @@ Example:
> char *string;
> };
>
> + UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src);
> +
> void qapi_free_UserDefOne(UserDefOne *obj);
>
> struct UserDefOneList {
> @@ -794,12 +796,31 @@ Example:
> UserDefOne *value;
> };
>
> + UserDefOneList *qapi_UserDefOneList_clone(const UserDefOneList *src);
> +
> void qapi_free_UserDefOneList(UserDefOneList *obj);
>
> #endif
> $ cat qapi-generated/example-qapi-types.c
> [Uninteresting stuff omitted...]
>
> + UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src)
> + {
> + QapiCloneVisitor *qcv;
> + Visitor *v;
> + UserDefOne *dst;
Tab damage. More of the same below.
> +
> + if (!src) {
> + return;
> + }
> +
> + qcv = qapi_clone_visitor_new(src);
> + v = qapi_clone_get_visitor(qcv);
> + visit_type_UserDefOne(v, NULL, &dst, NULL);
> + qapi_clone_visitor_cleanup(qcv);
> + return dst;
> + }
> +
> void qapi_free_UserDefOne(UserDefOne *obj)
> {
> QapiDeallocVisitor *qdv;
> @@ -815,6 +836,23 @@ Example:
> qapi_dealloc_visitor_cleanup(qdv);
> }
>
> + UserDefOneList *qapi_UserDefOneList_clone(const UserDefOneList *src)
> + {
> + QapiCloneVisitor *qcv;
> + Visitor *v;
> + UserDefOneList *dst;
> +
> + if (!dst) {
> + return;
> + }
> +
> + qcv = qapi_clone_visitor_new(src);
> + v = qapi_clone_get_visitor(qcv);
> + visit_type_UserDefOneList(v, NULL, &dst, NULL);
> + qapi_clone_visitor_cleanup(qcv);
> + return dst;
> + }
> +
> void qapi_free_UserDefOneList(UserDefOneList *obj)
> {
> QapiDeallocVisitor *qdv;
> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
> index b60e11b..1406df7 100644
> --- a/qapi/Makefile.objs
> +++ b/qapi/Makefile.objs
> @@ -1,6 +1,6 @@
> util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
> util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
> util-obj-y += string-input-visitor.o string-output-visitor.o
> -util-obj-y += opts-visitor.o json-output-visitor.o
> +util-obj-y += opts-visitor.o json-output-visitor.o qapi-clone-visitor.o
> util-obj-y += qmp-event.o
> util-obj-y += qapi-util.o
> diff --git a/tests/.gitignore b/tests/.gitignore
> index c2aad79..60ff7cc 100644
> --- a/tests/.gitignore
> +++ b/tests/.gitignore
> @@ -12,6 +12,7 @@ test-aio
> test-base64
> test-bitops
> test-blockjob-txn
> +test-clone-visitor
> test-coroutine
> test-crypto-afsplit
> test-crypto-block
> diff --git a/tests/Makefile b/tests/Makefile
> index 1f8a39d..10ed072 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -24,6 +24,8 @@ check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
> gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
> check-unit-y += tests/test-json-output-visitor$(EXESUF)
> gcov-files-test-json-output-visitor-y = qapi/json-output-visitor.c
> +check-unit-y += tests/test-clone-visitor$(EXESUF)
> +gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
> check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
> gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
> check-unit-y += tests/test-qmp-input-strict$(EXESUF)
> @@ -390,7 +392,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
> tests/check-qobject-json.o \
> tests/test-coroutine.o tests/test-string-output-visitor.o \
> tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
> - tests/test-json-output-visitor.o \
> + tests/test-clone-visitor.o tests/test-json-output-visitor.o \
> tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
> tests/test-qmp-commands.o tests/test-visitor-serialization.o \
> tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
> @@ -482,6 +484,7 @@ tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(te
> tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
> tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
> tests/test-json-output-visitor$(EXESUF): tests/test-json-output-visitor.o $(test-qapi-obj-y)
> +tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
> tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
> tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
> tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor
2016-05-02 17:54 ` Markus Armbruster
@ 2016-05-02 19:25 ` Eric Blake
2016-05-03 11:36 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-02 19:25 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 6575 bytes --]
On 05/02/2016 11:54 AM, Markus Armbruster wrote:
>
> qapi-types.h grows by 492 lines (19KiB, +19%). Roughly one non-blank
> line per non-simple type, including list types. It's included all over
> the place.
>
> qapi-types.c grows by 4212 lines (92KiB, +90%).
>
> Is it a good idea to generate all clone functions? As far as I can see,
> we use only a fraction of them.
Would it be worth creating a separate .h file, and only include that
file in the few places that actually want a clone, rather than making
every file pay the price of a larger .h?
>
> A sufficiently smart link-time optimizer can get rid of the ones we
> don't use. But I consider the "sufficiently smart optimizer" argument a
> cop out until proven otherwise.
We could also teach the qapi generator to mark specific types as
cloneable (and then the transitive closure of all subtypes included by
those types are also cloneable), and only generate the clone functions
where it matters, rather than for every type. I can play with that idea
(basically, adding a 'clone':true flag in the .json file, then figuring
out how to propagate that through all subtypes).
>
> We already generate plenty of code we don't use, but that's not exactly
> a reason for generating more code we don't use.
I guess it's a trade-off of whether we will find new uses rather than
the two immediate places that I fix in the next couple of patches - if
we need more types to be cloneable, how much maintenance effort is it to
mark those additional types cloneable?
>> +++ b/include/qapi/visitor-impl.h
>> @@ -35,6 +35,7 @@ typedef enum VisitorType {
>> VISITOR_INPUT,
>> VISITOR_OUTPUT,
>> VISITOR_DEALLOC,
>> + VISITOR_CLONE,
>
> It's *two* visitors, running in lockstep: an input visitor visiting
> @src, and an output visitor visiting the clone.
>
> To which one does the Visitor's type apply?
>
> Let's review how it's used:
>
> * visit_start_struct()
>
> if (obj && v->type == VISITOR_INPUT) {
> assert(!err != !*obj);
> }
>
> The clone visitor behaves like an input visitor here.
It doesn't actually set err, but is indeed allocating *obj.
> * visit_type_enum()
>
> if (v->type == VISITOR_INPUT) {
> input_type_enum(v, name, obj, strings, errp);
> } else if (v->type == VISITOR_OUTPUT) {
> output_type_enum(v, name, obj, strings, errp);
> }
>
> The clone visitor wants to override this completely.
The override is to have visit_type_enum() be a no-op (because we don't
visit top-level enums, and an enum which is a member of a struct or list
is cloned as part of the memdup() of pushing into the struct or list).
The dealloc visitor also has visit_type_enum() be a no-op.
>
> Hypothetical input from / output to binary visitors would probably
> want the same. Perhaps this part of commit da72ab0 wasn't a good
> idea.
The clone visitor is cloning C structs, not other representations (that
is, this is NOT a QObject cloner, and therefore even if we add
hypothetical binary representation input/output visitors, this wil NOT
be cloning those binary representations). The fact that we now have a
fourth category of visitor, where dealloc and clone visitors behave the
same for visit_type_enum, didn't seem too bad.
>> +def gen_type_clone_decl(name):
>> + return mcgen('''
>> +
>> +%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src);
>
> Hmm. It's qapi_free_FOO(), but qapi_FOO_clone().
and qapi_visit_FOO_members(). I'm fine swapping names to whatever you
find nicer, but think qapi_clone_FOO() is probably best now that you
mention it.
>> +static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>> + size_t size, Error **errp)
>> +{
>> + QapiCloneVisitor *qcv = to_qcv(v);
>> +
>> + if (!obj) {
>> + /* Nothing to allocate on the virtual walk during an
>> + * alternate, but we still have to push depth.
>
> I don't quite get this comment. I understand why we don't memdup() ---
> it's pointless without a place to store the duplicate. I think I
> understand why we want to increment qcv->depth unconditionally, but I
> might be wrong. What exactly is the meaning of member depth?
>
> What's the relation to alternate?
>
> I see this is temporary. Should I not worry and move on?
I can still make the comment better.
Basically, when we are doing visit_type_ALTERNATE() and happen to pick
the object branch of the alternate, we call visit_start_struct(NULL)
under the hood; the 'if (!obj)' should NOT be reachable for a top-level
visit (because we state that this visitor is limited and cannot do
virtual walks), but it IS reachable under the hood when walking an
alternate.
So maybe the comment should read:
if (!obj) {
/*
* Reachable only when visiting an alternate. Nothing to allocate,
* and visit_start_alternate() already copied memory, so we have
* nothing further to do except increment depth for proper
* bookkeeping during visit_end_struct().
*/
>
>> + * FIXME: passing obj to visit_end_struct would make this easier */
>> + assert(qcv->depth);
>> + qcv->depth++;
>> + return;
>> + }
>> +
>> + *obj = g_memdup(qcv->depth ? *obj : qcv->root, size);
>> + qcv->depth++;
>> +}
>> +static void qapi_clone_type_any(Visitor *v, const char *name, QObject **obj,
>> + Error **errp)
>> +{
>> + QapiCloneVisitor *qcv = to_qcv(v);
>> + assert(qcv->depth);
>> + /* Pointer was already copied by g_memdup; fix the refcount */
>> + qobject_incref(*obj);
>
> 'any' values are shared?
Unless you know of a way to deep clone QObject, and a reason that a deep
clone is better than sharing. The reason we have to deep clone the rest
of the QAPI object is that QAPI doesn't have ref-counting the way
QObject does.
>> + UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src)
>> + {
>> + QapiCloneVisitor *qcv;
>> + Visitor *v;
>> + UserDefOne *dst;
>
> Tab damage. More of the same below.
I can't make emacs figure out that the docs file doesn't need TABs, so
it's not the first time I've tab-damaged a patch (although sometimes
I've managed to catch it before leaking it to the list - not this time,
though).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor
2016-05-02 19:25 ` Eric Blake
@ 2016-05-03 11:36 ` Markus Armbruster
0 siblings, 0 replies; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 11:36 UTC (permalink / raw)
To: Eric Blake; +Cc: famz, qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 05/02/2016 11:54 AM, Markus Armbruster wrote:
>
>>
>> qapi-types.h grows by 492 lines (19KiB, +19%). Roughly one non-blank
>> line per non-simple type, including list types. It's included all over
>> the place.
>>
>> qapi-types.c grows by 4212 lines (92KiB, +90%).
>>
>> Is it a good idea to generate all clone functions? As far as I can see,
>> we use only a fraction of them.
>
> Would it be worth creating a separate .h file, and only include that
> file in the few places that actually want a clone, rather than making
> every file pay the price of a larger .h?
Possibly, but it might be a bother, as the generator scripts and
Makefiles are designed for one .h + one .c.
>> A sufficiently smart link-time optimizer can get rid of the ones we
>> don't use. But I consider the "sufficiently smart optimizer" argument a
>> cop out until proven otherwise.
>
> We could also teach the qapi generator to mark specific types as
> cloneable (and then the transitive closure of all subtypes included by
> those types are also cloneable), and only generate the clone functions
> where it matters, rather than for every type. I can play with that idea
> (basically, adding a 'clone':true flag in the .json file, then figuring
> out how to propagate that through all subtypes).
Related: generation of list types. We got away with DummyForceArrays
there.
I don't think you need the transitive closure: the generated clone
function doesn't recurse into generated clone functions, it sets up
recursion with the clone visitor.
Taking a step back: the generated clone functions feel like a case of
generitis. Here's their template:
%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src)
{
QapiCloneVisitor *qcv;
Visitor *v;
%(c_name)s *dst;
if (!src) {
return NULL;
}
qcv = qapi_clone_visitor_new(src);
v = qapi_clone_get_visitor(qcv);
visit_type_%(c_name)s(v, NULL, &dst, &error_abort);
qapi_clone_visitor_cleanup(qcv);
return dst;
}
The job could be done by a single function:
void *qapi_clone(const void *src,
void (*visit_type)(Visitor, const char *, void **,
Error **))
{
QapiCloneVisitor *qcv;
Visitor *v;
void *dst;
if (!src) {
return NULL;
}
qcv = qapi_clone_visitor_new(src);
v = qapi_clone_get_visitor(qcv);
visit_type(v, NULL, &dst, &error_abort);
qapi_clone_visitor_cleanup(qcv);
return dst;
}
However, passing a visit_type_FOO to this function requires a type cast.
Type-punning functions is always ugly.
A macro would do the job without the type punning. Inline:
#define QAPI_CLONE(type, src) \
(src ? ({ \
QapiCloneVisitor *qcv; \
Visitor *v; \
type *dst; \
\
qcv = qapi_clone_visitor_new(src); \
v = qapi_clone_get_visitor(qcv); \
visit_type_ ## type(v, NULL, &dst, &error_abort); \
qapi_clone_visitor_cleanup(qcv); \
dst; \
}) \
: NULL)
Out of line:
#define QAPI_DEFINE_CLONE(type, src) \
type *qapi_ ## type ## _clone(const type *src) \
{ \
QapiCloneVisitor *qcv; \
Visitor *v; \
type *dst; \
\
if (!src) { \
return NULL; \
} \
\
qcv = qapi_clone_visitor_new(src); \
v = qapi_clone_get_visitor(qcv); \
visit_type_ ## type(v, NULL, &dst, &error_abort); \
qapi_clone_visitor_cleanup(qcv); \
return dst; \
}
where non-generated code uses QAPI_DEFINE_CLONE() to define the ones we
actually need.
Differently ugly.
Note that the qapi_free_FOO() are similar. They're used more
frequently, which makes the inline macro less attractive.
>> We already generate plenty of code we don't use, but that's not exactly
>> a reason for generating more code we don't use.
>
> I guess it's a trade-off of whether we will find new uses rather than
> the two immediate places that I fix in the next couple of patches - if
> we need more types to be cloneable, how much maintenance effort is it to
> mark those additional types cloneable?
Compared to writing new code that needs a clone, adding one more line to
get the clone function is nothing. The time needed to realize you need
to do it and to figure out how to do it may not be nothing.
Documentation problem, I'd say.
>>> +++ b/include/qapi/visitor-impl.h
>>> @@ -35,6 +35,7 @@ typedef enum VisitorType {
>>> VISITOR_INPUT,
>>> VISITOR_OUTPUT,
>>> VISITOR_DEALLOC,
>>> + VISITOR_CLONE,
>>
>> It's *two* visitors, running in lockstep: an input visitor visiting
>> @src, and an output visitor visiting the clone.
>>
>> To which one does the Visitor's type apply?
>>
>> Let's review how it's used:
>>
>> * visit_start_struct()
>>
>> if (obj && v->type == VISITOR_INPUT) {
>> assert(!err != !*obj);
>> }
>>
>> The clone visitor behaves like an input visitor here.
>
> It doesn't actually set err, but is indeed allocating *obj.
If it could fail, it would set err then.
Followup question: should we assert this for a clone visitor, too?
>> * visit_type_enum()
>>
>> if (v->type == VISITOR_INPUT) {
>> input_type_enum(v, name, obj, strings, errp);
>> } else if (v->type == VISITOR_OUTPUT) {
>> output_type_enum(v, name, obj, strings, errp);
>> }
>>
>> The clone visitor wants to override this completely.
>
> The override is to have visit_type_enum() be a no-op (because we don't
> visit top-level enums, and an enum which is a member of a struct or list
> is cloned as part of the memdup() of pushing into the struct or list).
If we still had the type_enum() method removed in commit da72ab0, we'd
do enums exactly like other scalar types:
static void qapi_clone_type_enum(Visitor *v, const char *name, int *obj,
const char *const strings[],
Error **errp)
{
QapiCloneVisitor *qcv = to_qcv(v);
assert(qcv->depth);
/* Value was already cloned by g_memdup() */
}
> The dealloc visitor also has visit_type_enum() be a no-op.
The dealloc visitor could be viewed as output visitor with a relaxed
precondition: incompletely constructed input is okay. Anyway, it's
semantically special enough to require its own clauses in the visitor
documentation, so making its own visitor type is justifiable.
>> Hypothetical input from / output to binary visitors would probably
>> want the same. Perhaps this part of commit da72ab0 wasn't a good
>> idea.
>
> The clone visitor is cloning C structs, not other representations (that
> is, this is NOT a QObject cloner, and therefore even if we add
> hypothetical binary representation input/output visitors, this wil NOT
> be cloning those binary representations).
Of course. But I was trying to say something else. Let me try again.
Consider the five visitors QMP input, string input, JSON input
(hypothetical), binary input (hypothetical), clone.
All five build a tree of QAPI objects guided by some (visitor-specific)
input. Visitor-specific means the visitor core is oblivious of the kind
of input.
For the QMP input visitor, the input is a QObject.
For the string input visitor, the input is a string in idiosyncratic
syntax.
For the hypothetical JSON input visitor, the input would be an RFC 7159
JSON-text.
For the hypothetical binary input visitor, the input would be a bunch of
bytes.
For the clone visitor, the input is another tree of QAPI objects.
My point is: for the visitor core, there is no difference between these
five. This separation of concerns is only proper.
Except for one place, where we let a visitor detail bleed into the core:
enums. The bleeding happened in commit da72ab0. We did it because it
de-duplicated
> The fact that we now have a
> fourth category of visitor, where dealloc and clone visitors behave the
> same for visit_type_enum, didn't seem too bad.
Oh, it's certainly not *bad*. I'm just wondering whether it's
necessary, and whether the existing conditionals on v->type need
updating. I also hope to gain a better understanding of it in the
process.
>>> +def gen_type_clone_decl(name):
>>> + return mcgen('''
>>> +
>>> +%(c_name)s *qapi_%(c_name)s_clone(const %(c_name)s *src);
>>
>> Hmm. It's qapi_free_FOO(), but qapi_FOO_clone().
>
> and qapi_visit_FOO_members(). I'm fine swapping names to whatever you
> find nicer, but think qapi_clone_FOO() is probably best now that you
> mention it.
>
>
>>> +static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>>> + size_t size, Error **errp)
>>> +{
>>> + QapiCloneVisitor *qcv = to_qcv(v);
>>> +
>>> + if (!obj) {
>>> + /* Nothing to allocate on the virtual walk during an
>>> + * alternate, but we still have to push depth.
>>
>> I don't quite get this comment. I understand why we don't memdup() ---
>> it's pointless without a place to store the duplicate. I think I
>> understand why we want to increment qcv->depth unconditionally, but I
>> might be wrong. What exactly is the meaning of member depth?
>>
>> What's the relation to alternate?
>>
>> I see this is temporary. Should I not worry and move on?
>
> I can still make the comment better.
>
> Basically, when we are doing visit_type_ALTERNATE() and happen to pick
> the object branch of the alternate, we call visit_start_struct(NULL)
> under the hood; the 'if (!obj)' should NOT be reachable for a top-level
> visit (because we state that this visitor is limited and cannot do
> virtual walks), but it IS reachable under the hood when walking an
> alternate.
Right...
> So maybe the comment should read:
>
> if (!obj) {
> /*
> * Reachable only when visiting an alternate. Nothing to allocate,
an alternate's object branch
> * and visit_start_alternate() already copied memory, so we have
> * nothing further to do except increment depth for proper
> * bookkeeping during visit_end_struct().
> */
>
>>
>>> + * FIXME: passing obj to visit_end_struct would make this easier */
>>> + assert(qcv->depth);
>>> + qcv->depth++;
>>> + return;
>>> + }
>>> +
>>> + *obj = g_memdup(qcv->depth ? *obj : qcv->root, size);
>>> + qcv->depth++;
>>> +}
>
>>> +static void qapi_clone_type_any(Visitor *v, const char *name, QObject **obj,
>>> + Error **errp)
>>> +{
>>> + QapiCloneVisitor *qcv = to_qcv(v);
>>> + assert(qcv->depth);
>>> + /* Pointer was already copied by g_memdup; fix the refcount */
>>> + qobject_incref(*obj);
>>
>> 'any' values are shared?
>
> Unless you know of a way to deep clone QObject, and a reason that a deep
> clone is better than sharing. The reason we have to deep clone the rest
> of the QAPI object is that QAPI doesn't have ref-counting the way
> QObject does.
Simplifying management of the object life times is one reason to clone.
The other, more important one is to avoid unwanted sharing of
modifications. Only applies for mutable data, of course.
So yes, life time management is one reason for deep cloning QAPI, and it
doesn't apply to the 'any' values. But the other reason does apply,
since 'any' values can be mutable.
Options:
(1) Document 'any' values are shared between the clone and the original.
Dangerous trap, if you ask me.
(2) Don't implement type_any(). Attempts to clone something containing
'any' values crash. Document that as a restriction. Ugly: we can
generate clone functions that cannot be used. This leads to:
(3) Like (2), but additionally refrain from generating clone functions
that can crash. More restrive than (2), because with (2) such a
clone function still works when the 'any' values are optional and
absent, including empty ['any'].
(4) Implement QObject cloning.
>>> + UserDefOne *qapi_UserDefOne_clone(const UserDefOne *src)
>>> + {
>>> + QapiCloneVisitor *qcv;
>>> + Visitor *v;
>>> + UserDefOne *dst;
>>
>> Tab damage. More of the same below.
>
> I can't make emacs figure out that the docs file doesn't need TABs, so
> it's not the first time I've tab-damaged a patch (although sometimes
> I've managed to catch it before leaking it to the list - not this time,
> though).
We can't disable indent-tabs-mode for all files in .dir-locals.el,
because that would wreak havoc with Makefiles.
We could try disabling it for text-mode.
If we're bold, we could try disabling it for everything, then enable it
for makefile-mode.
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 16/18] sockets: Use new QAPI cloning
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (14 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 15/18] qapi: Add new clone visitor Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 8:30 ` Daniel P. Berrange
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 17/18] replay: " Eric Blake
` (2 subsequent siblings)
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz, Daniel P. Berrange, Gerd Hoffmann, Paolo Bonzini
Rather than rolling our own clone via an expensive conversion
in and back out of QObject, use the generated QAPI version.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
util/qemu-sockets.c | 22 +---------------------
1 file changed, 1 insertion(+), 21 deletions(-)
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 2a2c524..6b70944 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -1129,25 +1129,5 @@ SocketAddress *socket_remote_address(int fd, Error **errp)
void qapi_copy_SocketAddress(SocketAddress **p_dest,
SocketAddress *src)
{
- QmpOutputVisitor *qov;
- QmpInputVisitor *qiv;
- Visitor *ov, *iv;
- QObject *obj;
-
- *p_dest = NULL;
-
- qov = qmp_output_visitor_new();
- ov = qmp_output_get_visitor(qov);
- visit_type_SocketAddress(ov, NULL, &src, &error_abort);
- obj = qmp_output_get_qobject(qov);
- qmp_output_visitor_cleanup(qov);
- if (!obj) {
- return;
- }
-
- qiv = qmp_input_visitor_new(obj, true);
- iv = qmp_input_get_visitor(qiv);
- visit_type_SocketAddress(iv, NULL, p_dest, &error_abort);
- qmp_input_visitor_cleanup(qiv);
- qobject_decref(obj);
+ *p_dest = qapi_SocketAddress_clone(src);
}
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 16/18] sockets: Use new QAPI cloning
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 16/18] sockets: Use new QAPI cloning Eric Blake
@ 2016-04-29 8:30 ` Daniel P. Berrange
0 siblings, 0 replies; 78+ messages in thread
From: Daniel P. Berrange @ 2016-04-29 8:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, armbru, famz, Gerd Hoffmann, Paolo Bonzini
On Thu, Apr 28, 2016 at 10:23:37PM -0600, Eric Blake wrote:
> Rather than rolling our own clone via an expensive conversion
> in and back out of QObject, use the generated QAPI version.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: new patch
> ---
> util/qemu-sockets.c | 22 +---------------------
> 1 file changed, 1 insertion(+), 21 deletions(-)
>
> diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
> index 2a2c524..6b70944 100644
> --- a/util/qemu-sockets.c
> +++ b/util/qemu-sockets.c
> @@ -1129,25 +1129,5 @@ SocketAddress *socket_remote_address(int fd, Error **errp)
> void qapi_copy_SocketAddress(SocketAddress **p_dest,
> SocketAddress *src)
> {
> - QmpOutputVisitor *qov;
> - QmpInputVisitor *qiv;
> - Visitor *ov, *iv;
> - QObject *obj;
> -
> - *p_dest = NULL;
> -
> - qov = qmp_output_visitor_new();
> - ov = qmp_output_get_visitor(qov);
> - visit_type_SocketAddress(ov, NULL, &src, &error_abort);
> - obj = qmp_output_get_qobject(qov);
> - qmp_output_visitor_cleanup(qov);
> - if (!obj) {
> - return;
> - }
> -
> - qiv = qmp_input_visitor_new(obj, true);
> - iv = qmp_input_get_visitor(qiv);
> - visit_type_SocketAddress(iv, NULL, p_dest, &error_abort);
> - qmp_input_visitor_cleanup(qiv);
> - qobject_decref(obj);
> + *p_dest = qapi_SocketAddress_clone(src);
> }
Since it is only one line long, this method is now rather pointless.
I'd suggest just deleting it entirely and updating the callers to
use the real clone method directly.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 78+ messages in thread
* [Qemu-devel] [PATCH v3 17/18] replay: Use new QAPI cloning
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (15 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 16/18] sockets: Use new QAPI cloning Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_* Eric Blake
2016-05-09 8:50 ` [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Paolo Bonzini
18 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, famz
Rather than rolling our own clone via an expensive conversion
in and back out of QObject, use the generated QAPI version.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
replay/replay-input.c | 30 ++----------------------------
1 file changed, 2 insertions(+), 28 deletions(-)
diff --git a/replay/replay-input.c b/replay/replay-input.c
index 03e99d5..5a23d23 100644
--- a/replay/replay-input.c
+++ b/replay/replay-input.c
@@ -20,32 +20,6 @@
#include "qapi/qmp-input-visitor.h"
#include "qapi-visit.h"
-static InputEvent *qapi_clone_InputEvent(InputEvent *src)
-{
- QmpOutputVisitor *qov;
- QmpInputVisitor *qiv;
- Visitor *ov, *iv;
- QObject *obj;
- InputEvent *dst = NULL;
-
- qov = qmp_output_visitor_new();
- ov = qmp_output_get_visitor(qov);
- visit_type_InputEvent(ov, NULL, &src, &error_abort);
- obj = qmp_output_get_qobject(qov);
- qmp_output_visitor_cleanup(qov);
- if (!obj) {
- return NULL;
- }
-
- qiv = qmp_input_visitor_new(obj, true);
- iv = qmp_input_get_visitor(qiv);
- visit_type_InputEvent(iv, NULL, &dst, &error_abort);
- qmp_input_visitor_cleanup(qiv);
- qobject_decref(obj);
-
- return dst;
-}
-
void replay_save_input_event(InputEvent *evt)
{
InputKeyEvent *key;
@@ -143,7 +117,7 @@ InputEvent *replay_read_input_event(void)
break;
}
- return qapi_clone_InputEvent(&evt);
+ return qapi_InputEvent_clone(&evt);
}
void replay_input_event(QemuConsole *src, InputEvent *evt)
@@ -151,7 +125,7 @@ void replay_input_event(QemuConsole *src, InputEvent *evt)
if (replay_mode == REPLAY_MODE_PLAY) {
/* Nothing */
} else if (replay_mode == REPLAY_MODE_RECORD) {
- replay_add_input_event(qapi_clone_InputEvent(evt));
+ replay_add_input_event(qapi_InputEvent_clone(evt));
} else {
qemu_input_event_send_impl(src, evt);
}
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_*
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (16 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 17/18] replay: " Eric Blake
@ 2016-04-29 4:23 ` Eric Blake
2016-05-02 18:20 ` Markus Armbruster
2016-05-09 8:50 ` [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Paolo Bonzini
18 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-04-29 4:23 UTC (permalink / raw)
To: qemu-devel
Cc: armbru, famz, Kevin Wolf, Max Reitz, Michael Roth, David Gibson,
Alexander Graf, Michael S. Tsirkin, Juan Quintela, Amit Shah,
Andreas Färber, open list:Block layer core, open list:sPAPR
Rather than making the dealloc visitor track of stack of pointers
remembered during visit_start_* in order to free them during
visit_end_*, it's a lot easier to just make all callers pass the
same pointer to visit_end_*. The generated code has access to the
same pointer, while all other users are doing virtual walks and
can pass NULL. The dealloc visitor is then greatly simplified.
The clone visitor also gets a minor simplification of not having
to track quite as much depth.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v3: new patch
---
include/qapi/visitor.h | 32 ++++++++++++++++-----------
include/qapi/visitor-impl.h | 6 ++---
scripts/qapi-commands.py | 4 ++--
scripts/qapi-event.py | 2 +-
scripts/qapi-visit.py | 8 +++----
qapi/qapi-visit-core.c | 12 +++++-----
block/crypto.c | 4 ++--
hw/ppc/spapr_drc.c | 4 ++--
hw/virtio/virtio-balloon.c | 4 ++--
migration/savevm.c | 10 ++++-----
migration/vmstate.c | 10 ++++-----
qapi/json-output-visitor.c | 4 ++--
qapi/opts-visitor.c | 4 ++--
qapi/qapi-clone-visitor.c | 11 +++++-----
qapi/qapi-dealloc-visitor.c | 47 +++-------------------------------------
qapi/qmp-input-visitor.c | 2 +-
qapi/qmp-output-visitor.c | 4 ++--
qapi/string-input-visitor.c | 2 +-
qapi/string-output-visitor.c | 2 +-
qom/object.c | 2 +-
qom/object_interfaces.c | 4 ++--
tests/test-json-output-visitor.c | 4 ++--
tests/test-qmp-input-visitor.c | 2 +-
tests/test-qmp-output-visitor.c | 2 +-
docs/qapi-code-gen.txt | 4 ++--
25 files changed, 77 insertions(+), 113 deletions(-)
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 4c122cc..34fdd44 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -196,12 +196,12 @@
* goto outlist;
* }
* outlist:
- * visit_end_list(v);
+ * visit_end_list(v, NULL);
* if (!err) {
* visit_check_struct(v, &err);
* }
* outobj:
- * visit_end_struct(v);
+ * visit_end_struct(v, NULL);
* error_propagate(errp, err);
* ...clean up v...
* </example>
@@ -244,8 +244,8 @@ typedef struct GenericAlternate {
* After visit_start_struct() succeeds, the caller may visit its
* members one after the other, passing the member's name and address
* within the struct. Finally, visit_end_struct() needs to be called
- * to clean up, even if intermediate visits fail. See the examples
- * above.
+ * with the same @obj to clean up, even if intermediate visits fail.
+ * See the examples above.
*
* FIXME Should this be named visit_start_object, since it is also
* used for QAPI unions, and maps to JSON objects?
@@ -269,12 +269,14 @@ void visit_check_struct(Visitor *v, Error **errp);
/*
* Complete an object visit started earlier.
*
+ * @obj must match what was passed to the paired visit_start_struct().
+ *
* Must be called after any successful use of visit_start_struct(),
* even if intermediate processing was skipped due to errors, to allow
* the backend to release any resources. Destroying the visitor early
* behaves as if this was implicitly called.
*/
-void visit_end_struct(Visitor *v);
+void visit_end_struct(Visitor *v, void **obj);
/*** Visiting lists ***/
@@ -301,8 +303,9 @@ void visit_end_struct(Visitor *v);
* visit (where @obj is NULL) uses other means. For each list
* element, call the appropriate visit_type_FOO() with name set to
* NULL and obj set to the address of the value member of the list
- * element. Finally, visit_end_list() needs to be called to clean up,
- * even if intermediate visits fail. See the examples above.
+ * element. Finally, visit_end_list() needs to be called with the
+ * same @list to clean up, even if intermediate visits fail. See the
+ * examples above.
*/
void visit_start_list(Visitor *v, const char *name, GenericList **list,
size_t size, Error **errp);
@@ -326,12 +329,14 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size);
/*
* Complete a list visit started earlier.
*
+ * @list must match what was passed to the paired visit_start_list().
+ *
* Must be called after any successful use of visit_start_list(), even
* if intermediate processing was skipped due to errors, to allow the
* backend to release any resources. Destroying the visitor early
* behaves as if this was implicitly called.
*/
-void visit_end_list(Visitor *v);
+void visit_end_list(Visitor *v, void **list);
/*** Visiting alternates ***/
@@ -349,8 +354,9 @@ void visit_end_list(Visitor *v);
*
* If @promote_int, treat integers as QTYPE_FLOAT.
*
- * If successful, this must be paired with visit_end_alternate() to
- * clean up, even if visiting the contents of the alternate fails.
+ * If successful, this must be paired with visit_end_alternate() with
+ * the same @obj to clean up, even if visiting the contents of the
+ * alternate fails.
*/
void visit_start_alternate(Visitor *v, const char *name,
GenericAlternate **obj, size_t size,
@@ -359,15 +365,15 @@ void visit_start_alternate(Visitor *v, const char *name,
/*
* Finish visiting an alternate type.
*
+ * @obj must match what was passed to the paired visit_start_alternate().
+ *
* Must be called after any successful use of visit_start_alternate(),
* even if intermediate processing was skipped due to errors, to allow
* the backend to release any resources. Destroying the visitor early
* behaves as if this was implicitly called.
*
- * TODO: Should all the visit_end_* interfaces take obj parameter, so
- * that dealloc visitor need not track what was passed in visit_start?
*/
-void visit_end_alternate(Visitor *v);
+void visit_end_alternate(Visitor *v, void **obj);
/*** Other helpers ***/
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index a5a2dd0..fdc1f71 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -48,7 +48,7 @@ struct Visitor
void (*check_struct)(Visitor *v, Error **errp);
/* Must be set to visit structs */
- void (*end_struct)(Visitor *v);
+ void (*end_struct)(Visitor *v, void **obj);
/* Must be set; implementations may require @list to be non-null,
* but must document it. */
@@ -59,7 +59,7 @@ struct Visitor
GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);
/* Must be set */
- void (*end_list)(Visitor *v);
+ void (*end_list)(Visitor *v, void **list);
/* Must be set by input and dealloc visitors to visit alternates;
* optional for output visitors. */
@@ -68,7 +68,7 @@ struct Visitor
bool promote_int, Error **errp);
/* Optional, needed for dealloc visitor */
- void (*end_alternate)(Visitor *v);
+ void (*end_alternate)(Visitor *v, void **obj);
/* Must be set */
void (*type_int64)(Visitor *v, const char *name, int64_t *obj,
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 8c6acb3..971dc4e 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -129,7 +129,7 @@ def gen_marshal(name, arg_type, ret_type):
if (!err) {
visit_check_struct(v, &err);
}
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
if (err) {
goto out;
}
@@ -160,7 +160,7 @@ out:
v = qapi_dealloc_get_visitor(qdv);
visit_start_struct(v, NULL, NULL, 0, NULL);
visit_type_%(c_name)s_members(v, &arg, NULL);
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
qapi_dealloc_visitor_cleanup(qdv);
''',
c_name=arg_type.c_name())
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 21fb167..084c40a 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -101,7 +101,7 @@ def gen_event_send(name, arg_type):
if (!err) {
visit_check_struct(v, &err);
}
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
if (err) {
goto out;
}
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 70ea8ca..7b85d2b 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -129,7 +129,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
}
}
- visit_end_list(v);
+ visit_end_list(v, (void **)obj);
if (err && visit_is_input(v)) {
qapi_free_%(c_name)s(*obj);
*obj = NULL;
@@ -191,7 +191,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
if (!err) {
visit_check_struct(v, &err);
}
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
''',
c_type=var.type.c_name(),
c_name=c_name(var.name))
@@ -210,7 +210,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"%(name)s");
}
- visit_end_alternate(v);
+ visit_end_alternate(v, (void **)obj);
if (err && visit_is_input(v)) {
qapi_free_%(c_name)s(*obj);
*obj = NULL;
@@ -244,7 +244,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
}
visit_check_struct(v, &err);
out_obj:
- visit_end_struct(v);
+ visit_end_struct(v, (void **)obj);
if (err && visit_is_input(v)) {
qapi_free_%(c_name)s(*obj);
*obj = NULL;
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 838e5d5..f339dc2 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -43,9 +43,9 @@ void visit_check_struct(Visitor *v, Error **errp)
}
}
-void visit_end_struct(Visitor *v)
+void visit_end_struct(Visitor *v, void **obj)
{
- v->end_struct(v);
+ v->end_struct(v, obj);
}
void visit_start_list(Visitor *v, const char *name, GenericList **list,
@@ -67,9 +67,9 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
return v->next_list(v, tail, size);
}
-void visit_end_list(Visitor *v)
+void visit_end_list(Visitor *v, void **obj)
{
- v->end_list(v);
+ v->end_list(v, obj);
}
void visit_start_alternate(Visitor *v, const char *name,
@@ -89,10 +89,10 @@ void visit_start_alternate(Visitor *v, const char *name,
error_propagate(errp, err);
}
-void visit_end_alternate(Visitor *v)
+void visit_end_alternate(Visitor *v, void **obj)
{
if (v->end_alternate) {
- v->end_alternate(v);
+ v->end_alternate(v, obj);
}
}
diff --git a/block/crypto.c b/block/crypto.c
index 2424a4c..d2e710a 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -222,7 +222,7 @@ block_crypto_open_opts_init(QCryptoBlockFormat format,
visit_check_struct(opts_get_visitor(ov), &local_err);
}
- visit_end_struct(opts_get_visitor(ov));
+ visit_end_struct(opts_get_visitor(ov), NULL);
out:
if (local_err) {
@@ -269,7 +269,7 @@ block_crypto_create_opts_init(QCryptoBlockFormat format,
visit_check_struct(opts_get_visitor(ov), &local_err);
}
- visit_end_struct(opts_get_visitor(ov));
+ visit_end_struct(opts_get_visitor(ov), NULL);
out:
if (local_err) {
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 94c875d..c1da698 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -298,7 +298,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
/* shouldn't ever see an FDT_END_NODE before FDT_BEGIN_NODE */
g_assert(fdt_depth > 0);
visit_check_struct(v, &err);
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
if (err) {
error_propagate(errp, err);
return;
@@ -321,7 +321,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
return;
}
}
- visit_end_list(v);
+ visit_end_list(v, NULL);
break;
}
default:
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 8c15e09..3c1d03d 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -143,13 +143,13 @@ static void balloon_stats_get_all(Object *obj, Visitor *v, const char *name,
}
visit_check_struct(v, &err);
out_nested:
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
if (!err) {
visit_check_struct(v, &err);
}
out_end:
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
out:
error_propagate(errp, err);
}
diff --git a/migration/savevm.c b/migration/savevm.c
index 8cb3af9..ff6b192 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -665,8 +665,8 @@ static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
tmp = "buffer";
visit_type_str(vmdesc, "type", (char **)&tmp, &error_abort);
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
- visit_end_list(vmdesc);
+ visit_end_struct(vmdesc, NULL);
+ visit_end_list(vmdesc, NULL);
}
}
@@ -1112,7 +1112,7 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
save_section_footer(f, se);
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
+ visit_end_struct(vmdesc, NULL);
}
if (!in_postcopy) {
@@ -1120,9 +1120,9 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
qemu_put_byte(f, QEMU_VM_EOF);
}
- visit_end_list(vmdesc);
+ visit_end_list(vmdesc, NULL);
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
+ visit_end_struct(vmdesc, NULL);
vmdesc_str = json_output_get_string(vmdesc_jov);
vmdesc_len = strlen(vmdesc_str);
diff --git a/migration/vmstate.c b/migration/vmstate.c
index e7f894c..8c789a3 100644
--- a/migration/vmstate.c
+++ b/migration/vmstate.c
@@ -284,13 +284,13 @@ static void vmsd_desc_field_end(const VMStateDescription *vmsd,
if (field->flags & VMS_STRUCT) {
/* We printed a struct in between, close its child object */
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
+ visit_end_struct(vmdesc, NULL);
}
tmp = size;
visit_type_int(vmdesc, "size", &tmp, &error_abort);
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
+ visit_end_struct(vmdesc, NULL);
}
@@ -364,7 +364,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
}
if (vmdesc) {
- visit_end_list(vmdesc);
+ visit_end_list(vmdesc, NULL);
}
vmstate_subsection_save(f, vmsd, opaque, vmdesc);
@@ -464,14 +464,14 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
if (vmdesc) {
visit_check_struct(vmdesc, &error_abort);
- visit_end_struct(vmdesc);
+ visit_end_struct(vmdesc, NULL);
}
}
sub++;
}
if (vmdesc && subsection_found) {
- visit_end_list(vmdesc);
+ visit_end_list(vmdesc, NULL);
}
}
diff --git a/qapi/json-output-visitor.c b/qapi/json-output-visitor.c
index f479034..0d67337 100644
--- a/qapi/json-output-visitor.c
+++ b/qapi/json-output-visitor.c
@@ -58,7 +58,7 @@ static void json_output_start_struct(Visitor *v, const char *name, void **obj,
jov->depth++;
}
-static void json_output_end_struct(Visitor *v)
+static void json_output_end_struct(Visitor *v, void **obj)
{
JsonOutputVisitor *jov = to_jov(v);
assert(jov->depth);
@@ -87,7 +87,7 @@ static GenericList *json_output_next_list(Visitor *v, GenericList *tail,
return tail->next;
}
-static void json_output_end_list(Visitor *v)
+static void json_output_end_list(Visitor *v, void **obj)
{
JsonOutputVisitor *jov = to_jov(v);
assert(jov->depth);
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index 4cf1cf8..dcfbf92 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -180,7 +180,7 @@ opts_check_struct(Visitor *v, Error **errp)
static void
-opts_end_struct(Visitor *v)
+opts_end_struct(Visitor *v, void **obj)
{
OptsVisitor *ov = to_ov(v);
@@ -273,7 +273,7 @@ opts_next_list(Visitor *v, GenericList *tail, size_t size)
static void
-opts_end_list(Visitor *v)
+opts_end_list(Visitor *v, void **obj)
{
OptsVisitor *ov = to_ov(v);
diff --git a/qapi/qapi-clone-visitor.c b/qapi/qapi-clone-visitor.c
index 42384d3..92771e3 100644
--- a/qapi/qapi-clone-visitor.c
+++ b/qapi/qapi-clone-visitor.c
@@ -29,11 +29,8 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
QapiCloneVisitor *qcv = to_qcv(v);
if (!obj) {
- /* Nothing to allocate on the virtual walk during an
- * alternate, but we still have to push depth.
- * FIXME: passing obj to visit_end_struct would make this easier */
+ /* Nothing to allocate on the virtual walk */
assert(qcv->depth);
- qcv->depth++;
return;
}
@@ -41,11 +38,13 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
qcv->depth++;
}
-static void qapi_clone_end(Visitor *v)
+static void qapi_clone_end(Visitor *v, void **obj)
{
QapiCloneVisitor *qcv = to_qcv(v);
assert(qcv->depth);
- qcv->depth--;
+ if (obj) {
+ qcv->depth--;
+ }
}
static void qapi_clone_start_list(Visitor *v, const char *name,
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index cd68b55..9391dea 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -19,53 +19,18 @@
#include "qapi/qmp/types.h"
#include "qapi/visitor-impl.h"
-typedef struct StackEntry
-{
- void *value;
- QTAILQ_ENTRY(StackEntry) node;
-} StackEntry;
-
struct QapiDeallocVisitor
{
Visitor visitor;
- QTAILQ_HEAD(, StackEntry) stack;
};
-static QapiDeallocVisitor *to_qov(Visitor *v)
-{
- return container_of(v, QapiDeallocVisitor, visitor);
-}
-
-static void qapi_dealloc_push(QapiDeallocVisitor *qov, void *value)
-{
- StackEntry *e = g_malloc0(sizeof(*e));
-
- e->value = value;
-
- QTAILQ_INSERT_HEAD(&qov->stack, e, node);
-}
-
-static void *qapi_dealloc_pop(QapiDeallocVisitor *qov)
-{
- StackEntry *e = QTAILQ_FIRST(&qov->stack);
- QObject *value;
- QTAILQ_REMOVE(&qov->stack, e, node);
- value = e->value;
- g_free(e);
- return value;
-}
-
static void qapi_dealloc_start_struct(Visitor *v, const char *name, void **obj,
size_t unused, Error **errp)
{
- QapiDeallocVisitor *qov = to_qov(v);
- qapi_dealloc_push(qov, obj);
}
-static void qapi_dealloc_end_struct(Visitor *v)
+static void qapi_dealloc_end_struct(Visitor *v, void **obj)
{
- QapiDeallocVisitor *qov = to_qov(v);
- void **obj = qapi_dealloc_pop(qov);
if (obj) {
g_free(*obj);
}
@@ -75,14 +40,10 @@ static void qapi_dealloc_start_alternate(Visitor *v, const char *name,
GenericAlternate **obj, size_t size,
bool promote_int, Error **errp)
{
- QapiDeallocVisitor *qov = to_qov(v);
- qapi_dealloc_push(qov, obj);
}
-static void qapi_dealloc_end_alternate(Visitor *v)
+static void qapi_dealloc_end_alternate(Visitor *v, void **obj)
{
- QapiDeallocVisitor *qov = to_qov(v);
- void **obj = qapi_dealloc_pop(qov);
if (obj) {
g_free(*obj);
}
@@ -102,7 +63,7 @@ static GenericList *qapi_dealloc_next_list(Visitor *v, GenericList *tail,
return next;
}
-static void qapi_dealloc_end_list(Visitor *v)
+static void qapi_dealloc_end_list(Visitor *v, void **obj)
{
}
@@ -178,7 +139,5 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
v->visitor.type_any = qapi_dealloc_type_anything;
v->visitor.type_null = qapi_dealloc_type_null;
- QTAILQ_INIT(&v->stack);
-
return v;
}
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index aea90a1..84f32fc 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -145,7 +145,7 @@ static void qmp_input_check_struct(Visitor *v, Error **errp)
}
}
-static void qmp_input_pop(Visitor *v)
+static void qmp_input_pop(Visitor *v, void **obj)
{
QmpInputVisitor *qiv = to_qiv(v);
StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index ef9f62c..4bdc00d 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -107,7 +107,7 @@ static void qmp_output_start_struct(Visitor *v, const char *name, void **obj,
qmp_output_push(qov, dict);
}
-static void qmp_output_end_struct(Visitor *v)
+static void qmp_output_end_struct(Visitor *v, void **obj)
{
QmpOutputVisitor *qov = to_qov(v);
QObject *value = qmp_output_pop(qov);
@@ -131,7 +131,7 @@ static GenericList *qmp_output_next_list(Visitor *v, GenericList *tail,
return tail->next;
}
-static void qmp_output_end_list(Visitor *v)
+static void qmp_output_end_list(Visitor *v, void **end)
{
QmpOutputVisitor *qov = to_qov(v);
QObject *value = qmp_output_pop(qov);
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index 30b5879..1c66d6a 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -181,7 +181,7 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
return tail->next;
}
-static void end_list(Visitor *v)
+static void end_list(Visitor *v, void **obj)
{
}
diff --git a/qapi/string-output-visitor.c b/qapi/string-output-visitor.c
index d013196..1504a89 100644
--- a/qapi/string-output-visitor.c
+++ b/qapi/string-output-visitor.c
@@ -291,7 +291,7 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
return ret;
}
-static void end_list(Visitor *v)
+static void end_list(Visitor *v, void **obj)
{
StringOutputVisitor *sov = to_sov(v);
diff --git a/qom/object.c b/qom/object.c
index 3bc8a00..1562f7e 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -2038,7 +2038,7 @@ static void property_get_tm(Object *obj, Visitor *v, const char *name,
}
visit_check_struct(v, &err);
out_end:
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
out:
error_propagate(errp, err);
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index 51e62e2..926ec68 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -71,7 +71,7 @@ Object *user_creatable_add(const QDict *qdict,
obj = user_creatable_add_type(type, id, pdict, v, &local_err);
out_visit:
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
out:
QDECREF(pdict);
@@ -127,7 +127,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
if (!local_err) {
visit_check_struct(v, &local_err);
}
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
if (local_err) {
goto out;
}
diff --git a/tests/test-json-output-visitor.c b/tests/test-json-output-visitor.c
index 5bf6fc5..5e8a5c8 100644
--- a/tests/test-json-output-visitor.c
+++ b/tests/test-json-output-visitor.c
@@ -324,7 +324,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qobj = QOBJECT(qint_from_int(-42));
visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
visit_type_any(data->ov, NULL, &qobj, &error_abort);
- visit_end_list(data->ov);
+ visit_end_list(data->ov, NULL);
out = json_output_get_string(data->jov);
if (*pretty) {
g_assert_cmpstr(out, ==, "[\n -42\n]");
@@ -341,7 +341,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qobj = QOBJECT(qdict);
visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
visit_type_any(data->ov, NULL, &qobj, &error_abort);
- visit_end_list(data->ov);
+ visit_end_list(data->ov, NULL);
qobject_decref(qobj);
out = json_output_get_string(data->jov);
if (*pretty) {
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index d02306c..56ee943 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -305,7 +305,7 @@ static void test_visitor_in_null(TestInputVisitorData *data,
visit_type_null(v, "b", &err);
error_free_or_abort(&err);
visit_check_struct(v, &error_abort);
- visit_end_struct(v);
+ visit_end_struct(v, NULL);
}
static void test_visitor_in_union_flat(TestInputVisitorData *data,
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 050d65b..e2aa494 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -494,7 +494,7 @@ static void test_visitor_out_null(TestOutputVisitorData *data,
visit_start_struct(data->ov, NULL, NULL, 0, &error_abort);
visit_type_null(data->ov, "a", &error_abort);
visit_check_struct(data->ov, &error_abort);
- visit_end_struct(data->ov);
+ visit_end_struct(data->ov, NULL);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QDICT);
qdict = qobject_to_qdict(arg);
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 92fbb0e..48436aa 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -942,7 +942,7 @@ Example:
}
visit_check_struct(v, &err);
out_obj:
- visit_end_struct(v);
+ visit_end_struct(v, (void **)obj);
if (err && visit_is_input(v)) {
qapi_free_UserDefOne(*obj);
*obj = NULL;
@@ -970,7 +970,7 @@ Example:
}
}
- visit_end_list(v);
+ visit_end_list(v, (void **)obj);
if (err && visit_is_input(v)) {
qapi_free_UserDefOneList(*obj);
*obj = NULL;
--
2.5.5
^ permalink raw reply related [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_*
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_* Eric Blake
@ 2016-05-02 18:20 ` Markus Armbruster
2016-05-02 19:31 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-02 18:20 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, Kevin Wolf, Alexander Graf, famz,
open list:Block layer core, Michael S. Tsirkin, Juan Quintela,
Michael Roth, open list:sPAPR, Amit Shah, Max Reitz,
Andreas Färber, David Gibson
Eric Blake <eblake@redhat.com> writes:
> Rather than making the dealloc visitor track of stack of pointers
> remembered during visit_start_* in order to free them during
> visit_end_*, it's a lot easier to just make all callers pass the
> same pointer to visit_end_*. The generated code has access to the
> same pointer, while all other users are doing virtual walks and
> can pass NULL. The dealloc visitor is then greatly simplified.
>
> The clone visitor also gets a minor simplification of not having
> to track quite as much depth.
Do this before adding the clone visitor, to reduce churn?
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v3: new patch
> ---
> include/qapi/visitor.h | 32 ++++++++++++++++-----------
> include/qapi/visitor-impl.h | 6 ++---
> scripts/qapi-commands.py | 4 ++--
> scripts/qapi-event.py | 2 +-
> scripts/qapi-visit.py | 8 +++----
> qapi/qapi-visit-core.c | 12 +++++-----
> block/crypto.c | 4 ++--
> hw/ppc/spapr_drc.c | 4 ++--
> hw/virtio/virtio-balloon.c | 4 ++--
> migration/savevm.c | 10 ++++-----
> migration/vmstate.c | 10 ++++-----
> qapi/json-output-visitor.c | 4 ++--
> qapi/opts-visitor.c | 4 ++--
> qapi/qapi-clone-visitor.c | 11 +++++-----
> qapi/qapi-dealloc-visitor.c | 47 +++-------------------------------------
> qapi/qmp-input-visitor.c | 2 +-
> qapi/qmp-output-visitor.c | 4 ++--
> qapi/string-input-visitor.c | 2 +-
> qapi/string-output-visitor.c | 2 +-
> qom/object.c | 2 +-
> qom/object_interfaces.c | 4 ++--
> tests/test-json-output-visitor.c | 4 ++--
> tests/test-qmp-input-visitor.c | 2 +-
> tests/test-qmp-output-visitor.c | 2 +-
> docs/qapi-code-gen.txt | 4 ++--
> 25 files changed, 77 insertions(+), 113 deletions(-)
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index 4c122cc..34fdd44 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -196,12 +196,12 @@
> * goto outlist;
> * }
> * outlist:
> - * visit_end_list(v);
> + * visit_end_list(v, NULL);
> * if (!err) {
> * visit_check_struct(v, &err);
> * }
> * outobj:
> - * visit_end_struct(v);
> + * visit_end_struct(v, NULL);
> * error_propagate(errp, err);
> * ...clean up v...
> * </example>
> @@ -244,8 +244,8 @@ typedef struct GenericAlternate {
> * After visit_start_struct() succeeds, the caller may visit its
> * members one after the other, passing the member's name and address
> * within the struct. Finally, visit_end_struct() needs to be called
> - * to clean up, even if intermediate visits fail. See the examples
> - * above.
> + * with the same @obj to clean up, even if intermediate visits fail.
> + * See the examples above.
The examples don't show "the same @obj" very well, because it's NULL.
It'll do; the comment containing them is large enough as it is.
> *
> * FIXME Should this be named visit_start_object, since it is also
> * used for QAPI unions, and maps to JSON objects?
> @@ -269,12 +269,14 @@ void visit_check_struct(Visitor *v, Error **errp);
> /*
> * Complete an object visit started earlier.
> *
> + * @obj must match what was passed to the paired visit_start_struct().
> + *
> * Must be called after any successful use of visit_start_struct(),
> * even if intermediate processing was skipped due to errors, to allow
> * the backend to release any resources. Destroying the visitor early
> * behaves as if this was implicitly called.
> */
> -void visit_end_struct(Visitor *v);
> +void visit_end_struct(Visitor *v, void **obj);
>
>
> /*** Visiting lists ***/
> @@ -301,8 +303,9 @@ void visit_end_struct(Visitor *v);
> * visit (where @obj is NULL) uses other means. For each list
> * element, call the appropriate visit_type_FOO() with name set to
> * NULL and obj set to the address of the value member of the list
> - * element. Finally, visit_end_list() needs to be called to clean up,
> - * even if intermediate visits fail. See the examples above.
> + * element. Finally, visit_end_list() needs to be called with the
> + * same @list to clean up, even if intermediate visits fail. See the
> + * examples above.
> */
> void visit_start_list(Visitor *v, const char *name, GenericList **list,
> size_t size, Error **errp);
> @@ -326,12 +329,14 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size);
> /*
> * Complete a list visit started earlier.
> *
> + * @list must match what was passed to the paired visit_start_list().
> + *
> * Must be called after any successful use of visit_start_list(), even
> * if intermediate processing was skipped due to errors, to allow the
> * backend to release any resources. Destroying the visitor early
> * behaves as if this was implicitly called.
> */
> -void visit_end_list(Visitor *v);
> +void visit_end_list(Visitor *v, void **list);
>
>
> /*** Visiting alternates ***/
> @@ -349,8 +354,9 @@ void visit_end_list(Visitor *v);
> *
> * If @promote_int, treat integers as QTYPE_FLOAT.
> *
> - * If successful, this must be paired with visit_end_alternate() to
> - * clean up, even if visiting the contents of the alternate fails.
> + * If successful, this must be paired with visit_end_alternate() with
> + * the same @obj to clean up, even if visiting the contents of the
> + * alternate fails.
> */
> void visit_start_alternate(Visitor *v, const char *name,
> GenericAlternate **obj, size_t size,
> @@ -359,15 +365,15 @@ void visit_start_alternate(Visitor *v, const char *name,
> /*
> * Finish visiting an alternate type.
> *
> + * @obj must match what was passed to the paired visit_start_alternate().
> + *
> * Must be called after any successful use of visit_start_alternate(),
> * even if intermediate processing was skipped due to errors, to allow
> * the backend to release any resources. Destroying the visitor early
> * behaves as if this was implicitly called.
> *
> - * TODO: Should all the visit_end_* interfaces take obj parameter, so
> - * that dealloc visitor need not track what was passed in visit_start?
> */
> -void visit_end_alternate(Visitor *v);
> +void visit_end_alternate(Visitor *v, void **obj);
>
>
> /*** Other helpers ***/
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index a5a2dd0..fdc1f71 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -48,7 +48,7 @@ struct Visitor
> void (*check_struct)(Visitor *v, Error **errp);
>
> /* Must be set to visit structs */
> - void (*end_struct)(Visitor *v);
> + void (*end_struct)(Visitor *v, void **obj);
>
> /* Must be set; implementations may require @list to be non-null,
> * but must document it. */
> @@ -59,7 +59,7 @@ struct Visitor
> GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);
>
> /* Must be set */
> - void (*end_list)(Visitor *v);
> + void (*end_list)(Visitor *v, void **list);
Sure you want void ** and not GenericList **?
>
> /* Must be set by input and dealloc visitors to visit alternates;
> * optional for output visitors. */
> @@ -68,7 +68,7 @@ struct Visitor
> bool promote_int, Error **errp);
>
> /* Optional, needed for dealloc visitor */
> - void (*end_alternate)(Visitor *v);
> + void (*end_alternate)(Visitor *v, void **obj);
Sure you want void ** and not GenericAlternate **?
>
> /* Must be set */
> void (*type_int64)(Visitor *v, const char *name, int64_t *obj,
> diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
> index 8c6acb3..971dc4e 100644
> --- a/scripts/qapi-commands.py
> +++ b/scripts/qapi-commands.py
> @@ -129,7 +129,7 @@ def gen_marshal(name, arg_type, ret_type):
> if (!err) {
> visit_check_struct(v, &err);
> }
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> if (err) {
> goto out;
> }
> @@ -160,7 +160,7 @@ out:
> v = qapi_dealloc_get_visitor(qdv);
> visit_start_struct(v, NULL, NULL, 0, NULL);
> visit_type_%(c_name)s_members(v, &arg, NULL);
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> qapi_dealloc_visitor_cleanup(qdv);
> ''',
> c_name=arg_type.c_name())
> diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
> index 21fb167..084c40a 100644
> --- a/scripts/qapi-event.py
> +++ b/scripts/qapi-event.py
> @@ -101,7 +101,7 @@ def gen_event_send(name, arg_type):
> if (!err) {
> visit_check_struct(v, &err);
> }
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> if (err) {
> goto out;
> }
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index 70ea8ca..7b85d2b 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -129,7 +129,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> }
> }
>
> - visit_end_list(v);
> + visit_end_list(v, (void **)obj);
> if (err && visit_is_input(v)) {
> qapi_free_%(c_name)s(*obj);
> *obj = NULL;
> @@ -191,7 +191,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> if (!err) {
> visit_check_struct(v, &err);
> }
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> ''',
> c_type=var.type.c_name(),
> c_name=c_name(var.name))
> @@ -210,7 +210,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> "%(name)s");
> }
> - visit_end_alternate(v);
> + visit_end_alternate(v, (void **)obj);
> if (err && visit_is_input(v)) {
> qapi_free_%(c_name)s(*obj);
> *obj = NULL;
> @@ -244,7 +244,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> }
> visit_check_struct(v, &err);
> out_obj:
> - visit_end_struct(v);
> + visit_end_struct(v, (void **)obj);
> if (err && visit_is_input(v)) {
> qapi_free_%(c_name)s(*obj);
> *obj = NULL;
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 838e5d5..f339dc2 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -43,9 +43,9 @@ void visit_check_struct(Visitor *v, Error **errp)
> }
> }
>
> -void visit_end_struct(Visitor *v)
> +void visit_end_struct(Visitor *v, void **obj)
> {
> - v->end_struct(v);
> + v->end_struct(v, obj);
> }
>
> void visit_start_list(Visitor *v, const char *name, GenericList **list,
> @@ -67,9 +67,9 @@ GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
> return v->next_list(v, tail, size);
> }
>
> -void visit_end_list(Visitor *v)
> +void visit_end_list(Visitor *v, void **obj)
> {
> - v->end_list(v);
> + v->end_list(v, obj);
> }
>
> void visit_start_alternate(Visitor *v, const char *name,
> @@ -89,10 +89,10 @@ void visit_start_alternate(Visitor *v, const char *name,
> error_propagate(errp, err);
> }
>
> -void visit_end_alternate(Visitor *v)
> +void visit_end_alternate(Visitor *v, void **obj)
> {
> if (v->end_alternate) {
> - v->end_alternate(v);
> + v->end_alternate(v, obj);
> }
> }
>
> diff --git a/block/crypto.c b/block/crypto.c
> index 2424a4c..d2e710a 100644
> --- a/block/crypto.c
> +++ b/block/crypto.c
> @@ -222,7 +222,7 @@ block_crypto_open_opts_init(QCryptoBlockFormat format,
> visit_check_struct(opts_get_visitor(ov), &local_err);
> }
>
> - visit_end_struct(opts_get_visitor(ov));
> + visit_end_struct(opts_get_visitor(ov), NULL);
>
> out:
> if (local_err) {
> @@ -269,7 +269,7 @@ block_crypto_create_opts_init(QCryptoBlockFormat format,
> visit_check_struct(opts_get_visitor(ov), &local_err);
> }
>
> - visit_end_struct(opts_get_visitor(ov));
> + visit_end_struct(opts_get_visitor(ov), NULL);
>
> out:
> if (local_err) {
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index 94c875d..c1da698 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -298,7 +298,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
> /* shouldn't ever see an FDT_END_NODE before FDT_BEGIN_NODE */
> g_assert(fdt_depth > 0);
> visit_check_struct(v, &err);
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> if (err) {
> error_propagate(errp, err);
> return;
> @@ -321,7 +321,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name,
> return;
> }
> }
> - visit_end_list(v);
> + visit_end_list(v, NULL);
> break;
> }
> default:
> diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
> index 8c15e09..3c1d03d 100644
> --- a/hw/virtio/virtio-balloon.c
> +++ b/hw/virtio/virtio-balloon.c
> @@ -143,13 +143,13 @@ static void balloon_stats_get_all(Object *obj, Visitor *v, const char *name,
> }
> visit_check_struct(v, &err);
> out_nested:
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
>
> if (!err) {
> visit_check_struct(v, &err);
> }
> out_end:
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> out:
> error_propagate(errp, err);
> }
> diff --git a/migration/savevm.c b/migration/savevm.c
> index 8cb3af9..ff6b192 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -665,8 +665,8 @@ static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se,
> tmp = "buffer";
> visit_type_str(vmdesc, "type", (char **)&tmp, &error_abort);
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> - visit_end_list(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> + visit_end_list(vmdesc, NULL);
> }
> }
>
> @@ -1112,7 +1112,7 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
> save_section_footer(f, se);
>
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> }
>
> if (!in_postcopy) {
> @@ -1120,9 +1120,9 @@ void qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)
> qemu_put_byte(f, QEMU_VM_EOF);
> }
>
> - visit_end_list(vmdesc);
> + visit_end_list(vmdesc, NULL);
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> vmdesc_str = json_output_get_string(vmdesc_jov);
> vmdesc_len = strlen(vmdesc_str);
>
> diff --git a/migration/vmstate.c b/migration/vmstate.c
> index e7f894c..8c789a3 100644
> --- a/migration/vmstate.c
> +++ b/migration/vmstate.c
> @@ -284,13 +284,13 @@ static void vmsd_desc_field_end(const VMStateDescription *vmsd,
> if (field->flags & VMS_STRUCT) {
> /* We printed a struct in between, close its child object */
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> }
>
> tmp = size;
> visit_type_int(vmdesc, "size", &tmp, &error_abort);
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> }
>
>
> @@ -364,7 +364,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
> }
>
> if (vmdesc) {
> - visit_end_list(vmdesc);
> + visit_end_list(vmdesc, NULL);
> }
>
> vmstate_subsection_save(f, vmsd, opaque, vmdesc);
> @@ -464,14 +464,14 @@ static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
>
> if (vmdesc) {
> visit_check_struct(vmdesc, &error_abort);
> - visit_end_struct(vmdesc);
> + visit_end_struct(vmdesc, NULL);
> }
> }
> sub++;
> }
>
> if (vmdesc && subsection_found) {
> - visit_end_list(vmdesc);
> + visit_end_list(vmdesc, NULL);
> }
> }
>
> diff --git a/qapi/json-output-visitor.c b/qapi/json-output-visitor.c
> index f479034..0d67337 100644
> --- a/qapi/json-output-visitor.c
> +++ b/qapi/json-output-visitor.c
> @@ -58,7 +58,7 @@ static void json_output_start_struct(Visitor *v, const char *name, void **obj,
> jov->depth++;
> }
>
> -static void json_output_end_struct(Visitor *v)
> +static void json_output_end_struct(Visitor *v, void **obj)
> {
> JsonOutputVisitor *jov = to_jov(v);
> assert(jov->depth);
> @@ -87,7 +87,7 @@ static GenericList *json_output_next_list(Visitor *v, GenericList *tail,
> return tail->next;
> }
>
> -static void json_output_end_list(Visitor *v)
> +static void json_output_end_list(Visitor *v, void **obj)
> {
> JsonOutputVisitor *jov = to_jov(v);
> assert(jov->depth);
> diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
> index 4cf1cf8..dcfbf92 100644
> --- a/qapi/opts-visitor.c
> +++ b/qapi/opts-visitor.c
> @@ -180,7 +180,7 @@ opts_check_struct(Visitor *v, Error **errp)
>
>
> static void
> -opts_end_struct(Visitor *v)
> +opts_end_struct(Visitor *v, void **obj)
> {
> OptsVisitor *ov = to_ov(v);
>
> @@ -273,7 +273,7 @@ opts_next_list(Visitor *v, GenericList *tail, size_t size)
>
>
> static void
> -opts_end_list(Visitor *v)
> +opts_end_list(Visitor *v, void **obj)
> {
> OptsVisitor *ov = to_ov(v);
>
> diff --git a/qapi/qapi-clone-visitor.c b/qapi/qapi-clone-visitor.c
> index 42384d3..92771e3 100644
> --- a/qapi/qapi-clone-visitor.c
> +++ b/qapi/qapi-clone-visitor.c
> @@ -29,11 +29,8 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
> QapiCloneVisitor *qcv = to_qcv(v);
>
> if (!obj) {
> - /* Nothing to allocate on the virtual walk during an
> - * alternate, but we still have to push depth.
> - * FIXME: passing obj to visit_end_struct would make this easier */
> + /* Nothing to allocate on the virtual walk */
> assert(qcv->depth);
> - qcv->depth++;
> return;
> }
>
Why can we elide qcv->depth++? Do the assert(qcv->qdepth) still hold?
> @@ -41,11 +38,13 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
> qcv->depth++;
> }
>
> -static void qapi_clone_end(Visitor *v)
> +static void qapi_clone_end(Visitor *v, void **obj)
> {
> QapiCloneVisitor *qcv = to_qcv(v);
> assert(qcv->depth);
> - qcv->depth--;
> + if (obj) {
> + qcv->depth--;
> + }
> }
>
> static void qapi_clone_start_list(Visitor *v, const char *name,
> diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
> index cd68b55..9391dea 100644
> --- a/qapi/qapi-dealloc-visitor.c
> +++ b/qapi/qapi-dealloc-visitor.c
> @@ -19,53 +19,18 @@
> #include "qapi/qmp/types.h"
> #include "qapi/visitor-impl.h"
>
> -typedef struct StackEntry
> -{
> - void *value;
> - QTAILQ_ENTRY(StackEntry) node;
> -} StackEntry;
> -
> struct QapiDeallocVisitor
> {
> Visitor visitor;
> - QTAILQ_HEAD(, StackEntry) stack;
> };
>
> -static QapiDeallocVisitor *to_qov(Visitor *v)
> -{
> - return container_of(v, QapiDeallocVisitor, visitor);
> -}
> -
> -static void qapi_dealloc_push(QapiDeallocVisitor *qov, void *value)
> -{
> - StackEntry *e = g_malloc0(sizeof(*e));
> -
> - e->value = value;
> -
> - QTAILQ_INSERT_HEAD(&qov->stack, e, node);
> -}
> -
> -static void *qapi_dealloc_pop(QapiDeallocVisitor *qov)
> -{
> - StackEntry *e = QTAILQ_FIRST(&qov->stack);
> - QObject *value;
> - QTAILQ_REMOVE(&qov->stack, e, node);
> - value = e->value;
> - g_free(e);
> - return value;
> -}
> -
> static void qapi_dealloc_start_struct(Visitor *v, const char *name, void **obj,
> size_t unused, Error **errp)
> {
> - QapiDeallocVisitor *qov = to_qov(v);
> - qapi_dealloc_push(qov, obj);
> }
>
> -static void qapi_dealloc_end_struct(Visitor *v)
> +static void qapi_dealloc_end_struct(Visitor *v, void **obj)
> {
> - QapiDeallocVisitor *qov = to_qov(v);
> - void **obj = qapi_dealloc_pop(qov);
> if (obj) {
> g_free(*obj);
> }
> @@ -75,14 +40,10 @@ static void qapi_dealloc_start_alternate(Visitor *v, const char *name,
> GenericAlternate **obj, size_t size,
> bool promote_int, Error **errp)
> {
> - QapiDeallocVisitor *qov = to_qov(v);
> - qapi_dealloc_push(qov, obj);
> }
>
> -static void qapi_dealloc_end_alternate(Visitor *v)
> +static void qapi_dealloc_end_alternate(Visitor *v, void **obj)
> {
> - QapiDeallocVisitor *qov = to_qov(v);
> - void **obj = qapi_dealloc_pop(qov);
> if (obj) {
> g_free(*obj);
> }
> @@ -102,7 +63,7 @@ static GenericList *qapi_dealloc_next_list(Visitor *v, GenericList *tail,
> return next;
> }
>
> -static void qapi_dealloc_end_list(Visitor *v)
> +static void qapi_dealloc_end_list(Visitor *v, void **obj)
> {
> }
>
> @@ -178,7 +139,5 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
> v->visitor.type_any = qapi_dealloc_type_anything;
> v->visitor.type_null = qapi_dealloc_type_null;
>
> - QTAILQ_INIT(&v->stack);
> -
> return v;
> }
Much nicer.
> diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
> index aea90a1..84f32fc 100644
> --- a/qapi/qmp-input-visitor.c
> +++ b/qapi/qmp-input-visitor.c
> @@ -145,7 +145,7 @@ static void qmp_input_check_struct(Visitor *v, Error **errp)
> }
> }
>
> -static void qmp_input_pop(Visitor *v)
> +static void qmp_input_pop(Visitor *v, void **obj)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
You could assert @obj matches tos->obj. Same for the other visitors
that still need a stack.
> diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
> index ef9f62c..4bdc00d 100644
> --- a/qapi/qmp-output-visitor.c
> +++ b/qapi/qmp-output-visitor.c
> @@ -107,7 +107,7 @@ static void qmp_output_start_struct(Visitor *v, const char *name, void **obj,
> qmp_output_push(qov, dict);
> }
>
> -static void qmp_output_end_struct(Visitor *v)
> +static void qmp_output_end_struct(Visitor *v, void **obj)
> {
> QmpOutputVisitor *qov = to_qov(v);
> QObject *value = qmp_output_pop(qov);
> @@ -131,7 +131,7 @@ static GenericList *qmp_output_next_list(Visitor *v, GenericList *tail,
> return tail->next;
> }
>
> -static void qmp_output_end_list(Visitor *v)
> +static void qmp_output_end_list(Visitor *v, void **end)
> {
> QmpOutputVisitor *qov = to_qov(v);
> QObject *value = qmp_output_pop(qov);
> diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
> index 30b5879..1c66d6a 100644
> --- a/qapi/string-input-visitor.c
> +++ b/qapi/string-input-visitor.c
> @@ -181,7 +181,7 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
> return tail->next;
> }
>
> -static void end_list(Visitor *v)
> +static void end_list(Visitor *v, void **obj)
> {
> }
>
> diff --git a/qapi/string-output-visitor.c b/qapi/string-output-visitor.c
> index d013196..1504a89 100644
> --- a/qapi/string-output-visitor.c
> +++ b/qapi/string-output-visitor.c
> @@ -291,7 +291,7 @@ static GenericList *next_list(Visitor *v, GenericList *tail, size_t size)
> return ret;
> }
>
> -static void end_list(Visitor *v)
> +static void end_list(Visitor *v, void **obj)
> {
> StringOutputVisitor *sov = to_sov(v);
>
> diff --git a/qom/object.c b/qom/object.c
> index 3bc8a00..1562f7e 100644
> --- a/qom/object.c
> +++ b/qom/object.c
> @@ -2038,7 +2038,7 @@ static void property_get_tm(Object *obj, Visitor *v, const char *name,
> }
> visit_check_struct(v, &err);
> out_end:
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> out:
> error_propagate(errp, err);
>
> diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
> index 51e62e2..926ec68 100644
> --- a/qom/object_interfaces.c
> +++ b/qom/object_interfaces.c
> @@ -71,7 +71,7 @@ Object *user_creatable_add(const QDict *qdict,
> obj = user_creatable_add_type(type, id, pdict, v, &local_err);
>
> out_visit:
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
>
> out:
> QDECREF(pdict);
> @@ -127,7 +127,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
> if (!local_err) {
> visit_check_struct(v, &local_err);
> }
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> if (local_err) {
> goto out;
> }
> diff --git a/tests/test-json-output-visitor.c b/tests/test-json-output-visitor.c
> index 5bf6fc5..5e8a5c8 100644
> --- a/tests/test-json-output-visitor.c
> +++ b/tests/test-json-output-visitor.c
> @@ -324,7 +324,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
> qobj = QOBJECT(qint_from_int(-42));
> visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
> visit_type_any(data->ov, NULL, &qobj, &error_abort);
> - visit_end_list(data->ov);
> + visit_end_list(data->ov, NULL);
> out = json_output_get_string(data->jov);
> if (*pretty) {
> g_assert_cmpstr(out, ==, "[\n -42\n]");
> @@ -341,7 +341,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
> qobj = QOBJECT(qdict);
> visit_start_list(data->ov, NULL, NULL, 0, &error_abort);
> visit_type_any(data->ov, NULL, &qobj, &error_abort);
> - visit_end_list(data->ov);
> + visit_end_list(data->ov, NULL);
> qobject_decref(qobj);
> out = json_output_get_string(data->jov);
> if (*pretty) {
> diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
> index d02306c..56ee943 100644
> --- a/tests/test-qmp-input-visitor.c
> +++ b/tests/test-qmp-input-visitor.c
> @@ -305,7 +305,7 @@ static void test_visitor_in_null(TestInputVisitorData *data,
> visit_type_null(v, "b", &err);
> error_free_or_abort(&err);
> visit_check_struct(v, &error_abort);
> - visit_end_struct(v);
> + visit_end_struct(v, NULL);
> }
>
> static void test_visitor_in_union_flat(TestInputVisitorData *data,
> diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
> index 050d65b..e2aa494 100644
> --- a/tests/test-qmp-output-visitor.c
> +++ b/tests/test-qmp-output-visitor.c
> @@ -494,7 +494,7 @@ static void test_visitor_out_null(TestOutputVisitorData *data,
> visit_start_struct(data->ov, NULL, NULL, 0, &error_abort);
> visit_type_null(data->ov, "a", &error_abort);
> visit_check_struct(data->ov, &error_abort);
> - visit_end_struct(data->ov);
> + visit_end_struct(data->ov, NULL);
> arg = qmp_output_get_qobject(data->qov);
> g_assert(qobject_type(arg) == QTYPE_QDICT);
> qdict = qobject_to_qdict(arg);
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 92fbb0e..48436aa 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -942,7 +942,7 @@ Example:
> }
> visit_check_struct(v, &err);
> out_obj:
> - visit_end_struct(v);
> + visit_end_struct(v, (void **)obj);
> if (err && visit_is_input(v)) {
> qapi_free_UserDefOne(*obj);
> *obj = NULL;
> @@ -970,7 +970,7 @@ Example:
> }
> }
>
> - visit_end_list(v);
> + visit_end_list(v, (void **)obj);
> if (err && visit_is_input(v)) {
> qapi_free_UserDefOneList(*obj);
> *obj = NULL;
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_*
2016-05-02 18:20 ` Markus Armbruster
@ 2016-05-02 19:31 ` Eric Blake
2016-05-03 11:53 ` Markus Armbruster
0 siblings, 1 reply; 78+ messages in thread
From: Eric Blake @ 2016-05-02 19:31 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Kevin Wolf, Alexander Graf, famz,
open list:Block layer core, Michael S. Tsirkin, Juan Quintela,
Michael Roth, open list:sPAPR, Amit Shah, Max Reitz,
Andreas Färber, David Gibson
[-- Attachment #1: Type: text/plain, Size: 4332 bytes --]
On 05/02/2016 12:20 PM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Rather than making the dealloc visitor track of stack of pointers
>> remembered during visit_start_* in order to free them during
>> visit_end_*, it's a lot easier to just make all callers pass the
>> same pointer to visit_end_*. The generated code has access to the
>> same pointer, while all other users are doing virtual walks and
>> can pass NULL. The dealloc visitor is then greatly simplified.
>>
>> The clone visitor also gets a minor simplification of not having
>> to track quite as much depth.
>
> Do this before adding the clone visitor, to reduce churn?
I could. I first thought of it after, and kept it there as a
justification for keeping it (multiple visitors benefit), but even if
rebased earlier, the commit message can still mention that it will make
the clone visitor easier.
>> @@ -59,7 +59,7 @@ struct Visitor
>> GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);
>>
>> /* Must be set */
>> - void (*end_list)(Visitor *v);
>> + void (*end_list)(Visitor *v, void **list);
>
> Sure you want void ** and not GenericList **?
Yes. There are only two callers: dtc code passes NULL (where the type
doesn't matter), and generated visit_type_FOOList() has to already cast
to GenericList() during visit_start_list(); accepting void** here
instead of GenericList** removes the need for a second instance of that
cast.
>
>>
>> /* Must be set by input and dealloc visitors to visit alternates;
>> * optional for output visitors. */
>> @@ -68,7 +68,7 @@ struct Visitor
>> bool promote_int, Error **errp);
>>
>> /* Optional, needed for dealloc visitor */
>> - void (*end_alternate)(Visitor *v);
>> + void (*end_alternate)(Visitor *v, void **obj);
>
> Sure you want void ** and not GenericAlternate **?
Only one caller: generated code. Same story that we already have to cast
during visit_start_alternate(), so using void** here avoids the need for
a second cast.
Oh, and the clone visitor was easier to write with a single function
that takes void** for all three visit_end() implementations (whereas I'd
have to write three functions identical except for signature otherwise).
>> +++ b/qapi/qapi-clone-visitor.c
>> @@ -29,11 +29,8 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>> QapiCloneVisitor *qcv = to_qcv(v);
>>
>> if (!obj) {
>> - /* Nothing to allocate on the virtual walk during an
>> - * alternate, but we still have to push depth.
>> - * FIXME: passing obj to visit_end_struct would make this easier */
>> + /* Nothing to allocate on the virtual walk */
>> assert(qcv->depth);
>> - qcv->depth++;
>> return;
>> }
>>
>
> Why can we elide qcv->depth++? Do the assert(qcv->qdepth) still hold?
Yes, the assertion still holds: we can only reach this code underneath
visit_start_alternate(), ...
>
>> @@ -41,11 +38,13 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>> qcv->depth++;
>> }
>>
>> -static void qapi_clone_end(Visitor *v)
>> +static void qapi_clone_end(Visitor *v, void **obj)
>> {
>> QapiCloneVisitor *qcv = to_qcv(v);
>> assert(qcv->depth);
>> - qcv->depth--;
>> + if (obj) {
>> + qcv->depth--;
>> + }
...and this is the matching elision of the depth manipulations.
>> +++ b/qapi/qmp-input-visitor.c
>> @@ -145,7 +145,7 @@ static void qmp_input_check_struct(Visitor *v, Error **errp)
>> }
>> }
>>
>> -static void qmp_input_pop(Visitor *v)
>> +static void qmp_input_pop(Visitor *v, void **obj)
>> {
>> QmpInputVisitor *qiv = to_qiv(v);
>> StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
>
> You could assert @obj matches tos->obj. Same for the other visitors
> that still need a stack.
And brings me back to my question of whether qapi-visit-core.c should
maintain its own stack for asserting these types of sanity checks for
ALL callers, even when the visitor itself doesn't need a stack.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_*
2016-05-02 19:31 ` Eric Blake
@ 2016-05-03 11:53 ` Markus Armbruster
2016-05-03 12:41 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Markus Armbruster @ 2016-05-03 11:53 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, famz, Michael Roth, open list:Block layer core,
Michael S. Tsirkin, Juan Quintela, qemu-devel, Alexander Graf,
open list:sPAPR, Amit Shah, Max Reitz, Andreas Färber,
David Gibson
Eric Blake <eblake@redhat.com> writes:
> On 05/02/2016 12:20 PM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Rather than making the dealloc visitor track of stack of pointers
>>> remembered during visit_start_* in order to free them during
>>> visit_end_*, it's a lot easier to just make all callers pass the
>>> same pointer to visit_end_*. The generated code has access to the
>>> same pointer, while all other users are doing virtual walks and
>>> can pass NULL. The dealloc visitor is then greatly simplified.
>>>
>>> The clone visitor also gets a minor simplification of not having
>>> to track quite as much depth.
>>
>> Do this before adding the clone visitor, to reduce churn?
>
> I could. I first thought of it after, and kept it there as a
> justification for keeping it (multiple visitors benefit), but even if
> rebased earlier, the commit message can still mention that it will make
> the clone visitor easier.
Let's do it early.
>>> @@ -59,7 +59,7 @@ struct Visitor
>>> GenericList *(*next_list)(Visitor *v, GenericList *tail, size_t size);
>>>
>>> /* Must be set */
>>> - void (*end_list)(Visitor *v);
>>> + void (*end_list)(Visitor *v, void **list);
>>
>> Sure you want void ** and not GenericList **?
>
> Yes. There are only two callers: dtc code passes NULL (where the type
> doesn't matter), and generated visit_type_FOOList() has to already cast
> to GenericList() during visit_start_list(); accepting void** here
> instead of GenericList** removes the need for a second instance of that
> cast.
Fewer casts is good, but symmetry is also good.
>>>
>>> /* Must be set by input and dealloc visitors to visit alternates;
>>> * optional for output visitors. */
>>> @@ -68,7 +68,7 @@ struct Visitor
>>> bool promote_int, Error **errp);
>>>
>>> /* Optional, needed for dealloc visitor */
>>> - void (*end_alternate)(Visitor *v);
>>> + void (*end_alternate)(Visitor *v, void **obj);
>>
>> Sure you want void ** and not GenericAlternate **?
>
> Only one caller: generated code. Same story that we already have to cast
> during visit_start_alternate(), so using void** here avoids the need for
> a second cast.
>
> Oh, and the clone visitor was easier to write with a single function
> that takes void** for all three visit_end() implementations (whereas I'd
> have to write three functions identical except for signature otherwise).
Okay, I can buy this one.
>>> +++ b/qapi/qapi-clone-visitor.c
>>> @@ -29,11 +29,8 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>>> QapiCloneVisitor *qcv = to_qcv(v);
>>>
>>> if (!obj) {
>>> - /* Nothing to allocate on the virtual walk during an
>>> - * alternate, but we still have to push depth.
>>> - * FIXME: passing obj to visit_end_struct would make this easier */
>>> + /* Nothing to allocate on the virtual walk */
>>> assert(qcv->depth);
>>> - qcv->depth++;
>>> return;
>>> }
>>>
>>
>> Why can we elide qcv->depth++? Do the assert(qcv->qdepth) still hold?
>
> Yes, the assertion still holds: we can only reach this code underneath
> visit_start_alternate(), ...
>
>>
>>> @@ -41,11 +38,13 @@ static void qapi_clone_start_struct(Visitor *v, const char *name, void **obj,
>>> qcv->depth++;
>>> }
>>>
>>> -static void qapi_clone_end(Visitor *v)
>>> +static void qapi_clone_end(Visitor *v, void **obj)
>>> {
>>> QapiCloneVisitor *qcv = to_qcv(v);
>>> assert(qcv->depth);
>>> - qcv->depth--;
>>> + if (obj) {
>>> + qcv->depth--;
>>> + }
>
> ...and this is the matching elision of the depth manipulations.
Okay, I'm confused.
Consider BlockdevRef, defined as
{ 'alternate': 'BlockdevRef',
'data': { 'definition': 'BlockdevOptions',
'reference': 'str' } }
where BlockdevOptions is a (flat) union. Let's clone a BlockdevRef
holding a str. Sequence of calls:
qapi_BlockdevRef_clone(src)
qapi_clone_visitor_new(src)
// qcv->depth is now 0
visit_type_BlockdevRef(v, NULL, &dst, &error_abort)
visit_start_alternate(v, NULL, &dst,
sizeof(BlockdevRef), true, &error_abort)
qapi_clone_start_alternate(v, NULL, &dst,
sizeof(BlockdevRef), true, &error_abort)
qapi_clone_start_struct(v, NULL, &dst,
sizeof(BlockdevRef), &error_abort)
// does not increment qcv->depth
visit_type_str(v, NULL, &dst->u.references, &error_abort)
qapi_clone_type_str(v, NULL, &dst->u.references, &error_abort)
assert(qcv->depth) // why does this hold?
>>> +++ b/qapi/qmp-input-visitor.c
>>> @@ -145,7 +145,7 @@ static void qmp_input_check_struct(Visitor *v, Error **errp)
>>> }
>>> }
>>>
>>> -static void qmp_input_pop(Visitor *v)
>>> +static void qmp_input_pop(Visitor *v, void **obj)
>>> {
>>> QmpInputVisitor *qiv = to_qiv(v);
>>> StackObject *tos = &qiv->stack[qiv->nb_stack - 1];
>>
>> You could assert @obj matches tos->obj. Same for the other visitors
>> that still need a stack.
>
> And brings me back to my question of whether qapi-visit-core.c should
> maintain its own stack for asserting these types of sanity checks for
> ALL callers, even when the visitor itself doesn't need a stack.
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_*
2016-05-03 11:53 ` Markus Armbruster
@ 2016-05-03 12:41 ` Eric Blake
0 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-05-03 12:41 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, famz, Michael Roth, open list:Block layer core,
Michael S. Tsirkin, Juan Quintela, qemu-devel, Alexander Graf,
open list:sPAPR, Amit Shah, Max Reitz, Andreas Färber,
David Gibson
[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]
On 05/03/2016 05:53 AM, Markus Armbruster wrote:
> Okay, I'm confused.
>
> Consider BlockdevRef, defined as
>
> { 'alternate': 'BlockdevRef',
> 'data': { 'definition': 'BlockdevOptions',
> 'reference': 'str' } }
>
> where BlockdevOptions is a (flat) union. Let's clone a BlockdevRef
> holding a str. Sequence of calls:
>
> qapi_BlockdevRef_clone(src)
> qapi_clone_visitor_new(src)
> // qcv->depth is now 0
> visit_type_BlockdevRef(v, NULL, &dst, &error_abort)
> visit_start_alternate(v, NULL, &dst,
> sizeof(BlockdevRef), true, &error_abort)
> qapi_clone_start_alternate(v, NULL, &dst,
> sizeof(BlockdevRef), true, &error_abort)
so far, so good,
> qapi_clone_start_struct(v, NULL, &dst,
> sizeof(BlockdevRef), &error_abort)
Not reached if it holds a string. qapi_clone_start_alternate() sets
dst->type to QTYPE_QSTRING, but qapi_clone_start_struct() is only used
for QTYPE_QDICT.
> // does not increment qcv->depth
> visit_type_str(v, NULL, &dst->u.references, &error_abort)
> qapi_clone_type_str(v, NULL, &dst->u.references, &error_abort)
> assert(qcv->depth) // why does this hold?
Either we call:
visit_start_alternate()
visit_start_struct(NULL)
visit_end_struct()
visit_end_alternate()
or we elide the inner struct visit because the alternate's branch is
scalar. Either way, we only need +1 to the depth over the entire
alternate, because we only need 1 level of g_memdup() at the time we
first enter the alternate.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors
2016-04-29 4:23 [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Eric Blake
` (17 preceding siblings ...)
2016-04-29 4:23 ` [Qemu-devel] [PATCH v3 18/18] qapi: Add parameter to visit_end_* Eric Blake
@ 2016-05-09 8:50 ` Paolo Bonzini
2016-05-09 9:29 ` Paolo Bonzini
18 siblings, 1 reply; 78+ messages in thread
From: Paolo Bonzini @ 2016-05-09 8:50 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: famz, armbru
On 29/04/2016 06:23, Eric Blake wrote:
> Note that the series has a mutually exclusive choice: either
> patch 8, or patches 10-11. I still haven't gotten any feedback
> on which choice seems nicer.
I vote for patch 8.
Paolo
^ permalink raw reply [flat|nested] 78+ messages in thread* Re: [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors
2016-05-09 8:50 ` [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors Paolo Bonzini
@ 2016-05-09 9:29 ` Paolo Bonzini
2016-05-09 14:52 ` Eric Blake
0 siblings, 1 reply; 78+ messages in thread
From: Paolo Bonzini @ 2016-05-09 9:29 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: famz, armbru
On 09/05/2016 10:50, Paolo Bonzini wrote:
>
>
> On 29/04/2016 06:23, Eric Blake wrote:
>> Note that the series has a mutually exclusive choice: either
>> patch 8, or patches 10-11. I still haven't gotten any feedback
>> on which choice seems nicer.
>
> I vote for patch 8.
Nice first email after 3 weeks, I meant exactly the opposite. :) Having
two APIs for the same thing is unnecessary.
However, including all of patches 8/10/11 may also make sense (and
leaving out the revert). Then qjson is migrated first, then its users
move away from qjson, then qjson is removed.
Paolo
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [Qemu-devel] [PATCH v3 00/18] Add qapi-to-JSON and clone visitors
2016-05-09 9:29 ` Paolo Bonzini
@ 2016-05-09 14:52 ` Eric Blake
0 siblings, 0 replies; 78+ messages in thread
From: Eric Blake @ 2016-05-09 14:52 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: famz, armbru
[-- Attachment #1: Type: text/plain, Size: 1140 bytes --]
On 05/09/2016 03:29 AM, Paolo Bonzini wrote:
>
>
> On 09/05/2016 10:50, Paolo Bonzini wrote:
>>
>>
>> On 29/04/2016 06:23, Eric Blake wrote:
>>> Note that the series has a mutually exclusive choice: either
>>> patch 8, or patches 10-11. I still haven't gotten any feedback
>>> on which choice seems nicer.
>>
>> I vote for patch 8.
>
> Nice first email after 3 weeks, I meant exactly the opposite. :) Having
> two APIs for the same thing is unnecessary.
>
> However, including all of patches 8/10/11 may also make sense (and
> leaving out the revert). Then qjson is migrated first, then its users
> move away from qjson, then qjson is removed.
You'll want to catch up on the rest of the thread - for now, v4 of this
series will drop 8-11 entirely, because Dave was unwilling to accept any
new assertions on the migration path, so we instead moved qjson.c under
migration/ and added explicit comments why it is duplicating a
rudimentary JSON output engine but with no error checking or assertions.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread