* [Qemu-devel] [PULL 00/26] Block layer patches
@ 2018-06-15 14:20 Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 01/26] qemu-img: Fix assert when mapping unaligned raw file Kevin Wolf
` (26 more replies)
0 siblings, 27 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit 91fe7a376ad46e3cc5e82d418aad22173c948a3c:
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2018-06-15 11:41:44 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 6266e900b8083945cb766b45c124fb3c42932cb3:
block: Remove dead deprecation warning code (2018-06-15 14:49:44 +0200)
----------------------------------------------------------------
Block layer patches:
- Fix options that work only with -drive or -blockdev, but not with
both, because of QDict type confusion
- rbd: Add options 'auth-client-required' and 'key-secret'
- Remove deprecated -drive options serial/addr/cyls/heads/secs/trans
- rbd, iscsi: Remove deprecated 'filename' option
- Fix 'qemu-img map' crash with unaligned image size
- Improve QMP documentation for jobs
----------------------------------------------------------------
Eric Blake (2):
qemu-img: Fix assert when mapping unaligned raw file
iotests: Add test 221 to catch qemu-img map regression
John Snow (2):
jobs: fix stale wording
jobs: fix verb references in docs
Kevin Wolf (4):
block: Remove deprecated -drive geometry options
block: Remove deprecated -drive option addr
block: Remove deprecated -drive option serial
block: Remove dead deprecation warning code
Markus Armbruster (17):
rbd: Drop deprecated -drive parameter "filename"
iscsi: Drop deprecated -drive parameter "filename"
qobject: Move block-specific qdict code to block-qdict.c
block: Fix -blockdev for certain non-string scalars
block: Fix -drive for certain non-string scalars
block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts()
block: Factor out qobject_input_visitor_new_flat_confused()
block: Make remaining uses of qobject input visitor more robust
block-qdict: Simplify qdict_flatten_qdict()
block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist()
block-qdict: Clean up qdict_crumple() a bit
block-qdict: Simplify qdict_is_list() some
check-block-qdict: Rename qdict_flatten()'s variables for clarity
check-block-qdict: Cover flattening of empty lists and dictionaries
block: Fix -blockdev / blockdev-add for empty objects and arrays
rbd: New parameter auth-client-required
rbd: New parameter key-secret
Max Reitz (1):
block: Add block-specific QDict header
qapi/block-core.json | 19 ++
qapi/job.json | 23 +-
include/block/qdict.h | 34 +++
include/hw/block/block.h | 1 -
include/qapi/qmp/qdict.h | 17 --
include/sysemu/blockdev.h | 3 -
block.c | 1 +
block/block-backend.c | 1 -
block/crypto.c | 12 +-
block/gluster.c | 1 +
block/iscsi.c | 24 +-
block/nbd.c | 16 +-
block/nfs.c | 8 +-
block/parallels.c | 11 +-
block/qcow.c | 11 +-
block/qcow2.c | 11 +-
block/qed.c | 11 +-
block/quorum.c | 1 +
block/rbd.c | 85 +++---
block/sheepdog.c | 23 +-
block/snapshot.c | 1 +
block/ssh.c | 16 +-
block/vdi.c | 8 +-
block/vhdx.c | 11 +-
block/vpc.c | 11 +-
block/vvfat.c | 1 +
block/vxhs.c | 1 +
blockdev.c | 111 +------
device-hotplug.c | 4 -
hw/block/block.c | 27 --
hw/block/nvme.c | 1 -
hw/block/virtio-blk.c | 1 -
hw/ide/qdev.c | 1 -
hw/scsi/scsi-disk.c | 1 -
hw/usb/dev-storage.c | 1 -
qemu-img.c | 2 +-
qobject/block-qdict.c | 722 +++++++++++++++++++++++++++++++++++++++++++++
qobject/qdict.c | 628 ---------------------------------------
tests/ahci-test.c | 6 +-
tests/check-block-qdict.c | 690 +++++++++++++++++++++++++++++++++++++++++++
tests/check-qdict.c | 641 ----------------------------------------
tests/check-qobject.c | 1 +
tests/hd-geo-test.c | 37 +--
tests/ide-test.c | 8 +-
tests/test-replication.c | 1 +
util/qemu-config.c | 1 +
MAINTAINERS | 2 +
hmp-commands.hx | 1 -
qemu-doc.texi | 15 -
qemu-options.hx | 14 +-
qobject/Makefile.objs | 1 +
tests/Makefile.include | 4 +
tests/qemu-iotests/221 | 60 ++++
tests/qemu-iotests/221.out | 16 +
tests/qemu-iotests/group | 1 +
55 files changed, 1692 insertions(+), 1668 deletions(-)
create mode 100644 include/block/qdict.h
create mode 100644 qobject/block-qdict.c
create mode 100644 tests/check-block-qdict.c
create mode 100755 tests/qemu-iotests/221
create mode 100644 tests/qemu-iotests/221.out
^ permalink raw reply [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 01/26] qemu-img: Fix assert when mapping unaligned raw file
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 02/26] iotests: Add test 221 to catch qemu-img map regression Kevin Wolf
` (25 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Eric Blake <eblake@redhat.com>
Commit a290f085 exposed a latent bug in qemu-img map introduced
during the conversion of block status to be byte-based. Earlier in
commit 5e344dd8, the internal interface get_block_status() switched
to take byte-based parameters, but still called a sector-based
block layer function; as such, rounding was added in the lone
caller to obey the contract. However, commit 237d78f8 changed
get_block_status() to truly be byte-based, at which point rounding
to sector boundaries can result in calling bdrv_block_status() with
'bytes == 0' (a coding error) when the boundary between data and a
hole falls mid-sector (true for the past-EOF implicit hole present
in POSIX files). Fix things by removing the rounding that is now
no longer necessary.
See also https://bugzilla.redhat.com/1589738
Fixes: 237d78f8
Reported-by: Dan Kenigsberg <danken@redhat.com>
Reported-by: Nir Soffer <nsoffer@redhat.com>
Reported-by: Maor Lipchuk <mlipchuk@redhat.com>
CC: qemu-stable@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qemu-img.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/qemu-img.c b/qemu-img.c
index 1dcdd47254..e1a506f7f6 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2906,7 +2906,7 @@ static int img_map(int argc, char **argv)
int64_t n;
/* Probe up to 1 GiB at a time. */
- n = QEMU_ALIGN_DOWN(MIN(1 << 30, length - offset), BDRV_SECTOR_SIZE);
+ n = MIN(1 << 30, length - offset);
ret = get_block_status(bs, offset, n, &next);
if (ret < 0) {
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 02/26] iotests: Add test 221 to catch qemu-img map regression
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 01/26] qemu-img: Fix assert when mapping unaligned raw file Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 03/26] jobs: fix stale wording Kevin Wolf
` (24 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Eric Blake <eblake@redhat.com>
Although qemu-img creates aligned files (by rounding up), it
must also gracefully handle files that are not sector-aligned.
Test that the bug fixed in the previous patch does not recur.
It's a bit annoying that we can see the (implicit) hole past
the end of the file on to the next sector boundary, so if we
ever reach the point where we report a byte-accurate size rather
than our current behavior of always rounding up, this test will
probably need a slight modification.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/221 | 60 ++++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/221.out | 16 +++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 77 insertions(+)
create mode 100755 tests/qemu-iotests/221
create mode 100644 tests/qemu-iotests/221.out
diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221
new file mode 100755
index 0000000000..41c4e4bdf8
--- /dev/null
+++ b/tests/qemu-iotests/221
@@ -0,0 +1,60 @@
+#!/bin/bash
+#
+# Test qemu-img vs. unaligned images
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt raw
+_supported_proto file
+_supported_os Linux
+
+echo
+echo "=== Check mapping of unaligned raw image ==="
+echo
+
+_make_test_img 43009 # qemu-img create rounds size up
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+truncate --size=43009 "$TEST_IMG" # so we resize it and check again
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+$QEMU_IO -c 'w 43008 1' "$TEST_IMG" | _filter_qemu_io # writing also rounds up
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+truncate --size=43009 "$TEST_IMG" # so we resize it and check again
+$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map
+
+# success, all done
+echo '*** done'
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
new file mode 100644
index 0000000000..a9c0190aad
--- /dev/null
+++ b/tests/qemu-iotests/221.out
@@ -0,0 +1,16 @@
+QA output created by 221
+
+=== Check mapping of unaligned raw image ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=43009
+[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+wrote 1/1 bytes at offset 43008
+1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 0914c922d7..937a3d0a4d 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -218,3 +218,4 @@
217 rw auto quick
218 rw auto quick
219 rw auto
+221 rw auto quick
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 03/26] jobs: fix stale wording
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 01/26] qemu-img: Fix assert when mapping unaligned raw file Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 02/26] iotests: Add test 221 to catch qemu-img map regression Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 04/26] jobs: fix verb references in docs Kevin Wolf
` (23 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: John Snow <jsnow@redhat.com>
During the design for manual completion, we decided not to use the
"manual" property as a shorthand for both auto-dismiss and auto-finalize.
Fix the wording.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qapi/job.json | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/qapi/job.json b/qapi/job.json
index 17d10037c4..226443594b 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -50,16 +50,17 @@
# the last job in a transaction.
#
# @pending: The job has finished its work, but has finalization steps that it
-# needs to make prior to completing. These changes may require
-# manual intervention by the management process if manual was set
-# to true. These changes may still fail.
+# needs to make prior to completing. These changes will require
+# manual intervention via @job-finalize if auto-finalize was set to
+# false. These pending changes may still fail.
#
# @aborting: The job is in the process of being aborted, and will finish with
# an error. The job will afterwards report that it is @concluded.
# This status may not be visible to the management process.
#
-# @concluded: The job has finished all work. If manual was set to true, the job
-# will remain in the query list until it is dismissed.
+# @concluded: The job has finished all work. If auto-dismiss was set to false,
+# the job will remain in the query list until it is dismissed via
+# @job-dismiss.
#
# @null: The job is in the process of being dismantled. This state should not
# ever be visible externally.
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 04/26] jobs: fix verb references in docs
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (2 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 03/26] jobs: fix stale wording Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 05/26] rbd: Drop deprecated -drive parameter "filename" Kevin Wolf
` (22 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: John Snow <jsnow@redhat.com>
These point to the job versions now, not the blockjob versions which
don't really exist anymore.
Except set-speed, which does. It sticks out like a sore thumb. This
patch doesn't fix that, but it doesn't make it any worse, either.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qapi/job.json | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/qapi/job.json b/qapi/job.json
index 226443594b..9d074eb8d2 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -76,19 +76,19 @@
#
# Represents command verbs that can be applied to a job.
#
-# @cancel: see @block-job-cancel
+# @cancel: see @job-cancel
#
-# @pause: see @block-job-pause
+# @pause: see @job-pause
#
-# @resume: see @block-job-resume
+# @resume: see @job-resume
#
# @set-speed: see @block-job-set-speed
#
-# @complete: see @block-job-complete
+# @complete: see @job-complete
#
-# @dismiss: see @block-job-dismiss
+# @dismiss: see @job-dismiss
#
-# @finalize: see @block-job-finalize
+# @finalize: see @job-finalize
#
# Since: 2.12
##
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 05/26] rbd: Drop deprecated -drive parameter "filename"
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (3 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 04/26] jobs: fix verb references in docs Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 06/26] iscsi: " Kevin Wolf
` (21 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Parameter "filename" is deprecated since commit 91589d9e5ca, v2.10.0.
Time to get rid of it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/rbd.c | 16 ----------------
1 file changed, 16 deletions(-)
diff --git a/block/rbd.c b/block/rbd.c
index a16431e267..40c6e4185f 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -632,25 +632,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
QObject *crumpled = NULL;
const QDictEntry *e;
Error *local_err = NULL;
- const char *filename;
char *keypairs, *secretid;
int r;
- /* If we are given a filename, parse the filename, with precedence given to
- * filename encoded options */
- filename = qdict_get_try_str(options, "filename");
- if (filename) {
- warn_report("'filename' option specified. "
- "This is an unsupported option, and may be deprecated "
- "in the future");
- qemu_rbd_parse_filename(filename, options, &local_err);
- qdict_del(options, "filename");
- if (local_err) {
- error_propagate(errp, local_err);
- return -EINVAL;
- }
- }
-
keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs"));
if (keypairs) {
qdict_del(options, "=keyvalue-pairs");
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 06/26] iscsi: Drop deprecated -drive parameter "filename"
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (4 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 05/26] rbd: Drop deprecated -drive parameter "filename" Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 07/26] block: Add block-specific QDict header Kevin Wolf
` (20 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Parameter "filename" is deprecated since commit 5c3ad1a6a8f, v2.10.0.
Time to get rid of it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/iscsi.c | 23 ++---------------------
1 file changed, 2 insertions(+), 21 deletions(-)
diff --git a/block/iscsi.c b/block/iscsi.c
index c2fbd8a8aa..7e3ea72bd2 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -1713,10 +1713,6 @@ static QemuOptsList runtime_opts = {
.name = "timeout",
.type = QEMU_OPT_NUMBER,
},
- {
- .name = "filename",
- .type = QEMU_OPT_STRING,
- },
{ /* end of list */ }
},
};
@@ -1756,27 +1752,12 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags,
char *initiator_name = NULL;
QemuOpts *opts;
Error *local_err = NULL;
- const char *transport_name, *portal, *target, *filename;
+ const char *transport_name, *portal, *target;
#if LIBISCSI_API_VERSION >= (20160603)
enum iscsi_transport_type transport;
#endif
int i, ret = 0, timeout = 0, lun;
- /* If we are given a filename, parse the filename, with precedence given to
- * filename encoded options */
- filename = qdict_get_try_str(options, "filename");
- if (filename) {
- warn_report("'filename' option specified. "
- "This is an unsupported option, and may be deprecated "
- "in the future");
- iscsi_parse_filename(filename, options, &local_err);
- if (local_err) {
- ret = -EINVAL;
- error_propagate(errp, local_err);
- goto exit;
- }
- }
-
opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
qemu_opts_absorb_qdict(opts, options, &local_err);
if (local_err) {
@@ -2006,7 +1987,7 @@ out:
}
memset(iscsilun, 0, sizeof(IscsiLun));
}
-exit:
+
return ret;
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 07/26] block: Add block-specific QDict header
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (5 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 06/26] iscsi: " Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c Kevin Wolf
` (19 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Max Reitz <mreitz@redhat.com>
There are numerous QDict functions that have been introduced for and are
used only by the block layer. Move their declarations into an own
header file to reflect that.
While qdict_extract_subqdict() is in fact used outside of the block
layer (in util/qemu-config.c), it is still a function related very
closely to how the block layer works with nested QDicts, namely by
sometimes flattening them. Therefore, its declaration is put into this
header as well and util/qemu-config.c includes it with a comment stating
exactly which function it needs.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20180509165530.29561-7-mreitz@redhat.com>
[Copyright note tweaked, superfluous includes dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/block/qdict.h | 32 ++++++++++++++++++++++++++++++++
include/qapi/qmp/qdict.h | 17 -----------------
block.c | 1 +
block/gluster.c | 1 +
block/iscsi.c | 1 +
block/nbd.c | 1 +
block/nfs.c | 1 +
block/parallels.c | 1 +
block/qcow.c | 1 +
block/qcow2.c | 1 +
block/qed.c | 1 +
block/quorum.c | 1 +
block/rbd.c | 1 +
block/sheepdog.c | 1 +
block/snapshot.c | 1 +
block/ssh.c | 1 +
block/vhdx.c | 1 +
block/vpc.c | 1 +
block/vvfat.c | 1 +
block/vxhs.c | 1 +
blockdev.c | 1 +
qobject/qdict.c | 1 +
tests/check-qdict.c | 1 +
tests/check-qobject.c | 1 +
tests/test-replication.c | 1 +
util/qemu-config.c | 1 +
26 files changed, 56 insertions(+), 17 deletions(-)
create mode 100644 include/block/qdict.h
diff --git a/include/block/qdict.h b/include/block/qdict.h
new file mode 100644
index 0000000000..71c037afba
--- /dev/null
+++ b/include/block/qdict.h
@@ -0,0 +1,32 @@
+/*
+ * Special QDict functions used by the block layer
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#ifndef BLOCK_QDICT_H
+#define BLOCK_QDICT_H
+
+#include "qapi/qmp/qdict.h"
+
+void qdict_copy_default(QDict *dst, QDict *src, const char *key);
+void qdict_set_default_str(QDict *dst, const char *key, const char *val);
+
+void qdict_join(QDict *dest, QDict *src, bool overwrite);
+
+void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
+void qdict_array_split(QDict *src, QList **dst);
+int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(const QDict *src, Error **errp);
+void qdict_flatten(QDict *qdict);
+
+typedef struct QDictRenames {
+ const char *from;
+ const char *to;
+} QDictRenames;
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
+
+#endif
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 921a28d2d3..7f3ec10a10 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -67,23 +67,6 @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key,
bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value);
const char *qdict_get_try_str(const QDict *qdict, const char *key);
-void qdict_copy_default(QDict *dst, QDict *src, const char *key);
-void qdict_set_default_str(QDict *dst, const char *key, const char *val);
-
QDict *qdict_clone_shallow(const QDict *src);
-void qdict_flatten(QDict *qdict);
-
-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
-void qdict_array_split(QDict *src, QList **dst);
-int qdict_array_entries(QDict *src, const char *subqdict);
-QObject *qdict_crumple(const QDict *src, Error **errp);
-
-void qdict_join(QDict *dest, QDict *src, bool overwrite);
-
-typedef struct QDictRenames {
- const char *from;
- const char *to;
-} QDictRenames;
-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
#endif /* QDICT_H */
diff --git a/block.c b/block.c
index 50887087f3..afe30caac3 100644
--- a/block.c
+++ b/block.c
@@ -27,6 +27,7 @@
#include "block/block_int.h"
#include "block/blockjob.h"
#include "block/nbd.h"
+#include "block/qdict.h"
#include "qemu/error-report.h"
#include "module_block.h"
#include "qemu/module.h"
diff --git a/block/gluster.c b/block/gluster.c
index 9900b6420c..b5fe7f3e87 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -11,6 +11,7 @@
#include "qemu/osdep.h"
#include <glusterfs/api/glfs.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
diff --git a/block/iscsi.c b/block/iscsi.c
index 7e3ea72bd2..9f00fb47a5 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -33,6 +33,7 @@
#include "qemu/bitops.h"
#include "qemu/bitmap.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "scsi/constants.h"
#include "qemu/iov.h"
#include "qemu/option.h"
diff --git a/block/nbd.c b/block/nbd.c
index ff8333e3c1..d6c4c4ddbc 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -28,6 +28,7 @@
#include "qemu/osdep.h"
#include "nbd-client.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/uri.h"
#include "block/block_int.h"
diff --git a/block/nfs.c b/block/nfs.c
index 3349b67a76..3170b059b3 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -29,6 +29,7 @@
#include "qemu/error-report.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "trace.h"
#include "qemu/iov.h"
#include "qemu/option.h"
diff --git a/block/parallels.c b/block/parallels.c
index 6e9c37f44e..c1d9643498 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -31,6 +31,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/qcow.c b/block/qcow.c
index 1f866af0d3..8c08908fd8 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -26,6 +26,7 @@
#include "qapi/error.h"
#include "qemu/error-report.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/qcow2.c b/block/qcow2.c
index 6fa5e1d71a..d2d955f984 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -24,6 +24,7 @@
#include "qemu/osdep.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include <zlib.h>
diff --git a/block/qed.c b/block/qed.c
index 65cfe92393..324a953cbc 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -13,6 +13,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/timer.h"
#include "qemu/bswap.h"
diff --git a/block/quorum.c b/block/quorum.c
index b6476c405a..9152da8c58 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -17,6 +17,7 @@
#include "qemu/cutils.h"
#include "qemu/option.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qapi-events-block.h"
#include "qapi/qmp/qdict.h"
diff --git a/block/rbd.c b/block/rbd.c
index 40c6e4185f..9659c7361f 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -18,6 +18,7 @@
#include "qemu/error-report.h"
#include "qemu/option.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "crypto/secret.h"
#include "qemu/cutils.h"
#include "qapi/qmp/qstring.h"
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 7b98725af7..2e1f0e6eca 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -24,6 +24,7 @@
#include "qemu/option.h"
#include "qemu/sockets.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/bitops.h"
#include "qemu/cutils.h"
diff --git a/block/snapshot.c b/block/snapshot.c
index 2953d96c06..f9903bc94e 100644
--- a/block/snapshot.c
+++ b/block/snapshot.c
@@ -25,6 +25,7 @@
#include "qemu/osdep.h"
#include "block/snapshot.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qerror.h"
diff --git a/block/ssh.c b/block/ssh.c
index 4c4fa3ccfc..eec37dd27c 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -28,6 +28,7 @@
#include <libssh2_sftp.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/error.h"
#include "qemu/error-report.h"
#include "qemu/option.h"
diff --git a/block/vhdx.c b/block/vhdx.c
index 0831c5c5f4..2e32e24514 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -18,6 +18,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/vpc.c b/block/vpc.c
index 0ebfcd3cc8..41c8c980f1 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -26,6 +26,7 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
diff --git a/block/vvfat.c b/block/vvfat.c
index 662dca0114..4595f335b8 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -27,6 +27,7 @@
#include <dirent.h>
#include "qapi/error.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qemu/module.h"
#include "qemu/option.h"
#include "qemu/bswap.h"
diff --git a/block/vxhs.c b/block/vxhs.c
index 339e23218d..0cb0a007e9 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -12,6 +12,7 @@
#include <qnio/qnio_api.h>
#include <sys/param.h>
#include "block/block_int.h"
+#include "block/qdict.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
diff --git a/blockdev.c b/blockdev.c
index 4862323012..c24e261e37 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -35,6 +35,7 @@
#include "sysemu/blockdev.h"
#include "hw/block/block.h"
#include "block/blockjob.h"
+#include "block/qdict.h"
#include "block/throttle-groups.h"
#include "monitor/monitor.h"
#include "qemu/error-report.h"
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 22800eeceb..0554c64553 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -11,6 +11,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qbool.h"
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index eba5d3528e..93e2112b6d 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -11,6 +11,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnum.h"
diff --git a/tests/check-qobject.c b/tests/check-qobject.c
index 5cb08fcb63..16ccbde82c 100644
--- a/tests/check-qobject.c
+++ b/tests/check-qobject.c
@@ -8,6 +8,7 @@
*/
#include "qemu/osdep.h"
+#include "block/qdict.h"
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qlist.h"
diff --git a/tests/test-replication.c b/tests/test-replication.c
index 68c0d04f2a..c8165ae954 100644
--- a/tests/test-replication.c
+++ b/tests/test-replication.c
@@ -15,6 +15,7 @@
#include "qemu/option.h"
#include "replication.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#define IMG_SIZE (64 * 1024 * 1024)
diff --git a/util/qemu-config.c b/util/qemu-config.c
index 14d84022dc..9d2e278e29 100644
--- a/util/qemu-config.c
+++ b/util/qemu-config.c
@@ -1,4 +1,5 @@
#include "qemu/osdep.h"
+#include "block/qdict.h" /* for qdict_extract_subqdict() */
#include "qapi/error.h"
#include "qapi/qapi-commands-misc.h"
#include "qapi/qmp/qdict.h"
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (6 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 07/26] block: Add block-specific QDict header Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-19 19:29 ` Eric Blake
2018-06-15 14:20 ` [Qemu-devel] [PULL 09/26] block: Fix -blockdev for certain non-string scalars Kevin Wolf
` (18 subsequent siblings)
26 siblings, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Pure code motion, except for two brace placements and a comment
tweaked to appease checkpatch.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 640 ++++++++++++++++++++++++++++++++++++++++++++
qobject/qdict.c | 629 --------------------------------------------
tests/check-block-qdict.c | 655 ++++++++++++++++++++++++++++++++++++++++++++++
tests/check-qdict.c | 642 ---------------------------------------------
MAINTAINERS | 2 +
qobject/Makefile.objs | 1 +
tests/Makefile.include | 4 +
7 files changed, 1302 insertions(+), 1271 deletions(-)
create mode 100644 qobject/block-qdict.c
create mode 100644 tests/check-block-qdict.c
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
new file mode 100644
index 0000000000..fb92010dc5
--- /dev/null
+++ b/qobject/block-qdict.c
@@ -0,0 +1,640 @@
+/*
+ * Special QDict functions used by the block layer
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "block/qdict.h"
+#include "qapi/qmp/qlist.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+
+/**
+ * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
+ * value of 'key' in 'src' is copied there (and the refcount increased
+ * accordingly).
+ */
+void qdict_copy_default(QDict *dst, QDict *src, const char *key)
+{
+ QObject *val;
+
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ val = qdict_get(src, key);
+ if (val) {
+ qdict_put_obj(dst, key, qobject_ref(val));
+ }
+}
+
+/**
+ * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
+ * new QString initialised by 'val' is put there.
+ */
+void qdict_set_default_str(QDict *dst, const char *key, const char *val)
+{
+ if (qdict_haskey(dst, key)) {
+ return;
+ }
+
+ qdict_put_str(dst, key, val);
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target,
+ const char *prefix);
+
+static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QListEntry *entry;
+ char *new_key;
+ int i;
+
+ /* This function is never called with prefix == NULL, i.e., it is always
+ * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
+ * need to remove list entries during the iteration (the whole list will be
+ * deleted eventually anyway from qdict_flatten_qdict()). */
+ assert(prefix);
+
+ entry = qlist_first(qlist);
+
+ for (i = 0; entry; entry = qlist_next(entry), i++) {
+ value = qlist_entry_obj(entry);
+ new_key = g_strdup_printf("%s.%i", prefix, i);
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
+ } else {
+ /* All other types are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ }
+
+ g_free(new_key);
+ }
+}
+
+static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
+{
+ QObject *value;
+ const QDictEntry *entry, *next;
+ char *new_key;
+ bool delete;
+
+ entry = qdict_first(qdict);
+
+ while (entry != NULL) {
+
+ next = qdict_next(qdict, entry);
+ value = qdict_entry_value(entry);
+ new_key = NULL;
+ delete = false;
+
+ if (prefix) {
+ new_key = g_strdup_printf("%s.%s", prefix, entry->key);
+ }
+
+ if (qobject_type(value) == QTYPE_QDICT) {
+ /* Entries of QDicts are processed recursively, the QDict object
+ * itself disappears. */
+ qdict_flatten_qdict(qobject_to(QDict, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (qobject_type(value) == QTYPE_QLIST) {
+ qdict_flatten_qlist(qobject_to(QList, value), target,
+ new_key ? new_key : entry->key);
+ delete = true;
+ } else if (prefix) {
+ /* All other objects are moved to the target unchanged. */
+ qdict_put_obj(target, new_key, qobject_ref(value));
+ delete = true;
+ }
+
+ g_free(new_key);
+
+ if (delete) {
+ qdict_del(qdict, entry->key);
+
+ /* Restart loop after modifying the iterated QDict */
+ entry = qdict_first(qdict);
+ continue;
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_flatten(): For each nested QDict with key x, all fields with key y
+ * are moved to this QDict and their key is renamed to "x.y". For each nested
+ * QList with key x, the field at index y is moved to this QDict with the key
+ * "x.y" (i.e., the reverse of what qdict_array_split() does).
+ * This operation is applied recursively for nested QDicts and QLists.
+ */
+void qdict_flatten(QDict *qdict)
+{
+ qdict_flatten_qdict(qdict, qdict, NULL);
+}
+
+/* extract all the src QDict entries starting by start into dst */
+void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
+
+{
+ const QDictEntry *entry, *next;
+ const char *p;
+
+ *dst = qdict_new();
+ entry = qdict_first(src);
+
+ while (entry != NULL) {
+ next = qdict_next(src, entry);
+ if (strstart(entry->key, start, &p)) {
+ qdict_put_obj(*dst, p, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+ entry = next;
+ }
+}
+
+static int qdict_count_prefixed_entries(const QDict *src, const char *start)
+{
+ const QDictEntry *entry;
+ int count = 0;
+
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (strstart(entry->key, start, NULL)) {
+ if (count == INT_MAX) {
+ return -ERANGE;
+ }
+ count++;
+ }
+ }
+
+ return count;
+}
+
+/**
+ * qdict_array_split(): This function moves array-like elements of a QDict into
+ * a new QList. Every entry in the original QDict with a key "%u" or one
+ * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
+ * incrementally counting up, will be moved to a new QDict at index %u in the
+ * output QList with the key prefix removed, if that prefix is "%u.". If the
+ * whole key is just "%u", the whole QObject will be moved unchanged without
+ * creating a new QDict. The function terminates when there is no entry in the
+ * QDict with a prefix directly (incrementally) following the last one; it also
+ * returns if there are both entries with "%u" and "%u." for the same index %u.
+ * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
+ * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
+ * => [{"a": 42, "b": 23}, {"x": 0}, 66]
+ * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
+ */
+void qdict_array_split(QDict *src, QList **dst)
+{
+ unsigned i;
+
+ *dst = qlist_new();
+
+ for (i = 0; i < UINT_MAX; i++) {
+ QObject *subqobj;
+ bool is_subqdict;
+ QDict *subqdict;
+ char indexstr[32], prefix[32];
+ size_t snprintf_ret;
+
+ snprintf_ret = snprintf(indexstr, 32, "%u", i);
+ assert(snprintf_ret < 32);
+
+ subqobj = qdict_get(src, indexstr);
+
+ snprintf_ret = snprintf(prefix, 32, "%u.", i);
+ assert(snprintf_ret < 32);
+
+ /* Overflow is the same as positive non-zero results */
+ is_subqdict = qdict_count_prefixed_entries(src, prefix);
+
+ /*
+ * There may be either a single subordinate object (named
+ * "%u") or multiple objects (each with a key prefixed "%u."),
+ * but not both.
+ */
+ if (!subqobj == !is_subqdict) {
+ break;
+ }
+
+ if (is_subqdict) {
+ qdict_extract_subqdict(src, &subqdict, prefix);
+ assert(qdict_size(subqdict) > 0);
+ } else {
+ qobject_ref(subqobj);
+ qdict_del(src, indexstr);
+ }
+
+ qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
+ }
+}
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * e.g.
+ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned 'prefix'
+ * string. The 'suffix' string will be left in escaped format, so it
+ * can be fed back into the qdict_split_flat_key() key as the input
+ * later.
+ *
+ * The caller is responsible for freeing the string returned in @prefix
+ * using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix,
+ const char **suffix)
+{
+ const char *separator;
+ size_t i, j;
+
+ /* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+ separator = NULL;
+ do {
+ if (separator) {
+ separator += 2;
+ } else {
+ separator = key;
+ }
+ separator = strchr(separator, '.');
+ } while (separator && separator[1] == '.');
+
+ if (separator) {
+ *prefix = g_strndup(key, separator - key);
+ *suffix = separator + 1;
+ } else {
+ *prefix = g_strdup(key);
+ *suffix = NULL;
+ }
+
+ /* Unescape the '..' sequence into '.' */
+ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+ if ((*prefix)[i] == '.') {
+ assert((*prefix)[i + 1] == '.');
+ i++;
+ }
+ (*prefix)[j] = (*prefix)[i];
+ }
+ (*prefix)[j] = '\0';
+}
+
+/**
+ * qdict_is_list:
+ * @maybe_list: dict to check if keys represent list elements.
+ *
+ * Determine whether all keys in @maybe_list are valid list elements.
+ * If @maybe_list is non-zero in length and all the keys look like
+ * valid list indexes, this will return 1. If @maybe_list is zero
+ * length or all keys are non-numeric then it will return 0 to indicate
+ * it is a normal qdict. If there is a mix of numeric and non-numeric
+ * keys, or the list indexes are non-contiguous, an error is reported.
+ *
+ * Returns: 1 if a valid list, 0 if a dict, -1 on error
+ */
+static int qdict_is_list(QDict *maybe_list, Error **errp)
+{
+ const QDictEntry *ent;
+ ssize_t len = 0;
+ ssize_t max = -1;
+ int is_list = -1;
+ int64_t val;
+
+ for (ent = qdict_first(maybe_list); ent != NULL;
+ ent = qdict_next(maybe_list, ent)) {
+
+ if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
+ if (is_list == -1) {
+ is_list = 1;
+ } else if (!is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ len++;
+ if (val > max) {
+ max = val;
+ }
+ } else {
+ if (is_list == -1) {
+ is_list = 0;
+ } else if (is_list) {
+ error_setg(errp,
+ "Cannot mix list and non-list keys");
+ return -1;
+ }
+ }
+ }
+
+ if (is_list == -1) {
+ assert(!qdict_size(maybe_list));
+ is_list = 0;
+ }
+
+ /* NB this isn't a perfect check - e.g. it won't catch
+ * a list containing '1', '+1', '01', '3', but that
+ * does not matter - we've still proved that the
+ * input is a list. It is up the caller to do a
+ * stricter check if desired */
+ if (len != (max + 1)) {
+ error_setg(errp, "List indices are not contiguous, "
+ "saw %zd elements but %zd largest index",
+ len, max);
+ return -1;
+ }
+
+ return is_list;
+}
+
+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary (only scalar values) to crumple
+ *
+ * Takes a flat dictionary whose keys use '.' separator to indicate
+ * nesting, and values are scalars, and crumples it into a nested
+ * structure.
+ *
+ * To include a literal '.' in a key name, it must be escaped as '..'
+ *
+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in an output of:
+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *
+ * The following scenarios in the input dict will result in an
+ * error being returned:
+ *
+ * - Any values in @src are non-scalar types
+ * - If keys in @src imply that a particular level is both a
+ * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
+ * - If keys in @src imply that a particular level is a list,
+ * but the indices are non-contiguous. e.g. "foo.0.bar" and
+ * "foo.2.bar" without any "foo.1.bar" present.
+ * - If keys in @src represent list indexes, but are not in
+ * the "%zu" format. e.g. "foo.+0.bar"
+ *
+ * Returns: either a QDict or QList for the nested data structure, or NULL
+ * on error
+ */
+QObject *qdict_crumple(const QDict *src, Error **errp)
+{
+ const QDictEntry *ent;
+ QDict *two_level, *multi_level = NULL;
+ QObject *dst = NULL, *child;
+ size_t i;
+ char *prefix = NULL;
+ const char *suffix = NULL;
+ int is_list;
+
+ two_level = qdict_new();
+
+ /* Step 1: split our totally flat dict into a two level dict */
+ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
+ if (qobject_type(ent->value) == QTYPE_QDICT ||
+ qobject_type(ent->value) == QTYPE_QLIST) {
+ error_setg(errp, "Value %s is not a scalar",
+ ent->key);
+ goto error;
+ }
+
+ qdict_split_flat_key(ent->key, &prefix, &suffix);
+
+ child = qdict_get(two_level, prefix);
+ if (suffix) {
+ QDict *child_dict = qobject_to(QDict, child);
+ if (!child_dict) {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a scalar",
+ prefix);
+ goto error;
+ }
+
+ child_dict = qdict_new();
+ qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
+ }
+
+ qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
+ } else {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a dict",
+ prefix);
+ goto error;
+ }
+ qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
+ }
+
+ g_free(prefix);
+ prefix = NULL;
+ }
+
+ /* Step 2: optionally process the two level dict recursively
+ * into a multi-level dict */
+ multi_level = qdict_new();
+ for (ent = qdict_first(two_level); ent != NULL;
+ ent = qdict_next(two_level, ent)) {
+ QDict *dict = qobject_to(QDict, ent->value);
+ if (dict) {
+ child = qdict_crumple(dict, errp);
+ if (!child) {
+ goto error;
+ }
+
+ qdict_put_obj(multi_level, ent->key, child);
+ } else {
+ qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
+ }
+ }
+ qobject_unref(two_level);
+ two_level = NULL;
+
+ /* Step 3: detect if we need to turn our dict into list */
+ is_list = qdict_is_list(multi_level, errp);
+ if (is_list < 0) {
+ goto error;
+ }
+
+ if (is_list) {
+ dst = QOBJECT(qlist_new());
+
+ for (i = 0; i < qdict_size(multi_level); i++) {
+ char *key = g_strdup_printf("%zu", i);
+
+ child = qdict_get(multi_level, key);
+ g_free(key);
+
+ if (!child) {
+ error_setg(errp, "Missing list index %zu", i);
+ goto error;
+ }
+
+ qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
+ }
+ qobject_unref(multi_level);
+ multi_level = NULL;
+ } else {
+ dst = QOBJECT(multi_level);
+ }
+
+ return dst;
+
+ error:
+ g_free(prefix);
+ qobject_unref(multi_level);
+ qobject_unref(two_level);
+ qobject_unref(dst);
+ return NULL;
+}
+
+/**
+ * qdict_array_entries(): Returns the number of direct array entries if the
+ * sub-QDict of src specified by the prefix in subqdict (or src itself for
+ * prefix == "") is valid as an array, i.e. the length of the created list if
+ * the sub-QDict would become empty after calling qdict_array_split() on it. If
+ * the array is not valid, -EINVAL is returned.
+ */
+int qdict_array_entries(QDict *src, const char *subqdict)
+{
+ const QDictEntry *entry;
+ unsigned i;
+ unsigned entries = 0;
+ size_t subqdict_len = strlen(subqdict);
+
+ assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
+
+ /* qdict_array_split() loops until UINT_MAX, but as we want to return
+ * negative errors, we only have a signed return value here. Any additional
+ * entries will lead to -EINVAL. */
+ for (i = 0; i < INT_MAX; i++) {
+ QObject *subqobj;
+ int subqdict_entries;
+ char *prefix = g_strdup_printf("%s%u.", subqdict, i);
+
+ subqdict_entries = qdict_count_prefixed_entries(src, prefix);
+
+ /* Remove ending "." */
+ prefix[strlen(prefix) - 1] = 0;
+ subqobj = qdict_get(src, prefix);
+
+ g_free(prefix);
+
+ if (subqdict_entries < 0) {
+ return subqdict_entries;
+ }
+
+ /* There may be either a single subordinate object (named "%u") or
+ * multiple objects (each with a key prefixed "%u."), but not both. */
+ if (subqobj && subqdict_entries) {
+ return -EINVAL;
+ } else if (!subqobj && !subqdict_entries) {
+ break;
+ }
+
+ entries += subqdict_entries ? subqdict_entries : 1;
+ }
+
+ /* Consider everything handled that isn't part of the given sub-QDict */
+ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
+ if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
+ entries++;
+ }
+ }
+
+ /* Anything left in the sub-QDict that wasn't handled? */
+ if (qdict_size(src) != entries) {
+ return -EINVAL;
+ }
+
+ return i;
+}
+
+/**
+ * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
+ * elements from src to dest.
+ *
+ * If an element from src has a key already present in dest, it will not be
+ * moved unless overwrite is true.
+ *
+ * If overwrite is true, the conflicting values in dest will be discarded and
+ * replaced by the corresponding values from src.
+ *
+ * Therefore, with overwrite being true, the src QDict will always be empty when
+ * this function returns. If overwrite is false, the src QDict will be empty
+ * iff there were no conflicts.
+ */
+void qdict_join(QDict *dest, QDict *src, bool overwrite)
+{
+ const QDictEntry *entry, *next;
+
+ entry = qdict_first(src);
+ while (entry) {
+ next = qdict_next(src, entry);
+
+ if (overwrite || !qdict_haskey(dest, entry->key)) {
+ qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
+ qdict_del(src, entry->key);
+ }
+
+ entry = next;
+ }
+}
+
+/**
+ * qdict_rename_keys(): Rename keys in qdict according to the replacements
+ * specified in the array renames. The array must be terminated by an entry
+ * with from = NULL.
+ *
+ * The renames are performed individually in the order of the array, so entries
+ * may be renamed multiple times and may or may not conflict depending on the
+ * order of the renames array.
+ *
+ * Returns true for success, false in error cases.
+ */
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
+{
+ QObject *qobj;
+
+ while (renames->from) {
+ if (qdict_haskey(qdict, renames->from)) {
+ if (qdict_haskey(qdict, renames->to)) {
+ error_setg(errp, "'%s' and its alias '%s' can't be used at the "
+ "same time", renames->to, renames->from);
+ return false;
+ }
+
+ qobj = qdict_get(qdict, renames->from);
+ qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
+ qdict_del(qdict, renames->from);
+ }
+
+ renames++;
+ }
+ return true;
+}
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 0554c64553..3d8c2f7bbc 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -11,17 +11,11 @@
*/
#include "qemu/osdep.h"
-#include "block/qdict.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qbool.h"
-#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnull.h"
#include "qapi/qmp/qstring.h"
-#include "qapi/error.h"
-#include "qemu/queue.h"
-#include "qemu-common.h"
-#include "qemu/cutils.h"
/**
* qdict_new(): Create a new QDict
@@ -464,626 +458,3 @@ void qdict_destroy_obj(QObject *obj)
g_free(qdict);
}
-
-/**
- * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the
- * value of 'key' in 'src' is copied there (and the refcount increased
- * accordingly).
- */
-void qdict_copy_default(QDict *dst, QDict *src, const char *key)
-{
- QObject *val;
-
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- val = qdict_get(src, key);
- if (val) {
- qdict_put_obj(dst, key, qobject_ref(val));
- }
-}
-
-/**
- * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a
- * new QString initialised by 'val' is put there.
- */
-void qdict_set_default_str(QDict *dst, const char *key, const char *val)
-{
- if (qdict_haskey(dst, key)) {
- return;
- }
-
- qdict_put_str(dst, key, val);
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target,
- const char *prefix);
-
-static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
-{
- QObject *value;
- const QListEntry *entry;
- char *new_key;
- int i;
-
- /* This function is never called with prefix == NULL, i.e., it is always
- * called from within qdict_flatten_q(list|dict)(). Therefore, it does not
- * need to remove list entries during the iteration (the whole list will be
- * deleted eventually anyway from qdict_flatten_qdict()). */
- assert(prefix);
-
- entry = qlist_first(qlist);
-
- for (i = 0; entry; entry = qlist_next(entry), i++) {
- value = qlist_entry_obj(entry);
- new_key = g_strdup_printf("%s.%i", prefix, i);
-
- if (qobject_type(value) == QTYPE_QDICT) {
- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
- } else {
- /* All other types are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- }
-
- g_free(new_key);
- }
-}
-
-static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
-{
- QObject *value;
- const QDictEntry *entry, *next;
- char *new_key;
- bool delete;
-
- entry = qdict_first(qdict);
-
- while (entry != NULL) {
-
- next = qdict_next(qdict, entry);
- value = qdict_entry_value(entry);
- new_key = NULL;
- delete = false;
-
- if (prefix) {
- new_key = g_strdup_printf("%s.%s", prefix, entry->key);
- }
-
- if (qobject_type(value) == QTYPE_QDICT) {
- /* Entries of QDicts are processed recursively, the QDict object
- * itself disappears. */
- qdict_flatten_qdict(qobject_to(QDict, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target,
- new_key ? new_key : entry->key);
- delete = true;
- } else if (prefix) {
- /* All other objects are moved to the target unchanged. */
- qdict_put_obj(target, new_key, qobject_ref(value));
- delete = true;
- }
-
- g_free(new_key);
-
- if (delete) {
- qdict_del(qdict, entry->key);
-
- /* Restart loop after modifying the iterated QDict */
- entry = qdict_first(qdict);
- continue;
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_flatten(): For each nested QDict with key x, all fields with key y
- * are moved to this QDict and their key is renamed to "x.y". For each nested
- * QList with key x, the field at index y is moved to this QDict with the key
- * "x.y" (i.e., the reverse of what qdict_array_split() does).
- * This operation is applied recursively for nested QDicts and QLists.
- */
-void qdict_flatten(QDict *qdict)
-{
- qdict_flatten_qdict(qdict, qdict, NULL);
-}
-
-/* extract all the src QDict entries starting by start into dst */
-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start)
-
-{
- const QDictEntry *entry, *next;
- const char *p;
-
- *dst = qdict_new();
- entry = qdict_first(src);
-
- while (entry != NULL) {
- next = qdict_next(src, entry);
- if (strstart(entry->key, start, &p)) {
- qdict_put_obj(*dst, p, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
- entry = next;
- }
-}
-
-static int qdict_count_prefixed_entries(const QDict *src, const char *start)
-{
- const QDictEntry *entry;
- int count = 0;
-
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (strstart(entry->key, start, NULL)) {
- if (count == INT_MAX) {
- return -ERANGE;
- }
- count++;
- }
- }
-
- return count;
-}
-
-/**
- * qdict_array_split(): This function moves array-like elements of a QDict into
- * a new QList. Every entry in the original QDict with a key "%u" or one
- * prefixed "%u.", where %u designates an unsigned integer starting at 0 and
- * incrementally counting up, will be moved to a new QDict at index %u in the
- * output QList with the key prefix removed, if that prefix is "%u.". If the
- * whole key is just "%u", the whole QObject will be moved unchanged without
- * creating a new QDict. The function terminates when there is no entry in the
- * QDict with a prefix directly (incrementally) following the last one; it also
- * returns if there are both entries with "%u" and "%u." for the same index %u.
- * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66}
- * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66})
- * => [{"a": 42, "b": 23}, {"x": 0}, 66]
- * and {"4.y": 1, "o.o": 7} (remainder of the old QDict)
- */
-void qdict_array_split(QDict *src, QList **dst)
-{
- unsigned i;
-
- *dst = qlist_new();
-
- for (i = 0; i < UINT_MAX; i++) {
- QObject *subqobj;
- bool is_subqdict;
- QDict *subqdict;
- char indexstr[32], prefix[32];
- size_t snprintf_ret;
-
- snprintf_ret = snprintf(indexstr, 32, "%u", i);
- assert(snprintf_ret < 32);
-
- subqobj = qdict_get(src, indexstr);
-
- snprintf_ret = snprintf(prefix, 32, "%u.", i);
- assert(snprintf_ret < 32);
-
- /* Overflow is the same as positive non-zero results */
- is_subqdict = qdict_count_prefixed_entries(src, prefix);
-
- // There may be either a single subordinate object (named "%u") or
- // multiple objects (each with a key prefixed "%u."), but not both.
- if (!subqobj == !is_subqdict) {
- break;
- }
-
- if (is_subqdict) {
- qdict_extract_subqdict(src, &subqdict, prefix);
- assert(qdict_size(subqdict) > 0);
- } else {
- qobject_ref(subqobj);
- qdict_del(src, indexstr);
- }
-
- qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
- }
-}
-
-/**
- * qdict_split_flat_key:
- * @key: the key string to split
- * @prefix: non-NULL pointer to hold extracted prefix
- * @suffix: non-NULL pointer to remaining suffix
- *
- * Given a flattened key such as 'foo.0.bar', split it into two parts
- * at the first '.' separator. Allows double dot ('..') to escape the
- * normal separator.
- *
- * e.g.
- * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
- * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
- *
- * The '..' sequence will be unescaped in the returned 'prefix'
- * string. The 'suffix' string will be left in escaped format, so it
- * can be fed back into the qdict_split_flat_key() key as the input
- * later.
- *
- * The caller is responsible for freeing the string returned in @prefix
- * using g_free().
- */
-static void qdict_split_flat_key(const char *key, char **prefix,
- const char **suffix)
-{
- const char *separator;
- size_t i, j;
-
- /* Find first '.' separator, but if there is a pair '..'
- * that acts as an escape, so skip over '..' */
- separator = NULL;
- do {
- if (separator) {
- separator += 2;
- } else {
- separator = key;
- }
- separator = strchr(separator, '.');
- } while (separator && separator[1] == '.');
-
- if (separator) {
- *prefix = g_strndup(key, separator - key);
- *suffix = separator + 1;
- } else {
- *prefix = g_strdup(key);
- *suffix = NULL;
- }
-
- /* Unescape the '..' sequence into '.' */
- for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
- if ((*prefix)[i] == '.') {
- assert((*prefix)[i + 1] == '.');
- i++;
- }
- (*prefix)[j] = (*prefix)[i];
- }
- (*prefix)[j] = '\0';
-}
-
-/**
- * qdict_is_list:
- * @maybe_list: dict to check if keys represent list elements.
- *
- * Determine whether all keys in @maybe_list are valid list elements.
- * If @maybe_list is non-zero in length and all the keys look like
- * valid list indexes, this will return 1. If @maybe_list is zero
- * length or all keys are non-numeric then it will return 0 to indicate
- * it is a normal qdict. If there is a mix of numeric and non-numeric
- * keys, or the list indexes are non-contiguous, an error is reported.
- *
- * Returns: 1 if a valid list, 0 if a dict, -1 on error
- */
-static int qdict_is_list(QDict *maybe_list, Error **errp)
-{
- const QDictEntry *ent;
- ssize_t len = 0;
- ssize_t max = -1;
- int is_list = -1;
- int64_t val;
-
- for (ent = qdict_first(maybe_list); ent != NULL;
- ent = qdict_next(maybe_list, ent)) {
-
- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
- if (is_list == -1) {
- is_list = 1;
- } else if (!is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- len++;
- if (val > max) {
- max = val;
- }
- } else {
- if (is_list == -1) {
- is_list = 0;
- } else if (is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
- }
- }
-
- if (is_list == -1) {
- assert(!qdict_size(maybe_list));
- is_list = 0;
- }
-
- /* NB this isn't a perfect check - e.g. it won't catch
- * a list containing '1', '+1', '01', '3', but that
- * does not matter - we've still proved that the
- * input is a list. It is up the caller to do a
- * stricter check if desired */
- if (len != (max + 1)) {
- error_setg(errp, "List indices are not contiguous, "
- "saw %zd elements but %zd largest index",
- len, max);
- return -1;
- }
-
- return is_list;
-}
-
-/**
- * qdict_crumple:
- * @src: the original flat dictionary (only scalar values) to crumple
- *
- * Takes a flat dictionary whose keys use '.' separator to indicate
- * nesting, and values are scalars, and crumples it into a nested
- * structure.
- *
- * To include a literal '.' in a key name, it must be escaped as '..'
- *
- * For example, an input of:
- *
- * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
- * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
- *
- * will result in an output of:
- *
- * {
- * 'foo': [
- * { 'bar': 'one', 'wizz': '1' },
- * { 'bar': 'two', 'wizz': '2' }
- * ],
- * }
- *
- * The following scenarios in the input dict will result in an
- * error being returned:
- *
- * - Any values in @src are non-scalar types
- * - If keys in @src imply that a particular level is both a
- * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar".
- * - If keys in @src imply that a particular level is a list,
- * but the indices are non-contiguous. e.g. "foo.0.bar" and
- * "foo.2.bar" without any "foo.1.bar" present.
- * - If keys in @src represent list indexes, but are not in
- * the "%zu" format. e.g. "foo.+0.bar"
- *
- * Returns: either a QDict or QList for the nested data structure, or NULL
- * on error
- */
-QObject *qdict_crumple(const QDict *src, Error **errp)
-{
- const QDictEntry *ent;
- QDict *two_level, *multi_level = NULL;
- QObject *dst = NULL, *child;
- size_t i;
- char *prefix = NULL;
- const char *suffix = NULL;
- int is_list;
-
- two_level = qdict_new();
-
- /* Step 1: split our totally flat dict into a two level dict */
- for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT ||
- qobject_type(ent->value) == QTYPE_QLIST) {
- error_setg(errp, "Value %s is not a scalar",
- ent->key);
- goto error;
- }
-
- qdict_split_flat_key(ent->key, &prefix, &suffix);
-
- child = qdict_get(two_level, prefix);
- if (suffix) {
- QDict *child_dict = qobject_to(QDict, child);
- if (!child_dict) {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a scalar",
- prefix);
- goto error;
- }
-
- child_dict = qdict_new();
- qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
- }
-
- qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
- } else {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a dict",
- prefix);
- goto error;
- }
- qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
- }
-
- g_free(prefix);
- prefix = NULL;
- }
-
- /* Step 2: optionally process the two level dict recursively
- * into a multi-level dict */
- multi_level = qdict_new();
- for (ent = qdict_first(two_level); ent != NULL;
- ent = qdict_next(two_level, ent)) {
- QDict *dict = qobject_to(QDict, ent->value);
- if (dict) {
- child = qdict_crumple(dict, errp);
- if (!child) {
- goto error;
- }
-
- qdict_put_obj(multi_level, ent->key, child);
- } else {
- qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value));
- }
- }
- qobject_unref(two_level);
- two_level = NULL;
-
- /* Step 3: detect if we need to turn our dict into list */
- is_list = qdict_is_list(multi_level, errp);
- if (is_list < 0) {
- goto error;
- }
-
- if (is_list) {
- dst = QOBJECT(qlist_new());
-
- for (i = 0; i < qdict_size(multi_level); i++) {
- char *key = g_strdup_printf("%zu", i);
-
- child = qdict_get(multi_level, key);
- g_free(key);
-
- if (!child) {
- error_setg(errp, "Missing list index %zu", i);
- goto error;
- }
-
- qlist_append_obj(qobject_to(QList, dst), qobject_ref(child));
- }
- qobject_unref(multi_level);
- multi_level = NULL;
- } else {
- dst = QOBJECT(multi_level);
- }
-
- return dst;
-
- error:
- g_free(prefix);
- qobject_unref(multi_level);
- qobject_unref(two_level);
- qobject_unref(dst);
- return NULL;
-}
-
-/**
- * qdict_array_entries(): Returns the number of direct array entries if the
- * sub-QDict of src specified by the prefix in subqdict (or src itself for
- * prefix == "") is valid as an array, i.e. the length of the created list if
- * the sub-QDict would become empty after calling qdict_array_split() on it. If
- * the array is not valid, -EINVAL is returned.
- */
-int qdict_array_entries(QDict *src, const char *subqdict)
-{
- const QDictEntry *entry;
- unsigned i;
- unsigned entries = 0;
- size_t subqdict_len = strlen(subqdict);
-
- assert(!subqdict_len || subqdict[subqdict_len - 1] == '.');
-
- /* qdict_array_split() loops until UINT_MAX, but as we want to return
- * negative errors, we only have a signed return value here. Any additional
- * entries will lead to -EINVAL. */
- for (i = 0; i < INT_MAX; i++) {
- QObject *subqobj;
- int subqdict_entries;
- char *prefix = g_strdup_printf("%s%u.", subqdict, i);
-
- subqdict_entries = qdict_count_prefixed_entries(src, prefix);
-
- /* Remove ending "." */
- prefix[strlen(prefix) - 1] = 0;
- subqobj = qdict_get(src, prefix);
-
- g_free(prefix);
-
- if (subqdict_entries < 0) {
- return subqdict_entries;
- }
-
- /* There may be either a single subordinate object (named "%u") or
- * multiple objects (each with a key prefixed "%u."), but not both. */
- if (subqobj && subqdict_entries) {
- return -EINVAL;
- } else if (!subqobj && !subqdict_entries) {
- break;
- }
-
- entries += subqdict_entries ? subqdict_entries : 1;
- }
-
- /* Consider everything handled that isn't part of the given sub-QDict */
- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) {
- if (!strstart(qdict_entry_key(entry), subqdict, NULL)) {
- entries++;
- }
- }
-
- /* Anything left in the sub-QDict that wasn't handled? */
- if (qdict_size(src) != entries) {
- return -EINVAL;
- }
-
- return i;
-}
-
-/**
- * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all
- * elements from src to dest.
- *
- * If an element from src has a key already present in dest, it will not be
- * moved unless overwrite is true.
- *
- * If overwrite is true, the conflicting values in dest will be discarded and
- * replaced by the corresponding values from src.
- *
- * Therefore, with overwrite being true, the src QDict will always be empty when
- * this function returns. If overwrite is false, the src QDict will be empty
- * iff there were no conflicts.
- */
-void qdict_join(QDict *dest, QDict *src, bool overwrite)
-{
- const QDictEntry *entry, *next;
-
- entry = qdict_first(src);
- while (entry) {
- next = qdict_next(src, entry);
-
- if (overwrite || !qdict_haskey(dest, entry->key)) {
- qdict_put_obj(dest, entry->key, qobject_ref(entry->value));
- qdict_del(src, entry->key);
- }
-
- entry = next;
- }
-}
-
-/**
- * qdict_rename_keys(): Rename keys in qdict according to the replacements
- * specified in the array renames. The array must be terminated by an entry
- * with from = NULL.
- *
- * The renames are performed individually in the order of the array, so entries
- * may be renamed multiple times and may or may not conflict depending on the
- * order of the renames array.
- *
- * Returns true for success, false in error cases.
- */
-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
-{
- QObject *qobj;
-
- while (renames->from) {
- if (qdict_haskey(qdict, renames->from)) {
- if (qdict_haskey(qdict, renames->to)) {
- error_setg(errp, "'%s' and its alias '%s' can't be used at the "
- "same time", renames->to, renames->from);
- return false;
- }
-
- qobj = qdict_get(qdict, renames->from);
- qdict_put_obj(qdict, renames->to, qobject_ref(qobj));
- qdict_del(qdict, renames->from);
- }
-
- renames++;
- }
- return true;
-}
diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c
new file mode 100644
index 0000000000..5b9f4d506e
--- /dev/null
+++ b/tests/check-block-qdict.c
@@ -0,0 +1,655 @@
+/*
+ * Unit-tests for Block layer QDict extras
+ *
+ * Copyright (c) 2013-2018 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "block/qdict.h"
+#include "qapi/qmp/qlist.h"
+#include "qapi/qmp/qnum.h"
+#include "qapi/error.h"
+
+static void qdict_defaults_test(void)
+{
+ QDict *dict, *copy;
+
+ dict = qdict_new();
+ copy = qdict_new();
+
+ qdict_set_default_str(dict, "foo", "abc");
+ qdict_set_default_str(dict, "foo", "def");
+ g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc");
+ qdict_set_default_str(dict, "bar", "ghi");
+
+ qdict_copy_default(copy, dict, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc");
+ qdict_set_default_str(copy, "bar", "xyz");
+ qdict_copy_default(copy, dict, "bar");
+ g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz");
+
+ qobject_unref(copy);
+ qobject_unref(dict);
+}
+
+static void qdict_flatten_test(void)
+{
+ QList *list1 = qlist_new();
+ QList *list2 = qlist_new();
+ QDict *dict1 = qdict_new();
+ QDict *dict2 = qdict_new();
+ QDict *dict3 = qdict_new();
+
+ /*
+ * Test the flattening of
+ *
+ * {
+ * "e": [
+ * 42,
+ * [
+ * 23,
+ * 66,
+ * {
+ * "a": 0,
+ * "b": 1
+ * }
+ * ]
+ * ],
+ * "f": {
+ * "c": 2,
+ * "d": 3,
+ * },
+ * "g": 4
+ * }
+ *
+ * to
+ *
+ * {
+ * "e.0": 42,
+ * "e.1.0": 23,
+ * "e.1.1": 66,
+ * "e.1.2.a": 0,
+ * "e.1.2.b": 1,
+ * "f.c": 2,
+ * "f.d": 3,
+ * "g": 4
+ * }
+ */
+
+ qdict_put_int(dict1, "a", 0);
+ qdict_put_int(dict1, "b", 1);
+
+ qlist_append_int(list1, 23);
+ qlist_append_int(list1, 66);
+ qlist_append(list1, dict1);
+ qlist_append_int(list2, 42);
+ qlist_append(list2, list1);
+
+ qdict_put_int(dict2, "c", 2);
+ qdict_put_int(dict2, "d", 3);
+ qdict_put(dict3, "e", list2);
+ qdict_put(dict3, "f", dict2);
+ qdict_put_int(dict3, "g", 4);
+
+ qdict_flatten(dict3);
+
+ g_assert(qdict_get_int(dict3, "e.0") == 42);
+ g_assert(qdict_get_int(dict3, "e.1.0") == 23);
+ g_assert(qdict_get_int(dict3, "e.1.1") == 66);
+ g_assert(qdict_get_int(dict3, "e.1.2.a") == 0);
+ g_assert(qdict_get_int(dict3, "e.1.2.b") == 1);
+ g_assert(qdict_get_int(dict3, "f.c") == 2);
+ g_assert(qdict_get_int(dict3, "f.d") == 3);
+ g_assert(qdict_get_int(dict3, "g") == 4);
+
+ g_assert(qdict_size(dict3) == 8);
+
+ qobject_unref(dict3);
+}
+
+static void qdict_array_split_test(void)
+{
+ QDict *test_dict = qdict_new();
+ QDict *dict1, *dict2;
+ QNum *int1;
+ QList *test_list;
+
+ /*
+ * Test the split of
+ *
+ * {
+ * "1.x": 0,
+ * "4.y": 1,
+ * "0.a": 42,
+ * "o.o": 7,
+ * "0.b": 23,
+ * "2": 66
+ * }
+ *
+ * to
+ *
+ * [
+ * {
+ * "a": 42,
+ * "b": 23
+ * },
+ * {
+ * "x": 0
+ * },
+ * 66
+ * ]
+ *
+ * and
+ *
+ * {
+ * "4.y": 1,
+ * "o.o": 7
+ * }
+ *
+ * (remaining in the old QDict)
+ *
+ * This example is given in the comment of qdict_array_split().
+ */
+
+ qdict_put_int(test_dict, "1.x", 0);
+ qdict_put_int(test_dict, "4.y", 1);
+ qdict_put_int(test_dict, "0.a", 42);
+ qdict_put_int(test_dict, "o.o", 7);
+ qdict_put_int(test_dict, "0.b", 23);
+ qdict_put_int(test_dict, "2", 66);
+
+ qdict_array_split(test_dict, &test_list);
+
+ dict1 = qobject_to(QDict, qlist_pop(test_list));
+ dict2 = qobject_to(QDict, qlist_pop(test_list));
+ int1 = qobject_to(QNum, qlist_pop(test_list));
+
+ g_assert(dict1);
+ g_assert(dict2);
+ g_assert(int1);
+ g_assert(qlist_empty(test_list));
+
+ qobject_unref(test_list);
+
+ g_assert(qdict_get_int(dict1, "a") == 42);
+ g_assert(qdict_get_int(dict1, "b") == 23);
+
+ g_assert(qdict_size(dict1) == 2);
+
+ qobject_unref(dict1);
+
+ g_assert(qdict_get_int(dict2, "x") == 0);
+
+ g_assert(qdict_size(dict2) == 1);
+
+ qobject_unref(dict2);
+
+ g_assert_cmpint(qnum_get_int(int1), ==, 66);
+
+ qobject_unref(int1);
+
+ g_assert(qdict_get_int(test_dict, "4.y") == 1);
+ g_assert(qdict_get_int(test_dict, "o.o") == 7);
+
+ g_assert(qdict_size(test_dict) == 2);
+
+ qobject_unref(test_dict);
+
+ /*
+ * Test the split of
+ *
+ * {
+ * "0": 42,
+ * "1": 23,
+ * "1.x": 84
+ * }
+ *
+ * to
+ *
+ * [
+ * 42
+ * ]
+ *
+ * and
+ *
+ * {
+ * "1": 23,
+ * "1.x": 84
+ * }
+ *
+ * That is, test whether splitting stops if there is both an entry with key
+ * of "%u" and other entries with keys prefixed "%u." for the same index.
+ */
+
+ test_dict = qdict_new();
+
+ qdict_put_int(test_dict, "0", 42);
+ qdict_put_int(test_dict, "1", 23);
+ qdict_put_int(test_dict, "1.x", 84);
+
+ qdict_array_split(test_dict, &test_list);
+
+ int1 = qobject_to(QNum, qlist_pop(test_list));
+
+ g_assert(int1);
+ g_assert(qlist_empty(test_list));
+
+ qobject_unref(test_list);
+
+ g_assert_cmpint(qnum_get_int(int1), ==, 42);
+
+ qobject_unref(int1);
+
+ g_assert(qdict_get_int(test_dict, "1") == 23);
+ g_assert(qdict_get_int(test_dict, "1.x") == 84);
+
+ g_assert(qdict_size(test_dict) == 2);
+
+ qobject_unref(test_dict);
+}
+
+static void qdict_array_entries_test(void)
+{
+ QDict *dict = qdict_new();
+
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
+
+ qdict_put_int(dict, "bar", 0);
+ qdict_put_int(dict, "baz.0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
+
+ qdict_put_int(dict, "foo.1", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
+ qdict_put_int(dict, "foo.0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2);
+ qdict_put_int(dict, "foo.bar", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
+ qdict_del(dict, "foo.bar");
+
+ qdict_put_int(dict, "foo.2.a", 0);
+ qdict_put_int(dict, "foo.2.b", 0);
+ qdict_put_int(dict, "foo.2.c", 0);
+ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+
+ qobject_unref(dict);
+
+ dict = qdict_new();
+ qdict_put_int(dict, "1", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+ qdict_put_int(dict, "0", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2);
+ qdict_put_int(dict, "bar", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
+ qdict_del(dict, "bar");
+
+ qdict_put_int(dict, "2.a", 0);
+ qdict_put_int(dict, "2.b", 0);
+ qdict_put_int(dict, "2.c", 0);
+ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3);
+
+ qobject_unref(dict);
+}
+
+static void qdict_join_test(void)
+{
+ QDict *dict1, *dict2;
+ bool overwrite = false;
+ int i;
+
+ dict1 = qdict_new();
+ dict2 = qdict_new();
+
+ /* Test everything once without overwrite and once with */
+ do {
+ /* Test empty dicts */
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 0);
+ g_assert(qdict_size(dict2) == 0);
+
+ /* First iteration: Test movement */
+ /* Second iteration: Test empty source and non-empty destination */
+ qdict_put_int(dict2, "foo", 42);
+
+ for (i = 0; i < 2; i++) {
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 1);
+ g_assert(qdict_size(dict2) == 0);
+
+ g_assert(qdict_get_int(dict1, "foo") == 42);
+ }
+
+ /* Test non-empty source and destination without conflict */
+ qdict_put_int(dict2, "bar", 23);
+
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 2);
+ g_assert(qdict_size(dict2) == 0);
+
+ g_assert(qdict_get_int(dict1, "foo") == 42);
+ g_assert(qdict_get_int(dict1, "bar") == 23);
+
+ /* Test conflict */
+ qdict_put_int(dict2, "foo", 84);
+
+ qdict_join(dict1, dict2, overwrite);
+
+ g_assert(qdict_size(dict1) == 2);
+ g_assert(qdict_size(dict2) == !overwrite);
+
+ g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42));
+ g_assert(qdict_get_int(dict1, "bar") == 23);
+
+ if (!overwrite) {
+ g_assert(qdict_get_int(dict2, "foo") == 84);
+ }
+
+ /* Check the references */
+ g_assert(qdict_get(dict1, "foo")->base.refcnt == 1);
+ g_assert(qdict_get(dict1, "bar")->base.refcnt == 1);
+
+ if (!overwrite) {
+ g_assert(qdict_get(dict2, "foo")->base.refcnt == 1);
+ }
+
+ /* Clean up */
+ qdict_del(dict1, "foo");
+ qdict_del(dict1, "bar");
+
+ if (!overwrite) {
+ qdict_del(dict2, "foo");
+ }
+ } while (overwrite ^= true);
+
+ qobject_unref(dict1);
+ qobject_unref(dict2);
+}
+
+static void qdict_crumple_test_recursive(void)
+{
+ QDict *src, *dst, *rule, *vnc, *acl, *listen;
+ QList *rules;
+
+ src = qdict_new();
+ qdict_put_str(src, "vnc.listen.addr", "127.0.0.1");
+ qdict_put_str(src, "vnc.listen.port", "5901");
+ qdict_put_str(src, "vnc.acl.rules.0.match", "fred");
+ qdict_put_str(src, "vnc.acl.rules.0.policy", "allow");
+ qdict_put_str(src, "vnc.acl.rules.1.match", "bob");
+ qdict_put_str(src, "vnc.acl.rules.1.policy", "deny");
+ qdict_put_str(src, "vnc.acl.default", "deny");
+ qdict_put_str(src, "vnc.acl..name", "acl0");
+ qdict_put_str(src, "vnc.acl.rule..name", "acl0");
+
+ dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
+ g_assert(dst);
+ g_assert_cmpint(qdict_size(dst), ==, 1);
+
+ vnc = qdict_get_qdict(dst, "vnc");
+ g_assert(vnc);
+ g_assert_cmpint(qdict_size(vnc), ==, 3);
+
+ listen = qdict_get_qdict(vnc, "listen");
+ g_assert(listen);
+ g_assert_cmpint(qdict_size(listen), ==, 2);
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
+
+ acl = qdict_get_qdict(vnc, "acl");
+ g_assert(acl);
+ g_assert_cmpint(qdict_size(acl), ==, 3);
+
+ rules = qdict_get_qlist(acl, "rules");
+ g_assert(rules);
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ rule = qobject_to(QDict, qlist_pop(rules));
+ g_assert(rule);
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ qobject_unref(rule);
+
+ rule = qobject_to(QDict, qlist_pop(rules));
+ g_assert(rule);
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ qobject_unref(rule);
+
+ /* With recursive crumpling, we should see all names unescaped */
+ g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
+ g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
+
+ qobject_unref(src);
+ qobject_unref(dst);
+}
+
+static void qdict_crumple_test_empty(void)
+{
+ QDict *src, *dst;
+
+ src = qdict_new();
+
+ dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
+
+ g_assert_cmpint(qdict_size(dst), ==, 0);
+
+ qobject_unref(src);
+ qobject_unref(dst);
+}
+
+static int qdict_count_entries(QDict *dict)
+{
+ const QDictEntry *e;
+ int count = 0;
+
+ for (e = qdict_first(dict); e; e = qdict_next(dict, e)) {
+ count++;
+ }
+
+ return count;
+}
+
+static void qdict_rename_keys_test(void)
+{
+ QDict *dict = qdict_new();
+ QDict *copy;
+ QDictRenames *renames;
+ Error *local_err = NULL;
+
+ qdict_put_str(dict, "abc", "foo");
+ qdict_put_str(dict, "abcdef", "bar");
+ qdict_put_int(dict, "number", 42);
+ qdict_put_bool(dict, "flag", true);
+ qdict_put_null(dict, "nothing");
+
+ /* Empty rename list */
+ renames = (QDictRenames[]) {
+ { NULL, "this can be anything" }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Simple rename of all entries */
+ renames = (QDictRenames[]) {
+ { "abc", "str1" },
+ { "abcdef", "str2" },
+ { "number", "int" },
+ { "flag", "bool" },
+ { "nothing", "null" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert(!qdict_haskey(copy, "abc"));
+ g_assert(!qdict_haskey(copy, "abcdef"));
+ g_assert(!qdict_haskey(copy, "number"));
+ g_assert(!qdict_haskey(copy, "flag"));
+ g_assert(!qdict_haskey(copy, "nothing"));
+
+ g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Renames are processed top to bottom */
+ renames = (QDictRenames[]) {
+ { "abc", "tmp" },
+ { "abcdef", "abc" },
+ { "number", "abcdef" },
+ { "flag", "number" },
+ { "nothing", "flag" },
+ { "tmp", "nothing" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &error_abort);
+
+ g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL);
+ g_assert(!qdict_haskey(copy, "tmp"));
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Conflicting rename */
+ renames = (QDictRenames[]) {
+ { "abcdef", "abc" },
+ { NULL , NULL }
+ };
+ copy = qdict_clone_shallow(dict);
+ qdict_rename_keys(copy, renames, &local_err);
+
+ g_assert(local_err != NULL);
+ error_free(local_err);
+ local_err = NULL;
+
+ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+ g_assert_cmpint(qdict_count_entries(copy), ==, 5);
+
+ qobject_unref(copy);
+
+ /* Renames in an empty dict */
+ renames = (QDictRenames[]) {
+ { "abcdef", "abc" },
+ { NULL , NULL }
+ };
+
+ qobject_unref(dict);
+ dict = qdict_new();
+
+ qdict_rename_keys(dict, renames, &error_abort);
+ g_assert(qdict_first(dict) == NULL);
+
+ qobject_unref(dict);
+}
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+ QDict *src;
+ Error *error = NULL;
+
+ src = qdict_new();
+ /* rule.0 can't be both a string and a dict */
+ qdict_put_str(src, "rule.0", "fred");
+ qdict_put_str(src, "rule.0.policy", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* rule can't be both a list and a dict */
+ qdict_put_str(src, "rule.0", "fred");
+ qdict_put_str(src, "rule.a", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* The input should be flat, ie no dicts or lists */
+ qdict_put(src, "rule.a", qdict_new());
+ qdict_put_str(src, "rule.b", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* List indexes must not have gaps */
+ qdict_put_str(src, "rule.0", "deny");
+ qdict_put_str(src, "rule.3", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+
+ src = qdict_new();
+ /* List indexes must be in %zu format */
+ qdict_put_str(src, "rule.0", "deny");
+ qdict_put_str(src, "rule.+1", "allow");
+
+ g_assert(qdict_crumple(src, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ qobject_unref(src);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ g_test_add_func("/public/defaults", qdict_defaults_test);
+ g_test_add_func("/public/flatten", qdict_flatten_test);
+ g_test_add_func("/public/array_split", qdict_array_split_test);
+ g_test_add_func("/public/array_entries", qdict_array_entries_test);
+ g_test_add_func("/public/join", qdict_join_test);
+ g_test_add_func("/public/crumple/recursive",
+ qdict_crumple_test_recursive);
+ g_test_add_func("/public/crumple/empty",
+ qdict_crumple_test_empty);
+ g_test_add_func("/public/crumple/bad_inputs",
+ qdict_crumple_test_bad_inputs);
+
+ g_test_add_func("/public/rename_keys", qdict_rename_keys_test);
+
+ return g_test_run();
+}
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index 93e2112b6d..86e9fe7dc4 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -11,13 +11,7 @@
*/
#include "qemu/osdep.h"
-#include "block/qdict.h"
#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qlist.h"
-#include "qapi/qmp/qnum.h"
-#include "qapi/qmp/qstring.h"
-#include "qapi/error.h"
-#include "qemu-common.h"
/*
* Public Interface test-cases
@@ -157,28 +151,6 @@ static void qdict_get_try_str_test(void)
qobject_unref(tests_dict);
}
-static void qdict_defaults_test(void)
-{
- QDict *dict, *copy;
-
- dict = qdict_new();
- copy = qdict_new();
-
- qdict_set_default_str(dict, "foo", "abc");
- qdict_set_default_str(dict, "foo", "def");
- g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc");
- qdict_set_default_str(dict, "bar", "ghi");
-
- qdict_copy_default(copy, dict, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc");
- qdict_set_default_str(copy, "bar", "xyz");
- qdict_copy_default(copy, dict, "bar");
- g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz");
-
- qobject_unref(copy);
- qobject_unref(dict);
-}
-
static void qdict_haskey_not_test(void)
{
QDict *tests_dict = qdict_new();
@@ -254,606 +226,6 @@ static void qdict_iterapi_test(void)
qobject_unref(tests_dict);
}
-static void qdict_flatten_test(void)
-{
- QList *list1 = qlist_new();
- QList *list2 = qlist_new();
- QDict *dict1 = qdict_new();
- QDict *dict2 = qdict_new();
- QDict *dict3 = qdict_new();
-
- /*
- * Test the flattening of
- *
- * {
- * "e": [
- * 42,
- * [
- * 23,
- * 66,
- * {
- * "a": 0,
- * "b": 1
- * }
- * ]
- * ],
- * "f": {
- * "c": 2,
- * "d": 3,
- * },
- * "g": 4
- * }
- *
- * to
- *
- * {
- * "e.0": 42,
- * "e.1.0": 23,
- * "e.1.1": 66,
- * "e.1.2.a": 0,
- * "e.1.2.b": 1,
- * "f.c": 2,
- * "f.d": 3,
- * "g": 4
- * }
- */
-
- qdict_put_int(dict1, "a", 0);
- qdict_put_int(dict1, "b", 1);
-
- qlist_append_int(list1, 23);
- qlist_append_int(list1, 66);
- qlist_append(list1, dict1);
- qlist_append_int(list2, 42);
- qlist_append(list2, list1);
-
- qdict_put_int(dict2, "c", 2);
- qdict_put_int(dict2, "d", 3);
- qdict_put(dict3, "e", list2);
- qdict_put(dict3, "f", dict2);
- qdict_put_int(dict3, "g", 4);
-
- qdict_flatten(dict3);
-
- g_assert(qdict_get_int(dict3, "e.0") == 42);
- g_assert(qdict_get_int(dict3, "e.1.0") == 23);
- g_assert(qdict_get_int(dict3, "e.1.1") == 66);
- g_assert(qdict_get_int(dict3, "e.1.2.a") == 0);
- g_assert(qdict_get_int(dict3, "e.1.2.b") == 1);
- g_assert(qdict_get_int(dict3, "f.c") == 2);
- g_assert(qdict_get_int(dict3, "f.d") == 3);
- g_assert(qdict_get_int(dict3, "g") == 4);
-
- g_assert(qdict_size(dict3) == 8);
-
- qobject_unref(dict3);
-}
-
-static void qdict_array_split_test(void)
-{
- QDict *test_dict = qdict_new();
- QDict *dict1, *dict2;
- QNum *int1;
- QList *test_list;
-
- /*
- * Test the split of
- *
- * {
- * "1.x": 0,
- * "4.y": 1,
- * "0.a": 42,
- * "o.o": 7,
- * "0.b": 23,
- * "2": 66
- * }
- *
- * to
- *
- * [
- * {
- * "a": 42,
- * "b": 23
- * },
- * {
- * "x": 0
- * },
- * 66
- * ]
- *
- * and
- *
- * {
- * "4.y": 1,
- * "o.o": 7
- * }
- *
- * (remaining in the old QDict)
- *
- * This example is given in the comment of qdict_array_split().
- */
-
- qdict_put_int(test_dict, "1.x", 0);
- qdict_put_int(test_dict, "4.y", 1);
- qdict_put_int(test_dict, "0.a", 42);
- qdict_put_int(test_dict, "o.o", 7);
- qdict_put_int(test_dict, "0.b", 23);
- qdict_put_int(test_dict, "2", 66);
-
- qdict_array_split(test_dict, &test_list);
-
- dict1 = qobject_to(QDict, qlist_pop(test_list));
- dict2 = qobject_to(QDict, qlist_pop(test_list));
- int1 = qobject_to(QNum, qlist_pop(test_list));
-
- g_assert(dict1);
- g_assert(dict2);
- g_assert(int1);
- g_assert(qlist_empty(test_list));
-
- qobject_unref(test_list);
-
- g_assert(qdict_get_int(dict1, "a") == 42);
- g_assert(qdict_get_int(dict1, "b") == 23);
-
- g_assert(qdict_size(dict1) == 2);
-
- qobject_unref(dict1);
-
- g_assert(qdict_get_int(dict2, "x") == 0);
-
- g_assert(qdict_size(dict2) == 1);
-
- qobject_unref(dict2);
-
- g_assert_cmpint(qnum_get_int(int1), ==, 66);
-
- qobject_unref(int1);
-
- g_assert(qdict_get_int(test_dict, "4.y") == 1);
- g_assert(qdict_get_int(test_dict, "o.o") == 7);
-
- g_assert(qdict_size(test_dict) == 2);
-
- qobject_unref(test_dict);
-
- /*
- * Test the split of
- *
- * {
- * "0": 42,
- * "1": 23,
- * "1.x": 84
- * }
- *
- * to
- *
- * [
- * 42
- * ]
- *
- * and
- *
- * {
- * "1": 23,
- * "1.x": 84
- * }
- *
- * That is, test whether splitting stops if there is both an entry with key
- * of "%u" and other entries with keys prefixed "%u." for the same index.
- */
-
- test_dict = qdict_new();
-
- qdict_put_int(test_dict, "0", 42);
- qdict_put_int(test_dict, "1", 23);
- qdict_put_int(test_dict, "1.x", 84);
-
- qdict_array_split(test_dict, &test_list);
-
- int1 = qobject_to(QNum, qlist_pop(test_list));
-
- g_assert(int1);
- g_assert(qlist_empty(test_list));
-
- qobject_unref(test_list);
-
- g_assert_cmpint(qnum_get_int(int1), ==, 42);
-
- qobject_unref(int1);
-
- g_assert(qdict_get_int(test_dict, "1") == 23);
- g_assert(qdict_get_int(test_dict, "1.x") == 84);
-
- g_assert(qdict_size(test_dict) == 2);
-
- qobject_unref(test_dict);
-}
-
-static void qdict_array_entries_test(void)
-{
- QDict *dict = qdict_new();
-
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
-
- qdict_put_int(dict, "bar", 0);
- qdict_put_int(dict, "baz.0", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0);
-
- qdict_put_int(dict, "foo.1", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
- qdict_put_int(dict, "foo.0", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2);
- qdict_put_int(dict, "foo.bar", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL);
- qdict_del(dict, "foo.bar");
-
- qdict_put_int(dict, "foo.2.a", 0);
- qdict_put_int(dict, "foo.2.b", 0);
- qdict_put_int(dict, "foo.2.c", 0);
- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
-
- qobject_unref(dict);
-
- dict = qdict_new();
- qdict_put_int(dict, "1", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
- qdict_put_int(dict, "0", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2);
- qdict_put_int(dict, "bar", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL);
- qdict_del(dict, "bar");
-
- qdict_put_int(dict, "2.a", 0);
- qdict_put_int(dict, "2.b", 0);
- qdict_put_int(dict, "2.c", 0);
- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3);
-
- qobject_unref(dict);
-}
-
-static void qdict_join_test(void)
-{
- QDict *dict1, *dict2;
- bool overwrite = false;
- int i;
-
- dict1 = qdict_new();
- dict2 = qdict_new();
-
- /* Test everything once without overwrite and once with */
- do
- {
- /* Test empty dicts */
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 0);
- g_assert(qdict_size(dict2) == 0);
-
- /* First iteration: Test movement */
- /* Second iteration: Test empty source and non-empty destination */
- qdict_put_int(dict2, "foo", 42);
-
- for (i = 0; i < 2; i++) {
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 1);
- g_assert(qdict_size(dict2) == 0);
-
- g_assert(qdict_get_int(dict1, "foo") == 42);
- }
-
- /* Test non-empty source and destination without conflict */
- qdict_put_int(dict2, "bar", 23);
-
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 2);
- g_assert(qdict_size(dict2) == 0);
-
- g_assert(qdict_get_int(dict1, "foo") == 42);
- g_assert(qdict_get_int(dict1, "bar") == 23);
-
- /* Test conflict */
- qdict_put_int(dict2, "foo", 84);
-
- qdict_join(dict1, dict2, overwrite);
-
- g_assert(qdict_size(dict1) == 2);
- g_assert(qdict_size(dict2) == !overwrite);
-
- g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42));
- g_assert(qdict_get_int(dict1, "bar") == 23);
-
- if (!overwrite) {
- g_assert(qdict_get_int(dict2, "foo") == 84);
- }
-
- /* Check the references */
- g_assert(qdict_get(dict1, "foo")->base.refcnt == 1);
- g_assert(qdict_get(dict1, "bar")->base.refcnt == 1);
-
- if (!overwrite) {
- g_assert(qdict_get(dict2, "foo")->base.refcnt == 1);
- }
-
- /* Clean up */
- qdict_del(dict1, "foo");
- qdict_del(dict1, "bar");
-
- if (!overwrite) {
- qdict_del(dict2, "foo");
- }
- }
- while (overwrite ^= true);
-
- qobject_unref(dict1);
- qobject_unref(dict2);
-}
-
-static void qdict_crumple_test_recursive(void)
-{
- QDict *src, *dst, *rule, *vnc, *acl, *listen;
- QList *rules;
-
- src = qdict_new();
- qdict_put_str(src, "vnc.listen.addr", "127.0.0.1");
- qdict_put_str(src, "vnc.listen.port", "5901");
- qdict_put_str(src, "vnc.acl.rules.0.match", "fred");
- qdict_put_str(src, "vnc.acl.rules.0.policy", "allow");
- qdict_put_str(src, "vnc.acl.rules.1.match", "bob");
- qdict_put_str(src, "vnc.acl.rules.1.policy", "deny");
- qdict_put_str(src, "vnc.acl.default", "deny");
- qdict_put_str(src, "vnc.acl..name", "acl0");
- qdict_put_str(src, "vnc.acl.rule..name", "acl0");
-
- dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
- g_assert(dst);
- g_assert_cmpint(qdict_size(dst), ==, 1);
-
- vnc = qdict_get_qdict(dst, "vnc");
- g_assert(vnc);
- g_assert_cmpint(qdict_size(vnc), ==, 3);
-
- listen = qdict_get_qdict(vnc, "listen");
- g_assert(listen);
- g_assert_cmpint(qdict_size(listen), ==, 2);
- g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
- g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
-
- acl = qdict_get_qdict(vnc, "acl");
- g_assert(acl);
- g_assert_cmpint(qdict_size(acl), ==, 3);
-
- rules = qdict_get_qlist(acl, "rules");
- g_assert(rules);
- g_assert_cmpint(qlist_size(rules), ==, 2);
-
- rule = qobject_to(QDict, qlist_pop(rules));
- g_assert(rule);
- g_assert_cmpint(qdict_size(rule), ==, 2);
- g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
- g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
- qobject_unref(rule);
-
- rule = qobject_to(QDict, qlist_pop(rules));
- g_assert(rule);
- g_assert_cmpint(qdict_size(rule), ==, 2);
- g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
- g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
- qobject_unref(rule);
-
- /* With recursive crumpling, we should see all names unescaped */
- g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
- g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
-
- qobject_unref(src);
- qobject_unref(dst);
-}
-
-static void qdict_crumple_test_empty(void)
-{
- QDict *src, *dst;
-
- src = qdict_new();
-
- dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
-
- g_assert_cmpint(qdict_size(dst), ==, 0);
-
- qobject_unref(src);
- qobject_unref(dst);
-}
-
-static int qdict_count_entries(QDict *dict)
-{
- const QDictEntry *e;
- int count = 0;
-
- for (e = qdict_first(dict); e; e = qdict_next(dict, e)) {
- count++;
- }
-
- return count;
-}
-
-static void qdict_rename_keys_test(void)
-{
- QDict *dict = qdict_new();
- QDict *copy;
- QDictRenames *renames;
- Error *local_err = NULL;
-
- qdict_put_str(dict, "abc", "foo");
- qdict_put_str(dict, "abcdef", "bar");
- qdict_put_int(dict, "number", 42);
- qdict_put_bool(dict, "flag", true);
- qdict_put_null(dict, "nothing");
-
- /* Empty rename list */
- renames = (QDictRenames[]) {
- { NULL, "this can be anything" }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Simple rename of all entries */
- renames = (QDictRenames[]) {
- { "abc", "str1" },
- { "abcdef", "str2" },
- { "number", "int" },
- { "flag", "bool" },
- { "nothing", "null" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert(!qdict_haskey(copy, "abc"));
- g_assert(!qdict_haskey(copy, "abcdef"));
- g_assert(!qdict_haskey(copy, "number"));
- g_assert(!qdict_haskey(copy, "flag"));
- g_assert(!qdict_haskey(copy, "nothing"));
-
- g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Renames are processed top to bottom */
- renames = (QDictRenames[]) {
- { "abc", "tmp" },
- { "abcdef", "abc" },
- { "number", "abcdef" },
- { "flag", "number" },
- { "nothing", "flag" },
- { "tmp", "nothing" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &error_abort);
-
- g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL);
- g_assert(!qdict_haskey(copy, "tmp"));
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Conflicting rename */
- renames = (QDictRenames[]) {
- { "abcdef", "abc" },
- { NULL , NULL }
- };
- copy = qdict_clone_shallow(dict);
- qdict_rename_keys(copy, renames, &local_err);
-
- g_assert(local_err != NULL);
- error_free(local_err);
- local_err = NULL;
-
- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
- g_assert_cmpint(qdict_count_entries(copy), ==, 5);
-
- qobject_unref(copy);
-
- /* Renames in an empty dict */
- renames = (QDictRenames[]) {
- { "abcdef", "abc" },
- { NULL , NULL }
- };
-
- qobject_unref(dict);
- dict = qdict_new();
-
- qdict_rename_keys(dict, renames, &error_abort);
- g_assert(qdict_first(dict) == NULL);
-
- qobject_unref(dict);
-}
-
-static void qdict_crumple_test_bad_inputs(void)
-{
- QDict *src;
- Error *error = NULL;
-
- src = qdict_new();
- /* rule.0 can't be both a string and a dict */
- qdict_put_str(src, "rule.0", "fred");
- qdict_put_str(src, "rule.0.policy", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* rule can't be both a list and a dict */
- qdict_put_str(src, "rule.0", "fred");
- qdict_put_str(src, "rule.a", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* The input should be flat, ie no dicts or lists */
- qdict_put(src, "rule.a", qdict_new());
- qdict_put_str(src, "rule.b", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* List indexes must not have gaps */
- qdict_put_str(src, "rule.0", "deny");
- qdict_put_str(src, "rule.3", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-
- src = qdict_new();
- /* List indexes must be in %zu format */
- qdict_put_str(src, "rule.0", "deny");
- qdict_put_str(src, "rule.+1", "allow");
-
- g_assert(qdict_crumple(src, &error) == NULL);
- g_assert(error != NULL);
- error_free(error);
- error = NULL;
- qobject_unref(src);
-}
-
/*
* Errors test-cases
*/
@@ -987,29 +359,15 @@ int main(int argc, char **argv)
g_test_add_func("/public/get_try_int", qdict_get_try_int_test);
g_test_add_func("/public/get_str", qdict_get_str_test);
g_test_add_func("/public/get_try_str", qdict_get_try_str_test);
- g_test_add_func("/public/defaults", qdict_defaults_test);
g_test_add_func("/public/haskey_not", qdict_haskey_not_test);
g_test_add_func("/public/haskey", qdict_haskey_test);
g_test_add_func("/public/del", qdict_del_test);
g_test_add_func("/public/to_qdict", qobject_to_qdict_test);
g_test_add_func("/public/iterapi", qdict_iterapi_test);
- g_test_add_func("/public/flatten", qdict_flatten_test);
- g_test_add_func("/public/array_split", qdict_array_split_test);
- g_test_add_func("/public/array_entries", qdict_array_entries_test);
- g_test_add_func("/public/join", qdict_join_test);
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
- g_test_add_func("/public/crumple/recursive",
- qdict_crumple_test_recursive);
- g_test_add_func("/public/crumple/empty",
- qdict_crumple_test_empty);
- g_test_add_func("/public/crumple/bad_inputs",
- qdict_crumple_test_bad_inputs);
-
- g_test_add_func("/public/rename_keys", qdict_rename_keys_test);
-
/* The Big one */
if (g_test_slow()) {
g_test_add_func("/stress/test", qdict_stress_test);
diff --git a/MAINTAINERS b/MAINTAINERS
index 8a94517e9e..0fb5f38f9f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1369,6 +1369,8 @@ F: qemu-img*
F: qemu-io*
F: tests/qemu-iotests/
F: util/qemu-progress.c
+F: qobject/block-qdict.c
+F: test/check-block-qdict.c
T: git git://repo.or.cz/qemu/kevin.git block
Block I/O path
diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs
index 002d25873a..7b12c9cacf 100644
--- a/qobject/Makefile.objs
+++ b/qobject/Makefile.objs
@@ -1,2 +1,3 @@
util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o
util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
+util-obj-y += block-qdict.o
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 607afe5bed..ca91da26cb 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -39,6 +39,8 @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \
check-unit-y = tests/check-qdict$(EXESUF)
gcov-files-check-qdict-y = qobject/qdict.c
+check-unit-y = tests/check-block-qdict$(EXESUF)
+gcov-files-check-block-qdict-y = qobject/block-qdict.c
check-unit-y += tests/test-char$(EXESUF)
gcov-files-check-qdict-y = chardev/char.c
check-unit-y += tests/check-qnum$(EXESUF)
@@ -584,6 +586,7 @@ GENERATED_FILES += tests/test-qapi-types.h tests/test-qapi-visit.h \
test-obj-y = tests/check-qnum.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qnull.o tests/check-qobject.o \
tests/check-qjson.o tests/check-qlit.o \
+ tests/check-block-qtest.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \
tests/test-clone-visitor.o \
@@ -614,6 +617,7 @@ test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o
tests/check-qnum$(EXESUF): tests/check-qnum.o $(test-util-obj-y)
tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y)
tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y)
+tests/check-block-qdict$(EXESUF): tests/check-block-qdict.o $(test-util-obj-y)
tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y)
tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y)
tests/check-qobject$(EXESUF): tests/check-qobject.o $(test-util-obj-y)
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 09/26] block: Fix -blockdev for certain non-string scalars
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (7 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 10/26] block: Fix -drive " Kevin Wolf
` (17 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Configuration flows through the block subsystem in a rather peculiar
way. Configuration made with -drive enters it as QemuOpts.
Configuration made with -blockdev / blockdev-add enters it as QAPI
type BlockdevOptions. The block subsystem uses QDict, QemuOpts and
QAPI types internally. The precise flow is next to impossible to
explain (I tried for this commit message, but gave up after wasting
several hours). What I can explain is a flaw in the BlockDriver
interface that leads to this bug:
$ qemu-system-x86_64 -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234
qemu-system-x86_64: -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234: Internal error: parameter user invalid
QMP blockdev-add is broken the same way.
Here's what happens. The block layer passes configuration represented
as flat QDict (with dotted keys) to BlockDriver methods
.bdrv_file_open(). The QDict's members are typed according to the
QAPI schema.
nfs_file_open() converts it to QAPI type BlockdevOptionsNfs, with
qdict_crumple() and a qobject input visitor.
This visitor comes in two flavors. The plain flavor requires scalars
to be typed according to the QAPI schema. That's the case here. The
keyval flavor requires string scalars. That's not the case here.
nfs_file_open() uses the latter, and promptly falls apart for members
@user, @group, @tcp-syn-count, @readahead-size, @page-cache-size,
@debug.
Switching to the plain flavor would fix -blockdev, but break -drive,
because there the scalars arrive in nfs_file_open() as strings.
The proper fix would be to replace the QDict by QAPI type
BlockdevOptions in the BlockDriver interface. Sadly, that's beyond my
reach right now.
Next best would be to fix the block layer to always pass correctly
typed QDicts to the BlockDriver methods. Also beyond my reach.
What I can do is throw another hack onto the pile: have
nfs_file_open() convert all members to string, so use of the keyval
flavor actually works, by replacing qdict_crumple() by new function
qdict_crumple_for_keyval_qiv().
The pattern "pass result of qdict_crumple() to
qobject_input_visitor_new_keyval()" occurs several times more:
* qemu_rbd_open()
Same issue as nfs_file_open(), but since BlockdevOptionsRbd has only
string members, its only a latent bug. Fix it anyway.
* parallels_co_create_opts(), qcow_co_create_opts(),
qcow2_co_create_opts(), bdrv_qed_co_create_opts(),
sd_co_create_opts(), vhdx_co_create_opts(), vpc_co_create_opts()
These work, because they create the QDict with
qemu_opts_to_qdict_filtered(), which creates only string scalars.
The function sports a TODO comment asking for better typing; that's
going to be fun. Use qdict_crumple_for_keyval_qiv() to be safe.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/block/qdict.h | 1 +
block/nfs.c | 2 +-
block/parallels.c | 2 +-
block/qcow.c | 2 +-
block/qcow2.c | 2 +-
block/qed.c | 2 +-
block/rbd.c | 2 +-
block/sheepdog.c | 2 +-
block/vhdx.c | 2 +-
block/vpc.c | 2 +-
qobject/block-qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++
11 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/include/block/qdict.h b/include/block/qdict.h
index 71c037afba..47d9638c37 100644
--- a/include/block/qdict.h
+++ b/include/block/qdict.h
@@ -21,6 +21,7 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
QObject *qdict_crumple(const QDict *src, Error **errp);
+QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp);
void qdict_flatten(QDict *qdict);
typedef struct QDictRenames {
diff --git a/block/nfs.c b/block/nfs.c
index 3170b059b3..6935b611cc 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -561,7 +561,7 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options,
const QDictEntry *e;
Error *local_err = NULL;
- crumpled = qdict_crumple(options, errp);
+ crumpled = qdict_crumple_for_keyval_qiv(options, errp);
if (crumpled == NULL) {
return NULL;
}
diff --git a/block/parallels.c b/block/parallels.c
index c1d9643498..695899fc4b 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -653,7 +653,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "parallels");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/block/qcow.c b/block/qcow.c
index 8c08908fd8..860b058240 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -997,7 +997,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "qcow");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/block/qcow2.c b/block/qcow2.c
index d2d955f984..0a27afa2ef 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3152,7 +3152,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
qdict_put_str(qdict, "file", bs->node_name);
/* Now get the QAPI type BlockdevCreateOptions */
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/block/qed.c b/block/qed.c
index 324a953cbc..88fa36d409 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -763,7 +763,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "qed");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/block/rbd.c b/block/rbd.c
index 9659c7361f..09720e97c0 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -647,7 +647,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
}
/* Convert the remaining options into a QAPI object */
- crumpled = qdict_crumple(options, errp);
+ crumpled = qdict_crumple_for_keyval_qiv(options, errp);
if (crumpled == NULL) {
r = -EINVAL;
goto out;
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 2e1f0e6eca..a93f93d360 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -2217,7 +2217,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts,
}
/* Get the QAPI object */
- crumpled = qdict_crumple(qdict, errp);
+ crumpled = qdict_crumple_for_keyval_qiv(qdict, errp);
if (crumpled == NULL) {
ret = -EINVAL;
goto fail;
diff --git a/block/vhdx.c b/block/vhdx.c
index 2e32e24514..78b29d9fc7 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -2005,7 +2005,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "vhdx");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/block/vpc.c b/block/vpc.c
index 41c8c980f1..16178e5a90 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -1119,7 +1119,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "vpc");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple(qdict, errp);
+ qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index fb92010dc5..aba372c2eb 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -9,7 +9,10 @@
#include "qemu/osdep.h"
#include "block/qdict.h"
+#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qlist.h"
+#include "qapi/qmp/qnum.h"
+#include "qapi/qmp/qstring.h"
#include "qemu/cutils.h"
#include "qapi/error.h"
@@ -514,6 +517,60 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
}
/**
+ * qdict_crumple_for_keyval_qiv:
+ * @src: the flat dictionary (only scalar values) to crumple
+ * @errp: location to store error
+ *
+ * Like qdict_crumple(), but additionally transforms scalar values so
+ * the result can be passed to qobject_input_visitor_new_keyval().
+ *
+ * The block subsystem uses this function to prepare its flat QDict
+ * with possibly confused scalar types for a visit. It should not be
+ * used for anything else, and it should go away once the block
+ * subsystem has been cleaned up.
+ */
+QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp)
+{
+ QDict *tmp = NULL;
+ char *buf;
+ const char *s;
+ const QDictEntry *ent;
+ QObject *dst;
+
+ for (ent = qdict_first(src); ent; ent = qdict_next(src, ent)) {
+ buf = NULL;
+ switch (qobject_type(ent->value)) {
+ case QTYPE_QNULL:
+ case QTYPE_QSTRING:
+ continue;
+ case QTYPE_QNUM:
+ s = buf = qnum_to_string(qobject_to(QNum, ent->value));
+ break;
+ case QTYPE_QDICT:
+ case QTYPE_QLIST:
+ /* @src isn't flat; qdict_crumple() will fail */
+ continue;
+ case QTYPE_QBOOL:
+ s = qbool_get_bool(qobject_to(QBool, ent->value))
+ ? "on" : "off";
+ break;
+ default:
+ abort();
+ }
+
+ if (!tmp) {
+ tmp = qdict_clone_shallow(src);
+ }
+ qdict_put(tmp, ent->key, qstring_from_str(s));
+ g_free(buf);
+ }
+
+ dst = qdict_crumple(tmp ?: src, errp);
+ qobject_unref(tmp);
+ return dst;
+}
+
+/**
* qdict_array_entries(): Returns the number of direct array entries if the
* sub-QDict of src specified by the prefix in subqdict (or src itself for
* prefix == "") is valid as an array, i.e. the length of the created list if
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 10/26] block: Fix -drive for certain non-string scalars
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (8 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 09/26] block: Fix -blockdev for certain non-string scalars Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 11/26] block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() Kevin Wolf
` (16 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
The previous commit fixed -blockdev breakage due to misuse of the
qobject input visitor's keyval flavor in bdrv_file_open(). The commit
message explain why using the plain flavor would be just as wrong; it
would break -drive. Turns out we break it in three places:
nbd_open(), sd_open() and ssh_file_open(). They are even marked
FIXME. Example breakage:
$ qemu-system-x86 -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off
qemu-system-x86: -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off: Invalid parameter type for 'numeric', expected: boolean
Fix it the same way: replace qdict_crumple() by
qdict_crumple_for_keyval_qiv(), and switch from plain to the keyval
flavor.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/nbd.c | 12 ++----------
block/sheepdog.c | 12 ++----------
block/ssh.c | 12 ++----------
3 files changed, 6 insertions(+), 30 deletions(-)
diff --git a/block/nbd.c b/block/nbd.c
index d6c4c4ddbc..614dd9fec0 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -273,20 +273,12 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
goto done;
}
- crumpled_addr = qdict_crumple(addr, errp);
+ crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp);
if (!crumpled_addr) {
goto done;
}
- /*
- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive
- * server.type=inet. .to doesn't matter, it's ignored anyway.
- * That's because when @options come from -blockdev or
- * blockdev_add, members are typed according to the QAPI schema,
- * but when they come from -drive, they're all QString. The
- * visitor expects the former.
- */
- iv = qobject_input_visitor_new(crumpled_addr);
+ iv = qobject_input_visitor_new_keyval(crumpled_addr);
visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/sheepdog.c b/block/sheepdog.c
index a93f93d360..29e3e1eaaa 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -546,20 +546,12 @@ static SocketAddress *sd_server_config(QDict *options, Error **errp)
qdict_extract_subqdict(options, &server, "server.");
- crumpled_server = qdict_crumple(server, errp);
+ crumpled_server = qdict_crumple_for_keyval_qiv(server, errp);
if (!crumpled_server) {
goto done;
}
- /*
- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive
- * server.type=inet. .to doesn't matter, it's ignored anyway.
- * That's because when @options come from -blockdev or
- * blockdev_add, members are typed according to the QAPI schema,
- * but when they come from -drive, they're all QString. The
- * visitor expects the former.
- */
- iv = qobject_input_visitor_new(crumpled_server);
+ iv = qobject_input_visitor_new_keyval(crumpled_server);
visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/ssh.c b/block/ssh.c
index eec37dd27c..bd85d989d5 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -623,20 +623,12 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp)
}
/* Create the QAPI object */
- crumpled = qdict_crumple(options, errp);
+ crumpled = qdict_crumple_for_keyval_qiv(options, errp);
if (crumpled == NULL) {
goto fail;
}
- /*
- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive.
- * .to doesn't matter, it's ignored anyway.
- * That's because when @options come from -blockdev or
- * blockdev_add, members are typed according to the QAPI schema,
- * but when they come from -drive, they're all QString. The
- * visitor expects the former.
- */
- v = qobject_input_visitor_new(crumpled);
+ v = qobject_input_visitor_new_keyval(crumpled);
visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err);
visit_free(v);
qobject_unref(crumpled);
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 11/26] block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts()
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (9 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 10/26] block: Fix -drive " Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 12/26] block: Factor out qobject_input_visitor_new_flat_confused() Kevin Wolf
` (15 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
The following pattern occurs in the .bdrv_co_create_opts() methods of
parallels, qcow, qcow2, qed, vhdx and vpc:
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
qobject_unref(qdict);
qdict = qobject_to(QDict, qobj);
if (qdict == NULL) {
ret = -EINVAL;
goto done;
}
v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
[...]
ret = 0;
done:
qobject_unref(qdict);
[...]
return ret;
If qobject_to() fails, we return failure without setting errp. That's
wrong. As far as I can tell, it cannot fail here. Clean it up
anyway, by removing the useless conversion.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/parallels.c | 9 ++++-----
block/qcow.c | 9 ++++-----
block/qcow2.c | 9 ++++-----
block/qed.c | 9 ++++-----
block/vhdx.c | 9 ++++-----
block/vpc.c | 9 ++++-----
6 files changed, 24 insertions(+), 30 deletions(-)
diff --git a/block/parallels.c b/block/parallels.c
index 695899fc4b..ceb7a15d62 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -616,7 +616,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename,
BlockdevCreateOptions *create_options = NULL;
Error *local_err = NULL;
BlockDriverState *bs = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
int ret;
@@ -654,14 +654,13 @@ static int coroutine_fn parallels_co_create_opts(const char *filename,
qdict_put_str(qdict, "file", bs->node_name);
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto done;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qcow.c b/block/qcow.c
index 860b058240..2f81f081fd 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -946,7 +946,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename,
{
BlockdevCreateOptions *create_options = NULL;
BlockDriverState *bs = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
const char *val;
@@ -998,14 +998,13 @@ static int coroutine_fn qcow_co_create_opts(const char *filename,
qdict_put_str(qdict, "file", bs->node_name);
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qcow2.c b/block/qcow2.c
index 0a27afa2ef..8c338661db 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3080,7 +3080,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
Error **errp)
{
BlockdevCreateOptions *create_options = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
@@ -3153,14 +3153,13 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
/* Now get the QAPI type BlockdevCreateOptions */
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto finish;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qed.c b/block/qed.c
index 88fa36d409..fcec760b26 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -722,7 +722,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename,
Error **errp)
{
BlockdevCreateOptions *create_options = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
@@ -764,14 +764,13 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename,
qdict_put_str(qdict, "file", bs->node_name);
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/vhdx.c b/block/vhdx.c
index 78b29d9fc7..f2aec3d2cd 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -1965,7 +1965,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename,
Error **errp)
{
BlockdevCreateOptions *create_options = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
@@ -2006,14 +2006,13 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename,
qdict_put_str(qdict, "file", bs->node_name);
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/vpc.c b/block/vpc.c
index 16178e5a90..a9bb04149d 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -1081,7 +1081,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename,
QemuOpts *opts, Error **errp)
{
BlockdevCreateOptions *create_options = NULL;
- QDict *qdict = NULL;
+ QDict *qdict;
QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
@@ -1120,14 +1120,13 @@ static int coroutine_fn vpc_co_create_opts(const char *filename,
qdict_put_str(qdict, "file", bs->node_name);
qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- qobject_unref(qdict);
- qdict = qobject_to(QDict, qobj);
- if (qdict == NULL) {
+ if (!qobj) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_keyval(qobj);
+ qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 12/26] block: Factor out qobject_input_visitor_new_flat_confused()
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (10 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 11/26] block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 13/26] block: Make remaining uses of qobject input visitor more robust Kevin Wolf
` (14 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/block/qdict.h | 3 ++-
block/nbd.c | 7 ++-----
block/nfs.c | 7 ++-----
block/parallels.c | 7 ++-----
block/qcow.c | 7 ++-----
block/qcow2.c | 7 ++-----
block/qed.c | 7 ++-----
block/rbd.c | 7 ++-----
block/sheepdog.c | 14 ++++----------
block/ssh.c | 7 ++-----
block/vhdx.c | 7 ++-----
block/vpc.c | 7 ++-----
qobject/block-qdict.c | 28 +++++++++++++++++++++++++++-
13 files changed, 53 insertions(+), 62 deletions(-)
diff --git a/include/block/qdict.h b/include/block/qdict.h
index 47d9638c37..d8cb502d7d 100644
--- a/include/block/qdict.h
+++ b/include/block/qdict.h
@@ -21,7 +21,6 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
QObject *qdict_crumple(const QDict *src, Error **errp);
-QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp);
void qdict_flatten(QDict *qdict);
typedef struct QDictRenames {
@@ -30,4 +29,6 @@ typedef struct QDictRenames {
} QDictRenames;
bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp);
+Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict,
+ Error **errp);
#endif
diff --git a/block/nbd.c b/block/nbd.c
index 614dd9fec0..13db4030e6 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -263,7 +263,6 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
{
SocketAddress *saddr = NULL;
QDict *addr = NULL;
- QObject *crumpled_addr = NULL;
Visitor *iv = NULL;
Error *local_err = NULL;
@@ -273,12 +272,11 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
goto done;
}
- crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp);
- if (!crumpled_addr) {
+ iv = qobject_input_visitor_new_flat_confused(addr, errp);
+ if (!iv) {
goto done;
}
- iv = qobject_input_visitor_new_keyval(crumpled_addr);
visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
if (local_err) {
error_propagate(errp, local_err);
@@ -287,7 +285,6 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
done:
qobject_unref(addr);
- qobject_unref(crumpled_addr);
visit_free(iv);
return saddr;
}
diff --git a/block/nfs.c b/block/nfs.c
index 6935b611cc..743ca0450e 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -556,20 +556,17 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options,
Error **errp)
{
BlockdevOptionsNfs *opts = NULL;
- QObject *crumpled = NULL;
Visitor *v;
const QDictEntry *e;
Error *local_err = NULL;
- crumpled = qdict_crumple_for_keyval_qiv(options, errp);
- if (crumpled == NULL) {
+ v = qobject_input_visitor_new_flat_confused(options, errp);
+ if (!v) {
return NULL;
}
- v = qobject_input_visitor_new_keyval(crumpled);
visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err);
visit_free(v);
- qobject_unref(crumpled);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/parallels.c b/block/parallels.c
index ceb7a15d62..fd215e202a 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -617,7 +617,6 @@ static int coroutine_fn parallels_co_create_opts(const char *filename,
Error *local_err = NULL;
BlockDriverState *bs = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
int ret;
@@ -653,14 +652,12 @@ static int coroutine_fn parallels_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "parallels");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto done;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qcow.c b/block/qcow.c
index 2f81f081fd..5532731b9f 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -947,7 +947,6 @@ static int coroutine_fn qcow_co_create_opts(const char *filename,
BlockdevCreateOptions *create_options = NULL;
BlockDriverState *bs = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
const char *val;
Error *local_err = NULL;
@@ -997,14 +996,12 @@ static int coroutine_fn qcow_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "qcow");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qcow2.c b/block/qcow2.c
index 8c338661db..945132f692 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3081,7 +3081,6 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
{
BlockdevCreateOptions *create_options = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
Error *local_err = NULL;
@@ -3152,14 +3151,12 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt
qdict_put_str(qdict, "file", bs->node_name);
/* Now get the QAPI type BlockdevCreateOptions */
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto finish;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/qed.c b/block/qed.c
index fcec760b26..2363814538 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -723,7 +723,6 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename,
{
BlockdevCreateOptions *create_options = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
Error *local_err = NULL;
@@ -763,14 +762,12 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "qed");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/rbd.c b/block/rbd.c
index 09720e97c0..82346a2a5e 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -630,7 +630,6 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
BDRVRBDState *s = bs->opaque;
BlockdevOptionsRbd *opts = NULL;
Visitor *v;
- QObject *crumpled = NULL;
const QDictEntry *e;
Error *local_err = NULL;
char *keypairs, *secretid;
@@ -647,16 +646,14 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
}
/* Convert the remaining options into a QAPI object */
- crumpled = qdict_crumple_for_keyval_qiv(options, errp);
- if (crumpled == NULL) {
+ v = qobject_input_visitor_new_flat_confused(options, errp);
+ if (!v) {
r = -EINVAL;
goto out;
}
- v = qobject_input_visitor_new_keyval(crumpled);
visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err);
visit_free(v);
- qobject_unref(crumpled);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 29e3e1eaaa..665b1763eb 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -539,19 +539,17 @@ static void sd_aio_setup(SheepdogAIOCB *acb, BDRVSheepdogState *s,
static SocketAddress *sd_server_config(QDict *options, Error **errp)
{
QDict *server = NULL;
- QObject *crumpled_server = NULL;
Visitor *iv = NULL;
SocketAddress *saddr = NULL;
Error *local_err = NULL;
qdict_extract_subqdict(options, &server, "server.");
- crumpled_server = qdict_crumple_for_keyval_qiv(server, errp);
- if (!crumpled_server) {
+ iv = qobject_input_visitor_new_flat_confused(server, errp);
+ if (!iv) {
goto done;
}
- iv = qobject_input_visitor_new_keyval(crumpled_server);
visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
if (local_err) {
error_propagate(errp, local_err);
@@ -560,7 +558,6 @@ static SocketAddress *sd_server_config(QDict *options, Error **errp)
done:
visit_free(iv);
- qobject_unref(crumpled_server);
qobject_unref(server);
return saddr;
}
@@ -2173,7 +2170,6 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts,
{
BlockdevCreateOptions *create_options = NULL;
QDict *qdict, *location_qdict;
- QObject *crumpled;
Visitor *v;
char *redundancy;
Error *local_err = NULL;
@@ -2209,16 +2205,14 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts,
}
/* Get the QAPI object */
- crumpled = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (crumpled == NULL) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(crumpled);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
- qobject_unref(crumpled);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/ssh.c b/block/ssh.c
index bd85d989d5..da7bbf73e2 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -606,7 +606,6 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp)
BlockdevOptionsSsh *result = NULL;
QemuOpts *opts = NULL;
Error *local_err = NULL;
- QObject *crumpled;
const QDictEntry *e;
Visitor *v;
@@ -623,15 +622,13 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp)
}
/* Create the QAPI object */
- crumpled = qdict_crumple_for_keyval_qiv(options, errp);
- if (crumpled == NULL) {
+ v = qobject_input_visitor_new_flat_confused(options, errp);
+ if (!v) {
goto fail;
}
- v = qobject_input_visitor_new_keyval(crumpled);
visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err);
visit_free(v);
- qobject_unref(crumpled);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/block/vhdx.c b/block/vhdx.c
index f2aec3d2cd..a677703a9e 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -1966,7 +1966,6 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename,
{
BlockdevCreateOptions *create_options = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
Error *local_err = NULL;
@@ -2005,14 +2004,12 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "vhdx");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/block/vpc.c b/block/vpc.c
index a9bb04149d..bf294abfa7 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -1082,7 +1082,6 @@ static int coroutine_fn vpc_co_create_opts(const char *filename,
{
BlockdevCreateOptions *create_options = NULL;
QDict *qdict;
- QObject *qobj;
Visitor *v;
BlockDriverState *bs = NULL;
Error *local_err = NULL;
@@ -1119,14 +1118,12 @@ static int coroutine_fn vpc_co_create_opts(const char *filename,
qdict_put_str(qdict, "driver", "vpc");
qdict_put_str(qdict, "file", bs->node_name);
- qobj = qdict_crumple_for_keyval_qiv(qdict, errp);
- if (!qobj) {
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
ret = -EINVAL;
goto fail;
}
- v = qobject_input_visitor_new_keyval(qobj);
- qobject_unref(qobj);
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index aba372c2eb..41f39abc4a 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -13,6 +13,7 @@
#include "qapi/qmp/qlist.h"
#include "qapi/qmp/qnum.h"
#include "qapi/qmp/qstring.h"
+#include "qapi/qobject-input-visitor.h"
#include "qemu/cutils.h"
#include "qapi/error.h"
@@ -529,7 +530,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
* used for anything else, and it should go away once the block
* subsystem has been cleaned up.
*/
-QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp)
+static QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp)
{
QDict *tmp = NULL;
char *buf;
@@ -695,3 +696,28 @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
}
return true;
}
+
+/*
+ * Create a QObject input visitor for flat @qdict with possibly
+ * confused scalar types.
+ *
+ * The block subsystem uses this function to visit its flat QDict with
+ * possibly confused scalar types. It should not be used for anything
+ * else, and it should go away once the block subsystem has been
+ * cleaned up.
+ */
+Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict,
+ Error **errp)
+{
+ QObject *crumpled;
+ Visitor *v;
+
+ crumpled = qdict_crumple_for_keyval_qiv(qdict, errp);
+ if (!crumpled) {
+ return NULL;
+ }
+
+ v = qobject_input_visitor_new_keyval(crumpled);
+ qobject_unref(crumpled);
+ return v;
+}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 13/26] block: Make remaining uses of qobject input visitor more robust
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (11 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 12/26] block: Factor out qobject_input_visitor_new_flat_confused() Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 14/26] block-qdict: Simplify qdict_flatten_qdict() Kevin Wolf
` (13 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Remaining uses of qobject_input_visitor_new_keyval() in the block
subsystem:
* block_crypto_open_opts_init()
Currently doesn't visit any non-string scalars, thus safe. It's
called from
- block_crypto_open_luks()
Creates the QDict with qemu_opts_to_qdict_filtered(), which
creates only string scalars, but has a TODO asking for other types.
- qcow_open()
- qcow2_open(), qcow2_co_invalidate_cache(), qcow2_reopen_prepare()
* block_crypto_create_opts_init(), called from
- block_crypto_co_create_opts_luks()
Also creates the QDict with qemu_opts_to_qdict_filtered().
* vdi_co_create_opts()
Also creates the QDict with qemu_opts_to_qdict_filtered().
Replace these uses by qobject_input_visitor_new_flat_confused() for
robustness. This adds crumpling. Right now, that's a no-op, but if
we ever extend these things in non-flat ways, crumpling will be
needed.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/crypto.c | 12 +++++++++---
block/vdi.c | 8 ++++++--
2 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/block/crypto.c b/block/crypto.c
index bc322b50f5..82091c5f70 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -21,11 +21,11 @@
#include "qemu/osdep.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "crypto/block.h"
#include "qapi/opts-visitor.h"
#include "qapi/qapi-visit-crypto.h"
-#include "qapi/qmp/qdict.h"
#include "qapi/qobject-input-visitor.h"
#include "qapi/error.h"
#include "qemu/option.h"
@@ -159,7 +159,10 @@ block_crypto_open_opts_init(QCryptoBlockFormat format,
ret = g_new0(QCryptoBlockOpenOptions, 1);
ret->format = format;
- v = qobject_input_visitor_new_keyval(QOBJECT(opts));
+ v = qobject_input_visitor_new_flat_confused(opts, &local_err);
+ if (local_err) {
+ goto out;
+ }
visit_start_struct(v, NULL, NULL, 0, &local_err);
if (local_err) {
@@ -210,7 +213,10 @@ block_crypto_create_opts_init(QCryptoBlockFormat format,
ret = g_new0(QCryptoBlockCreateOptions, 1);
ret->format = format;
- v = qobject_input_visitor_new_keyval(QOBJECT(opts));
+ v = qobject_input_visitor_new_flat_confused(opts, &local_err);
+ if (local_err) {
+ goto out;
+ }
visit_start_struct(v, NULL, NULL, 0, &local_err);
if (local_err) {
diff --git a/block/vdi.c b/block/vdi.c
index 668af0a828..1d8ed67dbf 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -51,10 +51,10 @@
#include "qemu/osdep.h"
#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
#include "qapi/qobject-input-visitor.h"
#include "qapi/qapi-visit-block-core.h"
#include "block/block_int.h"
+#include "block/qdict.h"
#include "sysemu/block-backend.h"
#include "qemu/module.h"
#include "qemu/option.h"
@@ -934,7 +934,11 @@ static int coroutine_fn vdi_co_create_opts(const char *filename, QemuOpts *opts,
}
/* Get the QAPI object */
- v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ v = qobject_input_visitor_new_flat_confused(qdict, errp);
+ if (!v) {
+ ret = -EINVAL;
+ goto done;
+ }
visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err);
visit_free(v);
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 14/26] block-qdict: Simplify qdict_flatten_qdict()
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (12 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 13/26] block: Make remaining uses of qobject input visitor more robust Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 15/26] block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() Kevin Wolf
` (12 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
There's no need to restart the loop. We don't elsewhere, e.g. in
qdict_extract_subqdict(), qdict_join() and qemu_opts_absorb_qdict().
Simplify accordingly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 18 +++---------------
1 file changed, 3 insertions(+), 15 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index 41f39abc4a..f32df343e8 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -89,16 +89,13 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
QObject *value;
const QDictEntry *entry, *next;
char *new_key;
- bool delete;
entry = qdict_first(qdict);
while (entry != NULL) {
-
next = qdict_next(qdict, entry);
value = qdict_entry_value(entry);
new_key = NULL;
- delete = false;
if (prefix) {
new_key = g_strdup_printf("%s.%s", prefix, entry->key);
@@ -109,27 +106,18 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
* itself disappears. */
qdict_flatten_qdict(qobject_to(QDict, value), target,
new_key ? new_key : entry->key);
- delete = true;
+ qdict_del(qdict, entry->key);
} else if (qobject_type(value) == QTYPE_QLIST) {
qdict_flatten_qlist(qobject_to(QList, value), target,
new_key ? new_key : entry->key);
- delete = true;
+ qdict_del(qdict, entry->key);
} else if (prefix) {
/* All other objects are moved to the target unchanged. */
qdict_put_obj(target, new_key, qobject_ref(value));
- delete = true;
- }
-
- g_free(new_key);
-
- if (delete) {
qdict_del(qdict, entry->key);
-
- /* Restart loop after modifying the iterated QDict */
- entry = qdict_first(qdict);
- continue;
}
+ g_free(new_key);
entry = next;
}
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 15/26] block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist()
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (13 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 14/26] block-qdict: Simplify qdict_flatten_qdict() Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 16/26] block-qdict: Clean up qdict_crumple() a bit Kevin Wolf
` (11 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
qdict_flatten_qdict() skips copying scalars from @qdict to @target
when the two are the same. Fair enough, but it uses a non-obvious
test for "same". Replace it by the obvious one. While there, improve
comments.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index f32df343e8..a4e1c8d08f 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -71,12 +71,15 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
value = qlist_entry_obj(entry);
new_key = g_strdup_printf("%s.%i", prefix, i);
+ /*
+ * Flatten non-empty QDict and QList recursively into @target,
+ * copy other objects to @target
+ */
if (qobject_type(value) == QTYPE_QDICT) {
qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
} else if (qobject_type(value) == QTYPE_QLIST) {
qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
} else {
- /* All other types are moved to the target unchanged. */
qdict_put_obj(target, new_key, qobject_ref(value));
}
@@ -101,9 +104,11 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
new_key = g_strdup_printf("%s.%s", prefix, entry->key);
}
+ /*
+ * Flatten non-empty QDict and QList recursively into @target,
+ * copy other objects to @target
+ */
if (qobject_type(value) == QTYPE_QDICT) {
- /* Entries of QDicts are processed recursively, the QDict object
- * itself disappears. */
qdict_flatten_qdict(qobject_to(QDict, value), target,
new_key ? new_key : entry->key);
qdict_del(qdict, entry->key);
@@ -111,8 +116,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
qdict_flatten_qlist(qobject_to(QList, value), target,
new_key ? new_key : entry->key);
qdict_del(qdict, entry->key);
- } else if (prefix) {
- /* All other objects are moved to the target unchanged. */
+ } else if (target != qdict) {
qdict_put_obj(target, new_key, qobject_ref(value));
qdict_del(qdict, entry->key);
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 16/26] block-qdict: Clean up qdict_crumple() a bit
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (14 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 15/26] block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 17/26] block-qdict: Simplify qdict_is_list() some Kevin Wolf
` (10 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
When you mix scalar and non-scalar keys, whether you get an "already
set as scalar" or an "already set as dict" error depends on qdict
iteration order. Neither message makes much sense. Replace by
""Cannot mix scalar and non-scalar keys". This is similar to the
message we get for mixing list and non-list keys.
I find qdict_crumple()'s first loop hard to understand. Rearrange it
and add a comment.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index a4e1c8d08f..36cf58acc8 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -403,7 +403,7 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
QObject *qdict_crumple(const QDict *src, Error **errp)
{
const QDictEntry *ent;
- QDict *two_level, *multi_level = NULL;
+ QDict *two_level, *multi_level = NULL, *child_dict;
QObject *dst = NULL, *child;
size_t i;
char *prefix = NULL;
@@ -422,28 +422,28 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
}
qdict_split_flat_key(ent->key, &prefix, &suffix);
-
child = qdict_get(two_level, prefix);
+ child_dict = qobject_to(QDict, child);
+
+ if (child) {
+ /*
+ * If @child_dict, then all previous keys with this prefix
+ * had a suffix. If @suffix, this one has one as well,
+ * and we're good, else there's a clash.
+ */
+ if (!child_dict || !suffix) {
+ error_setg(errp, "Cannot mix scalar and non-scalar keys");
+ goto error;
+ }
+ }
+
if (suffix) {
- QDict *child_dict = qobject_to(QDict, child);
if (!child_dict) {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a scalar",
- prefix);
- goto error;
- }
-
child_dict = qdict_new();
- qdict_put_obj(two_level, prefix, QOBJECT(child_dict));
+ qdict_put(two_level, prefix, child_dict);
}
-
qdict_put_obj(child_dict, suffix, qobject_ref(ent->value));
} else {
- if (child) {
- error_setg(errp, "Key %s prefix is already set as a dict",
- prefix);
- goto error;
- }
qdict_put_obj(two_level, prefix, qobject_ref(ent->value));
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 17/26] block-qdict: Simplify qdict_is_list() some
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (15 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 16/26] block-qdict: Clean up qdict_crumple() a bit Kevin Wolf
@ 2018-06-15 14:20 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 18/26] check-block-qdict: Rename qdict_flatten()'s variables for clarity Kevin Wolf
` (9 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:20 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 27 +++++++++++----------------
1 file changed, 11 insertions(+), 16 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index 36cf58acc8..e51a3d2c0f 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -317,27 +317,22 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
for (ent = qdict_first(maybe_list); ent != NULL;
ent = qdict_next(maybe_list, ent)) {
+ int is_index = !qemu_strtoi64(ent->key, NULL, 10, &val);
- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) {
- if (is_list == -1) {
- is_list = 1;
- } else if (!is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
+ if (is_list == -1) {
+ is_list = is_index;
+ }
+
+ if (is_index != is_list) {
+ error_setg(errp, "Cannot mix list and non-list keys");
+ return -1;
+ }
+
+ if (is_index) {
len++;
if (val > max) {
max = val;
}
- } else {
- if (is_list == -1) {
- is_list = 0;
- } else if (is_list) {
- error_setg(errp,
- "Cannot mix list and non-list keys");
- return -1;
- }
}
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 18/26] check-block-qdict: Rename qdict_flatten()'s variables for clarity
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (16 preceding siblings ...)
2018-06-15 14:20 ` [Qemu-devel] [PULL 17/26] block-qdict: Simplify qdict_is_list() some Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 19/26] check-block-qdict: Cover flattening of empty lists and dictionaries Kevin Wolf
` (8 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/check-block-qdict.c | 57 ++++++++++++++++++++++++-----------------------
1 file changed, 29 insertions(+), 28 deletions(-)
diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c
index 5b9f4d506e..29f58a2d3d 100644
--- a/tests/check-block-qdict.c
+++ b/tests/check-block-qdict.c
@@ -37,11 +37,11 @@ static void qdict_defaults_test(void)
static void qdict_flatten_test(void)
{
- QList *list1 = qlist_new();
- QList *list2 = qlist_new();
- QDict *dict1 = qdict_new();
- QDict *dict2 = qdict_new();
- QDict *dict3 = qdict_new();
+ QList *e_1 = qlist_new();
+ QList *e = qlist_new();
+ QDict *e_1_2 = qdict_new();
+ QDict *f = qdict_new();
+ QDict *root = qdict_new();
/*
* Test the flattening of
@@ -79,35 +79,36 @@ static void qdict_flatten_test(void)
* }
*/
- qdict_put_int(dict1, "a", 0);
- qdict_put_int(dict1, "b", 1);
+ qdict_put_int(e_1_2, "a", 0);
+ qdict_put_int(e_1_2, "b", 1);
- qlist_append_int(list1, 23);
- qlist_append_int(list1, 66);
- qlist_append(list1, dict1);
- qlist_append_int(list2, 42);
- qlist_append(list2, list1);
+ qlist_append_int(e_1, 23);
+ qlist_append_int(e_1, 66);
+ qlist_append(e_1, e_1_2);
+ qlist_append_int(e, 42);
+ qlist_append(e, e_1);
- qdict_put_int(dict2, "c", 2);
- qdict_put_int(dict2, "d", 3);
- qdict_put(dict3, "e", list2);
- qdict_put(dict3, "f", dict2);
- qdict_put_int(dict3, "g", 4);
+ qdict_put_int(f, "c", 2);
+ qdict_put_int(f, "d", 3);
- qdict_flatten(dict3);
+ qdict_put(root, "e", e);
+ qdict_put(root, "f", f);
+ qdict_put_int(root, "g", 4);
- g_assert(qdict_get_int(dict3, "e.0") == 42);
- g_assert(qdict_get_int(dict3, "e.1.0") == 23);
- g_assert(qdict_get_int(dict3, "e.1.1") == 66);
- g_assert(qdict_get_int(dict3, "e.1.2.a") == 0);
- g_assert(qdict_get_int(dict3, "e.1.2.b") == 1);
- g_assert(qdict_get_int(dict3, "f.c") == 2);
- g_assert(qdict_get_int(dict3, "f.d") == 3);
- g_assert(qdict_get_int(dict3, "g") == 4);
+ qdict_flatten(root);
- g_assert(qdict_size(dict3) == 8);
+ g_assert(qdict_get_int(root, "e.0") == 42);
+ g_assert(qdict_get_int(root, "e.1.0") == 23);
+ g_assert(qdict_get_int(root, "e.1.1") == 66);
+ g_assert(qdict_get_int(root, "e.1.2.a") == 0);
+ g_assert(qdict_get_int(root, "e.1.2.b") == 1);
+ g_assert(qdict_get_int(root, "f.c") == 2);
+ g_assert(qdict_get_int(root, "f.d") == 3);
+ g_assert(qdict_get_int(root, "g") == 4);
- qobject_unref(dict3);
+ g_assert(qdict_size(root) == 8);
+
+ qobject_unref(root);
}
static void qdict_array_split_test(void)
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 19/26] check-block-qdict: Cover flattening of empty lists and dictionaries
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (17 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 18/26] check-block-qdict: Rename qdict_flatten()'s variables for clarity Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 20/26] block: Fix -blockdev / blockdev-add for empty objects and arrays Kevin Wolf
` (7 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/check-block-qdict.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c
index 29f58a2d3d..2da16f01a6 100644
--- a/tests/check-block-qdict.c
+++ b/tests/check-block-qdict.c
@@ -41,6 +41,8 @@ static void qdict_flatten_test(void)
QList *e = qlist_new();
QDict *e_1_2 = qdict_new();
QDict *f = qdict_new();
+ QList *y = qlist_new();
+ QDict *z = qdict_new();
QDict *root = qdict_new();
/*
@@ -62,7 +64,9 @@ static void qdict_flatten_test(void)
* "c": 2,
* "d": 3,
* },
- * "g": 4
+ * "g": 4,
+ * "y": [{}],
+ * "z": {"a": []}
* }
*
* to
@@ -77,6 +81,8 @@ static void qdict_flatten_test(void)
* "f.d": 3,
* "g": 4
* }
+ *
+ * Note that "y" and "z" get eaten.
*/
qdict_put_int(e_1_2, "a", 0);
@@ -91,9 +97,15 @@ static void qdict_flatten_test(void)
qdict_put_int(f, "c", 2);
qdict_put_int(f, "d", 3);
+ qlist_append(y, qdict_new());
+
+ qdict_put(z, "a", qlist_new());
+
qdict_put(root, "e", e);
qdict_put(root, "f", f);
qdict_put_int(root, "g", 4);
+ qdict_put(root, "y", y);
+ qdict_put(root, "z", z);
qdict_flatten(root);
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 20/26] block: Fix -blockdev / blockdev-add for empty objects and arrays
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (18 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 19/26] check-block-qdict: Cover flattening of empty lists and dictionaries Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 21/26] rbd: New parameter auth-client-required Kevin Wolf
` (6 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
-blockdev and blockdev-add silently ignore empty objects and arrays in
their argument. That's because qmp_blockdev_add() converts the
argument to a flat QDict, and qdict_flatten() eats empty QDict and
QList members. For instance, we ignore an empty BlockdevOptions
member @cache. No real harm, as absent means the same as empty there.
Thus, the flaw puts an artificial restriction on the QAPI schema: we
can't have potentially empty objects and arrays within
BlockdevOptions, except when they're optional and "empty" has the same
meaning as "absent".
Our QAPI schema satisfies this restriction (I checked), but it's a
trap for the unwary, and a temptation to employ awkward workarounds
for the wary. Let's get rid of it.
Change qdict_flatten() and qdict_crumple() to treat empty dictionaries
and lists exactly like scalars.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qobject/block-qdict.c | 54 +++++++++++++++++++++++++++++------------------
tests/check-block-qdict.c | 38 ++++++++++++++++++++++++++-------
2 files changed, 63 insertions(+), 29 deletions(-)
diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index e51a3d2c0f..df833083a7 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -56,6 +56,8 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
{
QObject *value;
const QListEntry *entry;
+ QDict *dict_val;
+ QList *list_val;
char *new_key;
int i;
@@ -69,16 +71,18 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix)
for (i = 0; entry; entry = qlist_next(entry), i++) {
value = qlist_entry_obj(entry);
+ dict_val = qobject_to(QDict, value);
+ list_val = qobject_to(QList, value);
new_key = g_strdup_printf("%s.%i", prefix, i);
/*
* Flatten non-empty QDict and QList recursively into @target,
* copy other objects to @target
*/
- if (qobject_type(value) == QTYPE_QDICT) {
- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key);
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target, new_key);
+ if (dict_val && qdict_size(dict_val)) {
+ qdict_flatten_qdict(dict_val, target, new_key);
+ } else if (list_val && !qlist_empty(list_val)) {
+ qdict_flatten_qlist(list_val, target, new_key);
} else {
qdict_put_obj(target, new_key, qobject_ref(value));
}
@@ -91,6 +95,8 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
{
QObject *value;
const QDictEntry *entry, *next;
+ QDict *dict_val;
+ QList *list_val;
char *new_key;
entry = qdict_first(qdict);
@@ -98,6 +104,8 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
while (entry != NULL) {
next = qdict_next(qdict, entry);
value = qdict_entry_value(entry);
+ dict_val = qobject_to(QDict, value);
+ list_val = qobject_to(QList, value);
new_key = NULL;
if (prefix) {
@@ -108,12 +116,12 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
* Flatten non-empty QDict and QList recursively into @target,
* copy other objects to @target
*/
- if (qobject_type(value) == QTYPE_QDICT) {
- qdict_flatten_qdict(qobject_to(QDict, value), target,
+ if (dict_val && qdict_size(dict_val)) {
+ qdict_flatten_qdict(dict_val, target,
new_key ? new_key : entry->key);
qdict_del(qdict, entry->key);
- } else if (qobject_type(value) == QTYPE_QLIST) {
- qdict_flatten_qlist(qobject_to(QList, value), target,
+ } else if (list_val && !qlist_empty(list_val)) {
+ qdict_flatten_qlist(list_val, target,
new_key ? new_key : entry->key);
qdict_del(qdict, entry->key);
} else if (target != qdict) {
@@ -127,10 +135,11 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix)
}
/**
- * qdict_flatten(): For each nested QDict with key x, all fields with key y
- * are moved to this QDict and their key is renamed to "x.y". For each nested
- * QList with key x, the field at index y is moved to this QDict with the key
- * "x.y" (i.e., the reverse of what qdict_array_split() does).
+ * qdict_flatten(): For each nested non-empty QDict with key x, all
+ * fields with key y are moved to this QDict and their key is renamed
+ * to "x.y". For each nested non-empty QList with key x, the field at
+ * index y is moved to this QDict with the key "x.y" (i.e., the
+ * reverse of what qdict_array_split() does).
* This operation is applied recursively for nested QDicts and QLists.
*/
void qdict_flatten(QDict *qdict)
@@ -361,8 +370,8 @@ static int qdict_is_list(QDict *maybe_list, Error **errp)
* @src: the original flat dictionary (only scalar values) to crumple
*
* Takes a flat dictionary whose keys use '.' separator to indicate
- * nesting, and values are scalars, and crumples it into a nested
- * structure.
+ * nesting, and values are scalars, empty dictionaries or empty lists,
+ * and crumples it into a nested structure.
*
* To include a literal '.' in a key name, it must be escaped as '..'
*
@@ -399,6 +408,8 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
{
const QDictEntry *ent;
QDict *two_level, *multi_level = NULL, *child_dict;
+ QDict *dict_val;
+ QList *list_val;
QObject *dst = NULL, *child;
size_t i;
char *prefix = NULL;
@@ -409,10 +420,11 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
/* Step 1: split our totally flat dict into a two level dict */
for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) {
- if (qobject_type(ent->value) == QTYPE_QDICT ||
- qobject_type(ent->value) == QTYPE_QLIST) {
- error_setg(errp, "Value %s is not a scalar",
- ent->key);
+ dict_val = qobject_to(QDict, ent->value);
+ list_val = qobject_to(QList, ent->value);
+ if ((dict_val && qdict_size(dict_val))
+ || (list_val && !qlist_empty(list_val))) {
+ error_setg(errp, "Value %s is not flat", ent->key);
goto error;
}
@@ -451,9 +463,9 @@ QObject *qdict_crumple(const QDict *src, Error **errp)
multi_level = qdict_new();
for (ent = qdict_first(two_level); ent != NULL;
ent = qdict_next(two_level, ent)) {
- QDict *dict = qobject_to(QDict, ent->value);
- if (dict) {
- child = qdict_crumple(dict, errp);
+ dict_val = qobject_to(QDict, ent->value);
+ if (dict_val && qdict_size(dict_val)) {
+ child = qdict_crumple(dict_val, errp);
if (!child) {
goto error;
}
diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c
index 2da16f01a6..1d20fccbd4 100644
--- a/tests/check-block-qdict.c
+++ b/tests/check-block-qdict.c
@@ -79,10 +79,10 @@ static void qdict_flatten_test(void)
* "e.1.2.b": 1,
* "f.c": 2,
* "f.d": 3,
- * "g": 4
+ * "g": 4,
+ * "y.0": {},
+ * "z.a": []
* }
- *
- * Note that "y" and "z" get eaten.
*/
qdict_put_int(e_1_2, "a", 0);
@@ -117,8 +117,10 @@ static void qdict_flatten_test(void)
g_assert(qdict_get_int(root, "f.c") == 2);
g_assert(qdict_get_int(root, "f.d") == 3);
g_assert(qdict_get_int(root, "g") == 4);
+ g_assert(!qdict_size(qdict_get_qdict(root, "y.0")));
+ g_assert(qlist_empty(qdict_get_qlist(root, "z.a")));
- g_assert(qdict_size(root) == 8);
+ g_assert(qdict_size(root) == 10);
qobject_unref(root);
}
@@ -387,7 +389,8 @@ static void qdict_join_test(void)
static void qdict_crumple_test_recursive(void)
{
QDict *src, *dst, *rule, *vnc, *acl, *listen;
- QList *rules;
+ QDict *empty, *empty_dict, *empty_list_0;
+ QList *rules, *empty_list, *empty_dict_a;
src = qdict_new();
qdict_put_str(src, "vnc.listen.addr", "127.0.0.1");
@@ -399,10 +402,12 @@ static void qdict_crumple_test_recursive(void)
qdict_put_str(src, "vnc.acl.default", "deny");
qdict_put_str(src, "vnc.acl..name", "acl0");
qdict_put_str(src, "vnc.acl.rule..name", "acl0");
+ qdict_put(src, "empty.dict.a", qlist_new());
+ qdict_put(src, "empty.list.0", qdict_new());
dst = qobject_to(QDict, qdict_crumple(src, &error_abort));
g_assert(dst);
- g_assert_cmpint(qdict_size(dst), ==, 1);
+ g_assert_cmpint(qdict_size(dst), ==, 2);
vnc = qdict_get_qdict(dst, "vnc");
g_assert(vnc);
@@ -440,6 +445,21 @@ static void qdict_crumple_test_recursive(void)
g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name"));
g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name"));
+ empty = qdict_get_qdict(dst, "empty");
+ g_assert(empty);
+ g_assert_cmpint(qdict_size(empty), ==, 2);
+ empty_dict = qdict_get_qdict(empty, "dict");
+ g_assert(empty_dict);
+ g_assert_cmpint(qdict_size(empty_dict), ==, 1);
+ empty_dict_a = qdict_get_qlist(empty_dict, "a");
+ g_assert(empty_dict_a && qlist_empty(empty_dict_a));
+ empty_list = qdict_get_qlist(empty, "list");
+ g_assert(empty_list);
+ g_assert_cmpint(qlist_size(empty_list), ==, 1);
+ empty_list_0 = qobject_to(QDict, qlist_pop(empty_list));
+ g_assert(empty_list_0);
+ g_assert_cmpint(qdict_size(empty_list_0), ==, 0);
+
qobject_unref(src);
qobject_unref(dst);
}
@@ -587,7 +607,7 @@ static void qdict_rename_keys_test(void)
static void qdict_crumple_test_bad_inputs(void)
{
- QDict *src;
+ QDict *src, *nested;
Error *error = NULL;
src = qdict_new();
@@ -614,7 +634,9 @@ static void qdict_crumple_test_bad_inputs(void)
src = qdict_new();
/* The input should be flat, ie no dicts or lists */
- qdict_put(src, "rule.a", qdict_new());
+ nested = qdict_new();
+ qdict_put(nested, "x", qdict_new());
+ qdict_put(src, "rule.a", nested);
qdict_put_str(src, "rule.b", "allow");
g_assert(qdict_crumple(src, &error) == NULL);
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 21/26] rbd: New parameter auth-client-required
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (19 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 20/26] block: Fix -blockdev / blockdev-add for empty objects and arrays Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 22/26] rbd: New parameter key-secret Kevin Wolf
` (5 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Parameter auth-client-required lets you configure authentication
methods. We tried to provide that in v2.9.0, but backed out due to
interface design doubts (commit 464444fcc16).
This commit is similar to what we backed out, but simpler: we use a
list of enumeration values instead of a list of objects with a member
of enumeration type.
Let's review our reasons for backing out the first try, as stated in
the commit message:
* The implementation uses deprecated rados_conf_set() key
"auth_supported". No biggie.
Fixed: we use "auth-client-required".
* The implementation makes -drive silently ignore invalid parameters
"auth" and "auth-supported.*.X" where X isn't "auth". Fixable (in
fact I'm going to fix similar bugs around parameter server), so
again no biggie.
That fix is commit 2836284db60. This commit doesn't bring the bugs
back.
* BlockdevOptionsRbd member @password-secret applies only to
authentication method cephx. Should it be a variant member of
RbdAuthMethod?
We've had time to ponder, and we decided to stick to the way Ceph
configuration works: the key configured separately, and silently
ignored if the authentication method doesn't use it.
* BlockdevOptionsRbd member @user could apply to both methods cephx
and none, but I'm not sure it's actually used with none. If it
isn't, should it be a variant member of RbdAuthMethod?
Likewise.
* The client offers a *set* of authentication methods, not a list.
Should the methods be optional members of BlockdevOptionsRbd instead
of members of list @auth-supported? The latter begs the question
what multiple entries for the same method mean. Trivial question
now that RbdAuthMethod contains nothing but @type, but less so when
RbdAuthMethod acquires other members, such the ones discussed above.
Again, we decided to stick to the way Ceph configuration works, except
we make auth-client-required a list of enumeration values instead of a
string containing keywords separated by delimiters.
* How BlockdevOptionsRbd member @auth-supported interacts with
settings from a configuration file specified with @conf is
undocumented. I suspect it's untested, too.
Not actually true, the documentation for @conf says "Values in the
configuration file will be overridden by options specified via QAPI",
and we've tested this.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qapi/block-core.json | 13 +++++++++++++
block/rbd.c | 42 ++++++++++++++++++++++++++++++++----------
2 files changed, 45 insertions(+), 10 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index fff23fc82b..0f68ca56f3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3178,6 +3178,14 @@
##
+# @RbdAuthMode:
+#
+# Since: 3.0
+##
+{ 'enum': 'RbdAuthMode',
+ 'data': [ 'cephx', 'none' ] }
+
+##
# @BlockdevOptionsRbd:
#
# @pool: Ceph pool name.
@@ -3192,6 +3200,10 @@
#
# @user: Ceph id name.
#
+# @auth-client-required: Acceptable authentication modes.
+# This maps to Ceph configuration option
+# "auth_client_required". (Since 3.0)
+#
# @server: Monitor host address and port. This maps
# to the "mon_host" Ceph option.
#
@@ -3203,6 +3215,7 @@
'*conf': 'str',
'*snapshot': 'str',
'*user': 'str',
+ '*auth-client-required': ['RbdAuthMode'],
'*server': ['InetSocketAddressBase'] } }
##
diff --git a/block/rbd.c b/block/rbd.c
index 82346a2a5e..ea0575d068 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -240,20 +240,42 @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp)
static int qemu_rbd_set_auth(rados_t cluster, const char *secretid,
+ BlockdevOptionsRbd *opts,
Error **errp)
{
- if (secretid == 0) {
- return 0;
- }
+ char *acr;
+ int r;
+ GString *accu;
+ RbdAuthModeList *auth;
+
+ if (secretid) {
+ gchar *secret = qcrypto_secret_lookup_as_base64(secretid,
+ errp);
+ if (!secret) {
+ return -1;
+ }
- gchar *secret = qcrypto_secret_lookup_as_base64(secretid,
- errp);
- if (!secret) {
- return -1;
+ rados_conf_set(cluster, "key", secret);
+ g_free(secret);
}
- rados_conf_set(cluster, "key", secret);
- g_free(secret);
+ if (opts->has_auth_client_required) {
+ accu = g_string_new("");
+ for (auth = opts->auth_client_required; auth; auth = auth->next) {
+ if (accu->str[0]) {
+ g_string_append_c(accu, ';');
+ }
+ g_string_append(accu, RbdAuthMode_str(auth->value));
+ }
+ acr = g_string_free(accu, FALSE);
+ r = rados_conf_set(cluster, "auth_client_required", acr);
+ g_free(acr);
+ if (r < 0) {
+ error_setg_errno(errp, -r,
+ "Could not set 'auth_client_required'");
+ return r;
+ }
+ }
return 0;
}
@@ -585,7 +607,7 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
}
}
- if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) {
+ if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) {
r = -EIO;
goto failed_shutdown;
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 22/26] rbd: New parameter key-secret
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (20 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 21/26] rbd: New parameter auth-client-required Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 23/26] block: Remove deprecated -drive geometry options Kevin Wolf
` (4 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Markus Armbruster <armbru@redhat.com>
Legacy -drive supports "password-secret" parameter that isn't
available with -blockdev / blockdev-add. That's because we backed out
our first try to provide it there due to interface design doubts, in
commit 577d8c9a811, v2.9.0.
This is the second try. It brings back the parameter, except it's
named "key-secret" now.
Let's review our reasons for backing out the first try, as stated in
the commit message:
* BlockdevOptionsRbd member @password-secret isn't actually a
password, it's a key generated by Ceph.
Addressed by the rename.
* We're not sure where member @password-secret belongs (see the
previous commit).
See previous commit.
* How @password-secret interacts with settings from a configuration
file specified with @conf is undocumented.
Not actually true, the documentation for @conf says "Values in the
configuration file will be overridden by options specified via QAPI",
and we've tested this.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qapi/block-core.json | 6 ++++++
block/rbd.c | 41 +++++++++++++++++++++++++----------------
2 files changed, 31 insertions(+), 16 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0f68ca56f3..ab629d1647 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3204,6 +3204,11 @@
# This maps to Ceph configuration option
# "auth_client_required". (Since 3.0)
#
+# @key-secret: ID of a QCryptoSecret object providing a key
+# for cephx authentication.
+# This maps to Ceph configuration option
+# "key". (Since 3.0)
+#
# @server: Monitor host address and port. This maps
# to the "mon_host" Ceph option.
#
@@ -3216,6 +3221,7 @@
'*snapshot': 'str',
'*user': 'str',
'*auth-client-required': ['RbdAuthMode'],
+ '*key-secret': 'str',
'*server': ['InetSocketAddressBase'] } }
##
diff --git a/block/rbd.c b/block/rbd.c
index ea0575d068..f2c6965418 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -239,24 +239,25 @@ static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp)
}
-static int qemu_rbd_set_auth(rados_t cluster, const char *secretid,
- BlockdevOptionsRbd *opts,
+static int qemu_rbd_set_auth(rados_t cluster, BlockdevOptionsRbd *opts,
Error **errp)
{
- char *acr;
+ char *key, *acr;
int r;
GString *accu;
RbdAuthModeList *auth;
- if (secretid) {
- gchar *secret = qcrypto_secret_lookup_as_base64(secretid,
- errp);
- if (!secret) {
- return -1;
+ if (opts->key_secret) {
+ key = qcrypto_secret_lookup_as_base64(opts->key_secret, errp);
+ if (!key) {
+ return -EIO;
+ }
+ r = rados_conf_set(cluster, "key", key);
+ g_free(key);
+ if (r < 0) {
+ error_setg_errno(errp, -r, "Could not set 'key'");
+ return r;
}
-
- rados_conf_set(cluster, "key", secret);
- g_free(secret);
}
if (opts->has_auth_client_required) {
@@ -367,9 +368,7 @@ static QemuOptsList runtime_opts = {
},
};
-/* FIXME Deprecate and remove keypairs or make it available in QMP.
- * password_secret should eventually be configurable in opts->location. Support
- * for it in .bdrv_open will make it work here as well. */
+/* FIXME Deprecate and remove keypairs or make it available in QMP. */
static int qemu_rbd_do_create(BlockdevCreateOptions *options,
const char *keypairs, const char *password_secret,
Error **errp)
@@ -575,6 +574,16 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
Error *local_err = NULL;
int r;
+ if (secretid) {
+ if (opts->key_secret) {
+ error_setg(errp,
+ "Legacy 'password-secret' clashes with 'key-secret'");
+ return -EINVAL;
+ }
+ opts->key_secret = g_strdup(secretid);
+ opts->has_key_secret = true;
+ }
+
mon_host = qemu_rbd_mon_host(opts, &local_err);
if (local_err) {
error_propagate(errp, local_err);
@@ -607,8 +616,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
}
}
- if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) {
- r = -EIO;
+ r = qemu_rbd_set_auth(*cluster, opts, errp);
+ if (r < 0) {
goto failed_shutdown;
}
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 23/26] block: Remove deprecated -drive geometry options
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (21 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 22/26] rbd: New parameter key-secret Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 24/26] block: Remove deprecated -drive option addr Kevin Wolf
` (3 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The -drive options cyls, heads, secs and trans were deprecated in
QEMU 2.10. It's time to remove them.
hd-geo-test tested both the old version with geometry options in -drive
and the new one with -device. Therefore the code using -drive doesn't
have to be replaced there, we just need to remove the -drive test cases.
This in turn allows some simplification of the code.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
include/sysemu/blockdev.h | 1 -
blockdev.c | 75 +----------------------------------------------
hw/block/block.c | 14 ---------
tests/hd-geo-test.c | 37 +++++------------------
hmp-commands.hx | 1 -
qemu-doc.texi | 5 ----
qemu-options.hx | 7 +----
7 files changed, 9 insertions(+), 131 deletions(-)
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index ac22f2ae1f..37ea39719e 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -35,7 +35,6 @@ struct DriveInfo {
int auto_del; /* see blockdev_mark_auto_del() */
bool is_default; /* Added by default_drive() ? */
int media_cd;
- int cyls, heads, secs, trans;
QemuOpts *opts;
char *serial;
QTAILQ_ENTRY(DriveInfo) next;
diff --git a/blockdev.c b/blockdev.c
index c24e261e37..bc9f34810f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -731,22 +731,6 @@ QemuOptsList qemu_legacy_drive_opts = {
.type = QEMU_OPT_STRING,
.help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)",
},{
- .name = "cyls",
- .type = QEMU_OPT_NUMBER,
- .help = "number of cylinders (ide disk geometry)",
- },{
- .name = "heads",
- .type = QEMU_OPT_NUMBER,
- .help = "number of heads (ide disk geometry)",
- },{
- .name = "secs",
- .type = QEMU_OPT_NUMBER,
- .help = "number of sectors (ide disk geometry)",
- },{
- .name = "trans",
- .type = QEMU_OPT_STRING,
- .help = "chs translation (auto, lba, none)",
- },{
.name = "addr",
.type = QEMU_OPT_STRING,
.help = "pci address (virtio only)",
@@ -792,7 +776,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
QemuOpts *legacy_opts;
DriveMediaType media = MEDIA_DISK;
BlockInterfaceType type;
- int cyls, heads, secs, translation;
int max_devs, bus_id, unit_id, index;
const char *devaddr;
const char *werror, *rerror;
@@ -803,7 +786,7 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
Error *local_err = NULL;
int i;
const char *deprecated[] = {
- "serial", "trans", "secs", "heads", "cyls", "addr"
+ "serial", "addr"
};
/* Change legacy command line options into QMP ones */
@@ -932,57 +915,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
type = block_default_type;
}
- /* Geometry */
- cyls = qemu_opt_get_number(legacy_opts, "cyls", 0);
- heads = qemu_opt_get_number(legacy_opts, "heads", 0);
- secs = qemu_opt_get_number(legacy_opts, "secs", 0);
-
- if (cyls || heads || secs) {
- if (cyls < 1) {
- error_report("invalid physical cyls number");
- goto fail;
- }
- if (heads < 1) {
- error_report("invalid physical heads number");
- goto fail;
- }
- if (secs < 1) {
- error_report("invalid physical secs number");
- goto fail;
- }
- }
-
- translation = BIOS_ATA_TRANSLATION_AUTO;
- value = qemu_opt_get(legacy_opts, "trans");
- if (value != NULL) {
- if (!cyls) {
- error_report("'%s' trans must be used with cyls, heads and secs",
- value);
- goto fail;
- }
- if (!strcmp(value, "none")) {
- translation = BIOS_ATA_TRANSLATION_NONE;
- } else if (!strcmp(value, "lba")) {
- translation = BIOS_ATA_TRANSLATION_LBA;
- } else if (!strcmp(value, "large")) {
- translation = BIOS_ATA_TRANSLATION_LARGE;
- } else if (!strcmp(value, "rechs")) {
- translation = BIOS_ATA_TRANSLATION_RECHS;
- } else if (!strcmp(value, "auto")) {
- translation = BIOS_ATA_TRANSLATION_AUTO;
- } else {
- error_report("'%s' invalid translation type", value);
- goto fail;
- }
- }
-
- if (media == MEDIA_CDROM) {
- if (cyls || secs || heads) {
- error_report("CHS can't be set with media=cdrom");
- goto fail;
- }
- }
-
/* Device address specified by bus/unit or index.
* If none was specified, try to find the first free one. */
bus_id = qemu_opt_get_number(legacy_opts, "bus", 0);
@@ -1105,11 +1037,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
dinfo = g_malloc0(sizeof(*dinfo));
dinfo->opts = all_opts;
- dinfo->cyls = cyls;
- dinfo->heads = heads;
- dinfo->secs = secs;
- dinfo->trans = translation;
-
dinfo->type = type;
dinfo->bus = bus_id;
dinfo->unit = unit_id;
diff --git a/hw/block/block.c b/hw/block/block.c
index b91e2b6d7e..b6c80ab0b7 100644
--- a/hw/block/block.c
+++ b/hw/block/block.c
@@ -108,20 +108,6 @@ bool blkconf_geometry(BlockConf *conf, int *ptrans,
unsigned cyls_max, unsigned heads_max, unsigned secs_max,
Error **errp)
{
- DriveInfo *dinfo;
-
- if (!conf->cyls && !conf->heads && !conf->secs) {
- /* try to fall back to value set with legacy -drive cyls=... */
- dinfo = blk_legacy_dinfo(conf->blk);
- if (dinfo) {
- conf->cyls = dinfo->cyls;
- conf->heads = dinfo->heads;
- conf->secs = dinfo->secs;
- if (ptrans) {
- *ptrans = dinfo->trans;
- }
- }
- }
if (!conf->cyls && !conf->heads && !conf->secs) {
hd_geometry_guess(conf->blk,
&conf->cyls, &conf->heads, &conf->secs,
diff --git a/tests/hd-geo-test.c b/tests/hd-geo-test.c
index 24870b38f4..ce665f1f83 100644
--- a/tests/hd-geo-test.c
+++ b/tests/hd-geo-test.c
@@ -201,7 +201,7 @@ static void setup_mbr(int img_idx, MBRcontents mbr)
static int setup_ide(int argc, char *argv[], int argv_sz,
int ide_idx, const char *dev, int img_idx,
- MBRcontents mbr, const char *opts)
+ MBRcontents mbr)
{
char *s1, *s2, *s3;
@@ -216,7 +216,7 @@ static int setup_ide(int argc, char *argv[], int argv_sz,
s3 = g_strdup(",media=cdrom");
}
argc = append_arg(argc, argv, argv_sz,
- g_strdup_printf("%s%s%s%s", s1, s2, s3, opts));
+ g_strdup_printf("%s%s%s", s1, s2, s3));
g_free(s1);
g_free(s2);
g_free(s3);
@@ -260,7 +260,7 @@ static void test_ide_mbr(bool use_device, MBRcontents mbr)
for (i = 0; i < backend_last; i++) {
cur_ide[i] = &hd_chst[i][mbr];
dev = use_device ? (is_hd(cur_ide[i]) ? "ide-hd" : "ide-cd") : NULL;
- argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr, "");
+ argc = setup_ide(argc, argv, ARGV_SIZE, i, dev, i, mbr);
}
args = g_strjoinv(" ", argv);
qtest_start(args);
@@ -327,16 +327,12 @@ static void test_ide_drive_user(const char *dev, bool trans)
const CHST expected_chst = { secs / (4 * 32) , 4, 32, trans };
argc = setup_common(argv, ARGV_SIZE);
- opts = g_strdup_printf("%s,%s%scyls=%d,heads=%d,secs=%d",
- dev ?: "",
- trans && dev ? "bios-chs-" : "",
- trans ? "trans=lba," : "",
+ opts = g_strdup_printf("%s,%scyls=%d,heads=%d,secs=%d",
+ dev, trans ? "bios-chs-trans=lba," : "",
expected_chst.cyls, expected_chst.heads,
expected_chst.secs);
cur_ide[0] = &expected_chst;
- argc = setup_ide(argc, argv, ARGV_SIZE,
- 0, dev ? opts : NULL, backend_small, mbr_chs,
- dev ? "" : opts);
+ argc = setup_ide(argc, argv, ARGV_SIZE, 0, opts, backend_small, mbr_chs);
g_free(opts);
args = g_strjoinv(" ", argv);
qtest_start(args);
@@ -347,22 +343,6 @@ static void test_ide_drive_user(const char *dev, bool trans)
}
/*
- * Test case: IDE device (if=ide) with explicit CHS
- */
-static void test_ide_drive_user_chs(void)
-{
- test_ide_drive_user(NULL, false);
-}
-
-/*
- * Test case: IDE device (if=ide) with explicit CHS and translation
- */
-static void test_ide_drive_user_chst(void)
-{
- test_ide_drive_user(NULL, true);
-}
-
-/*
* Test case: IDE device (if=none) with explicit CHS
*/
static void test_ide_device_user_chs(void)
@@ -392,8 +372,7 @@ static void test_ide_drive_cd_0(void)
for (i = 0; i <= backend_empty; i++) {
ide_idx = backend_empty - i;
cur_ide[ide_idx] = &hd_chst[i][mbr_blank];
- argc = setup_ide(argc, argv, ARGV_SIZE,
- ide_idx, NULL, i, mbr_blank, "");
+ argc = setup_ide(argc, argv, ARGV_SIZE, ide_idx, NULL, i, mbr_blank);
}
args = g_strjoinv(" ", argv);
qtest_start(args);
@@ -422,8 +401,6 @@ int main(int argc, char **argv)
qtest_add_func("hd-geo/ide/drive/mbr/blank", test_ide_drive_mbr_blank);
qtest_add_func("hd-geo/ide/drive/mbr/lba", test_ide_drive_mbr_lba);
qtest_add_func("hd-geo/ide/drive/mbr/chs", test_ide_drive_mbr_chs);
- qtest_add_func("hd-geo/ide/drive/user/chs", test_ide_drive_user_chs);
- qtest_add_func("hd-geo/ide/drive/user/chst", test_ide_drive_user_chst);
qtest_add_func("hd-geo/ide/drive/cd_0", test_ide_drive_cd_0);
qtest_add_func("hd-geo/ide/device/mbr/blank", test_ide_device_mbr_blank);
qtest_add_func("hd-geo/ide/device/mbr/lba", test_ide_device_mbr_lba);
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 0734fea931..0de7c4c29e 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1283,7 +1283,6 @@ ETEXI
.params = "[-n] [[<domain>:]<bus>:]<slot>\n"
"[file=file][,if=type][,bus=n]\n"
"[,unit=m][,media=d][,index=i]\n"
- "[,cyls=c,heads=h,secs=s[,trans=t]]\n"
"[,snapshot=on|off][,cache=on|off]\n"
"[,readonly=on|off][,copy-on-read=on|off]",
.help = "add drive to PCI storage controller",
diff --git a/qemu-doc.texi b/qemu-doc.texi
index cd05760cac..ab95bffc74 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -2850,11 +2850,6 @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir''
(for embedded NICs). The new syntax allows different settings to be
provided per NIC.
-@subsection -drive cyls=...,heads=...,secs=...,trans=... (since 2.10.0)
-
-The drive geometry arguments are replaced by the the geometry arguments
-that can be specified with the ``-device'' parameter.
-
@subsection -drive serial=... (since 2.10.0)
The drive serial argument is replaced by the the serial argument
diff --git a/qemu-options.hx b/qemu-options.hx
index c0d3951e9f..a14b9655c5 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -804,9 +804,8 @@ ETEXI
DEF("drive", HAS_ARG, QEMU_OPTION_drive,
"-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n"
- " [,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off]\n"
" [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
- " [,serial=s][,addr=A][,rerror=ignore|stop|report]\n"
+ " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n"
" [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
" [,readonly=on|off][,copy-on-read=on|off]\n"
" [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
@@ -847,10 +846,6 @@ This option defines where is connected the drive by using an index in the list
of available connectors of a given interface type.
@item media=@var{media}
This option defines the type of the media: disk or cdrom.
-@item cyls=@var{c},heads=@var{h},secs=@var{s}[,trans=@var{t}]
-Force disk physical geometry and the optional BIOS translation (trans=none or
-lba). These parameters are deprecated, use the corresponding parameters
-of @code{-device} instead.
@item snapshot=@var{snapshot}
@var{snapshot} is "on" or "off" and controls snapshot mode for the given drive
(see @option{-snapshot}).
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 24/26] block: Remove deprecated -drive option addr
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (22 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 23/26] block: Remove deprecated -drive geometry options Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
` (2 subsequent siblings)
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The -drive option addr was deprecated in QEMU 2.10. It's time to remove
it.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
---
include/sysemu/blockdev.h | 1 -
blockdev.c | 17 +----------------
device-hotplug.c | 4 ----
qemu-doc.texi | 5 -----
qemu-options.hx | 5 +----
5 files changed, 2 insertions(+), 30 deletions(-)
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index 37ea39719e..c0ae3700ec 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -28,7 +28,6 @@ typedef enum {
} BlockInterfaceType;
struct DriveInfo {
- const char *devaddr;
BlockInterfaceType type;
int bus;
int unit;
diff --git a/blockdev.c b/blockdev.c
index bc9f34810f..2984e400c2 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -731,10 +731,6 @@ QemuOptsList qemu_legacy_drive_opts = {
.type = QEMU_OPT_STRING,
.help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)",
},{
- .name = "addr",
- .type = QEMU_OPT_STRING,
- .help = "pci address (virtio only)",
- },{
.name = "serial",
.type = QEMU_OPT_STRING,
.help = "disk serial number",
@@ -777,7 +773,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
DriveMediaType media = MEDIA_DISK;
BlockInterfaceType type;
int max_devs, bus_id, unit_id, index;
- const char *devaddr;
const char *werror, *rerror;
bool read_only = false;
bool copy_on_read;
@@ -786,7 +781,7 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
Error *local_err = NULL;
int i;
const char *deprecated[] = {
- "serial", "addr"
+ "serial"
};
/* Change legacy command line options into QMP ones */
@@ -976,12 +971,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
}
/* Add virtio block device */
- devaddr = qemu_opt_get(legacy_opts, "addr");
- if (devaddr && type != IF_VIRTIO) {
- error_report("addr is not supported by this bus type");
- goto fail;
- }
-
if (type == IF_VIRTIO) {
QemuOpts *devopts;
devopts = qemu_opts_create(qemu_find_opts("device"), NULL, 0,
@@ -993,9 +982,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
}
qemu_opt_set(devopts, "drive", qdict_get_str(bs_opts, "id"),
&error_abort);
- if (devaddr) {
- qemu_opt_set(devopts, "addr", devaddr, &error_abort);
- }
}
filename = qemu_opt_get(legacy_opts, "file");
@@ -1040,7 +1026,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
dinfo->type = type;
dinfo->bus = bus_id;
dinfo->unit = unit_id;
- dinfo->devaddr = devaddr;
dinfo->serial = g_strdup(serial);
blk_set_legacy_dinfo(blk, dinfo);
diff --git a/device-hotplug.c b/device-hotplug.c
index 23fd6656f1..cd427e2c76 100644
--- a/device-hotplug.c
+++ b/device-hotplug.c
@@ -69,10 +69,6 @@ void hmp_drive_add(Monitor *mon, const QDict *qdict)
if (!dinfo) {
goto err;
}
- if (dinfo->devaddr) {
- monitor_printf(mon, "Parameter addr not supported\n");
- goto err;
- }
switch (dinfo->type) {
case IF_NONE:
diff --git a/qemu-doc.texi b/qemu-doc.texi
index ab95bffc74..338477725f 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -2855,11 +2855,6 @@ provided per NIC.
The drive serial argument is replaced by the the serial argument
that can be specified with the ``-device'' parameter.
-@subsection -drive addr=... (since 2.10.0)
-
-The drive addr argument is replaced by the the addr argument
-that can be specified with the ``-device'' parameter.
-
@subsection -usbdevice (since 2.10.0)
The ``-usbdevice DEV'' argument is now a synonym for setting
diff --git a/qemu-options.hx b/qemu-options.hx
index a14b9655c5..c2531e2f3c 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -805,7 +805,7 @@ ETEXI
DEF("drive", HAS_ARG, QEMU_OPTION_drive,
"-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n"
" [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
- " [,snapshot=on|off][,serial=s][,addr=A][,rerror=ignore|stop|report]\n"
+ " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n"
" [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
" [,readonly=on|off][,copy-on-read=on|off]\n"
" [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
@@ -883,9 +883,6 @@ an untrusted format header.
This option specifies the serial number to assign to the device. This
parameter is deprecated, use the corresponding parameter of @code{-device}
instead.
-@item addr=@var{addr}
-Specify the controller's PCI address (if=virtio only). This parameter is
-deprecated, use the corresponding parameter of @code{-device} instead.
@item werror=@var{action},rerror=@var{action}
Specify which @var{action} to take on write and read errors. Valid actions are:
"ignore" (ignore the error and try to continue), "stop" (pause QEMU),
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (23 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 24/26] block: Remove deprecated -drive option addr Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-22 11:38 ` Christian Borntraeger
2018-06-15 14:21 ` [Qemu-devel] [PULL 26/26] block: Remove dead deprecation warning code Kevin Wolf
2018-06-15 16:28 ` [Qemu-devel] [PULL 00/26] Block layer patches Peter Maydell
26 siblings, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The -drive option serial was deprecated in QEMU 2.10. It's time to
remove it.
Tests need to be updated to set the serial number with -global instead
of using the -drive option.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
---
include/hw/block/block.h | 1 -
include/sysemu/blockdev.h | 1 -
block/block-backend.c | 1 -
blockdev.c | 10 ----------
hw/block/block.c | 13 -------------
hw/block/nvme.c | 1 -
hw/block/virtio-blk.c | 1 -
hw/ide/qdev.c | 1 -
hw/scsi/scsi-disk.c | 1 -
hw/usb/dev-storage.c | 1 -
tests/ahci-test.c | 6 +++---
tests/ide-test.c | 8 ++++----
qemu-doc.texi | 5 -----
qemu-options.hx | 6 +-----
14 files changed, 8 insertions(+), 48 deletions(-)
diff --git a/include/hw/block/block.h b/include/hw/block/block.h
index d4f4dfffab..e9f9e2223f 100644
--- a/include/hw/block/block.h
+++ b/include/hw/block/block.h
@@ -72,7 +72,6 @@ static inline unsigned int get_physical_block_exp(BlockConf *conf)
/* Configuration helpers */
-void blkconf_serial(BlockConf *conf, char **serial);
bool blkconf_geometry(BlockConf *conf, int *trans,
unsigned cyls_max, unsigned heads_max, unsigned secs_max,
Error **errp);
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index c0ae3700ec..24954b94e0 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -35,7 +35,6 @@ struct DriveInfo {
bool is_default; /* Added by default_drive() ? */
int media_cd;
QemuOpts *opts;
- char *serial;
QTAILQ_ENTRY(DriveInfo) next;
};
diff --git a/block/block-backend.c b/block/block-backend.c
index d55c328736..2d1a3463e8 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -419,7 +419,6 @@ static void drive_info_del(DriveInfo *dinfo)
return;
}
qemu_opts_del(dinfo->opts);
- g_free(dinfo->serial);
g_free(dinfo);
}
diff --git a/blockdev.c b/blockdev.c
index 2984e400c2..d1ab425085 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -731,10 +731,6 @@ QemuOptsList qemu_legacy_drive_opts = {
.type = QEMU_OPT_STRING,
.help = "interface (ide, scsi, sd, mtd, floppy, pflash, virtio)",
},{
- .name = "serial",
- .type = QEMU_OPT_STRING,
- .help = "disk serial number",
- },{
.name = "file",
.type = QEMU_OPT_STRING,
.help = "file name",
@@ -776,12 +772,10 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
const char *werror, *rerror;
bool read_only = false;
bool copy_on_read;
- const char *serial;
const char *filename;
Error *local_err = NULL;
int i;
const char *deprecated[] = {
- "serial"
};
/* Change legacy command line options into QMP ones */
@@ -949,9 +943,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
goto fail;
}
- /* Serial number */
- serial = qemu_opt_get(legacy_opts, "serial");
-
/* no id supplied -> create one */
if (qemu_opts_id(all_opts) == NULL) {
char *new_id;
@@ -1026,7 +1017,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
dinfo->type = type;
dinfo->bus = bus_id;
dinfo->unit = unit_id;
- dinfo->serial = g_strdup(serial);
blk_set_legacy_dinfo(blk, dinfo);
diff --git a/hw/block/block.c b/hw/block/block.c
index b6c80ab0b7..cf0eb826f1 100644
--- a/hw/block/block.c
+++ b/hw/block/block.c
@@ -15,19 +15,6 @@
#include "qapi/qapi-types-block.h"
#include "qemu/error-report.h"
-void blkconf_serial(BlockConf *conf, char **serial)
-{
- DriveInfo *dinfo;
-
- if (!*serial) {
- /* try to fall back to value set with legacy -drive serial=... */
- dinfo = blk_legacy_dinfo(conf->blk);
- if (dinfo) {
- *serial = g_strdup(dinfo->serial);
- }
- }
-}
-
void blkconf_blocksizes(BlockConf *conf)
{
BlockBackend *blk = conf->blk;
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 811084b6a7..d5bf95b79b 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1215,7 +1215,6 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
return;
}
- blkconf_serial(&n->conf, &n->serial);
if (!n->serial) {
error_setg(errp, "serial property not set");
return;
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 50b5c869e3..225fe44b7a 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -935,7 +935,6 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp)
return;
}
- blkconf_serial(&conf->conf, &conf->serial);
if (!blkconf_apply_backend_options(&conf->conf,
blk_is_read_only(conf->conf.blk), true,
errp)) {
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index f395d24592..573b022e1e 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -188,7 +188,6 @@ static void ide_dev_initfn(IDEDevice *dev, IDEDriveKind kind, Error **errp)
return;
}
- blkconf_serial(&dev->conf, &dev->serial);
if (kind != IDE_CD) {
if (!blkconf_geometry(&dev->conf, &dev->chs_trans, 65535, 16, 255,
errp)) {
diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c
index ded23d36ca..aeaf611854 100644
--- a/hw/scsi/scsi-disk.c
+++ b/hw/scsi/scsi-disk.c
@@ -2368,7 +2368,6 @@ static void scsi_realize(SCSIDevice *dev, Error **errp)
return;
}
- blkconf_serial(&s->qdev.conf, &s->serial);
blkconf_blocksizes(&s->qdev.conf);
if (s->qdev.conf.logical_block_size >
diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c
index 481694a473..47b992f403 100644
--- a/hw/usb/dev-storage.c
+++ b/hw/usb/dev-storage.c
@@ -606,7 +606,6 @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp)
return;
}
- blkconf_serial(&s->conf, &dev->serial);
blkconf_blocksizes(&s->conf);
if (!blkconf_apply_backend_options(&s->conf, blk_is_read_only(blk), true,
errp)) {
diff --git a/tests/ahci-test.c b/tests/ahci-test.c
index 1a7b761304..937ed2f910 100644
--- a/tests/ahci-test.c
+++ b/tests/ahci-test.c
@@ -180,12 +180,12 @@ static AHCIQState *ahci_boot(const char *cli, ...)
s = ahci_vboot(cli, ap);
va_end(ap);
} else {
- cli = "-drive if=none,id=drive0,file=%s,cache=writeback,serial=%s"
- ",format=%s"
+ cli = "-drive if=none,id=drive0,file=%s,cache=writeback,format=%s"
" -M q35 "
"-device ide-hd,drive=drive0 "
+ "-global ide-hd.serial=%s "
"-global ide-hd.ver=%s";
- s = ahci_boot(cli, tmp_path, "testdisk", imgfmt, "version");
+ s = ahci_boot(cli, tmp_path, imgfmt, "testdisk", "version");
}
return s;
diff --git a/tests/ide-test.c b/tests/ide-test.c
index 2384c2c3e2..f39431b1a9 100644
--- a/tests/ide-test.c
+++ b/tests/ide-test.c
@@ -529,8 +529,8 @@ static void test_bmdma_no_busmaster(void)
static void test_bmdma_setup(void)
{
ide_test_start(
- "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw "
- "-global ide-hd.ver=%s",
+ "-drive file=%s,if=ide,cache=writeback,format=raw "
+ "-global ide-hd.serial=%s -global ide-hd.ver=%s",
tmp_path, "testdisk", "version");
qtest_irq_intercept_in(global_qtest, "ioapic");
}
@@ -561,8 +561,8 @@ static void test_identify(void)
int ret;
ide_test_start(
- "-drive file=%s,if=ide,serial=%s,cache=writeback,format=raw "
- "-global ide-hd.ver=%s",
+ "-drive file=%s,if=ide,cache=writeback,format=raw "
+ "-global ide-hd.serial=%s -global ide-hd.ver=%s",
tmp_path, "testdisk", "version");
dev = get_pci_device(&bmdma_bar, &ide_bar);
diff --git a/qemu-doc.texi b/qemu-doc.texi
index 338477725f..282bc3dc35 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -2850,11 +2850,6 @@ with ``-device ...,netdev=x''), or ``-nic user,smb=/some/dir''
(for embedded NICs). The new syntax allows different settings to be
provided per NIC.
-@subsection -drive serial=... (since 2.10.0)
-
-The drive serial argument is replaced by the the serial argument
-that can be specified with the ``-device'' parameter.
-
@subsection -usbdevice (since 2.10.0)
The ``-usbdevice DEV'' argument is now a synonym for setting
diff --git a/qemu-options.hx b/qemu-options.hx
index c2531e2f3c..d5b0c26e8e 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -805,7 +805,7 @@ ETEXI
DEF("drive", HAS_ARG, QEMU_OPTION_drive,
"-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n"
" [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
- " [,snapshot=on|off][,serial=s][,rerror=ignore|stop|report]\n"
+ " [,snapshot=on|off][,rerror=ignore|stop|report]\n"
" [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
" [,readonly=on|off][,copy-on-read=on|off]\n"
" [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
@@ -879,10 +879,6 @@ The default mode is @option{cache=writeback}.
Specify which disk @var{format} will be used rather than detecting
the format. Can be used to specify format=raw to avoid interpreting
an untrusted format header.
-@item serial=@var{serial}
-This option specifies the serial number to assign to the device. This
-parameter is deprecated, use the corresponding parameter of @code{-device}
-instead.
@item werror=@var{action},rerror=@var{action}
Specify which @var{action} to take on write and read errors. Valid actions are:
"ignore" (ignore the error and try to continue), "stop" (pause QEMU),
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* [Qemu-devel] [PULL 26/26] block: Remove dead deprecation warning code
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (24 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
@ 2018-06-15 14:21 ` Kevin Wolf
2018-06-15 16:28 ` [Qemu-devel] [PULL 00/26] Block layer patches Peter Maydell
26 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-15 14:21 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
We removed all options from the 'deprecated' array, so the code is dead
and can be removed as well.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
blockdev.c | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index d1ab425085..7f65cd7497 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -775,8 +775,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
const char *filename;
Error *local_err = NULL;
int i;
- const char *deprecated[] = {
- };
/* Change legacy command line options into QMP ones */
static const struct {
@@ -853,16 +851,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
goto fail;
}
- /* Other deprecated options */
- if (!qtest_enabled()) {
- for (i = 0; i < ARRAY_SIZE(deprecated); i++) {
- if (qemu_opt_get(legacy_opts, deprecated[i]) != NULL) {
- error_report("'%s' is deprecated, please use the corresponding "
- "option of '-device' instead", deprecated[i]);
- }
- }
- }
-
/* Media type */
value = qemu_opt_get(legacy_opts, "media");
if (value) {
--
2.13.6
^ permalink raw reply related [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 00/26] Block layer patches
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
` (25 preceding siblings ...)
2018-06-15 14:21 ` [Qemu-devel] [PULL 26/26] block: Remove dead deprecation warning code Kevin Wolf
@ 2018-06-15 16:28 ` Peter Maydell
26 siblings, 0 replies; 111+ messages in thread
From: Peter Maydell @ 2018-06-15 16:28 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Qemu-block, QEMU Developers
On 15 June 2018 at 15:20, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 91fe7a376ad46e3cc5e82d418aad22173c948a3c:
>
> Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2018-06-15 11:41:44 +0100)
>
> are available in the git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 6266e900b8083945cb766b45c124fb3c42932cb3:
>
> block: Remove dead deprecation warning code (2018-06-15 14:49:44 +0200)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Fix options that work only with -drive or -blockdev, but not with
> both, because of QDict type confusion
> - rbd: Add options 'auth-client-required' and 'key-secret'
> - Remove deprecated -drive options serial/addr/cyls/heads/secs/trans
> - rbd, iscsi: Remove deprecated 'filename' option
> - Fix 'qemu-img map' crash with unaligned image size
> - Improve QMP documentation for jobs
>
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c
2018-06-15 14:20 ` [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c Kevin Wolf
@ 2018-06-19 19:29 ` Eric Blake
0 siblings, 0 replies; 111+ messages in thread
From: Eric Blake @ 2018-06-19 19:29 UTC (permalink / raw)
To: Kevin Wolf, qemu-block; +Cc: qemu-devel, Markus Armbruster
On 06/15/2018 09:20 AM, Kevin Wolf wrote:
> From: Markus Armbruster <armbru@redhat.com>
>
> Pure code motion, except for two brace placements and a comment
> tweaked to appease checkpatch.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> qobject/block-qdict.c | 640 ++++++++++++++++++++++++++++++++++++++++++++
> qobject/qdict.c | 629 --------------------------------------------
> tests/check-block-qdict.c | 655 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/check-qdict.c | 642 ---------------------------------------------
> MAINTAINERS | 2 +
> qobject/Makefile.objs | 1 +
> tests/Makefile.include | 4 +
> 7 files changed, 1302 insertions(+), 1271 deletions(-)
> create mode 100644 qobject/block-qdict.c
> create mode 100644 tests/check-block-qdict.c
and missing a change to tests/.gitignore, so that
tests/check-block-qdict now shows up as an untracked file on an in-tree
build.
(We really should follow through with our threat of renaming all the
tests to use a consistent suffix-based pattern, as it's much easier to
gitignore a suffix than a prefix)
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-15 14:21 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
@ 2018-06-22 11:38 ` Christian Borntraeger
2018-06-22 12:51 ` [Qemu-devel] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial) Christian Borntraeger
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
0 siblings, 2 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 11:38 UTC (permalink / raw)
To: Kevin Wolf, qemu-block; +Cc: qemu-devel, Boris Fiuczynski, libvir-list
On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> The -drive option serial was deprecated in QEMU 2.10. It's time to
> remove it.
>
> Tests need to be updated to set the serial number with -global instead
> of using the -drive option.
libvirt 4.5 still creates those (at least on s390x)
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
<source file='/var/lib/libvirt/qemu/image.zhyp137'/>
<target dev='hda' bus='virtio'/>
<serial>skel</serial>
<boot order='1'/>
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
</disk>
->
[...]
-drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
[...]
2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
2018-06-22 11:25:21.098+0000: shutting down, reason=failed
So it seems that this breaks s390x.
^ permalink raw reply [flat|nested] 111+ messages in thread
* [Qemu-devel] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial)
2018-06-22 11:38 ` Christian Borntraeger
@ 2018-06-22 12:51 ` Christian Borntraeger
2018-06-22 20:08 ` [Qemu-devel] [qemu-s390x] " Thomas Huth
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
1 sibling, 1 reply; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 12:51 UTC (permalink / raw)
To: Kevin Wolf, qemu-block
Cc: qemu-devel, Boris Fiuczynski, libvir-list, qemu-s390x,
Markus Armbruster, Jeff Cody
adding more CC.
On 06/22/2018 01:38 PM, Christian Borntraeger wrote:
>
> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>> remove it.
>>
>> Tests need to be updated to set the serial number with -global instead
>> of using the -drive option.
>
> libvirt 4.5 still creates those (at least on s390x)
>
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> <target dev='hda' bus='virtio'/>
> <serial>skel</serial>
> <boot order='1'/>
> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> </disk>
>
>
> ->
> [...]
> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> [...]
>
> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>
> So it seems that this breaks s390x.
So what about reverting commit b0083267444a5e0f28391f6c2831a539f878d424
"block: Remove deprecated -drive option serial" and redo the removal in
qemu 3.1 (or 3.2) ?
Even if we fix libvirt today, this is certainly a too short period of
time to get things fixed in the field.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 11:38 ` Christian Borntraeger
2018-06-22 12:51 ` [Qemu-devel] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial) Christian Borntraeger
@ 2018-06-22 12:55 ` Kevin Wolf
2018-06-22 13:36 ` Christian Borntraeger
` (2 more replies)
1 sibling, 3 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-22 12:55 UTC (permalink / raw)
To: Christian Borntraeger
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake, pkrempa
Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>
> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > The -drive option serial was deprecated in QEMU 2.10. It's time to
> > remove it.
> >
> > Tests need to be updated to set the serial number with -global instead
> > of using the -drive option.
>
> libvirt 4.5 still creates those (at least on s390x)
>
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> <target dev='hda' bus='virtio'/>
> <serial>skel</serial>
> <boot order='1'/>
> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> </disk>
>
>
> ->
> [...]
> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> [...]
>
> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>
> So it seems that this breaks s390x.
Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
released.
Sadly, it also shows that deprecation warnings in log files go
unnoticed.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
@ 2018-06-22 13:36 ` Christian Borntraeger
2018-06-22 14:00 ` Christian Borntraeger
` (2 more replies)
2018-06-22 14:19 ` Markus Armbruster
2018-06-25 7:16 ` Peter Krempa
2 siblings, 3 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 13:36 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>
>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>> remove it.
>>>
>>> Tests need to be updated to set the serial number with -global instead
>>> of using the -drive option.
>>
>> libvirt 4.5 still creates those (at least on s390x)
>>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>> <target dev='hda' bus='virtio'/>
>> <serial>skel</serial>
>> <boot order='1'/>
>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>> </disk>
>>
>>
>> ->
>> [...]
>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>> [...]
>>
>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>
>> So it seems that this breaks s390x.
To me it seems that this is also broken on x86.
>
> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> released.
I think this is definitely too short notice. We should not break existing
setups just by insisting that users have to update libvirt when they update
QEMU. Yes, this might be our policy, but doing so "just because we can"
is certainly a very bad attitude. I see no fundamental technical reason why
we should not revert this change.
>
> Sadly, it also shows that deprecation warnings in log files go
> unnoticed.
In fact whoever added the deprication notice should have followed up
with the libvirt team to implement that change. no?
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 13:36 ` Christian Borntraeger
@ 2018-06-22 14:00 ` Christian Borntraeger
2018-06-22 14:02 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
2 siblings, 0 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 14:00 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
On 06/22/2018 03:36 PM, Christian Borntraeger wrote:
>
>
> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>>
>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>>> remove it.
>>>>
>>>> Tests need to be updated to set the serial number with -global instead
>>>> of using the -drive option.
>>>
>>> libvirt 4.5 still creates those (at least on s390x)
>>>
>>> <disk type='file' device='disk'>
>>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>>> <target dev='hda' bus='virtio'/>
>>> <serial>skel</serial>
>>> <boot order='1'/>
>>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>>> </disk>
>>>
>>>
>>> ->
>>> [...]
>>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>>> [...]
>>>
>>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>>
>>> So it seems that this breaks s390x.
>
> To me it seems that this is also broken on x86.
>>
>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
>> released.
>
> I think this is definitely too short notice. We should not break existing
> setups just by insisting that users have to update libvirt when they update
> QEMU. Yes, this might be our policy, but doing so "just because we can"
> is certainly a very bad attitude. I see no fundamental technical reason why
> we should not revert this change.
>
>
>>
>> Sadly, it also shows that deprecation warnings in log files go
>> unnoticed.
>
> In fact whoever added the deprication notice should have followed up
> with the libvirt team to implement that change. no?
FWIW cyls, heads, secs and trans also seem to be affected by this.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 13:36 ` Christian Borntraeger
2018-06-22 14:00 ` Christian Borntraeger
@ 2018-06-22 14:02 ` Daniel P. Berrangé
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
2 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-22 14:02 UTC (permalink / raw)
To: Christian Borntraeger
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, qemu-block,
libvir-list, qemu-devel, pkrempa
On Fri, Jun 22, 2018 at 03:36:50PM +0200, Christian Borntraeger wrote:
>
>
> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >>
> >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> >>> remove it.
> >>>
> >>> Tests need to be updated to set the serial number with -global instead
> >>> of using the -drive option.
> >>
> >> libvirt 4.5 still creates those (at least on s390x)
> >>
> >> <disk type='file' device='disk'>
> >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> >> <target dev='hda' bus='virtio'/>
> >> <serial>skel</serial>
> >> <boot order='1'/>
> >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> >> </disk>
> >>
> >>
> >> ->
> >> [...]
> >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> >> [...]
> >>
> >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> >>
> >> So it seems that this breaks s390x.
>
> To me it seems that this is also broken on x86.
Correct, this is not architecture specific.
> > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > released.
>
> I think this is definitely too short notice. We should not break existing
> setups just by insisting that users have to update libvirt when they update
> QEMU. Yes, this might be our policy, but doing so "just because we can"
> is certainly a very bad attitude. I see no fundamental technical reason why
> we should not revert this change.
>
> > Sadly, it also shows that deprecation warnings in log files go
> > unnoticed.
>
> In fact whoever added the deprication notice should have followed up
> with the libvirt team to implement that change. no?
On libvirt side I thought we had already stopped using the deprecated
syntax, but we clearly missed it :-(
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
2018-06-22 13:36 ` Christian Borntraeger
@ 2018-06-22 14:19 ` Markus Armbruster
2018-06-22 14:25 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-25 7:16 ` Peter Krempa
2 siblings, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-06-22 14:19 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, pkrempa
Kevin Wolf <kwolf@redhat.com> writes:
> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>
>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>> > The -drive option serial was deprecated in QEMU 2.10. It's time to
>> > remove it.
>> >
>> > Tests need to be updated to set the serial number with -global instead
>> > of using the -drive option.
>>
>> libvirt 4.5 still creates those (at least on s390x)
>>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>> <target dev='hda' bus='virtio'/>
>> <serial>skel</serial>
>> <boot order='1'/>
>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>> </disk>
>>
>>
>> ->
>> [...]
>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>> [...]
>>
>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>
>> So it seems that this breaks s390x.
>
> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> released.
>
> Sadly, it also shows that deprecation warnings in log files go
> unnoticed.
Nobody reads log files until things have gone belly up, and even then
unrelated log entries get ignored.
The way to get deprecation warnings noticed it to have the management
application fail its "make check".
Perhaps we could use a more structured notification, to make detecting
use of deprecated features programmatically trivial. A QMP event might
do.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 13:36 ` Christian Borntraeger
2018-06-22 14:00 ` Christian Borntraeger
2018-06-22 14:02 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
@ 2018-06-22 14:25 ` Kevin Wolf
2018-06-22 14:31 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
` (3 more replies)
2 siblings, 4 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-06-22 14:25 UTC (permalink / raw)
To: Christian Borntraeger
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>
>
> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >>
> >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> >>> remove it.
> >>>
> >>> Tests need to be updated to set the serial number with -global instead
> >>> of using the -drive option.
> >>
> >> libvirt 4.5 still creates those (at least on s390x)
> >>
> >> <disk type='file' device='disk'>
> >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> >> <target dev='hda' bus='virtio'/>
> >> <serial>skel</serial>
> >> <boot order='1'/>
> >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> >> </disk>
> >>
> >>
> >> ->
> >> [...]
> >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> >> [...]
> >>
> >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> >>
> >> So it seems that this breaks s390x.
>
> To me it seems that this is also broken on x86.
> >
> > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > released.
>
> I think this is definitely too short notice. We should not break existing
> setups just by insisting that users have to update libvirt when they update
> QEMU. Yes, this might be our policy, but doing so "just because we can"
> is certainly a very bad attitude. I see no fundamental technical reason why
> we should not revert this change.
This was in fact one release longer than our deprecation policy says.
Are we serious about the deprecation policy or aren't we?
I might consider reverting a change if it turned out that this requires
some massive work in libvirt. But I think this one should be rather easy
to fix in libvirt until 3.0 is released.
> > Sadly, it also shows that deprecation warnings in log files go
> > unnoticed.
>
> In fact whoever added the deprication notice should have followed up
> with the libvirt team to implement that change. no?
I expect the libvirt developers to read the QEMU Changelog at least for
incompatible changes and deprecations. We can't reasonably go and hunt
for developers for every management tool for QEMU that exists.
And anyway, if you come across a deprecation warning, that's the time
you should act, not only when it finally breaks.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:19 ` Markus Armbruster
@ 2018-06-22 14:25 ` Daniel P. Berrangé
2018-06-22 14:30 ` Daniel P. Berrangé
0 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-22 14:25 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, pkrempa
On Fri, Jun 22, 2018 at 04:19:29PM +0200, Markus Armbruster wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >>
> >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> >> > The -drive option serial was deprecated in QEMU 2.10. It's time to
> >> > remove it.
> >> >
> >> > Tests need to be updated to set the serial number with -global instead
> >> > of using the -drive option.
> >>
> >> libvirt 4.5 still creates those (at least on s390x)
> >>
> >> <disk type='file' device='disk'>
> >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> >> <target dev='hda' bus='virtio'/>
> >> <serial>skel</serial>
> >> <boot order='1'/>
> >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> >> </disk>
> >>
> >>
> >> ->
> >> [...]
> >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> >> [...]
> >>
> >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> >>
> >> So it seems that this breaks s390x.
> >
> > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > released.
> >
> > Sadly, it also shows that deprecation warnings in log files go
> > unnoticed.
>
> Nobody reads log files until things have gone belly up, and even then
> unrelated log entries get ignored.
>
> The way to get deprecation warnings noticed it to have the management
> application fail its "make check".
>
> Perhaps we could use a more structured notification, to make detecting
> use of deprecated features programmatically trivial. A QMP event might
> do.
Libvirt currently has CI that is largely focused on unit testing. We
recently did some work, however, to get our functional test suite
working properly again (Sys-Virt-TCK) and are trying to get some
new CI hardware. So if we get that running, we coud run tests on real
QEMU versions and check the /var/log/libvirt/qemu/$GUEST.logs to
make sure we're not triggering unexpected warnings from QEMU
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:25 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
@ 2018-06-22 14:30 ` Daniel P. Berrangé
2018-06-22 15:00 ` Eric Blake
0 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-22 14:30 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, pkrempa
On Fri, Jun 22, 2018 at 03:25:19PM +0100, Daniel P. Berrangé wrote:
> On Fri, Jun 22, 2018 at 04:19:29PM +0200, Markus Armbruster wrote:
> > Kevin Wolf <kwolf@redhat.com> writes:
> >
> > > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> > >>
> > >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > >> > The -drive option serial was deprecated in QEMU 2.10. It's time to
> > >> > remove it.
> > >> >
> > >> > Tests need to be updated to set the serial number with -global instead
> > >> > of using the -drive option.
> > >>
> > >> libvirt 4.5 still creates those (at least on s390x)
> > >>
> > >> <disk type='file' device='disk'>
> > >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > >> <target dev='hda' bus='virtio'/>
> > >> <serial>skel</serial>
> > >> <boot order='1'/>
> > >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > >> </disk>
> > >>
> > >>
> > >> ->
> > >> [...]
> > >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > >> [...]
> > >>
> > >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> > >>
> > >> So it seems that this breaks s390x.
> > >
> > > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > > released.
> > >
> > > Sadly, it also shows that deprecation warnings in log files go
> > > unnoticed.
> >
> > Nobody reads log files until things have gone belly up, and even then
> > unrelated log entries get ignored.
> >
> > The way to get deprecation warnings noticed it to have the management
> > application fail its "make check".
> >
> > Perhaps we could use a more structured notification, to make detecting
> > use of deprecated features programmatically trivial. A QMP event might
> > do.
>
> Libvirt currently has CI that is largely focused on unit testing. We
> recently did some work, however, to get our functional test suite
> working properly again (Sys-Virt-TCK) and are trying to get some
> new CI hardware. So if we get that running, we coud run tests on real
> QEMU versions and check the /var/log/libvirt/qemu/$GUEST.logs to
> make sure we're not triggering unexpected warnings from QEMU
This could be even easier if there was a --no-deprecations flag to
QEMU which triggered abort() whenever mgmt app uses a deprecated
feature.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
@ 2018-06-22 14:31 ` Daniel P. Berrangé
2018-06-25 9:53 ` Daniel P. Berrangé
2018-06-22 14:38 ` [Qemu-devel] " Christian Borntraeger
` (2 subsequent siblings)
3 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-22 14:31 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Peter Maydell, Boris Fiuczynski,
qemu-block, libvir-list, qemu-devel, pkrempa
On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> >
> >
> > On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> > >>
> > >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> > >>> remove it.
> > >>>
> > >>> Tests need to be updated to set the serial number with -global instead
> > >>> of using the -drive option.
> > >>
> > >> libvirt 4.5 still creates those (at least on s390x)
> > >>
> > >> <disk type='file' device='disk'>
> > >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > >> <target dev='hda' bus='virtio'/>
> > >> <serial>skel</serial>
> > >> <boot order='1'/>
> > >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > >> </disk>
> > >>
> > >>
> > >> ->
> > >> [...]
> > >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > >> [...]
> > >>
> > >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> > >>
> > >> So it seems that this breaks s390x.
> >
> > To me it seems that this is also broken on x86.
> > >
> > > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > > released.
> >
> > I think this is definitely too short notice. We should not break existing
> > setups just by insisting that users have to update libvirt when they update
> > QEMU. Yes, this might be our policy, but doing so "just because we can"
> > is certainly a very bad attitude. I see no fundamental technical reason why
> > we should not revert this change.
>
> This was in fact one release longer than our deprecation policy says.
> Are we serious about the deprecation policy or aren't we?
>
> I might consider reverting a change if it turned out that this requires
> some massive work in libvirt. But I think this one should be rather easy
> to fix in libvirt until 3.0 is released.
It is probably even possible for us to fix it in our July 1st
release
>
> > > Sadly, it also shows that deprecation warnings in log files go
> > > unnoticed.
> >
> > In fact whoever added the deprication notice should have followed up
> > with the libvirt team to implement that change. no?
>
> I expect the libvirt developers to read the QEMU Changelog at least for
> incompatible changes and deprecations. We can't reasonably go and hunt
> for developers for every management tool for QEMU that exists.
Yeah, from libvirt side we need todo a better job of checking this and
filing bugs against libvirt if there's something we tickle.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
2018-06-22 14:31 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
@ 2018-06-22 14:38 ` Christian Borntraeger
2018-06-22 14:47 ` Peter Maydell
2018-06-22 15:01 ` Kevin Wolf
2018-06-22 15:40 ` Daniel P. Berrangé
2018-06-25 7:44 ` Thomas Huth
3 siblings, 2 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 14:38 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
On 06/22/2018 04:25 PM, Kevin Wolf wrote:
> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>>
>>
>> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
>>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>>>
>>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>>>> remove it.
>>>>>
>>>>> Tests need to be updated to set the serial number with -global instead
>>>>> of using the -drive option.
>>>>
>>>> libvirt 4.5 still creates those (at least on s390x)
>>>>
>>>> <disk type='file' device='disk'>
>>>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>>>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>>>> <target dev='hda' bus='virtio'/>
>>>> <serial>skel</serial>
>>>> <boot order='1'/>
>>>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>>>> </disk>
>>>>
>>>>
>>>> ->
>>>> [...]
>>>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>>>> [...]
>>>>
>>>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>>>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>>>
>>>> So it seems that this breaks s390x.
>>
>> To me it seems that this is also broken on x86.
>>>
>>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
>>> released.
>>
>> I think this is definitely too short notice. We should not break existing
>> setups just by insisting that users have to update libvirt when they update
>> QEMU. Yes, this might be our policy, but doing so "just because we can"
>> is certainly a very bad attitude. I see no fundamental technical reason why
>> we should not revert this change.
>
> This was in fact one release longer than our deprecation policy says.
> Are we serious about the deprecation policy or aren't we?
I think it makes more sense to have 2 releases after everything was fixed
instead of 2 releases after it was announced.
So if everyone has adopted we can certainly follow our deprecation policy.
Now if deprecation breaks some real world cases it makes no sense to
"insist" on that deprecation policy. Really: If latest greatest libvirt
does not work 2 weeks before soft freeze I consider this too late.
Why: This breaks MY regression test setup before softfreeze. So I will stop
testing qemu in the most critical point in time.
If you would come up with your statement (taking deprecation policy more
serious than users) in the Linux kernel I can pretty much guarantee that
Linus would call you names.
>
> I might consider reverting a change if it turned out that this requires
> some massive work in libvirt. But I think this one should be rather easy
> to fix in libvirt until 3.0 is released.
I have not heard any reason what we gain by removing these features (and no
I dont believe that this increases your maintenance burden a lot). But it
clearly breaks things.
I suggest: revert the removal patches (all of them cyls,secs,serial etc)
and redo them for 3.1.
>
>>> Sadly, it also shows that deprecation warnings in log files go
>>> unnoticed.
>>
>> In fact whoever added the deprication notice should have followed up
>> with the libvirt team to implement that change. no?
>
> I expect the libvirt developers to read the QEMU Changelog at least for
> incompatible changes and deprecations. We can't reasonably go and hunt
> for developers for every management tool for QEMU that exists.
>
> And anyway, if you come across a deprecation warning, that's the time
> you should act, not only when it finally breaks.
>
> Kevin
>
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:38 ` [Qemu-devel] " Christian Borntraeger
@ 2018-06-22 14:47 ` Peter Maydell
2018-06-22 15:01 ` Kevin Wolf
1 sibling, 0 replies; 111+ messages in thread
From: Peter Maydell @ 2018-06-22 14:47 UTC (permalink / raw)
To: Christian Borntraeger
Cc: Kevin Wolf, Qemu-block, QEMU Developers, Boris Fiuczynski,
Libvirt, Eric Blake, Peter Krempa, Markus Armbruster, Jeff Cody
On 22 June 2018 at 15:38, Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> So if everyone has adopted we can certainly follow our deprecation policy.
> Now if deprecation breaks some real world cases it makes no sense to
> "insist" on that deprecation policy. Really: If latest greatest libvirt
> does not work 2 weeks before soft freeze I consider this too late.
>
> Why: This breaks MY regression test setup before softfreeze. So I will stop
> testing qemu in the most critical point in time.
>
> If you would come up with your statement (taking deprecation policy more
> serious than users) in the Linux kernel I can pretty much guarantee that
> Linus would call you names.
This is one of those areas where I like to think the QEMU
community is a more pleasant place to be than the kernel :-)
The fact we have a deprecation policy at all indicates that we
are (unlike the kernel) sometimes willing to break things that
previously worked for users; but I think we should be a bit
pragmatic as well. If one of our largest use cases (libvirt)
missed the memo on this one I don't think we do anybody any
favours by sticking to the letter of the rules.
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:30 ` Daniel P. Berrangé
@ 2018-06-22 15:00 ` Eric Blake
0 siblings, 0 replies; 111+ messages in thread
From: Eric Blake @ 2018-06-22 15:00 UTC (permalink / raw)
To: Daniel P. Berrangé, Markus Armbruster
Cc: Kevin Wolf, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, pkrempa
On 06/22/2018 09:30 AM, Daniel P. Berrangé wrote:
>>> Perhaps we could use a more structured notification, to make detecting
>>> use of deprecated features programmatically trivial. A QMP event might
>>> do.
>>
>> Libvirt currently has CI that is largely focused on unit testing. We
>> recently did some work, however, to get our functional test suite
>> working properly again (Sys-Virt-TCK) and are trying to get some
>> new CI hardware. So if we get that running, we coud run tests on real
>> QEMU versions and check the /var/log/libvirt/qemu/$GUEST.logs to
>> make sure we're not triggering unexpected warnings from QEMU
>
> This could be even easier if there was a --no-deprecations flag to
> QEMU which triggered abort() whenever mgmt app uses a deprecated
> feature.
Yes, a QMP event (which libvirt could then turn into a hard error if it
ever receives the event) or a qemu command line option to make
deprecated usage fatal (which libvirt would choose to enable) would both
be pragmatic approaches to quickly vetting whether libvirt is using
something that qemu has marked deprecated - provided that we are careful
to always wire up the event/abort into qemu at each location where we
also add a deprecation message. An event might be more flexible than
qemu aborting (as libvirt could make programmatic decisions on whether
to keep going in spite of the event, rather than the guest
unconditionally being lost).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:38 ` [Qemu-devel] " Christian Borntraeger
2018-06-22 14:47 ` Peter Maydell
@ 2018-06-22 15:01 ` Kevin Wolf
2018-06-22 15:50 ` Christian Borntraeger
1 sibling, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-06-22 15:01 UTC (permalink / raw)
To: Christian Borntraeger
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
Am 22.06.2018 um 16:38 hat Christian Borntraeger geschrieben:
>
>
> On 06/22/2018 04:25 PM, Kevin Wolf wrote:
> > Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> >>
> >>
> >> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> >>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >>>>
> >>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> >>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> >>>>> remove it.
> >>>>>
> >>>>> Tests need to be updated to set the serial number with -global instead
> >>>>> of using the -drive option.
> >>>>
> >>>> libvirt 4.5 still creates those (at least on s390x)
> >>>>
> >>>> <disk type='file' device='disk'>
> >>>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> >>>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> >>>> <target dev='hda' bus='virtio'/>
> >>>> <serial>skel</serial>
> >>>> <boot order='1'/>
> >>>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> >>>> </disk>
> >>>>
> >>>>
> >>>> ->
> >>>> [...]
> >>>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> >>>> [...]
> >>>>
> >>>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> >>>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> >>>>
> >>>> So it seems that this breaks s390x.
> >>
> >> To me it seems that this is also broken on x86.
> >>>
> >>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> >>> released.
> >>
> >> I think this is definitely too short notice. We should not break existing
> >> setups just by insisting that users have to update libvirt when they update
> >> QEMU. Yes, this might be our policy, but doing so "just because we can"
> >> is certainly a very bad attitude. I see no fundamental technical reason why
> >> we should not revert this change.
> >
> > This was in fact one release longer than our deprecation policy says.
> > Are we serious about the deprecation policy or aren't we?
>
> I think it makes more sense to have 2 releases after everything was fixed
> instead of 2 releases after it was announced.
This means effectively banning feature removal. The only time that
people actually starting fixing things is when it breaks. So if you
never remove it before everything is fixed, you just never remove it at
all.
It's unfortunate, but breaking things at some point is necessary. I hope
the breakage will only last a few days because libvirt will fix this.
Maybe one thing we could look into for the future is a special
deprecation warning function rather than just error_report(), and we
would make that one fatal in non-release builds so that things break
early, but you can still override it with a ./configure option.
> So if everyone has adopted we can certainly follow our deprecation policy.
> Now if deprecation breaks some real world cases it makes no sense to
> "insist" on that deprecation policy. Really: If latest greatest libvirt
> does not work 2 weeks before soft freeze I consider this too late.
>
> Why: This breaks MY regression test setup before softfreeze. So I will stop
> testing qemu in the most critical point in time.
>
> If you would come up with your statement (taking deprecation policy more
> serious than users) in the Linux kernel I can pretty much guarantee that
> Linus would call you names.
Users shouldn't use random git snapshots. Developers can revert the
change locally until libvirt is fixed.
If contrary to all expectations, libvirt doesn't manage to get this
fixed until 3.0-rc2, I will consider reverting the patch. But not
significantly earlier than that.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
2018-06-22 14:31 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-22 14:38 ` [Qemu-devel] " Christian Borntraeger
@ 2018-06-22 15:40 ` Daniel P. Berrangé
2018-06-22 17:54 ` Kevin Wolf
2018-06-25 10:01 ` Peter Maydell
2018-06-25 7:44 ` Thomas Huth
3 siblings, 2 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-22 15:40 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Peter Maydell, Boris Fiuczynski,
qemu-block, libvir-list, Jeff Cody, qemu-devel,
Markus Armbruster, pkrempa
On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> >
> >
> > On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> > >>
> > >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> > >>> remove it.
> > >>>
> > >>> Tests need to be updated to set the serial number with -global instead
> > >>> of using the -drive option.
> > >>
> > >> libvirt 4.5 still creates those (at least on s390x)
> > >>
> > >> <disk type='file' device='disk'>
> > >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > >> <target dev='hda' bus='virtio'/>
> > >> <serial>skel</serial>
> > >> <boot order='1'/>
> > >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > >> </disk>
> > >>
> > >>
> > >> ->
> > >> [...]
> > >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > >> [...]
> > >>
> > >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> > >>
> > >> So it seems that this breaks s390x.
> >
> > To me it seems that this is also broken on x86.
> > >
> > > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > > released.
> >
> > I think this is definitely too short notice. We should not break existing
> > setups just by insisting that users have to update libvirt when they update
> > QEMU. Yes, this might be our policy, but doing so "just because we can"
> > is certainly a very bad attitude. I see no fundamental technical reason why
> > we should not revert this change.
>
> This was in fact one release longer than our deprecation policy says.
> Are we serious about the deprecation policy or aren't we?
>
> I might consider reverting a change if it turned out that this requires
> some massive work in libvirt. But I think this one should be rather easy
> to fix in libvirt until 3.0 is released.
I've got a patch mostly ready that converts libvirt to setting these things
on the frontend device, however, I've got some queries...
- usb-storage - doesn't appear to support geometry or werror/rerror
Will we loose functionality by stopping use of -drive for werror
Loosing geometry feels relevant too, unless it was already ignored
when set opf -drive ?
- SD card - requires -drive with if=sd, no -device support AFAICT
Will we loose functionality be stopping use of -drive for if=sd, or
is this stuff ignored anyway ?
- ide-cd/scsi-id - doesn't support geometry
Presumably becuase its irrelevant for CDs
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 15:01 ` Kevin Wolf
@ 2018-06-22 15:50 ` Christian Borntraeger
0 siblings, 0 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-06-22 15:50 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, qemu-devel, Boris Fiuczynski, libvir-list, eblake,
pkrempa, Markus Armbruster, Jeff Cody, Peter Maydell
On 06/22/2018 05:01 PM, Kevin Wolf wrote:
> Am 22.06.2018 um 16:38 hat Christian Borntraeger geschrieben:
>>
>>
>> On 06/22/2018 04:25 PM, Kevin Wolf wrote:
>>> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>>>>
>>>>
>>>> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
>>>>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>>>>>
>>>>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>>>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>>>>>> remove it.
>>>>>>>
>>>>>>> Tests need to be updated to set the serial number with -global instead
>>>>>>> of using the -drive option.
>>>>>>
>>>>>> libvirt 4.5 still creates those (at least on s390x)
>>>>>>
>>>>>> <disk type='file' device='disk'>
>>>>>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>>>>>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>>>>>> <target dev='hda' bus='virtio'/>
>>>>>> <serial>skel</serial>
>>>>>> <boot order='1'/>
>>>>>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>>>>>> </disk>
>>>>>>
>>>>>>
>>>>>> ->
>>>>>> [...]
>>>>>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>>>>>> [...]
>>>>>>
>>>>>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>>>>>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>>>>>
>>>>>> So it seems that this breaks s390x.
>>>>
>>>> To me it seems that this is also broken on x86.
>>>>>
>>>>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
>>>>> released.
>>>>
>>>> I think this is definitely too short notice. We should not break existing
>>>> setups just by insisting that users have to update libvirt when they update
>>>> QEMU. Yes, this might be our policy, but doing so "just because we can"
>>>> is certainly a very bad attitude. I see no fundamental technical reason why
>>>> we should not revert this change.
>>>
>>> This was in fact one release longer than our deprecation policy says.
>>> Are we serious about the deprecation policy or aren't we?
>>
>> I think it makes more sense to have 2 releases after everything was fixed
>> instead of 2 releases after it was announced.
>
> This means effectively banning feature removal. The only time that
> people actually starting fixing things is when it breaks. So if you
> never remove it before everything is fixed, you just never remove it at
> all.
With the proposal that is floating around (like the --no-deprecation option
to be used for regression test suites) this could be solved.
>
> It's unfortunate, but breaking things at some point is necessary. I hope
> the breakage will only last a few days because libvirt will fix this.
>
> Maybe one thing we could look into for the future is a special
> deprecation warning function rather than just error_report(), and we
> would make that one fatal in non-release builds so that things break
> early, but you can still override it with a ./configure option.
>
>> So if everyone has adopted we can certainly follow our deprecation policy.
>> Now if deprecation breaks some real world cases it makes no sense to
>> "insist" on that deprecation policy. Really: If latest greatest libvirt
>> does not work 2 weeks before soft freeze I consider this too late.
>>
>> Why: This breaks MY regression test setup before softfreeze. So I will stop
>> testing qemu in the most critical point in time.
>>
>> If you would come up with your statement (taking deprecation policy more
>> serious than users) in the Linux kernel I can pretty much guarantee that
>> Linus would call you names.
>
> Users shouldn't use random git snapshots. Developers can revert the
> change locally until libvirt is fixed.g
>
> If contrary to all expectations, libvirt doesn't manage to get this
> fixed until 3.0-rc2, I will consider reverting the patch. But not
> significantly earlier than that.
I think I made it clear that I consider this wrong on so many levels.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 15:40 ` Daniel P. Berrangé
@ 2018-06-22 17:54 ` Kevin Wolf
2018-06-25 11:18 ` Daniel P. Berrangé
2018-06-25 10:01 ` Peter Maydell
1 sibling, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-06-22 17:54 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Christian Borntraeger, Peter Maydell, Boris Fiuczynski,
qemu-block, libvir-list, Jeff Cody, qemu-devel,
Markus Armbruster, pkrempa
Am 22.06.2018 um 17:40 hat Daniel P. Berrangé geschrieben:
> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > This was in fact one release longer than our deprecation policy says.
> > Are we serious about the deprecation policy or aren't we?
> >
> > I might consider reverting a change if it turned out that this requires
> > some massive work in libvirt. But I think this one should be rather easy
> > to fix in libvirt until 3.0 is released.
>
> I've got a patch mostly ready that converts libvirt to setting these things
> on the frontend device, however, I've got some queries...
>
> - usb-storage - doesn't appear to support geometry or werror/rerror
>
> Will we loose functionality by stopping use of -drive for werror
>
> Loosing geometry feels relevant too, unless it was already ignored
> when set opf -drive ?
You're right, usb-storage doesn't allow using -device to specify these,
and it does use the values from -drive.
For werror/rerror, we should clearly implement the option forwarding to
scsi-disk so that you can make use of it.
I'm not sure how sane specifying a non-standard CHS geometry for a USB
stick actually is. As an additional difficulty, usb-storage internally
creates a scsi-disk device (not scsi-hd), which is also considered
legacy and doesn't support the geometry options either, so it's not just
simple forwarding. We removed an actual feature there, but that feature
was probably never intended nor used.
If someone comes up with a compelling reason why they really need to
configure the CHS geometry of their USB sticks, I guess we can do it. My
real USB sticks I tested don't even support MODE_PAGE_HD_GEOMETRY
(though they have MODE_PAGE_FLEXIBLE_DISK_GEOMETRY).
> - SD card - requires -drive with if=sd, no -device support AFAICT
>
> Will we loose functionality be stopping use of -drive for if=sd, or
> is this stuff ignored anyway ?
This was already silently ignored.
> - ide-cd/scsi-id - doesn't support geometry
>
> Presumably becuase its irrelevant for CDs
Yes, geometry has no effect for CDs.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [qemu-s390x] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial)
2018-06-22 12:51 ` [Qemu-devel] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial) Christian Borntraeger
@ 2018-06-22 20:08 ` Thomas Huth
0 siblings, 0 replies; 111+ messages in thread
From: Thomas Huth @ 2018-06-22 20:08 UTC (permalink / raw)
To: Christian Borntraeger, Kevin Wolf, qemu-block
Cc: Boris Fiuczynski, libvir-list, Jeff Cody, qemu-devel,
Markus Armbruster, qemu-s390x, Max Reitz, Daniel P. Berrange
On 22.06.2018 14:51, Christian Borntraeger wrote:
> adding more CC.
>
> On 06/22/2018 01:38 PM, Christian Borntraeger wrote:
>>
>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>> remove it.
>>>
>>> Tests need to be updated to set the serial number with -global instead
>>> of using the -drive option.
>>
>> libvirt 4.5 still creates those (at least on s390x)
>>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>> <target dev='hda' bus='virtio'/>
>> <serial>skel</serial>
>> <boot order='1'/>
>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>> </disk>
>> ->
>> [...]
>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>> [...]
>>
>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>
>> So it seems that this breaks s390x.
I wonder why nobody noticed the deprecation messages before?
> So what about reverting commit b0083267444a5e0f28391f6c2831a539f878d424
> "block: Remove deprecated -drive option serial" and redo the removal in
> qemu 3.1 (or 3.2) ?
> Even if we fix libvirt today, this is certainly a too short period of
> time to get things fixed in the field.
Agreed, reverting that commit is likely the best thing we can do right
now, and then kill it in a later QEMU release. Note that you also need
to revert 6266e900b8083945cb766b45c124fb3c42932cb3 first.
Thomas
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
2018-06-22 13:36 ` Christian Borntraeger
2018-06-22 14:19 ` Markus Armbruster
@ 2018-06-25 7:16 ` Peter Krempa
2018-06-25 8:23 ` Thomas Huth
2 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-06-25 7:16 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2002 bytes --]
On Fri, Jun 22, 2018 at 14:55:02 +0200, Kevin Wolf wrote:
> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >
> > On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > > The -drive option serial was deprecated in QEMU 2.10. It's time to
> > > remove it.
> > >
> > > Tests need to be updated to set the serial number with -global instead
> > > of using the -drive option.
> >
> > libvirt 4.5 still creates those (at least on s390x)
> >
> > <disk type='file' device='disk'>
> > <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > <target dev='hda' bus='virtio'/>
> > <serial>skel</serial>
> > <boot order='1'/>
> > <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > </disk>
> >
> >
> > ->
> > [...]
> > -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > [...]
> >
> > 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> >
> > So it seems that this breaks s390x.
>
> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> released.
So, I already extracted the code that formats the attributes which
should be actually done for the frontend few months ago, but did not
swithc to it since I did not want to check when everything was added.
The fix will be rather simple if we are certain that the disk serial,
geometry and error policies were supported in qemu 1.5. In that case we
can switch unconditionally to the new format.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
` (2 preceding siblings ...)
2018-06-22 15:40 ` Daniel P. Berrangé
@ 2018-06-25 7:44 ` Thomas Huth
3 siblings, 0 replies; 111+ messages in thread
From: Thomas Huth @ 2018-06-25 7:44 UTC (permalink / raw)
To: Kevin Wolf, Christian Borntraeger
Cc: Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
Jeff Cody, qemu-devel, Markus Armbruster, pkrempa,
Daniel P. Berrange
On 22.06.2018 16:25, Kevin Wolf wrote:
> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>>
>>
>> On 06/22/2018 02:55 PM, Kevin Wolf wrote:
>>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>>>
>>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>>>> remove it.
>>>>>
>>>>> Tests need to be updated to set the serial number with -global instead
>>>>> of using the -drive option.
>>>>
>>>> libvirt 4.5 still creates those (at least on s390x)
>>>>
>>>> <disk type='file' device='disk'>
>>>> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
>>>> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
>>>> <target dev='hda' bus='virtio'/>
>>>> <serial>skel</serial>
>>>> <boot order='1'/>
>>>> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
>>>> </disk>
>>>>
>>>>
>>>> ->
>>>> [...]
>>>> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
>>>> [...]
>>>>
>>>> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
>>>> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
>>>>
>>>> So it seems that this breaks s390x.
>>
>> To me it seems that this is also broken on x86.
>>>
>>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
>>> released.
>>
>> I think this is definitely too short notice. We should not break existing
>> setups just by insisting that users have to update libvirt when they update
>> QEMU. Yes, this might be our policy, but doing so "just because we can"
>> is certainly a very bad attitude. I see no fundamental technical reason why
>> we should not revert this change.
>
> This was in fact one release longer than our deprecation policy says.
Actually, if we assume that the chapter in qemu-doc.texi is the
"official" way to deprecate things, these options are only officially
deprecated since QEMU v2.12, since we missed to add them to the
deprecation chapter earlier:
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=c08d46a9
(We've mentioned them in
https://wiki.qemu.org/ChangeLog/2.10#Deprecated_options already, though)
Thomas
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 7:16 ` Peter Krempa
@ 2018-06-25 8:23 ` Thomas Huth
2018-06-25 9:04 ` Daniel P. Berrangé
0 siblings, 1 reply; 111+ messages in thread
From: Thomas Huth @ 2018-06-25 8:23 UTC (permalink / raw)
To: Peter Krempa, Kevin Wolf
Cc: libvir-list, Christian Borntraeger, Boris Fiuczynski, qemu-devel,
qemu-block, Markus Armbruster
[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]
On 25.06.2018 09:16, Peter Krempa wrote:
> On Fri, Jun 22, 2018 at 14:55:02 +0200, Kevin Wolf wrote:
>> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
>>>
>>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
>>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
>>>> remove it.
>>>>
>>>> Tests need to be updated to set the serial number with -global instead
>>>> of using the -drive option.
[...]
>>> So it seems that this breaks s390x.
>>
>> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
>> released.
>
> So, I already extracted the code that formats the attributes which
> should be actually done for the frontend few months ago, but did not
> swithc to it since I did not want to check when everything was added.
>
> The fix will be rather simple if we are certain that the disk serial,
> geometry and error policies were supported in qemu 1.5. In that case we
> can switch unconditionally to the new format.
The CHS device properties are available since QEMU v1.2:
- IDE: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=ba801960
- SCSI: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=d252df48
- virtio-blk: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=e63e7fde2
The "serial" property and the PCI "addr" property are also available
since 2012 at least, I didn't fully track their origins, but QEMU v1.2
already supported them.
So yes, all these properties should be available in QEMU 1.5 already.
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 8:23 ` Thomas Huth
@ 2018-06-25 9:04 ` Daniel P. Berrangé
0 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-25 9:04 UTC (permalink / raw)
To: Thomas Huth
Cc: Peter Krempa, Kevin Wolf, Boris Fiuczynski, qemu-block,
libvir-list, qemu-devel, Markus Armbruster,
Christian Borntraeger
On Mon, Jun 25, 2018 at 10:23:03AM +0200, Thomas Huth wrote:
> On 25.06.2018 09:16, Peter Krempa wrote:
> > On Fri, Jun 22, 2018 at 14:55:02 +0200, Kevin Wolf wrote:
> >> Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> >>>
> >>> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> >>>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> >>>> remove it.
> >>>>
> >>>> Tests need to be updated to set the serial number with -global instead
> >>>> of using the -drive option.
> [...]
> >>> So it seems that this breaks s390x.
> >>
> >> Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> >> released.
> >
> > So, I already extracted the code that formats the attributes which
> > should be actually done for the frontend few months ago, but did not
> > swithc to it since I did not want to check when everything was added.
> >
> > The fix will be rather simple if we are certain that the disk serial,
> > geometry and error policies were supported in qemu 1.5. In that case we
> > can switch unconditionally to the new format.
>
> The CHS device properties are available since QEMU v1.2:
> - IDE: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=ba801960
> - SCSI: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=d252df48
> - virtio-blk: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=e63e7fde2
>
> The "serial" property and the PCI "addr" property are also available
> since 2012 at least, I didn't fully track their origins, but QEMU v1.2
> already supported them.
>
> So yes, all these properties should be available in QEMU 1.5 already.
The werror/rerror attributes were not added until 2.7.0, so we still
need a capabilities check for them
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 14:31 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
@ 2018-06-25 9:53 ` Daniel P. Berrangé
2018-06-25 11:41 ` Kevin Wolf
0 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-25 9:53 UTC (permalink / raw)
To: Kevin Wolf
Cc: Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, pkrempa
On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> > >
> > >
> > > On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > > > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> > > >>
> > > >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > > >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> > > >>> remove it.
> > > >>>
> > > >>> Tests need to be updated to set the serial number with -global instead
> > > >>> of using the -drive option.
> > > >>
> > > >> libvirt 4.5 still creates those (at least on s390x)
> > > >>
> > > >> <disk type='file' device='disk'>
> > > >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > > >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > > >> <target dev='hda' bus='virtio'/>
> > > >> <serial>skel</serial>
> > > >> <boot order='1'/>
> > > >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > > >> </disk>
> > > >>
> > > >>
> > > >> ->
> > > >> [...]
> > > >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > > >> [...]
> > > >>
> > > >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > > >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> > > >>
> > > >> So it seems that this breaks s390x.
> > >
> > > To me it seems that this is also broken on x86.
> > > >
> > > > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > > > released.
> > >
> > > I think this is definitely too short notice. We should not break existing
> > > setups just by insisting that users have to update libvirt when they update
> > > QEMU. Yes, this might be our policy, but doing so "just because we can"
> > > is certainly a very bad attitude. I see no fundamental technical reason why
> > > we should not revert this change.
> >
> > This was in fact one release longer than our deprecation policy says.
> > Are we serious about the deprecation policy or aren't we?
> >
> > I might consider reverting a change if it turned out that this requires
> > some massive work in libvirt. But I think this one should be rather easy
> > to fix in libvirt until 3.0 is released.
>
> It is probably even possible for us to fix it in our July 1st
> release
Fix posted here:
https://www.redhat.com/archives/libvir-list/2018-June/msg01598.html
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 15:40 ` Daniel P. Berrangé
2018-06-22 17:54 ` Kevin Wolf
@ 2018-06-25 10:01 ` Peter Maydell
2018-06-25 10:31 ` Peter Krempa
1 sibling, 1 reply; 111+ messages in thread
From: Peter Maydell @ 2018-06-25 10:01 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Kevin Wolf, Christian Borntraeger, Boris Fiuczynski, Qemu-block,
Libvirt, Jeff Cody, QEMU Developers, Markus Armbruster,
Peter Krempa
On 22 June 2018 at 16:40, Daniel P. Berrangé <berrange@redhat.com> wrote:
> - SD card - requires -drive with if=sd, no -device support AFAICT
Hmm? You can use -device if you want:
-drive if=none,id=mydrive,... -device sd-card,...,drive=mydrive
(the sd-card device then plugs into the sd controller device
via the sd-bus bus).
Most command lines floating around on the web use if=sd, though
(which doesn't require explicit creation of the sd-card device.)
I think one or two controllers might not yet have been converted
to use sd-bus.
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 10:01 ` Peter Maydell
@ 2018-06-25 10:31 ` Peter Krempa
2018-06-25 10:35 ` Peter Maydell
0 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-06-25 10:31 UTC (permalink / raw)
To: Peter Maydell
Cc: Daniel P. Berrangé,
Kevin Wolf, Christian Borntraeger, Boris Fiuczynski, Qemu-block,
Libvirt, Jeff Cody, QEMU Developers, Markus Armbruster
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
On Mon, Jun 25, 2018 at 11:01:46 +0100, Peter Maydell wrote:
> On 22 June 2018 at 16:40, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > - SD card - requires -drive with if=sd, no -device support AFAICT
>
> Hmm? You can use -device if you want:
> -drive if=none,id=mydrive,... -device sd-card,...,drive=mydrive
> (the sd-card device then plugs into the sd controller device
> via the sd-bus bus).
>
> Most command lines floating around on the web use if=sd, though
> (which doesn't require explicit creation of the sd-card device.)
When I've spoke with Markus during the KVM forum he found out that some
machine types still don't support that, so that why libvirt didn't
bother converting it yet.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 10:31 ` Peter Krempa
@ 2018-06-25 10:35 ` Peter Maydell
0 siblings, 0 replies; 111+ messages in thread
From: Peter Maydell @ 2018-06-25 10:35 UTC (permalink / raw)
To: Peter Krempa
Cc: Daniel P. Berrangé,
Kevin Wolf, Christian Borntraeger, Boris Fiuczynski, Qemu-block,
Libvirt, Jeff Cody, QEMU Developers, Markus Armbruster
On 25 June 2018 at 11:31, Peter Krempa <pkrempa@redhat.com> wrote:
> On Mon, Jun 25, 2018 at 11:01:46 +0100, Peter Maydell wrote:
>> On 22 June 2018 at 16:40, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> > - SD card - requires -drive with if=sd, no -device support AFAICT
>>
>> Hmm? You can use -device if you want:
>> -drive if=none,id=mydrive,... -device sd-card,...,drive=mydrive
>> (the sd-card device then plugs into the sd controller device
>> via the sd-bus bus).
>>
>> Most command lines floating around on the web use if=sd, though
>> (which doesn't require explicit creation of the sd-card device.)
>
> When I've spoke with Markus during the KVM forum he found out that some
> machine types still don't support that, so that why libvirt didn't
> bother converting it yet.
Yes, it's a bit of a stalled-in-progress transition...
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-22 17:54 ` Kevin Wolf
@ 2018-06-25 11:18 ` Daniel P. Berrangé
0 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-06-25 11:18 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Peter Maydell, Boris Fiuczynski,
qemu-block, libvir-list, Jeff Cody, qemu-devel,
Markus Armbruster, pkrempa
On Fri, Jun 22, 2018 at 07:54:00PM +0200, Kevin Wolf wrote:
> Am 22.06.2018 um 17:40 hat Daniel P. Berrangé geschrieben:
> > On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > > This was in fact one release longer than our deprecation policy says.
> > > Are we serious about the deprecation policy or aren't we?
> > >
> > > I might consider reverting a change if it turned out that this requires
> > > some massive work in libvirt. But I think this one should be rather easy
> > > to fix in libvirt until 3.0 is released.
> >
> > I've got a patch mostly ready that converts libvirt to setting these things
> > on the frontend device, however, I've got some queries...
> >
> > - usb-storage - doesn't appear to support geometry or werror/rerror
> >
> > Will we loose functionality by stopping use of -drive for werror
> >
> > Loosing geometry feels relevant too, unless it was already ignored
> > when set opf -drive ?
>
> You're right, usb-storage doesn't allow using -device to specify these,
> and it does use the values from -drive.
>
> For werror/rerror, we should clearly implement the option forwarding to
> scsi-disk so that you can make use of it.
This missing werror item looks like a blocker for libvirt to be
able to switch to using -blockdev without a regression.
> I'm not sure how sane specifying a non-standard CHS geometry for a USB
> stick actually is. As an additional difficulty, usb-storage internally
> creates a scsi-disk device (not scsi-hd), which is also considered
> legacy and doesn't support the geometry options either, so it's not just
> simple forwarding. We removed an actual feature there, but that feature
> was probably never intended nor used.
>
> If someone comes up with a compelling reason why they really need to
> configure the CHS geometry of their USB sticks, I guess we can do it. My
> real USB sticks I tested don't even support MODE_PAGE_HD_GEOMETRY
> (though they have MODE_PAGE_FLEXIBLE_DISK_GEOMETRY).
I don't think libvirt has a compelling reason. The ability to set CHS
on usb-storage is just something we got for free because it used the
same -drive syntax as other disk device frontends. It would be nice to
avoid the regression but I doubt people will actually notice in practice
as usb-storage users are few & far between, and even fewer of those will
care about CHS.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 9:53 ` Daniel P. Berrangé
@ 2018-06-25 11:41 ` Kevin Wolf
2018-06-25 11:45 ` Peter Krempa
0 siblings, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-06-25 11:41 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, pkrempa
Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> > On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > > Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> > > >
> > > >
> > > > On 06/22/2018 02:55 PM, Kevin Wolf wrote:
> > > > > Am 22.06.2018 um 13:38 hat Christian Borntraeger geschrieben:
> > > > >>
> > > > >> On 06/15/2018 04:21 PM, Kevin Wolf wrote:
> > > > >>> The -drive option serial was deprecated in QEMU 2.10. It's time to
> > > > >>> remove it.
> > > > >>>
> > > > >>> Tests need to be updated to set the serial number with -global instead
> > > > >>> of using the -drive option.
> > > > >>
> > > > >> libvirt 4.5 still creates those (at least on s390x)
> > > > >>
> > > > >> <disk type='file' device='disk'>
> > > > >> <driver name='qemu' type='qcow2' cache='none' io='native' iothread='1'/>
> > > > >> <source file='/var/lib/libvirt/qemu/image.zhyp137'/>
> > > > >> <target dev='hda' bus='virtio'/>
> > > > >> <serial>skel</serial>
> > > > >> <boot order='1'/>
> > > > >> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
> > > > >> </disk>
> > > > >>
> > > > >>
> > > > >> ->
> > > > >> [...]
> > > > >> -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native -device virtio-blk-ccw,iothread=iothread1,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
> > > > >> [...]
> > > > >>
> > > > >> 2018-06-22T11:25:20.946024Z qemu-system-s390x: -drive file=/var/lib/libvirt/qemu/image.zhyp137,format=qcow2,if=none,id=drive-virtio-disk0,serial=skel,cache=none,aio=native: Block format 'qcow2' does not support the option 'serial'
> > > > >> 2018-06-22 11:25:21.098+0000: shutting down, reason=failed
> > > > >>
> > > > >> So it seems that this breaks s390x.
> > > >
> > > > To me it seems that this is also broken on x86.
> > > > >
> > > > > Thanks for bringing this up. libvirt should fix this before QEMU 3.0 is
> > > > > released.
> > > >
> > > > I think this is definitely too short notice. We should not break existing
> > > > setups just by insisting that users have to update libvirt when they update
> > > > QEMU. Yes, this might be our policy, but doing so "just because we can"
> > > > is certainly a very bad attitude. I see no fundamental technical reason why
> > > > we should not revert this change.
> > >
> > > This was in fact one release longer than our deprecation policy says.
> > > Are we serious about the deprecation policy or aren't we?
> > >
> > > I might consider reverting a change if it turned out that this requires
> > > some massive work in libvirt. But I think this one should be rather easy
> > > to fix in libvirt until 3.0 is released.
> >
> > It is probably even possible for us to fix it in our July 1st
> > release
>
> Fix posted here:
>
> https://www.redhat.com/archives/libvir-list/2018-June/msg01598.html
Thanks!
I'll look into werror/rerror support for usb-storage. It shouldn't be
too hard, though it's strictly speaking a separate problem related to
using -blockdev rather than option deprecation.
If Peter wants to wait for QEMU support before converting werror/rerror
to -device, maybe it would make sense to split your patch for v2 so that
geometry and serial can get fixed right away?
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 11:41 ` Kevin Wolf
@ 2018-06-25 11:45 ` Peter Krempa
2018-07-02 8:04 ` Kevin Wolf
0 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-06-25 11:45 UTC (permalink / raw)
To: Kevin Wolf
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]
On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
> Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> > On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> > > On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > > > Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
[...]
>
> Thanks!
>
> I'll look into werror/rerror support for usb-storage. It shouldn't be
> too hard, though it's strictly speaking a separate problem related to
> using -blockdev rather than option deprecation.
>
> If Peter wants to wait for QEMU support before converting werror/rerror
Definitely. I don't want to keep around yet another hack that will
satisfy one specific case and then add another capability for it. We
should then gate the moving of the feature based on the presence of
werror for usb-storage.
> to -device, maybe it would make sense to split your patch for v2 so that
> geometry and serial can get fixed right away?
Yes this can be done right away.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-06-25 11:45 ` Peter Krempa
@ 2018-07-02 8:04 ` Kevin Wolf
2018-07-03 10:53 ` Christian Borntraeger
0 siblings, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-07-02 8:04 UTC (permalink / raw)
To: Peter Krempa
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]
Am 25.06.2018 um 13:45 hat Peter Krempa geschrieben:
> On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
> > Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> > > On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> > > > On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > > > > Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>
> [...]
>
> >
> > Thanks!
> >
> > I'll look into werror/rerror support for usb-storage. It shouldn't be
> > too hard, though it's strictly speaking a separate problem related to
> > using -blockdev rather than option deprecation.
> >
> > If Peter wants to wait for QEMU support before converting werror/rerror
>
> Definitely. I don't want to keep around yet another hack that will
> satisfy one specific case and then add another capability for it. We
> should then gate the moving of the feature based on the presence of
> werror for usb-storage.
>
> > to -device, maybe it would make sense to split your patch for v2 so that
> > geometry and serial can get fixed right away?
>
> Yes this can be done right away.
Has serial/gemoetry been fixed meanwhile and will it make it into the
next release?
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-02 8:04 ` Kevin Wolf
@ 2018-07-03 10:53 ` Christian Borntraeger
2018-07-03 11:22 ` Daniel P. Berrangé
0 siblings, 1 reply; 111+ messages in thread
From: Christian Borntraeger @ 2018-07-03 10:53 UTC (permalink / raw)
To: Kevin Wolf, Peter Krempa
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel
On 07/02/2018 10:04 AM, Kevin Wolf wrote:
> Am 25.06.2018 um 13:45 hat Peter Krempa geschrieben:
>> On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
>>> Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
>>>> On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
>>>>> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
>>>>>> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
>>
>> [...]
>>
>>>
>>> Thanks!
>>>
>>> I'll look into werror/rerror support for usb-storage. It shouldn't be
>>> too hard, though it's strictly speaking a separate problem related to
>>> using -blockdev rather than option deprecation.
>>>
>>> If Peter wants to wait for QEMU support before converting werror/rerror
>>
>> Definitely. I don't want to keep around yet another hack that will
>> satisfy one specific case and then add another capability for it. We
>> should then gate the moving of the feature based on the presence of
>> werror for usb-storage.
>>
>>> to -device, maybe it would make sense to split your patch for v2 so that
>>> geometry and serial can get fixed right away?
>>
>> Yes this can be done right away.
>
> Has serial/gemoetry been fixed meanwhile and will it make it into the
> next release?
I cannot find an archive that has it, but it is on the libvirt mailing
list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
Review seems done, but it has missed libvirt 4.5 which was released today.
Christian
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 10:53 ` Christian Borntraeger
@ 2018-07-03 11:22 ` Daniel P. Berrangé
2018-07-03 11:32 ` Kevin Wolf
0 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-03 11:22 UTC (permalink / raw)
To: Christian Borntraeger
Cc: Kevin Wolf, Peter Krempa, Peter Maydell, Boris Fiuczynski,
qemu-block, libvir-list, qemu-devel
On Tue, Jul 03, 2018 at 12:53:44PM +0200, Christian Borntraeger wrote:
>
>
> On 07/02/2018 10:04 AM, Kevin Wolf wrote:
> > Am 25.06.2018 um 13:45 hat Peter Krempa geschrieben:
> >> On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
> >>> Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> >>>> On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> >>>>> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> >>>>>> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> >>
> >> [...]
> >>
> >>>
> >>> Thanks!
> >>>
> >>> I'll look into werror/rerror support for usb-storage. It shouldn't be
> >>> too hard, though it's strictly speaking a separate problem related to
> >>> using -blockdev rather than option deprecation.
> >>>
> >>> If Peter wants to wait for QEMU support before converting werror/rerror
> >>
> >> Definitely. I don't want to keep around yet another hack that will
> >> satisfy one specific case and then add another capability for it. We
> >> should then gate the moving of the feature based on the presence of
> >> werror for usb-storage.
> >>
> >>> to -device, maybe it would make sense to split your patch for v2 so that
> >>> geometry and serial can get fixed right away?
> >>
> >> Yes this can be done right away.
> >
> > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > next release?
>
> I cannot find an archive that has it, but it is on the libvirt mailing
> list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> Review seems done, but it has missed libvirt 4.5 which was released today.
Just posted latest version here:
https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
It will be in the next release on ~ Aug 1st
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 11:22 ` Daniel P. Berrangé
@ 2018-07-03 11:32 ` Kevin Wolf
2018-07-03 11:35 ` Peter Maydell
` (2 more replies)
0 siblings, 3 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-07-03 11:32 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Christian Borntraeger, Peter Krempa, Peter Maydell,
Boris Fiuczynski, qemu-block, libvir-list, qemu-devel
Am 03.07.2018 um 13:22 hat Daniel P. Berrangé geschrieben:
> On Tue, Jul 03, 2018 at 12:53:44PM +0200, Christian Borntraeger wrote:
> >
> >
> > On 07/02/2018 10:04 AM, Kevin Wolf wrote:
> > > Am 25.06.2018 um 13:45 hat Peter Krempa geschrieben:
> > >> On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
> > >>> Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> > >>>> On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> > >>>>> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > >>>>>> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> > >>
> > >> [...]
> > >>
> > >>>
> > >>> Thanks!
> > >>>
> > >>> I'll look into werror/rerror support for usb-storage. It shouldn't be
> > >>> too hard, though it's strictly speaking a separate problem related to
> > >>> using -blockdev rather than option deprecation.
> > >>>
> > >>> If Peter wants to wait for QEMU support before converting werror/rerror
> > >>
> > >> Definitely. I don't want to keep around yet another hack that will
> > >> satisfy one specific case and then add another capability for it. We
> > >> should then gate the moving of the feature based on the presence of
> > >> werror for usb-storage.
> > >>
> > >>> to -device, maybe it would make sense to split your patch for v2 so that
> > >>> geometry and serial can get fixed right away?
> > >>
> > >> Yes this can be done right away.
> > >
> > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > next release?
> >
> > I cannot find an archive that has it, but it is on the libvirt mailing
> > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > Review seems done, but it has missed libvirt 4.5 which was released today.
>
> Just posted latest version here:
>
> https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
>
> It will be in the next release on ~ Aug 1st
It would have been a lot nicer to have it the July release because this
means that we'll have the released libvirt broken during almost the
whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
earliest, so I guess we're still okay. People using QEMU from git will
just need libvirt from git as well.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 11:32 ` Kevin Wolf
@ 2018-07-03 11:35 ` Peter Maydell
2018-07-03 12:38 ` Christian Borntraeger
2018-07-03 11:35 ` Daniel P. Berrangé
2018-07-04 13:02 ` Cornelia Huck
2 siblings, 1 reply; 111+ messages in thread
From: Peter Maydell @ 2018-07-03 11:35 UTC (permalink / raw)
To: Kevin Wolf
Cc: Daniel P. Berrangé,
Christian Borntraeger, Peter Krempa, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers
On 3 July 2018 at 12:32, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 03.07.2018 um 13:22 hat Daniel P. Berrangé geschrieben:
>> Just posted latest version here:
>>
>> https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
>>
>> It will be in the next release on ~ Aug 1st
>
> It would have been a lot nicer to have it the July release because this
> means that we'll have the released libvirt broken during almost the
> whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> earliest, so I guess we're still okay. People using QEMU from git will
> just need libvirt from git as well.
I'm still not clear what we gain from having a QEMU that's dropped
a feature that is still used by everything except leading-edge
not-yet-released versions of libvirt. Is there a strong reason we
can't just revert the deletion of the deprecated feature for a QEMU
release or two?
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 11:32 ` Kevin Wolf
2018-07-03 11:35 ` Peter Maydell
@ 2018-07-03 11:35 ` Daniel P. Berrangé
2018-07-04 13:02 ` Cornelia Huck
2 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-03 11:35 UTC (permalink / raw)
To: Kevin Wolf
Cc: Christian Borntraeger, Peter Krempa, Peter Maydell,
Boris Fiuczynski, qemu-block, libvir-list, qemu-devel
On Tue, Jul 03, 2018 at 01:32:29PM +0200, Kevin Wolf wrote:
> Am 03.07.2018 um 13:22 hat Daniel P. Berrangé geschrieben:
> > On Tue, Jul 03, 2018 at 12:53:44PM +0200, Christian Borntraeger wrote:
> > >
> > >
> > > On 07/02/2018 10:04 AM, Kevin Wolf wrote:
> > > > Am 25.06.2018 um 13:45 hat Peter Krempa geschrieben:
> > > >> On Mon, Jun 25, 2018 at 13:41:06 +0200, Kevin Wolf wrote:
> > > >>> Am 25.06.2018 um 11:53 hat Daniel P. Berrangé geschrieben:
> > > >>>> On Fri, Jun 22, 2018 at 03:31:46PM +0100, Daniel P. Berrangé wrote:
> > > >>>>> On Fri, Jun 22, 2018 at 04:25:13PM +0200, Kevin Wolf wrote:
> > > >>>>>> Am 22.06.2018 um 15:36 hat Christian Borntraeger geschrieben:
> > > >>
> > > >> [...]
> > > >>
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>> I'll look into werror/rerror support for usb-storage. It shouldn't be
> > > >>> too hard, though it's strictly speaking a separate problem related to
> > > >>> using -blockdev rather than option deprecation.
> > > >>>
> > > >>> If Peter wants to wait for QEMU support before converting werror/rerror
> > > >>
> > > >> Definitely. I don't want to keep around yet another hack that will
> > > >> satisfy one specific case and then add another capability for it. We
> > > >> should then gate the moving of the feature based on the presence of
> > > >> werror for usb-storage.
> > > >>
> > > >>> to -device, maybe it would make sense to split your patch for v2 so that
> > > >>> geometry and serial can get fixed right away?
> > > >>
> > > >> Yes this can be done right away.
> > > >
> > > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > > next release?
> > >
> > > I cannot find an archive that has it, but it is on the libvirt mailing
> > > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > > Review seems done, but it has missed libvirt 4.5 which was released today.
> >
> > Just posted latest version here:
> >
> > https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
> >
> > It will be in the next release on ~ Aug 1st
>
> It would have been a lot nicer to have it the July release because this
> means that we'll have the released libvirt broken during almost the
> whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> earliest, so I guess we're still okay. People using QEMU from git will
> just need libvirt from git as well.
Yeah, unfortunately i just missed the window of possibility to get it
into yesterday's release.
Fedora at least will be ok, and when other distros do pick up new QEMU
they'll likely pick up the corresponding libvirt release at same time
anyway.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 11:35 ` Peter Maydell
@ 2018-07-03 12:38 ` Christian Borntraeger
0 siblings, 0 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-07-03 12:38 UTC (permalink / raw)
To: Peter Maydell, Kevin Wolf
Cc: Daniel P. Berrangé,
Peter Krempa, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers
On 07/03/2018 01:35 PM, Peter Maydell wrote:
> On 3 July 2018 at 12:32, Kevin Wolf <kwolf@redhat.com> wrote:
>> Am 03.07.2018 um 13:22 hat Daniel P. Berrangé geschrieben:
>>> Just posted latest version here:
>>>
>>> https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
>>>
>>> It will be in the next release on ~ Aug 1st
>>
>> It would have been a lot nicer to have it the July release because this
>> means that we'll have the released libvirt broken during almost the
>> whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
>> earliest, so I guess we're still okay. People using QEMU from git will
>> just need libvirt from git as well.
>
> I'm still not clear what we gain from having a QEMU that's dropped
> a feature that is still used by everything except leading-edge
> not-yet-released versions of libvirt. Is there a strong reason we
> can't just revert the deletion of the deprecated feature for a QEMU
> release or two?
+1
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-03 11:32 ` Kevin Wolf
2018-07-03 11:35 ` Peter Maydell
2018-07-03 11:35 ` Daniel P. Berrangé
@ 2018-07-04 13:02 ` Cornelia Huck
2018-07-04 13:34 ` Kevin Wolf
2 siblings, 1 reply; 111+ messages in thread
From: Cornelia Huck @ 2018-07-04 13:02 UTC (permalink / raw)
To: Kevin Wolf
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, Peter Krempa
On Tue, 3 Jul 2018 13:32:29 +0200
Kevin Wolf <kwolf@redhat.com> wrote:
> > > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > > next release?
> > >
> > > I cannot find an archive that has it, but it is on the libvirt mailing
> > > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > > Review seems done, but it has missed libvirt 4.5 which was released today.
> >
> > Just posted latest version here:
> >
> > https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
> >
> > It will be in the next release on ~ Aug 1st
>
> It would have been a lot nicer to have it the July release because this
> means that we'll have the released libvirt broken during almost the
> whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> earliest, so I guess we're still okay. People using QEMU from git will
> just need libvirt from git as well.
Speaking as an innocent* bystander:
I would usually presume that I can use any recent libvirt to test
current QEMU, even bleeding edge. In this case, not even the latest
released libvirt version will be fine, I would also need to build
libvirt from git (which is probably not something a non-libvirt
developer will usually do). If everything goes according to plan, I can
only test QEMU with a released libvirt version at the very tail end of
hardfreeze, where only release blockers are appropriate.
I think it would be really beneficial to general QEMU test coverage to
push deleting this option back a release or two. We should make testing
QEMU in conjunction with libvirt as uncomplicated as possible.
*YMMV
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:02 ` Cornelia Huck
@ 2018-07-04 13:34 ` Kevin Wolf
2018-07-04 13:43 ` Daniel P. Berrangé
` (3 more replies)
0 siblings, 4 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-07-04 13:34 UTC (permalink / raw)
To: Cornelia Huck
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, Peter Krempa
Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben:
> On Tue, 3 Jul 2018 13:32:29 +0200
> Kevin Wolf <kwolf@redhat.com> wrote:
>
> > > > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > > > next release?
> > > >
> > > > I cannot find an archive that has it, but it is on the libvirt mailing
> > > > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > > > Review seems done, but it has missed libvirt 4.5 which was released today.
> > >
> > > Just posted latest version here:
> > >
> > > https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
> > >
> > > It will be in the next release on ~ Aug 1st
> >
> > It would have been a lot nicer to have it the July release because this
> > means that we'll have the released libvirt broken during almost the
> > whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> > earliest, so I guess we're still okay. People using QEMU from git will
> > just need libvirt from git as well.
>
> Speaking as an innocent* bystander:
>
> I would usually presume that I can use any recent libvirt to test
> current QEMU, even bleeding edge. In this case, not even the latest
> released libvirt version will be fine, I would also need to build
> libvirt from git (which is probably not something a non-libvirt
> developer will usually do). If everything goes according to plan, I can
> only test QEMU with a released libvirt version at the very tail end of
> hardfreeze, where only release blockers are appropriate.
I understand where you're coming from, but let's be honest: It's not as
if disk geometry or serial numbers were features that absolutely need
to be there to give QEMU any testing.
Also, my understanding has always been that we expect users to have a
libvirt version that isn't older than QEMU. It would be useful to set a
clear policy for this and document it.
> I think it would be really beneficial to general QEMU test coverage to
> push deleting this option back a release or two. We should make testing
> QEMU in conjunction with libvirt as uncomplicated as possible.
Essentially, what is important to me isn't getting these options dropped
exactly in 3.0, but not setting a bad precedence that deprecation isn't
actually worth anything. We may easily end up with this deprecation
process:
depreate a feature
release QEMU version n + 1
release QEMU version n + 2
remove the feature
while libvirt hasn't removed use of the feature:
# ...and why should it when everything is still working?
reinstate the feature
release QEMU version n + x
remove the feature
When management tools know that this is the process, the motivation to
remove the use of the feature gets even lower (not out of malice, but
because there will be always more important things), so this will
"optimise" itself into an endless loop and we're back to never actually
removing old cruft that impedes development.
The libvirt patch has just been merged (and I'm almost sure that this
wouldn't have happened so quickly if I had just reverted the patch right
away), so at least we know now that this specific instance of the loop
is going to terminate.
What's left is first and foremost that we need to sort out our broken
deprecation mechanism, and if that gets done, I don't mind if someone
wants to revert the patch for 3.0 as long as they also take care that it
gets back into 3.1.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:34 ` Kevin Wolf
@ 2018-07-04 13:43 ` Daniel P. Berrangé
2018-07-04 14:23 ` Kevin Wolf
2018-07-04 13:52 ` Christian Borntraeger
` (2 subsequent siblings)
3 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-04 13:43 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Peter Maydell, Boris Fiuczynski, qemu-block,
libvir-list, qemu-devel, Christian Borntraeger, Peter Krempa
On Wed, Jul 04, 2018 at 03:34:40PM +0200, Kevin Wolf wrote:
> Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben:
> > On Tue, 3 Jul 2018 13:32:29 +0200
> > Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > > > > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > > > > next release?
> > > > >
> > > > > I cannot find an archive that has it, but it is on the libvirt mailing
> > > > > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > > > > Review seems done, but it has missed libvirt 4.5 which was released today.
> > > >
> > > > Just posted latest version here:
> > > >
> > > > https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
> > > >
> > > > It will be in the next release on ~ Aug 1st
> > >
> > > It would have been a lot nicer to have it the July release because this
> > > means that we'll have the released libvirt broken during almost the
> > > whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> > > earliest, so I guess we're still okay. People using QEMU from git will
> > > just need libvirt from git as well.
> >
> > Speaking as an innocent* bystander:
> >
> > I would usually presume that I can use any recent libvirt to test
> > current QEMU, even bleeding edge. In this case, not even the latest
> > released libvirt version will be fine, I would also need to build
> > libvirt from git (which is probably not something a non-libvirt
> > developer will usually do). If everything goes according to plan, I can
> > only test QEMU with a released libvirt version at the very tail end of
> > hardfreeze, where only release blockers are appropriate.
>
> I understand where you're coming from, but let's be honest: It's not as
> if disk geometry or serial numbers were features that absolutely need
> to be there to give QEMU any testing.
I'd venture to suggest disk geometry is almost never used in practice.
serial strings, however, are pretty common, used by default in fact
by openstack.
> Also, my understanding has always been that we expect users to have a
> libvirt version that isn't older than QEMU. It would be useful to set a
> clear policy for this and document it.
My understanding was actually mostly the opposite. We'll try not to
/knowingly/ break existing libvirt version if reasonable. We've not
always followed that, either by accident, or even intentionally in
some cases. What's never defined is just how old libvirt we care about
not breaking. In the case of QEMU libvirt will in fact be fixed before
QEMU 3.0 is released, but only by a week or so.
I think its mostly a case of being pragmatic about what we do in this
respect, rather than defining a hard rule.
In this case I'd personally lean towards reverting this patch, since
merging it was not a critical blocker to other work that went into
3.0 IIUC. So block maintaniers burden shouldn't be impacted by a
temporary revert and then re-merge when 3.1 opens.
> > I think it would be really beneficial to general QEMU test coverage to
> > push deleting this option back a release or two. We should make testing
> > QEMU in conjunction with libvirt as uncomplicated as possible.
>
> Essentially, what is important to me isn't getting these options dropped
> exactly in 3.0, but not setting a bad precedence that deprecation isn't
> actually worth anything. We may easily end up with this deprecation
> process:
>
> depreate a feature
> release QEMU version n + 1
> release QEMU version n + 2
> remove the feature
> while libvirt hasn't removed use of the feature:
> # ...and why should it when everything is still working?
> reinstate the feature
> release QEMU version n + x
> remove the feature
Yeah, we definitely don't want to get into that kind of mess
in general.
> When management tools know that this is the process, the motivation to
> remove the use of the feature gets even lower (not out of malice, but
> because there will be always more important things), so this will
> "optimise" itself into an endless loop and we're back to never actually
> removing old cruft that impedes development.
>
> The libvirt patch has just been merged (and I'm almost sure that this
> wouldn't have happened so quickly if I had just reverted the patch right
> away), so at least we know now that this specific instance of the loop
> is going to terminate.
>
> What's left is first and foremost that we need to sort out our broken
> deprecation mechanism, and if that gets done, I don't mind if someone
> wants to revert the patch for 3.0 as long as they also take care that it
> gets back into 3.1.
What I think this really highlights is
- Lack of libvirt paying attention to deprecations
- Lack of a way to get good diagnostic of violations when testing
The first thing is just a human problem on libvirt side.
The second thing could be addressed in code if there was an env variable
or cli option to turn all use of deprecated features into abort()s in
QEMU. That way libvirt automated testing against git master would have
a way to identify places where we use deprecated feature.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:34 ` Kevin Wolf
2018-07-04 13:43 ` Daniel P. Berrangé
@ 2018-07-04 13:52 ` Christian Borntraeger
2018-07-04 13:58 ` Cornelia Huck
2018-07-04 16:14 ` Peter Maydell
3 siblings, 0 replies; 111+ messages in thread
From: Christian Borntraeger @ 2018-07-04 13:52 UTC (permalink / raw)
To: Kevin Wolf, Cornelia Huck
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Peter Krempa
On 07/04/2018 03:34 PM, Kevin Wolf wrote:
> Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben:
>> On Tue, 3 Jul 2018 13:32:29 +0200
>> Kevin Wolf <kwolf@redhat.com> wrote:
>>
>>>>>> Has serial/gemoetry been fixed meanwhile and will it make it into the
>>>>>> next release?
>>>>>
>>>>> I cannot find an archive that has it, but it is on the libvirt mailing
>>>>> list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
>>>>> Review seems done, but it has missed libvirt 4.5 which was released today.
>>>>
>>>> Just posted latest version here:
>>>>
>>>> https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
>>>>
>>>> It will be in the next release on ~ Aug 1st
>>>
>>> It would have been a lot nicer to have it the July release because this
>>> means that we'll have the released libvirt broken during almost the
>>> whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
>>> earliest, so I guess we're still okay. People using QEMU from git will
>>> just need libvirt from git as well.
>>
>> Speaking as an innocent* bystander:
>>
>> I would usually presume that I can use any recent libvirt to test
>> current QEMU, even bleeding edge. In this case, not even the latest
>> released libvirt version will be fine, I would also need to build
>> libvirt from git (which is probably not something a non-libvirt
>> developer will usually do). If everything goes according to plan, I can
>> only test QEMU with a released libvirt version at the very tail end of
>> hardfreeze, where only release blockers are appropriate.
>
> I understand where you're coming from, but let's be honest: It's not as
> if disk geometry or serial numbers were features that absolutely need
> to be there to give QEMU any testing.
For s390 it really has values when you use image files with a dasd geometry.
I also use the serial number a lot for mount by disk-id. So it certainly breaks
parts of my tests.
>
> Also, my understanding has always been that we expect users to have a
> libvirt version that isn't older than QEMU. It would be useful to set a
> clear policy for this and document it.
>
>> I think it would be really beneficial to general QEMU test coverage to
>> push deleting this option back a release or two. We should make testing
>> QEMU in conjunction with libvirt as uncomplicated as possible.
>
> Essentially, what is important to me isn't getting these options dropped
> exactly in 3.0, but not setting a bad precedence that deprecation isn't
> actually worth anything. We may easily end up with this deprecation
> process:
>
> depreate a feature
> release QEMU version n + 1
> release QEMU version n + 2
> remove the feature
> while libvirt hasn't removed use of the feature:
> # ...and why should it when everything is still working?
> reinstate the feature
> release QEMU version n + x
> remove the feature
>
> When management tools know that this is the process, the motivation to
> remove the use of the feature gets even lower (not out of malice, but
> because there will be always more important things), so this will
> "optimise" itself into an endless loop and we're back to never actually
> removing old cruft that impedes development.
I think this is really a theoretical issue that assumes that libvirt people
are evil and try to undermine our deprecation. And this is simply not true.
Everybody tries to do his best, but we are all busy. So we had the issue
because we were sloppy and have not dealt with the deprecation properly
(by pushing the libvirt people at time of deprecation) And the reason is
that we really have no process for qemu->libvirt requirements. I mean:
look at virtio-vsock and other things. How long it took from qemu to
libvirt show that the real issue is actually somewhere else and we might
want to talk about "how to improve qemu<->libvirt interaction" at the qemu
summit.
>
> The libvirt patch has just been merged (and I'm almost sure that this
> wouldn't have happened so quickly if I had just reverted the patch right
> away), so at least we know now that this specific instance of the loop
> is going to terminate.
>
> What's left is first and foremost that we need to sort out our broken
> deprecation mechanism, and if that gets done, I don't mind if someone
> wants to revert the patch for 3.0 as long as they also take care that it
> gets back into 3.1.
So what about the following:
- revert these patches now
- add these patches to the block-next-for-3.1 branch immediately (or whatever branch
name you have for that)
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:34 ` Kevin Wolf
2018-07-04 13:43 ` Daniel P. Berrangé
2018-07-04 13:52 ` Christian Borntraeger
@ 2018-07-04 13:58 ` Cornelia Huck
2018-07-04 16:14 ` Peter Maydell
3 siblings, 0 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-04 13:58 UTC (permalink / raw)
To: Kevin Wolf
Cc: Daniel P. Berrangé,
Peter Maydell, Boris Fiuczynski, qemu-block, libvir-list,
qemu-devel, Christian Borntraeger, Peter Krempa
On Wed, 4 Jul 2018 15:34:40 +0200
Kevin Wolf <kwolf@redhat.com> wrote:
> Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben:
> > On Tue, 3 Jul 2018 13:32:29 +0200
> > Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > > > > > Has serial/gemoetry been fixed meanwhile and will it make it into the
> > > > > > next release?
> > > > >
> > > > > I cannot find an archive that has it, but it is on the libvirt mailing
> > > > > list as "[libvirt] [PATCH v3] qemu: format serial and geometry on frontend disk device".
> > > > > Review seems done, but it has missed libvirt 4.5 which was released today.
> > > >
> > > > Just posted latest version here:
> > > >
> > > > https://www.redhat.com/archives/libvir-list/2018-July/msg00130.html
> > > >
> > > > It will be in the next release on ~ Aug 1st
> > >
> > > It would have been a lot nicer to have it the July release because this
> > > means that we'll have the released libvirt broken during almost the
> > > whole rc phase of QEMU 3.0, but the release is planned for Aug 8th the
> > > earliest, so I guess we're still okay. People using QEMU from git will
> > > just need libvirt from git as well.
> >
> > Speaking as an innocent* bystander:
> >
> > I would usually presume that I can use any recent libvirt to test
> > current QEMU, even bleeding edge. In this case, not even the latest
> > released libvirt version will be fine, I would also need to build
> > libvirt from git (which is probably not something a non-libvirt
> > developer will usually do). If everything goes according to plan, I can
> > only test QEMU with a released libvirt version at the very tail end of
> > hardfreeze, where only release blockers are appropriate.
>
> I understand where you're coming from, but let's be honest: It's not as
> if disk geometry or serial numbers were features that absolutely need
> to be there to give QEMU any testing.
Consider people running regression tests: They likely have a set of
machine definitions they use, some of which may include serial numbers
et al., and that suddenly breaks if they try with a newer QEMU. If they
can just update to a newer (released) libvirt, their regression tests
continue to work.
>
> Also, my understanding has always been that we expect users to have a
> libvirt version that isn't older than QEMU. It would be useful to set a
> clear policy for this and document it.
My understanding has been that while a recent-ish libvirt might not
exploit the latest QEMU features, it still continues to work. But yes,
this is a good point. We need to agree on a clear policy and document
that.
>
> > I think it would be really beneficial to general QEMU test coverage to
> > push deleting this option back a release or two. We should make testing
> > QEMU in conjunction with libvirt as uncomplicated as possible.
>
> Essentially, what is important to me isn't getting these options dropped
> exactly in 3.0, but not setting a bad precedence that deprecation isn't
> actually worth anything. We may easily end up with this deprecation
> process:
>
> depreate a feature
> release QEMU version n + 1
> release QEMU version n + 2
> remove the feature
> while libvirt hasn't removed use of the feature:
> # ...and why should it when everything is still working?
> reinstate the feature
> release QEMU version n + x
> remove the feature
>
> When management tools know that this is the process, the motivation to
> remove the use of the feature gets even lower (not out of malice, but
> because there will be always more important things), so this will
> "optimise" itself into an endless loop and we're back to never actually
> removing old cruft that impedes development.
I understand your concern, but not having a way out if something fell
through the cracks is hurting people testing QEMU -- IOW, people we
really don't want to hurt. Ideas on what could help:
- A clearer way to communicate to libvirt users that libvirt is using a
deprecated API, so that they can report it and libvirt can change its
code. The main problem is that people usually don't read logs if
everything seems to work fine...
- A documented way in our QEMU deprecation process to hold off for one
release, which can be used at most twice (or maybe just once?) No
endless loops that way :)
>
> The libvirt patch has just been merged (and I'm almost sure that this
> wouldn't have happened so quickly if I had just reverted the patch right
> away), so at least we know now that this specific instance of the loop
> is going to terminate.
>
> What's left is first and foremost that we need to sort out our broken
> deprecation mechanism, and if that gets done, I don't mind if someone
> wants to revert the patch for 3.0 as long as they also take care that it
> gets back into 3.1.
Fully agreed on sorting out our deprecation mechanism.
I can send a revert patch, if nobody else beats me to it.
Thanks!
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:43 ` Daniel P. Berrangé
@ 2018-07-04 14:23 ` Kevin Wolf
0 siblings, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-07-04 14:23 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Cornelia Huck, Peter Maydell, Boris Fiuczynski, qemu-block,
libvir-list, qemu-devel, Christian Borntraeger, Peter Krempa
Am 04.07.2018 um 15:43 hat Daniel P. Berrangé geschrieben:
> On Wed, Jul 04, 2018 at 03:34:40PM +0200, Kevin Wolf wrote:
> > I understand where you're coming from, but let's be honest: It's not as
> > if disk geometry or serial numbers were features that absolutely need
> > to be there to give QEMU any testing.
>
> I'd venture to suggest disk geometry is almost never used in practice.
>
> serial strings, however, are pretty common, used by default in fact
> by openstack.
Interesting, good to know.
> > Also, my understanding has always been that we expect users to have a
> > libvirt version that isn't older than QEMU. It would be useful to set a
> > clear policy for this and document it.
>
> My understanding was actually mostly the opposite. We'll try not to
> /knowingly/ break existing libvirt version if reasonable. We've not
> always followed that, either by accident, or even intentionally in
> some cases. What's never defined is just how old libvirt we care about
> not breaking.
Hm, that used to be true, yes. I still seem to remember that we've
always expected a new libvirt version. Was that only about new features
then?
However, the deprecation policy essentially says that we will now
knowingly break existing libvirt versions. With it, we also defined that
we definitely don't care about libvirt versions older than QEMU n-2.
You're right that we're not really making a statement for libvirt
versions newer than that.
> In the case of QEMU libvirt will in fact be fixed before QEMU 3.0 is
> released, but only by a week or so.
Probably two weeks because QEMU never does without the extra RC, but
that doesn't make a big difference.
> I think its mostly a case of being pragmatic about what we do in this
> respect, rather than defining a hard rule.
>
> In this case I'd personally lean towards reverting this patch, since
> merging it was not a critical blocker to other work that went into
> 3.0 IIUC. So block maintaniers burden shouldn't be impacted by a
> temporary revert and then re-merge when 3.1 opens.
Having to remember to really bring it back to life after the release is
a maintainer burden, which is one of the reasons why I don't want this
"extended" deprecation process to become the norm.
If it stays a one-time thing, and given that we're now in the freeze
and a block-next branch will be active anyway, I'm okay with a temporary
revert.
> > > I think it would be really beneficial to general QEMU test coverage to
> > > push deleting this option back a release or two. We should make testing
> > > QEMU in conjunction with libvirt as uncomplicated as possible.
> >
> > Essentially, what is important to me isn't getting these options dropped
> > exactly in 3.0, but not setting a bad precedence that deprecation isn't
> > actually worth anything. We may easily end up with this deprecation
> > process:
> >
> > depreate a feature
> > release QEMU version n + 1
> > release QEMU version n + 2
> > remove the feature
> > while libvirt hasn't removed use of the feature:
> > # ...and why should it when everything is still working?
> > reinstate the feature
> > release QEMU version n + x
> > remove the feature
>
> Yeah, we definitely don't want to get into that kind of mess
> in general.
>
> > When management tools know that this is the process, the motivation to
> > remove the use of the feature gets even lower (not out of malice, but
> > because there will be always more important things), so this will
> > "optimise" itself into an endless loop and we're back to never actually
> > removing old cruft that impedes development.
> >
> > The libvirt patch has just been merged (and I'm almost sure that this
> > wouldn't have happened so quickly if I had just reverted the patch right
> > away), so at least we know now that this specific instance of the loop
> > is going to terminate.
> >
> > What's left is first and foremost that we need to sort out our broken
> > deprecation mechanism, and if that gets done, I don't mind if someone
> > wants to revert the patch for 3.0 as long as they also take care that it
> > gets back into 3.1.
>
> What I think this really highlights is
>
> - Lack of libvirt paying attention to deprecations
> - Lack of a way to get good diagnostic of violations when testing
>
> The first thing is just a human problem on libvirt side.
>
> The second thing could be addressed in code if there was an env variable
> or cli option to turn all use of deprecated features into abort()s in
> QEMU. That way libvirt automated testing against git master would have
> a way to identify places where we use deprecated feature.
The first one is basically just a result of the second. I think the most
important improvement will be finding a way how to break things early
for libvirt developers, but keep them working with a warning for
everyone else.
As a more short-term thing, is someone from libvirt looking at other
present deprecations in QEMU and what needs to be done for them?
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 13:34 ` Kevin Wolf
` (2 preceding siblings ...)
2018-07-04 13:58 ` Cornelia Huck
@ 2018-07-04 16:14 ` Peter Maydell
2018-07-06 11:11 ` Cornelia Huck
3 siblings, 1 reply; 111+ messages in thread
From: Peter Maydell @ 2018-07-04 16:14 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Daniel P. Berrangé,
Boris Fiuczynski, Qemu-block, Libvirt, QEMU Developers,
Christian Borntraeger, Peter Krempa
On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> Essentially, what is important to me isn't getting these options dropped
> exactly in 3.0, but not setting a bad precedence that deprecation isn't
> actually worth anything. We may easily end up with this deprecation
> process:
>
> depreate a feature
> release QEMU version n + 1
> release QEMU version n + 2
> remove the feature
> while libvirt hasn't removed use of the feature:
> # ...and why should it when everything is still working?
> reinstate the feature
> release QEMU version n + x
> remove the feature
My take on the deprecation policy essentially is that it gives
us a *minimum* bar for how soon we can drop something. We
shouldn't be using it as an "always target this speed for
dropping something" -- we ought to be pragmatic. We can
drop stuff that's unused quickly, but should be slower
for things that still have major users (or reconsider
the deprecation entirely, potentially). There should be
a balance between making our work as developers easier and
inconveniencing our users.
In this particular case, reverting this deletion seems like
a fairly easy call to me.
We should also definitely work on improving how we can
let management-layer developers easily test that they're
not accidentally relying on deprecated features, certainly
(and also on better documentation for command line users
of how to switch away from deprecated features -- for
instance I am still using -redir in some of my scripts
because the warning about it being deprecated is not
precise about what exact command line option can be used
instead, especially for the case where the ethernet device
is builtin rather than created with -device...)
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-04 16:14 ` Peter Maydell
@ 2018-07-06 11:11 ` Cornelia Huck
2018-07-06 14:56 ` Kevin Wolf
2018-07-09 6:58 ` Thomas Huth
0 siblings, 2 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-06 11:11 UTC (permalink / raw)
To: Peter Maydell
Cc: Kevin Wolf, Daniel P. Berrangé,
Boris Fiuczynski, Qemu-block, Libvirt, QEMU Developers,
Christian Borntraeger, Peter Krempa
On Wed, 4 Jul 2018 17:14:02 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:
> On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> > Essentially, what is important to me isn't getting these options dropped
> > exactly in 3.0, but not setting a bad precedence that deprecation isn't
> > actually worth anything. We may easily end up with this deprecation
> > process:
> >
> > depreate a feature
> > release QEMU version n + 1
> > release QEMU version n + 2
> > remove the feature
> > while libvirt hasn't removed use of the feature:
> > # ...and why should it when everything is still working?
> > reinstate the feature
> > release QEMU version n + x
> > remove the feature
>
> My take on the deprecation policy essentially is that it gives
> us a *minimum* bar for how soon we can drop something. We
> shouldn't be using it as an "always target this speed for
> dropping something" -- we ought to be pragmatic. We can
> drop stuff that's unused quickly, but should be slower
> for things that still have major users (or reconsider
> the deprecation entirely, potentially). There should be
> a balance between making our work as developers easier and
> inconveniencing our users.
What about the following?
- put a feature on the "normal" deprecation list to remove after two
releases
Case (a): nobody complains, either within the deprecation period or
when it is finally removed
-> all is good
Case (b): the feature turns out to be widely used, and/or it turns out
that it offers value that currently can't be offered easily in another
way
-> remove from deprecation list; this obviously needs more thinking
Case (c): the feature is used, the users are willing to move away from
it, but they need a bit more time
-> put it on a "deprecation watchlist", listing the users we are
waiting for, and then remove after all are done (no +2)
That way, we can still easily remove old cruft (case (a)), but still
accommodate cases like this (case (c)). The obvious drawback is that
we'd need someone to curate the deprecation watchlist, to poke the
users we're waiting for, and probably remove anyway after some time if
they don't get their act together.
>
> In this particular case, reverting this deletion seems like
> a fairly easy call to me.
>
> We should also definitely work on improving how we can
> let management-layer developers easily test that they're
> not accidentally relying on deprecated features, certainly
> (and also on better documentation for command line users
> of how to switch away from deprecated features -- for
> instance I am still using -redir in some of my scripts
> because the warning about it being deprecated is not
> precise about what exact command line option can be used
> instead, especially for the case where the ethernet device
> is builtin rather than created with -device...)
Yes, this as well.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 11:11 ` Cornelia Huck
@ 2018-07-06 14:56 ` Kevin Wolf
2018-07-06 15:05 ` Daniel P. Berrangé
` (2 more replies)
2018-07-09 6:58 ` Thomas Huth
1 sibling, 3 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-07-06 14:56 UTC (permalink / raw)
To: Cornelia Huck
Cc: Peter Maydell, Daniel P. Berrangé,
Boris Fiuczynski, Qemu-block, Libvirt, QEMU Developers,
Christian Borntraeger, Peter Krempa
Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
> On Wed, 4 Jul 2018 17:14:02 +0100
> Peter Maydell <peter.maydell@linaro.org> wrote:
>
> > On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> > > Essentially, what is important to me isn't getting these options dropped
> > > exactly in 3.0, but not setting a bad precedence that deprecation isn't
> > > actually worth anything. We may easily end up with this deprecation
> > > process:
> > >
> > > depreate a feature
> > > release QEMU version n + 1
> > > release QEMU version n + 2
> > > remove the feature
> > > while libvirt hasn't removed use of the feature:
> > > # ...and why should it when everything is still working?
> > > reinstate the feature
> > > release QEMU version n + x
> > > remove the feature
> >
> > My take on the deprecation policy essentially is that it gives
> > us a *minimum* bar for how soon we can drop something. We
> > shouldn't be using it as an "always target this speed for
> > dropping something" -- we ought to be pragmatic. We can
> > drop stuff that's unused quickly, but should be slower
> > for things that still have major users (or reconsider
> > the deprecation entirely, potentially). There should be
> > a balance between making our work as developers easier and
> > inconveniencing our users.
>
> What about the following?
>
> - put a feature on the "normal" deprecation list to remove after two
> releases
> Case (a): nobody complains, either within the deprecation period or
> when it is finally removed
> -> all is good
> Case (b): the feature turns out to be widely used, and/or it turns out
> that it offers value that currently can't be offered easily in another
> way
> -> remove from deprecation list; this obviously needs more thinking
> Case (c): the feature is used, the users are willing to move away from
> it, but they need a bit more time
> -> put it on a "deprecation watchlist", listing the users we are
> waiting for, and then remove after all are done (no +2)
>
> That way, we can still easily remove old cruft (case (a)), but still
> accommodate cases like this (case (c)). The obvious drawback is that
> we'd need someone to curate the deprecation watchlist, to poke the
> users we're waiting for, and probably remove anyway after some time if
> they don't get their act together.
The problem is that things are only starting to move after two releases
have passed. The original idea was to already use that time. If we don't
use it, then waiting for two releases is pointless and we can just
directly let things go to a deprecation watchlist.
Maybe we can just use the existing wiki page:
https://wiki.qemu.org/index.php/Features/LegacyRemoval
And add a column for whether libvirt is ready? Of course, that only
makes sense if libvirt people make use of and contribute to this page.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 14:56 ` Kevin Wolf
@ 2018-07-06 15:05 ` Daniel P. Berrangé
2018-07-06 15:10 ` Peter Maydell
2018-07-09 7:29 ` Peter Krempa
2 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-06 15:05 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Peter Maydell, Boris Fiuczynski, Qemu-block,
Libvirt, QEMU Developers, Christian Borntraeger, Peter Krempa
On Fri, Jul 06, 2018 at 04:56:46PM +0200, Kevin Wolf wrote:
> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
> > On Wed, 4 Jul 2018 17:14:02 +0100
> > Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > > On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> > > > Essentially, what is important to me isn't getting these options dropped
> > > > exactly in 3.0, but not setting a bad precedence that deprecation isn't
> > > > actually worth anything. We may easily end up with this deprecation
> > > > process:
> > > >
> > > > depreate a feature
> > > > release QEMU version n + 1
> > > > release QEMU version n + 2
> > > > remove the feature
> > > > while libvirt hasn't removed use of the feature:
> > > > # ...and why should it when everything is still working?
> > > > reinstate the feature
> > > > release QEMU version n + x
> > > > remove the feature
> > >
> > > My take on the deprecation policy essentially is that it gives
> > > us a *minimum* bar for how soon we can drop something. We
> > > shouldn't be using it as an "always target this speed for
> > > dropping something" -- we ought to be pragmatic. We can
> > > drop stuff that's unused quickly, but should be slower
> > > for things that still have major users (or reconsider
> > > the deprecation entirely, potentially). There should be
> > > a balance between making our work as developers easier and
> > > inconveniencing our users.
> >
> > What about the following?
> >
> > - put a feature on the "normal" deprecation list to remove after two
> > releases
> > Case (a): nobody complains, either within the deprecation period or
> > when it is finally removed
> > -> all is good
> > Case (b): the feature turns out to be widely used, and/or it turns out
> > that it offers value that currently can't be offered easily in another
> > way
> > -> remove from deprecation list; this obviously needs more thinking
> > Case (c): the feature is used, the users are willing to move away from
> > it, but they need a bit more time
> > -> put it on a "deprecation watchlist", listing the users we are
> > waiting for, and then remove after all are done (no +2)
> >
> > That way, we can still easily remove old cruft (case (a)), but still
> > accommodate cases like this (case (c)). The obvious drawback is that
> > we'd need someone to curate the deprecation watchlist, to poke the
> > users we're waiting for, and probably remove anyway after some time if
> > they don't get their act together.
>
> The problem is that things are only starting to move after two releases
> have passed. The original idea was to already use that time. If we don't
> use it, then waiting for two releases is pointless and we can just
> directly let things go to a deprecation watchlist.
>
> Maybe we can just use the existing wiki page:
>
> https://wiki.qemu.org/index.php/Features/LegacyRemoval
>
> And add a column for whether libvirt is ready? Of course, that only
> makes sense if libvirt people make use of and contribute to this page.
FYI I checked the list of deprecated features, and aside from the drive
ones, libvirt is impacted by the VNC TLS stuff, and the change to
"target" in the query-cpus-fast QMP command.
I've got bugs open against libvirt to address all of these
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 14:56 ` Kevin Wolf
2018-07-06 15:05 ` Daniel P. Berrangé
@ 2018-07-06 15:10 ` Peter Maydell
2018-07-09 6:33 ` Markus Armbruster
2018-07-09 7:29 ` Peter Krempa
2 siblings, 1 reply; 111+ messages in thread
From: Peter Maydell @ 2018-07-06 15:10 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Daniel P. Berrangé,
Boris Fiuczynski, Qemu-block, Libvirt, QEMU Developers,
Christian Borntraeger, Peter Krempa
On 6 July 2018 at 15:56, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
>> That way, we can still easily remove old cruft (case (a)), but still
>> accommodate cases like this (case (c)). The obvious drawback is that
>> we'd need someone to curate the deprecation watchlist, to poke the
>> users we're waiting for, and probably remove anyway after some time if
>> they don't get their act together.
>
> The problem is that things are only starting to move after two releases
> have passed.
Right, so clearly just "put a note in the documentation" isn't
sufficient advertisement/prodding of things going away. (Also, two
releases is pretty fast. Many of our users will be using distro
packaged versions of QEMU which will lag further behind than
bleeding-edge users. The system version of QEMU on my desktop
machine is 2.5...)
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 15:10 ` Peter Maydell
@ 2018-07-09 6:33 ` Markus Armbruster
2018-07-09 11:08 ` Cornelia Huck
0 siblings, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-07-09 6:33 UTC (permalink / raw)
To: Peter Maydell
Cc: Kevin Wolf, Boris Fiuczynski, Qemu-block, Libvirt, Cornelia Huck,
QEMU Developers, Christian Borntraeger, Peter Krempa
Peter Maydell <peter.maydell@linaro.org> writes:
> On 6 July 2018 at 15:56, Kevin Wolf <kwolf@redhat.com> wrote:
>> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
>>> That way, we can still easily remove old cruft (case (a)), but still
>>> accommodate cases like this (case (c)). The obvious drawback is that
>>> we'd need someone to curate the deprecation watchlist, to poke the
>>> users we're waiting for, and probably remove anyway after some time if
>>> they don't get their act together.
>>
>> The problem is that things are only starting to move after two releases
>> have passed.
>
> Right, so clearly just "put a note in the documentation" isn't
> sufficient advertisement/prodding of things going away.
Yes. Ideas on more forceful notification have been tossed around, we
just have to act on them.
> (Also, two
> releases is pretty fast. Many of our users will be using distro
> packaged versions of QEMU which will lag further behind than
> bleeding-edge users. The system version of QEMU on my desktop
> machine is 2.5...)
If you consume QEMU in a way that's impacted by the changes the
deprecation policy guards, you have two sane options:
* Track upstream deprecation, either continuously, or at least right
after a QEMU release. Since 2.10, they're collected in qemu-doc
appendix "Deprecated features".
* Batch that work before you switch QEMU versions. Make sure to
allocate enough time.
If you consume only downstream versions of QEMU, and don't want to track
upstream, go ahead and pick the second option. It should do even if you
leap from 2.5 (December 2015) all the way to 3.0.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 11:11 ` Cornelia Huck
2018-07-06 14:56 ` Kevin Wolf
@ 2018-07-09 6:58 ` Thomas Huth
2018-07-09 11:58 ` Cornelia Huck
1 sibling, 1 reply; 111+ messages in thread
From: Thomas Huth @ 2018-07-09 6:58 UTC (permalink / raw)
To: Cornelia Huck, Peter Maydell
Cc: Kevin Wolf, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa,
Paolo Bonzini
On 06.07.2018 13:11, Cornelia Huck wrote:
> On Wed, 4 Jul 2018 17:14:02 +0100
> Peter Maydell <peter.maydell@linaro.org> wrote:
>
>> On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Essentially, what is important to me isn't getting these options dropped
>>> exactly in 3.0, but not setting a bad precedence that deprecation isn't
>>> actually worth anything. We may easily end up with this deprecation
>>> process:
>>>
>>> depreate a feature
>>> release QEMU version n + 1
>>> release QEMU version n + 2
>>> remove the feature
>>> while libvirt hasn't removed use of the feature:
>>> # ...and why should it when everything is still working?
>>> reinstate the feature
>>> release QEMU version n + x
>>> remove the feature
>>
>> My take on the deprecation policy essentially is that it gives
>> us a *minimum* bar for how soon we can drop something. We
>> shouldn't be using it as an "always target this speed for
>> dropping something" -- we ought to be pragmatic. We can
>> drop stuff that's unused quickly, but should be slower
>> for things that still have major users (or reconsider
>> the deprecation entirely, potentially). There should be
>> a balance between making our work as developers easier and
>> inconveniencing our users.
>
> What about the following?
>
> - put a feature on the "normal" deprecation list to remove after two
> releases
> Case (a): nobody complains, either within the deprecation period or
> when it is finally removed
> -> all is good
> Case (b): the feature turns out to be widely used, and/or it turns out
> that it offers value that currently can't be offered easily in another
> way
> -> remove from deprecation list; this obviously needs more thinking
> Case (c): the feature is used, the users are willing to move away from
> it, but they need a bit more time
> -> put it on a "deprecation watchlist", listing the users we are
> waiting for, and then remove after all are done (no +2)
That sounds like another indication that we should have a list of
"legacy" options in our qemu-doc, i.e. a list of interfaces that we
consider as old and unwanted, but do not intend to remove in 2 releases
yet. That idea has recently also come up already when we discussed the
"-enable-kvm" and "-no-kvm" options. The remainders "-usbdevice" is also
another good candidate for that list...
Thomas
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-06 14:56 ` Kevin Wolf
2018-07-06 15:05 ` Daniel P. Berrangé
2018-07-06 15:10 ` Peter Maydell
@ 2018-07-09 7:29 ` Peter Krempa
2018-07-10 5:59 ` Markus Armbruster
2 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-07-09 7:29 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Peter Maydell, Daniel P. Berrangé,
Boris Fiuczynski, Qemu-block, Libvirt, QEMU Developers,
Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]
On Fri, Jul 06, 2018 at 16:56:46 +0200, Kevin Wolf wrote:
> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
> > On Wed, 4 Jul 2018 17:14:02 +0100
> > Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > > On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> > > > Essentially, what is important to me isn't getting these options dropped
> > > > exactly in 3.0, but not setting a bad precedence that deprecation isn't
> > > > actually worth anything. We may easily end up with this deprecation
> > > > process:
> > > >
> > > > depreate a feature
> > > > release QEMU version n + 1
> > > > release QEMU version n + 2
> > > > remove the feature
> > > > while libvirt hasn't removed use of the feature:
> > > > # ...and why should it when everything is still working?
> > > > reinstate the feature
> > > > release QEMU version n + x
> > > > remove the feature
> > >
> > > My take on the deprecation policy essentially is that it gives
> > > us a *minimum* bar for how soon we can drop something. We
> > > shouldn't be using it as an "always target this speed for
> > > dropping something" -- we ought to be pragmatic. We can
> > > drop stuff that's unused quickly, but should be slower
> > > for things that still have major users (or reconsider
> > > the deprecation entirely, potentially). There should be
> > > a balance between making our work as developers easier and
> > > inconveniencing our users.
> >
> > What about the following?
> >
> > - put a feature on the "normal" deprecation list to remove after two
> > releases
> > Case (a): nobody complains, either within the deprecation period or
> > when it is finally removed
> > -> all is good
> > Case (b): the feature turns out to be widely used, and/or it turns out
> > that it offers value that currently can't be offered easily in another
> > way
> > -> remove from deprecation list; this obviously needs more thinking
> > Case (c): the feature is used, the users are willing to move away from
> > it, but they need a bit more time
> > -> put it on a "deprecation watchlist", listing the users we are
> > waiting for, and then remove after all are done (no +2)
> >
> > That way, we can still easily remove old cruft (case (a)), but still
> > accommodate cases like this (case (c)). The obvious drawback is that
> > we'd need someone to curate the deprecation watchlist, to poke the
> > users we're waiting for, and probably remove anyway after some time if
> > they don't get their act together.
>
> The problem is that things are only starting to move after two releases
> have passed. The original idea was to already use that time. If we don't
> use it, then waiting for two releases is pointless and we can just
> directly let things go to a deprecation watchlist.
>
> Maybe we can just use the existing wiki page:
>
> https://wiki.qemu.org/index.php/Features/LegacyRemoval
>
> And add a column for whether libvirt is ready? Of course, that only
> makes sense if libvirt people make use of and contribute to this page.
This should not be a problem, but I think we need some active
encouraging/prodding to remove deprecated stuff. Otherwise we might miss
the news.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-09 6:33 ` Markus Armbruster
@ 2018-07-09 11:08 ` Cornelia Huck
2018-07-09 11:17 ` Daniel P. Berrangé
0 siblings, 1 reply; 111+ messages in thread
From: Cornelia Huck @ 2018-07-09 11:08 UTC (permalink / raw)
To: Markus Armbruster
Cc: Peter Maydell, Kevin Wolf, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa
On Mon, 09 Jul 2018 08:33:05 +0200
Markus Armbruster <armbru@redhat.com> wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > On 6 July 2018 at 15:56, Kevin Wolf <kwolf@redhat.com> wrote:
> >> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
> >>> That way, we can still easily remove old cruft (case (a)), but still
> >>> accommodate cases like this (case (c)). The obvious drawback is that
> >>> we'd need someone to curate the deprecation watchlist, to poke the
> >>> users we're waiting for, and probably remove anyway after some time if
> >>> they don't get their act together.
> >>
> >> The problem is that things are only starting to move after two releases
> >> have passed.
> >
> > Right, so clearly just "put a note in the documentation" isn't
> > sufficient advertisement/prodding of things going away.
>
> Yes. Ideas on more forceful notification have been tossed around, we
> just have to act on them.
>
> > (Also, two
> > releases is pretty fast. Many of our users will be using distro
> > packaged versions of QEMU which will lag further behind than
> > bleeding-edge users. The system version of QEMU on my desktop
> > machine is 2.5...)
>
> If you consume QEMU in a way that's impacted by the changes the
> deprecation policy guards, you have two sane options:
>
> * Track upstream deprecation, either continuously, or at least right
> after a QEMU release. Since 2.10, they're collected in qemu-doc
> appendix "Deprecated features".
Can we draw more attention to this in any way? Point it out prominently
in the release notes? Send a list to known consumers (e.g. libvirt) on
release time?
>
> * Batch that work before you switch QEMU versions. Make sure to
> allocate enough time.
>
> If you consume only downstream versions of QEMU, and don't want to track
> upstream, go ahead and pick the second option. It should do even if you
> leap from 2.5 (December 2015) all the way to 3.0.
If you are a direct user of QEMU, you really need to follow development
more closely, or you won't be able to complain about removal of a
useful feature until it has already been gone for some time.
If you only use QEMU through libvirt, you should be fine as long as
libvirt (and distro packagers) do not mess up.
Do we have any idea about how many end users using libvirt or other
tooling vs. QEMU directly? Of those not going via libvirt, are they
more likely to follow development or use a distro package?
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-09 11:08 ` Cornelia Huck
@ 2018-07-09 11:17 ` Daniel P. Berrangé
2018-07-12 6:32 ` Markus Armbruster
0 siblings, 1 reply; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-09 11:17 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger,
Peter Krempa
On Mon, Jul 09, 2018 at 01:08:38PM +0200, Cornelia Huck wrote:
> On Mon, 09 Jul 2018 08:33:05 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > Peter Maydell <peter.maydell@linaro.org> writes:
> >
> > > On 6 July 2018 at 15:56, Kevin Wolf <kwolf@redhat.com> wrote:
> > >> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
> > >>> That way, we can still easily remove old cruft (case (a)), but still
> > >>> accommodate cases like this (case (c)). The obvious drawback is that
> > >>> we'd need someone to curate the deprecation watchlist, to poke the
> > >>> users we're waiting for, and probably remove anyway after some time if
> > >>> they don't get their act together.
> > >>
> > >> The problem is that things are only starting to move after two releases
> > >> have passed.
> > >
> > > Right, so clearly just "put a note in the documentation" isn't
> > > sufficient advertisement/prodding of things going away.
> >
> > Yes. Ideas on more forceful notification have been tossed around, we
> > just have to act on them.
> >
> > > (Also, two
> > > releases is pretty fast. Many of our users will be using distro
> > > packaged versions of QEMU which will lag further behind than
> > > bleeding-edge users. The system version of QEMU on my desktop
> > > machine is 2.5...)
> >
> > If you consume QEMU in a way that's impacted by the changes the
> > deprecation policy guards, you have two sane options:
> >
> > * Track upstream deprecation, either continuously, or at least right
> > after a QEMU release. Since 2.10, they're collected in qemu-doc
> > appendix "Deprecated features".
>
> Can we draw more attention to this in any way? Point it out prominently
> in the release notes? Send a list to known consumers (e.g. libvirt) on
> release time?
Yes, we should all newly deprecated stuff in the release notes.
For libvirt, I think whenever something is proposed for deprecation
we could just CC libvir-list, or ask one of the libvirt people to
confirm its not being used. If it is, then we should file BZ against
libvirt.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-09 6:58 ` Thomas Huth
@ 2018-07-09 11:58 ` Cornelia Huck
0 siblings, 0 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-09 11:58 UTC (permalink / raw)
To: Thomas Huth
Cc: Peter Maydell, Kevin Wolf, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa,
Paolo Bonzini
On Mon, 9 Jul 2018 08:58:05 +0200
Thomas Huth <thuth@redhat.com> wrote:
> On 06.07.2018 13:11, Cornelia Huck wrote:
> > On Wed, 4 Jul 2018 17:14:02 +0100
> > Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> >> On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
> >>> Essentially, what is important to me isn't getting these options dropped
> >>> exactly in 3.0, but not setting a bad precedence that deprecation isn't
> >>> actually worth anything. We may easily end up with this deprecation
> >>> process:
> >>>
> >>> depreate a feature
> >>> release QEMU version n + 1
> >>> release QEMU version n + 2
> >>> remove the feature
> >>> while libvirt hasn't removed use of the feature:
> >>> # ...and why should it when everything is still working?
> >>> reinstate the feature
> >>> release QEMU version n + x
> >>> remove the feature
> >>
> >> My take on the deprecation policy essentially is that it gives
> >> us a *minimum* bar for how soon we can drop something. We
> >> shouldn't be using it as an "always target this speed for
> >> dropping something" -- we ought to be pragmatic. We can
> >> drop stuff that's unused quickly, but should be slower
> >> for things that still have major users (or reconsider
> >> the deprecation entirely, potentially). There should be
> >> a balance between making our work as developers easier and
> >> inconveniencing our users.
> >
> > What about the following?
> >
> > - put a feature on the "normal" deprecation list to remove after two
> > releases
> > Case (a): nobody complains, either within the deprecation period or
> > when it is finally removed
> > -> all is good
> > Case (b): the feature turns out to be widely used, and/or it turns out
> > that it offers value that currently can't be offered easily in another
> > way
> > -> remove from deprecation list; this obviously needs more thinking
> > Case (c): the feature is used, the users are willing to move away from
> > it, but they need a bit more time
> > -> put it on a "deprecation watchlist", listing the users we are
> > waiting for, and then remove after all are done (no +2)
>
> That sounds like another indication that we should have a list of
> "legacy" options in our qemu-doc, i.e. a list of interfaces that we
> consider as old and unwanted, but do not intend to remove in 2 releases
> yet. That idea has recently also come up already when we discussed the
> "-enable-kvm" and "-no-kvm" options. The remainders "-usbdevice" is also
> another good candidate for that list...
Agree. It also might be a good idea to poke e.g. libvirt about those.
Related: Are there other widely-used management etc. programs that make
use of QEMU? (For some value of 'widely'.) We might consider poking
them as well.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-09 7:29 ` Peter Krempa
@ 2018-07-10 5:59 ` Markus Armbruster
2018-07-10 14:22 ` Cornelia Huck
0 siblings, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-07-10 5:59 UTC (permalink / raw)
To: Peter Krempa
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
Cornelia Huck, QEMU Developers, Christian Borntraeger
Peter Krempa <pkrempa@redhat.com> writes:
> On Fri, Jul 06, 2018 at 16:56:46 +0200, Kevin Wolf wrote:
>> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
>> > On Wed, 4 Jul 2018 17:14:02 +0100
>> > Peter Maydell <peter.maydell@linaro.org> wrote:
>> >
>> > > On 4 July 2018 at 14:34, Kevin Wolf <kwolf@redhat.com> wrote:
>> > > > Essentially, what is important to me isn't getting these options dropped
>> > > > exactly in 3.0, but not setting a bad precedence that deprecation isn't
>> > > > actually worth anything. We may easily end up with this deprecation
>> > > > process:
>> > > >
>> > > > depreate a feature
>> > > > release QEMU version n + 1
>> > > > release QEMU version n + 2
>> > > > remove the feature
>> > > > while libvirt hasn't removed use of the feature:
>> > > > # ...and why should it when everything is still working?
>> > > > reinstate the feature
>> > > > release QEMU version n + x
>> > > > remove the feature
>> > >
>> > > My take on the deprecation policy essentially is that it gives
>> > > us a *minimum* bar for how soon we can drop something. We
>> > > shouldn't be using it as an "always target this speed for
>> > > dropping something" -- we ought to be pragmatic. We can
>> > > drop stuff that's unused quickly, but should be slower
>> > > for things that still have major users (or reconsider
>> > > the deprecation entirely, potentially). There should be
>> > > a balance between making our work as developers easier and
>> > > inconveniencing our users.
>> >
>> > What about the following?
>> >
>> > - put a feature on the "normal" deprecation list to remove after two
>> > releases
>> > Case (a): nobody complains, either within the deprecation period or
>> > when it is finally removed
>> > -> all is good
>> > Case (b): the feature turns out to be widely used, and/or it turns out
>> > that it offers value that currently can't be offered easily in another
>> > way
>> > -> remove from deprecation list; this obviously needs more thinking
>> > Case (c): the feature is used, the users are willing to move away from
>> > it, but they need a bit more time
>> > -> put it on a "deprecation watchlist", listing the users we are
>> > waiting for, and then remove after all are done (no +2)
>> >
>> > That way, we can still easily remove old cruft (case (a)), but still
>> > accommodate cases like this (case (c)). The obvious drawback is that
>> > we'd need someone to curate the deprecation watchlist, to poke the
>> > users we're waiting for, and probably remove anyway after some time if
>> > they don't get their act together.
>>
>> The problem is that things are only starting to move after two releases
>> have passed. The original idea was to already use that time. If we don't
>> use it, then waiting for two releases is pointless and we can just
>> directly let things go to a deprecation watchlist.
>>
>> Maybe we can just use the existing wiki page:
>>
>> https://wiki.qemu.org/index.php/Features/LegacyRemoval
>>
>> And add a column for whether libvirt is ready? Of course, that only
>> makes sense if libvirt people make use of and contribute to this page.
>
> This should not be a problem, but I think we need some active
> encouraging/prodding to remove deprecated stuff. Otherwise we might miss
> the news.
In addition to actively pulling libvirt developers into review of
deprecation patches, we should pursue the idea to optionally let QEMU
fail on use of deprecated features, then have libvirt run its test suite
that way.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 5:59 ` Markus Armbruster
@ 2018-07-10 14:22 ` Cornelia Huck
2018-07-10 14:38 ` Kevin Wolf
` (2 more replies)
0 siblings, 3 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-10 14:22 UTC (permalink / raw)
To: Markus Armbruster
Cc: Peter Krempa, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
On Tue, 10 Jul 2018 07:59:15 +0200
Markus Armbruster <armbru@redhat.com> wrote:
> In addition to actively pulling libvirt developers into review of
> deprecation patches, we should pursue the idea to optionally let QEMU
> fail on use of deprecated features, then have libvirt run its test suite
> that way.
What about the following:
qemu_deprecated_option("old_option", "modern_option");
Which would then print (in normal operation)
"WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
to the monitor (or to stderr? to both?).
If you start QEMU with a -no-deprecated-options switch, it would print
"ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
and do an exit(1).
Would that be workable?
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 14:22 ` Cornelia Huck
@ 2018-07-10 14:38 ` Kevin Wolf
2018-07-12 6:38 ` Markus Armbruster
2018-07-10 14:39 ` Peter Krempa
2018-07-10 15:09 ` Peter Maydell
2 siblings, 1 reply; 111+ messages in thread
From: Kevin Wolf @ 2018-07-10 14:38 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Peter Krempa, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
> On Tue, 10 Jul 2018 07:59:15 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > In addition to actively pulling libvirt developers into review of
> > deprecation patches, we should pursue the idea to optionally let QEMU
> > fail on use of deprecated features, then have libvirt run its test suite
> > that way.
>
> What about the following:
>
> qemu_deprecated_option("old_option", "modern_option");
>
> Which would then print (in normal operation)
>
> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> to the monitor (or to stderr? to both?).
>
> If you start QEMU with a -no-deprecated-options switch, it would print
>
> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> and do an exit(1).
>
> Would that be workable?
I think the function should just take a message:
/* Works like error_report(), except for the WARNING/ERROR prefix
* and exit(1) if -no-deprecated-options is set */
void deprecation_report(const char *fmt, ...);
We don't necessarily deprecate only options, but we might also deprecate
monitor commands, specific options values (while keeping other values of
the same option) etc.
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 14:22 ` Cornelia Huck
2018-07-10 14:38 ` Kevin Wolf
@ 2018-07-10 14:39 ` Peter Krempa
2018-07-10 15:01 ` Cornelia Huck
2018-07-10 15:09 ` Peter Maydell
2 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-07-10 14:39 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 1941 bytes --]
On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
> On Tue, 10 Jul 2018 07:59:15 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > In addition to actively pulling libvirt developers into review of
> > deprecation patches, we should pursue the idea to optionally let QEMU
> > fail on use of deprecated features, then have libvirt run its test suite
> > that way.
>
> What about the following:
>
> qemu_deprecated_option("old_option", "modern_option");
I think this is too simplified. You can deprecate only a certain value
for an option or even just a combination of values and options. The
check will need to be programatic and error reporting probably can't be
reasonably machine readable anyways.
> Which would then print (in normal operation)
>
> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> to the monitor (or to stderr? to both?).
>
> If you start QEMU with a -no-deprecated-options switch, it would print
>
> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> and do an exit(1).
>
> Would that be workable?
For delivering the warnings via monitor you'll need a store that will
collect all the warnings and prepare them for delivery. You've got
basically two options:
1) monitor command to poll for deprecated options
2) event with deprecated options
Both require storing them since libvirt connects to the monitor only
after the command line is processed.
Warnings printed to stderr are nearly useless because until something
breaks nobody bothers to read the log files.
To make any reasonable use of -no-deprecated-options we'd also need
something that simulates qemu startup (no resources are touched in fact)
so that we can run it against the testsuite. Otherwise the use will be
limited to developers using it with the configuration they are
currently testing.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 14:39 ` Peter Krempa
@ 2018-07-10 15:01 ` Cornelia Huck
2018-07-10 15:24 ` Peter Krempa
` (2 more replies)
0 siblings, 3 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-10 15:01 UTC (permalink / raw)
To: Peter Krempa
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
On Tue, 10 Jul 2018 16:39:31 +0200
Peter Krempa <pkrempa@redhat.com> wrote:
> On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
> > On Tue, 10 Jul 2018 07:59:15 +0200
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> > > In addition to actively pulling libvirt developers into review of
> > > deprecation patches, we should pursue the idea to optionally let QEMU
> > > fail on use of deprecated features, then have libvirt run its test suite
> > > that way.
> >
> > What about the following:
> >
> > qemu_deprecated_option("old_option", "modern_option");
>
> I think this is too simplified. You can deprecate only a certain value
> for an option or even just a combination of values and options. The
> check will need to be programatic and error reporting probably can't be
> reasonably machine readable anyways.
Kevin's suggestion of using a simple message instead of this old/modern
option thing is actually better (and would cover more). My intention
behind printing a message (somewhere) was to make this friendly to
command line users as well.
>
> > Which would then print (in normal operation)
> >
> > "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> >
> > to the monitor (or to stderr? to both?).
> >
> > If you start QEMU with a -no-deprecated-options switch, it would print
> >
> > "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> >
> > and do an exit(1).
> >
> > Would that be workable?
>
> For delivering the warnings via monitor you'll need a store that will
> collect all the warnings and prepare them for delivery. You've got
> basically two options:
>
> 1) monitor command to poll for deprecated options
> 2) event with deprecated options
>
> Both require storing them since libvirt connects to the monitor only
> after the command line is processed.
>
> Warnings printed to stderr are nearly useless because until something
> breaks nobody bothers to read the log files.
So, from that I gather that a hard failure would be the easiest for
libvirt to detect (and everything else would become complicated really
quickly), right?
If we fail with exit(1), can libvirt check any message that is logged
right before that?
>
> To make any reasonable use of -no-deprecated-options we'd also need
> something that simulates qemu startup (no resources are touched in fact)
> so that we can run it against the testsuite. Otherwise the use will be
> limited to developers using it with the configuration they are
> currently testing.
Would that moan loudly that you should poke the libvirt developers if
some kind of testsuite failure is detected? Or am I misunderstanding?
Who is, in general, testing which libvirt version? I can think of:
- libvirt developers, which will probably run libvirt current git, but
more likely a released QEMU?
- QEMU (and other related tools) developers, who will probably use QEMU
current git, but a released libvirt
- normal (technical) users and (integration) testers, who will probably
use released versions of libvirt and QEMU
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 14:22 ` Cornelia Huck
2018-07-10 14:38 ` Kevin Wolf
2018-07-10 14:39 ` Peter Krempa
@ 2018-07-10 15:09 ` Peter Maydell
2018-07-10 16:59 ` Daniel P. Berrangé
2 siblings, 1 reply; 111+ messages in thread
From: Peter Maydell @ 2018-07-10 15:09 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Peter Krempa, Kevin Wolf, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
On 10 July 2018 at 15:22, Cornelia Huck <cohuck@redhat.com> wrote:
> On Tue, 10 Jul 2018 07:59:15 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> In addition to actively pulling libvirt developers into review of
>> deprecation patches, we should pursue the idea to optionally let QEMU
>> fail on use of deprecated features, then have libvirt run its test suite
>> that way.
>
> What about the following:
>
> qemu_deprecated_option("old_option", "modern_option");
>
> Which would then print (in normal operation)
>
> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> to the monitor (or to stderr? to both?).
>
> If you start QEMU with a -no-deprecated-options switch, it would print
>
> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
I'd prefer to see more hand-tailored deprecation messages that can
be more detailed about exactly what the new syntax needs to be.
The ideal would be if you could get right down to the detailed
level of "replace -old_thing -with_foo=x with -device new_thing,foo=x"...
thanks
-- PMM
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:01 ` Cornelia Huck
@ 2018-07-10 15:24 ` Peter Krempa
2018-07-11 6:53 ` Thomas Huth
2018-07-12 6:59 ` Markus Armbruster
2018-07-10 17:01 ` Daniel P. Berrangé
2018-07-11 13:48 ` Kashyap Chamarthy
2 siblings, 2 replies; 111+ messages in thread
From: Peter Krempa @ 2018-07-10 15:24 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 3582 bytes --]
On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
> On Tue, 10 Jul 2018 16:39:31 +0200
> Peter Krempa <pkrempa@redhat.com> wrote:
> > On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
> > > On Tue, 10 Jul 2018 07:59:15 +0200
> > > Markus Armbruster <armbru@redhat.com> wrote:
[...]
> > > "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> > >
> > > and do an exit(1).
> > >
> > > Would that be workable?
> >
> > For delivering the warnings via monitor you'll need a store that will
> > collect all the warnings and prepare them for delivery. You've got
> > basically two options:
> >
> > 1) monitor command to poll for deprecated options
> > 2) event with deprecated options
> >
> > Both require storing them since libvirt connects to the monitor only
> > after the command line is processed.
> >
> > Warnings printed to stderr are nearly useless because until something
> > breaks nobody bothers to read the log files.
>
> So, from that I gather that a hard failure would be the easiest for
> libvirt to detect (and everything else would become complicated really
> quickly), right?
People start complaining only when stuff breaks. If anything is optional
people will usually not enable it. That makes any non-mandatory option
not work in most cases.
Since we are talking about deprecation we can't really make any of this
default though so there will always be a level of user interaction
required.
An option is to do a automatic testing where one of this approaches will
be enabled. For that you need a way to generate configurations which
libvirt would use in real life. We have a rather big collection of XMLs
which describe a valid configuration but the problem with using them on
a real qemu is that most of the disk paths/network targets/other
resources are made up and making them work with a real qemu would range
from being painful to being impossible.
If we start from scratch you then lack coverage.
> If we fail with exit(1), can libvirt check any message that is logged
> right before that?
Yes we currently use this for very early failures which occur prior to
the monitor working.
> > To make any reasonable use of -no-deprecated-options we'd also need
> > something that simulates qemu startup (no resources are touched in fact)
> > so that we can run it against the testsuite. Otherwise the use will be
> > limited to developers using it with the configuration they are
> > currently testing.
>
> Would that moan loudly that you should poke the libvirt developers if
> some kind of testsuite failure is detected? Or am I misunderstanding?
Generally it should make somebody complain. But there is a problem.
Since we are talking deprecation it can't be enabled by default. And by
not making it default most of the users will not enable that option.
> Who is, in general, testing which libvirt version? I can think of:
> - libvirt developers, which will probably run libvirt current git, but
> more likely a released QEMU?
Speaking for myself I run git+patches libvirt and git version of
qemu as I'm usually working with new features. (sometimes git+patches
qemu if it's a bleeding edge feature or fix for a feature)
> - QEMU (and other related tools) developers, who will probably use QEMU
> current git, but a released libvirt
This is probably generally true.
> - normal (technical) users and (integration) testers, who will probably
> use released versions of libvirt and QEMU
This too
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:09 ` Peter Maydell
@ 2018-07-10 16:59 ` Daniel P. Berrangé
0 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-10 16:59 UTC (permalink / raw)
To: Peter Maydell
Cc: Cornelia Huck, Kevin Wolf, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa
On Tue, Jul 10, 2018 at 04:09:38PM +0100, Peter Maydell wrote:
> On 10 July 2018 at 15:22, Cornelia Huck <cohuck@redhat.com> wrote:
> > On Tue, 10 Jul 2018 07:59:15 +0200
> > Markus Armbruster <armbru@redhat.com> wrote:
> >
> >> In addition to actively pulling libvirt developers into review of
> >> deprecation patches, we should pursue the idea to optionally let QEMU
> >> fail on use of deprecated features, then have libvirt run its test suite
> >> that way.
> >
> > What about the following:
> >
> > qemu_deprecated_option("old_option", "modern_option");
> >
> > Which would then print (in normal operation)
> >
> > "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> >
> > to the monitor (or to stderr? to both?).
> >
> > If you start QEMU with a -no-deprecated-options switch, it would print
> >
> > "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>
> I'd prefer to see more hand-tailored deprecation messages that can
> be more detailed about exactly what the new syntax needs to be.
> The ideal would be if you could get right down to the detailed
> level of "replace -old_thing -with_foo=x with -device new_thing,foo=x"...
Agreed, the more detail we can provide the better for users.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:01 ` Cornelia Huck
2018-07-10 15:24 ` Peter Krempa
@ 2018-07-10 17:01 ` Daniel P. Berrangé
2018-07-11 13:48 ` Kashyap Chamarthy
2 siblings, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-10 17:01 UTC (permalink / raw)
To: Cornelia Huck
Cc: Peter Krempa, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Markus Armbruster, QEMU Developers,
Christian Borntraeger
On Tue, Jul 10, 2018 at 05:01:22PM +0200, Cornelia Huck wrote:
> Who is, in general, testing which libvirt version? I can think of:
> - libvirt developers, which will probably run libvirt current git, but
> more likely a released QEMU?
In general libvirt devs tend to run a mixture of whatever the default QEMU
is in Fedora / RHEL, along side current QEMU git master, depending on what
feature(s) we're working on.
> - QEMU (and other related tools) developers, who will probably use QEMU
> current git, but a released libvirt
> - normal (technical) users and (integration) testers, who will probably
> use released versions of libvirt and QEMU
We don't currently have any integration testing CI system, but we are in
process of getting hardware resource to allow us to run CI. With that we
can setup jobs which test various relevant QEMU versions, both git master
and released versions.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:24 ` Peter Krempa
@ 2018-07-11 6:53 ` Thomas Huth
2018-07-11 7:24 ` Cornelia Huck
2018-07-12 6:40 ` Markus Armbruster
2018-07-12 6:59 ` Markus Armbruster
1 sibling, 2 replies; 111+ messages in thread
From: Thomas Huth @ 2018-07-11 6:53 UTC (permalink / raw)
To: Peter Krempa, Cornelia Huck
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
Markus Armbruster, QEMU Developers, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]
On 10.07.2018 17:24, Peter Krempa wrote:
> On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
>> On Tue, 10 Jul 2018 16:39:31 +0200
>> Peter Krempa <pkrempa@redhat.com> wrote:
>>> On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
>>>> On Tue, 10 Jul 2018 07:59:15 +0200
>>>> Markus Armbruster <armbru@redhat.com> wrote:
>
> [...]
>
>>>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>>>
>>>> and do an exit(1).
>>>>
>>>> Would that be workable?
>>>
>>> For delivering the warnings via monitor you'll need a store that will
>>> collect all the warnings and prepare them for delivery. You've got
>>> basically two options:
>>>
>>> 1) monitor command to poll for deprecated options
>>> 2) event with deprecated options
>>>
>>> Both require storing them since libvirt connects to the monitor only
>>> after the command line is processed.
>>>
>>> Warnings printed to stderr are nearly useless because until something
>>> breaks nobody bothers to read the log files.
>>
>> So, from that I gather that a hard failure would be the easiest for
>> libvirt to detect (and everything else would become complicated really
>> quickly), right?
>
> People start complaining only when stuff breaks. If anything is optional
> people will usually not enable it. That makes any non-mandatory option
> not work in most cases.
So would it help if we "invert" the logic, i.e. deprecated_report()
would do exit(1) by default? Then, if the (human) users still want to
continue with the deprecated option, they have to add a
"--ignore-deprecation" command line switch to make QEMU start
successfully...
Thomas
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-11 6:53 ` Thomas Huth
@ 2018-07-11 7:24 ` Cornelia Huck
2018-07-12 6:40 ` Markus Armbruster
1 sibling, 0 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-11 7:24 UTC (permalink / raw)
To: Thomas Huth
Cc: Peter Krempa, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Markus Armbruster, QEMU Developers,
Christian Borntraeger
On Wed, 11 Jul 2018 08:53:20 +0200
Thomas Huth <thuth@redhat.com> wrote:
> On 10.07.2018 17:24, Peter Krempa wrote:
> > On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
> >> So, from that I gather that a hard failure would be the easiest for
> >> libvirt to detect (and everything else would become complicated really
> >> quickly), right?
> >
> > People start complaining only when stuff breaks. If anything is optional
> > people will usually not enable it. That makes any non-mandatory option
> > not work in most cases.
>
> So would it help if we "invert" the logic, i.e. deprecated_report()
> would do exit(1) by default? Then, if the (human) users still want to
> continue with the deprecated option, they have to add a
> "--ignore-deprecation" command line switch to make QEMU start
> successfully...
That is sure to get the attention of even 'normal' users, but we'd have
to make it really, really obvious (1) how to get it working again and
(2) what other things we'd like them to do (like 'if you are using a
management tool, please let them know about it'). I'm fearing a lot of
'I followed that ancient guide, and it does not work' type of reports,
though.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:01 ` Cornelia Huck
2018-07-10 15:24 ` Peter Krempa
2018-07-10 17:01 ` Daniel P. Berrangé
@ 2018-07-11 13:48 ` Kashyap Chamarthy
2 siblings, 0 replies; 111+ messages in thread
From: Kashyap Chamarthy @ 2018-07-11 13:48 UTC (permalink / raw)
To: Cornelia Huck
Cc: Peter Krempa, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
On Tue, Jul 10, 2018 at 05:01:22PM +0200, Cornelia Huck wrote:
[...]
> Who is, in general, testing which libvirt version? I can think of:
> - libvirt developers, which will probably run libvirt current git, but
> more likely a released QEMU?
> - QEMU (and other related tools) developers, who will probably use QEMU
> current git, but a released libvirt
> - normal (technical) users and (integration) testers, who will probably
> use released versions of libvirt and QEMU
There is another kind of integration tester -- e.g. FWIW, in Nova I've
been advocating to create a CI job that would do the following:
- QEMU from Git
- libvirt from Git
- Nova from Git
Along with:
- Latest QEMU release
- Latest libvirt release
- Nova from Git
With the aim of seeing what explodes in Nova and file bugs (for the
relevant low-leve components) and coordinate with relevant upstream
accordingly.
* * *
Further, Nova's libvirt driver defines several version constants. The
following two are set to a version that is available across a set of
"Linux distributions that matter"[*]
MIN_LIBVIRT_VERSION = (1, 3, 1)
MIN_QEMU_VERSION = (2, 5, 0)
(The above are the minimum required versions _today_.)
And at the start of each development cycle (lasts 6 months), we evaluate
what versions are available and update the version matrix[*]:
NEXT_MIN_LIBVIRT_VERSION = (3, 0, 0)
NEXT_MIN_QEMU_VERSION = (2, 8, 0)
(The above will be the versions for the _upcoming_ development cycle --
sometime end of this year.)
https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix
--
/kashyap
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-09 11:17 ` Daniel P. Berrangé
@ 2018-07-12 6:32 ` Markus Armbruster
2018-07-12 15:47 ` Thomas Huth
0 siblings, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 6:32 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Cornelia Huck, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Markus Armbruster, QEMU Developers,
Christian Borntraeger, Peter Krempa
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Mon, Jul 09, 2018 at 01:08:38PM +0200, Cornelia Huck wrote:
>> On Mon, 09 Jul 2018 08:33:05 +0200
>> Markus Armbruster <armbru@redhat.com> wrote:
>>
>> > Peter Maydell <peter.maydell@linaro.org> writes:
>> >
>> > > On 6 July 2018 at 15:56, Kevin Wolf <kwolf@redhat.com> wrote:
>> > >> Am 06.07.2018 um 13:11 hat Cornelia Huck geschrieben:
>> > >>> That way, we can still easily remove old cruft (case (a)), but still
>> > >>> accommodate cases like this (case (c)). The obvious drawback is that
>> > >>> we'd need someone to curate the deprecation watchlist, to poke the
>> > >>> users we're waiting for, and probably remove anyway after some time if
>> > >>> they don't get their act together.
>> > >>
>> > >> The problem is that things are only starting to move after two releases
>> > >> have passed.
>> > >
>> > > Right, so clearly just "put a note in the documentation" isn't
>> > > sufficient advertisement/prodding of things going away.
>> >
>> > Yes. Ideas on more forceful notification have been tossed around, we
>> > just have to act on them.
>> >
>> > > (Also, two
>> > > releases is pretty fast. Many of our users will be using distro
>> > > packaged versions of QEMU which will lag further behind than
>> > > bleeding-edge users. The system version of QEMU on my desktop
>> > > machine is 2.5...)
>> >
>> > If you consume QEMU in a way that's impacted by the changes the
>> > deprecation policy guards, you have two sane options:
>> >
>> > * Track upstream deprecation, either continuously, or at least right
>> > after a QEMU release. Since 2.10, they're collected in qemu-doc
>> > appendix "Deprecated features".
>>
>> Can we draw more attention to this in any way? Point it out prominently
>> in the release notes? Send a list to known consumers (e.g. libvirt) on
>> release time?
>
> Yes, we should all newly deprecated stuff in the release notes.
No-brainer.
> For libvirt, I think whenever something is proposed for deprecation
> we could just CC libvir-list, or ask one of the libvirt people to
> confirm its not being used. If it is, then we should file BZ against
> libvirt.
Makes sense, but relying on developers getting their cc: right every
time is a setting us up for failure.
Our tool to help with getting cc: wrong less often is the MAINTAINERS
file. Could one of the libvirt developers watch changes to qemu-doc
appendix "Deprecated features"? Would putting the appendix in its own
.texi help with that?
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 14:38 ` Kevin Wolf
@ 2018-07-12 6:38 ` Markus Armbruster
2018-07-12 6:51 ` Markus Armbruster
2018-07-12 7:00 ` Peter Krempa
0 siblings, 2 replies; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 6:38 UTC (permalink / raw)
To: Kevin Wolf
Cc: Cornelia Huck, Peter Maydell, Boris Fiuczynski, Qemu-block,
Libvirt, Markus Armbruster, QEMU Developers,
Christian Borntraeger, Peter Krempa
Kevin Wolf <kwolf@redhat.com> writes:
> Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
>> On Tue, 10 Jul 2018 07:59:15 +0200
>> Markus Armbruster <armbru@redhat.com> wrote:
>>
>> > In addition to actively pulling libvirt developers into review of
>> > deprecation patches, we should pursue the idea to optionally let QEMU
>> > fail on use of deprecated features, then have libvirt run its test suite
>> > that way.
>>
>> What about the following:
>>
>> qemu_deprecated_option("old_option", "modern_option");
>>
>> Which would then print (in normal operation)
>>
>> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>
>> to the monitor (or to stderr? to both?).
>>
>> If you start QEMU with a -no-deprecated-options switch, it would print
>>
>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>
>> and do an exit(1).
>>
>> Would that be workable?
>
> I think the function should just take a message:
>
> /* Works like error_report(), except for the WARNING/ERROR prefix
> * and exit(1) if -no-deprecated-options is set */
> void deprecation_report(const char *fmt, ...);
I like it. The contract could use a bit of polish, but that's detail.
> We don't necessarily deprecate only options, but we might also deprecate
> monitor commands, specific options values (while keeping other values of
> the same option) etc.
Exactly.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-11 6:53 ` Thomas Huth
2018-07-11 7:24 ` Cornelia Huck
@ 2018-07-12 6:40 ` Markus Armbruster
1 sibling, 0 replies; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 6:40 UTC (permalink / raw)
To: Thomas Huth
Cc: Peter Krempa, Cornelia Huck, Kevin Wolf, Peter Maydell,
Boris Fiuczynski, Qemu-block, Libvirt, Markus Armbruster,
QEMU Developers, Christian Borntraeger
Thomas Huth <thuth@redhat.com> writes:
> On 10.07.2018 17:24, Peter Krempa wrote:
>> On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
>>> On Tue, 10 Jul 2018 16:39:31 +0200
>>> Peter Krempa <pkrempa@redhat.com> wrote:
>>>> On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
>>>>> On Tue, 10 Jul 2018 07:59:15 +0200
>>>>> Markus Armbruster <armbru@redhat.com> wrote:
>>
>> [...]
>>
>>>>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>>>>
>>>>> and do an exit(1).
>>>>>
>>>>> Would that be workable?
>>>>
>>>> For delivering the warnings via monitor you'll need a store that will
>>>> collect all the warnings and prepare them for delivery. You've got
>>>> basically two options:
>>>>
>>>> 1) monitor command to poll for deprecated options
>>>> 2) event with deprecated options
>>>>
>>>> Both require storing them since libvirt connects to the monitor only
>>>> after the command line is processed.
>>>>
>>>> Warnings printed to stderr are nearly useless because until something
>>>> breaks nobody bothers to read the log files.
>>>
>>> So, from that I gather that a hard failure would be the easiest for
>>> libvirt to detect (and everything else would become complicated really
>>> quickly), right?
>>
>> People start complaining only when stuff breaks. If anything is optional
>> people will usually not enable it. That makes any non-mandatory option
>> not work in most cases.
>
> So would it help if we "invert" the logic, i.e. deprecated_report()
> would do exit(1) by default? Then, if the (human) users still want to
> continue with the deprecated option, they have to add a
> "--ignore-deprecation" command line switch to make QEMU start
> successfully...
You owe the God of Backward Compatibility one rubber chicken for
thinking this heretic thought!
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 6:38 ` Markus Armbruster
@ 2018-07-12 6:51 ` Markus Armbruster
2018-07-12 7:48 ` Cornelia Huck
2018-07-12 7:00 ` Peter Krempa
1 sibling, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 6:51 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Cornelia Huck, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger,
Peter Krempa
Markus Armbruster <armbru@redhat.com> writes:
> Kevin Wolf <kwolf@redhat.com> writes:
>
>> Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
>>> On Tue, 10 Jul 2018 07:59:15 +0200
>>> Markus Armbruster <armbru@redhat.com> wrote:
>>>
>>> > In addition to actively pulling libvirt developers into review of
>>> > deprecation patches, we should pursue the idea to optionally let QEMU
>>> > fail on use of deprecated features, then have libvirt run its test suite
>>> > that way.
>>>
>>> What about the following:
>>>
>>> qemu_deprecated_option("old_option", "modern_option");
>>>
>>> Which would then print (in normal operation)
>>>
>>> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>>
>>> to the monitor (or to stderr? to both?).
>>>
>>> If you start QEMU with a -no-deprecated-options switch, it would print
>>>
>>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>>>
>>> and do an exit(1).
>>>
>>> Would that be workable?
>>
>> I think the function should just take a message:
>>
>> /* Works like error_report(), except for the WARNING/ERROR prefix
>> * and exit(1) if -no-deprecated-options is set */
>> void deprecation_report(const char *fmt, ...);
>
> I like it. The contract could use a bit of polish, but that's detail.
Suggest --deprecated={silent,warn,error}, default silent.
[...]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-10 15:24 ` Peter Krempa
2018-07-11 6:53 ` Thomas Huth
@ 2018-07-12 6:59 ` Markus Armbruster
2018-07-12 7:19 ` Peter Krempa
1 sibling, 1 reply; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 6:59 UTC (permalink / raw)
To: Peter Krempa
Cc: Cornelia Huck, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Markus Armbruster, QEMU Developers,
Christian Borntraeger
Peter Krempa <pkrempa@redhat.com> writes:
> On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
>> On Tue, 10 Jul 2018 16:39:31 +0200
>> Peter Krempa <pkrempa@redhat.com> wrote:
>> > On Tue, Jul 10, 2018 at 16:22:08 +0200, Cornelia Huck wrote:
>> > > On Tue, 10 Jul 2018 07:59:15 +0200
>> > > Markus Armbruster <armbru@redhat.com> wrote:
>
> [...]
>
>> > > "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
>> > >
>> > > and do an exit(1).
>> > >
>> > > Would that be workable?
>> >
>> > For delivering the warnings via monitor you'll need a store that will
>> > collect all the warnings and prepare them for delivery. You've got
>> > basically two options:
>> >
>> > 1) monitor command to poll for deprecated options
>> > 2) event with deprecated options
>> >
>> > Both require storing them since libvirt connects to the monitor only
>> > after the command line is processed.
>> >
>> > Warnings printed to stderr are nearly useless because until something
>> > breaks nobody bothers to read the log files.
>>
>> So, from that I gather that a hard failure would be the easiest for
>> libvirt to detect (and everything else would become complicated really
>> quickly), right?
>
> People start complaining only when stuff breaks. If anything is optional
> people will usually not enable it. That makes any non-mandatory option
> not work in most cases.
>
> Since we are talking about deprecation we can't really make any of this
> default though so there will always be a level of user interaction
> required.
>
> An option is to do a automatic testing where one of this approaches will
> be enabled. For that you need a way to generate configurations which
> libvirt would use in real life. We have a rather big collection of XMLs
> which describe a valid configuration but the problem with using them on
> a real qemu is that most of the disk paths/network targets/other
> resources are made up and making them work with a real qemu would range
> from being painful to being impossible.
I sympathize. However, it's not clear which one's harder, providing
environments for a sufficiently wide range of configurations (possibly
mockups), or hacking QEMU to do nothing but check configuration. QEMU
isn't designed for that, and configuration checking is intertwined with
everything else. Complete disentanglement looks impractical to me. I
guess we could do something useful at the QAPI level, though. Yet
another reason to qapify the command line...
> If we start from scratch you then lack coverage.
>
>> If we fail with exit(1), can libvirt check any message that is logged
>> right before that?
>
> Yes we currently use this for very early failures which occur prior to
> the monitor working.
>
>> > To make any reasonable use of -no-deprecated-options we'd also need
>> > something that simulates qemu startup (no resources are touched in fact)
>> > so that we can run it against the testsuite. Otherwise the use will be
>> > limited to developers using it with the configuration they are
>> > currently testing.
>>>
>> Would that moan loudly that you should poke the libvirt developers if
>> some kind of testsuite failure is detected? Or am I misunderstanding?
>
> Generally it should make somebody complain. But there is a problem.
> Since we are talking deprecation it can't be enabled by default. And by
> not making it default most of the users will not enable that option.
I don't think end users should do the work of catching use of deprecated
features. It's a CI job.
In a CI context, we don't need fancy QMP infrastructure to communicate
"you used a deprecated feature", we can get away with printing an
explanation to stderr and exit(1). That should make CI fail, and the
failure should make a developer read the explanation. To unbreak CI, he
can either fix the problem right away, or file a BZ and suppress the CI
failure until it's fixed, say by downgrading --deprecated=error to
--deprecated=warn.
[...]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 6:38 ` Markus Armbruster
2018-07-12 6:51 ` Markus Armbruster
@ 2018-07-12 7:00 ` Peter Krempa
2018-07-12 11:19 ` Markus Armbruster
1 sibling, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-07-12 7:00 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Cornelia Huck, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 2725 bytes --]
On Thu, Jul 12, 2018 at 08:38:25 +0200, Markus Armbruster wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
> > Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
> >> On Tue, 10 Jul 2018 07:59:15 +0200
> >> Markus Armbruster <armbru@redhat.com> wrote:
> >> Would that be workable?
> >
> > I think the function should just take a message:
> >
> > /* Works like error_report(), except for the WARNING/ERROR prefix
> > * and exit(1) if -no-deprecated-options is set */
> > void deprecation_report(const char *fmt, ...);
>
> I like it. The contract could use a bit of polish, but that's detail.
>
> > We don't necessarily deprecate only options, but we might also deprecate
> > monitor commands, specific options values (while keeping other values of
> > the same option) etc.
>
> Exactly.
For monitor commands we luckily have QMP introspection which can help a
lot in this case. At least for deprecating stuff that is expressable by
the schema.
In libvirt we are actually doing schema validation of the blockdev-add
arguments generators and most commands which are covered by the
qemumonitorjsontest. The schema used is based on our capability
detection so it's gathered from the most-recent version of qemu we have
required for our tests (which is most of the time based on GIT version
of qemu if there are any significant new features).
If the deprecation will be expressable by the schema it should be rather
simple to modify the schema validator to catch the deprecation flags and
report errors in our testsuite.
CI-ifying of the above should be then also very simple. We'd just gather
fresh QMP schema rather than using one from our test case pantry.
Some time ago I also added testing of the commandline generator in
libvirt with the most recent capabilities rather than using the
historically declared capabilities that we've added when the test was
created. This means that we actually test some valid combinations and
also if stuff covered by our capability probing is removed the tests
will catch it.
I was also thinking of adding a tool which would use the above tests to
attempt starting of a qemu process until the monitor shows up. That test
then could also use -no-deprecated-options. I'm hoping waiting for the
monitor is sufficient to excercise most of the code which could contain
deprecation warnings. (Alternatively we can go through the
pre-cpu-startup setup done on the monitor as well). Unfortunately doing
this will not be as simple asi the test cases contain random disk paths
and other resources which may not be available. This means that it will
require some in-place modification and creative temporary resource
usage.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 6:59 ` Markus Armbruster
@ 2018-07-12 7:19 ` Peter Krempa
2018-07-12 11:33 ` Markus Armbruster
0 siblings, 1 reply; 111+ messages in thread
From: Peter Krempa @ 2018-07-12 7:19 UTC (permalink / raw)
To: Markus Armbruster
Cc: Cornelia Huck, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger
[-- Attachment #1: Type: text/plain, Size: 4602 bytes --]
On Thu, Jul 12, 2018 at 08:59:44 +0200, Markus Armbruster wrote:
> Peter Krempa <pkrempa@redhat.com> writes:
> > On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
> >> On Tue, 10 Jul 2018 16:39:31 +0200
> >> Peter Krempa <pkrempa@redhat.com> wrote:
[...]
> > An option is to do a automatic testing where one of this approaches will
> > be enabled. For that you need a way to generate configurations which
> > libvirt would use in real life. We have a rather big collection of XMLs
> > which describe a valid configuration but the problem with using them on
> > a real qemu is that most of the disk paths/network targets/other
> > resources are made up and making them work with a real qemu would range
> > from being painful to being impossible.
>
> I sympathize. However, it's not clear which one's harder, providing
> environments for a sufficiently wide range of configurations (possibly
> mockups), or hacking QEMU to do nothing but check configuration. QEMU
That's the main reason I think we should make it possible to use the
data for the 'qemuxml2argv' test suite in libvirt. We require that new
features are covered by this so that means that the testsuite is
possibly the most comprehensive collection of libvirt configurations I
know of.
It's not perfect as we in many cases don't test any possible
value but just try to excercise the code to generate them and others are
left behind.
Another historical problem was that we've defined a set of capabilities
rather than using any real example to do this so many of the
commandlines generated and tested are basically impossible to get in
real life.
That's why I added testing with real capabilities. We'll just need to
generate a bunch of files to achieve full coverage here.
> isn't designed for that, and configuration checking is intertwined with
> everything else. Complete disentanglement looks impractical to me. I
I agree. Getting anything special than the real codepath may create
bubbles of problems still. On the other hand we'll need some guidance on
what's sufficient to do to execute the deprecation detection code.
This may require some coding style guidelines in qemu. E.g. no
deprecation warnings after the vCPUs are started. Running a full
operating system to check the warinigns would be utterly impractical.
Preferrably we would get away with starting qemu and waiting for the
monitor to start.
> guess we could do something useful at the QAPI level, though. Yet
> another reason to qapify the command line...
That would be great, but I think that there's a subset of things that
can be deprecated but can't be expressed by schema. In such case we
still need to run the programatic checks to see.
> > If we start from scratch you then lack coverage.
> >
> >> If we fail with exit(1), can libvirt check any message that is logged
> >> right before that?
> >
> > Yes we currently use this for very early failures which occur prior to
> > the monitor working.
> >
> >> > To make any reasonable use of -no-deprecated-options we'd also need
> >> > something that simulates qemu startup (no resources are touched in fact)
> >> > so that we can run it against the testsuite. Otherwise the use will be
> >> > limited to developers using it with the configuration they are
> >> > currently testing.
> >>>
> >> Would that moan loudly that you should poke the libvirt developers if
> >> some kind of testsuite failure is detected? Or am I misunderstanding?
> >
> > Generally it should make somebody complain. But there is a problem.
> > Since we are talking deprecation it can't be enabled by default. And by
> > not making it default most of the users will not enable that option.
>
> I don't think end users should do the work of catching use of deprecated
> features. It's a CI job.
>
> In a CI context, we don't need fancy QMP infrastructure to communicate
> "you used a deprecated feature", we can get away with printing an
> explanation to stderr and exit(1). That should make CI fail, and the
> failure should make a developer read the explanation. To unbreak CI, he
> can either fix the problem right away, or file a BZ and suppress the CI
> failure until it's fixed, say by downgrading --deprecated=error to
> --deprecated=warn.
Definitely. Plain untranslated error message is fine. The only thing is
that it should be easy to detect. exit(1) is that solution. Or rather
exit($VALUE_SPECIFIC_FOR_DEPRECATION) so that we can automatically
discriminate test failures from deprecation warnings.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 6:51 ` Markus Armbruster
@ 2018-07-12 7:48 ` Cornelia Huck
2018-07-12 9:05 ` Kevin Wolf
2018-07-12 11:14 ` Markus Armbruster
0 siblings, 2 replies; 111+ messages in thread
From: Cornelia Huck @ 2018-07-12 7:48 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa
On Thu, 12 Jul 2018 08:51:16 +0200
Markus Armbruster <armbru@redhat.com> wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
> > Kevin Wolf <kwolf@redhat.com> writes:
> >
> >> Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
> >>> On Tue, 10 Jul 2018 07:59:15 +0200
> >>> Markus Armbruster <armbru@redhat.com> wrote:
> >>>
> >>> > In addition to actively pulling libvirt developers into review of
> >>> > deprecation patches, we should pursue the idea to optionally let QEMU
> >>> > fail on use of deprecated features, then have libvirt run its test suite
> >>> > that way.
> >>>
> >>> What about the following:
> >>>
> >>> qemu_deprecated_option("old_option", "modern_option");
> >>>
> >>> Which would then print (in normal operation)
> >>>
> >>> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> >>>
> >>> to the monitor (or to stderr? to both?).
> >>>
> >>> If you start QEMU with a -no-deprecated-options switch, it would print
> >>>
> >>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> >>>
> >>> and do an exit(1).
> >>>
> >>> Would that be workable?
> >>
> >> I think the function should just take a message:
> >>
> >> /* Works like error_report(), except for the WARNING/ERROR prefix
> >> * and exit(1) if -no-deprecated-options is set */
> >> void deprecation_report(const char *fmt, ...);
> >
> > I like it. The contract could use a bit of polish, but that's detail.
>
> Suggest --deprecated={silent,warn,error}, default silent.
I like that, but I'd prefer to default to warn (so that command line
users have a better chance to notice it).
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 7:48 ` Cornelia Huck
@ 2018-07-12 9:05 ` Kevin Wolf
2018-07-12 11:14 ` Markus Armbruster
1 sibling, 0 replies; 111+ messages in thread
From: Kevin Wolf @ 2018-07-12 9:05 UTC (permalink / raw)
To: Cornelia Huck
Cc: Markus Armbruster, Peter Maydell, Boris Fiuczynski, Qemu-block,
Libvirt, QEMU Developers, Christian Borntraeger, Peter Krempa
Am 12.07.2018 um 09:48 hat Cornelia Huck geschrieben:
> On Thu, 12 Jul 2018 08:51:16 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > Markus Armbruster <armbru@redhat.com> writes:
> >
> > > Kevin Wolf <kwolf@redhat.com> writes:
> > >
> > >> Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
> > >>> On Tue, 10 Jul 2018 07:59:15 +0200
> > >>> Markus Armbruster <armbru@redhat.com> wrote:
> > >>>
> > >>> > In addition to actively pulling libvirt developers into review of
> > >>> > deprecation patches, we should pursue the idea to optionally let QEMU
> > >>> > fail on use of deprecated features, then have libvirt run its test suite
> > >>> > that way.
> > >>>
> > >>> What about the following:
> > >>>
> > >>> qemu_deprecated_option("old_option", "modern_option");
> > >>>
> > >>> Which would then print (in normal operation)
> > >>>
> > >>> "WARNING: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> > >>>
> > >>> to the monitor (or to stderr? to both?).
> > >>>
> > >>> If you start QEMU with a -no-deprecated-options switch, it would print
> > >>>
> > >>> "ERROR: 'old_option' is deprecated and will be removed; use 'modern_option' instead"
> > >>>
> > >>> and do an exit(1).
> > >>>
> > >>> Would that be workable?
> > >>
> > >> I think the function should just take a message:
> > >>
> > >> /* Works like error_report(), except for the WARNING/ERROR prefix
> > >> * and exit(1) if -no-deprecated-options is set */
> > >> void deprecation_report(const char *fmt, ...);
> > >
> > > I like it. The contract could use a bit of polish, but that's detail.
Obviously, this comment wasn't meant to be copied into the source code,
but just to explain what I'm actually proposing there.
> > Suggest --deprecated={silent,warn,error}, default silent.
>
> I like that, but I'd prefer to default to warn (so that command line
> users have a better chance to notice it).
I agree that warn is the better default. (It's also consistent with what
we have been doing for deprecations so far.)
Kevin
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 7:48 ` Cornelia Huck
2018-07-12 9:05 ` Kevin Wolf
@ 2018-07-12 11:14 ` Markus Armbruster
1 sibling, 0 replies; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 11:14 UTC (permalink / raw)
To: Cornelia Huck
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa
Cornelia Huck <cohuck@redhat.com> writes:
> On Thu, 12 Jul 2018 08:51:16 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> Markus Armbruster <armbru@redhat.com> writes:
>>
>> > Kevin Wolf <kwolf@redhat.com> writes:
>> >
>> >> I think the function should just take a message:
>> >>
>> >> /* Works like error_report(), except for the WARNING/ERROR prefix
>> >> * and exit(1) if -no-deprecated-options is set */
>> >> void deprecation_report(const char *fmt, ...);
>> >
>> > I like it. The contract could use a bit of polish, but that's detail.
>>
>> Suggest --deprecated={silent,warn,error}, default silent.
>
> I like that, but I'd prefer to default to warn (so that command line
> users have a better chance to notice it).
Fair enough.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 7:00 ` Peter Krempa
@ 2018-07-12 11:19 ` Markus Armbruster
0 siblings, 0 replies; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 11:19 UTC (permalink / raw)
To: Peter Krempa
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
Cornelia Huck, QEMU Developers, Christian Borntraeger
Peter Krempa <pkrempa@redhat.com> writes:
> On Thu, Jul 12, 2018 at 08:38:25 +0200, Markus Armbruster wrote:
>> Kevin Wolf <kwolf@redhat.com> writes:
>> > Am 10.07.2018 um 16:22 hat Cornelia Huck geschrieben:
>> >> On Tue, 10 Jul 2018 07:59:15 +0200
>> >> Markus Armbruster <armbru@redhat.com> wrote:
>> >> Would that be workable?
>> >
>> > I think the function should just take a message:
>> >
>> > /* Works like error_report(), except for the WARNING/ERROR prefix
>> > * and exit(1) if -no-deprecated-options is set */
>> > void deprecation_report(const char *fmt, ...);
>>
>> I like it. The contract could use a bit of polish, but that's detail.
>>
>> > We don't necessarily deprecate only options, but we might also deprecate
>> > monitor commands, specific options values (while keeping other values of
>> > the same option) etc.
>>
>> Exactly.
>
> For monitor commands we luckily have QMP introspection which can help a
> lot in this case. At least for deprecating stuff that is expressable by
> the schema.
Introspection doesn't convey "deprecated", but...
> In libvirt we are actually doing schema validation of the blockdev-add
> arguments generators and most commands which are covered by the
> qemumonitorjsontest. The schema used is based on our capability
> detection so it's gathered from the most-recent version of qemu we have
> required for our tests (which is most of the time based on GIT version
> of qemu if there are any significant new features).
>
> If the deprecation will be expressable by the schema it should be rather
> simple to modify the schema validator to catch the deprecation flags and
> report errors in our testsuite.
... we can certainly make it if it's useful.
> CI-ifying of the above should be then also very simple. We'd just gather
> fresh QMP schema rather than using one from our test case pantry.
>
> Some time ago I also added testing of the commandline generator in
> libvirt with the most recent capabilities rather than using the
> historically declared capabilities that we've added when the test was
> created. This means that we actually test some valid combinations and
> also if stuff covered by our capability probing is removed the tests
> will catch it.
>
> I was also thinking of adding a tool which would use the above tests to
> attempt starting of a qemu process until the monitor shows up. That test
> then could also use -no-deprecated-options. I'm hoping waiting for the
> monitor is sufficient to excercise most of the code which could contain
> deprecation warnings. (Alternatively we can go through the
> pre-cpu-startup setup done on the monitor as well). Unfortunately doing
> this will not be as simple asi the test cases contain random disk paths
> and other resources which may not be available. This means that it will
> require some in-place modification and creative temporary resource
> usage.
Yes.
If you find QEMU makes testing something hard, we should talk. Together
we might find a reasonable way to make it easier.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 7:19 ` Peter Krempa
@ 2018-07-12 11:33 ` Markus Armbruster
0 siblings, 0 replies; 111+ messages in thread
From: Markus Armbruster @ 2018-07-12 11:33 UTC (permalink / raw)
To: Peter Krempa
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Cornelia Huck, QEMU Developers,
Christian Borntraeger
Peter Krempa <pkrempa@redhat.com> writes:
> On Thu, Jul 12, 2018 at 08:59:44 +0200, Markus Armbruster wrote:
>> Peter Krempa <pkrempa@redhat.com> writes:
>> > On Tue, Jul 10, 2018 at 17:01:22 +0200, Cornelia Huck wrote:
>> >> On Tue, 10 Jul 2018 16:39:31 +0200
>> >> Peter Krempa <pkrempa@redhat.com> wrote:
>
> [...]
>
>> > An option is to do a automatic testing where one of this approaches will
>> > be enabled. For that you need a way to generate configurations which
>> > libvirt would use in real life. We have a rather big collection of XMLs
>> > which describe a valid configuration but the problem with using them on
>> > a real qemu is that most of the disk paths/network targets/other
>> > resources are made up and making them work with a real qemu would range
>> > from being painful to being impossible.
>>
>> I sympathize. However, it's not clear which one's harder, providing
>> environments for a sufficiently wide range of configurations (possibly
>> mockups), or hacking QEMU to do nothing but check configuration. QEMU
>
> That's the main reason I think we should make it possible to use the
> data for the 'qemuxml2argv' test suite in libvirt. We require that new
> features are covered by this so that means that the testsuite is
> possibly the most comprehensive collection of libvirt configurations I
> know of.
>
> It's not perfect as we in many cases don't test any possible
> value but just try to excercise the code to generate them and others are
> left behind.
>
> Another historical problem was that we've defined a set of capabilities
> rather than using any real example to do this so many of the
> commandlines generated and tested are basically impossible to get in
> real life.
>
> That's why I added testing with real capabilities. We'll just need to
> generate a bunch of files to achieve full coverage here.
>
>
>> isn't designed for that, and configuration checking is intertwined with
>> everything else. Complete disentanglement looks impractical to me. I
>
> I agree. Getting anything special than the real codepath may create
> bubbles of problems still. On the other hand we'll need some guidance on
> what's sufficient to do to execute the deprecation detection code.
>
> This may require some coding style guidelines in qemu. E.g. no
> deprecation warnings after the vCPUs are started. Running a full
> operating system to check the warinigns would be utterly impractical.
Hot-plugging may get you deprecation warnings after vCPU start. But I
get what you mean. Rule of thumb: first check configuration is
well-formed, then do stuff that may fail when configuration asks for the
impossible, and only then do stuff that doesn't use configuration.
> Preferrably we would get away with starting qemu and waiting for the
> monitor to start.
We'll see how far that gets us.
>> guess we could do something useful at the QAPI level, though. Yet
>> another reason to qapify the command line...
>
> That would be great, but I think that there's a subset of things that
> can be deprecated but can't be expressed by schema. In such case we
> still need to run the programatic checks to see.
There will always be stuff the schema can't express without complicating
the schema language a lot, and stuff the schema could express, but only
at a cost in readability we prefer not to pay.
To get the most mileage out of schema introspection, we should strive
for making things visible in there whenever practical.
>> > If we start from scratch you then lack coverage.
>> >
>> >> If we fail with exit(1), can libvirt check any message that is logged
>> >> right before that?
>> >
>> > Yes we currently use this for very early failures which occur prior to
>> > the monitor working.
>> >
>> >> > To make any reasonable use of -no-deprecated-options we'd also need
>> >> > something that simulates qemu startup (no resources are touched in fact)
>> >> > so that we can run it against the testsuite. Otherwise the use will be
>> >> > limited to developers using it with the configuration they are
>> >> > currently testing.
>> >>>
>> >> Would that moan loudly that you should poke the libvirt developers if
>> >> some kind of testsuite failure is detected? Or am I misunderstanding?
>> >
>> > Generally it should make somebody complain. But there is a problem.
>> > Since we are talking deprecation it can't be enabled by default. And by
>> > not making it default most of the users will not enable that option.
>>
>> I don't think end users should do the work of catching use of deprecated
>> features. It's a CI job.
>>
>> In a CI context, we don't need fancy QMP infrastructure to communicate
>> "you used a deprecated feature", we can get away with printing an
>> explanation to stderr and exit(1). That should make CI fail, and the
>> failure should make a developer read the explanation. To unbreak CI, he
>> can either fix the problem right away, or file a BZ and suppress the CI
>> failure until it's fixed, say by downgrading --deprecated=error to
>> --deprecated=warn.
>
> Definitely. Plain untranslated error message is fine. The only thing is
> that it should be easy to detect. exit(1) is that solution. Or rather
> exit($VALUE_SPECIFIC_FOR_DEPRECATION) so that we can automatically
> discriminate test failures from deprecation warnings.
We'll have to search for of exit(X), where X is a bad idea.
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 6:32 ` Markus Armbruster
@ 2018-07-12 15:47 ` Thomas Huth
2018-07-13 11:35 ` Cornelia Huck
2018-07-16 9:33 ` Daniel P. Berrangé
0 siblings, 2 replies; 111+ messages in thread
From: Thomas Huth @ 2018-07-12 15:47 UTC (permalink / raw)
To: Markus Armbruster, Daniel P. Berrangé
Cc: Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
Cornelia Huck, QEMU Developers, Christian Borntraeger,
Peter Krempa
On 12.07.2018 08:32, Markus Armbruster wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
[...]
>> For libvirt, I think whenever something is proposed for deprecation
>> we could just CC libvir-list, or ask one of the libvirt people to
>> confirm its not being used. If it is, then we should file BZ against
>> libvirt.
>
> Makes sense, but relying on developers getting their cc: right every
> time is a setting us up for failure.
>
> Our tool to help with getting cc: wrong less often is the MAINTAINERS
> file. Could one of the libvirt developers watch changes to qemu-doc
> appendix "Deprecated features"? Would putting the appendix in its own
> .texi help with that?
Sound like a good idea to put the appendix in its own texi file. Then
add an "R: libvir-list" entry for that file to MAINTAINERS and we should
be fine (at least for the people who use the get_maintainers.pl script).
Thomas
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 15:47 ` Thomas Huth
@ 2018-07-13 11:35 ` Cornelia Huck
2018-07-16 10:06 ` Kashyap Chamarthy
2018-07-16 9:33 ` Daniel P. Berrangé
1 sibling, 1 reply; 111+ messages in thread
From: Cornelia Huck @ 2018-07-13 11:35 UTC (permalink / raw)
To: Thomas Huth
Cc: Markus Armbruster, Daniel P. Berrangé,
Kevin Wolf, Peter Maydell, Boris Fiuczynski, Qemu-block, Libvirt,
QEMU Developers, Christian Borntraeger, Peter Krempa
On Thu, 12 Jul 2018 17:47:00 +0200
Thomas Huth <thuth@redhat.com> wrote:
> On 12.07.2018 08:32, Markus Armbruster wrote:
> > Daniel P. Berrangé <berrange@redhat.com> writes:
> [...]
> >> For libvirt, I think whenever something is proposed for deprecation
> >> we could just CC libvir-list, or ask one of the libvirt people to
> >> confirm its not being used. If it is, then we should file BZ against
> >> libvirt.
> >
> > Makes sense, but relying on developers getting their cc: right every
> > time is a setting us up for failure.
> >
> > Our tool to help with getting cc: wrong less often is the MAINTAINERS
> > file. Could one of the libvirt developers watch changes to qemu-doc
> > appendix "Deprecated features"? Would putting the appendix in its own
> > .texi help with that?
>
> Sound like a good idea to put the appendix in its own texi file. Then
> add an "R: libvir-list" entry for that file to MAINTAINERS and we should
> be fine (at least for the people who use the get_maintainers.pl script).
+1, like that idea
Are there other consumers of QEMU's interfaces which should be cc:ed in
a similar way?
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-12 15:47 ` Thomas Huth
2018-07-13 11:35 ` Cornelia Huck
@ 2018-07-16 9:33 ` Daniel P. Berrangé
1 sibling, 0 replies; 111+ messages in thread
From: Daniel P. Berrangé @ 2018-07-16 9:33 UTC (permalink / raw)
To: Thomas Huth
Cc: Markus Armbruster, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, Cornelia Huck, QEMU Developers,
Christian Borntraeger, Peter Krempa
On Thu, Jul 12, 2018 at 05:47:00PM +0200, Thomas Huth wrote:
> On 12.07.2018 08:32, Markus Armbruster wrote:
> > Daniel P. Berrangé <berrange@redhat.com> writes:
> [...]
> >> For libvirt, I think whenever something is proposed for deprecation
> >> we could just CC libvir-list, or ask one of the libvirt people to
> >> confirm its not being used. If it is, then we should file BZ against
> >> libvirt.
> >
> > Makes sense, but relying on developers getting their cc: right every
> > time is a setting us up for failure.
> >
> > Our tool to help with getting cc: wrong less often is the MAINTAINERS
> > file. Could one of the libvirt developers watch changes to qemu-doc
> > appendix "Deprecated features"? Would putting the appendix in its own
> > .texi help with that?
>
> Sound like a good idea to put the appendix in its own texi file. Then
> add an "R: libvir-list" entry for that file to MAINTAINERS and we should
> be fine (at least for the people who use the get_maintainers.pl script).
That's a neat idea and gets my vote.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial
2018-07-13 11:35 ` Cornelia Huck
@ 2018-07-16 10:06 ` Kashyap Chamarthy
0 siblings, 0 replies; 111+ messages in thread
From: Kashyap Chamarthy @ 2018-07-16 10:06 UTC (permalink / raw)
To: Cornelia Huck
Cc: Thomas Huth, Kevin Wolf, Peter Maydell, Boris Fiuczynski,
Qemu-block, Libvirt, QEMU Developers, Christian Borntraeger,
Peter Krempa
On Fri, Jul 13, 2018 at 01:35:02PM +0200, Cornelia Huck wrote:
> On Thu, 12 Jul 2018 17:47:00 +0200
> Thomas Huth <thuth@redhat.com> wrote:
> > On 12.07.2018 08:32, Markus Armbruster wrote:
> > > Daniel P. Berrangé <berrange@redhat.com> writes:
[...]
> > > Our tool to help with getting cc: wrong less often is the MAINTAINERS
> > > file. Could one of the libvirt developers watch changes to qemu-doc
> > > appendix "Deprecated features"? Would putting the appendix in its own
> > > .texi help with that?
> >
> > Sound like a good idea to put the appendix in its own texi file. Then
> > add an "R: libvir-list" entry for that file to MAINTAINERS and we should
> > be fine (at least for the people who use the get_maintainers.pl script).
>
> +1, like that idea
>
> Are there other consumers of QEMU's interfaces which should be cc:ed in
> a similar way?
Perhaps starting with libvirt is fine -- as most of the open source
management applications rely on it. (But if we do know other consumers,
then they can be trivially added.)
--
/kashyap
^ permalink raw reply [flat|nested] 111+ messages in thread
end of thread, other threads:[~2018-07-16 10:06 UTC | newest]
Thread overview: 111+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-15 14:20 [Qemu-devel] [PULL 00/26] Block layer patches Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 01/26] qemu-img: Fix assert when mapping unaligned raw file Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 02/26] iotests: Add test 221 to catch qemu-img map regression Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 03/26] jobs: fix stale wording Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 04/26] jobs: fix verb references in docs Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 05/26] rbd: Drop deprecated -drive parameter "filename" Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 06/26] iscsi: " Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 07/26] block: Add block-specific QDict header Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c Kevin Wolf
2018-06-19 19:29 ` Eric Blake
2018-06-15 14:20 ` [Qemu-devel] [PULL 09/26] block: Fix -blockdev for certain non-string scalars Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 10/26] block: Fix -drive " Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 11/26] block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 12/26] block: Factor out qobject_input_visitor_new_flat_confused() Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 13/26] block: Make remaining uses of qobject input visitor more robust Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 14/26] block-qdict: Simplify qdict_flatten_qdict() Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 15/26] block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 16/26] block-qdict: Clean up qdict_crumple() a bit Kevin Wolf
2018-06-15 14:20 ` [Qemu-devel] [PULL 17/26] block-qdict: Simplify qdict_is_list() some Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 18/26] check-block-qdict: Rename qdict_flatten()'s variables for clarity Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 19/26] check-block-qdict: Cover flattening of empty lists and dictionaries Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 20/26] block: Fix -blockdev / blockdev-add for empty objects and arrays Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 21/26] rbd: New parameter auth-client-required Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 22/26] rbd: New parameter key-secret Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 23/26] block: Remove deprecated -drive geometry options Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 24/26] block: Remove deprecated -drive option addr Kevin Wolf
2018-06-15 14:21 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
2018-06-22 11:38 ` Christian Borntraeger
2018-06-22 12:51 ` [Qemu-devel] request a revert for "block: Remove deprecated -drive option serial" (was block: Remove deprecated -drive option serial) Christian Borntraeger
2018-06-22 20:08 ` [Qemu-devel] [qemu-s390x] " Thomas Huth
2018-06-22 12:55 ` [Qemu-devel] [PULL 25/26] block: Remove deprecated -drive option serial Kevin Wolf
2018-06-22 13:36 ` Christian Borntraeger
2018-06-22 14:00 ` Christian Borntraeger
2018-06-22 14:02 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-22 14:25 ` [Qemu-devel] " Kevin Wolf
2018-06-22 14:31 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-25 9:53 ` Daniel P. Berrangé
2018-06-25 11:41 ` Kevin Wolf
2018-06-25 11:45 ` Peter Krempa
2018-07-02 8:04 ` Kevin Wolf
2018-07-03 10:53 ` Christian Borntraeger
2018-07-03 11:22 ` Daniel P. Berrangé
2018-07-03 11:32 ` Kevin Wolf
2018-07-03 11:35 ` Peter Maydell
2018-07-03 12:38 ` Christian Borntraeger
2018-07-03 11:35 ` Daniel P. Berrangé
2018-07-04 13:02 ` Cornelia Huck
2018-07-04 13:34 ` Kevin Wolf
2018-07-04 13:43 ` Daniel P. Berrangé
2018-07-04 14:23 ` Kevin Wolf
2018-07-04 13:52 ` Christian Borntraeger
2018-07-04 13:58 ` Cornelia Huck
2018-07-04 16:14 ` Peter Maydell
2018-07-06 11:11 ` Cornelia Huck
2018-07-06 14:56 ` Kevin Wolf
2018-07-06 15:05 ` Daniel P. Berrangé
2018-07-06 15:10 ` Peter Maydell
2018-07-09 6:33 ` Markus Armbruster
2018-07-09 11:08 ` Cornelia Huck
2018-07-09 11:17 ` Daniel P. Berrangé
2018-07-12 6:32 ` Markus Armbruster
2018-07-12 15:47 ` Thomas Huth
2018-07-13 11:35 ` Cornelia Huck
2018-07-16 10:06 ` Kashyap Chamarthy
2018-07-16 9:33 ` Daniel P. Berrangé
2018-07-09 7:29 ` Peter Krempa
2018-07-10 5:59 ` Markus Armbruster
2018-07-10 14:22 ` Cornelia Huck
2018-07-10 14:38 ` Kevin Wolf
2018-07-12 6:38 ` Markus Armbruster
2018-07-12 6:51 ` Markus Armbruster
2018-07-12 7:48 ` Cornelia Huck
2018-07-12 9:05 ` Kevin Wolf
2018-07-12 11:14 ` Markus Armbruster
2018-07-12 7:00 ` Peter Krempa
2018-07-12 11:19 ` Markus Armbruster
2018-07-10 14:39 ` Peter Krempa
2018-07-10 15:01 ` Cornelia Huck
2018-07-10 15:24 ` Peter Krempa
2018-07-11 6:53 ` Thomas Huth
2018-07-11 7:24 ` Cornelia Huck
2018-07-12 6:40 ` Markus Armbruster
2018-07-12 6:59 ` Markus Armbruster
2018-07-12 7:19 ` Peter Krempa
2018-07-12 11:33 ` Markus Armbruster
2018-07-10 17:01 ` Daniel P. Berrangé
2018-07-11 13:48 ` Kashyap Chamarthy
2018-07-10 15:09 ` Peter Maydell
2018-07-10 16:59 ` Daniel P. Berrangé
2018-07-09 6:58 ` Thomas Huth
2018-07-09 11:58 ` Cornelia Huck
2018-06-22 14:38 ` [Qemu-devel] " Christian Borntraeger
2018-06-22 14:47 ` Peter Maydell
2018-06-22 15:01 ` Kevin Wolf
2018-06-22 15:50 ` Christian Borntraeger
2018-06-22 15:40 ` Daniel P. Berrangé
2018-06-22 17:54 ` Kevin Wolf
2018-06-25 11:18 ` Daniel P. Berrangé
2018-06-25 10:01 ` Peter Maydell
2018-06-25 10:31 ` Peter Krempa
2018-06-25 10:35 ` Peter Maydell
2018-06-25 7:44 ` Thomas Huth
2018-06-22 14:19 ` Markus Armbruster
2018-06-22 14:25 ` [Qemu-devel] [libvirt] " Daniel P. Berrangé
2018-06-22 14:30 ` Daniel P. Berrangé
2018-06-22 15:00 ` Eric Blake
2018-06-25 7:16 ` Peter Krempa
2018-06-25 8:23 ` Thomas Huth
2018-06-25 9:04 ` Daniel P. Berrangé
2018-06-15 14:21 ` [Qemu-devel] [PULL 26/26] block: Remove dead deprecation warning code Kevin Wolf
2018-06-15 16:28 ` [Qemu-devel] [PULL 00/26] Block layer patches Peter Maydell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.