[PULL 0/4] Block layer patches for 4.2.0-rc2

[PULL 0/4] Block layer patches for 4.2.0-rc2

[Kevin Wolf]

The following changes since commit bbe165740a96f67f48ecd3029dc26bdd7fd5192c:

  Merge remote-tracking branch 'remotes/vivier2/tags/ppc-for-4.2-pull-request' into staging (2019-11-18 11:09:06 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to b1f6a8e180367c945b67c6c9bbcec5c856be3779:

  iotests: Test multiple blockdev-snapshot calls (2019-11-18 17:50:08 +0100)

----------------------------------------------------------------
Block layer patches:

- Fix HMP savevm with -blockdev
- Minor iotests improvements

----------------------------------------------------------------
Kevin Wolf (3):
      iotests: Fix "no qualified output" error path
      block: Remove 'backing': null from bs->{explicit_,}options
      iotests: Test multiple blockdev-snapshot calls

Vladimir Sementsov-Ogievskiy (1):
      qemu-iotests/iotests.py: improve assert_qmp message

 block.c                       |   2 +
 tests/qemu-iotests/273        |  76 ++++++++++
 tests/qemu-iotests/273.out    | 337 ++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/check      |   2 +-
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |   4 +-
 6 files changed, 419 insertions(+), 3 deletions(-)
 create mode 100755 tests/qemu-iotests/273
 create mode 100644 tests/qemu-iotests/273.out

[PULL 1/4] qemu-iotests/iotests.py: improve assert_qmp message

[Kevin Wolf]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

From the two values compared, make it obvious which is found at path, and
which is expected.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/iotests.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 075f4739da..6a248472b9 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -731,8 +731,8 @@ class QMPTestCase(unittest.TestCase):
             self.fail('no match for "%s" in %s' % (str(result), str(value)))
         else:
             self.assertEqual(result, value,
-                             'values not equal "%s" and "%s"'
-                                 % (str(result), str(value)))
+                             '"%s" is "%s", expected "%s"'
+                                 % (path, str(result), str(value)))
 
     def assert_no_active_block_jobs(self):
         result = self.vm.qmp('query-block-jobs')
-- 
2.20.1

[PULL 2/4] iotests: Fix "no qualified output" error path

[Kevin Wolf]

The variable for error messages to be displayed is $results, not
$reason. Fix 'check' to print the "no qualified output" error message
again instead of having a failure without any message telling the user
why it failed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/check | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 71fe38834e..90970b0549 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -887,7 +887,7 @@ do
             if [ ! -f "$reference" ]
             then
                 status="fail"
-                reason="no qualified output"
+                results="no qualified output"
                 err=true
             else
                 if diff -w "$reference" $tmp.out >/dev/null 2>&1
-- 
2.20.1

[PULL 3/4] block: Remove 'backing': null from bs->{explicit_, }options

[Kevin Wolf]

bs->options and bs->explicit_options shouldn't contain any options for
child nodes. bdrv_open_inherited() takes care to remove any options that
match a child name after opening the image and the same is done when
reopening.

However, we miss the case of 'backing': null, which is a child option,
but results in no child being created. This means that a 'backing': null
remains in bs->options and bs->explicit_options.

A typical use for 'backing': null is in live snapshots: blockdev-add for
the qcow2 overlay makes sure not to open the backing file (because it is
already opened and blockdev-snapshot will attach it). After doing a
blockdev-snapshot, bs->options and bs->explicit_options become
inconsistent with the actual state (bs has a backing file now, but the
options still say null). On the next occasion that the image is
reopened, e.g. switching it from read-write to read-only when another
snapshot is taken, the option will take effect again and the node
incorrectly loses its backing file.

Fix bdrv_open_inherited() to remove the 'backing' option from
bs->options and bs->explicit_options even for the case where it
specifies that no backing file is wanted.

Reported-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
---
 block.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/block.c b/block.c
index 4cffc2bc35..473eb6eeaa 100644
--- a/block.c
+++ b/block.c
@@ -3019,6 +3019,8 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
                         "use \"backing\": null instead");
         }
         flags |= BDRV_O_NO_BACKING;
+        qdict_del(bs->explicit_options, "backing");
+        qdict_del(bs->options, "backing");
         qdict_del(options, "backing");
     }
 
-- 
2.20.1

[PULL 4/4] iotests: Test multiple blockdev-snapshot calls

[Kevin Wolf]

Test that doing a second blockdev-snapshot doesn't make the first
overlay's backing file go away.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
---
 tests/qemu-iotests/273     |  76 +++++++++
 tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 414 insertions(+)
 create mode 100755 tests/qemu-iotests/273
 create mode 100644 tests/qemu-iotests/273.out

diff --git a/tests/qemu-iotests/273 b/tests/qemu-iotests/273
new file mode 100755
index 0000000000..98a672516d
--- /dev/null
+++ b/tests/qemu-iotests/273
@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+#
+# Test multiple blockdev-snapshot calls with 'backing': null
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq=$(basename "$0")
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# This is a qcow2 regression test
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+do_run_qemu()
+{
+    echo Testing: "$@"
+    $QEMU -nographic -qmp-pretty stdio -nodefaults "$@"
+    echo
+}
+
+run_qemu()
+{
+    do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_qmp |
+        _filter_generated_node_ids | _filter_imgfmt | _filter_actual_image_size
+}
+
+TEST_IMG="$TEST_IMG.base" _make_test_img 64M
+TEST_IMG="$TEST_IMG.mid" _make_test_img -b "$TEST_IMG.base"
+_make_test_img -b "$TEST_IMG.mid"
+
+run_qemu \
+    -blockdev file,node-name=base,filename="$TEST_IMG.base" \
+     -blockdev file,node-name=midf,filename="$TEST_IMG.mid" \
+     -blockdev '{"driver":"qcow2","node-name":"mid","file":"midf","backing":null}' \
+     -blockdev file,node-name=topf,filename="$TEST_IMG" \
+     -blockdev '{"driver":"qcow2","file":"topf","node-name":"top","backing":null}' \
+<<EOF
+{"execute":"qmp_capabilities"}
+{"execute":"blockdev-snapshot","arguments":{"node":"base","overlay":"mid"}}
+{"execute":"blockdev-snapshot","arguments":{"node":"mid","overlay":"top"}}
+{"execute":"query-named-block-nodes"}
+{"execute":"x-debug-query-block-graph"}
+{"execute":"quit"}
+EOF
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/273.out b/tests/qemu-iotests/273.out
new file mode 100644
index 0000000000..c410fee5c4
--- /dev/null
+++ b/tests/qemu-iotests/273.out
@@ -0,0 +1,337 @@
+QA output created by 273
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.mid', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.mid
+Testing: -blockdev file,node-name=base,filename=TEST_DIR/t.IMGFMT.base -blockdev file,node-name=midf,filename=TEST_DIR/t.IMGFMT.mid -blockdev {"driver":"IMGFMT","node-name":"mid","file":"midf","backing":null} -blockdev file,node-name=topf,filename=TEST_DIR/t.IMGFMT -blockdev {"driver":"IMGFMT","file":"topf","node-name":"top","backing":null}
+{
+    QMP_VERSION
+}
+{
+    "return": {
+    }
+}
+{
+    "return": {
+    }
+}
+{
+    "return": {
+    }
+}
+{
+    "return": [
+        {
+            "iops_rd": 0,
+            "detect_zeroes": "off",
+            "image": {
+                "backing-image": {
+                    "backing-image": {
+                        "virtual-size": 197120,
+                        "filename": "TEST_DIR/t.IMGFMT.base",
+                        "format": "file",
+                        "actual-size": SIZE,
+                        "dirty-flag": false
+                    },
+                    "backing-filename-format": "file",
+                    "virtual-size": 67108864,
+                    "filename": "TEST_DIR/t.IMGFMT.mid",
+                    "cluster-size": 65536,
+                    "format": "IMGFMT",
+                    "actual-size": SIZE,
+                    "format-specific": {
+                        "type": "IMGFMT",
+                        "data": {
+                            "compat": "1.1",
+                            "lazy-refcounts": false,
+                            "refcount-bits": 16,
+                            "corrupt": false
+                        }
+                    },
+                    "full-backing-filename": "TEST_DIR/t.IMGFMT.base",
+                    "backing-filename": "TEST_DIR/t.IMGFMT.base",
+                    "dirty-flag": false
+                },
+                "backing-filename-format": "IMGFMT",
+                "virtual-size": 67108864,
+                "filename": "TEST_DIR/t.IMGFMT",
+                "cluster-size": 65536,
+                "format": "IMGFMT",
+                "actual-size": SIZE,
+                "format-specific": {
+                    "type": "IMGFMT",
+                    "data": {
+                        "compat": "1.1",
+                        "lazy-refcounts": false,
+                        "refcount-bits": 16,
+                        "corrupt": false
+                    }
+                },
+                "full-backing-filename": "TEST_DIR/t.IMGFMT.mid",
+                "backing-filename": "TEST_DIR/t.IMGFMT.mid",
+                "dirty-flag": false
+            },
+            "iops_wr": 0,
+            "ro": false,
+            "node-name": "top",
+            "backing_file_depth": 2,
+            "drv": "IMGFMT",
+            "iops": 0,
+            "bps_wr": 0,
+            "write_threshold": 0,
+            "backing_file": "TEST_DIR/t.IMGFMT.mid",
+            "encrypted": false,
+            "bps": 0,
+            "bps_rd": 0,
+            "cache": {
+                "no-flush": false,
+                "direct": false,
+                "writeback": true
+            },
+            "file": "TEST_DIR/t.IMGFMT",
+            "encryption_key_missing": false
+        },
+        {
+            "iops_rd": 0,
+            "detect_zeroes": "off",
+            "image": {
+                "virtual-size": 197120,
+                "filename": "TEST_DIR/t.IMGFMT",
+                "format": "file",
+                "actual-size": SIZE,
+                "dirty-flag": false
+            },
+            "iops_wr": 0,
+            "ro": false,
+            "node-name": "topf",
+            "backing_file_depth": 0,
+            "drv": "file",
+            "iops": 0,
+            "bps_wr": 0,
+            "write_threshold": 0,
+            "encrypted": false,
+            "bps": 0,
+            "bps_rd": 0,
+            "cache": {
+                "no-flush": false,
+                "direct": false,
+                "writeback": true
+            },
+            "file": "TEST_DIR/t.IMGFMT",
+            "encryption_key_missing": false
+        },
+        {
+            "iops_rd": 0,
+            "detect_zeroes": "off",
+            "image": {
+                "backing-image": {
+                    "virtual-size": 197120,
+                    "filename": "TEST_DIR/t.IMGFMT.base",
+                    "format": "file",
+                    "actual-size": SIZE,
+                    "dirty-flag": false
+                },
+                "backing-filename-format": "file",
+                "virtual-size": 67108864,
+                "filename": "TEST_DIR/t.IMGFMT.mid",
+                "cluster-size": 65536,
+                "format": "IMGFMT",
+                "actual-size": SIZE,
+                "format-specific": {
+                    "type": "IMGFMT",
+                    "data": {
+                        "compat": "1.1",
+                        "lazy-refcounts": false,
+                        "refcount-bits": 16,
+                        "corrupt": false
+                    }
+                },
+                "full-backing-filename": "TEST_DIR/t.IMGFMT.base",
+                "backing-filename": "TEST_DIR/t.IMGFMT.base",
+                "dirty-flag": false
+            },
+            "iops_wr": 0,
+            "ro": true,
+            "node-name": "mid",
+            "backing_file_depth": 1,
+            "drv": "IMGFMT",
+            "iops": 0,
+            "bps_wr": 0,
+            "write_threshold": 0,
+            "backing_file": "TEST_DIR/t.IMGFMT.base",
+            "encrypted": false,
+            "bps": 0,
+            "bps_rd": 0,
+            "cache": {
+                "no-flush": false,
+                "direct": false,
+                "writeback": true
+            },
+            "file": "TEST_DIR/t.IMGFMT.mid",
+            "encryption_key_missing": false
+        },
+        {
+            "iops_rd": 0,
+            "detect_zeroes": "off",
+            "image": {
+                "virtual-size": 197120,
+                "filename": "TEST_DIR/t.IMGFMT.mid",
+                "format": "file",
+                "actual-size": SIZE,
+                "dirty-flag": false
+            },
+            "iops_wr": 0,
+            "ro": false,
+            "node-name": "midf",
+            "backing_file_depth": 0,
+            "drv": "file",
+            "iops": 0,
+            "bps_wr": 0,
+            "write_threshold": 0,
+            "encrypted": false,
+            "bps": 0,
+            "bps_rd": 0,
+            "cache": {
+                "no-flush": false,
+                "direct": false,
+                "writeback": true
+            },
+            "file": "TEST_DIR/t.IMGFMT.mid",
+            "encryption_key_missing": false
+        },
+        {
+            "iops_rd": 0,
+            "detect_zeroes": "off",
+            "image": {
+                "virtual-size": 197120,
+                "filename": "TEST_DIR/t.IMGFMT.base",
+                "format": "file",
+                "actual-size": SIZE,
+                "dirty-flag": false
+            },
+            "iops_wr": 0,
+            "ro": true,
+            "node-name": "base",
+            "backing_file_depth": 0,
+            "drv": "file",
+            "iops": 0,
+            "bps_wr": 0,
+            "write_threshold": 0,
+            "encrypted": false,
+            "bps": 0,
+            "bps_rd": 0,
+            "cache": {
+                "no-flush": false,
+                "direct": false,
+                "writeback": true
+            },
+            "file": "TEST_DIR/t.IMGFMT.base",
+            "encryption_key_missing": false
+        }
+    ]
+}
+{
+    "return": {
+        "edges": [
+            {
+                "name": "file",
+                "parent": 5,
+                "shared-perm": [
+                    "graph-mod",
+                    "write-unchanged",
+                    "consistent-read"
+                ],
+                "perm": [
+                    "resize",
+                    "write",
+                    "consistent-read"
+                ],
+                "child": 4
+            },
+            {
+                "name": "backing",
+                "parent": 5,
+                "shared-perm": [
+                    "graph-mod",
+                    "resize",
+                    "write-unchanged",
+                    "write",
+                    "consistent-read"
+                ],
+                "perm": [
+                ],
+                "child": 3
+            },
+            {
+                "name": "file",
+                "parent": 3,
+                "shared-perm": [
+                    "graph-mod",
+                    "write-unchanged",
+                    "consistent-read"
+                ],
+                "perm": [
+                    "consistent-read"
+                ],
+                "child": 2
+            },
+            {
+                "name": "backing",
+                "parent": 3,
+                "shared-perm": [
+                    "graph-mod",
+                    "resize",
+                    "write-unchanged",
+                    "write",
+                    "consistent-read"
+                ],
+                "perm": [
+                ],
+                "child": 1
+            }
+        ],
+        "nodes": [
+            {
+                "name": "top",
+                "type": "block-driver",
+                "id": 5
+            },
+            {
+                "name": "topf",
+                "type": "block-driver",
+                "id": 4
+            },
+            {
+                "name": "mid",
+                "type": "block-driver",
+                "id": 3
+            },
+            {
+                "name": "midf",
+                "type": "block-driver",
+                "id": 2
+            },
+            {
+                "name": "base",
+                "type": "block-driver",
+                "id": 1
+            }
+        ]
+    }
+}
+{
+    "return": {
+    }
+}
+{
+    "timestamp": {
+        "seconds":  TIMESTAMP,
+        "microseconds":  TIMESTAMP
+    },
+    "event": "SHUTDOWN",
+    "data": {
+        "guest": false,
+        "reason": "host-qmp-quit"
+    }
+}
+
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 065040398d..c56bb69031 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -284,3 +284,4 @@
 268 rw auto quick
 270 rw backing quick
 272 rw
+273 backing quick
-- 
2.20.1

Re: [PULL 0/4] Block layer patches for 4.2.0-rc2

[Peter Maydell]

On Mon, 18 Nov 2019 at 17:03, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit bbe165740a96f67f48ecd3029dc26bdd7fd5192c:
>
>   Merge remote-tracking branch 'remotes/vivier2/tags/ppc-for-4.2-pull-request' into staging (2019-11-18 11:09:06 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to b1f6a8e180367c945b67c6c9bbcec5c856be3779:
>
>   iotests: Test multiple blockdev-snapshot calls (2019-11-18 17:50:08 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Fix HMP savevm with -blockdev
> - Minor iotests improvements
>
> ----------------------------------------------------------------
> Kevin Wolf (3):
>       iotests: Fix "no qualified output" error path
>       block: Remove 'backing': null from bs->{explicit_,}options
>       iotests: Test multiple blockdev-snapshot calls
>
> Vladimir Sementsov-Ogievskiy (1):
>       qemu-iotests/iotests.py: improve assert_qmp message


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2
for any user-visible changes.

-- PMM