[Qemu-devel] [PATCH for-3.0 0/3] Fix query-blockstats with -blockdev

[Qemu-devel] [PATCH for-3.0 0/3] Fix query-blockstats with -blockdev

[Kevin Wolf]

Kevin Wolf (3):
  block/qapi: Add 'qdev' field to query-blockstats result
  block/qapi: Include anonymous BBs in query-blockstats
  qemu-iotests: Test query-blockstats with -drive and -blockdev

 qapi/block-core.json       |   5 ++-
 block/qapi.c               |  16 ++++++-
 tests/qemu-iotests/227     | 101 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/227.out |  42 +++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 5 files changed, 163 insertions(+), 2 deletions(-)
 create mode 100755 tests/qemu-iotests/227
 create mode 100644 tests/qemu-iotests/227.out

-- 
2.13.6

[Qemu-devel] [PATCH for-3.0 1/3] block/qapi: Add 'qdev' field to query-blockstats result

[Kevin Wolf]

Like for query-block, the client needs to identify which BlockBackend
the returned data is for. Anonymous BlockBackends are identified by the
device model they are attached to. Add a 'qdev' field that contains the
qdev ID or QOM path of the attached device model.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 qapi/block-core.json |  5 ++++-
 block/qapi.c         | 10 ++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index d40d5ecc3b..3f09f62fa1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -866,6 +866,9 @@
 #
 # @node-name: The node name of the device. (Since 2.3)
 #
+# @qdev: The qdev ID, or if no ID is assigned, the QOM path of the block
+#        device. (since 3.0)
+#
 # @stats:  A @BlockDeviceStats for the device.
 #
 # @parent: This describes the file block device if it has one.
@@ -879,7 +882,7 @@
 # Since: 0.14.0
 ##
 { 'struct': 'BlockStats',
-  'data': {'*device': 'str', '*node-name': 'str',
+  'data': {'*device': 'str', '*qdev': 'str', '*node-name': 'str',
            'stats': 'BlockDeviceStats',
            '*parent': 'BlockStats',
            '*backing': 'BlockStats'} }
diff --git a/block/qapi.c b/block/qapi.c
index e12968fec8..50f867d634 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -597,11 +597,21 @@ BlockStatsList *qmp_query_blockstats(bool has_query_nodes,
             BlockStatsList *info = g_malloc0(sizeof(*info));
             AioContext *ctx = blk_get_aio_context(blk);
             BlockStats *s;
+            char *qdev;
 
             aio_context_acquire(ctx);
             s = bdrv_query_bds_stats(blk_bs(blk), true);
             s->has_device = true;
             s->device = g_strdup(blk_name(blk));
+
+            qdev = blk_get_attached_dev_id(blk);
+            if (qdev && *qdev) {
+                s->has_qdev = true;
+                s->qdev = qdev;
+            } else {
+                g_free(qdev);
+            }
+
             bdrv_query_blk_stats(s->stats, blk);
             aio_context_release(ctx);
 
-- 
2.13.6

Re: [Qemu-devel] [PATCH for-3.0 1/3] block/qapi: Add 'qdev' field to query-blockstats result

[Eric Blake]

On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> Like for query-block, the client needs to identify which BlockBackend
> the returned data is for. Anonymous BlockBackends are identified by the
> device model they are attached to. Add a 'qdev' field that contains the
> qdev ID or QOM path of the attached device model.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---

> @@ -879,7 +882,7 @@
>   # Since: 0.14.0
>   ##
>   { 'struct': 'BlockStats',
> -  'data': {'*device': 'str', '*node-name': 'str',
> +  'data': {'*device': 'str', '*qdev': 'str', '*node-name': 'str',
>              'stats': 'BlockDeviceStats',
>              '*parent': 'BlockStats',
>              '*backing': 'BlockStats'} }

Can we also update the example under query-blockstats a few lines later 
to show the added field?

At any rate, it looks helpful. Although this is borderline new feature, 
it resolves a need identified in libvirt using -blockdev, and so I can 
live with it as a 3.0 bugfix.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH for-3.0 1/3] block/qapi: Add 'qdev' field to query-blockstats result

[Kevin Wolf]

Am 27.07.2018 um 17:07 hat Eric Blake geschrieben:
> On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> > Like for query-block, the client needs to identify which BlockBackend
> > the returned data is for. Anonymous BlockBackends are identified by the
> > device model they are attached to. Add a 'qdev' field that contains the
> > qdev ID or QOM path of the attached device model.
> > 
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> 
> > @@ -879,7 +882,7 @@
> >   # Since: 0.14.0
> >   ##
> >   { 'struct': 'BlockStats',
> > -  'data': {'*device': 'str', '*node-name': 'str',
> > +  'data': {'*device': 'str', '*qdev': 'str', '*node-name': 'str',
> >              'stats': 'BlockDeviceStats',
> >              '*parent': 'BlockStats',
> >              '*backing': 'BlockStats'} }
> 
> Can we also update the example under query-blockstats a few lines later to
> show the added field?

I'll add the qdev field, but the example is hopelessly outdated anyway.
Sounds like something for another patch.

> At any rate, it looks helpful. Although this is borderline new feature, it
> resolves a need identified in libvirt using -blockdev, and so I can live
> with it as a 3.0 bugfix.
> 
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks.

Kevin

Re: [Qemu-devel] [PATCH for-3.0 1/3] block/qapi: Add 'qdev' field to query-blockstats result

[Peter Krempa]

[-- Attachment #1: Type: text/plain, Size: 1468 bytes --]

On Fri, Jul 27, 2018 at 17:58:54 +0200, Kevin Wolf wrote:
> Am 27.07.2018 um 17:07 hat Eric Blake geschrieben:
> > On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> > > Like for query-block, the client needs to identify which BlockBackend
> > > the returned data is for. Anonymous BlockBackends are identified by the
> > > device model they are attached to. Add a 'qdev' field that contains the
> > > qdev ID or QOM path of the attached device model.
> > > 
> > > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > > ---
> > 
> > > @@ -879,7 +882,7 @@
> > >   # Since: 0.14.0
> > >   ##
> > >   { 'struct': 'BlockStats',
> > > -  'data': {'*device': 'str', '*node-name': 'str',
> > > +  'data': {'*device': 'str', '*qdev': 'str', '*node-name': 'str',
> > >              'stats': 'BlockDeviceStats',
> > >              '*parent': 'BlockStats',
> > >              '*backing': 'BlockStats'} }
> > 
> > Can we also update the example under query-blockstats a few lines later to
> > show the added field?
> 
> I'll add the qdev field, but the example is hopelessly outdated anyway.
> Sounds like something for another patch.

When doing further cleanups it would be worth also noting that the
'query-nodes' argument of 'query-blockstats' does not return correct
information.

Right now it creates a false impression that it just changes the output
from nested to linear and includes filter nodes, but in fact the data is
not correct in that mode.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[Qemu-devel] [PATCH for-3.0 2/3] block/qapi: Include anonymous BBs in query-blockstats

[Kevin Wolf]

Consistent with query-block, query-blockstats should not only include
named BlockBackends, but also those that are anonmyous, but belong to a
device model.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block/qapi.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/block/qapi.c b/block/qapi.c
index 50f867d634..339727f0f4 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -593,12 +593,16 @@ BlockStatsList *qmp_query_blockstats(bool has_query_nodes,
             p_next = &info->next;
         }
     } else {
-        for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
+        for (blk = blk_all_next(NULL); blk; blk = blk_all_next(blk)) {
             BlockStatsList *info = g_malloc0(sizeof(*info));
             AioContext *ctx = blk_get_aio_context(blk);
             BlockStats *s;
             char *qdev;
 
+            if (!*blk_name(blk) && !blk_get_attached_dev(blk)) {
+                continue;
+            }
+
             aio_context_acquire(ctx);
             s = bdrv_query_bds_stats(blk_bs(blk), true);
             s->has_device = true;
-- 
2.13.6

Re: [Qemu-devel] [PATCH for-3.0 2/3] block/qapi: Include anonymous BBs in query-blockstats

[Eric Blake]

On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> Consistent with query-block, query-blockstats should not only include
> named BlockBackends, but also those that are anonmyous, but belong to a

s/anonmyous/anonymous/

> device model.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>   block/qapi.c | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH for-3.0 3/3] qemu-iotests: Test query-blockstats with -drive and -blockdev

[Kevin Wolf]

Make sure that query-blockstats returns information for every
BlockBackend that is named or attached to a device model (or both).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/227     | 101 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/227.out |  42 +++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 144 insertions(+)
 create mode 100755 tests/qemu-iotests/227
 create mode 100644 tests/qemu-iotests/227.out

diff --git a/tests/qemu-iotests/227 b/tests/qemu-iotests/227
new file mode 100755
index 0000000000..c840e0915d
--- /dev/null
+++ b/tests/qemu-iotests/227
@@ -0,0 +1,101 @@
+#!/bin/bash
+#
+# Test query-blockstats with different ways to create a BB
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=kwolf@redhat.com
+
+seq=$(basename $0)
+echo "QA output created by $seq"
+
+here=$PWD
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt generic
+_supported_proto file
+_supported_os Linux
+
+function do_run_qemu()
+{
+    echo Testing: "$@"
+    $QEMU -nographic -qmp stdio -serial none "$@"
+    echo
+}
+
+function run_qemu()
+{
+    do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \
+                          | _filter_qemu | _filter_imgfmt \
+                          | _filter_generated_node_ids
+}
+
+echo
+echo '=== blockstats with -drive if=virtio ==='
+echo
+
+run_qemu -drive driver=null-co,if=virtio <<EOF
+{ "execute": "qmp_capabilities" }
+{ "execute": "query-blockstats"}
+{ "execute": "quit" }
+EOF
+
+echo
+echo '=== blockstats with -drive if=none ==='
+echo
+
+run_qemu -drive driver=null-co,if=none <<EOF
+{ "execute": "qmp_capabilities" }
+{ "execute": "query-blockstats"}
+{ "execute": "quit" }
+EOF
+
+echo
+echo '=== blockstats with -blockdev ==='
+echo
+
+run_qemu -blockdev driver=null-co,node-name=null <<EOF
+{ "execute": "qmp_capabilities" }
+{ "execute": "query-blockstats"}
+{ "execute": "quit" }
+EOF
+
+echo
+echo '=== blockstats with -blockdev and -device ==='
+echo
+
+run_qemu -blockdev driver=null-co,node-name=null -device virtio-blk,drive=null,id=virtio0 <<EOF
+{ "execute": "qmp_capabilities" }
+{ "execute": "query-blockstats"}
+{ "execute": "quit" }
+EOF
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/227.out b/tests/qemu-iotests/227.out
new file mode 100644
index 0000000000..63373efe21
--- /dev/null
+++ b/tests/qemu-iotests/227.out
@@ -0,0 +1,42 @@
+QA output created by 227
+
+=== blockstats with -drive if=virtio ===
+
+Testing: -drive driver=null-co,if=virtio
+QMP_VERSION
+{"return": {}}
+{"return": [{"device": "virtio0", "stats": {"flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [], "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": true, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0}, "node-name": "NODE_NAME", "qdev": "/machine/peripheral-anon/device[0]/virtio-backend"}]}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
+
+
+=== blockstats with -drive if=none ===
+
+Testing: -drive driver=null-co,if=none
+QMP_VERSION
+{"return": {}}
+{"return": [{"device": "none0", "stats": {"flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [], "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": true, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0}, "node-name": "NODE_NAME"}]}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
+
+
+=== blockstats with -blockdev ===
+
+Testing: -blockdev driver=null-co,node-name=null
+QMP_VERSION
+{"return": {}}
+{"return": []}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
+
+
+=== blockstats with -blockdev and -device ===
+
+Testing: -blockdev driver=null-co,node-name=null -device virtio-blk,drive=null,id=virtio0
+QMP_VERSION
+{"return": {}}
+{"return": [{"device": "", "stats": {"flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [], "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0}, "node-name": "null", "qdev": "/machine/peripheral/virtio0/virtio-backend"}]}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}}
+
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 68f66259b2..b973dc842d 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -224,3 +224,4 @@
 223 rw auto quick
 225 rw auto quick
 226 auto quick
+227 auto quick
-- 
2.13.6

Re: [Qemu-devel] [PATCH for-3.0 3/3] qemu-iotests: Test query-blockstats with -drive and -blockdev

[Eric Blake]

On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> Make sure that query-blockstats returns information for every
> BlockBackend that is named or attached to a device model (or both).

Needed for libvirt switching over to -blockdev, so even though we're 
rather late in the 3.0 cycle, I can understand it going in now.

> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---

Reviewed-by: Eric Blake <eblake@redhat.com>

> +++ b/tests/qemu-iotests/227.out
> @@ -0,0 +1,42 @@
> +QA output created by 227
> +
> +=== blockstats with -drive if=virtio ===
> +
> +Testing: -drive driver=null-co,if=virtio
> +QMP_VERSION
> +{"return": {}}
> +{"return": [{"device": "virtio0", "stats": {"flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [], "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": true, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0}, "node-name": "NODE_NAME", "qdev": "/machine/peripheral-anon/device[0]/virtio-backend"}]}

Long lines. Would it be worth writing the test to use QMP pretty mode?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH for-3.0 3/3] qemu-iotests: Test query-blockstats with -drive and -blockdev

[Kevin Wolf]

Am 27.07.2018 um 17:12 hat Eric Blake geschrieben:
> On 07/27/2018 09:15 AM, Kevin Wolf wrote:
> > Make sure that query-blockstats returns information for every
> > BlockBackend that is named or attached to a device model (or both).
> 
> Needed for libvirt switching over to -blockdev, so even though we're rather
> late in the 3.0 cycle, I can understand it going in now.
> 
> > 
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> 
> Reviewed-by: Eric Blake <eblake@redhat.com>
> 
> > +++ b/tests/qemu-iotests/227.out
> > @@ -0,0 +1,42 @@
> > +QA output created by 227
> > +
> > +=== blockstats with -drive if=virtio ===
> > +
> > +Testing: -drive driver=null-co,if=virtio
> > +QMP_VERSION
> > +{"return": {}}
> > +{"return": [{"device": "virtio0", "stats": {"flush_total_time_ns": 0, "wr_highest_offset": 0, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [], "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 0, "flush_operations": 0, "wr_operations": 0, "rd_merged": 0, "rd_bytes": 0, "invalid_flush_operations": 0, "account_failed": true, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0}, "node-name": "NODE_NAME", "qdev": "/machine/peripheral-anon/device[0]/virtio-backend"}]}
> 
> Long lines. Would it be worth writing the test to use QMP pretty mode?

Good idea. I'll do that before applying the patches.

Kevin