* [PATCH v5 00/12] Exposing backing-chain allocation over NBD
@ 2020-10-23 18:36 Eric Blake
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
` (12 more replies)
0 siblings, 13 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, stefanha, qemu-block, rjones
v4 was here:
https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg02708.html
Since then:
- rebase to master
- patches 1, 2, and 12 are new based on Vladimir's observation of QAPI_LIST_ADD
- patches 10-11 are new based on prior discussion on exposing actual
depth in addition to a tri-state encoding
- patch 3 has a nasty bug fixed that was causing iotest failures
- patch 6 updated to take advantage of patch 2
- other minor tweaks based on review comments
001/12:[down] 'qapi: Move GenericList to qapi/util.h'
002/12:[down] 'qapi: Make QAPI_LIST_ADD() public'
003/12:[0002] [FC] 'nbd: Utilize QAPI_CLONE for type conversion'
004/12:[0010] [FC] 'nbd: Add new qemu:allocation-depth metadata context'
005/12:[----] [--] 'nbd: Add 'qemu-nbd -A' to expose allocation depth'
006/12:[0014] [FC] 'nbd: Update qapi to support exporting multiple bitmaps'
007/12:[----] [--] 'nbd: Simplify qemu bitmap context name'
008/12:[----] [--] 'nbd: Refactor counting of metadata contexts'
009/12:[0017] [FC] 'nbd: Allow export of multiple bitmaps for one device'
010/12:[down] 'block: Return depth level during bdrv_is_allocated_above'
011/12:[down] 'nbd: Expose actual depth in qemu:allocation-depth'
012/12:[down] 'qapi: Use QAPI_LIST_ADD() where possible'
Eric Blake (12):
qapi: Move GenericList to qapi/util.h
qapi: Make QAPI_LIST_ADD() public
nbd: Utilize QAPI_CLONE for type conversion
nbd: Add new qemu:allocation-depth metadata context
nbd: Add 'qemu-nbd -A' to expose allocation depth
nbd: Update qapi to support exporting multiple bitmaps
nbd: Simplify qemu bitmap context name
nbd: Refactor counting of metadata contexts
nbd: Allow export of multiple bitmaps for one device
block: Return depth level during bdrv_is_allocated_above
nbd: Expose actual depth in qemu:allocation-depth
qapi: Use QAPI_LIST_ADD() where possible
docs/devel/writing-qmp-commands.txt | 13 +-
docs/interop/nbd.txt | 31 +++-
docs/system/deprecated.rst | 4 +-
docs/tools/qemu-nbd.rst | 8 +-
qapi/block-core.json | 7 +-
qapi/block-export.json | 22 ++-
include/qapi/visitor.h | 9 +-
hw/net/rocker/rocker_fp.h | 2 +-
include/block/nbd.h | 14 +-
include/qapi/util.h | 16 ++
block/io.c | 11 +-
block.c | 15 +-
block/commit.c | 2 +-
block/gluster.c | 19 +--
block/mirror.c | 2 +-
block/stream.c | 2 +-
blockdev-nbd.c | 28 ++--
chardev/char.c | 21 ++-
hw/core/machine.c | 6 +-
hw/net/rocker/rocker.c | 8 +-
hw/net/rocker/rocker_fp.c | 14 +-
hw/net/virtio-net.c | 21 +--
migration/migration.c | 7 +-
migration/postcopy-ram.c | 7 +-
monitor/hmp-cmds.c | 11 +-
nbd/server.c | 222 ++++++++++++++++++++++------
qemu-img.c | 5 +-
qemu-nbd.c | 30 ++--
qga/commands-posix.c | 13 +-
qga/commands-win32.c | 17 +--
qga/commands.c | 6 +-
qom/qom-qmp-cmds.c | 29 +---
target/arm/helper.c | 6 +-
target/arm/monitor.c | 13 +-
target/i386/cpu.c | 6 +-
target/mips/helper.c | 6 +-
target/s390x/cpu_models.c | 12 +-
tests/test-clone-visitor.c | 7 +-
tests/test-qobject-output-visitor.c | 42 +++---
tests/test-visitor-serialization.c | 5 +-
trace/qmp.c | 22 ++-
ui/vnc.c | 21 +--
util/qemu-config.c | 14 +-
target/ppc/translate_init.c.inc | 12 +-
tests/qemu-iotests/291 | 6 +-
tests/qemu-iotests/309 | 73 +++++++++
tests/qemu-iotests/309.out | 22 +++
tests/qemu-iotests/group | 1 +
48 files changed, 529 insertions(+), 361 deletions(-)
create mode 100755 tests/qemu-iotests/309
create mode 100644 tests/qemu-iotests/309.out
--
2.29.0
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-24 9:06 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:18 ` Markus Armbruster
2020-10-23 18:36 ` [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public Eric Blake
` (11 subsequent siblings)
12 siblings, 2 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, vsementsov, Michael Roth, qemu-block, Markus Armbruster,
rjones, stefanha
Placing GenericList in util.h will make it easier for the next patch
to promote QAPI_LIST_ADD() into a public macro without requiring more
files to include the unrelated visitor.h.
However, we can't also move GenericAlternate; this is because it would
introduce a circular dependency: qapi-builtin-types.h needs a complete
definition of QEnumLookup (so it includes qapi/util.h), and
GenericAlternate needs a complete definition of QType (declared in
qapi-builtin-types.h). Leaving GenericAlternate in visitor.h breaks
the cycle, and doesn't matter since we don't have any further planned
uses for that type outside of visitors.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
include/qapi/visitor.h | 9 +--------
include/qapi/util.h | 8 ++++++++
2 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index ebc19ede7fff..8c2e1c29ad8b 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -16,6 +16,7 @@
#define QAPI_VISITOR_H
#include "qapi/qapi-builtin-types.h"
+#include "qapi/util.h"
/*
* The QAPI schema defines both a set of C data types, and a QMP wire
@@ -228,14 +229,6 @@
/*** Useful types ***/
-/* This struct is layout-compatible with all other *List structs
- * created by the QAPI generator. It is used as a typical
- * singly-linked list. */
-typedef struct GenericList {
- struct GenericList *next;
- char padding[];
-} GenericList;
-
/* This struct is layout-compatible with all Alternate types
* created by the QAPI generator. */
typedef struct GenericAlternate {
diff --git a/include/qapi/util.h b/include/qapi/util.h
index a7c3c6414874..50201896c7a4 100644
--- a/include/qapi/util.h
+++ b/include/qapi/util.h
@@ -11,6 +11,14 @@
#ifndef QAPI_UTIL_H
#define QAPI_UTIL_H
+/* This struct is layout-compatible with all other *List structs
+ * created by the QAPI generator. It is used as a typical
+ * singly-linked list. */
+typedef struct GenericList {
+ struct GenericList *next;
+ char padding[];
+} GenericList;
+
typedef struct QEnumLookup {
const char *const *array;
int size;
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-24 9:10 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:25 ` Markus Armbruster
2020-10-23 18:36 ` [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion Eric Blake
` (10 subsequent siblings)
12 siblings, 2 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, vsementsov, Michael Roth, qemu-block, Markus Armbruster,
rjones, Max Reitz, stefanha
We have a useful macro for inserting at the front of any
QAPI-generated list; move it from block.c to qapi/util.h so more
places can use it, including one earlier place in block.c.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
include/qapi/util.h | 8 ++++++++
block.c | 15 +++------------
2 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/include/qapi/util.h b/include/qapi/util.h
index 50201896c7a4..b6083055ce69 100644
--- a/include/qapi/util.h
+++ b/include/qapi/util.h
@@ -30,4 +30,12 @@ int qapi_enum_parse(const QEnumLookup *lookup, const char *buf,
int parse_qapi_name(const char *name, bool complete);
+/* For any GenericList @list, insert @element at the front. */
+#define QAPI_LIST_ADD(list, element) do { \
+ typeof(list) _tmp = g_new(typeof(*(list)), 1); \
+ _tmp->value = (element); \
+ _tmp->next = (list); \
+ (list) = _tmp; \
+} while (0)
+
#endif
diff --git a/block.c b/block.c
index 430edf79bb10..45bd79299611 100644
--- a/block.c
+++ b/block.c
@@ -39,6 +39,7 @@
#include "qapi/qmp/qstring.h"
#include "qapi/qobject-output-visitor.h"
#include "qapi/qapi-visit-block-core.h"
+#include "qapi/util.h"
#include "sysemu/block-backend.h"
#include "sysemu/sysemu.h"
#include "qemu/notify.h"
@@ -5211,7 +5212,7 @@ BlockDriverState *bdrv_find_node(const char *node_name)
BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
Error **errp)
{
- BlockDeviceInfoList *list, *entry;
+ BlockDeviceInfoList *list;
BlockDriverState *bs;
list = NULL;
@@ -5221,22 +5222,12 @@ BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
qapi_free_BlockDeviceInfoList(list);
return NULL;
}
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = list;
- list = entry;
+ QAPI_LIST_ADD(list, info);
}
return list;
}
-#define QAPI_LIST_ADD(list, element) do { \
- typeof(list) _tmp = g_new(typeof(*(list)), 1); \
- _tmp->value = (element); \
- _tmp->next = (list); \
- (list) = _tmp; \
-} while (0)
-
typedef struct XDbgBlockGraphConstructor {
XDbgBlockGraph *graph;
GHashTable *graph_nodes;
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
2020-10-23 18:36 ` [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-24 9:17 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:41 ` Markus Armbruster
2020-10-23 18:36 ` [PATCH v5 04/12] nbd: Add new qemu:allocation-depth metadata context Eric Blake
` (9 subsequent siblings)
12 siblings, 2 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, qemu-block, rjones, Max Reitz, stefanha
Rather than open-coding the translation from the deprecated
NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
better to utilize QAPI_CLONE_MEMBERS. This solves a couple of issues:
first, if we do any more refactoring of the base type (which an
upcoming patch plans to do), we don't have to revisit the open-coding.
Second, our assignment to arg->name is fishy: the generated QAPI code
currently does not visit it if arg->has_name is false, but if it DID
visit it, we would have introduced a double-free situation when arg is
finally freed.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
blockdev-nbd.c | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 8174023e5c47..cee9134b12eb 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -14,6 +14,8 @@
#include "sysemu/block-backend.h"
#include "hw/block/block.h"
#include "qapi/error.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/qapi-visit-block-export.h"
#include "qapi/qapi-commands-block-export.h"
#include "block/nbd.h"
#include "io/channel-socket.h"
@@ -195,7 +197,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
* the device name as a default here for compatibility.
*/
if (!arg->has_name) {
- arg->name = arg->device;
+ arg->has_name = true;
+ arg->name = g_strdup(arg->device);
}
export_opts = g_new(BlockExportOptions, 1);
@@ -205,15 +208,9 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
.node_name = g_strdup(bdrv_get_node_name(bs)),
.has_writable = arg->has_writable,
.writable = arg->writable,
- .u.nbd = {
- .has_name = true,
- .name = g_strdup(arg->name),
- .has_description = arg->has_description,
- .description = g_strdup(arg->description),
- .has_bitmap = arg->has_bitmap,
- .bitmap = g_strdup(arg->bitmap),
- },
};
+ QAPI_CLONE_MEMBERS(BlockExportOptionsNbd, &export_opts->u.nbd,
+ qapi_NbdServerAddOptions_base(arg));
/*
* nbd-server-add doesn't complain when a read-only device should be
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 04/12] nbd: Add new qemu:allocation-depth metadata context
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (2 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 05/12] nbd: Add 'qemu-nbd -A' to expose allocation depth Eric Blake
` (8 subsequent siblings)
12 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, qemu-block, rjones, Max Reitz, stefanha
'qemu-img map' provides a way to determine which extents of an image
come from the top layer vs. inherited from a backing chain. This is
useful information worth exposing over NBD. There is a proposal to
add a QMP command block-dirty-bitmap-populate which can create a dirty
bitmap that reflects allocation information, at which point the
qemu:dirty-bitmap:NAME metadata context can expose that information
via the creation of a temporary bitmap, but we can shorten the effort
by adding a new qemu:allocation-depth metadata context that does the
same thing without an intermediate bitmap (this patch does not
eliminate the need for that proposal, as it will have other uses as
well).
For this patch, I just encoded a tri-state value (DEPTH_CODE_MASK:
unallocated, from this layer, from any of the backing layers); an
obvious extension would be to provide the actual depth in bits 31-4 (a
new DEPTH_RAW_MASK 0xfffffff0) while keeping bits 1-0 as a tri-state
(leaving bits 3-2 unused, for ease of reading depth from a hex
number). But adding this extension would require
bdrv_is_allocated_above to return a depth number.
While documenting things, remember that although the NBD protocol has
NBD_OPT_SET_META_CONTEXT, the rest of its documentation refers to
'metadata context', which is a more apt description of what is
actually being used by NBD_CMD_BLOCK_STATUS: the user is requesting
metadata by passing one or more context names. So I also touched up
some existing wording to prefer the term 'metadata context' where it
makes sense.
Note that this patch does not actually enable any way to request a
server to enable this context; that will come in the next patch.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/interop/nbd.txt | 27 ++++++++++---
include/block/nbd.h | 12 ++++--
nbd/server.c | 92 ++++++++++++++++++++++++++++++++++++++++----
3 files changed, 116 insertions(+), 15 deletions(-)
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index f3b3cacc9621..7e948bd42218 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -17,19 +17,35 @@ namespace "qemu".
== "qemu" namespace ==
-The "qemu" namespace currently contains only one type of context,
-related to exposing the contents of a dirty bitmap alongside the
-associated disk contents. That context has the following form:
+The "qemu" namespace currently contains two available metadata context
+types. The first is related to exposing the contents of a dirty
+bitmap alongside the associated disk contents. That metadata context
+is named with the following form:
qemu:dirty-bitmap:<dirty-bitmap-export-name>
Each dirty-bitmap metadata context defines only one flag for extents
in reply for NBD_CMD_BLOCK_STATUS:
- bit 0: NBD_STATE_DIRTY, means that the extent is "dirty"
+ bit 0: NBD_STATE_DIRTY, set when the extent is "dirty"
+
+The second is related to exposing the source of various extents within
+the image, with a single metadata context named:
+
+ qemu:allocation-depth
+
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+ bits 0-1: 00: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated
+ 01: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the
+ top layer of the image
+ 10: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
+ backing layer
+ 11: invalid, never returned
For NBD_OPT_LIST_META_CONTEXT the following queries are supported
-in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
+in addition to the specific "qemu:allocation-depth" and
+"qemu:dirty-bitmap:<dirty-bitmap-export-name>":
* "qemu:" - returns list of all available metadata contexts in the
namespace.
@@ -55,3 +71,4 @@ the operation of that feature.
NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
* 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports,
NBD_CMD_FLAG_FAST_ZERO
+* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 3dd9a04546ec..956687f5c368 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2016-2019 Red Hat, Inc.
+ * Copyright (C) 2016-2020 Red Hat, Inc.
* Copyright (C) 2005 Anthony Liguori <anthony@codemonkey.ws>
*
* Network Block Device
@@ -47,7 +47,7 @@ typedef struct NBDOptionReply NBDOptionReply;
typedef struct NBDOptionReplyMetaContext {
NBDOptionReply h; /* h.type = NBD_REP_META_CONTEXT, h.length > 4 */
uint32_t context_id;
- /* meta context name follows */
+ /* metadata context name follows */
} QEMU_PACKED NBDOptionReplyMetaContext;
/* Transmission phase structs
@@ -229,7 +229,7 @@ enum {
#define NBD_MAX_BUFFER_SIZE (32 * 1024 * 1024)
/*
- * Maximum size of a protocol string (export name, meta context name,
+ * Maximum size of a protocol string (export name, metadata context name,
* etc.). Use malloc rather than stack allocation for storage of a
* string.
*/
@@ -259,6 +259,12 @@ enum {
/* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */
#define NBD_STATE_DIRTY (1 << 0)
+/* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */
+#define NBD_STATE_DEPTH_CODE_MASK 0x3
+#define NBD_STATE_DEPTH_UNALLOC 0x0
+#define NBD_STATE_DEPTH_LOCAL 0x1
+#define NBD_STATE_DEPTH_BACKING 0x2
+
static inline bool nbd_reply_type_is_error(int type)
{
return type & (1 << 15);
diff --git a/nbd/server.c b/nbd/server.c
index e75c825879aa..ae6f8a8e5429 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -27,7 +27,8 @@
#include "qemu/units.h"
#define NBD_META_ID_BASE_ALLOCATION 0
-#define NBD_META_ID_DIRTY_BITMAP 1
+#define NBD_META_ID_ALLOCATION_DEPTH 1
+#define NBD_META_ID_DIRTY_BITMAP 2
/*
* NBD_MAX_BLOCK_STATUS_EXTENTS: 1 MiB of extents data. An empirical
@@ -94,6 +95,7 @@ struct NBDExport {
BlockBackend *eject_notifier_blk;
Notifier eject_notifier;
+ bool allocation_depth;
BdrvDirtyBitmap *export_bitmap;
char *export_bitmap_context;
};
@@ -108,6 +110,7 @@ typedef struct NBDExportMetaContexts {
bool valid; /* means that negotiation of the option finished without
errors */
bool base_allocation; /* export base:allocation context (block status) */
+ bool allocation_depth; /* export qemu:allocation-depth */
bool bitmap; /* export qemu:dirty-bitmap:<export bitmap name> */
} NBDExportMetaContexts;
@@ -852,7 +855,8 @@ static bool nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta,
/* nbd_meta_qemu_query
*
* Handle queries to 'qemu' namespace. For now, only the qemu:dirty-bitmap:
- * context is available. Return true if @query has been handled.
+ * and qemu:allocation-depth contexts are available. Return true if @query
+ * has been handled.
*/
static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
const char *query)
@@ -864,12 +868,19 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
if (!*query) {
if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+ meta->allocation_depth = meta->exp->allocation_depth;
meta->bitmap = !!meta->exp->export_bitmap;
}
trace_nbd_negotiate_meta_query_parse("empty");
return true;
}
+ if (strcmp(query, "allocation-depth") == 0) {
+ trace_nbd_negotiate_meta_query_parse("allocation-depth");
+ meta->allocation_depth = meta->exp->allocation_depth;
+ return true;
+ }
+
if (nbd_strshift(&query, "dirty-bitmap:")) {
trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
if (!meta->exp->export_bitmap) {
@@ -884,7 +895,7 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
return true;
}
- trace_nbd_negotiate_meta_query_skip("not dirty-bitmap");
+ trace_nbd_negotiate_meta_query_skip("unknown qemu context");
return true;
}
@@ -984,6 +995,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) {
/* enable all known contexts */
meta->base_allocation = true;
+ meta->allocation_depth = meta->exp->allocation_depth;
meta->bitmap = !!meta->exp->export_bitmap;
} else {
for (i = 0; i < nb_queries; ++i) {
@@ -1003,6 +1015,15 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
}
}
+ if (meta->allocation_depth) {
+ ret = nbd_negotiate_send_meta_context(client, "qemu:allocation-depth",
+ NBD_META_ID_ALLOCATION_DEPTH,
+ errp);
+ if (ret < 0) {
+ return ret;
+ }
+ }
+
if (meta->bitmap) {
ret = nbd_negotiate_send_meta_context(client,
meta->exp->export_bitmap_context,
@@ -1961,6 +1982,40 @@ static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset,
return 0;
}
+static int blockalloc_to_extents(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, NBDExtentArray *ea)
+{
+ while (bytes) {
+ uint32_t flags;
+ int64_t num;
+ int ret = bdrv_is_allocated(bs, offset, bytes, &num);
+
+ if (ret < 0) {
+ return ret;
+ }
+
+ if (ret == 1) {
+ flags = NBD_STATE_DEPTH_LOCAL;
+ } else {
+ ret = bdrv_is_allocated_above(bs, NULL, false, offset, num,
+ &num);
+ if (ret < 0) {
+ return ret;
+ }
+ flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC;
+ }
+
+ if (nbd_extent_array_add(ea, num, flags) < 0) {
+ return 0;
+ }
+
+ offset += num;
+ bytes -= num;
+ }
+
+ return 0;
+}
+
/*
* nbd_co_send_extents
*
@@ -2000,7 +2055,11 @@ static int nbd_co_send_block_status(NBDClient *client, uint64_t handle,
unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BLOCK_STATUS_EXTENTS;
g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);
- ret = blockstatus_to_extents(bs, offset, length, ea);
+ if (context_id == NBD_META_ID_BASE_ALLOCATION) {
+ ret = blockstatus_to_extents(bs, offset, length, ea);
+ } else {
+ ret = blockalloc_to_extents(bs, offset, length, ea);
+ }
if (ret < 0) {
return nbd_co_send_structured_error(
client, handle, -ret, "can't get block status", errp);
@@ -2335,16 +2394,20 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
}
if (client->export_meta.valid &&
(client->export_meta.base_allocation ||
+ client->export_meta.allocation_depth ||
client->export_meta.bitmap))
{
bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE;
+ int contexts_remaining = client->export_meta.base_allocation +
+ client->export_meta.allocation_depth +
+ client->export_meta.bitmap;
if (client->export_meta.base_allocation) {
ret = nbd_co_send_block_status(client, request->handle,
blk_bs(exp->common.blk),
request->from,
request->len, dont_fragment,
- !client->export_meta.bitmap,
+ !--contexts_remaining,
NBD_META_ID_BASE_ALLOCATION,
errp);
if (ret < 0) {
@@ -2352,17 +2415,32 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
}
}
+ if (client->export_meta.allocation_depth) {
+ ret = nbd_co_send_block_status(client, request->handle,
+ blk_bs(exp->common.blk),
+ request->from, request->len,
+ dont_fragment,
+ !--contexts_remaining,
+ NBD_META_ID_ALLOCATION_DEPTH,
+ errp);
+ if (ret < 0) {
+ return ret;
+ }
+ }
+
if (client->export_meta.bitmap) {
ret = nbd_co_send_bitmap(client, request->handle,
client->exp->export_bitmap,
request->from, request->len,
- dont_fragment,
- true, NBD_META_ID_DIRTY_BITMAP, errp);
+ dont_fragment, !--contexts_remaining,
+ NBD_META_ID_DIRTY_BITMAP, errp);
if (ret < 0) {
return ret;
}
}
+ assert(!contexts_remaining);
+
return 0;
} else {
return nbd_send_generic_reply(client, request->handle, -EINVAL,
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 05/12] nbd: Add 'qemu-nbd -A' to expose allocation depth
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (3 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 04/12] nbd: Add new qemu:allocation-depth metadata context Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps Eric Blake
` (7 subsequent siblings)
12 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, vsementsov, qemu-block, Markus Armbruster, rjones,
stefanha, Max Reitz
Allow the server to expose an additional metacontext to be requested
by savvy clients. qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using block-export-add.
qemu as client can be hacked into viewing this new context by using
the now-misnamed x-dirty-bitmap option when creating an NBD blockdev
(even though our x- naming means we could rename it, I did not think
it worth breaking back-compat of tools that have been using it while
waiting for a better solution). It is worth noting the decoding of
how such context information will appear in 'qemu-img map
--output=json':
NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true
libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/tools/qemu-nbd.rst | 8 ++++-
qapi/block-core.json | 7 ++--
qapi/block-export.json | 12 +++++--
nbd/server.c | 2 ++
qemu-nbd.c | 26 +++++++++-----
tests/qemu-iotests/309 | 73 ++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/309.out | 22 ++++++++++++
tests/qemu-iotests/group | 1 +
8 files changed, 136 insertions(+), 15 deletions(-)
create mode 100755 tests/qemu-iotests/309
create mode 100644 tests/qemu-iotests/309.out
diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst
index 667861cb22e9..fe41336dc550 100644
--- a/docs/tools/qemu-nbd.rst
+++ b/docs/tools/qemu-nbd.rst
@@ -72,10 +72,16 @@ driver options if ``--image-opts`` is specified.
Export the disk as read-only.
+.. option:: -A, --allocation-depth
+
+ Expose allocation depth information via the
+ ``qemu:allocation-depth`` metadata context accessible through
+ NBD_OPT_SET_META_CONTEXT.
+
.. option:: -B, --bitmap=NAME
If *filename* has a qcow2 persistent bitmap *NAME*, expose
- that bitmap via the ``qemu:dirty-bitmap:NAME`` context
+ that bitmap via the ``qemu:dirty-bitmap:NAME`` metadata context
accessible through NBD_OPT_SET_META_CONTEXT.
.. option:: -s, --snapshot
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ee5ebef7f2ce..b6368f5fd9a1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3883,9 +3883,12 @@
#
# @tls-creds: TLS credentials ID
#
-# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of
+# @x-dirty-bitmap: A metadata context name such as "qemu:dirty-bitmap:NAME"
+# or "qemu:allocation-depth" to query in place of the
# traditional "base:allocation" block status (see
-# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0)
+# NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and
+# yes, naming this option x-context would have made
+# more sense) (since 3.0)
#
# @reconnect-delay: On an unexpected disconnect, the nbd client tries to
# connect again until succeeding or encountering a serious
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 65804834d905..893d5cde5dfe 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -75,14 +75,20 @@
# (Since 5.0)
#
# @bitmap: Also export the dirty bitmap reachable from @device, so the
-# NBD client can use NBD_OPT_SET_META_CONTEXT with
-# "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0)
+# NBD client can use NBD_OPT_SET_META_CONTEXT with the
+# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
+# bitmap. (since 4.0)
+#
+# @allocation-depth: Also export the allocation depth map for @device, so
+# the NBD client can use NBD_OPT_SET_META_CONTEXT with
+# the metadata context name "qemu:allocation-depth" to
+# inspect allocation details. (since 5.2)
#
# Since: 5.0
##
{ 'struct': 'BlockExportOptionsNbd',
'data': { '*name': 'str', '*description': 'str',
- '*bitmap': 'str' } }
+ '*bitmap': 'str', '*allocation-depth': 'bool' } }
##
# @NbdServerAddOptions:
diff --git a/nbd/server.c b/nbd/server.c
index ae6f8a8e5429..30cfe0eee467 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1597,6 +1597,8 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
}
+ exp->allocation_depth = arg->allocation_depth;
+
blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
QTAILQ_INSERT_TAIL(&exports, exp, next);
diff --git a/qemu-nbd.c b/qemu-nbd.c
index bc644a0670b6..847fde435a7f 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -99,6 +99,7 @@ static void usage(const char *name)
"\n"
"Exposing part of the image:\n"
" -o, --offset=OFFSET offset into the image\n"
+" -A, --allocation-depth expose the allocation depth\n"
" -B, --bitmap=NAME expose a persistent dirty bitmap\n"
"\n"
"General purpose options:\n"
@@ -519,7 +520,7 @@ int main(int argc, char **argv)
char *device = NULL;
QemuOpts *sn_opts = NULL;
const char *sn_id_or_name = NULL;
- const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L";
+ const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:AB:L";
struct option lopt[] = {
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
@@ -528,6 +529,7 @@ int main(int argc, char **argv)
{ "socket", required_argument, NULL, 'k' },
{ "offset", required_argument, NULL, 'o' },
{ "read-only", no_argument, NULL, 'r' },
+ { "allocation-depth", no_argument, NULL, 'A' },
{ "bitmap", required_argument, NULL, 'B' },
{ "connect", required_argument, NULL, 'c' },
{ "disconnect", no_argument, NULL, 'd' },
@@ -569,6 +571,7 @@ int main(int argc, char **argv)
QDict *options = NULL;
const char *export_name = NULL; /* defaults to "" later for server mode */
const char *export_description = NULL;
+ bool alloc_depth = false;
const char *bitmap = NULL;
const char *tlscredsid = NULL;
bool imageOpts = false;
@@ -694,6 +697,9 @@ int main(int argc, char **argv)
readonly = true;
flags &= ~BDRV_O_RDWR;
break;
+ case 'A':
+ alloc_depth = true;
+ break;
case 'B':
bitmap = optarg;
break;
@@ -791,8 +797,8 @@ int main(int argc, char **argv)
exit(EXIT_FAILURE);
}
if (export_name || export_description || dev_offset ||
- device || disconnect || fmt || sn_id_or_name || bitmap ||
- seen_aio || seen_discard || seen_cache) {
+ device || disconnect || fmt || sn_id_or_name || alloc_depth ||
+ bitmap || seen_aio || seen_discard || seen_cache) {
error_report("List mode is incompatible with per-device settings");
exit(EXIT_FAILURE);
}
@@ -1072,12 +1078,14 @@ int main(int argc, char **argv)
.has_writable = true,
.writable = !readonly,
.u.nbd = {
- .has_name = true,
- .name = g_strdup(export_name),
- .has_description = !!export_description,
- .description = g_strdup(export_description),
- .has_bitmap = !!bitmap,
- .bitmap = g_strdup(bitmap),
+ .has_name = true,
+ .name = g_strdup(export_name),
+ .has_description = !!export_description,
+ .description = g_strdup(export_description),
+ .has_bitmap = !!bitmap,
+ .bitmap = g_strdup(bitmap),
+ .has_allocation_depth = alloc_depth,
+ .allocation_depth = alloc_depth,
},
};
blk_exp_add(export_opts, &error_fatal);
diff --git a/tests/qemu-iotests/309 b/tests/qemu-iotests/309
new file mode 100755
index 000000000000..b6734794bb68
--- /dev/null
+++ b/tests/qemu-iotests/309
@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+#
+# Test qemu-nbd -A
+#
+# Copyright (C) 2018-2020 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+ nbd_server_stop
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.nbd
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+_require_command QEMU_NBD
+
+echo
+echo "=== Initial image setup ==="
+echo
+
+TEST_IMG="$TEST_IMG.base" _make_test_img 4M
+$QEMU_IO -c 'w 0 2M' -f $IMGFMT "$TEST_IMG.base" | _filter_qemu_io
+_make_test_img -b "$TEST_IMG.base" -F $IMGFMT 4M
+$QEMU_IO -c 'w 1M 2M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io
+
+echo
+echo "=== Check allocation over NBD ==="
+echo
+
+$QEMU_IMG map --output=json -f qcow2 "$TEST_IMG"
+IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket"
+nbd_server_start_unix_socket -r -f qcow2 -A "$TEST_IMG"
+# Normal -f raw NBD block status loses access to allocation information
+$QEMU_IMG map --output=json --image-opts \
+ "$IMG" | _filter_qemu_img_map
+# But since we used -A, and use x-dirty-bitmap as a hack for reading bitmaps,
+# we can reconstruct it, by abusing block status to report:
+# NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
+# NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
+# NBD_STATE_DEPTH_BACKING => "zero":true, "data":true
+$QEMU_IMG map --output=json --image-opts \
+ "$IMG,x-dirty-bitmap=qemu:allocation-depth" | _filter_qemu_img_map
+
+# success, all done
+echo '*** done'
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/309.out b/tests/qemu-iotests/309.out
new file mode 100644
index 000000000000..db75bb6b0df9
--- /dev/null
+++ b/tests/qemu-iotests/309.out
@@ -0,0 +1,22 @@
+QA output created by 309
+
+=== Initial image setup ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=4194304
+wrote 2097152/2097152 bytes at offset 0
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+wrote 2097152/2097152 bytes at offset 1048576
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+=== Check allocation over NBD ===
+
+[{ "start": 0, "length": 1048576, "depth": 1, "zero": false, "data": true, "offset": 327680},
+{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": 327680},
+{ "start": 3145728, "length": 1048576, "depth": 1, "zero": true, "data": false}]
+[{ "start": 0, "length": 3145728, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 3145728, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": true, "offset": OFFSET},
+{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": false},
+{ "start": 3145728, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 343298928350..2960dff72864 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -315,3 +315,4 @@
304 rw quick
305 rw quick
307 rw quick export
+309 rw auto quick
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (4 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 05/12] nbd: Add 'qemu-nbd -A' to expose allocation depth Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-26 10:50 ` Peter Krempa
2020-10-23 18:36 ` [PATCH v5 07/12] nbd: Simplify qemu bitmap context name Eric Blake
` (6 subsequent siblings)
12 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, vsementsov, qemu-block, reviewer:Incompatible changes,
Markus Armbruster, rjones, Max Reitz, stefanha
Since 'nbd-server-add' is deprecated, and 'block-export-add' is new to
5.2, we can still tweak the interface. Allowing 'bitmaps':['str'] is
nicer than 'bitmap':'str'. This wires up the qapi and qemu-nbd
changes to permit passing multiple bitmaps as distinct metadata
contexts that the NBD client may request, but the actual support for
more than one will require a further patch to the server.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/system/deprecated.rst | 4 +++-
qapi/block-export.json | 18 ++++++++++++------
blockdev-nbd.c | 13 +++++++++++++
nbd/server.c | 19 +++++++++++++------
qemu-nbd.c | 10 +++++-----
5 files changed, 46 insertions(+), 18 deletions(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 905628f3a0cb..d6cd027ac740 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -268,7 +268,9 @@ the 'wait' field, which is only applicable to sockets in server mode
''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Use the more generic commands ``block-export-add`` and ``block-export-del``
-instead.
+instead. As part of this deprecation, it is now preferred to export a
+list of dirty bitmaps via ``bitmaps``, rather than a single bitmap via
+``bitmap``.
Human Monitor Protocol (HMP) commands
-------------------------------------
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 893d5cde5dfe..c7c749d61097 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -74,10 +74,10 @@
# @description: Free-form description of the export, up to 4096 bytes.
# (Since 5.0)
#
-# @bitmap: Also export the dirty bitmap reachable from @device, so the
-# NBD client can use NBD_OPT_SET_META_CONTEXT with the
-# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
-# bitmap. (since 4.0)
+# @bitmaps: Also export each of the named dirty bitmaps reachable from
+# @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT with
+# the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect
+# each bitmap. (since 5.2)
#
# @allocation-depth: Also export the allocation depth map for @device, so
# the NBD client can use NBD_OPT_SET_META_CONTEXT with
@@ -88,7 +88,7 @@
##
{ 'struct': 'BlockExportOptionsNbd',
'data': { '*name': 'str', '*description': 'str',
- '*bitmap': 'str', '*allocation-depth': 'bool' } }
+ '*bitmaps': ['str'], '*allocation-depth': 'bool' } }
##
# @NbdServerAddOptions:
@@ -100,12 +100,18 @@
# @writable: Whether clients should be able to write to the device via the
# NBD connection (default false).
#
+# @bitmap: Also export a single dirty bitmap reachable from @device, so the
+# NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata
+# context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap
+# (since 4.0). Mutually exclusive with @bitmaps, and newer
+# clients should use that instead.
+#
# Since: 5.0
##
{ 'struct': 'NbdServerAddOptions',
'base': 'BlockExportOptionsNbd',
'data': { 'device': 'str',
- '*writable': 'bool' } }
+ '*writable': 'bool', '*bitmap': 'str' } }
##
# @nbd-server-add:
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index cee9134b12eb..cfd46223bf4d 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -192,6 +192,19 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
return;
}
+ /*
+ * New code should use the list 'bitmaps'; but until this code is
+ * gone, we must support the older single 'bitmap'. Use only one.
+ */
+ if (arg->has_bitmap) {
+ if (arg->has_bitmaps) {
+ error_setg(errp, "Can't mix 'bitmap' and 'bitmaps'");
+ return;
+ }
+ arg->has_bitmaps = true;
+ QAPI_LIST_ADD(arg->bitmaps, g_strdup(arg->bitmap));
+ }
+
/*
* block-export-add would default to the node-name, but we may have to use
* the device name as a default here for compatibility.
diff --git a/nbd/server.c b/nbd/server.c
index 30cfe0eee467..884ffa00f1bd 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1495,6 +1495,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
uint64_t perm, shared_perm;
bool readonly = !exp_args->writable;
bool shared = !exp_args->writable;
+ strList *bitmaps;
int ret;
assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
@@ -1556,12 +1557,18 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
}
exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
- if (arg->bitmap) {
+ /* XXX Allow more than one bitmap */
+ if (arg->bitmaps && arg->bitmaps->next) {
+ error_setg(errp, "multiple bitmaps per export not supported yet");
+ return -EOPNOTSUPP;
+ }
+ for (bitmaps = arg->bitmaps; bitmaps; bitmaps = bitmaps->next) {
+ const char *bitmap = bitmaps->value;
BlockDriverState *bs = blk_bs(blk);
BdrvDirtyBitmap *bm = NULL;
while (bs) {
- bm = bdrv_find_dirty_bitmap(bs, arg->bitmap);
+ bm = bdrv_find_dirty_bitmap(bs, bitmap);
if (bm != NULL) {
break;
}
@@ -1571,7 +1578,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
if (bm == NULL) {
ret = -ENOENT;
- error_setg(errp, "Bitmap '%s' is not found", arg->bitmap);
+ error_setg(errp, "Bitmap '%s' is not found", bitmap);
goto fail;
}
@@ -1585,15 +1592,15 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
ret = -EINVAL;
error_setg(errp,
"Enabled bitmap '%s' incompatible with readonly export",
- arg->bitmap);
+ bitmap);
goto fail;
}
bdrv_dirty_bitmap_set_busy(bm, true);
exp->export_bitmap = bm;
- assert(strlen(arg->bitmap) <= BDRV_BITMAP_MAX_NAME_SIZE);
+ assert(strlen(bitmap) <= BDRV_BITMAP_MAX_NAME_SIZE);
exp->export_bitmap_context = g_strdup_printf("qemu:dirty-bitmap:%s",
- arg->bitmap);
+ bitmap);
assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
}
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 847fde435a7f..5473821216f7 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -572,7 +572,7 @@ int main(int argc, char **argv)
const char *export_name = NULL; /* defaults to "" later for server mode */
const char *export_description = NULL;
bool alloc_depth = false;
- const char *bitmap = NULL;
+ strList *bitmaps = NULL;
const char *tlscredsid = NULL;
bool imageOpts = false;
bool writethrough = true;
@@ -701,7 +701,7 @@ int main(int argc, char **argv)
alloc_depth = true;
break;
case 'B':
- bitmap = optarg;
+ QAPI_LIST_ADD(bitmaps, g_strdup(optarg));
break;
case 'k':
sockpath = optarg;
@@ -798,7 +798,7 @@ int main(int argc, char **argv)
}
if (export_name || export_description || dev_offset ||
device || disconnect || fmt || sn_id_or_name || alloc_depth ||
- bitmap || seen_aio || seen_discard || seen_cache) {
+ bitmaps || seen_aio || seen_discard || seen_cache) {
error_report("List mode is incompatible with per-device settings");
exit(EXIT_FAILURE);
}
@@ -1082,8 +1082,8 @@ int main(int argc, char **argv)
.name = g_strdup(export_name),
.has_description = !!export_description,
.description = g_strdup(export_description),
- .has_bitmap = !!bitmap,
- .bitmap = g_strdup(bitmap),
+ .has_bitmaps = !!bitmaps,
+ .bitmaps = bitmaps,
.has_allocation_depth = alloc_depth,
.allocation_depth = alloc_depth,
},
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 07/12] nbd: Simplify qemu bitmap context name
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (5 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 08/12] nbd: Refactor counting of metadata contexts Eric Blake
` (5 subsequent siblings)
12 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, stefanha, qemu-block, rjones
Each dirty bitmap already knows its name; by reducing the scope of the
places where we construct "qemu:dirty-bitmap:NAME" strings, tracking
the name is more localized, and there are fewer per-export fields to
worry about. This in turn will make it easier for an upcoming patch
to export more than one bitmap at once.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
nbd/server.c | 19 +++++++++----------
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 884ffa00f1bd..05a8154975a1 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -97,7 +97,6 @@ struct NBDExport {
bool allocation_depth;
BdrvDirtyBitmap *export_bitmap;
- char *export_bitmap_context;
};
static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -882,14 +881,15 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
}
if (nbd_strshift(&query, "dirty-bitmap:")) {
+ const char *bm_name;
+
trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
if (!meta->exp->export_bitmap) {
trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
return true;
}
- if (nbd_meta_empty_or_pattern(client,
- meta->exp->export_bitmap_context +
- strlen("qemu:dirty-bitmap:"), query)) {
+ bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
+ if (nbd_meta_empty_or_pattern(client, bm_name, query)) {
meta->bitmap = true;
}
return true;
@@ -1025,8 +1025,11 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
}
if (meta->bitmap) {
- ret = nbd_negotiate_send_meta_context(client,
- meta->exp->export_bitmap_context,
+ const char *bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
+ g_autofree char *context = g_strdup_printf("qemu:dirty-bitmap:%s",
+ bm_name);
+
+ ret = nbd_negotiate_send_meta_context(client, context,
NBD_META_ID_DIRTY_BITMAP,
errp);
if (ret < 0) {
@@ -1599,9 +1602,6 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
bdrv_dirty_bitmap_set_busy(bm, true);
exp->export_bitmap = bm;
assert(strlen(bitmap) <= BDRV_BITMAP_MAX_NAME_SIZE);
- exp->export_bitmap_context = g_strdup_printf("qemu:dirty-bitmap:%s",
- bitmap);
- assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
}
exp->allocation_depth = arg->allocation_depth;
@@ -1681,7 +1681,6 @@ static void nbd_export_delete(BlockExport *blk_exp)
if (exp->export_bitmap) {
bdrv_dirty_bitmap_set_busy(exp->export_bitmap, false);
- g_free(exp->export_bitmap_context);
}
}
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 08/12] nbd: Refactor counting of metadata contexts
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (6 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 07/12] nbd: Simplify qemu bitmap context name Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 09/12] nbd: Allow export of multiple bitmaps for one device Eric Blake
` (4 subsequent siblings)
12 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, stefanha, qemu-block, rjones
Rather than open-code the count of negotiated contexts at several
sites, embed it directly into the struct.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
nbd/server.c | 23 +++++++++++------------
1 file changed, 11 insertions(+), 12 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 05a8154975a1..27d943529409 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -106,8 +106,7 @@ static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
* NBD_OPT_LIST_META_CONTEXT. */
typedef struct NBDExportMetaContexts {
NBDExport *exp;
- bool valid; /* means that negotiation of the option finished without
- errors */
+ size_t count; /* number of negotiated contexts */
bool base_allocation; /* export base:allocation context (block status) */
bool allocation_depth; /* export qemu:allocation-depth */
bool bitmap; /* export qemu:dirty-bitmap:<export bitmap name> */
@@ -448,7 +447,9 @@ static int nbd_negotiate_handle_list(NBDClient *client, Error **errp)
static void nbd_check_meta_export(NBDClient *client)
{
- client->export_meta.valid &= client->exp == client->export_meta.exp;
+ if (client->exp != client->export_meta.exp) {
+ client->export_meta.count = 0;
+ }
}
/* Send a reply to NBD_OPT_EXPORT_NAME.
@@ -956,6 +957,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
NBDExportMetaContexts local_meta;
uint32_t nb_queries;
int i;
+ size_t count = 0;
if (!client->structured_reply) {
return nbd_opt_invalid(client, errp,
@@ -1013,6 +1015,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+ count++;
}
if (meta->allocation_depth) {
@@ -1022,6 +1025,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+ count++;
}
if (meta->bitmap) {
@@ -1035,11 +1039,12 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+ count++;
}
ret = nbd_negotiate_send_rep(client, NBD_REP_ACK, errp);
if (ret == 0) {
- meta->valid = true;
+ meta->count = count;
}
return ret;
@@ -2400,15 +2405,9 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
return nbd_send_generic_reply(client, request->handle, -EINVAL,
"need non-zero length", errp);
}
- if (client->export_meta.valid &&
- (client->export_meta.base_allocation ||
- client->export_meta.allocation_depth ||
- client->export_meta.bitmap))
- {
+ if (client->export_meta.count) {
bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE;
- int contexts_remaining = client->export_meta.base_allocation +
- client->export_meta.allocation_depth +
- client->export_meta.bitmap;
+ int contexts_remaining = client->export_meta.count;
if (client->export_meta.base_allocation) {
ret = nbd_co_send_block_status(client, request->handle,
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 09/12] nbd: Allow export of multiple bitmaps for one device
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (7 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 08/12] nbd: Refactor counting of metadata contexts Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above Eric Blake
` (3 subsequent siblings)
12 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, qemu-block, rjones, Max Reitz, stefanha
With this, 'qemu-nbd -B b0 -B b1 -f qcow2 img.qcow2' can let you sniff
out multiple bitmaps from one server. qemu-img as client can still
only read one bitmap per client connection, but other NBD clients
(hello libnbd) can now read multiple bitmaps in a single pass.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
nbd/server.c | 100 ++++++++++++++++++++++++++++-------------
tests/qemu-iotests/291 | 6 +--
2 files changed, 72 insertions(+), 34 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 27d943529409..53526090b0a2 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -28,6 +28,7 @@
#define NBD_META_ID_BASE_ALLOCATION 0
#define NBD_META_ID_ALLOCATION_DEPTH 1
+/* Dirty bitmaps use 'NBD_META_ID_DIRTY_BITMAP + i', so keep this id last. */
#define NBD_META_ID_DIRTY_BITMAP 2
/*
@@ -96,7 +97,8 @@ struct NBDExport {
Notifier eject_notifier;
bool allocation_depth;
- BdrvDirtyBitmap *export_bitmap;
+ BdrvDirtyBitmap **export_bitmaps;
+ size_t nr_export_bitmaps;
};
static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -109,7 +111,10 @@ typedef struct NBDExportMetaContexts {
size_t count; /* number of negotiated contexts */
bool base_allocation; /* export base:allocation context (block status) */
bool allocation_depth; /* export qemu:allocation-depth */
- bool bitmap; /* export qemu:dirty-bitmap:<export bitmap name> */
+ bool *bitmaps; /*
+ * export qemu:dirty-bitmap:<export bitmap name>,
+ * sized by exp->nr_export_bitmaps
+ */
} NBDExportMetaContexts;
struct NBDClient {
@@ -861,6 +866,8 @@ static bool nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta,
static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
const char *query)
{
+ size_t i;
+
if (!nbd_strshift(&query, "qemu:")) {
return false;
}
@@ -869,7 +876,7 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
if (!*query) {
if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
meta->allocation_depth = meta->exp->allocation_depth;
- meta->bitmap = !!meta->exp->export_bitmap;
+ memset(meta->bitmaps, 1, meta->exp->nr_export_bitmaps);
}
trace_nbd_negotiate_meta_query_parse("empty");
return true;
@@ -882,17 +889,26 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
}
if (nbd_strshift(&query, "dirty-bitmap:")) {
- const char *bm_name;
-
trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
- if (!meta->exp->export_bitmap) {
- trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
+ if (!*query) {
+ if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+ memset(meta->bitmaps, 1, meta->exp->nr_export_bitmaps);
+ }
+ trace_nbd_negotiate_meta_query_parse("empty");
return true;
}
- bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
- if (nbd_meta_empty_or_pattern(client, bm_name, query)) {
- meta->bitmap = true;
+
+ for (i = 0; i < meta->exp->nr_export_bitmaps; i++) {
+ const char *bm_name;
+
+ bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmaps[i]);
+ if (strcmp(bm_name, query) == 0) {
+ meta->bitmaps[i] = true;
+ trace_nbd_negotiate_meta_query_parse(query);
+ return true;
+ }
}
+ trace_nbd_negotiate_meta_query_skip("no dirty-bitmap match");
return true;
}
@@ -954,9 +970,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
{
int ret;
g_autofree char *export_name = NULL;
- NBDExportMetaContexts local_meta;
+ g_autofree bool *bitmaps = NULL;
+ NBDExportMetaContexts local_meta = {0};
uint32_t nb_queries;
- int i;
+ size_t i;
size_t count = 0;
if (!client->structured_reply) {
@@ -971,6 +988,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
meta = &local_meta;
}
+ g_free(meta->bitmaps);
memset(meta, 0, sizeof(*meta));
ret = nbd_opt_read_name(client, &export_name, NULL, errp);
@@ -985,6 +1003,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
return nbd_opt_drop(client, NBD_REP_ERR_UNKNOWN, errp,
"export '%s' not present", sane_name);
}
+ meta->bitmaps = g_new0(bool, meta->exp->nr_export_bitmaps);
+ if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+ bitmaps = meta->bitmaps;
+ }
ret = nbd_opt_read(client, &nb_queries, sizeof(nb_queries), false, errp);
if (ret <= 0) {
@@ -998,7 +1020,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
/* enable all known contexts */
meta->base_allocation = true;
meta->allocation_depth = meta->exp->allocation_depth;
- meta->bitmap = !!meta->exp->export_bitmap;
+ memset(meta->bitmaps, 1, meta->exp->nr_export_bitmaps);
} else {
for (i = 0; i < nb_queries; ++i) {
ret = nbd_negotiate_meta_query(client, meta, errp);
@@ -1028,13 +1050,19 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
count++;
}
- if (meta->bitmap) {
- const char *bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
- g_autofree char *context = g_strdup_printf("qemu:dirty-bitmap:%s",
- bm_name);
+ for (i = 0; i < meta->exp->nr_export_bitmaps; i++) {
+ const char *bm_name;
+ g_autofree char *context = NULL;
+
+ if (!meta->bitmaps[i]) {
+ continue;
+ }
+
+ bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmaps[i]);
+ context = g_strdup_printf("qemu:dirty-bitmap:%s", bm_name);
ret = nbd_negotiate_send_meta_context(client, context,
- NBD_META_ID_DIRTY_BITMAP,
+ NBD_META_ID_DIRTY_BITMAP + i,
errp);
if (ret < 0) {
return ret;
@@ -1388,6 +1416,7 @@ void nbd_client_put(NBDClient *client)
QTAILQ_REMOVE(&client->exp->clients, client, next);
blk_exp_unref(&client->exp->common);
}
+ g_free(client->export_meta.bitmaps);
g_free(client);
}
}
@@ -1504,6 +1533,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
bool readonly = !exp_args->writable;
bool shared = !exp_args->writable;
strList *bitmaps;
+ size_t i;
int ret;
assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
@@ -1565,12 +1595,12 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
}
exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
- /* XXX Allow more than one bitmap */
- if (arg->bitmaps && arg->bitmaps->next) {
- error_setg(errp, "multiple bitmaps per export not supported yet");
- return -EOPNOTSUPP;
- }
for (bitmaps = arg->bitmaps; bitmaps; bitmaps = bitmaps->next) {
+ exp->nr_export_bitmaps++;
+ }
+ exp->export_bitmaps = g_new0(BdrvDirtyBitmap *, exp->nr_export_bitmaps);
+ for (i = 0, bitmaps = arg->bitmaps; bitmaps;
+ i++, bitmaps = bitmaps->next) {
const char *bitmap = bitmaps->value;
BlockDriverState *bs = blk_bs(blk);
BdrvDirtyBitmap *bm = NULL;
@@ -1604,11 +1634,15 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
goto fail;
}
- bdrv_dirty_bitmap_set_busy(bm, true);
- exp->export_bitmap = bm;
+ exp->export_bitmaps[i] = bm;
assert(strlen(bitmap) <= BDRV_BITMAP_MAX_NAME_SIZE);
}
+ /* Mark bitmaps busy in a separate loop, to simplify roll-back concerns. */
+ for (i = 0; i < exp->nr_export_bitmaps; i++) {
+ bdrv_dirty_bitmap_set_busy(exp->export_bitmaps[i], true);
+ }
+
exp->allocation_depth = arg->allocation_depth;
blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
@@ -1618,6 +1652,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
return 0;
fail:
+ g_free(exp->export_bitmaps);
g_free(exp->name);
g_free(exp->description);
return ret;
@@ -1667,6 +1702,7 @@ static void nbd_export_request_shutdown(BlockExport *blk_exp)
static void nbd_export_delete(BlockExport *blk_exp)
{
+ size_t i;
NBDExport *exp = container_of(blk_exp, NBDExport, common);
assert(exp->name == NULL);
@@ -1684,8 +1720,8 @@ static void nbd_export_delete(BlockExport *blk_exp)
blk_aio_detach, exp);
}
- if (exp->export_bitmap) {
- bdrv_dirty_bitmap_set_busy(exp->export_bitmap, false);
+ for (i = 0; i < exp->nr_export_bitmaps; i++) {
+ bdrv_dirty_bitmap_set_busy(exp->export_bitmaps[i], false);
}
}
@@ -2332,6 +2368,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
int flags;
NBDExport *exp = client->exp;
char *msg;
+ size_t i;
switch (request->type) {
case NBD_CMD_CACHE:
@@ -2435,12 +2472,15 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
}
}
- if (client->export_meta.bitmap) {
+ for (i = 0; i < client->exp->nr_export_bitmaps; i++) {
+ if (!client->export_meta.bitmaps[i]) {
+ continue;
+ }
ret = nbd_co_send_bitmap(client, request->handle,
- client->exp->export_bitmap,
+ client->exp->export_bitmaps[i],
request->from, request->len,
dont_fragment, !--contexts_remaining,
- NBD_META_ID_DIRTY_BITMAP, errp);
+ NBD_META_ID_DIRTY_BITMAP + i, errp);
if (ret < 0) {
return ret;
}
diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/291
index 4f837b205655..37848ac60bba 100755
--- a/tests/qemu-iotests/291
+++ b/tests/qemu-iotests/291
@@ -107,16 +107,14 @@ echo
# x-dirty-bitmap is a hack for reading bitmaps; it abuses block status to
# report "data":false for portions of the bitmap which are set
IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket"
-nbd_server_start_unix_socket -r -f qcow2 -B b0 "$TEST_IMG"
+nbd_server_start_unix_socket -r -f qcow2 \
+ -B b0 -B b1 -B b2 -B b3 "$TEST_IMG"
$QEMU_IMG map --output=json --image-opts \
"$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b0" | _filter_qemu_img_map
-nbd_server_start_unix_socket -r -f qcow2 -B b1 "$TEST_IMG"
$QEMU_IMG map --output=json --image-opts \
"$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b1" | _filter_qemu_img_map
-nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG"
$QEMU_IMG map --output=json --image-opts \
"$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map
-nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG"
$QEMU_IMG map --output=json --image-opts \
"$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b3" | _filter_qemu_img_map
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (8 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 09/12] nbd: Allow export of multiple bitmaps for one device Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-24 9:49 ` Vladimir Sementsov-Ogievskiy
2020-10-23 18:36 ` [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth Eric Blake
` (2 subsequent siblings)
12 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, Fam Zheng, vsementsov, qemu-block, rjones, Max Reitz,
stefanha, John Snow
When checking for allocation across a chain, it's already easy to
count the depth within the chain at which the allocation is found.
Instead of throwing that information away, return it to the caller.
Existing callers only cared about allocated/non-allocated, but having
a depth available will be used by NBD in the next patch.
Note that the previous code (since commit 188a7bbf94 in 2012) was
lazy: for each layer deeper in the backing chain, it was issuing a
bdrv_is_allocated request on the original 'bytes' amount, rather than
on any smaller amount 'n' learned from an upper layer. These
semantics happened to work, since if you have:
Base <- Active
XX-- -XX-
the consecutive results are offset 0: '11' with *pnum 2, followed by
offset 2: '1' with *pnum 1, followed by offset 3: '0' with *pnum 1;
the resulting sequence 1110 matches reality (the first three clusters
are indeed allocated somewhere in the given range). But post-patch,
we correctly give the results offset 0: '2' with *pnum 1, followed by
offset 1: '11' with *pnum 2, followed by offset 3: '0' with *pnum 1
(2110), without over-reporting the length of contributions from the
backing layers.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
block/io.c | 11 +++++++----
block/commit.c | 2 +-
block/mirror.c | 2 +-
block/stream.c | 2 +-
4 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/block/io.c b/block/io.c
index 54f0968aee27..4a4fa1c9ab1b 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2414,8 +2414,9 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState *bs, int64_t offset,
/*
* Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP]
*
- * Return 1 if (a prefix of) the given range is allocated in any image
- * between BASE and TOP (BASE is only included if include_base is set).
+ * Return a positive depth if (a prefix of) the given range is allocated
+ * in any image between BASE and TOP (BASE is only included if include_base
+ * is set). Depth 1 is TOP, 2 is the first backing layer, and so forth.
* BASE can be NULL to check if the given offset is allocated in any
* image of the chain. Return 0 otherwise, or negative errno on
* failure.
@@ -2435,6 +2436,7 @@ int bdrv_is_allocated_above(BlockDriverState *top,
{
BlockDriverState *intermediate;
int ret;
+ int depth = 0;
int64_t n = bytes;
assert(base || !include_base);
@@ -2444,14 +2446,15 @@ int bdrv_is_allocated_above(BlockDriverState *top,
int64_t pnum_inter;
int64_t size_inter;
+ depth++;
assert(intermediate);
- ret = bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter);
+ ret = bdrv_is_allocated(intermediate, offset, n, &pnum_inter);
if (ret < 0) {
return ret;
}
if (ret) {
*pnum = pnum_inter;
- return 1;
+ return depth;
}
size_inter = bdrv_getlength(intermediate);
diff --git a/block/commit.c b/block/commit.c
index 1e85c306cc41..71db7ba7472e 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -156,7 +156,7 @@ static int coroutine_fn commit_run(Job *job, Error **errp)
/* Copy if allocated above the base */
ret = bdrv_is_allocated_above(blk_bs(s->top), s->base_overlay, true,
offset, COMMIT_BUFFER_SIZE, &n);
- copy = (ret == 1);
+ copy = (ret > 0);
trace_commit_one_iteration(s, offset, n, ret);
if (copy) {
assert(n < SIZE_MAX);
diff --git a/block/mirror.c b/block/mirror.c
index 26acf4af6fb7..8e1ad6eceb57 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -846,7 +846,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)
}
assert(count);
- if (ret == 1) {
+ if (ret > 0) {
bdrv_set_dirty_bitmap(s->dirty_bitmap, offset, count);
}
offset += count;
diff --git a/block/stream.c b/block/stream.c
index 8ce6729a33da..236384f2f739 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -167,7 +167,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
n = len - offset;
}
- copy = (ret == 1);
+ copy = (ret > 0);
}
trace_stream_one_iteration(s, offset, n, ret);
if (copy) {
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (9 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-24 9:59 ` Vladimir Sementsov-Ogievskiy
2020-10-23 18:36 ` [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible Eric Blake
2020-10-23 18:44 ` [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
12 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, vsementsov, qemu-block, rjones, Max Reitz, stefanha
Preserve the tri-state encoding in the low bits, as that still remains
a valuable way to utilize qemu-img map with x-dirty-bitmap for
accessing quick information without needing a third-party NBD client.
But now that the block layer gives us an actual depth, we can easily
expose it in the remaining bits of our metadata context (leaving two
bits reserved, to make it easier to read depth out of a raw hex
number). This assumes no one runs a backing chain larger than 256M
elements.
iotest 309 remains unchanged (an example of the above-mentioned
x-dirty-bitmap hack); actually testing the new bits requires libnbd or
a similar client, and I didn't want to make iotests depend on libnbd
at this point in time; rather, see the libnbd project for interop
tests that exercise this new feature.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
docs/interop/nbd.txt | 6 +++++-
include/block/nbd.h | 2 ++
nbd/server.c | 27 +++++++++++++++------------
3 files changed, 22 insertions(+), 13 deletions(-)
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index 7e948bd42218..d90723ffe991 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -34,7 +34,8 @@ the image, with a single metadata context named:
qemu:allocation-depth
-In the allocation depth context, bits 0 and 1 form a tri-state value:
+In the allocation depth context, bits 0 and 1 form a tri-state value,
+along with 28 bits giving an actual depth:
bits 0-1: 00: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated
01: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the
@@ -42,6 +43,9 @@ In the allocation depth context, bits 0 and 1 form a tri-state value:
10: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
backing layer
11: invalid, never returned
+ bits 2-3: reserved, always 0
+ bits 4-31: NBD_STATE_DEPTH_RAW, the backing layer depth (0 if
+ UNALLOC, 1 for LOCAL, 2 or more for BACKING)
For NBD_OPT_LIST_META_CONTEXT the following queries are supported
in addition to the specific "qemu:allocation-depth" and
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 956687f5c368..3c0692aec642 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -264,6 +264,8 @@ enum {
#define NBD_STATE_DEPTH_UNALLOC 0x0
#define NBD_STATE_DEPTH_LOCAL 0x1
#define NBD_STATE_DEPTH_BACKING 0x2
+#define NBD_STATE_DEPTH_RAW_MASK 0xfffffff0
+#define NBD_STATE_DEPTH_RAW_SHIFT 4
static inline bool nbd_reply_type_is_error(int type)
{
diff --git a/nbd/server.c b/nbd/server.c
index 53526090b0a2..afa79e63a7a6 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2037,22 +2037,25 @@ static int blockalloc_to_extents(BlockDriverState *bs, uint64_t offset,
while (bytes) {
uint32_t flags;
int64_t num;
- int ret = bdrv_is_allocated(bs, offset, bytes, &num);
+ int depth = bdrv_is_allocated_above(bs, NULL, false, offset, bytes,
+ &num);
- if (ret < 0) {
- return ret;
- }
-
- if (ret == 1) {
+ switch (depth) {
+ case 0:
+ flags = NBD_STATE_DEPTH_UNALLOC;
+ break;
+ case 1:
flags = NBD_STATE_DEPTH_LOCAL;
- } else {
- ret = bdrv_is_allocated_above(bs, NULL, false, offset, num,
- &num);
- if (ret < 0) {
- return ret;
+ break;
+ default:
+ if (depth < 0) {
+ return depth;
}
- flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC;
+ flags = NBD_STATE_DEPTH_BACKING;
+ break;
}
+ assert(depth <= UINT32_MAX >> NBD_STATE_DEPTH_RAW_SHIFT);
+ flags |= depth << NBD_STATE_DEPTH_RAW_SHIFT;
if (nbd_extent_array_add(ea, num, flags) < 0) {
return 0;
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (10 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth Eric Blake
@ 2020-10-23 18:36 ` Eric Blake
2020-10-23 20:23 ` Eric Blake
2020-10-23 18:44 ` [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
12 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:36 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Michael S. Tsirkin, Jason Wang, Thomas Huth,
Jiaxun Yang, Philippe Mathieu-Daudé,
Gerd Hoffmann, Michael Roth, qemu-block, Juan Quintela,
David Hildenbrand, Markus Armbruster, Halil Pasic,
Christian Borntraeger, Marc-André Lureau, David Gibson,
Aleksandar Rikalo, Jiri Pirko, Eduardo Habkost,
Dr. David Alan Gilbert, open list:S390 KVM CPUs,
open list:GLUSTER, stefanha, Richard Henderson, kwolf,
vsementsov, Daniel P. Berrangé,
Cornelia Huck, rjones, Max Reitz, open list:ARM TCG CPUs,
open list:PowerPC TCG CPUs, Paolo Bonzini, Aurelien Jarno
Anywhere we create a list of just one item or by prepending items
(typically because order doesn't matter), we can use the now-public
macro. But places where we must keep the list in order by appending
remain open-coded.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
docs/devel/writing-qmp-commands.txt | 13 +++------
hw/net/rocker/rocker_fp.h | 2 +-
block/gluster.c | 19 +++++--------
chardev/char.c | 21 +++++++--------
hw/core/machine.c | 6 +----
hw/net/rocker/rocker.c | 8 +++---
hw/net/rocker/rocker_fp.c | 14 +++++-----
hw/net/virtio-net.c | 21 +++++----------
migration/migration.c | 7 ++---
migration/postcopy-ram.c | 7 ++---
monitor/hmp-cmds.c | 11 ++++----
qemu-img.c | 5 ++--
qga/commands-posix.c | 13 +++------
qga/commands-win32.c | 17 +++---------
qga/commands.c | 6 +----
qom/qom-qmp-cmds.c | 29 ++++++--------------
target/arm/helper.c | 6 +----
target/arm/monitor.c | 13 ++-------
target/i386/cpu.c | 6 +----
target/mips/helper.c | 6 +----
target/s390x/cpu_models.c | 12 ++-------
tests/test-clone-visitor.c | 7 ++---
tests/test-qobject-output-visitor.c | 42 ++++++++++++++---------------
tests/test-visitor-serialization.c | 5 +---
trace/qmp.c | 22 +++++++--------
ui/vnc.c | 21 +++++----------
util/qemu-config.c | 14 +++-------
target/ppc/translate_init.c.inc | 12 ++-------
28 files changed, 119 insertions(+), 246 deletions(-)
diff --git a/docs/devel/writing-qmp-commands.txt b/docs/devel/writing-qmp-commands.txt
index 46a6c48683f5..3e11eeaa1893 100644
--- a/docs/devel/writing-qmp-commands.txt
+++ b/docs/devel/writing-qmp-commands.txt
@@ -531,15 +531,10 @@ TimerAlarmMethodList *qmp_query_alarm_methods(Error **errp)
bool current = true;
for (p = alarm_timers; p->name; p++) {
- TimerAlarmMethodList *info = g_malloc0(sizeof(*info));
- info->value = g_malloc0(sizeof(*info->value));
- info->value->method_name = g_strdup(p->name);
- info->value->current = current;
-
- current = false;
-
- info->next = method_list;
- method_list = info;
+ TimerAlarmMethod *value = g_new0(TimerAlarmMethod, 1);
+ value->method_name = g_strdup(p->name);
+ value->current = current;
+ QAPI_LIST_ADD(method_list, value);
}
return method_list;
diff --git a/hw/net/rocker/rocker_fp.h b/hw/net/rocker/rocker_fp.h
index dbe1dd329a4b..4cb0bb9ccf81 100644
--- a/hw/net/rocker/rocker_fp.h
+++ b/hw/net/rocker/rocker_fp.h
@@ -28,7 +28,7 @@ int fp_port_eg(FpPort *port, const struct iovec *iov, int iovcnt);
char *fp_port_get_name(FpPort *port);
bool fp_port_get_link_up(FpPort *port);
-void fp_port_get_info(FpPort *port, RockerPortList *info);
+void fp_port_get_info(FpPort *port, RockerPort *info);
void fp_port_get_macaddr(FpPort *port, MACAddr *macaddr);
void fp_port_set_macaddr(FpPort *port, MACAddr *macaddr);
uint8_t fp_port_get_learning(FpPort *port);
diff --git a/block/gluster.c b/block/gluster.c
index 4f1448e2bc88..cf446c23f85d 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -359,8 +359,8 @@ static int qemu_gluster_parse_uri(BlockdevOptionsGluster *gconf,
return -EINVAL;
}
- gconf->server = g_new0(SocketAddressList, 1);
- gconf->server->value = gsconf = g_new0(SocketAddress, 1);
+ gsconf = g_new0(SocketAddress, 1);
+ QAPI_LIST_ADD(gconf->server, gsconf);
/* transport */
if (!uri->scheme || !strcmp(uri->scheme, "gluster")) {
@@ -514,7 +514,7 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster *gconf,
{
QemuOpts *opts;
SocketAddress *gsconf = NULL;
- SocketAddressList *curr = NULL;
+ SocketAddressList **curr;
QDict *backing_options = NULL;
Error *local_err = NULL;
char *str = NULL;
@@ -547,6 +547,7 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster *gconf,
}
gconf->path = g_strdup(ptr);
qemu_opts_del(opts);
+ curr = &gconf->server;
for (i = 0; i < num_servers; i++) {
str = g_strdup_printf(GLUSTER_OPT_SERVER_PATTERN"%d.", i);
@@ -655,15 +656,9 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster *gconf,
qemu_opts_del(opts);
}
- if (gconf->server == NULL) {
- gconf->server = g_new0(SocketAddressList, 1);
- gconf->server->value = gsconf;
- curr = gconf->server;
- } else {
- curr->next = g_new0(SocketAddressList, 1);
- curr->next->value = gsconf;
- curr = curr->next;
- }
+ *curr = g_new0(SocketAddressList, 1);
+ (*curr)->value = gsconf;
+ curr = &(*curr)->next;
gsconf = NULL;
qobject_unref(backing_options);
diff --git a/chardev/char.c b/chardev/char.c
index 78553125d311..8dd7ef4c5935 100644
--- a/chardev/char.c
+++ b/chardev/char.c
@@ -776,15 +776,14 @@ static int qmp_query_chardev_foreach(Object *obj, void *data)
{
Chardev *chr = CHARDEV(obj);
ChardevInfoList **list = data;
- ChardevInfoList *info = g_malloc0(sizeof(*info));
+ ChardevInfo *value;
- info->value = g_malloc0(sizeof(*info->value));
- info->value->label = g_strdup(chr->label);
- info->value->filename = g_strdup(chr->filename);
- info->value->frontend_open = chr->be && chr->be->fe_open;
+ value = g_malloc0(sizeof(*value));
+ value->label = g_strdup(chr->label);
+ value->filename = g_strdup(chr->filename);
+ value->frontend_open = chr->be && chr->be->fe_open;
- info->next = *list;
- *list = info;
+ QAPI_LIST_ADD(*list, value);
return 0;
}
@@ -803,12 +802,10 @@ static void
qmp_prepend_backend(const char *name, void *opaque)
{
ChardevBackendInfoList **list = opaque;
- ChardevBackendInfoList *info = g_malloc0(sizeof(*info));
+ ChardevBackendInfo *value = g_new0(ChardevBackendInfo, 1);
- info->value = g_malloc0(sizeof(*info->value));
- info->value->name = g_strdup(name);
- info->next = *list;
- *list = info;
+ value->name = g_strdup(name);
+ QAPI_LIST_ADD(*list, value);
}
ChardevBackendInfoList *qmp_query_chardev_backends(Error **errp)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index d740a7e96371..a972d2445a20 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -510,11 +510,7 @@ static void machine_set_nvdimm_persistence(Object *obj, const char *value,
void machine_class_allow_dynamic_sysbus_dev(MachineClass *mc, const char *type)
{
- strList *item = g_new0(strList, 1);
-
- item->value = g_strdup(type);
- item->next = mc->allowed_dynamic_sysbus_devices;
- mc->allowed_dynamic_sysbus_devices = item;
+ QAPI_LIST_ADD(mc->allowed_dynamic_sysbus_devices, g_strdup(type));
}
static void validate_sysbus_device(SysBusDevice *sbdev, void *opaque)
diff --git a/hw/net/rocker/rocker.c b/hw/net/rocker/rocker.c
index 1af1e6fa2f9b..a1137e11ff48 100644
--- a/hw/net/rocker/rocker.c
+++ b/hw/net/rocker/rocker.c
@@ -127,13 +127,11 @@ RockerPortList *qmp_query_rocker_ports(const char *name, Error **errp)
}
for (i = r->fp_ports - 1; i >= 0; i--) {
- RockerPortList *info = g_malloc0(sizeof(*info));
- info->value = g_malloc0(sizeof(*info->value));
+ RockerPort *value = g_malloc0(sizeof(*value));
struct fp_port *port = r->fp_port[i];
- fp_port_get_info(port, info);
- info->next = list;
- list = info;
+ fp_port_get_info(port, value);
+ QAPI_LIST_ADD(list, value);
}
return list;
diff --git a/hw/net/rocker/rocker_fp.c b/hw/net/rocker/rocker_fp.c
index 4aa7da79b81d..a616e709292e 100644
--- a/hw/net/rocker/rocker_fp.c
+++ b/hw/net/rocker/rocker_fp.c
@@ -51,14 +51,14 @@ bool fp_port_get_link_up(FpPort *port)
return !qemu_get_queue(port->nic)->link_down;
}
-void fp_port_get_info(FpPort *port, RockerPortList *info)
+void fp_port_get_info(FpPort *port, RockerPort *value)
{
- info->value->name = g_strdup(port->name);
- info->value->enabled = port->enabled;
- info->value->link_up = fp_port_get_link_up(port);
- info->value->speed = port->speed;
- info->value->duplex = port->duplex;
- info->value->autoneg = port->autoneg;
+ value->name = g_strdup(port->name);
+ value->enabled = port->enabled;
+ value->link_up = fp_port_get_link_up(port);
+ value->speed = port->speed;
+ value->duplex = port->duplex;
+ value->autoneg = port->autoneg;
}
void fp_port_get_macaddr(FpPort *port, MACAddr *macaddr)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 277289d56e76..6b13d3ca3c8f 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -437,17 +437,14 @@ static void rxfilter_notify(NetClientState *nc)
static intList *get_vlan_table(VirtIONet *n)
{
- intList *list, *entry;
+ intList *list;
int i, j;
list = NULL;
for (i = 0; i < MAX_VLAN >> 5; i++) {
for (j = 0; n->vlans[i] && j <= 0x1f; j++) {
if (n->vlans[i] & (1U << j)) {
- entry = g_malloc0(sizeof(*entry));
- entry->value = (i << 5) + j;
- entry->next = list;
- list = entry;
+ QAPI_LIST_ADD(list, (i << 5) + j);
}
}
}
@@ -460,7 +457,7 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)
VirtIONet *n = qemu_get_nic_opaque(nc);
VirtIODevice *vdev = VIRTIO_DEVICE(n);
RxFilterInfo *info;
- strList *str_list, *entry;
+ strList *str_list;
int i;
info = g_malloc0(sizeof(*info));
@@ -491,19 +488,15 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)
str_list = NULL;
for (i = 0; i < n->mac_table.first_multi; i++) {
- entry = g_malloc0(sizeof(*entry));
- entry->value = qemu_mac_strdup_printf(n->mac_table.macs + i * ETH_ALEN);
- entry->next = str_list;
- str_list = entry;
+ QAPI_LIST_ADD(str_list,
+ qemu_mac_strdup_printf(n->mac_table.macs + i * ETH_ALEN));
}
info->unicast_table = str_list;
str_list = NULL;
for (i = n->mac_table.first_multi; i < n->mac_table.in_use; i++) {
- entry = g_malloc0(sizeof(*entry));
- entry->value = qemu_mac_strdup_printf(n->mac_table.macs + i * ETH_ALEN);
- entry->next = str_list;
- str_list = entry;
+ QAPI_LIST_ADD(str_list,
+ qemu_mac_strdup_printf(n->mac_table.macs + i * ETH_ALEN));
}
info->multicast_table = str_list;
info->vlan_table = get_vlan_table(n);
diff --git a/migration/migration.c b/migration/migration.c
index 0575ecb37953..7b849e795699 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -378,12 +378,9 @@ int migration_incoming_enable_colo(void)
void migrate_add_address(SocketAddress *address)
{
MigrationIncomingState *mis = migration_incoming_get_current();
- SocketAddressList *addrs;
- addrs = g_new0(SocketAddressList, 1);
- addrs->next = mis->socket_address_list;
- mis->socket_address_list = addrs;
- addrs->value = QAPI_CLONE(SocketAddress, address);
+ QAPI_LIST_ADD(mis->socket_address_list,
+ QAPI_CLONE(SocketAddress, address));
}
void qemu_start_incoming_migration(const char *uri, Error **errp)
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 0a2f88a87d06..18ac7e06c581 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -145,14 +145,11 @@ static struct PostcopyBlocktimeContext *blocktime_context_new(void)
static uint32List *get_vcpu_blocktime_list(PostcopyBlocktimeContext *ctx)
{
MachineState *ms = MACHINE(qdev_get_machine());
- uint32List *list = NULL, *entry = NULL;
+ uint32List *list = NULL;
int i;
for (i = ms->smp.cpus - 1; i >= 0; i--) {
- entry = g_new0(uint32List, 1);
- entry->value = ctx->vcpu_blocktime[i];
- entry->next = list;
- list = entry;
+ QAPI_LIST_ADD(list, ctx->vcpu_blocktime[i]);
}
return list;
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 9789f4277f50..629c3d1bf741 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -1248,7 +1248,8 @@ void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict)
const char *cap = qdict_get_str(qdict, "capability");
bool state = qdict_get_bool(qdict, "state");
Error *err = NULL;
- MigrationCapabilityStatusList *caps = g_malloc0(sizeof(*caps));
+ MigrationCapabilityStatusList *caps = NULL;
+ MigrationCapabilityStatus *value = NULL;
int val;
val = qapi_enum_parse(&MigrationCapability_lookup, cap, -1, &err);
@@ -1256,10 +1257,10 @@ void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict)
goto end;
}
- caps->value = g_malloc0(sizeof(*caps->value));
- caps->value->capability = val;
- caps->value->state = state;
- caps->next = NULL;
+ value = g_malloc0(sizeof(*value));
+ value->capability = val;
+ value->state = state;
+ QAPI_LIST_ADD(caps, value);
qmp_migrate_set_capabilities(caps, &err);
end:
diff --git a/qemu-img.c b/qemu-img.c
index 2103507936ea..4cfa8bccc5e7 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1643,14 +1643,13 @@ static void do_dirty_bitmap_merge(const char *dst_node, const char *dst_name,
Error **errp)
{
BlockDirtyBitmapMergeSource *merge_src;
- BlockDirtyBitmapMergeSourceList *list;
+ BlockDirtyBitmapMergeSourceList *list = NULL;
merge_src = g_new0(BlockDirtyBitmapMergeSource, 1);
merge_src->type = QTYPE_QDICT;
merge_src->u.external.node = g_strdup(src_node);
merge_src->u.external.name = g_strdup(src_name);
- list = g_new0(BlockDirtyBitmapMergeSourceList, 1);
- list->value = merge_src;
+ QAPI_LIST_ADD(list, merge_src);
qmp_block_dirty_bitmap_merge(dst_node, dst_name, list, errp);
qapi_free_BlockDirtyBitmapMergeSourceList(list);
}
diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 3bffee99d4c9..06540425ded2 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -1211,7 +1211,7 @@ GuestFilesystemInfoList *qmp_guest_get_fsinfo(Error **errp)
{
FsMountList mounts;
struct FsMount *mount;
- GuestFilesystemInfoList *new, *ret = NULL;
+ GuestFilesystemInfoList *ret = NULL;
Error *local_err = NULL;
QTAILQ_INIT(&mounts);
@@ -1224,10 +1224,7 @@ GuestFilesystemInfoList *qmp_guest_get_fsinfo(Error **errp)
QTAILQ_FOREACH(mount, &mounts, next) {
g_debug("Building guest fsinfo for '%s'", mount->dirname);
- new = g_malloc0(sizeof(*ret));
- new->value = build_guest_fsinfo(mount, &local_err);
- new->next = ret;
- ret = new;
+ QAPI_LIST_ADD(ret, build_guest_fsinfo(mount, &local_err));
if (local_err) {
error_propagate(errp, local_err);
qapi_free_GuestFilesystemInfoList(ret);
@@ -1493,7 +1490,6 @@ GuestFilesystemTrimResponse *
qmp_guest_fstrim(bool has_minimum, int64_t minimum, Error **errp)
{
GuestFilesystemTrimResponse *response;
- GuestFilesystemTrimResultList *list;
GuestFilesystemTrimResult *result;
int ret = 0;
FsMountList mounts;
@@ -1517,10 +1513,7 @@ qmp_guest_fstrim(bool has_minimum, int64_t minimum, Error **errp)
result = g_malloc0(sizeof(*result));
result->path = g_strdup(mount->dirname);
- list = g_malloc0(sizeof(*list));
- list->value = result;
- list->next = response->paths;
- response->paths = list;
+ QAPI_LIST_ADD(response->paths, result);
fd = qemu_open_old(mount->dirname, O_RDONLY);
if (fd == -1) {
diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index 0c3c05484f5f..cc5736c3bba8 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -926,10 +926,8 @@ static GuestDiskAddressList *build_guest_disk_info(char *guid, Error **errp)
error_free(local_err);
goto out;
}
- list = g_malloc0(sizeof(*list));
- list->value = disk;
+ QAPI_LIST_ADD(list, disk);
disk = NULL;
- list->next = NULL;
goto out;
} else {
error_setg_win32(errp, GetLastError(),
@@ -1064,7 +1062,7 @@ free:
GuestFilesystemInfoList *qmp_guest_get_fsinfo(Error **errp)
{
HANDLE vol_h;
- GuestFilesystemInfoList *new, *ret = NULL;
+ GuestFilesystemInfoList *ret = NULL;
char guid[256];
vol_h = FindFirstVolume(guid, sizeof(guid));
@@ -1082,10 +1080,7 @@ GuestFilesystemInfoList *qmp_guest_get_fsinfo(Error **errp)
error_free(local_err);
continue;
}
- new = g_malloc(sizeof(*ret));
- new->value = info;
- new->next = ret;
- ret = new;
+ QAPI_LIST_ADD(ret, info);
} while (FindNextVolume(vol_h, guid, sizeof(guid)));
if (GetLastError() != ERROR_NO_MORE_FILES) {
@@ -1268,11 +1263,7 @@ qmp_guest_fstrim(bool has_minimum, int64_t minimum, Error **errp)
res->path = path;
- list = g_new0(GuestFilesystemTrimResultList, 1);
- list->value = res;
- list->next = resp->paths;
-
- resp->paths = list;
+ QAPI_LIST_ADD(resp->paths, res);
memset(argv, 0, sizeof(argv));
argv[0] = (gchar *)"defrag.exe";
diff --git a/qga/commands.c b/qga/commands.c
index 3dcd5fbe5c4d..27118df6caea 100644
--- a/qga/commands.c
+++ b/qga/commands.c
@@ -66,17 +66,13 @@ static void qmp_command_info(const QmpCommand *cmd, void *opaque)
{
GuestAgentInfo *info = opaque;
GuestAgentCommandInfo *cmd_info;
- GuestAgentCommandInfoList *cmd_info_list;
cmd_info = g_new0(GuestAgentCommandInfo, 1);
cmd_info->name = g_strdup(qmp_command_name(cmd));
cmd_info->enabled = qmp_command_is_enabled(cmd);
cmd_info->success_response = qmp_has_success_response(cmd);
- cmd_info_list = g_new0(GuestAgentCommandInfoList, 1);
- cmd_info_list->value = cmd_info;
- cmd_info_list->next = info->supported_commands;
- info->supported_commands = cmd_info_list;
+ QAPI_LIST_ADD(info->supported_commands, cmd_info);
}
struct GuestAgentInfo *qmp_guest_info(Error **errp)
diff --git a/qom/qom-qmp-cmds.c b/qom/qom-qmp-cmds.c
index 310ab2d0481d..5ac9272ffeea 100644
--- a/qom/qom-qmp-cmds.c
+++ b/qom/qom-qmp-cmds.c
@@ -46,14 +46,12 @@ ObjectPropertyInfoList *qmp_qom_list(const char *path, Error **errp)
object_property_iter_init(&iter, obj);
while ((prop = object_property_iter_next(&iter))) {
- ObjectPropertyInfoList *entry = g_malloc0(sizeof(*entry));
+ ObjectPropertyInfo *value = g_malloc0(sizeof(ObjectPropertyInfo));
- entry->value = g_malloc0(sizeof(ObjectPropertyInfo));
- entry->next = props;
- props = entry;
+ QAPI_LIST_ADD(props, value);
- entry->value->name = g_strdup(prop->name);
- entry->value->type = g_strdup(prop->type);
+ value->name = g_strdup(prop->name);
+ value->type = g_strdup(prop->type);
}
return props;
@@ -90,7 +88,7 @@ QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
static void qom_list_types_tramp(ObjectClass *klass, void *data)
{
- ObjectTypeInfoList *e, **pret = data;
+ ObjectTypeInfoList **pret = data;
ObjectTypeInfo *info;
ObjectClass *parent = object_class_get_parent(klass);
@@ -102,10 +100,7 @@ static void qom_list_types_tramp(ObjectClass *klass, void *data)
info->parent = g_strdup(object_class_get_name(parent));
}
- e = g_malloc0(sizeof(*e));
- e->value = info;
- e->next = *pret;
- *pret = e;
+ QAPI_LIST_ADD(*pret, info);
}
ObjectTypeInfoList *qmp_qom_list_types(bool has_implements,
@@ -155,7 +150,6 @@ ObjectPropertyInfoList *qmp_device_list_properties(const char *typename,
object_property_iter_init(&iter, obj);
while ((prop = object_property_iter_next(&iter))) {
ObjectPropertyInfo *info;
- ObjectPropertyInfoList *entry;
/* Skip Object and DeviceState properties */
if (strcmp(prop->name, "type") == 0 ||
@@ -181,10 +175,7 @@ ObjectPropertyInfoList *qmp_device_list_properties(const char *typename,
info->default_value = qobject_ref(prop->defval);
info->has_default_value = !!info->default_value;
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = prop_list;
- prop_list = entry;
+ QAPI_LIST_ADD(prop_list, info);
}
object_unref(obj);
@@ -222,7 +213,6 @@ ObjectPropertyInfoList *qmp_qom_list_properties(const char *typename,
}
while ((prop = object_property_iter_next(&iter))) {
ObjectPropertyInfo *info;
- ObjectPropertyInfoList *entry;
info = g_malloc0(sizeof(*info));
info->name = g_strdup(prop->name);
@@ -230,10 +220,7 @@ ObjectPropertyInfoList *qmp_qom_list_properties(const char *typename,
info->has_description = !!prop->description;
info->description = g_strdup(prop->description);
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = prop_list;
- prop_list = entry;
+ QAPI_LIST_ADD(prop_list, info);
}
object_unref(obj);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 97bb6b8c01b4..df150f3c3eeb 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -8293,7 +8293,6 @@ static void arm_cpu_add_definition(gpointer data, gpointer user_data)
{
ObjectClass *oc = data;
CpuDefinitionInfoList **cpu_list = user_data;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
const char *typename;
@@ -8303,10 +8302,7 @@ static void arm_cpu_add_definition(gpointer data, gpointer user_data)
strlen(typename) - strlen("-" TYPE_ARM_CPU));
info->q_typename = g_strdup(typename);
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = *cpu_list;
- *cpu_list = entry;
+ QAPI_LIST_ADD(*cpu_list, info);
}
CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
diff --git a/target/arm/monitor.c b/target/arm/monitor.c
index 169d8a64b651..771101656535 100644
--- a/target/arm/monitor.c
+++ b/target/arm/monitor.c
@@ -42,15 +42,6 @@ static GICCapability *gic_cap_new(int version)
return cap;
}
-static GICCapabilityList *gic_cap_list_add(GICCapabilityList *head,
- GICCapability *cap)
-{
- GICCapabilityList *item = g_new0(GICCapabilityList, 1);
- item->value = cap;
- item->next = head;
- return item;
-}
-
static inline void gic_cap_kvm_probe(GICCapability *v2, GICCapability *v3)
{
#ifdef CONFIG_KVM
@@ -84,8 +75,8 @@ GICCapabilityList *qmp_query_gic_capabilities(Error **errp)
gic_cap_kvm_probe(v2, v3);
- head = gic_cap_list_add(head, v2);
- head = gic_cap_list_add(head, v3);
+ QAPI_LIST_ADD(head, v2);
+ QAPI_LIST_ADD(head, v3);
return head;
}
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 0d8606958e9e..9ae6661f97e3 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4984,7 +4984,6 @@ static void x86_cpu_definition_entry(gpointer data, gpointer user_data)
ObjectClass *oc = data;
X86CPUClass *cc = X86_CPU_CLASS(oc);
CpuDefinitionInfoList **cpu_list = user_data;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
info = g_malloc0(sizeof(*info));
@@ -5009,10 +5008,7 @@ static void x86_cpu_definition_entry(gpointer data, gpointer user_data)
info->has_alias_of = !!info->alias_of;
}
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = *cpu_list;
- *cpu_list = entry;
+ QAPI_LIST_ADD(*cpu_list, info);
}
CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
diff --git a/target/mips/helper.c b/target/mips/helper.c
index afd78b1990be..036bacc24b22 100644
--- a/target/mips/helper.c
+++ b/target/mips/helper.c
@@ -1502,7 +1502,6 @@ static void mips_cpu_add_definition(gpointer data, gpointer user_data)
{
ObjectClass *oc = data;
CpuDefinitionInfoList **cpu_list = user_data;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
const char *typename;
@@ -1512,10 +1511,7 @@ static void mips_cpu_add_definition(gpointer data, gpointer user_data)
strlen(typename) - strlen("-" TYPE_MIPS_CPU));
info->q_typename = g_strdup(typename);
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = *cpu_list;
- *cpu_list = entry;
+ QAPI_LIST_ADD(*cpu_list, info);
}
CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index ca484bfda7be..89218bde049f 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -423,7 +423,6 @@ static void create_cpu_model_list(ObjectClass *klass, void *opaque)
{
struct CpuDefinitionInfoListData *cpu_list_data = opaque;
CpuDefinitionInfoList **cpu_list = &cpu_list_data->list;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
char *name = g_strdup(object_class_get_name(klass));
S390CPUClass *scc = S390_CPU_CLASS(klass);
@@ -450,10 +449,7 @@ static void create_cpu_model_list(ObjectClass *klass, void *opaque)
object_unref(obj);
}
- entry = g_new0(CpuDefinitionInfoList, 1);
- entry->value = info;
- entry->next = *cpu_list;
- *cpu_list = entry;
+ QAPI_LIST_ADD(*cpu_list, info);
}
CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
@@ -620,12 +616,8 @@ CpuModelExpansionInfo *qmp_query_cpu_model_expansion(CpuModelExpansionType type,
static void list_add_feat(const char *name, void *opaque)
{
strList **last = (strList **) opaque;
- strList *entry;
- entry = g_new0(strList, 1);
- entry->value = g_strdup(name);
- entry->next = *last;
- *last = entry;
+ QAPI_LIST_ADD(*last, g_strdup(name));
}
CpuModelCompareInfo *qmp_query_cpu_model_comparison(CpuModelInfo *infoa,
diff --git a/tests/test-clone-visitor.c b/tests/test-clone-visitor.c
index 5e1e8b2f5e8a..03788d355130 100644
--- a/tests/test-clone-visitor.c
+++ b/tests/test-clone-visitor.c
@@ -65,16 +65,13 @@ static void test_clone_alternate(void)
static void test_clone_list_union(void)
{
- uint8List *src, *dst;
+ uint8List *src = NULL, *dst;
uint8List *tmp = NULL;
int i;
/* Build list in reverse */
for (i = 10; i; i--) {
- src = g_new0(uint8List, 1);
- src->next = tmp;
- src->value = i;
- tmp = src;
+ QAPI_LIST_ADD(src, i);
}
dst = QAPI_CLONE(uint8List, src);
diff --git a/tests/test-qobject-output-visitor.c b/tests/test-qobject-output-visitor.c
index 1c856d9bd20a..95487b139801 100644
--- a/tests/test-qobject-output-visitor.c
+++ b/tests/test-qobject-output-visitor.c
@@ -223,7 +223,8 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
const void *unused)
{
const char *value_str = "list value";
- TestStructList *p, *head = NULL;
+ TestStruct *value;
+ TestStructList *head = NULL;
const int max_items = 10;
bool value_bool = true;
int value_int = 10;
@@ -233,14 +234,12 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
/* Build the list in reverse order... */
for (i = 0; i < max_items; i++) {
- p = g_malloc0(sizeof(*p));
- p->value = g_malloc0(sizeof(*p->value));
- p->value->integer = value_int + (max_items - i - 1);
- p->value->boolean = value_bool;
- p->value->string = g_strdup(value_str);
+ value = g_malloc0(sizeof(*value));
+ value->integer = value_int + (max_items - i - 1);
+ value->boolean = value_bool;
+ value->string = g_strdup(value_str);
- p->next = head;
- head = p;
+ QAPI_LIST_ADD(head, value);
}
visit_type_TestStructList(data->ov, NULL, &head, &error_abort);
@@ -270,26 +269,25 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
const void *unused)
{
- UserDefTwoList *p, *head = NULL;
+ UserDefTwo *value;
+ UserDefTwoList *head = NULL;
const char string[] = "foo bar";
int i, max_count = 1024;
for (i = 0; i < max_count; i++) {
- p = g_malloc0(sizeof(*p));
- p->value = g_malloc0(sizeof(*p->value));
+ value = g_malloc0(sizeof(*value));
- p->value->string0 = g_strdup(string);
- p->value->dict1 = g_new0(UserDefTwoDict, 1);
- p->value->dict1->string1 = g_strdup(string);
- p->value->dict1->dict2 = g_new0(UserDefTwoDictDict, 1);
- p->value->dict1->dict2->userdef = g_new0(UserDefOne, 1);
- p->value->dict1->dict2->userdef->string = g_strdup(string);
- p->value->dict1->dict2->userdef->integer = 42;
- p->value->dict1->dict2->string = g_strdup(string);
- p->value->dict1->has_dict3 = false;
+ value->string0 = g_strdup(string);
+ value->dict1 = g_new0(UserDefTwoDict, 1);
+ value->dict1->string1 = g_strdup(string);
+ value->dict1->dict2 = g_new0(UserDefTwoDictDict, 1);
+ value->dict1->dict2->userdef = g_new0(UserDefOne, 1);
+ value->dict1->dict2->userdef->string = g_strdup(string);
+ value->dict1->dict2->userdef->integer = 42;
+ value->dict1->dict2->string = g_strdup(string);
+ value->dict1->has_dict3 = false;
- p->next = head;
- head = p;
+ QAPI_LIST_ADD(head, value);
}
qapi_free_UserDefTwoList(head);
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 1c5a8b94ea87..efbf744fcf25 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -704,10 +704,7 @@ static void test_nested_struct_list(gconstpointer opaque)
int i = 0;
for (i = 0; i < 8; i++) {
- tmp = g_new0(UserDefTwoList, 1);
- tmp->value = nested_struct_create();
- tmp->next = listp;
- listp = tmp;
+ QAPI_LIST_ADD(listp, nested_struct_create());
}
ops->serialize(listp, &serialize_data, visit_nested_struct_list,
diff --git a/trace/qmp.c b/trace/qmp.c
index 38246e1aa692..8755835edabc 100644
--- a/trace/qmp.c
+++ b/trace/qmp.c
@@ -92,39 +92,37 @@ TraceEventInfoList *qmp_trace_event_get_state(const char *name,
/* Get states (all errors checked above) */
trace_event_iter_init(&iter, name);
while ((ev = trace_event_iter_next(&iter)) != NULL) {
- TraceEventInfoList *elem;
+ TraceEventInfo *value;
bool is_vcpu = trace_event_is_vcpu(ev);
if (has_vcpu && !is_vcpu) {
continue;
}
- elem = g_new(TraceEventInfoList, 1);
- elem->value = g_new(TraceEventInfo, 1);
- elem->value->vcpu = is_vcpu;
- elem->value->name = g_strdup(trace_event_get_name(ev));
+ value = g_new(TraceEventInfo, 1);
+ value->vcpu = is_vcpu;
+ value->name = g_strdup(trace_event_get_name(ev));
if (!trace_event_get_state_static(ev)) {
- elem->value->state = TRACE_EVENT_STATE_UNAVAILABLE;
+ value->state = TRACE_EVENT_STATE_UNAVAILABLE;
} else {
if (has_vcpu) {
if (is_vcpu) {
if (trace_event_get_vcpu_state_dynamic(cpu, ev)) {
- elem->value->state = TRACE_EVENT_STATE_ENABLED;
+ value->state = TRACE_EVENT_STATE_ENABLED;
} else {
- elem->value->state = TRACE_EVENT_STATE_DISABLED;
+ value->state = TRACE_EVENT_STATE_DISABLED;
}
}
/* else: already skipped above */
} else {
if (trace_event_get_state_dynamic(ev)) {
- elem->value->state = TRACE_EVENT_STATE_ENABLED;
+ value->state = TRACE_EVENT_STATE_ENABLED;
} else {
- elem->value->state = TRACE_EVENT_STATE_DISABLED;
+ value->state = TRACE_EVENT_STATE_DISABLED;
}
}
}
- elem->next = events;
- events = elem;
+ QAPI_LIST_ADD(events, value);
}
return events;
diff --git a/ui/vnc.c b/ui/vnc.c
index f006aa1afdb2..f39cfc952906 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -365,14 +365,11 @@ static VncDisplay *vnc_display_find(const char *id)
static VncClientInfoList *qmp_query_client_list(VncDisplay *vd)
{
- VncClientInfoList *cinfo, *prev = NULL;
+ VncClientInfoList *prev = NULL;
VncState *client;
QTAILQ_FOREACH(client, &vd->clients, next) {
- cinfo = g_new0(VncClientInfoList, 1);
- cinfo->value = qmp_query_vnc_client(client);
- cinfo->next = prev;
- prev = cinfo;
+ QAPI_LIST_ADD(prev, qmp_query_vnc_client(client));
}
return prev;
}
@@ -453,7 +450,6 @@ static VncServerInfo2List *qmp_query_server_entry(QIOChannelSocket *ioc,
int subauth,
VncServerInfo2List *prev)
{
- VncServerInfo2List *list;
VncServerInfo2 *info;
Error *err = NULL;
SocketAddress *addr;
@@ -476,10 +472,8 @@ static VncServerInfo2List *qmp_query_server_entry(QIOChannelSocket *ioc,
qmp_query_auth(auth, subauth, &info->auth,
&info->vencrypt, &info->has_vencrypt);
- list = g_new0(VncServerInfo2List, 1);
- list->value = info;
- list->next = prev;
- return list;
+ QAPI_LIST_ADD(prev, info);
+ return prev;
}
static void qmp_query_auth(int auth, int subauth,
@@ -554,7 +548,7 @@ static void qmp_query_auth(int auth, int subauth,
VncInfo2List *qmp_query_vnc_servers(Error **errp)
{
- VncInfo2List *item, *prev = NULL;
+ VncInfo2List *prev = NULL;
VncInfo2 *info;
VncDisplay *vd;
DeviceState *dev;
@@ -583,10 +577,7 @@ VncInfo2List *qmp_query_vnc_servers(Error **errp)
vd->ws_subauth, info->server);
}
- item = g_new0(VncInfo2List, 1);
- item->value = info;
- item->next = prev;
- prev = item;
+ QAPI_LIST_ADD(prev, info);
}
return prev;
}
diff --git a/util/qemu-config.c b/util/qemu-config.c
index 660f47b0050f..495ada45f3df 100644
--- a/util/qemu-config.c
+++ b/util/qemu-config.c
@@ -55,7 +55,7 @@ QemuOpts *qemu_find_opts_singleton(const char *group)
static CommandLineParameterInfoList *query_option_descs(const QemuOptDesc *desc)
{
- CommandLineParameterInfoList *param_list = NULL, *entry;
+ CommandLineParameterInfoList *param_list = NULL;
CommandLineParameterInfo *info;
int i;
@@ -87,10 +87,7 @@ static CommandLineParameterInfoList *query_option_descs(const QemuOptDesc *desc)
info->q_default = g_strdup(desc[i].def_value_str);
}
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = param_list;
- param_list = entry;
+ QAPI_LIST_ADD(param_list, info);
}
return param_list;
@@ -246,7 +243,7 @@ CommandLineOptionInfoList *qmp_query_command_line_options(bool has_option,
const char *option,
Error **errp)
{
- CommandLineOptionInfoList *conf_list = NULL, *entry;
+ CommandLineOptionInfoList *conf_list = NULL;
CommandLineOptionInfo *info;
int i;
@@ -262,10 +259,7 @@ CommandLineOptionInfoList *qmp_query_command_line_options(bool has_option,
info->parameters =
query_option_descs(vm_config_groups[i]->desc);
}
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = conf_list;
- conf_list = entry;
+ QAPI_LIST_ADD(conf_list, info);
}
}
diff --git a/target/ppc/translate_init.c.inc b/target/ppc/translate_init.c.inc
index bb66526280ef..5795d0e5af2c 100644
--- a/target/ppc/translate_init.c.inc
+++ b/target/ppc/translate_init.c.inc
@@ -10621,7 +10621,6 @@ static void ppc_cpu_defs_entry(gpointer data, gpointer user_data)
ObjectClass *oc = data;
CpuDefinitionInfoList **first = user_data;
const char *typename;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
typename = object_class_get_name(oc);
@@ -10629,10 +10628,7 @@ static void ppc_cpu_defs_entry(gpointer data, gpointer user_data)
info->name = g_strndup(typename,
strlen(typename) - strlen(POWERPC_CPU_TYPE_SUFFIX));
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = *first;
- *first = entry;
+ QAPI_LIST_ADD(*first, info);
}
CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
@@ -10648,7 +10644,6 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
for (i = 0; ppc_cpu_aliases[i].alias != NULL; i++) {
PowerPCCPUAlias *alias = &ppc_cpu_aliases[i];
ObjectClass *oc;
- CpuDefinitionInfoList *entry;
CpuDefinitionInfo *info;
oc = ppc_cpu_class_by_name(alias->model);
@@ -10660,10 +10655,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
info->name = g_strdup(alias->alias);
info->q_typename = g_strdup(object_class_get_name(oc));
- entry = g_malloc0(sizeof(*entry));
- entry->value = info;
- entry->next = cpu_list;
- cpu_list = entry;
+ QAPI_LIST_ADD(cpu_list, info);
}
return cpu_list;
--
2.29.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH v5 00/12] Exposing backing-chain allocation over NBD
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
` (11 preceding siblings ...)
2020-10-23 18:36 ` [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible Eric Blake
@ 2020-10-23 18:44 ` Eric Blake
2020-10-26 14:54 ` Markus Armbruster
12 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-23 18:44 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, vsementsov, qemu-block, Markus Armbruster, rjones, stefanha
On 10/23/20 1:36 PM, Eric Blake wrote:
> v4 was here:
> https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg02708.html
>
> Since then:
> - rebase to master
> - patches 1, 2, and 12 are new based on Vladimir's observation of QAPI_LIST_ADD
> - patches 10-11 are new based on prior discussion on exposing actual
> depth in addition to a tri-state encoding
> - patch 3 has a nasty bug fixed that was causing iotest failures
> - patch 6 updated to take advantage of patch 2
> - other minor tweaks based on review comments
>
> 001/12:[down] 'qapi: Move GenericList to qapi/util.h'
> 002/12:[down] 'qapi: Make QAPI_LIST_ADD() public'
> 003/12:[0002] [FC] 'nbd: Utilize QAPI_CLONE for type conversion'
> 004/12:[0010] [FC] 'nbd: Add new qemu:allocation-depth metadata context'
> 005/12:[----] [--] 'nbd: Add 'qemu-nbd -A' to expose allocation depth'
> 006/12:[0014] [FC] 'nbd: Update qapi to support exporting multiple bitmaps'
> 007/12:[----] [--] 'nbd: Simplify qemu bitmap context name'
> 008/12:[----] [--] 'nbd: Refactor counting of metadata contexts'
> 009/12:[0017] [FC] 'nbd: Allow export of multiple bitmaps for one device'
> 010/12:[down] 'block: Return depth level during bdrv_is_allocated_above'
> 011/12:[down] 'nbd: Expose actual depth in qemu:allocation-depth'
> 012/12:[down] 'qapi: Use QAPI_LIST_ADD() where possible'
and I meant to add:
Also available at:
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/nbd-alloc-depth-v5
patch 12 is the largest; it may be worth splitting that by maintainer,
or even pushing it off post-5.2. Logically, it can go in anywhere after
patch 2, but by putting it last, I'm hoping to send a pull request for
soft freeze next week for patches 1-11 for sure, and only include 12 if
we get enough positive review in time. I did not try to see if
Coccinelle could make the work done in patch 12 more automatable.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible
2020-10-23 18:36 ` [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible Eric Blake
@ 2020-10-23 20:23 ` Eric Blake
0 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-23 20:23 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Michael S. Tsirkin, Jason Wang, Aleksandar Rikalo,
Jiaxun Yang, Michael Roth, Gerd Hoffmann, qemu-block,
Juan Quintela, David Hildenbrand, Markus Armbruster, Halil Pasic,
Christian Borntraeger, Marc-André Lureau, Richard Henderson,
Thomas Huth, Jiri Pirko, Eduardo Habkost, Dr. David Alan Gilbert,
open list:S390 KVM CPUs, vsementsov, stefanha, David Gibson,
kwolf, open list:GLUSTER, Daniel P. Berrangé,
Cornelia Huck, Philippe Mathieu-Daudé,
Max Reitz, open list:ARM TCG CPUs, open list:PowerPC TCG CPUs,
Paolo Bonzini, Aurelien Jarno
On 10/23/20 1:36 PM, Eric Blake wrote:
> Anywhere we create a list of just one item or by prepending items
> (typically because order doesn't matter), we can use the now-public
> macro. But places where we must keep the list in order by appending
> remain open-coded.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> docs/devel/writing-qmp-commands.txt | 13 +++------
> +++ b/docs/devel/writing-qmp-commands.txt
> @@ -531,15 +531,10 @@ TimerAlarmMethodList *qmp_query_alarm_methods(Error **errp)
> bool current = true;
>
> for (p = alarm_timers; p->name; p++) {
> - TimerAlarmMethodList *info = g_malloc0(sizeof(*info));
> - info->value = g_malloc0(sizeof(*info->value));
> - info->value->method_name = g_strdup(p->name);
> - info->value->current = current;
> -
> - current = false;
> -
> - info->next = method_list;
> - method_list = info;
> + TimerAlarmMethod *value = g_new0(TimerAlarmMethod, 1);
> + value->method_name = g_strdup(p->name);
Oops, tab damage.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
@ 2020-10-24 9:06 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:18 ` Markus Armbruster
1 sibling, 0 replies; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-24 9:06 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: kwolf, qemu-block, stefanha, rjones, Markus Armbruster, Michael Roth
23.10.2020 21:36, Eric Blake wrote:
> Placing GenericList in util.h will make it easier for the next patch
> to promote QAPI_LIST_ADD() into a public macro without requiring more
> files to include the unrelated visitor.h.
>
> However, we can't also move GenericAlternate; this is because it would
> introduce a circular dependency: qapi-builtin-types.h needs a complete
> definition of QEnumLookup (so it includes qapi/util.h), and
> GenericAlternate needs a complete definition of QType (declared in
> qapi-builtin-types.h). Leaving GenericAlternate in visitor.h breaks
> the cycle, and doesn't matter since we don't have any further planned
> uses for that type outside of visitors.
>
> Suggested-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/qapi/visitor.h | 9 +--------
> include/qapi/util.h | 8 ++++++++
> 2 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index ebc19ede7fff..8c2e1c29ad8b 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -16,6 +16,7 @@
> #define QAPI_VISITOR_H
>
> #include "qapi/qapi-builtin-types.h"
> +#include "qapi/util.h"
>
> /*
> * The QAPI schema defines both a set of C data types, and a QMP wire
> @@ -228,14 +229,6 @@
>
> /*** Useful types ***/
>
> -/* This struct is layout-compatible with all other *List structs
> - * created by the QAPI generator. It is used as a typical
> - * singly-linked list. */
> -typedef struct GenericList {
> - struct GenericList *next;
> - char padding[];
> -} GenericList;
> -
> /* This struct is layout-compatible with all Alternate types
> * created by the QAPI generator. */
> typedef struct GenericAlternate {
> diff --git a/include/qapi/util.h b/include/qapi/util.h
> index a7c3c6414874..50201896c7a4 100644
> --- a/include/qapi/util.h
> +++ b/include/qapi/util.h
> @@ -11,6 +11,14 @@
> #ifndef QAPI_UTIL_H
> #define QAPI_UTIL_H
>
> +/* This struct is layout-compatible with all other *List structs
> + * created by the QAPI generator. It is used as a typical
> + * singly-linked list. */
doesn't checkpatch complain for comment style?
> +typedef struct GenericList {
> + struct GenericList *next;
> + char padding[];
> +} GenericList;
> +
> typedef struct QEnumLookup {
> const char *const *array;
> int size;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
2020-10-23 18:36 ` [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public Eric Blake
@ 2020-10-24 9:10 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:25 ` Markus Armbruster
1 sibling, 0 replies; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-24 9:10 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: kwolf, qemu-block, stefanha, rjones, Max Reitz,
Markus Armbruster, Michael Roth
23.10.2020 21:36, Eric Blake wrote:
> We have a useful macro for inserting at the front of any
> QAPI-generated list; move it from block.c to qapi/util.h so more
> places can use it, including one earlier place in block.c.
>
> Suggested-by: Vladimir Sementsov-Ogievskiy<vsementsov@virtuozzo.com>
> Signed-off-by: Eric Blake<eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion
2020-10-23 18:36 ` [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion Eric Blake
@ 2020-10-24 9:17 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:41 ` Markus Armbruster
1 sibling, 0 replies; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-24 9:17 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block, stefanha, rjones, Max Reitz
23.10.2020 21:36, Eric Blake wrote:
> Rather than open-coding the translation from the deprecated
> NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
> better to utilize QAPI_CLONE_MEMBERS. This solves a couple of issues:
> first, if we do any more refactoring of the base type (which an
> upcoming patch plans to do), we don't have to revisit the open-coding.
> Second, our assignment to arg->name is fishy: the generated QAPI code
> currently does not visit it if arg->has_name is false, but if it DID
> visit it, we would have introduced a double-free situation when arg is
> finally freed.
>
> Signed-off-by: Eric Blake<eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above
2020-10-23 18:36 ` [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above Eric Blake
@ 2020-10-24 9:49 ` Vladimir Sementsov-Ogievskiy
2020-10-26 12:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-24 9:49 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: kwolf, qemu-block, stefanha, rjones, John Snow, Max Reitz, Fam Zheng
23.10.2020 21:36, Eric Blake wrote:
> When checking for allocation across a chain, it's already easy to
> count the depth within the chain at which the allocation is found.
> Instead of throwing that information away, return it to the caller.
> Existing callers only cared about allocated/non-allocated, but having
> a depth available will be used by NBD in the next patch.
>
> Note that the previous code (since commit 188a7bbf94 in 2012) was
> lazy: for each layer deeper in the backing chain, it was issuing a
> bdrv_is_allocated request on the original 'bytes' amount, rather than
> on any smaller amount 'n' learned from an upper layer. These
> semantics happened to work, since if you have:
>
> Base <- Active
> XX-- -XX-
>
> the consecutive results are offset 0: '11' with *pnum 2, followed by
> offset 2: '1' with *pnum 1, followed by offset 3: '0' with *pnum 1;
> the resulting sequence 1110 matches reality (the first three clusters
> are indeed allocated somewhere in the given range). But post-patch,
> we correctly give the results offset 0: '2' with *pnum 1, followed by
> offset 1: '11' with *pnum 2, followed by offset 3: '0' with *pnum 1
> (2110), without over-reporting the length of contributions from the
> backing layers.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
This conflicts with block-status/is-allocated fix/merge patches in Stefan's "[PULL v3 00/28] Block patches"
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth
2020-10-23 18:36 ` [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth Eric Blake
@ 2020-10-24 9:59 ` Vladimir Sementsov-Ogievskiy
2020-10-26 12:31 ` Eric Blake
0 siblings, 1 reply; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-24 9:59 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block, stefanha, rjones, Max Reitz
23.10.2020 21:36, Eric Blake wrote:
> Preserve the tri-state encoding in the low bits, as that still remains
> a valuable way to utilize qemu-img map with x-dirty-bitmap for
> accessing quick information without needing a third-party NBD client.
Hmm.. that doesn't sound as a good reason for redundant information in the protocol. Previously good reason was additional effort needed to implement sever part, but you've implemented it. And if we export depth anyway, it seems better to hack a bit nbd_client_co_block_status to convert extent.flags appropriately if metadata context is "qemu:allocation-depth" (to keep x-dirty-bitmap working), than have a workaround at the protocol layer.
> But now that the block layer gives us an actual depth, we can easily
> expose it in the remaining bits of our metadata context (leaving two
> bits reserved, to make it easier to read depth out of a raw hex
> number). This assumes no one runs a backing chain larger than 256M
> elements.
>
> iotest 309 remains unchanged (an example of the above-mentioned
> x-dirty-bitmap hack); actually testing the new bits requires libnbd or
> a similar client, and I didn't want to make iotests depend on libnbd
> at this point in time; rather, see the libnbd project for interop
> tests that exercise this new feature.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> docs/interop/nbd.txt | 6 +++++-
> include/block/nbd.h | 2 ++
> nbd/server.c | 27 +++++++++++++++------------
> 3 files changed, 22 insertions(+), 13 deletions(-)
>
> diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
> index 7e948bd42218..d90723ffe991 100644
> --- a/docs/interop/nbd.txt
> +++ b/docs/interop/nbd.txt
> @@ -34,7 +34,8 @@ the image, with a single metadata context named:
>
> qemu:allocation-depth
>
> -In the allocation depth context, bits 0 and 1 form a tri-state value:
> +In the allocation depth context, bits 0 and 1 form a tri-state value,
> +along with 28 bits giving an actual depth:
>
> bits 0-1: 00: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated
> 01: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the
> @@ -42,6 +43,9 @@ In the allocation depth context, bits 0 and 1 form a tri-state value:
> 10: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
> backing layer
> 11: invalid, never returned
> + bits 2-3: reserved, always 0
> + bits 4-31: NBD_STATE_DEPTH_RAW, the backing layer depth (0 if
> + UNALLOC, 1 for LOCAL, 2 or more for BACKING)
>
> For NBD_OPT_LIST_META_CONTEXT the following queries are supported
> in addition to the specific "qemu:allocation-depth" and
> diff --git a/include/block/nbd.h b/include/block/nbd.h
> index 956687f5c368..3c0692aec642 100644
> --- a/include/block/nbd.h
> +++ b/include/block/nbd.h
> @@ -264,6 +264,8 @@ enum {
> #define NBD_STATE_DEPTH_UNALLOC 0x0
> #define NBD_STATE_DEPTH_LOCAL 0x1
> #define NBD_STATE_DEPTH_BACKING 0x2
> +#define NBD_STATE_DEPTH_RAW_MASK 0xfffffff0
> +#define NBD_STATE_DEPTH_RAW_SHIFT 4
>
> static inline bool nbd_reply_type_is_error(int type)
> {
> diff --git a/nbd/server.c b/nbd/server.c
> index 53526090b0a2..afa79e63a7a6 100644
> --- a/nbd/server.c
> +++ b/nbd/server.c
> @@ -2037,22 +2037,25 @@ static int blockalloc_to_extents(BlockDriverState *bs, uint64_t offset,
> while (bytes) {
> uint32_t flags;
> int64_t num;
> - int ret = bdrv_is_allocated(bs, offset, bytes, &num);
> + int depth = bdrv_is_allocated_above(bs, NULL, false, offset, bytes,
> + &num);
>
> - if (ret < 0) {
> - return ret;
> - }
> -
> - if (ret == 1) {
> + switch (depth) {
> + case 0:
> + flags = NBD_STATE_DEPTH_UNALLOC;
> + break;
> + case 1:
> flags = NBD_STATE_DEPTH_LOCAL;
> - } else {
> - ret = bdrv_is_allocated_above(bs, NULL, false, offset, num,
> - &num);
> - if (ret < 0) {
> - return ret;
> + break;
> + default:
> + if (depth < 0) {
> + return depth;
> }
> - flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC;
> + flags = NBD_STATE_DEPTH_BACKING;
> + break;
> }
> + assert(depth <= UINT32_MAX >> NBD_STATE_DEPTH_RAW_SHIFT);
> + flags |= depth << NBD_STATE_DEPTH_RAW_SHIFT;
>
> if (nbd_extent_array_add(ea, num, flags) < 0) {
> return 0;
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps
2020-10-23 18:36 ` [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps Eric Blake
@ 2020-10-26 10:50 ` Peter Krempa
2020-10-26 13:06 ` Eric Blake
0 siblings, 1 reply; 31+ messages in thread
From: Peter Krempa @ 2020-10-26 10:50 UTC (permalink / raw)
To: Eric Blake
Cc: kwolf, vsementsov, qemu-block, reviewer:Incompatible changes,
qemu-devel, Markus Armbruster, stefanha, Max Reitz
On Fri, Oct 23, 2020 at 13:36:46 -0500, Eric Blake wrote:
> Since 'nbd-server-add' is deprecated, and 'block-export-add' is new to
> 5.2, we can still tweak the interface. Allowing 'bitmaps':['str'] is
> nicer than 'bitmap':'str'. This wires up the qapi and qemu-nbd
> changes to permit passing multiple bitmaps as distinct metadata
> contexts that the NBD client may request, but the actual support for
> more than one will require a further patch to the server.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> docs/system/deprecated.rst | 4 +++-
> qapi/block-export.json | 18 ++++++++++++------
> blockdev-nbd.c | 13 +++++++++++++
> nbd/server.c | 19 +++++++++++++------
> qemu-nbd.c | 10 +++++-----
> 5 files changed, 46 insertions(+), 18 deletions(-)
>
> diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
> index 905628f3a0cb..d6cd027ac740 100644
> --- a/docs/system/deprecated.rst
> +++ b/docs/system/deprecated.rst
> @@ -268,7 +268,9 @@ the 'wait' field, which is only applicable to sockets in server mode
> ''''''''''''''''''''''''''''''''''''''''''''''''''''''''
>
> Use the more generic commands ``block-export-add`` and ``block-export-del``
> -instead.
> +instead. As part of this deprecation, it is now preferred to export a
> +list of dirty bitmaps via ``bitmaps``, rather than a single bitmap via
> +``bitmap``.
>
> Human Monitor Protocol (HMP) commands
> -------------------------------------
> diff --git a/qapi/block-export.json b/qapi/block-export.json
> index 893d5cde5dfe..c7c749d61097 100644
> --- a/qapi/block-export.json
> +++ b/qapi/block-export.json
> @@ -74,10 +74,10 @@
> # @description: Free-form description of the export, up to 4096 bytes.
> # (Since 5.0)
> #
> -# @bitmap: Also export the dirty bitmap reachable from @device, so the
> -# NBD client can use NBD_OPT_SET_META_CONTEXT with the
> -# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
> -# bitmap. (since 4.0)
> +# @bitmaps: Also export each of the named dirty bitmaps reachable from
> +# @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT with
> +# the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect
> +# each bitmap. (since 5.2)
Given unsynchronised release cycles between qemu and management apps
it's not cool to deprecate an interface without having at least one
release where the replacement interface is already stable.
This means that any project wanting to stay up to date will either have
to use deprecated interfaces for at least one release and develop the
replacement only when there's a stable interface or hope that they don't
have to change the interfaces too often.
This specifically impacts libvirt as we have validators which notify us
that a deprecated interface is used and we want to stay in sync, so that
there's one less group of users to worry about at the point qemu will
want to delete the interface.
> # @allocation-depth: Also export the allocation depth map for @device, so
> # the NBD client can use NBD_OPT_SET_META_CONTEXT with
> @@ -88,7 +88,7 @@
> ##
> { 'struct': 'BlockExportOptionsNbd',
> 'data': { '*name': 'str', '*description': 'str',
> - '*bitmap': 'str', '*allocation-depth': 'bool' } }
> + '*bitmaps': ['str'], '*allocation-depth': 'bool' } }
This adds 'bitmaps' also to nbd-server-add, which should not happen. You
probably want to stop using 'base' for 'NbdServerAddOptions' and just
duplicate everything else.
> # @NbdServerAddOptions:
> @@ -100,12 +100,18 @@
> # @writable: Whether clients should be able to write to the device via the
> # NBD connection (default false).
> #
> +# @bitmap: Also export a single dirty bitmap reachable from @device, so the
> +# NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata
> +# context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap
> +# (since 4.0). Mutually exclusive with @bitmaps, and newer
> +# clients should use that instead.
This doesn't make sense, nbd-server-add never had @bitmaps. Also adding
features to a deprecated interface doesn't IMO make sense if you want to
motivate users switch to thne new one.
> +#
> # Since: 5.0
> ##
> { 'struct': 'NbdServerAddOptions',
> 'base': 'BlockExportOptionsNbd',
> 'data': { 'device': 'str',
> - '*writable': 'bool' } }
> + '*writable': 'bool', '*bitmap': 'str' } }
>
> ##
> # @nbd-server-add:
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above
2020-10-24 9:49 ` Vladimir Sementsov-Ogievskiy
@ 2020-10-26 12:26 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 31+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-10-26 12:26 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: kwolf, qemu-block, stefanha, rjones, John Snow, Max Reitz, Fam Zheng
24.10.2020 12:49, Vladimir Sementsov-Ogievskiy wrote:
> 23.10.2020 21:36, Eric Blake wrote:
>> When checking for allocation across a chain, it's already easy to
>> count the depth within the chain at which the allocation is found.
>> Instead of throwing that information away, return it to the caller.
>> Existing callers only cared about allocated/non-allocated, but having
>> a depth available will be used by NBD in the next patch.
>>
>> Note that the previous code (since commit 188a7bbf94 in 2012) was
>> lazy: for each layer deeper in the backing chain, it was issuing a
>> bdrv_is_allocated request on the original 'bytes' amount, rather than
>> on any smaller amount 'n' learned from an upper layer. These
>> semantics happened to work, since if you have:
>>
>> Base <- Active
>> XX-- -XX-
>>
>> the consecutive results are offset 0: '11' with *pnum 2, followed by
>> offset 2: '1' with *pnum 1, followed by offset 3: '0' with *pnum 1;
>> the resulting sequence 1110 matches reality (the first three clusters
>> are indeed allocated somewhere in the given range). But post-patch,
>> we correctly give the results offset 0: '2' with *pnum 1, followed by
>> offset 1: '11' with *pnum 2, followed by offset 3: '0' with *pnum 1
>> (2110), without over-reporting the length of contributions from the
>> backing layers.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> This conflicts with block-status/is-allocated fix/merge patches in Stefan's "[PULL v3 00/28] Block patches"
>
the patches applied to master branch
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth
2020-10-24 9:59 ` Vladimir Sementsov-Ogievskiy
@ 2020-10-26 12:31 ` Eric Blake
0 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-26 12:31 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, Max Reitz, stefanha, qemu-block, rjones
On 10/24/20 4:59 AM, Vladimir Sementsov-Ogievskiy wrote:
> 23.10.2020 21:36, Eric Blake wrote:
>> Preserve the tri-state encoding in the low bits, as that still remains
>> a valuable way to utilize qemu-img map with x-dirty-bitmap for
>> accessing quick information without needing a third-party NBD client.
>
> Hmm.. that doesn't sound as a good reason for redundant information in
> the protocol. Previously good reason was additional effort needed to
> implement sever part, but you've implemented it. And if we export depth
> anyway, it seems better to hack a bit nbd_client_co_block_status to
> convert extent.flags appropriately if metadata context is
> "qemu:allocation-depth" (to keep x-dirty-bitmap working), than have a
> workaround at the protocol layer.
I'm happy to respin this to expose JUST a depth rather than redundant
information, but time is short if we want it in 5.2 (as soft freeze is
this week). I'll see what I can get to today; I'll rearrange the series
to put multiple bitmap exports first (as that appears ready), while
saving 'qemu-nbd -A' until we're happy with the qemu:allocation-depth
semantics. After all, once we release something, we've committed to
that user interface.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps
2020-10-26 10:50 ` Peter Krempa
@ 2020-10-26 13:06 ` Eric Blake
0 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-26 13:06 UTC (permalink / raw)
To: Peter Krempa
Cc: kwolf, vsementsov, qemu-block, reviewer:Incompatible changes,
qemu-devel, Markus Armbruster, stefanha, Max Reitz
On 10/26/20 5:50 AM, Peter Krempa wrote:
> On Fri, Oct 23, 2020 at 13:36:46 -0500, Eric Blake wrote:
>> Since 'nbd-server-add' is deprecated, and 'block-export-add' is new to
>> 5.2, we can still tweak the interface. Allowing 'bitmaps':['str'] is
>> nicer than 'bitmap':'str'. This wires up the qapi and qemu-nbd
>> changes to permit passing multiple bitmaps as distinct metadata
>> contexts that the NBD client may request, but the actual support for
>> more than one will require a further patch to the server.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> +++ b/qapi/block-export.json
>> @@ -74,10 +74,10 @@
>> # @description: Free-form description of the export, up to 4096 bytes.
>> # (Since 5.0)
>> #
>> -# @bitmap: Also export the dirty bitmap reachable from @device, so the
>> -# NBD client can use NBD_OPT_SET_META_CONTEXT with the
>> -# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
>> -# bitmap. (since 4.0)
>> +# @bitmaps: Also export each of the named dirty bitmaps reachable from
>> +# @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT with
>> +# the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect
>> +# each bitmap. (since 5.2)
>
> Given unsynchronised release cycles between qemu and management apps
> it's not cool to deprecate an interface without having at least one
> release where the replacement interface is already stable.
>
That's why I'm trying as hard as possible to get the block-export-add
interface perfect in its 5.2 release; if we agree that allowing qemu to
expose more than one bitmap is beneficial (and I argue that it is), then
the new interface MUST support that from the get-go, and not something
where we release it with 5.2 having 'bitmap' and 6.0 adding 'bitmaps'.
> This means that any project wanting to stay up to date will either have
> to use deprecated interfaces for at least one release and develop the
> replacement only when there's a stable interface or hope that they don't
> have to change the interfaces too often.
>
> This specifically impacts libvirt as we have validators which notify us
> that a deprecated interface is used and we want to stay in sync, so that
> there's one less group of users to worry about at the point qemu will
> want to delete the interface.
The deprecated interface is nbd-server-add; for _that_ interface, the
'bitmap' parameter will continue to work until nbd-server-add is removed
in 6.1 (so you have all of 5.2 and 6.0 to switch from nbd-server-add to
block-export-add).
What this patch does, then, is alter the deprecation from merely
changing the command from nbd-server-add to block-export-add with all
parameter names remaining the same, to instead changing both the command
name and the 'bitmap'=>'bitmaps' parameter. But I agree that libvirt
wants to do an all-or-none conversion: there is no reason for libvirt to
use block-export-add until 5.2 is actually released, at which point we
have locked in the new interface; and this patch is a demonstration that
we are still debating about a tweak to that interface before it becomes
locked in.
>
>> # @allocation-depth: Also export the allocation depth map for @device, so
>> # the NBD client can use NBD_OPT_SET_META_CONTEXT with
>> @@ -88,7 +88,7 @@
>> ##
>> { 'struct': 'BlockExportOptionsNbd',
>> 'data': { '*name': 'str', '*description': 'str',
>> - '*bitmap': 'str', '*allocation-depth': 'bool' } }
>> + '*bitmaps': ['str'], '*allocation-depth': 'bool' } }
>
> This adds 'bitmaps' also to nbd-server-add, which should not happen. You
> probably want to stop using 'base' for 'NbdServerAddOptions' and just
> duplicate everything else.
I can respin this to NOT add 'bitmaps' to the legacy 'nbd-server-add',
if you think that would be better. It is more complex in the QAPI code,
but not too much more difficulty in the glue code; and the glue code all
goes away in 6.1 when the deprecation cycle ends.
>
>> # @NbdServerAddOptions:
>> @@ -100,12 +100,18 @@
>> # @writable: Whether clients should be able to write to the device via the
>> # NBD connection (default false).
>> #
>> +# @bitmap: Also export a single dirty bitmap reachable from @device, so the
>> +# NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata
>> +# context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap
>> +# (since 4.0). Mutually exclusive with @bitmaps, and newer
>> +# clients should use that instead.
>
> This doesn't make sense, nbd-server-add never had @bitmaps. Also adding
> features to a deprecated interface doesn't IMO make sense if you want to
> motivate users switch to thne new one.
Fair enough. v6 coming up later today.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
2020-10-24 9:06 ` Vladimir Sementsov-Ogievskiy
@ 2020-10-26 14:18 ` Markus Armbruster
2020-10-26 14:22 ` Eric Blake
1 sibling, 1 reply; 31+ messages in thread
From: Markus Armbruster @ 2020-10-26 14:18 UTC (permalink / raw)
To: Eric Blake
Cc: kwolf, vsementsov, Michael Roth, qemu-block, rjones, qemu-devel,
stefanha
Eric Blake <eblake@redhat.com> writes:
> Placing GenericList in util.h will make it easier for the next patch
> to promote QAPI_LIST_ADD() into a public macro without requiring more
> files to include the unrelated visitor.h.
Is this true?
You don't actually need GenericList to make use of QAPI_LIST_ADD(), do
you? Any QAPI list type should do.
> However, we can't also move GenericAlternate; this is because it would
> introduce a circular dependency: qapi-builtin-types.h needs a complete
> definition of QEnumLookup (so it includes qapi/util.h), and
> GenericAlternate needs a complete definition of QType (declared in
> qapi-builtin-types.h). Leaving GenericAlternate in visitor.h breaks
> the cycle, and doesn't matter since we don't have any further planned
> uses for that type outside of visitors.
>
> Suggested-by: Markus Armbruster <armbru@redhat.com>
I did suggest to consider moving GenericList and GenericAlternate next
to QAPI_LIST_ADD(), because they're (loosely) related. We can't move
GenericAlternate. Moving only GenericList brings GenericList and
QAPI_LIST_ADD() together, but separates the more closely related
GenericList and GenericAlternate. Meh.
I'd leave it put.
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> include/qapi/visitor.h | 9 +--------
> include/qapi/util.h | 8 ++++++++
> 2 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index ebc19ede7fff..8c2e1c29ad8b 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -16,6 +16,7 @@
> #define QAPI_VISITOR_H
>
> #include "qapi/qapi-builtin-types.h"
> +#include "qapi/util.h"
Not necessary, qapi-builtin-types.h must include it for QEnumLookup.
> /*
> * The QAPI schema defines both a set of C data types, and a QMP wire
> @@ -228,14 +229,6 @@
>
> /*** Useful types ***/
>
> -/* This struct is layout-compatible with all other *List structs
> - * created by the QAPI generator. It is used as a typical
> - * singly-linked list. */
> -typedef struct GenericList {
> - struct GenericList *next;
> - char padding[];
> -} GenericList;
> -
> /* This struct is layout-compatible with all Alternate types
> * created by the QAPI generator. */
> typedef struct GenericAlternate {
> diff --git a/include/qapi/util.h b/include/qapi/util.h
> index a7c3c6414874..50201896c7a4 100644
> --- a/include/qapi/util.h
> +++ b/include/qapi/util.h
> @@ -11,6 +11,14 @@
> #ifndef QAPI_UTIL_H
> #define QAPI_UTIL_H
>
> +/* This struct is layout-compatible with all other *List structs
> + * created by the QAPI generator. It is used as a typical
> + * singly-linked list. */
> +typedef struct GenericList {
> + struct GenericList *next;
> + char padding[];
> +} GenericList;
> +
> typedef struct QEnumLookup {
> const char *const *array;
> int size;
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h
2020-10-26 14:18 ` Markus Armbruster
@ 2020-10-26 14:22 ` Eric Blake
0 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-26 14:22 UTC (permalink / raw)
To: Markus Armbruster
Cc: kwolf, vsementsov, Michael Roth, qemu-block, rjones, qemu-devel,
stefanha
On 10/26/20 9:18 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Placing GenericList in util.h will make it easier for the next patch
>> to promote QAPI_LIST_ADD() into a public macro without requiring more
>> files to include the unrelated visitor.h.
>
> Is this true?
>
> You don't actually need GenericList to make use of QAPI_LIST_ADD(), do
> you? Any QAPI list type should do.
Correct, compilation still works if I drop this patch.
>
>> However, we can't also move GenericAlternate; this is because it would
>> introduce a circular dependency: qapi-builtin-types.h needs a complete
>> definition of QEnumLookup (so it includes qapi/util.h), and
>> GenericAlternate needs a complete definition of QType (declared in
>> qapi-builtin-types.h). Leaving GenericAlternate in visitor.h breaks
>> the cycle, and doesn't matter since we don't have any further planned
>> uses for that type outside of visitors.
>>
>> Suggested-by: Markus Armbruster <armbru@redhat.com>
>
> I did suggest to consider moving GenericList and GenericAlternate next
> to QAPI_LIST_ADD(), because they're (loosely) related. We can't move
> GenericAlternate. Moving only GenericList brings GenericList and
> QAPI_LIST_ADD() together, but separates the more closely related
> GenericList and GenericAlternate. Meh.
>
> I'd leave it put.
Agreed. Dropping this patch in v6.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
2020-10-23 18:36 ` [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public Eric Blake
2020-10-24 9:10 ` Vladimir Sementsov-Ogievskiy
@ 2020-10-26 14:25 ` Markus Armbruster
2020-10-26 14:37 ` Eric Blake
1 sibling, 1 reply; 31+ messages in thread
From: Markus Armbruster @ 2020-10-26 14:25 UTC (permalink / raw)
To: Eric Blake
Cc: kwolf, vsementsov, Michael Roth, qemu-block, rjones, qemu-devel,
stefanha, Max Reitz
Eric Blake <eblake@redhat.com> writes:
> We have a useful macro for inserting at the front of any
> QAPI-generated list; move it from block.c to qapi/util.h so more
> places can use it, including one earlier place in block.c.
>
> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> include/qapi/util.h | 8 ++++++++
> block.c | 15 +++------------
> 2 files changed, 11 insertions(+), 12 deletions(-)
>
> diff --git a/include/qapi/util.h b/include/qapi/util.h
> index 50201896c7a4..b6083055ce69 100644
> --- a/include/qapi/util.h
> +++ b/include/qapi/util.h
> @@ -30,4 +30,12 @@ int qapi_enum_parse(const QEnumLookup *lookup, const char *buf,
>
> int parse_qapi_name(const char *name, bool complete);
>
> +/* For any GenericList @list, insert @element at the front. */
> +#define QAPI_LIST_ADD(list, element) do { \
> + typeof(list) _tmp = g_new(typeof(*(list)), 1); \
g_new(typeof(*p), 1) is an rather roundabout way to say
g_malloc(sizeof(*p). Yes, it returns typeof(p) instead of void *, but
the chance of this catching mistakes here rounds to zero.
Not this patch's problem. Ignore me :)
> + _tmp->value = (element); \
> + _tmp->next = (list); \
> + (list) = _tmp; \
> +} while (0)
> +
> #endif
> diff --git a/block.c b/block.c
> index 430edf79bb10..45bd79299611 100644
> --- a/block.c
> +++ b/block.c
> @@ -39,6 +39,7 @@
> #include "qapi/qmp/qstring.h"
> #include "qapi/qobject-output-visitor.h"
> #include "qapi/qapi-visit-block-core.h"
> +#include "qapi/util.h"
> #include "sysemu/block-backend.h"
> #include "sysemu/sysemu.h"
> #include "qemu/notify.h"
> @@ -5211,7 +5212,7 @@ BlockDriverState *bdrv_find_node(const char *node_name)
> BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
> Error **errp)
> {
> - BlockDeviceInfoList *list, *entry;
> + BlockDeviceInfoList *list;
> BlockDriverState *bs;
>
> list = NULL;
> @@ -5221,22 +5222,12 @@ BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
> qapi_free_BlockDeviceInfoList(list);
> return NULL;
> }
> - entry = g_malloc0(sizeof(*entry));
> - entry->value = info;
> - entry->next = list;
> - list = entry;
> + QAPI_LIST_ADD(list, info);
> }
>
> return list;
> }
PATCH 12 has more. I wonder whether PATCH 12 should be squashed into
this one. You decide.
>
> -#define QAPI_LIST_ADD(list, element) do { \
> - typeof(list) _tmp = g_new(typeof(*(list)), 1); \
> - _tmp->value = (element); \
> - _tmp->next = (list); \
> - (list) = _tmp; \
> -} while (0)
> -
> typedef struct XDbgBlockGraphConstructor {
> XDbgBlockGraph *graph;
> GHashTable *graph_nodes;
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
2020-10-26 14:25 ` Markus Armbruster
@ 2020-10-26 14:37 ` Eric Blake
2020-10-26 14:38 ` Eric Blake
0 siblings, 1 reply; 31+ messages in thread
From: Eric Blake @ 2020-10-26 14:37 UTC (permalink / raw)
To: Markus Armbruster
Cc: kwolf, vsementsov, Michael Roth, qemu-block, rjones, qemu-devel,
stefanha, Max Reitz
On 10/26/20 9:25 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> We have a useful macro for inserting at the front of any
>> QAPI-generated list; move it from block.c to qapi/util.h so more
>> places can use it, including one earlier place in block.c.
>>
>> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> include/qapi/util.h | 8 ++++++++
>> block.c | 15 +++------------
>> 2 files changed, 11 insertions(+), 12 deletions(-)
>>
>> diff --git a/include/qapi/util.h b/include/qapi/util.h
>> index 50201896c7a4..b6083055ce69 100644
>> --- a/include/qapi/util.h
>> +++ b/include/qapi/util.h
>> @@ -30,4 +30,12 @@ int qapi_enum_parse(const QEnumLookup *lookup, const char *buf,
>>
>> int parse_qapi_name(const char *name, bool complete);
>>
>> +/* For any GenericList @list, insert @element at the front. */
>> +#define QAPI_LIST_ADD(list, element) do { \
>> + typeof(list) _tmp = g_new(typeof(*(list)), 1); \
>
> g_new(typeof(*p), 1) is an rather roundabout way to say
> g_malloc(sizeof(*p). Yes, it returns typeof(p) instead of void *, but
> the chance of this catching mistakes here rounds to zero.
>
> Not this patch's problem. Ignore me :)
typeof is a gcc/clang extension; using sizeof makes the code slightly
more portable (but doesn't affect qemu's usage). I'm happy to change
it, although it would require more commit message finesse since that
becomes more than just code motion.
>> @@ -5221,22 +5222,12 @@ BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
>> qapi_free_BlockDeviceInfoList(list);
>> return NULL;
>> }
>> - entry = g_malloc0(sizeof(*entry));
>> - entry->value = info;
>> - entry->next = list;
>> - list = entry;
>> + QAPI_LIST_ADD(list, info);
>> }
>>
>> return list;
>> }
>
> PATCH 12 has more. I wonder whether PATCH 12 should be squashed into
> this one. You decide.
Patch 12 has a LOT more. And we're really close to soft freeze. I kept
them separate to minimize the risk of landing my QAPI changes (4/12)
before 5.2 (that MUST happen, or we've locked in a problematic API with
block-export-add), vs. cleanup changes that may require review from a
lot more areas in the tree. Given the timeline, I prefer to keep them
separate for v6.
I also still wonder if it is possible to make Coccinelle identify
candidates, rather than my manual audit that produced patch 12.
But for v6, I _will_ update the commit message to mention that more
conversions are possible, and saved for a later patch.
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Thanks; I think I can keep this even for v6, since all I am changing is
an enhanced commit message.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
2020-10-26 14:37 ` Eric Blake
@ 2020-10-26 14:38 ` Eric Blake
0 siblings, 0 replies; 31+ messages in thread
From: Eric Blake @ 2020-10-26 14:38 UTC (permalink / raw)
To: Markus Armbruster
Cc: kwolf, vsementsov, Michael Roth, qemu-block, rjones, qemu-devel,
stefanha, Max Reitz
On 10/26/20 9:37 AM, Eric Blake wrote:
>> PATCH 12 has more. I wonder whether PATCH 12 should be squashed into
>> this one. You decide.
>
> Patch 12 has a LOT more. And we're really close to soft freeze. I kept
> them separate to minimize the risk of landing my QAPI changes (4/12)
Correction - the QAPI change was 6/12 of v5 (whereas it moves earlier in
the series in my upcoming v6 posting)
> before 5.2 (that MUST happen, or we've locked in a problematic API with
> block-export-add), vs. cleanup changes that may require review from a
> lot more areas in the tree.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion
2020-10-23 18:36 ` [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion Eric Blake
2020-10-24 9:17 ` Vladimir Sementsov-Ogievskiy
@ 2020-10-26 14:41 ` Markus Armbruster
1 sibling, 0 replies; 31+ messages in thread
From: Markus Armbruster @ 2020-10-26 14:41 UTC (permalink / raw)
To: Eric Blake
Cc: kwolf, vsementsov, qemu-block, rjones, qemu-devel, Max Reitz, stefanha
Eric Blake <eblake@redhat.com> writes:
> Rather than open-coding the translation from the deprecated
> NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
> better to utilize QAPI_CLONE_MEMBERS. This solves a couple of issues:
> first, if we do any more refactoring of the base type (which an
> upcoming patch plans to do), we don't have to revisit the open-coding.
> Second, our assignment to arg->name is fishy: the generated QAPI code
> currently does not visit it if arg->has_name is false, but if it DID
> visit it, we would have introduced a double-free situation when arg is
> finally freed.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> blockdev-nbd.c | 15 ++++++---------
> 1 file changed, 6 insertions(+), 9 deletions(-)
>
> diff --git a/blockdev-nbd.c b/blockdev-nbd.c
> index 8174023e5c47..cee9134b12eb 100644
> --- a/blockdev-nbd.c
> +++ b/blockdev-nbd.c
> @@ -14,6 +14,8 @@
> #include "sysemu/block-backend.h"
> #include "hw/block/block.h"
> #include "qapi/error.h"
> +#include "qapi/clone-visitor.h"
> +#include "qapi/qapi-visit-block-export.h"
> #include "qapi/qapi-commands-block-export.h"
> #include "block/nbd.h"
> #include "io/channel-socket.h"
> @@ -195,7 +197,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
> * the device name as a default here for compatibility.
> */
> if (!arg->has_name) {
> - arg->name = arg->device;
> + arg->has_name = true;
> + arg->name = g_strdup(arg->device);
> }
This is the fix you mentioned in the commit message.
>
> export_opts = g_new(BlockExportOptions, 1);
> @@ -205,15 +208,9 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
*export_opts = (BlockExportOptions) {
.type = BLOCK_EXPORT_TYPE_NBD,
.id = g_strdup(arg->name),
> .node_name = g_strdup(bdrv_get_node_name(bs)),
> .has_writable = arg->has_writable,
> .writable = arg->writable,
Explicit initialization of all the common members, except for
@writethrough. @writethrough is optional, so not mentioning it makes it
absent. I don't mind.
> - .u.nbd = {
> - .has_name = true,
> - .name = g_strdup(arg->name),
> - .has_description = arg->has_description,
> - .description = g_strdup(arg->description),
> - .has_bitmap = arg->has_bitmap,
> - .bitmap = g_strdup(arg->bitmap),
Explicit initialization of all the variant members: copy of @arg.
> - },
> };
> + QAPI_CLONE_MEMBERS(BlockExportOptionsNbd, &export_opts->u.nbd,
> + qapi_NbdServerAddOptions_base(arg));
Another (and better) way to copy.
>
> /*
> * nbd-server-add doesn't complain when a read-only device should be
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v5 00/12] Exposing backing-chain allocation over NBD
2020-10-23 18:44 ` [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
@ 2020-10-26 14:54 ` Markus Armbruster
0 siblings, 0 replies; 31+ messages in thread
From: Markus Armbruster @ 2020-10-26 14:54 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, vsementsov, qemu-block, qemu-devel, rjones, stefanha
Eric Blake <eblake@redhat.com> writes:
> On 10/23/20 1:36 PM, Eric Blake wrote:
>> v4 was here:
>> https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg02708.html
>>
>> Since then:
>> - rebase to master
>> - patches 1, 2, and 12 are new based on Vladimir's observation of QAPI_LIST_ADD
>> - patches 10-11 are new based on prior discussion on exposing actual
>> depth in addition to a tri-state encoding
>> - patch 3 has a nasty bug fixed that was causing iotest failures
>> - patch 6 updated to take advantage of patch 2
>> - other minor tweaks based on review comments
>>
>> 001/12:[down] 'qapi: Move GenericList to qapi/util.h'
>> 002/12:[down] 'qapi: Make QAPI_LIST_ADD() public'
>> 003/12:[0002] [FC] 'nbd: Utilize QAPI_CLONE for type conversion'
>> 004/12:[0010] [FC] 'nbd: Add new qemu:allocation-depth metadata context'
>> 005/12:[----] [--] 'nbd: Add 'qemu-nbd -A' to expose allocation depth'
>> 006/12:[0014] [FC] 'nbd: Update qapi to support exporting multiple bitmaps'
>> 007/12:[----] [--] 'nbd: Simplify qemu bitmap context name'
>> 008/12:[----] [--] 'nbd: Refactor counting of metadata contexts'
>> 009/12:[0017] [FC] 'nbd: Allow export of multiple bitmaps for one device'
>> 010/12:[down] 'block: Return depth level during bdrv_is_allocated_above'
>> 011/12:[down] 'nbd: Expose actual depth in qemu:allocation-depth'
>> 012/12:[down] 'qapi: Use QAPI_LIST_ADD() where possible'
>
> and I meant to add:
>
> Also available at:
> https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/nbd-alloc-depth-v5
>
> patch 12 is the largest; it may be worth splitting that by maintainer,
I wouldn't split. The various parts will trickle in separately, and you
may well end up holding leftovers.
> or even pushing it off post-5.2. Logically, it can go in anywhere after
> patch 2, but by putting it last, I'm hoping to send a pull request for
> soft freeze next week for patches 1-11 for sure, and only include 12 if
> we get enough positive review in time. I did not try to see if
> Coccinelle could make the work done in patch 12 more automatable.
PATCH 12 serves a useful purpose even if we can't get it in right away:
it convinces me that making QAPI_LIST_ADD() public makes sense.
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2020-10-26 14:56 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-23 18:36 [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
2020-10-23 18:36 ` [PATCH v5 01/12] qapi: Move GenericList to qapi/util.h Eric Blake
2020-10-24 9:06 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:18 ` Markus Armbruster
2020-10-26 14:22 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public Eric Blake
2020-10-24 9:10 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:25 ` Markus Armbruster
2020-10-26 14:37 ` Eric Blake
2020-10-26 14:38 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion Eric Blake
2020-10-24 9:17 ` Vladimir Sementsov-Ogievskiy
2020-10-26 14:41 ` Markus Armbruster
2020-10-23 18:36 ` [PATCH v5 04/12] nbd: Add new qemu:allocation-depth metadata context Eric Blake
2020-10-23 18:36 ` [PATCH v5 05/12] nbd: Add 'qemu-nbd -A' to expose allocation depth Eric Blake
2020-10-23 18:36 ` [PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps Eric Blake
2020-10-26 10:50 ` Peter Krempa
2020-10-26 13:06 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 07/12] nbd: Simplify qemu bitmap context name Eric Blake
2020-10-23 18:36 ` [PATCH v5 08/12] nbd: Refactor counting of metadata contexts Eric Blake
2020-10-23 18:36 ` [PATCH v5 09/12] nbd: Allow export of multiple bitmaps for one device Eric Blake
2020-10-23 18:36 ` [PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above Eric Blake
2020-10-24 9:49 ` Vladimir Sementsov-Ogievskiy
2020-10-26 12:26 ` Vladimir Sementsov-Ogievskiy
2020-10-23 18:36 ` [PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth Eric Blake
2020-10-24 9:59 ` Vladimir Sementsov-Ogievskiy
2020-10-26 12:31 ` Eric Blake
2020-10-23 18:36 ` [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible Eric Blake
2020-10-23 20:23 ` Eric Blake
2020-10-23 18:44 ` [PATCH v5 00/12] Exposing backing-chain allocation over NBD Eric Blake
2020-10-26 14:54 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.