* [PATCH] block/copy-on-read: use bdrv_drop_filter() and drop s->active
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 6:44 ` [PATCH 01/21] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
` (20 subsequent siblings)
21 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Now, after huge update of block graph permission update algorithm, we
don't need this workaround with active state of the filter. Drop it and
use new smart bdrv_drop_filter() function.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-on-read.c | 33 +--------------------------------
1 file changed, 1 insertion(+), 32 deletions(-)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 9cad9e1b8c..c428682272 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -29,7 +29,6 @@
typedef struct BDRVStateCOR {
- bool active;
BlockDriverState *bottom_bs;
bool chain_frozen;
} BDRVStateCOR;
@@ -89,7 +88,6 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,
*/
bdrv_ref(bottom_bs);
}
- state->active = true;
state->bottom_bs = bottom_bs;
/*
@@ -112,17 +110,6 @@ static void cor_child_perm(BlockDriverState *bs, BdrvChild *c,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared)
{
- BDRVStateCOR *s = bs->opaque;
-
- if (!s->active) {
- /*
- * While the filter is being removed
- */
- *nperm = 0;
- *nshared = BLK_PERM_ALL;
- return;
- }
-
*nperm = perm & PERM_PASSTHROUGH;
*nshared = (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED;
@@ -280,32 +267,14 @@ static BlockDriver bdrv_copy_on_read = {
void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs)
{
- BdrvChild *child;
- BlockDriverState *bs;
BDRVStateCOR *s = cor_filter_bs->opaque;
- child = bdrv_filter_child(cor_filter_bs);
- if (!child) {
- return;
- }
- bs = child->bs;
-
- /* Retain the BDS until we complete the graph change. */
- bdrv_ref(bs);
- /* Hold a guest back from writing while permissions are being reset. */
- bdrv_drained_begin(bs);
- /* Drop permissions before the graph change. */
- s->active = false;
/* unfreeze, as otherwise bdrv_replace_node() will fail */
if (s->chain_frozen) {
s->chain_frozen = false;
bdrv_unfreeze_backing_chain(cor_filter_bs, s->bottom_bs);
}
- bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort);
- bdrv_replace_node(cor_filter_bs, bs, &error_abort);
-
- bdrv_drained_end(bs);
- bdrv_unref(bs);
+ bdrv_drop_filter(cor_filter_bs, &error_abort);
bdrv_unref(cor_filter_bs);
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
2021-05-17 6:44 ` [PATCH] block/copy-on-read: use bdrv_drop_filter() and drop s->active Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 12:09 ` Max Reitz
2021-05-17 6:44 ` [PATCH 02/21] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
` (19 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/block/block.h | 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/include/block/block.h b/include/block/block.h
index 82185965ff..f9d5fcb108 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
Error **errp);
int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+ Error **errp);
BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
int flags, Error **errp);
int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
diff --git a/block.c b/block.c
index 9ad725d205..755fa53d85 100644
--- a/block.c
+++ b/block.c
@@ -4961,6 +4961,42 @@ out:
return ret;
}
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+ Error **errp)
+{
+ int ret;
+ Transaction *tran = tran_new();
+ g_autoptr(GHashTable) found = NULL;
+ g_autoptr(GSList) refresh_list = NULL;
+ BlockDriverState *old_bs = child->bs;
+
+ if (old_bs) {
+ bdrv_ref(old_bs);
+ bdrv_drained_begin(old_bs);
+ }
+ bdrv_drained_begin(new_bs);
+
+ bdrv_replace_child(child, new_bs, tran);
+
+ found = g_hash_table_new(NULL, NULL);
+ if (old_bs) {
+ refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+ }
+ refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+ ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
+
+ tran_finalize(tran, ret);
+
+ if (old_bs) {
+ bdrv_drained_end(old_bs);
+ bdrv_unref(old_bs);
+ }
+ bdrv_drained_end(new_bs);
+
+ return ret;
+}
+
static void bdrv_delete(BlockDriverState *bs)
{
assert(bdrv_op_blocker_is_empty(bs));
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 6:44 ` [PATCH 01/21] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
@ 2021-05-17 12:09 ` Max Reitz
2021-05-17 14:30 ` Vladimir Sementsov-Ogievskiy
2021-05-19 10:12 ` Vladimir Sementsov-Ogievskiy
0 siblings, 2 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-17 12:09 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Add function to transactionally replace bs inside BdrvChild.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/block/block.h | 2 ++
> block.c | 36 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 38 insertions(+)
As you may guess, I know little about the rewritten replacing functions,
so this is kind of difficult to review for me. However, nothing looks
out of place, and the function looks sufficiently similar to
bdrv_replace_node_common() to make me happy.
> diff --git a/include/block/block.h b/include/block/block.h
> index 82185965ff..f9d5fcb108 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
> Error **errp);
> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
> Error **errp);
> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
> + Error **errp);
> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
> int flags, Error **errp);
> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
> diff --git a/block.c b/block.c
> index 9ad725d205..755fa53d85 100644
> --- a/block.c
> +++ b/block.c
> @@ -4961,6 +4961,42 @@ out:
> return ret;
> }
>
> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
> + Error **errp)
> +{
> + int ret;
> + Transaction *tran = tran_new();
> + g_autoptr(GHashTable) found = NULL;
> + g_autoptr(GSList) refresh_list = NULL;
> + BlockDriverState *old_bs = child->bs;
> +
> + if (old_bs) {
Hm. Can child->bs be ever NULL?
> + bdrv_ref(old_bs);
> + bdrv_drained_begin(old_bs);
> + }
> + bdrv_drained_begin(new_bs);
(I was wondering why we couldn’t handle the new_bs == NULL case here to
replace bdrv_remove_filter_or_cow_child(), but then I realized it’s
probably because that’s kind of difficult, precisely because child->bs
at least should generally be non-NULL. Which is why
bdrv_remove_filter_or_cow_child() needs to add its own transaction entry
to handle the BdrvChild object and the pointer to it.
Hence me wondering whether we could assume child->bs not to be NULL.)
> +
> + bdrv_replace_child(child, new_bs, tran);
> +
> + found = g_hash_table_new(NULL, NULL);
> + if (old_bs) {
> + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
> + }
> + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
> +
> + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t
refresh permissions. I think it’s correct to do it here, so the
following question doesn’t really concern this patch, but: Why don’t we
do it there?
I guess it’s because we expect the node to go away anyway, so we don’t
need to refresh the permissions. And that assumption should hold true
right now, given its callers. But is that a safe assumption in general?
Would there be a problem if we refreshed permissions there? Or is not
refreshing permissions just part of the function’s interface?
Max
> +
> + tran_finalize(tran, ret);
> +
> + if (old_bs) {
> + bdrv_drained_end(old_bs);
> + bdrv_unref(old_bs);
> + }
> + bdrv_drained_end(new_bs);
> +
> + return ret;
> +}
> +
> static void bdrv_delete(BlockDriverState *bs)
> {
> assert(bdrv_op_blocker_is_empty(bs));
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 12:09 ` Max Reitz
@ 2021-05-17 14:30 ` Vladimir Sementsov-Ogievskiy
2021-05-17 15:51 ` Max Reitz
2021-05-19 10:12 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 14:30 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 15:09, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> Add function to transactionally replace bs inside BdrvChild.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> include/block/block.h | 2 ++
>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>> 2 files changed, 38 insertions(+)
>
> As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.
>
>> diff --git a/include/block/block.h b/include/block/block.h
>> index 82185965ff..f9d5fcb108 100644
>> --- a/include/block/block.h
>> +++ b/include/block/block.h
>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
>> Error **errp);
>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>> Error **errp);
>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>> + Error **errp);
>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
>> int flags, Error **errp);
>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>> diff --git a/block.c b/block.c
>> index 9ad725d205..755fa53d85 100644
>> --- a/block.c
>> +++ b/block.c
>> @@ -4961,6 +4961,42 @@ out:
>> return ret;
>> }
>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>> + Error **errp)
>> +{
>> + int ret;
>> + Transaction *tran = tran_new();
>> + g_autoptr(GHashTable) found = NULL;
>> + g_autoptr(GSList) refresh_list = NULL;
>> + BlockDriverState *old_bs = child->bs;
>> +
>> + if (old_bs) {
>
> Hm. Can child->bs be ever NULL?
Hmm. Most probably not :)
In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.
>
>> + bdrv_ref(old_bs);
>> + bdrv_drained_begin(old_bs);
>> + }
>> + bdrv_drained_begin(new_bs);
>
> (I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL. Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.
>
> Hence me wondering whether we could assume child->bs not to be NULL.)
bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.
>
>> +
>> + bdrv_replace_child(child, new_bs, tran);
>> +
>> + found = g_hash_table_new(NULL, NULL);
>> + if (old_bs) {
>> + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
>> + }
>> + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
>> +
>> + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
>
> Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
>
> I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
>
Caller of bdrv_remove_filter_or_cow_child() should care about permissions: bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..
The main impact of previous big rework of permission is new scheme of working with permission update:
- first do all graph modifications, not thinking about permissions
- refresh permissions for the whole updated subgraph
- if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)
So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.
>
>> +
>> + tran_finalize(tran, ret);
>> +
>> + if (old_bs) {
>> + bdrv_drained_end(old_bs);
>> + bdrv_unref(old_bs);
>> + }
>> + bdrv_drained_end(new_bs);
>> +
>> + return ret;
>> +}
>> +
>> static void bdrv_delete(BlockDriverState *bs)
>> {
>> assert(bdrv_op_blocker_is_empty(bs));
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 14:30 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-17 15:51 ` Max Reitz
2021-05-17 18:05 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-17 15:51 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 16:30, Vladimir Sementsov-Ogievskiy wrote:
> 17.05.2021 15:09, Max Reitz wrote:
>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>> Add function to transactionally replace bs inside BdrvChild.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> include/block/block.h | 2 ++
>>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 38 insertions(+)
>>
>> As you may guess, I know little about the rewritten replacing
>> functions, so this is kind of difficult to review for me. However,
>> nothing looks out of place, and the function looks sufficiently
>> similar to bdrv_replace_node_common() to make me happy.
>>
>>> diff --git a/include/block/block.h b/include/block/block.h
>>> index 82185965ff..f9d5fcb108 100644
>>> --- a/include/block/block.h
>>> +++ b/include/block/block.h
>>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new,
>>> BlockDriverState *bs_top,
>>> Error **errp);
>>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>>> Error **errp);
>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>> + Error **errp);
>>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict
>>> *node_options,
>>> int flags, Error **errp);
>>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>>> diff --git a/block.c b/block.c
>>> index 9ad725d205..755fa53d85 100644
>>> --- a/block.c
>>> +++ b/block.c
>>> @@ -4961,6 +4961,42 @@ out:
>>> return ret;
>>> }
>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>> + Error **errp)
>>> +{
>>> + int ret;
>>> + Transaction *tran = tran_new();
>>> + g_autoptr(GHashTable) found = NULL;
>>> + g_autoptr(GSList) refresh_list = NULL;
>>> + BlockDriverState *old_bs = child->bs;
>>> +
>>> + if (old_bs) {
>>
>> Hm. Can child->bs be ever NULL?
>
> Hmm. Most probably not :)
>
> In some intermediate states we don't have bs in child, but it shouldn't
> be the place where bdrv_replace_child_bs is called.
>
>>
>>> + bdrv_ref(old_bs);
>>> + bdrv_drained_begin(old_bs);
>>> + }
>>> + bdrv_drained_begin(new_bs);
>>
>> (I was wondering why we couldn’t handle the new_bs == NULL case here
>> to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s
>> probably because that’s kind of difficult, precisely because child->bs
>> at least should generally be non-NULL. Which is why
>> bdrv_remove_filter_or_cow_child() needs to add its own transaction
>> entry to handle the BdrvChild object and the pointer to it.
>>
>> Hence me wondering whether we could assume child->bs not to be NULL.)
>
> bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't
> do drained section nor permission update. And new
> bdrv_replace_child_bs() is public function, which cares about these things.
>
>>
>>> +
>>> + bdrv_replace_child(child, new_bs, tran);
>>> +
>>> + found = g_hash_table_new(NULL, NULL);
>>> + if (old_bs) {
>>> + refresh_list = bdrv_topological_dfs(refresh_list, found,
>>> old_bs);
>>> + }
>>> + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
>>> +
>>> + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
>>
>> Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t
>> refresh permissions. I think it’s correct to do it here, so the
>> following question doesn’t really concern this patch, but: Why don’t
>> we do it there?
>>
>> I guess it’s because we expect the node to go away anyway, so we don’t
>> need to refresh the permissions. And that assumption should hold true
>> right now, given its callers. But is that a safe assumption in
>> general? Would there be a problem if we refreshed permissions there?
>> Or is not refreshing permissions just part of the function’s interface?
>>
>
> Caller of bdrv_remove_filter_or_cow_child() should care about
> permissions: bdrv_replace_node_common() do this, and
> bdrv_set_backing_noperm() has "_noperm" in the name..
OK. Makes me wonder why bdrv_remove_filter_or_cow_child() then doesn’t
have _noperm in its name, or why its comment doesn’t explain this
interface contract, but, well. :)
> The main impact of previous big rework of permission is new scheme of
> working with permission update:
>
> - first do all graph modifications, not thinking about permissions
> - refresh permissions for the whole updated subgraph
> - if refresh failed, rollback all the modifications (main sense if
> transactions here and there is possibility to do this rollback)
>
> So a lot of internal functions with @tran argument don't update
> permissions. But of course, we should care to update permissions after
> any graph modification.
Ah, OK. Makes sense, thanks.
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 15:51 ` Max Reitz
@ 2021-05-17 18:05 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 18:05 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 18:51, Max Reitz wrote:
> On 17.05.21 16:30, Vladimir Sementsov-Ogievskiy wrote:
>> 17.05.2021 15:09, Max Reitz wrote:
>>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>>> Add function to transactionally replace bs inside BdrvChild.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>> include/block/block.h | 2 ++
>>>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>>>> 2 files changed, 38 insertions(+)
>>>
>>> As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.
>>>
>>>> diff --git a/include/block/block.h b/include/block/block.h
>>>> index 82185965ff..f9d5fcb108 100644
>>>> --- a/include/block/block.h
>>>> +++ b/include/block/block.h
>>>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
>>>> Error **errp);
>>>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>>>> Error **errp);
>>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>>> + Error **errp);
>>>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
>>>> int flags, Error **errp);
>>>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>>>> diff --git a/block.c b/block.c
>>>> index 9ad725d205..755fa53d85 100644
>>>> --- a/block.c
>>>> +++ b/block.c
>>>> @@ -4961,6 +4961,42 @@ out:
>>>> return ret;
>>>> }
>>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>>> + Error **errp)
>>>> +{
>>>> + int ret;
>>>> + Transaction *tran = tran_new();
>>>> + g_autoptr(GHashTable) found = NULL;
>>>> + g_autoptr(GSList) refresh_list = NULL;
>>>> + BlockDriverState *old_bs = child->bs;
>>>> +
>>>> + if (old_bs) {
>>>
>>> Hm. Can child->bs be ever NULL?
>>
>> Hmm. Most probably not :)
>>
>> In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.
>>
>>>
>>>> + bdrv_ref(old_bs);
>>>> + bdrv_drained_begin(old_bs);
>>>> + }
>>>> + bdrv_drained_begin(new_bs);
>>>
>>> (I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL. Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.
>>>
>>> Hence me wondering whether we could assume child->bs not to be NULL.)
>>
>> bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.
>>
>>>
>>>> +
>>>> + bdrv_replace_child(child, new_bs, tran);
>>>> +
>>>> + found = g_hash_table_new(NULL, NULL);
>>>> + if (old_bs) {
>>>> + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
>>>> + }
>>>> + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
>>>> +
>>>> + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
>>>
>>> Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
>>>
>>> I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
>>>
>>
>> Caller of bdrv_remove_filter_or_cow_child() should care about permissions: bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..
>
> OK. Makes me wonder why bdrv_remove_filter_or_cow_child() then doesn’t have _noperm in its name, or why its comment doesn’t explain this interface contract, but, well. :)
You are right that's unclear. I'll make the patch that cleans that up.
>
>> The main impact of previous big rework of permission is new scheme of working with permission update:
>>
>> - first do all graph modifications, not thinking about permissions
>> - refresh permissions for the whole updated subgraph
>> - if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)
>>
>> So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.
>
> Ah, OK. Makes sense, thanks.
>
> Max
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-17 12:09 ` Max Reitz
2021-05-17 14:30 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-19 10:12 ` Vladimir Sementsov-Ogievskiy
2021-05-19 11:11 ` Max Reitz
1 sibling, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-19 10:12 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 15:09, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> Add function to transactionally replace bs inside BdrvChild.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> include/block/block.h | 2 ++
>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>> 2 files changed, 38 insertions(+)
>
> As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.
>
>> diff --git a/include/block/block.h b/include/block/block.h
>> index 82185965ff..f9d5fcb108 100644
>> --- a/include/block/block.h
>> +++ b/include/block/block.h
>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
>> Error **errp);
>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>> Error **errp);
>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>> + Error **errp);
>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
>> int flags, Error **errp);
>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>> diff --git a/block.c b/block.c
>> index 9ad725d205..755fa53d85 100644
>> --- a/block.c
>> +++ b/block.c
>> @@ -4961,6 +4961,42 @@ out:
>> return ret;
>> }
>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>> + Error **errp)
>> +{
>> + int ret;
>> + Transaction *tran = tran_new();
>> + g_autoptr(GHashTable) found = NULL;
>> + g_autoptr(GSList) refresh_list = NULL;
>> + BlockDriverState *old_bs = child->bs;
>> +
>> + if (old_bs) {
>
> Hm. Can child->bs be ever NULL?
Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(
qmp eject and blockdev-remove-medium seems do it too.
>
>> + bdrv_ref(old_bs);
>> + bdrv_drained_begin(old_bs);
>> + }
>> + bdrv_drained_begin(new_bs);
>
> (I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL. Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.
>
> Hence me wondering whether we could assume child->bs not to be NULL.)
>
>> +
>> + bdrv_replace_child(child, new_bs, tran);
>> +
>> + found = g_hash_table_new(NULL, NULL);
>> + if (old_bs) {
>> + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
>> + }
>> + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
>> +
>> + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
>
> Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
>
> I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
>
> Max
>
>> +
>> + tran_finalize(tran, ret);
>> +
>> + if (old_bs) {
>> + bdrv_drained_end(old_bs);
>> + bdrv_unref(old_bs);
>> + }
>> + bdrv_drained_end(new_bs);
>> +
>> + return ret;
>> +}
>> +
>> static void bdrv_delete(BlockDriverState *bs)
>> {
>> assert(bdrv_op_blocker_is_empty(bs));
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-19 10:12 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-19 11:11 ` Max Reitz
2021-05-19 11:14 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-19 11:11 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 19.05.21 12:12, Vladimir Sementsov-Ogievskiy wrote:
> 17.05.2021 15:09, Max Reitz wrote:
>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>> Add function to transactionally replace bs inside BdrvChild.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> include/block/block.h | 2 ++
>>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 38 insertions(+)
>>
>> As you may guess, I know little about the rewritten replacing
>> functions, so this is kind of difficult to review for me. However,
>> nothing looks out of place, and the function looks sufficiently
>> similar to bdrv_replace_node_common() to make me happy.
>>
>>> diff --git a/include/block/block.h b/include/block/block.h
>>> index 82185965ff..f9d5fcb108 100644
>>> --- a/include/block/block.h
>>> +++ b/include/block/block.h
>>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new,
>>> BlockDriverState *bs_top,
>>> Error **errp);
>>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>>> Error **errp);
>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>> + Error **errp);
>>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict
>>> *node_options,
>>> int flags, Error **errp);
>>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>>> diff --git a/block.c b/block.c
>>> index 9ad725d205..755fa53d85 100644
>>> --- a/block.c
>>> +++ b/block.c
>>> @@ -4961,6 +4961,42 @@ out:
>>> return ret;
>>> }
>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>> + Error **errp)
>>> +{
>>> + int ret;
>>> + Transaction *tran = tran_new();
>>> + g_autoptr(GHashTable) found = NULL;
>>> + g_autoptr(GSList) refresh_list = NULL;
>>> + BlockDriverState *old_bs = child->bs;
>>> +
>>> + if (old_bs) {
>>
>> Hm. Can child->bs be ever NULL?
>
> Seems it can. For example we have hmp_drive_del command, that removes bs
> from blk :(
>
> qmp eject and blockdev-remove-medium seems do it too.
blk_remove_bs() doesn’t eject the BDS from the BdrvChild blk->root,
though, it drops blk->root altogether. Doesn’t it?
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
2021-05-19 11:11 ` Max Reitz
@ 2021-05-19 11:14 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-19 11:14 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
19.05.2021 14:11, Max Reitz wrote:
> On 19.05.21 12:12, Vladimir Sementsov-Ogievskiy wrote:
>> 17.05.2021 15:09, Max Reitz wrote:
>>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>>> Add function to transactionally replace bs inside BdrvChild.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>> include/block/block.h | 2 ++
>>>> block.c | 36 ++++++++++++++++++++++++++++++++++++
>>>> 2 files changed, 38 insertions(+)
>>>
>>> As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.
>>>
>>>> diff --git a/include/block/block.h b/include/block/block.h
>>>> index 82185965ff..f9d5fcb108 100644
>>>> --- a/include/block/block.h
>>>> +++ b/include/block/block.h
>>>> @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
>>>> Error **errp);
>>>> int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
>>>> Error **errp);
>>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>>> + Error **errp);
>>>> BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
>>>> int flags, Error **errp);
>>>> int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
>>>> diff --git a/block.c b/block.c
>>>> index 9ad725d205..755fa53d85 100644
>>>> --- a/block.c
>>>> +++ b/block.c
>>>> @@ -4961,6 +4961,42 @@ out:
>>>> return ret;
>>>> }
>>>> +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
>>>> + Error **errp)
>>>> +{
>>>> + int ret;
>>>> + Transaction *tran = tran_new();
>>>> + g_autoptr(GHashTable) found = NULL;
>>>> + g_autoptr(GSList) refresh_list = NULL;
>>>> + BlockDriverState *old_bs = child->bs;
>>>> +
>>>> + if (old_bs) {
>>>
>>> Hm. Can child->bs be ever NULL?
>>
>> Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(
>>
>> qmp eject and blockdev-remove-medium seems do it too.
>
> blk_remove_bs() doesn’t eject the BDS from the BdrvChild blk->root, though, it drops blk->root altogether. Doesn’t it?
>
A hm, yes. What I say is that we can have empty blk. But probably we should never have BdrvChild with NULL bs. I'll check it
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 02/21] block: introduce blk_replace_bs
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
2021-05-17 6:44 ` [PATCH] block/copy-on-read: use bdrv_drop_filter() and drop s->active Vladimir Sementsov-Ogievskiy
2021-05-17 6:44 ` [PATCH 01/21] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 12:32 ` Max Reitz
2021-05-17 6:44 ` [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
` (18 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Add function to change bs inside blk.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/sysemu/block-backend.h | 1 +
block/block-backend.c | 8 ++++++++
2 files changed, 9 insertions(+)
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 880e903293..aec05ef0a0 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);
BlockDriverState *blk_bs(BlockBackend *blk);
void blk_remove_bs(BlockBackend *blk);
int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp);
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp);
bool bdrv_has_blk(BlockDriverState *bs);
bool bdrv_is_root_node(BlockDriverState *bs);
int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
diff --git a/block/block-backend.c b/block/block-backend.c
index de5496af66..b1abc6f3e6 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)
return 0;
}
+/*
+ * Change BlockDriverState associated with @blk.
+ */
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp)
+{
+ return bdrv_replace_child_bs(blk->root, new_bs, errp);
+}
+
/*
* Sets the permission bitmasks that the user of the BlockBackend needs.
*/
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 02/21] block: introduce blk_replace_bs
2021-05-17 6:44 ` [PATCH 02/21] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
@ 2021-05-17 12:32 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-17 12:32 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Add function to change bs inside blk.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/sysemu/block-backend.h | 1 +
> block/block-backend.c | 8 ++++++++
> 2 files changed, 9 insertions(+)
>
> diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
> index 880e903293..aec05ef0a0 100644
> --- a/include/sysemu/block-backend.h
> +++ b/include/sysemu/block-backend.h
> @@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);
> BlockDriverState *blk_bs(BlockBackend *blk);
> void blk_remove_bs(BlockBackend *blk);
> int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp);
> +int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp);
> bool bdrv_has_blk(BlockDriverState *bs);
> bool bdrv_is_root_node(BlockDriverState *bs);
> int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
> diff --git a/block/block-backend.c b/block/block-backend.c
> index de5496af66..b1abc6f3e6 100644
> --- a/block/block-backend.c
> +++ b/block/block-backend.c
> @@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)
> return 0;
> }
>
> +/*
> + * Change BlockDriverState associated with @blk.
> + */
> +int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp)
> +{
> + return bdrv_replace_child_bs(blk->root, new_bs, errp);
> +}
Reviewed-by: Max Reitz <mreitz@redhat.com>
(Looks indeed like we don’t need to do any of the things that
blk_insert_bs() and blk_remove_bs() do besides inserting and removing
the node.)
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 02/21] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 12:40 ` Max Reitz
2021-05-17 6:44 ` [PATCH 04/21] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
` (17 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Add field, so property can declare support for setting the property
when device is realized. To be used in the following commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/hw/qdev-properties.h | 1 +
hw/core/qdev-properties.c | 6 +++---
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
index 0ef97d60ce..007e1f69f4 100644
--- a/include/hw/qdev-properties.h
+++ b/include/hw/qdev-properties.h
@@ -32,6 +32,7 @@ struct PropertyInfo {
const char *name;
const char *description;
const QEnumLookup *enum_table;
+ bool realized_set_allowed;
int (*print)(Object *obj, Property *prop, char *dest, size_t len);
void (*set_default_value)(ObjectProperty *op, const Property *prop);
ObjectProperty *(*create)(ObjectClass *oc, const char *name,
diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c
index 50f40949f5..c34aac6ebc 100644
--- a/hw/core/qdev-properties.c
+++ b/hw/core/qdev-properties.c
@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
/* returns: true if property is allowed to be set, false otherwise */
static bool qdev_prop_allow_set(Object *obj, const char *name,
- Error **errp)
+ const PropertyInfo *info, Error **errp)
{
DeviceState *dev = DEVICE(obj);
- if (dev->realized) {
+ if (dev->realized && !info->realized_set_allowed) {
qdev_prop_set_after_realize(dev, name, errp);
return false;
}
@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
{
Property *prop = opaque;
- if (!qdev_prop_allow_set(obj, name, errp)) {
+ if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
return;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field
2021-05-17 6:44 ` [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
@ 2021-05-17 12:40 ` Max Reitz
2021-05-17 14:33 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-17 12:40 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Add field, so property can declare support for setting the property
> when device is realized. To be used in the following commit.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/hw/qdev-properties.h | 1 +
> hw/core/qdev-properties.c | 6 +++---
> 2 files changed, 4 insertions(+), 3 deletions(-)
Looks OK to me, although qdev isn’t my specialty.
> diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
> index 0ef97d60ce..007e1f69f4 100644
> --- a/include/hw/qdev-properties.h
> +++ b/include/hw/qdev-properties.h
> @@ -32,6 +32,7 @@ struct PropertyInfo {
> const char *name;
> const char *description;
> const QEnumLookup *enum_table;
> + bool realized_set_allowed;
I think a comment would be nice, though.
Max
> int (*print)(Object *obj, Property *prop, char *dest, size_t len);
> void (*set_default_value)(ObjectProperty *op, const Property *prop);
> ObjectProperty *(*create)(ObjectClass *oc, const char *name,
> diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c
> index 50f40949f5..c34aac6ebc 100644
> --- a/hw/core/qdev-properties.c
> +++ b/hw/core/qdev-properties.c
> @@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
>
> /* returns: true if property is allowed to be set, false otherwise */
> static bool qdev_prop_allow_set(Object *obj, const char *name,
> - Error **errp)
> + const PropertyInfo *info, Error **errp)
> {
> DeviceState *dev = DEVICE(obj);
>
> - if (dev->realized) {
> + if (dev->realized && !info->realized_set_allowed) {
> qdev_prop_set_after_realize(dev, name, errp);
> return false;
> }
> @@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
> {
> Property *prop = opaque;
>
> - if (!qdev_prop_allow_set(obj, name, errp)) {
> + if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
> return;
> }
>
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field
2021-05-17 12:40 ` Max Reitz
@ 2021-05-17 14:33 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 14:33 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 15:40, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> Add field, so property can declare support for setting the property
>> when device is realized. To be used in the following commit.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> include/hw/qdev-properties.h | 1 +
>> hw/core/qdev-properties.c | 6 +++---
>> 2 files changed, 4 insertions(+), 3 deletions(-)
>
> Looks OK to me, although qdev isn’t my specialty.
Neither my :) Thanks for looking anyway!
>
>> diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
>> index 0ef97d60ce..007e1f69f4 100644
>> --- a/include/hw/qdev-properties.h
>> +++ b/include/hw/qdev-properties.h
>> @@ -32,6 +32,7 @@ struct PropertyInfo {
>> const char *name;
>> const char *description;
>> const QEnumLookup *enum_table;
>> + bool realized_set_allowed;
>
> I think a comment would be nice, though.
>
Agree, will add.
>
>> int (*print)(Object *obj, Property *prop, char *dest, size_t len);
>> void (*set_default_value)(ObjectProperty *op, const Property *prop);
>> ObjectProperty *(*create)(ObjectClass *oc, const char *name,
>> diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c
>> index 50f40949f5..c34aac6ebc 100644
>> --- a/hw/core/qdev-properties.c
>> +++ b/hw/core/qdev-properties.c
>> @@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
>> /* returns: true if property is allowed to be set, false otherwise */
>> static bool qdev_prop_allow_set(Object *obj, const char *name,
>> - Error **errp)
>> + const PropertyInfo *info, Error **errp)
>> {
>> DeviceState *dev = DEVICE(obj);
>> - if (dev->realized) {
>> + if (dev->realized && !info->realized_set_allowed) {
>> qdev_prop_set_after_realize(dev, name, errp);
>> return false;
>> }
>> @@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
>> {
>> Property *prop = opaque;
>> - if (!qdev_prop_allow_set(obj, name, errp)) {
>> + if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
>> return;
>> }
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 04/21] qdev: allow setting drive property for realized device
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (3 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 15:48 ` Max Reitz
2021-05-17 6:44 ` [PATCH 05/21] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
` (16 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
We need an ability to insert filters above top block node, attached to
block device. It can't be achieved with blockdev-reopen command. So, we
want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A,
attached to some guest block device.
2. qom-set, to change bs attached to root blk from original node to
newly create filter.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
1 file changed, 22 insertions(+), 8 deletions(-)
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index 2760c21f11..7d97562654 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -93,16 +93,30 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
BlockBackend *blk;
bool blk_created = false;
int ret;
+ BlockDriverState *bs;
+ AioContext *ctx;
if (!visit_type_str(v, name, &str, errp)) {
return;
}
- /*
- * TODO Should this really be an error? If no, the old value
- * needs to be released before we store the new one.
- */
- if (!check_prop_still_unset(obj, name, *ptr, str, errp)) {
+ if (*ptr) {
+ /* BlockBackend alread exists. So, we want to change attached node */
+ blk = *ptr;
+ ctx = blk_get_aio_context(blk);
+ bs = bdrv_lookup_bs(NULL, str, errp);
+ if (!bs) {
+ return;
+ }
+
+ if (ctx != bdrv_get_aio_context(bs)) {
+ error_setg(errp, "Different aio context is not supported for new "
+ "node");
+ }
+
+ aio_context_acquire(ctx);
+ blk_replace_bs(blk, bs, errp);
+ aio_context_release(ctx);
return;
}
@@ -114,7 +128,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
blk = blk_by_name(str);
if (!blk) {
- BlockDriverState *bs = bdrv_lookup_bs(NULL, str, NULL);
+ bs = bdrv_lookup_bs(NULL, str, NULL);
if (bs) {
/*
* If the device supports iothreads, it will make sure to move the
@@ -123,8 +137,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
* aware of iothreads require their BlockBackends to be in the main
* AioContext.
*/
- AioContext *ctx = iothread ? bdrv_get_aio_context(bs) :
- qemu_get_aio_context();
+ ctx = iothread ? bdrv_get_aio_context(bs) : qemu_get_aio_context();
blk = blk_new(ctx, 0, BLK_PERM_ALL);
blk_created = true;
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
const PropertyInfo qdev_prop_drive = {
.name = "str",
.description = "Node name or ID of a block device to use as a backend",
+ .realized_set_allowed = true,
.get = get_drive,
.set = set_drive,
.release = release_drive,
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 04/21] qdev: allow setting drive property for realized device
2021-05-17 6:44 ` [PATCH 04/21] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
@ 2021-05-17 15:48 ` Max Reitz
2021-05-17 18:09 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-17 15:48 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> We need an ability to insert filters above top block node, attached to
> block device. It can't be achieved with blockdev-reopen command. So, we
> want do it with help of qom-set.
>
> Intended usage:
>
> 1. blockdev-add, creating the filter, which child is at top node A,
> attached to some guest block device.
Is a “not” missing here, i.e. “not attached to any guest block device”?
I would have thought one would create a filtered tree that is not in
use by any frontend, so that the filter need not take any permissions.
> 2. qom-set, to change bs attached to root blk from original node to
> newly create filter.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
> 1 file changed, 22 insertions(+), 8 deletions(-)
Looks good, just one question: (well, two, one was above)
> diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
> index 2760c21f11..7d97562654 100644
> --- a/hw/core/qdev-properties-system.c
> +++ b/hw/core/qdev-properties-system.c
[...]
> @@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
> const PropertyInfo qdev_prop_drive = {
> .name = "str",
> .description = "Node name or ID of a block device to use as a backend",
> + .realized_set_allowed = true,
> .get = get_drive,
> .set = set_drive,
> .release = release_drive,
Why not for qdev_prop_drive_iothread?
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 04/21] qdev: allow setting drive property for realized device
2021-05-17 15:48 ` Max Reitz
@ 2021-05-17 18:09 ` Vladimir Sementsov-Ogievskiy
2021-05-18 9:09 ` Max Reitz
0 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 18:09 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 18:48, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> We need an ability to insert filters above top block node, attached to
>> block device. It can't be achieved with blockdev-reopen command. So, we
>> want do it with help of qom-set.
>>
>> Intended usage:
>>
>> 1. blockdev-add, creating the filter, which child is at top node A,
>> attached to some guest block device.
>
> Is a “not” missing here, i.e. “not attached to any guest block device”? I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.
node A is attached.
So, we have [blk] --root-> [A}
And want to insert a filter between blk and A.
We do
1.
[filter] --file--\
v
[blk] --root--> [A]
2.
[blk] --root--> [filer] --file--> [A]
>
>> 2. qom-set, to change bs attached to root blk from original node to
>> newly create filter.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
>> 1 file changed, 22 insertions(+), 8 deletions(-)
>
> Looks good, just one question: (well, two, one was above)
>
>> diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
>> index 2760c21f11..7d97562654 100644
>> --- a/hw/core/qdev-properties-system.c
>> +++ b/hw/core/qdev-properties-system.c
>
> [...]
>
>> @@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
>> const PropertyInfo qdev_prop_drive = {
>> .name = "str",
>> .description = "Node name or ID of a block device to use as a backend",
>> + .realized_set_allowed = true,
>> .get = get_drive,
>> .set = set_drive,
>> .release = release_drive,
>
> Why not for qdev_prop_drive_iothread?
>
Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 04/21] qdev: allow setting drive property for realized device
2021-05-17 18:09 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-18 9:09 ` Max Reitz
2021-05-18 9:20 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 9:09 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 20:09, Vladimir Sementsov-Ogievskiy wrote:
> 17.05.2021 18:48, Max Reitz wrote:
>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>> We need an ability to insert filters above top block node, attached to
>>> block device. It can't be achieved with blockdev-reopen command. So, we
>>> want do it with help of qom-set.
>>>
>>> Intended usage:
>>>
>>> 1. blockdev-add, creating the filter, which child is at top node A,
>>> attached to some guest block device.
>>
>> Is a “not” missing here, i.e. “not attached to any guest block
>> device”? I would have thought one would create a filtered tree that
>> is not in use by any frontend, so that the filter need not take any
>> permissions.
>
> node A is attached.
>
> So, we have [blk] --root-> [A}
>
> And want to insert a filter between blk and A.
>
> We do
>
> 1.
>
> [filter] --file--\
> v
> [blk] --root--> [A]
Oh, so you mean node A is attached to a guest device. The sentence
sounded to me like the newly created filter tree were attached to it.
Yes, that’s how I expected it to be. I just find the sentence not quite
clear, because I found it ambiguous which node the “attached to some
guest block device” refers to.
Perhaps:
“Intended usage:
Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node B that has A as its child.
2. qom-set to change the node attached to the guest device’s
BlockBackend from A to B.”
?
>
> 2.
>
> [blk] --root--> [filer] --file--> [A]
>
>>
>>> 2. qom-set, to change bs attached to root blk from original node to
>>> newly create filter.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
>>> 1 file changed, 22 insertions(+), 8 deletions(-)
>>
>> Looks good, just one question: (well, two, one was above)
>>
>>> diff --git a/hw/core/qdev-properties-system.c
>>> b/hw/core/qdev-properties-system.c
>>> index 2760c21f11..7d97562654 100644
>>> --- a/hw/core/qdev-properties-system.c
>>> +++ b/hw/core/qdev-properties-system.c
>>
>> [...]
>>
>>> @@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char
>>> *name, void *opaque)
>>> const PropertyInfo qdev_prop_drive = {
>>> .name = "str",
>>> .description = "Node name or ID of a block device to use as a
>>> backend",
>>> + .realized_set_allowed = true,
>>> .get = get_drive,
>>> .set = set_drive,
>>> .release = release_drive,
>>
>> Why not for qdev_prop_drive_iothread?
>>
>
> Hmm, the only reason is that I missed that part of architecture around
> here, I'm new to qdev code. Will add with next version
OK. (I just saw it because it was right below this structure, too, it
isn’t like I actually know what I’m saying.)
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 04/21] qdev: allow setting drive property for realized device
2021-05-18 9:09 ` Max Reitz
@ 2021-05-18 9:20 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 9:20 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 12:09, Max Reitz wrote:
> On 17.05.21 20:09, Vladimir Sementsov-Ogievskiy wrote:
>> 17.05.2021 18:48, Max Reitz wrote:
>>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>>> We need an ability to insert filters above top block node, attached to
>>>> block device. It can't be achieved with blockdev-reopen command. So, we
>>>> want do it with help of qom-set.
>>>>
>>>> Intended usage:
>>>>
>>>> 1. blockdev-add, creating the filter, which child is at top node A,
>>>> attached to some guest block device.
>>>
>>> Is a “not” missing here, i.e. “not attached to any guest block device”? I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.
>>
>> node A is attached.
>>
>> So, we have [blk] --root-> [A}
>>
>> And want to insert a filter between blk and A.
>>
>> We do
>>
>> 1.
>>
>> [filter] --file--\
>> v
>> [blk] --root--> [A]
>
> Oh, so you mean node A is attached to a guest device. The sentence sounded to me like the newly created filter tree were attached to it.
>
> Yes, that’s how I expected it to be. I just find the sentence not quite clear, because I found it ambiguous which node the “attached to some guest block device” refers to.
>
> Perhaps:
> “Intended usage:
>
> Assume there is a node A that is attached to some guest device.
> 1. blockdev-add to create a filter node B that has A as its child.
> 2. qom-set to change the node attached to the guest device’s BlockBackend from A to B.”
>
> ?
Yes, sounds good, thanks
>
>>
>> 2.
>>
>> [blk] --root--> [filer] --file--> [A]
>>
>>>
>>>> 2. qom-set, to change bs attached to root blk from original node to
>>>> newly create filter.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>> hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
>>>> 1 file changed, 22 insertions(+), 8 deletions(-)
>>>
>>> Looks good, just one question: (well, two, one was above)
>>>
>>>> diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
>>>> index 2760c21f11..7d97562654 100644
>>>> --- a/hw/core/qdev-properties-system.c
>>>> +++ b/hw/core/qdev-properties-system.c
>>>
>>> [...]
>>>
>>>> @@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
>>>> const PropertyInfo qdev_prop_drive = {
>>>> .name = "str",
>>>> .description = "Node name or ID of a block device to use as a backend",
>>>> + .realized_set_allowed = true,
>>>> .get = get_drive,
>>>> .set = set_drive,
>>>> .release = release_drive,
>>>
>>> Why not for qdev_prop_drive_iothread?
>>>
>>
>> Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version
>
> OK. (I just saw it because it was right below this structure, too, it isn’t like I actually know what I’m saying.)
>
> Max
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 05/21] block: rename backup-top to copy-before-write
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (4 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 04/21] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 16:05 ` Max Reitz
2021-05-17 6:44 ` [PATCH 06/21] block/backup: drop support for copy_range Vladimir Sementsov-Ogievskiy
` (15 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
We are going to convert backup_top to full featured public filter,
which can be used in separate of backup job. Start from renaming from
"how it used" to "what it does".
While updating comments in 283 iotest, drop and rephrase also things
about ".active", as this field is now dropped, and filter doesn't have
"inactive" mode.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/{backup-top.h => copy-before-write.h} | 28 +++---
block/backup.c | 22 ++---
block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
MAINTAINERS | 4 +-
block/meson.build | 2 +-
tests/qemu-iotests/283 | 35 +++----
tests/qemu-iotests/283.out | 4 +-
7 files changed, 95 insertions(+), 100 deletions(-)
rename block/{backup-top.h => copy-before-write.h} (56%)
rename block/{backup-top.c => copy-before-write.c} (62%)
diff --git a/block/backup-top.h b/block/copy-before-write.h
similarity index 56%
rename from block/backup-top.h
rename to block/copy-before-write.h
index b28b0031c4..5977b7aa31 100644
--- a/block/backup-top.h
+++ b/block/copy-before-write.h
@@ -1,10 +1,10 @@
/*
- * backup-top filter driver
+ * copy-before-write filter driver
*
* The driver performs Copy-Before-Write (CBW) operation: it is injected above
* some node, and before each write it copies _old_ data to the target node.
*
- * Copyright (c) 2018-2019 Virtuozzo International GmbH.
+ * Copyright (c) 2018-2021 Virtuozzo International GmbH.
*
* Author:
* Sementsov-Ogievskiy Vladimir <vsementsov@virtuozzo.com>
@@ -23,20 +23,20 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-#ifndef BACKUP_TOP_H
-#define BACKUP_TOP_H
+#ifndef COPY_BEFORE_WRITE_H
+#define COPY_BEFORE_WRITE_H
#include "block/block_int.h"
#include "block/block-copy.h"
-BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
- BlockDriverState *target,
- const char *filter_node_name,
- uint64_t cluster_size,
- BackupPerf *perf,
- BdrvRequestFlags write_flags,
- BlockCopyState **bcs,
- Error **errp);
-void bdrv_backup_top_drop(BlockDriverState *bs);
+BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
+ BlockDriverState *target,
+ const char *filter_node_name,
+ uint64_t cluster_size,
+ BackupPerf *perf,
+ BdrvRequestFlags write_flags,
+ BlockCopyState **bcs,
+ Error **errp);
+void bdrv_cbw_drop(BlockDriverState *bs);
-#endif /* BACKUP_TOP_H */
+#endif /* COPY_BEFORE_WRITE_H */
diff --git a/block/backup.c b/block/backup.c
index 6cf2f974aa..0b5887269c 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -27,13 +27,13 @@
#include "qemu/bitmap.h"
#include "qemu/error-report.h"
-#include "block/backup-top.h"
+#include "block/copy-before-write.h"
#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
typedef struct BackupBlockJob {
BlockJob common;
- BlockDriverState *backup_top;
+ BlockDriverState *cbw;
BlockDriverState *source_bs;
BlockDriverState *target_bs;
@@ -104,7 +104,7 @@ static void backup_clean(Job *job)
{
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
block_job_remove_all_bdrv(&s->common);
- bdrv_backup_top_drop(s->backup_top);
+ bdrv_cbw_drop(s->cbw);
}
void backup_do_checkpoint(BlockJob *job, Error **errp)
@@ -408,7 +408,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BackupBlockJob *job = NULL;
int64_t cluster_size;
BdrvRequestFlags write_flags;
- BlockDriverState *backup_top = NULL;
+ BlockDriverState *cbw = NULL;
BlockCopyState *bcs = NULL;
assert(bs);
@@ -521,22 +521,22 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) |
(compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
- backup_top = bdrv_backup_top_append(bs, target, filter_node_name,
+ cbw = bdrv_cbw_append(bs, target, filter_node_name,
cluster_size, perf,
write_flags, &bcs, errp);
- if (!backup_top) {
+ if (!cbw) {
goto error;
}
/* job->len is fixed, so we can't allow resize */
- job = block_job_create(job_id, &backup_job_driver, txn, backup_top,
+ job = block_job_create(job_id, &backup_job_driver, txn, cbw,
0, BLK_PERM_ALL,
speed, creation_flags, cb, opaque, errp);
if (!job) {
goto error;
}
- job->backup_top = backup_top;
+ job->cbw = cbw;
job->source_bs = bs;
job->target_bs = target;
job->on_source_error = on_source_error;
@@ -552,7 +552,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
block_copy_set_progress_meter(bcs, &job->common.job.progress);
block_copy_set_speed(bcs, speed);
- /* Required permissions are already taken by backup-top target */
+ /* Required permissions are taken by copy-before-write filter target */
block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
&error_abort);
@@ -562,8 +562,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
if (sync_bitmap) {
bdrv_reclaim_dirty_bitmap(sync_bitmap, NULL);
}
- if (backup_top) {
- bdrv_backup_top_drop(backup_top);
+ if (cbw) {
+ bdrv_cbw_drop(cbw);
}
return NULL;
diff --git a/block/backup-top.c b/block/copy-before-write.c
similarity index 62%
rename from block/backup-top.c
rename to block/copy-before-write.c
index 425e3778be..40e91832d7 100644
--- a/block/backup-top.c
+++ b/block/copy-before-write.c
@@ -1,10 +1,10 @@
/*
- * backup-top filter driver
+ * copy-before-write filter driver
*
* The driver performs Copy-Before-Write (CBW) operation: it is injected above
* some node, and before each write it copies _old_ data to the target node.
*
- * Copyright (c) 2018-2019 Virtuozzo International GmbH.
+ * Copyright (c) 2018-2021 Virtuozzo International GmbH.
*
* Author:
* Sementsov-Ogievskiy Vladimir <vsementsov@virtuozzo.com>
@@ -32,25 +32,25 @@
#include "block/qdict.h"
#include "block/block-copy.h"
-#include "block/backup-top.h"
+#include "block/copy-before-write.h"
-typedef struct BDRVBackupTopState {
+typedef struct BDRVCopyBeforeWriteState {
BlockCopyState *bcs;
BdrvChild *target;
int64_t cluster_size;
-} BDRVBackupTopState;
+} BDRVCopyBeforeWriteState;
-static coroutine_fn int backup_top_co_preadv(
+static coroutine_fn int cbw_co_preadv(
BlockDriverState *bs, uint64_t offset, uint64_t bytes,
QEMUIOVector *qiov, int flags)
{
return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);
}
-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
- uint64_t bytes, BdrvRequestFlags flags)
+static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, BdrvRequestFlags flags)
{
- BDRVBackupTopState *s = bs->opaque;
+ BDRVCopyBeforeWriteState *s = bs->opaque;
uint64_t off, end;
if (flags & BDRV_REQ_WRITE_UNCHANGED) {
@@ -63,10 +63,10 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
return block_copy(s->bcs, off, end - off, true);
}
-static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
- int64_t offset, int bytes)
+static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
+ int64_t offset, int bytes)
{
- int ret = backup_top_cbw(bs, offset, bytes, 0);
+ int ret = cbw_cbw(bs, offset, bytes, 0);
if (ret < 0) {
return ret;
}
@@ -74,10 +74,10 @@ static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
return bdrv_co_pdiscard(bs->backing, offset, bytes);
}
-static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs,
+static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
int64_t offset, int bytes, BdrvRequestFlags flags)
{
- int ret = backup_top_cbw(bs, offset, bytes, flags);
+ int ret = cbw_cbw(bs, offset, bytes, flags);
if (ret < 0) {
return ret;
}
@@ -85,12 +85,12 @@ static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs,
return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
}
-static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs,
- uint64_t offset,
- uint64_t bytes,
- QEMUIOVector *qiov, int flags)
+static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
+ uint64_t offset,
+ uint64_t bytes,
+ QEMUIOVector *qiov, int flags)
{
- int ret = backup_top_cbw(bs, offset, bytes, flags);
+ int ret = cbw_cbw(bs, offset, bytes, flags);
if (ret < 0) {
return ret;
}
@@ -98,7 +98,7 @@ static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs,
return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags);
}
-static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
{
if (!bs->backing) {
return 0;
@@ -107,7 +107,7 @@ static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
return bdrv_co_flush(bs->backing->bs);
}
-static void backup_top_refresh_filename(BlockDriverState *bs)
+static void cbw_refresh_filename(BlockDriverState *bs)
{
if (bs->backing == NULL) {
/*
@@ -120,11 +120,11 @@ static void backup_top_refresh_filename(BlockDriverState *bs)
bs->backing->bs->filename);
}
-static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
- BdrvChildRole role,
- BlockReopenQueue *reopen_queue,
- uint64_t perm, uint64_t shared,
- uint64_t *nperm, uint64_t *nshared)
+static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
+ BdrvChildRole role,
+ BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
{
if (!(role & BDRV_CHILD_FILTERED)) {
/*
@@ -149,41 +149,41 @@ static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
-BlockDriver bdrv_backup_top_filter = {
- .format_name = "backup-top",
- .instance_size = sizeof(BDRVBackupTopState),
+BlockDriver bdrv_cbw_filter = {
+ .format_name = "copy-before-write",
+ .instance_size = sizeof(BDRVCopyBeforeWriteState),
- .bdrv_co_preadv = backup_top_co_preadv,
- .bdrv_co_pwritev = backup_top_co_pwritev,
- .bdrv_co_pwrite_zeroes = backup_top_co_pwrite_zeroes,
- .bdrv_co_pdiscard = backup_top_co_pdiscard,
- .bdrv_co_flush = backup_top_co_flush,
+ .bdrv_co_preadv = cbw_co_preadv,
+ .bdrv_co_pwritev = cbw_co_pwritev,
+ .bdrv_co_pwrite_zeroes = cbw_co_pwrite_zeroes,
+ .bdrv_co_pdiscard = cbw_co_pdiscard,
+ .bdrv_co_flush = cbw_co_flush,
- .bdrv_refresh_filename = backup_top_refresh_filename,
+ .bdrv_refresh_filename = cbw_refresh_filename,
- .bdrv_child_perm = backup_top_child_perm,
+ .bdrv_child_perm = cbw_child_perm,
.is_filter = true,
};
-BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
- BlockDriverState *target,
- const char *filter_node_name,
- uint64_t cluster_size,
- BackupPerf *perf,
- BdrvRequestFlags write_flags,
- BlockCopyState **bcs,
- Error **errp)
+BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
+ BlockDriverState *target,
+ const char *filter_node_name,
+ uint64_t cluster_size,
+ BackupPerf *perf,
+ BdrvRequestFlags write_flags,
+ BlockCopyState **bcs,
+ Error **errp)
{
ERRP_GUARD();
int ret;
- BDRVBackupTopState *state;
+ BDRVCopyBeforeWriteState *state;
BlockDriverState *top;
bool appended = false;
assert(source->total_sectors == target->total_sectors);
- top = bdrv_new_open_driver(&bdrv_backup_top_filter, filter_node_name,
+ top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name,
BDRV_O_RDWR, errp);
if (!top) {
return NULL;
@@ -210,7 +210,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
ret = bdrv_append(top, source, errp);
if (ret < 0) {
- error_prepend(errp, "Cannot append backup-top filter: ");
+ error_prepend(errp, "Cannot append copy-before-write filter: ");
goto fail;
}
appended = true;
@@ -231,7 +231,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
fail:
if (appended) {
- bdrv_backup_top_drop(top);
+ bdrv_cbw_drop(top);
} else {
bdrv_unref(top);
}
@@ -241,9 +241,9 @@ fail:
return NULL;
}
-void bdrv_backup_top_drop(BlockDriverState *bs)
+void bdrv_cbw_drop(BlockDriverState *bs)
{
- BDRVBackupTopState *s = bs->opaque;
+ BDRVCopyBeforeWriteState *s = bs->opaque;
bdrv_drop_filter(bs, &error_abort);
diff --git a/MAINTAINERS b/MAINTAINERS
index 78561a223f..4321f67fef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2281,8 +2281,8 @@ F: block/mirror.c
F: qapi/job.json
F: block/block-copy.c
F: include/block/block-copy.c
-F: block/backup-top.h
-F: block/backup-top.c
+F: block/copy-before-write.h
+F: block/copy-before-write.c
F: include/block/aio_task.h
F: block/aio_task.c
F: util/qemu-co-shared-resource.c
diff --git a/block/meson.build b/block/meson.build
index e687c54dbc..6e27aa9624 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -4,7 +4,7 @@ block_ss.add(files(
'aio_task.c',
'amend.c',
'backup.c',
- 'backup-top.c',
+ 'copy-before-write.c',
'blkdebug.c',
'blklogwrites.c',
'blkverify.c',
diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283
index 010c22f0a2..a09e0183ae 100755
--- a/tests/qemu-iotests/283
+++ b/tests/qemu-iotests/283
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# group: auto quick
#
-# Test for backup-top filter permission activation failure
+# Test for copy-before-write filter permission conflict
#
# Copyright (c) 2019 Virtuozzo International GmbH.
#
@@ -31,13 +31,13 @@ size = 1024 * 1024
""" Test description
When performing a backup, all writes on the source subtree must go through the
-backup-top filter so it can copy all data to the target before it is changed.
-backup-top filter is appended above source node, to achieve this thing, so all
-parents of source node are handled. A configuration with side parents of source
-sub-tree with write permission is unsupported (we'd have append several
-backup-top filter like nodes to handle such parents). The test create an
-example of such configuration and checks that a backup is then not allowed
-(blockdev-backup command should fail).
+copy-before-write filter so it can copy all data to the target before it is
+changed. copy-before-write filter is appended above source node, to achieve
+this thing, so all parents of source node are handled. A configuration with
+side parents of source sub-tree with write permission is unsupported (we'd have
+append several copy-before-write filter like nodes to handle such parents). The
+test create an example of such configuration and checks that a backup is then
+not allowed (blockdev-backup command should fail).
The configuration:
@@ -57,11 +57,10 @@ The configuration:
│ base │ ◀──────────── │ other │
└─────────────┘ └───────┘
-On activation (see .active field of backup-top state in block/backup-top.c),
-backup-top is going to unshare write permission on its source child. Write
-unsharing will be propagated to the "source->base" link and will conflict with
-other node write permission. So permission update will fail and backup job will
-not be started.
+copy-before-write filter wants to unshare write permission on its source child.
+Write unsharing will be propagated to the "source->base" link and will conflict
+with other node write permission. So permission update will fail and backup job
+will not be started.
Note, that the only thing which prevents backup of running on such
configuration is default permission propagation scheme. It may be altered by
@@ -99,13 +98,9 @@ vm.qmp_log('blockdev-backup', sync='full', device='source', target='target')
vm.shutdown()
-print('\n=== backup-top should be gone after job-finalize ===\n')
+print('\n=== copy-before-write filter should be gone after job-finalize ===\n')
-# Check that the backup-top node is gone after job-finalize.
-#
-# During finalization, the node becomes inactive and can no longer
-# function. If it is still present, new parents might be attached, and
-# there would be no meaningful way to handle their I/O requests.
+# Check that the copy-before-write node is gone after job-finalize.
vm = iotests.VM()
vm.launch()
@@ -131,7 +126,7 @@ vm.qmp_log('blockdev-backup',
vm.event_wait('BLOCK_JOB_PENDING', 5.0)
-# The backup-top filter should still be present prior to finalization
+# The copy-before-write filter should still be present prior to finalization
assert vm.node_info('backup-filter') is not None
vm.qmp_log('job-finalize', id='backup')
diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out
index 97e62a4c94..e08f807dab 100644
--- a/tests/qemu-iotests/283.out
+++ b/tests/qemu-iotests/283.out
@@ -5,9 +5,9 @@
{"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
{"return": {}}
{"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append backup-top filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
-=== backup-top should be gone after job-finalize ===
+=== copy-before-write filter should be gone after job-finalize ===
{"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name": "source"}}
{"return": {}}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 05/21] block: rename backup-top to copy-before-write
2021-05-17 6:44 ` [PATCH 05/21] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
@ 2021-05-17 16:05 ` Max Reitz
2021-05-17 19:42 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-17 16:05 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> We are going to convert backup_top to full featured public filter,
> which can be used in separate of backup job. Start from renaming from
> "how it used" to "what it does".
Is this safe? The name was externally visible in queries after all.
(I’m not saying it is unsafe, I just don’t know and would like to know
whether you’ve considered this already.)
(Regardless, renaming files and so on is fine, of course.)
> While updating comments in 283 iotest, drop and rephrase also things
> about ".active", as this field is now dropped, and filter doesn't have
> "inactive" mode.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/{backup-top.h => copy-before-write.h} | 28 +++---
> block/backup.c | 22 ++---
> block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
> MAINTAINERS | 4 +-
> block/meson.build | 2 +-
> tests/qemu-iotests/283 | 35 +++----
> tests/qemu-iotests/283.out | 4 +-
> 7 files changed, 95 insertions(+), 100 deletions(-)
> rename block/{backup-top.h => copy-before-write.h} (56%)
> rename block/{backup-top.c => copy-before-write.c} (62%)
[...]
> diff --git a/block/backup-top.c b/block/copy-before-write.c
> similarity index 62%
> rename from block/backup-top.c
> rename to block/copy-before-write.c
> index 425e3778be..40e91832d7 100644
> --- a/block/backup-top.c
> +++ b/block/copy-before-write.c
[...]
> @@ -32,25 +32,25 @@
[...]
> -static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
> - uint64_t bytes, BdrvRequestFlags flags)
> +static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
> + uint64_t bytes, BdrvRequestFlags flags)
I’m sure you noticed it, too, but cbw_cbw() is weird. Perhaps
cbw_do_cbw() at least?
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 05/21] block: rename backup-top to copy-before-write
2021-05-17 16:05 ` Max Reitz
@ 2021-05-17 19:42 ` Vladimir Sementsov-Ogievskiy
2021-05-18 9:37 ` Max Reitz
0 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 19:42 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 19:05, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> We are going to convert backup_top to full featured public filter,
>> which can be used in separate of backup job. Start from renaming from
>> "how it used" to "what it does".
>
> Is this safe? The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)
>
> (Regardless, renaming files and so on is fine, of course.)
Hmmm. I don't know.
It was visible yes.. But we've never documented it. And if someone depends on name of the format of the filter automatically inserted during backup job, it's a kind of "undocumented feature" use..
Another change I is changing child from backing to file in 11, from this point of view it's unsafe too. But ше even more reasonable than good name: having all public filters behave similar is a very good thing.
So, may be it a bit risky, but I think good interface worth that risk. And we always can say "sorry guys, but that was not documented, we didn't promise anything".
But I'm OK to go on with "backup-top" and "backing", is someone has strict opinion about this.
>
>> While updating comments in 283 iotest, drop and rephrase also things
>> about ".active", as this field is now dropped, and filter doesn't have
>> "inactive" mode.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/{backup-top.h => copy-before-write.h} | 28 +++---
>> block/backup.c | 22 ++---
>> block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
>> MAINTAINERS | 4 +-
>> block/meson.build | 2 +-
>> tests/qemu-iotests/283 | 35 +++----
>> tests/qemu-iotests/283.out | 4 +-
>> 7 files changed, 95 insertions(+), 100 deletions(-)
>> rename block/{backup-top.h => copy-before-write.h} (56%)
>> rename block/{backup-top.c => copy-before-write.c} (62%)
>
> [...]
>
>> diff --git a/block/backup-top.c b/block/copy-before-write.c
>> similarity index 62%
>> rename from block/backup-top.c
>> rename to block/copy-before-write.c
>> index 425e3778be..40e91832d7 100644
>> --- a/block/backup-top.c
>> +++ b/block/copy-before-write.c
>
> [...]
>
>> @@ -32,25 +32,25 @@
>
> [...]
>
>> -static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
>> - uint64_t bytes, BdrvRequestFlags flags)
>> +static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
>> + uint64_t bytes, BdrvRequestFlags flags)
>
> I’m sure you noticed it, too, but cbw_cbw() is weird. Perhaps cbw_do_cbw() at least?
>
OK. Maybe even cbw_do_copy_before_write()
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 05/21] block: rename backup-top to copy-before-write
2021-05-17 19:42 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-18 9:37 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 9:37 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 21:42, Vladimir Sementsov-Ogievskiy wrote:
> 17.05.2021 19:05, Max Reitz wrote:
>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>> We are going to convert backup_top to full featured public filter,
>>> which can be used in separate of backup job. Start from renaming from
>>> "how it used" to "what it does".
>>
>> Is this safe? The name was externally visible in queries after all.
>> (I’m not saying it is unsafe, I just don’t know and would like to know
>> whether you’ve considered this already.)
>>
>> (Regardless, renaming files and so on is fine, of course.)
>
> Hmmm. I don't know.
>
> It was visible yes.. But we've never documented it. And if someone
> depends on name of the format of the filter automatically inserted
> during backup job, it's a kind of "undocumented feature" use..
>
> Another change I is changing child from backing to file in 11, from this
> point of view it's unsafe too. But ше even more reasonable than good
> name: having all public filters behave similar is a very good thing.
>
> So, may be it a bit risky, but I think good interface worth that risk.
> And we always can say "sorry guys, but that was not documented, we
> didn't promise anything".
OK.
Max
> But I'm OK to go on with "backup-top" and "backing", is someone has
> strict opinion about this.
>
>>
>>> While updating comments in 283 iotest, drop and rephrase also things
>>> about ".active", as this field is now dropped, and filter doesn't have
>>> "inactive" mode.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> block/{backup-top.h => copy-before-write.h} | 28 +++---
>>> block/backup.c | 22 ++---
>>> block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
>>> MAINTAINERS | 4 +-
>>> block/meson.build | 2 +-
>>> tests/qemu-iotests/283 | 35 +++----
>>> tests/qemu-iotests/283.out | 4 +-
>>> 7 files changed, 95 insertions(+), 100 deletions(-)
>>> rename block/{backup-top.h => copy-before-write.h} (56%)
>>> rename block/{backup-top.c => copy-before-write.c} (62%)
>>
>> [...]
>>
>>> diff --git a/block/backup-top.c b/block/copy-before-write.c
>>> similarity index 62%
>>> rename from block/backup-top.c
>>> rename to block/copy-before-write.c
>>> index 425e3778be..40e91832d7 100644
>>> --- a/block/backup-top.c
>>> +++ b/block/copy-before-write.c
>>
>> [...]
>>
>>> @@ -32,25 +32,25 @@
>>
>> [...]
>>
>>> -static coroutine_fn int backup_top_cbw(BlockDriverState *bs,
>>> uint64_t offset,
>>> - uint64_t bytes,
>>> BdrvRequestFlags flags)
>>> +static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
>>> + uint64_t bytes, BdrvRequestFlags flags)
>>
>> I’m sure you noticed it, too, but cbw_cbw() is weird. Perhaps
>> cbw_do_cbw() at least?
>>
>
> OK. Maybe even cbw_do_copy_before_write()
>
>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 06/21] block/backup: drop support for copy_range
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (5 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 05/21] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 16:20 ` Max Reitz
2021-05-17 6:44 ` [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag Vladimir Sementsov-Ogievskiy
` (14 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
copy_range is not a default behavior since 6a30f663d4c0b3c, and it's
now available only though x-perf experimantal argument, so it's OK to
drop it.
Even when backup is used to copy disk to same filesystem, and
filesystem supports zero-copy copy_range, copy_range is probably not
what we want for backup: backup has good property of making a copy of
active disk, with no impact to active disk itself (unlike creating a
snapshot). And if copy_range instead of copying data adds fs-level
references, and on next guest write COW operation occurs, it's seems
most possible, that new block will be allocated for original vm disk,
not for backup disk. Thus, fragmentation of original disk will
increase.
We can simply add support back on demand. Now we want to publish
copy-before-write filter, and instead of thinking how to pass
use-copy-range argument to block-copy (create x-block-copy parameter
for new public filter driver, or may be set it by hand after filter
node creation?), instead of this let's just drop copy-range support in
backup for now.
After this patch copy-range support in block-copy becomes unused. Let's
keep it for a while, it won't hurt:
1. If there would be request for supporting copy_range in backup
(and/or in a new public copy-before-write filter), it will be easy
to satisfy it.
2. Probably, qemu-img convert will reuse block-copy, and qemu-img has
option to enable copy-range. qemu-img convert is not a backup, and
copy_range may be more reasonable for some cases in context of
qemu-img convert.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.h | 1 -
block/backup.c | 3 +--
block/copy-before-write.c | 4 +---
3 files changed, 2 insertions(+), 6 deletions(-)
diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index 5977b7aa31..e284dfb6a7 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -33,7 +33,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
- BackupPerf *perf,
BdrvRequestFlags write_flags,
BlockCopyState **bcs,
Error **errp);
diff --git a/block/backup.c b/block/backup.c
index 0b5887269c..7c7a88a0f7 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -522,8 +522,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
(compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
cbw = bdrv_cbw_append(bs, target, filter_node_name,
- cluster_size, perf,
- write_flags, &bcs, errp);
+ cluster_size, write_flags, &bcs, errp);
if (!cbw) {
goto error;
}
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 40e91832d7..fe087a7347 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -170,7 +170,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
- BackupPerf *perf,
BdrvRequestFlags write_flags,
BlockCopyState **bcs,
Error **errp)
@@ -217,8 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
state->cluster_size = cluster_size;
state->bcs = block_copy_state_new(top->backing, state->target,
- cluster_size, perf->use_copy_range,
- write_flags, errp);
+ cluster_size, false, write_flags, errp);
if (!state->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 06/21] block/backup: drop support for copy_range
2021-05-17 6:44 ` [PATCH 06/21] block/backup: drop support for copy_range Vladimir Sementsov-Ogievskiy
@ 2021-05-17 16:20 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-17 16:20 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> copy_range is not a default behavior since 6a30f663d4c0b3c, and it's
> now available only though x-perf experimantal argument, so it's OK to
> drop it.
>
> Even when backup is used to copy disk to same filesystem, and
> filesystem supports zero-copy copy_range, copy_range is probably not
> what we want for backup: backup has good property of making a copy of
> active disk, with no impact to active disk itself (unlike creating a
> snapshot). And if copy_range instead of copying data adds fs-level
> references, and on next guest write COW operation occurs, it's seems
> most possible, that new block will be allocated for original vm disk,
> not for backup disk. Thus, fragmentation of original disk will
> increase.
Good point.
> We can simply add support back on demand. Now we want to publish
> copy-before-write filter, and instead of thinking how to pass
> use-copy-range argument to block-copy (create x-block-copy parameter
> for new public filter driver, or may be set it by hand after filter
> node creation?), instead of this let's just drop copy-range support in
> backup for now.
>
> After this patch copy-range support in block-copy becomes unused. Let's
> keep it for a while, it won't hurt:
>
> 1. If there would be request for supporting copy_range in backup
> (and/or in a new public copy-before-write filter), it will be easy
> to satisfy it.
>
> 2. Probably, qemu-img convert will reuse block-copy, and qemu-img has
> option to enable copy-range. qemu-img convert is not a backup, and
> copy_range may be more reasonable for some cases in context of
> qemu-img convert.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.h | 1 -
> block/backup.c | 3 +--
> block/copy-before-write.c | 4 +---
> 3 files changed, 2 insertions(+), 6 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (6 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 06/21] block/backup: drop support for copy_range Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 16:46 ` Max Reitz
2021-05-17 6:44 ` [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size() Vladimir Sementsov-Ogievskiy
` (13 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
It won't hurt in common case, so let's not bother with detecting image
fleecing.
Also, we want to simplify initialization interface of copy-before-write
filter as we are going to make it public.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.h | 2 +-
include/block/block-copy.h | 3 +--
block/backup.c | 20 +-------------------
block/block-copy.c | 29 ++++++++++++++++++++++++++---
block/copy-before-write.c | 4 ++--
5 files changed, 31 insertions(+), 27 deletions(-)
diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index e284dfb6a7..538aab8bdb 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -33,7 +33,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
- BdrvRequestFlags write_flags,
+ bool compress,
BlockCopyState **bcs,
Error **errp);
void bdrv_cbw_drop(BlockDriverState *bs);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 338f2ea7fd..c013a20e1e 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -24,8 +24,7 @@ typedef struct BlockCopyCallState BlockCopyCallState;
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
int64_t cluster_size, bool use_copy_range,
- BdrvRequestFlags write_flags,
- Error **errp);
+ bool compress, Error **errp);
void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
diff --git a/block/backup.c b/block/backup.c
index 7c7a88a0f7..fe685e411b 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -407,7 +407,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
int64_t len, target_len;
BackupBlockJob *job = NULL;
int64_t cluster_size;
- BdrvRequestFlags write_flags;
BlockDriverState *cbw = NULL;
BlockCopyState *bcs = NULL;
@@ -504,25 +503,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- /*
- * If source is in backing chain of target assume that target is going to be
- * used for "image fleecing", i.e. it should represent a kind of snapshot of
- * source at backup-start point in time. And target is going to be read by
- * somebody (for example, used as NBD export) during backup job.
- *
- * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid
- * intersection of backup writes and third party reads from target,
- * otherwise reading from target we may occasionally read already updated by
- * guest data.
- *
- * For more information see commit f8d59dfb40bb and test
- * tests/qemu-iotests/222
- */
- write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) |
- (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
-
cbw = bdrv_cbw_append(bs, target, filter_node_name,
- cluster_size, write_flags, &bcs, errp);
+ cluster_size, compress, &bcs, errp);
if (!cbw) {
goto error;
}
diff --git a/block/block-copy.c b/block/block-copy.c
index 9b4af00614..daa1a2bf9f 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -245,7 +245,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
int64_t cluster_size, bool use_copy_range,
- BdrvRequestFlags write_flags, Error **errp)
+ bool compress, Error **errp)
{
BlockCopyState *s;
BdrvDirtyBitmap *copy_bitmap;
@@ -257,6 +257,28 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
}
bdrv_disable_dirty_bitmap(copy_bitmap);
+ /*
+ * Why we always set BDRV_REQ_SERIALISING write flag:
+ *
+ * Assume source is in backing chain of target assume that target is going
+ * to be used for "image fleecing", i.e. it should represent a kind of
+ * snapshot of source at backup-start point in time. And target is going to
+ * be read by somebody (for example, used as NBD export) during backup job.
+ *
+ * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid
+ * intersection of backup writes and third party reads from target,
+ * otherwise reading from target we may occasionally read already updated by
+ * guest data.
+ *
+ * For more information see commit f8d59dfb40bb and test
+ * tests/qemu-iotests/222
+ *
+ * Other cases? The only known reasonable case is "just copy to target, and
+ * target is not used for something else". In this case BDRV_REQ_SERIALISING
+ * change nothing, so let's not bother with detecting the "image fleecing"
+ * case and enabling BDRV_REQ_SERIALISING only for it.
+ */
+
s = g_new(BlockCopyState, 1);
*s = (BlockCopyState) {
.source = source,
@@ -264,7 +286,8 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
.copy_bitmap = copy_bitmap,
.cluster_size = cluster_size,
.len = bdrv_dirty_bitmap_size(copy_bitmap),
- .write_flags = write_flags,
+ .write_flags = BDRV_REQ_SERIALISING |
+ (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
.mem = shres_create(BLOCK_COPY_MAX_MEM),
};
@@ -277,7 +300,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
*/
s->use_copy_range = false;
s->copy_size = cluster_size;
- } else if (write_flags & BDRV_REQ_WRITE_COMPRESSED) {
+ } else if (compress) {
/* Compression supports only cluster-size writes and no copy-range. */
s->use_copy_range = false;
s->copy_size = cluster_size;
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index fe087a7347..19472912d1 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -170,7 +170,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
uint64_t cluster_size,
- BdrvRequestFlags write_flags,
+ bool compress,
BlockCopyState **bcs,
Error **errp)
{
@@ -216,7 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
state->cluster_size = cluster_size;
state->bcs = block_copy_state_new(top->backing, state->target,
- cluster_size, false, write_flags, errp);
+ cluster_size, false, compress, errp);
if (!state->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag
2021-05-17 6:44 ` [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag Vladimir Sementsov-Ogievskiy
@ 2021-05-17 16:46 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-17 16:46 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> It won't hurt in common case, so let's not bother with detecting image
> fleecing.
>
> Also, we want to simplify initialization interface of copy-before-write
> filter as we are going to make it public.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.h | 2 +-
> include/block/block-copy.h | 3 +--
> block/backup.c | 20 +-------------------
> block/block-copy.c | 29 ++++++++++++++++++++++++++---
> block/copy-before-write.c | 4 ++--
> 5 files changed, 31 insertions(+), 27 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (7 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 16:57 ` Max Reitz
2021-05-17 6:44 ` [PATCH 09/21] block/backup: move cluster size calculation to block-copy Vladimir Sementsov-Ogievskiy
` (12 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's
just error-out, it's simpler and safer.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/backup.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index fe685e411b..fe7a1f1e37 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
* targets with a backing file, try to avoid COW if possible.
*/
ret = bdrv_get_info(target, &bdi);
- if (ret == -ENOTSUP && !target_does_cow) {
+ if (ret < 0 && ret != -ENOTSUP) {
+ error_setg_errno(errp, -ret, "Failed to get target info");
+ return ret;
+ } else if (ret == -ENOTSUP && !target_does_cow) {
/* Cluster size is not defined */
warn_report("The target block device doesn't provide "
"information about the block size and it doesn't have a "
@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
"this default, the backup may be unusable",
BACKUP_CLUSTER_SIZE_DEFAULT);
return BACKUP_CLUSTER_SIZE_DEFAULT;
- } else if (ret < 0 && !target_does_cow) {
- error_setg_errno(errp, -ret,
- "Couldn't determine the cluster size of the target image, "
- "which has no backing file");
- error_append_hint(errp,
- "Aborting, since this may create an unusable destination image\n");
- return ret;
- } else if (ret < 0 && target_does_cow) {
+ } else if (ret == -ENOTSUP && target_does_cow) {
/* Not fatal; just trudge on ahead. */
return BACKUP_CLUSTER_SIZE_DEFAULT;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()
2021-05-17 6:44 ` [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size() Vladimir Sementsov-Ogievskiy
@ 2021-05-17 16:57 ` Max Reitz
2021-05-17 19:53 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-17 16:57 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's
> just error-out, it's simpler and safer.
Hm, doesn’t look that much simpler to me. Not sure how much safer it
is, because the point was that in the target_does_cow case, we would
like a cluster size hint, but it isn’t necessary. So if we don’t get
one, regardless of the reason, we use the default cluster size. I don’t
know why ENOTSUP should be treated in a special way there.
So I don’t know.
Max
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/backup.c | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/block/backup.c b/block/backup.c
> index fe685e411b..fe7a1f1e37 100644
> --- a/block/backup.c
> +++ b/block/backup.c
> @@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
> * targets with a backing file, try to avoid COW if possible.
> */
> ret = bdrv_get_info(target, &bdi);
> - if (ret == -ENOTSUP && !target_does_cow) {
> + if (ret < 0 && ret != -ENOTSUP) {
> + error_setg_errno(errp, -ret, "Failed to get target info");
> + return ret;
> + } else if (ret == -ENOTSUP && !target_does_cow) {
> /* Cluster size is not defined */
> warn_report("The target block device doesn't provide "
> "information about the block size and it doesn't have a "
> @@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
> "this default, the backup may be unusable",
> BACKUP_CLUSTER_SIZE_DEFAULT);
> return BACKUP_CLUSTER_SIZE_DEFAULT;
> - } else if (ret < 0 && !target_does_cow) {
> - error_setg_errno(errp, -ret,
> - "Couldn't determine the cluster size of the target image, "
> - "which has no backing file");
> - error_append_hint(errp,
> - "Aborting, since this may create an unusable destination image\n");
> - return ret;
> - } else if (ret < 0 && target_does_cow) {
> + } else if (ret == -ENOTSUP && target_does_cow) {
> /* Not fatal; just trudge on ahead. */
> return BACKUP_CLUSTER_SIZE_DEFAULT;
> }
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()
2021-05-17 16:57 ` Max Reitz
@ 2021-05-17 19:53 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 19:53 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
17.05.2021 19:57, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's
>> just error-out, it's simpler and safer.
>
> Hm, doesn’t look that much simpler to me. Not sure how much safer it is, because the point was that in the target_does_cow case, we would like a cluster size hint, but it isn’t necessary. So if we don’t get one, regardless of the reason, we use the default cluster size. I don’t know why ENOTSUP should be treated in a special way there.
>
> So I don’t know.
>
I'm probably OK to drop this for now and don't care. Still, I can share what brings me to this:
First I thought that cluster size should be easily available for any driver:
protocol drivers and not-backing-supporting format drivers can set it to 1 or to request_alignment, if they don't have a "cluster" in mind.
backing-supporting format drivers should of course provide actual cluster size
And I decided to just add bs->cluster_size variable, set on driver open, to simplify the whole thing and make it clean. Then, most this detect-cluster-size function would be just dropped.
But it occurs, that there is one driver, that has a good and rather tricky reason for ENOTSUP: vmdk can have several extents with different cluster size..
So I give up refactored, and finished with this one patch. It can be simply dropped, I am not really a fan of it..
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/backup.c | 14 +++++---------
>> 1 file changed, 5 insertions(+), 9 deletions(-)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index fe685e411b..fe7a1f1e37 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
>> * targets with a backing file, try to avoid COW if possible.
>> */
>> ret = bdrv_get_info(target, &bdi);
>> - if (ret == -ENOTSUP && !target_does_cow) {
>> + if (ret < 0 && ret != -ENOTSUP) {
>> + error_setg_errno(errp, -ret, "Failed to get target info");
>> + return ret;
>> + } else if (ret == -ENOTSUP && !target_does_cow) {
>> /* Cluster size is not defined */
>> warn_report("The target block device doesn't provide "
>> "information about the block size and it doesn't have a "
>> @@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
>> "this default, the backup may be unusable",
>> BACKUP_CLUSTER_SIZE_DEFAULT);
>> return BACKUP_CLUSTER_SIZE_DEFAULT;
>> - } else if (ret < 0 && !target_does_cow) {
>> - error_setg_errno(errp, -ret,
>> - "Couldn't determine the cluster size of the target image, "
>> - "which has no backing file");
>> - error_append_hint(errp,
>> - "Aborting, since this may create an unusable destination image\n");
>> - return ret;
>> - } else if (ret < 0 && target_does_cow) {
>> + } else if (ret == -ENOTSUP && target_does_cow) {
>> /* Not fatal; just trudge on ahead. */
>> return BACKUP_CLUSTER_SIZE_DEFAULT;
>> }
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 09/21] block/backup: move cluster size calculation to block-copy
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (8 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size() Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-17 17:09 ` Max Reitz
2021-05-17 6:44 ` [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
` (11 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
The main consumer of cluster-size is block-copy. Let's calculate it
here instead of passing through backup-top.
We are going to publish copy-before-write filter soon, so it will be
created through options. But we don't want for now to make explicit
option for cluster-size, let's continue to calculate it automatically.
So, now is the time to get rid of cluster_size argument for
bdrv_cbw_append().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.h | 1 -
include/block/block-copy.h | 5 ++--
block/backup.c | 58 ++++++--------------------------------
block/block-copy.c | 47 +++++++++++++++++++++++++++++-
block/copy-before-write.c | 10 +++----
5 files changed, 62 insertions(+), 59 deletions(-)
diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index 538aab8bdb..b386fd8f01 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -32,7 +32,6 @@
BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
- uint64_t cluster_size,
bool compress,
BlockCopyState **bcs,
Error **errp);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index c013a20e1e..8138686eb4 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -23,8 +23,8 @@ typedef struct BlockCopyState BlockCopyState;
typedef struct BlockCopyCallState BlockCopyCallState;
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size, bool use_copy_range,
- bool compress, Error **errp);
+ bool use_copy_range, bool compress,
+ Error **errp);
void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
@@ -86,6 +86,7 @@ void block_copy_kick(BlockCopyCallState *call_state);
void block_copy_call_cancel(BlockCopyCallState *call_state);
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
+int64_t block_copy_cluster_size(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
#endif /* BLOCK_COPY_H */
diff --git a/block/backup.c b/block/backup.c
index fe7a1f1e37..90cca1ca30 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -29,8 +29,6 @@
#include "block/copy-before-write.h"
-#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
-
typedef struct BackupBlockJob {
BlockJob common;
BlockDriverState *cbw;
@@ -354,39 +352,6 @@ static const BlockJobDriver backup_job_driver = {
.set_speed = backup_set_speed,
};
-static int64_t backup_calculate_cluster_size(BlockDriverState *target,
- Error **errp)
-{
- int ret;
- BlockDriverInfo bdi;
- bool target_does_cow = bdrv_backing_chain_next(target);
-
- /*
- * If there is no backing file on the target, we cannot rely on COW if our
- * backup cluster size is smaller than the target cluster size. Even for
- * targets with a backing file, try to avoid COW if possible.
- */
- ret = bdrv_get_info(target, &bdi);
- if (ret < 0 && ret != -ENOTSUP) {
- error_setg_errno(errp, -ret, "Failed to get target info");
- return ret;
- } else if (ret == -ENOTSUP && !target_does_cow) {
- /* Cluster size is not defined */
- warn_report("The target block device doesn't provide "
- "information about the block size and it doesn't have a "
- "backing file. The default block size of %u bytes is "
- "used. If the actual block size of the target exceeds "
- "this default, the backup may be unusable",
- BACKUP_CLUSTER_SIZE_DEFAULT);
- return BACKUP_CLUSTER_SIZE_DEFAULT;
- } else if (ret == -ENOTSUP && target_does_cow) {
- /* Not fatal; just trudge on ahead. */
- return BACKUP_CLUSTER_SIZE_DEFAULT;
- }
-
- return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
-}
-
BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
@@ -444,11 +409,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- cluster_size = backup_calculate_cluster_size(target, errp);
- if (cluster_size < 0) {
- goto error;
- }
-
if (perf->max_workers < 1) {
error_setg(errp, "max-workers must be greater than zero");
return NULL;
@@ -460,13 +420,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- if (perf->max_chunk && perf->max_chunk < cluster_size) {
- error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
- "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
- return NULL;
- }
-
-
if (sync_bitmap) {
/* If we need to write to this bitmap, check that we can: */
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -499,12 +452,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
- cbw = bdrv_cbw_append(bs, target, filter_node_name,
- cluster_size, compress, &bcs, errp);
+ cbw = bdrv_cbw_append(bs, target, filter_node_name, compress, &bcs, errp);
if (!cbw) {
goto error;
}
+ cluster_size = block_copy_cluster_size(bcs);
+
+ if (perf->max_chunk && perf->max_chunk < cluster_size) {
+ error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+ "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+ goto error;
+ }
+
/* job->len is fixed, so we can't allow resize */
job = block_job_create(job_id, &backup_job_driver, txn, cbw,
0, BLK_PERM_ALL,
diff --git a/block/block-copy.c b/block/block-copy.c
index daa1a2bf9f..9020234c6e 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -27,6 +27,7 @@
#define BLOCK_COPY_MAX_MEM (128 * MiB)
#define BLOCK_COPY_MAX_WORKERS 64
#define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */
+#define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16)
static coroutine_fn int block_copy_task_entry(AioTask *task);
@@ -243,13 +244,52 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)
target->bs->bl.max_transfer));
}
+static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
+ Error **errp)
+{
+ int ret;
+ BlockDriverInfo bdi;
+ bool target_does_cow = bdrv_backing_chain_next(target);
+
+ /*
+ * If there is no backing file on the target, we cannot rely on COW if our
+ * backup cluster size is smaller than the target cluster size. Even for
+ * targets with a backing file, try to avoid COW if possible.
+ */
+ ret = bdrv_get_info(target, &bdi);
+ if (ret < 0 && ret != -ENOTSUP) {
+ error_setg_errno(errp, -ret, "Failed to get target info");
+ return ret;
+ } else if (ret == -ENOTSUP && !target_does_cow) {
+ /* Cluster size is not defined */
+ warn_report("The target block device doesn't provide "
+ "information about the block size and it doesn't have a "
+ "backing file. The default block size of %u bytes is "
+ "used. If the actual block size of the target exceeds "
+ "this default, the backup may be unusable",
+ BLOCK_COPY_CLUSTER_SIZE_DEFAULT);
+ return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+ } else if (ret == -ENOTSUP && target_does_cow) {
+ /* Not fatal; just trudge on ahead. */
+ return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+ }
+
+ return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
+}
+
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int64_t cluster_size, bool use_copy_range,
+ bool use_copy_range,
bool compress, Error **errp)
{
BlockCopyState *s;
+ int64_t cluster_size;
BdrvDirtyBitmap *copy_bitmap;
+ cluster_size = block_copy_calculate_cluster_size(target->bs, errp);
+ if (cluster_size < 0) {
+ return NULL;
+ }
+
copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL,
errp);
if (!copy_bitmap) {
@@ -843,6 +883,11 @@ BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
return s->copy_bitmap;
}
+int64_t block_copy_cluster_size(BlockCopyState *s)
+{
+ return s->cluster_size;
+}
+
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
{
s->skip_unallocated = skip;
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 19472912d1..4776172f77 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -37,7 +37,6 @@
typedef struct BDRVCopyBeforeWriteState {
BlockCopyState *bcs;
BdrvChild *target;
- int64_t cluster_size;
} BDRVCopyBeforeWriteState;
static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
{
BDRVCopyBeforeWriteState *s = bs->opaque;
uint64_t off, end;
+ int64_t cluster_size = block_copy_cluster_size(s->bcs);
if (flags & BDRV_REQ_WRITE_UNCHANGED) {
return 0;
}
- off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
- end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
+ off = QEMU_ALIGN_DOWN(offset, cluster_size);
+ end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
return block_copy(s->bcs, off, end - off, true);
}
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {
BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
BlockDriverState *target,
const char *filter_node_name,
- uint64_t cluster_size,
bool compress,
BlockCopyState **bcs,
Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
}
appended = true;
- state->cluster_size = cluster_size;
state->bcs = block_copy_state_new(top->backing, state->target,
- cluster_size, false, compress, errp);
+ false, compress, errp);
if (!state->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 09/21] block/backup: move cluster size calculation to block-copy
2021-05-17 6:44 ` [PATCH 09/21] block/backup: move cluster size calculation to block-copy Vladimir Sementsov-Ogievskiy
@ 2021-05-17 17:09 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-17 17:09 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> The main consumer of cluster-size is block-copy. Let's calculate it
> here instead of passing through backup-top.
>
> We are going to publish copy-before-write filter soon, so it will be
> created through options. But we don't want for now to make explicit
> option for cluster-size, let's continue to calculate it automatically.
> So, now is the time to get rid of cluster_size argument for
> bdrv_cbw_append().
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.h | 1 -
> include/block/block-copy.h | 5 ++--
> block/backup.c | 58 ++++++--------------------------------
> block/block-copy.c | 47 +++++++++++++++++++++++++++++-
> block/copy-before-write.c | 10 +++----
> 5 files changed, 62 insertions(+), 59 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (9 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 09/21] block/backup: move cluster size calculation to block-copy Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 11:10 ` Max Reitz
2021-05-17 6:44 ` [PATCH 11/21] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
` (10 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
We are going to publish copy-before-write filter. So, user should be
able to create it with blockdev-add first, specifying both filtered and
target children. And then do blockdev-reopen, to actually insert the
filter where needed.
Currently, filter unshares write permission unconditionally on source
node. It's good, but it will not allow to do blockdev-add. So, let's
relax restrictions when filter doesn't have any parent.
Test output is modified, as now permission conflict happens only when
job creates a blk parent for filter node.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 8 +++++---
tests/qemu-iotests/283.out | 2 +-
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 4776172f77..af2bb97a30 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
bdrv_default_perms(bs, c, role, reopen_queue,
perm, shared, nperm, nshared);
- if (perm & BLK_PERM_WRITE) {
- *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+ if (!QLIST_EMPTY(&bs->parents)) {
+ if (perm & BLK_PERM_WRITE) {
+ *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+ }
+ *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
}
- *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
}
}
diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out
index e08f807dab..d5350ce7a7 100644
--- a/tests/qemu-iotests/283.out
+++ b/tests/qemu-iotests/283.out
@@ -5,7 +5,7 @@
{"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
{"return": {}}
{"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}
=== copy-before-write filter should be gone after job-finalize ===
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents
2021-05-17 6:44 ` [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
@ 2021-05-18 11:10 ` Max Reitz
2021-05-18 12:16 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 11:10 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> We are going to publish copy-before-write filter. So, user should be
> able to create it with blockdev-add first, specifying both filtered and
> target children. And then do blockdev-reopen, to actually insert the
> filter where needed.
>
> Currently, filter unshares write permission unconditionally on source
> node. It's good, but it will not allow to do blockdev-add. So, let's
> relax restrictions when filter doesn't have any parent.
>
> Test output is modified, as now permission conflict happens only when
> job creates a blk parent for filter node.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 8 +++++---
> tests/qemu-iotests/283.out | 2 +-
> 2 files changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
> index 4776172f77..af2bb97a30 100644
> --- a/block/copy-before-write.c
> +++ b/block/copy-before-write.c
> @@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
> bdrv_default_perms(bs, c, role, reopen_queue,
> perm, shared, nperm, nshared);
>
> - if (perm & BLK_PERM_WRITE) {
> - *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
> + if (!QLIST_EMPTY(&bs->parents)) {
I understand this works because with the transactional system, at the
time the permissions are checked, the graph has already been changed, yes?
I was wondering still whether there was any way to express this through
permissions alone. I guess we could check
“perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a
crutch to see whether there are any parents. I suppose this really is
about whether there are parents or not. So:
Reviewed-by: Max Reitz <mreitz@redhat.com>
> + if (perm & BLK_PERM_WRITE) {
> + *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
> + }
> + *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
> }
> - *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
> }
> }
>
> diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out
> index e08f807dab..d5350ce7a7 100644
> --- a/tests/qemu-iotests/283.out
> +++ b/tests/qemu-iotests/283.out
> @@ -5,7 +5,7 @@
> {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
> {"return": {}}
> {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
> -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
> +{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}
>
> === copy-before-write filter should be gone after job-finalize ===
>
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents
2021-05-18 11:10 ` Max Reitz
@ 2021-05-18 12:16 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 12:16 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 14:10, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> We are going to publish copy-before-write filter. So, user should be
>> able to create it with blockdev-add first, specifying both filtered and
>> target children. And then do blockdev-reopen, to actually insert the
>> filter where needed.
>>
>> Currently, filter unshares write permission unconditionally on source
>> node. It's good, but it will not allow to do blockdev-add. So, let's
>> relax restrictions when filter doesn't have any parent.
>>
>> Test output is modified, as now permission conflict happens only when
>> job creates a blk parent for filter node.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/copy-before-write.c | 8 +++++---
>> tests/qemu-iotests/283.out | 2 +-
>> 2 files changed, 6 insertions(+), 4 deletions(-)
>>
>> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
>> index 4776172f77..af2bb97a30 100644
>> --- a/block/copy-before-write.c
>> +++ b/block/copy-before-write.c
>> @@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
>> bdrv_default_perms(bs, c, role, reopen_queue,
>> perm, shared, nperm, nshared);
>> - if (perm & BLK_PERM_WRITE) {
>> - *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
>> + if (!QLIST_EMPTY(&bs->parents)) {
>
> I understand this works because with the transactional system, at the time the permissions are checked, the graph has already been changed, yes?
Right
>
> I was wondering still whether there was any way to express this through permissions alone. I guess we could check
> “perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a crutch to see whether there are any parents. I suppose this really is about whether there are parents or not. So:
It's a workaround for blockdev-add to work. It's a workaround, because it would be better for the filter to never share WRITE. But this way we'll have to make an interface to transactionally create and insert filter. Probably we will at some point, than this workaround could be dropped. So, if we have to special-case a situation when there no parents, let's check exactly this, to be more strict.
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
>> + if (perm & BLK_PERM_WRITE) {
>> + *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
>> + }
>> + *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
>> }
>> - *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
>> }
>> }
>> diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out
>> index e08f807dab..d5350ce7a7 100644
>> --- a/tests/qemu-iotests/283.out
>> +++ b/tests/qemu-iotests/283.out
>> @@ -5,7 +5,7 @@
>> {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
>> {"return": {}}
>> {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
>> -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
>> +{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}
>> === copy-before-write filter should be gone after job-finalize ===
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 11/21] block/copy-before-write: use file child instead of backing
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (10 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 11:47 ` Max Reitz
2021-05-17 6:44 ` [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
` (9 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
We are going to publish copy-before-write filter, and there no public
backing-child-based filter in Qemu. No reason to create a precedent, so
let's refactor copy-before-write filter instead.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------
1 file changed, 23 insertions(+), 17 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index af2bb97a30..2f06a524b8 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -43,7 +43,7 @@ static coroutine_fn int cbw_co_preadv(
BlockDriverState *bs, uint64_t offset, uint64_t bytes,
QEMUIOVector *qiov, int flags)
{
- return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);
+ return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
}
static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
@@ -71,7 +71,7 @@ static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
return ret;
}
- return bdrv_co_pdiscard(bs->backing, offset, bytes);
+ return bdrv_co_pdiscard(bs->file, offset, bytes);
}
static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
@@ -82,7 +82,7 @@ static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
return ret;
}
- return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
+ return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags);
}
static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
@@ -95,29 +95,22 @@ static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
return ret;
}
- return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags);
+ return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags);
}
static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
{
- if (!bs->backing) {
+ if (!bs->file) {
return 0;
}
- return bdrv_co_flush(bs->backing->bs);
+ return bdrv_co_flush(bs->file->bs);
}
static void cbw_refresh_filename(BlockDriverState *bs)
{
- if (bs->backing == NULL) {
- /*
- * we can be here after failed bdrv_attach_child in
- * bdrv_set_backing_hd
- */
- return;
- }
pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
- bs->backing->bs->filename);
+ bs->file->bs->filename);
}
static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
@@ -186,6 +179,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name,
BDRV_O_RDWR, errp);
if (!top) {
+ error_prepend(errp, "Cannot open driver: ");
return NULL;
}
@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
BDRV_CHILD_DATA, errp);
if (!state->target) {
+ error_prepend(errp, "Cannot attach target child: ");
bdrv_unref(target);
bdrv_unref(top);
return NULL;
}
+ bdrv_ref(source);
+ top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+ BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+ errp);
+ if (!top->file) {
+ error_prepend(errp, "Cannot attach file child: ");
+ bdrv_unref(source);
+ bdrv_unref(top);
+ return NULL;
+ }
+
bdrv_drained_begin(source);
- ret = bdrv_append(top, source, errp);
+ ret = bdrv_replace_node(source, top, errp);
if (ret < 0) {
error_prepend(errp, "Cannot append copy-before-write filter: ");
goto fail;
}
appended = true;
- state->bcs = block_copy_state_new(top->backing, state->target,
- false, compress, errp);
+ state->bcs = block_copy_state_new(top->file, state->target, false, compress,
+ errp);
if (!state->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing
2021-05-17 6:44 ` [PATCH 11/21] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
@ 2021-05-18 11:47 ` Max Reitz
2021-05-18 12:21 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 11:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> We are going to publish copy-before-write filter, and there no public
> backing-child-based filter in Qemu. No reason to create a precedent, so
> let's refactor copy-before-write filter instead.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------
> 1 file changed, 23 insertions(+), 17 deletions(-)
>
> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
> index af2bb97a30..2f06a524b8 100644
> --- a/block/copy-before-write.c
> +++ b/block/copy-before-write.c
[...]
> @@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
> state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
> BDRV_CHILD_DATA, errp);
> if (!state->target) {
> + error_prepend(errp, "Cannot attach target child: ");
> bdrv_unref(target);
> bdrv_unref(top);
> return NULL;
> }
>
> + bdrv_ref(source);
> + top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
> + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
> + errp);
> + if (!top->file) {
> + error_prepend(errp, "Cannot attach file child: ");
> + bdrv_unref(source);
Already existing in the code above, but why is the reference to @source
dropped here? Shouldn’t bdrv_attach_child()’s error path have done that
already?
Max
> + bdrv_unref(top);
> + return NULL;
> + }
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing
2021-05-18 11:47 ` Max Reitz
@ 2021-05-18 12:21 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 12:21 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 14:47, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> We are going to publish copy-before-write filter, and there no public
>> backing-child-based filter in Qemu. No reason to create a precedent, so
>> let's refactor copy-before-write filter instead.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------
>> 1 file changed, 23 insertions(+), 17 deletions(-)
>>
>> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
>> index af2bb97a30..2f06a524b8 100644
>> --- a/block/copy-before-write.c
>> +++ b/block/copy-before-write.c
>
> [...]
>
>> @@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
>> state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
>> BDRV_CHILD_DATA, errp);
>> if (!state->target) {
>> + error_prepend(errp, "Cannot attach target child: ");
>> bdrv_unref(target);
>> bdrv_unref(top);
>> return NULL;
>> }
>> + bdrv_ref(source);
>> + top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
>> + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
>> + errp);
>> + if (!top->file) {
>> + error_prepend(errp, "Cannot attach file child: ");
>> + bdrv_unref(source);
>
> Already existing in the code above, but why is the reference to @source dropped here? Shouldn’t bdrv_attach_child()’s error path have done that already?
>
Hmm, yes. Looks like a preexisting bug. will fix and update this patch.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (11 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 11/21] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 12:35 ` Max Reitz
2021-05-17 6:44 ` [PATCH 13/21] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
` (8 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Refactor the function to replace child at last. Thus we don't need to
revert it and code is simplified.
block-copy state initilization being done before replacing the child
doesn't need any drained section.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 33 +++++++++++----------------------
1 file changed, 11 insertions(+), 22 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 2f06a524b8..35a9aa163d 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -172,7 +172,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
int ret;
BDRVCopyBeforeWriteState *state;
BlockDriverState *top;
- bool appended = false;
assert(source->total_sectors == target->total_sectors);
@@ -197,8 +196,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
if (!state->target) {
error_prepend(errp, "Cannot attach target child: ");
bdrv_unref(target);
- bdrv_unref(top);
- return NULL;
+ goto fail;
}
bdrv_ref(source);
@@ -208,18 +206,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
if (!top->file) {
error_prepend(errp, "Cannot attach file child: ");
bdrv_unref(source);
- bdrv_unref(top);
- return NULL;
- }
-
- bdrv_drained_begin(source);
-
- ret = bdrv_replace_node(source, top, errp);
- if (ret < 0) {
- error_prepend(errp, "Cannot append copy-before-write filter: ");
goto fail;
}
- appended = true;
state->bcs = block_copy_state_new(top->file, state->target, false, compress,
errp);
@@ -227,21 +215,22 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
error_prepend(errp, "Cannot create block-copy-state: ");
goto fail;
}
- *bcs = state->bcs;
+ bdrv_drained_begin(source);
+ ret = bdrv_replace_node(source, top, errp);
bdrv_drained_end(source);
+ if (ret < 0) {
+ error_prepend(errp, "Cannot append copy-before-write filter: ");
+ goto fail;
+ }
+
+ *bcs = state->bcs;
return top;
fail:
- if (appended) {
- bdrv_cbw_drop(top);
- } else {
- bdrv_unref(top);
- }
-
- bdrv_drained_end(source);
-
+ block_copy_state_free(state->bcs);
+ bdrv_unref(top);
return NULL;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last
2021-05-17 6:44 ` [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
@ 2021-05-18 12:35 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 12:35 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Refactor the function to replace child at last. Thus we don't need to
> revert it and code is simplified.
>
> block-copy state initilization being done before replacing the child
*initialization
> doesn't need any drained section.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 33 +++++++++++----------------------
> 1 file changed, 11 insertions(+), 22 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 13/21] block/copy-before-write: introduce cbw_init()
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (12 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 12:53 ` Max Reitz
2021-05-17 6:44 ` [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
` (7 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Move part of bdrv_cbw_append() to new function cbw_open(). It's an
intermediate step for adding noramal .bdrv_open() handler to the
filter. With this commit no logic is changed, but we have a function
which will be turned into .bdrv_open() handler in future commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 73 +++++++++++++++++++++++----------------
1 file changed, 43 insertions(+), 30 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 35a9aa163d..94b1357fd8 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -144,6 +144,47 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
+static int cbw_init(BlockDriverState *top, BlockDriverState *source,
+ BlockDriverState *target, bool compress, Error **errp)
+{
+ BDRVCopyBeforeWriteState *state = top->opaque;
+
+ top->total_sectors = source->total_sectors;
+ top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+ (BDRV_REQ_FUA & source->supported_write_flags);
+ top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+ ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
+ source->supported_zero_flags);
+
+ bdrv_ref(target);
+ state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
+ BDRV_CHILD_DATA, errp);
+ if (!state->target) {
+ error_prepend(errp, "Cannot attach target child: ");
+ bdrv_unref(target);
+ return -EINVAL;
+ }
+
+ bdrv_ref(source);
+ top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+ BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+ errp);
+ if (!top->file) {
+ error_prepend(errp, "Cannot attach file child: ");
+ bdrv_unref(source);
+ return -EINVAL;
+ }
+
+ state->bcs = block_copy_state_new(top->file, state->target, false, compress,
+ errp);
+ if (!state->bcs) {
+ error_prepend(errp, "Cannot create block-copy-state: ");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
BlockDriver bdrv_cbw_filter = {
.format_name = "copy-before-write",
.instance_size = sizeof(BDRVCopyBeforeWriteState),
@@ -181,38 +222,10 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
error_prepend(errp, "Cannot open driver: ");
return NULL;
}
-
state = top->opaque;
- top->total_sectors = source->total_sectors;
- top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
- (BDRV_REQ_FUA & source->supported_write_flags);
- top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
- ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
- source->supported_zero_flags);
-
- bdrv_ref(target);
- state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
- BDRV_CHILD_DATA, errp);
- if (!state->target) {
- error_prepend(errp, "Cannot attach target child: ");
- bdrv_unref(target);
- goto fail;
- }
-
- bdrv_ref(source);
- top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
- BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
- errp);
- if (!top->file) {
- error_prepend(errp, "Cannot attach file child: ");
- bdrv_unref(source);
- goto fail;
- }
- state->bcs = block_copy_state_new(top->file, state->target, false, compress,
- errp);
- if (!state->bcs) {
- error_prepend(errp, "Cannot create block-copy-state: ");
+ ret = cbw_init(top, source, target, compress, errp);
+ if (ret < 0) {
goto fail;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 13/21] block/copy-before-write: introduce cbw_init()
2021-05-17 6:44 ` [PATCH 13/21] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
@ 2021-05-18 12:53 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 12:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Move part of bdrv_cbw_append() to new function cbw_open(). It's an
> intermediate step for adding noramal .bdrv_open() handler to the
> filter. With this commit no logic is changed, but we have a function
> which will be turned into .bdrv_open() handler in future commit.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 73 +++++++++++++++++++++++----------------
> 1 file changed, 43 insertions(+), 30 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (13 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 13/21] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 13:01 ` Max Reitz
2021-05-17 6:44 ` [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
` (6 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
One more step closer to real .bdrv_open() handler: use more usual names
for bs being initialized and its state.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 94b1357fd8..78dc421db2 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -144,40 +144,39 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
-static int cbw_init(BlockDriverState *top, BlockDriverState *source,
+static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
BlockDriverState *target, bool compress, Error **errp)
{
- BDRVCopyBeforeWriteState *state = top->opaque;
+ BDRVCopyBeforeWriteState *s = bs->opaque;
- top->total_sectors = source->total_sectors;
- top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+ bs->total_sectors = source->total_sectors;
+ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
(BDRV_REQ_FUA & source->supported_write_flags);
- top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
source->supported_zero_flags);
bdrv_ref(target);
- state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
- BDRV_CHILD_DATA, errp);
- if (!state->target) {
+ s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
+ BDRV_CHILD_DATA, errp);
+ if (!s->target) {
error_prepend(errp, "Cannot attach target child: ");
bdrv_unref(target);
return -EINVAL;
}
bdrv_ref(source);
- top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
- BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
- errp);
- if (!top->file) {
+ bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
+ BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+ errp);
+ if (!bs->file) {
error_prepend(errp, "Cannot attach file child: ");
bdrv_unref(source);
return -EINVAL;
}
- state->bcs = block_copy_state_new(top->file, state->target, false, compress,
- errp);
- if (!state->bcs) {
+ s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+ if (!s->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
return -EINVAL;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables
2021-05-17 6:44 ` [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
@ 2021-05-18 13:01 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 13:01 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> One more step closer to real .bdrv_open() handler: use more usual names
> for bs being initialized and its state.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 29 ++++++++++++++---------------
> 1 file changed, 14 insertions(+), 15 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (14 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 13:43 ` Max Reitz
2021-05-17 6:44 ` [PATCH 16/21] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
` (5 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
In the next commit we'll get rid of source argument of cbw_init().
Prepare to it now, to make next commit simpler: move the code block
that uses source below attaching the child and use bs->file->bs instead
of source variable.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 78dc421db2..ddd79b3686 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -149,13 +149,6 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
{
BDRVCopyBeforeWriteState *s = bs->opaque;
- bs->total_sectors = source->total_sectors;
- bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
- (BDRV_REQ_FUA & source->supported_write_flags);
- bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
- ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
- source->supported_zero_flags);
-
bdrv_ref(target);
s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
BDRV_CHILD_DATA, errp);
@@ -175,6 +168,13 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
return -EINVAL;
}
+ bs->total_sectors = bs->file->bs->total_sectors;
+ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+ (BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
+ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+ ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
+ bs->file->bs->supported_zero_flags);
+
s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
if (!s->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching
2021-05-17 6:44 ` [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
@ 2021-05-18 13:43 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 13:43 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> In the next commit we'll get rid of source argument of cbw_init().
> Prepare to it now, to make next commit simpler: move the code block
> that uses source below attaching the child and use bs->file->bs instead
> of source variable.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 16/21] block/copy-before-write: cbw_init(): use options
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (15 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 13:56 ` Max Reitz
2021-05-17 6:44 ` [PATCH 17/21] block/block-copy: switch to fully set bitmap by default Vladimir Sementsov-Ogievskiy
` (4 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
One more step closer to .bdrv_open(): use options instead of plain
arguments. Move to bdrv_open_child() calls, native for drive open
handlers.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
1 file changed, 20 insertions(+), 17 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index ddd79b3686..9ff1bf676c 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
- BlockDriverState *target, bool compress, Error **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
{
BDRVCopyBeforeWriteState *s = bs->opaque;
- bdrv_ref(target);
- s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
- BDRV_CHILD_DATA, errp);
- if (!s->target) {
- error_prepend(errp, "Cannot attach target child: ");
- bdrv_unref(target);
+ bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+ BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+ false, errp);
+ if (!bs->file) {
return -EINVAL;
}
- bdrv_ref(source);
- bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
- BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
- errp);
- if (!bs->file) {
- error_prepend(errp, "Cannot attach file child: ");
- bdrv_unref(source);
+ s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
+ BDRV_CHILD_DATA, false, errp);
+ if (!s->target) {
return -EINVAL;
}
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
- s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+ qdict_del(options, "cluster-size");
+ s->bcs = block_copy_state_new(bs->file, s->target, false,
+ qdict_get_try_bool(options, "x-deprecated-compress", false), errp);
+ qdict_del(options, "x-deprecated-compress");
if (!s->bcs) {
error_prepend(errp, "Cannot create block-copy-state: ");
return -EINVAL;
@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
int ret;
BDRVCopyBeforeWriteState *state;
BlockDriverState *top;
+ QDict *opts;
assert(source->total_sectors == target->total_sectors);
@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
}
state = top->opaque;
- ret = cbw_init(top, source, target, compress, errp);
+ opts = qdict_new();
+ qdict_put_str(opts, "file", bdrv_get_node_name(source));
+ qdict_put_str(opts, "target", bdrv_get_node_name(target));
+ qdict_put_bool(opts, "x-deprecated-compress", compress);
+
+ ret = cbw_init(top, opts, errp);
+ qobject_unref(opts);
if (ret < 0) {
goto fail;
}
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
2021-05-17 6:44 ` [PATCH 16/21] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
@ 2021-05-18 13:56 ` Max Reitz
2021-05-18 14:24 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 13:56 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> One more step closer to .bdrv_open(): use options instead of plain
> arguments. Move to bdrv_open_child() calls, native for drive open
> handlers.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
> 1 file changed, 20 insertions(+), 17 deletions(-)
>
> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
> index ddd79b3686..9ff1bf676c 100644
> --- a/block/copy-before-write.c
> +++ b/block/copy-before-write.c
> @@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
> }
> }
>
> -static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
> - BlockDriverState *target, bool compress, Error **errp)
> +static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
> {
> BDRVCopyBeforeWriteState *s = bs->opaque;
>
> - bdrv_ref(target);
> - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
> - BDRV_CHILD_DATA, errp);
> - if (!s->target) {
> - error_prepend(errp, "Cannot attach target child: ");
> - bdrv_unref(target);
> + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
> + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
> + false, errp);
> + if (!bs->file) {
> return -EINVAL;
> }
>
> - bdrv_ref(source);
> - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
> - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
> - errp);
> - if (!bs->file) {
> - error_prepend(errp, "Cannot attach file child: ");
> - bdrv_unref(source);
> + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
> + BDRV_CHILD_DATA, false, errp);
> + if (!s->target) {
> return -EINVAL;
> }
>
> @@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
> ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
> bs->file->bs->supported_zero_flags);
>
> - s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
> + qdict_del(options, "cluster-size");
What is this about?
> + s->bcs = block_copy_state_new(bs->file, s->target, false,
> + qdict_get_try_bool(options, "x-deprecated-compress", false), errp);
First, I’d keep the `compress` variable and use it to store the value,
because this doesn’t look very nice.
Second, what’s the story here? “deprecated” sounds to me like you’re
planning to use a different interface eventually, but looking ahead for
a bit I didn’t find anything yet.
Max
> + qdict_del(options, "x-deprecated-compress");
> if (!s->bcs) {
> error_prepend(errp, "Cannot create block-copy-state: ");
> return -EINVAL;
> @@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
> int ret;
> BDRVCopyBeforeWriteState *state;
> BlockDriverState *top;
> + QDict *opts;
>
> assert(source->total_sectors == target->total_sectors);
>
> @@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
> }
> state = top->opaque;
>
> - ret = cbw_init(top, source, target, compress, errp);
> + opts = qdict_new();
> + qdict_put_str(opts, "file", bdrv_get_node_name(source));
> + qdict_put_str(opts, "target", bdrv_get_node_name(target));
> + qdict_put_bool(opts, "x-deprecated-compress", compress);
> +
> + ret = cbw_init(top, opts, errp);
> + qobject_unref(opts);
> if (ret < 0) {
> goto fail;
> }
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
2021-05-18 13:56 ` Max Reitz
@ 2021-05-18 14:24 ` Vladimir Sementsov-Ogievskiy
2021-05-18 14:29 ` Max Reitz
0 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 14:24 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 16:56, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> One more step closer to .bdrv_open(): use options instead of plain
>> arguments. Move to bdrv_open_child() calls, native for drive open
>> handlers.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
>> 1 file changed, 20 insertions(+), 17 deletions(-)
>>
>> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
>> index ddd79b3686..9ff1bf676c 100644
>> --- a/block/copy-before-write.c
>> +++ b/block/copy-before-write.c
>> @@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
>> }
>> }
>> -static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
>> - BlockDriverState *target, bool compress, Error **errp)
>> +static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
>> {
>> BDRVCopyBeforeWriteState *s = bs->opaque;
>> - bdrv_ref(target);
>> - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
>> - BDRV_CHILD_DATA, errp);
>> - if (!s->target) {
>> - error_prepend(errp, "Cannot attach target child: ");
>> - bdrv_unref(target);
>> + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
>> + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
>> + false, errp);
>> + if (!bs->file) {
>> return -EINVAL;
>> }
>> - bdrv_ref(source);
>> - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
>> - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
>> - errp);
>> - if (!bs->file) {
>> - error_prepend(errp, "Cannot attach file child: ");
>> - bdrv_unref(source);
>> + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
>> + BDRV_CHILD_DATA, false, errp);
>> + if (!s->target) {
>> return -EINVAL;
>> }
>> @@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
>> ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
>> bs->file->bs->supported_zero_flags);
>> - s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
>> + qdict_del(options, "cluster-size");
>
> What is this about?
accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)
>
>> + s->bcs = block_copy_state_new(bs->file, s->target, false,
>> + qdict_get_try_bool(options, "x-deprecated-compress", false), errp);
>
> First, I’d keep the `compress` variable and use it to store the value, because this doesn’t look very nice.
OK
>
> Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
>
I should have described it in commit message.
We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.
>
>> + qdict_del(options, "x-deprecated-compress");
>> if (!s->bcs) {
>> error_prepend(errp, "Cannot create block-copy-state: ");
>> return -EINVAL;
>> @@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
>> int ret;
>> BDRVCopyBeforeWriteState *state;
>> BlockDriverState *top;
>> + QDict *opts;
>> assert(source->total_sectors == target->total_sectors);
>> @@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
>> }
>> state = top->opaque;
>> - ret = cbw_init(top, source, target, compress, errp);
>> + opts = qdict_new();
>> + qdict_put_str(opts, "file", bdrv_get_node_name(source));
>> + qdict_put_str(opts, "target", bdrv_get_node_name(target));
>> + qdict_put_bool(opts, "x-deprecated-compress", compress);
>> +
>> + ret = cbw_init(top, opts, errp);
>> + qobject_unref(opts);
>> if (ret < 0) {
>> goto fail;
>> }
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
2021-05-18 14:24 ` Vladimir Sementsov-Ogievskiy
@ 2021-05-18 14:29 ` Max Reitz
2021-05-18 14:32 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 14:29 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 18.05.21 16:24, Vladimir Sementsov-Ogievskiy wrote:
> 18.05.2021 16:56, Max Reitz wrote:
>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>> One more step closer to .bdrv_open(): use options instead of plain
>>> arguments. Move to bdrv_open_child() calls, native for drive open
>>> handlers.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
>>> 1 file changed, 20 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
>>> index ddd79b3686..9ff1bf676c 100644
>>> --- a/block/copy-before-write.c
>>> +++ b/block/copy-before-write.c
>>> @@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState
>>> *bs, BdrvChild *c,
>>> }
>>> }
>>> -static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
>>> - BlockDriverState *target, bool compress, Error
>>> **errp)
>>> +static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
>>> {
>>> BDRVCopyBeforeWriteState *s = bs->opaque;
>>> - bdrv_ref(target);
>>> - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
>>> - BDRV_CHILD_DATA, errp);
>>> - if (!s->target) {
>>> - error_prepend(errp, "Cannot attach target child: ");
>>> - bdrv_unref(target);
>>> + bs->file = bdrv_open_child(NULL, options, "file", bs,
>>> &child_of_bds,
>>> + BDRV_CHILD_FILTERED |
>>> BDRV_CHILD_PRIMARY,
>>> + false, errp);
>>> + if (!bs->file) {
>>> return -EINVAL;
>>> }
>>> - bdrv_ref(source);
>>> - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
>>> - BDRV_CHILD_FILTERED |
>>> BDRV_CHILD_PRIMARY,
>>> - errp);
>>> - if (!bs->file) {
>>> - error_prepend(errp, "Cannot attach file child: ");
>>> - bdrv_unref(source);
>>> + s->target = bdrv_open_child(NULL, options, "target", bs,
>>> &child_of_bds,
>>> + BDRV_CHILD_DATA, false, errp);
>>> + if (!s->target) {
>>> return -EINVAL;
>>> }
>>> @@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs,
>>> BlockDriverState *source,
>>> ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP |
>>> BDRV_REQ_NO_FALLBACK) &
>>> bs->file->bs->supported_zero_flags);
>>> - s->bcs = block_copy_state_new(bs->file, s->target, false,
>>> compress, errp);
>>> + qdict_del(options, "cluster-size");
>>
>> What is this about?
>
> accidental, will drop. (it's a remaining of my first solution where I
> tried to pass cluster-size, then I decided that's better move
> cluster-size detection to block_copy)
>
>>
>>> + s->bcs = block_copy_state_new(bs->file, s->target, false,
>>> + qdict_get_try_bool(options, "x-deprecated-compress",
>>> false), errp);
>>
>> First, I’d keep the `compress` variable and use it to store the value,
>> because this doesn’t look very nice.
>
> OK
>
>>
>> Second, what’s the story here? “deprecated” sounds to me like you’re
>> planning to use a different interface eventually, but looking ahead
>> for a bit I didn’t find anything yet.
>>
>
> I should have described it in commit message.
>
> We have "compress" filter driver. So instead adding "compress" option to
> every block job or filter, user should use "compress" filter. That's why
> I don't want to publish compress option for copy-before-write filter.
> Still we need it to maintain "compress" option of backup job. I also
> want to deprecate "compress" option in backup, then everything will be
> clear.
Oh, that’s true. Until backup’s option is deprecated, I think a comment
would be more useful than a note in the commit message, though.
Max
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
2021-05-18 14:29 ` Max Reitz
@ 2021-05-18 14:32 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 14:32 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 17:29, Max Reitz wrote:
> On 18.05.21 16:24, Vladimir Sementsov-Ogievskiy wrote:
>> 18.05.2021 16:56, Max Reitz wrote:
>>> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>>>> One more step closer to .bdrv_open(): use options instead of plain
>>>> arguments. Move to bdrv_open_child() calls, native for drive open
>>>> handlers.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>> block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
>>>> 1 file changed, 20 insertions(+), 17 deletions(-)
>>>>
>>>> diff --git a/block/copy-before-write.c b/block/copy-before-write.c
>>>> index ddd79b3686..9ff1bf676c 100644
>>>> --- a/block/copy-before-write.c
>>>> +++ b/block/copy-before-write.c
>>>> @@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
>>>> }
>>>> }
>>>> -static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
>>>> - BlockDriverState *target, bool compress, Error **errp)
>>>> +static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
>>>> {
>>>> BDRVCopyBeforeWriteState *s = bs->opaque;
>>>> - bdrv_ref(target);
>>>> - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
>>>> - BDRV_CHILD_DATA, errp);
>>>> - if (!s->target) {
>>>> - error_prepend(errp, "Cannot attach target child: ");
>>>> - bdrv_unref(target);
>>>> + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
>>>> + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
>>>> + false, errp);
>>>> + if (!bs->file) {
>>>> return -EINVAL;
>>>> }
>>>> - bdrv_ref(source);
>>>> - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
>>>> - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
>>>> - errp);
>>>> - if (!bs->file) {
>>>> - error_prepend(errp, "Cannot attach file child: ");
>>>> - bdrv_unref(source);
>>>> + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
>>>> + BDRV_CHILD_DATA, false, errp);
>>>> + if (!s->target) {
>>>> return -EINVAL;
>>>> }
>>>> @@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
>>>> ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
>>>> bs->file->bs->supported_zero_flags);
>>>> - s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
>>>> + qdict_del(options, "cluster-size");
>>>
>>> What is this about?
>>
>> accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)
>>
>>>
>>>> + s->bcs = block_copy_state_new(bs->file, s->target, false,
>>>> + qdict_get_try_bool(options, "x-deprecated-compress", false), errp);
>>>
>>> First, I’d keep the `compress` variable and use it to store the value, because this doesn’t look very nice.
>>
>> OK
>>
>>>
>>> Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
>>>
>>
>> I should have described it in commit message.
>>
>> We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.
>
> Oh, that’s true. Until backup’s option is deprecated, I think a comment would be more useful than a note in the commit message, though.
>
Agree, will add
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 17/21] block/block-copy: switch to fully set bitmap by default
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (16 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 16/21] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 14:22 ` Max Reitz
2021-05-17 6:44 ` [PATCH 18/21] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
` (3 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
block-copy has a bit inconvenient interface around dirty bitmap: user
should get pointer to it and than set by hand. We do need a possibility
to share the bitmap with backup job.
But default of empty bitmap is strange. Switch to full-set bitmap by
default. This way we will not care about setting dirty bitmap in
copy-before-write filter when publish it so that it can be used in
separate of backup job.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/backup.c | 16 +++++++---------
block/block-copy.c | 1 +
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 90cca1ca30..706c54fea1 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+ bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
NULL, true);
assert(ret);
- } else {
- if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
- /*
- * We can't hog the coroutine to initialize this thoroughly.
- * Set a flag and resume work when we are able to yield safely.
- */
- block_copy_set_skip_unallocated(job->bcs, true);
- }
- bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
+ } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
+ /*
+ * We can't hog the coroutine to initialize this thoroughly.
+ * Set a flag and resume work when we are able to yield safely.
+ */
+ block_copy_set_skip_unallocated(job->bcs, true);
}
estimate = bdrv_get_dirty_count(bcs_bitmap);
diff --git a/block/block-copy.c b/block/block-copy.c
index 9020234c6e..4126f7e8cc 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
return NULL;
}
bdrv_disable_dirty_bitmap(copy_bitmap);
+ bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
/*
* Why we always set BDRV_REQ_SERIALISING write flag:
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default
2021-05-17 6:44 ` [PATCH 17/21] block/block-copy: switch to fully set bitmap by default Vladimir Sementsov-Ogievskiy
@ 2021-05-18 14:22 ` Max Reitz
2021-05-18 14:31 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 14:22 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> block-copy has a bit inconvenient interface around dirty bitmap: user
> should get pointer to it and than set by hand. We do need a possibility
> to share the bitmap with backup job.
>
> But default of empty bitmap is strange.
I don’t know, I don’t find it strange. It expects its user to specify
what data to copy, so clearly it gives said user a blank slate.
> Switch to full-set bitmap by
> default. This way we will not care about setting dirty bitmap in
> copy-before-write filter when publish it so that it can be used in
> separate of backup job.
That’s a valid reason, though, so:
Reviewed-by: Max Reitz <mreitz@redhat.com>
Still, I find it stranger this way, because I’m more used to
“initialization to 0 by default”.
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/backup.c | 16 +++++++---------
> block/block-copy.c | 1 +
> 2 files changed, 8 insertions(+), 9 deletions(-)
>
> diff --git a/block/backup.c b/block/backup.c
> index 90cca1ca30..706c54fea1 100644
> --- a/block/backup.c
> +++ b/block/backup.c
> @@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
> BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
>
> if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
> + bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
> ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
> NULL, true);
> assert(ret);
> - } else {
> - if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
> - /*
> - * We can't hog the coroutine to initialize this thoroughly.
> - * Set a flag and resume work when we are able to yield safely.
> - */
> - block_copy_set_skip_unallocated(job->bcs, true);
> - }
> - bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
> + } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
> + /*
> + * We can't hog the coroutine to initialize this thoroughly.
> + * Set a flag and resume work when we are able to yield safely.
> + */
> + block_copy_set_skip_unallocated(job->bcs, true);
> }
>
> estimate = bdrv_get_dirty_count(bcs_bitmap);
> diff --git a/block/block-copy.c b/block/block-copy.c
> index 9020234c6e..4126f7e8cc 100644
> --- a/block/block-copy.c
> +++ b/block/block-copy.c
> @@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
> return NULL;
> }
> bdrv_disable_dirty_bitmap(copy_bitmap);
> + bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
>
> /*
> * Why we always set BDRV_REQ_SERIALISING write flag:
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default
2021-05-18 14:22 ` Max Reitz
@ 2021-05-18 14:31 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 14:31 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 17:22, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> block-copy has a bit inconvenient interface around dirty bitmap: user
>> should get pointer to it and than set by hand. We do need a possibility
>> to share the bitmap with backup job.
>>
>> But default of empty bitmap is strange.
>
> I don’t know, I don’t find it strange. It expects its user to specify what data to copy, so clearly it gives said user a blank slate.
>
>> Switch to full-set bitmap by
>> default. This way we will not care about setting dirty bitmap in
>> copy-before-write filter when publish it so that it can be used in
>> separate of backup job.
>
> That’s a valid reason, though, so:
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
> Still, I find it stranger this way, because I’m more used to “initialization to 0 by default”.
When I started to test this series it didn't work, because bitmap was not initialized. I fixed it and call original behavior strange :)
Probably better to keep block-copy as is and set bitmap in cbw_open. Backup code should be modified anyway, as it will get the bitmap after initialization in cbw_open.
Next step would be support for bitmap in copy-before-write filter anyway, and this place will be changed again.. So, I don't know, but it's not important thing.
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/backup.c | 16 +++++++---------
>> block/block-copy.c | 1 +
>> 2 files changed, 8 insertions(+), 9 deletions(-)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index 90cca1ca30..706c54fea1 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
>> BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
>> if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
>> + bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
>> ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
>> NULL, true);
>> assert(ret);
>> - } else {
>> - if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
>> - /*
>> - * We can't hog the coroutine to initialize this thoroughly.
>> - * Set a flag and resume work when we are able to yield safely.
>> - */
>> - block_copy_set_skip_unallocated(job->bcs, true);
>> - }
>> - bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
>> + } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
>> + /*
>> + * We can't hog the coroutine to initialize this thoroughly.
>> + * Set a flag and resume work when we are able to yield safely.
>> + */
>> + block_copy_set_skip_unallocated(job->bcs, true);
>> }
>> estimate = bdrv_get_dirty_count(bcs_bitmap);
>> diff --git a/block/block-copy.c b/block/block-copy.c
>> index 9020234c6e..4126f7e8cc 100644
>> --- a/block/block-copy.c
>> +++ b/block/block-copy.c
>> @@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
>> return NULL;
>> }
>> bdrv_disable_dirty_bitmap(copy_bitmap);
>> + bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
>> /*
>> * Why we always set BDRV_REQ_SERIALISING write flag:
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 18/21] block/block-copy: make setting progress optional
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (17 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 17/21] block/block-copy: switch to fully set bitmap by default Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 14:26 ` Max Reitz
2021-05-17 6:44 ` [PATCH 19/21] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Now block-copy will crash if user don't set progress meter by
block_copy_set_progress_meter(). copy-before-write filter will be used
in separate of backup job, and it doesn't want any progress meter (for
now). So, allow not setting it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/block-copy.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/block/block-copy.c b/block/block-copy.c
index 4126f7e8cc..63ec5c41a2 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -506,7 +506,7 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
if (ret < 0 && !t->call_state->ret) {
t->call_state->ret = ret;
t->call_state->error_is_read = error_is_read;
- } else {
+ } else if (t->s->progress) {
progress_work_done(t->s->progress, t->bytes);
}
co_put_to_shres(t->s->mem, t->bytes);
@@ -610,9 +610,11 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
if (!ret) {
bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
- progress_set_remaining(s->progress,
- bdrv_get_dirty_count(s->copy_bitmap) +
- s->in_flight_bytes);
+ if (s->progress) {
+ progress_set_remaining(s->progress,
+ bdrv_get_dirty_count(s->copy_bitmap) +
+ s->in_flight_bytes);
+ }
}
*count = bytes;
@@ -672,9 +674,11 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
}
if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) {
block_copy_task_end(task, 0);
- progress_set_remaining(s->progress,
- bdrv_get_dirty_count(s->copy_bitmap) +
- s->in_flight_bytes);
+ if (s->progress) {
+ progress_set_remaining(s->progress,
+ bdrv_get_dirty_count(s->copy_bitmap) +
+ s->in_flight_bytes);
+ }
trace_block_copy_skip_range(s, task->offset, task->bytes);
offset = task_end(task);
bytes = end - offset;
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 18/21] block/block-copy: make setting progress optional
2021-05-17 6:44 ` [PATCH 18/21] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
@ 2021-05-18 14:26 ` Max Reitz
2021-05-18 14:35 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 14:26 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Now block-copy will crash if user don't set progress meter by
> block_copy_set_progress_meter(). copy-before-write filter will be used
> in separate of backup job, and it doesn't want any progress meter (for
> now). So, allow not setting it.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/block-copy.c | 18 +++++++++++-------
> 1 file changed, 11 insertions(+), 7 deletions(-)
Makes me wonder whether an “if (not NULL)” check in progress_meter.h
wouldn’t be a nice addition, but:
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* Re: [PATCH 18/21] block/block-copy: make setting progress optional
2021-05-18 14:26 ` Max Reitz
@ 2021-05-18 14:35 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 14:35 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 17:26, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> Now block-copy will crash if user don't set progress meter by
>> block_copy_set_progress_meter(). copy-before-write filter will be used
>> in separate of backup job, and it doesn't want any progress meter (for
>> now). So, allow not setting it.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block/block-copy.c | 18 +++++++++++-------
>> 1 file changed, 11 insertions(+), 7 deletions(-)
>
> Makes me wonder whether an “if (not NULL)” check in progress_meter.h wouldn’t be a nice addition, but:
IMHO it isn't. It's unusual to check if (obj is not NULL) in all obj API functions.. The only place where it is usual is obj_free() like functions.
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
Thanks!
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 19/21] block/copy-before-write: make public block driver
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (18 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 18/21] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 14:46 ` Max Reitz
2021-05-17 6:44 ` [PATCH 20/21] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-05-17 6:44 ` [PATCH 21/21] itotests/222: add test-case for " Vladimir Sementsov-Ogievskiy
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Finally, copy-before-write gets own .bdrv_open and .bdrv_close
handlers, block_init() call and becomes available through bdrv_open().
To achieve this:
- cbw_init gets unused flags argument and becomes cbw_open
- block_copy_state_free() call moved to new cbw_close()
- in bdrv_cbw_append:
- options are completed with driver and node-name, and we can simply
use bdrv_insert_node() to do both open and drained replacing
- in bdrv_cbw_drop:
- cbw_close() is now responsible for freeing s->bcs, so don't do it
here
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/copy-before-write.c | 60 ++++++++++++++++++---------------------
1 file changed, 28 insertions(+), 32 deletions(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 9ff1bf676c..20c933bfb0 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -144,7 +144,8 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
}
}
-static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
+static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
{
BDRVCopyBeforeWriteState *s = bs->opaque;
@@ -180,10 +181,21 @@ static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
return 0;
}
+static void cbw_close(BlockDriverState *bs)
+{
+ BDRVCopyBeforeWriteState *s = bs->opaque;
+
+ block_copy_state_free(s->bcs);
+ s->bcs = NULL;
+}
+
BlockDriver bdrv_cbw_filter = {
.format_name = "copy-before-write",
.instance_size = sizeof(BDRVCopyBeforeWriteState),
+ .bdrv_open = cbw_open,
+ .bdrv_close = cbw_close,
+
.bdrv_co_preadv = cbw_co_preadv,
.bdrv_co_pwritev = cbw_co_pwritev,
.bdrv_co_pwrite_zeroes = cbw_co_pwrite_zeroes,
@@ -205,57 +217,41 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
Error **errp)
{
ERRP_GUARD();
- int ret;
BDRVCopyBeforeWriteState *state;
BlockDriverState *top;
QDict *opts;
assert(source->total_sectors == target->total_sectors);
- top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name,
- BDRV_O_RDWR, errp);
- if (!top) {
- error_prepend(errp, "Cannot open driver: ");
- return NULL;
- }
- state = top->opaque;
-
opts = qdict_new();
+ qdict_put_str(opts, "driver", "copy-before-write");
+ if (filter_node_name) {
+ qdict_put_str(opts, "node-name", filter_node_name);
+ }
qdict_put_str(opts, "file", bdrv_get_node_name(source));
qdict_put_str(opts, "target", bdrv_get_node_name(target));
qdict_put_bool(opts, "x-deprecated-compress", compress);
- ret = cbw_init(top, opts, errp);
- qobject_unref(opts);
- if (ret < 0) {
- goto fail;
- }
-
- bdrv_drained_begin(source);
- ret = bdrv_replace_node(source, top, errp);
- bdrv_drained_end(source);
- if (ret < 0) {
- error_prepend(errp, "Cannot append copy-before-write filter: ");
- goto fail;
+ top = bdrv_insert_node(source, opts, BDRV_O_RDWR, errp);
+ if (!top) {
+ return NULL;
}
+ state = top->opaque;
*bcs = state->bcs;
return top;
-
-fail:
- block_copy_state_free(state->bcs);
- bdrv_unref(top);
- return NULL;
}
void bdrv_cbw_drop(BlockDriverState *bs)
{
- BDRVCopyBeforeWriteState *s = bs->opaque;
-
bdrv_drop_filter(bs, &error_abort);
-
- block_copy_state_free(s->bcs);
-
bdrv_unref(bs);
}
+
+static void cbw_init(void)
+{
+ bdrv_register(&bdrv_cbw_filter);
+}
+
+block_init(cbw_init);
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 19/21] block/copy-before-write: make public block driver
2021-05-17 6:44 ` [PATCH 19/21] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
@ 2021-05-18 14:46 ` Max Reitz
0 siblings, 0 replies; 66+ messages in thread
From: Max Reitz @ 2021-05-18 14:46 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Finally, copy-before-write gets own .bdrv_open and .bdrv_close
> handlers, block_init() call and becomes available through bdrv_open().
>
> To achieve this:
>
> - cbw_init gets unused flags argument and becomes cbw_open
> - block_copy_state_free() call moved to new cbw_close()
> - in bdrv_cbw_append:
> - options are completed with driver and node-name, and we can simply
> use bdrv_insert_node() to do both open and drained replacing
> - in bdrv_cbw_drop:
> - cbw_close() is now responsible for freeing s->bcs, so don't do it
> here
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block/copy-before-write.c | 60 ++++++++++++++++++---------------------
> 1 file changed, 28 insertions(+), 32 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 20/21] qapi: publish copy-before-write filter
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (19 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 19/21] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 14:48 ` Max Reitz
2021-05-17 6:44 ` [PATCH 21/21] itotests/222: add test-case for " Vladimir Sementsov-Ogievskiy
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
qapi/block-core.json | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2ea294129e..f8fbcb6416 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2808,15 +2808,17 @@
# @blklogwrites: Since 3.0
# @blkreplay: Since 4.2
# @compress: Since 5.0
+# @copy-before-write: Since 6.1
#
# Since: 2.9
##
{ 'enum': 'BlockdevDriver',
'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
- 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
- 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
- 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
- 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+ 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
+ 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
+ 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
+ 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
+ 'qed', 'quorum', 'raw', 'rbd',
{ 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
@@ -3937,6 +3939,17 @@
'base': 'BlockdevOptionsGenericFormat',
'data': { '*bottom': 'str' } }
+##
+# @BlockdevOptionsCbw:
+#
+# Driver specific block device options for the copy-before-write driver.
+#
+# Since: 6.1
+##
+{ 'struct': 'BlockdevOptionsCbw',
+ 'base': 'BlockdevOptionsGenericFormat',
+ 'data': { 'target': 'BlockdevRef' } }
+
##
# @BlockdevOptions:
#
@@ -3989,6 +4002,7 @@
'bochs': 'BlockdevOptionsGenericFormat',
'cloop': 'BlockdevOptionsGenericFormat',
'compress': 'BlockdevOptionsGenericFormat',
+ 'copy-before-write':'BlockdevOptionsCbw',
'copy-on-read':'BlockdevOptionsCor',
'dmg': 'BlockdevOptionsGenericFormat',
'file': 'BlockdevOptionsFile',
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 20/21] qapi: publish copy-before-write filter
2021-05-17 6:44 ` [PATCH 20/21] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
@ 2021-05-18 14:48 ` Max Reitz
2021-05-18 14:56 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 14:48 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> qapi/block-core.json | 22 ++++++++++++++++++----
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 2ea294129e..f8fbcb6416 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2808,15 +2808,17 @@
> # @blklogwrites: Since 3.0
> # @blkreplay: Since 4.2
> # @compress: Since 5.0
> +# @copy-before-write: Since 6.1
> #
> # Since: 2.9
> ##
> { 'enum': 'BlockdevDriver',
> 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
> - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
> - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
> - 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
> - 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
> + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
> + 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
> + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
> + 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
> + 'qed', 'quorum', 'raw', 'rbd',
> { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
> 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
>
> @@ -3937,6 +3939,17 @@
> 'base': 'BlockdevOptionsGenericFormat',
> 'data': { '*bottom': 'str' } }
>
> +##
> +# @BlockdevOptionsCbw:
> +#
> +# Driver specific block device options for the copy-before-write driver.
> +#
I think there should be a description of @target here.
(Looks good otherwise, though.)
Max
> +# Since: 6.1
> +##
> +{ 'struct': 'BlockdevOptionsCbw',
> + 'base': 'BlockdevOptionsGenericFormat',
> + 'data': { 'target': 'BlockdevRef' } }
> +
> ##
> # @BlockdevOptions:
> #
> @@ -3989,6 +4002,7 @@
> 'bochs': 'BlockdevOptionsGenericFormat',
> 'cloop': 'BlockdevOptionsGenericFormat',
> 'compress': 'BlockdevOptionsGenericFormat',
> + 'copy-before-write':'BlockdevOptionsCbw',
> 'copy-on-read':'BlockdevOptionsCor',
> 'dmg': 'BlockdevOptionsGenericFormat',
> 'file': 'BlockdevOptionsFile',
>
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 20/21] qapi: publish copy-before-write filter
2021-05-18 14:48 ` Max Reitz
@ 2021-05-18 14:56 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 14:56 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 17:48, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> qapi/block-core.json | 22 ++++++++++++++++++----
>> 1 file changed, 18 insertions(+), 4 deletions(-)
>>
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 2ea294129e..f8fbcb6416 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -2808,15 +2808,17 @@
>> # @blklogwrites: Since 3.0
>> # @blkreplay: Since 4.2
>> # @compress: Since 5.0
>> +# @copy-before-write: Since 6.1
>> #
>> # Since: 2.9
>> ##
>> { 'enum': 'BlockdevDriver',
>> 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
>> - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
>> - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
>> - 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
>> - 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
>> + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
>> + 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
>> + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
>> + 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
>> + 'qed', 'quorum', 'raw', 'rbd',
>> { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
>> 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
>> @@ -3937,6 +3939,17 @@
>> 'base': 'BlockdevOptionsGenericFormat',
>> 'data': { '*bottom': 'str' } }
>> +##
>> +# @BlockdevOptionsCbw:
>> +#
>> +# Driver specific block device options for the copy-before-write driver,
which does so called copy-before-write operations: when data is written to
the filter, it firstly reads corresponding blocks from its file child and
copies them to @target child. After successful copying the write request is
propagated to file child. If copying filed, the original write request is
failed too and no data is written to file child.
>> +#
>
> I think there should be a description of @target here.
Right, I forget. Something like:
@target: The target for copy-before-write operations.
>
> (Looks good otherwise, though.)
>
> Max
>
>> +# Since: 6.1
>> +##
>> +{ 'struct': 'BlockdevOptionsCbw',
>> + 'base': 'BlockdevOptionsGenericFormat',
>> + 'data': { 'target': 'BlockdevRef' } }
>> +
>> ##
>> # @BlockdevOptions:
>> #
>> @@ -3989,6 +4002,7 @@
>> 'bochs': 'BlockdevOptionsGenericFormat',
>> 'cloop': 'BlockdevOptionsGenericFormat',
>> 'compress': 'BlockdevOptionsGenericFormat',
>> + 'copy-before-write':'BlockdevOptionsCbw',
>> 'copy-on-read':'BlockdevOptionsCor',
>> 'dmg': 'BlockdevOptionsGenericFormat',
>> 'file': 'BlockdevOptionsFile',
>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread
* [PATCH 21/21] itotests/222: add test-case for copy-before-write filter
2021-05-17 6:44 [PATCH 00/21] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
` (20 preceding siblings ...)
2021-05-17 6:44 ` [PATCH 20/21] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
@ 2021-05-17 6:44 ` Vladimir Sementsov-Ogievskiy
2021-05-18 15:24 ` Max Reitz
21 siblings, 1 reply; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-17 6:44 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini,
vsementsov, jsnow, mreitz, kwolf, den
New fleecing method becomes available: copy-before-write filter.
Actually we don't need backup job to setup image fleecing. Add test
fore new recommended way of image fleecing.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/222 | 56 ++++++++++++++++++++++-------
tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++
2 files changed, 115 insertions(+), 13 deletions(-)
diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222
index b48afe623e..8f8e1e4d7f 100755
--- a/tests/qemu-iotests/222
+++ b/tests/qemu-iotests/222
@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1]
("0xdc", "32M", "32k"), # Left-end of partial-right [2]
("0xcd", "0x3ff0000", "64k")] # patterns[3]
-with iotests.FilePath('base.img') as base_img_path, \
- iotests.FilePath('fleece.img') as fleece_img_path, \
- iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
- iotests.VM() as vm:
+def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
+ if use_cbw:
+ log('=== Test filter based fleecing ===')
+ else:
+ log('=== Test backup(sync=none) based fleecing ===')
+ log('')
log('--- Setting up images ---')
log('')
@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
log('--- Launching VM ---')
log('')
- vm.add_drive(base_img_path)
+ vm.add_drive(base_img_path, "node-name=node0")
vm.launch()
log('Done')
@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
"backing": src_node,
}))
- # Establish COW from source to fleecing node
- log(vm.qmp("blockdev-backup",
- device=src_node,
- target=tgt_node,
- sync="none"))
+ # Establish CBW from source to fleecing node
+ if use_cbw:
+ log(vm.qmp("blockdev-add", **{
+ "driver": "copy-before-write",
+ "node-name": "fl-cbw",
+ "file": src_node,
+ "target": tgt_node
+ }))
+
+ log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+ property="drive", value="fl-cbw"))
+ else:
+ log(vm.qmp("blockdev-backup",
+ device=src_node,
+ target=tgt_node,
+ sync="none"))
log('')
log('--- Setting up NBD Export ---')
@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
log('--- Cleanup ---')
log('')
- log(vm.qmp('block-job-cancel', device=src_node))
- log(vm.event_wait('BLOCK_JOB_CANCELLED'),
- filters=[iotests.filter_qmp_event])
+ if use_cbw:
+ log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+ property="drive", value="node0"))
+ log(vm.qmp("blockdev-del", node_name="fl-cbw"))
+ else:
+ log(vm.qmp('block-job-cancel', device=src_node))
+ log(vm.event_wait('BLOCK_JOB_CANCELLED'),
+ filters=[iotests.filter_qmp_event])
+
log(vm.qmp('nbd-server-stop'))
log(vm.qmp('blockdev-del', node_name=tgt_node))
vm.shutdown()
@@ -157,3 +176,14 @@ with iotests.FilePath('base.img') as base_img_path, \
log('')
log('Done')
+
+def test(use_cbw):
+ with iotests.FilePath('base.img') as base_img_path, \
+ iotests.FilePath('fleece.img') as fleece_img_path, \
+ iotests.FilePath('nbd.sock',
+ base_dir=iotests.sock_dir) as nbd_sock_path, \
+ iotests.VM() as vm:
+ do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm)
+
+test(False)
+test(True)
diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out
index 16643dde30..bdc0f7912f 100644
--- a/tests/qemu-iotests/222.out
+++ b/tests/qemu-iotests/222.out
@@ -1,3 +1,5 @@
+=== Test backup(sync=none) based fleecing ===
+
--- Setting up images ---
Done
@@ -65,3 +67,73 @@ read -P0xdc 32M 32k
read -P0xcd 0x3ff0000 64k
Done
+=== Test filter based fleecing ===
+
+--- Setting up images ---
+
+Done
+
+--- Launching VM ---
+
+Done
+
+--- Setting up Fleecing Graph ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
+
+--- Setting up NBD Export ---
+
+{"return": {}}
+{"return": {}}
+
+--- Sanity Check ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Testing COW ---
+
+write -P0xab 0 64k
+{"return": ""}
+write -P0xad 0x00f8000 64k
+{"return": ""}
+write -P0x1d 0x2008000 64k
+{"return": ""}
+write -P0xea 0x3fe0000 64k
+{"return": ""}
+
+--- Verifying Data ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Cleanup ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
+{"return": {}}
+
+--- Confirming writes ---
+
+read -P0xab 0 64k
+read -P0xad 0x00f8000 64k
+read -P0x1d 0x2008000 64k
+read -P0xea 0x3fe0000 64k
+read -P0xd5 0x108000 32k
+read -P0xdc 32M 32k
+read -P0xcd 0x3ff0000 64k
+
+Done
--
2.29.2
^ permalink raw reply related [flat|nested] 66+ messages in thread* Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter
2021-05-17 6:44 ` [PATCH 21/21] itotests/222: add test-case for " Vladimir Sementsov-Ogievskiy
@ 2021-05-18 15:24 ` Max Reitz
2021-05-18 15:41 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 66+ messages in thread
From: Max Reitz @ 2021-05-18 15:24 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-block
Cc: kwolf, berrange, ehabkost, jsnow, qemu-devel, armbru, den, pbonzini
On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
> New fleecing method becomes available: copy-before-write filter.
>
> Actually we don't need backup job to setup image fleecing. Add test
> fore new recommended way of image fleecing.
*for
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> tests/qemu-iotests/222 | 56 ++++++++++++++++++++++-------
> tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++
> 2 files changed, 115 insertions(+), 13 deletions(-)
Looks good, just wondering about some variable usage (why is src_node a
BB name?) and whether the virtio-blk qdev path couldn’t be expressed in
some nicer way.
> diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222
> index b48afe623e..8f8e1e4d7f 100755
> --- a/tests/qemu-iotests/222
> +++ b/tests/qemu-iotests/222
> @@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1]
> ("0xdc", "32M", "32k"), # Left-end of partial-right [2]
> ("0xcd", "0x3ff0000", "64k")] # patterns[3]
>
> -with iotests.FilePath('base.img') as base_img_path, \
> - iotests.FilePath('fleece.img') as fleece_img_path, \
> - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
> - iotests.VM() as vm:
> +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
> + if use_cbw:
> + log('=== Test filter based fleecing ===')
> + else:
> + log('=== Test backup(sync=none) based fleecing ===')
>
> + log('')
> log('--- Setting up images ---')
> log('')
>
> @@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
> log('--- Launching VM ---')
> log('')
>
> - vm.add_drive(base_img_path)
> + vm.add_drive(base_img_path, "node-name=node0")
> vm.launch()
> log('Done')
>
Afterwards, src_node is set to 'drive0'. So src_node is actually a BB
name...
> @@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
> "backing": src_node,
> }))
>
> - # Establish COW from source to fleecing node
> - log(vm.qmp("blockdev-backup",
> - device=src_node,
> - target=tgt_node,
> - sync="none"))
> + # Establish CBW from source to fleecing node
> + if use_cbw:
> + log(vm.qmp("blockdev-add", **{
> + "driver": "copy-before-write",
> + "node-name": "fl-cbw",
> + "file": src_node,
> + "target": tgt_node
Which makes this look strange, and also TIL (or maybe “today I was
reminded”?) that you can use BB names as node references.
(It’s also already weird in the `"backing": src_node` line right at the
beginning of this hunk...)
So I guess it works, but I’d find it would be nicer to distinguish
between the node name and the BB name.
> + }))
> +
> + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
> + property="drive", value="fl-cbw"))
Is that path portable across targets? Should we maybe instead manually
add a virtio-scsi device that we can give a proper name?
Or I suppose we could do
path = next(dev for dev in vm.qmp('query-block')['return']
if dev['device'] == 'drive0')['qdev']
> + else:
> + log(vm.qmp("blockdev-backup",
> + device=src_node,
> + target=tgt_node,
> + sync="none"))
>
> log('')
> log('--- Setting up NBD Export ---')
> @@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
> log('--- Cleanup ---')
> log('')
>
> - log(vm.qmp('block-job-cancel', device=src_node))
> - log(vm.event_wait('BLOCK_JOB_CANCELLED'),
> - filters=[iotests.filter_qmp_event])
> + if use_cbw:
> + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
> + property="drive", value="node0"))
If src_node were 'node0', we wouldn’t need to use the 'node0' literal
here and could use src_node instead. (Because it kind of seems like
this test wants to use those variables instead of literals. I mean, we
could also just call the node 'src-node', but, well.)
Max
> + log(vm.qmp("blockdev-del", node_name="fl-cbw"))
> + else:
> + log(vm.qmp('block-job-cancel', device=src_node))
> + log(vm.event_wait('BLOCK_JOB_CANCELLED'),
> + filters=[iotests.filter_qmp_event])
> +
> log(vm.qmp('nbd-server-stop'))
> log(vm.qmp('blockdev-del', node_name=tgt_node))
> vm.shutdown()
^ permalink raw reply [flat|nested] 66+ messages in thread* Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter
2021-05-18 15:24 ` Max Reitz
@ 2021-05-18 15:41 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 66+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-05-18 15:41 UTC (permalink / raw)
To: Max Reitz, qemu-block
Cc: qemu-devel, armbru, eblake, ehabkost, berrange, pbonzini, jsnow,
kwolf, den
18.05.2021 18:24, Max Reitz wrote:
> On 17.05.21 08:44, Vladimir Sementsov-Ogievskiy wrote:
>> New fleecing method becomes available: copy-before-write filter.
>>
>> Actually we don't need backup job to setup image fleecing. Add test
>> fore new recommended way of image fleecing.
>
> *for
>
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> tests/qemu-iotests/222 | 56 ++++++++++++++++++++++-------
>> tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 115 insertions(+), 13 deletions(-)
>
> Looks good, just wondering about some variable usage (why is src_node a BB name?) and whether the virtio-blk qdev path couldn’t be expressed in some nicer way.
>
>> diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222
>> index b48afe623e..8f8e1e4d7f 100755
>> --- a/tests/qemu-iotests/222
>> +++ b/tests/qemu-iotests/222
>> @@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1]
>> ("0xdc", "32M", "32k"), # Left-end of partial-right [2]
>> ("0xcd", "0x3ff0000", "64k")] # patterns[3]
>> -with iotests.FilePath('base.img') as base_img_path, \
>> - iotests.FilePath('fleece.img') as fleece_img_path, \
>> - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
>> - iotests.VM() as vm:
>> +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
>> + if use_cbw:
>> + log('=== Test filter based fleecing ===')
>> + else:
>> + log('=== Test backup(sync=none) based fleecing ===')
>> + log('')
>> log('--- Setting up images ---')
>> log('')
>> @@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
>> log('--- Launching VM ---')
>> log('')
>> - vm.add_drive(base_img_path)
>> + vm.add_drive(base_img_path, "node-name=node0")
>> vm.launch()
>> log('Done')
>>
>
> Afterwards, src_node is set to 'drive0'. So src_node is actually a BB name...
>
>> @@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
>> "backing": src_node,
>> }))
>> - # Establish COW from source to fleecing node
>> - log(vm.qmp("blockdev-backup",
>> - device=src_node,
>> - target=tgt_node,
>> - sync="none"))
>> + # Establish CBW from source to fleecing node
>> + if use_cbw:
>> + log(vm.qmp("blockdev-add", **{
>> + "driver": "copy-before-write",
>> + "node-name": "fl-cbw",
>> + "file": src_node,
>> + "target": tgt_node
>
> Which makes this look strange, and also TIL (or maybe “today I was reminded”?) that you can use BB names as node references.
>
> (It’s also already weird in the `"backing": src_node` line right at the beginning of this hunk...)
>
> So I guess it works, but I’d find it would be nicer to distinguish between the node name and the BB name.
Oh yes, that's all the mess, I'll improve.
>
>> + }))
>> +
>> + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
>> + property="drive", value="fl-cbw"))
>
> Is that path portable across targets? Should we maybe instead manually add a virtio-scsi device that we can give a proper name?
>
> Or I suppose we could do
>
> path = next(dev for dev in vm.qmp('query-block')['return']
> if dev['device'] == 'drive0')['qdev']
OK
>
>> + else:
>> + log(vm.qmp("blockdev-backup",
>> + device=src_node,
>> + target=tgt_node,
>> + sync="none"))
>> log('')
>> log('--- Setting up NBD Export ---')
>> @@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
>> log('--- Cleanup ---')
>> log('')
>> - log(vm.qmp('block-job-cancel', device=src_node))
>> - log(vm.event_wait('BLOCK_JOB_CANCELLED'),
>> - filters=[iotests.filter_qmp_event])
>> + if use_cbw:
>> + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
>> + property="drive", value="node0"))
>
> If src_node were 'node0', we wouldn’t need to use the 'node0' literal here and could use src_node instead. (Because it kind of seems like this test wants to use those variables instead of literals. I mean, we could also just call the node 'src-node', but, well.)
>
> Max
>
>> + log(vm.qmp("blockdev-del", node_name="fl-cbw"))
>> + else:
>> + log(vm.qmp('block-job-cancel', device=src_node))
>> + log(vm.event_wait('BLOCK_JOB_CANCELLED'),
>> + filters=[iotests.filter_qmp_event])
>> +
>> log(vm.qmp('nbd-server-stop'))
>> log(vm.qmp('blockdev-del', node_name=tgt_node))
>> vm.shutdown()
>
Great thanks for reviewing!
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 66+ messages in thread