All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, eblake@redhat.com, armbru@redhat.com,
	crosa@redhat.com, ehabkost@redhat.com, berrange@redhat.com,
	pbonzini@redhat.com, vsementsov@virtuozzo.com, jsnow@redhat.com,
	hreitz@redhat.com, kwolf@redhat.com,
	Max Reitz <mreitz@redhat.com>
Subject: [PATCH v8 09/34] block/backup: move cluster size calculation to block-copy
Date: Tue, 24 Aug 2021 11:38:31 +0300	[thread overview]
Message-ID: <20210824083856.17408-10-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20210824083856.17408-1-vsementsov@virtuozzo.com>

The main consumer of cluster-size is block-copy. Let's calculate it
here instead of passing through backup-top.

We are going to publish copy-before-write filter soon, so it will be
created through options. But we don't want for now to make explicit
option for cluster-size, let's continue to calculate it automatically.
So, now is the time to get rid of cluster_size argument for
bdrv_cbw_append().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 block/copy-before-write.h  |  1 -
 include/block/block-copy.h |  5 +--
 block/backup.c             | 62 ++++++--------------------------------
 block/block-copy.c         | 51 ++++++++++++++++++++++++++++++-
 block/copy-before-write.c  | 10 +++---
 5 files changed, 66 insertions(+), 63 deletions(-)

diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index 538aab8bdb..b386fd8f01 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -32,7 +32,6 @@
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index dca6c4ce36..b8a2d63545 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -25,8 +25,8 @@ typedef struct BlockCopyState BlockCopyState;
 typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size, bool use_copy_range,
-                                     bool compress, Error **errp);
+                                     bool use_copy_range, bool compress,
+                                     Error **errp);
 
 /* Function should be called prior any actual copy request */
 void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
@@ -91,6 +91,7 @@ void block_copy_kick(BlockCopyCallState *call_state);
 void block_copy_call_cancel(BlockCopyCallState *call_state);
 
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
+int64_t block_copy_cluster_size(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
 #endif /* BLOCK_COPY_H */
diff --git a/block/backup.c b/block/backup.c
index b31fd99ab3..83516297cb 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -29,8 +29,6 @@
 
 #include "block/copy-before-write.h"
 
-#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
-
 typedef struct BackupBlockJob {
     BlockJob common;
     BlockDriverState *cbw;
@@ -354,43 +352,6 @@ static const BlockJobDriver backup_job_driver = {
     .set_speed = backup_set_speed,
 };
 
-static int64_t backup_calculate_cluster_size(BlockDriverState *target,
-                                             Error **errp)
-{
-    int ret;
-    BlockDriverInfo bdi;
-    bool target_does_cow = bdrv_backing_chain_next(target);
-
-    /*
-     * If there is no backing file on the target, we cannot rely on COW if our
-     * backup cluster size is smaller than the target cluster size. Even for
-     * targets with a backing file, try to avoid COW if possible.
-     */
-    ret = bdrv_get_info(target, &bdi);
-    if (ret == -ENOTSUP && !target_does_cow) {
-        /* Cluster size is not defined */
-        warn_report("The target block device doesn't provide "
-                    "information about the block size and it doesn't have a "
-                    "backing file. The default block size of %u bytes is "
-                    "used. If the actual block size of the target exceeds "
-                    "this default, the backup may be unusable",
-                    BACKUP_CLUSTER_SIZE_DEFAULT);
-        return BACKUP_CLUSTER_SIZE_DEFAULT;
-    } else if (ret < 0 && !target_does_cow) {
-        error_setg_errno(errp, -ret,
-            "Couldn't determine the cluster size of the target image, "
-            "which has no backing file");
-        error_append_hint(errp,
-            "Aborting, since this may create an unusable destination image\n");
-        return ret;
-    } else if (ret < 0 && target_does_cow) {
-        /* Not fatal; just trudge on ahead. */
-        return BACKUP_CLUSTER_SIZE_DEFAULT;
-    }
-
-    return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
-}
-
 BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   BlockDriverState *target, int64_t speed,
                   MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
@@ -448,11 +409,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         return NULL;
     }
 
-    cluster_size = backup_calculate_cluster_size(target, errp);
-    if (cluster_size < 0) {
-        goto error;
-    }
-
     if (perf->max_workers < 1) {
         error_setg(errp, "max-workers must be greater than zero");
         return NULL;
@@ -464,13 +420,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         return NULL;
     }
 
-    if (perf->max_chunk && perf->max_chunk < cluster_size) {
-        error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
-                   "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
-        return NULL;
-    }
-
-
     if (sync_bitmap) {
         /* If we need to write to this bitmap, check that we can: */
         if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -503,12 +452,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
-    cbw = bdrv_cbw_append(bs, target, filter_node_name,
-                          cluster_size, false, &bcs, errp);
+    cbw = bdrv_cbw_append(bs, target, filter_node_name, false, &bcs, errp);
     if (!cbw) {
         goto error;
     }
 
+    cluster_size = block_copy_cluster_size(bcs);
+
+    if (perf->max_chunk && perf->max_chunk < cluster_size) {
+        error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+                   "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+        goto error;
+    }
+
     /* job->len is fixed, so we can't allow resize */
     job = block_job_create(job_id, &backup_job_driver, txn, cbw,
                            0, BLK_PERM_ALL,
diff --git a/block/block-copy.c b/block/block-copy.c
index e83870ff81..b0e0a38330 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -27,6 +27,7 @@
 #define BLOCK_COPY_MAX_MEM (128 * MiB)
 #define BLOCK_COPY_MAX_WORKERS 64
 #define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */
+#define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16)
 
 typedef enum {
     COPY_READ_WRITE_CLUSTER,
@@ -342,14 +343,57 @@ void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
     }
 }
 
+static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
+                                                 Error **errp)
+{
+    int ret;
+    BlockDriverInfo bdi;
+    bool target_does_cow = bdrv_backing_chain_next(target);
+
+    /*
+     * If there is no backing file on the target, we cannot rely on COW if our
+     * backup cluster size is smaller than the target cluster size. Even for
+     * targets with a backing file, try to avoid COW if possible.
+     */
+    ret = bdrv_get_info(target, &bdi);
+    if (ret == -ENOTSUP && !target_does_cow) {
+        /* Cluster size is not defined */
+        warn_report("The target block device doesn't provide "
+                    "information about the block size and it doesn't have a "
+                    "backing file. The default block size of %u bytes is "
+                    "used. If the actual block size of the target exceeds "
+                    "this default, the backup may be unusable",
+                    BLOCK_COPY_CLUSTER_SIZE_DEFAULT);
+        return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+    } else if (ret < 0 && !target_does_cow) {
+        error_setg_errno(errp, -ret,
+            "Couldn't determine the cluster size of the target image, "
+            "which has no backing file");
+        error_append_hint(errp,
+            "Aborting, since this may create an unusable destination image\n");
+        return ret;
+    } else if (ret < 0 && target_does_cow) {
+        /* Not fatal; just trudge on ahead. */
+        return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+    }
+
+    return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
+}
+
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size, bool use_copy_range,
+                                     bool use_copy_range,
                                      bool compress, Error **errp)
 {
     BlockCopyState *s;
+    int64_t cluster_size;
     BdrvDirtyBitmap *copy_bitmap;
     bool is_fleecing;
 
+    cluster_size = block_copy_calculate_cluster_size(target->bs, errp);
+    if (cluster_size < 0) {
+        return NULL;
+    }
+
     copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL,
                                            errp);
     if (!copy_bitmap) {
@@ -960,6 +1004,11 @@ BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
     return s->copy_bitmap;
 }
 
+int64_t block_copy_cluster_size(BlockCopyState *s)
+{
+    return s->cluster_size;
+}
+
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
 {
     qatomic_set(&s->skip_unallocated, skip);
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 235251a640..a7996d54db 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -37,7 +37,6 @@
 typedef struct BDRVCopyBeforeWriteState {
     BlockCopyState *bcs;
     BdrvChild *target;
-    int64_t cluster_size;
 } BDRVCopyBeforeWriteState;
 
 static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
     uint64_t off, end;
+    int64_t cluster_size = block_copy_cluster_size(s->bcs);
 
     if (flags & BDRV_REQ_WRITE_UNCHANGED) {
         return 0;
     }
 
-    off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
-    end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
+    off = QEMU_ALIGN_DOWN(offset, cluster_size);
+    end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
 
     return block_copy(s->bcs, off, end - off, true);
 }
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     }
     appended = true;
 
-    state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, false, compress, errp);
+                                      false, compress, errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
-- 
2.29.2



  parent reply	other threads:[~2021-08-24  8:49 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-24  8:38 [PATCH v8 00/34] block: publish backup-top filter Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 01/34] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 02/34] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 03/34] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 04/34] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 05/34] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 06/34] block-copy: move detecting fleecing scheme to block-copy Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 07/34] block/block-copy: introduce block_copy_set_copy_opts() Vladimir Sementsov-Ogievskiy
2021-08-24 13:32   ` Hanna Reitz
2021-08-24  8:38 ` [PATCH v8 08/34] block/backup: set copy_range and compress after filter insertion Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` Vladimir Sementsov-Ogievskiy [this message]
2021-09-01 11:57   ` [PATCH v8 09/34] block/backup: move cluster size calculation to block-copy Hanna Reitz
2021-09-01 12:04     ` Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 10/34] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 11/34] block/copy-before-write: drop extra bdrv_unref on failure path Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 12/34] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 13/34] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 14/34] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 15/34] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 16/34] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 17/34] block/copy-before-write: bdrv_cbw_append(): drop unused compress arg Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 18/34] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 19/34] block/copy-before-write: initialize block-copy bitmap Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 20/34] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 21/34] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 22/34] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 23/34] python/qemu/machine.py: refactor _qemu_args() Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 24/34] python/qemu/machine: QEMUMachine: improve qmp() method Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 25/34] python:QEMUMachine: template typing for self returning methods Vladimir Sementsov-Ogievskiy
2021-08-24 13:56   ` Hanna Reitz
2021-08-24  8:38 ` [PATCH v8 26/34] iotests/222: fix pylint and mypy complains Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 27/34] iotests/222: constantly use single quotes for strings Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 28/34] iotests: move 222 to tests/image-fleecing Vladimir Sementsov-Ogievskiy
2021-09-01 12:37   ` Hanna Reitz
2021-09-01 12:47     ` Vladimir Sementsov-Ogievskiy
2021-09-01 12:54       ` Hanna Reitz
2021-09-01 13:00     ` Daniel P. Berrangé
2021-08-24  8:38 ` [PATCH v8 29/34] iotests.py: hmp_qemu_io: support qdev Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 30/34] iotests/image-fleecing: proper source device Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 31/34] iotests/image-fleecing: rename tgt_node Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 32/34] iotests/image-fleecing: prepare for adding new test-case Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 33/34] iotests/image-fleecing: add test-case for copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-08-24  8:38 ` [PATCH v8 34/34] block/block-copy: block_copy_state_new(): drop extra arguments Vladimir Sementsov-Ogievskiy
2021-08-24 14:05   ` Hanna Reitz
2021-08-24 15:51 ` [PATCH v8 00/34] block: publish backup-top filter Hanna Reitz
2021-08-24 16:24   ` Vladimir Sementsov-Ogievskiy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210824083856.17408-10-vsementsov@virtuozzo.com \
    --to=vsementsov@virtuozzo.com \
    --cc=armbru@redhat.com \
    --cc=berrange@redhat.com \
    --cc=crosa@redhat.com \
    --cc=eblake@redhat.com \
    --cc=ehabkost@redhat.com \
    --cc=hreitz@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.