All of lore.kernel.org
 help / color / mirror / Atom feed
From: Hanna Reitz <hreitz@redhat.com>
To: qemu-block@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
	Peter Maydell <peter.maydell@linaro.org>,
	Hanna Reitz <hreitz@redhat.com>,
	qemu-devel@nongnu.org
Subject: [PULL 29/56] block/backup: move cluster size calculation to block-copy
Date: Wed,  1 Sep 2021 17:15:52 +0200	[thread overview]
Message-ID: <20210901151619.689075-30-hreitz@redhat.com> (raw)
In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com>

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The main consumer of cluster-size is block-copy. Let's calculate it
here instead of passing through backup-top.

We are going to publish copy-before-write filter soon, so it will be
created through options. But we don't want for now to make explicit
option for cluster-size, let's continue to calculate it automatically.
So, now is the time to get rid of cluster_size argument for
bdrv_cbw_append().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210824083856.17408-10-vsementsov@virtuozzo.com>
[hreitz: Add qemu/error-report.h include to block/block-copy.c]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/copy-before-write.h  |  1 -
 include/block/block-copy.h |  5 +--
 block/backup.c             | 62 ++++++--------------------------------
 block/block-copy.c         | 52 +++++++++++++++++++++++++++++++-
 block/copy-before-write.c  | 10 +++---
 5 files changed, 67 insertions(+), 63 deletions(-)

diff --git a/block/copy-before-write.h b/block/copy-before-write.h
index 538aab8bdb..b386fd8f01 100644
--- a/block/copy-before-write.h
+++ b/block/copy-before-write.h
@@ -32,7 +32,6 @@
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp);
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index dca6c4ce36..b8a2d63545 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -25,8 +25,8 @@ typedef struct BlockCopyState BlockCopyState;
 typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size, bool use_copy_range,
-                                     bool compress, Error **errp);
+                                     bool use_copy_range, bool compress,
+                                     Error **errp);
 
 /* Function should be called prior any actual copy request */
 void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
@@ -91,6 +91,7 @@ void block_copy_kick(BlockCopyCallState *call_state);
 void block_copy_call_cancel(BlockCopyCallState *call_state);
 
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
+int64_t block_copy_cluster_size(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
 #endif /* BLOCK_COPY_H */
diff --git a/block/backup.c b/block/backup.c
index b31fd99ab3..83516297cb 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -29,8 +29,6 @@
 
 #include "block/copy-before-write.h"
 
-#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
-
 typedef struct BackupBlockJob {
     BlockJob common;
     BlockDriverState *cbw;
@@ -354,43 +352,6 @@ static const BlockJobDriver backup_job_driver = {
     .set_speed = backup_set_speed,
 };
 
-static int64_t backup_calculate_cluster_size(BlockDriverState *target,
-                                             Error **errp)
-{
-    int ret;
-    BlockDriverInfo bdi;
-    bool target_does_cow = bdrv_backing_chain_next(target);
-
-    /*
-     * If there is no backing file on the target, we cannot rely on COW if our
-     * backup cluster size is smaller than the target cluster size. Even for
-     * targets with a backing file, try to avoid COW if possible.
-     */
-    ret = bdrv_get_info(target, &bdi);
-    if (ret == -ENOTSUP && !target_does_cow) {
-        /* Cluster size is not defined */
-        warn_report("The target block device doesn't provide "
-                    "information about the block size and it doesn't have a "
-                    "backing file. The default block size of %u bytes is "
-                    "used. If the actual block size of the target exceeds "
-                    "this default, the backup may be unusable",
-                    BACKUP_CLUSTER_SIZE_DEFAULT);
-        return BACKUP_CLUSTER_SIZE_DEFAULT;
-    } else if (ret < 0 && !target_does_cow) {
-        error_setg_errno(errp, -ret,
-            "Couldn't determine the cluster size of the target image, "
-            "which has no backing file");
-        error_append_hint(errp,
-            "Aborting, since this may create an unusable destination image\n");
-        return ret;
-    } else if (ret < 0 && target_does_cow) {
-        /* Not fatal; just trudge on ahead. */
-        return BACKUP_CLUSTER_SIZE_DEFAULT;
-    }
-
-    return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
-}
-
 BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   BlockDriverState *target, int64_t speed,
                   MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
@@ -448,11 +409,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         return NULL;
     }
 
-    cluster_size = backup_calculate_cluster_size(target, errp);
-    if (cluster_size < 0) {
-        goto error;
-    }
-
     if (perf->max_workers < 1) {
         error_setg(errp, "max-workers must be greater than zero");
         return NULL;
@@ -464,13 +420,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         return NULL;
     }
 
-    if (perf->max_chunk && perf->max_chunk < cluster_size) {
-        error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
-                   "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
-        return NULL;
-    }
-
-
     if (sync_bitmap) {
         /* If we need to write to this bitmap, check that we can: */
         if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -503,12 +452,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
-    cbw = bdrv_cbw_append(bs, target, filter_node_name,
-                          cluster_size, false, &bcs, errp);
+    cbw = bdrv_cbw_append(bs, target, filter_node_name, false, &bcs, errp);
     if (!cbw) {
         goto error;
     }
 
+    cluster_size = block_copy_cluster_size(bcs);
+
+    if (perf->max_chunk && perf->max_chunk < cluster_size) {
+        error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+                   "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+        goto error;
+    }
+
     /* job->len is fixed, so we can't allow resize */
     job = block_job_create(job_id, &backup_job_driver, txn, cbw,
                            0, BLK_PERM_ALL,
diff --git a/block/block-copy.c b/block/block-copy.c
index e83870ff81..5d0076ac93 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -21,12 +21,14 @@
 #include "qemu/units.h"
 #include "qemu/coroutine.h"
 #include "block/aio_task.h"
+#include "qemu/error-report.h"
 
 #define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB)
 #define BLOCK_COPY_MAX_BUFFER (1 * MiB)
 #define BLOCK_COPY_MAX_MEM (128 * MiB)
 #define BLOCK_COPY_MAX_WORKERS 64
 #define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */
+#define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16)
 
 typedef enum {
     COPY_READ_WRITE_CLUSTER,
@@ -342,14 +344,57 @@ void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range,
     }
 }
 
+static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
+                                                 Error **errp)
+{
+    int ret;
+    BlockDriverInfo bdi;
+    bool target_does_cow = bdrv_backing_chain_next(target);
+
+    /*
+     * If there is no backing file on the target, we cannot rely on COW if our
+     * backup cluster size is smaller than the target cluster size. Even for
+     * targets with a backing file, try to avoid COW if possible.
+     */
+    ret = bdrv_get_info(target, &bdi);
+    if (ret == -ENOTSUP && !target_does_cow) {
+        /* Cluster size is not defined */
+        warn_report("The target block device doesn't provide "
+                    "information about the block size and it doesn't have a "
+                    "backing file. The default block size of %u bytes is "
+                    "used. If the actual block size of the target exceeds "
+                    "this default, the backup may be unusable",
+                    BLOCK_COPY_CLUSTER_SIZE_DEFAULT);
+        return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+    } else if (ret < 0 && !target_does_cow) {
+        error_setg_errno(errp, -ret,
+            "Couldn't determine the cluster size of the target image, "
+            "which has no backing file");
+        error_append_hint(errp,
+            "Aborting, since this may create an unusable destination image\n");
+        return ret;
+    } else if (ret < 0 && target_does_cow) {
+        /* Not fatal; just trudge on ahead. */
+        return BLOCK_COPY_CLUSTER_SIZE_DEFAULT;
+    }
+
+    return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
+}
+
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size, bool use_copy_range,
+                                     bool use_copy_range,
                                      bool compress, Error **errp)
 {
     BlockCopyState *s;
+    int64_t cluster_size;
     BdrvDirtyBitmap *copy_bitmap;
     bool is_fleecing;
 
+    cluster_size = block_copy_calculate_cluster_size(target->bs, errp);
+    if (cluster_size < 0) {
+        return NULL;
+    }
+
     copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL,
                                            errp);
     if (!copy_bitmap) {
@@ -960,6 +1005,11 @@ BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
     return s->copy_bitmap;
 }
 
+int64_t block_copy_cluster_size(BlockCopyState *s)
+{
+    return s->cluster_size;
+}
+
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip)
 {
     qatomic_set(&s->skip_unallocated, skip);
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 235251a640..a7996d54db 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -37,7 +37,6 @@
 typedef struct BDRVCopyBeforeWriteState {
     BlockCopyState *bcs;
     BdrvChild *target;
-    int64_t cluster_size;
 } BDRVCopyBeforeWriteState;
 
 static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
     uint64_t off, end;
+    int64_t cluster_size = block_copy_cluster_size(s->bcs);
 
     if (flags & BDRV_REQ_WRITE_UNCHANGED) {
         return 0;
     }
 
-    off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
-    end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
+    off = QEMU_ALIGN_DOWN(offset, cluster_size);
+    end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
 
     return block_copy(s->bcs, off, end - off, true);
 }
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     }
     appended = true;
 
-    state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, false, compress, errp);
+                                      false, compress, errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
-- 
2.31.1



  parent reply	other threads:[~2021-09-01 15:49 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-01 15:15 [PULL 00/56] Block patches Hanna Reitz
2021-09-01 15:15 ` [PULL 01/56] python: qemu: add timer parameter for qmp.accept socket Hanna Reitz
2021-09-01 15:15 ` [PULL 02/56] python: Reduce strictness of pylint's duplicate-code check Hanna Reitz
2021-09-01 15:15 ` [PULL 03/56] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine Hanna Reitz
2021-09-01 15:15 ` [PULL 04/56] docs/devel/testing: add debug section to the QEMU iotests chapter Hanna Reitz
2021-09-01 15:15 ` [PULL 05/56] qemu-iotests: add option to attach gdbserver Hanna Reitz
2021-09-01 15:15 ` [PULL 06/56] qemu-iotests: delay QMP socket timers Hanna Reitz
2021-09-01 15:15 ` [PULL 07/56] qemu_iotests: insert gdbserver command line as wrapper for qemu binary Hanna Reitz
2021-09-01 15:15 ` [PULL 08/56] qemu-iotests: add gdbserver option to script tests too Hanna Reitz
2021-09-01 15:15 ` [PULL 09/56] docs/devel/testing: add -gdb option to the debugging section of QEMU iotests Hanna Reitz
2021-09-01 15:15 ` [PULL 10/56] qemu-iotests: extend the check script to prepare supporting valgrind for python tests Hanna Reitz
2021-09-01 15:15 ` [PULL 11/56] qemu-iotests: extend QMP socket timeout when using valgrind Hanna Reitz
2021-09-01 15:15 ` [PULL 12/56] qemu-iotests: allow valgrind to read/delete the generated log file Hanna Reitz
2021-09-01 15:15 ` [PULL 13/56] qemu-iotests: insert valgrind command line as wrapper for qemu binary Hanna Reitz
2021-09-01 15:15 ` [PULL 14/56] docs/devel/testing: add -valgrind option to the debug section of QEMU iotests Hanna Reitz
2021-09-01 15:15 ` [PULL 15/56] qemu-iotests: add option to show qemu binary logs on stdout Hanna Reitz
2021-09-01 15:15 ` [PULL 16/56] docs/devel/testing: add -p option to the debug section of QEMU iotests Hanna Reitz
2021-09-01 15:15 ` [PULL 17/56] block/monitor: Consolidate hmp_handle_error calls to reduce redundant code Hanna Reitz
2021-09-01 15:15 ` [PULL 18/56] raw-format: drop WRITE and RESIZE child perms when possible Hanna Reitz
2021-09-01 15:15 ` [PULL 19/56] iotests: use with-statement for open() calls Hanna Reitz
2021-09-01 15:15 ` [PULL 20/56] iotests: use subprocess.DEVNULL instead of open("/dev/null") Hanna Reitz
2021-09-01 15:15 ` [PULL 21/56] block: introduce bdrv_replace_child_bs() Hanna Reitz
2021-09-01 15:15 ` [PULL 22/56] block: introduce blk_replace_bs Hanna Reitz
2021-09-01 15:15 ` [PULL 23/56] qdev-properties: PropertyInfo: add realized_set_allowed field Hanna Reitz
2021-09-01 15:15 ` [PULL 24/56] qdev: allow setting drive property for realized device Hanna Reitz
2021-09-01 15:15 ` [PULL 25/56] block: rename backup-top to copy-before-write Hanna Reitz
2021-09-01 15:15 ` [PULL 26/56] block-copy: move detecting fleecing scheme to block-copy Hanna Reitz
2021-09-01 15:15 ` [PULL 27/56] block/block-copy: introduce block_copy_set_copy_opts() Hanna Reitz
2021-09-01 15:15 ` [PULL 28/56] block/backup: set copy_range and compress after filter insertion Hanna Reitz
2021-09-01 15:15 ` Hanna Reitz [this message]
2021-09-01 15:15 ` [PULL 30/56] block/copy-before-write: relax permission requirements when no parents Hanna Reitz
2021-09-01 15:15 ` [PULL 31/56] block/copy-before-write: drop extra bdrv_unref on failure path Hanna Reitz
2021-09-01 15:15 ` [PULL 32/56] block/copy-before-write: use file child instead of backing Hanna Reitz
2021-09-01 15:15 ` [PULL 33/56] block/copy-before-write: bdrv_cbw_append(): replace child at last Hanna Reitz
2021-09-01 15:15 ` [PULL 34/56] block/copy-before-write: introduce cbw_init() Hanna Reitz
2021-09-01 15:15 ` [PULL 35/56] block/copy-before-write: cbw_init(): rename variables Hanna Reitz
2021-09-01 15:15 ` [PULL 36/56] block/copy-before-write: cbw_init(): use file child after attaching Hanna Reitz
2021-09-01 15:16 ` [PULL 37/56] block/copy-before-write: bdrv_cbw_append(): drop unused compress arg Hanna Reitz
2021-09-01 15:16 ` [PULL 38/56] block/copy-before-write: cbw_init(): use options Hanna Reitz
2021-09-01 15:16 ` [PULL 39/56] block/copy-before-write: initialize block-copy bitmap Hanna Reitz
2021-09-01 15:16 ` [PULL 40/56] block/block-copy: make setting progress optional Hanna Reitz
2021-09-01 15:16 ` [PULL 41/56] block/copy-before-write: make public block driver Hanna Reitz
2021-09-20  9:41   ` Kevin Wolf
2021-09-20 10:08     ` Vladimir Sementsov-Ogievskiy
2021-09-01 15:16 ` [PULL 42/56] qapi: publish copy-before-write filter Hanna Reitz
2021-09-01 15:16 ` [PULL 43/56] python/qemu/machine.py: refactor _qemu_args() Hanna Reitz
2021-09-01 15:16 ` [PULL 44/56] python/qemu/machine: QEMUMachine: improve qmp() method Hanna Reitz
2021-09-01 15:16 ` [PULL 45/56] python:QEMUMachine: template typing for self returning methods Hanna Reitz
2021-09-01 15:16 ` [PULL 46/56] iotests/222: fix pylint and mypy complains Hanna Reitz
2021-09-01 15:16 ` [PULL 47/56] iotests/222: constantly use single quotes for strings Hanna Reitz
2021-09-01 15:16 ` [PULL 48/56] iotests: move 222 to tests/image-fleecing Hanna Reitz
2021-09-01 15:16 ` [PULL 49/56] iotests.py: hmp_qemu_io: support qdev Hanna Reitz
2021-09-01 15:16 ` [PULL 50/56] iotests/image-fleecing: proper source device Hanna Reitz
2021-09-01 15:16 ` [PULL 51/56] iotests/image-fleecing: rename tgt_node Hanna Reitz
2021-09-01 15:16 ` [PULL 52/56] iotests/image-fleecing: prepare for adding new test-case Hanna Reitz
2021-09-01 15:16 ` [PULL 53/56] iotests/image-fleecing: add test-case for copy-before-write filter Hanna Reitz
2021-09-01 15:16 ` [PULL 54/56] block/block-copy: block_copy_state_new(): drop extra arguments Hanna Reitz
2021-09-01 15:16 ` [PULL 55/56] block/export/fuse.c: fix fuse-lseek on uclibc or musl Hanna Reitz
2021-09-01 15:16 ` [PULL 56/56] block/file-win32: add reopen handlers Hanna Reitz
2021-09-02 11:07 ` [PULL 00/56] Block patches Peter Maydell
2021-09-02 11:21   ` Hanna Reitz
2021-09-02 13:56 ` Peter Maydell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210901151619.689075-30-hreitz@redhat.com \
    --to=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.