All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org, qemu-block@nongnu.org
Cc: Fam Zheng <fam@euphon.net>, Kevin Wolf <kwolf@redhat.com>,
	vsementsov@virtuozzo.com, Juan Quintela <quintela@redhat.com>,
	Wen Congyang <wencongyang2@huawei.com>,
	Xie Changlong <xiechanglong.d@gmail.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Markus Armbruster <armbru@redhat.com>,
	John Snow <jsnow@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Max Reitz <mreitz@redhat.com>
Subject: [Qemu-devel] [PATCH v3 12/18] block/backup: add 'always' bitmap sync policy
Date: Fri,  5 Jul 2019 16:16:25 -0400	[thread overview]
Message-ID: <20190705201631.26266-13-jsnow@redhat.com> (raw)
In-Reply-To: <20190705201631.26266-1-jsnow@redhat.com>

This adds an "always" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, the bitmap is *always* synchronized. This means
that for backups that fail part-way through, the bitmap retains a record of
which sectors need to be copied out to accomplish a new backup using the
old, partial result.

In effect, this allows us to "resume" a failed backup; however the new backup
will be from the new point in time, so it isn't a "resume" as much as it is
an "incremental retry." This can be useful in the case of extremely large
backups that fail considerably through the operation and we'd like to not waste
the work that was already performed.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 block/backup.c       | 27 +++++++++++++++++++--------
 qapi/block-core.json |  5 ++++-
 2 files changed, 23 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 3c08d057cc..b25e6179cf 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -268,18 +268,29 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
 {
     BdrvDirtyBitmap *bm;
     BlockDriverState *bs = blk_bs(job->common.blk);
+    bool sync = (((ret == 0) || (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS)) \
+                 && (job->bitmap_mode != BITMAP_SYNC_MODE_NEVER));
 
-    if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
+    if (sync) {
         /*
-         * Failure, or we don't want to synchronize the bitmap.
-         * Merge the successor back into the parent, delete nothing.
+         * We succeeded, or we always intended to sync the bitmap.
+         * Delete this bitmap and install the child.
          */
-        bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
-        assert(bm);
-    } else {
-        /* Everything is fine, delete this bitmap and install the backup. */
         bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
-        assert(bm);
+    } else {
+        /*
+         * We failed, or we never intended to sync the bitmap anyway.
+         * Merge the successor back into the parent, keeping all data.
+         */
+        bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
+    }
+
+    assert(bm);
+
+    if (ret < 0 && job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS) {
+        /* If we failed and synced, merge in the bits we didn't copy: */
+        bdrv_dirty_bitmap_merge_internal(bm, job->copy_bitmap,
+                                         NULL, true);
     }
 }
 
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b1aaaaa98e..5a578806c5 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1149,10 +1149,13 @@
 # @never: The bitmap is never synchronized with the operation, and is
 #         treated solely as a read-only manifest of blocks to copy.
 #
+# @always: The bitmap is always synchronized with the operation,
+#          regardless of whether or not the operation was successful.
+#
 # Since: 4.2
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['on-success', 'never'] }
+  'data': ['on-success', 'never', 'always'] }
 
 ##
 # @MirrorCopyMode:
-- 
2.21.0



  parent reply	other threads:[~2019-07-05 20:27 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-05 20:16 [Qemu-devel] [PATCH v3 00/18] bitmaps: introduce 'bitmap' sync mode John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 01/18] qapi/block-core: Introduce BackupCommon John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 02/18] drive-backup: create do_backup_common John Snow
2019-07-09 18:28   ` Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 03/18] blockdev-backup: utilize do_backup_common John Snow
2019-07-09 18:33   ` Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 04/18] qapi: add BitmapSyncMode enum John Snow
2019-07-09 18:34   ` Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 05/18] block/backup: Add mirror sync mode 'bitmap' John Snow
2019-07-09 18:38   ` Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 06/18] block/backup: add 'never' policy to bitmap sync mode John Snow
2019-07-09 18:38   ` Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 08/18] hbitmap: enable merging across granularities John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap John Snow
2019-07-05 20:16 ` John Snow [this message]
2019-07-09 18:46   ` [Qemu-devel] [PATCH v3 12/18] block/backup: add 'always' bitmap sync policy Max Reitz
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 13/18] iotests: add testing shim for script-style python tests John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 14/18] iotests: teach run_job to cancel pending jobs John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 15/18] iotests: teach FilePath to produce multiple paths John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 16/18] iotests: Add virtio-scsi device helper John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 17/18] iotests: add test 257 for bitmap-mode backups John Snow
2019-07-09 18:49   ` Max Reitz
2019-07-09 23:14     ` John Snow
2019-07-05 20:16 ` [Qemu-devel] [PATCH v3 18/18] block/backup: loosen restriction on readonly bitmaps John Snow

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190705201631.26266-13-jsnow@redhat.com \
    --to=jsnow@redhat.com \
    --cc=armbru@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=fam@euphon.net \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=stefanha@redhat.com \
    --cc=vsementsov@virtuozzo.com \
    --cc=wencongyang2@huawei.com \
    --cc=xiechanglong.d@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.