All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: qemu-block@nongnu.org, qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>, Fam Zheng <fam@euphon.net>,
	vsementsov@virtuozzo.com, Juan Quintela <quintela@redhat.com>,
	John Snow <jsnow@redhat.com>,
	Xie Changlong <xiechanglong.d@gmail.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Max Reitz <mreitz@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Wen Congyang <wencongyang2@huawei.com>,
	Markus Armbruster <armbru@redhat.com>
Subject: [Qemu-devel] [PATCH v4 12/18] block/backup: add 'always' bitmap sync policy
Date: Tue,  9 Jul 2019 19:25:44 -0400	[thread overview]
Message-ID: <20190709232550.10724-13-jsnow@redhat.com> (raw)
In-Reply-To: <20190709232550.10724-1-jsnow@redhat.com>

This adds an "always" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, the bitmap is *always* synchronized. This means
that for backups that fail part-way through, the bitmap retains a record of
which sectors need to be copied out to accomplish a new backup using the
old, partial result.

In effect, this allows us to "resume" a failed backup; however the new backup
will be from the new point in time, so it isn't a "resume" as much as it is
an "incremental retry." This can be useful in the case of extremely large
backups that fail considerably through the operation and we'd like to not waste
the work that was already performed.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
 block/backup.c       | 27 +++++++++++++++++++--------
 qapi/block-core.json |  5 ++++-
 2 files changed, 23 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index fb1b39c44e..acfe87b756 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -268,18 +268,29 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
 {
     BdrvDirtyBitmap *bm;
     BlockDriverState *bs = blk_bs(job->common.blk);
+    bool sync = (((ret == 0) || (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS)) \
+                 && (job->bitmap_mode != BITMAP_SYNC_MODE_NEVER));
 
-    if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
+    if (sync) {
         /*
-         * Failure, or we don't want to synchronize the bitmap.
-         * Merge the successor back into the parent, delete nothing.
+         * We succeeded, or we always intended to sync the bitmap.
+         * Delete this bitmap and install the child.
          */
-        bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
-        assert(bm);
-    } else {
-        /* Everything is fine, delete this bitmap and install the backup. */
         bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
-        assert(bm);
+    } else {
+        /*
+         * We failed, or we never intended to sync the bitmap anyway.
+         * Merge the successor back into the parent, keeping all data.
+         */
+        bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
+    }
+
+    assert(bm);
+
+    if (ret < 0 && job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS) {
+        /* If we failed and synced, merge in the bits we didn't copy: */
+        bdrv_dirty_bitmap_merge_internal(bm, job->copy_bitmap,
+                                         NULL, true);
     }
 }
 
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b1aaaaa98e..5a578806c5 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1149,10 +1149,13 @@
 # @never: The bitmap is never synchronized with the operation, and is
 #         treated solely as a read-only manifest of blocks to copy.
 #
+# @always: The bitmap is always synchronized with the operation,
+#          regardless of whether or not the operation was successful.
+#
 # Since: 4.2
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['on-success', 'never'] }
+  'data': ['on-success', 'never', 'always'] }
 
 ##
 # @MirrorCopyMode:
-- 
2.21.0



  parent reply	other threads:[~2019-07-09 23:59 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-09 23:25 [Qemu-devel] [PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 01/18] qapi/block-core: Introduce BackupCommon John Snow
2019-07-10  1:36   ` John Snow
2019-07-10  4:11     ` Markus Armbruster
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 02/18] drive-backup: create do_backup_common John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 03/18] blockdev-backup: utilize do_backup_common John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 04/18] qapi: add BitmapSyncMode enum John Snow
2019-07-10  4:15   ` Markus Armbruster
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 05/18] block/backup: Add mirror sync mode 'bitmap' John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 06/18] block/backup: add 'never' policy to bitmap sync mode John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 08/18] hbitmap: enable merging across granularities John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal John Snow
2019-07-10 14:38   ` Max Reitz
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap John Snow
2019-07-10 14:39   ` Max Reitz
2019-07-09 23:25 ` John Snow [this message]
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 13/18] iotests: add testing shim for script-style python tests John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 14/18] iotests: teach run_job to cancel pending jobs John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 15/18] iotests: teach FilePath to produce multiple paths John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 16/18] iotests: Add virtio-scsi device helper John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 17/18] iotests: add test 257 for bitmap-mode backups John Snow
2019-07-10 14:45   ` Max Reitz
2019-07-11  1:51   ` John Snow
2019-07-09 23:25 ` [Qemu-devel] [PATCH v4 18/18] block/backup: loosen restriction on readonly bitmaps John Snow
2019-07-10 14:48 ` [Qemu-devel] [PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode Max Reitz
2019-07-10 17:21   ` John Snow
2019-07-15 20:00 ` John Snow
2019-07-22 12:17 ` Fabian Grünbichler
2019-07-22 17:21   ` John Snow
2019-07-23  9:47     ` Fabian Grünbichler
2019-07-23 16:58       ` John Snow
2019-07-24  8:41         ` Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190709232550.10724-13-jsnow@redhat.com \
    --to=jsnow@redhat.com \
    --cc=armbru@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=fam@euphon.net \
    --cc=kwolf@redhat.com \
    --cc=mreitz@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=stefanha@redhat.com \
    --cc=vsementsov@virtuozzo.com \
    --cc=wencongyang2@huawei.com \
    --cc=xiechanglong.d@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.