From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54904) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQgs1-0002Zb-80 for qemu-devel@nongnu.org; Thu, 29 Jun 2017 17:21:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQgrw-00069S-Fd for qemu-devel@nongnu.org; Thu, 29 Jun 2017 17:21:01 -0400 References: <1498661881-9605-1-git-send-email-kchamart@redhat.com> <1498661881-9605-3-git-send-email-kchamart@redhat.com> From: John Snow Message-ID: Date: Thu, 29 Jun 2017 17:20:43 -0400 MIME-Version: 1.0 In-Reply-To: <1498661881-9605-3-git-send-email-kchamart@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH v4 2/2] bitmaps.md: Convert to rST; move it into 'interop' dir List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Kashyap Chamarthy , qemu-devel@nongnu.org Cc: kwolf@redhat.com, qemu-block@nongnu.org, stephen@that.guru, armbru@redhat.com, mreitz@redhat.com On 06/28/2017 10:58 AM, Kashyap Chamarthy wrote: > This is part of the on-going effort to convert QEMU upstream > documentation syntax to reStructuredText (rST). >=20 > The conversion to rST was done using: >=20 > $ pandoc -f markdown -t rst bitmaps.md -o bitmaps.rst >=20 > Then, make a couple of small syntactical adjustments. While at it, > reword a statement to avoid ambiguity. Addressing the feedback from > this thread: >=20 > https://lists.nongnu.org/archive/html/qemu-devel/2017-06/msg05428.h= tml >=20 > Signed-off-by: Kashyap Chamarthy Reviewed-by: John Snow > --- > * A Sphinx-rendered HTML version is here: > https://kashyapc.fedorapeople.org/v4-QEMU-Docs/_build/html/docs/bitma= ps.html >=20 > * The patch has "v4" subject prefix because I rolled this in along with > the other document (live-block-operations.rst) that is in review, > which is actually at v4. > --- > docs/devel/bitmaps.md | 505 ---------------------------------------= --- > docs/interop/bitmaps.rst | 555 +++++++++++++++++++++++++++++++++++++++= ++++++++ > 2 files changed, 555 insertions(+), 505 deletions(-) > delete mode 100644 docs/devel/bitmaps.md > create mode 100644 docs/interop/bitmaps.rst >=20 > diff --git a/docs/devel/bitmaps.md b/docs/devel/bitmaps.md > deleted file mode 100644 > index a2e8d51..0000000 > --- a/docs/devel/bitmaps.md > +++ /dev/null > @@ -1,505 +0,0 @@ > - > - > -# Dirty Bitmaps and Incremental Backup > - > -* Dirty Bitmaps are objects that track which data needs to be backed u= p for the > - next incremental backup. > - > -* Dirty bitmaps can be created at any time and attached to any node > - (not just complete drives.) > - > -## Dirty Bitmap Names > - > -* A dirty bitmap's name is unique to the node, but bitmaps attached to= different > - nodes can share the same name. > - > -* Dirty bitmaps created for internal use by QEMU may be anonymous and = have no > - name, but any user-created bitmaps may not be. There can be any numb= er of > - anonymous bitmaps per node. > - > -* The name of a user-created bitmap must not be empty (""). > - > -## Bitmap Modes > - > -* A Bitmap can be "frozen," which means that it is currently in-use by= a backup > - operation and cannot be deleted, renamed, written to, reset, > - etc. > - > -* The normal operating mode for a bitmap is "active." > - > -## Basic QMP Usage > - > -### Supported Commands ### > - > -* block-dirty-bitmap-add > -* block-dirty-bitmap-remove > -* block-dirty-bitmap-clear > - > -### Creation > - > -* To create a new bitmap, enabled, on the drive with id=3Ddrive0: > - > -```json > -{ "execute": "block-dirty-bitmap-add", > - "arguments": { > - "node": "drive0", > - "name": "bitmap0" > - } > -} > -``` > - > -* This bitmap will have a default granularity that matches the cluster= size of > - its associated drive, if available, clamped to between [4KiB, 64KiB]= . > - The current default for qcow2 is 64KiB. > - > -* To create a new bitmap that tracks changes in 32KiB segments: > - > -```json > -{ "execute": "block-dirty-bitmap-add", > - "arguments": { > - "node": "drive0", > - "name": "bitmap0", > - "granularity": 32768 > - } > -} > -``` > - > -### Deletion > - > -* Bitmaps that are frozen cannot be deleted. > - > -* Deleting the bitmap does not impact any other bitmaps attached to th= e same > - node, nor does it affect any backups already created from this node. > - > -* Because bitmaps are only unique to the node to which they are attach= ed, > - you must specify the node/drive name here, too. > - > -```json > -{ "execute": "block-dirty-bitmap-remove", > - "arguments": { > - "node": "drive0", > - "name": "bitmap0" > - } > -} > -``` > - > -### Resetting > - > -* Resetting a bitmap will clear all information it holds. > - > -* An incremental backup created from an empty bitmap will copy no data= , > - as if nothing has changed. > - > -```json > -{ "execute": "block-dirty-bitmap-clear", > - "arguments": { > - "node": "drive0", > - "name": "bitmap0" > - } > -} > -``` > - > -## Transactions > - > -### Justification > - > -Bitmaps can be safely modified when the VM is paused or halted by usin= g > -the basic QMP commands. For instance, you might perform the following = actions: > - > -1. Boot the VM in a paused state. > -2. Create a full drive backup of drive0. > -3. Create a new bitmap attached to drive0. > -4. Resume execution of the VM. > -5. Incremental backups are ready to be created. > - > -At this point, the bitmap and drive backup would be correctly in sync, > -and incremental backups made from this point forward would be correctl= y aligned > -to the full drive backup. > - > -This is not particularly useful if we decide we want to start incremen= tal > -backups after the VM has been running for a while, for which we will n= eed to > -perform actions such as the following: > - > -1. Boot the VM and begin execution. > -2. Using a single transaction, perform the following operations: > - * Create bitmap0. > - * Create a full drive backup of drive0. > -3. Incremental backups are now ready to be created. > - > -### Supported Bitmap Transactions > - > -* block-dirty-bitmap-add > -* block-dirty-bitmap-clear > - > -The usages are identical to their respective QMP commands, but see bel= ow > -for examples. > - > -### Example: New Incremental Backup > - > -As outlined in the justification, perhaps we want to create a new incr= emental > -backup chain attached to a drive. > - > -```json > -{ "execute": "transaction", > - "arguments": { > - "actions": [ > - {"type": "block-dirty-bitmap-add", > - "data": {"node": "drive0", "name": "bitmap0"} }, > - {"type": "drive-backup", > - "data": {"device": "drive0", "target": "/path/to/full_backup.im= g", > - "sync": "full", "format": "qcow2"} } > - ] > - } > -} > -``` > - > -### Example: New Incremental Backup Anchor Point > - > -Maybe we just want to create a new full backup with an existing bitmap= and > -want to reset the bitmap to track the new chain. > - > -```json > -{ "execute": "transaction", > - "arguments": { > - "actions": [ > - {"type": "block-dirty-bitmap-clear", > - "data": {"node": "drive0", "name": "bitmap0"} }, > - {"type": "drive-backup", > - "data": {"device": "drive0", "target": "/path/to/new_full_backu= p.img", > - "sync": "full", "format": "qcow2"} } > - ] > - } > -} > -``` > - > -## Incremental Backups > - > -The star of the show. > - > -**Nota Bene!** Only incremental backups of entire drives are supported= for now. > -So despite the fact that you can attach a bitmap to any arbitrary node= , they are > -only currently useful when attached to the root node. This is because > -drive-backup only supports drives/devices instead of arbitrary nodes. > - > -### Example: First Incremental Backup > - > -1. Create a full backup and sync it to the dirty bitmap, as in the tra= nsactional > -examples above; or with the VM offline, manually create a full copy an= d then > -create a new bitmap before the VM begins execution. > - > - * Let's assume the full backup is named 'full_backup.img'. > - * Let's assume the bitmap you created is 'bitmap0' attached to 'dr= ive0'. > - > -2. Create a destination image for the incremental backup that utilizes= the > -full backup as a backing image. > - > - * Let's assume it is named 'incremental.0.img'. > - > - ```sh > - # qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F= qcow2 > - ``` > - > -3. Issue the incremental backup command: > - > - ```json > - { "execute": "drive-backup", > - "arguments": { > - "device": "drive0", > - "bitmap": "bitmap0", > - "target": "incremental.0.img", > - "format": "qcow2", > - "sync": "incremental", > - "mode": "existing" > - } > - } > - ``` > - > -### Example: Second Incremental Backup > - > -1. Create a new destination image for the incremental backup that poin= ts to the > - previous one, e.g.: 'incremental.1.img' > - > - ```sh > - # qemu-img create -f qcow2 incremental.1.img -b incremental.0.img = -F qcow2 > - ``` > - > -2. Issue a new incremental backup command. The only difference here is= that we > - have changed the target image below. > - > - ```json > - { "execute": "drive-backup", > - "arguments": { > - "device": "drive0", > - "bitmap": "bitmap0", > - "target": "incremental.1.img", > - "format": "qcow2", > - "sync": "incremental", > - "mode": "existing" > - } > - } > - ``` > - > -## Errors > - > -* In the event of an error that occurs after a backup job is successfu= lly > - launched, either by a direct QMP command or a QMP transaction, the u= ser > - will receive a BLOCK_JOB_COMPLETE event with a failure message, acco= mpanied > - by a BLOCK_JOB_ERROR event. > - > -* In the case of an event being cancelled, the user will receive a > - BLOCK_JOB_CANCELLED event instead of a pair of COMPLETE and ERROR ev= ents. > - > -* In either case, the incremental backup data contained within the bit= map is > - safely rolled back, and the data within the bitmap is not lost. The = image > - file created for the failed attempt can be safely deleted. > - > -* Once the underlying problem is fixed (e.g. more storage space is fre= ed up), > - you can simply retry the incremental backup command with the same bi= tmap. > - > -### Example > - > -1. Create a target image: > - > - ```sh > - # qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F= qcow2 > - ``` > - > -2. Attempt to create an incremental backup via QMP: > - > - ```json > - { "execute": "drive-backup", > - "arguments": { > - "device": "drive0", > - "bitmap": "bitmap0", > - "target": "incremental.0.img", > - "format": "qcow2", > - "sync": "incremental", > - "mode": "existing" > - } > - } > - ``` > - > -3. Receive an event notifying us of failure: > - > - ```json > - { "timestamp": { "seconds": 1424709442, "microseconds": 844524 }, > - "data": { "speed": 0, "offset": 0, "len": 67108864, > - "error": "No space left on device", > - "device": "drive1", "type": "backup" }, > - "event": "BLOCK_JOB_COMPLETED" } > - ``` > - > -4. Delete the failed incremental, and re-create the image. > - > - ```sh > - # rm incremental.0.img > - # qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F= qcow2 > - ``` > - > -5. Retry the command after fixing the underlying problem, > - such as freeing up space on the backup volume: > - > - ```json > - { "execute": "drive-backup", > - "arguments": { > - "device": "drive0", > - "bitmap": "bitmap0", > - "target": "incremental.0.img", > - "format": "qcow2", > - "sync": "incremental", > - "mode": "existing" > - } > - } > - ``` > - > -6. Receive confirmation that the job completed successfully: > - > - ```json > - { "timestamp": { "seconds": 1424709668, "microseconds": 526525 }, > - "data": { "device": "drive1", "type": "backup", > - "speed": 0, "len": 67108864, "offset": 67108864}, > - "event": "BLOCK_JOB_COMPLETED" } > - ``` > - > -### Partial Transactional Failures > - > -* Sometimes, a transaction will succeed in launching and return succes= s, > - but then later the backup jobs themselves may fail. It is possible t= hat > - a management application may have to deal with a partial backup fail= ure > - after a successful transaction. > - > -* If multiple backup jobs are specified in a single transaction, when = one of > - them fails, it will not interact with the other backup jobs in any w= ay. > - > -* The job(s) that succeeded will clear the dirty bitmap associated wit= h the > - operation, but the job(s) that failed will not. It is not "safe" to = delete > - any incremental backups that were created successfully in this scena= rio, > - even though others failed. > - > -#### Example > - > -* QMP example highlighting two backup jobs: > - > - ```json > - { "execute": "transaction", > - "arguments": { > - "actions": [ > - { "type": "drive-backup", > - "data": { "device": "drive0", "bitmap": "bitmap0", > - "format": "qcow2", "mode": "existing", > - "sync": "incremental", "target": "d0-incr-1.qcow= 2" } }, > - { "type": "drive-backup", > - "data": { "device": "drive1", "bitmap": "bitmap1", > - "format": "qcow2", "mode": "existing", > - "sync": "incremental", "target": "d1-incr-1.qcow= 2" } }, > - ] > - } > - } > - ``` > - > -* QMP example response, highlighting one success and one failure: > - * Acknowledgement that the Transaction was accepted and jobs were = launched: > - ```json > - { "return": {} } > - ``` > - > - * Later, QEMU sends notice that the first job was completed: > - ```json > - { "timestamp": { "seconds": 1447192343, "microseconds": 615698= }, > - "data": { "device": "drive0", "type": "backup", > - "speed": 0, "len": 67108864, "offset": 67108864 }= , > - "event": "BLOCK_JOB_COMPLETED" > - } > - ``` > - > - * Later yet, QEMU sends notice that the second job has failed: > - ```json > - { "timestamp": { "seconds": 1447192399, "microseconds": 683015= }, > - "data": { "device": "drive1", "action": "report", > - "operation": "read" }, > - "event": "BLOCK_JOB_ERROR" } > - ``` > - > - ```json > - { "timestamp": { "seconds": 1447192399, "microseconds": 685853= }, > - "data": { "speed": 0, "offset": 0, "len": 67108864, > - "error": "Input/output error", > - "device": "drive1", "type": "backup" }, > - "event": "BLOCK_JOB_COMPLETED" } > - > -* In the above example, "d0-incr-1.qcow2" is valid and must be kept, > - but "d1-incr-1.qcow2" is invalid and should be deleted. If a VM-wide > - incremental backup of all drives at a point-in-time is to be made, > - new backups for both drives will need to be made, taking into accoun= t > - that a new incremental backup for drive0 needs to be based on top of > - "d0-incr-1.qcow2." > - > -### Grouped Completion Mode > - > -* While jobs launched by transactions normally complete or fail on the= ir own, > - it is possible to instruct them to complete or fail together as a gr= oup. > - > -* QMP transactions take an optional properties structure that can affe= ct > - the semantics of the transaction. > - > -* The "completion-mode" transaction property can be either "individual= " > - which is the default, legacy behavior described above, or "grouped," > - a new behavior detailed below. > - > -* Delayed Completion: In grouped completion mode, no jobs will report > - success until all jobs are ready to report success. > - > -* Grouped failure: If any job fails in grouped completion mode, all re= maining > - jobs will be cancelled. Any incremental backups will restore their d= irty > - bitmap objects as if no backup command was ever issued. > - > - * Regardless of if QEMU reports a particular incremental backup jo= b as > - CANCELLED or as an ERROR, the in-memory bitmap will be restored. > - > -#### Example > - > -* Here's the same example scenario from above with the new property: > - > - ```json > - { "execute": "transaction", > - "arguments": { > - "actions": [ > - { "type": "drive-backup", > - "data": { "device": "drive0", "bitmap": "bitmap0", > - "format": "qcow2", "mode": "existing", > - "sync": "incremental", "target": "d0-incr-1.qcow= 2" } }, > - { "type": "drive-backup", > - "data": { "device": "drive1", "bitmap": "bitmap1", > - "format": "qcow2", "mode": "existing", > - "sync": "incremental", "target": "d1-incr-1.qcow= 2" } }, > - ], > - "properties": { > - "completion-mode": "grouped" > - } > - } > - } > - ``` > - > -* QMP example response, highlighting a failure for drive2: > - * Acknowledgement that the Transaction was accepted and jobs were = launched: > - ```json > - { "return": {} } > - ``` > - > - * Later, QEMU sends notice that the second job has errored out, > - but that the first job was also cancelled: > - ```json > - { "timestamp": { "seconds": 1447193702, "microseconds": 632377= }, > - "data": { "device": "drive1", "action": "report", > - "operation": "read" }, > - "event": "BLOCK_JOB_ERROR" } > - ``` > - > - ```json > - { "timestamp": { "seconds": 1447193702, "microseconds": 640074= }, > - "data": { "speed": 0, "offset": 0, "len": 67108864, > - "error": "Input/output error", > - "device": "drive1", "type": "backup" }, > - "event": "BLOCK_JOB_COMPLETED" } > - ``` > - > - ```json > - { "timestamp": { "seconds": 1447193702, "microseconds": 640163= }, > - "data": { "device": "drive0", "type": "backup", "speed": 0, > - "len": 67108864, "offset": 16777216 }, > - "event": "BLOCK_JOB_CANCELLED" } > - ``` > - > - > diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst > new file mode 100644 > index 0000000..b154d54 > --- /dev/null > +++ b/docs/interop/bitmaps.rst > @@ -0,0 +1,555 @@ > +.. > + Copyright 2015 John Snow and Red Hat, Inc. > + All rights reserved. > + > + This file is licensed via The FreeBSD Documentation License, the fu= ll > + text of which is included at the end of this document. > + > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Dirty Bitmaps and Incremental Backup > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +- Dirty Bitmaps are objects that track which data needs to be backed = up > + for the next incremental backup. > + > +- Dirty bitmaps can be created at any time and attached to any node > + (not just complete drives.) > + > +.. contents:: > + > +Dirty Bitmap Names > +------------------ > + > +- A dirty bitmap's name is unique to the node, but bitmaps attached t= o > + different nodes can share the same name. > + > +- Dirty bitmaps created for internal use by QEMU may be anonymous and > + have no name, but any user-created bitmaps may not be. There can be > + any number of anonymous bitmaps per node. > + > +- The name of a user-created bitmap must not be empty (""). > + > +Bitmap Modes > +------------ > + > +- A bitmap can be "frozen," which means that it is currently in-use b= y > + a backup operation and cannot be deleted, renamed, written to, rese= t, > + etc. > + > +- The normal operating mode for a bitmap is "active." > + > +Basic QMP Usage > +--------------- > + > +Supported Commands > +~~~~~~~~~~~~~~~~~~ > + > +- ``block-dirty-bitmap-add`` > +- ``block-dirty-bitmap-remove`` > +- ``block-dirty-bitmap-clear`` > + > +Creation > +~~~~~~~~ > + > +- To create a new bitmap, enabled, on the drive with id=3Ddrive0: > + > +.. code:: json > + > + { "execute": "block-dirty-bitmap-add", > + "arguments": { > + "node": "drive0", > + "name": "bitmap0" > + } > + } > + > +- This bitmap will have a default granularity that matches the cluste= r > + size of its associated drive, if available, clamped to between [4Ki= B, > + 64KiB]. The current default for qcow2 is 64KiB. > + > +- To create a new bitmap that tracks changes in 32KiB segments: > + > +.. code:: json > + > + { "execute": "block-dirty-bitmap-add", > + "arguments": { > + "node": "drive0", > + "name": "bitmap0", > + "granularity": 32768 > + } > + } > + > +Deletion > +~~~~~~~~ > + > +- Bitmaps that are frozen cannot be deleted. > + > +- Deleting the bitmap does not impact any other bitmaps attached to t= he > + same node, nor does it affect any backups already created from this > + node. > + > +- Because bitmaps are only unique to the node to which they are > + attached, you must specify the node/drive name here, too. > + > +.. code:: json > + > + { "execute": "block-dirty-bitmap-remove", > + "arguments": { > + "node": "drive0", > + "name": "bitmap0" > + } > + } > + > +Resetting > +~~~~~~~~~ > + > +- Resetting a bitmap will clear all information it holds. > + > +- An incremental backup created from an empty bitmap will copy no dat= a, > + as if nothing has changed. > + > +.. code:: json > + > + { "execute": "block-dirty-bitmap-clear", > + "arguments": { > + "node": "drive0", > + "name": "bitmap0" > + } > + } > + > +Transactions > +------------ > + > +Justification > +~~~~~~~~~~~~~ > + > +Bitmaps can be safely modified when the VM is paused or halted by usin= g > +the basic QMP commands. For instance, you might perform the following > +actions: > + > +1. Boot the VM in a paused state. > +2. Create a full drive backup of drive0. > +3. Create a new bitmap attached to drive0. > +4. Resume execution of the VM. > +5. Incremental backups are ready to be created. > + > +At this point, the bitmap and drive backup would be correctly in sync, > +and incremental backups made from this point forward would be correctl= y > +aligned to the full drive backup. > + > +This is not particularly useful if we decide we want to start > +incremental backups after the VM has been running for a while, for whi= ch > +we will need to perform actions such as the following: > + > +1. Boot the VM and begin execution. > +2. Using a single transaction, perform the following operations: > + > + - Create ``bitmap0``. > + - Create a full drive backup of ``drive0``. > + > +3. Incremental backups are now ready to be created. > + > +Supported Bitmap Transactions > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +- ``block-dirty-bitmap-add`` > +- ``block-dirty-bitmap-clear`` > + > +The usages are identical to their respective QMP commands, but see bel= ow > +for examples. > + > +Example: New Incremental Backup > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +As outlined in the justification, perhaps we want to create a new > +incremental backup chain attached to a drive. > + > +.. code:: json > + > + { "execute": "transaction", > + "arguments": { > + "actions": [ > + {"type": "block-dirty-bitmap-add", > + "data": {"node": "drive0", "name": "bitmap0"} }, > + {"type": "drive-backup", > + "data": {"device": "drive0", "target": "/path/to/full_backu= p.img", > + "sync": "full", "format": "qcow2"} } > + ] > + } > + } > + > +Example: New Incremental Backup Anchor Point > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Maybe we just want to create a new full backup with an existing bitmap > +and want to reset the bitmap to track the new chain. > + > +.. code:: json > + > + { "execute": "transaction", > + "arguments": { > + "actions": [ > + {"type": "block-dirty-bitmap-clear", > + "data": {"node": "drive0", "name": "bitmap0"} }, > + {"type": "drive-backup", > + "data": {"device": "drive0", "target": "/path/to/new_full_b= ackup.img", > + "sync": "full", "format": "qcow2"} } > + ] > + } > + } > + > +Incremental Backups > +------------------- > + > +The star of the show. > + > +**Nota Bene!** Only incremental backups of entire drives are supported > +for now. So despite the fact that you can attach a bitmap to any > +arbitrary node, they are only currently useful when attached to the ro= ot > +node. This is because drive-backup only supports drives/devices instea= d > +of arbitrary nodes. > + > +Example: First Incremental Backup > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +1. Create a full backup and sync it to the dirty bitmap, as in the > + transactional examples above; or with the VM offline, manually crea= te > + a full copy and then create a new bitmap before the VM begins > + execution. > + > + - Let's assume the full backup is named ``full_backup.img``. > + - Let's assume the bitmap you created is ``bitmap0`` attached to > + ``drive0``. > + > +2. Create a destination image for the incremental backup that utilizes > + the full backup as a backing image. > + > + - Let's assume the new incremental image is named > + ``incremental.0.img``. > + > + .. code:: bash > + > + $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img= -F qcow2 > + > +3. Issue the incremental backup command: > + > + .. code:: json > + > + { "execute": "drive-backup", > + "arguments": { > + "device": "drive0", > + "bitmap": "bitmap0", > + "target": "incremental.0.img", > + "format": "qcow2", > + "sync": "incremental", > + "mode": "existing" > + } > + } > + > +Example: Second Incremental Backup > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +1. Create a new destination image for the incremental backup that poin= ts > + to the previous one, e.g.: ``incremental.1.img`` > + > + .. code:: bash > + > + $ qemu-img create -f qcow2 incremental.1.img -b incremental.0.i= mg -F qcow2 > + > +2. Issue a new incremental backup command. The only difference here is > + that we have changed the target image below. > + > + .. code:: json > + > + { "execute": "drive-backup", > + "arguments": { > + "device": "drive0", > + "bitmap": "bitmap0", > + "target": "incremental.1.img", > + "format": "qcow2", > + "sync": "incremental", > + "mode": "existing" > + } > + } > + > +Errors > +------ > + > +- In the event of an error that occurs after a backup job is > + successfully launched, either by a direct QMP command or a QMP > + transaction, the user will receive a ``BLOCK_JOB_COMPLETE`` event w= ith > + a failure message, accompanied by a ``BLOCK_JOB_ERROR`` event. > + > +- In the case of an event being cancelled, the user will receive a > + ``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERR= OR > + events. > + > +- In either case, the incremental backup data contained within the > + bitmap is safely rolled back, and the data within the bitmap is not > + lost. The image file created for the failed attempt can be safely > + deleted. > + > +- Once the underlying problem is fixed (e.g. more storage space is > + freed up), you can simply retry the incremental backup command with > + the same bitmap. > + > +Example > +~~~~~~~ > + > +1. Create a target image: > + > + .. code:: bash > + > + $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img= -F qcow2 > + > +2. Attempt to create an incremental backup via QMP: > + > + .. code:: json > + > + { "execute": "drive-backup", > + "arguments": { > + "device": "drive0", > + "bitmap": "bitmap0", > + "target": "incremental.0.img", > + "format": "qcow2", > + "sync": "incremental", > + "mode": "existing" > + } > + } > + > +3. Receive an event notifying us of failure: > + > + .. code:: json > + > + { "timestamp": { "seconds": 1424709442, "microseconds": 844524 = }, > + "data": { "speed": 0, "offset": 0, "len": 67108864, > + "error": "No space left on device", > + "device": "drive1", "type": "backup" }, > + "event": "BLOCK_JOB_COMPLETED" } > + > +4. Delete the failed incremental, and re-create the image. > + > + .. code:: bash > + > + $ rm incremental.0.img > + $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img= -F qcow2 > + > +5. Retry the command after fixing the underlying problem, such as > + freeing up space on the backup volume: > + > + .. code:: json > + > + { "execute": "drive-backup", > + "arguments": { > + "device": "drive0", > + "bitmap": "bitmap0", > + "target": "incremental.0.img", > + "format": "qcow2", > + "sync": "incremental", > + "mode": "existing" > + } > + } > + > +6. Receive confirmation that the job completed successfully: > + > + .. code:: json > + > + { "timestamp": { "seconds": 1424709668, "microseconds": 526525 = }, > + "data": { "device": "drive1", "type": "backup", > + "speed": 0, "len": 67108864, "offset": 67108864}, > + "event": "BLOCK_JOB_COMPLETED" } > + > +Partial Transactional Failures > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +- Sometimes, a transaction will succeed in launching and return > + success, but then later the backup jobs themselves may fail. It is > + possible that a management application may have to deal with a > + partial backup failure after a successful transaction. > + > +- If multiple backup jobs are specified in a single transaction, when > + one of them fails, it will not interact with the other backup jobs = in > + any way. > + > +- The job(s) that succeeded will clear the dirty bitmap associated wi= th > + the operation, but the job(s) that failed will not. It is not "safe= " > + to delete any incremental backups that were created successfully in > + this scenario, even though others failed. > + > +Example > +^^^^^^^ > + > +- QMP example highlighting two backup jobs: > + > + .. code:: json > + > + { "execute": "transaction", > + "arguments": { > + "actions": [ > + { "type": "drive-backup", > + "data": { "device": "drive0", "bitmap": "bitmap0", > + "format": "qcow2", "mode": "existing", > + "sync": "incremental", "target": "d0-incr-1.q= cow2" } }, > + { "type": "drive-backup", > + "data": { "device": "drive1", "bitmap": "bitmap1", > + "format": "qcow2", "mode": "existing", > + "sync": "incremental", "target": "d1-incr-1.q= cow2" } }, > + ] > + } > + } > + > +- QMP example response, highlighting one success and one failure: > + > + - Acknowledgement that the Transaction was accepted and jobs were > + launched: > + > + .. code:: json > + > + { "return": {} } > + > + - Later, QEMU sends notice that the first job was completed: > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447192343, "microseconds": 6156= 98 }, > + "data": { "device": "drive0", "type": "backup", > + "speed": 0, "len": 67108864, "offset": 67108864= }, > + "event": "BLOCK_JOB_COMPLETED" > + } > + > + - Later yet, QEMU sends notice that the second job has failed: > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447192399, "microseconds": 6830= 15 }, > + "data": { "device": "drive1", "action": "report", > + "operation": "read" }, > + "event": "BLOCK_JOB_ERROR" } > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447192399, "microseconds": > + 685853 }, "data": { "speed": 0, "offset": 0, "len": 67108864= , > + "error": "Input/output error", "device": "drive1", "type": > + "backup" }, "event": "BLOCK_JOB_COMPLETED" } > + > +- In the above example, ``d0-incr-1.qcow2`` is valid and must be kept= , > + but ``d1-incr-1.qcow2`` is invalid and should be deleted. If a VM-w= ide > + incremental backup of all drives at a point-in-time is to be made, > + new backups for both drives will need to be made, taking into accou= nt > + that a new incremental backup for drive0 needs to be based on top o= f > + ``d0-incr-1.qcow2``. > + > +Grouped Completion Mode > +~~~~~~~~~~~~~~~~~~~~~~~ > + > +- While jobs launched by transactions normally complete or fail on > + their own, it is possible to instruct them to complete or fail > + together as a group. > + > +- QMP transactions take an optional properties structure that can > + affect the semantics of the transaction. > + > +- The "completion-mode" transaction property can be either "individua= l" > + which is the default, legacy behavior described above, or "grouped,= " > + a new behavior detailed below. > + > +- Delayed Completion: In grouped completion mode, no jobs will report > + success until all jobs are ready to report success. > + > +- Grouped failure: If any job fails in grouped completion mode, all > + remaining jobs will be cancelled. Any incremental backups will > + restore their dirty bitmap objects as if no backup command was ever > + issued. > + > + - Regardless of if QEMU reports a particular incremental backup jo= b > + as CANCELLED or as an ERROR, the in-memory bitmap will be > + restored. > + > +Example > +^^^^^^^ > + > +- Here's the same example scenario from above with the new property: > + > + .. code:: json > + > + { "execute": "transaction", > + "arguments": { > + "actions": [ > + { "type": "drive-backup", > + "data": { "device": "drive0", "bitmap": "bitmap0", > + "format": "qcow2", "mode": "existing", > + "sync": "incremental", "target": "d0-incr-1.q= cow2" } }, > + { "type": "drive-backup", > + "data": { "device": "drive1", "bitmap": "bitmap1", > + "format": "qcow2", "mode": "existing", > + "sync": "incremental", "target": "d1-incr-1.q= cow2" } }, > + ], > + "properties": { > + "completion-mode": "grouped" > + } > + } > + } > + > +- QMP example response, highlighting a failure for ``drive2``: > + > + - Acknowledgement that the Transaction was accepted and jobs were > + launched: > + > + .. code:: json > + > + { "return": {} } > + > + - Later, QEMU sends notice that the second job has errored out, bu= t > + that the first job was also cancelled: > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447193702, "microseconds": 6323= 77 }, > + "data": { "device": "drive1", "action": "report", > + "operation": "read" }, > + "event": "BLOCK_JOB_ERROR" } > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447193702, "microseconds": 6400= 74 }, > + "data": { "speed": 0, "offset": 0, "len": 67108864, > + "error": "Input/output error", > + "device": "drive1", "type": "backup" }, > + "event": "BLOCK_JOB_COMPLETED" } > + > + .. code:: json > + > + { "timestamp": { "seconds": 1447193702, "microseconds": 6401= 63 }, > + "data": { "device": "drive0", "type": "backup", "speed": 0= , > + "len": 67108864, "offset": 16777216 }, > + "event": "BLOCK_JOB_CANCELLED" } > + > +.. raw:: html > + > + >=20 --=20 =E2=80=94js