* [Qemu-devel] [RFC] Differential Backups
@ 2015-04-29 22:51 John Snow
2015-05-05 10:25 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
0 siblings, 1 reply; 4+ messages in thread
From: John Snow @ 2015-04-29 22:51 UTC (permalink / raw)
To: Qemu-block; +Cc: Markus Armbruster, qemu-devel
This is a feature that should be very easy to add on top of the existing
incremental feature, since it's just a difference in how the bitmap is
treated:
Incremental
- Links to the last incremental (managed by libvirt)
- Clears the bitmap after creation
Differential:
- Links to the last full backup always (managed by libvirt)
- Does not clear the bitmap after creation
No biggie.
How it works currently: Incremental backups are created via the
MIRROR_SYNC_MODE_DIRTY_BITMAP backup mode. An early version of the
patchset actually had an additional parameter called the BitmapUseMode
that controlled how the bitmap was cleared, a concern that was later
made obsolete for other reasons.
I can add Differential backups in two ways:
(1) rename MIRROR_SYNC_MODE_DIRTY_BITMAP to
MIRROR_SYNC_MODE_INCREMENTAL, and then add
MIRROR_SYNC_MODE_DIFFERENTIAL. It's not too late to do this, since 2.4
has just started.
(2) Re-add the BitmapUseMode parameter and add some enums:
BITMAP_USE_MODE_INCREMENTAL
BITMAP_USE_MODE_DIFFERENTIAL
I think I am partial to #1, if only to cut down on additional
parameters, especially ones that are only useful to a small subset of
backup types.
I am also considering adding a QMP primitive to allow people to /copy/
bitmaps. This would allow users to differentiate backup chains after
they've already been started -- e.g.:
- User creates drive0, a full backup, and bitmap monthly0.
- Six days in, user decides it would be very nice to add a "weekly"
incremental backup series to the same drive, but doesn't want to lose
out on the monthly chain that was already started.
- User copies the monthly0 bitmap to a new weekly0 target.
- User can continue two independent incremental series intended for
different periodicity.
Actually, the copy primitive would allow people to do both differentials
and incrementals with the existing backup mode by copying the bitmap
before each usage, but that's slightly yucky, so I'd rather just add
both features to increase the flexibility of the delta backup system in
general.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [RFC] Differential Backups
2015-04-29 22:51 [Qemu-devel] [RFC] Differential Backups John Snow
@ 2015-05-05 10:25 ` Stefan Hajnoczi
2015-05-05 15:55 ` John Snow
0 siblings, 1 reply; 4+ messages in thread
From: Stefan Hajnoczi @ 2015-05-05 10:25 UTC (permalink / raw)
To: John Snow; +Cc: Markus Armbruster, Qemu-block, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 696 bytes --]
On Wed, Apr 29, 2015 at 06:51:08PM -0400, John Snow wrote:
> This is a feature that should be very easy to add on top of the existing
> incremental feature, since it's just a difference in how the bitmap is
> treated:
>
> Incremental
> - Links to the last incremental (managed by libvirt)
> - Clears the bitmap after creation
>
> Differential:
> - Links to the last full backup always (managed by libvirt)
> - Does not clear the bitmap after creation
>
> No biggie.
Differential backups can be done using incremental backup functionality
in QEMU:
The client application points QEMU to the same target repeatedly instead
of keeping separate incremental backups.
Stefan
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [RFC] Differential Backups
2015-05-05 10:25 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
@ 2015-05-05 15:55 ` John Snow
2015-05-06 9:17 ` Stefan Hajnoczi
0 siblings, 1 reply; 4+ messages in thread
From: John Snow @ 2015-05-05 15:55 UTC (permalink / raw)
To: Qemu-block, Stefan Hajnoczi; +Cc: Markus Armbruster, qemu-devel
On 05/05/2015 06:25 AM, Stefan Hajnoczi wrote:
> On Wed, Apr 29, 2015 at 06:51:08PM -0400, John Snow wrote:
>> This is a feature that should be very easy to add on top of the existing
>> incremental feature, since it's just a difference in how the bitmap is
>> treated:
>>
>> Incremental
>> - Links to the last incremental (managed by libvirt)
>> - Clears the bitmap after creation
>>
>> Differential:
>> - Links to the last full backup always (managed by libvirt)
>> - Does not clear the bitmap after creation
>>
>> No biggie.
>
> Differential backups can be done using incremental backup functionality
> in QEMU:
>
> The client application points QEMU to the same target repeatedly instead
> of keeping separate incremental backups.
>
> Stefan
>
Oh, so you're saying:
[anchor]<--[diff1]
And then when making a new incremental, we re-use diff1 as a target and
overwrite it so that it becomes:
[anchor]<--[diff2]
In effect giving us a differential.
OK, so it's possible, but we still lose out on some flexibility that a
slightly different mode would provide us, like the ability to keep
multiple differentials if desired. (Well, I suppose we *can* create
those by manually copying differentials after we create them, but that
seems hackier than necessary.)
Still, it would be such a paltry few lines of code and introduce no real
complexity to the subsystem, and it might make libvirt's time a little
easier for managing such things.
--js
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [RFC] Differential Backups
2015-05-05 15:55 ` John Snow
@ 2015-05-06 9:17 ` Stefan Hajnoczi
0 siblings, 0 replies; 4+ messages in thread
From: Stefan Hajnoczi @ 2015-05-06 9:17 UTC (permalink / raw)
To: John Snow; +Cc: libvir-list, Markus Armbruster, Qemu-block, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2786 bytes --]
On Tue, May 05, 2015 at 11:55:49AM -0400, John Snow wrote:
> On 05/05/2015 06:25 AM, Stefan Hajnoczi wrote:
> >On Wed, Apr 29, 2015 at 06:51:08PM -0400, John Snow wrote:
> >>This is a feature that should be very easy to add on top of the existing
> >>incremental feature, since it's just a difference in how the bitmap is
> >>treated:
> >>
> >>Incremental
> >>- Links to the last incremental (managed by libvirt)
> >>- Clears the bitmap after creation
> >>
> >>Differential:
> >>- Links to the last full backup always (managed by libvirt)
> >>- Does not clear the bitmap after creation
> >>
> >>No biggie.
> >
> >Differential backups can be done using incremental backup functionality
> >in QEMU:
> >
> >The client application points QEMU to the same target repeatedly instead
> >of keeping separate incremental backups.
> >
> >Stefan
> >
>
> Oh, so you're saying:
>
> [anchor]<--[diff1]
>
> And then when making a new incremental, we re-use diff1 as a target and
> overwrite it so that it becomes:
>
> [anchor]<--[diff2]
>
> In effect giving us a differential.
>
> OK, so it's possible, but we still lose out on some flexibility that a
> slightly different mode would provide us, like the ability to keep multiple
> differentials if desired. (Well, I suppose we *can* create those by manually
> copying differentials after we create them, but that seems hackier than
> necessary.)
>
> Still, it would be such a paltry few lines of code and introduce no real
> complexity to the subsystem, and it might make libvirt's time a little
> easier for managing such things.
I have CCed Eric Blake and the libvirt mailing list.
This is a good time to discuss libvirt requirements for backup APIs.
Current work for QEMU 2.4:
* QMP command to take incremental backup of guest disks in a single
atomic operation
* Dirty bitmap persistence across live migration and QEMU restart
Eric: Do you have any opinion on a differential backup feature in
addition to incremental backup.
My thoughts are that QEMU should only copy out changed blocks and let
the backup application worry about concepts like incremental vs
differential.
From a host performance perspective, copying out the same blocks that
have already been sent in a previous backup is a waste of I/O bandwidth.
Even backup applications that want differential backup may not actually
use the QEMU feature for this reason.
Regarding the size of the patch, there's a cost to adding the tests,
documentation, and commiting to QMP APIs. These costs dwarf the small
code change that preserves the dirty bitmap.
If there's a concrete requirement for this feature then I'm happy with
it, but let's not add bells and whistles just because we can.
Stefan
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-05-06 9:17 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-29 22:51 [Qemu-devel] [RFC] Differential Backups John Snow
2015-05-05 10:25 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2015-05-05 15:55 ` John Snow
2015-05-06 9:17 ` Stefan Hajnoczi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.