Re: [Qemu-devel] [PATCH v2 0/6] external backup api

["Denis V. Lunev" <den@virtuozzo.com>]

On 02/16/2016 08:09 PM, Stefan Hajnoczi wrote:
> On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> Hi all.
>>>>>>>
>>>>>>> These series which aims to add external backup api. This is needed to allow
>>>>>>> backup software use our dirty bitmaps.
>>>>>>>
>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>>>>> ..."?
>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>> their own formats or to the cloud etc.
>>>> As an example, there is a third-party backup format called VMA from
>>>> Proxmox.  A few years ago I posted a proof-of-concept external backup
>>>> tool in Python:
>>>>
>>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>>>>
>>>> It takes a full backup using drive-backup NBD (plus RAM/device state)
>>>> but the same can be done with incremental backups.
>>>>
>>>> Does this NBD approach meet your requirements?
>>>>
>>>> Stefan
>>> for us we should somehow provide implementation of
>>> calls posted by Vladimir. They are available in Parallels Server
>>> version 6 and should be available in the next QEMU based
>>> release using "Parallels SDK to libvirt" convertor. The problem
>>> for us is that this old approach is used in the other side
>>> of the product - in containers implementation while this
>>> SDK is a universal access tool to both things.
>> Point taken.  I think many other backup applications will expect a
>> similar API, so it's pragmatic to provide something compatible.
> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> concerns about the QMP monitor:
>
> Previously I described incremental backup in "push" mode (already
> supported today with drive-backup).  QEMU connects to a remote NBD
> server and writes out the contents of all dirty blocks:
>
>    QEMU ---Write dirty blocks--> Backup appliance (server)
>
> This doesn't lend itself well to existing backup applications that
> expect to iterate the dirty bitmap manually.
>
> Let's add a "pull" mode where the connection of the NBD connection is
> reversed.  The backup application connects to QEMU's NBD server (image
> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> Status command for querying block provisioning information.  Now the
> backup application can use Get LBA Status to fetch the dirty block
> information from QEMU.
>
>    QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>
> The dirty block information goes over the same NBD connection used to
> read the contents of the dirty blocks.  The QMP monitor is not used to
> transfer dirty block information.
>
> It may be necessary to extend the nbd-server-add command so that a
> bitmap name can be passed.  This bitmap will be used to answer Get LBA
> Status queries instead of using on bdrv_co_get_block_status().  This
> would be necessary if image fleecing (point in time snapshot) is used.
>
> Stefan
the idea seems feasible at the first glance. I'll need to check
the amount of the overhead of this approach. If we'll need
to perform an additional request for each <granularity>
block - the overhead is toooooo much.

Do you have the link to the spec/implementation?

Den