Re: [Qemu-devel] [PATCH v2 0/6] external backup api

["Denis V. Lunev" <den@virtuozzo.com>]

On 02/09/2016 09:12 PM, John Snow wrote:
>
> On 02/09/2016 11:58 AM, Denis V. Lunev wrote:
>> On 02/09/2016 07:49 PM, John Snow wrote:
>>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> Hi all.
>>>>>>>>
>>>>>>>> These series which aims to add external backup api. This is needed
>>>>>>>> to allow
>>>>>>>> backup software use our dirty bitmaps.
>>>>>>>>
>>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>>> What is the advantage of this appraoch over "drive-backup
>>>>>>> sync=incremental
>>>>>>> ..."?
>>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>>> their own formats or to the cloud etc.
>>>>> Backup software can implement NBD to receive the incremental blocks
>>>>> from
>>>>> QEMU.  Use drive-backup sync=incremental and the backup appliance as
>>>>> the
>>>>> NBD target.
>>>>>
>>>>> It's more complicated to add this QMP command flow to backup software
>>>>> than to implement NBD.
>>>>>
>>>>> Stefan
>>>> it can, but this is a matter of problem due to the nature of
>>>> how this software is implemented. Usually it is written
>>>> in a semi-standard way and it uses "plugins" to actually
>>>> collect the data, i.e. the code is written in standard
>>>> interface/real implementation pattern and interfaces are
>>>> basically the same.
>>>>
>>>> With this standard approach backup software is working
>>>> as an active side of the process, i.e. it performs operations
>>>> and controls the flow.
>>>>
>>>> This means that "non-standard" QEMU technology will be
>>>> pain here.
>>>>
>>>> Den
>>> Am I to understand that for e.g. VMWare the backup appliance is
>>> literally reading the disk image from storage directly while the VM is
>>> running?
>>>
>>> I'd be a bit surprised if that were true.
>> I think that backup software is asking alive VM about the data.
>>
>>> My biggest concern here is that there is not a safe way, today, to read
>>> from a disk image atomically while the VM is running. I think that'd
>>> take a lot of work to do and you'll not find a lot of support in
>>> implementing it.
>>>
>>> Of course, while the VM is paused/off is a different story, but for now
>>> I still feel like NBD is the right answer for getting block data from
>>> QEMU.
>>>
>>> What am I missing?
>>>
>>> --js
>> In general, in Parallels Server the backup was created using the
>> following approach:
>> - create external snapshot. In this case the base image (backing store
>> in QEMU terminology)
>>    will be READ-ONLY and could be safely read by any entity
>> - backup that read-only disk image (any way)
>> - merge snapshots
>>
> I see.
>
>> In this process backup software is active while PCS is passive.
> PCS?
Parallels Cloud Server. Sorry for abbreviation :(


>> With QEMU the approach looks the same to me:
>> - start a backup
>> - ask QEMU to give a data to be backuped (using NBD server in QEMU
>>    with Fam's image fleecing)
>> - finish backup
>>
>> Important bit here is that dirty bitmap should be provided
>> by QEMU by request. This dirty bitmap will be read-only at that
>> moment, current active dirty bitmap should be replaced with
>> new at backup start operation.
>>
>> Den
> I don't have any problems providing the bitmap data through an external
> API, but the part I want to be 100% clear on before I ACK it is the API
> portion where we allow an external client to split or merge bitmaps
> externally -- that's functionality you don't need if you query the data
> from QEMU itself.
That is fine :)

> If using NBD (either through the incremental backup tool or image
> fleecing) that is completely fine, too.
>
> My only reservations are over making direct reads to image files outside
> of QEMU while the VM is running.

Perfect :) We are on the same page. I would not
going to go something unsafe is this very sensitive
area.

Den