From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: John Snow <jsnow@redhat.com>, qemu-devel@nongnu.org
Cc: famz@redhat.com, den@virtuozzo.com,
Stefan Hajnoczi <stefanha@redhat.com>
Subject: Re: [Qemu-devel] [PATCH RFC] external backup api
Date: Sat, 6 Feb 2016 12:19:23 +0300 [thread overview]
Message-ID: <56B5BA9B.8050109@virtuozzo.com> (raw)
In-Reply-To: <56B4FC80.1070402@redhat.com>
On 05.02.2016 22:48, John Snow wrote:
>
> On 01/22/2016 12:07 PM, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all.
>>
>> This is the early begin of the series which aims to add external backup
>> api. This is needed to allow backup software use our dirty bitmaps.
>>
>> Vmware and Parallels Cloud Server have this feature.
>>
> Have a link to the equivalent feature that VMWare exposes? (Or Parallels
> Cloud Server) ... I'm curious about what the API there looks like.
For VMware you need their Virtual Disk Api Programming Guide
http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vddk60_programming.pdf
Look at Changed Block Tracking (CBT) , Backup and Restore.
For PCS here is part of SDK header, related to the topic:
====================================
/*
* Builds a map of the disk contents changes between 2 PITs.
Parameters
hDisk : A handle of type PHT_VIRTUAL_DISK identifying
the virtual disk.
sPit1Uuid : Uuid of the older PIT.
sPit2Uuid : Uuid of the later PIT.
phMap : A pointer to a variable which receives the
result (a handle of type PHT_VIRTUAL_DISK_MAP).
Returns
PRL_RESULT.
*/
PRL_METHOD_DECL( PARALLELS_API_VER_5,
PrlDisk_GetChangesMap_Local, (
PRL_HANDLE hDisk,
PRL_CONST_STR sPit1Uuid,
PRL_CONST_STR sPit2Uuid,
PRL_HANDLE_PTR phMap) );
/*
* Reports the number of significant bits in the map.
Parameters
hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying
the changes map.
phSize : A pointer to a variable which receives the
result.
Returns
PRL_RESULT.
*/
PRL_METHOD_DECL( PARALLELS_API_VER_5,
PrlDiskMap_GetSize, (
PRL_HANDLE hMap,
PRL_UINT32_PTR pnSize) );
/*
* Reports the size (in bytes) of a block mapped by a single bit
* in the map.
Parameters
hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying
the changes map.
phSize : A pointer to a variable which receives the
result.
Returns
PRL_RESULT.
*/
PRL_METHOD_DECL( PARALLELS_API_VER_5,
PrlDiskMap_GetGranularity, (
PRL_HANDLE hMap,
PRL_UINT32_PTR pnSize) );
/*
* Returns bits from the blocks map.
Parameters
hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying
the changes map.
pBuffer : A pointer to a store.
pnCapacity : A pointer to a variable holding the size
of the buffer and receiving the number of
bytes actually written.
Returns
PRL_RESULT.
*/
PRL_METHOD_DECL( PARALLELS_API_VER_5,
PrlDiskMap_Read, (
PRL_HANDLE hMap,
PRL_VOID_PTR pBuffer,
PRL_UINT32_PTR pnCapacity) );
=======================================
>
>> There is only one patch here, about querying dirty bitmap from qemu by
>> qmp command. It is just an updated and clipped (hmp command removed) old
>> my patch "[PATCH RFC v3 01/14] qmp: add query-block-dirty-bitmap".
>>
>> Before writing the whole thing I'd like to discuss the details. Or, may
>> be there are existing plans on this topic, or may be someone already
>> works on it?
>>
>> I see it like this:
>>
>> =====
>>
>> - add qmp commands for dirty-bitmap functions: create_successor, abdicate,
>> reclaime.
> Hm, why do we need such low-level control over splitting and merging
> bitmaps from an external client?
>
>> - make create-successor command transaction-able
>> - add query-block-dirty-bitmap qmp command
>>
>> then, external backup:
>>
>> qmp transaction {
>> external-snapshot
>> bitmap-create-successor
>> }
>>
>> qmp query frozen bitmap, not acquiring aio context.
>>
>> do external backup, using snapshot and bitmap
>>
>> if (success backup)
>> qmp bitmap-abdicate
>> else
>> qmp bitmap-reclaime
>>
>> qmp merge snapshot
>> =====
>>
> Hm, I see -- so you're hoping to manage the backup *entirely*
> externally, so you want to be able to reach inside of QEMU and control
> some status conditions to guarantee it'll be safe.
>
> I'm not convinced QEMU can guarantee such things -- due to various flush
> properties, race conditions on write, etc. QEMU handles all of this
> internally in a non-public way at the moment.
Hm, can you be more concrete? What operations are dangerous? We can do
them in paused state for example.
>
>> In the following patch query-bitmap acquires aio context. This must be
>> ofcourse dropped for frozen bitmap.
>> But to make it in true way, I think, I should check somehow that this is
>> not just frozen bitmap, but the bitmap frozen by qmp command, to avoid
>> incorrect quering of bitmap frozen by internal backup (or other
>> mechanizm).. May be, it is not necessary.
>>
>>
>>
>
>
--
Best regards,
Vladimir
next prev parent reply other threads:[~2016-02-06 9:19 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-22 17:07 [Qemu-devel] [PATCH RFC] external backup api Vladimir Sementsov-Ogievskiy
2016-01-22 17:07 ` [Qemu-devel] [PATCH] qmp: add query-block-dirty-bitmap Vladimir Sementsov-Ogievskiy
2016-01-22 17:22 ` Denis V. Lunev
2016-01-22 17:28 ` Vladimir Sementsov-Ogievskiy
2016-01-22 18:28 ` Denis V. Lunev
2016-01-22 18:43 ` Eric Blake
2016-02-05 19:48 ` [Qemu-devel] [PATCH RFC] external backup api John Snow
2016-02-06 9:19 ` Vladimir Sementsov-Ogievskiy [this message]
2016-02-08 21:14 ` John Snow
2016-02-09 15:54 ` Vladimir Sementsov-Ogievskiy
2016-02-09 16:51 ` John Snow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56B5BA9B.8050109@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=den@virtuozzo.com \
--cc=famz@redhat.com \
--cc=jsnow@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).