From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36081) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRz1r-0008Ti-H9 for qemu-devel@nongnu.org; Sat, 06 Feb 2016 04:19:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aRz1o-0007Tz-AW for qemu-devel@nongnu.org; Sat, 06 Feb 2016 04:19:43 -0500 Received: from mx2.parallels.com ([199.115.105.18]:37629) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRz1o-0007TS-20 for qemu-devel@nongnu.org; Sat, 06 Feb 2016 04:19:40 -0500 Message-ID: <56B5BA9B.8050109@virtuozzo.com> Date: Sat, 6 Feb 2016 12:19:23 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1453482459-80179-1-git-send-email-vsementsov@virtuozzo.com> <56B4FC80.1070402@redhat.com> In-Reply-To: <56B4FC80.1070402@redhat.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH RFC] external backup api List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: John Snow , qemu-devel@nongnu.org Cc: famz@redhat.com, den@virtuozzo.com, Stefan Hajnoczi On 05.02.2016 22:48, John Snow wrote: > > On 01/22/2016 12:07 PM, Vladimir Sementsov-Ogievskiy wrote: >> Hi all. >> >> This is the early begin of the series which aims to add external backup >> api. This is needed to allow backup software use our dirty bitmaps. >> >> Vmware and Parallels Cloud Server have this feature. >> > Have a link to the equivalent feature that VMWare exposes? (Or Parallels > Cloud Server) ... I'm curious about what the API there looks like. For VMware you need their Virtual Disk Api Programming Guide http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vddk60_programming.pdf Look at Changed Block Tracking (CBT) , Backup and Restore. For PCS here is part of SDK header, related to the topic: ==================================== /* * Builds a map of the disk contents changes between 2 PITs. Parameters hDisk : A handle of type PHT_VIRTUAL_DISK identifying the virtual disk. sPit1Uuid : Uuid of the older PIT. sPit2Uuid : Uuid of the later PIT. phMap : A pointer to a variable which receives the result (a handle of type PHT_VIRTUAL_DISK_MAP). Returns PRL_RESULT. */ PRL_METHOD_DECL( PARALLELS_API_VER_5, PrlDisk_GetChangesMap_Local, ( PRL_HANDLE hDisk, PRL_CONST_STR sPit1Uuid, PRL_CONST_STR sPit2Uuid, PRL_HANDLE_PTR phMap) ); /* * Reports the number of significant bits in the map. Parameters hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying the changes map. phSize : A pointer to a variable which receives the result. Returns PRL_RESULT. */ PRL_METHOD_DECL( PARALLELS_API_VER_5, PrlDiskMap_GetSize, ( PRL_HANDLE hMap, PRL_UINT32_PTR pnSize) ); /* * Reports the size (in bytes) of a block mapped by a single bit * in the map. Parameters hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying the changes map. phSize : A pointer to a variable which receives the result. Returns PRL_RESULT. */ PRL_METHOD_DECL( PARALLELS_API_VER_5, PrlDiskMap_GetGranularity, ( PRL_HANDLE hMap, PRL_UINT32_PTR pnSize) ); /* * Returns bits from the blocks map. Parameters hMap : A handle of type PHT_VIRTUAL_DISK_MAP identifying the changes map. pBuffer : A pointer to a store. pnCapacity : A pointer to a variable holding the size of the buffer and receiving the number of bytes actually written. Returns PRL_RESULT. */ PRL_METHOD_DECL( PARALLELS_API_VER_5, PrlDiskMap_Read, ( PRL_HANDLE hMap, PRL_VOID_PTR pBuffer, PRL_UINT32_PTR pnCapacity) ); ======================================= > >> There is only one patch here, about querying dirty bitmap from qemu by >> qmp command. It is just an updated and clipped (hmp command removed) old >> my patch "[PATCH RFC v3 01/14] qmp: add query-block-dirty-bitmap". >> >> Before writing the whole thing I'd like to discuss the details. Or, may >> be there are existing plans on this topic, or may be someone already >> works on it? >> >> I see it like this: >> >> ===== >> >> - add qmp commands for dirty-bitmap functions: create_successor, abdicate, >> reclaime. > Hm, why do we need such low-level control over splitting and merging > bitmaps from an external client? > >> - make create-successor command transaction-able >> - add query-block-dirty-bitmap qmp command >> >> then, external backup: >> >> qmp transaction { >> external-snapshot >> bitmap-create-successor >> } >> >> qmp query frozen bitmap, not acquiring aio context. >> >> do external backup, using snapshot and bitmap >> >> if (success backup) >> qmp bitmap-abdicate >> else >> qmp bitmap-reclaime >> >> qmp merge snapshot >> ===== >> > Hm, I see -- so you're hoping to manage the backup *entirely* > externally, so you want to be able to reach inside of QEMU and control > some status conditions to guarantee it'll be safe. > > I'm not convinced QEMU can guarantee such things -- due to various flush > properties, race conditions on write, etc. QEMU handles all of this > internally in a non-public way at the moment. Hm, can you be more concrete? What operations are dangerous? We can do them in paused state for example. > >> In the following patch query-bitmap acquires aio context. This must be >> ofcourse dropped for frozen bitmap. >> But to make it in true way, I think, I should check somehow that this is >> not just frozen bitmap, but the bitmap frozen by qmp command, to avoid >> incorrect quering of bitmap frozen by internal backup (or other >> mechanizm).. May be, it is not necessary. >> >> >> > > -- Best regards, Vladimir