From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58280) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aVjG8-0005Ry-PZ for qemu-devel@nongnu.org; Tue, 16 Feb 2016 12:17:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aVjG3-0001u9-PX for qemu-devel@nongnu.org; Tue, 16 Feb 2016 12:17:56 -0500 Received: from mx2.parallels.com ([199.115.105.18]:43685) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aVjG3-0001tL-Ju for qemu-devel@nongnu.org; Tue, 16 Feb 2016 12:17:51 -0500 Message-ID: <56C359B3.7050207@virtuozzo.com> Date: Tue, 16 Feb 2016 20:17:39 +0300 From: Vladimir Sementsov-Ogievskiy MIME-Version: 1.0 References: <1454151394-52320-1-git-send-email-vsementsov@virtuozzo.com> <20160203081418.GC25746@ad.usersys.redhat.com> <56B45D3A.405@virtuozzo.com> <20160209142852.GA13149@stefanha-x1.localdomain> <56B9FAAE.8040503@virtuozzo.com> <20160210101004.GB7317@stefanha-x1.localdomain> <20160216170943.GA31393@stefanha-x1.localdomain> In-Reply-To: <20160216170943.GA31393@stefanha-x1.localdomain> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v2 0/6] external backup api List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , "Denis V. Lunev" Cc: kwolf@redhat.com, jsnow@redhat.com, Fam Zheng , qemu-devel@nongnu.org, armbru@redhat.com On 16.02.2016 20:09, Stefan Hajnoczi wrote: > On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote: >> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote: >>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote: >>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote: >>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote: >>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote: >>>>>>> Hi all. >>>>>>> >>>>>>> These series which aims to add external backup api. This is needed to allow >>>>>>> backup software use our dirty bitmaps. >>>>>>> >>>>>>> Vmware and Parallels Cloud Server have this feature. >>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental >>>>>> ..."? >>>>> This will allow third-party vendors to backup QEMU VMs into >>>>> their own formats or to the cloud etc. >>>> As an example, there is a third-party backup format called VMA from >>>> Proxmox. A few years ago I posted a proof-of-concept external backup >>>> tool in Python: >>>> >>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html >>>> >>>> It takes a full backup using drive-backup NBD (plus RAM/device state) >>>> but the same can be done with incremental backups. >>>> >>>> Does this NBD approach meet your requirements? >>>> >>>> Stefan >>> for us we should somehow provide implementation of >>> calls posted by Vladimir. They are available in Parallels Server >>> version 6 and should be available in the next QEMU based >>> release using "Parallels SDK to libvirt" convertor. The problem >>> for us is that this old approach is used in the other side >>> of the product - in containers implementation while this >>> SDK is a universal access tool to both things. >> Point taken. I think many other backup applications will expect a >> similar API, so it's pragmatic to provide something compatible. > Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the > concerns about the QMP monitor: > > Previously I described incremental backup in "push" mode (already > supported today with drive-backup). QEMU connects to a remote NBD > server and writes out the contents of all dirty blocks: > > QEMU ---Write dirty blocks--> Backup appliance (server) > > This doesn't lend itself well to existing backup applications that > expect to iterate the dirty bitmap manually. > > Let's add a "pull" mode where the connection of the NBD connection is > reversed. The backup application connects to QEMU's NBD server (image > fleecing). The NBD protocol is extended to support the SCSI Get LBA > Status command for querying block provisioning information. Now the > backup application can use Get LBA Status to fetch the dirty block > information from QEMU. > > QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance > > The dirty block information goes over the same NBD connection used to > read the contents of the dirty blocks. The QMP monitor is not used to > transfer dirty block information. > > It may be necessary to extend the nbd-server-add command so that a > bitmap name can be passed. This bitmap will be used to answer Get LBA > Status queries instead of using on bdrv_co_get_block_status(). This > would be necessary if image fleecing (point in time snapshot) is used. > > Stefan Looks much easier than implement nbd-server for dirty bitmap, thanks. -- Best regards, Vladimir