From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dor Laor Subject: Re: [RFC] live snapshot, live merge, live block migration Date: Mon, 16 May 2011 10:53:40 +0300 Message-ID: <4DD0D804.7070305@redhat.com> References: <4DC7EEB0.3040007@redhat.com> <4DCBFDB2.6010801@redhat.com> <4DCCA278.1000508@sundar.org> <4DD04231.9010501@redhat.com> <4DD047D2.2020201@sundar.org> Reply-To: dlaor@redhat.com Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kevin Wolf , Anthony Liguori , "kvm@vger.kernel.org" , Jes Sorensen , Marcelo Tosatti , qemu-devel , Avi Kivity , Stefan Hajnoczi To: Jagane Sundar Return-path: In-Reply-To: <4DD047D2.2020201@sundar.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org On 05/16/2011 12:38 AM, Jagane Sundar wrote: > Hello Dor, >> One important advantage of live snapshot over live backup is support of >> multiple (consecutive) live snapshots while there can be only a single >> live backup at one time. >> >> This is why I tend to think that although live backup carry some benefit >> (no merge required), the live snapshot + live merge are more robust >> mechanism. >> > > The two things that concern me regarding the > live snapshot/live merge approach are: > 1. Performance considerations of having > multiple active snapshots? My description above was in accurate and I only hinted that multiple snapshots are possible but they are done consecutively - Live snapshot takes practically almost no time - just the time to get the guest virtagent to freeze the guest FS and to create the snapshot (for qcow2 is immediate). So if you like to have multiple snapshot, let's say 5 minute after you issued the first snapshot, there is no problem. The new writes will go to the snapshot while the former base is marked as read only. Eventually you like to (live) merge the snapshots together. This can be done in any point in time. > 2. Robustness of this solution in the face of > errors in the disk, etc. If any one of the snapshot > files were to get corrupted, the whole VM is > adversely impacted. Since the base images and any snapshot which is not a leaf is marked as read only there is no such risk. > > The primary goal of Livebackup architecture was to have zero > performance impact on the running VM. > > Livebackup impacts performance of the VM only when the > backup client connects to qemu to transfer the modified > blocks over, which should be, say 15 minutes a day, for a > daily backup schedule VM. In case there were lots of changing for example additional 50GB changes it will take more time and there will be a performance hit. > > One useful thing to do is to evaluate the important use cases > for this technology, and then decide which approach makes > most sense. As an example, let me state this use case: > - A IaaS cloud, where VMs are always on, running off of a local > disk, and need to be backed up once a day or so. > > Can you list some of the other use cases that live snapshot and > live merge were designed to solve. Perhaps we can put up a > single wiki page that describes all of these proposals. Both solutions can serve for the same scenario: With live snapshot the backup is done the following: 1. Take a live snapshot (s1) of image s0. 2. Newer writes goes to the snapshot s1 while s0 is read only. 3. Backup software processes s0 image. There are multiple ways for doing that - 1. Use qemu-img and get the dirty blocks from former backup. - Currently qemu-img does not support it. - Nevertheless, such mechanism will work for lvm, btrfs, NetApp 2. Mount the s0 image to another guest that runs traditional backup software at the file system level and let it do the backup. 4. Live merge s1->s0 We'll use live copy for that so each write is duplicated (like your live backup solution). 5. Delete s1 As you can see, both approaches are very similar, while live snapshot is more general and not tied to backup specifically. > > Thanks, > Jagane > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:39726) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QLscd-0005p8-Rp for qemu-devel@nongnu.org; Mon, 16 May 2011 03:53:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QLscc-00071v-Qc for qemu-devel@nongnu.org; Mon, 16 May 2011 03:53:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54809) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QLscc-00071m-Hz for qemu-devel@nongnu.org; Mon, 16 May 2011 03:53:46 -0400 Message-ID: <4DD0D804.7070305@redhat.com> Date: Mon, 16 May 2011 10:53:40 +0300 From: Dor Laor MIME-Version: 1.0 References: <4DC7EEB0.3040007@redhat.com> <4DCBFDB2.6010801@redhat.com> <4DCCA278.1000508@sundar.org> <4DD04231.9010501@redhat.com> <4DD047D2.2020201@sundar.org> In-Reply-To: <4DD047D2.2020201@sundar.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration Reply-To: dlaor@redhat.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jagane Sundar Cc: Kevin Wolf , Anthony Liguori , "kvm@vger.kernel.org" , Jes Sorensen , Marcelo Tosatti , qemu-devel , Avi Kivity , Stefan Hajnoczi On 05/16/2011 12:38 AM, Jagane Sundar wrote: > Hello Dor, >> One important advantage of live snapshot over live backup is support of >> multiple (consecutive) live snapshots while there can be only a single >> live backup at one time. >> >> This is why I tend to think that although live backup carry some benefit >> (no merge required), the live snapshot + live merge are more robust >> mechanism. >> > > The two things that concern me regarding the > live snapshot/live merge approach are: > 1. Performance considerations of having > multiple active snapshots? My description above was in accurate and I only hinted that multiple snapshots are possible but they are done consecutively - Live snapshot takes practically almost no time - just the time to get the guest virtagent to freeze the guest FS and to create the snapshot (for qcow2 is immediate). So if you like to have multiple snapshot, let's say 5 minute after you issued the first snapshot, there is no problem. The new writes will go to the snapshot while the former base is marked as read only. Eventually you like to (live) merge the snapshots together. This can be done in any point in time. > 2. Robustness of this solution in the face of > errors in the disk, etc. If any one of the snapshot > files were to get corrupted, the whole VM is > adversely impacted. Since the base images and any snapshot which is not a leaf is marked as read only there is no such risk. > > The primary goal of Livebackup architecture was to have zero > performance impact on the running VM. > > Livebackup impacts performance of the VM only when the > backup client connects to qemu to transfer the modified > blocks over, which should be, say 15 minutes a day, for a > daily backup schedule VM. In case there were lots of changing for example additional 50GB changes it will take more time and there will be a performance hit. > > One useful thing to do is to evaluate the important use cases > for this technology, and then decide which approach makes > most sense. As an example, let me state this use case: > - A IaaS cloud, where VMs are always on, running off of a local > disk, and need to be backed up once a day or so. > > Can you list some of the other use cases that live snapshot and > live merge were designed to solve. Perhaps we can put up a > single wiki page that describes all of these proposals. Both solutions can serve for the same scenario: With live snapshot the backup is done the following: 1. Take a live snapshot (s1) of image s0. 2. Newer writes goes to the snapshot s1 while s0 is read only. 3. Backup software processes s0 image. There are multiple ways for doing that - 1. Use qemu-img and get the dirty blocks from former backup. - Currently qemu-img does not support it. - Nevertheless, such mechanism will work for lvm, btrfs, NetApp 2. Mount the s0 image to another guest that runs traditional backup software at the file system level and let it do the backup. 4. Live merge s1->s0 We'll use live copy for that so each write is duplicated (like your live backup solution). 5. Delete s1 As you can see, both approaches are very similar, while live snapshot is more general and not tied to backup specifically. > > Thanks, > Jagane >