* [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage
@ 2009-11-02 13:40 lirans
2009-11-02 18:47 ` Avi Kivity
2009-11-03 7:33 ` [Qemu-devel] " Liran Schour
0 siblings, 2 replies; 10+ messages in thread
From: lirans @ 2009-11-02 13:40 UTC (permalink / raw)
To: qemu-devel; +Cc: Liran Schour
This series adds support for live migration without shared storage, means
copy the storage while migrating. It was tested with KVM. Supports 2 ways
to replicate the storage during migration:
1. Complete copy of storage to destination
2. Assuming the storage is cow based, copy only the allocated
data, time of the migration will be linear with the amount of allocated
data (user responsibility to verify that the same backend file reside
on src and destination).
Live migration will work as follows:
(qemu) migrate -d tcp:0:4444 # for ordinary live migration
(qemu) migrate -d blk tcp:0:4444 # for live migration with complete storage copy
(qemu) migrate -d blk inc tcp:0:4444 # for live migration with incremental storage copy, storage is cow based.
Changes from v4:
- Register block dirty tracking for each block device.
- Block.c will now manage a dirty bitmap per device once
bdrv_set_dirty_tracking() is called. Bitmap is polled by the upper
layer (block-migration.c).
- Global variabels moved to a global state structure allocated dynamically.
- Coding style issues.
Patch summary:
Makefile | 1 +
block-migration.c | 558 +++++++++++++++++++++++++++++++++++++++++++++++++++++
block-migration.h | 29 +++
block.c | 78 ++++++++-
block.h | 8 +-
block_int.h | 2 +
buffered_file.c | 10 +-
hw/hw.h | 9 +-
migration-exec.c | 9 +-
migration-fd.c | 7 +-
migration-tcp.c | 7 +-
migration-unix.c | 7 +-
migration.c | 23 ++-
migration.h | 20 ++-
monitor.c | 36 +++-
qemu-monitor.hx | 19 ++-
savevm.c | 62 +++++--
sysemu.h | 2 +-
vl.c | 9 +-
19 files changed, 841 insertions(+), 55 deletions(-)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage
2009-11-02 13:40 [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage lirans
@ 2009-11-02 18:47 ` Avi Kivity
2009-11-03 9:40 ` Liran Schour
2009-11-03 7:33 ` [Qemu-devel] " Liran Schour
1 sibling, 1 reply; 10+ messages in thread
From: Avi Kivity @ 2009-11-02 18:47 UTC (permalink / raw)
To: lirans; +Cc: qemu-devel
On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
> This series adds support for live migration without shared storage, means
> copy the storage while migrating. It was tested with KVM. Supports 2 ways
> to replicate the storage during migration:
> 1. Complete copy of storage to destination
> 2. Assuming the storage is cow based, copy only the allocated
> data, time of the migration will be linear with the amount of allocated
> data (user responsibility to verify that the same backend file reside
> on src and destination).
>
> Live migration will work as follows:
> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
> (qemu) migrate -d blk tcp:0:4444 # for live migration with complete storage copy
> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with incremental storage copy, storage is cow based.
>
I'd like to see the syntax generalized. For one, the guest may have
several disks; an install image cdrom might be available as an nfs image
but the main storage is local. Secondly, there can be several levels of
cow and we want to control which one we copy.
I'll leave the exact details to the qpeople, but if we can specify a
copy depth for each device, where 0=copy nothing, n=copy everything,
1=copy the last level (equivalent to -d blk inc) I think we'll have
covered everything.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage
2009-11-02 13:40 [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage lirans
2009-11-02 18:47 ` Avi Kivity
@ 2009-11-03 7:33 ` Liran Schour
1 sibling, 0 replies; 10+ messages in thread
From: Liran Schour @ 2009-11-03 7:33 UTC (permalink / raw)
To: qemu-devel
qemu-devel-bounces+lirans=il.ibm.com@nongnu.org wrote on 02/11/2009
15:40:25:
> Live migration will work as follows:
> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
> (qemu) migrate -d blk tcp:0:4444 # for live migration with complete
> storage copy
> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
> incremental storage copy, storage is cow based.
>
The up-to-date migrate command will work as follow:
(qemu) migrate -d tcp:0:4444 # for ordinary live migration
(qemu) migrate -d -b tcp:0:4444 # for live migration with complete storage
copy
(qemu) migrate -d -i tcp:0:4444 # for live migration with incremental
storage copy,
storage is cow based.
- Liran
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage
2009-11-02 18:47 ` Avi Kivity
@ 2009-11-03 9:40 ` Liran Schour
2009-11-03 10:38 ` Avi Kivity
0 siblings, 1 reply; 10+ messages in thread
From: Liran Schour @ 2009-11-03 9:40 UTC (permalink / raw)
To: Avi Kivity; +Cc: qemu-devel
- Liran
Avi Kivity <avi@redhat.com> wrote on 02/11/2009 20:47:34:
> On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
> > This series adds support for live migration without shared storage,
means
> > copy the storage while migrating. It was tested with KVM. Supports 2
ways
> > to replicate the storage during migration:
> > 1. Complete copy of storage to destination
> > 2. Assuming the storage is cow based, copy only the allocated
> > data, time of the migration will be linear with the amount of allocated
> > data (user responsibility to verify that the same backend file reside
> > on src and destination).
> >
> > Live migration will work as follows:
> > (qemu) migrate -d tcp:0:4444 # for ordinary live migration
> > (qemu) migrate -d blk tcp:0:4444 # for live migration with
> complete storage copy
> > (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
> incremental storage copy, storage is cow based.
> >
>
> I'd like to see the syntax generalized. For one, the guest may have
> several disks; an install image cdrom might be available as an nfs image
> but the main storage is local. Secondly, there can be several levels of
> cow and we want to control which one we copy.
>
> I'll leave the exact details to the qpeople, but if we can specify a
> copy depth for each device, where 0=copy nothing, n=copy everything,
> 1=copy the last level (equivalent to -d blk inc) I think we'll have
> covered everything.
I propose the following syntax:
migrate [-d] [-b [<device:copy_level> ...<device:copy_level>]]
tcp:<host>:<port>
Where device will be the name of the device: ide0-hd0. And copy_level will
be integer or n, 0=flat copy of the device, 1=copy the last level ...
n=copy all levels.
All HD devices that will not appear in the command will be treated as
copy_level=0. All devices != BDRV_TYPE_HD will be ignored.
What do you think?
- Liran
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage
2009-11-03 9:40 ` Liran Schour
@ 2009-11-03 10:38 ` Avi Kivity
2009-11-23 18:47 ` [Qemu-devel] " Jan Kiszka
0 siblings, 1 reply; 10+ messages in thread
From: Avi Kivity @ 2009-11-03 10:38 UTC (permalink / raw)
To: Liran Schour; +Cc: qemu-devel
On 11/03/2009 11:40 AM, Liran Schour wrote:
> - Liran
>
> Avi Kivity<avi@redhat.com> wrote on 02/11/2009 20:47:34:
>
>> On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
>>
>>> This series adds support for live migration without shared storage,
>>>
> means
>
>>> copy the storage while migrating. It was tested with KVM. Supports 2
>>>
> ways
>
>>> to replicate the storage during migration:
>>> 1. Complete copy of storage to destination
>>> 2. Assuming the storage is cow based, copy only the allocated
>>> data, time of the migration will be linear with the amount of allocated
>>> data (user responsibility to verify that the same backend file reside
>>> on src and destination).
>>>
>>> Live migration will work as follows:
>>> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
>>> (qemu) migrate -d blk tcp:0:4444 # for live migration with
>>>
>> complete storage copy
>>
>>> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
>>>
>> incremental storage copy, storage is cow based.
>>
>>>
>> I'd like to see the syntax generalized. For one, the guest may have
>> several disks; an install image cdrom might be available as an nfs image
>> but the main storage is local. Secondly, there can be several levels of
>> cow and we want to control which one we copy.
>>
>> I'll leave the exact details to the qpeople, but if we can specify a
>> copy depth for each device, where 0=copy nothing, n=copy everything,
>> 1=copy the last level (equivalent to -d blk inc) I think we'll have
>> covered everything.
>>
> I propose the following syntax:
> migrate [-d] [-b [<device:copy_level> ...<device:copy_level>]]
> tcp:<host>:<port>
> Where device will be the name of the device: ide0-hd0. And copy_level will
> be integer or n, 0=flat copy of the device, 1=copy the last level ...
> n=copy all levels.
> All HD devices that will not appear in the command will be treated as
> copy_level=0. All devices != BDRV_TYPE_HD will be ignored.
>
>
Looks good.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Qemu-devel] Re: [PATCH 0/3 v5] Live migration without shared storage
2009-11-03 10:38 ` Avi Kivity
@ 2009-11-23 18:47 ` Jan Kiszka
2009-11-24 14:11 ` Pierre Riteau
0 siblings, 1 reply; 10+ messages in thread
From: Jan Kiszka @ 2009-11-23 18:47 UTC (permalink / raw)
To: Liran Schour; +Cc: Anthony Liguori, Avi Kivity, qemu-devel
Avi Kivity wrote:
> On 11/03/2009 11:40 AM, Liran Schour wrote:
>> - Liran
>>
>> Avi Kivity<avi@redhat.com> wrote on 02/11/2009 20:47:34:
>>
>>> On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
>>>
>>>> This series adds support for live migration without shared storage,
>>>>
>> means
>>
>>>> copy the storage while migrating. It was tested with KVM. Supports 2
>>>>
>> ways
>>
>>>> to replicate the storage during migration:
>>>> 1. Complete copy of storage to destination
>>>> 2. Assuming the storage is cow based, copy only the allocated
>>>> data, time of the migration will be linear with the amount of allocated
>>>> data (user responsibility to verify that the same backend file reside
>>>> on src and destination).
>>>>
>>>> Live migration will work as follows:
>>>> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
>>>> (qemu) migrate -d blk tcp:0:4444 # for live migration with
>>>>
>>> complete storage copy
>>>
>>>> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
>>>>
>>> incremental storage copy, storage is cow based.
>>>
>>>>
>>> I'd like to see the syntax generalized. For one, the guest may have
>>> several disks; an install image cdrom might be available as an nfs image
>>> but the main storage is local. Secondly, there can be several levels of
>>> cow and we want to control which one we copy.
>>>
>>> I'll leave the exact details to the qpeople, but if we can specify a
>>> copy depth for each device, where 0=copy nothing, n=copy everything,
>>> 1=copy the last level (equivalent to -d blk inc) I think we'll have
>>> covered everything.
>>>
>> I propose the following syntax:
>> migrate [-d] [-b [<device:copy_level> ...<device:copy_level>]]
>> tcp:<host>:<port>
>> Where device will be the name of the device: ide0-hd0. And copy_level
>> will
>> be integer or n, 0=flat copy of the device, 1=copy the last level ...
>> n=copy all levels.
>> All HD devices that will not appear in the command will be treated as
>> copy_level=0. All devices != BDRV_TYPE_HD will be ignored.
>>
>>
>
> Looks good.
>
Do I get this correctly: this series is now "only" blocked by the
required rework of its user interface? Is there a schedule for this, a
v6 roll out? Will it be in time for qemu-0.12?
Jan
--
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] Re: [PATCH 0/3 v5] Live migration without shared storage
2009-11-23 18:47 ` [Qemu-devel] " Jan Kiszka
@ 2009-11-24 14:11 ` Pierre Riteau
2009-11-24 14:17 ` Jan Kiszka
0 siblings, 1 reply; 10+ messages in thread
From: Pierre Riteau @ 2009-11-24 14:11 UTC (permalink / raw)
To: Jan Kiszka; +Cc: Anthony Liguori, Avi Kivity, Liran Schour, qemu-devel
On 23 nov. 2009, at 19:47, Jan Kiszka wrote:
> Avi Kivity wrote:
>> On 11/03/2009 11:40 AM, Liran Schour wrote:
>>> - Liran
>>>
>>> Avi Kivity<avi@redhat.com> wrote on 02/11/2009 20:47:34:
>>>
>>>> On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
>>>>
>>>>> This series adds support for live migration without shared storage,
>>>>>
>>> means
>>>
>>>>> copy the storage while migrating. It was tested with KVM. Supports 2
>>>>>
>>> ways
>>>
>>>>> to replicate the storage during migration:
>>>>> 1. Complete copy of storage to destination
>>>>> 2. Assuming the storage is cow based, copy only the allocated
>>>>> data, time of the migration will be linear with the amount of allocated
>>>>> data (user responsibility to verify that the same backend file reside
>>>>> on src and destination).
>>>>>
>>>>> Live migration will work as follows:
>>>>> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
>>>>> (qemu) migrate -d blk tcp:0:4444 # for live migration with
>>>>>
>>>> complete storage copy
>>>>
>>>>> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
>>>>>
>>>> incremental storage copy, storage is cow based.
>>>>
>>>>>
>>>> I'd like to see the syntax generalized. For one, the guest may have
>>>> several disks; an install image cdrom might be available as an nfs image
>>>> but the main storage is local. Secondly, there can be several levels of
>>>> cow and we want to control which one we copy.
>>>>
>>>> I'll leave the exact details to the qpeople, but if we can specify a
>>>> copy depth for each device, where 0=copy nothing, n=copy everything,
>>>> 1=copy the last level (equivalent to -d blk inc) I think we'll have
>>>> covered everything.
>>>>
>>> I propose the following syntax:
>>> migrate [-d] [-b [<device:copy_level> ...<device:copy_level>]]
>>> tcp:<host>:<port>
>>> Where device will be the name of the device: ide0-hd0. And copy_level
>>> will
>>> be integer or n, 0=flat copy of the device, 1=copy the last level ...
>>> n=copy all levels.
>>> All HD devices that will not appear in the command will be treated as
>>> copy_level=0. All devices != BDRV_TYPE_HD will be ignored.
>>>
>>>
>>
>> Looks good.
>>
>
> Do I get this correctly: this series is now "only" blocked by the
> required rework of its user interface? Is there a schedule for this, a
> v6 roll out? Will it be in time for qemu-0.12?
>
> Jan
It was committed to the master branch a week ago.
--
Pierre Riteau -- http://perso.univ-rennes1.fr/pierre.riteau/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] Re: [PATCH 0/3 v5] Live migration without shared storage
2009-11-24 14:11 ` Pierre Riteau
@ 2009-11-24 14:17 ` Jan Kiszka
2009-11-24 14:27 ` Anthony Liguori
0 siblings, 1 reply; 10+ messages in thread
From: Jan Kiszka @ 2009-11-24 14:17 UTC (permalink / raw)
To: Pierre Riteau; +Cc: Anthony Liguori, Avi Kivity, Liran Schour, qemu-devel
Pierre Riteau wrote:
> On 23 nov. 2009, at 19:47, Jan Kiszka wrote:
>
>> Avi Kivity wrote:
>>> On 11/03/2009 11:40 AM, Liran Schour wrote:
>>>> - Liran
>>>>
>>>> Avi Kivity<avi@redhat.com> wrote on 02/11/2009 20:47:34:
>>>>
>>>>> On 11/02/2009 03:40 PM, lirans@il.ibm.com wrote:
>>>>>
>>>>>> This series adds support for live migration without shared storage,
>>>>>>
>>>> means
>>>>
>>>>>> copy the storage while migrating. It was tested with KVM. Supports 2
>>>>>>
>>>> ways
>>>>
>>>>>> to replicate the storage during migration:
>>>>>> 1. Complete copy of storage to destination
>>>>>> 2. Assuming the storage is cow based, copy only the allocated
>>>>>> data, time of the migration will be linear with the amount of allocated
>>>>>> data (user responsibility to verify that the same backend file reside
>>>>>> on src and destination).
>>>>>>
>>>>>> Live migration will work as follows:
>>>>>> (qemu) migrate -d tcp:0:4444 # for ordinary live migration
>>>>>> (qemu) migrate -d blk tcp:0:4444 # for live migration with
>>>>>>
>>>>> complete storage copy
>>>>>
>>>>>> (qemu) migrate -d blk inc tcp:0:4444 # for live migration with
>>>>>>
>>>>> incremental storage copy, storage is cow based.
>>>>>
>>>>> I'd like to see the syntax generalized. For one, the guest may have
>>>>> several disks; an install image cdrom might be available as an nfs image
>>>>> but the main storage is local. Secondly, there can be several levels of
>>>>> cow and we want to control which one we copy.
>>>>>
>>>>> I'll leave the exact details to the qpeople, but if we can specify a
>>>>> copy depth for each device, where 0=copy nothing, n=copy everything,
>>>>> 1=copy the last level (equivalent to -d blk inc) I think we'll have
>>>>> covered everything.
>>>>>
>>>> I propose the following syntax:
>>>> migrate [-d] [-b [<device:copy_level> ...<device:copy_level>]]
>>>> tcp:<host>:<port>
>>>> Where device will be the name of the device: ide0-hd0. And copy_level
>>>> will
>>>> be integer or n, 0=flat copy of the device, 1=copy the last level ...
>>>> n=copy all levels.
>>>> All HD devices that will not appear in the command will be treated as
>>>> copy_level=0. All devices != BDRV_TYPE_HD will be ignored.
>>>>
>>>>
>>> Looks good.
>>>
>> Do I get this correctly: this series is now "only" blocked by the
>> required rework of its user interface? Is there a schedule for this, a
>> v6 roll out? Will it be in time for qemu-0.12?
>>
>> Jan
>
> It was committed to the master branch a week ago.
Oh, indeed, thanks. Due to the fact that this discussion suggested that
there are still open issues, I did not even checked git. And the missing
merge notifications contributed to this as well.
Anyway, good to see this upstream. /me goes testing.
Jan
--
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] Re: [PATCH 0/3 v5] Live migration without shared storage
2009-11-24 14:17 ` Jan Kiszka
@ 2009-11-24 14:27 ` Anthony Liguori
2009-11-24 14:52 ` Liran Schour
0 siblings, 1 reply; 10+ messages in thread
From: Anthony Liguori @ 2009-11-24 14:27 UTC (permalink / raw)
To: Jan Kiszka
Cc: qemu-devel, Anthony Liguori, Avi Kivity, Liran Schour, Pierre Riteau
Jan Kiszka wrote:
> Oh, indeed, thanks. Due to the fact that this discussion suggested that
> there are still open issues, I did not even checked git.
They still need to be addressed. However, I wanted to do that work in
the tree vs. outside of the tree. With all the qmp stuff happening,
that seemed like the rationale thing to do.
--
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Qemu-devel] Re: [PATCH 0/3 v5] Live migration without shared storage
2009-11-24 14:27 ` Anthony Liguori
@ 2009-11-24 14:52 ` Liran Schour
0 siblings, 0 replies; 10+ messages in thread
From: Liran Schour @ 2009-11-24 14:52 UTC (permalink / raw)
To: Anthony Liguori
Cc: qemu-devel, Jan Kiszka, Anthony Liguori, Avi Kivity, Pierre Riteau
"Anthony Liguori" <aliguori@linux.vnet.ibm.com> wrote on 24/11/2009
16:27:58:
> Jan Kiszka wrote:
> > Oh, indeed, thanks. Due to the fact that this discussion suggested that
> > there are still open issues, I did not even checked git.
>
> They still need to be addressed. However, I wanted to do that work in
> the tree vs. outside of the tree. With all the qmp stuff happening,
> that seemed like the rationale thing to do.
>
What do you think about the following syntax:
migrate [-d] URI
[block:default_copy_level:[<device:copy_level>,.....,<device:copy_level>]
copy_level: 0 - flat copy.
1 - copy only first COW level.
..
n - copy all COW levels.
default_copy_level: use this copy level for all devices that are not
mentioned in the migrate command.
I intend to implement it this week, I am planning to implement on the first
phase only copy levels 0 and 1.
It suppose to solve the major problem today that a guest has one block
device that needs to be copied flat and the other block device that needs
to be COW copy.
- Liran
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2009-11-24 14:55 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-11-02 13:40 [Qemu-devel] [PATCH 0/3 v5] Live migration without shared storage lirans
2009-11-02 18:47 ` Avi Kivity
2009-11-03 9:40 ` Liran Schour
2009-11-03 10:38 ` Avi Kivity
2009-11-23 18:47 ` [Qemu-devel] " Jan Kiszka
2009-11-24 14:11 ` Pierre Riteau
2009-11-24 14:17 ` Jan Kiszka
2009-11-24 14:27 ` Anthony Liguori
2009-11-24 14:52 ` Liran Schour
2009-11-03 7:33 ` [Qemu-devel] " Liran Schour
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.