Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Nils Cant]

Hi,

I currently have a couple of Debian KVM servers with all a different 
version of kvm or qemu-kvm.

I can live migrate a guest OS from one server to the other just fine, as 
long as the version of qemu-kvm is the same. (Thanks to Gleb Natapov who 
helped me solve my ARP announce problem yesterday)

However, when I try to migrate a guest to a server running a newer (or 
older) version of qemu-kvm, I run into problems. I think this is because 
the xml configuration differs slightly between version.

For example, if I try to migrate from a server running qemu-kvm 0.11.0 
to one running qemu-kvm 0.12.4, I get the following error:

virt01:~# virsh migrate --live testserver qemu+ssh://192.168.1.7/system
error: internal error unable to reserve PCI address 0:0:3

When migrating from kvm 85 to qemu-kvm 0.11.0:

virt02:~# virsh migrate --live testserver qemu+ssh://192.168.1.2/system
error: Unknown failure

Migrating one from 0.12.4 to 0.11.0 just completely breaks libvirt, 
forcing me to kill -9 libvirtd and the kvm instance, etc.

I believe this is caused in part because different versions of qemu-kvm 
need different xml configurations. To run my testserver on 0.12.4 for 
example, I need to delete the lines with the PCI id's, just to be able 
to start the it.

So, is there a way to make this work? Perhaps without libvirt? Or tell 
libvirt to not send the config file when doing a live migrate, and use 
the xml in /etc/libvirt/qemu instead somehow?

Thanks in advance,

Nils

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Nils Cant]

Hey guys,

next try is without libvirt, but still no joy.

After issuing 'migrate -d <url>' on my sending host (qemu-kvm 0.11.0), I 
get the following output on the receiving host (qemu-kvm 0.12.4):

(qemu) Unknown savevm section or instance 'slirp' 0
load of migration failed

... leaving my vm broken.

Am I doing something wrong, or is the format of the savevm file just 
different and should I abandon all hope of ever doing a live migration 
to a newer version of qemu-kvm?

Thanks in advance,


Nils

---

Here are my upstart options on the 'sender':

/usr/bin/kvm -S \
         -M pc-0.11 \
         -enable-kvm \
         -m 512 \
         -smp 2,sockets=2,cores=1,threads=1 \
         -name testserver \
         -uuid 890a0156-0542-32d8-66d7-b36a711084cc \
         -monitor 
unix:/var/lib/libvirt/qemu/testserver.monitor,server,nowait \
         -boot c \
         -drive 
file=/dev/disk/by-path/ip-192.168.3.100:3260-iscsi-iqn.2003-10.com.lefthandnetworks:lefthand0:64:testserver-lun-0,if=virtio,boot=on 
\
         -drive media=cdrom \
         -usb \
         -vnc 0.0.0.0:10 \
         -k en-us \
         -vga cirrus &

And the 'receiver':

/usr/bin/kvm -S \
         -M pc-0.11 \
         -enable-kvm \
         -m 512 \
         -smp 2,sockets=2,cores=1,threads=1 \
         -name testserver \
         -uuid 890a0156-0542-32d8-66d7-b36a711084cc \
         -nodefaults \
         -chardev 
socket,id=monitor,path=/var/lib/libvirt/qemu/testserver.monitor,server,nowait 
\
         -mon chardev=monitor,mode=readline \
         -boot c \
         -drive 
file=/dev/disk/by-path/ip-192.168.3.100:3260-iscsi-iqn.2003-10.com.lefthandnetworks:lefthand0:64:testserver-lun-0,if=none,id=drive-virtio-disk0,boot=on 
\
         -drive if=none,media=cdrom,id=drive-ide0-1-0 \
         -device 
ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
         -device 
virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 \
         -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 \
         -chardev pty,id=serial0 \
         -device isa-serial,chardev=serial0 \
         -usb \
         -vnc 0.0.0.0:0 \
         -k en-us \
         -vga cirrus \
         -incoming tcp:0:4444 &

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Brian Jackson]

On Monday, August 30, 2010 07:04:51 am Nils Cant wrote:
> Hey guys,
> 
> next try is without libvirt, but still no joy.
> 
> After issuing 'migrate -d <url>' on my sending host (qemu-kvm 0.11.0), I
> get the following output on the receiving host (qemu-kvm 0.12.4):


A quick search of this and/or the qemu mailing lists would have told you this 
is unsupported.


> 
> (qemu) Unknown savevm section or instance 'slirp' 0
> load of migration failed
> 
> ... leaving my vm broken.
> 
> Am I doing something wrong, or is the format of the savevm file just
> different and should I abandon all hope of ever doing a live migration
> to a newer version of qemu-kvm?
> 
> Thanks in advance,
> 
> 
> Nils
> 
> ---
> 
> Here are my upstart options on the 'sender':
> 
> /usr/bin/kvm -S \
>          -M pc-0.11 \
>          -enable-kvm \
>          -m 512 \
>          -smp 2,sockets=2,cores=1,threads=1 \
>          -name testserver \
>          -uuid 890a0156-0542-32d8-66d7-b36a711084cc \
>          -monitor
> unix:/var/lib/libvirt/qemu/testserver.monitor,server,nowait \
>          -boot c \
>          -drive
> file=/dev/disk/by-path/ip-192.168.3.100:3260-iscsi-iqn.2003-10.com.lefthand
> networks:lefthand0:64:testserver-lun-0,if=virtio,boot=on \
>          -drive media=cdrom \
>          -usb \
>          -vnc 0.0.0.0:10 \
>          -k en-us \
>          -vga cirrus &
> 
> And the 'receiver':
> 
> /usr/bin/kvm -S \
>          -M pc-0.11 \
>          -enable-kvm \
>          -m 512 \
>          -smp 2,sockets=2,cores=1,threads=1 \
>          -name testserver \
>          -uuid 890a0156-0542-32d8-66d7-b36a711084cc \
>          -nodefaults \
>          -chardev
> socket,id=monitor,path=/var/lib/libvirt/qemu/testserver.monitor,server,nowa
> it \
>          -mon chardev=monitor,mode=readline \
>          -boot c \
>          -drive
> file=/dev/disk/by-path/ip-192.168.3.100:3260-iscsi-iqn.2003-10.com.lefthand
> networks:lefthand0:64:testserver-lun-0,if=none,id=drive-virtio-disk0,boot=o
> n \
>          -drive if=none,media=cdrom,id=drive-ide0-1-0 \
>          -device
> ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
>          -device
> virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0
> \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 \ -chardev
> pty,id=serial0 \
>          -device isa-serial,chardev=serial0 \
>          -usb \
>          -vnc 0.0.0.0:0 \
>          -k en-us \
>          -vga cirrus \
>          -incoming tcp:0:4444 &
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Avi Kivity]

  On 08/30/2010 11:16 PM, Brian Jackson wrote:
> On Monday, August 30, 2010 07:04:51 am Nils Cant wrote:
>> Hey guys,
>>
>> next try is without libvirt, but still no joy.
>>
>> After issuing 'migrate -d<url>' on my sending host (qemu-kvm 0.11.0), I
>> get the following output on the receiving host (qemu-kvm 0.12.4):
>
> A quick search of this and/or the qemu mailing lists would have told you this
> is unsupported.
>

It should work between 0.12 and 0.13, though.

-- 
error compiling committee.c: too many arguments to function

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Nils Cant]

Is this something we can depend on for future releases then?

If we decide now to build a virtual hosting farm on 0.12, will we be 
able to use live migration in the future if we decide we want to upgrade 
to 0.13 or later?

N.

On 08/31/2010 02:24 PM, Avi Kivity wrote:
>
> It should work between 0.12 and 0.13, though.
>

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Avi Kivity]

  On 09/03/2010 02:35 PM, Nils Cant wrote:

First, don't trim the cc list.  It's evil.

> Is this something we can depend on for future releases then?
>
> If we decide now to build a virtual hosting farm on 0.12, will we be 
> able to use live migration in the future if we decide we want to 
> upgrade to 0.13 or later?

I believe so.  Juan, can you confirm?

-- 
error compiling committee.c: too many arguments to function

Re: Is it possible to live migrate guest OS'es between different versions of kvm/qemu-kvm?

[Nils Cant]

Hey Juan,

do you also think future releases of kvm will have support for live 
migration from older versions? We're quite keen on knowing this, as we 
need to make a decision on what to build our stuff on. Being able to 
transparently move virtual servers onto a host running a newer version 
is definitely a feature we need...

Thanks in advance,

Nils

On 09/05/2010 10:39 AM, Avi Kivity wrote:
> On 09/03/2010 02:35 PM, Nils Cant wrote:
>
> First, don't trim the cc list. It's evil.
>
>> Is this something we can depend on for future releases then?
>>
>> If we decide now to build a virtual hosting farm on 0.12, will we be
>> able to use live migration in the future if we decide we want to
>> upgrade to 0.13 or later?
>
> I believe so. Juan, can you confirm?
>