From: Andrew Jones <drjones@redhat.com>
To: 张东旭 <xu910121@sina.com>
Cc: "marc.zyngier" <marc.zyngier@arm.com>,
qemu-devel <qemu-devel@nongnu.org>,
"Dave.Martin" <dave.martin@arm.com>,
kvmarm <kvmarm@lists.cs.columbia.edu>
Subject: Re: Kernel patch cases qemu live migration failed.
Date: Thu, 15 Oct 2020 15:26:29 +0200 [thread overview]
Message-ID: <20201015132629.n4k5k7wdruclcrev@kamzik.brq.redhat.com> (raw)
In-Reply-To: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn>
On Thu, Oct 15, 2020 at 12:06:39PM +0800, 张东旭 wrote:
> I'm so sorry for disturbing you.When I apply this kernel patch：KVM: arm64/sve: System register context switch and access supporthttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/kvm/sys_regs.c?id=73433762fcaeb9d59e84d299021c6b15466c96ddqemu live migration will failed with messages： qemu-kvm: Invalid value 233 expecting positive value <= 232 qemu-kvm: Failed to load cpu:cpreg_vmstate_array_lenNew version kernel exclude ID_AA64ZFR0_EL1 register when host not support SVE feature, so qemu ioctl kvm(KVM_GET_REG_LIST) will not contain ID_AA64ZFR0_EL1 register.
> I'm using CentOS Linux kernel, old kernel version：4.18.0-80.11.el8 (migration source) new kernel version：4.18.0-147.5.el8 (migration destination)
> CentOS linux kernel version 4.18.0-111.el8 applied this patch. So 4.18.0-147.5.el8 also applied this patch.
> Migration source and destination hosts have the same hardware, just kernel version is different, and the hardware on either side of the migration not support SVE.
> Is there some good suggestions，which can make sure old version kernel live migration to new version kernel with qemu?Thanks a lot.
ARM KVM guests must be run with CPU host passthrough. This means the
host hardware and host kernel versions must be identical in order
to guarantee successful migrations.
That said, upgrading the host kernel without shutting down the guests
is a reasonable thing to try. Unfortunately, in this case, the only
way to do it would be to hack QEMU on the destination to allow this
extra register in KVM_GET_REG_LIST. It should be harmless, as it's
not used. Allowing it is similar to allowing the destination to have
a larger number of registers than the source in KVM_GET_REG_LIST (which
is why the failing test is <=, not ==).
I wouldn't post a real patch to resolve this issue, though, as everything
is working as expected. The failing test is failing because it detected
a risky migration. And, KVM's filtering of registers from KVM_GET_REG_LIST
is also correct, even if previous KVM versions didn't do that. With CPU
models you could request the reg list for a particular model and expect
it to be the same across all kernel versions that support it, but we
don't have those.
Thanks,
drew
WARNING: multiple messages have this Message-ID (diff)
From: Andrew Jones <drjones@redhat.com>
To: 张东旭 <xu910121@sina.com>
Cc: "marc.zyngier" <marc.zyngier@arm.com>,
qemu-devel <qemu-devel@nongnu.org>,
"Dave.Martin" <dave.martin@arm.com>,
kvmarm <kvmarm@lists.cs.columbia.edu>
Subject: Re: Kernel patch cases qemu live migration failed.
Date: Thu, 15 Oct 2020 15:26:29 +0200 [thread overview]
Message-ID: <20201015132629.n4k5k7wdruclcrev@kamzik.brq.redhat.com> (raw)
In-Reply-To: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn>
On Thu, Oct 15, 2020 at 12:06:39PM +0800, 张东旭 wrote:
> I'm so sorry for disturbing you.When I apply this kernel patch：KVM: arm64/sve: System register context switch and access supporthttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/kvm/sys_regs.c?id=73433762fcaeb9d59e84d299021c6b15466c96ddqemu live migration will failed with messages： qemu-kvm: Invalid value 233 expecting positive value <= 232 qemu-kvm: Failed to load cpu:cpreg_vmstate_array_lenNew version kernel exclude ID_AA64ZFR0_EL1 register when host not support SVE feature, so qemu ioctl kvm(KVM_GET_REG_LIST) will not contain ID_AA64ZFR0_EL1 register.
> I'm using CentOS Linux kernel, old kernel version：4.18.0-80.11.el8 (migration source) new kernel version：4.18.0-147.5.el8 (migration destination)
> CentOS linux kernel version 4.18.0-111.el8 applied this patch. So 4.18.0-147.5.el8 also applied this patch.
> Migration source and destination hosts have the same hardware, just kernel version is different, and the hardware on either side of the migration not support SVE.
> Is there some good suggestions，which can make sure old version kernel live migration to new version kernel with qemu?Thanks a lot.
ARM KVM guests must be run with CPU host passthrough. This means the
host hardware and host kernel versions must be identical in order
to guarantee successful migrations.
That said, upgrading the host kernel without shutting down the guests
is a reasonable thing to try. Unfortunately, in this case, the only
way to do it would be to hack QEMU on the destination to allow this
extra register in KVM_GET_REG_LIST. It should be harmless, as it's
not used. Allowing it is similar to allowing the destination to have
a larger number of registers than the source in KVM_GET_REG_LIST (which
is why the failing test is <=, not ==).
I wouldn't post a real patch to resolve this issue, though, as everything
is working as expected. The failing test is failing because it detected
a risky migration. And, KVM's filtering of registers from KVM_GET_REG_LIST
is also correct, even if previous KVM versions didn't do that. With CPU
models you could request the reg list for a particular model and expect
it to be the same across all kernel versions that support it, but we
don't have those.
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
next prev parent reply other threads:[~2020-10-15 13:27 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-15 4:06 Kernel patch cases qemu live migration failed 张东旭
2020-10-15 4:06 ` 张东旭
2020-10-15 11:26 ` Marc Zyngier
2020-10-15 11:26 ` Marc Zyngier
2020-10-15 13:35 ` Andrew Jones
2020-10-15 13:35 ` Andrew Jones
2020-10-15 13:52 ` Marc Zyngier
2020-10-15 13:52 ` Marc Zyngier
2020-10-15 14:41 ` Andrew Jones
2020-10-15 14:41 ` Andrew Jones
2020-10-15 14:57 ` Peter Maydell
2020-10-15 14:57 ` Peter Maydell
2020-10-19 9:25 ` Andrew Jones
2020-10-19 9:25 ` Andrew Jones
2020-10-19 11:32 ` Dave Martin
2020-10-19 11:32 ` Dave Martin
2020-10-19 11:43 ` Peter Maydell
2020-10-19 11:43 ` Peter Maydell
2020-10-19 13:40 ` Andrew Jones
2020-10-19 13:40 ` Andrew Jones
2020-10-19 14:18 ` Peter Maydell
2020-10-19 14:18 ` Peter Maydell
2020-10-19 14:58 ` Dave Martin
2020-10-19 14:58 ` Dave Martin
2020-10-19 15:23 ` Andrew Jones
2020-10-19 15:23 ` Andrew Jones
2020-10-19 16:36 ` Dave Martin
2020-10-19 16:36 ` Dave Martin
2020-10-15 13:26 ` Andrew Jones [this message]
2020-10-15 13:26 ` Andrew Jones
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201015132629.n4k5k7wdruclcrev@kamzik.brq.redhat.com \
--to=drjones@redhat.com \
--cc=dave.martin@arm.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=marc.zyngier@arm.com \
--cc=qemu-devel@nongnu.org \
--cc=xu910121@sina.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.