From: Sean Christopherson <seanjc@google.com>
To: Peter Xu <peterx@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Sasha Levin <sashal@kernel.org>,
linux-kernel@vger.kernel.org, stable@vger.kernel.org,
Leonardo Bras <leobras@redhat.com>,
tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org,
chang.seok.bae@intel.com, luto@kernel.org, kvm@vger.kernel.org
Subject: Re: [PATCH AUTOSEL 5.16 07/28] x86/kvm/fpu: Limit guest user_xfeatures to supported bits of XCR0
Date: Tue, 7 Jun 2022 18:47:41 +0000 [thread overview]
Message-ID: <Yp+dTU4NOaIELJh8@google.com> (raw)
In-Reply-To: <Yp+WUoA+6x7ZpsaM@xz-m1.local>
On Tue, Jun 07, 2022, Peter Xu wrote:
> On Tue, Jun 07, 2022 at 03:04:27PM +0000, Sean Christopherson wrote:
> > On Tue, Jun 07, 2022, Paolo Bonzini wrote:
> > > On 6/6/22 23:27, Peter Xu wrote:
> > > > On Mon, Jun 06, 2022 at 06:18:12PM +0200, Paolo Bonzini wrote:
> > > > > > However there seems to be something missing at least to me, on why it'll
> > > > > > fail a migration from 5.15 (without this patch) to 5.18 (with this patch).
> > > > > > In my test case, user_xfeatures will be 0x7 (FP|SSE|YMM) if without this
> > > > > > patch, but 0x0 if with it.
> > > > >
> > > > > What CPU model are you using for the VM?
> > > >
> > > > I didn't specify it, assuming it's qemu64 with no extra parameters.
> > >
> > > Ok, so indeed it lacks AVX and this patch can have an effect.
> > >
> > > > > For example, if the source lacks this patch but the destination has it,
> > > > > the source will transmit YMM registers, but the destination will fail to
> > > > > set them if they are not available for the selected CPU model.
> > > > >
> > > > > See the commit message: "As a bonus, it will also fail if userspace tries to
> > > > > set fpu features (with the KVM_SET_XSAVE ioctl) that are not compatible to
> > > > > the guest configuration. Such features will never be returned by
> > > > > KVM_GET_XSAVE or KVM_GET_XSAVE2."
> > > >
> > > > IIUC you meant we should have failed KVM_SET_XSAVE when they're not aligned
> > > > (probably by failing validate_user_xstate_header when checking against the
> > > > user_xfeatures on dest host). But that's probably not my case, because here
> > > > KVM_SET_XSAVE succeeded, it's just that the guest gets a double fault after
> > > > the precopy migration completes (or for postcopy when the switchover is
> > > > done).
> > >
> > > Difficult to say what's happening without seeing at least the guest code
> > > around the double fault (above you said "fail a migration" and I thought
> > > that was a different scenario than the double fault), and possibly which was
> > > the first exception that contributed to the double fault.
> >
> > Regardless of why the guest explodes in the way it does, is someone planning on
> > bisecting this (if necessary?) and sending a backport to v5.15? There's another
> > bug report that is more than likely hitting the same bug.
>
> What's the bisection you mentioned? I actually did a bisection and I also
> checked reverting Leo's change can also fix this issue. Or do you mean
> something else?
Oooooh, sorry! I got completely turned around. You ran into a bug with the
fix. I thought that you were hitting the same issues as Mike where migrating
between hosts with different capabilities is broken in v5.15, but works in v5.18.
> > https://lore.kernel.org/all/48353e0d-e771-8a97-21d4-c65ff3bc4192@sentex.net
>
> That is kvm64, and I agree it could be the same problem since both qemu64
> and kvm64 models do not have any xsave feature bit declared in cpuid 0xd,
> so potentially we could be migrating some fpu states to it even with
> user_xfeatures==0 on dest host.
>
> So today I continued the investigation, and I think what's really missing
> is qemu seems to be ignoring the user_xfeatures check for KVM_SET_XSAVE and
> continues even if it returns -EINVAL. IOW, I'm wondering whether we should
> fail properly and start to check kvm_arch_put_registers() retcode. But
> that'll be a QEMU fix, and it'll at least not causing random faults
> (e.g. double faults) in guest but we should fail the migration gracefully.
>
> Sean: a side note is that I can also easily trigger one WARN_ON_ONCE() in
> your commit 98c25ead5eda5 in kvm_arch_vcpu_ioctl_run():
>
> WARN_ON_ONCE(kvm_lapic_hv_timer_in_use(vcpu));
>
> It'll be great if you'd like to check that up.
Ugh, userspace can force KVM_MP_STATE_UNINITIALIZED via KVM_SET_MP_STATE. Looks
like QEMU does that when emulating RESET.
Logically, a full RESET of the xAPIC seems like the right thing to do. I think
we can get away with that without breaking ABI? And kvm_lapic_reset() has a
related bug where it stops the HR timer but not doesn't handle the HV timer :-/
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index e69b83708f05..948aba894245 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2395,7 +2395,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event)
return;
/* Stop the timer in case it's a reset to an active apic */
- hrtimer_cancel(&apic->lapic_timer.timer);
+ cancel_apic_timer(&apic->lapic_timer.timer);
/* The xAPIC ID is set at RESET even if the APIC was already enabled. */
if (!init_event)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 540651cd28d7..ed2c7cb1642d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -10912,6 +10912,9 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
mp_state->mp_state == KVM_MP_STATE_INIT_RECEIVED))
goto out;
+ if (mp_state->mp_state == KVM_MP_STATE_UNINITIALIZED)
+ kvm_lapic_reset(vcpu, false);
+
if (mp_state->mp_state == KVM_MP_STATE_SIPI_RECEIVED) {
vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED;
set_bit(KVM_APIC_SIPI, &vcpu->arch.apic->pending_events);
next prev parent reply other threads:[~2022-06-07 21:03 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-01 20:13 [PATCH AUTOSEL 5.16 01/28] selftests/bpf: Add test for bpf_timer overwriting crash Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 02/28] swiotlb: fix info leak with DMA_FROM_DEVICE Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 03/28] usb: dwc3: pci: add support for the Intel Raptor Lake-S Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 04/28] pinctrl: tigerlake: Revert "Add Alder Lake-M ACPI ID" Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 05/28] KVM: Fix lockdep false negative during host resume Sasha Levin
2022-03-01 20:19 ` Paolo Bonzini
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 06/28] kvm: x86: Disable KVM_HC_CLOCK_PAIRING if tsc is in always catchup mode Sasha Levin
2022-03-01 20:19 ` Paolo Bonzini
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 07/28] x86/kvm/fpu: Limit guest user_xfeatures to supported bits of XCR0 Sasha Levin
2022-03-01 20:22 ` Paolo Bonzini
2022-06-03 18:40 ` Peter Xu
2022-06-06 16:18 ` Paolo Bonzini
2022-06-06 21:27 ` Peter Xu
2022-06-07 12:54 ` Paolo Bonzini
2022-06-07 15:04 ` Sean Christopherson
2022-06-07 18:17 ` Peter Xu
2022-06-07 18:47 ` Sean Christopherson [this message]
2022-06-07 21:01 ` Peter Xu
2022-06-07 18:55 ` Peter Xu
2022-06-08 20:34 ` Leonardo Bras Soares Passos
2022-06-08 20:53 ` Peter Xu
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 08/28] spi: rockchip: Fix error in getting num-cs property Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 09/28] spi: rockchip: terminate dma transmission when slave abort Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 10/28] drm/vc4: hdmi: Unregister codec device on unbind Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 11/28] of/fdt: move elfcorehdr reservation early for crash dump kernel Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 12/28] x86/kvm: Don't use pv tlb/ipi/sched_yield if on 1 vCPU Sasha Levin
2022-03-01 20:17 ` Paolo Bonzini
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 13/28] drivers: hamradio: 6pack: fix UAF bug caused by mod_timer() Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 14/28] net-sysfs: add check for netdevice being present to speed_show Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 15/28] sr9700: sanity check for packet length Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 16/28] hwmon: (pmbus) Clear pmbus fault/warning bits after read Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 17/28] nvme-tcp: send H2CData PDUs based on MAXH2CDATA Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 18/28] PCI: Mark all AMD Navi10 and Navi14 GPU ATS as broken Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 19/28] gpio: Return EPROBE_DEFER if gc->to_irq is NULL Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 20/28] drm/amdgpu: bypass tiling flag check in virtual display case (v2) Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 21/28] Revert "xen-netback: remove 'hotplug-status' once it has served its purpose" Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 22/28] Revert "xen-netback: Check for hotplug-status existence before watching" Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 23/28] ipv6: prevent a possible race condition with lifetimes Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 24/28] tracing: Ensure trace buffer is at least 4096 bytes large Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 25/28] tracing/osnoise: Make osnoise_main to sleep for microseconds Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 26/28] tracing: Fix selftest config check for function graph start up test Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 27/28] selftest/vm: fix map_fixed_noreplace test failure Sasha Levin
2022-03-01 20:13 ` [PATCH AUTOSEL 5.16 28/28] selftests/memfd: clean up mapping in mfd_fail_write Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yp+dTU4NOaIELJh8@google.com \
--to=seanjc@google.com \
--cc=bp@alien8.de \
--cc=chang.seok.bae@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=leobras@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=sashal@kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).