Re: Fwd: kvm: Windows Server 2003 VM fails to work on 6.1.44 (works fine on 6.1.43)

From: Maxim Levitsky <mlevitsk@redhat.com>
To: Vitaly Kuznetsov <vkuznets@redhat.com>,
	Bagas Sanjaya <bagasdotme@gmail.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Roman Mamedov <rm+bko@romanrm.net>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linux Regressions <regressions@lists.linux.dev>,
	Linux KVM <kvm@vger.kernel.org>, Borislav Petkov <bp@alien8.de>
Subject: Re: Fwd: kvm: Windows Server 2003 VM fails to work on 6.1.44 (works fine on 6.1.43)
Date: Wed, 16 Aug 2023 19:53:05 +0300	[thread overview]
Message-ID: <6b2aedbcff7625574596b363651e0bbd76b03140.camel@redhat.com> (raw)
In-Reply-To: <87cyzn5cln.fsf@redhat.com>

У ср, 2023-08-16 у 15:41 +0200, Vitaly Kuznetsov пише:
> Vitaly Kuznetsov <vkuznets@redhat.com> writes:
> 
> > Vitaly Kuznetsov <vkuznets@redhat.com> writes:
> > 
> > > Bagas Sanjaya <bagasdotme@gmail.com> writes:
> > > 
> > > > Hi,
> > > > 
> > > > I notice a regression report on Bugzilla [1]. Quoting from it:
> > > > 
> > > > > Hello,
> > > > > 
> > > > > I have a virtual machine running the old Windows Server 2003. On kernels 6.1.44 and 6.1.45, the QEMU VNC window stays dark, not switching to any of the guest's video modes and the VM process uses only ~64 MB of RAM of the assigned 2 GB, indefinitely. It's like the VM is paused/halted/stuck before even starting. The process can be killed successfully and then restarted again (with the same result), so it is not deadlocked in kernel or the like.
> > > > > 
> > > > > Kernel 6.1.43 works fine.
> > > > > 
> > > > > I have also tried downgrading CPU microcode from 20230808 to 20230719, but that did not help.
> > > > > 
> > > > > The CPU is AMD Ryzen 5900. I suspect some of the newly added mitigations may be the culprit?
> > > > 
> > > > See Bugzilla for the full thread.
> > > > 
> > > > Anyway, I'm adding it to regzbot as stable-specific regression:
> > > > 
> > > > #regzbot introduced: v6.1.43..v6.1.44 https://bugzilla.kernel.org/show_bug.cgi?id=217799
> > > > #regzbot title: Windows Server 2003 VM boot hang (only 64MB RAM allocated)
> > > > 
> > > > Thanks.
> > > > 
> > > > [1]: https://bugzilla.kernel.org/show_bug.cgi?id=217799
> > > 
> > > From KVM's PoV, I don't see any KVM/x86 patches v6.1.44..v6.1.45 
> > 
> > Oh, sorry, my bad, in the description of the BZ it is said that 6.1.44
> > is already broken, so it's most likely srso stuff then:
> > 
> > dd5f2ef16e3c x86: fix backwards merge of GDS/SRSO bit
> > 4f25355540ad x86/srso: Tie SBPB bit setting to microcode patch detection
> > 77cf32d0dbfb x86/srso: Add a forgotten NOENDBR annotation
> > c7f2cd045542 x86/srso: Fix return thunks in generated code
> > c9ae63d773ca x86/srso: Add IBPB on VMEXIT
> > 79c8091888ef x86/srso: Add IBPB
> > 98f62883e751 x86/srso: Add SRSO_NO support
> > 9139f4b6dd4f x86/srso: Add IBPB_BRTYPE support
> > ac41e90d8daa x86/srso: Add a Speculative RAS Overflow mitigation
> 
> Sean's https://lore.kernel.org/all/20230811155255.250835-1-seanjc@google.com/
> (alteady in 'tip') can actually be related and I see it was already
> tagged for stable@. Can anyone check if it really helps?
> 
> > dec3b91f2c4b x86/cpu, kvm: Add support for CPUID_80000021_EAX
> > 
> > it would still be great to try to bisect to the particular patch causing
> > the issue.

My 0.2 cents on something that might be related:

On my Intel laptop I can't boot a windows guest with hyperv enabled inside (either regular hyperv win10 or win11 with core isolation)
I know now that 'ibt=off' on host kernel line fixes this, but I didn't yet bisected it to see which commit started it.
(I took this from https://bugzilla.redhat.com/show_bug.cgi?id=2221531, which is unrelated but I just noticed it somehow and tried the solution)

I run upstream 6.4 kernel + kvm/queue on that laptop.

Best regards,
	Maxim Levitsky