Re: [PATCH 5/6] KVM: x86: never clear irr_pending in kvm_apic_update_apicv

From: Maxim Levitsky <mlevitsk@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>, kvm@vger.kernel.org
Cc: "open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)" 
	<linux-kernel@vger.kernel.org>,
	Wanpeng Li <wanpengli@tencent.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Joerg Roedel <joro@8bytes.org>, "H. Peter Anvin" <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	"maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)"
	<x86@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Jim Mattson <jmattson@google.com>,
	Sean Christopherson <seanjc@google.com>
Subject: Re: [PATCH 5/6] KVM: x86: never clear irr_pending in kvm_apic_update_apicv
Date: Fri, 10 Dec 2021 14:47:44 +0200	[thread overview]
Message-ID: <0a01229bbbb6d133ba164cb5495ad2300eb8d818.camel@redhat.com> (raw)
In-Reply-To: <fbf3e1665357d9517015ad49eee0c9825ed876d4.camel@redhat.com>

On Fri, 2021-12-10 at 14:20 +0200, Maxim Levitsky wrote:
> On Fri, 2021-12-10 at 13:07 +0100, Paolo Bonzini wrote:
> > On 12/9/21 12:54, Maxim Levitsky wrote:
> > > It is possible that during the AVIC incomplete IPI vmexit,
> > > its handler will set irr_pending to true,
> > > but the target vCPU will still see the IRR bit not set,
> > > due to the apparent lack of memory ordering between CPU's vIRR write
> > > that is supposed to happen prior to the AVIC incomplete IPI
> > > vmexit and the write of the irr_pending in that handler.
> > 
> > Are you sure about this?  Store-to-store ordering should be 
> > guaranteed---if not by the architecture---by existing memory barriers 
> > between vmrun returning and avic_incomplete_ipi_interception().  For 
> > example, srcu_read_lock implies an smp_mb().
> > 
> > Even more damning: no matter what internal black magic the processor 
> > could be using to write to IRR, the processor needs to order the writes 
> > against reads of IsRunning on processors without the erratum.  That 
> > would be equivalent to flushing the store buffer, and it would imply 
> > that the write of vIRR is ordered before the write to irr_pending.
> > 
> > Paolo
> > 
> Yes I almost 100% sure now that this patch is wrong.
> the code was just seeing irr_pending true because it is set
> to true while APICv/AVIC is use, and was not seeing yet the vIRR bits,
> because they didn't arrive yet. This this patch isn't needed.
> 
> Thanks again for help!
> I am testing your version of fixes to avic inhibition races,
> and then I'll send a new version of these patches.
> 
> Best regards,
> 	Maxim Levitsky

And yet that patch is needed for a differnt reason.

If the sender has AVIC enabled, it can turn on vIRR bits at any moment
without setting irr_pending = true - there are no VMexits happeing
on the sender side.

If we scan vIRR here and see no bits, and *then* disable AVIC,
there is a window where the they could legit be turned on without any cpu errata,
and we will not have irr_pending == true, and thus the following 
KVM_REQ_EVENT will make no difference.

Not touching irr_pending and letting just the KVM_REQ_EVENT do the work
will work too, and if the avic errata is present, reduce slightly
the chances of it happening.

Best regards,
	Maxim Levitsky