Re: Possible nohz-full/RCU issue in arm64 KVM

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
To: Mark Rutland <mark.rutland@arm.com>, Paolo Bonzini <pbonzini@redhat.com>
Cc: paulmck <paulmck@kernel.org>, maz <maz@kernel.org>,
	frederic <frederic@kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	rcu <rcu@vger.kernel.org>, Thomas Gleixner <tglx@linutronix.de>,
	Will Deacon <will@kernel.org>,
	kvmarm@lists.cs.columbia.edu,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
	Anup Patel <Anup.Patel@wdc.com>
Subject: Re: Possible nohz-full/RCU issue in arm64 KVM
Date: Tue, 11 Jan 2022 12:32:38 +0100	[thread overview]
Message-ID: <399d8805ca09f7d3c905b8c653abf60dd7141574.camel@redhat.com> (raw)
In-Reply-To: <YdR4N9QVYOzjowAb@FVFF77S0Q05N>

Hi Mark,

On Tue, 2022-01-04 at 16:39 +0000, Mark Rutland wrote:
> On Fri, Dec 17, 2021 at 04:54:22PM +0100, Paolo Bonzini wrote:
> > On 12/17/21 15:38, Mark Rutland wrote:
> > > For example kvm_guest_enter_irqoff() calls guest_enter_irq_off() which calls
> > > vtime_account_guest_enter(), but kvm_guest_exit_irqoff() doesn't call
> > > guest_exit_irq_off() and the call to vtime_account_guest_exit() is open-coded
> > > elsewhere. Also, guest_enter_irq_off() conditionally calls
> > > rcu_virt_note_context_switch(), but I can't immediately spot anything on the
> > > exit side that corresponded with that, which looks suspicious.
> > 
> > rcu_note_context_switch() is a point-in-time notification; it's not strictly
> > necessary, but it may improve performance a bit by avoiding unnecessary IPIs
> > from the RCU subsystem.
> > 
> > There's no benefit from doing it when you're back from the guest, because at
> > that point the CPU is just running normal kernel code.
> 
> I see.
> 
> My main issue here was just that it's really difficult to see how the
> entry/exit logic is balanced, and I reckon we can solve that by splitting
> guest_{enter,exit}_irqoff() into helper functions to handle the vtime
> accounting separately from the context tracking, so that arch code can do
> something like:
> 
>   guest_timing_enter_irqoff();
>   
>   guest_eqs_enter_irqoff();
>   < actually run vCPU here >
>   guest_eqs_exit_irqoff();
>   
>   < handle pending IRQs here >
>   
>   guest_timing_exit_irqoff();
> 
> ... which I hope should work for RISC-V too.
> 
> I've had a go, and I've pushed out a WIP to:
> 
>   https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/log/?h=arm64/kvm/rcu

Had a look at the patches and they seeem OK to me.

Thanks!

-- 
Nicolás Sáenz