From: Mark Rutland <mark.rutland@arm.com>
To: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>, paulmck <paulmck@kernel.org>,
maz <maz@kernel.org>, frederic <frederic@kernel.org>,
linux-kernel <linux-kernel@vger.kernel.org>,
rcu <rcu@vger.kernel.org>, Thomas Gleixner <tglx@linutronix.de>,
Will Deacon <will@kernel.org>,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
Anup Patel <Anup.Patel@wdc.com>
Subject: Re: Possible nohz-full/RCU issue in arm64 KVM
Date: Tue, 11 Jan 2022 12:23:10 +0000 [thread overview]
Message-ID: <Yd12rvDxyIWzEXWc@FVFF77S0Q05N> (raw)
In-Reply-To: <399d8805ca09f7d3c905b8c653abf60dd7141574.camel@redhat.com>
On Tue, Jan 11, 2022 at 12:32:38PM +0100, Nicolas Saenz Julienne wrote:
> Hi Mark,
>
> On Tue, 2022-01-04 at 16:39 +0000, Mark Rutland wrote:
> > On Fri, Dec 17, 2021 at 04:54:22PM +0100, Paolo Bonzini wrote:
> > > On 12/17/21 15:38, Mark Rutland wrote:
> > > > For example kvm_guest_enter_irqoff() calls guest_enter_irq_off() which calls
> > > > vtime_account_guest_enter(), but kvm_guest_exit_irqoff() doesn't call
> > > > guest_exit_irq_off() and the call to vtime_account_guest_exit() is open-coded
> > > > elsewhere. Also, guest_enter_irq_off() conditionally calls
> > > > rcu_virt_note_context_switch(), but I can't immediately spot anything on the
> > > > exit side that corresponded with that, which looks suspicious.
> > >
> > > rcu_note_context_switch() is a point-in-time notification; it's not strictly
> > > necessary, but it may improve performance a bit by avoiding unnecessary IPIs
> > > from the RCU subsystem.
> > >
> > > There's no benefit from doing it when you're back from the guest, because at
> > > that point the CPU is just running normal kernel code.
> >
> > I see.
> >
> > My main issue here was just that it's really difficult to see how the
> > entry/exit logic is balanced, and I reckon we can solve that by splitting
> > guest_{enter,exit}_irqoff() into helper functions to handle the vtime
> > accounting separately from the context tracking, so that arch code can do
> > something like:
> >
> > guest_timing_enter_irqoff();
> >
> > guest_eqs_enter_irqoff();
> > < actually run vCPU here >
> > guest_eqs_exit_irqoff();
> >
> > < handle pending IRQs here >
> >
> > guest_timing_exit_irqoff();
> >
> > ... which I hope should work for RISC-V too.
> >
> > I've had a go, and I've pushed out a WIP to:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/log/?h=arm64/kvm/rcu
>
> Had a look at the patches and they seeem OK to me.
>
> Thanks!
Cool.
FWIW I have an updated version at:
https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/log/?h=kvm/entry-rework
... which is largely the same approach, but the helpers got renamed, the
lockdep/tracing bits got fixed, and I've aligned mips, riscv, and x86 on the
same approach.
Once I get a free hour or so I intend to rebase that atop v5.16 and post that
out. I'll start a new thread with that, and rope in the relevant arch
maintainers (since e.g. I'm not sure what to do for ppc and s390).
Thanks,
Mark.
>
> --
> Nicolás Sáenz
>
next prev parent reply other threads:[~2022-01-11 12:23 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-17 11:51 Possible nohz-full/RCU issue in arm64 KVM Nicolas Saenz Julienne
2021-12-17 13:21 ` Mark Rutland
2021-12-17 14:15 ` Nicolas Saenz Julienne
2021-12-17 14:38 ` Mark Rutland
2021-12-17 15:54 ` Paolo Bonzini
2021-12-17 16:07 ` Paul E. McKenney
2021-12-17 16:20 ` Nicolas Saenz Julienne
2021-12-17 16:43 ` Paul E. McKenney
2021-12-17 16:34 ` Paolo Bonzini
2021-12-17 16:45 ` Paul E. McKenney
2021-12-17 17:02 ` Paolo Bonzini
2021-12-17 17:12 ` Paul E. McKenney
2021-12-17 17:23 ` Paolo Bonzini
2021-12-17 17:47 ` Paul E. McKenney
2022-01-04 16:39 ` Mark Rutland
2022-01-04 17:07 ` Paolo Bonzini
2022-01-11 11:32 ` Nicolas Saenz Julienne
2022-01-11 12:23 ` Mark Rutland [this message]
2021-12-17 14:51 ` Paolo Bonzini
2021-12-20 14:28 ` Marc Zyngier
2021-12-20 16:10 ` Frederic Weisbecker
2022-01-04 13:24 ` Mark Rutland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yd12rvDxyIWzEXWc@FVFF77S0Q05N \
--to=mark.rutland@arm.com \
--cc=Anup.Patel@wdc.com \
--cc=frederic@kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=nsaenzju@redhat.com \
--cc=paulmck@kernel.org \
--cc=pbonzini@redhat.com \
--cc=rcu@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).