Re: [PATCH 08/11] rcu/nocb: Limit number of softirq callbacks only on softirq

From: "Paul E. McKenney" <paulmck@kernel.org>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Peter Zijlstra <peterz@infradead.org>,
	Uladzislau Rezki <urezki@gmail.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Boqun Feng <boqun.feng@gmail.com>,
	Neeraj Upadhyay <neeraju@codeaurora.org>,
	Josh Triplett <josh@joshtriplett.org>,
	Joel Fernandes <joel@joelfernandes.org>,
	rcu@vger.kernel.org
Subject: Re: [PATCH 08/11] rcu/nocb: Limit number of softirq callbacks only on softirq
Date: Tue, 5 Oct 2021 13:55:11 -0700	[thread overview]
Message-ID: <20211005205511.GY880162@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <20211004134227.GC273854@lothringen>

On Mon, Oct 04, 2021 at 03:42:27PM +0200, Frederic Weisbecker wrote:
> On Fri, Oct 01, 2021 at 06:51:08PM +0100, Valentin Schneider wrote:
> > On 30/09/21 00:10, Frederic Weisbecker wrote:
> > > The current condition to limit the number of callbacks executed in a
> > > row checks the offloaded state of the rdp. Not only is it volatile
> > > but it is also misleading: the rcu_core() may well be executing
> > > callbacks concurrently with NOCB kthreads, and the offloaded state
> > > would then be verified on both cases. As a result the limit would
> > > spuriously not apply anymore on softirq while in the middle of
> > > (de-)offloading process.
> > >
> > > Another issue with the condition is that rcu_is_callbacks_kthread()
> > > doesn't check if we are actually running callbacks from rcuc itself or
> > > from a softirq interrupting rcuc.
> > >
> > 
> > Doesn't rcutree.use_softirq imply rcuc is never woken, in which case
> > RCU_SOFTIRQ can't interrupt rcuc (e.g. while run atop an IRQ exit)?
> > I suppose during the (de)offload sequence we could have RCU_SOFTIRQ running
> > atop the NOCB CB kthread, but that's not something
> > rcu_is_callbacks_kthread() detects.
> 
> Yes good point, I don't know if rcuc can be ever be interrupted by
> irq_exit() -> do_softirq() -> rcu_core() itself after all.
> 
> Paul can probably confirm your point?

This is supposed to be prohibited by the fact that rcutree.use_softirq
prevents the rcuc kthreads from being spawned in the first place.

That said, I do not recall having thought through the de-offloading
process with the rcuc kthreads in mind.  There could easily be bugs
in this case.  (I -think- that it is OK because each rcuc kthread is
confined to running on the corresponding CPU and because it disables BH.
But it is the system's opinion that counts.)

> > Also, why is rcu_is_callbacks_kthread() hardcoded to false for
> > !CONFIG_RCU_BOOST? Isn't it relevant for do_rcu_batch() ratelimiting
> > regardless (at least before your patches)?
> 
> I believe rcuc is only used on CONFIG_RCU_BOOST?

They are independent, though by default RT arranges for rcuc kthreads and
CONFIG_RCU_BOOST.  Now there are kthreads created for CONFIG_RCU_BOOST,
but these are the rcub kthreads, and they are not per CPU, but rather
per leaf rcu_node structure.

							Thanx, Paul