Re: [PATCH rcu 1/2] docs: Add documentation for rude and trace RCU flavors

["Paul E. McKenney" <paulmck@kernel.org>]

On Wed, Apr 20, 2022 at 12:19:06PM -0400, Steven Rostedt wrote:
> On Wed, 20 Apr 2022 09:13:19 -0700
> "Paul E. McKenney" <paulmck@kernel.org> wrote:
> 
> > > > +The tasks-rude-RCU API is also reader-marking-free and thus quite compact,
> > > > +consisting of call_rcu_tasks_rude(), synchronize_rcu_tasks_rude(),
> > > > +and rcu_barrier_tasks_rude().  
> > > 
> > > Are we going to be able to get rid of the "rude" version once we have all
> > > tracing in a RCU visible section?  
> > 
> > You tell me!  ;-)
> > 
> > If there are no longer any users, I would be happy to get rid of it.
> > As of v5.18-rc1, the only user is ftrace.
> > 
> > > > +
> > > > +Tasks Trace RCU
> > > > +~~~~~~~~~~~~~~~
> > > > +
> > > > +Some forms of tracing need to sleep in readers, but cannot tolerate
> > > > +SRCU's read-side overhead, which includes a full memory barrier in both
> > > > +srcu_read_lock() and srcu_read_unlock().  This need is handled by a
> > > > +Tasks Trace RCU that uses scheduler locking and IPIs to synchronize with
> > > > +readers.  Real-time systems that cannot tolerate IPIs may build their
> > > > +kernels with ``CONFIG_TASKS_TRACE_RCU_READ_MB=y``, which avoids the IPIs at
> > > > +the expense of adding full memory barriers to the read-side primitives.  
> > > 
> > > If NOHZ_FULL is enabled, is there a way to also be able to have this full
> > > mb on RT removed as well?

Ah, I did miss this question, apologies.  The tradeoff is IPIs during
each Tasks Trace RCU grace period on the one hand or the read-side
memory barrier on the other.

CONFIG_TASKS_TRACE_RCU_READ_MB=y gets you the read-side memory barriers.

CONFIG_TASKS_TRACE_RCU_READ_MB=n gets you the IPIs.

Choose wisely!  ;-)

More seriously, I could easily imagine an RT system being set up so that
Tasks Trace RCU grace periods never happen while the real-time application
is running.  This requires the system administrator being careful what
tracing facilities are used that those application is running, but
it seems doable to me.

Such an RT system could build with CONFIG_TASKS_TRACE_RCU_READ_MB=n to
avoid the read-side memory barriers, but also avoiding the IPIs while
the application was running.

Even more seriously, if the real-time application runs in nohz_full mode,
Tasks Trace RCU will avoid IPIing it.  In that case, the kernel can be
built with CONFIG_TASKS_TRACE_RCU_READ_MB=n and avoid both the read-side
memory barriers and the IPIs.

And the final bit of seriousness for this email, if your real-time
application didn't have a time-critical CPU-bound component, it might
be possible to avoid both read-side memory barriers and IPIs by
adjusting the rcu_tasks_trace_qs() code in the context-switch hook.

> Hmm, if we no longer need the rude version due to noinstr, if then we need
> to use something that adds full memory barriers at *every function call*
> then I rather keep the rude version.

A full memory barrier at every function call does sound more heavy weight
than would be good.  ;-)

							Thanx, Paul