Re: rcu/tree: Protect rcu_rdp_is_offloaded() invocations on RT

[Frederic Weisbecker <frederic@kernel.org>]

On Wed, Sep 22, 2021 at 01:27:31PM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-09-22 13:10:12 [+0200], Frederic Weisbecker wrote:
> > On Wed, Sep 22, 2021 at 08:32:08AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2021-09-22 01:45:18 [+0200], Frederic Weisbecker wrote:
> > > > 
> > > > Also while at it, I'm asking again: traditionally softirqs could assume that
> > > > manipulating a local state was safe against !irq_count() code fiddling with
> > > > the same state on the same CPU.
> > > > 
> > > > Now with preemptible softirqs, that assumption can be broken anytime. RCU was
> > > > fortunate enough to have a warning for that. But who knows how many issues like
> > > > this are lurking?
> > > 
> > > If "local state" is modified then it is safe as long as it is modified
> > > within a local_bh_disable() section. And we are in this section while
> > > invoking a forced-threaded interrupt. The special part about RCU is
> > > that it is used in_irq() as part of core-code.
> > 
> > But local_bh_disable() was deemed for protecting from interrupting softirqs,
> > not the other way around (softirqs being preempted by other tasks). The latter
> > semantic is new and nobody had that in mind until softirqs have been made
> > preemptible.
> > 
> > For example:
> > 
> >                              CPU 0
> >           -----------------------------------------------
> >           SOFTIRQ                            RANDOM TASK
> >           ------                             -----------
> >           int *X = &per_cpu(CPUX, 0)         int *X = &per_cpu(CPUX, 0)
> >           int A, B;                          WRITE_ONCE(*X, 0);
> >                                              WRITE_ONCE(*X, 1);
> >           A = READ_ONCE(*X);
> >           B = READ_ONCE(*X);
> > 
> > 
> > We used to have the guarantee that A == B. That's not true anymore. Now
> > some new explicit local_bh_disable() should be carefully placed on RANDOM_TASK
> > where it wasn't necessary before. RCU is not that special in this regard.
> 
> The part with rcutree.use_softirq=0 on RT does not make it any better,
> right?

The rcuc kthread disables softirqs before calling rcu_core(), so it behaves
pretty much the same as a softirq. Or am I missing something?

> So you rely on some implicit behaviour which breaks with RT such as:
> 
>                               CPU 0
>            -----------------------------------------------
>            RANDOM TASK-A                      RANDOM TASK-B
>            ------                             -----------
>            int *X = &per_cpu(CPUX, 0)         int *X = &per_cpu(CPUX, 0)
>            int A, B;                          
> 					      spin_lock(&D);
>            spin_lock(&C);
> 	   				      WRITE_ONCE(*X, 0);
>            A = READ_ONCE(*X);
>                                               WRITE_ONCE(*X, 1);
>            B = READ_ONCE(*X);
> 
> while spinlock C and D are just random locks not related to CPUX but it
> just happens that they are held at that time. So for !RT you guarantee
> that A == B while it is not the case on RT.

Not sure which spinlocks you are referring to here. Also most RCU spinlocks
are raw.

> 
> Sebastian