Re: [PATCH] rcu/nocb: Extend checks for offloaded rdp by migrate_disable

From: Thomas Gleixner <tglx@linutronix.de>
To: paulmck@kernel.org, Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: rcu@vger.kernel.org, Josh Triplett <josh@joshtriplett.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	Lai Jiangshan <jiangshanlai@gmail.com>,
	Joel Fernandes <joel@joelfernandes.org>,
	Frederic Weisbecker <frederic@kernel.org>
Subject: Re: [PATCH] rcu/nocb: Extend checks for offloaded rdp by migrate_disable
Date: Tue, 27 Jul 2021 21:33:42 +0200	[thread overview]
Message-ID: <87wnpbpn55.ffs@nanos.tec.linutronix.de> (raw)
In-Reply-To: <20210727172351.GC4397@paulmck-ThinkPad-P17-Gen-1>

On Tue, Jul 27 2021 at 10:23, Paul E. McKenney wrote:
> On Tue, Jul 27, 2021 at 06:38:15PM +0200, Sebastian Andrzej Siewior wrote:
>> One thing that has been overseen is that a task within a migrate-disable
>> region (as on PREEMPT_RT with disabled BH) is fully preemptible but may
>> not be migrated to another CPU which should be enough to guarantee that
>> rdp remains stable.
>> 
>> Check also disabled migration of the task if the RCU data pointer is
>> from current CPU. Put the whole check within an SMP ifdef block since
>> without SMP there are not CPU migrations to worry about (also
>> task_struct::migration_disabled is missing).
>> 
>> Cc: Frederic Weisbecker <frederic@kernel.org>
>> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>> ---
>> I don't fully understand why the CPU-hotplug lock matters here but this
>> is beside the point ;)
>
> If I remember correctly, any attempt to change the offloaded state
> must hold off CPU-hotplug operations.  So if the current thread is
> holding off CPU-hotplug operations, no other thread can be doing
> an offload or de-offload operation.

It only prevents unplugging of a CPU, but not plugging a CPU.

>>  kernel/rcu/tree_plugin.h | 6 ++++--
>>  1 file changed, 4 insertions(+), 2 deletions(-)
>> 
>> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
>> index 0ff5e4fb933e7..d8a623ba7d243 100644
>> --- a/kernel/rcu/tree_plugin.h
>> +++ b/kernel/rcu/tree_plugin.h
>> @@ -57,16 +57,18 @@ static bool rcu_rdp_is_offloaded(struct rcu_data *rdp)
>>  	 * timers have their own means of synchronization against the
>>  	 * offloaded state updaters.
>>  	 */
>> +#ifdef CONFIG_SMP
>>  	RCU_LOCKDEP_WARN(
>>  		!(lockdep_is_held(&rcu_state.barrier_mutex) ||
>>  		  (IS_ENABLED(CONFIG_HOTPLUG_CPU) && lockdep_is_cpus_held()) ||
>>  		  rcu_lockdep_is_held_nocb(rdp) ||
>>  		  (rdp == this_cpu_ptr(&rcu_data) &&
>> -		   !(IS_ENABLED(CONFIG_PREEMPT_COUNT) && preemptible())) ||
>> +		   (!(IS_ENABLED(CONFIG_PREEMPT_COUNT) && preemptible()) ||
>
>
>> +		    current->migration_disabled)) ||
>
> How does this change interact with the one proposed by Valentin?
>
> https://lore.kernel.org/lkml/20210721115118.729943-3-valentin.schneider@arm.com/

It does not interact, it conflicts and Valentin's is definitely the
better solution.

Thanks,

        tglx