All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	Paul McKenney <paulmck@linux.vnet.ibm.com>,
	Sebastian Siewior <bigeasy@linutronix.de>
Subject: [patch 8/8] locking/rt: Take RCU nesting into account for __might_resched()
Date: Thu, 23 Sep 2021 18:54:46 +0200 (CEST)	[thread overview]
Message-ID: <20210923165358.368305497@linutronix.de> (raw)
In-Reply-To: <20210923164145.466686140@linutronix.de>

The general rule that rcu_read_lock() held sections cannot voluntary sleep
does apply even on RT kernels. Though the substitution of spin/rw locks on
RT enabled kernels has to be exempt from that rule. On !RT a spin_lock()
can obviously nest inside a RCU read side critical section as the lock
acquisition is not going to block, but on RT this is not longer the case
due to the 'sleeping' spinlock substitution.

The RT patches contained a cheap hack to ignore the RCU nesting depth in
might_sleep() checks, which was a pragmatic but incorrect workaround.

Instead of generally ignoring the RCU nesting depth in __might_sleep() and
__might_resched() checks, pass the rcu_preempt_depth() via the offsets
argument to __might_resched() from spin/read/write_lock() which makes the
checks work correctly even in RCU read side critical sections.

The actual blocking on such a substituted lock within a RCU read side
critical section is already handled correctly in __schedule() by treating
it as a "preemption" of the RCU read side critical section.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/locking/spinlock_rt.c |   17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

--- a/kernel/locking/spinlock_rt.c
+++ b/kernel/locking/spinlock_rt.c
@@ -24,6 +24,17 @@
 #define RT_MUTEX_BUILD_SPINLOCKS
 #include "rtmutex.c"
 
+/*
+ * __might_resched() skips the state check as rtlocks are state
+ * preserving. Take RCU nesting into account as spin/read/write_lock() can
+ * legitimately nest into an RCU read side critical section.
+ */
+#define RTLOCK_RESCHED_OFFSETS						\
+	(rcu_preempt_depth() << MIGHT_RESCHED_RCU_SHIFT)
+
+#define rtlock_might_resched()						\
+	__might_resched(__FILE__, __LINE__, RTLOCK_RESCHED_OFFSETS)
+
 static __always_inline void rtlock_lock(struct rt_mutex_base *rtm)
 {
 	if (unlikely(!rt_mutex_cmpxchg_acquire(rtm, NULL, current)))
@@ -32,7 +43,7 @@ static __always_inline void rtlock_lock(
 
 static __always_inline void __rt_spin_lock(spinlock_t *lock)
 {
-	__might_resched(__FILE__, __LINE__, 0);
+	rtlock_might_resched();
 	rtlock_lock(&lock->lock);
 	rcu_read_lock();
 	migrate_disable();
@@ -210,7 +221,7 @@ EXPORT_SYMBOL(rt_write_trylock);
 
 void __sched rt_read_lock(rwlock_t *rwlock)
 {
-	__might_resched(__FILE__, __LINE__, 0);
+	rtlock_might_resched();
 	rwlock_acquire_read(&rwlock->dep_map, 0, 0, _RET_IP_);
 	rwbase_read_lock(&rwlock->rwbase, TASK_RTLOCK_WAIT);
 	rcu_read_lock();
@@ -220,7 +231,7 @@ EXPORT_SYMBOL(rt_read_lock);
 
 void __sched rt_write_lock(rwlock_t *rwlock)
 {
-	__might_resched(__FILE__, __LINE__, 0);
+	rtlock_might_resched();
 	rwlock_acquire(&rwlock->dep_map, 0, 0, _RET_IP_);
 	rwbase_write_lock(&rwlock->rwbase, TASK_RTLOCK_WAIT);
 	rcu_read_lock();


  parent reply	other threads:[~2021-09-23 16:55 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-23 16:54 [patch 0/8] sched: Clean up might_sleep() and make it RT aware Thomas Gleixner
2021-09-23 16:54 ` [patch 1/8] sched: Clean up the might_sleep() underscore zoo Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 2/8] sched: Make cond_resched_*lock() variants consistent vs. might_sleep() Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 3/8] sched: Remove preempt_offset argument from __might_sleep() Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 4/8] sched: Cleanup might_sleep() printks Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 5/8] sched: Make might_sleep() output less confusing Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 6/8] sched: Make RCU nest depth distinct in __might_resched() Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` [patch 7/8] sched: Make cond_resched_lock() variants RT aware Thomas Gleixner
2021-10-01 15:05   ` [tip: locking/core] " tip-bot2 for Thomas Gleixner
2021-09-23 16:54 ` Thomas Gleixner [this message]
2021-10-01 15:05   ` [tip: locking/core] locking/rt: Take RCU nesting into account for __might_resched() tip-bot2 for Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210923165358.368305497@linutronix.de \
    --to=tglx@linutronix.de \
    --cc=bigeasy@linutronix.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --subject='Re: [patch 8/8] locking/rt: Take RCU nesting into account for __might_resched()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.