From: Peter Zijlstra <peterz@infradead.org>
To: tglx@linutronix.de, boqun.feng@gmail.com
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org,
Ingo Molnar <mingo@kernel.org>,
Juri Lelli <juri.lelli@redhat.com>,
Steven Rostedt <rostedt@goodmis.org>,
Davidlohr Bueso <dave@stgolabs.net>,
Will Deacon <will@kernel.org>, Waiman Long <longman@redhat.com>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Mike Galbraith <efault@gmx.de>,
Daniel Bristot de Oliveira <bristot@redhat.com>
Subject: [PATCH 1/4] sched/wakeup: Strengthen current_save_and_set_rtlock_wait_state()
Date: Thu, 09 Sep 2021 12:59:16 +0200 [thread overview]
Message-ID: <20210909110203.767330253@infradead.org> (raw)
In-Reply-To: 20210909105915.757320973@infradead.org
While looking at current_save_and_set_rtlock_wait_state() I'm thinking
it really ought to use smp_store_mb(), because something like:
current_save_and_set_rtlock_wait_state();
for (;;) {
if (try_lock())
break;
raw_spin_unlock_irq(&lock->wait_lock);
schedule();
raw_spin_lock_irq(&lock->wait_lock);
set_current_state(TASK_RTLOCK_WAIT);
}
current_restore_rtlock_saved_state();
which is the advertised usage in the comment, is actually broken,
since trylock() will only need a load-acquire in general and that
could be re-ordered against the state store, which could lead to a
missed wakeup -> BAD (tm).
While there, make them consistent with the IRQ usage in
set_special_state().
Fixes: 5f220be21418 ("sched/wakeup: Prepare for RT sleeping spin/rwlocks")
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
include/linux/sched.h | 19 +++++++++++--------
1 file changed, 11 insertions(+), 8 deletions(-)
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -245,7 +245,8 @@ struct task_group;
* if (try_lock())
* break;
* raw_spin_unlock_irq(&lock->wait_lock);
- * schedule_rtlock();
+ * if (!cond)
+ * schedule_rtlock();
* raw_spin_lock_irq(&lock->wait_lock);
* set_current_state(TASK_RTLOCK_WAIT);
* }
@@ -253,22 +254,24 @@ struct task_group;
*/
#define current_save_and_set_rtlock_wait_state() \
do { \
- lockdep_assert_irqs_disabled(); \
- raw_spin_lock(¤t->pi_lock); \
+ unsigned long flags; /* may shadow */ \
+ \
+ raw_spin_lock_irqsave(¤t->pi_lock, flags); \
current->saved_state = current->__state; \
debug_rtlock_wait_set_state(); \
- WRITE_ONCE(current->__state, TASK_RTLOCK_WAIT); \
- raw_spin_unlock(¤t->pi_lock); \
+ smp_store_mb(current->__state, TASK_RTLOCK_WAIT); \
+ raw_spin_unlock_irqrestore(¤t->pi_lock, flags); \
} while (0);
#define current_restore_rtlock_saved_state() \
do { \
- lockdep_assert_irqs_disabled(); \
- raw_spin_lock(¤t->pi_lock); \
+ unsigned long flags; /* may shadow */ \
+ \
+ raw_spin_lock_irqsave(¤t->pi_lock, flags); \
debug_rtlock_wait_restore_state(); \
WRITE_ONCE(current->__state, current->saved_state); \
current->saved_state = TASK_RUNNING; \
- raw_spin_unlock(¤t->pi_lock); \
+ raw_spin_unlock_irqrestore(¤t->pi_lock, flags); \
} while (0);
#define get_current_state() READ_ONCE(current->__state)
next prev parent reply other threads:[~2021-09-09 11:04 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-09 10:59 [PATCH 0/4] locking/rwbase: Assorted fixes Peter Zijlstra
2021-09-09 10:59 ` Peter Zijlstra [this message]
2021-09-09 13:45 ` [PATCH 1/4] sched/wakeup: Strengthen current_save_and_set_rtlock_wait_state() Will Deacon
2021-09-09 14:27 ` Peter Zijlstra
2021-09-10 12:57 ` Will Deacon
2021-09-10 13:17 ` Peter Zijlstra
2021-09-10 14:01 ` Peter Zijlstra
2021-09-10 15:06 ` Will Deacon
2021-09-10 16:07 ` Waiman Long
2021-09-10 17:09 ` Peter Zijlstra
2021-09-12 3:57 ` Boqun Feng
2021-09-10 12:45 ` Sebastian Andrzej Siewior
2021-09-13 22:08 ` Thomas Gleixner
2021-09-13 22:52 ` Thomas Gleixner
2021-09-14 6:45 ` Peter Zijlstra
2021-09-09 10:59 ` [PATCH 2/4] locking/rwbase: Properly match set_and_save_state() to restore_state() Peter Zijlstra
2021-09-09 13:53 ` Will Deacon
2021-09-14 7:31 ` Thomas Gleixner
2021-09-16 11:59 ` [tip: locking/urgent] " tip-bot2 for Peter Zijlstra
2021-09-09 10:59 ` [PATCH 3/4] locking/rwbase: Fix rwbase_write_lock() vs __rwbase_read_lock() Peter Zijlstra
2021-09-14 7:45 ` Thomas Gleixner
2021-09-14 13:59 ` Peter Zijlstra
2021-09-14 15:00 ` Thomas Gleixner
2021-09-16 11:59 ` [tip: locking/urgent] locking/rwbase: Extract __rwbase_write_trylock() tip-bot2 for Peter Zijlstra
2021-09-09 10:59 ` [PATCH 4/4] locking/rwbase: Take care of ordering guarantee for fastpath reader Peter Zijlstra
2021-09-14 7:46 ` Thomas Gleixner
2021-09-16 11:59 ` [tip: locking/urgent] " tip-bot2 for Boqun Feng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210909110203.767330253@infradead.org \
--to=peterz@infradead.org \
--cc=bigeasy@linutronix.de \
--cc=boqun.feng@gmail.com \
--cc=bristot@redhat.com \
--cc=dave@stgolabs.net \
--cc=efault@gmx.de \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mingo@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).