From: Prateek Sood <prsood@codeaurora.org>
To: mingo@kernel.org, longman@redhat.com, peterz@infradead.org,
parri.andrea@gmail.com, dave@stgolabs.net
Cc: Prateek Sood <prsood@codeaurora.org>,
linux-kernel@vger.kernel.org, sramana@codeaurora.org
Subject: [PATCH] rwsem: fix missed wakeup due to reordering of load
Date: Thu, 7 Sep 2017 20:00:58 +0530 [thread overview]
Message-ID: <1504794658-15397-1-git-send-email-prsood@codeaurora.org> (raw)
If a spinner is present, there is a chance that the load of
rwsem_has_spinner() in rwsem_wake() can be reordered with
respect to decrement of rwsem count in __up_write() leading
to wakeup being missed.
spinning writer up_write caller
--------------- -----------------------
[S] osq_unlock() [L] osq
spin_lock(wait_lock)
sem->count=0xFFFFFFFF00000001
+0xFFFFFFFF00000000
count=sem->count
MB
sem->count=0xFFFFFFFE00000001
-0xFFFFFFFF00000001
spin_trylock(wait_lock)
return
rwsem_try_write_lock(count)
spin_unlock(wait_lock)
schedule()
Reordering of atomic_long_sub_return_release() in __up_write()
and rwsem_has_spinner() in rwsem_wake() can cause missing of
wakeup in up_write() context. In spinning writer, sem->count
and local variable count is 0XFFFFFFFE00000001. It would result
in rwsem_try_write_lock() failing to acquire rwsem and spinning
writer going to sleep in rwsem_down_write_failed().
The smp_rmb() will make sure that the spinner state is
consulted after sem->count is updated in up_write context.
Signed-off-by: Prateek Sood <prsood@codeaurora.org>
---
kernel/locking/rwsem-xadd.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c
index 02f6606..1fefe6d 100644
--- a/kernel/locking/rwsem-xadd.c
+++ b/kernel/locking/rwsem-xadd.c
@@ -613,6 +613,33 @@ struct rw_semaphore *rwsem_wake(struct rw_semaphore *sem)
DEFINE_WAKE_Q(wake_q);
/*
+ * __rwsem_down_write_failed_common(sem)
+ * rwsem_optimistic_spin(sem)
+ * osq_unlock(sem->osq)
+ * ...
+ * atomic_long_add_return(&sem->count)
+ *
+ * - VS -
+ *
+ * __up_write()
+ * if (atomic_long_sub_return_release(&sem->count) < 0)
+ * rwsem_wake(sem)
+ * osq_is_locked(&sem->osq)
+ *
+ * And __up_write() must observe !osq_is_locked() when it observes the
+ * atomic_long_add_return() in order to not miss a wakeup.
+ *
+ * This boils down to:
+ *
+ * [S.rel] X = 1 [RmW] r0 = (Y += 0)
+ * MB RMB
+ * [RmW] Y += 1 [L] r1 = X
+ *
+ * exists (r0=1 /\ r1=0)
+ */
+ smp_rmb();
+
+ /*
* If a spinner is present, it is not necessary to do the wakeup.
* Try to do wakeup only if the trylock succeeds to minimize
* spinlock contention which may introduce too much delay in the
--
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.,
is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
next reply other threads:[~2017-09-07 14:31 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-07 14:30 Prateek Sood [this message]
2017-09-19 14:05 ` [PATCH] rwsem: fix missed wakeup due to reordering of load Andrea Parri
2017-09-20 14:52 ` Davidlohr Bueso
2017-09-20 21:17 ` Andrea Parri
2017-09-27 21:20 ` Davidlohr Bueso
2017-09-26 18:37 ` Prateek Sood
2017-09-29 9:30 ` [tip:locking/urgent] locking/rwsem-xadd: Fix " tip-bot for Prateek Sood
-- strict thread matches above, loose matches on Subject: below --
2017-08-23 11:28 [PATCH] rwsem: fix " Prateek Sood
2017-08-24 11:29 ` Peter Zijlstra
2017-08-24 12:33 ` Peter Zijlstra
2017-08-24 12:52 ` Peter Zijlstra
2017-09-07 14:08 ` Prateek Sood
2017-07-26 20:17 Prateek Sood
2017-07-27 15:48 ` Waiman Long
2017-07-27 16:59 ` Peter Zijlstra
2017-08-10 8:32 ` Andrea Parri
2017-08-10 10:41 ` Peter Zijlstra
2017-08-10 10:44 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1504794658-15397-1-git-send-email-prsood@codeaurora.org \
--to=prsood@codeaurora.org \
--cc=dave@stgolabs.net \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mingo@kernel.org \
--cc=parri.andrea@gmail.com \
--cc=peterz@infradead.org \
--cc=sramana@codeaurora.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.