From: Peter Zijlstra <peterz@infradead.org>
To: tglx@linutronix.de
Cc: mingo@kernel.org, juri.lelli@arm.com, rostedt@goodmis.org,
xlpang@redhat.com, bigeasy@linutronix.de,
linux-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com,
jdesfossez@efficios.com, bristot@redhat.com,
dvhart@infradead.org, peterz@infradead.org
Subject: [PATCH -v5 09/14] futex: Rework inconsistent rt_mutex/futex_q state
Date: Sat, 04 Mar 2017 10:27:26 +0100 [thread overview]
Message-ID: <20170304093559.349677973@infradead.org> (raw)
In-Reply-To: 20170304092717.762954142@infradead.org
[-- Attachment #1: peter_zijlstra-futex_unlock_pi_wobbles.patch --]
[-- Type: text/plain, Size: 3582 bytes --]
There is a weird state in the futex_unlock_pi() path when it
interleaves with a concurrent futex_lock_pi() at the point where it
drops hb->lock.
In this case, it can happen that the rt_mutex wait_list and the
futex_q disagree on pending waiters, in particular rt_mutex will find
no pending waiters where futex_q thinks there are.
In this case the rt_mutex unlock code cannot assign an owner.
What the current code does in this case is use the futex_q waiter that
got us here; however when the rt_mutex_timed_futex_lock() has already
failed; this leaves things in a weird state, resulting in much
head-aches in fixup_owner().
Simplify all this by changing wake_futex_pi() to return -EAGAIN when
this situation occurs. This then gives the futex_lock_pi() code the
opportunity to continue and the retried futex_unlock_pi() will now
observe a coherent state.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/futex.c | 49 +++++++++++++------------------------------------
1 file changed, 13 insertions(+), 36 deletions(-)
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -1402,12 +1402,18 @@ static int wake_futex_pi(u32 __user *uad
new_owner = rt_mutex_next_owner(&pi_state->pi_mutex);
/*
- * It is possible that the next waiter (the one that brought
- * top_waiter owner to the kernel) timed out and is no longer
- * waiting on the lock.
+ * When we interleave with futex_lock_pi() where it does
+ * rt_mutex_timed_futex_lock(), we might observe @this futex_q waiter,
+ * but the rt_mutex's wait_list can be empty (either still, or again,
+ * depending on which side we land).
+ *
+ * When this happens, give up our locks and try again, giving the
+ * futex_lock_pi() instance time to complete and unqueue_me().
*/
- if (!new_owner)
- new_owner = top_waiter->task;
+ if (!new_owner) {
+ raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock);
+ return -EAGAIN;
+ }
/*
* We pass it to the next owner. The WAITERS bit is always
@@ -2324,7 +2330,6 @@ static long futex_wait_restart(struct re
*/
static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked)
{
- struct task_struct *owner;
int ret = 0;
if (locked) {
@@ -2338,43 +2343,15 @@ static int fixup_owner(u32 __user *uaddr
}
/*
- * Catch the rare case, where the lock was released when we were on the
- * way back before we locked the hash bucket.
- */
- if (q->pi_state->owner == current) {
- /*
- * Try to get the rt_mutex now. This might fail as some other
- * task acquired the rt_mutex after we removed ourself from the
- * rt_mutex waiters list.
- */
- if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) {
- locked = 1;
- goto out;
- }
-
- /*
- * pi_state is incorrect, some other task did a lock steal and
- * we returned due to timeout or signal without taking the
- * rt_mutex. Too late.
- */
- raw_spin_lock_irq(&q->pi_state->pi_mutex.wait_lock);
- owner = rt_mutex_owner(&q->pi_state->pi_mutex);
- if (!owner)
- owner = rt_mutex_next_owner(&q->pi_state->pi_mutex);
- raw_spin_unlock_irq(&q->pi_state->pi_mutex.wait_lock);
- ret = fixup_pi_state_owner(uaddr, q, owner);
- goto out;
- }
-
- /*
* Paranoia check. If we did not take the lock, then we should not be
* the owner of the rt_mutex.
*/
- if (rt_mutex_owner(&q->pi_state->pi_mutex) == current)
+ if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) {
printk(KERN_ERR "fixup_owner: ret = %d pi-mutex: %p "
"pi-state %p\n", ret,
q->pi_state->pi_mutex.owner,
q->pi_state->owner);
+ }
out:
return ret ? ret : locked;
next prev parent reply other threads:[~2017-03-04 10:07 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-04 9:27 [PATCH -v5 00/14] the saga of FUTEX_UNLOCK_PI wobbles continues Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 01/14] futex: Fix potential use-after-free in FUTEX_REQUEUE_PI Peter Zijlstra
2017-03-14 20:48 ` [tip:locking/urgent] " tip-bot for Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 02/14] futex: Add missing error handling to FUTEX_REQUEUE_PI Peter Zijlstra
2017-03-14 20:49 ` [tip:locking/urgent] " tip-bot for Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 03/14] futex: Cleanup variable names for futex_top_waiter() Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 04/14] futex: Use smp_store_release() in mark_wake_futex() Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 05/14] futex: Remove rt_mutex_deadlock_account_*() Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 06/14] futex,rt_mutex: Provide futex specific rt_mutex API Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 07/14] futex: Change locking rules Peter Zijlstra
2017-03-07 13:22 ` Thomas Gleixner
2017-03-07 16:47 ` Sebastian Andrzej Siewior
2017-03-07 18:01 ` Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 08/14] futex: Cleanup refcounting Peter Zijlstra
2017-03-04 9:27 ` Peter Zijlstra [this message]
2017-03-07 13:26 ` [PATCH -v5 09/14] futex: Rework inconsistent rt_mutex/futex_q state Thomas Gleixner
2017-03-04 9:27 ` [PATCH -v5 10/14] futex: Pull rt_mutex_futex_unlock() out from under hb->lock Peter Zijlstra
2017-03-07 14:08 ` Thomas Gleixner
2017-03-07 18:01 ` Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 11/14] futex,rt_mutex: Introduce rt_mutex_init_waiter() Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 12/14] futex,rt_mutex: Restructure rt_mutex_finish_proxy_lock() Peter Zijlstra
2017-03-07 14:18 ` Thomas Gleixner
2017-03-07 17:57 ` Peter Zijlstra
2017-03-07 17:59 ` Thomas Gleixner
2017-03-08 15:29 ` [PATCH] futex: move debug_rt_mutex_free_waiter() further down Sebastian Andrzej Siewior
2017-03-08 15:37 ` Sebastian Andrzej Siewior
2017-03-08 16:21 ` Steven Rostedt
2017-03-08 16:20 ` Steven Rostedt
2017-03-13 9:16 ` Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 13/14] futex: Rework futex_lock_pi() to use rt_mutex_*_proxy_lock() Peter Zijlstra
2017-03-04 9:27 ` [PATCH -v5 14/14] futex: futex_unlock_pi() determinism Peter Zijlstra
2017-03-07 14:31 ` Thomas Gleixner
2017-03-07 17:59 ` Peter Zijlstra
2017-03-13 9:25 ` Peter Zijlstra
2017-03-13 14:25 ` Thomas Gleixner
2017-03-13 15:11 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170304093559.349677973@infradead.org \
--to=peterz@infradead.org \
--cc=bigeasy@linutronix.de \
--cc=bristot@redhat.com \
--cc=dvhart@infradead.org \
--cc=jdesfossez@efficios.com \
--cc=juri.lelli@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=xlpang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).