From: Neeraj Upadhyay <neeraju@codeaurora.org>
To: paulmck@linux.vnet.ibm.com, josh@joshtriplett.org,
joel@joelfernandes.org, rostedt@goodmis.org,
mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com
Cc: linux-kernel@vger.kernel.org, rcu@vger.kernel.org,
pkondeti@codeaurora.org, prsood@codeaurora.org,
gkohli@codeaurora.org, Neeraj Upadhyay <neeraju@codeaurora.org>
Subject: [PATCH] rcu: Fix missed wakeup of exp_wq waiters
Date: Fri, 15 Nov 2019 22:58:14 +0530 [thread overview]
Message-ID: <1573838894-23027-1-git-send-email-neeraju@codeaurora.org> (raw)
For the tasks waiting in exp_wq inside exp_funnel_lock(),
there is a chance that they might be indefinitely blocked
in below scenario:
1. There is a task waiting on exp sequence 0b'100' inside
exp_funnel_lock().
_synchronize_rcu_expedited()
s = 0b'100
exp_funnel_lock()
wait_event(rnp->exp_wq[rcu_seq_ctr(s) & 0x3]
2. The Exp GP completes and task (task1) holding exp_mutex queues
worker and schedules out.
_synchronize_rcu_expedited()
s = 0b'100
queue_work(rcu_gp_wq, &rew.rew_work)
wake_up_worker()
schedule()
3. kworker A picks up the queued work and completes the exp gp
sequence.
rcu_exp_wait_wake()
rcu_exp_wait_wake()
rcu_exp_gp_seq_end(rsp) // rsp->expedited_sequence is incremented
// to 0b'100'
4. task1 does not enter wait queue, as sync_exp_work_done() returns true,
and releases exp_mutex.
wait_event(rnp->exp_wq[rcu_seq_ctr(s) & 0x3],
sync_exp_work_done(rsp, s));
mutex_unlock(&rsp->exp_mutex);
5. Next exp GP completes, and sequence number is incremented:
rcu_exp_wait_wake()
rcu_exp_wait_wake()
rcu_exp_gp_seq_end(rsp) // rsp->expedited_sequence = 0b'200'
6. As kworker A uses current expedited_sequence, it wakes up workers
from wrong wait queue index - it should have worken wait queue
corresponding to 0b'100' sequence, but wakes up the ones for
0b'200' sequence. This results in task at step 1 indefinitely blocked.
rcu_exp_wait_wake()
wake_up_all(&rnp->exp_wq[rcu_seq_ctr(rsp->expedited_sequence) & 0x3]);
Fix this by using the correct sequence number for wake_up_all() inside
rcu_exp_wait_wake().
Signed-off-by: Neeraj Upadhyay <neeraju@codeaurora.org>
---
kernel/rcu/tree_exp.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
index e4b77d3..28979d3 100644
--- a/kernel/rcu/tree_exp.h
+++ b/kernel/rcu/tree_exp.h
@@ -557,7 +557,7 @@ static void rcu_exp_wait_wake(unsigned long s)
spin_unlock(&rnp->exp_lock);
}
smp_mb(); /* All above changes before wakeup. */
- wake_up_all(&rnp->exp_wq[rcu_seq_ctr(rcu_state.expedited_sequence) & 0x3]);
+ wake_up_all(&rnp->exp_wq[rcu_seq_ctr(s) & 0x3]);
}
trace_rcu_exp_grace_period(rcu_state.name, s, TPS("endwake"));
mutex_unlock(&rcu_state.exp_wake_mutex);
--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation
next reply other threads:[~2019-11-15 17:28 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-15 17:28 Neeraj Upadhyay [this message]
2019-11-17 21:36 ` [PATCH] rcu: Fix missed wakeup of exp_wq waiters Paul E. McKenney
2019-11-18 9:28 ` Neeraj Upadhyay
[not found] ` <0101016e7dd7b824-50600058-ab5e-44d8-8d24-94cf095f1783-000000@us-west-2.amazonses.com>
2019-11-18 15:08 ` Paul E. McKenney
2019-11-18 16:41 ` Neeraj Upadhyay
[not found] ` <0101016e7f644106-90b40f19-ba9e-4974-bdbe-1062b52222a2-000000@us-west-2.amazonses.com>
2019-11-18 17:24 ` Paul E. McKenney
2019-11-19 3:35 ` Neeraj Upadhyay
[not found] ` <0101016e81ba8865-028ac62b-3fcd-481b-b657-be8519c4edf5-000000@us-west-2.amazonses.com>
2019-11-19 4:05 ` Paul E. McKenney
2019-11-19 7:03 ` Neeraj Upadhyay
2019-11-19 15:09 ` Paul E. McKenney
2019-11-19 20:06 ` Paul E. McKenney
2019-11-20 5:09 ` Neeraj Upadhyay
[not found] ` <eeb7de82-c24b-c914-05fb-683088c9f8e2@codeaurora.org>
2019-11-21 4:10 ` Paul E. McKenney
2019-11-21 4:36 ` Neeraj Upadhyay
[not found] ` <71de1f25-75c5-a4bb-49d9-19af5f8ae4fe@codeaurora.org>
2019-11-21 18:55 ` Paul E. McKenney
2019-11-22 3:44 ` Neeraj Upadhyay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1573838894-23027-1-git-send-email-neeraju@codeaurora.org \
--to=neeraju@codeaurora.org \
--cc=gkohli@codeaurora.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=pkondeti@codeaurora.org \
--cc=prsood@codeaurora.org \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).