From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org,
rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
fweisbec@gmail.com, oleg@redhat.com, joel.opensrc@gmail.com,
torvalds@linux-foundation.org, npiggin@gmail.com,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 02/21] rcu: Make rcu_start_future_gp()'s grace-period check more precise
Date: Sun, 22 Apr 2018 20:03:25 -0700 [thread overview]
Message-ID: <1524452624-27589-2-git-send-email-paulmck@linux.vnet.ibm.com> (raw)
In-Reply-To: <20180423030258.GA23370@linux.vnet.ibm.com>
The rcu_start_future_gp() function uses a sloppy check for a grace
period being in progress, which works today because there are a number
of code sequences that resolve the resulting races. However, some of
these race-resolution code sequences must acquire the root rcu_node
structure's ->lock, and contention on that lock has started manifesting.
This commit therefore makes rcu_start_future_gp() check more precise,
eliminating the sloppy lockless check of the rcu_state structure's ->gpnum
and ->completed fields. The effect is that rcu_start_future_gp() will
sometimes unnecessarily attempt to start a new grace period, but this
overhead will be reduced later using funnel locking.
Reported-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/tree.c | 18 +++++-------------
1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index f5ca72f2ed43..4bbba17422cd 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1705,20 +1705,12 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
}
/*
- * If either this rcu_node structure or the root rcu_node structure
- * believe that a grace period is in progress, then we must wait
- * for the one following, which is in "c". Because our request
- * will be noticed at the end of the current grace period, we don't
- * need to explicitly start one. We only do the lockless check
- * of rnp_root's fields if the current rcu_node structure thinks
- * there is no grace period in flight, and because we hold rnp->lock,
- * the only possible change is when rnp_root's two fields are
- * equal, in which case rnp_root->gpnum might be concurrently
- * incremented. But that is OK, as it will just result in our
- * doing some extra useless work.
+ * If this rcu_node structure believes that a grace period is in
+ * progress, then we must wait for the one following, which is in
+ * "c". Because our request will be noticed at the end of the
+ * current grace period, we don't need to explicitly start one.
*/
- if (rnp->gpnum != rnp->completed ||
- READ_ONCE(rnp_root->gpnum) != READ_ONCE(rnp_root->completed)) {
+ if (rnp->gpnum != rnp->completed) {
rnp->need_future_gp[c & 0x1]++;
trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleaf"));
goto out;
--
2.5.2
next prev parent reply other threads:[~2018-04-23 3:05 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-23 3:02 [PATCH tip/core/rcu 0/21] Contention reduction for v4.18 Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 01/21] rcu: Improve non-root rcu_cbs_completed() accuracy Paul E. McKenney
2018-04-23 3:03 ` Paul E. McKenney [this message]
2018-04-23 3:03 ` [PATCH tip/core/rcu 03/21] rcu: Add accessor macros for the ->need_future_gp[] array Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 04/21] rcu: Make rcu_gp_kthread() check for early-boot activity Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 05/21] rcu: Make rcu_gp_cleanup() more accurately predict need for new GP Paul E. McKenney
2018-05-10 7:21 ` [tip/core/rcu, " Joel Fernandes
2018-05-10 13:15 ` Paul E. McKenney
2018-05-10 17:22 ` Joel Fernandes
2018-05-11 16:22 ` Paul E. McKenney
2018-05-10 17:37 ` Joel Fernandes
2018-05-11 16:24 ` Paul E. McKenney
2018-05-11 16:27 ` Joel Fernandes
2018-04-23 3:03 ` [PATCH tip/core/rcu 06/21] rcu: Avoid losing ->need_future_gp[] values due to GP start/end races Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 07/21] rcu: Make rcu_future_needs_gp() check all ->need_future_gps[] elements Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 08/21] rcu: Convert ->need_future_gp[] array to boolean Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 09/21] rcu: Make rcu_migrate_callbacks wake GP kthread when needed Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 10/21] rcu: Avoid __call_rcu_core() root rcu_node ->lock acquisition Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 11/21] rcu: Switch __rcu_process_callbacks() to rcu_accelerate_cbs() Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 12/21] rcu: Cleanup, don't put ->completed into an int Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 13/21] rcu: Clear request other than RCU_GP_FLAG_INIT at GP end Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 14/21] rcu: Inline rcu_start_gp_advanced() into rcu_start_future_gp() Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 15/21] rcu: Make rcu_start_future_gp() caller select grace period Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 16/21] rcu: Add funnel locking to rcu_start_this_gp() Paul E. McKenney
2018-05-12 6:03 ` [tip/core/rcu,16/21] " Joel Fernandes
2018-05-12 14:40 ` Paul E. McKenney
2018-05-12 14:44 ` Paul E. McKenney
2018-05-12 23:53 ` Joel Fernandes
2018-05-13 15:38 ` Paul E. McKenney
2018-05-13 16:49 ` Joel Fernandes
2018-05-13 19:09 ` Paul E. McKenney
2018-05-13 19:51 ` Joel Fernandes
2018-05-14 2:22 ` Paul E. McKenney
2018-05-14 5:00 ` Joel Fernandes
2018-05-14 13:23 ` Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 17/21] rcu: Make rcu_start_this_gp() check for out-of-range requests Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 18/21] rcu: The rcu_gp_cleanup() function does not need cpu_needs_another_gp() Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 19/21] rcu: Simplify and inline cpu_needs_another_gp() Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 20/21] rcu: Drop early GP request check from rcu_gp_kthread() Paul E. McKenney
2018-04-23 3:03 ` [PATCH tip/core/rcu 21/21] rcu: Update list of rcu_future_grace_period() trace events Paul E. McKenney
2018-05-14 6:42 ` [PATCH tip/core/rcu 0/21] Contention reduction for v4.18 Nicholas Piggin
2018-05-14 16:09 ` Paul E. McKenney
2018-05-14 22:21 ` Nicholas Piggin
2018-05-14 22:42 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1524452624-27589-2-git-send-email-paulmck@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=jiangshanlai@gmail.com \
--cc=joel.opensrc@gmail.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=npiggin@gmail.com \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).