From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org,
rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com,
bobby.prani@gmail.com,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 14/18] rcu: Consolidate expedited CPU selection
Date: Tue, 6 Oct 2015 09:29:33 -0700 [thread overview]
Message-ID: <1444148977-14108-14-git-send-email-paulmck@linux.vnet.ibm.com> (raw)
In-Reply-To: <1444148977-14108-1-git-send-email-paulmck@linux.vnet.ibm.com>
Now that sync_sched_exp_select_cpus() and sync_rcu_exp_select_cpus()
are identical aside from the the argument to smp_call_function_single(),
this commit consolidates them with a functional argument.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/tree.c | 7 +++---
kernel/rcu/tree_plugin.h | 61 +-----------------------------------------------
2 files changed, 5 insertions(+), 63 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 500b1347554d..ff68087d93a3 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3654,7 +3654,8 @@ static void synchronize_sched_expedited_cpu_stop(void *data)
* Select the nodes that the upcoming expedited grace period needs
* to wait for.
*/
-static void sync_sched_exp_select_cpus(struct rcu_state *rsp)
+static void sync_rcu_exp_select_cpus(struct rcu_state *rsp,
+ smp_call_func_t func)
{
int cpu;
unsigned long flags;
@@ -3696,7 +3697,7 @@ static void sync_sched_exp_select_cpus(struct rcu_state *rsp)
for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask <<= 1) {
if (!(mask_ofl_ipi & mask))
continue;
- ret = smp_call_function_single(cpu, synchronize_sched_expedited_cpu_stop, NULL, 0);
+ ret = smp_call_function_single(cpu, func, rsp, 0);
if (!ret)
mask_ofl_ipi &= ~mask;
}
@@ -3788,7 +3789,7 @@ void synchronize_sched_expedited(void)
return; /* Someone else did our work for us. */
rcu_exp_gp_seq_start(rsp);
- sync_sched_exp_select_cpus(rsp);
+ sync_rcu_exp_select_cpus(rsp, synchronize_sched_expedited_cpu_stop);
synchronize_sched_expedited_wait(rsp);
rcu_exp_gp_seq_end(rsp);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 7880202f1e38..6cbfbfc58656 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -708,65 +708,6 @@ static void sync_rcu_exp_handler(void *info)
rcu_report_exp_rdp(rsp, rdp, true);
}
-/*
- * Select the nodes that the upcoming expedited grace period needs
- * to wait for.
- */
-static void sync_rcu_exp_select_cpus(struct rcu_state *rsp)
-{
- int cpu;
- unsigned long flags;
- unsigned long mask;
- unsigned long mask_ofl_test;
- unsigned long mask_ofl_ipi;
- int ret;
- struct rcu_node *rnp;
-
- sync_exp_reset_tree(rsp);
- rcu_for_each_leaf_node(rsp, rnp) {
- raw_spin_lock_irqsave(&rnp->lock, flags);
- smp_mb__after_unlock_lock();
-
- /* Each pass checks a CPU for identity, offline, and idle. */
- mask_ofl_test = 0;
- for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++) {
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
- struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
-
- if (raw_smp_processor_id() == cpu ||
- cpu_is_offline(cpu) ||
- !(atomic_add_return(0, &rdtp->dynticks) & 0x1))
- mask_ofl_test |= rdp->grpmask;
- }
- mask_ofl_ipi = rnp->expmask & ~mask_ofl_test;
-
- /*
- * Need to wait for any blocked tasks as well. Note that
- * additional blocking tasks will also block the expedited
- * GP until such time as the ->expmask bits are cleared.
- */
- if (rcu_preempt_has_tasks(rnp))
- rnp->exp_tasks = rnp->blkd_tasks.next;
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
-
- /* IPI the remaining CPUs for expedited quiescent state. */
- mask = 1;
- for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask <<= 1) {
- if (!(mask_ofl_ipi & mask))
- continue;
- ret = smp_call_function_single(cpu,
- sync_rcu_exp_handler,
- rsp, 0);
- if (!ret)
- mask_ofl_ipi &= ~mask;
- }
- /* Report quiescent states for those that went offline. */
- mask_ofl_test |= mask_ofl_ipi;
- if (mask_ofl_test)
- rcu_report_exp_cpu_mult(rsp, rnp, mask_ofl_test, false);
- }
-}
-
/**
* synchronize_rcu_expedited - Brute-force RCU grace period
*
@@ -795,7 +736,7 @@ void synchronize_rcu_expedited(void)
rcu_exp_gp_seq_start(rsp);
/* Initialize the rcu_node tree in preparation for the wait. */
- sync_rcu_exp_select_cpus(rsp);
+ sync_rcu_exp_select_cpus(rsp, sync_rcu_exp_handler);
/* Wait for snapshotted ->blkd_tasks lists to drain. */
rnp = rcu_get_root(rsp);
--
2.5.2
next prev parent reply other threads:[~2015-10-06 16:32 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-06 16:29 [PATCH tip/core/rcu 0/18] Expedited grace-period improvements for 4.4 Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 01/18] rcu: Use rsp->expedited_wq instead of sync_rcu_preempt_exp_wq Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation Paul E. McKenney
2015-10-06 20:29 ` Peter Zijlstra
2015-10-06 20:58 ` Paul E. McKenney
2015-10-07 7:51 ` Peter Zijlstra
2015-10-07 8:42 ` Mathieu Desnoyers
2015-10-07 11:01 ` Peter Zijlstra
2015-10-07 11:50 ` Peter Zijlstra
2015-10-07 12:03 ` Peter Zijlstra
2015-10-07 12:05 ` kbuild test robot
2015-10-07 12:09 ` kbuild test robot
2015-10-07 12:11 ` kbuild test robot
2015-10-07 12:17 ` Peter Zijlstra
2015-10-07 13:44 ` [kbuild-all] " Fengguang Wu
2015-10-07 13:55 ` Peter Zijlstra
2015-10-07 14:21 ` Fengguang Wu
2015-10-07 14:28 ` Peter Zijlstra
2015-10-07 15:18 ` Paul E. McKenney
2015-10-08 10:24 ` Peter Zijlstra
2015-10-07 15:15 ` Paul E. McKenney
2015-10-07 14:33 ` Paul E. McKenney
2015-10-07 14:40 ` Peter Zijlstra
2015-10-07 16:48 ` Paul E. McKenney
2015-10-08 9:49 ` Peter Zijlstra
2015-10-08 15:33 ` Paul E. McKenney
2015-10-08 17:12 ` Peter Zijlstra
2015-10-08 17:46 ` Paul E. McKenney
2015-10-09 0:10 ` Paul E. McKenney
2015-10-09 8:44 ` Peter Zijlstra
2015-10-06 16:29 ` [PATCH tip/core/rcu 03/18] rcu: Consolidate tree setup for synchronize_rcu_expedited() Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 04/18] rcu: Use single-stage IPI algorithm for RCU expedited grace period Paul E. McKenney
2015-10-07 13:24 ` Peter Zijlstra
2015-10-07 18:11 ` Paul E. McKenney
2015-10-07 13:35 ` Peter Zijlstra
2015-10-07 15:44 ` Paul E. McKenney
2015-10-07 13:43 ` Peter Zijlstra
2015-10-07 13:49 ` Peter Zijlstra
2015-10-07 16:14 ` Paul E. McKenney
2015-10-08 9:00 ` Peter Zijlstra
2015-10-07 16:13 ` Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 05/18] rcu: Move synchronize_sched_expedited() to combining tree Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 06/18] rcu: Rename qs_pending to core_needs_qs Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 07/18] rcu: Invert passed_quiesce and rename to cpu_no_qs Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 08/18] rcu: Make ->cpu_no_qs be a union for aggregate OR Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 09/18] rcu: Switch synchronize_sched_expedited() to IPI Paul E. McKenney
2015-10-07 14:18 ` Peter Zijlstra
2015-10-07 16:24 ` Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 10/18] rcu: Stop silencing lockdep false positive for expedited grace periods Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 11/18] rcu: Stop excluding CPU hotplug in synchronize_sched_expedited() Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 12/18] cpu: Remove try_get_online_cpus() Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 13/18] rcu: Prepare for consolidating expedited CPU selection Paul E. McKenney
2015-10-06 16:29 ` Paul E. McKenney [this message]
2015-10-06 16:29 ` [PATCH tip/core/rcu 15/18] rcu: Add online/offline info to expedited stall warning message Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 16/18] rcu: Add tasks to expedited stall-warning messages Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 17/18] rcu: Enable stall warnings for synchronize_rcu_expedited() Paul E. McKenney
2015-10-06 16:29 ` [PATCH tip/core/rcu 18/18] rcu: Better hotplug handling for synchronize_sched_expedited() Paul E. McKenney
2015-10-07 14:26 ` Peter Zijlstra
2015-10-07 16:26 ` Paul E. McKenney
2015-10-08 9:01 ` Peter Zijlstra
2015-10-08 15:06 ` Paul E. McKenney
2015-10-08 15:12 ` Peter Zijlstra
2015-10-08 15:19 ` Paul E. McKenney
2015-10-08 18:01 ` Josh Triplett
2015-10-09 0:11 ` Paul E. McKenney
2015-10-09 0:48 ` Josh Triplett
2015-10-09 3:54 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1444148977-14108-14-git-send-email-paulmck@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=bobby.prani@gmail.com \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=dvhart@linux.intel.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=jiangshanlai@gmail.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).