From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753414AbbJFQcI (ORCPT ); Tue, 6 Oct 2015 12:32:08 -0400 Received: from e37.co.us.ibm.com ([32.97.110.158]:35319 "EHLO e37.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752886AbbJFQ3t (ORCPT ); Tue, 6 Oct 2015 12:29:49 -0400 X-IBM-Helo: d03dlp01.boulder.ibm.com X-IBM-MailFrom: paulmck@linux.vnet.ibm.com X-IBM-RcptTo: linux-kernel@vger.kernel.org From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, "Paul E. McKenney" Subject: [PATCH tip/core/rcu 14/18] rcu: Consolidate expedited CPU selection Date: Tue, 6 Oct 2015 09:29:33 -0700 Message-Id: <1444148977-14108-14-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 2.5.2 In-Reply-To: <1444148977-14108-1-git-send-email-paulmck@linux.vnet.ibm.com> References: <20151006162907.GA12020@linux.vnet.ibm.com> <1444148977-14108-1-git-send-email-paulmck@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15100616-0025-0000-0000-00001DAA3174 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that sync_sched_exp_select_cpus() and sync_rcu_exp_select_cpus() are identical aside from the the argument to smp_call_function_single(), this commit consolidates them with a functional argument. Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 7 +++--- kernel/rcu/tree_plugin.h | 61 +----------------------------------------------- 2 files changed, 5 insertions(+), 63 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 500b1347554d..ff68087d93a3 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3654,7 +3654,8 @@ static void synchronize_sched_expedited_cpu_stop(void *data) * Select the nodes that the upcoming expedited grace period needs * to wait for. */ -static void sync_sched_exp_select_cpus(struct rcu_state *rsp) +static void sync_rcu_exp_select_cpus(struct rcu_state *rsp, + smp_call_func_t func) { int cpu; unsigned long flags; @@ -3696,7 +3697,7 @@ static void sync_sched_exp_select_cpus(struct rcu_state *rsp) for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask <<= 1) { if (!(mask_ofl_ipi & mask)) continue; - ret = smp_call_function_single(cpu, synchronize_sched_expedited_cpu_stop, NULL, 0); + ret = smp_call_function_single(cpu, func, rsp, 0); if (!ret) mask_ofl_ipi &= ~mask; } @@ -3788,7 +3789,7 @@ void synchronize_sched_expedited(void) return; /* Someone else did our work for us. */ rcu_exp_gp_seq_start(rsp); - sync_sched_exp_select_cpus(rsp); + sync_rcu_exp_select_cpus(rsp, synchronize_sched_expedited_cpu_stop); synchronize_sched_expedited_wait(rsp); rcu_exp_gp_seq_end(rsp); diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 7880202f1e38..6cbfbfc58656 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -708,65 +708,6 @@ static void sync_rcu_exp_handler(void *info) rcu_report_exp_rdp(rsp, rdp, true); } -/* - * Select the nodes that the upcoming expedited grace period needs - * to wait for. - */ -static void sync_rcu_exp_select_cpus(struct rcu_state *rsp) -{ - int cpu; - unsigned long flags; - unsigned long mask; - unsigned long mask_ofl_test; - unsigned long mask_ofl_ipi; - int ret; - struct rcu_node *rnp; - - sync_exp_reset_tree(rsp); - rcu_for_each_leaf_node(rsp, rnp) { - raw_spin_lock_irqsave(&rnp->lock, flags); - smp_mb__after_unlock_lock(); - - /* Each pass checks a CPU for identity, offline, and idle. */ - mask_ofl_test = 0; - for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++) { - struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu); - struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu); - - if (raw_smp_processor_id() == cpu || - cpu_is_offline(cpu) || - !(atomic_add_return(0, &rdtp->dynticks) & 0x1)) - mask_ofl_test |= rdp->grpmask; - } - mask_ofl_ipi = rnp->expmask & ~mask_ofl_test; - - /* - * Need to wait for any blocked tasks as well. Note that - * additional blocking tasks will also block the expedited - * GP until such time as the ->expmask bits are cleared. - */ - if (rcu_preempt_has_tasks(rnp)) - rnp->exp_tasks = rnp->blkd_tasks.next; - raw_spin_unlock_irqrestore(&rnp->lock, flags); - - /* IPI the remaining CPUs for expedited quiescent state. */ - mask = 1; - for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask <<= 1) { - if (!(mask_ofl_ipi & mask)) - continue; - ret = smp_call_function_single(cpu, - sync_rcu_exp_handler, - rsp, 0); - if (!ret) - mask_ofl_ipi &= ~mask; - } - /* Report quiescent states for those that went offline. */ - mask_ofl_test |= mask_ofl_ipi; - if (mask_ofl_test) - rcu_report_exp_cpu_mult(rsp, rnp, mask_ofl_test, false); - } -} - /** * synchronize_rcu_expedited - Brute-force RCU grace period * @@ -795,7 +736,7 @@ void synchronize_rcu_expedited(void) rcu_exp_gp_seq_start(rsp); /* Initialize the rcu_node tree in preparation for the wait. */ - sync_rcu_exp_select_cpus(rsp); + sync_rcu_exp_select_cpus(rsp, sync_rcu_exp_handler); /* Wait for snapshotted ->blkd_tasks lists to drain. */ rnp = rcu_get_root(rsp); -- 2.5.2