From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org,
rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com,
bobby.prani@gmail.com,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 10/20] rcu: Add functions to test for trivial grace periods
Date: Sat, 14 Jan 2017 01:13:11 -0800 [thread overview]
Message-ID: <1484385201-22227-10-git-send-email-paulmck@linux.vnet.ibm.com> (raw)
In-Reply-To: <20170114091255.GA20854@linux.vnet.ibm.com>
Under some circumstances, RCU grace periods are zero cost. For
RCU-preempt, this is the case during boot, and for RCU-bh and RCU-sched,
this is the case if there is only one CPU. This means that RCU users
might wish to dispense with grace-period-avoidance strategies when
grace periods are zero cost, so this commit adds rcu_trivial_gp(),
rcu_bh_trivial_gp(), and rcu_sched_trivial_gp() to test for these
conditions. Because the conditions leading to zero-cost grace periods
can change at any time (for example, when a second CPU is onlined), these
functions should be used as performance hints, and must not be relied
on for correctness. For example, even if rcu_trivial_gp() returns true,
you are required to invoke synchronize_rcu().
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
include/linux/rcupdate.h | 8 ++++++++
include/linux/rcutiny.h | 10 ++++++++++
include/linux/rcutree.h | 2 ++
kernel/rcu/tree.c | 24 ++++++++++++++++++++++++
kernel/rcu/tree_plugin.h | 11 +++++++++++
5 files changed, 55 insertions(+)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 01f71e1d2e94..a6222478b87d 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -293,6 +293,7 @@ void __rcu_read_lock(void);
void __rcu_read_unlock(void);
void rcu_read_unlock_special(struct task_struct *t);
void synchronize_rcu(void);
+bool rcu_trivial_gp(void);
/*
* Defined as a macro as it is a very low level header included from
@@ -448,6 +449,13 @@ bool __rcu_is_watching(void);
#define RCU_SCHEDULER_INIT 1
#define RCU_SCHEDULER_RUNNING 2
+#ifndef CONFIG_PREEMPT_RCU
+static inline bool rcu_trivial_gp(void)
+{
+ return rcu_sched_trivial_gp();
+}
+#endif /* #ifndef CONFIG_PREEMPT_RCU */
+
/*
* init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic
* initialization and destruction of rcu_head on the stack. rcu_head structures
diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index ac81e4063b40..a77dafe79813 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -82,6 +82,16 @@ static inline void synchronize_sched_expedited(void)
synchronize_sched();
}
+static inline bool rcu_sched_trivial_gp(void)
+{
+ return true;
+}
+
+static inline bool rcu_bh_trivial_gp(void)
+{
+ return true;
+}
+
static inline void kfree_call_rcu(struct rcu_head *head,
rcu_callback_t func)
{
diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
index 63a4e4cf40a5..fcd61cb08851 100644
--- a/include/linux/rcutree.h
+++ b/include/linux/rcutree.h
@@ -47,6 +47,8 @@ static inline void rcu_virt_note_context_switch(int cpu)
void synchronize_rcu_bh(void);
void synchronize_sched_expedited(void);
void synchronize_rcu_expedited(void);
+bool rcu_sched_trivial_gp(void);
+bool rcu_bh_trivial_gp(void);
void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func);
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index d7b63b88434b..ed5a17aca281 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3293,6 +3293,18 @@ void synchronize_sched(void)
EXPORT_SYMBOL_GPL(synchronize_sched);
/**
+ * rcu_sched_trivial_gp - Are RCU-sched grace periods trivially zero cost?
+ *
+ * Returns true if RCU-sched grace periods are currently zero cost, which
+ * they are if there is only one CPU. Note that unless you take steps to
+ * prevent it, the number of CPUs might change at any time.
+ */
+bool rcu_sched_trivial_gp(void)
+{
+ return rcu_blocking_is_gp();
+}
+
+/**
* synchronize_rcu_bh - wait until an rcu_bh grace period has elapsed.
*
* Control will return to the caller some time after a full rcu_bh grace
@@ -3320,6 +3332,18 @@ void synchronize_rcu_bh(void)
EXPORT_SYMBOL_GPL(synchronize_rcu_bh);
/**
+ * rcu_bh_trivial_gp - Are RCU-bh grace periods trivially zero cost?
+ *
+ * Returns true if RCU-bh grace periods are currently zero cost, which
+ * they are if there is only one CPU. Note that unless you take steps to
+ * prevent it, the number of CPUs might change at any time.
+ */
+bool rcu_bh_trivial_gp(void)
+{
+ return rcu_blocking_is_gp();
+}
+
+/**
* get_state_synchronize_rcu - Snapshot current RCU state
*
* Returns a cookie that is used by a later call to cond_synchronize_rcu()
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 56583e764ebf..e92d67a3fad2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -680,6 +680,17 @@ void synchronize_rcu(void)
EXPORT_SYMBOL_GPL(synchronize_rcu);
/**
+ * rcu_trivial_gp - Are RCU grace periods trivially zero cost?
+ *
+ * Returns true if RCU grace periods are currently zero cost, which
+ * they are during boot.
+ */
+bool rcu_trivial_gp(void)
+{
+ return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE;
+}
+
+/**
* rcu_barrier - Wait until all in-flight call_rcu() callbacks complete.
*
* Note that this primitive does not necessarily wait for an RCU grace period
--
2.5.2
next prev parent reply other threads:[~2017-01-14 9:16 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-14 9:12 [PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11 Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 01/20] rcu: update: Make RCU_EXPEDITE_BOOT be the default Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 02/20] lockdep: Make RCU suspicious-access splats use pr_err Paul E. McKenney
2017-01-16 7:53 ` Josh Triplett
2017-01-17 1:06 ` Paul E. McKenney
2017-01-16 17:21 ` Peter Zijlstra
2017-01-16 20:59 ` Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 03/20] Fix: Disable sys_membarrier when nohz_full is enabled Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 04/20] rcu: Only dump stalled-tasks stacks if there was a real stall Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 05/20] rcu: Remove unneeded rcu_process_callbacks() declarations Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 06/20] rcu: Remove unused but set variable Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 07/20] rcu: Remove short-term CPU kicking Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 08/20] rcu: Add long-term " Paul E. McKenney
2017-01-16 7:58 ` Josh Triplett
2017-01-17 1:07 ` Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 09/20] rcu: Once again use NMI-based stack traces in stall warnings Paul E. McKenney
2017-01-14 9:13 ` Paul E. McKenney [this message]
2017-01-16 8:01 ` [PATCH tip/core/rcu 10/20] rcu: Add functions to test for trivial grace periods Josh Triplett
2017-01-17 0:32 ` Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 11/20] sched,rcu: Make cond_resched() provide RCU quiescent state Paul E. McKenney
2017-01-16 8:08 ` Josh Triplett
2017-01-16 11:38 ` Paul E. McKenney
2017-01-16 17:11 ` Peter Zijlstra
2017-01-17 0:54 ` Paul E. McKenney
2017-01-17 10:51 ` Michal Hocko
2017-01-17 12:05 ` Paul E. McKenney
2017-01-17 12:11 ` Michal Hocko
2017-01-18 2:10 ` Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 12/20] rcu: Re-enable TASKS_RCU for User Mode Linux Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 13/20] rcu: Don't wake rcuc/X kthreads on NOCB CPUs Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 14/20] rcu: Add comment headers to expedited-grace-period counter functions Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 15/20] rcu: Make rcu_cpu_starting() use its "cpu" argument Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 16/20] rcu: Enable RCU tracepoints by default to aid in debugging Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 17/20] rcu: Fix comment in rcu_organize_nocb_kthreads() Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 18/20] llist: Clarify comments about when locking is needed Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 19/20] rcu: Eliminate unused expedited_normal counter Paul E. McKenney
2017-01-14 9:13 ` [PATCH tip/core/rcu 20/20] rcu: Add lockdep checks to synchronous expedited primitives Paul E. McKenney
2017-01-16 8:09 ` [PATCH tip/core/rcu 0/20] Miscellaneous fixes for 4.11 Josh Triplett
2017-01-17 1:07 ` Paul E. McKenney
2017-01-18 2:53 ` Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 01/18] rcu: update: Make RCU_EXPEDITE_BOOT be the default Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 02/18] lockdep: Make RCU suspicious-access splats use pr_err Paul E. McKenney
2017-01-21 20:40 ` Josh Triplett
2017-01-23 19:30 ` Paul E. McKenney
2017-01-23 22:33 ` Josh Triplett
2017-01-24 0:13 ` Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 03/18] Fix: Disable sys_membarrier when nohz_full is enabled Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 04/18] rcu: Only dump stalled-tasks stacks if there was a real stall Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 05/18] rcu: Remove unneeded rcu_process_callbacks() declarations Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 06/18] rcu: Remove unused but set variable Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 07/18] rcu: Add long-term CPU kicking Paul E. McKenney
2017-01-21 20:41 ` Josh Triplett
2017-01-21 20:42 ` Josh Triplett
2017-01-23 19:34 ` Paul E. McKenney
2017-01-23 20:25 ` Josh Triplett
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 08/18] rcu: Remove short-term " Paul E. McKenney
2017-01-21 20:43 ` Josh Triplett
2017-01-23 19:36 ` Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 09/18] rcu: Once again use NMI-based stack traces in stall warnings Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 10/18] rcu: Re-enable TASKS_RCU for User Mode Linux Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 11/18] rcu: Don't wake rcuc/X kthreads on NOCB CPUs Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 12/18] rcu: Add comment headers to expedited-grace-period counter functions Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 13/18] rcu: Make rcu_cpu_starting() use its "cpu" argument Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 14/18] rcu: Enable RCU tracepoints by default to aid in debugging Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 15/18] rcu: Fix comment in rcu_organize_nocb_kthreads() Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 16/18] llist: Clarify comments about when locking is needed Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 17/18] rcu: Eliminate unused expedited_normal counter Paul E. McKenney
2017-01-18 2:53 ` [PATCH v2 tip/core/rcu 18/18] rcu: Add lockdep checks to synchronous expedited primitives Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 0/18] Miscellaneous fixes for 4.11 Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 01/18] rcu: update: Make RCU_EXPEDITE_BOOT be the default Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 02/18] lockdep: Make RCU suspicious-access splats use pr_err Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 03/18] Fix: Disable sys_membarrier when nohz_full is enabled Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 04/18] rcu: Only dump stalled-tasks stacks if there was a real stall Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 05/18] rcu: Remove unneeded rcu_process_callbacks() declarations Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 06/18] rcu: Remove unused but set variable Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 07/18] rcu: Add long-term CPU kicking Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 08/18] rcu: Remove short-term " Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 09/18] rcu: Once again use NMI-based stack traces in stall warnings Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 10/18] rcu: Re-enable TASKS_RCU for User Mode Linux Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 11/18] rcu: Don't wake rcuc/X kthreads on NOCB CPUs Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 12/18] rcu: Add comment headers to expedited-grace-period counter functions Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 13/18] rcu: Make rcu_cpu_starting() use its "cpu" argument Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 14/18] rcu: Enable RCU tracepoints by default to aid in debugging Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 15/18] rcu: Fix comment in rcu_organize_nocb_kthreads() Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 16/18] llist: Clarify comments about when locking is needed Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 17/18] rcu: Eliminate unused expedited_normal counter Paul E. McKenney
2017-01-24 21:51 ` [PATCH v3 tip/core/rcu 18/18] rcu: Add lockdep checks to synchronous expedited primitives Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1484385201-22227-10-git-send-email-paulmck@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=bobby.prani@gmail.com \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=dvhart@linux.intel.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=jiangshanlai@gmail.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).