rcu.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: paulmck@kernel.org
To: rcu@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com,
	mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
	akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
	josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org,
	rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com,
	fweisbec@gmail.com, oleg@redhat.com, joel@joelfernandes.org,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Andrii Nakryiko <andriin@fb.com>
Subject: [PATCH RFC v2 tip/core/rcu 14/22] rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks
Date: Wed, 18 Mar 2020 17:10:52 -0700	[thread overview]
Message-ID: <20200319001100.24917-14-paulmck@kernel.org> (raw)
In-Reply-To: <20200319001024.GA28798@paulmck-ThinkPad-P72>

From: "Paul E. McKenney" <paulmck@kernel.org>

Because RCU does not watch exception early-entry/late-exit, idle-loop,
or CPU-hotplug execution, protection of tracing and BPF operations is
needlessly complicated.  This commit therefore adds a variant of
Tasks RCU that:

o	Has explicit read-side markers to allow finite grace periods in
	the face of in-kernel loops for PREEMPT=n builds.  These markers
	are rcu_read_lock_trace() and rcu_read_unlock_trace().

o	Protects code in the idle loop, exception entry/exit, and
	CPU-hotplug code paths.  In this respect, RCU-tasks trace is
	similar to SRCU, but with lighter-weight readers.

o	Avoids expensive read-side instruction, having overhead similar
	to that of Preemptible RCU.

There are of course downsides:

o	The grace-period code can send IPIs to CPUs, even when those CPUs
	are in the idle loop or in nohz_full userspace.  This will be
	addressed by later commits.

o	It is necessary to scan the full tasklist, much as for Tasks RCU.

o	There is a single callback queue guarded by a single lock,
	again, much as for Tasks RCU.  However, those early use cases
	that request multiple grace periods in quick succession are
	expected to do so from a single task, which makes the single
	lock almost irrelevant.  If needed, multiple callback queues
	can be provided using any number of schemes.

Perhaps most important, this variant of RCU does not affect the vanilla
flavors, rcu_preempt and rcu_sched.  The fact that RCU Tasks Trace
readers can operate from idle, offline, and exception entry/exit in no
way enables rcu_preempt and rcu_sched readers to do so.

This effort benefited greatly from off-list discussions of BPF
requirements with Alexei Starovoitov and Andrii Nakryiko.  At least
some of the on-list discussions are captured in the Link: tags below.
In addition, KCSAN was quite helpful in finding some early bugs.

Link: https://lore.kernel.org/lkml/20200219150744.428764577@infradead.org/
Link: https://lore.kernel.org/lkml/87mu8p797b.fsf@nanos.tec.linutronix.de/
Link: https://lore.kernel.org/lkml/20200225221305.605144982@linutronix.de/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrii Nakryiko <andriin@fb.com>
---
 include/linux/rcupdate_trace.h |  84 ++++++++++
 include/linux/sched.h          |   8 +
 init/init_task.c               |   4 +
 kernel/fork.c                  |   4 +
 kernel/rcu/Kconfig             |  12 +-
 kernel/rcu/tasks.h             | 359 ++++++++++++++++++++++++++++++++++++++++-
 6 files changed, 462 insertions(+), 9 deletions(-)
 create mode 100644 include/linux/rcupdate_trace.h

diff --git a/include/linux/rcupdate_trace.h b/include/linux/rcupdate_trace.h
new file mode 100644
index 0000000..ed97e10
--- /dev/null
+++ b/include/linux/rcupdate_trace.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Read-Copy Update mechanism for mutual exclusion, adapted for tracing.
+ *
+ * Copyright (C) 2020 Paul E. McKenney.
+ */
+
+#ifndef __LINUX_RCUPDATE_TRACE_H
+#define __LINUX_RCUPDATE_TRACE_H
+
+#include <linux/sched.h>
+#include <linux/rcupdate.h>
+
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+
+extern struct lockdep_map rcu_trace_lock_map;
+
+static inline int rcu_read_lock_trace_held(void)
+{
+	return lock_is_held(&rcu_trace_lock_map);
+}
+
+#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
+
+static inline int rcu_read_lock_trace_held(void)
+{
+	return 1;
+}
+
+#endif /* #else #ifdef CONFIG_DEBUG_LOCK_ALLOC */
+
+#ifdef CONFIG_TASKS_TRACE_RCU
+
+void rcu_read_unlock_trace_special(struct task_struct *t);
+
+/**
+ * rcu_read_lock_trace - mark beginning of RCU-trace read-side critical section
+ *
+ * When synchronize_rcu_trace() is invoked by one task, then that task
+ * is guaranteed to block until all other tasks exit their read-side
+ * critical sections.  Similarly, if call_rcu_trace() is invoked on one
+ * task while other tasks are within RCU read-side critical sections,
+ * invocation of the corresponding RCU callback is deferred until after
+ * the all the other tasks exit their critical sections.
+ *
+ * For more details, please see the documentation for rcu_read_lock().
+ */
+static inline void rcu_read_lock_trace(void)
+{
+	struct task_struct *t = current;
+
+	WRITE_ONCE(t->trc_reader_nesting, READ_ONCE(t->trc_reader_nesting) + 1);
+	rcu_lock_acquire(&rcu_trace_lock_map);
+}
+
+/**
+ * rcu_read_unlock_trace - mark end of RCU-trace read-side critical section
+ *
+ * Pairs with a preceding call to rcu_read_lock_trace(), and nesting is
+ * allowed.  Invoking a rcu_read_unlock_trace() when there is no matching
+ * rcu_read_lock_trace() is verboten, and will result in lockdep complaints.
+ *
+ * For more details, please see the documentation for rcu_read_unlock().
+ */
+static inline void rcu_read_unlock_trace(void)
+{
+	int nesting;
+	struct task_struct *t = current;
+
+	rcu_lock_release(&rcu_trace_lock_map);
+	nesting = READ_ONCE(t->trc_reader_nesting) - 1;
+	WRITE_ONCE(t->trc_reader_nesting, nesting);
+	if (likely(!READ_ONCE(t->trc_reader_need_end)) || nesting)
+		return;  // We assume shallow reader nesting.
+	rcu_read_unlock_trace_special(t);
+}
+
+void call_rcu_tasks_trace(struct rcu_head *rhp, rcu_callback_t func);
+void synchronize_rcu_tasks_trace(void);
+void rcu_barrier_tasks_trace(void);
+
+#endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
+
+#endif /* __LINUX_RCUPDATE_TRACE_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 621e4aa..ef68ae4 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -722,6 +722,14 @@ struct task_struct {
 	struct list_head		rcu_tasks_holdout_list;
 #endif /* #ifdef CONFIG_TASKS_RCU */
 
+#ifdef CONFIG_TASKS_TRACE_RCU
+	int				trc_reader_nesting;
+	int				trc_ipi_to_cpu;
+	bool				trc_reader_need_end;
+	bool				trc_reader_checked;
+	struct list_head		trc_holdout_list;
+#endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
+
 	struct sched_info		sched_info;
 
 	struct list_head		tasks;
diff --git a/init/init_task.c b/init/init_task.c
index 096191d..1b9ec3d 100644
--- a/init/init_task.c
+++ b/init/init_task.c
@@ -140,6 +140,10 @@ struct task_struct init_task
 	.rcu_tasks_holdout_list = LIST_HEAD_INIT(init_task.rcu_tasks_holdout_list),
 	.rcu_tasks_idle_cpu = -1,
 #endif
+#ifdef CONFIG_TASKS_TRACE_RCU
+	.trc_reader_nesting = 0,
+	.trc_holdout_list = LIST_HEAD_INIT(init_task.trc_holdout_list),
+#endif
 #ifdef CONFIG_CPUSETS
 	.mems_allowed_seq = SEQCNT_ZERO(init_task.mems_allowed_seq),
 #endif
diff --git a/kernel/fork.c b/kernel/fork.c
index e592e6f..d0e547c 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1685,6 +1685,10 @@ static inline void rcu_copy_process(struct task_struct *p)
 	INIT_LIST_HEAD(&p->rcu_tasks_holdout_list);
 	p->rcu_tasks_idle_cpu = -1;
 #endif /* #ifdef CONFIG_TASKS_RCU */
+#ifdef CONFIG_TASKS_TRACE_RCU
+	p->trc_reader_nesting = 0;
+	INIT_LIST_HEAD(&p->trc_holdout_list);
+#endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
 }
 
 struct pid *pidfd_pid(const struct file *file)
diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig
index 0d43ec1..187226b 100644
--- a/kernel/rcu/Kconfig
+++ b/kernel/rcu/Kconfig
@@ -71,7 +71,7 @@ config TREE_SRCU
 	  This option selects the full-fledged version of SRCU.
 
 config TASKS_RCU_GENERIC
-	def_bool TASKS_RCU || TASKS_RUDE_RCU
+	def_bool TASKS_RCU || TASKS_RUDE_RCU || TASKS_TRACE_RCU
 	select SRCU
 	help
 	  This option enables generic infrastructure code supporting
@@ -94,6 +94,16 @@ config TASKS_RUDE_RCU
 	  switches on all online CPUs, including idle ones, so use
 	  with caution.  Not for manual selection.
 
+config TASKS_TRACE_RCU
+	def_bool 0
+	default n
+	help
+	  This option enables a task-based RCU implementation that uses
+	  explicit rcu_read_lock_trace() read-side markers, and allows
+	  these readers to appear in the idle loop as well as on the CPU
+	  hotplug code paths.  It can force IPIs on online CPUs, including
+	  idle ones, so use with caution.  Not for manual selection.
+
 config RCU_STALL_COMMON
 	def_bool TREE_RCU
 	help
diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
index e959052..24f41ec 100644
--- a/kernel/rcu/tasks.h
+++ b/kernel/rcu/tasks.h
@@ -181,12 +181,17 @@ void exit_tasks_rcu_start(void) __acquires(&tasks_rcu_exit_srcu)
 	preempt_enable();
 }
 
+static void exit_tasks_rcu_finish_trace(struct task_struct *t);
+
 /* Do the srcu_read_unlock() for the above synchronize_srcu().  */
 void exit_tasks_rcu_finish(void) __releases(&tasks_rcu_exit_srcu)
 {
+	struct task_struct *t = current;
+
 	preempt_disable();
-	__srcu_read_unlock(&tasks_rcu_exit_srcu, current->rcu_tasks_idx);
+	__srcu_read_unlock(&tasks_rcu_exit_srcu, t->rcu_tasks_idx);
 	preempt_enable();
+	exit_tasks_rcu_finish_trace(t);
 }
 
 #ifndef CONFIG_TINY_RCU
@@ -196,15 +201,19 @@ void exit_tasks_rcu_finish(void) __releases(&tasks_rcu_exit_srcu)
  */
 static void __init rcu_tasks_bootup_oddness(void)
 {
-#ifdef CONFIG_TASKS_RCU
+#if defined(CONFIG_TASKS_RCU) || defined(CONFIG_TASKS_TRACE_RCU)
 	if (rcu_task_stall_timeout != RCU_TASK_STALL_TIMEOUT)
 		pr_info("\tTasks-RCU CPU stall warnings timeout set to %d (rcu_task_stall_timeout).\n", rcu_task_stall_timeout);
-	else
-		pr_info("\tTasks RCU enabled.\n");
+#endif /* #ifdef CONFIG_TASKS_RCU */
+#ifdef CONFIG_TASKS_RCU
+	pr_info("\tTrampoline variant of Tasks RCU enabled.\n");
 #endif /* #ifdef CONFIG_TASKS_RCU */
 #ifdef CONFIG_TASKS_RUDE_RCU
 	pr_info("\tRude variant of Tasks RCU enabled.\n");
 #endif /* #ifdef CONFIG_TASKS_RUDE_RCU */
+#ifdef CONFIG_TASKS_TRACE_RCU
+	pr_info("\tTracing variant of Tasks RCU enabled.\n");
+#endif /* #ifdef CONFIG_TASKS_TRACE_RCU */
 }
 
 #endif /* #ifndef CONFIG_TINY_RCU */
@@ -480,10 +489,10 @@ core_initcall(rcu_spawn_tasks_kthread);
 //
 // "Rude" variant of Tasks RCU, inspired by Steve Rostedt's trick of
 // passing an empty function to schedule_on_each_cpu().  This approach
-// provides an asynchronous call_rcu_rude() API and batching of concurrent
-// calls to the synchronous synchronize_rcu_rude() API.  This sends IPIs
-// far and wide and induces otherwise unnecessary context switches on all
-// online CPUs, whether online or not.
+// provides an asynchronous call_rcu_tasks_rude() API and batching
+// of concurrent calls to the synchronous synchronize_rcu_rude() API.
+// This sends IPIs far and wide and induces otherwise unnecessary context
+// switches on all online CPUs, whether online or not.
 
 // Empty function to allow workqueues to force a context switch.
 static void rcu_tasks_be_rude(struct work_struct *work)
@@ -569,3 +578,337 @@ static int __init rcu_spawn_tasks_rude_kthread(void)
 core_initcall(rcu_spawn_tasks_rude_kthread);
 
 #endif /* #ifdef CONFIG_TASKS_RUDE_RCU */
+
+////////////////////////////////////////////////////////////////////////
+//
+// Tracing variant of Tasks RCU.  This variant is designed to be used
+// to protect tracing hooks, including those of BPF.  This variant
+// therefore:
+//
+// 1.	Has explicit read-side markers to allow finite grace periods
+//	in the face of in-kernel loops for PREEMPT=n builds.
+//
+// 2.	Protects code in the idle loop, exception entry/exit, and
+//	CPU-hotplug code paths, similar to the capabilities of SRCU.
+//
+// 3.	Avoids expensive read-side instruction, having overhead similar
+//	to that of Preemptible RCU.
+//
+// There are of course downsides.  The grace-period code can send IPIs to
+// CPUs, even when those CPUs are in the idle loop or in nohz_full userspace.
+// It is necessary to scan the full tasklist, much as for Tasks RCU.  There
+// is a single callback queue guarded by a single lock, again, much as for
+// Tasks RCU.  If needed, these downsides can be at least partially remedied.
+//
+// Perhaps most important, this variant of RCU does not affect the vanilla
+// flavors, rcu_preempt and rcu_sched.  The fact that RCU Tasks Trace
+// readers can operate from idle, offline, and exception entry/exit in no
+// way allows rcu_preempt and rcu_sched readers to also do so.
+
+// The lockdep state must be outside of #ifdef to be useful.
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+static struct lock_class_key rcu_lock_trace_key;
+struct lockdep_map rcu_trace_lock_map =
+	STATIC_LOCKDEP_MAP_INIT("rcu_read_lock_trace", &rcu_lock_trace_key);
+EXPORT_SYMBOL_GPL(rcu_trace_lock_map);
+#endif /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
+
+#ifdef CONFIG_TASKS_TRACE_RCU
+
+atomic_t trc_n_readers_need_end;	// Number of waited-for readers.
+DECLARE_WAIT_QUEUE_HEAD(trc_wait);	// List of holdout tasks.
+
+// Record outstanding IPIs to each CPU.  No point in sending two...
+static DEFINE_PER_CPU(bool, trc_ipi_to_cpu);
+
+/* If we are the last reader, wake up the grace-period kthread. */
+void rcu_read_unlock_trace_special(struct task_struct *t)
+{
+	WRITE_ONCE(t->trc_reader_need_end, false);
+	if (atomic_dec_and_test(&trc_n_readers_need_end))
+		wake_up(&trc_wait);
+}
+EXPORT_SYMBOL_GPL(rcu_read_unlock_trace_special);
+
+/* Add a task to the holdout list, if it is not already on the list. */
+static void trc_add_holdout(struct task_struct *t, struct list_head *bhp)
+{
+	if (list_empty(&t->trc_holdout_list)) {
+		get_task_struct(t);
+		list_add(&t->trc_holdout_list, bhp);
+	}
+}
+
+/* Remove a task from the holdout list, if it is in fact present. */
+static void trc_del_holdout(struct task_struct *t)
+{
+	if (!list_empty(&t->trc_holdout_list)) {
+		list_del_init(&t->trc_holdout_list);
+		put_task_struct(t);
+	}
+}
+
+/* IPI handler to check task state. */
+static void trc_read_check_handler(void *t_in)
+{
+	struct task_struct *t = current;
+	struct task_struct *texp = t_in;
+
+	// If the task is no longer running on this CPU, leave.
+	if (unlikely(texp != t)) {
+		if (WARN_ON_ONCE(atomic_dec_and_test(&trc_n_readers_need_end)))
+			wake_up(&trc_wait);
+		goto reset_ipi; // Already on holdout list, so will check later.
+	}
+
+	// If the task is not in a read-side critical section, and
+	// if this is the last reader, awaken the grace-period kthread.
+	if (likely(!t->trc_reader_nesting)) {
+		if (WARN_ON_ONCE(atomic_dec_and_test(&trc_n_readers_need_end)))
+			wake_up(&trc_wait);
+		// Mark as checked after decrement to avoid false
+		// positives on the above WARN_ON_ONCE().
+		WRITE_ONCE(t->trc_reader_checked, true);
+		goto reset_ipi;
+	}
+	WRITE_ONCE(t->trc_reader_checked, true);
+
+	// Get here if the task is in a read-side critical section.  Set
+	// its state so that it will awaken the grace-period kthread upon
+	// exit from that critical section.
+	WARN_ON_ONCE(t->trc_reader_need_end);
+	WRITE_ONCE(t->trc_reader_need_end, true);
+
+reset_ipi:
+	// Allow future IPIs to be sent on CPU and for task.
+	// Also order this IPI handler against any later manipulations of
+	// the intended task.
+	smp_store_release(&per_cpu(trc_ipi_to_cpu, smp_processor_id()), false); // ^^^
+	smp_store_release(&texp->trc_ipi_to_cpu, -1); // ^^^
+}
+
+/* Callback function for scheduler to check non-running) task.  */
+static bool trc_inspect_reader_notrunning(struct task_struct *t, void *arg)
+{
+	if (task_curr(t))
+		return false;  // It is running, so decline to inspect it.
+
+	// Mark as checked.  Because this is called from the grace-period
+	// kthread, also remove the task from the holdout list.
+	t->trc_reader_checked = true;
+	trc_del_holdout(t);
+
+	// If the task is in a read-side critical section, set up its
+	// its state so that it will awaken the grace-period kthread upon
+	// exit from that critical section.
+	if (unlikely(t->trc_reader_nesting)) {
+		atomic_inc(&trc_n_readers_need_end); // One more to wait on.
+		WARN_ON_ONCE(t->trc_reader_need_end);
+		WRITE_ONCE(t->trc_reader_need_end, true);
+	}
+	return true;
+}
+
+/* Attempt to extract the state for the specified task. */
+static void trc_wait_for_one_reader(struct task_struct *t,
+				    struct list_head *bhp)
+{
+	int cpu;
+
+	// If a previous IPI is still in flight, let it complete.
+	if (smp_load_acquire(&t->trc_ipi_to_cpu) != -1) // Order IPI
+		return;
+
+	// The current task had better be in a quiescent state.
+	if (t == current) {
+		t->trc_reader_checked = true;
+		trc_del_holdout(t);
+		WARN_ON_ONCE(t->trc_reader_nesting);
+		return;
+	}
+
+	// Attempt to nail down the task for inspection.
+	if (try_invoke_on_locked_down_task(t, trc_inspect_reader_notrunning, t))
+		return;
+
+	// If currently running, send an IPI, either way, add to list.
+	trc_add_holdout(t, bhp);
+	if (task_curr(t)) {
+		// The task is currently running, so try IPIing it.
+		cpu = task_cpu(t);
+
+		// If there is already an IPI outstanding, let it happen.
+		if (per_cpu(trc_ipi_to_cpu, cpu) || t->trc_ipi_to_cpu >= 0)
+			return;
+
+		atomic_inc(&trc_n_readers_need_end);
+		per_cpu(trc_ipi_to_cpu, cpu) = true;
+		t->trc_ipi_to_cpu = cpu;
+		if (smp_call_function_single(cpu,
+					     trc_read_check_handler, t, 0)) {
+			per_cpu(trc_ipi_to_cpu, cpu) = false;
+			t->trc_ipi_to_cpu = cpu;
+		}
+	}
+}
+
+/* Initialize for a new RCU-tasks-trace grace period. */
+static void rcu_tasks_trace_pregp_step(void)
+{
+	int cpu;
+
+	// Wait for CPU-hotplug paths to complete.
+	cpus_read_lock();
+	cpus_read_unlock();
+
+	// Allow for fast-acting IPIs.
+	atomic_set(&trc_n_readers_need_end, 1);
+
+	// There shouldn't be any old IPIs, but...
+	for_each_possible_cpu(cpu)
+		WARN_ON_ONCE(per_cpu(trc_ipi_to_cpu, cpu));
+}
+
+/* Do first-round processing for the specified task. */
+static void rcu_tasks_trace_pertask(struct task_struct *t,
+				    struct list_head *hop)
+{
+	WRITE_ONCE(t->trc_reader_need_end, false);
+	t->trc_reader_checked = false;
+	t->trc_ipi_to_cpu = -1;
+	trc_wait_for_one_reader(t, hop);
+}
+
+/* Do intermediate processing between task and holdout scans. */
+static void rcu_tasks_trace_postscan(void)
+{
+	// Wait for late-stage exiting tasks to finish exiting.
+	// These might have passed the call to exit_tasks_rcu_finish().
+	synchronize_rcu();
+	// Any tasks that exit after this point will set ->trc_reader_checked.
+}
+
+/* Do one scan of the holdout list. */
+static void check_all_holdout_tasks_trace(struct list_head *hop,
+					  bool ndrpt, bool *frptp)
+{
+	struct task_struct *g, *t;
+
+	list_for_each_entry_safe(t, g, hop, trc_holdout_list) {
+		// If safe and needed, try to check the current task.
+		if (READ_ONCE(t->trc_ipi_to_cpu) == -1 &&
+		    !READ_ONCE(t->trc_reader_checked))
+			trc_wait_for_one_reader(t, hop);
+
+		// If check succeeded, remove this task from the list.
+		if (READ_ONCE(t->trc_reader_checked))
+			trc_del_holdout(t);
+	}
+}
+
+/* Wait for grace period to complete and provide ordering. */
+static void rcu_tasks_trace_postgp(void)
+{
+	// Remove the safety count.
+	smp_mb__before_atomic();  // Order vs. earlier atomics
+	atomic_dec(&trc_n_readers_need_end);
+	smp_mb__after_atomic();  // Order vs. later atomics
+
+	// Wait for readers.
+	wait_event_idle_exclusive(trc_wait,
+				  atomic_read(&trc_n_readers_need_end) == 0);
+
+	smp_mb(); // Caller's code must be ordered after wakeup.
+}
+
+/* Report any needed quiescent state for this exiting task. */
+void exit_tasks_rcu_finish_trace(struct task_struct *t)
+{
+	WRITE_ONCE(t->trc_reader_checked, true);
+	WARN_ON_ONCE(t->trc_reader_nesting);
+	WRITE_ONCE(t->trc_reader_nesting, 0);
+	if (WARN_ON_ONCE(READ_ONCE(t->trc_reader_need_end)))
+		rcu_read_unlock_trace_special(t);
+}
+
+void call_rcu_tasks_trace(struct rcu_head *rhp, rcu_callback_t func);
+DEFINE_RCU_TASKS(rcu_tasks_trace, rcu_tasks_wait_gp, call_rcu_tasks_trace,
+		 "RCU Tasks Trace");
+
+/**
+ * call_rcu_tasks_trace() - Queue a callback trace task-based grace period
+ * @rhp: structure to be used for queueing the RCU updates.
+ * @func: actual callback function to be invoked after the grace period
+ *
+ * The callback function will be invoked some time after a full grace
+ * period elapses, in other words after all currently executing RCU
+ * read-side critical sections have completed. call_rcu_tasks_trace()
+ * assumes that the read-side critical sections end at context switch,
+ * cond_resched_rcu_qs(), or transition to usermode execution.  As such,
+ * there are no read-side primitives analogous to rcu_read_lock() and
+ * rcu_read_unlock() because this primitive is intended to determine
+ * that all tasks have passed through a safe state, not so much for
+ * data-strcuture synchronization.
+ *
+ * See the description of call_rcu() for more detailed information on
+ * memory ordering guarantees.
+ */
+void call_rcu_tasks_trace(struct rcu_head *rhp, rcu_callback_t func)
+{
+	call_rcu_tasks_generic(rhp, func, &rcu_tasks_trace);
+}
+EXPORT_SYMBOL_GPL(call_rcu_tasks_trace);
+
+/**
+ * synchronize_rcu_tasks_trace - wait for a trace rcu-tasks grace period
+ *
+ * Control will return to the caller some time after a trace rcu-tasks
+ * grace period has elapsed, in other words after all currently
+ * executing rcu-tasks read-side critical sections have elapsed.  These
+ * read-side critical sections are delimited by calls to schedule(),
+ * cond_resched_tasks_rcu_qs(), userspace execution, and (in theory,
+ * anyway) cond_resched().
+ *
+ * This is a very specialized primitive, intended only for a few uses in
+ * tracing and other situations requiring manipulation of function preambles
+ * and profiling hooks.  The synchronize_rcu_tasks_trace() function is not
+ * (yet) intended for heavy use from multiple CPUs.
+ *
+ * See the description of synchronize_rcu() for more detailed information
+ * on memory ordering guarantees.
+ */
+void synchronize_rcu_tasks_trace(void)
+{
+	RCU_LOCKDEP_WARN(lock_is_held(&rcu_trace_lock_map), "Illegal synchronize_rcu_tasks_trace() in RCU Tasks Trace read-side critical section");
+	synchronize_rcu_tasks_generic(&rcu_tasks_trace);
+}
+EXPORT_SYMBOL_GPL(synchronize_rcu_tasks_trace);
+
+/**
+ * rcu_barrier_tasks_trace - Wait for in-flight call_rcu_tasks_trace() callbacks.
+ *
+ * Although the current implementation is guaranteed to wait, it is not
+ * obligated to, for example, if there are no pending callbacks.
+ */
+void rcu_barrier_tasks_trace(void)
+{
+	/* There is only one callback queue, so this is easy.  ;-) */
+	synchronize_rcu_tasks_trace();
+}
+EXPORT_SYMBOL_GPL(rcu_barrier_tasks_trace);
+
+static int __init rcu_spawn_tasks_trace_kthread(void)
+{
+	rcu_tasks_trace.pregp_func = rcu_tasks_trace_pregp_step;
+	rcu_tasks_trace.pertask_func = rcu_tasks_trace_pertask;
+	rcu_tasks_trace.postscan_func = rcu_tasks_trace_postscan;
+	rcu_tasks_trace.holdouts_func = check_all_holdout_tasks_trace;
+	rcu_tasks_trace.postgp_func = rcu_tasks_trace_postgp;
+	rcu_spawn_tasks_kthread_generic(&rcu_tasks_trace);
+	return 0;
+}
+core_initcall(rcu_spawn_tasks_trace_kthread);
+
+#else /* #ifdef CONFIG_TASKS_TRACE_RCU */
+void exit_tasks_rcu_finish_trace(struct task_struct *t) { }
+#endif /* #else #ifdef CONFIG_TASKS_TRACE_RCU */
-- 
2.9.5


  parent reply	other threads:[~2020-03-19  0:11 UTC|newest]

Thread overview: 171+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-12 18:16 [PATCH RFC tip/core/rcu 0/16] Prototype RCU usable from idle, exception, offline Paul E. McKenney
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 01/16] sched/core: Add function to sample state of non-running function paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 02/16] rcu: Add per-task state to RCU CPU stall warnings paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 03/16] rcutorture: Add flag to produce non-busy-wait task stalls paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 04/16] rcu-tasks: Move Tasks RCU to its own file paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 05/16] rcu-tasks: Create struct to hold state information paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 06/16] rcu: Reinstate synchronize_rcu_mult() paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 07/16] rcutorture: Add a test for synchronize_rcu_mult() paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 08/16] rcu-tasks: Refactor RCU-tasks to allow variants to be added paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 09/16] rcu-tasks: Add an RCU-tasks rude variant paulmck
2020-03-16 19:47   ` Joel Fernandes
2020-03-16 20:17     ` Joel Fernandes
2020-03-16 20:32       ` Paul E. McKenney
2020-03-16 21:32         ` Steven Rostedt
2020-03-16 21:45           ` Joel Fernandes
2020-03-16 22:03             ` Steven Rostedt
2020-03-16 22:11               ` Paul E. McKenney
2020-05-10  9:59               ` Lai Jiangshan
2020-05-10 15:49                 ` Paul E. McKenney
2020-05-11  0:06                   ` Lai Jiangshan
2020-05-11  2:44                     ` Paul E. McKenney
2020-05-11  0:44                 ` Masami Hiramatsu
2020-05-11 16:07                 ` Steven Rostedt
2020-05-11 19:48                 ` Steven Rostedt
2020-03-16 20:29     ` Paul E. McKenney
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 10/16] rcutorture: Add torture tests for RCU Tasks Rude paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 11/16] rcu-tasks: Use unique names for RCU-Tasks kthreads and messages paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 12/16] rcu-tasks: Further refactor RCU-tasks to allow adding more variants paulmck
2020-03-12 18:16 ` [PATCH RFC tip/core/rcu 13/16] rcu-tasks: Code movement to allow more Tasks RCU variants paulmck
2020-03-12 18:17 ` [PATCH RFC tip/core/rcu 14/16] rcu: Add an RCU Tasks Trace to simplify protection of tracing hooks paulmck
2020-03-12 18:17 ` [PATCH RFC tip/core/rcu 15/16] rcutorture: Add torture tests for RCU Tasks Trace paulmck
2020-03-12 18:17 ` [PATCH RFC tip/core/rcu 16/16] rcu-tasks: Add stall warnings " paulmck
2020-03-13 14:41 ` [PATCH RFC tip/core/rcu 0/16] Prototype RCU usable from idle, exception, offline Frederic Weisbecker
2020-03-13 15:42   ` Paul E. McKenney
2020-03-15 17:45     ` Mathieu Desnoyers
2020-03-15 17:59       ` Paul E. McKenney
2020-03-16 18:36         ` Steven Rostedt
2020-03-16 18:52           ` Paul E. McKenney
2020-03-16 14:45     ` Frederic Weisbecker
2020-03-16 15:39       ` Paul E. McKenney
2020-03-19  0:10 ` [PATCH RFC v2 tip/core/rcu 0/22] " Paul E. McKenney
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 01/22] sched/core: Add function to sample state of locked-down task paulmck
2020-03-19 17:22     ` Steven Rostedt
2020-03-19 17:35       ` Paul E. McKenney
2020-03-20  2:49         ` Paul E. McKenney
2020-03-20  3:09           ` Steven Rostedt
2020-03-20 16:27             ` Paul E. McKenney
2020-03-24  0:06           ` Joel Fernandes
2020-03-24  0:15             ` Joel Fernandes
2020-03-24 16:26               ` Paul E. McKenney
2020-03-24 15:48             ` Paul E. McKenney
2020-03-24 16:52               ` Joel Fernandes
2020-03-24 17:20                 ` Paul E. McKenney
2020-03-24 18:19                   ` Joel Fernandes
2020-03-25  0:58                     ` Paul E. McKenney
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 02/22] rcu: Add per-task state to RCU CPU stall warnings paulmck
2020-03-19 17:27     ` Steven Rostedt
2020-03-19 17:41       ` Paul E. McKenney
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 03/22] rcutorture: Add flag to produce non-busy-wait task stalls paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 04/22] rcu-tasks: Move Tasks RCU to its own file paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 05/22] rcu-tasks: Create struct to hold state information paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 06/22] rcu: Reinstate synchronize_rcu_mult() paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 07/22] rcutorture: Add a test for synchronize_rcu_mult() paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 08/22] rcu-tasks: Refactor RCU-tasks to allow variants to be added paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 09/22] rcu-tasks: Add an RCU-tasks rude variant paulmck
2020-03-19 19:04     ` Steven Rostedt
2020-03-19 23:58       ` Paul E. McKenney
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 10/22] rcutorture: Add torture tests for RCU Tasks Rude paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 11/22] rcu-tasks: Use unique names for RCU-Tasks kthreads and messages paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 12/22] rcu-tasks: Further refactor RCU-tasks to allow adding more variants paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 13/22] rcu-tasks: Code movement to allow more Tasks RCU variants paulmck
2020-03-19  0:10   ` paulmck [this message]
2020-03-19  1:37     ` [PATCH RFC v2 tip/core/rcu 14/22] rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks Joel Fernandes
2020-03-19  1:58       ` Joel Fernandes
2020-03-19  3:40         ` Paul E. McKenney
2020-03-19 19:42     ` Steven Rostedt
2020-03-20  0:28       ` Paul E. McKenney
2020-03-20  0:48         ` Steven Rostedt
2020-03-20  2:41           ` Paul E. McKenney
2020-03-28 14:06             ` Joel Fernandes
2020-03-28 15:34               ` Paul E. McKenney
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 15/22] rcutorture: Add torture tests for RCU Tasks Trace paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 16/22] rcu-tasks: Add stall warnings " paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 17/22] rcu-tasks: Move #ifdef into tasks.h paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 18/22] rcu-tasks: Add RCU tasks to rcutorture writer stall output paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 19/22] rcu-tasks: Make rcutorture writer stall output include GP state paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 20/22] rcu-tasks: Make RCU Tasks Trace make use of RCU scheduler hooks paulmck
2020-03-19  0:10   ` [PATCH RFC v2 tip/core/rcu 21/22] rcu-tasks: Add a grace-period start time for throttling and debug paulmck
2020-03-19  0:11   ` [PATCH RFC v2 tip/core/rcu 22/22] rcu-tasks: Provide boot parameter to delay IPIs until late in grace period paulmck
2020-03-19 11:31   ` [PATCH RFC v2 tip/core/rcu 0/22] Prototype RCU usable from idle, exception, offline Mathieu Desnoyers
2020-03-19 13:13     ` Paul E. McKenney
     [not found]   ` <20200319104614.11444-1-hdanton@sina.com>
2020-03-19 12:38     ` [PATCH RFC v2 tip/core/rcu 03/22] rcutorture: Add flag to produce non-busy-wait task stalls Paul E. McKenney
     [not found]     ` <20200319133947.12172-1-hdanton@sina.com>
2020-03-19 15:22       ` Paul E. McKenney
     [not found]       ` <20200320040329.9840-1-hdanton@sina.com>
2020-03-20 16:11         ` Paul E. McKenney
     [not found]   ` <20200320071228.9740-1-hdanton@sina.com>
2020-03-20 19:14     ` [PATCH RFC v2 tip/core/rcu 04/22] rcu-tasks: Move Tasks RCU to its own file Paul E. McKenney
2020-03-27 22:23   ` [PATCH RFC v3 tip/core/rcu 0/34] Prototype RCU usable from idle, exception, offline Paul E. McKenney
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 01/34] sched/core: Add function to sample state of locked-down task paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 02/34] rcu: Add per-task state to RCU CPU stall warnings paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 03/34] rcutorture: Add flag to produce non-busy-wait task stalls paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 04/34] rcu-tasks: Move Tasks RCU to its own file paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 05/34] rcu-tasks: Create struct to hold state information paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 06/34] rcu: Reinstate synchronize_rcu_mult() paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 07/34] rcutorture: Add a test for synchronize_rcu_mult() paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 08/34] rcu-tasks: Refactor RCU-tasks to allow variants to be added paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 09/34] rcu-tasks: Add an RCU-tasks rude variant paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 10/34] rcutorture: Add torture tests for RCU Tasks Rude paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 11/34] rcu-tasks: Use unique names for RCU-Tasks kthreads and messages paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 12/34] rcu-tasks: Further refactor RCU-tasks to allow adding more variants paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 13/34] rcu-tasks: Code movement to allow more Tasks RCU variants paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 14/34] rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 15/34] rcutorture: Add torture tests for RCU Tasks Trace paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 16/34] rcu-tasks: Add stall warnings " paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 17/34] rcu-tasks: Move #ifdef into tasks.h paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 18/34] rcu-tasks: Add RCU tasks to rcutorture writer stall output paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 19/34] rcu-tasks: Make rcutorture writer stall output include GP state paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 20/34] rcu-tasks: Make RCU Tasks Trace make use of RCU scheduler hooks paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 21/34] rcu-tasks: Add a grace-period start time for throttling and debug paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 22/34] rcu-tasks: Provide boot parameter to delay IPIs until late in grace period paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 23/34] rcu-tasks: Split ->trc_reader_need_end paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 24/34] rcu-tasks: Add grace-period and IPI counts to statistics paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 25/34] rcu-tasks: Add Kconfig option to mediate smp_mb() vs. IPI paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 26/34] rcu-tasks: Avoid IPIing userspace/idle tasks if kernel is so built paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 27/34] rcu-tasks: Allow rcu_read_unlock_trace() under scheduler locks paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 28/34] rcu-tasks: Disable CPU hotplug across RCU tasks trace scans paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 29/34] rcu-tasks: Handle the running-offline idle-task special case paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 30/34] rcu-tasks: Make RCU tasks trace also wait for idle tasks paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 31/34] rcu-tasks: Add rcu_dynticks_zero_in_eqs() effectiveness statistics paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 32/34] rcu-tasks: Add count for idle tasks on offline CPUs paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 33/34] rcutorture: Add TRACE02 scenario enabling RCU Tasks Trace IPIs paulmck
2020-03-27 22:24     ` [PATCH v3 tip/core/rcu 34/34] rcu-tasks: Add IPI failure count to statistics paulmck
2020-04-15 18:18     ` [PATCH v4 tip/core/rcu 0/38] Prototype RCU usable from idle, exception, offline Paul E. McKenney
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 01/38] rcu: Add comments marking transitions between RCU watching and not paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 02/38] rcu-tasks: Use context-switch hook for PREEMPT=y kernels paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 03/38] sched/core: Add function to sample state of locked-down task paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 04/38] rcu: Add per-task state to RCU CPU stall warnings paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 05/38] rcu-tasks: Move Tasks RCU to its own file paulmck
2020-05-10  7:42         ` Lai Jiangshan
2020-05-10 15:39           ` Paul E. McKenney
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 06/38] rcu-tasks: Create struct to hold state information paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 07/38] rcu: Reinstate synchronize_rcu_mult() paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 08/38] rcutorture: Add a test for synchronize_rcu_mult() paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 09/38] rcu-tasks: Refactor RCU-tasks to allow variants to be added paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 10/38] rcu-tasks: Add an RCU-tasks rude variant paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 11/38] rcutorture: Add torture tests for RCU Tasks Rude paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 12/38] rcu-tasks: Use unique names for RCU-Tasks kthreads and messages paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 13/38] rcu-tasks: Further refactor RCU-tasks to allow adding more variants paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 14/38] rcu-tasks: Code movement to allow more Tasks RCU variants paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 15/38] rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 16/38] rcutorture: Add torture tests for RCU Tasks Trace paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 17/38] rcu-tasks: Add stall warnings " paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 18/38] rcu-tasks: Move #ifdef into tasks.h paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 19/38] rcu-tasks: Add RCU tasks to rcutorture writer stall output paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 20/38] rcu-tasks: Make rcutorture writer stall output include GP state paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 21/38] rcu-tasks: Make RCU Tasks Trace make use of RCU scheduler hooks paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 22/38] rcu-tasks: Add a grace-period start time for throttling and debug paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 23/38] rcu-tasks: Provide boot parameter to delay IPIs until late in grace period paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 24/38] rcu-tasks: Split ->trc_reader_need_end paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 25/38] rcu-tasks: Add grace-period and IPI counts to statistics paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 26/38] rcu-tasks: Add Kconfig option to mediate smp_mb() vs. IPI paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 27/38] rcu-tasks: Avoid IPIing userspace/idle tasks if kernel is so built paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 28/38] rcu-tasks: Allow rcu_read_unlock_trace() under scheduler locks paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 29/38] rcu-tasks: Disable CPU hotplug across RCU tasks trace scans paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 30/38] rcu-tasks: Handle the running-offline idle-task special case paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 31/38] rcu-tasks: Make RCU tasks trace also wait for idle tasks paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 32/38] rcu-tasks: Add rcu_dynticks_zero_in_eqs() effectiveness statistics paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 33/38] rcu-tasks: Add count for idle tasks on offline CPUs paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 34/38] rcutorture: Add TRACE02 scenario enabling RCU Tasks Trace IPIs paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 35/38] rcu-tasks: Add IPI failure count to statistics paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 36/38] rcu-tasks: Allow standalone use of TASKS_{TRACE_,}RCU paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 37/38] ftrace: Use synchronize_rcu_tasks_rude() instead of ftrace_sync() paulmck
2020-04-15 18:19       ` [PATCH v4 tip/core/rcu 38/38] rcu: Don't acquire lock in NMI handler in rcu_nmi_enter_common() paulmck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200319001100.24917-14-paulmck@kernel.org \
    --to=paulmck@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andriin@fb.com \
    --cc=dhowells@redhat.com \
    --cc=dipankar@in.ibm.com \
    --cc=edumazet@google.com \
    --cc=fweisbec@gmail.com \
    --cc=jiangshanlai@gmail.com \
    --cc=joel@joelfernandes.org \
    --cc=josh@joshtriplett.org \
    --cc=kernel-team@fb.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mingo@kernel.org \
    --cc=oleg@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rcu@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).