[PATCH tip/core/rcu 0/5] Documentation and rcutorture changes

[PATCH tip/core/rcu 0/5] Documentation and rcutorture changes

[Paul E. McKenney]

Hello!

This series covers changes to rcutorture and documentation updates.
The individual patches in this series are as follows:

1.	Update rcutorture default values so that casual rcutorture
	users will do more aggressive testing.
2.	Make rcutorture track CPU-hotplug latency statistics.
3.	Document SRCU's new-found ability to be used by offline and
	idle CPUs, and also emphasize SRCU's limitations.
4.	Use the new pr_*() interfaces in rcutorture.
5.	Prevent kthread-initialization races in rcutorture.

							Thanx, Paul

------------------------------------------------------------------------

 b/Documentation/RCU/checklist.txt |    6 +
 b/Documentation/RCU/whatisRCU.txt |    9 +-
 b/kernel/rcutorture.c             |    4 -
 kernel/rcutorture.c               |  152 +++++++++++++++++++++++---------------
 4 files changed, 108 insertions(+), 63 deletions(-)

[PATCH tip/core/rcu 1/5] rcu: Update rcutorture defaults

[Paul E. McKenney]

From: "Paul E. McKenney" <paul.mckenney@linaro.org>

A number of new features have been added to rcutorture over the years, but
the defaults have not been updated to include them.  This commit therefore
turns on a couple of them that have proven helpful and trustworthy, namely
periodic progress reports and testing of NO_HZ.

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcutorture.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
index 25b1503..86315d3 100644
--- a/kernel/rcutorture.c
+++ b/kernel/rcutorture.c
@@ -53,10 +53,10 @@ MODULE_AUTHOR("Paul E. McKenney <paulmck@us.ibm.com> and Josh Triplett <josh@fre
 
 static int nreaders = -1;	/* # reader threads, defaults to 2*ncpus */
 static int nfakewriters = 4;	/* # fake writer threads */
-static int stat_interval;	/* Interval between stats, in seconds. */
+static int stat_interval = 60;	/* Interval between stats, in seconds. */
 				/*  Defaults to "only at end of test". */
 static bool verbose;		/* Print more debug info. */
-static bool test_no_idle_hz;	/* Test RCU's support for tickless idle CPUs. */
+static bool test_no_idle_hz = 1; /* Test RCU support for tickless idle CPUs. */
 static int shuffle_interval = 3; /* Interval between shuffles (in sec)*/
 static int stutter = 5;		/* Start/stop testing interval (in sec) */
 static int irqreader = 1;	/* RCU readers from irq (timers). */
-- 
1.7.8

[PATCH tip/core/rcu 2/5] rcu: Track CPU-hotplug duration statistics

[Paul E. McKenney]

From: "Paul E. McKenney" <paul.mckenney@linaro.org>

Many rcutorture runs include CPU-hotplug operations in their stress
testing.  This commit accumulates statistics on the durations of these
operations in deference to the recent concern about the overhead and
latency of these operations.

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcutorture.c |   42 +++++++++++++++++++++++++++++++++++++-----
 1 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
index 86315d3..c6cf6ff 100644
--- a/kernel/rcutorture.c
+++ b/kernel/rcutorture.c
@@ -176,8 +176,14 @@ static long n_rcu_torture_boosts;
 static long n_rcu_torture_timers;
 static long n_offline_attempts;
 static long n_offline_successes;
+static unsigned long sum_offline;
+static int min_offline = -1;
+static int max_offline;
 static long n_online_attempts;
 static long n_online_successes;
+static unsigned long sum_online;
+static int min_online = -1;
+static int max_online;
 static long n_barrier_attempts;
 static long n_barrier_successes;
 static struct list_head rcu_torture_removed;
@@ -1214,11 +1220,13 @@ rcu_torture_printk(char *page)
 		       n_rcu_torture_boost_failure,
 		       n_rcu_torture_boosts,
 		       n_rcu_torture_timers);
-	cnt += sprintf(&page[cnt], "onoff: %ld/%ld:%ld/%ld ",
-		       n_online_successes,
-		       n_online_attempts,
-		       n_offline_successes,
-		       n_offline_attempts);
+	cnt += sprintf(&page[cnt],
+		       "onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
+		       n_online_successes, n_online_attempts,
+		       n_offline_successes, n_offline_attempts,
+		       min_online, max_online,
+		       min_offline, max_offline,
+		       sum_online, sum_offline, HZ);
 	cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
 		       n_barrier_successes,
 		       n_barrier_attempts,
@@ -1490,8 +1498,10 @@ static int __cpuinit
 rcu_torture_onoff(void *arg)
 {
 	int cpu;
+	unsigned long delta;
 	int maxcpu = -1;
 	DEFINE_RCU_RANDOM(rand);
+	unsigned long starttime;
 
 	VERBOSE_PRINTK_STRING("rcu_torture_onoff task started");
 	for_each_online_cpu(cpu)
@@ -1509,6 +1519,7 @@ rcu_torture_onoff(void *arg)
 				printk(KERN_ALERT "%s" TORTURE_FLAG
 				       "rcu_torture_onoff task: offlining %d\n",
 				       torture_type, cpu);
+			starttime = jiffies;
 			n_offline_attempts++;
 			if (cpu_down(cpu) == 0) {
 				if (verbose)
@@ -1516,12 +1527,23 @@ rcu_torture_onoff(void *arg)
 					       "rcu_torture_onoff task: offlined %d\n",
 					       torture_type, cpu);
 				n_offline_successes++;
+				delta = jiffies - starttime;
+				sum_offline += delta;
+				if (min_offline < 0) {
+					min_offline = delta;
+					max_offline = delta;
+				}
+				if (min_offline > delta)
+					min_offline = delta;
+				if (max_offline < delta)
+					max_offline = delta;
 			}
 		} else if (cpu_is_hotpluggable(cpu)) {
 			if (verbose)
 				printk(KERN_ALERT "%s" TORTURE_FLAG
 				       "rcu_torture_onoff task: onlining %d\n",
 				       torture_type, cpu);
+			starttime = jiffies;
 			n_online_attempts++;
 			if (cpu_up(cpu) == 0) {
 				if (verbose)
@@ -1529,6 +1551,16 @@ rcu_torture_onoff(void *arg)
 					       "rcu_torture_onoff task: onlined %d\n",
 					       torture_type, cpu);
 				n_online_successes++;
+				delta = jiffies - starttime;
+				sum_online += delta;
+				if (min_online < 0) {
+					min_online = delta;
+					max_online = delta;
+				}
+				if (min_online > delta)
+					min_online = delta;
+				if (max_online < delta)
+					max_online = delta;
 			}
 		}
 		schedule_timeout_interruptible(onoff_interval * HZ);
-- 
1.7.8

[PATCH tip/core/rcu 3/5] rcu: Document SRCU dead-CPU capabilities, emphasize read-side limits

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

The current documentation did not help someone grepping for SRCU to
learn that disabling preemption is not a replacement for srcu_read_lock(),
so upgrade the documentation to bring this out, not just for SRCU,
but also for RCU-bh.  Also document the fact that SRCU readers are
respected on CPUs executing in user mode, idle CPUs, and even on
offline CPUs.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 Documentation/RCU/checklist.txt |    6 ++++++
 Documentation/RCU/whatisRCU.txt |    9 +++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
index fc103d7..cdb20d4 100644
--- a/Documentation/RCU/checklist.txt
+++ b/Documentation/RCU/checklist.txt
@@ -310,6 +310,12 @@ over a rather long period of time, but improvements are always welcome!
 	code under the influence of preempt_disable(), you instead
 	need to use synchronize_irq() or synchronize_sched().
 
+	This same limitation also applies to synchronize_rcu_bh()
+	and synchronize_srcu(), as well as to the asynchronous and
+	expedited forms of the three primitives, namely call_rcu(),
+	call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(),
+	synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited().
+
 12.	Any lock acquired by an RCU callback must be acquired elsewhere
 	with softirq disabled, e.g., via spin_lock_irqsave(),
 	spin_lock_bh(), etc.  Failing to disable irq on a given
diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
index 69ee188..bf0f6de 100644
--- a/Documentation/RCU/whatisRCU.txt
+++ b/Documentation/RCU/whatisRCU.txt
@@ -873,7 +873,7 @@ d.	Do you need to treat NMI handlers, hardirq handlers,
 	and code segments with preemption disabled (whether
 	via preempt_disable(), local_irq_save(), local_bh_disable(),
 	or some other mechanism) as if they were explicit RCU readers?
-	If so, you need RCU-sched.
+	If so, RCU-sched is the only choice that will work for you.
 
 e.	Do you need RCU grace periods to complete even in the face
 	of softirq monopolization of one or more of the CPUs?  For
@@ -884,7 +884,12 @@ f.	Is your workload too update-intensive for normal use of
 	RCU, but inappropriate for other synchronization mechanisms?
 	If so, consider SLAB_DESTROY_BY_RCU.  But please be careful!
 
-g.	Otherwise, use RCU.
+g.	Do you need read-side critical sections that are respected
+	even though they are in the middle of the idle loop, during
+	user-mode execution, or on an offlined CPU?  If so, SRCU is the
+	only choice that will work for you.
+
+h.	Otherwise, use RCU.
 
 Of course, this all assumes that you have determined that RCU is in fact
 the right tool for your job.
-- 
1.7.8

[PATCH tip/core/rcu 4/5] rcu: Switch rcutorture to pr_alert() and friends

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

Drop a few characters by switching kernel/rcutorture.c from
"printk(KERN_ALERT" to "pr_alert(".

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcutorture.c |  100 +++++++++++++++++++++++++-------------------------
 1 files changed, 50 insertions(+), 50 deletions(-)

diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
index c6cf6ff..8ea2726 100644
--- a/kernel/rcutorture.c
+++ b/kernel/rcutorture.c
@@ -119,11 +119,11 @@ MODULE_PARM_DESC(torture_type, "Type of RCU to torture (rcu, rcu_bh, srcu)");
 
 #define TORTURE_FLAG "-torture:"
 #define PRINTK_STRING(s) \
-	do { printk(KERN_ALERT "%s" TORTURE_FLAG s "\n", torture_type); } while (0)
+	do { pr_alert("%s" TORTURE_FLAG s "\n", torture_type); } while (0)
 #define VERBOSE_PRINTK_STRING(s) \
-	do { if (verbose) printk(KERN_ALERT "%s" TORTURE_FLAG s "\n", torture_type); } while (0)
+	do { if (verbose) pr_alert("%s" TORTURE_FLAG s "\n", torture_type); } while (0)
 #define VERBOSE_PRINTK_ERRSTRING(s) \
-	do { if (verbose) printk(KERN_ALERT "%s" TORTURE_FLAG "!!! " s "\n", torture_type); } while (0)
+	do { if (verbose) pr_alert("%s" TORTURE_FLAG "!!! " s "\n", torture_type); } while (0)
 
 static char printk_buf[4096];
 
@@ -241,7 +241,7 @@ rcutorture_shutdown_notify(struct notifier_block *unused1,
 	if (fullstop == FULLSTOP_DONTSTOP)
 		fullstop = FULLSTOP_SHUTDOWN;
 	else
-		printk(KERN_WARNING /* but going down anyway, so... */
+		pr_warning(/* but going down anyway, so... */
 		       "Concurrent 'rmmod rcutorture' and shutdown illegal!\n");
 	mutex_unlock(&fullstop_mutex);
 	return NOTIFY_DONE;
@@ -254,7 +254,7 @@ rcutorture_shutdown_notify(struct notifier_block *unused1,
 static void rcutorture_shutdown_absorb(char *title)
 {
 	if (ACCESS_ONCE(fullstop) == FULLSTOP_SHUTDOWN) {
-		printk(KERN_NOTICE
+		pr_notice(
 		       "rcutorture thread %s parking due to system shutdown\n",
 		       title);
 		schedule_timeout_uninterruptible(MAX_SCHEDULE_TIMEOUT);
@@ -1275,7 +1275,7 @@ rcu_torture_stats_print(void)
 	int cnt;
 
 	cnt = rcu_torture_printk(printk_buf);
-	printk(KERN_ALERT "%s", printk_buf);
+	pr_alert("%s", printk_buf);
 }
 
 /*
@@ -1388,20 +1388,20 @@ rcu_torture_stutter(void *arg)
 static inline void
 rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, char *tag)
 {
-	printk(KERN_ALERT "%s" TORTURE_FLAG
-		"--- %s: nreaders=%d nfakewriters=%d "
-		"stat_interval=%d verbose=%d test_no_idle_hz=%d "
-		"shuffle_interval=%d stutter=%d irqreader=%d "
-		"fqs_duration=%d fqs_holdoff=%d fqs_stutter=%d "
-		"test_boost=%d/%d test_boost_interval=%d "
-		"test_boost_duration=%d shutdown_secs=%d "
-		"onoff_interval=%d onoff_holdoff=%d\n",
-		torture_type, tag, nrealreaders, nfakewriters,
-		stat_interval, verbose, test_no_idle_hz, shuffle_interval,
-		stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
-		test_boost, cur_ops->can_boost,
-		test_boost_interval, test_boost_duration, shutdown_secs,
-		onoff_interval, onoff_holdoff);
+	pr_alert("%s" TORTURE_FLAG
+		 "--- %s: nreaders=%d nfakewriters=%d "
+		 "stat_interval=%d verbose=%d test_no_idle_hz=%d "
+		 "shuffle_interval=%d stutter=%d irqreader=%d "
+		 "fqs_duration=%d fqs_holdoff=%d fqs_stutter=%d "
+		 "test_boost=%d/%d test_boost_interval=%d "
+		 "test_boost_duration=%d shutdown_secs=%d "
+		 "onoff_interval=%d onoff_holdoff=%d\n",
+		 torture_type, tag, nrealreaders, nfakewriters,
+		 stat_interval, verbose, test_no_idle_hz, shuffle_interval,
+		 stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
+		 test_boost, cur_ops->can_boost,
+		 test_boost_interval, test_boost_duration, shutdown_secs,
+		 onoff_interval, onoff_holdoff);
 }
 
 static struct notifier_block rcutorture_shutdown_nb = {
@@ -1468,9 +1468,9 @@ rcu_torture_shutdown(void *arg)
 	       !kthread_should_stop()) {
 		delta = shutdown_time - jiffies_snap;
 		if (verbose)
-			printk(KERN_ALERT "%s" TORTURE_FLAG
-			       "rcu_torture_shutdown task: %lu jiffies remaining\n",
-			       torture_type, delta);
+			pr_alert("%s" TORTURE_FLAG
+				 "rcu_torture_shutdown task: %lu jiffies remaining\n",
+				 torture_type, delta);
 		schedule_timeout_interruptible(delta);
 		jiffies_snap = ACCESS_ONCE(jiffies);
 	}
@@ -1516,16 +1516,16 @@ rcu_torture_onoff(void *arg)
 		cpu = (rcu_random(&rand) >> 4) % (maxcpu + 1);
 		if (cpu_online(cpu) && cpu_is_hotpluggable(cpu)) {
 			if (verbose)
-				printk(KERN_ALERT "%s" TORTURE_FLAG
-				       "rcu_torture_onoff task: offlining %d\n",
-				       torture_type, cpu);
+				pr_alert("%s" TORTURE_FLAG
+					 "rcu_torture_onoff task: offlining %d\n",
+					 torture_type, cpu);
 			starttime = jiffies;
 			n_offline_attempts++;
 			if (cpu_down(cpu) == 0) {
 				if (verbose)
-					printk(KERN_ALERT "%s" TORTURE_FLAG
-					       "rcu_torture_onoff task: offlined %d\n",
-					       torture_type, cpu);
+					pr_alert("%s" TORTURE_FLAG
+						 "rcu_torture_onoff task: offlined %d\n",
+						 torture_type, cpu);
 				n_offline_successes++;
 				delta = jiffies - starttime;
 				sum_offline += delta;
@@ -1540,16 +1540,16 @@ rcu_torture_onoff(void *arg)
 			}
 		} else if (cpu_is_hotpluggable(cpu)) {
 			if (verbose)
-				printk(KERN_ALERT "%s" TORTURE_FLAG
-				       "rcu_torture_onoff task: onlining %d\n",
-				       torture_type, cpu);
+				pr_alert("%s" TORTURE_FLAG
+					 "rcu_torture_onoff task: onlining %d\n",
+					 torture_type, cpu);
 			starttime = jiffies;
 			n_online_attempts++;
 			if (cpu_up(cpu) == 0) {
 				if (verbose)
-					printk(KERN_ALERT "%s" TORTURE_FLAG
-					       "rcu_torture_onoff task: onlined %d\n",
-					       torture_type, cpu);
+					pr_alert("%s" TORTURE_FLAG
+						 "rcu_torture_onoff task: onlined %d\n",
+						 torture_type, cpu);
 				n_online_successes++;
 				delta = jiffies - starttime;
 				sum_online += delta;
@@ -1625,14 +1625,14 @@ static int __cpuinit rcu_torture_stall(void *args)
 	if (!kthread_should_stop()) {
 		stop_at = get_seconds() + stall_cpu;
 		/* RCU CPU stall is expected behavior in following code. */
-		printk(KERN_ALERT "rcu_torture_stall start.\n");
+		pr_alert("rcu_torture_stall start.\n");
 		rcu_read_lock();
 		preempt_disable();
 		while (ULONG_CMP_LT(get_seconds(), stop_at))
 			continue;  /* Induce RCU CPU stall warning. */
 		preempt_enable();
 		rcu_read_unlock();
-		printk(KERN_ALERT "rcu_torture_stall end.\n");
+		pr_alert("rcu_torture_stall end.\n");
 	}
 	rcutorture_shutdown_absorb("rcu_torture_stall");
 	while (!kthread_should_stop())
@@ -1748,12 +1748,12 @@ static int rcu_torture_barrier_init(void)
 	if (n_barrier_cbs == 0)
 		return 0;
 	if (cur_ops->call == NULL || cur_ops->cb_barrier == NULL) {
-		printk(KERN_ALERT "%s" TORTURE_FLAG
-		       " Call or barrier ops missing for %s,\n",
-		       torture_type, cur_ops->name);
-		printk(KERN_ALERT "%s" TORTURE_FLAG
-		       " RCU barrier testing omitted from run.\n",
-		       torture_type);
+		pr_alert("%s" TORTURE_FLAG
+			 " Call or barrier ops missing for %s,\n",
+			 torture_type, cur_ops->name);
+		pr_alert("%s" TORTURE_FLAG
+			 " RCU barrier testing omitted from run.\n",
+			 torture_type);
 		return 0;
 	}
 	atomic_set(&barrier_cbs_count, 0);
@@ -1846,7 +1846,7 @@ rcu_torture_cleanup(void)
 	mutex_lock(&fullstop_mutex);
 	rcutorture_record_test_transition();
 	if (fullstop == FULLSTOP_SHUTDOWN) {
-		printk(KERN_WARNING /* but going down anyway, so... */
+		pr_warning(/* but going down anyway, so... */
 		       "Concurrent 'rmmod rcutorture' and shutdown illegal!\n");
 		mutex_unlock(&fullstop_mutex);
 		schedule_timeout_uninterruptible(10);
@@ -1970,17 +1970,17 @@ rcu_torture_init(void)
 			break;
 	}
 	if (i == ARRAY_SIZE(torture_ops)) {
-		printk(KERN_ALERT "rcu-torture: invalid torture type: \"%s\"\n",
-		       torture_type);
-		printk(KERN_ALERT "rcu-torture types:");
+		pr_alert("rcu-torture: invalid torture type: \"%s\"\n",
+			 torture_type);
+		pr_alert("rcu-torture types:");
 		for (i = 0; i < ARRAY_SIZE(torture_ops); i++)
-			printk(KERN_ALERT " %s", torture_ops[i]->name);
-		printk(KERN_ALERT "\n");
+			pr_alert(" %s", torture_ops[i]->name);
+		pr_alert("\n");
 		mutex_unlock(&fullstop_mutex);
 		return -EINVAL;
 	}
 	if (cur_ops->fqs == NULL && fqs_duration != 0) {
-		printk(KERN_ALERT "rcu-torture: ->fqs NULL and non-zero fqs_duration, fqs disabled.\n");
+		pr_alert("rcu-torture: ->fqs NULL and non-zero fqs_duration, fqs disabled.\n");
 		fqs_duration = 0;
 	}
 	if (cur_ops->init)
-- 
1.7.8

[PATCH tip/core/rcu 5/5] rcu: Prevent initialization race in rcutorture kthreads

[Paul E. McKenney]

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>

When you do something like "t = kthread_run(...)", it is possible that
the kthread will start running before the assignment to "t" happens.
If the child kthread expects to find a pointer to its task_struct in "t",
it will then be fatally disappointed.  This commit therefore switches
such cases to kthread_create() followed by wake_up_process(), guaranteeing
that the assignment happens before the child kthread starts running.

Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
 kernel/rcutorture.c |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
index 8ea2726..3e2adae 100644
--- a/kernel/rcutorture.c
+++ b/kernel/rcutorture.c
@@ -2028,14 +2028,15 @@ rcu_torture_init(void)
 	/* Start up the kthreads. */
 
 	VERBOSE_PRINTK_STRING("Creating rcu_torture_writer task");
-	writer_task = kthread_run(rcu_torture_writer, NULL,
-				  "rcu_torture_writer");
+	writer_task = kthread_create(rcu_torture_writer, NULL,
+				     "rcu_torture_writer");
 	if (IS_ERR(writer_task)) {
 		firsterr = PTR_ERR(writer_task);
 		VERBOSE_PRINTK_ERRSTRING("Failed to create writer");
 		writer_task = NULL;
 		goto unwind;
 	}
+	wake_up_process(writer_task);
 	fakewriter_tasks = kzalloc(nfakewriters * sizeof(fakewriter_tasks[0]),
 				   GFP_KERNEL);
 	if (fakewriter_tasks == NULL) {
@@ -2150,14 +2151,15 @@ rcu_torture_init(void)
 	}
 	if (shutdown_secs > 0) {
 		shutdown_time = jiffies + shutdown_secs * HZ;
-		shutdown_task = kthread_run(rcu_torture_shutdown, NULL,
-					    "rcu_torture_shutdown");
+		shutdown_task = kthread_create(rcu_torture_shutdown, NULL,
+					       "rcu_torture_shutdown");
 		if (IS_ERR(shutdown_task)) {
 			firsterr = PTR_ERR(shutdown_task);
 			VERBOSE_PRINTK_ERRSTRING("Failed to create shutdown");
 			shutdown_task = NULL;
 			goto unwind;
 		}
+		wake_up_process(shutdown_task);
 	}
 	i = rcu_torture_onoff_init();
 	if (i != 0) {
-- 
1.7.8

Re: [PATCH tip/core/rcu 0/5] Documentation and rcutorture changes

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:44:48AM -0700, Paul E. McKenney wrote:
> Hello!
> 
> This series covers changes to rcutorture and documentation updates.
> The individual patches in this series are as follows:
> 
> 1.	Update rcutorture default values so that casual rcutorture
> 	users will do more aggressive testing.
> 2.	Make rcutorture track CPU-hotplug latency statistics.
> 3.	Document SRCU's new-found ability to be used by offline and
> 	idle CPUs, and also emphasize SRCU's limitations.
> 4.	Use the new pr_*() interfaces in rcutorture.
> 5.	Prevent kthread-initialization races in rcutorture.
> 
> 							Thanx, Paul
> 
> ------------------------------------------------------------------------
> 
>  b/Documentation/RCU/checklist.txt |    6 +
>  b/Documentation/RCU/whatisRCU.txt |    9 +-
>  b/kernel/rcutorture.c             |    4 -
>  kernel/rcutorture.c               |  152 +++++++++++++++++++++++---------------
>  4 files changed, 108 insertions(+), 63 deletions(-)

Something seems wrong with this diffstat; how'd the b/ prefixes get
there, and why does it list kernel/rcutorture.c twice, once with and
once without?

- Josh Triplett

Re: [PATCH tip/core/rcu 1/5] rcu: Update rcutorture defaults

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:45:08AM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> 
> A number of new features have been added to rcutorture over the years, but
> the defaults have not been updated to include them.  This commit therefore
> turns on a couple of them that have proven helpful and trustworthy, namely
> periodic progress reports and testing of NO_HZ.
> 
> Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
>  kernel/rcutorture.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> index 25b1503..86315d3 100644
> --- a/kernel/rcutorture.c
> +++ b/kernel/rcutorture.c
> @@ -53,10 +53,10 @@ MODULE_AUTHOR("Paul E. McKenney <paulmck@us.ibm.com> and Josh Triplett <josh@fre
>  
>  static int nreaders = -1;	/* # reader threads, defaults to 2*ncpus */
>  static int nfakewriters = 4;	/* # fake writer threads */
> -static int stat_interval;	/* Interval between stats, in seconds. */
> +static int stat_interval = 60;	/* Interval between stats, in seconds. */
>  				/*  Defaults to "only at end of test". */

Need to remove this comment about the default.

>  static bool verbose;		/* Print more debug info. */
> -static bool test_no_idle_hz;	/* Test RCU's support for tickless idle CPUs. */
> +static bool test_no_idle_hz = 1; /* Test RCU support for tickless idle CPUs. */

s/1/true/

>  static int shuffle_interval = 3; /* Interval between shuffles (in sec)*/
>  static int stutter = 5;		/* Start/stop testing interval (in sec) */
>  static int irqreader = 1;	/* RCU readers from irq (timers). */
> -- 
> 1.7.8
> 

Re: [PATCH tip/core/rcu 2/5] rcu: Track CPU-hotplug duration statistics

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:45:09AM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> 
> Many rcutorture runs include CPU-hotplug operations in their stress
> testing.  This commit accumulates statistics on the durations of these
> operations in deference to the recent concern about the overhead and
> latency of these operations.

How many jiffies, on average, do these operations take?  Measuring these
using jiffies seems highly prone to repeated rounding error.

> Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
>  kernel/rcutorture.c |   42 +++++++++++++++++++++++++++++++++++++-----
>  1 files changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> index 86315d3..c6cf6ff 100644
> --- a/kernel/rcutorture.c
> +++ b/kernel/rcutorture.c
> @@ -176,8 +176,14 @@ static long n_rcu_torture_boosts;
>  static long n_rcu_torture_timers;
>  static long n_offline_attempts;
>  static long n_offline_successes;
> +static unsigned long sum_offline;
> +static int min_offline = -1;
> +static int max_offline;
>  static long n_online_attempts;
>  static long n_online_successes;
> +static unsigned long sum_online;
> +static int min_online = -1;
> +static int max_online;
>  static long n_barrier_attempts;
>  static long n_barrier_successes;
>  static struct list_head rcu_torture_removed;
> @@ -1214,11 +1220,13 @@ rcu_torture_printk(char *page)
>  		       n_rcu_torture_boost_failure,
>  		       n_rcu_torture_boosts,
>  		       n_rcu_torture_timers);
> -	cnt += sprintf(&page[cnt], "onoff: %ld/%ld:%ld/%ld ",
> -		       n_online_successes,
> -		       n_online_attempts,
> -		       n_offline_successes,
> -		       n_offline_attempts);
> +	cnt += sprintf(&page[cnt],
> +		       "onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
> +		       n_online_successes, n_online_attempts,
> +		       n_offline_successes, n_offline_attempts,
> +		       min_online, max_online,
> +		       min_offline, max_offline,
> +		       sum_online, sum_offline, HZ);
>  	cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
>  		       n_barrier_successes,
>  		       n_barrier_attempts,
> @@ -1490,8 +1498,10 @@ static int __cpuinit
>  rcu_torture_onoff(void *arg)
>  {
>  	int cpu;
> +	unsigned long delta;
>  	int maxcpu = -1;
>  	DEFINE_RCU_RANDOM(rand);
> +	unsigned long starttime;
>  
>  	VERBOSE_PRINTK_STRING("rcu_torture_onoff task started");
>  	for_each_online_cpu(cpu)
> @@ -1509,6 +1519,7 @@ rcu_torture_onoff(void *arg)
>  				printk(KERN_ALERT "%s" TORTURE_FLAG
>  				       "rcu_torture_onoff task: offlining %d\n",
>  				       torture_type, cpu);
> +			starttime = jiffies;
>  			n_offline_attempts++;
>  			if (cpu_down(cpu) == 0) {
>  				if (verbose)
> @@ -1516,12 +1527,23 @@ rcu_torture_onoff(void *arg)
>  					       "rcu_torture_onoff task: offlined %d\n",
>  					       torture_type, cpu);
>  				n_offline_successes++;
> +				delta = jiffies - starttime;
> +				sum_offline += delta;
> +				if (min_offline < 0) {
> +					min_offline = delta;
> +					max_offline = delta;
> +				}
> +				if (min_offline > delta)
> +					min_offline = delta;
> +				if (max_offline < delta)
> +					max_offline = delta;
>  			}
>  		} else if (cpu_is_hotpluggable(cpu)) {
>  			if (verbose)
>  				printk(KERN_ALERT "%s" TORTURE_FLAG
>  				       "rcu_torture_onoff task: onlining %d\n",
>  				       torture_type, cpu);
> +			starttime = jiffies;
>  			n_online_attempts++;
>  			if (cpu_up(cpu) == 0) {
>  				if (verbose)
> @@ -1529,6 +1551,16 @@ rcu_torture_onoff(void *arg)
>  					       "rcu_torture_onoff task: onlined %d\n",
>  					       torture_type, cpu);
>  				n_online_successes++;
> +				delta = jiffies - starttime;
> +				sum_online += delta;
> +				if (min_online < 0) {
> +					min_online = delta;
> +					max_online = delta;
> +				}
> +				if (min_online > delta)
> +					min_online = delta;
> +				if (max_online < delta)
> +					max_online = delta;
>  			}
>  		}
>  		schedule_timeout_interruptible(onoff_interval * HZ);
> -- 
> 1.7.8
> 

Re: [PATCH tip/core/rcu 3/5] rcu: Document SRCU dead-CPU capabilities, emphasize read-side limits

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:45:10AM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> The current documentation did not help someone grepping for SRCU to
> learn that disabling preemption is not a replacement for srcu_read_lock(),
> so upgrade the documentation to bring this out, not just for SRCU,
> but also for RCU-bh.  Also document the fact that SRCU readers are
> respected on CPUs executing in user mode, idle CPUs, and even on
> offline CPUs.
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

>  Documentation/RCU/checklist.txt |    6 ++++++
>  Documentation/RCU/whatisRCU.txt |    9 +++++++--
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
> index fc103d7..cdb20d4 100644
> --- a/Documentation/RCU/checklist.txt
> +++ b/Documentation/RCU/checklist.txt
> @@ -310,6 +310,12 @@ over a rather long period of time, but improvements are always welcome!
>  	code under the influence of preempt_disable(), you instead
>  	need to use synchronize_irq() or synchronize_sched().
>  
> +	This same limitation also applies to synchronize_rcu_bh()
> +	and synchronize_srcu(), as well as to the asynchronous and
> +	expedited forms of the three primitives, namely call_rcu(),
> +	call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(),
> +	synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited().
> +
>  12.	Any lock acquired by an RCU callback must be acquired elsewhere
>  	with softirq disabled, e.g., via spin_lock_irqsave(),
>  	spin_lock_bh(), etc.  Failing to disable irq on a given
> diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
> index 69ee188..bf0f6de 100644
> --- a/Documentation/RCU/whatisRCU.txt
> +++ b/Documentation/RCU/whatisRCU.txt
> @@ -873,7 +873,7 @@ d.	Do you need to treat NMI handlers, hardirq handlers,
>  	and code segments with preemption disabled (whether
>  	via preempt_disable(), local_irq_save(), local_bh_disable(),
>  	or some other mechanism) as if they were explicit RCU readers?
> -	If so, you need RCU-sched.
> +	If so, RCU-sched is the only choice that will work for you.
>  
>  e.	Do you need RCU grace periods to complete even in the face
>  	of softirq monopolization of one or more of the CPUs?  For
> @@ -884,7 +884,12 @@ f.	Is your workload too update-intensive for normal use of
>  	RCU, but inappropriate for other synchronization mechanisms?
>  	If so, consider SLAB_DESTROY_BY_RCU.  But please be careful!
>  
> -g.	Otherwise, use RCU.
> +g.	Do you need read-side critical sections that are respected
> +	even though they are in the middle of the idle loop, during
> +	user-mode execution, or on an offlined CPU?  If so, SRCU is the
> +	only choice that will work for you.
> +
> +h.	Otherwise, use RCU.
>  
>  Of course, this all assumes that you have determined that RCU is in fact
>  the right tool for your job.
> -- 
> 1.7.8
> 

Re: [PATCH tip/core/rcu 4/5] rcu: Switch rcutorture to pr_alert() and friends

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:45:11AM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> Drop a few characters by switching kernel/rcutorture.c from
> "printk(KERN_ALERT" to "pr_alert(".
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

How about setting pr_fmt as well, and dropping the various "rcutorture:"
prefixes?  You'd still potentially want to add the torture type, though
you could do that with pr_fmt as well.

In any case:

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

Re: [PATCH tip/core/rcu 5/5] rcu: Prevent initialization race in rcutorture kthreads

[Josh Triplett]

On Thu, Aug 30, 2012 at 11:45:12AM -0700, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> When you do something like "t = kthread_run(...)", it is possible that
> the kthread will start running before the assignment to "t" happens.
> If the child kthread expects to find a pointer to its task_struct in "t",
> it will then be fatally disappointed.  This commit therefore switches
> such cases to kthread_create() followed by wake_up_process(), guaranteeing
> that the assignment happens before the child kthread starts running.
> 
> Reported-by: Fengguang Wu <fengguang.wu@intel.com>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Seems like you should go ahead and make this change for all the threads,
not just two of them.  A simple wrapper around kthread_run, taking a
struct task_struct ** to write to, would make this much simpler.  Such a
wrapper could also return an error code directly (for use in firsterr),
write NULL to the pointer on error, and perhaps print an error message,
which would remove most of the boilerplate currently duplicated for
every thread creation.

Arguably, all of those except the error message printing would make
sense as changes to kthread_run itself, but that's another patch. :)

- Josh Triplett

Re: [PATCH tip/core/rcu 2/5] rcu: Track CPU-hotplug duration statistics

[Paul E. McKenney]

On Thu, Aug 30, 2012 at 12:00:18PM -0700, Josh Triplett wrote:
> On Thu, Aug 30, 2012 at 11:45:09AM -0700, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > 
> > Many rcutorture runs include CPU-hotplug operations in their stress
> > testing.  This commit accumulates statistics on the durations of these
> > operations in deference to the recent concern about the overhead and
> > latency of these operations.
> 
> How many jiffies, on average, do these operations take?  Measuring these
> using jiffies seems highly prone to repeated rounding error.

On my laptop, 30-140 depending on what hotplug patches I have in place.
Some users have reported as few as 2-3 jiffies, but they don't use
rcutorture.

I eagerly look forward to the time when I need to change the timebase for
my own use.  ;-)

							Thanx, Paul

> > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > ---
> >  kernel/rcutorture.c |   42 +++++++++++++++++++++++++++++++++++++-----
> >  1 files changed, 37 insertions(+), 5 deletions(-)
> > 
> > diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> > index 86315d3..c6cf6ff 100644
> > --- a/kernel/rcutorture.c
> > +++ b/kernel/rcutorture.c
> > @@ -176,8 +176,14 @@ static long n_rcu_torture_boosts;
> >  static long n_rcu_torture_timers;
> >  static long n_offline_attempts;
> >  static long n_offline_successes;
> > +static unsigned long sum_offline;
> > +static int min_offline = -1;
> > +static int max_offline;
> >  static long n_online_attempts;
> >  static long n_online_successes;
> > +static unsigned long sum_online;
> > +static int min_online = -1;
> > +static int max_online;
> >  static long n_barrier_attempts;
> >  static long n_barrier_successes;
> >  static struct list_head rcu_torture_removed;
> > @@ -1214,11 +1220,13 @@ rcu_torture_printk(char *page)
> >  		       n_rcu_torture_boost_failure,
> >  		       n_rcu_torture_boosts,
> >  		       n_rcu_torture_timers);
> > -	cnt += sprintf(&page[cnt], "onoff: %ld/%ld:%ld/%ld ",
> > -		       n_online_successes,
> > -		       n_online_attempts,
> > -		       n_offline_successes,
> > -		       n_offline_attempts);
> > +	cnt += sprintf(&page[cnt],
> > +		       "onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
> > +		       n_online_successes, n_online_attempts,
> > +		       n_offline_successes, n_offline_attempts,
> > +		       min_online, max_online,
> > +		       min_offline, max_offline,
> > +		       sum_online, sum_offline, HZ);
> >  	cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
> >  		       n_barrier_successes,
> >  		       n_barrier_attempts,
> > @@ -1490,8 +1498,10 @@ static int __cpuinit
> >  rcu_torture_onoff(void *arg)
> >  {
> >  	int cpu;
> > +	unsigned long delta;
> >  	int maxcpu = -1;
> >  	DEFINE_RCU_RANDOM(rand);
> > +	unsigned long starttime;
> >  
> >  	VERBOSE_PRINTK_STRING("rcu_torture_onoff task started");
> >  	for_each_online_cpu(cpu)
> > @@ -1509,6 +1519,7 @@ rcu_torture_onoff(void *arg)
> >  				printk(KERN_ALERT "%s" TORTURE_FLAG
> >  				       "rcu_torture_onoff task: offlining %d\n",
> >  				       torture_type, cpu);
> > +			starttime = jiffies;
> >  			n_offline_attempts++;
> >  			if (cpu_down(cpu) == 0) {
> >  				if (verbose)
> > @@ -1516,12 +1527,23 @@ rcu_torture_onoff(void *arg)
> >  					       "rcu_torture_onoff task: offlined %d\n",
> >  					       torture_type, cpu);
> >  				n_offline_successes++;
> > +				delta = jiffies - starttime;
> > +				sum_offline += delta;
> > +				if (min_offline < 0) {
> > +					min_offline = delta;
> > +					max_offline = delta;
> > +				}
> > +				if (min_offline > delta)
> > +					min_offline = delta;
> > +				if (max_offline < delta)
> > +					max_offline = delta;
> >  			}
> >  		} else if (cpu_is_hotpluggable(cpu)) {
> >  			if (verbose)
> >  				printk(KERN_ALERT "%s" TORTURE_FLAG
> >  				       "rcu_torture_onoff task: onlining %d\n",
> >  				       torture_type, cpu);
> > +			starttime = jiffies;
> >  			n_online_attempts++;
> >  			if (cpu_up(cpu) == 0) {
> >  				if (verbose)
> > @@ -1529,6 +1551,16 @@ rcu_torture_onoff(void *arg)
> >  					       "rcu_torture_onoff task: onlined %d\n",
> >  					       torture_type, cpu);
> >  				n_online_successes++;
> > +				delta = jiffies - starttime;
> > +				sum_online += delta;
> > +				if (min_online < 0) {
> > +					min_online = delta;
> > +					max_online = delta;
> > +				}
> > +				if (min_online > delta)
> > +					min_online = delta;
> > +				if (max_online < delta)
> > +					max_online = delta;
> >  			}
> >  		}
> >  		schedule_timeout_interruptible(onoff_interval * HZ);
> > -- 
> > 1.7.8
> > 
> 

Re: [PATCH tip/core/rcu 1/5] rcu: Update rcutorture defaults

[Paul E. McKenney]

On Thu, Aug 30, 2012 at 11:57:05AM -0700, Josh Triplett wrote:
> On Thu, Aug 30, 2012 at 11:45:08AM -0700, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > 
> > A number of new features have been added to rcutorture over the years, but
> > the defaults have not been updated to include them.  This commit therefore
> > turns on a couple of them that have proven helpful and trustworthy, namely
> > periodic progress reports and testing of NO_HZ.
> > 
> > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > ---
> >  kernel/rcutorture.c |    4 ++--
> >  1 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> > index 25b1503..86315d3 100644
> > --- a/kernel/rcutorture.c
> > +++ b/kernel/rcutorture.c
> > @@ -53,10 +53,10 @@ MODULE_AUTHOR("Paul E. McKenney <paulmck@us.ibm.com> and Josh Triplett <josh@fre
> >  
> >  static int nreaders = -1;	/* # reader threads, defaults to 2*ncpus */
> >  static int nfakewriters = 4;	/* # fake writer threads */
> > -static int stat_interval;	/* Interval between stats, in seconds. */
> > +static int stat_interval = 60;	/* Interval between stats, in seconds. */
> >  				/*  Defaults to "only at end of test". */
> 
> Need to remove this comment about the default.

Good catch!  I have replaced it with "Zero means "only at end of test".

> >  static bool verbose;		/* Print more debug info. */
> > -static bool test_no_idle_hz;	/* Test RCU's support for tickless idle CPUs. */
> > +static bool test_no_idle_hz = 1; /* Test RCU support for tickless idle CPUs. */
> 
> s/1/true/

Good point, fixed.

Thank you for looking this over!

							Thanx, Paul

> >  static int shuffle_interval = 3; /* Interval between shuffles (in sec)*/
> >  static int stutter = 5;		/* Start/stop testing interval (in sec) */
> >  static int irqreader = 1;	/* RCU readers from irq (timers). */
> > -- 
> > 1.7.8
> > 
> 

Re: [PATCH tip/core/rcu 0/5] Documentation and rcutorture changes

[Paul E. McKenney]

On Thu, Aug 30, 2012 at 11:56:09AM -0700, Josh Triplett wrote:
> On Thu, Aug 30, 2012 at 11:44:48AM -0700, Paul E. McKenney wrote:
> > Hello!
> > 
> > This series covers changes to rcutorture and documentation updates.
> > The individual patches in this series are as follows:
> > 
> > 1.	Update rcutorture default values so that casual rcutorture
> > 	users will do more aggressive testing.
> > 2.	Make rcutorture track CPU-hotplug latency statistics.
> > 3.	Document SRCU's new-found ability to be used by offline and
> > 	idle CPUs, and also emphasize SRCU's limitations.
> > 4.	Use the new pr_*() interfaces in rcutorture.
> > 5.	Prevent kthread-initialization races in rcutorture.
> > 
> > 							Thanx, Paul
> > 
> > ------------------------------------------------------------------------
> > 
> >  b/Documentation/RCU/checklist.txt |    6 +
> >  b/Documentation/RCU/whatisRCU.txt |    9 +-
> >  b/kernel/rcutorture.c             |    4 -
> >  kernel/rcutorture.c               |  152 +++++++++++++++++++++++---------------
> >  4 files changed, 108 insertions(+), 63 deletions(-)
> 
> Something seems wrong with this diffstat; how'd the b/ prefixes get
> there, and why does it list kernel/rcutorture.c twice, once with and
> once without?

Hmmm...  It seems quite reproducible.  I did the usual git-format-patch
and ran the resulting set of patches through diffstat.  I seem to have a
broken diffstat...

However, git diff --stat v3.6-rc1..hotplug.2012.08.28a generates the
following:

 kernel/rcutree.c       |   93 +++++++++++-------------------------------------
 kernel/rcutree.h       |    3 --
 kernel/rcutree_trace.c |    4 +-
 kernel/sched/core.c    |   41 ++++++++++-----------
 4 files changed, 43 insertions(+), 98 deletions(-)

Which does look much better.

							Thanx, Paul

Re: [PATCH tip/core/rcu 0/5] Documentation and rcutorture changes

[Josh Triplett]

On Thu, Aug 30, 2012 at 02:46:03PM -0700, Paul E. McKenney wrote:
> On Thu, Aug 30, 2012 at 11:56:09AM -0700, Josh Triplett wrote:
> > On Thu, Aug 30, 2012 at 11:44:48AM -0700, Paul E. McKenney wrote:
> > > Hello!
> > > 
> > > This series covers changes to rcutorture and documentation updates.
> > > The individual patches in this series are as follows:
> > > 
> > > 1.	Update rcutorture default values so that casual rcutorture
> > > 	users will do more aggressive testing.
> > > 2.	Make rcutorture track CPU-hotplug latency statistics.
> > > 3.	Document SRCU's new-found ability to be used by offline and
> > > 	idle CPUs, and also emphasize SRCU's limitations.
> > > 4.	Use the new pr_*() interfaces in rcutorture.
> > > 5.	Prevent kthread-initialization races in rcutorture.
> > > 
> > > 							Thanx, Paul
> > > 
> > > ------------------------------------------------------------------------
> > > 
> > >  b/Documentation/RCU/checklist.txt |    6 +
> > >  b/Documentation/RCU/whatisRCU.txt |    9 +-
> > >  b/kernel/rcutorture.c             |    4 -
> > >  kernel/rcutorture.c               |  152 +++++++++++++++++++++++---------------
> > >  4 files changed, 108 insertions(+), 63 deletions(-)
> > 
> > Something seems wrong with this diffstat; how'd the b/ prefixes get
> > there, and why does it list kernel/rcutorture.c twice, once with and
> > once without?
> 
> Hmmm...  It seems quite reproducible.  I did the usual git-format-patch
> and ran the resulting set of patches through diffstat.  I seem to have a
> broken diffstat...
> 
> However, git diff --stat v3.6-rc1..hotplug.2012.08.28a generates the
> following:
> 
>  kernel/rcutree.c       |   93 +++++++++++-------------------------------------
>  kernel/rcutree.h       |    3 --
>  kernel/rcutree_trace.c |    4 +-
>  kernel/sched/core.c    |   41 ++++++++++-----------
>  4 files changed, 43 insertions(+), 98 deletions(-)
> 
> Which does look much better.

You might try generating your cover letter template via git format-patch
--cover-letter, which will automatically give you a list of patches and
a git-produced diffstat; much easier than trying to format a cover
letter by hand.  Meanwhile, you might consider sending your patches as a
bug report to diffstat upstream: Thomas E. Dickey
<dickey@invisible-island.net>.

- Josh Triplett

Re: [PATCH tip/core/rcu 2/5] rcu: Track CPU-hotplug duration statistics

[Josh Triplett]

On Thu, Aug 30, 2012 at 01:38:42PM -0700, Paul E. McKenney wrote:
> On Thu, Aug 30, 2012 at 12:00:18PM -0700, Josh Triplett wrote:
> > On Thu, Aug 30, 2012 at 11:45:09AM -0700, Paul E. McKenney wrote:
> > > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > > 
> > > Many rcutorture runs include CPU-hotplug operations in their stress
> > > testing.  This commit accumulates statistics on the durations of these
> > > operations in deference to the recent concern about the overhead and
> > > latency of these operations.
> > 
> > How many jiffies, on average, do these operations take?  Measuring these
> > using jiffies seems highly prone to repeated rounding error.
> 
> On my laptop, 30-140 depending on what hotplug patches I have in place.
> Some users have reported as few as 2-3 jiffies, but they don't use
> rcutorture.
> 
> I eagerly look forward to the time when I need to change the timebase for
> my own use.  ;-)

Fair enough.  In that case, this seems precise enough for the purpose it
serves.

> > > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

> > > ---
> > >  kernel/rcutorture.c |   42 +++++++++++++++++++++++++++++++++++++-----
> > >  1 files changed, 37 insertions(+), 5 deletions(-)
> > > 
> > > diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> > > index 86315d3..c6cf6ff 100644
> > > --- a/kernel/rcutorture.c
> > > +++ b/kernel/rcutorture.c
> > > @@ -176,8 +176,14 @@ static long n_rcu_torture_boosts;
> > >  static long n_rcu_torture_timers;
> > >  static long n_offline_attempts;
> > >  static long n_offline_successes;
> > > +static unsigned long sum_offline;
> > > +static int min_offline = -1;
> > > +static int max_offline;
> > >  static long n_online_attempts;
> > >  static long n_online_successes;
> > > +static unsigned long sum_online;
> > > +static int min_online = -1;
> > > +static int max_online;
> > >  static long n_barrier_attempts;
> > >  static long n_barrier_successes;
> > >  static struct list_head rcu_torture_removed;
> > > @@ -1214,11 +1220,13 @@ rcu_torture_printk(char *page)
> > >  		       n_rcu_torture_boost_failure,
> > >  		       n_rcu_torture_boosts,
> > >  		       n_rcu_torture_timers);
> > > -	cnt += sprintf(&page[cnt], "onoff: %ld/%ld:%ld/%ld ",
> > > -		       n_online_successes,
> > > -		       n_online_attempts,
> > > -		       n_offline_successes,
> > > -		       n_offline_attempts);
> > > +	cnt += sprintf(&page[cnt],
> > > +		       "onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
> > > +		       n_online_successes, n_online_attempts,
> > > +		       n_offline_successes, n_offline_attempts,
> > > +		       min_online, max_online,
> > > +		       min_offline, max_offline,
> > > +		       sum_online, sum_offline, HZ);
> > >  	cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
> > >  		       n_barrier_successes,
> > >  		       n_barrier_attempts,
> > > @@ -1490,8 +1498,10 @@ static int __cpuinit
> > >  rcu_torture_onoff(void *arg)
> > >  {
> > >  	int cpu;
> > > +	unsigned long delta;
> > >  	int maxcpu = -1;
> > >  	DEFINE_RCU_RANDOM(rand);
> > > +	unsigned long starttime;
> > >  
> > >  	VERBOSE_PRINTK_STRING("rcu_torture_onoff task started");
> > >  	for_each_online_cpu(cpu)
> > > @@ -1509,6 +1519,7 @@ rcu_torture_onoff(void *arg)
> > >  				printk(KERN_ALERT "%s" TORTURE_FLAG
> > >  				       "rcu_torture_onoff task: offlining %d\n",
> > >  				       torture_type, cpu);
> > > +			starttime = jiffies;
> > >  			n_offline_attempts++;
> > >  			if (cpu_down(cpu) == 0) {
> > >  				if (verbose)
> > > @@ -1516,12 +1527,23 @@ rcu_torture_onoff(void *arg)
> > >  					       "rcu_torture_onoff task: offlined %d\n",
> > >  					       torture_type, cpu);
> > >  				n_offline_successes++;
> > > +				delta = jiffies - starttime;
> > > +				sum_offline += delta;
> > > +				if (min_offline < 0) {
> > > +					min_offline = delta;
> > > +					max_offline = delta;
> > > +				}
> > > +				if (min_offline > delta)
> > > +					min_offline = delta;
> > > +				if (max_offline < delta)
> > > +					max_offline = delta;
> > >  			}
> > >  		} else if (cpu_is_hotpluggable(cpu)) {
> > >  			if (verbose)
> > >  				printk(KERN_ALERT "%s" TORTURE_FLAG
> > >  				       "rcu_torture_onoff task: onlining %d\n",
> > >  				       torture_type, cpu);
> > > +			starttime = jiffies;
> > >  			n_online_attempts++;
> > >  			if (cpu_up(cpu) == 0) {
> > >  				if (verbose)
> > > @@ -1529,6 +1551,16 @@ rcu_torture_onoff(void *arg)
> > >  					       "rcu_torture_onoff task: onlined %d\n",
> > >  					       torture_type, cpu);
> > >  				n_online_successes++;
> > > +				delta = jiffies - starttime;
> > > +				sum_online += delta;
> > > +				if (min_online < 0) {
> > > +					min_online = delta;
> > > +					max_online = delta;
> > > +				}
> > > +				if (min_online > delta)
> > > +					min_online = delta;
> > > +				if (max_online < delta)
> > > +					max_online = delta;
> > >  			}
> > >  		}
> > >  		schedule_timeout_interruptible(onoff_interval * HZ);
> > > -- 
> > > 1.7.8
> > > 
> > 
> 

Re: [PATCH tip/core/rcu 1/5] rcu: Update rcutorture defaults

[Josh Triplett]

On Thu, Aug 30, 2012 at 02:35:36PM -0700, Paul E. McKenney wrote:
> On Thu, Aug 30, 2012 at 11:57:05AM -0700, Josh Triplett wrote:
> > On Thu, Aug 30, 2012 at 11:45:08AM -0700, Paul E. McKenney wrote:
> > > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> > > 
> > > A number of new features have been added to rcutorture over the years, but
> > > the defaults have not been updated to include them.  This commit therefore
> > > turns on a couple of them that have proven helpful and trustworthy, namely
> > > periodic progress reports and testing of NO_HZ.
> > > 
> > > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > > ---
> > >  kernel/rcutorture.c |    4 ++--
> > >  1 files changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
> > > index 25b1503..86315d3 100644
> > > --- a/kernel/rcutorture.c
> > > +++ b/kernel/rcutorture.c
> > > @@ -53,10 +53,10 @@ MODULE_AUTHOR("Paul E. McKenney <paulmck@us.ibm.com> and Josh Triplett <josh@fre
> > >  
> > >  static int nreaders = -1;	/* # reader threads, defaults to 2*ncpus */
> > >  static int nfakewriters = 4;	/* # fake writer threads */
> > > -static int stat_interval;	/* Interval between stats, in seconds. */
> > > +static int stat_interval = 60;	/* Interval between stats, in seconds. */
> > >  				/*  Defaults to "only at end of test". */
> > 
> > Need to remove this comment about the default.
> 
> Good catch!  I have replaced it with "Zero means "only at end of test".

Good point, you definitely still need to document what zero means.

> > >  static bool verbose;		/* Print more debug info. */
> > > -static bool test_no_idle_hz;	/* Test RCU's support for tickless idle CPUs. */
> > > +static bool test_no_idle_hz = 1; /* Test RCU support for tickless idle CPUs. */
> > 
> > s/1/true/
> 
> Good point, fixed.
> 
> Thank you for looking this over!

With those two fixes:

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

Re: [PATCH tip/core/rcu 3/5] rcu: Document SRCU dead-CPU capabilities, emphasize read-side limits

[Lai Jiangshan]

On 08/31/2012 02:45 AM, Paul E. McKenney wrote:
> From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> 
> The current documentation did not help someone grepping for SRCU to
> learn that disabling preemption is not a replacement for srcu_read_lock(),
> so upgrade the documentation to bring this out, not just for SRCU,
> but also for RCU-bh.  Also document the fact that SRCU readers are
> respected on CPUs executing in user mode, idle CPUs, and even on
> offline CPUs.
> 
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Good.  (Sorry, I'm late.)

Reviewed-by: Lai Jiangshan <laijs@cn.fujitsu.com>

> ---
>  Documentation/RCU/checklist.txt |    6 ++++++
>  Documentation/RCU/whatisRCU.txt |    9 +++++++--
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
> index fc103d7..cdb20d4 100644
> --- a/Documentation/RCU/checklist.txt
> +++ b/Documentation/RCU/checklist.txt
> @@ -310,6 +310,12 @@ over a rather long period of time, but improvements are always welcome!
>  	code under the influence of preempt_disable(), you instead
>  	need to use synchronize_irq() or synchronize_sched().
>  
> +	This same limitation also applies to synchronize_rcu_bh()
> +	and synchronize_srcu(), as well as to the asynchronous and
> +	expedited forms of the three primitives, namely call_rcu(),
> +	call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(),
> +	synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited().
> +
>  12.	Any lock acquired by an RCU callback must be acquired elsewhere
>  	with softirq disabled, e.g., via spin_lock_irqsave(),
>  	spin_lock_bh(), etc.  Failing to disable irq on a given
> diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
> index 69ee188..bf0f6de 100644
> --- a/Documentation/RCU/whatisRCU.txt
> +++ b/Documentation/RCU/whatisRCU.txt
> @@ -873,7 +873,7 @@ d.	Do you need to treat NMI handlers, hardirq handlers,
>  	and code segments with preemption disabled (whether
>  	via preempt_disable(), local_irq_save(), local_bh_disable(),
>  	or some other mechanism) as if they were explicit RCU readers?
> -	If so, you need RCU-sched.
> +	If so, RCU-sched is the only choice that will work for you.
>  
>  e.	Do you need RCU grace periods to complete even in the face
>  	of softirq monopolization of one or more of the CPUs?  For
> @@ -884,7 +884,12 @@ f.	Is your workload too update-intensive for normal use of
>  	RCU, but inappropriate for other synchronization mechanisms?
>  	If so, consider SLAB_DESTROY_BY_RCU.  But please be careful!
>  
> -g.	Otherwise, use RCU.
> +g.	Do you need read-side critical sections that are respected
> +	even though they are in the middle of the idle loop, during
> +	user-mode execution, or on an offlined CPU?  If so, SRCU is the
> +	only choice that will work for you.
> +
> +h.	Otherwise, use RCU.
>  
>  Of course, this all assumes that you have determined that RCU is in fact
>  the right tool for your job.

Re: [PATCH tip/core/rcu 3/5] rcu: Document SRCU dead-CPU capabilities, emphasize read-side limits

[Paul E. McKenney]

On Tue, Sep 18, 2012 at 05:12:16PM +0800, Lai Jiangshan wrote:
> On 08/31/2012 02:45 AM, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > 
> > The current documentation did not help someone grepping for SRCU to
> > learn that disabling preemption is not a replacement for srcu_read_lock(),
> > so upgrade the documentation to bring this out, not just for SRCU,
> > but also for RCU-bh.  Also document the fact that SRCU readers are
> > respected on CPUs executing in user mode, idle CPUs, and even on
> > offline CPUs.
> > 
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> 
> Good.  (Sorry, I'm late.)

But, as it turns out, not too late.  ;-)

Thank you for the review!

							Thanx, Paul

> Reviewed-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> 
> > ---
> >  Documentation/RCU/checklist.txt |    6 ++++++
> >  Documentation/RCU/whatisRCU.txt |    9 +++++++--
> >  2 files changed, 13 insertions(+), 2 deletions(-)
> > 
> > diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
> > index fc103d7..cdb20d4 100644
> > --- a/Documentation/RCU/checklist.txt
> > +++ b/Documentation/RCU/checklist.txt
> > @@ -310,6 +310,12 @@ over a rather long period of time, but improvements are always welcome!
> >  	code under the influence of preempt_disable(), you instead
> >  	need to use synchronize_irq() or synchronize_sched().
> >  
> > +	This same limitation also applies to synchronize_rcu_bh()
> > +	and synchronize_srcu(), as well as to the asynchronous and
> > +	expedited forms of the three primitives, namely call_rcu(),
> > +	call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(),
> > +	synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited().
> > +
> >  12.	Any lock acquired by an RCU callback must be acquired elsewhere
> >  	with softirq disabled, e.g., via spin_lock_irqsave(),
> >  	spin_lock_bh(), etc.  Failing to disable irq on a given
> > diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
> > index 69ee188..bf0f6de 100644
> > --- a/Documentation/RCU/whatisRCU.txt
> > +++ b/Documentation/RCU/whatisRCU.txt
> > @@ -873,7 +873,7 @@ d.	Do you need to treat NMI handlers, hardirq handlers,
> >  	and code segments with preemption disabled (whether
> >  	via preempt_disable(), local_irq_save(), local_bh_disable(),
> >  	or some other mechanism) as if they were explicit RCU readers?
> > -	If so, you need RCU-sched.
> > +	If so, RCU-sched is the only choice that will work for you.
> >  
> >  e.	Do you need RCU grace periods to complete even in the face
> >  	of softirq monopolization of one or more of the CPUs?  For
> > @@ -884,7 +884,12 @@ f.	Is your workload too update-intensive for normal use of
> >  	RCU, but inappropriate for other synchronization mechanisms?
> >  	If so, consider SLAB_DESTROY_BY_RCU.  But please be careful!
> >  
> > -g.	Otherwise, use RCU.
> > +g.	Do you need read-side critical sections that are respected
> > +	even though they are in the middle of the idle loop, during
> > +	user-mode execution, or on an offlined CPU?  If so, SRCU is the
> > +	only choice that will work for you.
> > +
> > +h.	Otherwise, use RCU.
> >  
> >  Of course, this all assumes that you have determined that RCU is in fact
> >  the right tool for your job.
>