* [PATCH tip/core/rcu 0/2] rcu: more review feedback
@ 2009-09-27 6:47 Paul E. McKenney
2009-09-27 6:49 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-27 6:47 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
This patchset applies the last of the review comments:
o Whitespace fixes, updated comments, and code movement.
o Improve an existing abstraction and introduce two additional
abstractions for traversing the rcu_node tree.
Issues remaining:
o I can force hangs given sufficient rcutorture testing in
conjunction with random CPU-hotplug operations. I am working
on a simpler interface between RCU and CPU hotplug that I
believe will clear this up.
o TREE_PREEMPT_RCU currently uses the TREE_RCU variant of
synchronize_rcu_expedited, which is wrong. I am working on
an expedited variant for TREE_PREEMPT_RCU, but will fall
back on synchronize_rcu() for the short term if it gives me
much trouble.
o The IPI hyperactivity from call_rcu() noted by Nick Piggin.
I believe I have identified a clean design for a fix for this.
o Make RCU less dependent on IPIs for forcing grace periods
(but this might be best deferred to 2.6.33).
o RCU priority boosting (2.6.33!).
o TINY_PREEMPT_RCU (2.6.33!).
o Squeeze another 400 bytes or so out of TINY_RCU (2.6.33!).
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3
2009-09-27 6:47 [PATCH tip/core/rcu 0/2] rcu: more review feedback Paul E. McKenney
@ 2009-09-27 6:49 ` Paul E. McKenney
2009-09-27 15:41 ` Josh Triplett
2009-09-27 6:49 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
2 siblings, 1 reply; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-27 6:49 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells,
Paul E. McKenney
Whitespace fixes, updated comments, and trivial code movement.
o Fix whitespace error in RCU_HEAD_INIT()
o Move "So where is rcu_write_lock()" comment so that it does
not come between the rcu_read_unlock() header comment and
the rcu_read_unlock() definition.
o Move the module_param statements for blimit, qhimark, and qlowmark
to immediately follow the corresponding definitions.
o In __rcu_offline_cpu(), move the assignment to rdp_me inside the
"if" statement, given that rdp_me is not used outside of that "if"
statement.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
include/linux/rcupdate.h | 14 +++++++-------
kernel/rcutree.c | 10 +++++-----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index b2f1e10..6dd71fa 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -83,7 +83,7 @@ extern int rcu_scheduler_active;
#error "Unknown RCU implementation specified to kernel configuration"
#endif
-#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
+#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
#define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
#define INIT_RCU_HEAD(ptr) do { \
(ptr)->next = NULL; (ptr)->func = NULL; \
@@ -135,12 +135,6 @@ static inline void rcu_read_lock(void)
rcu_read_acquire();
}
-/**
- * rcu_read_unlock - marks the end of an RCU read-side critical section.
- *
- * See rcu_read_lock() for more information.
- */
-
/*
* So where is rcu_write_lock()? It does not exist, as there is no
* way for writers to lock out RCU readers. This is a feature, not
@@ -150,6 +144,12 @@ static inline void rcu_read_lock(void)
* used as well. RCU does not care how the writers keep out of each
* others' way, as long as they do so.
*/
+
+/**
+ * rcu_read_unlock - marks the end of an RCU read-side critical section.
+ *
+ * See rcu_read_lock() for more information.
+ */
static inline void rcu_read_unlock(void)
{
rcu_read_release();
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 45eb2d1..1ccdd5b 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -121,6 +121,10 @@ static int blimit = 10; /* Maximum callbacks per softirq. */
static int qhimark = 10000; /* If this many pending, ignore blimit. */
static int qlowmark = 100; /* Once only this many pending, use blimit. */
+module_param(blimit, int, 0);
+module_param(qhimark, int, 0);
+module_param(qlowmark, int, 0);
+
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
static int rcu_pending(int cpu);
@@ -873,8 +877,8 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)
* indefinitely delay callbacks, you have far worse things to
* be worrying about.
*/
- rdp_me = rsp->rda[smp_processor_id()];
if (rdp->nxtlist != NULL) {
+ rdp_me = rsp->rda[smp_processor_id()];
*rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxtlist;
rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
rdp->nxtlist = NULL;
@@ -1570,7 +1574,3 @@ void __init __rcu_init(void)
}
#include "rcutree_plugin.h"
-
-module_param(blimit, int, 0);
-module_param(qhimark, int, 0);
-module_param(qlowmark, int, 0);
--
1.5.2.5
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4
2009-09-27 6:47 [PATCH tip/core/rcu 0/2] rcu: more review feedback Paul E. McKenney
2009-09-27 6:49 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
@ 2009-09-27 6:49 ` Paul E. McKenney
2009-09-27 15:31 ` Josh Triplett
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
2 siblings, 1 reply; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-27 6:49 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells,
Paul E. McKenney
From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
These issues identified during an old-fashioned face-to-face code
review extending over many hours. This group improves an existing
abstraction and introduces two new ones.
o Make RCU_INIT_FLAVOR() declare its own variables, removing
the need to declare them at each call site.
o Create an rcu_for_each_leaf() macro that scans the leaf nodes
of the rcu_node tree.
o Create an rcu_for_each_node_breadth_first() macro that does
a breadth-first traversal of the rcu_node tree, AKA stepping
through the array in index-number order.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcutree.c | 53 ++++++++++++++++++++++++----------------------
kernel/rcutree.h | 12 ++++++++++
kernel/rcutree_plugin.h | 4 ---
3 files changed, 40 insertions(+), 29 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 1ccdd5b..bb313fd 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -461,8 +461,6 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
long delta;
unsigned long flags;
struct rcu_node *rnp = rcu_get_root(rsp);
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
/* Only let one CPU complain about others per time interval. */
@@ -473,18 +471,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
return;
}
rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
+
+ /*
+ * Now rat on any tasks that got kicked up to the root rcu_node
+ * due to CPU offlining.
+ */
+ rcu_print_task_stall(rnp);
spin_unlock_irqrestore(&rnp->lock, flags);
/* OK, time to rat on our buddy... */
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
rcu_print_task_stall(rnp);
- if (rnp_cur->qsmask == 0)
+ if (rnp->qsmask == 0)
continue;
- for (cpu = 0; cpu <= rnp_cur->grphi - rnp_cur->grplo; cpu++)
- if (rnp_cur->qsmask & (1UL << cpu))
- printk(" %d", rnp_cur->grplo + cpu);
+ for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
+ if (rnp->qsmask & (1UL << cpu))
+ printk(" %d", rnp->grplo + cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
smp_processor_id(), (long)(jiffies - rsp->gp_start));
@@ -644,7 +648,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
* one corresponding to this CPU, due to the fact that we have
* irqs disabled.
*/
- for (rnp = &rsp->node[0]; rnp < &rsp->node[NUM_RCU_NODES]; rnp++) {
+ rcu_for_each_node_breadth_first(rsp, rnp) {
spin_lock(&rnp->lock); /* irqs already disabled. */
rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
@@ -1037,33 +1041,32 @@ static int rcu_process_dyntick(struct rcu_state *rsp, long lastcomp,
int cpu;
unsigned long flags;
unsigned long mask;
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
+ struct rcu_node *rnp;
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
mask = 0;
- spin_lock_irqsave(&rnp_cur->lock, flags);
+ spin_lock_irqsave(&rnp->lock, flags);
if (rsp->completed != lastcomp) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
return 1;
}
- if (rnp_cur->qsmask == 0) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ if (rnp->qsmask == 0) {
+ spin_unlock_irqrestore(&rnp->lock, flags);
continue;
}
- cpu = rnp_cur->grplo;
+ cpu = rnp->grplo;
bit = 1;
- for (; cpu <= rnp_cur->grphi; cpu++, bit <<= 1) {
- if ((rnp_cur->qsmask & bit) != 0 && f(rsp->rda[cpu]))
+ for (; cpu <= rnp->grphi; cpu++, bit <<= 1) {
+ if ((rnp->qsmask & bit) != 0 && f(rsp->rda[cpu]))
mask |= bit;
}
if (mask != 0 && rsp->completed == lastcomp) {
- /* cpu_quiet_msk() releases rnp_cur->lock. */
- cpu_quiet_msk(mask, rsp, rnp_cur, flags);
+ /* cpu_quiet_msk() releases rnp->lock. */
+ cpu_quiet_msk(mask, rsp, rnp, flags);
continue;
}
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
}
return 0;
}
@@ -1545,6 +1548,10 @@ static void __init rcu_init_one(struct rcu_state *rsp)
*/
#define RCU_INIT_FLAVOR(rsp, rcu_data) \
do { \
+ int i; \
+ int j; \
+ struct rcu_node *rnp; \
+ \
rcu_init_one(rsp); \
rnp = (rsp)->level[NUM_RCU_LVLS - 1]; \
j = 0; \
@@ -1559,10 +1566,6 @@ do { \
void __init __rcu_init(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
rcu_bootup_announce();
#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
printk(KERN_INFO "RCU-based detection of stalled CPUs is enabled.\n");
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index e6ab31c..676eecd 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -106,6 +106,18 @@ struct rcu_node {
/* blocked_tasks[] array. */
} ____cacheline_internodealigned_in_smp;
+/*
+ * Do a full breadth-first scan of the rcu_node structures for the
+ * specified rcu_state structure.
+ */
+#define rcu_for_each_node_breadth_first(rsp, rnp) \
+ for ((rnp) = &(rsp)->node[0]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
+#define rcu_for_each_leaf_node(rsp, rnp) \
+ for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
/* Index values for nxttail array in struct rcu_data. */
#define RCU_DONE_TAIL 0 /* Also RCU_WAIT head. */
#define RCU_WAIT_TAIL 1 /* Also RCU_NEXT_READY head. */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index 2c73162..d88dfd3 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -424,10 +424,6 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu)
*/
static void __init __rcu_init_preempt(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
RCU_INIT_FLAVOR(&rcu_preempt_state, rcu_preempt_data);
}
--
1.5.2.5
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4
2009-09-27 6:49 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
@ 2009-09-27 15:31 ` Josh Triplett
2009-09-28 4:35 ` Paul E. McKenney
0 siblings, 1 reply; 15+ messages in thread
From: Josh Triplett @ 2009-09-27 15:31 UTC (permalink / raw)
To: Paul E. McKenney
Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers,
dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
On Sat, Sep 26, 2009 at 11:49:51PM -0700, Paul E. McKenney wrote:
> These issues identified during an old-fashioned face-to-face code
> review extending over many hours. This group improves an existing
> abstraction and introduces two new ones.
>
> o Make RCU_INIT_FLAVOR() declare its own variables, removing
> the need to declare them at each call site.
>
> o Create an rcu_for_each_leaf() macro that scans the leaf nodes
> of the rcu_node tree.
>
> o Create an rcu_for_each_node_breadth_first() macro that does
> a breadth-first traversal of the rcu_node tree, AKA stepping
> through the array in index-number order.
>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
I noticed one bit of unrelated code in this patch, which the commit
message doesn't mention:
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
[...]
> @@ -473,18 +471,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
> return;
> }
> rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
> +
> + /*
> + * Now rat on any tasks that got kicked up to the root rcu_node
> + * due to CPU offlining.
> + */
> + rcu_print_task_stall(rnp);
> spin_unlock_irqrestore(&rnp->lock, flags);
- Josh Triplett
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3
2009-09-27 6:49 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
@ 2009-09-27 15:41 ` Josh Triplett
0 siblings, 0 replies; 15+ messages in thread
From: Josh Triplett @ 2009-09-27 15:41 UTC (permalink / raw)
To: Paul E. McKenney
Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers,
dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
On Sat, Sep 26, 2009 at 11:49:50PM -0700, Paul E. McKenney wrote:
> Whitespace fixes, updated comments, and trivial code movement.
>
> o Fix whitespace error in RCU_HEAD_INIT()
>
> o Move "So where is rcu_write_lock()" comment so that it does
> not come between the rcu_read_unlock() header comment and
> the rcu_read_unlock() definition.
>
> o Move the module_param statements for blimit, qhimark, and qlowmark
> to immediately follow the corresponding definitions.
>
> o In __rcu_offline_cpu(), move the assignment to rdp_me inside the
> "if" statement, given that rdp_me is not used outside of that "if"
> statement.
>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Josh Triplett <josh@joshtriplett.org>
> ---
> include/linux/rcupdate.h | 14 +++++++-------
> kernel/rcutree.c | 10 +++++-----
> 2 files changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> index b2f1e10..6dd71fa 100644
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -83,7 +83,7 @@ extern int rcu_scheduler_active;
> #error "Unknown RCU implementation specified to kernel configuration"
> #endif
>
> -#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
> +#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
> #define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
> #define INIT_RCU_HEAD(ptr) do { \
> (ptr)->next = NULL; (ptr)->func = NULL; \
> @@ -135,12 +135,6 @@ static inline void rcu_read_lock(void)
> rcu_read_acquire();
> }
>
> -/**
> - * rcu_read_unlock - marks the end of an RCU read-side critical section.
> - *
> - * See rcu_read_lock() for more information.
> - */
> -
> /*
> * So where is rcu_write_lock()? It does not exist, as there is no
> * way for writers to lock out RCU readers. This is a feature, not
> @@ -150,6 +144,12 @@ static inline void rcu_read_lock(void)
> * used as well. RCU does not care how the writers keep out of each
> * others' way, as long as they do so.
> */
> +
> +/**
> + * rcu_read_unlock - marks the end of an RCU read-side critical section.
> + *
> + * See rcu_read_lock() for more information.
> + */
> static inline void rcu_read_unlock(void)
> {
> rcu_read_release();
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index 45eb2d1..1ccdd5b 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -121,6 +121,10 @@ static int blimit = 10; /* Maximum callbacks per softirq. */
> static int qhimark = 10000; /* If this many pending, ignore blimit. */
> static int qlowmark = 100; /* Once only this many pending, use blimit. */
>
> +module_param(blimit, int, 0);
> +module_param(qhimark, int, 0);
> +module_param(qlowmark, int, 0);
> +
> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> static int rcu_pending(int cpu);
>
> @@ -873,8 +877,8 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)
> * indefinitely delay callbacks, you have far worse things to
> * be worrying about.
> */
> - rdp_me = rsp->rda[smp_processor_id()];
> if (rdp->nxtlist != NULL) {
> + rdp_me = rsp->rda[smp_processor_id()];
> *rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxtlist;
> rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
> rdp->nxtlist = NULL;
> @@ -1570,7 +1574,3 @@ void __init __rcu_init(void)
> }
>
> #include "rcutree_plugin.h"
> -
> -module_param(blimit, int, 0);
> -module_param(qhimark, int, 0);
> -module_param(qlowmark, int, 0);
> --
> 1.5.2.5
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4
2009-09-27 15:31 ` Josh Triplett
@ 2009-09-28 4:35 ` Paul E. McKenney
0 siblings, 0 replies; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-28 4:35 UTC (permalink / raw)
To: Josh Triplett
Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers,
dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
On Sun, Sep 27, 2009 at 08:31:08AM -0700, Josh Triplett wrote:
> On Sat, Sep 26, 2009 at 11:49:51PM -0700, Paul E. McKenney wrote:
> > These issues identified during an old-fashioned face-to-face code
> > review extending over many hours. This group improves an existing
> > abstraction and introduces two new ones.
> >
> > o Make RCU_INIT_FLAVOR() declare its own variables, removing
> > the need to declare them at each call site.
> >
> > o Create an rcu_for_each_leaf() macro that scans the leaf nodes
> > of the rcu_node tree.
> >
> > o Create an rcu_for_each_node_breadth_first() macro that does
> > a breadth-first traversal of the rcu_node tree, AKA stepping
> > through the array in index-number order.
> >
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>
> I noticed one bit of unrelated code in this patch, which the commit
> message doesn't mention:
Good point -- this was not from our code review, but rather from my
mini-code-review introducing the relevant fixes.
> > --- a/kernel/rcutree.c
> > +++ b/kernel/rcutree.c
> [...]
> > @@ -473,18 +471,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
> > return;
> > }
> > rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
> > +
> > + /*
> > + * Now rat on any tasks that got kicked up to the root rcu_node
> > + * due to CPU offlining.
> > + */
> > + rcu_print_task_stall(rnp);
> > spin_unlock_irqrestore(&rnp->lock, flags);
The old code only checked for tasks blocked in RCU read-side critical
sections that were queued in the leaves of the rcu_node tree. Of
course, if all of the CPUs corresponding to a given rcu_node leaf have
gone offline since the tasks where queued, then they are moved to the
root rcu_node structure. So we need to check for stalled tasks in the
root rcu_node structure as well in the leaf rcu_node structures.
Thanx, Paul
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback
2009-09-27 6:47 [PATCH tip/core/rcu 0/2] rcu: more review feedback Paul E. McKenney
2009-09-27 6:49 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
2009-09-27 6:49 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
@ 2009-09-28 14:45 ` Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
` (3 more replies)
2 siblings, 4 replies; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-28 14:45 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
This patchset applies the last of the review comments:
o Whitespace fixes, updated comments, and code movement.
o Improve an existing abstraction and introduce two additional
abstractions for traversing the rcu_node tree.
Change since v1 (http://lkml.org/lkml/2009/9/27/17):
o Update second patch's changelog
Issues remaining:
o I can force hangs given sufficient rcutorture testing in
conjunction with random CPU-hotplug operations. I am working
on a simpler interface between RCU and CPU hotplug that I
believe will clear this up.
o TREE_PREEMPT_RCU currently uses the TREE_RCU variant of
synchronize_rcu_expedited, which is wrong. I am working on
an expedited variant for TREE_PREEMPT_RCU, but will fall
back on synchronize_rcu() for the short term if it gives me
much trouble.
o The IPI hyperactivity from call_rcu() noted by Nick Piggin.
I believe I have identified a clean design for a fix for this.
o Make RCU less dependent on IPIs for forcing grace periods
(but this might be best deferred to 2.6.33).
o RCU priority boosting (2.6.33!).
o TINY_PREEMPT_RCU (2.6.33!).
o Squeeze another 400 bytes or so out of TINY_RCU (2.6.33!).
b/include/linux/rcupdate.h | 14 +++++------
b/kernel/rcutree.c | 11 ++++-----
b/kernel/rcutree.h | 12 ++++++++++
b/kernel/rcutree_plugin.h | 5 ----
kernel/rcutree.c | 53 +++++++++++++++++++++++----------------------
5 files changed, 52 insertions(+), 43 deletions(-)
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
@ 2009-09-28 14:46 ` Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
2009-10-05 19:09 ` tip-bot for Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
` (2 subsequent siblings)
3 siblings, 2 replies; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-28 14:46 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells,
Paul E. McKenney
Whitespace fixes, updated comments, and trivial code movement.
o Fix whitespace error in RCU_HEAD_INIT()
o Move "So where is rcu_write_lock()" comment so that it does
not come between the rcu_read_unlock() header comment and
the rcu_read_unlock() definition.
o Move the module_param statements for blimit, qhimark, and qlowmark
to immediately follow the corresponding definitions.
o In __rcu_offline_cpu(), move the assignment to rdp_me inside the
"if" statement, given that rdp_me is not used outside of that "if"
statement.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
include/linux/rcupdate.h | 14 +++++++-------
kernel/rcutree.c | 10 +++++-----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index b2f1e10..6dd71fa 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -83,7 +83,7 @@ extern int rcu_scheduler_active;
#error "Unknown RCU implementation specified to kernel configuration"
#endif
-#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
+#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
#define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
#define INIT_RCU_HEAD(ptr) do { \
(ptr)->next = NULL; (ptr)->func = NULL; \
@@ -135,12 +135,6 @@ static inline void rcu_read_lock(void)
rcu_read_acquire();
}
-/**
- * rcu_read_unlock - marks the end of an RCU read-side critical section.
- *
- * See rcu_read_lock() for more information.
- */
-
/*
* So where is rcu_write_lock()? It does not exist, as there is no
* way for writers to lock out RCU readers. This is a feature, not
@@ -150,6 +144,12 @@ static inline void rcu_read_lock(void)
* used as well. RCU does not care how the writers keep out of each
* others' way, as long as they do so.
*/
+
+/**
+ * rcu_read_unlock - marks the end of an RCU read-side critical section.
+ *
+ * See rcu_read_lock() for more information.
+ */
static inline void rcu_read_unlock(void)
{
rcu_read_release();
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 45eb2d1..1ccdd5b 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -121,6 +121,10 @@ static int blimit = 10; /* Maximum callbacks per softirq. */
static int qhimark = 10000; /* If this many pending, ignore blimit. */
static int qlowmark = 100; /* Once only this many pending, use blimit. */
+module_param(blimit, int, 0);
+module_param(qhimark, int, 0);
+module_param(qlowmark, int, 0);
+
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
static int rcu_pending(int cpu);
@@ -873,8 +877,8 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)
* indefinitely delay callbacks, you have far worse things to
* be worrying about.
*/
- rdp_me = rsp->rda[smp_processor_id()];
if (rdp->nxtlist != NULL) {
+ rdp_me = rsp->rda[smp_processor_id()];
*rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxtlist;
rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
rdp->nxtlist = NULL;
@@ -1570,7 +1574,3 @@ void __init __rcu_init(void)
}
#include "rcutree_plugin.h"
-
-module_param(blimit, int, 0);
-module_param(qhimark, int, 0);
-module_param(qlowmark, int, 0);
--
1.5.2.5
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
@ 2009-09-28 14:46 ` Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
2009-10-05 19:10 ` tip-bot for Paul E. McKenney
2009-09-28 15:53 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Ingo Molnar
2009-09-28 16:09 ` Josh Triplett
3 siblings, 2 replies; 15+ messages in thread
From: Paul E. McKenney @ 2009-09-28 14:46 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells,
Paul E. McKenney
From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
These issues identified during an old-fashioned face-to-face code
review extending over many hours. This group improves an existing
abstraction and introduces two new ones. It also fixes an RCU
stall-warning bug found while making the other changes.
o Make RCU_INIT_FLAVOR() declare its own variables, removing
the need to declare them at each call site.
o Create an rcu_for_each_leaf() macro that scans the leaf nodes
of the rcu_node tree.
o Create an rcu_for_each_node_breadth_first() macro that does
a breadth-first traversal of the rcu_node tree, AKA stepping
through the array in index-number order.
o If all CPUs corresponding to a given leaf rcu_node structure
go offline, then any tasks queued on that leaf will be moved
to the root rcu_node structure. Therefore, the stall-warning
code must dump out tasks queued on the root rcu_node structure
as well as those queued on the leaf rcu_node structures.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcutree.c | 53 ++++++++++++++++++++++++----------------------
kernel/rcutree.h | 12 ++++++++++
kernel/rcutree_plugin.h | 4 ---
3 files changed, 40 insertions(+), 29 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 1ccdd5b..bb313fd 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -461,8 +461,6 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
long delta;
unsigned long flags;
struct rcu_node *rnp = rcu_get_root(rsp);
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
/* Only let one CPU complain about others per time interval. */
@@ -473,18 +471,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
return;
}
rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
+
+ /*
+ * Now rat on any tasks that got kicked up to the root rcu_node
+ * due to CPU offlining.
+ */
+ rcu_print_task_stall(rnp);
spin_unlock_irqrestore(&rnp->lock, flags);
/* OK, time to rat on our buddy... */
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
rcu_print_task_stall(rnp);
- if (rnp_cur->qsmask == 0)
+ if (rnp->qsmask == 0)
continue;
- for (cpu = 0; cpu <= rnp_cur->grphi - rnp_cur->grplo; cpu++)
- if (rnp_cur->qsmask & (1UL << cpu))
- printk(" %d", rnp_cur->grplo + cpu);
+ for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
+ if (rnp->qsmask & (1UL << cpu))
+ printk(" %d", rnp->grplo + cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
smp_processor_id(), (long)(jiffies - rsp->gp_start));
@@ -644,7 +648,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
* one corresponding to this CPU, due to the fact that we have
* irqs disabled.
*/
- for (rnp = &rsp->node[0]; rnp < &rsp->node[NUM_RCU_NODES]; rnp++) {
+ rcu_for_each_node_breadth_first(rsp, rnp) {
spin_lock(&rnp->lock); /* irqs already disabled. */
rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
@@ -1037,33 +1041,32 @@ static int rcu_process_dyntick(struct rcu_state *rsp, long lastcomp,
int cpu;
unsigned long flags;
unsigned long mask;
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
+ struct rcu_node *rnp;
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
mask = 0;
- spin_lock_irqsave(&rnp_cur->lock, flags);
+ spin_lock_irqsave(&rnp->lock, flags);
if (rsp->completed != lastcomp) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
return 1;
}
- if (rnp_cur->qsmask == 0) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ if (rnp->qsmask == 0) {
+ spin_unlock_irqrestore(&rnp->lock, flags);
continue;
}
- cpu = rnp_cur->grplo;
+ cpu = rnp->grplo;
bit = 1;
- for (; cpu <= rnp_cur->grphi; cpu++, bit <<= 1) {
- if ((rnp_cur->qsmask & bit) != 0 && f(rsp->rda[cpu]))
+ for (; cpu <= rnp->grphi; cpu++, bit <<= 1) {
+ if ((rnp->qsmask & bit) != 0 && f(rsp->rda[cpu]))
mask |= bit;
}
if (mask != 0 && rsp->completed == lastcomp) {
- /* cpu_quiet_msk() releases rnp_cur->lock. */
- cpu_quiet_msk(mask, rsp, rnp_cur, flags);
+ /* cpu_quiet_msk() releases rnp->lock. */
+ cpu_quiet_msk(mask, rsp, rnp, flags);
continue;
}
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
}
return 0;
}
@@ -1545,6 +1548,10 @@ static void __init rcu_init_one(struct rcu_state *rsp)
*/
#define RCU_INIT_FLAVOR(rsp, rcu_data) \
do { \
+ int i; \
+ int j; \
+ struct rcu_node *rnp; \
+ \
rcu_init_one(rsp); \
rnp = (rsp)->level[NUM_RCU_LVLS - 1]; \
j = 0; \
@@ -1559,10 +1566,6 @@ do { \
void __init __rcu_init(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
rcu_bootup_announce();
#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
printk(KERN_INFO "RCU-based detection of stalled CPUs is enabled.\n");
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index e6ab31c..676eecd 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -106,6 +106,18 @@ struct rcu_node {
/* blocked_tasks[] array. */
} ____cacheline_internodealigned_in_smp;
+/*
+ * Do a full breadth-first scan of the rcu_node structures for the
+ * specified rcu_state structure.
+ */
+#define rcu_for_each_node_breadth_first(rsp, rnp) \
+ for ((rnp) = &(rsp)->node[0]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
+#define rcu_for_each_leaf_node(rsp, rnp) \
+ for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
/* Index values for nxttail array in struct rcu_data. */
#define RCU_DONE_TAIL 0 /* Also RCU_WAIT head. */
#define RCU_WAIT_TAIL 1 /* Also RCU_NEXT_READY head. */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index 2c73162..d88dfd3 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -424,10 +424,6 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu)
*/
static void __init __rcu_init_preempt(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
RCU_INIT_FLAVOR(&rcu_preempt_state, rcu_preempt_data);
}
--
1.5.2.5
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
@ 2009-09-28 15:53 ` Ingo Molnar
2009-09-28 16:09 ` Josh Triplett
3 siblings, 0 replies; 15+ messages in thread
From: Ingo Molnar @ 2009-09-28 15:53 UTC (permalink / raw)
To: Paul E. McKenney
Cc: linux-kernel, laijs, dipankar, akpm, mathieu.desnoyers, josh,
dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> This patchset applies the last of the review comments:
>
> o Whitespace fixes, updated comments, and code movement.
>
> o Improve an existing abstraction and introduce two additional
> abstractions for traversing the rcu_node tree.
>
> Change since v1 (http://lkml.org/lkml/2009/9/27/17):
>
> o Update second patch's changelog
Applied, thanks a lot Paul!
Ingo
^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip:core/rcu] rcu: Clean up code based on review feedback from Josh Triplett, part 3
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
@ 2009-09-28 15:55 ` tip-bot for Paul E. McKenney
2009-10-05 19:09 ` tip-bot for Paul E. McKenney
1 sibling, 0 replies; 15+ messages in thread
From: tip-bot for Paul E. McKenney @ 2009-09-28 15:55 UTC (permalink / raw)
To: linux-tip-commits; +Cc: linux-kernel, paulmck, hpa, mingo, tglx, mingo
Commit-ID: 10905fde9671288030c616473eade4574f9894a1
Gitweb: http://git.kernel.org/tip/10905fde9671288030c616473eade4574f9894a1
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
AuthorDate: Mon, 28 Sep 2009 07:46:32 -0700
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Mon, 28 Sep 2009 17:52:38 +0200
rcu: Clean up code based on review feedback from Josh Triplett, part 3
Whitespace fixes, updated comments, and trivial code movement.
o Fix whitespace error in RCU_HEAD_INIT()
o Move "So where is rcu_write_lock()" comment so that it does
not come between the rcu_read_unlock() header comment and
the rcu_read_unlock() definition.
o Move the module_param statements for blimit, qhimark, and
qlowmark to immediately follow the corresponding
definitions.
o In __rcu_offline_cpu(), move the assignment to rdp_me
inside the "if" statement, given that rdp_me is not used
outside of that "if" statement.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference: <12541491931164-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
include/linux/rcupdate.h | 14 +++++++-------
kernel/rcutree.c | 10 +++++-----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index fe5c560..2f1bc42 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -83,7 +83,7 @@ extern int rcu_scheduler_active;
#error "Unknown RCU implementation specified to kernel configuration"
#endif
-#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
+#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
#define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
#define INIT_RCU_HEAD(ptr) do { \
(ptr)->next = NULL; (ptr)->func = NULL; \
@@ -135,12 +135,6 @@ static inline void rcu_read_lock(void)
rcu_read_acquire();
}
-/**
- * rcu_read_unlock - marks the end of an RCU read-side critical section.
- *
- * See rcu_read_lock() for more information.
- */
-
/*
* So where is rcu_write_lock()? It does not exist, as there is no
* way for writers to lock out RCU readers. This is a feature, not
@@ -150,6 +144,12 @@ static inline void rcu_read_lock(void)
* used as well. RCU does not care how the writers keep out of each
* others' way, as long as they do so.
*/
+
+/**
+ * rcu_read_unlock - marks the end of an RCU read-side critical section.
+ *
+ * See rcu_read_lock() for more information.
+ */
static inline void rcu_read_unlock(void)
{
rcu_read_release();
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 81af59b..d559783 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -122,6 +122,10 @@ static int blimit = 10; /* Maximum callbacks per softirq. */
static int qhimark = 10000; /* If this many pending, ignore blimit. */
static int qlowmark = 100; /* Once only this many pending, use blimit. */
+module_param(blimit, int, 0);
+module_param(qhimark, int, 0);
+module_param(qlowmark, int, 0);
+
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
static int rcu_pending(int cpu);
@@ -878,8 +882,8 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)
* indefinitely delay callbacks, you have far worse things to
* be worrying about.
*/
- rdp_me = rsp->rda[smp_processor_id()];
if (rdp->nxtlist != NULL) {
+ rdp_me = rsp->rda[smp_processor_id()];
*rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxtlist;
rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
rdp->nxtlist = NULL;
@@ -1575,7 +1579,3 @@ void __init __rcu_init(void)
}
#include "rcutree_plugin.h"
-
-module_param(blimit, int, 0);
-module_param(qhimark, int, 0);
-module_param(qlowmark, int, 0);
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [tip:core/rcu] rcu: Clean up code based on review feedback from Josh Triplett, part 4
2009-09-28 14:46 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
@ 2009-09-28 15:55 ` tip-bot for Paul E. McKenney
2009-10-05 19:10 ` tip-bot for Paul E. McKenney
1 sibling, 0 replies; 15+ messages in thread
From: tip-bot for Paul E. McKenney @ 2009-09-28 15:55 UTC (permalink / raw)
To: linux-tip-commits; +Cc: linux-kernel, paulmck, hpa, mingo, tglx, mingo
Commit-ID: 3a60838d46a718a6bd0bd69cb04eea10d8d59ede
Gitweb: http://git.kernel.org/tip/3a60838d46a718a6bd0bd69cb04eea10d8d59ede
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
AuthorDate: Mon, 28 Sep 2009 07:46:33 -0700
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Mon, 28 Sep 2009 17:52:39 +0200
rcu: Clean up code based on review feedback from Josh Triplett, part 4
These issues identified during an old-fashioned face-to-face code
review extending over many hours. This group improves an existing
abstraction and introduces two new ones. It also fixes an RCU
stall-warning bug found while making the other changes.
o Make RCU_INIT_FLAVOR() declare its own variables, removing
the need to declare them at each call site.
o Create an rcu_for_each_leaf() macro that scans the leaf
nodes of the rcu_node tree.
o Create an rcu_for_each_node_breadth_first() macro that does
a breadth-first traversal of the rcu_node tree, AKA
stepping through the array in index-number order.
o If all CPUs corresponding to a given leaf rcu_node
structure go offline, then any tasks queued on that leaf
will be moved to the root rcu_node structure. Therefore,
the stall-warning code must dump out tasks queued on the
root rcu_node structure as well as those queued on the leaf
rcu_node structures.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference: <12541491934126-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
kernel/rcutree.c | 53 ++++++++++++++++++++++++----------------------
kernel/rcutree.h | 12 ++++++++++
kernel/rcutree_plugin.h | 4 ---
3 files changed, 40 insertions(+), 29 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index d559783..e2e272b 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -462,8 +462,6 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
long delta;
unsigned long flags;
struct rcu_node *rnp = rcu_get_root(rsp);
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
/* Only let one CPU complain about others per time interval. */
@@ -474,18 +472,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
return;
}
rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
+
+ /*
+ * Now rat on any tasks that got kicked up to the root rcu_node
+ * due to CPU offlining.
+ */
+ rcu_print_task_stall(rnp);
spin_unlock_irqrestore(&rnp->lock, flags);
/* OK, time to rat on our buddy... */
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
rcu_print_task_stall(rnp);
- if (rnp_cur->qsmask == 0)
+ if (rnp->qsmask == 0)
continue;
- for (cpu = 0; cpu <= rnp_cur->grphi - rnp_cur->grplo; cpu++)
- if (rnp_cur->qsmask & (1UL << cpu))
- printk(" %d", rnp_cur->grplo + cpu);
+ for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
+ if (rnp->qsmask & (1UL << cpu))
+ printk(" %d", rnp->grplo + cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
smp_processor_id(), (long)(jiffies - rsp->gp_start));
@@ -649,7 +653,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
* one corresponding to this CPU, due to the fact that we have
* irqs disabled.
*/
- for (rnp = &rsp->node[0]; rnp < &rsp->node[NUM_RCU_NODES]; rnp++) {
+ rcu_for_each_node_breadth_first(rsp, rnp) {
spin_lock(&rnp->lock); /* irqs already disabled. */
rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
@@ -1042,33 +1046,32 @@ static int rcu_process_dyntick(struct rcu_state *rsp, long lastcomp,
int cpu;
unsigned long flags;
unsigned long mask;
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
+ struct rcu_node *rnp;
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
mask = 0;
- spin_lock_irqsave(&rnp_cur->lock, flags);
+ spin_lock_irqsave(&rnp->lock, flags);
if (rsp->completed != lastcomp) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
return 1;
}
- if (rnp_cur->qsmask == 0) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ if (rnp->qsmask == 0) {
+ spin_unlock_irqrestore(&rnp->lock, flags);
continue;
}
- cpu = rnp_cur->grplo;
+ cpu = rnp->grplo;
bit = 1;
- for (; cpu <= rnp_cur->grphi; cpu++, bit <<= 1) {
- if ((rnp_cur->qsmask & bit) != 0 && f(rsp->rda[cpu]))
+ for (; cpu <= rnp->grphi; cpu++, bit <<= 1) {
+ if ((rnp->qsmask & bit) != 0 && f(rsp->rda[cpu]))
mask |= bit;
}
if (mask != 0 && rsp->completed == lastcomp) {
- /* cpu_quiet_msk() releases rnp_cur->lock. */
- cpu_quiet_msk(mask, rsp, rnp_cur, flags);
+ /* cpu_quiet_msk() releases rnp->lock. */
+ cpu_quiet_msk(mask, rsp, rnp, flags);
continue;
}
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
}
return 0;
}
@@ -1550,6 +1553,10 @@ static void __init rcu_init_one(struct rcu_state *rsp)
*/
#define RCU_INIT_FLAVOR(rsp, rcu_data) \
do { \
+ int i; \
+ int j; \
+ struct rcu_node *rnp; \
+ \
rcu_init_one(rsp); \
rnp = (rsp)->level[NUM_RCU_LVLS - 1]; \
j = 0; \
@@ -1564,10 +1571,6 @@ do { \
void __init __rcu_init(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
rcu_bootup_announce();
#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
printk(KERN_INFO "RCU-based detection of stalled CPUs is enabled.\n");
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index e6ab31c..676eecd 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -106,6 +106,18 @@ struct rcu_node {
/* blocked_tasks[] array. */
} ____cacheline_internodealigned_in_smp;
+/*
+ * Do a full breadth-first scan of the rcu_node structures for the
+ * specified rcu_state structure.
+ */
+#define rcu_for_each_node_breadth_first(rsp, rnp) \
+ for ((rnp) = &(rsp)->node[0]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
+#define rcu_for_each_leaf_node(rsp, rnp) \
+ for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
/* Index values for nxttail array in struct rcu_data. */
#define RCU_DONE_TAIL 0 /* Also RCU_WAIT head. */
#define RCU_WAIT_TAIL 1 /* Also RCU_NEXT_READY head. */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index 6525021..57200fe 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -423,10 +423,6 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu)
*/
static void __init __rcu_init_preempt(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
RCU_INIT_FLAVOR(&rcu_preempt_state, rcu_preempt_data);
}
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
` (2 preceding siblings ...)
2009-09-28 15:53 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Ingo Molnar
@ 2009-09-28 16:09 ` Josh Triplett
3 siblings, 0 replies; 15+ messages in thread
From: Josh Triplett @ 2009-09-28 16:09 UTC (permalink / raw)
To: Paul E. McKenney
Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers,
dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells
On Mon, Sep 28, 2009 at 07:45:51AM -0700, Paul E. McKenney wrote:
> This patchset applies the last of the review comments:
>
> o Whitespace fixes, updated comments, and code movement.
>
> o Improve an existing abstraction and introduce two additional
> abstractions for traversing the rcu_node tree.
>
> Change since v1 (http://lkml.org/lkml/2009/9/27/17):
>
> o Update second patch's changelog
>
> Issues remaining:
>
> o I can force hangs given sufficient rcutorture testing in
> conjunction with random CPU-hotplug operations. I am working
> on a simpler interface between RCU and CPU hotplug that I
> believe will clear this up.
>
> o TREE_PREEMPT_RCU currently uses the TREE_RCU variant of
> synchronize_rcu_expedited, which is wrong. I am working on
> an expedited variant for TREE_PREEMPT_RCU, but will fall
> back on synchronize_rcu() for the short term if it gives me
> much trouble.
>
> o The IPI hyperactivity from call_rcu() noted by Nick Piggin.
> I believe I have identified a clean design for a fix for this.
>
> o Make RCU less dependent on IPIs for forcing grace periods
> (but this might be best deferred to 2.6.33).
>
> o RCU priority boosting (2.6.33!).
>
> o TINY_PREEMPT_RCU (2.6.33!).
>
> o Squeeze another 400 bytes or so out of TINY_RCU (2.6.33!).
>
>
> b/include/linux/rcupdate.h | 14 +++++------
> b/kernel/rcutree.c | 11 ++++-----
> b/kernel/rcutree.h | 12 ++++++++++
> b/kernel/rcutree_plugin.h | 5 ----
> kernel/rcutree.c | 53 +++++++++++++++++++++++----------------------
> 5 files changed, 52 insertions(+), 43 deletions(-)
For both patches:
Acked-by: Josh Triplett <josh@joshtriplett.org>
^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip:core/rcu] rcu: Clean up code based on review feedback from Josh Triplett, part 3
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
@ 2009-10-05 19:09 ` tip-bot for Paul E. McKenney
1 sibling, 0 replies; 15+ messages in thread
From: tip-bot for Paul E. McKenney @ 2009-10-05 19:09 UTC (permalink / raw)
To: linux-tip-commits; +Cc: linux-kernel, paulmck, hpa, mingo, tglx, mingo
Commit-ID: 3d76c082907e8f83c5d5c4572f38d53ad8f00c4b
Gitweb: http://git.kernel.org/tip/3d76c082907e8f83c5d5c4572f38d53ad8f00c4b
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
AuthorDate: Mon, 28 Sep 2009 07:46:32 -0700
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Mon, 5 Oct 2009 21:02:02 +0200
rcu: Clean up code based on review feedback from Josh Triplett, part 3
Whitespace fixes, updated comments, and trivial code movement.
o Fix whitespace error in RCU_HEAD_INIT()
o Move "So where is rcu_write_lock()" comment so that it does
not come between the rcu_read_unlock() header comment and
the rcu_read_unlock() definition.
o Move the module_param statements for blimit, qhimark, and
qlowmark to immediately follow the corresponding
definitions.
o In __rcu_offline_cpu(), move the assignment to rdp_me
inside the "if" statement, given that rdp_me is not used
outside of that "if" statement.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference: <12541491931164-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
include/linux/rcupdate.h | 14 +++++++-------
kernel/rcutree.c | 10 +++++-----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 7033121..3ebd0b7 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -77,7 +77,7 @@ extern int rcu_scheduler_active;
#error "Unknown RCU implementation specified to kernel configuration"
#endif
-#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
+#define RCU_HEAD_INIT { .next = NULL, .func = NULL }
#define RCU_HEAD(head) struct rcu_head head = RCU_HEAD_INIT
#define INIT_RCU_HEAD(ptr) do { \
(ptr)->next = NULL; (ptr)->func = NULL; \
@@ -129,12 +129,6 @@ static inline void rcu_read_lock(void)
rcu_read_acquire();
}
-/**
- * rcu_read_unlock - marks the end of an RCU read-side critical section.
- *
- * See rcu_read_lock() for more information.
- */
-
/*
* So where is rcu_write_lock()? It does not exist, as there is no
* way for writers to lock out RCU readers. This is a feature, not
@@ -144,6 +138,12 @@ static inline void rcu_read_lock(void)
* used as well. RCU does not care how the writers keep out of each
* others' way, as long as they do so.
*/
+
+/**
+ * rcu_read_unlock - marks the end of an RCU read-side critical section.
+ *
+ * See rcu_read_lock() for more information.
+ */
static inline void rcu_read_unlock(void)
{
rcu_read_release();
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 81af59b..d559783 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -122,6 +122,10 @@ static int blimit = 10; /* Maximum callbacks per softirq. */
static int qhimark = 10000; /* If this many pending, ignore blimit. */
static int qlowmark = 100; /* Once only this many pending, use blimit. */
+module_param(blimit, int, 0);
+module_param(qhimark, int, 0);
+module_param(qlowmark, int, 0);
+
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
static int rcu_pending(int cpu);
@@ -878,8 +882,8 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)
* indefinitely delay callbacks, you have far worse things to
* be worrying about.
*/
- rdp_me = rsp->rda[smp_processor_id()];
if (rdp->nxtlist != NULL) {
+ rdp_me = rsp->rda[smp_processor_id()];
*rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxtlist;
rdp_me->nxttail[RCU_NEXT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
rdp->nxtlist = NULL;
@@ -1575,7 +1579,3 @@ void __init __rcu_init(void)
}
#include "rcutree_plugin.h"
-
-module_param(blimit, int, 0);
-module_param(qhimark, int, 0);
-module_param(qlowmark, int, 0);
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [tip:core/rcu] rcu: Clean up code based on review feedback from Josh Triplett, part 4
2009-09-28 14:46 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
@ 2009-10-05 19:10 ` tip-bot for Paul E. McKenney
1 sibling, 0 replies; 15+ messages in thread
From: tip-bot for Paul E. McKenney @ 2009-10-05 19:10 UTC (permalink / raw)
To: linux-tip-commits; +Cc: linux-kernel, paulmck, hpa, mingo, tglx, mingo
Commit-ID: a0b6c9a78c41dc36732d6e1e90f0f2f57b29816f
Gitweb: http://git.kernel.org/tip/a0b6c9a78c41dc36732d6e1e90f0f2f57b29816f
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
AuthorDate: Mon, 28 Sep 2009 07:46:33 -0700
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Mon, 5 Oct 2009 21:02:04 +0200
rcu: Clean up code based on review feedback from Josh Triplett, part 4
These issues identified during an old-fashioned face-to-face code
review extending over many hours. This group improves an existing
abstraction and introduces two new ones. It also fixes an RCU
stall-warning bug found while making the other changes.
o Make RCU_INIT_FLAVOR() declare its own variables, removing
the need to declare them at each call site.
o Create an rcu_for_each_leaf() macro that scans the leaf
nodes of the rcu_node tree.
o Create an rcu_for_each_node_breadth_first() macro that does
a breadth-first traversal of the rcu_node tree, AKA
stepping through the array in index-number order.
o If all CPUs corresponding to a given leaf rcu_node
structure go offline, then any tasks queued on that leaf
will be moved to the root rcu_node structure. Therefore,
the stall-warning code must dump out tasks queued on the
root rcu_node structure as well as those queued on the leaf
rcu_node structures.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: akpm@linux-foundation.org
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference: <12541491934126-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
kernel/rcutree.c | 53 ++++++++++++++++++++++++----------------------
kernel/rcutree.h | 12 ++++++++++
kernel/rcutree_plugin.h | 4 ---
3 files changed, 40 insertions(+), 29 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index d559783..e2e272b 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -462,8 +462,6 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
long delta;
unsigned long flags;
struct rcu_node *rnp = rcu_get_root(rsp);
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
/* Only let one CPU complain about others per time interval. */
@@ -474,18 +472,24 @@ static void print_other_cpu_stall(struct rcu_state *rsp)
return;
}
rsp->jiffies_stall = jiffies + RCU_SECONDS_TILL_STALL_RECHECK;
+
+ /*
+ * Now rat on any tasks that got kicked up to the root rcu_node
+ * due to CPU offlining.
+ */
+ rcu_print_task_stall(rnp);
spin_unlock_irqrestore(&rnp->lock, flags);
/* OK, time to rat on our buddy... */
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
rcu_print_task_stall(rnp);
- if (rnp_cur->qsmask == 0)
+ if (rnp->qsmask == 0)
continue;
- for (cpu = 0; cpu <= rnp_cur->grphi - rnp_cur->grplo; cpu++)
- if (rnp_cur->qsmask & (1UL << cpu))
- printk(" %d", rnp_cur->grplo + cpu);
+ for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
+ if (rnp->qsmask & (1UL << cpu))
+ printk(" %d", rnp->grplo + cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
smp_processor_id(), (long)(jiffies - rsp->gp_start));
@@ -649,7 +653,7 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags)
* one corresponding to this CPU, due to the fact that we have
* irqs disabled.
*/
- for (rnp = &rsp->node[0]; rnp < &rsp->node[NUM_RCU_NODES]; rnp++) {
+ rcu_for_each_node_breadth_first(rsp, rnp) {
spin_lock(&rnp->lock); /* irqs already disabled. */
rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
@@ -1042,33 +1046,32 @@ static int rcu_process_dyntick(struct rcu_state *rsp, long lastcomp,
int cpu;
unsigned long flags;
unsigned long mask;
- struct rcu_node *rnp_cur = rsp->level[NUM_RCU_LVLS - 1];
- struct rcu_node *rnp_end = &rsp->node[NUM_RCU_NODES];
+ struct rcu_node *rnp;
- for (; rnp_cur < rnp_end; rnp_cur++) {
+ rcu_for_each_leaf_node(rsp, rnp) {
mask = 0;
- spin_lock_irqsave(&rnp_cur->lock, flags);
+ spin_lock_irqsave(&rnp->lock, flags);
if (rsp->completed != lastcomp) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
return 1;
}
- if (rnp_cur->qsmask == 0) {
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ if (rnp->qsmask == 0) {
+ spin_unlock_irqrestore(&rnp->lock, flags);
continue;
}
- cpu = rnp_cur->grplo;
+ cpu = rnp->grplo;
bit = 1;
- for (; cpu <= rnp_cur->grphi; cpu++, bit <<= 1) {
- if ((rnp_cur->qsmask & bit) != 0 && f(rsp->rda[cpu]))
+ for (; cpu <= rnp->grphi; cpu++, bit <<= 1) {
+ if ((rnp->qsmask & bit) != 0 && f(rsp->rda[cpu]))
mask |= bit;
}
if (mask != 0 && rsp->completed == lastcomp) {
- /* cpu_quiet_msk() releases rnp_cur->lock. */
- cpu_quiet_msk(mask, rsp, rnp_cur, flags);
+ /* cpu_quiet_msk() releases rnp->lock. */
+ cpu_quiet_msk(mask, rsp, rnp, flags);
continue;
}
- spin_unlock_irqrestore(&rnp_cur->lock, flags);
+ spin_unlock_irqrestore(&rnp->lock, flags);
}
return 0;
}
@@ -1550,6 +1553,10 @@ static void __init rcu_init_one(struct rcu_state *rsp)
*/
#define RCU_INIT_FLAVOR(rsp, rcu_data) \
do { \
+ int i; \
+ int j; \
+ struct rcu_node *rnp; \
+ \
rcu_init_one(rsp); \
rnp = (rsp)->level[NUM_RCU_LVLS - 1]; \
j = 0; \
@@ -1564,10 +1571,6 @@ do { \
void __init __rcu_init(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
rcu_bootup_announce();
#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
printk(KERN_INFO "RCU-based detection of stalled CPUs is enabled.\n");
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index e6ab31c..676eecd 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -106,6 +106,18 @@ struct rcu_node {
/* blocked_tasks[] array. */
} ____cacheline_internodealigned_in_smp;
+/*
+ * Do a full breadth-first scan of the rcu_node structures for the
+ * specified rcu_state structure.
+ */
+#define rcu_for_each_node_breadth_first(rsp, rnp) \
+ for ((rnp) = &(rsp)->node[0]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
+#define rcu_for_each_leaf_node(rsp, rnp) \
+ for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
+ (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+
/* Index values for nxttail array in struct rcu_data. */
#define RCU_DONE_TAIL 0 /* Also RCU_WAIT head. */
#define RCU_WAIT_TAIL 1 /* Also RCU_NEXT_READY head. */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index 6525021..57200fe 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -423,10 +423,6 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu)
*/
static void __init __rcu_init_preempt(void)
{
- int i; /* All used by RCU_INIT_FLAVOR(). */
- int j;
- struct rcu_node *rnp;
-
RCU_INIT_FLAVOR(&rcu_preempt_state, rcu_preempt_data);
}
^ permalink raw reply related [flat|nested] 15+ messages in thread
end of thread, other threads:[~2009-10-05 19:11 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-09-27 6:47 [PATCH tip/core/rcu 0/2] rcu: more review feedback Paul E. McKenney
2009-09-27 6:49 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
2009-09-27 15:41 ` Josh Triplett
2009-09-27 6:49 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
2009-09-27 15:31 ` Josh Triplett
2009-09-28 4:35 ` Paul E. McKenney
2009-09-28 14:45 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 1/2] rcu: Apply review feedback from Josh Triplett, part 3 Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
2009-10-05 19:09 ` tip-bot for Paul E. McKenney
2009-09-28 14:46 ` [PATCH tip/core/rcu 2/2] rcu: Apply review feedback from Josh Triplett, part 4 Paul E. McKenney
2009-09-28 15:55 ` [tip:core/rcu] rcu: Clean up code based on " tip-bot for Paul E. McKenney
2009-10-05 19:10 ` tip-bot for Paul E. McKenney
2009-09-28 15:53 ` [PATCH tip/core/rcu 0/2] rcu: v2: more review feedback Ingo Molnar
2009-09-28 16:09 ` Josh Triplett
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.