* [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors
@ 2017-05-02 16:21 Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 1/3] rcu: Open-code the rcu_cblist_empty() function Paul E. McKenney
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Paul E. McKenney @ 2017-05-02 16:21 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani
Hello!
And this series open-codes trivial rcu_cblist structure accessors:
1. Open-code rcu_cblist_empty(p) as !p->head.
2. Open-code rcu_cblist_n_cbs(p) as p->len.
3. Open-code rcu_cblist_n_lazy_cbs(p) as p->len_lazy.
Thanx, Paul
------------------------------------------------------------------------
rcu_segcblist.h | 20 +-------------------
tree.c | 20 +++++++++-----------
tree_plugin.h | 8 ++++----
tree_trace.c | 4 ++--
4 files changed, 16 insertions(+), 36 deletions(-)
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH tip/core/rcu 1/3] rcu: Open-code the rcu_cblist_empty() function
2017-05-02 16:21 [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Paul E. McKenney
@ 2017-05-02 16:24 ` Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 2/3] rcu: Open-code the rcu_cblist_n_cbs() function Paul E. McKenney
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2017-05-02 16:24 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani, Paul E. McKenney, Linus Torvalds
Because the rcu_cblist_empty() just samples the ->head pointer, and
because the rcu_cblist structure is quite straightforward, it makes
sense to open-code rcu_cblist_empty(p) as !p->head, cutting out a
level of indirection. This commit makes this change.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
kernel/rcu/rcu_segcblist.h | 8 +-------
kernel/rcu/tree.c | 9 ++++-----
kernel/rcu/tree_plugin.h | 4 ++--
3 files changed, 7 insertions(+), 14 deletions(-)
diff --git a/kernel/rcu/rcu_segcblist.h b/kernel/rcu/rcu_segcblist.h
index 86bc1101b806..7d18d41f0116 100644
--- a/kernel/rcu/rcu_segcblist.h
+++ b/kernel/rcu/rcu_segcblist.h
@@ -22,12 +22,6 @@
#include <linux/rcu_segcblist.h>
-/* Is simple callback list empty? */
-static inline bool rcu_cblist_empty(struct rcu_cblist *rclp)
-{
- return !rclp->head;
-}
-
/* Return number of callbacks in simple callback list. */
static inline long rcu_cblist_n_cbs(struct rcu_cblist *rclp)
{
@@ -66,7 +60,7 @@ static inline struct rcu_head *rcu_cblist_head(struct rcu_cblist *rclp)
*/
static inline struct rcu_head **rcu_cblist_tail(struct rcu_cblist *rclp)
{
- WARN_ON_ONCE(rcu_cblist_empty(rclp));
+ WARN_ON_ONCE(!rclp->head);
return rclp->tail;
}
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 91fff49d5869..35152414760d 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2647,9 +2647,9 @@ static void rcu_adopt_orphan_cbs(struct rcu_state *rsp, unsigned long flags)
/* First adopt the ready-to-invoke callbacks, then the done ones. */
rcu_segcblist_insert_done_cbs(&rdp->cblist, &rsp->orphan_done);
- WARN_ON_ONCE(!rcu_cblist_empty(&rsp->orphan_done));
+ WARN_ON_ONCE(rsp->orphan_done.head);
rcu_segcblist_insert_pend_cbs(&rdp->cblist, &rsp->orphan_pend);
- WARN_ON_ONCE(!rcu_cblist_empty(&rsp->orphan_pend));
+ WARN_ON_ONCE(rsp->orphan_pend.head);
WARN_ON_ONCE(rcu_segcblist_empty(&rdp->cblist) !=
!rcu_segcblist_n_cbs(&rdp->cblist));
}
@@ -2800,9 +2800,8 @@ static void rcu_do_batch(struct rcu_state *rsp, struct rcu_data *rdp)
local_irq_save(flags);
count = -rcu_cblist_n_cbs(&rcl);
- trace_rcu_batch_end(rsp->name, count, !rcu_cblist_empty(&rcl),
- need_resched(), is_idle_task(current),
- rcu_is_callbacks_kthread());
+ trace_rcu_batch_end(rsp->name, count, !!rcl.head, need_resched(),
+ is_idle_task(current), rcu_is_callbacks_kthread());
/* Update counts and requeue any remaining callbacks. */
rcu_segcblist_insert_done_cbs(&rdp->cblist, &rcl);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 7f1d677a2a25..6b4b1f8a272d 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1942,12 +1942,12 @@ static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
return false;
/* First, enqueue the donelist, if any. This preserves CB ordering. */
- if (!rcu_cblist_empty(&rsp->orphan_done)) {
+ if (rsp->orphan_done.head) {
__call_rcu_nocb_enqueue(rdp, rcu_cblist_head(&rsp->orphan_done),
rcu_cblist_tail(&rsp->orphan_done),
ql, qll, flags);
}
- if (!rcu_cblist_empty(&rsp->orphan_pend)) {
+ if (rsp->orphan_pend.head) {
__call_rcu_nocb_enqueue(rdp, rcu_cblist_head(&rsp->orphan_pend),
rcu_cblist_tail(&rsp->orphan_pend),
ql, qll, flags);
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH tip/core/rcu 2/3] rcu: Open-code the rcu_cblist_n_cbs() function
2017-05-02 16:21 [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 1/3] rcu: Open-code the rcu_cblist_empty() function Paul E. McKenney
@ 2017-05-02 16:24 ` Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 3/3] rcu: Open-code the rcu_cblist_n_lazy_cbs() function Paul E. McKenney
2017-05-02 16:47 ` [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Ingo Molnar
3 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2017-05-02 16:24 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani, Paul E. McKenney, Linus Torvalds
Because the rcu_cblist_n_cbs() just samples the ->len counter, and
because the rcu_cblist structure is quite straightforward, it makes
sense to open-code rcu_cblist_n_cbs(p) as p->len, cutting out a level
of indirection. This commit makes this change.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
kernel/rcu/rcu_segcblist.h | 6 ------
kernel/rcu/tree.c | 9 ++++-----
kernel/rcu/tree_plugin.h | 2 +-
kernel/rcu/tree_trace.c | 2 +-
4 files changed, 6 insertions(+), 13 deletions(-)
diff --git a/kernel/rcu/rcu_segcblist.h b/kernel/rcu/rcu_segcblist.h
index 7d18d41f0116..424a6b230921 100644
--- a/kernel/rcu/rcu_segcblist.h
+++ b/kernel/rcu/rcu_segcblist.h
@@ -22,12 +22,6 @@
#include <linux/rcu_segcblist.h>
-/* Return number of callbacks in simple callback list. */
-static inline long rcu_cblist_n_cbs(struct rcu_cblist *rclp)
-{
- return rclp->len;
-}
-
/* Return number of lazy callbacks in simple callback list. */
static inline long rcu_cblist_n_lazy_cbs(struct rcu_cblist *rclp)
{
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 35152414760d..942d529fccbc 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2633,9 +2633,8 @@ static void rcu_adopt_orphan_cbs(struct rcu_state *rsp, unsigned long flags)
return;
/* Do the accounting first. */
- rdp->n_cbs_adopted += rcu_cblist_n_cbs(&rsp->orphan_done);
- if (rcu_cblist_n_lazy_cbs(&rsp->orphan_done) !=
- rcu_cblist_n_cbs(&rsp->orphan_done))
+ rdp->n_cbs_adopted += rsp->orphan_done.len;
+ if (rcu_cblist_n_lazy_cbs(&rsp->orphan_done) != rsp->orphan_done.len)
rcu_idle_count_callbacks_posted();
rcu_segcblist_insert_count(&rdp->cblist, &rsp->orphan_done);
@@ -2792,14 +2791,14 @@ static void rcu_do_batch(struct rcu_state *rsp, struct rcu_data *rdp)
* Stop only if limit reached and CPU has something to do.
* Note: The rcl structure counts down from zero.
*/
- if (-rcu_cblist_n_cbs(&rcl) >= bl &&
+ if (-rcl.len >= bl &&
(need_resched() ||
(!is_idle_task(current) && !rcu_is_callbacks_kthread())))
break;
}
local_irq_save(flags);
- count = -rcu_cblist_n_cbs(&rcl);
+ count = -rcl.len;
trace_rcu_batch_end(rsp->name, count, !!rcl.head, need_resched(),
is_idle_task(current), rcu_is_callbacks_kthread());
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 6b4b1f8a272d..7ebe357df155 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1934,7 +1934,7 @@ static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
struct rcu_data *rdp,
unsigned long flags)
{
- long ql = rcu_cblist_n_cbs(&rsp->orphan_done);
+ long ql = rsp->orphan_done.len;
long qll = rcu_cblist_n_lazy_cbs(&rsp->orphan_done);
/* If this is not a no-CBs CPU, tell the caller to do it the old way. */
diff --git a/kernel/rcu/tree_trace.c b/kernel/rcu/tree_trace.c
index 30c5bf89ee58..b7743aa2965f 100644
--- a/kernel/rcu/tree_trace.c
+++ b/kernel/rcu/tree_trace.c
@@ -278,7 +278,7 @@ static void print_one_rcu_state(struct seq_file *m, struct rcu_state *rsp)
rsp->n_force_qs - rsp->n_force_qs_ngp,
READ_ONCE(rsp->n_force_qs_lh),
rcu_cblist_n_lazy_cbs(&rsp->orphan_done),
- rcu_cblist_n_cbs(&rsp->orphan_done));
+ rsp->orphan_done.len);
for (rnp = &rsp->node[0]; rnp - &rsp->node[0] < rcu_num_nodes; rnp++) {
if (rnp->level != level) {
seq_puts(m, "\n");
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH tip/core/rcu 3/3] rcu: Open-code the rcu_cblist_n_lazy_cbs() function
2017-05-02 16:21 [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 1/3] rcu: Open-code the rcu_cblist_empty() function Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 2/3] rcu: Open-code the rcu_cblist_n_cbs() function Paul E. McKenney
@ 2017-05-02 16:24 ` Paul E. McKenney
2017-05-02 16:47 ` [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Ingo Molnar
3 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2017-05-02 16:24 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani, Paul E. McKenney, Linus Torvalds
Because the rcu_cblist_n_lazy_cbs() just samples the ->len_lazy counter,
and because the rcu_cblist structure is quite straightforward, it makes
sense to open-code rcu_cblist_n_lazy_cbs(p) as p->len_lazy, cutting out
a level of indirection. This commit makes this change.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
kernel/rcu/rcu_segcblist.h | 6 ------
kernel/rcu/tree.c | 2 +-
kernel/rcu/tree_plugin.h | 2 +-
kernel/rcu/tree_trace.c | 2 +-
4 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/rcu/rcu_segcblist.h b/kernel/rcu/rcu_segcblist.h
index 424a6b230921..6e36e36478cd 100644
--- a/kernel/rcu/rcu_segcblist.h
+++ b/kernel/rcu/rcu_segcblist.h
@@ -22,12 +22,6 @@
#include <linux/rcu_segcblist.h>
-/* Return number of lazy callbacks in simple callback list. */
-static inline long rcu_cblist_n_lazy_cbs(struct rcu_cblist *rclp)
-{
- return rclp->len_lazy;
-}
-
/*
* Account for the fact that a previously dequeued callback turned out
* to be marked as lazy.
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 942d529fccbc..1205c8ad138a 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2634,7 +2634,7 @@ static void rcu_adopt_orphan_cbs(struct rcu_state *rsp, unsigned long flags)
/* Do the accounting first. */
rdp->n_cbs_adopted += rsp->orphan_done.len;
- if (rcu_cblist_n_lazy_cbs(&rsp->orphan_done) != rsp->orphan_done.len)
+ if (rsp->orphan_done.len_lazy != rsp->orphan_done.len)
rcu_idle_count_callbacks_posted();
rcu_segcblist_insert_count(&rdp->cblist, &rsp->orphan_done);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 7ebe357df155..c9a48657512a 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1935,7 +1935,7 @@ static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_state *rsp,
unsigned long flags)
{
long ql = rsp->orphan_done.len;
- long qll = rcu_cblist_n_lazy_cbs(&rsp->orphan_done);
+ long qll = rsp->orphan_done.len_lazy;
/* If this is not a no-CBs CPU, tell the caller to do it the old way. */
if (!rcu_is_nocb_cpu(smp_processor_id()))
diff --git a/kernel/rcu/tree_trace.c b/kernel/rcu/tree_trace.c
index b7743aa2965f..6cea17a1ea30 100644
--- a/kernel/rcu/tree_trace.c
+++ b/kernel/rcu/tree_trace.c
@@ -277,7 +277,7 @@ static void print_one_rcu_state(struct seq_file *m, struct rcu_state *rsp)
rsp->n_force_qs, rsp->n_force_qs_ngp,
rsp->n_force_qs - rsp->n_force_qs_ngp,
READ_ONCE(rsp->n_force_qs_lh),
- rcu_cblist_n_lazy_cbs(&rsp->orphan_done),
+ rsp->orphan_done.len_lazy,
rsp->orphan_done.len);
for (rnp = &rsp->node[0]; rnp - &rsp->node[0] < rcu_num_nodes; rnp++) {
if (rnp->level != level) {
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors
2017-05-02 16:21 [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Paul E. McKenney
` (2 preceding siblings ...)
2017-05-02 16:24 ` [PATCH tip/core/rcu 3/3] rcu: Open-code the rcu_cblist_n_lazy_cbs() function Paul E. McKenney
@ 2017-05-02 16:47 ` Ingo Molnar
2017-05-02 17:01 ` Paul E. McKenney
3 siblings, 1 reply; 6+ messages in thread
From: Ingo Molnar @ 2017-05-02 16:47 UTC (permalink / raw)
To: Paul E. McKenney
Cc: linux-kernel, jiangshanlai, dipankar, akpm, mathieu.desnoyers,
josh, tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> Hello!
>
> And this series open-codes trivial rcu_cblist structure accessors:
>
> 1. Open-code rcu_cblist_empty(p) as !p->head.
>
> 2. Open-code rcu_cblist_n_cbs(p) as p->len.
>
> 3. Open-code rcu_cblist_n_lazy_cbs(p) as p->len_lazy.
>
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> rcu_segcblist.h | 20 +-------------------
> tree.c | 20 +++++++++-----------
> tree_plugin.h | 8 ++++----
> tree_trace.c | 4 ++--
> 4 files changed, 16 insertions(+), 36 deletions(-)
Ok, these all look good to me.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors
2017-05-02 16:47 ` [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Ingo Molnar
@ 2017-05-02 17:01 ` Paul E. McKenney
0 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2017-05-02 17:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: linux-kernel, jiangshanlai, dipankar, akpm, mathieu.desnoyers,
josh, tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
bobby.prani
On Tue, May 02, 2017 at 06:47:10PM +0200, Ingo Molnar wrote:
>
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>
> > Hello!
> >
> > And this series open-codes trivial rcu_cblist structure accessors:
> >
> > 1. Open-code rcu_cblist_empty(p) as !p->head.
> >
> > 2. Open-code rcu_cblist_n_cbs(p) as p->len.
> >
> > 3. Open-code rcu_cblist_n_lazy_cbs(p) as p->len_lazy.
> >
> >
> > Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > rcu_segcblist.h | 20 +-------------------
> > tree.c | 20 +++++++++-----------
> > tree_plugin.h | 8 ++++----
> > tree_trace.c | 4 ++--
> > 4 files changed, 16 insertions(+), 36 deletions(-)
>
> Ok, these all look good to me.
Whew! ;-)
Thanx, Paul
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-05-02 17:01 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-02 16:21 [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 1/3] rcu: Open-code the rcu_cblist_empty() function Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 2/3] rcu: Open-code the rcu_cblist_n_cbs() function Paul E. McKenney
2017-05-02 16:24 ` [PATCH tip/core/rcu 3/3] rcu: Open-code the rcu_cblist_n_lazy_cbs() function Paul E. McKenney
2017-05-02 16:47 ` [PATCH tip/core/rcu 0/3] Open-code trivial rcu_cblist accessors Ingo Molnar
2017-05-02 17:01 ` Paul E. McKenney
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).