[RFC PATCH 0/5] rcu: fix sparse warnings

[RFC PATCH 0/5] rcu: fix sparse warnings

[Pranith Kumar]

The following patch series fixes sparse warning. There are still 3 warnings left
which I am figuring out. I wanted to send these to get feedback and see if I am
on the right path.

Patch 1 is the significant one which involves some refactoring. I am guessing
the 3 remaining sparse warnings will require similar refactoring.

Patch 4 remove the __noreturn attribute as this confuses sparse. I am not sure
if removing this is the right thing to do or if we should be fixing sparse.

The other 3 are trivial.

Pranith Kumar (5):
  kernel/rcu/tree.c:1272 fix a sparse warning 
  kernel/rcu/tree_plugin.h:1494 fix a sparse warning 
  kernel/rcu/tree_plugin.h:990 fix a sparse warning
  kernel/rcu/tree.c:3435 fix a sparse warning
  kernel/rcu/rcutorture.c:185 fix a sparse warning 

 kernel/rcu/rcutorture.c  |  2 +-
 kernel/rcu/tree.c        | 70 +++++++++++++++++++++++++++---------------------
 kernel/rcu/tree_plugin.h |  3 +++
 3 files changed, 43 insertions(+), 32 deletions(-)

-- 
1.9.1

[RFC PATCH 1/5] kernel/rcu/tree.c:1272 fix a sparse warning

[Pranith Kumar]

kernel/rcu/tree.c:1272:9: warning: context imbalance in 'rcu_start_future_gp' - different lock contexts for basic block

We can simplify the function by keeping the contexts together and removing
redundant checks.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree.c | 65 ++++++++++++++++++++++++++++++-------------------------
 1 file changed, 35 insertions(+), 30 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index f1ba773..9ab84d3 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1234,49 +1234,54 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 	}
 
 	/*
-	 * There might be no grace period in progress.  If we don't already
+	 * There is be no grace period in progress.  If we don't already
 	 * hold it, acquire the root rcu_node structure's lock in order to
-	 * start one (if needed).
+	 * start one.
 	 */
 	if (rnp != rnp_root) {
 		raw_spin_lock(&rnp_root->lock);
 		smp_mb__after_unlock_lock();
+		/*
+		 * Get a new grace-period number.  If there really is no grace
+		 * period in progress, it will be smaller than the one we obtained
+		 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
+		 */
+		c = rcu_cbs_completed(rdp->rsp, rnp_root);
+
+		/*
+		 * If the needed request for the required grace period is already
+		 * recorded, trace and leave.
+		 */
+		if (rnp_root->need_future_gp[c & 0x1]) {
+			trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
+			raw_spin_unlock(&rnp_root->lock);
+			goto out;
+		}
+
+		/* Record the need for the future grace period. */
+		rnp_root->need_future_gp[c & 0x1]++;
+
+		/*
+		 * Start a new grace period since it is not started
+		 */
+		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
+		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
+		raw_spin_unlock(&rnp_root->lock);
+		goto out;
 	}
 
+	/* rnp == rnp_root, we already hold the lock */
+	trace_rcu_future_gp(rnp, rdp, c, TPS("StartedLeaf"));
+	ret = rcu_start_gp_advanced(rdp->rsp, rnp, rdp);
+out:
 	/*
-	 * Get a new grace-period number.  If there really is no grace
-	 * period in progress, it will be smaller than the one we obtained
-	 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
-	 * CPUs have a ->nxtcompleted[] array, so no no-CBs checks needed.
+	 * Adjust callbacks as needed.  Note that even no-CBs CPUs
+	 * have a ->nxtcompleted[] array, so no no-CBs checks needed.
 	 */
-	c = rcu_cbs_completed(rdp->rsp, rnp_root);
 	for (i = RCU_DONE_TAIL; i < RCU_NEXT_TAIL; i++)
 		if (ULONG_CMP_LT(c, rdp->nxtcompleted[i]))
 			rdp->nxtcompleted[i] = c;
 
-	/*
-	 * If the needed for the required grace period is already
-	 * recorded, trace and leave.
-	 */
-	if (rnp_root->need_future_gp[c & 0x1]) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
-		goto unlock_out;
-	}
-
-	/* Record the need for the future grace period. */
-	rnp_root->need_future_gp[c & 0x1]++;
-
-	/* If a grace period is not already in progress, start one. */
-	if (rnp_root->gpnum != rnp_root->completed) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleafroot"));
-	} else {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
-		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
-	}
-unlock_out:
-	if (rnp != rnp_root)
-		raw_spin_unlock(&rnp_root->lock);
-out:
 	if (c_out != NULL)
 		*c_out = c;
 	return ret;
-- 
1.9.1

[RFC PATCH 2/5] kernel/rcu/tree_plugin.h:1494 fix a sparse warning

[Pranith Kumar]

kernel/rcu/tree_plugin.h:1494:13: warning: context imbalance in 'rcu_initiate_boost' - unexpected unlock

by annotating the function with releases()

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree_plugin.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index cbc2c45..0c955d9 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1256,6 +1256,7 @@ static int rcu_boost_kthread(void *arg)
  * about it going away.
  */
 static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
+	__releases(rnp->lock)
 {
 	struct task_struct *t;
 
@@ -1491,6 +1492,7 @@ static void rcu_prepare_kthreads(int cpu)
 #else /* #ifdef CONFIG_RCU_BOOST */
 
 static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
+	__releases(rnp->lock)
 {
 	raw_spin_unlock_irqrestore(&rnp->lock, flags);
 }
-- 
1.9.1

[RFC PATCH 3/5] kernel/rcu/tree_plugin.h:990 fix a sparse warning

[Pranith Kumar]

kernel/rcu/tree_plugin.h:990:13: warning: context imbalance in 'rcu_report_unblock_qs_rnp' - unexpected unlock

by annotating the function with __releases()

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree_plugin.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 0c955d9..9f85469 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -988,6 +988,7 @@ static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp)
 
 /* Because preemptible RCU does not exist, no quieting of tasks. */
 static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags)
+	__releases(rnp->lock)
 {
 	raw_spin_unlock_irqrestore(&rnp->lock, flags);
 }
-- 
1.9.1

[RFC PATCH 4/5] kernel/rcu/tree.c:3435 fix a sparse warning

[Pranith Kumar]

kernel/rcu/tree.c:3435:21: warning: incorrect type in argument 1 (different modifiers)
kernel/rcu/tree.c:3435:21:    expected int ( *threadfn )( ... )
kernel/rcu/tree.c:3435:21:    got int ( static [toplevel] [noreturn] *<noident> )( ... )

by removing __noreturn attribute and adding unreachable() as suggested on the
mailing list: http://www.kernelhub.org/?p=2&msg=436683

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 9ab84d3..6029a2e 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1689,7 +1689,7 @@ static void rcu_gp_cleanup(struct rcu_state *rsp)
 /*
  * Body of kthread that handles grace periods.
  */
-static int __noreturn rcu_gp_kthread(void *arg)
+static int rcu_gp_kthread(void *arg)
 {
 	int fqs_state;
 	int gf;
@@ -1777,6 +1777,9 @@ static int __noreturn rcu_gp_kthread(void *arg)
 		/* Handle grace-period end. */
 		rcu_gp_cleanup(rsp);
 	}
+
+	unreachable();
+	return 0;
 }
 
 /*
-- 
1.9.1

[RFC PATCH 5/5] kernel/rcu/rcutorture.c:185 fix a sparse warning

[Pranith Kumar]

fix the following sparse warning

kernel/rcu/rcutorture.c:185:1: warning: symbol 'boost_mutex' was not declared. Should it be static?

by marking boost_mutex as a static mutex

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/rcutorture.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 7fa34f8..1cd4b2d 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -182,7 +182,7 @@ static u64 notrace rcu_trace_clock_local(void)
 #endif /* #else #ifdef CONFIG_RCU_TRACE */
 
 static unsigned long boost_starttime;	/* jiffies of next boost test start. */
-DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
+static DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
 					/*  and boost task create/destroy. */
 static atomic_t barrier_cbs_count;	/* Barrier callbacks registered. */
 static bool barrier_phase;		/* Test phase. */
-- 
1.9.1

Re: [RFC PATCH 4/5] kernel/rcu/tree.c:3435 fix a sparse warning

[josh]

On Wed, Jun 11, 2014 at 04:39:42PM -0400, Pranith Kumar wrote:
> kernel/rcu/tree.c:3435:21: warning: incorrect type in argument 1 (different modifiers)
> kernel/rcu/tree.c:3435:21:    expected int ( *threadfn )( ... )
> kernel/rcu/tree.c:3435:21:    got int ( static [toplevel] [noreturn] *<noident> )( ... )
> 
> by removing __noreturn attribute and adding unreachable() as suggested on the
> mailing list: http://www.kernelhub.org/?p=2&msg=436683
> 
> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>

No, we should not do this.  And the mailing list post you point to seems
to explicitly recommend using noreturn rather than unreachable.

If sparse doesn't understand this, that's a bug in sparse, not in the
kernel.  Sparse needs to understand that it's OK to drop noreturn from a
function pointer type, just not OK to add it.

Rationale: If you call a noreturn function through a non-noreturn
function pointer, you might end up with unnecessary cleanup code, but
the call will work.  If you call a non-noreturn function through a
noreturn function pointer, the caller will not expect a return, and may
crash; *that* should require a cast.

>  kernel/rcu/tree.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 9ab84d3..6029a2e 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -1689,7 +1689,7 @@ static void rcu_gp_cleanup(struct rcu_state *rsp)
>  /*
>   * Body of kthread that handles grace periods.
>   */
> -static int __noreturn rcu_gp_kthread(void *arg)
> +static int rcu_gp_kthread(void *arg)
>  {
>  	int fqs_state;
>  	int gf;
> @@ -1777,6 +1777,9 @@ static int __noreturn rcu_gp_kthread(void *arg)
>  		/* Handle grace-period end. */
>  		rcu_gp_cleanup(rsp);
>  	}
> +
> +	unreachable();
> +	return 0;
>  }
>  
>  /*
> -- 
> 1.9.1
> 

Re: [RFC PATCH 5/5] kernel/rcu/rcutorture.c:185 fix a sparse warning

[josh]

On Wed, Jun 11, 2014 at 04:39:43PM -0400, Pranith Kumar wrote:
> fix the following sparse warning
> 
> kernel/rcu/rcutorture.c:185:1: warning: symbol 'boost_mutex' was not declared. Should it be static?
> 
> by marking boost_mutex as a static mutex
> 
> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>

Please preserve the comment alignment (by deleting a tab).  With that
fixed:
Reviewed-by: Josh Triplett <josh@joshtriplett.org>

>  kernel/rcu/rcutorture.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
> index 7fa34f8..1cd4b2d 100644
> --- a/kernel/rcu/rcutorture.c
> +++ b/kernel/rcu/rcutorture.c
> @@ -182,7 +182,7 @@ static u64 notrace rcu_trace_clock_local(void)
>  #endif /* #else #ifdef CONFIG_RCU_TRACE */
>  
>  static unsigned long boost_starttime;	/* jiffies of next boost test start. */
> -DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
> +static DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
>  					/*  and boost task create/destroy. */
>  static atomic_t barrier_cbs_count;	/* Barrier callbacks registered. */
>  static bool barrier_phase;		/* Test phase. */
> -- 
> 1.9.1
> 

Re: [RFC PATCH 4/5] kernel/rcu/tree.c:3435 fix a sparse warning

[Pranith Kumar]

On Wed, Jun 11, 2014 at 5:25 PM,  <josh@joshtriplett.org> wrote:
> On Wed, Jun 11, 2014 at 04:39:42PM -0400, Pranith Kumar wrote:
>> kernel/rcu/tree.c:3435:21: warning: incorrect type in argument 1 (different modifiers)
>> kernel/rcu/tree.c:3435:21:    expected int ( *threadfn )( ... )
>> kernel/rcu/tree.c:3435:21:    got int ( static [toplevel] [noreturn] *<noident> )( ... )
>>
>> by removing __noreturn attribute and adding unreachable() as suggested on the
>> mailing list: http://www.kernelhub.org/?p=2&msg=436683
>>
>> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
>
> No, we should not do this.  And the mailing list post you point to seems
> to explicitly recommend using noreturn rather than unreachable.
>
> If sparse doesn't understand this, that's a bug in sparse, not in the
> kernel.  Sparse needs to understand that it's OK to drop noreturn from a
> function pointer type, just not OK to add it.
>
> Rationale: If you call a noreturn function through a non-noreturn
> function pointer, you might end up with unnecessary cleanup code, but
> the call will work.  If you call a non-noreturn function through a
> noreturn function pointer, the caller will not expect a return, and may
> crash; *that* should require a cast.
>

Yes, I understand the rationale. I think this should be fixed in
sparse. Please drop this patch.

Thanks!
-- 
Pranith

Re: [RFC PATCH 1/5] kernel/rcu/tree.c:1272 fix a sparse warning

[Paul E. McKenney]

On Wed, Jun 11, 2014 at 04:39:39PM -0400, Pranith Kumar wrote:
> kernel/rcu/tree.c:1272:9: warning: context imbalance in 'rcu_start_future_gp' - different lock contexts for basic block
> 
> We can simplify the function by keeping the contexts together and removing
> redundant checks.
> 
> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
> ---
>  kernel/rcu/tree.c | 65 ++++++++++++++++++++++++++++++-------------------------
>  1 file changed, 35 insertions(+), 30 deletions(-)
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index f1ba773..9ab84d3 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -1234,49 +1234,54 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
>  	}
> 
>  	/*
> -	 * There might be no grace period in progress.  If we don't already
> +	 * There is be no grace period in progress.  If we don't already

We actually don't know at this point, unless rnp==rnp_root.  Otherwise,
the grace period might have started, but initialization might not yet
have reached rnp.

>  	 * hold it, acquire the root rcu_node structure's lock in order to
> -	 * start one (if needed).
> +	 * start one.
>  	 */
>  	if (rnp != rnp_root) {
>  		raw_spin_lock(&rnp_root->lock);
>  		smp_mb__after_unlock_lock();

I am not convinced that this transformation is correct, especially in
the rnp==rnp_root case.  For one thing, I don't see the need for a
future grace period being recorded in that case.

And I believe that if this transformation is fixed, there will be some
duplicate code, which scares me more than sparse false positives.  So I
am not willing to take this sort of transformation.  Or am I missing
something?

> +		/*
> +		 * Get a new grace-period number.  If there really is no grace
> +		 * period in progress, it will be smaller than the one we obtained
> +		 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
> +		 */
> +		c = rcu_cbs_completed(rdp->rsp, rnp_root);

But I believe that this statement could be moved into the preceding "if"
statement in the original code.  If this is really the case, it could
be a good change.

							Thanx, Paul

> +
> +		/*
> +		 * If the needed request for the required grace period is already
> +		 * recorded, trace and leave.
> +		 */
> +		if (rnp_root->need_future_gp[c & 0x1]) {
> +			trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
> +			raw_spin_unlock(&rnp_root->lock);
> +			goto out;
> +		}
> +
> +		/* Record the need for the future grace period. */
> +		rnp_root->need_future_gp[c & 0x1]++;
> +
> +		/*
> +		 * Start a new grace period since it is not started
> +		 */
> +		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
> +		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
> +		raw_spin_unlock(&rnp_root->lock);
> +		goto out;
>  	}
> 
> +	/* rnp == rnp_root, we already hold the lock */
> +	trace_rcu_future_gp(rnp, rdp, c, TPS("StartedLeaf"));
> +	ret = rcu_start_gp_advanced(rdp->rsp, rnp, rdp);
> +out:
>  	/*
> -	 * Get a new grace-period number.  If there really is no grace
> -	 * period in progress, it will be smaller than the one we obtained
> -	 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
> -	 * CPUs have a ->nxtcompleted[] array, so no no-CBs checks needed.
> +	 * Adjust callbacks as needed.  Note that even no-CBs CPUs
> +	 * have a ->nxtcompleted[] array, so no no-CBs checks needed.
>  	 */
> -	c = rcu_cbs_completed(rdp->rsp, rnp_root);
>  	for (i = RCU_DONE_TAIL; i < RCU_NEXT_TAIL; i++)
>  		if (ULONG_CMP_LT(c, rdp->nxtcompleted[i]))
>  			rdp->nxtcompleted[i] = c;
> 
> -	/*
> -	 * If the needed for the required grace period is already
> -	 * recorded, trace and leave.
> -	 */
> -	if (rnp_root->need_future_gp[c & 0x1]) {
> -		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
> -		goto unlock_out;
> -	}
> -
> -	/* Record the need for the future grace period. */
> -	rnp_root->need_future_gp[c & 0x1]++;
> -
> -	/* If a grace period is not already in progress, start one. */
> -	if (rnp_root->gpnum != rnp_root->completed) {
> -		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleafroot"));
> -	} else {
> -		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
> -		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
> -	}
> -unlock_out:
> -	if (rnp != rnp_root)
> -		raw_spin_unlock(&rnp_root->lock);
> -out:
>  	if (c_out != NULL)
>  		*c_out = c;
>  	return ret;
> -- 
> 1.9.1
> 

Re: [RFC PATCH 1/5] kernel/rcu/tree.c:1272 fix a sparse warning

[Pranith Kumar]

On 06/12/2014 07:16 PM, Paul E. McKenney wrote:
> On Wed, Jun 11, 2014 at 04:39:39PM -0400, Pranith Kumar wrote:
>> kernel/rcu/tree.c:1272:9: warning: context imbalance in 'rcu_start_future_gp' - different lock contexts for basic block
>>
>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
>> index f1ba773..9ab84d3 100644
>> --- a/kernel/rcu/tree.c
>> +++ b/kernel/rcu/tree.c
>> @@ -1234,49 +1234,54 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
>>  	}
>>
>>  	/*
>> -	 * There might be no grace period in progress.  If we don't already
>> +	 * There is be no grace period in progress.  If we don't already
> 
> We actually don't know at this point, unless rnp==rnp_root.  Otherwise,
> the grace period might have started, but initialization might not yet
> have reached rnp.

I should have mentioned that I wrote this on top of the previous patch where we
were checking the root node for presence of a grace period 
	ACCESS_ONCE(rnp_root->gpnum) != ACCESS_ONCE(rnp_root->completed)

But, I realize that even this does not guarantee that a grace period is in
progress as we do not yet have the lock for the root.

> 
>>  	 * hold it, acquire the root rcu_node structure's lock in order to
>> -	 * start one (if needed).
>> +	 * start one.
>>  	 */
>>  	if (rnp != rnp_root) {
>>  		raw_spin_lock(&rnp_root->lock);
>>  		smp_mb__after_unlock_lock();
> 
> I am not convinced that this transformation is correct, especially in
> the rnp==rnp_root case.  For one thing, I don't see the need for a
> future grace period being recorded in that case.
> 
> And I believe that if this transformation is fixed, there will be some
> duplicate code, which scares me more than sparse false positives.  So I
> am not willing to take this sort of transformation.  Or am I missing
> something?
> 
 
You are right. I knew I missed something! Even though this started as an
exercise to remove the sparse warning, I thought I could simplify the function
since I could see that we are doing some things twice.

Please find v2 below which takes care of the issues you mentioned. RFC please!

simplify the rcu_start_future_gp function. fix sparse warning as an added bonus :)

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree.c | 80 +++++++++++++++++++++++++------------------------------
 1 file changed, 37 insertions(+), 43 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index f1ba773..ee98d0b 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1192,44 +1192,60 @@ static void trace_rcu_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 }
 
 /*
+ * Adjust callbacks as needed.  Note that even no-CBs CPUs
+ * have a ->nxtcompleted[] array, so no no-CBs checks needed.
+ */
+static void rcu_adjust_callbacks(unsigned long c, struct rcu_data *rdp)
+{
+	int i;
+	for (i = RCU_DONE_TAIL; i < RCU_NEXT_TAIL; i++)
+		if (ULONG_CMP_LT(c, rdp->nxtcompleted[i]))
+			rdp->nxtcompleted[i] = c;
+}
+
+/*
  * Start some future grace period, as needed to handle newly arrived
  * callbacks.  The required future grace periods are recorded in each
  * rcu_node structure's ->need_future_gp field.  Returns true if there
  * is reason to awaken the grace-period kthread.
  *
  * The caller must hold the specified rcu_node structure's ->lock.
+ *
+ * This is called recursively at-most twice, once with a rcu_node and 
+ * once with the root rcu_node.
  */
 static bool __maybe_unused
 rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 		    unsigned long *c_out)
 {
 	unsigned long c;
-	int i;
 	bool ret = false;
 	struct rcu_node *rnp_root = rcu_get_root(rdp->rsp);
+	bool is_root = (rnp_root == rnp);
 
 	/*
 	 * Pick up grace-period number for new callbacks.  If this
 	 * grace period is already marked as needed, return to the caller.
 	 */
 	c = rcu_cbs_completed(rdp->rsp, rnp);
-	trace_rcu_future_gp(rnp, rdp, c, TPS("Startleaf"));
+	trace_rcu_future_gp(rnp, rdp, c, 
+			is_root ? TPS("Startedroot") : TPS("Startleaf"));
 	if (rnp->need_future_gp[c & 0x1]) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartleaf"));
+		trace_rcu_future_gp(rnp, rdp, c, 
+				is_root ? TPS("Prestartroot") :	TPS("Prestartleaf"));
 		goto out;
 	}
 
 	/*
-	 * If either this rcu_node structure or the root rcu_node structure
-	 * believe that a grace period is in progress, then we must wait
-	 * for the one following, which is in "c".  Because our request
-	 * will be noticed at the end of the current grace period, we don't
-	 * need to explicitly start one.
+	 * If this rcu_node structure believes that a grace period is in progress,
+	 * then we must wait for the one following, which is in "c".  
+	 * Because our request will be noticed at the end of the current grace
+	 * period, we don't need to explicitly start one.
 	 */
-	if (rnp->gpnum != rnp->completed ||
-	    ACCESS_ONCE(rnp->gpnum) != ACCESS_ONCE(rnp->completed)) {
+	if (rnp->gpnum != rnp->completed) {
 		rnp->need_future_gp[c & 0x1]++;
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleaf"));
+		trace_rcu_future_gp(rnp, rdp, c, 
+			is_root ? TPS("Startedleafroot") : TPS("Startleaf"));
 		goto out;
 	}
 
@@ -1241,41 +1257,19 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 	if (rnp != rnp_root) {
 		raw_spin_lock(&rnp_root->lock);
 		smp_mb__after_unlock_lock();
-	}
-
-	/*
-	 * Get a new grace-period number.  If there really is no grace
-	 * period in progress, it will be smaller than the one we obtained
-	 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
-	 * CPUs have a ->nxtcompleted[] array, so no no-CBs checks needed.
-	 */
-	c = rcu_cbs_completed(rdp->rsp, rnp_root);
-	for (i = RCU_DONE_TAIL; i < RCU_NEXT_TAIL; i++)
-		if (ULONG_CMP_LT(c, rdp->nxtcompleted[i]))
-			rdp->nxtcompleted[i] = c;
 
-	/*
-	 * If the needed for the required grace period is already
-	 * recorded, trace and leave.
-	 */
-	if (rnp_root->need_future_gp[c & 0x1]) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
-		goto unlock_out;
+		/*
+		 * Start a new grace period using the root node
+		 */
+		ret = rcu_start_future_gp(rnp_root, rdp, &c);
+		raw_spin_unlock(&rnp_root->lock);
+		goto out;
 	}
 
-	/* Record the need for the future grace period. */
-	rnp_root->need_future_gp[c & 0x1]++;
-
-	/* If a grace period is not already in progress, start one. */
-	if (rnp_root->gpnum != rnp_root->completed) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleafroot"));
-	} else {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
-		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
-	}
-unlock_out:
-	if (rnp != rnp_root)
-		raw_spin_unlock(&rnp_root->lock);
+	rcu_adjust_callbacks(c, rdp);
+	/* rnp == rnp_root, we already hold the lock */
+	trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
+	ret = rcu_start_gp_advanced(rdp->rsp, rnp, rdp);
 out:
 	if (c_out != NULL)
 		*c_out = c;
-- 
1.9.1

Re: [RFC PATCH 1/5] kernel/rcu/tree.c:1272 fix a sparse warning

[Pranith Kumar]

On 06/13/2014 12:54 AM, Pranith Kumar wrote:
> On 06/12/2014 07:16 PM, Paul E. McKenney wrote:
>> On Wed, Jun 11, 2014 at 04:39:39PM -0400, Pranith Kumar wrote:
>>> kernel/rcu/tree.c:1272:9: warning: context imbalance in 'rcu_start_future_gp' - different lock contexts for basic block
>>>
>>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
>>> index f1ba773..9ab84d3 100644
>>> --- a/kernel/rcu/tree.c
>>> +++ b/kernel/rcu/tree.c
>>> @@ -1234,49 +1234,54 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
>>>  	}
>>>
>>>  	/*
>>> -	 * There might be no grace period in progress.  If we don't already
>>> +	 * There is be no grace period in progress.  If we don't already
>>
>> We actually don't know at this point, unless rnp==rnp_root.  Otherwise,
>> the grace period might have started, but initialization might not yet
>> have reached rnp.
> 
> I should have mentioned that I wrote this on top of the previous patch where we
> were checking the root node for presence of a grace period 
> 	ACCESS_ONCE(rnp_root->gpnum) != ACCESS_ONCE(rnp_root->completed)
> 
> But, I realize that even this does not guarantee that a grace period is in
> progress as we do not yet have the lock for the root.
> 
>>
>>>  	 * hold it, acquire the root rcu_node structure's lock in order to
>>> -	 * start one (if needed).
>>> +	 * start one.
>>>  	 */
>>>  	if (rnp != rnp_root) {
>>>  		raw_spin_lock(&rnp_root->lock);
>>>  		smp_mb__after_unlock_lock();
>>
>> I am not convinced that this transformation is correct, especially in
>> the rnp==rnp_root case.  For one thing, I don't see the need for a
>> future grace period being recorded in that case.
>>
>> And I believe that if this transformation is fixed, there will be some
>> duplicate code, which scares me more than sparse false positives.  So I
>> am not willing to take this sort of transformation.  Or am I missing
>> something?
>>
>  
> You are right. I knew I missed something! Even though this started as an
> exercise to remove the sparse warning, I thought I could simplify the function
> since I could see that we are doing some things twice.
> 
> Please find v2 below which takes care of the issues you mentioned. RFC please!
> 

Please find v3 which removes an unnecessary function I introduced.

simplify the function. fix sparse warning as an added bonus :)

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 kernel/rcu/tree.c | 65 +++++++++++++++++++++++--------------------------------
 1 file changed, 27 insertions(+), 38 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index f1ba773..639d7a0 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1198,6 +1198,9 @@ static void trace_rcu_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
  * is reason to awaken the grace-period kthread.
  *
  * The caller must hold the specified rcu_node structure's ->lock.
+ *
+ * This is called recursively at-most twice, once with a rcu_node and a root
+ * rcu_node.
  */
 static bool __maybe_unused
 rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
@@ -1207,29 +1210,31 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 	int i;
 	bool ret = false;
 	struct rcu_node *rnp_root = rcu_get_root(rdp->rsp);
+	bool is_root = (rnp_root == rnp);
 
 	/*
 	 * Pick up grace-period number for new callbacks.  If this
 	 * grace period is already marked as needed, return to the caller.
 	 */
 	c = rcu_cbs_completed(rdp->rsp, rnp);
-	trace_rcu_future_gp(rnp, rdp, c, TPS("Startleaf"));
+	trace_rcu_future_gp(rnp, rdp, c, 
+			is_root ? TPS("Startedroot") : TPS("Startleaf"));
 	if (rnp->need_future_gp[c & 0x1]) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartleaf"));
+		trace_rcu_future_gp(rnp, rdp, c, 
+				is_root ? TPS("Prestartroot") :	TPS("Prestartleaf"));
 		goto out;
 	}
 
 	/*
-	 * If either this rcu_node structure or the root rcu_node structure
-	 * believe that a grace period is in progress, then we must wait
-	 * for the one following, which is in "c".  Because our request
-	 * will be noticed at the end of the current grace period, we don't
-	 * need to explicitly start one.
+	 * If this rcu_node structure believes that a grace period is in progress,
+	 * then we must wait for the one following, which is in "c".  
+	 * Because our request will be noticed at the end of the current grace
+	 * period, we don't need to explicitly start one.
 	 */
-	if (rnp->gpnum != rnp->completed ||
-	    ACCESS_ONCE(rnp->gpnum) != ACCESS_ONCE(rnp->completed)) {
+	if (rnp->gpnum != rnp->completed) {
 		rnp->need_future_gp[c & 0x1]++;
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleaf"));
+		trace_rcu_future_gp(rnp, rdp, c, 
+			is_root ? TPS("Startedleafroot") : TPS("Startleaf"));
 		goto out;
 	}
 
@@ -1241,41 +1246,25 @@ rcu_start_future_gp(struct rcu_node *rnp, struct rcu_data *rdp,
 	if (rnp != rnp_root) {
 		raw_spin_lock(&rnp_root->lock);
 		smp_mb__after_unlock_lock();
+
+		/*
+		 * Start a new grace period with the root node
+		 */
+		ret = rcu_start_future_gp(rnp_root, rdp, &c);
+		raw_spin_unlock(&rnp_root->lock);
+		goto out;
 	}
 
 	/*
-	 * Get a new grace-period number.  If there really is no grace
-	 * period in progress, it will be smaller than the one we obtained
-	 * earlier.  Adjust callbacks as needed.  Note that even no-CBs
-	 * CPUs have a ->nxtcompleted[] array, so no no-CBs checks needed.
+	 * Adjust callbacks as needed.  Note that even no-CBs CPUs
+	 * have a ->nxtcompleted[] array, so no no-CBs checks needed.
 	 */
-	c = rcu_cbs_completed(rdp->rsp, rnp_root);
 	for (i = RCU_DONE_TAIL; i < RCU_NEXT_TAIL; i++)
 		if (ULONG_CMP_LT(c, rdp->nxtcompleted[i]))
 			rdp->nxtcompleted[i] = c;
-
-	/*
-	 * If the needed for the required grace period is already
-	 * recorded, trace and leave.
-	 */
-	if (rnp_root->need_future_gp[c & 0x1]) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Prestartedroot"));
-		goto unlock_out;
-	}
-
-	/* Record the need for the future grace period. */
-	rnp_root->need_future_gp[c & 0x1]++;
-
-	/* If a grace period is not already in progress, start one. */
-	if (rnp_root->gpnum != rnp_root->completed) {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedleafroot"));
-	} else {
-		trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
-		ret = rcu_start_gp_advanced(rdp->rsp, rnp_root, rdp);
-	}
-unlock_out:
-	if (rnp != rnp_root)
-		raw_spin_unlock(&rnp_root->lock);
+	/* rnp == rnp_root, we already hold the lock */
+	trace_rcu_future_gp(rnp, rdp, c, TPS("Startedroot"));
+	ret = rcu_start_gp_advanced(rdp->rsp, rnp, rdp);
 out:
 	if (c_out != NULL)
 		*c_out = c;
-- 
1.9.1

Re: [RFC PATCH 2/5] kernel/rcu/tree_plugin.h:1494 fix a sparse warning

[Paul E. McKenney]

On Wed, Jun 11, 2014 at 04:39:40PM -0400, Pranith Kumar wrote:
> kernel/rcu/tree_plugin.h:1494:13: warning: context imbalance in 'rcu_initiate_boost' - unexpected unlock
> 
> by annotating the function with releases()
> 
> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>

Hearing no objections, I have queued this for 3.17.

							Thanx, Paul

> ---
>  kernel/rcu/tree_plugin.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index cbc2c45..0c955d9 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -1256,6 +1256,7 @@ static int rcu_boost_kthread(void *arg)
>   * about it going away.
>   */
>  static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
> +	__releases(rnp->lock)
>  {
>  	struct task_struct *t;
> 
> @@ -1491,6 +1492,7 @@ static void rcu_prepare_kthreads(int cpu)
>  #else /* #ifdef CONFIG_RCU_BOOST */
> 
>  static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
> +	__releases(rnp->lock)
>  {
>  	raw_spin_unlock_irqrestore(&rnp->lock, flags);
>  }
> -- 
> 1.9.1
> 

Re: [RFC PATCH 3/5] kernel/rcu/tree_plugin.h:990 fix a sparse warning

[Paul E. McKenney]

On Wed, Jun 11, 2014 at 04:39:41PM -0400, Pranith Kumar wrote:
> kernel/rcu/tree_plugin.h:990:13: warning: context imbalance in 'rcu_report_unblock_qs_rnp' - unexpected unlock
> 
> by annotating the function with __releases()
> 
> Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>

Hearing no objections, I have queued this for 3.17.

							Thanx, Paul

> ---
>  kernel/rcu/tree_plugin.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index 0c955d9..9f85469 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -988,6 +988,7 @@ static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp)
> 
>  /* Because preemptible RCU does not exist, no quieting of tasks. */
>  static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags)
> +	__releases(rnp->lock)
>  {
>  	raw_spin_unlock_irqrestore(&rnp->lock, flags);
>  }
> -- 
> 1.9.1
> 

Re: [RFC PATCH 5/5] kernel/rcu/rcutorture.c:185 fix a sparse warning

[Paul E. McKenney]

On Wed, Jun 11, 2014 at 02:47:52PM -0700, josh@joshtriplett.org wrote:
> On Wed, Jun 11, 2014 at 04:39:43PM -0400, Pranith Kumar wrote:
> > fix the following sparse warning
> > 
> > kernel/rcu/rcutorture.c:185:1: warning: symbol 'boost_mutex' was not declared. Should it be static?
> > 
> > by marking boost_mutex as a static mutex
> > 
> > Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
> 
> Please preserve the comment alignment (by deleting a tab).  With that
> fixed:
> Reviewed-by: Josh Triplett <josh@joshtriplett.org>

Queued for 3.18.

But Pranith, next time Josh gives you a review comment, could you please
respond with the appropriate update?

							Thanx, Paul

> >  kernel/rcu/rcutorture.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
> > index 7fa34f8..1cd4b2d 100644
> > --- a/kernel/rcu/rcutorture.c
> > +++ b/kernel/rcu/rcutorture.c
> > @@ -182,7 +182,7 @@ static u64 notrace rcu_trace_clock_local(void)
> >  #endif /* #else #ifdef CONFIG_RCU_TRACE */
> >  
> >  static unsigned long boost_starttime;	/* jiffies of next boost test start. */
> > -DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
> > +static DEFINE_MUTEX(boost_mutex);		/* protect setting boost_starttime */
> >  					/*  and boost task create/destroy. */
> >  static atomic_t barrier_cbs_count;	/* Barrier callbacks registered. */
> >  static bool barrier_phase;		/* Test phase. */
> > -- 
> > 1.9.1
> > 
> 

Re: [RFC PATCH 5/5] kernel/rcu/rcutorture.c:185 fix a sparse warning

[Pranith Kumar]

On Tue, Jul 8, 2014 at 6:35 PM, Paul E. McKenney
<paulmck@linux.vnet.ibm.com> wrote:
> On Wed, Jun 11, 2014 at 02:47:52PM -0700, josh@joshtriplett.org wrote:
>> On Wed, Jun 11, 2014 at 04:39:43PM -0400, Pranith Kumar wrote:
>> > fix the following sparse warning
>> >
>> > kernel/rcu/rcutorture.c:185:1: warning: symbol 'boost_mutex' was not declared. Should it be static?
>> >
>> > by marking boost_mutex as a static mutex
>> >
>> > Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
>>
>> Please preserve the comment alignment (by deleting a tab).  With that
>> fixed:
>> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>
> Queued for 3.18.
>
> But Pranith, next time Josh gives you a review comment, could you please
> respond with the appropriate update?
>

I was away from my work desktop for the past few days. I actually sent
a fixed patch 10 min ago, but in a new series.

Sorry for missing this, I will drop an update promptly next time.


-- 
Pranith