RCU: Question on force_qs_rnp

RCU: Question on force_qs_rnp

[Zhang, Qiang]

Hello Paul

I have some questions for you .
in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0, 
we will check blocked readers in this rnp node,  instead of jumping directly to the next node .


Thanks
Qiang

Re: RCU: Question on force_qs_rnp

[Joel Fernandes]

On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
> Hello Paul
> 
> I have some questions for you .
> in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
> report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
> after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0, 
> we will check blocked readers in this rnp node,  instead of jumping directly to the next node .

Could you clarify what good is this going to do? What problem are you trying to
address?

You could have a task that is blocked in an RCU leaf node, but the
force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
CPU could be dyntick-idle and a quiescent state is reported. However, the GP
must not end and the rcu leaf node should still be present in its parent
intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
any relevance.

Or am I missing the point of the question?

thanks,

 - Joel

Re: RCU: Question on force_qs_rnp

[Paul E. McKenney]

On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
> On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
> > Hello Paul
> > 
> > I have some questions for you .
> > in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
> > report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
> > after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0, 
> > we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
> 
> Could you clarify what good is this going to do? What problem are you trying to
> address?
> 
> You could have a task that is blocked in an RCU leaf node, but the
> force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
> CPU could be dyntick-idle and a quiescent state is reported. However, the GP
> must not end and the rcu leaf node should still be present in its parent
> intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
> any relevance.
> 
> Or am I missing the point of the question?

Hello, Qiang,

Another way of making Joel's point is to say that the additional check
you are asking for is already being done, but by rcu_report_qs_rnp().

							Thanx, Paul

回复: RCU: Question on force_qs_rnp

[Zhang, Qiang]

________________________________________
发件人: Paul E. McKenney <paulmck@kernel.org>
发送时间: 2020年9月15日 4:56
收件人: Joel Fernandes
抄送: Zhang, Qiang; Uladzislau Rezki; josh@joshtriplett.org; rostedt@goodmis.org; mathieu.desnoyers@efficios.com; Lai Jiangshan; rcu@vger.kernel.org; LKML
主题: Re: RCU: Question on force_qs_rnp

On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
> On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
> > Hello Paul
> >
> > I have some questions for you .
> > in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
> > report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
> > after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0,
> > we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
>
> Could you clarify what good is this going to do? What problem are you trying to
> address?
>
> You could have a task that is blocked in an RCU leaf node, but the
> force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
> CPU could be dyntick-idle and a quiescent state is reported. However, the GP
> must not end and the rcu leaf node should still be present in its parent
> intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
> any relevance.
>
> Or am I missing the point of the question?

>Hello, Qiang,

>Another way of making Joel's point is to say that the additional check
>you are asking for is already being done, but by rcu_report_qs_rnp().

>                                                        Thanx, Paul

Hello Pual,  Joel

What I want to express is as follows :

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 7623128d0020..beb554539f01 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2622,6 +2622,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
                if (mask != 0) {
                        /* Idle/offline CPUs, report (releases rnp->lock). */
                        rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
+                       raw_spin_lock_irqsave_rcu_node(rnp, flags);
+                       if (rnp->qsmask == 0 && rcu_preempt_blocked_readers_cgp(rnp))
+                               rcu_initiate_boost(rnp, flags);
+                       else
+                               raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
                } else {
                        /* Nothing to do here, so just drop the lock. */
                        raw_spin_unlock_irqrestore_rcu_node(rnp, flags);


Thanks

Qiang

Re: 回复: RCU: Question on force_qs_rnp

[Paul E. McKenney]

On Tue, Sep 15, 2020 at 03:18:23AM +0000, Zhang, Qiang wrote:
> 
> 
> ________________________________________
> 发件人: Paul E. McKenney <paulmck@kernel.org>
> 发送时间: 2020年9月15日 4:56
> 收件人: Joel Fernandes
> 抄送: Zhang, Qiang; Uladzislau Rezki; josh@joshtriplett.org; rostedt@goodmis.org; mathieu.desnoyers@efficios.com; Lai Jiangshan; rcu@vger.kernel.org; LKML
> 主题: Re: RCU: Question on force_qs_rnp
> 
> On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
> > On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
> > > Hello Paul
> > >
> > > I have some questions for you .
> > > in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
> > > report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
> > > after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0,
> > > we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
> >
> > Could you clarify what good is this going to do? What problem are you trying to
> > address?
> >
> > You could have a task that is blocked in an RCU leaf node, but the
> > force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
> > CPU could be dyntick-idle and a quiescent state is reported. However, the GP
> > must not end and the rcu leaf node should still be present in its parent
> > intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
> > any relevance.
> >
> > Or am I missing the point of the question?
> 
> >Hello, Qiang,
> 
> >Another way of making Joel's point is to say that the additional check
> >you are asking for is already being done, but by rcu_report_qs_rnp().
> 
> >                                                        Thanx, Paul
> 
> Hello Pual,  Joel
> 
> What I want to express is as follows :
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 7623128d0020..beb554539f01 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -2622,6 +2622,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
>                 if (mask != 0) {
>                         /* Idle/offline CPUs, report (releases rnp->lock). */
>                         rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
> +                       raw_spin_lock_irqsave_rcu_node(rnp, flags);
> +                       if (rnp->qsmask == 0 && rcu_preempt_blocked_readers_cgp(rnp))
> +                               rcu_initiate_boost(rnp, flags);
> +                       else
> +                               raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>                 } else {
>                         /* Nothing to do here, so just drop the lock. */
>                         raw_spin_unlock_irqrestore_rcu_node(rnp, flags);

But in that case, why duplicate the code from rcu_initiate_boost()?

							Thanx, Paul

Re: 回复: RCU: Question on force_qs_rnp

[Zhang,Qiang]

On 9/15/20 11:41 AM, Paul E. McKenney wrote:
> On Tue, Sep 15, 2020 at 03:18:23AM +0000, Zhang, Qiang wrote:
>>
>>
>> ________________________________________
>> 发件人: Paul E. McKenney <paulmck@kernel.org>
>> 发送时间: 2020年9月15日 4:56
>> 收件人: Joel Fernandes
>> 抄送: Zhang, Qiang; Uladzislau Rezki; josh@joshtriplett.org; rostedt@goodmis.org; mathieu.desnoyers@efficios.com; Lai Jiangshan; rcu@vger.kernel.org; LKML
>> 主题: Re: RCU: Question on force_qs_rnp
>>
>> On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
>>> On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
>>>> Hello Paul
>>>>
>>>> I have some questions for you .
>>>> in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
>>>> report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
>>>> after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0,
>>>> we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
>>>
>>> Could you clarify what good is this going to do? What problem are you trying to
>>> address?
>>>
>>> You could have a task that is blocked in an RCU leaf node, but the
>>> force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
>>> CPU could be dyntick-idle and a quiescent state is reported. However, the GP
>>> must not end and the rcu leaf node should still be present in its parent
>>> intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
>>> any relevance.
>>>
>>> Or am I missing the point of the question?
>>
>>> Hello, Qiang,
>>
>>> Another way of making Joel's point is to say that the additional check
>>> you are asking for is already being done, but by rcu_report_qs_rnp().
>>
>>>                                                         Thanx, Paul
>>
>> Hello Pual,  Joel
>>
>> What I want to express is as follows :
>>
>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
>> index 7623128d0020..beb554539f01 100644
>> --- a/kernel/rcu/tree.c
>> +++ b/kernel/rcu/tree.c
>> @@ -2622,6 +2622,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
>>                  if (mask != 0) {
>>                          /* Idle/offline CPUs, report (releases rnp->lock). */
>>                          rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
>> +                       raw_spin_lock_irqsave_rcu_node(rnp, flags);
>> +                       if (rnp->qsmask == 0 && rcu_preempt_blocked_readers_cgp(rnp))
>> +                               rcu_initiate_boost(rnp, flags);
>> +                       else
>> +                               raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>>                  } else {
>>                          /* Nothing to do here, so just drop the lock. */
>>                          raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
> 
> But in that case, why duplicate the code from rcu_initiate_boost()?
> 
> 							Thanx, Paul
> 

Hello Paul

When we force a qs for rnp, we first check the leaf node "rnp->qsmask" 
if it is reached zero, will check if there are some blocked readers in 
this leaf rnp node, if so we need to priority-boost blocked readers.
if not we will check cpu dyntick-idle and report leaf node qs, after 
this leaf rnp node report qs, there is may be some blocked readers in 
this node, should we also need to priority-boost blocked readers?

Thanks

Qiang

Re: 回复: RCU: Question on force_qs_rnp

[Paul E. McKenney]

On Tue, Sep 15, 2020 at 01:16:39PM +0800, Zhang,Qiang wrote:
> 
> 
> On 9/15/20 11:41 AM, Paul E. McKenney wrote:
> > On Tue, Sep 15, 2020 at 03:18:23AM +0000, Zhang, Qiang wrote:
> > > 
> > > 
> > > ________________________________________
> > > 发件人: Paul E. McKenney <paulmck@kernel.org>
> > > 发送时间: 2020年9月15日 4:56
> > > 收件人: Joel Fernandes
> > > 抄送: Zhang, Qiang; Uladzislau Rezki; josh@joshtriplett.org; rostedt@goodmis.org; mathieu.desnoyers@efficios.com; Lai Jiangshan; rcu@vger.kernel.org; LKML
> > > 主题: Re: RCU: Question on force_qs_rnp
> > > 
> > > On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
> > > > On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
> > > > > Hello Paul
> > > > > 
> > > > > I have some questions for you .
> > > > > in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
> > > > > report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
> > > > > after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0,
> > > > > we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
> > > > 
> > > > Could you clarify what good is this going to do? What problem are you trying to
> > > > address?
> > > > 
> > > > You could have a task that is blocked in an RCU leaf node, but the
> > > > force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
> > > > CPU could be dyntick-idle and a quiescent state is reported. However, the GP
> > > > must not end and the rcu leaf node should still be present in its parent
> > > > intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
> > > > any relevance.
> > > > 
> > > > Or am I missing the point of the question?
> > > 
> > > > Hello, Qiang,
> > > 
> > > > Another way of making Joel's point is to say that the additional check
> > > > you are asking for is already being done, but by rcu_report_qs_rnp().
> > > 
> > > >                                                         Thanx, Paul
> > > 
> > > Hello Pual,  Joel
> > > 
> > > What I want to express is as follows :
> > > 
> > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > > index 7623128d0020..beb554539f01 100644
> > > --- a/kernel/rcu/tree.c
> > > +++ b/kernel/rcu/tree.c
> > > @@ -2622,6 +2622,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
> > >                  if (mask != 0) {
> > >                          /* Idle/offline CPUs, report (releases rnp->lock). */
> > >                          rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
> > > +                       raw_spin_lock_irqsave_rcu_node(rnp, flags);
> > > +                       if (rnp->qsmask == 0 && rcu_preempt_blocked_readers_cgp(rnp))
> > > +                               rcu_initiate_boost(rnp, flags);
> > > +                       else
> > > +                               raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
> > >                  } else {
> > >                          /* Nothing to do here, so just drop the lock. */
> > >                          raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
> > 
> > But in that case, why duplicate the code from rcu_initiate_boost()?
> > 
> > 							Thanx, Paul
> > 
> 
> Hello Paul
> 
> When we force a qs for rnp, we first check the leaf node "rnp->qsmask" if it
> is reached zero, will check if there are some blocked readers in this leaf
> rnp node, if so we need to priority-boost blocked readers.
> if not we will check cpu dyntick-idle and report leaf node qs, after this
> leaf rnp node report qs, there is may be some blocked readers in this node,
> should we also need to priority-boost blocked readers?

Yes, but we will do that on the next time around, a few milliseconds
later.  And by that time, it is quite possible that the reader will have
completed, which will save us from having to priority-boost it.

							Thanx, Paul

Re: 回复: RCU: Question on force_qs_rnp

[Zhang,Qiang]

On 9/16/20 2:06 AM, Paul E. McKenney wrote:
> On Tue, Sep 15, 2020 at 01:16:39PM +0800, Zhang,Qiang wrote:
>>
>>
>> On 9/15/20 11:41 AM, Paul E. McKenney wrote:
>>> On Tue, Sep 15, 2020 at 03:18:23AM +0000, Zhang, Qiang wrote:
>>>>
>>>>
>>>> ________________________________________
>>>> 发件人: Paul E. McKenney <paulmck@kernel.org>
>>>> 发送时间: 2020年9月15日 4:56
>>>> 收件人: Joel Fernandes
>>>> 抄送: Zhang, Qiang; Uladzislau Rezki; josh@joshtriplett.org; rostedt@goodmis.org; mathieu.desnoyers@efficios.com; Lai Jiangshan; rcu@vger.kernel.org; LKML
>>>> 主题: Re: RCU: Question on force_qs_rnp
>>>>
>>>> On Mon, Sep 14, 2020 at 03:42:08PM -0400, Joel Fernandes wrote:
>>>>> On Mon, Sep 14, 2020 at 07:55:18AM +0000, Zhang, Qiang wrote:
>>>>>> Hello Paul
>>>>>>
>>>>>> I have some questions for you .
>>>>>> in force_qs_rnp func ,  if  "f(rdp)" func return true we will call rcu_report_qs_rnp func
>>>>>> report a quiescent state for this rnp node, and clear grpmask form rnp->qsmask.
>>>>>> after that ,  can we make a check for this rnp->qsmask,  if  rnp->qsmask == 0,
>>>>>> we will check blocked readers in this rnp node,  instead of jumping directly to the next node .
>>>>>
>>>>> Could you clarify what good is this going to do? What problem are you trying to
>>>>> address?
>>>>>
>>>>> You could have a task that is blocked in an RCU leaf node, but the
>>>>> force_qs_rnp() decided to call rcu_report_qs_rnp(). This is perfectly Ok. The
>>>>> CPU could be dyntick-idle and a quiescent state is reported. However, the GP
>>>>> must not end and the rcu leaf node should still be present in its parent
>>>>> intermediate nodes ->qsmask. In this case, the ->qsmask == 0 does not have
>>>>> any relevance.
>>>>>
>>>>> Or am I missing the point of the question?
>>>>
>>>>> Hello, Qiang,
>>>>
>>>>> Another way of making Joel's point is to say that the additional check
>>>>> you are asking for is already being done, but by rcu_report_qs_rnp().
>>>>
>>>>>                                                          Thanx, Paul
>>>>
>>>> Hello Pual,  Joel
>>>>
>>>> What I want to express is as follows :
>>>>
>>>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
>>>> index 7623128d0020..beb554539f01 100644
>>>> --- a/kernel/rcu/tree.c
>>>> +++ b/kernel/rcu/tree.c
>>>> @@ -2622,6 +2622,11 @@ static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
>>>>                   if (mask != 0) {
>>>>                           /* Idle/offline CPUs, report (releases rnp->lock). */
>>>>                           rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
>>>> +                       raw_spin_lock_irqsave_rcu_node(rnp, flags);
>>>> +                       if (rnp->qsmask == 0 && rcu_preempt_blocked_readers_cgp(rnp))
>>>> +                               rcu_initiate_boost(rnp, flags);
>>>> +                       else
>>>> +                               raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>>>>                   } else {
>>>>                           /* Nothing to do here, so just drop the lock. */
>>>>                           raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
>>>
>>> But in that case, why duplicate the code from rcu_initiate_boost()?
>>>
>>> 							Thanx, Paul
>>>
>>
>> Hello Paul
>>
>> When we force a qs for rnp, we first check the leaf node "rnp->qsmask" if it
>> is reached zero, will check if there are some blocked readers in this leaf
>> rnp node, if so we need to priority-boost blocked readers.
>> if not we will check cpu dyntick-idle and report leaf node qs, after this
>> leaf rnp node report qs, there is may be some blocked readers in this node,
>> should we also need to priority-boost blocked readers?
> 
> Yes, but we will do that on the next time around, a few milliseconds
> later.  And by that time, it is quite possible that the reader will have
> completed, which will save us from having to priority-boost it.
> 
> 							Thanx, Paul
> 

Thanks Paul, I see.