[PATCH] sched/rt: Document why has_pushable_tasks() isn't called with a runqueue lock

[PATCH] sched/rt: Document why has_pushable_tasks() isn't called with a runqueue lock

[Steven Rostedt]

From: Steven Rostedt (VMware) <rostedt@goodmis.org>

While reviewing the RT scheduling IPI logic, I was thinking that it was
a bug that has_pushable_tasks(rq) was not called under the runqueue
lock. But then I realized that there isn't a case where a race would
cause a problem, as to update has_pushable_tasks() would trigger a
push_rt_task() call from the CPU doing the update.

This subtle logic deserves a comment.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 4101f9d..f39449b 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1976,6 +1976,16 @@ static void try_to_push_tasks(void *arg)
 	src_rq = rq_of_rt_rq(rt_rq);
 
 again:
+	/*
+	 * Normally, has_pushable_tasks() would be performed within the
+	 * runqueue lock being held. But if it was not set when entering
+	 * this hard interrupt handler function, then to have it set would
+	 * require a wake up. A wake up of an RT task will either cause a
+	 * schedule if the woken task is higher priority than the running
+	 * task, or it would try to do a push from the CPU doing the wake
+	 * up. Grabbing the runqueue lock in such a case would more likely
+	 * just cause unnecessary contention.
+	 */
 	if (has_pushable_tasks(rq)) {
 		raw_spin_lock(&rq->lock);
 		push_rt_task(rq);

Re: [PATCH] sched/rt: Document why has_pushable_tasks() isn't called with a runqueue lock

[Peter Zijlstra]

On Tue, Feb 28, 2017 at 04:48:56PM -0500, Steven Rostedt wrote:

> +	/*
> +	 * Normally, has_pushable_tasks() would be performed within the
> +	 * runqueue lock being held. But if it was not set when entering

"not set" what? I'm having trouble parsing this.

> +	 * this hard interrupt handler function, then to have it set would
> +	 * require a wake up. A wake up of an RT task will either cause a
> +	 * schedule if the woken task is higher priority than the running
> +	 * task, or it would try to do a push from the CPU doing the wake
> +	 * up. Grabbing the runqueue lock in such a case would more likely
> +	 * just cause unnecessary contention.
> +	 */
>  	if (has_pushable_tasks(rq)) {
>  		raw_spin_lock(&rq->lock);
>  		push_rt_task(rq);

Re: [PATCH] sched/rt: Document why has_pushable_tasks() isn't called with a runqueue lock

[Steven Rostedt]

On Wed, 1 Mar 2017 09:37:01 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Tue, Feb 28, 2017 at 04:48:56PM -0500, Steven Rostedt wrote:
> 
> > +	/*
> > +	 * Normally, has_pushable_tasks() would be performed within the
> > +	 * runqueue lock being held. But if it was not set when entering  
> 
> "not set" what? I'm having trouble parsing this.

I always forgot that with documentation, pronouns should be avoided.

"But if has_pushable_tasks is false when entering"

> 
> > +	 * this hard interrupt handler function, then to have it set would

", then to have it set to true would"

> > +	 * require a wake up. A wake up of an RT task will either cause a
> > +	 * schedule if the woken task is higher priority than the running
> > +	 * task, or it would try to do a push from the CPU doing the wake
> > +	 * up. Grabbing the runqueue lock in such a case would more likely
> > +	 * just cause unnecessary contention.
> > +	 */
> >  	if (has_pushable_tasks(rq)) {
> >  		raw_spin_lock(&rq->lock);
> >  		push_rt_task(rq);