[PATCH] sched/rt: Do not do push/pull when there is only one CPU

[PATCH] sched/rt: Do not do push/pull when there is only one CPU

[Steven Rostedt]

Daniel Wagner reported a crash on the beaglebone black. This is a
single CPU architecture, and does not have a functional:
arch_send_call_function_single_ipi() and can crash if that is called.

As it only has one CPU, it shouldn't be called, but if the kernel is
compiled for SMP, the push/pull RT scheduling logic now calls it for
irq_work if the one CPU is overloaded, it can use that function to call
itself and crash the kernel.

There's no reason for the push/pull logic to even be called if there's
only one CPU online. Have it bail if it sees that's the case.

Link: http://lkml.kernel.org/r/8c913cc2-b2e3-8c2e-e503-aff1428f8ff5@monom.org
Fixes: 4bdced5c9 ("sched/rt: Simplify the IPI based RT balancing logic")
Cc: stable@vger.kernel.org
Reported-by: Daniel Wagner <wagi@monom.org>
---
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 4056c19ca3f0..50d2f8179f70 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1784,6 +1784,10 @@ static int push_rt_task(struct rq *rq)
 	if (!rq->rt.overloaded)
 		return 0;
 
+	/* If we are the only CPU, don't bother */
+	if (num_online_cpus() == 1)
+		return 0;
+
 	next_task = pick_next_pushable_task(rq);
 	if (!next_task)
 		return 0;
@@ -2038,6 +2042,10 @@ static void pull_rt_task(struct rq *this_rq)
 	if (likely(!rt_overloaded(this_rq)))
 		return;
 
+	/* If we are the only CPU, don't bother */
+	if (num_online_cpus() == 1)
+		return;
+
 	/*
 	 * Match the barrier from rt_set_overloaded; this guarantees that if we
 	 * see overloaded we must also see the rto_mask bit.

Re: [PATCH] sched/rt: Do not do push/pull when there is only one CPU

[Sebastian Andrzej Siewior]

On 2017-12-01 13:32:22 [-0500], Steven Rostedt wrote:
> 
> Daniel Wagner reported a crash on the beaglebone black. This is a
> single CPU architecture, and does not have a functional:
> arch_send_call_function_single_ipi() and can crash if that is called.
> 
> As it only has one CPU, it shouldn't be called, but if the kernel is
> compiled for SMP, the push/pull RT scheduling logic now calls it for
> irq_work if the one CPU is overloaded, it can use that function to call
> itself and crash the kernel.
> 
> There's no reason for the push/pull logic to even be called if there's
> only one CPU online. Have it bail if it sees that's the case.
> 
> Link: http://lkml.kernel.org/r/8c913cc2-b2e3-8c2e-e503-aff1428f8ff5@monom.org
> Fixes: 4bdced5c9 ("sched/rt: Simplify the IPI based RT balancing logic")
> Cc: stable@vger.kernel.org
> Reported-by: Daniel Wagner <wagi@monom.org>
> ---
> diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> index 4056c19ca3f0..50d2f8179f70 100644
> --- a/kernel/sched/rt.c
> +++ b/kernel/sched/rt.c
> @@ -1784,6 +1784,10 @@ static int push_rt_task(struct rq *rq)
>  	if (!rq->rt.overloaded)
>  		return 0;
>  
> +	/* If we are the only CPU, don't bother */
> +	if (num_online_cpus() == 1)
> +		return 0;
> +

what about a check next to sched_feat(RT_PUSH_IPI)? I don't know if this
is a hot path or not (due to bitmap_weight). If it is, then I would
suggest something like a jump-label which is enabled if more than one
CPU has been enabled on boot.

>  	next_task = pick_next_pushable_task(rq);
>  	if (!next_task)
>  		return 0;
> @@ -2038,6 +2042,10 @@ static void pull_rt_task(struct rq *this_rq)
>  	if (likely(!rt_overloaded(this_rq)))
>  		return;
>  
> +	/* If we are the only CPU, don't bother */
> +	if (num_online_cpus() == 1)
> +		return;
> +
>  	/*
>  	 * Match the barrier from rt_set_overloaded; this guarantees that if we
>  	 * see overloaded we must also see the rto_mask bit.

Sebastian

Re: [PATCH] sched/rt: Do not do push/pull when there is only one CPU

[Peter Zijlstra]

On Sat, Dec 02, 2017 at 01:53:31PM +0100, Sebastian Andrzej Siewior wrote:
> On 2017-12-01 13:32:22 [-0500], Steven Rostedt wrote:
> > 
> > Daniel Wagner reported a crash on the beaglebone black. This is a
> > single CPU architecture, and does not have a functional:
> > arch_send_call_function_single_ipi() and can crash if that is called.
> > 
> > As it only has one CPU, it shouldn't be called, but if the kernel is
> > compiled for SMP, the push/pull RT scheduling logic now calls it for
> > irq_work if the one CPU is overloaded, it can use that function to call
> > itself and crash the kernel.
> > 
> > There's no reason for the push/pull logic to even be called if there's
> > only one CPU online. Have it bail if it sees that's the case.
> > 
> > Link: http://lkml.kernel.org/r/8c913cc2-b2e3-8c2e-e503-aff1428f8ff5@monom.org
> > Fixes: 4bdced5c9 ("sched/rt: Simplify the IPI based RT balancing logic")
> > Cc: stable@vger.kernel.org
> > Reported-by: Daniel Wagner <wagi@monom.org>
> > ---
> > diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> > index 4056c19ca3f0..50d2f8179f70 100644
> > --- a/kernel/sched/rt.c
> > +++ b/kernel/sched/rt.c
> > @@ -1784,6 +1784,10 @@ static int push_rt_task(struct rq *rq)
> >  	if (!rq->rt.overloaded)
> >  		return 0;
> >  
> > +	/* If we are the only CPU, don't bother */
> > +	if (num_online_cpus() == 1)
> > +		return 0;
> > +
> 
> what about a check next to sched_feat(RT_PUSH_IPI)? I don't know if this
> is a hot path or not (due to bitmap_weight). If it is, then I would
> suggest something like a jump-label which is enabled if more than one
> CPU has been enabled on boot.

Yeah good point; bitmap_weight can be quite expensive.

Re: [PATCH] sched/rt: Do not do push/pull when there is only one CPU

[Steven Rostedt]

On Sat, 2 Dec 2017 13:53:31 +0100
Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:

> what about a check next to sched_feat(RT_PUSH_IPI)? I don't know if this
> is a hot path or not (due to bitmap_weight). If it is, then I would
> suggest something like a jump-label which is enabled if more than one
> CPU has been enabled on boot.

Yeah I didn't like that because of the overhead. But I was being
optimistic that the cpu weight function would be a nit to the actual
pull logic. But I have a better plan.

I would like to disable RT_PUSH_IPI, but that's a sched feature and
that is a constant if we build without sched debugging.

v2 coming up.

-- Steve

Re: [PATCH] sched/rt: Do not do push/pull when there is only one CPU

[Steven Rostedt]

On Sat, 2 Dec 2017 12:12:52 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> v2 coming up.

Not really a v2, but a different approach. Look for the patch with the
subject:

 "[PATCH] sched/rt: Do not pull from current CPU if only one cpu to pull"

-- Steve