Re: [PATCH] generic-ipi: fix deadlock in __smp_call_function_single

[Heiko Carstens <heiko.carstens@de.ibm.com>]

On Sat, Sep 11, 2010 at 09:42:16AM -0700, Venkatesh Pallipadi wrote:
> On Sat, Sep 11, 2010 at 2:20 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> > On Fri, 2010-09-10 at 17:28 -0700, Andrew Morton wrote:
> >> Where is this scheduler bug?  Did it occur because someone didn't
> >> understand __smp_call_function_single()?  Or did it occur because the
> >> scheduler code is doing something which its implementors did not expect
> >> or intend?
> >
> >
> > It comes from 83cd4fe2 (sched: Change nohz idle load balancing logic to
> > push model), where nohz_balance_kick() simply needs to kick the
> > designated driver into action.
> >
> > I take it Venki assumed __smp_call_function_single() works like
> > smp_call_function_single() where you can use it for the local cpu as
> > well.
> 
> Yes. This was an oversight while moving from using send_remote_softirq
> to using __smp_call_function_single.
> Also, as we don't have rq lock around this point, it seems possible
> that the CPU that was busy and wants to kick idle load balance on
> remote CPU, could have become idle and nominated itself as idle load
> balancer.
> 
> Below patch looks good to me.
> 
> Acked-by: Venkatesh Pallipadi <venki@google.com>
> 
> I guess, we also need a WARN_ON_ONCE for (cpu == smp_processor_id())
> in __smp_call_function_single(), as the eventual result of this bug
> that Heiko saw was a deadlock

Either that or my generic IPI patch should be applied. At least to me
it was rather surprising to see that smp_call_function_single() and
__smp_call_function_single() behave differently when the 'remote' cpu
is the current cpu.