smp_call_function() friends and irq/bottom_half context

smp_call_function() friends and irq/bottom_half context

[류호은]

Hi, all.

I'm reading kernel/smp.c code and I found comments on smp_call_function()
and smp_call_function_[single/many]
saying that these functions are cannot be called in interrupt disabled
context or irq/bottom half handlers.

I understand that there is a potential deadlock issue when caller CPU of
the functions is waiting for the completion of the callback of other CPUs.
But I was wondering if this is the case even when the caller CPU doesn't
wait for the completion (wait == 0).

Thank you.

Re: smp_call_function() friends and irq/bottom_half context

[Peter Zijlstra]

On Thu, May 10, 2018 at 01:58:29PM +0900, ��ȣ�� wrote:
> Hi, all.
> 
> I'm reading kernel/smp.c code and I found comments on smp_call_function()
> and smp_call_function_[single/many]
> saying that these functions are cannot be called in interrupt disabled
> context or irq/bottom half handlers.
> 
> I understand that there is a potential deadlock issue when caller CPU of
> the functions is waiting for the completion of the callback of other CPUs.
> But I was wondering if this is the case even when the caller CPU doesn't
> wait for the completion (wait == 0).

IIRC yes, because csd_lock(). You can however use
smp_call_function_single_async() with your own csd. Be very careful
though, it is very easy to construct deadlocks.

RE: smp_call_function() friends and irq/bottom_half context

[Hoeun Ryu]

> -----Original Message-----
> From: Peter Zijlstra [mailto:peterz@infradead.org]
> Sent: Thursday, May 10, 2018 7:17 PM
> To: ��ȣ�� <hoeun.ryu@lge.com>
> Cc: mingo@kernel.org; aaron.lu@intel.com; adobriyan@gmail.com;
> frederic@kernel.org; ying.huang@intel.com; linux-kernel@vger.kernel.org
> Subject: Re: smp_call_function() friends and irq/bottom_half context
> 
> On Thu, May 10, 2018 at 01:58:29PM +0900, ��ȣ�� wrote:
> > Hi, all.
> >
> > I'm reading kernel/smp.c code and I found comments on smp_call_function()
> > and smp_call_function_[single/many]
> > saying that these functions are cannot be called in interrupt disabled
> > context or irq/bottom half handlers.
> >
> > I understand that there is a potential deadlock issue when caller CPU of
> > the functions is waiting for the completion of the callback of other
> CPUs.
> > But I was wondering if this is the case even when the caller CPU doesn't
> > wait for the completion (wait == 0).
> 
> IIRC yes, because csd_lock(). You can however use
> smp_call_function_single_async() with your own csd. Be very careful
> though, it is very easy to construct deadlocks.

Thank you for the explanation and the suggestion.

Would you please explain more on csd lock and irq disabled or irq/bh context ?
How can deadlocks happen when calling smp_call_function(wait=0) with irq disabled ?
How can deadlocks happen when calling smp_call_function(wait=0) from irq or bottom half context ?

I think it as 2d array like call_single_data[caller cpu][target cpu].flags.
csd_lock(csd[caller][target]) spins on the flag only when LOCKED is already set
which means that the caller CPU already called smp_call_function(wait=0)
and at the second calling of the function, the caller CPU spins on
csd_lock(csd[caller][target]) but the target CPU is still processing its callback list
(maybe processing other csd[other cpu][target]) and it does not yet process
csd[caller][target]. But the callback list will be finally flushed and LOCKED flags
is cleared of csd[caller][target] eventually ?