Re: threadirqs deadlocks

From: Johan Hovold <johan@kernel.org>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andy Shevchenko <andy.shevchenko@gmail.com>,
	linux-serial@vger.kernel.org, linux-kernel@vger.kernel.org,
	Peter Zijlstra <peterz@infradead.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: Re: threadirqs deadlocks
Date: Wed, 17 Mar 2021 15:00:28 +0100	[thread overview]
Message-ID: <YFILfPT1SFypmOAj@hovoldconsulting.com> (raw)
In-Reply-To: <87eegdzzez.fsf@nanos.tec.linutronix.de>

On Wed, Mar 17, 2021 at 02:24:04PM +0100, Thomas Gleixner wrote:

> On Tue, Mar 16 2021 at 11:56, Johan Hovold wrote:

> > It seems to me that forced interrupt threading cannot generally work
> > without updating drivers that expose locks that can be taken by other
> > interrupt handlers, for example, by using spin_lock_irqsave() in their
> > interrupt handlers or marking their interrupts as IRQF_NO_THREAD.
> 
> The latter is the worst option because that will break PREEMPT_RT.
> 
> > What are your thoughts on this given that forced threading isn't that
> > widely used and was said to be "mostly a debug option". Do we need to
> > vet all current and future drivers and adapt them for "threadirqs"?
> >
> > Note that we now have people sending cleanup patches for interrupt
> > handlers by search-and-replacing spin_lock_irqsave() with spin_lock()
> > which can end up exposing this more.
> 
> It's true that for !RT it's primarily a debug option, but occasionaly a
> very valuable one because it does not take the whole machine down when
> something explodes in an interrupt handler. Used it just a couple of
> weeks ago successfully :)
> 
> So we have several ways out of that:
> 
>   1) Do the lock() -> lock_irqsave() dance
> 
>   2) Delay printing from hard interrupt context (which is what RT does)

While this is probably mostly an issue for console drivers, the problem
is more general and we'd need to identify and add workarounds for any
lock that could be taken by a second interrupt handler.

>   3) Actually disable interrupts before calling the force threaded
>      handler.
> 
> I'd say #3 is the right fix here. It's preserving the !RT semantics
> and the usefulness of threadirqs for debugging and spare us dealing with
> the script kiddies.

I was hoping you'd say that. :) Just wasn't sure whether it would
cripple threadirqs too much.

> Something like the below.

Looks good to me. Do you want to spin that into a patch or shall I do
it after some testing?

Johan