Re: [RFC PATCH] arm64: defconfig: Disable fine-grained task level IRQ time accounting

From: Vladimir Oltean <olteanv@gmail.com>
To: Kurt Kanzenbach <kurt.kanzenbach@linutronix.de>
Cc: Alison Wang <alison.wang@nxp.com>,
	catalin.marinas@arm.com, will@kernel.org, paulmck@kernel.org,
	mw@semihalf.com, leoyang.li@nxp.com, vladimir.oltean@nxp.com,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org,
	Anna-Maria Gleixner <anna-maria@linutronix.de>,
	Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [RFC PATCH] arm64: defconfig: Disable fine-grained task level IRQ time accounting
Date: Mon, 3 Aug 2020 11:16:25 +0300	[thread overview]
Message-ID: <20200803081625.czdfwcpw5emcd4ls@skbuf> (raw)
In-Reply-To: <873654m9zi.fsf@kurt>

On Mon, Aug 03, 2020 at 10:04:01AM +0200, Kurt Kanzenbach wrote:
> On Thu Jul 30 2020, Vladimir Oltean wrote:
> > On Thu, Jul 30, 2020 at 09:23:44AM +0200, Kurt Kanzenbach wrote:
> >> On Wed Jul 29 2020, Vladimir Oltean wrote:
> >> > For more context, here is my original report of the issue:
> >> > https://lkml.org/lkml/2020/6/4/1062
> >> >
> >> > Just like you, I could not reproduce the RCU stalls and system hang on a
> >> > 5.6-rt kernel, just on mainline and derivatives, using the plain
> >> > defconfig.
> >> >
> >> > The issue is not specific to Layerscape or i.MX8, but rather I was able
> >> > to see the same behavior on Marvell Armada 37xx as well as Qualcomm
> >> > MSM8976.
> >> >
> >> > So, while of course I agree that disabling IRQ time accounting for arm64
> >> > isn't a real solution, it isn't by far an exaggerated proposal either.
> >> > Nonetheless, the patch is just a RFC and should be treated as such. We
> >> > are at a loss when it comes to debugging this any further and we would
> >> > appreciate some pointers.
> >> 
> >> Yeah, sure. I'll try to reproduce this issue first. So it triggers with:
> >> 
> >>  * arm64
> >>  * mainline, not -rt kernel
> >>  * opened serial console
> >>  * irq accounting enabled
> >> 
> >> Anything else?
> >> 
> >> Thanks,
> >> Kurt
> >
> > Thanks for giving a helping hand, Kurt. The defconfig should be enough.
> > In the interest of full disclosure, the only arm64 device on which we
> > didn't reproduce this was the 16-core LX2160A. But we did reproduce on
> > that with maxcpus=1 though. And also on msm8976 with all 8 cores booted.
> > Just mentioning this in case you're testing on a 16-core system, you
> > might want to reduce the number a bit.
> 
> OK. I've reproduced it on a Marvell Armada SoC with v5.6 mainline. See
> splats below. Running with irq time accounting enabled, kills the
> machine immediately. However, I'm not getting the possible deadlock
> warnings in 8250 as you did. So that might be unrelated.
> 

Yes, the console lockdep warnings are unrelated. They are discussed
here:
https://lore.kernel.org/lkml/CAHQZ30BnfX+gxjPm1DUd5psOTqbyDh4EJE=2=VAMW_VDafctkA@mail.gmail.com/

> Unfortunately I have no idea what to debug here.
> 
> Thanks,
> Kurt

So, this means we could submit a formal version of this patch? :)

Thanks,
-Vladimir