Re: Boot stall regression from "printk for 5.19" merge

From: "Marek Behún" <kabel@kernel.org>
To: Sergey Senozhatsky <senozhatsky@chromium.org>
Cc: John Ogness <john.ogness@linutronix.de>,
	Petr Mladek <pmladek@suse.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-kernel@vger.kernel.org
Subject: Re: Boot stall regression from "printk for 5.19" merge
Date: Mon, 20 Jun 2022 12:02:34 +0200	[thread overview]
Message-ID: <20220620120234.5a65ac97@thinkpad> (raw)
In-Reply-To: <YrAEUM20n3Rc4aOn@google.com>

On Mon, 20 Jun 2022 14:23:28 +0900
Sergey Senozhatsky <senozhatsky@chromium.org> wrote:

> On (22/06/19 20:49), Marek Behún wrote:
> [..]
> > causes a regression on arm64 (Marvell CN9130-CRB board) where the
> > system boot freezes in most cases (and is unusable until restarted by
> > watchdog), or, in some cases boots, but the console output gets mangled
> > for a while (the serial console spits garbage characters).  
> 
> Can you please try disabling console kthreads and see how the boot
> process goes? Just `return 0` from printk_activate_kthreads() (I think
> this should do the trick).

This indeed makes the problem go away...

> 
> > The stall example:
> > 
> >   ...
> >   [    1.108141] io scheduler mq-deadline registered
> >   [    1.108143] io scheduler kyber registered
> >   [    1.163484] armada-ap806-pinctrl f06f4000.system-controller:pinctrl: registered pinctrl driver
> >   [
> > 
> > Or another:
> > 
> >   ...
> >   [    1.108155] io scheduler mq-deadline registered
> >   [    1.108156] io scheduler kyber registered
> >   [    1.162991] armada-ap806-pinctrl f06f4000.system-controller:pinctrl: registered pinctrl driver
> >   [    1.163361] armada-cp110-pinctrl f2440000.system-controller:pinctrl: registered pinctrl driver
> >   [    1.180827] mv_xor_v2 f0400000.xor: Marvell Version 2 XOR driver
> >   [  
> > 
> > The garbage example:
> > 
> >   ...
> >   [    0.920951] raid6: using neon recovery algorithm
> >   [    0.921228] iommu: Default domain type: Translated
> >   %
> > 
> >           gb@k+cFL/[    4.954974] DSA: tree 0 setup
> >   [    4.955286] cfg80211: Loading compiled-in X.509 certificates for regulatory database  
> 
> This is pretty suspicious. I don't see how console kthreads would
> corrupt the output. I suspect that something else is going on, some
> memory corruption, etc.

Maybe multiple threads are writing to serial registers, or something...

Marek