Re: CONFIG_THUMB2_KERNEL=y boot failure after Spectre BHB fixes

From: Christian Eggers <ceggers@arri.de>
To: Russell King <linux@armlinux.org.uk>,
	<linux-arm-kernel@lists.infradead.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	<linux-arm-kernel@lists.infradead.org>,
	<linux-kernel@vger.kernel.org>, <llvm@lists.linux.dev>,
	Nathan Chancellor <nathan@kernel.org>
Subject: Re: CONFIG_THUMB2_KERNEL=y boot failure after Spectre BHB fixes
Date: Tue, 22 Mar 2022 18:49:17 +0100	[thread overview]
Message-ID: <2109354.irdbgypaU6@localhost.localdomain> (raw)
In-Reply-To: <YipOoAaBIHjeCKOq@dev-arch.thelio-3990X>

Hi Nathan, hi Russel,

I stumbled today over the same problem (no output on serial console
with v5.15.28-rt36). During `git bisect`, I had also some commits
where a few lines of output were visible.

At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of
sections"), the system boots up to here:

start_kernel()
+--setup_arch()
   +--paging_init()
      +--devicemaps_init()
         +--eary_trap_init(vectors_base = 0xC7FFE000)
            +--copy_from_lma(vectors_base = 0xC7FFE000, __vectors_start=0x0, __vectors_end=0x20)
               +--__memcpy()

               copy_template.S:113 
               ldr8w   r1, r3, r4, r5, r6, r7, r8, ip, lr, abort=20f
               r1 = 0

With the final v5.15.28-rt36 I found out that the system boots fine
after disabling CONFIG_HARDEN_BRANCH_HISTORY.

Is there anything else I could analyze? My SoC system is a NXP i.MX6LL
with a ARMv7 core. I have access to a JTAG debugger.

regards
Christian

On Thursday, 10 March 2022, 20:16:48 CET, Nathan Chancellor wrote:
> Hi Russell,
> 
> Apologies if this has already been reported, I did not see anything when
> sifting through lore.kernel.org and I can still reproduce this with
> current mainline (1db333d9a51f).
> 
> I noticed a QEMU boot failure with multi_v7_defconfig with
> CONFIG_THUMB2_KERNEL=y in our continuous integration [1]. It does not
> appear to be compiler specific, as it reproduces with a bunch of
> different clang versions and GCC 11.2.0 (I didn't try other GCC
> versions).
> 
> At commit 04e91b732476 ("ARM: early traps initialisation"), everything
> boots fine.
> 
> At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of
> sections"), there is no output from QEMU at all.
> 
> At commit b9baf5c8c5c3 ("ARM: Spectre-BHB workaround"), there is some
> output but the boot still hangs before init. I have included a log of
> the output of QEMU at this revision along with the command line I am
> using, which comes from [2]. If I disable CONFIG_HARDEN_BRANCH_HISTORY,
> the kernel boots.
> 
> If there is any further information I can provide or patches I can try,
> I am happy to do so.
> 
> [1]: https://github.com/ClangBuiltLinux/continuous-integration2/runs/5496036256?check_suite_focus=true
> [2]: https://github.com/ClangBuiltLinux/boot-utils
> 
> Cheers,
> Nathan
> 