RPi4 U-Boot freeze

From: Stefan Agner <stefan@agner.ch>
To: u-boot@lists.denx.de
Subject: RPi4 U-Boot freeze
Date: Sun, 23 Aug 2020 19:06:39 +0200	[thread overview]
Message-ID: <f96631871ef577ada93b3abd6c1a1140@agner.ch> (raw)

Hi,

I noticed a quite common freeze when running 32-bit U-Boot 2020.01
(rpi_4_32b_defconfig) on a 2GB RPi4 model:

U-Boot 2020.01 (Aug 07 2020 - 13:00:23 +0000)

DRAM:  1.9 GiB
<freeze, no more output>

This happens fairly often, I would say 4 out of 5 boot tries. However,
if it boots, everything seems to run fine.

The issue seems to go away when using 2020.04 or any newer release,
however, when trying to find the actual patch fixing the issue using git
bisect I ended up with a MMC merge request which really seems unrelated
(36bdcf7f3b). It seems that the problem is quite evasive and disappears
if certain structure are aligned differently etc.

Enabling initcall debugging showed that U-Boot crashes right after
relocation:

...
initcall: 00016f2c

RAM Configuration:
Bank #0: 0 948 MiB
Bank #1: 40000000 1 GiB
Bank #2: 0 0 Bytes
Bank #3: 0 0 Bytes

DRAM:  1.9 GiB
initcall: 00016bb8
New Stack Pointer is: 3af6d9e0
initcall: 00016da4
initcall: 00016ef0
initcall: 00016ef8
initcall: 00016d38
Relocation Offset is: 3b375000
Relocating to 3b37d000, new gd at 3af78ec0, sp at 3af6d9e0
initcall: 00016ec8 [clear_bss]
initcall: 0004465c [display_options?? only appears sometimes]
<freeze>

I realized when using CONFIG_SYS_(I|D)CACHE_OFF=y the problem seems to
disappear. But to be 100% certain that it is cache related, I used my
original configuration (which is known to "reliably" freeze), and
replaced 00016ec8 with 00008688 manually in the binary, essentially
swapping out function pointers in "init_sequence_f" [00008688 is
cleanup_before_linux, which flushes and disables I-cache/D-cache]. And
indeed, that hacked up binary does boot reliably every time:

...
initcall: 00016f2c

RAM Configuration:
Bank #0: 0 948 MiB
Bank #1: 40000000 1 GiB
Bank #2: 0 0 Bytes
Bank #3: 0 0 Bytes

DRAM:  1.9 GiB
initcall: 00016bb8
New Stack Pointer is: 3af6d9e0
initcall: 00016da4
initcall: 00016ef0
initcall: 00016ef8
initcall: 00016d38
Relocation Offset is: 3b375000
Relocating to 3b37d000, new gd at 3af78ec0, sp at 3af6d9e0
initcall: 00008688
initcall: 3b38c10c
initcall: 3b38c114
initcall: 000172e0 (relocated to 3b38c2e0)
initcall: 0001712c (relocated to 3b38c12c)
...

From what I understand on RPi4 caches are enabled when entering U-Boot.
I was wondering if the relocation code really can handle that?

--
Stefan