U-Boot failures on CM4 and Pi4-8Gb

U-Boot failures on CM4 and Pi4-8Gb

[Dave Jones]

Hi,

We've recently run across some failures with U-Boot on a couple of 
platforms, specifically the relatively new Pi Compute Module 4, and the 
Pi 4 8GB of RAM. On the CM4, U-Boot failed to identify the eMMC storage, 
and on the Pi 4 8GB, a crash and reset occurred when U-Boot tried to 
read from the boot file-system on the SD card, but only when booted with 
no monitor attached (attaching a monitor results in a successful boot, 
at least under arm64). These were with more-or-less stock 
rpi_4_defconfig and rpi_4_32b_defconfig configurations from v2020.10 
(our local config changes bump the env-size, enable OF_BOARD, and 
RAW_INITRD, but that's it).

I've bisected the source and figured out roughly the commits 
responsible; for the CM4 eMMC case it's the enabling of DMA for the 
SDHCI interface in c6b9fbf756.

The Pi 4 8GB failure was a little more complex in that the bisection 
pointed to 3113c84ba2 (a merge), however that gave me a few clues to try 
and after a little experimenting I found that disabling 
CONFIG_PCI_BRCMSTB (effectively disabling the PCI driver for the 
platform) got things working happily again.

I'm happy to submit the local patches I'm adding to the Ubuntu build to 
work around these, but obviously those fixes are "brute force and 
ignorance" patches that just disable the relevant config; would such 
patches be wanted, or would the community prefer to figure out the root 
causes in each case? I'm happy to test any patches on the relevant 
hardware if anyone doesn't have access to the necessary boards.

Best regards,

Dave Jones.

U-Boot failures on CM4 and Pi4-8Gb

[Matthias Brugger]

Hi Dave,

On 16/12/2020 15:23, Dave Jones wrote:
> Hi,
> 
> We've recently run across some failures with U-Boot on a couple of platforms,
> specifically the relatively new Pi Compute Module 4, and the Pi 4 8GB of RAM. On
> the CM4, U-Boot failed to identify the eMMC storage, and on the Pi 4 8GB, a
> crash and reset occurred when U-Boot tried to read from the boot file-system on
> the SD card, but only when booted with no monitor attached (attaching a monitor
> results in a successful boot, at least under arm64). These were with
> more-or-less stock rpi_4_defconfig and rpi_4_32b_defconfig configurations from
> v2020.10 (our local config changes bump the env-size, enable OF_BOARD, and
> RAW_INITRD, but that's it).
> 
> I've bisected the source and figured out roughly the commits responsible; for
> the CM4 eMMC case it's the enabling of DMA for the SDHCI interface in c6b9fbf756.
> 
> The Pi 4 8GB failure was a little more complex in that the bisection pointed to
> 3113c84ba2 (a merge), however that gave me a few clues to try and after a little
> experimenting I found that disabling CONFIG_PCI_BRCMSTB (effectively disabling
> the PCI driver for the platform) got things working happily again.
> 
> I'm happy to submit the local patches I'm adding to the Ubuntu build to work
> around these, but obviously those fixes are "brute force and ignorance" patches
> that just disable the relevant config; would such patches be wanted, or would
> the community prefer to figure out the root causes in each case? I'm happy to
> test any patches on the relevant hardware if anyone doesn't have access to the
> necessary boards.
> 

Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
mind to test this series, if this fixes your problems:
https://patchwork.ozlabs.org/user/todo/uboot/?series=220661

Regards,
Matthias

U-Boot failures on CM4 and Pi4-8Gb

[Peter Robinson]

On Wed, Dec 16, 2020 at 4:15 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
>
> Hi Dave,
>
> On 16/12/2020 15:23, Dave Jones wrote:
> > Hi,
> >
> > We've recently run across some failures with U-Boot on a couple of platforms,
> > specifically the relatively new Pi Compute Module 4, and the Pi 4 8GB of RAM. On
> > the CM4, U-Boot failed to identify the eMMC storage, and on the Pi 4 8GB, a
> > crash and reset occurred when U-Boot tried to read from the boot file-system on
> > the SD card, but only when booted with no monitor attached (attaching a monitor
> > results in a successful boot, at least under arm64). These were with
> > more-or-less stock rpi_4_defconfig and rpi_4_32b_defconfig configurations from
> > v2020.10 (our local config changes bump the env-size, enable OF_BOARD, and
> > RAW_INITRD, but that's it).
> >
> > I've bisected the source and figured out roughly the commits responsible; for
> > the CM4 eMMC case it's the enabling of DMA for the SDHCI interface in c6b9fbf756.
> >
> > The Pi 4 8GB failure was a little more complex in that the bisection pointed to
> > 3113c84ba2 (a merge), however that gave me a few clues to try and after a little
> > experimenting I found that disabling CONFIG_PCI_BRCMSTB (effectively disabling
> > the PCI driver for the platform) got things working happily again.
> >
> > I'm happy to submit the local patches I'm adding to the Ubuntu build to work
> > around these, but obviously those fixes are "brute force and ignorance" patches
> > that just disable the relevant config; would such patches be wanted, or would
> > the community prefer to figure out the root causes in each case? I'm happy to
> > test any patches on the relevant hardware if anyone doesn't have access to the
> > necessary boards.
> >
>
> Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
> PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
> mind to test this series, if this fixes your problems:
> https://patchwork.ozlabs.org/user/todo/uboot/?series=220661

That's your todo list so the link doesn't work for an anonymous consumer.

U-Boot failures on CM4 and Pi4-8Gb

[Matthias Brugger]

On 16/12/2020 17:20, Peter Robinson wrote:
> On Wed, Dec 16, 2020 at 4:15 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
>>
>> Hi Dave,
>>
>> On 16/12/2020 15:23, Dave Jones wrote:
>>> Hi,
>>>
>>> We've recently run across some failures with U-Boot on a couple of platforms,
>>> specifically the relatively new Pi Compute Module 4, and the Pi 4 8GB of RAM. On
>>> the CM4, U-Boot failed to identify the eMMC storage, and on the Pi 4 8GB, a
>>> crash and reset occurred when U-Boot tried to read from the boot file-system on
>>> the SD card, but only when booted with no monitor attached (attaching a monitor
>>> results in a successful boot, at least under arm64). These were with
>>> more-or-less stock rpi_4_defconfig and rpi_4_32b_defconfig configurations from
>>> v2020.10 (our local config changes bump the env-size, enable OF_BOARD, and
>>> RAW_INITRD, but that's it).
>>>
>>> I've bisected the source and figured out roughly the commits responsible; for
>>> the CM4 eMMC case it's the enabling of DMA for the SDHCI interface in c6b9fbf756.
>>>
>>> The Pi 4 8GB failure was a little more complex in that the bisection pointed to
>>> 3113c84ba2 (a merge), however that gave me a few clues to try and after a little
>>> experimenting I found that disabling CONFIG_PCI_BRCMSTB (effectively disabling
>>> the PCI driver for the platform) got things working happily again.
>>>
>>> I'm happy to submit the local patches I'm adding to the Ubuntu build to work
>>> around these, but obviously those fixes are "brute force and ignorance" patches
>>> that just disable the relevant config; would such patches be wanted, or would
>>> the community prefer to figure out the root causes in each case? I'm happy to
>>> test any patches on the relevant hardware if anyone doesn't have access to the
>>> necessary boards.
>>>
>>
>> Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
>> PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
>> mind to test this series, if this fixes your problems:
>> https://patchwork.ozlabs.org/user/todo/uboot/?series=220661
> 
> That's your todo list so the link doesn't work for an anonymous consumer.
> 

Right, sorry not my day today:
https://patchwork.ozlabs.org/project/uboot/list/?series=220661

Regards,
Matthias

U-Boot failures on CM4 and Pi4-8Gb

[Peter Robinson]

On Wed, Dec 16, 2020 at 4:21 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
>
>
>
> On 16/12/2020 17:20, Peter Robinson wrote:
> > On Wed, Dec 16, 2020 at 4:15 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
> >>
> >> Hi Dave,
> >>
> >> On 16/12/2020 15:23, Dave Jones wrote:
> >>> Hi,
> >>>
> >>> We've recently run across some failures with U-Boot on a couple of platforms,
> >>> specifically the relatively new Pi Compute Module 4, and the Pi 4 8GB of RAM. On
> >>> the CM4, U-Boot failed to identify the eMMC storage, and on the Pi 4 8GB, a
> >>> crash and reset occurred when U-Boot tried to read from the boot file-system on
> >>> the SD card, but only when booted with no monitor attached (attaching a monitor
> >>> results in a successful boot, at least under arm64). These were with
> >>> more-or-less stock rpi_4_defconfig and rpi_4_32b_defconfig configurations from
> >>> v2020.10 (our local config changes bump the env-size, enable OF_BOARD, and
> >>> RAW_INITRD, but that's it).
> >>>
> >>> I've bisected the source and figured out roughly the commits responsible; for
> >>> the CM4 eMMC case it's the enabling of DMA for the SDHCI interface in c6b9fbf756.
> >>>
> >>> The Pi 4 8GB failure was a little more complex in that the bisection pointed to
> >>> 3113c84ba2 (a merge), however that gave me a few clues to try and after a little
> >>> experimenting I found that disabling CONFIG_PCI_BRCMSTB (effectively disabling
> >>> the PCI driver for the platform) got things working happily again.
> >>>
> >>> I'm happy to submit the local patches I'm adding to the Ubuntu build to work
> >>> around these, but obviously those fixes are "brute force and ignorance" patches
> >>> that just disable the relevant config; would such patches be wanted, or would
> >>> the community prefer to figure out the root causes in each case? I'm happy to
> >>> test any patches on the relevant hardware if anyone doesn't have access to the
> >>> necessary boards.
> >>>
> >>
> >> Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
> >> PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
> >> mind to test this series, if this fixes your problems:
> >> https://patchwork.ozlabs.org/user/todo/uboot/?series=220661
> >
> > That's your todo list so the link doesn't work for an anonymous consumer.
> >
>
> Right, sorry not my day today:
> https://patchwork.ozlabs.org/project/uboot/list/?series=220661

I see a crash even with that series on both the 8Gb and 4Gb models, I
was going to reply on the thread but I've not had a chance to look
closer into it. I'm not sure if they're identical but the details of
the two are below:

U-Boot 2021.01-rc3 (Dec 15 2020 - 00:00:00 +0000)

DRAM:  3.9 GiB
RPI 4 Model B (0xc03111)
MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   eth0: ethernet at 7d580000
PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
starting USB...
Bus xhci_pci: Register 5000420 NbrPorts 5
Starting the controller
Port not available.
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found EFI removable media binary efi/fedora/grubaa64.efi
Card did not respond to voltage select! : -110
Scanning disk mmcnr at 7e300000.blk...
Disk mmcnr at 7e300000.blk not ready
Scanning disk emmc2 at 7e340000.blk...
Found 4 disks
No EFI system partition
BootOrder not defined
EFI boot manager: Cannot load any image
EFI LOAD FAILED: continuing...
Card did not respond to voltage select! : -110
starting USB...
Bus xhci_pci: Register 5000420 NbrPorts 5
Starting the controller
Port not available.
USB is stopped. Please issue 'usb start' first.
starting USB...
Bus xhci_pci: "Synchronous Abort" handler, esr 0x96000004
elr: 00000000000b9bfc lr : 00000000000b9be8 (reloc)
elr: 000000003df91bfc lr : 000000003df91be8
x0 : 000000003dc3c040 x1 : 0000000000000000
x2 : 000000003dc3e000 x3 : 29e0ff7f6b43c03d
x4 : 0000000000000000 x5 : 0000000000000000
x6 : 0000000000000000 x7 : 000000003dee8600
x8 : fffffffffffffff0 x9 : 0000000000000008
x10: 0000000000000648 x11: 000000003db471dc
x12: 0000000000000130 x13: 000000003db47198
x14: 000000003db48000 x15: 0000000000000002
x16: 000000003df7fff4 x17: 0b639001e8370785
x18: 000000003db53da0 x19: 000000003db67f80
x20: 000000000000000c x21: 0000000600000020
x22: 000000003dbf0c50 x23: 00000000000002aa
x24: 0000000000001550 x25: 000000003dc3e000
x26: 00000000002aa000 x27: 00000000002aa000
x28: 000000003db67da0 x29: 000000003db471d0

Code: f9400263 1ad42022 f94002c0 8b020322 (f9405063)
Resetting CPU ...

resetting ...



U-Boot 2021.01-rc3 (Dec 15 2020 - 21:04:13 +0000)

DRAM:  7.9 GiB
RPI 4 Model B (0xd03114)
MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   eth0: ethernet at 7d580000
PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
starting USB...
Bus xhci_pci: probe failed, error -110
No working controllers found
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
"Synchronous Abort" handler, esr 0x96000004
elr: 000000000009c180 lr : 000000000009c734 (reloc)
elr: 000000003df74180 lr : 000000003df74734
x0 : 6d6d20676e696e6e x1 : 000000000000005c
x2 : 0000000000800000 x3 : 000000003dfd23e8
x4 : 0000000000000020 x5 : 000000000000005c
x6 : 000000003dfd23e8 x7 : fffffffffffffff0
x8 : fffffffffffffff0 x9 : 0000000000000008
x10: 000000003db673d0 x11: 000000003db62f40
x12: 0000000000000000 x13: 0000000000000200
x14: 000000003db47fe0 x15: 0000000000000021
x16: 000000003df76248 x17: 30c67e855f36071b
x18: 000000003db53da0 x19: 0000000000000030
x20: 000000003dfd1e28 x21: 0000000000000020
x22: 000000003db66af0 x23: 000000003dfd1e28
x24: 0000000000000000 x25: 0000000000000030
x26: 000000003db65a70 x27: 0000000000000003
x28: 000000003db65aa0 x29: 000000003db46bf0

Code: eb03005f 54ffff43 f9400ca4 17ffffe0 (f9400404)
Resetting CPU ...

U-Boot failures on CM4 and Pi4-8Gb

[Peter Robinson]

On Wed, Dec 16, 2020 at 4:27 PM Peter Robinson <pbrobinson@gmail.com> wrote:
>
> On Wed, Dec 16, 2020 at 4:21 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
> >
> >
> >
> > On 16/12/2020 17:20, Peter Robinson wrote:
> > > On Wed, Dec 16, 2020 at 4:15 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
> > >>
> > >> Hi Dave,
> > >>
> > >> On 16/12/2020 15:23, Dave Jones wrote:
> > >>> Hi,
> > >>>
> > >>> We've recently run across some failures with U-Boot on a couple of platforms,
> > >>> specifically the relatively new Pi Compute Module 4, and the Pi 4 8GB of RAM. On
> > >>> the CM4, U-Boot failed to identify the eMMC storage, and on the Pi 4 8GB, a
> > >>> crash and reset occurred when U-Boot tried to read from the boot file-system on
> > >>> the SD card, but only when booted with no monitor attached (attaching a monitor
> > >>> results in a successful boot, at least under arm64). These were with
> > >>> more-or-less stock rpi_4_defconfig and rpi_4_32b_defconfig configurations from
> > >>> v2020.10 (our local config changes bump the env-size, enable OF_BOARD, and
> > >>> RAW_INITRD, but that's it).
> > >>>
> > >>> I've bisected the source and figured out roughly the commits responsible; for
> > >>> the CM4 eMMC case it's the enabling of DMA for the SDHCI interface in c6b9fbf756.
> > >>>
> > >>> The Pi 4 8GB failure was a little more complex in that the bisection pointed to
> > >>> 3113c84ba2 (a merge), however that gave me a few clues to try and after a little
> > >>> experimenting I found that disabling CONFIG_PCI_BRCMSTB (effectively disabling
> > >>> the PCI driver for the platform) got things working happily again.
> > >>>
> > >>> I'm happy to submit the local patches I'm adding to the Ubuntu build to work
> > >>> around these, but obviously those fixes are "brute force and ignorance" patches
> > >>> that just disable the relevant config; would such patches be wanted, or would
> > >>> the community prefer to figure out the root causes in each case? I'm happy to
> > >>> test any patches on the relevant hardware if anyone doesn't have access to the
> > >>> necessary boards.
> > >>>
> > >>
> > >> Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
> > >> PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
> > >> mind to test this series, if this fixes your problems:
> > >> https://patchwork.ozlabs.org/user/todo/uboot/?series=220661
> > >
> > > That's your todo list so the link doesn't work for an anonymous consumer.
> > >
> >
> > Right, sorry not my day today:
> > https://patchwork.ozlabs.org/project/uboot/list/?series=220661
>
> I see a crash even with that series on both the 8Gb and 4Gb models, I
> was going to reply on the thread but I've not had a chance to look
> closer into it. I'm not sure if they're identical but the details of
> the two are below:

Also note that is using the Dec 15th firmware from commit ce59305, the
new DTs pushed later yesterday for 5.10 have a different/continuous
crash

> U-Boot 2021.01-rc3 (Dec 15 2020 - 00:00:00 +0000)
>
> DRAM:  3.9 GiB
> RPI 4 Model B (0xc03111)
> MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
> Loading Environment from FAT... *** Warning - bad CRC, using default environment
>
> In:    serial
> Out:   serial
> Err:   serial
> Net:   eth0: ethernet at 7d580000
> PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
> starting USB...
> Bus xhci_pci: Register 5000420 NbrPorts 5
> Starting the controller
> Port not available.
> Hit any key to stop autoboot:  0
> switch to partitions #0, OK
> mmc0 is current device
> Scanning mmc 0:1...
> Found EFI removable media binary efi/fedora/grubaa64.efi
> Card did not respond to voltage select! : -110
> Scanning disk mmcnr at 7e300000.blk...
> Disk mmcnr at 7e300000.blk not ready
> Scanning disk emmc2 at 7e340000.blk...
> Found 4 disks
> No EFI system partition
> BootOrder not defined
> EFI boot manager: Cannot load any image
> EFI LOAD FAILED: continuing...
> Card did not respond to voltage select! : -110
> starting USB...
> Bus xhci_pci: Register 5000420 NbrPorts 5
> Starting the controller
> Port not available.
> USB is stopped. Please issue 'usb start' first.
> starting USB...
> Bus xhci_pci: "Synchronous Abort" handler, esr 0x96000004
> elr: 00000000000b9bfc lr : 00000000000b9be8 (reloc)
> elr: 000000003df91bfc lr : 000000003df91be8
> x0 : 000000003dc3c040 x1 : 0000000000000000
> x2 : 000000003dc3e000 x3 : 29e0ff7f6b43c03d
> x4 : 0000000000000000 x5 : 0000000000000000
> x6 : 0000000000000000 x7 : 000000003dee8600
> x8 : fffffffffffffff0 x9 : 0000000000000008
> x10: 0000000000000648 x11: 000000003db471dc
> x12: 0000000000000130 x13: 000000003db47198
> x14: 000000003db48000 x15: 0000000000000002
> x16: 000000003df7fff4 x17: 0b639001e8370785
> x18: 000000003db53da0 x19: 000000003db67f80
> x20: 000000000000000c x21: 0000000600000020
> x22: 000000003dbf0c50 x23: 00000000000002aa
> x24: 0000000000001550 x25: 000000003dc3e000
> x26: 00000000002aa000 x27: 00000000002aa000
> x28: 000000003db67da0 x29: 000000003db471d0
>
> Code: f9400263 1ad42022 f94002c0 8b020322 (f9405063)
> Resetting CPU ...
>
> resetting ...
>
>
>
> U-Boot 2021.01-rc3 (Dec 15 2020 - 21:04:13 +0000)
>
> DRAM:  7.9 GiB
> RPI 4 Model B (0xd03114)
> MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
> Loading Environment from FAT... *** Warning - bad CRC, using default environment
>
> In:    serial
> Out:   vidconsole
> Err:   vidconsole
> Net:   eth0: ethernet at 7d580000
> PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
> starting USB...
> Bus xhci_pci: probe failed, error -110
> No working controllers found
> Hit any key to stop autoboot:  0
> switch to partitions #0, OK
> mmc0 is current device
> "Synchronous Abort" handler, esr 0x96000004
> elr: 000000000009c180 lr : 000000000009c734 (reloc)
> elr: 000000003df74180 lr : 000000003df74734
> x0 : 6d6d20676e696e6e x1 : 000000000000005c
> x2 : 0000000000800000 x3 : 000000003dfd23e8
> x4 : 0000000000000020 x5 : 000000000000005c
> x6 : 000000003dfd23e8 x7 : fffffffffffffff0
> x8 : fffffffffffffff0 x9 : 0000000000000008
> x10: 000000003db673d0 x11: 000000003db62f40
> x12: 0000000000000000 x13: 0000000000000200
> x14: 000000003db47fe0 x15: 0000000000000021
> x16: 000000003df76248 x17: 30c67e855f36071b
> x18: 000000003db53da0 x19: 0000000000000030
> x20: 000000003dfd1e28 x21: 0000000000000020
> x22: 000000003db66af0 x23: 000000003dfd1e28
> x24: 0000000000000000 x25: 0000000000000030
> x26: 000000003db65a70 x27: 0000000000000003
> x28: 000000003db65aa0 x29: 000000003db46bf0
>
> Code: eb03005f 54ffff43 f9400ca4 17ffffe0 (f9400404)
> Resetting CPU ...

U-Boot failures on CM4 and Pi4-8Gb

[Dave Jones]

Hi Matthias,

On Wed, Dec 16, 2020 at 05:21:50PM +0100, Matthias Brugger wrote:
>
>
>On 16/12/2020 17:20, Peter Robinson wrote:
>> On Wed, Dec 16, 2020 at 4:15 PM Matthias Brugger <matthias.bgg@gmail.com> wrote:
[snip]
>>>
>>> Thanks for looking into this. I'm aware of problems booting CM4 and RPi 400 with
>>> PCI. I wasn't aware that RPi4 8GB problem was related to PCI as well. Would you
>>> mind to test this series, if this fixes your problems:
>>> https://patchwork.ozlabs.org/user/todo/uboot/?series=220661
>>
>> That's your todo list so the link doesn't work for an anonymous consumer.
>>
>
>Right, sorry not my day today:
>https://patchwork.ozlabs.org/project/uboot/list/?series=220661

Thanks for the swift reply! I've tried this patch series on top of our 
v2020.10 build (excluding my hack to disable CONFIG_PCI_BRCMSTB) and 
unfortunately I get similar results to Peter:


     U-Boot 2020.10-dirty (Dec 16 2020 - 17:39:11 +0000)

     DRAM:  7.9 GiB
     RPI 4 Model B (0xd03114)
     MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
     Loading Environment from FAT... *** Warning - bad CRC, using default environment

     In:    serial
     Out:   serial
     Err:   serial
     Net:   eth0: ethernet at 7d580000
     PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
     starting USB...
     Bus xhci_pci: probe failed, error -110
     No working controllers found
     Hit any key to stop autoboot:  0
     switch to partitions #0, OK
     mmc0 is current device
     Scanning mmc 0:1...
     Found extlinux/extlinux.conf
     Invalid filesystem: 0x02400000
     SCRIPT FAILED: continuing...
     "Synchronous Abort" handler, esr 0x96000004
     elr: 00000000000dab8c lr : 000000000009257c (reloc)
     elr: 000000003b3b2b8c lr : 000000003b36a57c
     x0 : 0378696600000005 x1 : 000000003af4a5f0
     x2 : 00000000ffffffd5 x3 : 0000000000000000
     x4 : 0378696600000005 x5 : 0000000000000048
     x6 : 0000000000000021 x7 : 000000003afd6840
     x8 : 0000000000000001 x9 : 0000000000000008
     x10: 0000000000000002 x11: 000000003af46dd0
     x12: 0000000000000000 x13: 0000000000000200
     x14: 000000003af2bf30 x15: 0000000000000021
     x16: 000000003b38ad4c x17: 0000000000000007
     x18: 000000003af37d90 x19: 0000000000000000
     x20: 000000003af4a100 x21: 000000003af4a5f7
     x22: 000000003af4a5f0 x23: 0000000000000000
     x24: 000000003b3e5000 x25: 0000000000000000
     x26: 000000003af49b60 x27: 000000003af4a610
     x28: 000000003af49c00 x29: 000000003af2a7f0

     Code: 3900007f d65f03c0 aa0003e4 d2800003 (38636885)
     Resetting CPU ...

     resetting ...


However, when I tried the patch on top of master (a439136599), things 
worked perfectly:


     U-Boot 2021.01-rc3-00159-ga439136599-dirty (Dec 16 2020 - 20:00:16 
     +0000)

     DRAM:  7.9 GiB
     RPI 4 Model B (0xd03114)
     MMC:   mmcnr at 7e300000: 1, emmc2 at 7e340000: 0
     Loading Environment from FAT... *** Warning - bad CRC, using default environment

     In:    serial
     Out:   serial
     Err:   serial
     Net:   eth0: ethernet at 7d580000
     PCIe BRCM: link up, 5.0 Gbps x1 (SSC)
     starting USB...
     Bus xhci_pci: probe failed, error -110
     No working controllers found
     Hit any key to stop autoboot:  0
     switch to partitions #0, OK
     mmc0 is current device
     Scanning mmc 0:1...
     Found U-Boot script /boot.scr
     2603 bytes read in 35 ms (72.3 KiB/s)
     ## Executing script at 02400000
     8340400 bytes read in 626 ms (12.7 MiB/s)
     Total of 1 halfword(s) were the same
     Decompressing kernel...
     Uncompressed size: 23824896 = 0x16B8A00
     29335440 bytes read in 2167 ms (12.9 MiB/s)
     Booting Ubuntu (with booti) from mmc 0:...
     ## Flattened Device Tree blob at 02600000
        Booting using the fdt blob at 0x2600000
        Using Device Tree in place at 0000000002600000, end 000000000260ee5b

     Starting kernel ...


Just to make certain it was the patch-set responsible for fixing things 
on master, I rebuilt without your patch-set and tried that. However, 
that also worked perfectly (same output as above) so I'm guessing some 
other commit was responsible?

I'm happy to do another bisect run to try and figure out what's fixing 
it if that would be helpful?

Still, given it looks like the next version may "just fix it" anyway 
(though I haven't tested CM4 / 400 on it yet) I'd be content with 
waiting for that to land upstream in Debian and just getting by with my 
hacky patches for now.

Thanks,

Dave