Device Tree Overlays and FIT Images - Are they supported together fully?

Device Tree Overlays and FIT Images - Are they supported together fully?

[Westermann, Oliver]

Hey,

We're supporting a device that has some hardware variants. As those variants can be of A different kinds on B different interfaces, we end up with A*B = N different hardware variations.
Currently we have N device trees in the FIT image, but we noticed that it doesn't scale anymore: Any new hardware variants B+1 causes a number of A new device trees. We decided to invest some time into FDT overlays and managed to get it to work, but not completely to our expectations.

Our current flow is to load the fit image and boot a configuration from the fit image using

bootm ${loadaddr}#${fit_config}

I followed the documentation at doc/uImage.FIT/overlay-fdt-boot.txt & a bit of hacking in my yocto later, I have a working FIT image that has the structure as described in the docs. I can manually copy out & modify dtb's and dtbo's as well as apply them, but I can't do whats described under "Configuration using overlays and feature selection":

If I do
bootm ${loadaddr}#${fit_config}#bar
with bar being a very small dtbo, U-Boot fails to apply the overlay stating "Overlayed FDT requires relocation", output by boot_get_fdt_fit() in boot/image-fit.c#L2341.

Reading around that code, it seems that what is described in the docs is not really supported: I can't find any codepath that would copy the base dtb out of the FIT image area before applying any overlays.

Is my doubt reasonable or am I missing a step somewhere?
I can provide more info (detailed load & boot steps, fit image iminfo) if needed.

- Olli

RE: Device Tree Overlays and FIT Images - Are they supported together fully?

[Westermann, Oliver]

On Tue, 1 Feb 2022 at 10:50, Oliver Westermann <Oliver.Westermann at cognex.com> wrote:

> Hey,
> 
> Is my doubt reasonable or am I missing a step somewhere?
> I can provide more info (detailed load & boot steps, fit image iminfo) if needed.
> 
> - Olli
> 

Hey,

I'm not sure if this is the correct list or my initial information was lacking/incomplete.
I'm currently looking into adding support for automatic relocation and resizing of the base dtb in the function described above.

Is there some documentation or information about the space that can be expected beyond ${loadaddr} in regular usecases or if there is another common patter for the need of restructuring the boot data?

Best regards, Olli

Re: Device Tree Overlays and FIT Images - Are they supported together fully?

[Heinrich Schuchardt]

On 2/11/22 08:19, Westermann, Oliver wrote:
> On Tue, 1 Feb 2022 at 10:50, Oliver Westermann <Oliver.Westermann at cognex.com> wrote:
>
>> Hey,
>>
>> Is my doubt reasonable or am I missing a step somewhere?

You can pass the name of a configuration and overlays to apply to the
bootm command. See doc/uImage.FIT/overlay-fdt-boot.txt

Here are examples of usage:

* configs/am65x_hs_evm_a53_defconfig
* include/configs/ti_armv7_common.h


>> I can provide more info (detailed load & boot steps, fit image iminfo) if needed.
>>
>> - Olli
>>
>
> Hey,
>
> I'm not sure if this is the correct list or my initial information was lacking/incomplete.
> I'm currently looking into adding support for automatic relocation and resizing of the base dtb in the function described above.

Which function?

>
> Is there some documentation or information about the space that can be expected beyond ${loadaddr} in regular usecases or if there is another common patter for the need of restructuring the boot data?

U-Boot relocates itself to the top of memory. The available memory is
shown by command bdinfo.

Run 'printenv' to see which other memory addresses are defined.

Best regards

Heinrich

>
> Best regards, Olli

RE: [EXTERNAL] Re: Device Tree Overlays and FIT Images - Are they supported together fully?

[Westermann, Oliver]

Hey,

On Fri, 11 Feb 2022 at 08:54, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> You can pass the name of a configuration and overlays to apply to the
> bootm command. See doc/uImage.FIT/overlay-fdt-boot.txt
> 
> Here are examples of usage:
> 
> * configs/am65x_hs_evm_a53_defconfig
> * include/configs/ti_armv7_common.h

It seems my first mail got lost, I'm not really used to mailinglists :(
My initial mail got send Tue Feb 1 10:50:48, not sure if you can still find it. It's in the archives at least.

I'm following the doc/uImage.FIT/overlay-fdt-boot.txt and it seems my FIT image (and dtbs contained) are okay, but I've issues with the bootm command:

> Which function?

My current setup uses bootm ${loadaddr}#${fit_config} which works fine, fit_config containing a configuration node name. When calling bootm ${loadaddr}#${fit_config}#bar to apply the bar overlay to ${fit_config}, I get the "Overlayed FDT requires relocation" error from boot/image-fit.c#L2341 out of function boot_get_fdt_fit().

I might be doing something wrong, but I did some classic printf debugging to understand the codepath leading up to this location. I could not find any code that would copy the dtb contained in my configuration out of the fit image, which means the check in L2339 should always trigger. I'm wondering if there is some "copy dtb out of FIT" step in overlay-fdt-boot.txt which I'm not seeing, but I feel like I triple checked that document by now ;-) So for me the base dtb sits in the FIT and u-boot declines to apply an overlay. The check itself makes sense to me as we can't check if there is sufficent padding beyond the base-dtb to prevent growing it into the next blob.

> U-Boot relocates itself to the top of memory. The available memory is
> shown by command bdinfo.
> 
> Run 'printenv' to see which other memory addresses are defined.

Thanks, I will check that for my testing, though I'm still wondering what I'm doing wrong.

Best regards, Olli

Re: [EXTERNAL] Re: Device Tree Overlays and FIT Images - Are they supported together fully?

[Marcel Ziswiler]

Hi Olli

On Fri, 2022-02-11 at 08:13 +0000, Westermann, Oliver wrote:

[snip]

> My current setup uses bootm ${loadaddr}#${fit_config} which works fine, fit_config containing a configuration
> node name. When calling bootm ${loadaddr}#${fit_config}#bar to apply the bar overlay to ${fit_config}, I get
> the "Overlayed FDT requires relocation" error from boot/image-fit.c#L2341 out of function boot_get_fdt_fit().
> 
> I might be doing something wrong, but I did some classic printf debugging to understand the codepath leading
> up to this location. I could not find any code that would copy the dtb contained in my configuration out of
> the fit image, which means the check in L2339 should always trigger. I'm wondering if there is some "copy dtb
> out of FIT" step in overlay-fdt-boot.txt which I'm not seeing, but I feel like I triple checked that document
> by now ;-) So for me the base dtb sits in the FIT and u-boot declines to apply an overlay. The check itself
> makes sense to me as we can't check if there is sufficent padding beyond the base-dtb to prevent growing it
> into the next blob.

I am not sure whether I fully understand the issue(s) you are having. I only know that we are using a similar
technique to actually boot our Toradex Easy Installer. Anyway, I hope you find this helpful.

We do use a script there to get this going. Let me just post the (hopefully) relevant excerpts:

env set fdtfile ${fdt_prefix}imx8mp-verdin-wifi-dev.dtb
env set ramdisk_addr_r 0x47400000

# Re-enable fdt relocation since in place fdt edits corrupt the ramdisk
# in a FIT image...
env set fdt_high
env set fdt_resize true
env set fitconf_fdt_overlays

env set set_default_overlays 'env set fdt_overlays "verdin-imx8mp_native-hdmi_overlay.dtbo verdin-
imx8mp_lt8912_overlay.dtbo"'

env set set_load_overlays_file 'env set load_overlays_file "env import -t 0x42e10000 0x200"'

env set set_apply_overlays 'env set apply_overlays "for overlay_file in \"\\${fdt_overlays}\"; do env set
fitconf_fdt_overlays \"\\"\\${fitconf_fdt_overlays}#config@\\${overlay_file}\\"\"; env set overlay_file; done;
true"'

env set bootcmd_run 'echo "Bootargs: \${bootargs}" && bootm
${ramdisk_addr_r}#config@freescale_\${fdtfile}\${fitconf_fdt_overlays}'

run set_load_overlays_file
run set_apply_overlays

run load_overlays_file
run apply_overlays

run bootcmd

And that is how that whole thing then looks upon boot:

## Executing script at 42e00000
Bootargs: quiet drm.edid_firmware=edid/1280x720.bin video=HDMI-A-1:1280x720-16@6
0D video=HDMI-A-2:1280x720-16@60D initcall_blacklist=vpu_driver_init rootfstype=
squashfs root=/dev/ram autoinstall clk_ignore_unused pci=nomsi
## Loading kernel from FIT Image at 47400000 ...
   Using 'config@freescale_imx8mp-verdin-wifi-dev.dtb' configuration
   Trying 'kernel@1' kernel subimage
     Description:  Linux kernel
     Type:         Kernel Image
     Compression:  gzip compressed
     Data Start:   0x47400108
     Data Size:    10389245 Bytes = 9.9 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x40000000
     Entry Point:  0x40000000
     Hash algo:    sha1
     Hash value:   039972cffe32d2806cff06f64707f56ecb214806
   Verifying Hash Integrity ... sha1+ OK
## Loading ramdisk from FIT Image at 47400000 ...
   Using 'config@freescale_imx8mp-verdin-wifi-dev.dtb' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description:  tezi-initramfs
     Type:         RAMDisk Image
     Compression:  uncompressed
     Data Start:   0x47e440ec
     Data Size:    35405824 Bytes = 33.8 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x60000000
     Entry Point:  unavailable
     Hash algo:    sha1
     Hash value:   d6acbdc4dbce430542b0ebc98cf78c8f1596fb6e
   Verifying Hash Integrity ... sha1+ OK
   Loading ramdisk from 0x47e440ec to 0x60000000
## Loading fdt from FIT Image at 47400000 ...
   Using 'config@freescale_imx8mp-verdin-wifi-dev.dtb' configuration
   Trying 'fdt@freescale_imx8mp-verdin-wifi-dev.dtb' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x47e27800
     Data Size:    86327 Bytes = 84.3 KiB
     Architecture: AArch64
     Load Address: 0x44000000
     Hash algo:    sha1
     Hash value:   f0830ff873080ac797feaea43bcff0c365abfc37
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0x47e27800 to 0x44000000
## Loading fdt from FIT Image at 47400000 ...
   Using 'config@verdin-imx8mp_native-hdmi_overlay.dtbo' configuration
   Trying 'fdt@verdin-imx8mp_native-hdmi_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x47e41e44
     Data Size:    1860 Bytes = 1.8 KiB
     Architecture: AArch64
     Load Address: 0x46000000
     Hash algo:    sha1
     Hash value:   e012dc61089de5a93eecd9cf4b0bb3b6cb58bec6
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0x47e41e44 to 0x46000000
## Loading fdt from FIT Image at 47400000 ...
   Using 'config@verdin-imx8mp_lt8912_overlay.dtbo' configuration
   Trying 'fdt@verdin-imx8mp_lt8912_overlay.dtbo' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x47e3ea40
     Data Size:    1987 Bytes = 1.9 KiB
     Architecture: AArch64
     Load Address: 0x46000000
     Hash algo:    sha1
     Hash value:   5a8add06641fe0b39df350cb658d54d295d69a3a
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0x47e3ea40 to 0x46000000
   Booting using the fdt blob at 0x44000000
   Uncompressing Kernel Image
   Loading Device Tree to 00000000fdbdd000, end 00000000fdbf5245 ... OK

Starting kernel ...

> > U-Boot relocates itself to the top of memory. The available memory is
> > shown by command bdinfo.
> > 
> > Run 'printenv' to see which other memory addresses are defined.
> 
> Thanks, I will check that for my testing, though I'm still wondering what I'm doing wrong.

I hope that gives you some ideas how to overcome your struggle. Good luck!

> Best regards, Olli

Cheers

Marcel

RE: [EXTERNAL] Re: Device Tree Overlays and FIT Images - Are they supported together fully?

[Westermann, Oliver]

Hey Marcel, 

> On Sat, 2022-02-19 at 00:38 +0000, Ziswiler, Marcel wrote: 
> I am not sure whether I fully understand the issue(s) you are having. 

Same here ;-) I did some more work on this, best described as copying a lot of commands from my notepad back and forth. I think you confirmed my underlying issue here:
Your process is "take a FIT, move the dtb out of FIT image, apply the overlays, boot it". The documentation in overlay-fdt-boot.txt never mentions to move the dtb file out of the FIT image and it bugged me a lot that I was doing a (seemingly) unneeded step, but it seems the step is correct and just not documented.

> Let me just post the (hopefully) relevant excerpts:

First of all, this looks like how I got it to work, so this tells me at least I wasn't completly off-track here.

> # Re-enable fdt relocation since in place fdt edits corrupt the ramdisk # in a FIT image...
> env set fdt_high
> env set fdt_resize true

Can you give me a bit more info on your use of the fdt_resize? I know the fdt resize command and I assume thats what you're calling underneath, but I would like to get a better understanding on what is done so I'm not missing anything.

> I hope that gives you some ideas how to overcome your struggle. Good luck!

I did quite a lot, thanks!

- Olli