[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Alexander Kurtz]

Hi!

I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3. 

Recently, the Debian kernel grew beyond 16 MiB:

    4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
    4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
    4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]

https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:

    #define ENV_MEM_LAYOUT_SETTINGS \
    "fdt_high=ffffffff\0" \
    "initrd_high=ffffffff\0" \
    "fdt_addr_r=0x00000100\0" \
    "pxefile_addr_r=0x00100000\0" \
    "kernel_addr_r=0x01000000\0" \
    "scriptaddr=0x02000000\0" \
    "ramdisk_addr_r=0x02100000\0" \

Am I correct in assuming that this default configuration will break
with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
wanted to ask for explicit confirmation here.

The background is that ever since the upgrade to 4.14, the kernel has
been failing to unpack the initramfs, complaining about "junk in [the]
compressed archive" and I am currently trying to find where to best
file a bug report about this!

Best regards

Alexander Kurtz

[0] http://snapshot.debian.org/package/linux/4.13.13-1/#linux-image-4.13.0-1-arm64_4.13.13-1
[1] http://snapshot.debian.org/package/linux/4.14.17-1/#linux-image-4.14.0-3-arm64_4.14.17-1
[2] http://snapshot.debian.org/package/linux/4.15.4-1/#linux-image-4.15.0-1-arm64_4.15.4-1
[3] https://github.com/u-boot/u-boot/blob/master/doc/README.distro#L250
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180222/e6799b63/attachment.sig>

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Jaehoon Chung]

On 02/23/2018 04:58 AM, Alexander Kurtz wrote:
> Hi!
> 
> I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
> dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3. 
> 
> Recently, the Debian kernel grew beyond 16 MiB:
> 
>     4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
>     4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
>     4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]
> 
> https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:
> 
>     #define ENV_MEM_LAYOUT_SETTINGS \
>     "fdt_high=ffffffff\0" \
>     "initrd_high=ffffffff\0" \
>     "fdt_addr_r=0x00000100\0" \
>     "pxefile_addr_r=0x00100000\0" \
>     "kernel_addr_r=0x01000000\0" \
>     "scriptaddr=0x02000000\0" \
>     "ramdisk_addr_r=0x02100000\0" \
> 
> Am I correct in assuming that this default configuration will break
> with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
> wanted to ask for explicit confirmation here.

Especially, this is occurred about arm64 kernel.
Because it wil be overlapped with other image address, if kernel image is over than 16MB.

In my case, i'm using the private boot.scr.img for booting script.
I had just added the kernel_addr_r=0x02d000000.

Just Refer to below:
https://review.tizen.org/git/?p=platform/kernel/u-boot.git;a=commit;h=8fdfbbeb8a4b2f8c8e66824709e48d9abdb23534

Best Regards,
Jaehoon Chung

> 
> The background is that ever since the upgrade to 4.14, the kernel has
> been failing to unpack the initramfs, complaining about "junk in [the]
> compressed archive" and I am currently trying to find where to best
> file a bug report about this!
> 
> Best regards
> 
> Alexander Kurtz
> 
> [0] http://snapshot.debian.org/package/linux/4.13.13-1/#linux-image-4.13.0-1-arm64_4.13.13-1
> [1] http://snapshot.debian.org/package/linux/4.14.17-1/#linux-image-4.14.0-3-arm64_4.14.17-1
> [2] http://snapshot.debian.org/package/linux/4.15.4-1/#linux-image-4.15.0-1-arm64_4.15.4-1
> [3] https://github.com/u-boot/u-boot/blob/master/doc/README.distro#L250
> 
> 
> 
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> https://lists.denx.de/listinfo/u-boot
> 

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Alexander Kurtz]

Hi!

Thanks for confirming this! I see four ways how this can be solved:

   1. I can locally adjust the memory locations to make my > 16MiB kernel
      work. The commit mentioned by Jaehoon Chung does this via a custom
      boot.scr, I did this by bumping $scriptaddr and $ramdisk_addr_r
      interactively during bootup and storing the changes to the Micro SD
      card with the "saveenv" command.
   2. I could place a "uEnv.txt on the Micro SD card with those bumped
      values in it. This would make this work-around deployable via
      Puppet, but unfortunately reading "uEnv.txt" on bootup seems to be a
      board-specific feature, and the RPI code currently doesn't seem to
      have this feature. [0]
   3. I could ask Debian to patch the upstream default values for these
      memory locations. After all, a distro bootloader which doesn't boot
      the distro kernel is pretty useless, so I imagine they would
      probably consider this.
   4. U-Boot upstream could adjust the memory locations themselves. I
      would be happy to submit a patch for this, however I want get some
      feedback first: Would you guys accept such a patch, bumping the
      supported kernel size to e.g. 32 MiB or even 64 MiB?

I would prefer 4>3>2>1, what do you think?

Best regards

Alexander Kurtz

[0] https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180223/a4f07ea5/attachment.sig>

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Tom Rini]

On Fri, Feb 23, 2018 at 04:47:06PM +0900, Jaehoon Chung wrote:
> On 02/23/2018 04:58 AM, Alexander Kurtz wrote:
> > Hi!
> > 
> > I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
> > dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3. 
> > 
> > Recently, the Debian kernel grew beyond 16 MiB:
> > 
> >     4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
> >     4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
> >     4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]
> > 
> > https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:
> > 
> >     #define ENV_MEM_LAYOUT_SETTINGS \
> >     "fdt_high=ffffffff\0" \
> >     "initrd_high=ffffffff\0" \
> >     "fdt_addr_r=0x00000100\0" \
> >     "pxefile_addr_r=0x00100000\0" \
> >     "kernel_addr_r=0x01000000\0" \
> >     "scriptaddr=0x02000000\0" \
> >     "ramdisk_addr_r=0x02100000\0" \
> > 
> > Am I correct in assuming that this default configuration will break
> > with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
> > wanted to ask for explicit confirmation here.
> 
> Especially, this is occurred about arm64 kernel.
> Because it wil be overlapped with other image address, if kernel image is over than 16MB.
> 
> In my case, i'm using the private boot.scr.img for booting script.
> I had just added the kernel_addr_r=0x02d000000.
> 
> Just Refer to below:
> https://review.tizen.org/git/?p=platform/kernel/u-boot.git;a=commit;h=8fdfbbeb8a4b2f8c8e66824709e48d9abdb23534

I would strongly recommend setting the values here based on what
linux/Documentation/arm/Booting and linux/Documentation/arm64/boot.txt
describe as the maximum limits for each of these locations.  And take a
peek at include/configs/ti_armv7_common.h in U-Boot on how I set these
for the various TI platforms.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180223/912214c5/attachment.sig>

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Stephen Warren]

On 02/22/2018 12:58 PM, Alexander Kurtz wrote:
> Hi!
> 
> I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
> dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3.
> 
> Recently, the Debian kernel grew beyond 16 MiB:
> 
>      4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
>      4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
>      4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]
> 
> https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:
> 
>      #define ENV_MEM_LAYOUT_SETTINGS \
>      "fdt_high=ffffffff\0" \
>      "initrd_high=ffffffff\0" \
>      "fdt_addr_r=0x00000100\0" \
>      "pxefile_addr_r=0x00100000\0" \
>      "kernel_addr_r=0x01000000\0" \
>      "scriptaddr=0x02000000\0" \
>      "ramdisk_addr_r=0x02100000\0" \
> 
> Am I correct in assuming that this default configuration will break
> with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
> wanted to ask for explicit confirmation here.

Yes, I expect those values are broken for kernels over 16MiB (and 
perhaps even smaller ones too, which may use memory after the compressed 
image for BSS during boot, which could overlap the ramdisk).

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Tuomas Tynkkynen]

+ agraf

On Fri, 23 Feb 2018 09:57:30 -0500
Tom Rini <trini@konsulko.com> wrote:

> On Fri, Feb 23, 2018 at 04:47:06PM +0900, Jaehoon Chung wrote:
> > On 02/23/2018 04:58 AM, Alexander Kurtz wrote:  
> > > Hi!
> > > 
> > > I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
> > > dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3. 
> > > 
> > > Recently, the Debian kernel grew beyond 16 MiB:
> > > 
> > >     4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
> > >     4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
> > >     4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]
> > > 
> > > https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:
> > > 
> > >     #define ENV_MEM_LAYOUT_SETTINGS \
> > >     "fdt_high=ffffffff\0" \
> > >     "initrd_high=ffffffff\0" \
> > >     "fdt_addr_r=0x00000100\0" \
> > >     "pxefile_addr_r=0x00100000\0" \
> > >     "kernel_addr_r=0x01000000\0" \
> > >     "scriptaddr=0x02000000\0" \
> > >     "ramdisk_addr_r=0x02100000\0" \
> > > 
> > > Am I correct in assuming that this default configuration will break
> > > with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
> > > wanted to ask for explicit confirmation here.  
> > 
> > Especially, this is occurred about arm64 kernel.
> > Because it wil be overlapped with other image address, if kernel image is over than 16MB.
> > 
> > In my case, i'm using the private boot.scr.img for booting script.
> > I had just added the kernel_addr_r=0x02d000000.
> > 
> > Just Refer to below:
> > https://review.tizen.org/git/?p=platform/kernel/u-boot.git;a=commit;h=8fdfbbeb8a4b2f8c8e66824709e48d9abdb23534  
> 
> I would strongly recommend setting the values here based on what
> linux/Documentation/arm/Booting and linux/Documentation/arm64/boot.txt
> describe as the maximum limits for each of these locations.  And take a
> peek at include/configs/ti_armv7_common.h in U-Boot on how I set these
> for the various TI platforms.
> 

I sent a patch for the load addresses a while ago, for which I never got
around to writing a follow-up it seems:

https://lists.denx.de/pipermail/u-boot/2017-June/295889.html

FWIW, I have my doubts of the relocation mechanism used for DTs and initrds
because on ARM, the size of the lowmem mapping in Linux (which dictates the
highest permitted address) depends on not only build-time options like
CONFIG_VMSPLIT_* but also on boot-time options like vmalloc=xxxM.

Also, at least when using the extlinux.conf boot method, because the files
(kernel, DTB, initrd) are first loaded to RAM to the addresses specified
in the environment and only then relocated, the files can still smash each
other in memory during load time. So enabling relocation doesn't actually
help, tweaking the load addresses is still required.

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Tom Rini]

On Fri, Feb 23, 2018 at 08:00:19PM +0200, Tuomas Tynkkynen wrote:
> + agraf
> 
> On Fri, 23 Feb 2018 09:57:30 -0500
> Tom Rini <trini@konsulko.com> wrote:
> 
> > On Fri, Feb 23, 2018 at 04:47:06PM +0900, Jaehoon Chung wrote:
> > > On 02/23/2018 04:58 AM, Alexander Kurtz wrote:  
> > > > Hi!
> > > > 
> > > > I am using U-Boot to boot Debian Buster arm64 (standard kernel with a
> > > > dracut-based initramfs) via an extlinux.conf on my Raspberry Pi 3. 
> > > > 
> > > > Recently, the Debian kernel grew beyond 16 MiB:
> > > > 
> > > >     4.13: /boot/vmlinuz-4.13.0-1-arm64 == 16448000 bytes (< 16 MiB) [0]
> > > >     4.14: /boot/vmlinuz-4.14.0-3-arm64 == 17539584 bytes (> 16 MiB) [1]
> > > >     4.15: /boot/vmlinuz-4.15.0-1-arm64 == 17867264 bytes (> 16 MiB) [2]
> > > > 
> > > > https://github.com/u-boot/u-boot/blob/master/include/configs/rpi.h#L129 says:
> > > > 
> > > >     #define ENV_MEM_LAYOUT_SETTINGS \
> > > >     "fdt_high=ffffffff\0" \
> > > >     "initrd_high=ffffffff\0" \
> > > >     "fdt_addr_r=0x00000100\0" \
> > > >     "pxefile_addr_r=0x00100000\0" \
> > > >     "kernel_addr_r=0x01000000\0" \
> > > >     "scriptaddr=0x02000000\0" \
> > > >     "ramdisk_addr_r=0x02100000\0" \
> > > > 
> > > > Am I correct in assuming that this default configuration will break
> > > > with kernels > 16 MiB? The Readme [3] seems to confirm this, but I
> > > > wanted to ask for explicit confirmation here.  
> > > 
> > > Especially, this is occurred about arm64 kernel.
> > > Because it wil be overlapped with other image address, if kernel image is over than 16MB.
> > > 
> > > In my case, i'm using the private boot.scr.img for booting script.
> > > I had just added the kernel_addr_r=0x02d000000.
> > > 
> > > Just Refer to below:
> > > https://review.tizen.org/git/?p=platform/kernel/u-boot.git;a=commit;h=8fdfbbeb8a4b2f8c8e66824709e48d9abdb23534  
> > 
> > I would strongly recommend setting the values here based on what
> > linux/Documentation/arm/Booting and linux/Documentation/arm64/boot.txt
> > describe as the maximum limits for each of these locations.  And take a
> > peek at include/configs/ti_armv7_common.h in U-Boot on how I set these
> > for the various TI platforms.
> > 
> 
> I sent a patch for the load addresses a while ago, for which I never got
> around to writing a follow-up it seems:
> 
> https://lists.denx.de/pipermail/u-boot/2017-June/295889.html
> 
> FWIW, I have my doubts of the relocation mechanism used for DTs and initrds
> because on ARM, the size of the lowmem mapping in Linux (which dictates the
> highest permitted address) depends on not only build-time options like
> CONFIG_VMSPLIT_* but also on boot-time options like vmalloc=xxxM.
> 
> Also, at least when using the extlinux.conf boot method, because the files
> (kernel, DTB, initrd) are first loaded to RAM to the addresses specified
> in the environment and only then relocated, the files can still smash each
> other in memory during load time. So enabling relocation doesn't actually
> help, tweaking the load addresses is still required.

It's true that on ARM you can have different splits, but at least when I
wrote the limits for TI, I either picked the smallest possible, or said
"if you have less than 256MB on your system, you need to make these
manually".  I honestly don't recall which atm.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180223/11a221a7/attachment.sig>

[U-Boot] Support of kernels > 16 MiB on Raspberry Pi

[Alexander Kurtz]

Hi!

I just wanted to let you know that I recently upgraded to U-Boot
2018.07 from Debian Experimental [0] and everything works now! Thanks
to everybody involved [1]!

Best regards

Alexander Kurtz

[0] https://tracker.debian.org/pkg/u-boot
[1] https://github.com/u-boot/u-boot/commits/v2018.07/include/configs/rpi.h
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180721/c20c22ee/attachment.sig>