Driver model at UEFI runtime

Driver model at UEFI runtime

[Heinrich Schuchardt]

Hello Simon,

The EBBR specification requires that the UEFI SystemReset() runtime
service is available to the operating system.

Up to now this has been implemented by overriding function
efi_reset_system() which is marked as __efi_runtime.

Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
and the System Reset Extension for SBI on RISC-V. This has kept the
number of implementations low. The only exceptions are:

* arch/arm/cpu/armv8/fsl-layerscape/cpu.c
* arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
* arch/sandbox/cpu/start.c

Bin has suggested in
https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
reset drivers based on the driver model.

Currently after ExitBootServices() the driver model does not exist anymore.

When evaluating Bin's suggestion one has to keep in mind that after
invoking ExitBootServices() most operating systems call
SetVirtualAddressMap(). Due to the change of the address map all
pointers used by U-Boot afterwards must be updated to match the new
memory map.

The impression that Ilias and I have is that keeping the driver model
alive after SetVirtualAddressMap() would incur:

* a high development effort
* a significant code size increase
* an enlarged attack surface

For RISC-V it has been clarified in the platform specification that the
SBI must implement the system reset extension. For ARMv8 using TF-A and
PSCI is what ARM suggests.

So for these architectures we do not expect a growth in the number of
drivers needed.

Ilias and my favorite would be keeping the design as is.

What is your view on this?

Best regards

Heinrich

Re: Driver model at UEFI runtime

[Simon Glass]

Hi Heinrich,

On Thu, 9 Sept 2021 at 05:29, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Hello Simon,
>
> The EBBR specification requires that the UEFI SystemReset() runtime
> service is available to the operating system.
>
> Up to now this has been implemented by overriding function
> efi_reset_system() which is marked as __efi_runtime.
>
> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> and the System Reset Extension for SBI on RISC-V. This has kept the
> number of implementations low. The only exceptions are:
>
> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> * arch/sandbox/cpu/start.c
>
> Bin has suggested in
> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> reset drivers based on the driver model.
>
> Currently after ExitBootServices() the driver model does not exist anymore.
>
> When evaluating Bin's suggestion one has to keep in mind that after
> invoking ExitBootServices() most operating systems call
> SetVirtualAddressMap(). Due to the change of the address map all
> pointers used by U-Boot afterwards must be updated to match the new
> memory map.
>
> The impression that Ilias and I have is that keeping the driver model
> alive after SetVirtualAddressMap() would incur:
>
> * a high development effort
> * a significant code size increase
> * an enlarged attack surface
>
> For RISC-V it has been clarified in the platform specification that the
> SBI must implement the system reset extension. For ARMv8 using TF-A and
> PSCI is what ARM suggests.
>
> So for these architectures we do not expect a growth in the number of
> drivers needed.
>
> Ilias and my favorite would be keeping the design as is.
>
> What is your view on this?

Not to dump on the original author but here again we are paying the
price for the shortcuts taken at the time and not since revisited.

My original request then was to create a new build of U-Boot, since we
need to build (and load) the runtime stuff separately. Then we can
avoid all this mess and just use the normal U-Boot code (and driver
model). It also scales up to whatever else we want to do to the untime
stuff in the future.

This will be somewhat easier with the VPL series applied, and even
easier if we can make the different builds more easily configurable.
TBD on that and Tom has suggested an approach with CONFIG options that
should tidy things up.

Of course the 'tools' builds solve this by copying/symlinking the C
files into a different directory so they can be built again in a
different context. I think that could get quite out of hand though. So
I favour a separate build.

Regards,
Simon

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

On 9/9/21 10:00 PM, Simon Glass wrote:
> Hi Heinrich,
>
> On Thu, 9 Sept 2021 at 05:29, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> Hello Simon,
>>
>> The EBBR specification requires that the UEFI SystemReset() runtime
>> service is available to the operating system.
>>
>> Up to now this has been implemented by overriding function
>> efi_reset_system() which is marked as __efi_runtime.
>>
>> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>> and the System Reset Extension for SBI on RISC-V. This has kept the
>> number of implementations low. The only exceptions are:
>>
>> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>> * arch/sandbox/cpu/start.c
>>
>> Bin has suggested in
>> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
>> reset drivers based on the driver model.
>>
>> Currently after ExitBootServices() the driver model does not exist anymore.
>>
>> When evaluating Bin's suggestion one has to keep in mind that after
>> invoking ExitBootServices() most operating systems call
>> SetVirtualAddressMap(). Due to the change of the address map all
>> pointers used by U-Boot afterwards must be updated to match the new
>> memory map.
>>
>> The impression that Ilias and I have is that keeping the driver model
>> alive after SetVirtualAddressMap() would incur:
>>
>> * a high development effort
>> * a significant code size increase
>> * an enlarged attack surface
>>
>> For RISC-V it has been clarified in the platform specification that the
>> SBI must implement the system reset extension. For ARMv8 using TF-A and
>> PSCI is what ARM suggests.
>>
>> So for these architectures we do not expect a growth in the number of
>> drivers needed.
>>
>> Ilias and my favorite would be keeping the design as is.
>>
>> What is your view on this?
>
> Not to dump on the original author but here again we are paying the
> price for the shortcuts taken at the time and not since revisited.
>
> My original request then was to create a new build of U-Boot, since we
> need to build (and load) the runtime stuff separately. Then we can

Do you mean by new build something like TPL, SPL?

Tom is right in complaining that the UEFI implementation is getting too
big for some boards. Duplicating a lot of binary code, e.g. the complete
libfdt or everything needed for UEFI variables, does not look a viable
option. The good thing about tagging functions as __efi_runtime is
minimizing binary code duplication.

It would be possible to leave the whole U-Boot binary in memory when
launching the operating system at the cost of loosing < 1MiB of RAM.
This could eliminate the __efi_runtime tagging.

The problematic stuff are the memory structures that we need to convey
between the boottime and the runtime. It is here where pointers need to
be updated. You cannot resolve this data side problem by duplicating code.

The first thing we should work on is an easily parsable structure
without pointers for conveying runtime device information.

Something like a concatenation of structures with

* length
* driver id
* private data

might be sufficient.

> avoid all this mess and just use the normal U-Boot code (and driver
> model). It also scales up to whatever else we want to do to the runtime
> stuff in the future.
>
> This will be somewhat easier with the VPL series applied, and even

VPL? Please, provide a link.

When thinking of which drivers are needed at runtime it is restricted to
the following:

* reset driver. Some like SBI can be blindly called at runtime because
   configuration tells us that there is an SBI. For others, e.g GPIO,
   we need information from the runtime devicetree. For others we may
   want the result of probing at boottime to avoid code duplication.

* tee driver: for managing variables at runtime it would be
   good to have access to non-volatile memory managed by the TEE.
   This has not been realized yet.

All devices that are managed by the operating system must not be touched
by the runtime firmware.

Best regards

Heinrich

> easier if we can make the different builds more easily configurable.
> TBD on that and Tom has suggested an approach with CONFIG options that
> should tidy things up.
>
> Of course the 'tools' builds solve this by copying/symlinking the C
> files into a different directory so they can be built again in a
> different context. I think that could get quite out of hand though. So
> I favour a separate build.
>
> Regards,
> Simon
>

Re: Driver model at UEFI runtime

[Simon Glass]

Hi Heinrich,

On Fri, 10 Sept 2021 at 08:19, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
>
>
> On 9/9/21 10:00 PM, Simon Glass wrote:
> > Hi Heinrich,
> >
> > On Thu, 9 Sept 2021 at 05:29, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> Hello Simon,
> >>
> >> The EBBR specification requires that the UEFI SystemReset() runtime
> >> service is available to the operating system.
> >>
> >> Up to now this has been implemented by overriding function
> >> efi_reset_system() which is marked as __efi_runtime.
> >>
> >> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> >> and the System Reset Extension for SBI on RISC-V. This has kept the
> >> number of implementations low. The only exceptions are:
> >>
> >> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> >> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> >> * arch/sandbox/cpu/start.c
> >>
> >> Bin has suggested in
> >> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> >> reset drivers based on the driver model.
> >>
> >> Currently after ExitBootServices() the driver model does not exist anymore.
> >>
> >> When evaluating Bin's suggestion one has to keep in mind that after
> >> invoking ExitBootServices() most operating systems call
> >> SetVirtualAddressMap(). Due to the change of the address map all
> >> pointers used by U-Boot afterwards must be updated to match the new
> >> memory map.
> >>
> >> The impression that Ilias and I have is that keeping the driver model
> >> alive after SetVirtualAddressMap() would incur:
> >>
> >> * a high development effort
> >> * a significant code size increase
> >> * an enlarged attack surface
> >>
> >> For RISC-V it has been clarified in the platform specification that the
> >> SBI must implement the system reset extension. For ARMv8 using TF-A and
> >> PSCI is what ARM suggests.
> >>
> >> So for these architectures we do not expect a growth in the number of
> >> drivers needed.
> >>
> >> Ilias and my favorite would be keeping the design as is.
> >>
> >> What is your view on this?
> >
> > Not to dump on the original author but here again we are paying the
> > price for the shortcuts taken at the time and not since revisited.
> >
> > My original request then was to create a new build of U-Boot, since we
> > need to build (and load) the runtime stuff separately. Then we can
>
> Do you mean by new build something like TPL, SPL?

I suppose, but we need to move it to PHASE instead, I think. BTW I
sent a series that shows how we can drop TPL_SPL_ once we complete the
CONFIG migration.

>
> Tom is right in complaining that the UEFI implementation is getting too
> big for some boards. Duplicating a lot of binary code, e.g. the complete
> libfdt or everything needed for UEFI variables, does not look a viable
> option. The good thing about tagging functions as __efi_runtime is
> minimizing binary code duplication.

That's true, but it is going to become impossible to maintain this
mess at some point. For example there is a duplicated reset driver and
the UEFI runtime specifically avoiding using driver model. Where does
it end?!

IMO EFI runtime is its own binary and we're going to have to accept
that at some point.

>
> It would be possible to leave the whole U-Boot binary in memory when
> launching the operating system at the cost of loosing < 1MiB of RAM.
> This could eliminate the __efi_runtime tagging.

Yes, but will people complain about the size?

>
> The problematic stuff are the memory structures that we need to convey
> between the boottime and the runtime. It is here where pointers need to
> be updated. You cannot resolve this data side problem by duplicating code.
>
> The first thing we should work on is an easily parsable structure
> without pointers for conveying runtime device information.
>
> Something like a concatenation of structures with
>
> * length
> * driver id
> * private data
>
> might be sufficient.

Well yes that's a whole other problem. I suppose we ultimately end up
running dm_init() again when we start up the runtime? :-(

>
> > avoid all this mess and just use the normal U-Boot code (and driver
> > model). It also scales up to whatever else we want to do to the runtime
> > stuff in the future.
> >
> > This will be somewhat easier with the VPL series applied, and even
>
> VPL? Please, provide a link.

http://patchwork.ozlabs.org/project/uboot/list/?series=263034

>
> When thinking of which drivers are needed at runtime it is restricted to
> the following:
>
> * reset driver. Some like SBI can be blindly called at runtime because
>    configuration tells us that there is an SBI. For others, e.g GPIO,
>    we need information from the runtime devicetree. For others we may
>    want the result of probing at boottime to avoid code duplication.
>
> * tee driver: for managing variables at runtime it would be
>    good to have access to non-volatile memory managed by the TEE.
>    This has not been realized yet.
>
> All devices that are managed by the operating system must not be touched
> by the runtime firmware.

OK.

My eyes are glazing over at this point. As you say, EFI runtime as a
separate binary will further increase the size of EFI, but we will end
up there in the end as people need to call more U-Boot code. No one
ever claimed that EFI was svelte.

Regards,
Simon

Re: Driver model at UEFI runtime

[Bin Meng]

Hi Heinrich,

On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Hello Simon,
>
> The EBBR specification requires that the UEFI SystemReset() runtime
> service is available to the operating system.
>
> Up to now this has been implemented by overriding function
> efi_reset_system() which is marked as __efi_runtime.
>
> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> and the System Reset Extension for SBI on RISC-V. This has kept the
> number of implementations low. The only exceptions are:
>
> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> * arch/sandbox/cpu/start.c
>
> Bin has suggested in
> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> reset drivers based on the driver model.
>
> Currently after ExitBootServices() the driver model does not exist anymore.
>
> When evaluating Bin's suggestion one has to keep in mind that after
> invoking ExitBootServices() most operating systems call
> SetVirtualAddressMap(). Due to the change of the address map all
> pointers used by U-Boot afterwards must be updated to match the new
> memory map.
>

Yeah, this was discussed 3 years ago:
https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken

> The impression that Ilias and I have is that keeping the driver model
> alive after SetVirtualAddressMap() would incur:
>
> * a high development effort
> * a significant code size increase
> * an enlarged attack surface
>
> For RISC-V it has been clarified in the platform specification that the
> SBI must implement the system reset extension. For ARMv8 using TF-A and
> PSCI is what ARM suggests.
>
> So for these architectures we do not expect a growth in the number of
> drivers needed.
>
> Ilias and my favorite would be keeping the design as is.
>
> What is your view on this?

Is U-Boot's UEFI loader implementation supposed to be the recommended
UEFI firmware on ARM and RISC-V on a production / deployment system?
Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
expect to load grub.efi which chain-loads a kernel without
CONFIG_EFI_STUB? What do distributions normally do? What's our
position when compared to EDK II?

Regards,
Bin

Re: Driver model at UEFI runtime

[François Ozog]

Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :

> Hi Heinrich,
>
> On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de>
> wrote:
> >
> > Hello Simon,
> >
> > The EBBR specification requires that the UEFI SystemReset() runtime
> > service is available to the operating system.
> >
> > Up to now this has been implemented by overriding function
> > efi_reset_system() which is marked as __efi_runtime.
> >
> > Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> > and the System Reset Extension for SBI on RISC-V. This has kept the
> > number of implementations low. The only exceptions are:
> >
> > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> > * arch/sandbox/cpu/start.c
> >
> > Bin has suggested in
> > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> > reset drivers based on the driver model.
> >
> > Currently after ExitBootServices() the driver model does not exist
> anymore.
> >
> > When evaluating Bin's suggestion one has to keep in mind that after
> > invoking ExitBootServices() most operating systems call
> > SetVirtualAddressMap(). Due to the change of the address map all
> > pointers used by U-Boot afterwards must be updated to match the new
> > memory map.
> >
>
> Yeah, this was discussed 3 years ago:
>
> https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>
> > The impression that Ilias and I have is that keeping the driver model
> > alive after SetVirtualAddressMap() would incur:
> >
> > * a high development effort
> > * a significant code size increase
> > * an enlarged attack surface
> >
> > For RISC-V it has been clarified in the platform specification that the
> > SBI must implement the system reset extension. For ARMv8 using TF-A and
> > PSCI is what ARM suggests.
> >
> > So for these architectures we do not expect a growth in the number of
> > drivers needed.
> >
> > Ilias and my favorite would be keeping the design as is.
> >
> > What is your view on this?
>
> Is U-Boot's UEFI loader implementation supposed to be the recommended
> UEFI firmware on ARM and RISC-V on a production / deployment system?

For Arm: yes, that is SystemReady spec.

>
> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
> expect to load grub.efi which chain-loads a kernel without
> CONFIG_EFI_STUB?

all paths should be possible , and the shim.efi is to be supported too.
With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
other words I don’t see grub.efi booting a non config_efi_stub.

> What do distributions normally do?

At least Red Hat made it clear at multiple Linaro Connect that they want
standards, and SystemReady is one that makes the life of embedded distros
easy.
Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.

> What's our
> position when compared to EDK II?

the typical distro boot flow is not the most efficient and drags concept
dating where the Microsoft certs had to be part of the picture. A direct
U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
(grub), avoids convoluted platform cert management (shim) and just enhance
security and boot time. Note: Since kernel 5.10, initrd can be measured
(with TPM) when loaded by efi-stub.

>
>
> Regards,
> Bin
>
-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Bin Meng]

On Thu, Sep 30, 2021 at 2:23 PM François Ozog <francois.ozog@linaro.org>
wrote:

>
>
> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :
>
>> Hi Heinrich,
>>
>> On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de>
>> wrote:
>> >
>> > Hello Simon,
>> >
>> > The EBBR specification requires that the UEFI SystemReset() runtime
>> > service is available to the operating system.
>> >
>> > Up to now this has been implemented by overriding function
>> > efi_reset_system() which is marked as __efi_runtime.
>> >
>> > Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>> > and the System Reset Extension for SBI on RISC-V. This has kept the
>> > number of implementations low. The only exceptions are:
>> >
>> > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>> > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>> > * arch/sandbox/cpu/start.c
>> >
>> > Bin has suggested in
>> > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to
>> use
>> > reset drivers based on the driver model.
>> >
>> > Currently after ExitBootServices() the driver model does not exist
>> anymore.
>> >
>> > When evaluating Bin's suggestion one has to keep in mind that after
>> > invoking ExitBootServices() most operating systems call
>> > SetVirtualAddressMap(). Due to the change of the address map all
>> > pointers used by U-Boot afterwards must be updated to match the new
>> > memory map.
>> >
>>
>> Yeah, this was discussed 3 years ago:
>>
>> https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>>
>> > The impression that Ilias and I have is that keeping the driver model
>> > alive after SetVirtualAddressMap() would incur:
>> >
>> > * a high development effort
>> > * a significant code size increase
>> > * an enlarged attack surface
>> >
>> > For RISC-V it has been clarified in the platform specification that the
>> > SBI must implement the system reset extension. For ARMv8 using TF-A and
>> > PSCI is what ARM suggests.
>> >
>> > So for these architectures we do not expect a growth in the number of
>> > drivers needed.
>> >
>> > Ilias and my favorite would be keeping the design as is.
>> >
>> > What is your view on this?
>>
>> Is U-Boot's UEFI loader implementation supposed to be the recommended
>> UEFI firmware on ARM and RISC-V on a production / deployment system?
>
> For Arm: yes, that is SystemReady spec.
>

How about EDK II? Is EDK II supposed to be used in the server environment
where UEFI + ACPI is the way to go? Does any board that ships EDK II with
UEFI + DTB? Can we safely assume that U-Boot's UEFI loader is the reference
implementation for UEFI + DTB on Arm?


>
>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
>> expect to load grub.efi which chain-loads a kernel without
>> CONFIG_EFI_STUB?
>
> all paths should be possible , and the shim.efi is to be supported too.
> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
> other words I don’t see grub.efi booting a non config_efi_stub.
>
>> What do distributions normally do?
>
> At least Red Hat made it clear at multiple Linaro Connect that they want
> standards, and SystemReady is one that makes the life of embedded distros
> easy.
> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.
>
>> What's our
>> position when compared to EDK II?
>
> the typical distro boot flow is not the most efficient and drags concept
> dating where the Microsoft certs had to be part of the picture. A direct
> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
> (grub), avoids convoluted platform cert management (shim) and just enhance
> security and boot time. Note: Since kernel 5.10, initrd can be measured
> (with TPM) when loaded by efi-stub.
>

Regards,
Bin

Re: Driver model at UEFI runtime

[François Ozog]

Le jeu. 30 sept. 2021 à 08:38, Bin Meng <bmeng.cn@gmail.com> a écrit :

> On Thu, Sep 30, 2021 at 2:23 PM François Ozog <francois.ozog@linaro.org>
> wrote:
>
>>
>>
>> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :
>>
>>> Hi Heinrich,
>>>
>>> On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de>
>>> wrote:
>>> >
>>> > Hello Simon,
>>> >
>>> > The EBBR specification requires that the UEFI SystemReset() runtime
>>> > service is available to the operating system.
>>> >
>>> > Up to now this has been implemented by overriding function
>>> > efi_reset_system() which is marked as __efi_runtime.
>>> >
>>> > Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>>> > and the System Reset Extension for SBI on RISC-V. This has kept the
>>> > number of implementations low. The only exceptions are:
>>> >
>>> > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>>> > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>>> > * arch/sandbox/cpu/start.c
>>> >
>>> > Bin has suggested in
>>> > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to
>>> use
>>> > reset drivers based on the driver model.
>>> >
>>> > Currently after ExitBootServices() the driver model does not exist
>>> anymore.
>>> >
>>> > When evaluating Bin's suggestion one has to keep in mind that after
>>> > invoking ExitBootServices() most operating systems call
>>> > SetVirtualAddressMap(). Due to the change of the address map all
>>> > pointers used by U-Boot afterwards must be updated to match the new
>>> > memory map.
>>> >
>>>
>>> Yeah, this was discussed 3 years ago:
>>>
>>> https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>>>
>>> > The impression that Ilias and I have is that keeping the driver model
>>> > alive after SetVirtualAddressMap() would incur:
>>> >
>>> > * a high development effort
>>> > * a significant code size increase
>>> > * an enlarged attack surface
>>> >
>>> > For RISC-V it has been clarified in the platform specification that the
>>> > SBI must implement the system reset extension. For ARMv8 using TF-A and
>>> > PSCI is what ARM suggests.
>>> >
>>> > So for these architectures we do not expect a growth in the number of
>>> > drivers needed.
>>> >
>>> > Ilias and my favorite would be keeping the design as is.
>>> >
>>> > What is your view on this?
>>>
>>> Is U-Boot's UEFI loader implementation supposed to be the recommended
>>> UEFI firmware on ARM and RISC-V on a production / deployment system?
>>
>> For Arm: yes, that is SystemReady spec.
>>
>
> How about EDK II? Is EDK II supposed to be used in the server environment
> where UEFI + ACPI is the way to go?
>
Arm server boot is defined in SystemReady-SR.

> Does any board that ships EDK II with UEFI + DTB?
>
there are boards with EDK II with DTB (Socionext Synquacer has a boot menu
to select between ACPI and DT).

> Can we safely assume that U-Boot's UEFI loader is the reference
> implementation for UEFI + DTB on Arm?
>
yes: that is SystemReady-IR that Arm, Linaro and SoC vendors have made a
reality in the past year and a half.  The key benefit is non brickable with
anti rollback protection you get of all firmware through the UEFI capsule.
The UEFi capsule is complemented by an Arm architecture capsule through FFA
interface.
You will have a number os SystemReady sessions at the forthcoming Arm Dev
Summit.
You will also see VMWare supporting SystemReady-ES, and I am hopeful that
Andrei@VMware will also support SystemReady-IR.

>
>
>>
>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
>>> expect to load grub.efi which chain-loads a kernel without
>>> CONFIG_EFI_STUB?
>>
>> all paths should be possible , and the shim.efi is to be supported too.
>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
>> other words I don’t see grub.efi booting a non config_efi_stub.
>>
>>> What do distributions normally do?
>>
>> At least Red Hat made it clear at multiple Linaro Connect that they want
>> standards, and SystemReady is one that makes the life of embedded distros
>> easy.
>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
>> SecureBoot.
>>
>>> What's our
>>> position when compared to EDK II?
>>
>> the typical distro boot flow is not the most efficient and drags concept
>> dating where the Microsoft certs had to be part of the picture. A direct
>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
>> (grub), avoids convoluted platform cert management (shim) and just enhance
>> security and boot time. Note: Since kernel 5.10, initrd can be measured
>> (with TPM) when loaded by efi-stub.
>>
>
> Regards,
> Bin
>
-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Ilias Apalodimas]

On Thu, 30 Sept 2021 at 09:38, Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Thu, Sep 30, 2021 at 2:23 PM François Ozog <francois.ozog@linaro.org> wrote:
>>
>>
>>
>> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :
>>>
>>> Hi Heinrich,
>>>
>>> On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>> >
>>> > Hello Simon,
>>> >
>>> > The EBBR specification requires that the UEFI SystemReset() runtime
>>> > service is available to the operating system.
>>> >
>>> > Up to now this has been implemented by overriding function
>>> > efi_reset_system() which is marked as __efi_runtime.
>>> >
>>> > Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>>> > and the System Reset Extension for SBI on RISC-V. This has kept the
>>> > number of implementations low. The only exceptions are:
>>> >
>>> > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>>> > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>>> > * arch/sandbox/cpu/start.c
>>> >
>>> > Bin has suggested in
>>> > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
>>> > reset drivers based on the driver model.
>>> >
>>> > Currently after ExitBootServices() the driver model does not exist anymore.
>>> >
>>> > When evaluating Bin's suggestion one has to keep in mind that after
>>> > invoking ExitBootServices() most operating systems call
>>> > SetVirtualAddressMap(). Due to the change of the address map all
>>> > pointers used by U-Boot afterwards must be updated to match the new
>>> > memory map.
>>> >
>>>
>>> Yeah, this was discussed 3 years ago:
>>> https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>>>
>>> > The impression that Ilias and I have is that keeping the driver model
>>> > alive after SetVirtualAddressMap() would incur:
>>> >
>>> > * a high development effort
>>> > * a significant code size increase
>>> > * an enlarged attack surface
>>> >
>>> > For RISC-V it has been clarified in the platform specification that the
>>> > SBI must implement the system reset extension. For ARMv8 using TF-A and
>>> > PSCI is what ARM suggests.
>>> >
>>> > So for these architectures we do not expect a growth in the number of
>>> > drivers needed.
>>> >
>>> > Ilias and my favorite would be keeping the design as is.
>>> >
>>> > What is your view on this?
>>>
>>> Is U-Boot's UEFI loader implementation supposed to be the recommended
>>> UEFI firmware on ARM and RISC-V on a production / deployment system?
>>
>> For Arm: yes, that is SystemReady spec.
>
>
> How about EDK II? Is EDK II supposed to be used in the server environment where UEFI + ACPI is the way to go?
>

Yes ACPI pretty much means EDK2.  Specifically for Arm
SystemReady-ES/SR (embedded server ready and server ready), require
ACPI.  Boards passing that certification today use EDK2.

> Does any board that ships EDK II with UEFI + DTB?

The Socionext SynQuacer box is the only board I know off, that works
with EDK2 and can use either DT or ACPI.

> Can we safely assume that U-Boot's UEFI loader is the reference implementation for UEFI + DTB on Arm?

There's even more to that. EDK2 is bound to the PI spec as well as the
UEFI spec.  U-Boot is a UEFI implementation that doesn't need to
adhere to that.

Regards
/Ilias
>>>
>>>
>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
>>> expect to load grub.efi which chain-loads a kernel without
>>> CONFIG_EFI_STUB?
>>
>> all paths should be possible , and the shim.efi is to be supported too. With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In other words I don’t see grub.efi booting a non config_efi_stub.
>>>
>>> What do distributions normally do?
>>
>> At least Red Hat made it clear at multiple Linaro Connect that they want standards, and SystemReady is one that makes the life of embedded distros easy.
>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.
>>>
>>> What's our
>>> position when compared to EDK II?
>>
>> the typical distro boot flow is not the most efficient and drags concept dating where the Microsoft certs had to be part of the picture. A direct U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path (grub), avoids convoluted platform cert management (shim) and just enhance security and boot time. Note: Since kernel 5.10, initrd can be measured (with TPM) when loaded by efi-stub.
>
>
> Regards,
> Bin

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

On 9/30/21 8:23 AM, François Ozog wrote:
>
>
> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com
> <mailto:bmeng.cn@gmail.com>> a écrit :
>
>     Hi Heinrich,
>
>     On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt
>     <xypron.glpk@gmx.de <mailto:xypron.glpk@gmx.de>> wrote:
>      >
>      > Hello Simon,
>      >
>      > The EBBR specification requires that the UEFI SystemReset() runtime
>      > service is available to the operating system.
>      >
>      > Up to now this has been implemented by overriding function
>      > efi_reset_system() which is marked as __efi_runtime.
>      >
>      > Both ARM and RISC-V support generic interfaces for reset. PSCI
>     for ARM
>      > and the System Reset Extension for SBI on RISC-V. This has kept the
>      > number of implementations low. The only exceptions are:
>      >
>      > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>      > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>      > * arch/sandbox/cpu/start.c
>      >
>      > Bin has suggested in
>      > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html
>     <https://lists.denx.de/pipermail/u-boot/2021-September/459865.html>
>     to use
>      > reset drivers based on the driver model.
>      >
>      > Currently after ExitBootServices() the driver model does not
>     exist anymore.
>      >
>      > When evaluating Bin's suggestion one has to keep in mind that after
>      > invoking ExitBootServices() most operating systems call
>      > SetVirtualAddressMap(). Due to the change of the address map all
>      > pointers used by U-Boot afterwards must be updated to match the new
>      > memory map.
>      >
>
>     Yeah, this was discussed 3 years ago:
>     https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>     <https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken>
>
>      > The impression that Ilias and I have is that keeping the driver model
>      > alive after SetVirtualAddressMap() would incur:
>      >
>      > * a high development effort
>      > * a significant code size increase
>      > * an enlarged attack surface
>      >
>      > For RISC-V it has been clarified in the platform specification
>     that the
>      > SBI must implement the system reset extension. For ARMv8 using
>     TF-A and
>      > PSCI is what ARM suggests.
>      >
>      > So for these architectures we do not expect a growth in the number of
>      > drivers needed.
>      >
>      > Ilias and my favorite would be keeping the design as is.
>      >
>      > What is your view on this?
>
>     Is U-Boot's UEFI loader implementation supposed to be the recommended
>     UEFI firmware on ARM and RISC-V on a production / deployment system?
>
> For Arm: yes, that is SystemReady spec.

For RISC-V it is required by the RISC-V platform specification.

>
>
>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
>     expect to load grub.efi which chain-loads a kernel without
>     CONFIG_EFI_STUB?
>
> all paths should be possible , and the shim.efi is to be supported too.
> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
> other words I don’t see grub.efi booting a non config_efi_stub.
>
>     What do distributions normally do?
>
> At least Red Hat made it clear at multiple Linaro Connect that they want
> standards, and SystemReady is one that makes the life of embedded
> distros easy.
> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.

For Fedora see
https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description

SUSE started the UEFI implementation to boot on all architectures in the
same way. The current ARM and RISC-V images uses UEFI.

Debian and Ubuntu install for booting via GRUB if the installer is
invoked via UEFI. A fallback solution using the legacy Linux entry point
exists.

For BSD the only way to boot on ARM is via UEFI.

>
>     What's our
>     position when compared to EDK II?

U-Boot implements only a subset of UEFI defined in the EBBR specification.

For servers you need a larger scope which is offered by EDK II. This is
required both by the RISC-V platform specification as well as the ARM
SystemReady ServerReady profile.

The number of boards supported by upstream EDK II is very low. But
proprietary firmware based on EDK II exists.

>
> the typical distro boot flow is not the most efficient and drags concept
> dating where the Microsoft certs had to be part of the picture. A direct
> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
> (grub), avoids convoluted platform cert management (shim) and just

This is why U-Boot supports UEFI boot options specifying both a binary
as well as an initial RAM disk.

Best regards

Heinrich

> enhance security and boot time. Note: Since kernel 5.10, initrd can be
> measured (with TPM) when loaded by efi-stub.
>
>
>
>     Regards,
>     Bin
>
> --
>
> François-Frédéric Ozog | /Director Business Development/
> T: +33.67221.6485
> francois.ozog@linaro.org <mailto:francois.ozog@linaro.org> | Skype: ffozog
>
>

Re: Driver model at UEFI runtime

[Michael Walle]

Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
> On 9/30/21 8:23 AM, François Ozog wrote:
[..]
>>     Is U-Boot's UEFI loader implementation supposed to be the 
>> recommended
>>     UEFI firmware on ARM and RISC-V on a production / deployment 
>> system?
>> 
>> For Arm: yes, that is SystemReady spec.
> 
> For RISC-V it is required by the RISC-V platform specification.
> 
>> 
>> 
>>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do 
>> we
>>     expect to load grub.efi which chain-loads a kernel without
>>     CONFIG_EFI_STUB?
>> 
>> all paths should be possible , and the shim.efi is to be supported 
>> too.
>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. 
>> In
>> other words I don’t see grub.efi booting a non config_efi_stub.
>> 
>>     What do distributions normally do?
>> 
>> At least Red Hat made it clear at multiple Linaro Connect that they 
>> want
>> standards, and SystemReady is one that makes the life of embedded
>> distros easy.
>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi 
>> SecureBoot.
> 
> For Fedora see
> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
> 
> SUSE started the UEFI implementation to boot on all architectures in 
> the
> same way. The current ARM and RISC-V images uses UEFI.
> 
> Debian and Ubuntu install for booting via GRUB if the installer is
> invoked via UEFI. A fallback solution using the legacy Linux entry 
> point
> exists.
> 
> For BSD the only way to boot on ARM is via UEFI.
> 
>> 
>>     What's our
>>     position when compared to EDK II?
> 
> U-Boot implements only a subset of UEFI defined in the EBBR 
> specification.
> 
> For servers you need a larger scope which is offered by EDK II. This is
> required both by the RISC-V platform specification as well as the ARM
> SystemReady ServerReady profile.
> 
> The number of boards supported by upstream EDK II is very low. But
> proprietary firmware based on EDK II exists.
> 
>> 
>> the typical distro boot flow is not the most efficient and drags 
>> concept
>> dating where the Microsoft certs had to be part of the picture. A 
>> direct
>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot 
>> path
>> (grub), avoids convoluted platform cert management (shim) and just
> 
> This is why U-Boot supports UEFI boot options specifying both a binary
> as well as an initial RAM disk.

I might be late to this, but where does the devicetree come from? As far
as I understand it right now, for most (all) the SytemReady IR certified
boards, the compiled-in one from u-boot is passed to linux. This won't 
work
in the long run, because the devicetrees keep getting incompatible 
changes.
So while it work for one kernel version it might not work on the next
version.

-michael

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle <michael@walle.cc>:
>Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
>> On 9/30/21 8:23 AM, François Ozog wrote:
>[..]
>>>     Is U-Boot's UEFI loader implementation supposed to be the 
>>> recommended
>>>     UEFI firmware on ARM and RISC-V on a production / deployment 
>>> system?
>>> 
>>> For Arm: yes, that is SystemReady spec.
>> 
>> For RISC-V it is required by the RISC-V platform specification.
>> 
>>> 
>>> 
>>>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do 
>>> we
>>>     expect to load grub.efi which chain-loads a kernel without
>>>     CONFIG_EFI_STUB?
>>> 
>>> all paths should be possible , and the shim.efi is to be supported 
>>> too.
>>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. 
>>> In
>>> other words I don’t see grub.efi booting a non config_efi_stub.
>>> 
>>>     What do distributions normally do?
>>> 
>>> At least Red Hat made it clear at multiple Linaro Connect that they 
>>> want
>>> standards, and SystemReady is one that makes the life of embedded
>>> distros easy.
>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi 
>>> SecureBoot.
>> 
>> For Fedora see
>> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
>> 
>> SUSE started the UEFI implementation to boot on all architectures in 
>> the
>> same way. The current ARM and RISC-V images uses UEFI.
>> 
>> Debian and Ubuntu install for booting via GRUB if the installer is
>> invoked via UEFI. A fallback solution using the legacy Linux entry 
>> point
>> exists.
>> 
>> For BSD the only way to boot on ARM is via UEFI.
>> 
>>> 
>>>     What's our
>>>     position when compared to EDK II?
>> 
>> U-Boot implements only a subset of UEFI defined in the EBBR 
>> specification.
>> 
>> For servers you need a larger scope which is offered by EDK II. This is
>> required both by the RISC-V platform specification as well as the ARM
>> SystemReady ServerReady profile.
>> 
>> The number of boards supported by upstream EDK II is very low. But
>> proprietary firmware based on EDK II exists.
>> 
>>> 
>>> the typical distro boot flow is not the most efficient and drags 
>>> concept
>>> dating where the Microsoft certs had to be part of the picture. A 
>>> direct
>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot 
>>> path
>>> (grub), avoids convoluted platform cert management (shim) and just
>> 
>> This is why U-Boot supports UEFI boot options specifying both a binary
>> as well as an initial RAM disk.
>
>I might be late to this, but where does the devicetree come from? As far
>as I understand it right now, for most (all) the SytemReady IR certified
>boards, the compiled-in one from u-boot is passed to linux. This won't 
>work
>in the long run, because the devicetrees keep getting incompatible 
>changes.
>So while it work for one kernel version it might not work on the next
>version.

It would make sense to add the fdt devicepath to the bootoption like we did it for the initrd.

Best regards

Heinrich

>
>-michael

Re: Driver model at UEFI runtime

[Mark Kettenis]

> From: Simon Glass <sjg@chromium.org>
> Date: Wed, 29 Sep 2021 22:08:49 -0600

Hi Simon,

> Hi Heinrich,
> 
> On Fri, 10 Sept 2021 at 08:19, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >
> >
> >
> > On 9/9/21 10:00 PM, Simon Glass wrote:
> > > Hi Heinrich,
> > >
> > > On Thu, 9 Sept 2021 at 05:29, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> > >>
> > >> Hello Simon,
> > >>
> > >> The EBBR specification requires that the UEFI SystemReset() runtime
> > >> service is available to the operating system.
> > >>
> > >> Up to now this has been implemented by overriding function
> > >> efi_reset_system() which is marked as __efi_runtime.
> > >>
> > >> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> > >> and the System Reset Extension for SBI on RISC-V. This has kept the
> > >> number of implementations low. The only exceptions are:
> > >>
> > >> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> > >> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> > >> * arch/sandbox/cpu/start.c
> > >>
> > >> Bin has suggested in
> > >> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> > >> reset drivers based on the driver model.
> > >>
> > >> Currently after ExitBootServices() the driver model does not exist anymore.
> > >>
> > >> When evaluating Bin's suggestion one has to keep in mind that after
> > >> invoking ExitBootServices() most operating systems call
> > >> SetVirtualAddressMap(). Due to the change of the address map all
> > >> pointers used by U-Boot afterwards must be updated to match the new
> > >> memory map.
> > >>
> > >> The impression that Ilias and I have is that keeping the driver model
> > >> alive after SetVirtualAddressMap() would incur:
> > >>
> > >> * a high development effort
> > >> * a significant code size increase
> > >> * an enlarged attack surface
> > >>
> > >> For RISC-V it has been clarified in the platform specification that the
> > >> SBI must implement the system reset extension. For ARMv8 using TF-A and
> > >> PSCI is what ARM suggests.
> > >>
> > >> So for these architectures we do not expect a growth in the number of
> > >> drivers needed.
> > >>
> > >> Ilias and my favorite would be keeping the design as is.
> > >>
> > >> What is your view on this?
> > >
> > > Not to dump on the original author but here again we are paying the
> > > price for the shortcuts taken at the time and not since revisited.
> > >
> > > My original request then was to create a new build of U-Boot, since we
> > > need to build (and load) the runtime stuff separately. Then we can
> >
> > Do you mean by new build something like TPL, SPL?
> 
> I suppose, but we need to move it to PHASE instead, I think. BTW I
> sent a series that shows how we can drop TPL_SPL_ once we complete the
> CONFIG migration.
> 
> >
> > Tom is right in complaining that the UEFI implementation is getting too
> > big for some boards. Duplicating a lot of binary code, e.g. the complete
> > libfdt or everything needed for UEFI variables, does not look a viable
> > option. The good thing about tagging functions as __efi_runtime is
> > minimizing binary code duplication.
> 
> That's true, but it is going to become impossible to maintain this
> mess at some point. For example there is a duplicated reset driver and
> the UEFI runtime specifically avoiding using driver model. Where does
> it end?!

It ends with whatever functionality we decide that the EFI runtime has
to support after ExitBootServices() has been called.  In EBBR all
runtime services are optional at this stage, with the exception of
SetVirtualAddressMap() and ConvertPointer(). So the only thing that
you really need is a dummy implementation that returns EFI_UNSUPPORTED
for everything else and a way to relocate that implementation to a
virtual address chosen by the OS.  So not even a reset implementation
is needed.  And then there is no reason to get DM involved.

Of course the OS will need some way to reset/shutdown the machine.  In
principle this is supposed to be implemented in other firmware
components (PSCI on ARM, SBI on RISC-V) and the UEFI implementation is
supposed to just make the appropriate firmware call.  In that case I'd
say implementing ResetSystem() would be so trivial that involving DM
makes very little sense.  Unfortunately not all PSCI/SBI
implementations actually implement this.  But even then, as long as
the hardware needed to reset the system is exposed in the DT passed to
the OS, the OS can just take care of this itself.

It's the capsule update stuff and EFI variables stuff that creates
complications if you want to support those while the OS is running.
But the former doesn't really work without the latter and I fear that
implementing EFI variables is just not possible on the majority of the
boards we support in U-Boot.  Especially if you care about verified
boot.  You pretty much have to have something like OP-TEE running in a
secure enclave to do this properly.  There is a good reason why this
was made optional in EBBR.

> IMO EFI runtime is its own binary and we're going to have to accept
> that at some point.

Yes.  I think so.  Just like we do for the PSCI implementation that is
provided for some of the boards that don't come with a TF-A
implementation.  The minimal implementation I suggest above would be
tiny and would fit very well in this model.  My suggestion would be
that before ExitBootServices() gets called, runtime services would
just call normal U-Boot code using DM and all.  And when
ExitBootServices() gets called we just switch in the standalone dummy
implementation.

Personally, I would really prefer this approach where the runtime code
is as minimal as possible.  It would be way easier to audit and
convince myself as an OS developer that yes, if I make an EFI runtime
call it isn't going to do nasty stuff behind my back.

> > It would be possible to leave the whole U-Boot binary in memory when
> > launching the operating system at the cost of loosing < 1MiB of RAM.
> > This could eliminate the __efi_runtime tagging.
> 
> Yes, but will people complain about the size?

Yes.  This would leave a ton of executable code in memory that could
be exploited using a kernel ROP attack.

> > The problematic stuff are the memory structures that we need to convey
> > between the boottime and the runtime. It is here where pointers need to
> > be updated. You cannot resolve this data side problem by duplicating code.
> >
> > The first thing we should work on is an easily parsable structure
> > without pointers for conveying runtime device information.
> >
> > Something like a concatenation of structures with
> >
> > * length
> > * driver id
> > * private data
> >
> > might be sufficient.
> 
> Well yes that's a whole other problem. I suppose we ultimately end up
> running dm_init() again when we start up the runtime? :-(
> 
> >
> > > avoid all this mess and just use the normal U-Boot code (and driver
> > > model). It also scales up to whatever else we want to do to the runtime
> > > stuff in the future.
> > >
> > > This will be somewhat easier with the VPL series applied, and even
> >
> > VPL? Please, provide a link.
> 
> http://patchwork.ozlabs.org/project/uboot/list/?series=263034
> 
> >
> > When thinking of which drivers are needed at runtime it is restricted to
> > the following:
> >
> > * reset driver. Some like SBI can be blindly called at runtime because
> >    configuration tells us that there is an SBI. For others, e.g GPIO,
> >    we need information from the runtime devicetree. For others we may
> >    want the result of probing at boottime to avoid code duplication.
> >
> > * tee driver: for managing variables at runtime it would be
> >    good to have access to non-volatile memory managed by the TEE.
> >    This has not been realized yet.
> >
> > All devices that are managed by the operating system must not be touched
> > by the runtime firmware.
> 
> OK.
> 
> My eyes are glazing over at this point. As you say, EFI runtime as a
> separate binary will further increase the size of EFI, but we will end
> up there in the end as people need to call more U-Boot code. No one
> ever claimed that EFI was svelte.
> 
> Regards,
> Simon
> 

Re: Driver model at UEFI runtime

[Mark Kettenis]

> From: François Ozog <francois.ozog@linaro.org>
> Date: Thu, 30 Sep 2021 08:23:34 +0200
> 
> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :
> 
> > Hi Heinrich,
> >
> > On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de>
> > wrote:
> > >
> > > Hello Simon,
> > >
> > > The EBBR specification requires that the UEFI SystemReset() runtime
> > > service is available to the operating system.
> > >
> > > Up to now this has been implemented by overriding function
> > > efi_reset_system() which is marked as __efi_runtime.
> > >
> > > Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
> > > and the System Reset Extension for SBI on RISC-V. This has kept the
> > > number of implementations low. The only exceptions are:
> > >
> > > * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> > > * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
> > > * arch/sandbox/cpu/start.c
> > >
> > > Bin has suggested in
> > > https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
> > > reset drivers based on the driver model.
> > >
> > > Currently after ExitBootServices() the driver model does not exist
> > anymore.
> > >
> > > When evaluating Bin's suggestion one has to keep in mind that after
> > > invoking ExitBootServices() most operating systems call
> > > SetVirtualAddressMap(). Due to the change of the address map all
> > > pointers used by U-Boot afterwards must be updated to match the new
> > > memory map.
> > >
> >
> > Yeah, this was discussed 3 years ago:
> >
> > https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
> >
> > > The impression that Ilias and I have is that keeping the driver model
> > > alive after SetVirtualAddressMap() would incur:
> > >
> > > * a high development effort
> > > * a significant code size increase
> > > * an enlarged attack surface
> > >
> > > For RISC-V it has been clarified in the platform specification that the
> > > SBI must implement the system reset extension. For ARMv8 using TF-A and
> > > PSCI is what ARM suggests.
> > >
> > > So for these architectures we do not expect a growth in the number of
> > > drivers needed.
> > >
> > > Ilias and my favorite would be keeping the design as is.
> > >
> > > What is your view on this?
> >
> > Is U-Boot's UEFI loader implementation supposed to be the recommended
> > UEFI firmware on ARM and RISC-V on a production / deployment system?
> 
> For Arm: yes, that is SystemReady spec.
> 
> >
> > Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
> > expect to load grub.efi which chain-loads a kernel without
> > CONFIG_EFI_STUB?
> 
> all paths should be possible , and the shim.efi is to be supported too.
> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
> other words I don’t see grub.efi booting a non config_efi_stub.
> 
> > What do distributions normally do?
> 
> At least Red Hat made it clear at multiple Linaro Connect that they want
> standards, and SystemReady is one that makes the life of embedded distros
> easy.
> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.
> 
> > What's our
> > position when compared to EDK II?
> 
> the typical distro boot flow is not the most efficient and drags concept
> dating where the Microsoft certs had to be part of the picture. A direct
> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
> (grub), avoids convoluted platform cert management (shim) and just enhance
> security and boot time. Note: Since kernel 5.10, initrd can be measured
> (with TPM) when loaded by efi-stub.

One major attraction of using the UEFI boot flow on non-x86
architectures is that it allows OS distributions to use nearly
identical logic for installing themselves as on a x86 systems.  And
as a result the user experience is largely the same.

I only have basic understanding of how the Linux EFI boot stub works
in practice, but I think it means that the Linux kernel you want to
boot has to live on a filesystem that can be accessed by the UEFI
firmware.  And that pretty much means it has to live on a VFAT
filesystem, which is undesirable from a Linux distro perspective I'd
say.  It also means that you have to rely on the EFI boot manager to
switch between kernels, which significantly changes the user
experience.

Re: Driver model at UEFI runtime

[Mark Kettenis]

> From: Ilias Apalodimas <ilias.apalodimas@linaro.org>
> Date: Thu, 30 Sep 2021 09:46:30 +0300
> 
> On Thu, 30 Sept 2021 at 09:38, Bin Meng <bmeng.cn@gmail.com> wrote:
> 
> > Does any board that ships EDK II with UEFI + DTB?
> 
> The Socionext SynQuacer box is the only board I know off, that works
> with EDK2 and can use either DT or ACPI.

The SoftIron Overdrive 1000/3000, SolidRun MACCHIATObin and SolidRun
HoneyComb LX2 all come with EDK2-based firmware that can switch
between DT and ACPI.

The PFTF firmware for the Raspberry Pi also uses EDK2 and can be
switched between DT and ACPI.

On all these systems DT exposes a more complete hardware description
than ACPI so people actually use the DT option.

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

On 9/30/21 13:13, Mark Kettenis wrote:
>> From: Simon Glass <sjg@chromium.org>
>> Date: Wed, 29 Sep 2021 22:08:49 -0600
>
> Hi Simon,
>
>> Hi Heinrich,
>>
>> On Fri, 10 Sept 2021 at 08:19, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>
>>>
>>>
>>> On 9/9/21 10:00 PM, Simon Glass wrote:
>>>> Hi Heinrich,
>>>>
>>>> On Thu, 9 Sept 2021 at 05:29, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>>>>
>>>>> Hello Simon,
>>>>>
>>>>> The EBBR specification requires that the UEFI SystemReset() runtime
>>>>> service is available to the operating system.
>>>>>
>>>>> Up to now this has been implemented by overriding function
>>>>> efi_reset_system() which is marked as __efi_runtime.
>>>>>
>>>>> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>>>>> and the System Reset Extension for SBI on RISC-V. This has kept the
>>>>> number of implementations low. The only exceptions are:
>>>>>
>>>>> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>>>>> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>>>>> * arch/sandbox/cpu/start.c
>>>>>
>>>>> Bin has suggested in
>>>>> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
>>>>> reset drivers based on the driver model.
>>>>>
>>>>> Currently after ExitBootServices() the driver model does not exist anymore.
>>>>>
>>>>> When evaluating Bin's suggestion one has to keep in mind that after
>>>>> invoking ExitBootServices() most operating systems call
>>>>> SetVirtualAddressMap(). Due to the change of the address map all
>>>>> pointers used by U-Boot afterwards must be updated to match the new
>>>>> memory map.
>>>>>
>>>>> The impression that Ilias and I have is that keeping the driver model
>>>>> alive after SetVirtualAddressMap() would incur:
>>>>>
>>>>> * a high development effort
>>>>> * a significant code size increase
>>>>> * an enlarged attack surface
>>>>>
>>>>> For RISC-V it has been clarified in the platform specification that the
>>>>> SBI must implement the system reset extension. For ARMv8 using TF-A and
>>>>> PSCI is what ARM suggests.
>>>>>
>>>>> So for these architectures we do not expect a growth in the number of
>>>>> drivers needed.
>>>>>
>>>>> Ilias and my favorite would be keeping the design as is.
>>>>>
>>>>> What is your view on this?
>>>>
>>>> Not to dump on the original author but here again we are paying the
>>>> price for the shortcuts taken at the time and not since revisited.
>>>>
>>>> My original request then was to create a new build of U-Boot, since we
>>>> need to build (and load) the runtime stuff separately. Then we can
>>>
>>> Do you mean by new build something like TPL, SPL?
>>
>> I suppose, but we need to move it to PHASE instead, I think. BTW I
>> sent a series that shows how we can drop TPL_SPL_ once we complete the
>> CONFIG migration.
>>
>>>
>>> Tom is right in complaining that the UEFI implementation is getting too
>>> big for some boards. Duplicating a lot of binary code, e.g. the complete
>>> libfdt or everything needed for UEFI variables, does not look a viable
>>> option. The good thing about tagging functions as __efi_runtime is
>>> minimizing binary code duplication.
>>
>> That's true, but it is going to become impossible to maintain this
>> mess at some point. For example there is a duplicated reset driver and
>> the UEFI runtime specifically avoiding using driver model. Where does
>> it end?!
>
> It ends with whatever functionality we decide that the EFI runtime has
> to support after ExitBootServices() has been called.  In EBBR all
> runtime services are optional at this stage, with the exception of
> SetVirtualAddressMap() and ConvertPointer(). So the only thing that
> you really need is a dummy implementation that returns EFI_UNSUPPORTED
> for everything else and a way to relocate that implementation to a
> virtual address chosen by the OS.  So not even a reset implementation
> is needed.  And then there is no reason to get DM involved.
>
> Of course the OS will need some way to reset/shutdown the machine.  In
> principle this is supposed to be implemented in other firmware
> components (PSCI on ARM, SBI on RISC-V) and the UEFI implementation is
> supposed to just make the appropriate firmware call.  In that case I'd
> say implementing ResetSystem() would be so trivial that involving DM
> makes very little sense.  Unfortunately not all PSCI/SBI
> implementations actually implement this.  But even then, as long as
> the hardware needed to reset the system is exposed in the DT passed to
> the OS, the OS can just take care of this itself.
>
> It's the capsule update stuff and EFI variables stuff that creates
> complications if you want to support those while the OS is running.
> But the former doesn't really work without the latter and I fear that
> implementing EFI variables is just not possible on the majority of the
> boards we support in U-Boot.  Especially if you care about verified
> boot.  You pretty much have to have something like OP-TEE running in a
> secure enclave to do this properly.  There is a good reason why this
> was made optional in EBBR.

UEFI variables are already implemented. After ExitBootServices() we
support GetVariable() and GetNextVeriableName(). SetVariable() is not
yet implemented.

>
>> IMO EFI runtime is its own binary and we're going to have to accept
>> that at some point.
>
> Yes.  I think so.  Just like we do for the PSCI implementation that is
> provided for some of the boards that don't come with a TF-A
> implementation.  The minimal implementation I suggest above would be
> tiny and would fit very well in this model.  My suggestion would be
> that before ExitBootServices() gets called, runtime services would
> just call normal U-Boot code using DM and all.  And when
> ExitBootServices() gets called we just switch in the standalone dummy
> implementation.

This switch is currently done in efi_runtime_detach() and in event
handlers for the ExitBootService event.

>
> Personally, I would really prefer this approach where the runtime code
> is as minimal as possible.  It would be way easier to audit and
> convince myself as an OS developer that yes, if I make an EFI runtime
> call it isn't going to do nasty stuff behind my back.

These are the *.c files containing __efi_runtime code.

arch/arm/cpu/armv8/fsl-layerscape/cpu.c
arch/arm/lib/sections.c
arch/arm/mach-bcm283x/reset.c
arch/riscv/lib/sbi.c (if we support runtime reset)
arch/sandbox/cpu/start.c
arch/sandbox/lib/sections.c
arch/x86/lib/sections.c
drivers/firmware/psci.c
drivers/sysreset/sysreset_sbi.c (if we support runtime reset)
drivers/sysreset/sysreset_x86.c
lib/charset.c
lib/crc32.c
lib/efi_loader/efi_boottime.c
lib/efi_loader/efi_memory.c
lib/efi_loader/efi_runtime.c
lib/efi_loader/efi_var_common.c
lib/efi_loader/efi_var_mem.c
lib/efi_loader/efi_variable.c
lib/efi_loader/efi_variable_tee.c

>
>>> It would be possible to leave the whole U-Boot binary in memory when
>>> launching the operating system at the cost of loosing < 1MiB of RAM.
>>> This could eliminate the __efi_runtime tagging.
>>
>> Yes, but will people complain about the size?
>
> Yes.  This would leave a ton of executable code in memory that could
> be exploited using a kernel ROP attack.

So let's exclude this option.

Best regards

Heinrich

>
>>> The problematic stuff are the memory structures that we need to convey
>>> between the boottime and the runtime. It is here where pointers need to
>>> be updated. You cannot resolve this data side problem by duplicating code.
>>>
>>> The first thing we should work on is an easily parsable structure
>>> without pointers for conveying runtime device information.
>>>
>>> Something like a concatenation of structures with
>>>
>>> * length
>>> * driver id
>>> * private data
>>>
>>> might be sufficient.
>>
>> Well yes that's a whole other problem. I suppose we ultimately end up
>> running dm_init() again when we start up the runtime? :-(
>>
>>>
>>>> avoid all this mess and just use the normal U-Boot code (and driver
>>>> model). It also scales up to whatever else we want to do to the runtime
>>>> stuff in the future.
>>>>
>>>> This will be somewhat easier with the VPL series applied, and even
>>>
>>> VPL? Please, provide a link.
>>
>> http://patchwork.ozlabs.org/project/uboot/list/?series=263034
>>
>>>
>>> When thinking of which drivers are needed at runtime it is restricted to
>>> the following:
>>>
>>> * reset driver. Some like SBI can be blindly called at runtime because
>>>     configuration tells us that there is an SBI. For others, e.g GPIO,
>>>     we need information from the runtime devicetree. For others we may
>>>     want the result of probing at boottime to avoid code duplication.
>>>
>>> * tee driver: for managing variables at runtime it would be
>>>     good to have access to non-volatile memory managed by the TEE.
>>>     This has not been realized yet.
>>>
>>> All devices that are managed by the operating system must not be touched
>>> by the runtime firmware.
>>
>> OK.
>>
>> My eyes are glazing over at this point. As you say, EFI runtime as a
>> separate binary will further increase the size of EFI, but we will end
>> up there in the end as people need to call more U-Boot code. No one
>> ever claimed that EFI was svelte.
>>
>> Regards,
>> Simon
>>

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

On 9/30/21 13:31, Mark Kettenis wrote:
>> From: François Ozog <francois.ozog@linaro.org>
>> Date: Thu, 30 Sep 2021 08:23:34 +0200
>>
>> Le jeu. 30 sept. 2021 à 07:12, Bin Meng <bmeng.cn@gmail.com> a écrit :
>>
>>> Hi Heinrich,
>>>
>>> On Thu, Sep 9, 2021 at 7:16 PM Heinrich Schuchardt <xypron.glpk@gmx.de>
>>> wrote:
>>>>
>>>> Hello Simon,
>>>>
>>>> The EBBR specification requires that the UEFI SystemReset() runtime
>>>> service is available to the operating system.
>>>>
>>>> Up to now this has been implemented by overriding function
>>>> efi_reset_system() which is marked as __efi_runtime.
>>>>
>>>> Both ARM and RISC-V support generic interfaces for reset. PSCI for ARM
>>>> and the System Reset Extension for SBI on RISC-V. This has kept the
>>>> number of implementations low. The only exceptions are:
>>>>
>>>> * arch/arm/cpu/armv8/fsl-layerscape/cpu.c
>>>> * arch/arm/mach-bcm283x/reset.c for the Raspberry PIs
>>>> * arch/sandbox/cpu/start.c
>>>>
>>>> Bin has suggested in
>>>> https://lists.denx.de/pipermail/u-boot/2021-September/459865.html to use
>>>> reset drivers based on the driver model.
>>>>
>>>> Currently after ExitBootServices() the driver model does not exist
>>> anymore.
>>>>
>>>> When evaluating Bin's suggestion one has to keep in mind that after
>>>> invoking ExitBootServices() most operating systems call
>>>> SetVirtualAddressMap(). Due to the change of the address map all
>>>> pointers used by U-Boot afterwards must be updated to match the new
>>>> memory map.
>>>>
>>>
>>> Yeah, this was discussed 3 years ago:
>>>
>>> https://u-boot.denx.narkive.com/mA8xIbLk/efi-loader-runtime-services-implementation-broken
>>>
>>>> The impression that Ilias and I have is that keeping the driver model
>>>> alive after SetVirtualAddressMap() would incur:
>>>>
>>>> * a high development effort
>>>> * a significant code size increase
>>>> * an enlarged attack surface
>>>>
>>>> For RISC-V it has been clarified in the platform specification that the
>>>> SBI must implement the system reset extension. For ARMv8 using TF-A and
>>>> PSCI is what ARM suggests.
>>>>
>>>> So for these architectures we do not expect a growth in the number of
>>>> drivers needed.
>>>>
>>>> Ilias and my favorite would be keeping the design as is.
>>>>
>>>> What is your view on this?
>>>
>>> Is U-Boot's UEFI loader implementation supposed to be the recommended
>>> UEFI firmware on ARM and RISC-V on a production / deployment system?
>>
>> For Arm: yes, that is SystemReady spec.
>>
>>>
>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do we
>>> expect to load grub.efi which chain-loads a kernel without
>>> CONFIG_EFI_STUB?
>>
>> all paths should be possible , and the shim.efi is to be supported too.
>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot. In
>> other words I don’t see grub.efi booting a non config_efi_stub.
>>
>>> What do distributions normally do?
>>
>> At least Red Hat made it clear at multiple Linaro Connect that they want
>> standards, and SystemReady is one that makes the life of embedded distros
>> easy.
>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi SecureBoot.
>>
>>> What's our
>>> position when compared to EDK II?
>>
>> the typical distro boot flow is not the most efficient and drags concept
>> dating where the Microsoft certs had to be part of the picture. A direct
>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot path
>> (grub), avoids convoluted platform cert management (shim) and just enhance
>> security and boot time. Note: Since kernel 5.10, initrd can be measured
>> (with TPM) when loaded by efi-stub.
>
> One major attraction of using the UEFI boot flow on non-x86
> architectures is that it allows OS distributions to use nearly
> identical logic for installing themselves as on a x86 systems.  And
> as a result the user experience is largely the same.
>
> I only have basic understanding of how the Linux EFI boot stub works
> in practice, but I think it means that the Linux kernel you want to
> boot has to live on a filesystem that can be accessed by the UEFI
> firmware.  And that pretty much means it has to live on a VFAT
> filesystem, which is undesirable from a Linux distro perspective I'd
> say.  It also means that you have to rely on the EFI boot manager to
> switch between kernels, which significantly changes the user
> experience.

The Linux EFI stub does not access media and it does not care about the
location from where it was loaded.

The UEFI specification requires support for FAT but does not disallow
supporting further file systems. It might be seen as a deficiency of EDK
II that it only supports FAT and firmware filesystems.

U-Boot can load the kernel, initrd, and fdt from any supported device
irrespective of the boot method. When using UEFI boot options we
currently only support block devices. We do not yet support UEFI boot
options for loading from the network.

Best regards

Heinrich

Re: Driver model at UEFI runtime

[Michael Walle]

Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle 
> <michael@walle.cc>:
>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
>>> On 9/30/21 8:23 AM, François Ozog wrote:
>> [..]
>>>>     Is U-Boot's UEFI loader implementation supposed to be the
>>>> recommended
>>>>     UEFI firmware on ARM and RISC-V on a production / deployment
>>>> system?
>>>> 
>>>> For Arm: yes, that is SystemReady spec.
>>> 
>>> For RISC-V it is required by the RISC-V platform specification.
>>> 
>>>> 
>>>> 
>>>>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or 
>>>> do
>>>> we
>>>>     expect to load grub.efi which chain-loads a kernel without
>>>>     CONFIG_EFI_STUB?
>>>> 
>>>> all paths should be possible , and the shim.efi is to be supported
>>>> too.
>>>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot.
>>>> In
>>>> other words I don’t see grub.efi booting a non config_efi_stub.
>>>> 
>>>>     What do distributions normally do?
>>>> 
>>>> At least Red Hat made it clear at multiple Linaro Connect that they
>>>> want
>>>> standards, and SystemReady is one that makes the life of embedded
>>>> distros easy.
>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
>>>> SecureBoot.
>>> 
>>> For Fedora see
>>> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
>>> 
>>> SUSE started the UEFI implementation to boot on all architectures in
>>> the
>>> same way. The current ARM and RISC-V images uses UEFI.
>>> 
>>> Debian and Ubuntu install for booting via GRUB if the installer is
>>> invoked via UEFI. A fallback solution using the legacy Linux entry
>>> point
>>> exists.
>>> 
>>> For BSD the only way to boot on ARM is via UEFI.
>>> 
>>>> 
>>>>     What's our
>>>>     position when compared to EDK II?
>>> 
>>> U-Boot implements only a subset of UEFI defined in the EBBR
>>> specification.
>>> 
>>> For servers you need a larger scope which is offered by EDK II. This 
>>> is
>>> required both by the RISC-V platform specification as well as the ARM
>>> SystemReady ServerReady profile.
>>> 
>>> The number of boards supported by upstream EDK II is very low. But
>>> proprietary firmware based on EDK II exists.
>>> 
>>>> 
>>>> the typical distro boot flow is not the most efficient and drags
>>>> concept
>>>> dating where the Microsoft certs had to be part of the picture. A
>>>> direct
>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot
>>>> path
>>>> (grub), avoids convoluted platform cert management (shim) and just
>>> 
>>> This is why U-Boot supports UEFI boot options specifying both a 
>>> binary
>>> as well as an initial RAM disk.
>> 
>> I might be late to this, but where does the devicetree come from? As 
>> far
>> as I understand it right now, for most (all) the SytemReady IR 
>> certified
>> boards, the compiled-in one from u-boot is passed to linux. This won't
>> work
>> in the long run, because the devicetrees keep getting incompatible
>> changes.
>> So while it work for one kernel version it might not work on the next
>> version.
> 
> It would make sense to add the fdt devicepath to the bootoption like
> we did it for the initrd.

I haven't followed the much of the recent development, are there any
commits/files I can look at?

And I'm not just talking about the use case where the EFI stub of linux
is booted directy, but also when there is grub in between.

The distribution would supply a bunch of devicetrees along with
the kernel (and initrd). Possibly also different versions of these.
In grub you would choose the desired kernel/initrd/devicetree
combination.

But this means grub would need to (1) know the filename of the
devicetree, or (2) it is hardcoded in the grub config which is
generated by the distribution.

For (1) you'd need to convey the information from u-boot to grub.
(2) would mean the distribution will have to figure out the suitable
devicetree.

To make things worse, there are boards which have several
different devicetrees or there might even be overlays. Eg. my
board has different devicetrees which are expected to be chosen
by the user by setting the fdtfile variable.

-michael

Re: Driver model at UEFI runtime

[Heinrich Schuchardt]

On 9/30/21 14:07, Michael Walle wrote:
> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
>> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
>> <michael@walle.cc>:
>>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
>>>> On 9/30/21 8:23 AM, François Ozog wrote:
>>> [..]
>>>>>     Is U-Boot's UEFI loader implementation supposed to be the
>>>>> recommended
>>>>>     UEFI firmware on ARM and RISC-V on a production / deployment
>>>>> system?
>>>>>
>>>>> For Arm: yes, that is SystemReady spec.
>>>>
>>>> For RISC-V it is required by the RISC-V platform specification.
>>>>
>>>>>
>>>>>
>>>>>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or do
>>>>> we
>>>>>     expect to load grub.efi which chain-loads a kernel without
>>>>>     CONFIG_EFI_STUB?
>>>>>
>>>>> all paths should be possible , and the shim.efi is to be supported
>>>>> too.
>>>>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot.
>>>>> In
>>>>> other words I don’t see grub.efi booting a non config_efi_stub.
>>>>>
>>>>>     What do distributions normally do?
>>>>>
>>>>> At least Red Hat made it clear at multiple Linaro Connect that they
>>>>> want
>>>>> standards, and SystemReady is one that makes the life of embedded
>>>>> distros easy.
>>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
>>>>> SecureBoot.
>>>>
>>>> For Fedora see
>>>> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
>>>>
>>>>
>>>> SUSE started the UEFI implementation to boot on all architectures in
>>>> the
>>>> same way. The current ARM and RISC-V images uses UEFI.
>>>>
>>>> Debian and Ubuntu install for booting via GRUB if the installer is
>>>> invoked via UEFI. A fallback solution using the legacy Linux entry
>>>> point
>>>> exists.
>>>>
>>>> For BSD the only way to boot on ARM is via UEFI.
>>>>
>>>>>
>>>>>     What's our
>>>>>     position when compared to EDK II?
>>>>
>>>> U-Boot implements only a subset of UEFI defined in the EBBR
>>>> specification.
>>>>
>>>> For servers you need a larger scope which is offered by EDK II. This is
>>>> required both by the RISC-V platform specification as well as the ARM
>>>> SystemReady ServerReady profile.
>>>>
>>>> The number of boards supported by upstream EDK II is very low. But
>>>> proprietary firmware based on EDK II exists.
>>>>
>>>>>
>>>>> the typical distro boot flow is not the most efficient and drags
>>>>> concept
>>>>> dating where the Microsoft certs had to be part of the picture. A
>>>>> direct
>>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot
>>>>> path
>>>>> (grub), avoids convoluted platform cert management (shim) and just
>>>>
>>>> This is why U-Boot supports UEFI boot options specifying both a binary
>>>> as well as an initial RAM disk.
>>>
>>> I might be late to this, but where does the devicetree come from? As far
>>> as I understand it right now, for most (all) the SytemReady IR certified
>>> boards, the compiled-in one from u-boot is passed to linux. This won't
>>> work
>>> in the long run, because the devicetrees keep getting incompatible
>>> changes.
>>> So while it work for one kernel version it might not work on the next
>>> version.
>>
>> It would make sense to add the fdt devicepath to the bootoption like
>> we did it for the initrd.
>
> I haven't followed the much of the recent development, are there any
> commits/files I can look at?

For FDT, not yet.

>
> And I'm not just talking about the use case where the EFI stub of linux
> is booted directy, but also when there is grub in between.
>
> The distribution would supply a bunch of devicetrees along with
> the kernel (and initrd). Possibly also different versions of these.
> In grub you would choose the desired kernel/initrd/devicetree
> combination.

The best way to handle this would be the devicetree command in GRUB. In
U-Boot we have implemented a protocol to fixup devicetrees:
https://github.com/U-Boot-EFI/EFI_DT_FIXUP_PROTOCOL
This is already used by systemd-boot.

I prepared some patches for GRUB:
https://lists.gnu.org/archive/html/grub-devel/2021-02/msg00012.html

Unfortunately GRUB maintainers have decided to disable the devicetree
command when secure boot is enabled.

Best regards

Heinrich

>
> But this means grub would need to (1) know the filename of the
> devicetree, or (2) it is hardcoded in the grub config which is
> generated by the distribution.
>
> For (1) you'd need to convey the information from u-boot to grub.
> (2) would mean the distribution will have to figure out the suitable
> devicetree.
>
> To make things worse, there are boards which have several
> different devicetrees or there might even be overlays. Eg. my
> board has different devicetrees which are expected to be chosen
> by the user by setting the fdtfile variable.
>
> -michael

Re: Driver model at UEFI runtime

[François Ozog]

On Thu, 30 Sept 2021 at 14:07, Michael Walle <michael@walle.cc> wrote:

> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
> > Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
> > <michael@walle.cc>:
> >> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
> >>> On 9/30/21 8:23 AM, François Ozog wrote:
> >> [..]
> >>>>     Is U-Boot's UEFI loader implementation supposed to be the
> >>>> recommended
> >>>>     UEFI firmware on ARM and RISC-V on a production / deployment
> >>>> system?
> >>>>
> >>>> For Arm: yes, that is SystemReady spec.
> >>>
> >>> For RISC-V it is required by the RISC-V platform specification.
> >>>
> >>>>
> >>>>
> >>>>     Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB, or
> >>>> do
> >>>> we
> >>>>     expect to load grub.efi which chain-loads a kernel without
> >>>>     CONFIG_EFI_STUB?
> >>>>
> >>>> all paths should be possible , and the shim.efi is to be supported
> >>>> too.
> >>>> With UEFI, I always see that UEFI is kept down to OS for SecureBoot.
> >>>> In
> >>>> other words I don’t see grub.efi booting a non config_efi_stub.
> >>>>
> >>>>     What do distributions normally do?
> >>>>
> >>>> At least Red Hat made it clear at multiple Linaro Connect that they
> >>>> want
> >>>> standards, and SystemReady is one that makes the life of embedded
> >>>> distros easy.
> >>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
> >>>> SecureBoot.
> >>>
> >>> For Fedora see
> >>>
> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
> >>>
> >>> SUSE started the UEFI implementation to boot on all architectures in
> >>> the
> >>> same way. The current ARM and RISC-V images uses UEFI.
> >>>
> >>> Debian and Ubuntu install for booting via GRUB if the installer is
> >>> invoked via UEFI. A fallback solution using the legacy Linux entry
> >>> point
> >>> exists.
> >>>
> >>> For BSD the only way to boot on ARM is via UEFI.
> >>>
> >>>>
> >>>>     What's our
> >>>>     position when compared to EDK II?
> >>>
> >>> U-Boot implements only a subset of UEFI defined in the EBBR
> >>> specification.
> >>>
> >>> For servers you need a larger scope which is offered by EDK II. This
> >>> is
> >>> required both by the RISC-V platform specification as well as the ARM
> >>> SystemReady ServerReady profile.
> >>>
> >>> The number of boards supported by upstream EDK II is very low. But
> >>> proprietary firmware based on EDK II exists.
> >>>
> >>>>
> >>>> the typical distro boot flow is not the most efficient and drags
> >>>> concept
> >>>> dating where the Microsoft certs had to be part of the picture. A
> >>>> direct
> >>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the boot
> >>>> path
> >>>> (grub), avoids convoluted platform cert management (shim) and just
> >>>
> >>> This is why U-Boot supports UEFI boot options specifying both a
> >>> binary
> >>> as well as an initial RAM disk.
> >>
> >> I might be late to this, but where does the devicetree come from? As
> >> far
> >> as I understand it right now, for most (all) the SytemReady IR
> >> certified
> >> boards, the compiled-in one from u-boot is passed to linux. This won't
> >> work
> >> in the long run, because the devicetrees keep getting incompatible
> >> changes.
> >> So while it work for one kernel version it might not work on the next
> >> version.
> >
> > It would make sense to add the fdt devicepath to the bootoption like
> > we did it for the initrd.
>
> I haven't followed the much of the recent development, are there any
> commits/files I can look at?
>
> And I'm not just talking about the use case where the EFI stub of linux
> is booted directy, but also when there is grub in between.
>
> The distribution would supply a bunch of devicetrees along with
> the kernel (and initrd). Possibly also different versions of these.
> In grub you would choose the desired kernel/initrd/devicetree
> combination.
>
> With SystemReady, DT from distros are ignored.
With SecureBoot on, dtb=<file> on the command line is disabled.
UEFI can have multiple boot options. Each boot option comes with:
- a list of "device paths" --> dp[n]
- a command line
The "device paths" are not your typical file path in operating system. They
can include configuration information (line speed for a UART along with
terminal emulation parameters). Each dp can be given a specific class. An
initrd "vendor media" class was created.
At present, dp[0] is loaded as the UEFI command and dp[1..n] are ignored.
Linux kernel EFI_STUB can ask "load the "initrd vendor media here"; the EFI
subsystem with browse through dp[1-n] for that element and load it.
If measured boot is on, the initrd will be measured by the UEFI subsystem.
So by having multiple BootXXX variables, one can achieve multiple kernel
selections.

But this means grub would need to (1) know the filename of the
> devicetree, or (2) it is hardcoded in the grub config which is
> generated by the distribution.
>
> For (1) you'd need to convey the information from u-boot to grub.
> (2) would mean the distribution will have to figure out the suitable
> devicetree.
>
> To make things worse, there are boards which have several
> different devicetrees or there might even be overlays. Eg. my
> board has different devicetrees which are expected to be chosen
> by the user by setting the fdtfile variable.
>
> That was practical when the "user" is also a developer. Things have to
mature to a different model, yet offering same flexibility.
For example some NXP layerscape processors can define the function of pins
by configuring a SerDes. A pin can thus be a PCI lane or an ethernet MII
lane. A board is built with either PCI or Ethernet connectors or devices.
It has nothing to do with a "user" choice: the DT passed to the OS need to
reflect the hardware configuration. U-Boot need the SerDes device DT to be
able to configure the SoC to match the board hardware. The RichOS shall not
see the SerDes, just the PCI or Ethernet things. Those are known in advance
and associated with the board itself.
Could you describe the use case for multiple device trees for one board?



> -michael
>


-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Michael Walle]

[adding Vladimir, because he showed interest in this, too]

Am 2021-09-30 15:56, schrieb François Ozog:
> On Thu, 30 Sept 2021 at 14:07, Michael Walle <michael@walle.cc> wrote:
> 
>> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
>>> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
>>> <michael@walle.cc>:
>>>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
>>>>> On 9/30/21 8:23 AM, François Ozog wrote:
>>>> [..]
>>>>>> Is U-Boot's UEFI loader implementation supposed to be the
>>>>>> recommended
>>>>>> UEFI firmware on ARM and RISC-V on a production /
>> deployment
>>>>>> system?
>>>>>> 
>>>>>> For Arm: yes, that is SystemReady spec.
>>>>> 
>>>>> For RISC-V it is required by the RISC-V platform specification.
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB,
>> or
>>>>>> do
>>>>>> we
>>>>>> expect to load grub.efi which chain-loads a kernel without
>>>>>> CONFIG_EFI_STUB?
>>>>>> 
>>>>>> all paths should be possible , and the shim.efi is to be
>> supported
>>>>>> too.
>>>>>> With UEFI, I always see that UEFI is kept down to OS for
>> SecureBoot.
>>>>>> In
>>>>>> other words I don’t see grub.efi booting a non
>> config_efi_stub.
>>>>>> 
>>>>>> What do distributions normally do?
>>>>>> 
>>>>>> At least Red Hat made it clear at multiple Linaro Connect that
>> they
>>>>>> want
>>>>>> standards, and SystemReady is one that makes the life of
>> embedded
>>>>>> distros easy.
>>>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
>>>>>> SecureBoot.
>>>>> 
>>>>> For Fedora see
>>>>> 
>> 
> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
>>>>> 
>>>>> SUSE started the UEFI implementation to boot on all
>> architectures in
>>>>> the
>>>>> same way. The current ARM and RISC-V images uses UEFI.
>>>>> 
>>>>> Debian and Ubuntu install for booting via GRUB if the installer
>> is
>>>>> invoked via UEFI. A fallback solution using the legacy Linux
>> entry
>>>>> point
>>>>> exists.
>>>>> 
>>>>> For BSD the only way to boot on ARM is via UEFI.
>>>>> 
>>>>>> 
>>>>>> What's our
>>>>>> position when compared to EDK II?
>>>>> 
>>>>> U-Boot implements only a subset of UEFI defined in the EBBR
>>>>> specification.
>>>>> 
>>>>> For servers you need a larger scope which is offered by EDK II.
>> This
>>>>> is
>>>>> required both by the RISC-V platform specification as well as
>> the ARM
>>>>> SystemReady ServerReady profile.
>>>>> 
>>>>> The number of boards supported by upstream EDK II is very low.
>> But
>>>>> proprietary firmware based on EDK II exists.
>>>>> 
>>>>>> 
>>>>>> the typical distro boot flow is not the most efficient and
>> drags
>>>>>> concept
>>>>>> dating where the Microsoft certs had to be part of the picture.
>> A
>>>>>> direct
>>>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the
>> boot
>>>>>> path
>>>>>> (grub), avoids convoluted platform cert management (shim) and
>> just
>>>>> 
>>>>> This is why U-Boot supports UEFI boot options specifying both a
>>>>> binary
>>>>> as well as an initial RAM disk.
>>>> 
>>>> I might be late to this, but where does the devicetree come from?
>> As
>>>> far
>>>> as I understand it right now, for most (all) the SytemReady IR
>>>> certified
>>>> boards, the compiled-in one from u-boot is passed to linux. This
>> won't
>>>> work
>>>> in the long run, because the devicetrees keep getting
>> incompatible
>>>> changes.
>>>> So while it work for one kernel version it might not work on the
>> next
>>>> version.
>>> 
>>> It would make sense to add the fdt devicepath to the bootoption
>> like
>>> we did it for the initrd.
>> 
>> I haven't followed the much of the recent development, are there any
>> commits/files I can look at?
>> 
>> And I'm not just talking about the use case where the EFI stub of
>> linux
>> is booted directy, but also when there is grub in between.
>> 
>> The distribution would supply a bunch of devicetrees along with
>> the kernel (and initrd). Possibly also different versions of these.
>> In grub you would choose the desired kernel/initrd/devicetree
>> combination.
> 
> With SystemReady, DT from distros are ignored.

Err? Is this really true, I know about some incompatible changes
to the device tree which prevents you from using usb (or even a
kernel panic) with the imx8mm and I know that on the ls1028a
flexspi wont work if the devicetree doesn't match the kernel.
I.e. if you have a device tree from kernel 5.14 you cannot
use it on 5.10. If you have one from 5.10 you cannot use it on
a later kernel.

So, IMHO this is alreay a non-starter for SystemReady if it
should work beyond having a serial console and some storage :(

> With SecureBoot on, dtb=<file> on the command line is disabled.
> UEFI can have multiple boot options. Each boot option comes with:
> - a list of "device paths" --> dp[n]
> - a command line
> The "device paths" are not your typical file path in operating system.
> They can include configuration information (line speed for a UART
> along with terminal emulation parameters). Each dp can be given a
> specific class. An initrd "vendor media" class was created.
> At present, dp[0] is loaded as the UEFI command and dp[1..n] are
> ignored.
> Linux kernel EFI_STUB can ask "load the "initrd vendor media here";
> the EFI subsystem with browse through dp[1-n] for that element and
> load it.
> If measured boot is on, the initrd will be measured by the UEFI
> subsystem.
> So by having multiple BootXXX variables, one can achieve multiple
> kernel selections.
> 
>> But this means grub would need to (1) know the filename of the
>> devicetree, or (2) it is hardcoded in the grub config which is
>> generated by the distribution.
>> 
>> For (1) you'd need to convey the information from u-boot to grub.
>> (2) would mean the distribution will have to figure out the suitable
>> devicetree.
>> 
>> To make things worse, there are boards which have several
>> different devicetrees or there might even be overlays. Eg. my
>> board has different devicetrees which are expected to be chosen
>> by the user by setting the fdtfile variable.
> 
> That was practical when the "user" is also a developer. Things have to
> mature to a different model, yet offering same flexibility.
> For example some NXP layerscape processors can define the function of
> pins by configuring a SerDes. A pin can thus be a PCI lane or an
> ethernet MII lane. A board is built with either PCI or Ethernet
> connectors or devices. It has nothing to do with a "user" choice: the
> DT passed to the OS need to reflect the hardware configuration. U-Boot
> need the SerDes device DT to be able to configure the SoC to match the
> board hardware. The RichOS shall not see the SerDes, just the PCI or
> Ethernet things. Those are known in advance and associated with the
> board itself.

Thats understood and clear. But there are also different things to
consider, where the user might actually change something not related
to hardware but which is also contained in the devicetree. The
internal switch connection (and configuration) on the LS1028A comes
to mind (maybe Vladimir can be more specific here). In general, there
might be more than just a description of the hardware in the devicetree.

> Could you describe the use case for multiple device trees for one
> board?

The board is actually a plug-in module which you can put into
different carriers. The module comes in different variants (1
to 4). This is detected by the bootloader and the correct variant
device tree is loaded, but OTOH the carrier is not detected. This
is something the user has to provide (by now, they have to set
the fdtfile variable to match their carrier).

You can also have a look at
linux/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28*dts
for actual devicetrees.

-michael

Re: Driver model at UEFI runtime

[François Ozog]

On Thu, 30 Sept 2021 at 17:12, Michael Walle <michael@walle.cc> wrote:

> [adding Vladimir, because he showed interest in this, too]
>
> Am 2021-09-30 15:56, schrieb François Ozog:
> > On Thu, 30 Sept 2021 at 14:07, Michael Walle <michael@walle.cc> wrote:
> >
> >> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
> >>> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
> >>> <michael@walle.cc>:
> >>>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
> >>>>> On 9/30/21 8:23 AM, François Ozog wrote:
> >>>> [..]
> >>>>>> Is U-Boot's UEFI loader implementation supposed to be the
> >>>>>> recommended
> >>>>>> UEFI firmware on ARM and RISC-V on a production /
> >> deployment
> >>>>>> system?
> >>>>>>
> >>>>>> For Arm: yes, that is SystemReady spec.
> >>>>>
> >>>>> For RISC-V it is required by the RISC-V platform specification.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB,
> >> or
> >>>>>> do
> >>>>>> we
> >>>>>> expect to load grub.efi which chain-loads a kernel without
> >>>>>> CONFIG_EFI_STUB?
> >>>>>>
> >>>>>> all paths should be possible , and the shim.efi is to be
> >> supported
> >>>>>> too.
> >>>>>> With UEFI, I always see that UEFI is kept down to OS for
> >> SecureBoot.
> >>>>>> In
> >>>>>> other words I don’t see grub.efi booting a non
> >> config_efi_stub.
> >>>>>>
> >>>>>> What do distributions normally do?
> >>>>>>
> >>>>>> At least Red Hat made it clear at multiple Linaro Connect that
> >> they
> >>>>>> want
> >>>>>> standards, and SystemReady is one that makes the life of
> >> embedded
> >>>>>> distros easy.
> >>>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from UEFi
> >>>>>> SecureBoot.
> >>>>>
> >>>>> For Fedora see
> >>>>>
> >>
> > https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
> >>>>>
> >>>>> SUSE started the UEFI implementation to boot on all
> >> architectures in
> >>>>> the
> >>>>> same way. The current ARM and RISC-V images uses UEFI.
> >>>>>
> >>>>> Debian and Ubuntu install for booting via GRUB if the installer
> >> is
> >>>>> invoked via UEFI. A fallback solution using the legacy Linux
> >> entry
> >>>>> point
> >>>>> exists.
> >>>>>
> >>>>> For BSD the only way to boot on ARM is via UEFI.
> >>>>>
> >>>>>>
> >>>>>> What's our
> >>>>>> position when compared to EDK II?
> >>>>>
> >>>>> U-Boot implements only a subset of UEFI defined in the EBBR
> >>>>> specification.
> >>>>>
> >>>>> For servers you need a larger scope which is offered by EDK II.
> >> This
> >>>>> is
> >>>>> required both by the RISC-V platform specification as well as
> >> the ARM
> >>>>> SystemReady ServerReady profile.
> >>>>>
> >>>>> The number of boards supported by upstream EDK II is very low.
> >> But
> >>>>> proprietary firmware based on EDK II exists.
> >>>>>
> >>>>>>
> >>>>>> the typical distro boot flow is not the most efficient and
> >> drags
> >>>>>> concept
> >>>>>> dating where the Microsoft certs had to be part of the picture.
> >> A
> >>>>>> direct
> >>>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in the
> >> boot
> >>>>>> path
> >>>>>> (grub), avoids convoluted platform cert management (shim) and
> >> just
> >>>>>
> >>>>> This is why U-Boot supports UEFI boot options specifying both a
> >>>>> binary
> >>>>> as well as an initial RAM disk.
> >>>>
> >>>> I might be late to this, but where does the devicetree come from?
> >> As
> >>>> far
> >>>> as I understand it right now, for most (all) the SytemReady IR
> >>>> certified
> >>>> boards, the compiled-in one from u-boot is passed to linux. This
> >> won't
> >>>> work
> >>>> in the long run, because the devicetrees keep getting
> >> incompatible
> >>>> changes.
> >>>> So while it work for one kernel version it might not work on the
> >> next
> >>>> version.
> >>>
> >>> It would make sense to add the fdt devicepath to the bootoption
> >> like
> >>> we did it for the initrd.
> >>
> >> I haven't followed the much of the recent development, are there any
> >> commits/files I can look at?
> >>
> >> And I'm not just talking about the use case where the EFI stub of
> >> linux
> >> is booted directy, but also when there is grub in between.
> >>
> >> The distribution would supply a bunch of devicetrees along with
> >> the kernel (and initrd). Possibly also different versions of these.
> >> In grub you would choose the desired kernel/initrd/devicetree
> >> combination.
> >
> > With SystemReady, DT from distros are ignored.
>
> Err? Is this really true, I know about some incompatible changes
> to the device tree which prevents you from using usb (or even a
> kernel panic) with the imx8mm and I know that on the ls1028a
> flexspi wont work if the devicetree doesn't match the kernel.
> I.e. if you have a device tree from kernel 5.14 you cannot
> use it on 5.10. If you have one from 5.10 you cannot use it on
> a later kernel.
>
> What you describe is the situation we want to avoid and that comes from
years of tinkering.
https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
lists a number of certified boards and the list is going to grow
significantly.
On those boards, you will be able to boot any kernel.
The image I have in mind with OS provided DT is:
as a French driver, my DT says that the steering wheel is on the left hand
side of the car.
Shall I whine about the cars in England that do not comply to my DT? or
accept to use the car provided DT?
The situation is comfortable for tinkerers, but not sustainable. It is a
matter of organization and not technology to solve the problems you
describe.


> So, IMHO this is alreay a non-starter for SystemReady if it
> should work beyond having a serial console and some storage :(
>
> > With SecureBoot on, dtb=<file> on the command line is disabled.
> > UEFI can have multiple boot options. Each boot option comes with:
> > - a list of "device paths" --> dp[n]
> > - a command line
> > The "device paths" are not your typical file path in operating system.
> > They can include configuration information (line speed for a UART
> > along with terminal emulation parameters). Each dp can be given a
> > specific class. An initrd "vendor media" class was created.
> > At present, dp[0] is loaded as the UEFI command and dp[1..n] are
> > ignored.
> > Linux kernel EFI_STUB can ask "load the "initrd vendor media here";
> > the EFI subsystem with browse through dp[1-n] for that element and
> > load it.
> > If measured boot is on, the initrd will be measured by the UEFI
> > subsystem.
> > So by having multiple BootXXX variables, one can achieve multiple
> > kernel selections.
> >
> >> But this means grub would need to (1) know the filename of the
> >> devicetree, or (2) it is hardcoded in the grub config which is
> >> generated by the distribution.
> >>
> >> For (1) you'd need to convey the information from u-boot to grub.
> >> (2) would mean the distribution will have to figure out the suitable
> >> devicetree.
> >>
> >> To make things worse, there are boards which have several
> >> different devicetrees or there might even be overlays. Eg. my
> >> board has different devicetrees which are expected to be chosen
> >> by the user by setting the fdtfile variable.
> >
> > That was practical when the "user" is also a developer. Things have to
> > mature to a different model, yet offering same flexibility.
> > For example some NXP layerscape processors can define the function of
> > pins by configuring a SerDes. A pin can thus be a PCI lane or an
> > ethernet MII lane. A board is built with either PCI or Ethernet
> > connectors or devices. It has nothing to do with a "user" choice: the
> > DT passed to the OS need to reflect the hardware configuration. U-Boot
> > need the SerDes device DT to be able to configure the SoC to match the
> > board hardware. The RichOS shall not see the SerDes, just the PCI or
> > Ethernet things. Those are known in advance and associated with the
> > board itself.
>
> Thats understood and clear. But there are also different things to
> consider, where the user might actually change something not related
> to hardware but which is also contained in the devicetree. The
> internal switch connection (and configuration) on the LS1028A comes
> to mind (maybe Vladimir can be more specific here). In general, there
> might be more than just a description of the hardware in the devicetree.
>
> Indeed and DPAA2 configuration also comes to mind. But then where shall
the use configuration be described?
DT shall describe the hardware, and not its configuration which tends to be
OS dependent or more precisely, driver dependent.
U-Boot is probably the best placed to do some of the work. It can have ways
to describe the desired configuration in auxiliary persistent configuration
data.
That auxiliary information can be stored in various places under various
formats. One possible way is to have overlays.
Another one is to have U-Boot use DT format as way to stored some
configuration (U-Boot private bindings). This method is actually used by
TF-A, for instance to describe the static secure partitions.

> Could you describe the use case for multiple device trees for one
> > board?
>
> The board is actually a plug-in module which you can put into
> different carriers. The module comes in different variants (1
> to 4). This is detected by the bootloader and the correct variant
> device tree is loaded, but OTOH the carrier is not detected. This
> is something the user has to provide (by now, they have to set
> the fdtfile variable to match their carrier).
>
> You can also have a look at
> linux/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28*dts
> for actual devicetrees.
>
> For hats, capes, SoMs, carrier boards and others, we need to better
organize.
The EEPROM based approach by RPI and Beagle shall give us hints on how to
deal with this at scale in an industrial manner.
We have EEPROM SPDs for DIMMs: it feels so natural to leverage the same
concept to bring enumeration to those hot/cold pluggable elements.
It is my intent to sit down with RPI, Beagle and 96Boards to agree on
something around those lines.
In the meantime, manual composition is a nuisance with have to deal with.
But for sure, that should be entirely transparent to the OS.

Side note to read in a positive mindset even if it does not look positive:
If we keep things like they are, what may end up happening is that business
leaders will direct their teams to use ACPI to avoid DT related pains.
One would say ACPI does not support everything and it comes with EDK2...
Well, let's just be patient and we will witness that business guys have a
different view of what is a solution to a problem.

-michael
>


-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Michael Walle]

Am 2021-09-30 17:47, schrieb François Ozog:
> On Thu, 30 Sept 2021 at 17:12, Michael Walle <michael@walle.cc> wrote:
> 
>> [adding Vladimir, because he showed interest in this, too]
>> 
>> Am 2021-09-30 15:56, schrieb François Ozog:
>>> On Thu, 30 Sept 2021 at 14:07, Michael Walle <michael@walle.cc>
>> wrote:
>>> 
>>>> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
>>>>> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
>>>>> <michael@walle.cc>:
>>>>>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
>>>>>>> On 9/30/21 8:23 AM, François Ozog wrote:
>>>>>> [..]
>>>>>>>> Is U-Boot's UEFI loader implementation supposed to be the
>>>>>>>> recommended
>>>>>>>> UEFI firmware on ARM and RISC-V on a production /
>>>> deployment
>>>>>>>> system?
>>>>>>>> 
>>>>>>>> For Arm: yes, that is SystemReady spec.
>>>>>>> 
>>>>>>> For RISC-V it is required by the RISC-V platform
>> specification.
>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB,
>>>> or
>>>>>>>> do
>>>>>>>> we
>>>>>>>> expect to load grub.efi which chain-loads a kernel without
>>>>>>>> CONFIG_EFI_STUB?
>>>>>>>> 
>>>>>>>> all paths should be possible , and the shim.efi is to be
>>>> supported
>>>>>>>> too.
>>>>>>>> With UEFI, I always see that UEFI is kept down to OS for
>>>> SecureBoot.
>>>>>>>> In
>>>>>>>> other words I don’t see grub.efi booting a non
>>>> config_efi_stub.
>>>>>>>> 
>>>>>>>> What do distributions normally do?
>>>>>>>> 
>>>>>>>> At least Red Hat made it clear at multiple Linaro Connect
>> that
>>>> they
>>>>>>>> want
>>>>>>>> standards, and SystemReady is one that makes the life of
>>>> embedded
>>>>>>>> distros easy.
>>>>>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from
>> UEFi
>>>>>>>> SecureBoot.
>>>>>>> 
>>>>>>> For Fedora see
>>>>>>> 
>>>> 
>>> 
>> 
> https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
>>>>>>> 
>>>>>>> SUSE started the UEFI implementation to boot on all
>>>> architectures in
>>>>>>> the
>>>>>>> same way. The current ARM and RISC-V images uses UEFI.
>>>>>>> 
>>>>>>> Debian and Ubuntu install for booting via GRUB if the
>> installer
>>>> is
>>>>>>> invoked via UEFI. A fallback solution using the legacy Linux
>>>> entry
>>>>>>> point
>>>>>>> exists.
>>>>>>> 
>>>>>>> For BSD the only way to boot on ARM is via UEFI.
>>>>>>> 
>>>>>>>> 
>>>>>>>> What's our
>>>>>>>> position when compared to EDK II?
>>>>>>> 
>>>>>>> U-Boot implements only a subset of UEFI defined in the EBBR
>>>>>>> specification.
>>>>>>> 
>>>>>>> For servers you need a larger scope which is offered by EDK
>> II.
>>>> This
>>>>>>> is
>>>>>>> required both by the RISC-V platform specification as well as
>>>> the ARM
>>>>>>> SystemReady ServerReady profile.
>>>>>>> 
>>>>>>> The number of boards supported by upstream EDK II is very low.
>>>> But
>>>>>>> proprietary firmware based on EDK II exists.
>>>>>>> 
>>>>>>>> 
>>>>>>>> the typical distro boot flow is not the most efficient and
>>>> drags
>>>>>>>> concept
>>>>>>>> dating where the Microsoft certs had to be part of the
>> picture.
>>>> A
>>>>>>>> direct
>>>>>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in
>> the
>>>> boot
>>>>>>>> path
>>>>>>>> (grub), avoids convoluted platform cert management (shim) and
>>>> just
>>>>>>> 
>>>>>>> This is why U-Boot supports UEFI boot options specifying both
>> a
>>>>>>> binary
>>>>>>> as well as an initial RAM disk.
>>>>>> 
>>>>>> I might be late to this, but where does the devicetree come
>> from?
>>>> As
>>>>>> far
>>>>>> as I understand it right now, for most (all) the SytemReady IR
>>>>>> certified
>>>>>> boards, the compiled-in one from u-boot is passed to linux.
>> This
>>>> won't
>>>>>> work
>>>>>> in the long run, because the devicetrees keep getting
>>>> incompatible
>>>>>> changes.
>>>>>> So while it work for one kernel version it might not work on
>> the
>>>> next
>>>>>> version.
>>>>> 
>>>>> It would make sense to add the fdt devicepath to the bootoption
>>>> like
>>>>> we did it for the initrd.
>>>> 
>>>> I haven't followed the much of the recent development, are there
>> any
>>>> commits/files I can look at?
>>>> 
>>>> And I'm not just talking about the use case where the EFI stub of
>>>> linux
>>>> is booted directy, but also when there is grub in between.
>>>> 
>>>> The distribution would supply a bunch of devicetrees along with
>>>> the kernel (and initrd). Possibly also different versions of
>> these.
>>>> In grub you would choose the desired kernel/initrd/devicetree
>>>> combination.
>>> 
>>> With SystemReady, DT from distros are ignored.
>> 
>> Err? Is this really true, I know about some incompatible changes
>> to the device tree which prevents you from using usb (or even a
>> kernel panic) with the imx8mm and I know that on the ls1028a
>> flexspi wont work if the devicetree doesn't match the kernel.
>> I.e. if you have a device tree from kernel 5.14 you cannot
>> use it on 5.10. If you have one from 5.10 you cannot use it on
>> a later kernel.
> 
> What you describe is the situation we want to avoid and that comes
> from years of tinkering.

how is that tinkering? That means once you have a device tree, it is
set in stone. which isn't reality, sorry.

Consider the ls1028a and the flexspi "bug". For the 5.10 kernel/dtb
there was a wrong clock connected to the flexspi device. There wasn't
even a driver for the correct clock. Thus, I introduced a new clock
driver and wired it correctly in the device tree. Which was introduced
in 5.11 (or 5.12 I don't know anymore). u-boot is now providing a
devicetree from the 5.14 kernel. Thus, it has the clock input connected
to the new clock driver. But debian, for example, has kernel 5.10. Which
doesn't have the clock driver, oops. The flexspi tries to enable the
clock which fails and the whole probe will fail.

Regarding the imx8mm usb error, apparently the node was renamed. If 
you're
using older kernels with newer dtbs, the kernel oopes. If you're using a
newer kernel with older dtbs, USB doesn't get probed. (Although I was
told that there is a "fix" for this, that is, both node names are 
tried.)

> https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
> lists a number of certified boards and the list is going to grow
> significantly.

And the sl28 board will likely be there soon, too.

> On those boards, you will be able to boot any kernel.

Actually I don't think so, because you might hit the imx8mm bug, too. 
May
just be lucky by the devicetree/kernel combination.

> The image I have in mind with OS provided DT is:
> as a French driver, my DT says that the steering wheel is on the left
> hand side of the car.
> Shall I whine about the cars in England that do not comply to my DT?
> or accept to use the car provided DT?
> The situation is comfortable for tinkerers, but not sustainable. It is
> a matter of organization and not technology to solve the problems you
> describe.

Mh, I'm not sure I understand what you're trying to tell me. The
distribution also provides you the kernel, why shouldn't it provide
devicetree which exactly matches this kernel and was also tested
against.

>> So, IMHO this is alreay a non-starter for SystemReady if it
>> should work beyond having a serial console and some storage :(
>> 
>>> With SecureBoot on, dtb=<file> on the command line is disabled.
>>> UEFI can have multiple boot options. Each boot option comes with:
>>> - a list of "device paths" --> dp[n]
>>> - a command line
>>> The "device paths" are not your typical file path in operating
>> system.
>>> They can include configuration information (line speed for a UART
>>> along with terminal emulation parameters). Each dp can be given a
>>> specific class. An initrd "vendor media" class was created.
>>> At present, dp[0] is loaded as the UEFI command and dp[1..n] are
>>> ignored.
>>> Linux kernel EFI_STUB can ask "load the "initrd vendor media
>> here";
>>> the EFI subsystem with browse through dp[1-n] for that element and
>>> load it.
>>> If measured boot is on, the initrd will be measured by the UEFI
>>> subsystem.
>>> So by having multiple BootXXX variables, one can achieve multiple
>>> kernel selections.
>>> 
>>>> But this means grub would need to (1) know the filename of the
>>>> devicetree, or (2) it is hardcoded in the grub config which is
>>>> generated by the distribution.
>>>> 
>>>> For (1) you'd need to convey the information from u-boot to grub.
>>>> (2) would mean the distribution will have to figure out the
>> suitable
>>>> devicetree.
>>>> 
>>>> To make things worse, there are boards which have several
>>>> different devicetrees or there might even be overlays. Eg. my
>>>> board has different devicetrees which are expected to be chosen
>>>> by the user by setting the fdtfile variable.
>>> 
>>> That was practical when the "user" is also a developer. Things
>> have to
>>> mature to a different model, yet offering same flexibility.
>>> For example some NXP layerscape processors can define the function
>> of
>>> pins by configuring a SerDes. A pin can thus be a PCI lane or an
>>> ethernet MII lane. A board is built with either PCI or Ethernet
>>> connectors or devices. It has nothing to do with a "user" choice:
>> the
>>> DT passed to the OS need to reflect the hardware configuration.
>> U-Boot
>>> need the SerDes device DT to be able to configure the SoC to match
>> the
>>> board hardware. The RichOS shall not see the SerDes, just the PCI
>> or
>>> Ethernet things. Those are known in advance and associated with
>> the
>>> board itself.
>> 
>> Thats understood and clear. But there are also different things to
>> consider, where the user might actually change something not related
>> to hardware but which is also contained in the devicetree. The
>> internal switch connection (and configuration) on the LS1028A comes
>> to mind (maybe Vladimir can be more specific here). In general,
>> there
>> might be more than just a description of the hardware in the
>> devicetree.
> 
> Indeed and DPAA2 configuration also comes to mind. But then where
> shall the use configuration be described?
> DT shall describe the hardware, and not its configuration which tends
> to be OS dependent or more precisely, driver dependent.
> U-Boot is probably the best placed to do some of the work. It can have
> ways to describe the desired configuration in auxiliary persistent
> configuration data.
> That auxiliary information can be stored in various places under
> various formats. One possible way is to have overlays.
> Another one is to have U-Boot use DT format as way to stored some
> configuration (U-Boot private bindings). This method is actually used
> by TF-A, for instance to describe the static secure partitions.
> 
>>> Could you describe the use case for multiple device trees for one
>>> board?
>> 
>> The board is actually a plug-in module which you can put into
>> different carriers. The module comes in different variants (1
>> to 4). This is detected by the bootloader and the correct variant
>> device tree is loaded, but OTOH the carrier is not detected. This
>> is something the user has to provide (by now, they have to set
>> the fdtfile variable to match their carrier).
>> 
>> You can also have a look at
>> linux/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28*dts
>> for actual devicetrees.
> 
> For hats, capes, SoMs, carrier boards and others, we need to better
> organize.
> The EEPROM based approach by RPI and Beagle shall give us hints on how
> to deal with this at scale in an industrial manner.
> We have EEPROM SPDs for DIMMs: it feels so natural to leverage the
> same concept to bring enumeration to those hot/cold pluggable
> elements.
> It is my intent to sit down with RPI, Beagle and 96Boards to agree on
> something around those lines.
> In the meantime, manual composition is a nuisance with have to deal
> with. But for sure, that should be entirely transparent to the OS.

Well there are boards out there, where auto detection is not possible,
or intentionally not wanted. And of course it should be transparent
to the operation system. But you have a way to load a different
devicetree but the _only_ one you got with your firmware (or let it
be a set of devicetrees, doesn't matter).

-michael

> Side note to read in a positive mindset even if it does not look
> positive:
> If we keep things like they are, what may end up happening is that
> business leaders will direct their teams to use ACPI to avoid DT
> related pains.
> One would say ACPI does not support everything and it comes with
> EDK2...
> Well, let's just be patient and we will witness that business guys
> have a different view of what is a solution to a problem.

Re: Driver model at UEFI runtime

[François Ozog]

On Thu, 30 Sept 2021 at 18:25, Michael Walle <michael@walle.cc> wrote:

> Am 2021-09-30 17:47, schrieb François Ozog:
> > On Thu, 30 Sept 2021 at 17:12, Michael Walle <michael@walle.cc> wrote:
> >
> >> [adding Vladimir, because he showed interest in this, too]
> >>
> >> Am 2021-09-30 15:56, schrieb François Ozog:
> >>> On Thu, 30 Sept 2021 at 14:07, Michael Walle <michael@walle.cc>
> >> wrote:
> >>>
> >>>> Am 2021-09-30 12:50, schrieb Heinrich Schuchardt:
> >>>>> Am 30. September 2021 11:53:47 MESZ schrieb Michael Walle
> >>>>> <michael@walle.cc>:
> >>>>>> Am 2021-09-30 08:56, schrieb Heinrich Schuchardt:
> >>>>>>> On 9/30/21 8:23 AM, François Ozog wrote:
> >>>>>> [..]
> >>>>>>>> Is U-Boot's UEFI loader implementation supposed to be the
> >>>>>>>> recommended
> >>>>>>>> UEFI firmware on ARM and RISC-V on a production /
> >>>> deployment
> >>>>>>>> system?
> >>>>>>>>
> >>>>>>>> For Arm: yes, that is SystemReady spec.
> >>>>>>>
> >>>>>>> For RISC-V it is required by the RISC-V platform
> >> specification.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Do we expect bootefi to boot a kernel with CONFIG_EFI_STUB,
> >>>> or
> >>>>>>>> do
> >>>>>>>> we
> >>>>>>>> expect to load grub.efi which chain-loads a kernel without
> >>>>>>>> CONFIG_EFI_STUB?
> >>>>>>>>
> >>>>>>>> all paths should be possible , and the shim.efi is to be
> >>>> supported
> >>>>>>>> too.
> >>>>>>>> With UEFI, I always see that UEFI is kept down to OS for
> >>>> SecureBoot.
> >>>>>>>> In
> >>>>>>>> other words I don’t see grub.efi booting a non
> >>>> config_efi_stub.
> >>>>>>>>
> >>>>>>>> What do distributions normally do?
> >>>>>>>>
> >>>>>>>> At least Red Hat made it clear at multiple Linaro Connect
> >> that
> >>>> they
> >>>>>>>> want
> >>>>>>>> standards, and SystemReady is one that makes the life of
> >>>> embedded
> >>>>>>>> distros easy.
> >>>>>>>> Distros boot shim.efi, grub.efi, Linux.efi to benefit from
> >> UEFi
> >>>>>>>> SecureBoot.
> >>>>>>>
> >>>>>>> For Fedora see
> >>>>>>>
> >>>>
> >>>
> >>
> > https://fedoraproject.org/wiki/Changes/uEFIforARMv7#Detailed_Description
> >>>>>>>
> >>>>>>> SUSE started the UEFI implementation to boot on all
> >>>> architectures in
> >>>>>>> the
> >>>>>>> same way. The current ARM and RISC-V images uses UEFI.
> >>>>>>>
> >>>>>>> Debian and Ubuntu install for booting via GRUB if the
> >> installer
> >>>> is
> >>>>>>> invoked via UEFI. A fallback solution using the legacy Linux
> >>>> entry
> >>>>>>> point
> >>>>>>> exists.
> >>>>>>>
> >>>>>>> For BSD the only way to boot on ARM is via UEFI.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> What's our
> >>>>>>>> position when compared to EDK II?
> >>>>>>>
> >>>>>>> U-Boot implements only a subset of UEFI defined in the EBBR
> >>>>>>> specification.
> >>>>>>>
> >>>>>>> For servers you need a larger scope which is offered by EDK
> >> II.
> >>>> This
> >>>>>>> is
> >>>>>>> required both by the RISC-V platform specification as well as
> >>>> the ARM
> >>>>>>> SystemReady ServerReady profile.
> >>>>>>>
> >>>>>>> The number of boards supported by upstream EDK II is very low.
> >>>> But
> >>>>>>> proprietary firmware based on EDK II exists.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> the typical distro boot flow is not the most efficient and
> >>>> drags
> >>>>>>>> concept
> >>>>>>>> dating where the Microsoft certs had to be part of the
> >> picture.
> >>>> A
> >>>>>>>> direct
> >>>>>>>> U-Boot Linux.efi is my preferred; avoids yet another OS in
> >> the
> >>>> boot
> >>>>>>>> path
> >>>>>>>> (grub), avoids convoluted platform cert management (shim) and
> >>>> just
> >>>>>>>
> >>>>>>> This is why U-Boot supports UEFI boot options specifying both
> >> a
> >>>>>>> binary
> >>>>>>> as well as an initial RAM disk.
> >>>>>>
> >>>>>> I might be late to this, but where does the devicetree come
> >> from?
> >>>> As
> >>>>>> far
> >>>>>> as I understand it right now, for most (all) the SytemReady IR
> >>>>>> certified
> >>>>>> boards, the compiled-in one from u-boot is passed to linux.
> >> This
> >>>> won't
> >>>>>> work
> >>>>>> in the long run, because the devicetrees keep getting
> >>>> incompatible
> >>>>>> changes.
> >>>>>> So while it work for one kernel version it might not work on
> >> the
> >>>> next
> >>>>>> version.
> >>>>>
> >>>>> It would make sense to add the fdt devicepath to the bootoption
> >>>> like
> >>>>> we did it for the initrd.
> >>>>
> >>>> I haven't followed the much of the recent development, are there
> >> any
> >>>> commits/files I can look at?
> >>>>
> >>>> And I'm not just talking about the use case where the EFI stub of
> >>>> linux
> >>>> is booted directy, but also when there is grub in between.
> >>>>
> >>>> The distribution would supply a bunch of devicetrees along with
> >>>> the kernel (and initrd). Possibly also different versions of
> >> these.
> >>>> In grub you would choose the desired kernel/initrd/devicetree
> >>>> combination.
> >>>
> >>> With SystemReady, DT from distros are ignored.
> >>
> >> Err? Is this really true, I know about some incompatible changes
> >> to the device tree which prevents you from using usb (or even a
> >> kernel panic) with the imx8mm and I know that on the ls1028a
> >> flexspi wont work if the devicetree doesn't match the kernel.
> >> I.e. if you have a device tree from kernel 5.14 you cannot
> >> use it on 5.10. If you have one from 5.10 you cannot use it on
> >> a later kernel.
> >
> > What you describe is the situation we want to avoid and that comes
> > from years of tinkering.
>
> how is that tinkering? That means once you have a device tree, it is
> set in stone. which isn't reality, sorry.
>
> Consider the ls1028a and the flexspi "bug". For the 5.10 kernel/dtb
> there was a wrong clock connected to the flexspi device. There wasn't
> even a driver for the correct clock.

The clock could have been described even though there was no Linux driver.
DT is there to describe HW, not the capacity of a particular OS or boot
firmware to deal with it.

> Thus, I introduced a new clock
> driver and wired it correctly in the device tree. Which was introduced
> in 5.11 (or 5.12 I don't know anymore). u-boot is now providing a
> devicetree from the 5.14 kernel. Thus, it has the clock input connected
> to the new clock driver. But debian, for example, has kernel 5.10. Which
> doesn't have the clock driver, oops. The flexspi tries to enable the
> clock which fails and the whole probe will fail.


>
Regarding the imx8mm usb error, apparently the node was renamed. If
> you're
> using older kernels with newer dtbs, the kernel oopes. If you're using a
> newer kernel with older dtbs, USB doesn't get probed. (Although I was
> told that there is a "fix" for this, that is, both node names are
> tried.)
>
> I keep seeing those issues about node renames or compatible string changes.
That's the tinkering I am talking about.
There is no in-kernel ABI but Linus bang heads if anyone touches
userland-kernel ABI inappropriately.
As DT is mostly handled in-kernel, people treat DT as no-ABI.
But it is part of firmware-kernel ABI.
Until we properly organize this firmware-kernel ABI, the problem you
describe and more will continue.
Now the DT lifecycle before the firmware-kernel ABI also needs to be
specified so that we properly handle hats, capes, SoMs, carrier boards...

>
> https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
> > lists a number of certified boards and the list is going to grow
> > significantly.
>
> And the sl28 board will likely be there soon, too.
>
> > On those boards, you will be able to boot any kernel.
>
> Actually I don't think so, because you might hit the imx8mm bug, too.
> May
> just be lucky by the devicetree/kernel combination.
>
> > The image I have in mind with OS provided DT is:
> > as a French driver, my DT says that the steering wheel is on the left
> > hand side of the car.
> > Shall I whine about the cars in England that do not comply to my DT?
> > or accept to use the car provided DT?
> > The situation is comfortable for tinkerers, but not sustainable. It is
> > a matter of organization and not technology to solve the problems you
> > describe.
>
> Mh, I'm not sure I understand what you're trying to tell me. The
> distribution also provides you the kernel, why shouldn't it provide
> devicetree which exactly matches this kernel and was also tested
> against.
>
Because the kernel has no clue which hats, capes has been added for
instance.
The kernel provided DTB works fine when the firmware builder and OS
builders are the same.
This traditional model is evolving and the team that deals with OS may not
even be in the same company as the one providing the firmware.
Ask the Fedora IoT architect what he thinks about the distro provided DTs
;-)
There is a need to deal with DT bugs. OS provided DT is a bad solution to a
real problem.
U-Boot patches look a possibility for those bugs.

>
> >> So, IMHO this is alreay a non-starter for SystemReady if it
> >> should work beyond having a serial console and some storage :(
> >>
> >>> With SecureBoot on, dtb=<file> on the command line is disabled.
> >>> UEFI can have multiple boot options. Each boot option comes with:
> >>> - a list of "device paths" --> dp[n]
> >>> - a command line
> >>> The "device paths" are not your typical file path in operating
> >> system.
> >>> They can include configuration information (line speed for a UART
> >>> along with terminal emulation parameters). Each dp can be given a
> >>> specific class. An initrd "vendor media" class was created.
> >>> At present, dp[0] is loaded as the UEFI command and dp[1..n] are
> >>> ignored.
> >>> Linux kernel EFI_STUB can ask "load the "initrd vendor media
> >> here";
> >>> the EFI subsystem with browse through dp[1-n] for that element and
> >>> load it.
> >>> If measured boot is on, the initrd will be measured by the UEFI
> >>> subsystem.
> >>> So by having multiple BootXXX variables, one can achieve multiple
> >>> kernel selections.
> >>>
> >>>> But this means grub would need to (1) know the filename of the
> >>>> devicetree, or (2) it is hardcoded in the grub config which is
> >>>> generated by the distribution.
> >>>>
> >>>> For (1) you'd need to convey the information from u-boot to grub.
> >>>> (2) would mean the distribution will have to figure out the
> >> suitable
> >>>> devicetree.
> >>>>
> >>>> To make things worse, there are boards which have several
> >>>> different devicetrees or there might even be overlays. Eg. my
> >>>> board has different devicetrees which are expected to be chosen
> >>>> by the user by setting the fdtfile variable.
> >>>
> >>> That was practical when the "user" is also a developer. Things
> >> have to
> >>> mature to a different model, yet offering same flexibility.
> >>> For example some NXP layerscape processors can define the function
> >> of
> >>> pins by configuring a SerDes. A pin can thus be a PCI lane or an
> >>> ethernet MII lane. A board is built with either PCI or Ethernet
> >>> connectors or devices. It has nothing to do with a "user" choice:
> >> the
> >>> DT passed to the OS need to reflect the hardware configuration.
> >> U-Boot
> >>> need the SerDes device DT to be able to configure the SoC to match
> >> the
> >>> board hardware. The RichOS shall not see the SerDes, just the PCI
> >> or
> >>> Ethernet things. Those are known in advance and associated with
> >> the
> >>> board itself.
> >>
> >> Thats understood and clear. But there are also different things to
> >> consider, where the user might actually change something not related
> >> to hardware but which is also contained in the devicetree. The
> >> internal switch connection (and configuration) on the LS1028A comes
> >> to mind (maybe Vladimir can be more specific here). In general,
> >> there
> >> might be more than just a description of the hardware in the
> >> devicetree.
> >
> > Indeed and DPAA2 configuration also comes to mind. But then where
> > shall the use configuration be described?
> > DT shall describe the hardware, and not its configuration which tends
> > to be OS dependent or more precisely, driver dependent.
> > U-Boot is probably the best placed to do some of the work. It can have
> > ways to describe the desired configuration in auxiliary persistent
> > configuration data.
> > That auxiliary information can be stored in various places under
> > various formats. One possible way is to have overlays.
> > Another one is to have U-Boot use DT format as way to stored some
> > configuration (U-Boot private bindings). This method is actually used
> > by TF-A, for instance to describe the static secure partitions.
> >
> >>> Could you describe the use case for multiple device trees for one
> >>> board?
> >>
> >> The board is actually a plug-in module which you can put into
> >> different carriers. The module comes in different variants (1
> >> to 4). This is detected by the bootloader and the correct variant
> >> device tree is loaded, but OTOH the carrier is not detected. This
> >> is something the user has to provide (by now, they have to set
> >> the fdtfile variable to match their carrier).
> >>
> >> You can also have a look at
> >> linux/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28*dts
> >> for actual devicetrees.
> >
> > For hats, capes, SoMs, carrier boards and others, we need to better
> > organize.
> > The EEPROM based approach by RPI and Beagle shall give us hints on how
> > to deal with this at scale in an industrial manner.
> > We have EEPROM SPDs for DIMMs: it feels so natural to leverage the
> > same concept to bring enumeration to those hot/cold pluggable
> > elements.
> > It is my intent to sit down with RPI, Beagle and 96Boards to agree on
> > something around those lines.
> > In the meantime, manual composition is a nuisance with have to deal
> > with. But for sure, that should be entirely transparent to the OS.
>
> Well there are boards out there, where auto detection is not possible,
> or intentionally not wanted. And of course it should be transparent
> to the operation system. But you have a way to load a different
> devicetree but the _only_ one you got with your firmware (or let it
> be a set of devicetrees, doesn't matter).
>
Indeed. and this is not about solving the world hunger. In the vast
majority of cases it is possible and is becoming more and more desirable
because of the fragmentation of the software value chain in embedded (many
companies involved in solution building rather than one providing from DRAM
controller programming to Linux kernel). For those cases, let's agree on an
acceptable zero-touch way to solve the problem, EEPROM being only one
possible solution.

>
> -michael
>
> > Side note to read in a positive mindset even if it does not look
> > positive:
> > If we keep things like they are, what may end up happening is that
> > business leaders will direct their teams to use ACPI to avoid DT
> > related pains.
> > One would say ACPI does not support everything and it comes with
> > EDK2...
> > Well, let's just be patient and we will witness that business guys
> > have a different view of what is a solution to a problem.
>


-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Michael Walle]

>>>>> With SystemReady, DT from distros are ignored.
>>>> 
>>>> Err? Is this really true, I know about some incompatible changes
>>>> to the device tree which prevents you from using usb (or even a
>>>> kernel panic) with the imx8mm and I know that on the ls1028a
>>>> flexspi wont work if the devicetree doesn't match the kernel.
>>>> I.e. if you have a device tree from kernel 5.14 you cannot
>>>> use it on 5.10. If you have one from 5.10 you cannot use it on
>>>> a later kernel.
>>> 
>>> What you describe is the situation we want to avoid and that comes
>>> from years of tinkering.
>> 
>> how is that tinkering? That means once you have a device tree, it is
>> set in stone. which isn't reality, sorry.
>> 
>> Consider the ls1028a and the flexspi "bug". For the 5.10 kernel/dtb
>> there was a wrong clock connected to the flexspi device. There
>> wasn't
>> even a driver for the correct clock.
> 
> The clock could have been described even though there was no Linux
> driver.
> DT is there to describe HW, not the capacity of a particular OS or
> boot firmware to deal with it.

You're missing my point. It's not about what would be the perfect
scenario. But what actually happens in reality. And unfortunately,
it happens, so you have to deal with devicetree incompatibilies.
Just ignoring this and keeping just one devicetree in the firmware
is doomed to fail sooner or later.

[..]

>> Regarding the imx8mm usb error, apparently the node was renamed. If
>> you're
>> using older kernels with newer dtbs, the kernel oopes. If you're
>> using a
>> newer kernel with older dtbs, USB doesn't get probed. (Although I
>> was
>> told that there is a "fix" for this, that is, both node names are
>> tried.)
> 
> I keep seeing those issues about node renames or compatible string
> changes.
> That's the tinkering I am talking about.
> There is no in-kernel ABI but Linus bang heads if anyone touches
> userland-kernel ABI inappropriately.
> 
> As DT is mostly handled in-kernel, people treat DT as no-ABI.
> But it is part of firmware-kernel ABI.
> Until we properly organize this firmware-kernel ABI, the problem you
> describe and more will continue.

Sure, but until you reaches that point (I predict it wont happen soon
or at all) you'll have to deal with per kernel devicetrees. Just
saying, we are just providing one devicetree supplied by the firmware
just doesn't work with the current situation. So IMHO if SystemReady is
really "it just works", then you have to consider this. And so far,
it seems all SystemReady certified boards just throw the u-boot
devicetree at linux and hope for the best.

> Now the DT lifecycle before the firmware-kernel ABI also needs to be
> specified so that we properly handle hats, capes, SoMs, carrier
> boards...
> 
>>> 
>> 
> https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
>>> lists a number of certified boards and the list is going to grow
>>> significantly.
>> 
>> And the sl28 board will likely be there soon, too.
>> 
>>> On those boards, you will be able to boot any kernel.
>> 
>> Actually I don't think so, because you might hit the imx8mm bug,
>> too.
>> May
>> just be lucky by the devicetree/kernel combination.
>> 
>>> The image I have in mind with OS provided DT is:
>>> as a French driver, my DT says that the steering wheel is on the
>> left
>>> hand side of the car.
>>> Shall I whine about the cars in England that do not comply to my
>> DT?
>>> or accept to use the car provided DT?
>>> The situation is comfortable for tinkerers, but not sustainable.
>> It is
>>> a matter of organization and not technology to solve the problems
>> you
>>> describe.
>> 
>> Mh, I'm not sure I understand what you're trying to tell me. The
>> distribution also provides you the kernel, why shouldn't it provide
>> devicetree which exactly matches this kernel and was also tested
>> against.
> 
> Because the kernel has no clue which hats, capes has been added for
> instance.

And the same might be true for the firmware as pointed out before.

> The kernel provided DTB works fine when the firmware builder and OS
> builders are the same.

Ehh? It is just the other way around. The distro supplies the kernel
and thus it also have the corresponding devicetrees for this particular
kernel. The firmware can remain the same. Now Mark might disagree here,
because OpenBSD doesn't provide devicetrees (I really don't know).

I agree, that in a perfect world, there would be just one (or a set of)
stable devicetree(s) which can be used by any OS/Distribution/Kernel.
But this simply isn't the case.

> This traditional model is evolving and the team that deals with OS may
> not even be in the same company as the one providing the firmware.
> Ask the Fedora IoT architect what he thinks about the distro provided
> DTs ;-)

I don't even know who "the fedora iot architect" is, nor what her/his
arguments are.

> There is a need to deal with DT bugs. OS provided DT is a bad solution
> to a real problem.
> U-Boot patches look a possibility for those bugs.

And then you always have to update the firmware and duplicate the 
"code".
I.e. theres an incompatible change, the devicetree is update in linux
and you have to replicate just this as a fixup in u-boot. And you have
to detect when and if it should be applied.

-michael

Re: Driver model at UEFI runtime

[François Ozog]

Le ven. 1 oct. 2021 à 00:00, Michael Walle <michael@walle.cc> a écrit :

> >>>>> With SystemReady, DT from distros are ignored.
> >>>>
> >>>> Err? Is this really true, I know about some incompatible changes
> >>>> to the device tree which prevents you from using usb (or even a
> >>>> kernel panic) with the imx8mm and I know that on the ls1028a
> >>>> flexspi wont work if the devicetree doesn't match the kernel.
> >>>> I.e. if you have a device tree from kernel 5.14 you cannot
> >>>> use it on 5.10. If you have one from 5.10 you cannot use it on
> >>>> a later kernel.
> >>>
> >>> What you describe is the situation we want to avoid and that comes
> >>> from years of tinkering.
> >>
> >> how is that tinkering? That means once you have a device tree, it is
> >> set in stone. which isn't reality, sorry.
> >>
> >> Consider the ls1028a and the flexspi "bug". For the 5.10 kernel/dtb
> >> there was a wrong clock connected to the flexspi device. There
> >> wasn't
> >> even a driver for the correct clock.
> >
> > The clock could have been described even though there was no Linux
> > driver.
> > DT is there to describe HW, not the capacity of a particular OS or
> > boot firmware to deal with it.
>
> You're missing my point. It's not about what would be the perfect
> scenario. But what actually happens in reality. And unfortunately,
> it happens, so you have to deal with devicetree incompatibilies.
> Just ignoring this and keeping just one devicetree in the firmware
> is doomed to fail sooner or later.

We have an example of firmware provided hardware description that works
well (Ok: 99% of the time): ACPI.
We also are 100% sure that the current situation is messy, hairy, buggy but
familiar.

>
>
> [..]
>
> >> Regarding the imx8mm usb error, apparently the node was renamed. If
> >> you're
> >> using older kernels with newer dtbs, the kernel oopes. If you're
> >> using a
> >> newer kernel with older dtbs, USB doesn't get probed. (Although I
> >> was
> >> told that there is a "fix" for this, that is, both node names are
> >> tried.)
> >
> > I keep seeing those issues about node renames or compatible string
> > changes.
> > That's the tinkering I am talking about.
> > There is no in-kernel ABI but Linus bang heads if anyone touches
> > userland-kernel ABI inappropriately.
> >
> > As DT is mostly handled in-kernel, people treat DT as no-ABI.
> > But it is part of firmware-kernel ABI.
> > Until we properly organize this firmware-kernel ABI, the problem you
> > describe and more will continue.
>
> Sure, but until you reaches that point (I predict it wont happen soon
> or at all) you'll have to deal with per kernel devicetrees. Just
> saying, we are just providing one devicetree supplied by the firmware
> just doesn't work with the current situation. So IMHO if SystemReady is
> really "it just works", then you have to consider this. And so far,
> it seems all SystemReady certified boards just throw the u-boot
> devicetree at linux and hope for the best.

SystemReady is not meant to be all boards, starting now and mandatory. It
is a selected of boards for which everyone in the value chain is willing to
spend the energy to solve the problem as if we were living in a perfect
world.

>
>
> > Now the DT lifecycle before the firmware-kernel ABI also needs to be
> > specified so that we properly handle hats, capes, SoMs, carrier
> > boards...
> >
> >>>
> >>
> >
> https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
> >>> lists a number of certified boards and the list is going to grow
> >>> significantly.
> >>
> >> And the sl28 board will likely be there soon, too.
> >>
> >>> On those boards, you will be able to boot any kernel.
> >>
> >> Actually I don't think so, because you might hit the imx8mm bug,
> >> too.
> >> May
> >> just be lucky by the devicetree/kernel combination.
> >>
> >>> The image I have in mind with OS provided DT is:
> >>> as a French driver, my DT says that the steering wheel is on the
> >> left
> >>> hand side of the car.
> >>> Shall I whine about the cars in England that do not comply to my
> >> DT?
> >>> or accept to use the car provided DT?
> >>> The situation is comfortable for tinkerers, but not sustainable.
> >> It is
> >>> a matter of organization and not technology to solve the problems
> >> you
> >>> describe.
> >>
> >> Mh, I'm not sure I understand what you're trying to tell me. The
> >> distribution also provides you the kernel, why shouldn't it provide
> >> devicetree which exactly matches this kernel and was also tested
> >> against.
> >
> > Because the kernel has no clue which hats, capes has been added for
> > instance.
>
> And the same might be true for the firmware as pointed out before.
>
> > The kernel provided DTB works fine when the firmware builder and OS
> > builders are the same.
>
> Ehh? It is just the other way around. The distro supplies the kernel
> and thus it also have the corresponding devicetrees for this particular
> kernel. The firmware can remain the same. Now Mark might disagree here,
> because OpenBSD doesn't provide devicetrees (I really don't know).
>
> I agree, that in a perfect world, there would be just one (or a set of)
> stable devicetree(s) which can be used by any OS/Distribution/Kernel.
> But this simply isn't the case.

Agreed: this is not the case today. But some vendors have decided that for
some boards it will work this way following EBBR for Arm and RISC-V.
Based on the current maturity of the DT, we are at a pivot moment when we
can do this for many boards without too much issues.


>
> > This traditional model is evolving and the team that deals with OS may
> > not even be in the same company as the one providing the firmware.
> > Ask the Fedora IoT architect what he thinks about the distro provided
> > DTs ;-)
>
> I don't even know who "the fedora iot architect" is, nor what her/his
> arguments are.
>
> > There is a need to deal with DT bugs. OS provided DT is a bad solution
> > to a real problem.
> > U-Boot patches look a possibility for those bugs.
>
> And then you always have to update the firmware and duplicate the
> "code".
> I.e. theres an incompatible change, the devicetree is update in linux
> and you have to replicate just this as a fixup in u-boot. And you have
> to detect when and if it should be applied.

DT should not change based on driver change. DT has been used as a driver
parameter facility and that created issues. DT is not meant for that. Linux
has driver load options and /etc/modules.d/*.conf for that as well as many
other facilities such as ethtool for example. If drivers stop using DT as a
parameter/config store you avoid most of the problems. And then you can
really share DT between OSes (*BSD…).

>
>
> -michael
>
-- 
François-Frédéric Ozog | *Director Business Development*
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Driver model at UEFI runtime

[Michael Walle]

Am 2021-10-01 00:20, schrieb François Ozog:
> Le ven. 1 oct. 2021 à 00:00, Michael Walle <michael@walle.cc> a
> écrit :
> 
>>>>>>> With SystemReady, DT from distros are ignored.
>>>>>> 
>>>>>> Err? Is this really true, I know about some incompatible
>> changes
>>>>>> to the device tree which prevents you from using usb (or even a
>>>>>> kernel panic) with the imx8mm and I know that on the ls1028a
>>>>>> flexspi wont work if the devicetree doesn't match the kernel.
>>>>>> I.e. if you have a device tree from kernel 5.14 you cannot
>>>>>> use it on 5.10. If you have one from 5.10 you cannot use it on
>>>>>> a later kernel.
>>>>> 
>>>>> What you describe is the situation we want to avoid and that
>> comes
>>>>> from years of tinkering.
>>>> 
>>>> how is that tinkering? That means once you have a device tree, it
>> is
>>>> set in stone. which isn't reality, sorry.
>>>> 
>>>> Consider the ls1028a and the flexspi "bug". For the 5.10
>> kernel/dtb
>>>> there was a wrong clock connected to the flexspi device. There
>>>> wasn't
>>>> even a driver for the correct clock.
>>> 
>>> The clock could have been described even though there was no Linux
>>> driver.
>>> DT is there to describe HW, not the capacity of a particular OS or
>>> boot firmware to deal with it.
>> 
>> You're missing my point. It's not about what would be the perfect
>> scenario. But what actually happens in reality. And unfortunately,
>> it happens, so you have to deal with devicetree incompatibilies.
>> Just ignoring this and keeping just one devicetree in the firmware
>> is doomed to fail sooner or later.
> 
> We have an example of firmware provided hardware description that
> works well (Ok: 99% of the time): ACPI.

Mh, I can't really comment on this as I am not familiar with ACPI.
But judging by all the linux acpi fixups and bios incompatiblies,
my gut tells me that it doesn't work _that_ well ;)

> We also are 100% sure that the current situation is messy, hairy,
> buggy but familiar.
> 
>> [..]
>> 
>>>> Regarding the imx8mm usb error, apparently the node was renamed.
>> If
>>>> you're
>>>> using older kernels with newer dtbs, the kernel oopes. If you're
>>>> using a
>>>> newer kernel with older dtbs, USB doesn't get probed. (Although I
>>>> was
>>>> told that there is a "fix" for this, that is, both node names are
>>>> tried.)
>>> 
>>> I keep seeing those issues about node renames or compatible string
>>> changes.
>>> That's the tinkering I am talking about.
>>> There is no in-kernel ABI but Linus bang heads if anyone touches
>>> userland-kernel ABI inappropriately.
>>> 
>>> As DT is mostly handled in-kernel, people treat DT as no-ABI.
>>> But it is part of firmware-kernel ABI.
>>> Until we properly organize this firmware-kernel ABI, the problem
>> you
>>> describe and more will continue.
>> 
>> Sure, but until you reaches that point (I predict it wont happen
>> soon
>> or at all) you'll have to deal with per kernel devicetrees. Just
>> saying, we are just providing one devicetree supplied by the
>> firmware
>> just doesn't work with the current situation. So IMHO if SystemReady
>> is
>> really "it just works", then you have to consider this. And so far,
>> it seems all SystemReady certified boards just throw the u-boot
>> devicetree at linux and hope for the best.
> 
> SystemReady is not meant to be all boards, starting now and mandatory.
> It is a selected of boards for which everyone in the value chain is
> willing to spend the energy to solve the problem as if we were living
> in a perfect world.

And here I am, trying to find a solution to a real problem I'm facing
with my board and the systemready cerification. But all I'm hearing is
that it should work the way linaro/ARM have in mind, but it clearly
doesn't.

>>> Now the DT lifecycle before the firmware-kernel ABI also needs to
>> be
>>> specified so that we properly handle hats, capes, SoMs, carrier
>>> boards...
>>> 
>>>>> 
>>>> 
>>> 
>> 
> https://developer.arm.com/architectures/system-architectures/arm-systemready/ir
>>>>> lists a number of certified boards and the list is going to grow
>>>>> significantly.
>>>> 
>>>> And the sl28 board will likely be there soon, too.
>>>> 
>>>>> On those boards, you will be able to boot any kernel.
>>>> 
>>>> Actually I don't think so, because you might hit the imx8mm bug,
>>>> too.
>>>> May
>>>> just be lucky by the devicetree/kernel combination.
>>>> 
>>>>> The image I have in mind with OS provided DT is:
>>>>> as a French driver, my DT says that the steering wheel is on the
>>>> left
>>>>> hand side of the car.
>>>>> Shall I whine about the cars in England that do not comply to my
>>>> DT?
>>>>> or accept to use the car provided DT?
>>>>> The situation is comfortable for tinkerers, but not sustainable.
>>>> It is
>>>>> a matter of organization and not technology to solve the
>> problems
>>>> you
>>>>> describe.
>>>> 
>>>> Mh, I'm not sure I understand what you're trying to tell me. The
>>>> distribution also provides you the kernel, why shouldn't it
>> provide
>>>> devicetree which exactly matches this kernel and was also tested
>>>> against.
>>> 
>>> Because the kernel has no clue which hats, capes has been added
>> for
>>> instance.
>> 
>> And the same might be true for the firmware as pointed out before.
>> 
>>> The kernel provided DTB works fine when the firmware builder and
>> OS
>>> builders are the same.
>> 
>> Ehh? It is just the other way around. The distro supplies the kernel
>> and thus it also have the corresponding devicetrees for this
>> particular
>> kernel. The firmware can remain the same. Now Mark might disagree
>> here,
>> because OpenBSD doesn't provide devicetrees (I really don't know).
>> 
>> I agree, that in a perfect world, there would be just one (or a set
>> of)
>> stable devicetree(s) which can be used by any
>> OS/Distribution/Kernel.
>> But this simply isn't the case.
> 
> Agreed: this is not the case today. But some vendors have decided that
> for some boards it will work this way following EBBR for Arm and
> RISC-V.
> Based on the current maturity of the DT, we are at a pivot moment when
> we can do this for many boards without too much issues.
> 
>>> This traditional model is evolving and the team that deals with OS
>> may
>>> not even be in the same company as the one providing the firmware.
>>> Ask the Fedora IoT architect what he thinks about the distro
>> provided
>>> DTs ;-)
>> 
>> I don't even know who "the fedora iot architect" is, nor what
>> her/his
>> arguments are.
>> 
>>> There is a need to deal with DT bugs. OS provided DT is a bad
>> solution
>>> to a real problem.
>>> U-Boot patches look a possibility for those bugs.
>> 
>> And then you always have to update the firmware and duplicate the
>> "code".
>> I.e. theres an incompatible change, the devicetree is update in
>> linux
>> and you have to replicate just this as a fixup in u-boot. And you
>> have
>> to detect when and if it should be applied.
> 
> DT should not change based on driver change. DT has been used as a
> driver parameter facility and that created issues. DT is not meant for
> that. Linux has driver load options and /etc/modules.d/*.conf for that
> as well as many other facilities such as ethtool for example. If
> drivers stop using DT as a parameter/config store you avoid most of
> the problems. And then you can really share DT between OSes (*BSD…).

Mh, it seems we are just repeating our own arguments and this doesn't
really lead to anything more of value.

-michael