Re: [PATCH 00/28] Initial implementation of bootmethod/bootflow

[Ilias Apalodimas <ilias.apalodimas@linaro.org>]

Hi Mark, 

> > > > > > > > > > >

[...]

> > > > > > > > > > > Well, there's "find the next stage", which is boot_targets environment
> > > > > > > > > > > variable, and then "where that next stage looks for stuff" which is
> > > > > > > > > > > OS-dependent.  Sometimes the ESP grub.cfg file is just enough to tell
> > > > > > > > > > > grub to find the full grub.cfg file elsewhere, and sometimes it's a full
> > > > > > > > > > > grub.cfg file.  I think Mark is talking about the former, and you've
> > > > > > > > > > > said it's not part of this series, yet, but on the TODO list.
> > > > > > > > > >
> > > > > > > > > > Right.  With the current distroboot code the order of the devices that
> > > > > > > > > > appears in boot_targets is determined by per-board/SOC/machine config
> > > > > > > > > > files and the order isn't the same for all of them.  Users can change
> > > > > > > > > > the order if necessary by modifying the environment variable and
> > > > > > > > > > saving the environment.  And for a one-off boot from a different
> > > > > > > > > > device they can simply run an appropriate boot command.  The
> > > > > > > > > > boot_targets variable in particular is documented in various install
> > > > > > > > > > documents so it would probably be good of the new "bootmethod" code
> > > > > > > > > > would respect this variable.
> > > > > > > > > >
> > > > > > > > > > For OpenBSD I'm not really interested in the bootflow part.  As I
> > > > > > > > > > explained in the past, that part of the problem is solved in a
> > > > > > > > > > (mostly) uniform way across platforms by the OpenBSD bootloader which
> > > > > > > > > > can read an /etc/boot.conf that allows bootflow customization.  So as
> > > > > > > > > > long as the default of the new code still results in
> > > > > > > > > > \EFI\BOOT\BOOT{machine type short-name}.EFI being loaded and run if
> > > > > > > > > > there is no U-Boot specific bootflow configured, I'm happy.
> > > > > > > > >
> > > > > > > > >  Mostly the same for FreeBSD, as long as the efi boot<arch>.efi is
> > > > > > > > > loaded and run by default (respecting the boot_targets order) we will
> > > > > > > > > be fine.
> > > > > > > > 
> > > > > > > > OK thanks for the info. My expectation is that bootmethod/bootflow can
> > > > > > > > support this easily enough (it is actually simpler than distro boot).
> > > > > > > > 
> > > > > > > > >
> > > > > > > > > > I can't speak for the other BSDs, but my impression is that they are
> > > > > > > > > > pretty much in the same position.  The FreeBSD bootloader for example
> > > > > > > > > > supports a high-degree of "bootflow" customization and I doubt that
> > > > > > > > > > taking it out of the loop is a viable option for most users.
> > > > > > > > 
> > > > > > > > I think the same may happen with grub. E.g. with Ubuntu I see quite a
> > > > > > > > bit of code in the grub.cfg file and it's not clear to me that it can
> > > > > > > > be replaced with a 'data instead of code' approach. Still, a valid
> > > > > > > > bootflow is simply to jump to an EFI app, which seems to be what is
> > > > > > > > happening here. The bootflow side is really just about describing what
> > > > > > > > to do, and this case is no different. For now I see three types of
> > > > > > > > bootflow, PXE/syslinux, EFI boot manager and EFI app.
> > > > > > > 
> > > > > > > By "EFI app", do you mean a way of booting "/efi/boot/bootXX.efi"
> > > > > > > (default file name in case that no image path is specified)?
> > > > > > > 
> > > > > > > In fact, this behavior, or removable media support, is defined
> > > > > > > as part of UEFI boot manager in UEFI specification. (See section 3.5)
> > > > > > > What this means is that the boot order, including a removable media
> > > > > > > case and user-provided BootXXXX cases, should be controlled solely
> > > > > > > by "BootOrder" variable.
> > > > > > > So the current combination of distro_bootcmd + UEFI boot manger doesn't
> > > > > > > fully comply with the specification.
> > > > > > > 
> > > > > > > Even if those two cases are integrated, I don't know how "BootOrder"
> > > > > > > semantics can be preserved in your approach.
> > > > > > 
> > > > > > I think the high level answer is that whereas today part of
> > > > > > distro_bootcmd (and so iterating over boot_targets) "bootefi bootmgr"
> > > > > > gets run, with what Simon is proposing we would have an easier / quicker
> > > > > > way to get over to just running that.  Perhaps a clean-up to just use
> > > > > > that, even?  Or are we not to the point yet where we could remove the
> > > > > > direct fall-back to /efi/boot/bootXX.efi ?
> > > > > 
> > > > > I think "bootefi bootmgr" only works if the BootOrder variable is
> > > > > defined, and currently that isn't the case.
> > > > > 
> > > > > The boot manager behaviour as specified in the UEFI specification is
> > > > > somewhat problematic to implement in U-Boot because of the limitations
> > > > > in how variables can be set at runtime.  This is one of the reasons
> > > > > OpenBSD doesn't actually bother with setting the variables and simple
> > > > > relies on the "removable media" support mentioned above.  All my
> > > > > OpenBSD systems that use U-Boot print the follwing lines:
> > > > > 
> > > > >   BootOrder not defined
> > > > >   EFI boot manager: Cannot load any image
> > > > >   Found EFI removable media binary efi/boot/bootaa64.efi
> > > > > 
> > > > > But maybe that last step can be intgrated into bootefi bootmgr at some
> > > > > point?
> > > > > 
> > > > > Also note that manually manipulating the EFI variables to change the
> > > > > boot order is quite cumbersome; it isn't a matter of just changed the
> > > > > aforementioned BootOrder variable.  That's why I think boot_targets is
> > > > > the preferable way to define the order in which devices should be
> > > > > booted.  I don't think that violates the UEFI specification.  It
> > > > > merely is the way U-Boot implements the boot device selection that
> > > > > more traditional UEFI implementations implement using a menu.
> > > > 
> > > > As I don't want to side-track Simon's thread even further, I would like
> > > > to see a bit more detailed explanation of why U-Boot cannot support EFI
> > > > variables, or if it's just a matter of someone doing the work, and it's
> > > > not been a priority yet.
> > > 
> > > U-Boot has some support for EFI variables, but there are some
> > > fundamental problems that make "full" support for them hard or even
> > > impossible.
> > > 
> > > Some non-volatile storage is necessary for these variables such that
> > > they can be persistent across boots.  Obviously this very much applies
> > > to the BootOrder variable.  EFI defines calls to manipulate variables
> > > as part of its runtime services.  This means that the NV storage has
> > > to implemented in a way that doesn't interfere with normal OS usage of
> > > the hardware.  That pretty much means that you need dedicated hardware
> > > for this, which most devices supported by U-Boot simply don't have.
> > > Having the EFI variables in the U-Boot environment on a reserved part
> > > of a uSD card isn't going to work if the OS assumes it has full
> > > control over the uSD controller.
> > > 
> > > Recent versions of the UEFI have made the implementation of some of
> > > the runtime services optional (more or less at the request of the EBBR
> > > folks) and allow certain calls (e.g. the SetVariable() call) to fail.
> > > This poses a bit of a problem though, which I'll try to sketch here:
> > > 
> > > The way things typically work on a x86 EFI system is that you boot
> > > your OS installer from removable media.  The OS installer does its
> > > thing (partitions the disk, creates filesystems, installs the OS
> > > kernel, etc.) and at the end creates a boot option for the boot
> > > manager by creating an apropriate Boot#### variable and possibly
> > > modifying the BootOrder variable to include the newly created boot
> > > option.  A typical x86 Linux distro will create a Boot#### variable
> > > that is effectively a devicepath pointing at grub.efi.  Unfortunately
> > > that won't work if the SetVariable() EFI runtime interface doesn't
> > > work.
> > > 
> > > I'm not sure how the EBBR folks envisaged the OS installation user
> > > experience on these systems.  Maybe Takahiro can explain.  But as long
> > > as you don't really care about booting multiple OSes on a system,
> > > relying on the default removable media boot path works fine in most
> > > cases in that it automatically boots into the installed OS when you
> > > reboot after removing the installation media.
> > 
> > Ah right, run-time variables are where it gets tricky.  I would think
> > that when ENV_IS_IN_MMC/etc, where it's a hard location on something,
> > and not a file (which would be hard to share since it's likely mounted
> > via the OS) would let us get past that.
> 
> Not really.  As soon as the OS takes over the MMC controller, U-Boot
> can't touch it anymore.  The only thing that really works is using a
> dedicated device that isn't exposed to the OS.  So U-Boot would need
> to remove its nodes from the device tree.  But even then things get
> tricky with shared clocks and such.
> 
> The only really safe option is to use something like TrustZone on ARM
> devices to completely hide things from the non-secure world. 

That's how it also works on servers. They all have dedicated hardware

> I believe that is what Linaro is actually doing in their U-Boot based
> implementation where the EFI variable store is actually implemented in
> OP-TEE running in the secure world.  However that places demands on
> the hardware that many SoCs and boards won't be able to fulfill.
> 

There's two general categories here
- SPI/whatever flash on the secure world. That pretty much solves
  everything, but as you say, imposes hardware restrictions and the runtime
  part is missing from U-Boot.
- store the variables on an RPMB partition.  This currently works on
  u-boot. It doesn't directly solve the problem with runtime variables, but
  you can get away with it if the OS agrees (which is against what EFI is
  trying to do :)).  So the RPMB variable storage goes through OP-TEE, I
  have some out of tree patches for Linux, were the bootloader is
  installing an empty config table. Upon discovery the OS changes all the
  runtime callbacks to OP-TEE provided functions, so you can read/write
  variables properly.  I understand that this isn't the best solution, but
  honestly from the discussions and thoughts I've had up to now, it seems
  the sanest way to solve the setvariable problem, if the OS is in control
  of the medium after ExitBootServices. We are in discussions internally to 
  see if there's a better way to do it, and I am open to suggestions. In
  any case the OS has to do 'something' about it, since it controls the
  device.

> A lot of the more advance EFI features that have been
> implemented/proposed (secure boot, capsule updates) do rely on having
> proper run-time variable support. 

0fa5020c024e 'fixes' capsule update on-disk (but kinda breaks the spec)

> So Simon does have a point that the
> EFI based approach for that has some serious issues that a "native"
> U-Boot implementation could avoid and that enabling these EFI features
> by default adds unecessary "bloat".

Regards
/Ilias