On Thu, Aug 26, 2021 at 02:01:07PM +0200, Mark Kettenis wrote: > > Date: Wed, 25 Aug 2021 18:06:05 -0400 > > From: Tom Rini > > > > On Wed, Aug 25, 2021 at 11:54:58PM +0200, Mark Kettenis wrote: > > > > Date: Wed, 25 Aug 2021 10:56:35 -0400 > > > > From: Tom Rini > > > > > > > > On Wed, Aug 25, 2021 at 11:42:51PM +0900, AKASHI Takahiro wrote: > > > > > Simon, > > > > > > > > > > On Wed, Aug 25, 2021 at 07:11:44AM -0600, Simon Glass wrote: > > > > > > Hi, > > > > > > > > > > > > On Wed, 25 Aug 2021 at 04:45, Emmanuel Vadot wrote: > > > > > > > > > > > > > > On Tue, 24 Aug 2021 12:22:42 +0200 (CEST) > > > > > > > Mark Kettenis wrote: > > > > > > > > > > > > > > > > Date: Mon, 23 Aug 2021 16:01:46 -0400 > > > > > > > > > From: Tom Rini > > > > > > > > > > > > > > > > > > On Mon, Aug 23, 2021 at 11:25:42AM -0600, Simon Glass wrote: > > > > > > > > > > Hi Mark, > > > > > > > > > > > > > > > > > > > > On Mon, 23 Aug 2021 at 05:54, Mark Kettenis wrote: > > > > > > > > > > > > > > > > > > > > > > > From: Simon Glass > > > > > > > > > > > > Date: Wed, 18 Aug 2021 21:45:33 -0600 > > > > > > > > > > > > > > > > > > > > > > > > Bootmethod and bootflow provide a built-in way for U-Boot to automatically boot > > > > > > > > > > > > an Operating System without custom scripting and other customisation: > > > > > > > > > > > > > > > > > > > > > > > > - bootmethod - a method to scan a device to find bootflows (owned by U-Boot) > > > > > > > > > > > > - bootflow - a description of how to boot (owned by the distro) > > > > > > > > > > > > > > > > > > > > > > > > This series provides an initial implementation of these, enable to scan > > > > > > > > > > > > for bootflows from MMC and Ethernet. The only bootflow supported is > > > > > > > > > > > > distro boot, i.e. an extlinux.conf file included on a filesystem or > > > > > > > > > > > > tftp server. It works similiarly to the existing script-based approach, > > > > > > > > > > > > but is native to U-Boot. > > > > > > > > > > > > > > > > > > > > > > > > With this we can boot on a Raspberry Pi 3 with just one command: > > > > > > > > > > > > > > > > > > > > > > > > bootflow scan -lb > > > > > > > > > > > > > > > > > > > > > > > > which means to scan, listing (-l) each bootflow and trying to boot each > > > > > > > > > > > > one (-b). The final patch shows this. > > > > > > > > > > > > > > > > > > > > > > > > It is intended that this approach be expanded to support mechanisms other > > > > > > > > > > > > than distro boot, including EFI-related ones. With a standard way to > > > > > > > > > > > > identify boot devices, these features become easier. It also should > > > > > > > > > > > > support U-Boot scripts, for backwards compatibility only. > > > > > > > > > > > > > > > > > > > > > > > > The first patch of this series moves boot-related code out of common/ and > > > > > > > > > > > > into a new boot/ directory. This helps to collect these related files > > > > > > > > > > > > in one place, as common/ is quite large. > > > > > > > > > > > > > > > > > > > > > > > > Like sysboot, this feature makes use of the existing PXE implementation. > > > > > > > > > > > > Much of this series consists of cleaning up that code and refactoring it > > > > > > > > > > > > into something closer to a module that can be called, teasing apart its > > > > > > > > > > > > reliance on the command-line interpreter to access filesystems and the > > > > > > > > > > > > like. Also it now uses function arguments and its own context struct > > > > > > > > > > > > internally rather than environment variables, which is very hard to > > > > > > > > > > > > follow. No core functional change is included in the included PXE patches. > > > > > > > > > > > > > > > > > > > > > > > > For documentation, see the 'doc' patch. > > > > > > > > > > > > > > > > > > > > > > > > There is quite a long list of future work included in the documentation. > > > > > > > > > > > > One question is the choice of naming. Since this is a bootloader, should > > > > > > > > > > > > we just call this a 'method' and a 'flow' ? The 'boot' prefix is already > > > > > > > > > > > > shared by other commands like bootm, booti, etc. > > > > > > > > > > > > > > > > > > > > > > > > The design is described here: > > > > > > > > > > > > > > > > > > > > > > > > https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing > > > > > > > > > > > > > > > > > > > > > > > > The series is available at u-boot-dm/bmea-working > > > > > > > > > > > > > > > > > > > > > > How does the user control the order in which devices are scanned/booted? > > > > > > > > > > > > > > > > > > > > That is not supported in distroboot at present, at least so far as I > > > > > > > > > > can see. For Fedora it seems to happen in grub. Do I have that right? > > > > > > > > > > > > > > > > > > Well, there's "find the next stage", which is boot_targets environment > > > > > > > > > variable, and then "where that next stage looks for stuff" which is > > > > > > > > > OS-dependent. Sometimes the ESP grub.cfg file is just enough to tell > > > > > > > > > grub to find the full grub.cfg file elsewhere, and sometimes it's a full > > > > > > > > > grub.cfg file. I think Mark is talking about the former, and you've > > > > > > > > > said it's not part of this series, yet, but on the TODO list. > > > > > > > > > > > > > > > > Right. With the current distroboot code the order of the devices that > > > > > > > > appears in boot_targets is determined by per-board/SOC/machine config > > > > > > > > files and the order isn't the same for all of them. Users can change > > > > > > > > the order if necessary by modifying the environment variable and > > > > > > > > saving the environment. And for a one-off boot from a different > > > > > > > > device they can simply run an appropriate boot command. The > > > > > > > > boot_targets variable in particular is documented in various install > > > > > > > > documents so it would probably be good of the new "bootmethod" code > > > > > > > > would respect this variable. > > > > > > > > > > > > > > > > For OpenBSD I'm not really interested in the bootflow part. As I > > > > > > > > explained in the past, that part of the problem is solved in a > > > > > > > > (mostly) uniform way across platforms by the OpenBSD bootloader which > > > > > > > > can read an /etc/boot.conf that allows bootflow customization. So as > > > > > > > > long as the default of the new code still results in > > > > > > > > \EFI\BOOT\BOOT{machine type short-name}.EFI being loaded and run if > > > > > > > > there is no U-Boot specific bootflow configured, I'm happy. > > > > > > > > > > > > > > Mostly the same for FreeBSD, as long as the efi boot.efi is > > > > > > > loaded and run by default (respecting the boot_targets order) we will > > > > > > > be fine. > > > > > > > > > > > > OK thanks for the info. My expectation is that bootmethod/bootflow can > > > > > > support this easily enough (it is actually simpler than distro boot). > > > > > > > > > > > > > > > > > > > > > I can't speak for the other BSDs, but my impression is that they are > > > > > > > > pretty much in the same position. The FreeBSD bootloader for example > > > > > > > > supports a high-degree of "bootflow" customization and I doubt that > > > > > > > > taking it out of the loop is a viable option for most users. > > > > > > > > > > > > I think the same may happen with grub. E.g. with Ubuntu I see quite a > > > > > > bit of code in the grub.cfg file and it's not clear to me that it can > > > > > > be replaced with a 'data instead of code' approach. Still, a valid > > > > > > bootflow is simply to jump to an EFI app, which seems to be what is > > > > > > happening here. The bootflow side is really just about describing what > > > > > > to do, and this case is no different. For now I see three types of > > > > > > bootflow, PXE/syslinux, EFI boot manager and EFI app. > > > > > > > > > > By "EFI app", do you mean a way of booting "/efi/boot/bootXX.efi" > > > > > (default file name in case that no image path is specified)? > > > > > > > > > > In fact, this behavior, or removable media support, is defined > > > > > as part of UEFI boot manager in UEFI specification. (See section 3.5) > > > > > What this means is that the boot order, including a removable media > > > > > case and user-provided BootXXXX cases, should be controlled solely > > > > > by "BootOrder" variable. > > > > > So the current combination of distro_bootcmd + UEFI boot manger doesn't > > > > > fully comply with the specification. > > > > > > > > > > Even if those two cases are integrated, I don't know how "BootOrder" > > > > > semantics can be preserved in your approach. > > > > > > > > I think the high level answer is that whereas today part of > > > > distro_bootcmd (and so iterating over boot_targets) "bootefi bootmgr" > > > > gets run, with what Simon is proposing we would have an easier / quicker > > > > way to get over to just running that. Perhaps a clean-up to just use > > > > that, even? Or are we not to the point yet where we could remove the > > > > direct fall-back to /efi/boot/bootXX.efi ? > > > > > > I think "bootefi bootmgr" only works if the BootOrder variable is > > > defined, and currently that isn't the case. > > > > > > The boot manager behaviour as specified in the UEFI specification is > > > somewhat problematic to implement in U-Boot because of the limitations > > > in how variables can be set at runtime. This is one of the reasons > > > OpenBSD doesn't actually bother with setting the variables and simple > > > relies on the "removable media" support mentioned above. All my > > > OpenBSD systems that use U-Boot print the follwing lines: > > > > > > BootOrder not defined > > > EFI boot manager: Cannot load any image > > > Found EFI removable media binary efi/boot/bootaa64.efi > > > > > > But maybe that last step can be intgrated into bootefi bootmgr at some > > > point? > > > > > > Also note that manually manipulating the EFI variables to change the > > > boot order is quite cumbersome; it isn't a matter of just changed the > > > aforementioned BootOrder variable. That's why I think boot_targets is > > > the preferable way to define the order in which devices should be > > > booted. I don't think that violates the UEFI specification. It > > > merely is the way U-Boot implements the boot device selection that > > > more traditional UEFI implementations implement using a menu. > > > > As I don't want to side-track Simon's thread even further, I would like > > to see a bit more detailed explanation of why U-Boot cannot support EFI > > variables, or if it's just a matter of someone doing the work, and it's > > not been a priority yet. > > U-Boot has some support for EFI variables, but there are some > fundamental problems that make "full" support for them hard or even > impossible. > > Some non-volatile storage is necessary for these variables such that > they can be persistent across boots. Obviously this very much applies > to the BootOrder variable. EFI defines calls to manipulate variables > as part of its runtime services. This means that the NV storage has > to implemented in a way that doesn't interfere with normal OS usage of > the hardware. That pretty much means that you need dedicated hardware > for this, which most devices supported by U-Boot simply don't have. > Having the EFI variables in the U-Boot environment on a reserved part > of a uSD card isn't going to work if the OS assumes it has full > control over the uSD controller. > > Recent versions of the UEFI have made the implementation of some of > the runtime services optional (more or less at the request of the EBBR > folks) and allow certain calls (e.g. the SetVariable() call) to fail. > This poses a bit of a problem though, which I'll try to sketch here: > > The way things typically work on a x86 EFI system is that you boot > your OS installer from removable media. The OS installer does its > thing (partitions the disk, creates filesystems, installs the OS > kernel, etc.) and at the end creates a boot option for the boot > manager by creating an apropriate Boot#### variable and possibly > modifying the BootOrder variable to include the newly created boot > option. A typical x86 Linux distro will create a Boot#### variable > that is effectively a devicepath pointing at grub.efi. Unfortunately > that won't work if the SetVariable() EFI runtime interface doesn't > work. > > I'm not sure how the EBBR folks envisaged the OS installation user > experience on these systems. Maybe Takahiro can explain. But as long > as you don't really care about booting multiple OSes on a system, > relying on the default removable media boot path works fine in most > cases in that it automatically boots into the installed OS when you > reboot after removing the installation media. Ah right, run-time variables are where it gets tricky. I would think that when ENV_IS_IN_MMC/etc, where it's a hard location on something, and not a file (which would be hard to share since it's likely mounted via the OS) would let us get past that. -- Tom