Re: [PATCH v2 00/41] Initial implementation of standard boot

[Simon Glass <sjg@chromium.org>]

+u-boot-board-maintainers@lists.denx.de
+u-boot-custodians@lists.denx.de

On Sat, 23 Oct 2021 at 17:26, Simon Glass <sjg@chromium.org> wrote:
>
>
> The bootflow feature provide a built-in way for U-Boot to automatically
> boot an Operating System without custom scripting and other customisation.
> This is called 'standard boot' since it provides a standard way for
> U-Boot to boot a distro, without scripting.
>
> It introduces the following concepts:
>
>    - bootdev - a device which can hold a distro
>    - bootmeth - a method to scan a bootdev to find bootflows (owned by
>                 U-Boot)
>    - bootflow - a description of how to boot (owned by the distro)
>
> This series provides an implementation of these, enabled to scan for
> bootflows from MMC, USB and Ethernet. It supports the existing distro
> boot as well as the EFI loader flow (bootefi/bootmgr). It works
> similiarly to the existing script-based approach, but is native to
> U-Boot.
>
> With this we can boot on a Raspberry Pi 3 with just one command:
>
>    bootflow scan -lb
>
> which means to scan, listing (-l) each bootflow and trying to boot each
> one (-b). The final patch shows this.
>
> With a standard way to identify boot devices, booting become easier. It
> also should be possible to support U-Boot scripts, for backwards
> compatibility only.
>
> This series relies on the PXE clean-up series, posted here:
>
>    https://patchwork.ozlabs.org/project/uboot/list/?series=267078
>
> For documentation, see the 'doc' patch.
>
> For version 2, a new naming scheme is used as above:
>
>    - bootdev is used instead of bootdevice, because 'device' is overused,
>        is everywhere in U-Boot, can be confused with udevice
>    - bootmeth - because 'method' is too vanilla, appears 1300 times in
>        U-Boot
>
> Also in version 2, drivers are introduced for the boot methods, to make
> it more extensible. Booting a custom OS is simply a matter of creating a
> bootmeth for it and implementing the read_file() and boot() methods.
>
> The design is described in these two documents:
>
> https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing
>
> https://drive.google.com/file/d/1kTrflO9vvGlKp-ZH_jlgb9TY3WYG6FF9/view?usp=sharing
>
> The series is available at u-boot-dm/bme-working
>
> Sample log on rpi_3_32b:
>
> U-Boot 2021.10-rc2-00043-gccd453aa918-dirty (Aug 28 2021 - 13:58:46 -0600)
>
> DRAM:  992 MiB
> RPI 3 Model B (0xa22082)
> MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
> Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1... In:    serial
> Out:   vidconsole
> Err:   vidconsole
> Net:   No ethernet found.
> starting USB...
> Bus usb@7e980000: USB DWC2
> scanning bus usb@7e980000 for devices... usb_kbd usb_kbd: Timeout poll on interrupt endpoint
> Failed to get keyboard state from device 0c40:8000
> 4 USB Device(s) found
>        scanning usb for storage devices... 0 Storage Device(s) found
> Hit any key to stop autoboot:  0
> Scanning for bootflows in all bootmethods
> Seq  Type         State   Uclass    Part  Name                      Filename
> ---  -----------  ------  --------  ----  ------------------------  ----------------
> Scanning bootmethod 'mmc@7e202000.bootmethod':
>   0  efi-loader   loaded  mmc          1  mmc@7e202000.bootmethod.p efi/boot/bootarm.efi
> ** Booting bootflow 'mmc@7e202000.bootmethod.part_1'
> Scanning disk mmc@7e202000.blk...
> ** Unrecognized filesystem type **
> Card did not respond to voltage select! : -110
> Scanning disk sdhci@7e300000.blk...
> Disk sdhci@7e300000.blk not ready
> Found 4 disks
> No EFI system partition
> Booting /efi\boot
> Waiting for Ethernet connection... done.
>
>       Fedora (5.11.12-300.fc34.armv7hl) 34 (Workstation Edition)
>       UEFI Firmware Settings
>
>       Use the ▲ and ▼ keys to change the selection.
>       Press 'e' to edit the selected item, or 'c' for a command prompt. Press Escape to return to the previous menu.
>    The selected entry will be started automatically in 0s.
>
> Changes in v2:
> - Free the memory buffer, to avoid running out of memory
> - Unmap the file
> - Reorder struct sandbox_mmc_priv
>
> Simon Glass (41):
>   lib: Add a way to find the postiion of a trailing number
>   Makefile: Allow LTO to be disabled for a build
>   test/py: Allow passing input to a program
>   sandbox: Support unmapping a file
>   sandbox: mmc: Support a backing file
>   mmc: Allow for children other than the block device
>   mbr: Correct verification check
>   disk: part_dos: Fix a NULL pointer error
>   common: Allow a smaller console-recording pre-reloc
>   dm: core: Add tests for stringlist functions
>   dm: core: Fix handling of uclass pre_unbind method
>   dm: core: Fix up string-function documentation
>   dm: core: Add a way to obtain a string list
>   dm: core: Allow finding children / uclasses by partial name
>   dm: core: Add a way to count the devices in a uclass
>   sandbox: Enable HEXDUMP for sandbox_flattree
>   test/py: Relax the naming rules for unit tests
>   test/py: Raise a ValueError if a command fails
>   bootstd: Add the concept of a bootflow
>   bootstd: Add the bootstd uclass and core implementation
>   bootstd: Add the bootdev uclass
>   bootstd: Add the bootmeth uclass and helpers
>   bootstd: Add support for bootflows
>   bootstd: Add a bootdev command
>   bootstd: Add a bootflow ommand
>   bootstd: Add a bootmeth command
>   bootstd: Add an implementation of distro boot
>   bootstd: mmc: Add a bootdev driver
>   bootstd: ethernet: Add a bootdev driver
>   bootstd: Add an implementation of distro PXE boot
>   bootstd: Add an implementation of EFI boot
>   bootstd: Add a system bootdev for strange boot methods
>   bootstd: Add an implementation of EFI bootmgr
>   bootstd: Add a sandbox bootmeth driver
>   bootstd: usb: Add a bootdev driver
>   test: fastboot: Avoid using mmc1
>   test: dm: Restart USB before assuming it is stopped
>   bootstd: Add tests for bootstd including all uclasses
>   bootstd: Add setup for the bootflow tests
>   bootstd: doc: Add documentation
>   RFC: Switch rpi over to use bootstd
>
>  MAINTAINERS                                  |  21 +
>  Makefile                                     |  18 +-
>  arch/arm/config.mk                           |   4 +-
>  arch/arm/include/asm/global_data.h           |   2 +-
>  arch/sandbox/cpu/os.c                        |  10 +
>  arch/sandbox/dts/test.dts                    |  19 +
>  boot/Kconfig                                 |  77 ++-
>  boot/Makefile                                |  14 +
>  boot/bootdev-uclass.c                        | 439 ++++++++++++++
>  boot/bootflow.c                              | 576 ++++++++++++++++++
>  boot/bootmeth-uclass.c                       |  59 ++
>  boot/bootmeth_distro.c                       | 217 +++++++
>  boot/bootmeth_efi.c                          | 263 ++++++++
>  boot/bootmeth_efi_mgr.c                      |  84 +++
>  boot/bootmeth_pxe.c                          | 183 ++++++
>  boot/bootmeth_sandbox.c                      |  67 +++
>  boot/bootstd-uclass.c                        | 176 ++++++
>  boot/system_bootdev.c                        |  64 ++
>  cmd/Kconfig                                  |  15 +
>  cmd/Makefile                                 |   1 +
>  cmd/bootdev.c                                | 120 ++++
>  cmd/bootflow.c                               | 386 ++++++++++++
>  cmd/bootmeth.c                               | 158 +++++
>  cmd/mbr.c                                    |   8 +-
>  common/Kconfig                               |  10 +
>  common/console.c                             |   4 +-
>  common/usb_storage.c                         |  12 +
>  configs/amcore_defconfig                     |   1 +
>  configs/rcar3_salvator-x_defconfig           |   1 +
>  configs/sandbox_defconfig                    |   3 +-
>  configs/sandbox_flattree_defconfig           |   4 +-
>  configs/tbs2910_defconfig                    |   1 +
>  disk/part_dos.c                              |  10 +-
>  doc/build/gcc.rst                            |  17 +
>  doc/develop/bootstd.rst                      | 600 +++++++++++++++++++
>  doc/develop/distro.rst                       |   3 +
>  doc/develop/index.rst                        |   1 +
>  doc/device-tree-bindings/bootdev.txt         |  26 +
>  doc/device-tree-bindings/bootmeth.txt        |  31 +
>  doc/device-tree-bindings/bootstd.txt         |  36 ++
>  doc/device-tree-bindings/mmc/sandbox,mmc.txt |  18 +
>  doc/usage/bootdev.rst                        | 135 +++++
>  doc/usage/bootflow.rst                       | 427 +++++++++++++
>  doc/usage/bootmeth.rst                       | 106 ++++
>  doc/usage/index.rst                          |   3 +
>  drivers/core/device-remove.c                 |   9 +-
>  drivers/core/device.c                        |  13 +-
>  drivers/core/of_access.c                     |   3 +-
>  drivers/core/ofnode.c                        |  26 +
>  drivers/core/read.c                          |   6 +
>  drivers/core/uclass.c                        |  29 +-
>  drivers/mmc/Makefile                         |   5 +
>  drivers/mmc/mmc-uclass.c                     |  27 +-
>  drivers/mmc/mmc_bootdev.c                    |  62 ++
>  drivers/mmc/sandbox_mmc.c                    |  60 +-
>  drivers/usb/host/Makefile                    |   4 +
>  drivers/usb/host/usb_bootdev.c               |  61 ++
>  include/bootdev.h                            | 258 ++++++++
>  include/bootflow.h                           | 304 ++++++++++
>  include/bootmeth.h                           | 156 +++++
>  include/bootstd.h                            |  80 +++
>  include/configs/rpi.h                        |  41 +-
>  include/distro.h                             |  24 +
>  include/dm/device.h                          |  12 +
>  include/dm/ofnode.h                          |  24 +-
>  include/dm/read.h                            |  28 +
>  include/dm/uclass-id.h                       |   3 +
>  include/dm/uclass-internal.h                 |  14 +-
>  include/dm/uclass.h                          |  17 +
>  include/mmc.h                                |  12 +-
>  include/os.h                                 |   9 +
>  include/test/suites.h                        |   2 +
>  include/vsprintf.h                           |  18 +
>  lib/strto.c                                  |  14 +-
>  net/Kconfig                                  |   9 +
>  net/Makefile                                 |   1 +
>  net/eth-uclass.c                             |   8 +
>  net/eth_bootdev.c                            | 101 ++++
>  test/Makefile                                |   1 +
>  test/boot/Makefile                           |   5 +
>  test/boot/bootdev.c                          | 200 +++++++
>  test/boot/bootflow.c                         | 398 ++++++++++++
>  test/boot/bootmeth.c                         |  90 +++
>  test/boot/bootstd_common.c                   |  35 ++
>  test/boot/bootstd_common.h                   |  27 +
>  test/cmd_ut.c                                |   7 +
>  test/dm/blk.c                                |   6 +
>  test/dm/fastboot.c                           |   4 +-
>  test/dm/ofnode.c                             |  96 +++
>  test/py/conftest.py                          |   2 +-
>  test/py/multiplexed_log.py                   |  10 +-
>  test/py/tests/bootstd/mmc1.img.xz            | Bin 0 -> 4448 bytes
>  test/py/tests/test_ut.py                     | 103 ++++
>  test/py/u_boot_utils.py                      |   5 +-
>  94 files changed, 6765 insertions(+), 94 deletions(-)
>  create mode 100644 boot/bootdev-uclass.c
>  create mode 100644 boot/bootflow.c
>  create mode 100644 boot/bootmeth-uclass.c
>  create mode 100644 boot/bootmeth_distro.c
>  create mode 100644 boot/bootmeth_efi.c
>  create mode 100644 boot/bootmeth_efi_mgr.c
>  create mode 100644 boot/bootmeth_pxe.c
>  create mode 100644 boot/bootmeth_sandbox.c
>  create mode 100644 boot/bootstd-uclass.c
>  create mode 100644 boot/system_bootdev.c
>  create mode 100644 cmd/bootdev.c
>  create mode 100644 cmd/bootflow.c
>  create mode 100644 cmd/bootmeth.c
>  create mode 100644 doc/develop/bootstd.rst
>  create mode 100644 doc/device-tree-bindings/bootdev.txt
>  create mode 100644 doc/device-tree-bindings/bootmeth.txt
>  create mode 100644 doc/device-tree-bindings/bootstd.txt
>  create mode 100644 doc/device-tree-bindings/mmc/sandbox,mmc.txt
>  create mode 100644 doc/usage/bootdev.rst
>  create mode 100644 doc/usage/bootflow.rst
>  create mode 100644 doc/usage/bootmeth.rst
>  create mode 100644 drivers/mmc/mmc_bootdev.c
>  create mode 100644 drivers/usb/host/usb_bootdev.c
>  create mode 100644 include/bootdev.h
>  create mode 100644 include/bootflow.h
>  create mode 100644 include/bootmeth.h
>  create mode 100644 include/bootstd.h
>  create mode 100644 include/distro.h
>  create mode 100644 net/eth_bootdev.c
>  create mode 100644 test/boot/Makefile
>  create mode 100644 test/boot/bootdev.c
>  create mode 100644 test/boot/bootflow.c
>  create mode 100644 test/boot/bootmeth.c
>  create mode 100644 test/boot/bootstd_common.c
>  create mode 100644 test/boot/bootstd_common.h
>  create mode 100644 test/py/tests/bootstd/mmc1.img.xz
>
> --
> 2.33.0.1079.g6e70778dc9-goog
>