[Buildroot] [PATCH v3 0/7] Add support for ISO9660 image compatible with Legacy and EFI BIOS

[Buildroot] [PATCH v3 0/7] Add support for ISO9660 image compatible with Legacy and EFI BIOS

[Kory Maincent]

This series of patches aims to support the generation of an ISO9660 hybrid
image compatible with Legacy BIOS and EFI BIOS. To implement this, we need
to improve the grub2 package and modify the ISO9660 image support

Grub2 was written to build only one configuration at a time. For the hybrid
image we need to have several configuration of Grub2 in the same image.
For example we might want to have Grub2 built for BIOS, EFI 32 bits and EFI
64 bits in the same image. To support this, we chose to fill a list of
configuration name tuples, and then build each Grub2 configuration in a
separate build-$(tuple) folder. It seems simpler than having multiple
duplicated grub2 packages for each possible configuration.

The generation of ISO9660 image was only supporting bootloaders based on
Legacy BIOS boot. We first change the ISO9660 image generation to use
xorriso instead of genimageiso, in order to be able to build an image
compatible with both legacy and EFI BIOS. Then we add the generation of an
EFI System Partition in iso9660 so that we can install the EFI-compatible
bootloader."

In detail:

 - PATCH 1 drop the usage of the useless startup.nsh file
 - PATCH 2 implements simultaneous build of GRUB for different configurations
 - PATCH 3 implements the generation of ISO9660 image booting on a EFI BIOS
 - PATCH 4 implements the generation of hybrid image compatible with Legacy
   and EFI BIOS
 - PATCH 5 updates the encoding of the text return from testing emulator
 - PATCH 6 add support to i386 architecture to edk2 package
 - PATCH 7 updates iso9660 tests and implements a test for EFI image and
   hybrid image.

Changes in v2:
- Update the typo of Grub2 configuration tuples to make more legible code
- Expand explanation in few commit messages.
- Fix typos.
- Fix Grub2 Legacy builtin configurations.
- Add mkfs and mcopy parameters in iso9660 package to build reproducible
  images.
- Remove the implementation of host-efi-bios package.
- Add support for i386 architecture to edk2 package.

Changes in v3:
- Add a patch to drop all the startup.nsh usage in Buildroot. The EFI
  payload naming is sufficient to choose the right boot payload.
- Update the Grub2 tuples management to have a more readable package.
- Fix ISO9660 EFI image generation options.
- Fix Kconfig options between EFI and BIOS bootloader in the ISO9660
  package.

Thanks in advance for your review and feedback

Kory Maincent (7):
  board, boot, package: remove usage of startup.nsh in EFI partition
  boot/grub2: add support to build multiple Grub2 configurations in the
    same build
  fs/iso9660: add support to Grub EFI bootloader in the image
  fs/iso9660: add support for hybrid image using Grub bootloader on BIOS
    and EFI
  support/testing/infra/emulator.py: update encoding when calling qemu
  boot/edk2: add support to i386 architecture
  support/testing/tests/fs/test_iso9660.py: add support to test using
    EFI BIOS

 Config.in.legacy                         |  24 +++
 board/aarch64-efi/genimage-efi.cfg       |   3 -
 board/intel/galileo/genimage.cfg         |   3 -
 board/minnowboard/genimage.cfg           |   3 -
 board/pc/genimage-efi.cfg                |   3 -
 board/pc/post-build.sh                   |   2 +-
 board/qemu/aarch64-sbsa/genimage.cfg     |   3 -
 boot/edk2/Config.in                      |  12 +-
 boot/edk2/edk2.mk                        |  12 +-
 boot/grub2/Config.in                     |  53 +++++--
 boot/grub2/grub2.mk                      | 181 +++++++++++++----------
 boot/gummiboot/gummiboot.mk              |   2 -
 fs/iso9660/Config.in                     |  27 +++-
 fs/iso9660/iso9660.mk                    |  64 +++++++-
 package/systemd/systemd.mk               |   2 -
 support/testing/conf/grub2-efi.cfg       |   2 +
 support/testing/infra/emulator.py        |   2 +-
 support/testing/tests/fs/test_iso9660.py |  80 +++++++++-
 18 files changed, 342 insertions(+), 136 deletions(-)
 create mode 100644 support/testing/conf/grub2-efi.cfg

-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

[Kory Maincent]

The startup.nsh file is useless to boot EFI payloads. We just need to
follow the naming detection specified in the UEFI spec.
The EFI payload need to be placed in the boot/efi folder in the EFI partition
and follow the architecture naming as described below:
32bit : bootia32.efi
x64 : bootx64.efi
aarch32 : bootarm.efi
aarch64 : bootaa64.efi

This naming is already right in the packages involved (systemd, grub2,
gummiboot), therefore we just need to drop the generation of the
startup.nsh file.

The usage of the startup.nsh in genimage is also dropped to avoid errors in
the image generation.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---
 board/aarch64-efi/genimage-efi.cfg   | 3 ---
 board/intel/galileo/genimage.cfg     | 3 ---
 board/minnowboard/genimage.cfg       | 3 ---
 board/pc/genimage-efi.cfg            | 3 ---
 board/pc/post-build.sh               | 2 +-
 board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
 boot/grub2/grub2.mk                  | 8 --------
 boot/gummiboot/gummiboot.mk          | 2 --
 package/systemd/systemd.mk           | 2 --
 9 files changed, 1 insertion(+), 28 deletions(-)

diff --git a/board/aarch64-efi/genimage-efi.cfg b/board/aarch64-efi/genimage-efi.cfg
index 34cca1d64e..2daa70c361 100644
--- a/board/aarch64-efi/genimage-efi.cfg
+++ b/board/aarch64-efi/genimage-efi.cfg
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
diff --git a/board/intel/galileo/genimage.cfg b/board/intel/galileo/genimage.cfg
index 31add21e67..b930c0deb9 100644
--- a/board/intel/galileo/genimage.cfg
+++ b/board/intel/galileo/genimage.cfg
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
diff --git a/board/minnowboard/genimage.cfg b/board/minnowboard/genimage.cfg
index c5b07179b4..521385aadd 100644
--- a/board/minnowboard/genimage.cfg
+++ b/board/minnowboard/genimage.cfg
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
diff --git a/board/pc/genimage-efi.cfg b/board/pc/genimage-efi.cfg
index ec7e85b06c..34befb7d69 100644
--- a/board/pc/genimage-efi.cfg
+++ b/board/pc/genimage-efi.cfg
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
diff --git a/board/pc/post-build.sh b/board/pc/post-build.sh
index ed37b3b87e..db23795e27 100755
--- a/board/pc/post-build.sh
+++ b/board/pc/post-build.sh
@@ -5,7 +5,7 @@ set -e
 BOARD_DIR=$(dirname "$0")
 
 # Detect boot strategy, EFI or BIOS
-if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
+if [ -d "$BINARIES_DIR/efi-part/" ]; then
     cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
 else
     cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
diff --git a/board/qemu/aarch64-sbsa/genimage.cfg b/board/qemu/aarch64-sbsa/genimage.cfg
index 285b308d90..c75d9fcca1 100644
--- a/board/qemu/aarch64-sbsa/genimage.cfg
+++ b/board/qemu/aarch64-sbsa/genimage.cfg
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
index 52e9199ae9..ad2edf17aa 100644
--- a/boot/grub2/grub2.mk
+++ b/boot/grub2/grub2.mk
@@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
 	$(GRUB2_IMAGE_INSTALL_ELTORITO)
 endef
 
-ifeq ($(GRUB2_PLATFORM),efi)
-define GRUB2_EFI_STARTUP_NSH
-	echo $(notdir $(GRUB2_IMAGE)) > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
-endef
-GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
-endif
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
diff --git a/boot/gummiboot/gummiboot.mk b/boot/gummiboot/gummiboot.mk
index 748e87030e..eb1f3da78c 100644
--- a/boot/gummiboot/gummiboot.mk
+++ b/boot/gummiboot/gummiboot.mk
@@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
 define GUMMIBOOT_INSTALL_IMAGES_CMDS
 	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
-	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
index 5a76f2f246..c403896759 100644
--- a/package/systemd/systemd.mk
+++ b/package/systemd/systemd.mk
@@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
 define SYSTEMD_INSTALL_BOOT_FILES
 	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
-	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Kory Maincent]

When Grub2 is build it is configured only for one boot set-up, BIOS Legacy,
EFI 32 bit or EFI 64 bit. It can not deal with several boot set-up on the
same image.

This patch allows to build Grub2 for different configurations simultaneously.
To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI
BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more
reasonable to just extend the grub2 package into 3 packages.

We can no longer use autotools-package as a consequence of this multi-build, and
we have to resort to generic-package and a partial duplication of
the autotools-infra. Grub2 was already using custom option like --prefix or
--exec-prefix so this won't add much more weirdness.

We use a GRUB2_TUPLES list to describe all the configurations selected.
For each boot case described in the GRUB2_TUPLES list, it configures and
builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.

We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the
BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---

Changes in v2:
Following the review from Yann E. Morin:
- Update the tuples configurations to makes the code more readable.
- Move "--target" and "--with-platform" parameters into the
  GRUB2_CONFIGURE_CMD.
- Fix startup.nsh installation.
- Fix Grub2 configuration to be sure that at least one platform is always enabled.
- Fix legacy builtin configurations to string in place of bool.

Changes in v3:
Following the review from Yann E. Morin:
- Update the tuples configurations to remove the conditions and makes it
  more readable.
- Update the tuples configurations for builtin config and modules options.
- Fix typo.
- Remove the generation of startup.nsh.

 Config.in.legacy                         |  24 ++++
 boot/grub2/Config.in                     |  53 +++++--
 boot/grub2/grub2.mk                      | 175 +++++++++++++----------
 support/testing/tests/fs/test_iso9660.py |   6 +-
 4 files changed, 169 insertions(+), 89 deletions(-)

diff --git a/Config.in.legacy b/Config.in.legacy
index 35a11f4dc6..33cee0202b 100644
--- a/Config.in.legacy
+++ b/Config.in.legacy
@@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
 
 comment "Legacy options removed in 2021.08"
 
+config BR2_TARGET_GRUB2_BUILTIN_MODULES
+	string "the grub2 builtin modules has been renamed"
+	help
+	  This option has been split to separate the builtin modules
+	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
+	select BR2_LEGACY
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+	string "the grub2 builtin configuration has been renamed"
+	help
+	  This option has been split to separate the builtin configuration
+	  between BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_CONFIG != ""
+	select BR2_LEGACY
+
 config BR2_PACKAGE_LIBMCRYPT
 	bool "libmcrypt package was removed"
 	select BR2_LEGACY
diff --git a/boot/grub2/Config.in b/boot/grub2/Config.in
index e45133999e..10bba6c5e4 100644
--- a/boot/grub2/Config.in
+++ b/boot/grub2/Config.in
@@ -10,6 +10,13 @@ config BR2_TARGET_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2_ARCH_SUPPORTS
 	depends on BR2_USE_WCHAR
+	select BR2_TARGET_GRUB2_I386_PC if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		(BR2_i386 || BR2_x86_64)
+	select BR2_TARGET_GRUB2_ARM_UBOOT if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		BR2_arm
+	select BR2_TARGET_GRUB2_ARM64_EFI if BR2_aarch64
 	help
 	  GNU GRUB is a Multiboot boot loader. It was derived from
 	  GRUB, the GRand Unified Bootloader, which was originally
@@ -25,10 +32,10 @@ config BR2_TARGET_GRUB2
 
 	  http://www.gnu.org/software/grub/
 
-if BR2_TARGET_GRUB2
+config BR2_TARGET_GRUB2_HAS_PTF
+	bool
 
-choice
-	prompt "Platform"
+if BR2_TARGET_GRUB2
 
 config BR2_TARGET_GRUB2_I386_PC
 	bool "i386-pc"
@@ -40,6 +47,7 @@ config BR2_TARGET_GRUB2_I386_PC
 config BR2_TARGET_GRUB2_I386_EFI
 	bool "i386-efi"
 	depends on BR2_i386 || BR2_x86_64
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting has a
 	  32 bits EFI BIOS. Note that some x86-64 platforms use a 32
@@ -48,6 +56,7 @@ config BR2_TARGET_GRUB2_I386_EFI
 config BR2_TARGET_GRUB2_X86_64_EFI
 	bool "x86-64-efi"
 	depends on BR2_x86_64
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting has a
 	  64 bits EFI BIOS.
@@ -63,6 +72,7 @@ config BR2_TARGET_GRUB2_ARM_UBOOT
 config BR2_TARGET_GRUB2_ARM_EFI
 	bool "arm-efi"
 	depends on BR2_arm
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting is an
 	  ARM platform and you want to boot Grub 2 as an EFI
@@ -76,10 +86,10 @@ config BR2_TARGET_GRUB2_ARM64_EFI
 	  Aarch64 platform and you want to boot Grub 2 as an EFI
 	  application.
 
-endchoice
-
 if BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
 
+comment "Options for the x86 legacy BIOS or ARM U-Boot support"
+
 config BR2_TARGET_GRUB2_BOOT_PARTITION
 	string "boot partition"
 	default "hd0,msdos1"
@@ -89,24 +99,43 @@ config BR2_TARGET_GRUB2_BOOT_PARTITION
 	  first disk if using a legacy partition table, or 'hd0,gpt1'
 	  if using GPT partition table.
 
-endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
-
-config BR2_TARGET_GRUB2_BUILTIN_MODULES
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_PC
 	string "builtin modules"
+	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
 	default "boot linux ext2 fat squash4 part_msdos part_gpt normal biosdisk" if BR2_TARGET_GRUB2_I386_PC
-	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop" \
-		if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
-		   BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
 	default "linux ext2 fat part_msdos normal" if BR2_TARGET_GRUB2_ARM_UBOOT
 
-config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC
+	string "builtin config"
+	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
+	help
+	  Path to a Grub 2 configuration file that will be embedded
+	  into the Grub image itself. This allows to set the root
+	  device and other configuration parameters, but however menu
+	  entries cannot be described in this embedded configuration.
+
+endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
+
+if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
+	BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
+
+comment "Options for the EFI BIOS or ARM EFI support"
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI
+	string "builtin modules"
+	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
+	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop"
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI
 	string "builtin config"
+	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
 	help
 	  Path to a Grub 2 configuration file that will be embedded
 	  into the Grub image itself. This allows to set the root
 	  device and other configuration parameters, but however menu
 	  entries cannot be described in this embedded configuration.
 
+endif
 config BR2_TARGET_GRUB2_INSTALL_TOOLS
 	bool "install tools"
 	help
diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
index ad2edf17aa..e01ebb2edb 100644
--- a/boot/grub2/grub2.mk
+++ b/boot/grub2/grub2.mk
@@ -57,53 +57,65 @@ GRUB2_INSTALL_TARGET = NO
 endif
 GRUB2_CPE_ID_VENDOR = gnu
 
-GRUB2_BUILTIN_MODULES = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES))
-GRUB2_BUILTIN_CONFIG = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG))
+GRUB2_BUILTIN_MODULES_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC))
+GRUB2_BUILTIN_MODULES_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI))
+GRUB2_BUILTIN_CONFIG_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC))
+GRUB2_BUILTIN_CONFIG_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI))
 GRUB2_BOOT_PARTITION = $(call qstrip,$(BR2_TARGET_GRUB2_BOOT_PARTITION))
 
-ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/grub.img
-GRUB2_CFG = $(TARGET_DIR)/boot/grub/grub.cfg
-GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
-GRUB2_TUPLE = i386-pc
-GRUB2_TARGET = i386
-GRUB2_PLATFORM = pc
-else ifeq ($(BR2_TARGET_GRUB2_I386_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = i386-efi
-GRUB2_TARGET = i386
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_X86_64_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = x86_64-efi
-GRUB2_TARGET = x86_64
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_ARM_UBOOT),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/boot-part/grub/grub.img
-GRUB2_CFG = $(BINARIES_DIR)/boot-part/grub/grub.cfg
-GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
-GRUB2_TUPLE = arm-uboot
-GRUB2_TARGET = arm
-GRUB2_PLATFORM = uboot
-else ifeq ($(BR2_TARGET_GRUB2_ARM_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = arm-efi
-GRUB2_TARGET = arm
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_ARM64_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = arm64-efi
-GRUB2_TARGET = aarch64
-GRUB2_PLATFORM = efi
-endif
+GRUB2_IMAGE_i386-pc = $(BINARIES_DIR)/grub.img
+GRUB2_CFG_i386-pc = $(TARGET_DIR)/boot/grub/grub.cfg
+GRUB2_PREFIX_i386-pc = ($(GRUB2_BOOT_PARTITION))/boot/grub
+GRUB2_TARGET_i386-pc = i386
+GRUB2_PLATFORM_i386-pc = pc
+GRUB2_BUILTIN_CONFIG_i386-pc = $(GRUB2_BUILTIN_CONFIG_PC)
+GRUB2_BUILTIN_MODULES_i386-pc = $(GRUB2_BUILTIN_MODULES_PC)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_PC) += i386-pc
+
+GRUB2_IMAGE_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
+GRUB2_CFG_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_i386-efi = /EFI/BOOT
+GRUB2_TARGET_i386-efi = i386
+GRUB2_PLATFORM_i386-efi = efi
+GRUB2_BUILTIN_CONFIG_i386-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_i386-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_EFI) += i386-efi
+
+GRUB2_IMAGE_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
+GRUB2_CFG_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_x86_64-efi = /EFI/BOOT
+GRUB2_TARGET_x86_64-efi = x86_64
+GRUB2_PLATFORM_x86_64-efi = efi
+GRUB2_BUILTIN_CONFIG_x86_64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_x86_64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_X86_64_EFI) += x86_64-efi
+
+GRUB2_IMAGE_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.img
+GRUB2_CFG_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.cfg
+GRUB2_PREFIX_arm-uboot = ($(GRUB2_BOOT_PARTITION))/boot/grub
+GRUB2_TARGET_arm-uboot = arm
+GRUB2_PLATFORM_arm-uboot = uboot
+GRUB2_BUILTIN_CONFIG_arm-uboot = $(GRUB2_BUILTIN_CONFIG_PC)
+GRUB2_BUILTIN_MODULES_arm-uboot = $(GRUB2_BUILTIN_MODULES_PC)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_UBOOT) += arm-uboot
+
+GRUB2_IMAGE_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
+GRUB2_CFG_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_arm-efi = /EFI/BOOT
+GRUB2_TARGET_arm-efi = arm
+GRUB2_PLATFORM_arm-efi = efi
+GRUB2_BUILTIN_CONFIG_arm-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_arm-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_EFI) += arm-efi
+
+GRUB2_IMAGE_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
+GRUB2_CFG_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_arm64-efi = /EFI/BOOT
+GRUB2_TARGET_arm64-efi = aarch64
+GRUB2_PLATFORM_arm64-efi = efi
+GRUB2_BUILTIN_CONFIG_arm64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_arm64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi
 
 # Grub2 is kind of special: it considers CC, LD and so on to be the
 # tools to build the host programs and uses TARGET_CC, TARGET_CFLAGS,
@@ -127,18 +139,6 @@ GRUB2_CONF_ENV = \
 	TARGET_OBJCOPY="$(TARGET_OBJCOPY)" \
 	TARGET_STRIP="$(TARGET_CROSS)strip"
 
-GRUB2_CONF_OPTS = \
-	--target=$(GRUB2_TARGET) \
-	--with-platform=$(GRUB2_PLATFORM) \
-	--prefix=/ \
-	--exec-prefix=/ \
-	--disable-grub-mkfont \
-	--enable-efiemu=no \
-	ac_cv_lib_lzma_lzma_code=no \
-	--enable-device-mapper=no \
-	--enable-libzfs=no \
-	--disable-werror
-
 HOST_GRUB2_CONF_OPTS = \
 	--disable-grub-mkfont \
 	--enable-efiemu=no \
@@ -147,26 +147,53 @@ HOST_GRUB2_CONF_OPTS = \
 	--enable-libzfs=no \
 	--disable-werror
 
-ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
-define GRUB2_IMAGE_INSTALL_ELTORITO
-	cat $(HOST_DIR)/lib/grub/$(GRUB2_TUPLE)/cdboot.img $(GRUB2_IMAGE) > \
-		$(BINARIES_DIR)/grub-eltorito.img
+define GRUB2_CONFIGURE_CMDS
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		mkdir -p $(@D)/build-$(tuple) ; \
+		cd $(@D)/build-$(tuple) ; \
+		$(TARGET_CONFIGURE_OPTS) \
+		$(TARGET_CONFIGURE_ARGS) \
+		$(GRUB2_CONF_ENV) \
+		../configure \
+			--target=$(GRUB2_TARGET_$(tuple)) \
+			--with-platform=$(GRUB2_PLATFORM_$(tuple)) \
+			--host=$(GNU_TARGET_NAME) \
+			--build=$(GNU_HOST_NAME) \
+			--prefix=/ \
+			--exec-prefix=/ \
+			--disable-grub-mkfont \
+			--enable-efiemu=no \
+			ac_cv_lib_lzma_lzma_code=no \
+			--enable-device-mapper=no \
+			--enable-libzfs=no \
+			--disable-werror
+	)
+endef
+
+define GRUB2_BUILD_CMDS
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
+	)
 endef
-endif
 
 define GRUB2_INSTALL_IMAGES_CMDS
-	mkdir -p $(dir $(GRUB2_IMAGE))
-	$(HOST_DIR)/usr/bin/grub-mkimage \
-		-d $(@D)/grub-core/ \
-		-O $(GRUB2_TUPLE) \
-		-o $(GRUB2_IMAGE) \
-		-p "$(GRUB2_PREFIX)" \
-		$(if $(GRUB2_BUILTIN_CONFIG),-c $(GRUB2_BUILTIN_CONFIG)) \
-		$(GRUB2_BUILTIN_MODULES)
-	mkdir -p $(dir $(GRUB2_CFG))
-	$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG)
-	$(GRUB2_IMAGE_INSTALL_ELTORITO)
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
+		$(HOST_DIR)/usr/bin/grub-mkimage \
+			-d $(@D)/build-$(tuple)/grub-core/ \
+			-O $(tuple) \
+			-o $(GRUB2_IMAGE_$(tuple)) \
+			-p "$(GRUB2_PREFIX_$(tuple))" \
+			$(if $(GRUB2_BUILTIN_CONFIG_$(tuple)), \
+				-c $(GRUB2_BUILTIN_CONFIG_$(tuple))) \
+			$(GRUB2_BUILTIN_MODULES_$(tuple)) ; \
+		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple)) ; \
+		$(if $(findstring $(GRUB2_PLATFORM_$(tuple)), pc), \
+			cat $(HOST_DIR)/lib/grub/$(tuple)/cdboot.img $(GRUB2_IMAGE_$(tuple)) > \
+				$(BINARIES_DIR)/grub-eltorito.img ; \
+		) \
+	)
 endef
 
-$(eval $(autotools-package))
+$(eval $(generic-package))
 $(eval $(host-autotools-package))
diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
index 68f4840852..1b699e1fef 100644
--- a/support/testing/tests/fs/test_iso9660.py
+++ b/support/testing/tests/fs/test_iso9660.py
@@ -54,7 +54,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -75,7 +75,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -95,7 +95,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_INITRD=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 3/7] fs/iso9660: add support to Grub EFI bootloader in the image

[Kory Maincent]

Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660
image.
For that we need to create EFI System Partition (ESP). The ESP is a vfat
partition which contain the Grub2 binary at the /EFI/BOOT/ location.
xorriso command will generate the iso image including the ESP.

We notice Grub can not read and mount the ESP, therefore we place the Grub
configuration file in the ISO9660 partition. A Grub2 builtin configuration
need to be used to tell Grub2 to search automatically its configuration
file in the ISO9660 partition. Use 'set root=(cd0)' in the configuration
file passed to BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---

Changes in v2:
Following the review from Yann E. Morin:
- Add parameter to mkfs and mcopy to allow reproducible builds
- Fix typos and variable multi-line assignement

Changes in v3:
- Fix Kconfig dependencies.
- Add exclusive configuration choice between BIOS bootloader and EFI
  bootloader in Kconfig.
- Fix namings and reproducibility issues.
- Update image generation options to separate common options.

 fs/iso9660/Config.in  | 29 ++++++++++++++++++++++-------
 fs/iso9660/iso9660.mk | 43 ++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 62 insertions(+), 10 deletions(-)

diff --git a/fs/iso9660/Config.in b/fs/iso9660/Config.in
index 6f001c0640..f8f577506d 100644
--- a/fs/iso9660/Config.in
+++ b/fs/iso9660/Config.in
@@ -2,8 +2,7 @@ config BR2_TARGET_ROOTFS_ISO9660
 	bool "iso image"
 	depends on (BR2_i386 || BR2_x86_64)
 	depends on BR2_LINUX_KERNEL
-	depends on BR2_TARGET_GRUB2_I386_PC || \
-		BR2_TARGET_SYSLINUX_ISOLINUX
+	depends on BR2_TARGET_GRUB2 || BR2_TARGET_SYSLINUX_ISOLINUX
 	select BR2_LINUX_KERNEL_INSTALL_TARGET \
 	       if (!BR2_TARGET_ROOTFS_ISO9660_INITRD && !BR2_TARGET_ROOTFS_INITRAMFS)
 	help
@@ -27,12 +26,15 @@ choice
 
 config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 	bool "grub2"
-	depends on BR2_TARGET_GRUB2_I386_PC
+	depends on BR2_TARGET_GRUB2
 	help
 	  Use Grub 2 as the bootloader for the ISO9660 image. Make
 	  sure to enable the 'iso9660' module in
-	  BR2_TARGET_GRUB2_BUILTIN_MODULES and to use 'cd' as the boot
-	  partition in BR2_TARGET_GRUB2_BOOT_PARTITION=.
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_PC or
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI. Use 'cd' as the boot
+	  partition in BR2_TARGET_GRUB2_BOOT_PARTITION= for GRUB on BIOS
+	  or 'set root=(cd0)' in the configuration file passed to
+	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI for GRUB on EFI.
 
 config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 	bool "isolinux"
@@ -40,6 +42,19 @@ config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 
 endchoice
 
+choice
+	prompt "Boot payload"
+
+config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
+	bool "legacy bios"
+	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
+
+config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
+	bool "UEFI"
+	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
+
+endchoice
+
 config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
 	string "Boot menu config file"
 	default "fs/iso9660/grub.cfg" if BR2_TARGET_ROOTFS_ISO9660_GRUB2
@@ -83,7 +98,7 @@ config BR2_TARGET_ROOTFS_ISO9660_HYBRID
 
 endif
 
-comment "iso image needs a Linux kernel and either grub2 i386-pc or isolinux to be built"
+comment "iso image needs a Linux kernel and either grub2 or isolinux to be built"
 	depends on BR2_i386 || BR2_x86_64
 	depends on !BR2_LINUX_KERNEL || \
-		!(BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_SYSLINUX_ISOLINUX)
+		!(BR2_TARGET_GRUB2 || BR2_TARGET_SYSLINUX_ISOLINUX)
diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
index 23421a9a5c..921efa1b02 100644
--- a/fs/iso9660/iso9660.mk
+++ b/fs/iso9660/iso9660.mk
@@ -57,7 +57,12 @@ else
 ROOTFS_ISO9660_TMP_TARGET_DIR = $(TARGET_DIR)
 endif
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2),y)
+ifeq ($(BR2_REPRODUCIBLE),y)
+ROOTFS_ISO9660_VFAT_OPTS = --invariant
+ROOTFS_ISO9660_FIX_TIME = touch $(SOURCE_DATE_EPOCH)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),yy)
 ROOTFS_ISO9660_DEPENDENCIES += grub2
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
@@ -66,6 +71,22 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
 endef
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
+ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi
+ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
+ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
+	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
+	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
+	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
+	$(ROOTFS_ISO9660_FIX_TIME)
+	$(HOST_DIR)/sbin/mkfs.vfat $(ROOTFS_ISO9660_VFAT_OPTS) $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
+	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
+		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/
+endef
 else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
 ROOTFS_ISO9660_DEPENDENCIES += syslinux
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
@@ -128,9 +149,25 @@ ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_DISABLE_EXTERNAL_INITRD
 
 endif # ROOTFS_ISO9660_USE_INITRD
 
+ROOTFS_ISO9660_OPTS += \
+	-J \
+	-R \
+	-no-emul-boot
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
+ROOTFS_ISO9660_OPTS += \
+	-boot-load-size 4 \
+	-boot-info-table \
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
+ROOTFS_ISO9660_OPTS += \
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+endif
+
 define ROOTFS_ISO9660_CMD
-	$(HOST_DIR)/bin/xorriso -as mkisofs -J -R -b $(ROOTFS_ISO9660_BOOT_IMAGE) \
-		-no-emul-boot -boot-load-size 4 -boot-info-table \
+	$(HOST_DIR)/bin/xorriso -as mkisofs \
 		$(ROOTFS_ISO9660_OPTS) \
 		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
 endef
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

[Kory Maincent]

Add support for building an hybrid ISO9660 image compatible with legacy
and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command
to generate the hybrid image.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---

Changes in v2:
- Fix typos

Changes in v3:
- Follow changes brougt by modification of the third patch.

 fs/iso9660/Config.in  | 16 +++++++--------
 fs/iso9660/iso9660.mk | 45 ++++++++++++++++++++++++++++---------------
 2 files changed, 36 insertions(+), 25 deletions(-)

diff --git a/fs/iso9660/Config.in b/fs/iso9660/Config.in
index f8f577506d..8c4e7badf8 100644
--- a/fs/iso9660/Config.in
+++ b/fs/iso9660/Config.in
@@ -27,6 +27,10 @@ choice
 config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER \
+		if BR2_TARGET_GRUB2_I386_PC
+	select BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER \
+		if (BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI)
 	help
 	  Use Grub 2 as the bootloader for the ISO9660 image. Make
 	  sure to enable the 'iso9660' module in
@@ -39,21 +43,15 @@ config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 	bool "isolinux"
 	depends on BR2_TARGET_SYSLINUX_ISOLINUX
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
 
 endchoice
 
-choice
-	prompt "Boot payload"
-
 config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
-	bool "legacy bios"
-	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
-	bool "UEFI"
-	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
-
-endchoice
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
 	string "Boot menu config file"
diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
index 921efa1b02..9c2535d102 100644
--- a/fs/iso9660/iso9660.mk
+++ b/fs/iso9660/iso9660.mk
@@ -67,18 +67,20 @@ ROOTFS_ISO9660_DEPENDENCIES += grub2
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
 ROOTFS_ISO9660_BOOT_IMAGE = boot/grub/grub-eltorito.img
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
 endef
-else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
 ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
 ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi
 ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
 ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI
 	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
 	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
 	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
@@ -87,12 +89,14 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
 	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
 		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/
 endef
-else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
 ROOTFS_ISO9660_DEPENDENCIES += syslinux
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/isolinux.cfg
 ROOTFS_ISO9660_BOOT_IMAGE = isolinux/isolinux.bin
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/syslinux/* \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/
 	$(INSTALL) -D -m 0644 $(HOST_DIR)/share/syslinux/ldlinux.c32 \
@@ -105,7 +109,8 @@ define ROOTFS_ISO9660_PREPARATION
 		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
 	$(SED) "s%__KERNEL_PATH__%/boot/$(LINUX_IMAGE_NAME)%" \
 		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
-	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER)
+	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS)
+	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI)
 endef
 
 ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_PREPARATION
@@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
 
 ROOTFS_ISO9660_OPTS += \
 	-J \
-	-R \
-	-no-emul-boot
+	-R
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
+ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
+	-no-emul-boot \
 	-boot-load-size 4 \
-	-boot-info-table \
-	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
-endif
+	-boot-info-table
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
-	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
+	-no-emul-boot
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
+ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
 endif
 
 define ROOTFS_ISO9660_CMD
 	$(HOST_DIR)/bin/xorriso -as mkisofs \
 		$(ROOTFS_ISO9660_OPTS) \
+		$(ROOTFS_ISO9660_BOOTLOADER_OPTS) \
 		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
 endef
 
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Kory Maincent]

With UTF-8 got issue with wrong character returned by Qemu when using EFI
BIOS. This breaks the test process with the following error. Update to
ISO-8859-1 encoding to avoid it.

    emulator.login()
  File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
    index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
    return self.expect_list(compiled_pattern_list,
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
    return super(spawn, self).read_nonblocking(size)
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
    s = self._decoder.decode(s, final=False)
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---
 support/testing/infra/emulator.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 0a77eb80fc..1fab9caad8 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -76,7 +76,7 @@ class Emulator(object):
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
         self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
                                   timeout=5 * self.timeout_multiplier,
-                                  encoding='utf-8',
+                                  encoding='ISO-8859-1',
                                   env={"QEMU_AUDIO_DRV": "none"})
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

[Kory Maincent]

Add support the build the firmware for QEMU i386 pc machine

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---
 boot/edk2/Config.in | 12 +++++++++++-
 boot/edk2/edk2.mk   | 12 ++++++++++--
 2 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/boot/edk2/Config.in b/boot/edk2/Config.in
index 150806899f..9f8988bcca 100644
--- a/boot/edk2/Config.in
+++ b/boot/edk2/Config.in
@@ -1,6 +1,6 @@
 config BR2_TARGET_EDK2
 	bool "EDK2"
-	depends on BR2_x86_64 || BR2_aarch64
+	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386
 	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
 	select BR2_PACKAGE_EDK2_PLATFORMS
 	help
@@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
 
 choice
 	prompt "Platform"
+	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
 	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
 	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
 
+config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
+	bool "i386"
+	depends on BR2_i386 || BR2_x86_64
+	help
+	  Platform configuration for a generic i386 target.
+	  This platform will boot from flash address 0x0.
+	  It should therefore be used as the first bootloader.
+
 config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	bool "x86-64"
 	depends on BR2_x86_64
@@ -94,6 +103,7 @@ endchoice
 
 config BR2_TARGET_EDK2_FD_NAME
 	string
+	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
 	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
diff --git a/boot/edk2/edk2.mk b/boot/edk2/edk2.mk
index fabd0c5b45..ab3cdad464 100644
--- a/boot/edk2/edk2.mk
+++ b/boot/edk2/edk2.mk
@@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
 EDK2_INSTALL_TARGET = NO
 EDK2_INSTALL_IMAGES = YES
 
-ifeq ($(BR2_x86_64),y)
+ifeq ($(BR2_i386),y)
+EDK2_ARCH = IA32
+else ifeq ($(BR2_x86_64),y)
 EDK2_ARCH = X64
 else ifeq ($(BR2_aarch64),y)
 EDK2_ARCH = AARCH64
@@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
 EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
 EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
 
-ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
+ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
+EDK2_DEPENDENCIES += host-nasm
+EDK2_PACKAGE_NAME = OvmfPkg
+EDK2_PLATFORM_NAME = OvmfPkgIa32
+EDK2_BUILD_DIR = OvmfIa32
+
+else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
 EDK2_DEPENDENCIES += host-nasm
 EDK2_PACKAGE_NAME = OvmfPkg
 EDK2_PLATFORM_NAME = OvmfPkgX64
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 7/7] support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS

[Kory Maincent]

The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
Add dedicated Grub2 builtin config for the EFI compatible cases.

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
---
 support/testing/conf/grub2-efi.cfg       |  2 +
 support/testing/tests/fs/test_iso9660.py | 74 +++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 2 deletions(-)
 create mode 100644 support/testing/conf/grub2-efi.cfg

diff --git a/support/testing/conf/grub2-efi.cfg b/support/testing/conf/grub2-efi.cfg
new file mode 100644
index 0000000000..11c32e880e
--- /dev/null
+++ b/support/testing/conf/grub2-efi.cfg
@@ -0,0 +1,2 @@
+set root=(cd0)
+set prefix=/boot/grub
diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
index 1b699e1fef..8315442604 100644
--- a/support/testing/tests/fs/test_iso9660.py
+++ b/support/testing/tests/fs/test_iso9660.py
@@ -25,9 +25,13 @@ BASIC_CONFIG = \
     """.format(infra.filepath("conf/minimal-x86-qemu-kernel.config"))
 
 
-def test_mount_internal_external(emulator, builddir, internal=True):
+def test_mount_internal_external(emulator, builddir, internal=True, efi=False):
     img = os.path.join(builddir, "images", "rootfs.iso9660")
-    emulator.boot(arch="i386", options=["-cdrom", img])
+    if efi:
+        efi_img = os.path.join(builddir, "images", "OVMF.fd")
+        emulator.boot(arch="i386", options=["-cdrom", img, "-bios", efi_img])
+    else:
+        emulator.boot(arch="i386", options=["-cdrom", img])
     emulator.login()
 
     if internal:
@@ -53,6 +57,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660=y
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -74,6 +79,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -94,6 +100,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660=y
         BR2_TARGET_ROOTFS_ISO9660_INITRD=y
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -107,6 +114,69 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         exit_code = test_touch_file(self.emulator)
         self.assertEqual(exit_code, 0)
 
+
+class TestIso9660Grub2EFI(infra.basetest.BRTest):
+    config = BASIC_CONFIG + \
+        """
+        BR2_TARGET_ROOTFS_ISO9660=y
+        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
+        BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_EFI=y
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat part_msdos part_gpt normal iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
+        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
+        BR2_TARGET_EDK2=y
+        """.format(infra.filepath("conf/grub2-efi.cfg"),
+                   infra.filepath("conf/grub2.cfg"))
+
+    def test_run(self):
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=True)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+
+class TestIso9660Grub2Hybrid(infra.basetest.BRTest):
+    config = BASIC_CONFIG + \
+        """
+        BR2_TARGET_ROOTFS_ISO9660=y
+        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
+        BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
+        BR2_TARGET_GRUB2_I386_EFI=y
+        BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 biosdisk"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC=""
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 efi_gop"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
+        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
+        BR2_TARGET_EDK2=y
+        """.format(infra.filepath("conf/grub2-efi.cfg"),
+                   infra.filepath("conf/grub2.cfg"))
+
+    def test_run(self):
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=False)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+        self.emulator.stop()
+
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=True)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+
 #
 # Syslinux
 
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

[Yann E. MORIN]

Köry, All,

+Dick, Erico, Gerome, Nicholas, Peter, Romain:

This patch touches boards for which you are registered in the DEVELOPERS
file. Could you verify that those boards still boot with this patch
applied, please?

Otherwise, I have no comment on the patch; good! :-)

Regards,
Yann E. MORIN.

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> The startup.nsh file is useless to boot EFI payloads. We just need to
> follow the naming detection specified in the UEFI spec.
> The EFI payload need to be placed in the boot/efi folder in the EFI partition
> and follow the architecture naming as described below:
> 32bit : bootia32.efi
> x64 : bootx64.efi
> aarch32 : bootarm.efi
> aarch64 : bootaa64.efi
> 
> This naming is already right in the packages involved (systemd, grub2,
> gummiboot), therefore we just need to drop the generation of the
> startup.nsh file.
> 
> The usage of the startup.nsh in genimage is also dropped to avoid errors in
> the image generation.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  board/aarch64-efi/genimage-efi.cfg   | 3 ---
>  board/intel/galileo/genimage.cfg     | 3 ---
>  board/minnowboard/genimage.cfg       | 3 ---
>  board/pc/genimage-efi.cfg            | 3 ---
>  board/pc/post-build.sh               | 2 +-
>  board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
>  boot/grub2/grub2.mk                  | 8 --------
>  boot/gummiboot/gummiboot.mk          | 2 --
>  package/systemd/systemd.mk           | 2 --
>  9 files changed, 1 insertion(+), 28 deletions(-)
> 
> diff --git a/board/aarch64-efi/genimage-efi.cfg b/board/aarch64-efi/genimage-efi.cfg
> index 34cca1d64e..2daa70c361 100644
> --- a/board/aarch64-efi/genimage-efi.cfg
> +++ b/board/aarch64-efi/genimage-efi.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/board/intel/galileo/genimage.cfg b/board/intel/galileo/genimage.cfg
> index 31add21e67..b930c0deb9 100644
> --- a/board/intel/galileo/genimage.cfg
> +++ b/board/intel/galileo/genimage.cfg
> @@ -1,9 +1,6 @@
>  # Create an image of the efi partition
>  image efi-part.vfat {
>  	vfat {
> -		file startup.nsh {
> -			image = "efi-part/startup.nsh"
> -		}
>  		file EFI {
>  			image = "efi-part/EFI"
>  		}
> diff --git a/board/minnowboard/genimage.cfg b/board/minnowboard/genimage.cfg
> index c5b07179b4..521385aadd 100644
> --- a/board/minnowboard/genimage.cfg
> +++ b/board/minnowboard/genimage.cfg
> @@ -1,9 +1,6 @@
>  # Create an image of the efi partition
>  image efi-part.vfat {
>  	vfat {
> -		file startup.nsh {
> -			image = "efi-part/startup.nsh"
> -		}
>  		file EFI {
>  			image = "efi-part/EFI"
>  		}
> diff --git a/board/pc/genimage-efi.cfg b/board/pc/genimage-efi.cfg
> index ec7e85b06c..34befb7d69 100644
> --- a/board/pc/genimage-efi.cfg
> +++ b/board/pc/genimage-efi.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/board/pc/post-build.sh b/board/pc/post-build.sh
> index ed37b3b87e..db23795e27 100755
> --- a/board/pc/post-build.sh
> +++ b/board/pc/post-build.sh
> @@ -5,7 +5,7 @@ set -e
>  BOARD_DIR=$(dirname "$0")
>  
>  # Detect boot strategy, EFI or BIOS
> -if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
> +if [ -d "$BINARIES_DIR/efi-part/" ]; then
>      cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
>  else
>      cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
> diff --git a/board/qemu/aarch64-sbsa/genimage.cfg b/board/qemu/aarch64-sbsa/genimage.cfg
> index 285b308d90..c75d9fcca1 100644
> --- a/board/qemu/aarch64-sbsa/genimage.cfg
> +++ b/board/qemu/aarch64-sbsa/genimage.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
> index 52e9199ae9..ad2edf17aa 100644
> --- a/boot/grub2/grub2.mk
> +++ b/boot/grub2/grub2.mk
> @@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
>  	$(GRUB2_IMAGE_INSTALL_ELTORITO)
>  endef
>  
> -ifeq ($(GRUB2_PLATFORM),efi)
> -define GRUB2_EFI_STARTUP_NSH
> -	echo $(notdir $(GRUB2_IMAGE)) > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
> -endef
> -GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
> -endif
> -
>  $(eval $(autotools-package))
>  $(eval $(host-autotools-package))
> diff --git a/boot/gummiboot/gummiboot.mk b/boot/gummiboot/gummiboot.mk
> index 748e87030e..eb1f3da78c 100644
> --- a/boot/gummiboot/gummiboot.mk
> +++ b/boot/gummiboot/gummiboot.mk
> @@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
>  define GUMMIBOOT_INSTALL_IMAGES_CMDS
>  	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
>  		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
> -	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
>  	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
>  		$(BINARIES_DIR)/efi-part/loader/loader.conf
>  	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
> diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
> index 5a76f2f246..c403896759 100644
> --- a/package/systemd/systemd.mk
> +++ b/package/systemd/systemd.mk
> @@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
>  define SYSTEMD_INSTALL_BOOT_FILES
>  	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
>  		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
> -	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
>  	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
>  		$(BINARIES_DIR)/efi-part/loader/loader.conf
>  	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
> -- 
> 2.25.1
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@lists.buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

[Erico Nunes]

On Thu, Sep 23, 2021 at 10:07 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Köry, All,
>
> +Dick, Erico, Gerome, Nicholas, Peter, Romain:
>
> This patch touches boards for which you are registered in the DEVELOPERS
> file. Could you verify that those boards still boot with this patch
> applied, please?

The patch looks ok and on a quick test, both aarch64_efi_defconfig and
pc_x86_64_efi_defconfig continue to work as usual. So looks good to
me.

Erico
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> The startup.nsh file is useless to boot EFI payloads. We just need to
> follow the naming detection specified in the UEFI spec.
> The EFI payload need to be placed in the boot/efi folder in the EFI partition
> and follow the architecture naming as described below:
> 32bit : bootia32.efi
> x64 : bootx64.efi
> aarch32 : bootarm.efi
> aarch64 : bootaa64.efi
> 
> This naming is already right in the packages involved (systemd, grub2,
> gummiboot), therefore we just need to drop the generation of the
> startup.nsh file.
> 
> The usage of the startup.nsh in genimage is also dropped to avoid errors in
> the image generation.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  board/aarch64-efi/genimage-efi.cfg   | 3 ---
>  board/intel/galileo/genimage.cfg     | 3 ---
>  board/minnowboard/genimage.cfg       | 3 ---
>  board/pc/genimage-efi.cfg            | 3 ---
>  board/pc/post-build.sh               | 2 +-
>  board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
>  boot/grub2/grub2.mk                  | 8 --------
>  boot/gummiboot/gummiboot.mk          | 2 --
>  package/systemd/systemd.mk           | 2 --
>  9 files changed, 1 insertion(+), 28 deletions(-)
> 
> diff --git a/board/aarch64-efi/genimage-efi.cfg b/board/aarch64-efi/genimage-efi.cfg
> index 34cca1d64e..2daa70c361 100644
> --- a/board/aarch64-efi/genimage-efi.cfg
> +++ b/board/aarch64-efi/genimage-efi.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/board/intel/galileo/genimage.cfg b/board/intel/galileo/genimage.cfg
> index 31add21e67..b930c0deb9 100644
> --- a/board/intel/galileo/genimage.cfg
> +++ b/board/intel/galileo/genimage.cfg
> @@ -1,9 +1,6 @@
>  # Create an image of the efi partition
>  image efi-part.vfat {
>  	vfat {
> -		file startup.nsh {
> -			image = "efi-part/startup.nsh"
> -		}
>  		file EFI {
>  			image = "efi-part/EFI"
>  		}
> diff --git a/board/minnowboard/genimage.cfg b/board/minnowboard/genimage.cfg
> index c5b07179b4..521385aadd 100644
> --- a/board/minnowboard/genimage.cfg
> +++ b/board/minnowboard/genimage.cfg
> @@ -1,9 +1,6 @@
>  # Create an image of the efi partition
>  image efi-part.vfat {
>  	vfat {
> -		file startup.nsh {
> -			image = "efi-part/startup.nsh"
> -		}
>  		file EFI {
>  			image = "efi-part/EFI"
>  		}
> diff --git a/board/pc/genimage-efi.cfg b/board/pc/genimage-efi.cfg
> index ec7e85b06c..34befb7d69 100644
> --- a/board/pc/genimage-efi.cfg
> +++ b/board/pc/genimage-efi.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/board/pc/post-build.sh b/board/pc/post-build.sh
> index ed37b3b87e..db23795e27 100755
> --- a/board/pc/post-build.sh
> +++ b/board/pc/post-build.sh
> @@ -5,7 +5,7 @@ set -e
>  BOARD_DIR=$(dirname "$0")
>  
>  # Detect boot strategy, EFI or BIOS
> -if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
> +if [ -d "$BINARIES_DIR/efi-part/" ]; then
>      cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
>  else
>      cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
> diff --git a/board/qemu/aarch64-sbsa/genimage.cfg b/board/qemu/aarch64-sbsa/genimage.cfg
> index 285b308d90..c75d9fcca1 100644
> --- a/board/qemu/aarch64-sbsa/genimage.cfg
> +++ b/board/qemu/aarch64-sbsa/genimage.cfg
> @@ -1,8 +1,5 @@
>  image efi-part.vfat {
>    vfat {
> -    file startup.nsh {
> -      image = "efi-part/startup.nsh"
> -    }
>      file EFI {
>        image = "efi-part/EFI"
>      }
> diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
> index 52e9199ae9..ad2edf17aa 100644
> --- a/boot/grub2/grub2.mk
> +++ b/boot/grub2/grub2.mk
> @@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
>  	$(GRUB2_IMAGE_INSTALL_ELTORITO)
>  endef
>  
> -ifeq ($(GRUB2_PLATFORM),efi)
> -define GRUB2_EFI_STARTUP_NSH
> -	echo $(notdir $(GRUB2_IMAGE)) > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
> -endef
> -GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
> -endif
> -
>  $(eval $(autotools-package))
>  $(eval $(host-autotools-package))
> diff --git a/boot/gummiboot/gummiboot.mk b/boot/gummiboot/gummiboot.mk
> index 748e87030e..eb1f3da78c 100644
> --- a/boot/gummiboot/gummiboot.mk
> +++ b/boot/gummiboot/gummiboot.mk
> @@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
>  define GUMMIBOOT_INSTALL_IMAGES_CMDS
>  	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
>  		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
> -	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
>  	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
>  		$(BINARIES_DIR)/efi-part/loader/loader.conf
>  	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
> diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
> index 5a76f2f246..c403896759 100644
> --- a/package/systemd/systemd.mk
> +++ b/package/systemd/systemd.mk
> @@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
>  define SYSTEMD_INSTALL_BOOT_FILES
>  	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
>  		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
> -	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
> -		$(BINARIES_DIR)/efi-part/startup.nsh
>  	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
>  		$(BINARIES_DIR)/efi-part/loader/loader.conf
>  	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

[Yann E. MORIN]

Erico, All,

On 2021-09-24 22:28 +0200, Erico Nunes spake thusly:
> On Thu, Sep 23, 2021 at 10:07 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > +Dick, Erico, Gerome, Nicholas, Peter, Romain:
> >
> > This patch touches boards for which you are registered in the DEVELOPERS
> > file. Could you verify that those boards still boot with this patch
> > applied, please?
> The patch looks ok and on a quick test, both aarch64_efi_defconfig and
> pc_x86_64_efi_defconfig continue to work as usual. So looks good to
> me.

Thanks for the feedback; I've made that a Tested-by tag from you when
applying. :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> This patch allows to build Grub2 for different configurations simultaneously.
> To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI
> BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more
> reasonable to just extend the grub2 package into 3 packages.
[--SNIP--]
> diff --git a/Config.in.legacy b/Config.in.legacy
> index 35a11f4dc6..33cee0202b 100644
> --- a/Config.in.legacy
> +++ b/Config.in.legacy
> @@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
>  
>  comment "Legacy options removed in 2021.08"
>  
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES
> +	string "the grub2 builtin modules has been renamed"
> +	help
> +	  This option has been split to separate the builtin modules
> +	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
> +	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.
> +
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
> +	bool
> +	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
> +	select BR2_LEGACY
> +
> +config BR2_TARGET_GRUB2_BUILTIN_CONFIG
> +	string "the grub2 builtin configuration has been renamed"
> +	help
> +	  This option has been split to separate the builtin configuration

    $ make check-package
    Config.in.legacy:186: help text: <tab><2 spaces><62 chars>
    (http://nightly.buildroot.org/#writing-rules-config-in)

[--SNIP--]
> diff --git a/boot/grub2/Config.in b/boot/grub2/Config.in
> index e45133999e..10bba6c5e4 100644
> --- a/boot/grub2/Config.in
> +++ b/boot/grub2/Config.in
> @@ -10,6 +10,13 @@ config BR2_TARGET_GRUB2
>  	bool "grub2"
>  	depends on BR2_TARGET_GRUB2_ARCH_SUPPORTS
>  	depends on BR2_USE_WCHAR
> +	select BR2_TARGET_GRUB2_I386_PC if \
> +		!BR2_TARGET_GRUB2_HAS_PTF && \
> +		(BR2_i386 || BR2_x86_64)
> +	select BR2_TARGET_GRUB2_ARM_UBOOT if \
> +		!BR2_TARGET_GRUB2_HAS_PTF && \
> +		BR2_arm
> +	select BR2_TARGET_GRUB2_ARM64_EFI if BR2_aarch64
>  	help
>  	  GNU GRUB is a Multiboot boot loader. It was derived from
>  	  GRUB, the GRand Unified Bootloader, which was originally
> @@ -25,10 +32,10 @@ config BR2_TARGET_GRUB2
>  
>  	  http://www.gnu.org/software/grub/
>  
> -if BR2_TARGET_GRUB2
> +config BR2_TARGET_GRUB2_HAS_PTF
> +	bool

I've moved this option inside the 'if' conditional, otherwise the
suboptions are not properly indented.

Applied to master with the above two fixed, thanks.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 3/7] fs/iso9660: add support to Grub EFI bootloader in the image

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660
> image.
> For that we need to create EFI System Partition (ESP). The ESP is a vfat
> partition which contain the Grub2 binary at the /EFI/BOOT/ location.
> xorriso command will generate the iso image including the ESP.
> 
> We notice Grub can not read and mount the ESP, therefore we place the Grub
> configuration file in the ISO9660 partition. A Grub2 builtin configuration
> need to be used to tell Grub2 to search automatically its configuration
> file in the ISO9660 partition. Use 'set root=(cd0)' in the configuration
> file passed to BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
[--SNIP--]
> diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
> index 23421a9a5c..921efa1b02 100644
> --- a/fs/iso9660/iso9660.mk
> +++ b/fs/iso9660/iso9660.mk
> @@ -57,7 +57,12 @@ else
>  ROOTFS_ISO9660_TMP_TARGET_DIR = $(TARGET_DIR)
>  endif
>  
> -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2),y)
> +ifeq ($(BR2_REPRODUCIBLE),y)
> +ROOTFS_ISO9660_VFAT_OPTS = --invariant
> +ROOTFS_ISO9660_FIX_TIME = touch $(SOURCE_DATE_EPOCH)

I think you may have missed something about SOURCE_DATE_EPOCH: it is a
UNIX timestamp, as a number of seconds elapsed since 1970-01-01T00:00:00+0000.
Right now, it's around 1632775187.

So, what you wrote would create a file named '1632775187' in the
Buildroot top-level directory... Probably not what is expected...

So I've changed to:

    ifeq ($(BR2_REPRODUCIBLE),y)
    ROOTFS_ISO9660_VFAT_OPTS = --invariant
    ROOTFS_ISO9660_FIX_TIME = touch -d @$(SOURCE_DATE_EPOCH)
    else
    ROOTFS_ISO9660_FIX_TIME = @:
    endif

... and then...

> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),yy)
>  ROOTFS_ISO9660_DEPENDENCIES += grub2
>  ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
>  	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
> @@ -66,6 +71,22 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
>  	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
>  		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
>  endef
> +else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
> +ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
> +ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi
> +ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
> +ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
> +ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
> +	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
> +define ROOTFS_ISO9660_INSTALL_BOOTLOADER
> +	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
> +	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
> +	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
> +	$(ROOTFS_ISO9660_FIX_TIME)
> +	$(HOST_DIR)/sbin/mkfs.vfat $(ROOTFS_ISO9660_VFAT_OPTS) $(ROOTFS_ISO9660_EFI_PARTITION_PATH)

... here:

    $(ROOTFS_ISO9660_FIX_TIME) $(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/*

... and...

> +	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
> +		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/

... here:

    $(ROOTFS_ISO9660_FIX_TIME) $(ROOTFS_ISO9660_EFI_PARTITION_PATH)

Applied to master with the above changes, thanks.

Regards,
Yann E. MORIN.

> +endef
>  else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
>  ROOTFS_ISO9660_DEPENDENCIES += syslinux
>  ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
> @@ -128,9 +149,25 @@ ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_DISABLE_EXTERNAL_INITRD
>  
>  endif # ROOTFS_ISO9660_USE_INITRD
>  
> +ROOTFS_ISO9660_OPTS += \
> +	-J \
> +	-R \
> +	-no-emul-boot
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> +ROOTFS_ISO9660_OPTS += \
> +	-boot-load-size 4 \
> +	-boot-info-table \
> +	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
> +endif
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> +ROOTFS_ISO9660_OPTS += \
> +	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
> +endif
> +
>  define ROOTFS_ISO9660_CMD
> -	$(HOST_DIR)/bin/xorriso -as mkisofs -J -R -b $(ROOTFS_ISO9660_BOOT_IMAGE) \
> -		-no-emul-boot -boot-load-size 4 -boot-info-table \
> +	$(HOST_DIR)/bin/xorriso -as mkisofs \
>  		$(ROOTFS_ISO9660_OPTS) \
>  		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
>  endef
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> Add support for building an hybrid ISO9660 image compatible with legacy
> and UEFI BIOS.
> The option -eltorito-alt-boot need to be used in the xorriso command
> to generate the hybrid image.

So I was wondering why you had to rename the variables, and why you
split the values with -eltorito-alt-boot right in between...

As I understand it from the xorriso command line, -eltorito-alt-boot
basically means "OK, we're done with the previous boot parameters, now
we start a new set of boot parameters", so the order of options *is*
important.

Right?

Furthermore, -eltorito-alt-boot and -no-emul-boot are not conflicting,
because the former is really this separator as discussed above, while
the latter specifies the type of the current boot image.

Right?

(For my information: how many such alternate boot parameters can we
define: is it limited to just two, or can we add more?)

> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
[--SNIP--]
> diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
> index 921efa1b02..9c2535d102 100644
> --- a/fs/iso9660/iso9660.mk
> +++ b/fs/iso9660/iso9660.mk
[--SNIP--]
> @@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
>  
>  ROOTFS_ISO9660_OPTS += \
>  	-J \
> -	-R \
> -	-no-emul-boot
> +	-R
>  
> -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> -ROOTFS_ISO9660_OPTS += \
> +ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \

    ROOTFS_ISO9660_OPTS_BIOS

> +	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
> +	-no-emul-boot \
>  	-boot-load-size 4 \
> -	-boot-info-table \
> -	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
> -endif
> +	-boot-info-table

You are also reordering the remaining options. Is that really necessary?
Or is it for symmetry with the EFI case where the "image" option comes
first?

> -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> -ROOTFS_ISO9660_OPTS += \
> -	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \

    ROOTFS_ISO9660_OPTS_EFI

> +	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
> +	-no-emul-boot
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
> +ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)

Hmmm This is pretty ugly... What about:

    ROOTFS_ISO9660_OPTS += \
        $(ROOTFS_ISO9660_OPTS_BIOS) \
        -eltorito-alt-boot \
        $(ROOTFS_ISO9660_OPTS_EFI)

Also, does BIOS really has to come before EFI? If so, why?

> +else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)

    ROOTFS_ISO9660_OPTS += $(ROOTFS_ISO9660_OPTS_BIOS)

> +else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)

    ROOTFS_ISO9660_OPTS += $(ROOTFS_ISO9660_OPTS_EFI)

>  endif
>  
>  define ROOTFS_ISO9660_CMD
>  	$(HOST_DIR)/bin/xorriso -as mkisofs \
>  		$(ROOTFS_ISO9660_OPTS) \
> +		$(ROOTFS_ISO9660_BOOTLOADER_OPTS) \

And thus you don't need the new variable here.

Regards,
Yann E. MORIN.

>  		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
>  endef
>  
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 3/7] fs/iso9660: add support to Grub EFI bootloader in the image

[Yann E. MORIN]

Köry, All,

On 2021-09-27 22:43 +0200, Yann E. MORIN spake thusly:
> On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660
> > image.
[--SNIP--]
> > -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2),y)
> > +ifeq ($(BR2_REPRODUCIBLE),y)
> > +ROOTFS_ISO9660_VFAT_OPTS = --invariant
> > +ROOTFS_ISO9660_FIX_TIME = touch $(SOURCE_DATE_EPOCH)
[--SNIP--]
> So I've changed to:
>     ifeq ($(BR2_REPRODUCIBLE),y)
>     ROOTFS_ISO9660_VFAT_OPTS = --invariant
>     ROOTFS_ISO9660_FIX_TIME = touch -d @$(SOURCE_DATE_EPOCH)
>     else
>     ROOTFS_ISO9660_FIX_TIME = @:

Arg, I made a mistake: the commands fir filesystems are executed as a
stadnard Makefile rule; instead, they are stashed into a script and
executed by the shell (under fakeroot).

So, we can quiesce a command by prefixing it with '@'.

I've pushed a fix.

Sorry for the mess...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

[Köry Maincent]

Yann,

On Mon, 27 Sep 2021 23:05:14 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > Add support for building an hybrid ISO9660 image compatible with legacy
> > and UEFI BIOS.
> > The option -eltorito-alt-boot need to be used in the xorriso command
> > to generate the hybrid image.  
> 
> So I was wondering why you had to rename the variables, and why you
> split the values with -eltorito-alt-boot right in between...
> 
> As I understand it from the xorriso command line, -eltorito-alt-boot
> basically means "OK, we're done with the previous boot parameters, now
> we start a new set of boot parameters", so the order of options *is*
> important.
> 
> Right?

Right

> 
> Furthermore, -eltorito-alt-boot and -no-emul-boot are not conflicting,
> because the former is really this separator as discussed above, while
> the latter specifies the type of the current boot image.
> 
> Right?

Right

> 
> (For my information: how many such alternate boot parameters can we
> define: is it limited to just two, or can we add more?)

According to xorriso manual we can add up to 32 boot catalog entries.

> 
> > Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> > ---  
> [--SNIP--]
> > diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
> > index 921efa1b02..9c2535d102 100644
> > --- a/fs/iso9660/iso9660.mk
> > +++ b/fs/iso9660/iso9660.mk  
> [--SNIP--]
> > @@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
> >  
> >  ROOTFS_ISO9660_OPTS += \
> >  	-J \
> > -	-R \
> > -	-no-emul-boot
> > +	-R
> >  
> > -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> > -ROOTFS_ISO9660_OPTS += \
> > +ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \  
> 
>     ROOTFS_ISO9660_OPTS_BIOS
> 
> > +	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
> > +	-no-emul-boot \
> >  	-boot-load-size 4 \
> > -	-boot-info-table \
> > -	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
> > -endif
> > +	-boot-info-table  
> 
> You are also reordering the remaining options. Is that really necessary?
> Or is it for symmetry with the EFI case where the "image" option comes
> first?

It was for symmetry.

> 
> > -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> > -ROOTFS_ISO9660_OPTS += \
> > -	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
> > +ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \  
> 
>     ROOTFS_ISO9660_OPTS_EFI
> 
> > +	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
> > +	-no-emul-boot
> > +
> > +ifeq
> > ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
> > +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
> > +ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
> > +ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)  
> 
> Hmmm This is pretty ugly... What about:
> 
>     ROOTFS_ISO9660_OPTS += \
>         $(ROOTFS_ISO9660_OPTS_BIOS) \
>         -eltorito-alt-boot \
>         $(ROOTFS_ISO9660_OPTS_EFI)
> 
> Also, does BIOS really has to come before EFI? If so, why?

It seems BIOS need to be the first boot option. I have not found any
information about it but if I invert BIOS and EFI options it doesn't boot. 

Regards,
Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> Add support for building an hybrid ISO9660 image compatible with legacy
> and UEFI BIOS.
> The option -eltorito-alt-boot need to be used in the xorriso command
> to generate the hybrid image.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>

Applied to master, thanks, with the folowing changes;

  - note about hybrid vs. (iso)hybrid
  - explain -eltorito-alt-boot
  - explain duplication of -no-emul-boot
  - rename the variables
  - note about the BIOS-EFI ordering

Since I had done a few changes in the previous patches, of course I had
to resolve the conflicts when applying this patch... I've tried to be as
carefull as I could, but I may have (again!) let some issues slip
theough...

Could you please review/test that it is till works as expected?

I don't think the remaining patches need rebasing, though, so I hope I
can handle more than one next evening... :-/

Regards,
Yann E. MORIN.

> ---
> 
> Changes in v2:
> - Fix typos
> 
> Changes in v3:
> - Follow changes brougt by modification of the third patch.
> 
>  fs/iso9660/Config.in  | 16 +++++++--------
>  fs/iso9660/iso9660.mk | 45 ++++++++++++++++++++++++++++---------------
>  2 files changed, 36 insertions(+), 25 deletions(-)
> 
> diff --git a/fs/iso9660/Config.in b/fs/iso9660/Config.in
> index f8f577506d..8c4e7badf8 100644
> --- a/fs/iso9660/Config.in
> +++ b/fs/iso9660/Config.in
> @@ -27,6 +27,10 @@ choice
>  config BR2_TARGET_ROOTFS_ISO9660_GRUB2
>  	bool "grub2"
>  	depends on BR2_TARGET_GRUB2
> +	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER \
> +		if BR2_TARGET_GRUB2_I386_PC
> +	select BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER \
> +		if (BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI)
>  	help
>  	  Use Grub 2 as the bootloader for the ISO9660 image. Make
>  	  sure to enable the 'iso9660' module in
> @@ -39,21 +43,15 @@ config BR2_TARGET_ROOTFS_ISO9660_GRUB2
>  config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
>  	bool "isolinux"
>  	depends on BR2_TARGET_SYSLINUX_ISOLINUX
> +	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
>  
>  endchoice
>  
> -choice
> -	prompt "Boot payload"
> -
>  config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
> -	bool "legacy bios"
> -	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
> +	bool
>  
>  config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
> -	bool "UEFI"
> -	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
> -
> -endchoice
> +	bool
>  
>  config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
>  	string "Boot menu config file"
> diff --git a/fs/iso9660/iso9660.mk b/fs/iso9660/iso9660.mk
> index 921efa1b02..9c2535d102 100644
> --- a/fs/iso9660/iso9660.mk
> +++ b/fs/iso9660/iso9660.mk
> @@ -67,18 +67,20 @@ ROOTFS_ISO9660_DEPENDENCIES += grub2
>  ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
>  	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
>  ROOTFS_ISO9660_BOOT_IMAGE = boot/grub/grub-eltorito.img
> -define ROOTFS_ISO9660_INSTALL_BOOTLOADER
> +define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
>  	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
>  		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
>  endef
> -else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
> +endif
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
>  ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
>  ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi
>  ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
>  ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
>  ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
>  	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
> -define ROOTFS_ISO9660_INSTALL_BOOTLOADER
> +define ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI
>  	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
>  	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
>  	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
> @@ -87,12 +89,14 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
>  	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
>  		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/
>  endef
> -else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
> +endif
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
>  ROOTFS_ISO9660_DEPENDENCIES += syslinux
>  ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
>  	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/isolinux.cfg
>  ROOTFS_ISO9660_BOOT_IMAGE = isolinux/isolinux.bin
> -define ROOTFS_ISO9660_INSTALL_BOOTLOADER
> +define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
>  	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/syslinux/* \
>  		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/
>  	$(INSTALL) -D -m 0644 $(HOST_DIR)/share/syslinux/ldlinux.c32 \
> @@ -105,7 +109,8 @@ define ROOTFS_ISO9660_PREPARATION
>  		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
>  	$(SED) "s%__KERNEL_PATH__%/boot/$(LINUX_IMAGE_NAME)%" \
>  		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
> -	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER)
> +	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS)
> +	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI)
>  endef
>  
>  ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_PREPARATION
> @@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
>  
>  ROOTFS_ISO9660_OPTS += \
>  	-J \
> -	-R \
> -	-no-emul-boot
> +	-R
>  
> -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> -ROOTFS_ISO9660_OPTS += \
> +ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \
> +	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
> +	-no-emul-boot \
>  	-boot-load-size 4 \
> -	-boot-info-table \
> -	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
> -endif
> +	-boot-info-table
>  
> -ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> -ROOTFS_ISO9660_OPTS += \
> -	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \
> +	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
> +	-no-emul-boot
> +
> +ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
> +ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
> +else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
> +else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
> +ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
>  endif
>  
>  define ROOTFS_ISO9660_CMD
>  	$(HOST_DIR)/bin/xorriso -as mkisofs \
>  		$(ROOTFS_ISO9660_OPTS) \
> +		$(ROOTFS_ISO9660_BOOTLOADER_OPTS) \
>  		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
>  endef
>  
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

[Köry Maincent]

Hello Yann,

Thanks for the merge.

On Wed, 29 Sep 2021 23:26:14 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > Add support for building an hybrid ISO9660 image compatible with legacy
> > and UEFI BIOS.
> > The option -eltorito-alt-boot need to be used in the xorriso command
> > to generate the hybrid image.
> > 
> > Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>  
> 
> Applied to master, thanks, with the folowing changes;
> 
>   - note about hybrid vs. (iso)hybrid
>   - explain -eltorito-alt-boot
>   - explain duplication of -no-emul-boot
>   - rename the variables
>   - note about the BIOS-EFI ordering
> 
> Since I had done a few changes in the previous patches, of course I had
> to resolve the conflicts when applying this patch... I've tried to be as
> carefull as I could, but I may have (again!) let some issues slip
> theough...
> 
> Could you please review/test that it is till works as expected?

I do not see any issue in your change, the bench of tests (last patch) ends
well.

> 
> I don't think the remaining patches need rebasing, though, so I hope I
> can handle more than one next evening... :-/

They indeed don't need rebasing.

Regards,
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> Add support the build the firmware for QEMU i386 pc machine
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  boot/edk2/Config.in | 12 +++++++++++-
>  boot/edk2/edk2.mk   | 12 ++++++++++--
>  2 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/boot/edk2/Config.in b/boot/edk2/Config.in
> index 150806899f..9f8988bcca 100644
> --- a/boot/edk2/Config.in
> +++ b/boot/edk2/Config.in
> @@ -1,6 +1,6 @@
>  config BR2_TARGET_EDK2
>  	bool "EDK2"
> -	depends on BR2_x86_64 || BR2_aarch64
> +	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386

With three archicteures, it is now nicer to introduce a symbol dedicated
to list the arch support:

    config BR2_TARGET_EDK2_ARCH_SUPPORTS
        bool
        default y if BR2_aarch64
        default y if BR2_i386
        default y if BR2_x86_64

    config BR2_TARGET_EDK2
        bool "edk2"
        depends on BR2_TARGET_EDK2_ARCH_SUPPORTS
        [...]

>  	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
>  	select BR2_PACKAGE_EDK2_PLATFORMS
>  	help
> @@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
>  
>  choice
>  	prompt "Platform"
> +	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
>  	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
>  	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
>  
> +config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
> +	bool "i386"
> +	depends on BR2_i386 || BR2_x86_64

So, I was wondering on the above could work [...]

> +	help
> +	  Platform configuration for a generic i386 target.
> +	  This platform will boot from flash address 0x0.
> +	  It should therefore be used as the first bootloader.
> +
>  config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
>  	bool "x86-64"
>  	depends on BR2_x86_64
> @@ -94,6 +103,7 @@ endchoice
>  
>  config BR2_TARGET_EDK2_FD_NAME
>  	string
> +	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
>  	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
>  	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
>  	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
> diff --git a/boot/edk2/edk2.mk b/boot/edk2/edk2.mk
> index fabd0c5b45..ab3cdad464 100644
> --- a/boot/edk2/edk2.mk
> +++ b/boot/edk2/edk2.mk
> @@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
>  EDK2_INSTALL_TARGET = NO
>  EDK2_INSTALL_IMAGES = YES
>  
> -ifeq ($(BR2_x86_64),y)
> +ifeq ($(BR2_i386),y)
> +EDK2_ARCH = IA32
> +else ifeq ($(BR2_x86_64),y)

[...]
with these settings when the target is actually an x86_64. So I came up
with this minimal defconfig:

    BR2_x86_64=y
    BR2_x86_corei7=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_EDK2=y
    BR2_TARGET_EDK2_PLATFORM_OVMF_I386=y

And indeed, it does not work:

    $ make edk2
    [...]
    >>> edk2 edk2-stable202102 Building
    [...]
    Ran 285 tests in 1.458s

    OK
    Build environment: Linux-5.4.0-84-generic-x86_64-with-glibc2.31
    Build start time: 21:35:13, Sep.30 2021

    WORKSPACE        = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102
    PACKAGES_PATH    = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102:/home/ymorin/dev/buildroot/O>
    EDK_TOOLS_PATH   = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/BaseTools
    CONF_PATH        = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/Conf
    PYTHON_COMMAND   = /home/ymorin/dev/buildroot/O/host/bin/python3

    build.py...
     : error 2000: Invalid parameter
        Invalid ARCH specified. [Valid ARCH: IA32]

    - Failed -
    Build end time: 21:35:13, Sep.30 2021
    Build total time: 00:00:00

    make[1]: *** [package/pkg-generic.mk:294: /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/.stamp_built] Error 1
    make: *** [Makefile:23: _all] Error 2

Setting EDK2_ARCH to IA32 fixes the build, but I have no way to know if
the build is "correct"...

So, in this case, I think the EDK2_ARCH should depend on one of the
BR2_TARGET_EDK2_PLATFORM_OVMF_xxx, not the target architecture.

Alternatively, we could maybe restrict BR2_TARGET_EDK2_PLATFORM_OVMF_I386
to just i386?

Regards,
Yann E. MORIN.

>  EDK2_ARCH = X64
>  else ifeq ($(BR2_aarch64),y)
>  EDK2_ARCH = AARCH64
> @@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
>  EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
>  EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
>  
> -ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
> +ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
> +EDK2_DEPENDENCIES += host-nasm
> +EDK2_PACKAGE_NAME = OvmfPkg
> +EDK2_PLATFORM_NAME = OvmfPkgIa32
> +EDK2_BUILD_DIR = OvmfIa32
> +
> +else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
>  EDK2_DEPENDENCIES += host-nasm
>  EDK2_PACKAGE_NAME = OvmfPkg
>  EDK2_PLATFORM_NAME = OvmfPkgX64
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Yann E. MORIN]

Köry, All,

+Arnout, +Petr for their python3 expertise

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> With UTF-8 got issue with wrong character returned by Qemu when using EFI
> BIOS. This breaks the test process with the following error. Update to
> ISO-8859-1 encoding to avoid it.

I am always skeptical about switching away from UTF-8. Rather, I'd like
to undersand why that is the case.

Is it specific to just the edk2 tests you added later in the series?
What if you set your locale to 'C' (LC_ALL=C LANG=C) when runing the
test-suite? Or to 'C.UTF-8'?

>     emulator.login()
>   File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
>     index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
>     return self.expect_list(compiled_pattern_list,
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
>     return exp.expect_loop(timeout)
>   File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
>     incoming = spawn.read_nonblocking(spawn.maxread, timeout)
>   File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
>     return super(spawn, self).read_nonblocking(size)
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
>     s = self._decoder.decode(s, final=False)
>   File "/usr/lib/python3.8/codecs.py", line 322, in decode
>     (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

0xda in iso8859-15 is Ú (LATIN CAPITAL LETTER U WITH ACUTE). This is a
weird character to see... And his is the first byte... Is edk2 just
spitting actual binary?

So I'd really like to understand why that is... 

Regards,
Yann E. MORIN.

> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  support/testing/infra/emulator.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
> index 0a77eb80fc..1fab9caad8 100644
> --- a/support/testing/infra/emulator.py
> +++ b/support/testing/infra/emulator.py
> @@ -76,7 +76,7 @@ class Emulator(object):
>          self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
>          self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
>                                    timeout=5 * self.timeout_multiplier,
> -                                  encoding='utf-8',
> +                                  encoding='ISO-8859-1',
>                                    env={"QEMU_AUDIO_DRV": "none"})
>          # We want only stdout into the log to avoid double echo
>          self.qemu.logfile_read = self.logfile
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Yann E. MORIN]

Köry, All,

On 2021-09-30 22:28 +0200, Yann E. MORIN spake thusly:
> On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > With UTF-8 got issue with wrong character returned by Qemu when using EFI
> > BIOS. This breaks the test process with the following error. Update to
> > ISO-8859-1 encoding to avoid it.
> I am always skeptical about switching away from UTF-8. Rather, I'd like
> to undersand why that is the case.
[--SNIP--]
> 0xda in iso8859-15 is Ú (LATIN CAPITAL LETTER U WITH ACUTE). This is a
> weird character to see... And his is the first byte... Is edk2 just
> spitting actual binary?

It's not the first byte; and it's not edk2, but grub2:

    00000150  6d 1b 5b 34 30 6d 1b 5b  30 32 3b 33 30 48 47 4e  |m.[40m.[02;30HGN|
    00000160  55 20 47 52 55 42 20 20  76 65 72 73 69 6f 6e 20  |U GRUB  version |
    00000170  32 2e 30 34 0a 0d 0a 0d  1b 5b 30 34 3b 30 32 48  |2.04.....[04;02H|
    00000180  da c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    00000190  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001a0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001b0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001c0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 bf 1b 5b  |...............[|
    000001d0  30 35 3b 30 32 48 b3 1b  5b 30 35 3b 37 39 48 b3  |05;02H..[05;79H.|

So, what is this "da c4...c4 bf" sequence? It looks a bit more obvious
when looking at the folowing lines:

    000001d0  30 35 3b 30 32 48 b3 1b  5b 30 35 3b 37 39 48 b3  |05;02H..[05;79H.|
    000001e0  1b 5b 30 36 3b 30 32 48  b3 1b 5b 30 36 3b 37 39  |.[06;02H..[06;79|
    000001f0  48 b3 1b 5b 30 37 3b 30  32 48 b3 1b 5b 30 37 3b  |H..[07;02H..[07;|
    00000200  37 39 48 b3 1b 5b 30 38  3b 30 32 48 b3 1b 5b 30  |79H..[08;02H..[0|
    00000210  38 3b 37 39 48 b3 1b 5b  30 39 3b 30 32 48 b3 1b  |8;79H..[09;02H..|
    00000220  5b 30 39 3b 37 39 48 b3  1b 5b 31 30 3b 30 32 48  |[09;79H..[10;02H|
    00000230  b3 1b 5b 31 30 3b 37 39  48 b3 1b 5b 31 31 3b 30  |..[10;79H..[11;0|
    00000240  32 48 b3 1b 5b 31 31 3b  37 39 48 b3 1b 5b 31 32  |2H..[11;79H..[12|
    00000250  3b 30 32 48 b3 1b 5b 31  32 3b 37 39 48 b3 1b 5b  |;02H..[12;79H..[|
    00000260  31 33 3b 30 32 48 b3 1b  5b 31 33 3b 37 39 48 b3  |13;02H..[13;79H.|
    00000270  1b 5b 31 34 3b 30 32 48  b3 1b 5b 31 34 3b 37 39  |.[14;02H..[14;79|
    00000280  48 b3 1b 5b 31 35 3b 30  32 48 b3 1b 5b 31 35 3b  |H..[15;02H..[15;|
    00000290  37 39 48 b3 1b 5b 31 36  3b 30 32 48 b3 1b 5b 31  |79H..[16;02H..[1|
    000002a0  36 3b 37 39 48 b3 1b 5b  31 37 3b 30 32 48 b3 1b  |6;79H..[17;02H..|
    000002b0  5b 31 37 3b 37 39 48 b3  1b 5b 31 38 3b 30 32 48  |[17;79H..[18;02H|
    000002c0  c0 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002d0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002e0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002f0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    00000300  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 d9 1b 5b  |...............[|
    00000310  31 39 3b 30 32 48 1b 5b  32 30 3b 30 32 48 20 20  |19;02H.[20;02H  |

OK, so there are a lot of ANSI Escape Sequences, which are CSI commands:

    1b 5b 30 35 3b 37 39 48     - ESC [ 05 ; 79 H

This is CSI for "Cursor Position", i.e. move sursor to row 5, column 79.
OK, it is drawing a box! da is top-left corner, c4 is horizontal line,
bf is top-right corner, b3 is vertical line, c0 id lower-left corner,
and d9 is lower-right corner.

This is definitely not ISO-8859-15; it is CP437 [0]:
    https://en.wikipedia.org/wiki/Code_page_437

So, the solution to switch over to iso-8859-15 is not the proper
solution. We don't want to switch to CP437 either!

What I suggest, instead, is that we tell pexpect.spawn to just fix any
non-decodable character, by passing the parameter:
    codec_errors='replace'

At least, it makes the two EFI run-test succeed!

If that is OK, I can do that when applying, and fix the commit log
accordingly.

Regards,
Yann E. MORIN.

[0] Oh boy, does that bring fond memories... ;-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Thomas Petazzoni]

On Sat, 2 Oct 2021 22:28:05 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> OK, so there are a lot of ANSI Escape Sequences, which are CSI commands:
> 
>     1b 5b 30 35 3b 37 39 48     - ESC [ 05 ; 79 H
> 
> This is CSI for "Cursor Position", i.e. move sursor to row 5, column 79.
> OK, it is drawing a box! da is top-left corner, c4 is horizontal line,
> bf is top-right corner, b3 is vertical line, c0 id lower-left corner,
> and d9 is lower-right corner.
> 
> This is definitely not ISO-8859-15; it is CP437 [0]:
>     https://en.wikipedia.org/wiki/Code_page_437
> 
> So, the solution to switch over to iso-8859-15 is not the proper
> solution. We don't want to switch to CP437 either!

Wow, thanks for the investigation!

> What I suggest, instead, is that we tell pexpect.spawn to just fix any
> non-decodable character, by passing the parameter:
>     codec_errors='replace'
> 
> At least, it makes the two EFI run-test succeed!
> 
> If that is OK, I can do that when applying, and fix the commit log
> accordingly.

This indeed looks like a better option.

Thanks again!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> With UTF-8 got issue with wrong character returned by Qemu when using EFI
> BIOS. This breaks the test process with the following error. Update to
> ISO-8859-1 encoding to avoid it.
> 
>     emulator.login()
>   File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
>     index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
>     return self.expect_list(compiled_pattern_list,
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
>     return exp.expect_loop(timeout)
>   File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
>     incoming = spawn.read_nonblocking(spawn.maxread, timeout)
>   File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
>     return super(spawn, self).read_nonblocking(size)
>   File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
>     s = self._decoder.decode(s, final=False)
>   File "/usr/lib/python3.8/codecs.py", line 322, in decode
>     (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  support/testing/infra/emulator.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
> index 0a77eb80fc..1fab9caad8 100644
> --- a/support/testing/infra/emulator.py
> +++ b/support/testing/infra/emulator.py
> @@ -76,7 +76,7 @@ class Emulator(object):
>          self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
>          self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
>                                    timeout=5 * self.timeout_multiplier,
> -                                  encoding='utf-8',
> +                                  encoding='ISO-8859-1',

After review from Thomas, I've indeed reverted to using utf-8, but using
codecerrors='replace' to fix invalid utf-8 sequences.

I've also reworded the commit log accordingly.

Applied to master, thanks.

Regards,
Yann E. MORIN.

>                                    env={"QEMU_AUDIO_DRV": "none"})
>          # We want only stdout into the log to avoid double echo
>          self.qemu.logfile_read = self.logfile
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> Add support the build the firmware for QEMU i386 pc machine
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  boot/edk2/Config.in | 12 +++++++++++-
>  boot/edk2/edk2.mk   | 12 ++++++++++--
>  2 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/boot/edk2/Config.in b/boot/edk2/Config.in
> index 150806899f..9f8988bcca 100644
> --- a/boot/edk2/Config.in
> +++ b/boot/edk2/Config.in
> @@ -1,6 +1,6 @@
>  config BR2_TARGET_EDK2
>  	bool "EDK2"
> -	depends on BR2_x86_64 || BR2_aarch64
> +	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386
>  	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
>  	select BR2_PACKAGE_EDK2_PLATFORMS
>  	help
> @@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
>  
>  choice
>  	prompt "Platform"
> +	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
>  	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
>  	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
>  
> +config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
> +	bool "i386"
> +	depends on BR2_i386 || BR2_x86_64

As there was a build failure with BR2_x86_64=y, I've just dropped that
possibility: BR2_TARGET_EDK2_PLATFORM_OVMF_I386 is only available for
BR2_i386.

If you have the opportunity/need to look into that, a further patch is
most welcome!

Applied to master, thanks.

Regards,
Yann E. MORIN.

> +	help
> +	  Platform configuration for a generic i386 target.
> +	  This platform will boot from flash address 0x0.
> +	  It should therefore be used as the first bootloader.
> +
>  config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
>  	bool "x86-64"
>  	depends on BR2_x86_64
> @@ -94,6 +103,7 @@ endchoice
>  
>  config BR2_TARGET_EDK2_FD_NAME
>  	string
> +	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
>  	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
>  	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
>  	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
> diff --git a/boot/edk2/edk2.mk b/boot/edk2/edk2.mk
> index fabd0c5b45..ab3cdad464 100644
> --- a/boot/edk2/edk2.mk
> +++ b/boot/edk2/edk2.mk
> @@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
>  EDK2_INSTALL_TARGET = NO
>  EDK2_INSTALL_IMAGES = YES
>  
> -ifeq ($(BR2_x86_64),y)
> +ifeq ($(BR2_i386),y)
> +EDK2_ARCH = IA32
> +else ifeq ($(BR2_x86_64),y)
>  EDK2_ARCH = X64
>  else ifeq ($(BR2_aarch64),y)
>  EDK2_ARCH = AARCH64
> @@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
>  EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
>  EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
>  
> -ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
> +ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
> +EDK2_DEPENDENCIES += host-nasm
> +EDK2_PACKAGE_NAME = OvmfPkg
> +EDK2_PLATFORM_NAME = OvmfPkgIa32
> +EDK2_BUILD_DIR = OvmfIa32
> +
> +else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
>  EDK2_DEPENDENCIES += host-nasm
>  EDK2_PACKAGE_NAME = OvmfPkg
>  EDK2_PLATFORM_NAME = OvmfPkgX64
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 7/7] support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> The ISO9660 tests are only testing BIOS Legacy.
> Add support to test an ISO9660 image based on EFI BIOS.
> Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS.
> Add dedicated Grub2 builtin config for the EFI compatible cases.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
>  support/testing/conf/grub2-efi.cfg       |  2 +
>  support/testing/tests/fs/test_iso9660.py | 74 +++++++++++++++++++++++-
>  2 files changed, 74 insertions(+), 2 deletions(-)
>  create mode 100644 support/testing/conf/grub2-efi.cfg
> 
> diff --git a/support/testing/conf/grub2-efi.cfg b/support/testing/conf/grub2-efi.cfg
> new file mode 100644
> index 0000000000..11c32e880e
> --- /dev/null
> +++ b/support/testing/conf/grub2-efi.cfg
> @@ -0,0 +1,2 @@
> +set root=(cd0)
> +set prefix=/boot/grub
> diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
> index 1b699e1fef..8315442604 100644
> --- a/support/testing/tests/fs/test_iso9660.py
> +++ b/support/testing/tests/fs/test_iso9660.py
> @@ -25,9 +25,13 @@ BASIC_CONFIG = \
>      """.format(infra.filepath("conf/minimal-x86-qemu-kernel.config"))
>  
>  
> -def test_mount_internal_external(emulator, builddir, internal=True):
> +def test_mount_internal_external(emulator, builddir, internal=True, efi=False):
>      img = os.path.join(builddir, "images", "rootfs.iso9660")
> -    emulator.boot(arch="i386", options=["-cdrom", img])
> +    if efi:
> +        efi_img = os.path.join(builddir, "images", "OVMF.fd")
> +        emulator.boot(arch="i386", options=["-cdrom", img, "-bios", efi_img])
> +    else:
> +        emulator.boot(arch="i386", options=["-cdrom", img])
>      emulator.login()
>  
>      if internal:
> @@ -53,6 +57,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
>          BR2_TARGET_ROOTFS_ISO9660=y
>          # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
>          BR2_TARGET_GRUB2=y
> +        BR2_TARGET_GRUB2_I386_PC=y

This is no longer needed now that we ensure that it is the default
setting. Ditto for the following two, of course.

Applied to master, thanks.

Regards,
Yann E. MORIN.

>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
>          BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
> @@ -74,6 +79,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
>          # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
>          BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
>          BR2_TARGET_GRUB2=y
> +        BR2_TARGET_GRUB2_I386_PC=y
>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
>          BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
> @@ -94,6 +100,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
>          BR2_TARGET_ROOTFS_ISO9660=y
>          BR2_TARGET_ROOTFS_ISO9660_INITRD=y
>          BR2_TARGET_GRUB2=y
> +        BR2_TARGET_GRUB2_I386_PC=y
>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
>          BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
> @@ -107,6 +114,69 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
>          exit_code = test_touch_file(self.emulator)
>          self.assertEqual(exit_code, 0)
>  
> +
> +class TestIso9660Grub2EFI(infra.basetest.BRTest):
> +    config = BASIC_CONFIG + \
> +        """
> +        BR2_TARGET_ROOTFS_ISO9660=y
> +        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
> +        BR2_TARGET_GRUB2=y
> +        BR2_TARGET_GRUB2_I386_EFI=y
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat part_msdos part_gpt normal iso9660"
> +        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
> +        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
> +        BR2_TARGET_EDK2=y
> +        """.format(infra.filepath("conf/grub2-efi.cfg"),
> +                   infra.filepath("conf/grub2.cfg"))
> +
> +    def test_run(self):
> +        exit_code = test_mount_internal_external(self.emulator,
> +                                                 self.builddir, internal=True,
> +                                                 efi=True)
> +        self.assertEqual(exit_code, 0)
> +
> +        exit_code = test_touch_file(self.emulator)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestIso9660Grub2Hybrid(infra.basetest.BRTest):
> +    config = BASIC_CONFIG + \
> +        """
> +        BR2_TARGET_ROOTFS_ISO9660=y
> +        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
> +        BR2_TARGET_GRUB2=y
> +        BR2_TARGET_GRUB2_I386_PC=y
> +        BR2_TARGET_GRUB2_I386_EFI=y
> +        BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 biosdisk"
> +        BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC=""
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 efi_gop"
> +        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
> +        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
> +        BR2_TARGET_EDK2=y
> +        """.format(infra.filepath("conf/grub2-efi.cfg"),
> +                   infra.filepath("conf/grub2.cfg"))
> +
> +    def test_run(self):
> +        exit_code = test_mount_internal_external(self.emulator,
> +                                                 self.builddir, internal=True,
> +                                                 efi=False)
> +        self.assertEqual(exit_code, 0)
> +
> +        exit_code = test_touch_file(self.emulator)
> +        self.assertEqual(exit_code, 0)
> +
> +        self.emulator.stop()
> +
> +        exit_code = test_mount_internal_external(self.emulator,
> +                                                 self.builddir, internal=True,
> +                                                 efi=True)
> +        self.assertEqual(exit_code, 0)
> +
> +        exit_code = test_touch_file(self.emulator)
> +        self.assertEqual(exit_code, 0)
> +
> +
>  #
>  # Syslinux
>  
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Köry Maincent]

Hello Yann,

On Sun, 3 Oct 2021 14:47:19 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> After review from Thomas, I've indeed reverted to using utf-8, but using
> codecerrors='replace' to fix invalid utf-8 sequences.
> 
> I've also reworded the commit log accordingly.
> 
> Applied to master, thanks.

Thanks for this investigation on the utf8 character issue and for completing the
merging process :)

Regards,
Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

[Köry Maincent]

Yann, 

On Sun, 3 Oct 2021 14:49:00 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > +config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
> > +	bool "i386"
> > +	depends on BR2_i386 || BR2_x86_64  
> 
> As there was a build failure with BR2_x86_64=y, I've just dropped that
> possibility: BR2_TARGET_EDK2_PLATFORM_OVMF_I386 is only available for
> BR2_i386.
> 
> If you have the opportunity/need to look into that, a further patch is
> most welcome!

Found! The issue is that EDK2_ARCH is configured for X64 even when we select
the BR2_TARGET_EDK2_PLATFORM_OVMF_I386 configuration.
> >  else ifeq ($(BR2_x86_64),y)
> >  EDK2_ARCH = X64

I will send a patch to fix this.

Regards,

Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > Köry, All,
 > On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
 >> With UTF-8 got issue with wrong character returned by Qemu when using EFI
 >> BIOS. This breaks the test process with the following error. Update to
 >> ISO-8859-1 encoding to avoid it.
 >> 
 >> emulator.login()
 >> File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
 >> index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
 >> File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
 >> return self.expect_list(compiled_pattern_list,
 >> File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
 >> return exp.expect_loop(timeout)
 >> File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
 >> incoming = spawn.read_nonblocking(spawn.maxread, timeout)
 >> File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
 >> return super(spawn, self).read_nonblocking(size)
 >> File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
 >> s = self._decoder.decode(s, final=False)
 >> File "/usr/lib/python3.8/codecs.py", line 322, in decode
 >> (result, consumed) = self._buffer_decode(data, self.errors, final)
 >> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
 >> 
 >> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
 >> ---
 >> support/testing/infra/emulator.py | 2 +-
 >> 1 file changed, 1 insertion(+), 1 deletion(-)
 >> 
 >> diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
 >> index 0a77eb80fc..1fab9caad8 100644
 >> --- a/support/testing/infra/emulator.py
 >> +++ b/support/testing/infra/emulator.py
 >> @@ -76,7 +76,7 @@ class Emulator(object):
 >> self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
 >> self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
 >> timeout=5 * self.timeout_multiplier,
 >> -                                  encoding='utf-8',
 >> +                                  encoding='ISO-8859-1',

 > After review from Thomas, I've indeed reverted to using utf-8, but using
 > codecerrors='replace' to fix invalid utf-8 sequences.

 > I've also reworded the commit log accordingly.

 > Applied to master, thanks.

Committed to 2021.02.x, 2021.05.x and 2021.08.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
[--SNIP--]
> We can no longer use autotools-package as a consequence of this multi-build, and
> we have to resort to generic-package and a partial duplication of
> the autotools-infra. Grub2 was already using custom option like --prefix or
> --exec-prefix so this won't add much more weirdness.

But this had an unfortunate side effect: the grub2 tools, like
grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no
longer installed in the target now.

Indeed, it worked so far because there is a default target install
command for autotools packages, which does basivally call
    make install DESTDIR=$(TARGET_DIR)

But generic-package has no default install command, so declaring that
the package installs in target does nothing.

I see two options:

 1. provide a custom command for one of the enabled platforms; e.g.
    arbitrarily choose the first in the tuple and install in target
    from that one tuple

 2. revert to using an autotools package, but build for no platform at
    all and just install the tools. For the platforms, move the current
    foreach loops into post-build and post-install-image hooks

Care to look into that, please?

Regards,
Yann E. MORIN.

> We use a GRUB2_TUPLES list to describe all the configurations selected.
> For each boot case described in the GRUB2_TUPLES list, it configures and
> builds Grub2 in a separate folder named build-$(tuple).
> We use a foreach loop to make actions on each tuple selected.
> 
> We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the
> BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.
> 
> Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> ---
> 
> Changes in v2:
> Following the review from Yann E. Morin:
> - Update the tuples configurations to makes the code more readable.
> - Move "--target" and "--with-platform" parameters into the
>   GRUB2_CONFIGURE_CMD.
> - Fix startup.nsh installation.
> - Fix Grub2 configuration to be sure that at least one platform is always enabled.
> - Fix legacy builtin configurations to string in place of bool.
> 
> Changes in v3:
> Following the review from Yann E. Morin:
> - Update the tuples configurations to remove the conditions and makes it
>   more readable.
> - Update the tuples configurations for builtin config and modules options.
> - Fix typo.
> - Remove the generation of startup.nsh.
> 
>  Config.in.legacy                         |  24 ++++
>  boot/grub2/Config.in                     |  53 +++++--
>  boot/grub2/grub2.mk                      | 175 +++++++++++++----------
>  support/testing/tests/fs/test_iso9660.py |   6 +-
>  4 files changed, 169 insertions(+), 89 deletions(-)
> 
> diff --git a/Config.in.legacy b/Config.in.legacy
> index 35a11f4dc6..33cee0202b 100644
> --- a/Config.in.legacy
> +++ b/Config.in.legacy
> @@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
>  
>  comment "Legacy options removed in 2021.08"
>  
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES
> +	string "the grub2 builtin modules has been renamed"
> +	help
> +	  This option has been split to separate the builtin modules
> +	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
> +	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.
> +
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
> +	bool
> +	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
> +	select BR2_LEGACY
> +
> +config BR2_TARGET_GRUB2_BUILTIN_CONFIG
> +	string "the grub2 builtin configuration has been renamed"
> +	help
> +	  This option has been split to separate the builtin configuration
> +	  between BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC and
> +	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.
> +
> +config BR2_TARGET_GRUB2_BUILTIN_CONFIG_WRAP
> +	bool
> +	default y if BR2_TARGET_GRUB2_BUILTIN_CONFIG != ""
> +	select BR2_LEGACY
> +
>  config BR2_PACKAGE_LIBMCRYPT
>  	bool "libmcrypt package was removed"
>  	select BR2_LEGACY
> diff --git a/boot/grub2/Config.in b/boot/grub2/Config.in
> index e45133999e..10bba6c5e4 100644
> --- a/boot/grub2/Config.in
> +++ b/boot/grub2/Config.in
> @@ -10,6 +10,13 @@ config BR2_TARGET_GRUB2
>  	bool "grub2"
>  	depends on BR2_TARGET_GRUB2_ARCH_SUPPORTS
>  	depends on BR2_USE_WCHAR
> +	select BR2_TARGET_GRUB2_I386_PC if \
> +		!BR2_TARGET_GRUB2_HAS_PTF && \
> +		(BR2_i386 || BR2_x86_64)
> +	select BR2_TARGET_GRUB2_ARM_UBOOT if \
> +		!BR2_TARGET_GRUB2_HAS_PTF && \
> +		BR2_arm
> +	select BR2_TARGET_GRUB2_ARM64_EFI if BR2_aarch64
>  	help
>  	  GNU GRUB is a Multiboot boot loader. It was derived from
>  	  GRUB, the GRand Unified Bootloader, which was originally
> @@ -25,10 +32,10 @@ config BR2_TARGET_GRUB2
>  
>  	  http://www.gnu.org/software/grub/
>  
> -if BR2_TARGET_GRUB2
> +config BR2_TARGET_GRUB2_HAS_PTF
> +	bool
>  
> -choice
> -	prompt "Platform"
> +if BR2_TARGET_GRUB2
>  
>  config BR2_TARGET_GRUB2_I386_PC
>  	bool "i386-pc"
> @@ -40,6 +47,7 @@ config BR2_TARGET_GRUB2_I386_PC
>  config BR2_TARGET_GRUB2_I386_EFI
>  	bool "i386-efi"
>  	depends on BR2_i386 || BR2_x86_64
> +	select BR2_TARGET_GRUB2_HAS_PTF
>  	help
>  	  Select this option if the platform you're targetting has a
>  	  32 bits EFI BIOS. Note that some x86-64 platforms use a 32
> @@ -48,6 +56,7 @@ config BR2_TARGET_GRUB2_I386_EFI
>  config BR2_TARGET_GRUB2_X86_64_EFI
>  	bool "x86-64-efi"
>  	depends on BR2_x86_64
> +	select BR2_TARGET_GRUB2_HAS_PTF
>  	help
>  	  Select this option if the platform you're targetting has a
>  	  64 bits EFI BIOS.
> @@ -63,6 +72,7 @@ config BR2_TARGET_GRUB2_ARM_UBOOT
>  config BR2_TARGET_GRUB2_ARM_EFI
>  	bool "arm-efi"
>  	depends on BR2_arm
> +	select BR2_TARGET_GRUB2_HAS_PTF
>  	help
>  	  Select this option if the platform you're targetting is an
>  	  ARM platform and you want to boot Grub 2 as an EFI
> @@ -76,10 +86,10 @@ config BR2_TARGET_GRUB2_ARM64_EFI
>  	  Aarch64 platform and you want to boot Grub 2 as an EFI
>  	  application.
>  
> -endchoice
> -
>  if BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
>  
> +comment "Options for the x86 legacy BIOS or ARM U-Boot support"
> +
>  config BR2_TARGET_GRUB2_BOOT_PARTITION
>  	string "boot partition"
>  	default "hd0,msdos1"
> @@ -89,24 +99,43 @@ config BR2_TARGET_GRUB2_BOOT_PARTITION
>  	  first disk if using a legacy partition table, or 'hd0,gpt1'
>  	  if using GPT partition table.
>  
> -endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
> -
> -config BR2_TARGET_GRUB2_BUILTIN_MODULES
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES_PC
>  	string "builtin modules"
> +	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
>  	default "boot linux ext2 fat squash4 part_msdos part_gpt normal biosdisk" if BR2_TARGET_GRUB2_I386_PC
> -	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop" \
> -		if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
> -		   BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
>  	default "linux ext2 fat part_msdos normal" if BR2_TARGET_GRUB2_ARM_UBOOT
>  
> -config BR2_TARGET_GRUB2_BUILTIN_CONFIG
> +config BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC
> +	string "builtin config"
> +	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
> +	help
> +	  Path to a Grub 2 configuration file that will be embedded
> +	  into the Grub image itself. This allows to set the root
> +	  device and other configuration parameters, but however menu
> +	  entries cannot be described in this embedded configuration.
> +
> +endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
> +
> +if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
> +	BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
> +
> +comment "Options for the EFI BIOS or ARM EFI support"
> +
> +config BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI
> +	string "builtin modules"
> +	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
> +	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop"
> +
> +config BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI
>  	string "builtin config"
> +	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
>  	help
>  	  Path to a Grub 2 configuration file that will be embedded
>  	  into the Grub image itself. This allows to set the root
>  	  device and other configuration parameters, but however menu
>  	  entries cannot be described in this embedded configuration.
>  
> +endif
>  config BR2_TARGET_GRUB2_INSTALL_TOOLS
>  	bool "install tools"
>  	help
> diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
> index ad2edf17aa..e01ebb2edb 100644
> --- a/boot/grub2/grub2.mk
> +++ b/boot/grub2/grub2.mk
> @@ -57,53 +57,65 @@ GRUB2_INSTALL_TARGET = NO
>  endif
>  GRUB2_CPE_ID_VENDOR = gnu
>  
> -GRUB2_BUILTIN_MODULES = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES))
> -GRUB2_BUILTIN_CONFIG = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG))
> +GRUB2_BUILTIN_MODULES_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC))
> +GRUB2_BUILTIN_MODULES_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI))
> +GRUB2_BUILTIN_CONFIG_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC))
> +GRUB2_BUILTIN_CONFIG_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI))
>  GRUB2_BOOT_PARTITION = $(call qstrip,$(BR2_TARGET_GRUB2_BOOT_PARTITION))
>  
> -ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/grub.img
> -GRUB2_CFG = $(TARGET_DIR)/boot/grub/grub.cfg
> -GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
> -GRUB2_TUPLE = i386-pc
> -GRUB2_TARGET = i386
> -GRUB2_PLATFORM = pc
> -else ifeq ($(BR2_TARGET_GRUB2_I386_EFI),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
> -GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> -GRUB2_PREFIX = /EFI/BOOT
> -GRUB2_TUPLE = i386-efi
> -GRUB2_TARGET = i386
> -GRUB2_PLATFORM = efi
> -else ifeq ($(BR2_TARGET_GRUB2_X86_64_EFI),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
> -GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> -GRUB2_PREFIX = /EFI/BOOT
> -GRUB2_TUPLE = x86_64-efi
> -GRUB2_TARGET = x86_64
> -GRUB2_PLATFORM = efi
> -else ifeq ($(BR2_TARGET_GRUB2_ARM_UBOOT),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/boot-part/grub/grub.img
> -GRUB2_CFG = $(BINARIES_DIR)/boot-part/grub/grub.cfg
> -GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
> -GRUB2_TUPLE = arm-uboot
> -GRUB2_TARGET = arm
> -GRUB2_PLATFORM = uboot
> -else ifeq ($(BR2_TARGET_GRUB2_ARM_EFI),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
> -GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> -GRUB2_PREFIX = /EFI/BOOT
> -GRUB2_TUPLE = arm-efi
> -GRUB2_TARGET = arm
> -GRUB2_PLATFORM = efi
> -else ifeq ($(BR2_TARGET_GRUB2_ARM64_EFI),y)
> -GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
> -GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> -GRUB2_PREFIX = /EFI/BOOT
> -GRUB2_TUPLE = arm64-efi
> -GRUB2_TARGET = aarch64
> -GRUB2_PLATFORM = efi
> -endif
> +GRUB2_IMAGE_i386-pc = $(BINARIES_DIR)/grub.img
> +GRUB2_CFG_i386-pc = $(TARGET_DIR)/boot/grub/grub.cfg
> +GRUB2_PREFIX_i386-pc = ($(GRUB2_BOOT_PARTITION))/boot/grub
> +GRUB2_TARGET_i386-pc = i386
> +GRUB2_PLATFORM_i386-pc = pc
> +GRUB2_BUILTIN_CONFIG_i386-pc = $(GRUB2_BUILTIN_CONFIG_PC)
> +GRUB2_BUILTIN_MODULES_i386-pc = $(GRUB2_BUILTIN_MODULES_PC)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_PC) += i386-pc
> +
> +GRUB2_IMAGE_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
> +GRUB2_CFG_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> +GRUB2_PREFIX_i386-efi = /EFI/BOOT
> +GRUB2_TARGET_i386-efi = i386
> +GRUB2_PLATFORM_i386-efi = efi
> +GRUB2_BUILTIN_CONFIG_i386-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
> +GRUB2_BUILTIN_MODULES_i386-efi = $(GRUB2_BUILTIN_MODULES_EFI)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_EFI) += i386-efi
> +
> +GRUB2_IMAGE_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
> +GRUB2_CFG_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> +GRUB2_PREFIX_x86_64-efi = /EFI/BOOT
> +GRUB2_TARGET_x86_64-efi = x86_64
> +GRUB2_PLATFORM_x86_64-efi = efi
> +GRUB2_BUILTIN_CONFIG_x86_64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
> +GRUB2_BUILTIN_MODULES_x86_64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_X86_64_EFI) += x86_64-efi
> +
> +GRUB2_IMAGE_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.img
> +GRUB2_CFG_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.cfg
> +GRUB2_PREFIX_arm-uboot = ($(GRUB2_BOOT_PARTITION))/boot/grub
> +GRUB2_TARGET_arm-uboot = arm
> +GRUB2_PLATFORM_arm-uboot = uboot
> +GRUB2_BUILTIN_CONFIG_arm-uboot = $(GRUB2_BUILTIN_CONFIG_PC)
> +GRUB2_BUILTIN_MODULES_arm-uboot = $(GRUB2_BUILTIN_MODULES_PC)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_UBOOT) += arm-uboot
> +
> +GRUB2_IMAGE_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
> +GRUB2_CFG_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> +GRUB2_PREFIX_arm-efi = /EFI/BOOT
> +GRUB2_TARGET_arm-efi = arm
> +GRUB2_PLATFORM_arm-efi = efi
> +GRUB2_BUILTIN_CONFIG_arm-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
> +GRUB2_BUILTIN_MODULES_arm-efi = $(GRUB2_BUILTIN_MODULES_EFI)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_EFI) += arm-efi
> +
> +GRUB2_IMAGE_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
> +GRUB2_CFG_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
> +GRUB2_PREFIX_arm64-efi = /EFI/BOOT
> +GRUB2_TARGET_arm64-efi = aarch64
> +GRUB2_PLATFORM_arm64-efi = efi
> +GRUB2_BUILTIN_CONFIG_arm64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
> +GRUB2_BUILTIN_MODULES_arm64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
> +GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi
>  
>  # Grub2 is kind of special: it considers CC, LD and so on to be the
>  # tools to build the host programs and uses TARGET_CC, TARGET_CFLAGS,
> @@ -127,18 +139,6 @@ GRUB2_CONF_ENV = \
>  	TARGET_OBJCOPY="$(TARGET_OBJCOPY)" \
>  	TARGET_STRIP="$(TARGET_CROSS)strip"
>  
> -GRUB2_CONF_OPTS = \
> -	--target=$(GRUB2_TARGET) \
> -	--with-platform=$(GRUB2_PLATFORM) \
> -	--prefix=/ \
> -	--exec-prefix=/ \
> -	--disable-grub-mkfont \
> -	--enable-efiemu=no \
> -	ac_cv_lib_lzma_lzma_code=no \
> -	--enable-device-mapper=no \
> -	--enable-libzfs=no \
> -	--disable-werror
> -
>  HOST_GRUB2_CONF_OPTS = \
>  	--disable-grub-mkfont \
>  	--enable-efiemu=no \
> @@ -147,26 +147,53 @@ HOST_GRUB2_CONF_OPTS = \
>  	--enable-libzfs=no \
>  	--disable-werror
>  
> -ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
> -define GRUB2_IMAGE_INSTALL_ELTORITO
> -	cat $(HOST_DIR)/lib/grub/$(GRUB2_TUPLE)/cdboot.img $(GRUB2_IMAGE) > \
> -		$(BINARIES_DIR)/grub-eltorito.img
> +define GRUB2_CONFIGURE_CMDS
> +	$(foreach tuple, $(GRUB2_TUPLES-y), \
> +		mkdir -p $(@D)/build-$(tuple) ; \
> +		cd $(@D)/build-$(tuple) ; \
> +		$(TARGET_CONFIGURE_OPTS) \
> +		$(TARGET_CONFIGURE_ARGS) \
> +		$(GRUB2_CONF_ENV) \
> +		../configure \
> +			--target=$(GRUB2_TARGET_$(tuple)) \
> +			--with-platform=$(GRUB2_PLATFORM_$(tuple)) \
> +			--host=$(GNU_TARGET_NAME) \
> +			--build=$(GNU_HOST_NAME) \
> +			--prefix=/ \
> +			--exec-prefix=/ \
> +			--disable-grub-mkfont \
> +			--enable-efiemu=no \
> +			ac_cv_lib_lzma_lzma_code=no \
> +			--enable-device-mapper=no \
> +			--enable-libzfs=no \
> +			--disable-werror
> +	)
> +endef
> +
> +define GRUB2_BUILD_CMDS
> +	$(foreach tuple, $(GRUB2_TUPLES-y), \
> +		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
> +	)
>  endef
> -endif
>  
>  define GRUB2_INSTALL_IMAGES_CMDS
> -	mkdir -p $(dir $(GRUB2_IMAGE))
> -	$(HOST_DIR)/usr/bin/grub-mkimage \
> -		-d $(@D)/grub-core/ \
> -		-O $(GRUB2_TUPLE) \
> -		-o $(GRUB2_IMAGE) \
> -		-p "$(GRUB2_PREFIX)" \
> -		$(if $(GRUB2_BUILTIN_CONFIG),-c $(GRUB2_BUILTIN_CONFIG)) \
> -		$(GRUB2_BUILTIN_MODULES)
> -	mkdir -p $(dir $(GRUB2_CFG))
> -	$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG)
> -	$(GRUB2_IMAGE_INSTALL_ELTORITO)
> +	$(foreach tuple, $(GRUB2_TUPLES-y), \
> +		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
> +		$(HOST_DIR)/usr/bin/grub-mkimage \
> +			-d $(@D)/build-$(tuple)/grub-core/ \
> +			-O $(tuple) \
> +			-o $(GRUB2_IMAGE_$(tuple)) \
> +			-p "$(GRUB2_PREFIX_$(tuple))" \
> +			$(if $(GRUB2_BUILTIN_CONFIG_$(tuple)), \
> +				-c $(GRUB2_BUILTIN_CONFIG_$(tuple))) \
> +			$(GRUB2_BUILTIN_MODULES_$(tuple)) ; \
> +		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple)) ; \
> +		$(if $(findstring $(GRUB2_PLATFORM_$(tuple)), pc), \
> +			cat $(HOST_DIR)/lib/grub/$(tuple)/cdboot.img $(GRUB2_IMAGE_$(tuple)) > \
> +				$(BINARIES_DIR)/grub-eltorito.img ; \
> +		) \
> +	)
>  endef
>  
> -$(eval $(autotools-package))
> +$(eval $(generic-package))
>  $(eval $(host-autotools-package))
> diff --git a/support/testing/tests/fs/test_iso9660.py b/support/testing/tests/fs/test_iso9660.py
> index 68f4840852..1b699e1fef 100644
> --- a/support/testing/tests/fs/test_iso9660.py
> +++ b/support/testing/tests/fs/test_iso9660.py
> @@ -54,7 +54,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
>          # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
>          BR2_TARGET_GRUB2=y
>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
> -        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
>          """.format(infra.filepath("conf/grub2.cfg"))
>  
> @@ -75,7 +75,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
>          BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
>          BR2_TARGET_GRUB2=y
>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
> -        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
>          """.format(infra.filepath("conf/grub2.cfg"))
>  
> @@ -95,7 +95,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
>          BR2_TARGET_ROOTFS_ISO9660_INITRD=y
>          BR2_TARGET_GRUB2=y
>          BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
> -        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
> +        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
>          BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
>          """.format(infra.filepath("conf/grub2.cfg"))
>  
> -- 
> 2.25.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@lists.buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

On Wed, 6 Oct 2021 20:11:04 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> [--SNIP--]
> > We can no longer use autotools-package as a consequence of this
> > multi-build, and we have to resort to generic-package and a partial
> > duplication of the autotools-infra. Grub2 was already using custom option
> > like --prefix or --exec-prefix so this won't add much more weirdness.  
> 
> But this had an unfortunate side effect: the grub2 tools, like
> grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no
> longer installed in the target now.

Doh, didn't thought of that!
 
> But generic-package has no default install command, so declaring that
> the package installs in target does nothing.
> 
> I see two options:
> 
>  1. provide a custom command for one of the enabled platforms; e.g.
>     arbitrarily choose the first in the tuple and install in target
>     from that one tuple
> 
>  2. revert to using an autotools package, but build for no platform at
>     all and just install the tools. For the platforms, move the current
>     foreach loops into post-build and post-install-image hooks
> 
> Care to look into that, please?

mmh that's tricky, because in the case we want to use the Grub modules it need
to be generated for each platforms.
For example we can face a case where we will have these three Grub modules
repositories for each platform:
target/lib/grub/i386-pc/
target/lib/grub/i386-efi/
target/lib/grub/x86_64-efi/

What about just adding the "make install" command in the foreach loop?
This will overwrite the binaries, the locale and the /etc configurations for
each "make" loop but we will have all the Grub modules repositories. I don't
think the overwrites will cause issues because it is just tools.

Regards,

Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-10-07 10:23 +0200, Köry Maincent spake thusly:
> On Wed, 6 Oct 2021 20:11:04 +0200
> "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
> > On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > [--SNIP--]
> > > We can no longer use autotools-package as a consequence of this
> > > multi-build, and we have to resort to generic-package and a partial
> > > duplication of the autotools-infra. Grub2 was already using custom option
> > > like --prefix or --exec-prefix so this won't add much more weirdness.  
> > But this had an unfortunate side effect: the grub2 tools, like
> > grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no
> > longer installed in the target now.
> Doh, didn't thought of that!

Neither did I...

[--SNIP--]
> > Care to look into that, please?
> 
> mmh that's tricky, because in the case we want to use the Grub modules it need
> to be generated for each platforms.
> For example we can face a case where we will have these three Grub modules
> repositories for each platform:
> target/lib/grub/i386-pc/
> target/lib/grub/i386-efi/
> target/lib/grub/x86_64-efi/
> 
> What about just adding the "make install" command in the foreach loop?
> This will overwrite the binaries, the locale and the /etc configurations for
> each "make" loop but we will have all the Grub modules repositories. I don't
> think the overwrites will cause issues because it is just tools.

I was thinking about something along the lines of:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..bb853cace6 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -127,6 +127,9 @@ GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) +=
    arm64-efi
     HOST_GRUB2_CONF_ENV = \
        CPP="$(HOSTCC) -E"
     
    +# Main build: only build tools
    +GRUB2_CONF_OPTS = --with-platform=none --blabla...
    +
     GRUB2_CONF_ENV = \
        CPP="$(TARGET_CC) -E" \
        TARGET_CC="$(TARGET_CC)" \
    @@ -147,7 +150,7 @@ HOST_GRUB2_CONF_OPTS = \
        --enable-libzfs=no \
        --disable-werror
     
    -define GRUB2_CONFIGURE_CMDS
    +define GRUB2_CONFIGURE_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            mkdir -p $(@D)/build-$(tuple) ; \
            cd $(@D)/build-$(tuple) ; \
    @@ -169,14 +172,16 @@ define GRUB2_CONFIGURE_CMDS
                --disable-werror
        )
     endef
    +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
     
    -define GRUB2_BUILD_CMDS
    +define GRUB2_BUILD_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
        )
     endef
    +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
     
    -define GRUB2_INSTALL_IMAGES_CMDS
    +define GRUB2_INSTALL_IMAGES_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
            $(HOST_DIR)/usr/bin/grub-mkimage \
    @@ -194,6 +199,7 @@ define GRUB2_INSTALL_IMAGES_CMDS
            ) \
        )
     endef
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
     
    -$(eval $(generic-package))
    +$(eval $(autotools-package))
     $(eval $(host-autotools-package))

By using --with-platform=none in the main CONF_OPTS, we should be able
to disable building any of the "images", and just build the tools.

Totally untested, of course...

Alternatively, we could introduce a new package, grub2-tools, not unlike
uboot-tools, which sole responsibility would be to build and install
those tools.

Regards,
Yann E. MORIN.

> Regards,
> 
> Köry

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-10-07 10:23 +0200, Köry Maincent spake thusly:
> > On Wed, 6 Oct 2021 20:11:04 +0200
> > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:  
> > > On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
> > > [--SNIP--]  
> > > > We can no longer use autotools-package as a consequence of this
> > > > multi-build, and we have to resort to generic-package and a partial
> > > > duplication of the autotools-infra. Grub2 was already using custom
> > > > option like --prefix or --exec-prefix so this won't add much more
> > > > weirdness.    
> > > But this had an unfortunate side effect: the grub2 tools, like
> > > grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no
> > > longer installed in the target now.  
> > Doh, didn't thought of that!  
> 
> Neither did I...
> 
> [--SNIP--]
> > > Care to look into that, please?  
> > 
> > mmh that's tricky, because in the case we want to use the Grub modules it
> > need to be generated for each platforms.
> > For example we can face a case where we will have these three Grub modules
> > repositories for each platform:
> > target/lib/grub/i386-pc/
> > target/lib/grub/i386-efi/
> > target/lib/grub/x86_64-efi/
> > 
> > What about just adding the "make install" command in the foreach loop?
> > This will overwrite the binaries, the locale and the /etc configurations for
> > each "make" loop but we will have all the Grub modules repositories. I don't
> > think the overwrites will cause issues because it is just tools.  

> 
> By using --with-platform=none in the main CONF_OPTS, we should be able
> to disable building any of the "images", and just build the tools.
> 
> Totally untested, of course...

As I said if we do that we won't have all the Grub modules in the target
directory. I think these modules are quiet important, an user that wants to use
grub-mkimage will surely use them for example. 

Regards,

Köry,

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:
> On Thu, 7 Oct 2021 11:53:26 +0200
> "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
[--SNIP--]
> > By using --with-platform=none in the main CONF_OPTS, we should be able
> > to disable building any of the "images", and just build the tools.
> > 
> > Totally untested, of course...
> As I said if we do that we won't have all the Grub modules in the target
> directory. I think these modules are quiet important, an user that wants to use
> grub-mkimage will surely use them for example. 

OK, I had a hard time understanding what you meant... But indeed,
BR2_TARGET_GRUB2_INSTALL_TOOLS *also* wants to install the modules in
target/

In this case, this should not be too complex either; we just need a
post-image hook that copies all modules into TARGET_DIR, like:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..0bb10d765d 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -50,11 +50,6 @@ GRUB2_IGNORE_CVES += CVE-2019-14865
     # version available in Buildroot.
     GRUB2_IGNORE_CVES += CVE-2020-15705
     
    -ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    -GRUB2_INSTALL_TARGET = YES
    -else
    -GRUB2_INSTALL_TARGET = NO
    -endif
     GRUB2_CPE_ID_VENDOR = gnu
     
     GRUB2_BUILTIN_MODULES_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC))
    @@ -127,6 +122,9 @@ GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi
     HOST_GRUB2_CONF_ENV = \
         CPP="$(HOSTCC) -E"
     
    +# Main build: only build tools
    +GRUB2_CONF_OPTS = --with-platform=none --blabla...
    +
     GRUB2_CONF_ENV = \
         CPP="$(TARGET_CC) -E" \
         TARGET_CC="$(TARGET_CC)" \
    @@ -147,7 +145,7 @@ HOST_GRUB2_CONF_OPTS = \
         --enable-libzfs=no \
         --disable-werror
     
    -define GRUB2_CONFIGURE_CMDS
    +define GRUB2_CONFIGURE_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(@D)/build-$(tuple) ; \
             cd $(@D)/build-$(tuple) ; \
    @@ -169,14 +167,16 @@ define GRUB2_CONFIGURE_CMDS
                 --disable-werror
         )
     endef
    +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
     
    -define GRUB2_BUILD_CMDS
    +define GRUB2_BUILD_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
         )
     endef
    +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
     
    -define GRUB2_INSTALL_IMAGES_CMDS
    +define GRUB2_INSTALL_IMAGES_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
             $(HOST_DIR)/usr/bin/grub-mkimage \
    @@ -194,6 +194,23 @@ define GRUB2_INSTALL_IMAGES_CMDS
             ) \
         )
     endef
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
     
    -$(eval $(generic-package))
    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +GRUB2_INSTALL_TARGET = YES
    +define GRUB2_INSTALL_MODS_IN_TARGET
    +    mkdir -p $(TARGET_DIR)/boot
    +    $(foreach tuple, $(GRUB2_TUPLES-y), \
    +        cp -a $(GRUB2_IMAGE_$(tuple)) $(TARGET_DIR)/boot/
    +    )
    +endef
    +# **YES** this is a post-install-image hook that installs in target/
    +# **AND** we need it to be registered **LAST**, after all the per-tuple
    +# image-install hooks
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET
    +else
    +GRUB2_INSTALL_TARGET = NO
    +endif
    +
    +$(eval $(autotools-package))
     $(eval $(host-autotools-package))


NOTE: yes, I know that $(GRUB2_IMAGE_$(tuple)) is not exactly what we
need to copy, and that it will need some additional tweaking.

But then, maybe installing all the per-tuple to target/ might just
work in the end. That would not be nice, but as Thomas already said,
grub2 is already not nice anyway...

Also, I just noticed that we copy the cdboot.img from the host dir! Are
the modules all there?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

On Thu, 7 Oct 2021 18:29:31 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:
> > On Thu, 7 Oct 2021 11:53:26 +0200
> > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:  
> [--SNIP--]
> > > By using --with-platform=none in the main CONF_OPTS, we should be able
> > > to disable building any of the "images", and just build the tools.
> > > 
> > > Totally untested, of course...  
> > As I said if we do that we won't have all the Grub modules in the target
> > directory. I think these modules are quiet important, an user that wants to
> > use grub-mkimage will surely use them for example.   
> 
> OK, I had a hard time understanding what you meant... But indeed,
> BR2_TARGET_GRUB2_INSTALL_TOOLS *also* wants to install the modules in
> target/

Sorry I was maybe not clear enough.
I was speaking about all the .mod and .module files installed in the
/target/lib/grub/$(GRUB2_TUPLES-$(tuple)) directory when calling the "make
install" command. If we use none as platform they are not generated, therefore
installed.

Here is the list of installed file if I add "make install" in the foreach
loop: https://termbin.com/8b4w
Here is the list of installed file in the case of none
platform: https://termbin.com/x45h

> 
> In this case, this should not be too complex either; we just need a
> post-image hook that copies all modules into TARGET_DIR, like:

Yes it could works, but the installation of the module is not really proper, see
below.

> 
>     diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
>     index e01ebb2edb..0bb10d765d 100644
>     --- a/boot/grub2/grub2.mk
>     +++ b/boot/grub2/grub2.mk
>     @@ -50,11 +50,6 @@ GRUB2_IGNORE_CVES += CVE-2019-14865
>      # version available in Buildroot.
>      GRUB2_IGNORE_CVES += CVE-2020-15705
>      
>     -ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
>     -GRUB2_INSTALL_TARGET = YES
>     -else
>     -GRUB2_INSTALL_TARGET = NO
>     -endif
>      GRUB2_CPE_ID_VENDOR = gnu
>      
>      GRUB2_BUILTIN_MODULES_PC = $(call
> qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC)) @@ -127,6 +122,9 @@
> GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi HOST_GRUB2_CONF_ENV =
> \ CPP="$(HOSTCC) -E"
>      
>     +# Main build: only build tools
>     +GRUB2_CONF_OPTS = --with-platform=none --blabla...
>     +
>      GRUB2_CONF_ENV = \
>          CPP="$(TARGET_CC) -E" \
>          TARGET_CC="$(TARGET_CC)" \
>     @@ -147,7 +145,7 @@ HOST_GRUB2_CONF_OPTS = \
>          --enable-libzfs=no \
>          --disable-werror
>      
>     -define GRUB2_CONFIGURE_CMDS
>     +define GRUB2_CONFIGURE_CMDS_PTF
>          $(foreach tuple, $(GRUB2_TUPLES-y), \
>              mkdir -p $(@D)/build-$(tuple) ; \
>              cd $(@D)/build-$(tuple) ; \
>     @@ -169,14 +167,16 @@ define GRUB2_CONFIGURE_CMDS
>                  --disable-werror
>          )
>      endef
>     +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
>      
>     -define GRUB2_BUILD_CMDS
>     +define GRUB2_BUILD_CMDS_PTF
>          $(foreach tuple, $(GRUB2_TUPLES-y), \
>              $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
>          )
>      endef
>     +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
>      
>     -define GRUB2_INSTALL_IMAGES_CMDS
>     +define GRUB2_INSTALL_IMAGES_CMDS_PTF
>          $(foreach tuple, $(GRUB2_TUPLES-y), \
>              mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
>              $(HOST_DIR)/usr/bin/grub-mkimage \
>     @@ -194,6 +194,23 @@ define GRUB2_INSTALL_IMAGES_CMDS
>              ) \
>          )
>      endef
>     +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
>      
>     -$(eval $(generic-package))
>     +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
>     +GRUB2_INSTALL_TARGET = YES
>     +define GRUB2_INSTALL_MODS_IN_TARGET
>     +    mkdir -p $(TARGET_DIR)/boot
>     +    $(foreach tuple, $(GRUB2_TUPLES-y), \
>     +        cp -a $(GRUB2_IMAGE_$(tuple)) $(TARGET_DIR)/boot/

Here you copy only the image generated, not the modules. It should more be like
that:
	$(foreach tuple, $(GRUB2_TUPLES-y), \
		mkdir -p $(TARGET_DIR)/usr/lib/$(tuple) \
		cp -a $(@D)/build-$(tuple)/grub-core/{*.mod,*.module} \
			$(TARGET_DIR)/usr/lib/$(tuple)

We might also need *.img file because cdboot.img is needed to create CDROM
drive image.

Or we also could "make install" each platform in a temporary folder then copy
the content of tmp-folder-$(tuple)/usr/lib/$(tuple) in the $(TARGET_DIR) but I
am not sure it is better than just "make install" directly in the $(TARGET_DIR).

>     +    )
>     +endef
>     +# **YES** this is a post-install-image hook that installs in target/
>     +# **AND** we need it to be registered **LAST**, after all the per-tuple
>     +# image-install hooks
>     +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET
>     +else
>     +GRUB2_INSTALL_TARGET = NO
>     +endif
>     +
>     +$(eval $(autotools-package))
>      $(eval $(host-autotools-package))
> 
> 
> NOTE: yes, I know that $(GRUB2_IMAGE_$(tuple)) is not exactly what we
> need to copy, and that it will need some additional tweaking.
> 
> But then, maybe installing all the per-tuple to target/ might just
> work in the end. That would not be nice, but as Thomas already said,
> grub2 is already not nice anyway...

Yeah, not sure what is the better solution. 

> Also, I just noticed that we copy the cdboot.img from the host dir! Are
> the modules all there?

You may have copy a wrong code because it won't install cdboot.img in the
$(TARGET_DIR) with the code below.

Regards,
Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

Hello Yann,

On Thu, 7 Oct 2021 18:29:31 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:
> > On Thu, 7 Oct 2021 11:53:26 +0200
> > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:  
> [--SNIP--]

> In this case, this should not be too complex either; we just need a
> post-image hook that copies all modules into TARGET_DIR, like:

Is something like the following code seems okay for you.
I use temporary install-$(tuple) folder for the different platforms.
We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the
CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder.
Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other
configure command without receive this error message:
  configure: error: source directory already configured; run "make distclean"
  there first

To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK.
I have tested it and it seems functioning.


diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
index e01ebb2edb..e339eed6de 100644
--- a/boot/grub2/grub2.mk
+++ b/boot/grub2/grub2.mk
@@ -139,6 +139,22 @@ GRUB2_CONF_ENV = \
        TARGET_OBJCOPY="$(TARGET_OBJCOPY)" \
        TARGET_STRIP="$(TARGET_CROSS)strip"
 
+GRUB2_CONF_COMMON_OPTS = \
+       --host=$(GNU_TARGET_NAME) \
+       --build=$(GNU_HOST_NAME) \
+       --prefix=/ \
+       --exec-prefix=/ \
+       --disable-grub-mkfont \
+       --enable-efiemu=no \
+       ac_cv_lib_lzma_lzma_code=no \
+       --enable-device-mapper=no \
+       --enable-libzfs=no \
+       --disable-werror
+
+GRUB2_CONF_OPTS = \
+       $(GRUB2_CONF_COMMON_OPTS) \
+       --with-platform=none
+
 HOST_GRUB2_CONF_OPTS = \
        --disable-grub-mkfont \
        --enable-efiemu=no \
@@ -147,7 +163,7 @@ HOST_GRUB2_CONF_OPTS = \
        --enable-libzfs=no \
        --disable-werror

-define GRUB2_CONFIGURE_CMDS
+define GRUB2_CONFIGURE_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                mkdir -p $(@D)/build-$(tuple) ; \
                cd $(@D)/build-$(tuple) ; \
@@ -157,26 +173,19 @@ define GRUB2_CONFIGURE_CMDS
                ../configure \
                        --target=$(GRUB2_TARGET_$(tuple)) \
                        --with-platform=$(GRUB2_PLATFORM_$(tuple)) \
-                       --host=$(GNU_TARGET_NAME) \
-                       --build=$(GNU_HOST_NAME) \
-                       --prefix=/ \
-                       --exec-prefix=/ \
-                       --disable-grub-mkfont \
-                       --enable-efiemu=no \
-                       ac_cv_lib_lzma_lzma_code=no \
-                       --enable-device-mapper=no \
-                       --enable-libzfs=no \
-                       --disable-werror
+                       $(GRUB2_CONF_COMMON_OPTS)
        )
 endef
+GRUB2_PRE_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
 
-define GRUB2_BUILD_CMDS
+define GRUB2_BUILD_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
        )
 endef
+GRUB2_PRE_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
 
-define GRUB2_INSTALL_IMAGES_CMDS
+define GRUB2_INSTALL_IMAGES_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
                $(HOST_DIR)/usr/bin/grub-mkimage \
@@ -194,6 +203,20 @@ define GRUB2_INSTALL_IMAGES_CMDS
                ) \
        )
 endef
+GRUB2_PRE_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_CMDS_PTF
+
+ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
+GRUB2_INSTALL_TARGET = YES
+define GRUB2_INSTALL_MODS_IN_TARGET
+       $(foreach tuple, $(GRUB2_TUPLES-y), \
+               $(TARGET_MAKE_ENV) $(MAKE) DESTDIR=$(@D)/$(install-$(tuple))
install -C $(@D)/build-$(tuple) ; \
+               cp -a $(@D)/$(install-$(tuple))/lib/* $(TARGET_DIR)/lib/ ;
+       )
+endef
+GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET
+else
+GRUB2_INSTALL_TARGET = NO
+endif
 
-$(eval $(generic-package))
+$(eval $(autotools-package))
 $(eval $(host-autotools-package))

Regards,
Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

Sorry for my late reply, I've been pretty busy here...

On 2021-10-11 12:27 +0200, Köry Maincent spake thusly:
> On Thu, 7 Oct 2021 18:29:31 +0200
> "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
> > On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:
> > > On Thu, 7 Oct 2021 11:53:26 +0200
> > > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:  
> > [--SNIP--]
> 
> > In this case, this should not be too complex either; we just need a
> > post-image hook that copies all modules into TARGET_DIR, like:
> 
> Is something like the following code seems okay for you.
> I use temporary install-$(tuple) folder for the different platforms.
> We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the
> CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder.
> Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other
> configure command without receive this error message:
>   configure: error: source directory already configured; run "make distclean"
>   there first

Arg, I hadn't thought about this...

> To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK.
> I have tested it and it seems functioning.

But I'm afraid it is going to fail if you try to reconfigure the thing,
like:

    $ make grub2-reconfigure

So, in the end, your original proposal, to install each and every tuples
into target/, is probably the best option we have, and it can be as
simple as something like:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..a18696b6cc 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -195,5 +195,13 @@ define GRUB2_INSTALL_IMAGES_CMDS
        )
     endef
     
    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +define GRUB2_INSTALL_TARGET_CMDS
    +	$(foreach tuple, $(GRUB2_TUPLES-y), \
    +		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple) DESTDIR=$(TARGET_DIR) install
    +	)
    +endef
    +endif
    +
     $(eval $(generic-package))
     $(eval $(host-autotools-package))

Yes, this will indeed install the same tools more than once, but this is
not a problem: it is _not_ a case of a package overwriting files from
another package, but overwriting its own files, so we don't care.

Test-built with:

    BR2_x86_pentium4=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_GRUB2=y
    BR2_TARGET_GRUB2_I386_PC=y
    BR2_TARGET_GRUB2_I386_EFI=y
    BR2_TARGET_GRUB2_INSTALL_TOOLS=y

Side note, unrelated to this issue: we have a parallel build issue with
grub2:

    2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
    2021-10-14 21:45:52 config.status: creating config-util.h
    2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script ../grub-core/script/parser.y
    2021-10-14 21:45:52 ../grub-core/script/parser.y:92.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated]
    2021-10-14 21:45:52    92 | %pure-parser
    2021-10-14 21:45:52       | ^~~~~~~~~~~~
    2021-10-14 21:45:52       | %define api.pure
    2021-10-14 21:45:52 ../grub-core/script/parser.y: warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
    2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
    2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
    2021-10-14 21:45:52    20 | #include <config-util.h>
    2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
    2021-10-14 21:45:52 compilation terminated.
    2021-10-14 21:45:52 make[2]: *** [Makefile:13107: grub_fstest.pp] Error 1
    2021-10-14 21:45:52 make[2]: *** Waiting for unfinished jobs....

And re-starting the build succeeds.

But just looking at how config-util.h is actually created (in Makefile.in)
makes me cry:

   3389 config-util.h: stamp-h1
   3390     @test -f $@ || rm -f stamp-h1
   3391     @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1
   3392
   3393 stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
   3394     @rm -f stamp-h1
   3395     cd $(top_builddir) && $(SHELL) ./config.status config-util.h

Whaaa?

But fortunately, it looks liek it is fixed with upstream commit
42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

Could you look into both the tools and the backport of the commit?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
> Side note, unrelated to this issue: we have a parallel build issue with
> grub2:
[--SNIP--]
> But fortunately, it looks liek it is fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And of course, since this commit touches Makefile.am, but we avoid very
hard re-running autoreconf for grub2, we also need to patch Makefile.in
(the same hunk applies quite OK to both file, fortunately).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Adam Duskett]

Hello;

On Thu, Oct 14, 2021 at 1:02 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Köry, All,
>
> Sorry for my late reply, I've been pretty busy here...
>
> On 2021-10-11 12:27 +0200, Köry Maincent spake thusly:
> > On Thu, 7 Oct 2021 18:29:31 +0200
> > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
> > > On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:
> > > > On Thu, 7 Oct 2021 11:53:26 +0200
> > > > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
> > > [--SNIP--]
> >
> > > In this case, this should not be too complex either; we just need a
> > > post-image hook that copies all modules into TARGET_DIR, like:
> >
> > Is something like the following code seems okay for you.
> > I use temporary install-$(tuple) folder for the different platforms.
> > We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the
> > CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder.
> > Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other
> > configure command without receive this error message:
> >   configure: error: source directory already configured; run "make distclean"
> >   there first
>
> Arg, I hadn't thought about this...
>
> > To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK.
> > I have tested it and it seems functioning.
>
> But I'm afraid it is going to fail if you try to reconfigure the thing,
> like:
>
>     $ make grub2-reconfigure
>
> So, in the end, your original proposal, to install each and every tuples
> into target/, is probably the best option we have, and it can be as
> simple as something like:
>
>     diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
>     index e01ebb2edb..a18696b6cc 100644
>     --- a/boot/grub2/grub2.mk
>     +++ b/boot/grub2/grub2.mk
>     @@ -195,5 +195,13 @@ define GRUB2_INSTALL_IMAGES_CMDS
>         )
>      endef
>
>     +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
>     +define GRUB2_INSTALL_TARGET_CMDS
>     +   $(foreach tuple, $(GRUB2_TUPLES-y), \
>     +           $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple) DESTDIR=$(TARGET_DIR) install
>     +   )
>     +endef
>     +endif
>     +
>      $(eval $(generic-package))
>      $(eval $(host-autotools-package))
>
This indeed works properly!
Tested-by: Adam Duskett <aduskett@gmail.com>
> Yes, this will indeed install the same tools more than once, but this is
> not a problem: it is _not_ a case of a package overwriting files from
> another package, but overwriting its own files, so we don't care.
>
> Test-built with:
>
>     BR2_x86_pentium4=y
>     BR2_TOOLCHAIN_EXTERNAL=y
>     BR2_TARGET_GRUB2=y
>     BR2_TARGET_GRUB2_I386_PC=y
>     BR2_TARGET_GRUB2_I386_EFI=y
>     BR2_TARGET_GRUB2_INSTALL_TOOLS=y
>
> Side note, unrelated to this issue: we have a parallel build issue with
> grub2:
>
>     2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
>     2021-10-14 21:45:52 config.status: creating config-util.h
>     2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script ../grub-core/script/parser.y
>     2021-10-14 21:45:52 ../grub-core/script/parser.y:92.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated]
>     2021-10-14 21:45:52    92 | %pure-parser
>     2021-10-14 21:45:52       | ^~~~~~~~~~~~
>     2021-10-14 21:45:52       | %define api.pure
>     2021-10-14 21:45:52 ../grub-core/script/parser.y: warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
>     2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
>     2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
>     2021-10-14 21:45:52    20 | #include <config-util.h>
>     2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
>     2021-10-14 21:45:52 compilation terminated.
>     2021-10-14 21:45:52 make[2]: *** [Makefile:13107: grub_fstest.pp] Error 1
>     2021-10-14 21:45:52 make[2]: *** Waiting for unfinished jobs....
>
> And re-starting the build succeeds.
>
> But just looking at how config-util.h is actually created (in Makefile.in)
> makes me cry:
>
>    3389 config-util.h: stamp-h1
>    3390     @test -f $@ || rm -f stamp-h1
>    3391     @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1
>    3392
>    3393 stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
>    3394     @rm -f stamp-h1
>    3395     cd $(top_builddir) && $(SHELL) ./config.status config-util.h
>
> Whaaa?
>
> But fortunately, it looks liek it is fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).
>
> Could you look into both the tools and the backport of the commit?
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
> So, in the end, your original proposal, to install each and every tuples
> into target/, is probably the best option we have, [...]

Additionally, to test what was failing, I had instrumented the build
with the following patch (note that it mixes fixes, like no semi-colons
but separate lines, or using && not sime-colon, with the actual calls
to MESSAGE):

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..5400538837 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -149,8 +149,9 @@ HOST_GRUB2_CONF_OPTS = \
     
     define GRUB2_CONFIGURE_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    -		mkdir -p $(@D)/build-$(tuple) ; \
    -		cd $(@D)/build-$(tuple) ; \
    +		@$(call MESSAGE,Configuring $(tuple))
    +		mkdir -p $(@D)/build-$(tuple)
    +		cd $(@D)/build-$(tuple) && \
     		$(TARGET_CONFIGURE_OPTS) \
     				$(TARGET_CONFIGURE_ARGS) \
     		$(GRUB2_CONF_ENV) \
    @@ -172,13 +173,15 @@ endef
     
     define GRUB2_BUILD_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    +		@$(call MESSAGE,Building $(tuple))
     		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
     	)
     endef
     
     define GRUB2_INSTALL_IMAGES_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    -		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
    +		@$(call MESSAGE,Installing $(tuple) to images directory)
    +		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple)))
     		$(HOST_DIR)/usr/bin/grub-mkimage \
     			-d $(@D)/build-$(tuple)/grub-core/ \
     			-O $(tuple) \
    @@ -186,12 +189,12 @@ define GRUB2_INSTALL_IMAGES_CMDS
     			-p "$(GRUB2_PREFIX_$(tuple))" \
     			$(if $(GRUB2_BUILTIN_CONFIG_$(tuple)), \
     				-c $(GRUB2_BUILTIN_CONFIG_$(tuple))) \
    -				$(GRUB2_BUILTIN_MODULES_$(tuple)) ; \
    -		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple)) ; \
    +				$(GRUB2_BUILTIN_MODULES_$(tuple))
    +		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple))
     		$(if $(findstring $(GRUB2_PLATFORM_$(tuple)), pc), \
     			cat $(HOST_DIR)/lib/grub/$(tuple)/cdboot.img $(GRUB2_IMAGE_$(tuple)) > \
    -				$(BINARIES_DIR)/grub-eltorito.img ; \
    -		) \
    +				$(BINARIES_DIR)/grub-eltorito.img \
    +		)
     	)
     endef
     

That way, it is easier to see what is going on, and the output when
filtered with brmake, looks nice ;-)

2021-10-14 22:52:36 >>> grub2 2.04 Extracting
2021-10-14 22:52:36 >>> grub2 2.04 Patching
2021-10-14 22:52:39 >>> grub2 2.04 Configuring
2021-10-14 22:52:39 >>> grub2 2.04 Configuring i386-pc
2021-10-14 22:52:56 >>> grub2 2.04 Configuring i386-efi
2021-10-14 22:53:13 >>> grub2 2.04 Building
2021-10-14 22:53:13 >>> grub2 2.04 Building i386-pc
2021-10-14 22:53:51 >>> grub2 2.04 Building i386-efi
2021-10-14 22:54:28 >>> grub2 2.04 Installing to target
2021-10-14 22:54:28 >>> grub2 2.04 Installing i386-pc to target
2021-10-14 22:54:31 >>> grub2 2.04 Installing i386-efi to target
2021-10-14 22:54:34 >>> grub2 2.04 Installing to images directory
2021-10-14 22:54:34 >>> grub2 2.04 Installing i386-pc to images directory
2021-10-14 22:54:34 >>> grub2 2.04 Installing i386-efi to images directory

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

Hello Yann,

On Thu, 14 Oct 2021 22:02:07 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Köry, All,
> 
> Sorry for my late reply, I've been pretty busy here...

No problem, be a maintainer is a full-time job done on your own time and you
do great work! :)

> So, in the end, your original proposal, to install each and every tuples
> into target/, is probably the best option we have, and it can be as
> simple as something like:

Alright

> 
> Test-built with:
> 
>     BR2_x86_pentium4=y
>     BR2_TOOLCHAIN_EXTERNAL=y
>     BR2_TARGET_GRUB2=y
>     BR2_TARGET_GRUB2_I386_PC=y
>     BR2_TARGET_GRUB2_I386_EFI=y
>     BR2_TARGET_GRUB2_INSTALL_TOOLS=y
> 
> Side note, unrelated to this issue: we have a parallel build issue with
> grub2:
> 
>     2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
>     2021-10-14 21:45:52 config.status: creating config-util.h
>     2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script
> ../grub-core/script/parser.y 2021-10-14 21:45:52
> ../grub-core/script/parser.y:92.1-12: warning: deprecated directive:
> ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated] 2021-10-14 21:45:52
> 92 | %pure-parser 2021-10-14 21:45:52       | ^~~~~~~~~~~~ 2021-10-14
> 21:45:52       | %define api.pure 2021-10-14 21:45:52
> ../grub-core/script/parser.y: warning: fix-its can be applied.  Rerun with
> option '--update'. [-Wother] 2021-10-14 21:45:52
> /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I.
> -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include
> -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include
> -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib
> -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
> -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64
> -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c
> ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c
> ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
> 2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error:
> config-util.h: No such file or directory 2021-10-14 21:45:52    20 | #include
> <config-util.h> 2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
> 2021-10-14 21:45:52 compilation terminated. 2021-10-14 21:45:52 make[2]: ***
> [Makefile:13107: grub_fstest.pp] Error 1 2021-10-14 21:45:52 make[2]: ***
> Waiting for unfinished jobs....
> 
> And re-starting the build succeeds.

Weird, I do not face this build error with the same defconfig.

Regards,
Köry
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Thomas Petazzoni]

On Fri, 15 Oct 2021 11:19:39 +0200
Köry Maincent <kory.maincent@bootlin.com> wrote:

> > And re-starting the build succeeds.  
> 
> Weird, I do not face this build error with the same defconfig.

It's a parallel build issue, which means it's a race condition. It will
not happen all the time. Try on a build server with lots of CPU cores,
and do the build in a loop multiple times.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Yann E. MORIN]

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
[--SNIP--]
> Side note, unrelated to this issue: we have a parallel build issue with
> grub2:
>     2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
>     2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
>     2021-10-14 21:45:52    20 | #include <config-util.h>
>     2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
[--SNIP--]
> But fortunately, it looks liek it is fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And apparently, this is not enough, because I still hit the issue, about
one in five builds, always when building i386-efi... :-(

But given how inventive the grub buildsystem is, I dread to ever have to
look at it once more... :-(

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Adam Duskett]

Hey Kory;

Do you plan on submitting a patch to fix this? If not, I can do so.
Just let me know!

Adam

On Fri, Oct 15, 2021 at 1:50 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Köry, All,
>
> On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
> [--SNIP--]
> > Side note, unrelated to this issue: we have a parallel build issue with
> > grub2:
> >     2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
> >     2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
> >     2021-10-14 21:45:52    20 | #include <config-util.h>
> >     2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
> [--SNIP--]
> > But fortunately, it looks liek it is fixed with upstream commit
> > 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).
>
> And apparently, this is not enough, because I still hit the issue, about
> one in five builds, always when building i386-efi... :-(
>
> But given how inventive the grub buildsystem is, I dread to ever have to
> look at it once more... :-(
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

[Köry Maincent]

Hello Adam,

On Tue, 19 Oct 2021 09:30:10 -0700
Adam Duskett <aduskett@gmail.com> wrote:

> Hey Kory;
> 
> Do you plan on submitting a patch to fix this? If not, I can do so.
> Just let me know!

I was about to do it before Yann spot the parallel build error. As I did not
managed to face the error I have stopped my action on the patch. I can however
send the first part of it that fixing tools install, then wait the fix for the
parallel build issue.

Regards,
Köry

> 
> Adam
> 
> On Fri, Oct 15, 2021 at 1:50 PM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> >
> > Köry, All,
> >
> > On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
> > [--SNIP--]  
> > > Side note, unrelated to this issue: we have a parallel build issue with
> > > grub2:
> > >     2021-10-14 21:45:52
> > > /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H
> > > -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include
> > > -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include
> > > -I./include -I../grub-core/lib/libgcrypt-grub/src/
> > > -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE
> > > -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector
> > > -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@'
> > > ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../
> > > grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm
> > > -f grub_fstest.pp; exit 1) 2021-10-14 21:45:52
> > > ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such
> > > file or directory 2021-10-14 21:45:52    20 | #include <config-util.h>
> > > 2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~  
> > [--SNIP--]  
> > > But fortunately, it looks liek it is fixed with upstream commit
> > > 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).  
> >
> > And apparently, this is not enough, because I still hit the issue, about
> > one in five builds, always when building i386-efi... :-(
> >
> > But given how inventive the grub buildsystem is, I dread to ever have to
> > look at it once more... :-(
> >

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot