[PATCH v6 0/2] arm/efi: Add dom0less support to UEFI boot

[PATCH v6 0/2] arm/efi: Add dom0less support to UEFI boot

[Luca Fancellu]

This serie introduces a way to start a dom0less setup when Xen is booting as EFI
application.
Using the device tree it's now possible to fetch from the disk and load in
memory all the modules needed to start any domU defined in the DT.
Dom0less for now is supported only by the arm architecture.

This serie was originally formed by 3 patch, the first one was merged.

Luca Fancellu (2):
  arm/efi: Use dom0less configuration when using EFI boot
  arm/efi: load dom0 modules from DT using UEFI

 docs/misc/arm/device-tree/booting.txt |  29 +++
 docs/misc/efi.pandoc                  | 261 ++++++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 343 +++++++++++++++++++++++++-
 xen/common/efi/boot.c                 |  55 +++--
 4 files changed, 671 insertions(+), 17 deletions(-)

-- 
2.17.1

[PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Luca Fancellu]

This patch introduces the support for dom0less configuration
when using UEFI boot on ARM, it permits the EFI boot to
continue if no dom0 kernel is specified but at least one domU
is found.

Introduce the new property "xen,uefi-binary" for device tree boot
module nodes that are subnode of "xen,domain" compatible nodes.
The property holds a string containing the file name of the
binary that shall be loaded by the uefi loader from the filesystem.

Introduce a new call efi_check_dt_boot(...) called during EFI boot
that checks for module to be loaded using device tree.
Architectures that don't support device tree don't have to
provide this function.

Update efi documentation about how to start a dom0less
setup using UEFI

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v6:
- change is_boot_module() to check for every multiboot,module
instead of multiboot,{kernel,ramdisk,device-tree} (Julien), as
a result of that remove the function and put the check inside of
handle_module_node(...)
Changes in v5:
- Removed unneeded variable initialization
- Fixed comment
- Fixed error message for the absence of an initial domain kernel
- changed efi_arch_check_dt_boot to efi_check_dt_boot and add
a stub if CONFIG_HAS_DEVICE_TREE is not declared, updated commit
message about the call introduction in the EFI boot flow.
Changes in v4:
- update uefi,cfg-load to xen,uefi-cfg-load in documentation
- fixed comments and code style
- changed variable name from dt_module_found to dt_modules_found
in boot.c
- removed stub efi_arch_check_dt_boot from x86 code because the
architecture does not support DT, protected call with #ifdef
in the common code.
- add comment to explain the result from efi_arch_check_dt_boot
just looking the common code
- Add space before comment in boot.c
- renamed uefi,binary property to xen,uefi-binary
Changes in v3:
- fixed documentation
- fixed name len in strlcpy
- fixed some style issues
- closed filesystem handle before calling blexit
- passed runtime errors up to the stack instead
of calling blexit
- renamed names and function to make them more
general in prevision to load also Dom0 kernel
and ramdisk from DT
Changes in v2:
- remove array of struct file
- fixed some int types
- Made the code use filesystem even when configuration
file is skipped.
- add documentation of uefi,binary in booting.txt
- add documentation on how to boot all configuration
for Xen using UEFI in efi.pandoc
---
 docs/misc/arm/device-tree/booting.txt |  21 ++
 docs/misc/efi.pandoc                  | 203 +++++++++++++++++
 xen/arch/arm/efi/efi-boot.h           | 299 +++++++++++++++++++++++++-
 xen/common/efi/boot.c                 |  39 +++-
 4 files changed, 550 insertions(+), 12 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 352b0ec43a..7258e7e1ec 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
 
     Command line parameters for the guest kernel.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 The ramdisk sub-node has the following properties:
 
 - compatible
@@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
     Specifies the physical address of the ramdisk in RAM and its
     length.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 
 Example
 =======
@@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
     Specifies the physical address of the device tree binary fragment
     RAM and its length.
 
+- xen,uefi-binary (UEFI boot only)
+
+    String property that specifies the file name to be loaded by the UEFI boot
+    for this module. If this is specified, there is no need to specify the reg
+    property because it will be created by the UEFI stub on boot.
+    This option is needed only when UEFI boot is used.
+
 As an example:
 
         module@0xc000000 {
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index ed85351541..876cd55005 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,3 +167,206 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+
+## UEFI boot and dom0less on ARM
+
+Dom0less feature is supported by ARM and it is possible to use it when Xen is
+started as an EFI application.
+The way to specify the domU domains is by Device Tree as specified in the
+[dom0less](dom0less.html) documentation page under the "Device Tree
+configuration" section, but instead of declaring the reg property in the boot
+module, the user must specify the "xen,uefi-binary" property containing the name
+of the binary file that has to be loaded in memory.
+The UEFI stub will load the binary in memory and it will add the reg property
+accordingly.
+
+An example here:
+
+domU1 {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	compatible = "xen,domain";
+	memory = <0 0x20000>;
+	cpus = <1>;
+	vpl011;
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "console=ttyAMA0";
+	};
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+	module@3 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "passthrough.dtb";
+	};
+};
+
+## How to boot different Xen setup using UEFI
+
+These are the different ways to boot a Xen system from UEFI:
+
+ - Boot Xen and Dom0 (minimum required)
+ - Boot Xen and DomU(s) (true dom0less, only on ARM)
+ - Boot Xen, Dom0 and DomU(s) (only on ARM)
+
+### Boot Xen and Dom0
+
+This configuration can be started using the Xen configuration file in the
+example above.
+
+### Boot Xen and DomU(s)
+
+This configuration needs the domU domain(s) specified in the /chosen node,
+examples of how to do that are provided by the documentation about dom0less
+and the example above shows how to use the "xen,uefi-binary" property to use the
+UEFI stub for module loading.
+When adding DomU modules to device tree, also add the property
+xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
+Otherwise, Xen will skip the config file and rely on device tree alone.
+
+Example 1 of how to boot a true dom0less configuration:
+
+Xen configuration file: skipped.
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "<Xen command line>"
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+Example 2 of how to boot a true dom0less configuration:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,uefi-cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+	domU2 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0x100000>;
+		vpl011;
+
+		module@2 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu2.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+### Boot Xen, Dom0 and DomU(s)
+
+This configuration is a mix of the two configuration above, to boot this one
+the configuration file must be processed so the /chosen node must have the
+"xen,uefi-cfg-load" property.
+
+Here an example:
+
+Xen configuration file:
+
+```
+[global]
+default=xen
+
+[xen]
+options=<Xen command line>
+kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
+ramdisk=initrd-3.0.31-0.4-xen
+dtb=<optional DTB>
+```
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,uefi-cfg-load;
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
+
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
+
+
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index a3e46453d4..f35e035b22 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -8,9 +8,49 @@
 #include <asm/setup.h>
 #include <asm/smp.h>
 
+typedef struct {
+    char *name;
+    unsigned int name_len;
+    EFI_PHYSICAL_ADDRESS addr;
+    UINTN size;
+} module_name;
+
+/*
+ * Binaries will be translated into bootmodules, the maximum number for them is
+ * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
+ */
+#define MAX_UEFI_MODULES (MAX_MODULES - 2)
+static struct file __initdata module_binary;
+static module_name __initdata modules[MAX_UEFI_MODULES];
+static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
+static unsigned int __initdata modules_idx;
+
+#define ERROR_BINARY_FILE_NOT_FOUND (-1)
+#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
+#define ERROR_ALLOC_MODULE_NAME     (-2)
+#define ERROR_MISSING_DT_PROPERTY   (-3)
+#define ERROR_RENAME_MODULE_NAME    (-4)
+#define ERROR_SET_REG_PROPERTY      (-5)
+#define ERROR_CHECK_MODULE_COMPAT   (-6)
+#define ERROR_DT_MODULE_DOMU        (-1)
+#define ERROR_DT_CHOSEN_NODE        (-2)
+
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
 
+static int get_module_file_index(const char *name, unsigned int name_len);
+static void PrintMessage(const CHAR16 *s);
+static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                const char *name,
+                                unsigned int name_len);
+static int handle_module_node(EFI_FILE_HANDLE dir_handle,
+                              int module_node_offset,
+                              int reg_addr_cells,
+                              int reg_size_cells);
+static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                       int domain_node);
+static int efi_check_dt_boot(EFI_FILE_HANDLE dir_handle);
+
 #define DEVICE_TREE_GUID \
 {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
 
@@ -552,8 +592,254 @@ static void __init efi_arch_handle_module(const struct file *file,
                          kernel.size) < 0 )
             blexit(L"Unable to set reg property.");
     }
-    else
+    else if ( file != &module_binary )
+        /*
+         * If file is not a dom0 module file and it's not a domU module,
+         * stop here.
+         */
         blexit(L"Unknown module type");
+
+    /*
+     * modules_available is decremented here because for each dom0 file added
+     * from the configuration file, there will be an additional bootmodule,
+     * so the number of available slots will be decremented because there is a
+     * maximum amount of bootmodules that can be loaded.
+     */
+    modules_available--;
+}
+
+/*
+ * This function checks for a binary previously loaded with a give name, it
+ * returns the index of the file in the modules array or a negative number if no
+ * file with that name is found.
+ */
+static int __init get_module_file_index(const char *name,
+                                        unsigned int name_len)
+{
+    unsigned int i;
+    int ret = ERROR_BINARY_FILE_NOT_FOUND;
+
+    for ( i = 0; i < modules_idx; i++ )
+    {
+        module_name *mod = &modules[i];
+        if ( (mod->name_len == name_len) &&
+             (strncmp(mod->name, name, name_len) == 0) )
+        {
+            ret = i;
+            break;
+        }
+    }
+    return ret;
+}
+
+static void __init PrintMessage(const CHAR16 *s)
+{
+    PrintStr(s);
+    PrintStr(newline);
+}
+
+/*
+ * This function allocates a binary and keeps track of its name, it returns the
+ * index of the file in the modules array or a negative number on error.
+ */
+static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
+                                       const char *name,
+                                       unsigned int name_len)
+{
+    module_name *file_name;
+    union string module_name;
+    int ret;
+
+    /*
+     * Check if there is any space left for a module, the variable
+     * modules_available is updated each time we use read_file(...)
+     * successfully.
+     */
+    if ( !modules_available )
+    {
+        PrintMessage(L"No space left for modules");
+        return ERROR_ALLOC_MODULE_NO_SPACE;
+    }
+
+    module_name.cs = name;
+    ret = modules_idx;
+
+    /* Save at this index the name of this binary */
+    file_name = &modules[ret];
+
+    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
+                              (void**)&file_name->name) != EFI_SUCCESS )
+    {
+        PrintMessage(L"Error allocating memory for module binary name");
+        return ERROR_ALLOC_MODULE_NAME;
+    }
+
+    /* Save name and length of the binary in the data structure */
+    strlcpy(file_name->name, name, name_len + 1);
+    file_name->name_len = name_len;
+
+    /* Load the binary in memory */
+    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
+
+    /* Save address and size */
+    file_name->addr = module_binary.addr;
+    file_name->size = module_binary.size;
+
+    /* s2w(...) allocates some memory, free it */
+    efi_bs->FreePool(module_name.w);
+
+    modules_idx++;
+
+    return ret;
+}
+
+/*
+ * This function checks for the presence of the xen,uefi-binary property in the
+ * module, if found it loads the binary as module and sets the right address
+ * for the reg property into the module DT node.
+ */
+static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
+                                     int module_node_offset,
+                                     int reg_addr_cells,
+                                     int reg_size_cells)
+{
+    const void *uefi_name_prop;
+    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
+    int uefi_name_len, file_idx, module_compat;
+    module_name *file;
+
+    /* Check if the node is a multiboot,module otherwise return */
+    module_compat = fdt_node_check_compatible(fdt, module_node_offset,
+                                              "multiboot,module");
+    if ( module_compat < 0 )
+        /* Error while checking the compatible string */
+        return ERROR_CHECK_MODULE_COMPAT;
+
+    if ( module_compat != 0 )
+        /* Module is not a multiboot,module */
+        return 0;
+
+    /* Read xen,uefi-binary property to get the file name. */
+    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
+                                 &uefi_name_len);
+
+    if ( !uefi_name_prop )
+        /* Property not found */
+        return 0;
+
+    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
+    if ( file_idx < 0 )
+    {
+        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
+                                        uefi_name_len);
+        if ( file_idx < 0 )
+            return file_idx;
+    }
+
+    file = &modules[file_idx];
+
+    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
+
+    /* Rename the module to be module@{address} */
+    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
+    {
+        PrintMessage(L"Unable to modify module node name.");
+        return ERROR_RENAME_MODULE_NAME;
+    }
+
+    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
+                     file->addr, file->size) < 0 )
+    {
+        PrintMessage(L"Unable to set module reg property.");
+        return ERROR_SET_REG_PROPERTY;
+    }
+
+    return 0;
+}
+
+/*
+ * This function checks for boot modules under the domU guest domain node
+ * in the DT.
+ * Returns 0 on success, negative number on error.
+ */
+static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
+                                              int domain_node)
+{
+    int module_node, addr_cells, size_cells, len;
+    const struct fdt_property *prop;
+
+    /* Get #address-cells and #size-cells from domain node */
+    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#address-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
+    if ( !prop )
+    {
+        PrintMessage(L"#size-cells not found in domain node.");
+        return ERROR_MISSING_DT_PROPERTY;
+    }
+
+    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
+
+    /* Check for nodes compatible with multiboot,module inside this node */
+    for ( module_node = fdt_first_subnode(fdt, domain_node);
+          module_node > 0;
+          module_node = fdt_next_subnode(fdt, module_node) )
+    {
+        int ret = handle_module_node(dir_handle, module_node, addr_cells,
+                                        size_cells);
+        if ( ret < 0 )
+            return ret;
+    }
+
+    return 0;
+}
+
+/*
+ * This function checks for xen domain nodes under the /chosen node for possible
+ * domU guests to be loaded.
+ * Returns the number of modules loaded or a negative number for error.
+ */
+static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
+{
+    int chosen, node, addr_len, size_len;
+    unsigned int i = 0;
+
+    /* Check for the chosen node in the current DTB */
+    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
+    if ( chosen < 0 )
+    {
+        PrintMessage(L"Unable to setup chosen node");
+        return ERROR_DT_CHOSEN_NODE;
+    }
+
+    /* Check for nodes compatible with xen,domain under the chosen node */
+    for ( node = fdt_first_subnode(fdt, chosen);
+          node > 0;
+          node = fdt_next_subnode(fdt, node) )
+    {
+        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
+        {
+            /* Found a node with compatible xen,domain; handle this node. */
+            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
+                return ERROR_DT_MODULE_DOMU;
+        }
+    }
+
+    /* Free boot modules file names if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free boot modules binary names */
+        efi_bs->FreePool(modules[i].name);
+    }
+
+    return modules_idx;
 }
 
 static void __init efi_arch_cpu(void)
@@ -562,8 +848,19 @@ static void __init efi_arch_cpu(void)
 
 static void __init efi_arch_blexit(void)
 {
+    unsigned int i = 0;
+
     if ( dtbfile.need_to_free )
         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
+    /* Free boot modules file names if any */
+    for ( ; i < modules_idx; i++ )
+    {
+        /* Free boot modules binary names */
+        efi_bs->FreePool(modules[i].name);
+        /* Free modules binaries */
+        efi_bs->FreePages(modules[i].addr,
+                          PFN_UP(modules[i].size));
+    }
     if ( memmap )
         efi_bs->FreePool(memmap);
 }
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 758f9d74d2..7879b93f93 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
     StdErr->OutputString(StdErr, (CHAR16 *)s );
 }
 
+#ifndef CONFIG_HAS_DEVICE_TREE
+static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
+{
+    return 0;
+}
+#endif
+
 /*
  * Include architecture specific implementation here, which references the
  * static globals defined above.
@@ -1136,6 +1143,8 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     bool base_video = false;
     const char *option_str;
     bool use_cfg_file;
+    int dt_modules_found;
+    EFI_FILE_HANDLE dir_handle;
 
     __set_bit(EFI_BOOT, &efi_flags);
     __set_bit(EFI_LOADER, &efi_flags);
@@ -1216,9 +1225,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
     efi_arch_relocate_image(0);
 
+    /* Get the file system interface. */
+    dir_handle = get_parent_handle(loaded_image, &file_name);
+
     if ( use_cfg_file )
     {
-        EFI_FILE_HANDLE dir_handle;
         UINTN depth, cols, rows, size;
 
         size = cols = rows = depth = 0;
@@ -1229,9 +1240,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
 
         gop = efi_get_gop();
 
-        /* Get the file system interface. */
-        dir_handle = get_parent_handle(loaded_image, &file_name);
-
         /* Read and parse the config file. */
         if ( read_section(loaded_image, L"config", &cfg, NULL) )
             PrintStr(L"Using builtin config file\r\n");
@@ -1285,14 +1293,12 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
             efi_bs->FreePool(name.w);
         }
 
-        if ( !name.s )
-            blexit(L"No Dom0 kernel image specified.");
-
         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
 
-        option_str = split_string(name.s);
+        option_str = name.s ? split_string(name.s) : NULL;
 
-        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
+        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
+             name.s )
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
@@ -1361,12 +1367,23 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
         cfg.addr = 0;
 
-        dir_handle->Close(dir_handle);
-
         if ( gop && !base_video )
             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
     }
 
+    /* Get the number of boot modules specified on the DT or an error (<0) */
+    dt_modules_found = efi_check_dt_boot(dir_handle);
+
+    dir_handle->Close(dir_handle);
+
+    if ( dt_modules_found < 0 )
+        /* efi_check_dt_boot throws some error */
+        blexit(L"Error processing boot modules on DT.");
+
+    /* Check if at least one of Dom0 or DomU(s) is specified */
+    if ( !dt_modules_found && !kernel.ptr )
+        blexit(L"No initial domain kernel specified.");
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Stefano Stabellini]

On Mon, 11 Oct 2021, Luca Fancellu wrote:
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "xen,uefi-binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Introduce a new call efi_check_dt_boot(...) called during EFI boot
> that checks for module to be loaded using device tree.
> Architectures that don't support device tree don't have to
> provide this function.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v6:
> - change is_boot_module() to check for every multiboot,module
> instead of multiboot,{kernel,ramdisk,device-tree} (Julien), as
> a result of that remove the function and put the check inside of
> handle_module_node(...)
> Changes in v5:
> - Removed unneeded variable initialization
> - Fixed comment
> - Fixed error message for the absence of an initial domain kernel
> - changed efi_arch_check_dt_boot to efi_check_dt_boot and add
> a stub if CONFIG_HAS_DEVICE_TREE is not declared, updated commit
> message about the call introduction in the EFI boot flow.
> Changes in v4:
> - update uefi,cfg-load to xen,uefi-cfg-load in documentation
> - fixed comments and code style
> - changed variable name from dt_module_found to dt_modules_found
> in boot.c
> - removed stub efi_arch_check_dt_boot from x86 code because the
> architecture does not support DT, protected call with #ifdef
> in the common code.
> - add comment to explain the result from efi_arch_check_dt_boot
> just looking the common code
> - Add space before comment in boot.c
> - renamed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - fixed documentation
> - fixed name len in strlcpy
> - fixed some style issues
> - closed filesystem handle before calling blexit
> - passed runtime errors up to the stack instead
> of calling blexit
> - renamed names and function to make them more
> general in prevision to load also Dom0 kernel
> and ramdisk from DT
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
>  docs/misc/arm/device-tree/booting.txt |  21 ++
>  docs/misc/efi.pandoc                  | 203 +++++++++++++++++
>  xen/arch/arm/efi/efi-boot.h           | 299 +++++++++++++++++++++++++-
>  xen/common/efi/boot.c                 |  39 +++-
>  4 files changed, 550 insertions(+), 12 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 352b0ec43a..7258e7e1ec 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
>  
>      Command line parameters for the guest kernel.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  The ramdisk sub-node has the following properties:
>  
>  - compatible
> @@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
>      Specifies the physical address of the ramdisk in RAM and its
>      length.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  
>  Example
>  =======
> @@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
>      Specifies the physical address of the device tree binary fragment
>      RAM and its length.
>  
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
>  As an example:
>  
>          module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ed85351541..876cd55005 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
>  	--output xen.signed.efi \
>  	xen.unified.efi
>  ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "xen,uefi-binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	compatible = "xen,domain";
> +	memory = <0 0x20000>;
> +	cpus = <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "console=ttyAMA0";
> +	};
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +These are the different ways to boot a Xen system from UEFI:
> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in the
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen node,
> +examples of how to do that are provided by the documentation about dom0less
> +and the example above shows how to use the "xen,uefi-binary" property to use the
> +UEFI stub for module loading.
> +When adding DomU modules to device tree, also add the property
> +xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
> +Otherwise, Xen will skip the config file and rely on device tree alone.
> +
> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this one
> +the configuration file must be processed so the /chosen node must have the
> +"xen,uefi-cfg-load" property.
> +
> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index a3e46453d4..f35e035b22 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,49 @@
>  #include <asm/setup.h>
>  #include <asm/smp.h>
>  
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_UEFI_MODULES (MAX_MODULES - 2)
> +static struct file __initdata module_binary;
> +static module_name __initdata modules[MAX_UEFI_MODULES];
> +static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
> +static unsigned int __initdata modules_idx;
> +
> +#define ERROR_BINARY_FILE_NOT_FOUND (-1)
> +#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
> +#define ERROR_ALLOC_MODULE_NAME     (-2)
> +#define ERROR_MISSING_DT_PROPERTY   (-3)
> +#define ERROR_RENAME_MODULE_NAME    (-4)
> +#define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_CHECK_MODULE_COMPAT   (-6)
> +#define ERROR_DT_MODULE_DOMU        (-1)
> +#define ERROR_DT_CHOSEN_NODE        (-2)
> +
>  void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>  void __flush_dcache_area(const void *vaddr, unsigned long size);
>  
> +static int get_module_file_index(const char *name, unsigned int name_len);
> +static void PrintMessage(const CHAR16 *s);
> +static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                const char *name,
> +                                unsigned int name_len);
> +static int handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                              int module_node_offset,
> +                              int reg_addr_cells,
> +                              int reg_size_cells);
> +static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                       int domain_node);
> +static int efi_check_dt_boot(EFI_FILE_HANDLE dir_handle);
> +
>  #define DEVICE_TREE_GUID \
>  {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
>  
> @@ -552,8 +592,254 @@ static void __init efi_arch_handle_module(const struct file *file,
>                           kernel.size) < 0 )
>              blexit(L"Unable to set reg property.");
>      }
> -    else
> +    else if ( file != &module_binary )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>          blexit(L"Unknown module type");
> +
> +    /*
> +     * modules_available is decremented here because for each dom0 file added
> +     * from the configuration file, there will be an additional bootmodule,
> +     * so the number of available slots will be decremented because there is a
> +     * maximum amount of bootmodules that can be loaded.
> +     */
> +    modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the modules array or a negative number if no
> + * file with that name is found.
> + */
> +static int __init get_module_file_index(const char *name,
> +                                        unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret = ERROR_BINARY_FILE_NOT_FOUND;
> +
> +    for ( i = 0; i < modules_idx; i++ )
> +    {
> +        module_name *mod = &modules[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +static void __init PrintMessage(const CHAR16 *s)
> +{
> +    PrintStr(s);
> +    PrintStr(newline);
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it returns the
> + * index of the file in the modules array or a negative number on error.
> + */
> +static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                       const char *name,
> +                                       unsigned int name_len)
> +{
> +    module_name *file_name;
> +    union string module_name;
> +    int ret;
> +
> +    /*
> +     * Check if there is any space left for a module, the variable
> +     * modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !modules_available )
> +    {
> +        PrintMessage(L"No space left for modules");
> +        return ERROR_ALLOC_MODULE_NO_SPACE;
> +    }
> +
> +    module_name.cs = name;
> +    ret = modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &modules[ret];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +    {
> +        PrintMessage(L"Error allocating memory for module binary name");
> +        return ERROR_ALLOC_MODULE_NAME;
> +    }
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len + 1);
> +    file_name->name_len = name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr = module_binary.addr;
> +    file_name->size = module_binary.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    modules_idx++;
> +
> +    return ret;
> +}
> +
> +/*
> + * This function checks for the presence of the xen,uefi-binary property in the
> + * module, if found it loads the binary as module and sets the right address
> + * for the reg property into the module DT node.
> + */
> +static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                                     int module_node_offset,
> +                                     int reg_addr_cells,
> +                                     int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx, module_compat;
> +    module_name *file;
> +
> +    /* Check if the node is a multiboot,module otherwise return */
> +    module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> +                                              "multiboot,module");
> +    if ( module_compat < 0 )
> +        /* Error while checking the compatible string */
> +        return ERROR_CHECK_MODULE_COMPAT;
> +
> +    if ( module_compat != 0 )
> +        /* Module is not a multiboot,module */
> +        return 0;
> +
> +    /* Read xen,uefi-binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return 0;
> +
> +    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
> +    if ( file_idx < 0 )
> +    {
> +        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
> +                                        uefi_name_len);
> +        if ( file_idx < 0 )
> +            return file_idx;
> +    }
> +
> +    file = &modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +    {
> +        PrintMessage(L"Unable to modify module node name.");
> +        return ERROR_RENAME_MODULE_NAME;
> +    }
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +    {
> +        PrintMessage(L"Unable to set module reg property.");
> +        return ERROR_SET_REG_PROPERTY;
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + * Returns 0 on success, negative number on error.
> + */
> +static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                              int domain_node)
> +{
> +    int module_node, addr_cells, size_cells, len;
> +    const struct fdt_property *prop;
> +
> +    /* Get #address-cells and #size-cells from domain node */
> +    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#address-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#size-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    /* Check for nodes compatible with multiboot,module inside this node */
> +    for ( module_node = fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +    {
> +        int ret = handle_module_node(dir_handle, module_node, addr_cells,
> +                                        size_cells);
> +        if ( ret < 0 )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + * Returns the number of modules loaded or a negative number for error.
> + */
> +static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen, node, addr_len, size_len;
> +    unsigned int i = 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +    {
> +        PrintMessage(L"Unable to setup chosen node");
> +        return ERROR_DT_CHOSEN_NODE;
> +    }
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
> +        {
> +            /* Found a node with compatible xen,domain; handle this node. */
> +            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> +                return ERROR_DT_MODULE_DOMU;
> +        }
> +    }
> +
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +    }
> +
> +    return modules_idx;
>  }
>  
>  static void __init efi_arch_cpu(void)
> @@ -562,8 +848,19 @@ static void __init efi_arch_cpu(void)
>  
>  static void __init efi_arch_blexit(void)
>  {
> +    unsigned int i = 0;
> +
>      if ( dtbfile.need_to_free )
>          efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +        /* Free modules binaries */
> +        efi_bs->FreePages(modules[i].addr,
> +                          PFN_UP(modules[i].size));
> +    }
>      if ( memmap )
>          efi_bs->FreePool(memmap);
>  }
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..7879b93f93 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
>      StdErr->OutputString(StdErr, (CHAR16 *)s );
>  }
>  
> +#ifndef CONFIG_HAS_DEVICE_TREE
> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    return 0;
> +}
> +#endif
> +
>  /*
>   * Include architecture specific implementation here, which references the
>   * static globals defined above.
> @@ -1136,6 +1143,8 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>      bool base_video = false;
>      const char *option_str;
>      bool use_cfg_file;
> +    int dt_modules_found;
> +    EFI_FILE_HANDLE dir_handle;
>  
>      __set_bit(EFI_BOOT, &efi_flags);
>      __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1225,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>      efi_arch_relocate_image(0);
>  
> +    /* Get the file system interface. */
> +    dir_handle = get_parent_handle(loaded_image, &file_name);
> +
>      if ( use_cfg_file )
>      {
> -        EFI_FILE_HANDLE dir_handle;
>          UINTN depth, cols, rows, size;
>  
>          size = cols = rows = depth = 0;
> @@ -1229,9 +1240,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>  
>          gop = efi_get_gop();
>  
> -        /* Get the file system interface. */
> -        dir_handle = get_parent_handle(loaded_image, &file_name);
> -
>          /* Read and parse the config file. */
>          if ( read_section(loaded_image, L"config", &cfg, NULL) )
>              PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1293,12 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>              efi_bs->FreePool(name.w);
>          }
>  
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>          efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>  
> -        option_str = split_string(name.s);
> +        option_str = name.s ? split_string(name.s) : NULL;
>  
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
> +             name.s )
>          {
>              read_file(dir_handle, s2w(&name), &kernel, option_str);
>              efi_bs->FreePool(name.w);
> @@ -1361,12 +1367,23 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>          efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>          cfg.addr = 0;
>  
> -        dir_handle->Close(dir_handle);
> -
>          if ( gop && !base_video )
>              gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>      }
>  
> +    /* Get the number of boot modules specified on the DT or an error (<0) */
> +    dt_modules_found = efi_check_dt_boot(dir_handle);
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if ( dt_modules_found < 0 )
> +        /* efi_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");
> +
> +    /* Check if at least one of Dom0 or DomU(s) is specified */
> +    if ( !dt_modules_found && !kernel.ptr )
> +        blexit(L"No initial domain kernel specified.");
> +
>      efi_arch_edd();
>  
>      /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Bertrand Marquis]

Hi Luca,

> On 11 Oct 2021, at 19:15, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
> 
> This patch introduces the support for dom0less configuration
> when using UEFI boot on ARM, it permits the EFI boot to
> continue if no dom0 kernel is specified but at least one domU
> is found.
> 
> Introduce the new property "xen,uefi-binary" for device tree boot
> module nodes that are subnode of "xen,domain" compatible nodes.
> The property holds a string containing the file name of the
> binary that shall be loaded by the uefi loader from the filesystem.
> 
> Introduce a new call efi_check_dt_boot(...) called during EFI boot
> that checks for module to be loaded using device tree.
> Architectures that don't support device tree don't have to
> provide this function.
> 
> Update efi documentation about how to start a dom0less
> setup using UEFI
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
> Changes in v6:
> - change is_boot_module() to check for every multiboot,module
> instead of multiboot,{kernel,ramdisk,device-tree} (Julien), as
> a result of that remove the function and put the check inside of
> handle_module_node(...)
> Changes in v5:
> - Removed unneeded variable initialization
> - Fixed comment
> - Fixed error message for the absence of an initial domain kernel
> - changed efi_arch_check_dt_boot to efi_check_dt_boot and add
> a stub if CONFIG_HAS_DEVICE_TREE is not declared, updated commit
> message about the call introduction in the EFI boot flow.
> Changes in v4:
> - update uefi,cfg-load to xen,uefi-cfg-load in documentation
> - fixed comments and code style
> - changed variable name from dt_module_found to dt_modules_found
> in boot.c
> - removed stub efi_arch_check_dt_boot from x86 code because the
> architecture does not support DT, protected call with #ifdef
> in the common code.
> - add comment to explain the result from efi_arch_check_dt_boot
> just looking the common code
> - Add space before comment in boot.c
> - renamed uefi,binary property to xen,uefi-binary
> Changes in v3:
> - fixed documentation
> - fixed name len in strlcpy
> - fixed some style issues
> - closed filesystem handle before calling blexit
> - passed runtime errors up to the stack instead
> of calling blexit
> - renamed names and function to make them more
> general in prevision to load also Dom0 kernel
> and ramdisk from DT
> Changes in v2:
> - remove array of struct file
> - fixed some int types
> - Made the code use filesystem even when configuration
> file is skipped.
> - add documentation of uefi,binary in booting.txt
> - add documentation on how to boot all configuration
> for Xen using UEFI in efi.pandoc
> ---
> docs/misc/arm/device-tree/booting.txt |  21 ++
> docs/misc/efi.pandoc                  | 203 +++++++++++++++++
> xen/arch/arm/efi/efi-boot.h           | 299 +++++++++++++++++++++++++-
> xen/common/efi/boot.c                 |  39 +++-
> 4 files changed, 550 insertions(+), 12 deletions(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 352b0ec43a..7258e7e1ec 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -190,6 +190,13 @@ The kernel sub-node has the following properties:
> 
>     Command line parameters for the guest kernel.
> 
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
> The ramdisk sub-node has the following properties:
> 
> - compatible
> @@ -201,6 +208,13 @@ The ramdisk sub-node has the following properties:
>     Specifies the physical address of the ramdisk in RAM and its
>     length.
> 
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
> 
> Example
> =======
> @@ -265,6 +279,13 @@ The dtb sub-node should have the following properties:
>     Specifies the physical address of the device tree binary fragment
>     RAM and its length.
> 
> +- xen,uefi-binary (UEFI boot only)
> +
> +    String property that specifies the file name to be loaded by the UEFI boot
> +    for this module. If this is specified, there is no need to specify the reg
> +    property because it will be created by the UEFI stub on boot.
> +    This option is needed only when UEFI boot is used.
> +
> As an example:
> 
>         module@0xc000000 {
> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
> index ed85351541..876cd55005 100644
> --- a/docs/misc/efi.pandoc
> +++ b/docs/misc/efi.pandoc
> @@ -167,3 +167,206 @@ sbsign \
> 	--output xen.signed.efi \
> 	xen.unified.efi
> ```
> +
> +## UEFI boot and dom0less on ARM
> +
> +Dom0less feature is supported by ARM and it is possible to use it when Xen is
> +started as an EFI application.
> +The way to specify the domU domains is by Device Tree as specified in the
> +[dom0less](dom0less.html) documentation page under the "Device Tree
> +configuration" section, but instead of declaring the reg property in the boot
> +module, the user must specify the "xen,uefi-binary" property containing the name
> +of the binary file that has to be loaded in memory.
> +The UEFI stub will load the binary in memory and it will add the reg property
> +accordingly.
> +
> +An example here:
> +
> +domU1 {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	compatible = "xen,domain";
> +	memory = <0 0x20000>;
> +	cpus = <1>;
> +	vpl011;
> +
> +	module@1 {
> +		compatible = "multiboot,kernel", "multiboot,module";
> +		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
> +		bootargs = "console=ttyAMA0";
> +	};
> +	module@2 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
> +	};
> +	module@3 {
> +		compatible = "multiboot,ramdisk", "multiboot,module";
> +		xen,uefi-binary = "passthrough.dtb";
> +	};
> +};
> +
> +## How to boot different Xen setup using UEFI
> +
> +These are the different ways to boot a Xen system from UEFI:
> +
> + - Boot Xen and Dom0 (minimum required)
> + - Boot Xen and DomU(s) (true dom0less, only on ARM)
> + - Boot Xen, Dom0 and DomU(s) (only on ARM)
> +
> +### Boot Xen and Dom0
> +
> +This configuration can be started using the Xen configuration file in the
> +example above.
> +
> +### Boot Xen and DomU(s)
> +
> +This configuration needs the domU domain(s) specified in the /chosen node,
> +examples of how to do that are provided by the documentation about dom0less
> +and the example above shows how to use the "xen,uefi-binary" property to use the
> +UEFI stub for module loading.
> +When adding DomU modules to device tree, also add the property
> +xen,uefi-cfg-load under chosen for Xen to load the Xen config file.
> +Otherwise, Xen will skip the config file and rely on device tree alone.
> +
> +Example 1 of how to boot a true dom0less configuration:
> +
> +Xen configuration file: skipped.
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,xen-bootargs = "<Xen command line>"
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +Example 2 of how to boot a true dom0less configuration:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +	domU2 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0x100000>;
> +		vpl011;
> +
> +		module@2 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu2.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +### Boot Xen, Dom0 and DomU(s)
> +
> +This configuration is a mix of the two configuration above, to boot this one
> +the configuration file must be processed so the /chosen node must have the
> +"xen,uefi-cfg-load" property.
> +
> +Here an example:
> +
> +Xen configuration file:
> +
> +```
> +[global]
> +default=xen
> +
> +[xen]
> +options=<Xen command line>
> +kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> +ramdisk=initrd-3.0.31-0.4-xen
> +dtb=<optional DTB>
> +```
> +
> +Device tree:
> +
> +```
> +chosen {
> +	#size-cells = <0x1>;
> +	#address-cells = <0x1>;
> +	xen,uefi-cfg-load;
> +
> +	domU1 {
> +		#size-cells = <0x1>;
> +		#address-cells = <0x1>;
> +		compatible = "xen,domain";
> +		cpus = <0x1>;
> +		memory = <0x0 0xc0000>;
> +		vpl011;
> +
> +		module@1 {
> +			compatible = "multiboot,kernel", "multiboot,module";
> +			xen,uefi-binary = "Image-domu1.bin";
> +			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
> +		};
> +	};
> +};
> +```
> +
> +
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index a3e46453d4..f35e035b22 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -8,9 +8,49 @@
> #include <asm/setup.h>
> #include <asm/smp.h>
> 
> +typedef struct {
> +    char *name;
> +    unsigned int name_len;
> +    EFI_PHYSICAL_ADDRESS addr;
> +    UINTN size;
> +} module_name;
> +
> +/*
> + * Binaries will be translated into bootmodules, the maximum number for them is
> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> + */
> +#define MAX_UEFI_MODULES (MAX_MODULES - 2)
> +static struct file __initdata module_binary;
> +static module_name __initdata modules[MAX_UEFI_MODULES];
> +static unsigned int __initdata modules_available = MAX_UEFI_MODULES;
> +static unsigned int __initdata modules_idx;
> +
> +#define ERROR_BINARY_FILE_NOT_FOUND (-1)
> +#define ERROR_ALLOC_MODULE_NO_SPACE (-1)
> +#define ERROR_ALLOC_MODULE_NAME     (-2)
> +#define ERROR_MISSING_DT_PROPERTY   (-3)
> +#define ERROR_RENAME_MODULE_NAME    (-4)
> +#define ERROR_SET_REG_PROPERTY      (-5)
> +#define ERROR_CHECK_MODULE_COMPAT   (-6)
> +#define ERROR_DT_MODULE_DOMU        (-1)
> +#define ERROR_DT_CHOSEN_NODE        (-2)
> +
> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
> void __flush_dcache_area(const void *vaddr, unsigned long size);
> 
> +static int get_module_file_index(const char *name, unsigned int name_len);
> +static void PrintMessage(const CHAR16 *s);
> +static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                const char *name,
> +                                unsigned int name_len);
> +static int handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                              int module_node_offset,
> +                              int reg_addr_cells,
> +                              int reg_size_cells);
> +static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                       int domain_node);
> +static int efi_check_dt_boot(EFI_FILE_HANDLE dir_handle);
> +
> #define DEVICE_TREE_GUID \
> {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 0xe0}}
> 
> @@ -552,8 +592,254 @@ static void __init efi_arch_handle_module(const struct file *file,
>                          kernel.size) < 0 )
>             blexit(L"Unable to set reg property.");
>     }
> -    else
> +    else if ( file != &module_binary )
> +        /*
> +         * If file is not a dom0 module file and it's not a domU module,
> +         * stop here.
> +         */
>         blexit(L"Unknown module type");
> +
> +    /*
> +     * modules_available is decremented here because for each dom0 file added
> +     * from the configuration file, there will be an additional bootmodule,
> +     * so the number of available slots will be decremented because there is a
> +     * maximum amount of bootmodules that can be loaded.
> +     */
> +    modules_available--;
> +}
> +
> +/*
> + * This function checks for a binary previously loaded with a give name, it
> + * returns the index of the file in the modules array or a negative number if no
> + * file with that name is found.
> + */
> +static int __init get_module_file_index(const char *name,
> +                                        unsigned int name_len)
> +{
> +    unsigned int i;
> +    int ret = ERROR_BINARY_FILE_NOT_FOUND;
> +
> +    for ( i = 0; i < modules_idx; i++ )
> +    {
> +        module_name *mod = &modules[i];
> +        if ( (mod->name_len == name_len) &&
> +             (strncmp(mod->name, name, name_len) == 0) )
> +        {
> +            ret = i;
> +            break;
> +        }
> +    }
> +    return ret;
> +}
> +
> +static void __init PrintMessage(const CHAR16 *s)
> +{
> +    PrintStr(s);
> +    PrintStr(newline);
> +}
> +
> +/*
> + * This function allocates a binary and keeps track of its name, it returns the
> + * index of the file in the modules array or a negative number on error.
> + */
> +static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
> +                                       const char *name,
> +                                       unsigned int name_len)
> +{
> +    module_name *file_name;
> +    union string module_name;
> +    int ret;
> +
> +    /*
> +     * Check if there is any space left for a module, the variable
> +     * modules_available is updated each time we use read_file(...)
> +     * successfully.
> +     */
> +    if ( !modules_available )
> +    {
> +        PrintMessage(L"No space left for modules");
> +        return ERROR_ALLOC_MODULE_NO_SPACE;
> +    }
> +
> +    module_name.cs = name;
> +    ret = modules_idx;
> +
> +    /* Save at this index the name of this binary */
> +    file_name = &modules[ret];
> +
> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
> +                              (void**)&file_name->name) != EFI_SUCCESS )
> +    {
> +        PrintMessage(L"Error allocating memory for module binary name");
> +        return ERROR_ALLOC_MODULE_NAME;
> +    }
> +
> +    /* Save name and length of the binary in the data structure */
> +    strlcpy(file_name->name, name, name_len + 1);
> +    file_name->name_len = name_len;
> +
> +    /* Load the binary in memory */
> +    read_file(dir_handle, s2w(&module_name), &module_binary, NULL);
> +
> +    /* Save address and size */
> +    file_name->addr = module_binary.addr;
> +    file_name->size = module_binary.size;
> +
> +    /* s2w(...) allocates some memory, free it */
> +    efi_bs->FreePool(module_name.w);
> +
> +    modules_idx++;
> +
> +    return ret;
> +}
> +
> +/*
> + * This function checks for the presence of the xen,uefi-binary property in the
> + * module, if found it loads the binary as module and sets the right address
> + * for the reg property into the module DT node.
> + */
> +static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> +                                     int module_node_offset,
> +                                     int reg_addr_cells,
> +                                     int reg_size_cells)
> +{
> +    const void *uefi_name_prop;
> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
> +    int uefi_name_len, file_idx, module_compat;
> +    module_name *file;
> +
> +    /* Check if the node is a multiboot,module otherwise return */
> +    module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> +                                              "multiboot,module");
> +    if ( module_compat < 0 )
> +        /* Error while checking the compatible string */
> +        return ERROR_CHECK_MODULE_COMPAT;
> +
> +    if ( module_compat != 0 )
> +        /* Module is not a multiboot,module */
> +        return 0;
> +
> +    /* Read xen,uefi-binary property to get the file name. */
> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
> +                                 &uefi_name_len);
> +
> +    if ( !uefi_name_prop )
> +        /* Property not found */
> +        return 0;
> +
> +    file_idx = get_module_file_index(uefi_name_prop, uefi_name_len);
> +    if ( file_idx < 0 )
> +    {
> +        file_idx = allocate_module_file(dir_handle, uefi_name_prop,
> +                                        uefi_name_len);
> +        if ( file_idx < 0 )
> +            return file_idx;
> +    }
> +
> +    file = &modules[file_idx];
> +
> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
> +
> +    /* Rename the module to be module@{address} */
> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
> +    {
> +        PrintMessage(L"Unable to modify module node name.");
> +        return ERROR_RENAME_MODULE_NAME;
> +    }
> +
> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, reg_size_cells,
> +                     file->addr, file->size) < 0 )
> +    {
> +        PrintMessage(L"Unable to set module reg property.");
> +        return ERROR_SET_REG_PROPERTY;
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for boot modules under the domU guest domain node
> + * in the DT.
> + * Returns 0 on success, negative number on error.
> + */
> +static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> +                                              int domain_node)
> +{
> +    int module_node, addr_cells, size_cells, len;
> +    const struct fdt_property *prop;
> +
> +    /* Get #address-cells and #size-cells from domain node */
> +    prop = fdt_get_property(fdt, domain_node, "#address-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#address-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    prop = fdt_get_property(fdt, domain_node, "#size-cells", &len);
> +    if ( !prop )
> +    {
> +        PrintMessage(L"#size-cells not found in domain node.");
> +        return ERROR_MISSING_DT_PROPERTY;
> +    }
> +
> +    size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
> +
> +    /* Check for nodes compatible with multiboot,module inside this node */
> +    for ( module_node = fdt_first_subnode(fdt, domain_node);
> +          module_node > 0;
> +          module_node = fdt_next_subnode(fdt, module_node) )
> +    {
> +        int ret = handle_module_node(dir_handle, module_node, addr_cells,
> +                                        size_cells);
> +        if ( ret < 0 )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
> +/*
> + * This function checks for xen domain nodes under the /chosen node for possible
> + * domU guests to be loaded.
> + * Returns the number of modules loaded or a negative number for error.
> + */
> +static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    int chosen, node, addr_len, size_len;
> +    unsigned int i = 0;
> +
> +    /* Check for the chosen node in the current DTB */
> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
> +    if ( chosen < 0 )
> +    {
> +        PrintMessage(L"Unable to setup chosen node");
> +        return ERROR_DT_CHOSEN_NODE;
> +    }
> +
> +    /* Check for nodes compatible with xen,domain under the chosen node */
> +    for ( node = fdt_first_subnode(fdt, chosen);
> +          node > 0;
> +          node = fdt_next_subnode(fdt, node) )
> +    {
> +        if ( !fdt_node_check_compatible(fdt, node, "xen,domain") )
> +        {
> +            /* Found a node with compatible xen,domain; handle this node. */
> +            if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> +                return ERROR_DT_MODULE_DOMU;
> +        }
> +    }
> +
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +    }
> +
> +    return modules_idx;
> }
> 
> static void __init efi_arch_cpu(void)
> @@ -562,8 +848,19 @@ static void __init efi_arch_cpu(void)
> 
> static void __init efi_arch_blexit(void)
> {
> +    unsigned int i = 0;
> +
>     if ( dtbfile.need_to_free )
>         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
> +    /* Free boot modules file names if any */
> +    for ( ; i < modules_idx; i++ )
> +    {
> +        /* Free boot modules binary names */
> +        efi_bs->FreePool(modules[i].name);
> +        /* Free modules binaries */
> +        efi_bs->FreePages(modules[i].addr,
> +                          PFN_UP(modules[i].size));
> +    }
>     if ( memmap )
>         efi_bs->FreePool(memmap);
> }
> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
> index 758f9d74d2..7879b93f93 100644
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
>     StdErr->OutputString(StdErr, (CHAR16 *)s );
> }
> 
> +#ifndef CONFIG_HAS_DEVICE_TREE
> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> +{
> +    return 0;
> +}
> +#endif
> +
> /*
>  * Include architecture specific implementation here, which references the
>  * static globals defined above.
> @@ -1136,6 +1143,8 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>     bool base_video = false;
>     const char *option_str;
>     bool use_cfg_file;
> +    int dt_modules_found;
> +    EFI_FILE_HANDLE dir_handle;
> 
>     __set_bit(EFI_BOOT, &efi_flags);
>     __set_bit(EFI_LOADER, &efi_flags);
> @@ -1216,9 +1225,11 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
> 
>     efi_arch_relocate_image(0);
> 
> +    /* Get the file system interface. */
> +    dir_handle = get_parent_handle(loaded_image, &file_name);
> +
>     if ( use_cfg_file )
>     {
> -        EFI_FILE_HANDLE dir_handle;
>         UINTN depth, cols, rows, size;
> 
>         size = cols = rows = depth = 0;
> @@ -1229,9 +1240,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
> 
>         gop = efi_get_gop();
> 
> -        /* Get the file system interface. */
> -        dir_handle = get_parent_handle(loaded_image, &file_name);
> -
>         /* Read and parse the config file. */
>         if ( read_section(loaded_image, L"config", &cfg, NULL) )
>             PrintStr(L"Using builtin config file\r\n");
> @@ -1285,14 +1293,12 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>             efi_bs->FreePool(name.w);
>         }
> 
> -        if ( !name.s )
> -            blexit(L"No Dom0 kernel image specified.");
> -
>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
> 
> -        option_str = split_string(name.s);
> +        option_str = name.s ? split_string(name.s) : NULL;
> 
> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
> +        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) &&
> +             name.s )
>         {
>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>             efi_bs->FreePool(name.w);
> @@ -1361,12 +1367,23 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
>         efi_bs->FreePages(cfg.addr, PFN_UP(cfg.size));
>         cfg.addr = 0;
> 
> -        dir_handle->Close(dir_handle);
> -
>         if ( gop && !base_video )
>             gop_mode = efi_find_gop_mode(gop, cols, rows, depth);
>     }
> 
> +    /* Get the number of boot modules specified on the DT or an error (<0) */
> +    dt_modules_found = efi_check_dt_boot(dir_handle);
> +
> +    dir_handle->Close(dir_handle);
> +
> +    if ( dt_modules_found < 0 )
> +        /* efi_check_dt_boot throws some error */
> +        blexit(L"Error processing boot modules on DT.");
> +
> +    /* Check if at least one of Dom0 or DomU(s) is specified */
> +    if ( !dt_modules_found && !kernel.ptr )
> +        blexit(L"No initial domain kernel specified.");
> +
>     efi_arch_edd();
> 
>     /* XXX Collect EDID info. */
> -- 
> 2.17.1
> 
> 

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Jan Beulich]

On 11.10.2021 20:15, Luca Fancellu wrote:
> --- a/xen/common/efi/boot.c
> +++ b/xen/common/efi/boot.c
> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
>      StdErr->OutputString(StdErr, (CHAR16 *)s );
>  }
>  
> +#ifndef CONFIG_HAS_DEVICE_TREE
> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)

Didn't we agree that you would drop "inline" from here?

Jan

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Luca Fancellu]

> On 12 Oct 2021, at 09:45, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 11.10.2021 20:15, Luca Fancellu wrote:
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
>>     StdErr->OutputString(StdErr, (CHAR16 *)s );
>> }
>> 
>> +#ifndef CONFIG_HAS_DEVICE_TREE
>> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> 
> Didn't we agree that you would drop "inline" from here?
> 

Yes we did, really sorry I forgot to drop it, I’ll push another serie, given the inline
Dropped are you ok with the patch?

> Jan
> 

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Jan Beulich]

On 12.10.2021 11:05, Luca Fancellu wrote:
>> On 12 Oct 2021, at 09:45, Jan Beulich <jbeulich@suse.com> wrote:
>> On 11.10.2021 20:15, Luca Fancellu wrote:
>>> --- a/xen/common/efi/boot.c
>>> +++ b/xen/common/efi/boot.c
>>> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
>>>     StdErr->OutputString(StdErr, (CHAR16 *)s );
>>> }
>>>
>>> +#ifndef CONFIG_HAS_DEVICE_TREE
>>> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>>
>> Didn't we agree that you would drop "inline" from here?
> 
> Yes we did, really sorry I forgot to drop it, I’ll push another serie, given the inline
> Dropped are you ok with the patch?

Well, yes - I had given my ack for it already.

Jan

Re: [PATCH v6 1/2] arm/efi: Use dom0less configuration when using EFI boot

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 920 bytes --]

On Tue, 12 Oct 2021, Jan Beulich wrote:
> On 12.10.2021 11:05, Luca Fancellu wrote:
> >> On 12 Oct 2021, at 09:45, Jan Beulich <jbeulich@suse.com> wrote:
> >> On 11.10.2021 20:15, Luca Fancellu wrote:
> >>> --- a/xen/common/efi/boot.c
> >>> +++ b/xen/common/efi/boot.c
> >>> @@ -166,6 +166,13 @@ static void __init PrintErr(const CHAR16 *s)
> >>>     StdErr->OutputString(StdErr, (CHAR16 *)s );
> >>> }
> >>>
> >>> +#ifndef CONFIG_HAS_DEVICE_TREE
> >>> +static inline int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> >>
> >> Didn't we agree that you would drop "inline" from here?
> > 
> > Yes we did, really sorry I forgot to drop it, I’ll push another serie, given the inline
> > Dropped are you ok with the patch?
> 
> Well, yes - I had given my ack for it already.

I made this change on commit and added your ack.

FYI I gave my reviewed-by to patch #2. Patch #2 needs your ack as Luca
dropped it on v6.

[PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Luca Fancellu]

Add support to load Dom0 boot modules from
the device tree using the xen,uefi-binary property.

Update documentation about that.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
---
Changes in v6:
- given the changes to is_boot_module() in previous patch,
a check to avoid declaration of xsm in DT and cfg file is
introduced and the call to is_boot_module is removed.
Changes in v5:
- renamed missing uefi,binary string
- used kernel.ptr instead of kernel.addr to be consistent
to the surrounding code
- Changed a comment referring to efi_arch_check_dt_boot
that now is efi_check_dt_boot
Changes in v4:
- Add check to avoid double definition of Dom0 ramdisk
from cfg file and DT
- Fix if conditions indentation in boot.c
- Moved Dom0 kernel verification code after check for
presence for Dom0 or DomU(s)
- Changed uefi,binary property to xen,uefi-binary
Changes in v3:
- new patch
---
 docs/misc/arm/device-tree/booting.txt |  8 ++++
 docs/misc/efi.pandoc                  | 64 +++++++++++++++++++++++++--
 xen/arch/arm/efi/efi-boot.h           | 52 ++++++++++++++++++++--
 xen/common/efi/boot.c                 | 16 ++++---
 4 files changed, 128 insertions(+), 12 deletions(-)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 7258e7e1ec..c6a775f4e8 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -70,6 +70,14 @@ Each node contains the following properties:
 	priority of this field vs. other mechanisms of specifying the
 	bootargs for the kernel.
 
+- xen,uefi-binary (UEFI boot only)
+
+	String property that specifies the file name to be loaded by the UEFI
+	boot for this module. If this is specified, there is no need to specify
+	the reg property because it will be created by the UEFI stub on boot.
+	This option is needed only when UEFI boot is used, the node needs to be
+	compatible with multiboot,kernel or multiboot,ramdisk.
+
 Examples
 ========
 
diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
index 876cd55005..4abbb5bb82 100644
--- a/docs/misc/efi.pandoc
+++ b/docs/misc/efi.pandoc
@@ -167,6 +167,28 @@ sbsign \
 	--output xen.signed.efi \
 	xen.unified.efi
 ```
+## UEFI boot and Dom0 modules on ARM
+
+When booting using UEFI on ARM, it is possible to specify the Dom0 modules
+directly from the device tree without using the Xen configuration file, here an
+example:
+
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+}
 
 ## UEFI boot and dom0less on ARM
 
@@ -326,10 +348,10 @@ chosen {
 ### Boot Xen, Dom0 and DomU(s)
 
 This configuration is a mix of the two configuration above, to boot this one
-the configuration file must be processed so the /chosen node must have the
-"xen,uefi-cfg-load" property.
+the configuration file can be processed or the Dom0 modules can be read from
+the device tree.
 
-Here an example:
+Here the first example:
 
 Xen configuration file:
 
@@ -369,4 +391,40 @@ chosen {
 };
 ```
 
+Here the second example:
+
+Device tree:
+
+```
+chosen {
+	#size-cells = <0x1>;
+	#address-cells = <0x1>;
+	xen,xen-bootargs = "[Xen boot arguments]"
+
+	module@1 {
+		compatible = "multiboot,kernel", "multiboot,module";
+		xen,uefi-binary = "vmlinuz-3.0.31-0.4-xen";
+		bootargs = "[domain 0 command line options]";
+	};
+
+	module@2 {
+		compatible = "multiboot,ramdisk", "multiboot,module";
+		xen,uefi-binary = "initrd-3.0.31-0.4-xen";
+	};
+
+	domU1 {
+		#size-cells = <0x1>;
+		#address-cells = <0x1>;
+		compatible = "xen,domain";
+		cpus = <0x1>;
+		memory = <0x0 0xc0000>;
+		vpl011;
 
+		module@1 {
+			compatible = "multiboot,kernel", "multiboot,module";
+			xen,uefi-binary = "Image-domu1.bin";
+			bootargs = "console=ttyAMA0 root=/dev/ram0 rw";
+		};
+	};
+};
+```
diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index f35e035b22..840728d6c0 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -32,8 +32,12 @@ static unsigned int __initdata modules_idx;
 #define ERROR_RENAME_MODULE_NAME    (-4)
 #define ERROR_SET_REG_PROPERTY      (-5)
 #define ERROR_CHECK_MODULE_COMPAT   (-6)
+#define ERROR_DOM0_ALREADY_FOUND    (-7)
+#define ERROR_DOM0_RAMDISK_FOUND    (-8)
+#define ERROR_XSM_ALREADY_FOUND     (-9)
 #define ERROR_DT_MODULE_DOMU        (-1)
 #define ERROR_DT_CHOSEN_NODE        (-2)
+#define ERROR_DT_MODULE_DOM0        (-3)
 
 void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
 void __flush_dcache_area(const void *vaddr, unsigned long size);
@@ -46,7 +50,8 @@ static int allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int handle_module_node(EFI_FILE_HANDLE dir_handle,
                               int module_node_offset,
                               int reg_addr_cells,
-                              int reg_size_cells);
+                              int reg_size_cells,
+                              bool is_domu_module);
 static int handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
                                        int domain_node);
 static int efi_check_dt_boot(EFI_FILE_HANDLE dir_handle);
@@ -701,7 +706,8 @@ static int __init allocate_module_file(EFI_FILE_HANDLE dir_handle,
 static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
                                      int module_node_offset,
                                      int reg_addr_cells,
-                                     int reg_size_cells)
+                                     int reg_size_cells,
+                                     bool is_domu_module)
 {
     const void *uefi_name_prop;
     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
@@ -754,6 +760,41 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
         return ERROR_SET_REG_PROPERTY;
     }
 
+    if ( !is_domu_module )
+    {
+        if ( (fdt_node_check_compatible(fdt, module_node_offset,
+                                    "multiboot,kernel") == 0) )
+        {
+            /*
+            * This is the Dom0 kernel, wire it to the kernel variable because it
+            * will be verified by the shim lock protocol later in the common
+            * code.
+            */
+            if ( kernel.addr )
+            {
+                PrintMessage(L"Dom0 kernel already found in cfg file.");
+                return ERROR_DOM0_ALREADY_FOUND;
+            }
+            kernel.need_to_free = false; /* Freed using the module array */
+            kernel.addr = file->addr;
+            kernel.size = file->size;
+        }
+        else if ( ramdisk.addr &&
+                  (fdt_node_check_compatible(fdt, module_node_offset,
+                                             "multiboot,ramdisk") == 0) )
+        {
+            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
+            return ERROR_DOM0_RAMDISK_FOUND;
+        }
+        else if ( xsm.addr &&
+                  (fdt_node_check_compatible(fdt, module_node_offset,
+                                             "xen,xsm-policy") == 0) )
+        {
+            PrintMessage(L"XSM policy already found in cfg file.");
+            return ERROR_XSM_ALREADY_FOUND;
+        }
+    }
+
     return 0;
 }
 
@@ -793,7 +834,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
           module_node = fdt_next_subnode(fdt, module_node) )
     {
         int ret = handle_module_node(dir_handle, module_node, addr_cells,
-                                        size_cells);
+                                     size_cells, true);
         if ( ret < 0 )
             return ret;
     }
@@ -803,7 +844,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
 
 /*
  * This function checks for xen domain nodes under the /chosen node for possible
- * domU guests to be loaded.
+ * dom0 and domU guests to be loaded.
  * Returns the number of modules loaded or a negative number for error.
  */
 static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
@@ -830,6 +871,9 @@ static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
                 return ERROR_DT_MODULE_DOMU;
         }
+        else if ( handle_module_node(dir_handle, node, addr_len, size_len,
+                                     false) < 0 )
+                 return ERROR_DT_MODULE_DOM0;
     }
 
     /* Free boot modules file names if any */
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 7879b93f93..531975326f 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1302,11 +1302,6 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
         {
             read_file(dir_handle, s2w(&name), &kernel, option_str);
             efi_bs->FreePool(name.w);
-
-            if ( !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
-                            (void **)&shim_lock)) &&
-                 (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
-                PrintErrMesg(L"Dom0 kernel image could not be verified", status);
         }
 
         if ( !read_section(loaded_image, L"ramdisk", &ramdisk, NULL) )
@@ -1384,6 +1379,17 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
     if ( !dt_modules_found && !kernel.ptr )
         blexit(L"No initial domain kernel specified.");
 
+    /*
+     * The Dom0 kernel can be loaded from the configuration file or by the
+     * device tree through the efi_check_dt_boot function, in this stage
+     * verify it.
+     */
+    if ( kernel.ptr &&
+         !EFI_ERROR(efi_bs->LocateProtocol(&shim_lock_guid, NULL,
+                                           (void **)&shim_lock)) &&
+         (status = shim_lock->Verify(kernel.ptr, kernel.size)) != EFI_SUCCESS )
+        PrintErrMesg(L"Dom0 kernel image could not be verified", status);
+
     efi_arch_edd();
 
     /* XXX Collect EDID info. */
-- 
2.17.1

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

On Mon, 11 Oct 2021, Luca Fancellu wrote:
> Add support to load Dom0 boot modules from
> the device tree using the xen,uefi-binary property.
> 
> Update documentation about that.
> 
> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>

Unfortunately, due to the change to the previous patch, this patch now
has one issue, see below.


> @@ -754,6 +760,41 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>          return ERROR_SET_REG_PROPERTY;
>      }
>  
> +    if ( !is_domu_module )
> +    {
> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
> +                                    "multiboot,kernel") == 0) )
> +        {
> +            /*
> +            * This is the Dom0 kernel, wire it to the kernel variable because it
> +            * will be verified by the shim lock protocol later in the common
> +            * code.
> +            */
> +            if ( kernel.addr )
> +            {
> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
> +                return ERROR_DOM0_ALREADY_FOUND;
> +            }
> +            kernel.need_to_free = false; /* Freed using the module array */
> +            kernel.addr = file->addr;
> +            kernel.size = file->size;
> +        }
> +        else if ( ramdisk.addr &&
> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> +                                             "multiboot,ramdisk") == 0) )
> +        {
> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> +            return ERROR_DOM0_RAMDISK_FOUND;
> +        }
> +        else if ( xsm.addr &&
> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> +                                             "xen,xsm-policy") == 0) )
> +        {
> +            PrintMessage(L"XSM policy already found in cfg file.");
> +            return ERROR_XSM_ALREADY_FOUND;
> +        }
> +    }
> +
>      return 0;
>  }
>  
> @@ -793,7 +834,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>            module_node = fdt_next_subnode(fdt, module_node) )
>      {
>          int ret = handle_module_node(dir_handle, module_node, addr_cells,
> -                                        size_cells);
> +                                     size_cells, true);
>          if ( ret < 0 )
>              return ret;
>      }
> @@ -803,7 +844,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>  
>  /*
>   * This function checks for xen domain nodes under the /chosen node for possible
> - * domU guests to be loaded.
> + * dom0 and domU guests to be loaded.
>   * Returns the number of modules loaded or a negative number for error.
>   */
>  static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> @@ -830,6 +871,9 @@ static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>              if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>                  return ERROR_DT_MODULE_DOMU;
>          }
> +        else if ( handle_module_node(dir_handle, node, addr_len, size_len,
> +                                     false) < 0 )
> +                 return ERROR_DT_MODULE_DOM0;
>      }

handle_module_node comes with a "multiboot,module" compatible check now,
which is fine for dom0less DomU modules, but it is not OK for dom0
modules.

That is because it is also possible to write the dom0 modules (kernel
and ramdisk) with the following compabile strings:

/chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module"
/chosen/dom0-ramdisk compatible "xen,linux-initrd" "xen,multiboot-module"

They are legacy but we are not meant to break support for them. Also
third party tools might still use them -- I checked and even
ImageBuilder still uses them.

One way to solve the problem is to make the "multiboot,module"
compatible check at the beginning of handle_module_node conditional on
!is_domu_module.

Or maybe just ignore compabible errors if !is_domu_module. Something like:

diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 840728d6c0..cbfcd55449 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -721,7 +721,7 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
         /* Error while checking the compatible string */
         return ERROR_CHECK_MODULE_COMPAT;
 
-    if ( module_compat != 0 )
+    if ( is_domu_module && module_compat != 0 )
         /* Module is not a multiboot,module */
         return 0;
 

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Luca Fancellu]

> On 11 Oct 2021, at 20:53, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Mon, 11 Oct 2021, Luca Fancellu wrote:
>> Add support to load Dom0 boot modules from
>> the device tree using the xen,uefi-binary property.
>> 
>> Update documentation about that.
>> 
>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> 

Hi Stefano,

> Unfortunately, due to the change to the previous patch, this patch now
> has one issue, see below.
> 
> 
>> @@ -754,6 +760,41 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>>         return ERROR_SET_REG_PROPERTY;
>>     }
>> 
>> +    if ( !is_domu_module )
>> +    {
>> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
>> +                                    "multiboot,kernel") == 0) )
>> +        {
>> +            /*
>> +            * This is the Dom0 kernel, wire it to the kernel variable because it
>> +            * will be verified by the shim lock protocol later in the common
>> +            * code.
>> +            */
>> +            if ( kernel.addr )
>> +            {
>> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
>> +                return ERROR_DOM0_ALREADY_FOUND;
>> +            }
>> +            kernel.need_to_free = false; /* Freed using the module array */
>> +            kernel.addr = file->addr;
>> +            kernel.size = file->size;
>> +        }
>> +        else if ( ramdisk.addr &&
>> +                  (fdt_node_check_compatible(fdt, module_node_offset,
>> +                                             "multiboot,ramdisk") == 0) )
>> +        {
>> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
>> +            return ERROR_DOM0_RAMDISK_FOUND;
>> +        }
>> +        else if ( xsm.addr &&
>> +                  (fdt_node_check_compatible(fdt, module_node_offset,
>> +                                             "xen,xsm-policy") == 0) )
>> +        {
>> +            PrintMessage(L"XSM policy already found in cfg file.");
>> +            return ERROR_XSM_ALREADY_FOUND;
>> +        }
>> +    }
>> +
>>     return 0;
>> }
>> 
>> @@ -793,7 +834,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>>           module_node = fdt_next_subnode(fdt, module_node) )
>>     {
>>         int ret = handle_module_node(dir_handle, module_node, addr_cells,
>> -                                        size_cells);
>> +                                     size_cells, true);
>>         if ( ret < 0 )
>>             return ret;
>>     }
>> @@ -803,7 +844,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>> 
>> /*
>>  * This function checks for xen domain nodes under the /chosen node for possible
>> - * domU guests to be loaded.
>> + * dom0 and domU guests to be loaded.
>>  * Returns the number of modules loaded or a negative number for error.
>>  */
>> static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>> @@ -830,6 +871,9 @@ static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
>>             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
>>                 return ERROR_DT_MODULE_DOMU;
>>         }
>> +        else if ( handle_module_node(dir_handle, node, addr_len, size_len,
>> +                                     false) < 0 )
>> +                 return ERROR_DT_MODULE_DOM0;
>>     }
> 
> handle_module_node comes with a "multiboot,module" compatible check now,
> which is fine for dom0less DomU modules, but it is not OK for dom0
> modules.
> 
> That is because it is also possible to write the dom0 modules (kernel
> and ramdisk) with the following compabile strings:
> 
> /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module"
> /chosen/dom0-ramdisk compatible "xen,linux-initrd" "xen,multiboot-module"

Oh ok I’m surprised because I think even before I was not managing any module
with “xen,multiboot-module”, just any multiboot,{kernel,ramdisk,device-tree}

> 
> They are legacy but we are not meant to break support for them. Also
> third party tools might still use them -- I checked and even
> ImageBuilder still uses them.
> 
> One way to solve the problem is to make the "multiboot,module"
> compatible check at the beginning of handle_module_node conditional on
> !is_domu_module.
> 
> Or maybe just ignore compabible errors if !is_domu_module. Something like:
> 
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 840728d6c0..cbfcd55449 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -721,7 +721,7 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>         /* Error while checking the compatible string */
>         return ERROR_CHECK_MODULE_COMPAT;
> 
> -    if ( module_compat != 0 )
> +    if ( is_domu_module && module_compat != 0 )
>         /* Module is not a multiboot,module */
>         return 0;
> 

I can be ok with this change but it means that any node under chosen that is not a “xen,domain”
will be handled as it is a Dom0 boot module (if it has xen,uefi-binary), is it always true?

Otherwise I can do these changes:

--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -721,10 +721,19 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
         /* Error while checking the compatible string */
         return ERROR_CHECK_MODULE_COMPAT;
 
-    if ( module_compat != 0 )
+    if ( is_domu_module && (module_compat != 0) )
         /* Module is not a multiboot,module */
         return 0;
 
+    /*
+     * For Dom0 boot modules can be specified also using the legacy compatible
+     * xen,multiboot-module
+     */
+    if ( !is_domu_module && module_compat &&
+         (fdt_node_check_compatible(fdt, module_node_offset,
+                                    "xen,multiboot-module") != 0) )
+         return 0;
+
     /* Read xen,uefi-binary property to get the file name. */
     uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
                                  &uefi_name_len);
@@ -763,7 +772,9 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
     if ( !is_domu_module )
     {
         if ( (fdt_node_check_compatible(fdt, module_node_offset,
-                                    "multiboot,kernel") == 0) )
+                                        "multiboot,kernel") == 0) ||
+             (fdt_node_check_compatible(fdt, module_node_offset,
+                                        "xen,linux-zimage") == 0) )
         {
             /*
             * This is the Dom0 kernel, wire it to the kernel variable because it
@@ -780,8 +791,10 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
             kernel.size = file->size;
         }
         else if ( ramdisk.addr &&
-                  (fdt_node_check_compatible(fdt, module_node_offset,
-                                             "multiboot,ramdisk") == 0) )
+                  ((fdt_node_check_compatible(fdt, module_node_offset,
+                                              "multiboot,ramdisk") == 0) ||
+                   (fdt_node_check_compatible(fdt, module_node_offset,
+                                              "xen,linux-initrd") == 0)) )
         {
             PrintMessage(L"Dom0 ramdisk already found in cfg file.");
             return ERROR_DOM0_RAMDISK_FOUND;


I would need to check for “xen,linux-zimage” and “xen,linux-initrd” however
to be sure the user is not specifying the kernel and ramdisk twice.

Please let me know if you agree or if there is some issue with them.

Cheers,
Luca

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 9405 bytes --]

On Mon, 11 Oct 2021, Luca Fancellu wrote:
> > On 11 Oct 2021, at 20:53, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Mon, 11 Oct 2021, Luca Fancellu wrote:
> >> Add support to load Dom0 boot modules from
> >> the device tree using the xen,uefi-binary property.
> >> 
> >> Update documentation about that.
> >> 
> >> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> > 
> 
> Hi Stefano,
> 
> > Unfortunately, due to the change to the previous patch, this patch now
> > has one issue, see below.
> > 
> > 
> >> @@ -754,6 +760,41 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> >>         return ERROR_SET_REG_PROPERTY;
> >>     }
> >> 
> >> +    if ( !is_domu_module )
> >> +    {
> >> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
> >> +                                    "multiboot,kernel") == 0) )
> >> +        {
> >> +            /*
> >> +            * This is the Dom0 kernel, wire it to the kernel variable because it
> >> +            * will be verified by the shim lock protocol later in the common
> >> +            * code.
> >> +            */
> >> +            if ( kernel.addr )
> >> +            {
> >> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
> >> +                return ERROR_DOM0_ALREADY_FOUND;
> >> +            }
> >> +            kernel.need_to_free = false; /* Freed using the module array */
> >> +            kernel.addr = file->addr;
> >> +            kernel.size = file->size;
> >> +        }
> >> +        else if ( ramdisk.addr &&
> >> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> >> +                                             "multiboot,ramdisk") == 0) )
> >> +        {
> >> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> >> +            return ERROR_DOM0_RAMDISK_FOUND;
> >> +        }
> >> +        else if ( xsm.addr &&
> >> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> >> +                                             "xen,xsm-policy") == 0) )
> >> +        {
> >> +            PrintMessage(L"XSM policy already found in cfg file.");
> >> +            return ERROR_XSM_ALREADY_FOUND;
> >> +        }
> >> +    }
> >> +
> >>     return 0;
> >> }
> >> 
> >> @@ -793,7 +834,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> >>           module_node = fdt_next_subnode(fdt, module_node) )
> >>     {
> >>         int ret = handle_module_node(dir_handle, module_node, addr_cells,
> >> -                                        size_cells);
> >> +                                     size_cells, true);
> >>         if ( ret < 0 )
> >>             return ret;
> >>     }
> >> @@ -803,7 +844,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> >> 
> >> /*
> >>  * This function checks for xen domain nodes under the /chosen node for possible
> >> - * domU guests to be loaded.
> >> + * dom0 and domU guests to be loaded.
> >>  * Returns the number of modules loaded or a negative number for error.
> >>  */
> >> static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> >> @@ -830,6 +871,9 @@ static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> >>             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> >>                 return ERROR_DT_MODULE_DOMU;
> >>         }
> >> +        else if ( handle_module_node(dir_handle, node, addr_len, size_len,
> >> +                                     false) < 0 )
> >> +                 return ERROR_DT_MODULE_DOM0;
> >>     }
> > 
> > handle_module_node comes with a "multiboot,module" compatible check now,
> > which is fine for dom0less DomU modules, but it is not OK for dom0
> > modules.
> > 
> > That is because it is also possible to write the dom0 modules (kernel
> > and ramdisk) with the following compabile strings:
> > 
> > /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module"
> > /chosen/dom0-ramdisk compatible "xen,linux-initrd" "xen,multiboot-module"
> 
> Oh ok I’m surprised because I think even before I was not managing any module
> with “xen,multiboot-module”, just any multiboot,{kernel,ramdisk,device-tree}

At least by looking at the code it seemed to work before, although we
weren't explicitly checking for this case 

 
> > They are legacy but we are not meant to break support for them. Also
> > third party tools might still use them -- I checked and even
> > ImageBuilder still uses them.
> > 
> > One way to solve the problem is to make the "multiboot,module"
> > compatible check at the beginning of handle_module_node conditional on
> > !is_domu_module.
> > 
> > Or maybe just ignore compabible errors if !is_domu_module. Something like:
> > 
> > diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> > index 840728d6c0..cbfcd55449 100644
> > --- a/xen/arch/arm/efi/efi-boot.h
> > +++ b/xen/arch/arm/efi/efi-boot.h
> > @@ -721,7 +721,7 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> >         /* Error while checking the compatible string */
> >         return ERROR_CHECK_MODULE_COMPAT;
> > 
> > -    if ( module_compat != 0 )
> > +    if ( is_domu_module && module_compat != 0 )
> >         /* Module is not a multiboot,module */
> >         return 0;
> > 
> 
> I can be ok with this change but it means that any node under chosen that is not a “xen,domain”
> will be handled as it is a Dom0 boot module (if it has xen,uefi-binary), is it always true?
 
Good point. I don't think it is a safe assumption.


> Otherwise I can do these changes:
> 
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -721,10 +721,19 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>          /* Error while checking the compatible string */
>          return ERROR_CHECK_MODULE_COMPAT;
>  
> -    if ( module_compat != 0 )
> +    if ( is_domu_module && (module_compat != 0) )
>          /* Module is not a multiboot,module */
>          return 0;
>  
> +    /*
> +     * For Dom0 boot modules can be specified also using the legacy compatible
> +     * xen,multiboot-module
> +     */
> +    if ( !is_domu_module && module_compat &&
> +         (fdt_node_check_compatible(fdt, module_node_offset,
> +                                    "xen,multiboot-module") != 0) )
> +         return 0;
> +
>      /* Read xen,uefi-binary property to get the file name. */
>      uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
>                                   &uefi_name_len);
> @@ -763,7 +772,9 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>      if ( !is_domu_module )
>      {
>          if ( (fdt_node_check_compatible(fdt, module_node_offset,
> -                                    "multiboot,kernel") == 0) )
> +                                        "multiboot,kernel") == 0) ||
> +             (fdt_node_check_compatible(fdt, module_node_offset,
> +                                        "xen,linux-zimage") == 0) )
>          {
>              /*
>              * This is the Dom0 kernel, wire it to the kernel variable because it
> @@ -780,8 +791,10 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>              kernel.size = file->size;
>          }
>          else if ( ramdisk.addr &&
> -                  (fdt_node_check_compatible(fdt, module_node_offset,
> -                                             "multiboot,ramdisk") == 0) )
> +                  ((fdt_node_check_compatible(fdt, module_node_offset,
> +                                              "multiboot,ramdisk") == 0) ||
> +                   (fdt_node_check_compatible(fdt, module_node_offset,
> +                                              "xen,linux-initrd") == 0)) )
>          {
>              PrintMessage(L"Dom0 ramdisk already found in cfg file.");
>              return ERROR_DOM0_RAMDISK_FOUND;
> 
> 
> I would need to check for “xen,linux-zimage” and “xen,linux-initrd” however
> to be sure the user is not specifying the kernel and ramdisk twice.
> 
> Please let me know if you agree or if there is some issue with them.

I have another idea: I don't think we need to actually check for
"xen,linux-zimage" or "xen,linux-initrd" because I am pretty sure they
were always used in conjunction with "xen,multiboot-module".

So maybe it is enough to check for:

- for dom0: "xen,multiboot-module"
- domU: "multiboot,module"


E.g.:

diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
index 840728d6c0..076b827bdd 100644
--- a/xen/arch/arm/efi/efi-boot.h
+++ b/xen/arch/arm/efi/efi-boot.h
@@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
     char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
     int uefi_name_len, file_idx, module_compat;
     module_name *file;
+    const char *compat_string = is_domu_module ? "multiboot,module" :
+                                "xen,multiboot-module";
 
     /* Check if the node is a multiboot,module otherwise return */
     module_compat = fdt_node_check_compatible(fdt, module_node_offset,
-                                              "multiboot,module");
+                                              compat_string);
     if ( module_compat < 0 )
         /* Error while checking the compatible string */
         return ERROR_CHECK_MODULE_COMPAT;

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 10119 bytes --]

On Mon, 11 Oct 2021, Stefano Stabellini wrote:
> On Mon, 11 Oct 2021, Luca Fancellu wrote:
> > > On 11 Oct 2021, at 20:53, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > > 
> > > On Mon, 11 Oct 2021, Luca Fancellu wrote:
> > >> Add support to load Dom0 boot modules from
> > >> the device tree using the xen,uefi-binary property.
> > >> 
> > >> Update documentation about that.
> > >> 
> > >> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
> > > 
> > 
> > Hi Stefano,
> > 
> > > Unfortunately, due to the change to the previous patch, this patch now
> > > has one issue, see below.
> > > 
> > > 
> > >> @@ -754,6 +760,41 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> > >>         return ERROR_SET_REG_PROPERTY;
> > >>     }
> > >> 
> > >> +    if ( !is_domu_module )
> > >> +    {
> > >> +        if ( (fdt_node_check_compatible(fdt, module_node_offset,
> > >> +                                    "multiboot,kernel") == 0) )
> > >> +        {
> > >> +            /*
> > >> +            * This is the Dom0 kernel, wire it to the kernel variable because it
> > >> +            * will be verified by the shim lock protocol later in the common
> > >> +            * code.
> > >> +            */
> > >> +            if ( kernel.addr )
> > >> +            {
> > >> +                PrintMessage(L"Dom0 kernel already found in cfg file.");
> > >> +                return ERROR_DOM0_ALREADY_FOUND;
> > >> +            }
> > >> +            kernel.need_to_free = false; /* Freed using the module array */
> > >> +            kernel.addr = file->addr;
> > >> +            kernel.size = file->size;
> > >> +        }
> > >> +        else if ( ramdisk.addr &&
> > >> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> > >> +                                             "multiboot,ramdisk") == 0) )
> > >> +        {
> > >> +            PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> > >> +            return ERROR_DOM0_RAMDISK_FOUND;
> > >> +        }
> > >> +        else if ( xsm.addr &&
> > >> +                  (fdt_node_check_compatible(fdt, module_node_offset,
> > >> +                                             "xen,xsm-policy") == 0) )
> > >> +        {
> > >> +            PrintMessage(L"XSM policy already found in cfg file.");
> > >> +            return ERROR_XSM_ALREADY_FOUND;
> > >> +        }
> > >> +    }
> > >> +
> > >>     return 0;
> > >> }
> > >> 
> > >> @@ -793,7 +834,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> > >>           module_node = fdt_next_subnode(fdt, module_node) )
> > >>     {
> > >>         int ret = handle_module_node(dir_handle, module_node, addr_cells,
> > >> -                                        size_cells);
> > >> +                                     size_cells, true);
> > >>         if ( ret < 0 )
> > >>             return ret;
> > >>     }
> > >> @@ -803,7 +844,7 @@ static int __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
> > >> 
> > >> /*
> > >>  * This function checks for xen domain nodes under the /chosen node for possible
> > >> - * domU guests to be loaded.
> > >> + * dom0 and domU guests to be loaded.
> > >>  * Returns the number of modules loaded or a negative number for error.
> > >>  */
> > >> static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> > >> @@ -830,6 +871,9 @@ static int __init efi_check_dt_boot(EFI_FILE_HANDLE dir_handle)
> > >>             if ( handle_dom0less_domain_node(dir_handle, node) < 0 )
> > >>                 return ERROR_DT_MODULE_DOMU;
> > >>         }
> > >> +        else if ( handle_module_node(dir_handle, node, addr_len, size_len,
> > >> +                                     false) < 0 )
> > >> +                 return ERROR_DT_MODULE_DOM0;
> > >>     }
> > > 
> > > handle_module_node comes with a "multiboot,module" compatible check now,
> > > which is fine for dom0less DomU modules, but it is not OK for dom0
> > > modules.
> > > 
> > > That is because it is also possible to write the dom0 modules (kernel
> > > and ramdisk) with the following compabile strings:
> > > 
> > > /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module"
> > > /chosen/dom0-ramdisk compatible "xen,linux-initrd" "xen,multiboot-module"
> > 
> > Oh ok I’m surprised because I think even before I was not managing any module
> > with “xen,multiboot-module”, just any multiboot,{kernel,ramdisk,device-tree}
> 
> At least by looking at the code it seemed to work before, although we
> weren't explicitly checking for this case 
> 
>  
> > > They are legacy but we are not meant to break support for them. Also
> > > third party tools might still use them -- I checked and even
> > > ImageBuilder still uses them.
> > > 
> > > One way to solve the problem is to make the "multiboot,module"
> > > compatible check at the beginning of handle_module_node conditional on
> > > !is_domu_module.
> > > 
> > > Or maybe just ignore compabible errors if !is_domu_module. Something like:
> > > 
> > > diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> > > index 840728d6c0..cbfcd55449 100644
> > > --- a/xen/arch/arm/efi/efi-boot.h
> > > +++ b/xen/arch/arm/efi/efi-boot.h
> > > @@ -721,7 +721,7 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> > >         /* Error while checking the compatible string */
> > >         return ERROR_CHECK_MODULE_COMPAT;
> > > 
> > > -    if ( module_compat != 0 )
> > > +    if ( is_domu_module && module_compat != 0 )
> > >         /* Module is not a multiboot,module */
> > >         return 0;
> > > 
> > 
> > I can be ok with this change but it means that any node under chosen that is not a “xen,domain”
> > will be handled as it is a Dom0 boot module (if it has xen,uefi-binary), is it always true?
>  
> Good point. I don't think it is a safe assumption.
> 
> 
> > Otherwise I can do these changes:
> > 
> > --- a/xen/arch/arm/efi/efi-boot.h
> > +++ b/xen/arch/arm/efi/efi-boot.h
> > @@ -721,10 +721,19 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> >          /* Error while checking the compatible string */
> >          return ERROR_CHECK_MODULE_COMPAT;
> >  
> > -    if ( module_compat != 0 )
> > +    if ( is_domu_module && (module_compat != 0) )
> >          /* Module is not a multiboot,module */
> >          return 0;
> >  
> > +    /*
> > +     * For Dom0 boot modules can be specified also using the legacy compatible
> > +     * xen,multiboot-module
> > +     */
> > +    if ( !is_domu_module && module_compat &&
> > +         (fdt_node_check_compatible(fdt, module_node_offset,
> > +                                    "xen,multiboot-module") != 0) )
> > +         return 0;
> > +
> >      /* Read xen,uefi-binary property to get the file name. */
> >      uefi_name_prop = fdt_getprop(fdt, module_node_offset, "xen,uefi-binary",
> >                                   &uefi_name_len);
> > @@ -763,7 +772,9 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> >      if ( !is_domu_module )
> >      {
> >          if ( (fdt_node_check_compatible(fdt, module_node_offset,
> > -                                    "multiboot,kernel") == 0) )
> > +                                        "multiboot,kernel") == 0) ||
> > +             (fdt_node_check_compatible(fdt, module_node_offset,
> > +                                        "xen,linux-zimage") == 0) )
> >          {
> >              /*
> >              * This is the Dom0 kernel, wire it to the kernel variable because it
> > @@ -780,8 +791,10 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
> >              kernel.size = file->size;
> >          }
> >          else if ( ramdisk.addr &&
> > -                  (fdt_node_check_compatible(fdt, module_node_offset,
> > -                                             "multiboot,ramdisk") == 0) )
> > +                  ((fdt_node_check_compatible(fdt, module_node_offset,
> > +                                              "multiboot,ramdisk") == 0) ||
> > +                   (fdt_node_check_compatible(fdt, module_node_offset,
> > +                                              "xen,linux-initrd") == 0)) )
> >          {
> >              PrintMessage(L"Dom0 ramdisk already found in cfg file.");
> >              return ERROR_DOM0_RAMDISK_FOUND;
> > 
> > 
> > I would need to check for “xen,linux-zimage” and “xen,linux-initrd” however
> > to be sure the user is not specifying the kernel and ramdisk twice.
> > 
> > Please let me know if you agree or if there is some issue with them.
> 
> I have another idea: I don't think we need to actually check for
> "xen,linux-zimage" or "xen,linux-initrd" because I am pretty sure they
> were always used in conjunction with "xen,multiboot-module".
> 
> So maybe it is enough to check for:
> 
> - for dom0: "xen,multiboot-module"
> - domU: "multiboot,module"
> 
> 
> E.g.:
> 
> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> index 840728d6c0..076b827bdd 100644
> --- a/xen/arch/arm/efi/efi-boot.h
> +++ b/xen/arch/arm/efi/efi-boot.h
> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
>      int uefi_name_len, file_idx, module_compat;
>      module_name *file;
> +    const char *compat_string = is_domu_module ? "multiboot,module" :
> +                                "xen,multiboot-module";
>  
>      /* Check if the node is a multiboot,module otherwise return */
>      module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> -                                              "multiboot,module");
> +                                              compat_string);
>      if ( module_compat < 0 )
>          /* Error while checking the compatible string */
>          return ERROR_CHECK_MODULE_COMPAT;


Well... not exactly like this because this would stop a normal
"multiboot,module" dom0 kernel from being recognized.

So we need for domU: only "multiboot,module"
For Dom0, either "multiboot,module" or "xen,multiboot-module"

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Julien Grall]

Hi Stefano,

On 11/10/2021 22:24, Stefano Stabellini wrote:
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index 840728d6c0..076b827bdd 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE dir_handle,
>>       char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 */
>>       int uefi_name_len, file_idx, module_compat;
>>       module_name *file;
>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
>> +                                "xen,multiboot-module";
>>   
>>       /* Check if the node is a multiboot,module otherwise return */
>>       module_compat = fdt_node_check_compatible(fdt, module_node_offset,
>> -                                              "multiboot,module");
>> +                                              compat_string);
>>       if ( module_compat < 0 )
>>           /* Error while checking the compatible string */
>>           return ERROR_CHECK_MODULE_COMPAT;
> 
> 
> Well... not exactly like this because this would stop a normal
> "multiboot,module" dom0 kernel from being recognized.
> 
> So we need for domU: only "multiboot,module"
> For Dom0, either "multiboot,module" or "xen,multiboot-module"

Looking at the history, xen,multiboot-module has been considered as a 
legacy binding since before UEFI was introduced. In fact, without this 
series, I believe, there is limited reasons for the compatible to be 
present in the DT as you would either use grub (which use the new 
compatible) or xen.cfg (the stub will create the node).

So I would argue that this compatible should not be used in combination 
with UEFI and therefore we should not handle it. This would make the 
code simpler :).

Cheers,

-- 
Julien Grall

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

On Mon, 11 Oct 2021, Julien Grall wrote:
> Hi Stefano,
> 
> On 11/10/2021 22:24, Stefano Stabellini wrote:
> > > diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> > > index 840728d6c0..076b827bdd 100644
> > > --- a/xen/arch/arm/efi/efi-boot.h
> > > +++ b/xen/arch/arm/efi/efi-boot.h
> > > @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
> > > dir_handle,
> > >       char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
> > > \0 */
> > >       int uefi_name_len, file_idx, module_compat;
> > >       module_name *file;
> > > +    const char *compat_string = is_domu_module ? "multiboot,module" :
> > > +                                "xen,multiboot-module";
> > >         /* Check if the node is a multiboot,module otherwise return */
> > >       module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> > > -                                              "multiboot,module");
> > > +                                              compat_string);
> > >       if ( module_compat < 0 )
> > >           /* Error while checking the compatible string */
> > >           return ERROR_CHECK_MODULE_COMPAT;
> > 
> > 
> > Well... not exactly like this because this would stop a normal
> > "multiboot,module" dom0 kernel from being recognized.
> > 
> > So we need for domU: only "multiboot,module"
> > For Dom0, either "multiboot,module" or "xen,multiboot-module"
> 
> Looking at the history, xen,multiboot-module has been considered as a legacy
> binding since before UEFI was introduced. In fact, without this series, I
> believe, there is limited reasons for the compatible to be present in the DT
> as you would either use grub (which use the new compatible) or xen.cfg (the
> stub will create the node).
> 
> So I would argue that this compatible should not be used in combination with
> UEFI and therefore we should not handle it. This would make the code simpler
> :).

What you suggested is a viable option, however ImageBuilder is still
using the "xen,multiboot-module" format somehow today (no idea why) and
we have the following written in docs/misc/arm/device-tree/booting.txt:

	Xen 4.4 supported a different set of legacy compatible strings
	which remain supported such that systems supporting both 4.4
	and later can use a single DTB.

	- "xen,multiboot-module" equivalent to "multiboot,module"
	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"

	For compatibility with Xen 4.4 the more specific "xen,linux-*"
	names are non-optional and must be included.

My preference is to avoid breaking compatibility (even with UEFI
booting). The way I suggested above is one way to do it.

But I don't feel strongly about this at all, I am fine with ignoring
"xen,multiboot-module" in the EFI stub. I can get ImageBuilder fixed
very quickly (I should do that in any case). If we are going to ignore
"xen,multiboot-module" then we probably want to update the text in
docs/misc/arm/device-tree/booting.txt also.

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Luca Fancellu]

> On 12 Oct 2021, at 02:31, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> On Mon, 11 Oct 2021, Julien Grall wrote:
>> Hi Stefano,
>> 
>> On 11/10/2021 22:24, Stefano Stabellini wrote:
>>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>>>> index 840728d6c0..076b827bdd 100644
>>>> --- a/xen/arch/arm/efi/efi-boot.h
>>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
>>>> dir_handle,
>>>>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
>>>> \0 */
>>>>      int uefi_name_len, file_idx, module_compat;
>>>>      module_name *file;
>>>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
>>>> +                                "xen,multiboot-module";
>>>>        /* Check if the node is a multiboot,module otherwise return */
>>>>      module_compat = fdt_node_check_compatible(fdt, module_node_offset,
>>>> -                                              "multiboot,module");
>>>> +                                              compat_string);
>>>>      if ( module_compat < 0 )
>>>>          /* Error while checking the compatible string */
>>>>          return ERROR_CHECK_MODULE_COMPAT;
>>> 
>>> 
>>> Well... not exactly like this because this would stop a normal
>>> "multiboot,module" dom0 kernel from being recognized.
>>> 
>>> So we need for domU: only "multiboot,module"
>>> For Dom0, either "multiboot,module" or "xen,multiboot-module"
>> 
>> Looking at the history, xen,multiboot-module has been considered as a legacy
>> binding since before UEFI was introduced. In fact, without this series, I
>> believe, there is limited reasons for the compatible to be present in the DT
>> as you would either use grub (which use the new compatible) or xen.cfg (the
>> stub will create the node).
>> 
>> So I would argue that this compatible should not be used in combination with
>> UEFI and therefore we should not handle it. This would make the code simpler
>> :).
> 

Hi Stefano,

> What you suggested is a viable option, however ImageBuilder is still
> using the "xen,multiboot-module" format somehow today (no idea why) and
> we have the following written in docs/misc/arm/device-tree/booting.txt:
> 
> 	Xen 4.4 supported a different set of legacy compatible strings
> 	which remain supported such that systems supporting both 4.4
> 	and later can use a single DTB.
> 
> 	- "xen,multiboot-module" equivalent to "multiboot,module"
> 	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
> 	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"
> 
> 	For compatibility with Xen 4.4 the more specific "xen,linux-*"
> 	names are non-optional and must be included.
> 
> My preference is to avoid breaking compatibility (even with UEFI
> booting). The way I suggested above is one way to do it.
> 
> But I don't feel strongly about this at all, I am fine with ignoring
> "xen,multiboot-module" in the EFI stub. I can get ImageBuilder fixed
> very quickly (I should do that in any case). If we are going to ignore
> "xen,multiboot-module" then we probably want to update the text in
> docs/misc/arm/device-tree/booting.txt also.

The changes to support legacy compatible strings can be done but it will result in
complex code, I would go for Julien suggestion to just drop it for UEFI.

I can add a note on docs/misc/arm/device-tree/booting.txt saying that for UEFI boot
the legacy strings are not supported.

Something like:

--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -51,6 +51,8 @@ Each node contains the following properties:
        Xen 4.4 supported a different set of legacy compatible strings
        which remain supported such that systems supporting both 4.4
        and later can use a single DTB.
+       However when booting Xen using UEFI and Device Tree, the legacy compatible
+       strings are not supported.
 
        - "xen,multiboot-module" equivalent to "multiboot,module"
        - "xen,linux-zimage"     equivalent to "multiboot,kernel”


What do you think about that?

Cheers,
Luca

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 4525 bytes --]

On Tue, 12 Oct 2021, Luca Fancellu wrote:
> > On 12 Oct 2021, at 02:31, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > 
> > On Mon, 11 Oct 2021, Julien Grall wrote:
> >> Hi Stefano,
> >> 
> >> On 11/10/2021 22:24, Stefano Stabellini wrote:
> >>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> >>>> index 840728d6c0..076b827bdd 100644
> >>>> --- a/xen/arch/arm/efi/efi-boot.h
> >>>> +++ b/xen/arch/arm/efi/efi-boot.h
> >>>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
> >>>> dir_handle,
> >>>>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
> >>>> \0 */
> >>>>      int uefi_name_len, file_idx, module_compat;
> >>>>      module_name *file;
> >>>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
> >>>> +                                "xen,multiboot-module";
> >>>>        /* Check if the node is a multiboot,module otherwise return */
> >>>>      module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> >>>> -                                              "multiboot,module");
> >>>> +                                              compat_string);
> >>>>      if ( module_compat < 0 )
> >>>>          /* Error while checking the compatible string */
> >>>>          return ERROR_CHECK_MODULE_COMPAT;
> >>> 
> >>> 
> >>> Well... not exactly like this because this would stop a normal
> >>> "multiboot,module" dom0 kernel from being recognized.
> >>> 
> >>> So we need for domU: only "multiboot,module"
> >>> For Dom0, either "multiboot,module" or "xen,multiboot-module"
> >> 
> >> Looking at the history, xen,multiboot-module has been considered as a legacy
> >> binding since before UEFI was introduced. In fact, without this series, I
> >> believe, there is limited reasons for the compatible to be present in the DT
> >> as you would either use grub (which use the new compatible) or xen.cfg (the
> >> stub will create the node).
> >> 
> >> So I would argue that this compatible should not be used in combination with
> >> UEFI and therefore we should not handle it. This would make the code simpler
> >> :).
> > 
> 
> Hi Stefano,
> 
> > What you suggested is a viable option, however ImageBuilder is still
> > using the "xen,multiboot-module" format somehow today (no idea why) and
> > we have the following written in docs/misc/arm/device-tree/booting.txt:
> > 
> > 	Xen 4.4 supported a different set of legacy compatible strings
> > 	which remain supported such that systems supporting both 4.4
> > 	and later can use a single DTB.
> > 
> > 	- "xen,multiboot-module" equivalent to "multiboot,module"
> > 	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
> > 	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"
> > 
> > 	For compatibility with Xen 4.4 the more specific "xen,linux-*"
> > 	names are non-optional and must be included.
> > 
> > My preference is to avoid breaking compatibility (even with UEFI
> > booting). The way I suggested above is one way to do it.
> > 
> > But I don't feel strongly about this at all, I am fine with ignoring
> > "xen,multiboot-module" in the EFI stub. I can get ImageBuilder fixed
> > very quickly (I should do that in any case). If we are going to ignore
> > "xen,multiboot-module" then we probably want to update the text in
> > docs/misc/arm/device-tree/booting.txt also.
> 
> The changes to support legacy compatible strings can be done but it will result in
> complex code, I would go for Julien suggestion to just drop it for UEFI.
> 
> I can add a note on docs/misc/arm/device-tree/booting.txt saying that for UEFI boot
> the legacy strings are not supported.
> 
> Something like:
> 
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -51,6 +51,8 @@ Each node contains the following properties:
>         Xen 4.4 supported a different set of legacy compatible strings
>         which remain supported such that systems supporting both 4.4
>         and later can use a single DTB.
> +       However when booting Xen using UEFI and Device Tree, the legacy compatible
> +       strings are not supported.
>  
>         - "xen,multiboot-module" equivalent to "multiboot,module"
>         - "xen,linux-zimage"     equivalent to "multiboot,kernel”
> 
> 
> What do you think about that?

Also reading Julien's reply, I am fine with a doc-only change in a
separate patch.

Yes, something along those lines is OK.

So for this patch:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Jan Beulich]

On 13.10.2021 02:49, Stefano Stabellini wrote:
> On Tue, 12 Oct 2021, Luca Fancellu wrote:
>>> On 12 Oct 2021, at 02:31, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>>
>>> On Mon, 11 Oct 2021, Julien Grall wrote:
>>>> Hi Stefano,
>>>>
>>>> On 11/10/2021 22:24, Stefano Stabellini wrote:
>>>>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>>>>>> index 840728d6c0..076b827bdd 100644
>>>>>> --- a/xen/arch/arm/efi/efi-boot.h
>>>>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>>>>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
>>>>>> dir_handle,
>>>>>>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
>>>>>> \0 */
>>>>>>      int uefi_name_len, file_idx, module_compat;
>>>>>>      module_name *file;
>>>>>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
>>>>>> +                                "xen,multiboot-module";
>>>>>>        /* Check if the node is a multiboot,module otherwise return */
>>>>>>      module_compat = fdt_node_check_compatible(fdt, module_node_offset,
>>>>>> -                                              "multiboot,module");
>>>>>> +                                              compat_string);
>>>>>>      if ( module_compat < 0 )
>>>>>>          /* Error while checking the compatible string */
>>>>>>          return ERROR_CHECK_MODULE_COMPAT;
>>>>>
>>>>>
>>>>> Well... not exactly like this because this would stop a normal
>>>>> "multiboot,module" dom0 kernel from being recognized.
>>>>>
>>>>> So we need for domU: only "multiboot,module"
>>>>> For Dom0, either "multiboot,module" or "xen,multiboot-module"
>>>>
>>>> Looking at the history, xen,multiboot-module has been considered as a legacy
>>>> binding since before UEFI was introduced. In fact, without this series, I
>>>> believe, there is limited reasons for the compatible to be present in the DT
>>>> as you would either use grub (which use the new compatible) or xen.cfg (the
>>>> stub will create the node).
>>>>
>>>> So I would argue that this compatible should not be used in combination with
>>>> UEFI and therefore we should not handle it. This would make the code simpler
>>>> :).
>>>
>>
>> Hi Stefano,
>>
>>> What you suggested is a viable option, however ImageBuilder is still
>>> using the "xen,multiboot-module" format somehow today (no idea why) and
>>> we have the following written in docs/misc/arm/device-tree/booting.txt:
>>>
>>> 	Xen 4.4 supported a different set of legacy compatible strings
>>> 	which remain supported such that systems supporting both 4.4
>>> 	and later can use a single DTB.
>>>
>>> 	- "xen,multiboot-module" equivalent to "multiboot,module"
>>> 	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
>>> 	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"
>>>
>>> 	For compatibility with Xen 4.4 the more specific "xen,linux-*"
>>> 	names are non-optional and must be included.
>>>
>>> My preference is to avoid breaking compatibility (even with UEFI
>>> booting). The way I suggested above is one way to do it.
>>>
>>> But I don't feel strongly about this at all, I am fine with ignoring
>>> "xen,multiboot-module" in the EFI stub. I can get ImageBuilder fixed
>>> very quickly (I should do that in any case). If we are going to ignore
>>> "xen,multiboot-module" then we probably want to update the text in
>>> docs/misc/arm/device-tree/booting.txt also.
>>
>> The changes to support legacy compatible strings can be done but it will result in
>> complex code, I would go for Julien suggestion to just drop it for UEFI.
>>
>> I can add a note on docs/misc/arm/device-tree/booting.txt saying that for UEFI boot
>> the legacy strings are not supported.
>>
>> Something like:
>>
>> --- a/docs/misc/arm/device-tree/booting.txt
>> +++ b/docs/misc/arm/device-tree/booting.txt
>> @@ -51,6 +51,8 @@ Each node contains the following properties:
>>         Xen 4.4 supported a different set of legacy compatible strings
>>         which remain supported such that systems supporting both 4.4
>>         and later can use a single DTB.
>> +       However when booting Xen using UEFI and Device Tree, the legacy compatible
>> +       strings are not supported.
>>  
>>         - "xen,multiboot-module" equivalent to "multiboot,module"
>>         - "xen,linux-zimage"     equivalent to "multiboot,kernel”
>>
>>
>> What do you think about that?
> 
> Also reading Julien's reply, I am fine with a doc-only change in a
> separate patch.
> 
> Yes, something along those lines is OK.
> 
> So for this patch:
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

Then applicable parts
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 4940 bytes --]

On Wed, 13 Oct 2021, Jan Beulich wrote:
> On 13.10.2021 02:49, Stefano Stabellini wrote:
> > On Tue, 12 Oct 2021, Luca Fancellu wrote:
> >>> On 12 Oct 2021, at 02:31, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >>>
> >>> On Mon, 11 Oct 2021, Julien Grall wrote:
> >>>> Hi Stefano,
> >>>>
> >>>> On 11/10/2021 22:24, Stefano Stabellini wrote:
> >>>>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
> >>>>>> index 840728d6c0..076b827bdd 100644
> >>>>>> --- a/xen/arch/arm/efi/efi-boot.h
> >>>>>> +++ b/xen/arch/arm/efi/efi-boot.h
> >>>>>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
> >>>>>> dir_handle,
> >>>>>>      char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
> >>>>>> \0 */
> >>>>>>      int uefi_name_len, file_idx, module_compat;
> >>>>>>      module_name *file;
> >>>>>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
> >>>>>> +                                "xen,multiboot-module";
> >>>>>>        /* Check if the node is a multiboot,module otherwise return */
> >>>>>>      module_compat = fdt_node_check_compatible(fdt, module_node_offset,
> >>>>>> -                                              "multiboot,module");
> >>>>>> +                                              compat_string);
> >>>>>>      if ( module_compat < 0 )
> >>>>>>          /* Error while checking the compatible string */
> >>>>>>          return ERROR_CHECK_MODULE_COMPAT;
> >>>>>
> >>>>>
> >>>>> Well... not exactly like this because this would stop a normal
> >>>>> "multiboot,module" dom0 kernel from being recognized.
> >>>>>
> >>>>> So we need for domU: only "multiboot,module"
> >>>>> For Dom0, either "multiboot,module" or "xen,multiboot-module"
> >>>>
> >>>> Looking at the history, xen,multiboot-module has been considered as a legacy
> >>>> binding since before UEFI was introduced. In fact, without this series, I
> >>>> believe, there is limited reasons for the compatible to be present in the DT
> >>>> as you would either use grub (which use the new compatible) or xen.cfg (the
> >>>> stub will create the node).
> >>>>
> >>>> So I would argue that this compatible should not be used in combination with
> >>>> UEFI and therefore we should not handle it. This would make the code simpler
> >>>> :).
> >>>
> >>
> >> Hi Stefano,
> >>
> >>> What you suggested is a viable option, however ImageBuilder is still
> >>> using the "xen,multiboot-module" format somehow today (no idea why) and
> >>> we have the following written in docs/misc/arm/device-tree/booting.txt:
> >>>
> >>> 	Xen 4.4 supported a different set of legacy compatible strings
> >>> 	which remain supported such that systems supporting both 4.4
> >>> 	and later can use a single DTB.
> >>>
> >>> 	- "xen,multiboot-module" equivalent to "multiboot,module"
> >>> 	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
> >>> 	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"
> >>>
> >>> 	For compatibility with Xen 4.4 the more specific "xen,linux-*"
> >>> 	names are non-optional and must be included.
> >>>
> >>> My preference is to avoid breaking compatibility (even with UEFI
> >>> booting). The way I suggested above is one way to do it.
> >>>
> >>> But I don't feel strongly about this at all, I am fine with ignoring
> >>> "xen,multiboot-module" in the EFI stub. I can get ImageBuilder fixed
> >>> very quickly (I should do that in any case). If we are going to ignore
> >>> "xen,multiboot-module" then we probably want to update the text in
> >>> docs/misc/arm/device-tree/booting.txt also.
> >>
> >> The changes to support legacy compatible strings can be done but it will result in
> >> complex code, I would go for Julien suggestion to just drop it for UEFI.
> >>
> >> I can add a note on docs/misc/arm/device-tree/booting.txt saying that for UEFI boot
> >> the legacy strings are not supported.
> >>
> >> Something like:
> >>
> >> --- a/docs/misc/arm/device-tree/booting.txt
> >> +++ b/docs/misc/arm/device-tree/booting.txt
> >> @@ -51,6 +51,8 @@ Each node contains the following properties:
> >>         Xen 4.4 supported a different set of legacy compatible strings
> >>         which remain supported such that systems supporting both 4.4
> >>         and later can use a single DTB.
> >> +       However when booting Xen using UEFI and Device Tree, the legacy compatible
> >> +       strings are not supported.
> >>  
> >>         - "xen,multiboot-module" equivalent to "multiboot,module"
> >>         - "xen,linux-zimage"     equivalent to "multiboot,kernel”
> >>
> >>
> >> What do you think about that?
> > 
> > Also reading Julien's reply, I am fine with a doc-only change in a
> > separate patch.
> > 
> > Yes, something along those lines is OK.
> > 
> > So for this patch:
> > 
> > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> Then applicable parts
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks! Patch committed.

Re: [PATCH v6 2/2] arm/efi: load dom0 modules from DT using UEFI

[Julien Grall]

Hi Stefano,

On 12/10/2021 02:31, Stefano Stabellini wrote:
> On Mon, 11 Oct 2021, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 11/10/2021 22:24, Stefano Stabellini wrote:
>>>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>>>> index 840728d6c0..076b827bdd 100644
>>>> --- a/xen/arch/arm/efi/efi-boot.h
>>>> +++ b/xen/arch/arm/efi/efi-boot.h
>>>> @@ -713,10 +713,12 @@ static int __init handle_module_node(EFI_FILE_HANDLE
>>>> dir_handle,
>>>>        char mod_string[24]; /* Placeholder for module@ + a 64-bit number +
>>>> \0 */
>>>>        int uefi_name_len, file_idx, module_compat;
>>>>        module_name *file;
>>>> +    const char *compat_string = is_domu_module ? "multiboot,module" :
>>>> +                                "xen,multiboot-module";
>>>>          /* Check if the node is a multiboot,module otherwise return */
>>>>        module_compat = fdt_node_check_compatible(fdt, module_node_offset,
>>>> -                                              "multiboot,module");
>>>> +                                              compat_string);
>>>>        if ( module_compat < 0 )
>>>>            /* Error while checking the compatible string */
>>>>            return ERROR_CHECK_MODULE_COMPAT;
>>>
>>>
>>> Well... not exactly like this because this would stop a normal
>>> "multiboot,module" dom0 kernel from being recognized.
>>>
>>> So we need for domU: only "multiboot,module"
>>> For Dom0, either "multiboot,module" or "xen,multiboot-module"
>>
>> Looking at the history, xen,multiboot-module has been considered as a legacy
>> binding since before UEFI was introduced. In fact, without this series, I
>> believe, there is limited reasons for the compatible to be present in the DT
>> as you would either use grub (which use the new compatible) or xen.cfg (the
>> stub will create the node).
>>
>> So I would argue that this compatible should not be used in combination with
>> UEFI and therefore we should not handle it. This would make the code simpler
>> :).
> 
> What you suggested is a viable option, however ImageBuilder is still
> using the "xen,multiboot-module" format somehow today (no idea why) and
> we have the following written in docs/misc/arm/device-tree/booting.txt:
> 
> 	Xen 4.4 supported a different set of legacy compatible strings
> 	which remain supported such that systems supporting both 4.4
> 	and later can use a single DTB.
> 
> 	- "xen,multiboot-module" equivalent to "multiboot,module"
> 	- "xen,linux-zimage"     equivalent to "multiboot,kernel"
> 	- "xen,linux-initrd"     equivalent to "multiboot,ramdisk"
> 
> 	For compatibility with Xen 4.4 the more specific "xen,linux-*"
> 	names are non-optional and must be included.
> 
> My preference is to avoid breaking compatibility (even with UEFI
> booting). The way I suggested above is one way to do it.

I understand that from the documentation PoV we claim that the legacy 
bindings is supported in EFI. However, looking at the code, I don't 
think we ever supported them.

Indeed, the function efi_arch_use_config_file() has always only looked 
for nodes contains the compatible "multiboot,module". If there are none 
present, then the stub would require to have a xen.cfg present.

> 
> But I don't feel strongly about this at all, I am fine with ignoring
> "xen,multiboot-module" in the EFI stub.

I think this has always been the case for the past 7 years (or so). This 
leads me to think that nobody ever used UEFI in combination of 
ImageBuilder and therefore I think...

> I can get ImageBuilder fixed
> very quickly (I should do that in any case).

... it is more suitable to fix ImageBuilder over trying to add 
retrospectively a legacy binding in the UEFI stub.

> If we are going to ignore
> "xen,multiboot-module" then we probably want to update the text in
> docs/misc/arm/device-tree/booting.txt also.

I agree the docs probably wants to be updated. Although, I think this 
should happen in a separate patch because this doesn't look a new problem.

Cheers,

-- 
Julien Grall