On Fri, 17 Sep 2021, Luca Fancellu wrote: > > On 16 Sep 2021, at 21:16, Stefano Stabellini wrote: > > > > On Thu, 16 Sep 2021, Jan Beulich wrote: > >> On 16.09.2021 17:07, Luca Fancellu wrote: > >>> I explain here my understanding on dom0less, this feature is used to start domUs at > >>> Xen boot in parallel, the name is misleading but it doesn’t require dom0 to be absent. > >>> > >>> So if you have a dom0 kernel embed in the image, it's completely fine to start it and it > >>> doesn’t have to be skipped. > >>> > >>> Here the possible user cases: > >>> 1) start only dom0 [dom0 modules on xen.cfg or embedded in Xen image] > >>> 2) start only domUs, true dom0less [no dom0 modules on xen.cfg or embedded in Xen image, domUs on DT] > >>> 3) start dom0 and domUs [(dom0 modules on xen.cfg or embedded in Xen image) and domUs on DT] > >> > >> If that's the intention - fine. Stefano? > > > > Hi Stefano, > > > What do you mean by dom0 modules embedded in the Xen image? I am not > > familiar with it, but I imagine it doesn't involve any multiboot,module > > nodes in device tree, right? > > > > Putting aside "dom0 modules embedded in Xen image" that I didn't fully > > understand, there are three ways to load Dom0: > > > > - dom0 kernel (and ramdisk, optional) on xen.cfg > > - dom0 kernel (and ramdisk, optional) on device tree using the "reg" property > > - dom0 kernel (and ramdisk, optional) on device tree using the "uefi,binary" property > > True for the #1 and #2, the last one is not implemented. The uefi,binary property > for now is only used to load domU modules. Yeah, it is no problem that is not currently implemented, but from a device tree binding / efi interface perspective it should be possible. > > Then, the other use cases are: > > > > - true dom0less, domUs on device tree using the "reg" property > > - true dom0less, domUs on device tree using the "uefi,binary" property > > > > And of course all the possible combinations between Dom0 and DomU > > loading. > > > > > > Currently, patch #1 checks for the presence of a Dom0 kernel node and, if > > present, it decides not to proceed with xen.cfg. So if the Dom0 kernel > > node is *not* present, efi_arch_use_config_file returns true. > > > > However, this could be a true dom0less configuration without any Dom0 > > kernel. If so, you might not want to load xen.cfg at all because it is > > not needed. In a true dom0less configuration, we probably want > > efi_arch_use_config_file to return false. > > In a true dom0less configuration we might need to read xen.cfg to retrieve the > Xen command line, The Xen command line could also be on device tree (/chosen/xen,xen-bootargs). > but following the actual implementation of the common code > there is more. I’m going to explain. > > What efi_arch_use_config_file really does is not only choosing to read xen.cfg > or not. Following the common code (xen/common/efi/boot.c) and what its result activate > along the path, it basically decides if the UEFI stub is used as a loader from filesystem > or not. We need the UEFI stub as a loader to be 100% UEFI and load our modules. > > The original check basically says “if there are multiboot,module in the DT, then some > bootloader has loaded in memory the required modules so I’m not gonna load anything > from the filesystem because I assume it puts everything in place for me to boot.” OK, I am following. It looks like this is the source of the issue. > >From misc/efi.txt: > When booted as an EFI application, Xen requires a configuration file as described below unless a bootloader, > such as GRUB, has loaded the modules and describes them in the device tree provided to Xen. If a bootloader > provides a device tree containing modules then any configuration files are ignored, and the bootloader is > responsible for populating all relevant device tree nodes. > > What I’m doing in patch #1 is restricting that check to just the multiboot,module that are > Dom0 module, why? Because with the introduction of dom0less we need to specify > multiboot,modules for domUs, but the presence or not of dom0 modules is the only > Information we need to understand if the user decided to start Xen with everything > in places (modules in memory, xen command line, dtb) or if the job is demanded to the > UEFI stub and its configuration file. I don't think so. Imagine a case where the user has everything in device tree, doesn't need xen.cfg, but dom0 and domUs are specified as uefi,binary. We don't want xen.cfg but we do want to be able to load files from the filesystem. This might not be currently implemented but from an bindings perspective it should be possible. > By the configuration file you can also load in memory the Xen dtb, so Xen can > be started as an EFI application without the DTB and then load it using the EFI stub. This can be very useful but it would follow the !fdt check and return true from efi_arch_use_config_file. So it doesn't really conflict with anything we would around multiboot,module and xen,cfg-loading. > I’m not against this new property “xen,cfg-loading”, I just think it is not needed because > we have all the information we need without it and in any case we need to read the > configuration file because otherwise we won’t have access to the Xen command line. We don't necessarely need to read the Xen command line from xen.cfg :-) > Now I’m going to show you examples of all use cases that are valid with the introduction > of this serie: > > 1) Start Xen as EFI application and load only Dom0 > > Xen.cfg: > [global] > default=xen_dom0 > > [xen_dom0] > options= > kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options] > ramdisk=initrd-3.0.31-0.4-xen > dtb= > > DT: > {no modification} > > 2) Start Xen as EFI application to load a true dom0less setup > > Xen.cfg: > [global] > default=xen_true_dom0less > > [xen_true_dom0less] > options= > dtb= > > DT: > chosen { > #size-cells = <0x1>; > #address-cells = <0x1>; > > domU1 { > #size-cells = <0x1>; > #address-cells = <0x1>; > compatible = "xen,domain”; > cpus = <1>; > memory = <0 0xC0000>; > > module@1 { > compatible = "multiboot,kernel", "multiboot,module”; > bootargs = "console=ttyAMA0 root=/dev/ram0 rw”; > uefi,binary = “domU_kernel1.bin" > }; > > module@2 { > compatible = “multiboot,ramdisk", "multiboot,module”; > uefi,binary = “domU_ramdisk1.bin" > }; > > module@3 { > compatible = "multiboot,device-tree", "multiboot,module”; > uefi,binary = “domU_passthrough1.dtb" > }; > }; > > domU2 { }; > } > > 3) Start Xen as EFI application to load Dom0 and DomUs > > Xen.cfg: > [global] > default=xen_dom0_domUs > > [xen_dom0_domUs] > options= > kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options] > ramdisk=initrd-3.0.31-0.4-xen > dtb= > > DT: > chosen { > #size-cells = <0x1>; > #address-cells = <0x1>; > > domU1 { > #size-cells = <0x1>; > #address-cells = <0x1>; > compatible = "xen,domain”; > cpus = <1>; > memory = <0 0xC0000>; > > module@1 { > compatible = "multiboot,kernel", "multiboot,module”; > bootargs = "console=ttyAMA0 root=/dev/ram0 rw”; > uefi,binary = “domU_kernel1.bin" > }; > > module@2 { > compatible = “multiboot,ramdisk", "multiboot,module”; > uefi,binary = “domU_ramdisk1.bin" > }; > > module@3 { > compatible = "multiboot,device-tree", "multiboot,module”; > uefi,binary = “domU_passthrough1.dtb" > }; > }; > > domU2 { }; > } > > So as you see every case is covered without the introduction of the > property. > > Please let me know what do you think. I think that from an interface perspective (not a code perspective) we need to be able to account for cases like: 4) Start Xen as EFI application and load only Dom0 no Xen.cfg DT: xen,xen-bootargs dom0/uefi,binary domUs/uefi,binary But in any case, even disregarding this case, past experience has taught me that it is always better to have an explicit flag to trigger a new behavior, rather than relying on "guesswork". If we introduce "xen,cfg-loading", we are going to be a lot more future-proof that if we don't introduce it in terms of backward and forward compatibility in case we need to change anything.