Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start

[Luca Fancellu <luca.fancellu@arm.com>]

> On 7 Sep 2021, at 13:30, Julien Grall <julien@xen.org> wrote:
> 
> 
> 
> On 07/09/2021 12:51, Luca Fancellu wrote:
>>> On 7 Sep 2021, at 10:35, Julien Grall <julien@xen.org> wrote:
>>> 
>>> Hi Luca,
>>> 
>>> On 07/09/2021 07:52, Luca Fancellu wrote:
>>>> Add a design describing a proposal to improve the EFI
>>>> configuration file, adding keywords to describe domU
>>>> guests and allowing to start a dom0less system.
>>>> Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
>>>> ---
>>>>  docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
>>>>  1 file changed, 105 insertions(+)
>>>>  create mode 100644 docs/designs/efi-arm-dom0less.md
>>>> diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
>>>> new file mode 100644
>>>> index 0000000000..8d8fa2243f
>>>> --- /dev/null
>>>> +++ b/docs/designs/efi-arm-dom0less.md
>>>> @@ -0,0 +1,105 @@
>>>> +# Xen EFI configuration file
>>>> +
>>>> +The current configuration file used by Xen when it is started as an EFI
>>>> +application is considering only the dom0 guest and doesn't have any
>>>> +property to describe and load in memory domU guests.
>>> 
>>> From my understanding, the problem is less about properties (we already have them in the Device-Tree) but more about where are the binaries located in memory as we don't know in advance.
>> Hi Julien,
> Hi Luca,
> 
>> I think I used the wrong word there, I meant “keyword” instead of “property” because I was referring about the
>> lack of keywords to describe a domu guest in the Xen EFI configuration file.
>> I agree with you that on systems with static allocation, the kernel and ramdisk binaries must be at certain locations
>> that are out of control when we use the EFI boot services, the thing we can do is provide a keyword to specify the
>> addresses and then use the CopyMem() function to relocate the kernel/ramdisk in the address we want.
> 
> I wasn't specifically referring to static allocation here, sorry if this wasn't clear. I was pointing out that most of the information you create in the xen.cfg is going to be similar to what we already provide in the Device-Tree.
> 
> My main concern is everytime we add a new feature in Dom0less, a developer would need to write code for the DT and UEFI. This will increase the code size and maintenance.
> 
> The same can be said for the admin as if they want to switch from plain U-boot to UEFI, they would also need to fully rewrite the bindings.
> 
> AFAICT, most of the information provided in the Device-Tree are usable even when using UEFI. So I would prefer if we try to re-use what's existing. This is what my proposal below was about.
> 
>>> 
>>> So I would like to propose something that build on top of the Device-Tree work we did. Note this is early thoughts.
>>> 
>>> The problematic nodes in the DT are:
>>> 
>>>        module@0x4a000000 {
>>>            compatible = "multiboot,kernel", "multiboot,module";
>>>            reg = <0x0 0x4a000000 0xffffff>;
>>>            bootargs = "console=ttyAMA0 init=/bin/sh";
>>>        };
>>> 
>>>        module@0x4b000000 {
>>>            compatible = "multiboot,ramdisk", "multiboot,module";
>>>            reg = <0x0 0x4b000000 0xffffff>;
>>>        };
>>> 
>>> In particular the property "reg" cannot be known in advance because the UEFI stub will be responsible to load the binaries in memory.
>> Yes that’s true, the UEFI stub is using from the UEFI boot service the AllocatePages function that is giving back an address out of our control,
>> then using another function the binary is read from the disk and copied at that address, finally the UEFI stub is writing the node in the device tree that
>> will be used by Xen later.
> 
> I am not sure to follow. Are you saying the UEFI stub will create the dom0less node in the DT based on the xen.cfg?

Yes, in my mind the UEFI stub would have created all necessary node into the chosen {} to create the system, now the stub does that
but only for the dom0 modules.

> 
>>> 
>>> What we could do is providing a list of binaries to load and associate a key for each of them. Something like:
>>> 
>>> binary=<binary> <key>
>>> binary=<binary2> <key2>
>>> ....
>>> 
>>> We can then replace the property "reg" with a new property "uefi,key" that will contain the name of the binary.
>>> 
>>> What do you think?
>> Here I’m lost, because I don’t understand what we are going to do with the name of the binary.
> 
> <binaryX> would be used by the UEFI stub to load the binary in memory. Each binary will have a <keyX> which helps to refer them in the Device-Tree. To give a concrete example, let say we have two dom0less domains:
>  - DomA: 2 vCPUs, 128MB
>  - DomB: 3 vCPUs, 512MB
> 
> DomA and DomB will be using the same kernel but a different ramdisk. xen.cfg, would look like:
> 
> [global]
> default=section1
> 
> [section1]
> options=console=vga,com1 com1=57600 loglvl=all noreboot
> kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
> ramdisk=initrd-3.0.31-0.4-xen
> xsm=<filename>
> dtb=devtree.dtb
> binary=vmlinuz-guest domu-kernel
> binary=ramdisk-domA.img domA-ramdisk
> binary=ramdisk-domB.img domB-ramdisk
> 
> The chosen node in the DT would look like:
> 
> chosen {
>    domU1 {
>        compatible = "xen,domain";
>        #address-cells = <0x2>;
>        #size-cells = <0x1>;
>        memory = <0 0x8000000>;
>        cpus = <2>;
> 
>        module@1 {
>            compatible = "multiboot,kernel", "multiboot,module";
>            uefi,binary = "domu-kernel";
>            bootargs = "console=ttyAMA0 init=/bin/sh";
>        };
> 
>        module@2 {
>            compatible = "multiboot,ramdisk", "multiboot,module";
>            uefi,binary = "domA-ramdisk";
>        };
>    };
> 
>    domU2 {
>        compatible = "xen,domain";
>        #address-cells = <0x3>;
>        #size-cells = <0x1>;
>        memory = <0 0x20000000>;
>        cpus = <3>;
> 
>        module@1 {
>            compatible = "multiboot,kernel", "multiboot,module";
>            uefi,binary = "domu-kernel";
>            bootargs = "console=ttyAMA0 init=/bin/sh";
>        };
> 
>        module@2 {
>            compatible = "multiboot,ramdisk", "multiboot,module";
>            uefi,binary = "domA-ramdisk";
>        };
>    };
> };
> 
> With this approach, the change is quite minimal to move between an classic U-boot boot and EFI boot.

Ok now I see, yes this approach can work and can save some code, in the current code we have that if
a "multiboot,module” is found in the dtb, the Xen EFI configuration file is skipped, but if we use the
module@XX {} without the compatible it can work, the UEFI stub will load the binary and update all
the needed properties (compatible, reg).

Please let me know if we all agree on that

Cheers,

Luca


> 
> Cheers,
> 
> -- 
> Julien Grall