From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27E74C433F5 for ; Wed, 3 Nov 2021 13:18:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 80199610EA for ; Wed, 3 Nov 2021 13:18:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 80199610EA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:41632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1miGA5-00049W-HM for qemu-devel@archiver.kernel.org; Wed, 03 Nov 2021 09:18:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1miG8u-0003Q0-F2 for qemu-devel@nongnu.org; Wed, 03 Nov 2021 09:17:28 -0400 Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]:41758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1miG8r-0003RM-GK for qemu-devel@nongnu.org; Wed, 03 Nov 2021 09:17:27 -0400 Received: by mail-ed1-x532.google.com with SMTP id ee33so9107421edb.8 for ; Wed, 03 Nov 2021 06:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=to3VWalT1oF8s8W0kCuV93KO1EvHg21dh734ZDF3AaA=; b=oo9j4sQCKbP5KcU2bPOd3CAA5jliX7rShMdbIDLJ25RUxikTN9+nzO7lRWVjmOb//s xxObJCGSOvelf01MxPpKOrg50eUAQLq8L72aab1DfmwzRQKPeV3xSRt0eMxqSkAAeuBD jZMQwYlKMZs9DnyylUB/rvp02+vQacnjqjHT8uakVQO+YId7xisHgYkyvBm5SlVba52d vMMXBm3AfRJyLHncYX0l2WAIk+rjx6TrIx75ROaJF7tZ+K8762r61/xai+apfxmX5HcG TuJjkOGIBDXHiy3rjtzq57vx7i27wR2OMqxVZujxK8GWDUFmw636LFyfQu6yb6Q20mpe 26cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=to3VWalT1oF8s8W0kCuV93KO1EvHg21dh734ZDF3AaA=; b=lud+RaSrPB2fPGjak39zWAXin36xJ06EqXT44n8liDg9HalJMmnW3yUJpgG9Dw2OmS FxN9p8hg2z399OwlE6bCXEDu8qZIOZaMBnDgVYcmR6VXUztfB2jo+99DXTMa+Fpe7vhA S8+7bSEjv/gsFSF0349CMiFWcmSf55QZttK94HtFuDBLu8UcrA68T2UIDUq2jjXtDEiU Z7dZRDAYOqZdDQXs1U2uBZxPClrlvtQ8NtOdxVmel8apekeueODuqqinimQs3arOLbqA NvhxiUVyqU7ldKWqC48pBCOm9pvhQt4/Ca2zD3z7OiD3FSYVQ84WJ7fYW1QVWhoRREfH TjVQ== X-Gm-Message-State: AOAM531ooKRl8RyW084CpsboiVHx9kZkvJLdxTuaT6P3Jo7RAPeGVLCb WPmB/XKQFg0VYVG6ZOLn3ScozkK3eRbUmndsU4sHzA== X-Google-Smtp-Source: ABdhPJyHubuDOvvl030FWLo8+pqEXE6ssozLlSN4us8fQx2VpTXE2RN81N4g3riFy816kMm8x/c9aKR9LtA+1eSW6DU= X-Received: by 2002:a05:6402:2707:: with SMTP id y7mr6527730edd.26.1635945442589; Wed, 03 Nov 2021 06:17:22 -0700 (PDT) MIME-Version: 1.0 References: <20210926183410.256484-1-sjg@chromium.org> <87ilx9jw7t.fsf@linaro.org> In-Reply-To: <87ilx9jw7t.fsf@linaro.org> From: =?UTF-8?Q?Fran=C3=A7ois_Ozog?= Date: Wed, 3 Nov 2021 14:17:11 +0100 Message-ID: Subject: Re: [PATCH] hw/arm/virt: Allow additions to the generated device tree To: =?UTF-8?B?QWxleCBCZW5uw6ll?= Content-Type: multipart/alternative; boundary="000000000000fa63aa05cfe23afb" Received-SPF: pass client-ip=2a00:1450:4864:20::532; envelope-from=francois.ozog@linaro.org; helo=mail-ed1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Eduardo Habkost , Simon Glass , QEMU Developers , qemu-arm@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --000000000000fa63aa05cfe23afb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Thanks Alex to patch me in. I'd like to present another perspective on the motivation as I can't really comment on the actual "how". On real Arm boards, firmware is often assembled into a FIP. That FIP can contain quite a good deal of things, including an NT_FW_CONFIG, NonTrusted_FirmWare_CONFIGuration (NT_FW =3D BL33 which is U-Boot in our case). So the expected typical content for that section is a DT fragment/overlay. That section is to be used in different ways but one is https://trustedfirmware-a.readthedocs.io/en/latest/components/fconf/index.h= tml . For SystemReady systems we will almost inevitably put a device tree fragment in this section and have BL2 merge it with the board DT before handing it over to BL33 (U-Boot is one of them). In some real world examples based on carrier board + som for instance, it may contain SerDes configuration for U-Boot that will result in appropriate PCI lanes or MDIO lanes for the booted OS. So I could say there is precedence in Simon's effort. In any case, when we will have made the changes in TFA for the SystemReady boards we work on, booting the full SystemReady stack (TFA, OP-TEE, U-Boot) on Qemu will allow that late merge based through the FIP. Other boot flows such as VBE (without TFA but with TPL/SPL/U-Boot proper) need a similar facility. Hence I am supporting Simon's proposal at least on the intent. On the how exactly, that is outside my skillset. future comments below On Wed, 3 Nov 2021 at 12:48, Alex Benn=C3=A9e wrot= e: > > Peter Maydell writes: > > > On Mon, 27 Sept 2021 at 16:18, Simon Glass wrote: > >> On Mon, 27 Sept 2021 at 02:48, Peter Maydell > wrote: > >> > So what is missing in the QEMU-provided DTB that it needs? > >> > >> Quite a lot. Here are some examples: > >> > >> U-Boot has limited pre-relocation memory so tries to avoid > >> binding/probing devices that are not used before relocation: > >> > >> > https://u-boot.readthedocs.io/en/latest/develop/driver-model/design.html#= pre-relocation-support > > > > It's up to u-boot to decide what it wants to touch and > > what it does not. QEMU tells u-boot what all the available > > devices are; I don't think we should have extra stuff saying > > "and if you are u-boot, do something odd". > > > >> There is a configuration node (which is likely to change form in > >> future releases, but will still be there) > >> > >> > https://github.com/u-boot/u-boot/blob/master/doc/device-tree-bindings/con= fig.txt > > > > I think u-boot should be storing this kind of thing somewhere > > else (e.g. as part of the binary blob that is u-boot itself, > > or stored in flash or RAM as a separate blob). > > > >> Then there are various features which put things in U-Boot's control > >> dtb, such as verified boot, which adds public keys during signing: > >> > >> > https://github.com/u-boot/u-boot/blob/master/doc/uImage.FIT/signature.txt= #L135 > >> > >> More generally, the U-Boot tree has hundreds of files which add > >> properties for each board, since we try to keep the U-Boot-specific > >> things out of the Linux tree: > >> > >> $ find . -name *u-boot.dtsi |wc -l > >> 398 > > > > If any of this is actual information about the hardware then you > > should sort out getting the bindings documented officially > > (which I think is still in the Linux tree), and then QEMU can > > provide them. > > > >> Quite a bit of this is to do with SPL and so far it seems that QEMU > >> mostly runs U-Boot proper only, although I see that SPL is starting to > >> creep in too in the U-Boot CI. > >> > >> So at present QEMU is not able to support U-Boot fully. > > > > My take is that this is u-boot doing weird custom things with > > the DTB that aren't "describe the hardware". You should be able > > to boot u-boot by putting those custom DTB extra things in a > > separate blob and having u-boot combine that with the > > actual DTB when it starts. > > It's not entirely without precedent - for SPL (which I hope is secondary > program loading) we have things like the guest loader which expands the > plain HW DTB with some information needed by the bootloader and the > primary OS to load additional blobs which have been put into memory. > > In effect the DTB is being expanded as a signalling mechanism similar to > things like fw_cfg and other things we use to control boot up. Whether > this affects the "purity" of DTB as a "just the HW" description is > probably a philosophical question. > > More than a philosophical question: a key aspect of supply chain that nee= d change from quite inflexible and tightly coupled to loosely coupled. A key aspect of it is to maintain "pure" hardware description DTBs at rest= . Composition of DTBs at build time (for products) or runtime (for development boards) should be a simple thing. Another aspect to take into account is System Device Trees. U-Boot only deal with Cortex-As on a platform, so there are multiple device trees for each compute domain. Communities are working on System Device Tree to define the overall platform with its power and clock domains. A tool "lopper" is being developed to slide the SDT into diverse domain DTs. One of them being included into the FIP as the basis for the computing element (Carrier, SoM...). Those attempts to cleanup passed DTBs from configuration data (drivers, bootloaders...) is not incompatible with merging fragments at runtime (for dev boards) or build time (for products). > I agree with Peter that just allowing the merging of arbitrary data into > the QEMU generated DTB is going to lead to confusion and breakages. > Indeed I wrote the guest-loader because instructions for booting Xen up > until that point involved dumpdtb and hand hacking the data which was > silly because this is stuff QEMU already knew about. > > > > > -- PMM > > > -- > Alex Benn=C3=A9e > --=20 Fran=C3=A7ois-Fr=C3=A9d=C3=A9ric Ozog | *Director Business Development* T: +33.67221.6485 francois.ozog@linaro.org | Skype: ffozog --000000000000fa63aa05cfe23afb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Thanks Alex to pat= ch me in.

I'd like to present another perspect= ive on the motivation as I can't really comment on the actual "how= ".

On real Arm boards, firmware is often asse= mbled into a FIP.=C2=A0
That FIP can contain quite a good deal of= things, including an NT_FW_CONFIG, NonTrusted_FirmWare_CONFIGuration (NT_F= W =3D BL33 which is U-Boot in our case).
So the expected typical = content for that section is a DT fragment/overlay.
That section i= s to be used in different ways but one is=C2=A0https://trustedfirmware-a.readthedocs.io/en/latest/components/fconf/i= ndex.html.
For SystemReady systems we will almost inevitably = put a device tree fragment in this section and have BL2 merge it with the b= oard DT before handing it over to BL33 (U-Boot is one of them).
I= n some real world examples based on carrier board + som for instance, it ma= y contain SerDes configuration for U-Boot that will result in appropriate P= CI lanes or MDIO lanes for the booted OS.

So I cou= ld say there is precedence in Simon's effort.

= In any case, when we will have made the changes in TFA for the SystemReady = boards we work on, booting the full SystemReady stack=C2=A0(TFA, OP-TEE, U-= Boot) on Qemu will allow that late merge based through the FIP.
<= br>
Other boot flows such as VBE (without TFA but with TPL/SPL/U-= Boot proper) need a similar facility.=C2=A0

Hence = I am supporting Simon's proposal at least on the intent. On the how exa= ctly, that is outside my skillset.

future comments= below


On Wed, 3 Nov 2021 at 12:48, Alex Benn=C3=A9e &l= t;alex.bennee@l= inaro.org> wrote:

Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 27 Sept 2021 at 16:18, Simon Glass <sjg@chromium.org> wrote:
>> On Mon, 27 Sept 2021 at 02:48, Peter Maydell <peter.maydell@linaro.org&g= t; wrote:
>> > So what is missing in the QEMU-provided DTB that it needs? >>
>> Quite a lot. Here are some examples:
>>
>> U-Boot has limited pre-relocation memory so tries to avoid
>> binding/probing devices that are not used before relocation:
>>
>> https://u-boot.readthedocs.io/en/latest/develop/driver-model/design.htm= l#pre-relocation-support
>
> It's up to u-boot to decide what it wants to touch and
> what it does not. QEMU tells u-boot what all the available
> devices are; I don't think we should have extra stuff saying
> "and if you are u-boot, do something odd".
>
>> There is a configuration node (which is likely to change form in >> future releases, but will still be there)
>>
>> https://git= hub.com/u-boot/u-boot/blob/master/doc/device-tree-bindings/config.txt >
> I think u-boot should be storing this kind of thing somewhere
> else (e.g. as part of the binary blob that is u-boot itself,
> or stored in flash or RAM as a separate blob).
>
>> Then there are various features which put things in U-Boot's c= ontrol
>> dtb, such as verified boot, which adds public keys during signing:=
>>
>> https://githu= b.com/u-boot/u-boot/blob/master/doc/uImage.FIT/signature.txt#L135
>>
>> More generally, the U-Boot tree has hundreds of files which add >> properties for each board, since we try to keep the U-Boot-specifi= c
>> things out of the Linux tree:
>>
>> $ find . -name *u-boot.dtsi |wc -l
>> 398
>
> If any of this is actual information about the hardware then you
> should sort out getting the bindings documented officially
> (which I think is still in the Linux tree), and then QEMU can
> provide them.
>
>> Quite a bit of this is to do with SPL and so far it seems that QEM= U
>> mostly runs U-Boot proper only, although I see that SPL is startin= g to
>> creep in too in the U-Boot CI.
>>
>> So at present QEMU is not able to support U-Boot fully.
>
> My take is that this is u-boot doing weird custom things with
> the DTB that aren't "describe the hardware". You should = be able
> to boot u-boot by putting those custom DTB extra things in a
> separate blob and having u-boot combine that with the
> actual DTB when it starts.

It's not entirely without precedent - for SPL (which I hope is secondar= y
program loading) we have things like the guest loader which expands the
plain HW DTB with some information needed by the bootloader and the
primary OS to load additional blobs which have been put into memory.

In effect the DTB is being expanded as a signalling mechanism similar to things like fw_cfg and other things we use to control boot up. Whether
this affects the "purity" of DTB as a "just the HW" des= cription is
probably a philosophical question.

More than a philosophical question: a key aspect of s= upply chain that need change from
quite inflexible and tightly co= upled to loosely coupled.
=C2=A0A key aspect of it is to maintain= "pure" hardware description DTBs at rest.
Composition = of DTBs at build time (for products) or runtime (for development boards) sh= ould be a simple thing.
Another aspect to take into account is Sy= stem Device Trees. U-Boot only deal with Cortex-As on a platform,=C2=A0
so there are multiple device trees for each compute domain. Communit= ies are working on System Device Tree
to define the overall platf= orm with its power and clock domains. A tool "lopper" is being de= veloped to slide the SDT into diverse domain DTs.
One of them bei= ng included into the FIP as the basis for the computing element (Carrier, S= oM...).
Those attempts to cleanup passed DTBs from configuration = data (drivers, bootloaders...) is not incompatible=C2=A0
with mer= ging fragments at runtime (for dev boards) or build time (for products).
I agree with Peter that just allowing the merging of arbitrary data into the QEMU generated DTB is going to lead to confusion and breakages.
Indeed I wrote the guest-loader because instructions for booting Xen up
until that point involved dumpdtb and hand hacking the data which was
silly because this is stuff QEMU already knew about.

>
> -- PMM


--
Alex Benn=C3=A9e


--
=
Fran=C3=A7ois-Fr=C3=A9d=C3=A9ric Ozog=C2= =A0|=C2=A0Director Busines= s Development
T:=C2=A0+33.67221.6485
francois.ozog@linaro.org=C2=A0|=C2=A0Skype:=C2=A0ffozog

<= /div>
--000000000000fa63aa05cfe23afb--