From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Gibson Subject: Re: [PATCH v6 1/4] dtc: Document the dynamic plugin internals Date: Wed, 25 May 2016 13:41:04 +1000 Message-ID: <20160525034104.GI17226@voom.fritz.box> References: <1462477724-8092-1-git-send-email-pantelis.antoniou@konsulko.com> <1462477724-8092-2-git-send-email-pantelis.antoniou@konsulko.com> <20160524045840.GC29005@voom.fritz.box> <53E9201A-5D63-4A8E-8179-F96980F76BED@konsulko.com> <20160524105002.GD17226@voom.fritz.box> <45A08C18-0368-48B2-B6E3-BC352E345125@konsulko.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nO3oAMapP4dBpMZi" Return-path: Content-Disposition: inline In-Reply-To: <45A08C18-0368-48B2-B6E3-BC352E345125-OWPKS81ov/FWk0Htik3J/w@public.gmane.org> Sender: devicetree-compiler-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Pantelis Antoniou Cc: Jon Loeliger , Grant Likely , Rob Herring , Frank Rowand , Mark Rutland , Jan Luebbe , Sascha Hauer , Matt Porter , devicetree-compiler-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: devicetree@vger.kernel.org --nO3oAMapP4dBpMZi Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 24, 2016 at 03:53:07PM +0300, Pantelis Antoniou wrote: > Hi David, >=20 > > On May 24, 2016, at 13:50 , David Gibson = wrote: > >=20 > > On Tue, May 24, 2016 at 10:43:29AM +0300, Pantelis Antoniou wrote: > >> Hi David, > >>=20 > >>> On May 24, 2016, at 07:58 , David Gibson wrote: > >>>=20 > >>> One small nit in the document itself. > >>>=20 > >>> I have other comments, but they're about the overlay format itself, > >>> rather than this patch as such. > >>>=20 > >>=20 > >> OK. > >>=20 > >>> On Thu, May 05, 2016 at 10:48:41PM +0300, Pantelis Antoniou wrote: > >>>> Provides the document explaining the internal mechanics of > >>>> plugins and options. > >>>>=20 > >>>> Signed-off-by: Pantelis Antoniou > >>>> --- > >>>> Documentation/dt-object-internal.txt | 318 +++++++++++++++++++++++++= ++++++++++ > >>>> 1 file changed, 318 insertions(+) > >>>> create mode 100644 Documentation/dt-object-internal.txt > >>>>=20 > >>>> diff --git a/Documentation/dt-object-internal.txt b/Documentation/dt= -object-internal.txt > >>>> new file mode 100644 > >>>> index 0000000..734f447 > >>>> --- /dev/null > >>>> +++ b/Documentation/dt-object-internal.txt > >>>> @@ -0,0 +1,318 @@ > >>>> +Device Tree Dynamic Object format internals > >>>> +------------------------------------------- > >>>> + > >>>> +The Device Tree for most platforms is a static representation of > >>>> +the hardware capabilities. This is insufficient for many platforms > >>>> +that need to dynamically insert device tree fragments to the > >>>> +running kernel's live tree. > >>>> + > >>>> +This document explains the the device tree object format and the > >>>> +modifications made to the device tree compiler, which make it possi= ble. > >>>> + > >>>> +1. Simplified Problem Definition > >>>> +-------------------------------- > >>>> + > >>>> +Assume we have a platform which boots using following simplified de= vice tree. > >>>> + > >>>> +---- foo.dts ------------------------------------------------------= ----------- > >>>> + /* FOO platform */ > >>>> + / { > >>>> + compatible =3D "corp,foo"; > >>>> + > >>>> + /* shared resources */ > >>>> + res: res { > >>>> + }; > >>>> + > >>>> + /* On chip peripherals */ > >>>> + ocp: ocp { > >>>> + /* peripherals that are always instantiated */ > >>>> + peripheral1 { ... }; > >>>> + }; > >>>> + }; > >>>> +---- foo.dts ------------------------------------------------------= ----------- > >>>> + > >>>> +We have a number of peripherals that after probing (using some unde= fined method) > >>>> +should result in different device tree configuration. > >>>> + > >>>> +We cannot boot with this static tree because due to the configurati= on of the > >>>> +foo platform there exist multiple conficting peripherals DT fragmen= ts. > >>>> + > >>>> +So for the bar peripheral we would have this: > >>>> + > >>>> +---- foo+bar.dts --------------------------------------------------= ----------- > >>>> + /* FOO platform + bar peripheral */ > >>>> + / { > >>>> + compatible =3D "corp,foo"; > >>>> + > >>>> + /* shared resources */ > >>>> + res: res { > >>>> + }; > >>>> + > >>>> + /* On chip peripherals */ > >>>> + ocp: ocp { > >>>> + /* peripherals that are always instantiated */ > >>>> + peripheral1 { ... }; > >>>> + > >>>> + /* bar peripheral */ > >>>> + bar { > >>>> + compatible =3D "corp,bar"; > >>>> + ... /* various properties and child nodes */ > >>>> + }; > >>>> + }; > >>>> + }; > >>>> +---- foo+bar.dts --------------------------------------------------= ----------- > >>>> + > >>>> +While for the baz peripheral we would have this: > >>>> + > >>>> +---- foo+baz.dts --------------------------------------------------= ----------- > >>>> + /* FOO platform + baz peripheral */ > >>>> + / { > >>>> + compatible =3D "corp,foo"; > >>>> + > >>>> + /* shared resources */ > >>>> + res: res { > >>>> + /* baz resources */ > >>>> + baz_res: res_baz { ... }; > >>>> + }; > >>>> + > >>>> + /* On chip peripherals */ > >>>> + ocp: ocp { > >>>> + /* peripherals that are always instantiated */ > >>>> + peripheral1 { ... }; > >>>> + > >>>> + /* baz peripheral */ > >>>> + baz { > >>>> + compatible =3D "corp,baz"; > >>>> + /* reference to another point in the tree */ > >>>> + ref-to-res =3D <&baz_res>; > >>>> + ... /* various properties and child nodes */ > >>>> + }; > >>>> + }; > >>>> + }; > >>>> +---- foo+baz.dts --------------------------------------------------= ----------- > >>>> + > >>>> +We note that the baz case is more complicated, since the baz periph= eral needs to > >>>> +reference another node in the DT tree. > >>>> + > >>>> +2. Device Tree Object Format Requirements > >>>> +----------------------------------------- > >>>> + > >>>> +Since the device tree is used for booting a number of very differen= t hardware > >>>> +platforms it is imperative that we tread very carefully. > >>>> + > >>>> +2.a) No changes to the Device Tree binary format for the base tree.= We cannot > >>>> +modify the tree format at all and all the information we require sh= ould be > >>>> +encoded using device tree itself. We can add nodes that can be safe= ly ignored > >>>> +by both bootloaders and the kernel. The plugin dtb's are optionally= tagged > >>>> +with a different magic number in the header but otherwise they too = are simple > >>>> +blobs. > >>>> + > >>>> +2.b) Changes to the DTS source format should be absolutely minimal,= and should > >>>> +only be needed for the DT fragment definitions, and not the base bo= ot DT. > >>>> + > >>>> +2.c) An explicit option should be used to instruct DTC to generate = the required > >>>> +information needed for object resolution. Platforms that don't use = the > >>>> +dynamic object format can safely ignore it. > >>>> + > >>>> +2.d) Finally, DT syntax changes should be kept to a minimum. It sho= uld be > >>>> +possible to express everything using the existing DT syntax. > >>>> + > >>>> +3. Implementation > >>>> +----------------- > >>>> + > >>>> +The basic unit of addressing in Device Tree is the phandle. Turns o= ut it's > >>>> +relatively simple to extend the way phandles are generated and refe= renced > >>>> +so that it's possible to dynamically convert symbolic references (l= abels) > >>>> +to phandle values. This is a valid assumption as long as the author= uses > >>>> +reference syntax and does not assign phandle values manually (which= might > >>>> +be a problem with decompiled source files). > >>>> + > >>>> +We can roughly divide the operation into two steps. > >>>> + > >>>> +3.a) Compilation of the base board DTS file using the '-@' option > >>>> +generates a valid DT blob with an added __symbols__ node at the roo= t node, > >>>> +containing a list of all nodes that are marked with a label. > >>>> + > >>>> +Using the foo.dts file above the following node will be generated; > >>>> + > >>>> +$ dtc -@ -O dtb -o foo.dtb -b 0 foo.dts > >>>> +$ fdtdump foo.dtb > >>>> +... > >>>> +/ { > >>>> + ... > >>>> + res { > >>>> + ... > >>>> + phandle =3D <0x00000001>; > >>>> + ... > >>>> + }; > >>>> + ocp { > >>>> + ... > >>>> + phandle =3D <0x00000002>; > >>>> + ... > >>>> + }; > >>>> + __symbols__ { > >>>> + res=3D"/res"; > >>>> + ocp=3D"/ocp"; > >>>> + }; > >>>> +}; > >>>> + > >>>> +Notice that all the nodes that had a reference have been recorded, = and that > >>>=20 > >>> s/reference/label/ > >>>=20 > >>=20 > >> OK > >>=20 > >>>> +phandles have been generated for them. > >>>=20 > >>>=20 > >>>> +This blob can be used to boot the board normally, the __symbols__ n= ode will > >>>> +be safely ignored both by the bootloader and the kernel (the only l= oss will > >>>> +be a few bytes of memory and disk space). > >>>> + > >>>> +3.b) The Device Tree fragments must be compiled with the same optio= n but they > >>>> +must also have a tag (/plugin/) that allows undefined references to= nodes > >>>> +that are not present at compilation time to be recorded so that the= runtime > >>>> +loader can fix them. > >>>> + > >>>> +So the bar peripheral's DTS format would be of the form: > >>>> + > >>>> +/dts-v1/ /plugin/; /* allow undefined references and record them */ > >>>> +/ { > >>>> + .... /* various properties for loader use; i.e. part id etc. */ > >>>> + fragment@0 { > >>>> + target =3D <&ocp>; > >>>> + __overlay__ { > >>>> + /* bar peripheral */ > >>>> + bar { > >>>> + compatible =3D "corp,bar"; > >>>> + ... /* various properties and child nodes */ > >>>> + } > >>>> + }; > >>>> + }; > >>>> +}; > >>>> + > >>>> +Note that there's a target property that specifies the location whe= re the > >>>> +contents of the overlay node will be placed, and it references the = node > >>>> +in the foo.dts file. > >>>=20 > >>> Ugh.. I really don't like the target stuff appearing in the dts like > >>> this. I thought we were changing this so these appeared in the blob, > >>> but in the source we just used the existing overlay syntax, so for the > >>> above, something like: > >>>=20 > >>> &ocp { > >>> ... > >>> }; > >>>=20 > >>=20 > >> This works, but it=E2=80=99s just syntactic sugar. > >=20 > > Hmmm.... The target=3D property and fragment@ nodes are part of the > > internal overlay glue, rather than actual DT content. So, I *really* > > dislike including it inline in the dts file. Come to that, I dislike > > including it in the dtb, but I can see the rationale and we're kind of > > stuck with it anyway. The dts, not so much. > >=20 >=20 > There are more target variants to come, but they don=E2=80=99t require any > dtc changes. >=20 > The &label { }; transforms to fragment@0 { target =3D <&label>; __overlay= __ { }; }; >=20 > >> It does not cover the cases where the target is a path, or a different > >> kind of target. > >=20 > > Huh? It certainly covers the case of a path > > &{/some/path} { ... } > > What other sort of target did you have in mind? > >=20 >=20 > That is just not supported right now. There is no phandle to resolve. >=20 > I=E2=80=99ll take a stab at it, but it will transform to the target-path = variant. Right, but my point is there's an obvious dts syntex for path-targetted fragments. My inclination is to create syntax for any new fragment types, rather than expose the encoding of the overlay metadata as if it were in-band information. > >> Besides the sugary part, a target is something that doesn=E2=80=99t ha= ve anything to > >> do with the plugin format. > >>=20 > >>> Or have I gotten confused by the history of things. > >>>=20 > >>=20 > >> It=E2=80=99s got a long history for sure :) > >>=20 > >>>> +$ dtc -@ -O dtb -o bar.dtbo -b 0 bar.dts > >>>> +$ fdtdump bar.dtbo > >>>> +... > >>>> +/ { > >>>> + ... /* properties */ > >>>> + fragment@0 { > >>>> + target =3D <0xffffffff>; > >>>> + __overlay__ { > >>>> + bar { > >>>> + compatible =3D "corp,bar"; > >>>> + ... /* various properties and child nodes */ > >>>> + } > >>>> + }; > >>>> + }; > >>>> + __fixups__ { > >>>> + ocp =3D "/fragment@0:target:0"; > >>>=20 > >>> I still hate this parse-requiring string, but I guess we're stuck with > >>> it. > >>>=20 > >>>> + }; > >>>> +}; > >>>> + > >>>> +No __symbols__ has been generated (no label in bar.dts). > >>>> +Note that the target's ocp label is undefined, so the phandle handle > >>>> +value is filled with the illegal value '0xffffffff', while a __fixu= ps__ > >>>> +node has been generated, which marks the location in the tree where > >>>> +the label lookup should store the runtime phandle value of the ocp = node. > >>>> + > >>>> +The format of the __fixups__ node entry is > >>>> + > >>>> +