devicetree.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
Cc: "Rob Herring" <robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"Devicetree Compiler"
	<devicetree-compiler-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"devicetree-spec-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<devicetree-spec-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"Jon Loeliger" <jdl-CYoMK+44s/E@public.gmane.org>,
	"devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"Pantelis Antoniou"
	<pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w@public.gmane.org>,
	"Pantelis
	Antomarek.vasut-Re5JQEeQqe/2sr8fMPgRzw@public.gmane.org"
	<panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org>,
	"Grant Likely"
	<grant.likely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org>,
	"Marek Vašut"
	<marek.vasut-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	"Tom Rini" <trini-OWPKS81ov/FWk0Htik3J/w@public.gmane.org>,
	"Kyle Evans" <kevans-h+KGxgPPiopAfugRpC6u6w@public.gmane.org>,
	"Geert Uytterhoeven"
	<geert-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org>,
	"Alan Tull" <atull-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	"Michael Ellerman" <mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org>
Subject: Re: [RFC] devicetree: new FDT format version
Date: Thu, 25 Jan 2018 12:01:07 -0800	[thread overview]
Message-ID: <72d30756-a963-92c9-1838-3e3f80c57e39@gmail.com> (raw)
In-Reply-To: <20180125122912.GB2099@umbus>

On 01/25/18 04:29, David Gibson wrote:
> On Wed, Jan 24, 2018 at 04:22:15PM -0800, Frank Rowand wrote:
>> On 01/24/18 13:16, Frank Rowand wrote:
>>> On 01/24/18 07:47, Rob Herring wrote:
>>>> On Tue, Jan 23, 2018 at 3:17 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>> On 01/23/18 04:42, David Gibson wrote:
>>>>>> On Mon, Jan 22, 2018 at 03:01:52PM -0600, Rob Herring wrote:
>>>>>>> On Mon, Jan 22, 2018 at 2:09 AM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I've tried to create a decent distribution list, but I'm sure I've missed
>>>>>>>> someone or some important list.  Please share this with anyone you think
>>>>>>>> will be affected.
>>>>>>>>
>>>>>>>> I have been playing around with some thoughts for some additions to
>>>>>>>> the devicetree FDT aka blob format.
>>>>>>>>
>>>>>>>> I would like to get the affected parties thinking about how additions to
>>>>>>>> the format could improve whichever pieces of FDT related technology you
>>>>>>>> work on or care about.  In my opinion, the FDT format should change
>>>>>>>> very infrequently because of the impact on so many projects that have
>>>>>>>> to work together to create a final solution, plus the many many users
>>>>>>>> of those projects.
>>>>>>>
>>>>>>> A few things discussed before:
>>>>>>> - Adding type information Even just tagging phandles would be good.
>>>>>>
>>>>>> I'm a bit dubious about this.  It would have to be "hints" only -
>>>>>> there's not really anyway we can put in authoritative type
>>>>>> information, since dtc itself doesn't really know that.  It's also
>>>>>> hard to see how that could be done in a way which wouldn't either a)
>>>>>> require very awkward parallel lookup of the data and type information
>>>>>> or b) not be backwards compatible (even read only).
>>>>
>>>> I never said it was possible. :) I'm just trying to enumerate possible
>>>> FDT format changes because I don't think we want to continuously
>>>> trickle out FDT changes even if they are backwards compatible.
>>>
>>> Yes, I'm trying to capture any pending changes in a single version change.
>>>
>>>
>>>>>>> - Allow applying overlays by just appending to the blob. The need for
>>>>>>> this is somewhat gone now that libfdt can just fully apply overlays.
>>>>>>
>>>>>> I'm not really sure what you want here.  I mean you could easily allow
>>>>>> the format to allow multiple appended overlays, and define that to
>>>>>> mean the overlaid result.  At some point *something* is going to have
>>>>>> to really do the application, so I'm not sure that it really buys you
>>>>>> that much.  It also makes nearly every operation on the tree in libfdt
>>>>>> horrible to implement, at least within the other constraints the
>>>>>> interface was designed around; you'll continually have to scan the
>>>>>> entire tree just in case some other overlay fragment has some bearing
>>>>>> on the thing you're looking at.  It confuses the interface too: what
>>>>>> does "node offset" mean if the same node could be built up from
>>>>>> overlay fragments at multiple offsets.
>>>>
>>>> The idea was to avoid applying overlays to flattened trees at all.
>>>> You're just passing the problem to the next stage (typically the
>>>> kernel). But we have applying overlays to flattened trees now, so
>>>> maybe there's no need anymore.
>>>>
>>>>> Somewhat echoing David's comment, I'm also not sure what you mean.
>>>>> And trying to not overly influence this conversation with preconceptions
>>>>> from what I'm going to propose.
>>>>>
>>>>> My first shot at the new format added a field to the FDT to indicate
>>>>> that an overlay FDT was concatenated to the FDT (and the overlay FDT
>>>>> in turn could set it's field to concatenate another overlay FDT).
>>>>
>>>> Yes, something like this is what I meant. This was something Grant had
>>>> talked about.
>>>>
>>>>> But in the end I decided that information belonged outside the FDT,
>>>>> and it was sufficient to require that all FDTs be padded out so that
>>>>> if an overlay FDT was concatenated to the FDT, the overlay FDT would
>>>>> be properly aligned.
>>>>
>>>> I can't think of why this wouldn't work either.
>>>>
>>>>> For ease of typing, I'll call this "chaining" or "chained".  For
>>>>> Linux, I am envisioning no kernel use of data from a chained FDT
>>>>> until after the tree is unflattened.  I haven't done an exhaustive
>>>>> search to determine all of the uses of data directly from the
>>>>> flattened FDT, but I am optimistic that there will not be any such
>>>>> access that would require data from a chained FDT (and we could
>>>>> make this a rule).
>>>>
>>>> This would be a major downside to leaving it up to the kernel because
>>>> what can't be touched is hard to enumerate and could change. For
>>>> example, we added earlycon and now the uart node can't be modified.
>>>
>>> What you say makes sense.  So I'll reverse myself and say that for
>>> Linux, we should update the FDT read code to scan all chained
>>> overlay FDT(s) as well as the base FDT.
>>
>> < snip >
>>
>> What I wrote was somewhat ambiguous.  What I meant by "FDT read
>> code" was functions that check for existence of nodes in the
>> FDT or read property values from the FDT.
> 
> Oh.. not just FDT unflattening code.
> 
> The trouble with this is that scanning for a specific node or property
> in a set of chained overlays is highly nontrivial.  Even if you set
> aside the arguably self-imposed design constraints in libfdt, you
> can't just do the same lookup in each fragment: along the way you need
> to resolve the path at which each fragment applies.  That in itself is
> non-trivial.  If you have overlays applying on top of other overlays,
> that could involve recursive lookups of things from previous overlays.
> It's spectacularly complicated and we have to do it on *every single*
> read operation.

I totally overlooked having to resolve each fragment.  You are right,
that makes the problem very complex instead of trivial.


> Either fully applying the overlay in flattened form, or (even
> temporarily) unflattening the tree are better solutions.
> 
>> The other way one could read what I wrote was that when Linux
>> unflattens the FDT, it would unflatten the FDT and all of the
>> chained FDTs.  Obviously also true, but not what I was trying
>> to say here.
>>
>> -Frank
> 

  reply	other threads:[~2018-01-25 20:01 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-22  8:09 Frank Rowand
     [not found] ` <b96829f9-2e8b-fdc5-5090-58591e2260cf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-22  8:14   ` Frank Rowand
     [not found]     ` <ec68cfb3-4702-ba00-e926-3ad63b20b199-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-22 20:08       ` Frank Rowand
2018-01-22 20:08   ` Frank Rowand
     [not found]     ` <bc44f051-835d-1c8d-a928-be0fd4ef80b5-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-25  0:27       ` Frank Rowand
     [not found]         ` <b5a72db1-45b2-f21f-9afd-0991b288840e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-27  8:48           ` David Gibson
2018-01-29  8:08             ` Frank Rowand
     [not found]               ` <2c352396-154e-ea75-c50b-0f2bfafd19da-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-29 10:56                 ` David Gibson
2018-01-30  1:29                   ` Frank Rowand
2018-01-22 21:01   ` Rob Herring
     [not found]     ` <CAL_JsqJR2y7bMw_-9TBAGWZ_kf7_sZo5qvqvRowJ8jiy=4G0Bg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-23 12:42       ` David Gibson
2018-01-23 21:17         ` Frank Rowand
     [not found]           ` <20328477-e511-e875-7dc4-253640f2219e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-24 15:47             ` Rob Herring
     [not found]               ` <CAL_Jsq+fvFrGhqO0zbEUE_i23FkU=G4Z1-e0vnXHi4KbS2oK0A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-24 21:16                 ` Frank Rowand
     [not found]                   ` <e1bec77d-4dac-200b-34be-23573bf738f0-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-24 22:27                     ` Alan Tull
2018-01-25  0:22                     ` Frank Rowand
     [not found]                       ` <e986435b-481b-2629-7600-10d9e21ac58e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-25 12:29                         ` David Gibson
2018-01-25 20:01                           ` Frank Rowand [this message]
     [not found]                             ` <72d30756-a963-92c9-1838-3e3f80c57e39-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-29 18:32                               ` Grant Likely
     [not found]                                 ` <CACxGe6tUB3NhNhOtqzoJfThx=RE1_=TSDQz+wQUm=sdE5JirZQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-29 23:15                                   ` David Gibson
2018-01-26  8:56                           ` Geert Uytterhoeven
     [not found]                             ` <CAMuHMdV0jdj90uzqMx_wtvz=-KaagJG2_UQTm1DW3gzt6cNG6A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-26  8:59                               ` Geert Uytterhoeven
2018-01-26 22:08                               ` Frank Rowand
     [not found]                                 ` <83008da0-7383-ba2d-a239-e11ad7d1327d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-27  9:00                                   ` David Gibson
2018-01-27  8:56                               ` David Gibson
2018-01-25 23:11                     ` Frank Rowand
2018-01-25 12:22                 ` David Gibson
2018-01-25  9:14             ` Marek Vasut
     [not found]               ` <90983180-ae3b-5a31-9dc0-b62b978a0fba-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-25 12:37                 ` David Gibson
2018-01-27 20:30                   ` Marek Vasut
     [not found]                     ` <5a943937-3b59-514b-3939-df25daea5470-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-29  0:53                       ` David Gibson
2018-01-23 12:05   ` David Gibson
2018-01-23 21:28     ` Frank Rowand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=72d30756-a963-92c9-1838-3e3f80c57e39@gmail.com \
    --to=frowand.list-re5jqeeqqe8avxtiumwx3w@public.gmane.org \
    --cc=atull-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org \
    --cc=devicetree-compiler-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=devicetree-spec-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=geert-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org \
    --cc=grant.likely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org \
    --cc=jdl-CYoMK+44s/E@public.gmane.org \
    --cc=kevans-h+KGxgPPiopAfugRpC6u6w@public.gmane.org \
    --cc=marek.vasut-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org \
    --cc=pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w@public.gmane.org \
    --cc=panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org \
    --cc=robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=trini-OWPKS81ov/FWk0Htik3J/w@public.gmane.org \
    --subject='Re: [RFC] devicetree: new FDT format version' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox