From: Pantelis Antoniou <panto@antoniou-consulting.com>
To: Grant Likely <grant.likely@secretlab.ca>
Cc: Joel A Fernandes <agnel.joel@gmail.com>,
Rob Herring <robherring2@gmail.com>,
Deepak Saxena <dsaxena@linaro.org>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Scott Wood <scottwood@freescale.com>,
Tony Lindgren <tony@atomide.com>, Russ Dill <Russ.Dill@ti.com>,
Felipe Balbi <balbi@ti.com>, Benoit Cousson <b-cousson@ti.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
Koen Kooi <koen@dominion.thruhere.net>,
Matt Porter <mporter@ti.com>,
linux-omap@vger.kernel.org, Kevin Hilman <khilman@ti.com>,
Paul Walmsley <paul@pwsan.com>,
devicetree-discuss@lists.ozlabs.org
Subject: Re: [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2)
Date: Mon, 12 Nov 2012 13:47:59 +0200 [thread overview]
Message-ID: <14968148-872C-441A-B162-0FF93A70E489@antoniou-consulting.com> (raw)
In-Reply-To: <CACxGe6vL9gbAKyDWcZWWAQ4WO=nTgdAioXS-h2e1jtJK9pmZUg@mail.gmail.com>
Hi Grant,
On Nov 9, 2012, at 11:22 PM, Grant Likely wrote:
> On Fri, Nov 9, 2012 at 5:32 AM, Joel A Fernandes <agnel.joel@gmail.com> wrote:
>> Hi Pantelis,
>>
>> I hope I'm not too late to reply as I'm traveling.
>>
>> On Nov 6, 2012, at 5:30 AM, Pantelis Antoniou
>> <panto@antoniou-consulting.com> wrote:
>>
>>>
>>>>
>>>> Joanne has purchased one of Jane's capes and packaged it into a rugged
>>>> case for data logging. As far as Joanne is concerned, the BeagleBone and
>>>> cape together are a single unit and she'd prefer a single monolithic FDT
>>>> instead of using an FDT overlay.
>>>> Option A: Using dtc, she uses the BeagleBone and cape .dts source files
>>>> to generate a single .dtb for the entire system which is
>>>> loaded by U-Boot. -or-
>>>
>>> Unlikely.
>>>> Option B: Joanne uses a tool to merge the BeagleBone and cape .dtb files
>>>> (instead of .dts files), -or-
>>> Possible but low probability.
>>>
>>>> Option C: U-Boot loads both the base and overlay FDT files, merges them,
>>>> and passes the resolved tree to the kernel.
>>>>
>>>
>>> Could be made to work. Only really required if Joanne wants the
>>> cape interface to work for u-boot too. For example if the cape has some
>>> kind of network interface that u-boot will use to boot from.
>>>
>>
>> I love Grant's hashing idea a lot keeping the phandle problem for
>> compile time and not requiring fixups.
>>
>> IMO it is still a cleaner approach if u-boot does the simple tree merging for
>> all cases, and not the kernel reasons mentioned below.
>
> I'm more than sufficiently convinced that making changes at runtime
> from userspace is pretty much required.
>
> Also consider: it is order of magnitudes easier to modify the kernel
> than it is to change the firmware for end users.
>
Complete agreement here.
>> (1)
>> From a development standpoint, very little or nothing will
>> have to be changed in kernel (except for scripts/dtc) considering we
>> are moving forward with hashing.
>>
>> (2)
>> Also this discussed a while back but at some point is going to brought
>> up again- loading of dt fragment directly from EEPROM and merging at
>> run time. If we were to implement this in kernel, we would have to add
>> cape specific EEPROM reading code, merge the tree before it is
>> unflattened and parse.
>
> Unless it is required for boot to userspace I'm not considering
> merging before userspace starts. That's well after the tree is
> unflattened into the live form. If it is require to boot then I agree
> that is should be done in firmware. I see zero problem with having a
> beaglebone specific cape driver that knows to read the eeprom and
> request a specific configuration file. Heck, the kernel doesn't even
> need to parse the eeprom data. It can be read from userspace and
> userspace decides which overlay to provide. There's nothing stopping
> userspace from reading the eeprom, looking up the correct dts for the
> board, downloading the file from the Internet, compiling it with dtc
> and installing it.... and yes that is getting a little extreme.
>
We're trying to come up with the method that will work best for us.
>From an ease of use perspective, having a kernel driver doing the
probing and performing the DT fragment insertion looks the best.
It's especially nice for the manufacturer, since he can make sure
that when he ships a board a single kernel image will contain everything
with no possibility of RMAs.
For h/w prototyping, where the user is tinkering around with his
own design, the user space approach would be best.
The downloading over the internet DTS file, is a bit extreme for now :)
>> I think doing tree merging in kernel is messy
>> and we should do it in uboot considering we might have to read EEPROM for
>> this use case. Ideally reading the fragment from the EEPROM for all capes
>> and merging without worrying about version detection, Doing the merge and
>> passing the merged blob to the kernel which (kernel) works the same way
>> it does today.
>>
>>>> It may be sufficient to solve it by making the phandle values less
>>>> volatile. Right now dtc generates phandles linearly. Generated phandles
>>>> could be overridden with explicit phandle properties, but it isn't a
>>>> fantastic solution. Perhaps generating the phandle from a hash of the
>>>> node name would be sufficient.
>>>>
>>>
>>> I doubt the hash method will work reliably. We only have 32 bits to work with,
>>> nothing like the SHA hashes of git.
>>>
>>
>> I was wondering I have worked with kernel's crypto code in the past to
>> generate 32 bit md5sums of 1000s of dataitems, from what I've seen,
>> collisions are rare and since we are talking about just a few nodes
>> that are being referenced in the base dt. I think the probability is
>> even less (ofcourse such an analysis strongly depends on dataset).
>> this method also takes away a lot of complexity with having it to do
>> runtime fixups and will help us get off the ground quickly.
>
> It wouldn't be hard to put together a test and run it on all the .dts
> files in the kernel; generating md5 sums for all the full_name paths
> and seeing if we've got any collisions yet.
>
>> We can also put in a collision handling mechanism if needed.
>> I think it is worthy doing a sample hash of all nodes in all dts we
>> have in a script and see for once if we have collisions and what it
>> looks like.
>
> Collision handling is a must, but again this is all internal to dtc. I
> don't foresee any problems with it.
>
>> Alternatively to hashing, reading David Gibson's paper I followed,
>> phandle is supposed to 'uniquely' identity node. I wonder why the node
>> name itself is not sufficient to uniquely identify.
>
> Simply because the FDT draws upon the existing OFW bindings which use
> a 32bit value to reference other nodes.
>
>> The code that does
>> the tree walking can then just strcmp the node name while it walks the
>> tree instead of having to find a node with a phandle number. I guess
>> the reason is phandles are small to store as data values. Another
>> approach can be to arrange the string block in alphabetical order
>> (unless it already is), and store phandle as index of the node name
>> referenced relative to the starting of the strong block. This will not
>> affect nodes in dtb being moved around since they will still have the
>> same index value. the problem being adding or removing nodes Changes
>> the index of all other nodes in the string block as well.. Hmm.
>
> And that still doesn't help find all the phandle locations in the tree
> for doing fixups. It would need to be a table of
> nodes+properties+offsets that contain phandles for fixup to work, but
> I shy away from that approach because I think it will be fragile.
>
> g.
Maybe, but IMHO it will work. I don't know, we're at the point where
we have to start coding and see what come out.
Regards
-- Pantelis
next prev parent reply other threads:[~2012-11-12 11:48 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-05 20:40 [RFC] Device Tree Overlays Proposal (Was Re: capebus moving omap_devices to mach-omap2) Grant Likely
2012-11-05 21:40 ` Tabi Timur-B04825
2012-11-05 23:22 ` Tony Lindgren
2012-11-09 12:06 ` Grant Likely
2012-11-06 0:07 ` Grant Likely
2012-11-06 10:31 ` Pantelis Antoniou
2012-11-07 22:35 ` Ryan Mallon
2012-11-08 13:28 ` Koen Kooi
2012-11-08 14:09 ` Timur Tabi
2012-11-08 17:00 ` Mitch Bradley
2012-11-06 10:30 ` Pantelis Antoniou
2012-11-06 11:14 ` Grant Likely
2012-11-06 18:35 ` Tony Lindgren
2012-11-06 19:29 ` Russ Dill
2012-11-06 19:41 ` Pantelis Antoniou
2012-11-06 22:17 ` Stephen Warren
2012-11-06 19:34 ` Pantelis Antoniou
2012-11-06 20:45 ` Grant Likely
2012-11-06 20:50 ` Grant Likely
2012-11-07 8:06 ` Pantelis Antoniou
2012-11-07 15:33 ` Alan Tull
2012-11-09 17:03 ` Grant Likely
2012-11-07 8:13 ` Pantelis Antoniou
2012-11-07 10:19 ` Benoit Cousson
2012-11-07 11:02 ` Pantelis Antoniou
2012-11-07 11:12 ` Benoit Cousson
2012-11-07 11:23 ` Pantelis Antoniou
2012-11-09 20:33 ` Grant Likely
2012-11-12 11:34 ` Pantelis Antoniou
2012-11-12 13:01 ` Grant Likely
2012-11-07 17:25 ` Stephen Warren
2012-11-07 22:10 ` Pantelis Antoniou
2012-11-08 10:36 ` Cousson, Benoit
2012-11-09 5:32 ` Joel A Fernandes
2012-11-09 14:29 ` David Gibson
2012-11-10 3:15 ` Joel A Fernandes
2012-11-09 21:22 ` Grant Likely
2012-11-12 11:47 ` Pantelis Antoniou [this message]
2012-11-13 3:59 ` Joel A Fernandes
2012-11-09 22:59 ` Stephen Warren
[not found] ` <-4237940489086529028@unknownmsgid>
[not found] ` <559B8433-67C3-4A1A-A5D6-859907655176@antoniou-consulting.com>
2012-11-10 3:36 ` Joel A Fernandes
2012-11-12 12:48 ` Pantelis Antoniou
2012-11-13 2:28 ` David Gibson
2012-11-06 22:37 ` Stephen Warren
2012-11-07 0:54 ` Mitch Bradley
2012-11-09 17:02 ` Grant Likely
2012-11-12 11:29 ` Pantelis Antoniou
2012-11-07 8:47 ` Pantelis Antoniou
2012-11-07 17:18 ` Stephen Warren
2012-11-07 22:08 ` Pantelis Antoniou
2012-11-09 16:28 ` Grant Likely
2012-11-09 23:23 ` Stephen Warren
2012-11-09 23:40 ` Grant Likely
2012-11-12 10:53 ` Koen Kooi
2012-11-12 12:10 ` Pantelis Antoniou
2012-11-12 16:52 ` Stephen Warren
2012-11-13 7:25 ` David Gibson
2012-11-13 8:09 ` Pantelis Antoniou
2012-11-13 12:24 ` Grant Likely
2012-11-13 13:38 ` Pantelis Antoniou
2012-11-15 4:57 ` David Gibson
2012-11-13 17:10 ` Stephen Warren
2012-11-13 23:30 ` David Gibson
2012-11-14 0:00 ` Pantelis Antoniou
2012-11-13 16:57 ` Stephen Warren
2012-11-13 18:10 ` Mitch Bradley
2012-11-13 18:29 ` Stephen Warren
2012-11-13 19:09 ` Mitch Bradley
2012-11-13 19:11 ` Pantelis Antoniou
2012-11-17 22:27 ` Jean-Christophe PLAGNIOL-VILLARD
2012-11-20 17:09 ` Grant Likely
2012-11-11 20:47 ` Rob Landley
2012-11-12 12:50 ` Pantelis Antoniou
2012-11-12 16:54 ` Stephen Warren
2012-11-12 11:23 ` Pantelis Antoniou
2012-11-12 16:49 ` Stephen Warren
2012-11-12 17:00 ` Pantelis Antoniou
2012-11-12 17:10 ` Stephen Warren
2012-11-12 17:19 ` Pantelis Antoniou
2012-11-12 17:29 ` Stephen Warren
2012-11-12 17:38 ` Pantelis Antoniou
2012-11-12 20:16 ` Russ Dill
2012-11-12 16:45 ` Stephen Warren
2012-11-09 2:26 ` David Gibson
2012-11-09 15:40 ` Pantelis Antoniou
2012-11-13 0:03 ` David Gibson
2012-11-09 21:08 ` Grant Likely
2012-11-13 0:05 ` David Gibson
2012-11-09 21:42 ` Grant Likely
2012-11-13 1:05 ` David Gibson
2012-11-13 5:22 ` Stephen Warren
2012-11-13 6:54 ` David Gibson
2012-11-09 22:57 ` Stephen Warren
2012-11-09 23:27 ` Grant Likely
2012-11-12 12:05 ` Pantelis Antoniou
2012-11-09 23:14 ` Stephen Warren
2012-11-09 23:06 ` Stephen Warren
2012-11-09 23:32 ` Grant Likely
2012-11-12 11:03 ` Koen Kooi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=14968148-872C-441A-B162-0FF93A70E489@antoniou-consulting.com \
--to=panto@antoniou-consulting.com \
--cc=Russ.Dill@ti.com \
--cc=agnel.joel@gmail.com \
--cc=b-cousson@ti.com \
--cc=balbi@ti.com \
--cc=benh@kernel.crashing.org \
--cc=devicetree-discuss@lists.ozlabs.org \
--cc=dsaxena@linaro.org \
--cc=grant.likely@secretlab.ca \
--cc=khilman@ti.com \
--cc=koen@dominion.thruhere.net \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-omap@vger.kernel.org \
--cc=mporter@ti.com \
--cc=paul@pwsan.com \
--cc=robherring2@gmail.com \
--cc=scottwood@freescale.com \
--cc=tony@atomide.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).