[RFC] Best practices for hardware shipping device trees

[RFC] Best practices for hardware shipping device trees

[Tom Rini]

Hey all,

Do we have a document yet talking about the best practices for how we
would like a hardware vendor to ship, store and possibly update a device
tree, on the hardware?  "However they like" seems likely to invite
problems down the line with everyone trying their own thing.  Thanks!

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Jason Cooper]

Hey Tom,

On Wed, Aug 14, 2013 at 11:13:45AM -0400, Tom Rini wrote:
> Do we have a document yet talking about the best practices for how we
> would like a hardware vendor to ship, store and possibly update a device
> tree, on the hardware?  "However they like" seems likely to invite
> problems down the line with everyone trying their own thing.  Thanks!

Speaking from my experience with the Marvell SoCs and after market
installation of mainline code (bootloaders, kernel, etc), I can say what
I'd like to see, if that helps ;-)

  1) individually upgradable (bootloader, dtb, config, kernel, etc)
      - separate flash partitions for each

  2) bootloader uses as well as passes off the dtb
      - Good for scenarios where user wants to modify flash partitions,
        he would only need to update the dtb.
      - facilitates fixes after deployment since dtb not bound to
        bootloader.

I'm sure there's more, but those jumped to mind when I read your
question.

thx,

Jason.

Re: [RFC] Best practices for hardware shipping device trees

[Stephen Warren]

On 08/14/2013 09:13 AM, Tom Rini wrote:
> Hey all,
> 
> Do we have a document yet talking about the best practices for how we
> would like a hardware vendor to ship, store and possibly update a device
> tree, on the hardware?  "However they like" seems likely to invite
> problems down the line with everyone trying their own thing.  Thanks!

I don't believe there's any written guidance, no.

The initial guidance Grant gave (IIRC at the first Linaro Connect last
year, or perhaps the ARM workshop in Prague, or perhaps also in various
ARM kernel list threads?) was that the DTBs should be stored/accessed in
exactly the same way as the kernel, which on many systems implies it's a
file in /boot (although MTD partitions, ... are also possible kernel
locations). The idea here was to explicitly make upgrading the DTB as
easy as upgrading the kernel, and explicitly without having to upgrade
any firmware, since that's a more dangerous process in most cases.

Now perhaps that advice was only intended to apply very early on when DT
was really new on ARM, and has "aged out" by now? If so, I don't recall
that every being explicitly mentioned or communicated later.

Certainly, that approach actively causes issues whereby every distro
must somehow install ensure the DTBs get into /boot, rather than the
system "just providing" them somehow. Having distros have to manage this
probably doesn't scale very well, hence people are advocating that the
DTB be part of the firmware image, so it's already there without the
distro having to put it into the filesystem.

Re: [RFC] Best practices for hardware shipping device trees

[Jason Cooper]

On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
...
> Certainly, that approach actively causes issues whereby every distro
> must somehow install ensure the DTBs get into /boot, rather than the
> system "just providing" them somehow. Having distros have to manage this
> probably doesn't scale very well, hence people are advocating that the
> DTB be part of the firmware image, so it's already there without the
> distro having to put it into the filesystem.

I advocate that it be stored on the same media as the bootloader (eg
flash), but separate from the bootloader, similar to how u-boot has a
separate flash partition for it's environment.

This way, a distro can upgrade the DTB if they _choose_ to, but managing
the dtbs *isn't* mandatory.

It makes bootloader init via DT a little more difficult, since the flash
chip/bus parameters would have to be hardcoded into the bootloader.
Plus the offset to the dtb.  But bootloader init via DT is at best a
nice-to-have, imho.  I don't think it should get in the way of a
properly upgradeable system.

thx,

Jason.

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
> On 08/14/2013 09:13 AM, Tom Rini wrote:
> > Hey all,
> > 
> > Do we have a document yet talking about the best practices for how we
> > would like a hardware vendor to ship, store and possibly update a device
> > tree, on the hardware?  "However they like" seems likely to invite
> > problems down the line with everyone trying their own thing.  Thanks!
> 
> I don't believe there's any written guidance, no.
> 
> The initial guidance Grant gave (IIRC at the first Linaro Connect last
> year, or perhaps the ARM workshop in Prague, or perhaps also in various
> ARM kernel list threads?) was that the DTBs should be stored/accessed in
> exactly the same way as the kernel, which on many systems implies it's a
> file in /boot (although MTD partitions, ... are also possible kernel
> locations). The idea here was to explicitly make upgrading the DTB as
> easy as upgrading the kernel, and explicitly without having to upgrade
> any firmware, since that's a more dangerous process in most cases.
> 
> Now perhaps that advice was only intended to apply very early on when DT
> was really new on ARM, and has "aged out" by now? If so, I don't recall
> that every being explicitly mentioned or communicated later.
[snip out a bit more of Stephen's answer]

Yes, this notion certainly is the opposite of what was suggested on the
cross-distro list, both as part of a "what should a bootloader provide
to get commodity distros to support the board" thread and the "where
should a device tree live in the filesystem" thread.  Cc'ing them now
because this is one of those things that feels like it needs solving,
solving soon, and done in a way the least number of folks get surprised
about it.

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Wed, Aug 14, 2013 at 12:38:23PM -0400, Jason Cooper wrote:
> Hey Tom,
> 
> On Wed, Aug 14, 2013 at 11:13:45AM -0400, Tom Rini wrote:
> > Do we have a document yet talking about the best practices for how we
> > would like a hardware vendor to ship, store and possibly update a device
> > tree, on the hardware?  "However they like" seems likely to invite
> > problems down the line with everyone trying their own thing.  Thanks!
> 
> Speaking from my experience with the Marvell SoCs and after market
> installation of mainline code (bootloaders, kernel, etc), I can say what
> I'd like to see, if that helps ;-)
> 
>   1) individually upgradable (bootloader, dtb, config, kernel, etc)
>       - separate flash partitions for each

Well, that assumes it comes from the factory with flash flashed.  Forgot
to update my mutt rules before sending the first post, but a lot of TI
boards don't ship with NAND programmed, just an SD card, even in the
cases of boards with NAND.  On some families (am335x) we include a
relatively large EEPROM on the references, of which some customers throw
out and some keep and re-purpose with their own data.

>   2) bootloader uses as well as passes off the dtb
>       - Good for scenarios where user wants to modify flash partitions,
>         he would only need to update the dtb.
>       - facilitates fixes after deployment since dtb not bound to
>         bootloader.

Putting my U-Boot guy hat on, I'm happy to go down this path more, in
certain areas where we can afford it, once things in DT-land are stable
(and we have examples of this today, even).

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Jason Cooper]

On Wed, Aug 14, 2013 at 02:41:48PM -0400, Tom Rini wrote:
> On Wed, Aug 14, 2013 at 12:38:23PM -0400, Jason Cooper wrote:
> > On Wed, Aug 14, 2013 at 11:13:45AM -0400, Tom Rini wrote:
> > > Do we have a document yet talking about the best practices for how we
> > > would like a hardware vendor to ship, store and possibly update a device
> > > tree, on the hardware?  "However they like" seems likely to invite
> > > problems down the line with everyone trying their own thing.  Thanks!
> > 
> > Speaking from my experience with the Marvell SoCs and after market
> > installation of mainline code (bootloaders, kernel, etc), I can say what
> > I'd like to see, if that helps ;-)
> > 
> >   1) individually upgradable (bootloader, dtb, config, kernel, etc)
> >       - separate flash partitions for each
> 
> Well, that assumes it comes from the factory with flash flashed.  Forgot
> to update my mutt rules before sending the first post, but a lot of TI
> boards don't ship with NAND programmed, just an SD card, even in the
> cases of boards with NAND.  On some families (am335x) we include a
> relatively large EEPROM on the references, of which some customers throw
> out and some keep and re-purpose with their own data.

Ah, okay.  I'm more used to dealing with end-user devices that are sold
as consumer products (routers, NASs, APs, etc).  I suppose I could
amend the above to be 'place the dtb on the same media as the bootloader
as a separate entity'.  So if there's a filesystem, it's a separate
file, if it's a partitioned flash, then a separate partition.

thx,

Jason.

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Wed, 14 Aug 2013, Tom Rini wrote:

> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
> > On 08/14/2013 09:13 AM, Tom Rini wrote:
> > > Hey all,
> > > 
> > > Do we have a document yet talking about the best practices for how we
> > > would like a hardware vendor to ship, store and possibly update a device
> > > tree, on the hardware?  "However they like" seems likely to invite
> > > problems down the line with everyone trying their own thing.  Thanks!
> > 
> > I don't believe there's any written guidance, no.
> > 
> > The initial guidance Grant gave (IIRC at the first Linaro Connect last
> > year, or perhaps the ARM workshop in Prague, or perhaps also in various
> > ARM kernel list threads?) was that the DTBs should be stored/accessed in
> > exactly the same way as the kernel, which on many systems implies it's a
> > file in /boot (although MTD partitions, ... are also possible kernel
> > locations). The idea here was to explicitly make upgrading the DTB as
> > easy as upgrading the kernel, and explicitly without having to upgrade
> > any firmware, since that's a more dangerous process in most cases.
> > 
> > Now perhaps that advice was only intended to apply very early on when DT
> > was really new on ARM, and has "aged out" by now? If so, I don't recall
> > that every being explicitly mentioned or communicated later.
> [snip out a bit more of Stephen's answer]
> 
> Yes, this notion certainly is the opposite of what was suggested on the
> cross-distro list, both as part of a "what should a bootloader provide
> to get commodity distros to support the board" thread and the "where
> should a device tree live in the filesystem" thread.  Cc'ing them now
> because this is one of those things that feels like it needs solving,
> solving soon, and done in a way the least number of folks get surprised
> about it.

I don't think the "however they like" approach is that bad.  It is 
certainly less of a problem than "exactly the same way as the kernel" is.

Using /boot implies a distro filesystem and we'd rather wish for DT to 
be independent from a distro or even Linux.  The DTB should therefore be 
stored and accessed in a way which is hardware/bootloader specific.  The 
distro being installed on top shouldn't have to care.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> On Wed, 14 Aug 2013, Tom Rini wrote:
> 
>> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
>>> On 08/14/2013 09:13 AM, Tom Rini wrote:
>>>> Hey all,
>>>>
>>>> Do we have a document yet talking about the best practices for how we
>>>> would like a hardware vendor to ship, store and possibly update a device
>>>> tree, on the hardware?  "However they like" seems likely to invite
>>>> problems down the line with everyone trying their own thing.  Thanks!
>>>
>>> I don't believe there's any written guidance, no.
>>>
>>> The initial guidance Grant gave (IIRC at the first Linaro Connect last
>>> year, or perhaps the ARM workshop in Prague, or perhaps also in various
>>> ARM kernel list threads?) was that the DTBs should be stored/accessed in
>>> exactly the same way as the kernel, which on many systems implies it's a
>>> file in /boot (although MTD partitions, ... are also possible kernel
>>> locations). The idea here was to explicitly make upgrading the DTB as
>>> easy as upgrading the kernel, and explicitly without having to upgrade
>>> any firmware, since that's a more dangerous process in most cases.
>>>
>>> Now perhaps that advice was only intended to apply very early on when DT
>>> was really new on ARM, and has "aged out" by now? If so, I don't recall
>>> that every being explicitly mentioned or communicated later.
>> [snip out a bit more of Stephen's answer]
>>
>> Yes, this notion certainly is the opposite of what was suggested on the
>> cross-distro list, both as part of a "what should a bootloader provide
>> to get commodity distros to support the board" thread and the "where
>> should a device tree live in the filesystem" thread.  Cc'ing them now
>> because this is one of those things that feels like it needs solving,
>> solving soon, and done in a way the least number of folks get surprised
>> about it.
> 
> I don't think the "however they like" approach is that bad.  It is 
> certainly less of a problem than "exactly the same way as the kernel" is.
> 
> Using /boot implies a distro filesystem and we'd rather wish for DT to 
> be independent from a distro or even Linux.  The DTB should therefore be 
> stored and accessed in a way which is hardware/bootloader specific.  The 
> distro being installed on top shouldn't have to care.

Right, the expectation has been set that the DT isn't supposed to be
tied to the kernel and I don't wish to re-hash that.  I'm asking, should
we provide some expectations / guidance on how to store the DT?  Should
quirks be expected to be worked around at the DT consumer level (kernel,
bootloader, whatever) or in updating the stored copy?  That there rules
out, or not, certain choices.

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Wed, 14 Aug 2013, Tom Rini wrote:

> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > On Wed, 14 Aug 2013, Tom Rini wrote:
> > 
> >> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
> >>> On 08/14/2013 09:13 AM, Tom Rini wrote:
> >>>> Hey all,
> >>>>
> >>>> Do we have a document yet talking about the best practices for how we
> >>>> would like a hardware vendor to ship, store and possibly update a device
> >>>> tree, on the hardware?  "However they like" seems likely to invite
> >>>> problems down the line with everyone trying their own thing.  Thanks!
> >>>
> >>> I don't believe there's any written guidance, no.
> >>>
> >>> The initial guidance Grant gave (IIRC at the first Linaro Connect last
> >>> year, or perhaps the ARM workshop in Prague, or perhaps also in various
> >>> ARM kernel list threads?) was that the DTBs should be stored/accessed in
> >>> exactly the same way as the kernel, which on many systems implies it's a
> >>> file in /boot (although MTD partitions, ... are also possible kernel
> >>> locations). The idea here was to explicitly make upgrading the DTB as
> >>> easy as upgrading the kernel, and explicitly without having to upgrade
> >>> any firmware, since that's a more dangerous process in most cases.
> >>>
> >>> Now perhaps that advice was only intended to apply very early on when DT
> >>> was really new on ARM, and has "aged out" by now? If so, I don't recall
> >>> that every being explicitly mentioned or communicated later.
> >> [snip out a bit more of Stephen's answer]
> >>
> >> Yes, this notion certainly is the opposite of what was suggested on the
> >> cross-distro list, both as part of a "what should a bootloader provide
> >> to get commodity distros to support the board" thread and the "where
> >> should a device tree live in the filesystem" thread.  Cc'ing them now
> >> because this is one of those things that feels like it needs solving,
> >> solving soon, and done in a way the least number of folks get surprised
> >> about it.
> > 
> > I don't think the "however they like" approach is that bad.  It is 
> > certainly less of a problem than "exactly the same way as the kernel" is.
> > 
> > Using /boot implies a distro filesystem and we'd rather wish for DT to 
> > be independent from a distro or even Linux.  The DTB should therefore be 
> > stored and accessed in a way which is hardware/bootloader specific.  The 
> > distro being installed on top shouldn't have to care.
> 
> Right, the expectation has been set that the DT isn't supposed to be
> tied to the kernel and I don't wish to re-hash that.  I'm asking, should
> we provide some expectations / guidance on how to store the DT?  Should
> quirks be expected to be worked around at the DT consumer level (kernel,
> bootloader, whatever) or in updating the stored copy?  That there rules
> out, or not, certain choices.

Well, the hard guideline should require that the DTB be updateable and 
not linked with nor generated by the bootloader or firmware.  That 
implies some storage separate from the bootloader but this doesn't need 
to be a filesystem.

And the DTB should be updated far less often than the kernel.  So that 
might be acceptable to have a special bootloader-specific procedure to 
update it.

I expect that the kernel will have to deal with quirks from a bad DTB 
that has already been deployed.  But those quirks workarounds only have 
to make things minimally work while the optimal state would come from an 
updated DTB.  Same story as with bad BIOSes on a X86 system.

More importantly, a DTB update is likely to be useful in providing new 
data the previous DTB didn't include e.g. if some bindings weren't 
finalized when the initial deployment took place.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> On Wed, 14 Aug 2013, Tom Rini wrote:
> 
> > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > 
> > >> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
> > >>> On 08/14/2013 09:13 AM, Tom Rini wrote:
> > >>>> Hey all,
> > >>>>
> > >>>> Do we have a document yet talking about the best practices for how we
> > >>>> would like a hardware vendor to ship, store and possibly update a device
> > >>>> tree, on the hardware?  "However they like" seems likely to invite
> > >>>> problems down the line with everyone trying their own thing.  Thanks!
> > >>>
> > >>> I don't believe there's any written guidance, no.
> > >>>
> > >>> The initial guidance Grant gave (IIRC at the first Linaro Connect last
> > >>> year, or perhaps the ARM workshop in Prague, or perhaps also in various
> > >>> ARM kernel list threads?) was that the DTBs should be stored/accessed in
> > >>> exactly the same way as the kernel, which on many systems implies it's a
> > >>> file in /boot (although MTD partitions, ... are also possible kernel
> > >>> locations). The idea here was to explicitly make upgrading the DTB as
> > >>> easy as upgrading the kernel, and explicitly without having to upgrade
> > >>> any firmware, since that's a more dangerous process in most cases.
> > >>>
> > >>> Now perhaps that advice was only intended to apply very early on when DT
> > >>> was really new on ARM, and has "aged out" by now? If so, I don't recall
> > >>> that every being explicitly mentioned or communicated later.
> > >> [snip out a bit more of Stephen's answer]
> > >>
> > >> Yes, this notion certainly is the opposite of what was suggested on the
> > >> cross-distro list, both as part of a "what should a bootloader provide
> > >> to get commodity distros to support the board" thread and the "where
> > >> should a device tree live in the filesystem" thread.  Cc'ing them now
> > >> because this is one of those things that feels like it needs solving,
> > >> solving soon, and done in a way the least number of folks get surprised
> > >> about it.
> > > 
> > > I don't think the "however they like" approach is that bad.  It is 
> > > certainly less of a problem than "exactly the same way as the kernel" is.
> > > 
> > > Using /boot implies a distro filesystem and we'd rather wish for DT to 
> > > be independent from a distro or even Linux.  The DTB should therefore be 
> > > stored and accessed in a way which is hardware/bootloader specific.  The 
> > > distro being installed on top shouldn't have to care.
> > 
> > Right, the expectation has been set that the DT isn't supposed to be
> > tied to the kernel and I don't wish to re-hash that.  I'm asking, should
> > we provide some expectations / guidance on how to store the DT?  Should
> > quirks be expected to be worked around at the DT consumer level (kernel,
> > bootloader, whatever) or in updating the stored copy?  That there rules
> > out, or not, certain choices.
> 
> Well, the hard guideline should require that the DTB be updateable and 
> not linked with nor generated by the bootloader or firmware.  That 
> implies some storage separate from the bootloader but this doesn't need 
> to be a filesystem.

I assume generated doesn't mean not updated.  Do we want to limit the
nodes that can be touched here?  For example, a number of platforms have
incorrect memory size (and on PowerPC, intentionally zero) so that the
bootloader can correct them.  And not linked with the bootloader may (or
may not, Stephen, can you comment on what Tegra is doing at the moment?)
make it harder to use in those cases.

> And the DTB should be updated far less often than the kernel.  So that 
> might be acceptable to have a special bootloader-specific procedure to 
> update it.

Right.  But you wouldn't want it to be a hardware mod to make things be
writable again (thinking the EEPROM I mentioned before).

> I expect that the kernel will have to deal with quirks from a bad DTB 
> that has already been deployed.  But those quirks workarounds only have 
> to make things minimally work while the optimal state would come from an 
> updated DTB.  Same story as with bad BIOSes on a X86 system.
> 
> More importantly, a DTB update is likely to be useful in providing new 
> data the previous DTB didn't include e.g. if some bindings weren't 
> finalized when the initial deployment took place.

And what bindings it should use in the interm depends on the general
outcome of when bindings get locked in and how all of that.  But does
this mean that the expectation, on the kernel side, would be that a
developer should use the latest published bindings for a platform, or
the bindings that shipped the very first time?  For bootloaders the rule
is "whatever shipped on the hardware shouldn't be touched".

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Stephen Warren]

On 08/15/2013 08:53 AM, Tom Rini wrote:
> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>
>>> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>>>
>>>>> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
>>>>>> On 08/14/2013 09:13 AM, Tom Rini wrote:
>>>>>>> Hey all,
>>>>>>>
>>>>>>> Do we have a document yet talking about the best practices for how we
>>>>>>> would like a hardware vendor to ship, store and possibly update a device
>>>>>>> tree, on the hardware?  "However they like" seems likely to invite
>>>>>>> problems down the line with everyone trying their own thing.  Thanks!
>>>>>>
>>>>>> I don't believe there's any written guidance, no.
>>>>>>
>>>>>> The initial guidance Grant gave (IIRC at the first Linaro Connect last
>>>>>> year, or perhaps the ARM workshop in Prague, or perhaps also in various
>>>>>> ARM kernel list threads?) was that the DTBs should be stored/accessed in
>>>>>> exactly the same way as the kernel, which on many systems implies it's a
>>>>>> file in /boot (although MTD partitions, ... are also possible kernel
>>>>>> locations). The idea here was to explicitly make upgrading the DTB as
>>>>>> easy as upgrading the kernel, and explicitly without having to upgrade
>>>>>> any firmware, since that's a more dangerous process in most cases.
>>>>>>
>>>>>> Now perhaps that advice was only intended to apply very early on when DT
>>>>>> was really new on ARM, and has "aged out" by now? If so, I don't recall
>>>>>> that every being explicitly mentioned or communicated later.
>>>>> [snip out a bit more of Stephen's answer]
>>>>>
>>>>> Yes, this notion certainly is the opposite of what was suggested on the
>>>>> cross-distro list, both as part of a "what should a bootloader provide
>>>>> to get commodity distros to support the board" thread and the "where
>>>>> should a device tree live in the filesystem" thread.  Cc'ing them now
>>>>> because this is one of those things that feels like it needs solving,
>>>>> solving soon, and done in a way the least number of folks get surprised
>>>>> about it.
>>>>
>>>> I don't think the "however they like" approach is that bad.  It is 
>>>> certainly less of a problem than "exactly the same way as the kernel" is.
>>>>
>>>> Using /boot implies a distro filesystem and we'd rather wish for DT to 
>>>> be independent from a distro or even Linux.  The DTB should therefore be 
>>>> stored and accessed in a way which is hardware/bootloader specific.  The 
>>>> distro being installed on top shouldn't have to care.
>>>
>>> Right, the expectation has been set that the DT isn't supposed to be
>>> tied to the kernel and I don't wish to re-hash that.  I'm asking, should
>>> we provide some expectations / guidance on how to store the DT?  Should
>>> quirks be expected to be worked around at the DT consumer level (kernel,
>>> bootloader, whatever) or in updating the stored copy?  That there rules
>>> out, or not, certain choices.
>>
>> Well, the hard guideline should require that the DTB be updateable and 
>> not linked with nor generated by the bootloader or firmware.  That 
>> implies some storage separate from the bootloader but this doesn't need 
>> to be a filesystem.
> 
> I assume generated doesn't mean not updated.  Do we want to limit the
> nodes that can be touched here?  For example, a number of platforms have
> incorrect memory size (and on PowerPC, intentionally zero) so that the
> bootloader can correct them.  And not linked with the bootloader may (or
> may not, Stephen, can you comment on what Tegra is doing at the moment?)
> make it harder to use in those cases.

On Tegra, there are two DTBs:

The first is attached to the U-Boot image, and parameterizes U-Boot
itself. The bindings used in this are often quite different from the
kernel:-(

The second is the DTB passed to the kernel. Right now, we're using the
model of putting this in /boot on the root filesystem and having the
U-Boot boot script load it (with filename generated as
${soc}-${board}.dtb) and pass it to the kernel. Moving this into boot
flash at some well-known location rather than the filesystem might be
possible in the long term.

Perhaps the two should be unified, although there isn't much interest in
bringing the U-Boot DT content up to scratch, it seems:-( I would just
ignore the U-Boot copy for now, and treat it as an internal
implementation detail of U-Boot.

We certainly expect U-Boot to update the DT that's passed to the kernel,
for:

* Memory size (although it's typically already correct in all the DTBs
anyway)
* Command-line
* Possibly to add a "simplefb" node, until the kernel has LCD panel
support (CDF).

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Thu, 15 Aug 2013, Tom Rini wrote:

> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> > On Wed, 14 Aug 2013, Tom Rini wrote:
> > 
> > > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > > I don't think the "however they like" approach is that bad.  It is 
> > > > certainly less of a problem than "exactly the same way as the kernel" is.
> > > > 
> > > > Using /boot implies a distro filesystem and we'd rather wish for DT to 
> > > > be independent from a distro or even Linux.  The DTB should therefore be 
> > > > stored and accessed in a way which is hardware/bootloader specific.  The 
> > > > distro being installed on top shouldn't have to care.
> > > 
> > > Right, the expectation has been set that the DT isn't supposed to be
> > > tied to the kernel and I don't wish to re-hash that.  I'm asking, should
> > > we provide some expectations / guidance on how to store the DT?  Should
> > > quirks be expected to be worked around at the DT consumer level (kernel,
> > > bootloader, whatever) or in updating the stored copy?  That there rules
> > > out, or not, certain choices.
> > 
> > Well, the hard guideline should require that the DTB be updateable and 
> > not linked with nor generated by the bootloader or firmware.  That 
> > implies some storage separate from the bootloader but this doesn't need 
> > to be a filesystem.
> 
> I assume generated doesn't mean not updated.

Indeed.  For instance, the bootloader is likely to update it with some 
user provided kernel cmdline string, and possibly ethernet MAC addresses 
and/or some board serial number, etc.

BTW providing a unique serial number via DT to the kernel would help a 
lot in the seeding of the kernel's random pool.

> Do we want to limit the
> nodes that can be touched here?  For example, a number of platforms have
> incorrect memory size (and on PowerPC, intentionally zero) so that the
> bootloader can correct them.  And not linked with the bootloader may (or
> may not, Stephen, can you comment on what Tegra is doing at the moment?)
> make it harder to use in those cases.

U-Boot let you update any node you want once you've told it where in 
memory the DTB is.  I suppose such thing can be scripted or even added 
to the board specific U-Boot support easily.  As long as people are not 
overriding every nodes I suppose this is reasonable.

Especially if the hardware may accommodate different amount of RAM, in 
that case there is no other way than runtime probing of that RAM and the 
DTB must be updated accordingly.

> > And the DTB should be updated far less often than the kernel.  So that 
> > might be acceptable to have a special bootloader-specific procedure to 
> > update it.
> 
> Right.  But you wouldn't want it to be a hardware mod to make things be
> writable again (thinking the EEPROM I mentioned before).

No.

It must be end user updatable, even if the bootloader itself might not 
always be.

> > I expect that the kernel will have to deal with quirks from a bad DTB 
> > that has already been deployed.  But those quirks workarounds only have 
> > to make things minimally work while the optimal state would come from an 
> > updated DTB.  Same story as with bad BIOSes on a X86 system.
> > 
> > More importantly, a DTB update is likely to be useful in providing new 
> > data the previous DTB didn't include e.g. if some bindings weren't 
> > finalized when the initial deployment took place.
> 
> And what bindings it should use in the interm depends on the general
> outcome of when bindings get locked in and how all of that.  But does
> this mean that the expectation, on the kernel side, would be that a
> developer should use the latest published bindings for a platform, or
> the bindings that shipped the very first time?  For bootloaders the rule
> is "whatever shipped on the hardware shouldn't be touched".

I think the debate on this very topic is still ongoing.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Thu, Aug 15, 2013 at 09:30:01AM -0600, Stephen Warren wrote:
> On 08/15/2013 08:53 AM, Tom Rini wrote:
> > On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> >> On Wed, 14 Aug 2013, Tom Rini wrote:
> >>
> >>> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> >>>> On Wed, 14 Aug 2013, Tom Rini wrote:
> >>>>
> >>>>> On Wed, Aug 14, 2013 at 10:44:22AM -0600, Stephen Warren wrote:
> >>>>>> On 08/14/2013 09:13 AM, Tom Rini wrote:
> >>>>>>> Hey all,
> >>>>>>>
> >>>>>>> Do we have a document yet talking about the best practices for how we
> >>>>>>> would like a hardware vendor to ship, store and possibly update a device
> >>>>>>> tree, on the hardware?  "However they like" seems likely to invite
> >>>>>>> problems down the line with everyone trying their own thing.  Thanks!
> >>>>>>
> >>>>>> I don't believe there's any written guidance, no.
> >>>>>>
> >>>>>> The initial guidance Grant gave (IIRC at the first Linaro Connect last
> >>>>>> year, or perhaps the ARM workshop in Prague, or perhaps also in various
> >>>>>> ARM kernel list threads?) was that the DTBs should be stored/accessed in
> >>>>>> exactly the same way as the kernel, which on many systems implies it's a
> >>>>>> file in /boot (although MTD partitions, ... are also possible kernel
> >>>>>> locations). The idea here was to explicitly make upgrading the DTB as
> >>>>>> easy as upgrading the kernel, and explicitly without having to upgrade
> >>>>>> any firmware, since that's a more dangerous process in most cases.
> >>>>>>
> >>>>>> Now perhaps that advice was only intended to apply very early on when DT
> >>>>>> was really new on ARM, and has "aged out" by now? If so, I don't recall
> >>>>>> that every being explicitly mentioned or communicated later.
> >>>>> [snip out a bit more of Stephen's answer]
> >>>>>
> >>>>> Yes, this notion certainly is the opposite of what was suggested on the
> >>>>> cross-distro list, both as part of a "what should a bootloader provide
> >>>>> to get commodity distros to support the board" thread and the "where
> >>>>> should a device tree live in the filesystem" thread.  Cc'ing them now
> >>>>> because this is one of those things that feels like it needs solving,
> >>>>> solving soon, and done in a way the least number of folks get surprised
> >>>>> about it.
> >>>>
> >>>> I don't think the "however they like" approach is that bad.  It is 
> >>>> certainly less of a problem than "exactly the same way as the kernel" is.
> >>>>
> >>>> Using /boot implies a distro filesystem and we'd rather wish for DT to 
> >>>> be independent from a distro or even Linux.  The DTB should therefore be 
> >>>> stored and accessed in a way which is hardware/bootloader specific.  The 
> >>>> distro being installed on top shouldn't have to care.
> >>>
> >>> Right, the expectation has been set that the DT isn't supposed to be
> >>> tied to the kernel and I don't wish to re-hash that.  I'm asking, should
> >>> we provide some expectations / guidance on how to store the DT?  Should
> >>> quirks be expected to be worked around at the DT consumer level (kernel,
> >>> bootloader, whatever) or in updating the stored copy?  That there rules
> >>> out, or not, certain choices.
> >>
> >> Well, the hard guideline should require that the DTB be updateable and 
> >> not linked with nor generated by the bootloader or firmware.  That 
> >> implies some storage separate from the bootloader but this doesn't need 
> >> to be a filesystem.
> > 
> > I assume generated doesn't mean not updated.  Do we want to limit the
> > nodes that can be touched here?  For example, a number of platforms have
> > incorrect memory size (and on PowerPC, intentionally zero) so that the
> > bootloader can correct them.  And not linked with the bootloader may (or
> > may not, Stephen, can you comment on what Tegra is doing at the moment?)
> > make it harder to use in those cases.
> 
> On Tegra, there are two DTBs:
> 
> The first is attached to the U-Boot image, and parameterizes U-Boot
> itself. The bindings used in this are often quite different from the
> kernel:-(
[snip]
> Perhaps the two should be unified, although there isn't much interest in
> bringing the U-Boot DT content up to scratch, it seems:-( I would just
> ignore the U-Boot copy for now, and treat it as an internal
> implementation detail of U-Boot.

I would very much like to see them unified and used more in U-Boot.
Most U-Boot development is about scratching ones own itch, is the
biggest hurdle towards mass conversion to DTs (followed by the number of
platforms that aren't being updated to DT in the kernel, followed by
needing more conversion done in the kernel, bindings finalized, etc).

But that's why I bring up "not attached to bootloader/firmware" as a
question here.  If we get the bindings in-sync, and make it easy to trim
down a tree, we could still attach what the firmware needs to know to
know about the rest of the system and have the full DT somewhere.

> We certainly expect U-Boot to update the DT that's passed to the kernel,
> for:
> 
> * Memory size (although it's typically already correct in all the DTBs
> anyway)

... for Tegra (for clarity).

> * Command-line
> * Possibly to add a "simplefb" node, until the kernel has LCD panel
> support (CDF).

Can we talk about that last one more?  That sounds like some sort of
temporary binding.  Or am I using the terms wrong?  But, it sounds like
we're saying add whatever nodes might be missing?

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Thu, Aug 15, 2013 at 11:45:57AM -0400, Nicolas Pitre wrote:
> On Thu, 15 Aug 2013, Tom Rini wrote:
> 
> > On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > 
> > > > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > > > I don't think the "however they like" approach is that bad.  It is 
> > > > > certainly less of a problem than "exactly the same way as the kernel" is.
> > > > > 
> > > > > Using /boot implies a distro filesystem and we'd rather wish for DT to 
> > > > > be independent from a distro or even Linux.  The DTB should therefore be 
> > > > > stored and accessed in a way which is hardware/bootloader specific.  The 
> > > > > distro being installed on top shouldn't have to care.
> > > > 
> > > > Right, the expectation has been set that the DT isn't supposed to be
> > > > tied to the kernel and I don't wish to re-hash that.  I'm asking, should
> > > > we provide some expectations / guidance on how to store the DT?  Should
> > > > quirks be expected to be worked around at the DT consumer level (kernel,
> > > > bootloader, whatever) or in updating the stored copy?  That there rules
> > > > out, or not, certain choices.
> > > 
> > > Well, the hard guideline should require that the DTB be updateable and 
> > > not linked with nor generated by the bootloader or firmware.  That 
> > > implies some storage separate from the bootloader but this doesn't need 
> > > to be a filesystem.
> > 
> > I assume generated doesn't mean not updated.
> 
> Indeed.  For instance, the bootloader is likely to update it with some 
> user provided kernel cmdline string, and possibly ethernet MAC addresses 
> and/or some board serial number, etc.

Right, command line, memory and mac addresses are what U-Boot does
today.  Is there a standard binding for board serial number?  If so we
could update that too, if set.

> BTW providing a unique serial number via DT to the kernel would help a 
> lot in the seeding of the kernel's random pool.
> 
> > Do we want to limit the
> > nodes that can be touched here?  For example, a number of platforms have
> > incorrect memory size (and on PowerPC, intentionally zero) so that the
> > bootloader can correct them.  And not linked with the bootloader may (or
> > may not, Stephen, can you comment on what Tegra is doing at the moment?)
> > make it harder to use in those cases.
> 
> U-Boot let you update any node you want once you've told it where in 
> memory the DTB is.  I suppose such thing can be scripted or even added 
> to the board specific U-Boot support easily.  As long as people are not 
> overriding every nodes I suppose this is reasonable.

Right, U-Boot has a 'fdt' command for fiddling with the in-memory DT as
well as updating certain nodes as part of booting the kernel.  This
could be used to "hotfix" the stored device tree if desired.  Today this
is used more (or at least, so it seems) while developing your DT, or
simply functionality people didn't widely know was there.

> Especially if the hardware may accommodate different amount of RAM, in 
> that case there is no other way than runtime probing of that RAM and the 
> DTB must be updated accordingly.

This is probably why many PowerPC platforms have a zero size memory
node, yes.  At one point I was talking about saying the standard should
be zero, and then we assume that a non-zero value must be correct (for
platforms with lots and lots of memory where U-Boot is setting things
wrong now).

> > > And the DTB should be updated far less often than the kernel.  So that 
> > > might be acceptable to have a special bootloader-specific procedure to 
> > > update it.
> > 
> > Right.  But you wouldn't want it to be a hardware mod to make things be
> > writable again (thinking the EEPROM I mentioned before).
> 
> No.
> 
> It must be end user updatable, even if the bootloader itself might not 
> always be.

So one of the rules for the guidelines :)

> > > I expect that the kernel will have to deal with quirks from a bad DTB 
> > > that has already been deployed.  But those quirks workarounds only have 
> > > to make things minimally work while the optimal state would come from an 
> > > updated DTB.  Same story as with bad BIOSes on a X86 system.
> > > 
> > > More importantly, a DTB update is likely to be useful in providing new 
> > > data the previous DTB didn't include e.g. if some bindings weren't 
> > > finalized when the initial deployment took place.
> > 
> > And what bindings it should use in the interm depends on the general
> > outcome of when bindings get locked in and how all of that.  But does
> > this mean that the expectation, on the kernel side, would be that a
> > developer should use the latest published bindings for a platform, or
> > the bindings that shipped the very first time?  For bootloaders the rule
> > is "whatever shipped on the hardware shouldn't be touched".
> 
> I think the debate on this very topic is still ongoing.

Right.  And that's part of why I brought the topic up.  Using my magic
VPN-based crystal ball, I can see when for example TI plans to release
platforms, think a little about timelines and go "Gee, if we want the
device tree included with the hardware when it ships, I need to know
what's expected somewhere between a month from now and 3 months ago,
crap.".

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Stephen Warren]

On 08/15/2013 10:37 AM, Tom Rini wrote:
> On Thu, Aug 15, 2013 at 09:30:01AM -0600, Stephen Warren wrote:
...
>> On Tegra, there are two DTBs:
>>
>> The first is attached to the U-Boot image, and parameterizes U-Boot
>> itself. The bindings used in this are often quite different from the
>> kernel:-(
> [snip]
>> Perhaps the two should be unified, although there isn't much interest in
>> bringing the U-Boot DT content up to scratch, it seems:-( I would just
>> ignore the U-Boot copy for now, and treat it as an internal
>> implementation detail of U-Boot.
> 
> I would very much like to see them unified and used more in U-Boot.
> Most U-Boot development is about scratching ones own itch, is the
> biggest hurdle towards mass conversion to DTs (followed by the number of
> platforms that aren't being updated to DT in the kernel, followed by
> needing more conversion done in the kernel, bindings finalized, etc).

I was mainly talking about the people specifically working on Tegra
support in U-Boot rather than U-Boot as a generality.

...
>> We certainly expect U-Boot to update the DT that's passed to the kernel,
>> for:
...
>> * Possibly to add a "simplefb" node, until the kernel has LCD panel
>> support (CDF).
> 
> Can we talk about that last one more?  That sounds like some sort of
> temporary binding.  Or am I using the terms wrong?  But, it sounds like
> we're saying add whatever nodes might be missing?

For Tegra, this should be temporary, although temporary will probably be
a long time! For other SoCs, it may be a reasonable long-term solution.

The simplefb DT binding and associated Linux driver simply defines a
memory region as being the frame-buffer, so that the OS can just write
pixels to the memory region, and they'll be displayed. It's assumed that
the HW to do the scanout is already set up before the OS starts.

For Tegra, this is useful on some platforms that have built-in LCDs,
since the kernel support for LCDs, which has to be the CDF (Common
Display Framework) under the new DT and common driver regime, is
perpetually missing. Having U-Boot create a simplefb node in DT and
enabling the simplefb driver in the kernel is a way of getting useful
display support prior to CDF getting merged.

The same argument applies to upstream kernel support for the Samsung
Exynos CPU used in the ARM ChromeBook.

For the Raspberry Pi, I wanted to avoid defining DT bindings for the RPC
communication channel out to the VideoCore co-processor which controls
the display path. Relying on simplefb for the display allowed me to
defer creating bindings/drivers for the RPC channel etc.

On other HW, the display HW may be so simple that it's just not worth
writing an OS driver; the best way to support the HW may always be to
have a simplefb node in DT and have a single driver in the bootloader
that's already initialized the HW, rather than writing an OS driver just
to re-do what the bootloader already did.

simplefb information could get into DT in a couple ways:

1) The DTB for the platform could include the simplefb node right from
the start.

This model would be appropriate for more permanent use of simplefb,
since the node appears in the orignial DT.

In this case, the bootloader would at least have to fill in the physical
memory address of the frame-buffer that it allocated. If the display is
device with variable resolution, rather than a fixed built-in LCD, the
bootloader would also have to fill in the resolution properties.

2) The DTB for the platform doesn't include the simplefb node.

This would be appropriate for platforms that use simplefb as a temporary
step until complete OS support for the display HW is present, so a real
driver can be used. This prevents DTBs from ever having the
soon-to-be-removed simplefb node encoded into them.

In this case, the bootloader would have to add a completely new simplefb
node to the DT, and fill in all the properties.

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On Thu, Aug 15, 2013 at 11:31:54AM -0600, Stephen Warren wrote:
> On 08/15/2013 10:37 AM, Tom Rini wrote:
> > On Thu, Aug 15, 2013 at 09:30:01AM -0600, Stephen Warren wrote:
> ...
> >> On Tegra, there are two DTBs:
> >>
> >> The first is attached to the U-Boot image, and parameterizes U-Boot
> >> itself. The bindings used in this are often quite different from the
> >> kernel:-(
> > [snip]
> >> Perhaps the two should be unified, although there isn't much interest in
> >> bringing the U-Boot DT content up to scratch, it seems:-( I would just
> >> ignore the U-Boot copy for now, and treat it as an internal
> >> implementation detail of U-Boot.
> > 
> > I would very much like to see them unified and used more in U-Boot.
> > Most U-Boot development is about scratching ones own itch, is the
> > biggest hurdle towards mass conversion to DTs (followed by the number of
> > platforms that aren't being updated to DT in the kernel, followed by
> > needing more conversion done in the kernel, bindings finalized, etc).
> 
> I was mainly talking about the people specifically working on Tegra
> support in U-Boot rather than U-Boot as a generality.

Right.  And with my U-Boot guy hat on, I want to be clear that this is a
direction I want to see the project move in, once things have settled
down a bit with the kernel side of things (since we'll have our own
headaches like "how little of the system can we hard-code" and "how can
we merge device trees, or do we discard that first one?").

> ...
> >> We certainly expect U-Boot to update the DT that's passed to the kernel,
> >> for:
> ...
> >> * Possibly to add a "simplefb" node, until the kernel has LCD panel
> >> support (CDF).
> > 
> > Can we talk about that last one more?  That sounds like some sort of
> > temporary binding.  Or am I using the terms wrong?  But, it sounds like
> > we're saying add whatever nodes might be missing?
[snip useful explanation of what simplefb is]
> simplefb information could get into DT in a couple ways:
> 
> 1) The DTB for the platform could include the simplefb node right from
> the start.
> 
> This model would be appropriate for more permanent use of simplefb,
> since the node appears in the orignial DT.
> 
> In this case, the bootloader would at least have to fill in the physical
> memory address of the frame-buffer that it allocated. If the display is
> device with variable resolution, rather than a fixed built-in LCD, the
> bootloader would also have to fill in the resolution properties.
> 
> 2) The DTB for the platform doesn't include the simplefb node.
> 
> This would be appropriate for platforms that use simplefb as a temporary
> step until complete OS support for the display HW is present, so a real
> driver can be used. This prevents DTBs from ever having the
> soon-to-be-removed simplefb node encoded into them.
> 
> In this case, the bootloader would have to add a completely new simplefb
> node to the DT, and fill in all the properties.

OK, now why wouldn't the right thing to do here be to #1 and later in
time, the DT is updated when the better drivers are merged in?

I don't think it's strictly a bad thing to say that firmware/bootloader
is allowed to construct temporary nodes, but we should make it clear
when that's an OK practice.  Yes?

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Stephen Warren]

On 08/15/2013 12:56 PM, Tom Rini wrote:
> On Thu, Aug 15, 2013 at 11:31:54AM -0600, Stephen Warren wrote:
>> On 08/15/2013 10:37 AM, Tom Rini wrote:
>>> On Thu, Aug 15, 2013 at 09:30:01AM -0600, Stephen Warren wrote:
...
>>>> We certainly expect U-Boot to update the DT that's passed to the kernel,
>>>> for:
>> ...
>>>> * Possibly to add a "simplefb" node, until the kernel has LCD panel
>>>> support (CDF).
>>>
>>> Can we talk about that last one more?  That sounds like some sort of
>>> temporary binding.  Or am I using the terms wrong?  But, it sounds like
>>> we're saying add whatever nodes might be missing?
> [snip useful explanation of what simplefb is]
>> simplefb information could get into DT in a couple ways:
>>
>> 1) The DTB for the platform could include the simplefb node right from
>> the start.
>>
>> This model would be appropriate for more permanent use of simplefb,
>> since the node appears in the orignial DT.
>>
>> In this case, the bootloader would at least have to fill in the physical
>> memory address of the frame-buffer that it allocated. If the display is
>> device with variable resolution, rather than a fixed built-in LCD, the
>> bootloader would also have to fill in the resolution properties.
>>
>> 2) The DTB for the platform doesn't include the simplefb node.
>>
>> This would be appropriate for platforms that use simplefb as a temporary
>> step until complete OS support for the display HW is present, so a real
>> driver can be used. This prevents DTBs from ever having the
>> soon-to-be-removed simplefb node encoded into them.
>>
>> In this case, the bootloader would have to add a completely new simplefb
>> node to the DT, and fill in all the properties.
> 
> OK, now why wouldn't the right thing to do here be to #1 and later in
> time, the DT is updated when the better drivers are merged in?
> 
> I don't think it's strictly a bad thing to say that firmware/bootloader
> is allowed to construct temporary nodes, but we should make it clear
> when that's an OK practice.  Yes?

For systems where simplefb is a transitional measure (i.e. where it
would eventually make sense to define a complete binding for the HW),
I'd prefer to avoid churn in the *.dts files by never including the
simplefb node in them, but rather adding it dynamically. In other words,
we don't add something to the file just to take it away later.

That's the logic behind why I wrote separate "add node" and "fill in
node" functions for the simplefb in U-Boot.

One thing that's slightly different about simplefb is that the node
isn't useful without the bootloader filling in some information, which
generally isn't true of DT content.

It'd certainly be technically possible to just add the simplefb node
into DT in all cases, then swap it out later when something else came along.

Re: [RFC] Best practices for hardware shipping device trees

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 920 bytes --]

On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> On Wed, 14 Aug 2013, Tom Rini wrote:
> > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > On Wed, 14 Aug 2013, Tom Rini wrote:
[snip]
> Well, the hard guideline should require that the DTB be updateable and 
> not linked with nor generated by the bootloader or firmware.  That 
> implies some storage separate from the bootloader but this doesn't need 
> to be a filesystem.

Wait, what!?

Much as I think a bunch of the current problems have been caused by
being overly keen to push the dtb into firmware, we shouldn't *ban*
the original Open Firmware model of the device tree, where it is
generated by the firmware and consumed by the OS.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Fri, 16 Aug 2013, David Gibson wrote:

> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > > On Wed, 14 Aug 2013, Tom Rini wrote:
> [snip]
> > Well, the hard guideline should require that the DTB be updateable and 
> > not linked with nor generated by the bootloader or firmware.  That 
> > implies some storage separate from the bootloader but this doesn't need 
> > to be a filesystem.
> 
> Wait, what!?
> 
> Much as I think a bunch of the current problems have been caused by
> being overly keen to push the dtb into firmware, we shouldn't *ban*
> the original Open Firmware model of the device tree, where it is
> generated by the firmware and consumed by the OS.

If the DTB generating firmware can be updated by the end user just as 
easily and safely as a standalone DTB then that's probably fine.  But we 
do know that many people/organizations are not willing to let end users 
upgrade bootloaders due to the risks associated with such an operation.  
So in that case we may not suggest the DTB be tied to the 
bootloader/firmware.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

On Thu, Aug 15, 2013 at 10:51:18PM -0400, Nicolas Pitre wrote:
> On Fri, 16 Aug 2013, David Gibson wrote:
> 
> > On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > [snip]
> > > Well, the hard guideline should require that the DTB be updateable and 
> > > not linked with nor generated by the bootloader or firmware.  That 
> > > implies some storage separate from the bootloader but this doesn't need 
> > > to be a filesystem.
> > 
> > Wait, what!?
> > 
> > Much as I think a bunch of the current problems have been caused by
> > being overly keen to push the dtb into firmware, we shouldn't *ban*
> > the original Open Firmware model of the device tree, where it is
> > generated by the firmware and consumed by the OS.
> 
> If the DTB generating firmware can be updated by the end user just as 
> easily and safely as a standalone DTB then that's probably fine.  But we 
> do know that many people/organizations are not willing to let end users 
> upgrade bootloaders due to the risks associated with such an operation.  
> So in that case we may not suggest the DTB be tied to the 
> bootloader/firmware.

No, even then.  I really don't think trying to ban the actual,
original Open Firmware model of device tree usage is sensible.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Mon, 19 Aug 2013, David Gibson wrote:

> On Thu, Aug 15, 2013 at 10:51:18PM -0400, Nicolas Pitre wrote:
> > On Fri, 16 Aug 2013, David Gibson wrote:
> > 
> > > On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> > > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > > > On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> > > > > > On Wed, 14 Aug 2013, Tom Rini wrote:
> > > [snip]
> > > > Well, the hard guideline should require that the DTB be updateable and 
> > > > not linked with nor generated by the bootloader or firmware.  That 
> > > > implies some storage separate from the bootloader but this doesn't need 
> > > > to be a filesystem.
> > > 
> > > Wait, what!?
> > > 
> > > Much as I think a bunch of the current problems have been caused by
> > > being overly keen to push the dtb into firmware, we shouldn't *ban*
> > > the original Open Firmware model of the device tree, where it is
> > > generated by the firmware and consumed by the OS.
> > 
> > If the DTB generating firmware can be updated by the end user just as 
> > easily and safely as a standalone DTB then that's probably fine.  But we 
> > do know that many people/organizations are not willing to let end users 
> > upgrade bootloaders due to the risks associated with such an operation.  
> > So in that case we may not suggest the DTB be tied to the 
> > bootloader/firmware.
> 
> No, even then.  I really don't think trying to ban the actual,
> original Open Firmware model of device tree usage is sensible.

I don't think we should *ban* anything.  This is rather about 
recommending best practice to people.

And if your bootloader produces a bad DTB and you cannot let end users 
upgrade the bootloader then this is certainly against best practice.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[Alexander Graf]

On 19.08.2013, at 19:43, Nicolas Pitre wrote:

> On Mon, 19 Aug 2013, David Gibson wrote:
> 
>> On Thu, Aug 15, 2013 at 10:51:18PM -0400, Nicolas Pitre wrote:
>>> On Fri, 16 Aug 2013, David Gibson wrote:
>>> 
>>>> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
>>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>>>>> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
>>>>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>>> [snip]
>>>>> Well, the hard guideline should require that the DTB be updateable and 
>>>>> not linked with nor generated by the bootloader or firmware.  That 
>>>>> implies some storage separate from the bootloader but this doesn't need 
>>>>> to be a filesystem.
>>>> 
>>>> Wait, what!?
>>>> 
>>>> Much as I think a bunch of the current problems have been caused by
>>>> being overly keen to push the dtb into firmware, we shouldn't *ban*
>>>> the original Open Firmware model of the device tree, where it is
>>>> generated by the firmware and consumed by the OS.
>>> 
>>> If the DTB generating firmware can be updated by the end user just as 
>>> easily and safely as a standalone DTB then that's probably fine.  But we 
>>> do know that many people/organizations are not willing to let end users 
>>> upgrade bootloaders due to the risks associated with such an operation.  
>>> So in that case we may not suggest the DTB be tied to the 
>>> bootloader/firmware.
>> 
>> No, even then.  I really don't think trying to ban the actual,
>> original Open Firmware model of device tree usage is sensible.
> 
> I don't think we should *ban* anything.  This is rather about 
> recommending best practice to people.
> 
> And if your bootloader produces a bad DTB and you cannot let end users 
> upgrade the bootloader then this is certainly against best practice.

But that's exactly how everyone else works. On x86 you get your ACPI tables from firmware, on PPC you get your device tree from firmware and you really don't update your firmware just because your OS thinks the tables are wrong.

Of course, this means that we get a bunch of quirks in Linux to support partially broken device description facilities, but it at least gives you a proper split between "firmware developer" as someone who knows the hardware and "kernel developer" as someone who tries to be as agnostic as possible.

Long story short, with UEFI coming I'm pretty sure things on ARM will move to the firmware delivered description tables anyway. So what exactly are we discussing here?


Alex

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Tue, 20 Aug 2013, Alexander Graf wrote:

> 
> On 19.08.2013, at 19:43, Nicolas Pitre wrote:
> 
> > On Mon, 19 Aug 2013, David Gibson wrote:
> > 
> >> On Thu, Aug 15, 2013 at 10:51:18PM -0400, Nicolas Pitre wrote:
> >>> On Fri, 16 Aug 2013, David Gibson wrote:
> >>> 
> >>>> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
> >>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
> >>>>>> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
> >>>>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
> >>>> [snip]
> >>>>> Well, the hard guideline should require that the DTB be updateable and 
> >>>>> not linked with nor generated by the bootloader or firmware.  That 
> >>>>> implies some storage separate from the bootloader but this doesn't need 
> >>>>> to be a filesystem.
> >>>> 
> >>>> Wait, what!?
> >>>> 
> >>>> Much as I think a bunch of the current problems have been caused by
> >>>> being overly keen to push the dtb into firmware, we shouldn't *ban*
> >>>> the original Open Firmware model of the device tree, where it is
> >>>> generated by the firmware and consumed by the OS.
> >>> 
> >>> If the DTB generating firmware can be updated by the end user just as 
> >>> easily and safely as a standalone DTB then that's probably fine.  But we 
> >>> do know that many people/organizations are not willing to let end users 
> >>> upgrade bootloaders due to the risks associated with such an operation.  
> >>> So in that case we may not suggest the DTB be tied to the 
> >>> bootloader/firmware.
> >> 
> >> No, even then.  I really don't think trying to ban the actual,
> >> original Open Firmware model of device tree usage is sensible.
> > 
> > I don't think we should *ban* anything.  This is rather about 
> > recommending best practice to people.
> > 
> > And if your bootloader produces a bad DTB and you cannot let end users 
> > upgrade the bootloader then this is certainly against best practice.
> 
> But that's exactly how everyone else works. On x86 you get your ACPI 
> tables from firmware, on PPC you get your device tree from firmware 
> and you really don't update your firmware just because your OS thinks 
> the tables are wrong.

And everyone, except maybe yourself, despise that state of affair.

Sorry but we ought to push for something better when we can.  As device 
tree is being established on ARM, now is the time to learn from past 
mistakes and break away from the broken legacy methods.


Nicolas

Re: [RFC] Best practices for hardware shipping device trees

[Tom Rini]

On 08/20/2013 02:40 AM, Alexander Graf wrote:
> 
> On 19.08.2013, at 19:43, Nicolas Pitre wrote:
> 
>> On Mon, 19 Aug 2013, David Gibson wrote:
>>
>>> On Thu, Aug 15, 2013 at 10:51:18PM -0400, Nicolas Pitre wrote:
>>>> On Fri, 16 Aug 2013, David Gibson wrote:
>>>>
>>>>> On Wed, Aug 14, 2013 at 10:57:36PM -0400, Nicolas Pitre wrote:
>>>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>>>>>> On 08/14/2013 08:37 PM, Nicolas Pitre wrote:
>>>>>>>> On Wed, 14 Aug 2013, Tom Rini wrote:
>>>>> [snip]
>>>>>> Well, the hard guideline should require that the DTB be updateable and 
>>>>>> not linked with nor generated by the bootloader or firmware.  That 
>>>>>> implies some storage separate from the bootloader but this doesn't need 
>>>>>> to be a filesystem.
>>>>>
>>>>> Wait, what!?
>>>>>
>>>>> Much as I think a bunch of the current problems have been caused by
>>>>> being overly keen to push the dtb into firmware, we shouldn't *ban*
>>>>> the original Open Firmware model of the device tree, where it is
>>>>> generated by the firmware and consumed by the OS.
>>>>
>>>> If the DTB generating firmware can be updated by the end user just as 
>>>> easily and safely as a standalone DTB then that's probably fine.  But we 
>>>> do know that many people/organizations are not willing to let end users 
>>>> upgrade bootloaders due to the risks associated with such an operation.  
>>>> So in that case we may not suggest the DTB be tied to the 
>>>> bootloader/firmware.
>>>
>>> No, even then.  I really don't think trying to ban the actual,
>>> original Open Firmware model of device tree usage is sensible.
>>
>> I don't think we should *ban* anything.  This is rather about 
>> recommending best practice to people.
>>
>> And if your bootloader produces a bad DTB and you cannot let end users 
>> upgrade the bootloader then this is certainly against best practice.
> 
> But that's exactly how everyone else works. On x86 you get your ACPI
> tables from firmware, on PPC you get your device tree from firmware
> and you really don't update your firmware just because your OS thinks
> the tables are wrong.

I think others have pointed out elsewhere that it's not "OS thinks the
tables are wrong" but "ACPI tables are wrong, update your firmware" and
there are firmware updates to fix these problems required.

> Of course, this means that we get a bunch of quirks in Linux to
> support partially broken device description facilities, but it at
> least gives you a proper split between "firmware developer" as
> someone who knows the hardware and "kernel developer" as someone who
> tries to be as agnostic as possible.
> 
> Long story short, with UEFI coming I'm pretty sure things on ARM will
> move to the firmware delivered description tables anyway. So what
> exactly are we discussing here?

ARMv8 is moving towards having 3 combinations available, yes.
UEFI+ACPI, UEFI+Device Tree and not-UEFI+Device Tree.  And we're also
talking about ARMv7 and so on that while people are pushing UEFI in
places will probably be like x86 and UEFI, possible but not recommended
(and still end up being device trees, not ACPI).

-- 
Tom

Re: [RFC] Best practices for hardware shipping device trees

[Matt Sealey]

On Tue, Aug 20, 2013 at 7:02 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Tue, 20 Aug 2013, Alexander Graf wrote:
>
>
>> > And if your bootloader produces a bad DTB and you cannot let end users
>> > upgrade the bootloader then this is certainly against best practice.
>>
>> But that's exactly how everyone else works. On x86 you get your ACPI
>> tables from firmware, on PPC you get your device tree from firmware
>> and you really don't update your firmware just because your OS thinks
>> the tables are wrong.
>
> And everyone, except maybe yourself, despise that state of affair.
>
> Sorry but we ought to push for something better when we can.  As device
> tree is being established on ARM, now is the time to learn from past
> mistakes and break away from the broken legacy methods.

Well, provide support for real OpenFirmware or firmware-provided DTB,
and allow people to append DTBs to their kernels if they need to
update them.

The better state of affairs is that vendors who wish to support Linux
on the systems they ship would allow the firmware to be upgraded and
collaborate in the process. If they do not wish this or wish to
cooperate with this procedure, you can vote with your wallet; don't
buy their systems.

The idea that you could say "we accept some device trees but not all"
even if they follow standards at the time and the standards changed is
not going to fly, simply because what you're pushing is device trees,
and yes.. sometimes things change, and bindings move on, and they may
not be compatible.

In the case of a real OpenFirmware implementation it is ridiculously
easy to either patch the device tree in the firmware (load a Forth
script - I can point at two examples for this) or do it in the kernel
via CIF.

In the case of other bootloaders like U-Boot, libfdt provides the
ability to modify and insert loaded device trees to match (in which
case, the DTB bundled in the flash of some board is just a starting
point).

The worst case scenario is that it would be completely hosed, and in
this case an appended (or overridden by loading from filesystem or
other flash location) DTB is the solution - just like you can load
ACPI tables for some systems where the vendor shipped version is
totally broken.

There isn't a way around this, it's a fact of life in an imperfect
world. What you can do is keep Linux clean and move all device tree
modification/fixup crap to the vendor/bootloader responsibility where
it should be. If any of the above is not possible then it's going to
be incredibly difficult - to the point that insisting on complete
simplification and standardization of bootloader configurations and
"sanity" of device trees doesn't apply anyway. Sometimes vendors will
do what they want to do and not what you want to do; Linux no work,
and that is that.

Re: [RFC] Best practices for hardware shipping device trees

[Nicolas Pitre]

On Tue, 20 Aug 2013, Matt Sealey wrote:

> On Tue, Aug 20, 2013 at 7:02 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Tue, 20 Aug 2013, Alexander Graf wrote:
> >
> >
> >> > And if your bootloader produces a bad DTB and you cannot let end users
> >> > upgrade the bootloader then this is certainly against best practice.
> >>
> >> But that's exactly how everyone else works. On x86 you get your ACPI
> >> tables from firmware, on PPC you get your device tree from firmware
> >> and you really don't update your firmware just because your OS thinks
> >> the tables are wrong.
> >
> > And everyone, except maybe yourself, despise that state of affair.
> >
> > Sorry but we ought to push for something better when we can.  As device
> > tree is being established on ARM, now is the time to learn from past
> > mistakes and break away from the broken legacy methods.
> 
> Well, provide support for real OpenFirmware or firmware-provided DTB,
> and allow people to append DTBs to their kernels if they need to
> update them.

Forget about appending DTBs to the kernel.  That's only a temporary 
hack which would add more maintenance work into distro people's plate if 
that was to be promoted as a recommended practice.

> The better state of affairs is that vendors who wish to support Linux
> on the systems they ship would allow the firmware to be upgraded and
> collaborate in the process.

That's what I've been saying all along.  Either the firmware is fixable 
by anyone (meaning source code is available) and upgradeable by end 
users, or the firmware allows for easy update by end users of the DTB it 
passes to the kernel if the OEM is too nervous about firmware upgrades 
or unwilling to publish the source for it.

> If they do not wish this or wish to cooperate with this procedure, you 
> can vote with your wallet; don't buy their systems.

This thread is about providing them with good guidelines they can follow 
_before_ they make bad decisions, most of the time out of ignorance 
rather than malice, about their firmware implementation.

> In the case of a real OpenFirmware implementation it is ridiculously
> easy to either patch the device tree in the firmware (load a Forth
> script - I can point at two examples for this) or do it in the kernel
> via CIF.
> 
> In the case of other bootloaders like U-Boot, libfdt provides the
> ability to modify and insert loaded device trees to match (in which
> case, the DTB bundled in the flash of some board is just a starting
> point).
> 
> The worst case scenario is that it would be completely hosed, and in
> this case an appended (or overridden by loading from filesystem or
> other flash location) DTB is the solution - just like you can load
> ACPI tables for some systems where the vendor shipped version is
> totally broken.

Sure.  But that's not what "best practices for hardware shipping device 
trees" should be about.  Working around a broken one is always possible, 
but if we pass the message to OEMs that we're happy to paper over their 
mistakes with a side channel they might decide not to care about 
providing a method allowing the unavoidable mistakes to be fixed at its 
source.

We want the DTB to live with the firmware or bootloader, and the ability 
to fix a broken DTB at that level.  That's all.


Nicolas