Device Tree Evolution Project - call notes - 29th January

Device Tree Evolution Project - call notes - 29th January

[Steve McIntyre]

Hi folks!

I'm sharing the notes from our regular meeting that was held
yesterday. We had updates around several of our initiatives,
particularly around System DT and runtime identification of DTB.

More details about the meeting etc. at the end.

Attendees
=========

SteveM - Arm
GrantL - Arm
MarkB - Arm
RobH - Arm
AlexandreT - ST
BenjaminG - ST
EricF - ST
LoïcP - ST
BillM - TI
FrankR - ??
BruceA - Xilinx
StefanoS - Xilinx
MathieuP - Linaro
BillF - Linaro
KumarG - Linaro
VincentG - Linaro
TrilokS - Qualcomm

Notes
=====

1. DTE-18 - DTB runtime ID
   a. Alexandre posted patches, went through review
   b. v2 coming soon
      1. Don't modify DTC directly, but tweak how things are built
         slightly
      2. ETA? maybe 3w

2. DTE-19 - working out what's changed since the DTB build
   a. Time to talk on the list to see how to do this best
      1. Checksum per node, one of 2 ways:
         a. Modify the sources to include the checksum?
         b. Update the DTB format to add support directly?
      2. Discuss on-list

3. DTE-17: Arnd's prototype work for external DT repo
   a. Status update? No Arnd this week, so not this time
   b. Not everybody will be at FOSDEM or the Arm kernel summit in
      Cambridge, so limited scope for discussion about this
   c. SM will try to organise something for Cambridge next week

4. DTE-2: System DT
   a. Adding spec for new bindings is ok, but need to add more
      explanation of how things should be used (e.g. how to map
      interrupts to multiple domains, multiple controllers)
   b. Stefano promises more docs soon to describe things
   c. ETA for having a working demo?
      1. Will change again before it heads upstream
      2. Things work ok for a demo already! :-)
      3. Just needs more docs to describe

5. DTE-8: DTB lifecycle
   a. Trying to get LEDGE group working on this, working on planning
      meetings for next week
   b. Interested parties - please say so and we'll try and include you
      too
   c. Specifically thinking about bootloader-applied overlays, not
      dynamic stuff applied by kernel at runtime
   d. TI really looking for a collaboration space here
   e. ST interested in U-Boot starting with one DT, then applying an
      overlay before passing it to kernel
      1. Source-level problem here - whey do we have two different
         (almost-complete) DTs in both U-Boot and Linux?
      2. DTE-17 could help here

6. Meetings at BUD20 Connect?
   a. Let Steve know about availability and what you'd like to talk
      about
   b. Sessions in the BUD20 schedule:
      1. Thu 11:00: System DT
      2. Mon 15:30: New approach to dynamically manage hardware bus
         firewalls

7. Linux on Arm summit in Cambridge next week
   a. Some of our group will be around and may have some informal
      discussion
   b. Not a formal DT sprint!


Background information about DTE
================================

Linaro engineers are working on a range of initiatives in the DT
space, collected together as a project called Device Tree Evolution
(DTE). We hold a discussion call every second Wednesday at 
1700 GMT / 1200 EST / 0900 PST. If you would like to be invited, please
ask me (Steve McIntyre).

This is a summary of the notes from the most recent meeting. I aim to
tidy up and post the meeting notes shortly after each meeting. The raw
notes are published at

https://docs.google.com/document/d/e/2PACX-1vRVDrVFWjIOascqZFCO--T8pIqyFB_MDh9cvgyoqhI6Y0tqaA9TcCcvQhcmxi5IY7CG44JfIrCdAUDL/pub

For more information about DTE, see:

 * https://www.linaro.org/engineering/core/devicetree-evolution/
 * https://www.linaro.org/assets/pdf/Linaro-White-Paper--Device-Tree-Evolution.pdf

Cheers,
-- 
Steve McIntyre                                steve.mcintyre-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs

RE: Device Tree Evolution Project - call notes - 29th January

[Mills, William]

Steve,

Can TI get on the agenda for this week's meeting?

We have a simple approach to make progress on boot loader applied overlays.
It is a small repo with just the overlays.
It builds the dtb's from the upstream kernel repo and the overlays from this repo.
We are going to do this at git.ti.com so we can make progress.
However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
Does it make sense to host something like this in devicetree.org github account?

I think this is a very reasonable way forward and more practical than:
* moving all DT's out of the kernel (or sync'ing two homes etc)
* reworking overlay support to be the "one final good way"
* everyone keeping this stuff in their own vendor trees, each doing something different and generating N solutions

We will push the repo somewhere public before the meeting.

(CC'ing Tom and Tony does not constituent their endorsement, we are starting to talk with them now also)

Bill

-----Original Message-----
From: devicetree-spec-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:devicetree-spec-owner@vger.kernel.org] On Behalf Of Steve McIntyre
Sent: Thursday, January 30, 2020 8:56 AM
To: devicetree-spec-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: dte-all-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
Subject: Device Tree Evolution Project - call notes - 29th January

Hi folks!

I'm sharing the notes from our regular meeting that was held
yesterday. We had updates around several of our initiatives,
particularly around System DT and runtime identification of DTB.

More details about the meeting etc. at the end.

Attendees
=========

SteveM - Arm
GrantL - Arm
MarkB - Arm
RobH - Arm
AlexandreT - ST
BenjaminG - ST
EricF - ST
LoïcP - ST
BillM - TI
FrankR - ??
BruceA - Xilinx
StefanoS - Xilinx
MathieuP - Linaro
BillF - Linaro
KumarG - Linaro
VincentG - Linaro
TrilokS - Qualcomm

Notes
=====

1. DTE-18 - DTB runtime ID
   a. Alexandre posted patches, went through review
   b. v2 coming soon
      1. Don't modify DTC directly, but tweak how things are built
         slightly
      2. ETA? maybe 3w

2. DTE-19 - working out what's changed since the DTB build
   a. Time to talk on the list to see how to do this best
      1. Checksum per node, one of 2 ways:
         a. Modify the sources to include the checksum?
         b. Update the DTB format to add support directly?
      2. Discuss on-list

3. DTE-17: Arnd's prototype work for external DT repo
   a. Status update? No Arnd this week, so not this time
   b. Not everybody will be at FOSDEM or the Arm kernel summit in
      Cambridge, so limited scope for discussion about this
   c. SM will try to organise something for Cambridge next week

4. DTE-2: System DT
   a. Adding spec for new bindings is ok, but need to add more
      explanation of how things should be used (e.g. how to map
      interrupts to multiple domains, multiple controllers)
   b. Stefano promises more docs soon to describe things
   c. ETA for having a working demo?
      1. Will change again before it heads upstream
      2. Things work ok for a demo already! :-)
      3. Just needs more docs to describe

5. DTE-8: DTB lifecycle
   a. Trying to get LEDGE group working on this, working on planning
      meetings for next week
   b. Interested parties - please say so and we'll try and include you
      too
   c. Specifically thinking about bootloader-applied overlays, not
      dynamic stuff applied by kernel at runtime
   d. TI really looking for a collaboration space here
   e. ST interested in U-Boot starting with one DT, then applying an
      overlay before passing it to kernel
      1. Source-level problem here - whey do we have two different
         (almost-complete) DTs in both U-Boot and Linux?
      2. DTE-17 could help here

6. Meetings at BUD20 Connect?
   a. Let Steve know about availability and what you'd like to talk
      about
   b. Sessions in the BUD20 schedule:
      1. Thu 11:00: System DT
      2. Mon 15:30: New approach to dynamically manage hardware bus
         firewalls

7. Linux on Arm summit in Cambridge next week
   a. Some of our group will be around and may have some informal
      discussion
   b. Not a formal DT sprint!


Background information about DTE
================================

Linaro engineers are working on a range of initiatives in the DT
space, collected together as a project called Device Tree Evolution
(DTE). We hold a discussion call every second Wednesday at 
1700 GMT / 1200 EST / 0900 PST. If you would like to be invited, please
ask me (Steve McIntyre).

This is a summary of the notes from the most recent meeting. I aim to
tidy up and post the meeting notes shortly after each meeting. The raw
notes are published at

https://docs.google.com/document/d/e/2PACX-1vRVDrVFWjIOascqZFCO--T8pIqyFB_MDh9cvgyoqhI6Y0tqaA9TcCcvQhcmxi5IY7CG44JfIrCdAUDL/pub

For more information about DTE, see:

 * https://www.linaro.org/engineering/core/devicetree-evolution/
 * https://www.linaro.org/assets/pdf/Linaro-White-Paper--Device-Tree-Evolution.pdf

Cheers,
-- 
Steve McIntyre                                steve.mcintyre-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs

Re: Device Tree Evolution Project - call notes - 29th January

[Grant Likely]

On 10/02/2020 16:58, 'Mills, William' via dte-all wrote:
> Steve,
>
> Can TI get on the agenda for this week's meeting?
>
> We have a simple approach to make progress on boot loader applied overlays.
> It is a small repo with just the overlays.
> It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> We are going to do this at git.ti.com so we can make progress.
> However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> Does it make sense to host something like this in devicetree.org github account?

Possibly. Do you want a test repo on github.com/devicetree prior to the
meeting? What would you call it?

g.

>
> I think this is a very reasonable way forward and more practical than:
> * moving all DT's out of the kernel (or sync'ing two homes etc)
> * reworking overlay support to be the "one final good way"
> * everyone keeping this stuff in their own vendor trees, each doing something different and generating N solutions
>
> We will push the repo somewhere public before the meeting.
>
> (CC'ing Tom and Tony does not constituent their endorsement, we are starting to talk with them now also)
>
> Bill
>
> -----Original Message-----
> From: devicetree-spec-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:devicetree-spec-owner@vger.kernel.org] On Behalf Of Steve McIntyre
> Sent: Thursday, January 30, 2020 8:56 AM
> To: devicetree-spec-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: dte-all-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
> Subject: Device Tree Evolution Project - call notes - 29th January
>
> Hi folks!
>
> I'm sharing the notes from our regular meeting that was held
> yesterday. We had updates around several of our initiatives,
> particularly around System DT and runtime identification of DTB.
>
> More details about the meeting etc. at the end.
>
> Attendees
> =========
>
> SteveM - Arm
> GrantL - Arm
> MarkB - Arm
> RobH - Arm
> AlexandreT - ST
> BenjaminG - ST
> EricF - ST
> LoïcP - ST
> BillM - TI
> FrankR - ??
> BruceA - Xilinx
> StefanoS - Xilinx
> MathieuP - Linaro
> BillF - Linaro
> KumarG - Linaro
> VincentG - Linaro
> TrilokS - Qualcomm
>
> Notes
> =====
>
> 1. DTE-18 - DTB runtime ID
>     a. Alexandre posted patches, went through review
>     b. v2 coming soon
>        1. Don't modify DTC directly, but tweak how things are built
>           slightly
>        2. ETA? maybe 3w
>
> 2. DTE-19 - working out what's changed since the DTB build
>     a. Time to talk on the list to see how to do this best
>        1. Checksum per node, one of 2 ways:
>           a. Modify the sources to include the checksum?
>           b. Update the DTB format to add support directly?
>        2. Discuss on-list
>
> 3. DTE-17: Arnd's prototype work for external DT repo
>     a. Status update? No Arnd this week, so not this time
>     b. Not everybody will be at FOSDEM or the Arm kernel summit in
>        Cambridge, so limited scope for discussion about this
>     c. SM will try to organise something for Cambridge next week
>
> 4. DTE-2: System DT
>     a. Adding spec for new bindings is ok, but need to add more
>        explanation of how things should be used (e.g. how to map
>        interrupts to multiple domains, multiple controllers)
>     b. Stefano promises more docs soon to describe things
>     c. ETA for having a working demo?
>        1. Will change again before it heads upstream
>        2. Things work ok for a demo already! :-)
>        3. Just needs more docs to describe
>
> 5. DTE-8: DTB lifecycle
>     a. Trying to get LEDGE group working on this, working on planning
>        meetings for next week
>     b. Interested parties - please say so and we'll try and include you
>        too
>     c. Specifically thinking about bootloader-applied overlays, not
>        dynamic stuff applied by kernel at runtime
>     d. TI really looking for a collaboration space here
>     e. ST interested in U-Boot starting with one DT, then applying an
>        overlay before passing it to kernel
>        1. Source-level problem here - whey do we have two different
>           (almost-complete) DTs in both U-Boot and Linux?
>        2. DTE-17 could help here
>
> 6. Meetings at BUD20 Connect?
>     a. Let Steve know about availability and what you'd like to talk
>        about
>     b. Sessions in the BUD20 schedule:
>        1. Thu 11:00: System DT
>        2. Mon 15:30: New approach to dynamically manage hardware bus
>           firewalls
>
> 7. Linux on Arm summit in Cambridge next week
>     a. Some of our group will be around and may have some informal
>        discussion
>     b. Not a formal DT sprint!
>
>
> Background information about DTE
> ================================
>
> Linaro engineers are working on a range of initiatives in the DT
> space, collected together as a project called Device Tree Evolution
> (DTE). We hold a discussion call every second Wednesday at
> 1700 GMT / 1200 EST / 0900 PST. If you would like to be invited, please
> ask me (Steve McIntyre).
>
> This is a summary of the notes from the most recent meeting. I aim to
> tidy up and post the meeting notes shortly after each meeting. The raw
> notes are published at
>
> https://docs.google.com/document/d/e/2PACX-1vRVDrVFWjIOascqZFCO--T8pIqyFB_MDh9cvgyoqhI6Y0tqaA9TcCcvQhcmxi5IY7CG44JfIrCdAUDL/pub
>
> For more information about DTE, see:
>
>   * https://www.linaro.org/engineering/core/devicetree-evolution/
>   * https://www.linaro.org/assets/pdf/Linaro-White-Paper--Device-Tree-Evolution.pdf
>
> Cheers,
>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Device Tree Evolution Project - call notes - 29th January

[Steve McIntyre]

On Mon, Feb 10, 2020 at 04:58:26PM +0000, William Mills wrote:
>Steve,
>
>Can TI get on the agenda for this week's meeting?
>
>We have a simple approach to make progress on boot loader applied overlays.
>It is a small repo with just the overlays.
>It builds the dtb's from the upstream kernel repo and the overlays from this repo.
>We are going to do this at git.ti.com so we can make progress.
>However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
>Does it make sense to host something like this in devicetree.org github account?

I think so, yes - I see Grant has already responded there.

>I think this is a very reasonable way forward and more practical than:
>* moving all DT's out of the kernel (or sync'ing two homes etc)
>* reworking overlay support to be the "one final good way"
>* everyone keeping this stuff in their own vendor trees, each doing something different and generating N solutions
>
>We will push the repo somewhere public before the meeting.

Cool. :-)

>(CC'ing Tom and Tony does not constituent their endorsement, we are starting to talk with them now also)

ACK!

Cheers,
-- 
Steve McIntyre                                steve.mcintyre-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs

Re: Device Tree Evolution Project - call notes - 29th January

[Rob Herring]

On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
>
> Steve,
>
> Can TI get on the agenda for this week's meeting?
>
> We have a simple approach to make progress on boot loader applied overlays.
> It is a small repo with just the overlays.
> It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> We are going to do this at git.ti.com so we can make progress.
> However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.

For the record, I'm in favor of hosting overlays in the kernel (of
course I have opinions on the details). Frank is not in favor IIRC,
but Frank is only maintainer of 'drivers/of/' so ultimately not his
decision.

> Does it make sense to host something like this in devicetree.org github account?

Yes, but I would like to see this coordinated with hosting
'devicetree-rebasing' there. We should be able to use the same
makefiles from it. Perhaps the overlay repo should work as a git
submodule of it as well. That would help keep things in sync.

A concern I have is what happens when someone wants to split some
portions of an existing dts into an overlay? Then the base dts becomes
incomplete. Users will have regressions from missing functionality if
they don't know to apply some overlay. And how do we track what base
DTs overlays apply to? Not that we have a solution when they are in
one tree, but 2 trees makes that harder.

Rob

Re: Device Tree Evolution Project - call notes - 29th January

[Tony Lindgren]

* Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> [200211 16:33]:
> On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
> >
> > Steve,
> >
> > Can TI get on the agenda for this week's meeting?
> >
> > We have a simple approach to make progress on boot loader applied overlays.
> > It is a small repo with just the overlays.
> > It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> > We are going to do this at git.ti.com so we can make progress.
> > However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> 
> For the record, I'm in favor of hosting overlays in the kernel (of
> course I have opinions on the details). Frank is not in favor IIRC,
> but Frank is only maintainer of 'drivers/of/' so ultimately not his
> decision.

I agree, having them all together would be best. AFAIK the only
issue for that earlier was people did not want dangling dts files
that could not be checked and compiled. Sounds like that part has
been now taken care of.

> > Does it make sense to host something like this in devicetree.org github account?
> 
> Yes, but I would like to see this coordinated with hosting
> 'devicetree-rebasing' there. We should be able to use the same
> makefiles from it. Perhaps the overlay repo should work as a git
> submodule of it as well. That would help keep things in sync.
> 
> A concern I have is what happens when someone wants to split some
> portions of an existing dts into an overlay? Then the base dts becomes
> incomplete. Users will have regressions from missing functionality if
> they don't know to apply some overlay. And how do we track what base
> DTs overlays apply to? Not that we have a solution when they are in
> one tree, but 2 trees makes that harder.

Yeah good point.

Regards,

Tony

Re: Device Tree Evolution Project - call notes - 29th January

[Francois Ozog]

On Tue, 11 Feb 2020 at 17:32, Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
> >
> > Steve,
> >
> > Can TI get on the agenda for this week's meeting?
> >
> > We have a simple approach to make progress on boot loader applied overlays.
> > It is a small repo with just the overlays.
> > It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> > We are going to do this at git.ti.com so we can make progress.
> > However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
>
> For the record, I'm in favor of hosting overlays in the kernel (of
> course I have opinions on the details). Frank is not in favor IIRC,
> but Frank is only maintainer of 'drivers/of/' so ultimately not his
> decision.
>
> > Does it make sense to host something like this in devicetree.org github account?
>
> Yes, but I would like to see this coordinated with hosting
> 'devicetree-rebasing' there. We should be able to use the same
> makefiles from it. Perhaps the overlay repo should work as a git
> submodule of it as well. That would help keep things in sync.
>
> A concern I have is what happens when someone wants to split some
> portions of an existing dts into an overlay? Then the base dts becomes
> incomplete. Users will have regressions from missing functionality if
> they don't know to apply some overlay. And how do we track what base
> DTs overlays apply to? Not that we have a solution when they are in
> one tree, but 2 trees makes that harder.

DT lifecycle shall be selectable by the board vendor. If, for any reason
(and there may be plenty), a vendor believes it is a better organization to
handle the DT assembly (static + overlays) outside the kernel, then it
should be possible to do so.
Note: On the ACPI front, a form of overlays (needed for DT too) will
be handled by U-Boot:
https://lists.denx.de/pipermail/u-boot/2020-January/397886.html

>
> Rob
>
>

Re: Device Tree Evolution Project - call notes - 29th January

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

On Tue, Feb 11, 2020 at 06:44:31PM +0100, Francois Ozog wrote:
> On Tue, 11 Feb 2020 at 17:32, Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> >
> > On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
> > >
> > > Steve,
> > >
> > > Can TI get on the agenda for this week's meeting?
> > >
> > > We have a simple approach to make progress on boot loader applied overlays.
> > > It is a small repo with just the overlays.
> > > It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> > > We are going to do this at git.ti.com so we can make progress.
> > > However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> >
> > For the record, I'm in favor of hosting overlays in the kernel (of
> > course I have opinions on the details). Frank is not in favor IIRC,
> > but Frank is only maintainer of 'drivers/of/' so ultimately not his
> > decision.
> >
> > > Does it make sense to host something like this in devicetree.org github account?
> >
> > Yes, but I would like to see this coordinated with hosting
> > 'devicetree-rebasing' there. We should be able to use the same
> > makefiles from it. Perhaps the overlay repo should work as a git
> > submodule of it as well. That would help keep things in sync.
> >
> > A concern I have is what happens when someone wants to split some
> > portions of an existing dts into an overlay? Then the base dts becomes
> > incomplete. Users will have regressions from missing functionality if
> > they don't know to apply some overlay. And how do we track what base
> > DTs overlays apply to? Not that we have a solution when they are in
> > one tree, but 2 trees makes that harder.
> 
> DT lifecycle shall be selectable by the board vendor. If, for any reason
> (and there may be plenty), a vendor believes it is a better organization to
> handle the DT assembly (static + overlays) outside the kernel, then it
> should be possible to do so.
> Note: On the ACPI front, a form of overlays (needed for DT too) will
> be handled by U-Boot:
> https://lists.denx.de/pipermail/u-boot/2020-January/397886.html

I'll be on the next call as well, but I do want to caution against
reading too much in to what we're doing to deal with x86 / ACPI and its
different set of challenges.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Device Tree Evolution Project - call notes - 29th January

[Rob Herring]

On Tue, Feb 11, 2020 at 11:44 AM Francois Ozog <francois.ozog-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>
> On Tue, 11 Feb 2020 at 17:32, Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> >
> > On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
> > >
> > > Steve,
> > >
> > > Can TI get on the agenda for this week's meeting?
> > >
> > > We have a simple approach to make progress on boot loader applied overlays.
> > > It is a small repo with just the overlays.
> > > It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> > > We are going to do this at git.ti.com so we can make progress.
> > > However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> >
> > For the record, I'm in favor of hosting overlays in the kernel (of
> > course I have opinions on the details). Frank is not in favor IIRC,
> > but Frank is only maintainer of 'drivers/of/' so ultimately not his
> > decision.
> >
> > > Does it make sense to host something like this in devicetree.org github account?
> >
> > Yes, but I would like to see this coordinated with hosting
> > 'devicetree-rebasing' there. We should be able to use the same
> > makefiles from it. Perhaps the overlay repo should work as a git
> > submodule of it as well. That would help keep things in sync.
> >
> > A concern I have is what happens when someone wants to split some
> > portions of an existing dts into an overlay? Then the base dts becomes
> > incomplete. Users will have regressions from missing functionality if
> > they don't know to apply some overlay. And how do we track what base
> > DTs overlays apply to? Not that we have a solution when they are in
> > one tree, but 2 trees makes that harder.
>
> DT lifecycle shall be selectable by the board vendor. If, for any reason
> (and there may be plenty), a vendor believes it is a better organization to
> handle the DT assembly (static + overlays) outside the kernel, then it
> should be possible to do so.

Certainly, I don't think I said anything that would prevent that. In
that case, I would assume the board is not already upstream and
there's not any backwards compatibility (ABI) to worry about. I'm only
talking about cases where the split is changed such that we go from a
monolithic dtb to a base dtb plus 1 or more overlay dtbs. IOW, we
can't just change the split because that is an ABI. We still need to
build and provide that original single dtb. There may be exceptions to
that, but by default I have to care about backwards compatibility (and
overlays add a new dimension to that). I suppose we could decide that
a collection of base and overlay DTs are a monolithic unit and there
is no ABI between them. However, we'd need to define some way to
distinguish both cases. It's plausible that you'd want to do both on
the same platform (base and overlay dtbs from the board vendor and 3rd
party overlays).

Besides the above reason, there is something much more basic. We need
build time assembly of base plus overlays just to validate overlays
apply successfully. We don't want the only way to check an overlay
applies is by booting the specific board that works with a given base
and overlay dtb. Whether overlays apply or not is just the first step
in validation. There's all the schema validation to do which is going
to have its own issues.

Rob

Re: Device Tree Evolution Project - call notes - 29th January

[William Mills]

On 2/11/20 4:53 PM, Rob Herring wrote:
> On Tue, Feb 11, 2020 at 11:44 AM Francois Ozog <francois.ozog-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>>
>> On Tue, 11 Feb 2020 at 17:32, Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>>>
>>> On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
>>>>
>>>> Steve,
>>>>
>>>> Can TI get on the agenda for this week's meeting?
>>>>
>>>> We have a simple approach to make progress on boot loader applied overlays.
>>>> It is a small repo with just the overlays.
>>>> It builds the dtb's from the upstream kernel repo and the overlays from this repo.
>>>> We are going to do this at git.ti.com so we can make progress.
>>>> However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
>>>
>>> For the record, I'm in favor of hosting overlays in the kernel (of
>>> course I have opinions on the details). Frank is not in favor IIRC,
>>> but Frank is only maintainer of 'drivers/of/' so ultimately not his
>>> decision.
>>>
>>>> Does it make sense to host something like this in devicetree.org github account?
>>>
>>> Yes, but I would like to see this coordinated with hosting
>>> 'devicetree-rebasing' there. We should be able to use the same
>>> makefiles from it. Perhaps the overlay repo should work as a git
>>> submodule of it as well. That would help keep things in sync.
>>>

ACK on that.  Lots of good points raised in this thread. (I guess I
should not have recycled the subject line :)

I promised to publish so this is what we have so far:
https://github.com/wmamills/dt-overlays

Bill

>>> A concern I have is what happens when someone wants to split some
>>> portions of an existing dts into an overlay? Then the base dts becomes
>>> incomplete. Users will have regressions from missing functionality if
>>> they don't know to apply some overlay. And how do we track what base
>>> DTs overlays apply to? Not that we have a solution when they are in
>>> one tree, but 2 trees makes that harder.
>>
>> DT lifecycle shall be selectable by the board vendor. If, for any reason
>> (and there may be plenty), a vendor believes it is a better organization to
>> handle the DT assembly (static + overlays) outside the kernel, then it
>> should be possible to do so.
> 
> Certainly, I don't think I said anything that would prevent that. In
> that case, I would assume the board is not already upstream and
> there's not any backwards compatibility (ABI) to worry about. I'm only
> talking about cases where the split is changed such that we go from a
> monolithic dtb to a base dtb plus 1 or more overlay dtbs. IOW, we
> can't just change the split because that is an ABI. We still need to
> build and provide that original single dtb. There may be exceptions to
> that, but by default I have to care about backwards compatibility (and
> overlays add a new dimension to that). I suppose we could decide that
> a collection of base and overlay DTs are a monolithic unit and there
> is no ABI between them. However, we'd need to define some way to
> distinguish both cases. It's plausible that you'd want to do both on
> the same platform (base and overlay dtbs from the board vendor and 3rd
> party overlays).
> 
> Besides the above reason, there is something much more basic. We need
> build time assembly of base plus overlays just to validate overlays
> apply successfully. We don't want the only way to check an overlay
> applies is by booting the specific board that works with a given base
> and overlay dtb. Whether overlays apply or not is just the first step
> in validation. There's all the schema validation to do which is going
> to have its own issues.
> 
> Rob
> 

Re: Device Tree Evolution Project - call notes - 29th January

[Frank Rowand]

On 2/11/20 10:31 AM, Rob Herring wrote:
> On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
>>
>> Steve,
>>
>> Can TI get on the agenda for this week's meeting?
>>
>> We have a simple approach to make progress on boot loader applied overlays.
>> It is a small repo with just the overlays.
>> It builds the dtb's from the upstream kernel repo and the overlays from this repo.
>> We are going to do this at git.ti.com so we can make progress.
>> However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> 
> For the record, I'm in favor of hosting overlays in the kernel (of
> course I have opinions on the details). Frank is not in favor IIRC,
> but Frank is only maintainer of 'drivers/of/' so ultimately not his
> decision.

I agree that the kernel is the place to host overlay sources.

-Frank

> 
>> Does it make sense to host something like this in devicetree.org github account?
> 
> Yes, but I would like to see this coordinated with hosting
> 'devicetree-rebasing' there. We should be able to use the same
> makefiles from it. Perhaps the overlay repo should work as a git
> submodule of it as well. That would help keep things in sync.
> 
> A concern I have is what happens when someone wants to split some
> portions of an existing dts into an overlay? Then the base dts becomes
> incomplete. Users will have regressions from missing functionality if
> they don't know to apply some overlay. And how do we track what base
> DTs overlays apply to? Not that we have a solution when they are in
> one tree, but 2 trees makes that harder.
> 
> Rob
> 

Re: Device Tree Evolution Project - call notes - 29th January

[Francois Ozog]

On Wed, 12 Feb 2020 at 03:49, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> On 2/11/20 10:31 AM, Rob Herring wrote:
> > On Mon, Feb 10, 2020 at 10:58 AM Mills, William <wmills-l0cyMroinI0@public.gmane.org> wrote:
> >>
> >> Steve,
> >>
> >> Can TI get on the agenda for this week's meeting?
> >>
> >> We have a simple approach to make progress on boot loader applied overlays.
> >> It is a small repo with just the overlays.
> >> It builds the dtb's from the upstream kernel repo and the overlays from this repo.
> >> We are going to do this at git.ti.com so we can make progress.
> >> However we would like to suggest this as a simple way forward to collectively host things the kernel maintainers do not want in the kernel.
> >
> > For the record, I'm in favor of hosting overlays in the kernel (of
> > course I have opinions on the details). Frank is not in favor IIRC,
> > but Frank is only maintainer of 'drivers/of/' so ultimately not his
> > decision.
>
> I agree that the kernel is the place to host overlay sources.
>

I would like to fully understand the *why* overlays before diving in the *how*.
Even if addressed in previous calls or meeting minutes: it may be good
to restate the use cases in the mail thread.
Here are the ones I have been exposed to:

1) static hypervisor partitioning
Some devices of the board may be under the control of an RTOS or even
a bare metal app in a partition while most devices are under a Linux
partition
Hypervisor could be Xen, VMWare ESXi, Hafnium, Jailhouse.

2) driver isolation from main OS
A kernel may run "driver-less" and have devices handled in isolated
domains or VMs.
Reasons for that many not be just about security but abstraction for instance.

3) conditional parameters or device access
Presence or Absence of OPTEE memory reservation is not an operating
system selection.
We may want to boot the same image regardless if OPTEE is activated or not.
If we want to have embedded systems as easy as servers, we do not want
to have a fixed assumption by
distros providers. Boot loader can properly "decorate" the DT with
memory reservations if OPTEE present.
In addition, OPTEE could vouch for certain device access. It can give
actual device access to kernel if validated.
The BAR(s) may be actually be controlled by OPTEE.
(2) and (3) may be combined

4) Lego-style hardware
hardware may come in pieces that are assembled at different moment of
the product lifecycle:
- dev board extensions with capes
- composable products such as Insta360 configurable camara (or defunct
Google project ARA)
The running platform DT can only be a composition DTs either at runtime.

5) other cases?

Let's not talk about (4) for the moment. Are there other cases?

(1), (2) and (3) is all about correct chopping of a platform DT into
meaningful and self-contained pieces.
It is mandatory that, when fully assembled, it is a coherent and
compliant entity that the kernel can use.
Linux kernel should assume all devices for itself and hence all dtso
should be present in the kernel for this "end-to-end" case.

Now the origin of the pieces may be "upstream", meaning the Linux
kernel would pull all pieces and assemble all of them at normal
compilation time.
Other components (bootloader, TEE, hypervisor, RTOS) can pull the
dtsos and assemble them in different ways.
The good thing is that slices are known a-priori. An OS driver making
use of a DT element not in a single dtso may end-up having issues if
used in (1) for instance.

Those other components could "refer" to Linux kernel source for dtso
origin. But it looks strange as the DT is describing hardware, not a
Linux component...

To conclude:
- dtso are in a single multi-vendor repo
- Linux pull them all and assemble them all so that it validates the
end-to-end schema. drivers shall only deal with single dtso.
- other OSes, bootloaders, hypervisors, TEE pull them and assemble
them as they please
- Linux receives the actual DT from whatever component loads it and
know that the DT it deals with is made of the dtso it already knows
and for which the drivers have been verified


> -Frank
>
> >
> >> Does it make sense to host something like this in devicetree.org github account?
> >
> > Yes, but I would like to see this coordinated with hosting
> > 'devicetree-rebasing' there. We should be able to use the same
> > makefiles from it. Perhaps the overlay repo should work as a git
> > submodule of it as well. That would help keep things in sync.
> >
> > A concern I have is what happens when someone wants to split some
> > portions of an existing dts into an overlay? Then the base dts becomes
> > incomplete. Users will have regressions from missing functionality if
> > they don't know to apply some overlay. And how do we track what base
> > DTs overlays apply to? Not that we have a solution when they are in
> > one tree, but 2 trees makes that harder.
> >
> > Rob
> >
>
>