devicetree repository separation/migration

devicetree repository separation/migration

[Jason Cooper]

All,

At last weeks devicetree irc meeting, I took on the task of writing this
email.  I'm a bit behind.

One of the outcomes of the ARM/devicetree discussion at the 2013 Kernel
Summit was that we were going to hold off on separating the devicetree
from the Linux kernel tree.  The primary reason for this was to get
through the backlog of patches.

It's been several months, and we're seeing evidence of other projects
having difficulty keeping in sync with the kernel tree.  Specifically,
barebox is having trouble syncing:

  http://list-archives.org/2014/02/07/barebox-lists-infradead-org/devicetree-maintenance-in-barebox/f/5820726136

U-boot has some questionable (admittedly from early on) nodes.  I'll
refrain from singling out a board file or commit.  Interested
individuals can look through the irc archives for #devicetree.  At any
rate, reportedly, Wolfgang confirmed at the U-Boot minisummit that the
dts files in U-Boot would match the ones from the kernel.  So U-Boot is
in the same situation as barebox, wrt syncing.

BSD is also adopting devicetree, but I'm not personally familiar with
the status of that.  Nor whether it's diverged from what we have.

On the Linux side, it appears to me that we have successfully removed
the bottleneck, and patches seem to be flowing fairly well now.

For the past few months, Ian has been maintaining a devicetree repo at

  http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git;a=summary


Now for the questions:

  - Is the Linux development workflow ready for devicetree to move out
    of the Linux Kernel?

  - Would having the devicetree in it's own repo, with it's own workflow
    and release schedule help the other users of it?

  - Where should it be hosted?

  - How do we envision projects will use it?  git submodule?  reference
    a version tag?  (this is primarily targeted at bootloaders that need
    to compile in a dtb or subset of a dtb into the bootloader)

Other thoughts I may have missed?

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> Interested individuals can look through the irc archives for
> #devicetree.

oops, My mistake.  They are supposed to be at irclogs.linaro.org, but it
hasn't been set up yet.  You'll just have to ask nicely for someone to
go through their scrollback buffer. :)

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> All,
> 
> At last weeks devicetree irc meeting, I took on the task of writing this
> email.  I'm a bit behind.
> 
> One of the outcomes of the ARM/devicetree discussion at the 2013 Kernel
> Summit was that we were going to hold off on separating the devicetree
> from the Linux kernel tree.  The primary reason for this was to get
> through the backlog of patches.
> 
> It's been several months, and we're seeing evidence of other projects
> having difficulty keeping in sync with the kernel tree.  Specifically,
> barebox is having trouble syncing:
> 
>   http://list-archives.org/2014/02/07/barebox-lists-infradead-org/devicetree-maintenance-in-barebox/f/5820726136
> 
> U-boot has some questionable (admittedly from early on) nodes.  I'll
> refrain from singling out a board file or commit.  Interested
> individuals can look through the irc archives for #devicetree.  At any
> rate, reportedly, Wolfgang confirmed at the U-Boot minisummit that the
> dts files in U-Boot would match the ones from the kernel.  So U-Boot is
> in the same situation as barebox, wrt syncing.
> 
> BSD is also adopting devicetree, but I'm not personally familiar with
> the status of that.  Nor whether it's diverged from what we have.
> 
> On the Linux side, it appears to me that we have successfully removed
> the bottleneck, and patches seem to be flowing fairly well now.
> 
> For the past few months, Ian has been maintaining a devicetree repo at
> 
>   http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git;a=summary
> 
> 
> Now for the questions:
> 
>   - Is the Linux development workflow ready for devicetree to move out
>     of the Linux Kernel?

I hope so since keeping the devicetrees in sync with the kernel is a
pain for all external users.

> 
>   - Would having the devicetree in it's own repo, with it's own workflow
>     and release schedule help the other users of it?
> 
>   - Where should it be hosted?
> 
>   - How do we envision projects will use it?  git submodule?  reference
>     a version tag?  (this is primarily targeted at bootloaders that need
>     to compile in a dtb or subset of a dtb into the bootloader)

I would prefer to use it as a submodule. I'll likely need some barebox
specific additions to the devicetrees. Right now our idea is to leave
the provided devicetrees untouched and instead of compiling the board
dts files directly we create <boardname>-barebox.dts files which include
the original board files. That would allow us to provide additional
information to barebox without having to carry patches for the
devicetrees.

> 
> Other thoughts I may have missed?

It will be interesting to see which rules should apply for merging new
bindings. I know that devicetrees should be OS agnostic, but sometimes
they are modelled after how Linux currently works. What happens when the
*BSD guys have different ideas how a good binding looks like? How will
such conflicts be resolved?

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
...
> >   - Is the Linux development workflow ready for devicetree to move out
> >     of the Linux Kernel?
> 
> I hope so since keeping the devicetrees in sync with the kernel is a
> pain for all external users.

Well, I haven't heard any screams yet.  I suspect people are waiting for
details on the exact form it would take before complaining...

> >   - How do we envision projects will use it?  git submodule?  reference
> >     a version tag?  (this is primarily targeted at bootloaders that need
> >     to compile in a dtb or subset of a dtb into the bootloader)
> 
> I would prefer to use it as a submodule.

ok.  I've often thought that was the right solution for several things
(dtc.git inside the kernel tree), but no one ever seemed to speak of it
or bring it up.  Kinda like leprosy.

It does add an extra step to build process for new users.  Although that
could be handled in the Makefile.

> I'll likely need some barebox specific additions to the devicetrees.
> Right now our idea is to leave the provided devicetrees untouched and
> instead of compiling the board dts files directly we create
> <boardname>-barebox.dts files which include the original board files.
> That would allow us to provide additional information to barebox
> without having to carry patches for the devicetrees.

So the resulting <boardname>-barebox.dtb is compiled into the barebox
binary?  Is the dtb passed to the kernel independently upgradeable?

Why not post binding/dts patches for 'barebox,...' attributes that you
need?

> > Other thoughts I may have missed?
> 
> It will be interesting to see which rules should apply for merging new
> bindings. I know that devicetrees should be OS agnostic, but sometimes
> they are modelled after how Linux currently works. What happens when the
> *BSD guys have different ideas how a good binding looks like? How will
> such conflicts be resolved?

That's more a question for Grant.  I assume we'll all put on our big-boy
pants and pick the best technical solution based on their merits. :)

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Stephen Warren]

On 02/18/2014 11:18 AM, Jason Cooper wrote:
> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> ...
>>>   - Is the Linux development workflow ready for devicetree to move out
>>>     of the Linux Kernel?
>>
>> I hope so since keeping the devicetrees in sync with the kernel is a
>> pain for all external users.
> 
> Well, I haven't heard any screams yet.  I suspect people are waiting for
> details on the exact form it would take before complaining...
> 
>>>   - How do we envision projects will use it?  git submodule?  reference
>>>     a version tag?  (this is primarily targeted at bootloaders that need
>>>     to compile in a dtb or subset of a dtb into the bootloader)
>>
>> I would prefer to use it as a submodule.
> 
> ok.  I've often thought that was the right solution for several things
> (dtc.git inside the kernel tree), but no one ever seemed to speak of it
> or bring it up.  Kinda like leprosy.
> 
> It does add an extra step to build process for new users.  Although that
> could be handled in the Makefile.

My limited experience of git submodules implies that comparing them to
Leprosy isn't a bad comparison:-)

If they are separated out, I'd vastly prefer they simply be a standalone
project completed divorced from the kernel. Playing games with git
submodules to try and make it easier seems more likely to actually make
this more complicated.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Tue, Feb 18, 2014 at 12:09:16PM -0700, Stephen Warren wrote:
> On 02/18/2014 11:18 AM, Jason Cooper wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> >>>   - Is the Linux development workflow ready for devicetree to move out
> >>>     of the Linux Kernel?
> >>
> >> I hope so since keeping the devicetrees in sync with the kernel is a
> >> pain for all external users.
> > 
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> > 
> >>>   - How do we envision projects will use it?  git submodule?  reference
> >>>     a version tag?  (this is primarily targeted at bootloaders that need
> >>>     to compile in a dtb or subset of a dtb into the bootloader)
> >>
> >> I would prefer to use it as a submodule.
> > 
> > ok.  I've often thought that was the right solution for several things
> > (dtc.git inside the kernel tree), but no one ever seemed to speak of it
> > or bring it up.  Kinda like leprosy.
> > 
> > It does add an extra step to build process for new users.  Although that
> > could be handled in the Makefile.
> 
> My limited experience of git submodules implies that comparing them to
> Leprosy isn't a bad comparison:-)
> 
> If they are separated out, I'd vastly prefer they simply be a standalone
> project completed divorced from the kernel.

Yes, I probably wasn't clear.  The devicetree repo _would_ be it's own
repo divorced from all other projects, including the kernel.

How projects wish to integrate the tree in order to produce the dtbs is
the question at hand.  The outcome of the discussion may have an effect
on how we structure the tree.  eg Makefile's and such.

> Playing games with git submodules to try and make it easier seems more
> likely to actually make this more complicated.

I've played around with them a bit, and I think they are most similar to
round-abouts.  It looks like chaos.  But as long as people follow a few
simple rules, it works out fine.

>From the devicetree pov, though, it would just be maintaining the
makefile(s) to be amenable to that scenario.

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Tue, Feb 18, 2014 at 12:09:16PM -0700, Stephen Warren wrote:
> On 02/18/2014 11:18 AM, Jason Cooper wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> >>>   - Is the Linux development workflow ready for devicetree to move out
> >>>     of the Linux Kernel?
> >>
> >> I hope so since keeping the devicetrees in sync with the kernel is a
> >> pain for all external users.
> > 
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> > 
> >>>   - How do we envision projects will use it?  git submodule?  reference
> >>>     a version tag?  (this is primarily targeted at bootloaders that need
> >>>     to compile in a dtb or subset of a dtb into the bootloader)
> >>
> >> I would prefer to use it as a submodule.
> > 
> > ok.  I've often thought that was the right solution for several things
> > (dtc.git inside the kernel tree), but no one ever seemed to speak of it
> > or bring it up.  Kinda like leprosy.
> > 
> > It does add an extra step to build process for new users.  Although that
> > could be handled in the Makefile.
> 
> My limited experience of git submodules implies that comparing them to
> Leprosy isn't a bad comparison:-)
> 
> If they are separated out, I'd vastly prefer they simply be a standalone
> project completed divorced from the kernel.

Yes, I probably wasn't clear.  The devicetree repo _would_ be it's own
repo divorced from all other projects, including the kernel.

How projects wish to integrate the tree in order to produce the dtbs is
the question at hand.  The outcome of the discussion may have an effect
on how we structure the tree.  eg Makefile's and such.

> Playing games with git submodules to try and make it easier seems more
> likely to actually make this more complicated.

I've played around with them a bit, and I think they are most similar to
round-abouts.  It looks like chaos.  But as long as people follow a few
simple rules, it works out fine.

From the devicetree pov, though, it would just be maintaining the
makefile(s) to be amenable to that scenario.

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Olof Johansson]

On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> ...
>> >   - Is the Linux development workflow ready for devicetree to move out
>> >     of the Linux Kernel?
>>
>> I hope so since keeping the devicetrees in sync with the kernel is a
>> pain for all external users.
>
> Well, I haven't heard any screams yet.  I suspect people are waiting for
> details on the exact form it would take before complaining...

I'M SCREAMING NOW. :-)

Honestly though, I think we need to do this carefully. Even though we
don't like it, there are still lots of bindings in flux and
cross-dependencies between two independent repos will be a major pain.

I think we have two options:

1. Bring out everything in the current kernel repo to a separate one,
but do it my mirroring over. Changes go into the kernel repo first and
then comes over to this one, but other projects can mirror the
standalone repo without downloading a whole kernel tree.

2. Remove the kernel contents and move it over to the new repo. This
should be done independently for each platform, and the maintainers
get to decide if, when and how they do it. Some platforms are ready
for it (some have been for a long time), others are not. And it's up
to the maintainer, since they are the ones we will yell at when they
make our life miserable by adding cross-dependencies with an external
repo. Breakage due to the move is something we should have to put up
with, etc.

>> I'll likely need some barebox specific additions to the devicetrees.
>> Right now our idea is to leave the provided devicetrees untouched and
>> instead of compiling the board dts files directly we create
>> <boardname>-barebox.dts files which include the original board files.
>> That would allow us to provide additional information to barebox
>> without having to carry patches for the devicetrees.
>
> So the resulting <boardname>-barebox.dtb is compiled into the barebox
> binary?  Is the dtb passed to the kernel independently upgradeable?
>
> Why not post binding/dts patches for 'barebox,...' attributes that you
> need?

+1. These should ideally be posted and reviewed too -- maybe they make
sense to share between barebox and other firmware stacks, for example.

>
>> > Other thoughts I may have missed?
>>
>> It will be interesting to see which rules should apply for merging new
>> bindings. I know that devicetrees should be OS agnostic, but sometimes
>> they are modelled after how Linux currently works. What happens when the
>> *BSD guys have different ideas how a good binding looks like? How will
>> such conflicts be resolved?
>
> That's more a question for Grant.  I assume we'll all put on our big-boy
> pants and pick the best technical solution based on their merits. :)

A good binding focuses on describing hardware, so as long as we keep
our focus on that instead of how a specific implementation of a driver
will use it, I think we can avoid at least some of the potential
conflicts.


-Olof
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/18/2014 10:18 AM, Jason Cooper wrote:
> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> ...
>>>   - Is the Linux development workflow ready for devicetree to move out
>>>     of the Linux Kernel?
>>
>> I hope so since keeping the devicetrees in sync with the kernel is a
>> pain for all external users.
> 
> Well, I haven't heard any screams yet.  I suspect people are waiting for
> details on the exact form it would take before complaining...

< snip >

Let me join the chorus of screams.

I would venture that the number of linux kernel developers actively
touching device tree source files (and impacted by changes to them)
is vastly larger than any other set of developers.

My experience in the kernel is already that finding working device
tree fragments that match current under development code is difficult.
(I am not trying to generalize for all systems, just the ones I use.)

Moving the device tree source files out of the kernel git tree will
only make things worse.

-Frank

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Olof Johansson]

On Tue, Feb 18, 2014 at 11:47 AM, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
> Breakage due to the move is something we should have to put up
> with, etc.

Typo. We absolutely must not have breakage due to this move.


-Olof
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Tim Bird]

I'm not in favor of separating the device tree information from the kernel.

If we switch, then whatever synchronization issues other projects
are having now with synching with the device tree info from the kernel will
just then become the problem of the kernel developers, who will then
have to sync with the device tree info from another repository.  If the
sync issues can't be solved now for them, why or how would it be solved
post-separation for us?  (It sounds like a zero-sum game of pain transfer
to me.)

I'm relatively unfamiliar with the arguments.  Can someone provide
a brief list of reasons this is needed, and how the inconvenience to Linux
kernel developers will be minimized, should it proceed?

Thanks,
 -- Tim


On Tue, Feb 18, 2014 at 2:21 PM, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
> On Tue, Feb 18, 2014 at 11:47 AM, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
>> Breakage due to the move is something we should have to put up
>> with, etc.
>
> Typo. We absolutely must not have breakage due to this move.
>
>
> -Olof
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
 -- Tim Bird
Senior Software Engineer, Sony Mobile
Architecture Group Chair, CE Workgroup, Linux Foundation
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Tue, Feb 18, 2014 at 01:18:54PM -0500, Jason Cooper wrote:
> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> > On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> ...
> > >   - Is the Linux development workflow ready for devicetree to move out
> > >     of the Linux Kernel?
> > 
> > I hope so since keeping the devicetrees in sync with the kernel is a
> > pain for all external users.
> 
> Well, I haven't heard any screams yet.  I suspect people are waiting for
> details on the exact form it would take before complaining...
> 
> > >   - How do we envision projects will use it?  git submodule?  reference
> > >     a version tag?  (this is primarily targeted at bootloaders that need
> > >     to compile in a dtb or subset of a dtb into the bootloader)
> > 
> > I would prefer to use it as a submodule.
> 
> ok.  I've often thought that was the right solution for several things
> (dtc.git inside the kernel tree), but no one ever seemed to speak of it
> or bring it up.  Kinda like leprosy.
> 
> It does add an extra step to build process for new users.  Although that
> could be handled in the Makefile.
> 
> > I'll likely need some barebox specific additions to the devicetrees.
> > Right now our idea is to leave the provided devicetrees untouched and
> > instead of compiling the board dts files directly we create
> > <boardname>-barebox.dts files which include the original board files.
> > That would allow us to provide additional information to barebox
> > without having to carry patches for the devicetrees.
> 
> So the resulting <boardname>-barebox.dtb is compiled into the barebox
> binary?  Is the dtb passed to the kernel independently upgradeable?

Yes, it's compiled into barebox. barebox uses it for probing from
devicetree. You can pass this devicetree to the kernel or provide
another one should you want or have to.

> 
> Why not post binding/dts patches for 'barebox,...' attributes that you
> need?

I wasn't clear. The bindings themselves should be posted and merged
mainline. mtd partitions are a good example for what I mean. I don't
want to see them in the mainline dts files because that would mean my
partitioning would change with each mainline change. Nevertheless I have
the partitions in the barebox dts files.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
> I'm not in favor of separating the device tree information from the kernel.
> 
> If we switch, then whatever synchronization issues other projects
> are having now with synching with the device tree info from the kernel will
> just then become the problem of the kernel developers, who will then
> have to sync with the device tree info from another repository.  If the
> sync issues can't be solved now for them, why or how would it be solved
> post-separation for us?  (It sounds like a zero-sum game of pain transfer
> to me.)
> 
> I'm relatively unfamiliar with the arguments.  Can someone provide
> a brief list of reasons this is needed, and how the inconvenience to Linux
> kernel developers will be minimized, should it proceed?

One of the reasons for doing devicetrees is to separate the hardware
description from the code so that:
- Other OSes (and bootloaders) can use the same description to start on
  a given hardware
- A generic Kernel can be started on any hardware
- A hardware describes itself, makes itself more introspecitve so we can
  go away from very specialized kernels

This can't be archieved when the devicetrees are constantly changing. So
we should separate the devicetrees from the kernel to make them usable
for other projects, but also to make the kernel more universally usable.
Compatibility issues will be far more obvious when kernel and
devicetrees are separated, but this will make people behave more
carefully and helps making the devicetree interface more stable and
usable.

Just to make that sure: It's an illusion that future kernels will be
100% compatible with old devicetrees, but we should at least follow a
best effort approach. If they are compatible enough to at least bring up
the hardware then this is at least enough to install a better
devicetree.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Tue, Feb 18, 2014 at 11:47:56AM -0800, Olof Johansson wrote:
> On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> >> >   - Is the Linux development workflow ready for devicetree to move out
> >> >     of the Linux Kernel?
> >>
> >> I hope so since keeping the devicetrees in sync with the kernel is a
> >> pain for all external users.
> >
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> 
> I'M SCREAMING NOW. :-)
> 
> Honestly though, I think we need to do this carefully. Even though we
> don't like it, there are still lots of bindings in flux and
> cross-dependencies between two independent repos will be a major pain.
> 
> I think we have two options:
> 
> 1. Bring out everything in the current kernel repo to a separate one,
> but do it my mirroring over. Changes go into the kernel repo first and
> then comes over to this one, but other projects can mirror the
> standalone repo without downloading a whole kernel tree.
> 
> 2. Remove the kernel contents and move it over to the new repo. This
> should be done independently for each platform, and the maintainers
> get to decide if, when and how they do it. Some platforms are ready
> for it (some have been for a long time), others are not. And it's up
> to the maintainer, since they are the ones we will yell at when they
> make our life miserable by adding cross-dependencies with an external
> repo. Breakage due to the move is something we should have to put up
> with, etc.

Doing the move on a SoC/Maintainer basis is a good idea. I think the
move only makes sense for SoCs where the basic infrastructure devices
which are used by other devices are ironed out, namely interrupts,
GPIOs, DMA channels, pinctrl. As long as these are not present in a
devicetree it won't be usable by future kernels.

For i.MX I made the experience that the builtin barebox devicetree very
often is enough for my usecases so that I just throw a
imx_v6_v7_defcfonfig kernel to the board and don't have to think about
devicetrees anymore (This comes to an abrupt end when graphics is involved
though)

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Tue, Feb 18, 2014 at 11:47:56AM -0800, Olof Johansson wrote:
> On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> >> >   - Is the Linux development workflow ready for devicetree to move out
> >> >     of the Linux Kernel?
> >>
> >> I hope so since keeping the devicetrees in sync with the kernel is a
> >> pain for all external users.
> >
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> 
> I'M SCREAMING NOW. :-)

.:. I just pooped my pants. :)

> Honestly though, I think we need to do this carefully. Even though we
> don't like it, there are still lots of bindings in flux and
> cross-dependencies between two independent repos will be a major pain.

Agreed.

> I think we have two options:
> 
> 1. Bring out everything in the current kernel repo to a separate one,
> but do it my mirroring over. Changes go into the kernel repo first and
> then comes over to this one, but other projects can mirror the
> standalone repo without downloading a whole kernel tree.

I prefer this one.  Assuming that a separate repo is mostly agreed upon,
this allows us to provide the tree in an easily digestible way without
impacting the current workflow.

Also, if it works for the other projects, no one says we have to move
beyond this step.

> 2. Remove the kernel contents and move it over to the new repo. This
> should be done independently for each platform, and the maintainers
> get to decide if, when and how they do it. Some platforms are ready
> for it (some have been for a long time), others are not. And it's up
> to the maintainer, since they are the ones we will yell at when they
> make our life miserable by adding cross-dependencies with an external
> repo. Breakage due to the move is something we should have to put up
> with, etc.
> 
> >> I'll likely need some barebox specific additions to the devicetrees.
> >> Right now our idea is to leave the provided devicetrees untouched and
> >> instead of compiling the board dts files directly we create
> >> <boardname>-barebox.dts files which include the original board files.
> >> That would allow us to provide additional information to barebox
> >> without having to carry patches for the devicetrees.
> >
> > So the resulting <boardname>-barebox.dtb is compiled into the barebox
> > binary?  Is the dtb passed to the kernel independently upgradeable?
> >
> > Why not post binding/dts patches for 'barebox,...' attributes that you
> > need?
> 
> +1. These should ideally be posted and reviewed too -- maybe they make
> sense to share between barebox and other firmware stacks, for example.

Agreed, so is there anything preventing that from happening currently?

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Tue, Feb 18, 2014 at 02:14:51PM -0800, Frank Rowand wrote:
> On 2/18/2014 10:18 AM, Jason Cooper wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> >>>   - Is the Linux development workflow ready for devicetree to move out
> >>>     of the Linux Kernel?
> >>
> >> I hope so since keeping the devicetrees in sync with the kernel is a
> >> pain for all external users.
> > 
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> 
> < snip >
> 
> Let me join the chorus of screams.
> 
> I would venture that the number of linux kernel developers actively
> touching device tree source files (and impacted by changes to them)
> is vastly larger than any other set of developers.
> 
> My experience in the kernel is already that finding working device
> tree fragments that match current under development code is difficult.
> (I am not trying to generalize for all systems, just the ones I use.)
> 
> Moving the device tree source files out of the kernel git tree will
> only make things worse.

Having written/applied a significant portion of the arm devicetree files
for several SoCs over the past couple of years, I share your concern.
The reason I bring this up is because I keep seeing the little churn
here and there because we _can_.

I find it very similar to a teenager reaching adulthood.  At a certain
point, the parents have to say "Get out, get a job, get your own place."
It's hard for all parties involved, but it's the best thing for the
young adult in the long run.  There's no substitute for living on your
own and paying bills.

I think the long term health of the devicetree would benefit greatly
from being more detached than it currently is.  It would force everyone
to "Pay the bills".

The time might not be right now, and it probably will take one of the
more gradual forms Olof suggested.  But I think it should happen at some
point to help it grow up.

And we still have the short term problem of facilitating other projects
use of the devicetree.  Which can only make it more robust and accepted
by default.

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Jason Cooper]

On Wed, Feb 19, 2014 at 09:32:32AM +0100, Sascha Hauer wrote:
> On Tue, Feb 18, 2014 at 01:18:54PM -0500, Jason Cooper wrote:
> > On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> > > On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
> > ...
> > > >   - Is the Linux development workflow ready for devicetree to move out
> > > >     of the Linux Kernel?
> > > 
> > > I hope so since keeping the devicetrees in sync with the kernel is a
> > > pain for all external users.
> > 
> > Well, I haven't heard any screams yet.  I suspect people are waiting for
> > details on the exact form it would take before complaining...
> > 
> > > >   - How do we envision projects will use it?  git submodule?  reference
> > > >     a version tag?  (this is primarily targeted at bootloaders that need
> > > >     to compile in a dtb or subset of a dtb into the bootloader)
> > > 
> > > I would prefer to use it as a submodule.
> > 
> > ok.  I've often thought that was the right solution for several things
> > (dtc.git inside the kernel tree), but no one ever seemed to speak of it
> > or bring it up.  Kinda like leprosy.
> > 
> > It does add an extra step to build process for new users.  Although that
> > could be handled in the Makefile.
> > 
> > > I'll likely need some barebox specific additions to the devicetrees.
> > > Right now our idea is to leave the provided devicetrees untouched and
> > > instead of compiling the board dts files directly we create
> > > <boardname>-barebox.dts files which include the original board files.
> > > That would allow us to provide additional information to barebox
> > > without having to carry patches for the devicetrees.
> > 
> > So the resulting <boardname>-barebox.dtb is compiled into the barebox
> > binary?  Is the dtb passed to the kernel independently upgradeable?
> 
> Yes, it's compiled into barebox. barebox uses it for probing from
> devicetree. You can pass this devicetree to the kernel or provide
> another one should you want or have to.

Ok.

> > Why not post binding/dts patches for 'barebox,...' attributes that you
> > need?
> 
> I wasn't clear. The bindings themselves should be posted and merged
> mainline. mtd partitions are a good example for what I mean. I don't
> want to see them in the mainline dts files because that would mean my
> partitioning would change with each mainline change. Nevertheless I have
> the partitions in the barebox dts files.

Ack.  We should probably address those kinds of properties in a more
uniform way.  A good example is the OpenBlocks platform.  It can use
standard ram of various sizes in it's socket.  So the memory value in
the mainline tree is completely bogus and needs to be rewritten at boot
either by the bootloader or atags_to_fdt.

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/19/2014 1:08 AM, Sascha Hauer wrote:
> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>> I'm not in favor of separating the device tree information from the kernel.
>>
>> If we switch, then whatever synchronization issues other projects
>> are having now with synching with the device tree info from the kernel will
>> just then become the problem of the kernel developers, who will then
>> have to sync with the device tree info from another repository.  If the
>> sync issues can't be solved now for them, why or how would it be solved
>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>> to me.)
>>
>> I'm relatively unfamiliar with the arguments.  Can someone provide
>> a brief list of reasons this is needed, and how the inconvenience to Linux
>> kernel developers will be minimized, should it proceed?
> 


> One of the reasons for doing devicetrees is to separate the hardware
> description from the code so that:
> - Other OSes (and bootloaders) can use the same description to start on
>   a given hardware
> - A generic Kernel can be started on any hardware
> - A hardware describes itself, makes itself more introspecitve so we can
>   go away from very specialized kernels

Tim knows this ^^^^.  He was asking for the arguments for moving dts files
out of the linux kernel source tree.

Can you answer his question?

> 
> This can't be archieved when the devicetrees are constantly changing. So
> we should separate the devicetrees from the kernel to make them usable
> for other projects, but also to make the kernel more universally usable.
> Compatibility issues will be far more obvious when kernel and
> devicetrees are separated, but this will make people behave more
> carefully and helps making the devicetree interface more stable and
> usable.

So come up with a way of making dts files more stable while still in
the linux kernel source tree.  Moving the files to another repository
is not going to magically make them more stable.

> 
> Just to make that sure: It's an illusion that future kernels will be
> 100% compatible with old devicetrees, but we should at least follow a
> best effort approach. If they are compatible enough to at least bring up
> the hardware then this is at least enough to install a better
> devicetree.
> 
> Sascha
> 

-Frank
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/19/2014 10:46 AM, Jason Cooper wrote:
> On Tue, Feb 18, 2014 at 02:14:51PM -0800, Frank Rowand wrote:
>> On 2/18/2014 10:18 AM, Jason Cooper wrote:
>>> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>>>> On Mon, Feb 17, 2014 at 01:05:44PM -0500, Jason Cooper wrote:
>>> ...
>>>>>   - Is the Linux development workflow ready for devicetree to move out
>>>>>     of the Linux Kernel?
>>>>
>>>> I hope so since keeping the devicetrees in sync with the kernel is a
>>>> pain for all external users.
>>>
>>> Well, I haven't heard any screams yet.  I suspect people are waiting for
>>> details on the exact form it would take before complaining...
>>
>> < snip >
>>
>> Let me join the chorus of screams.
>>
>> I would venture that the number of linux kernel developers actively
>> touching device tree source files (and impacted by changes to them)
>> is vastly larger than any other set of developers.
>>
>> My experience in the kernel is already that finding working device
>> tree fragments that match current under development code is difficult.
>> (I am not trying to generalize for all systems, just the ones I use.)
>>
>> Moving the device tree source files out of the kernel git tree will
>> only make things worse.
> 
> Having written/applied a significant portion of the arm devicetree files
> for several SoCs over the past couple of years, I share your concern.
> The reason I bring this up is because I keep seeing the little churn
> here and there because we _can_.
> 
> I find it very similar to a teenager reaching adulthood.  At a certain
> point, the parents have to say "Get out, get a job, get your own place."
> It's hard for all parties involved, but it's the best thing for the
> young adult in the long run.  There's no substitute for living on your
> own and paying bills.

That is not exactly a technical reason (the teenager explanation).

> 
> I think the long term health of the devicetree would benefit greatly
> from being more detached than it currently is.  It would force everyone
> to "Pay the bills".
> 
> The time might not be right now, and it probably will take one of the
> more gradual forms Olof suggested.  But I think it should happen at some
> point to help it grow up.
> 
> And we still have the short term problem of facilitating other projects
> use of the devicetree.  Which can only make it more robust and accepted
> by default.

I am predicting increased pain for little gain if the dts files move out
of the linux kernel repository.

-Frnak

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 19, 2014, at 11:32 AM, Jason Cooper wrote:

> On Tue, Feb 18, 2014 at 11:47:56AM -0800, Olof Johansson wrote:
>> On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
>> I think we have two options:
>> 
>> 1. Bring out everything in the current kernel repo to a separate one,
>> but do it my mirroring over. Changes go into the kernel repo first and
>> then comes over to this one, but other projects can mirror the
>> standalone repo without downloading a whole kernel tree.
> 
> I prefer this one.  Assuming that a separate repo is mostly agreed upon,
> this allows us to provide the tree in an easily digestible way without
> impacting the current workflow.
> 
> Also, if it works for the other projects, no one says we have to move
> beyond this step.

I just joined this list...  What's the scope of what would move into the new
repo? The dts files with the binding docs, or also the code to implement
those bindings?

Warner

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 19, 2014, at 11:46 AM, Jason Cooper wrote:
> I think the long term health of the devicetree would benefit greatly
> from being more detached than it currently is.  It would force everyone
> to "Pay the bills".
> 
> The time might not be right now, and it probably will take one of the
> more gradual forms Olof suggested.  But I think it should happen at some
> point to help it grow up.
> 
> And we still have the short term problem of facilitating other projects
> use of the devicetree.  Which can only make it more robust and accepted
> by default.

I know it would be easier to import new device tree files into FreeBSD, and
support the latest version of the spec if there was a clean way of doing so.
Right now, with the files comingled with other files in the Linux tree it is harder
to do so than if they were stand-alone. Especially if the compiler could also
come from a near-by place.

Warner

P.S. I've been championing the use of fdt in FreeBSD for a few years now. I
am presently converting over all our Atmel support to use fdt, as well as
championing cross-SoC and cross-architecture use of fdt + gpio, fdt + i2c, etc.
Just so you know who I am, in case the name in unfamiliar to you...--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Tue, Feb 18, 2014 at 6:18 PM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>> It will be interesting to see which rules should apply for merging new
>> bindings. I know that devicetrees should be OS agnostic, but sometimes
>> they are modelled after how Linux currently works. What happens when the
>> *BSD guys have different ideas how a good binding looks like? How will
>> such conflicts be resolved?
>
> That's more a question for Grant.  I assume we'll all put on our big-boy
> pants and pick the best technical solution based on their merits. :)

I think you've answered it pretty competently.

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Rob Herring]

On Tue, Feb 18, 2014 at 4:44 PM, Tim Bird <tbird20d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> I'm not in favor of separating the device tree information from the kernel.
>
> If we switch, then whatever synchronization issues other projects
> are having now with synching with the device tree info from the kernel will
> just then become the problem of the kernel developers, who will then
> have to sync with the device tree info from another repository.  If the
> sync issues can't be solved now for them, why or how would it be solved
> post-separation for us?  (It sounds like a zero-sum game of pain transfer
> to me.)
>
> I'm relatively unfamiliar with the arguments.  Can someone provide
> a brief list of reasons this is needed, and how the inconvenience to Linux
> kernel developers will be minimized, should it proceed?

- Primarily, other projects like u-boot, barebox, FreeBSD and possibly
TianoCore (UEFI) are using DT now. Leaving them in the kernel will
cause fragmentation. The statements about barebox needing to add
barebox properties to the dtb is exactly the kind of fragmentation I'm
worried about.
- The need to share dts fragments across arches. This is a bit
orthogonal, but this restructuring would be easier done outside the
kernel tree. Restructuring everything in the kernel tree and then
moving it out would be a lot of churn.
- As we add schema checking, we need somewhere to put those.

One way to minimize the inconvenience is keep versioning and dev
cycles in sync with the kernel. We could also start doing things to
align the kernel workflow with how things will work when we do have a
separate repository.

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>> I'm not in favor of separating the device tree information from the kernel.
>>>
>>> If we switch, then whatever synchronization issues other projects
>>> are having now with synching with the device tree info from the kernel will
>>> just then become the problem of the kernel developers, who will then
>>> have to sync with the device tree info from another repository.  If the
>>> sync issues can't be solved now for them, why or how would it be solved
>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>> to me.)
>>>
>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>> kernel developers will be minimized, should it proceed?
>>
>
>
>> One of the reasons for doing devicetrees is to separate the hardware
>> description from the code so that:
>> - Other OSes (and bootloaders) can use the same description to start on
>>   a given hardware
>> - A generic Kernel can be started on any hardware
>> - A hardware describes itself, makes itself more introspecitve so we can
>>   go away from very specialized kernels
>
> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
> out of the linux kernel source tree.

We've made the decision that devicetree bindings need to be treated as
ABI, but as long as the .dts files live in the kernel there will
always be the temptation to just tweak things in lock-step and nobody
will notice. Splitting the files out gives that extra push to think
about whether changes to a binding will backwards compatible with a
tree that doesn't have those changes because the chances are a lot
higher that someone will hit that combination.

The other argument is shared source between
BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
is no good way to share the database of hardware descriptions.

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Olof Johansson]

On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> One way to minimize the inconvenience is keep versioning and dev
> cycles in sync with the kernel. We could also start doing things to
> align the kernel workflow with how things will work when we do have a
> separate repository.

I don't think aligning development cycles is what we want most here it
might be useful for us in Linux but it'll make things difficult for
other projects since they're not aware of our release cycles. The
device tree bindings and DT contents in that repo should be "always
stable", i.e. no merge window / rc concept. As soon as something goes
in it's live, and from then out only fixes to the DTS files (or
appending the binding).

For example, I don't want to have to track two trees to test against
-- I'll want to keep one repo of the very latest DT files and always
use those to boot any and all boards.


-Olof
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/19/2014 1:12 PM, Rob Herring wrote:
> On Tue, Feb 18, 2014 at 4:44 PM, Tim Bird <tbird20d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> I'm not in favor of separating the device tree information from the kernel.
>>
>> If we switch, then whatever synchronization issues other projects
>> are having now with synching with the device tree info from the kernel will
>> just then become the problem of the kernel developers, who will then
>> have to sync with the device tree info from another repository.  If the
>> sync issues can't be solved now for them, why or how would it be solved
>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>> to me.)
>>
>> I'm relatively unfamiliar with the arguments.  Can someone provide
>> a brief list of reasons this is needed, and how the inconvenience to Linux
>> kernel developers will be minimized, should it proceed?
> 
> - Primarily, other projects like u-boot, barebox, FreeBSD and possibly
> TianoCore (UEFI) are using DT now. Leaving them in the kernel will
> cause fragmentation. The statements about barebox needing to add
> barebox properties to the dtb is exactly the kind of fragmentation I'm
> worried about.

"Devicetree only describes the hardware" (paraphrasing a claim often made).
If the linux kernel dts files do not fully describe the hardware then it
is appropriate to add the missing info.

> - The need to share dts fragments across arches. This is a bit
> orthogonal, but this restructuring would be easier done outside the
> kernel tree. Restructuring everything in the kernel tree and then
> moving it out would be a lot of churn.

The churn will occur no matter what repository the files are in.

> - As we add schema checking, we need somewhere to put those.

And why does _that_ need to be in an external repository?

> 
> One way to minimize the inconvenience is keep versioning and dev
> cycles in sync with the kernel. We could also start doing things to
> align the kernel workflow with how things will work when we do have a
> separate repository.

If you stay in sync with the kernel then you are still tied to the
linux kernel source repository.  No gain.

-Frank

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/19/2014 1:15 PM, Grant Likely wrote:
> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>
>>>> If we switch, then whatever synchronization issues other projects
>>>> are having now with synching with the device tree info from the kernel will
>>>> just then become the problem of the kernel developers, who will then
>>>> have to sync with the device tree info from another repository.  If the
>>>> sync issues can't be solved now for them, why or how would it be solved
>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>> to me.)
>>>>
>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>> kernel developers will be minimized, should it proceed?
>>>
>>
>>
>>> One of the reasons for doing devicetrees is to separate the hardware
>>> description from the code so that:
>>> - Other OSes (and bootloaders) can use the same description to start on
>>>   a given hardware
>>> - A generic Kernel can be started on any hardware
>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>   go away from very specialized kernels
>>
>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>> out of the linux kernel source tree.
> 
> We've made the decision that devicetree bindings need to be treated as
> ABI, but as long as the .dts files live in the kernel there will
> always be the temptation to just tweak things in lock-step and nobody
> will notice. Splitting the files out gives that extra push to think
> about whether changes to a binding will backwards compatible with a
> tree that doesn't have those changes because the chances are a lot
> higher that someone will hit that combination.

We have other ABI in the kernel.  The maintainers have to develop a
spine if they are letting the supposed ABI changes occur they way
you suggest.

I thought that the actions flowing out of Edinburgh were supposed to
move us in the direction of developing the ABI mentality, and the
process to allow development and experimentation to occur (with a
clear label that they were not yet ABI) then a way to decide that
the development of a devicetree binding was complete enough to be
ABI.

> 
> The other argument is shared source between
> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
> is no good way to share the database of hardware descriptions.
> 
> g.
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 19, 2014, at 2:09 PM, Grant Likely wrote:

> On Tue, Feb 18, 2014 at 6:18 PM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
>> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>>> It will be interesting to see which rules should apply for merging new
>>> bindings. I know that devicetrees should be OS agnostic, but sometimes
>>> they are modelled after how Linux currently works. What happens when the
>>> *BSD guys have different ideas how a good binding looks like? How will
>>> such conflicts be resolved?
>> 
>> That's more a question for Grant.  I assume we'll all put on our big-boy
>> pants and pick the best technical solution based on their merits. :)
> 
> I think you've answered it pretty competently.

What, the BSDs don't get a free pass to dump junk into the process. I'm shocked :)

But we have bug-boy pants over in BSD land, so that shouldn't be a problem.

Warner

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/19/2014 1:15 PM, Grant Likely wrote:
> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>
>>>> If we switch, then whatever synchronization issues other projects
>>>> are having now with synching with the device tree info from the kernel will
>>>> just then become the problem of the kernel developers, who will then
>>>> have to sync with the device tree info from another repository.  If the
>>>> sync issues can't be solved now for them, why or how would it be solved
>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>> to me.)
>>>>
>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>> kernel developers will be minimized, should it proceed?
>>>
>>
>>
>>> One of the reasons for doing devicetrees is to separate the hardware
>>> description from the code so that:
>>> - Other OSes (and bootloaders) can use the same description to start on
>>>   a given hardware
>>> - A generic Kernel can be started on any hardware
>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>   go away from very specialized kernels
>>
>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>> out of the linux kernel source tree.
> 
> We've made the decision that devicetree bindings need to be treated as
> ABI, but as long as the .dts files live in the kernel there will
> always be the temptation to just tweak things in lock-step and nobody
> will notice. Splitting the files out gives that extra push to think
> about whether changes to a binding will backwards compatible with a
> tree that doesn't have those changes because the chances are a lot
> higher that someone will hit that combination.
> 
> The other argument is shared source between
> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
> is no good way to share the database of hardware descriptions.

We could provide an easy export (see below).  What do you think?

-Frank



Proof of concept: export devicetree source and header files

Not-signed-off-by: Frank Rowand <frank.rowand-/MT0OVThwyLZJqsBc5GL+g@public.gmane.org>


---
 Makefile                   |   27 	26 +	1 -	0 !
 scripts/Makefile.dtsexport |   19 	19 +	0 -	0 !
 scripts/dts.sh             |   11 	11 +	0 -	0 !
 3 files changed, 56 insertions(+), 1 deletion(-)

Index: b/Makefile
===================================================================
--- a/Makefile
+++ b/Makefile
@@ -234,6 +234,9 @@ endif
 # Where to locate arch specific headers
 hdr-arch  := $(SRCARCH)
 
+# Where to locate arch specific dts
+dts-arch  := $(SRCARCH)
+
 KCONFIG_CONFIG	?= .config
 export KCONFIG_CONFIG
 
@@ -463,7 +466,7 @@ version_h := include/generated/uapi/linu
 no-dot-config-targets := clean mrproper distclean \
 			 cscope gtags TAGS tags help %docs check% coccicheck \
 			 $(version_h) headers_% archheaders archscripts \
-			 kernelversion %src-pkg
+			 kernelversion %src-pkg dts_export%
 
 config-targets := 0
 mixed-targets  := 0
@@ -933,6 +936,26 @@ firmware_install: FORCE
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
 
 # ---------------------------------------------------------------------------
+# devicetree source and headers export
+
+#Default location for installed headers
+export EXPORT_DTS_PATH = $(KBUILD_OUTPUT)/usr/dts
+
+PHONY += dts_export_headers
+dts_export_headers: scripts_basic
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=include/dt-bindings
+	@echo $(KERNELRELEASE) >$(EXPORT_DTS_PATH)/linux_version
+
+PHONY += dts_export_all
+dts_export_all: dts_export_headers
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/dts.sh
+
+PHONY += dts_export
+dts_export:
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=arch/$(dts-arch)/boot/dts
+
+
+# ---------------------------------------------------------------------------
 # Kernel headers
 
 #Default location for installed headers
@@ -1160,6 +1183,8 @@ help:
 	@echo  '  modules_prepare - Set up for building external modules'
 	@echo  '  tags/TAGS	  - Generate tags file for editors'
 	@echo  '  cscope	  - Generate cscope index'
+	@echo  '  dts_export_all  - Export devicetree source and headers to EXPORT_DTS_PATH'
+	@echo  '                    (default: $(EXPORT_DTS_PATH))'
 	@echo  '  gtags           - Generate GNU GLOBAL index'
 	@echo  '  kernelrelease	  - Output the release version string'
 	@echo  '  kernelversion	  - Output the version stored in Makefile'
Index: b/scripts/Makefile.dtsexport
===================================================================
--- /dev/null
+++ b/scripts/Makefile.dtsexport
@@ -0,0 +1,19 @@
+# ==========================================================================
+# Exporting dts source and header files
+#
+# ==========================================================================
+
+srcpath := $(srctree)/$(obj)/*
+dstpath := $(EXPORT_DTS_PATH)/$(obj)
+
+include scripts/Kbuild.include
+
+
+quiet_cmd_install = EXPORT $(subst $(srctree)/,,$(srcpath))
+      cmd_install = mkdir -p $(dstpath); cp -a $(srcpath) $(dstpath)
+
+
+.PHONY: export
+export:
+	$(call cmd,install)
+
Index: b/scripts/dts.sh
===================================================================
--- /dev/null
+++ b/scripts/dts.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+# Run dts_export command for all architectures
+
+# Stop on error
+set -e
+
+for arch in $(ls ${srctree}/arch); do
+	if [ -d ${srctree}/arch/${arch}/boot/dts ]; then
+		make ARCH=${arch} dts_export
+	fi
+done
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Thu, Feb 20, 2014 at 4:58 AM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 2/19/2014 1:15 PM, Grant Likely wrote:
>> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>>
>>>>> If we switch, then whatever synchronization issues other projects
>>>>> are having now with synching with the device tree info from the kernel will
>>>>> just then become the problem of the kernel developers, who will then
>>>>> have to sync with the device tree info from another repository.  If the
>>>>> sync issues can't be solved now for them, why or how would it be solved
>>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>>> to me.)
>>>>>
>>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>>> kernel developers will be minimized, should it proceed?
>>>>
>>>
>>>
>>>> One of the reasons for doing devicetrees is to separate the hardware
>>>> description from the code so that:
>>>> - Other OSes (and bootloaders) can use the same description to start on
>>>>   a given hardware
>>>> - A generic Kernel can be started on any hardware
>>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>>   go away from very specialized kernels
>>>
>>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>>> out of the linux kernel source tree.
>>
>> We've made the decision that devicetree bindings need to be treated as
>> ABI, but as long as the .dts files live in the kernel there will
>> always be the temptation to just tweak things in lock-step and nobody
>> will notice. Splitting the files out gives that extra push to think
>> about whether changes to a binding will backwards compatible with a
>> tree that doesn't have those changes because the chances are a lot
>> higher that someone will hit that combination.
>>
>> The other argument is shared source between
>> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
>> is no good way to share the database of hardware descriptions.
>
> We could provide an easy export (see below).  What do you think?

Ian Campbell is already maintaining an export tree as a staging area
for an eventual split. He's had it up and running for almost a year
now:

http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Thu, 2014-02-20 at 08:14 +0000, Grant Likely wrote:
> >> The other argument is shared source between
> >> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there

                            ^/Xen ;-)

> >> is no good way to share the database of hardware descriptions.
> >
> > We could provide an easy export (see below).  What do you think?
> 
> Ian Campbell is already maintaining an export tree as a staging area
> for an eventual split. He's had it up and running for almost a year
> now:
> 
> http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git

NB, per the name it is potentially rebasing, effectively until if/when
it becomes the primary. In practice it hasn't been rebased since the
early days while I was sorting out the translations.

I've got an open action to move it somewhere other than xenbits,
probably git.kernel.org unless someone can supply a more neutral place.

WRT people wanting a more staged transition rather than a flag day: I
think I could tweaking things to rewrite each arch onto its own branch,
which could all be merged up into a single master branch. Then as $ARCH
moves over to using device.tree.git rather than linux.git as their
primary location for DTS we can simply drop the conversion for that
branch and start accepting patches onto master. I think doing anything
more fine-grained than per-arch would be tricky, at least to do
automagically.

Ian.



--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Wed, 2014-02-19 at 13:20 -0800, Olof Johansson wrote:
> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > One way to minimize the inconvenience is keep versioning and dev
> > cycles in sync with the kernel. We could also start doing things to
> > align the kernel workflow with how things will work when we do have a
> > separate repository.
> 
> I don't think aligning development cycles is what we want most here it
> might be useful for us in Linux but it'll make things difficult for
> other projects since they're not aware of our release cycles. The
> device tree bindings and DT contents in that repo should be "always
> stable", i.e. no merge window / rc concept. As soon as something goes
> in it's live, and from then out only fixes to the DTS files (or
> appending the binding).

I agree, but I also think it would be useful to draw the occasional line
in the sand and do e.g. monthly or quarterly tagged releases e.g. for
distros who want a tarball package to work off.

There's no reason for that to be tied to any other projects dev cycle
though.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Wed, 2014-02-19 at 12:18 -0800, Frank Rowand wrote:
> I am predicting increased pain for little gain if the dts files move out
> of the linux kernel repository.

That's only the case if you take a very Linux centric point of view and
as others have pointed out there are numerous other projects which would
like to both consume and contribute to the set of DTS files, forcing all
those people to take part in Linux development in order to do so is a
barrier which has been shown to be too high in many cases, which is
already leading to fragmentation.

If DTB is to be taken seriously as a generic standard for describing
hardware then IMHO it cannot afford to remain so closely tied to a
single consumer.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Wed, 2014-02-19 at 13:23 -0700, Warner Losh wrote:
> On Feb 19, 2014, at 11:32 AM, Jason Cooper wrote:
> 
> > On Tue, Feb 18, 2014 at 11:47:56AM -0800, Olof Johansson wrote:
> >> On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> >> I think we have two options:
> >> 
> >> 1. Bring out everything in the current kernel repo to a separate one,
> >> but do it my mirroring over. Changes go into the kernel repo first and
> >> then comes over to this one, but other projects can mirror the
> >> standalone repo without downloading a whole kernel tree.
> > 
> > I prefer this one.  Assuming that a separate repo is mostly agreed upon,
> > this allows us to provide the tree in an easily digestible way without
> > impacting the current workflow.
> > 
> > Also, if it works for the other projects, no one says we have to move
> > beyond this step.
> 
> I just joined this list...  What's the scope of what would move into the new
> repo? The dts files with the binding docs, or also the code to implement
> those bindings?

Just the dts(i) and Bindings docs, not the code.

e.g. something like
http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git
would be the ultimate goal.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Wed, 19 Feb 2014 13:20:15 -0800, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > One way to minimize the inconvenience is keep versioning and dev
> > cycles in sync with the kernel. We could also start doing things to
> > align the kernel workflow with how things will work when we do have a
> > separate repository.
> 
> I don't think aligning development cycles is what we want most here it
> might be useful for us in Linux but it'll make things difficult for
> other projects since they're not aware of our release cycles. The
> device tree bindings and DT contents in that repo should be "always
> stable", i.e. no merge window / rc concept. As soon as something goes
> in it's live, and from then out only fixes to the DTS files (or
> appending the binding).
> 
> For example, I don't want to have to track two trees to test against
> -- I'll want to keep one repo of the very latest DT files and always
> use those to boot any and all boards.

This approach does have the subtle side effect that differs from what we
discussed in Edinburgh.  We've talked about always being able to boot a
new kernel on an old devicetree, but not a new devicetree on an old
kernel. With a separate board database repo we are going to hit both
cases. At least to a limited extent we're going to need older kernels
booting with the latest devicetree, and we'll need some rules about how
that gets applied.

The alternative is that binding changes land in .dts files after a
kernel release, and I don't think we want that situation at all.

This is an issue for new bindings, only when a binding is getting
extended or replaced. If the change is merely adding new properties then
there shouldn't be a problem (the old kernel will ignore the new
properties). If it removes properties or creates a new compatible string
(dropping the one supported by an older kernel) then it won't boot.

Ultimately this is probably the right thing to do, but it will be
difficult. Keeping a staging process for new bindings in lock step
with the kernel is probably the way to mitigate this.

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Thu, 20 Feb 2014 09:50:31 +0000, Ian Campbell <ijc-KcIKpvwj1kUDXYZnReoRVg@public.gmane.org> wrote:
> On Wed, 2014-02-19 at 13:23 -0700, Warner Losh wrote:
> > On Feb 19, 2014, at 11:32 AM, Jason Cooper wrote:
> > 
> > > On Tue, Feb 18, 2014 at 11:47:56AM -0800, Olof Johansson wrote:
> > >> On Tue, Feb 18, 2014 at 10:18 AM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
> > >> I think we have two options:
> > >> 
> > >> 1. Bring out everything in the current kernel repo to a separate one,
> > >> but do it my mirroring over. Changes go into the kernel repo first and
> > >> then comes over to this one, but other projects can mirror the
> > >> standalone repo without downloading a whole kernel tree.
> > > 
> > > I prefer this one.  Assuming that a separate repo is mostly agreed upon,
> > > this allows us to provide the tree in an easily digestible way without
> > > impacting the current workflow.
> > > 
> > > Also, if it works for the other projects, no one says we have to move
> > > beyond this step.
> > 
> > I just joined this list...  What's the scope of what would move into the new
> > repo? The dts files with the binding docs, or also the code to implement
> > those bindings?
> 
> Just the dts(i) and Bindings docs, not the code.
> 
> e.g. something like
> http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git
> would be the ultimate goal.

And schemas files when we have them.

g.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Wed, 19 Feb 2014 14:43:58 -0700, Warner Losh <wlosh@bsdimp.com> wrote:
> 
> On Feb 19, 2014, at 2:09 PM, Grant Likely wrote:
> 
> > On Tue, Feb 18, 2014 at 6:18 PM, Jason Cooper <jason@lakedaemon.net> wrote:
> >> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
> >>> It will be interesting to see which rules should apply for merging new
> >>> bindings. I know that devicetrees should be OS agnostic, but sometimes
> >>> they are modelled after how Linux currently works. What happens when the
> >>> *BSD guys have different ideas how a good binding looks like? How will
> >>> such conflicts be resolved?
> >> 
> >> That's more a question for Grant.  I assume we'll all put on our big-boy
> >> pants and pick the best technical solution based on their merits. :)
> > 
> > I think you've answered it pretty competently.
> 
> What, the BSDs don't get a free pass to dump junk into the process. I'm shocked :)
> 
> But we have bug-boy pants over in BSD land, so that shouldn't be a problem.

Bug-boy pants? Sounds sticky.

g.

Re: devicetree repository separation/migration

[Rob Herring]

On Wed, Feb 19, 2014 at 3:20 PM, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> One way to minimize the inconvenience is keep versioning and dev
>> cycles in sync with the kernel. We could also start doing things to
>> align the kernel workflow with how things will work when we do have a
>> separate repository.
>
> I don't think aligning development cycles is what we want most here it
> might be useful for us in Linux but it'll make things difficult for
> other projects since they're not aware of our release cycles. The
> device tree bindings and DT contents in that repo should be "always
> stable", i.e. no merge window / rc concept. As soon as something goes
> in it's live, and from then out only fixes to the DTS files (or
> appending the binding).

I agree we wouldn't really want or need to follow rc's and master
always being stable should be the target, but people will want
"releases." There is no reason you can't support both. Take dtc as an
example, the release process is someone requests Jon to make a release
because they want some feature not in a tagged release, so he tags
master and we have a "release." So we can either make something up for
versions and cadence, follow the kernel, or do both. I would view
syncing with kernel versions to be only for a transition period to
ease the concerns of those who are keeping their dtb version aligned
to kernel version.

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Rob Herring]

On Wed, Feb 19, 2014 at 3:32 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 2/19/2014 1:12 PM, Rob Herring wrote:
>> On Tue, Feb 18, 2014 at 4:44 PM, Tim Bird <tbird20d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> I'm not in favor of separating the device tree information from the kernel.
>>>
>>> If we switch, then whatever synchronization issues other projects
>>> are having now with synching with the device tree info from the kernel will
>>> just then become the problem of the kernel developers, who will then
>>> have to sync with the device tree info from another repository.  If the
>>> sync issues can't be solved now for them, why or how would it be solved
>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>> to me.)
>>>
>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>> kernel developers will be minimized, should it proceed?
>>
>> - Primarily, other projects like u-boot, barebox, FreeBSD and possibly
>> TianoCore (UEFI) are using DT now. Leaving them in the kernel will
>> cause fragmentation. The statements about barebox needing to add
>> barebox properties to the dtb is exactly the kind of fragmentation I'm
>> worried about.
>
> "Devicetree only describes the hardware" (paraphrasing a claim often made).
> If the linux kernel dts files do not fully describe the hardware then it
> is appropriate to add the missing info.

Yes, but if hardware description is what is being added, then that is
not bootloader specific. I would guess this case is not h/w
description, but I don't know as none of it has been posted for
review. The real problem here is adding bindings without review.

Not fully describing the h/w and adding to the dts is common and not
really an issue as long as it is done in an ABI compatible way.

>> - The need to share dts fragments across arches. This is a bit
>> orthogonal, but this restructuring would be easier done outside the
>> kernel tree. Restructuring everything in the kernel tree and then
>> moving it out would be a lot of churn.
>
> The churn will occur no matter what repository the files are in.

Yes, but doing this in the kernel is much harder to coordinate with
Linus and architecture maintainers. I don't think we want to go thru
that process when we plan to just remove everything.

>> - As we add schema checking, we need somewhere to put those.
>
> And why does _that_ need to be in an external repository?

For the same reason as everything else (binding docs, dts files), so
people outside of kernel developers will contribute.

>>
>> One way to minimize the inconvenience is keep versioning and dev
>> cycles in sync with the kernel. We could also start doing things to
>> align the kernel workflow with how things will work when we do have a
>> separate repository.
>
> If you stay in sync with the kernel then you are still tied to the
> linux kernel source repository.  No gain.

The only synchronization would be tagging the repo at the same time
and using the same version numbering. Using that versus working on
master all the time would be completely up to end users. It's really
only encouraging people to test a specific combination of kernel and
dtb. The other option is we simply make up our own tagging/release
scheme.

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/20/2014 12:14 AM, Grant Likely wrote:
> On Thu, Feb 20, 2014 at 4:58 AM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On 2/19/2014 1:15 PM, Grant Likely wrote:
>>> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>>>
>>>>>> If we switch, then whatever synchronization issues other projects
>>>>>> are having now with synching with the device tree info from the kernel will
>>>>>> just then become the problem of the kernel developers, who will then
>>>>>> have to sync with the device tree info from another repository.  If the
>>>>>> sync issues can't be solved now for them, why or how would it be solved
>>>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>>>> to me.)
>>>>>>
>>>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>>>> kernel developers will be minimized, should it proceed?
>>>>>
>>>>
>>>>
>>>>> One of the reasons for doing devicetrees is to separate the hardware
>>>>> description from the code so that:
>>>>> - Other OSes (and bootloaders) can use the same description to start on
>>>>>   a given hardware
>>>>> - A generic Kernel can be started on any hardware
>>>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>>>   go away from very specialized kernels
>>>>
>>>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>>>> out of the linux kernel source tree.
>>>
>>> We've made the decision that devicetree bindings need to be treated as
>>> ABI, but as long as the .dts files live in the kernel there will
>>> always be the temptation to just tweak things in lock-step and nobody
>>> will notice. Splitting the files out gives that extra push to think
>>> about whether changes to a binding will backwards compatible with a
>>> tree that doesn't have those changes because the chances are a lot
>>> higher that someone will hit that combination.
>>>
>>> The other argument is shared source between
>>> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
>>> is no good way to share the database of hardware descriptions.
>>
>> We could provide an easy export (see below).  What do you think?
> 
> Ian Campbell is already maintaining an export tree as a staging area
> for an eventual split. He's had it up and running for almost a year
> now:
> 
> http://xenbits.xen.org/gitweb/?p=people/ianc/device-tree-rebasing.git
> 
> g.
> 

So there already is a "good way to share the database of hardware
descriptions" in addition to the one I provided.

-Frank
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/17/2014 10:05 AM, Jason Cooper wrote:
> All,
> 
> At last weeks devicetree irc meeting, I took on the task of writing this
> email.  I'm a bit behind.
> 
> One of the outcomes of the ARM/devicetree discussion at the 2013 Kernel
> Summit was that we were going to hold off on separating the devicetree
> from the Linux kernel tree.  The primary reason for this was to get
> through the backlog of patches.
> 
> It's been several months, and we're seeing evidence of other projects
> having difficulty keeping in sync with the kernel tree.  Specifically,
> barebox is having trouble syncing:
> 
>   http://list-archives.org/2014/02/07/barebox-lists-infradead-org/devicetree-maintenance-in-barebox/f/5820726136

< snip >

Sascha,

(Directing this to you, because the devicetree-maintenance-in-barebox thread
begins with an email from you.)

I have read through the referenced thread, but do not yet understand the cause
of the issues the barebox project is facing.  What I got from the thread is
that the barebox project maintains some devicetree changes in the project
repository, and it is difficult to manage these changes as the upstream
project (the Linux kernel) makes changes.

What are the barebox changes to dts files?

Why are the changes not submitted upstream?  (Or if they were submitted, why
were they not accepted?)

I'm not sure what else to ask to try to understand the issues for barebox.  Is
there anything else you can say to help me understand?

How would moving the devicetree files to another repository (also external to
the barebox project) resolve the issues for the barebox project?

Thanks,

-Frank

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Thu, Feb 20, 2014 at 05:07:12PM -0800, Frank Rowand wrote:
> On 2/17/2014 10:05 AM, Jason Cooper wrote:
> > All,
> > 
> > At last weeks devicetree irc meeting, I took on the task of writing this
> > email.  I'm a bit behind.
> > 
> > One of the outcomes of the ARM/devicetree discussion at the 2013 Kernel
> > Summit was that we were going to hold off on separating the devicetree
> > from the Linux kernel tree.  The primary reason for this was to get
> > through the backlog of patches.
> > 
> > It's been several months, and we're seeing evidence of other projects
> > having difficulty keeping in sync with the kernel tree.  Specifically,
> > barebox is having trouble syncing:
> > 
> >   http://list-archives.org/2014/02/07/barebox-lists-infradead-org/devicetree-maintenance-in-barebox/f/5820726136
> 
> < snip >
> 
> Sascha,
> 
> (Directing this to you, because the devicetree-maintenance-in-barebox thread
> begins with an email from you.)
> 
> I have read through the referenced thread, but do not yet understand the cause
> of the issues the barebox project is facing.  What I got from the thread is
> that the barebox project maintains some devicetree changes in the project
> repository, and it is difficult to manage these changes as the upstream
> project (the Linux kernel) makes changes.
> 
> What are the barebox changes to dts files?

Mostly mtd partitions and descriptions where barebox can find its
environment.

> 
> Why are the changes not submitted upstream?  (Or if they were submitted, why
> were they not accepted?)

The descriptions where to find the environment are obviously barebox
specific and not acceptable upstream. mtd partitions shouldn't be
upstream either since they should be changeable on a per project or even
per board basis.

As mentioned before we can put the changes into barebox specific dts
files which include the original upstream dts files. This way we can
keep the changes separated from the original files and don't have to
patch them.

> 
> I'm not sure what else to ask to try to understand the issues for barebox.  Is
> there anything else you can say to help me understand?
> 
> How would moving the devicetree files to another repository (also external to
> the barebox project) resolve the issues for the barebox project?

When I start supporting a new board I start with porting the bootloader.
For doing so I create an initial dts for the board which is then part of
barebox. At some point barebox development is done, further work is done
in the kernel, so the dts is copied there. The graphics nodes are added
which are irrelevant for barebox. Along the way I find some bugs in
the dts which are relevant for barebox, so I have to sync the dts back
to barebox at some point.
When someone posts barebox patches for a new board I normally would have
to ask him to post the patches for Linux inclusion first, because
otherwise I will get conflicting devicetrees downstream in barebox
later when the board is ported to the kernel.
None of the above is magically solved when the dts files are external to
the kernel, it just simplifies the process. It would allow me to use
version control systems to do the sync rather than copying files back
and forth.
Ians dts repository is a good start, but it contains a complete kernel
history and this is not very suitable as a submodule for other projects.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 19, 2014, at 9:58 PM, Frank Rowand wrote:

> On 2/19/2014 1:15 PM, Grant Likely wrote:
>> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>> 
>>>>> If we switch, then whatever synchronization issues other projects
>>>>> are having now with synching with the device tree info from the kernel will
>>>>> just then become the problem of the kernel developers, who will then
>>>>> have to sync with the device tree info from another repository.  If the
>>>>> sync issues can't be solved now for them, why or how would it be solved
>>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>>> to me.)
>>>>> 
>>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>>> kernel developers will be minimized, should it proceed?
>>>> 
>>> 
>>> 
>>>> One of the reasons for doing devicetrees is to separate the hardware
>>>> description from the code so that:
>>>> - Other OSes (and bootloaders) can use the same description to start on
>>>>  a given hardware
>>>> - A generic Kernel can be started on any hardware
>>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>>  go away from very specialized kernels
>>> 
>>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>>> out of the linux kernel source tree.
>> 
>> We've made the decision that devicetree bindings need to be treated as
>> ABI, but as long as the .dts files live in the kernel there will
>> always be the temptation to just tweak things in lock-step and nobody
>> will notice. Splitting the files out gives that extra push to think
>> about whether changes to a binding will backwards compatible with a
>> tree that doesn't have those changes because the chances are a lot
>> higher that someone will hit that combination.
>> 
>> The other argument is shared source between
>> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
>> is no good way to share the database of hardware descriptions.
> 
> We could provide an easy export (see below).  What do you think?

So what would the process be to get changes to those files upstream if you did this? It would make it marginally easier to USE, but once disconnected from the git world, a lot harder to track and fix.

Also, you should export the device tree docs too, since they are, in theory, a set. And honestly, the device tree doc files need more work than the .dts and .dtsi files.

Warner

> -Frank
> 
> 
> 
> Proof of concept: export devicetree source and header files
> 
> Not-signed-off-by: Frank Rowand <frank.rowand-/MT0OVThwyLZJqsBc5GL+g@public.gmane.org>
> 
> 
> ---
> Makefile                   |   27 	26 +	1 -	0 !
> scripts/Makefile.dtsexport |   19 	19 +	0 -	0 !
> scripts/dts.sh             |   11 	11 +	0 -	0 !
> 3 files changed, 56 insertions(+), 1 deletion(-)
> 
> Index: b/Makefile
> ===================================================================
> --- a/Makefile
> +++ b/Makefile
> @@ -234,6 +234,9 @@ endif
> # Where to locate arch specific headers
> hdr-arch  := $(SRCARCH)
> 
> +# Where to locate arch specific dts
> +dts-arch  := $(SRCARCH)
> +
> KCONFIG_CONFIG	?= .config
> export KCONFIG_CONFIG
> 
> @@ -463,7 +466,7 @@ version_h := include/generated/uapi/linu
> no-dot-config-targets := clean mrproper distclean \
> 			 cscope gtags TAGS tags help %docs check% coccicheck \
> 			 $(version_h) headers_% archheaders archscripts \
> -			 kernelversion %src-pkg
> +			 kernelversion %src-pkg dts_export%
> 
> config-targets := 0
> mixed-targets  := 0
> @@ -933,6 +936,26 @@ firmware_install: FORCE
> 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
> 
> # ---------------------------------------------------------------------------
> +# devicetree source and headers export
> +
> +#Default location for installed headers
> +export EXPORT_DTS_PATH = $(KBUILD_OUTPUT)/usr/dts
> +
> +PHONY += dts_export_headers
> +dts_export_headers: scripts_basic
> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=include/dt-bindings
> +	@echo $(KERNELRELEASE) >$(EXPORT_DTS_PATH)/linux_version
> +
> +PHONY += dts_export_all
> +dts_export_all: dts_export_headers
> +	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/dts.sh
> +
> +PHONY += dts_export
> +dts_export:
> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=arch/$(dts-arch)/boot/dts
> +
> +
> +# ---------------------------------------------------------------------------
> # Kernel headers
> 
> #Default location for installed headers
> @@ -1160,6 +1183,8 @@ help:
> 	@echo  '  modules_prepare - Set up for building external modules'
> 	@echo  '  tags/TAGS	  - Generate tags file for editors'
> 	@echo  '  cscope	  - Generate cscope index'
> +	@echo  '  dts_export_all  - Export devicetree source and headers to EXPORT_DTS_PATH'
> +	@echo  '                    (default: $(EXPORT_DTS_PATH))'
> 	@echo  '  gtags           - Generate GNU GLOBAL index'
> 	@echo  '  kernelrelease	  - Output the release version string'
> 	@echo  '  kernelversion	  - Output the version stored in Makefile'
> Index: b/scripts/Makefile.dtsexport
> ===================================================================
> --- /dev/null
> +++ b/scripts/Makefile.dtsexport
> @@ -0,0 +1,19 @@
> +# ==========================================================================
> +# Exporting dts source and header files
> +#
> +# ==========================================================================
> +
> +srcpath := $(srctree)/$(obj)/*
> +dstpath := $(EXPORT_DTS_PATH)/$(obj)
> +
> +include scripts/Kbuild.include
> +
> +
> +quiet_cmd_install = EXPORT $(subst $(srctree)/,,$(srcpath))
> +      cmd_install = mkdir -p $(dstpath); cp -a $(srcpath) $(dstpath)
> +
> +
> +.PHONY: export
> +export:
> +	$(call cmd,install)
> +
> Index: b/scripts/dts.sh
> ===================================================================
> --- /dev/null
> +++ b/scripts/dts.sh
> @@ -0,0 +1,11 @@
> +#!/bin/sh
> +# Run dts_export command for all architectures
> +
> +# Stop on error
> +set -e
> +
> +for arch in $(ls ${srctree}/arch); do
> +	if [ -d ${srctree}/arch/${arch}/boot/dts ]; then
> +		make ARCH=${arch} dts_export
> +	fi
> +done
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 20, 2014, at 4:38 AM, Grant Likely wrote:

> On Wed, 19 Feb 2014 13:20:15 -0800, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
>> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> One way to minimize the inconvenience is keep versioning and dev
>>> cycles in sync with the kernel. We could also start doing things to
>>> align the kernel workflow with how things will work when we do have a
>>> separate repository.
>> 
>> I don't think aligning development cycles is what we want most here it
>> might be useful for us in Linux but it'll make things difficult for
>> other projects since they're not aware of our release cycles. The
>> device tree bindings and DT contents in that repo should be "always
>> stable", i.e. no merge window / rc concept. As soon as something goes
>> in it's live, and from then out only fixes to the DTS files (or
>> appending the binding).
>> 
>> For example, I don't want to have to track two trees to test against
>> -- I'll want to keep one repo of the very latest DT files and always
>> use those to boot any and all boards.
> 
> This approach does have the subtle side effect that differs from what we
> discussed in Edinburgh.  We've talked about always being able to boot a
> new kernel on an old devicetree, but not a new devicetree on an old
> kernel. With a separate board database repo we are going to hit both
> cases. At least to a limited extent we're going to need older kernels
> booting with the latest devicetree, and we'll need some rules about how
> that gets applied.

I wasn't in Edinburgh...  Was this at the binary level or at the source level? I'm thinking specifically about the move to cpp in the back of my mind...

> The alternative is that binding changes land in .dts files after a
> kernel release, and I don't think we want that situation at all.
> 
> This is an issue for new bindings, only when a binding is getting
> extended or replaced. If the change is merely adding new properties then
> there shouldn't be a problem (the old kernel will ignore the new
> properties). If it removes properties or creates a new compatible string
> (dropping the one supported by an older kernel) then it won't boot.

Versioning here wouldn't save you either...

> Ultimately this is probably the right thing to do, but it will be
> difficult. Keeping a staging process for new bindings in lock step
> with the kernel is probably the way to mitigate this.

You could have a property like linux,version-min="3.22" if that becomes an issue...

Warner

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Warner Losh]

On Feb 20, 2014, at 4:39 AM, Grant Likely wrote:

> On Wed, 19 Feb 2014 14:43:58 -0700, Warner Losh <wlosh-uzTCJ5RojNnQT0dZR+AlfA@public.gmane.org> wrote:
>> 
>> On Feb 19, 2014, at 2:09 PM, Grant Likely wrote:
>> 
>>> On Tue, Feb 18, 2014 at 6:18 PM, Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org> wrote:
>>>> On Tue, Feb 18, 2014 at 04:57:50PM +0100, Sascha Hauer wrote:
>>>>> It will be interesting to see which rules should apply for merging new
>>>>> bindings. I know that devicetrees should be OS agnostic, but sometimes
>>>>> they are modelled after how Linux currently works. What happens when the
>>>>> *BSD guys have different ideas how a good binding looks like? How will
>>>>> such conflicts be resolved?
>>>> 
>>>> That's more a question for Grant.  I assume we'll all put on our big-boy
>>>> pants and pick the best technical solution based on their merits. :)
>>> 
>>> I think you've answered it pretty competently.
>> 
>> What, the BSDs don't get a free pass to dump junk into the process. I'm shocked :)
>> 
>> But we have bug-boy pants over in BSD land, so that shouldn't be a problem.
> 
> Bug-boy pants? Sounds sticky.

Yea, gotta those accidental typos...

Warner

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Olof Johansson]

On Fri, Feb 21, 2014 at 6:11 AM, Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org> wrote:
> On Thu, Feb 20, 2014 at 05:07:12PM -0800, Frank Rowand wrote:

>> Why are the changes not submitted upstream?  (Or if they were submitted, why
>> were they not accepted?)
>
> The descriptions where to find the environment are obviously barebox
> specific and not acceptable upstream.

Why not? Wouldn't it be useful if you could find where the environment
is from the running Linux image so that you can change variables
without rebooting into Barebox first?


-Olof
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Olof Johansson]

On Thu, Feb 20, 2014 at 3:38 AM, Grant Likely <grant.likely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org> wrote:
> On Wed, 19 Feb 2014 13:20:15 -0800, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
>> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> > One way to minimize the inconvenience is keep versioning and dev
>> > cycles in sync with the kernel. We could also start doing things to
>> > align the kernel workflow with how things will work when we do have a
>> > separate repository.
>>
>> I don't think aligning development cycles is what we want most here it
>> might be useful for us in Linux but it'll make things difficult for
>> other projects since they're not aware of our release cycles. The
>> device tree bindings and DT contents in that repo should be "always
>> stable", i.e. no merge window / rc concept. As soon as something goes
>> in it's live, and from then out only fixes to the DTS files (or
>> appending the binding).
>>
>> For example, I don't want to have to track two trees to test against
>> -- I'll want to keep one repo of the very latest DT files and always
>> use those to boot any and all boards.
>
> This approach does have the subtle side effect that differs from what we
> discussed in Edinburgh.  We've talked about always being able to boot a
> new kernel on an old devicetree, but not a new devicetree on an old
> kernel. With a separate board database repo we are going to hit both
> cases. At least to a limited extent we're going to need older kernels
> booting with the latest devicetree, and we'll need some rules about how
> that gets applied.

That's true. For the most part this should work well as we enable IP
for DT, since before then we'll fall back on old configuration ways.
Where it gets complicated is if we deprecate a whole binding and make
a new one, or if we change the meaning of properties. Due to the ABI
properties, we shouldn't be doing the latter anyway. The former is
obviously a problem but we might just have to live with it. I don't
think we're likely to hit it much in reality once bindings stabilize
-- main case would be if we want to rewrite some of the very old and
maybe not so good bindings (like the mtd driver binding that the at91
guys came up with early). But in reality we're likely to just stick
with them and not bother.

So, yes, it does change the formality but in reality I don't expect
that much of a difference.

> The alternative is that binding changes land in .dts files after a
> kernel release, and I don't think we want that situation at all.

Nope.

> This is an issue for new bindings, only when a binding is getting

s/is/isn't/, which I think was your intent.

> extended or replaced. If the change is merely adding new properties then
> there shouldn't be a problem (the old kernel will ignore the new
> properties). If it removes properties or creates a new compatible string
> (dropping the one supported by an older kernel) then it won't boot.

Yes, I think I said the same thing above in my reply now. :-)

> Ultimately this is probably the right thing to do, but it will be
> difficult. Keeping a staging process for new bindings in lock step
> with the kernel is probably the way to mitigate this.

This might work with the mirroring case, but it'll get awkward if we
have completely separate repos and thus potentially separate
reviewers/approvers, if things get additional scrutiny between the
staging phase in the kernel and when it gets merged upstream,
especially if the DTS has already been moved out. I'm not quite sure
how that would work, really.

-Olof
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Frank Rowand]

On 2/21/2014 10:22 AM, Warner Losh wrote:
> 
> On Feb 19, 2014, at 9:58 PM, Frank Rowand wrote:
> 
>> On 2/19/2014 1:15 PM, Grant Likely wrote:
>>> On Wed, Feb 19, 2014 at 8:16 PM, Frank Rowand <frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> On 2/19/2014 1:08 AM, Sascha Hauer wrote:
>>>>> On Tue, Feb 18, 2014 at 02:44:15PM -0800, Tim Bird wrote:
>>>>>> I'm not in favor of separating the device tree information from the kernel.
>>>>>>
>>>>>> If we switch, then whatever synchronization issues other projects
>>>>>> are having now with synching with the device tree info from the kernel will
>>>>>> just then become the problem of the kernel developers, who will then
>>>>>> have to sync with the device tree info from another repository.  If the
>>>>>> sync issues can't be solved now for them, why or how would it be solved
>>>>>> post-separation for us?  (It sounds like a zero-sum game of pain transfer
>>>>>> to me.)
>>>>>>
>>>>>> I'm relatively unfamiliar with the arguments.  Can someone provide
>>>>>> a brief list of reasons this is needed, and how the inconvenience to Linux
>>>>>> kernel developers will be minimized, should it proceed?
>>>>>
>>>>
>>>>
>>>>> One of the reasons for doing devicetrees is to separate the hardware
>>>>> description from the code so that:
>>>>> - Other OSes (and bootloaders) can use the same description to start on
>>>>>  a given hardware
>>>>> - A generic Kernel can be started on any hardware
>>>>> - A hardware describes itself, makes itself more introspecitve so we can
>>>>>  go away from very specialized kernels
>>>>
>>>> Tim knows this ^^^^.  He was asking for the arguments for moving dts files
>>>> out of the linux kernel source tree.
>>>
>>> We've made the decision that devicetree bindings need to be treated as
>>> ABI, but as long as the .dts files live in the kernel there will
>>> always be the temptation to just tweak things in lock-step and nobody
>>> will notice. Splitting the files out gives that extra push to think
>>> about whether changes to a binding will backwards compatible with a
>>> tree that doesn't have those changes because the chances are a lot
>>> higher that someone will hit that combination.
>>>
>>> The other argument is shared source between
>>> BSD/U-Boot/Barebox/Linux/etc. Until we have a separate .dts repo there
>>> is no good way to share the database of hardware descriptions.
>>
>> We could provide an easy export (see below).  What do you think?
> 
> So what would the process be to get changes to those files upstream if you did this? It would make it marginally easier to USE, but once disconnected from the git world, a lot harder to track and fix.

Changes would be through the normal upstream project (the Linux kernel).  Just like when
the Linux kernel uses a driver from BSD, any changes to the upstream are done through
the BSD process.

> 
> Also, you should export the device tree docs too, since they are, in theory, a set.

Yes, absolutely.  And the kbuild docs need to be updated.  And ....  The patch was
just a proof of concept to show the scope of the changes that would be required and
that it was possible.

> And honestly, the device tree doc files need more work than the .dts and .dtsi files.
> 
> Warner
> 
>> -Frank
>>
>>
>>
>> Proof of concept: export devicetree source and header files
>>
>> Not-signed-off-by: Frank Rowand <frank.rowand-/MT0OVThwyLZJqsBc5GL+g@public.gmane.org>
>>
>>
>> ---
>> Makefile                   |   27 	26 +	1 -	0 !
>> scripts/Makefile.dtsexport |   19 	19 +	0 -	0 !
>> scripts/dts.sh             |   11 	11 +	0 -	0 !
>> 3 files changed, 56 insertions(+), 1 deletion(-)
>>
>> Index: b/Makefile
>> ===================================================================
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -234,6 +234,9 @@ endif
>> # Where to locate arch specific headers
>> hdr-arch  := $(SRCARCH)
>>
>> +# Where to locate arch specific dts
>> +dts-arch  := $(SRCARCH)
>> +
>> KCONFIG_CONFIG	?= .config
>> export KCONFIG_CONFIG
>>
>> @@ -463,7 +466,7 @@ version_h := include/generated/uapi/linu
>> no-dot-config-targets := clean mrproper distclean \
>> 			 cscope gtags TAGS tags help %docs check% coccicheck \
>> 			 $(version_h) headers_% archheaders archscripts \
>> -			 kernelversion %src-pkg
>> +			 kernelversion %src-pkg dts_export%
>>
>> config-targets := 0
>> mixed-targets  := 0
>> @@ -933,6 +936,26 @@ firmware_install: FORCE
>> 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
>>
>> # ---------------------------------------------------------------------------
>> +# devicetree source and headers export
>> +
>> +#Default location for installed headers
>> +export EXPORT_DTS_PATH = $(KBUILD_OUTPUT)/usr/dts
>> +
>> +PHONY += dts_export_headers
>> +dts_export_headers: scripts_basic
>> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=include/dt-bindings
>> +	@echo $(KERNELRELEASE) >$(EXPORT_DTS_PATH)/linux_version
>> +
>> +PHONY += dts_export_all
>> +dts_export_all: dts_export_headers
>> +	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/dts.sh
>> +
>> +PHONY += dts_export
>> +dts_export:
>> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.dtsexport obj=arch/$(dts-arch)/boot/dts
>> +
>> +
>> +# ---------------------------------------------------------------------------
>> # Kernel headers
>>
>> #Default location for installed headers
>> @@ -1160,6 +1183,8 @@ help:
>> 	@echo  '  modules_prepare - Set up for building external modules'
>> 	@echo  '  tags/TAGS	  - Generate tags file for editors'
>> 	@echo  '  cscope	  - Generate cscope index'
>> +	@echo  '  dts_export_all  - Export devicetree source and headers to EXPORT_DTS_PATH'
>> +	@echo  '                    (default: $(EXPORT_DTS_PATH))'
>> 	@echo  '  gtags           - Generate GNU GLOBAL index'
>> 	@echo  '  kernelrelease	  - Output the release version string'
>> 	@echo  '  kernelversion	  - Output the version stored in Makefile'
>> Index: b/scripts/Makefile.dtsexport
>> ===================================================================
>> --- /dev/null
>> +++ b/scripts/Makefile.dtsexport
>> @@ -0,0 +1,19 @@
>> +# ==========================================================================
>> +# Exporting dts source and header files
>> +#
>> +# ==========================================================================
>> +
>> +srcpath := $(srctree)/$(obj)/*
>> +dstpath := $(EXPORT_DTS_PATH)/$(obj)
>> +
>> +include scripts/Kbuild.include
>> +
>> +
>> +quiet_cmd_install = EXPORT $(subst $(srctree)/,,$(srcpath))
>> +      cmd_install = mkdir -p $(dstpath); cp -a $(srcpath) $(dstpath)
>> +
>> +
>> +.PHONY: export
>> +export:
>> +	$(call cmd,install)
>> +
>> Index: b/scripts/dts.sh
>> ===================================================================
>> --- /dev/null
>> +++ b/scripts/dts.sh
>> @@ -0,0 +1,11 @@
>> +#!/bin/sh
>> +# Run dts_export command for all architectures
>> +
>> +# Stop on error
>> +set -e
>> +
>> +for arch in $(ls ${srctree}/arch); do
>> +	if [ -d ${srctree}/arch/${arch}/boot/dts ]; then
>> +		make ARCH=${arch} dts_export
>> +	fi
>> +done
>> --
>> To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Fri, Feb 21, 2014 at 11:27:29AM -0700, Warner Losh wrote:
> 
> On Feb 20, 2014, at 4:38 AM, Grant Likely wrote:
> 
> > On Wed, 19 Feb 2014 13:20:15 -0800, Olof Johansson <olof-nZhT3qVonbNeoWH0uzbU5w@public.gmane.org> wrote:
> >> On Wed, Feb 19, 2014 at 1:12 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>> One way to minimize the inconvenience is keep versioning and dev
> >>> cycles in sync with the kernel. We could also start doing things to
> >>> align the kernel workflow with how things will work when we do have a
> >>> separate repository.
> >> 
> >> I don't think aligning development cycles is what we want most here it
> >> might be useful for us in Linux but it'll make things difficult for
> >> other projects since they're not aware of our release cycles. The
> >> device tree bindings and DT contents in that repo should be "always
> >> stable", i.e. no merge window / rc concept. As soon as something goes
> >> in it's live, and from then out only fixes to the DTS files (or
> >> appending the binding).
> >> 
> >> For example, I don't want to have to track two trees to test against
> >> -- I'll want to keep one repo of the very latest DT files and always
> >> use those to boot any and all boards.
> > 
> > This approach does have the subtle side effect that differs from what we
> > discussed in Edinburgh.  We've talked about always being able to boot a
> > new kernel on an old devicetree, but not a new devicetree on an old
> > kernel. With a separate board database repo we are going to hit both
> > cases. At least to a limited extent we're going to need older kernels
> > booting with the latest devicetree, and we'll need some rules about how
> > that gets applied.
> 
> I wasn't in Edinburgh...  Was this at the binary level or at the
> source level? I'm thinking specifically about the move to cpp in the
> back of my mind...

Only the binary compatibility matters. The source can change
without breaking compatibility between kernels and devicetrees.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Fri, 2014-02-21 at 15:11 +0100, Sascha Hauer wrote:
> Ians dts repository is a good start, but it contains a complete kernel
> history and this is not very suitable as a submodule for other
> projects. 

It only contains the full history for the files which it contains, not a
complete kernel history. This is deliberate so that "git annotate" etc
still works to tell you where a particular line came from.

The are a lot of merge NULL-commits which aren't strictly needed (they
have no content and only a single parent) but I didn't manage to get git
rewrite-branch to omit them. They are mostly harmless I think.

I'm not sure how any of that makes it unsuitable for use as a submodule
though, the history contained in a git tree seems pretty orthogonal to
that to me.

TBH, I'm not sure what the requirements for a submodule are -- IME when
something is used as a submodule it is up to the outer git tree to call
into the inner build system in the correct way, whatever that may be.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Mon, Feb 24, 2014 at 03:59:17PM +0000, Ian Campbell wrote:
> On Fri, 2014-02-21 at 15:11 +0100, Sascha Hauer wrote:
> > Ians dts repository is a good start, but it contains a complete kernel
> > history and this is not very suitable as a submodule for other
> > projects. 
> 
> It only contains the full history for the files which it contains, not a
> complete kernel history. This is deliberate so that "git annotate" etc
> still works to tell you where a particular line came from.

I have cloned git://xenbits.xen.org/people/ianc/device-tree-rebasing.git.
.git is 843MB in size and after a 'git checkout v3.13' I see a vanilla v3.13
checked out. I may have done something wrong, but I don't see what it
could be.

> 
> The are a lot of merge NULL-commits which aren't strictly needed (they
> have no content and only a single parent) but I didn't manage to get git
> rewrite-branch to omit them. They are mostly harmless I think.
> 
> I'm not sure how any of that makes it unsuitable for use as a submodule
> though, the history contained in a git tree seems pretty orthogonal to
> that to me.

My concerns are only about the size of the repository. Of cause if
you're willing to wait long enough that problem is orthogonal. But
anyway, what I cloned from your repository is likely not what it should
look like.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Tue, 2014-02-25 at 08:26 +0100, Sascha Hauer wrote:
> On Mon, Feb 24, 2014 at 03:59:17PM +0000, Ian Campbell wrote:
> > On Fri, 2014-02-21 at 15:11 +0100, Sascha Hauer wrote:
> > > Ians dts repository is a good start, but it contains a complete kernel
> > > history and this is not very suitable as a submodule for other
> > > projects. 
> > 
> > It only contains the full history for the files which it contains, not a
> > complete kernel history. This is deliberate so that "git annotate" etc
> > still works to tell you where a particular line came from.
> 
> I have cloned git://xenbits.xen.org/people/ianc/device-tree-rebasing.git.
> .git is 843MB in size and after a 'git checkout v3.13' I see a vanilla v3.13
> checked out. I may have done something wrong, but I don't see what it
> could be.

There is a branch with the full Linux stuff in there too. It is needed
in the tree doing the conversion but doesn't really need to be
published. I pushed it in the early days without really thinking about
the size impact. I'll remove that stuff from the published tree.

The master branch, which you should have got by default, contains the
converted stuff and the tags have -dts appended (e.g. checkout
v3.13-dts). Checkout one of those and you should see the expected thing.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-compiler" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Ian Campbell]

On Tue, 2014-02-25 at 07:36 +0000, Ian Campbell wrote:
> On Tue, 2014-02-25 at 08:26 +0100, Sascha Hauer wrote:
> > On Mon, Feb 24, 2014 at 03:59:17PM +0000, Ian Campbell wrote:
> > > On Fri, 2014-02-21 at 15:11 +0100, Sascha Hauer wrote:
> > > > Ians dts repository is a good start, but it contains a complete kernel
> > > > history and this is not very suitable as a submodule for other
> > > > projects. 
> > > 
> > > It only contains the full history for the files which it contains, not a
> > > complete kernel history. This is deliberate so that "git annotate" etc
> > > still works to tell you where a particular line came from.
> > 
> > I have cloned git://xenbits.xen.org/people/ianc/device-tree-rebasing.git.
> > .git is 843MB in size and after a 'git checkout v3.13' I see a vanilla v3.13
> > checked out. I may have done something wrong, but I don't see what it
> > could be.
> 
> There is a branch with the full Linux stuff in there too. It is needed
> in the tree doing the conversion but doesn't really need to be
> published. I pushed it in the early days without really thinking about
> the size impact. I'll remove that stuff from the published tree.

Done. The tree now has only a master branch and the vX.Y-dts tags (it is
still pushing the historical ones, they'll be there soon).

.git of a fresh clone is now 22M, which is more like it ;-)

I've moved the tree with all the conversion state aside into
device-tree-conversion.git. Most people won't need what is in there.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Wed, Feb 26, 2014 at 09:00:32AM +0000, Ian Campbell wrote:
> On Tue, 2014-02-25 at 07:36 +0000, Ian Campbell wrote:
> > On Tue, 2014-02-25 at 08:26 +0100, Sascha Hauer wrote:
> > > On Mon, Feb 24, 2014 at 03:59:17PM +0000, Ian Campbell wrote:
> > > > On Fri, 2014-02-21 at 15:11 +0100, Sascha Hauer wrote:
> > > > > Ians dts repository is a good start, but it contains a complete kernel
> > > > > history and this is not very suitable as a submodule for other
> > > > > projects. 
> > > > 
> > > > It only contains the full history for the files which it contains, not a
> > > > complete kernel history. This is deliberate so that "git annotate" etc
> > > > still works to tell you where a particular line came from.
> > > 
> > > I have cloned git://xenbits.xen.org/people/ianc/device-tree-rebasing.git.
> > > .git is 843MB in size and after a 'git checkout v3.13' I see a vanilla v3.13
> > > checked out. I may have done something wrong, but I don't see what it
> > > could be.
> > 
> > There is a branch with the full Linux stuff in there too. It is needed
> > in the tree doing the conversion but doesn't really need to be
> > published. I pushed it in the early days without really thinking about
> > the size impact. I'll remove that stuff from the published tree.
> 
> Done. The tree now has only a master branch and the vX.Y-dts tags (it is
> still pushing the historical ones, they'll be there soon).
> 
> .git of a fresh clone is now 22M, which is more like it ;-)

I just cloned it. This is much more managable now. Thanks for doing
this.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Sascha Hauer]

On Fri, Feb 21, 2014 at 10:47:09AM -0800, Olof Johansson wrote:
> On Fri, Feb 21, 2014 at 6:11 AM, Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org> wrote:
> > On Thu, Feb 20, 2014 at 05:07:12PM -0800, Frank Rowand wrote:
> 
> >> Why are the changes not submitted upstream?  (Or if they were submitted, why
> >> were they not accepted?)
> >
> > The descriptions where to find the environment are obviously barebox
> > specific and not acceptable upstream.
> 
> Why not? Wouldn't it be useful if you could find where the environment
> is from the running Linux image so that you can change variables
> without rebooting into Barebox first?

Indeed that would be very useful. Generally that would require some
generic binding for referring to partitions on mtd devices, partitions
on block devices and EEPROMs. If you think such a binding would have a
chance upstream I could try and come up with a binding.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: devicetree repository separation/migration

[Grant Likely]

On Fri, 28 Feb 2014 09:00:25 +0100, Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org> wrote:
> On Fri, Feb 21, 2014 at 10:47:09AM -0800, Olof Johansson wrote:
> > On Fri, Feb 21, 2014 at 6:11 AM, Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org> wrote:
> > > On Thu, Feb 20, 2014 at 05:07:12PM -0800, Frank Rowand wrote:
> > 
> > >> Why are the changes not submitted upstream?  (Or if they were submitted, why
> > >> were they not accepted?)
> > >
> > > The descriptions where to find the environment are obviously barebox
> > > specific and not acceptable upstream.
> > 
> > Why not? Wouldn't it be useful if you could find where the environment
> > is from the running Linux image so that you can change variables
> > without rebooting into Barebox first?
> 
> Indeed that would be very useful. Generally that would require some
> generic binding for referring to partitions on mtd devices, partitions
> on block devices and EEPROMs. If you think such a binding would have a
> chance upstream I could try and come up with a binding.

I agree. I want to explore doing the same thing for UEFI also. Currently
the UEFI spec pretty much requires UEFI runtime services to 'own' the
device that holds variable storage, but that doesn't work on
cost-sensitive devices that can't afford separate flash dedicated to
UEFI. I'd like to investigate a spec extension that defines the variable
storage format on a block device so that Linux can directly manipulate
it without a runtime services call.

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html