Arm device tree wonder

Arm device tree wonder

[Pierre Beaumon]

>From I understanding the goal of the FDT, is to limit board code for each soc and simplify machine code.

=== info provided by FDT ===

We describe the mapping of controllers on the soc (irq, memory map). But this info won't change for a given SOC.
What's the point of describing the interrupt controller, timer on FDT ?
Do we really expect to support new SOC without any kernel modification ?
If not why not include for once this info in the kernel and avoid to duplicate it for each board ?

Also most controller aren't stable across SOC version :
- there are bugs in hardware (for example omap3 errata is 100 pages) and new revisions correct them
- new revisions add new features

How the driver is aware of the revision on the controller. Most of the time hardware guys are lazy and don't provide this info in the controller (no version register). The driver need to deduce it from the SOC version/revision (with dynamic detection).

FDT doesn't seem to provide this info and shouldn't (what if a board got different SOC revisions).
This means there still a link between driver and machine code for hardware probing.

=== in or out kernel tree FDT ? ===

Also when the driver is improved (support new feature, ...), how new configuration should be provided to FDT ?
Can we break the FDT API or should we provide only new property ?

If we want to support out of kernel FDT, I think we can't break the API.

But what happen if a board/SOC vendor provide a board with a crappy FDT mapping, we can rewrite it and do new mapping in mainline kernel.
But that means the vendor FDT is not supported anymore, and we can't boot new kernel on this board without overriding the vendor FDT.
This is the same problem with acpi table. In theory we could rewrite them, but in practice we add workaround in the interpreter.

An alternative should be to always include the FDT with the kernel.
But in this case :
- we don't solve unique board id problem
- we need tags in order bootloader pass limited information
- we can't boot new board of generic kernel
- FDT is only an abstraction for board/machine code that is provided by platform driver/device.

=== flexibility of FDT ===

The current code use platform data to configure driver (sdcard bus width, reset pin, speed info, ...).
This is easy to convert it to FDT (textual conversion).

But in the current board code conditional can be used. According to the board revision (gpio, fuse, rev atag), different platform data can be provided. Same apply for static external device (i2c, spi, ...).
How can this be done with FDT ?
Can FDT support conditional stuff ?
Should we have a FDT per revision, should the bootloader patch the FDT, should the kernel select/patch the FDT ?

If we have a FDT per revision, I don't believe it is acceptable to be able to flash the FDT with the correct revision.
Production chain can be complex and making the flashing stage detect the revision and flashing the correct FDT can't be easy to do (tools they use can be basic, ...).

If I do a generic module (ie a daughter board containing the SOC/pmu) that can be connected to different motherboards.
The size of memory can change, the external devices can change (spi, usb phy present or not, lcd, ...).
The current model can allow to do a generic board code driven by gpio/fuse to do the motherboard configuration. And the customer (that integrate module on their motherboard) doesn't need any kernel modification.

How this will work with FDT ?

Some platform data need callback. How this will work with FDT ?

Sometime when doing a board, we discover too late board bug that can be workaround in the board code.
How this will work with FDT ?
Should be do like in x86 with quirk in kernel according to dmi indentifier or have machine code somewhere ?

The kernel FDT can help to unify how platform declare controller resource, but I wonder if it will be enough. It could be interesting to have a soc_bus (or even emulate a pci_bus) that is look like the pci_bus.
The advantage of the pci_bus is that it is widely used on most architecture.
Also it provide interesting functions that are not provided by the platform_bus :
- vendor/device id/revision
- pci_(enable|disable)_device (were the clock could be manipulated)
- pci_prepare_to_sleep/pci_back_from_sleep (for suspend/resume stuff)

====

Arm is different of other architecture (sparc/powerpc) where FDT is used because there are lot's of Soc with different controllers. Nearly all vendor got its own uart, i2c, ...
I am waiting to see how it will work on arm.

But please don't forget arm isn't only used on laptop/netbook that want generic kernel.
It is used for a wide variety of applications. These applications may not be found in the mainline kernel, but that don't mean nobody use them.

Also I believe one big arm problem is a lack of common infrastructure that lead to code duplication.
Some machine try to do too much things. omap2 (mach-omap2 + plat-omap) is about 80 KSLOC over 400KSLOC (76 mach + plat directories). That 8 times bigger than the average ((80/2)/(400/76))! There are either common stuff that should be shared, or some stuff should be moved in drivers.

Also the directory organization of mach-*, plat-* is quite complex.
Why not put a top level directory with the soc family name, and then put in that directory mach-*/plat-* directories.

Re: Arm device tree wonder

[Scott Wood]

On 08/30/2011 03:09 PM, Pierre Beaumon wrote:
> From I understanding the goal of the FDT, is to limit board code for each soc and simplify machine code.
> 
> === info provided by FDT ===
> 
> We describe the mapping of controllers on the soc (irq, memory map). But this info won't change for a given SOC.
> What's the point of describing the interrupt controller, timer on FDT ?
> Do we really expect to support new SOC without any kernel modification ?

Depending on how closely it resembles a previously supported SoC, the
changes might be minimal.

> If not why not include for once this info in the kernel and avoid to duplicate it for each board ?

DTC recently grew the ability to do overlay trees, which in combination
with include files lets you factor out the SoC stuff into its own file.

For example, see arch/powerpc/boot/dts/p4080si.dtsi and
arch/powerpc/boot/dts/p4080ds.dts

> Also most controller aren't stable across SOC version :
> - there are bugs in hardware (for example omap3 errata is 100 pages) and new revisions correct them
> - new revisions add new features
> 
> How the driver is aware of the revision on the controller. Most of
> the time hardware guys are lazy and don't provide this info in the
> controller (no version register). The driver need to deduce it from
> the SOC version/revision (with dynamic detection).

You can still use the SOC version info.  You could also encode whatever
information is relevant into the device's compatible string, or other
properties.

> FDT doesn't seem to provide this info and shouldn't (what if a board got different SOC revisions).

On Freescale powerpc boards we usually patch up things like this from
U-Boot, though this has some drawbacks.  It could also be done from
early Linux platform code, or some update program that sits between
firmware and Linux (such as a U-Boot standalone application).  We use
these updates for board-level things as well, such as when certain
devices are enabled/disabled by jumper or similar config.

It's also possible to have the driver just find out about the SOC
revision from elsewhere, though this can be awkward in a virtualized
environment if you need to mix emulated devices that belong to different
SOC types.

> This means there still a link between driver and machine code for hardware probing.
> 
> === in or out kernel tree FDT ? ===
> 
> Also when the driver is improved (support new feature, ...), how new
> configuration should be provided to FDT ? Can we break the FDT API or
> should we provide only new property ?

The FDT is not a driver configuration medium.  It is supposed to just
describe the hardware.

> If we want to support out of kernel FDT, I think we can't break the API.
> 
> But what happen if a board/SOC vendor provide a board with a crappy
> FDT mapping, we can rewrite it and do new mapping in mainline
> kernel. But that means the vendor FDT is not supported anymore, and
> we can't boot new kernel on this board without overriding the vendor
> FDT.

In general, if you're relying on a binding that hasn't been accepted
into mainline Linux (or possibly some other designated binding
repository, such as devicetree.org), you're at risk of breakage.  Same
as with other APIs.

You can usually have the code support both the new and old bindings, though.

> Sometime when doing a board, we discover too late board bug that can
> be workaround in the board code. How this will work with FDT ? Should
> be do like in x86 with quirk in kernel according to dmi indentifier
> or have machine code somewhere ?

On powerpc we still have board code (identified by string, not
enumeration).  Even if we eventually develop the ability to boot on an
unrecognized board purely based on the other things in the device tree
(provided all the needed support is actually there), the ability to
match the toplevel compatible when special handling is needed would
still be there.

> Arm is different of other architecture (sparc/powerpc) where FDT is
> used because there are lot's of Soc with different controllers.
> Nearly all vendor got its own uart, i2c, ... I am waiting to see how
> it will work on arm.

How is this different from powerpc?

> But please don't forget arm isn't only used on laptop/netbook that
> want generic kernel. It is used for a wide variety of applications.
> These applications may not be found in the mainline kernel, but that
> don't mean nobody use them.

This can also be said of powerpc.

> Also I believe one big arm problem is a lack of common infrastructure
> that lead to code duplication. Some machine try to do too much
> things. omap2 (mach-omap2 + plat-omap) is about 80 KSLOC over
> 400KSLOC (76 mach + plat directories). That 8 times bigger than the
> average ((80/2)/(400/76))! There are either common stuff that should
> be shared, or some stuff should be moved in drivers.
> 
> Also the directory organization of mach-*, plat-* is quite complex. 
> Why not put a top level directory with the soc family name, and then
> put in that directory mach-*/plat-* directories.

Why have such a rigid structure at all, versus just having various bits
of code that support various things, glued together however the device
tree and/or board code specify?

-Scott

Arm device tree wonder

[Scott Wood]

On 08/30/2011 03:09 PM, Pierre Beaumon wrote:
> From I understanding the goal of the FDT, is to limit board code for each soc and simplify machine code.
> 
> === info provided by FDT ===
> 
> We describe the mapping of controllers on the soc (irq, memory map). But this info won't change for a given SOC.
> What's the point of describing the interrupt controller, timer on FDT ?
> Do we really expect to support new SOC without any kernel modification ?

Depending on how closely it resembles a previously supported SoC, the
changes might be minimal.

> If not why not include for once this info in the kernel and avoid to duplicate it for each board ?

DTC recently grew the ability to do overlay trees, which in combination
with include files lets you factor out the SoC stuff into its own file.

For example, see arch/powerpc/boot/dts/p4080si.dtsi and
arch/powerpc/boot/dts/p4080ds.dts

> Also most controller aren't stable across SOC version :
> - there are bugs in hardware (for example omap3 errata is 100 pages) and new revisions correct them
> - new revisions add new features
> 
> How the driver is aware of the revision on the controller. Most of
> the time hardware guys are lazy and don't provide this info in the
> controller (no version register). The driver need to deduce it from
> the SOC version/revision (with dynamic detection).

You can still use the SOC version info.  You could also encode whatever
information is relevant into the device's compatible string, or other
properties.

> FDT doesn't seem to provide this info and shouldn't (what if a board got different SOC revisions).

On Freescale powerpc boards we usually patch up things like this from
U-Boot, though this has some drawbacks.  It could also be done from
early Linux platform code, or some update program that sits between
firmware and Linux (such as a U-Boot standalone application).  We use
these updates for board-level things as well, such as when certain
devices are enabled/disabled by jumper or similar config.

It's also possible to have the driver just find out about the SOC
revision from elsewhere, though this can be awkward in a virtualized
environment if you need to mix emulated devices that belong to different
SOC types.

> This means there still a link between driver and machine code for hardware probing.
> 
> === in or out kernel tree FDT ? ===
> 
> Also when the driver is improved (support new feature, ...), how new
> configuration should be provided to FDT ? Can we break the FDT API or
> should we provide only new property ?

The FDT is not a driver configuration medium.  It is supposed to just
describe the hardware.

> If we want to support out of kernel FDT, I think we can't break the API.
> 
> But what happen if a board/SOC vendor provide a board with a crappy
> FDT mapping, we can rewrite it and do new mapping in mainline
> kernel. But that means the vendor FDT is not supported anymore, and
> we can't boot new kernel on this board without overriding the vendor
> FDT.

In general, if you're relying on a binding that hasn't been accepted
into mainline Linux (or possibly some other designated binding
repository, such as devicetree.org), you're at risk of breakage.  Same
as with other APIs.

You can usually have the code support both the new and old bindings, though.

> Sometime when doing a board, we discover too late board bug that can
> be workaround in the board code. How this will work with FDT ? Should
> be do like in x86 with quirk in kernel according to dmi indentifier
> or have machine code somewhere ?

On powerpc we still have board code (identified by string, not
enumeration).  Even if we eventually develop the ability to boot on an
unrecognized board purely based on the other things in the device tree
(provided all the needed support is actually there), the ability to
match the toplevel compatible when special handling is needed would
still be there.

> Arm is different of other architecture (sparc/powerpc) where FDT is
> used because there are lot's of Soc with different controllers.
> Nearly all vendor got its own uart, i2c, ... I am waiting to see how
> it will work on arm.

How is this different from powerpc?

> But please don't forget arm isn't only used on laptop/netbook that
> want generic kernel. It is used for a wide variety of applications.
> These applications may not be found in the mainline kernel, but that
> don't mean nobody use them.

This can also be said of powerpc.

> Also I believe one big arm problem is a lack of common infrastructure
> that lead to code duplication. Some machine try to do too much
> things. omap2 (mach-omap2 + plat-omap) is about 80 KSLOC over
> 400KSLOC (76 mach + plat directories). That 8 times bigger than the
> average ((80/2)/(400/76))! There are either common stuff that should
> be shared, or some stuff should be moved in drivers.
> 
> Also the directory organization of mach-*, plat-* is quite complex. 
> Why not put a top level directory with the soc family name, and then
> put in that directory mach-*/plat-* directories.

Why have such a rigid structure at all, versus just having various bits
of code that support various things, glued together however the device
tree and/or board code specify?

-Scott

Re: Arm device tree wonder

[Paul Walmsley]

On Tue, 30 Aug 2011, Pierre Beaumon wrote:

> Also I believe one big arm problem is a lack of common infrastructure 
> that lead to code duplication. Some machine try to do too much things. 
> omap2 (mach-omap2 + plat-omap) is about 80 KSLOC over 400KSLOC (76 mach 
> + plat directories). That 8 times bigger than the average 
> ((80/2)/(400/76))! There are either common stuff that should be shared, 
> or some stuff should be moved in drivers.

Here are some additional data points you might consider adding to your 
analysis:

- Consider taking into account what percentage of those lines are for 
  SoC-specific device data files.  Where should those files be moved, 
  before a DT (or other data format) conversion?

- Consider comparing the number of on-chip devices supported by the core 
  OMAP code to the number of devices supported by the average SoC core 
  code, in mainline.

- Consider accounting for the extra code and data needed for fine-grained 
  power management, compared to the average SoC core code, assuming the
  SoC supports it.

- Consider taking into account the number of OMAP variants supported by 
  the core OMAP code (at least thirty, not counting silicon revision 
  differences, which can be significant) compared to the average SoC.

- Consider comparing the number of boards supported by the core OMAP code
  to the number supported by the average SoC code.  To what other shared
  directory or drivers/ subdirectory should these files be moved to,
  before a DT conversion?

To be sure, there's a fair amount of OMAP core code that should be moved 
out to drivers.  My guess is about 10-15% of the current total.  Work 
along those lines is proceeding.  If you're offering to help, we could use 
it.

But I don't think that the way to deal with the remaining 85-90% of the 
core code and data is quite as simple as your rough summary suggests.


- Paul

Arm device tree wonder

[Paul Walmsley]

On Tue, 30 Aug 2011, Pierre Beaumon wrote:

> Also I believe one big arm problem is a lack of common infrastructure 
> that lead to code duplication. Some machine try to do too much things. 
> omap2 (mach-omap2 + plat-omap) is about 80 KSLOC over 400KSLOC (76 mach 
> + plat directories). That 8 times bigger than the average 
> ((80/2)/(400/76))! There are either common stuff that should be shared, 
> or some stuff should be moved in drivers.

Here are some additional data points you might consider adding to your 
analysis:

- Consider taking into account what percentage of those lines are for 
  SoC-specific device data files.  Where should those files be moved, 
  before a DT (or other data format) conversion?

- Consider comparing the number of on-chip devices supported by the core 
  OMAP code to the number of devices supported by the average SoC core 
  code, in mainline.

- Consider accounting for the extra code and data needed for fine-grained 
  power management, compared to the average SoC core code, assuming the
  SoC supports it.

- Consider taking into account the number of OMAP variants supported by 
  the core OMAP code (at least thirty, not counting silicon revision 
  differences, which can be significant) compared to the average SoC.

- Consider comparing the number of boards supported by the core OMAP code
  to the number supported by the average SoC code.  To what other shared
  directory or drivers/ subdirectory should these files be moved to,
  before a DT conversion?

To be sure, there's a fair amount of OMAP core code that should be moved 
out to drivers.  My guess is about 10-15% of the current total.  Work 
along those lines is proceeding.  If you're offering to help, we could use 
it.

But I don't think that the way to deal with the remaining 85-90% of the 
core code and data is quite as simple as your rough summary suggests.


- Paul

Re : Arm device tree wonder

[Pierre Beaumon]

Hello Paul,

thanks for your reply.

> On Tue, 30 Aug 2011, Pierre Beaumon wrote:
> 
> > Also I believe one big arm problem is a lack of common
> > infrastructure that lead to code duplication. Some machine try to
> > do too much things. omap2 (mach-omap2 + plat-omap) is about 80
> > KSLOC over 400KSLOC (76 mach 
> > + plat directories). That 8 times bigger than the average 
> > ((80/2)/(400/76))! There are either common stuff that should be
> > shared, or some stuff should be moved in drivers.
> 
> Here are some additional data points you might consider adding to
> your analysis:
> 
> - Consider taking into account what percentage of those lines are for 
>   SoC-specific device data files.  Where should those files be moved, 
>   before a DT (or other data format) conversion?
Soc specific data are mainly in file starting with board* on omap and take only 15 KSLOC.

Most of code come from  omap_device, clock, powerdomain, hwmod, mux code (in mach-omap2 more than 30KSLOC for clock* omap_hwmod* mux*)
This code won't be removed when we switch to DT, and I believe this code can be shared with others.

For each driver/framework in bsp (plat-*/mach-*) it could be interesting to have an answer to the following questions : 
- what it does
- why it is specific to this SOC
- can it be usefull for other SOC
- can an existing framework be used/enhanced

For example the mux code it a common problem to all SOC and is not specific to omap. Having a generic framework could be great (pinmux ?) 


hwmod [1] doesn't look like specific to OMAP and can apply to any arm SOC. Why should it be in omap code ?
Platform_bus is limited (see my example on pci like bus) and arm may need a better bus. Why not omap_device/hwmod ?

This make me think of the time when linux didn't get a wifi stack and each driver came with its own stack.


> - Consider accounting for the extra code and data needed for
> fine-grained power management, compared to the average SoC core code,
> assuming the SoC supports it.
Yes omap is fine tuned for power management, but the power management concept is not limited to omap and can be shared.
See for example the pm_runtime framework that can be used by everybody (pc, arm, ...).



[1]
 * One way to view an OMAP SoC is as a collection of largely unrelated
 * IP blocks connected by interconnects.  The IP blocks include
 * devices such as ARM processors, audio serial interfaces, UARTs,
 * etc.  Some of these devices, like the DSP, are created by TI;
 * others, like the SGX, largely originate from external vendors.  In
 * TI's documentation, on-chip devices are referred to as "OMAP
 * modules."  Some of these IP blocks are identical across several
 * OMAP versions.  Others are revised frequently.
[...]
* OMAP hwmod provides a consistent way to describe the on-chip
 * hardware blocks and their integration into the rest of the chip.
 * This description can be automatically generated from the TI
 * hardware database.  OMAP hwmod provides a standard, consistent API
 * to reset, enable, idle, and disable these hardware blocks.  And
 * hwmod provides a way for other core code, such as the Linux device 
 * code or the OMAP power management and address space mapping code,
 * code or the OMAP power management and add
ress space mapping code,
 * to query the hardware database.

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html