All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Gibson <david@gibson.dropbear.id.au>
To: u-boot@lists.denx.de
Subject: [U-Boot] [PATCH v3 00/12] cmd: fdt: Add device tree overlays support
Date: Mon, 27 Jun 2016 01:37:19 +1000	[thread overview]
Message-ID: <20160626153719.GP15625@voom.fritz.box> (raw)
In-Reply-To: <20160624142757.32735-1-maxime.ripard@free-electrons.com>

On Fri, Jun 24, 2016 at 04:27:45PM +0200, Maxime Ripard wrote:
> Hi,
> 
> The device tree overlays are a great solution to the issue raised by
> the bunch expandable boards we find everywhere these days, like the
> Beaglebone, Raspberry Pi or CHIP.
> 
> However, most of the time, the overlays are applied through a
> mechanism involving the firmware request interface in Linux, that is
> only fully functional once the userspace has been mounted and is
> running.
> 
> Some expansion boards might need to be enabled before that, because
> they simply need to patch the DT early on, or need to be initialized
> early in order to be fully functional, or because they provide access
> to the root filesystem.
> 
> In these cases, having the bootloader applying the overlay before
> Linux starts seems like the easiest solution.
> 
> This implementation doesn't provide all the Linux fancyness though,
> there's no transactional application, which means that if the overlay
> cannot be applied for a reason while you're still halfway through the
> application, you're probably screwed. It also cannot remove an
> overlay, but I don't think that it is currently a use-case.
> 
> There's still a bunch of work to extend the libfdt unit tests to test
> the new functions introduced, but these patches will be submitted
> in a near future.

Are you planning to send a new version of (the relevant portions of)
these against upstream libfdt?

> 
> Let me know what you think,
> Maxime
> 
> Changes from v2:
>   - Add Kconfig option for the libfdt overlay support
>   - Reworked the code to deal with Pantelis and David numerous
>     comments, among which:
>     * Remove the need for malloc in the overlay code, and added some
>       libfdt functions to do that
>     * Remove the DT magic in case of an error to not be able to use it
>       anymore
>     * Removed the fdt_ and _ function prefix for the static functions
>     * Plus the usual bunch of rework, error checking and optimizations.
> 
>   - Added new tests to deal with bugs reported by David (the overlay
>     was not applying when you add a subnode declared that was already
>     in the base device tree, and using a local phandle was only
>     working if the property storing it only had a length of 4).
> 
> Changes from v1:
>   - Moved the overlay code to libfdt
>   - Added unit tests
>   - Refactored the code to reduce the amount of memory allocation
>   - No longer modify the overlay itself, but create a copy to operate
>     on instead.
>   - Removed the limitations on the fixups path, names and properties
>     length
>   - Fixed a few things here and there according to comments		    
> 
> Maxime Ripard (12):
>   cmd: fdt: Narrow the check for fdt addr
>   scripts: Makefile.lib: Sanitize DTB names
>   vsprintf: Include stdarg for va_list
>   libfdt: Add new headers and defines
>   libfdt: Add iterator over properties
>   libfdt: Add max phandle retrieval function
>   libfdt: Add fdt_setprop_inplace_by_index
>   libfdt: Add fdt_path_offset_namelen
>   libfdt: Add fdt_getprop_namelen_w
>   libfdt: Add overlay application function
>   cmd: fdt: add fdt overlay application subcommand
>   tests: Introduce DT overlay tests
> 
>  Makefile                          |   1 +
>  cmd/fdt.c                         |  26 ++-
>  include/libfdt.h                  | 124 ++++++++++++-
>  include/libfdt_env.h              |   6 +
>  include/test/overlay.h            |  16 ++
>  include/test/suites.h             |   1 +
>  include/vsprintf.h                |   2 +
>  lib/Kconfig                       |   5 +
>  lib/libfdt/Makefile               |   2 +
>  lib/libfdt/fdt_overlay.c          | 381 ++++++++++++++++++++++++++++++++++++++
>  lib/libfdt/fdt_ro.c               |  44 ++++-
>  lib/libfdt/fdt_wip.c              |  13 +-
>  scripts/Makefile.lib              |   8 +-
>  test/Kconfig                      |   1 +
>  test/cmd_ut.c                     |   6 +
>  test/overlay/Kconfig              |  11 ++
>  test/overlay/Makefile             |  15 ++
>  test/overlay/cmd_ut_overlay.c     | 243 ++++++++++++++++++++++++
>  test/overlay/test-fdt-base.dts    |  21 +++
>  test/overlay/test-fdt-overlay.dts |  88 +++++++++
>  20 files changed, 992 insertions(+), 22 deletions(-)
>  create mode 100644 include/test/overlay.h
>  create mode 100644 lib/libfdt/fdt_overlay.c
>  create mode 100644 test/overlay/Kconfig
>  create mode 100644 test/overlay/Makefile
>  create mode 100644 test/overlay/cmd_ut_overlay.c
>  create mode 100644 test/overlay/test-fdt-base.dts
>  create mode 100644 test/overlay/test-fdt-overlay.dts
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160627/7ab7960b/attachment.sig>

WARNING: multiple messages have this Message-ID (diff)
From: David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>
To: Maxime Ripard
	<maxime.ripard-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
Cc: "Pantelis Antoniou"
	<pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w@public.gmane.org>,
	"Simon Glass" <sjg-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>,
	"Boris Brezillon"
	<boris.brezillon-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>,
	"Alexander Kaplan" <alex-MflLfwwFzuz+yO7R74ARew@public.gmane.org>,
	"Thomas Petazzoni"
	<thomas.petazzoni-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>,
	devicetree-compiler-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	"Antoine Ténart"
	<antoine.tenart-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>,
	"Hans de Goede"
	<hdegoede-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	"Tom Rini" <trini-OWPKS81ov/FWk0Htik3J/w@public.gmane.org>,
	u-boot-0aAXYlwwYIKGBzrmiIFOJg@public.gmane.org,
	"Stefan Agner" <stefan-XLVq0VzYD2Y@public.gmane.org>
Subject: Re: [PATCH v3 00/12] cmd: fdt: Add device tree overlays support
Date: Mon, 27 Jun 2016 01:37:19 +1000	[thread overview]
Message-ID: <20160626153719.GP15625@voom.fritz.box> (raw)
In-Reply-To: <20160624142757.32735-1-maxime.ripard-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>

[-- Attachment #1: Type: text/plain, Size: 4903 bytes --]

On Fri, Jun 24, 2016 at 04:27:45PM +0200, Maxime Ripard wrote:
> Hi,
> 
> The device tree overlays are a great solution to the issue raised by
> the bunch expandable boards we find everywhere these days, like the
> Beaglebone, Raspberry Pi or CHIP.
> 
> However, most of the time, the overlays are applied through a
> mechanism involving the firmware request interface in Linux, that is
> only fully functional once the userspace has been mounted and is
> running.
> 
> Some expansion boards might need to be enabled before that, because
> they simply need to patch the DT early on, or need to be initialized
> early in order to be fully functional, or because they provide access
> to the root filesystem.
> 
> In these cases, having the bootloader applying the overlay before
> Linux starts seems like the easiest solution.
> 
> This implementation doesn't provide all the Linux fancyness though,
> there's no transactional application, which means that if the overlay
> cannot be applied for a reason while you're still halfway through the
> application, you're probably screwed. It also cannot remove an
> overlay, but I don't think that it is currently a use-case.
> 
> There's still a bunch of work to extend the libfdt unit tests to test
> the new functions introduced, but these patches will be submitted
> in a near future.

Are you planning to send a new version of (the relevant portions of)
these against upstream libfdt?

> 
> Let me know what you think,
> Maxime
> 
> Changes from v2:
>   - Add Kconfig option for the libfdt overlay support
>   - Reworked the code to deal with Pantelis and David numerous
>     comments, among which:
>     * Remove the need for malloc in the overlay code, and added some
>       libfdt functions to do that
>     * Remove the DT magic in case of an error to not be able to use it
>       anymore
>     * Removed the fdt_ and _ function prefix for the static functions
>     * Plus the usual bunch of rework, error checking and optimizations.
> 
>   - Added new tests to deal with bugs reported by David (the overlay
>     was not applying when you add a subnode declared that was already
>     in the base device tree, and using a local phandle was only
>     working if the property storing it only had a length of 4).
> 
> Changes from v1:
>   - Moved the overlay code to libfdt
>   - Added unit tests
>   - Refactored the code to reduce the amount of memory allocation
>   - No longer modify the overlay itself, but create a copy to operate
>     on instead.
>   - Removed the limitations on the fixups path, names and properties
>     length
>   - Fixed a few things here and there according to comments		    
> 
> Maxime Ripard (12):
>   cmd: fdt: Narrow the check for fdt addr
>   scripts: Makefile.lib: Sanitize DTB names
>   vsprintf: Include stdarg for va_list
>   libfdt: Add new headers and defines
>   libfdt: Add iterator over properties
>   libfdt: Add max phandle retrieval function
>   libfdt: Add fdt_setprop_inplace_by_index
>   libfdt: Add fdt_path_offset_namelen
>   libfdt: Add fdt_getprop_namelen_w
>   libfdt: Add overlay application function
>   cmd: fdt: add fdt overlay application subcommand
>   tests: Introduce DT overlay tests
> 
>  Makefile                          |   1 +
>  cmd/fdt.c                         |  26 ++-
>  include/libfdt.h                  | 124 ++++++++++++-
>  include/libfdt_env.h              |   6 +
>  include/test/overlay.h            |  16 ++
>  include/test/suites.h             |   1 +
>  include/vsprintf.h                |   2 +
>  lib/Kconfig                       |   5 +
>  lib/libfdt/Makefile               |   2 +
>  lib/libfdt/fdt_overlay.c          | 381 ++++++++++++++++++++++++++++++++++++++
>  lib/libfdt/fdt_ro.c               |  44 ++++-
>  lib/libfdt/fdt_wip.c              |  13 +-
>  scripts/Makefile.lib              |   8 +-
>  test/Kconfig                      |   1 +
>  test/cmd_ut.c                     |   6 +
>  test/overlay/Kconfig              |  11 ++
>  test/overlay/Makefile             |  15 ++
>  test/overlay/cmd_ut_overlay.c     | 243 ++++++++++++++++++++++++
>  test/overlay/test-fdt-base.dts    |  21 +++
>  test/overlay/test-fdt-overlay.dts |  88 +++++++++
>  20 files changed, 992 insertions(+), 22 deletions(-)
>  create mode 100644 include/test/overlay.h
>  create mode 100644 lib/libfdt/fdt_overlay.c
>  create mode 100644 test/overlay/Kconfig
>  create mode 100644 test/overlay/Makefile
>  create mode 100644 test/overlay/cmd_ut_overlay.c
>  create mode 100644 test/overlay/test-fdt-base.dts
>  create mode 100644 test/overlay/test-fdt-overlay.dts
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

  parent reply	other threads:[~2016-06-26 15:37 UTC|newest]

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-06-24 14:27 [U-Boot] [PATCH v3 00/12] cmd: fdt: Add device tree overlays support Maxime Ripard
2016-06-24 14:27 ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 01/12] cmd: fdt: Narrow the check for fdt addr Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 02/12] scripts: Makefile.lib: Sanitize DTB names Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 03/12] vsprintf: Include stdarg for va_list Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 04/12] libfdt: Add new headers and defines Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-26 15:39   ` [U-Boot] " David Gibson
2016-06-26 15:39     ` David Gibson
2016-06-27  7:25     ` [U-Boot] " Maxime Ripard
2016-06-27  7:25       ` Maxime Ripard
2016-06-28  3:02       ` [U-Boot] " David Gibson
2016-06-28  3:02         ` David Gibson
2016-06-24 14:27 ` [U-Boot] [PATCH v3 05/12] libfdt: Add iterator over properties Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-24 14:27 ` [U-Boot] [PATCH v3 06/12] libfdt: Add max phandle retrieval function Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-24 14:27 ` [U-Boot] [PATCH v3 07/12] libfdt: Add fdt_setprop_inplace_by_index Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-26  8:09     ` [U-Boot] " Maxime Ripard
2016-06-26  8:09       ` Maxime Ripard
2016-06-26 15:45   ` [U-Boot] " David Gibson
2016-06-26 15:45     ` David Gibson
2016-06-27  9:16     ` [U-Boot] " Maxime Ripard
2016-06-27  9:16       ` Maxime Ripard
2016-06-28  3:05       ` [U-Boot] " David Gibson
2016-06-28  3:05         ` David Gibson
2016-06-24 14:27 ` [U-Boot] [PATCH v3 08/12] libfdt: Add fdt_path_offset_namelen Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-26  8:10     ` [U-Boot] " Maxime Ripard
2016-06-26  8:10       ` Maxime Ripard
2016-06-29  3:27       ` [U-Boot] " Simon Glass
2016-06-29  3:27         ` Simon Glass
2016-06-26 15:47   ` [U-Boot] " David Gibson
2016-06-26 15:47     ` David Gibson
2016-06-24 14:27 ` [U-Boot] [PATCH v3 09/12] libfdt: Add fdt_getprop_namelen_w Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-26  8:11     ` [U-Boot] " Maxime Ripard
2016-06-26  8:11       ` Maxime Ripard
2016-06-26 15:49   ` [U-Boot] " David Gibson
2016-06-26 15:49     ` David Gibson
2016-06-27  7:29     ` [U-Boot] " Maxime Ripard
2016-06-27  7:29       ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 10/12] libfdt: Add overlay application function Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-26  2:53   ` [U-Boot] " Simon Glass
2016-06-26  2:53     ` Simon Glass
2016-06-27  7:13     ` [U-Boot] " Maxime Ripard
2016-06-27  7:13       ` Maxime Ripard
2016-06-27  5:26   ` [U-Boot] " David Gibson
2016-06-27  5:26     ` David Gibson
2016-06-27 11:40     ` [U-Boot] " Maxime Ripard
2016-06-27 11:40       ` Maxime Ripard
2016-06-28  3:12       ` [U-Boot] " David Gibson
2016-06-28  3:12         ` David Gibson
2016-06-30  2:34         ` [U-Boot] " Frank Rowand
2016-06-30  2:34           ` Frank Rowand
2016-06-30  5:08           ` [U-Boot] " David Gibson
2016-06-30  5:08             ` David Gibson
2016-06-24 14:27 ` [U-Boot] [PATCH v3 11/12] cmd: fdt: add fdt overlay application subcommand Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-24 14:27 ` [U-Boot] [PATCH v3 12/12] tests: Introduce DT overlay tests Maxime Ripard
2016-06-24 14:27   ` Maxime Ripard
2016-06-27  5:26   ` [U-Boot] " David Gibson
2016-06-27  5:26     ` David Gibson
2016-06-27  6:30     ` [U-Boot] " Maxime Ripard
2016-06-27  6:30       ` Maxime Ripard
2016-06-27  6:47       ` [U-Boot] " David Gibson
2016-06-27  6:47         ` David Gibson
2016-06-26 15:37 ` David Gibson [this message]
2016-06-26 15:37   ` [PATCH v3 00/12] cmd: fdt: Add device tree overlays support David Gibson
2016-06-27  6:29   ` [U-Boot] " Maxime Ripard
2016-06-27  6:29     ` Maxime Ripard
2016-06-29  3:28     ` [U-Boot] " Simon Glass
2016-06-29  3:28       ` Simon Glass

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160626153719.GP15625@voom.fritz.box \
    --to=david@gibson.dropbear.id.au \
    --cc=u-boot@lists.denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.