[OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[Petr Kubizňák]

Add DT_FILES variable to allow the user of the class to select specific
dts files to build. This is useful for packages featuring dts files
for multiple machines.

To make DT_FILES consistent with KERNEL_DEVICETREE, the list works
with both dts and dtb files.

Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
---
 meta/classes-recipe/devicetree.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
index ed2a92e447..bd50d7fa1d 100644
--- a/meta/classes-recipe/devicetree.bbclass
+++ b/meta/classes-recipe/devicetree.bbclass
@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
 
 DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
 DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
-DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
+DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
 DT_FILES_PATH ?= "${S}"
+DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
+DT_FILES ?= ""
 
 DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
 DT_PADDING_SIZE ??= "0x3000"
@@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
     subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 
 python devicetree_do_compile() {
+    import re
     includes = expand_includes("DT_INCLUDE", d)
+    dtfiles = d.getVar("DT_FILES").split()
+    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
     listpath = d.getVar("DT_FILES_PATH")
-    for dts in os.listdir(listpath):
+    for dts in dtfiles or os.listdir(listpath):
         dtspath = os.path.join(listpath, dts)
         try:
             if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
-- 
2.30.2

Re: [OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[Richard Purdie]

On Wed, 2023-04-26 at 09:22 +0200, Petr Kubizňák wrote:
> Add DT_FILES variable to allow the user of the class to select specific
> dts files to build. This is useful for packages featuring dts files
> for multiple machines.
> 
> To make DT_FILES consistent with KERNEL_DEVICETREE, the list works
> with both dts and dtb files.
> 
> Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
> ---
>  meta/classes-recipe/devicetree.bbclass | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
> index ed2a92e447..bd50d7fa1d 100644
> --- a/meta/classes-recipe/devicetree.bbclass
> +++ b/meta/classes-recipe/devicetree.bbclass
> @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
>  
>  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
>  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
>  DT_FILES_PATH ?= "${S}"
> +DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
> +DT_FILES ?= ""
>  
>  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
>  DT_PADDING_SIZE ??= "0x3000"
> @@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
>      subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>  
>  python devicetree_do_compile() {
> +    import re
>      includes = expand_includes("DT_INCLUDE", d)
> +    dtfiles = d.getVar("DT_FILES").split()
> +    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
>      listpath = d.getVar("DT_FILES_PATH")
> -    for dts in os.listdir(listpath):
> +    for dts in dtfiles or os.listdir(listpath):
>          dtspath = os.path.join(listpath, dts)
>          try:
>              if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):

I've taken this but could you send a patch to the documentation to add
this variable to the glossary and to the devicetree class section
please?

Thanks,

Richard

Re: [OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[Petr Kubizňák - 2N]

Hi Richard,

Thanks for accepting the patch.

I can for sure send a doc patch, just want to make sure it is desired since the glossary does not list any DT_* variable at the moment, and the devicetree class section in this respect only refers to the class sources. So I think both DT_FILES and DT_FILES_PATH variables should be documented at least, right?

Cheers,
Petr

________________________________________
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Sent: Thursday, May 4, 2023 1:38 PM
To: Petr Kubizňák - 2N; openembedded-core@lists.openembedded.org
Cc: Michael Opdenacker
Subject: Re: [OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

On Wed, 2023-04-26 at 09:22 +0200, Petr Kubizňák wrote:
> Add DT_FILES variable to allow the user of the class to select specific
> dts files to build. This is useful for packages featuring dts files
> for multiple machines.
>
> To make DT_FILES consistent with KERNEL_DEVICETREE, the list works
> with both dts and dtb files.
>
> Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
> ---
>  meta/classes-recipe/devicetree.bbclass | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
> index ed2a92e447..bd50d7fa1d 100644
> --- a/meta/classes-recipe/devicetree.bbclass
> +++ b/meta/classes-recipe/devicetree.bbclass
> @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
>
>  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
>  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
>  DT_FILES_PATH ?= "${S}"
> +DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
> +DT_FILES ?= ""
>
>  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
>  DT_PADDING_SIZE ??= "0x3000"
> @@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
>      subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>
>  python devicetree_do_compile() {
> +    import re
>      includes = expand_includes("DT_INCLUDE", d)
> +    dtfiles = d.getVar("DT_FILES").split()
> +    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
>      listpath = d.getVar("DT_FILES_PATH")
> -    for dts in os.listdir(listpath):
> +    for dts in dtfiles or os.listdir(listpath):
>          dtspath = os.path.join(listpath, dts)
>          try:
>              if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):

I've taken this but could you send a patch to the documentation to add
this variable to the glossary and to the devicetree class section
please?

Thanks,

Richard

Re: [OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[Richard Purdie]

On Fri, 2023-05-05 at 10:59 +0000, Petr Kubizňák - 2N wrote:
> Thanks for accepting the patch.
> 
> I can for sure send a doc patch, just want to make sure it is desired
> since the glossary does not list any DT_* variable at the moment, and
> the devicetree class section in this respect only refers to the class
> sources. So I think both DT_FILES and DT_FILES_PATH variables should
> be documented at least, right?

Ideally all the variables in use would be documented in the manual.
We're trying to fix and improve the docs so yes, adding docs for the
ones you added would be great, bonus marks if there are others you can
add at the same time!

Cheers,

Richard

Re: [OE-core][PATCH v4] devicetree.bbclass: Allow selection of dts files to build

[Michael Opdenacker]

On 05.05.23 at 13:10, Richard Purdie wrote:
> On Fri, 2023-05-05 at 10:59 +0000, Petr Kubizňák - 2N wrote:
>> Thanks for accepting the patch.
>>
>> I can for sure send a doc patch, just want to make sure it is desired
>> since the glossary does not list any DT_* variable at the moment, and
>> the devicetree class section in this respect only refers to the class
>> sources. So I think both DT_FILES and DT_FILES_PATH variables should
>> be documented at least, right?
> Ideally all the variables in use would be documented in the manual.
> We're trying to fix and improve the docs so yes, adding docs for the
> ones you added would be great, bonus marks if there are others you can
> add at the same time!


I have the same challenge with 
https://git.yoctoproject.org/poky/tree/meta/classes-recipe/uboot-sign.bbclass

The number of variables which can be modified is overwhelming! I guess 
what matters in the documentation is to refer to the variables for which 
there is a legitimate reason for changing the default value, right?
Like UBOOT_FIT_ADDRESS_CELLS, which was introduced recently.

Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com