Re: [PATCH] devicetree.bbclass: User/BSP device tree source compilation class

[Jaewon Lee <JAEWON@xilinx.com>]

Hi Nathan, 

What do you think about adding support for dts include files as well (.dtsi)?

Thanks,
Jaewon
 

-----Original Message-----
From: openembedded-core-bounces@lists.openembedded.org [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Nathan Rossi
Sent: Thursday, August 2, 2018 1:55 AM
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH] devicetree.bbclass: User/BSP device tree source compilation class

This bbclass implements the device tree compilation for user provided device trees. In order to use this class, it should be inherited in a BSP recipe which provides the sources. The default setup enables inclusion of kernel device tree sources (though can be disabled by the recipe by overriding DT_INCLUDE or KERNEL_INCLUDE).

This provides an additional mechanism for BSPs to provide device trees and device tree overlays for their target machines. Whilst still enabling access to the kernel device trees for base SoC includes and headers.

This approach to providing device trees has benefits for certain use cases over patching the device trees into the kernel source.

* device trees are separated from kernel source, allows for selection of kernel and or kernel versions without needing to explicitly patch the kernel (or appending to the kernel recipes).

* providing device trees from separate sources, from the layer, generated by the recipe or other recipes.

This class also implements some additional features that are not available in the kernel-devicetree flow. This includes population of device tree blobs into the sysroot which allows for other recipes to consume built dtbs (e.g. U-Boot with EXT_DTB compilation), device tree overlay compilation and customizing DTC compilation args (boot cpu/padding/etc.).

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Acked-by: Martin Hundebøll <mnhu@prevas.dk>
---
In addition to this patch I have written some documentation for the Yocto BSP guide that covers details about both methods.

The meta-xilinx layer has a recipe (device-tree.bb) that behaves exactly like this class for some time. It has been very useful for out-of-kernel device tree compilation as well as for a mechanism so that users of Xilinx SoCs can provide their customizations (FPGA devices) and board specific configuration. This compilation flow also makes sense for other BSPs, which is the reason for creating this bbclass for common shared use by other BSPs.

Examples of this classes usage can be seen for meta-xilinx:
https://github.com/nathanrossi/meta-xilinx/blob/nrossi/devicetree-classify/meta-xilinx-bsp/recipes-bsp/device-tree/device-tree.bb
---
 meta/classes/devicetree.bbclass | 140 ++++++++++++++++++++++++++++++++
 1 file changed, 140 insertions(+)
 create mode 100644 meta/classes/devicetree.bbclass

diff --git a/meta/classes/devicetree.bbclass b/meta/classes/devicetree.bbclass new file mode 100644 index 0000000000..dbc83f2a1d
--- /dev/null
+++ b/meta/classes/devicetree.bbclass
@@ -0,0 +1,140 @@
+# This bbclass implements device tree compliation for user provided 
+device tree # sources. The compilation of the device tree sources is 
+the same as the kernel # device tree compilation process, this includes 
+being able to include sources # from the kernel such as soc dtsi files 
+or header files such as gpio.h. In # addition to device trees this 
+bbclass also handles compilation of device tree # overlays.
+#
+# The output of this class behaves similar to how 
+kernel-devicetree.bbclass # operates in that the output files are installed into /boot/devicetree.
+# However this class on purpose separates the deployed device trees 
+into the # 'devicetree' subdirectory. This prevents clashes with the 
+kernel-devicetree # output. Additionally the device trees are populated 
+into the sysroot for # access via the sysroot from within other recipes.
+
+SECTION ?= "bsp"
+
+# The default inclusion of kernel device tree includes and headers 
+means that # device trees built with them are at least GPLv2 (and in 
+some cases dual # licensed). Default to GPLv2 if the recipe does not specify a license.
+LICENSE ?= "GPLv2"
+LIC_FILES_CHKSUM ?= "file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"
+
+INHIBIT_DEFAULT_DEPS = "1"
+DEPENDS += "dtc-native"
+
+inherit deploy kernel-arch
+
+COMPATIBLE_MACHINE ?= "^$"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+SYSROOT_DIRS += "/boot/devicetree"
+FILES_${PN} = "/boot/devicetree/*.dtb /boot/devicetree/*.dtbo"
+
+S = "${WORKDIR}"
+B = "${WORKDIR}/build"
+
+# Default kernel includes, these represent what are normally used for 
+in-kernel # sources.
+KERNEL_INCLUDE ??= " \
+        ${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts \
+        ${STAGING_KERNEL_DIR}/arch/${ARCH}/boot/dts/* \
+        ${STAGING_KERNEL_DIR}/scripts/dtc/include-prefixes \
+        "
+
+DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
+DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
+DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
+DT_FILES_PATH ?= "${S}"
+
+DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
+DT_PADDING_SIZE ??= "0x3000"
+DT_RESERVED_MAP[doc] = "Number of reserved map entires."
+DT_RESERVED_MAP ??= "8"
+DT_BOOT_CPU[doc] = "The boot cpu, defaults to 0"
+DT_BOOT_CPU ??= "0"
+
+DTC_FLAGS ?= "-R ${DT_RESERVED_MAP} -p ${DT_PADDING_SIZE} -b ${DT_BOOT_CPU}"
+DTC_PPFLAGS ?= "-nostdinc -undef -D__DTS__ -x assembler-with-cpp"
+DTC_OFLAGS ?= "-@ -H epapr"
+
+python () {
+    if d.getVar("KERNEL_INCLUDE"):
+        # auto add dependency on kernel tree, but only if kernel include paths
+        # are specified.
+        d.appendVarFlag("do_compile", "depends", " 
+virtual/kernel:do_configure") }
+
+def expand_includes(varname, d):
+    import glob
+    includes = set()
+    # expand all includes with glob
+    for i in (d.getVar(varname) or "").split():
+        for g in glob.glob(i):
+            if os.path.isdir(g): # only add directories to include path
+                includes.add(g)
+    return includes
+
+def devicetree_source_is_overlay(path):
+    # determine if a dts file is an overlay by checking if it uses "/plugin/;"
+    with open(path, "r") as f:
+        for i in f:
+            if i.startswith("/plugin/;"):
+                return True
+    return False
+
+def devicetree_compile(dtspath, includes, d):
+    import subprocess
+    dts = os.path.basename(dtspath)
+    dtname = os.path.splitext(dts)[0]
+    bb.note("Processing {0} [{1}]".format(dtname, dts))
+
+    # preprocess
+    ppargs = d.getVar("BUILD_CPP").split()
+    ppargs += (d.getVar("DTC_PPFLAGS") or "").split()
+    for i in includes:
+        ppargs.append("-I{0}".format(i))
+    ppargs += ["-o", "{0}.pp".format(dts), dtspath]
+    bb.note("Running {0}".format(" ".join(ppargs)))
+    subprocess.run(ppargs, check = True)
+
+    # determine if the file is an overlay or not (using the preprocessed file)
+    isoverlay = devicetree_source_is_overlay("{0}.pp".format(dts))
+
+    # compile
+    dtcargs = ["dtc"] + (d.getVar("DTC_FLAGS") or "").split()
+    if isoverlay:
+        dtcargs += (d.getVar("DTC_OFLAGS") or "").split()
+    for i in includes:
+        dtcargs += ["-i", i]
+    dtcargs += ["-o", "{0}.{1}".format(dtname, "dtbo" if isoverlay else "dtb")]
+    dtcargs += ["-I", "dts", "-O", "dtb", "{0}.pp".format(dts)]
+    bb.note("Running {0}".format(" ".join(dtcargs)))
+    subprocess.run(dtcargs, check = True)
+
+python devicetree_do_compile() {
+    includes = expand_includes("DT_INCLUDE", d)
+    listpath = d.getVar("DT_FILES_PATH")
+    for dts in os.listdir(listpath):
+        if not dts.endswith(".dts"):
+            continue # skip non-.dts files
+        dtspath = os.path.join(listpath, dts)
+        devicetree_compile(dtspath, includes, d) }
+
+devicetree_do_install() {
+    for DTB_FILE in `ls *.dtb *.dtbo`; do
+        install -Dm 0644 ${B}/${DTB_FILE} ${D}/boot/devicetree/${DTB_FILE}
+    done
+}
+
+devicetree_do_deploy() {
+    for DTB_FILE in `ls *.dtb *.dtbo`; do
+        install -Dm 0644 ${B}/${DTB_FILE} ${DEPLOYDIR}/devicetree/${DTB_FILE}
+    done
+}
+addtask deploy before do_build after do_install
+
+EXPORT_FUNCTIONS do_compile do_install do_deploy
+
--
2.18.0

--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core