[PATCH 0/9] Add basic fitImage support

[PATCH 0/9] Add basic fitImage support

[Marek Vasut]

This series cleans up the code in kernel.bbclass and
separates out the uImage generation. The 7th patch
in this series then adds support for generation of the
advanced successor of the uImage, the fitImage.

This series adds only very rudimentary support for
generation of fitImage. The advanced features might
be added in subsequent patches.

I would also like to thank Richard for explaining to
me how to properly do the implementation and I hope
this series is at least close to that.

Marek Vasut (9):
  kernel: Clean up KERNEL_IMAGETYPE_FOR_MAKE
  kernel: Rework do_uboot_mkimage
  kernel: Pull out the linux.bin generation
  kernel: Pull uImage generation into separate class
  kernel: Separate out uboot_prep_kimage
  kernel: Build DTBs early
  kernel: Add basic fitImage support
  kernel: Build uImage only when really needed
  doc: Document new KERNEL_CLASSES variable

 documentation/ref-manual/ref-variables.xml |  18 +++
 meta/classes/kernel-fitimage.bbclass       | 234 +++++++++++++++++++++++++++++
 meta/classes/kernel-uboot.bbclass          |  21 +++
 meta/classes/kernel-uimage.bbclass         |  36 +++++
 meta/classes/kernel.bbclass                |  52 +++----
 meta/conf/documentation.conf               |   1 +
 meta/recipes-kernel/linux/linux-dtb.inc    |  14 +-
 7 files changed, 343 insertions(+), 33 deletions(-)
 create mode 100644 meta/classes/kernel-fitimage.bbclass
 create mode 100644 meta/classes/kernel-uboot.bbclass
 create mode 100644 meta/classes/kernel-uimage.bbclass

-- 
2.1.4

[PATCH 1/9] kernel: Clean up KERNEL_IMAGETYPE_FOR_MAKE

[Marek Vasut]

Remove the lambda function setting KERNEL_IMAGETYPE_FOR_MAKE and instead
set it in the anonymous python function. This also allows us to handle
image types which are not supported directly by kernel, but require some
other kernel target to be built. This is the case for example with the
fitImage, which is the uImage successor.

There is no functional change.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 125ed88..75bfd76 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -17,12 +17,16 @@ INITRAMFS_TASK ?= ""
 INITRAMFS_IMAGE_BUNDLE ?= ""
 
 python __anonymous () {
+    import re
+
     kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
     if kerneltype == 'uImage':
         depends = d.getVar("DEPENDS", True)
         depends = "%s u-boot-mkimage-native" % depends
         d.setVar("DEPENDS", depends)
 
+    d.setVar("KERNEL_IMAGETYPE_FOR_MAKE", re.sub(r'\.gz$', '', kerneltype))
+
     image = d.getVar('INITRAMFS_IMAGE', True)
     if image:
         d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
@@ -104,8 +108,6 @@ KERNEL_ALT_IMAGETYPE ??= ""
 # they are staged.
 KERNEL_SRC_PATH = "/usr/src/kernel"
 
-KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
-
 copy_initramfs() {
 	echo "Copying initramfs into ./usr ..."
 	# In case the directory is not created yet from the first pass compile:
-- 
2.1.4

[PATCH 2/9] kernel: Rework do_uboot_mkimage

[Marek Vasut]

Rework the function so part it's internals can be re-used by fitImage
image type. The name of the temporary file , linux.bin , is recycled
a little more as it's now used for both the case where it is gzip
compressed and where it is not. This should be fine, since the file
is temporary and removed after the uImage was created anyway.

There is no functional change here.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel.bbclass | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 75bfd76..e85e73b 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -435,22 +435,32 @@ MODULE_TARBALL_DEPLOY ?= "1"
 do_uboot_mkimage() {
 	if test "x${KERNEL_IMAGETYPE}" = "xuImage" ; then 
 		if test "x${KEEPUIMAGE}" != "xyes" ; then
+			if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
+				vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
+				linux_suffix=""
+				linux_comp="none"
+			else
+				vmlinux_path="vmlinux"
+				linux_suffix=".gz"
+				linux_comp="gzip"
+			fi
+
+			${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
+
+			if [ "${linux_comp}" != "none" ] ; then
+				rm -f linux.bin
+				gzip -9 linux.bin
+				mv -f "linux.bin${linux_suffix}" linux.bin
+			fi
+
 			ENTRYPOINT=${UBOOT_ENTRYPOINT}
 			if test -n "${UBOOT_ENTRYSYMBOL}"; then
 				ENTRYPOINT=`${HOST_PREFIX}nm ${S}/vmlinux | \
 					awk '$3=="${UBOOT_ENTRYSYMBOL}" {print $1}'`
 			fi
-			if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
-				${OBJCOPY} -O binary -R .note -R .comment -S arch/${ARCH}/boot/compressed/vmlinux linux.bin
-				uboot-mkimage -A ${UBOOT_ARCH} -O linux -T kernel -C none -a ${UBOOT_LOADADDRESS} -e $ENTRYPOINT -n "${DISTRO_NAME}/${PV}/${MACHINE}" -d linux.bin arch/${ARCH}/boot/uImage
-				rm -f linux.bin
-			else
-				${OBJCOPY} -O binary -R .note -R .comment -S vmlinux linux.bin
-				rm -f linux.bin.gz
-				gzip -9 linux.bin
-				uboot-mkimage -A ${UBOOT_ARCH} -O linux -T kernel -C gzip -a ${UBOOT_LOADADDRESS} -e $ENTRYPOINT -n "${DISTRO_NAME}/${PV}/${MACHINE}" -d linux.bin.gz arch/${ARCH}/boot/uImage
-				rm -f linux.bin.gz
-			fi
+
+			uboot-mkimage -A ${UBOOT_ARCH} -O linux -T kernel -C "${linux_comp}" -a ${UBOOT_LOADADDRESS} -e $ENTRYPOINT -n "${DISTRO_NAME}/${PV}/${MACHINE}" -d linux.bin arch/${ARCH}/boot/uImage
+			rm -f linux.bin
 		fi
 	fi
 }
-- 
2.1.4

[PATCH 3/9] kernel: Pull out the linux.bin generation

[Marek Vasut]

Pull the generation of linux.bin image, which is then packed into uImage,
into a separate function. No functional change.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel.bbclass | 40 +++++++++++++++++++++++-----------------
 1 file changed, 23 insertions(+), 17 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e85e73b..2895e5e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -432,26 +432,32 @@ MODULE_TARBALL_BASE_NAME ?= "${MODULE_IMAGE_BASE_NAME}.tgz"
 MODULE_TARBALL_SYMLINK_NAME ?= "modules-${MACHINE}.tgz"
 MODULE_TARBALL_DEPLOY ?= "1"
 
+uboot_prep_kimage() {
+	if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
+		vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
+		linux_suffix=""
+		linux_comp="none"
+	else
+		vmlinux_path="vmlinux"
+		linux_suffix=".gz"
+		linux_comp="gzip"
+	fi
+
+	${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
+
+	if [ "${linux_comp}" != "none" ] ; then
+		rm -f linux.bin
+		gzip -9 linux.bin
+		mv -f "linux.bin${linux_suffix}" linux.bin
+	fi
+
+	echo "${linux_comp}"
+}
+
 do_uboot_mkimage() {
 	if test "x${KERNEL_IMAGETYPE}" = "xuImage" ; then 
 		if test "x${KEEPUIMAGE}" != "xyes" ; then
-			if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
-				vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
-				linux_suffix=""
-				linux_comp="none"
-			else
-				vmlinux_path="vmlinux"
-				linux_suffix=".gz"
-				linux_comp="gzip"
-			fi
-
-			${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
-
-			if [ "${linux_comp}" != "none" ] ; then
-				rm -f linux.bin
-				gzip -9 linux.bin
-				mv -f "linux.bin${linux_suffix}" linux.bin
-			fi
+			uboot_prep_kimage
 
 			ENTRYPOINT=${UBOOT_ENTRYPOINT}
 			if test -n "${UBOOT_ENTRYSYMBOL}"; then
-- 
2.1.4

[PATCH V2 4/9] kernel: Pull uImage generation into separate class

[Marek Vasut]

Pull the uImage image format generation from kernel.bbclass into
a separate kernel-uimage.bbclass. Introduce new KERNEL_CLASSES
variable, which allows registration of additional classes which
implement new kernel image types. The default value of is to
register kernel-uimage to preserve the original behavior.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-uimage.bbclass | 48 +++++++++++++++++++++++++++++
 meta/classes/kernel.bbclass        | 62 +++++++++++---------------------------
 2 files changed, 65 insertions(+), 45 deletions(-)
 create mode 100644 meta/classes/kernel-uimage.bbclass


V2: Implement KERNEL_CLASSES variable.

diff --git a/meta/classes/kernel-uimage.bbclass b/meta/classes/kernel-uimage.bbclass
new file mode 100644
index 0000000..8a3efc6
--- /dev/null
+++ b/meta/classes/kernel-uimage.bbclass
@@ -0,0 +1,48 @@
+python __anonymous () {
+    kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
+    if kerneltype == 'uImage':
+        depends = d.getVar("DEPENDS", True)
+        depends = "%s u-boot-mkimage-native" % depends
+        d.setVar("DEPENDS", depends)
+}
+
+uboot_prep_kimage() {
+	if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
+		vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
+		linux_suffix=""
+		linux_comp="none"
+	else
+		vmlinux_path="vmlinux"
+		linux_suffix=".gz"
+		linux_comp="gzip"
+	fi
+
+	${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
+
+	if [ "${linux_comp}" != "none" ] ; then
+		rm -f linux.bin
+		gzip -9 linux.bin
+		mv -f "linux.bin${linux_suffix}" linux.bin
+	fi
+
+	echo "${linux_comp}"
+}
+
+do_uboot_mkimage() {
+	if test "x${KERNEL_IMAGETYPE}" = "xuImage" ; then
+		if test "x${KEEPUIMAGE}" != "xyes" ; then
+			uboot_prep_kimage
+
+			ENTRYPOINT=${UBOOT_ENTRYPOINT}
+			if test -n "${UBOOT_ENTRYSYMBOL}"; then
+				ENTRYPOINT=`${HOST_PREFIX}nm ${S}/vmlinux | \
+					awk '$3=="${UBOOT_ENTRYSYMBOL}" {print $1}'`
+			fi
+
+			uboot-mkimage -A ${UBOOT_ARCH} -O linux -T kernel -C "${linux_comp}" -a ${UBOOT_LOADADDRESS} -e $ENTRYPOINT -n "${DISTRO_NAME}/${PV}/${MACHINE}" -d linux.bin arch/${ARCH}/boot/uImage
+			rm -f linux.bin
+		fi
+	fi
+}
+
+addtask uboot_mkimage before do_install after do_compile
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 2895e5e..74e925f 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -20,10 +20,6 @@ python __anonymous () {
     import re
 
     kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
-    if kerneltype == 'uImage':
-        depends = d.getVar("DEPENDS", True)
-        depends = "%s u-boot-mkimage-native" % depends
-        d.setVar("DEPENDS", depends)
 
     d.setVar("KERNEL_IMAGETYPE_FOR_MAKE", re.sub(r'\.gz$', '', kerneltype))
 
@@ -40,6 +36,23 @@ python __anonymous () {
         d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
 }
 
+# Here we pull in all various kernel image types which we support.
+#
+# In case you're wondering why kernel.bbclass inherits the other image
+# types instead of the other way around, the reason for that is to
+# maintain compatibility with various currently existing meta-layers.
+# By pulling in the various kernel image types here, we retain the
+# original behavior of kernel.bbclass, so no meta-layers should get
+# broken.
+#
+# KERNEL_CLASSES by default pulls in kernel-uimage.bbclass, since this
+# used to be the default behavior when only uImage was supported. This
+# variable can be appended by users who implement support for new kernel
+# image types.
+
+KERNEL_CLASSES ?= " kernel-uimage "
+inherit ${KERNEL_CLASSES}
+
 # Old style kernels may set ${S} = ${WORKDIR}/git for example
 # We need to move these over to STAGING_KERNEL_DIR. We can't just
 # create the symlink in advance as the git fetcher can't cope with
@@ -432,47 +445,6 @@ MODULE_TARBALL_BASE_NAME ?= "${MODULE_IMAGE_BASE_NAME}.tgz"
 MODULE_TARBALL_SYMLINK_NAME ?= "modules-${MACHINE}.tgz"
 MODULE_TARBALL_DEPLOY ?= "1"
 
-uboot_prep_kimage() {
-	if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
-		vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
-		linux_suffix=""
-		linux_comp="none"
-	else
-		vmlinux_path="vmlinux"
-		linux_suffix=".gz"
-		linux_comp="gzip"
-	fi
-
-	${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
-
-	if [ "${linux_comp}" != "none" ] ; then
-		rm -f linux.bin
-		gzip -9 linux.bin
-		mv -f "linux.bin${linux_suffix}" linux.bin
-	fi
-
-	echo "${linux_comp}"
-}
-
-do_uboot_mkimage() {
-	if test "x${KERNEL_IMAGETYPE}" = "xuImage" ; then 
-		if test "x${KEEPUIMAGE}" != "xyes" ; then
-			uboot_prep_kimage
-
-			ENTRYPOINT=${UBOOT_ENTRYPOINT}
-			if test -n "${UBOOT_ENTRYSYMBOL}"; then
-				ENTRYPOINT=`${HOST_PREFIX}nm ${S}/vmlinux | \
-					awk '$3=="${UBOOT_ENTRYSYMBOL}" {print $1}'`
-			fi
-
-			uboot-mkimage -A ${UBOOT_ARCH} -O linux -T kernel -C "${linux_comp}" -a ${UBOOT_LOADADDRESS} -e $ENTRYPOINT -n "${DISTRO_NAME}/${PV}/${MACHINE}" -d linux.bin arch/${ARCH}/boot/uImage
-			rm -f linux.bin
-		fi
-	fi
-}
-
-addtask uboot_mkimage before do_install after do_compile
-
 kernel_do_deploy() {
 	install -m 0644 ${KERNEL_OUTPUT} ${DEPLOYDIR}/${KERNEL_IMAGE_BASE_NAME}.bin
 	if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
-- 
2.1.4

[PATCH 5/9] kernel: Separate out uboot_prep_kimage

[Marek Vasut]

Separate the function which prepares the kernel for packing into
uImage into separate class, so this function can be reused by the
fitImage class.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-uboot.bbclass  | 21 +++++++++++++++++++++
 meta/classes/kernel-uimage.bbclass | 24 ++----------------------
 2 files changed, 23 insertions(+), 22 deletions(-)
 create mode 100644 meta/classes/kernel-uboot.bbclass

diff --git a/meta/classes/kernel-uboot.bbclass b/meta/classes/kernel-uboot.bbclass
new file mode 100644
index 0000000..8ab30b8
--- /dev/null
+++ b/meta/classes/kernel-uboot.bbclass
@@ -0,0 +1,21 @@
+uboot_prep_kimage() {
+	if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
+		vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
+		linux_suffix=""
+		linux_comp="none"
+	else
+		vmlinux_path="vmlinux"
+		linux_suffix=".gz"
+		linux_comp="gzip"
+	fi
+
+	${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
+
+	if [ "${linux_comp}" != "none" ] ; then
+		rm -f linux.bin
+		gzip -9 linux.bin
+		mv -f "linux.bin${linux_suffix}" linux.bin
+	fi
+
+	echo "${linux_comp}"
+}
diff --git a/meta/classes/kernel-uimage.bbclass b/meta/classes/kernel-uimage.bbclass
index 8a3efc6..ce8f96f 100644
--- a/meta/classes/kernel-uimage.bbclass
+++ b/meta/classes/kernel-uimage.bbclass
@@ -1,3 +1,5 @@
+inherit kernel-uboot
+
 python __anonymous () {
     kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
     if kerneltype == 'uImage':
@@ -6,28 +8,6 @@ python __anonymous () {
         d.setVar("DEPENDS", depends)
 }
 
-uboot_prep_kimage() {
-	if test -e arch/${ARCH}/boot/compressed/vmlinux ; then
-		vmlinux_path="arch/${ARCH}/boot/compressed/vmlinux"
-		linux_suffix=""
-		linux_comp="none"
-	else
-		vmlinux_path="vmlinux"
-		linux_suffix=".gz"
-		linux_comp="gzip"
-	fi
-
-	${OBJCOPY} -O binary -R .note -R .comment -S "${vmlinux_path}" linux.bin
-
-	if [ "${linux_comp}" != "none" ] ; then
-		rm -f linux.bin
-		gzip -9 linux.bin
-		mv -f "linux.bin${linux_suffix}" linux.bin
-	fi
-
-	echo "${linux_comp}"
-}
-
 do_uboot_mkimage() {
 	if test "x${KERNEL_IMAGETYPE}" = "xuImage" ; then
 		if test "x${KEEPUIMAGE}" != "xyes" ; then
-- 
2.1.4

[PATCH 6/9] kernel: Build DTBs early

[Marek Vasut]

Pull out the compilation of the DTB blobs right after the kernel's
own do_compile function finishes. This makes them available just in
time for the kernel image construction functions.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/recipes-kernel/linux/linux-dtb.inc | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-dtb.inc b/meta/recipes-kernel/linux/linux-dtb.inc
index 6b8f1a5..ee3a5e1 100644
--- a/meta/recipes-kernel/linux/linux-dtb.inc
+++ b/meta/recipes-kernel/linux/linux-dtb.inc
@@ -5,6 +5,18 @@ python __anonymous () {
     d.appendVar("PACKAGES", " kernel-devicetree")
 }
 
+do_compile_append() {
+	if test -n "${KERNEL_DEVICETREE}"; then
+		for DTB in ${KERNEL_DEVICETREE}; do
+			if echo ${DTB} | grep -q '/dts/'; then
+				bbwarn "${DTB} contains the full path to the the dts file, but only the dtb name should be used."
+				DTB=`basename ${DTB} | sed 's,\.dts$,.dtb,g'`
+			fi
+			oe_runmake ${DTB}
+		done
+	fi
+}
+
 do_install_append() {
 	if test -n "${KERNEL_DEVICETREE}"; then
 		for DTB in ${KERNEL_DEVICETREE}; do
@@ -13,10 +25,8 @@ do_install_append() {
 				DTB=`basename ${DTB} | sed 's,\.dts$,.dtb,g'`
 			fi
 			DTB_BASE_NAME=`basename ${DTB} .dtb`
-			DTB_NAME=`echo ${KERNEL_IMAGE_BASE_NAME} | sed "s/${MACHINE}/${DTB_BASE_NAME}/g"`
 			DTB_SYMLINK_NAME=`echo ${KERNEL_IMAGE_SYMLINK_NAME} | sed "s/${MACHINE}/${DTB_BASE_NAME}/g"`
 			DTB_PATH="${B}/arch/${ARCH}/boot/dts/${DTB}"
-			oe_runmake ${DTB}
 			if [ ! -e "${DTB_PATH}" ]; then
 				DTB_PATH="${B}/arch/${ARCH}/boot/${DTB}"
 			fi
-- 
2.1.4

[PATCH 7/9] kernel: Add basic fitImage support

[Marek Vasut]

This patch adds support for generating a kernel fitImage, which is
a a successor to the uImage format. Unlike uImage, which could only
contain the kernel image itself, the fitImage can contain all kinds
of artifacts, like the kernel image, device tree blobs, initramfs
images, binary firmwares etc. Furthermore, the fitImage supports
different kinds of checksums, not only CRC32 like the uImage did.
Last, but not least, fitImage supports signatures such that either
the whole image or it's parts can be signed and then in turn can
be verified by the bootloader.

So far we only add support for wrapping the kernel image and DTB
into the fitImage. The fitImage uses the sha1 checksum, which is
the default.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-fitimage.bbclass | 234 +++++++++++++++++++++++++++++++++++
 1 file changed, 234 insertions(+)
 create mode 100644 meta/classes/kernel-fitimage.bbclass

diff --git a/meta/classes/kernel-fitimage.bbclass b/meta/classes/kernel-fitimage.bbclass
new file mode 100644
index 0000000..2a56a54
--- /dev/null
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -0,0 +1,234 @@
+inherit kernel-uboot
+
+python __anonymous () {
+    kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
+    if kerneltype == 'fitImage':
+        depends = d.getVar("DEPENDS", True)
+        depends = "%s u-boot-mkimage-native dtc-native" % depends
+        d.setVar("DEPENDS", depends)
+
+	# Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
+	# to kernel.bbclass . We have to override it, since we pack zImage
+	# (at least for now) into the fitImage .
+        d.setVar("KERNEL_IMAGETYPE_FOR_MAKE", "zImage")
+
+        image = d.getVar('INITRAMFS_IMAGE', True)
+        if image:
+            d.appendVarFlag('do_assemble_fitimage', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
+}
+
+#
+# Emit the fitImage ITS header
+#
+fitimage_emit_fit_header() {
+	cat << EOF >> fit-image.its
+/dts-v1/;
+
+/ {
+        description = "U-Boot fitImage for ${DISTRO_NAME}/${PV}/${MACHINE}";
+        #address-cells = <1>;
+EOF
+}
+
+#
+# Emit the fitImage section bits
+#
+# $1 ... Section bit type: imagestart - image section start
+#                          confstart  - configuration section start
+#                          sectend    - section end
+#                          fitend     - fitimage end
+#
+fitimage_emit_section_maint() {
+	case $1 in
+	imagestart)
+		cat << EOF >> fit-image.its
+
+        images {
+EOF
+	;;
+	confstart)
+		cat << EOF >> fit-image.its
+
+        configurations {
+EOF
+	;;
+	sectend)
+		cat << EOF >> fit-image.its
+	};
+EOF
+	;;
+	fitend)
+		cat << EOF >> fit-image.its
+};
+EOF
+	;;
+	esac
+}
+
+#
+# Emit the fitImage ITS kernel section
+#
+# $1 ... Image counter
+# $2 ... Path to kernel image
+# $3 ... Compression type
+fitimage_emit_section_kernel() {
+
+	kernel_csum="sha1"
+
+	ENTRYPOINT=${UBOOT_ENTRYPOINT}
+	if test -n "${UBOOT_ENTRYSYMBOL}"; then
+		ENTRYPOINT=`${HOST_PREFIX}nm ${S}/vmlinux | \
+			awk '$3=="${UBOOT_ENTRYSYMBOL}" {print $1}'`
+	fi
+
+	cat << EOF >> fit-image.its
+                kernel@${1} {
+                        description = "Linux kernel";
+                        data = /incbin/("${2}");
+                        type = "kernel";
+                        arch = "${UBOOT_ARCH}";
+                        os = "linux";
+                        compression = "${3}";
+                        load = <${UBOOT_LOADADDRESS}>;
+                        entry = <${ENTRYPOINT}>;
+                        hash@1 {
+                                algo = "${kernel_csum}";
+                        };
+                };
+EOF
+}
+
+#
+# Emit the fitImage ITS DTB section
+#
+# $1 ... Image counter
+# $2 ... Path to DTB image
+fitimage_emit_section_dtb() {
+
+	dtb_csum="sha1"
+
+	cat << EOF >> fit-image.its
+                fdt@${1} {
+                        description = "Flattened Device Tree blob";
+                        data = /incbin/("${2}");
+                        type = "flat_dt";
+                        arch = "${UBOOT_ARCH}";
+                        compression = "none";
+                        hash@1 {
+                                algo = "${dtb_csum}";
+                        };
+                };
+EOF
+}
+
+#
+# Emit the fitImage ITS configuration section
+#
+# $1 ... Linux kernel ID
+# $2 ... DTB image ID
+fitimage_emit_section_config() {
+
+	conf_csum="sha1"
+
+	# Test if we have any DTBs at all
+	if [ -z "${2}" ] ; then
+		conf_desc="Boot Linux kernel"
+		fdt_line=""
+	else
+		conf_desc="Boot Linux kernel with FDT blob"
+		fdt_line="fdt = \"fdt@${2}\";"
+	fi
+	kernel_line="kernel = \"kernel@${1}\";"
+
+	cat << EOF >> fit-image.its
+                default = "conf@1";
+                conf@1 {
+                        description = "${conf_desc}";
+			${kernel_line}
+			${fdt_line}
+                        hash@1 {
+                                algo = "${conf_csum}";
+                        };
+                };
+EOF
+}
+
+do_assemble_fitimage() {
+	if test "x${KERNEL_IMAGETYPE}" = "xfitImage" ; then
+		kernelcount=1
+		dtbcount=""
+		rm -f fit-image.its
+
+		fitimage_emit_fit_header
+
+		#
+		# Step 1: Prepare a kernel image section.
+		#
+		fitimage_emit_section_maint imagestart
+
+		uboot_prep_kimage
+		fitimage_emit_section_kernel "${kernelcount}" linux.bin "${linux_comp}"
+
+		#
+		# Step 2: Prepare a DTB image section
+		#
+		if test -n "${KERNEL_DEVICETREE}"; then
+			dtbcount=1
+			for DTB in ${KERNEL_DEVICETREE}; do
+				if echo ${DTB} | grep -q '/dts/'; then
+					bbwarn "${DTB} contains the full path to the the dts file, but only the dtb name should be used."
+					DTB=`basename ${DTB} | sed 's,\.dts$,.dtb,g'`
+				fi
+				DTB_PATH="arch/${ARCH}/boot/dts/${DTB}"
+				if [ ! -e "${DTB_PATH}" ]; then
+					DTB_PATH="arch/${ARCH}/boot/${DTB}"
+				fi
+
+				fitimage_emit_section_dtb ${dtbcount} ${DTB_PATH}
+				dtbcount=`expr ${dtbcount} + 1`
+			done
+		fi
+
+		fitimage_emit_section_maint sectend
+
+		# Force the first Kernel and DTB in the default config
+		kernelcount=1
+		dtbcount=1
+
+		#
+		# Step 3: Prepare a configurations section
+		#
+		fitimage_emit_section_maint confstart
+
+		fitimage_emit_section_config ${kernelcount} ${dtbcount}
+
+		fitimage_emit_section_maint sectend
+
+		fitimage_emit_section_maint fitend
+
+		#
+		# Step 4: Assemble the image
+		#
+		uboot-mkimage -f fit-image.its arch/${ARCH}/boot/fitImage
+	fi
+}
+
+addtask assemble_fitimage before do_install after do_compile
+
+kernel_do_deploy_append() {
+	# Update deploy directory
+	if test "x${KERNEL_IMAGETYPE}" = "xfitImage" ; then
+		cd ${B}
+		echo "Copying fit-image.its source file..."
+		its_base_name="${KERNEL_IMAGETYPE}-its-${PV}-${PR}-${MACHINE}-${DATETIME}"
+		its_symlink_name=${KERNEL_IMAGETYPE}-its-${MACHINE}
+		install -m 0644 fit-image.its ${DEPLOYDIR}/${its_base_name}.its
+		linux_bin_base_name="${KERNEL_IMAGETYPE}-linux.bin-${PV}-${PR}-${MACHINE}-${DATETIME}"
+		linux_bin_symlink_name=${KERNEL_IMAGETYPE}-linux.bin-${MACHINE}
+		install -m 0644 linux.bin ${DEPLOYDIR}/${linux_bin_base_name}.bin
+
+		cd ${DEPLOYDIR}
+		ln -sf ${its_base_name}.its ${its_symlink_name}.its
+		ln -sf ${linux_bin_base_name}.bin ${linux_bin_symlink_name}.bin
+	fi
+}
-- 
2.1.4

[PATCH V2 8/9] kernel: Build uImage only when really needed

[Marek Vasut]

Build the uImage file using the kernel build system only when
it is really required, which is only in case KEEPUIMAGE == yes.
Otherwise, just build zImage, since the Yocto build system will
handle the uImage generation for us.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-uimage.bbclass | 8 ++++++++
 1 file changed, 8 insertions(+)

V2: Fix the KEEPUIMAGE check in __anonymous, the logic must
    be inverted. (thanks Bruce)

diff --git a/meta/classes/kernel-uimage.bbclass b/meta/classes/kernel-uimage.bbclass
index ce8f96f..f73965b 100644
--- a/meta/classes/kernel-uimage.bbclass
+++ b/meta/classes/kernel-uimage.bbclass
@@ -6,6 +6,14 @@ python __anonymous () {
         depends = d.getVar("DEPENDS", True)
         depends = "%s u-boot-mkimage-native" % depends
         d.setVar("DEPENDS", depends)
+
+	# Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
+	# to kernel.bbclass . We override the variable here, since we need
+	# to build uImage using the kernel build system if and only if
+	# KEEPUIMAGE == yes. Otherwise, we pack compressed vmlinux into
+	# the uImage .
+	if d.getVar("KEEPUIMAGE", True) != 'yes':
+            d.setVar("KERNEL_IMAGETYPE_FOR_MAKE", "zImage")
 }
 
 do_uboot_mkimage() {
-- 
2.1.4

[PATCH 9/9] doc: Document new KERNEL_CLASSES variable

[Marek Vasut]

Document the KERNEL_CLASSES variable, which is used to register
support for various kernel image types.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Koen Kooi <koen@dominion.thruhere.net>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 documentation/ref-manual/ref-variables.xml | 18 ++++++++++++++++++
 meta/conf/documentation.conf               |  1 +
 2 files changed, 19 insertions(+)

diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index ee5cbc2..2349207 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -6025,6 +6025,24 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
             </glossdef>
         </glossentry>
 
+        <glossentry id='var-KERNEL_CLASSES'><glossterm>KERNEL_CLASSES</glossterm>
+            <info>
+                KERNEL_CLASSES[doc] = "A list of classes defining kernel image types that kernel class should inherit."
+            </info>
+            <glossdef>
+                <para role="glossdeffirst">
+<!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
+		    A list of classes defining kernel image types that
+		    kernel class should inherit. You typically append this
+		    variable to enable extended image types. An example is
+		    the "kernel-fitimage", which enables fitImage support
+		    and resides in meta/classes/kernel-fitimage.bbclass .
+		    Custom kernel image types can be registered with the
+		    kernel class using this variable.
+                </para>
+            </glossdef>
+        </glossentry>
+
         <glossentry id='var-KERNEL_EXTRA_ARGS'><glossterm>KERNEL_EXTRA_ARGS</glossterm>
             <info>
                 KERNEL_EXTRA_ARGS[doc] = "Specifies additional make command-line arguments the OpenEmbedded build system passes on when compiling the kernel."
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 3a918e8..d55276b 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -241,6 +241,7 @@ IPK_FEED_URIS[doc] = "List of ipkg feed records to put into generated image."
 KARCH[doc] = "Defines the kernel architecture used when assembling the configuration. You define the KARCH variable in the BSP Descriptions."
 KBRANCH[doc] = "A regular expression used by the build process to explicitly identify the kernel branch that is validated, patched and configured during a build."
 KBRANCH_DEFAULT[doc] = "Defines the Linux kernel source repository's default branch used to build the Linux kernel. Unless you specify otherwise, the variable initializes to 'master'."
+KERNEL_CLASSES[doc] = "A list of classes defining kernel image types that kernel class should inherit."
 KERNEL_EXTRA_ARGS[doc] = "Specifies additional make command-line arguments the OpenEmbedded build system passes on when compiling the kernel."
 KERNEL_FEATURES[doc] = "Includes additional metadata from the Yocto Project kernel Git repository. The metadata you add through this variable includes config fragments and features descriptions."
 KERNEL_IMAGETYPE[doc] = "The type of kernel to build for a device, usually set by the machine configuration files and defaults to 'zImage'."
-- 
2.1.4

Re: [PATCH 1/9] kernel: Clean up KERNEL_IMAGETYPE_FOR_MAKE

[Bruce Ashfield]

The series looks good to me.

Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>

On 2015-05-14 08:31 AM, Marek Vasut wrote:
> Remove the lambda function setting KERNEL_IMAGETYPE_FOR_MAKE and instead
> set it in the anonymous python function. This also allows us to handle
> image types which are not supported directly by kernel, but require some
> other kernel target to be built. This is the case for example with the
> fitImage, which is the uImage successor.
>
> There is no functional change.
>
> Signed-off-by: Marek Vasut <marex@denx.de>
> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
> Cc: Koen Kooi <koen@dominion.thruhere.net>
> Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
> Cc: Ross Burton <ross.burton@intel.com>
> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
> ---
>   meta/classes/kernel.bbclass | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 125ed88..75bfd76 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -17,12 +17,16 @@ INITRAMFS_TASK ?= ""
>   INITRAMFS_IMAGE_BUNDLE ?= ""
>
>   python __anonymous () {
> +    import re
> +
>       kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
>       if kerneltype == 'uImage':
>           depends = d.getVar("DEPENDS", True)
>           depends = "%s u-boot-mkimage-native" % depends
>           d.setVar("DEPENDS", depends)
>
> +    d.setVar("KERNEL_IMAGETYPE_FOR_MAKE", re.sub(r'\.gz$', '', kerneltype))
> +
>       image = d.getVar('INITRAMFS_IMAGE', True)
>       if image:
>           d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
> @@ -104,8 +108,6 @@ KERNEL_ALT_IMAGETYPE ??= ""
>   # they are staged.
>   KERNEL_SRC_PATH = "/usr/src/kernel"
>
> -KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
> -
>   copy_initramfs() {
>   	echo "Copying initramfs into ./usr ..."
>   	# In case the directory is not created yet from the first pass compile:
>

Re: [PATCH 1/9] kernel: Clean up KERNEL_IMAGETYPE_FOR_MAKE

[Marek Vasut]

On Thursday, May 14, 2015 at 05:15:55 PM, Bruce Ashfield wrote:
> The series looks good to me.
> 
> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>

Hi,

any chance this could get applied so it can get further testing in-tree
please ?

Thank you!

Best regards,
Marek Vasut