[PATCH V3 0/3] Introduces kernel-initramfs recipe to resolve a implicit dependency issue

[PATCH V3 0/3] Introduces kernel-initramfs recipe to resolve a implicit dependency issue

[Ming Liu]

Changes in V3:
1 Adjusted to the tip:2a6e061712cfe9cb4738806a0c351a64e0d30144
[ cmake: update to 3.4.2 ]
2 Fixed a KERNEL_PRIORITY expanding issue
3 Tested with following configuration in local.conf:
  a) INITRAMFS_IMAGE = "initramfs-kexecboot-klibc-image"
     INITRAMFS_IMAGE_BUNDLE = "1"
     IMAGE_INSTALL_append = "kernel-initramfs"

  b) INITRAMFS_IMAGE = "core-image-minimal-initramfs"
     INITRAMFS_IMAGE_BUNDLE = "1"
     IMAGE_INSTALL_append = "kernel-initramfs"

  c) # INITRAMFS_IMAGE = "core-image-minimal-initramfs"
     INITRAMFS_IMAGE_BUNDLE = "1"
     IMAGE_INSTALL_append = "kernel-initramfs"

  d) # INITRAMFS_IMAGE = "core-image-minimal-initramfs"
     # INITRAMFS_IMAGE_BUNDLE = "1"
     # IMAGE_INSTALL_append = "kernel-initramfs"

Test was based on qemux86-64 machine, core-image-minimal.
With a and b, the kernel-initramfs package has been installed into rootfs correclty.
With c, got a error message like:
......
ERROR: Nothing RPROVIDES 'kernel-initramfs' (but /data/poky/meta/recipes-core/images/core-image-minimal.bb RDEPENDS on or otherwise requires it)
ERROR: kernel-initramfs was skipped: Set INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE to enable it
NOTE: Runtime target 'kernel-initramfs' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['kernel-initramfs']
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'kernel-initramfs']
......

With d, nothing changed comparing to without the patches.

-----------------------------------------------------------------------
In current initramfs bundled kernel packaging policy, there are several
dependency chains co-existing:

| "core-image-minimal.do_build" -> "core-image-minimal.do_bundle_initramfs"
| "core-image-minimal.do_bundle_initramfs" -> "virtual/kernel.do_bundle_initramfs"
| "core-image-minimal.do_bundle_initramfs" -> "core-image-minimal.do_rootfs"
| "core-image-minimal.do_rootfs" -> "virtual/kernel.do_package_write_${IMAGE_PKGTYPE}"
| "virtual/kernel.do_package_write_${IMAGE_PKGTYPE}" -> "virtual/kernel.do_package"
| "virtual/kernel.do_package" -> "virtual/kernel.do_install"
|
| "virtual/kernel.do_deploy" -> "virtual/kernel.do_bundle_initramfs"
| "virtual/kernel.do_bundle_initramfs" -> "virtual/kernel.do_install"

We could see, virtual/kernel.do_package is not explicitly depending on
virtual/kernel.do_bundle_initramfs so far, therefore, there is not a
workable way to add initramfs bundled kernel image into rootfs, because
kernel's do_bundle_initramfs could run parallelly with its do_package,
which means we will get a implicit kernel-image package that sometimes it
contains the initramfs bundled kernel or sometimes it doesn't.

To fix this problem, the idea is to let the initramfs bundled kernel
packaging depend on do_bundle_initramfs, meanwhile, to avoid the circular
dependency issue that commit: 609d5a9ab9e58bb1c2bcc2145399fbc8b701b85a
[ kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling ]

was trying to address, this dependency has to be splitted from kernel
recipe(at least, I could not figure out another way to achieve it), so a
new kernel-initramfs is introduced, in which a dependency chain is created:

| "kernel-initramfs.do_install" -> "virtual/kernel.do_deploy"
| "virtual/kernel.do_deploy" -> "virtual/kernel.do_bundle_initramfs"

Then the users can add initramfs bundled kernel image into rootfs by:

IMAGE_INSTALL_append = " kernel-initramfs"

without introducing any circular dependencies.

Ming Liu (3):
  kernel.bbclass: do not install initramfs bundled kernel image
  image.bbclass: removes bundle_initramfs related code
  kernel-initramfs: new recipe, creates initramfs bundled kernel
    packaging

 meta/classes/image.bbclass                    | 13 -----
 meta/classes/kernel.bbclass                   |  4 --
 meta/recipes-kernel/linux/kernel-initramfs.bb | 69 +++++++++++++++++++++++++++
 3 files changed, 69 insertions(+), 17 deletions(-)
 create mode 100644 meta/recipes-kernel/linux/kernel-initramfs.bb

-- 
1.9.1

[PATCH V3 1/3] kernel.bbclass: do not install initramfs bundled kernel image

[Ming Liu]

From: Ming Liu <peter.x.liu@external.atlascopco.com>

It makes no sense to install a initramfs bundled kernel image since
do_package does not depend on do_bundle_initramfs at all, otherwise, it
leads to a impicit kernel-image package depending on do_package run before
or after do_bundle_initramfs.

And for some reason, we can not add a intertask dependency between
do_package and do_bundle_initramfs, that would introduce a circular
dependency in following scenarios:
1 Some kernel modules have been added into INITRAMFS_IMAGE.
2 INITRAMFS_IMAGE has been bundled into kernel image.
3 kernel-image package has been added into core image's IMAGE_INSTALL.

We need firgure out another way to fix this problem.

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
---
 meta/classes/kernel.bbclass | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index f37affc..a3b33d0 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -179,10 +179,6 @@ do_bundle_initramfs () {
 		kernel_do_compile
 		mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.initramfs
 		mv -f ${KERNEL_OUTPUT}.bak ${KERNEL_OUTPUT}
-		# Update install area
-		echo "There is kernel image bundled with initramfs: ${B}/${KERNEL_OUTPUT}.initramfs"
-		install -m 0644 ${B}/${KERNEL_OUTPUT}.initramfs ${D}/boot/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin
-		echo "${B}/${KERNEL_OUTPUT}.initramfs"
 	fi
 }
 
-- 
1.9.1

[PATCH V3 2/3] image.bbclass: removes bundle_initramfs related code

[Ming Liu]

From: Ming Liu <peter.x.liu@external.atlascopco.com>

The original purpose of these code is to repackage initramfs bundled kernel
into rootfs by making image.bbclass build depend on bundle_initramfs of
virtual/kernel. But this is not going to succeed because virtual/kernel's
do_package is not depending on its bundle_initramfs.

We must figure out another way to package initramfs bundled kernel.

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
---
 meta/classes/image.bbclass | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 56a49e7..87645a0 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -167,10 +167,6 @@ python () {
     d.setVar('IMAGE_FEATURES', ' '.join(list(remain_features)))
 
     check_image_features(d)
-    initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
-    if initramfs_image != "":
-        d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" %  d.getVar('PN', True))
-        d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_image_complete" % initramfs_image)
 }
 
 IMAGE_CLASSES += "image_types"
@@ -511,12 +507,3 @@ do_packagedata[noexec] = "1"
 do_package_write_ipk[noexec] = "1"
 do_package_write_deb[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
-
-# Allow the kernel to be repacked with the initramfs and boot image file as a single file
-do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
-do_bundle_initramfs[nostamp] = "1"
-do_bundle_initramfs[noexec] = "1"
-do_bundle_initramfs () {
-	:
-}
-addtask bundle_initramfs after do_image_complete
-- 
1.9.1

[PATCH V3 3/3] kernel-initramfs: new recipe, creates initramfs bundled kernel packaging

[Ming Liu]

From: Ming Liu <peter.x.liu@external.atlascopco.com>

So far, there is not a workable way to package the initramfs bundled kernel
image, the only place is tmp/deploy from where we could get it, but if a
user want to add it into a certain image, there is no guarantee that he
will achieve it because a explicit dependency is missing between package
and bundle_initramfs, and this was designed to avoid introducing circular
dependencies when using kernel modules in the initramfs image.bb that is
caused by kernel.bbclass trying to build the initramfs before the kernel
packaging.

To fix this problem, the idea is to split the initramfs bundled kernel
packaging to a new recipe, make the do_install depend on kernel's deploy
task, then with the following config:

IMAGE_INSTALL_append = " kernel-initramfs"

The initramfs bundled kernel could be added to the image without
introducing any circular dependencies.

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
---
 meta/recipes-kernel/linux/kernel-initramfs.bb | 69 +++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 meta/recipes-kernel/linux/kernel-initramfs.bb

diff --git a/meta/recipes-kernel/linux/kernel-initramfs.bb b/meta/recipes-kernel/linux/kernel-initramfs.bb
new file mode 100644
index 0000000..49d539f
--- /dev/null
+++ b/meta/recipes-kernel/linux/kernel-initramfs.bb
@@ -0,0 +1,69 @@
+SUMMARY = "Initramfs bundled kernel image"
+DESCRIPTION = "When built, it packages a initramfs bundled kernel image of the \
+preferred virtual/kernel provider."
+
+SECTION = "kernel"
+
+LICENSE = "GPLv2"
+LIC_FILES_CHKSUM = "[file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6]file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"
+
+inherit linux-kernel-base
+
+# Whilst not a module, this ensures we don't get multilib extended. (which would make no sense)
+inherit module-base
+
+S = "${STAGING_KERNEL_DIR}"
+B = "${WORKDIR}/build"
+
+# we dont need the default dependencies.
+INHIBIT_DEFAULT_DEPS = "1"
+
+KERNEL_IMAGETYPE ?= "zImage"
+KERNEL_IMAGEDEST = "boot"
+KERNEL_VERSION = "${@['1.0.0', get_kernelversion_file('${STAGING_KERNEL_BUILDDIR}')][get_kernelversion_file('${STAGING_KERNEL_BUILDDIR}') != None]}"
+
+KERNEL_PRIORITY ?= "${@int(d.getVar('KERNEL_VERSION', True).split('-')[0].split('+')[0].split('.')[0]) * 10000 + \
+                       int(d.getVar('KERNEL_VERSION', True).split('-')[0].split('+')[0].split('.')[1]) * 100 + \
+                       int(d.getVar('KERNEL_VERSION', True).split('-')[0].split('+')[0].split('.')[-1])}"
+
+PACKAGES = "${PN}"
+FILES_${PN} = "/${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}*"
+
+PKG_${PN} = "${PN}-${@legitimize_package_name('${KERNEL_VERSION}')}"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+# Skip processing of this recipe if INITRAMFS_IMAGE or INITRAMFS_IMAGE_BUNDLE
+# is not set correctly, to avoid generate a single empty package which makes
+# no sense.
+python __anonymous () {
+    if not d.getVar('INITRAMFS_IMAGE', True) or d.getVar('INITRAMFS_IMAGE_BUNDLE', True) != '1':
+        raise bb.parse.SkipPackage("Set INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE to enable it")
+}
+
+# Need the output of deploy.
+do_install[depends] += "virtual/kernel:do_deploy"
+
+# We only need the packaging tasks - disable the rest
+do_fetch[noexec] = "1"
+do_unpack[noexec] = "1"
+do_patch[noexec] = "1"
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+do_populate_sysroot[noexec] = "1"
+
+do_install() {
+	if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
+		echo "Copying initramfs bundled kernel image from ${DEPLOY_DIR_IMAGE}..."
+		install -d ${D}/${KERNEL_IMAGEDEST}
+		install -m 0644 ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin ${D}/${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-initramfs-${KERNEL_VERSION}
+	fi
+}
+
+pkg_postinst_${PN} () {
+	update-alternatives --install /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-initramfs ${KERNEL_IMAGETYPE}-initramfs /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-initramfs-${KERNEL_VERSION} ${KERNEL_PRIORITY} || true
+}
+
+pkg_postrm_${PN} () {
+	update-alternatives --remove ${KERNEL_IMAGETYPE}-initramfs ${KERNEL_IMAGETYPE}-initramfs-${KERNEL_VERSION} || true
+}
-- 
1.9.1