* [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
@ 2013-08-22 23:04 Jason Wessel
2013-08-23 6:16 ` Khem Raj
2013-08-24 17:15 ` Richard Purdie
0 siblings, 2 replies; 10+ messages in thread
From: Jason Wessel @ 2013-08-22 23:04 UTC (permalink / raw)
To: Openembedded-core
This patch aims to fix the following two cases for the INITRAMFS generation.
1) Allow an image recipe to specify a paired INITRAMFS recipe such
as core-image-minimal-initramfs. This allows building a base
image which always generates the needed initramfs image in one step
2) Allow building a single binary which contains a kernel and
the initramfs.
A key requirement of the initramfs is to be able to add kernel
modules. The current implementation of the INITRAMFS_IMAGE variable
has a circular dependency when using kernel modules in the initramfs
image.bb file that is caused by kernel.bbclass trying to build the
initramfs before the kernel's do_install rule.
The solution for this problem is to have the kernel's
do_bundle_initramfs_image task depend on the do_rootfs from the
INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
will also sets up dependencies to make the initramfs creation task run
last.
The code to bundle the kernel and initramfs together has been added.
At a high level, all it is doing is invoking a second compilation of
the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
to the generated initramfs from the image recipe.
[YOCTO #4072]
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
meta/classes/image.bbclass | 12 ++++++
meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
meta/conf/local.conf.sample | 20 +++++++++
3 files changed, 116 insertions(+), 12 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 909702a..23967ec 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -130,6 +130,10 @@ python () {
d.setVar('MULTILIB_VENDORS', ml_vendor_list)
check_image_features(d)
+ initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
+ if initramfs_image != "":
+ d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
+ d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
}
#
@@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
do_package_write_rpm[noexec] = "1"
addtask rootfs before do_build
+# Allow the kernel to be repacked with the initramfs and boot image file as a single file
+do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
+do_bundle_initramfs[nostamp] = "1"
+do_bundle_initramfs[noexec] = "1"
+do_bundle_initramfs () {
+ :
+}
+addtask bundle_initramfs after do_rootfs
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e039dfc..8cf66ce 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
KERNEL_IMAGETYPE ?= "zImage"
INITRAMFS_IMAGE ?= ""
INITRAMFS_TASK ?= ""
+INITRAMFS_IMAGE_BUNDLE ?= ""
python __anonymous () {
kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
@@ -19,7 +20,15 @@ python __anonymous () {
image = d.getVar('INITRAMFS_IMAGE', True)
if image:
- d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
+ d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
+
+ # NOTE: setting INITRAMFS_TASK is for backward compatibility
+ # The preferred method is to set INITRAMFS_IMAGE, because
+ # this INITRAMFS_TASK has circular dependency problems
+ # if the initramfs requires kernel modules
+ image_task = d.getVar('INITRAMFS_TASK', True)
+ if image_task:
+ d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
}
inherit kernel-arch deploy
@@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
+copy_initramfs() {
+ echo "Copying initramfs into ./usr ..."
+ # Find and use the first initramfs image archive type we find
+ rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
+ for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
+ if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
+ cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
+ case $img in
+ *gz)
+ echo "gzip decompressing image"
+ gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
+ break
+ ;;
+ *lzo)
+ echo "lzo decompressing image"
+ lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
+ break
+ ;;
+ *lzma)
+ echo "lzma decompressing image"
+ lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
+ break
+ ;;
+ *xz)
+ echo "xz decompressing image"
+ xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
+ break
+ ;;
+ esac
+ fi
+ done
+ echo "Finished copy of initramfs into ./usr"
+}
+
+INITRAMFS_BASE_NAME = "${KERNEL_IMAGETYPE}-initramfs-${PV}-${PR}-${MACHINE}-${DATETIME}"
+INITRAMFS_BASE_NAME[vardepsexclude] = "DATETIME"
+do_bundle_initramfs () {
+ if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
+ echo "Creating a kernel image with a bundled initramfs..."
+ copy_initramfs
+ if [ -e ${KERNEL_OUTPUT} ] ; then
+ mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.bak
+ fi
+ use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
+ kernel_do_compile
+ mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.initramfs
+ mv -f ${KERNEL_OUTPUT}.bak ${KERNEL_OUTPUT}
+ # Update install area
+ echo "There is kernel image bundled with initramfs: ${B}/${KERNEL_OUTPUT}.initramfs"
+ install -m 0644 ${B}/${KERNEL_OUTPUT}.initramfs ${D}/boot/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin
+ echo "${B}/${KERNEL_OUTPUT}.initramfs"
+ cd ${B}
+ # Update deploy directory
+ if [ -e "${KERNEL_OUTPUT}.initramfs" ]; then
+ echo "Copying deploy kernel-initramfs image and setting up links..."
+ initramfs_base_name=${INITRAMFS_BASE_NAME}
+ initramfs_symlink_name=${KERNEL_IMAGETYPE}-initramfs-${MACHINE}
+ install -m 0644 ${KERNEL_OUTPUT}.initramfs ${DEPLOY_DIR_IMAGE}/${initramfs_base_name}.bin
+ cd ${DEPLOY_DIR_IMAGE}
+ ln -sf ${initramfs_base_name}.bin ${initramfs_symlink_name}.bin
+ fi
+ fi
+}
+do_bundle_initramfs[nostamp] = "1"
+addtask bundle_initramfs after do_compile
+
kernel_do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
- oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+ # The $use_alternate_initrd is only set from
+ # do_bundle_initramfs() This variable is specifically for the
+ # case where we are making a second pass at the kernel
+ # compilation and we want to force the kernel build to use a
+ # different initramfs image. The way to do that in the kernel
+ # is to specify:
+ # make ...args... CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio
+ oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}"; then
gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" > "${KERNEL_OUTPUT}"
fi
@@ -219,18 +301,8 @@ kernel_do_configure() {
cp "${WORKDIR}/defconfig" "${B}/.config"
fi
yes '' | oe_runmake oldconfig
-
- if [ ! -z "${INITRAMFS_IMAGE}" ]; then
- for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
- if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
- cp "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" initramfs.$img
- fi
- done
- fi
}
-do_configure[depends] += "${INITRAMFS_TASK}"
-
do_savedefconfig() {
oe_runmake savedefconfig
}
diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
index 2b078d0..17733ab 100644
--- a/meta/conf/local.conf.sample
+++ b/meta/conf/local.conf.sample
@@ -146,6 +146,26 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
#
+# Kernel image features
+#
+# The INITRAMFS_IMAGE image variable will cause an additional recipe to
+# be built as a dependency to the what ever rootfs recipe you might be
+# using such as core-image-sato. The initramfs might be needed for
+# the initial boot of of the target system such as to load kernel
+# modules prior to mounting the root file system.
+#
+# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
+# specified by the INITRAMFS_IMAGE will be run through an extra pass
+# through the kernel compilation in order to build a single binary
+# which contains both the kernel image and the initramfs. The
+# combined binary will be deposited into the tmp/deploy directory.
+# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
+# INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
+#
+#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
+#INITRAMFS_IMAGE_BUNDLE = "1"
+
+#
# Runtime testing of images
#
# The build system can test booting virtual machine images under qemu (an emulator)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-08-22 23:04 [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling Jason Wessel
@ 2013-08-23 6:16 ` Khem Raj
2013-08-23 12:56 ` Jason Wessel
2013-08-24 17:15 ` Richard Purdie
1 sibling, 1 reply; 10+ messages in thread
From: Khem Raj @ 2013-08-23 6:16 UTC (permalink / raw)
To: Jason Wessel; +Cc: Patches and discussions about the oe-core layer
On Thu, Aug 22, 2013 at 4:04 PM, Jason Wessel
<jason.wessel@windriver.com> wrote:
> This patch aims to fix the following two cases for the INITRAMFS generation.
> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
> as core-image-minimal-initramfs. This allows building a base
> image which always generates the needed initramfs image in one step
> 2) Allow building a single binary which contains a kernel and
> the initramfs.
I think this could be a bit simpler. Have a full kernel image recipe (
kernel + initramfs)
separate. It fits into the equation nicely. The final image can bundle
initramfs which has modules from practically same kernel build is
staged step 1 of kernel build which
automatically happens today for any image build.
The new recipe for kernel+initramfs requires the existing kernel
recipes and tweaks the .config to enable INITRAMFS_IMAGE
You can share the sources for both stages where kernel is built like
gcc is putting the sources under work-shared and building all gcc
recipes out of this location.
>
> A key requirement of the initramfs is to be able to add kernel
> modules. The current implementation of the INITRAMFS_IMAGE variable
> has a circular dependency when using kernel modules in the initramfs
> image.bb file that is caused by kernel.bbclass trying to build the
> initramfs before the kernel's do_install rule.
>
> The solution for this problem is to have the kernel's
> do_bundle_initramfs_image task depend on the do_rootfs from the
> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
> will also sets up dependencies to make the initramfs creation task run
> last.
>
> The code to bundle the kernel and initramfs together has been added.
> At a high level, all it is doing is invoking a second compilation of
> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
> to the generated initramfs from the image recipe.
>
> [YOCTO #4072]
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>
> ---
> meta/classes/image.bbclass | 12 ++++++
> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
> meta/conf/local.conf.sample | 20 +++++++++
> 3 files changed, 116 insertions(+), 12 deletions(-)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 909702a..23967ec 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -130,6 +130,10 @@ python () {
> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>
> check_image_features(d)
> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
> + if initramfs_image != "":
> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
> }
>
> #
> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
> do_package_write_rpm[noexec] = "1"
>
> addtask rootfs before do_build
> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
> +do_bundle_initramfs[nostamp] = "1"
> +do_bundle_initramfs[noexec] = "1"
> +do_bundle_initramfs () {
> + :
> +}
> +addtask bundle_initramfs after do_rootfs
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index e039dfc..8cf66ce 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
> KERNEL_IMAGETYPE ?= "zImage"
> INITRAMFS_IMAGE ?= ""
> INITRAMFS_TASK ?= ""
> +INITRAMFS_IMAGE_BUNDLE ?= ""
>
> python __anonymous () {
> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
> @@ -19,7 +20,15 @@ python __anonymous () {
>
> image = d.getVar('INITRAMFS_IMAGE', True)
> if image:
> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
> +
> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
> + # The preferred method is to set INITRAMFS_IMAGE, because
> + # this INITRAMFS_TASK has circular dependency problems
> + # if the initramfs requires kernel modules
> + image_task = d.getVar('INITRAMFS_TASK', True)
> + if image_task:
> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
> }
>
> inherit kernel-arch deploy
> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>
> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>
> +copy_initramfs() {
> + echo "Copying initramfs into ./usr ..."
> + # Find and use the first initramfs image archive type we find
> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
> + case $img in
> + *gz)
> + echo "gzip decompressing image"
> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *lzo)
> + echo "lzo decompressing image"
> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *lzma)
> + echo "lzma decompressing image"
> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *xz)
> + echo "xz decompressing image"
> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + esac
> + fi
> + done
> + echo "Finished copy of initramfs into ./usr"
> +}
> +
> +INITRAMFS_BASE_NAME = "${KERNEL_IMAGETYPE}-initramfs-${PV}-${PR}-${MACHINE}-${DATETIME}"
> +INITRAMFS_BASE_NAME[vardepsexclude] = "DATETIME"
> +do_bundle_initramfs () {
> + if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
> + echo "Creating a kernel image with a bundled initramfs..."
> + copy_initramfs
> + if [ -e ${KERNEL_OUTPUT} ] ; then
> + mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.bak
> + fi
> + use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> + kernel_do_compile
> + mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.initramfs
> + mv -f ${KERNEL_OUTPUT}.bak ${KERNEL_OUTPUT}
> + # Update install area
> + echo "There is kernel image bundled with initramfs: ${B}/${KERNEL_OUTPUT}.initramfs"
> + install -m 0644 ${B}/${KERNEL_OUTPUT}.initramfs ${D}/boot/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin
> + echo "${B}/${KERNEL_OUTPUT}.initramfs"
> + cd ${B}
> + # Update deploy directory
> + if [ -e "${KERNEL_OUTPUT}.initramfs" ]; then
> + echo "Copying deploy kernel-initramfs image and setting up links..."
> + initramfs_base_name=${INITRAMFS_BASE_NAME}
> + initramfs_symlink_name=${KERNEL_IMAGETYPE}-initramfs-${MACHINE}
> + install -m 0644 ${KERNEL_OUTPUT}.initramfs ${DEPLOY_DIR_IMAGE}/${initramfs_base_name}.bin
> + cd ${DEPLOY_DIR_IMAGE}
> + ln -sf ${initramfs_base_name}.bin ${initramfs_symlink_name}.bin
> + fi
> + fi
> +}
> +do_bundle_initramfs[nostamp] = "1"
> +addtask bundle_initramfs after do_compile
> +
> kernel_do_compile() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> - oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> + # The $use_alternate_initrd is only set from
> + # do_bundle_initramfs() This variable is specifically for the
> + # case where we are making a second pass at the kernel
> + # compilation and we want to force the kernel build to use a
> + # different initramfs image. The way to do that in the kernel
> + # is to specify:
> + # make ...args... CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio
> + oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
> if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}"; then
> gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" > "${KERNEL_OUTPUT}"
> fi
> @@ -219,18 +301,8 @@ kernel_do_configure() {
> cp "${WORKDIR}/defconfig" "${B}/.config"
> fi
> yes '' | oe_runmake oldconfig
> -
> - if [ ! -z "${INITRAMFS_IMAGE}" ]; then
> - for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
> - if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
> - cp "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" initramfs.$img
> - fi
> - done
> - fi
> }
>
> -do_configure[depends] += "${INITRAMFS_TASK}"
> -
> do_savedefconfig() {
> oe_runmake savedefconfig
> }
> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
> index 2b078d0..17733ab 100644
> --- a/meta/conf/local.conf.sample
> +++ b/meta/conf/local.conf.sample
> @@ -146,6 +146,26 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
> USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>
> #
> +# Kernel image features
> +#
> +# The INITRAMFS_IMAGE image variable will cause an additional recipe to
> +# be built as a dependency to the what ever rootfs recipe you might be
> +# using such as core-image-sato. The initramfs might be needed for
> +# the initial boot of of the target system such as to load kernel
> +# modules prior to mounting the root file system.
> +#
> +# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
> +# specified by the INITRAMFS_IMAGE will be run through an extra pass
> +# through the kernel compilation in order to build a single binary
> +# which contains both the kernel image and the initramfs. The
> +# combined binary will be deposited into the tmp/deploy directory.
> +# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
> +# INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
> +#
> +#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
> +#INITRAMFS_IMAGE_BUNDLE = "1"
> +
> +#
> # Runtime testing of images
> #
> # The build system can test booting virtual machine images under qemu (an emulator)
> --
> 1.7.9.5
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-08-23 6:16 ` Khem Raj
@ 2013-08-23 12:56 ` Jason Wessel
2013-08-26 8:33 ` Andrea Adami
0 siblings, 1 reply; 10+ messages in thread
From: Jason Wessel @ 2013-08-23 12:56 UTC (permalink / raw)
To: Khem Raj; +Cc: Patches and discussions about the oe-core layer
On 08/23/2013 01:16 AM, Khem Raj wrote:
> On Thu, Aug 22, 2013 at 4:04 PM, Jason Wessel
> <jason.wessel@windriver.com> wrote:
>> This patch aims to fix the following two cases for the INITRAMFS generation.
>> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>> as core-image-minimal-initramfs. This allows building a base
>> image which always generates the needed initramfs image in one step
>> 2) Allow building a single binary which contains a kernel and
>> the initramfs.
> I think this could be a bit simpler. Have a full kernel image recipe (
> kernel + initramfs)
> separate. It fits into the equation nicely. The final image can bundle
> initramfs which has modules from practically same kernel build is
> staged step 1 of kernel build which
> automatically happens today for any image build.
The changes I put together are work around the circular dependency problem. Staging and using another recipe is certainly an option.
What I really wanted to do was have just another image type not unlike cpio, ext3, etc... But this was not possible because there was no way to get back to the point that the kernel could be re-linked and make the call to the kernel methods and get the context correct. I had tried using the internal method for exec_task from the image recipe context but there appeared to be no direct way to get this to work.
>
> The new recipe for kernel+initramfs requires the existing kernel
> recipes and tweaks the .config to enable INITRAMFS_IMAGE
>
> You can share the sources for both stages where kernel is built like
> gcc is putting the sources under work-shared and building all gcc
> recipes out of this location.
Ideally we could use the sysroot to regenerate the combined image, but this was not low hanging fruit. If we came up with a way to generate the combined kernel+initramfs image directly from the sysroot this could easily be controlled by just having it as an IMAGE_FEATURE as opposed to making distinct image types.
Jason.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-08-22 23:04 [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling Jason Wessel
2013-08-23 6:16 ` Khem Raj
@ 2013-08-24 17:15 ` Richard Purdie
2013-09-27 8:07 ` Andrea Adami
1 sibling, 1 reply; 10+ messages in thread
From: Richard Purdie @ 2013-08-24 17:15 UTC (permalink / raw)
To: Jason Wessel; +Cc: Openembedded-core
On Thu, 2013-08-22 at 18:04 -0500, Jason Wessel wrote:
> This patch aims to fix the following two cases for the INITRAMFS generation.
> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
> as core-image-minimal-initramfs. This allows building a base
> image which always generates the needed initramfs image in one step
> 2) Allow building a single binary which contains a kernel and
> the initramfs.
>
> A key requirement of the initramfs is to be able to add kernel
> modules. The current implementation of the INITRAMFS_IMAGE variable
> has a circular dependency when using kernel modules in the initramfs
> image.bb file that is caused by kernel.bbclass trying to build the
> initramfs before the kernel's do_install rule.
>
> The solution for this problem is to have the kernel's
> do_bundle_initramfs_image task depend on the do_rootfs from the
> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
> will also sets up dependencies to make the initramfs creation task run
> last.
>
> The code to bundle the kernel and initramfs together has been added.
> At a high level, all it is doing is invoking a second compilation of
> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
> to the generated initramfs from the image recipe.
>
> [YOCTO #4072]
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
I have a couple of things I'd really like to see get resolved here. One
is below, the other is I'm worried about the packaged output differences
since we can package the kernel into a package file and now its going to
be different.
I appreciate its a hard problem to solve but not impossible. Basically
we move the package generation for that single package into a separate
recipe and have it depend on the bundling task if/as/when needed. The
bundle task stashes the kernel in the sysroot, the other recipe simply
packages it. Its a little bit of a dance but should ensure we get
everything consistent.
> ---
> meta/classes/image.bbclass | 12 ++++++
> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
> meta/conf/local.conf.sample | 20 +++++++++
> 3 files changed, 116 insertions(+), 12 deletions(-)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 909702a..23967ec 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -130,6 +130,10 @@ python () {
> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>
> check_image_features(d)
> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
> + if initramfs_image != "":
> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
> }
>
> #
> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
> do_package_write_rpm[noexec] = "1"
>
> addtask rootfs before do_build
> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
> +do_bundle_initramfs[nostamp] = "1"
> +do_bundle_initramfs[noexec] = "1"
> +do_bundle_initramfs () {
> + :
> +}
> +addtask bundle_initramfs after do_rootfs
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index e039dfc..8cf66ce 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
> KERNEL_IMAGETYPE ?= "zImage"
> INITRAMFS_IMAGE ?= ""
> INITRAMFS_TASK ?= ""
> +INITRAMFS_IMAGE_BUNDLE ?= ""
>
> python __anonymous () {
> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
> @@ -19,7 +20,15 @@ python __anonymous () {
>
> image = d.getVar('INITRAMFS_IMAGE', True)
> if image:
> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
> +
> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
> + # The preferred method is to set INITRAMFS_IMAGE, because
> + # this INITRAMFS_TASK has circular dependency problems
> + # if the initramfs requires kernel modules
> + image_task = d.getVar('INITRAMFS_TASK', True)
> + if image_task:
> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
> }
>
> inherit kernel-arch deploy
> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>
> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>
> +copy_initramfs() {
> + echo "Copying initramfs into ./usr ..."
> + # Find and use the first initramfs image archive type we find
> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
> + case $img in
> + *gz)
> + echo "gzip decompressing image"
> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *lzo)
> + echo "lzo decompressing image"
> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *lzma)
> + echo "lzma decompressing image"
> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + *xz)
> + echo "xz decompressing image"
> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> + break
> + ;;
> + esac
> + fi
> + done
> + echo "Finished copy of initramfs into ./usr"
> +}
But what about my bzip2'd image? ;-)
I'd suggest we rid of this and instead ensure that we're generating an
uncompressed cpio image. The image generation code will happily sort
that our for us if we ask it for that specific image type.
I'd also wondered if we could remove INITRAMFS_TASK since its just going
to confuse things and I'd prefer to maintain only one way of doing this
if at all possible.
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-08-23 12:56 ` Jason Wessel
@ 2013-08-26 8:33 ` Andrea Adami
0 siblings, 0 replies; 10+ messages in thread
From: Andrea Adami @ 2013-08-26 8:33 UTC (permalink / raw)
To: Jason Wessel; +Cc: Patches and discussions about the oe-core layer
On Fri, Aug 23, 2013 at 2:56 PM, Jason Wessel
<jason.wessel@windriver.com> wrote:
> On 08/23/2013 01:16 AM, Khem Raj wrote:
>> On Thu, Aug 22, 2013 at 4:04 PM, Jason Wessel
>> <jason.wessel@windriver.com> wrote:
>>> This patch aims to fix the following two cases for the INITRAMFS generation.
>>> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>>> as core-image-minimal-initramfs. This allows building a base
>>> image which always generates the needed initramfs image in one step
>>> 2) Allow building a single binary which contains a kernel and
>>> the initramfs.
>> I think this could be a bit simpler. Have a full kernel image recipe (
>> kernel + initramfs)
>> separate. It fits into the equation nicely. The final image can bundle
>> initramfs which has modules from practically same kernel build is
>> staged step 1 of kernel build which
>> automatically happens today for any image build.
>
> The changes I put together are work around the circular dependency problem. Staging and using another recipe is certainly an option.
>
> What I really wanted to do was have just another image type not unlike cpio, ext3, etc... But this was not possible because there was no way to get back to the point that the kernel could be re-linked and make the call to the kernel methods and get the context correct. I had tried using the internal method for exec_task from the image recipe context but there appeared to be no direct way to get this to work.
>
>
>
>
>>
>> The new recipe for kernel+initramfs requires the existing kernel
>> recipes and tweaks the .config to enable INITRAMFS_IMAGE
>>
>> You can share the sources for both stages where kernel is built like
>> gcc is putting the sources under work-shared and building all gcc
>> recipes out of this location.
>
>
> Ideally we could use the sysroot to regenerate the combined image, but this was not low hanging fruit. If we came up with a way to generate the combined kernel+initramfs image directly from the sysroot this could easily be controlled by just having it as an IMAGE_FEATURE as opposed to making distinct image types.
>
>
> Jason.
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Some more thoughts:
I'd like to point out that we introduced a variable called INITRAMFS_FSTYPES
http://cgit.openembedded.org/openembedded-core/commit/meta/conf/bitbake.conf?id=17f7f3a43e863d9e2a16dd02face5137a4f4b225
and that we have an initial initramfs framework (modular, similar to
the one in oe-classic) :
http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-core/initrdscripts?id=7b69ad2167a1f0e57db82817b98a0cbcb70a0dd3
Maybe this framework could be extended for the typical simple needs (
I know our linux-yocto-tiny-kexecboot is a rather complicated
example).
Cheers
Andrea
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-08-24 17:15 ` Richard Purdie
@ 2013-09-27 8:07 ` Andrea Adami
2013-09-27 9:05 ` Richard Purdie
0 siblings, 1 reply; 10+ messages in thread
From: Andrea Adami @ 2013-09-27 8:07 UTC (permalink / raw)
To: Richard Purdie, Bruce Ashfield; +Cc: Openembedded-core
On Sat, Aug 24, 2013 at 7:15 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2013-08-22 at 18:04 -0500, Jason Wessel wrote:
>> This patch aims to fix the following two cases for the INITRAMFS generation.
>> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>> as core-image-minimal-initramfs. This allows building a base
>> image which always generates the needed initramfs image in one step
>> 2) Allow building a single binary which contains a kernel and
>> the initramfs.
>>
>> A key requirement of the initramfs is to be able to add kernel
>> modules. The current implementation of the INITRAMFS_IMAGE variable
>> has a circular dependency when using kernel modules in the initramfs
>> image.bb file that is caused by kernel.bbclass trying to build the
>> initramfs before the kernel's do_install rule.
>>
>> The solution for this problem is to have the kernel's
>> do_bundle_initramfs_image task depend on the do_rootfs from the
>> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
>> will also sets up dependencies to make the initramfs creation task run
>> last.
>>
>> The code to bundle the kernel and initramfs together has been added.
>> At a high level, all it is doing is invoking a second compilation of
>> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
>> to the generated initramfs from the image recipe.
>>
>> [YOCTO #4072]
>>
>> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>
> I have a couple of things I'd really like to see get resolved here. One
> is below, the other is I'm worried about the packaged output differences
> since we can package the kernel into a package file and now its going to
> be different.
>
> I appreciate its a hard problem to solve but not impossible. Basically
> we move the package generation for that single package into a separate
> recipe and have it depend on the bundling task if/as/when needed. The
> bundle task stashes the kernel in the sysroot, the other recipe simply
> packages it. Its a little bit of a dance but should ensure we get
> everything consistent.
>
>
>> ---
>> meta/classes/image.bbclass | 12 ++++++
>> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
>> meta/conf/local.conf.sample | 20 +++++++++
>> 3 files changed, 116 insertions(+), 12 deletions(-)
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 909702a..23967ec 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -130,6 +130,10 @@ python () {
>> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>>
>> check_image_features(d)
>> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
>> + if initramfs_image != "":
>> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
>> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
>> }
>>
>> #
>> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
>> do_package_write_rpm[noexec] = "1"
>>
>> addtask rootfs before do_build
>> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
>> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
>> +do_bundle_initramfs[nostamp] = "1"
>> +do_bundle_initramfs[noexec] = "1"
>> +do_bundle_initramfs () {
>> + :
>> +}
>> +addtask bundle_initramfs after do_rootfs
>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> index e039dfc..8cf66ce 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
>> KERNEL_IMAGETYPE ?= "zImage"
>> INITRAMFS_IMAGE ?= ""
>> INITRAMFS_TASK ?= ""
>> +INITRAMFS_IMAGE_BUNDLE ?= ""
>>
>> python __anonymous () {
>> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
>> @@ -19,7 +20,15 @@ python __anonymous () {
>>
>> image = d.getVar('INITRAMFS_IMAGE', True)
>> if image:
>> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
>> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
>> +
>> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
>> + # The preferred method is to set INITRAMFS_IMAGE, because
>> + # this INITRAMFS_TASK has circular dependency problems
>> + # if the initramfs requires kernel modules
>> + image_task = d.getVar('INITRAMFS_TASK', True)
>> + if image_task:
>> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
>> }
>>
>> inherit kernel-arch deploy
>> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>>
>> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>>
>> +copy_initramfs() {
>> + echo "Copying initramfs into ./usr ..."
>> + # Find and use the first initramfs image archive type we find
>> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
>> + case $img in
>> + *gz)
>> + echo "gzip decompressing image"
>> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> + break
>> + ;;
>> + *lzo)
>> + echo "lzo decompressing image"
>> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> + break
>> + ;;
>> + *lzma)
>> + echo "lzma decompressing image"
>> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> + break
>> + ;;
>> + *xz)
>> + echo "xz decompressing image"
>> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> + break
>> + ;;
>> + esac
>> + fi
>> + done
>> + echo "Finished copy of initramfs into ./usr"
>> +}
>
> But what about my bzip2'd image? ;-)
>
> I'd suggest we rid of this and instead ensure that we're generating an
> uncompressed cpio image. The image generation code will happily sort
> that our for us if we ask it for that specific image type.
>
> I'd also wondered if we could remove INITRAMFS_TASK since its just going
> to confuse things and I'd prefer to maintain only one way of doing this
> if at all possible.
>
> Cheers,
>
> Richard
Though our observations, in an effort to solve Yocto #4072 the
patchset has been merged.
As side effect it broke the 'old style' creation of initramfs so some
recipes are now unbuildable.
I'm in touch with Bruce and Jason and I have already a patch for
kernel.bbclass restoring the old functionality by adding one more
variable in each recipe: INITRAMFS_TASK.
I'm pretty sure we could have built the same kind of images containing
modules in one pass before as well and still dislike the idea of
having to set a variable in local.conf to spread it to the kernel and
to the image (to any image...)
With 1.5 approaching I'd like to have the issue solved as soon as possible.
Richard, when is deadline for core-patches?
Cheers
Andrea
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-09-27 8:07 ` Andrea Adami
@ 2013-09-27 9:05 ` Richard Purdie
2013-09-27 11:59 ` Andrea Adami
2013-09-27 16:33 ` Jason Wessel
0 siblings, 2 replies; 10+ messages in thread
From: Richard Purdie @ 2013-09-27 9:05 UTC (permalink / raw)
To: Andrea Adami; +Cc: Openembedded-core
On Fri, 2013-09-27 at 10:07 +0200, Andrea Adami wrote:
> On Sat, Aug 24, 2013 at 7:15 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2013-08-22 at 18:04 -0500, Jason Wessel wrote:
> >> This patch aims to fix the following two cases for the INITRAMFS generation.
> >> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
> >> as core-image-minimal-initramfs. This allows building a base
> >> image which always generates the needed initramfs image in one step
> >> 2) Allow building a single binary which contains a kernel and
> >> the initramfs.
> >>
> >> A key requirement of the initramfs is to be able to add kernel
> >> modules. The current implementation of the INITRAMFS_IMAGE variable
> >> has a circular dependency when using kernel modules in the initramfs
> >> image.bb file that is caused by kernel.bbclass trying to build the
> >> initramfs before the kernel's do_install rule.
> >>
> >> The solution for this problem is to have the kernel's
> >> do_bundle_initramfs_image task depend on the do_rootfs from the
> >> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
> >> will also sets up dependencies to make the initramfs creation task run
> >> last.
> >>
> >> The code to bundle the kernel and initramfs together has been added.
> >> At a high level, all it is doing is invoking a second compilation of
> >> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
> >> to the generated initramfs from the image recipe.
> >>
> >> [YOCTO #4072]
> >>
> >> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> >> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> >
> > I have a couple of things I'd really like to see get resolved here. One
> > is below, the other is I'm worried about the packaged output differences
> > since we can package the kernel into a package file and now its going to
> > be different.
> >
> > I appreciate its a hard problem to solve but not impossible. Basically
> > we move the package generation for that single package into a separate
> > recipe and have it depend on the bundling task if/as/when needed. The
> > bundle task stashes the kernel in the sysroot, the other recipe simply
> > packages it. Its a little bit of a dance but should ensure we get
> > everything consistent.
> >
> >
> >> ---
> >> meta/classes/image.bbclass | 12 ++++++
> >> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
> >> meta/conf/local.conf.sample | 20 +++++++++
> >> 3 files changed, 116 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> >> index 909702a..23967ec 100644
> >> --- a/meta/classes/image.bbclass
> >> +++ b/meta/classes/image.bbclass
> >> @@ -130,6 +130,10 @@ python () {
> >> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
> >>
> >> check_image_features(d)
> >> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
> >> + if initramfs_image != "":
> >> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
> >> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
> >> }
> >>
> >> #
> >> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
> >> do_package_write_rpm[noexec] = "1"
> >>
> >> addtask rootfs before do_build
> >> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
> >> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
> >> +do_bundle_initramfs[nostamp] = "1"
> >> +do_bundle_initramfs[noexec] = "1"
> >> +do_bundle_initramfs () {
> >> + :
> >> +}
> >> +addtask bundle_initramfs after do_rootfs
> >> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> >> index e039dfc..8cf66ce 100644
> >> --- a/meta/classes/kernel.bbclass
> >> +++ b/meta/classes/kernel.bbclass
> >> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
> >> KERNEL_IMAGETYPE ?= "zImage"
> >> INITRAMFS_IMAGE ?= ""
> >> INITRAMFS_TASK ?= ""
> >> +INITRAMFS_IMAGE_BUNDLE ?= ""
> >>
> >> python __anonymous () {
> >> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
> >> @@ -19,7 +20,15 @@ python __anonymous () {
> >>
> >> image = d.getVar('INITRAMFS_IMAGE', True)
> >> if image:
> >> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
> >> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
> >> +
> >> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
> >> + # The preferred method is to set INITRAMFS_IMAGE, because
> >> + # this INITRAMFS_TASK has circular dependency problems
> >> + # if the initramfs requires kernel modules
> >> + image_task = d.getVar('INITRAMFS_TASK', True)
> >> + if image_task:
> >> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
> >> }
> >>
> >> inherit kernel-arch deploy
> >> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
> >>
> >> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
> >>
> >> +copy_initramfs() {
> >> + echo "Copying initramfs into ./usr ..."
> >> + # Find and use the first initramfs image archive type we find
> >> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
> >> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
> >> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
> >> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
> >> + case $img in
> >> + *gz)
> >> + echo "gzip decompressing image"
> >> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> >> + break
> >> + ;;
> >> + *lzo)
> >> + echo "lzo decompressing image"
> >> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> >> + break
> >> + ;;
> >> + *lzma)
> >> + echo "lzma decompressing image"
> >> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> >> + break
> >> + ;;
> >> + *xz)
> >> + echo "xz decompressing image"
> >> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
> >> + break
> >> + ;;
> >> + esac
> >> + fi
> >> + done
> >> + echo "Finished copy of initramfs into ./usr"
> >> +}
> >
> > But what about my bzip2'd image? ;-)
> >
> > I'd suggest we rid of this and instead ensure that we're generating an
> > uncompressed cpio image. The image generation code will happily sort
> > that our for us if we ask it for that specific image type.
> >
> > I'd also wondered if we could remove INITRAMFS_TASK since its just going
> > to confuse things and I'd prefer to maintain only one way of doing this
> > if at all possible.
> >
> > Cheers,
> >
> > Richard
>
>
> Though our observations, in an effort to solve Yocto #4072 the
> patchset has been merged.
> As side effect it broke the 'old style' creation of initramfs so some
> recipes are now unbuildable.
>
> I'm in touch with Bruce and Jason and I have already a patch for
> kernel.bbclass restoring the old functionality by adding one more
> variable in each recipe: INITRAMFS_TASK.
>
> I'm pretty sure we could have built the same kind of images containing
> modules in one pass before as well and still dislike the idea of
> having to set a variable in local.conf to spread it to the kernel and
> to the image (to any image...)
>
> With 1.5 approaching I'd like to have the issue solved as soon as possible.
> Richard, when is deadline for core-patches?
Basically ASAP. The next release build is the start of next week but any
patches going into that need to have been run through an autobuilder
cycle first.
This is the first I've heard of the problem, other than some comments
about possible problems on irc. Can someone please open up a bug for
this and clearly describe what used to work and now doesn't and what the
possible fixes or workarounds are?
I don't think anyone likes regressions however if we don't have the open
bug, its very hard to track.
You say you've talked to Bruce/Jason but why didn't the discussion
happen on the mailing list? That way others may have been able to help
and now I could read back the list and see for myself what the issue is.
As it is, I simply don't know other than the need to set a new variable
which is hinted at above...
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-09-27 9:05 ` Richard Purdie
@ 2013-09-27 11:59 ` Andrea Adami
2013-09-27 12:33 ` Bruce Ashfield
2013-09-27 16:33 ` Jason Wessel
1 sibling, 1 reply; 10+ messages in thread
From: Andrea Adami @ 2013-09-27 11:59 UTC (permalink / raw)
To: Richard Purdie; +Cc: Openembedded-core
On Fri, Sep 27, 2013 at 11:05 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2013-09-27 at 10:07 +0200, Andrea Adami wrote:
>> On Sat, Aug 24, 2013 at 7:15 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Thu, 2013-08-22 at 18:04 -0500, Jason Wessel wrote:
>> >> This patch aims to fix the following two cases for the INITRAMFS generation.
>> >> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>> >> as core-image-minimal-initramfs. This allows building a base
>> >> image which always generates the needed initramfs image in one step
>> >> 2) Allow building a single binary which contains a kernel and
>> >> the initramfs.
>> >>
>> >> A key requirement of the initramfs is to be able to add kernel
>> >> modules. The current implementation of the INITRAMFS_IMAGE variable
>> >> has a circular dependency when using kernel modules in the initramfs
>> >> image.bb file that is caused by kernel.bbclass trying to build the
>> >> initramfs before the kernel's do_install rule.
>> >>
>> >> The solution for this problem is to have the kernel's
>> >> do_bundle_initramfs_image task depend on the do_rootfs from the
>> >> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
>> >> will also sets up dependencies to make the initramfs creation task run
>> >> last.
>> >>
>> >> The code to bundle the kernel and initramfs together has been added.
>> >> At a high level, all it is doing is invoking a second compilation of
>> >> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
>> >> to the generated initramfs from the image recipe.
>> >>
>> >> [YOCTO #4072]
>> >>
>> >> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>> >> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>> >
>> > I have a couple of things I'd really like to see get resolved here. One
>> > is below, the other is I'm worried about the packaged output differences
>> > since we can package the kernel into a package file and now its going to
>> > be different.
>> >
>> > I appreciate its a hard problem to solve but not impossible. Basically
>> > we move the package generation for that single package into a separate
>> > recipe and have it depend on the bundling task if/as/when needed. The
>> > bundle task stashes the kernel in the sysroot, the other recipe simply
>> > packages it. Its a little bit of a dance but should ensure we get
>> > everything consistent.
>> >
>> >
>> >> ---
>> >> meta/classes/image.bbclass | 12 ++++++
>> >> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
>> >> meta/conf/local.conf.sample | 20 +++++++++
>> >> 3 files changed, 116 insertions(+), 12 deletions(-)
>> >>
>> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> >> index 909702a..23967ec 100644
>> >> --- a/meta/classes/image.bbclass
>> >> +++ b/meta/classes/image.bbclass
>> >> @@ -130,6 +130,10 @@ python () {
>> >> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>> >>
>> >> check_image_features(d)
>> >> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
>> >> + if initramfs_image != "":
>> >> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
>> >> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
>> >> }
>> >>
>> >> #
>> >> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
>> >> do_package_write_rpm[noexec] = "1"
>> >>
>> >> addtask rootfs before do_build
>> >> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
>> >> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
>> >> +do_bundle_initramfs[nostamp] = "1"
>> >> +do_bundle_initramfs[noexec] = "1"
>> >> +do_bundle_initramfs () {
>> >> + :
>> >> +}
>> >> +addtask bundle_initramfs after do_rootfs
>> >> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> >> index e039dfc..8cf66ce 100644
>> >> --- a/meta/classes/kernel.bbclass
>> >> +++ b/meta/classes/kernel.bbclass
>> >> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
>> >> KERNEL_IMAGETYPE ?= "zImage"
>> >> INITRAMFS_IMAGE ?= ""
>> >> INITRAMFS_TASK ?= ""
>> >> +INITRAMFS_IMAGE_BUNDLE ?= ""
>> >>
>> >> python __anonymous () {
>> >> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
>> >> @@ -19,7 +20,15 @@ python __anonymous () {
>> >>
>> >> image = d.getVar('INITRAMFS_IMAGE', True)
>> >> if image:
>> >> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
>> >> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
>> >> +
>> >> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
>> >> + # The preferred method is to set INITRAMFS_IMAGE, because
>> >> + # this INITRAMFS_TASK has circular dependency problems
>> >> + # if the initramfs requires kernel modules
>> >> + image_task = d.getVar('INITRAMFS_TASK', True)
>> >> + if image_task:
>> >> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
>> >> }
>> >>
>> >> inherit kernel-arch deploy
>> >> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>> >>
>> >> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>> >>
>> >> +copy_initramfs() {
>> >> + echo "Copying initramfs into ./usr ..."
>> >> + # Find and use the first initramfs image archive type we find
>> >> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>> >> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>> >> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>> >> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
>> >> + case $img in
>> >> + *gz)
>> >> + echo "gzip decompressing image"
>> >> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> >> + break
>> >> + ;;
>> >> + *lzo)
>> >> + echo "lzo decompressing image"
>> >> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> >> + break
>> >> + ;;
>> >> + *lzma)
>> >> + echo "lzma decompressing image"
>> >> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> >> + break
>> >> + ;;
>> >> + *xz)
>> >> + echo "xz decompressing image"
>> >> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>> >> + break
>> >> + ;;
>> >> + esac
>> >> + fi
>> >> + done
>> >> + echo "Finished copy of initramfs into ./usr"
>> >> +}
>> >
>> > But what about my bzip2'd image? ;-)
>> >
>> > I'd suggest we rid of this and instead ensure that we're generating an
>> > uncompressed cpio image. The image generation code will happily sort
>> > that our for us if we ask it for that specific image type.
>> >
>> > I'd also wondered if we could remove INITRAMFS_TASK since its just going
>> > to confuse things and I'd prefer to maintain only one way of doing this
>> > if at all possible.
>> >
>> > Cheers,
>> >
>> > Richard
>>
>>
>> Though our observations, in an effort to solve Yocto #4072 the
>> patchset has been merged.
>> As side effect it broke the 'old style' creation of initramfs so some
>> recipes are now unbuildable.
>>
>> I'm in touch with Bruce and Jason and I have already a patch for
>> kernel.bbclass restoring the old functionality by adding one more
>> variable in each recipe: INITRAMFS_TASK.
>>
>> I'm pretty sure we could have built the same kind of images containing
>> modules in one pass before as well and still dislike the idea of
>> having to set a variable in local.conf to spread it to the kernel and
>> to the image (to any image...)
>>
>> With 1.5 approaching I'd like to have the issue solved as soon as possible.
>> Richard, when is deadline for core-patches?
>
> Basically ASAP. The next release build is the start of next week but any
> patches going into that need to have been run through an autobuilder
> cycle first.
>
> This is the first I've heard of the problem, other than some comments
> about possible problems on irc. Can someone please open up a bug for
> this and clearly describe what used to work and now doesn't and what the
> possible fixes or workarounds are?
>
> I don't think anyone likes regressions however if we don't have the open
> bug, its very hard to track.
>
> You say you've talked to Bruce/Jason but why didn't the discussion
> happen on the mailing list? That way others may have been able to help
> and now I could read back the list and see for myself what the issue is.
> As it is, I simply don't know other than the need to set a new variable
> which is hinted at above...
>
I'm waiting for feedback because my patch is just restoring some lines
of code in kernel_do_configure removed by the a.m. patch and should be
harmless wrt 'new-style' builds.
I don't know exactly which image has to be built for testing: I
successfully embedded core-image-base and its kernel+modules.
The point is, with the actual kernel.bbclass, it could *perhaps* be
necessary to add one more check before copying the cpio to the kernel
/usr.
- if [ ! -z "${INITRAMFS_IMAGE}" ]; then
+ if [ ! -z "${INITRAMFS_IMAGE}" -a ! -z "${INITRAMFS_TASK}" ]; then
I don't think so anyway, and personally dislike this second var
INITRAMFS_TASK, which would need to be put in every kernel recipe
asking for old-compat build.
Alternatively one could write in the recipe do_configure[depends] +=
"my-image:do_rootfs" but imho is better to handle that in
kernel.bbclass and just set INITRAMFS_IMAGE in the recipe.
Patch follows (prolly mangled): will be resent once agreed
From 8d399441b32ae9d64c6045ca99b3a80f67590f8b Mon Sep 17 00:00:00 2001
From: Andrea Adami <andrea.adami@gmail.com>
Date: Mon, 16 Sep 2013 20:10:14 +0200
Subject: [PATCH] kernel.bbclass: copy the initramfs cpio in
kernel_do_configure()
The straight build of a kernel with embedded image fails
on do_compile after
commmit 609d5a9ab9e58bb1c2bcc2145399fbc8b701b85a
kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and
bundling
To keep the old behavior kernel recipe must declare both
INITRAMFS_IMAGE and the respective INITRAMFS_TASK.
Signed-off-by: Andrea Adami <andrea.adami@gmail.com>
---
meta/classes/kernel.bbclass | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 8cf66ce..a534579 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -301,6 +301,14 @@ kernel_do_configure() {
cp "${WORKDIR}/defconfig" "${B}/.config"
fi
yes '' | oe_runmake oldconfig
+
+ if [ ! -z "${INITRAMFS_IMAGE}" -a ! -z "${INITRAMFS_TASK}" ]; then
+ for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
+ if [ -e
"${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
+ cp
"${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img"
initramfs.$img
+ fi
+ done
+ fi
}
do_savedefconfig() {
--
1.8.1.5
> Cheers,
>
> Richard
>
>
>
Sorry to put you in hurry :/
Cheers
Andrea
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-09-27 11:59 ` Andrea Adami
@ 2013-09-27 12:33 ` Bruce Ashfield
0 siblings, 0 replies; 10+ messages in thread
From: Bruce Ashfield @ 2013-09-27 12:33 UTC (permalink / raw)
To: Andrea Adami, Richard Purdie; +Cc: Openembedded-core
On 13-09-27 07:59 AM, Andrea Adami wrote:
> On Fri, Sep 27, 2013 at 11:05 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Fri, 2013-09-27 at 10:07 +0200, Andrea Adami wrote:
>>> On Sat, Aug 24, 2013 at 7:15 PM, Richard Purdie
>>> <richard.purdie@linuxfoundation.org> wrote:
>>>> On Thu, 2013-08-22 at 18:04 -0500, Jason Wessel wrote:
>>>>> This patch aims to fix the following two cases for the INITRAMFS generation.
>>>>> 1) Allow an image recipe to specify a paired INITRAMFS recipe such
>>>>> as core-image-minimal-initramfs. This allows building a base
>>>>> image which always generates the needed initramfs image in one step
>>>>> 2) Allow building a single binary which contains a kernel and
>>>>> the initramfs.
>>>>>
>>>>> A key requirement of the initramfs is to be able to add kernel
>>>>> modules. The current implementation of the INITRAMFS_IMAGE variable
>>>>> has a circular dependency when using kernel modules in the initramfs
>>>>> image.bb file that is caused by kernel.bbclass trying to build the
>>>>> initramfs before the kernel's do_install rule.
>>>>>
>>>>> The solution for this problem is to have the kernel's
>>>>> do_bundle_initramfs_image task depend on the do_rootfs from the
>>>>> INITRAMFS_IMAGE and not some intermediate point. The image.bbclass
>>>>> will also sets up dependencies to make the initramfs creation task run
>>>>> last.
>>>>>
>>>>> The code to bundle the kernel and initramfs together has been added.
>>>>> At a high level, all it is doing is invoking a second compilation of
>>>>> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
>>>>> to the generated initramfs from the image recipe.
>>>>>
>>>>> [YOCTO #4072]
>>>>>
>>>>> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>>>>> Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>>
>>>> I have a couple of things I'd really like to see get resolved here. One
>>>> is below, the other is I'm worried about the packaged output differences
>>>> since we can package the kernel into a package file and now its going to
>>>> be different.
>>>>
>>>> I appreciate its a hard problem to solve but not impossible. Basically
>>>> we move the package generation for that single package into a separate
>>>> recipe and have it depend on the bundling task if/as/when needed. The
>>>> bundle task stashes the kernel in the sysroot, the other recipe simply
>>>> packages it. Its a little bit of a dance but should ensure we get
>>>> everything consistent.
>>>>
>>>>
>>>>> ---
>>>>> meta/classes/image.bbclass | 12 ++++++
>>>>> meta/classes/kernel.bbclass | 96 +++++++++++++++++++++++++++++++++++++------
>>>>> meta/conf/local.conf.sample | 20 +++++++++
>>>>> 3 files changed, 116 insertions(+), 12 deletions(-)
>>>>>
>>>>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>>>>> index 909702a..23967ec 100644
>>>>> --- a/meta/classes/image.bbclass
>>>>> +++ b/meta/classes/image.bbclass
>>>>> @@ -130,6 +130,10 @@ python () {
>>>>> d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>>>>>
>>>>> check_image_features(d)
>>>>> + initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
>>>>> + if initramfs_image != "":
>>>>> + d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs" % d.getVar('PN', True))
>>>>> + d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs" % initramfs_image)
>>>>> }
>>>>>
>>>>> #
>>>>> @@ -629,3 +633,11 @@ do_package_write_deb[noexec] = "1"
>>>>> do_package_write_rpm[noexec] = "1"
>>>>>
>>>>> addtask rootfs before do_build
>>>>> +# Allow the kernel to be repacked with the initramfs and boot image file as a single file
>>>>> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
>>>>> +do_bundle_initramfs[nostamp] = "1"
>>>>> +do_bundle_initramfs[noexec] = "1"
>>>>> +do_bundle_initramfs () {
>>>>> + :
>>>>> +}
>>>>> +addtask bundle_initramfs after do_rootfs
>>>>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>>>>> index e039dfc..8cf66ce 100644
>>>>> --- a/meta/classes/kernel.bbclass
>>>>> +++ b/meta/classes/kernel.bbclass
>>>>> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
>>>>> KERNEL_IMAGETYPE ?= "zImage"
>>>>> INITRAMFS_IMAGE ?= ""
>>>>> INITRAMFS_TASK ?= ""
>>>>> +INITRAMFS_IMAGE_BUNDLE ?= ""
>>>>>
>>>>> python __anonymous () {
>>>>> kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
>>>>> @@ -19,7 +20,15 @@ python __anonymous () {
>>>>>
>>>>> image = d.getVar('INITRAMFS_IMAGE', True)
>>>>> if image:
>>>>> - d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
>>>>> + d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_rootfs')
>>>>> +
>>>>> + # NOTE: setting INITRAMFS_TASK is for backward compatibility
>>>>> + # The preferred method is to set INITRAMFS_IMAGE, because
>>>>> + # this INITRAMFS_TASK has circular dependency problems
>>>>> + # if the initramfs requires kernel modules
>>>>> + image_task = d.getVar('INITRAMFS_TASK', True)
>>>>> + if image_task:
>>>>> + d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
>>>>> }
>>>>>
>>>>> inherit kernel-arch deploy
>>>>> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>>>>>
>>>>> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>>>>>
>>>>> +copy_initramfs() {
>>>>> + echo "Copying initramfs into ./usr ..."
>>>>> + # Find and use the first initramfs image archive type we find
>>>>> + rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>>>>> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>>>>> + if [ -e "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>>>>> + cp ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
>>>>> + case $img in
>>>>> + *gz)
>>>>> + echo "gzip decompressing image"
>>>>> + gunzip -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>>> + break
>>>>> + ;;
>>>>> + *lzo)
>>>>> + echo "lzo decompressing image"
>>>>> + lzop -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>>> + break
>>>>> + ;;
>>>>> + *lzma)
>>>>> + echo "lzma decompressing image"
>>>>> + lzmash -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>>> + break
>>>>> + ;;
>>>>> + *xz)
>>>>> + echo "xz decompressing image"
>>>>> + xz -df ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>>>> + break
>>>>> + ;;
>>>>> + esac
>>>>> + fi
>>>>> + done
>>>>> + echo "Finished copy of initramfs into ./usr"
>>>>> +}
>>>>
>>>> But what about my bzip2'd image? ;-)
>>>>
>>>> I'd suggest we rid of this and instead ensure that we're generating an
>>>> uncompressed cpio image. The image generation code will happily sort
>>>> that our for us if we ask it for that specific image type.
>>>>
>>>> I'd also wondered if we could remove INITRAMFS_TASK since its just going
>>>> to confuse things and I'd prefer to maintain only one way of doing this
>>>> if at all possible.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>
>>>
>>> Though our observations, in an effort to solve Yocto #4072 the
>>> patchset has been merged.
>>> As side effect it broke the 'old style' creation of initramfs so some
>>> recipes are now unbuildable.
>>>
>>> I'm in touch with Bruce and Jason and I have already a patch for
>>> kernel.bbclass restoring the old functionality by adding one more
>>> variable in each recipe: INITRAMFS_TASK.
>>>
>>> I'm pretty sure we could have built the same kind of images containing
>>> modules in one pass before as well and still dislike the idea of
>>> having to set a variable in local.conf to spread it to the kernel and
>>> to the image (to any image...)
>>>
>>> With 1.5 approaching I'd like to have the issue solved as soon as possible.
>>> Richard, when is deadline for core-patches?
>>
>> Basically ASAP. The next release build is the start of next week but any
>> patches going into that need to have been run through an autobuilder
>> cycle first.
>>
>> This is the first I've heard of the problem, other than some comments
>> about possible problems on irc. Can someone please open up a bug for
>> this and clearly describe what used to work and now doesn't and what the
>> possible fixes or workarounds are?
>>
>> I don't think anyone likes regressions however if we don't have the open
>> bug, its very hard to track.
>>
>> You say you've talked to Bruce/Jason but why didn't the discussion
>> happen on the mailing list? That way others may have been able to help
>> and now I could read back the list and see for myself what the issue is.
>> As it is, I simply don't know other than the need to set a new variable
>> which is hinted at above...
>>
>
> I'm waiting for feedback because my patch is just restoring some lines
> of code in kernel_do_configure removed by the a.m. patch and should be
> harmless wrt 'new-style' builds.
> I don't know exactly which image has to be built for testing: I
> successfully embedded core-image-base and its kernel+modules.
>
> The point is, with the actual kernel.bbclass, it could *perhaps* be
> necessary to add one more check before copying the cpio to the kernel
> /usr.
>
> - if [ ! -z "${INITRAMFS_IMAGE}" ]; then
> + if [ ! -z "${INITRAMFS_IMAGE}" -a ! -z "${INITRAMFS_TASK}" ]; then
>
> I don't think so anyway, and personally dislike this second var
> INITRAMFS_TASK, which would need to be put in every kernel recipe
> asking for old-compat build.
>
> Alternatively one could write in the recipe do_configure[depends] +=
> "my-image:do_rootfs" but imho is better to handle that in
> kernel.bbclass and just set INITRAMFS_IMAGE in the recipe.
>
I'm back at this again today, yesterday was a complete write off for
me. Sorry about that. But I'm here to drive this to resolution now.
>
> Patch follows (prolly mangled): will be resent once agreed
>
> From 8d399441b32ae9d64c6045ca99b3a80f67590f8b Mon Sep 17 00:00:00 2001
> From: Andrea Adami <andrea.adami@gmail.com>
> Date: Mon, 16 Sep 2013 20:10:14 +0200
> Subject: [PATCH] kernel.bbclass: copy the initramfs cpio in
> kernel_do_configure()
>
> The straight build of a kernel with embedded image fails
> on do_compile after
>
> commmit 609d5a9ab9e58bb1c2bcc2145399fbc8b701b85a
> kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and
> bundling
>
> To keep the old behavior kernel recipe must declare both
> INITRAMFS_IMAGE and the respective INITRAMFS_TASK.
This definitely restores things in the short term, but I've got a
few things I tried yeterday to check on, and see if the builds
worked at all.
That being said, even the new changes I was working on have a
larger footprint than this, so at this point, I don't object to this.
As everyone recalls in the original threads, I was adamant that we
not break any existing workflows when trying to add this bundling
code.
I'll follow up shortly with a better response.
Cheers,
Bruce
>
> Signed-off-by: Andrea Adami <andrea.adami@gmail.com>
> ---
> meta/classes/kernel.bbclass | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 8cf66ce..a534579 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -301,6 +301,14 @@ kernel_do_configure() {
> cp "${WORKDIR}/defconfig" "${B}/.config"
> fi
> yes '' | oe_runmake oldconfig
> +
> + if [ ! -z "${INITRAMFS_IMAGE}" -a ! -z "${INITRAMFS_TASK}" ]; then
> + for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
> + if [ -e
> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
> + cp
> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img"
> initramfs.$img
> + fi
> + done
> + fi
> }
>
> do_savedefconfig() {
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling
2013-09-27 9:05 ` Richard Purdie
2013-09-27 11:59 ` Andrea Adami
@ 2013-09-27 16:33 ` Jason Wessel
1 sibling, 0 replies; 10+ messages in thread
From: Jason Wessel @ 2013-09-27 16:33 UTC (permalink / raw)
To: Richard Purdie; +Cc: Openembedded-core
On 09/27/2013 04:05 AM, Richard Purdie wrote:
> On Fri, 2013-09-27 at 10:07 +0200, Andrea Adami wrote:
>> On Sat, Aug 24, 2013 at 7:15 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> With 1.5 approaching I'd like to have the issue solved as soon as possible.
>> Richard, when is deadline for core-patches?
>
> Basically ASAP. The next release build is the start of next week but any
> patches going into that need to have been run through an autobuilder
> cycle first.
>
> This is the first I've heard of the problem, other than some comments
> about possible problems on irc. Can someone please open up a bug for
> this and clearly describe what used to work and now doesn't and what the
> possible fixes or workarounds are?
>
> I don't think anyone likes regressions however if we don't have the open
> bug, its very hard to track.
>
> You say you've talked to Bruce/Jason but why didn't the discussion
> happen on the mailing list? That way others may have been able to help
> and now I could read back the list and see for myself what the issue is.
> As it is, I simply don't know other than the need to set a new variable
> which is hinted at above...
>
What was needed most was a defect detailing the configuration setup. The reason I have not hit this problem nor any of the other autobuilders was that it required a very specific way to setup the recipes to depend on each other, but not circularly.
Andrea sent full details to Bruce, who sent them to me and I built it this morning. It took very little time to isolate the problem but it was one of those oh so subtle issues (patch is sent to oe-core). Perhaps in the future we can just get a Bugzilla opened right away. I don't like regressions either. :-)
Andrea replied to me in IRC, "I'll test the new patch soon".
Cheers,
Jason.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-09-27 16:33 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-22 23:04 [v2 PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling Jason Wessel
2013-08-23 6:16 ` Khem Raj
2013-08-23 12:56 ` Jason Wessel
2013-08-26 8:33 ` Andrea Adami
2013-08-24 17:15 ` Richard Purdie
2013-09-27 8:07 ` Andrea Adami
2013-09-27 9:05 ` Richard Purdie
2013-09-27 11:59 ` Andrea Adami
2013-09-27 12:33 ` Bruce Ashfield
2013-09-27 16:33 ` Jason Wessel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.