From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f52.google.com (mail-oi0-f52.google.com [209.85.218.52]) by mail.openembedded.org (Postfix) with ESMTP id 64FBC60745 for ; Mon, 14 Dec 2015 22:31:56 +0000 (UTC) Received: by oian133 with SMTP id n133so23476550oia.3 for ; Mon, 14 Dec 2015 14:31:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=LuMNbiOtIBm9YBbjUP8puXmxXZbJoSiFnKbSdHxf9lE=; b=toKLuKExIp2GrQNUjkOIdxAigg08T6PJx3+qmJzTC2skrMAzy6xUAQOLmo4kL4HWzk zosp5t2w8AtCFopQwxtCmABX12AIY9JHGBvSGGVvfT4Dt0np91d0WIIhML3I3U7HY/+2 y05wyG93UJEOrFzIRazcNgu7rYIDVCfTxdGU+JHCS2wD2YFP4jWGE5t4CeoWTA6wKuk/ Pu+zbQ2anlc9uiOTelkDG4xNN3Gvyvo67sMcdk/Q/a+jkUCLe7ftHkrji9P4SzQj3Cet JuRGKpPNF8d+55BdgkZQFVZHCUp4XroLGMq19Iw4dN+eAGZ7pLOk5zlQiFZC/+BUMJU8 fCHw== X-Received: by 10.202.241.11 with SMTP id p11mr25915392oih.51.1450132316831; Mon, 14 Dec 2015 14:31:56 -0800 (PST) Received: from harisdt.amer.corp.natinst.com ([130.164.62.171]) by smtp.googlemail.com with ESMTPSA id iq6sm15923354obb.0.2015.12.14.14.31.56 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 14 Dec 2015 14:31:56 -0800 (PST) To: Bruce Ashfield , Haris Okanovic References: <1450119998-2912-1-git-send-email-haris.okanovic@ni.com> From: Haris Okanovic Message-ID: <566F435B.1060208@gmail.com> Date: Mon, 14 Dec 2015 16:31:55 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Cc: Gratian Crisan , Patches and discussions about the oe-core layer Subject: Re: [PATCH] kernel: Add support for multiple kernel packages X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Dec 2015 22:32:01 -0000 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Hi Bruce, Thanks for taking time to look over our patch! On 12/14/2015 02:02 PM, Bruce Ashfield wrote: > > > On Mon, Dec 14, 2015 at 2:17 PM, Bruce Ashfield > > wrote: > > > > On Mon, Dec 14, 2015 at 2:06 PM, Haris Okanovic > > wrote: > > From: Gratian Crisan > > > Templetize kernel package name: > > Add a "weak" variable KERNEL_PACKAGE_NAME used as the base name > for kernel > packages. It defaults to the old hard-coded name 'kernel' but it > can be > redefined by recipes that provide additional kernel packages in > order > to avoid build conflicts. > > Change hard-coded 'kernel' references to KERNEL_PACKAGE_NAME in > kernel bbclass-es. > > Build alternate kernels from WORKDIR instead of STAGING_KERNEL_DIR: > > Prior to this change, kernel recipes would all fetch source to > STAGING_KERNEL_DIR as defined by bitbake/distro confs. This broke > parallel builds when more than one kernel recipes are defined in > the distribution, since they all attempted to fetch() and patch() > in a shared source dir. > > With this change, alternate kernel recipes fetch source into their > ${WORKDIR} so that they may build in parallel to each other and the > default kernel recipe, which still fetches to STAGING_KERNEL_DIR. > > > > But what's the system level use case for this capability ? Honestly, > it just > looks complicated .. and I'm not seeing exactly multiple kernels are > being > built in parallel in a single build. We've been forcing everything > explicitly > to be in a single, shared location .. and I see that this is both > complicating > and a possible source of errors. > > It's probably that I'm just misunderstanding the point of the change. > > If I had to guess, you are building recovery, or kexec/kdump kernels > along > side the main kernel ? > > > Aha. I see the debug kernel reference in the comments (as one example). > We should have made this a little more clear in the change description. I added a short summary at the top to document our use case. > As a follow up. If these kernel's are building via kernel.bbclass, how > are you > avoiding the artifacts all ending up in STAGING_KERNEL_BUILDDIR ? I did > a quick scan, and couldn't see if it was being considered. > Good catch. It looks like I missed that one. I modified kernel.bbclass to override STAGING_KERNEL_DIR and STAGING_KERNEL_BUILDDIR directly when KERNEL_PACKAGE_NAME != kernel, and leave S unchanged. Built 2 kernel recipes side-by-side and verified source and artifacts are where they should be: work-shared dir for default kernel, work dir for other. I'll post a V2 patch shortly. > Bruce > > > Cheers, > > Bruce > > > Testing: > > Built linux-yocto-4.1.13 for qemux86 and verified it produced kernel > image and modules packages. Added kernel recipe with non-default > KERNEL_PACKAGE_NAME and verified it produces kernel image and > modules > packages with alternate name next to default kernel. > > Signed-off-by: Gratian Crisan > > Signed-off-by: Haris Okanovic > > Coauthored-by: Haris Okanovic > > Coauthored-by: Josh Hernstrom > > Natinst-ReviewBoard-ID: 120348 > Natinst-ReviewBoard-ID: 120447 > --- > meta/classes/kernel-module-split.bbclass | 9 ++-- > meta/classes/kernel.bbclass | 71 > ++++++++++++++++++-------------- > meta/conf/documentation.conf | 1 + > meta/recipes-kernel/linux/linux-dtb.inc | 2 +- > 4 files changed, 49 insertions(+), 34 deletions(-) > > diff --git a/meta/classes/kernel-module-split.bbclass > b/meta/classes/kernel-module-split.bbclass > index e1a70e6..7415ec8 100644 > --- a/meta/classes/kernel-module-split.bbclass > +++ b/meta/classes/kernel-module-split.bbclass > @@ -28,7 +28,7 @@ do_install_append() { > > PACKAGESPLITFUNCS_prepend = "split_kernel_module_packages " > > -KERNEL_MODULES_META_PACKAGE ?= "kernel-modules" > +KERNEL_MODULES_META_PACKAGE ?= "${KERNEL_PACKAGE_NAME}-modules" > > python split_kernel_module_packages () { > import re > @@ -179,14 +179,17 @@ python split_kernel_module_packages () { > # Avoid automatic -dev recommendations for modules > ending with -dev. > d.setVarFlag('RRECOMMENDS_' + pkg, 'nodeprrecs', 1) > > + kernel_package_name = d.getVar("KERNEL_PACKAGE_NAME", True) > + kernel_version = d.getVar("KERNEL_VERSION", True) > + > module_deps = parse_depmod() > module_regex = '^(.*)\.k?o$' > - module_pattern = 'kernel-module-%s' > + module_pattern = '%s-module-%%s' % kernel_package_name > > postinst = d.getVar('pkg_postinst_modules', True) > postrm = d.getVar('pkg_postrm_modules', True) > > - modules = do_split_packages(d, root='/lib/modules', > file_regex=module_regex, output_pattern=module_pattern, > description='%s kernel module', postinst=postinst, > postrm=postrm, recursive=True, hook=frob_metadata, > extra_depends='kernel-%s' % (d.getVar("KERNEL_VERSION", True))) > + modules = do_split_packages(d, root='/lib/modules', > file_regex=module_regex, output_pattern=module_pattern, > description='%s kernel module', postinst=postinst, > postrm=postrm, recursive=True, hook=frob_metadata, > extra_depends='%s-%s' % (kernel_package_name, kernel_version)) > if modules: > metapkg = d.getVar('KERNEL_MODULES_META_PACKAGE', True) > d.appendVar('RDEPENDS_' + metapkg, ' '+' '.join(modules)) > diff --git a/meta/classes/kernel.bbclass > b/meta/classes/kernel.bbclass > index b75a462..cc27364 100644 > --- a/meta/classes/kernel.bbclass > +++ b/meta/classes/kernel.bbclass > @@ -3,7 +3,6 @@ inherit linux-kernel-base kernel-module-split > PROVIDES += "virtual/kernel" > DEPENDS += "virtual/${TARGET_PREFIX}binutils > virtual/${TARGET_PREFIX}gcc kmod-native depmodwrapper-cross > bc-native" > > -S = "${STAGING_KERNEL_DIR}" > B = "${WORKDIR}/build" > KBUILD_OUTPUT = "${B}" > OE_TERMINAL_EXPORTS += "KBUILD_OUTPUT" > @@ -11,6 +10,7 @@ OE_TERMINAL_EXPORTS += "KBUILD_OUTPUT" > # we include gcc above, we dont need virtual/libc > INHIBIT_DEFAULT_DEPS = "1" > > +KERNEL_PACKAGE_NAME ??= "kernel" > KERNEL_IMAGETYPE ?= "zImage" > INITRAMFS_IMAGE ?= "" > INITRAMFS_TASK ?= "" > @@ -34,6 +34,17 @@ python __anonymous () { > image_task = d.getVar('INITRAMFS_TASK', True) > if image_task: > d.appendVarFlag('do_configure', 'depends', ' > ${INITRAMFS_TASK}') > + > + # The default kernel recipe fetches it's source to > + # STAGING_KERNEL_DIR as defined by bitbake/distro confs. > + # Alternate kernel recipes (E.g. debug kernels) fetch > source into > + # their work dir so that they may build in parallel. > + if d.getVar("KERNEL_PACKAGE_NAME", True) == "kernel": > + d.setVar("S", d.getVar("STAGING_KERNEL_DIR", True)) > + else: > + workdir = d.getVar("WORKDIR", True) > + kernelSourceDir = os.path.join(workdir, 'kernel-source') > + d.setVar("S", kernelSourceDir) > } > > # Here we pull in all various kernel image types which we support. > @@ -79,9 +90,9 @@ base_do_unpack_append () { > > inherit kernel-arch deploy > > -PACKAGES_DYNAMIC += "^kernel-module-.*" > -PACKAGES_DYNAMIC += "^kernel-image-.*" > -PACKAGES_DYNAMIC += "^kernel-firmware-.*" > +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-module-.*" > +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-image-.*" > +PACKAGES_DYNAMIC += "^${KERNEL_PACKAGE_NAME}-firmware-.*" > > export OS = "${TARGET_OS}" > export CROSS_COMPILE = "${TARGET_PREFIX}" > @@ -270,9 +281,9 @@ do_shared_workdir_setscene () { > > emit_depmod_pkgdata() { > # Stash data for depmod > - install -d ${PKGDESTWORK}/kernel-depmod/ > - echo "${KERNEL_VERSION}" > > ${PKGDESTWORK}/kernel-depmod/kernel-abiversion > - cp ${B}/System.map > ${PKGDESTWORK}/kernel-depmod/System.map-${KERNEL_VERSION} > + install -d ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/ > + echo "${KERNEL_VERSION}" > > ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/${KERNEL_PACKAGE_NAME}-abiversion > + cp ${B}/System.map > ${PKGDESTWORK}/${KERNEL_PACKAGE_NAME}-depmod/System.map-${KERNEL_VERSION} > } > > PACKAGEFUNCS += "emit_depmod_pkgdata" > @@ -287,7 +298,7 @@ do_shared_workdir () { > # Store the kernel version in sysroots for > module-base.bbclass > # > > - echo "${KERNEL_VERSION}" > $kerneldir/kernel-abiversion > + echo "${KERNEL_VERSION}" > > $kerneldir/${KERNEL_PACKAGE_NAME}-abiversion > > # Copy files required for module builds > cp System.map $kerneldir/System.map-${KERNEL_VERSION} > @@ -361,28 +372,28 @@ EXPORT_FUNCTIONS do_compile do_install > do_configure > > # kernel-base becomes kernel-${KERNEL_VERSION} > # kernel-image becomes kernel-image-${KERNEL_VERISON} > -PACKAGES = "kernel kernel-base kernel-vmlinux kernel-image > kernel-dev kernel-modules" > +PACKAGES = "${KERNEL_PACKAGE_NAME} ${KERNEL_PACKAGE_NAME}-base > ${KERNEL_PACKAGE_NAME}-vmlinux ${KERNEL_PACKAGE_NAME}-image > ${KERNEL_PACKAGE_NAME}-dev ${KERNEL_PACKAGE_NAME}-modules" > FILES_${PN} = "" > -FILES_kernel-base = > "/lib/modules/${KERNEL_VERSION}/modules.order > /lib/modules/${KERNEL_VERSION}/modules.builtin" > -FILES_kernel-image = "/boot/${KERNEL_IMAGETYPE}*" > -FILES_kernel-dev = "/boot/System.map* /boot/Module.symvers* > /boot/config* ${KERNEL_SRC_PATH} > /lib/modules/${KERNEL_VERSION}/build" > -FILES_kernel-vmlinux = "/boot/vmlinux*" > -FILES_kernel-modules = "" > -RDEPENDS_kernel = "kernel-base" > +FILES_${KERNEL_PACKAGE_NAME}-base = > "/lib/modules/${KERNEL_VERSION}/modules.order > /lib/modules/${KERNEL_VERSION}/modules.builtin" > +FILES_${KERNEL_PACKAGE_NAME}-image = "/boot/${KERNEL_IMAGETYPE}*" > +FILES_${KERNEL_PACKAGE_NAME}-dev = "/boot/System.map* > /boot/Module.symvers* /boot/config* ${KERNEL_SRC_PATH} > /lib/modules/${KERNEL_VERSION}/build" > +FILES_${KERNEL_PACKAGE_NAME}-vmlinux = "/boot/vmlinux*" > +FILES_${KERNEL_PACKAGE_NAME}-modules = "" > +RDEPENDS_${KERNEL_PACKAGE_NAME} = "${KERNEL_PACKAGE_NAME}-base" > # Allow machines to override this dependency if kernel image > files are > # not wanted in images as standard > -RDEPENDS_kernel-base ?= "kernel-image" > -PKG_kernel-image = > "kernel-image-${@legitimize_package_name('${KERNEL_VERSION}')}" > -RDEPENDS_kernel-image += > "${@base_conditional('KERNEL_IMAGETYPE', 'vmlinux', > 'kernel-vmlinux', '', d)}" > -PKG_kernel-base = > "kernel-${@legitimize_package_name('${KERNEL_VERSION}')}" > -RPROVIDES_kernel-base += "kernel-${KERNEL_VERSION}" > -ALLOW_EMPTY_kernel = "1" > -ALLOW_EMPTY_kernel-base = "1" > -ALLOW_EMPTY_kernel-image = "1" > -ALLOW_EMPTY_kernel-modules = "1" > -DESCRIPTION_kernel-modules = "Kernel modules meta package" > - > -pkg_postinst_kernel-base () { > +RDEPENDS_${KERNEL_PACKAGE_NAME}-base ?= > "${KERNEL_PACKAGE_NAME}-image" > +PKG_${KERNEL_PACKAGE_NAME}-image = > "${KERNEL_PACKAGE_NAME}-image-${@legitimize_package_name('${KERNEL_VERSION}')}" > +RDEPENDS_${KERNEL_PACKAGE_NAME}-image += > "${@base_conditional('KERNEL_IMAGETYPE', 'vmlinux', > '${KERNEL_PACKAGE_NAME}-vmlinux', '', d)}" > +PKG_${KERNEL_PACKAGE_NAME}-base = > "${KERNEL_PACKAGE_NAME}-${@legitimize_package_name('${KERNEL_VERSION}')}" > +RPROVIDES_${KERNEL_PACKAGE_NAME}-base += > "${KERNEL_PACKAGE_NAME}-${KERNEL_VERSION}" > +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME} = "1" > +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-base = "1" > +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-image = "1" > +ALLOW_EMPTY_${KERNEL_PACKAGE_NAME}-modules = "1" > +DESCRIPTION_${KERNEL_PACKAGE_NAME}-modules = "Kernel modules > meta package" > + > +pkg_postinst_${KERNEL_PACKAGE_NAME}-base () { > if [ ! -e "$D/lib/modules/${KERNEL_VERSION}" ]; then > mkdir -p $D/lib/modules/${KERNEL_VERSION} > fi > @@ -393,18 +404,18 @@ pkg_postinst_kernel-base () { > fi > } > > -pkg_postinst_kernel-image () { > +pkg_postinst_${KERNEL_PACKAGE_NAME}-image () { > update-alternatives --install > /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE} ${KERNEL_IMAGETYPE} > /${KERNEL_IMAGEDEST}/${KERNEL_IMAGETYPE}-${KERNEL_VERSION} > ${KERNEL_PRIORITY} || true > } > > -pkg_postrm_kernel-image () { > +pkg_postrm_${KERNEL_PACKAGE_NAME}-image () { > update-alternatives --remove ${KERNEL_IMAGETYPE} > ${KERNEL_IMAGETYPE}-${KERNEL_VERSION} || true > } > > PACKAGESPLITFUNCS_prepend = "split_kernel_packages " > > python split_kernel_packages () { > - do_split_packages(d, root='/lib/firmware', > file_regex='^(.*)\.(bin|fw|cis|dsp)$', > output_pattern='kernel-firmware-%s', description='Firmware for > %s', recursive=True, extra_depends='') > + do_split_packages(d, root='/lib/firmware', > file_regex='^(.*)\.(bin|fw|cis|dsp)$', > output_pattern='${KERNEL_PACKAGE_NAME}-firmware-%s', > description='Firmware for %s', recursive=True, extra_depends='') > } > > do_strip() { > diff --git a/meta/conf/documentation.conf > b/meta/conf/documentation.conf > index 1b5c071..fb5e03f 100644 > --- a/meta/conf/documentation.conf > +++ b/meta/conf/documentation.conf > @@ -250,6 +250,7 @@ KERNEL_FEATURES[doc] = "Includes additional > metadata from the Yocto Project kern > KERNEL_IMAGETYPE[doc] = "The type of kernel to build for a > device, usually set by the machine configuration files and > defaults to 'zImage'." > KERNEL_MODULE_AUTOLOAD[doc] = "Lists kernel modules that need > to be auto-loaded during boot" > KERNEL_MODULE_PROBECONF[doc] = "Lists kernel modules for which > the build system expects to find module_conf_* values that > specify configuration for each of the modules" > +KERNEL_PACKAGE_NAME[doc] = "Name prefix for kernel packages. > Defaults to 'kernel'." > KERNEL_PATH[doc] = "The location of the kernel sources. This > variable is set to the value of the STAGING_KERNEL_DIR within > the module class (module.bbclass)." > KERNEL_SRC[doc] = "The location of the kernel sources. This > variable is set to the value of the STAGING_KERNEL_DIR within > the module class (module.bbclass)." > KFEATURE_DESCRIPTION[doc] = "Provides a short description of a > configuration fragment. You use this variable in the .scc file > that describes a configuration fragment file." > diff --git a/meta/recipes-kernel/linux/linux-dtb.inc > b/meta/recipes-kernel/linux/linux-dtb.inc > index 772adcb..c27a3cb 100644 > --- a/meta/recipes-kernel/linux/linux-dtb.inc > +++ b/meta/recipes-kernel/linux/linux-dtb.inc > @@ -2,7 +2,7 @@ > FILES_kernel-devicetree = "/${KERNEL_IMAGEDEST}/devicetree*" > > python __anonymous () { > - d.appendVar("PACKAGES", " kernel-devicetree") > + d.appendVar("PACKAGES", " ${KERNEL_PACKAGE_NAME}-devicetree") > } > > normalize_dtb () { > -- > 2.6.2 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > > > -- > "Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end" > > > > > -- > "Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end" > > -- Haris