* [PATCHv5 1/8] image-artifact-names: introduce new bbclass and move some variables into it
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 2/8] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Martin Jansa
` (8 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* similar to kernel-artifact-names for other recipes/bbclasses which
need to use some deployed artifacts
* bitbake.conf: move IMAGE_BASENAME, IMAGE_VERSION_SUFFIX, IMAGE_NAME,
IMAGE_LINK_NAME variables
* image_types.bbclass: move IMAGE_NAME_SUFFIX variable
* currently IMAGE_NAME_SUFFIX is used only by image.bbclass,
image_types.bbclass and meta/recipes-core/images/build-appliance-image_15.0.0.bb
but if it's needed by some recipe which isn't itself an image, then
it's useful in bitbake.conf, e.g. we have a recipe for creating
VirtualBox appliances which combines .wic.vmdk with .ovf file to
create .zip with appliance, but for that we need the filename of
.wic.vmdk which now contains IMAGE_NAME_SUFFIX
https://github.com/webOS-ports/meta-webos-ports/blob/4980ce52a43ac6897657602810313af359f0b839/meta-luneos/recipes-core/images/luneos-emulator-appliance.inc#L24
* we were hardcoding .rootfs suffix where needed, but for quite long
time it's configurable with IMAGE_NAME_SUFFIX since:
commit 380ee36811939d947024bf78de907e3c071b834f
Author: Patrick Ohly <patrick.ohly@intel.com>
Date: Mon Mar 7 18:07:52 2016 +0100
image creation: allow overriding .rootfs suffix
and might not match with hardcoded .rootfs, so make it easier to
use IMAGE_NAME_SUFFIX where needed even without inheritting whole
image_types.bbclass
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/buildhistory.bbclass | 2 ++
meta/classes/image-artifact-names.bbclass | 15 +++++++++++++++
meta/classes/image-live.bbclass | 2 +-
meta/classes/image_types.bbclass | 9 ++-------
meta/classes/kernel-artifact-names.bbclass | 8 ++++++++
meta/classes/qemuboot.bbclass | 2 ++
meta/classes/rootfs-postcommands.bbclass | 2 ++
meta/classes/testimage.bbclass | 2 ++
meta/conf/bitbake.conf | 5 -----
9 files changed, 34 insertions(+), 13 deletions(-)
create mode 100644 meta/classes/image-artifact-names.bbclass
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index affdf272d7..cabdb9b935 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -7,6 +7,8 @@
# Copyright (C) 2007-2011 Koen Kooi <koen@openembedded.org>
#
+inherit image-artifact-names
+
BUILDHISTORY_FEATURES ?= "image package sdk"
BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
diff --git a/meta/classes/image-artifact-names.bbclass b/meta/classes/image-artifact-names.bbclass
new file mode 100644
index 0000000000..5ab8f1b7aa
--- /dev/null
+++ b/meta/classes/image-artifact-names.bbclass
@@ -0,0 +1,15 @@
+##################################################################
+# Specific image creation and rootfs population info.
+##################################################################
+
+IMAGE_BASENAME = "${PN}"
+IMAGE_VERSION_SUFFIX = "-${DATETIME}"
+IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
+IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
+IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
+
+# IMAGE_NAME is the base name for everything produced when building images.
+# The actual image that contains the rootfs has an additional suffix (.rootfs
+# by default) followed by additional suffices which describe the format (.ext4,
+# .ext4.xz, etc.).
+IMAGE_NAME_SUFFIX ??= ".rootfs"
diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass
index 54058b350d..9ea5ddc312 100644
--- a/meta/classes/image-live.bbclass
+++ b/meta/classes/image-live.bbclass
@@ -22,7 +22,7 @@
# ${HDDIMG_ID} - FAT image volume-id
# ${ROOTFS} - indicates a filesystem image to include as the root filesystem (optional)
-inherit live-vm-common
+inherit live-vm-common image-artifact-names
do_bootimg[depends] += "dosfstools-native:do_populate_sysroot \
mtools-native:do_populate_sysroot \
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index f82f1d8862..c512bba405 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -1,9 +1,3 @@
-# IMAGE_NAME is the base name for everything produced when building images.
-# The actual image that contains the rootfs has an additional suffix (.rootfs
-# by default) followed by additional suffices which describe the format (.ext4,
-# .ext4.xz, etc.).
-IMAGE_NAME_SUFFIX ??= ".rootfs"
-
# The default aligment of the size of the rootfs is set to 1KiB. In case
# you're using the SD card emulation of a QEMU system simulator you may
# set this value to 2048 (2MiB alignment).
@@ -231,7 +225,8 @@ IMAGE_CMD_f2fs () {
EXTRA_IMAGECMD = ""
-inherit siteinfo kernel-arch
+inherit siteinfo kernel-arch image-artifact-names
+
JFFS2_ENDIANNESS ?= "${@oe.utils.conditional('SITEINFO_ENDIANNESS', 'le', '-l', '-b', d)}"
JFFS2_ERASEBLOCK ?= "0x40000"
EXTRA_IMAGECMD_jffs2 ?= "--pad ${JFFS2_ENDIANNESS} --eraseblock=${JFFS2_ERASEBLOCK} --no-cleanmarkers"
diff --git a/meta/classes/kernel-artifact-names.bbclass b/meta/classes/kernel-artifact-names.bbclass
index bbeecba7bd..a65cdddb3e 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -1,3 +1,11 @@
+##################################################################
+# Specific kernel creation info
+# for recipes/bbclasses which need to reuse some of the kernel
+# artifacts, but aren't kernel recipes themselves
+##################################################################
+
+inherit image-artifact-names
+
KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
diff --git a/meta/classes/qemuboot.bbclass b/meta/classes/qemuboot.bbclass
index 15a9e63f2b..694798ad2b 100644
--- a/meta/classes/qemuboot.bbclass
+++ b/meta/classes/qemuboot.bbclass
@@ -67,6 +67,8 @@ QB_NETWORK_DEVICE ?= "-device virtio-net-pci,netdev=net0,mac=@MAC@"
# This should be kept align with ROOT_VM
QB_DRIVE_TYPE ?= "/dev/sd"
+inherit image-artifact-names
+
# Create qemuboot.conf
addtask do_write_qemuboot_conf after do_rootfs before do_image
IMGDEPLOYDIR ?= "${WORKDIR}/deploy-${PN}-image-complete"
diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index 2f171836fa..14a0cf4af0 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -39,6 +39,8 @@ ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd"
ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;'
+inherit image-artifact-names
+
# Sort the user and group entries in /etc by ID in order to make the content
# deterministic. Package installs are not deterministic, causing the ordering
# of entries to change between builds. In case that this isn't desired,
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 844ed87944..a1d0ddbb47 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -3,6 +3,8 @@
# Released under the MIT license (see COPYING.MIT)
inherit metadata_scm
+inherit image-artifact-names
+
# testimage.bbclass enables testing of qemu images using python unittests.
# Most of the tests are commands run on target image over ssh.
# To use it add testimage to global inherit and call your target image with -c testimage
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 62b5466b71..3c1e7589e6 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -449,11 +449,6 @@ STAGING_KERNEL_BUILDDIR = "${TMPDIR}/work-shared/${MACHINE}/kernel-build-artifac
##################################################################
IMAGE_ROOTFS = "${WORKDIR}/rootfs"
-IMAGE_BASENAME = "${PN}"
-IMAGE_VERSION_SUFFIX = "-${DATETIME}"
-IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
-IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
-IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
# This option allows for a percentage overage of the actual image size rather than a
# fixed extra space, this is space needed for initial startup and basic operations.
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 2/8] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
2020-01-14 9:12 ` [PATCHv5 1/8] image-artifact-names: introduce new bbclass and move some variables into it Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 3/8] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME Martin Jansa
` (7 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* just RFC, the part for images isn't finished yet and there is
still some issue with DATETIME when kernel artifacts are used
from sstate, this is just to validate the idea behind
[YOCTO #12937] before finishing the implementation (it's already
finished and used by various LGE builds, but having simpler
way of doing it directly in oe-core mighe be useful for others).
* move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME
that way e.g. kernel.do_deploy can be reused from sstate to
provide "version-less" artifacts and then very fast
do_deploy_links task just adds links with consistent suffixes
(by default the version from the recipe but could be easily set
to e.g. some release name when building some products).
* create hard links instead of symlinks, so that whatever version
the filename says is really there
* some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file
to be removed first or they might either append or update the
content of the image instead of creating new image file from
scratch - I have seen this only with one proprietary format we
generate with our own tool, so hopefully this isn't very common
* this is basically the mechanism are using in webOS with
WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from
jenkins job and then all artifacts (images as well as corresponding
kernel files) have the same version string)
* without this, you can still easily set the variables to contain
the version from jenkins job (excluded from sstate signature like
DATETIME currently is to prevent rebuilding it everytime even when
the content didn't change) but then when kernel is reused from sstate
you can have version 1.0 used on kernel artifacts and 2.0 on image
artifacts.
* if you don't exclude the version string with vardepsexclude, then
you get the right version in the filenames but for cost of
re-executing do_deploy every single time, which with rm_work will
cause all kernel tasks to be re-executed (together with everything
which depends on it like external modules etc).
* the implementation "from outside" is a bit tricky as shown in webOS
OSE, because first you need to reverse the meaning of IMAGE_NAME
and IMAGE_LINK_NAME like here, but also replace all symlinks with
hardlinks and then adjust all recipes/bbclasses to depend on our
do_deploy_fixup task instead of the original do_deploy
see the variable modifications:
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65
and then various bbclasses to hook do_webos_deploy_fixup task creating
the hardlinks for possible artifacts:
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/kernel.bbclass
https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/image.bbclass
so hopefully with all these changes in oe-core other project can
achieve the same just by setting one variable IMAGE_VERSION_SUFFIX
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
kernel
---
meta/classes/cve-check.bbclass | 4 +-
meta/classes/image-artifact-names.bbclass | 4 +-
meta/classes/image.bbclass | 10 ++---
meta/classes/kernel-artifact-names.bbclass | 4 +-
meta/classes/kernel-devicetree.bbclass | 21 +++++++++--
meta/classes/kernel.bbclass | 43 ++++++++++++++++------
meta/classes/qemuboot.bbclass | 2 +-
meta/classes/rootfs-postcommands.bbclass | 4 +-
8 files changed, 63 insertions(+), 29 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 74124364b2..f266bf2489 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -102,10 +102,10 @@ python cve_check_write_rootfs_manifest () {
if manifest_name and os.path.exists(manifest_name):
manifest_link = os.path.join(deploy_dir, "%s.cve" % link_name)
- # If we already have another manifest, update symlinks
+ # If we already have another manifest, update hardlinks
if os.path.exists(os.path.realpath(manifest_link)):
os.remove(manifest_link)
- os.symlink(os.path.basename(manifest_name), manifest_link)
+ os.link(manifest_name, manifest_link)
bb.plain("Image CVE report stored in: %s" % manifest_name)
}
diff --git a/meta/classes/image-artifact-names.bbclass b/meta/classes/image-artifact-names.bbclass
index 5ab8f1b7aa..d5ba035f5a 100644
--- a/meta/classes/image-artifact-names.bbclass
+++ b/meta/classes/image-artifact-names.bbclass
@@ -5,8 +5,8 @@
IMAGE_BASENAME = "${PN}"
IMAGE_VERSION_SUFFIX = "-${DATETIME}"
IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
-IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
-IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
+IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}"
+IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}"
# IMAGE_NAME is the base name for everything produced when building images.
# The actual image that contains the rootfs has an additional suffix (.rootfs
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index c2824395c9..363552e7cf 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -589,14 +589,14 @@ python create_symlinks() {
return
for type in subimages:
dst = os.path.join(deploy_dir, link_name + "." + type)
- src = img_name + imgsuffix + type
- if os.path.exists(os.path.join(deploy_dir, src)):
- bb.note("Creating symlink: %s -> %s" % (dst, src))
+ src = os.path.join(deploy_dir, img_name + imgsuffix + type)
+ if os.path.exists(src):
+ bb.note("Creating hardlink: %s -> %s" % (dst, src))
if os.path.islink(dst):
os.remove(dst)
- os.symlink(src, dst)
+ os.link(src, dst)
else:
- bb.note("Skipping symlink, source does not exist: %s -> %s" % (dst, src))
+ bb.note("Skipping hardlink, source does not exist: %s -> %s" % (dst, src))
}
MULTILIBRE_ALLOW_REP =. "${base_bindir}|${base_sbindir}|${bindir}|${sbindir}|${libexecdir}|${sysconfdir}|${nonarch_base_libdir}/udev|/lib/modules/[^/]*/modules.*|"
diff --git a/meta/classes/kernel-artifact-names.bbclass b/meta/classes/kernel-artifact-names.bbclass
index a65cdddb3e..529e0c565e 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -6,8 +6,8 @@
inherit image-artifact-names
-KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
-KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
+KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}"
+KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
diff --git a/meta/classes/kernel-devicetree.bbclass b/meta/classes/kernel-devicetree.bbclass
index 522c46575d..91f4a30232 100644
--- a/meta/classes/kernel-devicetree.bbclass
+++ b/meta/classes/kernel-devicetree.bbclass
@@ -74,19 +74,32 @@ do_deploy_append() {
install -d $deployDir
install -m 0644 ${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext
- ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do
if [ "$type" = "zImage" ] && [ "${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
cat ${D}/${KERNEL_IMAGEDEST}/$type \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
> $deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
- ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
- $deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
if [ -e "${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
cat ${KERNEL_OUTPUT_DIR}/${type}.initramfs \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
> $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
- ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+ fi
+ fi
+ done
+ done
+}
+do_deploy_links_append() {
+ for dtbf in ${KERNEL_DEVICETREE}; do
+ dtb=`normalize_dtb "$dtbf"`
+ dtb_ext=${dtb##*.}
+ dtb_base_name=`basename $dtb .$dtb_ext`
+ ln -vf $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+ for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do
+ if [ "$type" = "zImage" ] && [ "${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
+ ln -vf $deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+ $deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ if [ -e "${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
+ ln -vf $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
fi
fi
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 750988f4e5..954b761094 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -686,17 +686,13 @@ kernel_do_deploy() {
fi
for imageType in ${KERNEL_IMAGETYPES} ; do
- base_name=${imageType}-${KERNEL_IMAGE_NAME}
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} $deployDir/${base_name}.bin
- symlink_name=${imageType}-${KERNEL_IMAGE_LINK_NAME}
- ln -sf ${base_name}.bin $deployDir/${symlink_name}.bin
- ln -sf ${base_name}.bin $deployDir/${imageType}
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin
+ ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}
done
if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
mkdir -p ${D}${root_prefix}/lib
tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib
- ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
fi
if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
@@ -704,10 +700,7 @@ kernel_do_deploy() {
if [ "$imageType" = "fitImage" ] ; then
continue
fi
- initramfs_base_name=${imageType}-${INITRAMFS_NAME}
- initramfs_symlink_name=${imageType}-${INITRAMFS_LINK_NAME}
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${initramfs_base_name}.bin
- ln -sf ${initramfs_base_name}.bin $deployDir/${initramfs_symlink_name}.bin
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${imageType}-${INITRAMFS_NAME}.bin
done
fi
}
@@ -717,7 +710,35 @@ do_deploy[prefuncs] += "package_get_auto_pr"
addtask deploy after do_populate_sysroot do_packagedata
-EXPORT_FUNCTIONS do_deploy
+kernel_do_deploy_links() {
+ deployDir="${DEPLOY_DIR_IMAGE}"
+ if [ -n "${KERNEL_DEPLOYSUBDIR}" ]; then
+ deployDir="${DEPLOY_DIR_IMAGE}/${KERNEL_DEPLOYSUBDIR}"
+ mkdir "$deployDir"
+ fi
+
+ for imageType in ${KERNEL_IMAGETYPES} ; do
+ ln -vf $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}-${KERNEL_IMAGE_LINK_NAME}.bin
+ done
+
+ if [ ${MODULE_TARBALL_DEPLOY} = "1" -a -f $deployDir/modules-${MODULE_TARBALL_NAME}.tgz ] ; then
+ ln -vf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
+ fi
+
+ if [ ! -z "${INITRAMFS_IMAGE}" -a "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
+ for imageType in ${KERNEL_IMAGETYPES} ; do
+ if [ "$imageType" = "fitImage" ] ; then
+ continue
+ fi
+ ln -vf $deployDir/${imageType}-${INITRAMFS_NAME}.bin $deployDir/${imageType}-${INITRAMFS_LINK_NAME}.bin
+ done
+ fi
+}
+do_deploy_links[prefuncs] += "package_get_auto_pr"
+
+addtask deploy_links after do_deploy before do_build
+
+EXPORT_FUNCTIONS do_deploy do_deploy_links
# Add using Device Tree support
inherit kernel-devicetree
diff --git a/meta/classes/qemuboot.bbclass b/meta/classes/qemuboot.bbclass
index 694798ad2b..fbcd1f8098 100644
--- a/meta/classes/qemuboot.bbclass
+++ b/meta/classes/qemuboot.bbclass
@@ -122,5 +122,5 @@ python do_write_qemuboot_conf() {
if qemuboot_link != qemuboot:
if os.path.lexists(qemuboot_link):
os.remove(qemuboot_link)
- os.symlink(os.path.basename(qemuboot), qemuboot_link)
+ os.link(qemuboot, qemuboot_link)
}
diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index 14a0cf4af0..14fe0d2f80 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -271,7 +271,7 @@ python write_image_manifest () {
manifest_link = deploy_dir + "/" + link_name + ".manifest"
if os.path.lexists(manifest_link):
os.remove(manifest_link)
- os.symlink(os.path.basename(manifest_name), manifest_link)
+ os.link(manifest_name, manifest_link)
}
# Can be used to create /etc/timestamp during image construction to give a reasonably
@@ -339,7 +339,7 @@ python write_image_test_data() {
testdata_link = os.path.join(deploy_dir, "%s.testdata.json" % link_name)
if os.path.lexists(testdata_link):
os.remove(testdata_link)
- os.symlink(os.path.basename(testdata_name), testdata_link)
+ os.link(testdata_name, testdata_link)
}
write_image_test_data[vardepsexclude] += "TOPDIR"
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 3/8] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
2020-01-14 9:12 ` [PATCHv5 1/8] image-artifact-names: introduce new bbclass and move some variables into it Martin Jansa
2020-01-14 9:12 ` [PATCHv5 2/8] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-15 10:11 ` Richard Purdie
2020-01-14 9:12 ` [PATCHv5 4/8] kernel.bbclass: imageType without {} Martin Jansa
` (6 subsequent siblings)
9 siblings, 1 reply; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* otherwise PKGR seen in do_install, do_deploy and do_deploy_links will
have different value in each of them (PRSERV will return different
value of EXTENDPRAUTO because TASKHASH is different for each of these
tasks and also cause unnecessary multiple EXTENDPRAUTO increments for
each build).
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/kernel-artifact-names.bbclass | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/classes/kernel-artifact-names.bbclass b/meta/classes/kernel-artifact-names.bbclass
index 529e0c565e..41ef6e884d 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -6,7 +6,12 @@
inherit image-artifact-names
-KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}"
+# Intentionally use PR instead of PKGR, because EXTENDPRAUTO included
+# in PKGR will have different value for do_install/do_deploy/do_deploy_links
+# tasks with different TASKHASH, causing multiple EXTENDPRAUTO increments for
+# each kernel build and more importantly preventing do_deploy_links to
+# reference artifacts created do_deploy task
+KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PR}-${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCHv5 3/8] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME
2020-01-14 9:12 ` [PATCHv5 3/8] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME Martin Jansa
@ 2020-01-15 10:11 ` Richard Purdie
0 siblings, 0 replies; 12+ messages in thread
From: Richard Purdie @ 2020-01-15 10:11 UTC (permalink / raw)
To: Martin Jansa, openembedded-core
On Tue, 2020-01-14 at 10:12 +0100, Martin Jansa wrote:
> * otherwise PKGR seen in do_install, do_deploy and do_deploy_links
> will
> have different value in each of them (PRSERV will return different
> value of EXTENDPRAUTO because TASKHASH is different for each of
> these
> tasks and also cause unnecessary multiple EXTENDPRAUTO increments
> for
> each build).
>
> [YOCTO #12937]
>
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
> meta/classes/kernel-artifact-names.bbclass | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel-artifact-names.bbclass
> b/meta/classes/kernel-artifact-names.bbclass
> index 529e0c565e..41ef6e884d 100644
> --- a/meta/classes/kernel-artifact-names.bbclass
> +++ b/meta/classes/kernel-artifact-names.bbclass
> @@ -6,7 +6,12 @@
>
> inherit image-artifact-names
>
> -KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}"
> +# Intentionally use PR instead of PKGR, because EXTENDPRAUTO
> included
> +# in PKGR will have different value for
> do_install/do_deploy/do_deploy_links
> +# tasks with different TASKHASH, causing multiple EXTENDPRAUTO
> increments for
> +# each kernel build and more importantly preventing do_deploy_links
> to
> +# reference artifacts created do_deploy task
> +KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PR}-${MACHINE}"
> KERNEL_ARTIFACT_LINK_NAME ?=
> "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
Perhaps this is why patch 5/8 is necessary? ("kernel.bbclass: drop
unnecessary package_get_auto_pr for do_install")
I suspect there are links between these two patches...
Cheers,
Richard
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCHv5 4/8] kernel.bbclass: imageType without {}
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (2 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 3/8] kernel-artifact-names.bbclass: use PR instead of PKGR in KERNEL_ARTIFACT_NAME Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 5/8] kernel.bbclass: drop unnecessary package_get_auto_pr for do_install Martin Jansa
` (5 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* just to make sure it looks like bash variable not bitbake variable in
run.do_* scripts
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/kernel.bbclass | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 954b761094..ff5673fb56 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -373,9 +373,9 @@ kernel_do_install() {
install -d ${D}/${KERNEL_IMAGEDEST}
install -d ${D}/boot
for imageType in ${KERNEL_IMAGETYPES} ; do
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} ${D}/${KERNEL_IMAGEDEST}/${imageType}-${KERNEL_VERSION}
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType ${D}/${KERNEL_IMAGEDEST}/$imageType-${KERNEL_VERSION}
if [ "${KERNEL_PACKAGE_NAME}" = "kernel" ]; then
- ln -sf ${imageType}-${KERNEL_VERSION} ${D}/${KERNEL_IMAGEDEST}/${imageType}
+ ln -sf $imageType-${KERNEL_VERSION} ${D}/${KERNEL_IMAGEDEST}/$imageType
fi
done
install -m 0644 System.map ${D}/boot/System.map-${KERNEL_VERSION}
@@ -686,8 +686,8 @@ kernel_do_deploy() {
fi
for imageType in ${KERNEL_IMAGETYPES} ; do
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin
- ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType $deployDir/$imageType-${KERNEL_IMAGE_NAME}.bin
+ ln -sf $imageType-${KERNEL_IMAGE_NAME}.bin $deployDir/$imageType
done
if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
@@ -700,7 +700,7 @@ kernel_do_deploy() {
if [ "$imageType" = "fitImage" ] ; then
continue
fi
- install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${imageType}-${INITRAMFS_NAME}.bin
+ install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType.initramfs $deployDir/$imageType-${INITRAMFS_NAME}.bin
done
fi
}
@@ -718,7 +718,7 @@ kernel_do_deploy_links() {
fi
for imageType in ${KERNEL_IMAGETYPES} ; do
- ln -vf $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}-${KERNEL_IMAGE_LINK_NAME}.bin
+ ln -vf $deployDir/$imageType-${KERNEL_IMAGE_NAME}.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
done
if [ ${MODULE_TARBALL_DEPLOY} = "1" -a -f $deployDir/modules-${MODULE_TARBALL_NAME}.tgz ] ; then
@@ -730,7 +730,7 @@ kernel_do_deploy_links() {
if [ "$imageType" = "fitImage" ] ; then
continue
fi
- ln -vf $deployDir/${imageType}-${INITRAMFS_NAME}.bin $deployDir/${imageType}-${INITRAMFS_LINK_NAME}.bin
+ ln -vf $deployDir/$imageType-${INITRAMFS_NAME}.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin
done
fi
}
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 5/8] kernel.bbclass: drop unnecessary package_get_auto_pr for do_install
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (3 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 4/8] kernel.bbclass: imageType without {} Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 6/8] *-artifact-names: include version only in the artifact links Martin Jansa
` (4 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* do_install doesn't use whole "version" as do_deploy does, e.g.
${PKGE}-${PKGV}-${PKGR}-${MACHINE}
it installs only the files with only ${KERNEL_VERSION} in filename or
path, so it doesn't need expanded AUTOINC value in PKGV nor
EXPANDPRAUTO in PKGR like do_deploy does
* it was introduced in
http://git.openembedded.org/openembedded-core/commit/?id=1392f959cb8cd50b5a4492899e54f3ed68ef56d7
but it's not clear why it was needed back then, but doesn't seem to be
useful at all currently, only causes multiple EXTENDPRAUTO bumps every
time different linux-yocto is being built.
* There are currently 4 EXTENDPRAUTO bumps during each build as shown in
prserv:
$ sqlite3 cache/prserv.sqlite3
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> select * from PRMAIN_nohist where version like 'linux-yocto-dev%';
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|20601304a6e4fa0b7ac13fa1262040c976c862d177077799dc15492215fa51df|0
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|2820d331b7eba5165943bc016a1c274d42e7605e24244873b15cc1c9c6f657e2|1
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|4f29da98c268aa5bf1c4767bb2bb157fc6077b1d76dfd434028b18bf3252e0c0|2
linux-yocto-dev-4.19+gitAUTOINC+57b791cb9f_122d468967-r0|qemux86|23d8d17b23bc6db1dd7f0f30086f0ec6ade2b2180e787a78d89b6e43b8c4fad6|3
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c|0
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993|1
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89|2
linux-yocto-dev-5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0|qemux86|dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7|3
the TASKHASHes correspond to do_install, do_package, do_deploy, do_deploy_links tasks:
$ ls tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_*sigdata*{2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c,dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993,09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89,dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7}*
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_install.sigdata.2a23d8783f794b3e79b438889ec60661ca635f9ec09d0519176a31d832377f1c
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_package.sigdata.dafc2a636e7e18357b7efbf99981af45234105c3f46b056edfd2142d5a5d4993
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_deploy.sigdata.09798369f303700fb8d42550d959e310a05fb4573b71646df51acc00d3a6fe89
tmp-glibc/stamps/qemux86-webos-linux/linux-yocto-dev/5.0~rc6+gitAUTOINC+e721b5d6ab_8b7d7ef74a-r0.do_deploy_links.sigdata.dab07eb869034df28be59ae13914989ab88bdca5a9a9362ca96b4eb38180afd7
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/kernel.bbclass | 1 -
1 file changed, 1 deletion(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index ff5673fb56..249704b6f8 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -385,7 +385,6 @@ kernel_do_install() {
install -d ${D}${sysconfdir}/modules-load.d
install -d ${D}${sysconfdir}/modprobe.d
}
-do_install[prefuncs] += "package_get_auto_pr"
# Must be ran no earlier than after do_kernel_checkout or else Makefile won't be in ${S}/Makefile
do_kernel_version_sanity_check() {
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 6/8] *-artifact-names: include version only in the artifact links
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (4 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 5/8] kernel.bbclass: drop unnecessary package_get_auto_pr for do_install Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 7/8] *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX Martin Jansa
` (3 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* drop ${PKGE}-${PKGV}-${PR} from kernel artifacts names (this is the
latest build) and add it only in hardlinks created in do_deploy_links
so that we can use PKGR there again (because these links are generally
used only by human operators and they don't have their own TASKHASH or
the IMAGE_VERSION_SUFFIX might be set to some release name which they
do understand
* this allows to drop package_get_auto_pr from kernel do_deploy as well,
leaving only 2 EXTENDPRAUTO bumps for each kernel build (do_package
and do_deploy_links, unfortunatelly these will still have different
value, so if you're looking for the exact kernel image in deploy
directory based on kernel image package version seen on the device the
EXTENDPRAUTO part of PR will be different).
[YOCTO #12937]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/image-artifact-names.bbclass | 2 +-
meta/classes/kernel-artifact-names.bbclass | 7 +------
meta/classes/kernel.bbclass | 1 -
3 files changed, 2 insertions(+), 8 deletions(-)
diff --git a/meta/classes/image-artifact-names.bbclass b/meta/classes/image-artifact-names.bbclass
index d5ba035f5a..b23cef22ca 100644
--- a/meta/classes/image-artifact-names.bbclass
+++ b/meta/classes/image-artifact-names.bbclass
@@ -3,7 +3,7 @@
##################################################################
IMAGE_BASENAME = "${PN}"
-IMAGE_VERSION_SUFFIX = "-${DATETIME}"
+IMAGE_VERSION_SUFFIX = "${PKGE}-${PKGV}-${PKGR}-${DATETIME}"
IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME"
IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}"
IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}"
diff --git a/meta/classes/kernel-artifact-names.bbclass b/meta/classes/kernel-artifact-names.bbclass
index 41ef6e884d..92e08297cc 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -6,12 +6,7 @@
inherit image-artifact-names
-# Intentionally use PR instead of PKGR, because EXTENDPRAUTO included
-# in PKGR will have different value for do_install/do_deploy/do_deploy_links
-# tasks with different TASKHASH, causing multiple EXTENDPRAUTO increments for
-# each kernel build and more importantly preventing do_deploy_links to
-# reference artifacts created do_deploy task
-KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PR}-${MACHINE}"
+KERNEL_ARTIFACT_NAME ?= "${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 249704b6f8..ebe5378301 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -705,7 +705,6 @@ kernel_do_deploy() {
}
do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy[dirs] = "${DEPLOYDIR} ${B}"
-do_deploy[prefuncs] += "package_get_auto_pr"
addtask deploy after do_populate_sysroot do_packagedata
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 7/8] *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (5 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 6/8] *-artifact-names: include version only in the artifact links Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:12 ` [PATCHv5 8/8] images: respect IMAGE_NAME_SUFFIX also for *-testdata.json and *-qemuboot.conf files Martin Jansa
` (2 subsequent siblings)
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* fixes the issue when image-live.bbclass expects the image
ending with just INITRAMFS_FSTYPES:
image-live.bbclass:INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}"
while by default it now was with .rootfs suffix:
-rw-r--r-- 2 bitbake bitbake 1.5K Oct 25 16:12 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs.env
-rw-r--r-- 4 bitbake bitbake 11M Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.cpio.gz
-rw-r--r-- 4 bitbake bitbake 1.2K Oct 25 16:11 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.manifest
-rw-r--r-- 4 bitbake bitbake 1.3K Oct 25 16:12 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.qemuboot.conf
-rw-r--r-- 4 bitbake bitbake 196K Oct 25 16:11 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.testdata.json
-rw-r--r-- 4 bitbake bitbake 118M Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.wic
-rw-r--r-- 4 bitbake bitbake 3.1K Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64-1.0-r0-20191025154349.wic.bmap
-rw-r--r-- 4 bitbake bitbake 1.3K Oct 25 16:12 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.qemuboot.conf
-rw-r--r-- 4 bitbake bitbake 11M Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.rootfs.cpio.gz
-rw-r--r-- 4 bitbake bitbake 1.2K Oct 25 16:11 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.rootfs.manifest
-rw-r--r-- 4 bitbake bitbake 118M Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.rootfs.wic
-rw-r--r-- 4 bitbake bitbake 3.1K Oct 25 16:13 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.rootfs.wic.bmap
-rw-r--r-- 4 bitbake bitbake 196K Oct 25 16:11 /OE/build/oe-core/tmp/deploy/images/genericx86-64/core-image-minimal-initramfs-genericx86-64.testdata.json
initramfs images aren't normally used for rootfs, so just set
the suffix to empty, people using different artifact names might
still need to set INITRD_LIVE (e.g. when their images don't end
with "-${MACHINE}" as well)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/recipes-core/images/core-image-minimal-initramfs.bb | 1 +
meta/recipes-core/images/core-image-tiny-initramfs.bb | 1 +
meta/recipes-extended/images/core-image-testmaster-initramfs.bb | 1 +
3 files changed, 3 insertions(+)
diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
index 7df8ab1ebd..055db7b4a9 100644
--- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
+++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
@@ -17,6 +17,7 @@ PACKAGE_INSTALL = "${INITRAMFS_SCRIPTS} ${VIRTUAL-RUNTIME_base-utils} udev base-
IMAGE_FEATURES = ""
export IMAGE_BASENAME = "${MLPREFIX}core-image-minimal-initramfs"
+IMAGE_NAME_SUFFIX ?= ""
IMAGE_LINGUAS = ""
LICENSE = "MIT"
diff --git a/meta/recipes-core/images/core-image-tiny-initramfs.bb b/meta/recipes-core/images/core-image-tiny-initramfs.bb
index 0eca6d9944..5849900742 100644
--- a/meta/recipes-core/images/core-image-tiny-initramfs.bb
+++ b/meta/recipes-core/images/core-image-tiny-initramfs.bb
@@ -13,6 +13,7 @@ PACKAGE_INSTALL = "initramfs-live-boot-tiny packagegroup-core-boot dropbear ${VI
IMAGE_FEATURES = ""
export IMAGE_BASENAME = "core-image-tiny-initramfs"
+IMAGE_NAME_SUFFIX ?= ""
IMAGE_LINGUAS = ""
LICENSE = "MIT"
diff --git a/meta/recipes-extended/images/core-image-testmaster-initramfs.bb b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
index 09a6d16042..1a2e0af27b 100644
--- a/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
+++ b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
@@ -8,6 +8,7 @@ PACKAGE_INSTALL = "initramfs-live-boot initramfs-live-install-testfs initramfs-l
IMAGE_FEATURES = ""
export IMAGE_BASENAME = "core-image-testmaster-initramfs"
+IMAGE_NAME_SUFFIX ?= ""
IMAGE_LINGUAS = ""
LICENSE = "MIT"
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv5 8/8] images: respect IMAGE_NAME_SUFFIX also for *-testdata.json and *-qemuboot.conf files
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (6 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 7/8] *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX Martin Jansa
@ 2020-01-14 9:12 ` Martin Jansa
2020-01-14 9:32 ` ✗ patchtest: failure for YOCTO #12937 - Consistent naming scheme for deployed artifacts (rev4) Patchwork
2020-01-15 10:08 ` [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Richard Purdie
9 siblings, 0 replies; 12+ messages in thread
From: Martin Jansa @ 2020-01-14 9:12 UTC (permalink / raw)
To: openembedded-core
* it might be cleaner to move these including the suffix into
image-artifacts.bbclass
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
meta/classes/image-live.bbclass | 10 +++++-----
meta/classes/qemuboot.bbclass | 2 +-
meta/classes/rootfs-postcommands.bbclass | 2 +-
meta/classes/syslinux.bbclass | 2 +-
meta/classes/testexport.bbclass | 10 ++++++----
meta/classes/testimage.bbclass | 5 +++--
6 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass
index 9ea5ddc312..a20a409721 100644
--- a/meta/classes/image-live.bbclass
+++ b/meta/classes/image-live.bbclass
@@ -130,14 +130,14 @@ build_iso() {
if [ "${PCBIOS}" = "1" ] && [ "${EFI}" != "1" ] ; then
# PCBIOS only media
mkisofs -V ${BOOTIMG_VOLUME_ID} \
- -o ${IMGDEPLOYDIR}/${IMAGE_NAME}.iso \
+ -o ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.iso \
-b ${ISO_BOOTIMG} -c ${ISO_BOOTCAT} \
$mkisofs_compress_opts \
${MKISOFS_OPTIONS} $mkisofs_iso_level ${ISODIR}
else
# EFI only OR EFI+PCBIOS
mkisofs -A ${BOOTIMG_VOLUME_ID} -V ${BOOTIMG_VOLUME_ID} \
- -o ${IMGDEPLOYDIR}/${IMAGE_NAME}.iso \
+ -o ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.iso \
-b ${ISO_BOOTIMG} -c ${ISO_BOOTCAT} \
$mkisofs_compress_opts ${MKISOFS_OPTIONS} $mkisofs_iso_level \
-eltorito-alt-boot -eltorito-platform efi \
@@ -146,7 +146,7 @@ build_iso() {
isohybrid_args="-u"
fi
- isohybrid $isohybrid_args ${IMGDEPLOYDIR}/${IMAGE_NAME}.iso
+ isohybrid $isohybrid_args ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.iso
}
build_fat_img() {
@@ -238,13 +238,13 @@ build_hddimg() {
fi
fi
- build_fat_img ${HDDDIR} ${IMGDEPLOYDIR}/${IMAGE_NAME}.hddimg
+ build_fat_img ${HDDDIR} ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.hddimg
if [ "${PCBIOS}" = "1" ]; then
syslinux_hddimg_install
fi
- chmod 644 ${IMGDEPLOYDIR}/${IMAGE_NAME}.hddimg
+ chmod 644 ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.hddimg
fi
}
diff --git a/meta/classes/qemuboot.bbclass b/meta/classes/qemuboot.bbclass
index fbcd1f8098..7924b27689 100644
--- a/meta/classes/qemuboot.bbclass
+++ b/meta/classes/qemuboot.bbclass
@@ -85,7 +85,7 @@ do_write_qemuboot_conf[vardepsexclude] += "TOPDIR"
python do_write_qemuboot_conf() {
import configparser
- qemuboot = "%s/%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'), d.getVar('IMAGE_NAME'))
+ qemuboot = "%s/%s%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'), d.getVar('IMAGE_NAME'), d.getVar('IMAGE_NAME_SUFFIX'))
qemuboot_link = "%s/%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'), d.getVar('IMAGE_LINK_NAME'))
finalpath = d.getVar("DEPLOY_DIR_IMAGE")
topdir = d.getVar('TOPDIR')
diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index 14fe0d2f80..00f57bcc06 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -330,7 +330,7 @@ python write_image_test_data() {
deploy_dir = d.getVar('IMGDEPLOYDIR')
link_name = d.getVar('IMAGE_LINK_NAME')
- testdata_name = os.path.join(deploy_dir, "%s.testdata.json" % d.getVar('IMAGE_NAME'))
+ testdata_name = os.path.join(deploy_dir, "%s%s.testdata.json" % (d.getVar('IMAGE_NAME'), d.getVar('IMAGE_NAME_SUFFIX')))
searchString = "%s/"%(d.getVar("TOPDIR")).replace("//","/")
export2json(d, testdata_name, searchString=searchString, replaceString="")
diff --git a/meta/classes/syslinux.bbclass b/meta/classes/syslinux.bbclass
index 894f6b3718..fab7bc0972 100644
--- a/meta/classes/syslinux.bbclass
+++ b/meta/classes/syslinux.bbclass
@@ -72,7 +72,7 @@ syslinux_hddimg_populate() {
}
syslinux_hddimg_install() {
- syslinux ${IMGDEPLOYDIR}/${IMAGE_NAME}.hddimg
+ syslinux ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.hddimg
}
python build_syslinux_cfg () {
diff --git a/meta/classes/testexport.bbclass b/meta/classes/testexport.bbclass
index 59cbaefbf9..c5f0bb768f 100644
--- a/meta/classes/testexport.bbclass
+++ b/meta/classes/testexport.bbclass
@@ -49,8 +49,9 @@ def testexport_main(d):
from oeqa.runtime.context import OERuntimeTestContext
from oeqa.runtime.context import OERuntimeTestContextExecutor
- image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
- d.getVar('IMAGE_LINK_NAME')))
+ image_name = ("%s/%s%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_NAME'),
+ d.getVar('IMAGE_NAME_SUFFIX')))
tdname = "%s.testdata.json" % image_name
td = json.load(open(tdname, "r"))
@@ -122,8 +123,9 @@ def copy_needed_files(d, tc):
shutil.copy2(json_file, cases_path)
# Copy test data
- image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
- d.getVar('IMAGE_LINK_NAME')))
+ image_name = ("%s/%s%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_NAME'),
+ d.getVar('IMAGE_NAME_SUFFIX')))
image_manifest = "%s.manifest" % image_name
tdname = "%s.testdata.json" % image_name
test_data_path = os.path.join(export_path, 'data')
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index a1d0ddbb47..68b5bd8ce6 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -183,8 +183,9 @@ def testimage_main(d):
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
- image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
- d.getVar('IMAGE_LINK_NAME')))
+ image_name = ("%s/%s%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_NAME'),
+ d.getVar('IMAGE_NAME_SUFFIX')))
tdname = "%s.testdata.json" % image_name
try:
--
2.20.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* ✗ patchtest: failure for YOCTO #12937 - Consistent naming scheme for deployed artifacts (rev4)
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (7 preceding siblings ...)
2020-01-14 9:12 ` [PATCHv5 8/8] images: respect IMAGE_NAME_SUFFIX also for *-testdata.json and *-qemuboot.conf files Martin Jansa
@ 2020-01-14 9:32 ` Patchwork
2020-01-15 10:08 ` [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Richard Purdie
9 siblings, 0 replies; 12+ messages in thread
From: Patchwork @ 2020-01-14 9:32 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
== Series Details ==
Series: YOCTO #12937 - Consistent naming scheme for deployed artifacts (rev4)
Revision: 4
URL : https://patchwork.openembedded.org/series/19321/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:
* Patch [v5, 2/8] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link
Issue Commit shortlog is too long [test_shortlog_length]
Suggested fix Edit shortlog so that it is 90 characters or less (currently 119 characters)
If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts
2020-01-14 9:12 [PATCHv5 0/6] YOCTO #12937 - Consistent naming scheme for deployed artifacts Martin Jansa
` (8 preceding siblings ...)
2020-01-14 9:32 ` ✗ patchtest: failure for YOCTO #12937 - Consistent naming scheme for deployed artifacts (rev4) Patchwork
@ 2020-01-15 10:08 ` Richard Purdie
9 siblings, 0 replies; 12+ messages in thread
From: Richard Purdie @ 2020-01-15 10:08 UTC (permalink / raw)
To: Martin Jansa, openembedded-core
On Tue, 2020-01-14 at 10:12 +0100, Martin Jansa wrote:
> The following changes since commit
> b6d4150f9c74f25a4022a3fa0bd489a8e85deb77:
>
> gettext: fix typo in Upstream-Status (2020-01-13 13:07:32 +0000)
>
> are available in the Git repository at:
>
> git://git.openembedded.org/openembedded-core-contrib
> jansa/artifacts
>
> http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
>
> Martin Jansa (8):
> image-artifact-names: introduce new bbclass and move some variables
> into it
> bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in
> the _LINK_NAME variables and change it to hard link
> kernel-artifact-names.bbclass: use PR instead of PKGR in
> KERNEL_ARTIFACT_NAME
> kernel.bbclass: imageType without {}
> kernel.bbclass: drop unnecessary package_get_auto_pr for do_install
> *-artifact-names: include version only in the artifact links
> *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX
> images: respect IMAGE_NAME_SUFFIX also for *-testdata.json and
> *-qemuboot.conf files
Unfortunately this triggered a lot of failures in testing:
selftest:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/612
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/615
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/621
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/611
qa-extras:
https://autobuilder.yoctoproject.org/typhoon/#/builders/69/builds/1451
qa-extras2:
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1482
qemuarm:
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/1451
qemuarm64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/1444
qemumips:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1444
qemumips64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1446
qemuppc:
https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/1436
qemux86:
https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/1444
qemux86-64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/1448
A lot of errors but possibly only from a smaller number of root causes,
the selftest and step3d failures look like specific patterns.
Cheers,
Richard
^ permalink raw reply [flat|nested] 12+ messages in thread