[PATCH 0/1] classes/kernel*: allow disabling symlink creation

[PATCH 0/1] classes/kernel*: allow disabling symlink creation

[Paul Eggleton]

The following changes since commit 6e3785a3f1f3cf68f5fe101cd6bebe91db165973:

  uninative: Don't use single sstate for pseudo-native (2020-11-24 15:53:04 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/kernel-symlinks
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/kernel-symlinks

Paul Eggleton (1):
  classes/kernel*: allow disabling symlink creation

 meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------
 meta/classes/kernel-fitimage.bbclass   | 16 ++++++++++++----
 meta/classes/kernel.bbclass            | 14 ++++++++++----
 3 files changed, 34 insertions(+), 14 deletions(-)

-- 
1.8.3.1

[PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

Allow setting any of the *_LINK_NAME variables to empty string in order
to disable creating symlinks. This is particularly useful for situations
where symlinks in DEPLOY_DIR_IMAGE will be treated externally as
duplicates of the linked files (in which case you would probably set
KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------
 meta/classes/kernel-fitimage.bbclass   | 16 ++++++++++++----
 meta/classes/kernel.bbclass            | 14 ++++++++++----
 3 files changed, 34 insertions(+), 14 deletions(-)

diff --git a/meta/classes/kernel-devicetree.bbclass b/meta/classes/kernel-devicetree.bbclass
index 81dda80..7cde549 100644
--- a/meta/classes/kernel-devicetree.bbclass
+++ b/meta/classes/kernel-devicetree.bbclass
@@ -73,21 +73,27 @@ do_deploy_append() {
 		dtb_base_name=`basename $dtb .$dtb_ext`
 		install -d $deployDir
 		install -m 0644 ${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
-		ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext
-		ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+		if [ -n "${KERNEL_DTB_LINK_NAME}" ] ; then
+			ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext
+			ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+		fi
 		for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do
 			if [ "$type" = "zImage" ] && [ "${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
 				cat ${D}/${KERNEL_IMAGEDEST}/$type \
 					$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
 					> $deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
-				ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-					$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+				if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
+					ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+						$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+				fi
 				if [ -e "${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
 					cat ${KERNEL_OUTPUT_DIR}/${type}.initramfs \
 						$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
 						>  $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
-					ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-						$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+					if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
+						ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+							$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+					fi
 				fi
 			fi
 		done
diff --git a/meta/classes/kernel-fitimage.bbclass b/meta/classes/kernel-fitimage.bbclass
index bb2f3c4..e91c95f 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -573,20 +573,28 @@ kernel_do_deploy_append() {
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
 		echo "Copying fit-image.its source file..."
 		install -m 0644 ${B}/fit-image.its "$deployDir/fitImage-its-${KERNEL_FIT_NAME}.its"
-		ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}"
+		if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+			ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}"
+		fi
 
 		echo "Copying linux.bin file..."
 		install -m 0644 ${B}/linux.bin $deployDir/fitImage-linux.bin-${KERNEL_FIT_NAME}.bin
-		ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}"
+		if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+			ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}"
+		fi
 
 		if [ -n "${INITRAMFS_IMAGE}" ]; then
 			echo "Copying fit-image-${INITRAMFS_IMAGE}.its source file..."
 			install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"
-			ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+				ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			fi
 
 			echo "Copying fitImage-${INITRAMFS_IMAGE} file..."
 			install -m 0644 ${B}/arch/${ARCH}/boot/fitImage-${INITRAMFS_IMAGE} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin"
-			ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+				ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			fi
 		fi
 		if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ] ; then
 			# UBOOT_DTB_IMAGE is a realfile, but we can't use
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 1a444ef..299e559 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -725,8 +725,10 @@ kernel_do_deploy() {
 	for imageType in ${KERNEL_IMAGETYPES} ; do
 		baseName=$imageType-${KERNEL_IMAGE_NAME}
 		install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType $deployDir/$baseName.bin
-		ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
-		ln -sf $baseName.bin $deployDir/$imageType
+		if [ -n "${KERNEL_IMAGE_LINK_NAME}" ] ; then
+			ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
+			ln -sf $baseName.bin $deployDir/$imageType
+		fi
 	done
 
 	if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
@@ -739,7 +741,9 @@ kernel_do_deploy() {
 		TAR_ARGS="$TAR_ARGS --owner=0 --group=0"
 		tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip -9n > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
 
-		ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
+		if [ -n "${MODULE_TARBALL_LINK_NAME}" ] ; then
+			ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
+		fi
 	fi
 
 	if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
@@ -749,7 +753,9 @@ kernel_do_deploy() {
 			fi
 			initramfsBaseName=$imageType-${INITRAMFS_NAME}
 			install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType.initramfs $deployDir/$initramfsBaseName.bin
-			ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin
+			if [ -n "${INITRAMFS_LINK_NAME}" ] ; then
+				ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin
+			fi
 		done
 	fi
 }
-- 
1.8.3.1

Re: [OE-core] [PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Paul Eggleton]

On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:
> From: Paul Eggleton <paul.eggleton@microsoft.com>
> 
> Allow setting any of the *_LINK_NAME variables to empty string in order
> to disable creating symlinks. This is particularly useful for situations
> where symlinks in DEPLOY_DIR_IMAGE will be treated externally as
> duplicates of the linked files (in which case you would probably set
> KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).

I missed something - v2 incoming.

Cheers
Paul

Re: [OE-core] [PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

Can we use a separate variable controlling this behavior other than
*_LINK_NAME variables being set to empty?

Then we could easily extend it to stop creating the links also for image
artifacts as well.

On Thu, Nov 26, 2020 at 7:55 PM Paul Eggleton <
bluelightning@bluelightning.org> wrote:

> On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:
> > From: Paul Eggleton <paul.eggleton@microsoft.com>
> >
> > Allow setting any of the *_LINK_NAME variables to empty string in order
> > to disable creating symlinks. This is particularly useful for situations
> > where symlinks in DEPLOY_DIR_IMAGE will be treated externally as
> > duplicates of the linked files (in which case you would probably set
> > KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).
>
> I missed something - v2 incoming.
>
> Cheers
> Paul
>
>
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 1328 bytes --]

Re: [OE-core] [PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Paul Eggleton]

Hi Martin

(replying all this time)

On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:
> Can we use a separate variable controlling this behavior other than
> *_LINK_NAME variables being set to empty?
>
> Then we could easily extend it to stop creating the links also for image
> artifacts as well.

Well, I didn't call it out explicitly but I was just mirroring the image 
artifacts behaviour - we set those to "" as well and there are checks there 
already to make that disable the symlink in that case.

Paul

Re: [OE-core] [PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

Hi Paul,

I see, I was just worried that this case will block me pushing the last bit
of [YOCTO #12937] which changes symlinks to hardlinks:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts

Which makes the use case you're fixing worse (as even less systems will
correctly handle hardlinks), so I support the idea to disable having the
links. My current work around is to just remove the unnecessary files when
rsyncing to file-server.

But the main issue which I'm seeing with my current implementation (and
with webos_deploy this is based on) is that we need the version-less
artifacts to be available, because e.g. image recipe searching kernel
artifact when there is DATETIME or SRCPV in the kernel artifact-name won't
be able to find it, because the image might be built at different DATETIME
and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the
build is using the version-less artifacts (always the latest one).

I guess your implementation might fail in such DATETIME cases as well.

With separate variable to disable the links we can somehow work around this
or at least warn user that without the version-less filenames, the
VERSION_SUFFIX variable needs to be stable (as no PV/SRCPV/DATATIME in it).

Cheers,

On Tue, Dec 1, 2020 at 5:13 PM Paul Eggleton <
bluelightning@bluelightning.org> wrote:

> Hi Martin
>
> (replying all this time)
>
> On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:
> > Can we use a separate variable controlling this behavior other than
> > *_LINK_NAME variables being set to empty?
> >
> > Then we could easily extend it to stop creating the links also for image
> > artifacts as well.
>
> Well, I didn't call it out explicitly but I was just mirroring the image
> artifacts behaviour - we set those to "" as well and there are checks
> there
> already to make that disable the symlink in that case.
>
> Paul
>
>
>

[-- Attachment #2: Type: text/html, Size: 2567 bytes --]

Re: [OE-core] [PATCH 1/1] classes/kernel*: allow disabling symlink creation

[Paul Eggleton]

On Wednesday, 2 December 2020 05:22:43 NZDT Martin Jansa wrote:
> I see, I was just worried that this case will block me pushing the last bit
> of [YOCTO #12937] which changes symlinks to hardlinks:
> https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifact
> s
> 
> Which makes the use case you're fixing worse (as even less systems will
> correctly handle hardlinks), so I support the idea to disable having the
> links. My current work around is to just remove the unnecessary files when
> rsyncing to file-server.
> 
> But the main issue which I'm seeing with my current implementation (and
> with webos_deploy this is based on) is that we need the version-less
> artifacts to be available, because e.g. image recipe searching kernel
> artifact when there is DATETIME or SRCPV in the kernel artifact-name won't
> be able to find it, because the image might be built at different DATETIME
> and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the
> build is using the version-less artifacts (always the latest one).
> 
> I guess your implementation might fail in such DATETIME cases as well.
> 
> With separate variable to disable the links we can somehow work around this
> or at least warn user that without the version-less filenames, the
> VERSION_SUFFIX variable needs to be stable (as no PV/SRCPV/DATATIME in it).

That is a good point and we have the same problem here - what I'm planning to
do is to set a bunch of the other variables such that there are no versioned
artifacts - i.e. the real files effectively appear in place of the symlinks and
the versioned ones disappear. It seems to work in my tests so far. With my
patchset this is what I set:

------------ snip ------------
IMAGE_LINK_NAME = ""
IMAGE_VERSION_SUFFIX = ""
IMAGE_NAME_SUFFIX = ""
IMAGE_MANIFEST_distroname = "${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.manifest"

KERNEL_ARTIFACT_NAME = "${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME = ""
------------ snip ------------

I have found that I still need the plain type named kernel artifacts (e.g. vmlinux),
and since these don't actually use the link name variable I have changed those to
be written out even if the link name is blank (v2 patchset still in testing). We could
introduce another variable to disable those but I hadn't planned to do that
for the moment.

Cheers,
Paul