[gatesgarth][PATCH] image_types.bbclass: tar: support EXTRA_IMGAGECMD

[gatesgarth][PATCH] image_types.bbclass: tar: support EXTRA_IMGAGECMD

[S. Lockwood-Childs]

The primary motivation for making tar image options configurable
is to provide choice on a per-machine basis between gnu and posix
formats. Neither one is suitable for all use cases.
* gnu isn't compatible when xattrs are required
* posix can cause locale headaches
  https://groups.google.com/g/swupdate/c/zUOSqmkk7cA

One could say "just fix your image installer" but it's nice to be able
to still produce images that currently-deployed systems can handle,
for instance to make update packages to update your updater :)

Default is still posix format, so current behavior is unchanged
unless EXTRA_IMAGECMD_tar is customized.

Signed-off-by: S. Lockwood-Childs <sjl@vctlabs.com>
---
 meta/classes/image_types.bbclass | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 30951ae366..c0dca351d5 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -110,7 +110,7 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS} ${IMGDEPLOYDIR}/${IMAGE_NAM
 
 IMAGE_CMD_TAR ?= "tar"
 # ignore return code 1 "file changed as we read it" as other tasks(e.g. do_image_wic) may be hardlinking rootfs
-IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --format=posix --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
+IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --numeric-owner ${EXTRA_IMAGECMD} -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
 
 do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
 IMAGE_CMD_cpio () {
@@ -228,6 +228,9 @@ EXTRA_IMAGECMD_ext4 ?= "-i 4096"
 EXTRA_IMAGECMD_btrfs ?= "-n 4096"
 EXTRA_IMAGECMD_f2fs ?= ""
 
+# Customize tar format if needed, since gnu and posix have different drawbacks
+EXTRA_IMAGECMD_tar ?= "--format=posix"
+
 do_image_cpio[depends] += "cpio-native:do_populate_sysroot"
 do_image_jffs2[depends] += "mtd-utils-native:do_populate_sysroot"
 do_image_cramfs[depends] += "util-linux-native:do_populate_sysroot"
-- 
2.20.1

Re: [OE-core] [gatesgarth][PATCH] image_types.bbclass: tar: support EXTRA_IMGAGECMD

[Anuj Mittal]

Hi,

On Wed, 2021-04-14 at 19:32 -0700, S. Lockwood-Childs wrote:
> The primary motivation for making tar image options configurable
> is to provide choice on a per-machine basis between gnu and posix
> formats. Neither one is suitable for all use cases.
> * gnu isn't compatible when xattrs are required
> * posix can cause locale headaches
>   https://groups.google.com/g/swupdate/c/zUOSqmkk7cA
> 
> One could say "just fix your image installer" but it's nice to be able
> to still produce images that currently-deployed systems can handle,
> for instance to make update packages to update your updater :)
> 
> Default is still posix format, so current behavior is unchanged
> unless EXTRA_IMAGECMD_tar is customized.
> 

This would need to be proposed for and merged in master branch first
before it can be backported to a stable branch.

Thanks,

Anuj

> Signed-off-by: S. Lockwood-Childs <sjl@vctlabs.com>
> ---
>  meta/classes/image_types.bbclass | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image_types.bbclass
> b/meta/classes/image_types.bbclass
> index 30951ae366..c0dca351d5 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -110,7 +110,7 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs
> ${IMAGE_ROOTFS} ${IMGDEPLOYDIR}/${IMAGE_NAM
>  
>  IMAGE_CMD_TAR ?= "tar"
>  # ignore return code 1 "file changed as we read it" as other
> tasks(e.g. do_image_wic) may be hardlinking rootfs
> -IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --format=posix --
> numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar
> -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
> +IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --numeric-owner
> ${EXTRA_IMAGECMD} -cf
> ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C
> ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
>  
>  do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
>  IMAGE_CMD_cpio () {
> @@ -228,6 +228,9 @@ EXTRA_IMAGECMD_ext4 ?= "-i 4096"
>  EXTRA_IMAGECMD_btrfs ?= "-n 4096"
>  EXTRA_IMAGECMD_f2fs ?= ""
>  
> +# Customize tar format if needed, since gnu and posix have different
> drawbacks
> +EXTRA_IMAGECMD_tar ?= "--format=posix"
> +
>  do_image_cpio[depends] += "cpio-native:do_populate_sysroot"
>  do_image_jffs2[depends] += "mtd-utils-native:do_populate_sysroot"
>  do_image_cramfs[depends] += "util-linux-native:do_populate_sysroot"
> 
> 
> 

Re: [OE-core] [gatesgarth][PATCH] image_types.bbclass: tar: support EXTRA_IMGAGECMD

[Khem Raj]

On 4/14/21 7:32 PM, S. Lockwood-Childs wrote:
> The primary motivation for making tar image options configurable
> is to provide choice on a per-machine basis between gnu and posix
> formats. Neither one is suitable for all use cases.
> * gnu isn't compatible when xattrs are required
> * posix can cause locale headaches
>    https://groups.google.com/g/swupdate/c/zUOSqmkk7cA
> 
> One could say "just fix your image installer" but it's nice to be able
> to still produce images that currently-deployed systems can handle,
> for instance to make update packages to update your updater :)
> 
> Default is still posix format, so current behavior is unchanged
> unless EXTRA_IMAGECMD_tar is customized.
> 
> Signed-off-by: S. Lockwood-Childs <sjl@vctlabs.com>
> ---
>   meta/classes/image_types.bbclass | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index 30951ae366..c0dca351d5 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -110,7 +110,7 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS} ${IMGDEPLOYDIR}/${IMAGE_NAM
>   
>   IMAGE_CMD_TAR ?= "tar"
>   # ignore return code 1 "file changed as we read it" as other tasks(e.g. do_image_wic) may be hardlinking rootfs
> -IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --format=posix --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
> +IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --numeric-owner ${EXTRA_IMAGECMD} -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
>   
>   do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
>   IMAGE_CMD_cpio () {
> @@ -228,6 +228,9 @@ EXTRA_IMAGECMD_ext4 ?= "-i 4096"
>   EXTRA_IMAGECMD_btrfs ?= "-n 4096"
>   EXTRA_IMAGECMD_f2fs ?= ""
>   
> +# Customize tar format if needed, since gnu and posix have different drawbacks
> +EXTRA_IMAGECMD_tar ?= "--format=posix"
> +

in local.conf maybe you can just do


IMAGE_CMD_tar_remove = "--format=posix"
IMAGE_CMD_TAR += "--format=gnu"

will that work ?

this would mean no changes would be needed in core metadata

>   do_image_cpio[depends] += "cpio-native:do_populate_sysroot"
>   do_image_jffs2[depends] += "mtd-utils-native:do_populate_sysroot"
>   do_image_cramfs[depends] += "util-linux-native:do_populate_sysroot"
> 
> 
> 
> 
>