[PATCH] do_image_cpio: Improve reproducibility

[PATCH] do_image_cpio: Improve reproducibility

[Ernst Sjöstrand]

Find lists files in the order that the filesystem has registered them,
which can vary. Adding a sort should have minimal performance impact.
Also use the --reproducible option to cpio.

Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
---
 meta/classes/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 2eeffbb366..c7af889075 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -128,7 +128,7 @@ IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NA
 
 do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
 IMAGE_CMD_cpio () {
-	(cd ${IMAGE_ROOTFS} && find . | cpio -o -H newc >${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
+	(cd ${IMAGE_ROOTFS} && find . | sort | cpio --reproducible -o -H newc >${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
 	# We only need the /init symlink if we're building the real
 	# image. The -dbg image doesn't need it! By being clever
 	# about this we also avoid 'touch' below failing, as it
-- 
2.24.0

✗ patchtest: failure for do_image_cpio: Improve reproducibility

[Patchwork]

== Series Details ==

Series: do_image_cpio: Improve reproducibility
Revision: 1
URL   : https://patchwork.openembedded.org/series/21190/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at dfb3b56641)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [PATCH] do_image_cpio: Improve reproducibility

[Ross Burton]

On 18/11/2019 09:40, Ernst Sjöstrand wrote:
> Find lists files in the order that the filesystem has registered them,
> which can vary. Adding a sort should have minimal performance impact.
> Also use the --reproducible option to cpio.

For some reason this doesn't apply:

Applying: do_image_cpio: Improve reproducibility
Using index info to reconstruct a base tree...
error: patch failed: meta/classes/image_types.bbclass:128
error: meta/classes/image_types.bbclass: patch does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 do_image_cpio: Improve reproducibility

Ross

Re: [PATCH] do_image_cpio: Improve reproducibility

[Ernst Sjöstrand]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

I see, it was created against the yocto/poky git. However I made sure it applied to openembedded-core with "git am" also.
If you use that it should work. I can create a new version if you want.

Regards
//Ernst
________________________________
From: openembedded-core-bounces@lists.openembedded.org <openembedded-core-bounces@lists.openembedded.org> on behalf of Ross Burton <ross.burton@intel.com>
Sent: Monday, November 18, 2019 15:35
To: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PATCH] do_image_cpio: Improve reproducibility

On 18/11/2019 09:40, Ernst Sjöstrand wrote:
> Find lists files in the order that the filesystem has registered them,
> which can vary. Adding a sort should have minimal performance impact.
> Also use the --reproducible option to cpio.

For some reason this doesn't apply:

Applying: do_image_cpio: Improve reproducibility
Using index info to reconstruct a base tree...
error: patch failed: meta/classes/image_types.bbclass:128
error: meta/classes/image_types.bbclass: patch does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 do_image_cpio: Improve reproducibility

Ross
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
https://urldefense.com/v3/__http://lists.openembedded.org/mailman/listinfo/openembedded-core__;!!BFCLnRDDbM3FOmw!pzTjX-9fH0eNz5xLqWdhAP0q_Ns8zaAaf7amSkaV7fLGmXkB-29C2Awxih5o3OdxaDAFDw$

[-- Attachment #2: Type: text/html, Size: 3039 bytes --]

Re: [PATCH] do_image_cpio: Improve reproducibility

[Ross Burton]

On 18/11/2019 15:19, Ernst Sjöstrand wrote:
> I see, it was created against the yocto/poky git. However I made sure it 
> applied to openembedded-core with "git am" also.
> If you use that it should work. I can create a new version if you want.

This was applying to poky... maybe your mailer mangled the whitespace.

Ross

Re: [PATCH v2] do_image_cpio: Improve reproducibility

[Ernst Sjöstrand]

Find lists files in the order that the filesystem has registered them,
which can vary. Adding a sort should have minimal performance impact.
Also use the --reproducible option to cpio.

Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
---

Trying again, looks like tabs were converted to spaces the first time.

 meta/classes/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 2eeffbb366..c7af889075 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -128,7 +128,7 @@ IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NA
 
 do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
 IMAGE_CMD_cpio () {
-	(cd ${IMAGE_ROOTFS} && find . | cpio -o -H newc >${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
+	(cd ${IMAGE_ROOTFS} && find . | sort | cpio --reproducible -o -H newc >${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
 	# We only need the /init symlink if we're building the real
 	# image. The -dbg image doesn't need it! By being clever
 	# about this we also avoid 'touch' below failing, as it
-- 
2.24.0