[PATCH 1/9] rpm: Ensure compression parallelism isn't coded into rpms

[PATCH 1/9] rpm: Ensure compression parallelism isn't coded into rpms

[Richard Purdie]

We don't want the compression thread numbers to be encoded into the rpm
since this results in the rpm not being deterministic. Add a patch
from Alex Kanavin which addresses this issue (was queued for rpm 4.17
but we need to fix this with 4.16 too).

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 ...not-insert-payloadflags-into-.rpm-me.patch | 28 +++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_4.16.1.3.bb     |  1 +
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0001-build-pack.c-do-not-insert-payloadflags-into-.rpm-me.patch

diff --git a/meta/recipes-devtools/rpm/files/0001-build-pack.c-do-not-insert-payloadflags-into-.rpm-me.patch b/meta/recipes-devtools/rpm/files/0001-build-pack.c-do-not-insert-payloadflags-into-.rpm-me.patch
new file mode 100644
index 00000000000..79b168257ee
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0001-build-pack.c-do-not-insert-payloadflags-into-.rpm-me.patch
@@ -0,0 +1,28 @@
+From 2d351c666f09cc1b9e368422653fb42ac8b86249 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex@linutronix.de>
+Date: Tue, 31 Aug 2021 10:37:05 +0200
+Subject: [PATCH] build/pack.c: do not insert payloadflags into .rpm metadata
+
+The flags look like '19T56' where 19 is the compression level
+(deterministic), and 56 is the amount of threads (varies from one
+host to the next and breaks reproducibility for .rpm).
+
+Upstream-Status: Inappropriate [oe-core specific]
+Signed-off-by: Alexander Kanavin <alex@linutronix.de>
+---
+ build/pack.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/build/pack.c b/build/pack.c
+index 932cb213e..b45d0726f 100644
+--- a/build/pack.c
++++ b/build/pack.c
+@@ -328,7 +328,7 @@ static char *getIOFlags(Package pkg)
+ 	    headerPutString(pkg->header, RPMTAG_PAYLOADCOMPRESSOR, compr);
+ 	buf = xstrdup(rpmio_flags);
+ 	buf[s - rpmio_flags] = '\0';
+-	headerPutString(pkg->header, RPMTAG_PAYLOADFLAGS, buf+1);
++	headerPutString(pkg->header, RPMTAG_PAYLOADFLAGS, "");
+ 	free(buf);
+     }
+ exit:
diff --git a/meta/recipes-devtools/rpm/rpm_4.16.1.3.bb b/meta/recipes-devtools/rpm/rpm_4.16.1.3.bb
index 189da924366..60181f26c70 100644
--- a/meta/recipes-devtools/rpm/rpm_4.16.1.3.bb
+++ b/meta/recipes-devtools/rpm/rpm_4.16.1.3.bb
@@ -40,6 +40,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm;branch=rpm-4.16.x \
            file://0016-rpmscript.c-change-logging-level-around-scriptlets-t.patch \
            file://0001-lib-transaction.c-fix-file-conflicts-for-MIPS64-N32.patch \
            file://0001-tools-Add-error.h-for-non-glibc-case.patch \
+           file://0001-build-pack.c-do-not-insert-payloadflags-into-.rpm-me.patch \
            "
 
 PE = "1"
-- 
2.32.0

[PATCH 2/9] package: Ensure pclist files are deterministic and don't use full paths

[Richard Purdie]

Currently the pkgconfig pclist files contain full paths which are build
host specific and the order of entries is not deterministic.

Fix both these issues so the files are deterministic.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/package.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 460997ad543..985dfacd09c 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -2112,12 +2112,12 @@ python package_do_pkgconfig () {
     for pkg in packages.split():
         pkgconfig_provided[pkg] = []
         pkgconfig_needed[pkg] = []
-        for file in pkgfiles[pkg]:
+        for file in sorted(pkgfiles[pkg]):
                 m = pc_re.match(file)
                 if m:
                     pd = bb.data.init()
                     name = m.group(1)
-                    pkgconfig_provided[pkg].append(name)
+                    pkgconfig_provided[pkg].append(os.path.basename(name))
                     if not os.access(file, os.R_OK):
                         continue
                     with open(file, 'r') as f:
@@ -2140,7 +2140,7 @@ python package_do_pkgconfig () {
         pkgs_file = os.path.join(shlibswork_dir, pkg + ".pclist")
         if pkgconfig_provided[pkg] != []:
             with open(pkgs_file, 'w') as f:
-                for p in pkgconfig_provided[pkg]:
+                for p in sorted(pkgconfig_provided[pkg]):
                     f.write('%s\n' % p)
 
     # Go from least to most specific since the last one found wins
-- 
2.32.0

[PATCH 3/9] gnupg: Be deterministic about sendmail

[Richard Purdie]

Set a path to where sendmail would exist making the output deterministic
as it no longer depends on the build host and the presense of sendmail
there.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-support/gnupg/gnupg_2.3.1.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-support/gnupg/gnupg_2.3.1.bb b/meta/recipes-support/gnupg/gnupg_2.3.1.bb
index b8b0314d2f0..411ea9579dd 100644
--- a/meta/recipes-support/gnupg/gnupg_2.3.1.bb
+++ b/meta/recipes-support/gnupg/gnupg_2.3.1.bb
@@ -31,6 +31,7 @@ EXTRA_OECONF = "--disable-ldap \
 		--with-zlib=${STAGING_LIBDIR}/.. \
 		--with-bzip2=${STAGING_LIBDIR}/.. \
 		--with-readline=${STAGING_LIBDIR}/.. \
+		--with-mailprog=${sbindir}/sendmail \
 		--enable-gpg-is-gpg2 \
                "
 
-- 
2.32.0

[PATCH 4/9] mesa: Ensure megadrivers runtime mappings are deterministic

[Richard Purdie]

Add a sort to ensure the package dependency output is determnistic.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-graphics/mesa/mesa.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-graphics/mesa/mesa.inc b/meta/recipes-graphics/mesa/mesa.inc
index 0a7a3ca7bca..282671d30f3 100644
--- a/meta/recipes-graphics/mesa/mesa.inc
+++ b/meta/recipes-graphics/mesa/mesa.inc
@@ -257,7 +257,7 @@ python mesa_populate_packages() {
     import re
     dri_drivers_root = oe.path.join(d.getVar('PKGD'), d.getVar('libdir'), "dri")
     if os.path.isdir(dri_drivers_root):
-        dri_pkgs = os.listdir(dri_drivers_root)
+        dri_pkgs = sorted(os.listdir(dri_drivers_root))
         lib_name = d.expand("${MLPREFIX}mesa-megadriver")
         for p in dri_pkgs:
             m = re.match(r'^(.*)_dri\.so$', p)
-- 
2.32.0

[PATCH 5/9] util-linux: Fix reproducibility

[Richard Purdie]

Sort the list of files to ensure the pkgdata output is deterministic.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-core/util-linux/util-linux_2.37.2.bb | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/util-linux/util-linux_2.37.2.bb b/meta/recipes-core/util-linux/util-linux_2.37.2.bb
index 6f2808cc441..d609c30067f 100644
--- a/meta/recipes-core/util-linux/util-linux_2.37.2.bb
+++ b/meta/recipes-core/util-linux/util-linux_2.37.2.bb
@@ -37,12 +37,13 @@ python util_linux_binpackages () {
                     continue
 
                 pkg = os.path.basename(os.readlink(file))
-                extras[pkg] = extras.get(pkg, '') + ' ' + file.replace(dvar, '', 1)
+                extras.setdefault(pkg, [])
+                extras[pkg].append(file.replace(dvar, '', 1))
 
     pn = d.getVar('PN')
     for pkg, links in extras.items():
         of = d.getVar('FILES:' + pn + '-' + pkg)
-        links = of + links
+        links = of + " " + " ".join(sorted(links))
         d.setVar('FILES:' + pn + '-' + pkg, links)
 }
 
-- 
2.32.0

[PATCH 6/9] libtool: Allow libtool-cross to reproduce

[Richard Purdie]

The hostname removal from the script is useful to make libtool-cross
reproduce. Apply the patch everywhere as it doesn't cause any issues.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-devtools/libtool/libtool-2.4.6.inc | 1 +
 meta/recipes-devtools/libtool/libtool_2.4.6.bb  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/libtool/libtool-2.4.6.inc b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
index 6748d7468e7..7104c98c203 100644
--- a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
+++ b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
@@ -24,6 +24,7 @@ SRC_URI = "${GNU_MIRROR}/libtool/libtool-${PV}.tar.gz \
            file://0001-Makefile.am-make-sure-autoheader-run-before-autoconf.patch \
            file://0001-Makefile.am-make-sure-autoheader-run-before-automake.patch \
            file://lto-prefix.patch \
+           file://debian-no_hostname.patch \
           "
 
 SRC_URI[md5sum] = "addf44b646ddb4e3919805aa88fa7c5e"
diff --git a/meta/recipes-devtools/libtool/libtool_2.4.6.bb b/meta/recipes-devtools/libtool/libtool_2.4.6.bb
index 95bbc875f09..fb40ce7317a 100644
--- a/meta/recipes-devtools/libtool/libtool_2.4.6.bb
+++ b/meta/recipes-devtools/libtool/libtool_2.4.6.bb
@@ -1,6 +1,6 @@
 require libtool-${PV}.inc
 
-SRC_URI += "file://multilib.patch file://debian-no_hostname.patch"
+SRC_URI += "file://multilib.patch"
 
 RDEPENDS:${PN} += "bash"
 
-- 
2.32.0

[PATCH 7/9] gobject-introspection: Don't write $HOME into scripts

[Richard Purdie]

Writing an expanded version of $HOME into the wrapper script breaks
reproducibility. We don't need this here so don't.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../gobject-introspection/gobject-introspection_1.68.0.bb       | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.68.0.bb b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.68.0.bb
index cf1349f830a..b1d3d8abdcc 100644
--- a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.68.0.bb
+++ b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.68.0.bb
@@ -93,7 +93,7 @@ EOF
         # from the target sysroot.
         cat > ${B}/g-ir-scanner-wrapper << EOF
 #!/bin/sh
-# This prevents g-ir-scanner from writing cache data to $HOME
+# This prevents g-ir-scanner from writing cache data to user's HOME dir
 export GI_SCANNER_DISABLE_CACHE=1
 
 g-ir-scanner --lib-dirs-envvar=GIR_EXTRA_LIBS_PATH --use-binary-wrapper=${STAGING_BINDIR}/g-ir-scanner-qemuwrapper --use-ldd-wrapper=${STAGING_BINDIR}/g-ir-scanner-lddwrapper --add-include-path=${STAGING_DATADIR}/gir-1.0 --add-include-path=${STAGING_LIBDIR}/gir-1.0 "\$@"
-- 
2.32.0

[PATCH 8/9] oeqa/selftest/bbtests: Add uuid to force build test

[Richard Purdie]

With hash equivalence working properly, this test breaks as it detects
an unchanged tasks and doesn't rerun subsequent tasks.

Add a uuid to the payload to ensure it doesn't ever match
equivalence and the test works as intended.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/selftest/cases/bbtests.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/bbtests.py b/meta/lib/oeqa/selftest/cases/bbtests.py
index 8831de60658..65623640746 100644
--- a/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -83,8 +83,10 @@ class BitbakeTests(OESelftestTestCase):
 
     def test_force_task_1(self):
         # test 1 from bug 5875
+        import uuid
         test_recipe = 'zlib'
-        test_data = "Microsoft Made No Profit From Anyone's Zunes Yo"
+        # Need to use uuid otherwise hash equivlance would change the workflow
+        test_data = "Microsoft Made No Profit From Anyone's Zunes Yo %s" % uuid.uuid1()
         bb_vars = get_bb_vars(['D', 'PKGDEST', 'mandir'], test_recipe)
         image_dir = bb_vars['D']
         pkgsplit_dir = bb_vars['PKGDEST']
-- 
2.32.0

[PATCH 9/9] image: Exclude IMAGE_VERSION_SUFFIX from expansion in image tasks

[Richard Purdie]

Similarly to DATETIME, exclude IMAGE_VERSION_SUFFIX from expansion in
image tasks to ensure the sstate signatures are calculated correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/image.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 1d88ccd8192..c2f32320278 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -434,6 +434,7 @@ python () {
         localdata.delVar('DATETIME')
         localdata.delVar('DATE')
         localdata.delVar('TMPDIR')
+        localdata.delVar('IMAGE_VERSION_SUFFIX')
         vardepsexclude = (d.getVarFlag('IMAGE_CMD:' + realt, 'vardepsexclude', True) or '').split()
         for dep in vardepsexclude:
             localdata.delVar(dep)
-- 
2.32.0