[PATCH] glibc: improve reproducibility with multilib

[PATCH] glibc: improve reproducibility with multilib

[Juro Bystricky]

Multilib builds specify several loaders which will end up embedded in
some binaries or script files. To support reproducible builds, we must
ensure the loaders are always in deterministic order.

[YOCTO #2655]
[YOCTO #12478]
[YOCTO #12480]

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
---
 meta/recipes-core/glibc/glibc-ld.inc  | 4 ++--
 meta/recipes-core/glibc/glibc_2.26.bb | 8 +++-----
 2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-core/glibc/glibc-ld.inc b/meta/recipes-core/glibc/glibc-ld.inc
index c1d635d..607b27c 100644
--- a/meta/recipes-core/glibc/glibc-ld.inc
+++ b/meta/recipes-core/glibc/glibc-ld.inc
@@ -38,8 +38,8 @@ def glibc_dl_info(d):
         overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + item
         localdata.setVar("OVERRIDES", overrides)
         ld_append_if_tune_exists(localdata, infos, ld_info_all)
-    infos['ldconfig'] = ','.join(infos['ldconfig'])
-    infos['lddrewrite'] = ' '.join(infos['lddrewrite'])
+    infos['ldconfig'] = ','.join(sorted(infos['ldconfig']))
+    infos['lddrewrite'] = ' '.join(sorted(infos['lddrewrite']))
     return infos
 
 EGLIBC_KNOWN_INTERPRETER_NAMES = "${@glibc_dl_info(d)['ldconfig']}"
diff --git a/meta/recipes-core/glibc/glibc_2.26.bb b/meta/recipes-core/glibc/glibc_2.26.bb
index 7eb56b3..df520bd 100644
--- a/meta/recipes-core/glibc/glibc_2.26.bb
+++ b/meta/recipes-core/glibc/glibc_2.26.bb
@@ -133,12 +133,10 @@ do_compile () {
 	if [ -n "${RTLDLIST}" ]
 	then
 		prevrtld=`cat ${B}/elf/ldd | grep "^RTLDLIST=" | sed 's#^RTLDLIST="\?\([^"]*\)"\?$#\1#'`
-		if [ "${prevrtld}" != "${RTLDLIST}" ]
-		then
-			sed -i ${B}/elf/ldd -e "s#^RTLDLIST=.*\$#RTLDLIST=\"${prevrtld} ${RTLDLIST}\"#"
-		fi
+		# remove duplicate entries
+		newrtld=`echo "${prevrtld} ${RTLDLIST}" | xargs -n1 | sort -u | xargs`
+		sed -i ${B}/elf/ldd -e "s#^RTLDLIST=.*\$#RTLDLIST=\"${newrtld}\"#"
 	fi
-
 }
 
 # Use the host locale archive when built for nativesdk so that we don't need to
-- 
2.7.4

Re: [PATCH] glibc: improve reproducibility with multilib

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

On Mon, Feb 5, 2018 at 11:26 AM, Juro Bystricky <juro.bystricky@intel.com>
wrote:

> Multilib builds specify several loaders which will end up embedded in
> some binaries or script files. To support reproducible builds, we must
> ensure the loaders are always in deterministic order.
>
> [YOCTO #2655]
> [YOCTO #12478]
> [YOCTO #12480]
>
> Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
> ---
>  meta/recipes-core/glibc/glibc-ld.inc  | 4 ++--
>  meta/recipes-core/glibc/glibc_2.26.bb | 8 +++-----
>  2 files changed, 5 insertions(+), 7 deletions(-)
>
> diff --git a/meta/recipes-core/glibc/glibc-ld.inc
> b/meta/recipes-core/glibc/glibc-ld.inc
> index c1d635d..607b27c 100644
> --- a/meta/recipes-core/glibc/glibc-ld.inc
> +++ b/meta/recipes-core/glibc/glibc-ld.inc
> @@ -38,8 +38,8 @@ def glibc_dl_info(d):
>          overrides = localdata.getVar("OVERRIDES", False) +
> ":virtclass-multilib-" + item
>          localdata.setVar("OVERRIDES", overrides)
>          ld_append_if_tune_exists(localdata, infos, ld_info_all)
> -    infos['ldconfig'] = ','.join(infos['ldconfig'])
> -    infos['lddrewrite'] = ' '.join(infos['lddrewrite'])
> +    infos['ldconfig'] = ','.join(sorted(infos['ldconfig']))
> +    infos['lddrewrite'] = ' '.join(sorted(infos['lddrewrite']))
>      return infos
>
>  EGLIBC_KNOWN_INTERPRETER_NAMES = "${@glibc_dl_info(d)['ldconfig']}"
> diff --git a/meta/recipes-core/glibc/glibc_2.26.bb
> b/meta/recipes-core/glibc/glibc_2.26.bb
> index 7eb56b3..df520bd 100644
> --- a/meta/recipes-core/glibc/glibc_2.26.bb
> +++ b/meta/recipes-core/glibc/glibc_2.26.bb
> @@ -133,12 +133,10 @@ do_compile () {
>         if [ -n "${RTLDLIST}" ]
>         then
>                 prevrtld=`cat ${B}/elf/ldd | grep "^RTLDLIST=" | sed
> 's#^RTLDLIST="\?\([^"]*\)"\?$#\1#'`
> -               if [ "${prevrtld}" != "${RTLDLIST}" ]
> -               then
> -                       sed -i ${B}/elf/ldd -e
> "s#^RTLDLIST=.*\$#RTLDLIST=\"${prevrtld} ${RTLDLIST}\"#"
> -               fi
> +               # remove duplicate entries
> +               newrtld=`echo "${prevrtld} ${RTLDLIST}" | xargs -n1 | sort
> -u | xargs`
>


This “| xargs” at the end seems redundant. xargs what? xargs is used to run
a command on each word of the input..
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3783 bytes --]

Re: [PATCH] glibc: improve reproducibility with multilib

[Bystricky, Juro]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]


> This “| xargs” at the end seems redundant. xargs what? xargs is used to run a command on each word of > the input..

It forces the words into one line:

$ echo "zebra fox elephant elephant lion wolf fox " | xargs -n1 | sort -u | xargs
elephant fox lion wolf zebra

$echo "zebra fox elephant elephant lion wolf fox " | xargs -n1 | sort -u
elephant
fox
lion
wolf
zebra



[-- Attachment #2: Type: text/html, Size: 998 bytes --]