[master][PATCH 0/4] Reproducible LTO builds

[master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

Currently, enabling Link Time Optimization via:

require conf/distro/include/lto.inc
DISTRO_FEATURES_append = " lto"

makes the build non-reproducible (as in
https://reproducible-builds.org/).  This patch set fixes some of the
problems, but is not yet a complete solution.

This patch set addresses the following bugs:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490

openembedded-core patches
1/4: lto.inc: disable LTO for grub
2/4: gcc: Backport patch to make LTO builds more reproducible
3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS

openembedded-devel patches
4/4: net-snmp: fix QA Issue after LDFLAGS change

Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
TARGET_CFLAGS).  This could possibly be done differently, such as by
removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
if there is a better/preferred way to do it.

The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
included a patch to fix that.  Other packages might have been broken as
well, although I haven't found any others.

With all these patches, there are still many shared libraries in /lib
and /usr/lib that are still not reproducible with LTO enabled because
x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
the libbz2 package from the bzip2 recipe is a good example.  I do not
have a patch for that problem.  I will leave that to someone else.

So this is what we need to achieve reproducible LTO builds:
*) This patch set
*) A fix for libtool
*) Add LTO to reproducible builds testing if appropriate (e.g.
meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
*) Fixes for individual packages, if needed
*) Once everything works, update the documentation again (I previously
requested a documentation change for this bug, which will eventually
show up at
http://docs.yoctoproject.org/test-manual/reproducible-builds.html).

Tony Battersby
Cybernetics

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Randy MacLeod]

On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> Currently, enabling Link Time Optimization via:
> 
> require conf/distro/include/lto.inc
> DISTRO_FEATURES_append = " lto"
> 
> makes the build non-reproducible (as in
> https://reproducible-builds.org/).  This patch set fixes some of the
> problems, but is not yet a complete solution.
> 
> This patch set addresses the following bugs:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> 
> openembedded-core patches
> 1/4: lto.inc: disable LTO for grub
> 2/4: gcc: Backport patch to make LTO builds more reproducible
> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> 
> openembedded-devel patches
> 4/4: net-snmp: fix QA Issue after LDFLAGS change
> 
> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> TARGET_CFLAGS).  This could possibly be done differently, such as by
> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> if there is a better/preferred way to do it.
> 
> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> included a patch to fix that.  Other packages might have been broken as
> well, although I haven't found any others.
> 
> With all these patches, there are still many shared libraries in /lib
> and /usr/lib that are still not reproducible with LTO enabled because
> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> the libbz2 package from the bzip2 recipe is a good example.  I do not
> have a patch for that problem.  I will leave that to someone else.
> 
> So this is what we need to achieve reproducible LTO builds:
> *) This patch set
> *) A fix for libtool
> *) Add LTO to reproducible builds testing if appropriate (e.g.
> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> *) Fixes for individual packages, if needed
> *) Once everything works, update the documentation again (I previously
> requested a documentation change for this bug, which will eventually
> show up at
> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).

This is certainly a few steps in the right direction with a good outline
of what's next.

I suspect that Richard has been busy with other things like overrides
and the update to glibc-2.34.

Do you have any suggestions or questions about what we should do
with this patch series Richard?

../Randy


> 
> Tony Battersby
> Cybernetics
> 
> 
> 
> 
> 


-- 
# Randy MacLeod
# Wind River Linux

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Richard Purdie]

On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> > Currently, enabling Link Time Optimization via:
> > 
> > require conf/distro/include/lto.inc
> > DISTRO_FEATURES_append = " lto"
> > 
> > makes the build non-reproducible (as in
> > https://reproducible-builds.org/).  This patch set fixes some of the
> > problems, but is not yet a complete solution.
> > 
> > This patch set addresses the following bugs:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> > 
> > openembedded-core patches
> > 1/4: lto.inc: disable LTO for grub
> > 2/4: gcc: Backport patch to make LTO builds more reproducible
> > 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> > 
> > openembedded-devel patches
> > 4/4: net-snmp: fix QA Issue after LDFLAGS change
> > 
> > Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> > TARGET_CFLAGS).  This could possibly be done differently, such as by
> > removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> > TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> > if there is a better/preferred way to do it.
> > 
> > The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> > included a patch to fix that.  Other packages might have been broken as
> > well, although I haven't found any others.
> > 
> > With all these patches, there are still many shared libraries in /lib
> > and /usr/lib that are still not reproducible with LTO enabled because
> > x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> > patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> > the libbz2 package from the bzip2 recipe is a good example.  I do not
> > have a patch for that problem.  I will leave that to someone else.
> > 
> > So this is what we need to achieve reproducible LTO builds:
> > *) This patch set
> > *) A fix for libtool
> > *) Add LTO to reproducible builds testing if appropriate (e.g.
> > meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> > *) Fixes for individual packages, if needed
> > *) Once everything works, update the documentation again (I previously
> > requested a documentation change for this bug, which will eventually
> > show up at
> > http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> 
> This is certainly a few steps in the right direction with a good outline
> of what's next.
> 
> I suspect that Richard has been busy with other things like overrides
> and the update to glibc-2.34.
> 
> Do you have any suggestions or questions about what we should do
> with this patch series Richard?

Thanks for the reminder, I've been meaning to queue/test this which I will
do. The gcc version has been bumped but I ported the patch forward to it
and the patches are in master-next for testing. It is great to see this
support moving forward.

Much as it pains me to admit having any libtool knowledge, I may be able to
help with that bit. There is an untested patch below which may address that
issue, or at least give a good hint on how to do it if it doesn't work. I
don't have an environment to test easily right now.

Cheers,

Richard


From 7681b0a1461f22034e1f70328a060db34474c60f Mon Sep 17 00:00:00 2001
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Thu, 5 Aug 2021 11:18:20 +0100
Subject: libtool: Fix lto option passing?

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-devtools/libtool/libtool-2.4.6.inc        |  1 +
 meta/recipes-devtools/libtool/libtool/lto-prefix.patch | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)
 create mode 100644 meta/recipes-devtools/libtool/libtool/lto-prefix.patch

diff --git a/meta/recipes-devtools/libtool/libtool-2.4.6.inc b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
index c1cbceb51f..6748d7468e 100644
--- a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
+++ b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
@@ -23,6 +23,7 @@ SRC_URI = "${GNU_MIRROR}/libtool/libtool-${PV}.tar.gz \
            file://0001-libtool-Check-for-static-libs-for-internal-compiler-.patch \
            file://0001-Makefile.am-make-sure-autoheader-run-before-autoconf.patch \
            file://0001-Makefile.am-make-sure-autoheader-run-before-automake.patch \
+           file://lto-prefix.patch \
           "
 
 SRC_URI[md5sum] = "addf44b646ddb4e3919805aa88fa7c5e"
diff --git a/meta/recipes-devtools/libtool/libtool/lto-prefix.patch b/meta/recipes-
devtools/libtool/libtool/lto-prefix.patch
new file mode 100644
index 0000000000..f788b3fc27
--- /dev/null
+++ b/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
@@ -0,0 +1,16 @@
+Index: libtool-2.4.6/build-aux/ltmain.in
+===================================================================
+--- libtool-2.4.6.orig/build-aux/ltmain.in
++++ libtool-2.4.6/build-aux/ltmain.in
+@@ -5424,9 +5424,10 @@ func_mode_link ()
+       # --sysroot=*          for sysroot support
+       # -O*, -g*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time optimization
+       # -stdlib=*            select c++ std lib with clang
++      # -f*-prefix-map*      needed for lto linking
+       -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
+       -t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
+-      -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*)
++      -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-f*-prefix-map*)
+         func_quote_for_eval "$arg"
+ 	arg=$func_quote_for_eval_result
+         func_append compile_command " $arg"

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/5/21 6:21 AM, Richard Purdie wrote:
> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>> Currently, enabling Link Time Optimization via:
>>>
>>> require conf/distro/include/lto.inc
>>> DISTRO_FEATURES_append = " lto"
>>>
>>> makes the build non-reproducible (as in
>>> https://reproducible-builds.org/).  This patch set fixes some of the
>>> problems, but is not yet a complete solution.
>>>
>>> This patch set addresses the following bugs:
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>
>>> openembedded-core patches
>>> 1/4: lto.inc: disable LTO for grub
>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>
>>> openembedded-devel patches
>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>
>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>>> if there is a better/preferred way to do it.
>>>
>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>> included a patch to fix that.  Other packages might have been broken as
>>> well, although I haven't found any others.
>>>
>>> With all these patches, there are still many shared libraries in /lib
>>> and /usr/lib that are still not reproducible with LTO enabled because
>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>>> have a patch for that problem.  I will leave that to someone else.
>>>
>>> So this is what we need to achieve reproducible LTO builds:
>>> *) This patch set
>>> *) A fix for libtool
>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>> *) Fixes for individual packages, if needed
>>> *) Once everything works, update the documentation again (I previously
>>> requested a documentation change for this bug, which will eventually
>>> show up at
>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>> This is certainly a few steps in the right direction with a good outline
>> of what's next.
>>
>> I suspect that Richard has been busy with other things like overrides
>> and the update to glibc-2.34.
>>
>> Do you have any suggestions or questions about what we should do
>> with this patch series Richard?
> Thanks for the reminder, I've been meaning to queue/test this which I will
> do. The gcc version has been bumped but I ported the patch forward to it
> and the patches are in master-next for testing. It is great to see this
> support moving forward.
>
> Much as it pains me to admit having any libtool knowledge, I may be able to
> help with that bit. There is an untested patch below which may address that
> issue, or at least give a good hint on how to do it if it doesn't work. I
> don't have an environment to test easily right now.
>
> Cheers,
>
> Richard

I tested your libtool patch in master-next with the bzip2 recipe.  It
works as intended; libbz2.so.1.0.6 and all the other output files are
now reproducible with LTO enabled.  I also verified that libtool is
passing the -f*-prefix-map flags to the link command in log.do_compile
as intended.

I also verified that core-image-minimal builds successfully with the
libtool patch.

Tony Battersby


>
> >From 7681b0a1461f22034e1f70328a060db34474c60f Mon Sep 17 00:00:00 2001
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date: Thu, 5 Aug 2021 11:18:20 +0100
> Subject: libtool: Fix lto option passing?
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/recipes-devtools/libtool/libtool-2.4.6.inc        |  1 +
>  meta/recipes-devtools/libtool/libtool/lto-prefix.patch | 16 ++++++++++++++++
>  2 files changed, 17 insertions(+)
>  create mode 100644 meta/recipes-devtools/libtool/libtool/lto-prefix.patch
>
> diff --git a/meta/recipes-devtools/libtool/libtool-2.4.6.inc b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> index c1cbceb51f..6748d7468e 100644
> --- a/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> +++ b/meta/recipes-devtools/libtool/libtool-2.4.6.inc
> @@ -23,6 +23,7 @@ SRC_URI = "${GNU_MIRROR}/libtool/libtool-${PV}.tar.gz \
>             file://0001-libtool-Check-for-static-libs-for-internal-compiler-.patch \
>             file://0001-Makefile.am-make-sure-autoheader-run-before-autoconf.patch \
>             file://0001-Makefile.am-make-sure-autoheader-run-before-automake.patch \
> +           file://lto-prefix.patch \
>            "
>  
>  SRC_URI[md5sum] = "addf44b646ddb4e3919805aa88fa7c5e"
> diff --git a/meta/recipes-devtools/libtool/libtool/lto-prefix.patch b/meta/recipes-
> devtools/libtool/libtool/lto-prefix.patch
> new file mode 100644
> index 0000000000..f788b3fc27
> --- /dev/null
> +++ b/meta/recipes-devtools/libtool/libtool/lto-prefix.patch
> @@ -0,0 +1,16 @@
> +Index: libtool-2.4.6/build-aux/ltmain.in
> +===================================================================
> +--- libtool-2.4.6.orig/build-aux/ltmain.in
> ++++ libtool-2.4.6/build-aux/ltmain.in
> +@@ -5424,9 +5424,10 @@ func_mode_link ()
> +       # --sysroot=*          for sysroot support
> +       # -O*, -g*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time optimization
> +       # -stdlib=*            select c++ std lib with clang
> ++      # -f*-prefix-map*      needed for lto linking
> +       -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
> +       -t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
> +-      -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*)
> ++      -O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-f*-prefix-map*)
> +         func_quote_for_eval "$arg"
> + 	arg=$func_quote_for_eval_result
> +         func_append compile_command " $arg"
>
>
>

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Richard Purdie]

On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
> On 8/5/21 6:21 AM, Richard Purdie wrote:
> > On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> > > On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> > > > Currently, enabling Link Time Optimization via:
> > > > 
> > > > require conf/distro/include/lto.inc
> > > > DISTRO_FEATURES_append = " lto"
> > > > 
> > > > makes the build non-reproducible (as in
> > > > https://reproducible-builds.org/).  This patch set fixes some of the
> > > > problems, but is not yet a complete solution.
> > > > 
> > > > This patch set addresses the following bugs:
> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> > > > 
> > > > openembedded-core patches
> > > > 1/4: lto.inc: disable LTO for grub
> > > > 2/4: gcc: Backport patch to make LTO builds more reproducible
> > > > 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> > > > 
> > > > openembedded-devel patches
> > > > 4/4: net-snmp: fix QA Issue after LDFLAGS change
> > > > 
> > > > Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> > > > TARGET_CFLAGS).  This could possibly be done differently, such as by
> > > > removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> > > > TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> > > > if there is a better/preferred way to do it.
> > > > 
> > > > The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> > > > included a patch to fix that.  Other packages might have been broken as
> > > > well, although I haven't found any others.
> > > > 
> > > > With all these patches, there are still many shared libraries in /lib
> > > > and /usr/lib that are still not reproducible with LTO enabled because
> > > > x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> > > > patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> > > > the libbz2 package from the bzip2 recipe is a good example.  I do not
> > > > have a patch for that problem.  I will leave that to someone else.
> > > > 
> > > > So this is what we need to achieve reproducible LTO builds:
> > > > *) This patch set
> > > > *) A fix for libtool
> > > > *) Add LTO to reproducible builds testing if appropriate (e.g.
> > > > meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> > > > *) Fixes for individual packages, if needed
> > > > *) Once everything works, update the documentation again (I previously
> > > > requested a documentation change for this bug, which will eventually
> > > > show up at
> > > > http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> > > This is certainly a few steps in the right direction with a good outline
> > > of what's next.
> > > 
> > > I suspect that Richard has been busy with other things like overrides
> > > and the update to glibc-2.34.
> > > 
> > > Do you have any suggestions or questions about what we should do
> > > with this patch series Richard?
> > Thanks for the reminder, I've been meaning to queue/test this which I will
> > do. The gcc version has been bumped but I ported the patch forward to it
> > and the patches are in master-next for testing. It is great to see this
> > support moving forward.
> > 
> > Much as it pains me to admit having any libtool knowledge, I may be able to
> > help with that bit. There is an untested patch below which may address that
> > issue, or at least give a good hint on how to do it if it doesn't work. I
> > don't have an environment to test easily right now.
> > 
> > Cheers,
> > 
> > Richard
> 
> I tested your libtool patch in master-next with the bzip2 recipe.  It
> works as intended; libbz2.so.1.0.6 and all the other output files are
> now reproducible with LTO enabled.  I also verified that libtool is
> passing the -f*-prefix-map flags to the link command in log.do_compile
> as intended.
> 
> I also verified that core-image-minimal builds successfully with the
> libtool patch.

Thanks for testing that, sounds like it helps that issue. I'll queue that patch.

Unfortunately even with the ruby fix I queued, we still see reproduciblity
issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg 
packages:

https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio

We'll need to resolve those before I can merge the flags change.

Diffoscope output:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/

Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.

Cheers,

Richard

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/6/21 1:33 AM, Richard Purdie wrote:
> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>>>> Currently, enabling Link Time Optimization via:
>>>>>
>>>>> require conf/distro/include/lto.inc
>>>>> DISTRO_FEATURES_append = " lto"
>>>>>
>>>>> makes the build non-reproducible (as in
>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
>>>>> problems, but is not yet a complete solution.
>>>>>
>>>>> This patch set addresses the following bugs:
>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>>>
>>>>> openembedded-core patches
>>>>> 1/4: lto.inc: disable LTO for grub
>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>>>
>>>>> openembedded-devel patches
>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>>>
>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>>>>> if there is a better/preferred way to do it.
>>>>>
>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>>>> included a patch to fix that.  Other packages might have been broken as
>>>>> well, although I haven't found any others.
>>>>>
>>>>> With all these patches, there are still many shared libraries in /lib
>>>>> and /usr/lib that are still not reproducible with LTO enabled because
>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>>>>> have a patch for that problem.  I will leave that to someone else.
>>>>>
>>>>> So this is what we need to achieve reproducible LTO builds:
>>>>> *) This patch set
>>>>> *) A fix for libtool
>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>>>> *) Fixes for individual packages, if needed
>>>>> *) Once everything works, update the documentation again (I previously
>>>>> requested a documentation change for this bug, which will eventually
>>>>> show up at
>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>>>> This is certainly a few steps in the right direction with a good outline
>>>> of what's next.
>>>>
>>>> I suspect that Richard has been busy with other things like overrides
>>>> and the update to glibc-2.34.
>>>>
>>>> Do you have any suggestions or questions about what we should do
>>>> with this patch series Richard?
>>> Thanks for the reminder, I've been meaning to queue/test this which I will
>>> do. The gcc version has been bumped but I ported the patch forward to it
>>> and the patches are in master-next for testing. It is great to see this
>>> support moving forward.
>>>
>>> Much as it pains me to admit having any libtool knowledge, I may be able to
>>> help with that bit. There is an untested patch below which may address that
>>> issue, or at least give a good hint on how to do it if it doesn't work. I
>>> don't have an environment to test easily right now.
>>>
>>> Cheers,
>>>
>>> Richard
>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>> works as intended; libbz2.so.1.0.6 and all the other output files are
>> now reproducible with LTO enabled.  I also verified that libtool is
>> passing the -f*-prefix-map flags to the link command in log.do_compile
>> as intended.
>>
>> I also verified that core-image-minimal builds successfully with the
>> libtool patch.
> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
>
> Unfortunately even with the ruby fix I queued, we still see reproduciblity
> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg 
> packages:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>
> We'll need to resolve those before I can merge the flags change.
>
> Diffoscope output:
>
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>
> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>
> Cheers,
>
> Richard
>
It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h. 
There is already a patch
poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
that removes some of these items for the sake of reproducibiltiy.  But I
do not know ruby, so I do not know the implications of doing the same
thing for MJIT_DLDFLAGS.

Tony Battersby

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/6/21 12:44 PM, Tony Battersby wrote:
> On 8/6/21 1:33 AM, Richard Purdie wrote:
>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>>>>> Currently, enabling Link Time Optimization via:
>>>>>>
>>>>>> require conf/distro/include/lto.inc
>>>>>> DISTRO_FEATURES_append = " lto"
>>>>>>
>>>>>> makes the build non-reproducible (as in
>>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
>>>>>> problems, but is not yet a complete solution.
>>>>>>
>>>>>> This patch set addresses the following bugs:
>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>>>>
>>>>>> openembedded-core patches
>>>>>> 1/4: lto.inc: disable LTO for grub
>>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>>>>
>>>>>> openembedded-devel patches
>>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>>>>
>>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>>>>>> if there is a better/preferred way to do it.
>>>>>>
>>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>>>>> included a patch to fix that.  Other packages might have been broken as
>>>>>> well, although I haven't found any others.
>>>>>>
>>>>>> With all these patches, there are still many shared libraries in /lib
>>>>>> and /usr/lib that are still not reproducible with LTO enabled because
>>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>>>>>> have a patch for that problem.  I will leave that to someone else.
>>>>>>
>>>>>> So this is what we need to achieve reproducible LTO builds:
>>>>>> *) This patch set
>>>>>> *) A fix for libtool
>>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>>>>> *) Fixes for individual packages, if needed
>>>>>> *) Once everything works, update the documentation again (I previously
>>>>>> requested a documentation change for this bug, which will eventually
>>>>>> show up at
>>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>>>>> This is certainly a few steps in the right direction with a good outline
>>>>> of what's next.
>>>>>
>>>>> I suspect that Richard has been busy with other things like overrides
>>>>> and the update to glibc-2.34.
>>>>>
>>>>> Do you have any suggestions or questions about what we should do
>>>>> with this patch series Richard?
>>>> Thanks for the reminder, I've been meaning to queue/test this which I will
>>>> do. The gcc version has been bumped but I ported the patch forward to it
>>>> and the patches are in master-next for testing. It is great to see this
>>>> support moving forward.
>>>>
>>>> Much as it pains me to admit having any libtool knowledge, I may be able to
>>>> help with that bit. There is an untested patch below which may address that
>>>> issue, or at least give a good hint on how to do it if it doesn't work. I
>>>> don't have an environment to test easily right now.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>>> works as intended; libbz2.so.1.0.6 and all the other output files are
>>> now reproducible with LTO enabled.  I also verified that libtool is
>>> passing the -f*-prefix-map flags to the link command in log.do_compile
>>> as intended.
>>>
>>> I also verified that core-image-minimal builds successfully with the
>>> libtool patch.
>> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
>>
>> Unfortunately even with the ruby fix I queued, we still see reproduciblity
>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg 
>> packages:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>>
>> We'll need to resolve those before I can merge the flags change.
>>
>> Diffoscope output:
>>
>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>>
>> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>>
>> Cheers,
>>
>> Richard
>>
> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h. 
> There is already a patch
> poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
> that removes some of these items for the sake of reproducibiltiy.  But I
> do not know ruby, so I do not know the implications of doing the same
> thing for MJIT_DLDFLAGS.
>
> Tony Battersby
>
>
>
Richard,

I noticed that you already fixed the problems in curl and gettext in
master-next.  Here is a patch for ruby if you haven't fixed that already.

Tony Battersby

From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
From: Tony Battersby <tonyb@cybernetics.com>
Date: Fri, 6 Aug 2021 16:47:14 -0400
Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS

If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
longer reproducible.  Fix this.

Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
---
 ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
 meta/recipes-devtools/ruby/ruby_3.0.2.bb      |  1 +
 2 files changed, 43 insertions(+)
 create mode 100644 meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch

diff --git a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
new file mode 100644
index 0000000000..9387506c26
--- /dev/null
+++ b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
@@ -0,0 +1,42 @@
+Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
+
+If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
+reproducible.  Fix this.
+
+Upstream-Status: Inapproppriate [oe-core specific]
+Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
+---
+--- a/tool/mjit_archflag.sh
++++ b/tool/mjit_archflag.sh
+@@ -7,6 +7,20 @@ quote() {
+     echo
+ }
+ 
++quote_filtered() {
++    printf "#${indent}define $1"
++    while shift && [ "$#" -gt 0 ]; do
++	case "$1" in
++	    -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
++		;;
++	    *)
++		printf ' "%s"'$sep "$1"
++		;;
++	esac
++    done
++    echo
++}
++
+ archs=""
+ arch_flag=""
+ 
+--- a/template/Makefile.in
++++ b/template/Makefile.in
+@@ -666,7 +666,7 @@ mjit_config.h:
+ 	quote "MJIT_OPTFLAGS   " $(MJIT_OPTFLAGS); \
+ 	quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
+ 	quote "MJIT_LDSHARED   " ; \
+-	quote "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
++	quote_filtered "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
+ 	quote "MJIT_LIBS       " $(LIBRUBYARG_SHARED); \
+ 	quote 'PRELOADENV       "@PRELOADENV@"'; \
+ 	indent=$${archs:+'  '}; \
diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
index 0b9c342def..38e594a59e 100644
--- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
+++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
@@ -6,6 +6,7 @@ SRC_URI += " \
            file://remove_has_include_macros.patch \
            file://run-ptest \
            file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
+           file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
            "
 
 SRC_URI[sha256sum] = "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
-- 
2.25.1

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Khem Raj]

I am seeing bunch of failures on meta-oe and meta-atmel

https://errors.yoctoproject.org/Errors/Details/600064/
https://errors.yoctoproject.org/Errors/Details/600065/
https://errors.yoctoproject.org/Errors/Details/600071/

On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <tonyb@cybernetics.com> wrote:
>
> On 8/6/21 12:44 PM, Tony Battersby wrote:
> > On 8/6/21 1:33 AM, Richard Purdie wrote:
> >> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
> >>> On 8/5/21 6:21 AM, Richard Purdie wrote:
> >>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> >>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> >>>>>> Currently, enabling Link Time Optimization via:
> >>>>>>
> >>>>>> require conf/distro/include/lto.inc
> >>>>>> DISTRO_FEATURES_append = " lto"
> >>>>>>
> >>>>>> makes the build non-reproducible (as in
> >>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
> >>>>>> problems, but is not yet a complete solution.
> >>>>>>
> >>>>>> This patch set addresses the following bugs:
> >>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> >>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> >>>>>>
> >>>>>> openembedded-core patches
> >>>>>> 1/4: lto.inc: disable LTO for grub
> >>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
> >>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> >>>>>>
> >>>>>> openembedded-devel patches
> >>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
> >>>>>>
> >>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> >>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
> >>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> >>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> >>>>>> if there is a better/preferred way to do it.
> >>>>>>
> >>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> >>>>>> included a patch to fix that.  Other packages might have been broken as
> >>>>>> well, although I haven't found any others.
> >>>>>>
> >>>>>> With all these patches, there are still many shared libraries in /lib
> >>>>>> and /usr/lib that are still not reproducible with LTO enabled because
> >>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> >>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> >>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
> >>>>>> have a patch for that problem.  I will leave that to someone else.
> >>>>>>
> >>>>>> So this is what we need to achieve reproducible LTO builds:
> >>>>>> *) This patch set
> >>>>>> *) A fix for libtool
> >>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
> >>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> >>>>>> *) Fixes for individual packages, if needed
> >>>>>> *) Once everything works, update the documentation again (I previously
> >>>>>> requested a documentation change for this bug, which will eventually
> >>>>>> show up at
> >>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> >>>>> This is certainly a few steps in the right direction with a good outline
> >>>>> of what's next.
> >>>>>
> >>>>> I suspect that Richard has been busy with other things like overrides
> >>>>> and the update to glibc-2.34.
> >>>>>
> >>>>> Do you have any suggestions or questions about what we should do
> >>>>> with this patch series Richard?
> >>>> Thanks for the reminder, I've been meaning to queue/test this which I will
> >>>> do. The gcc version has been bumped but I ported the patch forward to it
> >>>> and the patches are in master-next for testing. It is great to see this
> >>>> support moving forward.
> >>>>
> >>>> Much as it pains me to admit having any libtool knowledge, I may be able to
> >>>> help with that bit. There is an untested patch below which may address that
> >>>> issue, or at least give a good hint on how to do it if it doesn't work. I
> >>>> don't have an environment to test easily right now.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Richard
> >>> I tested your libtool patch in master-next with the bzip2 recipe.  It
> >>> works as intended; libbz2.so.1.0.6 and all the other output files are
> >>> now reproducible with LTO enabled.  I also verified that libtool is
> >>> passing the -f*-prefix-map flags to the link command in log.do_compile
> >>> as intended.
> >>>
> >>> I also verified that core-image-minimal builds successfully with the
> >>> libtool patch.
> >> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
> >>
> >> Unfortunately even with the ruby fix I queued, we still see reproduciblity
> >> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg
> >> packages:
> >>
> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
> >>
> >> We'll need to resolve those before I can merge the flags change.
> >>
> >> Diffoscope output:
> >>
> >> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
> >>
> >> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
> >>
> >> Cheers,
> >>
> >> Richard
> >>
> > It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
> > build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h.
> > There is already a patch
> > poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
> > that removes some of these items for the sake of reproducibiltiy.  But I
> > do not know ruby, so I do not know the implications of doing the same
> > thing for MJIT_DLDFLAGS.
> >
> > Tony Battersby
> >
> >
> >
> Richard,
>
> I noticed that you already fixed the problems in curl and gettext in
> master-next.  Here is a patch for ruby if you haven't fixed that already.
>
> Tony Battersby
>
> From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
> From: Tony Battersby <tonyb@cybernetics.com>
> Date: Fri, 6 Aug 2021 16:47:14 -0400
> Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS
>
> If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
> longer reproducible.  Fix this.
>
> Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
> ---
>  ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
>  meta/recipes-devtools/ruby/ruby_3.0.2.bb      |  1 +
>  2 files changed, 43 insertions(+)
>  create mode 100644 meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>
> diff --git a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
> new file mode 100644
> index 0000000000..9387506c26
> --- /dev/null
> +++ b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
> @@ -0,0 +1,42 @@
> +Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
> +
> +If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
> +reproducible.  Fix this.
> +
> +Upstream-Status: Inapproppriate [oe-core specific]
> +Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
> +---
> +--- a/tool/mjit_archflag.sh
> ++++ b/tool/mjit_archflag.sh
> +@@ -7,6 +7,20 @@ quote() {
> +     echo
> + }
> +
> ++quote_filtered() {
> ++    printf "#${indent}define $1"
> ++    while shift && [ "$#" -gt 0 ]; do
> ++      case "$1" in
> ++          -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
> ++              ;;
> ++          *)
> ++              printf ' "%s"'$sep "$1"
> ++              ;;
> ++      esac
> ++    done
> ++    echo
> ++}
> ++
> + archs=""
> + arch_flag=""
> +
> +--- a/template/Makefile.in
> ++++ b/template/Makefile.in
> +@@ -666,7 +666,7 @@ mjit_config.h:
> +       quote "MJIT_OPTFLAGS   " $(MJIT_OPTFLAGS); \
> +       quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
> +       quote "MJIT_LDSHARED   " ; \
> +-      quote "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
> ++      quote_filtered "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
> +       quote "MJIT_LIBS       " $(LIBRUBYARG_SHARED); \
> +       quote 'PRELOADENV       "@PRELOADENV@"'; \
> +       indent=$${archs:+'  '}; \
> diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> index 0b9c342def..38e594a59e 100644
> --- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> +++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> @@ -6,6 +6,7 @@ SRC_URI += " \
>             file://remove_has_include_macros.patch \
>             file://run-ptest \
>             file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
> +           file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
>             "
>
>  SRC_URI[sha256sum] = "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
> --
> 2.25.1
>

[meta-networking][master][PATCH 1/2] curlpp: fix QA Issue after LDFLAGS change

[Tony Battersby]

Adding -f*-prefix-map to LDFLAGS caused the following issue:

QA Issue: curlpp.pc failed sanity test (tmpdir)

Fix by filtering out -f*-prefix-map from *.pc files.

[YOCTO #14481]

Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
---
 meta-networking/recipes-support/curlpp/curlpp_0.8.1.bb | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta-networking/recipes-support/curlpp/curlpp_0.8.1.bb b/meta-networking/recipes-support/curlpp/curlpp_0.8.1.bb
index 6ce52d717..263de81c7 100644
--- a/meta-networking/recipes-support/curlpp/curlpp_0.8.1.bb
+++ b/meta-networking/recipes-support/curlpp/curlpp_0.8.1.bb
@@ -16,3 +16,10 @@ S = "${WORKDIR}/git"
 inherit cmake pkgconfig binconfig
 
 BBCLASSEXTEND = "native nativesdk"
+
+do_install:append() {
+    sed -e 's@[^ ]*-ffile-prefix-map=[^ "]*@@g' \
+        -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \
+        -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \
+        -i ${D}${libdir}/pkgconfig/*.pc
+}
-- 
2.25.1

[meta-oe][master][PATCH 2/2] ldns: fix QA Issue after LDFLAGS change

[Tony Battersby]

Adding -f*-prefix-map to LDFLAGS caused the following issue:

QA Issue: ldns.pc failed sanity test (tmpdir)

Fix by filtering out -f*-prefix-map from *.pc files.

[YOCTO #14481]

Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
---
 meta-oe/recipes-devtools/ldns/ldns_1.7.1.bb | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta-oe/recipes-devtools/ldns/ldns_1.7.1.bb b/meta-oe/recipes-devtools/ldns/ldns_1.7.1.bb
index 2a52dd688..2ce669154 100644
--- a/meta-oe/recipes-devtools/ldns/ldns_1.7.1.bb
+++ b/meta-oe/recipes-devtools/ldns/ldns_1.7.1.bb
@@ -16,3 +16,10 @@ PACKAGECONFIG[drill] = "--with-drill,--without-drill"
 
 EXTRA_OECONF = "--with-ssl=${STAGING_EXECPREFIXDIR} \
                 libtool=${TARGET_PREFIX}libtool"
+
+do_install:append() {
+    sed -e 's@[^ ]*-ffile-prefix-map=[^ "]*@@g' \
+        -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \
+        -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \
+        -i ${D}${libdir}/pkgconfig/*.pc
+}
-- 
2.25.1

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Tony Battersby]

On 8/6/21 10:19 PM, Khem Raj wrote:
> I am seeing bunch of failures on meta-oe and meta-atmel
>
> https://errors.yoctoproject.org/Errors/Details/600064/
> https://errors.yoctoproject.org/Errors/Details/600065/
> https://errors.yoctoproject.org/Errors/Details/600071/
>
>
I sent patches for meta-openembedded, but I am not sure how to build
meta-atmel:

bitbake libegl
ERROR: Layer atmel is not compatible with the core layer which only
supports these series: honister (layer is compatible with hardknott)

But the fix is probably the same as the ones I sent for the other
packages (which I copied from
meta-openembedded/meta-networking/recipes-protocols/net-snmp/net-snmp_5.9.1.bb).

Tony Battersby


> On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <tonyb@cybernetics.com> wrote:
>> On 8/6/21 12:44 PM, Tony Battersby wrote:
>>> On 8/6/21 1:33 AM, Richard Purdie wrote:
>>>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>>>>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>>>>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>>>>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>>>>>>> Currently, enabling Link Time Optimization via:
>>>>>>>>
>>>>>>>> require conf/distro/include/lto.inc
>>>>>>>> DISTRO_FEATURES_append = " lto"
>>>>>>>>
>>>>>>>> makes the build non-reproducible (as in
>>>>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
>>>>>>>> problems, but is not yet a complete solution.
>>>>>>>>
>>>>>>>> This patch set addresses the following bugs:
>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>>>>>>
>>>>>>>> openembedded-core patches
>>>>>>>> 1/4: lto.inc: disable LTO for grub
>>>>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>>>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>>>>>>
>>>>>>>> openembedded-devel patches
>>>>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>>>>>>
>>>>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>>>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>>>>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>>>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>>>>>>>> if there is a better/preferred way to do it.
>>>>>>>>
>>>>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>>>>>>> included a patch to fix that.  Other packages might have been broken as
>>>>>>>> well, although I haven't found any others.
>>>>>>>>
>>>>>>>> With all these patches, there are still many shared libraries in /lib
>>>>>>>> and /usr/lib that are still not reproducible with LTO enabled because
>>>>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>>>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>>>>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>>>>>>>> have a patch for that problem.  I will leave that to someone else.
>>>>>>>>
>>>>>>>> So this is what we need to achieve reproducible LTO builds:
>>>>>>>> *) This patch set
>>>>>>>> *) A fix for libtool
>>>>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>>>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>>>>>>> *) Fixes for individual packages, if needed
>>>>>>>> *) Once everything works, update the documentation again (I previously
>>>>>>>> requested a documentation change for this bug, which will eventually
>>>>>>>> show up at
>>>>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>>>>>>> This is certainly a few steps in the right direction with a good outline
>>>>>>> of what's next.
>>>>>>>
>>>>>>> I suspect that Richard has been busy with other things like overrides
>>>>>>> and the update to glibc-2.34.
>>>>>>>
>>>>>>> Do you have any suggestions or questions about what we should do
>>>>>>> with this patch series Richard?
>>>>>> Thanks for the reminder, I've been meaning to queue/test this which I will
>>>>>> do. The gcc version has been bumped but I ported the patch forward to it
>>>>>> and the patches are in master-next for testing. It is great to see this
>>>>>> support moving forward.
>>>>>>
>>>>>> Much as it pains me to admit having any libtool knowledge, I may be able to
>>>>>> help with that bit. There is an untested patch below which may address that
>>>>>> issue, or at least give a good hint on how to do it if it doesn't work. I
>>>>>> don't have an environment to test easily right now.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Richard
>>>>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>>>>> works as intended; libbz2.so.1.0.6 and all the other output files are
>>>>> now reproducible with LTO enabled.  I also verified that libtool is
>>>>> passing the -f*-prefix-map flags to the link command in log.do_compile
>>>>> as intended.
>>>>>
>>>>> I also verified that core-image-minimal builds successfully with the
>>>>> libtool patch.
>>>> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
>>>>
>>>> Unfortunately even with the ruby fix I queued, we still see reproduciblity
>>>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg
>>>> packages:
>>>>
>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>>>>
>>>> We'll need to resolve those before I can merge the flags change.
>>>>
>>>> Diffoscope output:
>>>>
>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>>>>
>>>> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>>
>>> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
>>> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h.
>>> There is already a patch
>>> poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
>>> that removes some of these items for the sake of reproducibiltiy.  But I
>>> do not know ruby, so I do not know the implications of doing the same
>>> thing for MJIT_DLDFLAGS.
>>>
>>> Tony Battersby
>>>
>>>
>>>
>> Richard,
>>
>> I noticed that you already fixed the problems in curl and gettext in
>> master-next.  Here is a patch for ruby if you haven't fixed that already.
>>
>> Tony Battersby
>>
>> From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
>> From: Tony Battersby <tonyb@cybernetics.com>
>> Date: Fri, 6 Aug 2021 16:47:14 -0400
>> Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS
>>
>> If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
>> longer reproducible.  Fix this.
>>
>> Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
>> ---
>>  ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
>>  meta/recipes-devtools/ruby/ruby_3.0.2.bb      |  1 +
>>  2 files changed, 43 insertions(+)
>>  create mode 100644 meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>>
>> diff --git a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>> new file mode 100644
>> index 0000000000..9387506c26
>> --- /dev/null
>> +++ b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>> @@ -0,0 +1,42 @@
>> +Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
>> +
>> +If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
>> +reproducible.  Fix this.
>> +
>> +Upstream-Status: Inapproppriate [oe-core specific]
>> +Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
>> +---
>> +--- a/tool/mjit_archflag.sh
>> ++++ b/tool/mjit_archflag.sh
>> +@@ -7,6 +7,20 @@ quote() {
>> +     echo
>> + }
>> +
>> ++quote_filtered() {
>> ++    printf "#${indent}define $1"
>> ++    while shift && [ "$#" -gt 0 ]; do
>> ++      case "$1" in
>> ++          -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
>> ++              ;;
>> ++          *)
>> ++              printf ' "%s"'$sep "$1"
>> ++              ;;
>> ++      esac
>> ++    done
>> ++    echo
>> ++}
>> ++
>> + archs=""
>> + arch_flag=""
>> +
>> +--- a/template/Makefile.in
>> ++++ b/template/Makefile.in
>> +@@ -666,7 +666,7 @@ mjit_config.h:
>> +       quote "MJIT_OPTFLAGS   " $(MJIT_OPTFLAGS); \
>> +       quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
>> +       quote "MJIT_LDSHARED   " ; \
>> +-      quote "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
>> ++      quote_filtered "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
>> +       quote "MJIT_LIBS       " $(LIBRUBYARG_SHARED); \
>> +       quote 'PRELOADENV       "@PRELOADENV@"'; \
>> +       indent=$${archs:+'  '}; \
>> diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>> index 0b9c342def..38e594a59e 100644
>> --- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>> +++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>> @@ -6,6 +6,7 @@ SRC_URI += " \
>>             file://remove_has_include_macros.patch \
>>             file://run-ptest \
>>             file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
>> +           file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
>>             "
>>
>>  SRC_URI[sha256sum] = "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
>> --
>> 2.25.1
>>

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Khem Raj]

On 8/9/21 7:39 AM, Tony Battersby wrote:
> On 8/6/21 10:19 PM, Khem Raj wrote:
>> I am seeing bunch of failures on meta-oe and meta-atmel
>>
>> https://errors.yoctoproject.org/Errors/Details/600064/
>> https://errors.yoctoproject.org/Errors/Details/600065/
>> https://errors.yoctoproject.org/Errors/Details/600071/
>>
>>
> I sent patches for meta-openembedded, but I am not sure how to build
> meta-atmel:
> 
> bitbake libegl
> ERROR: Layer atmel is not compatible with the core layer which only
> supports these series: honister (layer is compatible with hardknott)

you can use my branch which has fixes for override problem

https://github.com/YoeDistro/meta-atmel/tree/yoe/mut


> 
> But the fix is probably the same as the ones I sent for the other
> packages (which I copied from
> meta-openembedded/meta-networking/recipes-protocols/net-snmp/net-snmp_5.9.1.bb).
> 
> Tony Battersby
> 
> 
>> On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <tonyb@cybernetics.com> wrote:
>>> On 8/6/21 12:44 PM, Tony Battersby wrote:
>>>> On 8/6/21 1:33 AM, Richard Purdie wrote:
>>>>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
>>>>>> On 8/5/21 6:21 AM, Richard Purdie wrote:
>>>>>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
>>>>>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
>>>>>>>>> Currently, enabling Link Time Optimization via:
>>>>>>>>>
>>>>>>>>> require conf/distro/include/lto.inc
>>>>>>>>> DISTRO_FEATURES_append = " lto"
>>>>>>>>>
>>>>>>>>> makes the build non-reproducible (as in
>>>>>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
>>>>>>>>> problems, but is not yet a complete solution.
>>>>>>>>>
>>>>>>>>> This patch set addresses the following bugs:
>>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
>>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
>>>>>>>>>
>>>>>>>>> openembedded-core patches
>>>>>>>>> 1/4: lto.inc: disable LTO for grub
>>>>>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
>>>>>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
>>>>>>>>>
>>>>>>>>> openembedded-devel patches
>>>>>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
>>>>>>>>>
>>>>>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
>>>>>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
>>>>>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
>>>>>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
>>>>>>>>> if there is a better/preferred way to do it.
>>>>>>>>>
>>>>>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
>>>>>>>>> included a patch to fix that.  Other packages might have been broken as
>>>>>>>>> well, although I haven't found any others.
>>>>>>>>>
>>>>>>>>> With all these patches, there are still many shared libraries in /lib
>>>>>>>>> and /usr/lib that are still not reproducible with LTO enabled because
>>>>>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
>>>>>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
>>>>>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
>>>>>>>>> have a patch for that problem.  I will leave that to someone else.
>>>>>>>>>
>>>>>>>>> So this is what we need to achieve reproducible LTO builds:
>>>>>>>>> *) This patch set
>>>>>>>>> *) A fix for libtool
>>>>>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
>>>>>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
>>>>>>>>> *) Fixes for individual packages, if needed
>>>>>>>>> *) Once everything works, update the documentation again (I previously
>>>>>>>>> requested a documentation change for this bug, which will eventually
>>>>>>>>> show up at
>>>>>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
>>>>>>>> This is certainly a few steps in the right direction with a good outline
>>>>>>>> of what's next.
>>>>>>>>
>>>>>>>> I suspect that Richard has been busy with other things like overrides
>>>>>>>> and the update to glibc-2.34.
>>>>>>>>
>>>>>>>> Do you have any suggestions or questions about what we should do
>>>>>>>> with this patch series Richard?
>>>>>>> Thanks for the reminder, I've been meaning to queue/test this which I will
>>>>>>> do. The gcc version has been bumped but I ported the patch forward to it
>>>>>>> and the patches are in master-next for testing. It is great to see this
>>>>>>> support moving forward.
>>>>>>>
>>>>>>> Much as it pains me to admit having any libtool knowledge, I may be able to
>>>>>>> help with that bit. There is an untested patch below which may address that
>>>>>>> issue, or at least give a good hint on how to do it if it doesn't work. I
>>>>>>> don't have an environment to test easily right now.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Richard
>>>>>> I tested your libtool patch in master-next with the bzip2 recipe.  It
>>>>>> works as intended; libbz2.so.1.0.6 and all the other output files are
>>>>>> now reproducible with LTO enabled.  I also verified that libtool is
>>>>>> passing the -f*-prefix-map flags to the link command in log.do_compile
>>>>>> as intended.
>>>>>>
>>>>>> I also verified that core-image-minimal builds successfully with the
>>>>>> libtool patch.
>>>>> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
>>>>>
>>>>> Unfortunately even with the ruby fix I queued, we still see reproduciblity
>>>>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg
>>>>> packages:
>>>>>
>>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
>>>>>
>>>>> We'll need to resolve those before I can merge the flags change.
>>>>>
>>>>> Diffoscope output:
>>>>>
>>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
>>>>>
>>>>> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Richard
>>>>>
>>>> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
>>>> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h.
>>>> There is already a patch
>>>> poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
>>>> that removes some of these items for the sake of reproducibiltiy.  But I
>>>> do not know ruby, so I do not know the implications of doing the same
>>>> thing for MJIT_DLDFLAGS.
>>>>
>>>> Tony Battersby
>>>>
>>>>
>>>>
>>> Richard,
>>>
>>> I noticed that you already fixed the problems in curl and gettext in
>>> master-next.  Here is a patch for ruby if you haven't fixed that already.
>>>
>>> Tony Battersby
>>>
>>>  From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
>>> From: Tony Battersby <tonyb@cybernetics.com>
>>> Date: Fri, 6 Aug 2021 16:47:14 -0400
>>> Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS
>>>
>>> If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
>>> longer reproducible.  Fix this.
>>>
>>> Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
>>> ---
>>>   ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
>>>   meta/recipes-devtools/ruby/ruby_3.0.2.bb      |  1 +
>>>   2 files changed, 43 insertions(+)
>>>   create mode 100644 meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>>>
>>> diff --git a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>>> new file mode 100644
>>> index 0000000000..9387506c26
>>> --- /dev/null
>>> +++ b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
>>> @@ -0,0 +1,42 @@
>>> +Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
>>> +
>>> +If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
>>> +reproducible.  Fix this.
>>> +
>>> +Upstream-Status: Inapproppriate [oe-core specific]
>>> +Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
>>> +---
>>> +--- a/tool/mjit_archflag.sh
>>> ++++ b/tool/mjit_archflag.sh
>>> +@@ -7,6 +7,20 @@ quote() {
>>> +     echo
>>> + }
>>> +
>>> ++quote_filtered() {
>>> ++    printf "#${indent}define $1"
>>> ++    while shift && [ "$#" -gt 0 ]; do
>>> ++      case "$1" in
>>> ++          -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
>>> ++              ;;
>>> ++          *)
>>> ++              printf ' "%s"'$sep "$1"
>>> ++              ;;
>>> ++      esac
>>> ++    done
>>> ++    echo
>>> ++}
>>> ++
>>> + archs=""
>>> + arch_flag=""
>>> +
>>> +--- a/template/Makefile.in
>>> ++++ b/template/Makefile.in
>>> +@@ -666,7 +666,7 @@ mjit_config.h:
>>> +       quote "MJIT_OPTFLAGS   " $(MJIT_OPTFLAGS); \
>>> +       quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
>>> +       quote "MJIT_LDSHARED   " ; \
>>> +-      quote "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
>>> ++      quote_filtered "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
>>> +       quote "MJIT_LIBS       " $(LIBRUBYARG_SHARED); \
>>> +       quote 'PRELOADENV       "@PRELOADENV@"'; \
>>> +       indent=$${archs:+'  '}; \
>>> diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>>> index 0b9c342def..38e594a59e 100644
>>> --- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>>> +++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
>>> @@ -6,6 +6,7 @@ SRC_URI += " \
>>>              file://remove_has_include_macros.patch \
>>>              file://run-ptest \
>>>              file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
>>> +           file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
>>>              "
>>>
>>>   SRC_URI[sha256sum] = "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
>>> --
>>> 2.25.1
>>>
> 

Re: [OE-core] [master][PATCH 0/4] Reproducible LTO builds

[Khem Raj]

when using ubuntu 18.04 host tcsh fails now  see [1]

gcc: error: unrecognized command line option
‘-fmacro-prefix-map=TOPDIR/build/tmp/work/core2-32-yoe-linux/tcsh/6.22.04-r0=/usr/src/debug/tcsh/6.22.04-r0’
Makefile:450: recipe for target 'gethost' failed
make: *** [gethost] Error 1

[1] https://errors.yoctoproject.org/Errors/Details/600645/

On Mon, Aug 9, 2021 at 10:18 AM Khem Raj <raj.khem@gmail.com> wrote:
>
>
>
> On 8/9/21 7:39 AM, Tony Battersby wrote:
> > On 8/6/21 10:19 PM, Khem Raj wrote:
> >> I am seeing bunch of failures on meta-oe and meta-atmel
> >>
> >> https://errors.yoctoproject.org/Errors/Details/600064/
> >> https://errors.yoctoproject.org/Errors/Details/600065/
> >> https://errors.yoctoproject.org/Errors/Details/600071/
> >>
> >>
> > I sent patches for meta-openembedded, but I am not sure how to build
> > meta-atmel:
> >
> > bitbake libegl
> > ERROR: Layer atmel is not compatible with the core layer which only
> > supports these series: honister (layer is compatible with hardknott)
>
> you can use my branch which has fixes for override problem
>
> https://github.com/YoeDistro/meta-atmel/tree/yoe/mut
>
>
> >
> > But the fix is probably the same as the ones I sent for the other
> > packages (which I copied from
> > meta-openembedded/meta-networking/recipes-protocols/net-snmp/net-snmp_5.9.1.bb).
> >
> > Tony Battersby
> >
> >
> >> On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <tonyb@cybernetics.com> wrote:
> >>> On 8/6/21 12:44 PM, Tony Battersby wrote:
> >>>> On 8/6/21 1:33 AM, Richard Purdie wrote:
> >>>>> On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
> >>>>>> On 8/5/21 6:21 AM, Richard Purdie wrote:
> >>>>>>> On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
> >>>>>>>> On 2021-07-27 9:34 a.m., Tony Battersby wrote:
> >>>>>>>>> Currently, enabling Link Time Optimization via:
> >>>>>>>>>
> >>>>>>>>> require conf/distro/include/lto.inc
> >>>>>>>>> DISTRO_FEATURES_append = " lto"
> >>>>>>>>>
> >>>>>>>>> makes the build non-reproducible (as in
> >>>>>>>>> https://reproducible-builds.org/).  This patch set fixes some of the
> >>>>>>>>> problems, but is not yet a complete solution.
> >>>>>>>>>
> >>>>>>>>> This patch set addresses the following bugs:
> >>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
> >>>>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
> >>>>>>>>>
> >>>>>>>>> openembedded-core patches
> >>>>>>>>> 1/4: lto.inc: disable LTO for grub
> >>>>>>>>> 2/4: gcc: Backport patch to make LTO builds more reproducible
> >>>>>>>>> 3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
> >>>>>>>>>
> >>>>>>>>> openembedded-devel patches
> >>>>>>>>> 4/4: net-snmp: fix QA Issue after LDFLAGS change
> >>>>>>>>>
> >>>>>>>>> Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
> >>>>>>>>> TARGET_CFLAGS).  This could possibly be done differently, such as by
> >>>>>>>>> removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
> >>>>>>>>> TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those.  Let me know
> >>>>>>>>> if there is a better/preferred way to do it.
> >>>>>>>>>
> >>>>>>>>> The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
> >>>>>>>>> included a patch to fix that.  Other packages might have been broken as
> >>>>>>>>> well, although I haven't found any others.
> >>>>>>>>>
> >>>>>>>>> With all these patches, there are still many shared libraries in /lib
> >>>>>>>>> and /usr/lib that are still not reproducible with LTO enabled because
> >>>>>>>>> x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
> >>>>>>>>> patch 3 when linking shared libraries.  /usr/lib/libbz2.so.1.0.6 from
> >>>>>>>>> the libbz2 package from the bzip2 recipe is a good example.  I do not
> >>>>>>>>> have a patch for that problem.  I will leave that to someone else.
> >>>>>>>>>
> >>>>>>>>> So this is what we need to achieve reproducible LTO builds:
> >>>>>>>>> *) This patch set
> >>>>>>>>> *) A fix for libtool
> >>>>>>>>> *) Add LTO to reproducible builds testing if appropriate (e.g.
> >>>>>>>>> meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
> >>>>>>>>> *) Fixes for individual packages, if needed
> >>>>>>>>> *) Once everything works, update the documentation again (I previously
> >>>>>>>>> requested a documentation change for this bug, which will eventually
> >>>>>>>>> show up at
> >>>>>>>>> http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
> >>>>>>>> This is certainly a few steps in the right direction with a good outline
> >>>>>>>> of what's next.
> >>>>>>>>
> >>>>>>>> I suspect that Richard has been busy with other things like overrides
> >>>>>>>> and the update to glibc-2.34.
> >>>>>>>>
> >>>>>>>> Do you have any suggestions or questions about what we should do
> >>>>>>>> with this patch series Richard?
> >>>>>>> Thanks for the reminder, I've been meaning to queue/test this which I will
> >>>>>>> do. The gcc version has been bumped but I ported the patch forward to it
> >>>>>>> and the patches are in master-next for testing. It is great to see this
> >>>>>>> support moving forward.
> >>>>>>>
> >>>>>>> Much as it pains me to admit having any libtool knowledge, I may be able to
> >>>>>>> help with that bit. There is an untested patch below which may address that
> >>>>>>> issue, or at least give a good hint on how to do it if it doesn't work. I
> >>>>>>> don't have an environment to test easily right now.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Richard
> >>>>>> I tested your libtool patch in master-next with the bzip2 recipe.  It
> >>>>>> works as intended; libbz2.so.1.0.6 and all the other output files are
> >>>>>> now reproducible with LTO enabled.  I also verified that libtool is
> >>>>>> passing the -f*-prefix-map flags to the link command in log.do_compile
> >>>>>> as intended.
> >>>>>>
> >>>>>> I also verified that core-image-minimal builds successfully with the
> >>>>>> libtool patch.
> >>>>> Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
> >>>>>
> >>>>> Unfortunately even with the ruby fix I queued, we still see reproduciblity
> >>>>> issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg
> >>>>> packages:
> >>>>>
> >>>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
> >>>>>
> >>>>> We'll need to resolve those before I can merge the flags change.
> >>>>>
> >>>>> Diffoscope output:
> >>>>>
> >>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
> >>>>>
> >>>>> Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Richard
> >>>>>
> >>>> It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
> >>>> build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h.
> >>>> There is already a patch
> >>>> poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
> >>>> that removes some of these items for the sake of reproducibiltiy.  But I
> >>>> do not know ruby, so I do not know the implications of doing the same
> >>>> thing for MJIT_DLDFLAGS.
> >>>>
> >>>> Tony Battersby
> >>>>
> >>>>
> >>>>
> >>> Richard,
> >>>
> >>> I noticed that you already fixed the problems in curl and gettext in
> >>> master-next.  Here is a patch for ruby if you haven't fixed that already.
> >>>
> >>> Tony Battersby
> >>>
> >>>  From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
> >>> From: Tony Battersby <tonyb@cybernetics.com>
> >>> Date: Fri, 6 Aug 2021 16:47:14 -0400
> >>> Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS
> >>>
> >>> If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
> >>> longer reproducible.  Fix this.
> >>>
> >>> Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
> >>> ---
> >>>   ...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
> >>>   meta/recipes-devtools/ruby/ruby_3.0.2.bb      |  1 +
> >>>   2 files changed, 43 insertions(+)
> >>>   create mode 100644 meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
> >>>
> >>> diff --git a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
> >>> new file mode 100644
> >>> index 0000000000..9387506c26
> >>> --- /dev/null
> >>> +++ b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
> >>> @@ -0,0 +1,42 @@
> >>> +Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
> >>> +
> >>> +If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
> >>> +reproducible.  Fix this.
> >>> +
> >>> +Upstream-Status: Inapproppriate [oe-core specific]
> >>> +Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
> >>> +---
> >>> +--- a/tool/mjit_archflag.sh
> >>> ++++ b/tool/mjit_archflag.sh
> >>> +@@ -7,6 +7,20 @@ quote() {
> >>> +     echo
> >>> + }
> >>> +
> >>> ++quote_filtered() {
> >>> ++    printf "#${indent}define $1"
> >>> ++    while shift && [ "$#" -gt 0 ]; do
> >>> ++      case "$1" in
> >>> ++          -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
> >>> ++              ;;
> >>> ++          *)
> >>> ++              printf ' "%s"'$sep "$1"
> >>> ++              ;;
> >>> ++      esac
> >>> ++    done
> >>> ++    echo
> >>> ++}
> >>> ++
> >>> + archs=""
> >>> + arch_flag=""
> >>> +
> >>> +--- a/template/Makefile.in
> >>> ++++ b/template/Makefile.in
> >>> +@@ -666,7 +666,7 @@ mjit_config.h:
> >>> +       quote "MJIT_OPTFLAGS   " $(MJIT_OPTFLAGS); \
> >>> +       quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
> >>> +       quote "MJIT_LDSHARED   " ; \
> >>> +-      quote "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
> >>> ++      quote_filtered "MJIT_DLDFLAGS    MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
> >>> +       quote "MJIT_LIBS       " $(LIBRUBYARG_SHARED); \
> >>> +       quote 'PRELOADENV       "@PRELOADENV@"'; \
> >>> +       indent=$${archs:+'  '}; \
> >>> diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> >>> index 0b9c342def..38e594a59e 100644
> >>> --- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> >>> +++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
> >>> @@ -6,6 +6,7 @@ SRC_URI += " \
> >>>              file://remove_has_include_macros.patch \
> >>>              file://run-ptest \
> >>>              file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
> >>> +           file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
> >>>              "
> >>>
> >>>   SRC_URI[sha256sum] = "5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
> >>> --
> >>> 2.25.1
> >>>
> >

[meta-oe][master][PATCH] tcsh: fix compile error after LDFLAGS change

[Tony Battersby]

Adding -f*-prefix-map to LDFLAGS caused the following issue when
compiling tcsh on Ubuntu 18.04:

gcc: error: unrecognized command line option ‘-fmacro-prefix-map=...’

Fix by using BUILD_LDFLAGS instead of LDFLAGS for gethost.

[YOCTO #14481]

Signed-off-by: Tony Battersby <tonyb@cybernetics.com>
---

NOTE: tested only on Ubuntu 20.04.

 meta-oe/recipes-shells/tcsh/tcsh_6.22.04.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-oe/recipes-shells/tcsh/tcsh_6.22.04.bb b/meta-oe/recipes-shells/tcsh/tcsh_6.22.04.bb
index ac6c6db81..c4da5cd83 100644
--- a/meta-oe/recipes-shells/tcsh/tcsh_6.22.04.bb
+++ b/meta-oe/recipes-shells/tcsh/tcsh_6.22.04.bb
@@ -20,7 +20,7 @@ EXTRA_OEMAKE += "CC_FOR_GETHOST='${BUILD_CC}'"
 inherit autotools
 
 do_compile:prepend() {
-    oe_runmake CC_FOR_GETHOST='${BUILD_CC}' CFLAGS='${BUILD_CFLAGS}' gethost
+    oe_runmake CC_FOR_GETHOST='${BUILD_CC}' CFLAGS='${BUILD_CFLAGS}' LDFLAGS='${BUILD_LDFLAGS}' gethost
 }
 
 do_install:append () {
-- 
2.25.1