[Buildroot] [PATCH RFC] package/binutils: build deterministic archives by default

[Buildroot] [PATCH RFC] package/binutils: build deterministic archives by default

[Romain Naour]

The option --enable-deterministic-archives that build deterministic
archives is supported by binutils since binutils 2.23 [1][2].

See "Static libraries" chapter [3].
https://reproducible-builds.org/docs/archives/

Enable this option for target and host variant.

It was added to Yocto Morty (2.2) in 2016 [4].

[1] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=2e30cb575a1d2b1eb8f776356ed59b0722a943d1
[2] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=9cb80f72d8b8b2f59562692d9385cd5a533e9b3f
[3] https://reproducible-builds.org/docs/archives
[4] https://github.com/openembedded/openembedded-core/commit/df0d525c02780b5a0bd7a177a249c55f41797476

Signed-off-by: Romain Naour <romain.naour@gmail.com>
---
 package/binutils/binutils.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/binutils/binutils.mk b/package/binutils/binutils.mk
index 8ec76134bc..c5c685ac85 100644
--- a/package/binutils/binutils.mk
+++ b/package/binutils/binutils.mk
@@ -57,6 +57,7 @@ BINUTILS_CONF_OPTS = \
 	--target=$(GNU_TARGET_NAME) \
 	--enable-install-libiberty \
 	--enable-build-warnings=no \
+	--enable-deterministic-archives \
 	$(BINUTILS_DISABLE_GDB_CONF_OPTS) \
 	$(BINUTILS_EXTRA_CONFIG_OPTIONS)
 
@@ -93,6 +94,7 @@ HOST_BINUTILS_CONF_OPTS = \
 	--disable-shared \
 	--enable-static \
 	--with-sysroot=$(STAGING_DIR) \
+	--enable-deterministic-archives \
 	--enable-poison-system-directories \
 	--without-debuginfod \
 	$(BINUTILS_DISABLE_GDB_CONF_OPTS) \
-- 
2.31.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH RFC] package/binutils: build deterministic archives by default

[Arnout Vandecappelle]

On 30/09/2021 00:00, Romain Naour wrote:
> The option --enable-deterministic-archives that build deterministic
> archives is supported by binutils since binutils 2.23 [1][2].

  Should we make this dependent on BR2_REPRODUCIBLE? AFAICS it sets the date 
fields in static libraries to zero.

  Is this going to fix a reproducibility issue detected by the autobuilders?

  Regards,
  Arnout

> 
> See "Static libraries" chapter [3].
> https://reproducible-builds.org/docs/archives/
> 
> Enable this option for target and host variant.
> 
> It was added to Yocto Morty (2.2) in 2016 [4].
> 
> [1] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=2e30cb575a1d2b1eb8f776356ed59b0722a943d1
> [2] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=9cb80f72d8b8b2f59562692d9385cd5a533e9b3f
> [3] https://reproducible-builds.org/docs/archives
> [4] https://github.com/openembedded/openembedded-core/commit/df0d525c02780b5a0bd7a177a249c55f41797476
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> ---
>   package/binutils/binutils.mk | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/package/binutils/binutils.mk b/package/binutils/binutils.mk
> index 8ec76134bc..c5c685ac85 100644
> --- a/package/binutils/binutils.mk
> +++ b/package/binutils/binutils.mk
> @@ -57,6 +57,7 @@ BINUTILS_CONF_OPTS = \
>   	--target=$(GNU_TARGET_NAME) \
>   	--enable-install-libiberty \
>   	--enable-build-warnings=no \
> +	--enable-deterministic-archives \
>   	$(BINUTILS_DISABLE_GDB_CONF_OPTS) \
>   	$(BINUTILS_EXTRA_CONFIG_OPTIONS)
>   
> @@ -93,6 +94,7 @@ HOST_BINUTILS_CONF_OPTS = \
>   	--disable-shared \
>   	--enable-static \
>   	--with-sysroot=$(STAGING_DIR) \
> +	--enable-deterministic-archives \
>   	--enable-poison-system-directories \
>   	--without-debuginfod \
>   	$(BINUTILS_DISABLE_GDB_CONF_OPTS) \
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH RFC] package/binutils: build deterministic archives by default

[Romain Naour]

Hello Arnout,

Le 03/10/2021 à 12:14, Arnout Vandecappelle a écrit :
> 
> 
> On 30/09/2021 00:00, Romain Naour wrote:
>> The option --enable-deterministic-archives that build deterministic
>> archives is supported by binutils since binutils 2.23 [1][2].
> 
>  Should we make this dependent on BR2_REPRODUCIBLE? AFAICS it sets the date
> fields in static libraries to zero.

I'm not sure since it impact host-binutils, it change the default behavior of
the archiver (ar).

Instead of this patch, we could provide -D option to ar when BR2_REPRODUCIBLE is
set.

> 
>  Is this going to fix a reproducibility issue detected by the autobuilders?

No, I noticed this option while looking at binutils git tree.

Best regards,
Romain


> 
>  Regards,
>  Arnout
> 
>>
>> See "Static libraries" chapter [3].
>> https://reproducible-builds.org/docs/archives/
>>
>> Enable this option for target and host variant.
>>
>> It was added to Yocto Morty (2.2) in 2016 [4].
>>
>> [1]
>> https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=2e30cb575a1d2b1eb8f776356ed59b0722a943d1
>>
>> [2]
>> https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=9cb80f72d8b8b2f59562692d9385cd5a533e9b3f
>>
>> [3] https://reproducible-builds.org/docs/archives
>> [4]
>> https://github.com/openembedded/openembedded-core/commit/df0d525c02780b5a0bd7a177a249c55f41797476
>>
>>
>> Signed-off-by: Romain Naour <romain.naour@gmail.com>
>> ---
>>   package/binutils/binutils.mk | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/package/binutils/binutils.mk b/package/binutils/binutils.mk
>> index 8ec76134bc..c5c685ac85 100644
>> --- a/package/binutils/binutils.mk
>> +++ b/package/binutils/binutils.mk
>> @@ -57,6 +57,7 @@ BINUTILS_CONF_OPTS = \
>>       --target=$(GNU_TARGET_NAME) \
>>       --enable-install-libiberty \
>>       --enable-build-warnings=no \
>> +    --enable-deterministic-archives \
>>       $(BINUTILS_DISABLE_GDB_CONF_OPTS) \
>>       $(BINUTILS_EXTRA_CONFIG_OPTIONS)
>>   @@ -93,6 +94,7 @@ HOST_BINUTILS_CONF_OPTS = \
>>       --disable-shared \
>>       --enable-static \
>>       --with-sysroot=$(STAGING_DIR) \
>> +    --enable-deterministic-archives \
>>       --enable-poison-system-directories \
>>       --without-debuginfod \
>>       $(BINUTILS_DISABLE_GDB_CONF_OPTS) \
>>

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH RFC] package/binutils: build deterministic archives by default

[Thomas Petazzoni]

Hello Romain,

On Thu, 30 Sep 2021 00:00:27 +0200
Romain Naour <romain.naour@gmail.com> wrote:

> The option --enable-deterministic-archives that build deterministic
> archives is supported by binutils since binutils 2.23 [1][2].
> 
> See "Static libraries" chapter [3].
> https://reproducible-builds.org/docs/archives/
> 
> Enable this option for target and host variant.
> 
> It was added to Yocto Morty (2.2) in 2016 [4].
> 
> [1] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=2e30cb575a1d2b1eb8f776356ed59b0722a943d1
> [2] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=9cb80f72d8b8b2f59562692d9385cd5a533e9b3f
> [3] https://reproducible-builds.org/docs/archives
> [4] https://github.com/openembedded/openembedded-core/commit/df0d525c02780b5a0bd7a177a249c55f41797476
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>

The drawback of this approach is that it doesn't address the problem
for external toolchains, which have not necessarily been built with
--enable-deterministic-archives.

We could either pass -D in TARGET_AR (but of course that is subject to
packages obeying to that), or we could extend the toolchain wrapper to
also cover the ar tool ?

For this to be worth the effort, it would be useful to get some actual
case where this is causing a problem. Indeed, in Buildroot, we don't
really care about the reproducibility of .a archives themselves. So
unless their lack of reproducibility trickles down to the
statically-linked executables that result from using those .a, I'm not
sure this is really worth it.

I'll mark the patch as Changes Requested, but feel free to disagree and
put it back in the New state.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot