[PATCH v1 0/2] Improve reproducible builds

[PATCH v1 0/2] Improve reproducible builds

[Frédéric Pierret (fepitre)]

This two fixes improve reproducibility of resulting Xen binaries

Frédéric Pierret (fepitre) (2):
  No insert of the build timestamp into the x86 xen efi binary
  xen/common/makefile: remove gzip timestamp

 xen/arch/x86/Makefile | 1 +
 xen/common/Makefile   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

-- 
2.26.2

[PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Frédéric Pierret (fepitre)]

This is for improving reproducible builds.

Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
---
 xen/arch/x86/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index b388861679..f5a529afd5 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
 EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
 EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
+EFI_LDFLAGS += --no-insert-timestamp
 
 # Check if the compiler supports the MS ABI.
 export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
-- 
2.26.2

[PATCH v1 2/2] xen/common/makefile: remove gzip timestamp

[Frédéric Pierret (fepitre)]

This is for improving reproducible builds.

Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
---
 xen/common/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/common/Makefile b/xen/common/Makefile
index 06881d023c..32cd650ba8 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -77,7 +77,7 @@ obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
 
 CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(XEN_ROOT)/xen/)$(KCONFIG_CONFIG)
 config.gz: $(CONF_FILE)
-	gzip -c $< >$@
+	gzip -n -c $< >$@
 
 config_data.o: config.gz
 
-- 
2.26.2

Re: [PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Jan Beulich]

On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:

> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
>  EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
>  EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
>  EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
> +EFI_LDFLAGS += --no-insert-timestamp

Generally I prefer binaries to carry timestamps, when they are
intended to do so (i.e. when they have a respective field). So
I think if no timestamp is wanted, it should be as an option
(not sure about the default).

This said, I didn't think time stamps got meaningfully in the
way of reproducible builds - ignoring the minor differences
cause by them, especially when they sit at well known offsets
in the binaries, shouldn't be a big deal.

Jan

Re: [PATCH v1 2/2] xen/common/makefile: remove gzip timestamp

[Jan Beulich]

On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:
> This is for improving reproducible builds.
> 
> Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>

Acked-by: Jan Beulich <jbeulich@suse.com>

Albeit I'd like to ask for the title to actually mention whose
gzip time stamp it is that gets squashed. Perhaps "xen: don't
have timestamp inserted in config.gz"?

Jan

Re: [PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote:
> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:
> 
> > --- a/xen/arch/x86/Makefile
> > +++ b/xen/arch/x86/Makefile
> > @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
> >  EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
> >  EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
> >  EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
> > +EFI_LDFLAGS += --no-insert-timestamp
> 
> Generally I prefer binaries to carry timestamps, when they are
> intended to do so (i.e. when they have a respective field). So
> I think if no timestamp is wanted, it should be as an option
> (not sure about the default).

What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if
present? Of course if there is an option to set explicit timestamp
value.

[1] https://reproducible-builds.org/docs/source-date-epoch/

> This said, I didn't think time stamps got meaningfully in the
> way of reproducible builds - ignoring the minor differences
> cause by them, especially when they sit at well known offsets
> in the binaries, shouldn't be a big deal.

It is a big deal. There is a huge difference between running sha256sum
(or your other favorite hash) on two build artifacts, and using a
specialized tool/script to compare each file separately. Note the
xen.efi may be buried very deep in the thing you compare, for example
inside deb/rpm and then ISO image (installation image), at which point
it's far from "they sit at well known offsets".

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Jan Beulich]

On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote:
> On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote:
>> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:
>>
>>> --- a/xen/arch/x86/Makefile
>>> +++ b/xen/arch/x86/Makefile
>>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
>>>  EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
>>>  EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
>>>  EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
>>> +EFI_LDFLAGS += --no-insert-timestamp
>>
>> Generally I prefer binaries to carry timestamps, when they are
>> intended to do so (i.e. when they have a respective field). So
>> I think if no timestamp is wanted, it should be as an option
>> (not sure about the default).
> 
> What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if
> present? Of course if there is an option to set explicit timestamp
> value.
> 
> [1] https://reproducible-builds.org/docs/source-date-epoch/

Why not.

>> This said, I didn't think time stamps got meaningfully in the
>> way of reproducible builds - ignoring the minor differences
>> cause by them, especially when they sit at well known offsets
>> in the binaries, shouldn't be a big deal.
> 
> It is a big deal. There is a huge difference between running sha256sum
> (or your other favorite hash) on two build artifacts, and using a
> specialized tool/script to compare each file separately. Note the
> xen.efi may be buried very deep in the thing you compare, for example
> inside deb/rpm and then ISO image (installation image), at which point
> it's far from "they sit at well known offsets".

If you care about checking images / blobs where binaries with time
stamps are merely constituent parts, why not strip the time stamps
at the time of creation of those images / blobs (or as a minor
intermediate step, in case you want to e.g. record the hashes for
later reference)?

Jan

Re: [PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Andrew Cooper]

On 30/10/2020 12:48, Jan Beulich wrote:
> On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote:
>> On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote:
>>> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:
>>>
>>>> --- a/xen/arch/x86/Makefile
>>>> +++ b/xen/arch/x86/Makefile
>>>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
>>>>  EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
>>>>  EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
>>>>  EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
>>>> +EFI_LDFLAGS += --no-insert-timestamp
>>> Generally I prefer binaries to carry timestamps, when they are
>>> intended to do so (i.e. when they have a respective field). So
>>> I think if no timestamp is wanted, it should be as an option
>>> (not sure about the default).
>> What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if
>> present? Of course if there is an option to set explicit timestamp
>> value.
>>
>> [1] https://reproducible-builds.org/docs/source-date-epoch/
> Why not.

SOURCE_DATE_EPOCH is the right way to fix this.

It probably wants to default to something sane in the root Makefile, so
it covers tools as well.

>>> This said, I didn't think time stamps got meaningfully in the
>>> way of reproducible builds - ignoring the minor differences
>>> cause by them, especially when they sit at well known offsets
>>> in the binaries, shouldn't be a big deal.
>> It is a big deal. There is a huge difference between running sha256sum
>> (or your other favorite hash) on two build artifacts, and using a
>> specialized tool/script to compare each file separately. Note the
>> xen.efi may be buried very deep in the thing you compare, for example
>> inside deb/rpm and then ISO image (installation image), at which point
>> it's far from "they sit at well known offsets".
> If you care about checking images / blobs where binaries with time
> stamps are merely constituent parts, why not strip the time stamps
> at the time of creation of those images / blobs (or as a minor
> intermediate step, in case you want to e.g. record the hashes for
> later reference)?

Because that is a disaster to maintain.  A critical part of reproducible
builds is not needing custom comparison logic for every binary artefact.

~Andrew

Re: [PATCH v1 1/2] No insert of the build timestamp into the x86 xen efi binary

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

On Fri, Oct 30, 2020 at 01:30:08PM +0000, Andrew Cooper wrote:
> On 30/10/2020 12:48, Jan Beulich wrote:
> > On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote:
> >> On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote:
> >>> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote:
> >>>
> >>>> --- a/xen/arch/x86/Makefile
> >>>> +++ b/xen/arch/x86/Makefile
> >>>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
> >>>>  EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
> >>>>  EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
> >>>>  EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
> >>>> +EFI_LDFLAGS += --no-insert-timestamp
> >>> Generally I prefer binaries to carry timestamps, when they are
> >>> intended to do so (i.e. when they have a respective field). So
> >>> I think if no timestamp is wanted, it should be as an option
> >>> (not sure about the default).
> >> What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if
> >> present? Of course if there is an option to set explicit timestamp
> >> value.
> >>
> >> [1] https://reproducible-builds.org/docs/source-date-epoch/
> > Why not.
> 
> SOURCE_DATE_EPOCH is the right way to fix this.

Hmm, reading 'ld' man page, I don't see an option to set explicit value,
on --insert-timestamp / --no-insert-timestamp.

> It probably wants to default to something sane in the root Makefile, so
> it covers tools as well.

In practice, the package build system (deb, rpm, etc) do set it based on
last package changelog entry, so _for package build case_ it isn't
needed. But probably nice addition.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]