[XEN PATCH 0/3] Improvements for reproducible builds

[XEN PATCH 0/3] Improvements for reproducible builds

[Maximilian Engelhardt]

While working on a reproducible build of xen in Debian I came up with
the following three patches which are needed to build xen reproducibly
in Debian. Reproducible builds are useful to verify the binary has
actually been built from the corresponding source.

The first patch is an extension of [1] which only omits the timestamp if
SOURCE_DATE_EPOCH is defined. Patch two fixes an ordering issue in the
generated documentation and the last patch uses the date from
SOURCE_DATE_EPOCH if available in the documentation.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html

Maximilian Engelhardt (3):
  xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
  docs: use predictable ordering in generated documentation
  docs: set date to SOURCE_DATE_EPOCH if available

 docs/Makefile         | 8 +++++++-
 docs/xen-headers      | 2 +-
 xen/arch/x86/Makefile | 6 ++++++
 3 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.20.1

[XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined

[Maximilian Engelhardt]

By default a timestamp gets added to the xen efi binary. Unfortunately
ld doesn't seem to provide a way to set a custom date, like from
SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option
--no-insert-timestamp) if SOURCE_DATE_EPOCH is defined. This makes
reproducible builds possible.

This is an alternative to the patch suggested in [1]. This patch only
omits the timestamp when SOURCE_DATE_EPOCH is defined.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 xen/arch/x86/Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 8f2180485b..863aed043f 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -184,6 +184,12 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
 EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
 EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
+# It seems ld unfortunately can't set a custom timestamp, so add a zero value
+# for the timestamp (option --no-insert-timestamp) if SOURCE_DATE_EPOCH is
+# defined to make reproducible builds possible.
+ifdef SOURCE_DATE_EPOCH
+EFI_LDFLAGS += --no-insert-timestamp
+endif
 
 $(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
 $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
-- 
2.20.1

Re: [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined

[Jan Beulich]

On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> By default a timestamp gets added to the xen efi binary. Unfortunately
> ld doesn't seem to provide a way to set a custom date, like from
> SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option
> --no-insert-timestamp) if SOURCE_DATE_EPOCH is defined. This makes
> reproducible builds possible.
> 
> This is an alternative to the patch suggested in [1]. This patch only
> omits the timestamp when SOURCE_DATE_EPOCH is defined.
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html
> 
> Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>

Acked-by: Jan Beulich <jbeulich@suse.com>

[XEN PATCH 2/3] docs: use predictable ordering in generated documentation

[Maximilian Engelhardt]

When the seq number is equal, sort by the title to get predictable
output ordering. This is useful for reproducible builds.

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 docs/xen-headers | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/xen-headers b/docs/xen-headers
index 54155632c4..8c434d77e2 100755
--- a/docs/xen-headers
+++ b/docs/xen-headers
@@ -331,7 +331,7 @@ sub output_index () {
 <h2>Starting points</h2>
 <ul>
 END
-    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} } @incontents) {
+    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} or $a->{Title} cmp $b->{Title} } @incontents) {
         $o .= "<li><a href=\"$ic->{Href}\">$ic->{Title}</a></li>\n";
     }
     $o .= "</ul>\n";
-- 
2.20.1

Re: [XEN PATCH 2/3] docs: use predictable ordering in generated documentation

[Andrew Cooper]

On 18/12/2020 20:42, Maximilian Engelhardt wrote:
> When the seq number is equal, sort by the title to get predictable
> output ordering. This is useful for reproducible builds.
>
> Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

[XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available

[Maximilian Engelhardt]

Use the solution described in [1] to replace the call to the 'date'
command with a version that uses SOURCE_DATE_EPOCH if available. This
is needed for reproducible builds.

[1] https://reproducible-builds.org/docs/source-date-epoch/

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 docs/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/docs/Makefile b/docs/Makefile
index 8de1efb6f5..ac6792ff7c 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
 -include $(XEN_ROOT)/config/Docs.mk
 
 VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion)
-DATE		:= $(shell date +%Y-%m-%d)
+
+DATE_FMT	:= +%Y-%m-%d
+ifdef SOURCE_DATE_EPOCH
+DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")
+else
+DATE		:= $(shell date "$(DATE_FMT)")
+endif
 
 DOC_ARCHES      := arm x86_32 x86_64
 MAN_SECTIONS    := 1 5 7 8
-- 
2.20.1

Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available

[Jan Beulich]

On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
>  -include $(XEN_ROOT)/config/Docs.mk
>  
>  VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion)
> -DATE		:= $(shell date +%Y-%m-%d)
> +
> +DATE_FMT	:= +%Y-%m-%d
> +ifdef SOURCE_DATE_EPOCH
> +DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")

Looking at the doc for a (deliberately) old "date", I can't find
any mention of the -d "@..." syntax. I take it the command would
fail on that system. It would then go on to try the -r variant,
which has entirely different meaning on GNU (Linux) systems.

docs/ being subject to configuring, why don't you determine the
capabilities of "date" there and invoke just the one command
that was found suitable for the system?

Jan

Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available

[Maximilian Engelhardt]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

On Montag, 21. Dezember 2020 10:01:14 CET Jan Beulich wrote:
> On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> > --- a/docs/Makefile
> > +++ b/docs/Makefile
> > @@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
> > 
> >  -include $(XEN_ROOT)/config/Docs.mk
> >  
> >  VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory
> >  xenversion)> 
> > -DATE		:= $(shell date +%Y-%m-%d)
> > +
> > +DATE_FMT	:= +%Y-%m-%d
> > +ifdef SOURCE_DATE_EPOCH
> > +DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)"
> > 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)"
> > 2>/dev/null || date -u "$(DATE_FMT)")
> Looking at the doc for a (deliberately) old "date", I can't find
> any mention of the -d "@..." syntax. I take it the command would
> fail on that system. It would then go on to try the -r variant,
> which has entirely different meaning on GNU (Linux) systems.
> 
> docs/ being subject to configuring, why don't you determine the
> capabilities of "date" there and invoke just the one command
> that was found suitable for the system?
> 
> Jan

Hi Jan,

I did some research. The -d "@..." syntax was introduced about 2005. Testing a 
live CD from 2006 (KNOPPIX_V5.0.1CD-2006-06-01-EN.iso) it was supported there. 
The documentation about this syntax has only been added in 2011 to the date 
command. I'm wondering if anybody running such an old system wants to use 
SOURCE_DATE_EPOCH.

However, I came up with a patch to determine which suitable date version is 
available and only call that, as you suggested. I will post the new patch 
soon.

Maxi

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]