All of lore.kernel.org
 help / color / mirror / Atom feed
* [XEN PATCH 0/3] Improvements for reproducible builds
@ 2020-12-18 20:42 Maximilian Engelhardt
  2020-12-18 20:42 ` [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined Maximilian Engelhardt
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Maximilian Engelhardt @ 2020-12-18 20:42 UTC (permalink / raw)
  To: xen-devel
  Cc: Maximilian Engelhardt, Jan Beulich, Andrew Cooper,
	Roger Pau Monné,
	Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
	Stefano Stabellini

While working on a reproducible build of xen in Debian I came up with
the following three patches which are needed to build xen reproducibly
in Debian. Reproducible builds are useful to verify the binary has
actually been built from the corresponding source.

The first patch is an extension of [1] which only omits the timestamp if
SOURCE_DATE_EPOCH is defined. Patch two fixes an ordering issue in the
generated documentation and the last patch uses the date from
SOURCE_DATE_EPOCH if available in the documentation.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html

Maximilian Engelhardt (3):
  xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
  docs: use predictable ordering in generated documentation
  docs: set date to SOURCE_DATE_EPOCH if available

 docs/Makefile         | 8 +++++++-
 docs/xen-headers      | 2 +-
 xen/arch/x86/Makefile | 6 ++++++
 3 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.20.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
  2020-12-18 20:42 [XEN PATCH 0/3] Improvements for reproducible builds Maximilian Engelhardt
@ 2020-12-18 20:42 ` Maximilian Engelhardt
  2020-12-21  8:50   ` Jan Beulich
  2020-12-18 20:42 ` [XEN PATCH 2/3] docs: use predictable ordering in generated documentation Maximilian Engelhardt
  2020-12-18 20:42 ` [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available Maximilian Engelhardt
  2 siblings, 1 reply; 8+ messages in thread
From: Maximilian Engelhardt @ 2020-12-18 20:42 UTC (permalink / raw)
  To: xen-devel
  Cc: Maximilian Engelhardt, Jan Beulich, Andrew Cooper,
	Roger Pau Monné,
	Wei Liu

By default a timestamp gets added to the xen efi binary. Unfortunately
ld doesn't seem to provide a way to set a custom date, like from
SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option
--no-insert-timestamp) if SOURCE_DATE_EPOCH is defined. This makes
reproducible builds possible.

This is an alternative to the patch suggested in [1]. This patch only
omits the timestamp when SOURCE_DATE_EPOCH is defined.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 xen/arch/x86/Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 8f2180485b..863aed043f 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -184,6 +184,12 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
 EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION)
 EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
+# It seems ld unfortunately can't set a custom timestamp, so add a zero value
+# for the timestamp (option --no-insert-timestamp) if SOURCE_DATE_EPOCH is
+# defined to make reproducible builds possible.
+ifdef SOURCE_DATE_EPOCH
+EFI_LDFLAGS += --no-insert-timestamp
+endif
 
 $(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
 $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
-- 
2.20.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [XEN PATCH 2/3] docs: use predictable ordering in generated documentation
  2020-12-18 20:42 [XEN PATCH 0/3] Improvements for reproducible builds Maximilian Engelhardt
  2020-12-18 20:42 ` [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined Maximilian Engelhardt
@ 2020-12-18 20:42 ` Maximilian Engelhardt
  2020-12-22 10:38   ` Andrew Cooper
  2020-12-18 20:42 ` [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available Maximilian Engelhardt
  2 siblings, 1 reply; 8+ messages in thread
From: Maximilian Engelhardt @ 2020-12-18 20:42 UTC (permalink / raw)
  To: xen-devel
  Cc: Maximilian Engelhardt, Andrew Cooper, George Dunlap, Ian Jackson,
	Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu

When the seq number is equal, sort by the title to get predictable
output ordering. This is useful for reproducible builds.

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 docs/xen-headers | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/xen-headers b/docs/xen-headers
index 54155632c4..8c434d77e2 100755
--- a/docs/xen-headers
+++ b/docs/xen-headers
@@ -331,7 +331,7 @@ sub output_index () {
 <h2>Starting points</h2>
 <ul>
 END
-    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} } @incontents) {
+    foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} or $a->{Title} cmp $b->{Title} } @incontents) {
         $o .= "<li><a href=\"$ic->{Href}\">$ic->{Title}</a></li>\n";
     }
     $o .= "</ul>\n";
-- 
2.20.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available
  2020-12-18 20:42 [XEN PATCH 0/3] Improvements for reproducible builds Maximilian Engelhardt
  2020-12-18 20:42 ` [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined Maximilian Engelhardt
  2020-12-18 20:42 ` [XEN PATCH 2/3] docs: use predictable ordering in generated documentation Maximilian Engelhardt
@ 2020-12-18 20:42 ` Maximilian Engelhardt
  2020-12-21  9:01   ` Jan Beulich
  2 siblings, 1 reply; 8+ messages in thread
From: Maximilian Engelhardt @ 2020-12-18 20:42 UTC (permalink / raw)
  To: xen-devel; +Cc: Maximilian Engelhardt, Ian Jackson, Wei Liu

Use the solution described in [1] to replace the call to the 'date'
command with a version that uses SOURCE_DATE_EPOCH if available. This
is needed for reproducible builds.

[1] https://reproducible-builds.org/docs/source-date-epoch/

Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
---
 docs/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/docs/Makefile b/docs/Makefile
index 8de1efb6f5..ac6792ff7c 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
 -include $(XEN_ROOT)/config/Docs.mk
 
 VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion)
-DATE		:= $(shell date +%Y-%m-%d)
+
+DATE_FMT	:= +%Y-%m-%d
+ifdef SOURCE_DATE_EPOCH
+DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")
+else
+DATE		:= $(shell date "$(DATE_FMT)")
+endif
 
 DOC_ARCHES      := arm x86_32 x86_64
 MAN_SECTIONS    := 1 5 7 8
-- 
2.20.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
  2020-12-18 20:42 ` [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined Maximilian Engelhardt
@ 2020-12-21  8:50   ` Jan Beulich
  0 siblings, 0 replies; 8+ messages in thread
From: Jan Beulich @ 2020-12-21  8:50 UTC (permalink / raw)
  To: Maximilian Engelhardt
  Cc: Andrew Cooper, Roger Pau Monné, Wei Liu, xen-devel

On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> By default a timestamp gets added to the xen efi binary. Unfortunately
> ld doesn't seem to provide a way to set a custom date, like from
> SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option
> --no-insert-timestamp) if SOURCE_DATE_EPOCH is defined. This makes
> reproducible builds possible.
> 
> This is an alternative to the patch suggested in [1]. This patch only
> omits the timestamp when SOURCE_DATE_EPOCH is defined.
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html
> 
> Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>

Acked-by: Jan Beulich <jbeulich@suse.com>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available
  2020-12-18 20:42 ` [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available Maximilian Engelhardt
@ 2020-12-21  9:01   ` Jan Beulich
  2020-12-23 16:10     ` Maximilian Engelhardt
  0 siblings, 1 reply; 8+ messages in thread
From: Jan Beulich @ 2020-12-21  9:01 UTC (permalink / raw)
  To: Maximilian Engelhardt; +Cc: Ian Jackson, Wei Liu, xen-devel

On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
>  -include $(XEN_ROOT)/config/Docs.mk
>  
>  VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion)
> -DATE		:= $(shell date +%Y-%m-%d)
> +
> +DATE_FMT	:= +%Y-%m-%d
> +ifdef SOURCE_DATE_EPOCH
> +DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")

Looking at the doc for a (deliberately) old "date", I can't find
any mention of the -d "@..." syntax. I take it the command would
fail on that system. It would then go on to try the -r variant,
which has entirely different meaning on GNU (Linux) systems.

docs/ being subject to configuring, why don't you determine the
capabilities of "date" there and invoke just the one command
that was found suitable for the system?

Jan


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [XEN PATCH 2/3] docs: use predictable ordering in generated documentation
  2020-12-18 20:42 ` [XEN PATCH 2/3] docs: use predictable ordering in generated documentation Maximilian Engelhardt
@ 2020-12-22 10:38   ` Andrew Cooper
  0 siblings, 0 replies; 8+ messages in thread
From: Andrew Cooper @ 2020-12-22 10:38 UTC (permalink / raw)
  To: Maximilian Engelhardt, xen-devel
  Cc: George Dunlap, Ian Jackson, Jan Beulich, Julien Grall,
	Stefano Stabellini, Wei Liu

On 18/12/2020 20:42, Maximilian Engelhardt wrote:
> When the seq number is equal, sort by the title to get predictable
> output ordering. This is useful for reproducible builds.
>
> Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available
  2020-12-21  9:01   ` Jan Beulich
@ 2020-12-23 16:10     ` Maximilian Engelhardt
  0 siblings, 0 replies; 8+ messages in thread
From: Maximilian Engelhardt @ 2020-12-23 16:10 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Ian Jackson, Wei Liu, xen-devel

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

On Montag, 21. Dezember 2020 10:01:14 CET Jan Beulich wrote:
> On 18.12.2020 21:42, Maximilian Engelhardt wrote:
> > --- a/docs/Makefile
> > +++ b/docs/Makefile
> > @@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk
> > 
> >  -include $(XEN_ROOT)/config/Docs.mk
> >  
> >  VERSION		:= $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory
> >  xenversion)> 
> > -DATE		:= $(shell date +%Y-%m-%d)
> > +
> > +DATE_FMT	:= +%Y-%m-%d
> > +ifdef SOURCE_DATE_EPOCH
> > +DATE		:= $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)"
> > 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)"
> > 2>/dev/null || date -u "$(DATE_FMT)")
> Looking at the doc for a (deliberately) old "date", I can't find
> any mention of the -d "@..." syntax. I take it the command would
> fail on that system. It would then go on to try the -r variant,
> which has entirely different meaning on GNU (Linux) systems.
> 
> docs/ being subject to configuring, why don't you determine the
> capabilities of "date" there and invoke just the one command
> that was found suitable for the system?
> 
> Jan

Hi Jan,

I did some research. The -d "@..." syntax was introduced about 2005. Testing a 
live CD from 2006 (KNOPPIX_V5.0.1CD-2006-06-01-EN.iso) it was supported there. 
The documentation about this syntax has only been added in 2011 to the date 
command. I'm wondering if anybody running such an old system wants to use 
SOURCE_DATE_EPOCH.

However, I came up with a patch to determine which suitable date version is 
available and only call that, as you suggested. I will post the new patch 
soon.

Maxi

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2020-12-23 16:10 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-18 20:42 [XEN PATCH 0/3] Improvements for reproducible builds Maximilian Engelhardt
2020-12-18 20:42 ` [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined Maximilian Engelhardt
2020-12-21  8:50   ` Jan Beulich
2020-12-18 20:42 ` [XEN PATCH 2/3] docs: use predictable ordering in generated documentation Maximilian Engelhardt
2020-12-22 10:38   ` Andrew Cooper
2020-12-18 20:42 ` [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available Maximilian Engelhardt
2020-12-21  9:01   ` Jan Beulich
2020-12-23 16:10     ` Maximilian Engelhardt

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.