[PATCH 0/4 v2] tools: various changes

[PATCH 0/4 v2] tools: various changes

[Olaf Hering]

A resend of this series, with comments addressed:
http://lists.xenproject.org/archives/html/xen-devel/2015-03/msg02645.html

* The XENSTORED_ARGS= should be fine according to IanJ

* The changes for reproducible builds are really just interesting for
  distro packages. Passing all the variables via a rpm spec file for
  example can be automated in such build environment. The current
  behaviour is preserved.

Please review once more and apply.

Thanks,
Olaf

Olaf Hering (4):
  tools/hotplug: introduce XENSTORED_ARGS= in sysconfig file.
  INSTALL: mention variables for reproducible builds
  hvmloader: add knob for fixed SMBIOS date string
  hvmloader: add knob for fixed VGABIOS date string

 INSTALL                                            | 8 ++++++++
 tools/firmware/hvmloader/Makefile                  | 5 ++++-
 tools/firmware/vgabios/Makefile                    | 4 ++--
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 8 ++++++++
 4 files changed, 22 insertions(+), 3 deletions(-)

[PATCH 1/4] tools/hotplug: introduce XENSTORED_ARGS= in sysconfig file.

[Olaf Hering]

It is already used in the runlevel script and the service file.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index c12fc8a..f0fa98d 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -21,6 +21,14 @@
 #XENSTORED=@XENSTORED@
 
 ## Type: string
+## Default: ""
+#
+# Additional commandline arguments to start xenstored,
+# like "--trace-file /var/log/xen/xenstored-trace.log"
+# See "@sbindir@/xenstored --help" for possible options.
+XENSTORED_ARGS=
+
+## Type: string
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages

[PATCH 2/4 v2] INSTALL: mention variables for reproducible builds

[Olaf Hering]

Mention two variables introduced by commit ac977f5 ("use more fixed
strings to build the hypervisor").

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
Cc: Tim Deegan <tim@xen.org>
---
v2: adjust format for XEN_BUILD_DATE

 INSTALL | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/INSTALL b/INSTALL
index 13d7a17..1ed44bb 100644
--- a/INSTALL
+++ b/INSTALL
@@ -211,6 +211,12 @@ changed with these variables.
 XEN_WHOAMI=
 XEN_DOMAIN=
 
+Some components of xen and tools will include an unpredictable timestamp
+into the binaries. To allow reproducible builds the following variables
+can be used to provide fixed timestamps in the expected format.
+XEN_BUILD_DATE=<output of date(1)>
+XEN_BUILD_TIME=hh:mm:ss
+
 The following variables can be used to tweak some aspects of the
 hypervisor build.
 verbose=y

[PATCH 3/4] hvmloader: add knob for fixed SMBIOS date string

[Olaf Hering]

To allow reproducible builds of hvmloader introduce a make variable
SMBIOS_REL_DATE=mm/dd/yyyy to provide a fixed date string. Without this
change the hvmloader binary changes with every rebuild.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Jan Beulich <jbeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 INSTALL                           | 1 +
 tools/firmware/hvmloader/Makefile | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/INSTALL b/INSTALL
index 1ed44bb..bf412ef 100644
--- a/INSTALL
+++ b/INSTALL
@@ -216,6 +216,7 @@ into the binaries. To allow reproducible builds the following variables
 can be used to provide fixed timestamps in the expected format.
 XEN_BUILD_DATE=<output of date(1)>
 XEN_BUILD_TIME=hh:mm:ss
+SMBIOS_REL_DATE=mm/dd/yyyy
 
 The following variables can be used to tweak some aspects of the
 hypervisor build.
diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile
index cf967fd..34008d2 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -26,6 +26,9 @@ SUBDIRS := acpi
 # The HVM loader is started in 32-bit mode at the address below:
 LOADADDR = 0x100000
 
+# SMBIOS spec requires format mm/dd/yyyy
+SMBIOS_REL_DATE ?= $(shell date +%m/%d/%Y)
+
 CFLAGS += $(CFLAGS_xeninclude)
 
 OBJS  = hvmloader.o mp_tables.o util.o smbios.o 
@@ -88,7 +91,7 @@ all: subdirs-all
 	$(MAKE) hvmloader
 
 ovmf.o rombios.o seabios.o hvmloader.o: roms.inc
-smbios.o: CFLAGS += -D__SMBIOS_DATE__="\"$(shell date +%m/%d/%Y)\""
+smbios.o: CFLAGS += -D__SMBIOS_DATE__="$(SMBIOS_REL_DATE)"
 
 hvmloader: $(OBJS) acpi/acpi.a
 	$(LD) $(LDFLAGS_DIRECT) -N -Ttext $(LOADADDR) -o hvmloader.tmp $^

[PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string

[Olaf Hering]

To allow reproducible builds of hvmloader introduce a make variable
VGABIOS_REL_DATE="dd Mon yyyy" to provide a fixed date string. Without
this change the hvmloader binary changes with every rebuild.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 INSTALL                         | 1 +
 tools/firmware/vgabios/Makefile | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/INSTALL b/INSTALL
index bf412ef..a0f2e7b 100644
--- a/INSTALL
+++ b/INSTALL
@@ -217,6 +217,7 @@ can be used to provide fixed timestamps in the expected format.
 XEN_BUILD_DATE=<output of date(1)>
 XEN_BUILD_TIME=hh:mm:ss
 SMBIOS_REL_DATE=mm/dd/yyyy
+VGABIOS_REL_DATE="dd Mon yyyy"
 
 The following variables can be used to tweak some aspects of the
 hypervisor build.
diff --git a/tools/firmware/vgabios/Makefile b/tools/firmware/vgabios/Makefile
index 51d9e6e..3284812 100644
--- a/tools/firmware/vgabios/Makefile
+++ b/tools/firmware/vgabios/Makefile
@@ -5,10 +5,10 @@ BCC = bcc
 AS86 = as86
 
 RELEASE = `pwd | sed "s-.*/--"`
-RELDATE = `date '+%d %b %Y'`
+VGABIOS_REL_DATE ?= `date '+%d %b %Y'`
 RELVERS = `pwd | sed "s-.*/--" | sed "s/vgabios//" | sed "s/-//"`
 
-VGABIOS_DATE = "-DVGABIOS_DATE=\"$(RELDATE)\""
+VGABIOS_DATE = "-DVGABIOS_DATE=\"$(VGABIOS_REL_DATE)\""
 
 .PHONY: all
 all: bios cirrus-bios

Re: [PATCH 2/4 v2] INSTALL: mention variables for reproducible builds

[Ian Campbell]

On Wed, 2015-04-01 at 13:28 +0000, Olaf Hering wrote:
> Mention two variables introduced by commit ac977f5 ("use more fixed
> strings to build the hypervisor").
> 
> Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string

[Ian Campbell]

On Wed, 2015-04-01 at 13:28 +0000, Olaf Hering wrote:
> To allow reproducible builds of hvmloader introduce a make variable
> VGABIOS_REL_DATE="dd Mon yyyy" to provide a fixed date string. Without
> this change the hvmloader binary changes with every rebuild.
> 
> Signed-off-by: Olaf Hering <olaf@aepfle.de>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

It's not clear if vgabios should be maintained by us tools folks or if
it should have been taken under the hvmloader umbrella and maintained by
the hypervisor folks.

I've CC-d Jan, but I'm not going to block on an ack.

> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
>  INSTALL                         | 1 +
>  tools/firmware/vgabios/Makefile | 4 ++--
>  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/INSTALL b/INSTALL
> index bf412ef..a0f2e7b 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -217,6 +217,7 @@ can be used to provide fixed timestamps in the expected format.
>  XEN_BUILD_DATE=<output of date(1)>
>  XEN_BUILD_TIME=hh:mm:ss
>  SMBIOS_REL_DATE=mm/dd/yyyy
> +VGABIOS_REL_DATE="dd Mon yyyy"
>  
>  The following variables can be used to tweak some aspects of the
>  hypervisor build.
> diff --git a/tools/firmware/vgabios/Makefile b/tools/firmware/vgabios/Makefile
> index 51d9e6e..3284812 100644
> --- a/tools/firmware/vgabios/Makefile
> +++ b/tools/firmware/vgabios/Makefile
> @@ -5,10 +5,10 @@ BCC = bcc
>  AS86 = as86
>  
>  RELEASE = `pwd | sed "s-.*/--"`
> -RELDATE = `date '+%d %b %Y'`
> +VGABIOS_REL_DATE ?= `date '+%d %b %Y'`
>  RELVERS = `pwd | sed "s-.*/--" | sed "s/vgabios//" | sed "s/-//"`
>  
> -VGABIOS_DATE = "-DVGABIOS_DATE=\"$(RELDATE)\""
> +VGABIOS_DATE = "-DVGABIOS_DATE=\"$(VGABIOS_REL_DATE)\""
>  
>  .PHONY: all
>  all: bios cirrus-bios

Re: [PATCH 0/4 v2] tools: various changes

[Ian Campbell]

On Wed, 2015-04-01 at 13:28 +0000, Olaf Hering wrote:
> A resend of this series, with comments addressed:
> http://lists.xenproject.org/archives/html/xen-devel/2015-03/msg02645.html

Applied with my acks on 1,2&4 and Jan's on 3.

Re: [PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string

[Jan Beulich]

>>> On 02.04.15 at 11:45, <ian.campbell@citrix.com> wrote:
> On Wed, 2015-04-01 at 13:28 +0000, Olaf Hering wrote:
>> To allow reproducible builds of hvmloader introduce a make variable
>> VGABIOS_REL_DATE="dd Mon yyyy" to provide a fixed date string. Without
>> this change the hvmloader binary changes with every rebuild.
>> 
>> Signed-off-by: Olaf Hering <olaf@aepfle.de>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> 
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
> 
> It's not clear if vgabios should be maintained by us tools folks or if
> it should have been taken under the hvmloader umbrella and maintained by
> the hypervisor folks.

I'm not sure either - hvmloader is really code we grew and hence
maintaining it together with its hypervisor side makes sense. For
the various firmware blobs, otoh, I think they match more closely
with the qemu/tools pattern.

Jan

Re: [PATCH 4/4] hvmloader: add knob for fixed VGABIOS date string

[Ian Campbell]

On Mon, 2015-04-13 at 09:51 +0100, Jan Beulich wrote:
> >>> On 02.04.15 at 11:45, <ian.campbell@citrix.com> wrote:
> > On Wed, 2015-04-01 at 13:28 +0000, Olaf Hering wrote:
> >> To allow reproducible builds of hvmloader introduce a make variable
> >> VGABIOS_REL_DATE="dd Mon yyyy" to provide a fixed date string. Without
> >> this change the hvmloader binary changes with every rebuild.
> >> 
> >> Signed-off-by: Olaf Hering <olaf@aepfle.de>
> >> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> >> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > 
> > Acked-by: Ian Campbell <ian.campbell@citrix.com>
> > 
> > It's not clear if vgabios should be maintained by us tools folks or if
> > it should have been taken under the hvmloader umbrella and maintained by
> > the hypervisor folks.
> 
> I'm not sure either - hvmloader is really code we grew and hence
> maintaining it together with its hypervisor side makes sense. For
> the various firmware blobs, otoh, I think they match more closely
> with the qemu/tools pattern.

OK, I think that makes sense.

We have specific maintainers for OVMF and SeaBIOS, I think the other
stuff (rombios and vgabios) is frozen/inactive enough (and specific to
qemu-trad) that we can muddle through via the tools/* umbrella
maintainership.

I think ipxe is only used with trad too, but I'm not sure if it is quite
so inactive or if someone ought to be keeping an eye on e.g. new
upstream versions. I think we can just continue as we have been there
too.

Ian.