[U-Boot] [PATCH v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[U-Boot] [PATCH v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Paul Kocialkowski]

In order to achieve reproducible builds in U-Boot, timestamps that are defined
at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
built reproducibly. This is the case for e.g. sunxi devices.

However, some other devices might need some more tweaks, especially regarding
the image generation tools.

Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
---
 Makefile              |  7 ++++---
 README                | 12 ++++++++++++
 tools/default_image.c | 21 ++++++++++++++++++++-
 3 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 54ef2cd..596974c 100644
--- a/Makefile
+++ b/Makefile
@@ -1232,9 +1232,10 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date +'#define U_BOOT_TZ "%z"')
+	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
 endef
 
 $(version_h): include/config/uboot.release FORCE
diff --git a/README b/README
index 4e0ff9f..1bcb63c 100644
--- a/README
+++ b/README
@@ -5081,6 +5081,18 @@ This firmware often needs to be loaded during U-Boot booting.
 - CONFIG_SYS_MEM_TOP_HIDE_MIN
 	Define minimum DDR size to be hided from top of the DDR memory
 
+Reproducible builds
+-------------------
+
+In order to achieve reproducible builds, timestamps used in the U-Boot build
+process have to be set to a fixed value.
+
+This is done using the SOURCE_DATE_EPOCH environment variable.
+SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
+option for U-Boot or an environment variable in U-Boot.
+
+SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
+
 Building the Software:
 ======================
 
diff --git a/tools/default_image.c b/tools/default_image.c
index cf5c0d4..18940af 100644
--- a/tools/default_image.c
+++ b/tools/default_image.c
@@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
 				struct image_tool_params *params)
 {
 	uint32_t checksum;
+	char *source_date_epoch;
+	struct tm *time_universal;
+	time_t time;
 
 	image_header_t * hdr = (image_header_t *)ptr;
 
@@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
 				sizeof(image_header_t)),
 			sbuf->st_size - sizeof(image_header_t));
 
+	source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+	if (source_date_epoch != NULL) {
+		time = (time_t) strtol(source_date_epoch, NULL, 10);
+
+		time_universal = gmtime(&time);
+		if (time_universal == NULL) {
+			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
+				__func__);
+			time = 0;
+		} else {
+			time = mktime(time_universal);
+		}
+	} else {
+		time = sbuf->st_mtime;
+	}
+
 	/* Build new header */
 	image_set_magic(hdr, IH_MAGIC);
-	image_set_time(hdr, sbuf->st_mtime);
+	image_set_time(hdr, time);
 	image_set_size(hdr, sbuf->st_size - sizeof(image_header_t));
 	image_set_load(hdr, params->addr);
 	image_set_ep(hdr, params->ep);
-- 
1.9.1

[U-Boot] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Tom Rini]

On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:

> In order to achieve reproducible builds in U-Boot, timestamps that are defined
> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
> variable allows setting a fixed value for those timestamps.
> 
> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
> built reproducibly. This is the case for e.g. sunxi devices.
> 
> However, some other devices might need some more tweaks, especially regarding
> the image generation tools.
> 
> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150728/784dbdb0/attachment.sig>

[U-Boot] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Bin Meng]

Hi,

On Tue, Jul 28, 2015 at 11:00 PM, Tom Rini <trini@konsulko.com> wrote:
> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
>
>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
>> variable allows setting a fixed value for those timestamps.
>>
>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
>> built reproducibly. This is the case for e.g. sunxi devices.
>>
>> However, some other devices might need some more tweaks, especially regarding
>> the image generation tools.
>>
>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
>
> Applied to u-boot/master, thanks!
>
> --

This commit breaks the following commit:

commit f3f431a712729a1af94d01bd1bfde17a252ff02c
Author: Chris Packham <judge.packham@gmail.com>
Date:   Sun May 10 21:02:09 2015 +1200

    Makefile: Add U_BOOT_TZ and include in version

    Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to
    include the timezone in the version output.

    Acked-by: Simon Glass <sjg@chromium.org>
    Signed-off-by: Chris Packham <judge.packham@gmail.com>

Before this commit I have:
U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)

After this commit I have:
U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)

As you see: the timezone information is missing, and U-Boot's
timestamp is now GMT+0 (the correct one should be GMT+8)

Is this intended behavior? Or if not, please fix it.

Regards,
Bin

[U-Boot] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Chris Packham]

On Fri, Jul 31, 2015 at 2:54 PM, Bin Meng <bmeng.cn@gmail.com> wrote:
> Hi,
>
> On Tue, Jul 28, 2015 at 11:00 PM, Tom Rini <trini@konsulko.com> wrote:
>> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
>>
>>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
>>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
>>> variable allows setting a fixed value for those timestamps.
>>>
>>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
>>> built reproducibly. This is the case for e.g. sunxi devices.
>>>
>>> However, some other devices might need some more tweaks, especially regarding
>>> the image generation tools.
>>>
>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
>>
>> Applied to u-boot/master, thanks!
>>
>> --
>
> This commit breaks the following commit:
>
> commit f3f431a712729a1af94d01bd1bfde17a252ff02c
> Author: Chris Packham <judge.packham@gmail.com>
> Date:   Sun May 10 21:02:09 2015 +1200
>
>     Makefile: Add U_BOOT_TZ and include in version
>
>     Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to
>     include the timezone in the version output.
>
>     Acked-by: Simon Glass <sjg@chromium.org>
>     Signed-off-by: Chris Packham <judge.packham@gmail.com>
>
> Before this commit I have:
> U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)
>
> After this commit I have:
> U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)
>
> As you see: the timezone information is missing, and U-Boot's
> timestamp is now GMT+0 (the correct one should be GMT+8)
>
> Is this intended behavior? Or if not, please fix it.
>
> Regards,
> Bin

The problem is that date -u implies UTC. So +0000 is right if you do
want the time to be displayed in UTC (for me living at +12/+13 UTC
makes my head hurt because all the dates are yesterday).

The intent of f3f431a7 was to reflect the timezone that the build
machine was set to. Dropping the -u would probably be sufficient but
perhaps it would be better to make whole lot conditional on
SOURCE_DATE_EPOCH being set. Something like this (untested, apologies
gmail web interface)

ifneq ($(SOURCE_DATE_EPOCH),)
define filechk_timestamp.h
       (SOURCE_DATE="$(SOURCE_DATE_EPOCH)"; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"' )
endef
else
define filechk_timestamp.h
       (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
       LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date +'#define U_BOOT_TZ "%z"')
endef
endif

[U-Boot] [PATCH] Makefile: Use correct timezone for U_BOOT_TZ

[Chris Packham]

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
building normally the timezone is taken from the build machine's locale
setting.

Fixes: f3f431a71272 ("Reproducible U-Boot build support, using
SOURCE_DATE_EPOCH")
Signed-off-by: Chris Packham <judge.packham@gmail.com>
---
Hopefully this will suit everyone. I've done a quick sanity test with
sandbox and the date shown in the version output looks to be what I
expect with/without SOURCE_DATE_EPOCH.

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 394ed09..f75c730 100644
--- a/Makefile
+++ b/Makefile
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
+		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+	else \
+		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+	fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
-- 
2.5.0.rc0

[U-Boot] [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Ximin Luo]

On 31/07/15 07:25, Chris Packham wrote:
> The problem is that date -u implies UTC. So +0000 is right if you do
> want the time to be displayed in UTC (for me living at +12/+13 UTC
> makes my head hurt because all the dates are yesterday).
> 

One of the main problems with human-readable date/time is that it implies a calendar, with all its exceptions like leap years, changing DST, etc. Many RFCs, and the POSIX date functions, are highly flawed and conflate calendars with absolute time, e.g. don't specify they mean the Gregorian calendar, and don't specify what to do with dates before when the Gregorian calendar was introduced.

If you want to be precise about these things (so as to not cause problems in the future), then you need to specify all of that when dealing with human-readable dates. This is why for now, the Reproducible Builds timestamps proposal only defines SOURCE_DATE_EPOCH. We might extend it later, though.

> The intent of f3f431a7 was to reflect the timezone that the build
> machine was set to. Dropping the -u would probably be sufficient but
> perhaps it would be better to make whole lot conditional on
> SOURCE_DATE_EPOCH being set. Something like this (untested, apologies
> gmail web interface)
> 
> ifneq ($(SOURCE_DATE_EPOCH),)
> define filechk_timestamp.h
>        (SOURCE_DATE="$(SOURCE_DATE_EPOCH)"; \
>        LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
> %d %C%y"'; \
>        LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
>        LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"' )
> endef
> else
> define filechk_timestamp.h
>        (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>        LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>        LC_ALL=C date +'#define U_BOOT_TZ "%z"')
> endef
> endif
> 

Yes, it is best to force the TZ as UTC for now, if SOURCE_DATE_EPOCH is set.

At some point later, we might add SOURCE_DATE_TZOFFSET, or other formats that include a timezone offset, or something. (That would match Git's internal timestamp format, which is "$epochts $tzoffset", and can be translated unambiguously into ISO8601 and RFC2822 formats). For now though, we decided to keep things simple. But please say so, if you think that is something you (or others who use timestamps in build systems) would really want.

BTW, you need to do "@$${SOURCE_DATE_EPOCH}" when giving timestamps to date(1):

$ date -d @1438330000 +%s
1438330000
$ date -d 1438330000 +%s
date: invalid date ?1438330000?
1

X

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150731/79985a4a/attachment.sig>

[U-Boot] [PATCH] Makefile: Use correct timezone for U_BOOT_TZ

[Bin Meng]

On Fri, Jul 31, 2015 at 6:04 PM, Chris Packham <judge.packham@gmail.com> wrote:
> When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
> building normally the timezone is taken from the build machine's locale
> setting.
>
> Fixes: f3f431a71272 ("Reproducible U-Boot build support, using
> SOURCE_DATE_EPOCH")
> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> ---
> Hopefully this will suit everyone. I've done a quick sanity test with
> sandbox and the date shown in the version output looks to be what I
> expect with/without SOURCE_DATE_EPOCH.
>
>  Makefile | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 394ed09..f75c730 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1230,10 +1230,16 @@ define filechk_version.h
>  endef
>
>  define filechk_timestamp.h
> -       (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
> +       (if test -n "$${SOURCE_DATE_EPOCH}"; then \
> +               SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +       else \
> +               LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +               LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> +               LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
> +       fi)
>  endef
>
>  $(version_h): include/config/uboot.release FORCE
> --

Tested-by: Bin Meng <bmeng.cn@gmail.com>

[U-Boot] [PATCH] Makefile: Use correct timezone for U_BOOT_TZ

[Paul Kocialkowski]

Le vendredi 31 juillet 2015 ? 22:04 +1200, Chris Packham a ?crit :
> When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
> building normally the timezone is taken from the build machine's locale
> setting.

Thanks a lot for introducing this, I thought it might disturb a few
people.

> Fixes: f3f431a71272 ("Reproducible U-Boot build support, using
> SOURCE_DATE_EPOCH")

It is incorrect to say that your patch fixes my commit, because my
commit didn't introduce anything unexpected or faulty.

If anything, it was merged before people gave their opinion about
whether UTC should be default. I was also going to introduce a patch
like yours later. For the record, UTC is default for Coreboot's
timestamps.

Thus, please remove that line.

Otherwise, this is:
Tested-By: Paul Kocialkowski <contact@paulk.fr>

> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> ---
> Hopefully this will suit everyone. I've done a quick sanity test with
> sandbox and the date shown in the version output looks to be what I
> expect with/without SOURCE_DATE_EPOCH.
> 
>  Makefile | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 394ed09..f75c730 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1230,10 +1230,16 @@ define filechk_version.h
>  endef
>  
>  define filechk_timestamp.h
> -	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
> -	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
> -	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
> +	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
> +		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> +		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +	else \
> +		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> +		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
> +	fi)
>  endef
>  
>  $(version_h): include/config/uboot.release FORCE

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150731/e7d7218c/attachment-0001.sig>

[U-Boot] [PATCH] Makefile: Use correct timezone for U_BOOT_TZ

[Chris Packham]

(note I had some patman config that erroneously included extra people
on the original Cc list, I've now dropped them)

On Sat, Aug 1, 2015 at 5:05 AM, Paul Kocialkowski <contact@paulk.fr> wrote:
> Le vendredi 31 juillet 2015 ? 22:04 +1200, Chris Packham a ?crit :
>> When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
>> building normally the timezone is taken from the build machine's locale
>> setting.
>
> Thanks a lot for introducing this, I thought it might disturb a few
> people.
>
>> Fixes: f3f431a71272 ("Reproducible U-Boot build support, using
>> SOURCE_DATE_EPOCH")
>
> It is incorrect to say that your patch fixes my commit, because my
> commit didn't introduce anything unexpected or faulty.
>
> If anything, it was merged before people gave their opinion about
> whether UTC should be default. I was also going to introduce a patch
> like yours later. For the record, UTC is default for Coreboot's
> timestamps.
>
> Thus, please remove that line.

Done. v2 on it's way.

The other mailing list thread also talked about the possibility of
having SOURCE_DATE_TZOFFSET I'll look at a proof of concept for that.

> Otherwise, this is:
> Tested-By: Paul Kocialkowski <contact@paulk.fr>
>
>> Signed-off-by: Chris Packham <judge.packham@gmail.com>
>> ---
>> Hopefully this will suit everyone. I've done a quick sanity test with
>> sandbox and the date shown in the version output looks to be what I
>> expect with/without SOURCE_DATE_EPOCH.
>>
>>  Makefile | 14 ++++++++++----
>>  1 file changed, 10 insertions(+), 4 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 394ed09..f75c730 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1230,10 +1230,16 @@ define filechk_version.h
>>  endef
>>
>>  define filechk_timestamp.h
>> -     (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
>> -     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
>> -     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
>> -     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
>> +     (if test -n "$${SOURCE_DATE_EPOCH}"; then \
>> +             SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
>> +             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
>> +             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
>> +             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>> +     else \
>> +             LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>> +             LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>> +             LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
>> +     fi)
>>  endef
>>
>>  $(version_h): include/config/uboot.release FORCE
>

[U-Boot] [PATCH v2] Makefile: Use correct timezone for U_BOOT_TZ

[Chris Packham]

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
building normally the timezone is taken from the build machine's locale
setting.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Paul Kocialkowski <contact@paulk.fr>
---

Changes in v2:
- Collect some tested-by tags
- Remove reference to f3f431a71272 in the commit message
- Drop Ccs that were erroneously added when submitting v1, remaining Ccs
  are from the original mailing list thread

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 394ed09..f75c730 100644
--- a/Makefile
+++ b/Makefile
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
+		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+	else \
+		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+	fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
-- 
2.5.0.rc0

[U-Boot] [RFC PATCH] Makefile: Add SOURCE_DATE_TZ

[Chris Packham]

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a
build with a specific date timestamp. This allows the verification of
source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify
what will appear in the output of the version command. If SOURCE_DATE_TZ
is not specified UTC will be used.  SOURCE_DATE_TZ on it's own will not
have an affect.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
---
This is a quick proof of concept that allows some control of how the
timezone is displayed.

 Makefile | 7 ++++---
 README   | 9 ++++++---
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index f75c730..5102a9c 100644
--- a/Makefile
+++ b/Makefile
@@ -1232,9 +1232,10 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		SOURCE_TZ="$${SOURCE_DATE_TZ:-UTC}"; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
diff --git a/README b/README
index 1bcb63c..d2e3e94 100644
--- a/README
+++ b/README
@@ -5087,11 +5087,14 @@ Reproducible builds
 In order to achieve reproducible builds, timestamps used in the U-Boot build
 process have to be set to a fixed value.
 
-This is done using the SOURCE_DATE_EPOCH environment variable.
-SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
-option for U-Boot or an environment variable in U-Boot.
+This is done using the SOURCE_DATE_EPOCH and SOURCE_DATE_TZ environment
+variables. These are to be set on the build host's shell, not as a
+configuration option for U-Boot or an environment variable in U-Boot.
 
 SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
+SOURCE_DATE_TZ will default to UTC but can be set to another timezone and is
+used to determine how the date is displayed (tzselect(1) can be used to
+determine an appropriate value).
 
 Building the Software:
 ======================
-- 
2.5.0.rc0

[U-Boot] [RFC PATCH] Makefile: Add SOURCE_DATE_TZ

[Paul Kocialkowski]

Le samedi 01 ao?t 2015 ? 22:32 +1200, Chris Packham a ?crit :
> Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a
> build with a specific date timestamp. This allows the verification of
> source supplied with a pre-compiled binary.
> 
> If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify
> what will appear in the output of the version command. If SOURCE_DATE_TZ
> is not specified UTC will be used.  SOURCE_DATE_TZ on it's own will not
> have an affect.

Well, I worked with the assumption that SOURCE_DATE_EPOCH would always
be provided in UTC, but I see no harm in providing SOURCE_DATE_TZ as
well, provided that it falls back to UTC when not set, as is the current
behaviour of tour patch.

Looks good to me!

> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> ---
> This is a quick proof of concept that allows some control of how the
> timezone is displayed.
> 
>  Makefile | 7 ++++---
>  README   | 9 ++++++---
>  2 files changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index f75c730..5102a9c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1232,9 +1232,10 @@ endef
>  define filechk_timestamp.h
>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +		SOURCE_TZ="$${SOURCE_DATE_TZ:-UTC}"; \
> +		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>  	else \
>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> diff --git a/README b/README
> index 1bcb63c..d2e3e94 100644
> --- a/README
> +++ b/README
> @@ -5087,11 +5087,14 @@ Reproducible builds
>  In order to achieve reproducible builds, timestamps used in the U-Boot build
>  process have to be set to a fixed value.
>  
> -This is done using the SOURCE_DATE_EPOCH environment variable.
> -SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
> -option for U-Boot or an environment variable in U-Boot.
> +This is done using the SOURCE_DATE_EPOCH and SOURCE_DATE_TZ environment
> +variables. These are to be set on the build host's shell, not as a
> +configuration option for U-Boot or an environment variable in U-Boot.
>  
>  SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
> +SOURCE_DATE_TZ will default to UTC but can be set to another timezone and is
> +used to determine how the date is displayed (tzselect(1) can be used to
> +determine an appropriate value).
>  
>  Building the Software:
>  ======================

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150801/ae46b64a/attachment.sig>

[U-Boot] [RFC PATCH] Makefile: Add SOURCE_DATE_TZ

[Ximin Luo]

On 01/08/15 20:47, Paul Kocialkowski wrote:
> Le samedi 01 ao?t 2015 ? 22:32 +1200, Chris Packham a ?crit :
>> Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a
>> build with a specific date timestamp. This allows the verification of
>> source supplied with a pre-compiled binary.
>>
>> If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify
>> what will appear in the output of the version command. If SOURCE_DATE_TZ
>> is not specified UTC will be used.  SOURCE_DATE_TZ on it's own will not
>> have an affect.
> 
> Well, I worked with the assumption that SOURCE_DATE_EPOCH would always
> be provided in UTC, but I see no harm in providing SOURCE_DATE_TZ as
> well, provided that it falls back to UTC when not set, as is the current
> behaviour of tour patch.
> 

To clarify, SOURCE_DATE_EPOCH is a unix timestamp which is defined as the number of seconds (excluding leap seconds) since Jan 1 1970 UTC. There is no way to specify this "in another timezone"; there is no ambiguity here.

However, I am not sure that us at Reproducible Builds will actually adopt this variable. We haven't talked about it beyond my previous email [1], it was just me quickly skimming off my thoughts and firing off quick ideas. My personal concern about SOURCE_DATE_TZ is that it implies that it could take actual named time zones, like "EST" or "Europe/London"; it is **extremely unlikely** that we will do anything like this soon, because this would involve timezone databases and complex things like that. This is why in my previous email I suggested the SOURCE_DATE_TZOFFSET variable. Also, the TZ variable as specified by POSIX specifies the offset in the *opposite* direction to what ISO8601 and RFC2822 displays.

The git internal timestamp format only supports timezone offsets in the common direction, opposite to TZ, i.e. positive for east of Greenwich and negative for west.

I'd suggest to leave out SOURCE_DATE_TZ for now, until we come up with a more precisely defined *meaning* for this variable. It is meant to be a standard across all tools, so some time to think through the issues is necessary. Otherwise we risk leaving a clusterfuck of a legacy, like the POSIX time functions.

X

[1] https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20150727/002562.html or http://lists.denx.de/pipermail/u-boot/2015-July/221301.html

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150802/4bc9a798/attachment.sig>

[U-Boot] [RFC PATCH] Makefile: Add SOURCE_DATE_TZ

[Ximin Luo]

On 02/08/15 00:02, Ximin Luo wrote:
> However, I am not sure that us at Reproducible Builds will actually adopt this variable. We haven't talked about it beyond my previous email [1], it was just me quickly skimming off my thoughts and firing off quick ideas.

Whoops, I mean the _TZ variable. The _EPOCH variable has been decided and we're trying to get tools to support this.

-- 
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150802/3e352a9f/attachment.sig>

[U-Boot] [PATCH v2] Makefile: Use correct timezone for U_BOOT_TZ

[Chris Packham]

Hi Tom,

On Sat, Aug 1, 2015 at 9:43 PM, Chris Packham <judge.packham@gmail.com> wrote:
> When building with SOURCE_DATE_EPOCH the timezone is in UTC. When
> building normally the timezone is taken from the build machine's locale
> setting.
>
> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> Tested-by: Bin Meng <bmeng.cn@gmail.com>
> Tested-by: Paul Kocialkowski <contact@paulk.fr>
> ---
>
> Changes in v2:
> - Collect some tested-by tags
> - Remove reference to f3f431a71272 in the commit message
> - Drop Ccs that were erroneously added when submitting v1, remaining Ccs
>   are from the original mailing list thread
>
>  Makefile | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 394ed09..f75c730 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1230,10 +1230,16 @@ define filechk_version.h
>  endef
>
>  define filechk_timestamp.h
> -       (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
> -       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
> +       (if test -n "$${SOURCE_DATE_EPOCH}"; then \
> +               SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +       else \
> +               LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +               LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> +               LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
> +       fi)
>  endef
>
>  $(version_h): include/config/uboot.release FORCE
> --
> 2.5.0.rc0
>

Ping? I think this may have been missed due to an excessive Cc list
(which I've now culled).

[U-Boot] [U-Boot,RFC] Makefile: Add SOURCE_DATE_TZ

[Tom Rini]

On Sat, Aug 01, 2015 at 10:32:49PM +1200, Chris Packham wrote:

> Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a
> build with a specific date timestamp. This allows the verification of
> source supplied with a pre-compiled binary.
> 
> If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify
> what will appear in the output of the version command. If SOURCE_DATE_TZ
> is not specified UTC will be used.  SOURCE_DATE_TZ on it's own will not
> have an affect.
> 
> Signed-off-by: Chris Packham <judge.packham@gmail.com>
> ---
> This is a quick proof of concept that allows some control of how the
> timezone is displayed.

Can you please re-submit this, and the related EPOC patches as a series
together?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150812/2f249189/attachment.sig>

[U-Boot] [U-Boot,RFC] Makefile: Add SOURCE_DATE_TZ

[Chris Packham]

On Thu, Aug 13, 2015 at 4:40 AM, Tom Rini <trini@konsulko.com> wrote:
> On Sat, Aug 01, 2015 at 10:32:49PM +1200, Chris Packham wrote:
>
>> Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a
>> build with a specific date timestamp. This allows the verification of
>> source supplied with a pre-compiled binary.
>>
>> If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify
>> what will appear in the output of the version command. If SOURCE_DATE_TZ
>> is not specified UTC will be used.  SOURCE_DATE_TZ on it's own will not
>> have an affect.
>>
>> Signed-off-by: Chris Packham <judge.packham@gmail.com>
>> ---
>> This is a quick proof of concept that allows some control of how the
>> timezone is displayed.
>
> Can you please re-submit this, and the related EPOC patches as a series
> together?  Thanks!
>

Hi Tom,

Will do although Ximin Luo had some concerns[1]. The code works but
there may (will?) be some churn in the naming/concept for the TZ part.

--
[1] - http://marc.info/?l=u-boot&m=143851366020937&w=2

[U-Boot] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Andreas Bießmann]

On 07/28/2015 05:00 PM, Tom Rini wrote:
> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
> 
>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
>> variable allows setting a fixed value for those timestamps.
>>
>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
>> built reproducibly. This is the case for e.g. sunxi devices.
>>
>> However, some other devices might need some more tweaks, especially regarding
>> the image generation tools.
>>
>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
> 
> Applied to u-boot/master, thanks!

This commit breaks build on non GNU hosts (like OS X and persumably
other *BSD hosts). Before, those hosts where supported, so for me this
has to be fixed for 2015.10

We need a) some mechanism to search for the GNU date variant or b) some
wrapper to provide the correct output on those host machines.

I vote for a), it is acceptable to have the GNU date available but we
should error on 'no GNU date available'. Furthermore we need to have the
date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

Andreas

[U-Boot] [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Vagrant Cascadian]

On 2015-08-25, Andreas Bie?mann wrote:
> On 07/28/2015 05:00 PM, Tom Rini wrote:
>> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
>>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
>>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
>>> variable allows setting a fixed value for those timestamps.
>>>
>>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
>>> built reproducibly. This is the case for e.g. sunxi devices.
>>>
>>> However, some other devices might need some more tweaks, especially regarding
>>> the image generation tools.
>>>
>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
>> 
>> Applied to u-boot/master, thanks!
>
> This commit breaks build on non GNU hosts (like OS X and persumably
> other *BSD hosts). Before, those hosts where supported, so for me this
> has to be fixed for 2015.10
>
> We need a) some mechanism to search for the GNU date variant or b) some
> wrapper to provide the correct output on those host machines.
>
> I vote for a), it is acceptable to have the GNU date available but we
> should error on 'no GNU date available'. Furthermore we need to have the
> date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

There was a proposed patch which only uses the GNU date extensions if
SOURCE_DATE_EPOCH environment variable is set, would this sufficiently
address your concerns, at least for the short term?

  Message-Id: <1438337042-30762-1-git-send-email-judge.packham@gmail.com>
  http://lists.denx.de/pipermail/u-boot/2015-August/221429.html


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/928dc33c/attachment.sig>

[U-Boot] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Fabio Estevam]

On Thu, Jul 30, 2015 at 11:54 PM, Bin Meng <bmeng.cn@gmail.com> wrote:

> This commit breaks the following commit:
>
> commit f3f431a712729a1af94d01bd1bfde17a252ff02c
> Author: Chris Packham <judge.packham@gmail.com>
> Date:   Sun May 10 21:02:09 2015 +1200
>
>     Makefile: Add U_BOOT_TZ and include in version
>
>     Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to
>     include the timezone in the version output.
>
>     Acked-by: Simon Glass <sjg@chromium.org>
>     Signed-off-by: Chris Packham <judge.packham@gmail.com>
>
> Before this commit I have:
> U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)
>
> After this commit I have:
> U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)
>
> As you see: the timezone information is missing, and U-Boot's
> timestamp is now GMT+0 (the correct one should be GMT+8)
>
> Is this intended behavior? Or if not, please fix it.

I notice the same behavior here and I agree it would be nice to fix this.

Thanks

[U-Boot] [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Andreas Bießmann]

On 08/25/2015 11:55 AM, Vagrant Cascadian wrote:
> On 2015-08-25, Andreas Bie?mann wrote:
>> On 07/28/2015 05:00 PM, Tom Rini wrote:
>>> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
>>>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
>>>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
>>>> variable allows setting a fixed value for those timestamps.
>>>>
>>>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
>>>> built reproducibly. This is the case for e.g. sunxi devices.
>>>>
>>>> However, some other devices might need some more tweaks, especially regarding
>>>> the image generation tools.
>>>>
>>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
>>>
>>> Applied to u-boot/master, thanks!
>>
>> This commit breaks build on non GNU hosts (like OS X and persumably
>> other *BSD hosts). Before, those hosts where supported, so for me this
>> has to be fixed for 2015.10
>>
>> We need a) some mechanism to search for the GNU date variant or b) some
>> wrapper to provide the correct output on those host machines.
>>
>> I vote for a), it is acceptable to have the GNU date available but we
>> should error on 'no GNU date available'. Furthermore we need to have the
>> date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.
> 
> There was a proposed patch which only uses the GNU date extensions if
> SOURCE_DATE_EPOCH environment variable is set, would this sufficiently
> address your concerns, at least for the short term?
> 
>   Message-Id: <1438337042-30762-1-git-send-email-judge.packham@gmail.com>
>   http://lists.denx.de/pipermail/u-boot/2015-August/221429.html

thanks for the pointer, normal builds work with that change.

Andreas

[U-Boot] SOURCE_DATE_EPOCH must not be linux only... (was Re: [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH)

[Holger Levsen]

Hi,

(just stating the (IMHO) obvious here?)

On Dienstag, 25. August 2015, Vagrant Cascadian wrote:
> There was a proposed patch which only uses the GNU date extensions if
> SOURCE_DATE_EPOCH environment variable is set, would this sufficiently
> address your concerns, at least for the short term?

there also should be a solution which works on BSD in the not so long term if 
we want the SOURCE_DATE_EPOCH specification to take off..!

(and this probably also affects other software where we want to (or did) 
introduce SOURCE_DATE_EPOCH!)


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/225583c0/attachment.sig>

[U-Boot] [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Paul Kocialkowski]

Le mardi 25 ao?t 2015 ? 12:20 +0200, Andreas Bie?mann a ?crit :
> On 08/25/2015 11:55 AM, Vagrant Cascadian wrote:
> > On 2015-08-25, Andreas Bie?mann wrote:
> >> On 07/28/2015 05:00 PM, Tom Rini wrote:
> >>> On Sun, Jul 26, 2015 at 06:48:15PM +0200, Paul Kocialkowski wrote:
> >>>> In order to achieve reproducible builds in U-Boot, timestamps that are defined
> >>>> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
> >>>> variable allows setting a fixed value for those timestamps.
> >>>>
> >>>> Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be
> >>>> built reproducibly. This is the case for e.g. sunxi devices.
> >>>>
> >>>> However, some other devices might need some more tweaks, especially regarding
> >>>> the image generation tools.
> >>>>
> >>>> Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
> >>>
> >>> Applied to u-boot/master, thanks!
> >>
> >> This commit breaks build on non GNU hosts (like OS X and persumably
> >> other *BSD hosts). Before, those hosts where supported, so for me this
> >> has to be fixed for 2015.10
> >>
> >> We need a) some mechanism to search for the GNU date variant or b) some
> >> wrapper to provide the correct output on those host machines.
> >>
> >> I vote for a), it is acceptable to have the GNU date available but we
> >> should error on 'no GNU date available'. Furthermore we need to have the
> >> date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.
> > 
> > There was a proposed patch which only uses the GNU date extensions if
> > SOURCE_DATE_EPOCH environment variable is set, would this sufficiently
> > address your concerns, at least for the short term?
> > 
> >   Message-Id: <1438337042-30762-1-git-send-email-judge.packham@gmail.com>
> >   http://lists.denx.de/pipermail/u-boot/2015-August/221429.html
> 
> thanks for the pointer, normal builds work with that change.

I think we should get that patch merged so that it doesn't affect
regular builds on non-GNU hosts. It is also relevant for the purpose it
serves initially, too.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/2d93b03d/attachment.sig>

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Andreas Bießmann]

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
variant of date. Respect this and search it, error on missing GNU date.

Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
---
This commit tries to figure out if we have a GNU date variant
available. It errors on missing GNU date when it is required (for
SOURCE_DATE_EPOCH set). The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make SOURCE_DATE_EPOCH="`date -R`" O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1303: *** "No GNU date found".  Stop.
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
  CHK     include/generated/timestamp_autogenerated.h
  UPD     include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

 Makefile | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index b9b2375..95eae64 100644
--- a/Makefile
+++ b/Makefile
@@ -346,6 +346,9 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+GNUDATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1284,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1298,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(GNUDATE)),)
+	$(error "No GNU date found")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------
-- 
2.1.4

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Marek Vasut]

On Thursday, August 27, 2015 at 10:13:49 AM, Andreas Bie?mann wrote:
> The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
> variant of date.

Why does it require the GNU date ?

> Respect this and search it, error on missing GNU date.

Wouldn't it make more sense to fix the code to NOT depend on GNU extensions ?

> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>

[...]

Best regards,
Marek Vasut

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Paul Kocialkowski]

Le jeudi 27 ao?t 2015 ? 10:13 +0200, Andreas Bie?mann a ?crit :
> The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
> variant of date. Respect this and search it, error on missing GNU date.

Well, IMHO we shouldn't check for GNU date but for the extensions it
implements. Those could be included in other date implementations. For
instance, it looks like busybox's date implements the -u and -d options
too.

> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
> ---
> This commit tries to figure out if we have a GNU date variant
> available. It errors on missing GNU date when it is required (for
> SOURCE_DATE_EPOCH set). The result is:

Also, I don't think it should be up to the Makefile to figure out which
date binary to use. It should be up to the user to put the right one as
"date", e.g. with an alias. That is, unless the name for GNU date is
something really standardized on non-GNU systems.

> ---8<---
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make SOURCE_DATE_EPOCH="`date -R`" O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
> /home/abiessmann/src/u-boot/Makefile:1303: *** "No GNU date found".  Stop.
> make[1]: Leaving directory '/tmp/picosam'
> Makefile:146: recipe for target 'sub-make' failed
> make: *** [sub-make] Error 2
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
>   CHK     include/generated/timestamp_autogenerated.h
>   UPD     include/generated/timestamp_autogenerated.h
> make[1]: Leaving directory '/tmp/picosam'
> --->8---
> 
> It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).
> 
>  Makefile | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index b9b2375..95eae64 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -346,6 +346,9 @@ PERL		= perl
>  PYTHON		= python
>  DTC		= dtc
>  CHECK		= sparse
> +GNUDATE	       := $(foreach date,gdate date.gnu date, \
> +		    $(shell _date=`which $(date)`; \
> +		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))

I advise running date with the needed options (-u -d) and checked the
error code instead of this. And of course, the variable names shouldn't
mention GNU date.

>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
>  		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
> @@ -1281,9 +1284,9 @@ endef
>  define filechk_timestamp.h
>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>  	else \
>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> @@ -1295,6 +1298,11 @@ $(version_h): include/config/uboot.release FORCE
>  	$(call filechk,version.h)
>  
>  $(timestamp_h): $(srctree)/Makefile FORCE
> +ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
> +ifeq ($(strip $(GNUDATE)),)
> +	$(error "No GNU date found")
> +endif
> +endif
>  	$(call filechk,timestamp.h)
>  
>  # ---------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/c052cc01/attachment.sig>

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Andreas Bießmann]

On 08/27/2015 10:32 AM, Paul Kocialkowski wrote:
> Le jeudi 27 ao?t 2015 ? 10:13 +0200, Andreas Bie?mann a ?crit :
>> The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
>> variant of date. Respect this and search it, error on missing GNU date.
> 
> Well, IMHO we shouldn't check for GNU date but for the extensions it
> implements. Those could be included in other date implementations. For
> instance, it looks like busybox's date implements the -u and -d options
> too.

But BSD date used the -d switch to set the timezone [1] ...

---8<---
  -d	dst  Set the kernel's value for	daylight saving	time.  If dst
             is non-zero, future calls	to gettimeofday(2) will	return
             a non-zero for tz_dsttime.
--->8---

And it is the same for OS X [2].

>> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
>> ---
>> This commit tries to figure out if we have a GNU date variant
>> available. It errors on missing GNU date when it is required (for
>> SOURCE_DATE_EPOCH set). The result is:
> 
> Also, I don't think it should be up to the Makefile to figure out which
> date binary to use. It should be up to the user to put the right one as
> "date", e.g. with an alias. That is, unless the name for GNU date is
> something really standardized on non-GNU systems.

Most *BSD systems use gdate or date.gnu for the GNU variant of date, if
available. The date tool available is the one of the *BSD distributiuon.
HAving a shell alias for date covering gdate is for some users not an
option. So it has to be some make variable. For me it would be neat to
have the make find some possible solution (e.g. search for the GNU
variant on systems which have it).

>> diff --git a/Makefile b/Makefile
>> index b9b2375..95eae64 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -346,6 +346,9 @@ PERL		= perl
>>  PYTHON		= python
>>  DTC		= dtc
>>  CHECK		= sparse
>> +GNUDATE	       := $(foreach date,gdate date.gnu date, \
>> +		    $(shell _date=`which $(date)`; \
>> +		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))
> 
> I advise running date with the needed options (-u -d) and checked the
> error code instead of this. And of course, the variable names shouldn't
> mention GNU date.

Good point, let's see how my OS X date behaves with a set '-d' switch.

Andreas

[1] http://www.freebsd.org/cgi/man.cgi?query=date
[2]
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/date.1.html

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Andreas Bießmann]

Hi Marek,

On 08/27/2015 10:28 AM, Marek Vasut wrote:
> On Thursday, August 27, 2015 at 10:13:49 AM, Andreas Bie?mann wrote:
>> The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
>> variant of date.
> 
> Why does it require the GNU date ?

Cause of the -d switch. It is used in GNU date variants to set the time
to print the string from. On the other hand *BSD variants use it to set
the DST flag.

>> Respect this and search it, error on missing GNU date.
> 
> Wouldn't it make more sense to fix the code to NOT depend on GNU extensions ?

That would be great, but so we need to check for the tool and use the
right switches depending on that.
I think forcing the user to provide the right tool here is Ok.

Best regards

Andreas

[U-Boot] [RFC PATCH v2] Makefile: search for GNU date

[Andreas Bießmann]

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date
with -u and -d switch to print the date string of another time. In other
words it requires some date that behaves like the GNU date.

Respect this and search a working date, error on no working version.

Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
---
This commit tries to figure out if we have a date variant
available that supports the '-u' and '-d "@0"' switches.
It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and
suffixes are widespread used for the GNU variant of a tool also avialable on
*BSD systems.

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
  CHK     include/generated/timestamp_autogenerated.h
  UPD     include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="0" ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1304: *** "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!".  Stop.
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index b9b2375..b797e38 100644
--- a/Makefile
+++ b/Makefile
@@ -346,6 +346,10 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+DATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} -u -d "@0" >/dev/null 2>&1; \
+		      test $$? -eq 0 && echo $${_date}))
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1285,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(DATE)),)
+	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------
-- 
2.1.4

[U-Boot] [RFC PATCH] Makefile: search for GNU date

[Marek Vasut]

On Thursday, August 27, 2015 at 11:01:19 AM, Andreas Bie?mann wrote:
> Hi Marek,

Hi!

> On 08/27/2015 10:28 AM, Marek Vasut wrote:
> > On Thursday, August 27, 2015 at 10:13:49 AM, Andreas Bie?mann wrote:
> >> The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU
> >> variant of date.
> > 
> > Why does it require the GNU date ?
> 
> Cause of the -d switch. It is used in GNU date variants to set the time
> to print the string from. On the other hand *BSD variants use it to set
> the DST flag.

This sort of stuff should be in the commit message :)

> >> Respect this and search it, error on missing GNU date.
> > 
> > Wouldn't it make more sense to fix the code to NOT depend on GNU
> > extensions ?
> 
> That would be great, but so we need to check for the tool and use the
> right switches depending on that.
> I think forcing the user to provide the right tool here is Ok.

I think we don't have a better option here anyway, right ? Maybe try
checking uname ? But you can have linux with bsd userspace ... and
probably BSD with GNU userspace, meh.

Best regards,
Marek Vasut

[U-Boot] [RFC PATCH v2] Makefile: search for GNU date

[Paul Kocialkowski]

Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :
> The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date
> with -u and -d switch to print the date string of another time. In other
> words it requires some date that behaves like the GNU date.
> 
> Respect this and search a working date, error on no working version.

Looks good to me, except one nitpick, see below.

> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
> ---
> This commit tries to figure out if we have a date variant
> available that supports the '-u' and '-d "@0"' switches.
> It errors on non-working variants of date.
>
> To respect *BSD host systems search for gdate and date.gnu. Those pre- and
> suffixes are widespread used for the GNU variant of a tool also avialable on
> *BSD systems.

Fair enough then, that's fine with me.

> The result is:
> 
> ---8<---
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
>   CHK     include/generated/timestamp_autogenerated.h
>   UPD     include/generated/timestamp_autogenerated.h
> make[1]: Leaving directory '/tmp/picosam'
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="0" ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
> /home/abiessmann/src/u-boot/Makefile:1304: *** "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!".  Stop.
> make[1]: Leaving directory '/tmp/picosam'
> Makefile:146: recipe for target 'sub-make' failed
> make: *** [sub-make] Error 2
> --->8---
> 
> It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).
> 
> Changes in v2:
> * check for '-u' and '-d "@0"' switch rather than for the GNU variant
> 
>  Makefile | 15 ++++++++++++---
>  1 file changed, 12 insertions(+), 3 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index b9b2375..b797e38 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -346,6 +346,10 @@ PERL		= perl
>  PYTHON		= python
>  DTC		= dtc
>  CHECK		= sparse
> +DATE	       := $(foreach date,gdate date.gnu date, \
> +		    $(shell _date=`which $(date)`; \
> +		      $${_date} -u -d "@0" >/dev/null 2>&1; \
> +		      test $$? -eq 0 && echo $${_date}))

First, I don't understand why you need to call date with the full path:
if which can find it, then calling the binary without its full path
should do just as well, right?

Then, correct me if I'm wrong, but calling test and using && is
overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 &&
echo $${_date}

So in the end, the whole line would look like:

DATE	       := $(foreach date,gdate date.gnu date, \
		    $($${date} -u -d "@0" >/dev/null 2>&1 \
		      && echo $${date}))

Let me know what you think (and please test it as well).

>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
>  		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
> @@ -1281,9 +1285,9 @@ endef
>  define filechk_timestamp.h
>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>  	else \
>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> @@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
>  	$(call filechk,version.h)
>  
>  $(timestamp_h): $(srctree)/Makefile FORCE
> +ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
> +ifeq ($(strip $(DATE)),)
> +	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
> +endif
> +endif
>  	$(call filechk,timestamp.h)
>  
>  # ---------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/06986ace/attachment.sig>

[U-Boot] [RFC PATCH v2] Makefile: search for GNU date

[Andreas Bießmann]

On 08/27/2015 03:03 PM, Paul Kocialkowski wrote:
> Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :


>> Changes in v2:
>> * check for '-u' and '-d "@0"' switch rather than for the GNU variant
>>
>>  Makefile | 15 ++++++++++++---
>>  1 file changed, 12 insertions(+), 3 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index b9b2375..b797e38 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -346,6 +346,10 @@ PERL		= perl
>>  PYTHON		= python
>>  DTC		= dtc
>>  CHECK		= sparse
>> +DATE	       := $(foreach date,gdate date.gnu date, \
>> +		    $(shell _date=`which $(date)`; \
>> +		      $${_date} -u -d "@0" >/dev/null 2>&1; \
>> +		      test $$? -eq 0 && echo $${_date}))
> 
> First, I don't understand why you need to call date with the full path:
> if which can find it, then calling the binary without its full path
> should do just as well, right?

You'r right.

> Then, correct me if I'm wrong, but calling test and using && is
> overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 &&
> echo $${_date}

Also true.

> So in the end, the whole line would look like:
> 
> DATE	       := $(foreach date,gdate date.gnu date, \
> 		    $($${date} -u -d "@0" >/dev/null 2>&1 \
> 		      && echo $${date}))
> 
> Let me know what you think (and please test it as well).

That should work. I wonder however why we don't include this snippet in
the filechk_timestamp.h script directly. In fact the $(DATE) (a date
that support -d '@0' switch) is just used there, so why clobber the
Makefile with it?

---8<---
define filechk_timestamp.h
 (if test -n "$${SOURCE_DATE_EPOCH}"; then \
   date=""; \
   for _date in gdate date.gnu date; do \
   $${_date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && \
      date="$${_date}"; \
   done; \
   if test -n "$${date}"; then \
    SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
   else \
    return 42; \
   fi; \
  else \
   LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
   LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
   LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
  fi)
endef
--->8---


>>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
>>  		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
>> @@ -1281,9 +1285,9 @@ endef
>>  define filechk_timestamp.h
>>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
>>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
>> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
>> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
>> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
>> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
>> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>>  	else \
>>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>> @@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
>>  	$(call filechk,version.h)
>>  
>>  $(timestamp_h): $(srctree)/Makefile FORCE
>> +ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
>> +ifeq ($(strip $(DATE)),)
>> +	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
>> +endif
>> +endif
>>  	$(call filechk,timestamp.h)
>>  
>>  # ---------------------------------------------------------------------------
> 

[U-Boot] [RFC PATCH v2] Makefile: search for GNU date

[Paul Kocialkowski]

Le jeudi 27 ao?t 2015 ? 15:52 +0200, Andreas Bie?mann a ?crit :
> On 08/27/2015 03:03 PM, Paul Kocialkowski wrote:
> > Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :
> 
> 
> >> Changes in v2:
> >> * check for '-u' and '-d "@0"' switch rather than for the GNU variant
> >>
> >>  Makefile | 15 ++++++++++++---
> >>  1 file changed, 12 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/Makefile b/Makefile
> >> index b9b2375..b797e38 100644
> >> --- a/Makefile
> >> +++ b/Makefile
> >> @@ -346,6 +346,10 @@ PERL		= perl
> >>  PYTHON		= python
> >>  DTC		= dtc
> >>  CHECK		= sparse
> >> +DATE	       := $(foreach date,gdate date.gnu date, \
> >> +		    $(shell _date=`which $(date)`; \
> >> +		      $${_date} -u -d "@0" >/dev/null 2>&1; \
> >> +		      test $$? -eq 0 && echo $${_date}))
> > 
> > First, I don't understand why you need to call date with the full path:
> > if which can find it, then calling the binary without its full path
> > should do just as well, right?
> 
> You'r right.
> 
> > Then, correct me if I'm wrong, but calling test and using && is
> > overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 &&
> > echo $${_date}
> 
> Also true.
> 
> > So in the end, the whole line would look like:
> > 
> > DATE	       := $(foreach date,gdate date.gnu date, \
> > 		    $($${date} -u -d "@0" >/dev/null 2>&1 \
> > 		      && echo $${date}))
> > 
> > Let me know what you think (and please test it as well).
> 
> That should work. I wonder however why we don't include this snippet in
> the filechk_timestamp.h script directly. In fact the $(DATE) (a date
> that support -d '@0' switch) is just used there, so why clobber the
> Makefile with it?

I'm fine with both options, but your suggestion also makes it more
obvious why we're doing that check at all.

> ---8<---
> define filechk_timestamp.h
>  (if test -n "$${SOURCE_DATE_EPOCH}"; then \
>    date=""; \
>    for _date in gdate date.gnu date; do \
>    $${_date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && \
>       date="$${_date}"; \
>    done; \
>    if test -n "$${date}"; then \
>     SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
>     LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
> %d %C%y"'; \
>     LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
>     LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
>    else \
>     return 42; \
>    fi; \
>   else \
>    LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>    LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
>    LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
>   fi)
> endef
> --->8---
> 
> 
> >>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
> >>  		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
> >> @@ -1281,9 +1285,9 @@ endef
> >>  define filechk_timestamp.h
> >>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
> >>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> >> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> >> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> >> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> >> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> >> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> >> +		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> >>  	else \
> >>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
> >>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
> >> @@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
> >>  	$(call filechk,version.h)
> >>  
> >>  $(timestamp_h): $(srctree)/Makefile FORCE
> >> +ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
> >> +ifeq ($(strip $(DATE)),)
> >> +	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
> >> +endif
> >> +endif
> >>  	$(call filechk,timestamp.h)
> >>  
> >>  # ---------------------------------------------------------------------------
> > 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/bc4b8558/attachment.sig>

[U-Boot] [PATCH v3] Makefile: fix SOURCE_DATE_EPOCH for *BSD host

[Andreas Bießmann]

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1)
with -d switch to print the relevant date and time strings of another point of
time.

In other words it requires some date(1) that behaves like the GNU date(1) [1].
The BSD date(1) [2] on the other hand has the same switch but with a different
meaning.

Respect this and check the date(1) abilities before usage, error on non
working version.  Use the well known pre- and suffixes for the GNU variant of
a tool on *BSD hosts to search for a working date(1) version.

[1] http://man7.org/linux/man-pages/man1/date.1.html [2]
http://www.freebsd.org/cgi/man.cgi?query=date

Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
---
This commit tries to figure out if we have a date variant
available that supports the '-u' and '-d "@0"' switches.
It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and
suffixes are widespread used for the GNU variant of a tool also avialable on
*BSD systems.

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
  CHK     include/generated/timestamp_autogenerated.h
  UPD     include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="$(date +'%s')" ARCH=arm include/generated/timestamp_autogenerated.h
make[1]: Entering directory '/tmp/picosam'
  CHK     include/generated/timestamp_autogenerated.h
/home/abiessmann/src/u-boot/Makefile:1311: recipe for target 'include/generated/timestamp_autogenerated.h' failed
make[1]: *** [include/generated/timestamp_autogenerated.h] Error 42
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v3:
 * move the check in the timestamp.h generation script
 * remove RFC
 * reword commit message

Changes in v2:
 * check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index e8ea71c..81f12b8 100644
--- a/Makefile
+++ b/Makefile
@@ -1279,12 +1279,24 @@ define filechk_version.h
 	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
 endef
 
+# The SOURCE_DATE_EPOCH mechanism requires a date that behaves like GNU date.
+# The BSD date on the other hand behaves different and would produce errors
+# with the misused '-d' switch.  Respect that and search a working date with
+# well known pre- and suffixes for the GNU variant of date.
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		DATE=""; \
+		for date in gdate date.gnu date; do \
+			$${date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && DATE="$${date}"; \
+		done; \
+		if test -n "$${DATE}"; then \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		else \
+			return 42; \
+		fi; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
-- 
2.1.4

[U-Boot] [U-Boot, v3] Makefile: fix SOURCE_DATE_EPOCH for *BSD host

[Tom Rini]

On Fri, Aug 28, 2015 at 10:29:55AM +0200, Andreas Bie?mann wrote:

> The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1)
> with -d switch to print the relevant date and time strings of another point of
> time.
> 
> In other words it requires some date(1) that behaves like the GNU date(1) [1].
> The BSD date(1) [2] on the other hand has the same switch but with a different
> meaning.
> 
> Respect this and check the date(1) abilities before usage, error on non
> working version.  Use the well known pre- and suffixes for the GNU variant of
> a tool on *BSD hosts to search for a working date(1) version.
> 
> [1] http://man7.org/linux/man-pages/man1/date.1.html [2]
> http://www.freebsd.org/cgi/man.cgi?query=date
> 
> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150828/4758b9aa/attachment.sig>

[U-Boot] [U-Boot, v3] Makefile: fix SOURCE_DATE_EPOCH for *BSD host

[Holger Levsen]

Hi,

On Freitag, 28. August 2015, Tom Rini wrote:
> Applied to u-boot/master, thanks!

yay, glad to see this thread come to a happy ending, both nicely and quickly! 
;-) Thanks to everyone involved!


cheers,
	Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150828/1aa06476/attachment.sig>

[U-Boot] [PATCH v3] Makefile: fix SOURCE_DATE_EPOCH for *BSD host

[Paul Kocialkowski]

Le vendredi 28 ao?t 2015 ? 10:29 +0200, Andreas Bie?mann a ?crit :
> The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1)
> with -d switch to print the relevant date and time strings of another point of
> time.
> 
> In other words it requires some date(1) that behaves like the GNU date(1) [1].
> The BSD date(1) [2] on the other hand has the same switch but with a different
> meaning.
> 
> Respect this and check the date(1) abilities before usage, error on non
> working version.  Use the well known pre- and suffixes for the GNU variant of
> a tool on *BSD hosts to search for a working date(1) version.

Looks good to me! Have you actually tried setting SOURCE_DATE_EPOCH to a
fixed value, building U-Boot twice and checking that the produced file
is the very same?

> [1] http://man7.org/linux/man-pages/man1/date.1.html [2]
> http://www.freebsd.org/cgi/man.cgi?query=date
> 
> Signed-off-by: Andreas Bie?mann <andreas.devel@googlemail.com>
> ---
> This commit tries to figure out if we have a date variant
> available that supports the '-u' and '-d "@0"' switches.
> It errors on non-working variants of date.
> 
> To respect *BSD host systems search for gdate and date.gnu. Those pre- and
> suffixes are widespread used for the GNU variant of a tool also avialable on
> *BSD systems.
> 
> The result is:
> 
> ---8<---
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
>   CHK     include/generated/timestamp_autogenerated.h
>   UPD     include/generated/timestamp_autogenerated.h
> make[1]: Leaving directory '/tmp/picosam'
> abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="$(date +'%s')" ARCH=arm include/generated/timestamp_autogenerated.h
> make[1]: Entering directory '/tmp/picosam'
>   CHK     include/generated/timestamp_autogenerated.h
> /home/abiessmann/src/u-boot/Makefile:1311: recipe for target 'include/generated/timestamp_autogenerated.h' failed
> make[1]: *** [include/generated/timestamp_autogenerated.h] Error 42
> make[1]: Leaving directory '/tmp/picosam'
> Makefile:146: recipe for target 'sub-make' failed
> make: *** [sub-make] Error 2
> --->8---
> 
> It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).
> 
> Changes in v3:
>  * move the check in the timestamp.h generation script
>  * remove RFC
>  * reword commit message
> 
> Changes in v2:
>  * check for '-u' and '-d "@0"' switch rather than for the GNU variant
> 
>  Makefile | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index e8ea71c..81f12b8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1279,12 +1279,24 @@ define filechk_version.h
>  	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
>  endef
>  
> +# The SOURCE_DATE_EPOCH mechanism requires a date that behaves like GNU date.
> +# The BSD date on the other hand behaves different and would produce errors
> +# with the misused '-d' switch.  Respect that and search a working date with
> +# well known pre- and suffixes for the GNU variant of date.
>  define filechk_timestamp.h
>  	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
>  		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> -		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +		DATE=""; \
> +		for date in gdate date.gnu date; do \
> +			$${date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && DATE="$${date}"; \
> +		done; \
> +		if test -n "$${DATE}"; then \
> +			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
> +			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
> +			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
> +		else \
> +			return 42; \
> +		fi; \
>  	else \
>  		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
>  		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150901/f5fbd352/attachment.sig>

[U-Boot] [PATCH v3] Makefile: fix SOURCE_DATE_EPOCH for *BSD host

[Andreas Bießmann]

On 09/01/2015 07:03 PM, Paul Kocialkowski wrote:
> Le vendredi 28 ao?t 2015 ? 10:29 +0200, Andreas Bie?mann a ?crit :
>> The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1)
>> with -d switch to print the relevant date and time strings of another point of
>> time.
>>
>> In other words it requires some date(1) that behaves like the GNU date(1) [1].
>> The BSD date(1) [2] on the other hand has the same switch but with a different
>> meaning.
>>
>> Respect this and check the date(1) abilities before usage, error on non
>> working version.  Use the well known pre- and suffixes for the GNU variant of
>> a tool on *BSD hosts to search for a working date(1) version.
> 
> Looks good to me! Have you actually tried setting SOURCE_DATE_EPOCH to a
> fixed value, building U-Boot twice and checking that the produced file
> is the very same?

It works:

---8<---
andreas at imac % shasum /tmp/u-boot.img.1441178617 u-boot.img
/tmp/u-boot.bin.1441178617 u-boot.bin /tmp/boot.bin.1441178617 boot.bin
1a779ba79efa1874a5b307650392737d861005bb  /tmp/u-boot.img.1441178617
1a779ba79efa1874a5b307650392737d861005bb  u-boot.img
dbe56f02e510ee251a2e5f9f8b6ce430884557bb  /tmp/u-boot.bin.1441178617
dbe56f02e510ee251a2e5f9f8b6ce430884557bb  u-boot.bin
4aa9b3ba9641febae0bfdbb374c54c287b463ac0  /tmp/boot.bin.1441178617
4aa9b3ba9641febae0bfdbb374c54c287b463ac0  boot.bin
andreas at imac % uname -a
Darwin imac 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT
2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64
andreas@imac % git describe
v2015.10-rc2-304-gb7e84c9
--->8---

Both are fresh builds with the same SOURCE_DATE_EPOCH set. However using
the time stamp from timestamp_autogenerated.h does _not_ work due to the
wrong TZ (+0200 for me, UTC with SOURCE_DATE_EPOCH). But I guess this is
a known problem.

Andreas

[U-Boot] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Vagrant Cascadian]

On 2015-07-26, Paul Kocialkowski wrote:
> In order to achieve reproducible builds in U-Boot, timestamps that are defined
> at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
> variable allows setting a fixed value for those timestamps.
...
> However, some other devices might need some more tweaks, especially regarding
> the image generation tools.

With this patch, there is still variation based on timezone in any of
the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian
packages:

  https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html

The good news is that all the u-boot.bin targets are produced
reproducibly, so here's to progress!


I think the use of "time = mktime(time_universal);" is where the problem
lies:

> diff --git a/tools/default_image.c b/tools/default_image.c
> index cf5c0d4..18940af 100644
> --- a/tools/default_image.c
> +++ b/tools/default_image.c
> @@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
>  				sizeof(image_header_t)),
>  			sbuf->st_size - sizeof(image_header_t));
>  
> +	source_date_epoch = getenv("SOURCE_DATE_EPOCH");
> +	if (source_date_epoch != NULL) {
> +		time = (time_t) strtol(source_date_epoch, NULL, 10);
> +
> +		time_universal = gmtime(&time);
> +		if (time_universal == NULL) {
> +			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
> +				__func__);
> +			time = 0;
> +		} else {
> +			time = mktime(time_universal);
> +		}
> +	} else {
> +		time = sbuf->st_mtime;
> +	}
> +
>  	/* Build new header */
>  	image_set_magic(hdr, IH_MAGIC);
> -	image_set_time(hdr, sbuf->st_mtime);
> +	image_set_time(hdr, time);
>  	image_set_size(hdr, sbuf->st_size - sizeof(image_header_t));
>  	image_set_load(hdr, params->addr);
>  	image_set_ep(hdr, params->ep);
> -- 
> 1.9.1

According to the mktime manpage:

       The  mktime()  function converts a broken-down time structure,
       expressed as local time, to calendar time representation.  

So my interpetation is that it's taking the UTC time and converts it
into local time using the configured timezone... not sure what would be
a viable alternative to mktime.

Running with the TZ=UTC environment variable exported works around the
problem; not sure if it would be appropriate to always run with TZ=UTC
when SOURCE_DATE_EPOCH is set...


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150924/4e92e27e/attachment.sig>

[U-Boot] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Paul Kocialkowski]

Le jeudi 24 septembre 2015 ? 09:05 -0700, Vagrant Cascadian a ?crit :
> On 2015-07-26, Paul Kocialkowski wrote:
> > In order to achieve reproducible builds in U-Boot, timestamps that are defined
> > at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
> > variable allows setting a fixed value for those timestamps.
> ...
> > However, some other devices might need some more tweaks, especially regarding
> > the image generation tools.
> 
> With this patch, there is still variation based on timezone in any of
> the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian
> packages:
> 
>   https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html

Thanks for reporting this!

> The good news is that all the u-boot.bin targets are produced
> reproducibly, so here's to progress!

Good, that's a nice first step forward.

> I think the use of "time = mktime(time_universal);" is where the problem
> lies:

[?]

> According to the mktime manpage:
> 
>        The  mktime()  function converts a broken-down time structure,
>        expressed as local time, to calendar time representation.  
> 
> So my interpetation is that it's taking the UTC time and converts it
> into local time using the configured timezone... not sure what would be
> a viable alternative to mktime.

That seems to make sense. Come to think of it, it probably was not
necessary to call gmtime in the first place: if SOURCE_DATE_EPOCH is
always in UTC, we should be able to stick that as-is in the time
variable. At best, gmtime + mktime (assuming mktime working in UTC)
would give us back the same timestamp.

What do you think? Please let me know if I'm wrong.

> Running with the TZ=UTC environment variable exported works around the
> problem; not sure if it would be appropriate to always run with TZ=UTC
> when SOURCE_DATE_EPOCH is set...

Well that's too much of a workaround to be a reliable solution for the
long term, IMHO.

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: https://www.replicant.us/
Blog: https://blog.replicant.us/
Wiki/tracker/forums: https://redmine.replicant.us/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150928/b0842d2c/attachment.sig>

[U-Boot] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Siarhei Siamashka]

On Thu, 24 Sep 2015 09:05:10 -0700
Vagrant Cascadian <vagrant@debian.org> wrote:

> On 2015-07-26, Paul Kocialkowski wrote:
> > In order to achieve reproducible builds in U-Boot, timestamps that are defined
> > at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment
> > variable allows setting a fixed value for those timestamps.
> ...
> > However, some other devices might need some more tweaks, especially regarding
> > the image generation tools.
> 
> With this patch, there is still variation based on timezone in any of
> the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian
> packages:
> 
>   https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html
> 
> The good news is that all the u-boot.bin targets are produced
> reproducibly, so here's to progress!

Hello,

I see that Debian is building U-Boot 2015.07 according to the
information from that link. It may be a good idea to ensure that
the following simple patch is also applied:
    http://git.denx.de/?p=u-boot.git;a=commitdiff;h=bfb05d0187d70274c77d02dc0de5e728e1f8be05

It had been submitted at the day when somebody (probably you) was
troubleshooting reproducible build issues and asking questions in
the U-Boot IRC channel.

-- 
Best regards,
Siarhei Siamashka

[U-Boot] [Reproducible-builds] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Vagrant Cascadian]

On 2015-09-28, Paul Kocialkowski wrote:
> Le jeudi 24 septembre 2015 ? 09:05 -0700, Vagrant Cascadian a ?crit :
>> I think the use of "time = mktime(time_universal);" is where the problem
>> lies:
>
> [?]
>
>> According to the mktime manpage:
>> 
>>        The  mktime()  function converts a broken-down time structure,
>>        expressed as local time, to calendar time representation.  
>> 
>> So my interpetation is that it's taking the UTC time and converts it
>> into local time using the configured timezone... not sure what would be
>> a viable alternative to mktime.
>
> That seems to make sense. Come to think of it, it probably was not
> necessary to call gmtime in the first place: if SOURCE_DATE_EPOCH is
> always in UTC, we should be able to stick that as-is in the time
> variable. At best, gmtime + mktime (assuming mktime working in UTC)
> would give us back the same timestamp.
>
> What do you think? Please let me know if I'm wrong.

This patch on top of 2015.10-rc4 seems to resolve the issue for me:

Index: u-boot/tools/default_image.c
===================================================================
--- u-boot.orig/tools/default_image.c
+++ u-boot/tools/default_image.c
@@ -108,8 +108,6 @@ static void image_set_header(void *ptr,
 			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
 				__func__);
 			time = 0;
-		} else {
-			time = mktime(time_universal);
 		}
 	} else {
 		time = sbuf->st_mtime;


It still checks for the validity of SOURCE_DATE_EPOCH using gmtime, but
doesn't call mktime at all, just re-uses the value set from
SOURCE_DATE_EPOCH.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150930/a5900ac9/attachment.sig>

[U-Boot] [Reproducible-builds] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

[Paul Kocialkowski]

Le mercredi 30 septembre 2015 ? 08:50 -0700, Vagrant Cascadian a ?crit :
> On 2015-09-28, Paul Kocialkowski wrote:
> > What do you think? Please let me know if I'm wrong.
> 
> This patch on top of 2015.10-rc4 seems to resolve the issue for me:
> 
> Index: u-boot/tools/default_image.c
> ===================================================================
> --- u-boot.orig/tools/default_image.c
> +++ u-boot/tools/default_image.c
> @@ -108,8 +108,6 @@ static void image_set_header(void *ptr,
>  			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
>  				__func__);
>  			time = 0;
> -		} else {
> -			time = mktime(time_universal);
>  		}
>  	} else {
>  		time = sbuf->st_mtime;
> 
> 
> It still checks for the validity of SOURCE_DATE_EPOCH using gmtime, but
> doesn't call mktime at all, just re-uses the value set from
> SOURCE_DATE_EPOCH.

That's a good plan! I guess we should also fully get rid of the
time_universal variable and make the check inline:

if (gmtime(&time) == NULL)

and of course drop the else statement.

Would you like to craft that patch for upstream U-Boot?
If not, I'd be happy to do it.

Thanks for your work!

-- 
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: https://www.replicant.us/
Blog: https://blog.replicant.us/
Wiki/tracker/forums: https://redmine.replicant.us/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151002/e45fb4e0/attachment.sig>