[PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Alex Kiernan]

When REPRODUCIBLE_TIMESTAMP_ROOTFS is unset and we want to parse one
from git, use COREBASE as the base for the git command so we have a
known repository which we're using. Without this the build may fail
if the current directory is not part of a git repository.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
---

 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8ef06c0..2e88024 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -651,7 +651,7 @@ POPULATE_SDK_PRE_TARGET_COMMAND += "${@bb.utils.contains('DISTRO_FEATURES', 'usr
 reproducible_final_image_task () {
     if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
         if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
-            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git log -1 --pretty=%ct`
+            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct`
         fi
         # Set mtime of all files to a reproducible value
         bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
-- 
2.7.4

[PATCH 2/2] kernel: Set SOURCE_DATE_EPOCH to kernel git timestamp if not set

[Alex Kiernan]

If SOURCE_DATE_EPOCH is unset (in addition to the existing "0" behaviour)
parse out the top most commit timestamp from the kernel tree to use as the
timestamp.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
---

 meta/classes/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 972ac52..3213b93 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -288,7 +288,7 @@ kernel_do_compile() {
 	if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
 		# kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
 		# be set....
-		if [ "$SOURCE_DATE_EPOCH" = "0" ]; then
+		if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
 			olddir=`pwd`
 			cd ${S}
 			SOURCE_DATE_EPOCH=`git log  -1 --pretty=%ct`
-- 
2.7.4

Re: [PATCH 2/2] kernel: Set SOURCE_DATE_EPOCH to kernel git timestamp if not set

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Thanks, this seems to be fixing the issue I've reported long time ago:
http://lists.openembedded.org/pipermail/openembedded-commits/2018-March/220579.html



On Tue, Jun 26, 2018 at 2:10 PM Alex Kiernan <alex.kiernan@gmail.com> wrote:

> If SOURCE_DATE_EPOCH is unset (in addition to the existing "0" behaviour)
> parse out the top most commit timestamp from the kernel tree to use as the
> timestamp.
>
> Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> ---
>
>  meta/classes/kernel.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 972ac52..3213b93 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -288,7 +288,7 @@ kernel_do_compile() {
>         if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
>                 # kernel sources do not use do_unpack, so
> SOURCE_DATE_EPOCH may not
>                 # be set....
> -               if [ "$SOURCE_DATE_EPOCH" = "0" ]; then
> +               if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}"
> = "0" ]; then
>                         olddir=`pwd`
>                         cd ${S}
>                         SOURCE_DATE_EPOCH=`git log  -1 --pretty=%ct`
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 2370 bytes --]

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Peter Kjellerstedt]

There is no guarantee that ${COREBASE} refers to a path that is under 
Git control. E.g., in our case it refers to a directory that is under 
repo control. "${COREBASE}/meta" is probably a better option.

//Peter

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of
> Alex Kiernan
> Sent: den 26 juni 2018 14:09
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [OE-Core][PATCH 1/2] image: Set COREBASE as the git
> directory for timestamp
> 
> When REPRODUCIBLE_TIMESTAMP_ROOTFS is unset and we want to parse one
> from git, use COREBASE as the base for the git command so we have a
> known repository which we're using. Without this the build may fail
> if the current directory is not part of a git repository.
> 
> Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> ---
> 
>  meta/classes/image.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 8ef06c0..2e88024 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -651,7 +651,7 @@ POPULATE_SDK_PRE_TARGET_COMMAND +=
> "${@bb.utils.contains('DISTRO_FEATURES', 'usr
>  reproducible_final_image_task () {
>      if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
>          if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> -            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git log -1 --pretty=%ct`
> +            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1
> --pretty=%ct`
>          fi
>          # Set mtime of all files to a reproducible value
>          bbnote "reproducible_final_image_task: mtime set to
> $REPRODUCIBLE_TIMESTAMP_ROOTFS"
> --
> 2.7.4
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Richard Purdie]

On Thu, 2018-06-28 at 15:18 +0000, Peter Kjellerstedt wrote:
> There is no guarantee that ${COREBASE} refers to a path that is
> under Git control. E.g., in our case it refers to a directory that is
> under repo control. "${COREBASE}/meta" is probably a better option.

You could make that argument about /meta too.

I'd suggest the code checks for ${COREBASE}/.git and if that doesn't
exist, fall back to the timestamp of the
${COREBASE}/meta/conf/bitbake.conf file.

Cheers,

Richard

> //Peter
> 
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org
> > [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf
> > Of
> > Alex Kiernan
> > Sent: den 26 juni 2018 14:09
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [OE-Core][PATCH 1/2] image: Set COREBASE as the
> > git
> > directory for timestamp
> > 
> > When REPRODUCIBLE_TIMESTAMP_ROOTFS is unset and we want to parse
> > one
> > from git, use COREBASE as the base for the git command so we have a
> > known repository which we're using. Without this the build may fail
> > if the current directory is not part of a git repository.
> > 
> > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > ---
> > 
> >  meta/classes/image.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass
> > index 8ef06c0..2e88024 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -651,7 +651,7 @@ POPULATE_SDK_PRE_TARGET_COMMAND +=
> > "${@bb.utils.contains('DISTRO_FEATURES', 'usr
> >  reproducible_final_image_task () {
> >      if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> >          if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> > -            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git log -1 --
> > pretty=%ct`
> > +            REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}"
> > log -1
> > --pretty=%ct`
> >          fi
> >          # Set mtime of all files to a reproducible value
> >          bbnote "reproducible_final_image_task: mtime set to
> > $REPRODUCIBLE_TIMESTAMP_ROOTFS"
> > --
> > 2.7.4
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Thu, Jun 28, 2018 at 8:29 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2018-06-28 at 15:18 +0000, Peter Kjellerstedt wrote:
> > There is no guarantee that ${COREBASE} refers to a path that is
> > under Git control. E.g., in our case it refers to a directory that is
> > under repo control. "${COREBASE}/meta" is probably a better option.
>
> You could make that argument about /meta too.
>
> I'd suggest the code checks for ${COREBASE}/.git and if that doesn't
> exist, fall back to the timestamp of the
> ${COREBASE}/meta/conf/bitbake.conf file.
>

That's a good idea, but I'd suggest using bb.utils.which() to locate
conf/bibake.conf rather than hardcoding that path.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1330 bytes --]

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Alex Kiernan]

On Thu, Jun 28, 2018 at 4:31 PM Christopher Larson <kergoth@gmail.com> wrote:
>
>
>
> On Thu, Jun 28, 2018 at 8:29 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>>
>> On Thu, 2018-06-28 at 15:18 +0000, Peter Kjellerstedt wrote:
>> > There is no guarantee that ${COREBASE} refers to a path that is
>> > under Git control. E.g., in our case it refers to a directory that is
>> > under repo control. "${COREBASE}/meta" is probably a better option.
>>
>> You could make that argument about /meta too.
>>
>> I'd suggest the code checks for ${COREBASE}/.git and if that doesn't

In a poky checkout from repo (which is what I'm also using),
${COREBASE}/meta/.git doesn't exist, but ${COREBASE}/.git does, but
that all works because it's climbing the directory tree to find the
.git directory.

>> exist, fall back to the timestamp of the
>> ${COREBASE}/meta/conf/bitbake.conf file.
>
>
> That's a good idea, but I'd suggest using bb.utils.which() to locate conf/bibake.conf rather than hardcoding that path.

How about (totally untested):

if git -C "${COREBASE}/meta" rev-parse --show-toplevel; then
  REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}/meta" log -1 --pretty=%ct`
else
  REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(bbpath,
"conf/bitbake.conf")}`
fi

?

--
Alex Kiernan

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Peter Kjellerstedt]

> -----Original Message-----
> From: Alex Kiernan [mailto:alex.kiernan@gmail.com]
> Sent: den 28 juni 2018 17:57
> To: kergoth@gmail.com
> Cc: richard.purdie@linuxfoundation.org; Peter Kjellerstedt
> <peter.kjellerstedt@axis.com>; openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [OE-Core][PATCH 1/2] image: Set COREBASE as the
> git directory for timestamp
> 
> On Thu, Jun 28, 2018 at 4:31 PM Christopher Larson <kergoth@gmail.com>
> wrote:
> >
> > On Thu, Jun 28, 2018 at 8:29 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >>
> >> On Thu, 2018-06-28 at 15:18 +0000, Peter Kjellerstedt wrote:
> >> > There is no guarantee that ${COREBASE} refers to a path that is
> >> > under Git control. E.g., in our case it refers to a directory that
> >> > is under repo control. "${COREBASE}/meta" is probably a better
> >> > option.
> >>
> >> You could make that argument about /meta too.
> >>
> >> I'd suggest the code checks for ${COREBASE}/.git and if that doesn't
> 
> In a poky checkout from repo (which is what I'm also using),
> ${COREBASE}/meta/.git doesn't exist, but ${COREBASE}/.git does, but
> that all works because it's climbing the directory tree to find the
> .git directory.
> 
> >> exist, fall back to the timestamp of the
> >> ${COREBASE}/meta/conf/bitbake.conf file.
> >
> > That's a good idea, but I'd suggest using bb.utils.which() to locate
> > conf/bibake.conf rather than hardcoding that path.
> 
> How about (totally untested):
> 
> if git -C "${COREBASE}/meta" rev-parse --show-toplevel; then
>   REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}/meta" log -1 --pretty=%ct`
> else
>   REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(bbpath, "conf/bitbake.conf")}`
> fi
> 
> ?

You do not really need the first git call above:

REPRODUCIBLE_TIMESTAMP_ROOTFS=$(git -C "${COREBASE}/meta" log -1 --pretty=%ct 2>/dev/null)
if [ -z "$REPRODUCIBLE_TIMESTAMP_ROOTFS" ]; then
	REPRODUCIBLE_TIMESTAMP_ROOTFS=$(stat -c%Y ${@bb.utils.which(bbpath, "conf/bitbake.conf")})
fi

> --
> Alex Kiernan

//Peter

Re: [PATCH 1/2] image: Set COREBASE as the git directory for timestamp

[Alex Kiernan]

On Thu, Jun 28, 2018 at 7:34 PM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> > -----Original Message-----
> > From: Alex Kiernan [mailto:alex.kiernan@gmail.com]
> > Sent: den 28 juni 2018 17:57
> > To: kergoth@gmail.com
> > Cc: richard.purdie@linuxfoundation.org; Peter Kjellerstedt
> > <peter.kjellerstedt@axis.com>; openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [OE-Core][PATCH 1/2] image: Set COREBASE as the
> > git directory for timestamp
> >
> > On Thu, Jun 28, 2018 at 4:31 PM Christopher Larson <kergoth@gmail.com>
> > wrote:
> > >
> > > On Thu, Jun 28, 2018 at 8:29 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > >>
> > >> On Thu, 2018-06-28 at 15:18 +0000, Peter Kjellerstedt wrote:
> > >> > There is no guarantee that ${COREBASE} refers to a path that is
> > >> > under Git control. E.g., in our case it refers to a directory that
> > >> > is under repo control. "${COREBASE}/meta" is probably a better
> > >> > option.
> > >>
> > >> You could make that argument about /meta too.
> > >>
> > >> I'd suggest the code checks for ${COREBASE}/.git and if that doesn't
> >
> > In a poky checkout from repo (which is what I'm also using),
> > ${COREBASE}/meta/.git doesn't exist, but ${COREBASE}/.git does, but
> > that all works because it's climbing the directory tree to find the
> > .git directory.
> >
> > >> exist, fall back to the timestamp of the
> > >> ${COREBASE}/meta/conf/bitbake.conf file.
> > >
> > > That's a good idea, but I'd suggest using bb.utils.which() to locate
> > > conf/bibake.conf rather than hardcoding that path.
> >
> > How about (totally untested):
> >
> > if git -C "${COREBASE}/meta" rev-parse --show-toplevel; then
> >   REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}/meta" log -1 --pretty=%ct`
> > else
> >   REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(bbpath, "conf/bitbake.conf")}`
> > fi
> >
> > ?
>
> You do not really need the first git call above:
>
> REPRODUCIBLE_TIMESTAMP_ROOTFS=$(git -C "${COREBASE}/meta" log -1 --pretty=%ct 2>/dev/null)
> if [ -z "$REPRODUCIBLE_TIMESTAMP_ROOTFS" ]; then
>         REPRODUCIBLE_TIMESTAMP_ROOTFS=$(stat -c%Y ${@bb.utils.which(bbpath, "conf/bitbake.conf")})
> fi
>

That's true... I'll post an update

--
Alex Kiernan