[PATCH] bitbake.conf: Add comm to HOSTTOOLS

[PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Niko Mauno]

This mitigates following issues during u-boot do_compile() step --
otherwise, if comm is not available, they are quietly ignored:

  .../scripts/check-config.sh: line 33: comm: command not found
  .../scripts/check-config.sh: line 39: comm: command not found

Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS
was considered a lower impact fix compared to adding coreutils-native
buildtime dependency to u-boot recipe.

Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 93afb13166..e1d2f3c3e2 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -473,7 +473,7 @@ HOSTTOOLS_DIR = "${TMPDIR}/hosttools"
 
 # Tools needed to run builds with OE-Core
 HOSTTOOLS += " \
-    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp cp cpio \
+    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \
     cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
     fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \
     head hostname id install ld ldd ln ls make makeinfo md5sum mkdir mknod \
-- 
2.11.0

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Manjukumar Harthikote Matha]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Niko
> Mauno
> Sent: Monday, January 29, 2018 9:47 AM
> To: openembedded-core@lists.openembedded.org
> Cc: Niko Mauno <niko.mauno@vaisala.com>
> Subject: [OE-core] [PATCH] bitbake.conf: Add comm to HOSTTOOLS
> 
> This mitigates following issues during u-boot do_compile() step -- otherwise, if
> comm is not available, they are quietly ignored:
> 
>   .../scripts/check-config.sh: line 33: comm: command not found
>   .../scripts/check-config.sh: line 39: comm: command not found
> 
> Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS was
> considered a lower impact fix compared to adding coreutils-native buildtime
> dependency to u-boot recipe.
> 

Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>

> Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
> ---
>  meta/conf/bitbake.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index
> 93afb13166..e1d2f3c3e2 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -473,7 +473,7 @@ HOSTTOOLS_DIR = "${TMPDIR}/hosttools"
> 
>  # Tools needed to run builds with OE-Core  HOSTTOOLS += " \
> -    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp cp cpio \
> +    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp
> + comm cp cpio \
>      cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
>      fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \
>      head hostname id install ld ldd ln ls make makeinfo md5sum mkdir mknod \
> --
> 2.11.0
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Alexander Kanavin]

On 01/29/2018 07:47 PM, Niko Mauno wrote:
> This mitigates following issues during u-boot do_compile() step --
> otherwise, if comm is not available, they are quietly ignored:
> 
>    .../scripts/check-config.sh: line 33: comm: command not found
>    .../scripts/check-config.sh: line 39: comm: command not found
> 
> Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS
> was considered a lower impact fix compared to adding coreutils-native
> buildtime dependency to u-boot recipe.

I disagree. Expanding HOSTTOOLS just to cater to the needs of a single 
recipe is not optimal, IMO.

Alex

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Khem Raj]

On 1/29/18 10:38 PM, Alexander Kanavin wrote:
> On 01/29/2018 07:47 PM, Niko Mauno wrote:
>> This mitigates following issues during u-boot do_compile() step --
>> otherwise, if comm is not available, they are quietly ignored:
>>
>>    .../scripts/check-config.sh: line 33: comm: command not found
>>    .../scripts/check-config.sh: line 39: comm: command not found
>>
>> Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS
>> was considered a lower impact fix compared to adding coreutils-native
>> buildtime dependency to u-boot recipe.
> 
> I disagree. Expanding HOSTTOOLS just to cater to the needs of a single 
> recipe is not optimal, IMO.

alternative are
build time dep on coreutils-native.
fix uboot script to not use comm
do not invoke check-config.sh at all.

none of them are any better.

> 
> Alex

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Alexander Kanavin]

On 01/30/2018 09:07 AM, Khem Raj wrote:

>> I disagree. Expanding HOSTTOOLS just to cater to the needs of a single 
>> recipe is not optimal, IMO.
> 
> alternative are
> build time dep on coreutils-native.
> fix uboot script to not use comm
> do not invoke check-config.sh at all.
> 
> none of them are any better.

Before we can declare that, we need to look what comm is actually used 
for during build, and why its absence is not a problem and doesn't cause 
errors.

Consider also that using comm from host may break reproducibility, if 
absence or presence of comm on the host results in a different output 
from the build.

Alex

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Manjukumar Harthikote Matha]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of
> Khem Raj
> Sent: Monday, January 29, 2018 11:07 PM
> To: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] bitbake.conf: Add comm to HOSTTOOLS
> 
> 
> 
> On 1/29/18 10:38 PM, Alexander Kanavin wrote:
> > On 01/29/2018 07:47 PM, Niko Mauno wrote:
> >> This mitigates following issues during u-boot do_compile() step --
> >> otherwise, if comm is not available, they are quietly ignored:
> >>
> >>    .../scripts/check-config.sh: line 33: comm: command not found
> >>    .../scripts/check-config.sh: line 39: comm: command not found
> >>
> >> Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS
> >> was considered a lower impact fix compared to adding coreutils-native
> >> buildtime dependency to u-boot recipe.
> >
> > I disagree. Expanding HOSTTOOLS just to cater to the needs of a single
> > recipe is not optimal, IMO.
>
> alternative are
> build time dep on coreutils-native.

This works, tested it however u-boot now will depend on coreutils-native.

> fix uboot script to not use comm
> do not invoke check-config.sh at all.
> 
This might not be an option for us :)

> none of them are any better.
> 

Agree with Khem here, if it is not in HOSTTOOLS, it will trickle down to coreutils-native till u-boot script gets patched.   

Thanks,
Manju

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Mauno Niko]

On 01/30/2018 04:20 PM, Alexander Kanavin wrote:
> On 01/30/2018 09:07 AM, Khem Raj wrote:
> 
>>> I disagree. Expanding HOSTTOOLS just to cater to the needs of a 
>>> single recipe is not optimal, IMO.
>>
>> alternative are
>> build time dep on coreutils-native.
>> fix uboot script to not use comm
>> do not invoke check-config.sh at all.
>>
>> none of them are any better.
> 
> Before we can declare that, we need to look what comm is actually used 
> for during build, and why its absence is not a problem and doesn't cause 
> errors.
> 
> Consider also that using comm from host may break reproducibility, if 
> absence or presence of comm on the host results in a different output 
> from the build.

My patch initially did add coreutils-native to u-boot DEPENDS.

Should I resubmit it? (did not apply due to mismatch in recipe filename (version))

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Alexander Kanavin]

On 02/01/2018 01:36 PM, Mauno Niko wrote:
>> Before we can declare that, we need to look what comm is actually used
>> for during build, and why its absence is not a problem and doesn't cause
>> errors.
>>
>> Consider also that using comm from host may break reproducibility, if
>> absence or presence of comm on the host results in a different output
>> from the build.
> 
> My patch initially did add coreutils-native to u-boot DEPENDS.
> 
> Should I resubmit it? (did not apply due to mismatch in recipe filename (version))

RP has already merged the patch you sent, I think :)

Still, in my opinion - yes, it's better to undo the HOSTTOOLS change, 
and add coreutils-native. But first, research a bit further for what 
purpose comm is used during build and why its absence doesn't break it.

Alex

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

On 1 February 2018 at 14:39, Alexander Kanavin <
alexander.kanavin@linux.intel.com> wrote:

> On 02/01/2018 01:36 PM, Mauno Niko wrote:
>
>> Before we can declare that, we need to look what comm is actually used
>>> for during build, and why its absence is not a problem and doesn't cause
>>> errors.
>>>
>>> Consider also that using comm from host may break reproducibility, if
>>> absence or presence of comm on the host results in a different output
>>> from the build.
>>>
>>
>> My patch initially did add coreutils-native to u-boot DEPENDS.
>>
>> Should I resubmit it? (did not apply due to mismatch in recipe filename
>> (version))
>>
>
> RP has already merged the patch you sent, I think :)
>
> Still, in my opinion - yes, it's better to undo the HOSTTOOLS change, and
> add coreutils-native. But first, research a bit further for what purpose
> comm is used during build and why its absence doesn't break it.
>

coreutils is on 100% of linux desktops (rounding up), and this would be the
only recipe to build-depend on coreutils-native.  I don't see the problem
with depending on the host for this.

Ross

[-- Attachment #2: Type: text/html, Size: 1755 bytes --]

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Luca Ceresoli]

Hi,

On 29/01/2018 18:47, Niko Mauno wrote:
> This mitigates following issues during u-boot do_compile() step --
> otherwise, if comm is not available, they are quietly ignored:
> 
>   .../scripts/check-config.sh: line 33: comm: command not found
>   .../scripts/check-config.sh: line 39: comm: command not found

In current U-Boot master (since [0]), the absence of 'comm' is not
silently ignored but rather causes a build failure.

Thus I think this patch should be cherry-picked on the stable yocto
branches. Otherwise people using a post-2018.3 U-Boot will see their
builds fail and will have to find out on their own how to fix it.

[0]
http://git.denx.de/?p=u-boot.git;a=commit;h=f69dce5081396a83b87d6df2693764b99466a18d

-- 
Luca

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Burton, Ross]

Please backport the patch to rocko and submit it to the list.

Ross

On 28 March 2018 at 15:55, Luca Ceresoli <luca@lucaceresoli.net> wrote:
> Hi,
>
> On 29/01/2018 18:47, Niko Mauno wrote:
>> This mitigates following issues during u-boot do_compile() step --
>> otherwise, if comm is not available, they are quietly ignored:
>>
>>   .../scripts/check-config.sh: line 33: comm: command not found
>>   .../scripts/check-config.sh: line 39: comm: command not found
>
> In current U-Boot master (since [0]), the absence of 'comm' is not
> silently ignored but rather causes a build failure.
>
> Thus I think this patch should be cherry-picked on the stable yocto
> branches. Otherwise people using a post-2018.3 U-Boot will see their
> builds fail and will have to find out on their own how to fix it.
>
> [0]
> http://git.denx.de/?p=u-boot.git;a=commit;h=f69dce5081396a83b87d6df2693764b99466a18d
>
> --
> Luca
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Luca Ceresoli]

Hi Ross,

On 28/03/2018 17:23, Burton, Ross wrote:
> Please backport the patch to rocko and submit it to the list.

OK, thanks.
Just sent.
-- 
Luca

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Denys Dmytriyenko]

On Mon, Jan 29, 2018 at 06:04:37PM +0000, Manjukumar Harthikote Matha wrote:
> 
> 
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org
> > [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Niko
> > Mauno
> > Sent: Monday, January 29, 2018 9:47 AM
> > To: openembedded-core@lists.openembedded.org
> > Cc: Niko Mauno <niko.mauno@vaisala.com>
> > Subject: [OE-core] [PATCH] bitbake.conf: Add comm to HOSTTOOLS
> > 
> > This mitigates following issues during u-boot do_compile() step -- otherwise, if
> > comm is not available, they are quietly ignored:
> > 
> >   .../scripts/check-config.sh: line 33: comm: command not found
> >   .../scripts/check-config.sh: line 39: comm: command not found
> > 
> > Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS was
> > considered a lower impact fix compared to adding coreutils-native buildtime
> > dependency to u-boot recipe.
> > 
> 
> Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>

????????
https://lists.x.org/archives/xorg-devel/2009-October/003036.html


> > Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
> > ---
> >  meta/conf/bitbake.conf | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index
> > 93afb13166..e1d2f3c3e2 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -473,7 +473,7 @@ HOSTTOOLS_DIR = "${TMPDIR}/hosttools"
> > 
> >  # Tools needed to run builds with OE-Core  HOSTTOOLS += " \
> > -    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp cp cpio \
> > +    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp
> > + comm cp cpio \
> >      cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
> >      fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \
> >      head hostname id install ld ldd ln ls make makeinfo md5sum mkdir mknod \
> > --
> > 2.11.0
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Denys Dmytriyenko]

On Wed, Mar 28, 2018 at 06:50:03PM +0200, Luca Ceresoli wrote:
> Hi Ross,
> 
> On 28/03/2018 17:23, Burton, Ross wrote:
> > Please backport the patch to rocko and submit it to the list.
> 
> OK, thanks.
> Just sent.

Thanks! Looking forward to getting it merged to Rocko.

-- 
Denys

Re: [PATCH] bitbake.conf: Add comm to HOSTTOOLS

[Manjukumar Harthikote Matha]

> -----Original Message-----
> From: Denys Dmytriyenko [mailto:denis@denix.org]
> Sent: Thursday, March 29, 2018 3:26 PM
> To: Manjukumar Harthikote Matha <MANJUKUM@xilinx.com>
> Cc: Niko Mauno <niko.mauno@vaisala.com>; openembedded-
> core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] bitbake.conf: Add comm to HOSTTOOLS
> 
> On Mon, Jan 29, 2018 at 06:04:37PM +0000, Manjukumar Harthikote Matha
> wrote:
> >
> >
> > > -----Original Message-----
> > > From: openembedded-core-bounces@lists.openembedded.org
> > > [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of
> Niko
> > > Mauno
> > > Sent: Monday, January 29, 2018 9:47 AM
> > > To: openembedded-core@lists.openembedded.org
> > > Cc: Niko Mauno <niko.mauno@vaisala.com>
> > > Subject: [OE-core] [PATCH] bitbake.conf: Add comm to HOSTTOOLS
> > >
> > > This mitigates following issues during u-boot do_compile() step -- otherwise, if
> > > comm is not available, they are quietly ignored:
> > >
> > >   .../scripts/check-config.sh: line 33: comm: command not found
> > >   .../scripts/check-config.sh: line 39: comm: command not found
> > >
> > > Since 'comm' is provided by coreutils package, adding it to HOSTTOOLS was
> > > considered a lower impact fix compared to adding coreutils-native buildtime
> > > dependency to u-boot recipe.
> > >
> >
> > Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
> 
> ????????
> https://lists.x.org/archives/xorg-devel/2009-October/003036.html
> 

Sorry my bad, intention was Tested-by


> 
> > > Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
> > > ---
> > >  meta/conf/bitbake.conf | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index
> > > 93afb13166..e1d2f3c3e2 100644
> > > --- a/meta/conf/bitbake.conf
> > > +++ b/meta/conf/bitbake.conf
> > > @@ -473,7 +473,7 @@ HOSTTOOLS_DIR = "${TMPDIR}/hosttools"
> > >
> > >  # Tools needed to run builds with OE-Core  HOSTTOOLS += " \
> > > -    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp cp
> cpio \
> > > +    [ ar as awk basename bash bzip2 cat chgrp chmod chown chrpath cmp
> > > + comm cp cpio \
> > >      cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
> > >      fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \
> > >      head hostname id install ld ldd ln ls make makeinfo md5sum mkdir mknod \
> > > --
> > > 2.11.0
> > >
> > > --
> > > _______________________________________________
> > > Openembedded-core mailing list
> > > Openembedded-core@lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core