[PATCH -next 1/2] arm: Remove enforced Os flag for LZ4 decompressor

[PATCH -next 1/2] arm: Remove enforced Os flag for LZ4 decompressor

[Kyungsik Lee]

-Os is enforced here, based on the test result of decompression time
below, slightly faster than -O2.

But further tests with UA show that using -O2 will be the right choice
especially in the case of the unaligned access enabled and the gap,
few counts in the normal decompression mode is small enough to remove -Os.

Decompression Time(counts)
     Normal    UA enabled
-Os  6717      3447
-O2  6720      2728

Note: ARM v7, Kernel 3.4
      counter freq. = 32768 HZ
      UA(Unaligned Access)
      gcc version 4.6.2

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
---
 arch/arm/boot/compressed/Makefile | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 001a13a..198a4ad 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -27,9 +27,6 @@ OBJS	+= misc.o decompress.o
 ifeq ($(CONFIG_DEBUG_UNCOMPRESS),y)
 OBJS	+= debug.o
 endif
-ifeq ($(CONFIG_KERNEL_LZ4),y)
-CFLAGS_decompress.o := -Os
-endif
 FONTC	= $(srctree)/drivers/video/console/font_acorn_8x8.c
 
 # string library code (-Os is enforced to keep it much smaller)
-- 
1.8.1.1

[PATCH -next 1/2] arm: Remove enforced Os flag for LZ4 decompressor

[Kyungsik Lee]

-Os is enforced here, based on the test result of decompression time
below, slightly faster than -O2.

But further tests with UA show that using -O2 will be the right choice
especially in the case of the unaligned access enabled and the gap,
few counts in the normal decompression mode is small enough to remove -Os.

Decompression Time(counts)
     Normal    UA enabled
-Os  6717      3447
-O2  6720      2728

Note: ARM v7, Kernel 3.4
      counter freq. = 32768 HZ
      UA(Unaligned Access)
      gcc version 4.6.2

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
---
 arch/arm/boot/compressed/Makefile | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 001a13a..198a4ad 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -27,9 +27,6 @@ OBJS	+= misc.o decompress.o
 ifeq ($(CONFIG_DEBUG_UNCOMPRESS),y)
 OBJS	+= debug.o
 endif
-ifeq ($(CONFIG_KERNEL_LZ4),y)
-CFLAGS_decompress.o := -Os
-endif
 FONTC	= $(srctree)/drivers/video/console/font_acorn_8x8.c
 
 # string library code (-Os is enforced to keep it much smaller)
-- 
1.8.1.1

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
lz4demo is replaced by lz4c. lz4c supports both the new streaming and
legacy format with -l option.

This patch makes use of lz4c to support legacy format which is
used for LZ4 De/compression in the linux kernel.

Link: https://code.google.com/p/lz4/source/checkout
Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
Cc: Chanho Min <chanho.min@lge.com>
---
 scripts/Makefile.lib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index a0ab6d7..c9bfbb0 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
 
 quiet_cmd_lz4 = LZ4     $@
 cmd_lz4 = (cat $(filter-out FORCE,$^) | \
-	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
+	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
 	(rm -f $@ ; false)
 
 # U-Boot mkimage
-- 
1.8.1.1

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
lz4demo is replaced by lz4c. lz4c supports both the new streaming and
legacy format with -l option.

This patch makes use of lz4c to support legacy format which is
used for LZ4 De/compression in the linux kernel.

Link: https://code.google.com/p/lz4/source/checkout
Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Florian Fainelli <florian@openwrt.org>
Cc: Yann Collet <yann.collet.73@gmail.com>
Cc: Chanho Min <chanho.min@lge.com>
---
 scripts/Makefile.lib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index a0ab6d7..c9bfbb0 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
 
 quiet_cmd_lz4 = LZ4     $@
 cmd_lz4 = (cat $(filter-out FORCE,$^) | \
-	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
+	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
 	(rm -f $@ ; false)
 
 # U-Boot mkimage
-- 
1.8.1.1

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Mon, May 06, 2013 at 05:42:55PM +0900, Kyungsik Lee wrote:
> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
> legacy format with -l option.
> 
> This patch makes use of lz4c to support legacy format which is
> used for LZ4 De/compression in the linux kernel.
> 
> Link: https://code.google.com/p/lz4/source/checkout
> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Russell King <rmk@arm.linux.org.uk>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Florian Fainelli <florian@openwrt.org>
> Cc: Yann Collet <yann.collet.73@gmail.com>
> Cc: Chanho Min <chanho.min@lge.com>
> ---
>  scripts/Makefile.lib | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index a0ab6d7..c9bfbb0 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>  
>  quiet_cmd_lz4 = LZ4     $@
>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \

You probably want to check for the presence of lz4c on the system and
bail with an informative message if absent.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Mon, May 06, 2013 at 05:42:55PM +0900, Kyungsik Lee wrote:
> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
> legacy format with -l option.
> 
> This patch makes use of lz4c to support legacy format which is
> used for LZ4 De/compression in the linux kernel.
> 
> Link: https://code.google.com/p/lz4/source/checkout
> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Russell King <rmk@arm.linux.org.uk>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Florian Fainelli <florian@openwrt.org>
> Cc: Yann Collet <yann.collet.73@gmail.com>
> Cc: Chanho Min <chanho.min@lge.com>
> ---
>  scripts/Makefile.lib | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index a0ab6d7..c9bfbb0 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>  
>  quiet_cmd_lz4 = LZ4     $@
>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \

You probably want to check for the presence of lz4c on the system and
bail with an informative message if absent.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Mon, May 6, 2013 at 11:51 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Mon, May 06, 2013 at 05:42:55PM +0900, Kyungsik Lee wrote:
>> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
>> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
>> legacy format with -l option.
>>
>> This patch makes use of lz4c to support legacy format which is
>> used for LZ4 De/compression in the linux kernel.
>>
>> Link: https://code.google.com/p/lz4/source/checkout
>> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>> Cc: Ingo Molnar <mingo@elte.hu>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Russell King <rmk@arm.linux.org.uk>
>> Cc: Borislav Petkov <bp@alien8.de>
>> Cc: Florian Fainelli <florian@openwrt.org>
>> Cc: Yann Collet <yann.collet.73@gmail.com>
>> Cc: Chanho Min <chanho.min@lge.com>
>> ---
>>  scripts/Makefile.lib | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index a0ab6d7..c9bfbb0 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>>
>>  quiet_cmd_lz4 = LZ4     $@
>>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
>> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>
> You probably want to check for the presence of lz4c on the system and
> bail with an informative message if absent.

Yep, x86_64-randconfig
(http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):

  LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
/bin/sh: lz4c: command not found

Also, several of the blackfin builds started failing due to a
compression-related
issue, e.g. http://kisskb.ellerman.id.au/kisskb/buildresult/9101322/:

  UIMAGE  arch/blackfin/boot/uImage.lzma

Invalid Compression Type - valid names are: none, bzip2, gzip
Usage: /usr/local/bin/mkimage -l image
          -l ==> list image header information
       /usr/local/bin/mkimage [-x] -A arch -O os -T type -C comp -a
addr -e ep -n name -d data_file[:data_file...] image
          -A ==> set architecture to 'arch'
          -O ==> set operating system to 'os'
          -T ==> set image type to 'type'
          -C ==> set compression type 'comp'
          -a ==> set load address to 'addr' (hex)
          -e ==> set entry point to 'ep' (hex)
          -n ==> set image name to 'name'
          -d ==> use image data from 'datafile'
          -x ==> set XIP (execute in place)
make[2]: *** [arch/blackfin/boot/uImage.lzma] Error 1

May be unrelated, though.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Mon, May 6, 2013 at 11:51 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Mon, May 06, 2013 at 05:42:55PM +0900, Kyungsik Lee wrote:
>> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
>> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
>> legacy format with -l option.
>>
>> This patch makes use of lz4c to support legacy format which is
>> used for LZ4 De/compression in the linux kernel.
>>
>> Link: https://code.google.com/p/lz4/source/checkout
>> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>> Cc: Ingo Molnar <mingo@elte.hu>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Russell King <rmk@arm.linux.org.uk>
>> Cc: Borislav Petkov <bp@alien8.de>
>> Cc: Florian Fainelli <florian@openwrt.org>
>> Cc: Yann Collet <yann.collet.73@gmail.com>
>> Cc: Chanho Min <chanho.min@lge.com>
>> ---
>>  scripts/Makefile.lib | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index a0ab6d7..c9bfbb0 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>>
>>  quiet_cmd_lz4 = LZ4     $@
>>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
>> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>
> You probably want to check for the presence of lz4c on the system and
> bail with an informative message if absent.

Yep, x86_64-randconfig
(http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):

  LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
/bin/sh: lz4c: command not found

Also, several of the blackfin builds started failing due to a
compression-related
issue, e.g. http://kisskb.ellerman.id.au/kisskb/buildresult/9101322/:

  UIMAGE  arch/blackfin/boot/uImage.lzma

Invalid Compression Type - valid names are: none, bzip2, gzip
Usage: /usr/local/bin/mkimage -l image
          -l ==> list image header information
       /usr/local/bin/mkimage [-x] -A arch -O os -T type -C comp -a
addr -e ep -n name -d data_file[:data_file...] image
          -A ==> set architecture to 'arch'
          -O ==> set operating system to 'os'
          -T ==> set image type to 'type'
          -C ==> set compression type 'comp'
          -a ==> set load address to 'addr' (hex)
          -e ==> set entry point to 'ep' (hex)
          -n ==> set image name to 'name'
          -d ==> use image data from 'datafile'
          -x ==> set XIP (execute in place)
make[2]: *** [arch/blackfin/boot/uImage.lzma] Error 1

May be unrelated, though.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Mon, May 6, 2013 at 11:51 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Mon, May 06, 2013 at 05:42:55PM +0900, Kyungsik Lee wrote:
>> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
>> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
>> legacy format with -l option.
>>
>> This patch makes use of lz4c to support legacy format which is
>> used for LZ4 De/compression in the linux kernel.
>>
>> Link: https://code.google.com/p/lz4/source/checkout
>> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>> Cc: Ingo Molnar <mingo@elte.hu>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Russell King <rmk@arm.linux.org.uk>
>> Cc: Borislav Petkov <bp@alien8.de>
>> Cc: Florian Fainelli <florian@openwrt.org>
>> Cc: Yann Collet <yann.collet.73@gmail.com>
>> Cc: Chanho Min <chanho.min@lge.com>
>> ---
>>  scripts/Makefile.lib | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index a0ab6d7..c9bfbb0 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>>
>>  quiet_cmd_lz4 = LZ4     $@
>>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
>> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>
> You probably want to check for the presence of lz4c on the system and
> bail with an informative message if absent.

Yep, x86_64-randconfig
(http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):

  LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
/bin/sh: lz4c: command not found

Also, several of the blackfin builds started failing due to a
compression-related
issue, e.g. http://kisskb.ellerman.id.au/kisskb/buildresult/9101322/:

  UIMAGE  arch/blackfin/boot/uImage.lzma

Invalid Compression Type - valid names are: none, bzip2, gzip
Usage: /usr/local/bin/mkimage -l image
          -l ==> list image header information
       /usr/local/bin/mkimage [-x] -A arch -O os -T type -C comp -a
addr -e ep -n name -d data_file[:data_file...] image
          -A ==> set architecture to 'arch'
          -O ==> set operating system to 'os'
          -T ==> set image type to 'type'
          -C ==> set compression type 'comp'
          -a ==> set load address to 'addr' (hex)
          -e ==> set entry point to 'ep' (hex)
          -n ==> set image name to 'name'
          -d ==> use image data from 'datafile'
          -x ==> set XIP (execute in place)
make[2]: *** [arch/blackfin/boot/uImage.lzma] Error 1

May be unrelated, though.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> >> index a0ab6d7..c9bfbb0 100644
> >> --- a/scripts/Makefile.lib
> >> +++ b/scripts/Makefile.lib
> >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> >>
> >>  quiet_cmd_lz4 = LZ4     $@
> >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >
> > You probably want to check for the presence of lz4c on the system and
> > bail with an informative message if absent.
> 
> Yep, x86_64-randconfig
> (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> 
>   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> /bin/sh: lz4c: command not found

Sure, it had to be 2 months and to hit upstream for there still to be no
fix. Geez...

Anyone reading this and willing to try his skills with an
introductory-level patch, feel free to address this. I'll help testing
and upstreaming it.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> >> index a0ab6d7..c9bfbb0 100644
> >> --- a/scripts/Makefile.lib
> >> +++ b/scripts/Makefile.lib
> >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> >>
> >>  quiet_cmd_lz4 = LZ4     $@
> >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >
> > You probably want to check for the presence of lz4c on the system and
> > bail with an informative message if absent.
> 
> Yep, x86_64-randconfig
> (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> 
>   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> /bin/sh: lz4c: command not found

Sure, it had to be 2 months and to hit upstream for there still to be no
fix. Geez...

Anyone reading this and willing to try his skills with an
introductory-level patch, feel free to address this. I'll help testing
and upstreaming it.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> >> index a0ab6d7..c9bfbb0 100644
> >> --- a/scripts/Makefile.lib
> >> +++ b/scripts/Makefile.lib
> >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> >>
> >>  quiet_cmd_lz4 = LZ4     $@
> >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >
> > You probably want to check for the presence of lz4c on the system and
> > bail with an informative message if absent.
> 
> Yep, x86_64-randconfig
> (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> 
>   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> /bin/sh: lz4c: command not found

Sure, it had to be 2 months and to hit upstream for there still to be no
fix. Geez...

Anyone reading this and willing to try his skills with an
introductory-level patch, feel free to address this. I'll help testing
and upstreaming it.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Markus Trippelsdorf]

On 2013.07.10 at 11:36 +0200, Borislav Petkov wrote:
> On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> > >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > >> index a0ab6d7..c9bfbb0 100644
> > >> --- a/scripts/Makefile.lib
> > >> +++ b/scripts/Makefile.lib
> > >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> > >>
> > >>  quiet_cmd_lz4 = LZ4     $@
> > >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >
> > > You probably want to check for the presence of lz4c on the system and
> > > bail with an informative message if absent.
> > 
> > Yep, x86_64-randconfig
> > (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> > 
> >   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> > /bin/sh: lz4c: command not found
> 
> Sure, it had to be 2 months and to hit upstream for there still to be no
> fix. Geez...

Well, there's also no hand-holding when lzop is missing for LZO.
And the error message is clear enough. No?

BTW speaking of introductory-level patches, what about the following
one:

Now that lz4 kernel compression is available, add *.lz4 to .gitignore

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno


-- 
Markus

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Markus Trippelsdorf]

On 2013.07.10 at 11:36 +0200, Borislav Petkov wrote:
> On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> > >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > >> index a0ab6d7..c9bfbb0 100644
> > >> --- a/scripts/Makefile.lib
> > >> +++ b/scripts/Makefile.lib
> > >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> > >>
> > >>  quiet_cmd_lz4 = LZ4     $@
> > >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >
> > > You probably want to check for the presence of lz4c on the system and
> > > bail with an informative message if absent.
> > 
> > Yep, x86_64-randconfig
> > (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> > 
> >   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> > /bin/sh: lz4c: command not found
> 
> Sure, it had to be 2 months and to hit upstream for there still to be no
> fix. Geez...

Well, there's also no hand-holding when lzop is missing for LZO.
And the error message is clear enough. No?

BTW speaking of introductory-level patches, what about the following
one:

Now that lz4 kernel compression is available, add *.lz4 to .gitignore

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno


-- 
Markus

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Markus Trippelsdorf]

On 2013.07.10 at 11:36 +0200, Borislav Petkov wrote:
> On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> > >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > >> index a0ab6d7..c9bfbb0 100644
> > >> --- a/scripts/Makefile.lib
> > >> +++ b/scripts/Makefile.lib
> > >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> > >>
> > >>  quiet_cmd_lz4 = LZ4     $@
> > >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >
> > > You probably want to check for the presence of lz4c on the system and
> > > bail with an informative message if absent.
> > 
> > Yep, x86_64-randconfig
> > (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> > 
> >   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> > /bin/sh: lz4c: command not found
> 
> Sure, it had to be 2 months and to hit upstream for there still to be no
> fix. Geez...

Well, there's also no hand-holding when lzop is missing for LZO.
And the error message is clear enough. No?

BTW speaking of introductory-level patches, what about the following
one:

Now that lz4 kernel compression is available, add *.lz4 to .gitignore

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno


-- 
Markus

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Markus Trippelsdorf]

On 2013.07.10 at 11:36 +0200, Borislav Petkov wrote:
> On Wed, Jul 10, 2013 at 10:12:46AM +0200, Geert Uytterhoeven wrote:
> > >> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > >> index a0ab6d7..c9bfbb0 100644
> > >> --- a/scripts/Makefile.lib
> > >> +++ b/scripts/Makefile.lib
> > >> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
> > >>
> > >>  quiet_cmd_lz4 = LZ4     $@
> > >>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > >> -     lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >> +     lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > >
> > > You probably want to check for the presence of lz4c on the system and
> > > bail with an informative message if absent.
> > 
> > Yep, x86_64-randconfig
> > (http://kisskb.ellerman.id.au/kisskb/buildresult/9110794/):
> > 
> >   LZ4     arch/x86/boot/compressed/vmlinux.bin.lz4
> > /bin/sh: lz4c: command not found
> 
> Sure, it had to be 2 months and to hit upstream for there still to be no
> fix. Geez...

Well, there's also no hand-holding when lzop is missing for LZO.
And the error message is clear enough. No?

BTW speaking of introductory-level patches, what about the following
one:

Now that lz4 kernel compression is available, add *.lz4 to .gitignore

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno


-- 
Markus

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 08:28:15PM +0200, Markus Trippelsdorf wrote:
> Well, there's also no hand-holding when lzop is missing for LZO. And
> the error message is clear enough. No?

Actually, we should error out more gracefully than that. Maybe check for
the presence of the executable first and if not, exit out of the build
with an informational message that lz4c is not present on the system.
Ditto for lzop.

Currently, you wait for the whole build to complete just to see that
there's no executable at the end. Not nice.

> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore

Yep. That makes sense.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 08:28:15PM +0200, Markus Trippelsdorf wrote:
> Well, there's also no hand-holding when lzop is missing for LZO. And
> the error message is clear enough. No?

Actually, we should error out more gracefully than that. Maybe check for
the presence of the executable first and if not, exit out of the build
with an informational message that lz4c is not present on the system.
Ditto for lzop.

Currently, you wait for the whole build to complete just to see that
there's no executable at the end. Not nice.

> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore

Yep. That makes sense.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 08:28:15PM +0200, Markus Trippelsdorf wrote:
> Well, there's also no hand-holding when lzop is missing for LZO. And
> the error message is clear enough. No?

Actually, we should error out more gracefully than that. Maybe check for
the presence of the executable first and if not, exit out of the build
with an informational message that lz4c is not present on the system.
Ditto for lzop.

Currently, you wait for the whole build to complete just to see that
there's no executable at the end. Not nice.

> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore

Yep. That makes sense.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 10, 2013 at 08:28:15PM +0200, Markus Trippelsdorf wrote:
> Well, there's also no hand-holding when lzop is missing for LZO. And
> the error message is clear enough. No?

Actually, we should error out more gracefully than that. Maybe check for
the presence of the executable first and if not, exit out of the build
with an informational message that lz4c is not present on the system.
Ditto for lzop.

Currently, you wait for the whole build to complete just to see that
there's no executable at the end. Not nice.

> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore

Yep. That makes sense.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

> 
> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore
> 
> diff --git a/.gitignore b/.gitignore
> index 3b8b9b3..7e9932e 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -29,6 +29,7 @@ modules.builtin
>  *.bz2
>  *.lzma
>  *.xz
> +*.lz4
>  *.lzo
>  *.patch
>  *.gcno
> 
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

Thanks,
Kyungsik

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

> 
> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore
> 
> diff --git a/.gitignore b/.gitignore
> index 3b8b9b3..7e9932e 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -29,6 +29,7 @@ modules.builtin
>  *.bz2
>  *.lzma
>  *.xz
> +*.lz4
>  *.lzo
>  *.patch
>  *.gcno
> 
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

Thanks,
Kyungsik

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

> 
> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore
> 
> diff --git a/.gitignore b/.gitignore
> index 3b8b9b3..7e9932e 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -29,6 +29,7 @@ modules.builtin
>  *.bz2
>  *.lzma
>  *.xz
> +*.lz4
>  *.lzo
>  *.patch
>  *.gcno
> 
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

Thanks,
Kyungsik

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Kyungsik Lee]

> 
> BTW speaking of introductory-level patches, what about the following
> one:
> 
> Now that lz4 kernel compression is available, add *.lz4 to .gitignore
> 
> diff --git a/.gitignore b/.gitignore
> index 3b8b9b3..7e9932e 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -29,6 +29,7 @@ modules.builtin
>  *.bz2
>  *.lzma
>  *.xz
> +*.lz4
>  *.lzo
>  *.patch
>  *.gcno
> 
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

Thanks,
Kyungsik

[PATCH] .gitignore: ignore *.lz4 files

[Markus Trippelsdorf]

Now that lz4 kernel compression is available, add *.lz4 to .gitignore.

Signed-off-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno
-- 
Markus

[PATCH] .gitignore: ignore *.lz4 files

[Markus Trippelsdorf]

Now that lz4 kernel compression is available, add *.lz4 to .gitignore.

Signed-off-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno
-- 
Markus

[PATCH] .gitignore: ignore *.lz4 files

[Markus Trippelsdorf]

Now that lz4 kernel compression is available, add *.lz4 to .gitignore.

Signed-off-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno
-- 
Markus

[PATCH] .gitignore: ignore *.lz4 files

[Markus Trippelsdorf]

Now that lz4 kernel compression is available, add *.lz4 to .gitignore.

Signed-off-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Acked-by: Kyungsik Lee <kyungsik.lee@lge.com>

diff --git a/.gitignore b/.gitignore
index 3b8b9b3..7e9932e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,6 +29,7 @@ modules.builtin
 *.bz2
 *.lzma
 *.xz
+*.lz4
 *.lzo
 *.patch
 *.gcno
-- 
Markus

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Kyungsik Lee <kyungsik.lee@lge.com> wrote:

> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
> legacy format with -l option.
> 
> This patch makes use of lz4c to support legacy format which is
> used for LZ4 De/compression in the linux kernel.
> 
> Link: https://code.google.com/p/lz4/source/checkout
> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Russell King <rmk@arm.linux.org.uk>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Florian Fainelli <florian@openwrt.org>
> Cc: Yann Collet <yann.collet.73@gmail.com>
> Cc: Chanho Min <chanho.min@lge.com>
> ---
>  scripts/Makefile.lib | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index a0ab6d7..c9bfbb0 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>  
>  quiet_cmd_lz4 = LZ4     $@
>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>  	(rm -f $@ ; false)

None of the major distros I tried has the lz4 or lz4c binary available, 
and this breaks randconfig builds:

  /bin/sh: lz4c: command not found

If:

 CONFIG_HAVE_KERNEL_LZ4=y
 CONFIG_KERNEL_LZ4=y
 CONFIG_RD_LZ4=y
 CONFIG_LZ4_DECOMPRESS=y
 CONFIG_DECOMPRESS_LZ4=y

If a utility is not widely available yet and if the utility is not in the 
kernel proper, could you please at least make sure that randconfig does 
not stumble over non-buildable kernels?

Thanks,

	Ingo

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Kyungsik Lee <kyungsik.lee@lge.com> wrote:

> LZ4 has been updated with LZ4 Streaming Format specification(v1.3).
> lz4demo is replaced by lz4c. lz4c supports both the new streaming and
> legacy format with -l option.
> 
> This patch makes use of lz4c to support legacy format which is
> used for LZ4 De/compression in the linux kernel.
> 
> Link: https://code.google.com/p/lz4/source/checkout
> Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Russell King <rmk@arm.linux.org.uk>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: Florian Fainelli <florian@openwrt.org>
> Cc: Yann Collet <yann.collet.73@gmail.com>
> Cc: Chanho Min <chanho.min@lge.com>
> ---
>  scripts/Makefile.lib | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index a0ab6d7..c9bfbb0 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -313,7 +313,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
>  
>  quiet_cmd_lz4 = LZ4     $@
>  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
>  	(rm -f $@ ; false)

None of the major distros I tried has the lz4 or lz4c binary available, 
and this breaks randconfig builds:

  /bin/sh: lz4c: command not found

If:

 CONFIG_HAVE_KERNEL_LZ4=y
 CONFIG_KERNEL_LZ4=y
 CONFIG_RD_LZ4=y
 CONFIG_LZ4_DECOMPRESS=y
 CONFIG_DECOMPRESS_LZ4=y

If a utility is not widely available yet and if the utility is not in the 
kernel proper, could you please at least make sure that randconfig does 
not stumble over non-buildable kernels?

Thanks,

	Ingo

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 09:56:02AM +0200, Ingo Molnar wrote:
> If a utility is not widely available yet and if the utility is not in
> the kernel proper, could you please at least make sure that randconfig
> does not stumble over non-buildable kernels?

Two months ago I've complained about this and other people have burned
themselves too in the meantime...

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 09:56:02AM +0200, Ingo Molnar wrote:
> If a utility is not widely available yet and if the utility is not in
> the kernel proper, could you please at least make sure that randconfig
> does not stumble over non-buildable kernels?

Two months ago I've complained about this and other people have burned
themselves too in the meantime...

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Borislav Petkov <bp@alien8.de> wrote:

> On Fri, Jul 12, 2013 at 09:56:02AM +0200, Ingo Molnar wrote:
> >
> > If a utility is not widely available yet and if the utility is not in 
> > the kernel proper, could you please at least make sure that randconfig 
> > does not stumble over non-buildable kernels?
> 
> Two months ago I've complained about this and other people have burned 
> themselves too in the meantime...

and now it is upstream already, via -mm:

   e76e1fdfa8f8 lib: add support for LZ4-compressed kernel


Thanks,

	Ingo

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Borislav Petkov <bp@alien8.de> wrote:

> On Fri, Jul 12, 2013 at 09:56:02AM +0200, Ingo Molnar wrote:
> >
> > If a utility is not widely available yet and if the utility is not in 
> > the kernel proper, could you please at least make sure that randconfig 
> > does not stumble over non-buildable kernels?
> 
> Two months ago I've complained about this and other people have burned 
> themselves too in the meantime...

and now it is upstream already, via -mm:

   e76e1fdfa8f8 lib: add support for LZ4-compressed kernel


Thanks,

	Ingo

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> and now it is upstream already, via -mm:
> 
>    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel

Sure. Let's see if it manages to get released b0rked like it is right
now.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> and now it is upstream already, via -mm:
> 
>    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel

Sure. Let's see if it manages to get released b0rked like it is right
now.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Fri, 12 Jul 2013 12:23:26 +0200 Borislav Petkov <bp@alien8.de> wrote:

> On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> > and now it is upstream already, via -mm:
> > 
> >    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel
> 
> Sure. Let's see if it manages to get released b0rked like it is right
> now.

Well y'know, if whining fixed bugs then there would be no problem here
at all.

This problem came up briefly and I thought it had been adequately
addressed.  Now a month or two later we find out who are the people who
can't be bothered testing -next, but think it terribly important that
others do so.

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Fri, 12 Jul 2013 12:23:26 +0200 Borislav Petkov <bp@alien8.de> wrote:

> On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> > and now it is upstream already, via -mm:
> > 
> >    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel
> 
> Sure. Let's see if it manages to get released b0rked like it is right
> now.

Well y'know, if whining fixed bugs then there would be no problem here
at all.

This problem came up briefly and I thought it had been adequately
addressed.  Now a month or two later we find out who are the people who
can't be bothered testing -next, but think it terribly important that
others do so.

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Fri, 12 Jul 2013 12:23:26 +0200 Borislav Petkov <bp@alien8.de> wrote:
> 
> > On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> > > and now it is upstream already, via -mm:
> > > 
> > >    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel
> > 
> > Sure. Let's see if it manages to get released b0rked like it is right
> > now.
> 
> Well y'know, if whining fixed bugs then there would be no problem here 
> at all.
> 
> This problem came up briefly and I thought it had been adequately 
> addressed.  Now a month or two later we find out who are the people who 
> can't be bothered testing -next, but think it terribly important that 
> others do so.

At least with my limited resources -next is unusable for my randconfig 
testing. I can randconfig test the fresh new bugs I introduce plus the 
bugs that slip upstream - which naturally peak during the merge window.

Anyway, I hacked it around locally, sh*t happens, it should be easy enough 
to fix.

Thanks,

	Ingo

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Fri, 12 Jul 2013 12:23:26 +0200 Borislav Petkov <bp@alien8.de> wrote:
> 
> > On Fri, Jul 12, 2013 at 12:13:19PM +0200, Ingo Molnar wrote:
> > > and now it is upstream already, via -mm:
> > > 
> > >    e76e1fdfa8f8 lib: add support for LZ4-compressed kernel
> > 
> > Sure. Let's see if it manages to get released b0rked like it is right
> > now.
> 
> Well y'know, if whining fixed bugs then there would be no problem here 
> at all.
> 
> This problem came up briefly and I thought it had been adequately 
> addressed.  Now a month or two later we find out who are the people who 
> can't be bothered testing -next, but think it terribly important that 
> others do so.

At least with my limited resources -next is unusable for my randconfig 
testing. I can randconfig test the fresh new bugs I introduce plus the 
bugs that slip upstream - which naturally peak during the merge window.

Anyway, I hacked it around locally, sh*t happens, it should be easy enough 
to fix.

Thanks,

	Ingo

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 04:18:10AM -0700, Andrew Morton wrote:
> Well y'know, if whining fixed bugs then there would be no problem here
> at all.

I know. I decided to whine this time and not try to fix it because
attempting to fix every fallout I'm seeing in the tree with randconfigs
is a full-time job and the *actual* job doesn't see any progress.

And in all fairness, if someone were reporting an issue against my code,
I'll try to do my very best to fix it as quickly aas possible, instead
of ignoring it.

So actually, the right thing to do in such cases is back out half-cooked
stuff from the tree until it gets fixed. That'll give the authors the
right incentive. But this is just me.

> This problem came up briefly and I thought it had been adequately
> addressed. Now a month or two later we find out who are the people who
> can't be bothered testing -next, but think it terribly important that
> others do so.

Yeah, if everyone would put their money where their mouth is...

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 04:18:10AM -0700, Andrew Morton wrote:
> Well y'know, if whining fixed bugs then there would be no problem here
> at all.

I know. I decided to whine this time and not try to fix it because
attempting to fix every fallout I'm seeing in the tree with randconfigs
is a full-time job and the *actual* job doesn't see any progress.

And in all fairness, if someone were reporting an issue against my code,
I'll try to do my very best to fix it as quickly aas possible, instead
of ignoring it.

So actually, the right thing to do in such cases is back out half-cooked
stuff from the tree until it gets fixed. That'll give the authors the
right incentive. But this is just me.

> This problem came up briefly and I thought it had been adequately
> addressed. Now a month or two later we find out who are the people who
> can't be bothered testing -next, but think it terribly important that
> others do so.

Yeah, if everyone would put their money where their mouth is...

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

Hello,

2013/7/12 Borislav Petkov <bp@alien8.de>:
> On Fri, Jul 12, 2013 at 04:18:10AM -0700, Andrew Morton wrote:
>> Well y'know, if whining fixed bugs then there would be no problem here
>> at all.
>
> I know. I decided to whine this time and not try to fix it because
> attempting to fix every fallout I'm seeing in the tree with randconfigs
> is a full-time job and the *actual* job doesn't see any progress.
>
> And in all fairness, if someone were reporting an issue against my code,
> I'll try to do my very best to fix it as quickly aas possible, instead
> of ignoring it.
>
> So actually, the right thing to do in such cases is back out half-cooked
> stuff from the tree until it gets fixed. That'll give the authors the
> right incentive. But this is just me.
>
>> This problem came up briefly and I thought it had been adequately
>> addressed. Now a month or two later we find out who are the people who
>> can't be bothered testing -next, but think it terribly important that
>> others do so.
>
> Yeah, if everyone would put their money where their mouth is...

Shall we provide a shell script wrapper for all of these compressor
commands which would test for the utility presence and issue a BIG FAT
WARNING if not, but still create the file not to make the kernel build
fail? I guess we could probably do this in the Makefile directly as
well.
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

Hello,

2013/7/12 Borislav Petkov <bp@alien8.de>:
> On Fri, Jul 12, 2013 at 04:18:10AM -0700, Andrew Morton wrote:
>> Well y'know, if whining fixed bugs then there would be no problem here
>> at all.
>
> I know. I decided to whine this time and not try to fix it because
> attempting to fix every fallout I'm seeing in the tree with randconfigs
> is a full-time job and the *actual* job doesn't see any progress.
>
> And in all fairness, if someone were reporting an issue against my code,
> I'll try to do my very best to fix it as quickly aas possible, instead
> of ignoring it.
>
> So actually, the right thing to do in such cases is back out half-cooked
> stuff from the tree until it gets fixed. That'll give the authors the
> right incentive. But this is just me.
>
>> This problem came up briefly and I thought it had been adequately
>> addressed. Now a month or two later we find out who are the people who
>> can't be bothered testing -next, but think it terribly important that
>> others do so.
>
> Yeah, if everyone would put their money where their mouth is...

Shall we provide a shell script wrapper for all of these compressor
commands which would test for the utility presence and issue a BIG FAT
WARNING if not, but still create the file not to make the kernel build
fail? I guess we could probably do this in the Makefile directly as
well.
--
Florian

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

Hello,

2013/7/12 Borislav Petkov <bp@alien8.de>:
> On Fri, Jul 12, 2013 at 04:18:10AM -0700, Andrew Morton wrote:
>> Well y'know, if whining fixed bugs then there would be no problem here
>> at all.
>
> I know. I decided to whine this time and not try to fix it because
> attempting to fix every fallout I'm seeing in the tree with randconfigs
> is a full-time job and the *actual* job doesn't see any progress.
>
> And in all fairness, if someone were reporting an issue against my code,
> I'll try to do my very best to fix it as quickly aas possible, instead
> of ignoring it.
>
> So actually, the right thing to do in such cases is back out half-cooked
> stuff from the tree until it gets fixed. That'll give the authors the
> right incentive. But this is just me.
>
>> This problem came up briefly and I thought it had been adequately
>> addressed. Now a month or two later we find out who are the people who
>> can't be bothered testing -next, but think it terribly important that
>> others do so.
>
> Yeah, if everyone would put their money where their mouth is...

Shall we provide a shell script wrapper for all of these compressor
commands which would test for the utility presence and issue a BIG FAT
WARNING if not, but still create the file not to make the kernel build
fail? I guess we could probably do this in the Makefile directly as
well.
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 01:34:38PM +0100, Florian Fainelli wrote:
> Shall we provide a shell script wrapper for all of these compressor
> commands which would test for the utility presence and issue a BIG FAT
> WARNING if not, but still create the file not to make the kernel build
> fail? I guess we could probably do this in the Makefile directly as
> well.

I don't know how that would work with the main kernel Makefile
but we do those kinds of tests in perf before we build it so
that we can save ourselves the time spent on a failed build:
2209001fd895e8932ae2c85bfca233758234499a.

This should probably work if you add a prerequisite target which runs
early in the build process and checks for all the userspace tools
required for the build.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 01:34:38PM +0100, Florian Fainelli wrote:
> Shall we provide a shell script wrapper for all of these compressor
> commands which would test for the utility presence and issue a BIG FAT
> WARNING if not, but still create the file not to make the kernel build
> fail? I guess we could probably do this in the Makefile directly as
> well.

I don't know how that would work with the main kernel Makefile
but we do those kinds of tests in perf before we build it so
that we can save ourselves the time spent on a failed build:
2209001fd895e8932ae2c85bfca233758234499a.

This should probably work if you add a prerequisite target which runs
early in the build process and checks for all the userspace tools
required for the build.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Fri, Jul 12, 2013 at 01:34:38PM +0100, Florian Fainelli wrote:
> Shall we provide a shell script wrapper for all of these compressor
> commands which would test for the utility presence and issue a BIG FAT
> WARNING if not, but still create the file not to make the kernel build
> fail? I guess we could probably do this in the Makefile directly as
> well.

I don't know how that would work with the main kernel Makefile
but we do those kinds of tests in perf before we build it so
that we can save ourselves the time spent on a failed build:
2209001fd895e8932ae2c85bfca233758234499a.

This should probably work if you add a prerequisite target which runs
early in the build process and checks for all the userspace tools
required for the build.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Fri, 12 Jul 2013 09:56:02 +0200 Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Kyungsik Lee <kyungsik.lee@lge.com> wrote:
> >  quiet_cmd_lz4 = LZ4     $@
> >  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >  	(rm -f $@ ; false)
> 
> None of the major distros I tried has the lz4 or lz4c binary available, 
> and this breaks randconfig builds:
> 
>   /bin/sh: lz4c: command not found
> 
> If:
> 
>  CONFIG_HAVE_KERNEL_LZ4=y
>  CONFIG_KERNEL_LZ4=y
>  CONFIG_RD_LZ4=y
>  CONFIG_LZ4_DECOMPRESS=y
>  CONFIG_DECOMPRESS_LZ4=y
> 
> If a utility is not widely available yet and if the utility is not in the 
> kernel proper, could you please at least make sure that randconfig does 
> not stumble over non-buildable kernels?

I don't know how to do this.  Any suggestions?

It has to be done at `make config' time.  We'd need to probe for the
presence of lz4c and then....  what?

Is there any precedent for this?

I don't think we can just ignore the absence of lz4c - the user has
selected a config which his system cannot build.  The problem lies
within randconfig itself.

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Fri, 12 Jul 2013 09:56:02 +0200 Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Kyungsik Lee <kyungsik.lee@lge.com> wrote:
> >  quiet_cmd_lz4 = LZ4     $@
> >  cmd_lz4 = (cat $(filter-out FORCE,$^) | \
> > -	lz4demo -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> > +	lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> >  	(rm -f $@ ; false)
> 
> None of the major distros I tried has the lz4 or lz4c binary available, 
> and this breaks randconfig builds:
> 
>   /bin/sh: lz4c: command not found
> 
> If:
> 
>  CONFIG_HAVE_KERNEL_LZ4=y
>  CONFIG_KERNEL_LZ4=y
>  CONFIG_RD_LZ4=y
>  CONFIG_LZ4_DECOMPRESS=y
>  CONFIG_DECOMPRESS_LZ4=y
> 
> If a utility is not widely available yet and if the utility is not in the 
> kernel proper, could you please at least make sure that randconfig does 
> not stumble over non-buildable kernels?

I don't know how to do this.  Any suggestions?

It has to be done at `make config' time.  We'd need to probe for the
presence of lz4c and then....  what?

Is there any precedent for this?

I don't think we can just ignore the absence of lz4c - the user has
selected a config which his system cannot build.  The problem lies
within randconfig itself.

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[H. Peter Anvin]

On 07/15/2013 03:03 PM, Andrew Morton wrote:
> 
> I don't know how to do this.  Any suggestions?
> 
> It has to be done at `make config' time.  We'd need to probe for the
> presence of lz4c and then....  what?
> 
> Is there any precedent for this?
> 
> I don't think we can just ignore the absence of lz4c - the user has
> selected a config which his system cannot build.  The problem lies
> within randconfig itself.
> 

We keep running over the need to be able to have kconfig run tests on
the build system (for toolchain support or for optional tools needed);
running them in the Makefiles (i.e. at Kbuild time) is simply too late.

	-hpa

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[H. Peter Anvin]

On 07/15/2013 03:03 PM, Andrew Morton wrote:
> 
> I don't know how to do this.  Any suggestions?
> 
> It has to be done at `make config' time.  We'd need to probe for the
> presence of lz4c and then....  what?
> 
> Is there any precedent for this?
> 
> I don't think we can just ignore the absence of lz4c - the user has
> selected a config which his system cannot build.  The problem lies
> within randconfig itself.
> 

We keep running over the need to be able to have kconfig run tests on
the build system (for toolchain support or for optional tools needed);
running them in the Makefiles (i.e. at Kbuild time) is simply too late.

	-hpa

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:

> On 07/15/2013 03:03 PM, Andrew Morton wrote:
> > 
> > I don't know how to do this.  Any suggestions?
> > 
> > It has to be done at `make config' time.  We'd need to probe for the
> > presence of lz4c and then....  what?
> > 
> > Is there any precedent for this?
> > 
> > I don't think we can just ignore the absence of lz4c - the user has
> > selected a config which his system cannot build.  The problem lies
> > within randconfig itself.
> > 
> 
> We keep running over the need to be able to have kconfig run tests on
> the build system (for toolchain support or for optional tools needed);
> running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> 

Would it make sense to extend Kconfig's `depends'?

	depends on $(shell-command)

I don't know how practical that would be to implement...

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:

> On 07/15/2013 03:03 PM, Andrew Morton wrote:
> > 
> > I don't know how to do this.  Any suggestions?
> > 
> > It has to be done at `make config' time.  We'd need to probe for the
> > presence of lz4c and then....  what?
> > 
> > Is there any precedent for this?
> > 
> > I don't think we can just ignore the absence of lz4c - the user has
> > selected a config which his system cannot build.  The problem lies
> > within randconfig itself.
> > 
> 
> We keep running over the need to be able to have kconfig run tests on
> the build system (for toolchain support or for optional tools needed);
> running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> 

Would it make sense to extend Kconfig's `depends'?

	depends on $(shell-command)

I don't know how practical that would be to implement...

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> 
> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> > > 
> > > I don't know how to do this.  Any suggestions?
> > > 
> > > It has to be done at `make config' time.  We'd need to probe for the
> > > presence of lz4c and then....  what?
> > > 
> > > Is there any precedent for this?
> > > 
> > > I don't think we can just ignore the absence of lz4c - the user has
> > > selected a config which his system cannot build.  The problem lies
> > > within randconfig itself.
> > > 
> > 
> > We keep running over the need to be able to have kconfig run tests on
> > the build system (for toolchain support or for optional tools needed);
> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > 
> 
> Would it make sense to extend Kconfig's `depends'?
> 
> 	depends on $(shell-command)
> 
> I don't know how practical that would be to implement...

Or, easier and faster, run some front-end script which generates
once-off Kconfig symbols.

	if [ -x /bin/lz4c ]
	then
		echo CONFIG_HAVE_LZ4C
	fi

then munge the output of that script into the Kconfig run and do

	depends on HAVE_LZ4C

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> 
> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> > > 
> > > I don't know how to do this.  Any suggestions?
> > > 
> > > It has to be done at `make config' time.  We'd need to probe for the
> > > presence of lz4c and then....  what?
> > > 
> > > Is there any precedent for this?
> > > 
> > > I don't think we can just ignore the absence of lz4c - the user has
> > > selected a config which his system cannot build.  The problem lies
> > > within randconfig itself.
> > > 
> > 
> > We keep running over the need to be able to have kconfig run tests on
> > the build system (for toolchain support or for optional tools needed);
> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > 
> 
> Would it make sense to extend Kconfig's `depends'?
> 
> 	depends on $(shell-command)
> 
> I don't know how practical that would be to implement...

Or, easier and faster, run some front-end script which generates
once-off Kconfig symbols.

	if [ -x /bin/lz4c ]
	then
		echo CONFIG_HAVE_LZ4C
	fi

then munge the output of that script into the Kconfig run and do

	depends on HAVE_LZ4C

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Andrew, All,

On Tuesday 16 July 2013 09:56:11 Andrew Morton wrote:
> On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> > > On 07/15/2013 03:03 PM, Andrew Morton wrote:
[--SNIP--]
> > > We keep running over the need to be able to have kconfig run tests on
> > > the build system (for toolchain support or for optional tools needed);
> > > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > > 
> > 
> > Would it make sense to extend Kconfig's `depends'?
> > 
> > 	depends on $(shell-command)
> > 
> > I don't know how practical that would be to implement...

I'm afraid this will get rather ugly and not trivial.

Can we mix 'depends on SYMBOL' and 'depends on $(command)' ?
Can we mix both in a boolean expression such as 'depends on SYMBOL
&& $(command)' ?

What would be the condition for evaluating the dependency rule? Evaluation
at Kconfig read-time might not be enough, given this construct:

    config FOO
        depends on $(foo)
    comment "'foo' is missing, please install it"
        depends on !$(foo)

Also, I believe Kconfig should stay a config-only language, without
much esoteric features.

> Or, easier and faster, run some front-end script which generates
> once-off Kconfig symbols.
> 
> 	if [ -x /bin/lz4c ]
> 	then
> 		echo CONFIG_HAVE_LZ4C
> 	fi
> 
> then munge the output of that script into the Kconfig run and do
> 
> 	depends on HAVE_LZ4C

Yes, this is a better solution.

For what it's worth, this is what I'm doing in crosstool-NG: a script
checks for optional pre-requisites, spits out a Kconfig blob which is
included by the top-level Kconfig file.

Here is a snippet of generated Kconfig blob:
    config HAVE_XZ
        def_bool y
    config HAVE_LZMA
        bool

Which means we do have 'xz', but not 'lzma'. This is relatively trivial
to do, so I'll tackle this this evening when I'm back home (unless
someone beats me to it).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Andrew, All,

On Tuesday 16 July 2013 09:56:11 Andrew Morton wrote:
> On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> > > On 07/15/2013 03:03 PM, Andrew Morton wrote:
[--SNIP--]
> > > We keep running over the need to be able to have kconfig run tests on
> > > the build system (for toolchain support or for optional tools needed);
> > > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > > 
> > 
> > Would it make sense to extend Kconfig's `depends'?
> > 
> > 	depends on $(shell-command)
> > 
> > I don't know how practical that would be to implement...

I'm afraid this will get rather ugly and not trivial.

Can we mix 'depends on SYMBOL' and 'depends on $(command)' ?
Can we mix both in a boolean expression such as 'depends on SYMBOL
&& $(command)' ?

What would be the condition for evaluating the dependency rule? Evaluation
at Kconfig read-time might not be enough, given this construct:

    config FOO
        depends on $(foo)
    comment "'foo' is missing, please install it"
        depends on !$(foo)

Also, I believe Kconfig should stay a config-only language, without
much esoteric features.

> Or, easier and faster, run some front-end script which generates
> once-off Kconfig symbols.
> 
> 	if [ -x /bin/lz4c ]
> 	then
> 		echo CONFIG_HAVE_LZ4C
> 	fi
> 
> then munge the output of that script into the Kconfig run and do
> 
> 	depends on HAVE_LZ4C

Yes, this is a better solution.

For what it's worth, this is what I'm doing in crosstool-NG: a script
checks for optional pre-requisites, spits out a Kconfig blob which is
included by the top-level Kconfig file.

Here is a snippet of generated Kconfig blob:
    config HAVE_XZ
        def_bool y
    config HAVE_LZMA
        bool

Which means we do have 'xz', but not 'lzma'. This is relatively trivial
to do, so I'll tackle this this evening when I'm back home (unless
someone beats me to it).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Tue, 16 Jul 2013 10:08:07 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > Or, easier and faster, run some front-end script which generates
> > once-off Kconfig symbols.
> > 
> > 	if [ -x /bin/lz4c ]
> > 	then
> > 		echo CONFIG_HAVE_LZ4C
> > 	fi
> > 
> > then munge the output of that script into the Kconfig run and do
> > 
> > 	depends on HAVE_LZ4C
> 
> Yes, this is a better solution.
> 
> For what it's worth, this is what I'm doing in crosstool-NG: a script
> checks for optional pre-requisites, spits out a Kconfig blob which is
> included by the top-level Kconfig file.
> 
> Here is a snippet of generated Kconfig blob:
>     config HAVE_XZ
>         def_bool y
>     config HAVE_LZMA
>         bool
> 
> Which means we do have 'xz', but not 'lzma'. This is relatively trivial
> to do, so I'll tackle this this evening when I'm back home (unless
> someone beats me to it).

Cool, thanks.

It seems a bit inefficient to be evaluating commands in a Kconfig
setting where doesn't need that done.  For example, if someone has
selected LZO compression then they don't need to probe for lz4c.

That example sounds hard to solve in a simple fashion, but what about
the case where the config system is not going to look at an entire
subsystem?  Suppose for example, drivers/media/firewire wants to probe
for some executable, but the user hasn't selected firewire at all.

What I'm angling at is, rather than a single global front-end script,
can we embed the scripts in some fashion within the various Kconfig
files?  Say,

	script ./some-script.sh

and the config system will only evaluate that command if it is working
on that Kconfig file.  Obviously that requires a multiple-pass thing.

It's late, but you see what I mean ;)

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Tue, 16 Jul 2013 10:08:07 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > Or, easier and faster, run some front-end script which generates
> > once-off Kconfig symbols.
> > 
> > 	if [ -x /bin/lz4c ]
> > 	then
> > 		echo CONFIG_HAVE_LZ4C
> > 	fi
> > 
> > then munge the output of that script into the Kconfig run and do
> > 
> > 	depends on HAVE_LZ4C
> 
> Yes, this is a better solution.
> 
> For what it's worth, this is what I'm doing in crosstool-NG: a script
> checks for optional pre-requisites, spits out a Kconfig blob which is
> included by the top-level Kconfig file.
> 
> Here is a snippet of generated Kconfig blob:
>     config HAVE_XZ
>         def_bool y
>     config HAVE_LZMA
>         bool
> 
> Which means we do have 'xz', but not 'lzma'. This is relatively trivial
> to do, so I'll tackle this this evening when I'm back home (unless
> someone beats me to it).

Cool, thanks.

It seems a bit inefficient to be evaluating commands in a Kconfig
setting where doesn't need that done.  For example, if someone has
selected LZO compression then they don't need to probe for lz4c.

That example sounds hard to solve in a simple fashion, but what about
the case where the config system is not going to look at an entire
subsystem?  Suppose for example, drivers/media/firewire wants to probe
for some executable, but the user hasn't selected firewire at all.

What I'm angling at is, rather than a single global front-end script,
can we embed the scripts in some fashion within the various Kconfig
files?  Say,

	script ./some-script.sh

and the config system will only evaluate that command if it is working
on that Kconfig file.  Obviously that requires a multiple-pass thing.

It's late, but you see what I mean ;)

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 01:27:56AM -0700, Andrew Morton wrote:
> What I'm angling at is, rather than a single global front-end script,
> can we embed the scripts in some fashion within the various Kconfig
> files?  Say,
> 
> 	script ./some-script.sh
> 
> and the config system will only evaluate that command if it is working
> on that Kconfig file.  Obviously that requires a multiple-pass thing.

What's wrong with simply grepping the .config we've just created for
*enabled* symbols which require userspace support, check for the
presence of said support and bail out if none?

I.e., those steps:

1. make <whatever>config
2. make => A prereq. target runs the shell script.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 01:27:56AM -0700, Andrew Morton wrote:
> What I'm angling at is, rather than a single global front-end script,
> can we embed the scripts in some fashion within the various Kconfig
> files?  Say,
> 
> 	script ./some-script.sh
> 
> and the config system will only evaluate that command if it is working
> on that Kconfig file.  Obviously that requires a multiple-pass thing.

What's wrong with simply grepping the .config we've just created for
*enabled* symbols which require userspace support, check for the
presence of said support and bail out if none?

I.e., those steps:

1. make <whatever>config
2. make => A prereq. target runs the shell script.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:05:46 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 01:27:56AM -0700, Andrew Morton wrote:
> > What I'm angling at is, rather than a single global front-end script,
> > can we embed the scripts in some fashion within the various Kconfig
> > files?  Say,
> > 
> > 	script ./some-script.sh
> > 
> > and the config system will only evaluate that command if it is working
> > on that Kconfig file.  Obviously that requires a multiple-pass thing.
> 
> What's wrong with simply grepping the .config we've just created for
> *enabled* symbols which require userspace support, check for the
> presence of said support and bail out if none?

The goal is to avoid generating a non-buildable kernel in the first place.
This is very usefull for automatic check harness that run randconfig, for
example (but may have more use-cases).

> I.e., those steps:
> 1. make <whatever>config
> 2. make => A prereq. target runs the shell script.

This is too late, since this will bail out, and will give a false-positive
failure.

While if we were to check for _known_ needed tools before calling into
Kconfig, we can simply hide non-available config options, and even inform
the user *why* the option is not available.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:05:46 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 01:27:56AM -0700, Andrew Morton wrote:
> > What I'm angling at is, rather than a single global front-end script,
> > can we embed the scripts in some fashion within the various Kconfig
> > files?  Say,
> > 
> > 	script ./some-script.sh
> > 
> > and the config system will only evaluate that command if it is working
> > on that Kconfig file.  Obviously that requires a multiple-pass thing.
> 
> What's wrong with simply grepping the .config we've just created for
> *enabled* symbols which require userspace support, check for the
> presence of said support and bail out if none?

The goal is to avoid generating a non-buildable kernel in the first place.
This is very usefull for automatic check harness that run randconfig, for
example (but may have more use-cases).

> I.e., those steps:
> 1. make <whatever>config
> 2. make => A prereq. target runs the shell script.

This is too late, since this will bail out, and will give a false-positive
failure.

While if we were to check for _known_ needed tools before calling into
Kconfig, we can simply hide non-available config options, and even inform
the user *why* the option is not available.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
>> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
>>
>> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
>> > >
>> > > I don't know how to do this.  Any suggestions?
>> > >
>> > > It has to be done at `make config' time.  We'd need to probe for the
>> > > presence of lz4c and then....  what?
>> > >
>> > > Is there any precedent for this?
>> > >
>> > > I don't think we can just ignore the absence of lz4c - the user has
>> > > selected a config which his system cannot build.  The problem lies
>> > > within randconfig itself.
>> > >
>> >
>> > We keep running over the need to be able to have kconfig run tests on
>> > the build system (for toolchain support or for optional tools needed);
>> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
>> >
>>
>> Would it make sense to extend Kconfig's `depends'?
>>
>>       depends on $(shell-command)
>>
>> I don't know how practical that would be to implement...
>
> Or, easier and faster, run some front-end script which generates
> once-off Kconfig symbols.
>
>         if [ -x /bin/lz4c ]
>         then
>                 echo CONFIG_HAVE_LZ4C
>         fi
>
> then munge the output of that script into the Kconfig run and do
>
>         depends on HAVE_LZ4C

That does sound nice to avoid the build error, but will eventually
make it harder to diagnose why the kernel has not been compressed with
the specific compression tool, some kind of warning should also be
emitted maybe?
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
>> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
>>
>> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
>> > >
>> > > I don't know how to do this.  Any suggestions?
>> > >
>> > > It has to be done at `make config' time.  We'd need to probe for the
>> > > presence of lz4c and then....  what?
>> > >
>> > > Is there any precedent for this?
>> > >
>> > > I don't think we can just ignore the absence of lz4c - the user has
>> > > selected a config which his system cannot build.  The problem lies
>> > > within randconfig itself.
>> > >
>> >
>> > We keep running over the need to be able to have kconfig run tests on
>> > the build system (for toolchain support or for optional tools needed);
>> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
>> >
>>
>> Would it make sense to extend Kconfig's `depends'?
>>
>>       depends on $(shell-command)
>>
>> I don't know how practical that would be to implement...
>
> Or, easier and faster, run some front-end script which generates
> once-off Kconfig symbols.
>
>         if [ -x /bin/lz4c ]
>         then
>                 echo CONFIG_HAVE_LZ4C
>         fi
>
> then munge the output of that script into the Kconfig run and do
>
>         depends on HAVE_LZ4C

That does sound nice to avoid the build error, but will eventually
make it harder to diagnose why the kernel has not been compressed with
the specific compression tool, some kind of warning should also be
emitted maybe?
--
Florian

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
>> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
>>
>> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
>> > >
>> > > I don't know how to do this.  Any suggestions?
>> > >
>> > > It has to be done at `make config' time.  We'd need to probe for the
>> > > presence of lz4c and then....  what?
>> > >
>> > > Is there any precedent for this?
>> > >
>> > > I don't think we can just ignore the absence of lz4c - the user has
>> > > selected a config which his system cannot build.  The problem lies
>> > > within randconfig itself.
>> > >
>> >
>> > We keep running over the need to be able to have kconfig run tests on
>> > the build system (for toolchain support or for optional tools needed);
>> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
>> >
>>
>> Would it make sense to extend Kconfig's `depends'?
>>
>>       depends on $(shell-command)
>>
>> I don't know how practical that would be to implement...
>
> Or, easier and faster, run some front-end script which generates
> once-off Kconfig symbols.
>
>         if [ -x /bin/lz4c ]
>         then
>                 echo CONFIG_HAVE_LZ4C
>         fi
>
> then munge the output of that script into the Kconfig run and do
>
>         depends on HAVE_LZ4C

That does sound nice to avoid the build error, but will eventually
make it harder to diagnose why the kernel has not been compressed with
the specific compression tool, some kind of warning should also be
emitted maybe?
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:12:17AM +0200, Yann E. MORIN wrote:
> The goal is to avoid generating a non-buildable kernel in the first place.

I know, that's why I'm proposing to fail the build with the script run
as one of the first things by make.

> > I.e., those steps:
> > 1. make <whatever>config
> > 2. make => A prereq. target runs the shell script.
> 
> This is too late, since this will bail out, and will give a false-positive
> failure.

Huh? This will tell you: "you've selected X, but I don't have toolX
needed for the build".

You install the tool and run make again. Checks pass, the build
continues.

> While if we were to check for _known_ needed tools before calling into
> Kconfig,

How would you do that? How do you know what the user is going to select
*before* she even selects it?

You need to have a prepared .config to act upon.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:12:17AM +0200, Yann E. MORIN wrote:
> The goal is to avoid generating a non-buildable kernel in the first place.

I know, that's why I'm proposing to fail the build with the script run
as one of the first things by make.

> > I.e., those steps:
> > 1. make <whatever>config
> > 2. make => A prereq. target runs the shell script.
> 
> This is too late, since this will bail out, and will give a false-positive
> failure.

Huh? This will tell you: "you've selected X, but I don't have toolX
needed for the build".

You install the tool and run make again. Checks pass, the build
continues.

> While if we were to check for _known_ needed tools before calling into
> Kconfig,

How would you do that? How do you know what the user is going to select
*before* she even selects it?

You need to have a prepared .config to act upon.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Florian, All,

On Tuesday 16 July 2013 11:13:24 Florian Fainelli wrote:
> 2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> > On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> >> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> >>
> >> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> >> > >
> >> > > I don't know how to do this.  Any suggestions?
> >> > >
> >> > > It has to be done at `make config' time.  We'd need to probe for the
> >> > > presence of lz4c and then....  what?
> >> > >
> >> > > Is there any precedent for this?
> >> > >
> >> > > I don't think we can just ignore the absence of lz4c - the user has
> >> > > selected a config which his system cannot build.  The problem lies
> >> > > within randconfig itself.
> >> > >
> >> >
> >> > We keep running over the need to be able to have kconfig run tests on
> >> > the build system (for toolchain support or for optional tools needed);
> >> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> >> >
> >>
> >> Would it make sense to extend Kconfig's `depends'?
> >>
> >>       depends on $(shell-command)
> >>
> >> I don't know how practical that would be to implement...
> >
> > Or, easier and faster, run some front-end script which generates
> > once-off Kconfig symbols.
> >
> >         if [ -x /bin/lz4c ]
> >         then
> >                 echo CONFIG_HAVE_LZ4C
> >         fi
> >
> > then munge the output of that script into the Kconfig run and do
> >
> >         depends on HAVE_LZ4C
> 
> That does sound nice to avoid the build error, but will eventually
> make it harder to diagnose why the kernel has not been compressed with
> the specific compression tool, some kind of warning should also be
> emitted maybe?

Either a warning at check-time, or a comment in Kconfig, such as:
    config COMPRESS_LZ4
        depends on HAVE_LZ4
    comment "'lz4' missing, LZ4 compression not available"
        depends on !HAVE_LZ4

Either way is fine with me. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Florian, All,

On Tuesday 16 July 2013 11:13:24 Florian Fainelli wrote:
> 2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> > On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> >> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> >>
> >> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> >> > >
> >> > > I don't know how to do this.  Any suggestions?
> >> > >
> >> > > It has to be done at `make config' time.  We'd need to probe for the
> >> > > presence of lz4c and then....  what?
> >> > >
> >> > > Is there any precedent for this?
> >> > >
> >> > > I don't think we can just ignore the absence of lz4c - the user has
> >> > > selected a config which his system cannot build.  The problem lies
> >> > > within randconfig itself.
> >> > >
> >> >
> >> > We keep running over the need to be able to have kconfig run tests on
> >> > the build system (for toolchain support or for optional tools needed);
> >> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> >> >
> >>
> >> Would it make sense to extend Kconfig's `depends'?
> >>
> >>       depends on $(shell-command)
> >>
> >> I don't know how practical that would be to implement...
> >
> > Or, easier and faster, run some front-end script which generates
> > once-off Kconfig symbols.
> >
> >         if [ -x /bin/lz4c ]
> >         then
> >                 echo CONFIG_HAVE_LZ4C
> >         fi
> >
> > then munge the output of that script into the Kconfig run and do
> >
> >         depends on HAVE_LZ4C
> 
> That does sound nice to avoid the build error, but will eventually
> make it harder to diagnose why the kernel has not been compressed with
> the specific compression tool, some kind of warning should also be
> emitted maybe?

Either a warning at check-time, or a comment in Kconfig, such as:
    config COMPRESS_LZ4
        depends on HAVE_LZ4
    comment "'lz4' missing, LZ4 compression not available"
        depends on !HAVE_LZ4

Either way is fine with me. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Florian, All,

On Tuesday 16 July 2013 11:13:24 Florian Fainelli wrote:
> 2013/7/16 Andrew Morton <akpm@linux-foundation.org>:
> > On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> >> On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> >>
> >> > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> >> > >
> >> > > I don't know how to do this.  Any suggestions?
> >> > >
> >> > > It has to be done at `make config' time.  We'd need to probe for the
> >> > > presence of lz4c and then....  what?
> >> > >
> >> > > Is there any precedent for this?
> >> > >
> >> > > I don't think we can just ignore the absence of lz4c - the user has
> >> > > selected a config which his system cannot build.  The problem lies
> >> > > within randconfig itself.
> >> > >
> >> >
> >> > We keep running over the need to be able to have kconfig run tests on
> >> > the build system (for toolchain support or for optional tools needed);
> >> > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> >> >
> >>
> >> Would it make sense to extend Kconfig's `depends'?
> >>
> >>       depends on $(shell-command)
> >>
> >> I don't know how practical that would be to implement...
> >
> > Or, easier and faster, run some front-end script which generates
> > once-off Kconfig symbols.
> >
> >         if [ -x /bin/lz4c ]
> >         then
> >                 echo CONFIG_HAVE_LZ4C
> >         fi
> >
> > then munge the output of that script into the Kconfig run and do
> >
> >         depends on HAVE_LZ4C
> 
> That does sound nice to avoid the build error, but will eventually
> make it harder to diagnose why the kernel has not been compressed with
> the specific compression tool, some kind of warning should also be
> emitted maybe?

Either a warning at check-time, or a comment in Kconfig, such as:
    config COMPRESS_LZ4
        depends on HAVE_LZ4
    comment "'lz4' missing, LZ4 compression not available"
        depends on !HAVE_LZ4

Either way is fine with me. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:22:42 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 11:12:17AM +0200, Yann E. MORIN wrote:
> > The goal is to avoid generating a non-buildable kernel in the first place.
> 
> I know, that's why I'm proposing to fail the build with the script run
> as one of the first things by make.

No, this is already too late: the given .config *is* already broken and
unbuildable.

> > > I.e., those steps:
> > > 1. make <whatever>config
> > > 2. make => A prereq. target runs the shell script.
> > 
> > This is too late, since this will bail out, and will give a false-positive
> > failure.
> 
> Huh? This will tell you: "you've selected X, but I don't have toolX
> needed for the build".
> 
> You install the tool and run make again. Checks pass, the build
> continues.

That does not work in two cases:
  - older distros that do not have the tool packaged
  - automatic test-harness that run thousands of randconfig a day

In the first case, this /could/ be overcome by the user compiling and
installing the package manually, but is mostly undoable in enterprise
environment with shared build machines, where each user would have to
install the same tool(s) again and again.

In the second case, this would yield a lot of false-positive in daily
reports.

Your solution mostly works with minimum overhead for a human user
interactively building a kernel, not with automated test harnesses.

> > While if we were to check for _known_ needed tools before calling into
> > Kconfig,
> 
> How would you do that? How do you know what the user is going to select
> *before* she even selects it?

The idea is to *avoid* the user being able to select unavailable options.

> You need to have a prepared .config to act upon.

The idea is to be always have a .config that is buildable with the current
toolset of the system, especially for esoteric and/or recent tools that are
not packaged and/or installed by default by the distros.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:22:42 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 11:12:17AM +0200, Yann E. MORIN wrote:
> > The goal is to avoid generating a non-buildable kernel in the first place.
> 
> I know, that's why I'm proposing to fail the build with the script run
> as one of the first things by make.

No, this is already too late: the given .config *is* already broken and
unbuildable.

> > > I.e., those steps:
> > > 1. make <whatever>config
> > > 2. make => A prereq. target runs the shell script.
> > 
> > This is too late, since this will bail out, and will give a false-positive
> > failure.
> 
> Huh? This will tell you: "you've selected X, but I don't have toolX
> needed for the build".
> 
> You install the tool and run make again. Checks pass, the build
> continues.

That does not work in two cases:
  - older distros that do not have the tool packaged
  - automatic test-harness that run thousands of randconfig a day

In the first case, this /could/ be overcome by the user compiling and
installing the package manually, but is mostly undoable in enterprise
environment with shared build machines, where each user would have to
install the same tool(s) again and again.

In the second case, this would yield a lot of false-positive in daily
reports.

Your solution mostly works with minimum overhead for a human user
interactively building a kernel, not with automated test harnesses.

> > While if we were to check for _known_ needed tools before calling into
> > Kconfig,
> 
> How would you do that? How do you know what the user is going to select
> *before* she even selects it?

The idea is to *avoid* the user being able to select unavailable options.

> You need to have a prepared .config to act upon.

The idea is to be always have a .config that is buildable with the current
toolset of the system, especially for esoteric and/or recent tools that are
not packaged and/or installed by default by the distros.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:32:39AM +0200, Yann E. MORIN wrote:
> No, this is already too late: the given .config *is* already broken and
> unbuildable.

Yes, and we don't build it because we fail the build.

> That does not work in two cases:
>   - older distros that do not have the tool packaged

This point is moot - distro kernels are built by the distro people.

>   - automatic test-harness that run thousands of randconfig a day

randconfigs are not guaranteed to build successfully.

> The idea is to be always have a .config that is buildable with the
> current toolset of the system, especially for esoteric and/or recent
> tools that are not packaged and/or installed by default by the
> distros.

How are you going to tell the user running make <something>config to
install the missing tools?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:32:39AM +0200, Yann E. MORIN wrote:
> No, this is already too late: the given .config *is* already broken and
> unbuildable.

Yes, and we don't build it because we fail the build.

> That does not work in two cases:
>   - older distros that do not have the tool packaged

This point is moot - distro kernels are built by the distro people.

>   - automatic test-harness that run thousands of randconfig a day

randconfigs are not guaranteed to build successfully.

> The idea is to be always have a .config that is buildable with the
> current toolset of the system, especially for esoteric and/or recent
> tools that are not packaged and/or installed by default by the
> distros.

How are you going to tell the user running make <something>config to
install the missing tools?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:38:20 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 11:32:39AM +0200, Yann E. MORIN wrote:
> > That does not work in two cases:
> >   - older distros that do not have the tool packaged
> 
> This point is moot - distro kernels are built by the distro people.

It's not about building distro kernels. It's about building the current
kernel on a distro, for example for cross-compilation.

> >   - automatic test-harness that run thousands of randconfig a day
> randconfigs are not guaranteed to build successfully.

Well, mostly agreed. However, there was a request (in this thread) that
using such new tools that are not yet widely available in distros, does
not break randconfig.

This is a *new* requirement.

> > The idea is to be always have a .config that is buildable with the
> > current toolset of the system, especially for esoteric and/or recent
> > tools that are not packaged and/or installed by default by the
> > distros.
> How are you going to tell the user running make <something>config to
> install the missing tools?

As discussed with Florian in this thread, either one of:
  - a warning at check time
  - a comment in Kconfig

which both are easy enough to achieve. As I said earlier in this thread,
I've been using a similar scheme in crosstool-NG (which uses Kconfig as
its configuration infrastructure), and it has been proven effective
over the past feew years.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Borislav, All,

On Tuesday 16 July 2013 11:38:20 Borislav Petkov wrote:
> On Tue, Jul 16, 2013 at 11:32:39AM +0200, Yann E. MORIN wrote:
> > That does not work in two cases:
> >   - older distros that do not have the tool packaged
> 
> This point is moot - distro kernels are built by the distro people.

It's not about building distro kernels. It's about building the current
kernel on a distro, for example for cross-compilation.

> >   - automatic test-harness that run thousands of randconfig a day
> randconfigs are not guaranteed to build successfully.

Well, mostly agreed. However, there was a request (in this thread) that
using such new tools that are not yet widely available in distros, does
not break randconfig.

This is a *new* requirement.

> > The idea is to be always have a .config that is buildable with the
> > current toolset of the system, especially for esoteric and/or recent
> > tools that are not packaged and/or installed by default by the
> > distros.
> How are you going to tell the user running make <something>config to
> install the missing tools?

As discussed with Florian in this thread, either one of:
  - a warning at check time
  - a comment in Kconfig

which both are easy enough to achieve. As I said earlier in this thread,
I've been using a similar scheme in crosstool-NG (which uses Kconfig as
its configuration infrastructure), and it has been proven effective
over the past feew years.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:45:27AM +0200, Yann E. MORIN wrote:
> As discussed with Florian in this thread, either one of:
>   - a warning at check time
>   - a comment in Kconfig

Whatever you do, make sure to make it quite clear to users what she
needs to do.

What I really have a problem with is options being hidden just because I
haven't selected something they depend on and having to run around the
Kconfig maze to find what I also need to select in order for the option
to appear.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Tue, Jul 16, 2013 at 11:45:27AM +0200, Yann E. MORIN wrote:
> As discussed with Florian in this thread, either one of:
>   - a warning at check time
>   - a comment in Kconfig

Whatever you do, make sure to make it quite clear to users what she
needs to do.

What I really have a problem with is options being hidden just because I
haven't selected something they depend on and having to run around the
Kconfig maze to find what I also need to select in order for the option
to appear.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Borislav Petkov <bp@alien8.de>:
> On Tue, Jul 16, 2013 at 11:45:27AM +0200, Yann E. MORIN wrote:
>> As discussed with Florian in this thread, either one of:
>>   - a warning at check time
>>   - a comment in Kconfig
>
> Whatever you do, make sure to make it quite clear to users what she
> needs to do.
>
> What I really have a problem with is options being hidden just because I
> haven't selected something they depend on and having to run around the
> Kconfig maze to find what I also need to select in order for the option
> to appear.

Yes, me too, which is why I suggested the warning, as in "a warning
printed to stdout/stderr" rather than a Kconfig comment which will be
easy to be ignored.
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Borislav Petkov <bp@alien8.de>:
> On Tue, Jul 16, 2013 at 11:45:27AM +0200, Yann E. MORIN wrote:
>> As discussed with Florian in this thread, either one of:
>>   - a warning at check time
>>   - a comment in Kconfig
>
> Whatever you do, make sure to make it quite clear to users what she
> needs to do.
>
> What I really have a problem with is options being hidden just because I
> haven't selected something they depend on and having to run around the
> Kconfig maze to find what I also need to select in order for the option
> to appear.

Yes, me too, which is why I suggested the warning, as in "a warning
printed to stdout/stderr" rather than a Kconfig comment which will be
easy to be ignored.
--
Florian

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Florian Fainelli]

2013/7/16 Borislav Petkov <bp@alien8.de>:
> On Tue, Jul 16, 2013 at 11:45:27AM +0200, Yann E. MORIN wrote:
>> As discussed with Florian in this thread, either one of:
>>   - a warning at check time
>>   - a comment in Kconfig
>
> Whatever you do, make sure to make it quite clear to users what she
> needs to do.
>
> What I really have a problem with is options being hidden just because I
> haven't selected something they depend on and having to run around the
> Kconfig maze to find what I also need to select in order for the option
> to appear.

Yes, me too, which is why I suggested the warning, as in "a warning
printed to stdout/stderr" rather than a Kconfig comment which will be
easy to be ignored.
--
Florian

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

On Tue, Jul 16, 2013 at 10:08:07AM +0200, Yann E. MORIN wrote:
> Andrew, All,
> 
> On Tuesday 16 July 2013 09:56:11 Andrew Morton wrote:
> > On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > > On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> > > > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> [--SNIP--]
> > > > We keep running over the need to be able to have kconfig run tests on
> > > > the build system (for toolchain support or for optional tools needed);
> > > > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > > > 
> > > 
> > > Would it make sense to extend Kconfig's `depends'?
> > > 
> > > 	depends on $(shell-command)
> > > 
> > > I don't know how practical that would be to implement...
> 
> I'm afraid this will get rather ugly and not trivial.
> 
> Can we mix 'depends on SYMBOL' and 'depends on $(command)' ?
> Can we mix both in a boolean expression such as 'depends on SYMBOL
> && $(command)' ?
> 
> What would be the condition for evaluating the dependency rule? Evaluation
> at Kconfig read-time might not be enough, given this construct:
> 
>     config FOO
>         depends on $(foo)
>     comment "'foo' is missing, please install it"
>         depends on !$(foo)
> 
> Also, I believe Kconfig should stay a config-only language, without
> much esoteric features.

We could extend the symbol option part to retreive values from a binary.
Something like this:

config FOOBAR
        bool
        option exec="true"

FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
And similar conversions for other types.

This only extendt Kconfig slightly - using an already present method to import
external values.

The drawback I see with this approach is that we may execute a lot of small programs
where the value is never used.
Implementing lazy evaluation of a symbol value will not be easy I think.

	Sam

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

On Tue, Jul 16, 2013 at 10:08:07AM +0200, Yann E. MORIN wrote:
> Andrew, All,
> 
> On Tuesday 16 July 2013 09:56:11 Andrew Morton wrote:
> > On Tue, 16 Jul 2013 00:47:27 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > > On Mon, 15 Jul 2013 15:08:20 -0700 "H. Peter Anvin" <hpa@zytor.com> wrote:
> > > > On 07/15/2013 03:03 PM, Andrew Morton wrote:
> [--SNIP--]
> > > > We keep running over the need to be able to have kconfig run tests on
> > > > the build system (for toolchain support or for optional tools needed);
> > > > running them in the Makefiles (i.e. at Kbuild time) is simply too late.
> > > > 
> > > 
> > > Would it make sense to extend Kconfig's `depends'?
> > > 
> > > 	depends on $(shell-command)
> > > 
> > > I don't know how practical that would be to implement...
> 
> I'm afraid this will get rather ugly and not trivial.
> 
> Can we mix 'depends on SYMBOL' and 'depends on $(command)' ?
> Can we mix both in a boolean expression such as 'depends on SYMBOL
> && $(command)' ?
> 
> What would be the condition for evaluating the dependency rule? Evaluation
> at Kconfig read-time might not be enough, given this construct:
> 
>     config FOO
>         depends on $(foo)
>     comment "'foo' is missing, please install it"
>         depends on !$(foo)
> 
> Also, I believe Kconfig should stay a config-only language, without
> much esoteric features.

We could extend the symbol option part to retreive values from a binary.
Something like this:

config FOOBAR
        bool
        option exec="true"

FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
And similar conversions for other types.

This only extendt Kconfig slightly - using an already present method to import
external values.

The drawback I see with this approach is that we may execute a lot of small programs
where the value is never used.
Implementing lazy evaluation of a symbol value will not be easy I think.

	Sam

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

> 
> We could extend the symbol option part to retreive values from a binary.
> Something like this:
> 
> config FOOBAR
>         bool
>         option exec="true"
> 
> FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> And similar conversions for other types.
> 
> This only extendt Kconfig slightly - using an already present method to import
> external values.
> 
> The drawback I see with this approach is that we may execute a lot of small programs
> where the value is never used.

Following is a quick patch implmenting this idea.
You need to run gperf manually to enable this.

"gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"

I did not figure out how to use the built-in rules to generate this file :-(

I have tested this lightly - as we should discuss if this is a viable way forward.
For now I used popen() - so return value of the executed program are ignored.

To test this I used:

config FOOBAR
	bool
	option exec="echo y"

config FOOBAR_SELECTOR
	bool
	default FOOBAR

Nothing fancy - but it shows the idea.

	Sam


diff --git a/scripts/kconfig/expr.h b/scripts/kconfig/expr.h
index df198a5..a68258b 100644
--- a/scripts/kconfig/expr.h
+++ b/scripts/kconfig/expr.h
@@ -133,6 +133,7 @@ enum prop_type {
 	P_SELECT,   /* select BAR */
 	P_RANGE,    /* range 7..100 (for a symbol) */
 	P_ENV,      /* value from environment variable */
+	P_EXEC,     /* value from executable (using symbol option) */
 	P_SYMBOL,   /* where a symbol is defined */
 };
 
diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h
index 09f4edf..371f73b 100644
--- a/scripts/kconfig/lkc.h
+++ b/scripts/kconfig/lkc.h
@@ -61,6 +61,7 @@ enum conf_def_mode {
 #define T_OPT_MODULES		1
 #define T_OPT_DEFCONFIG_LIST	2
 #define T_OPT_ENV		3
+#define T_OPT_EXEC		4
 
 struct kconf_id {
 	int name;
diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c
index 7e233a6..cfa7aaa 100644
--- a/scripts/kconfig/menu.c
+++ b/scripts/kconfig/menu.c
@@ -213,6 +213,9 @@ void menu_add_option(int token, char *arg)
 	case T_OPT_ENV:
 		prop_add_env(arg);
 		break;
+	case T_OPT_EXEC:
+		prop_add_exec(arg);
+		break;
 	}
 }
 
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index d550300..b7179a6 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -1331,6 +1331,8 @@ const char *prop_get_type_name(enum prop_type type)
 		return "prompt";
 	case P_ENV:
 		return "env";
+	case P_EXEC:
+		return "exec";
 	case P_COMMENT:
 		return "comment";
 	case P_MENU:
@@ -1379,3 +1381,56 @@ static void prop_add_env(const char *env)
 	else
 		menu_warn(current_entry, "environment variable %s undefined", env);
 }
+
+static void exec_command(const char *command, struct symbol *sym)
+{
+	char buffer[2048];
+	FILE *stream;
+
+	stream = popen(command, "r");
+
+	if (stream != NULL) {
+		if (fgets(buffer, sizeof(buffer), stream) != NULL) {
+			int i;
+
+			buffer[sizeof(buffer) - 1] = '\0';
+
+			/* Drop any trialing newlines */
+			i = strlen(buffer);
+			while (i > 0 && buffer[i - 1] == '\n') {
+				buffer[i - 1] = '\0';
+				i--;
+			}
+			/* Validate the output of the command */
+			if (strlen(buffer) == 0) {
+				menu_warn(current_entry,
+				          "command '%s' - invalid (empty?) return value: \"%s\"",
+				          command, buffer);
+				return;
+			}
+
+			menu_warn(current_entry, "default: %s", buffer);
+			sym_add_default(sym, buffer);
+		} else {
+			menu_warn(current_entry, "command '%s' - empty return value", command);
+		}
+		pclose(stream);
+	} else {
+		menu_warn(current_entry, "command '%s' failed to execute", command);
+	}
+}
+
+static void prop_add_exec(const char *command)
+{
+	struct property *prop;
+	struct symbol *sym;
+
+	sym = current_entry->sym;
+	sym->flags |= SYMBOL_AUTO;
+
+	prop = prop_alloc(P_EXEC, sym);
+	prop->expr = expr_alloc_symbol(sym_lookup(command, SYMBOL_CONST));
+
+	exec_command(command, sym);
+}
+
diff --git a/scripts/kconfig/zconf.gperf b/scripts/kconfig/zconf.gperf
index f14ab41..02c6a59 100644
--- a/scripts/kconfig/zconf.gperf
+++ b/scripts/kconfig/zconf.gperf
@@ -44,4 +44,5 @@ on,		T_ON,		TF_PARAM
 modules,	T_OPT_MODULES,	TF_OPTION
 defconfig_list,	T_OPT_DEFCONFIG_LIST,TF_OPTION
 env,		T_OPT_ENV,	TF_OPTION
+exec,		T_OPT_EXEC,	TF_OPTION
 %%

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

> 
> We could extend the symbol option part to retreive values from a binary.
> Something like this:
> 
> config FOOBAR
>         bool
>         option exec="true"
> 
> FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> And similar conversions for other types.
> 
> This only extendt Kconfig slightly - using an already present method to import
> external values.
> 
> The drawback I see with this approach is that we may execute a lot of small programs
> where the value is never used.

Following is a quick patch implmenting this idea.
You need to run gperf manually to enable this.

"gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"

I did not figure out how to use the built-in rules to generate this file :-(

I have tested this lightly - as we should discuss if this is a viable way forward.
For now I used popen() - so return value of the executed program are ignored.

To test this I used:

config FOOBAR
	bool
	option exec="echo y"

config FOOBAR_SELECTOR
	bool
	default FOOBAR

Nothing fancy - but it shows the idea.

	Sam


diff --git a/scripts/kconfig/expr.h b/scripts/kconfig/expr.h
index df198a5..a68258b 100644
--- a/scripts/kconfig/expr.h
+++ b/scripts/kconfig/expr.h
@@ -133,6 +133,7 @@ enum prop_type {
 	P_SELECT,   /* select BAR */
 	P_RANGE,    /* range 7..100 (for a symbol) */
 	P_ENV,      /* value from environment variable */
+	P_EXEC,     /* value from executable (using symbol option) */
 	P_SYMBOL,   /* where a symbol is defined */
 };
 
diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h
index 09f4edf..371f73b 100644
--- a/scripts/kconfig/lkc.h
+++ b/scripts/kconfig/lkc.h
@@ -61,6 +61,7 @@ enum conf_def_mode {
 #define T_OPT_MODULES		1
 #define T_OPT_DEFCONFIG_LIST	2
 #define T_OPT_ENV		3
+#define T_OPT_EXEC		4
 
 struct kconf_id {
 	int name;
diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c
index 7e233a6..cfa7aaa 100644
--- a/scripts/kconfig/menu.c
+++ b/scripts/kconfig/menu.c
@@ -213,6 +213,9 @@ void menu_add_option(int token, char *arg)
 	case T_OPT_ENV:
 		prop_add_env(arg);
 		break;
+	case T_OPT_EXEC:
+		prop_add_exec(arg);
+		break;
 	}
 }
 
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index d550300..b7179a6 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -1331,6 +1331,8 @@ const char *prop_get_type_name(enum prop_type type)
 		return "prompt";
 	case P_ENV:
 		return "env";
+	case P_EXEC:
+		return "exec";
 	case P_COMMENT:
 		return "comment";
 	case P_MENU:
@@ -1379,3 +1381,56 @@ static void prop_add_env(const char *env)
 	else
 		menu_warn(current_entry, "environment variable %s undefined", env);
 }
+
+static void exec_command(const char *command, struct symbol *sym)
+{
+	char buffer[2048];
+	FILE *stream;
+
+	stream = popen(command, "r");
+
+	if (stream != NULL) {
+		if (fgets(buffer, sizeof(buffer), stream) != NULL) {
+			int i;
+
+			buffer[sizeof(buffer) - 1] = '\0';
+
+			/* Drop any trialing newlines */
+			i = strlen(buffer);
+			while (i > 0 && buffer[i - 1] == '\n') {
+				buffer[i - 1] = '\0';
+				i--;
+			}
+			/* Validate the output of the command */
+			if (strlen(buffer) == 0) {
+				menu_warn(current_entry,
+				          "command '%s' - invalid (empty?) return value: \"%s\"",
+				          command, buffer);
+				return;
+			}
+
+			menu_warn(current_entry, "default: %s", buffer);
+			sym_add_default(sym, buffer);
+		} else {
+			menu_warn(current_entry, "command '%s' - empty return value", command);
+		}
+		pclose(stream);
+	} else {
+		menu_warn(current_entry, "command '%s' failed to execute", command);
+	}
+}
+
+static void prop_add_exec(const char *command)
+{
+	struct property *prop;
+	struct symbol *sym;
+
+	sym = current_entry->sym;
+	sym->flags |= SYMBOL_AUTO;
+
+	prop = prop_alloc(P_EXEC, sym);
+	prop->expr = expr_alloc_symbol(sym_lookup(command, SYMBOL_CONST));
+
+	exec_command(command, sym);
+}
+
diff --git a/scripts/kconfig/zconf.gperf b/scripts/kconfig/zconf.gperf
index f14ab41..02c6a59 100644
--- a/scripts/kconfig/zconf.gperf
+++ b/scripts/kconfig/zconf.gperf
@@ -44,4 +44,5 @@ on,		T_ON,		TF_PARAM
 modules,	T_OPT_MODULES,	TF_OPTION
 defconfig_list,	T_OPT_DEFCONFIG_LIST,TF_OPTION
 env,		T_OPT_ENV,	TF_OPTION
+exec,		T_OPT_EXEC,	TF_OPTION
 %%

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 17, 2013 at 11:16:49PM +0200, Sam Ravnborg wrote:
> +
> +static void exec_command(const char *command, struct symbol *sym)
> +{
> +	char buffer[2048];
> +	FILE *stream;

Just some indentation level saving:

> +
> +	stream = popen(command, "r");
> +
> +	if (stream != NULL) {

	if (!stream) {
		menu_warn(current_entry, "command '%s' failed to execute", command);
		return;
	}

and the rest starts one level less to the right:

	if (fgets(buffer, sizeof(buffer), stream) != NULL) {
		int i;

		buffer[sizeof(buffer) - 1] = '\0';

and so on...

> +
> +			/* Drop any trialing newlines */
> +			i = strlen(buffer);
> +			while (i > 0 && buffer[i - 1] == '\n') {
> +				buffer[i - 1] = '\0';
> +				i--;
> +			}
> +			/* Validate the output of the command */
> +			if (strlen(buffer) == 0) {
> +				menu_warn(current_entry,
> +				          "command '%s' - invalid (empty?) return value: \"%s\"",
> +				          command, buffer);
> +				return;
> +			}
> +
> +			menu_warn(current_entry, "default: %s", buffer);
> +			sym_add_default(sym, buffer);
> +		} else {
> +			menu_warn(current_entry, "command '%s' - empty return value", command);
> +		}
> +		pclose(stream);
> +	} else {
> +		menu_warn(current_entry, "command '%s' failed to execute", command);
> +	}
> +}

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Borislav Petkov]

On Wed, Jul 17, 2013 at 11:16:49PM +0200, Sam Ravnborg wrote:
> +
> +static void exec_command(const char *command, struct symbol *sym)
> +{
> +	char buffer[2048];
> +	FILE *stream;

Just some indentation level saving:

> +
> +	stream = popen(command, "r");
> +
> +	if (stream != NULL) {

	if (!stream) {
		menu_warn(current_entry, "command '%s' failed to execute", command);
		return;
	}

and the rest starts one level less to the right:

	if (fgets(buffer, sizeof(buffer), stream) != NULL) {
		int i;

		buffer[sizeof(buffer) - 1] = '\0';

and so on...

> +
> +			/* Drop any trialing newlines */
> +			i = strlen(buffer);
> +			while (i > 0 && buffer[i - 1] == '\n') {
> +				buffer[i - 1] = '\0';
> +				i--;
> +			}
> +			/* Validate the output of the command */
> +			if (strlen(buffer) == 0) {
> +				menu_warn(current_entry,
> +				          "command '%s' - invalid (empty?) return value: \"%s\"",
> +				          command, buffer);
> +				return;
> +			}
> +
> +			menu_warn(current_entry, "default: %s", buffer);
> +			sym_add_default(sym, buffer);
> +		} else {
> +			menu_warn(current_entry, "command '%s' - empty return value", command);
> +		}
> +		pclose(stream);
> +	} else {
> +		menu_warn(current_entry, "command '%s' failed to execute", command);
> +	}
> +}

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Sam, All,

Well, while I was fighting on this on my side... ;-)

On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > We could extend the symbol option part to retreive values from a binary.
> > Something like this:
> > 
> > config FOOBAR
> >         bool
> >         option exec="true"
> > 
> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > And similar conversions for other types.
> > 
> > This only extendt Kconfig slightly - using an already present method to import
> > external values.
> > 
> > The drawback I see with this approach is that we may execute a lot of small programs
> > where the value is never used.
> 
> Following is a quick patch implmenting this idea.
> You need to run gperf manually to enable this.
> 
> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> 
> I did not figure out how to use the built-in rules to generate this file :-(

make REGENERATE_PARSERS=y menuconfig

> I have tested this lightly - as we should discuss if this is a viable way forward.

Instead of extending the Kconfig language, I was thinking (as initially
suggested by Andrew) of generating a Kconfig file before all config
targets, and source that Kconfig file from $(TOPDIR)/Kconfig.

I'm not a fan od extending the Kconfig language in this way. It feels
weird to me. Especially when we have a way to solve this specific issue
with a script that generates the needed Kconfig symbols before any
config target is called.

At the very least we'd have to decide if this is available only for
booleans/tristates, or for any type? Your implementation seems to make
it valid for strings/ints, too.

> For now I used popen() - so return value of the executed program are ignored.

Yet, pclose returns the exit status of the command.

If option exec was to be valid only for booleans, we could use the exit
value rather than the output, which would seem more logical. It would be
however a bit moretriclky for tristates, though...

Maybe using output is the best, after all...

So, how do you expect we use that to check for an external tool?
Something like this?
    config HAVE_LZ4
        bool
        option exec="which lz4c >/dev/null 2>&1 && echo y"

    config KERNEL_LZ4
        bool "compress the kernel with lz4"
        depends on HAVE_LZ4

Ot did I miss something?

> diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
> index d550300..b7179a6 100644
> --- a/scripts/kconfig/symbol.c
> +++ b/scripts/kconfig/symbol.c
[--SNIP--]
> @@ -1379,3 +1381,56 @@ static void prop_add_env(const char *env)
>  	else
>  		menu_warn(current_entry, "environment variable %s undefined", env);
>  }
> +
> +static void exec_command(const char *command, struct symbol *sym)
> +{
> +	char buffer[2048];
> +	FILE *stream;
> +
> +	stream = popen(command, "r");
> +
> +	if (stream != NULL) {
> +		if (fgets(buffer, sizeof(buffer), stream) != NULL) {
> +			int i;
> +
> +			buffer[sizeof(buffer) - 1] = '\0';
> +
> +			/* Drop any trialing newlines */
> +			i = strlen(buffer);
> +			while (i > 0 && buffer[i - 1] == '\n') {
> +				buffer[i - 1] = '\0';
> +				i--;
> +			}
> +			/* Validate the output of the command */
> +			if (strlen(buffer) == 0) {
> +				menu_warn(current_entry,
> +				          "command '%s' - invalid (empty?) return value: \"%s\"",
> +				          command, buffer);
> +				return;

Missing pclose() before return.

[--SNIP--]

I'll give it a spin here. Thank you!

BTW, I still need your help on solving that one:
    randconfig does randomise choice entries, unless KCONFIG_ALLCONFIG
    is specified.
    http://marc.info/?l=linux-kernel&m=137209757414433&w=2

My two previous attempts failed due to introducing regressions, so they
were both reverted... :-(
Any suggestion? ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Sam, All,

Well, while I was fighting on this on my side... ;-)

On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > We could extend the symbol option part to retreive values from a binary.
> > Something like this:
> > 
> > config FOOBAR
> >         bool
> >         option exec="true"
> > 
> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > And similar conversions for other types.
> > 
> > This only extendt Kconfig slightly - using an already present method to import
> > external values.
> > 
> > The drawback I see with this approach is that we may execute a lot of small programs
> > where the value is never used.
> 
> Following is a quick patch implmenting this idea.
> You need to run gperf manually to enable this.
> 
> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> 
> I did not figure out how to use the built-in rules to generate this file :-(

make REGENERATE_PARSERS=y menuconfig

> I have tested this lightly - as we should discuss if this is a viable way forward.

Instead of extending the Kconfig language, I was thinking (as initially
suggested by Andrew) of generating a Kconfig file before all config
targets, and source that Kconfig file from $(TOPDIR)/Kconfig.

I'm not a fan od extending the Kconfig language in this way. It feels
weird to me. Especially when we have a way to solve this specific issue
with a script that generates the needed Kconfig symbols before any
config target is called.

At the very least we'd have to decide if this is available only for
booleans/tristates, or for any type? Your implementation seems to make
it valid for strings/ints, too.

> For now I used popen() - so return value of the executed program are ignored.

Yet, pclose returns the exit status of the command.

If option exec was to be valid only for booleans, we could use the exit
value rather than the output, which would seem more logical. It would be
however a bit moretriclky for tristates, though...

Maybe using output is the best, after all...

So, how do you expect we use that to check for an external tool?
Something like this?
    config HAVE_LZ4
        bool
        option exec="which lz4c >/dev/null 2>&1 && echo y"

    config KERNEL_LZ4
        bool "compress the kernel with lz4"
        depends on HAVE_LZ4

Ot did I miss something?

> diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
> index d550300..b7179a6 100644
> --- a/scripts/kconfig/symbol.c
> +++ b/scripts/kconfig/symbol.c
[--SNIP--]
> @@ -1379,3 +1381,56 @@ static void prop_add_env(const char *env)
>  	else
>  		menu_warn(current_entry, "environment variable %s undefined", env);
>  }
> +
> +static void exec_command(const char *command, struct symbol *sym)
> +{
> +	char buffer[2048];
> +	FILE *stream;
> +
> +	stream = popen(command, "r");
> +
> +	if (stream != NULL) {
> +		if (fgets(buffer, sizeof(buffer), stream) != NULL) {
> +			int i;
> +
> +			buffer[sizeof(buffer) - 1] = '\0';
> +
> +			/* Drop any trialing newlines */
> +			i = strlen(buffer);
> +			while (i > 0 && buffer[i - 1] == '\n') {
> +				buffer[i - 1] = '\0';
> +				i--;
> +			}
> +			/* Validate the output of the command */
> +			if (strlen(buffer) == 0) {
> +				menu_warn(current_entry,
> +				          "command '%s' - invalid (empty?) return value: \"%s\"",
> +				          command, buffer);
> +				return;

Missing pclose() before return.

[--SNIP--]

I'll give it a spin here. Thank you!

BTW, I still need your help on solving that one:
    randconfig does randomise choice entries, unless KCONFIG_ALLCONFIG
    is specified.
    http://marc.info/?l=linux-kernel&m=137209757414433&w=2

My two previous attempts failed due to introducing regressions, so they
were both reverted... :-(
Any suggestion? ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[H. Peter Anvin]

On 07/17/2013 03:30 PM, Yann E. MORIN wrote:
> 
> At the very least we'd have to decide if this is available only for
> booleans/tristates, or for any type? Your implementation seems to make
> it valid for strings/ints, too.
> 

I feel the test should *export* a boolean.

It is worth noting that one of the things that keep getting brought up
as something that need this is gcc/binutils support for a specific feature.

	-hpa

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[H. Peter Anvin]

On 07/17/2013 03:30 PM, Yann E. MORIN wrote:
> 
> At the very least we'd have to decide if this is available only for
> booleans/tristates, or for any type? Your implementation seems to make
> it valid for strings/ints, too.
> 

I feel the test should *export* a boolean.

It is worth noting that one of the things that keep getting brought up
as something that need this is gcc/binutils support for a specific feature.

	-hpa

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
>> > We could extend the symbol option part to retreive values from a binary.
>> > Something like this:
>> >
>> > config FOOBAR
>> >         bool
>> >         option exec="true"
>> >
>> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
>> > And similar conversions for other types.
>> >
>> > This only extendt Kconfig slightly - using an already present method to import
>> > external values.
>> >
>> > The drawback I see with this approach is that we may execute a lot of small programs
>> > where the value is never used.
>>
>> Following is a quick patch implmenting this idea.
>> You need to run gperf manually to enable this.
>>
>> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
>>
>> I did not figure out how to use the built-in rules to generate this file :-(
>
> make REGENERATE_PARSERS=y menuconfig
>
>> I have tested this lightly - as we should discuss if this is a viable way forward.
>
> Instead of extending the Kconfig language, I was thinking (as initially
> suggested by Andrew) of generating a Kconfig file before all config
> targets, and source that Kconfig file from $(TOPDIR)/Kconfig.

I also prefer the generated Kconfig file.
It keeps all these checks in a single place, instead of spreading it over all
Kconfig files. This allows to keep better control over the list of checks, and
notice when it gets out-of-hand.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
>> > We could extend the symbol option part to retreive values from a binary.
>> > Something like this:
>> >
>> > config FOOBAR
>> >         bool
>> >         option exec="true"
>> >
>> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
>> > And similar conversions for other types.
>> >
>> > This only extendt Kconfig slightly - using an already present method to import
>> > external values.
>> >
>> > The drawback I see with this approach is that we may execute a lot of small programs
>> > where the value is never used.
>>
>> Following is a quick patch implmenting this idea.
>> You need to run gperf manually to enable this.
>>
>> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
>>
>> I did not figure out how to use the built-in rules to generate this file :-(
>
> make REGENERATE_PARSERS=y menuconfig
>
>> I have tested this lightly - as we should discuss if this is a viable way forward.
>
> Instead of extending the Kconfig language, I was thinking (as initially
> suggested by Andrew) of generating a Kconfig file before all config
> targets, and source that Kconfig file from $(TOPDIR)/Kconfig.

I also prefer the generated Kconfig file.
It keeps all these checks in a single place, instead of spreading it over all
Kconfig files. This allows to keep better control over the list of checks, and
notice when it gets out-of-hand.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
>> > We could extend the symbol option part to retreive values from a binary.
>> > Something like this:
>> >
>> > config FOOBAR
>> >         bool
>> >         option exec="true"
>> >
>> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
>> > And similar conversions for other types.
>> >
>> > This only extendt Kconfig slightly - using an already present method to import
>> > external values.
>> >
>> > The drawback I see with this approach is that we may execute a lot of small programs
>> > where the value is never used.
>>
>> Following is a quick patch implmenting this idea.
>> You need to run gperf manually to enable this.
>>
>> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
>>
>> I did not figure out how to use the built-in rules to generate this file :-(
>
> make REGENERATE_PARSERS=y menuconfig
>
>> I have tested this lightly - as we should discuss if this is a viable way forward.
>
> Instead of extending the Kconfig language, I was thinking (as initially
> suggested by Andrew) of generating a Kconfig file before all config
> targets, and source that Kconfig file from $(TOPDIR)/Kconfig.

I also prefer the generated Kconfig file.
It keeps all these checks in a single place, instead of spreading it over all
Kconfig files. This allows to keep better control over the list of checks, and
notice when it gets out-of-hand.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> >> > We could extend the symbol option part to retreive values from a binary.
> >> > Something like this:
> >> >
> >> > config FOOBAR
> >> >         bool
> >> >         option exec="true"
> >> >
> >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> >> > And similar conversions for other types.
> >> >
> >> > This only extendt Kconfig slightly - using an already present method to import
> >> > external values.
> >> >
> >> > The drawback I see with this approach is that we may execute a lot of small programs
> >> > where the value is never used.
> >>
> >> Following is a quick patch implmenting this idea.
> >> You need to run gperf manually to enable this.
> >>
> >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> >>
> >> I did not figure out how to use the built-in rules to generate this file :-(
> >
> > make REGENERATE_PARSERS=y menuconfig
> >
> >> I have tested this lightly - as we should discuss if this is a viable way forward.
> >
> > Instead of extending the Kconfig language, I was thinking (as initially
> > suggested by Andrew) of generating a Kconfig file before all config
> > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> 
> I also prefer the generated Kconfig file.
> It keeps all these checks in a single place, instead of spreading it over all
> Kconfig files. This allows to keep better control over the list of checks, and
> notice when it gets out-of-hand.

I prefer the "option exec" approach, actually.  That way the shell-outs
are colocated with the code which uses them and they will only be executed
if you've actually selected that subsystem for building (I think?).

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> >> > We could extend the symbol option part to retreive values from a binary.
> >> > Something like this:
> >> >
> >> > config FOOBAR
> >> >         bool
> >> >         option exec="true"
> >> >
> >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> >> > And similar conversions for other types.
> >> >
> >> > This only extendt Kconfig slightly - using an already present method to import
> >> > external values.
> >> >
> >> > The drawback I see with this approach is that we may execute a lot of small programs
> >> > where the value is never used.
> >>
> >> Following is a quick patch implmenting this idea.
> >> You need to run gperf manually to enable this.
> >>
> >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> >>
> >> I did not figure out how to use the built-in rules to generate this file :-(
> >
> > make REGENERATE_PARSERS=y menuconfig
> >
> >> I have tested this lightly - as we should discuss if this is a viable way forward.
> >
> > Instead of extending the Kconfig language, I was thinking (as initially
> > suggested by Andrew) of generating a Kconfig file before all config
> > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> 
> I also prefer the generated Kconfig file.
> It keeps all these checks in a single place, instead of spreading it over all
> Kconfig files. This allows to keep better control over the list of checks, and
> notice when it gets out-of-hand.

I prefer the "option exec" approach, actually.  That way the shell-outs
are colocated with the code which uses them and they will only be executed
if you've actually selected that subsystem for building (I think?).

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> >> > We could extend the symbol option part to retreive values from a binary.
> >> > Something like this:
> >> >
> >> > config FOOBAR
> >> >         bool
> >> >         option exec="true"
> >> >
> >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> >> > And similar conversions for other types.
> >> >
> >> > This only extendt Kconfig slightly - using an already present method to import
> >> > external values.
> >> >
> >> > The drawback I see with this approach is that we may execute a lot of small programs
> >> > where the value is never used.
> >>
> >> Following is a quick patch implmenting this idea.
> >> You need to run gperf manually to enable this.
> >>
> >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> >>
> >> I did not figure out how to use the built-in rules to generate this file :-(
> >
> > make REGENERATE_PARSERS=y menuconfig
> >
> >> I have tested this lightly - as we should discuss if this is a viable way forward.
> >
> > Instead of extending the Kconfig language, I was thinking (as initially
> > suggested by Andrew) of generating a Kconfig file before all config
> > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> 
> I also prefer the generated Kconfig file.
> It keeps all these checks in a single place, instead of spreading it over all
> Kconfig files. This allows to keep better control over the list of checks, and
> notice when it gets out-of-hand.

I prefer the "option exec" approach, actually.  That way the shell-outs
are colocated with the code which uses them and they will only be executed
if you've actually selected that subsystem for building (I think?).

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Andrew, All,

On Thursday 18 July 2013 09:34:08 Andrew Morton wrote:
> On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > >> > We could extend the symbol option part to retreive values from a binary.
> > >> > Something like this:
> > >> >
> > >> > config FOOBAR
> > >> >         bool
> > >> >         option exec="true"
> > >> >
> > >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > >> > And similar conversions for other types.
> > >> >
> > >> > This only extendt Kconfig slightly - using an already present method to import
> > >> > external values.
> > >> Following is a quick patch implmenting this idea.
> > >> You need to run gperf manually to enable this.
> > >>
> > >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> > >>
> > >> I did not figure out how to use the built-in rules to generate this file :-(
> > >
> > > make REGENERATE_PARSERS=y menuconfig
> > >
> > >> I have tested this lightly - as we should discuss if this is a viable way forward.
> > >
> > > Instead of extending the Kconfig language, I was thinking (as initially
> > > suggested by Andrew) of generating a Kconfig file before all config
> > > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> > 
> > I also prefer the generated Kconfig file.
> > It keeps all these checks in a single place, instead of spreading it over all
> > Kconfig files. This allows to keep better control over the list of checks, and
> > notice when it gets out-of-hand.
> 
> I prefer the "option exec" approach, actually.  That way the shell-outs
> are colocated with the code which uses

Indeed, but in this case, all the checks will be spread-out in the Kconfig
files, and not easily locatable. Having all in a single script will also
more easily raise eyebrows when that script appears in a diffstat. Noticing
the 'exec' option risks being a bit less easy.

But, that's not my call to decide. ;-)

> them and they will only be executed
> if you've actually selected that subsystem for building (I think?).

If I understand the code correctly (which is still to be proven), the
exec option is run at parse-time, once.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Andrew, All,

On Thursday 18 July 2013 09:34:08 Andrew Morton wrote:
> On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > >> > We could extend the symbol option part to retreive values from a binary.
> > >> > Something like this:
> > >> >
> > >> > config FOOBAR
> > >> >         bool
> > >> >         option exec="true"
> > >> >
> > >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > >> > And similar conversions for other types.
> > >> >
> > >> > This only extendt Kconfig slightly - using an already present method to import
> > >> > external values.
> > >> Following is a quick patch implmenting this idea.
> > >> You need to run gperf manually to enable this.
> > >>
> > >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> > >>
> > >> I did not figure out how to use the built-in rules to generate this file :-(
> > >
> > > make REGENERATE_PARSERS=y menuconfig
> > >
> > >> I have tested this lightly - as we should discuss if this is a viable way forward.
> > >
> > > Instead of extending the Kconfig language, I was thinking (as initially
> > > suggested by Andrew) of generating a Kconfig file before all config
> > > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> > 
> > I also prefer the generated Kconfig file.
> > It keeps all these checks in a single place, instead of spreading it over all
> > Kconfig files. This allows to keep better control over the list of checks, and
> > notice when it gets out-of-hand.
> 
> I prefer the "option exec" approach, actually.  That way the shell-outs
> are colocated with the code which uses

Indeed, but in this case, all the checks will be spread-out in the Kconfig
files, and not easily locatable. Having all in a single script will also
more easily raise eyebrows when that script appears in a diffstat. Noticing
the 'exec' option risks being a bit less easy.

But, that's not my call to decide. ;-)

> them and they will only be executed
> if you've actually selected that subsystem for building (I think?).

If I understand the code correctly (which is still to be proven), the
exec option is run at parse-time, once.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Yann E. MORIN]

Andrew, All,

On Thursday 18 July 2013 09:34:08 Andrew Morton wrote:
> On Thu, 18 Jul 2013 09:22:58 +0200 Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Thu, Jul 18, 2013 at 12:30 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > > On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > >> > We could extend the symbol option part to retreive values from a binary.
> > >> > Something like this:
> > >> >
> > >> > config FOOBAR
> > >> >         bool
> > >> >         option exec="true"
> > >> >
> > >> > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > >> > And similar conversions for other types.
> > >> >
> > >> > This only extendt Kconfig slightly - using an already present method to import
> > >> > external values.
> > >> Following is a quick patch implmenting this idea.
> > >> You need to run gperf manually to enable this.
> > >>
> > >> "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> > >>
> > >> I did not figure out how to use the built-in rules to generate this file :-(
> > >
> > > make REGENERATE_PARSERS=y menuconfig
> > >
> > >> I have tested this lightly - as we should discuss if this is a viable way forward.
> > >
> > > Instead of extending the Kconfig language, I was thinking (as initially
> > > suggested by Andrew) of generating a Kconfig file before all config
> > > targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> > 
> > I also prefer the generated Kconfig file.
> > It keeps all these checks in a single place, instead of spreading it over all
> > Kconfig files. This allows to keep better control over the list of checks, and
> > notice when it gets out-of-hand.
> 
> I prefer the "option exec" approach, actually.  That way the shell-outs
> are colocated with the code which uses

Indeed, but in this case, all the checks will be spread-out in the Kconfig
files, and not easily locatable. Having all in a single script will also
more easily raise eyebrows when that script appears in a diffstat. Noticing
the 'exec' option risks being a bit less easy.

But, that's not my call to decide. ;-)

> them and they will only be executed
> if you've actually selected that subsystem for building (I think?).

If I understand the code correctly (which is still to be proven), the
exec option is run at parse-time, once.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:47:02 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > 
> > I prefer the "option exec" approach, actually.  That way the shell-outs
> > are colocated with the code which uses
> 
> Indeed, but in this case, all the checks will be spread-out in the Kconfig
> files, and not easily locatable. Having all in a single script will also
> more easily raise eyebrows when that script appears in a diffstat. Noticing
> the 'exec' option risks being a bit less easy.

Eh, there are a million ways to screw up the kernel.

> But, that's not my call to decide. ;-)
> 
> > them and they will only be executed
> > if you've actually selected that subsystem for building (I think?).
> 
> If I understand the code correctly (which is still to be proven), the
> exec option is run at parse-time, once.

Yes, but not every Kconfig file in the tree gets parsed every time. 
For example, if arch/arm/Kconfig uses "option exec=some-slow-thing",
x86 users won't execute some-slow-thing.  This is good.

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:47:02 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > 
> > I prefer the "option exec" approach, actually.  That way the shell-outs
> > are colocated with the code which uses
> 
> Indeed, but in this case, all the checks will be spread-out in the Kconfig
> files, and not easily locatable. Having all in a single script will also
> more easily raise eyebrows when that script appears in a diffstat. Noticing
> the 'exec' option risks being a bit less easy.

Eh, there are a million ways to screw up the kernel.

> But, that's not my call to decide. ;-)
> 
> > them and they will only be executed
> > if you've actually selected that subsystem for building (I think?).
> 
> If I understand the code correctly (which is still to be proven), the
> exec option is run at parse-time, once.

Yes, but not every Kconfig file in the tree gets parsed every time. 
For example, if arch/arm/Kconfig uses "option exec=some-slow-thing",
x86 users won't execute some-slow-thing.  This is good.

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Andrew Morton]

On Thu, 18 Jul 2013 09:47:02 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > 
> > I prefer the "option exec" approach, actually.  That way the shell-outs
> > are colocated with the code which uses
> 
> Indeed, but in this case, all the checks will be spread-out in the Kconfig
> files, and not easily locatable. Having all in a single script will also
> more easily raise eyebrows when that script appears in a diffstat. Noticing
> the 'exec' option risks being a bit less easy.

Eh, there are a million ways to screw up the kernel.

> But, that's not my call to decide. ;-)
> 
> > them and they will only be executed
> > if you've actually selected that subsystem for building (I think?).
> 
> If I understand the code correctly (which is still to be proven), the
> exec option is run at parse-time, once.

Yes, but not every Kconfig file in the tree gets parsed every time. 
For example, if arch/arm/Kconfig uses "option exec=some-slow-thing",
x86 users won't execute some-slow-thing.  This is good.

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 1:22 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> It is worth noting that one of the things that keep getting brought up
> as something that need this is gcc/binutils support for a specific feature.

Yeah, then those tests would be run only once, at kconfig time. I guess we
still run some of them many times, due to the use of recursive make?

Note that these will end up in your .config --- which is not necesarily a bad
thing --- but it does mean that if e.g. you sent me your .config,
several options
may change when I try to use it.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 1:22 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> It is worth noting that one of the things that keep getting brought up
> as something that need this is gcc/binutils support for a specific feature.

Yeah, then those tests would be run only once, at kconfig time. I guess we
still run some of them many times, due to the use of recursive make?

Note that these will end up in your .config --- which is not necesarily a bad
thing --- but it does mean that if e.g. you sent me your .config,
several options
may change when I try to use it.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Geert Uytterhoeven]

On Thu, Jul 18, 2013 at 1:22 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> It is worth noting that one of the things that keep getting brought up
> as something that need this is gcc/binutils support for a specific feature.

Yeah, then those tests would be run only once, at kconfig time. I guess we
still run some of them many times, due to the use of recursive make?

Note that these will end up in your .config --- which is not necesarily a bad
thing --- but it does mean that if e.g. you sent me your .config,
several options
may change when I try to use it.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

On Thu, Jul 18, 2013 at 12:30:33AM +0200, Yann E. MORIN wrote:
> Sam, All,
> 
> Well, while I was fighting on this on my side... ;-)
> 
> On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > > We could extend the symbol option part to retreive values from a binary.
> > > Something like this:
> > > 
> > > config FOOBAR
> > >         bool
> > >         option exec="true"
> > > 
> > > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > > And similar conversions for other types.
> > > 
> > > This only extendt Kconfig slightly - using an already present method to import
> > > external values.
> > > 
> > > The drawback I see with this approach is that we may execute a lot of small programs
> > > where the value is never used.
> > 
> > Following is a quick patch implmenting this idea.
> > You need to run gperf manually to enable this.
> > 
> > "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> > 
> > I did not figure out how to use the built-in rules to generate this file :-(
> 
> make REGENERATE_PARSERS=y menuconfig
Thanks!

> 
> > I have tested this lightly - as we should discuss if this is a viable way forward.
> 
> Instead of extending the Kconfig language, I was thinking (as initially
> suggested by Andrew) of generating a Kconfig file before all config
> targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> 
> I'm not a fan od extending the Kconfig language in this way. It feels
> weird to me. Especially when we have a way to solve this specific issue
> with a script that generates the needed Kconfig symbols before any
> config target is called.

The problem we want to solve is that we want some external state to influence
the Kconfig symbols.
In this case if we support lz4.
In other cases if we support a specific gcc feature or whatever.

The relevant external state may not be relevant in all cases.
We may have something that is arm specific - and we do not want to waste
time on this when we are building for sparc.

We can consider two general cases.
- Where the external state is static and independent on any other config symbols.
  lz4 is such a case.
- Where the external state depends on a config symbol.
  As we specify CROSS_COMPILE as a config symbol this may influence if gcc
  support a specific feature or not.

Both approaches suggested (generate Kconfig and exec option) supports that
we only execute relevant stuff. When we generate Kconfig we can call arch specific
scripts and for exec option we include the arch specific stuff in the arch specific
Kconfig files.

None of the approaches presented so far can handle the fact that a config symbol
may influence the external state.
But I think the exec option can be updated to support this which is why
that is my preference.

This extends the Kconfig language - but at least it is a very minor extension.

	Sam

[PATCH -next 2/2] kbuild: fix for updated LZ4 tool with the new streaming format

[Sam Ravnborg]

On Thu, Jul 18, 2013 at 12:30:33AM +0200, Yann E. MORIN wrote:
> Sam, All,
> 
> Well, while I was fighting on this on my side... ;-)
> 
> On 2013-07-17 23:16 +0200, Sam Ravnborg spake thusly:
> > > We could extend the symbol option part to retreive values from a binary.
> > > Something like this:
> > > 
> > > config FOOBAR
> > >         bool
> > >         option exec="true"
> > > 
> > > FOOBAR would assume the value "y" if the command true has exit code == 0, otherwise "n".
> > > And similar conversions for other types.
> > > 
> > > This only extendt Kconfig slightly - using an already present method to import
> > > external values.
> > > 
> > > The drawback I see with this approach is that we may execute a lot of small programs
> > > where the value is never used.
> > 
> > Following is a quick patch implmenting this idea.
> > You need to run gperf manually to enable this.
> > 
> > "gperf -C scripts/kconfig/zconf.gperf > scripts/kconfig/zconf.hash.c"
> > 
> > I did not figure out how to use the built-in rules to generate this file :-(
> 
> make REGENERATE_PARSERS=y menuconfig
Thanks!

> 
> > I have tested this lightly - as we should discuss if this is a viable way forward.
> 
> Instead of extending the Kconfig language, I was thinking (as initially
> suggested by Andrew) of generating a Kconfig file before all config
> targets, and source that Kconfig file from $(TOPDIR)/Kconfig.
> 
> I'm not a fan od extending the Kconfig language in this way. It feels
> weird to me. Especially when we have a way to solve this specific issue
> with a script that generates the needed Kconfig symbols before any
> config target is called.

The problem we want to solve is that we want some external state to influence
the Kconfig symbols.
In this case if we support lz4.
In other cases if we support a specific gcc feature or whatever.

The relevant external state may not be relevant in all cases.
We may have something that is arm specific - and we do not want to waste
time on this when we are building for sparc.

We can consider two general cases.
- Where the external state is static and independent on any other config symbols.
  lz4 is such a case.
- Where the external state depends on a config symbol.
  As we specify CROSS_COMPILE as a config symbol this may influence if gcc
  support a specific feature or not.

Both approaches suggested (generate Kconfig and exec option) supports that
we only execute relevant stuff. When we generate Kconfig we can call arch specific
scripts and for exec option we include the arch specific stuff in the arch specific
Kconfig files.

None of the approaches presented so far can handle the fact that a config symbol
may influence the external state.
But I think the exec option can be updated to support this which is why
that is my preference.

This extends the Kconfig language - but at least it is a very minor extension.

	Sam