[Patch] Read CONFIG_RD_ variables for initramfs compression

[Patch] Read CONFIG_RD_ variables for initramfs compression

[P J P]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1068 bytes --]

    Hello Andrew,

When expert configuration option(CONFIG_EXPERT) is enabled, menuconfig offers 
choice of compression algorithm, to compress initial ramfs image. This choice 
is stored into CONFIG_RD_<GZIP/BZIP2/...> variables.

But, usr/Makefile reads from earlier INITRAMFS_COMPRESSION_* variables to set 
the ramfs file extension(.gzip/.bz2/...), which is later(in 
scripts/gen_initramfs_list.sh) used to apply actual compression.

Since none of the INITRAMFS_COMPRESSION_* variables are defined, the resulting 
'usr/initramfs_data.cpio' file remains un-compressed.

The patch attached herein, updates the usr/Makefile to read CONFIG_RD_* 
variables, and also adds support for the Lz4 compression algorithm.

Could you please review it?
==
PS:
IIUC, I think INITRAMFS_COMPRESSION_* variables are not used anywhere at all. 
Though I'm not sure about the EMBEDDED configurations. But if they are not 
used, it could be better to remove their description/references from 
usr/Kconfig and other places.


Thank you.
--
Prasad J Pandit / Red Hat Security Response Team

[-- Attachment #2: Type: TEXT/PLAIN, Size: 2876 bytes --]

From 0304fb400c27227106c1f9a57fe3de13ca03cca2 Mon Sep 17 00:00:00 2001
From: P J P <prasad@redhat.com>
Date: Tue, 15 Oct 2013 19:28:40 +0530
Subject: Read CONFIG_RD_ variables for initramfs compression

When expert configuration option(CONFIG_EXPERT) is enabled,
menuconfig offers a choice of compression algorithm to compress
initial ramfs image; This choice is stored into CONFIG_RD_*
variables. But usr/Makefile uses earlier INITRAMFS_COMPRESSION_*
macros to build initial ramfs file. Since none of them is defined,
resulting 'initramfs_data.cpio' file remains un-compressed.

This patch updates the Makefile to use CONFIG_RD_* variables and
adds support for LZ4 compression algorithm.

Signed-off-by: P J P <prasad@redhat.com>

diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index b482f16..5761973 100644
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -246,6 +246,7 @@ case "$arg" in
 		echo "$output_file" | grep -q "\.xz$" && \
 				compr="xz --check=crc32 --lzma2=dict=1MiB"
 		echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.lz4$" && compr="lz4 -9 -f"
 		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
 		shift
 		;;
diff --git a/usr/Makefile b/usr/Makefile
index 029ffe6..06ad948 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -7,19 +7,22 @@ PHONY += klibcdirs
 
 
 # Gzip
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_GZIP)   = .gz
+suffix_$(CONFIG_RD_GZIP)   = .gz
 
 # Bzip2
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)  = .bz2
+suffix_$(CONFIG_RD_BZIP2)  = .bz2
 
 # Lzma
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZMA)   = .lzma
+suffix_$(CONFIG_RD_LZMA)   = .lzma
 
 # XZ
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_XZ)     = .xz
+suffix_$(CONFIG_RD_XZ)     = .xz
 
 # Lzo
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZO)   = .lzo
+suffix_$(CONFIG_RD_LZO)    = .lzo
+
+# Lz4
+suffix_$(CONFIG_RD_LZ4)    = .lz4
 
 AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/initramfs_data.cpio$(suffix_y)"
 
@@ -53,7 +56,10 @@ endif
 quiet_cmd_initfs = GEN     $@
       cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
 
-targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.xz initramfs_data.cpio.lzo initramfs_data.cpio
+targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 \
+	initramfs_data.cpio.lzma initramfs_data.cpio.xz \
+	initramfs_data.cpio.lzo initramfs_data.cpio.lz4 \
+	initramfs_data.cpio
 # do not try to update files included in initramfs
 $(deps_initramfs): ;
 
@@ -66,4 +72,3 @@ $(deps_initramfs): klibcdirs
 $(obj)/initramfs_data.cpio$(suffix_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
 	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/.initramfs_data.cpio.d
 	$(call if_changed,initfs)
-
-- 
1.8.3.1

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[Andrew Morton]

On Tue, 15 Oct 2013 20:25:57 +0530 (IST) P J P <ppandit@redhat.com> wrote:

> When expert configuration option(CONFIG_EXPERT) is enabled,
> menuconfig offers a choice of compression algorithm to compress
> initial ramfs image; This choice is stored into CONFIG_RD_*
> variables. But usr/Makefile uses earlier INITRAMFS_COMPRESSION_*
> macros to build initial ramfs file. Since none of them is defined,
> resulting 'initramfs_data.cpio' file remains un-compressed.
> 
> This patch updates the Makefile to use CONFIG_RD_* variables and
> adds support for LZ4 compression algorithm.

This patch breaks my x86_64 allmodconfig build, because I don't have
the lz4 executable installed:

/usr/src/25/scripts/gen_initramfs_list.sh: line 307: lz4: command not found
make[1]: *** [usr/initramfs_data.cpio.lz4] Error 1

This obviously isn't acceptable!

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[P J P]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1065 bytes --]

  Hello Andrew,

+-- On Tue, 29 Oct 2013, Andrew Morton wrote --+
| On Tue, 15 Oct 2013 20:25:57 +0530 (IST) P J P <ppandit@redhat.com> wrote:
| This patch breaks my x86_64 allmodconfig build, because I don't have
| the lz4 executable installed:
| 
| /usr/src/25/scripts/gen_initramfs_list.sh: line 307: lz4: command not found
| make[1]: *** [usr/initramfs_data.cpio.lz4] Error 1
| 
| This obviously isn't acceptable!

  Oops! '$ make allmodconfig' seems to enables all compression algorithms; So 
the last one overrides the previous choices in usr/Makefile.

===
...
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
===

Please see an updated patch herein. I've patched 'gen_initramfs_list.sh' 
script to check if a selected compression command is accessible or not; And 
fall-back to the default gzip(1) format when it is not. usr/Makefile also 
defaults to '.gz' format when all are enabled.

I hope this works well; Sorry about the trouble though.

Thank you!
--
Prasad J Pandit / Red Hat Security Response Team

[-- Attachment #2: Type: TEXT/PLAIN, Size: 3990 bytes --]

From c903542c15ea30785d209ad7dd8fa0d461e084cb Mon Sep 17 00:00:00 2001
From: P J P <prasad@redhat.com>
Date: Wed, 30 Oct 2013 15:32:16 +0530
Subject: Read CONFIG_RD_ variables for initramfs compression

When expert configuration option(CONFIG_EXPERT) is enabled,
menuconfig offers a choice of compression algorithm to compress
initial ramfs image; This choice is stored into CONFIG_RD_*
variables. But usr/Makefile uses earlier INITRAMFS_COMPRESSION_*
macros to build initial ramfs file. Since none of them is defined,
resulting 'initramfs_data.cpio' file remains un-compressed.

This patch updates the Makefile to use CONFIG_RD_* variables and
adds support for LZ4 compression algorithm. Also updates the
'gen_initramfs_list.sh' script to check whether a selected
compression command is accessible or not. And fall-back to default
gzip(1) compression when it is not.

Signed-off-by: P J P <prasad@redhat.com>

diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index b482f16..2889a83 100644
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -240,12 +240,18 @@ case "$arg" in
 		output_file="$1"
 		cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
 		output=${cpio_list}
-		echo "$output_file" | grep -q "\.gz$" && compr="gzip -n -9 -f"
-		echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f"
-		echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f"
-		echo "$output_file" | grep -q "\.xz$" && \
-				compr="xz --check=crc32 --lzma2=dict=1MiB"
-		echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.gz$" && [ -x "/bin/gzip" ] \
+                && compr="gzip -n -9 -f"
+		echo "$output_file" | grep -q "\.bz2$" && [ -x "/bin/bzip2" ] \
+                && compr="bzip2 -9 -f"
+		echo "$output_file" | grep -q "\.lzma$" && [ -x "/bin/lzma" ] \
+                && compr="lzma -9 -f"
+		echo "$output_file" | grep -q "\.xz$" && [ -x "/bin/xz" ] \
+                && compr="xz --check=crc32 --lzma2=dict=1MiB"
+		echo "$output_file" | grep -q "\.lzo$" && [ -x "/bin/lzop" ] \
+                && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.lz4$" && [ -x "/bin/lz4" ] \
+                && compr="lz4 -9 -f"
 		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
 		shift
 		;;
diff --git a/usr/Makefile b/usr/Makefile
index 029ffe6..e767f01 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -6,20 +6,23 @@ klibcdirs:;
 PHONY += klibcdirs
 
 
-# Gzip
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_GZIP)   = .gz
-
 # Bzip2
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)  = .bz2
+suffix_$(CONFIG_RD_BZIP2)  = .bz2
 
 # Lzma
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZMA)   = .lzma
+suffix_$(CONFIG_RD_LZMA)   = .lzma
 
 # XZ
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_XZ)     = .xz
+suffix_$(CONFIG_RD_XZ)     = .xz
 
 # Lzo
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZO)   = .lzo
+suffix_$(CONFIG_RD_LZO)    = .lzo
+
+# Lz4
+suffix_$(CONFIG_RD_LZ4)    = .lz4
+
+# Gzip
+suffix_$(CONFIG_RD_GZIP)   = .gz
 
 AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/initramfs_data.cpio$(suffix_y)"
 
@@ -53,7 +56,10 @@ endif
 quiet_cmd_initfs = GEN     $@
       cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
 
-targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.xz initramfs_data.cpio.lzo initramfs_data.cpio
+targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 \
+	initramfs_data.cpio.lzma initramfs_data.cpio.xz \
+	initramfs_data.cpio.lzo initramfs_data.cpio.lz4 \
+	initramfs_data.cpio
 # do not try to update files included in initramfs
 $(deps_initramfs): ;
 
@@ -66,4 +72,3 @@ $(deps_initramfs): klibcdirs
 $(obj)/initramfs_data.cpio$(suffix_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
 	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/.initramfs_data.cpio.d
 	$(call if_changed,initfs)
-
-- 
1.8.3.1

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[Andrew Morton]

On Wed, 30 Oct 2013 15:57:20 +0530 (IST) P J P <ppandit@redhat.com> wrote:

> +-- On Tue, 29 Oct 2013, Andrew Morton wrote --+
> | On Tue, 15 Oct 2013 20:25:57 +0530 (IST) P J P <ppandit@redhat.com> wrote:
> | This patch breaks my x86_64 allmodconfig build, because I don't have
> | the lz4 executable installed:
> | 
> | /usr/src/25/scripts/gen_initramfs_list.sh: line 307: lz4: command not found
> | make[1]: *** [usr/initramfs_data.cpio.lz4] Error 1
> | 
> | This obviously isn't acceptable!
> 
>   Oops! '$ make allmodconfig' seems to enables all compression algorithms; So 
> the last one overrides the previous choices in usr/Makefile.
> 
> ===
> ...
> CONFIG_RD_GZIP=y
> CONFIG_RD_BZIP2=y
> CONFIG_RD_LZMA=y
> CONFIG_RD_XZ=y
> CONFIG_RD_LZO=y
> CONFIG_RD_LZ4=y
> ===
> 
> Please see an updated patch herein. I've patched 'gen_initramfs_list.sh' 
> script to check if a selected compression command is accessible or not; And 
> fall-back to the default gzip(1) format when it is not. usr/Makefile also 
> defaults to '.gz' format when all are enabled.

Below is the delta.

Requiring that the executables exist in /bin is unpleasant.  Isn't
there some convenient way of testing for the presence of an executable
in $PATH?

The shell-builtin `which' seems to dtrt:

akpm3:/usr/src/25> if which gzip > /dev/null ; then echo foo ; fi 
foo
akpm3:/usr/src/25> if which lz4 > /dev/null ; then echo foo ; fi    
akpm3:/usr/src/25> 

/bin/which should do the same thing.

There's probably some kbuild-approved way of doing this, but I
don't know what it is off-hand.



From: P J P <ppandit@redhat.com>
Subject: initramfs-read-config_rd_-variables-for-initramfs-compression-fix

Oops!  '$ make allmodconfig' seems to enables all compression algorithms;
So the last one overrides the previous choices in usr/Makefile.

Please see an updated patch herein.  I've patched 'gen_initramfs_list.sh'
script to check if a selected compression command is accessible or not;
And fall-back to the default gzip(1) format when it is not.  usr/Makefile
also defaults to '.gz' format when all are enabled.

Signed-off-by: P J P <prasad@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 scripts/gen_initramfs_list.sh |   19 ++++++++++++-------
 usr/Makefile                  |    6 +++---
 2 files changed, 15 insertions(+), 10 deletions(-)

diff -puN scripts/gen_initramfs_list.sh~initramfs-read-config_rd_-variables-for-initramfs-compression-fix scripts/gen_initramfs_list.sh
--- a/scripts/gen_initramfs_list.sh~initramfs-read-config_rd_-variables-for-initramfs-compression-fix
+++ a/scripts/gen_initramfs_list.sh
@@ -240,13 +240,18 @@ case "$arg" in
 		output_file="$1"
 		cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
 		output=${cpio_list}
-		echo "$output_file" | grep -q "\.gz$" && compr="gzip -n -9 -f"
-		echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f"
-		echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f"
-		echo "$output_file" | grep -q "\.xz$" && \
-				compr="xz --check=crc32 --lzma2=dict=1MiB"
-		echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
-		echo "$output_file" | grep -q "\.lz4$" && compr="lz4 -9 -f"
+		echo "$output_file" | grep -q "\.gz$" && [ -x "/bin/gzip" ] \
+                && compr="gzip -n -9 -f"
+		echo "$output_file" | grep -q "\.bz2$" && [ -x "/bin/bzip2" ] \
+                && compr="bzip2 -9 -f"
+		echo "$output_file" | grep -q "\.lzma$" && [ -x "/bin/lzma" ] \
+                && compr="lzma -9 -f"
+		echo "$output_file" | grep -q "\.xz$" && [ -x "/bin/xz" ] \
+                && compr="xz --check=crc32 --lzma2=dict=1MiB"
+		echo "$output_file" | grep -q "\.lzo$" && [ -x "/bin/lzop" ] \
+                && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.lz4$" && [ -x "/bin/lz4" ] \
+                && compr="lz4 -9 -f"
 		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
 		shift
 		;;
diff -puN usr/Makefile~initramfs-read-config_rd_-variables-for-initramfs-compression-fix usr/Makefile
--- a/usr/Makefile~initramfs-read-config_rd_-variables-for-initramfs-compression-fix
+++ a/usr/Makefile
@@ -6,9 +6,6 @@ klibcdirs:;
 PHONY += klibcdirs
 
 
-# Gzip
-suffix_$(CONFIG_RD_GZIP)   = .gz
-
 # Bzip2
 suffix_$(CONFIG_RD_BZIP2)  = .bz2
 
@@ -24,6 +21,9 @@ suffix_$(CONFIG_RD_LZO)    = .lzo
 # Lz4
 suffix_$(CONFIG_RD_LZ4)    = .lz4
 
+# Gzip
+suffix_$(CONFIG_RD_GZIP)   = .gz
+
 AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/initramfs_data.cpio$(suffix_y)"
 
 # Generate builtin.o based on initramfs_data.o
_

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[P J P]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 321 bytes --]

  Hello Andrew,

+-- On Wed, 30 Oct 2013, Andrew Morton wrote --+
| Isn't there some convenient way of testing for the presence of an 
| executable in $PATH?
| 
| The shell-builtin `which' seems to dtrt:

  True. Please see an updated patch attached herein.

Thank you.
--
Prasad J Pandit / Red Hat Security Response Team

[-- Attachment #2: Type: TEXT/PLAIN, Size: 4206 bytes --]

From 352781fd2846c54e92ae0c37dd972dc5fcdfb695 Mon Sep 17 00:00:00 2001
From: P J P <prasad@redhat.com>
Date: Thu, 31 Oct 2013 12:03:00 +0530
Subject: Read CONFIG_RD_ variables for initramfs compression

When expert configuration option(CONFIG_EXPERT) is enabled,
menuconfig offers a choice of compression algorithm to compress
initial ramfs image; This choice is stored into CONFIG_RD_*
variables. But usr/Makefile uses earlier INITRAMFS_COMPRESSION_*
macros to build initial ramfs file. Since none of them is defined,
resulting 'initramfs_data.cpio' file remains un-compressed.

This patch updates the Makefile to use CONFIG_RD_* variables and
adds support for LZ4 compression algorithm. Also updates the
'gen_initramfs_list.sh' script to check whether a selected
compression command is accessible or not. And fall-back to default
gzip(1) compression when it is not.

Signed-off-by: P J P <prasad@redhat.com>

diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index b482f16..ef47409 100644
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -240,12 +240,24 @@ case "$arg" in
 		output_file="$1"
 		cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
 		output=${cpio_list}
-		echo "$output_file" | grep -q "\.gz$" && compr="gzip -n -9 -f"
-		echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f"
-		echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f"
-		echo "$output_file" | grep -q "\.xz$" && \
-				compr="xz --check=crc32 --lzma2=dict=1MiB"
-		echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.gz$" \
+                && [ -x "`which gzip 2> /dev/null`" ] \
+                && compr="gzip -n -9 -f"
+		echo "$output_file" | grep -q "\.bz2$" \
+                && [ -x "`which bzip2 2> /dev/null`" ] \
+                && compr="bzip2 -9 -f"
+		echo "$output_file" | grep -q "\.lzma$" \
+                && [ -x "`which lzma 2> /dev/null`" ] \
+                && compr="lzma -9 -f"
+		echo "$output_file" | grep -q "\.xz$" \
+                && [ -x "`which xz 2> /dev/null`" ] \
+                && compr="xz --check=crc32 --lzma2=dict=1MiB"
+		echo "$output_file" | grep -q "\.lzo$" \
+                && [ -x "`which lzop 2> /dev/null`" ] \
+                && compr="lzop -9 -f"
+		echo "$output_file" | grep -q "\.lz4$" \
+                && [ -x "`which lz4 2> /dev/null`" ] \
+                && compr="lz4 -9 -f"
 		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
 		shift
 		;;
diff --git a/usr/Makefile b/usr/Makefile
index 029ffe6..e767f01 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -6,20 +6,23 @@ klibcdirs:;
 PHONY += klibcdirs
 
 
-# Gzip
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_GZIP)   = .gz
-
 # Bzip2
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)  = .bz2
+suffix_$(CONFIG_RD_BZIP2)  = .bz2
 
 # Lzma
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZMA)   = .lzma
+suffix_$(CONFIG_RD_LZMA)   = .lzma
 
 # XZ
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_XZ)     = .xz
+suffix_$(CONFIG_RD_XZ)     = .xz
 
 # Lzo
-suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZO)   = .lzo
+suffix_$(CONFIG_RD_LZO)    = .lzo
+
+# Lz4
+suffix_$(CONFIG_RD_LZ4)    = .lz4
+
+# Gzip
+suffix_$(CONFIG_RD_GZIP)   = .gz
 
 AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/initramfs_data.cpio$(suffix_y)"
 
@@ -53,7 +56,10 @@ endif
 quiet_cmd_initfs = GEN     $@
       cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
 
-targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.xz initramfs_data.cpio.lzo initramfs_data.cpio
+targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 \
+	initramfs_data.cpio.lzma initramfs_data.cpio.xz \
+	initramfs_data.cpio.lzo initramfs_data.cpio.lz4 \
+	initramfs_data.cpio
 # do not try to update files included in initramfs
 $(deps_initramfs): ;
 
@@ -66,4 +72,3 @@ $(deps_initramfs): klibcdirs
 $(obj)/initramfs_data.cpio$(suffix_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
 	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/.initramfs_data.cpio.d
 	$(call if_changed,initfs)
-
-- 
1.8.3.1

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[Simon Guinot]

[-- Attachment #1: Type: text/plain, Size: 5348 bytes --]

On Thu, Oct 31, 2013 at 12:35:05PM +0530, P J P wrote:
>   Hello Andrew,
> 
> +-- On Wed, 30 Oct 2013, Andrew Morton wrote --+
> | Isn't there some convenient way of testing for the presence of an 
> | executable in $PATH?
> | 
> | The shell-builtin `which' seems to dtrt:
> 
>   True. Please see an updated patch attached herein.
> 
> Thank you.
> --
> Prasad J Pandit / Red Hat Security Response Team

> From 352781fd2846c54e92ae0c37dd972dc5fcdfb695 Mon Sep 17 00:00:00 2001
> From: P J P <prasad@redhat.com>
> Date: Thu, 31 Oct 2013 12:03:00 +0530
> Subject: Read CONFIG_RD_ variables for initramfs compression
> 
> When expert configuration option(CONFIG_EXPERT) is enabled,
> menuconfig offers a choice of compression algorithm to compress
> initial ramfs image; This choice is stored into CONFIG_RD_*
> variables. But usr/Makefile uses earlier INITRAMFS_COMPRESSION_*
> macros to build initial ramfs file. Since none of them is defined,
> resulting 'initramfs_data.cpio' file remains un-compressed.
> 
> This patch updates the Makefile to use CONFIG_RD_* variables and
> adds support for LZ4 compression algorithm. Also updates the
> 'gen_initramfs_list.sh' script to check whether a selected
> compression command is accessible or not. And fall-back to default
> gzip(1) compression when it is not.
> 
> Signed-off-by: P J P <prasad@redhat.com>

Hi P J P,

IIUC this patch, the INITRAMFS_COMPRESSION_* options are now
ignored/useless. Don't you think we should remove them from the
usr/Kconfig file ?

Actually, I think this patch makes the initramfs compression
configuration quite confusing. Consider the following configuration
for a 3.13-rc3 kernel:

CONFIG_RD_GZIP=y
CONFIG_RD_LZMA=y
CONFIG_INITRAMFS_COMPRESSION_LZMA=y

This now produces a gzipped initramfs_data.cpio against a lzma one
previously. 

Regards,

Simon

> 
> diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
> index b482f16..ef47409 100644
> --- a/scripts/gen_initramfs_list.sh
> +++ b/scripts/gen_initramfs_list.sh
> @@ -240,12 +240,24 @@ case "$arg" in
>  		output_file="$1"
>  		cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
>  		output=${cpio_list}
> -		echo "$output_file" | grep -q "\.gz$" && compr="gzip -n -9 -f"
> -		echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f"
> -		echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f"
> -		echo "$output_file" | grep -q "\.xz$" && \
> -				compr="xz --check=crc32 --lzma2=dict=1MiB"
> -		echo "$output_file" | grep -q "\.lzo$" && compr="lzop -9 -f"
> +		echo "$output_file" | grep -q "\.gz$" \
> +                && [ -x "`which gzip 2> /dev/null`" ] \
> +                && compr="gzip -n -9 -f"
> +		echo "$output_file" | grep -q "\.bz2$" \
> +                && [ -x "`which bzip2 2> /dev/null`" ] \
> +                && compr="bzip2 -9 -f"
> +		echo "$output_file" | grep -q "\.lzma$" \
> +                && [ -x "`which lzma 2> /dev/null`" ] \
> +                && compr="lzma -9 -f"
> +		echo "$output_file" | grep -q "\.xz$" \
> +                && [ -x "`which xz 2> /dev/null`" ] \
> +                && compr="xz --check=crc32 --lzma2=dict=1MiB"
> +		echo "$output_file" | grep -q "\.lzo$" \
> +                && [ -x "`which lzop 2> /dev/null`" ] \
> +                && compr="lzop -9 -f"
> +		echo "$output_file" | grep -q "\.lz4$" \
> +                && [ -x "`which lz4 2> /dev/null`" ] \
> +                && compr="lz4 -9 -f"
>  		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
>  		shift
>  		;;
> diff --git a/usr/Makefile b/usr/Makefile
> index 029ffe6..e767f01 100644
> --- a/usr/Makefile
> +++ b/usr/Makefile
> @@ -6,20 +6,23 @@ klibcdirs:;
>  PHONY += klibcdirs
>  
>  
> -# Gzip
> -suffix_$(CONFIG_INITRAMFS_COMPRESSION_GZIP)   = .gz
> -
>  # Bzip2
> -suffix_$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)  = .bz2
> +suffix_$(CONFIG_RD_BZIP2)  = .bz2
>  
>  # Lzma
> -suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZMA)   = .lzma
> +suffix_$(CONFIG_RD_LZMA)   = .lzma
>  
>  # XZ
> -suffix_$(CONFIG_INITRAMFS_COMPRESSION_XZ)     = .xz
> +suffix_$(CONFIG_RD_XZ)     = .xz
>  
>  # Lzo
> -suffix_$(CONFIG_INITRAMFS_COMPRESSION_LZO)   = .lzo
> +suffix_$(CONFIG_RD_LZO)    = .lzo
> +
> +# Lz4
> +suffix_$(CONFIG_RD_LZ4)    = .lz4
> +
> +# Gzip
> +suffix_$(CONFIG_RD_GZIP)   = .gz
>  
>  AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/initramfs_data.cpio$(suffix_y)"
>  
> @@ -53,7 +56,10 @@ endif
>  quiet_cmd_initfs = GEN     $@
>        cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
>  
> -targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 initramfs_data.cpio.lzma initramfs_data.cpio.xz initramfs_data.cpio.lzo initramfs_data.cpio
> +targets := initramfs_data.cpio.gz initramfs_data.cpio.bz2 \
> +	initramfs_data.cpio.lzma initramfs_data.cpio.xz \
> +	initramfs_data.cpio.lzo initramfs_data.cpio.lz4 \
> +	initramfs_data.cpio
>  # do not try to update files included in initramfs
>  $(deps_initramfs): ;
>  
> @@ -66,4 +72,3 @@ $(deps_initramfs): klibcdirs
>  $(obj)/initramfs_data.cpio$(suffix_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
>  	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/.initramfs_data.cpio.d
>  	$(call if_changed,initfs)
> -
> -- 
> 1.8.3.1
> 


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [Patch] Read CONFIG_RD_ variables for initramfs compression

[P J P]

  Hello Simon, Andrew

+-- On Wed, 11 Dec 2013, Simon Guinot wrote --+
| IIUC this patch, the INITRAMFS_COMPRESSION_* options are now
| ignored/useless. Don't you think we should remove them from the
| usr/Kconfig file ?

  -> https://lkml.org/lkml/2013/11/25/21

I'v pushed a patch from Mr Hristo to the same effect. I guess it's still in 
the queue. I haven't received any review for it yet. (...Andrew?)

| Actually, I think this patch makes the initramfs compression
| configuration quite confusing. Consider the following configuration
| for a 3.13-rc3 kernel:
| 
| CONFIG_RD_GZIP=y
| CONFIG_RD_LZMA=y
| CONFIG_INITRAMFS_COMPRESSION_LZMA=y
| 
| This now produces a gzipped initramfs_data.cpio against a lzma one
| previously. 

  That is because, when multiple options are set, CONFIG_RD_GZIP is checked 
last in the usr/Makefile.
  ...
  # Gzip
  suffix_$(CONFIG_RD_GZIP)   = .gz

Hope it helps.
--
Prasad J Pandit / Red Hat Security Response Team