[PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[Masahiro Yamada]

Since commit ddd09bcc899f ("initramfs: make compression options not
depend on INITRAMFS_SOURCE"), Kconfig asks the compression mode for
the built-in initramfs regardless of INITRAMFS_SOURCE.

It is technically simpler, but pointless from a UI perspective,
Linus says [1].

When INITRAMFS_SOURCE is empty, usr/Makefile creates a tiny default
cpio, which is so small that nobody cares about the compression.

This commit hides the Kconfig choice in that case. The default cpio
is embedded without compression, which was the original behavior.

[1]: https://lkml.org/lkml/2020/2/1/160

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Linus,

Will you directly pick up this if you are OK with it?



 usr/Kconfig  | 1 +
 usr/Makefile | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/usr/Kconfig b/usr/Kconfig
index 529caab1a328..bdf5bbd40727 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -102,6 +102,7 @@ config RD_LZ4
 
 choice
 	prompt "Built-in initramfs compression mode"
+	depends on INITRAMFS_SOURCE != ""
 	help
 	  This option allows you to decide by which algorithm the builtin
 	  initramfs will be compressed.  Several compression algorithms are
diff --git a/usr/Makefile b/usr/Makefile
index 18aed2ab98da..c12e6b15ce72 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -8,7 +8,7 @@
 # with unpack_to_rootfs(). Make size_append no-op.
 override size_append := :
 
-compress-$(CONFIG_INITRAMFS_COMPRESSION_NONE)	:= shipped
+compress-y					:= shipped
 compress-$(CONFIG_INITRAMFS_COMPRESSION_GZIP)	:= gzip
 compress-$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)	:= bzip2
 compress-$(CONFIG_INITRAMFS_COMPRESSION_LZMA)	:= lzma
-- 
2.17.1

Re: [PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[Linus Torvalds]

On Mon, Feb 3, 2020 at 4:47 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Will you directly pick up this if you are OK with it?

Done. Thanks,

              Linus

Re: [PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[Linus Torvalds]

On Mon, Feb 3, 2020 at 4:47 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> This commit hides the Kconfig choice in that case. The default cpio
> is embedded without compression, which was the original behavior.

Btw, is there any way to figure out automatically what the initrd
compression is (for the external case)?

Because I think it would be lovely to be able to have sane defaults
for the CONFIG_RD_xyz things.

I'm assuming the answer is "no", simply because it comes from distro
installs, but I thought I'd check.

                 Linus

Re: [PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[Masahiro Yamada]

Hi Linus,

On Tue, Feb 4, 2020 at 2:35 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mon, Feb 3, 2020 at 4:47 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > This commit hides the Kconfig choice in that case. The default cpio
> > is embedded without compression, which was the original behavior.
>
> Btw, is there any way to figure out automatically what the initrd
> compression is (for the external case)?
>
> Because I think it would be lovely to be able to have sane defaults
> for the CONFIG_RD_xyz things.
>
> I'm assuming the answer is "no", simply because it comes from distro
> installs, but I thought I'd check.

I think the answer is no.

For external initrd, it is up to distros,
and we have no way to know the compression type beforehand.

We know it after booting.

init/initramfs.c has the following line:
pr_debug("Detected %s compressed data\n", compress_name);



-- 
Best Regards
Masahiro Yamada

Re: [PATCH] initramfs: do not show compression mode choice if INITRAMFS_SOURCE is empty

[Arvind Sankar]

On Tue, Feb 04, 2020 at 01:47:08AM +0900, Masahiro Yamada wrote:
> Since commit ddd09bcc899f ("initramfs: make compression options not
> depend on INITRAMFS_SOURCE"), Kconfig asks the compression mode for
> the built-in initramfs regardless of INITRAMFS_SOURCE.
> 
> It is technically simpler, but pointless from a UI perspective,
> Linus says [1].
> 
> When INITRAMFS_SOURCE is empty, usr/Makefile creates a tiny default
> cpio, which is so small that nobody cares about the compression.
> 
> This commit hides the Kconfig choice in that case. The default cpio
> is embedded without compression, which was the original behavior.
> 
> [1]: https://lkml.org/lkml/2020/2/1/160
> 
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---

Would it be feasible to check if the kernel is going to be compressed,
i.e. one of the KERNEL_{GZIP,BZIP2,...} options other than
KERNEL_UNCOMPRESSED is set, and default built-in initramfs to
uncompressed in that case as well?

Thanks.

[PATCH] initramfs: don't double-compress built-in initramfs if the kernel is compressed

[Arvind Sankar]

If the kernel is going to be compressed anyway, there is no point in
double-compressing the built-in initramfs. Hide the built-in initramfs
compression choice in this case.

Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>
---
 init/Kconfig | 9 +++++++++
 usr/Kconfig  | 2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/init/Kconfig b/init/Kconfig
index 24b23d843df1..88da0976bbfa 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -176,6 +176,9 @@ config HAVE_KERNEL_LZ4
 config HAVE_KERNEL_UNCOMPRESSED
 	bool
 
+config KERNEL_COMPRESSED
+	bool
+
 choice
 	prompt "Kernel compression mode"
 	default KERNEL_GZIP
@@ -201,6 +204,7 @@ choice
 config KERNEL_GZIP
 	bool "Gzip"
 	depends on HAVE_KERNEL_GZIP
+	select KERNEL_COMPRESSED
 	help
 	  The old and tried gzip compression. It provides a good balance
 	  between compression ratio and decompression speed.
@@ -208,6 +212,7 @@ config KERNEL_GZIP
 config KERNEL_BZIP2
 	bool "Bzip2"
 	depends on HAVE_KERNEL_BZIP2
+	select KERNEL_COMPRESSED
 	help
 	  Its compression ratio and speed is intermediate.
 	  Decompression speed is slowest among the choices.  The kernel
@@ -218,6 +223,7 @@ config KERNEL_BZIP2
 config KERNEL_LZMA
 	bool "LZMA"
 	depends on HAVE_KERNEL_LZMA
+	select KERNEL_COMPRESSED
 	help
 	  This compression algorithm's ratio is best.  Decompression speed
 	  is between gzip and bzip2.  Compression is slowest.
@@ -226,6 +232,7 @@ config KERNEL_LZMA
 config KERNEL_XZ
 	bool "XZ"
 	depends on HAVE_KERNEL_XZ
+	select KERNEL_COMPRESSED
 	help
 	  XZ uses the LZMA2 algorithm and instruction set specific
 	  BCJ filters which can improve compression ratio of executable
@@ -241,6 +248,7 @@ config KERNEL_XZ
 config KERNEL_LZO
 	bool "LZO"
 	depends on HAVE_KERNEL_LZO
+	select KERNEL_COMPRESSED
 	help
 	  Its compression ratio is the poorest among the choices. The kernel
 	  size is about 10% bigger than gzip; however its speed
@@ -249,6 +257,7 @@ config KERNEL_LZO
 config KERNEL_LZ4
 	bool "LZ4"
 	depends on HAVE_KERNEL_LZ4
+	select KERNEL_COMPRESSED
 	help
 	  LZ4 is an LZ77-type compressor with a fixed, byte-oriented encoding.
 	  A preliminary version of LZ4 de/compression tool is available at
diff --git a/usr/Kconfig b/usr/Kconfig
index bdf5bbd40727..f39eeb9ea2e3 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -102,7 +102,7 @@ config RD_LZ4
 
 choice
 	prompt "Built-in initramfs compression mode"
-	depends on INITRAMFS_SOURCE != ""
+	depends on INITRAMFS_SOURCE != "" && !KERNEL_COMPRESSED
 	help
 	  This option allows you to decide by which algorithm the builtin
 	  initramfs will be compressed.  Several compression algorithms are
-- 
2.24.1

Re: [PATCH] initramfs: don't double-compress built-in initramfs if the kernel is compressed

[Masahiro Yamada]

Hi.

On Tue, Feb 4, 2020 at 6:16 AM Arvind Sankar <nivedita@alum.mit.edu> wrote:
>
> If the kernel is going to be compressed anyway, there is no point in
> double-compressing the built-in initramfs. Hide the built-in initramfs
> compression choice in this case.
>
> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>


Double-compression slightly reduces the data size.
And, keeping the bare vmlinux small is beneficial, IMHO.




> ---
>  init/Kconfig | 9 +++++++++
>  usr/Kconfig  | 2 +-
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/init/Kconfig b/init/Kconfig
> index 24b23d843df1..88da0976bbfa 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -176,6 +176,9 @@ config HAVE_KERNEL_LZ4
>  config HAVE_KERNEL_UNCOMPRESSED
>         bool
>
> +config KERNEL_COMPRESSED
> +       bool
> +
>  choice
>         prompt "Kernel compression mode"
>         default KERNEL_GZIP
> @@ -201,6 +204,7 @@ choice
>  config KERNEL_GZIP
>         bool "Gzip"
>         depends on HAVE_KERNEL_GZIP
> +       select KERNEL_COMPRESSED
>         help
>           The old and tried gzip compression. It provides a good balance
>           between compression ratio and decompression speed.
> @@ -208,6 +212,7 @@ config KERNEL_GZIP
>  config KERNEL_BZIP2
>         bool "Bzip2"
>         depends on HAVE_KERNEL_BZIP2
> +       select KERNEL_COMPRESSED
>         help
>           Its compression ratio and speed is intermediate.
>           Decompression speed is slowest among the choices.  The kernel
> @@ -218,6 +223,7 @@ config KERNEL_BZIP2
>  config KERNEL_LZMA
>         bool "LZMA"
>         depends on HAVE_KERNEL_LZMA
> +       select KERNEL_COMPRESSED
>         help
>           This compression algorithm's ratio is best.  Decompression speed
>           is between gzip and bzip2.  Compression is slowest.
> @@ -226,6 +232,7 @@ config KERNEL_LZMA
>  config KERNEL_XZ
>         bool "XZ"
>         depends on HAVE_KERNEL_XZ
> +       select KERNEL_COMPRESSED
>         help
>           XZ uses the LZMA2 algorithm and instruction set specific
>           BCJ filters which can improve compression ratio of executable
> @@ -241,6 +248,7 @@ config KERNEL_XZ
>  config KERNEL_LZO
>         bool "LZO"
>         depends on HAVE_KERNEL_LZO
> +       select KERNEL_COMPRESSED
>         help
>           Its compression ratio is the poorest among the choices. The kernel
>           size is about 10% bigger than gzip; however its speed
> @@ -249,6 +257,7 @@ config KERNEL_LZO
>  config KERNEL_LZ4
>         bool "LZ4"
>         depends on HAVE_KERNEL_LZ4
> +       select KERNEL_COMPRESSED
>         help
>           LZ4 is an LZ77-type compressor with a fixed, byte-oriented encoding.
>           A preliminary version of LZ4 de/compression tool is available at
> diff --git a/usr/Kconfig b/usr/Kconfig
> index bdf5bbd40727..f39eeb9ea2e3 100644
> --- a/usr/Kconfig
> +++ b/usr/Kconfig
> @@ -102,7 +102,7 @@ config RD_LZ4
>
>  choice
>         prompt "Built-in initramfs compression mode"
> -       depends on INITRAMFS_SOURCE != ""
> +       depends on INITRAMFS_SOURCE != "" && !KERNEL_COMPRESSED
>         help
>           This option allows you to decide by which algorithm the builtin
>           initramfs will be compressed.  Several compression algorithms are
> --
> 2.24.1
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH] initramfs: don't double-compress built-in initramfs if the kernel is compressed

[Arvind Sankar]

On Tue, Feb 04, 2020 at 11:22:19AM +0900, Masahiro Yamada wrote:
> Hi.
> 
> On Tue, Feb 4, 2020 at 6:16 AM Arvind Sankar <nivedita@alum.mit.edu> wrote:
> >
> > If the kernel is going to be compressed anyway, there is no point in
> > double-compressing the built-in initramfs. Hide the built-in initramfs
> > compression choice in this case.
> >
> > Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>
> 
> 
> Double-compression slightly reduces the data size.
> And, keeping the bare vmlinux small is beneficial, IMHO.
> 

Ok. In my setup (kernel is compressed by xz), none for the builtin
initramfs is optimal by a measurable amount, but I that will depend on
config/initramfs contents/compression methods, so might well not be true
in general. Maybe add a note to the help text?

none    7030816 arch/x86/boot/bzImage
gz      7382144 arch/x86/boot/bzImage
bz2     7353696 arch/x86/boot/bzImage
lzma    7118176 arch/x86/boot/bzImage
xz      7126400 arch/x86/boot/bzImage
lzo     7379360 arch/x86/boot/bzImage
lz4     7335712 arch/x86/boot/bzImage


diff --git a/usr/Kconfig b/usr/Kconfig
index a6b68503d177..b68910b225fc 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -125,6 +125,12 @@ choice
 	  low on RAM, since it reduces the memory consumption during
 	  boot.
 
+	  Note that if the kernel itself is compressed, the builtin initramfs
+	  will be compressed twice, first by the algorithm selected here, and
+	  then by the kernel compression method chosen. This may increase the
+	  final image size, so you should test whether compressing the builtin
+	  initramfs provides any savings.
+
 	  Keep in mind that your build system needs to provide the appropriate
 	  compression tool to compress the generated initram cpio file for
 	  embedding.