All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nick Terrell <nickrterrell@gmail.com>
To: Borislav Petkov <bp@alien8.de>, Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org, Chris Mason <clm@fb.com>,
	linux-kbuild@vger.kernel.org, x86@kernel.org,
	gregkh@linuxfoundation.org, Petr Malat <oss@malat.biz>,
	Kees Cook <keescook@chromium.org>,
	Kernel Team <Kernel-team@fb.com>,
	Nick Terrell <nickrterrell@gmail.com>,
	Adam Borowski <kilobyte@angband.pl>,
	Patrick Williams <patrickw3@fb.com>,
	rmikey@fb.com, mingo@kernel.org,
	Patrick Williams <patrick@stwcx.xyz>,
	Sedat Dilek <sedat.dilek@gmail.com>,
	Norbert Lange <nolange79@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alex Xu <alex_y_xu@yahoo.ca>,
	Arvind Sankar <nivedita@alum.mit.edu>,
	Nick Terrell <terrelln@fb.com>
Subject: [PATCH v8 4/7] usr: add support for zstd compressed initramfs
Date: Thu, 23 Jul 2020 12:27:58 -0700	[thread overview]
Message-ID: <20200723192801.351114-5-nickrterrell@gmail.com> (raw)
In-Reply-To: <20200723192801.351114-1-nickrterrell@gmail.com>

From: Nick Terrell <terrelln@fb.com>

* Add support for a zstd compressed initramfs.
* Add compression for compressing built-in initramfs with zstd.

I have tested this patch by boot testing with buildroot and QEMU.
Specifically, I booted the kernel with both a zstd and gzip compressed
initramfs, both built into the kernel and separate. I ensured that the
correct compression algorithm was used. I tested on arm, aarch64, i386,
and x86_64.

This patch has been tested in production on aarch64 and x86_64 devices.

Additionally, I have performance measurements from internal use in
production. On an aarch64 device we saw 19 second boot time improvement
from switching from lzma to zstd (27 seconds to 8 seconds). On an x86_64
device we saw a 9 second boot time reduction from switching from xz to
zstd.

Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Terrell <terrelln@fb.com>
---
 usr/Kconfig  | 20 ++++++++++++++++++++
 usr/Makefile |  1 +
 2 files changed, 21 insertions(+)

diff --git a/usr/Kconfig b/usr/Kconfig
index 96afb03b65f9..2599bc21c1b2 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -100,6 +100,15 @@ config RD_LZ4
 	  Support loading of a LZ4 encoded initial ramdisk or cpio buffer
 	  If unsure, say N.
 
+config RD_ZSTD
+	bool "Support initial ramdisk/ramfs compressed using ZSTD"
+	default y
+	depends on BLK_DEV_INITRD
+	select DECOMPRESS_ZSTD
+	help
+	  Support loading of a ZSTD encoded initial ramdisk or cpio buffer.
+	  If unsure, say N.
+
 choice
 	prompt "Built-in initramfs compression mode"
 	depends on INITRAMFS_SOURCE != ""
@@ -196,6 +205,17 @@ config INITRAMFS_COMPRESSION_LZ4
 	  If you choose this, keep in mind that most distros don't provide lz4
 	  by default which could cause a build failure.
 
+config INITRAMFS_COMPRESSION_ZSTD
+	bool "ZSTD"
+	depends on RD_ZSTD
+	help
+	  ZSTD is a compression algorithm targeting intermediate compression
+	  with fast decompression speed. It will compress better than GZIP and
+	  decompress around the same speed as LZO, but slower than LZ4.
+
+	  If you choose this, keep in mind that you may need to install the zstd
+	  tool to be able to compress the initram.
+
 config INITRAMFS_COMPRESSION_NONE
 	bool "None"
 	help
diff --git a/usr/Makefile b/usr/Makefile
index c12e6b15ce72..b1a81a40eab1 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -15,6 +15,7 @@ compress-$(CONFIG_INITRAMFS_COMPRESSION_LZMA)	:= lzma
 compress-$(CONFIG_INITRAMFS_COMPRESSION_XZ)	:= xzmisc
 compress-$(CONFIG_INITRAMFS_COMPRESSION_LZO)	:= lzo
 compress-$(CONFIG_INITRAMFS_COMPRESSION_LZ4)	:= lz4
+compress-$(CONFIG_INITRAMFS_COMPRESSION_ZSTD)	:= zstd
 
 obj-$(CONFIG_BLK_DEV_INITRD) := initramfs_data.o
 
-- 
2.27.0


  parent reply	other threads:[~2020-07-23 19:30 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-23 19:27 [GIT PULL][PATCH v8 0/7] Add support for ZSTD-compressed kernel and initramfs Nick Terrell
2020-07-23 19:27 ` [PATCH v8 1/7] lib: prepare zstd for preboot environment Nick Terrell
2020-07-23 19:27 ` [PATCH v8 2/7] lib: add zstd support to decompress Nick Terrell
2020-07-24 12:12   ` Ingo Molnar
2020-07-23 19:27 ` [PATCH v8 3/7] init: add support for zstd compressed kernel Nick Terrell
2020-07-23 20:30   ` Randy Dunlap
2020-07-24 12:17     ` Ingo Molnar
2020-07-24 14:09       ` Randy Dunlap
2020-07-23 19:27 ` Nick Terrell [this message]
2020-07-23 19:27 ` [PATCH v8 5/7] x86: bump ZO_z_extra_bytes margin for zstd Nick Terrell
2020-07-23 19:28 ` [PATCH v8 6/7] x86: Add support for ZSTD compressed kernel Nick Terrell
2020-07-24  0:50   ` Sedat Dilek
2020-07-24 14:30     ` Arvind Sankar
2020-07-24 14:48       ` Sedat Dilek
2020-07-24 17:38         ` Arvind Sankar
2020-07-27 22:44       ` Nick Terrell
2020-07-24 12:26   ` Ingo Molnar
2020-07-24 13:46     ` Adam Borowski
2020-07-27 22:43     ` Nick Terrell
2020-07-23 19:28 ` [PATCH v8 7/7] .gitignore: add ZSTD-compressed files Nick Terrell
2020-07-24  0:49 ` [GIT PULL][PATCH v8 0/7] Add support for ZSTD-compressed kernel and initramfs Sedat Dilek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200723192801.351114-5-nickrterrell@gmail.com \
    --to=nickrterrell@gmail.com \
    --cc=Kernel-team@fb.com \
    --cc=akpm@linux-foundation.org \
    --cc=alex_y_xu@yahoo.ca \
    --cc=bp@alien8.de \
    --cc=clm@fb.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=keescook@chromium.org \
    --cc=kilobyte@angband.pl \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=nivedita@alum.mit.edu \
    --cc=nolange79@gmail.com \
    --cc=oss@malat.biz \
    --cc=patrick@stwcx.xyz \
    --cc=patrickw3@fb.com \
    --cc=rmikey@fb.com \
    --cc=sedat.dilek@gmail.com \
    --cc=terrelln@fb.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.