[PATCH v2 0/2] pstore: Replace crypto API compression with zlib calls

[PATCH v2 0/2] pstore: Replace crypto API compression with zlib calls

[Ard Biesheuvel]

The pstore layer implements support for compression of kernel log
output, using a variety of compression algorithms provided by the
[deprecated] crypto API 'comp' interface.

This appears to have been somebody's pet project rather than a solution
to a real problem: the original deflate compression is reasonably fast,
compresses well and is comparatively small in terms of code footprint,
and so the flexibility that the crypto API integration provides does
little more than complicate the code for no reason.

So let's get rid of this complexity, and switch back to zlib deflate
using the library interface.

Changes since v1:
- add missing vfree() of zlib compression workspace
- implement improvements and simplifications suggested by Eric
- add missing zlib_in/deflateEnd() calls
- add code comment to document that the use of a library interface is
  deliberate, and doesn't require a future 'upgrade' to the crypto API

Cc: Kees Cook <keescook@chromium.org>
Cc: "Guilherme G. Piccoli" <gpiccoli@igalia.com>
Cc: Eric Biggers <ebiggers@kernel.org>

Ard Biesheuvel (1):
  pstore: Replace crypto API compression with zlib_deflate library calls

Kees Cook (1):
  pstore: Remove worst-case compression size logic

 fs/pstore/Kconfig    | 100 +-----
 fs/pstore/platform.c | 321 ++++++--------------
 2 files changed, 101 insertions(+), 320 deletions(-)

-- 
2.39.2

[PATCH v2 1/2] pstore: Remove worst-case compression size logic

[Ard Biesheuvel]

From: Kees Cook <keescook@chromium.org>

The worst case compression size used by pstore gives an upper bound for
how much the data might inadvertently *grow* due to encapsulation
overhead if the input is not compressible at all.

Given that pstore records have individual 'compressed' flags, we can
simply store the uncompressed data if compressing it would end up using
more space, making the worst case identical to the uncompressed case.

This means we can just drop all the elaborate logic that reasons about
upper bounds for each individual compression algorithm, and just store
the uncompressed data directly if compression fails for any reason.

Co-developed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 fs/pstore/platform.c | 202 ++------------------
 1 file changed, 15 insertions(+), 187 deletions(-)

diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c
index cbc0b468c1ab6ca1..52d7b438c4882a43 100644
--- a/fs/pstore/platform.c
+++ b/fs/pstore/platform.c
@@ -16,15 +16,6 @@
 #include <linux/console.h>
 #include <linux/module.h>
 #include <linux/pstore.h>
-#if IS_ENABLED(CONFIG_PSTORE_LZO_COMPRESS)
-#include <linux/lzo.h>
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_LZ4_COMPRESS) || IS_ENABLED(CONFIG_PSTORE_LZ4HC_COMPRESS)
-#include <linux/lz4.h>
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_ZSTD_COMPRESS)
-#include <linux/zstd.h>
-#endif
 #include <linux/crypto.h>
 #include <linux/string.h>
 #include <linux/timer.h>
@@ -97,13 +88,7 @@ MODULE_PARM_DESC(kmsg_bytes, "amount of kernel log to snapshot (in bytes)");
 /* Compression parameters */
 static struct crypto_comp *tfm;
 
-struct pstore_zbackend {
-	int (*zbufsize)(size_t size);
-	const char *name;
-};
-
 static char *big_oops_buf;
-static size_t big_oops_buf_sz;
 
 void pstore_set_kmsg_bytes(int bytes)
 {
@@ -168,105 +153,6 @@ static bool pstore_cannot_block_path(enum kmsg_dump_reason reason)
 	}
 }
 
-#if IS_ENABLED(CONFIG_PSTORE_DEFLATE_COMPRESS)
-static int zbufsize_deflate(size_t size)
-{
-	size_t cmpr;
-
-	switch (size) {
-	/* buffer range for efivars */
-	case 1000 ... 2000:
-		cmpr = 56;
-		break;
-	case 2001 ... 3000:
-		cmpr = 54;
-		break;
-	case 3001 ... 3999:
-		cmpr = 52;
-		break;
-	/* buffer range for nvram, erst */
-	case 4000 ... 10000:
-		cmpr = 45;
-		break;
-	default:
-		cmpr = 60;
-		break;
-	}
-
-	return (size * 100) / cmpr;
-}
-#endif
-
-#if IS_ENABLED(CONFIG_PSTORE_LZO_COMPRESS)
-static int zbufsize_lzo(size_t size)
-{
-	return lzo1x_worst_compress(size);
-}
-#endif
-
-#if IS_ENABLED(CONFIG_PSTORE_LZ4_COMPRESS) || IS_ENABLED(CONFIG_PSTORE_LZ4HC_COMPRESS)
-static int zbufsize_lz4(size_t size)
-{
-	return LZ4_compressBound(size);
-}
-#endif
-
-#if IS_ENABLED(CONFIG_PSTORE_842_COMPRESS)
-static int zbufsize_842(size_t size)
-{
-	return size;
-}
-#endif
-
-#if IS_ENABLED(CONFIG_PSTORE_ZSTD_COMPRESS)
-static int zbufsize_zstd(size_t size)
-{
-	return zstd_compress_bound(size);
-}
-#endif
-
-static const struct pstore_zbackend *zbackend __ro_after_init;
-
-static const struct pstore_zbackend zbackends[] = {
-#if IS_ENABLED(CONFIG_PSTORE_DEFLATE_COMPRESS)
-	{
-		.zbufsize	= zbufsize_deflate,
-		.name		= "deflate",
-	},
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_LZO_COMPRESS)
-	{
-		.zbufsize	= zbufsize_lzo,
-		.name		= "lzo",
-	},
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_LZ4_COMPRESS)
-	{
-		.zbufsize	= zbufsize_lz4,
-		.name		= "lz4",
-	},
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_LZ4HC_COMPRESS)
-	{
-		.zbufsize	= zbufsize_lz4,
-		.name		= "lz4hc",
-	},
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_842_COMPRESS)
-	{
-		.zbufsize	= zbufsize_842,
-		.name		= "842",
-	},
-#endif
-#if IS_ENABLED(CONFIG_PSTORE_ZSTD_COMPRESS)
-	{
-		.zbufsize	= zbufsize_zstd,
-		.name		= "zstd",
-	},
-#endif
-	{ }
-};
-
 static int pstore_compress(const void *in, void *out,
 			   unsigned int inlen, unsigned int outlen)
 {
@@ -287,50 +173,42 @@ static int pstore_compress(const void *in, void *out,
 static void allocate_buf_for_compression(void)
 {
 	struct crypto_comp *ctx;
-	int size;
 	char *buf;
 
 	/* Skip if not built-in or compression backend not selected yet. */
-	if (!IS_ENABLED(CONFIG_PSTORE_COMPRESS) || !zbackend)
+	if (!IS_ENABLED(CONFIG_PSTORE_COMPRESS) || !compress)
 		return;
 
 	/* Skip if no pstore backend yet or compression init already done. */
 	if (!psinfo || tfm)
 		return;
 
-	if (!crypto_has_comp(zbackend->name, 0, 0)) {
-		pr_err("Unknown compression: %s\n", zbackend->name);
+	if (!crypto_has_comp(compress, 0, 0)) {
+		pr_err("Unknown compression: %s\n", compress);
 		return;
 	}
 
-	size = zbackend->zbufsize(psinfo->bufsize);
-	if (size <= 0) {
-		pr_err("Invalid compression size for %s: %d\n",
-		       zbackend->name, size);
-		return;
-	}
-
-	buf = kmalloc(size, GFP_KERNEL);
+	/* Worst-case compression should never be more than uncompressed. */
+	buf = kmalloc(psinfo->bufsize, GFP_KERNEL);
 	if (!buf) {
-		pr_err("Failed %d byte compression buffer allocation for: %s\n",
-		       size, zbackend->name);
+		pr_err("Failed %zu byte compression buffer allocation for: %s\n",
+		       psinfo->bufsize, compress);
 		return;
 	}
 
-	ctx = crypto_alloc_comp(zbackend->name, 0, 0);
+	ctx = crypto_alloc_comp(compress, 0, 0);
 	if (IS_ERR_OR_NULL(ctx)) {
 		kfree(buf);
-		pr_err("crypto_alloc_comp('%s') failed: %ld\n", zbackend->name,
+		pr_err("crypto_alloc_comp('%s') failed: %ld\n", compress,
 		       PTR_ERR(ctx));
 		return;
 	}
 
 	/* A non-NULL big_oops_buf indicates compression is available. */
 	tfm = ctx;
-	big_oops_buf_sz = size;
 	big_oops_buf = buf;
 
-	pr_info("Using crash dump compression: %s\n", zbackend->name);
+	pr_info("Using crash dump compression: %s\n", compress);
 }
 
 static void free_buf_for_compression(void)
@@ -341,33 +219,6 @@ static void free_buf_for_compression(void)
 	}
 	kfree(big_oops_buf);
 	big_oops_buf = NULL;
-	big_oops_buf_sz = 0;
-}
-
-/*
- * Called when compression fails, since the printk buffer
- * would be fetched for compression calling it again when
- * compression fails would have moved the iterator of
- * printk buffer which results in fetching old contents.
- * Copy the recent messages from big_oops_buf to psinfo->buf
- */
-static size_t copy_kmsg_to_buffer(int hsize, size_t len)
-{
-	size_t total_len;
-	size_t diff;
-
-	total_len = hsize + len;
-
-	if (total_len > psinfo->bufsize) {
-		diff = total_len - psinfo->bufsize + hsize;
-		memcpy(psinfo->buf, big_oops_buf, hsize);
-		memcpy(psinfo->buf + hsize, big_oops_buf + diff,
-					psinfo->bufsize - hsize);
-		total_len = psinfo->bufsize;
-	} else
-		memcpy(psinfo->buf, big_oops_buf, total_len);
-
-	return total_len;
 }
 
 void pstore_record_init(struct pstore_record *record,
@@ -426,13 +277,8 @@ static void pstore_dump(struct kmsg_dumper *dumper,
 		record.part = part;
 		record.buf = psinfo->buf;
 
-		if (big_oops_buf) {
-			dst = big_oops_buf;
-			dst_size = big_oops_buf_sz;
-		} else {
-			dst = psinfo->buf;
-			dst_size = psinfo->bufsize;
-		}
+		dst = big_oops_buf ?: psinfo->buf;
+		dst_size = psinfo->bufsize;
 
 		/* Write dump header. */
 		header_size = snprintf(dst, dst_size, "%s#%d Part%u\n", why,
@@ -453,8 +299,8 @@ static void pstore_dump(struct kmsg_dumper *dumper,
 				record.compressed = true;
 				record.size = zipped_len;
 			} else {
-				record.size = copy_kmsg_to_buffer(header_size,
-								  dump_size);
+				record.size = header_size + dump_size;
+				memcpy(psinfo->buf, dst, record.size);
 			}
 		} else {
 			record.size = header_size + dump_size;
@@ -703,8 +549,7 @@ static void decompress_record(struct pstore_record *record)
 	}
 
 	/* Allocate enough space to hold max decompression and ECC. */
-	unzipped_len = big_oops_buf_sz;
-	workspace = kmalloc(unzipped_len + record->ecc_notice_size,
+	workspace = kmalloc(psinfo->bufsize + record->ecc_notice_size,
 			    GFP_KERNEL);
 	if (!workspace)
 		return;
@@ -818,27 +663,10 @@ static void pstore_timefunc(struct timer_list *unused)
 	pstore_timer_kick();
 }
 
-static void __init pstore_choose_compression(void)
-{
-	const struct pstore_zbackend *step;
-
-	if (!compress)
-		return;
-
-	for (step = zbackends; step->name; step++) {
-		if (!strcmp(compress, step->name)) {
-			zbackend = step;
-			return;
-		}
-	}
-}
-
 static int __init pstore_init(void)
 {
 	int ret;
 
-	pstore_choose_compression();
-
 	/*
 	 * Check if any pstore backends registered earlier but did not
 	 * initialize compression because crypto was not ready. If so,
-- 
2.39.2

Re: [PATCH v2 1/2] pstore: Remove worst-case compression size logic

[Guilherme G. Piccoli]

On 07/07/2023 05:34, Ard Biesheuvel wrote:
> From: Kees Cook <keescook@chromium.org>
> [...] 
> 
> Co-developed-by: Kees Cook <keescook@chromium.org>
> Signed-off-by: Kees Cook <keescook@chromium.org>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> [...]


Out of curiosity, why Kees is author but at the same time, he is a
co-developer? heh

Maybe it was Ard as either author or co-developer?
Cheers,


Guilherme

Re: [PATCH v2 1/2] pstore: Remove worst-case compression size logic

[Kees Cook]

On July 7, 2023 1:57:29 PM PDT, "Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:
>On 07/07/2023 05:34, Ard Biesheuvel wrote:
>> From: Kees Cook <keescook@chromium.org>
>> [...] 
>> 
>> Co-developed-by: Kees Cook <keescook@chromium.org>
>> Signed-off-by: Kees Cook <keescook@chromium.org>
>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>> [...]
>
>
>Out of curiosity, why Kees is author but at the same time, he is a
>co-developer? heh
>
>Maybe it was Ard as either author or co-developer?

I think that when Ard added the co-dev tag for me he forget to reset the "author" according to git.

-Kees


-- 
Kees Cook

Re: [PATCH v2 1/2] pstore: Remove worst-case compression size logic

[Eric Biggers]

On Fri, Jul 07, 2023 at 10:34:55AM +0200, Ard Biesheuvel wrote:
> From: Kees Cook <keescook@chromium.org>

As was mentioned by others, you should reset the author.

> +	/* Worst-case compression should never be more than uncompressed. */
> +	buf = kmalloc(psinfo->bufsize, GFP_KERNEL);

I don't think the above comment really conveys the reasoning.  Maybe write
something along the lines of:

	/*
         * The compression buffer only needs to be as large as the maximum
         * uncompressed record size, since any record that would be expanded by
         * compression is just stored uncompressed.
	 */

- Eric

[PATCH v2 2/2] pstore: Replace crypto API compression with zlib_deflate library calls

[Ard Biesheuvel]

Pstore supports compression using a variety of algorithms exposed by the
crypto API. This uses the deprecated comp (as opposed to scomp/acomp)
API, and so we should stop using that, and either move to the new API,
or switch to a different approach entirely.

Given that we only compress ASCII text in pstore, and considering that
this happens when the system is likely to be in an unstable state, the
flexibility that the complex crypto API provides does not outweigh its
impact on the risk that we might encounter additional problems when
trying to commit the kernel log contents to the pstore backend.

So let's switch [back] to the zlib deflate library API, and remove all
the complexity that really has no place in a low-level diagnostic
facility. Note that, while more modern compression algorithms have been
added to the kernel in recent years, the code size of zlib deflate is
substantially smaller than, e.g., zstd, while its performance in terms
of compression ratio is better than most for ASCII text, and speed is
deemed irrelevant in this context.

Note that this means that compressed pstore records may no longer be
accessible after a kernel upgrade, but this has never been part of the
contract. (The choice of compression algorithm is not stored in the
pstore records either)

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 fs/pstore/Kconfig    | 100 ++-------------
 fs/pstore/platform.c | 131 +++++++++++++-------
 2 files changed, 92 insertions(+), 139 deletions(-)

diff --git a/fs/pstore/Kconfig b/fs/pstore/Kconfig
index c49d554cc9ae9f80..3acc38600cd1a2d0 100644
--- a/fs/pstore/Kconfig
+++ b/fs/pstore/Kconfig
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 config PSTORE
 	tristate "Persistent store support"
-	select CRYPTO if PSTORE_COMPRESS
 	default n
 	help
 	   This option enables generic access to platform level
@@ -22,99 +21,18 @@ config PSTORE_DEFAULT_KMSG_BYTES
 	  Defines default size of pstore kernel log storage.
 	  Can be enlarged if needed, not recommended to shrink it.
 
-config PSTORE_DEFLATE_COMPRESS
-	tristate "DEFLATE (ZLIB) compression"
-	default y
-	depends on PSTORE
-	select CRYPTO_DEFLATE
-	help
-	  This option enables DEFLATE (also known as ZLIB) compression
-	  algorithm support.
-
-config PSTORE_LZO_COMPRESS
-	tristate "LZO compression"
-	depends on PSTORE
-	select CRYPTO_LZO
-	help
-	  This option enables LZO compression algorithm support.
-
-config PSTORE_LZ4_COMPRESS
-	tristate "LZ4 compression"
-	depends on PSTORE
-	select CRYPTO_LZ4
-	help
-	  This option enables LZ4 compression algorithm support.
-
-config PSTORE_LZ4HC_COMPRESS
-	tristate "LZ4HC compression"
-	depends on PSTORE
-	select CRYPTO_LZ4HC
-	help
-	  This option enables LZ4HC (high compression) mode algorithm.
-
-config PSTORE_842_COMPRESS
-	bool "842 compression"
-	depends on PSTORE
-	select CRYPTO_842
-	help
-	  This option enables 842 compression algorithm support.
-
-config PSTORE_ZSTD_COMPRESS
-	bool "zstd compression"
-	depends on PSTORE
-	select CRYPTO_ZSTD
-	help
-	  This option enables zstd compression algorithm support.
-
 config PSTORE_COMPRESS
-	def_bool y
+	bool "Pstore compression (deflate)"
 	depends on PSTORE
-	depends on PSTORE_DEFLATE_COMPRESS || PSTORE_LZO_COMPRESS ||	\
-		   PSTORE_LZ4_COMPRESS || PSTORE_LZ4HC_COMPRESS ||	\
-		   PSTORE_842_COMPRESS || PSTORE_ZSTD_COMPRESS
-
-choice
-	prompt "Default pstore compression algorithm"
-	depends on PSTORE_COMPRESS
+	select ZLIB_INFLATE
+	select ZLIB_DEFLATE
+	default y
 	help
-	  This option chooses the default active compression algorithm.
-	  This change be changed at boot with "pstore.compress=..." on
-	  the kernel command line.
-
-	  Currently, pstore has support for 6 compression algorithms:
-	  deflate, lzo, lz4, lz4hc, 842 and zstd.
-
-	  The default compression algorithm is deflate.
-
-	config PSTORE_DEFLATE_COMPRESS_DEFAULT
-		bool "deflate" if PSTORE_DEFLATE_COMPRESS
-
-	config PSTORE_LZO_COMPRESS_DEFAULT
-		bool "lzo" if PSTORE_LZO_COMPRESS
-
-	config PSTORE_LZ4_COMPRESS_DEFAULT
-		bool "lz4" if PSTORE_LZ4_COMPRESS
-
-	config PSTORE_LZ4HC_COMPRESS_DEFAULT
-		bool "lz4hc" if PSTORE_LZ4HC_COMPRESS
-
-	config PSTORE_842_COMPRESS_DEFAULT
-		bool "842" if PSTORE_842_COMPRESS
-
-	config PSTORE_ZSTD_COMPRESS_DEFAULT
-		bool "zstd" if PSTORE_ZSTD_COMPRESS
-
-endchoice
-
-config PSTORE_COMPRESS_DEFAULT
-	string
-	depends on PSTORE_COMPRESS
-	default "deflate" if PSTORE_DEFLATE_COMPRESS_DEFAULT
-	default "lzo" if PSTORE_LZO_COMPRESS_DEFAULT
-	default "lz4" if PSTORE_LZ4_COMPRESS_DEFAULT
-	default "lz4hc" if PSTORE_LZ4HC_COMPRESS_DEFAULT
-	default "842" if PSTORE_842_COMPRESS_DEFAULT
-	default "zstd" if PSTORE_ZSTD_COMPRESS_DEFAULT
+	  Whether pstore records should be compressed before being written to
+	  the backing store. This is implemented using the zlib 'deflate'
+	  algorithm, using the library implementation instead of using the full
+	  blown crypto API. This reduces the risk of secondary oopses or other
+	  problems while pstore is recording panic metadata.
 
 config PSTORE_CONSOLE
 	bool "Log kernel console messages"
diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c
index 52d7b438c4882a43..9a0db551e8af67fa 100644
--- a/fs/pstore/platform.c
+++ b/fs/pstore/platform.c
@@ -16,13 +16,14 @@
 #include <linux/console.h>
 #include <linux/module.h>
 #include <linux/pstore.h>
-#include <linux/crypto.h>
 #include <linux/string.h>
 #include <linux/timer.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
 #include <linux/jiffies.h>
+#include <linux/vmalloc.h>
 #include <linux/workqueue.h>
+#include <linux/zlib.h>
 
 #include "internal.h"
 
@@ -71,12 +72,22 @@ static char *backend;
 module_param(backend, charp, 0444);
 MODULE_PARM_DESC(backend, "specific backend to use");
 
-static char *compress =
-#ifdef CONFIG_PSTORE_COMPRESS_DEFAULT
-		CONFIG_PSTORE_COMPRESS_DEFAULT;
-#else
-		NULL;
-#endif
+/*
+ * pstore no longer implements compression via the crypto API, and only
+ * supports zlib deflate compression implemented using the zlib library
+ * interface. This removes additional complexity which is hard to justify for a
+ * diagnostic facility that has to operate in conditions where the system may
+ * have become unstable. Zlib deflate is comparatively small in terms of code
+ * size, and compresses ASCII text as well as any other compression algorithm
+ * available in the kernel. In terms of compression speed, deflate is not the
+ * best performer but for recording the log output on a kernel panic, this is
+ * not considered critical.
+ *
+ * The only remaining arguments supported by the compress= module parameter are
+ * 'deflate' and 'none'. To retain compatibility with existing installations,
+ * all other values are logged and replaced with 'deflate'.
+ */
+static char *compress = "deflate";
 module_param(compress, charp, 0444);
 MODULE_PARM_DESC(compress, "compression to use");
 
@@ -85,8 +96,7 @@ unsigned long kmsg_bytes = CONFIG_PSTORE_DEFAULT_KMSG_BYTES;
 module_param(kmsg_bytes, ulong, 0444);
 MODULE_PARM_DESC(kmsg_bytes, "amount of kernel log to snapshot (in bytes)");
 
-/* Compression parameters */
-static struct crypto_comp *tfm;
+static void *compress_workspace;
 
 static char *big_oops_buf;
 
@@ -156,36 +166,49 @@ static bool pstore_cannot_block_path(enum kmsg_dump_reason reason)
 static int pstore_compress(const void *in, void *out,
 			   unsigned int inlen, unsigned int outlen)
 {
+	struct z_stream_s zstream = {
+		.next_in	= in,
+		.avail_in	= inlen,
+		.next_out	= out,
+		.avail_out	= outlen,
+		.workspace	= compress_workspace,
+	};
 	int ret;
 
 	if (!IS_ENABLED(CONFIG_PSTORE_COMPRESS))
 		return -EINVAL;
 
-	ret = crypto_comp_compress(tfm, in, inlen, out, &outlen);
-	if (ret) {
-		pr_err("crypto_comp_compress failed, ret = %d!\n", ret);
-		return ret;
-	}
+	ret = zlib_deflateInit2(&zstream, Z_DEFAULT_COMPRESSION, Z_DEFLATED,
+				-MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY);
+	if (ret != Z_OK)
+		return -EINVAL;
 
-	return outlen;
+	ret = zlib_deflate(&zstream, Z_FINISH);
+	if (ret != Z_STREAM_END)
+		return -EINVAL;
+
+	ret = zlib_deflateEnd(&zstream);
+	if (ret != Z_OK)
+		pr_warn_once("zlib_deflateEnd() failed: %d\n", ret);
+
+	return zstream.total_out;
 }
 
 static void allocate_buf_for_compression(void)
 {
-	struct crypto_comp *ctx;
 	char *buf;
 
-	/* Skip if not built-in or compression backend not selected yet. */
-	if (!IS_ENABLED(CONFIG_PSTORE_COMPRESS) || !compress)
+	/* Skip if not built-in or compression disabled. */
+	if (!IS_ENABLED(CONFIG_PSTORE_COMPRESS) || !compress ||
+	    !strcmp(compress, "none")) {
+		compress = NULL;
 		return;
+	}
 
-	/* Skip if no pstore backend yet or compression init already done. */
-	if (!psinfo || tfm)
-		return;
-
-	if (!crypto_has_comp(compress, 0, 0)) {
-		pr_err("Unknown compression: %s\n", compress);
-		return;
+	if (strcmp(compress, "deflate")) {
+		pr_err("Unsupported compression '%s', falling back to deflate\n",
+		       compress);
+		compress = "deflate";
 	}
 
 	/* Worst-case compression should never be more than uncompressed. */
@@ -196,16 +219,15 @@ static void allocate_buf_for_compression(void)
 		return;
 	}
 
-	ctx = crypto_alloc_comp(compress, 0, 0);
-	if (IS_ERR_OR_NULL(ctx)) {
+	compress_workspace =
+		vmalloc(zlib_deflate_workspacesize(MAX_WBITS, DEF_MEM_LEVEL));
+	if (!compress_workspace) {
+		pr_err("Failed to allocate zlib deflate workspace\n");
 		kfree(buf);
-		pr_err("crypto_alloc_comp('%s') failed: %ld\n", compress,
-		       PTR_ERR(ctx));
 		return;
 	}
 
 	/* A non-NULL big_oops_buf indicates compression is available. */
-	tfm = ctx;
 	big_oops_buf = buf;
 
 	pr_info("Using crash dump compression: %s\n", compress);
@@ -213,10 +235,7 @@ static void allocate_buf_for_compression(void)
 
 static void free_buf_for_compression(void)
 {
-	if (IS_ENABLED(CONFIG_PSTORE_COMPRESS) && tfm) {
-		crypto_free_comp(tfm);
-		tfm = NULL;
-	}
+	vfree(compress_workspace);
 	kfree(big_oops_buf);
 	big_oops_buf = NULL;
 }
@@ -527,7 +546,8 @@ void pstore_unregister(struct pstore_info *psi)
 }
 EXPORT_SYMBOL_GPL(pstore_unregister);
 
-static void decompress_record(struct pstore_record *record)
+static void decompress_record(struct pstore_record *record,
+			      struct z_stream_s *zstream)
 {
 	int ret;
 	int unzipped_len;
@@ -548,21 +568,32 @@ static void decompress_record(struct pstore_record *record)
 		return;
 	}
 
+	ret = zlib_inflateReset(zstream);
+	if (ret != Z_OK) {
+		pr_err("zlib_inflateReset() failed, ret = %d!\n", ret);
+		return;
+	}
+
 	/* Allocate enough space to hold max decompression and ECC. */
 	workspace = kmalloc(psinfo->bufsize + record->ecc_notice_size,
 			    GFP_KERNEL);
 	if (!workspace)
 		return;
 
-	/* After decompression "unzipped_len" is almost certainly smaller. */
-	ret = crypto_comp_decompress(tfm, record->buf, record->size,
-					  workspace, &unzipped_len);
-	if (ret) {
-		pr_err("crypto_comp_decompress failed, ret = %d!\n", ret);
+	zstream->next_in	= record->buf;
+	zstream->avail_in	= record->size;
+	zstream->next_out	= workspace;
+	zstream->avail_out	= psinfo->bufsize;
+
+	ret = zlib_inflate(zstream, Z_FINISH);
+	if (ret != Z_STREAM_END) {
+		pr_err("zlib_inflate() failed, ret = %d!\n", ret);
 		kfree(workspace);
 		return;
 	}
 
+	unzipped_len = zstream->total_out;
+
 	/* Append ECC notice to decompressed buffer. */
 	memcpy(workspace + unzipped_len, record->buf + record->size,
 	       record->ecc_notice_size);
@@ -592,10 +623,17 @@ void pstore_get_backend_records(struct pstore_info *psi,
 {
 	int failed = 0;
 	unsigned int stop_loop = 65536;
+	struct z_stream_s zstream;
 
 	if (!psi || !root)
 		return;
 
+	if (IS_ENABLED(CONFIG_PSTORE_COMPRESS) && compress) {
+		zstream.workspace = kvmalloc(zlib_inflate_workspacesize(),
+					     GFP_KERNEL);
+		zlib_inflateInit2(&zstream, -DEF_WBITS);
+	}
+
 	mutex_lock(&psi->read_mutex);
 	if (psi->open && psi->open(psi))
 		goto out;
@@ -624,7 +662,7 @@ void pstore_get_backend_records(struct pstore_info *psi,
 			break;
 		}
 
-		decompress_record(record);
+		decompress_record(record, &zstream);
 		rc = pstore_mkfile(root, record);
 		if (rc) {
 			/* pstore_mkfile() did not take record, so free it. */
@@ -639,6 +677,10 @@ void pstore_get_backend_records(struct pstore_info *psi,
 		psi->close(psi);
 out:
 	mutex_unlock(&psi->read_mutex);
+	kvfree(zstream.workspace);
+
+	if (zlib_inflateEnd(&zstream) != Z_OK)
+		pr_warn("zlib_inflateEnd() failed\n");
 
 	if (failed)
 		pr_warn("failed to create %d record(s) from '%s'\n",
@@ -667,13 +709,6 @@ static int __init pstore_init(void)
 {
 	int ret;
 
-	/*
-	 * Check if any pstore backends registered earlier but did not
-	 * initialize compression because crypto was not ready. If so,
-	 * initialize compression now.
-	 */
-	allocate_buf_for_compression();
-
 	ret = pstore_init_fs();
 	if (ret)
 		free_buf_for_compression();
-- 
2.39.2

Re: [PATCH v2 2/2] pstore: Replace crypto API compression with zlib_deflate library calls

[Eric Biggers]

On Fri, Jul 07, 2023 at 10:34:56AM +0200, Ard Biesheuvel wrote:
> +/*
> + * pstore no longer implements compression via the crypto API, and only
> + * supports zlib deflate compression implemented using the zlib library
> + * interface. This removes additional complexity which is hard to justify for a
> + * diagnostic facility that has to operate in conditions where the system may
> + * have become unstable. Zlib deflate is comparatively small in terms of code
> + * size, and compresses ASCII text as well as any other compression algorithm
> + * available in the kernel. In terms of compression speed, deflate is not the
> + * best performer but for recording the log output on a kernel panic, this is
> + * not considered critical.
> + *
> + * The only remaining arguments supported by the compress= module parameter are
> + * 'deflate' and 'none'. To retain compatibility with existing installations,
> + * all other values are logged and replaced with 'deflate'.
> + */
> +static char *compress = "deflate";

I would soften the claim "compresses ASCII text as well as any other compression
algorithm available in the kernel" to something like "compresses ASCII text
comparatively well".  This is because zstd can indeed compress ASCII text
slightly more than deflate.

> static void free_buf_for_compression(void)
> {
> -       if (IS_ENABLED(CONFIG_PSTORE_COMPRESS) && tfm) {
> -               crypto_free_comp(tfm);
> -               tfm = NULL;
> -       }
> +       vfree(compress_workspace);
>         kfree(big_oops_buf);
>         big_oops_buf = NULL;
>  }

Maybe set compress_workspace to NULL after it is freed.

> @@ -592,10 +623,17 @@ void pstore_get_backend_records(struct pstore_info *psi,
>  {
>  	int failed = 0;
>  	unsigned int stop_loop = 65536;
> +	struct z_stream_s zstream;
>  
>  	if (!psi || !root)
>  		return;
>  
> +	if (IS_ENABLED(CONFIG_PSTORE_COMPRESS) && compress) {
> +		zstream.workspace = kvmalloc(zlib_inflate_workspacesize(),
> +					     GFP_KERNEL);
> +		zlib_inflateInit2(&zstream, -DEF_WBITS);
> +	}
> +

zstream.workspace is never checked for NULL.  Maybe do:

	struct z_stream_s zstream = {};

and have decompress_record() check it.  Also I think it should replace the check
for big_oops_buf, as big_oops_buf is really for compression, not decompression.


>  	mutex_lock(&psi->read_mutex);
>  	if (psi->open && psi->open(psi))
>  		goto out;
> @@ -624,7 +662,7 @@ void pstore_get_backend_records(struct pstore_info *psi,
>  			break;
>  		}
>  
> -		decompress_record(record);
> +		decompress_record(record, &zstream);
>  		rc = pstore_mkfile(root, record);
>  		if (rc) {
>  			/* pstore_mkfile() did not take record, so free it. */
> @@ -639,6 +677,10 @@ void pstore_get_backend_records(struct pstore_info *psi,
>  		psi->close(psi);
>  out:
>  	mutex_unlock(&psi->read_mutex);
> +	kvfree(zstream.workspace);
> +
> +	if (zlib_inflateEnd(&zstream) != Z_OK)
> +		pr_warn("zlib_inflateEnd() failed\n");

The destruction code above needs to be guarded by the same condition as the
initialization code ('IS_ENABLED(CONFIG_PSTORE_COMPRESS) && compress').

- Eric

Re: [PATCH v2 0/2] pstore: Replace crypto API compression with zlib calls

[Guilherme G. Piccoli]

On 07/07/2023 05:34, Ard Biesheuvel wrote:
> The pstore layer implements support for compression of kernel log
> output, using a variety of compression algorithms provided by the
> [deprecated] crypto API 'comp' interface.
> 
> This appears to have been somebody's pet project rather than a solution
> to a real problem: the original deflate compression is reasonably fast,
> compresses well and is comparatively small in terms of code footprint,
> and so the flexibility that the crypto API integration provides does
> little more than complicate the code for no reason.
> 
> So let's get rid of this complexity, and switch back to zlib deflate
> using the library interface.
> 
> Changes since v1:
> - add missing vfree() of zlib compression workspace
> - implement improvements and simplifications suggested by Eric
> - add missing zlib_in/deflateEnd() calls
> - add code comment to document that the use of a library interface is
>   deliberate, and doesn't require a future 'upgrade' to the crypto API
> [...]

Thanks a lot Ard/Kees, quite a great improvement! I've tested on top of
6.4.0 in the Steam Deck and it's working perfectly fine - no noticeable
extra delay in the log collecting during panic compared to zstd.

Feel free to add (to the series):
Tested-by: Guilherme G. Piccoli <gpiccoli@igalia.com> # Steam Deck

Cheers,


Guilherme