[PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Takashi Iwai]

Hi,

it seems that ZSTD format is getting popular, and I've been asked
about the firmware loader support.  So I took a quick glance, and it
turned out that it's fairly easy thanks to the existing ZSTD API.
Now high time to submit something.

The first patch adds a new Kconfig CONFIG_FW_LOADER_COMPRESS_ZSTD and
the corresponding decompression function to the firmware loader code.
For the already supported XZ-compression, CONFIG_FW_LOADER_COMPRESS_XZ
is added to make it selectable explicitly, too.

The rest three patches are for selftest: a cleanup, a fix and the
additional support of ZSTD format.

Currently, I have no idea whether any distro would use ZSTD files for
firmware files in near future, though.  That's the reason of this
patch set being an RFC for now.


thanks,

Takashi

===

Takashi Iwai (4):
  firmware: Add the support for ZSTD-compressed firmware files
  selftests: firmware: Simplify test patterns
  selftest: firmware: Fix the request_firmware_into_buf() test for XZ
    format
  selftest: firmware: Add ZSTD compressed file tests

 drivers/base/firmware_loader/Kconfig          |  21 ++-
 drivers/base/firmware_loader/main.c           |  74 +++++++-
 .../selftests/firmware/fw_filesystem.sh       | 167 +++++++++---------
 tools/testing/selftests/firmware/fw_lib.sh    |  12 +-
 4 files changed, 175 insertions(+), 99 deletions(-)

-- 
2.26.2

[PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Takashi Iwai]

Due to the popular demands on ZSTD, here is a patch to add a support
of ZSTD-compressed firmware files via the direct firmware loader.
It's just like XZ-compressed file support, providing a decompressor
with ZSTD.  Since ZSTD API can give the decompression size beforehand,
the code is even simpler than XZ.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 drivers/base/firmware_loader/Kconfig | 21 ++++++--
 drivers/base/firmware_loader/main.c  | 74 ++++++++++++++++++++++++++--
 2 files changed, 87 insertions(+), 8 deletions(-)

diff --git a/drivers/base/firmware_loader/Kconfig b/drivers/base/firmware_loader/Kconfig
index 5b24f3959255..f5307978927c 100644
--- a/drivers/base/firmware_loader/Kconfig
+++ b/drivers/base/firmware_loader/Kconfig
@@ -157,17 +157,28 @@ config FW_LOADER_USER_HELPER_FALLBACK
 
 config FW_LOADER_COMPRESS
 	bool "Enable compressed firmware support"
-	select FW_LOADER_PAGED_BUF
-	select XZ_DEC
 	help
 	  This option enables the support for loading compressed firmware
 	  files. The caller of firmware API receives the decompressed file
 	  content. The compressed file is loaded as a fallback, only after
 	  loading the raw file failed at first.
 
-	  Currently only XZ-compressed files are supported, and they have to
-	  be compressed with either none or crc32 integrity check type (pass
-	  "-C crc32" option to xz command).
+if FW_LOADER_COMPRESS
+config FW_LOADER_COMPRESS_XZ
+	bool "Enable XZ-compressed firmware support"
+	select FW_LOADER_PAGED_BUF
+	select XZ_DEC
+	help
+	  This option adds the support for XZ-compressed files.
+	  The files have to be compressed with either none or crc32
+	  integrity check type (pass "-C crc32" option to xz command).
+
+config FW_LOADER_COMPRESS_ZSTD
+	bool "Enable ZSTD-compressed firmware support"
+	select ZSTD_DECOMPRESS
+	help
+	  This option adds the support for ZSTD-compressed files.
+endif # FW_LOADER_COMPRESS
 
 config FW_CACHE
 	bool "Enable firmware caching during suspend"
diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c
index 78355095e00d..71332ed4959d 100644
--- a/drivers/base/firmware_loader/main.c
+++ b/drivers/base/firmware_loader/main.c
@@ -34,6 +34,7 @@
 #include <linux/syscore_ops.h>
 #include <linux/reboot.h>
 #include <linux/security.h>
+#include <linux/zstd.h>
 #include <linux/xz.h>
 
 #include <generated/utsrelease.h>
@@ -362,10 +363,72 @@ int fw_map_paged_buf(struct fw_priv *fw_priv)
 }
 #endif
 
+/*
+ * ZSTD-compressed firmware support
+ */
+#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
+static int fw_decompress_zstd(struct device *dev, struct fw_priv *fw_priv,
+			      size_t in_size, const void *in_buffer)
+{
+	size_t len, out_size, workspace_size;
+	void *workspace, *out_buf;
+	ZSTD_DCtx *ctx;
+	int err;
+
+	if (fw_priv->data) {
+		out_size = fw_priv->allocated_size;
+		out_buf = fw_priv->data;
+	} else {
+		out_size = ZSTD_findDecompressedSize(in_buffer, in_size);
+		if (out_size == ZSTD_CONTENTSIZE_UNKNOWN ||
+		    out_size == ZSTD_CONTENTSIZE_ERROR) {
+			dev_dbg(dev, "%s: invalid decompression size\n", __func__);
+			return -EINVAL;
+		}
+		out_buf = vzalloc(out_size);
+		if (!out_buf)
+			return -ENOMEM;
+	}
+
+	workspace_size = ZSTD_DCtxWorkspaceBound();
+	workspace = kvzalloc(workspace_size, GFP_KERNEL);
+	if (!workspace) {
+		err = -ENOMEM;
+		goto error;
+	}
+
+	ctx = ZSTD_initDCtx(workspace, workspace_size);
+	if (!ctx) {
+		dev_dbg(dev, "%s: failed to initialize context\n", __func__);
+		err = -EINVAL;
+		goto error;
+	}
+
+	len = ZSTD_decompressDCtx(ctx, out_buf, out_size, in_buffer, in_size);
+	if (ZSTD_isError(len)) {
+		dev_dbg(dev, "%s: failed to decompress: %d\n", __func__,
+			ZSTD_getErrorCode(len));
+		err = -EINVAL;
+		goto error;
+	}
+
+	fw_priv->size = len;
+	if (!fw_priv->data)
+		fw_priv->data = out_buf;
+	err = 0;
+
+ error:
+	kvfree(workspace);
+	if (!fw_priv->data)
+		vfree(out_buf);
+	return err;
+}
+#endif /* CONFIG_FW_LOADER_COMPRESS_ZSTD */
+
 /*
  * XZ-compressed firmware support
  */
-#ifdef CONFIG_FW_LOADER_COMPRESS
+#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
 /* show an error and return the standard error code */
 static int fw_decompress_xz_error(struct device *dev, enum xz_ret xz_ret)
 {
@@ -459,7 +522,7 @@ static int fw_decompress_xz(struct device *dev, struct fw_priv *fw_priv,
 	else
 		return fw_decompress_xz_pages(dev, fw_priv, in_size, in_buffer);
 }
-#endif /* CONFIG_FW_LOADER_COMPRESS */
+#endif /* CONFIG_FW_LOADER_COMPRESS_XZ */
 
 /* direct firmware loading support */
 static char fw_path_para[256];
@@ -814,7 +877,12 @@ _request_firmware(const struct firmware **firmware_p, const char *name,
 	if (!(opt_flags & FW_OPT_PARTIAL))
 		nondirect = true;
 
-#ifdef CONFIG_FW_LOADER_COMPRESS
+#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
+	if (ret == -ENOENT && nondirect)
+		ret = fw_get_filesystem_firmware(device, fw->priv, ".zst",
+						 fw_decompress_zstd);
+#endif
+#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
 	if (ret == -ENOENT && nondirect)
 		ret = fw_get_filesystem_firmware(device, fw->priv, ".xz",
 						 fw_decompress_xz);
-- 
2.26.2

[PATCH RFC 2/4] selftests: firmware: Simplify test patterns

[Takashi Iwai]

The test patterns are almost same in three sequential tests.
Make the unified helper function for improving the readability.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 .../selftests/firmware/fw_filesystem.sh       | 106 +++++-------------
 1 file changed, 30 insertions(+), 76 deletions(-)

diff --git a/tools/testing/selftests/firmware/fw_filesystem.sh b/tools/testing/selftests/firmware/fw_filesystem.sh
index c2a2a100114b..2424a97da65b 100755
--- a/tools/testing/selftests/firmware/fw_filesystem.sh
+++ b/tools/testing/selftests/firmware/fw_filesystem.sh
@@ -435,6 +435,32 @@ test_request_partial_firmware_into_buf()
 	echo "OK"
 }
 
+do_tests ()
+{
+	mode="$1"
+	suffix="$2"
+
+	for i in $(seq 1 5); do
+		test_batched_request_firmware$suffix $i $mode
+	done
+
+	for i in $(seq 1 5); do
+		test_batched_request_firmware_into_buf$suffix $i $mode
+	done
+
+	for i in $(seq 1 5); do
+		test_batched_request_firmware_direct$suffix $i $mode
+	done
+
+	for i in $(seq 1 5); do
+		test_request_firmware_nowait_uevent$suffix $i $mode
+	done
+
+	for i in $(seq 1 5); do
+		test_request_firmware_nowait_custom$suffix $i $mode
+	done
+}
+
 # Only continue if batched request triggers are present on the
 # test-firmware driver
 test_config_present
@@ -442,25 +468,7 @@ test_config_present
 # test with the file present
 echo
 echo "Testing with the file present..."
-for i in $(seq 1 5); do
-	test_batched_request_firmware $i normal
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_into_buf $i normal
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_direct $i normal
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_uevent $i normal
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_custom $i normal
-done
+do_tests normal
 
 # Partial loads cannot use fallback, so do not repeat tests.
 test_request_partial_firmware_into_buf 0 10
@@ -472,25 +480,7 @@ test_request_partial_firmware_into_buf 2 10
 # a hung task, which would require a hard reset.
 echo
 echo "Testing with the file missing..."
-for i in $(seq 1 5); do
-	test_batched_request_firmware_nofile $i
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_into_buf_nofile $i
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_direct_nofile $i
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_uevent_nofile $i
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_custom_nofile $i
-done
+do_tests nofile _nofile
 
 # Partial loads cannot use fallback, so do not repeat tests.
 test_request_partial_firmware_into_buf_nofile 0 10
@@ -505,48 +495,12 @@ xz -9 -C crc32 -k $FW
 config_set_name $NAME
 echo
 echo "Testing with both plain and xz files present..."
-for i in $(seq 1 5); do
-	test_batched_request_firmware $i both
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_into_buf $i both
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_direct $i both
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_uevent $i both
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_custom $i both
-done
+do_tests both
 
 # test with only xz file present
 mv "$FW" "${FW}-orig"
 echo
 echo "Testing with only xz file present..."
-for i in $(seq 1 5); do
-	test_batched_request_firmware $i xzonly
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_into_buf $i xzonly
-done
-
-for i in $(seq 1 5); do
-	test_batched_request_firmware_direct $i xzonly
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_uevent $i xzonly
-done
-
-for i in $(seq 1 5); do
-	test_request_firmware_nowait_custom $i xzonly
-done
+do_tests xzonly
 
 exit 0
-- 
2.26.2

[PATCH RFC 3/4] selftest: firmware: Fix the request_firmware_into_buf() test for XZ format

[Takashi Iwai]

The test uses a different firmware name, and we forgot to adapt for
the XZ compressed file tests.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 tools/testing/selftests/firmware/fw_filesystem.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/testing/selftests/firmware/fw_filesystem.sh b/tools/testing/selftests/firmware/fw_filesystem.sh
index 2424a97da65b..f1976e650672 100755
--- a/tools/testing/selftests/firmware/fw_filesystem.sh
+++ b/tools/testing/selftests/firmware/fw_filesystem.sh
@@ -492,6 +492,7 @@ test "$HAS_FW_LOADER_COMPRESS" != "yes" && exit 0
 
 # test with both files present
 xz -9 -C crc32 -k $FW
+xz -9 -C crc32 -k $FW_INTO_BUF
 config_set_name $NAME
 echo
 echo "Testing with both plain and xz files present..."
@@ -499,6 +500,7 @@ do_tests both
 
 # test with only xz file present
 mv "$FW" "${FW}-orig"
+mv "$FW_INTO_BUF" "${FW_INTO_BUF}-orig"
 echo
 echo "Testing with only xz file present..."
 do_tests xzonly
-- 
2.26.2

[PATCH RFC 4/4] selftest: firmware: Add ZSTD compressed file tests

[Takashi Iwai]

It's similar like XZ compressed files.  For the simplicity, both XZ
and ZSTD tests are done in a single function.  The format is specified
via $COMPRESS_FORMAT and the compression function is pre-defined.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 .../selftests/firmware/fw_filesystem.sh       | 75 ++++++++++++++-----
 tools/testing/selftests/firmware/fw_lib.sh    | 12 ++-
 2 files changed, 64 insertions(+), 23 deletions(-)

diff --git a/tools/testing/selftests/firmware/fw_filesystem.sh b/tools/testing/selftests/firmware/fw_filesystem.sh
index f1976e650672..abaceac83d6e 100755
--- a/tools/testing/selftests/firmware/fw_filesystem.sh
+++ b/tools/testing/selftests/firmware/fw_filesystem.sh
@@ -211,7 +211,7 @@ read_firmwares()
 	else
 		fwfile="$FW"
 	fi
-	if [ "$1" = "xzonly" ]; then
+	if [ "$1" = "componly" ]; then
 		fwfile="${fwfile}-orig"
 	fi
 	for i in $(seq 0 3); do
@@ -235,7 +235,7 @@ read_partial_firmwares()
 		fwfile="${FW}"
 	fi
 
-	if [ "$1" = "xzonly" ]; then
+	if [ "$1" = "componly" ]; then
 		fwfile="${fwfile}-orig"
 	fi
 
@@ -409,10 +409,8 @@ test_request_firmware_nowait_custom()
 	config_unset_uevent
 	RANDOM_FILE_PATH=$(setup_random_file)
 	RANDOM_FILE="$(basename $RANDOM_FILE_PATH)"
-	if [ "$2" = "both" ]; then
-		xz -9 -C crc32 -k $RANDOM_FILE_PATH
-	elif [ "$2" = "xzonly" ]; then
-		xz -9 -C crc32 $RANDOM_FILE_PATH
+	if [ -n "$2" -a "$2" != "normal" ]; then
+		compress-"$2"-$COMPRESS_FORMAT $RANDOM_FILE_PATH
 	fi
 	config_set_name $RANDOM_FILE
 	config_trigger_async
@@ -488,21 +486,58 @@ test_request_partial_firmware_into_buf_nofile 0 5
 test_request_partial_firmware_into_buf_nofile 1 6
 test_request_partial_firmware_into_buf_nofile 2 10
 
-test "$HAS_FW_LOADER_COMPRESS" != "yes" && exit 0
+test_request_firmware_compressed ()
+{
+	export COMPRESS_FORMAT="$1"
 
-# test with both files present
-xz -9 -C crc32 -k $FW
-xz -9 -C crc32 -k $FW_INTO_BUF
-config_set_name $NAME
-echo
-echo "Testing with both plain and xz files present..."
-do_tests both
+	# test with both files present
+	compress-both-$COMPRESS_FORMAT $FW
+	compress-both-$COMPRESS_FORMAT $FW_INTO_BUF
 
-# test with only xz file present
-mv "$FW" "${FW}-orig"
-mv "$FW_INTO_BUF" "${FW_INTO_BUF}-orig"
-echo
-echo "Testing with only xz file present..."
-do_tests xzonly
+	config_set_name $NAME
+	echo
+	echo "Testing with both plain and $COMPRESS_FORMAT files present..."
+	do_tests both
+
+	# test with only compressed file present
+	mv "$FW" "${FW}-orig"
+	mv "$FW_INTO_BUF" "${FW_INTO_BUF}-orig"
+
+	config_set_name $NAME
+	echo
+	echo "Testing with only $COMPRESS_FORMAT file present..."
+	do_tests componly
+
+	mv "${FW}-orig" "$FW"
+	mv "${FW_INTO_BUF}-orig" "$FW_INTO_BUF"
+}
+
+compress-both-XZ ()
+{
+	xz -k -9 -C crc32 "$@"
+}
+
+compress-componly-XZ ()
+{
+	xz -9 -C crc32 "$@"
+}
+
+compress-both-ZSTD ()
+{
+	zstd -q -k "$@"
+}
+
+compress-componly-ZSTD ()
+{
+	zstd -q --rm "$@"
+}
+
+if test "$HAS_FW_LOADER_COMPRESS_XZ" = "yes"; then
+	test_request_firmware_compressed XZ
+fi
+
+if test "$HAS_FW_LOADER_COMPRESS_ZSTD" = "yes"; then
+	test_request_firmware_compressed ZSTD
+fi
 
 exit 0
diff --git a/tools/testing/selftests/firmware/fw_lib.sh b/tools/testing/selftests/firmware/fw_lib.sh
index 5b8c0fedee76..3fa8282b053b 100755
--- a/tools/testing/selftests/firmware/fw_lib.sh
+++ b/tools/testing/selftests/firmware/fw_lib.sh
@@ -62,7 +62,8 @@ check_setup()
 {
 	HAS_FW_LOADER_USER_HELPER="$(kconfig_has CONFIG_FW_LOADER_USER_HELPER=y)"
 	HAS_FW_LOADER_USER_HELPER_FALLBACK="$(kconfig_has CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y)"
-	HAS_FW_LOADER_COMPRESS="$(kconfig_has CONFIG_FW_LOADER_COMPRESS=y)"
+	HAS_FW_LOADER_COMPRESS_XZ="$(kconfig_has CONFIG_FW_LOADER_COMPRESS_XZ=y)"
+	HAS_FW_LOADER_COMPRESS_ZSTD="$(kconfig_has CONFIG_FW_LOADER_COMPRESS_ZSTD=y)"
 	PROC_FW_IGNORE_SYSFS_FALLBACK="0"
 	PROC_FW_FORCE_SYSFS_FALLBACK="0"
 
@@ -98,9 +99,14 @@ check_setup()
 
 	OLD_FWPATH="$(cat /sys/module/firmware_class/parameters/path)"
 
-	if [ "$HAS_FW_LOADER_COMPRESS" = "yes" ]; then
+	if [ "$HAS_FW_LOADER_COMPRESS_XZ" = "yes" ]; then
 		if ! which xz 2> /dev/null > /dev/null; then
-			HAS_FW_LOADER_COMPRESS=""
+			HAS_FW_LOADER_COMPRESS_XZ=""
+		fi
+	fi
+	if [ "$HAS_FW_LOADER_COMPRESS_ZSTD" = "yes" ]; then
+		if ! which zstd 2> /dev/null > /dev/null; then
+			HAS_FW_LOADER_COMPRESS_ZSTD=""
 		fi
 	fi
 }
-- 
2.26.2

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Greg Kroah-Hartman]

On Wed, Jan 27, 2021 at 04:49:35PM +0100, Takashi Iwai wrote:
> Hi,
> 
> it seems that ZSTD format is getting popular, and I've been asked
> about the firmware loader support.  So I took a quick glance, and it
> turned out that it's fairly easy thanks to the existing ZSTD API.
> Now high time to submit something.
> 
> The first patch adds a new Kconfig CONFIG_FW_LOADER_COMPRESS_ZSTD and
> the corresponding decompression function to the firmware loader code.
> For the already supported XZ-compression, CONFIG_FW_LOADER_COMPRESS_XZ
> is added to make it selectable explicitly, too.
> 
> The rest three patches are for selftest: a cleanup, a fix and the
> additional support of ZSTD format.
> 
> Currently, I have no idea whether any distro would use ZSTD files for
> firmware files in near future, though.  That's the reason of this
> patch set being an RFC for now.

Looks sane enough to me, if we have a real user, I see no reason why to
not merge this.

thanks,

greg k-h

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Luis Chamberlain]

On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> Due to the popular demands on ZSTD, here is a patch to add a support
> of ZSTD-compressed firmware files via the direct firmware loader.
> It's just like XZ-compressed file support, providing a decompressor
> with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> the code is even simpler than XZ.
> 
> Signed-off-by: Takashi Iwai <tiwai@suse.de>
> ---
>  drivers/base/firmware_loader/Kconfig | 21 ++++++--
>  drivers/base/firmware_loader/main.c  | 74 ++++++++++++++++++++++++++--
>  2 files changed, 87 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/base/firmware_loader/Kconfig b/drivers/base/firmware_loader/Kconfig
> index 5b24f3959255..f5307978927c 100644
> --- a/drivers/base/firmware_loader/Kconfig
> +++ b/drivers/base/firmware_loader/Kconfig
> @@ -157,17 +157,28 @@ config FW_LOADER_USER_HELPER_FALLBACK
>  
>  config FW_LOADER_COMPRESS
>  	bool "Enable compressed firmware support"
> -	select FW_LOADER_PAGED_BUF
> -	select XZ_DEC
>  	help
>  	  This option enables the support for loading compressed firmware
>  	  files. The caller of firmware API receives the decompressed file
>  	  content. The compressed file is loaded as a fallback, only after
>  	  loading the raw file failed at first.
>  
> -	  Currently only XZ-compressed files are supported, and they have to
> -	  be compressed with either none or crc32 integrity check type (pass
> -	  "-C crc32" option to xz command).
> +if FW_LOADER_COMPRESS
> +config FW_LOADER_COMPRESS_XZ
> +	bool "Enable XZ-compressed firmware support"
> +	select FW_LOADER_PAGED_BUF
> +	select XZ_DEC
> +	help
> +	  This option adds the support for XZ-compressed files.
> +	  The files have to be compressed with either none or crc32
> +	  integrity check type (pass "-C crc32" option to xz command).
> +
> +config FW_LOADER_COMPRESS_ZSTD
> +	bool "Enable ZSTD-compressed firmware support"
> +	select ZSTD_DECOMPRESS
> +	help
> +	  This option adds the support for ZSTD-compressed files.
> +endif # FW_LOADER_COMPRESS
>  
>  config FW_CACHE
>  	bool "Enable firmware caching during suspend"
> diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c
> index 78355095e00d..71332ed4959d 100644
> --- a/drivers/base/firmware_loader/main.c
> +++ b/drivers/base/firmware_loader/main.c
> @@ -34,6 +34,7 @@
>  #include <linux/syscore_ops.h>
>  #include <linux/reboot.h>
>  #include <linux/security.h>
> +#include <linux/zstd.h>
>  #include <linux/xz.h>
>  
>  #include <generated/utsrelease.h>
> @@ -362,10 +363,72 @@ int fw_map_paged_buf(struct fw_priv *fw_priv)
>  }
>  #endif
>  
> +/*
> + * ZSTD-compressed firmware support
> + */
> +#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
> +static int fw_decompress_zstd(struct device *dev, struct fw_priv *fw_priv,
> +			      size_t in_size, const void *in_buffer)
> +{
> +	size_t len, out_size, workspace_size;
> +	void *workspace, *out_buf;
> +	ZSTD_DCtx *ctx;
> +	int err;
> +
> +	if (fw_priv->data) {
> +		out_size = fw_priv->allocated_size;
> +		out_buf = fw_priv->data;
> +	} else {
> +		out_size = ZSTD_findDecompressedSize(in_buffer, in_size);
> +		if (out_size == ZSTD_CONTENTSIZE_UNKNOWN ||
> +		    out_size == ZSTD_CONTENTSIZE_ERROR) {
> +			dev_dbg(dev, "%s: invalid decompression size\n", __func__);
> +			return -EINVAL;
> +		}
> +		out_buf = vzalloc(out_size);
> +		if (!out_buf)
> +			return -ENOMEM;
> +	}
> +
> +	workspace_size = ZSTD_DCtxWorkspaceBound();
> +	workspace = kvzalloc(workspace_size, GFP_KERNEL);
> +	if (!workspace) {
> +		err = -ENOMEM;
> +		goto error;
> +	}
> +
> +	ctx = ZSTD_initDCtx(workspace, workspace_size);
> +	if (!ctx) {
> +		dev_dbg(dev, "%s: failed to initialize context\n", __func__);
> +		err = -EINVAL;
> +		goto error;
> +	}
> +
> +	len = ZSTD_decompressDCtx(ctx, out_buf, out_size, in_buffer, in_size);
> +	if (ZSTD_isError(len)) {
> +		dev_dbg(dev, "%s: failed to decompress: %d\n", __func__,
> +			ZSTD_getErrorCode(len));
> +		err = -EINVAL;
> +		goto error;
> +	}
> +
> +	fw_priv->size = len;
> +	if (!fw_priv->data)
> +		fw_priv->data = out_buf;
> +	err = 0;
> +
> + error:
> +	kvfree(workspace);
> +	if (!fw_priv->data)
> +		vfree(out_buf);
> +	return err;
> +}
> +#endif /* CONFIG_FW_LOADER_COMPRESS_ZSTD */
> +
>  /*
>   * XZ-compressed firmware support
>   */
> -#ifdef CONFIG_FW_LOADER_COMPRESS
> +#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
>  /* show an error and return the standard error code */
>  static int fw_decompress_xz_error(struct device *dev, enum xz_ret xz_ret)
>  {
> @@ -459,7 +522,7 @@ static int fw_decompress_xz(struct device *dev, struct fw_priv *fw_priv,
>  	else
>  		return fw_decompress_xz_pages(dev, fw_priv, in_size, in_buffer);
>  }
> -#endif /* CONFIG_FW_LOADER_COMPRESS */
> +#endif /* CONFIG_FW_LOADER_COMPRESS_XZ */
>  
>  /* direct firmware loading support */
>  static char fw_path_para[256];
> @@ -814,7 +877,12 @@ _request_firmware(const struct firmware **firmware_p, const char *name,
>  	if (!(opt_flags & FW_OPT_PARTIAL))
>  		nondirect = true;
>  
> -#ifdef CONFIG_FW_LOADER_COMPRESS
> +#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
> +	if (ret == -ENOENT && nondirect)
> +		ret = fw_get_filesystem_firmware(device, fw->priv, ".zst",
> +						 fw_decompress_zstd);
> +#endif
> +#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
>  	if (ret == -ENOENT && nondirect)
>  		ret = fw_get_filesystem_firmware(device, fw->priv, ".xz",
>  						 fw_decompress_xz);

Since this is growing, it would be nicer if we avoid the #ifdef mess and
simply have code iterate over supported compressions, and this is
observed here in one line.

  Luis

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Takashi Iwai]

On Wed, 17 Feb 2021 14:16:44 +0100,
Luis Chamberlain wrote:
> 
> On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> > Due to the popular demands on ZSTD, here is a patch to add a support
> > of ZSTD-compressed firmware files via the direct firmware loader.
> > It's just like XZ-compressed file support, providing a decompressor
> > with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> > the code is even simpler than XZ.
> > 
> > Signed-off-by: Takashi Iwai <tiwai@suse.de>
> > ---
> >  drivers/base/firmware_loader/Kconfig | 21 ++++++--
> >  drivers/base/firmware_loader/main.c  | 74 ++++++++++++++++++++++++++--
> >  2 files changed, 87 insertions(+), 8 deletions(-)
> > 
> > diff --git a/drivers/base/firmware_loader/Kconfig b/drivers/base/firmware_loader/Kconfig
> > index 5b24f3959255..f5307978927c 100644
> > --- a/drivers/base/firmware_loader/Kconfig
> > +++ b/drivers/base/firmware_loader/Kconfig
> > @@ -157,17 +157,28 @@ config FW_LOADER_USER_HELPER_FALLBACK
> >  
> >  config FW_LOADER_COMPRESS
> >  	bool "Enable compressed firmware support"
> > -	select FW_LOADER_PAGED_BUF
> > -	select XZ_DEC
> >  	help
> >  	  This option enables the support for loading compressed firmware
> >  	  files. The caller of firmware API receives the decompressed file
> >  	  content. The compressed file is loaded as a fallback, only after
> >  	  loading the raw file failed at first.
> >  
> > -	  Currently only XZ-compressed files are supported, and they have to
> > -	  be compressed with either none or crc32 integrity check type (pass
> > -	  "-C crc32" option to xz command).
> > +if FW_LOADER_COMPRESS
> > +config FW_LOADER_COMPRESS_XZ
> > +	bool "Enable XZ-compressed firmware support"
> > +	select FW_LOADER_PAGED_BUF
> > +	select XZ_DEC
> > +	help
> > +	  This option adds the support for XZ-compressed files.
> > +	  The files have to be compressed with either none or crc32
> > +	  integrity check type (pass "-C crc32" option to xz command).
> > +
> > +config FW_LOADER_COMPRESS_ZSTD
> > +	bool "Enable ZSTD-compressed firmware support"
> > +	select ZSTD_DECOMPRESS
> > +	help
> > +	  This option adds the support for ZSTD-compressed files.
> > +endif # FW_LOADER_COMPRESS
> >  
> >  config FW_CACHE
> >  	bool "Enable firmware caching during suspend"
> > diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c
> > index 78355095e00d..71332ed4959d 100644
> > --- a/drivers/base/firmware_loader/main.c
> > +++ b/drivers/base/firmware_loader/main.c
> > @@ -34,6 +34,7 @@
> >  #include <linux/syscore_ops.h>
> >  #include <linux/reboot.h>
> >  #include <linux/security.h>
> > +#include <linux/zstd.h>
> >  #include <linux/xz.h>
> >  
> >  #include <generated/utsrelease.h>
> > @@ -362,10 +363,72 @@ int fw_map_paged_buf(struct fw_priv *fw_priv)
> >  }
> >  #endif
> >  
> > +/*
> > + * ZSTD-compressed firmware support
> > + */
> > +#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
> > +static int fw_decompress_zstd(struct device *dev, struct fw_priv *fw_priv,
> > +			      size_t in_size, const void *in_buffer)
> > +{
> > +	size_t len, out_size, workspace_size;
> > +	void *workspace, *out_buf;
> > +	ZSTD_DCtx *ctx;
> > +	int err;
> > +
> > +	if (fw_priv->data) {
> > +		out_size = fw_priv->allocated_size;
> > +		out_buf = fw_priv->data;
> > +	} else {
> > +		out_size = ZSTD_findDecompressedSize(in_buffer, in_size);
> > +		if (out_size == ZSTD_CONTENTSIZE_UNKNOWN ||
> > +		    out_size == ZSTD_CONTENTSIZE_ERROR) {
> > +			dev_dbg(dev, "%s: invalid decompression size\n", __func__);
> > +			return -EINVAL;
> > +		}
> > +		out_buf = vzalloc(out_size);
> > +		if (!out_buf)
> > +			return -ENOMEM;
> > +	}
> > +
> > +	workspace_size = ZSTD_DCtxWorkspaceBound();
> > +	workspace = kvzalloc(workspace_size, GFP_KERNEL);
> > +	if (!workspace) {
> > +		err = -ENOMEM;
> > +		goto error;
> > +	}
> > +
> > +	ctx = ZSTD_initDCtx(workspace, workspace_size);
> > +	if (!ctx) {
> > +		dev_dbg(dev, "%s: failed to initialize context\n", __func__);
> > +		err = -EINVAL;
> > +		goto error;
> > +	}
> > +
> > +	len = ZSTD_decompressDCtx(ctx, out_buf, out_size, in_buffer, in_size);
> > +	if (ZSTD_isError(len)) {
> > +		dev_dbg(dev, "%s: failed to decompress: %d\n", __func__,
> > +			ZSTD_getErrorCode(len));
> > +		err = -EINVAL;
> > +		goto error;
> > +	}
> > +
> > +	fw_priv->size = len;
> > +	if (!fw_priv->data)
> > +		fw_priv->data = out_buf;
> > +	err = 0;
> > +
> > + error:
> > +	kvfree(workspace);
> > +	if (!fw_priv->data)
> > +		vfree(out_buf);
> > +	return err;
> > +}
> > +#endif /* CONFIG_FW_LOADER_COMPRESS_ZSTD */
> > +
> >  /*
> >   * XZ-compressed firmware support
> >   */
> > -#ifdef CONFIG_FW_LOADER_COMPRESS
> > +#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
> >  /* show an error and return the standard error code */
> >  static int fw_decompress_xz_error(struct device *dev, enum xz_ret xz_ret)
> >  {
> > @@ -459,7 +522,7 @@ static int fw_decompress_xz(struct device *dev, struct fw_priv *fw_priv,
> >  	else
> >  		return fw_decompress_xz_pages(dev, fw_priv, in_size, in_buffer);
> >  }
> > -#endif /* CONFIG_FW_LOADER_COMPRESS */
> > +#endif /* CONFIG_FW_LOADER_COMPRESS_XZ */
> >  
> >  /* direct firmware loading support */
> >  static char fw_path_para[256];
> > @@ -814,7 +877,12 @@ _request_firmware(const struct firmware **firmware_p, const char *name,
> >  	if (!(opt_flags & FW_OPT_PARTIAL))
> >  		nondirect = true;
> >  
> > -#ifdef CONFIG_FW_LOADER_COMPRESS
> > +#ifdef CONFIG_FW_LOADER_COMPRESS_ZSTD
> > +	if (ret == -ENOENT && nondirect)
> > +		ret = fw_get_filesystem_firmware(device, fw->priv, ".zst",
> > +						 fw_decompress_zstd);
> > +#endif
> > +#ifdef CONFIG_FW_LOADER_COMPRESS_XZ
> >  	if (ret == -ENOENT && nondirect)
> >  		ret = fw_get_filesystem_firmware(device, fw->priv, ".xz",
> >  						 fw_decompress_xz);
> 
> Since this is growing, it would be nicer if we avoid the #ifdef mess and
> simply have code iterate over supported compressions, and this is
> observed here in one line.

If it ever grows further, yeah, but I guess this won't happen so
often :)  As of now, I chose the open-code as there are only two.


thanks,

Takashi

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Luis Chamberlain]

On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> Due to the popular demands on ZSTD, here is a patch to add a support
> of ZSTD-compressed firmware files via the direct firmware loader.
> It's just like XZ-compressed file support, providing a decompressor
> with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> the code is even simpler than XZ.
> 
> Signed-off-by: Takashi Iwai <tiwai@suse.de>

It also occurs to me that having a simple like #define HAVE_FIRMWARE_COMPRESS_ZSTD
on include/linux/firmware.h would enable userspace to be aware (if they
have kernel sources) to determine if the kernels supports this format.

  Luis

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Takashi Iwai]

On Wed, 17 Feb 2021 14:24:19 +0100,
Luis Chamberlain wrote:
> 
> On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> > Due to the popular demands on ZSTD, here is a patch to add a support
> > of ZSTD-compressed firmware files via the direct firmware loader.
> > It's just like XZ-compressed file support, providing a decompressor
> > with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> > the code is even simpler than XZ.
> > 
> > Signed-off-by: Takashi Iwai <tiwai@suse.de>
> 
> It also occurs to me that having a simple like #define HAVE_FIRMWARE_COMPRESS_ZSTD
> on include/linux/firmware.h would enable userspace to be aware (if they
> have kernel sources) to determine if the kernels supports this format.

Extending that idea, we might want to have a sysfs entry showing the
supported formats instead?  This will allow to judge dynamically.


thanks,

Takashi

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Greg Kroah-Hartman]

On Wed, Feb 17, 2021 at 02:34:34PM +0100, Takashi Iwai wrote:
> On Wed, 17 Feb 2021 14:24:19 +0100,
> Luis Chamberlain wrote:
> > 
> > On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> > > Due to the popular demands on ZSTD, here is a patch to add a support
> > > of ZSTD-compressed firmware files via the direct firmware loader.
> > > It's just like XZ-compressed file support, providing a decompressor
> > > with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> > > the code is even simpler than XZ.
> > > 
> > > Signed-off-by: Takashi Iwai <tiwai@suse.de>
> > 
> > It also occurs to me that having a simple like #define HAVE_FIRMWARE_COMPRESS_ZSTD
> > on include/linux/firmware.h would enable userspace to be aware (if they
> > have kernel sources) to determine if the kernels supports this format.
> 
> Extending that idea, we might want to have a sysfs entry showing the
> supported formats instead?  This will allow to judge dynamically.

What could userspace do with that information?

Re: [PATCH RFC 1/4] firmware: Add the support for ZSTD-compressed firmware files

[Takashi Iwai]

On Wed, 17 Feb 2021 15:17:52 +0100,
Greg Kroah-Hartman wrote:
> 
> On Wed, Feb 17, 2021 at 02:34:34PM +0100, Takashi Iwai wrote:
> > On Wed, 17 Feb 2021 14:24:19 +0100,
> > Luis Chamberlain wrote:
> > > 
> > > On Wed, Jan 27, 2021 at 04:49:36PM +0100, Takashi Iwai wrote:
> > > > Due to the popular demands on ZSTD, here is a patch to add a support
> > > > of ZSTD-compressed firmware files via the direct firmware loader.
> > > > It's just like XZ-compressed file support, providing a decompressor
> > > > with ZSTD.  Since ZSTD API can give the decompression size beforehand,
> > > > the code is even simpler than XZ.
> > > > 
> > > > Signed-off-by: Takashi Iwai <tiwai@suse.de>
> > > 
> > > It also occurs to me that having a simple like #define HAVE_FIRMWARE_COMPRESS_ZSTD
> > > on include/linux/firmware.h would enable userspace to be aware (if they
> > > have kernel sources) to determine if the kernels supports this format.
> > 
> > Extending that idea, we might want to have a sysfs entry showing the
> > supported formats instead?  This will allow to judge dynamically.
> 
> What could userspace do with that information?

Decide whether to store the firmware files on the system in the
supported compression format or not.

But admittedly it can be little help, as that's rather information you
want not for the running kernel but for the system to be installed...


Takashi

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Hideki Yamane]

On Wed, 27 Jan 2021 18:09:48 +0100
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > 
> > Currently, I have no idea whether any distro would use ZSTD files for
> > firmware files in near future, though.  That's the reason of this
> > patch set being an RFC for now.
> 
> Looks sane enough to me, if we have a real user, I see no reason why to
> not merge this.

 Just curious, any progress for this?

-- 
Hideki Yamane <henrich@iijmio-mail.jp>

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Greg Kroah-Hartman]

On Fri, Jan 28, 2022 at 03:22:13AM +0900, Hideki Yamane wrote:
> On Wed, 27 Jan 2021 18:09:48 +0100
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > 
> > > Currently, I have no idea whether any distro would use ZSTD files for
> > > firmware files in near future, though.  That's the reason of this
> > > patch set being an RFC for now.
> > 
> > Looks sane enough to me, if we have a real user, I see no reason why to
> > not merge this.
> 
>  Just curious, any progress for this?

What is "this"?  You are responding to a year-old email message :)

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Hideki Yamane]

On Thu, 27 Jan 2022 19:41:18 +0100
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> What is "this"?  You are responding to a year-old email message :)

 Oh, yes, sorry.

 What's the status for adding "ZSTD-compressed file support" for loading
 firmware feature config FW_LOADER_COMPRESS in drivers/base/firmware_loader/Kconfig
 has still XZ_DEC only, where should I check to know its progress?


-- 
Hideki Yamane <henrich@iijmio-mail.jp>

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Greg Kroah-Hartman]

On Fri, Jan 28, 2022 at 09:33:35AM +0900, Hideki Yamane wrote:
> On Thu, 27 Jan 2022 19:41:18 +0100
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > What is "this"?  You are responding to a year-old email message :)
> 
>  Oh, yes, sorry.
> 
>  What's the status for adding "ZSTD-compressed file support" for loading
>  firmware feature config FW_LOADER_COMPRESS in drivers/base/firmware_loader/Kconfig
>  has still XZ_DEC only, where should I check to know its progress?

If you need this feature, take the patches and rebase them to the latest
kernel tree and submit them for inclusion.

thanks,

greg k-h

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Takashi Iwai]

On Fri, 28 Jan 2022 07:54:22 +0100,
Greg Kroah-Hartman wrote:
> 
> On Fri, Jan 28, 2022 at 09:33:35AM +0900, Hideki Yamane wrote:
> > On Thu, 27 Jan 2022 19:41:18 +0100
> > Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > > What is "this"?  You are responding to a year-old email message :)
> > 
> >  Oh, yes, sorry.
> > 
> >  What's the status for adding "ZSTD-compressed file support" for loading
> >  firmware feature config FW_LOADER_COMPRESS in drivers/base/firmware_loader/Kconfig
> >  has still XZ_DEC only, where should I check to know its progress?
> 
> If you need this feature, take the patches and rebase them to the latest
> kernel tree and submit them for inclusion.

Yes, it's a kind of chicken-and-egg problem, and we want that the
patches have actually users before merging to the mainline.

I rebased the patches to 5.17-rc1 with the adaption for the latest
ZSTD API and pushed to topic/fw-loader-zstd-5.17 branch of my
sound.git tree.  It's totally untested, and it'd be appreciated if
anyone can confirm that it's working and used.  Then we can happily
re-submit the patches for the merge.


thanks,

Takashi

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Hideki Yamane]

Hi,

On Fri, 28 Jan 2022 09:13:36 +0100
Takashi Iwai <tiwai@suse.de> wrote:
> It's totally untested, and it'd be appreciated if
> anyone can confirm that it's working and used.  Then we can happily
> re-submit the patches for the merge.

 I've built it with 5.16.4 based one in Debian experimental repository
 and it seems to work (note: build failure with 5.15.15 in Debian unstable).


 How to check:
  1. compress firmware files under /lib/firmware with zstd
  2. boot with distro kernel that is not enable this feature
  3. Wifi that needs firmware doesn't work as expected (*1)
  4. boot with patched kernel
  5. Wifi works :) (*2)


 logs:

(*1)

 2月 02 02:09:43 elitebook830 kernel: Intel(R) Wireless WiFi driver for Linux
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: enabling device (0000 -> 0002)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-36.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-36.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-35.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-35.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-34.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-34.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-33.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-33.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-32.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-32.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-31.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-31.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-30.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-30.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-29.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-29.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-28.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-28.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-27.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-27.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-26.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-26.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-25.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-25.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-24.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-24.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-23.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-23.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-22.ucode (-2)
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-22.ucode failed with error -2
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: minimum version required: iwlwifi-8265-22
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: maximum version supported: iwlwifi-8265-36
 2月 02 02:09:43 elitebook830 kernel: iwlwifi 0000:01:00.0: check git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git


(*2)

 2月 02 02:12:35 elitebook830 kernel: Intel(R) Wireless WiFi driver for Linux
 2月 02 02:12:35 elitebook830 kernel: iwlwifi 0000:01:00.0: enabling device (0000 -> 0002)
 2月 02 02:12:35 elitebook830 kernel: usb 3-1.3.2.1.4.2: FTDI USB Serial Device converter now attached to ttyUSB0
 2月 02 02:12:35 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: failed to load iwlwifi-8265-36.ucode (-2)
 2月 02 02:12:35 elitebook830 kernel: firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware
 2月 02 02:12:35 elitebook830 kernel: iwlwifi 0000:01:00.0: firmware: direct-loading firmware iwlwifi-8265-36.ucode
 2月 02 02:12:35 elitebook830 kernel: iwlwifi 0000:01:00.0: loaded firmware version 36.ca7b901d.0 8265-36.ucode op_mode iwlmvm






-- 
Hideki Yamane <henrich@iijmio-mail.jp>

Re: [PATCH RFC 0/4] firmware: Add ZSTD-compressed file support

[Takashi Iwai]

On Tue, 01 Feb 2022 18:36:18 +0100,
Hideki Yamane wrote:
> 
> Hi,
> 
> On Fri, 28 Jan 2022 09:13:36 +0100
> Takashi Iwai <tiwai@suse.de> wrote:
> > It's totally untested, and it'd be appreciated if
> > anyone can confirm that it's working and used.  Then we can happily
> > re-submit the patches for the merge.
> 
>  I've built it with 5.16.4 based one in Debian experimental repository
>  and it seems to work (note: build failure with 5.15.15 in Debian unstable).
> 
> 
>  How to check:
>   1. compress firmware files under /lib/firmware with zstd
>   2. boot with distro kernel that is not enable this feature
>   3. Wifi that needs firmware doesn't work as expected (*1)
>   4. boot with patched kernel
>   5. Wifi works :) (*2)

Good to know that it's working.  (BTW, you can see more details when
you enable the debug, e.g. "firmware_class.dyndbg=+p" boot option.)

The next step is actually using this in some distro (or privately).
If the feature is confirmed to be needed, I'll resubmit the latest
patches for the merge request.


thanks,

Takashi