Re: PROBLEM: zstd bzImage decompression fails for some x86_32 config on 5.9-rc1

From: Sedat Dilek <sedat.dilek@gmail.com>
To: Feng Tang <feng.tang@intel.com>
Cc: Nick Terrell <terrelln@fb.com>,
	linux-kernel@vger.kernel.org, x86@kernel.org,
	Ingo Molnar <mingo@kernel.org>,
	Masahiro Yamada <masahiroy@kernel.org>,
	linux-kbuild@vger.kernel.org, rong.a.chen@intel.com,
	philip.li@intel.com
Subject: Re: PROBLEM: zstd bzImage decompression fails for some x86_32 config on 5.9-rc1
Date: Mon, 28 Sep 2020 11:53:54 +0200	[thread overview]
Message-ID: <CA+icZUWvVB_D2USqUivohvywjtxe5jfW1phcA84Rfz9Cwy9jfQ@mail.gmail.com> (raw)
In-Reply-To: <20200928085505.GA22244@shbuild999.sh.intel.com>

On Mon, Sep 28, 2020 at 10:55 AM Feng Tang <feng.tang@intel.com> wrote:
>
> Hi Nick,
>
> 0day has found some kernel decomprssion failure case since 5.9-rc1 (X86_32
> build), and it could be related with ZSTD code, though initially we bisected
> to some other commits.
>
> The error messages are:
>
>         early console in setup code
>         Wrong EFI loader signature.
>         early console in extract_kernel
>         input_data: 0x046f50b4
>         input_len: 0x01ebbeb6
>         output: 0x01000000
>         output_len: 0x04fc535c
>         kernel_total_size: 0x055f5000
>         needed_size: 0x055f5000
>
>         Decompressing Linux...
>
>         ZSTD-compressed data is corrupt
>
> This could be reproduced by compiling the kernel with attached config,
> and use QEMU to boot it.
>
> We suspect it could be related with the kernel size, as we only see
> it on big kernel, and some more info are:
>
> * If we remove a lot of kernel config to build a much smaller kernel,
>   it will boot fine
>
> * If we change the zstd algorithm from zstd22 to zstd19, the kernel will
>   boot fine with below patch
>

Hi,

Recently, Debian has updated initramfs-tools/initramfs-tools-core packages
with ZSTD support:

[ /usr/sbin/mkinitramfs ]

case "${compress}" in
gzip)   # If we're doing a reproducible build, use gzip -n
       if [ -n "${SOURCE_DATE_EPOCH}" ]; then
               compress="gzip -n"
       # Otherwise, substitute pigz if it's available
       elif command -v pigz >/dev/null; then
               compress=pigz
       fi
       ;;
lz4)    compress="lz4 -9 -l" ;;
zstd)   compress="zstd -q -19 -T0" ;;
xz)     compress="xz --check=crc32"
       # If we're not doing a reproducible build, enable multithreading
       test -z "${SOURCE_DATE_EPOCH}" && compress="$compress --threads=0"
       ;;
bzip2|lzma|lzop)
       # no parameters needed
       ;;
*)      echo "W: Unknown compression command ${compress}" >&2 ;;
esac

As you can see it uses compression-level 19 for ZSTD.
Maybe it is good to change from 22 to 19 in the Linux-kernel sources?

Thanks.

Regards,
- Sedat -

>         diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
>         index 3962f59..8fe71ba 100644
>         --- a/arch/x86/boot/compressed/Makefile
>         +++ b/arch/x86/boot/compressed/Makefile
>         @@ -147,7 +147,7 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE
>          $(obj)/vmlinux.bin.zst: $(vmlinux.bin.all-y) FORCE
>         -       $(call if_changed,zstd22)
>         +       $(call if_changed,zstd)
>
>
> Please let me know if you need more info, and sorry for the late report
> as we just tracked down to this point.
>
> Thanks,
> Feng
>
>
>