All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] kbuild: buildtar: add comments about inconsistent package generation
@ 2024-04-14 17:41 ` Masahiro Yamada
  0 siblings, 0 replies; 8+ messages in thread
From: Masahiro Yamada @ 2024-04-14 17:41 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Albert Ou, Nathan Chancellor, Nicolas Schier,
	Palmer Dabbelt, Paul Walmsley, linux-kernel, linux-riscv

scripts/package/buildtar checks some kernel packages, and copies the
first image found. This may potentially produce an inconsistent (and
possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
and vmlinuz.efi, then copies the first image found, which might be a
stale image.

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
expected behavior.

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
which will remain in the build directory unless you clean it. Even if
CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
precedence over vmlinuz.efi.

In summary, the code "[ -f ... ] && cp" does not consistently produce
the desired outcome.

The other package scripts are deterministic; scripts/package/mkdebian,
for example, chooses a copied kernel image based on CONFIG options.

I removed [ -f ... ] checks from x86, alpha, parisc, and the default
because they have a single kernel image to copy. If it is missing, it
should be an error.

I did not modify the code for mips, arm64, riscv. Instead, I left some
comments. Eventually, someone may fix the code, or at the very least,
it may discourage the copy-pasting of incorrect code.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/package/buildtar | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/scripts/package/buildtar b/scripts/package/buildtar
index 72c91a1b832f..ed8d9b496305 100755
--- a/scripts/package/buildtar
+++ b/scripts/package/buildtar
@@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 #
 # Install arch-specific kernel image(s)
 #
+# Note:
+#   mips, arm64, and riscv copy the first image found. This may not produce
+#   the desired outcome because it may pick up a stale file remaining in the
+#   build tree.
+#
 case "${ARCH}" in
 	x86|i386|x86_64)
-		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	alpha)
-		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	parisc*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
 		;;
 	mips)
+		# Please note the following code may copy a stale file.
 		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
 			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
@@ -86,6 +92,7 @@ case "${ARCH}" in
 		fi
 		;;
 	arm64)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
 			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
@@ -94,6 +101,7 @@ case "${ARCH}" in
 		done
 		;;
 	riscv)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image; do
 			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
@@ -102,7 +110,7 @@ case "${ARCH}" in
 		done
 		;;
 	*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
 		echo "" >&2
 		echo '** ** **  WARNING  ** ** **' >&2
 		echo "" >&2
-- 
2.40.1


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH] kbuild: buildtar: add comments about inconsistent package generation
@ 2024-04-14 17:41 ` Masahiro Yamada
  0 siblings, 0 replies; 8+ messages in thread
From: Masahiro Yamada @ 2024-04-14 17:41 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Albert Ou, Nathan Chancellor, Nicolas Schier,
	Palmer Dabbelt, Paul Walmsley, linux-kernel, linux-riscv

scripts/package/buildtar checks some kernel packages, and copies the
first image found. This may potentially produce an inconsistent (and
possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
and vmlinuz.efi, then copies the first image found, which might be a
stale image.

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
expected behavior.

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
which will remain in the build directory unless you clean it. Even if
CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
precedence over vmlinuz.efi.

In summary, the code "[ -f ... ] && cp" does not consistently produce
the desired outcome.

The other package scripts are deterministic; scripts/package/mkdebian,
for example, chooses a copied kernel image based on CONFIG options.

I removed [ -f ... ] checks from x86, alpha, parisc, and the default
because they have a single kernel image to copy. If it is missing, it
should be an error.

I did not modify the code for mips, arm64, riscv. Instead, I left some
comments. Eventually, someone may fix the code, or at the very least,
it may discourage the copy-pasting of incorrect code.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/package/buildtar | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/scripts/package/buildtar b/scripts/package/buildtar
index 72c91a1b832f..ed8d9b496305 100755
--- a/scripts/package/buildtar
+++ b/scripts/package/buildtar
@@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 #
 # Install arch-specific kernel image(s)
 #
+# Note:
+#   mips, arm64, and riscv copy the first image found. This may not produce
+#   the desired outcome because it may pick up a stale file remaining in the
+#   build tree.
+#
 case "${ARCH}" in
 	x86|i386|x86_64)
-		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	alpha)
-		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	parisc*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
 		;;
 	mips)
+		# Please note the following code may copy a stale file.
 		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
 			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
@@ -86,6 +92,7 @@ case "${ARCH}" in
 		fi
 		;;
 	arm64)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
 			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
@@ -94,6 +101,7 @@ case "${ARCH}" in
 		done
 		;;
 	riscv)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image; do
 			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
@@ -102,7 +110,7 @@ case "${ARCH}" in
 		done
 		;;
 	*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
 		echo "" >&2
 		echo '** ** **  WARNING  ** ** **' >&2
 		echo "" >&2
-- 
2.40.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
  2024-04-14 17:41 ` Masahiro Yamada
@ 2024-04-15  6:59   ` Emil Renner Berthing
  -1 siblings, 0 replies; 8+ messages in thread
From: Emil Renner Berthing @ 2024-04-15  6:59 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Albert Ou, Nathan Chancellor, Nicolas Schier, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv

Masahiro Yamada wrote:
> scripts/package/buildtar checks some kernel packages, and copies the
> first image found. This may potentially produce an inconsistent (and
> possibly wrong) package.
>
> For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> and vmlinuz.efi, then copies the first image found, which might be a
> stale image.
>
> When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> expected behavior.
>
> If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
> which will remain in the build directory unless you clean it. Even if
> CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
> stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
> precedence over vmlinuz.efi.
>
> In summary, the code "[ -f ... ] && cp" does not consistently produce
> the desired outcome.
>
> The other package scripts are deterministic; scripts/package/mkdebian,
> for example, chooses a copied kernel image based on CONFIG options.
>
> I removed [ -f ... ] checks from x86, alpha, parisc, and the default
> because they have a single kernel image to copy. If it is missing, it
> should be an error.
>
> I did not modify the code for mips, arm64, riscv. Instead, I left some
> comments. Eventually, someone may fix the code, or at the very least,
> it may discourage the copy-pasting of incorrect code.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Hi Masahiro,

Thanks for the patch. I wonder why arm64 and riscv is not just using
KBUILD_IMAGE as the source path of the image to copy like some other
architectures do. In any case:

Reviewed-by: Emil Renner Berthing <emil.renner.berthing@canonical.com>

> ---
>
>  scripts/package/buildtar | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/package/buildtar b/scripts/package/buildtar
> index 72c91a1b832f..ed8d9b496305 100755
> --- a/scripts/package/buildtar
> +++ b/scripts/package/buildtar
> @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  #
>  # Install arch-specific kernel image(s)
>  #
> +# Note:
> +#   mips, arm64, and riscv copy the first image found. This may not produce
> +#   the desired outcome because it may pick up a stale file remaining in the
> +#   build tree.
> +#
>  case "${ARCH}" in
>  	x86|i386|x86_64)
> -		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		;;
>  	alpha)
> -		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		;;
>  	parisc*)
> -		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> +		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
>  		;;
>  	mips)
> +		# Please note the following code may copy a stale file.
>  		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
>  			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
> @@ -86,6 +92,7 @@ case "${ARCH}" in
>  		fi
>  		;;
>  	arm64)
> +		# Please note the following code may copy a stale file.
>  		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
>  			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
>  				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> @@ -94,6 +101,7 @@ case "${ARCH}" in
>  		done
>  		;;
>  	riscv)
> +		# Please note the following code may copy a stale file.
>  		for i in Image.bz2 Image.gz Image; do
>  			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
>  				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> @@ -102,7 +110,7 @@ case "${ARCH}" in
>  		done
>  		;;
>  	*)
> -		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
> +		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
>  		echo "" >&2
>  		echo '** ** **  WARNING  ** ** **' >&2
>  		echo "" >&2
> --
> 2.40.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
@ 2024-04-15  6:59   ` Emil Renner Berthing
  0 siblings, 0 replies; 8+ messages in thread
From: Emil Renner Berthing @ 2024-04-15  6:59 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Albert Ou, Nathan Chancellor, Nicolas Schier, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv

Masahiro Yamada wrote:
> scripts/package/buildtar checks some kernel packages, and copies the
> first image found. This may potentially produce an inconsistent (and
> possibly wrong) package.
>
> For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> and vmlinuz.efi, then copies the first image found, which might be a
> stale image.
>
> When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> expected behavior.
>
> If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
> which will remain in the build directory unless you clean it. Even if
> CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
> stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
> precedence over vmlinuz.efi.
>
> In summary, the code "[ -f ... ] && cp" does not consistently produce
> the desired outcome.
>
> The other package scripts are deterministic; scripts/package/mkdebian,
> for example, chooses a copied kernel image based on CONFIG options.
>
> I removed [ -f ... ] checks from x86, alpha, parisc, and the default
> because they have a single kernel image to copy. If it is missing, it
> should be an error.
>
> I did not modify the code for mips, arm64, riscv. Instead, I left some
> comments. Eventually, someone may fix the code, or at the very least,
> it may discourage the copy-pasting of incorrect code.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Hi Masahiro,

Thanks for the patch. I wonder why arm64 and riscv is not just using
KBUILD_IMAGE as the source path of the image to copy like some other
architectures do. In any case:

Reviewed-by: Emil Renner Berthing <emil.renner.berthing@canonical.com>

> ---
>
>  scripts/package/buildtar | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/package/buildtar b/scripts/package/buildtar
> index 72c91a1b832f..ed8d9b496305 100755
> --- a/scripts/package/buildtar
> +++ b/scripts/package/buildtar
> @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  #
>  # Install arch-specific kernel image(s)
>  #
> +# Note:
> +#   mips, arm64, and riscv copy the first image found. This may not produce
> +#   the desired outcome because it may pick up a stale file remaining in the
> +#   build tree.
> +#
>  case "${ARCH}" in
>  	x86|i386|x86_64)
> -		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		;;
>  	alpha)
> -		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		;;
>  	parisc*)
> -		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> +		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
>  		;;
>  	mips)
> +		# Please note the following code may copy a stale file.
>  		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
>  			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>  		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
> @@ -86,6 +92,7 @@ case "${ARCH}" in
>  		fi
>  		;;
>  	arm64)
> +		# Please note the following code may copy a stale file.
>  		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
>  			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
>  				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> @@ -94,6 +101,7 @@ case "${ARCH}" in
>  		done
>  		;;
>  	riscv)
> +		# Please note the following code may copy a stale file.
>  		for i in Image.bz2 Image.gz Image; do
>  			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
>  				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> @@ -102,7 +110,7 @@ case "${ARCH}" in
>  		done
>  		;;
>  	*)
> -		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
> +		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
>  		echo "" >&2
>  		echo '** ** **  WARNING  ** ** **' >&2
>  		echo "" >&2
> --
> 2.40.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
  2024-04-14 17:41 ` Masahiro Yamada
@ 2024-04-16  2:07   ` Masahiro Yamada
  -1 siblings, 0 replies; 8+ messages in thread
From: Masahiro Yamada @ 2024-04-16  2:07 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Albert Ou, Nathan Chancellor, Nicolas Schier, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv

On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> scripts/package/buildtar checks some kernel packages, and copies the
> first image found. This may potentially produce an inconsistent (and
> possibly wrong) package.
>
> For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> and vmlinuz.efi, then copies the first image found, which might be a
> stale image.
>
> When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> expected behavior.
>
> If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,


The correct sentence is

If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...







> which will remain in the build directory unless you clean it. Even if
> CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
> stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
> precedence over vmlinuz.efi.
>
> In summary, the code "[ -f ... ] && cp" does not consistently produce
> the desired outcome.
>
> The other package scripts are deterministic; scripts/package/mkdebian,
> for example, chooses a copied kernel image based on CONFIG options.
>
> I removed [ -f ... ] checks from x86, alpha, parisc, and the default
> because they have a single kernel image to copy. If it is missing, it
> should be an error.
>
> I did not modify the code for mips, arm64, riscv. Instead, I left some
> comments. Eventually, someone may fix the code, or at the very least,
> it may discourage the copy-pasting of incorrect code.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
>  scripts/package/buildtar | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/package/buildtar b/scripts/package/buildtar
> index 72c91a1b832f..ed8d9b496305 100755
> --- a/scripts/package/buildtar
> +++ b/scripts/package/buildtar
> @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  #
>  # Install arch-specific kernel image(s)
>  #
> +# Note:
> +#   mips, arm64, and riscv copy the first image found. This may not produce
> +#   the desired outcome because it may pick up a stale file remaining in the
> +#   build tree.
> +#
>  case "${ARCH}" in
>         x86|i386|x86_64)
> -               [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +               cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 ;;
>         alpha)
> -               [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +               cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 ;;
>         parisc*)
> -               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> +               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>                 [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
>                 ;;
>         mips)
> +               # Please note the following code may copy a stale file.
>                 if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
>                         cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
> @@ -86,6 +92,7 @@ case "${ARCH}" in
>                 fi
>                 ;;
>         arm64)
> +               # Please note the following code may copy a stale file.
>                 for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
>                         if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
>                                 cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> @@ -94,6 +101,7 @@ case "${ARCH}" in
>                 done
>                 ;;
>         riscv)
> +               # Please note the following code may copy a stale file.
>                 for i in Image.bz2 Image.gz Image; do
>                         if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
>                                 cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> @@ -102,7 +110,7 @@ case "${ARCH}" in
>                 done
>                 ;;
>         *)
> -               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
> +               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
>                 echo "" >&2
>                 echo '** ** **  WARNING  ** ** **' >&2
>                 echo "" >&2
> --
> 2.40.1
>


--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
@ 2024-04-16  2:07   ` Masahiro Yamada
  0 siblings, 0 replies; 8+ messages in thread
From: Masahiro Yamada @ 2024-04-16  2:07 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Albert Ou, Nathan Chancellor, Nicolas Schier, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv

On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> scripts/package/buildtar checks some kernel packages, and copies the
> first image found. This may potentially produce an inconsistent (and
> possibly wrong) package.
>
> For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> and vmlinuz.efi, then copies the first image found, which might be a
> stale image.
>
> When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> expected behavior.
>
> If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,


The correct sentence is

If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...







> which will remain in the build directory unless you clean it. Even if
> CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy
> stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes
> precedence over vmlinuz.efi.
>
> In summary, the code "[ -f ... ] && cp" does not consistently produce
> the desired outcome.
>
> The other package scripts are deterministic; scripts/package/mkdebian,
> for example, chooses a copied kernel image based on CONFIG options.
>
> I removed [ -f ... ] checks from x86, alpha, parisc, and the default
> because they have a single kernel image to copy. If it is missing, it
> should be an error.
>
> I did not modify the code for mips, arm64, riscv. Instead, I left some
> comments. Eventually, someone may fix the code, or at the very least,
> it may discourage the copy-pasting of incorrect code.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
>  scripts/package/buildtar | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/package/buildtar b/scripts/package/buildtar
> index 72c91a1b832f..ed8d9b496305 100755
> --- a/scripts/package/buildtar
> +++ b/scripts/package/buildtar
> @@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>  #
>  # Install arch-specific kernel image(s)
>  #
> +# Note:
> +#   mips, arm64, and riscv copy the first image found. This may not produce
> +#   the desired outcome because it may pick up a stale file remaining in the
> +#   build tree.
> +#
>  case "${ARCH}" in
>         x86|i386|x86_64)
> -               [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +               cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 ;;
>         alpha)
> -               [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> +               cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 ;;
>         parisc*)
> -               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> +               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
>                 [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
>                 ;;
>         mips)
> +               # Please note the following code may copy a stale file.
>                 if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
>                         cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
>                 elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
> @@ -86,6 +92,7 @@ case "${ARCH}" in
>                 fi
>                 ;;
>         arm64)
> +               # Please note the following code may copy a stale file.
>                 for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
>                         if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
>                                 cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
> @@ -94,6 +101,7 @@ case "${ARCH}" in
>                 done
>                 ;;
>         riscv)
> +               # Please note the following code may copy a stale file.
>                 for i in Image.bz2 Image.gz Image; do
>                         if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
>                                 cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
> @@ -102,7 +110,7 @@ case "${ARCH}" in
>                 done
>                 ;;
>         *)
> -               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
> +               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
>                 echo "" >&2
>                 echo '** ** **  WARNING  ** ** **' >&2
>                 echo "" >&2
> --
> 2.40.1
>


--
Best Regards
Masahiro Yamada

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
  2024-04-16  2:07   ` Masahiro Yamada
@ 2024-04-17 19:37     ` Nicolas Schier
  -1 siblings, 0 replies; 8+ messages in thread
From: Nicolas Schier @ 2024-04-17 19:37 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: linux-kbuild, Albert Ou, Nathan Chancellor, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv


[-- Attachment #1.1: Type: text/plain, Size: 1003 bytes --]

On Tue 16 Apr 2024 11:07:48 GMT, Masahiro Yamada wrote:
> On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada 
> <masahiroy@kernel.org> wrote:
> >
> > scripts/package/buildtar checks some kernel packages, and copies the
> > first image found. This may potentially produce an inconsistent (and
> > possibly wrong) package.
> >
> > For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> > and vmlinuz.efi, then copies the first image found, which might be a
> > stale image.
> >
> > When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> > 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> > expected behavior.
> >
> > If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
> 
> 
> The correct sentence is
> 
> If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...

Thanks, I appreciate the detailed descriptions and comments. 

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

Kind regards,
Nicolas


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 161 bytes --]

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] kbuild: buildtar: add comments about inconsistent package generation
@ 2024-04-17 19:37     ` Nicolas Schier
  0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Schier @ 2024-04-17 19:37 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: linux-kbuild, Albert Ou, Nathan Chancellor, Palmer Dabbelt,
	Paul Walmsley, linux-kernel, linux-riscv

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

On Tue 16 Apr 2024 11:07:48 GMT, Masahiro Yamada wrote:
> On Mon, Apr 15, 2024 at 2:42 AM Masahiro Yamada 
> <masahiroy@kernel.org> wrote:
> >
> > scripts/package/buildtar checks some kernel packages, and copies the
> > first image found. This may potentially produce an inconsistent (and
> > possibly wrong) package.
> >
> > For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo},
> > and vmlinuz.efi, then copies the first image found, which might be a
> > stale image.
> >
> > When CONFIG_EFI_ZBOOT is enabled in the pristine source tree,
> > 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the
> > expected behavior.
> >
> > If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,
> 
> 
> The correct sentence is
> 
> If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...

Thanks, I appreciate the detailed descriptions and comments. 

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

Kind regards,
Nicolas


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2024-04-17 20:01 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-14 17:41 [PATCH] kbuild: buildtar: add comments about inconsistent package generation Masahiro Yamada
2024-04-14 17:41 ` Masahiro Yamada
2024-04-15  6:59 ` Emil Renner Berthing
2024-04-15  6:59   ` Emil Renner Berthing
2024-04-16  2:07 ` Masahiro Yamada
2024-04-16  2:07   ` Masahiro Yamada
2024-04-17 19:37   ` Nicolas Schier
2024-04-17 19:37     ` Nicolas Schier

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.