[PATCH v2 0/4] x86, kbuild: Some improvement for fdimage/isoimage generation

[PATCH v2 0/4] x86, kbuild: Some improvement for fdimage/isoimage generation

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

This serias is a expansion of below original patch after discussion with Ingo
and Yamada.
[PATCH] x86, build: Improve the isolinux searching of isoimage generation

v2: some improvement for commit messages.

Changbin Du (4):
  x86, build: Factor out fdimage/isoimage generation commands to
    standalone script
  x86, build: Add new paths for isolinux.bin and ldlinux.c32
  x86, build: Specify -input-charset=utf-8 for mkisofs
  x86, boot: Add some generated files to the .gitignore file

 arch/x86/boot/.gitignore  |   3 ++
 arch/x86/boot/Makefile    |  59 +++++------------------
 arch/x86/boot/genimage.sh | 119 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 133 insertions(+), 48 deletions(-)
 create mode 100644 arch/x86/boot/genimage.sh

-- 
2.7.4

[PATCH v2 1/4] x86, build: Factor out fdimage/isoimage generation commands to standalone script

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

The build message for fdimage/isoimage are pretty unstructured. The raw
shell command blocks are printed. We can improve them as regular build
system messages. Besides, writing commands in a shell script is much more
easy than in a Makefile.

See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.

This patch factors out the commands used for fdimage/isoimage generation
from arch/x86/boot/Makefile to a new script arch/x86/boot/genimage.sh.
Then it adds the new kbuild command 'genimage' which invokes the new script.
All fdimages/isoimage files are now generated by a call to 'genimage' with
different parameters.

Now 'make isoimage' becomes:
...
Kernel: arch/x86/boot/bzImage is ready  (#30)
  GENIMAGE arch/x86/boot/image.iso
Size of boot image is 4 sectors -> No emulation
 15.37% done, estimate finish Sun Nov  5 23:36:57 2017
 30.68% done, estimate finish Sun Nov  5 23:36:57 2017
 46.04% done, estimate finish Sun Nov  5 23:36:57 2017
 61.35% done, estimate finish Sun Nov  5 23:36:57 2017
 76.69% done, estimate finish Sun Nov  5 23:36:57 2017
 92.00% done, estimate finish Sun Nov  5 23:36:57 2017
Total translation table size: 2048
Total rockridge attributes bytes: 659
Total directory bytes: 0
Path table size(bytes): 10
Max brk space used 0
32608 extents written (63 MB)
Kernel: arch/x86/boot/image.iso is ready

Before:
Kernel: arch/x86/boot/bzImage is ready  (#63)
rm -rf arch/x86/boot/isoimage
mkdir arch/x86/boot/isoimage
for i in lib lib64 share end ; do \
	if [ -f /usr/$i/syslinux/isolinux.bin ] ; then \
		cp /usr/$i/syslinux/isolinux.bin arch/x86/boot/isoimage ; \
		if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then \
			cp /usr/$i/syslinux/ldlinux.c32 arch/x86/boot/isoimage ; \
		fi ; \
		break ; \
	fi ; \
	if [ $i = end ] ; then exit 1 ; fi ; \
done
...

Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>

---
v2: Fix grammar errors in commit message. (Ingo)
---
 arch/x86/boot/Makefile    |  59 +++++---------------------
 arch/x86/boot/genimage.sh | 105 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 116 insertions(+), 48 deletions(-)
 create mode 100644 arch/x86/boot/genimage.sh

diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index d88a2fd..9b5adae 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -123,63 +123,26 @@ image_cmdline = default linux $(FDARGS) $(if $(FDINITRD),initrd=initrd.img,)
 $(obj)/mtools.conf: $(src)/mtools.conf.in
 	sed -e 's|@OBJ@|$(obj)|g' < $< > $@
 
+quiet_cmd_genimage = GENIMAGE $3
+cmd_genimage = sh $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
+			$(obj)/mtools.conf '$(image_cmdline)' $(FDINITRD)
+
 # This requires write access to /dev/fd0
 bzdisk: $(obj)/bzImage $(obj)/mtools.conf
-	MTOOLSRC=$(obj)/mtools.conf mformat a:			; sync
-	syslinux /dev/fd0					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(src)/mtools.conf mcopy - a:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' a:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage a:linux	; sync
+	$(call cmd,genimage,bzdisk,/dev/fd0)
 
 # These require being root or having syslinux 2.02 or higher installed
 fdimage fdimage144: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=1440
-	MTOOLSRC=$(obj)/mtools.conf mformat v:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - v:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' v:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage v:linux	; sync
+	$(call cmd,genimage,fdimage144,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 fdimage288: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=2880
-	MTOOLSRC=$(obj)/mtools.conf mformat w:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - w:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' w:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage w:linux	; sync
+	$(call cmd,genimage,fdimage288,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 isoimage: $(obj)/bzImage
-	-rm -rf $(obj)/isoimage
-	mkdir $(obj)/isoimage
-	for i in lib lib64 share end ; do \
-		if [ -f /usr/$$i/syslinux/isolinux.bin ] ; then \
-			cp /usr/$$i/syslinux/isolinux.bin $(obj)/isoimage ; \
-			if [ -f /usr/$$i/syslinux/ldlinux.c32 ]; then \
-				cp /usr/$$i/syslinux/ldlinux.c32 $(obj)/isoimage ; \
-			fi ; \
-			break ; \
-		fi ; \
-		if [ $$i = end ] ; then exit 1 ; fi ; \
-	done
-	cp $(obj)/bzImage $(obj)/isoimage/linux
-	echo '$(image_cmdline)' > $(obj)/isoimage/isolinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		cp '$(FDINITRD)' $(obj)/isoimage/initrd.img ; \
-	fi
-	mkisofs -J -r -o $(obj)/image.iso -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table \
-		$(obj)/isoimage
-	isohybrid $(obj)/image.iso 2>/dev/null || true
-	rm -rf $(obj)/isoimage
+	$(call cmd,genimage,isoimage,$(obj)/image.iso)
+	@$(kecho) 'Kernel: $(obj)/image.iso is ready'
 
 bzlilo: $(obj)/bzImage
 	if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz $(INSTALL_PATH)/vmlinuz.old; fi
diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
new file mode 100644
index 0000000..75a9de1
--- /dev/null
+++ b/arch/x86/boot/genimage.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 2017 by Changbin Du <changbin.du@intel.com>
+#
+# Adapted from code in arch/x86/boot/Makefile by H. Peter Anvin and others
+#
+# "make fdimage/fdimage144/fdimage288/isoimage" script for x86 architecture
+#
+# Arguments:
+#   $1 - fdimage format
+#   $2 - target image file
+#   $3 - kernel bzImage file
+#   $4 - mtool configuration file
+#   $5 - kernel cmdline
+#   $6 - inird image file
+#
+
+verify () {
+	if [ ! -f "$1" ]; then
+		echo ""                                                   1>&2
+		echo " *** Missing file: $1"                              1>&2
+		echo ""                                                   1>&2
+		exit 1
+	fi
+}
+
+
+export MTOOLSRC=$4
+FIMAGE=$2
+FBZIMAGE=$3
+KCMDLINE=$5
+FDINITRD=$6
+
+# Make sure the files actually exist
+verify "$FBZIMAGE"
+verify "$MTOOLSRC"
+
+genbzdisk() {
+	mformat a:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - a:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" a:initrd.img
+	fi
+	mcopy $FBZIMAGE a:linux
+}
+
+genfdimage144() {
+	dd if=/dev/zero of=$FIMAGE bs=1024 count=1440
+	mformat v:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - v:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" v:initrd.img
+	fi
+	mcopy $FBZIMAGE v:linux
+}
+
+genfdimage288() {
+	dd if=/dev/zero of=$FIMAGE bs=1024 count=2880
+	mformat w:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - W:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" w:initrd.img
+	fi
+	mcopy $FBZIMAGE w:linux
+}
+
+genisoimage() {
+	tmp_dir=`dirname $FIMAGE`/isoimage
+	rm -rf $tmp_dir
+	mkdir $tmp_dir
+	for i in lib lib64 share end ; do
+		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then
+			cp /usr/$i/syslinux/isolinux.bin $tmp_dir
+			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then
+				cp /usr/$i/syslinux/ldlinux.c32 $tmp_dir
+			fi
+			break
+		fi
+		if [ $i = end ] ; then exit 1 ; fi ;
+	done
+	cp $FBZIMAGE $tmp_dir/linux
+	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		cp "$FDINITRD" $tmp_dir/initrd.img
+	fi
+	mkisofs -J -r -o $FIMAGE -b isolinux.bin -c boot.cat \
+		-no-emul-boot -boot-load-size 4 -boot-info-table $tmp_dir
+	isohybrid $FIMAGE 2>/dev/null || true
+	rm -rf $tmp_dir
+}
+
+case $1 in
+	bzdisk)     genbzdisk;;
+	fdimage144) genfdimage144;;
+	fdimage288) genfdimage288;;
+	isoimage)   genisoimage;;
+	*)          echo 'Unknown image format'; exit 1;
+esac
-- 
2.7.4

[PATCH v2 2/4] x86, build: Add new paths for isolinux.bin and ldlinux.c32

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

Recently I failed to build isoimage target, because the path of isolinux.bin
changed to /usr/xxx/ISOLINUX/isolinux.bin, as well as ldlinux.c32 which
changed to /usr/xxx/syslinux/modules/bios/ldlinux.c32.

This patch improves the file search logic:
  - Show a error message instead of silent fail.
  - Add above new paths.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Ingo Molnar <mingo@kernel.org>

---
v2: Improve description. (Ingo)
---
 arch/x86/boot/genimage.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
index 75a9de1..6c8100b 100644
--- a/arch/x86/boot/genimage.sh
+++ b/arch/x86/boot/genimage.sh
@@ -76,14 +76,27 @@ genisoimage() {
 	rm -rf $tmp_dir
 	mkdir $tmp_dir
 	for i in lib lib64 share end ; do
-		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then
-			cp /usr/$i/syslinux/isolinux.bin $tmp_dir
-			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then
-				cp /usr/$i/syslinux/ldlinux.c32 $tmp_dir
+		for j in syslinux ISOLINUX ; do
+			if [ -f /usr/$i/$j/isolinux.bin ] ; then
+				isolinux=/usr/$i/$j/isolinux.bin
+				echo "Using $isolinux"
+				cp $isolinux $tmp_dir
 			fi
+		done
+		for j in syslinux syslinux/modules/bios ; do
+			if [ -f /usr/$i/$j/ldlinux.c32 ]; then
+				ldlinux=/usr/$i/$j/ldlinux.c32
+				echo "Using $ldlinux"
+				cp $ldlinux $tmp_dir
+			fi
+		done
+		if [ -n "$isolinux" -a -n "$ldlinux" ] ; then
 			break
 		fi
-		if [ $i = end ] ; then exit 1 ; fi ;
+		if [ $i = end -a -z "$isolinux" ] ; then
+			echo 'Need an isolinux.bin file, please install syslinux/isolinux.'
+			exit 1
+		fi
 	done
 	cp $FBZIMAGE $tmp_dir/linux
 	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg
-- 
2.7.4

[PATCH v2 3/4] x86, build: Specify -input-charset=utf-8 for mkisofs

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

It avoids the following warning triggered by newer versions of mkisofs::
I: -input-charset not specified, using utf-8 (detected in locale settings)

Signed-off-by: Changbin Du <changbin.du@intel.com>
---
 arch/x86/boot/genimage.sh | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
index 6c8100b..628e936 100644
--- a/arch/x86/boot/genimage.sh
+++ b/arch/x86/boot/genimage.sh
@@ -103,8 +103,9 @@ genisoimage() {
 	if [ -f "$FDINITRD" ] ; then
 		cp "$FDINITRD" $tmp_dir/initrd.img
 	fi
-	mkisofs -J -r -o $FIMAGE -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table $tmp_dir
+	mkisofs -J -r -input-charset=utf-8 -o $FIMAGE -b isolinux.bin \
+		-c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table \
+		$tmp_dir
 	isohybrid $FIMAGE 2>/dev/null || true
 	rm -rf $tmp_dir
 }
-- 
2.7.4

[PATCH v2 4/4] x86, boot: Add some generated files to the .gitignore file

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

Ignore generated files fdimage,mtools.conf and image.iso.

Signed-off-by: Changbin Du <changbin.du@intel.com>
---
 arch/x86/boot/.gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/boot/.gitignore b/arch/x86/boot/.gitignore
index e3cf9f6..09d25dd 100644
--- a/arch/x86/boot/.gitignore
+++ b/arch/x86/boot/.gitignore
@@ -7,3 +7,6 @@ zoffset.h
 setup
 setup.bin
 setup.elf
+fdimage
+mtools.conf
+image.iso
-- 
2.7.4

Re: [PATCH v2 1/4] x86, build: Factor out fdimage/isoimage generation commands to standalone script

[Ingo Molnar]

* changbin.du@intel.com <changbin.du@intel.com> wrote:

> From: Changbin Du <changbin.du@intel.com>
> 
> The build message for fdimage/isoimage are pretty unstructured. The raw
> shell command blocks are printed. We can improve them as regular build
> system messages. Besides, writing commands in a shell script is much more
> easy than in a Makefile.
> 
> See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.
> 
> This patch factors out the commands used for fdimage/isoimage generation
> from arch/x86/boot/Makefile to a new script arch/x86/boot/genimage.sh.
> Then it adds the new kbuild command 'genimage' which invokes the new script.
> All fdimages/isoimage files are now generated by a call to 'genimage' with
> different parameters.
> 
> Now 'make isoimage' becomes:
> ...
> Kernel: arch/x86/boot/bzImage is ready  (#30)
>   GENIMAGE arch/x86/boot/image.iso
> Size of boot image is 4 sectors -> No emulation
>  15.37% done, estimate finish Sun Nov  5 23:36:57 2017
>  30.68% done, estimate finish Sun Nov  5 23:36:57 2017
>  46.04% done, estimate finish Sun Nov  5 23:36:57 2017
>  61.35% done, estimate finish Sun Nov  5 23:36:57 2017
>  76.69% done, estimate finish Sun Nov  5 23:36:57 2017
>  92.00% done, estimate finish Sun Nov  5 23:36:57 2017
> Total translation table size: 2048
> Total rockridge attributes bytes: 659
> Total directory bytes: 0
> Path table size(bytes): 10
> Max brk space used 0
> 32608 extents written (63 MB)
> Kernel: arch/x86/boot/image.iso is ready

Could we please also do another patch, to make it emit only this by default:

>   GENIMAGE arch/x86/boot/image.iso

None of the other messages really matter to the user.

Maybe emit them when building with V=1 - but none of that info is really 
interesting IMHO.

Thanks,

	Ingo

Re: [PATCH v2 1/4] x86, build: Factor out fdimage/isoimage generation commands to standalone script

[Du, Changbin]

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

On Tue, Nov 07, 2017 at 11:20:29AM +0100, Ingo Molnar wrote:
> 
> * changbin.du@intel.com <changbin.du@intel.com> wrote:
> 
> > From: Changbin Du <changbin.du@intel.com>
> > 
> > The build message for fdimage/isoimage are pretty unstructured. The raw
> > shell command blocks are printed. We can improve them as regular build
> > system messages. Besides, writing commands in a shell script is much more
> > easy than in a Makefile.
> > 
> > See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.
> > 
> > This patch factors out the commands used for fdimage/isoimage generation
> > from arch/x86/boot/Makefile to a new script arch/x86/boot/genimage.sh.
> > Then it adds the new kbuild command 'genimage' which invokes the new script.
> > All fdimages/isoimage files are now generated by a call to 'genimage' with
> > different parameters.
> > 
> > Now 'make isoimage' becomes:
> > ...
> > Kernel: arch/x86/boot/bzImage is ready  (#30)
> >   GENIMAGE arch/x86/boot/image.iso
> > Size of boot image is 4 sectors -> No emulation
> >  15.37% done, estimate finish Sun Nov  5 23:36:57 2017
> >  30.68% done, estimate finish Sun Nov  5 23:36:57 2017
> >  46.04% done, estimate finish Sun Nov  5 23:36:57 2017
> >  61.35% done, estimate finish Sun Nov  5 23:36:57 2017
> >  76.69% done, estimate finish Sun Nov  5 23:36:57 2017
> >  92.00% done, estimate finish Sun Nov  5 23:36:57 2017
> > Total translation table size: 2048
> > Total rockridge attributes bytes: 659
> > Total directory bytes: 0
> > Path table size(bytes): 10
> > Max brk space used 0
> > 32608 extents written (63 MB)
> > Kernel: arch/x86/boot/image.iso is ready
> 
> Could we please also do another patch, to make it emit only this by default:
> 
> >   GENIMAGE arch/x86/boot/image.iso
> 
> None of the other messages really matter to the user.
> 
> Maybe emit them when building with V=1 - but none of that info is really 
> interesting IMHO.
> 
Ingo, do you mean that omit below messages?
Size of boot image is 4 sectors -> No emulation
15.37% done, estimate finish Sun Nov  5 23:36:57 2017
...
32608 extents written (63 MB)

These are printed by mkisofs. How about just add '-quite' parameter to it?
Kernel: arch/x86/boot/bzImage is ready  (#75)
GENIMAGE arch/x86/boot/image.iso
Using /usr/lib/ISOLINUX/isolinux.bin
Using /usr/lib/syslinux/modules/bios/ldlinux.c32
Kernel: arch/x86/boot/image.iso is ready

> Thanks,
> 
> 	Ingo

-- 
Thanks,
Changbin Du

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [PATCH v2 1/4] x86, build: Factor out fdimage/isoimage generation commands to standalone script

[Ingo Molnar]

* Du, Changbin <changbin.du@intel.com> wrote:

> On Tue, Nov 07, 2017 at 11:20:29AM +0100, Ingo Molnar wrote:
> > 
> > * changbin.du@intel.com <changbin.du@intel.com> wrote:
> > 
> > > From: Changbin Du <changbin.du@intel.com>
> > > 
> > > The build message for fdimage/isoimage are pretty unstructured. The raw
> > > shell command blocks are printed. We can improve them as regular build
> > > system messages. Besides, writing commands in a shell script is much more
> > > easy than in a Makefile.
> > > 
> > > See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.
> > > 
> > > This patch factors out the commands used for fdimage/isoimage generation
> > > from arch/x86/boot/Makefile to a new script arch/x86/boot/genimage.sh.
> > > Then it adds the new kbuild command 'genimage' which invokes the new script.
> > > All fdimages/isoimage files are now generated by a call to 'genimage' with
> > > different parameters.
> > > 
> > > Now 'make isoimage' becomes:
> > > ...
> > > Kernel: arch/x86/boot/bzImage is ready  (#30)
> > >   GENIMAGE arch/x86/boot/image.iso
> > > Size of boot image is 4 sectors -> No emulation
> > >  15.37% done, estimate finish Sun Nov  5 23:36:57 2017
> > >  30.68% done, estimate finish Sun Nov  5 23:36:57 2017
> > >  46.04% done, estimate finish Sun Nov  5 23:36:57 2017
> > >  61.35% done, estimate finish Sun Nov  5 23:36:57 2017
> > >  76.69% done, estimate finish Sun Nov  5 23:36:57 2017
> > >  92.00% done, estimate finish Sun Nov  5 23:36:57 2017
> > > Total translation table size: 2048
> > > Total rockridge attributes bytes: 659
> > > Total directory bytes: 0
> > > Path table size(bytes): 10
> > > Max brk space used 0
> > > 32608 extents written (63 MB)
> > > Kernel: arch/x86/boot/image.iso is ready
> > 
> > Could we please also do another patch, to make it emit only this by default:
> > 
> > >   GENIMAGE arch/x86/boot/image.iso
> > 
> > None of the other messages really matter to the user.
> > 
> > Maybe emit them when building with V=1 - but none of that info is really 
> > interesting IMHO.
> > 
> Ingo, do you mean that omit below messages?
> Size of boot image is 4 sectors -> No emulation
> 15.37% done, estimate finish Sun Nov  5 23:36:57 2017
> ...
> 32608 extents written (63 MB)

Yes, and all the other unnecessary verbosity.

> These are printed by mkisofs. How about just add '-quite' parameter to it?

Yes, or redirect it to /dev/null or any other solution.

Thanks,

	Ingo

[tip:x86/asm] x86/build: Factor out fdimage/isoimage generation commands to standalone script

[tip-bot for Changbin Du]

Commit-ID:  4366d57af19e89d23ef9154414b2539f1c3d18fa
Gitweb:     https://git.kernel.org/tip/4366d57af19e89d23ef9154414b2539f1c3d18fa
Author:     Changbin Du <changbin.du@intel.com>
AuthorDate: Mon, 6 Nov 2017 11:32:56 +0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 7 Nov 2017 11:22:20 +0100

x86/build: Factor out fdimage/isoimage generation commands to standalone script

The build messages for fdimage/isoimage generation are pretty unstructured,
just the raw shell command blocks are printed.

Emit shortened messages similar to existing kbuild messages, and move
the Makefile commands into a separate shell script - which is much
easier to handle.

This patch factors out the commands used for fdimage/isoimage generation
from arch/x86/boot/Makefile to a new script arch/x86/boot/genimage.sh.
Then it adds the new kbuild command 'genimage' which invokes the new script.
All fdimages/isoimage files are now generated by a call to 'genimage' with
different parameters.

Now 'make isoimage' becomes:

	...
	Kernel: arch/x86/boot/bzImage is ready  (#30)
	  GENIMAGE arch/x86/boot/image.iso
	Size of boot image is 4 sectors -> No emulation
	 15.37% done, estimate finish Sun Nov  5 23:36:57 2017
	 30.68% done, estimate finish Sun Nov  5 23:36:57 2017
	 46.04% done, estimate finish Sun Nov  5 23:36:57 2017
	 61.35% done, estimate finish Sun Nov  5 23:36:57 2017
	 76.69% done, estimate finish Sun Nov  5 23:36:57 2017
	 92.00% done, estimate finish Sun Nov  5 23:36:57 2017
	Total translation table size: 2048
	Total rockridge attributes bytes: 659
	Total directory bytes: 0
	Path table size(bytes): 10
	Max brk space used 0
	32608 extents written (63 MB)
	Kernel: arch/x86/boot/image.iso is ready

Before:

	Kernel: arch/x86/boot/bzImage is ready  (#63)
	rm -rf arch/x86/boot/isoimage
	mkdir arch/x86/boot/isoimage
	for i in lib lib64 share end ; do \
		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then \
			cp /usr/$i/syslinux/isolinux.bin arch/x86/boot/isoimage ; \
			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then \
				cp /usr/$i/syslinux/ldlinux.c32 arch/x86/boot/isoimage ; \
			fi ; \
			break ; \
		fi ; \
		if [ $i = end ] ; then exit 1 ; fi ; \
	done
	...

Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1509939179-7556-2-git-send-email-changbin.du@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/boot/Makefile    |  59 +++++---------------------
 arch/x86/boot/genimage.sh | 105 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 116 insertions(+), 48 deletions(-)

diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index d88a2fd..9b5adae 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -123,63 +123,26 @@ image_cmdline = default linux $(FDARGS) $(if $(FDINITRD),initrd=initrd.img,)
 $(obj)/mtools.conf: $(src)/mtools.conf.in
 	sed -e 's|@OBJ@|$(obj)|g' < $< > $@
 
+quiet_cmd_genimage = GENIMAGE $3
+cmd_genimage = sh $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
+			$(obj)/mtools.conf '$(image_cmdline)' $(FDINITRD)
+
 # This requires write access to /dev/fd0
 bzdisk: $(obj)/bzImage $(obj)/mtools.conf
-	MTOOLSRC=$(obj)/mtools.conf mformat a:			; sync
-	syslinux /dev/fd0					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(src)/mtools.conf mcopy - a:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' a:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage a:linux	; sync
+	$(call cmd,genimage,bzdisk,/dev/fd0)
 
 # These require being root or having syslinux 2.02 or higher installed
 fdimage fdimage144: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=1440
-	MTOOLSRC=$(obj)/mtools.conf mformat v:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - v:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' v:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage v:linux	; sync
+	$(call cmd,genimage,fdimage144,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 fdimage288: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=2880
-	MTOOLSRC=$(obj)/mtools.conf mformat w:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - w:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' w:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage w:linux	; sync
+	$(call cmd,genimage,fdimage288,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 isoimage: $(obj)/bzImage
-	-rm -rf $(obj)/isoimage
-	mkdir $(obj)/isoimage
-	for i in lib lib64 share end ; do \
-		if [ -f /usr/$$i/syslinux/isolinux.bin ] ; then \
-			cp /usr/$$i/syslinux/isolinux.bin $(obj)/isoimage ; \
-			if [ -f /usr/$$i/syslinux/ldlinux.c32 ]; then \
-				cp /usr/$$i/syslinux/ldlinux.c32 $(obj)/isoimage ; \
-			fi ; \
-			break ; \
-		fi ; \
-		if [ $$i = end ] ; then exit 1 ; fi ; \
-	done
-	cp $(obj)/bzImage $(obj)/isoimage/linux
-	echo '$(image_cmdline)' > $(obj)/isoimage/isolinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		cp '$(FDINITRD)' $(obj)/isoimage/initrd.img ; \
-	fi
-	mkisofs -J -r -o $(obj)/image.iso -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table \
-		$(obj)/isoimage
-	isohybrid $(obj)/image.iso 2>/dev/null || true
-	rm -rf $(obj)/isoimage
+	$(call cmd,genimage,isoimage,$(obj)/image.iso)
+	@$(kecho) 'Kernel: $(obj)/image.iso is ready'
 
 bzlilo: $(obj)/bzImage
 	if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz $(INSTALL_PATH)/vmlinuz.old; fi
diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
new file mode 100644
index 0000000..75a9de1
--- /dev/null
+++ b/arch/x86/boot/genimage.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 2017 by Changbin Du <changbin.du@intel.com>
+#
+# Adapted from code in arch/x86/boot/Makefile by H. Peter Anvin and others
+#
+# "make fdimage/fdimage144/fdimage288/isoimage" script for x86 architecture
+#
+# Arguments:
+#   $1 - fdimage format
+#   $2 - target image file
+#   $3 - kernel bzImage file
+#   $4 - mtool configuration file
+#   $5 - kernel cmdline
+#   $6 - inird image file
+#
+
+verify () {
+	if [ ! -f "$1" ]; then
+		echo ""                                                   1>&2
+		echo " *** Missing file: $1"                              1>&2
+		echo ""                                                   1>&2
+		exit 1
+	fi
+}
+
+
+export MTOOLSRC=$4
+FIMAGE=$2
+FBZIMAGE=$3
+KCMDLINE=$5
+FDINITRD=$6
+
+# Make sure the files actually exist
+verify "$FBZIMAGE"
+verify "$MTOOLSRC"
+
+genbzdisk() {
+	mformat a:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - a:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" a:initrd.img
+	fi
+	mcopy $FBZIMAGE a:linux
+}
+
+genfdimage144() {
+	dd if=/dev/zero of=$FIMAGE bs=1024 count=1440
+	mformat v:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - v:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" v:initrd.img
+	fi
+	mcopy $FBZIMAGE v:linux
+}
+
+genfdimage288() {
+	dd if=/dev/zero of=$FIMAGE bs=1024 count=2880
+	mformat w:
+	syslinux $FIMAGE
+	echo "$KCMDLINE" | mcopy - W:syslinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		mcopy "$FDINITRD" w:initrd.img
+	fi
+	mcopy $FBZIMAGE w:linux
+}
+
+genisoimage() {
+	tmp_dir=`dirname $FIMAGE`/isoimage
+	rm -rf $tmp_dir
+	mkdir $tmp_dir
+	for i in lib lib64 share end ; do
+		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then
+			cp /usr/$i/syslinux/isolinux.bin $tmp_dir
+			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then
+				cp /usr/$i/syslinux/ldlinux.c32 $tmp_dir
+			fi
+			break
+		fi
+		if [ $i = end ] ; then exit 1 ; fi ;
+	done
+	cp $FBZIMAGE $tmp_dir/linux
+	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg
+	if [ -f "$FDINITRD" ] ; then
+		cp "$FDINITRD" $tmp_dir/initrd.img
+	fi
+	mkisofs -J -r -o $FIMAGE -b isolinux.bin -c boot.cat \
+		-no-emul-boot -boot-load-size 4 -boot-info-table $tmp_dir
+	isohybrid $FIMAGE 2>/dev/null || true
+	rm -rf $tmp_dir
+}
+
+case $1 in
+	bzdisk)     genbzdisk;;
+	fdimage144) genfdimage144;;
+	fdimage288) genfdimage288;;
+	isoimage)   genisoimage;;
+	*)          echo 'Unknown image format'; exit 1;
+esac

[tip:x86/asm] x86/build: Add new paths for isolinux.bin and ldlinux.c32

[tip-bot for Changbin Du]

Commit-ID:  8a7546a04ecb7c9388a3e935c6cfcc7c2c4c6b67
Gitweb:     https://git.kernel.org/tip/8a7546a04ecb7c9388a3e935c6cfcc7c2c4c6b67
Author:     Changbin Du <changbin.du@intel.com>
AuthorDate: Mon, 6 Nov 2017 11:32:57 +0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 7 Nov 2017 11:22:20 +0100

x86/build: Add new paths for isolinux.bin and ldlinux.c32

Recently I failed to build isoimage target, because the path of isolinux.bin
changed to /usr/xxx/ISOLINUX/isolinux.bin, as well as ldlinux.c32 which
changed to /usr/xxx/syslinux/modules/bios/ldlinux.c32.

This patch improves the file search logic:
  - Show a error message instead of silent fail.
  - Add above new paths.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: yamada.masahiro@socionext.com
Link: http://lkml.kernel.org/r/1509939179-7556-3-git-send-email-changbin.du@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/boot/genimage.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
index 75a9de1..6c8100b 100644
--- a/arch/x86/boot/genimage.sh
+++ b/arch/x86/boot/genimage.sh
@@ -76,14 +76,27 @@ genisoimage() {
 	rm -rf $tmp_dir
 	mkdir $tmp_dir
 	for i in lib lib64 share end ; do
-		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then
-			cp /usr/$i/syslinux/isolinux.bin $tmp_dir
-			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then
-				cp /usr/$i/syslinux/ldlinux.c32 $tmp_dir
+		for j in syslinux ISOLINUX ; do
+			if [ -f /usr/$i/$j/isolinux.bin ] ; then
+				isolinux=/usr/$i/$j/isolinux.bin
+				echo "Using $isolinux"
+				cp $isolinux $tmp_dir
 			fi
+		done
+		for j in syslinux syslinux/modules/bios ; do
+			if [ -f /usr/$i/$j/ldlinux.c32 ]; then
+				ldlinux=/usr/$i/$j/ldlinux.c32
+				echo "Using $ldlinux"
+				cp $ldlinux $tmp_dir
+			fi
+		done
+		if [ -n "$isolinux" -a -n "$ldlinux" ] ; then
 			break
 		fi
-		if [ $i = end ] ; then exit 1 ; fi ;
+		if [ $i = end -a -z "$isolinux" ] ; then
+			echo 'Need an isolinux.bin file, please install syslinux/isolinux.'
+			exit 1
+		fi
 	done
 	cp $FBZIMAGE $tmp_dir/linux
 	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg

[tip:x86/asm] x86/build: Specify -input-charset=utf-8 for mkisofs

[tip-bot for Changbin Du]

Commit-ID:  c306ba7b909408dbf935bb4a8dc62fae64cb337c
Gitweb:     https://git.kernel.org/tip/c306ba7b909408dbf935bb4a8dc62fae64cb337c
Author:     Changbin Du <changbin.du@intel.com>
AuthorDate: Mon, 6 Nov 2017 11:32:58 +0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 7 Nov 2017 11:22:20 +0100

x86/build: Specify -input-charset=utf-8 for mkisofs

It avoids the following warning triggered by newer versions of mkisofs:

    -input-charset not specified, using utf-8 (detected in locale settings)

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: yamada.masahiro@socionext.com
Link: http://lkml.kernel.org/r/1509939179-7556-4-git-send-email-changbin.du@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/boot/genimage.sh | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/x86/boot/genimage.sh b/arch/x86/boot/genimage.sh
index 6c8100b..628e936 100644
--- a/arch/x86/boot/genimage.sh
+++ b/arch/x86/boot/genimage.sh
@@ -103,8 +103,9 @@ genisoimage() {
 	if [ -f "$FDINITRD" ] ; then
 		cp "$FDINITRD" $tmp_dir/initrd.img
 	fi
-	mkisofs -J -r -o $FIMAGE -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table $tmp_dir
+	mkisofs -J -r -input-charset=utf-8 -o $FIMAGE -b isolinux.bin \
+		-c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table \
+		$tmp_dir
 	isohybrid $FIMAGE 2>/dev/null || true
 	rm -rf $tmp_dir
 }

[tip:x86/asm] x86/build: Add more generated files to the .gitignore file

[tip-bot for Changbin Du]

Commit-ID:  d786f05175fae09cc39dfa5769c62649341caeb4
Gitweb:     https://git.kernel.org/tip/d786f05175fae09cc39dfa5769c62649341caeb4
Author:     Changbin Du <changbin.du@intel.com>
AuthorDate: Mon, 6 Nov 2017 11:32:59 +0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 7 Nov 2017 11:22:21 +0100

x86/build: Add more generated files to the .gitignore file

Some of the files generated by the build process were not listed.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: yamada.masahiro@socionext.com
Link: http://lkml.kernel.org/r/1509939179-7556-5-git-send-email-changbin.du@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/boot/.gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/boot/.gitignore b/arch/x86/boot/.gitignore
index e3cf9f6..09d25dd 100644
--- a/arch/x86/boot/.gitignore
+++ b/arch/x86/boot/.gitignore
@@ -7,3 +7,6 @@ zoffset.h
 setup
 setup.bin
 setup.elf
+fdimage
+mtools.conf
+image.iso