linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 00/13] initramfs: a lot of cleanups
@ 2020-01-04 15:02 Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
                   ` (13 more replies)
  0 siblings, 14 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen,
	Mauro Carvalho Chehab, Sam Ravnborg, linux-kernel



Masahiro Yamada (13):
  initramfs: replace klibcdirs in Makefile with FORCE
  gen_initramfs_list.sh: remove unused variable 'default_list'
  gen_initramfs_list.sh: fix the tool name in the comment
  initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  initramfs: remove redundant dependency on BLK_DEV_INITRD
  initramfs: make compression options not depend on INITRAMFS_SOURCE
  initramfs: make initramfs compression choice non-optional
  initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in
    Makefile
  initramfs: generate dependency list and cpio at the same time
  initramfs: add default_cpio_list, and delete -d option support
  gen_initramfs.sh: always output cpio even without -o option
  initramfs: refactor the initramfs build rules
  gen_initramfs.sh: remove intermediate cpio_list on errors

 usr/.gitignore                                |   8 +-
 usr/Kconfig                                   |  26 ---
 usr/Makefile                                  |  97 ++++++----
 usr/default_cpio_list                         |   6 +
 ...gen_initramfs_list.sh => gen_initramfs.sh} | 167 +++++-------------
 usr/initramfs_data.S                          |   5 +-
 6 files changed, 112 insertions(+), 197 deletions(-)
 create mode 100644 usr/default_cpio_list
 rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (53%)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:33   ` Greg Thelen
  2020-01-07 10:46   ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list' Masahiro Yamada
                   ` (12 subsequent siblings)
  13 siblings, 2 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen, Sam Ravnborg, linux-kernel

'klibcdirs' was added by commit d39a206bc35d ("kbuild: rebuild initramfs
if content of initramfs changes"). If this is just a matter of forcing
execution of the recipe line, we can replace it with FORCE.

I do not understand the purpose of

   $(deps_initramfs): klibcdirs

Remove it.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2:
  - New patch (I forgot to submit this in v1 series)

 usr/Makefile | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/usr/Makefile b/usr/Makefile
index e6f7cb2f81db..55c942da01cd 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -3,9 +3,6 @@
 # kbuild file for usr/ - including initramfs image
 #
 
-klibcdirs:;
-PHONY += klibcdirs
-
 suffix_y = $(subst $\",,$(CONFIG_INITRAMFS_COMPRESSION))
 datafile_y = initramfs_data.cpio$(suffix_y)
 datafile_d_y = .$(datafile_y).d
@@ -50,13 +47,12 @@ targets := $(datafile_y)
 # do not try to update files included in initramfs
 $(deps_initramfs): ;
 
-$(deps_initramfs): klibcdirs
 # We rebuild initramfs_data.cpio if:
 # 1) Any included file is newer than initramfs_data.cpio
 # 2) There are changes in which files are included (added or deleted)
 # 3) If gen_init_cpio are newer than initramfs_data.cpio
 # 4) Arguments to gen_initramfs.sh changes
-$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
+$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) FORCE
 	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/$(datafile_d_y)
 	$(call if_changed,initfs)
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list'
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:34   ` Greg Thelen
  2020-01-04 15:02 ` [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment Masahiro Yamada
                   ` (11 subsequent siblings)
  13 siblings, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

This is assigned, but not referenced.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/gen_initramfs_list.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs_list.sh
index 2bbac73e6477..909a7b865aa7 100755
--- a/usr/gen_initramfs_list.sh
+++ b/usr/gen_initramfs_list.sh
@@ -280,7 +280,6 @@ while [ $# -gt 0 ]; do
 			shift
 			;;
 		"-d")	# display default initramfs list
-			default_list="$arg"
 			${dep_list}default_initramfs
 			;;
 		"-h")
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list' Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:34   ` Greg Thelen
  2020-01-04 15:02 ` [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh Masahiro Yamada
                   ` (10 subsequent siblings)
  13 siblings, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

There is no tool named "gen_initramfs". The correct name is
"gen_init_cpio".

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/gen_initramfs_list.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs_list.sh
index 909a7b865aa7..1fd93414fb37 100755
--- a/usr/gen_initramfs_list.sh
+++ b/usr/gen_initramfs_list.sh
@@ -189,7 +189,7 @@ dir_filelist() {
 
 # if only one file is specified and it is .cpio file then use it direct as fs
 # if a directory is specified then add all files in given direcotry to fs
-# if a regular file is specified assume it is in gen_initramfs format
+# if a regular file is specified assume it is in gen_init_cpio format
 input_file() {
 	source="$1"
 	if [ -f "$1" ]; then
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (2 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:43   ` Greg Thelen
  2020-01-04 15:02 ` [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD Masahiro Yamada
                   ` (9 subsequent siblings)
  13 siblings, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen, Sam Ravnborg, linux-kernel

The comments in usr/Makefile wrongly refer to the script name (twice).

Line 37:
    # The dependency list is generated by gen_initramfs.sh -l

Line 54:
    # 4) Arguments to gen_initramfs.sh changes

There does not exist such a script.

I was going to fix the comments, but after some consideration, I thought
"gen_initramfs.sh" would be more suitable than "gen_initramfs_list.sh"
because it generates an initramfs image in the common usage.

The script generates a list that can be fed to gen_init_cpio only when
it is directly run without -o or -l option.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Makefile                                    | 2 +-
 usr/{gen_initramfs_list.sh => gen_initramfs.sh} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (100%)

diff --git a/usr/Makefile b/usr/Makefile
index 55c942da01cd..e44a66b8c051 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -24,7 +24,7 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
 # Generate the initramfs cpio archive
 
 hostprogs-y := gen_init_cpio
-initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs_list.sh
+initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs.sh
 ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
 			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
 ramfs-args  := \
diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs.sh
similarity index 100%
rename from usr/gen_initramfs_list.sh
rename to usr/gen_initramfs.sh
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (3 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:44   ` Greg Thelen
  2020-01-04 15:02 ` [PATCH v2 06/13] initramfs: make compression options not depend on INITRAMFS_SOURCE Masahiro Yamada
                   ` (8 subsequent siblings)
  13 siblings, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, Mauro Carvalho Chehab, linux-kernel

init/Kconfig includes usr/Kconfig inside the "if BLK_DEV_INITRD" ...
"endif" block:

    if BLK_DEV_INITRD

    source "usr/Kconfig"

    endif

Hence, all the defines in usr/Kconfig depend on BLK_DEV_INITRD.

Remove the redundant "depends on BLK_DEV_INITRD".

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Kconfig | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/usr/Kconfig b/usr/Kconfig
index a6b68503d177..72f50a21c051 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -54,7 +54,6 @@ config INITRAMFS_ROOT_GID
 
 config RD_GZIP
 	bool "Support initial ramdisk/ramfs compressed using gzip"
-	depends on BLK_DEV_INITRD
 	default y
 	select DECOMPRESS_GZIP
 	help
@@ -64,7 +63,6 @@ config RD_GZIP
 config RD_BZIP2
 	bool "Support initial ramdisk/ramfs compressed using bzip2"
 	default y
-	depends on BLK_DEV_INITRD
 	select DECOMPRESS_BZIP2
 	help
 	  Support loading of a bzip2 encoded initial ramdisk or cpio buffer
@@ -73,7 +71,6 @@ config RD_BZIP2
 config RD_LZMA
 	bool "Support initial ramdisk/ramfs compressed using LZMA"
 	default y
-	depends on BLK_DEV_INITRD
 	select DECOMPRESS_LZMA
 	help
 	  Support loading of a LZMA encoded initial ramdisk or cpio buffer
@@ -81,7 +78,6 @@ config RD_LZMA
 
 config RD_XZ
 	bool "Support initial ramdisk/ramfs compressed using XZ"
-	depends on BLK_DEV_INITRD
 	default y
 	select DECOMPRESS_XZ
 	help
@@ -91,7 +87,6 @@ config RD_XZ
 config RD_LZO
 	bool "Support initial ramdisk/ramfs compressed using LZO"
 	default y
-	depends on BLK_DEV_INITRD
 	select DECOMPRESS_LZO
 	help
 	  Support loading of a LZO encoded initial ramdisk or cpio buffer
@@ -100,7 +95,6 @@ config RD_LZO
 config RD_LZ4
 	bool "Support initial ramdisk/ramfs compressed using LZ4"
 	default y
-	depends on BLK_DEV_INITRD
 	select DECOMPRESS_LZ4
 	help
 	  Support loading of a LZ4 encoded initial ramdisk or cpio buffer
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 06/13] initramfs: make compression options not depend on INITRAMFS_SOURCE
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (4 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 07/13] initramfs: make initramfs compression choice non-optional Masahiro Yamada
                   ` (7 subsequent siblings)
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, Mauro Carvalho Chehab, linux-kernel

Even if INITRAMFS_SOURCE is empty, usr/gen_initramfs.sh generates a
tiny default initramfs, which is embedded in vmlinux.

So, defining INITRAMFS_COMPRESSION* options should be valid irrespective
of INITRAMFS_SOURCE.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Kconfig | 2 --
 1 file changed, 2 deletions(-)

diff --git a/usr/Kconfig b/usr/Kconfig
index 72f50a21c051..43934c128010 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -102,7 +102,6 @@ config RD_LZ4
 
 choice
 	prompt "Built-in initramfs compression mode"
-	depends on INITRAMFS_SOURCE!=""
 	optional
 	help
 	  This option allows you to decide by which algorithm the builtin
@@ -211,7 +210,6 @@ config INITRAMFS_COMPRESSION_LZ4
 endchoice
 
 config INITRAMFS_COMPRESSION
-	depends on INITRAMFS_SOURCE!=""
 	string
 	default ""      if INITRAMFS_COMPRESSION_NONE
 	default ".gz"   if INITRAMFS_COMPRESSION_GZIP
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 07/13] initramfs: make initramfs compression choice non-optional
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (5 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 06/13] initramfs: make compression options not depend on INITRAMFS_SOURCE Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile Masahiro Yamada
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, Mauro Carvalho Chehab, linux-kernel

Currently, the choice of the initramfs compression mode is too complex
because users are allowed to not specify the compression mode at all.

I think it makes more sense to require users to choose the compression
explicitly, and delete the fallback defaults of INITRAMFS_COMPRESSION.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Kconfig | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/usr/Kconfig b/usr/Kconfig
index 43934c128010..ab61e81165e0 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -102,7 +102,6 @@ config RD_LZ4
 
 choice
 	prompt "Built-in initramfs compression mode"
-	optional
 	help
 	  This option allows you to decide by which algorithm the builtin
 	  initramfs will be compressed.  Several compression algorithms are
@@ -218,10 +217,3 @@ config INITRAMFS_COMPRESSION
 	default ".xz"   if INITRAMFS_COMPRESSION_XZ
 	default ".lzo"  if INITRAMFS_COMPRESSION_LZO
 	default ".lz4"  if INITRAMFS_COMPRESSION_LZ4
-	default ".gz"   if RD_GZIP
-	default ".lz4"  if RD_LZ4
-	default ".lzo"  if RD_LZO
-	default ".xz"   if RD_XZ
-	default ".lzma" if RD_LZMA
-	default ".bz2"  if RD_BZIP2
-	default ""
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (6 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 07/13] initramfs: make initramfs compression choice non-optional Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-06  7:51   ` Greg Thelen
  2020-01-04 15:02 ` [PATCH v2 09/13] initramfs: generate dependency list and cpio at the same time Masahiro Yamada
                   ` (5 subsequent siblings)
  13 siblings, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen, Sam Ravnborg, linux-kernel

Specify the dependency directly in the Makefile.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Makefile         | 7 +++----
 usr/gen_initramfs.sh | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/usr/Makefile b/usr/Makefile
index e44a66b8c051..7df6559f0f30 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -24,7 +24,6 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
 # Generate the initramfs cpio archive
 
 hostprogs-y := gen_init_cpio
-initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs.sh
 ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
 			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
 ramfs-args  := \
@@ -40,7 +39,7 @@ ifneq ($(wildcard $(obj)/$(datafile_d_y)),)
 endif
 
 quiet_cmd_initfs = GEN     $@
-      cmd_initfs = $(initramfs) -o $@ $(ramfs-args) $(ramfs-input)
+      cmd_initfs = $< -o $@ $(ramfs-args) $(ramfs-input)
 
 targets := $(datafile_y)
 
@@ -52,8 +51,8 @@ $(deps_initramfs): ;
 # 2) There are changes in which files are included (added or deleted)
 # 3) If gen_init_cpio are newer than initramfs_data.cpio
 # 4) Arguments to gen_initramfs.sh changes
-$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) FORCE
-	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/$(datafile_d_y)
+$(obj)/$(datafile_y): $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
+	$(Q)$< -l $(ramfs-input) > $(obj)/$(datafile_d_y)
 	$(call if_changed,initfs)
 
 subdir-$(CONFIG_UAPI_HEADER_TEST) += include
diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index 1fd93414fb37..68b6ddfd5f96 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -235,7 +235,7 @@ arg="$1"
 case "$arg" in
 	"-l")	# files included in initramfs - used by kbuild
 		dep_list="list_"
-		echo "deps_initramfs := $0 \\"
+		echo "deps_initramfs := \\"
 		shift
 		;;
 	"-o")	# generate compressed cpio image named $1
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 09/13] initramfs: generate dependency list and cpio at the same time
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (7 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 10/13] initramfs: add default_cpio_list, and delete -d option support Masahiro Yamada
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen, Sam Ravnborg, linux-kernel

Currently, this script is run twice, for the dependency list, and then
for the cpio archive.

The first one is re-run every time although its build log is suppressed
so nobody notices it.

Make it work more efficiently by generating the cpio and the dependency
list at the same time.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Makefile         |   3 +-
 usr/gen_initramfs.sh | 108 ++++++++++++++++++-------------------------
 2 files changed, 47 insertions(+), 64 deletions(-)

diff --git a/usr/Makefile b/usr/Makefile
index 7df6559f0f30..3ae8b45bfc61 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -39,7 +39,7 @@ ifneq ($(wildcard $(obj)/$(datafile_d_y)),)
 endif
 
 quiet_cmd_initfs = GEN     $@
-      cmd_initfs = $< -o $@ $(ramfs-args) $(ramfs-input)
+      cmd_initfs = $< -o $@ -l $(obj)/$(datafile_d_y) $(ramfs-args) $(ramfs-input)
 
 targets := $(datafile_y)
 
@@ -52,7 +52,6 @@ $(deps_initramfs): ;
 # 3) If gen_init_cpio are newer than initramfs_data.cpio
 # 4) Arguments to gen_initramfs.sh changes
 $(obj)/$(datafile_y): $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
-	$(Q)$< -l $(ramfs-input) > $(obj)/$(datafile_d_y)
 	$(call if_changed,initfs)
 
 subdir-$(CONFIG_UAPI_HEADER_TEST) += include
diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index 68b6ddfd5f96..49a4e22147b5 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -15,9 +15,10 @@ set -e
 usage() {
 cat << EOF
 Usage:
-$0 [-o <file>] [-u <uid>] [-g <gid>] {-d | <cpio_source>} ...
+$0 [-o <file>] [-l <dep_list>] [-u <uid>] [-g <gid>] {-d | <cpio_source>} ...
 	-o <file>      Create compressed initramfs file named <file> using
 		       gen_init_cpio and compressor depending on the extension
+	-l <dep_list>  Create dependency list named <dep_list>
 	-u <uid>       User ID to map to user ID 0 (root).
 		       <uid> is only meaningful if <cpio_source> is a
 		       directory.  "squash" forces all files to uid 0.
@@ -42,11 +43,6 @@ field() {
 	shift $1 ; echo $1
 }
 
-list_default_initramfs() {
-	# echo usr/kinit/kinit
-	:
-}
-
 default_initramfs() {
 	cat <<-EOF >> ${output}
 		# This is a very simple, default initramfs
@@ -81,10 +77,6 @@ filetype() {
 	return 0
 }
 
-list_print_mtime() {
-	:
-}
-
 print_mtime() {
 	local my_mtime="0"
 
@@ -97,10 +89,10 @@ print_mtime() {
 }
 
 list_parse() {
-	if [ -L "$1" ]; then
+	if [ -z "$dep_list" -o -L "$1" ]; then
 		return
 	fi
-	echo "$1" | sed 's/:/\\:/g; s/$/ \\/'
+	echo "$1" | sed 's/:/\\:/g; s/$/ \\/' >> $dep_list
 }
 
 # for each file print a line in following format
@@ -161,28 +153,25 @@ unknown_option() {
 	exit 1
 }
 
-list_header() {
-	:
-}
-
 header() {
 	printf "\n#####################\n# $1\n" >> ${output}
 }
 
 # process one directory (incl sub-directories)
 dir_filelist() {
-	${dep_list}header "$1"
+	header "$1"
 
 	srcdir=$(echo "$1" | sed -e 's://*:/:g')
 	dirlist=$(find "${srcdir}" -printf "%p %m %U %G\n" | LANG=C sort)
 
 	# If $dirlist is only one line, then the directory is empty
 	if [  "$(echo "${dirlist}" | wc -l)" -gt 1 ]; then
-		${dep_list}print_mtime "$1"
+		print_mtime "$1"
 
 		echo "${dirlist}" | \
 		while read x; do
-			${dep_list}parse ${x}
+			list_parse $x
+			parse $x
 		done
 	fi
 }
@@ -193,22 +182,21 @@ dir_filelist() {
 input_file() {
 	source="$1"
 	if [ -f "$1" ]; then
-		${dep_list}header "$1"
+		header "$1"
 		is_cpio="$(echo "$1" | sed 's/^.*\.cpio\(\..*\)\{0,1\}/cpio/')"
 		if [ $2 -eq 0 -a ${is_cpio} = "cpio" ]; then
 			cpio_file=$1
 			echo "$1" | grep -q '^.*\.cpio\..*' && is_cpio_compressed="compressed"
-			[ ! -z ${dep_list} ] && echo "$1"
+			[ -n "$dep_list" ] && echo "$1" >> $dep_list
 			return 0
 		fi
-		if [ -z ${dep_list} ]; then
-			print_mtime "$1" >> ${output}
-			cat "$1"         >> ${output}
-		else
-		        echo "$1 \\"
+		print_mtime "$1" >> ${output}
+		cat "$1"         >> ${output}
+		if [ -n "$dep_list" ]; then
+		        echo "$1 \\"  >> $dep_list
 			cat "$1" | while read type dir file perm ; do
 				if [ "$type" = "file" ]; then
-					echo "$file \\";
+					echo "$file \\" >> $dep_list
 				fi
 			done
 		fi
@@ -231,44 +219,40 @@ output_file=""
 is_cpio_compressed=
 compr="gzip -n -9 -f"
 
-arg="$1"
-case "$arg" in
-	"-l")	# files included in initramfs - used by kbuild
-		dep_list="list_"
-		echo "deps_initramfs := \\"
-		shift
-		;;
-	"-o")	# generate compressed cpio image named $1
-		shift
-		output_file="$1"
-		cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
-		output=${cpio_list}
-		echo "$output_file" | grep -q "\.gz$" \
-                && [ -x "`which gzip 2> /dev/null`" ] \
-                && compr="gzip -n -9 -f"
-		echo "$output_file" | grep -q "\.bz2$" \
-                && [ -x "`which bzip2 2> /dev/null`" ] \
-                && compr="bzip2 -9 -f"
-		echo "$output_file" | grep -q "\.lzma$" \
-                && [ -x "`which lzma 2> /dev/null`" ] \
-                && compr="lzma -9 -f"
-		echo "$output_file" | grep -q "\.xz$" \
-                && [ -x "`which xz 2> /dev/null`" ] \
-                && compr="xz --check=crc32 --lzma2=dict=1MiB"
-		echo "$output_file" | grep -q "\.lzo$" \
-                && [ -x "`which lzop 2> /dev/null`" ] \
-                && compr="lzop -9 -f"
-		echo "$output_file" | grep -q "\.lz4$" \
-                && [ -x "`which lz4 2> /dev/null`" ] \
-                && compr="lz4 -l -9 -f"
-		echo "$output_file" | grep -q "\.cpio$" && compr="cat"
-		shift
-		;;
-esac
 while [ $# -gt 0 ]; do
 	arg="$1"
 	shift
 	case "$arg" in
+		"-l")	# files included in initramfs - used by kbuild
+			dep_list="$1"
+			echo "deps_initramfs := \\" > $dep_list
+			shift
+			;;
+		"-o")	# generate compressed cpio image named $1
+			output_file="$1"
+			cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
+			output=${cpio_list}
+			echo "$output_file" | grep -q "\.gz$" \
+			&& [ -x "`which gzip 2> /dev/null`" ] \
+			&& compr="gzip -n -9 -f"
+			echo "$output_file" | grep -q "\.bz2$" \
+			&& [ -x "`which bzip2 2> /dev/null`" ] \
+			&& compr="bzip2 -9 -f"
+			echo "$output_file" | grep -q "\.lzma$" \
+			&& [ -x "`which lzma 2> /dev/null`" ] \
+			&& compr="lzma -9 -f"
+			echo "$output_file" | grep -q "\.xz$" \
+			&& [ -x "`which xz 2> /dev/null`" ] \
+			&& compr="xz --check=crc32 --lzma2=dict=1MiB"
+			echo "$output_file" | grep -q "\.lzo$" \
+			&& [ -x "`which lzop 2> /dev/null`" ] \
+			&& compr="lzop -9 -f"
+			echo "$output_file" | grep -q "\.lz4$" \
+			&& [ -x "`which lz4 2> /dev/null`" ] \
+			&& compr="lz4 -l -9 -f"
+			echo "$output_file" | grep -q "\.cpio$" && compr="cat"
+			shift
+			;;
 		"-u")	# map $1 to uid=0 (root)
 			root_uid="$1"
 			[ "$root_uid" = "-1" ] && root_uid=$(id -u || echo 0)
@@ -280,7 +264,7 @@ while [ $# -gt 0 ]; do
 			shift
 			;;
 		"-d")	# display default initramfs list
-			${dep_list}default_initramfs
+			default_initramfs
 			;;
 		"-h")
 			usage
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 10/13] initramfs: add default_cpio_list, and delete -d option support
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (8 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 09/13] initramfs: generate dependency list and cpio at the same time Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 11/13] gen_initramfs.sh: always output cpio even without -o option Masahiro Yamada
                   ` (3 subsequent siblings)
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen, Sam Ravnborg, linux-kernel

When CONFIG_INITRAMFS_SOURCE is empty, the Makefile passes the -d
option to gen_initramfs.sh to create the default initramfs, which
contains /dev, /dev/console, and /root.

This commit simplifies the default behavior; remove the -d option,
and add the default cpio list.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/Makefile          |  2 +-
 usr/default_cpio_list |  6 ++++++
 usr/gen_initramfs.sh  | 16 ----------------
 3 files changed, 7 insertions(+), 17 deletions(-)
 create mode 100644 usr/default_cpio_list

diff --git a/usr/Makefile b/usr/Makefile
index 3ae8b45bfc61..9256a5b189ee 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -25,7 +25,7 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
 
 hostprogs-y := gen_init_cpio
 ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
-			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
+			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),$(srctree)/$(src)/default_cpio_list)
 ramfs-args  := \
         $(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \
         $(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID))
diff --git a/usr/default_cpio_list b/usr/default_cpio_list
new file mode 100644
index 000000000000..37b3864066e8
--- /dev/null
+++ b/usr/default_cpio_list
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# This is a very simple, default initramfs
+
+dir /dev 0755 0 0
+nod /dev/console 0600 0 0 c 5 1
+dir /root 0700 0 0
diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index 49a4e22147b5..e6808a8c3b2b 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -28,7 +28,6 @@ $0 [-o <file>] [-l <dep_list>] [-u <uid>] [-g <gid>] {-d | <cpio_source>} ...
 	<cpio_source>  File list or directory for cpio archive.
 		       If <cpio_source> is a .cpio file it will be used
 		       as direct input to initramfs.
-	-d             Output the default cpio list.
 
 All options except -o and -l may be repeated and are interpreted
 sequentially and immediately.  -u and -g states are preserved across
@@ -43,18 +42,6 @@ field() {
 	shift $1 ; echo $1
 }
 
-default_initramfs() {
-	cat <<-EOF >> ${output}
-		# This is a very simple, default initramfs
-
-		dir /dev 0755 0 0
-		nod /dev/console 0600 0 0 c 5 1
-		dir /root 0700 0 0
-		# file /kinit usr/kinit/kinit 0755 0 0
-		# slink /init kinit 0755 0 0
-	EOF
-}
-
 filetype() {
 	local argv1="$1"
 
@@ -263,9 +250,6 @@ while [ $# -gt 0 ]; do
 			[ "$root_gid" = "-1" ] && root_gid=$(id -g || echo 0)
 			shift
 			;;
-		"-d")	# display default initramfs list
-			default_initramfs
-			;;
 		"-h")
 			usage
 			exit 0
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 11/13] gen_initramfs.sh: always output cpio even without -o option
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (9 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 10/13] initramfs: add default_cpio_list, and delete -d option support Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 12/13] initramfs: refactor the initramfs build rules Masahiro Yamada
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

Currently, this script outputs a cpio file when -o option is
given, but otherwise a text file in the format recognized by
gen_init_cpio.

This behavior is unclear. Make it always output a cpio file.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/gen_initramfs.sh | 58 +++++++++++++++++++++-----------------------
 1 file changed, 27 insertions(+), 31 deletions(-)

diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index e6808a8c3b2b..1efb87bda545 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -6,7 +6,6 @@
 #
 # Generate a cpio packed initramfs. It uses gen_init_cpio to generate
 # the cpio archive, and then compresses it.
-# The script may also be used to generate the inputfile used for gen_init_cpio
 # This script assumes that gen_init_cpio is located in usr/ directory
 
 # error out on errors
@@ -71,8 +70,8 @@ print_mtime() {
 		my_mtime=$(find "$1" -printf "%T@\n" | sort -r | head -n 1)
 	fi
 
-	echo "# Last modified: ${my_mtime}" >> ${output}
-	echo "" >> ${output}
+	echo "# Last modified: ${my_mtime}" >> $cpio_list
+	echo "" >> $cpio_list
 }
 
 list_parse() {
@@ -125,7 +124,7 @@ parse() {
 			;;
 	esac
 
-	echo "${str}" >> ${output}
+	echo "${str}" >> $cpio_list
 
 	return 0
 }
@@ -141,7 +140,7 @@ unknown_option() {
 }
 
 header() {
-	printf "\n#####################\n# $1\n" >> ${output}
+	printf "\n#####################\n# $1\n" >> $cpio_list
 }
 
 # process one directory (incl sub-directories)
@@ -177,8 +176,8 @@ input_file() {
 			[ -n "$dep_list" ] && echo "$1" >> $dep_list
 			return 0
 		fi
-		print_mtime "$1" >> ${output}
-		cat "$1"         >> ${output}
+		print_mtime "$1" >> $cpio_list
+		cat "$1"         >> $cpio_list
 		if [ -n "$dep_list" ]; then
 		        echo "$1 \\"  >> $dep_list
 			cat "$1" | while read type dir file perm ; do
@@ -200,9 +199,9 @@ root_uid=0
 root_gid=0
 dep_list=
 cpio_file=
-cpio_list=
+cpio_list=$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)
 output="/dev/stdout"
-output_file=""
+output_file="/dev/stdout"
 is_cpio_compressed=
 compr="gzip -n -9 -f"
 
@@ -217,8 +216,7 @@ while [ $# -gt 0 ]; do
 			;;
 		"-o")	# generate compressed cpio image named $1
 			output_file="$1"
-			cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)"
-			output=${cpio_list}
+			output=$cpio_list
 			echo "$output_file" | grep -q "\.gz$" \
 			&& [ -x "`which gzip 2> /dev/null`" ] \
 			&& compr="gzip -n -9 -f"
@@ -269,27 +267,25 @@ done
 
 # If output_file is set we will generate cpio archive and compress it
 # we are careful to delete tmp files
-if [ ! -z ${output_file} ]; then
-	if [ -z ${cpio_file} ]; then
-		timestamp=
-		if test -n "$KBUILD_BUILD_TIMESTAMP"; then
-			timestamp="$(date -d"$KBUILD_BUILD_TIMESTAMP" +%s || :)"
-			if test -n "$timestamp"; then
-				timestamp="-t $timestamp"
-			fi
+if [ -z ${cpio_file} ]; then
+	timestamp=
+	if test -n "$KBUILD_BUILD_TIMESTAMP"; then
+		timestamp="$(date -d"$KBUILD_BUILD_TIMESTAMP" +%s || :)"
+		if test -n "$timestamp"; then
+			timestamp="-t $timestamp"
 		fi
-		cpio_tfile="$(mktemp ${TMPDIR:-/tmp}/cpiofile.XXXXXX)"
-		usr/gen_init_cpio $timestamp ${cpio_list} > ${cpio_tfile}
-	else
-		cpio_tfile=${cpio_file}
-	fi
-	rm ${cpio_list}
-	if [ "${is_cpio_compressed}" = "compressed" ]; then
-		cat ${cpio_tfile} > ${output_file}
-	else
-		(cat ${cpio_tfile} | ${compr}  - > ${output_file}) \
-		|| (rm -f ${output_file} ; false)
 	fi
-	[ -z ${cpio_file} ] && rm ${cpio_tfile}
+	cpio_tfile="$(mktemp ${TMPDIR:-/tmp}/cpiofile.XXXXXX)"
+	usr/gen_init_cpio $timestamp ${cpio_list} > ${cpio_tfile}
+else
+	cpio_tfile=${cpio_file}
+fi
+rm ${cpio_list}
+if [ "${is_cpio_compressed}" = "compressed" ]; then
+	cat ${cpio_tfile} > ${output_file}
+else
+	(cat ${cpio_tfile} | ${compr}  - > ${output_file}) \
+	|| (rm -f ${output_file} ; false)
 fi
+[ -z ${cpio_file} ] && rm ${cpio_tfile}
 exit 0
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 12/13] initramfs: refactor the initramfs build rules
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (10 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 11/13] gen_initramfs.sh: always output cpio even without -o option Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-04 15:02 ` [PATCH v2 13/13] gen_initramfs.sh: remove intermediate cpio_list on errors Masahiro Yamada
  2020-01-21 15:59 ` [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Greg Thelen,
	Mauro Carvalho Chehab, Sam Ravnborg, linux-kernel

Currently, usr/gen_initramfs.sh takes care of all the use-cases:

 [1] generates a cpio file unless CONFIG_INITRAMFS_SOURCE points to
     a single cpio archive

 [2] If CONFIG_INITRAMFS_SOURCE is the path to a cpio archive,
     use it as-is.

 [3] Compress the cpio file according to CONFIG_INITRAMFS_COMPRESSION_*
     unless it is passed a compressed archive.

To simplify the script, move [2] and [3] to usr/Makefile.

If CONFIG_INITRAMFS_SOURCE is the path to a cpio archive, there is
no need to run this shell script.

For the cpio archive compression, you can re-use the rules from
scripts/Makefile.lib .

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2:
  - improve the comments so that they match to the code
  - use $(strip ...) instead of $(words ...)
    for the case CONFIG_INITRAMFS_SOURCE is empty

 usr/.gitignore       |  8 +---
 usr/Kconfig          | 10 -----
 usr/Makefile         | 89 +++++++++++++++++++++++++++++---------------
 usr/gen_initramfs.sh | 77 ++++++++------------------------------
 usr/initramfs_data.S |  5 +--
 5 files changed, 78 insertions(+), 111 deletions(-)

diff --git a/usr/.gitignore b/usr/.gitignore
index be5eae1df7eb..610de736b75e 100644
--- a/usr/.gitignore
+++ b/usr/.gitignore
@@ -1,9 +1,3 @@
-#
-# Generated files
-#
 gen_init_cpio
 initramfs_data.cpio
-initramfs_data.cpio.gz
-initramfs_data.cpio.bz2
-initramfs_data.cpio.lzma
-initramfs_list
+/initramfs_inc_data
diff --git a/usr/Kconfig b/usr/Kconfig
index ab61e81165e0..529caab1a328 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -207,13 +207,3 @@ config INITRAMFS_COMPRESSION_LZ4
 	  by default which could cause a build failure.
 
 endchoice
-
-config INITRAMFS_COMPRESSION
-	string
-	default ""      if INITRAMFS_COMPRESSION_NONE
-	default ".gz"   if INITRAMFS_COMPRESSION_GZIP
-	default ".bz2"  if INITRAMFS_COMPRESSION_BZIP2
-	default ".lzma" if INITRAMFS_COMPRESSION_LZMA
-	default ".xz"   if INITRAMFS_COMPRESSION_XZ
-	default ".lzo"  if INITRAMFS_COMPRESSION_LZO
-	default ".lz4"  if INITRAMFS_COMPRESSION_LZ4
diff --git a/usr/Makefile b/usr/Makefile
index 9256a5b189ee..e03753888ad4 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -3,55 +3,86 @@
 # kbuild file for usr/ - including initramfs image
 #
 
-suffix_y = $(subst $\",,$(CONFIG_INITRAMFS_COMPRESSION))
-datafile_y = initramfs_data.cpio$(suffix_y)
-datafile_d_y = .$(datafile_y).d
-AFLAGS_initramfs_data.o += -DINITRAMFS_IMAGE="usr/$(datafile_y)"
+# cmd_bzip2, cmd_lzma, cmd_lzo, cmd_lz4 from scripts/Makefile.lib appends the
+# size at the end of the compressed file, which unfortunately does not work
+# with unpack_to_rootfs(). Make size_append no-op.
+override size_append := :
 
-# clean rules do not have CONFIG_INITRAMFS_COMPRESSION.  So clean up after all
-# possible compression formats.
-clean-files += initramfs_data.cpio*
+compress-$(CONFIG_INITRAMFS_COMPRESSION_NONE)	:= shipped
+compress-$(CONFIG_INITRAMFS_COMPRESSION_GZIP)	:= gzip
+compress-$(CONFIG_INITRAMFS_COMPRESSION_BZIP2)	:= bzip2
+compress-$(CONFIG_INITRAMFS_COMPRESSION_LZMA)	:= lzma
+compress-$(CONFIG_INITRAMFS_COMPRESSION_XZ)	:= xzmisc
+compress-$(CONFIG_INITRAMFS_COMPRESSION_LZO)	:= lzo
+compress-$(CONFIG_INITRAMFS_COMPRESSION_LZ4)	:= lz4
 
-# Generate builtin.o based on initramfs_data.o
 obj-$(CONFIG_BLK_DEV_INITRD) := initramfs_data.o
 
-# initramfs_data.o contains the compressed initramfs_data.cpio image.
-# The image is included using .incbin, a dependency which is not
-# tracked automatically.
-$(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
+$(obj)/initramfs_data.o: $(obj)/initramfs_inc_data
 
-#####
-# Generate the initramfs cpio archive
+ramfs-input := $(strip $(shell echo $(CONFIG_INITRAMFS_SOURCE)))
+cpio-data :=
+
+# If CONFIG_INITRAMFS_SOURCE is empty, generate a small initramfs with the
+# default contents.
+ifeq ($(ramfs-input),)
+ramfs-input := $(srctree)/$(src)/default_cpio_list
+endif
+
+ifeq ($(words $(ramfs-input)),1)
+
+# If CONFIG_INITRAMFS_SOURCE specifies a single file, and it is suffixed with
+# .cpio, use it directly as an initramfs.
+ifneq ($(filter %.cpio,$(ramfs-input)),)
+cpio-data := $(ramfs-input)
+endif
+
+# If CONFIG_INITRAMFS_SOURCE specifies a single file, and it is suffixed with
+# .cpio.*, use it directly as an initramfs, and avoid double compression.
+ifeq ($(words $(subst .cpio.,$(space),$(ramfs-input))),2)
+cpio-data := $(ramfs-input)
+compress-y := shipped
+endif
+
+endif
+
+# For other cases, generate the initramfs cpio archive based on the contents
+# specified by CONFIG_INITRAMFS_SOURCE.
+ifeq ($(cpio-data),)
+
+cpio-data := $(obj)/initramfs_data.cpio
 
 hostprogs-y := gen_init_cpio
-ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
-			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),$(srctree)/$(src)/default_cpio_list)
-ramfs-args  := \
-        $(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \
-        $(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID))
 
-# $(datafile_d_y) is used to identify all files included
+# .initramfs_data.cpio.d is used to identify all files included
 # in initramfs and to detect if any files are added/removed.
 # Removed files are identified by directory timestamp being updated
 # The dependency list is generated by gen_initramfs.sh -l
-ifneq ($(wildcard $(obj)/$(datafile_d_y)),)
-	include $(obj)/$(datafile_d_y)
-endif
-
-quiet_cmd_initfs = GEN     $@
-      cmd_initfs = $< -o $@ -l $(obj)/$(datafile_d_y) $(ramfs-args) $(ramfs-input)
-
-targets := $(datafile_y)
+-include $(obj)/.initramfs_data.cpio.d
 
 # do not try to update files included in initramfs
 $(deps_initramfs): ;
 
+quiet_cmd_initfs = GEN     $@
+      cmd_initfs = \
+	$< -o $@ -l $(obj)/.initramfs_data.cpio.d \
+	$(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \
+	$(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID)) \
+	$(ramfs-input)
+
 # We rebuild initramfs_data.cpio if:
 # 1) Any included file is newer than initramfs_data.cpio
 # 2) There are changes in which files are included (added or deleted)
 # 3) If gen_init_cpio are newer than initramfs_data.cpio
 # 4) Arguments to gen_initramfs.sh changes
-$(obj)/$(datafile_y): $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
+$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE
 	$(call if_changed,initfs)
 
+endif
+
+$(obj)/initramfs_inc_data: $(cpio-data) FORCE
+	$(call if_changed,$(compress-y))
+
+targets += initramfs_data.cpio initramfs_inc_data
+
 subdir-$(CONFIG_UAPI_HEADER_TEST) += include
diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index 1efb87bda545..4e6715f8ff7f 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -5,7 +5,7 @@
 # Released under the terms of the GNU GPL
 #
 # Generate a cpio packed initramfs. It uses gen_init_cpio to generate
-# the cpio archive, and then compresses it.
+# the cpio archive.
 # This script assumes that gen_init_cpio is located in usr/ directory
 
 # error out on errors
@@ -15,8 +15,7 @@ usage() {
 cat << EOF
 Usage:
 $0 [-o <file>] [-l <dep_list>] [-u <uid>] [-g <gid>] {-d | <cpio_source>} ...
-	-o <file>      Create compressed initramfs file named <file> using
-		       gen_init_cpio and compressor depending on the extension
+	-o <file>      Create initramfs file named <file> by using gen_init_cpio
 	-l <dep_list>  Create dependency list named <dep_list>
 	-u <uid>       User ID to map to user ID 0 (root).
 		       <uid> is only meaningful if <cpio_source> is a
@@ -162,20 +161,12 @@ dir_filelist() {
 	fi
 }
 
-# if only one file is specified and it is .cpio file then use it direct as fs
-# if a directory is specified then add all files in given direcotry to fs
-# if a regular file is specified assume it is in gen_init_cpio format
 input_file() {
 	source="$1"
 	if [ -f "$1" ]; then
+		# If a regular file is specified, assume it is in
+		# gen_init_cpio format
 		header "$1"
-		is_cpio="$(echo "$1" | sed 's/^.*\.cpio\(\..*\)\{0,1\}/cpio/')"
-		if [ $2 -eq 0 -a ${is_cpio} = "cpio" ]; then
-			cpio_file=$1
-			echo "$1" | grep -q '^.*\.cpio\..*' && is_cpio_compressed="compressed"
-			[ -n "$dep_list" ] && echo "$1" >> $dep_list
-			return 0
-		fi
 		print_mtime "$1" >> $cpio_list
 		cat "$1"         >> $cpio_list
 		if [ -n "$dep_list" ]; then
@@ -187,6 +178,7 @@ input_file() {
 			done
 		fi
 	elif [ -d "$1" ]; then
+		# If a directory is specified then add all files in it to fs
 		dir_filelist "$1"
 	else
 		echo "  ${prog}: Cannot open '$1'" >&2
@@ -198,12 +190,8 @@ prog=$0
 root_uid=0
 root_gid=0
 dep_list=
-cpio_file=
 cpio_list=$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)
 output="/dev/stdout"
-output_file="/dev/stdout"
-is_cpio_compressed=
-compr="gzip -n -9 -f"
 
 while [ $# -gt 0 ]; do
 	arg="$1"
@@ -214,28 +202,8 @@ while [ $# -gt 0 ]; do
 			echo "deps_initramfs := \\" > $dep_list
 			shift
 			;;
-		"-o")	# generate compressed cpio image named $1
-			output_file="$1"
-			output=$cpio_list
-			echo "$output_file" | grep -q "\.gz$" \
-			&& [ -x "`which gzip 2> /dev/null`" ] \
-			&& compr="gzip -n -9 -f"
-			echo "$output_file" | grep -q "\.bz2$" \
-			&& [ -x "`which bzip2 2> /dev/null`" ] \
-			&& compr="bzip2 -9 -f"
-			echo "$output_file" | grep -q "\.lzma$" \
-			&& [ -x "`which lzma 2> /dev/null`" ] \
-			&& compr="lzma -9 -f"
-			echo "$output_file" | grep -q "\.xz$" \
-			&& [ -x "`which xz 2> /dev/null`" ] \
-			&& compr="xz --check=crc32 --lzma2=dict=1MiB"
-			echo "$output_file" | grep -q "\.lzo$" \
-			&& [ -x "`which lzop 2> /dev/null`" ] \
-			&& compr="lzop -9 -f"
-			echo "$output_file" | grep -q "\.lz4$" \
-			&& [ -x "`which lz4 2> /dev/null`" ] \
-			&& compr="lz4 -l -9 -f"
-			echo "$output_file" | grep -q "\.cpio$" && compr="cat"
+		"-o")	# generate cpio image named $1
+			output="$1"
 			shift
 			;;
 		"-u")	# map $1 to uid=0 (root)
@@ -258,34 +226,21 @@ while [ $# -gt 0 ]; do
 					unknown_option
 					;;
 				*)	# input file/dir - process it
-					input_file "$arg" "$#"
+					input_file "$arg"
 					;;
 			esac
 			;;
 	esac
 done
 
-# If output_file is set we will generate cpio archive and compress it
+# If output_file is set we will generate cpio archive
 # we are careful to delete tmp files
-if [ -z ${cpio_file} ]; then
-	timestamp=
-	if test -n "$KBUILD_BUILD_TIMESTAMP"; then
-		timestamp="$(date -d"$KBUILD_BUILD_TIMESTAMP" +%s || :)"
-		if test -n "$timestamp"; then
-			timestamp="-t $timestamp"
-		fi
+timestamp=
+if test -n "$KBUILD_BUILD_TIMESTAMP"; then
+	timestamp="$(date -d"$KBUILD_BUILD_TIMESTAMP" +%s || :)"
+	if test -n "$timestamp"; then
+		timestamp="-t $timestamp"
 	fi
-	cpio_tfile="$(mktemp ${TMPDIR:-/tmp}/cpiofile.XXXXXX)"
-	usr/gen_init_cpio $timestamp ${cpio_list} > ${cpio_tfile}
-else
-	cpio_tfile=${cpio_file}
-fi
-rm ${cpio_list}
-if [ "${is_cpio_compressed}" = "compressed" ]; then
-	cat ${cpio_tfile} > ${output_file}
-else
-	(cat ${cpio_tfile} | ${compr}  - > ${output_file}) \
-	|| (rm -f ${output_file} ; false)
 fi
-[ -z ${cpio_file} ] && rm ${cpio_tfile}
-exit 0
+usr/gen_init_cpio $timestamp $cpio_list > $output
+rm $cpio_list
diff --git a/usr/initramfs_data.S b/usr/initramfs_data.S
index d07648f05bbf..cd67edc38797 100644
--- a/usr/initramfs_data.S
+++ b/usr/initramfs_data.S
@@ -22,12 +22,9 @@
   in the ELF header, as required by certain architectures.
 */
 
-#include <linux/stringify.h>
-#include <asm-generic/vmlinux.lds.h>
-
 .section .init.ramfs,"a"
 __irf_start:
-.incbin __stringify(INITRAMFS_IMAGE)
+.incbin "usr/initramfs_inc_data"
 __irf_end:
 .section .init.ramfs.info,"a"
 .globl __initramfs_size
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [PATCH v2 13/13] gen_initramfs.sh: remove intermediate cpio_list on errors
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (11 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 12/13] initramfs: refactor the initramfs build rules Masahiro Yamada
@ 2020-01-04 15:02 ` Masahiro Yamada
  2020-01-21 15:59 ` [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-04 15:02 UTC (permalink / raw)
  To: linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

This script sets the -e option, so it exits on any error, in which
case it exits without cleaning up the intermediate cpio_list.

Make sure to delete it on exit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v2: None

 usr/gen_initramfs.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/usr/gen_initramfs.sh b/usr/gen_initramfs.sh
index 4e6715f8ff7f..8ae831657e5d 100755
--- a/usr/gen_initramfs.sh
+++ b/usr/gen_initramfs.sh
@@ -193,6 +193,8 @@ dep_list=
 cpio_list=$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)
 output="/dev/stdout"
 
+trap "rm -f $cpio_list" EXIT
+
 while [ $# -gt 0 ]; do
 	arg="$1"
 	shift
@@ -243,4 +245,3 @@ if test -n "$KBUILD_BUILD_TIMESTAMP"; then
 	fi
 fi
 usr/gen_init_cpio $timestamp $cpio_list > $output
-rm $cpio_list
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE
  2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
@ 2020-01-06  7:33   ` Greg Thelen
  2020-01-07 10:46   ` Masahiro Yamada
  1 sibling, 0 replies; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:33 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Sam Ravnborg, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> 'klibcdirs' was added by commit d39a206bc35d ("kbuild: rebuild initramfs
> if content of initramfs changes"). If this is just a matter of forcing
> execution of the recipe line, we can replace it with FORCE.
>
> I do not understand the purpose of
>
>    $(deps_initramfs): klibcdirs
>
> Remove it.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Reviewed-by: Greg Thelen <gthelen@google.com>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list'
  2020-01-04 15:02 ` [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list' Masahiro Yamada
@ 2020-01-06  7:34   ` Greg Thelen
  0 siblings, 0 replies; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:34 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> This is assigned, but not referenced.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Reviewed-by: Greg Thelen <gthelen@google.com>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment
  2020-01-04 15:02 ` [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment Masahiro Yamada
@ 2020-01-06  7:34   ` Greg Thelen
  0 siblings, 0 replies; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:34 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild; +Cc: Masahiro Yamada, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> There is no tool named "gen_initramfs". The correct name is
> "gen_init_cpio".
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Reviewed-by: Greg Thelen <gthelen@google.com>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  2020-01-04 15:02 ` [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh Masahiro Yamada
@ 2020-01-06  7:43   ` Greg Thelen
  2020-01-06 10:26     ` Masahiro Yamada
  0 siblings, 1 reply; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:43 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Sam Ravnborg, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> The comments in usr/Makefile wrongly refer to the script name (twice).
>
> Line 37:
>     # The dependency list is generated by gen_initramfs.sh -l
>
> Line 54:
>     # 4) Arguments to gen_initramfs.sh changes
>
> There does not exist such a script.
>
> I was going to fix the comments, but after some consideration, I thought
> "gen_initramfs.sh" would be more suitable than "gen_initramfs_list.sh"
> because it generates an initramfs image in the common usage.
>
> The script generates a list that can be fed to gen_init_cpio only when
> it is directly run without -o or -l option.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v2: None
>
>  usr/Makefile                                    | 2 +-
>  usr/{gen_initramfs_list.sh => gen_initramfs.sh} | 0
>  2 files changed, 1 insertion(+), 1 deletion(-)
>  rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (100%)

Will this break klibc?  It might have a ref to the old name.
https://git.kernel.org/pub/scm/libs/klibc/klibc.git/tree/usr/Kbuild#n55

> diff --git a/usr/Makefile b/usr/Makefile
> index 55c942da01cd..e44a66b8c051 100644
> --- a/usr/Makefile
> +++ b/usr/Makefile
> @@ -24,7 +24,7 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
>  # Generate the initramfs cpio archive
>  
>  hostprogs-y := gen_init_cpio
> -initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs_list.sh
> +initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs.sh
>  ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
>  			$(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
>  ramfs-args  := \
> diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs.sh
> similarity index 100%
> rename from usr/gen_initramfs_list.sh
> rename to usr/gen_initramfs.sh

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD
  2020-01-04 15:02 ` [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD Masahiro Yamada
@ 2020-01-06  7:44   ` Greg Thelen
  0 siblings, 0 replies; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:44 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Masahiro Yamada, Mauro Carvalho Chehab, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> init/Kconfig includes usr/Kconfig inside the "if BLK_DEV_INITRD" ...
> "endif" block:
>
>     if BLK_DEV_INITRD
>
>     source "usr/Kconfig"
>
>     endif
>
> Hence, all the defines in usr/Kconfig depend on BLK_DEV_INITRD.
>
> Remove the redundant "depends on BLK_DEV_INITRD".
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Reviewed-by: Greg Thelen <gthelen@google.com>


> ---
>
> Changes in v2: None
>
>  usr/Kconfig | 6 ------
>  1 file changed, 6 deletions(-)
>
> diff --git a/usr/Kconfig b/usr/Kconfig
> index a6b68503d177..72f50a21c051 100644
> --- a/usr/Kconfig
> +++ b/usr/Kconfig
> @@ -54,7 +54,6 @@ config INITRAMFS_ROOT_GID
>  
>  config RD_GZIP
>  	bool "Support initial ramdisk/ramfs compressed using gzip"
> -	depends on BLK_DEV_INITRD
>  	default y
>  	select DECOMPRESS_GZIP
>  	help
> @@ -64,7 +63,6 @@ config RD_GZIP
>  config RD_BZIP2
>  	bool "Support initial ramdisk/ramfs compressed using bzip2"
>  	default y
> -	depends on BLK_DEV_INITRD
>  	select DECOMPRESS_BZIP2
>  	help
>  	  Support loading of a bzip2 encoded initial ramdisk or cpio buffer
> @@ -73,7 +71,6 @@ config RD_BZIP2
>  config RD_LZMA
>  	bool "Support initial ramdisk/ramfs compressed using LZMA"
>  	default y
> -	depends on BLK_DEV_INITRD
>  	select DECOMPRESS_LZMA
>  	help
>  	  Support loading of a LZMA encoded initial ramdisk or cpio buffer
> @@ -81,7 +78,6 @@ config RD_LZMA
>  
>  config RD_XZ
>  	bool "Support initial ramdisk/ramfs compressed using XZ"
> -	depends on BLK_DEV_INITRD
>  	default y
>  	select DECOMPRESS_XZ
>  	help
> @@ -91,7 +87,6 @@ config RD_XZ
>  config RD_LZO
>  	bool "Support initial ramdisk/ramfs compressed using LZO"
>  	default y
> -	depends on BLK_DEV_INITRD
>  	select DECOMPRESS_LZO
>  	help
>  	  Support loading of a LZO encoded initial ramdisk or cpio buffer
> @@ -100,7 +95,6 @@ config RD_LZO
>  config RD_LZ4
>  	bool "Support initial ramdisk/ramfs compressed using LZ4"
>  	default y
> -	depends on BLK_DEV_INITRD
>  	select DECOMPRESS_LZ4
>  	help
>  	  Support loading of a LZ4 encoded initial ramdisk or cpio buffer

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile
  2020-01-04 15:02 ` [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile Masahiro Yamada
@ 2020-01-06  7:51   ` Greg Thelen
  0 siblings, 0 replies; 26+ messages in thread
From: Greg Thelen @ 2020-01-06  7:51 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Masahiro Yamada, Andrew Morton, Sam Ravnborg, linux-kernel

Masahiro Yamada <masahiroy@kernel.org> wrote:

> Specify the dependency directly in the Makefile.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Reviewed-by: Greg Thelen <gthelen@google.com>

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  2020-01-06  7:43   ` Greg Thelen
@ 2020-01-06 10:26     ` Masahiro Yamada
  2020-01-07  3:02       ` Ben Hutchings
  2020-01-07  3:21       ` hpa
  0 siblings, 2 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-06 10:26 UTC (permalink / raw)
  To: Greg Thelen, Ben Hutchings, H. Peter Anvin
  Cc: Linux Kbuild mailing list, Andrew Morton, Sam Ravnborg,
	Linux Kernel Mailing List

(+CC Ben Hutchings, H. Peter Anvin)

In my understanding, the klibc build system is standalone.
So, the change in Linux kernel does not affect klibc at all.
Only the depending part is UAPI headers (make headers_install).

So, this patch
(https://lore.kernel.org/patchwork/patch/1175336/)
should be OK.

Please correct me if I am wrong.


On Mon, Jan 6, 2020 at 4:43 PM Greg Thelen <gthelen@google.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> > The comments in usr/Makefile wrongly refer to the script name (twice).
> >
> > Line 37:
> >     # The dependency list is generated by gen_initramfs.sh -l
> >
> > Line 54:
> >     # 4) Arguments to gen_initramfs.sh changes
> >
> > There does not exist such a script.
> >
> > I was going to fix the comments, but after some consideration, I thought
> > "gen_initramfs.sh" would be more suitable than "gen_initramfs_list.sh"
> > because it generates an initramfs image in the common usage.
> >
> > The script generates a list that can be fed to gen_init_cpio only when
> > it is directly run without -o or -l option.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> > Changes in v2: None
> >
> >  usr/Makefile                                    | 2 +-
> >  usr/{gen_initramfs_list.sh => gen_initramfs.sh} | 0
> >  2 files changed, 1 insertion(+), 1 deletion(-)
> >  rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (100%)
>
> Will this break klibc?  It might have a ref to the old name.
> https://git.kernel.org/pub/scm/libs/klibc/klibc.git/tree/usr/Kbuild#n55

I do not think so.

As I stated above, the klibc build system is independent of
any script in the Linux kernel.

The klibc Makefile refers to
scripts/gen_initramfs_list.sh, which does not exist.

My path is renaming
usr/gen_initramfs_list.sh to usr/gen_initramfs.sh


If the renaming had been problematic for klibc,
commit f6f57a46435d7253a52a1a07a58183678ad266a0
("initramfs: move gen_initramfs_list.sh from scripts/ to usr/")
would have already caused a problem.



Ben, Hans,
Is usr/Kbuild in klibc used?
If it is not used, is it better to delete it to avoid confusion?


Masahiro Yamada




> > diff --git a/usr/Makefile b/usr/Makefile
> > index 55c942da01cd..e44a66b8c051 100644
> > --- a/usr/Makefile
> > +++ b/usr/Makefile
> > @@ -24,7 +24,7 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y) FORCE
> >  # Generate the initramfs cpio archive
> >
> >  hostprogs-y := gen_init_cpio
> > -initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs_list.sh
> > +initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs.sh
> >  ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
> >                       $(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
> >  ramfs-args  := \
> > diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs.sh
> > similarity index 100%
> > rename from usr/gen_initramfs_list.sh
> > rename to usr/gen_initramfs.sh



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  2020-01-06 10:26     ` Masahiro Yamada
@ 2020-01-07  3:02       ` Ben Hutchings
  2020-01-07  3:21       ` hpa
  1 sibling, 0 replies; 26+ messages in thread
From: Ben Hutchings @ 2020-01-07  3:02 UTC (permalink / raw)
  To: Masahiro Yamada, Greg Thelen, H. Peter Anvin
  Cc: Linux Kbuild mailing list, Andrew Morton, Sam Ravnborg,
	Linux Kernel Mailing List

[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]

On Mon, 2020-01-06 at 19:26 +0900, Masahiro Yamada wrote:
> (+CC Ben Hutchings, H. Peter Anvin)
> 
> In my understanding, the klibc build system is standalone.
> So, the change in Linux kernel does not affect klibc at all.
> Only the depending part is UAPI headers (make headers_install).
> 
> So, this patch
> (https://lore.kernel.org/patchwork/patch/1175336/)
> should be OK.
> 
> Please correct me if I am wrong.

I think you're right.

[...]
> I do not think so.
> 
> As I stated above, the klibc build system is independent of
> any script in the Linux kernel.
> 
> The klibc Makefile refers to
> scripts/gen_initramfs_list.sh, which does not exist.
[...]

Right.  I believe the original intent was to include klibc in the
kernel tree, so this would have existed because of that.

There's no sign of that in the current klibc.git, but I think it might
have gone through "git filter-branch" at some point after it was
decided not to include it upstream.  In any case, this was already
broken for as long as klibc has been separate.

Ben.

-- 
Ben Hutchings
Larkinson's Law: All laws are basically false.



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
  2020-01-06 10:26     ` Masahiro Yamada
  2020-01-07  3:02       ` Ben Hutchings
@ 2020-01-07  3:21       ` hpa
  1 sibling, 0 replies; 26+ messages in thread
From: hpa @ 2020-01-07  3:21 UTC (permalink / raw)
  To: Masahiro Yamada, Greg Thelen, Ben Hutchings
  Cc: Linux Kbuild mailing list, Andrew Morton, Sam Ravnborg,
	Linux Kernel Mailing List

On January 6, 2020 2:26:22 AM PST, Masahiro Yamada <masahiroy@kernel.org> wrote:
>(+CC Ben Hutchings, H. Peter Anvin)
>
>In my understanding, the klibc build system is standalone.
>So, the change in Linux kernel does not affect klibc at all.
>Only the depending part is UAPI headers (make headers_install).
>
>So, this patch
>(https://lore.kernel.org/patchwork/patch/1175336/)
>should be OK.
>
>Please correct me if I am wrong.
>
>
>On Mon, Jan 6, 2020 at 4:43 PM Greg Thelen <gthelen@google.com> wrote:
>>
>> Masahiro Yamada <masahiroy@kernel.org> wrote:
>>
>> > The comments in usr/Makefile wrongly refer to the script name
>(twice).
>> >
>> > Line 37:
>> >     # The dependency list is generated by gen_initramfs.sh -l
>> >
>> > Line 54:
>> >     # 4) Arguments to gen_initramfs.sh changes
>> >
>> > There does not exist such a script.
>> >
>> > I was going to fix the comments, but after some consideration, I
>thought
>> > "gen_initramfs.sh" would be more suitable than
>"gen_initramfs_list.sh"
>> > because it generates an initramfs image in the common usage.
>> >
>> > The script generates a list that can be fed to gen_init_cpio only
>when
>> > it is directly run without -o or -l option.
>> >
>> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>> > ---
>> >
>> > Changes in v2: None
>> >
>> >  usr/Makefile                                    | 2 +-
>> >  usr/{gen_initramfs_list.sh => gen_initramfs.sh} | 0
>> >  2 files changed, 1 insertion(+), 1 deletion(-)
>> >  rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (100%)
>>
>> Will this break klibc?  It might have a ref to the old name.
>>
>https://git.kernel.org/pub/scm/libs/klibc/klibc.git/tree/usr/Kbuild#n55
>
>I do not think so.
>
>As I stated above, the klibc build system is independent of
>any script in the Linux kernel.
>
>The klibc Makefile refers to
>scripts/gen_initramfs_list.sh, which does not exist.
>
>My path is renaming
>usr/gen_initramfs_list.sh to usr/gen_initramfs.sh
>
>
>If the renaming had been problematic for klibc,
>commit f6f57a46435d7253a52a1a07a58183678ad266a0
>("initramfs: move gen_initramfs_list.sh from scripts/ to usr/")
>would have already caused a problem.
>
>
>
>Ben, Hans,
>Is usr/Kbuild in klibc used?
>If it is not used, is it better to delete it to avoid confusion?
>
>
>Masahiro Yamada
>
>
>
>
>> > diff --git a/usr/Makefile b/usr/Makefile
>> > index 55c942da01cd..e44a66b8c051 100644
>> > --- a/usr/Makefile
>> > +++ b/usr/Makefile
>> > @@ -24,7 +24,7 @@ $(obj)/initramfs_data.o: $(obj)/$(datafile_y)
>FORCE
>> >  # Generate the initramfs cpio archive
>> >
>> >  hostprogs-y := gen_init_cpio
>> > -initramfs   := $(CONFIG_SHELL)
>$(srctree)/$(src)/gen_initramfs_list.sh
>> > +initramfs   := $(CONFIG_SHELL) $(srctree)/$(src)/gen_initramfs.sh
>> >  ramfs-input := $(if $(filter-out "",$(CONFIG_INITRAMFS_SOURCE)), \
>> >                       $(shell echo $(CONFIG_INITRAMFS_SOURCE)),-d)
>> >  ramfs-args  := \
>> > diff --git a/usr/gen_initramfs_list.sh b/usr/gen_initramfs.sh
>> > similarity index 100%
>> > rename from usr/gen_initramfs_list.sh
>> > rename to usr/gen_initramfs.sh

That is correct.

The klibc integration work was mainly fine in an actual kernel tree, which pulled the klibc tree.

It has obviously bitrotted somewhat, but it wouldn't be all that hard to resurrect it if there is ever interest.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE
  2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
  2020-01-06  7:33   ` Greg Thelen
@ 2020-01-07 10:46   ` Masahiro Yamada
  2020-01-07 17:13     ` hpa
  1 sibling, 1 reply; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-07 10:46 UTC (permalink / raw)
  To: Linux Kbuild mailing list
  Cc: Andrew Morton, Greg Thelen, Sam Ravnborg,
	Linux Kernel Mailing List, Ben Hutchings, H. Peter Anvin

(+CC: Ben Hutchings, H. Peter Anvin)

On Sun, Jan 5, 2020 at 12:03 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> 'klibcdirs' was added by commit d39a206bc35d ("kbuild: rebuild initramfs
> if content of initramfs changes"). If this is just a matter of forcing
> execution of the recipe line, we can replace it with FORCE.
>
> I do not understand the purpose of
>
>    $(deps_initramfs): klibcdirs


Perhaps, the 'klibcdirs' target might be intended
to control the directory descending
in case klibc is dropped in the kernel tree.

Anyway, klibc is built independently
of Linux kernel, and this 'klibcdirs' target
is just a no-op stub as far as the kernel tree is concerned.

Clean it up.


> Remove it.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v2:
>   - New patch (I forgot to submit this in v1 series)
>
>  usr/Makefile | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
>
> diff --git a/usr/Makefile b/usr/Makefile
> index e6f7cb2f81db..55c942da01cd 100644
> --- a/usr/Makefile
> +++ b/usr/Makefile
> @@ -3,9 +3,6 @@
>  # kbuild file for usr/ - including initramfs image
>  #
>
> -klibcdirs:;
> -PHONY += klibcdirs
> -
>  suffix_y = $(subst $\",,$(CONFIG_INITRAMFS_COMPRESSION))
>  datafile_y = initramfs_data.cpio$(suffix_y)
>  datafile_d_y = .$(datafile_y).d
> @@ -50,13 +47,12 @@ targets := $(datafile_y)
>  # do not try to update files included in initramfs
>  $(deps_initramfs): ;
>
> -$(deps_initramfs): klibcdirs
>  # We rebuild initramfs_data.cpio if:
>  # 1) Any included file is newer than initramfs_data.cpio
>  # 2) There are changes in which files are included (added or deleted)
>  # 3) If gen_init_cpio are newer than initramfs_data.cpio
>  # 4) Arguments to gen_initramfs.sh changes
> -$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
> +$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) FORCE
>         $(Q)$(initramfs) -l $(ramfs-input) > $(obj)/$(datafile_d_y)
>         $(call if_changed,initfs)
>
> --
> 2.17.1
>


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE
  2020-01-07 10:46   ` Masahiro Yamada
@ 2020-01-07 17:13     ` hpa
  0 siblings, 0 replies; 26+ messages in thread
From: hpa @ 2020-01-07 17:13 UTC (permalink / raw)
  To: Masahiro Yamada, Linux Kbuild mailing list
  Cc: Andrew Morton, Greg Thelen, Sam Ravnborg,
	Linux Kernel Mailing List, Ben Hutchings

On January 7, 2020 2:46:29 AM PST, Masahiro Yamada <masahiroy@kernel.org> wrote:
>(+CC: Ben Hutchings, H. Peter Anvin)
>
>On Sun, Jan 5, 2020 at 12:03 AM Masahiro Yamada <masahiroy@kernel.org>
>wrote:
>>
>> 'klibcdirs' was added by commit d39a206bc35d ("kbuild: rebuild
>initramfs
>> if content of initramfs changes"). If this is just a matter of
>forcing
>> execution of the recipe line, we can replace it with FORCE.
>>
>> I do not understand the purpose of
>>
>>    $(deps_initramfs): klibcdirs
>
>
>Perhaps, the 'klibcdirs' target might be intended
>to control the directory descending
>in case klibc is dropped in the kernel tree.
>
>Anyway, klibc is built independently
>of Linux kernel, and this 'klibcdirs' target
>is just a no-op stub as far as the kernel tree is concerned.
>
>Clean it up.
>
>
>> Remove it.
>>
>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>> ---
>>
>> Changes in v2:
>>   - New patch (I forgot to submit this in v1 series)
>>
>>  usr/Makefile | 6 +-----
>>  1 file changed, 1 insertion(+), 5 deletions(-)
>>
>> diff --git a/usr/Makefile b/usr/Makefile
>> index e6f7cb2f81db..55c942da01cd 100644
>> --- a/usr/Makefile
>> +++ b/usr/Makefile
>> @@ -3,9 +3,6 @@
>>  # kbuild file for usr/ - including initramfs image
>>  #
>>
>> -klibcdirs:;
>> -PHONY += klibcdirs
>> -
>>  suffix_y = $(subst $\",,$(CONFIG_INITRAMFS_COMPRESSION))
>>  datafile_y = initramfs_data.cpio$(suffix_y)
>>  datafile_d_y = .$(datafile_y).d
>> @@ -50,13 +47,12 @@ targets := $(datafile_y)
>>  # do not try to update files included in initramfs
>>  $(deps_initramfs): ;
>>
>> -$(deps_initramfs): klibcdirs
>>  # We rebuild initramfs_data.cpio if:
>>  # 1) Any included file is newer than initramfs_data.cpio
>>  # 2) There are changes in which files are included (added or
>deleted)
>>  # 3) If gen_init_cpio are newer than initramfs_data.cpio
>>  # 4) Arguments to gen_initramfs.sh changes
>> -$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs)
>klibcdirs
>> +$(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) FORCE
>>         $(Q)$(initramfs) -l $(ramfs-input) > $(obj)/$(datafile_d_y)
>>         $(call if_changed,initfs)
>>
>> --
>> 2.17.1
>>

Yes, it is/was a hook for the klibc integration tree.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [PATCH v2 00/13] initramfs: a lot of cleanups
  2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
                   ` (12 preceding siblings ...)
  2020-01-04 15:02 ` [PATCH v2 13/13] gen_initramfs.sh: remove intermediate cpio_list on errors Masahiro Yamada
@ 2020-01-21 15:59 ` Masahiro Yamada
  13 siblings, 0 replies; 26+ messages in thread
From: Masahiro Yamada @ 2020-01-21 15:59 UTC (permalink / raw)
  To: Linux Kbuild mailing list
  Cc: Andrew Morton, Greg Thelen, Mauro Carvalho Chehab, Sam Ravnborg,
	Linux Kernel Mailing List

On Sun, Jan 5, 2020 at 12:03 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
>
>
> Masahiro Yamada (13):
>   initramfs: replace klibcdirs in Makefile with FORCE
>   gen_initramfs_list.sh: remove unused variable 'default_list'
>   gen_initramfs_list.sh: fix the tool name in the comment
>   initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh
>   initramfs: remove redundant dependency on BLK_DEV_INITRD
>   initramfs: make compression options not depend on INITRAMFS_SOURCE
>   initramfs: make initramfs compression choice non-optional
>   initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in
>     Makefile
>   initramfs: generate dependency list and cpio at the same time
>   initramfs: add default_cpio_list, and delete -d option support
>   gen_initramfs.sh: always output cpio even without -o option
>   initramfs: refactor the initramfs build rules
>   gen_initramfs.sh: remove intermediate cpio_list on errors

All, applied to linux-kbuild.


>  usr/.gitignore                                |   8 +-
>  usr/Kconfig                                   |  26 ---
>  usr/Makefile                                  |  97 ++++++----
>  usr/default_cpio_list                         |   6 +
>  ...gen_initramfs_list.sh => gen_initramfs.sh} | 167 +++++-------------
>  usr/initramfs_data.S                          |   5 +-
>  6 files changed, 112 insertions(+), 197 deletions(-)
>  create mode 100644 usr/default_cpio_list
>  rename usr/{gen_initramfs_list.sh => gen_initramfs.sh} (53%)
>
> --
> 2.17.1
>


--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2020-01-21 16:00 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-04 15:02 [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 01/13] initramfs: replace klibcdirs in Makefile with FORCE Masahiro Yamada
2020-01-06  7:33   ` Greg Thelen
2020-01-07 10:46   ` Masahiro Yamada
2020-01-07 17:13     ` hpa
2020-01-04 15:02 ` [PATCH v2 02/13] gen_initramfs_list.sh: remove unused variable 'default_list' Masahiro Yamada
2020-01-06  7:34   ` Greg Thelen
2020-01-04 15:02 ` [PATCH v2 03/13] gen_initramfs_list.sh: fix the tool name in the comment Masahiro Yamada
2020-01-06  7:34   ` Greg Thelen
2020-01-04 15:02 ` [PATCH v2 04/13] initramfs: rename gen_initramfs_list.sh to gen_initramfs.sh Masahiro Yamada
2020-01-06  7:43   ` Greg Thelen
2020-01-06 10:26     ` Masahiro Yamada
2020-01-07  3:02       ` Ben Hutchings
2020-01-07  3:21       ` hpa
2020-01-04 15:02 ` [PATCH v2 05/13] initramfs: remove redundant dependency on BLK_DEV_INITRD Masahiro Yamada
2020-01-06  7:44   ` Greg Thelen
2020-01-04 15:02 ` [PATCH v2 06/13] initramfs: make compression options not depend on INITRAMFS_SOURCE Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 07/13] initramfs: make initramfs compression choice non-optional Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 08/13] initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile Masahiro Yamada
2020-01-06  7:51   ` Greg Thelen
2020-01-04 15:02 ` [PATCH v2 09/13] initramfs: generate dependency list and cpio at the same time Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 10/13] initramfs: add default_cpio_list, and delete -d option support Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 11/13] gen_initramfs.sh: always output cpio even without -o option Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 12/13] initramfs: refactor the initramfs build rules Masahiro Yamada
2020-01-04 15:02 ` [PATCH v2 13/13] gen_initramfs.sh: remove intermediate cpio_list on errors Masahiro Yamada
2020-01-21 15:59 ` [PATCH v2 00/13] initramfs: a lot of cleanups Masahiro Yamada

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).