[PATCH 0/6] kbuild: mergeconfig: fix bugs, improve readability, and add generic entry point

[PATCH 0/6] kbuild: mergeconfig: fix bugs, improve readability, and add generic entry point

[Masahiro Yamada]

This series is related to mergeconfig (scripts/kconfig/merge_config.sh):

1/6 and 2/6 fix bugs related to the parallel build. (-j option).
3/6 thru 5/6 are minor clean-ups.
6/6 is a new feature; add a generic entry point of mergeconfig.

The last one is RFC.

Masahiro Yamada (6):
  kbuild: mergeconfig: fix "jobserver unavailable" warning
  kbuild: mergeconfig: move an error check to merge_config.sh
  kbuild: mergeconfig: remove redundant $(objtree)
  merge_config.sh: improve indentation
  merge_config.sh: rename MAKE to RUNMAKE
  kbuild: add generic mergeconfig target, %.config

 scripts/kconfig/Makefile        | 20 +++++++++-----------
 scripts/kconfig/merge_config.sh | 25 ++++++++++++++-----------
 2 files changed, 23 insertions(+), 22 deletions(-)

-- 
1.9.1

[PATCH 1/6] kbuild: mergeconfig: fix "jobserver unavailable" warning

[Masahiro Yamada]

If "make kvmconfig" is run with "-j" option, a warning message,
"jobserver unavailable: using -j1.  Add `+' to parent make rule.",
is displayed.

  $ make -s defconfig
  *** Default configuration is based on 'x86_64_defconfig'
  #
  # configuration written to .config
  #
  $ make -j8 kvmconfig
  Using ./.config as base
  Merging ./arch/x86/configs/kvm_guest.config
    [ snip ]
  #
  # merged configuration written to ./.config (needs make)
  #
  make[2]: warning: jobserver unavailable: using -j1.  Add `+' to
  parent make rule.
  scripts/kconfig/conf --oldconfig Kconfig
    [ snip ]
  #
  # configuration written to .config
  #

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 scripts/kconfig/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 9645c07..fc34f46 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -110,7 +110,7 @@ define mergeconfig
 $(if $(wildcard $(objtree)/.config),, $(error You need an existing .config for this target))
 $(if $(call configfiles,$(1)),, $(error No configuration exists for this target on this architecture))
 $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m -O $(objtree) $(objtree)/.config $(call configfiles,$(1))
-$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
++$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 endef
 
 PHONY += kvmconfig
-- 
1.9.1

[PATCH 2/6] kbuild: mergeconfig: move an error check to merge_config.sh

[Masahiro Yamada]

Currently, "make tinyconfig" does not work with "-j" option.

  $ make mrproper
  $ make -j8 tinyconfig
    HOSTCC  scripts/basic/fixdep
    HOSTCC  scripts/kconfig/conf.o
    SHIPPED scripts/kconfig/zconf.tab.c
    SHIPPED scripts/kconfig/zconf.lex.c
    SHIPPED scripts/kconfig/zconf.hash.c
    HOSTCC  scripts/kconfig/zconf.tab.o
    HOSTLD  scripts/kconfig/conf
  scripts/kconfig/conf --allnoconfig Kconfig
  #
  # configuration written to .config
  #
  scripts/kconfig/Makefile:122: *** You need an existing .config
  for this target.  Stop.
  make: *** [tinyconfig] Error 2

As shown above, "allnoconfig" has created the .config file before
mergeconfig is called, but Make still raises a false alarm because
of some sort of race condition.

We can fix this issue by moving the error check to the shell script.

Anyway, scripts/kconfig/merge_config.sh always requires an existing
.config as a base file.  It is reasonable to check its existence in
the shell script.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 scripts/kconfig/Makefile        | 1 -
 scripts/kconfig/merge_config.sh | 5 +++++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index fc34f46..be0fad4 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -107,7 +107,6 @@ endif
 configfiles=$(wildcard $(srctree)/kernel/configs/$(1).config $(srctree)/arch/$(SRCARCH)/configs/$(1).config)
 
 define mergeconfig
-$(if $(wildcard $(objtree)/.config),, $(error You need an existing .config for this target))
 $(if $(call configfiles,$(1)),, $(error No configuration exists for this target on this architecture))
 $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m -O $(objtree) $(objtree)/.config $(call configfiles,$(1))
 +$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
index 2118af8..88d89b2 100755
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -85,6 +85,11 @@ fi
 INITFILE=$1
 shift;
 
+if [ ! -r "$INITFILE" ]; then
+	echo "The base file '$INITFILE' does not exist.  Exit." >&2
+	exit 1
+fi
+
 MERGE_LIST=$*
 SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
 TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX)
-- 
1.9.1

[PATCH 3/6] kbuild: mergeconfig: remove redundant $(objtree)

[Masahiro Yamada]

Kbuild always runs in $(objtree).  Actually, $(objtree) is always
set to "." by the top-level Makefile.

We can omit "-O $(objtree)" and "$(objtree)/".

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 scripts/kconfig/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index be0fad4..b0e4be2 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -108,7 +108,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$(1).config $(srctree)/arch/$(S
 
 define mergeconfig
 $(if $(call configfiles,$(1)),, $(error No configuration exists for this target on this architecture))
-$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m -O $(objtree) $(objtree)/.config $(call configfiles,$(1))
+$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(call configfiles,$(1))
 +$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 endef
 
-- 
1.9.1

[PATCH 4/6] merge_config.sh: improve indentation

[Masahiro Yamada]

It is true that we do not want to move the code too far to the
right, but something like below is not preferred:

    if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
    echo Value of $CFG is redefined by fragment $MERGE_FILE:
    echo Previous  value: $PREV_VAL
    echo New value:       $NEW_VAL
    echo
    elif [ "$WARNREDUN" = "true" ]; then
    echo Value of $CFG is redundant by fragment $MERGE_FILE:
    fi

To fix this, call "continue" if the "grep" command fails to find the
given CONFIG.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 scripts/kconfig/merge_config.sh | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
index 88d89b2..56584b1 100755
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -103,20 +103,18 @@ for MERGE_FILE in $MERGE_LIST ; do
 	CFG_LIST=$(sed -n "$SED_CONFIG_EXP" $MERGE_FILE)
 
 	for CFG in $CFG_LIST ; do
-		grep -q -w $CFG $TMP_FILE
-		if [ $? -eq 0 ] ; then
-			PREV_VAL=$(grep -w $CFG $TMP_FILE)
-			NEW_VAL=$(grep -w $CFG $MERGE_FILE)
-			if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
+		grep -q -w $CFG $TMP_FILE || continue
+		PREV_VAL=$(grep -w $CFG $TMP_FILE)
+		NEW_VAL=$(grep -w $CFG $MERGE_FILE)
+		if [ "x$PREV_VAL" != "x$NEW_VAL" ] ; then
 			echo Value of $CFG is redefined by fragment $MERGE_FILE:
 			echo Previous  value: $PREV_VAL
 			echo New value:       $NEW_VAL
 			echo
-			elif [ "$WARNREDUN" = "true" ]; then
+		elif [ "$WARNREDUN" = "true" ]; then
 			echo Value of $CFG is redundant by fragment $MERGE_FILE:
-			fi
-			sed -i "/$CFG[ =]/d" $TMP_FILE
 		fi
+		sed -i "/$CFG[ =]/d" $TMP_FILE
 	done
 	cat $MERGE_FILE >> $TMP_FILE
 done
-- 
1.9.1

[PATCH 5/6] merge_config.sh: rename MAKE to RUNMAKE

[Masahiro Yamada]

The variable "MAKE" is used to store the command name that has
invoked the Makefile.  (Actually, it is already set to "make"
if you run this script from a Makefile.)

In this script, however, it is used to determine if Make should be
run or not.  It is not what we usually expect.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

 scripts/kconfig/merge_config.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh
index 56584b1..ec8e203 100755
--- a/scripts/kconfig/merge_config.sh
+++ b/scripts/kconfig/merge_config.sh
@@ -35,7 +35,7 @@ usage() {
 	echo "  -O    dir to put generated output files"
 }
 
-MAKE=true
+RUNMAKE=true
 ALLTARGET=alldefconfig
 WARNREDUN=false
 OUTPUT=.
@@ -48,7 +48,7 @@ while true; do
 		continue
 		;;
 	"-m")
-		MAKE=false
+		RUNMAKE=false
 		shift
 		continue
 		;;
@@ -119,7 +119,7 @@ for MERGE_FILE in $MERGE_LIST ; do
 	cat $MERGE_FILE >> $TMP_FILE
 done
 
-if [ "$MAKE" = "false" ]; then
+if [ "$RUNMAKE" = "false" ]; then
 	cp $TMP_FILE $OUTPUT/.config
 	echo "#"
 	echo "# merged configuration written to $OUTPUT/.config (needs make)"
-- 
1.9.1

[RFC PATCH 6/6] kbuild: add generic mergeconfig target, %.config

[Masahiro Yamada]

"scripts/kconfig/merge_config.sh && make oldconfig" works well
enough for merging local config fragments, but Kbuild currently has
the entry points only for "kvmconfig" and "tinyconfig".

This commit provides the generic target for mergeconfig, so we can
manage our own config fragments easily:
put "foo.config" in arch/$(SRCARCH)/configs/ or kernel/configs/,
and then run "make foo.config".

Now "make kvmconfig" is just a shorthand of "make kvm_guest.config".
Likewise, "make tinyconfig" is equivalent to
"make allnoconfig tiny.config".

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Perhaps, "make foo_mergeconfig" is better than "make foo.config"?


 scripts/kconfig/Makefile | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index b0e4be2..cb2cf54 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -104,21 +104,20 @@ endif
 %_defconfig: $(obj)/conf
 	$(Q)$< --defconfig=arch/$(SRCARCH)/configs/$@ $(Kconfig)
 
-configfiles=$(wildcard $(srctree)/kernel/configs/$(1).config $(srctree)/arch/$(SRCARCH)/configs/$(1).config)
+configfiles=$(wildcard $(srctree)/kernel/configs/$@ $(srctree)/arch/$(SRCARCH)/configs/$@)
 
-define mergeconfig
-$(if $(call configfiles,$(1)),, $(error No configuration exists for this target on this architecture))
-$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(call configfiles,$(1))
-+$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
-endef
+%.config: $(obj)/conf
+	$(if $(call configfiles),, $(error No configuration exists for this target on this architecture))
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(configfiles)
+	+$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 
 PHONY += kvmconfig
-kvmconfig:
-	$(call mergeconfig,kvm_guest)
+kvmconfig: kvm_guest.config
+	@:
 
 PHONY += tinyconfig
-tinyconfig: allnoconfig
-	$(call mergeconfig,tiny)
+tinyconfig:
+	$(Q)$(MAKE) -f $(srctree)/Makefile allnoconfig tiny.config
 
 # Help text used by make help
 help:
-- 
1.9.1

Re: [PATCH 0/6] kbuild: mergeconfig: fix bugs, improve readability, and add generic entry point

[Josh Triplett]

On Fri, Mar 13, 2015 at 03:21:37PM +0900, Masahiro Yamada wrote:
> 
> This series is related to mergeconfig (scripts/kconfig/merge_config.sh):
> 
> 1/6 and 2/6 fix bugs related to the parallel build. (-j option).
> 3/6 thru 5/6 are minor clean-ups.
> 6/6 is a new feature; add a generic entry point of mergeconfig.
> 
> The last one is RFC.
> 
> Masahiro Yamada (6):
>   kbuild: mergeconfig: fix "jobserver unavailable" warning
>   kbuild: mergeconfig: move an error check to merge_config.sh
>   kbuild: mergeconfig: remove redundant $(objtree)
>   merge_config.sh: improve indentation
>   merge_config.sh: rename MAKE to RUNMAKE
>   kbuild: add generic mergeconfig target, %.config

For all six:

Reviewed-by: Josh Triplett <josh@joshtriplett.org>

Re: [PATCH 0/6] kbuild: mergeconfig: fix bugs, improve readability, and add generic entry point

[Darren Hart]

On 3/13/15, 1:27 AM, "Josh Triplett" <josh@joshtriplett.org> wrote:

>On Fri, Mar 13, 2015 at 03:21:37PM +0900, Masahiro Yamada wrote:
>> 
>> This series is related to mergeconfig (scripts/kconfig/merge_config.sh):
>> 
>> 1/6 and 2/6 fix bugs related to the parallel build. (-j option).
>> 3/6 thru 5/6 are minor clean-ups.
>> 6/6 is a new feature; add a generic entry point of mergeconfig.
>> 
>> The last one is RFC.
>> 
>> Masahiro Yamada (6):
>>   kbuild: mergeconfig: fix "jobserver unavailable" warning
>>   kbuild: mergeconfig: move an error check to merge_config.sh
>>   kbuild: mergeconfig: remove redundant $(objtree)
>>   merge_config.sh: improve indentation
>>   merge_config.sh: rename MAKE to RUNMAKE
>>   kbuild: add generic mergeconfig target, %.config

Nice improvements, thank you.

>
>For all six:
>
>Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>

Likewise:

Reviewed-by: Darren Hart <dvhart@linux.intel.com>

-- 
Darren Hart
Intel Open Source Technology Center

Re: [PATCH 0/6] kbuild: mergeconfig: fix bugs, improve readability, and add generic entry point

[Michal Marek]

On 2015-03-13 22:29, Darren Hart wrote:
> On 3/13/15, 1:27 AM, "Josh Triplett" <josh@joshtriplett.org> wrote:
> 
>> On Fri, Mar 13, 2015 at 03:21:37PM +0900, Masahiro Yamada wrote:
>>>
>>> This series is related to mergeconfig (scripts/kconfig/merge_config.sh):
>>>
>>> 1/6 and 2/6 fix bugs related to the parallel build. (-j option).
>>> 3/6 thru 5/6 are minor clean-ups.
>>> 6/6 is a new feature; add a generic entry point of mergeconfig.
>>>
>>> The last one is RFC.
>>>
>>> Masahiro Yamada (6):
>>>   kbuild: mergeconfig: fix "jobserver unavailable" warning
>>>   kbuild: mergeconfig: move an error check to merge_config.sh
>>>   kbuild: mergeconfig: remove redundant $(objtree)
>>>   merge_config.sh: improve indentation
>>>   merge_config.sh: rename MAKE to RUNMAKE
>>>   kbuild: add generic mergeconfig target, %.config
> 
> Nice improvements, thank you.
> 
>>
>> For all six:
>>
>> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>>
> 
> Likewise:
> 
> Reviewed-by: Darren Hart <dvhart@linux.intel.com>

Added to kbuild.git#kconfig.

Michal