[Buildroot] [PATCH v2 0/5] Makefile: Support merged and nested defconfigs.

[Buildroot] [PATCH v2 0/5] Makefile: Support merged and nested defconfigs.

[Sam Bobroff]

This is a combination and cleanup of the previous efforts by
Sam Bobroff and Elizabeth Liner to introduce merged defconfigs
and nested defconfig directories respectively.

Merged defconfigs allow us to create defconfig snippets for aspects
that are common across multiple machines and create a combined defconfig
from the snippets.  The support scripts for merged kernel defconfigs is reused.

Nested defconfig directories allow us to organize the defconfig
directory into sub-directories.  A likely use for this is to place
defconfigs into architecture-specific subdirectories.


Changes v1 -> v2:

This is a complete rework of the config merging system, discarding the .merge
files and instead using Makefile fragments included from the configuration
directories that invoke a Make function. This allows much better error handling
as well as a cleaner looking system (somewhat similar to the package .mk
files).

I've also added some documentation to explain what is supported and an example
to show how it's used.  I haven't touched the documentation before and I'm not
familiar with the formatting so I hope I've done it the right way and that it's
useful (feedback would be great). "make manual-html" seems to work at least.


Elizabeth Liner (1):
  Makefile: Support nested config directories

Patrick Williams (3):
  Makefile: Generate %_defconfig recipes from macro.
  Makefile: Add nested config dirs to list-defconfigs
  Makefile: Add merged defconfigs to list-defconfigs.

Sam Bobroff (1):
  Add support for merged defconfigs

 Makefile                             | 58 +++++++++++++++++++++++++++++-----
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 114 insertions(+), 8 deletions(-)
 create mode 100644 docs/manual/merged-defconfigs.txt

-- 
2.1.0

[Buildroot] [PATCH v2 1/5] Add support for merged defconfigs

[Sam Bobroff]

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 Makefile                             | 38 +++++++++++++++++++++++
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+)
 create mode 100644 docs/manual/merged-defconfigs.txt

diff --git a/Makefile b/Makefile
index 78b44c5..9f3fc92 100644
--- a/Makefile
+++ b/Makefile
@@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 .PHONY: defconfig savedefconfig
 
+# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
+# Missing or duplicate matches must be checked for elsewhere.
+define FIND_CONFIG_FILE =
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
+endef
+
+define CONFIG_FILE_LIST =
+$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
+endef
+
+# Check that each config file is found once and only once.
+define CHECK_CONFIG_FILE_LIST =
+$(foreach f,$(1),
+ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
+$$(error ERROR: Missing input file: $(f))
+else
+ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
+$$(error ERROR: Duplicate input file: $(f))
+endif
+endif
+)
+endef
+
+# To be called by configuration fragments (*.mk) to set up defconfigs built
+# by merge_config.sh, via the include that follows:
+define merge_config =
+$(call CHECK_CONFIG_FILE_LIST,$2 $3)
+$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
+	@mkdir $$(CONFIG_DIR)/.merge_config
+	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
+		$(call CONFIG_FILE_LIST,$2 $3)
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
+		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
+	@rm -rf $$(CONFIG_DIR)/.merge_config
+endef
+
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+
 ################################################################################
 #
 # Cleanup and misc junk
diff --git a/docs/manual/adding-board-support.txt b/docs/manual/adding-board-support.txt
index f6d74ae..086a67e 100644
--- a/docs/manual/adding-board-support.txt
+++ b/docs/manual/adding-board-support.txt
@@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file@the
 root of the Buildroot source tree. Move this file into the +configs/+
 directory, and rename it +<boardname>_defconfig+.
 
+It is also possible to create configurations by specifying differences from
+an existing configuration. See xref:merged-defconfigs[].
+
 It is recommended to use as much as possible upstream versions of the
 Linux kernel and bootloaders, and to use as much as possible default
 kernel and bootloader configurations. If they are incorrect for your
diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
index 87a20bd..9abc37e 100644
--- a/docs/manual/appendix.txt
+++ b/docs/manual/appendix.txt
@@ -3,6 +3,7 @@
 
 include::makedev-syntax.txt[]
 include::makeusers-syntax.txt[]
+include::merged-defconfigs.txt[]
 
 
 // Automatically generated lists:
diff --git a/docs/manual/merged-defconfigs.txt b/docs/manual/merged-defconfigs.txt
new file mode 100644
index 0000000..e1f50fd
--- /dev/null
+++ b/docs/manual/merged-defconfigs.txt
@@ -0,0 +1,60 @@
+// -*- mode:doc -*- ;
+// vim: set syntax=asciidoc:
+
+[[merged-defconfigs]]
+== Merged configurations
+
+Buildroot supports the use of the Linux kernel's +merge_config.sh+ script to
+create configurations by merging a set of partial configurations (or overlays)
+on top of a base configuration. This may be useful when there are several
+configurations that are specializations of a single generic configuration.
+
+Files in the +configs/+ directory with the extension
++.mk+ will be read by the top level Makefile and the Make function
++merge_config+ can be used within them to define new merged configurations.
+
+The merge_config function takes three parameters: the first is the name of the
+new configuration (which should end with +_defconfig+), the second is the name
+of the base configuration file and the third is a space separated list of the
+partial configuration files to merge from the +configs/+ directory. All files
+should be specified relative to the +configs/+ directory and the name of the
+Make fragment file should match the name of the configuration it creates.
+
+The new configuration can be selected as if it were a normal configuration, i.e. with
++make foo_defconfig+.
+
+Partial configurations have the same format as full configuration
+files except that they contain only a subset of values. They can be created by
+starting with a normal configuration file and deleting lines as appropriate.
+
+External configuration files from the +$(BR2_EXTERNAL)/configs+ directory are fully
+supported, and merge configurations may freely refer to files from either
+location.
+
+Example:
+
+If configs/foo_defconfig contains:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+-----
+
+And configs/bar_defconfig.mk contains:
+-----
+$(eval $(call merge_config,bar_defconfig,foo_defconfig,b.config))
+-----
+
+And configs/b.config contains:
+-----
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
+
+After running the command:
+-----
+make bar_defconfig
+-----
+
+The resulting .config file would contain:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
-- 
2.1.0

[Buildroot] [PATCH v2 2/5] Makefile: Generate %_defconfig recipes from macro.

[Sam Bobroff]

From: Patrick Williams <patrick@stwcx.xyz>

To reduce duplication in the %_defconfig recipes with $(TOPDIR) and
$(BR2_EXTERNAL) versions, generate these from a macro.  The macro is
now called on a list of directories containing the appropriate ones.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
---
 Makefile | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index 9f3fc92..e78b9e2 100644
--- a/Makefile
+++ b/Makefile
@@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
-		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
+define CREATE_DEFCONFIG_RECIPES
+%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
+		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
+endef
 
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
-		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< \
@@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
 	@rm -rf $$(CONFIG_DIR)/.merge_config
 endef
 
-$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
 
 ################################################################################
 #
-- 
2.1.0

[Buildroot] [PATCH v2 3/5] Makefile: Support nested config directories

[Sam Bobroff]

From: Elizabeth Liner <eliner@us.ibm.com>

Extend the BR2_DEFCONFIG_PATH to support nested config directories.
This allows us to place config files in both .../config and
.../config/<arch>.

Both $(TOPDIR) and $(BR2_EXTERNAL) are supported.

Signed-off-by: Elizabeth Liner <eliner@us.ibm.com>
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index e78b9e2..fd07f8c 100644
--- a/Makefile
+++ b/Makefile
@@ -848,7 +848,8 @@ define CREATE_DEFCONFIG_RECIPES
 		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
 endef
 
-BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+BR2_DEFCONFIG_PATHS=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/))) \
+		$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
 $(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
-- 
2.1.0

[Buildroot] [PATCH v2 4/5] Makefile: Add nested config dirs to list-defconfigs

[Sam Bobroff]

From: Patrick Williams <patrick@stwcx.xyz>

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
---
 Makefile | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index fd07f8c..cd94f52 100644
--- a/Makefile
+++ b/Makefile
@@ -206,6 +206,10 @@ LEGAL_LICENSES_TXT_HOST = $(LEGAL_INFO_DIR)/host-licenses.txt
 LEGAL_WARNINGS = $(LEGAL_INFO_DIR)/.warnings
 LEGAL_REPORT = $(LEGAL_INFO_DIR)/README
 
+BR2_DEFCONFIG_PATHS_LOCAL=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/)))
+BR2_DEFCONFIG_PATHS_USER=$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
+BR2_DEFCONFIG_PATHS=$(BR2_DEFCONFIG_PATHS_LOCAL) $(BR2_DEFCONFIG_PATHS_USER)
+
 BR2_CONFIG = $(CONFIG_DIR)/.config
 
 # Pull in the user's configuration file
@@ -848,8 +852,6 @@ define CREATE_DEFCONFIG_RECIPES
 		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
 endef
 
-BR2_DEFCONFIG_PATHS=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/))) \
-		$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
 $(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@@ -1009,12 +1011,12 @@ help:
 
 list-defconfigs:
 	@echo 'Built-in configs:'
-	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(wildcard $(path)/*_defconfig)))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),)
 	@echo
 	@echo 'User-provided configs:'
-	@$(foreach b, $(sort $(notdir $(wildcard $(BR2_EXTERNAL)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(wildcard $(path)/*_defconfig)))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 endif
 	@echo
-- 
2.1.0

[Buildroot] [PATCH v2 5/5] Makefile: Add merged defconfigs to list-defconfigs.

[Sam Bobroff]

From: Patrick Williams <patrick@stwcx.xyz>

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index cd94f52..461066f 100644
--- a/Makefile
+++ b/Makefile
@@ -1011,12 +1011,12 @@ help:
 
 list-defconfigs:
 	@echo 'Built-in configs:'
-	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(wildcard $(path)/*_defconfig)))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(basename $(wildcard $(path)/*_defconfig $(path)/*_defconfig.mk))))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),)
 	@echo
 	@echo 'User-provided configs:'
-	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(wildcard $(path)/*_defconfig)))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(basename $(wildcard $(path)/*_defconfig $(path)/*_defconfig.mk))))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 endif
 	@echo
-- 
2.1.0

[Buildroot] [PATCH v2 1/5] Add support for merged defconfigs

[Romain Naour]

Hi Sam, All,

Thanks for this new series!

I tested it by merging qemu_arm_vexpress_defconfig as base configuration file
with a arm_toolchain.config and efl.config partial configuration files.
The merged defconfig is named efl_on_arm_defconfig.

I think it's a "must have" for 2016.08.

Some comments bellow.

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :
> Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
> ---
>  Makefile                             | 38 +++++++++++++++++++++++
>  docs/manual/adding-board-support.txt |  3 ++
>  docs/manual/appendix.txt             |  1 +
>  docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
>  4 files changed, 102 insertions(+)
>  create mode 100644 docs/manual/merged-defconfigs.txt

I would be good to split in two commits the Makefile changes and the documentation.

> 
> diff --git a/Makefile b/Makefile
> index 78b44c5..9f3fc92 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  
>  .PHONY: defconfig savedefconfig
>  
> +# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
> +# Missing or duplicate matches must be checked for elsewhere.
> +define FIND_CONFIG_FILE =
> +$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
> +endef
> +
> +define CONFIG_FILE_LIST =
> +$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
> +endef
> +
> +# Check that each config file is found once and only once.
> +define CHECK_CONFIG_FILE_LIST =
> +$(foreach f,$(1),
> +ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
> +$$(error ERROR: Missing input file: $(f))
> +else
> +ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
> +$$(error ERROR: Duplicate input file: $(f))
> +endif
> +endif
> +)
> +endef
> +
> +# To be called by configuration fragments (*.mk) to set up defconfigs built
> +# by merge_config.sh, via the include that follows:
> +define merge_config =
> +$(call CHECK_CONFIG_FILE_LIST,$2 $3)
> +$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
> +	@mkdir $$(CONFIG_DIR)/.merge_config
> +	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
> +		$(call CONFIG_FILE_LIST,$2 $3)
> +	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
> +		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
> +	@rm -rf $$(CONFIG_DIR)/.merge_config
> +endef

There is a problem here with make savedefconfig since BR2_DEFCONFIG =
$$(CONFIG_DIR)/.merge_config/.config and .merge_config directory is removed with
the last command.

I get:
*** Error while saving defconfig to: test/efl-1.17.1-merged/.merge_config/.config

BR2_DEFCONFIG should be set to configs/efl_on_arm_defconfig

I need more time to continue the review and testing.

Best regards,
Romain Naour

> +
> +$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
> +
>  ################################################################################
>  #
>  # Cleanup and misc junk
> diff --git a/docs/manual/adding-board-support.txt b/docs/manual/adding-board-support.txt
> index f6d74ae..086a67e 100644
> --- a/docs/manual/adding-board-support.txt
> +++ b/docs/manual/adding-board-support.txt
> @@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file at the
>  root of the Buildroot source tree. Move this file into the +configs/+
>  directory, and rename it +<boardname>_defconfig+.
>  
> +It is also possible to create configurations by specifying differences from
> +an existing configuration. See xref:merged-defconfigs[].
> +
>  It is recommended to use as much as possible upstream versions of the
>  Linux kernel and bootloaders, and to use as much as possible default
>  kernel and bootloader configurations. If they are incorrect for your
> diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
> index 87a20bd..9abc37e 100644
> --- a/docs/manual/appendix.txt
> +++ b/docs/manual/appendix.txt
> @@ -3,6 +3,7 @@
>  
>  include::makedev-syntax.txt[]
>  include::makeusers-syntax.txt[]
> +include::merged-defconfigs.txt[]
>  
>  
>  // Automatically generated lists:
> diff --git a/docs/manual/merged-defconfigs.txt b/docs/manual/merged-defconfigs.txt
> new file mode 100644
> index 0000000..e1f50fd
> --- /dev/null
> +++ b/docs/manual/merged-defconfigs.txt
> @@ -0,0 +1,60 @@
> +// -*- mode:doc -*- ;
> +// vim: set syntax=asciidoc:
> +
> +[[merged-defconfigs]]
> +== Merged configurations
> +
> +Buildroot supports the use of the Linux kernel's +merge_config.sh+ script to
> +create configurations by merging a set of partial configurations (or overlays)
> +on top of a base configuration. This may be useful when there are several
> +configurations that are specializations of a single generic configuration.
> +
> +Files in the +configs/+ directory with the extension
> ++.mk+ will be read by the top level Makefile and the Make function
> ++merge_config+ can be used within them to define new merged configurations.
> +
> +The merge_config function takes three parameters: the first is the name of the
> +new configuration (which should end with +_defconfig+), the second is the name
> +of the base configuration file and the third is a space separated list of the
> +partial configuration files to merge from the +configs/+ directory. All files
> +should be specified relative to the +configs/+ directory and the name of the
> +Make fragment file should match the name of the configuration it creates.
> +
> +The new configuration can be selected as if it were a normal configuration, i.e. with
> ++make foo_defconfig+.
> +
> +Partial configurations have the same format as full configuration
> +files except that they contain only a subset of values. They can be created by
> +starting with a normal configuration file and deleting lines as appropriate.
> +
> +External configuration files from the +$(BR2_EXTERNAL)/configs+ directory are fully
> +supported, and merge configurations may freely refer to files from either
> +location.
> +
> +Example:
> +
> +If configs/foo_defconfig contains:
> +-----
> +BR2_TARGET_GENERIC_HOSTNAME="buildroot"
> +-----
> +
> +And configs/bar_defconfig.mk contains:
> +-----
> +$(eval $(call merge_config,bar_defconfig,foo_defconfig,b.config))
> +-----
> +
> +And configs/b.config contains:
> +-----
> +BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
> +-----
> +
> +After running the command:
> +-----
> +make bar_defconfig
> +-----
> +
> +The resulting .config file would contain:
> +-----
> +BR2_TARGET_GENERIC_HOSTNAME="buildroot"
> +BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
> +-----
> 

[Buildroot] [PATCH v2 2/5] Makefile: Generate %_defconfig recipes from macro.

[Romain Naour]

Hi Sam, All

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :
> From: Patrick Williams <patrick@stwcx.xyz>
> 
> To reduce duplication in the %_defconfig recipes with $(TOPDIR) and
> $(BR2_EXTERNAL) versions, generate these from a macro.  The macro is
> now called on a list of directories containing the appropriate ones.
> 
> Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

Your Sob is missing here and for the following patches, see
http://nightly.buildroot.org/manual.html#submitting-patches

Best regards,
Romain

> ---
>  Makefile | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 9f3fc92..e78b9e2 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
>  
>  # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
> -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
> -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
> -		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
> +define CREATE_DEFCONFIG_RECIPES
> +%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
> +	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
> +		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
> +endef
>  
> -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
> -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
> -		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
> +BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
> +$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
>  
>  savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@$(COMMON_CONFIG_ENV) $< \
> @@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
>  	@rm -rf $$(CONFIG_DIR)/.merge_config
>  endef
>  
> -$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
> +$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
>  
>  ################################################################################
>  #
> 

[Buildroot] [PATCH v2 1/5] Add support for merged defconfigs

[Sam Bobroff]

On Wed, Jun 22, 2016 at 12:09:14AM +0200, Romain Naour wrote:
> Hi Sam, All,
> 
> Thanks for this new series!
> 
> I tested it by merging qemu_arm_vexpress_defconfig as base configuration file
> with a arm_toolchain.config and efl.config partial configuration files.
> The merged defconfig is named efl_on_arm_defconfig.
> 
> I think it's a "must have" for 2016.08.
> 
> Some comments bellow.
> 
> Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :
> > Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
> > ---
> >  Makefile                             | 38 +++++++++++++++++++++++
> >  docs/manual/adding-board-support.txt |  3 ++
> >  docs/manual/appendix.txt             |  1 +
> >  docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
> >  4 files changed, 102 insertions(+)
> >  create mode 100644 docs/manual/merged-defconfigs.txt
> 
> I would be good to split in two commits the Makefile changes and the documentation.

Will do.

> > diff --git a/Makefile b/Makefile
> > index 78b44c5..9f3fc92 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >  
> >  .PHONY: defconfig savedefconfig
> >  
> > +# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
> > +# Missing or duplicate matches must be checked for elsewhere.
> > +define FIND_CONFIG_FILE =
> > +$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
> > +endef
> > +
> > +define CONFIG_FILE_LIST =
> > +$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
> > +endef
> > +
> > +# Check that each config file is found once and only once.
> > +define CHECK_CONFIG_FILE_LIST =
> > +$(foreach f,$(1),
> > +ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
> > +$$(error ERROR: Missing input file: $(f))
> > +else
> > +ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
> > +$$(error ERROR: Duplicate input file: $(f))
> > +endif
> > +endif
> > +)
> > +endef
> > +
> > +# To be called by configuration fragments (*.mk) to set up defconfigs built
> > +# by merge_config.sh, via the include that follows:
> > +define merge_config =
> > +$(call CHECK_CONFIG_FILE_LIST,$2 $3)
> > +$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
> > +	@mkdir $$(CONFIG_DIR)/.merge_config
> > +	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
> > +		$(call CONFIG_FILE_LIST,$2 $3)
> > +	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
> > +		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
> > +	@rm -rf $$(CONFIG_DIR)/.merge_config
> > +endef
> 
> There is a problem here with make savedefconfig since BR2_DEFCONFIG =
> $$(CONFIG_DIR)/.merge_config/.config and .merge_config directory is removed with
> the last command.
> 
> I get:
> *** Error while saving defconfig to: test/efl-1.17.1-merged/.merge_config/.config
> 
> BR2_DEFCONFIG should be set to configs/efl_on_arm_defconfig

Ah, I'll take a look at that.

> I need more time to continue the review and testing.

Thanks for the review :-) I'll wait for more comments before I send v3.

> Best regards,
> Romain Naour
> 
> > +
> > +$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
> > +
> >  ################################################################################
> >  #
> >  # Cleanup and misc junk
> > diff --git a/docs/manual/adding-board-support.txt b/docs/manual/adding-board-support.txt
> > index f6d74ae..086a67e 100644
> > --- a/docs/manual/adding-board-support.txt
> > +++ b/docs/manual/adding-board-support.txt
> > @@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file at the
> >  root of the Buildroot source tree. Move this file into the +configs/+
> >  directory, and rename it +<boardname>_defconfig+.
> >  
> > +It is also possible to create configurations by specifying differences from
> > +an existing configuration. See xref:merged-defconfigs[].
> > +
> >  It is recommended to use as much as possible upstream versions of the
> >  Linux kernel and bootloaders, and to use as much as possible default
> >  kernel and bootloader configurations. If they are incorrect for your
> > diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
> > index 87a20bd..9abc37e 100644
> > --- a/docs/manual/appendix.txt
> > +++ b/docs/manual/appendix.txt
> > @@ -3,6 +3,7 @@
> >  
> >  include::makedev-syntax.txt[]
> >  include::makeusers-syntax.txt[]
> > +include::merged-defconfigs.txt[]
> >  
> >  
> >  // Automatically generated lists:
> > diff --git a/docs/manual/merged-defconfigs.txt b/docs/manual/merged-defconfigs.txt
> > new file mode 100644
> > index 0000000..e1f50fd
> > --- /dev/null
> > +++ b/docs/manual/merged-defconfigs.txt
> > @@ -0,0 +1,60 @@
> > +// -*- mode:doc -*- ;
> > +// vim: set syntax=asciidoc:
> > +
> > +[[merged-defconfigs]]
> > +== Merged configurations
> > +
> > +Buildroot supports the use of the Linux kernel's +merge_config.sh+ script to
> > +create configurations by merging a set of partial configurations (or overlays)
> > +on top of a base configuration. This may be useful when there are several
> > +configurations that are specializations of a single generic configuration.
> > +
> > +Files in the +configs/+ directory with the extension
> > ++.mk+ will be read by the top level Makefile and the Make function
> > ++merge_config+ can be used within them to define new merged configurations.
> > +
> > +The merge_config function takes three parameters: the first is the name of the
> > +new configuration (which should end with +_defconfig+), the second is the name
> > +of the base configuration file and the third is a space separated list of the
> > +partial configuration files to merge from the +configs/+ directory. All files
> > +should be specified relative to the +configs/+ directory and the name of the
> > +Make fragment file should match the name of the configuration it creates.
> > +
> > +The new configuration can be selected as if it were a normal configuration, i.e. with
> > ++make foo_defconfig+.
> > +
> > +Partial configurations have the same format as full configuration
> > +files except that they contain only a subset of values. They can be created by
> > +starting with a normal configuration file and deleting lines as appropriate.
> > +
> > +External configuration files from the +$(BR2_EXTERNAL)/configs+ directory are fully
> > +supported, and merge configurations may freely refer to files from either
> > +location.
> > +
> > +Example:
> > +
> > +If configs/foo_defconfig contains:
> > +-----
> > +BR2_TARGET_GENERIC_HOSTNAME="buildroot"
> > +-----
> > +
> > +And configs/bar_defconfig.mk contains:
> > +-----
> > +$(eval $(call merge_config,bar_defconfig,foo_defconfig,b.config))
> > +-----
> > +
> > +And configs/b.config contains:
> > +-----
> > +BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
> > +-----
> > +
> > +After running the command:
> > +-----
> > +make bar_defconfig
> > +-----
> > +
> > +The resulting .config file would contain:
> > +-----
> > +BR2_TARGET_GENERIC_HOSTNAME="buildroot"
> > +BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
> > +-----
> > 

[Buildroot] [PATCH v2 2/5] Makefile: Generate %_defconfig recipes from macro.

[Sam Bobroff]

On Wed, Jun 22, 2016 at 12:12:02AM +0200, Romain Naour wrote:
> Hi Sam, All
> 
> Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :
> > From: Patrick Williams <patrick@stwcx.xyz>
> > 
> > To reduce duplication in the %_defconfig recipes with $(TOPDIR) and
> > $(BR2_EXTERNAL) versions, generate these from a macro.  The macro is
> > now called on a list of directories containing the appropriate ones.
> > 
> > Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
> 
> Your Sob is missing here and for the following patches, see
> http://nightly.buildroot.org/manual.html#submitting-patches

Ah, thanks. I'll fix that for the next version.

Cheers,
Sam.

> Best regards,
> Romain
> 
> > ---
> >  Makefile | 15 ++++++++-------
> >  1 file changed, 8 insertions(+), 7 deletions(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index 9f3fc92..e78b9e2 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
> >  
> >  # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
> > -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
> > -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
> > -		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
> > +define CREATE_DEFCONFIG_RECIPES
> > +%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
> > +	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
> > +		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
> > +endef
> >  
> > -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
> > -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
> > -		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
> > +BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
> > +$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
> >  
> >  savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >  	@$(COMMON_CONFIG_ENV) $< \
> > @@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
> >  	@rm -rf $$(CONFIG_DIR)/.merge_config
> >  endef
> >  
> > -$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
> > +$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
> >  
> >  ################################################################################
> >  #
> >