[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Yann E. MORIN]

One of the selling points for br2-external is to provide a mean to add
new packages. However, it is not supported that a package be defined by
Buildroot and then redefined in a br2-external tree.

This situation may occur without the user noticing or even willing to
redefine the package, for example:
  - br2-external is first created against a version of Buildroot
  - a package (missing in Buildroot) is added to that br2-external tree
  - upstream Buildroot adds this package
  - user updates to the new Buildroot

In this case, the result in undefined, and we can't make any guarantee
on the result (working or not).

Add a sanity check so that a package redefinition gets caught.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile               | 1 +
 package/pkg-generic.mk | 9 +++++++++
 2 files changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index dd8959f..da78f18 100644
--- a/Makefile
+++ b/Makefile
@@ -336,6 +336,7 @@ unexport O
 GNU_HOST_NAME := $(shell support/gnuconfig/config.guess)
 
 PACKAGES :=
+PACKAGES_ALL :=
 
 # silent mode requested?
 QUIET := $(if $(findstring s,$(filter-out --%,$(MAKEFLAGS))),-q)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index ffef4d3..7f0c2ab 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -341,6 +341,14 @@ endef
 
 define inner-generic-package
 
+# Ensure the package is only declared once, i.e. do not accept that a
+# package be re-defined by a br2-external tree
+ifneq ($(call strip,$(filter $(1),$(PACKAGES_ALL))),)
+$$(error Package '$(1)' defined a second time in '$(pkgdir)'; \
+	previous definition was in '$$($(2)_PKGDIR)')
+endif
+PACKAGES_ALL += $(1)
+
 # Define default values for various package-related variables, if not
 # already defined. For some variables (version, source, site and
 # subdir), if they are undefined, we try to see if a variable without
@@ -351,6 +359,7 @@ define inner-generic-package
 $(2)_TYPE                       =  $(4)
 $(2)_NAME			=  $(1)
 $(2)_RAWNAME			=  $$(patsubst host-%,%,$(1))
+$(2)_PKGDIR			=  $(pkgdir)
 
 # Keep the package version that may contain forward slashes in the _DL_VERSION
 # variable, then replace all forward slashes ('/') by underscores ('_') to
-- 
1.9.1

[Buildroot] [PATCH 02/21 v2] docs/manual: document $(FOO_PKGDIR)

[Yann E. MORIN]

In the manual, add a note that packages can use $(FOO_PKGDIR) to get the
path to the directory containg the .mk and Config.in files, if they need
it to e.g. copy files from there, like startup scripts, systemd units or
any other resource.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 docs/manual/adding-packages-generic.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index 9d6401f..17ada16 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -491,6 +491,9 @@ endef
 
 In the action definitions, you can use the following variables:
 
+* +$(FOO_PKGDIR)+ contains the path to the directory containing the
+  +foo.mk+ and +Config.in+ files.
+
 * +$(@D)+, which contains the directory in which the package source
   code has been uncompressed.
 
-- 
1.9.1

[Buildroot] [PATCH 03/21 v2] core: move pkg-utils.mk to support/

[Yann E. MORIN]

pkg-utils.mk contains various definitions that are used in the package
infrastructures and packages themselves.

However, those definitions can be useful in other parts of Buildroot,
and are already used in a few places that are not related to the package
infrastructure. Also, $(sep) will be needed early in the Makefile when
we eventually support multiple br2-external trees.

Since this file only contains definitions, we can include it anytime.

So, consider that file to no longer be specific to the package infras:
  - move it to support and rename it,
  - move a few similar definitions from the main Makefile to that file.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
---
 Makefile              | 10 ++-----
 package/pkg-utils.mk  | 59 ----------------------------------------
 support/misc/utils.mk | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 76 insertions(+), 67 deletions(-)
 create mode 100644 support/misc/utils.mk

diff --git a/Makefile b/Makefile
index da78f18..60cea32 100644
--- a/Makefile
+++ b/Makefile
@@ -111,14 +111,8 @@ else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
 BR_BUILDING = y
 endif
 
-# Strip quotes and then whitespaces
-qstrip = $(strip $(subst ",,$(1)))
-#"))
-
-# Variables for use in Make constructs
-comma := ,
-empty :=
-space := $(empty) $(empty)
+# Include some helper macros and variables
+include support/misc/utils.mk
 
 ifneq ("$(origin O)", "command line")
 O := output
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
index 44bd2c9..28db481 100644
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -5,43 +5,6 @@
 #
 ################################################################################
 
-# Case conversion macros. This is inspired by the 'up' macro from gmsl
-# (http://gmsl.sf.net). It is optimised very heavily because these macros
-# are used a lot. It is about 5 times faster than forking a shell and tr.
-#
-# The caseconvert-helper creates a definition of the case conversion macro.
-# After expansion by the outer $(eval ), the UPPERCASE macro is defined as:
-# $(strip $(eval __tmp := $(1))  $(eval __tmp := $(subst a,A,$(__tmp))) ... )
-# In other words, every letter is substituted one by one.
-#
-# The caseconvert-helper allows us to create this definition out of the
-# [FROM] and [TO] lists, so we don't need to write down every substition
-# manually. The uses of $ and $$ quoting are chosen in order to do as
-# much expansion as possible up-front.
-#
-# Note that it would be possible to conceive a slightly more optimal
-# implementation that avoids the use of __tmp, but that would be even
-# more unreadable and is not worth the effort.
-
-[FROM] := a b c d e f g h i j k l m n o p q r s t u v w x y z - .
-[TO]   := A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ _
-
-define caseconvert-helper
-$(1) = $$(strip \
-	$$(eval __tmp := $$(1))\
-	$(foreach c, $(2),\
-		$$(eval __tmp := $$(subst $(word 1,$(subst :, ,$c)),$(word 2,$(subst :, ,$c)),$$(__tmp))))\
-	$$(__tmp))
-endef
-
-$(eval $(call caseconvert-helper,UPPERCASE,$(join $(addsuffix :,$([FROM])),$([TO]))))
-$(eval $(call caseconvert-helper,LOWERCASE,$(join $(addsuffix :,$([TO])),$([FROM]))))
-
-# Sanitize macro cleans up generic strings so it can be used as a filename
-# and in rules. Particularly useful for VCS version strings, that can contain
-# slashes, colons (OK in filenames but not in rules), and spaces.
-sanitize = $(subst $(space),_,$(subst :,_,$(subst /,_,$(strip $(1)))))
-
 #
 # Manipulation of .config files based on the Kconfig
 # infrastructure. Used by the BusyBox package, the Linux kernel
@@ -82,28 +45,6 @@ INFLATE.tar  = cat
 # suitable-extractor(filename): returns extractor based on suffix
 suitable-extractor = $(INFLATE$(suffix $(1)))
 
-# MESSAGE Macro -- display a message in bold type
-MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)_NAME) $($(PKG)_VERSION) $(call qstrip,$(1))$(TERM_RESET)"
-TERM_BOLD := $(shell tput smso)
-TERM_RESET := $(shell tput rmso)
-
-# Utility functions for 'find'
-# findfileclauses(filelist) => -name 'X' -o -name 'Y'
-findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
-# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
-finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
-
-# Miscellaneous utility functions
-# notfirstword(wordlist): returns all but the first word in wordlist
-notfirstword = $(wordlist 2,$(words $(1)),$(1))
-
-# Needed for the foreach loops to loop over the list of hooks, so that
-# each hook call is properly separated by a newline.
-define sep
-
-
-endef
-
 # check-deprecated-variable -- throw an error on deprecated variables
 # example:
 #   $(eval $(call check-deprecated-variable,FOO_MAKE_OPT,FOO_MAKE_OPTS))
diff --git a/support/misc/utils.mk b/support/misc/utils.mk
new file mode 100644
index 0000000..7984ea4
--- /dev/null
+++ b/support/misc/utils.mk
@@ -0,0 +1,74 @@
+################################################################################
+#
+# This file contains various utility macros and variables used about
+# everywhere in make constructs.
+#
+################################################################################
+
+# Strip quotes and then whitespaces
+qstrip = $(strip $(subst ",,$(1)))
+#"))
+
+# Variables for use in Make constructs
+comma := ,
+empty :=
+space := $(empty) $(empty)
+
+# Case conversion macros. This is inspired by the 'up' macro from gmsl
+# (http://gmsl.sf.net). It is optimised very heavily because these macros
+# are used a lot. It is about 5 times faster than forking a shell and tr.
+#
+# The caseconvert-helper creates a definition of the case conversion macro.
+# After expansion by the outer $(eval ), the UPPERCASE macro is defined as:
+# $(strip $(eval __tmp := $(1))  $(eval __tmp := $(subst a,A,$(__tmp))) ... )
+# In other words, every letter is substituted one by one.
+#
+# The caseconvert-helper allows us to create this definition out of the
+# [FROM] and [TO] lists, so we don't need to write down every substition
+# manually. The uses of $ and $$ quoting are chosen in order to do as
+# much expansion as possible up-front.
+#
+# Note that it would be possible to conceive a slightly more optimal
+# implementation that avoids the use of __tmp, but that would be even
+# more unreadable and is not worth the effort.
+
+[FROM] := a b c d e f g h i j k l m n o p q r s t u v w x y z - .
+[TO]   := A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ _
+
+define caseconvert-helper
+$(1) = $$(strip \
+	$$(eval __tmp := $$(1))\
+	$(foreach c, $(2),\
+		$$(eval __tmp := $$(subst $(word 1,$(subst :, ,$c)),$(word 2,$(subst :, ,$c)),$$(__tmp))))\
+	$$(__tmp))
+endef
+
+$(eval $(call caseconvert-helper,UPPERCASE,$(join $(addsuffix :,$([FROM])),$([TO]))))
+$(eval $(call caseconvert-helper,LOWERCASE,$(join $(addsuffix :,$([TO])),$([FROM]))))
+
+# Sanitize macro cleans up generic strings so it can be used as a filename
+# and in rules. Particularly useful for VCS version strings, that can contain
+# slashes, colons (OK in filenames but not in rules), and spaces.
+sanitize = $(subst $(space),_,$(subst :,_,$(subst /,_,$(strip $(1)))))
+
+# MESSAGE Macro -- display a message in bold type
+MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)_NAME) $($(PKG)_VERSION) $(call qstrip,$(1))$(TERM_RESET)"
+TERM_BOLD := $(shell tput smso)
+TERM_RESET := $(shell tput rmso)
+
+# Utility functions for 'find'
+# findfileclauses(filelist) => -name 'X' -o -name 'Y'
+findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
+# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
+finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
+
+# Miscellaneous utility functions
+# notfirstword(wordlist): returns all but the first word in wordlist
+notfirstword = $(wordlist 2,$(words $(1)),$(1))
+
+# Needed for the foreach loops to loop over the list of hooks, so that
+# each hook call is properly separated by a newline.
+define sep
+
+
+endef
-- 
1.9.1

[Buildroot] [PATCH 04/21 v2] core: commonalise the bundled and br2-external %_defconfig rules

[Yann E. MORIN]

The code for both cases is exactly the same, and only differs in the
location where defconfig files are looked for.

We use an intermediate macro to generate the corresponding rules,
because directly generating the rules is ugly and needs lots of escaping
and double-dollar-ing for the $(eval ...) and $(foreach ...) calls to
play nicely together.

Furthermore, that will be tremendously useful when we support multiple
br2-external trees.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index 60cea32..052f58a 100644
--- a/Makefile
+++ b/Makefile
@@ -789,13 +789,12 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
-		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
-
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
-		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+define PERCENT_DEFCONFIG
+%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig outputmakefile
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(1)/configs/$$@ \
+		$$< --defconfig=$(1)/configs/$$@ $$(CONFIG_CONFIG_IN)
+endef
+$(eval $(foreach d,$(TOPDIR) $(BR2_EXTERNAL),$(call PERCENT_DEFCONFIG,$(d))$(sep)))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< \
-- 
1.9.1

[Buildroot] [PATCH 05/21 v2] core: remove .br-external on distclean

[Yann E. MORIN]

distclean is supposed to return the current directory, whether in-tree
or out-of-tree, into pristine conditions, which means we should also
forget about any br2-external tree on distclean.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 052f58a..0e25bd1 100644
--- a/Makefile
+++ b/Makefile
@@ -839,7 +839,7 @@ ifeq ($(O),output)
 	rm -rf $(O)
 endif
 	rm -rf $(BR2_CONFIG) $(CONFIG_DIR)/.config.old $(CONFIG_DIR)/..config.tmp \
-		$(CONFIG_DIR)/.auto.deps
+		$(CONFIG_DIR)/.auto.deps $(BR2_EXTERNAL_FILE)
 
 help:
 	@echo 'Cleaning:'
-- 
1.9.1

[Buildroot] [PATCH 06/21 v2] docs/manual: do not override BR2_EXTERNAL

[Yann E. MORIN]

Currently, we forcibly override BR2_EXTERNAL when building the manual,
so as to avoid referring to packages therein from the manual.

However, when generating the lists of packages, we limit ourselves to
scanning for packages in (Buildroot's) TOPDIR and never search in
BR2_EXTERNAL. So, we do not really need to override BR2_EXTERNAL when
generating the manual to achieve the same result.

Furthermore, we're only looking for packages that are defined in the
sub-menu "Target packages", and never anywhere else, and especially not
in the sub-menu "User-provided options".

Finally, we're soon to completely eliminate use of BR2_EXTERNAL in
Kconfig altogether, so that would no longer have any impact anyway.

So, just leave BR2_EXTERNAL alone when generating the manual.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 docs/manual/manual.mk | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index ad9bd90..caf080a 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -8,13 +8,10 @@ MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
 MANUAL_RESOURCES = $(TOPDIR)/docs/images
 
 # Our manual needs to generate lists
-# Packages included in BR2_EXTERNAL are not part of buildroot, so they
-# should not be included in the manual.
 define MANUAL_GEN_LISTS
 	$(Q)$(call MESSAGE,"Updating the manual lists...")
 	$(Q)$(COMMON_CONFIG_ENV) \
 		BR2_DEFCONFIG="" \
-		BR2_EXTERNAL=$(TOPDIR)/support/dummy-external \
 		TOPDIR=$(TOPDIR) \
 		O=$(@D) \
 		python -B $(TOPDIR)/support/scripts/gen-manual-lists.py
-- 
1.9.1

[Buildroot] [PATCH 07/21 v2] doc/asciidoc: add possibility to define document dependencies

[Yann E. MORIN]

Currently, a document can not have dependencies, except for the purely
internal ones (like checking asciidoc version, and presence of dblatex).

For our own manual, this will come in handy when we introduce a
generated kconfig snippet, so we can actually make the manual depend on
that snippet being generated first.

For external documents, it can be used to depend on host-packages if
need be (e.g. a custom host packages that generates specific media files
included in the manual).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
---
 docs/manual/adding-packages-asciidoc.txt | 3 +++
 package/doc-asciidoc.mk                  | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/docs/manual/adding-packages-asciidoc.txt b/docs/manual/adding-packages-asciidoc.txt
index 6e21786..0c10fd1 100644
--- a/docs/manual/adding-packages-asciidoc.txt
+++ b/docs/manual/adding-packages-asciidoc.txt
@@ -63,6 +63,9 @@ information is (assuming the document name is +foo+) :
   to one or more directories containing so-called resources (like CSS or
   images). By default, empty.
 
+* +FOO_DEPENDENCIES+, optional, the list of packages (most probably,
+  host-packages) that must be built before building this document.
+
 There are also additional hooks (see xref:hooks[] for general information
 on hooks), that a document may set to define extra actions to be done at
 various steps:
diff --git a/package/doc-asciidoc.mk b/package/doc-asciidoc.mk
index 6ab5ad2..ede6ed1 100644
--- a/package/doc-asciidoc.mk
+++ b/package/doc-asciidoc.mk
@@ -140,7 +140,7 @@ endef
 ################################################################################
 define ASCIIDOC
 # Single line, because splitting a foreach is not easy...
-$(1)-check-dependencies: asciidoc-check-dependencies
+$(1)-check-dependencies: asciidoc-check-dependencies $$($(2)_DEPENDENCIES)
 	$$(Q)$$(foreach hook,$$($(2)_CHECK_DEPENDENCIES_HOOKS),$$(call $$(hook))$$(sep))
 
 # Single line, because splitting a foreach is not easy...
-- 
1.9.1

[Buildroot] [PATCH 08/21 v2] core: introduce an intermediate rule before the configurators

[Yann E. MORIN]

Currently, all configurators depend on generating the out-of-tree
Makefile wrapper.

In an upcoming patch, we'll need to also generate a kconfig fragment,
so it will have to kick in before we run the configurators.

Introduce a new intermediate "prepare-kconfig" rule, so we can
commonalise the dependencies of the configurators. Move the dependency
on the Makefile wrapper to that new intermediate rule.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
---
 Makefile | 36 +++++++++++++++++++-----------------
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/Makefile b/Makefile
index 0e25bd1..21ad930 100644
--- a/Makefile
+++ b/Makefile
@@ -702,6 +702,8 @@ endif # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 HOSTCFLAGS = $(CFLAGS_FOR_BUILD)
 export HOSTCFLAGS
 
+prepare-kconfig: outputmakefile
+
 $(BUILD_DIR)/buildroot-config/%onf:
 	mkdir -p $(@D)/lxdialog
 	PKG_CONFIG_PATH="$(HOST_PKG_CONFIG_PATH)" $(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)" \
@@ -720,19 +722,19 @@ COMMON_CONFIG_ENV = \
 	BR2_EXTERNAL=$(BR2_EXTERNAL) \
 	SKIP_LEGACY=
 
-xconfig: $(BUILD_DIR)/buildroot-config/qconf outputmakefile
+xconfig: $(BUILD_DIR)/buildroot-config/qconf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-gconfig: $(BUILD_DIR)/buildroot-config/gconf outputmakefile
+gconfig: $(BUILD_DIR)/buildroot-config/gconf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) srctree=$(TOPDIR) $< $(CONFIG_CONFIG_IN)
 
-menuconfig: $(BUILD_DIR)/buildroot-config/mconf outputmakefile
+menuconfig: $(BUILD_DIR)/buildroot-config/mconf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-nconfig: $(BUILD_DIR)/buildroot-config/nconf outputmakefile
+nconfig: $(BUILD_DIR)/buildroot-config/nconf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-config: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+config: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
 # For the config targets that automatically select options, we pass
@@ -740,22 +742,22 @@ config: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 # no values are set for the legacy options so a subsequent oldconfig
 # will query them. Therefore, run an additional olddefconfig.
 
-oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+oldconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< --oldconfig $(CONFIG_CONFIG_IN)
 
-randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+randconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --randconfig $(CONFIG_CONFIG_IN)
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allyesconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --allyesconfig $(CONFIG_CONFIG_IN)
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allnoconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --allnoconfig $(CONFIG_CONFIG_IN)
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+randpackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
 		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
@@ -763,7 +765,7 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
 		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
@@ -771,7 +773,7 @@ allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
 		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
@@ -779,24 +781,24 @@ allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
 
-silentoldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+silentoldconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	$(COMMON_CONFIG_ENV) $< --silentoldconfig $(CONFIG_CONFIG_IN)
 
-olddefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+olddefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN)
 
-defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+defconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
 define PERCENT_DEFCONFIG
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig outputmakefile
+%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig prepare-kconfig
 	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(1)/configs/$$@ \
 		$$< --defconfig=$(1)/configs/$$@ $$(CONFIG_CONFIG_IN)
 endef
 $(eval $(foreach d,$(TOPDIR) $(BR2_EXTERNAL),$(call PERCENT_DEFCONFIG,$(d))$(sep)))
 
-savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+savedefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< \
 		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
 		$(CONFIG_CONFIG_IN)
-- 
1.9.1

[Buildroot] [PATCH 09/21 v2] core: move rule to create basic directories

[Yann E. MORIN]

Some of those directories will be needed even during configuration, like
BUILD_DIR, where we'll store the generated kconfig snippet.

So, move the rule to create them outside the HAVE_CONFIG block.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
---
 Makefile | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/Makefile b/Makefile
index 21ad930..6cc9877 100644
--- a/Makefile
+++ b/Makefile
@@ -205,6 +205,15 @@ LEGAL_LICENSES_TXT_HOST = $(LEGAL_INFO_DIR)/host-licenses.txt
 LEGAL_WARNINGS = $(LEGAL_INFO_DIR)/.warnings
 LEGAL_REPORT = $(LEGAL_INFO_DIR)/README
 
+################################################################################
+#
+# staging and target directories do NOT list these as
+# dependencies anywhere else
+#
+################################################################################
+$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
+	@mkdir -p $@
+
 BR2_CONFIG = $(CONFIG_DIR)/.config
 
 # Pull in the user's configuration file
@@ -435,15 +444,6 @@ world: target-post-image
 	legal-info legal-info-prepare legal-info-clean printvars help \
 	list-defconfigs target-finalize target-post-image source-check
 
-################################################################################
-#
-# staging and target directories do NOT list these as
-# dependencies anywhere else
-#
-################################################################################
-$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
-	@mkdir -p $@
-
 # We make a symlink lib32->lib or lib64->lib as appropriate
 # MIPS64/n32 requires lib32 even though it's a 64-bit arch.
 ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
-- 
1.9.1

[Buildroot] [PATCH 10/21 v2] core: introduce a generated kconfig snippet

[Yann E. MORIN]

Add the infrastructure for adding generated kconfig snippet in the
menuconfig.

For now, the kconfig snippet is generated empty, the recipe for filling
it in will be introduced in sub-sequent patches.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 Config.in             | 6 ++++++
 Makefile              | 7 ++++++-
 docs/manual/manual.mk | 3 +++
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/Config.in b/Config.in
index d795361..a0fb1ea 100644
--- a/Config.in
+++ b/Config.in
@@ -18,6 +18,10 @@ config BR2_EXTERNAL
 	string
 	option env="BR2_EXTERNAL"
 
+config BR2_BUILD_DIR
+	string
+	option env="BUILD_DIR"
+
 # Hidden boolean selected by packages in need of Java in order to build
 # (example: xbmc)
 config BR2_NEEDS_HOST_JAVA
@@ -647,3 +651,5 @@ menu "User-provided options"
 source "$BR2_EXTERNAL/Config.in"
 
 endmenu
+
+source "$BR2_BUILD_DIR/.br2-external.in"
diff --git a/Makefile b/Makefile
index 6cc9877..af9aab4 100644
--- a/Makefile
+++ b/Makefile
@@ -702,7 +702,7 @@ endif # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 HOSTCFLAGS = $(CFLAGS_FOR_BUILD)
 export HOSTCFLAGS
 
-prepare-kconfig: outputmakefile
+prepare-kconfig: outputmakefile $(BUILD_DIR)/.br2-external.in
 
 $(BUILD_DIR)/buildroot-config/%onf:
 	mkdir -p $(@D)/lxdialog
@@ -720,6 +720,7 @@ COMMON_CONFIG_ENV = \
 	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
 	BR2_CONFIG=$(BR2_CONFIG) \
 	BR2_EXTERNAL=$(BR2_EXTERNAL) \
+	BUILD_DIR=$(BUILD_DIR) \
 	SKIP_LEGACY=
 
 xconfig: $(BUILD_DIR)/buildroot-config/qconf prepare-kconfig
@@ -820,6 +821,10 @@ ifeq ($(NEED_WRAPPER),y)
 	$(Q)$(TOPDIR)/support/scripts/mkmakefile $(TOPDIR) $(O)
 endif
 
+.PHONY: $(BUILD_DIR)/.br2-external.in
+$(BUILD_DIR)/.br2-external.in: $(BUILD_DIR)
+	@touch $@
+
 # printvars prints all the variables currently defined in our Makefiles
 printvars:
 	@$(foreach V, \
diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index caf080a..b26f912 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -7,6 +7,9 @@
 MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
 MANUAL_RESOURCES = $(TOPDIR)/docs/images
 
+# Ensure the kconfig snippet is generated (for MANUAL_GEN_LISTS, below):
+MANUAL_DEPENDENCIES += prepare-kconfig
+
 # Our manual needs to generate lists
 define MANUAL_GEN_LISTS
 	$(Q)$(call MESSAGE,"Updating the manual lists...")
-- 
1.9.1

[Buildroot] [PATCH 11/21 v2] docs/manual: prepare-config can be used as a dependency of documents

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
---
 docs/manual/adding-packages-asciidoc.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/manual/adding-packages-asciidoc.txt b/docs/manual/adding-packages-asciidoc.txt
index 0c10fd1..a278d44 100644
--- a/docs/manual/adding-packages-asciidoc.txt
+++ b/docs/manual/adding-packages-asciidoc.txt
@@ -65,6 +65,8 @@ information is (assuming the document name is +foo+) :
 
 * +FOO_DEPENDENCIES+, optional, the list of packages (most probably,
   host-packages) that must be built before building this document.
+  If a hook of your document needs to access the _Kconfig_ structure,
+  you may add +prepare-kconfig+ to the list of dependencies.
 
 There are also additional hooks (see xref:hooks[] for general information
 on hooks), that a document may set to define extra actions to be done at
-- 
1.9.1

[Buildroot] [PATCH 12/21 v2] core: do not hard-code inclusion of br2-external in Kconfig

[Yann E. MORIN]

Move the inclusion of br2-external's Config.in to the generated kconfig
snippet.

This will ultimately allow us to use more than one br2-external tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Config.in | 7 -------
 Makefile  | 8 +++++++-
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/Config.in b/Config.in
index a0fb1ea..512c049 100644
--- a/Config.in
+++ b/Config.in
@@ -645,11 +645,4 @@ source "package/Config.in.host"
 
 source "Config.in.legacy"
 
-menu "User-provided options"
-	depends on BR2_EXTERNAL != "support/dummy-external"
-
-source "$BR2_EXTERNAL/Config.in"
-
-endmenu
-
 source "$BR2_BUILD_DIR/.br2-external.in"
diff --git a/Makefile b/Makefile
index af9aab4..1ee44a6 100644
--- a/Makefile
+++ b/Makefile
@@ -823,7 +823,13 @@ endif
 
 .PHONY: $(BUILD_DIR)/.br2-external.in
 $(BUILD_DIR)/.br2-external.in: $(BUILD_DIR)
-	@touch $@
+	$(Q)( \
+		printf "#\n# Automatically generated file; DO NOT EDIT.\n#\n\n"; \
+		printf 'menu "User-provided options"\n'; \
+		printf '\tdepends on BR2_EXTERNAL != "support/dummy-external"\n\n'; \
+		printf 'source "%s/Config.in"\n\n' $$(cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd); \
+		printf 'endmenu # User-provided options\n'; \
+	) >$@
 
 # printvars prints all the variables currently defined in our Makefiles
 printvars:
-- 
1.9.1

[Buildroot] [PATCH 13/21 v2] core: get rid of our dummy br2-external tree

[Yann E. MORIN]

Now that we generate a kconfig snippet, we can conditionally include the
BR2_EXTERNAL's Config.in only when BR2_EXTERNAL is supplied by the user,
which means our empty/dummy Config.in is no longer useful to us.

As for external.mk, we can also include it only when BR2_EXTERNAL is
supplied by the user, which means our empty/dummy external.mk is no
longer of any use to us.

Ditch both of those files, and:

  - only generate actual content in the Kconfig snippet when we actually
    do have a BR2_EXTERNAL provided by the user (i.e. BR2_EXTERNAL is not
    empty);

  - add a variable that contains the path to the external.mk provided by
    the user, or empty if none, and include the path set in that variable
    (make can 'include' nothing without any problem! ;-) )

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile                           | 18 +++++++-----------
 support/dummy-external/Config.in   |  0
 support/dummy-external/external.mk |  0
 3 files changed, 7 insertions(+), 11 deletions(-)
 delete mode 100644 support/dummy-external/Config.in
 delete mode 100644 support/dummy-external/external.mk

diff --git a/Makefile b/Makefile
index 1ee44a6..80de431 100644
--- a/Makefile
+++ b/Makefile
@@ -152,16 +152,11 @@ $(if $(BASE_DIR),, $(error output directory "$(O)" does not exist))
 # on the command line, therefore the file is re-created every time make is run.
 #
 # When BR2_EXTERNAL is set to an empty value (e.g. explicitly in command
-# line), the .br-external file is removed and we point to
-# support/dummy-external. This makes sure we can unconditionally include the
-# Config.in and external.mk from the BR2_EXTERNAL directory. In this case,
-# override is necessary so the user can clear BR2_EXTERNAL from the command
-# line, but the dummy path is still used internally.
+# line), the .br-external file is removed.
 
 BR2_EXTERNAL_FILE = $(BASE_DIR)/.br-external
 -include $(BR2_EXTERNAL_FILE)
 ifeq ($(BR2_EXTERNAL),)
-  override BR2_EXTERNAL = support/dummy-external
   $(shell rm -f $(BR2_EXTERNAL_FILE))
 else
   _BR2_EXTERNAL = $(shell cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd)
@@ -170,6 +165,7 @@ else
   endif
   override BR2_EXTERNAL := $(_BR2_EXTERNAL)
   $(shell echo BR2_EXTERNAL ?= $(BR2_EXTERNAL) > $(BR2_EXTERNAL_FILE))
+  BR2_EXTERNAL_MK = $(BR2_EXTERNAL)/external.mk
 endif
 
 # To make sure that the environment variable overrides the .config option,
@@ -428,7 +424,8 @@ include boot/common.mk
 include linux/linux.mk
 include fs/common.mk
 
-include $(BR2_EXTERNAL)/external.mk
+# Nothing to include if no BR2_EXTERNAL tree in use
+include $(BR2_EXTERNAL_MK)
 
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR)
@@ -823,13 +820,12 @@ endif
 
 .PHONY: $(BUILD_DIR)/.br2-external.in
 $(BUILD_DIR)/.br2-external.in: $(BUILD_DIR)
-	$(Q)( \
+	$(Q)if [ -n '$(BR2_EXTERNAL)' ]; then \
 		printf "#\n# Automatically generated file; DO NOT EDIT.\n#\n\n"; \
-		printf 'menu "User-provided options"\n'; \
-		printf '\tdepends on BR2_EXTERNAL != "support/dummy-external"\n\n'; \
+		printf 'menu "User-provided options"\n\n'; \
 		printf 'source "%s/Config.in"\n\n' $$(cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd); \
 		printf 'endmenu # User-provided options\n'; \
-	) >$@
+	fi >$@
 
 # printvars prints all the variables currently defined in our Makefiles
 printvars:
diff --git a/support/dummy-external/Config.in b/support/dummy-external/Config.in
deleted file mode 100644
index e69de29..0000000
diff --git a/support/dummy-external/external.mk b/support/dummy-external/external.mk
deleted file mode 100644
index e69de29..0000000
-- 
1.9.1

[Buildroot] [PATCH 14/21 v2] core: introduce per br2-external ID

[Yann E. MORIN]

This unique ID is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(ID), which contains the path to the br2-external tree.

This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.

Also, display the BR2_EXTERNAL_$(ID) and its path as a comment in the
menuconfig.

This will ultimately allow us to support multiple br2-external trees at
once, with that ID (and thus BR2_EXTERNAL_$(ID)) uniquely defining which
br2-external tree is being used.

Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on (second hunk) will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we must un-quote it
before we include the br2-external's makefile (third hunk). That's
unfortunate, but there is no easy way around that as we want the two
variables to be named the same in Makefile and Kconfig (and we can't
ask the user to un-quote that variable himself either), hence the
little dirty trick.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 80de431..f5202ba 100644
--- a/Makefile
+++ b/Makefile
@@ -153,6 +153,9 @@ $(if $(BASE_DIR),, $(error output directory "$(O)" does not exist))
 #
 # When BR2_EXTERNAL is set to an empty value (e.g. explicitly in command
 # line), the .br-external file is removed.
+#
+# If the br2-external tree defines its ID, then export the path in the
+# BR2_EXTERNAL_$(ID) variable.
 
 BR2_EXTERNAL_FILE = $(BASE_DIR)/.br-external
 -include $(BR2_EXTERNAL_FILE)
@@ -165,6 +168,13 @@ else
   endif
   override BR2_EXTERNAL := $(_BR2_EXTERNAL)
   $(shell echo BR2_EXTERNAL ?= $(BR2_EXTERNAL) > $(BR2_EXTERNAL_FILE))
+  ifneq ($(wildcard $(BR2_EXTERNAL)/external.id),)
+    BR2_EXTERNAL_ID := $(shell cat $(BR2_EXTERNAL)/external.id 2>/dev/null)
+    ifeq ($(BR2_EXTERNAL_ID),)
+    $(error $(BR2_EXTERNAL) has no ID (in file 'external.id'))
+    endif
+    BR2_EXTERNAL_$(BR2_EXTERNAL_ID) = $(BR2_EXTERNAL)
+  endif
   BR2_EXTERNAL_MK = $(BR2_EXTERNAL)/external.mk
 endif
 
@@ -424,6 +434,10 @@ include boot/common.mk
 include linux/linux.mk
 include fs/common.mk
 
+# If the br2-external tree defines its ID, then the BR2_EXTERNAL_$(ID)
+# variable is also present in .config, so it is quoted. We must unquote
+# it before feeding it to the br2-external makefile.
+BR2_EXTERNAL_$(BR2_EXTERNAL_ID) := $(call qstrip,$(BR2_EXTERNAL_$(BR2_EXTERNAL_ID)))
 # Nothing to include if no BR2_EXTERNAL tree in use
 include $(BR2_EXTERNAL_MK)
 
@@ -823,7 +837,13 @@ $(BUILD_DIR)/.br2-external.in: $(BUILD_DIR)
 	$(Q)if [ -n '$(BR2_EXTERNAL)' ]; then \
 		printf "#\n# Automatically generated file; DO NOT EDIT.\n#\n\n"; \
 		printf 'menu "User-provided options"\n\n'; \
-		printf 'source "%s/Config.in"\n\n' $$(cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd); \
+		if [ -z "$(BR2_EXTERNAL_ID)" ]; then \
+			printf 'source "%s/Config.in"\n\n' $$(cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd); \
+		else \
+			printf 'config BR2_EXTERNAL_%s\n' $(BR2_EXTERNAL_ID); \
+			printf '\tstring\n\tdefault "%s"\n\n' $(BR2_EXTERNAL_$(BR2_EXTERNAL_ID)); \
+			printf 'source "$$BR2_EXTERNAL_%s/Config.in"\n\n' $(BR2_EXTERNAL_ID); \
+		fi; \
 		printf 'endmenu # User-provided options\n'; \
 	fi >$@
 
-- 
1.9.1

[Buildroot] [PATCH 15/21 v2] core: handle .br-external in a make rule

[Yann E. MORIN]

Currently, we treat the case where we have no br2-external tree
(BR2_EXTERNAL is empty) differently from the case where we do have
one (BR2_EXTERNAL is not empty).

There is now no reason to treat those two cases differently:

  - the kconfig snippet is always generated appropriately (i.e. it would
    include the br2-external tree if set, or include nothing otherwise);

  - we no longer have a dummy br-external tree either.

As such, generate the .br-external cache file in both cases.

This will make it much easy to handle when we introduce support for
multiple br2-external trees.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index f5202ba..d31144b 100644
--- a/Makefile
+++ b/Makefile
@@ -151,23 +151,17 @@ $(if $(BASE_DIR),, $(error output directory "$(O)" does not exist))
 # On subsequent invocations of make, it is read in. It can still be overridden
 # on the command line, therefore the file is re-created every time make is run.
 #
-# When BR2_EXTERNAL is set to an empty value (e.g. explicitly in command
-# line), the .br-external file is removed.
-#
 # If the br2-external tree defines its ID, then export the path in the
 # BR2_EXTERNAL_$(ID) variable.
 
 BR2_EXTERNAL_FILE = $(BASE_DIR)/.br-external
 -include $(BR2_EXTERNAL_FILE)
-ifeq ($(BR2_EXTERNAL),)
-  $(shell rm -f $(BR2_EXTERNAL_FILE))
-else
+ifneq ($(BR2_EXTERNAL),)
   _BR2_EXTERNAL = $(shell cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd)
   ifeq ($(_BR2_EXTERNAL),)
     $(error BR2_EXTERNAL='$(BR2_EXTERNAL)' does not exist, relative to $(TOPDIR))
   endif
   override BR2_EXTERNAL := $(_BR2_EXTERNAL)
-  $(shell echo BR2_EXTERNAL ?= $(BR2_EXTERNAL) > $(BR2_EXTERNAL_FILE))
   ifneq ($(wildcard $(BR2_EXTERNAL)/external.id),)
     BR2_EXTERNAL_ID := $(shell cat $(BR2_EXTERNAL)/external.id 2>/dev/null)
     ifeq ($(BR2_EXTERNAL_ID),)
@@ -178,6 +172,11 @@ else
   BR2_EXTERNAL_MK = $(BR2_EXTERNAL)/external.mk
 endif
 
+# This needs to be *after* we compute BR_EXTERNAL, above.
+.PHONY: $(BR2_EXTERNAL_FILE)
+$(BR2_EXTERNAL_FILE):
+	@echo BR2_EXTERNAL ?= $(BR_EXTERNAL) >$@
+
 # To make sure that the environment variable overrides the .config option,
 # set this before including .config.
 ifneq ($(BR2_DL_DIR),)
-- 
1.9.1

[Buildroot] [PATCH 0/21 v2] br2-external: support multiple trees at once (branch yem/multi-br2-external-5)

[Yann E. MORIN]

Hello All!

This rather complex series introduces support for using multiple
br2-external trees at once.


Currently, Buildroot can use what we call a br2-external tree, that is a
location where one can store customisations outisde of the Buildroot
tree:

  - defconfig files;
  - packages;
  - custom make rules and logic;
  - company-, project- and board-related configurations files, like a
    kernel config file, a rootfs overlay;
  - any other content that does not belong to the Buildroot tree...

This is very nice and very handy!

However, it is not possible to use more than one such br2-external tree
at a time; Buildroot only supports using at most one br2-external tree.


Having support for using more than one br2-external tree at the same
time would allow more flexibility organising those out-of-tree
customisations. One could use a br2-external tree to store FLOSS
packages not yet in Buildroot and a second br2-external tree for
proprietary packages; such a separation would make compliance with
FLOSS licenses easier (by allowing distributing only the br2-external
tree with FLOSS packages, keeping the proprietary bits private).

It could also be used to commonalise br2-external trees from different
internal projects, where each would provide a set of extensions (new
packages, new boards defconfigs...) that a project could cherry-pick.

And countless other possibilities.

This series is an attempt at making it possible to use such setups.


The series is organised as such:

  - patches 1-2 ensure no br2-external tree can redefine a package;
    they introduce a per-package variable that points to the directory
    with the package's .mk and Config.in files; that variable is added
    to the manual;

  - patch 3 is a little cleanup to commonalise utility functions that
    are used throughout the Makefiles;

  - patches 4-10 pave the way to introducing a generated kconfig
    snippet;

  - patches 11-13 and 15 remove hard-wired and conditional handling of
    br2-external, to always handle it in non-conditional Makefile code,
    with only the content of the generated kconfig snippet depending on
    the value of $(BR2_EXTERNAL);

  - patch 14 introduces a per-br2-external variable that holds the path
    to that tree; see below for an in-depth expanation for the reason
    for that ID;

  - patch 16 is the core of the series; it eventually introduces support
    for using multiple br2-external trees at once. Tanks to all the
    previous changes, it is pretty light (67+, 31-), even if a bit
    complex;

  - patch 17 adds some information about a missing ID when a
    br2-external tree is used alone. It is not necessary to keep that
    patch and it can be dropped if that informative message is not
    wanted;

  - patches 18-21 add the necessary documentation about all this new
    multi-br2-external support, and adds packages from br2-external
    trees in the manual.

(Note: patch 15 could/should have been before patch 14, but they were
made in that order and fipping them was a bit difficult; so here they
come as-is.)


The pivotal change in this series is implemented in patch 14, and aims
at providing br2-external trees with a way to find their own files,
given that their locations at runtime is completely unkown, and can
not even be deduced relative to Buildroot's own TOPDIR; this is
explained below.

Unlike Makefiles, Kconfig does not allow us to redefine a variable at
various moments of the parsing, because the values are only valid and
acted upon when the complete Kconfig code has been parsed. So, we need
a way to provide Config.in files from each br2-external with their
actual location. For that, we need different variables, one for each
br2-external tree.

So, we need a way to uniquely identify each br2-external tree, and we
need a stable scheme. We can't use indexes; we can't use path-based
hashing (because paths are not stable accross users/machines/time);
we can't use content-based hashing (external.mk and/or Config.in may
all be empty, giving the same hash); we can't use hashes altogether
because they are not human-friendly.

The solution is to require br2-external trees to provide a unique ID
that is used as a kind of 'name-space' for that tree: we eventually
provide a single variable, BR2_EXTERNAL_$(ID) that points to the
location of that tree; because $(ID) is known to the external tree,
it also knows BR2_EXTERNAL_$(ID) exists, so it can use it, and we
export it both in Config.in and external.mk .

Therefore, br2-external trees must now use $(BR2_EXTERNAL_$(MY_ID)) to
refer to their own files. It also prevent them from (accidentally)
including files from other br2-external trees.

Note: IDs are optional, so that existing br2-external trees continue to
work as before. However, if a br2-external tree does not provide an ID,
it will not be useable in conjunction with other br2-external trees.
Using multiple br2-external trees mandates that all of them provide an
ID.


Please have a look at the various commit logs, they are usually pretty
copious in details (sometimes even much bigger than the actual change
they are commenting!).


And that's about all for this time. ;-)


Changes RFC -> v2:
  - much. :-]


Regards,
Yann E. MORIN.


The following changes since commit 301e8ffbb252f7a6718f9a5610bb27976efe0ac7:

  gst1-plugins-bad: update configure options (2015-10-21 23:36:37 +0200)

are available in the git repository at:

  git://git.busybox.net/~ymorin/git/buildroot yem/multi-br2-external-5

for you to fetch changes up to 4ae06b5c21e26de732cbd8f41870a31721227b74:

  docs/manual: document multi br2-external (2015-10-22 21:46:38 +0200)

----------------------------------------------------------------
Yann E. MORIN (21):
      core: do not accept multiple definitions of a package
      docs/manual: document $(FOO_PKGDIR)
      core: move pkg-utils.mk to support/
      core: commonalise the bundled and br2-external %_defconfig rules
      core: remove .br-external on distclean
      docs/manual: do not override BR2_EXTERNAL
      doc/asciidoc: add possibility to define document dependencies
      core: introduce an intermediate rule before the configurators
      core: move rule to create basic directories
      core: introduce a generated kconfig snippet
      docs/manual: prepare-config can be used as a dependency of documents
      core: do not hard-code inclusion of br2-external in Kconfig
      core: get rid of our dummy br2-external tree
      core: introduce per br2-external ID
      core: handle .br-external in a make rule
      core: add support for multiple br2-external trees
      core: br2-external trees without ID can't be used with other trees
      support/scripts: teach gen-manual-lists about BR2_EXTERNAL
      docs/manual: include packages from BR2_EXTERNAL if set
      docs/manual: document the br2-external ID
      docs/manual: document multi br2-external

 Config.in                                |  11 +-
 Makefile                                 | 185 ++++++++++++++++++++-----------
 docs/manual/adding-packages-asciidoc.txt |   5 +
 docs/manual/adding-packages-generic.txt  |   3 +
 docs/manual/appendix.txt                 |   5 +
 docs/manual/customize-outside-br.txt     | 110 ++++++++++++------
 docs/manual/manual.mk                    |  24 +++-
 package/Makefile.in                      |   2 +-
 package/doc-asciidoc.mk                  |   2 +-
 package/pkg-generic.mk                   |   9 ++
 package/pkg-utils.mk                     |  59 ----------
 support/dummy-external/Config.in         |   0
 support/dummy-external/external.mk       |   0
 support/misc/utils.mk                    |  74 +++++++++++++
 support/scripts/gen-manual-lists.py      |  24 +++-
 15 files changed, 340 insertions(+), 173 deletions(-)
 delete mode 100644 support/dummy-external/Config.in
 delete mode 100644 support/dummy-external/external.mk
 create mode 100644 support/misc/utils.mk

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 16/21 v2] core: add support for multiple br2-external trees

[Yann E. MORIN]

Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.

A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.

Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.

Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.

What we do is to treat BR2_EXTERNAL as a space-separated list of paths,
which we iterate to construct:

  - the list of all br2-external IDs, BR_EXTERNAL_IDS

  - the per-br2-external tree BR2_EXTERNAL_$(ID) variables, which points
    to the actual location of the corresponding tree,

  - EXTRA_ENV which now needs to contain all BR2_EXTERNAL_$(ID)
    variables, and thus needs to be set early.

Once we have all those variables, we replace references to BR2_EXTERNAL
with either:

  - a $(patsubst ...) to include the external.mk files (and docs),
    or

  - a $(foreach ...) iteration to generate other Makefile code.

To be noted: we also use $(foreach ...) in the kconfig-snippet
generating rule, because it is cleaner than doing so in the shell

Finally, we double-quote $(BR2_EXTERNAL) when assigning it to
environment variables.

Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's ID (prefixed with 'BR2_EXTERNAL_') and the
sub-menu's first item is a comment with the path to that br2-external
tree.

If there's only one br2-external tree, then there is no sub-menu, and
the comment with the path is omitted.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Changes v3 -> v4:
  - drop check for file existence  (Thomas)
  - fix check for empty ID  (Thomas)
  - simpler code to check number of br2-external trees
---
 Makefile            | 96 ++++++++++++++++++++++++++++++++++++-----------------
 package/Makefile.in |  2 +-
 2 files changed, 67 insertions(+), 31 deletions(-)

diff --git a/Makefile b/Makefile
index d31144b..141b173 100644
--- a/Makefile
+++ b/Makefile
@@ -150,32 +150,57 @@ $(if $(BASE_DIR),, $(error output directory "$(O)" does not exist))
 # The value of BR2_EXTERNAL is stored in .br-external in the output directory.
 # On subsequent invocations of make, it is read in. It can still be overridden
 # on the command line, therefore the file is re-created every time make is run.
-#
-# If the br2-external tree defines its ID, then export the path in the
-# BR2_EXTERNAL_$(ID) variable.
 
 BR2_EXTERNAL_FILE = $(BASE_DIR)/.br-external
 -include $(BR2_EXTERNAL_FILE)
-ifneq ($(BR2_EXTERNAL),)
-  _BR2_EXTERNAL = $(shell cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd)
-  ifeq ($(_BR2_EXTERNAL),)
-    $(error BR2_EXTERNAL='$(BR2_EXTERNAL)' does not exist, relative to $(TOPDIR))
-  endif
-  override BR2_EXTERNAL := $(_BR2_EXTERNAL)
-  ifneq ($(wildcard $(BR2_EXTERNAL)/external.id),)
-    BR2_EXTERNAL_ID := $(shell cat $(BR2_EXTERNAL)/external.id 2>/dev/null)
-    ifeq ($(BR2_EXTERNAL_ID),)
-    $(error $(BR2_EXTERNAL) has no ID (in file 'external.id'))
-    endif
-    BR2_EXTERNAL_$(BR2_EXTERNAL_ID) = $(BR2_EXTERNAL)
-  endif
-  BR2_EXTERNAL_MK = $(BR2_EXTERNAL)/external.mk
-endif
 
-# This needs to be *after* we compute BR_EXTERNAL, above.
 .PHONY: $(BR2_EXTERNAL_FILE)
 $(BR2_EXTERNAL_FILE):
-	@echo BR2_EXTERNAL ?= $(BR_EXTERNAL) >$@
+	@echo BR2_EXTERNAL ?= $(BR2_EXTERNAL) >$@
+
+# Those two variables need to be defined as simply-expanded variables, they
+# can't be recursively-expanded, because the values they are assigned change
+# with each iteration of the foreach, below.
+BR_EXTERNAL_IDS :=
+EXTRA_ENV :=
+
+# If there is no or one br2-external trees used, then we don't require (but
+# accept) an ID; otherwise (i.e. there are two or more br2-external trees)
+# we require they do all define their ID.
+ifeq ($(filter-out 0 1,$(words $(BR2_EXTERNAL))),)
+BR2_EXTERNAL_NEED_ID := loose
+else
+BR2_EXTERNAL_NEED_ID := strict
+endif
+
+# Validate the br2-external tree passed as $(1):
+# - check the directory actually exists
+# - check if we need and have a non-empty ID
+# - check the ID is not a duplicate
+# - set variables for later use
+define BR2_EXTERNAL_VALIDATE
+  _BR_EXT_DIR := $$(shell cd $(1) >/dev/null 2>&1 && pwd)
+  ifeq ($$(_BR_EXT_DIR),)
+    $$(error BR2_EXTERNAL='$(1)' does not exist, relative to $$(TOPDIR))
+  endif
+  _BR_EXT_ID := $$(shell cat $$(_BR_EXT_DIR)/external.id 2>/dev/null)
+  ifeq ($$(_BR_EXT_ID),)
+    ifeq ($(BR2_EXTERNAL_NEED_ID),strict)
+      $$(error BR2_EXTERNAL='$(1)' has no ID (in file 'external.id'),\
+               mandatory to use more than one br2-external tree at once)
+    endif # BR2_EXTERNAL_NEED_ID strict
+  endif # No ID
+  ifneq ($$(filter $$(_BR_EXT_ID),$$(BR_EXTERNAL_IDS)),)
+    $$(error Duplicate ID '$$(_BR_EXT_ID)' in '$(1)', previously defined in '$$(BR2_EXTERNAL_$$(_BR_EXT_ID))')
+  endif
+  ifneq ($$(_BR_EXT_ID),)
+    BR2_EXTERNAL_$$(_BR_EXT_ID) := $$(_BR_EXT_DIR)
+    BR_EXTERNAL_IDS += $$(_BR_EXT_ID)
+    EXTRA_ENV += BR2_EXTERNAL_$$(_BR_EXT_ID)=$$(_BR_EXT_DIR)
+  endif # _BR_EXT_ID not empty
+endef # BR2_EXTERNAL_VALIDATE
+
+$(eval $(foreach d,$(BR2_EXTERNAL),$(call BR2_EXTERNAL_VALIDATE,$(d))$(sep)))
 
 # To make sure that the environment variable overrides the .config option,
 # set this before including .config.
@@ -436,9 +461,9 @@ include fs/common.mk
 # If the br2-external tree defines its ID, then the BR2_EXTERNAL_$(ID)
 # variable is also present in .config, so it is quoted. We must unquote
 # it before feeding it to the br2-external makefile.
-BR2_EXTERNAL_$(BR2_EXTERNAL_ID) := $(call qstrip,$(BR2_EXTERNAL_$(BR2_EXTERNAL_ID)))
+$(eval $(foreach id,$(BR_EXTERNAL_IDS),BR2_EXTERNAL_$(id) := $(call qstrip,$(BR2_EXTERNAL_$(id)))$(sep)))
 # Nothing to include if no BR2_EXTERNAL tree in use
-include $(BR2_EXTERNAL_MK)
+include $(patsubst %,%/external.mk,$(BR2_EXTERNAL))
 
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR)
@@ -729,7 +754,7 @@ COMMON_CONFIG_ENV = \
 	KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \
 	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
 	BR2_CONFIG=$(BR2_CONFIG) \
-	BR2_EXTERNAL=$(BR2_EXTERNAL) \
+	BR2_EXTERNAL="$(BR2_EXTERNAL)" \
 	BUILD_DIR=$(BUILD_DIR) \
 	SKIP_LEGACY=
 
@@ -833,15 +858,26 @@ endif
 
 .PHONY: $(BUILD_DIR)/.br2-external.in
 $(BUILD_DIR)/.br2-external.in: $(BUILD_DIR)
+	$(Q)echo "  GEN     $@"
 	$(Q)if [ -n '$(BR2_EXTERNAL)' ]; then \
 		printf "#\n# Automatically generated file; DO NOT EDIT.\n#\n\n"; \
 		printf 'menu "User-provided options"\n\n'; \
-		if [ -z "$(BR2_EXTERNAL_ID)" ]; then \
+		if [ -z "$(call strip,$(BR_EXTERNAL_IDS))" ]; then \
 			printf 'source "%s/Config.in"\n\n' $$(cd $(BR2_EXTERNAL) >/dev/null 2>&1 && pwd); \
 		else \
-			printf 'config BR2_EXTERNAL_%s\n' $(BR2_EXTERNAL_ID); \
-			printf '\tstring\n\tdefault "%s"\n\n' $(BR2_EXTERNAL_$(BR2_EXTERNAL_ID)); \
-			printf 'source "$$BR2_EXTERNAL_%s/Config.in"\n\n' $(BR2_EXTERNAL_ID); \
+			$(foreach id,$(BR_EXTERNAL_IDS),\
+				for i in $$(seq 1 80); do printf '#'; done; printf '\n\n'; \
+				if [ $(words $(call strip,$(BR_EXTERNAL_IDS))) -gt 1 ]; then \
+					printf 'menu "BR2_EXTERNAL_%s"\n\n' $(id); \
+					printf 'comment "%s"\n\n' $(BR2_EXTERNAL_$(id)); \
+				fi; \
+				printf 'config BR2_EXTERNAL_%s\n' $(id); \
+				printf '\tstring\n\tdefault "%s"\n\n' $(BR2_EXTERNAL_$(id)); \
+				printf 'source "$$BR2_EXTERNAL_%s/Config.in"\n\n' $(id); \
+				if [ $(words $(call strip,$(BR_EXTERNAL_IDS))) -gt 1 ]; then \
+					printf 'endmenu # BR2_EXTERNAL_%s\n\n' $(id); \
+				fi; ) \
+			for i in $$(seq 1 80); do printf '#'; done; printf '\n\n'; \
 		fi; \
 		printf 'endmenu # User-provided options\n'; \
 	fi >$@
@@ -954,10 +990,10 @@ list-defconfigs:
 	@echo 'Built-in configs:'
 	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
-ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),)
+ifneq ($(wildcard $(patsubst %,%/configs/*_defconfig,$(BR2_EXTERNAL))),)
 	@echo
 	@echo 'User-provided configs:'
-	@$(foreach b, $(sort $(notdir $(wildcard $(BR2_EXTERNAL)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(wildcard $(patsubst %,%/configs/*_defconfig,$(BR2_EXTERNAL))))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 endif
 	@echo
@@ -979,7 +1015,7 @@ print-version:
 	@echo $(BR2_VERSION_FULL)
 
 include docs/manual/manual.mk
--include $(BR2_EXTERNAL)/docs/*/*.mk
+-include $(patsubst %,%/docs/*/*.mk,$(BR2_EXTERNAL))
 
 .PHONY: $(noconfig_targets)
 
diff --git a/package/Makefile.in b/package/Makefile.in
index 8a592d4..8c93b7c 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -316,7 +316,7 @@ EXTRA_ENV = \
 	PATH=$(BR_PATH) \
 	BR2_DL_DIR=$(BR2_DL_DIR) \
 	BUILD_DIR=$(BUILD_DIR) \
-	BR2_EXTERNAL=$(BR2_EXTERNAL)
+	BR2_EXTERNAL="$(BR2_EXTERNAL)"
 
 ################################################################################
 # settings we need to pass to configure
-- 
1.9.1

[Buildroot] [PATCH 17/21 v2] core: br2-external trees without ID can't be used with other trees

[Yann E. MORIN]

Inform users who use a single br2-external tree without an ID that this
br2-external tree will not be useable in conjunction with other
br2-external trees.

This is only an informative message, not even a warning, since we do
want to continue to accept that situation for the forseeable future.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index 141b173..b827323 100644
--- a/Makefile
+++ b/Makefile
@@ -188,6 +188,9 @@ define BR2_EXTERNAL_VALIDATE
     ifeq ($(BR2_EXTERNAL_NEED_ID),strict)
       $$(error BR2_EXTERNAL='$(1)' has no ID (in file 'external.id'),\
                mandatory to use more than one br2-external tree at once)
+    else
+      $$(info BR2_EXTERNAL='$(1)' has no ID (in file 'external.id');\
+              it will not be useable with other br2-external trees.)
     endif # BR2_EXTERNAL_NEED_ID strict
   endif # No ID
   ifneq ($$(filter $$(_BR_EXT_ID),$$(BR_EXTERNAL_IDS)),)
-- 
1.9.1

[Buildroot] [PATCH 18/21 v2] support/scripts: teach gen-manual-lists about BR2_EXTERNAL

[Yann E. MORIN]

When using BR2_EXTERNAL, generate the list of packages from there.

This is currently effectively unused for now, as BR2_EXTERNAL is not
available (not exported in the environment); that will be updated in a
later patch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
---
 support/scripts/gen-manual-lists.py | 24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/support/scripts/gen-manual-lists.py b/support/scripts/gen-manual-lists.py
index d231eda..a672b7b 100644
--- a/support/scripts/gen-manual-lists.py
+++ b/support/scripts/gen-manual-lists.py
@@ -141,7 +141,6 @@ class Buildroot:
 
     """
     root_config = "Config.in"
-    package_dirname = "package"
     package_prefixes = ["BR2_PACKAGE_", "BR2_PACKAGE_HOST_"]
     re_pkg_prefix = re.compile(r"^(" + "|".join(package_prefixes) + ").*")
     deprecated_symbol = "BR2_DEPRECATED"
@@ -182,12 +181,22 @@ class Buildroot:
             'format': "_format_symbol_prompt_location",
             'sorted': False,
         },
+        'external': {
+            'filename': "br2-external-package-list",
+            'root_menu': "User-provided options",
+            'filter': "_is_real_package",
+            'format': "_format_symbol_prompt_location",
+            'sorted': True,
+        },
     }
 
     def __init__(self):
         self.base_dir = os.environ.get("TOPDIR")
         self.output_dir = os.environ.get("O")
-        self.package_dir = os.path.join(self.base_dir, self.package_dirname)
+        self.package_dirs = []
+        self.package_dirs.append(os.path.join(self.base_dir, "package"))
+        if os.getenv('BR2_EXTERNAL'):
+            self.package_dirs.extend(os.environ["BR2_EXTERNAL"].split())
         self.config = kconfiglib.Config(os.path.join(self.base_dir,
                                                      self.root_config),
                                         self.base_dir)
@@ -259,9 +268,10 @@ class Buildroot:
         #   symbol.
         if not hasattr(self, "_package_list"):
             pkg_list = []
-            for _, _, files in os.walk(self.package_dir):
-                for file_ in (f for f in files if f.endswith(".mk")):
-                    pkg_list.append(re.sub(r"(.*?)\.mk", r"\1", file_))
+            for d in self.package_dirs:
+                for _, _, files in os.walk(d):
+                    for file_ in (f for f in files if f.endswith(".mk")):
+                        pkg_list.append(re.sub(r"(.*?)\.mk", r"\1", file_))
             setattr(self, "_package_list", pkg_list)
         for pkg in getattr(self, "_package_list"):
             if type == 'real':
@@ -491,6 +501,8 @@ class Buildroot:
 
 if __name__ == '__main__':
     list_types = ['target-packages', 'host-packages', 'virtual-packages', 'deprecated']
+    if os.getenv('BR2_EXTERNAL'):
+        list_types.append('external')
     parser = ArgumentParser()
     parser.add_argument("list_type", nargs="?", choices=list_types,
                         help="""\
@@ -505,6 +517,8 @@ Generate the given list (generate all lists if unspecified)""")
                         help="Output virtual package file")
     parser.add_argument("--output-deprecated", dest="output_deprecated",
                         help="Output deprecated file")
+    parser.add_argument("--output-external", dest="output_external",
+                        help="Output br2-external file")
     args = parser.parse_args()
     lists = [args.list_type] if args.list_type else list_types
     buildroot = Buildroot()
-- 
1.9.1

[Buildroot] [PATCH 19/21 v2] docs/manual: include packages from BR2_EXTERNAL if set

[Yann E. MORIN]

If the user is using one or more br2-external trees, add the list of
packages therein to the manual.

Note that we do re-introduce BR2_EXTERNAL (when it was removed in
[-SHA1-] (docs/manual: do not override BR2_EXTERNAL). The reason for
the removal is that BR2_EXTERNAL is no longer used as a Kconfig option.

However, BR2_EXTERNAL is now used by the gen-manual-lists.py script,
which needs it to find packages' .mk files in the br2-external trees,
not as a Kconfig option (which it is not).

So, even though that looks like a revert, that's not: the purpose is
different, and we only pass it when not empty.

In case we have no br2-external, we want to create an empty file, so
we can't just touch it, in case it was previously generated with
br2-external trees; instead, we printf an empty string in there to
ensure the file exists and is empty.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
---
 docs/manual/appendix.txt |  5 +++++
 docs/manual/manual.mk    | 18 ++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
index 87a20bd..7ef8bbc 100644
--- a/docs/manual/appendix.txt
+++ b/docs/manual/appendix.txt
@@ -40,3 +40,8 @@ year following deprecation. For example, a symbol depending on
 +BR2_DEPRECATED_SINCE_2013_05+ can be removed from 2014.05 onwards.
 
 include::deprecated-list.txt[]
+
+// This file is generated:
+// - with the list of br2-external packages if BR2_EXTERNAL is set
+// - empty otherwise
+include::br2-external.txt[]
diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index b26f912..2e711b8 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -10,14 +10,32 @@ MANUAL_RESOURCES = $(TOPDIR)/docs/images
 # Ensure the kconfig snippet is generated (for MANUAL_GEN_LISTS, below):
 MANUAL_DEPENDENCIES += prepare-kconfig
 
+# If using one or more BR2_EXTERNAL trees, include the list of
+# packages therein, otherwise, generate an empty file
+ifeq ($(BR2_EXTERNAL),)
+define MANUAL_GEN_LISTS_BR2_EXTERNAL
+	$(Q)printf "" >$(@D)/br2-external.txt
+endef
+else
+define MANUAL_GEN_LISTS_BR2_EXTERNAL
+	$(Q)( \
+		printf "== List of BR2_EXTERNAL packages\n\n"; \
+		printf "include::br2-external-package-list.txt[]\n"; \
+	) >$(@D)/br2-external.txt
+endef
+MANUAL_GEN_LISTS_ENV = BR2_EXTERNAL="$(BR2_EXTERNAL)"
+endif
+
 # Our manual needs to generate lists
 define MANUAL_GEN_LISTS
 	$(Q)$(call MESSAGE,"Updating the manual lists...")
 	$(Q)$(COMMON_CONFIG_ENV) \
+		$(MANUAL_GEN_LISTS_ENV) \
 		BR2_DEFCONFIG="" \
 		TOPDIR=$(TOPDIR) \
 		O=$(@D) \
 		python -B $(TOPDIR)/support/scripts/gen-manual-lists.py
+	$(MANUAL_GEN_LISTS_BR2_EXTERNAL)
 endef
 MANUAL_POST_RSYNC_HOOKS += MANUAL_GEN_LISTS
 
-- 
1.9.1

[Buildroot] [PATCH 20/21 v2] docs/manual: document the br2-external ID

[Yann E. MORIN]

Update the manual with the new external.id mandatory file.

Take the opportunity to add a section listing all mandatory files,
Config.in, external.mk and the new external.id, instead of just hinting
about them in the external package recipes section.

Change the example to use the ID-suffixed variable instead of the raw
BR2_EXTERNAL variable, which won't be useable once we introduce support
for multiple br2-external trees, as it would then contain more that one
path.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 docs/manual/customize-outside-br.txt | 50 ++++++++++++++++++++++++------------
 1 file changed, 34 insertions(+), 16 deletions(-)

diff --git a/docs/manual/customize-outside-br.txt b/docs/manual/customize-outside-br.txt
index 9ad177d..afc5b78 100644
--- a/docs/manual/customize-outside-br.txt
+++ b/docs/manual/customize-outside-br.txt
@@ -57,19 +57,41 @@ Or disable the usage of external definitions:
 buildroot/ $ make BR2_EXTERNAL= xconfig
 -----
 
-+BR2_EXTERNAL+ allows three different things:
+A +BR2_EXTERNAL+ tree must contain at least those three files:
+
++external.id+::
+   That file should contain the ID for the +BR2_EXTERNAL+ tree.
+   That ID is used to construct the +BR2_EXTERNAL_$(ID)+ variable,
+   available in +Config.in+ and +external.mk+ (see below), so that ID
+   must be a valid make and Kconfig variable. Buildroot sets
+   +BR2_EXTERNAL_$(ID)+ to the path of the +BR2_EXTERNAL+ tree.
++
+Examples:
++
+  ** +FOO+ -> +BR2_EXTERNAL_FOO+
+  ** +BAR_42+ -> +BR2_EXTERNAL_BAR_42+
+  ** ...
+
++Config.in+::
++external.mk+::
+   Those files (which may each be empty) can be used to define package
+   recipes, like for packages bundled in Buildroot itself, or other
+   custom configuration options.
+
+Using +BR2_EXTERNAL+ then allows three different things:
 
  * One can store all the board-specific configuration files there,
    such as the kernel configuration, the root filesystem overlay, or
    any other configuration file for which Buildroot allows to set its
-   location. The +BR2_EXTERNAL+ value is available within the
-   Buildroot configuration using +$(BR2_EXTERNAL)+. As an example, one
-   could set the +BR2_ROOTFS_OVERLAY+ Buildroot option to
-   +$(BR2_EXTERNAL)/board/<boardname>/overlay/+ (to specify a root
+   location. The location of the +BR2_EXTERNAL+ tree is available in the
+   Buildroot configuration using the +BR2_EXTERNAL_$(ID)+ variable. As
+   an example, (considering the ID +BAR_42+), one could set the
+   +BR2_ROOTFS_OVERLAY+ Buildroot option to
+   +$(BR2_EXTERNAL_BAR_42)/board/<boardname>/overlay/+ (to specify a root
    filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+
    Buildroot option to
-   +$(BR2_EXTERNAL)/board/<boardname>/kernel.config+ (to specify the
-   location of the kernel configuration file).
+   +$(BR2_EXTERNAL_BAR_42)/board/<boardname>/kernel.config+ (to specify
+   the location of the kernel configuration file).
 
  * One can store package recipes (i.e. +Config.in+ and
    +<packagename>.mk+), or even custom configuration options and make
@@ -77,27 +99,23 @@ buildroot/ $ make BR2_EXTERNAL= xconfig
    make it appear in the top-level configuration menu, and includes
    +$(BR2_EXTERNAL)/external.mk+ with the rest of the makefile logic.
 +
-.Note
-Providing +Config.in+ and +external.mk+ is mandatory, but they can be
-   empty.
-+
 The main usage of this is to store package recipes. The recommended
    way to do this is to write a +$(BR2_EXTERNAL)/Config.in+ file that
    looks like:
 +
 ------
-source "$BR2_EXTERNAL/package/package1/Config.in"
-source "$BR2_EXTERNAL/package/package2/Config.in"
+source "$BR2_EXTERNAL_BAR_42/package/package1/Config.in"
+source "$BR2_EXTERNAL_BAR_42/package/package2/Config.in"
 ------
 +
 Then, have a +$(BR2_EXTERNAL)/external.mk+ file that looks like:
 +
 ------
-include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
+include $(sort $(wildcard $(BR2_EXTERNAL_BAR_42)/package/*/*.mk))
 ------
 +
-And then in +$(BR2_EXTERNAL)/package/package1+ and
-   +$(BR2_EXTERNAL)/package/package2+ create normal Buildroot
+And then in +$(BR2_EXTERNAL_FOO_42)/package/package1+ and
+   +$(BR2_EXTERNAL_FOO_42)/package/package2+ create normal Buildroot
    package recipes, as explained in xref:adding-packages[].
    If you prefer, you can also group the packages in subdirectories
    called <boardname> and adapt the above paths accordingly.
-- 
1.9.1

[Buildroot] [PATCH 21/21 v2] docs/manual: document multi br2-external

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 docs/manual/customize-outside-br.txt | 62 ++++++++++++++++++++++++------------
 1 file changed, 41 insertions(+), 21 deletions(-)

diff --git a/docs/manual/customize-outside-br.txt b/docs/manual/customize-outside-br.txt
index afc5b78..9d5b521 100644
--- a/docs/manual/customize-outside-br.txt
+++ b/docs/manual/customize-outside-br.txt
@@ -17,19 +17,18 @@ place project-specific customizations in two locations:
    having them nicely integrated in the build logic. This section
    explains how to use +BR2_EXTERNAL+.
 
-+BR2_EXTERNAL+ is an environment variable that can be used to point to
-a directory that contains Buildroot customizations. It can be passed
-to any Buildroot +make+ invocation. It is automatically saved in the
-hidden +.br-external+ file in the output directory. Thanks to this,
-there is no need to pass +BR2_EXTERNAL+ at every +make+ invocation. It
-can however be changed at any time by passing a new value, and can be
++BR2_EXTERNAL+ is an environment variable that can be set to contain the
+paths to one or more directories that contains Buildroot customizations.
+It can be passed to any Buildroot +make+ invocation. It is automatically
+saved in the hidden +.br-external+ file in the output directory. Thanks
+to this, there is no need to pass +BR2_EXTERNAL+ at every +make+ invocation.
+It can however be changed at any time by passing a new value, and can be
 removed by passing an empty value.
 
 .Note
-The +BR2_EXTERNAL+ path can be either an absolute or a relative path,
-but if it's passed as a relative path, it is important to note that it
-is interpreted relative to the main Buildroot source directory, *not*
-to the Buildroot output directory.
+Paths in +BR2_EXTERNAL+ can be either absolute or relative paths, but it
+is important to note that relative paths are interpreted relative to the
+main Buildroot source directory, *not* to the Buildroot output directory.
 
 Some examples:
 
@@ -51,6 +50,12 @@ We can switch to another external definitions directory at any time:
 buildroot/ $ make BR2_EXTERNAL=/where/we/have/barfoo xconfig
 -----
 
+We can also use multiple +BR2_EXTERNAL+ locations:
+
+----
+buildroot/ $ make BR2_EXTERNAL="/path/to/foobar /where/we/have/barfoo" menuconfig
+----
+
 Or disable the usage of external definitions:
 
 -----
@@ -60,12 +65,22 @@ buildroot/ $ make BR2_EXTERNAL= xconfig
 A +BR2_EXTERNAL+ tree must contain at least those three files:
 
 +external.id+::
-   That file should contain the ID for the +BR2_EXTERNAL+ tree.
+   That file should contain the ID for that +BR2_EXTERNAL+ tree.
    That ID is used to construct the +BR2_EXTERNAL_$(ID)+ variable,
    available in +Config.in+ and +external.mk+ (see below), so that ID
    must be a valid make and Kconfig variable. Buildroot sets
    +BR2_EXTERNAL_$(ID)+ to the path of the +BR2_EXTERNAL+ tree.
 +
+.Note:
+Since it is possible to use multiple +BR2_EXTERNAL+ trees at once, this
+  ID is used by Buildroot to generate variables for each of those trees.
+  That ID is used to identify your +BR2_EXTERNAL+ tree, so try to come up
+  with an ID that really describes your +BR2_EXTERNAL+ tree, in order for
+  it to be relatively unique, so that it does not clash with another ID
+  from another +BR2_EXTERNAL+ tree, especially if you are planning on
+  somehow sharing your +BR2_EXTERNAL+ tree with third parties or using
+  +BR2_EXTERNAL+ external trees from third parties.
++
 Examples:
 +
   ** +FOO+ -> +BR2_EXTERNAL_FOO+
@@ -93,22 +108,22 @@ Using +BR2_EXTERNAL+ then allows three different things:
    +$(BR2_EXTERNAL_BAR_42)/board/<boardname>/kernel.config+ (to specify
    the location of the kernel configuration file).
 
- * One can store package recipes (i.e. +Config.in+ and
-   +<packagename>.mk+), or even custom configuration options and make
-   logic. Buildroot automatically includes +$(BR2_EXTERNAL)/Config.in+ to
-   make it appear in the top-level configuration menu, and includes
-   +$(BR2_EXTERNAL)/external.mk+ with the rest of the makefile logic.
+ * One can store package recipes (i.e. +Config.in+ and +<packagename>.mk+),
+   or even custom configuration options and make logic. Buildroot
+   automatically includes the +Config.in+ from each paths in +BR2_EXTERNAL+
+   to make it appear in a sub-menu labelled _User-provided options_ at the
+   root of the main configuration menu, and includes the +external.mk+ from
+   each paths in +BR2_EXTERNAL+ with the rest of the makefile logic.
 +
 The main usage of this is to store package recipes. The recommended
-   way to do this is to write a +$(BR2_EXTERNAL)/Config.in+ file that
-   looks like:
+   way to do this is to write +Config.in+ files that look like:
 +
 ------
 source "$BR2_EXTERNAL_BAR_42/package/package1/Config.in"
 source "$BR2_EXTERNAL_BAR_42/package/package2/Config.in"
 ------
 +
-Then, have a +$(BR2_EXTERNAL)/external.mk+ file that looks like:
+Then, write +/external.mk+ files that look like:
 +
 ------
 include $(sort $(wildcard $(BR2_EXTERNAL_BAR_42)/package/*/*.mk))
@@ -121,7 +136,12 @@ And then in +$(BR2_EXTERNAL_FOO_42)/package/package1+ and
    called <boardname> and adapt the above paths accordingly.
 
  * One can store Buildroot defconfigs in the +configs+ subdirectory of
-   +$(BR2_EXTERNAL)+. Buildroot will automatically show them in the
+   a +BR2_EXTERNAL+ tree. Buildroot will automatically show them in the
    output of +make list-defconfigs+ and allow them to be loaded with the
    normal +make <name>_defconfig+ command. They will be visible under the
-   +User-provided configs+' label in the 'make list-defconfigs' output.
+   +User-provided configs+ label in the 'make list-defconfigs' output.
++
+.Note
+If a defconfig file is present in more than one +BR2_EXTERNAL+ location,
+   then the last one is used. It is also possible to override a defconfig
+   bundled in Buildroot.
-- 
1.9.1

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Arnout Vandecappelle]

On 22-10-15 22:33, Yann E. MORIN wrote:
> One of the selling points for br2-external is to provide a mean to add
> new packages. However, it is not supported that a package be defined by
> Buildroot and then redefined in a br2-external tree.
> 
> This situation may occur without the user noticing or even willing to
> redefine the package, for example:
>   - br2-external is first created against a version of Buildroot
>   - a package (missing in Buildroot) is added to that br2-external tree
>   - upstream Buildroot adds this package
>   - user updates to the new Buildroot
> 
> In this case, the result in undefined, and we can't make any guarantee
> on the result (working or not).
> 
> Add a sanity check so that a package redefinition gets caught.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 For you multi-br2-external haters: please commit independently of be multi-
feature :-)


 But see below.

> ---
>  Makefile               | 1 +
>  package/pkg-generic.mk | 9 +++++++++
>  2 files changed, 10 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index dd8959f..da78f18 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -336,6 +336,7 @@ unexport O
>  GNU_HOST_NAME := $(shell support/gnuconfig/config.guess)
>  
>  PACKAGES :=
> +PACKAGES_ALL :=
>  
>  # silent mode requested?
>  QUIET := $(if $(findstring s,$(filter-out --%,$(MAKEFLAGS))),-q)
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index ffef4d3..7f0c2ab 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -341,6 +341,14 @@ endef
>  
>  define inner-generic-package
>  
> +# Ensure the package is only declared once, i.e. do not accept that a
> +# package be re-defined by a br2-external tree
> +ifneq ($(call strip,$(filter $(1),$(PACKAGES_ALL))),)
> +$$(error Package '$(1)' defined a second time in '$(pkgdir)'; \
> +	previous definition was in '$$($(2)_PKGDIR)')
> +endif
> +PACKAGES_ALL += $(1)
> +
>  # Define default values for various package-related variables, if not
>  # already defined. For some variables (version, source, site and
>  # subdir), if they are undefined, we try to see if a variable without
> @@ -351,6 +359,7 @@ define inner-generic-package
>  $(2)_TYPE                       =  $(4)
>  $(2)_NAME			=  $(1)
>  $(2)_RAWNAME			=  $$(patsubst host-%,%,$(1))
> +$(2)_PKGDIR			=  $(pkgdir)

 Now we have this variable, it would be nice to replace all occurences of
$(PKGDIR) with $($(PKG)_PKGDIR) and remove the (IMHO ugly)

$$($(2)_TARGET_PATCH):                   PKGDIR=$(pkgdir)

(obviously in a follow-up patch, which I might produce myself if I feel like it).

 Regards,
 Arnout

>  
>  # Keep the package version that may contain forward slashes in the _DL_VERSION
>  # variable, then replace all forward slashes ('/') by underscores ('_') to
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Yann E. MORIN]

Arnout, All,

On 2015-10-22 23:01 +0200, Arnout Vandecappelle spake thusly:
> On 22-10-15 22:33, Yann E. MORIN wrote:
> > One of the selling points for br2-external is to provide a mean to add
> > new packages. However, it is not supported that a package be defined by
> > Buildroot and then redefined in a br2-external tree.
> > 
> > This situation may occur without the user noticing or even willing to
> > redefine the package, for example:
> >   - br2-external is first created against a version of Buildroot
> >   - a package (missing in Buildroot) is added to that br2-external tree
> >   - upstream Buildroot adds this package
> >   - user updates to the new Buildroot
> > 
> > In this case, the result in undefined, and we can't make any guarantee
> > on the result (working or not).
> > 
> > Add a sanity check so that a package redefinition gets caught.
> > 
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Peter Korsgaard <jacmet@uclibc.org>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> 
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Thanks! :-)

>  For you multi-br2-external haters: please commit independently of be multi-
> feature :-)

Yes, this is independent.

It's in this series because it was prompted by my work on multi-br2-external,
but I could very well have submitted it separately.
 
[--SNIP--]
> > @@ -351,6 +359,7 @@ define inner-generic-package
> >  $(2)_TYPE                       =  $(4)
> >  $(2)_NAME			=  $(1)
> >  $(2)_RAWNAME			=  $$(patsubst host-%,%,$(1))
> > +$(2)_PKGDIR			=  $(pkgdir)
> 
>  Now we have this variable, it would be nice to replace all occurences of
> $(PKGDIR) with $($(PKG)_PKGDIR) and remove the (IMHO ugly)
> 
> $$($(2)_TARGET_PATCH):                   PKGDIR=$(pkgdir)
> 
> (obviously in a follow-up patch, which I might produce myself if I feel like it).

I already started doing so, jsut to see to what extent that would have
an impact, but it is not ready at all.

I also wanted to replace every occurences of hard-coded paths like:
    package/foo/my-file

with:
    $(FOO_PKGDIR)/my-file

but this is an insane amount of work. I'll cary it if/when this patch is
applied.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 03/21 v2] core: move pkg-utils.mk to support/

[Arnout Vandecappelle]

On 22-10-15 22:33, Yann E. MORIN wrote:
> pkg-utils.mk contains various definitions that are used in the package
> infrastructures and packages themselves.
> 
> However, those definitions can be useful in other parts of Buildroot,
> and are already used in a few places that are not related to the package
> infrastructure. Also, $(sep) will be needed early in the Makefile when
> we eventually support multiple br2-external trees.
> 
> Since this file only contains definitions, we can include it anytime.
> 
> So, consider that file to no longer be specific to the package infras:
>   - move it to support and rename it,
>   - move a few similar definitions from the main Makefile to that file.

 Actually I think a lot more should move to support. For instance, pkg-* is
actually not just for packages but also for linux, bootloaders, toolchain. And
even package/Makefile.in (with a better name then :-).

> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> ---
>  Makefile              | 10 ++-----
>  package/pkg-utils.mk  | 59 ----------------------------------------
>  support/misc/utils.mk | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++

 If you agree that more stuff should move to support, I think there should be a
new directory support/make or support/mk.

 BTW, why didn't the rename detection catch this? Did you change too much? In
that case, perhaps it's better to do it in two patches? :-P


Regards,
Arnout

>  3 files changed, 76 insertions(+), 67 deletions(-)
>  create mode 100644 support/misc/utils.mk
> 
[snip]


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Arnout Vandecappelle]

On 22-10-15 23:09, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2015-10-22 23:01 +0200, Arnout Vandecappelle spake thusly:
>> On 22-10-15 22:33, Yann E. MORIN wrote:
[snip]
>>> @@ -351,6 +359,7 @@ define inner-generic-package
>>>  $(2)_TYPE                       =  $(4)
>>>  $(2)_NAME			=  $(1)
>>>  $(2)_RAWNAME			=  $$(patsubst host-%,%,$(1))
>>> +$(2)_PKGDIR			=  $(pkgdir)
>>
>>  Now we have this variable, it would be nice to replace all occurences of
>> $(PKGDIR) with $($(PKG)_PKGDIR) and remove the (IMHO ugly)

 BTW, an alternative is to to globally assign:

PKGDIR = $($(PKG)_PKGDIR)

then we can keep on using $(PKGDIR). But that probably is not really clear for
most developers...

>>
>> $$($(2)_TARGET_PATCH):                   PKGDIR=$(pkgdir)
>>
>> (obviously in a follow-up patch, which I might produce myself if I feel like it).
> 
> I already started doing so, jsut to see to what extent that would have
> an impact, but it is not ready at all.
> 
> I also wanted to replace every occurences of hard-coded paths like:
>     package/foo/my-file
> 
> with:
>     $(FOO_PKGDIR)/my-file
> 
> but this is an insane amount of work. I'll cary it if/when this patch is
> applied.

 I don't think it makes sense to make a mass change like that. If you want to
make a mass change, calculate some hashes :-)

 Regards,
 Arnout

> 
> Regards,
> Yann E. MORIN.
> 

-- 
Arnout Vandecappelle      arnout dot vandecappelle at essensium dot com
Senior Embedded Software Architect . . . . . . +32-478-010353 (mobile)
Essensium, Mind division . . . . . . . . . . . . . . http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium . . . . . BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 03/21 v2] core: move pkg-utils.mk to support/

[Yann E. MORIN]

Arnout, All,

On 2015-10-22 23:10 +0200, Arnout Vandecappelle spake thusly:
> On 22-10-15 22:33, Yann E. MORIN wrote:
> > pkg-utils.mk contains various definitions that are used in the package
> > infrastructures and packages themselves.
> > 
> > However, those definitions can be useful in other parts of Buildroot,
> > and are already used in a few places that are not related to the package
> > infrastructure. Also, $(sep) will be needed early in the Makefile when
> > we eventually support multiple br2-external trees.
> > 
> > Since this file only contains definitions, we can include it anytime.
> > 
> > So, consider that file to no longer be specific to the package infras:
> >   - move it to support and rename it,
> >   - move a few similar definitions from the main Makefile to that file.
> 
>  Actually I think a lot more should move to support. For instance, pkg-* is
> actually not just for packages but also for linux, bootloaders, toolchain. And
> even package/Makefile.in (with a better name then :-).

I do agree. But I think this can be done in a separate series.

The current series is about adding multi-br2-external support, so only
includes whatever is necessary to achieve that goal.

I can queue a rework of the support materials as you suggest, but that
will be done in a separate series (and hopefully not a requirement
before this one is condered! ;-) ).

> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Peter Korsgaard <jacmet@uclibc.org>
> > ---
> >  Makefile              | 10 ++-----
> >  package/pkg-utils.mk  | 59 ----------------------------------------
> >  support/misc/utils.mk | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 
>  If you agree that more stuff should move to support, I think there should be a
> new directory support/make or support/mk.

As I said, we can move it later when doing the support rework.

>  BTW, why didn't the rename detection catch this? Did you change too much? In
> that case, perhaps it's better to do it in two patches? :-P

It was not caught because it is less than 50% (actually, 49%!) so git did
not detect a copy (or a rename either).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 20/21 v2] docs/manual: document the br2-external ID

[Jeremy Rosen]

> 
> -+BR2_EXTERNAL+ allows three different things:
> +A +BR2_EXTERNAL+ tree must contain at least those three files:
> +
> ++external.id+::
> +   That file should contain the ID for the +BR2_EXTERNAL+ tree.
> +   That ID is used to construct the +BR2_EXTERNAL_$(ID)+ variable,
> +   available in +Config.in+ and +external.mk+ (see below), so that ID
> +   must be a valid make and Kconfig variable. Buildroot sets
> +   +BR2_EXTERNAL_$(ID)+ to the path of the +BR2_EXTERNAL+ tree.
> ++
> +Examples:
> ++
> +  ** +FOO+ -> +BR2_EXTERNAL_FOO+
> +  ** +BAR_42+ -> +BR2_EXTERNAL_BAR_42+
> +  ** ...
> +

those are example of valid ID, but what is the exact syntax of the file ? 
is it a single line with just the ID ? this is not ver clear here....


Regards

Jeremy

[Buildroot] [PATCH 20/21 v2] docs/manual: document the br2-external ID

[Yann E. MORIN]

Jeremy, All,

On 2015-10-23 09:40 +0200, Jeremy Rosen spake thusly:
> > -+BR2_EXTERNAL+ allows three different things:
> > +A +BR2_EXTERNAL+ tree must contain at least those three files:
> > +
> > ++external.id+::
> > +   That file should contain the ID for the +BR2_EXTERNAL+ tree.
> > +   That ID is used to construct the +BR2_EXTERNAL_$(ID)+ variable,
> > +   available in +Config.in+ and +external.mk+ (see below), so that ID
> > +   must be a valid make and Kconfig variable. Buildroot sets
> > +   +BR2_EXTERNAL_$(ID)+ to the path of the +BR2_EXTERNAL+ tree.
> > ++
> > +Examples:
> > ++
> > +  ** +FOO+ -> +BR2_EXTERNAL_FOO+
> > +  ** +BAR_42+ -> +BR2_EXTERNAL_BAR_42+
> > +  ** ...
> > +
> 
> those are example of valid ID, but what is the exact syntax of the file ? 
> is it a single line with just the ID ? this is not ver clear here....

Good point. Indeed, the format of the file is to contain just the ID,
like so:

    $ cat external.id
    BAR_42

I'll update the description to make it obvious. Thanks! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Samuel Martin]

Yann,

On Thu, Oct 22, 2015 at 10:33 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> One of the selling points for br2-external is to provide a mean to add
> new packages. However, it is not supported that a package be defined by
> Buildroot and then redefined in a br2-external tree.
>
> This situation may occur without the user noticing or even willing to
> redefine the package, for example:
>   - br2-external is first created against a version of Buildroot
>   - a package (missing in Buildroot) is added to that br2-external tree
>   - upstream Buildroot adds this package
>   - user updates to the new Buildroot
>
> In this case, the result in undefined, and we can't make any guarantee
> on the result (working or not).
>
> Add a sanity check so that a package redefinition gets caught.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Reviewed-by: Samuel Martin <s.martin49@gmail.com>

> ---
>  Makefile               | 1 +
>  package/pkg-generic.mk | 9 +++++++++
>  2 files changed, 10 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index dd8959f..da78f18 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -336,6 +336,7 @@ unexport O
>  GNU_HOST_NAME := $(shell support/gnuconfig/config.guess)
>
>  PACKAGES :=
> +PACKAGES_ALL :=
>
>  # silent mode requested?
>  QUIET := $(if $(findstring s,$(filter-out --%,$(MAKEFLAGS))),-q)
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index ffef4d3..7f0c2ab 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -341,6 +341,14 @@ endef
>
>  define inner-generic-package
>
> +# Ensure the package is only declared once, i.e. do not accept that a
> +# package be re-defined by a br2-external tree
> +ifneq ($(call strip,$(filter $(1),$(PACKAGES_ALL))),)
> +$$(error Package '$(1)' defined a second time in '$(pkgdir)'; \
> +       previous definition was in '$$($(2)_PKGDIR)')
> +endif
> +PACKAGES_ALL += $(1)
> +
>  # Define default values for various package-related variables, if not
>  # already defined. For some variables (version, source, site and
>  # subdir), if they are undefined, we try to see if a variable without
> @@ -351,6 +359,7 @@ define inner-generic-package
>  $(2)_TYPE                       =  $(4)
>  $(2)_NAME                      =  $(1)
>  $(2)_RAWNAME                   =  $$(patsubst host-%,%,$(1))
> +$(2)_PKGDIR                    =  $(pkgdir)

This change can also provide clean/unique way of handling packages'
files coming within Buildroot and br2-external trees. :-)
(I'm not a big fan of the package/foo/foo.conf vs.
$(BR2_EXTERNAL)/package/bar/bar.conf thing, depending whether foo is
in the Builldroot tree, and bar in the br2-external one).

Regards,

-- 
Samuel

[Buildroot] [PATCH 02/21 v2] docs/manual: document $(FOO_PKGDIR)

[Samuel Martin]

Yann,

On Thu, Oct 22, 2015 at 10:33 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> In the manual, add a note that packages can use $(FOO_PKGDIR) to get the
> path to the directory containg the .mk and Config.in files, if they need
> it to e.g. copy files from there, like startup scripts, systemd units or
> any other resource.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Samuel Martin <s.martin49@gmail.com>
> ---
>  docs/manual/adding-packages-generic.txt | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
> index 9d6401f..17ada16 100644
> --- a/docs/manual/adding-packages-generic.txt
> +++ b/docs/manual/adding-packages-generic.txt
> @@ -491,6 +491,9 @@ endef
>
>  In the action definitions, you can use the following variables:
>
> +* +$(FOO_PKGDIR)+ contains the path to the directory containing the
> +  +foo.mk+ and +Config.in+ files.

No mention to what/when this new variable could be useful?
e.g. refer to some packages files coming within the Buildroot tree
such as runtime configuration files...

> +
>  * +$(@D)+, which contains the directory in which the package source
>    code has been uncompressed.
>
> --
> 1.9.1
>

Regards,

-- 
Samuel

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Yann E. MORIN]

Samuel, All,

On 2015-10-23 21:39 +0200, Samuel Martin spake thusly:
> On Thu, Oct 22, 2015 at 10:33 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > One of the selling points for br2-external is to provide a mean to add
> > new packages. However, it is not supported that a package be defined by
> > Buildroot and then redefined in a br2-external tree.
> >
> > This situation may occur without the user noticing or even willing to
> > redefine the package, for example:
> >   - br2-external is first created against a version of Buildroot
> >   - a package (missing in Buildroot) is added to that br2-external tree
> >   - upstream Buildroot adds this package
> >   - user updates to the new Buildroot
> >
> > In this case, the result in undefined, and we can't make any guarantee
> > on the result (working or not).
> >
> > Add a sanity check so that a package redefinition gets caught.
> >
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Peter Korsgaard <jacmet@uclibc.org>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> 
> Reviewed-by: Samuel Martin <s.martin49@gmail.com>
> 
[--SNIP--]
> > @@ -351,6 +359,7 @@ define inner-generic-package
> >  $(2)_TYPE                       =  $(4)
> >  $(2)_NAME                      =  $(1)
> >  $(2)_RAWNAME                   =  $$(patsubst host-%,%,$(1))
> > +$(2)_PKGDIR                    =  $(pkgdir)
> 
> This change can also provide clean/unique way of handling packages'
> files coming within Buildroot and br2-external trees. :-)
> (I'm not a big fan of the package/foo/foo.conf vs.
> $(BR2_EXTERNAL)/package/bar/bar.conf thing, depending whether foo is
> in the Builldroot tree, and bar in the br2-external one).

See what I already replied to Arnout! ;-)

I don;t like it either, and I'd like we switch to using that variable
instead.

Still, that's pretty much of a change, and Arnout is right: such a bulk
change is not needed right now; we can switch over time, as packages are
updated/fixed...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 02/21 v2] docs/manual: document $(FOO_PKGDIR)

[Yann E. MORIN]

Samuel, All,

On 2015-10-23 21:39 +0200, Samuel Martin spake thusly:
> On Thu, Oct 22, 2015 at 10:33 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > In the manual, add a note that packages can use $(FOO_PKGDIR) to get the
> > path to the directory containg the .mk and Config.in files, if they need
> > it to e.g. copy files from there, like startup scripts, systemd units or
> > any other resource.
> >
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Samuel Martin <s.martin49@gmail.com>
> > ---
> >  docs/manual/adding-packages-generic.txt | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
> > index 9d6401f..17ada16 100644
> > --- a/docs/manual/adding-packages-generic.txt
> > +++ b/docs/manual/adding-packages-generic.txt
> > @@ -491,6 +491,9 @@ endef
> >
> >  In the action definitions, you can use the following variables:
> >
> > +* +$(FOO_PKGDIR)+ contains the path to the directory containing the
> > +  +foo.mk+ and +Config.in+ files.
> 
> No mention to what/when this new variable could be useful?
> e.g. refer to some packages files coming within the Buildroot tree
> such as runtime configuration files...

OK, I will add a bit of comment about the use of this variable.

Thanks! :-)

Regards,
Yann E. MORIN.

> >  * +$(@D)+, which contains the directory in which the package source
> >    code has been uncompressed.
> >
> > --
> > 1.9.1
> >
> 
> Regards,
> 
> -- 
> Samuel

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 03/21 v2] core: move pkg-utils.mk to support/

[Arnout Vandecappelle]

On 22-10-15 23:20, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2015-10-22 23:10 +0200, Arnout Vandecappelle spake thusly:
>> On 22-10-15 22:33, Yann E. MORIN wrote:
>>> pkg-utils.mk contains various definitions that are used in the package
>>> infrastructures and packages themselves.
>>>
>>> However, those definitions can be useful in other parts of Buildroot,
>>> and are already used in a few places that are not related to the package
>>> infrastructure. Also, $(sep) will be needed early in the Makefile when
>>> we eventually support multiple br2-external trees.
>>>
>>> Since this file only contains definitions, we can include it anytime.
>>>
>>> So, consider that file to no longer be specific to the package infras:
>>>   - move it to support and rename it,
>>>   - move a few similar definitions from the main Makefile to that file.
>>
>>  Actually I think a lot more should move to support. For instance, pkg-* is
>> actually not just for packages but also for linux, bootloaders, toolchain. And
>> even package/Makefile.in (with a better name then :-).
> 
> I do agree. But I think this can be done in a separate series.

 Yeah sorry I should have mentioned that: as a separate series indeed.


> The current series is about adding multi-br2-external support, so only
> includes whatever is necessary to achieve that goal.
> 
> I can queue a rework of the support materials as you suggest, but that
> will be done in a separate series (and hopefully not a requirement
> before this one is condered! ;-) ).
> 
>>> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
>>> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>>> Cc: Peter Korsgaard <jacmet@uclibc.org>
>>> ---
>>>  Makefile              | 10 ++-----
>>>  package/pkg-utils.mk  | 59 ----------------------------------------
>>>  support/misc/utils.mk | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>  If you agree that more stuff should move to support, I think there should be a
>> new directory support/make or support/mk.
> 
> As I said, we can move it later when doing the support rework.

 OK.

>>  BTW, why didn't the rename detection catch this? Did you change too much? In
>> that case, perhaps it's better to do it in two patches? :-P
> 
> It was not caught because it is less than 50% (actually, 49%!) so git did
> not detect a copy (or a rename either).

 If you send a v2, could you set -M40% so the rename is detected?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 04/21 v2] core: commonalise the bundled and br2-external %_defconfig rules

[Arnout Vandecappelle]

On 22-10-15 22:33, Yann E. MORIN wrote:
> The code for both cases is exactly the same, and only differs in the
> location where defconfig files are looked for.
> 
> We use an intermediate macro to generate the corresponding rules,
> because directly generating the rules is ugly and needs lots of escaping
> and double-dollar-ing for the $(eval ...) and $(foreach ...) calls to
> play nicely together.
> 
> Furthermore, that will be tremendously useful when we support multiple
> br2-external trees.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>

 Bunch of small nits. I'm not giving it my reviewed-by yet because I'm not
convinced that this change really makes things clearer, so it only really
applies in function of the multi-external and for that I first have to see the
whole series.

> ---
>  Makefile | 13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 60cea32..052f58a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -789,13 +789,12 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
>  
>  # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig

 This comment should go inside the define

> -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
> -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
> -		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
> -
> -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
> -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
> -		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
> +define PERCENT_DEFCONFIG

 The macros we've added recently were all lowercase, and I like it that way.

 Regards,
 Arnout

> +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(1)/configs/%_defconfig outputmakefile
> +	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(1)/configs/$$@ \
> +		$$< --defconfig=$(1)/configs/$$@ $$(CONFIG_CONFIG_IN)
> +endef
> +$(eval $(foreach d,$(TOPDIR) $(BR2_EXTERNAL),$(call PERCENT_DEFCONFIG,$(d))$(sep)))
>  
>  savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  	@$(COMMON_CONFIG_ENV) $< \
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 05/21 v2] core: remove .br-external on distclean

[Arnout Vandecappelle]

On 22-10-15 22:34, Yann E. MORIN wrote:
> distclean is supposed to return the current directory, whether in-tree
> or out-of-tree, into pristine conditions, which means we should also
> forget about any br2-external tree on distclean.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 That said: why don't we just remove $(O) unconditionally?


 Note that this can be applied independently of the rest of the series.


 Regards,
 Arnout


> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index 052f58a..0e25bd1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -839,7 +839,7 @@ ifeq ($(O),output)
>  	rm -rf $(O)
>  endif
>  	rm -rf $(BR2_CONFIG) $(CONFIG_DIR)/.config.old $(CONFIG_DIR)/..config.tmp \
> -		$(CONFIG_DIR)/.auto.deps
> +		$(CONFIG_DIR)/.auto.deps $(BR2_EXTERNAL_FILE)
>  
>  help:
>  	@echo 'Cleaning:'
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 04/21 v2] core: commonalise the bundled and br2-external %_defconfig rules

[Yann E. MORIN]

Arnout, All,

On 2015-10-26 21:27 +0100, Arnout Vandecappelle spake thusly:
> On 22-10-15 22:33, Yann E. MORIN wrote:
> > The code for both cases is exactly the same, and only differs in the
> > location where defconfig files are looked for.
> > 
> > We use an intermediate macro to generate the corresponding rules,
> > because directly generating the rules is ugly and needs lots of escaping
> > and double-dollar-ing for the $(eval ...) and $(foreach ...) calls to
> > play nicely together.
> > 
> > Furthermore, that will be tremendously useful when we support multiple
> > br2-external trees.
> > 
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Peter Korsgaard <jacmet@uclibc.org>
> > Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> > Cc: Arnout Vandecappelle <arnout@mind.be>
> 
>  Bunch of small nits. I'm not giving it my reviewed-by yet because I'm not
> convinced that this change really makes things clearer, so it only really
> applies in function of the multi-external

Indeed, without the multi br2-external support, it is not that much
usefull.

> and for that I first have to see the whole series.

Of course. :-)

> > ---
> >  Makefile | 13 ++++++-------
> >  1 file changed, 6 insertions(+), 7 deletions(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index 60cea32..052f58a 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -789,13 +789,12 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> >  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
> >  
> >  # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
> 
>  This comment should go inside the define

Why? The macro only generates Makefile code, so we don;t care that
comment being replicated for both cases.

> > -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
> > -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
> > -		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
> > -
> > -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
> > -	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
> > -		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
> > +define PERCENT_DEFCONFIG
> 
>  The macros we've added recently were all lowercase, and I like it that way.

OK.

Thanks! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 06/21 v2] docs/manual: do not override BR2_EXTERNAL

[Arnout Vandecappelle]

On 22-10-15 22:34, Yann E. MORIN wrote:
> Currently, we forcibly override BR2_EXTERNAL when building the manual,
> so as to avoid referring to packages therein from the manual.
> 
> However, when generating the lists of packages, we limit ourselves to
> scanning for packages in (Buildroot's) TOPDIR and never search in
> BR2_EXTERNAL. So, we do not really need to override BR2_EXTERNAL when
> generating the manual to achieve the same result.
> 
> Furthermore, we're only looking for packages that are defined in the
> sub-menu "Target packages", and never anywhere else, and especially not
> in the sub-menu "User-provided options".
> 
> Finally, we're soon to completely eliminate use of BR2_EXTERNAL in
> Kconfig altogether, so that would no longer have any impact anyway.
> 
> So, just leave BR2_EXTERNAL alone when generating the manual.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>


 Again, can be applied independently of the rest of the series.

 Regards,
 Arnout

> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Samuel Martin <s.martin49@gmail.com>
> ---
>  docs/manual/manual.mk | 3 ---
>  1 file changed, 3 deletions(-)
> 
> diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
> index ad9bd90..caf080a 100644
> --- a/docs/manual/manual.mk
> +++ b/docs/manual/manual.mk
> @@ -8,13 +8,10 @@ MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
>  MANUAL_RESOURCES = $(TOPDIR)/docs/images
>  
>  # Our manual needs to generate lists
> -# Packages included in BR2_EXTERNAL are not part of buildroot, so they
> -# should not be included in the manual.
>  define MANUAL_GEN_LISTS
>  	$(Q)$(call MESSAGE,"Updating the manual lists...")
>  	$(Q)$(COMMON_CONFIG_ENV) \
>  		BR2_DEFCONFIG="" \
> -		BR2_EXTERNAL=$(TOPDIR)/support/dummy-external \
>  		TOPDIR=$(TOPDIR) \
>  		O=$(@D) \
>  		python -B $(TOPDIR)/support/scripts/gen-manual-lists.py
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 04/21 v2] core: commonalise the bundled and br2-external %_defconfig rules

[Arnout Vandecappelle]

On 26-10-15 21:56, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2015-10-26 21:27 +0100, Arnout Vandecappelle spake thusly:
>> On 22-10-15 22:33, Yann E. MORIN wrote:
[snip]
>>> ---
>>>  Makefile | 13 ++++++-------
>>>  1 file changed, 6 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 60cea32..052f58a 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -789,13 +789,12 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>>>  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
>>>  
>>>  # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
>>
>>  This comment should go inside the define
> 
> Why? The macro only generates Makefile code, so we don;t care that
> comment being replicated for both cases.

 Because the comment should be as close as possible to statement it relates to.
So actually it should be placed after the %_defconfig (which is possible now
since there is only one).

 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 01/21 v2] core: do not accept multiple definitions of a package

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 22 Oct 2015 22:33:56 +0200, Yann E. MORIN wrote:
> One of the selling points for br2-external is to provide a mean to add
> new packages. However, it is not supported that a package be defined by
> Buildroot and then redefined in a br2-external tree.
> 
> This situation may occur without the user noticing or even willing to
> redefine the package, for example:
>   - br2-external is first created against a version of Buildroot
>   - a package (missing in Buildroot) is added to that br2-external tree
>   - upstream Buildroot adds this package
>   - user updates to the new Buildroot
> 
> In this case, the result in undefined, and we can't make any guarantee
> on the result (working or not).
> 
> Add a sanity check so that a package redefinition gets caught.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  Makefile               | 1 +
>  package/pkg-generic.mk | 9 +++++++++
>  2 files changed, 10 insertions(+)

Applied, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 05/21 v2] core: remove .br-external on distclean

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 22 Oct 2015 22:34:00 +0200, Yann E. MORIN wrote:
> distclean is supposed to return the current directory, whether in-tree
> or out-of-tree, into pristine conditions, which means we should also
> forget about any br2-external tree on distclean.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Applied, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 06/21 v2] docs/manual: do not override BR2_EXTERNAL

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 22 Oct 2015 22:34:01 +0200, Yann E. MORIN wrote:
> Currently, we forcibly override BR2_EXTERNAL when building the manual,
> so as to avoid referring to packages therein from the manual.
> 
> However, when generating the lists of packages, we limit ourselves to
> scanning for packages in (Buildroot's) TOPDIR and never search in
> BR2_EXTERNAL. So, we do not really need to override BR2_EXTERNAL when
> generating the manual to achieve the same result.
> 
> Furthermore, we're only looking for packages that are defined in the
> sub-menu "Target packages", and never anywhere else, and especially not
> in the sub-menu "User-provided options".
> 
> Finally, we're soon to completely eliminate use of BR2_EXTERNAL in
> Kconfig altogether, so that would no longer have any impact anyway.
> 
> So, just leave BR2_EXTERNAL alone when generating the manual.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Peter Korsgaard <jacmet@uclibc.org>
> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Samuel Martin <s.martin49@gmail.com>
> ---
>  docs/manual/manual.mk | 3 ---
>  1 file changed, 3 deletions(-)

Applied, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com