[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Hello,

Following the e-mail from Ryan Barnett entitled "[RFC] New feature to
handle a custom board folder", I finally took a bit of time to
implement how I think the support for externally stored Buildroot
customization should be done.

I really think the approach of having board-related configuration
options within Buildroot is wrong: it is the Buildroot configuration
as a whole that defines what the configuration of the user system is:
a combination of a kernel, a root filesystem with various packages, a
selection of a root filesystem image format, some system configuration
tuning, a bootloader, etc.

In Ryan e-mail, I believe the most important thing is the wish to
store the board-specific details outside of the Buildroot tree. In
fact, this is already possible: nothing prevents anyone from using
$(TOPDIR)/../company/kernel.config as the kernel configuration file
for example, where $(TOPDIR)/../company/ is outside of the Buildroot
tree, and versioned in a completely different way.

However, it seems like this possibility is not 'visible' enough, and
admittedly only usable for some Buildroot configuration elements,
while an user may also be interested in storing Buildroot package
recipes and Buildroot deconfig files outside of the main Buildroot
tree.

So, this patch set adds the BR2_EXTERNAL mechanism. It is very
lightweight, and doesn't change anything to the Buildroot overall
logic. Basically, BR2_EXTERNAL is an environment variable that one can
point to a directory that will contain Buildroot customizations. It is
then usable for three things:

 * It is available as $(BR2_EXTERNAL) within all of the Buildroot
   configuration options, to define the location of the kernel
   configuration file, the root filesystem overlay, etc.

 * It allows to store package recipes in $(BR2_EXTERNAL)/package/,
   exactly like the package/ directory of Buildroot. Buildroot will
   automatically include $(BR2_EXTERNAL)/Config.in under the top-level
   menu 'Target packages' menu, and will include
   $(BR2_EXTERNAL)/external.mk at the makefile level. So by adding a
   simple $(BR2_EXTERNAL)/Config.in and $(BR2_EXTERNAL)/external.mk,
   one can store package recipes outside Buildroot. See the manual
   update (PATCH 3) for details.

 * It allows to store defconfigs in $(BR2_EXTERNAL)/configs/. They
   appear automatically in 'make help' and can be loaded like any
   other defconfig using 'make <something>_defconfig'.

Best regards,

Thomas

Thomas Petazzoni (3):
  Makefile: factorize *config dependencies
  Add support for BR2_EXTERNAL
  docs/manual: add explanations about BR2_EXTERNAL

 Config.in                            |   6 +-
 Makefile                             |  64 +++++++++++++-------
 docs/manual/customize-outside-br.txt | 110 +++++++++++++++++++++++++++++++++++
 docs/manual/customize.txt            |   2 +
 docs/manual/manual.mk                |   2 +-
 5 files changed, 161 insertions(+), 23 deletions(-)
 create mode 100644 docs/manual/customize-outside-br.txt

-- 
1.8.1.2

[Buildroot] [PATCH 1/3] Makefile: factorize *config dependencies

[Thomas Petazzoni]

All of the *config targets have common dependencies. Currently, the
only common dependency is the 'outputmakefile' target, but as we are
going to add one more, it makes sense to factorize these common
dependencies under the COMMON_CONFIG_DEPS variable.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/Makefile b/Makefile
index 0d7476a..fc55b87 100644
--- a/Makefile
+++ b/Makefile
@@ -633,43 +633,46 @@ COMMON_CONFIG_ENV = \
 	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
 	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG)
 
-xconfig: $(BUILD_DIR)/buildroot-config/qconf outputmakefile
+COMMON_CONFIG_DEPS = \
+	outputmakefile
+
+xconfig: $(BUILD_DIR)/buildroot-config/qconf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-gconfig: $(BUILD_DIR)/buildroot-config/gconf outputmakefile
+gconfig: $(BUILD_DIR)/buildroot-config/gconf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) srctree=$(TOPDIR) $< $(CONFIG_CONFIG_IN)
 
-menuconfig: $(BUILD_DIR)/buildroot-config/mconf outputmakefile
+menuconfig: $(BUILD_DIR)/buildroot-config/mconf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-nconfig: $(BUILD_DIR)/buildroot-config/nconf outputmakefile
+nconfig: $(BUILD_DIR)/buildroot-config/nconf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-config: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+config: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
 
-oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+oldconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --oldconfig $(CONFIG_CONFIG_IN)
 
-randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+randconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
 
-allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allyesconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
 
-allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allnoconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --allnoconfig $(CONFIG_CONFIG_IN)
 
-randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+randpackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
@@ -680,7 +683,7 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 		$< --randconfig $(CONFIG_CONFIG_IN)
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 
-allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
@@ -691,7 +694,7 @@ allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 		$< --allyesconfig $(CONFIG_CONFIG_IN)
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 
-allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
 	@$(COMMON_CONFIG_ENV) \
@@ -699,23 +702,23 @@ allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 		$< --allnoconfig $(CONFIG_CONFIG_IN)
 	@rm -f $(CONFIG_DIR)/.config.nopkg
 
-silentoldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+silentoldconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	$(COMMON_CONFIG_ENV) $< --silentoldconfig $(CONFIG_CONFIG_IN)
 
-olddefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+olddefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN)
 
-defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+defconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
+%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
 
-savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
+savedefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< \
 		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
-- 
1.8.1.2

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Thomas Petazzoni]

This commit adds support for an environment variable named
BR2_EXTERNAL, which the user can point to a directory outside of
Buildroot that contains root filesystem overlays, kernel configuration
files, package recipes, defconfigs, etc. It allows users to keep their
specific Buildroot customizations outside of the Buildroot tree.

BR2_EXTERNAL allows:

 (1) To use $(BR2_EXTERNAL) in all Buildroot configuration options
     that take a path as argument. This allows to reference a root
     filesystem overlay, a kernel configuration file, a Barebox
     configuration file located in the BR2_EXTERNAL directory.

 (2) To store external package or makefile logic, since the
     BR2_EXTERNAL/external.mk file is automatically included by the
     Makefile logic, and the BR2_EXTERNAL/Config.in file is
     automatically included, and appears in the top-level menu. The
     typical usage would be to create a BR2_EXTERNAL/package/
     directory to contain package definitions.

 (3) To store defconfig files under BR2_EXTERNAL/configs/. They will
     be visible in 'make help' and usable through 'make
     <someboard>_defconfig'.

In terms of implementation, it is relatively straightforward:

 * A BR2_EXTERNAL kconfig variable is added, which is set to the value
   of the BR2_EXTERNAL environment variable.

 * The top-level Config.in file given to kconfig is no longer the main
   Config.in in the Buildroot sources, but instead a toplevel.in file
   generated in the output directory, which includes the top-level
   Buildroot Config.in file and the BR2_EXTERNAL/Config.in file if
   provided. Since is needed because the kconfig 'source' statement
   errors out if the included file doesn't exist. I have written
   patches that add support for an 'isource' statement in kconfig
   (that silently ignores the inclusion if the pointed file doesn't
   exist), but keeping feature patches against kconfig doesn't seem
   like a good idea. Note that the "mainmenu" statement is part of
   this generated file, because it must be the first statement seen in
   the toplevel Config.in file passed to kconfig.

 * The BR2_EXTERNAL/external.mk makefile gets included.

 * The BR2_EXTERNAL environment variable gets passed in the
   environment of kconfig and when executing the kconfiglib-based
   Python script that updates the manual, so that the references to
   the BR2_EXTERNAL variable within Config.in files can be resolved.

 * The 'make help' and 'make %_defconfig' targets are updated to take
   into account the defconfig files stored under
   BR2_EXTERNAL/configs/.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Config.in             |  6 ++++--
 Makefile              | 29 +++++++++++++++++++++++++----
 docs/manual/manual.mk |  2 +-
 3 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/Config.in b/Config.in
index cb246a4..7e253c6 100644
--- a/Config.in
+++ b/Config.in
@@ -1,7 +1,5 @@
 #
 
-mainmenu "Buildroot $BR2_VERSION Configuration"
-
 config BR2_HAVE_DOT_CONFIG
 	bool
 	default y
@@ -14,6 +12,10 @@ config BR2_HOSTARCH
 	string
 	option env="HOSTARCH"
 
+config BR2_EXTERNAL
+	string
+	option env="BR2_EXTERNAL"
+
 # Hidden boolean selected by pre-built packages for x86, when they
 # need to run on x86-64 machines (example: pre-built external
 # toolchains, binary tools like SAM-BA, etc.).
diff --git a/Makefile b/Makefile
index fc55b87..1014399 100644
--- a/Makefile
+++ b/Makefile
@@ -47,7 +47,6 @@ export HOSTARCH := $(shell uname -m | \
 
 # absolute path
 TOPDIR:=$(shell pwd)
-CONFIG_CONFIG_IN=Config.in
 CONFIG=support/kconfig
 DATE:=$(shell date +%Y%m%d)
 
@@ -339,6 +338,10 @@ include boot/common.mk
 include linux/linux.mk
 include system/system.mk
 
+ifneq ($(BR2_EXTERNAL),)
+include $(BR2_EXTERNAL)/external.mk
+endif
+
 TARGETS+=target-finalize
 
 ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)
@@ -622,6 +625,18 @@ $(BUILD_DIR)/buildroot-config/%onf:
 	mkdir -p $(@D)/lxdialog
 	$(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)" obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
 
+CONFIG_CONFIG_IN=$(BUILD_DIR)/buildroot-config/toplevel.in
+
+# This is intentionally a virtual target so that the file gets
+# regenerated everytime this target is invoked.
+toplevelin-generate:
+	mkdir -p $(dir $(CONFIG_CONFIG_IN))
+	echo "mainmenu \"Buildroot $$BR2_VERSION Configuration\"" > $(CONFIG_CONFIG_IN)
+	echo "source \"Config.in\"" >> $(CONFIG_CONFIG_IN)
+ifneq ($(BR2_EXTERNAL),)
+	echo "source \"$$BR2_EXTERNAL/Config.in\"" >> $(CONFIG_CONFIG_IN)
+endif
+
 DEFCONFIG = $(call qstrip,$(BR2_DEFCONFIG))
 
 # We don't want to fully expand BR2_DEFCONFIG here, so Kconfig will
@@ -631,10 +646,12 @@ COMMON_CONFIG_ENV = \
 	KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \
 	KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \
 	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
-	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG)
+	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG) \
+	BR2_EXTERNAL=$(BR2_EXTERNAL)
 
 COMMON_CONFIG_DEPS = \
-	outputmakefile
+	outputmakefile \
+	toplevelin-generate
 
 xconfig: $(BUILD_DIR)/buildroot-config/qconf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
@@ -718,6 +735,10 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
 
+%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig $(COMMON_CONFIG_DEPS)
+	@mkdir -p $(BUILD_DIR)/buildroot-config
+	@$(COMMON_CONFIG_ENV) $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@$(COMMON_CONFIG_ENV) $< \
@@ -829,7 +850,7 @@ endif
 	@echo '  make V=0|1             - 0 => quiet build (default), 1 => verbose build'
 	@echo '  make O=dir             - Locate all output files in "dir", including .config'
 	@echo
-	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig $(BR2_EXTERNAL)/configs/*_defconfig))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 	@echo
 	@echo 'See docs/README, or generate the Buildroot manual for further details'
diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
index 4906bc8..8e0ab30 100644
--- a/docs/manual/manual.mk
+++ b/docs/manual/manual.mk
@@ -1,6 +1,6 @@
 manual-update-lists:
 	$(Q)$(call MESSAGE,"Updating the manual lists...")
-	$(Q)BR2_DEFCONFIG="" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \
+	$(Q)BR2_DEFCONFIG="" BR2_EXTERNAL="$(BR2_EXTERNAL)" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \
 		$(TOPDIR)/support/scripts/gen-manual-lists.py
 
 ################################################################################
-- 
1.8.1.2

[Buildroot] [PATCH 3/3] docs/manual: add explanations about BR2_EXTERNAL

[Thomas Petazzoni]

This commit updates the manual to add details on how to use the
BR2_EXTERNAL feature.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/customize-outside-br.txt | 110 +++++++++++++++++++++++++++++++++++
 docs/manual/customize.txt            |   2 +
 2 files changed, 112 insertions(+)
 create mode 100644 docs/manual/customize-outside-br.txt

diff --git a/docs/manual/customize-outside-br.txt b/docs/manual/customize-outside-br.txt
new file mode 100644
index 0000000..81a8911
--- /dev/null
+++ b/docs/manual/customize-outside-br.txt
@@ -0,0 +1,110 @@
+// -*- mode:doc -*- ;
+
+[[outside-br-custom]]
+Keeping customization outside Buildroot
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While the Buildroot community recommends and encourages upstreaming to
+the official Buildroot version the packages and boards support that
+are written by developers, it is sometimes not possible or desirable,
+due to these packages or boards being highly specific or proprietary.
+
+In this case, Buildroot users are offered two choices:
+
+ * They can add their packages, board support and configuration files
+   directly within the Buildroot tree, and maintain them by using
+   branches in a version control system.
+
+ * They can use the +BR2_EXTERNAL+ mechanism, which allows to keep
+   package recipes, board support and configuration files outside of
+   the Buildroot tree, while still having them nicely integrated in
+   the build logic. The following paragraphs give details on how to
+   use +BR2_EXTERNAL+.
+
++BR2_EXTERNAL+ is an environment variable that one can use to point to
+a directory that contains Buildroot customizations. This environment
+variable must be passed everytime a Buildroot +make+ command is
+invoked, be it at configuration time (+make menuconfig+, +make
+xconfig+, etc.), at build time (+make+) or any other Buildroot command
+(+make source+, +make legal-info+, etc.).
+
+Examples:
+
+-----
+ buildroot/ $ make BR2_EXTERNAL=../company menuconfig
+ buildroot/ $ make BR2_EXTERNAL=../company
+-----
+
+This +BR2_EXTERNAL+ then allows three different things:
+
+ * One can store all the board-specific configuration files here, such
+   as the kernel configuration, the root filesystem overlay, or any
+   other configuration file for which Buildroot allows to set its
+   location. The +BR2_EXTERNAL+ value is available within the
+   Buildroot configuration using +$(BR2_EXTERNAL)+. As an example, one
+   could set the +BR2_ROOTFS_OVERLAY+ Buildroot option to
+   +$(BR2_EXTERNAL)/board/someboard/overlay/+ (to specify a root
+   filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+
+   Buildroot option to +$(BR2_EXTERNAL)/board/someboard/kernel.config+
+   (to specify the location of the kernel configuration file).
++
+To achieve this, it is recommended but not mandatory, to store those
+   details in directories called +board/<boardname>/+ under
+   +BR2_EXTERNAL+.
+
+ * One can store package recipes (i.e +Config.in+ and
+   +<packagename>.mk+), or even custom configuration options and make
+   logic. Buildroot automatically includes +BR2_EXTERNAL/Config.in+ to
+   make it appear in the top-level configuration menu, and includes
+   +BR2_EXTERNAL/external.mk+ with the rest of the makefile logic.
++
+The main usage of this would be to store package recipes. The
+   recommended way to do this is to write a +BR2_EXTERNAL/Config.in+
+   that looks like:
++
+------
+menu "<somecompany> packages"
+
+source "$BR2_EXTERNAL/package/package1/Config.in"
+source "$BR2_EXTERNAL/package/package2/Config.in"
+
+endmenu
+------
++
+Then, have a +BR2_EXTERNAL/external.mk' file that looks like:
++
+------
+include $(BR2_EXTERNAL)/package/*/*.mk
+------
++
+And then in +BR2_EXTERNAL/package/package1+ and
+   +BR2_EXTERNAL/package/package2+ create normal Buildroot package
+   recipes, as explained in xref:adding-packages[].
+
+ * One can store Buildroot defconfigs in the +configs+ subdirectory of
+   +BR2_EXTERNAL+. Buildroot will automatically show them in the
+   output of +make help+ and allow them to be loaded with the normal
+   +make <name>_defconfig+ command.
+
+In the end, a typical +BR2_EXTERNAL+ directory organization would
+generally be:
+
+-----
+??? Config.in
+??? external.mk
+??? board/
+?   ??? <someboard>/
+?       ??? overlay/
+?           ??? etc/
+?               ??? <some file>
+??? configs/
+?   ??? <some board>_defconfig
+??? package/
+    ??? Config.in
+    ??? package1/
+        ??? Config.in
+        ??? package1.mk
+    ??? package2/
+        ??? Config.in
+        ??? package2.mk
+------
diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
index 0456ef1..7e46fd8 100644
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -17,3 +17,5 @@ include::customize-toolchain.txt[]
 include::customize-store.txt[]
 
 include::customize-packages.txt[]
+
+include::customize-outside-br.txt[]
-- 
1.8.1.2

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[rjbarnet at rockwellcollins.com]

Thomas - 

Thank you so much for this patchset as it address the biggest issue that
my company currently has with buildroot and will help us become a much 
bigger
contributer to buildroot since we can move to a model that is conducive to
contributing changes back up to buildroot.

See my other comments below. I'm currently testing the patchset out and 
will
be commenting on this shortly.

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
08:15:26 AM:

> Hello,
> 
> Following the e-mail from Ryan Barnett entitled "[RFC] New feature to
> handle a custom board folder", I finally took a bit of time to
> implement how I think the support for externally stored Buildroot
> customization should be done.

After getting feedback from Arnout my idea of how this shifted towards 
the model that you've implemented here in this patchset. However, without
intimate knowledge of top-level make structure/kconfig, I didn't quite
visualize this implementation.

With  the Buildroot mailing list I did not see this patchset until this
morning so you can disregard my thoughts about my second revision of how 
to approach this problem even though I was starting to gravitate towards
this approach. However, my reasoning of why I would like to push for some
thing like this still stand.

> I really think the approach of having board-related configuration
> options within Buildroot is wrong: it is the Buildroot configuration
> as a whole that defines what the configuration of the user system is:
> a combination of a kernel, a root filesystem with various packages, a
> selection of a root filesystem image format, some system configuration
> tuning, a bootloader, etc.

I definitely agree!

> In Ryan e-mail, I believe the most important thing is the wish to
> store the board-specific details outside of the Buildroot tree. In
> fact, this is already possible: nothing prevents anyone from using
> $(TOPDIR)/../company/kernel.config as the kernel configuration file
> for example, where $(TOPDIR)/../company/ is outside of the Buildroot
> tree, and versioned in a completely different way.

Yes this is correct. The biggest challenge is differentating our company's
proprity changes with changes that we have brought in from the buildroot
comminuity as a project progresses through it's development stages. It
is definitely a change when not being able to use git.

> However, it seems like this possibility is not 'visible' enough, and
> admittedly only usable for some Buildroot configuration elements,
> while an user may also be interested in storing Buildroot package
> recipes and Buildroot deconfig files outside of the main Buildroot
> tree.

I agree because usually our changes were scattered between board/<company>
and <package/company>. This model allows for use to consolidate outside
of buildroot in one toplevel directory.

> So, this patch set adds the BR2_EXTERNAL mechanism. It is very
> lightweight, and doesn't change anything to the Buildroot overall
> logic. Basically, BR2_EXTERNAL is an environment variable that one can
> point to a directory that will contain Buildroot customizations. It is
> then usable for three things:

Agree that it is very lightweight and minimal.

Thanks Again,
-Ryan



Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/2e3c6953/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/2e3c6953/attachment.gif>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[rjbarnet at rockwellcollins.com]

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
08:15:28 AM:

> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Config.in             |  6 ++++--
>  Makefile              | 29 +++++++++++++++++++++++++----
>  docs/manual/manual.mk |  2 +-

One idea that I have, is for the make file that is genreated for the an
external output directory, if we could preserve the the value of 
BR2_EXTERNAL
as follows. I don't think my logic is fully there but there is more of a 
sudo implementation:

diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
--- a/support/scripts/mkmakefile
+++ b/support/scripts/mkmakefile
@@ -27,6 +27,7 @@ makedir := \$(dir \$(call lastword,\$(MAKEFILE_LIST)))

 MAKEARGS := -C $1
 MAKEARGS += O=\$(if \$(patsubst /%,,\$(makedir)),\$(CURDIR)/)\$(patsubst 
%/,%,\$(makedir))
+MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))

 MAKEFLAGS += --no-print-directory

Again this is only a suggestion but otherwise I'll ack the patchset.

Thanks,
-Ryan




Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/0f73d9eb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/0f73d9eb/attachment.gif>

[Buildroot] [PATCH 1/3] Makefile: factorize *config dependencies

[rjbarnet at rockwellcollins.com]

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
08:15:27 AM:

> All of the *config targets have common dependencies. Currently, the
> only common dependency is the 'outputmakefile' target, but as we are
> going to add one more, it makes sense to factorize these common
> dependencies under the COMMON_CONFIG_DEPS variable.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Ryan Barnett <rjbarnet@rockwellcollins.com>



Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/e6ac348b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/e6ac348b/attachment-0001.gif>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[rjbarnet at rockwellcollins.com]

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
08:15:28 AM

> This commit adds support for an environment variable named
> BR2_EXTERNAL, which the user can point to a directory outside of
> Buildroot that contains root filesystem overlays, kernel configuration
> files, package recipes, defconfigs, etc. It allows users to keep their
> specific Buildroot customizations outside of the Buildroot tree.
> 
> BR2_EXTERNAL allows:
> 
>  (1) To use $(BR2_EXTERNAL) in all Buildroot configuration options
>      that take a path as argument. This allows to reference a root
>      filesystem overlay, a kernel configuration file, a Barebox
>      configuration file located in the BR2_EXTERNAL directory.
> 
>  (2) To store external package or makefile logic, since the
>      BR2_EXTERNAL/external.mk file is automatically included by the
>      Makefile logic, and the BR2_EXTERNAL/Config.in file is
>      automatically included, and appears in the top-level menu. The
>      typical usage would be to create a BR2_EXTERNAL/package/
>      directory to contain package definitions.
> 
>  (3) To store defconfig files under BR2_EXTERNAL/configs/. They will
>      be visible in 'make help' and usable through 'make
>      <someboard>_defconfig'.
> 
> In terms of implementation, it is relatively straightforward:
> 
>  * A BR2_EXTERNAL kconfig variable is added, which is set to the value
>    of the BR2_EXTERNAL environment variable.
> 
>  * The top-level Config.in file given to kconfig is no longer the main
>    Config.in in the Buildroot sources, but instead a toplevel.in file
>    generated in the output directory, which includes the top-level
>    Buildroot Config.in file and the BR2_EXTERNAL/Config.in file if
>    provided. Since is needed because the kconfig 'source' statement
>    errors out if the included file doesn't exist. I have written
>    patches that add support for an 'isource' statement in kconfig
>    (that silently ignores the inclusion if the pointed file doesn't
>    exist), but keeping feature patches against kconfig doesn't seem
>    like a good idea. Note that the "mainmenu" statement is part of
>    this generated file, because it must be the first statement seen in
>    the toplevel Config.in file passed to kconfig.
> 
>  * The BR2_EXTERNAL/external.mk makefile gets included.
> 
>  * The BR2_EXTERNAL environment variable gets passed in the
>    environment of kconfig and when executing the kconfiglib-based
>    Python script that updates the manual, so that the references to
>    the BR2_EXTERNAL variable within Config.in files can be resolved.
> 
>  * The 'make help' and 'make %_defconfig' targets are updated to take
>    into account the defconfig files stored under
>    BR2_EXTERNAL/configs/.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Ryan Barnett <rjbarnet@rockwellcollins.com>



Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/c1e8074b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/c1e8074b/attachment.gif>

[Buildroot] [PATCH 3/3] docs/manual: add explanations about BR2_EXTERNAL

[rjbarnet at rockwellcollins.com]

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
08:15:29 AM

> This commit updates the manual to add details on how to use the
> BR2_EXTERNAL feature.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Ryan Barnett <rjbarnet@rockwellcollins.com>



Ryan J Barnett / Software Engineer / Platform SW 
MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
Phone: 319-263-3880 / VPN: 263-3880 
rjbarnet at rockwellcollins.com
www.rockwellcollins.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/6920bda5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2004 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130910/6920bda5/attachment.gif>

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Baruch Siach]

Hi Thomas,

On Sun, Sep 08, 2013 at 03:15:26PM +0200, Thomas Petazzoni wrote:
> Following the e-mail from Ryan Barnett entitled "[RFC] New feature to
> handle a custom board folder", I finally took a bit of time to
> implement how I think the support for externally stored Buildroot
> customization should be done.

Thanks! This set of patched would be very useful for projects I'm involved in.

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Ryan,

First of all, thanks for the feedback.

On Tue, 10 Sep 2013 20:32:32 -0500, rjbarnet at rockwellcollins.com wrote:

> Thank you so much for this patchset as it address the biggest issue that
> my company currently has with buildroot and will help us become a much 
> bigger
> contributer to buildroot since we can move to a model that is conducive to
> contributing changes back up to buildroot.

Glad to hear this!

> > Following the e-mail from Ryan Barnett entitled "[RFC] New feature to
> > handle a custom board folder", I finally took a bit of time to
> > implement how I think the support for externally stored Buildroot
> > customization should be done.
> 
> After getting feedback from Arnout my idea of how this shifted towards 
> the model that you've implemented here in this patchset. However, without
> intimate knowledge of top-level make structure/kconfig, I didn't quite
> visualize this implementation.
> 
> With  the Buildroot mailing list I did not see this patchset until this
> morning so you can disregard my thoughts about my second revision of how 
> to approach this problem even though I was starting to gravitate towards
> this approach. However, my reasoning of why I would like to push for some
> thing like this still stand.

Ok. Note that after I posted my patches, I had a discussion with Yann
Morin on IRC and he was skeptical about it, for two reasons:

 (1) Almost all of what BR2_EXTERNAL allows is possible today with the
     current Buildroot. For board level stuff, nothing forces you to
     store it in board/<foo>/ within the Buildroot tree: all of the
     configuration options in Buildroot can use
     $(TOPDIR)/../company/<foo>/ to reference a kernel configuration
     file, or a root filesystem overlay.

     For packages, the 'local.mk' file is already automatically
     included in Buildroot if it exists. So you can create such a file
     and put a "include $(TOPDIR)/../company/external.mk' line in it.
     This external.mk will include all the makefiles of the external
     packages. Since Config.in can't be integrated as easily, one
     solution would be to hardcode BR2_PACKAGE_FOOBAR=y directly within
     external.mk to have it always enabled.

     So, it's already possible today, but I believe the current
     situation is (a) not very easy to use, and (b) not visible enough.
     (b) could be solved by more documentation, but I believe
     BR2_EXTERNAL does not make Buildroot more complicated and solves
     (a) quite nicely.

 (2) Yann's feeling was that by giving a too easy solution to keep
     things out of tree would discourage BR users from contributing
     their new packages upstream, since they can keep it outside of the
     tree nicely. While I certainly admit it can be the case, your
     introduction to this e-mail interestingly points exactly the
     opposite :-)

Best regards,

Thomas Petazzoni
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Ryan Barnett]

Thomas,

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/11/2013 
02:17:00 AM:

> Ryan,
> 
> First of all, thanks for the feedback.

You're welcome.

> On Tue, 10 Sep 2013 20:32:32 -0500, rjbarnet at rockwellcollins.com wrote:
> 
> > Thank you so much for this patchset as it address the biggest issue 
that
> > my company currently has with buildroot and will help us become a much 

> > bigger
> > contributer to buildroot since we can move to a model that is 
conducive to
> > contributing changes back up to buildroot.
> 
> Glad to hear this!

I'll give bit more detail on how patch could allow our company to be
better contribute back to the buildroot project with this change. During 
development of a project, we could use git to track mainline and any 
third party package that we add support for, we would contribute them 
back upstream. The hope would be that by the time we get to the point
where we have to lock into a specific version of buildroot, our changes
would have been incorporated into buildroot. If not, we could move the
packages that were rejected into the BR2_EXTERNAL location but still
be able to use an off-the-shelve version of buildroot.

> Ok. Note that after I posted my patches, I had a discussion with Yann
> Morin on IRC and he was skeptical about it, for two reasons:
> 
>  (1) Almost all of what BR2_EXTERNAL allows is possible today with the
>      current Buildroot. For board level stuff, nothing forces you to
>      store it in board/<foo>/ within the Buildroot tree: all of the
>      configuration options in Buildroot can use
>      $(TOPDIR)/../company/<foo>/ to reference a kernel configuration
>      file, or a root filesystem overlay.
> 
>      For packages, the 'local.mk' file is already automatically
>      included in Buildroot if it exists. So you can create such a file
>      and put a "include $(TOPDIR)/../company/external.mk' line in it.
>      This external.mk will include all the makefiles of the external
>      packages. Since Config.in can't be integrated as easily, one
>      solution would be to hardcode BR2_PACKAGE_FOOBAR=y directly within
>      external.mk to have it always enabled.

That just becomes nasty and doesn't really easily allow for different 
configurations where you don't always want all your propriety packages
to build. Also it doesn't make your propriety packages visible within
the buildroot menuconfig. The other nice feature is the is that company's
configs are automatically pulled in a separate area.

>      So, it's already possible today, but I believe the current
>      situation is (a) not very easy to use, and (b) not visible enough.
>      (b) could be solved by more documentation, but I believe
>      BR2_EXTERNAL does not make Buildroot more complicated and solves
>      (a) quite nicely.

I definitely agree

>  (2) Yann's feeling was that by giving a too easy solution to keep
>      things out of tree would discourage BR users from contributing
>      their new packages upstream, since they can keep it outside of the
>      tree nicely. While I certainly admit it can be the case, your
>      introduction to this e-mail interestingly points exactly the
>      opposite :-)

While I do understand Yann's concerns, however, I do think that this goes
against buildroot's philosophy - "Buildroot: make Embedded Linux easy". 
Right now it isn't easy for one to make customization to buildroot without
modifying buildroot's source.

Yann - I would welcome you input into this discussion on mailing list 
here.

I'm probably beating this issue to death but I guess I'm very excited 
about
the possibilities that this patchset opens up for my company.

Thanks,
-Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130911/cde71f90/attachment-0001.html>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Yann E. MORIN]

Ryan, All,

On 2013-09-10 21:03 -0500, rjbarnet at rockwellcollins.com spake thusly:
> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/08/2013 
> 08:15:28 AM:
> 
> > 
> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > ---
> >  Config.in             |  6 ++++--
> >  Makefile              | 29 +++++++++++++++++++++++++----
> >  docs/manual/manual.mk |  2 +-
> 
> One idea that I have, is for the make file that is genreated for the an
> external output directory, if we could preserve the the value of 
> BR2_EXTERNAL
> as follows. I don't think my logic is fully there but there is more of a 
> sudo implementation:
> 
> diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
> --- a/support/scripts/mkmakefile
> +++ b/support/scripts/mkmakefile
> @@ -27,6 +27,7 @@ makedir := \$(dir \$(call lastword,\$(MAKEFILE_LIST)))
> 
>  MAKEARGS := -C $1
>  MAKEARGS += O=\$(if \$(patsubst /%,,\$(makedir)),\$(CURDIR)/)\$(patsubst 
> %/,%,\$(makedir))
> +MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))

Why realpath? I think we should pass the exact value of BR2_EXTERNAL.

Otherwise, I agree that this could be usefull.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Ryan Barnett]

"Yann E. MORIN" <yann.morin.1998@gmail.com> wrote on 09/11/2013 12:03:40 
PM:

> > One idea that I have, is for the make file that is genreated for the 
an
> > external output directory, if we could preserve the the value of 
> > BR2_EXTERNAL
> > as follows. I don't think my logic is fully there but there is more of 
a 
> > sudo implementation:
> > 
> > diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
> > --- a/support/scripts/mkmakefile
> > +++ b/support/scripts/mkmakefile
> > @@ -27,6 +27,7 @@ makedir := \$(dir \$(call 
lastword,\$(MAKEFILE_LIST)))
> > 
> >  MAKEARGS := -C $1
> >  MAKEARGS += O=\$(if \$(patsubst 
/%,,\$(makedir)),\$(CURDIR)/)\$(patsubst 
> > %/,%,\$(makedir))
> > +MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))
> 
> Why realpath? I think we should pass the exact value of BR2_EXTERNAL.

I would agree, like I said when I suggested this I didn't think it all the
way through other than this feature would be very useful.

> Otherwise, I agree that this could be usefull.
> 
> Regards,
> Yann E. MORIN.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130911/5ef6fd3e/attachment.html>

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Ryan, Thomas, All,

On 2013-09-11 10:55 -0500, Ryan Barnett spake thusly:
> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/11/2013 
> 02:17:00 AM:
> > On Tue, 10 Sep 2013 20:32:32 -0500, rjbarnet at rockwellcollins.com wrote:
> > > Thank you so much for this patchset as it address the biggest issue 
> that
> > > my company currently has with buildroot and will help us become a much 
> 
> > > bigger
> > > contributer to buildroot since we can move to a model that is 
> conducive to
> > > contributing changes back up to buildroot.
> > 
> > Glad to hear this!

Yes, indeed. It's good to read this would in the end allow people to
contribute more instead of /fighting the system/ ! :-)

> I'll give bit more detail on how patch could allow our company to be
> better contribute back to the buildroot project with this change. During 
> development of a project, we could use git to track mainline and any 
> third party package that we add support for, we would contribute them 
> back upstream. The hope would be that by the time we get to the point
> where we have to lock into a specific version of buildroot, our changes
> would have been incorporated into buildroot. If not, we could move the
> packages that were rejected into the BR2_EXTERNAL location but still
> be able to use an off-the-shelve version of buildroot.

That's funny, because I would have expected exactly the opposite: do
everything in BR2_EXTERNAL, and move them to Buildroot just before
upstreaming them.

But if you (and others) see BR2_EXTERNAL as a faillover for stuff that
could not be upstreamed before you settle on a version of Buildroot,
then I have to say that I'm really happy to read that. :-)

So many companies only upstream their work after-the-fact that it is
indeed a relief to find some that do actually want to push as much as
possible upstream first. Kuddos to you! :-)

> > Ok. Note that after I posted my patches, I had a discussion with Yann
> > Morin on IRC and he was skeptical about it, for two reasons:
> > 
> >  (1) Almost all of what BR2_EXTERNAL allows is possible today with the
> >      current Buildroot. For board level stuff, nothing forces you to
> >      store it in board/<foo>/ within the Buildroot tree: all of the
> >      configuration options in Buildroot can use
> >      $(TOPDIR)/../company/<foo>/ to reference a kernel configuration
> >      file, or a root filesystem overlay.
> > 
> >      For packages, the 'local.mk' file is already automatically
> >      included in Buildroot if it exists. So you can create such a file
> >      and put a "include $(TOPDIR)/../company/external.mk' line in it.
> >      This external.mk will include all the makefiles of the external
> >      packages. Since Config.in can't be integrated as easily, one
> >      solution would be to hardcode BR2_PACKAGE_FOOBAR=y directly within
> >      external.mk to have it always enabled.
> 
> That just becomes nasty and doesn't really easily allow for different 
> configurations where you don't always want all your propriety packages
> to build. Also it doesn't make your propriety packages visible within
> the buildroot menuconfig. The other nice feature is the is that company's
> configs are automatically pulled in a separate area.

The way *I* see this is that Buildroot is not the main /frontend/ of the
build system, but rather a backend. That's the way I use it: I have a
kind of /upper-layer/ build system that uses Buildroot as a kind of
/backend/. I've already introduced it briefly here some time ago, and
wanted to make it a bit more complete before a real annoucement, but
here it is if you want to have a look (it's been public all the time
sicne the beginning  anyway):
    http://ymorin.is-a-geek.org/git/buildroot.config/

(Note, this thread is not about the above, so I think we should not
discuss it further in this thread; let's start another thread if people
are interested).

> >      So, it's already possible today, but I believe the current
> >      situation is (a) not very easy to use, and (b) not visible enough.
> >      (b) could be solved by more documentation, but I believe
> >      BR2_EXTERNAL does not make Buildroot more complicated and solves
> >      (a) quite nicely.
> 
> I definitely agree

Although I do not like the idea on a philosophical point of view, I
agree there is such a need, even more so since you're not the first to
come up with, and try to address, this issue.

So I'm all for making people's lives^Wwork easier, and Thomas' patchset
looks pretty rad (I haven't tested it yet, but I think I could even use
it from my Buildroot.config project! ;-) )

> >  (2) Yann's feeling was that by giving a too easy solution to keep
> >      things out of tree would discourage BR users from contributing
> >      their new packages upstream, since they can keep it outside of the
> >      tree nicely. While I certainly admit it can be the case, your
> >      introduction to this e-mail interestingly points exactly the
> >      opposite :-)
> 
> While I do understand Yann's concerns, however, I do think that this goes
> against buildroot's philosophy - "Buildroot: make Embedded Linux easy". 
> Right now it isn't easy for one to make customization to buildroot without
> modifying buildroot's source.
> 
> Yann - I would welcome you input into this discussion on mailing list 
> here.

Yes, as I said above, I would have expected companies (well, people at
companies) be stressed by projects deadlines, and that they would work
in BR2_EXTERNAL, and only try to upstream later, when it would be too
late (for them!) to have proper reviews, since they would be internally
committed to their changes in BR2_EXTERNAL, and we would not be able to
take their changes.

But if people use BR2_EXTERNAL as a fallback for anything that did not
make it upstream (by lack of time, or of interest), or for purely
internal stuff, then I don't see a reason for concern. All the better! :-)

> I'm probably beating this issue to death but I guess I'm very excited 
> about
> the possibilities that this patchset opens up for my company.

Not speaking for the others, but I for one am very exited by this
patchset too, if it makes people very exited about what it will allow
them to submit back upstream! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/3] Makefile: factorize *config dependencies

[Yann E. MORIN]

Thomas, All,

On 2013-09-08 15:15 +0200, Thomas Petazzoni spake thusly:
> All of the *config targets have common dependencies. Currently, the
> only common dependency is the 'outputmakefile' target, but as we are
> going to add one more, it makes sense to factorize these common
> dependencies under the COMMON_CONFIG_DEPS variable.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

> ---
>  Makefile | 37 ++++++++++++++++++++-----------------
>  1 file changed, 20 insertions(+), 17 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 0d7476a..fc55b87 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -633,43 +633,46 @@ COMMON_CONFIG_ENV = \
>  	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
>  	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG)
>  
> -xconfig: $(BUILD_DIR)/buildroot-config/qconf outputmakefile
> +COMMON_CONFIG_DEPS = \
> +	outputmakefile
> +
> +xconfig: $(BUILD_DIR)/buildroot-config/qconf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
>  
> -gconfig: $(BUILD_DIR)/buildroot-config/gconf outputmakefile
> +gconfig: $(BUILD_DIR)/buildroot-config/gconf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) srctree=$(TOPDIR) $< $(CONFIG_CONFIG_IN)
>  
> -menuconfig: $(BUILD_DIR)/buildroot-config/mconf outputmakefile
> +menuconfig: $(BUILD_DIR)/buildroot-config/mconf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
>  
> -nconfig: $(BUILD_DIR)/buildroot-config/nconf outputmakefile
> +nconfig: $(BUILD_DIR)/buildroot-config/nconf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
>  
> -config: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +config: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< $(CONFIG_CONFIG_IN)
>  
> -oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +oldconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --oldconfig $(CONFIG_CONFIG_IN)
>  
> -randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +randconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
>  
> -allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +allyesconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
>  
> -allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +allnoconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --allnoconfig $(CONFIG_CONFIG_IN)
>  
> -randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +randpackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
>  	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
> @@ -680,7 +683,7 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  		$< --randconfig $(CONFIG_CONFIG_IN)
>  	@rm -f $(CONFIG_DIR)/.config.nopkg
>  
> -allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
>  	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
> @@ -691,7 +694,7 @@ allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  		$< --allyesconfig $(CONFIG_CONFIG_IN)
>  	@rm -f $(CONFIG_DIR)/.config.nopkg
>  
> -allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@grep -v BR2_PACKAGE_ $(BUILDROOT_CONFIG) > $(CONFIG_DIR)/.config.nopkg
>  	@$(COMMON_CONFIG_ENV) \
> @@ -699,23 +702,23 @@ allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  		$< --allnoconfig $(CONFIG_CONFIG_IN)
>  	@rm -f $(CONFIG_DIR)/.config.nopkg
>  
> -silentoldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +silentoldconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	$(COMMON_CONFIG_ENV) $< --silentoldconfig $(CONFIG_CONFIG_IN)
>  
> -olddefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +olddefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN)
>  
> -defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +defconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
>  
> -%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
> +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
>  
> -savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
> +savedefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>  	@mkdir -p $(BUILD_DIR)/buildroot-config
>  	@$(COMMON_CONFIG_ENV) $< \
>  		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
> -- 
> 1.8.1.2
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas De Schampheleire]

Hi,

On Wed, Sep 11, 2013 at 7:27 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Ryan, Thomas, All,
>
[..]
>
> Yes, as I said above, I would have expected companies (well, people at
> companies) be stressed by projects deadlines, and that they would work
> in BR2_EXTERNAL, and only try to upstream later, when it would be too
> late (for them!) to have proper reviews, since they would be internally
> committed to their changes in BR2_EXTERNAL, and we would not be able to
> take their changes.
>
> But if people use BR2_EXTERNAL as a fallback for anything that did not
> make it upstream (by lack of time, or of interest), or for purely
> internal stuff, then I don't see a reason for concern. All the better! :-)
>

Not to ruin the atmosphere, but even though Ryan and his colleagues
may be using BR2_EXTERNAL 'the good way', ultimately helping the
buildroot project with their contributions, the danger of others not
doing that is still real. Of course, _not_ contributing changes made
in a company project was also possible previously, without
BR2_EXTERNAL, so this is no reason to block the patchset.

In case it may be useful to others: here is how I am using buildroot
in a company environment. We are not directly tracking mainline, but
base ourselves on release tarballs. We have one Mercurial repository
in which each of these release tarballs are extracted (the pristine
repository). In this repository, no changes are made other than the
extracting. Since these are release tarballs, there is obviously no
information about the individual commits made in buildroot git, but
this is generally not a problem. In case we do want to know who made a
particular change, we can still look that up in buildroot git.
A second repository is cloned from this first one, and does contain
our own changes. This two-repository approach is similar to having a
pristine branch and a master branch for development, it's a matter of
choice.
Our changes are mainly in package/company, board/company and configs.
This makes updating to a new buildroot release pretty straightforward:
first the pristine repo is updated with a new tarball. Then, these
changes are pulled into the development repo, and merged. Most merges
will be trivial because most changes are in separate files.
If necessary, changes made upstream are imported as separate commits
on top of the releases, but this happens only occasionally.

Of course, we do sometimes make changes in standard buildroot files.
These can be new non-proprietary packages, or other changes throughout
the tree. We take care to keep this upstreamable, and do effectively
upstream when/where possible. It are these files that can pose a merge
conflict, because for example the implementation originally committed
is slightly different from the one accepted upstream. Or maybe, the
patch has not yet been accepted upstream when a new release is taken
in.
The upstreaming work is currently done by one person (me) and this is
clearly the weak point. If I stop paying attention to upstreaming, or
to keeping the changes we make upstreamable, the delta between our
development repo and mainline buildroot becomes so large that it is a
nightmare to keep up with newer releases.

In our overall build process, at some point the buildroot repo will be
cloned at the right revision, the relevant config set, and 'make' will
be run in buildroot. Afterwards, the result files will be collected
from output/images and post-processed into the right format to put on
our boards. One could consider buildroot here as a backend, as Yann
referred to.

Upstreaming is also done with Mercurial: I convert buildroot git to hg
using the hg-git extension. Patches are directly sent from Mercurial
through the patchbomb extension. The only git command I execute is
'git pull' in the git repo, and possible 'git checkout next' during
the time there is a master and active next branch.


Best regards,
Thomas

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Wed, 11 Sep 2013 19:27:09 +0200, Yann E. MORIN wrote:

> > That just becomes nasty and doesn't really easily allow for different 
> > configurations where you don't always want all your propriety packages
> > to build. Also it doesn't make your propriety packages visible within
> > the buildroot menuconfig. The other nice feature is the is that company's
> > configs are automatically pulled in a separate area.
> 
> The way *I* see this is that Buildroot is not the main /frontend/ of the
> build system, but rather a backend. That's the way I use it: I have a
> kind of /upper-layer/ build system that uses Buildroot as a kind of
> /backend/.

I think that's one way of using it, but that's not necessarily the only
way we should support. In all the Buildroot based projects I've done,
I've also tried to make Buildroot directly be the front-end, and
include whatever build logic is needed within Buildroot.

I can only remember of one project in which I did not do that, and it
was because the customer had already written his build logic using
Buildroot as a back-end.

> Yes, as I said above, I would have expected companies (well, people at
> companies) be stressed by projects deadlines, and that they would work
> in BR2_EXTERNAL, and only try to upstream later, when it would be too
> late (for them!) to have proper reviews, since they would be internally
> committed to their changes in BR2_EXTERNAL, and we would not be able to
> take their changes.
> 
> But if people use BR2_EXTERNAL as a fallback for anything that did not
> make it upstream (by lack of time, or of interest), or for purely
> internal stuff, then I don't see a reason for concern. All the better! :-)

If companies are in a hurry, they are not going to upstream anything
anyway, so we're not loosing something. Today, without BR2_EXTERNAL,
companies can already fork Buildroot, do a massive amount of changes
directly within Buildroot, and not upstream anything at all. Therefore,
adding BR2_EXTERNAL is not going to make things worse.

I believe it could potentially make things better. It might allow
companies to separate what they consider "company-specific" in the
BR2_EXTERNAL directory, while still making the Buildroot core changes,
or open-source package additions in the main Buildroot tree, and submit
those changes upstream. It allows them to keep cleanly separated the
changes that will never be upstreamed because it doesn't make sense
(specific board support, highly specific and proprietary packages) from
the things that might be upstreamed if the submission effort is made.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Thu, 12 Sep 2013 09:54:56 +0200, Thomas De Schampheleire wrote:

> > Yes, as I said above, I would have expected companies (well, people at
> > companies) be stressed by projects deadlines, and that they would work
> > in BR2_EXTERNAL, and only try to upstream later, when it would be too
> > late (for them!) to have proper reviews, since they would be internally
> > committed to their changes in BR2_EXTERNAL, and we would not be able to
> > take their changes.
> >
> > But if people use BR2_EXTERNAL as a fallback for anything that did not
> > make it upstream (by lack of time, or of interest), or for purely
> > internal stuff, then I don't see a reason for concern. All the better! :-)
> >
> 
> Not to ruin the atmosphere, but even though Ryan and his colleagues
> may be using BR2_EXTERNAL 'the good way', ultimately helping the
> buildroot project with their contributions, the danger of others not
> doing that is still real. Of course, _not_ contributing changes made
> in a company project was also possible previously, without
> BR2_EXTERNAL, so this is no reason to block the patchset.

Yes, that's what I believe: companies and people are forking Buildroot
nowadays. As I highlighted in my reply to Yann's e-mail, BR2_EXTERNAL
may in fact encourage companies in separating their specific changes
from the changes that could potentially be upstreamed, and therefore
encourage upstream.

I really don't think BR2_EXTERNAL is going to make things worse in
terms of upstreaming.

> In case it may be useful to others: here is how I am using buildroot
> in a company environment. We are not directly tracking mainline, but
> base ourselves on release tarballs. We have one Mercurial repository
> in which each of these release tarballs are extracted (the pristine
> repository). In this repository, no changes are made other than the
> extracting. Since these are release tarballs, there is obviously no
> information about the individual commits made in buildroot git, but
> this is generally not a problem. In case we do want to know who made a
> particular change, we can still look that up in buildroot git.
> A second repository is cloned from this first one, and does contain
> our own changes. This two-repository approach is similar to having a
> pristine branch and a master branch for development, it's a matter of
> choice.
> Our changes are mainly in package/company, board/company and configs.
> This makes updating to a new buildroot release pretty straightforward:
> first the pristine repo is updated with a new tarball. Then, these
> changes are pulled into the development repo, and merged. Most merges
> will be trivial because most changes are in separate files.
> If necessary, changes made upstream are imported as separate commits
> on top of the releases, but this happens only occasionally.
> 
> Of course, we do sometimes make changes in standard buildroot files.
> These can be new non-proprietary packages, or other changes throughout
> the tree. We take care to keep this upstreamable, and do effectively
> upstream when/where possible. It are these files that can pose a merge
> conflict, because for example the implementation originally committed
> is slightly different from the one accepted upstream. Or maybe, the
> patch has not yet been accepted upstream when a new release is taken
> in.
> The upstreaming work is currently done by one person (me) and this is
> clearly the weak point. If I stop paying attention to upstreaming, or
> to keeping the changes we make upstreamable, the delta between our
> development repo and mainline buildroot becomes so large that it is a
> nightmare to keep up with newer releases.

Do you think there is something we can do at the Buildroot level to
ease this process?

> In our overall build process, at some point the buildroot repo will be
> cloned at the right revision, the relevant config set, and 'make' will
> be run in buildroot. Afterwards, the result files will be collected
> from output/images and post-processed into the right format to put on
> our boards. One could consider buildroot here as a backend, as Yann
> referred to.
> 
> Upstreaming is also done with Mercurial: I convert buildroot git to hg
> using the hg-git extension. Patches are directly sent from Mercurial
> through the patchbomb extension. The only git command I execute is
> 'git pull' in the git repo, and possible 'git checkout next' during
> the time there is a master and active next branch.

Thanks a lot for this write-up. I think we should keep it somewhere in
a section of the manual.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[ANDY KENNEDY]

> -----Original Message-----
> From: buildroot-bounces at busybox.net [mailto:buildroot-bounces at busybox.net] On Behalf Of Thomas
> Petazzoni
> Sent: Thursday, September 12, 2013 1:22 PM
> To: Thomas De Schampheleire
> Cc: buildroot at uclibc.org; Yann E. MORIN
> Subject: Re: [Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization
> 
> Dear Thomas De Schampheleire,
> 
> On Thu, 12 Sep 2013 09:54:56 +0200, Thomas De Schampheleire wrote:
> 
> > > Yes, as I said above, I would have expected companies (well, people at
> > > companies) be stressed by projects deadlines, and that they would work
> > > in BR2_EXTERNAL, and only try to upstream later, when it would be too
> > > late (for them!) to have proper reviews, since they would be internally
> > > committed to their changes in BR2_EXTERNAL, and we would not be able to
> > > take their changes.
> > >
> > > But if people use BR2_EXTERNAL as a fallback for anything that did not
> > > make it upstream (by lack of time, or of interest), or for purely
> > > internal stuff, then I don't see a reason for concern. All the better! :-)
> > >
> >
> > Not to ruin the atmosphere, but even though Ryan and his colleagues
> > may be using BR2_EXTERNAL 'the good way', ultimately helping the
> > buildroot project with their contributions, the danger of others not
> > doing that is still real. Of course, _not_ contributing changes made
> > in a company project was also possible previously, without
> > BR2_EXTERNAL, so this is no reason to block the patchset.
> 
> Yes, that's what I believe: companies and people are forking Buildroot
> nowadays. As I highlighted in my reply to Yann's e-mail, BR2_EXTERNAL
> may in fact encourage companies in separating their specific changes
> from the changes that could potentially be upstreamed, and therefore
> encourage upstream.
> 
> I really don't think BR2_EXTERNAL is going to make things worse in
> terms of upstreaming.

<snip>

This sounds much like the way Linux does things for the IP related
drivers.  Is that the intent we are going for (personally, I think this
is a GREAT idea, as it allows companies to have IP related widgets in
BuildRoot without the fear of being REQUIRED to push back their secret
sauce)?

If that is what you intend, you have my vote!

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Dear ANDY KENNEDY,

On Thu, 12 Sep 2013 18:25:40 +0000, ANDY KENNEDY wrote:

> This sounds much like the way Linux does things for the IP related
> drivers.  Is that the intent we are going for (personally, I think this
> is a GREAT idea, as it allows companies to have IP related widgets in
> BuildRoot without the fear of being REQUIRED to push back their secret
> sauce)?
> 
> If that is what you intend, you have my vote!

I am not sure what you mean by "IP related drivers". Do you mean
proprietary drivers?

It is true that the BR2_EXTERNAL thing raises a licensing question:
should the BR2_EXTERNAL contents also be released under GPLv2, like the
rest of Buildroot? Do we really want the root filesystem overlays and
other highly project-specific contents be released under GPLv2 ?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[ANDY KENNEDY]

> -----Original Message-----
> From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com]
> Sent: Thursday, September 12, 2013 1:34 PM
> To: ANDY KENNEDY
> Cc: Thomas De Schampheleire; buildroot at uclibc.org; Yann E. MORIN
> Subject: Re: [Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization
> 
> Dear ANDY KENNEDY,
> 
> On Thu, 12 Sep 2013 18:25:40 +0000, ANDY KENNEDY wrote:
> 
> > This sounds much like the way Linux does things for the IP related
> > drivers.  Is that the intent we are going for (personally, I think this
> > is a GREAT idea, as it allows companies to have IP related widgets in
> > BuildRoot without the fear of being REQUIRED to push back their secret
> > sauce)?
> >
> > If that is what you intend, you have my vote!
> 
> I am not sure what you mean by "IP related drivers". Do you mean
> proprietary drivers?

Sorry, I thought that was a universal acronym.  Yes, IP is
Intellectual Property.

> 
> It is true that the BR2_EXTERNAL thing raises a licensing question:
> should the BR2_EXTERNAL contents also be released under GPLv2, like the
> rest of Buildroot? Do we really want the root filesystem overlays and
> other highly project-specific contents be released under GPLv2 ?

Whether one uses BuildRoot as their build system, I would think, would
not be a concern of those of us who work on BuildRoot.  Am I the only
one that feels this way?  I see Linux, and derivations of it, in this
case BuildRoot, as an ends to a means.  I hold the opinion that, if I
have added some minor tool like htop to the RFS, and someone can then
make a butt-load of money by my work then KUDOS TO THEM and shame on me
for not thinking of it in the first place.

I know that there are those that believe that if I walked near a person
who was thinking of GPL related code and I get an idea (not necessarily
related to what the individual was doing), that this idea should then be
GPL code as well (this seems to be the intent of GPLv3, though I stretch
the scenario somewhat).  I don't hold that view.  If I release something
back to BuildRoot, I'm doing so in the spirit in which I used BuildRoot
to begin with:  I am giving back to a community of friends that help me
make money.  If I can help you make money in return, I am happy to have
helped you.

Am I alone in this thinking?  I hope not.

Andy

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Arnout Vandecappelle]

  Hi Thomas,

  It's funny how people can change their mind...

  I've put Tzu-Jung Lee in CC, who wrote something similar a while ago.

On 08/09/13 15:15, Thomas Petazzoni wrote:
> This commit adds support for an environment variable named
> BR2_EXTERNAL, which the user can point to a directory outside of
> Buildroot that contains root filesystem overlays, kernel configuration
> files, package recipes, defconfigs, etc. It allows users to keep their
> specific Buildroot customizations outside of the Buildroot tree.
>
> BR2_EXTERNAL allows:

  I would split this into three separate patches for the three separate 
features. I think (1) and (3) are a lot less controversial than (2).

>
>   (1) To use $(BR2_EXTERNAL) in all Buildroot configuration options
>       that take a path as argument. This allows to reference a root
>       filesystem overlay, a kernel configuration file, a Barebox
>       configuration file located in the BR2_EXTERNAL directory.

  Note to list: this was already possible before, of course, the 
difference is that now this path will be stored in the .config.

  There is one small issue with this: I don't think that all the use 
cases support relative paths. Ideally, the Makefile should be smart 
enough to prepend $(TOPDIR) if it doesn't start with a /.


>   (2) To store external package or makefile logic, since the
>       BR2_EXTERNAL/external.mk file is automatically included by the
>       Makefile logic, and the BR2_EXTERNAL/Config.in file is
>       automatically included, and appears in the top-level menu. The
>       typical usage would be to create a BR2_EXTERNAL/package/
>       directory to contain package definitions.

  I'm not really convinced by the principle. The external.mk is exactly 
the same as the local override .mk; the Config.in appears at the very end 
of the top-level menu. Instead, I think we should enforce the buildroot 
hierarchy in the external dir, i.e.

source <path-to-external-dir>/package/Config.in

at the top of package/Config.in, and

-include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))

in the top-level Makefile.

  Of course, that Config.in thing is a lot trickier because it's almost 
impossible to refer to the output directory from the Config.in.

>   (3) To store defconfig files under BR2_EXTERNAL/configs/. They will
>       be visible in 'make help' and usable through 'make
>       <someboard>_defconfig'.

  That's a great idea.


> In terms of implementation, it is relatively straightforward:
>
>   * A BR2_EXTERNAL kconfig variable is added, which is set to the value
>     of the BR2_EXTERNAL environment variable.
>
>   * The top-level Config.in file given to kconfig is no longer the main
>     Config.in in the Buildroot sources, but instead a toplevel.in file
>     generated in the output directory, which includes the top-level
>     Buildroot Config.in file and the BR2_EXTERNAL/Config.in file if
>     provided. Since is needed because the kconfig 'source' statement
>     errors out if the included file doesn't exist. I have written
>     patches that add support for an 'isource' statement in kconfig
>     (that silently ignores the inclusion if the pointed file doesn't
>     exist), but keeping feature patches against kconfig doesn't seem
>     like a good idea.

  It took me a while to realize that that was _not_ the approach you had 
taken...

>     Note that the "mainmenu" statement is part of
>     this generated file, because it must be the first statement seen in
>     the toplevel Config.in file passed to kconfig.
>
>   * The BR2_EXTERNAL/external.mk makefile gets included.
>
>   * The BR2_EXTERNAL environment variable gets passed in the
>     environment of kconfig and when executing the kconfiglib-based
>     Python script that updates the manual, so that the references to
>     the BR2_EXTERNAL variable within Config.in files can be resolved.

  So this means that the external packages will appear in the package 
list in the manual? I'm not sure if that is what you typically want.


>   * The 'make help' and 'make %_defconfig' targets are updated to take
>     into account the defconfig files stored under
>     BR2_EXTERNAL/configs/.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>   Config.in             |  6 ++++--
>   Makefile              | 29 +++++++++++++++++++++++++----
>   docs/manual/manual.mk |  2 +-
>   3 files changed, 30 insertions(+), 7 deletions(-)
>
> diff --git a/Config.in b/Config.in
> index cb246a4..7e253c6 100644
> --- a/Config.in
> +++ b/Config.in
> @@ -1,7 +1,5 @@
>   #
>
> -mainmenu "Buildroot $BR2_VERSION Configuration"
> -
>   config BR2_HAVE_DOT_CONFIG
>   	bool
>   	default y
> @@ -14,6 +12,10 @@ config BR2_HOSTARCH
>   	string
>   	option env="HOSTARCH"
>
> +config BR2_EXTERNAL
> +	string
> +	option env="BR2_EXTERNAL"
> +
>   # Hidden boolean selected by pre-built packages for x86, when they
>   # need to run on x86-64 machines (example: pre-built external
>   # toolchains, binary tools like SAM-BA, etc.).
> diff --git a/Makefile b/Makefile
> index fc55b87..1014399 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -47,7 +47,6 @@ export HOSTARCH := $(shell uname -m | \
>
>   # absolute path
>   TOPDIR:=$(shell pwd)
> -CONFIG_CONFIG_IN=Config.in
>   CONFIG=support/kconfig
>   DATE:=$(shell date +%Y%m%d)
>
> @@ -339,6 +338,10 @@ include boot/common.mk
>   include linux/linux.mk
>   include system/system.mk
>
> +ifneq ($(BR2_EXTERNAL),)
> +include $(BR2_EXTERNAL)/external.mk
> +endif
> +
>   TARGETS+=target-finalize
>
>   ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)
> @@ -622,6 +625,18 @@ $(BUILD_DIR)/buildroot-config/%onf:
>   	mkdir -p $(@D)/lxdialog
>   	$(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)" obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
>
> +CONFIG_CONFIG_IN=$(BUILD_DIR)/buildroot-config/toplevel.in
> +
> +# This is intentionally a virtual target so that the file gets
> +# regenerated everytime this target is invoked.
> +toplevelin-generate:

  I guess you mean phony target. You should also add
.PHONY: toplevelin-generate
for the (unlikely) case that someone creates a file with this name.

  BTW I don't like the target's name. generate-config-dot-in maybe?

> +	mkdir -p $(dir $(CONFIG_CONFIG_IN))
> +	echo "mainmenu \"Buildroot $$BR2_VERSION Configuration\"" > $(CONFIG_CONFIG_IN)
> +	echo "source \"Config.in\"" >> $(CONFIG_CONFIG_IN)
> +ifneq ($(BR2_EXTERNAL),)
> +	echo "source \"$$BR2_EXTERNAL/Config.in\"" >> $(CONFIG_CONFIG_IN)

  $(BR2_EXTERNAL) would be a lot more readable than $$BR2_EXTERNAL IMHO. 
Then you can also put it in single quotes and remove the \.

> +endif
> +
>   DEFCONFIG = $(call qstrip,$(BR2_DEFCONFIG))
>
>   # We don't want to fully expand BR2_DEFCONFIG here, so Kconfig will
> @@ -631,10 +646,12 @@ COMMON_CONFIG_ENV = \
>   	KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \
>   	KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \
>   	KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
> -	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG)
> +	BUILDROOT_CONFIG=$(BUILDROOT_CONFIG) \
> +	BR2_EXTERNAL=$(BR2_EXTERNAL)

  There is one tricky issue:

cd ~/src/myproject/output
make -C ~/src/buildroot O=$PWD menuconfig
Hack hack hack
Hm, I need an external dir...
BR2_EXTERNAL=.. make menuconfig
=> BR2_EXTERNAL will point to ~/src instead of ~/src/myproject

  I.e., it doesn't work well for relative paths.

  I don't know if there is an elegant way to solve that.


>   COMMON_CONFIG_DEPS = \
> -	outputmakefile
> +	outputmakefile \
> +	toplevelin-generate
>
>   xconfig: $(BUILD_DIR)/buildroot-config/qconf $(COMMON_CONFIG_DEPS)
>   	@mkdir -p $(BUILD_DIR)/buildroot-config
> @@ -718,6 +735,10 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>   	@mkdir -p $(BUILD_DIR)/buildroot-config
>   	@$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
>
> +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig $(COMMON_CONFIG_DEPS)
> +	@mkdir -p $(BUILD_DIR)/buildroot-config
> +	@$(COMMON_CONFIG_ENV) $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)

  This should be in a
ifneq ($(BR2_EXTERNAL),)


  Regards,
  Arnout

> +
>   savedefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>   	@mkdir -p $(BUILD_DIR)/buildroot-config
>   	@$(COMMON_CONFIG_ENV) $< \
> @@ -829,7 +850,7 @@ endif
>   	@echo '  make V=0|1             - 0 => quiet build (default), 1 => verbose build'
>   	@echo '  make O=dir             - Locate all output files in "dir", including .config'
>   	@echo
> -	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \
> +	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig $(BR2_EXTERNAL)/configs/*_defconfig))), \
>   	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
>   	@echo
>   	@echo 'See docs/README, or generate the Buildroot manual for further details'
> diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
> index 4906bc8..8e0ab30 100644
> --- a/docs/manual/manual.mk
> +++ b/docs/manual/manual.mk
> @@ -1,6 +1,6 @@
>   manual-update-lists:
>   	$(Q)$(call MESSAGE,"Updating the manual lists...")
> -	$(Q)BR2_DEFCONFIG="" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \
> +	$(Q)BR2_DEFCONFIG="" BR2_EXTERNAL="$(BR2_EXTERNAL)" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \
>   		$(TOPDIR)/support/scripts/gen-manual-lists.py
>
>   ################################################################################
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Arnout Vandecappelle]

On 11/09/13 04:03, rjbarnet at rockwellcollins.com wrote:
> Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on
> 09/08/2013 08:15:28 AM:
>
>  >
>  > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>  > ---
>  >  Config.in   |  6 ++++--
>  >  Makefile    | 29 +++++++++++++++++++++++++----
>  >  docs/manual/manual.mk |  2 +-
>
> One idea that I have, is for the make file that is genreated for the an
> external output directory, if we could preserve the the value of
> BR2_EXTERNAL
> as follows.

  Thomas's patch already does that, because BR2_EXTERNAL is stored in the 
config file.

> I don't think my logic is fully there but there is more of a
> sudo implementation:
>
> diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
> --- a/support/scripts/mkmakefile
> +++ b/support/scripts/mkmakefile
> @@ -27,6 +27,7 @@ makedir := \$(dir \$(call lastword,\$(MAKEFILE_LIST)))
>
>   MAKEARGS := -C $1
>   MAKEARGS += O=\$(if \$(patsubst
> /%,,\$(makedir)),\$(CURDIR)/)\$(patsubst %/,%,\$(makedir))
> +MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))
>
>   MAKEFLAGS += --no-print-directory
>
> Again this is only a suggestion but otherwise I'll ack the patchset.
>
> Thanks,
> -Ryan
>
> -------------------------------------------------------------------------
> 	*Ryan J Barnett* / Software Engineer / Platform SW
> MS 137-157, 855 35th St NE, Cedar Rapids, IA, 52498-3161, US
> Phone: 319-263-3880 / VPN: 263-3880 _
> __rjbarnet at rockwellcollins.com_ <mailto:rjbarnet@rockwellcollins.com>_
> __www.rockwellcollins.com_ <http://www.rockwellcollins.com/>
>
>
>
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Ryan Barnett]

Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:05:18 PM:

> On 11/09/13 04:03, rjbarnet at rockwellcollins.com wrote:
> > Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on
> > 09/08/2013 08:15:28 AM:
> >
> >  >
> >  > Signed-off-by: Thomas Petazzoni 
<thomas.petazzoni@free-electrons.com>
> >  > ---
> >  >  Config.in   |  6 ++++--
> >  >  Makefile    | 29 +++++++++++++++++++++++++----
> >  >  docs/manual/manual.mk |  2 +-
> >
> > One idea that I have, is for the make file that is genreated for the 
an
> > external output directory, if we could preserve the the value of
> > BR2_EXTERNAL
> > as follows.
> 
>   Thomas's patch already does that, because BR2_EXTERNAL is stored in 
the 
> config file.
> 

I don't believe it gets stored in the config since Config.in pulls
the variable in from the shell's or make's environment. 

+config BR2_EXTERNAL
+       string
+       option env="BR2_EXTERNAL"

In order to utilize the BR2_EXTERNAL feature I must have BR2_EXTERNAL
either exported into my environment or be called as 

'make BR2_EXTERNAL=/some/path <target>'


When utilizing the BR2_EXTERNAL I usual want to use some defconfig from
my BR2_EXTERNAL so it adding it the makefile that is generated in your 
build directory when specifying O= would be extermely useful as that 
way I can switch shells and my definition of BR2_EXTERNAL would still 
be present.

'make O=/build/path BR2_EXTERNAL=/path/to/external defconfig'

> > I don't think my logic is fully there but there is more of a
> > sudo implementation:
> >
> > diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
> > --- a/support/scripts/mkmakefile
> > +++ b/support/scripts/mkmakefile
> > @@ -27,6 +27,7 @@ makedir := \$(dir \$(call 
lastword,\$(MAKEFILE_LIST)))
> >
> >   MAKEARGS := -C $1
> >   MAKEARGS += O=\$(if \$(patsubst
> > /%,,\$(makedir)),\$(CURDIR)/)\$(patsubst %/,%,\$(makedir))
> > +MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))
> >
> >   MAKEFLAGS += --no-print-directory
> >
> > Again this is only a suggestion but otherwise I'll ack the patchset.
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/e06f840b/attachment.html>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Arnout Vandecappelle]

On 12/09/13 23:30, Ryan Barnett wrote:
> Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:05:18 PM:
>
>  > On 11/09/13 04:03, rjbarnet at rockwellcollins.com wrote:
>  > > Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on
>  > > 09/08/2013 08:15:28 AM:
>  > >
>  > >  >
>  > >  > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>  > >  > ---
>  > >  >  Config.in   |  6 ++++--
>  > >  >  Makefile    | 29 +++++++++++++++++++++++++----
>  > >  >  docs/manual/manual.mk |  2 +-
>  > >
>  > > One idea that I have, is for the make file that is genreated for the an
>  > > external output directory, if we could preserve the the value of
>  > > BR2_EXTERNAL
>  > > as follows.
>  >
>  >   Thomas's patch already does that, because BR2_EXTERNAL is stored in the
>  > config file.
>  >
>
> I don't believe it gets stored in the config since Config.in pulls
> the variable in from the shell's or make's environment.
>
> +config BR2_EXTERNAL
> +       string
> +       option env="BR2_EXTERNAL"
>
> In order to utilize the BR2_EXTERNAL feature I must have BR2_EXTERNAL
> either exported into my environment or be called as

  The Makefile exports it in the environment of the Kconfig targets. So 
once it is in the .config, it will stay there. Try it and you'll see :-)

>
> 'make BR2_EXTERNAL=/some/path <target>'
>
>
> When utilizing the BR2_EXTERNAL I usual want to use some defconfig from
> my BR2_EXTERNAL so it adding it the makefile that is generated in your
> build directory when specifying O= would be extermely useful as that
> way I can switch shells and my definition of BR2_EXTERNAL would still
> be present.
>
> 'make O=/build/path BR2_EXTERNAL=/path/to/external defconfig'

  In this particular case, you're not using the Makefile in the output 
dir but only in the buildroot dir so it's not relevant.

  The only case where it would be relevant is when you have an existing 
output directory, you remove the .config first, and then you do a
make foo_defconfig. Seems a bit an unlikely scenario.

  Regards,
  Arnout

>
>  > > I don't think my logic is fully there but there is more of a
>  > > sudo implementation:
>  > >
>  > > diff --git a/support/scripts/mkmakefile b/support/scripts/mkmakefile
>  > > --- a/support/scripts/mkmakefile
>  > > +++ b/support/scripts/mkmakefile
>  > > @@ -27,6 +27,7 @@ makedir := \$(dir \$(call lastword,\$(MAKEFILE_LIST)))
>  > >
>  > >   MAKEARGS := -C $1
>  > >   MAKEARGS += O=\$(if \$(patsubst
>  > > /%,,\$(makedir)),\$(CURDIR)/)\$(patsubst %/,%,\$(makedir))
>  > > +MAKEARGS += BR2_EXTERNAL=\$(realpath \$(BR2_EXTERNAL))
>  > >
>  > >   MAKEFLAGS += --no-print-directory
>  > >
>  > > Again this is only a suggestion but otherwise I'll ack the patchset.
>  > >
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 3/3] docs/manual: add explanations about BR2_EXTERNAL

[Arnout Vandecappelle]

On 08/09/13 15:15, Thomas Petazzoni wrote:
> This commit updates the manual to add details on how to use the
> BR2_EXTERNAL feature.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>   docs/manual/customize-outside-br.txt | 110 +++++++++++++++++++++++++++++++++++
>   docs/manual/customize.txt            |   2 +
>   2 files changed, 112 insertions(+)
>   create mode 100644 docs/manual/customize-outside-br.txt
>
> diff --git a/docs/manual/customize-outside-br.txt b/docs/manual/customize-outside-br.txt
> new file mode 100644
> index 0000000..81a8911
> --- /dev/null
> +++ b/docs/manual/customize-outside-br.txt
> @@ -0,0 +1,110 @@
> +// -*- mode:doc -*- ;
> +
> +[[outside-br-custom]]
> +Keeping customization outside Buildroot
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +While the Buildroot community recommends and encourages upstreaming to
> +the official Buildroot version the packages and boards support that
> +are written by developers, it is sometimes not possible or desirable,
> +due to these packages or boards being highly specific or proprietary.
> +
> +In this case, Buildroot users are offered two choices:
> +
> + * They can add their packages, board support and configuration files
> +   directly within the Buildroot tree, and maintain them by using
> +   branches in a version control system.
> +
> + * They can use the +BR2_EXTERNAL+ mechanism, which allows to keep
> +   package recipes, board support and configuration files outside of
> +   the Buildroot tree, while still having them nicely integrated in
> +   the build logic. The following paragraphs give details on how to
> +   use +BR2_EXTERNAL+.
> +
> ++BR2_EXTERNAL+ is an environment variable that one can use to point to
> +a directory that contains Buildroot customizations. This environment
> +variable must be passed everytime a Buildroot +make+ command is
> +invoked, be it at configuration time (+make menuconfig+, +make
> +xconfig+, etc.), at build time (+make+) or any other Buildroot command
> +(+make source+, +make legal-info+, etc.).

  Hang on... It's stored in the .config, right? So no need to pass it 
every time?

  In fact, if the config option would get a prompt, you could even enter 
it through the config instead of through the environment. The only reason 
to pass it through the environment is to be able to list the available 
defconfigs.

  Or am I missing something?

> +
> +Examples:
> +
> +-----
> + buildroot/ $ make BR2_EXTERNAL=../company menuconfig
> + buildroot/ $ make BR2_EXTERNAL=../company
> +-----
> +
> +This +BR2_EXTERNAL+ then allows three different things:
> +
> + * One can store all the board-specific configuration files here, such
> +   as the kernel configuration, the root filesystem overlay, or any
> +   other configuration file for which Buildroot allows to set its
> +   location. The +BR2_EXTERNAL+ value is available within the
> +   Buildroot configuration using +$(BR2_EXTERNAL)+. As an example, one
> +   could set the +BR2_ROOTFS_OVERLAY+ Buildroot option to
> +   +$(BR2_EXTERNAL)/board/someboard/overlay/+ (to specify a root
> +   filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+
> +   Buildroot option to +$(BR2_EXTERNAL)/board/someboard/kernel.config+
> +   (to specify the location of the kernel configuration file).
> ++
> +To achieve this, it is recommended but not mandatory, to store those
> +   details in directories called +board/<boardname>/+ under

  You use someboard instead of <boardname> earlier.

> +   +BR2_EXTERNAL+.
> +
> + * One can store package recipes (i.e +Config.in+ and
> +   +<packagename>.mk+), or even custom configuration options and make
> +   logic. Buildroot automatically includes +BR2_EXTERNAL/Config.in+ to
> +   make it appear in the top-level configuration menu, and includes
> +   +BR2_EXTERNAL/external.mk+ with the rest of the makefile logic.
> ++
> +The main usage of this would be to store package recipes. The
> +   recommended way to do this is to write a +BR2_EXTERNAL/Config.in+
> +   that looks like:
> ++
> +------
> +menu "<somecompany> packages"
> +
> +source "$BR2_EXTERNAL/package/package1/Config.in"
> +source "$BR2_EXTERNAL/package/package2/Config.in"

  Wow, does this work? Then actually, you don't need that generated 
Config.in at all... Instead, you can add

source "$BR2_EXTERNAL/package/Config.in"

to the top of package/Config.in. And set a non-empty default for 
BR2_EXTERNAL, e.g. $(TOPDIR)/support/dummy_external, which contains an 
empty package/Config.in.

  Regards,
  Arnout

> +
> +endmenu
> +------
> ++
> +Then, have a +BR2_EXTERNAL/external.mk' file that looks like:
> ++
> +------
> +include $(BR2_EXTERNAL)/package/*/*.mk
> +------
> ++
> +And then in +BR2_EXTERNAL/package/package1+ and
> +   +BR2_EXTERNAL/package/package2+ create normal Buildroot package
> +   recipes, as explained in xref:adding-packages[].
> +
> + * One can store Buildroot defconfigs in the +configs+ subdirectory of
> +   +BR2_EXTERNAL+. Buildroot will automatically show them in the
> +   output of +make help+ and allow them to be loaded with the normal
> +   +make <name>_defconfig+ command.
> +
> +In the end, a typical +BR2_EXTERNAL+ directory organization would
> +generally be:
> +
> +-----
> +??? Config.in
> +??? external.mk
> +??? board/
> +?   ??? <someboard>/
> +?       ??? overlay/
> +?           ??? etc/
> +?               ??? <some file>
> +??? configs/
> +?   ??? <some board>_defconfig
> +??? package/
> +    ??? Config.in
> +    ??? package1/
> +        ??? Config.in
> +        ??? package1.mk
> +    ??? package2/
> +        ??? Config.in
> +        ??? package2.mk
> +------
> diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
> index 0456ef1..7e46fd8 100644
> --- a/docs/manual/customize.txt
> +++ b/docs/manual/customize.txt
> @@ -17,3 +17,5 @@ include::customize-toolchain.txt[]
>   include::customize-store.txt[]
>
>   include::customize-packages.txt[]
> +
> +include::customize-outside-br.txt[]
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Thomas, All,

On 2013-09-12 09:54 +0200, Thomas De Schampheleire spake thusly:
> On Wed, Sep 11, 2013 at 7:27 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > Ryan, Thomas, All,
> >
> [..]
> >
> > Yes, as I said above, I would have expected companies (well, people at
> > companies) be stressed by projects deadlines, and that they would work
> > in BR2_EXTERNAL, and only try to upstream later, when it would be too
> > late (for them!) to have proper reviews, since they would be internally
> > committed to their changes in BR2_EXTERNAL, and we would not be able to
> > take their changes.
> >
> > But if people use BR2_EXTERNAL as a fallback for anything that did not
> > make it upstream (by lack of time, or of interest), or for purely
> > internal stuff, then I don't see a reason for concern. All the better! :-)
> >
> 
> Not to ruin the atmosphere, but even though Ryan and his colleagues
> may be using BR2_EXTERNAL 'the good way', ultimately helping the
> buildroot project with their contributions, the danger of others not
> doing that is still real.

Yes, of course. Don't stop me from being optimistic for once, please! ;-)

> Of course, _not_ contributing changes made
> in a company project was also possible previously, without
> BR2_EXTERNAL, so this is no reason to block the patchset.

Quite the opposite: Thomas P. did convince me on IRC and with this
patchset (which I have already started to ack). I still have to ack
patch 2, and did not do so since I haven't yet reviewed and tested here
first.

So, no, I will not try to block this, but rather try to push it since it
is a oft-wanted feature, and Thomas P has come up with a rather sane
solution to do it the Buildroot way. This will eventually stop people
from banging their heads on this for good. :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Ryan Barnett]

Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:41:49 PM:

> On 12/09/13 23:30, Ryan Barnett wrote:
> > Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:05:18 PM:
> >
> >  > On 11/09/13 04:03, rjbarnet at rockwellcollins.com wrote:
> >  > > Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on
> >  > > 09/08/2013 08:15:28 AM:
> >  > >
> >  > >  >
> >  > >  > Signed-off-by: Thomas Petazzoni 
<thomas.petazzoni@free-electrons.com>
> >  > >  > ---
> >  > >  >  Config.in   |  6 ++++--
> >  > >  >  Makefile    | 29 +++++++++++++++++++++++++----
> >  > >  >  docs/manual/manual.mk |  2 +-
> >  > >
> >  > > One idea that I have, is for the make file that is genreated for 
the an
> >  > > external output directory, if we could preserve the the value of
> >  > > BR2_EXTERNAL
> >  > > as follows.
> >  >
> >  >   Thomas's patch already does that, because BR2_EXTERNAL is stored 
in the
> >  > config file.
> >  >
> >
> > I don't believe it gets stored in the config since Config.in pulls
> > the variable in from the shell's or make's environment.
> >
> > +config BR2_EXTERNAL
> > +       string
> > +       option env="BR2_EXTERNAL"
> >
> > In order to utilize the BR2_EXTERNAL feature I must have BR2_EXTERNAL
> > either exported into my environment or be called as
> 
>   The Makefile exports it in the environment of the Kconfig targets. So 
> once it is in the .config, it will stay there. Try it and you'll see :-)

I have looked as I'm experiment with this and it definitely isn't there :) 

Also according to the documentation you need to define it every time :)

> >
> > 'make BR2_EXTERNAL=/some/path <target>'
> >
> >
> > When utilizing the BR2_EXTERNAL I usual want to use some defconfig 
from
> > my BR2_EXTERNAL so it adding it the makefile that is generated in your
> > build directory when specifying O= would be extermely useful as that
> > way I can switch shells and my definition of BR2_EXTERNAL would still
> > be present.
> >
> > 'make O=/build/path BR2_EXTERNAL=/path/to/external defconfig'
> 
>   In this particular case, you're not using the Makefile in the output 
> dir but only in the buildroot dir so it's not relevant.
> 
>   The only case where it would be relevant is when you have an existing 
> output directory, you remove the .config first, and then you do a
> make foo_defconfig. Seems a bit an unlikely scenario.
> 
>   Regards,
>   Arnout

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/1906f3f7/attachment.html>

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Thomas, All,

On 2013-09-12 20:21 +0200, Thomas Petazzoni spake thusly:
> On Thu, 12 Sep 2013 09:54:56 +0200, Thomas De Schampheleire wrote:
> 
> > > Yes, as I said above, I would have expected companies (well, people at
> > > companies) be stressed by projects deadlines, and that they would work
> > > in BR2_EXTERNAL, and only try to upstream later, when it would be too
> > > late (for them!) to have proper reviews, since they would be internally
> > > committed to their changes in BR2_EXTERNAL, and we would not be able to
> > > take their changes.
> > >
> > > But if people use BR2_EXTERNAL as a fallback for anything that did not
> > > make it upstream (by lack of time, or of interest), or for purely
> > > internal stuff, then I don't see a reason for concern. All the better! :-)
> > >
> > 
> > Not to ruin the atmosphere, but even though Ryan and his colleagues
> > may be using BR2_EXTERNAL 'the good way', ultimately helping the
> > buildroot project with their contributions, the danger of others not
> > doing that is still real. Of course, _not_ contributing changes made
> > in a company project was also possible previously, without
> > BR2_EXTERNAL, so this is no reason to block the patchset.
> 
> Yes, that's what I believe: companies and people are forking Buildroot
> nowadays. As I highlighted in my reply to Yann's e-mail, BR2_EXTERNAL
> may in fact encourage companies in separating their specific changes
> from the changes that could potentially be upstreamed, and therefore
> encourage upstream.
> 
> I really don't think BR2_EXTERNAL is going to make things worse in
> terms of upstreaming.

Agreed.

It makes it easier fro them not to upstream, but since it was not
complex to do so before, that does virtually change nothing.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Fwd: [PATCH 2/3] Add support for BR2_EXTERNAL

[Ryan Barnett]

While utilizing this patch, I found the following that should be fixed

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote on 09/12/2013 
04:39 PM:

> diff --git a/Makefile b/Makefile
> index fc55b87..1014399 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -47,7 +47,6 @@ export HOSTARCH := $(shell uname -m | \
> 
>  # absolute path
>  TOPDIR:=$(shell pwd)
> -CONFIG_CONFIG_IN=Config.in
>  CONFIG=support/kconfig
>  DATE:=$(shell date +%Y%m%d)
> 
> @@ -339,6 +338,10 @@ include boot/common.mk
>  include linux/linux.mk
>  include system/system.mk
> 
> +ifneq ($(BR2_EXTERNAL),)
> +include $(BR2_EXTERNAL)/external.mk
> +endif
> +
>  TARGETS+=target-finalize
> 
>  ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)
> @@ -622,6 +625,18 @@ $(BUILD_DIR)/buildroot-config/%onf:
>         mkdir -p $(@D)/lxdialog
>         $(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)"
> obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
> 
> +CONFIG_CONFIG_IN=$(BUILD_DIR)/buildroot-config/toplevel.in
> +
> +# This is intentionally a virtual target so that the file gets
> +# regenerated everytime this target is invoked.
> +toplevelin-generate:
> +       mkdir -p $(dir $(CONFIG_CONFIG_IN))
> +       echo "mainmenu \"Buildroot $$BR2_VERSION Configuration\"" >
> $(CONFIG_CONFIG_IN)
> +       echo "source \"Config.in\"" >> $(CONFIG_CONFIG_IN)
> +ifneq ($(BR2_EXTERNAL),)
> +       echo "source \"$$BR2_EXTERNAL/Config.in\"" >> 
$(CONFIG_CONFIG_IN)
> +endif
> +
>  DEFCONFIG = $(call qstrip,$(BR2_DEFCONFIG))

Each command above should have a @ symbol since the commands print out 
every
time. As follows:

+toplevelin-generate:
+       @mkdir -p $(dir $(CONFIG_CONFIG_IN))
+       @echo "mainmenu \"Buildroot $$BR2_VERSION Configuration\"" >
$(CONFIG_CONFIG_IN)
+       @echo "source \"Config.in\"" >> $(CONFIG_CONFIG_IN)
+ifneq ($(BR2_EXTERNAL),)
+       @echo "source \"$$BR2_EXTERNAL/Config.in\"" >> $(CONFIG_CONFIG_IN)
+endif
+

>  # We don't want to fully expand BR2_DEFCONFIG here, so Kconfig will
> @@ -631,10 +646,12 @@ COMMON_CONFIG_ENV = \
>         KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \
>         KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \
>         KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \
> -       BUILDROOT_CONFIG=$(BUILDROOT_CONFIG)
> +       BUILDROOT_CONFIG=$(BUILDROOT_CONFIG) \
> +       BR2_EXTERNAL=$(BR2_EXTERNAL)
> 
>  COMMON_CONFIG_DEPS = \
> -       outputmakefile
> +       outputmakefile \
> +       toplevelin-generate
> 
>  xconfig: $(BUILD_DIR)/buildroot-config/qconf $(COMMON_CONFIG_DEPS)
>         @mkdir -p $(BUILD_DIR)/buildroot-config
> @@ -718,6 +735,10 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf
> $(COMMON_CONFIG_DEPS)
>         @mkdir -p $(BUILD_DIR)/buildroot-config
>         @$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@
> $(CONFIG_CONFIG_IN)
> 
> +%_defconfig: $(BUILD_DIR)/buildroot-config/conf
> $(BR2_EXTERNAL)/configs/%_defconfig $(COMMON_CONFIG_DEPS)
> +       @mkdir -p $(BUILD_DIR)/buildroot-config
> +       @$(COMMON_CONFIG_ENV) $<
> --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
> +
>  savedefconfig: $(BUILD_DIR)/buildroot-config/conf $(COMMON_CONFIG_DEPS)
>         @mkdir -p $(BUILD_DIR)/buildroot-config
>         @$(COMMON_CONFIG_ENV) $< \
> @@ -829,7 +850,7 @@ endif
>         @echo '  make V=0|1             - 0 => quiet build (default),
> 1 => verbose build'
>         @echo '  make O=dir             - Locate all output files in
> "dir", including .config'
>         @echo
> -       @$(foreach b, $(sort $(notdir $(wildcard
> $(TOPDIR)/configs/*_defconfig))), \
> +       @$(foreach b, $(sort $(notdir $(wildcard
> $(TOPDIR)/configs/*_defconfig $(BR2_EXTERNAL)/configs/*_defconfig))),
> \
>           printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
>         @echo
>         @echo 'See docs/README, or generate the Buildroot manual for
> further details'
> diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk
> index 4906bc8..8e0ab30 100644
> --- a/docs/manual/manual.mk
> +++ b/docs/manual/manual.mk
> @@ -1,6 +1,6 @@
>  manual-update-lists:
>         $(Q)$(call MESSAGE,"Updating the manual lists...")
> -       $(Q)BR2_DEFCONFIG="" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build 
\
> +       $(Q)BR2_DEFCONFIG="" BR2_EXTERNAL="$(BR2_EXTERNAL)"
> TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \
>                 $(TOPDIR)/support/scripts/gen-manual-lists.py
> 
> 
> 
################################################################################
> --
> 1.8.1.2
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/3f5ae746/attachment.html>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Arnout Vandecappelle]

On 12/09/13 23:51, Ryan Barnett wrote:
>  > > In order to utilize the BR2_EXTERNAL feature I must have BR2_EXTERNAL
>  > > either exported into my environment or be called as
>  >
>  >   The Makefile exports it in the environment of the Kconfig targets. So
>  > once it is in the .config, it will stay there. Try it and you'll see :-)
>
> I have looked as I'm experiment with this and it definitely isn't there :)
> Also according to the documentation you need to define it every time :)
>

  Ah, yes indeed, sorry about that - I was trying to be too smart.

  To save it in the .config, it needs a prompt, and to be able to take it 
from the environment it needs a helper variable. Like BR2_DEFCONFIG.


  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Arnout Vandecappelle]

On 12/09/13 20:33, Thomas Petazzoni wrote:
> Dear ANDY KENNEDY,
>
> On Thu, 12 Sep 2013 18:25:40 +0000, ANDY KENNEDY wrote:
>
>> This sounds much like the way Linux does things for the IP related
>> drivers.  Is that the intent we are going for (personally, I think this
>> is a GREAT idea, as it allows companies to have IP related widgets in
>> BuildRoot without the fear of being REQUIRED to push back their secret
>> sauce)?
>>
>> If that is what you intend, you have my vote!
>
> I am not sure what you mean by "IP related drivers". Do you mean
> proprietary drivers?
>
> It is true that the BR2_EXTERNAL thing raises a licensing question:
> should the BR2_EXTERNAL contents also be released under GPLv2, like the
> rest of Buildroot? Do we really want the root filesystem overlays and
> other highly project-specific contents be released under GPLv2 ?

  You can never be sure unless a judge makes a ruling, but the GPL is 
pretty clear on this:

"If identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works."

  i.e. you can distribute the source of buildroot itself separately from 
your external directory.

  GPL continues:

"But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License."

  As long as your external stuff merely aggregates with the GPL'd stuff 
under buildroot, there shouldn't be an issue.


  Regards,
  Arnout



-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Thomas, All,

On 2013-09-12 20:33 +0200, Thomas Petazzoni spake thusly:
> On Thu, 12 Sep 2013 18:25:40 +0000, ANDY KENNEDY wrote:
> 
> > This sounds much like the way Linux does things for the IP related
> > drivers.  Is that the intent we are going for (personally, I think this
> > is a GREAT idea, as it allows companies to have IP related widgets in
> > BuildRoot without the fear of being REQUIRED to push back their secret
> > sauce)?
> > 
> > If that is what you intend, you have my vote!
> 
> I am not sure what you mean by "IP related drivers". Do you mean
> proprietary drivers?
> 
> It is true that the BR2_EXTERNAL thing raises a licensing question:
> should the BR2_EXTERNAL contents also be released under GPLv2, like the
> rest of Buildroot? Do we really want the root filesystem overlays and
> other highly project-specific contents be released under GPLv2 ?

IANAL, this is not legal advice, talk to your lawyer.

Here are however my toughts on this:

 1- all the Config.in and package.mk files, plus the external.mk file,
    and all other files sourced or otherwise included by a file from
    Buildroot (and thus use Buildroot's infrastructure) could be
    considered a derived work of Buildroot, and thus must be licensed
    with a compatible license,
 2- the defconfigs could be considered as what the GPL identifies as the
    "scripts used to control compilation and installation of the
    executable", since without the .config file, an end-user can not
    reproduce the executable(s),
 3- everything else might be considered not a derived work of Buildroot,
    and might not be required to be licensed with a license compatible
    with Buildroot's license.

Explanations:

1- It's not because it is to be licensed under a license compatible with
   the Buildroot  license that it *has* to be distributed. Just that if
   it *is* distributed, the licensing terms on that should be compatible
   with Buildroot's licensing terms.

2- That's because Buildroot will most probably build a GPL/LGPL program
   that will end up in the ytarget that this applies, because of the
   license of those programs, not because of Buildroot's license. Even
   if Buildroot was under a different license, this would still apply as
   long as a GPL/LGPL program ends up in the target.

3- By "everything else", I mean: a package (in Buildroot's terminology:
   the Config.in and associated .mk file) that directly embbeds the source
   files (.c and .h and ...) for this package (like the host-mkpasswd).
   These source files can not be considered a derived work of Buildroot,
   so Buildroot's license does not apply to those.

So, I think the make legal-stuff should descend into BR2_EXTERNAL, as if
it was an integral part of Buildroot.

BR2_EXTERNAL is here *just* to separate parts of the recipes from the
upstream ones, as a mean to clearly separate local changes from the
upstream reference.

But again: IANAL, this is not legal advice, talk to your legal counsel.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Ryan Barnett]

Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:57:04 PM:

> On 12/09/13 23:51, Ryan Barnett wrote:
> >  > > In order to utilize the BR2_EXTERNAL feature I must have 
BR2_EXTERNAL
> >  > > either exported into my environment or be called as
> >  >
> >  >   The Makefile exports it in the environment of the Kconfig 
targets. So
> >  > once it is in the .config, it will stay there. Try it and you'll 
see :-)
> >
> > I have looked as I'm experiment with this and it definitely isn't 
there :)
> > Also according to the documentation you need to define it every time 
:)
> >
> 
>   Ah, yes indeed, sorry about that - I was trying to be too smart.

Happens to the best of us.

>   To save it in the .config, it needs a prompt, and to be able to take 
it 
> from the environment it needs a helper variable. Like BR2_DEFCONFIG.

I like to view BR2_EXTERNAL as something that is similar to "O=" which 
means
that I have the ability to change it at a moments notice. However, like it 

is done in mkmakefile when doing the following from within buildroot 
toplevel
source directory:

make O=/path/to/build defconfig

O= is stored in the generated makefile (as you know). However, it gets 
very
annoying to type BR2_EXTERNAL=/path/to/external and I don't always want to
export it as shell environment variable. Thus, adding that line to 
mkmakefile is handy since usually when you specify O= you are doing a 
specific build which BR2_EXTERNAL would definitely be a part of.

Thanks,
-Ryan


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130912/a4e733ea/attachment.html>

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

On 2013-09-13 00:04 +0200, Arnout Vandecappelle spake thusly:
> On 12/09/13 20:33, Thomas Petazzoni wrote:
> >Dear ANDY KENNEDY,
> >
> >On Thu, 12 Sep 2013 18:25:40 +0000, ANDY KENNEDY wrote:
> >
> >>This sounds much like the way Linux does things for the IP related
> >>drivers.  Is that the intent we are going for (personally, I think this
> >>is a GREAT idea, as it allows companies to have IP related widgets in
> >>BuildRoot without the fear of being REQUIRED to push back their secret
> >>sauce)?
> >>
> >>If that is what you intend, you have my vote!
> >
> >I am not sure what you mean by "IP related drivers". Do you mean
> >proprietary drivers?
> >
> >It is true that the BR2_EXTERNAL thing raises a licensing question:
> >should the BR2_EXTERNAL contents also be released under GPLv2, like the
> >rest of Buildroot? Do we really want the root filesystem overlays and
> >other highly project-specific contents be released under GPLv2 ?
> 
>  You can never be sure unless a judge makes a ruling, 

Agreed: only a judge can decide.

So: IANAL, and so on...

> but the GPL is pretty
> clear on this:
> 
> "If identifiable sections of that work are not derived from the Program,
> and can be reasonably considered independent and separate works in
> themselves, then this License, and its terms, do not apply to those
> sections when you distribute them as separate works."
> 
>  i.e. you can distribute the source of buildroot itself separately from your
> external directory.

Yes, but I believe BR2_EXTERNAL *is* a derived work of Buildroot, so as
such should be licensed under a license compatible with Buildroot's.

>  As long as your external stuff merely aggregates with the GPL'd stuff under
> buildroot, there shouldn't be an issue.

But that's not aggregation in my opinion, since it does leverage all or
parts of the Buildroot infrastrucutre, and was clearly written with this
intent, and as such can not be considered mere aggregation.

Of course, all this only applies _if and when_ BR2_EXTERNAL gets
distributed to a third party.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

On 2013-09-12 20:18 +0200, Thomas Petazzoni spake thusly:
> Dear Yann E. MORIN,
> 
> On Wed, 11 Sep 2013 19:27:09 +0200, Yann E. MORIN wrote:
> 
> > > That just becomes nasty and doesn't really easily allow for different 
> > > configurations where you don't always want all your propriety packages
> > > to build. Also it doesn't make your propriety packages visible within
> > > the buildroot menuconfig. The other nice feature is the is that company's
> > > configs are automatically pulled in a separate area.
> > 
> > The way *I* see this is that Buildroot is not the main /frontend/ of the
> > build system, but rather a backend. That's the way I use it: I have a
> > kind of /upper-layer/ build system that uses Buildroot as a kind of
> > /backend/.
> 
> I think that's one way of using it, but that's not necessarily the only
> way we should support.

Sorry, I never said so. I just said how *I* use it in my own use-case,
to illustrate yet another way to use Buildroot.

> In all the Buildroot based projects I've done,
> I've also tried to make Buildroot directly be the front-end, and
> include whatever build logic is needed within Buildroot.
> 
> I can only remember of one project in which I did not do that, and it
> was because the customer had already written his build logic using
> Buildroot as a back-end.

Yes, we should keep Buildroot a self-contained project. That it gets
also used as a kind of backend does not mean Buildrot *is* a backend.

> > Yes, as I said above, I would have expected companies (well, people at
> > companies) be stressed by projects deadlines, and that they would work
> > in BR2_EXTERNAL, and only try to upstream later, when it would be too
> > late (for them!) to have proper reviews, since they would be internally
> > committed to their changes in BR2_EXTERNAL, and we would not be able to
> > take their changes.
> > 
> > But if people use BR2_EXTERNAL as a fallback for anything that did not
> > make it upstream (by lack of time, or of interest), or for purely
> > internal stuff, then I don't see a reason for concern. All the better! :-)
> 
> If companies are in a hurry, they are not going to upstream anything
> anyway, so we're not loosing something. Today, without BR2_EXTERNAL,
> companies can already fork Buildroot, do a massive amount of changes
> directly within Buildroot, and not upstream anything at all. Therefore,
> adding BR2_EXTERNAL is not going to make things worse.
> 
> I believe it could potentially make things better. It might allow
> companies to separate what they consider "company-specific" in the
> BR2_EXTERNAL directory, while still making the Buildroot core changes,
> or open-source package additions in the main Buildroot tree, and submit
> those changes upstream. It allows them to keep cleanly separated the
> changes that will never be upstreamed because it doesn't make sense
> (specific board support, highly specific and proprietary packages) from
> the things that might be upstreamed if the submission effort is made.

I completely agree with you.

You showed me the Light, and I've seen the Light, now! :-)

Anyway, maybe we should make this clear in the manual what BR2_EXTERNAL
is really for non-FLOSS, proprietary packages, and that FLOSS packages
should be done in Buildroot, nit in BR2_EXTERNAL.

Alternatively, I can see BR2_EXTERNAL as a staging location where a
company may introduce new (FLOSS) packages, and move them to Buildroot
just prior to upstreaming. Enforcing Buildroot's layout in BR2_EXTERNAL
would surely help in this regard, I believe.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[ANDY KENNEDY]

> Yann E. MORIN wrote:
> IANAL, this is not legal advice, talk to your lawyer.
> 
> Here are however my toughts on this:
> 
>  1- all the Config.in and package.mk files, plus the external.mk file,
>     and all other files sourced or otherwise included by a file from
>     Buildroot (and thus use Buildroot's infrastructure) could be
>     considered a derived work of Buildroot, and thus must be licensed
>     with a compatible license,
>  2- the defconfigs could be considered as what the GPL identifies as the
>     "scripts used to control compilation and installation of the
>     executable", since without the .config file, an end-user can not
>     reproduce the executable(s),
>  3- everything else might be considered not a derived work of Buildroot,
>     and might not be required to be licensed with a license compatible
>     with Buildroot's license.
> 
> Explanations:
> 
> 1- It's not because it is to be licensed under a license compatible with
>    the Buildroot  license that it *has* to be distributed. Just that if
>    it *is* distributed, the licensing terms on that should be compatible
>    with Buildroot's licensing terms.
> 
> 2- That's because Buildroot will most probably build a GPL/LGPL program
>    that will end up in the ytarget that this applies, because of the
>    license of those programs, not because of Buildroot's license. Even
>    if Buildroot was under a different license, this would still apply as
>    long as a GPL/LGPL program ends up in the target.
> 
> 3- By "everything else", I mean: a package (in Buildroot's terminology:
>    the Config.in and associated .mk file) that directly embbeds the source
>    files (.c and .h and ...) for this package (like the host-mkpasswd).
>    These source files can not be considered a derived work of Buildroot,
>    so Buildroot's license does not apply to those.
> 
> So, I think the make legal-stuff should descend into BR2_EXTERNAL, as if
> it was an integral part of Buildroot.
> 
> BR2_EXTERNAL is here *just* to separate parts of the recipes from the
> upstream ones, as a mean to clearly separate local changes from the
> upstream reference.
> 
> But again: IANAL, this is not legal advice, talk to your legal counsel.


Since tone is difficult to understand through e-mail, please understand
that I'm not attempting to start a flame war, nor am I throwing stones. . .
just brainstorming (or in my case, perhaps a light shower).

You speak of BuildRoot as a third party.  BuildRoot is not really a third
party but is first person from my point of view.  Although I have not added
NEAR as much as other people, I still contend that I am partial owner in
BuildRoot.  To that end, I have a voice in the decision making over any
external (or internal for that matter) infrastructure and/or tie-in.

If we choose to allow outside companies to use BuildRoot to develop systems
with Intellectual Property (IP) -- which is why we all contribute to
BuildRoot in the first place right?  I mean, surely I'm not the only one
here who has paid for my house, cars, kids, etc by using Linux (and by
extension BuildRoot) to develop embedded systems. -- then we are here-by
GRANTING permissions, as the current developers of BuildRoot, that Sure
"Super Company XYZ" you can use our previous work to go forth and make
Linux products.  IMO, the more users we have of Open Sourced software, the
better the software becomes.  Just because I push something back to the
community doesn't mean that I have given away some sort of trade secrete.
How I use the widget may be the trade secret, but that is none of anyone
else's business (and EXCEEDS the scope of the GPL).

So, can we not attempt to come to a consensus within our own little
BuildRoot family of how we will allow others to use such a tie-in?  If the
general populous of this family decides that we want to protect what we
have done, so be it.  But, if most of us feels that we should allow
companies/individuals to expand what is built with BuildRoot without the
fear of the FSF coming to hunt down all their children, then we should, as
the property holders, grant that permission EXPLICITLY.

Again, If I'm the only one that feels this way, tell me to shut up and I'll
say nothing else about it.  I just don't think that using Linus's kernel
under the hood, and the GNU project's additions to it requires the developers
of a product to "open source" the product just because they are, within the
confines of the granted licenses, and within GOOD FAITH of the company and
free software community, using said base.

Andy

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Andy, All,

On 2013-09-12 22:28 +0000, ANDY KENNEDY spake thusly:
> > Yann E. MORIN wrote:
> > IANAL, this is not legal advice, talk to your lawyer.
> > 
> > Here are however my toughts on this:
> > 
> >  1- all the Config.in and package.mk files, plus the external.mk file,
> >     and all other files sourced or otherwise included by a file from
> >     Buildroot (and thus use Buildroot's infrastructure) could be
> >     considered a derived work of Buildroot, and thus must be licensed
> >     with a compatible license,
> >  2- the defconfigs could be considered as what the GPL identifies as the
> >     "scripts used to control compilation and installation of the
> >     executable", since without the .config file, an end-user can not
> >     reproduce the executable(s),
> >  3- everything else might be considered not a derived work of Buildroot,
> >     and might not be required to be licensed with a license compatible
> >     with Buildroot's license.
> > 
> > Explanations:
> > 
> > 1- It's not because it is to be licensed under a license compatible with
> >    the Buildroot  license that it *has* to be distributed. Just that if
> >    it *is* distributed, the licensing terms on that should be compatible
> >    with Buildroot's licensing terms.
> > 
> > 2- That's because Buildroot will most probably build a GPL/LGPL program
> >    that will end up in the ytarget that this applies, because of the
> >    license of those programs, not because of Buildroot's license. Even
> >    if Buildroot was under a different license, this would still apply as
> >    long as a GPL/LGPL program ends up in the target.
> > 
> > 3- By "everything else", I mean: a package (in Buildroot's terminology:
> >    the Config.in and associated .mk file) that directly embbeds the source
> >    files (.c and .h and ...) for this package (like the host-mkpasswd).
> >    These source files can not be considered a derived work of Buildroot,
> >    so Buildroot's license does not apply to those.
> > 
> > So, I think the make legal-stuff should descend into BR2_EXTERNAL, as if
> > it was an integral part of Buildroot.
> > 
> > BR2_EXTERNAL is here *just* to separate parts of the recipes from the
> > upstream ones, as a mean to clearly separate local changes from the
> > upstream reference.
> > 
> > But again: IANAL, this is not legal advice, talk to your legal counsel.
> 
> 
> Since tone is difficult to understand through e-mail, please understand
> that I'm not attempting to start a flame war, nor am I throwing stones. . .
> just brainstorming (or in my case, perhaps a light shower).

No problem, there are no feelings in my words (OK, just a very little bit!).

> You speak of BuildRoot as a third party.  BuildRoot is not really a third
> party but is first person from my point of view.

No, Buildroot is *not* a third party.

Consider this situation:
  Buildroot community
    -> makes Buildroot available under GPLv2

  Company FooCorp
    -> uses Buildroot
    -> creates BR2_EXTERNAL
    -> distributes BR2_EXTERNAL

  You or Me
    -> gets BR2_EXTERNAL from FooCorp

The third party I referred to (in my other mail) would be "You or Me"
here.

>  Although I have not added
> NEAR as much as other people, I still contend that I am partial owner in
> BuildRoot.  To that end, I have a voice in the decision making over any
> external (or internal for that matter) infrastructure and/or tie-in.

Yes, but since Buildroot pre-existed under the GPLv2 prior to your
changes, and your changes were made explicitly to work with Buildroot
(by being modifications to Buildroot), then those changes are GPLv2.

[--SNIP--]
What I'm arguing is that the content of BR2_EXTERNAL that _interacts
with Buildroot's internal infrastructure_ *are* a derived work of
Buildroot.

Let's take a (very simplistic) example:

    BR2_EXTERNAL/
        package/
            pkg-1/
                Config.in           *
                pkg-1.mk            *
                pkg-1-main.c        -
                pkg-1.h             -

* : a derived work of Buildroot
- : not a derived work of Buildroot

Config.in and pkg-1.mk are clearly (in my opinion) a derived work of
Buildroot, since they are written with the very intent to be interacting
with Buildroot's internal infrastructure.

But pkg-1-main.c and pkg-1.h (which are the C sources of pkg-1) are *not*
a derived work of Buildroot, and thus can well be under whatever license.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Thu, 12 Sep 2013 23:04:13 +0200, Arnout Vandecappelle wrote:

>   It's funny how people can change their mind...

Isn't it? In french, we have an expression that more or less says:
"only idiots don't change their mind" :-)

> On 08/09/13 15:15, Thomas Petazzoni wrote:
> > This commit adds support for an environment variable named
> > BR2_EXTERNAL, which the user can point to a directory outside of
> > Buildroot that contains root filesystem overlays, kernel configuration
> > files, package recipes, defconfigs, etc. It allows users to keep their
> > specific Buildroot customizations outside of the Buildroot tree.
> >
> > BR2_EXTERNAL allows:
> 
>   I would split this into three separate patches for the three separate 
> features. I think (1) and (3) are a lot less controversial than (2).

Hum, right, I might be splitting things, but I believe than (1) and (3)
without (2) doesn't make much sense.

> >   (1) To use $(BR2_EXTERNAL) in all Buildroot configuration options
> >       that take a path as argument. This allows to reference a root
> >       filesystem overlay, a kernel configuration file, a Barebox
> >       configuration file located in the BR2_EXTERNAL directory.
> 
>   Note to list: this was already possible before, of course, the 
> difference is that now this path will be stored in the .config.

I'm not sure why you think it's going to be stored in the .config. The
'config BR2_EXTERNAL' option is here only to "forward" the environment
variable as a kconfig value, and it doesn't seem to be stored in
the .config file, as far as I can see:

$ make BR2_EXTERNAL=../company/ menuconfig
$ grep BR2_EXTERNAL .config
$

>   There is one small issue with this: I don't think that all the use 
> cases support relative paths. Ideally, the Makefile should be smart 
> enough to prepend $(TOPDIR) if it doesn't start with a /.

So far the cases I've tested did support relative paths, but I agree
that turning it into an absolute path within the Makefile is probably
safer.

> >   (2) To store external package or makefile logic, since the
> >       BR2_EXTERNAL/external.mk file is automatically included by the
> >       Makefile logic, and the BR2_EXTERNAL/Config.in file is
> >       automatically included, and appears in the top-level menu. The
> >       typical usage would be to create a BR2_EXTERNAL/package/
> >       directory to contain package definitions.
> 
>   I'm not really convinced by the principle. The external.mk is exactly 
> the same as the local override .mk; the Config.in appears at the very end 
> of the top-level menu. Instead, I think we should enforce the buildroot 
> hierarchy in the external dir, i.e.
> 
> source <path-to-external-dir>/package/Config.in
> 
> at the top of package/Config.in, and
> 
> -include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
> 
> in the top-level Makefile.

This is what I had originally done, but it seemed to me that the
external stuff may wish to add other Kconfig options than just
packages. That said, I've done this BR2_EXTERNAL stuff mainly for
packages, so if people think that we should restrict this to packages,
I'm fine with that.

>   Of course, that Config.in thing is a lot trickier because it's almost 
> impossible to refer to the output directory from the Config.in.

I'm not sure to follow you on this one.

> >   * The top-level Config.in file given to kconfig is no longer the main
> >     Config.in in the Buildroot sources, but instead a toplevel.in file
> >     generated in the output directory, which includes the top-level
> >     Buildroot Config.in file and the BR2_EXTERNAL/Config.in file if
> >     provided. Since is needed because the kconfig 'source' statement
> >     errors out if the included file doesn't exist. I have written
> >     patches that add support for an 'isource' statement in kconfig
> >     (that silently ignores the inclusion if the pointed file doesn't
> >     exist), but keeping feature patches against kconfig doesn't seem
> >     like a good idea.
> 
>   It took me a while to realize that that was _not_ the approach you had 
> taken...

Yeah, sorry if the wording was confusing. I initially experimented by
doing a kconfig modification, and then realized it could be done
without changing kconfig at all, which is obviously nicer.

> >   * The BR2_EXTERNAL environment variable gets passed in the
> >     environment of kconfig and when executing the kconfiglib-based
> >     Python script that updates the manual, so that the references to
> >     the BR2_EXTERNAL variable within Config.in files can be resolved.
> 
>   So this means that the external packages will appear in the package 
> list in the manual? I'm not sure if that is what you typically want.

I remember having an issue with the manual generation script, but it
might have been caused by the earlier kconfig-based implementation. I
agree with quite certainly don't want to have the BR2_EXTERNAL packages
listed in the manual.


>   I guess you mean phony target. You should also add
> .PHONY: toplevelin-generate
> for the (unlikely) case that someone creates a file with this name.

Ok.

>   BTW I don't like the target's name. generate-config-dot-in maybe?

Ok.

> > +	mkdir -p $(dir $(CONFIG_CONFIG_IN))
> > +	echo "mainmenu \"Buildroot $$BR2_VERSION Configuration\"" > $(CONFIG_CONFIG_IN)
> > +	echo "source \"Config.in\"" >> $(CONFIG_CONFIG_IN)
> > +ifneq ($(BR2_EXTERNAL),)
> > +	echo "source \"$$BR2_EXTERNAL/Config.in\"" >> $(CONFIG_CONFIG_IN)
> 
>   $(BR2_EXTERNAL) would be a lot more readable than $$BR2_EXTERNAL IMHO. 
> Then you can also put it in single quotes and remove the \.

Right.


>   There is one tricky issue:
> 
> cd ~/src/myproject/output
> make -C ~/src/buildroot O=$PWD menuconfig
> Hack hack hack
> Hm, I need an external dir...
> BR2_EXTERNAL=.. make menuconfig
> => BR2_EXTERNAL will point to ~/src instead of ~/src/myproject
> 
>   I.e., it doesn't work well for relative paths.
> 
>   I don't know if there is an elegant way to solve that.

I'll have a look into this.


>   This should be in a
> ifneq ($(BR2_EXTERNAL),)

Right.

Thanks for the review!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Tzu-Jung Lee]

Hi Thomas, Arnout, and all,

It's been a while... Arnout has a good memory :-)
And yes we have been using similar approach since then.

Thomas's patch is comprehensive, and we'd like to switch to the official
one whenever possible.

On Fri, Sep 13, 2013 at 11:48 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Arnout Vandecappelle,
>
> On Thu, 12 Sep 2013 23:04:13 +0200, Arnout Vandecappelle wrote:
>
>>   There is one small issue with this: I don't think that all the use
>> cases support relative paths. Ideally, the Makefile should be smart
>> enough to prepend $(TOPDIR) if it doesn't start with a /.
>
> So far the cases I've tested did support relative paths, but I agree
> that turning it into an absolute path within the Makefile is probably
> safer.

>> >   (2) To store external package or makefile logic, since the
>> >       BR2_EXTERNAL/external.mk file is automatically included by the
>> >       Makefile logic, and the BR2_EXTERNAL/Config.in file is
>> >       automatically included, and appears in the top-level menu. The
>> >       typical usage would be to create a BR2_EXTERNAL/package/
>> >       directory to contain package definitions.
>>
>>   I'm not really convinced by the principle. The external.mk is exactly
>> the same as the local override .mk; the Config.in appears at the very end
>> of the top-level menu. Instead, I think we should enforce the buildroot
>> hierarchy in the external dir, i.e.
>>
>> source <path-to-external-dir>/package/Config.in
>>
>> at the top of package/Config.in, and
>>
>> -include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
>>
>> in the top-level Makefile.
>
> This is what I had originally done, but it seemed to me that the
> external stuff may wish to add other Kconfig options than just
> packages. That said, I've done this BR2_EXTERNAL stuff mainly for
> packages, so if people think that we should restrict this to packages,
> I'm fine with that.
>

We do want to support the absolute paths for both Config.in and *.mk.

As a "buildroot overlay", we have similar structures in overlay/ like
the buildroot/,
such as configs, fs, packages, etc.

This requires the buildroot to support only only the *.mk and
Config.in in the top
level over overlay/, but also those in their sub directories.

The *.mk case may be get rid of with Arnount's suggestion.
But I'm wondering could we solve the Config.in case in a less trickier way?

The following is the approach I've been used locally.

The overlay/Makefile plays a trick, which allows us to work under overlay/ as we
work under buildroot/

Suppose we have the following configs,

  overlay/configs/foo_defconfig

Our build sequence will be:

  $ cd overlay
  $ make O=../output/foo foo_defconfig
  $ make O=../output/foo

With either Thomas's patch or our local patch, other targets supported by
buildroot, such as menuconfig, help, etc also works under overlay/ as they do
under buildroot/.

##### overlay/Makefile #######
EXTERNAL_OVERLAY       ?= $(realpath $(dir $(lastword $(MAKEFILE_LIST))))
export EXTERNAL_OVERLAY

BUILDROOT_TOP           = ../buildroot

all:
        ln -fs ../overlay $(BUILDROOT_TOP)
        make -C $(BUILDROOT_TOP)

%:
        ln -fs ../overlay $(BUILDROOT_TOP)
        make -C $(BUILDROOT_TOP) $(MAKECMDGOALS)

##### overlay/local.mk #######
-include overlay/package/*/*.mk
-include overlay/package/*/*/*.mk

-include overlay/cust/*/*.mk
-include overlay/cust/*/*/*.mk

-include overlay/linux/*.mk

###### overlay/Config.in ######
source "overlay/package/Config.in"
source "overlay/cust/Config.in"

###### overlay/package/Config.in ######
source "overlay/package/xxx/Config.in"


Thanks.

Roy

[Buildroot] [PATCH 3/3] docs/manual: add explanations about BR2_EXTERNAL

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Thu, 12 Sep 2013 23:46:22 +0200, Arnout Vandecappelle wrote:

> > ++BR2_EXTERNAL+ is an environment variable that one can use to point to
> > +a directory that contains Buildroot customizations. This environment
> > +variable must be passed everytime a Buildroot +make+ command is
> > +invoked, be it at configuration time (+make menuconfig+, +make
> > +xconfig+, etc.), at build time (+make+) or any other Buildroot command
> > +(+make source+, +make legal-info+, etc.).
> 
>   Hang on... It's stored in the .config, right? So no need to pass it 
> every time?

No, it is *not* stored in the .config. See other parts of the thread.

>   In fact, if the config option would get a prompt, you could even enter 
> it through the config instead of through the environment. The only reason 
> to pass it through the environment is to be able to list the available 
> defconfigs.

No, that's not the only reason. In Config.in, we are sourcing
$(BR2_EXTERNAL)/Config.in, so we need to know the value of
$(BR2_EXTERNAL) before running menuconfig/xconfig.

If BR2_EXTERNAL is just a Config.in option, you have a chicken-and-egg
problem: you need its value before running menuconfig, but to define
its value, you need to run menuconfig.

The BR2_EXTERNAL Config.in option is *only* here to "transfer" the
environment variable value into a Config.in option so that Config.in
files can use $BR2_EXTERNAL to reference other Config.in files. That's
the only reason why we have this BR2_EXTERNAL Config.in option. And as
discussed previously, the value of this option is not even stored in
the .config file.

> > +To achieve this, it is recommended but not mandatory, to store those
> > +   details in directories called +board/<boardname>/+ under
> 
>   You use someboard instead of <boardname> earlier.

Will fix, thanks.


> > +------
> > +menu "<somecompany> packages"
> > +
> > +source "$BR2_EXTERNAL/package/package1/Config.in"
> > +source "$BR2_EXTERNAL/package/package2/Config.in"
> 
>   Wow, does this work?

Are you sure you read/understand the BR2_EXTERNAL completely? :-)

The example above is an example of a $(BR2_EXTERNAL)/Config.in file.
This file is the only one that gets conditionally included (i.e
included when BR2_EXTERNAL is set, not included when BR2_EXTERNAL is
not set).

The Config.in files included by $(BR2_EXTERNAL)/Config.in are the ones
added by the user in his external directory, so obviously he will only
be listing Config.in files that really exist, just like we do in
Buildroot.

There must be something really not clear in the documentation I've
written, because it seems you did not get how it was all working. Do
not hesitate to point out what was causing confusion, so that I can
improve the wording of the documentation.

Maybe you should look at the directory/file hierarchy example shown
below.

> Then actually, you don't need that generated 
> Config.in at all... Instead, you can add
> 
> source "$BR2_EXTERNAL/package/Config.in"
> 
> to the top of package/Config.in. And set a non-empty default for 
> BR2_EXTERNAL, e.g. $(TOPDIR)/support/dummy_external, which contains an 
> empty package/Config.in.

Humm, this however looks like a very interesting trick. I'll try this
out.

> > +In the end, a typical +BR2_EXTERNAL+ directory organization would
> > +generally be:
> > +
> > +-----
> > +??? Config.in
> > +??? external.mk
> > +??? board/
> > +?   ??? <someboard>/
> > +?       ??? overlay/
> > +?           ??? etc/
> > +?               ??? <some file>
> > +??? configs/
> > +?   ??? <some board>_defconfig
> > +??? package/
> > +    ??? Config.in
> > +    ??? package1/
> > +        ??? Config.in
> > +        ??? package1.mk
> > +    ??? package2/
> > +        ??? Config.in
> > +        ??? package2.mk

Have you had a look at this part of the documentation?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Thomas Petazzoni]

Dear Tzu-Jung Lee,

On Fri, 13 Sep 2013 14:43:11 +0800, Tzu-Jung Lee wrote:

> >>   I'm not really convinced by the principle. The external.mk is exactly
> >> the same as the local override .mk; the Config.in appears at the very end
> >> of the top-level menu. Instead, I think we should enforce the buildroot
> >> hierarchy in the external dir, i.e.
> >>
> >> source <path-to-external-dir>/package/Config.in
> >>
> >> at the top of package/Config.in, and
> >>
> >> -include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
> >>
> >> in the top-level Makefile.
> >
> > This is what I had originally done, but it seemed to me that the
> > external stuff may wish to add other Kconfig options than just
> > packages. That said, I've done this BR2_EXTERNAL stuff mainly for
> > packages, so if people think that we should restrict this to packages,
> > I'm fine with that.
> >
> 
> We do want to support the absolute paths for both Config.in and *.mk.
> 
> As a "buildroot overlay", we have similar structures in overlay/ like
> the buildroot/,
> such as configs, fs, packages, etc.
> 
> This requires the buildroot to support only only the *.mk and
> Config.in in the top
> level over overlay/, but also those in their sub directories.
> 
> The *.mk case may be get rid of with Arnount's suggestion.
> But I'm wondering could we solve the Config.in case in a less trickier way?

I am not sure what you mean here. What do you mean by "support the
absolute paths for both Config.in and *.mk" ?

> The overlay/Makefile plays a trick, which allows us to work under overlay/ as we
> work under buildroot/
> 
> Suppose we have the following configs,
> 
>   overlay/configs/foo_defconfig
> 
> Our build sequence will be:
> 
>   $ cd overlay
>   $ make O=../output/foo foo_defconfig
>   $ make O=../output/foo
> 
> With either Thomas's patch or our local patch, other targets supported by
> buildroot, such as menuconfig, help, etc also works under overlay/ as they do
> under buildroot/.
> 
> ##### overlay/Makefile #######
> EXTERNAL_OVERLAY       ?= $(realpath $(dir $(lastword $(MAKEFILE_LIST))))
> export EXTERNAL_OVERLAY
> 
> BUILDROOT_TOP           = ../buildroot
> 
> all:
>         ln -fs ../overlay $(BUILDROOT_TOP)
>         make -C $(BUILDROOT_TOP)
> 
> %:
>         ln -fs ../overlay $(BUILDROOT_TOP)
>         make -C $(BUILDROOT_TOP) $(MAKECMDGOALS)
> 
> ##### overlay/local.mk #######
> -include overlay/package/*/*.mk
> -include overlay/package/*/*/*.mk
> 
> -include overlay/cust/*/*.mk
> -include overlay/cust/*/*/*.mk
> 
> -include overlay/linux/*.mk
> 
> ###### overlay/Config.in ######
> source "overlay/package/Config.in"
> source "overlay/cust/Config.in"
> 
> ###### overlay/package/Config.in ######
> source "overlay/package/xxx/Config.in"

This is what BR2_EXTERNAL allows. Of course, doing a
BR2_EXTERNAL/Makefile that allows to run Buildroot from the external
(overlay) directory is something left for the person who creates the
overlay, I believe.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas De Schampheleire]

Hi Thomas,

On Thu, Sep 12, 2013 at 8:21 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Thomas De Schampheleire,
>
> On Thu, 12 Sep 2013 09:54:56 +0200, Thomas De Schampheleire wrote:
>

>> The upstreaming work is currently done by one person (me) and this is
>> clearly the weak point. If I stop paying attention to upstreaming, or
>> to keeping the changes we make upstreamable, the delta between our
>> development repo and mainline buildroot becomes so large that it is a
>> nightmare to keep up with newer releases.
>
> Do you think there is something we can do at the Buildroot level to
> ease this process?

I don't think there is much to do about that. The buildroot community
is very open and active, so there is no real reason why others
couldn't do what I do. The 'time' aspect is the only hurdle I have
experienced in the past, in two ways:
1. upstreaming changes requires effort and time from the developer
side, in order to prepare the patch in the first place, send it, and
update after comments. All this in several cycles depending on the
comments. In a typical company mindset, upstreaming is not a first
priority, so these tasks go in-between other stuff.
Except for encouraging companies and other forks to upstream their
changes as much as possible, there is little the buildroot community
can do here.

2. sometimes patches get very little, if any, response. Especially in
the beginning of my contributions, this was very frustrating. I pinged
patches, re-sent them, even multiple times, and still not even a
comment was given. I can imagine that not everyone continues trying,
and several people will just give up, tagging the buildroot community
as 'not accepting patches'.
The situation has certainly improved since then, I think: the
introduction of patchwork helps keeping track of open patches, and
although I have not kept numbers I think the community certainly has
grown, and there are now several more people systematically reviewing
patches so that the burden does not fall entirely on Peter and
yourself.
I think we should still keep an eye on this though. I think it still
happens that patches are sent by 'new' people in the community, and
there is little or no reaction. Sometimes the patch does not really
fit in our general belief of how buildroot should be, but maybe we
don't always dare to say it. Or sometimes the patch is so specific
that it is hard to  give a good review, so one waits until someone
else with more experience in that area gives some comments, which may
never happen.
Ideally, patches should stay no longer than a fixed amount of time in
patchwork. Either the patch is given some comments and we
wait/encourage for the submitter to send an updated version, or we
reject the patch with some explanation, or we ask more details so we
can either accept/reject it with more background. This may not be
feasible for all patches. Sometimes there is simply no-one who really
needs the extra feature/fix except the submitter himself, but that is
not necessarily a reason not to accept the patch.

[..]
>
> Thanks a lot for this write-up. I think we should keep it somewhere in
> a section of the manual.

Thanks. I'm adding Samuel on this one.
If we add this to the manual, then maybe we should add some other ways
in which buildroot can be used: a number of recipes that work and from
which beginning users can choose or get ideas from.

Best regards,
Thomas

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Tzu-Jung Lee]

Hi Thomas,

On Fri, Sep 13, 2013 at 3:10 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Tzu-Jung Lee,
>
> On Fri, 13 Sep 2013 14:43:11 +0800, Tzu-Jung Lee wrote:
>
>> >>   I'm not really convinced by the principle. The external.mk is exactly
>> >> the same as the local override .mk; the Config.in appears at the very end
>> >> of the top-level menu. Instead, I think we should enforce the buildroot
>> >> hierarchy in the external dir, i.e.
>> >>
>> >> source <path-to-external-dir>/package/Config.in
>> >>
>> >> at the top of package/Config.in, and
>> >>
>> >> -include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
>> >>
>> >> in the top-level Makefile.
>> >
>> > This is what I had originally done, but it seemed to me that the
>> > external stuff may wish to add other Kconfig options than just
>> > packages. That said, I've done this BR2_EXTERNAL stuff mainly for
>> > packages, so if people think that we should restrict this to packages,
>> > I'm fine with that.
>> >
>>
>> We do want to support the absolute paths for both Config.in and *.mk.
>>
>> As a "buildroot overlay", we have similar structures in overlay/ like
>> the buildroot/,
>> such as configs, fs, packages, etc.
>>
>> This requires the buildroot to support only only the *.mk and
>> Config.in in the top
>> level over overlay/, but also those in their sub directories.
>>
>> The *.mk case may be get rid of with Arnount's suggestion.
>> But I'm wondering could we solve the Config.in case in a less trickier way?
>
> I am not sure what you mean here. What do you mean by "support the
> absolute paths for both Config.in and *.mk" ?
>
>> The overlay/Makefile plays a trick, which allows us to work under overlay/ as we
>> work under buildroot/
>>
>> Suppose we have the following configs,
>>
>>   overlay/configs/foo_defconfig
>>
>> Our build sequence will be:
>>
>>   $ cd overlay
>>   $ make O=../output/foo foo_defconfig
>>   $ make O=../output/foo
>>
>> With either Thomas's patch or our local patch, other targets supported by
>> buildroot, such as menuconfig, help, etc also works under overlay/ as they do
>> under buildroot/.
>>
>> ##### overlay/Makefile #######
>> EXTERNAL_OVERLAY       ?= $(realpath $(dir $(lastword $(MAKEFILE_LIST))))
>> export EXTERNAL_OVERLAY
>>
>> BUILDROOT_TOP           = ../buildroot
>>
>> all:
>>         ln -fs ../overlay $(BUILDROOT_TOP)
>>         make -C $(BUILDROOT_TOP)
>>
>> %:
>>         ln -fs ../overlay $(BUILDROOT_TOP)
>>         make -C $(BUILDROOT_TOP) $(MAKECMDGOALS)
>>
>> ##### overlay/local.mk #######
>> -include overlay/package/*/*.mk
>> -include overlay/package/*/*/*.mk
>>
>> -include overlay/cust/*/*.mk
>> -include overlay/cust/*/*/*.mk
>>
>> -include overlay/linux/*.mk
>>
>> ###### overlay/Config.in ######
>> source "overlay/package/Config.in"
>> source "overlay/cust/Config.in"
>>
>> ###### overlay/package/Config.in ######
>> source "overlay/package/xxx/Config.in"
>
> This is what BR2_EXTERNAL allows. Of course, doing a
> BR2_EXTERNAL/Makefile that allows to run Buildroot from the external
> (overlay) directory is something left for the person who creates the
> overlay, I believe.


By the absolute path, I mean the paths of sub Config.in included by the
BR2_EXTERNAL/Config.in, not the path of itself.

For example, suppose we have private packages in the overlay

  overlay/Config.in
  overlay/packages/foo/Config.in
  overlay/packages/bar/Config.in

The overlay/Config.in can't include them without support of absolute paths

  source "overlay/packages/foo/Config.in"
  source "overlay/packages/bar/Config.in"

In ths case, Buildroot tries to include them from the

  myproj/buildroot/overlay/packages/foo/Config.in

instead of

  myproj/overlay/packages/foo/Config.in

Thanks.

Roy

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Ryan Barnett]

Thomas De Schampheleire <patrickdepinguin@gmail.com> wrote on 
09/13/2013 02:35:04 AM:

> Hi Thomas,
> 
> On Thu, Sep 12, 2013 at 8:21 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Dear Thomas De Schampheleire,
> >
> > On Thu, 12 Sep 2013 09:54:56 +0200, Thomas De Schampheleire wrote:
> >
> 
> >> The upstreaming work is currently done by one person (me) and this is
> >> clearly the weak point. If I stop paying attention to upstreaming, or
> >> to keeping the changes we make upstreamable, the delta between our
> >> development repo and mainline buildroot becomes so large that it is a
> >> nightmare to keep up with newer releases.
> >
> > Do you think there is something we can do at the Buildroot level to
> > ease this process?
> 
> I don't think there is much to do about that. The buildroot community
> is very open and active, so there is no real reason why others
> couldn't do what I do. The 'time' aspect is the only hurdle I have
> experienced in the past, in two ways:
> 1. upstreaming changes requires effort and time from the developer
> side, in order to prepare the patch in the first place, send it, and
> update after comments. All this in several cycles depending on the
> comments. In a typical company mindset, upstreaming is not a first
> priority, so these tasks go in-between other stuff.
> Except for encouraging companies and other forks to upstream their
> changes as much as possible, there is little the buildroot community
> can do here.
> 
> 2. sometimes patches get very little, if any, response. Especially in
> the beginning of my contributions, this was very frustrating. I pinged
> patches, re-sent them, even multiple times, and still not even a
> comment was given. I can imagine that not everyone continues trying,
> and several people will just give up, tagging the buildroot community
> as 'not accepting patches'.

As being someone who is rather new member of the community, I guess 
I've received almost quite the opposite reaction from the community. 
When posting patches back to mailing list I've received good feedback
that has been very critical and much better than anything our company
has been able to do internally. I've even made my biggest issue known
and in attempt for comment for comments on how I could come up with 
something to fix the problem - Thomas Petazzoni addressed it. So
in a little way I feel great about the discussion I've spawned.

> The situation has certainly improved since then, I think: the
> introduction of patchwork helps keeping track of open patches, and
> although I have not kept numbers I think the community certainly has
> grown, and there are now several more people systematically reviewing
> patches so that the burden does not fall entirely on Peter and
> yourself.

I'm going to start a new discussion thread - [RFC] Effective Patchwork 
Use and Patch Review - on my thoughts about the review process and
I how I'm finding it difficult to use. 

> I think we should still keep an eye on this though. I think it still
> happens that patches are sent by 'new' people in the community, and
> there is little or no reaction. Sometimes the patch does not really
> fit in our general belief of how buildroot should be, but maybe we
> don't always dare to say it. Or sometimes the patch is so specific
> that it is hard to  give a good review, so one waits until someone
> else with more experience in that area gives some comments, which may
> never happen.
> Ideally, patches should stay no longer than a fixed amount of time in
> patchwork. Either the patch is given some comments and we
> wait/encourage for the submitter to send an updated version, or we
> reject the patch with some explanation, or we ask more details so we
> can either accept/reject it with more background. This may not be
> feasible for all patches. Sometimes there is simply no-one who really
> needs the extra feature/fix except the submitter himself, but that is
> not necessarily a reason not to accept the patch.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130913/997f042d/attachment.html>

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Arnout Vandecappelle]

On 13/09/13 00:11, Ryan Barnett wrote:
> Arnout Vandecappelle <arnout@mind.be> wrote on 09/12/2013 04:57:04 PM:
>
>  > On 12/09/13 23:51, Ryan Barnett wrote:
>  > >  > > In order to utilize the BR2_EXTERNAL feature I must have
> BR2_EXTERNAL
>  > >  > > either exported into my environment or be called as
>  > >  >
>  > >  >   The Makefile exports it in the environment of the Kconfig
> targets. So
>  > >  > once it is in the .config, it will stay there. Try it and you'll
> see :-)
>  > >
>  > > I have looked as I'm experiment with this and it definitely isn't
> there :)
>  > > Also according to the documentation you need to define it every time :)
>  > >
>  >
>  >   Ah, yes indeed, sorry about that - I was trying to be too smart.
>
> Happens to the best of us.
>
>  >   To save it in the .config, it needs a prompt, and to be able to take it
>  > from the environment it needs a helper variable. Like BR2_DEFCONFIG.
>
> I like to view BR2_EXTERNAL as something that is similar to "O=" which means
> that I have the ability to change it at a moments notice. However, like it
> is done in mkmakefile when doing the following from within buildroot
> toplevel
> source directory:
>
> make O=/path/to/build defconfig
>
> O= is stored in the generated makefile (as you know). However, it gets very
> annoying to type BR2_EXTERNAL=/path/to/external and I don't always want to
> export it as shell environment variable. Thus, adding that line to
> mkmakefile is handy since usually when you specify O= you are doing a
> specific build which BR2_EXTERNAL would definitely be a part of.

  My point is: it should (and can) be made part of the .config. It really 
has to be part of the .config, because some options in the .config will 
refer to the packages that are defined in the external directory.

  Regards,
  Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Arnout Vandecappelle]

On 13/09/13 00:12, Yann E. MORIN wrote:
> On 2013-09-13 00:04 +0200, Arnout Vandecappelle spake thusly:
[snip]
>> but the GPL is pretty
>> clear on this:
>>
>> "If identifiable sections of that work are not derived from the Program,
>> and can be reasonably considered independent and separate works in
>> themselves, then this License, and its terms, do not apply to those
>> sections when you distribute them as separate works."
>>
>>   i.e. you can distribute the source of buildroot itself separately from your
>> external directory.
>
> Yes, but I believe BR2_EXTERNAL *is* a derived work of Buildroot, so as
> such should be licensed under a license compatible with Buildroot's.
>
>>   As long as your external stuff merely aggregates with the GPL'd stuff under
>> buildroot, there shouldn't be an issue.
>
> But that's not aggregation in my opinion, since it does leverage all or
> parts of the Buildroot infrastrucutre, and was clearly written with this
> intent, and as such can not be considered mere aggregation.
>
> Of course, all this only applies _if and when_ BR2_EXTERNAL gets
> distributed to a third party.

  The more I think about it, the more confused I get :-) Although your 
other reply to Andy sums things up pretty clearly.

  Let me try out a few scenarios (I'm making things up as I go here).

  I have made some changes to buildroot, and I have also made an external 
tree.


1. I want to distribute the modified buildroot tree to someone. Do I also 
have to make the external one available?

  => No, because the buildroot tree in fact doesn't refer to the external 
tree. You run "BR2_EXTERNAL=... make".


2. I distribute a rootfs which contains a GPL program (bash). Do I have 
to make the buildroot tree available?

  => Yes, because I have to make the build scripts for bash available, 
and buildroot is the build script that I've used.


3. I use buildroot to build lua. I copy the lua executable out of the 
target/ tree and distribute it. Do I have to make the buildroot tree 
available?

  => No, because lua is MIT licensed.


4. I distribute a rootfs which contains a GPL program (bash). It also 
contains a proprietary program (foo). I have added a foo.mk file to the 
buildroot tree to build my proprietary program. Do I have to make the 
modified buildroot tree available?

  => Probably not. I have to make the build script for bash available, 
but the original buildroot tree already does the job.

  I believe that this reasoning holds regardless of whether the foo.mk is 
added directly in the buildroot tree or through BR2_EXTERNAL (or through 
override.mk).


5. I add a proprietary binary to the default skeleton and distribute the 
rootfs built by buildroot. Do I have to make the modified buildroot with 
the proprietary binary available?

  => I don't think so, because the proprietary binary is merely 
aggregated with the rest of the buildroot-generated stuff.


6. Can I give the buildroot tree + the external tree to someone else?

  => Yes, but only under the GPL - the GPL applies to the external tree 
as well.



  After thinking about it this much, it looks like the buildroot license 
is a lot more lenient than I first thought. But of course, I don't really 
now.


  Maybe we should organize a discussion about this in the legal track at 
FOSDEM.


  Regards,
  Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 2/3] Add support for BR2_EXTERNAL

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Fri, 13 Sep 2013 22:56:59 +0200, Arnout Vandecappelle wrote:

> > O= is stored in the generated makefile (as you know). However, it gets very
> > annoying to type BR2_EXTERNAL=/path/to/external and I don't always want to
> > export it as shell environment variable. Thus, adding that line to
> > mkmakefile is handy since usually when you specify O= you are doing a
> > specific build which BR2_EXTERNAL would definitely be a part of.
> 
>   My point is: it should (and can) be made part of the .config. It really 
> has to be part of the .config, because some options in the .config will 
> refer to the packages that are defined in the external directory.

I believe it cannot be part of the .config, because there is a
chicken-and-egg problem due to the fact that running 'make menuconfig'
requires to already know the BR2_EXTERNAL value.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Arnout, All,

Once again, I am not a lawyer, and the below is not legal advice.
Anyone sane would consult a lawyer for a definitive legal counsel.

On 2013-09-13 23:50 +0200, Arnout Vandecappelle spake thusly:
> On 13/09/13 00:12, Yann E. MORIN wrote:
> >On 2013-09-13 00:04 +0200, Arnout Vandecappelle spake thusly:
> [snip]
> >>but the GPL is pretty
> >>clear on this:
> >>
> >>"If identifiable sections of that work are not derived from the Program,
> >>and can be reasonably considered independent and separate works in
> >>themselves, then this License, and its terms, do not apply to those
> >>sections when you distribute them as separate works."
> >>
> >>  i.e. you can distribute the source of buildroot itself separately from your
> >>external directory.
> >
> >Yes, but I believe BR2_EXTERNAL *is* a derived work of Buildroot, so as
> >such should be licensed under a license compatible with Buildroot's.
> >
> >>  As long as your external stuff merely aggregates with the GPL'd stuff under
> >>buildroot, there shouldn't be an issue.
> >
> >But that's not aggregation in my opinion, since it does leverage all or
> >parts of the Buildroot infrastrucutre, and was clearly written with this
> >intent, and as such can not be considered mere aggregation.
> >
> >Of course, all this only applies _if and when_ BR2_EXTERNAL gets
> >distributed to a third party.
> 
>  The more I think about it, the more confused I get :-) Although your other
> reply to Andy sums things up pretty clearly.
> 
>  Let me try out a few scenarios (I'm making things up as I go here).
> 
>  I have made some changes to buildroot, and I have also made an external
> tree.
> 
> 
> 1. I want to distribute the modified buildroot tree to someone. Do I also
> have to make the external one available?
> 
>  => No, because the buildroot tree in fact doesn't refer to the external
> tree. You run "BR2_EXTERNAL=... make".

Agreed, I think.

> 2. I distribute a rootfs which contains a GPL program (bash). Do I have to
> make the buildroot tree available?
> 
>  => Yes, because I have to make the build scripts for bash available, and
> buildroot is the build script that I've used.

Yes, because of section 3 of the GPL this program is licensed under.

Although strict compliance does not _require_ you to distribute your
Buildroot tree (since by giving the 'make' commands would be enough),
giving your Buildroot tree makes this so much easy that you would
probably want to do that rather than extract the commands run by
Buildroot.

> 3. I use buildroot to build lua. I copy the lua executable out of the
> target/ tree and distribute it. Do I have to make the buildroot tree
> available?
> 
>  => No, because lua is MIT licensed.

Indeed, no. But that does not preclude any other reason why you would
have to distribute your Buildroot tree (eg. case 2, above).

> 4. I distribute a rootfs which contains a GPL program (bash). It also
> contains a proprietary program (foo). I have added a foo.mk file to the
> buildroot tree to build my proprietary program. Do I have to make the
> modified buildroot tree available?
> 
>  => Probably not. I have to make the build script for bash available, but
> the original buildroot tree already does the job.
> 
>  I believe that this reasoning holds regardless of whether the foo.mk is
> added directly in the buildroot tree or through BR2_EXTERNAL (or through
> override.mk).

These are muddy waters.

If the unmodified Buildroot tree is enough to rebuild the GPL program,
then I would not see any valid reason to object.

But you'd have to have a *very good* process that ensures that your
Buildroot modifications do *not* impact the build of the GPL program.
A very good process, indeed, as soon as your team is more than a (very)
few people.

> 5. I add a proprietary binary to the default skeleton and distribute the
> rootfs built by buildroot. Do I have to make the modified buildroot with the
> proprietary binary available?
> 
>  => I don't think so, because the proprietary binary is merely aggregated
> with the rest of the buildroot-generated stuff.

Same answer as case 4, above. Be sure to have a very good process.

> 6. Can I give the buildroot tree + the external tree to someone else?
> 
>  => Yes, but only under the GPL - the GPL applies to the external tree as
> well.

(Note: when I say 'BR2_EXTERNAL', I'm speaking about the content of the
directory pointed to by the value of the variable BR2_EXTERNAL, unless
otherwise stated.)

That's all I was arguing about: BR2_EXTERNAL is a derived work of
Buildroot, so is bound to the same licensing terms as Buildroot is.

Since Buildroot's license is the GPLv2, BR2_EXTERNAL is covered by the
GPLv2.

Since the GPLv2 is a license which terms apply at the time of
distribution, as long as you do not distribute BR2_EXTERNAL, you are
free to do whatever you want with BR2_EXTERNAL. But as soon as you
distribute BR2_EXTERNAL (either because you decide to do so on your own
will, or because you are required to, as in case 2, above), you are
bound to distribute it under the terms of the GPLv2 (or arguably under a
license that is compatible with the GPLv2, eg. BSD).


Now let me add a few more cases:

7. I add a new package to Buildroot, for a GPL program, but do so in
   BR2_EXTERNAL.

   Do I have to distribute my BR2_EXTERNAL?
   Do I have to distribute my Buildroot tree?

=> Yes to both questions, for the same reasons as case 2, above.


8. My BR2_EXTERNAL contains the package described in case 7, above, plus
   a package for a proprietary program.

   Can I expunge the package for the proprietary program from BR2_EXTERNAL
   before I distribute it?

=> Yes, this is the same as cases 4 and 5, above.

====

Also, I was wondering if we could not have BR2_EXTERNAL (the variable)
be a list of external directories (colon-separated, a-la PATH)?

I would use it that way:
    BR2_EXTERNAL="/path/to/my/local/GPL/packages"
    BR2_EXTERNAL+=":/path/to/my/proprietary/packages"
    export BR2_EXTERNAL
    make ....

This would even make it much easier to separate proprietary packages, so
they do not leak, while making it easy to comply for GPL-like packages.

Thoughts?

====

>  After thinking about it this much, it looks like the buildroot license is a
> lot more lenient than I first thought. But of course, I don't really now.

I don't believe you could call the GPLv2 'lenient'. In fact I would call
it anything but lenient: anything that is a derived work of Buildroot is
covered by its license.

Note: the target rootfs is *not* a derived work of Buildroot.

>  Maybe we should organize a discussion about this in the legal track at
> FOSDEM.

Yes, that would make for a very good case on the (L)GPL.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas De Schampheleire]

On Fri, Sep 13, 2013 at 12:47 AM, Yann E. MORIN <yann.morin.1998@free.fr>
wrote:
> Andy, All,
>
> On 2013-09-12 22:28 +0000, ANDY KENNEDY spake thusly:
[..]
> What I'm arguing is that the content of BR2_EXTERNAL that _interacts
> with Buildroot's internal infrastructure_ *are* a derived work of
> Buildroot.
>
> Let's take a (very simplistic) example:
>
> BR2_EXTERNAL/
> package/
> pkg-1/
> Config.in *
> pkg-1.mk *
> pkg-1-main.c -
> pkg-1.h -
>
> * : a derived work of Buildroot
> - : not a derived work of Buildroot
>
> Config.in and pkg-1.mk are clearly (in my opinion) a derived work of
> Buildroot, since they are written with the very intent to be interacting
> with Buildroot's internal infrastructure.
>
> But pkg-1-main.c and pkg-1.h (which are the C sources of pkg-1) are *not*
> a derived work of Buildroot, and thus can well be under whatever license.

I agree with Yann here: the location of files is not relevant in this case:
clearly the .mk and Config files derive from buildroot. The .mk file would
use infrastructure like generic-package, and define setting such as
FOO_SOURCE, FOO_DEPENDENCIES, specifically to work with buildroot's core
infrastructure. This means that these derived files also become GPL.

Things would be different if we decide that the generic-package and other
infrastructures are an interface to buildroot (which I don't believe they
are). This is similar to why user-space applications running under the
Linux kernel (GPLv2) are not automatically GPLv2 too: the system call
interface is considered the standard interface between the kernel and
applications.

Regarding Andy's argument about making money: I have no objections about
someone selling a product made with buildroot. Same for creating a derived
build system, then selling it for big money. However, while doing so, the
gpl license must be honored, so source code of buildroot must be
distributed. As for anything in the gpl world, this means that purely
selling a product (the derived build system) doesn't make a lot of sense,
as any of your clients can distribute it freely. Instead, the value is now
in the innovation and service you can provide. On both aspects, the
buildroot community itself is already doing a great job at this for free,
so there is little value in forking buildroot for money IMO.


Best regards,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130915/068ff317/attachment.html>

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[ANDY KENNEDY]

> From: Yann E. MORIN [mailto:yann.morin.1998 at gmail.com] On Behalf Of Yann E. MORIN
> Sent: Saturday, September 14, 2013 5:16 PM

Everything you guys said here was sound, logical reasoning.  This is
what I believe to be the spirit of what *some* in the free software community
intended (though there are a few radicals out there which have taken it
down a different road).

<snip>
> > >Yes, but I believe BR2_EXTERNAL *is* a derived work of Buildroot, so as
> > >such should be licensed under a license compatible with BuildRoot's.

About this:  There was a BIG case here in the States within the last
two years.  Google vs. Sun Micro systems.  The argument was over the header
files of the language Java (if you already know this, sorry for the recap).
It was decided that an INTERFACE is not protected by _anything_.  So,
from a certain point of view, the *.mk files contain various variables
and then make a FUNCTION call back into the system.  This would negate there
being a "derived work" for anyone adding on an IP app to the build system.

However, you are dead-on about the GPL apps that may be built that way as
one would ABSOLUTELY have to release the build process with the binary.

So, whether this would work in courts across the globe, I have no idea.
Ethically speaking (and I'm big on ethics), I would not want to release IP
code that could _ONLY_ be built using BuildRoot, however, adding something
in that doesn't DEPEND on being built by BuildRoot is clearly okay by
our license and the GPL under which BuildRoot is released.

Now, as Yann has stated previously: the information above and 1.25USD will
buy you a Coke!

Andy

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Hello,

On Sun, 15 Sep 2013 00:16:13 +0200, Yann E. MORIN wrote:

> Since Buildroot's license is the GPLv2, BR2_EXTERNAL is covered by the
> GPLv2.

Probably going to make waves, but I'm wondering if the GPLv2 is the
right license for Buildroot. I believe it is not very easy to
understand how the terms of the GPL apply to something such as a build
system, and I am not sure that the GPL copyleft requirements are really
benefiting to Buildroot in any way. I am pretty sure that the vast
majority of companies using Buildroot are not really realizing it's
licensed under the GPL and therefore are not complying with the
Buildroot license terms (while they probably do realize that the
kernel, U-Boot, etc. are under the GPL and comply with their terms). Of
course that's not an argument to change the license, but I believe it
also highlights how hard it is to understand the GPL requirements on
the Buildroot case.

So, I believe that we should either:

 (1) Clarify and document how we believe the GPL terms apply to
     Buildroot (this would probably be a long discussion process, in
     which the SFLC should probably participate). When I see the
     discussions around BR2_EXTERNAL where the package .mk files and
     Config.in files may be seen as derivative work, but not the root
     filesystem overlay, or that package .mk files for GPL packages
     should be under the GPL, but not necessarily .mk files for non-GPL
     packages, I believe it is way too complicated for users. To me, it
     seems like complying with the Buildroot license is more
     complicated than using Buildroot itself, which is kind of silly.

 (2) Change the Buildroot license to a non-copyleft license. Of course,
     that requires contacting a lot of people, but maybe not so much:
     over the last 3-4 years, the vast majority of the Buildroot code
     base has been rewritten, and many of the people having worked on
     that are still around today.

Thoughts?

What other build systems are doing:

 * Yocto/OpenEmbeded: bitbake is under GPLv2, the rest (package
   recipes) is under MIT.

 * PTXdist is under GPLv2, but has a small license clarification "Note:
   PTXdist is a build system which generates a distribution for
   embedded systems. This license does only cover the build system
   itself and doesn't affect the license of managed software in any
   way, even if it might be distributed together with the build
   system." I believe it doesn't really clarify completely how the GPL
   applies to a build system.

 * OpenBricks is under GPLv2.

 * OpenWRT is under GPLv2, since it is originally a fork of Buildroot.

 * LTIB is under GPLv2.

Any other build system around worth having a look at?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Thomas, All,

On 2013-09-16 18:21 +0200, Thomas Petazzoni spake thusly:
> On Sun, 15 Sep 2013 00:16:13 +0200, Yann E. MORIN wrote:
> 
> > Since Buildroot's license is the GPLv2, BR2_EXTERNAL is covered by the
> > GPLv2.
> 
> Probably going to make waves,

Probably, yes. ;-)

> but I'm wondering if the GPLv2 is the
> right license for Buildroot. I believe it is not very easy to
> understand how the terms of the GPL apply to something such as a build
> system, and I am not sure that the GPL copyleft requirements are really
> benefiting to Buildroot in any way. I am pretty sure that the vast
> majority of companies using Buildroot are not really realizing it's
> licensed under the GPL and therefore are not complying with the
> Buildroot license terms (while they probably do realize that the
> kernel, U-Boot, etc. are under the GPL and comply with their terms).

On the other hand, the GPLv2 only applies at the time of distribution.
So long as the Buildroot tre is not distributed, there is no reason to
fear anything.

Now, let's try to make things clear:

  - on an embedded system, the probability that there is a GPL program
    is rather high (eg. busybox, the Linux kernel);

  - lets assume Buildroot is used to build those programs;

  - the GPL (as applied to _those_ programs, not Buildroot) mandates
    that the script to control compilation and installation of those
    programs be made available (section of GPLv2);

  - so the easiest way to comply with those programs' GPL is to
    distribute the Buildroot tree that was used to build the target
    filesystem, since it does contain all required recipes (aka the
    scripts of section 3 of the GPLv2)

  - Buildroot is itself GPLv2, so by distributing the Buildroot tree, a
    company has to release it under GPLv2.

> Of
> course that's not an argument to change the license, but I believe it
> also highlights how hard it is to understand the GPL requirements on
> the Buildroot case.

Sorry, but it is very clear to me, and I am no lawyer. And any decent
company will have a rather decent team of lawyers to help in these types
of cases.

> So, I believe that we should either:
> 
>  (1) Clarify and document how we believe the GPL terms apply to
>      Buildroot (this would probably be a long discussion process, in
>      which the SFLC should probably participate). When I see the
>      discussions around BR2_EXTERNAL where the package .mk files and
>      Config.in files may be seen as derivative work, but not the root
>      filesystem overlay, or that package .mk files for GPL packages
>      should be under the GPL,

It all revolves around the definition of a "derivative work".

We have to decide whether BR2_EXTERNAL is, or parts of it are, a derived
work of Buildroot. I think you now know my position, I guess. ;-)

>      but not necessarily .mk files for non-GPL
>      packages,

Wrong. *If* they are distributed, they'll be under the GPLv2.
But since they build proprietary (or weak-copyleft licensed) packages,
it is _not_ required that they be distributed.

>      I believe it is way too complicated for users. To me, it
>      seems like complying with the Buildroot license is more
>      complicated than using Buildroot itself, which is kind of silly.


>  (2) Change the Buildroot license to a non-copyleft license.

That would be a *hard* thing to do, I'm afraid.

>      Of course,
>      that requires contacting a lot of people, but maybe not so much:
>      over the last 3-4 years, the vast majority of the Buildroot code
>      base has been rewritten, and many of the people having worked on
>      that are still around today.
> 
> Thoughts?

I lean toward an explanation in the manual (with proper disclaimer that
entice the user to seek legal counsel).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Yann E. MORIN]

Andy, All,

On 2013-09-16 15:43 +0000, ANDY KENNEDY spake thusly:
> > From: Yann E. MORIN [mailto:yann.morin.1998 at gmail.com] On Behalf Of Yann E. MORIN
> > Sent: Saturday, September 14, 2013 5:16 PM
> 
> Everything you guys said here was sound, logical reasoning.  This is
> what I believe to be the spirit of what *some* in the free software community
> intended (though there are a few radicals out there which have taken it
> down a different road).
> 
> <snip>
> > > >Yes, but I believe BR2_EXTERNAL *is* a derived work of Buildroot, so as
> > > >such should be licensed under a license compatible with BuildRoot's.
> 
> About this:  There was a BIG case here in the States within the last
> two years.  Google vs. Sun Micro systems.

Yes, that story has crossed the ocean. ;-)

>  The argument was over the header
> files of the language Java (if you already know this, sorry for the recap).
> It was decided that an INTERFACE is not protected by _anything_.

That's where we can argue: is the Buildroot-specific infrastructure an
interface, or is it not an interface.

That's the same argument that applies to kernel modules: if a module is
written explicitly for the Linux kernel, then some kernel developpers
believe that module is a derived work of the Linux kernel. If the module
is pre-existing, and merely ported over to the Linux interfaces, it
might not be a derived work.

Although this is still considered a "gray area", and has not been tried
yet, so we can't use that as anything but a guideline of thought.

>  So,
> from a certain point of view, the *.mk files contain various variables
> and then make a FUNCTION call back into the system.  This would negate there
> being a "derived work" for anyone adding on an IP app to the build system.

I think you are confused by your "IP app" and the "recipes to build your
IP app".

Of course your proprietary application is not covered by the Buildroot
license. So yes, you can bundle it in your rootfs, and not be bound to
provide a means to rebuild it.

> However, you are dead-on about the GPL apps that may be built that way as
> one would ABSOLUTELY have to release the build process with the binary.

Yes, this is section 3 of the GPLv2. Excerpt:

    ---8<---
    For an executable work, complete source
    code means all the source code for all modules it contains, plus any
    associated interface definition files, plus the scripts used to
    control compilation and installation of the executable.
    ---8<---

So, if you distribute a binary of busybox on your device, you have to
provide the recipient of that device with:

  - the source code of busybox itself (ie the .c and .h files)

  - the build instructions to reproduce that busybox binary

To fulfill that second point, you have two options:

  - provide your Buildroot tree
or
  - provide the exact commands to rebuild busybox

Since the former is way easier than the latter, you'd probably opt for
providing the Buildroot tree.

Now, if your Buildroot tree also contains recipes for proprietary
application (what you call "IP app"), you can well expunge your Buildroot
tree from those recipes, and distribute that expunged Buildroot tree to
the recipient of your device, since that expunged Buildroot tree is
sufficient to rebuild the aforementioned busybox binary.

Indeed, that's not the exact Buildroot tree used to build busybox, but
since it is absolutely sufficient, that would not be a problem at all.

But expunging the recipes for the proprietary app would need a proper
process that guarantees it does not "leak" changes to other recipes for
non-proprietary apps.

Enters now BR2_EXTERNAL. If your BR2_EXTERNAL only contains recipes for
proprietary applications, and there are none in your Buildroot tree, you
will *not* have to provide that BR2_EXTERNAL tree@all.

But if your BR2_EXTERNAL tree contains recipes for GPL applications that
end up in your device, you will *have* to provide *those* recipes.

Ditto for the Buildroot tree, you may expunge the recipes for your
proprietary apps from the BR2_EXTERNAL tree you distribute, since they
are not required to rebuild the GPL applications.

> So, whether this would work in courts across the globe, I have no idea.
> Ethically speaking (and I'm big on ethics), I would not want to release IP
> code that could _ONLY_ be built using BuildRoot, however, adding something
> in that doesn't DEPEND on being built by BuildRoot is clearly okay by
> our license and the GPL under which BuildRoot is released.

OK, so you basically said there is no problem. Good! :-)

> Now, as Yann has stated previously: the information above and 1.25USD will
> buy you a Coke!

Indeed, the above would have to be tried in a court of law.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[ANDY KENNEDY]

> From: Yann E. MORIN [mailto:yann.morin.1998 at gmail.com] On Behalf Of Yann E. MORIN
> Sent: Monday, September 16, 2013 12:08 PM
> 
> Thomas, All,
> 
> On 2013-09-16 18:21 +0200, Thomas Petazzoni spake thusly:
> > On Sun, 15 Sep 2013 00:16:13 +0200, Yann E. MORIN wrote:
> >
> > > Since Buildroot's license is the GPLv2, BR2_EXTERNAL is covered by the
> > > GPLv2.
> >
> > Probably going to make waves,
> 
> Probably, yes. ;-)
> 
> > but I'm wondering if the GPLv2 is the
> > right license for Buildroot. I believe it is not very easy to
> > understand how the terms of the GPL apply to something such as a build
> > system, and I am not sure that the GPL copyleft requirements are really
> > benefiting to Buildroot in any way. I am pretty sure that the vast
> > majority of companies using Buildroot are not really realizing it's
> > licensed under the GPL and therefore are not complying with the
> > Buildroot license terms (while they probably do realize that the
> > kernel, U-Boot, etc. are under the GPL and comply with their terms).
> 
> On the other hand, the GPLv2 only applies at the time of distribution.
> So long as the Buildroot tre is not distributed, there is no reason to
> fear anything.
> 
> Now, let's try to make things clear:
> 
>   - on an embedded system, the probability that there is a GPL program
>     is rather high (eg. busybox, the Linux kernel);

Not always.  There is still a high contingent in the States that use
VXWorks, and then a lesser population use a home-grown kernel without
every touching GPL code.


> 
>   - lets assume Buildroot is used to build those programs;
> 
>   - the GPL (as applied to _those_ programs, not Buildroot) mandates
>     that the script to control compilation and installation of those
>     programs be made available (section of GPLv2);
> 
>   - so the easiest way to comply with those programs' GPL is to
>     distribute the Buildroot tree that was used to build the target
>     filesystem, since it does contain all required recipes (aka the
>     scripts of section 3 of the GPLv2)
> 
>   - Buildroot is itself GPLv2, so by distributing the Buildroot tree, a
>     company has to release it under GPLv2.

I agree with Yann on this point.  I don't see a valid reason to change
the license of BuildRoot (even if you wanted to make it a less restrictive
license for the main package/*.mk files).  In my current company, I have
discussed in great detail exactly what must be done in order to comply
with the GPL.  The team I'm on has taken great care to isolate the code
into two sections:  Open Sourced code (including MIT license stuff, etc)
and IP code.  I have stressed the importance of carefully considering
how applications are made, what dependencies we have on the applications,
etc.  My company is on-board with (when we get to the point in which
we are shipping Linux as the OS on a product) releasing all the
non-company code.

I believe that, as Yann states below (more to comment on this), a company
acting in "good faith" has the desire to "do the right thing"(tm) ;).  I
would expect that the part that Thomas said is more likely the case.
People who are using BuildRoot may be breaking the license agreement
without intending to, nor without the knowledge they are doing so. In
keeping with his thought, I would support writing a fairly detailed
document (probably should even put a short "comment" in Kconfig about
it) to detail what we believe to be the "right thing".

> 
> > Of
> > course that's not an argument to change the license, but I believe it
> > also highlights how hard it is to understand the GPL requirements on
> > the Buildroot case.
> 
> Sorry, but it is very clear to me, and I am no lawyer. And any decent
> company will have a rather decent team of lawyers to help in these types
> of cases.

Don't assume to much on this one.  Though I have been in the Opened Source
community since 1993, I have seen many companies that didn?t have any
legal support on this.  In fact, I have seen SEVERAL law companies that,
though they are experts on licensing, etc, have never dealt with the GPL.

In the previous three companies that I worked for, there was hardly a
concern about licensing issues as the customer was the US army :\.  So,
we were careful, in those other companies, to package up all the open
sourced code, but we didn't really _do_ anything with it.

> 
> > So, I believe that we should either:
> >
> >  (1) Clarify and document how we believe the GPL terms apply to
> >      Buildroot (this would probably be a long discussion process, in
> >      which the SFLC should probably participate). When I see the
> >      discussions around BR2_EXTERNAL where the package .mk files and
> >      Config.in files may be seen as derivative work, but not the root
> >      filesystem overlay, or that package .mk files for GPL packages
> >      should be under the GPL,
> 
> It all revolves around the definition of a "derivative work".
> 
> We have to decide whether BR2_EXTERNAL is, or parts of it are, a derived
> work of Buildroot. I think you now know my position, I guess. ;-)

Well, In your e-mail (which predates this e-mail by a few minutes)
that starts out: 

	Andy, All,
	
	On 2013-09-16 15:43 +0000, ANDY KENNEDY spake thusly:

You have expounded on what you believe to a derived work.  What you stated
clearly in that e-mail was not the same meaning I derived from your former
e-mails.  I have no problem with your definition of a derived work, thus
I you and I are on the same side of this issue.

<snip>
<Crap!  snipped too much>

Yann, You stated that you would like to see further documentation that
clearly sets guidelines on this matter.  I stated this last week several
times, therefore I completely agree with you.

How ever we choose to license this, we should EXPLICITLY state this in
documentation and CLEARLY show where this document is.  If it comes down
to a vote of current developers, I would like to even put that into the
document, perhaps with a  S H O R T  comment as to the viewpoint of the
voter ;) so that in 10 years there is no confusion about our points of
view to the next generation of BuildRoot users and developers.

Andy

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Hello,

On Mon, 16 Sep 2013 19:08:15 +0200, Yann E. MORIN wrote:

> > but I'm wondering if the GPLv2 is the
> > right license for Buildroot. I believe it is not very easy to
> > understand how the terms of the GPL apply to something such as a
> > build system, and I am not sure that the GPL copyleft requirements
> > are really benefiting to Buildroot in any way. I am pretty sure
> > that the vast majority of companies using Buildroot are not really
> > realizing it's licensed under the GPL and therefore are not
> > complying with the Buildroot license terms (while they probably do
> > realize that the kernel, U-Boot, etc. are under the GPL and comply
> > with their terms).
> 
> On the other hand, the GPLv2 only applies at the time of distribution.
> So long as the Buildroot tre is not distributed, there is no reason to
> fear anything.

Right, that's true.

> Now, let's try to make things clear:
> 
>   - on an embedded system, the probability that there is a GPL program
>     is rather high (eg. busybox, the Linux kernel);
> 
>   - lets assume Buildroot is used to build those programs;
> 
>   - the GPL (as applied to _those_ programs, not Buildroot) mandates
>     that the script to control compilation and installation of those
>     programs be made available (section of GPLv2);

Whether Buildroot is part of such scripts or not remains a slightly
open question. That the kernel makefiles and scripts, the Busybox
makefiles and scripts should remain part of the kernel and Busybox
sources is clear. That the tool that is used to orchestrate the overall
building process is part of those "script to control compilation and
installation" is not so clear-cut from my point of view.

At least so far, I don't think I've seen many companies using
Buildroot, on products that include GPL components, providing the
source code for Buildroot.

>   - so the easiest way to comply with those programs' GPL is to
>     distribute the Buildroot tree that was used to build the target
>     filesystem, since it does contain all required recipes (aka the
>     scripts of section 3 of the GPLv2)

Right, but it is not necessarily easy to separate within Buildroot the
thing that you are ready to distribute (package recipes of open-source
programs) from the package recipes of proprietary programs, your root
filesystem overlay and so on.

>   - Buildroot is itself GPLv2, so by distributing the Buildroot tree,
> a company has to release it under GPLv2.

Yes, that's for sure, I'm not questioning that.

What I'm questioning is really the case where a company makes an
embedded product, and has used Buildroot to generate a rootfs that
includes GPL programs, is this company required to distribute Buildroot.

> > Of
> > course that's not an argument to change the license, but I believe
> > it also highlights how hard it is to understand the GPL
> > requirements on the Buildroot case.
> 
> Sorry, but it is very clear to me, and I am no lawyer. And any decent
> company will have a rather decent team of lawyers to help in these
> types of cases.

That might be the case where you work, but I honestly don't believe
it's the case in many companies, especially SMEs doing embedded
products.

> > So, I believe that we should either:
> > 
> >  (1) Clarify and document how we believe the GPL terms apply to
> >      Buildroot (this would probably be a long discussion process, in
> >      which the SFLC should probably participate). When I see the
> >      discussions around BR2_EXTERNAL where the package .mk files and
> >      Config.in files may be seen as derivative work, but not the
> > root filesystem overlay, or that package .mk files for GPL packages
> >      should be under the GPL,
> 
> It all revolves around the definition of a "derivative work".
> 
> We have to decide whether BR2_EXTERNAL is, or parts of it are, a
> derived work of Buildroot. I think you now know my position, I
> guess. ;-)

Right.

> >      but not necessarily .mk files for non-GPL
> >      packages,
> 
> Wrong. *If* they are distributed, they'll be under the GPLv2.
> But since they build proprietary (or weak-copyleft licensed) packages,
> it is _not_ required that they be distributed.

Yes, hence I said "not necessarily".

> >      I believe it is way too complicated for users. To me, it
> >      seems like complying with the Buildroot license is more
> >      complicated than using Buildroot itself, which is kind of
> > silly.
> 
> 
> >  (2) Change the Buildroot license to a non-copyleft license.
> 
> That would be a *hard* thing to do, I'm afraid.

Yes, license changes are difficult. Not impossible (VLC changed its
license, for example), but certainly annoying.

> I lean toward an explanation in the manual (with proper disclaimer
> that entice the user to seek legal counsel).

Fair enough. Care to submit a patch? :-)

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Dear ANDY KENNEDY,

On Mon, 16 Sep 2013 17:45:00 +0000, ANDY KENNEDY wrote:

> >   - on an embedded system, the probability that there is a GPL
> > program is rather high (eg. busybox, the Linux kernel);
> 
> Not always.  There is still a high contingent in the States that use
> VXWorks, and then a lesser population use a home-grown kernel without
> every touching GPL code.

I believe Yann was implicitly saying "embedded Linux system". Buildroot
is of no-use for VxWorks based systems or other non-Linux operating
systems.

> I agree with Yann on this point.  I don't see a valid reason to change
> the license of BuildRoot (even if you wanted to make it a less
> restrictive license for the main package/*.mk files).  In my current
> company, I have discussed in great detail exactly what must be done
> in order to comply with the GPL.  The team I'm on has taken great
> care to isolate the code into two sections:  Open Sourced code
> (including MIT license stuff, etc) and IP code.  I have stressed the
> importance of carefully considering how applications are made, what
> dependencies we have on the applications, etc.  My company is
> on-board with (when we get to the point in which we are shipping
> Linux as the OS on a product) releasing all the non-company code.

Sure. But in Buildroot, things are more complicated: within Buildroot
*itself* there may be parts that you have to redistribute (i.e package
recipes for GPL programs), and some other parts you may not be willing
to redistribute (i.e package recipes for your own applications, or your
root filesystem overlay). This makes it quite difficult for a company
using Buildroot to easily separate what must be distributed from what
shouldn't be distributed.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Thomas, All,

On 2013-09-16 19:58 +0200, Thomas Petazzoni spake thusly:
> On Mon, 16 Sep 2013 19:08:15 +0200, Yann E. MORIN wrote:
> > Now, let's try to make things clear:
> > 
> >   - on an embedded system, the probability that there is a GPL program
> >     is rather high (eg. busybox, the Linux kernel);
> > 
> >   - lets assume Buildroot is used to build those programs;
> > 
> >   - the GPL (as applied to _those_ programs, not Buildroot) mandates
> >     that the script to control compilation and installation of those
> >     programs be made available (section of GPLv2);
> 
> Whether Buildroot is part of such scripts or not remains a slightly
> open question. That the kernel makefiles and scripts, the Busybox
> makefiles and scripts should remain part of the kernel and Busybox
> sources is clear. That the tool that is used to orchestrate the overall
> building process is part of those "script to control compilation and
> installation" is not so clear-cut from my point of view.
> 
> At least so far, I don't think I've seen many companies using
> Buildroot, on products that include GPL components, providing the
> source code for Buildroot.

I never said you *had* to provide the Buildroot tree. I said that
providing it was *easier* than trying to separately provide build
instructions for each GPL program on the device.

> >   - so the easiest way to comply with those programs' GPL is to
> >     distribute the Buildroot tree that was used to build the target
> >     filesystem, since it does contain all required recipes (aka the
> >     scripts of section 3 of the GPLv2)
> 
> Right, but it is not necessarily easy to separate within Buildroot the
> thing that you are ready to distribute (package recipes of open-source
> programs) from the package recipes of proprietary programs, your root
> filesystem overlay and so on.

But your BR2_EXTERNAL will make this really trivial:
  - do all FLOSS stuff in Buildroot tree
  - do all proprietary stuff in BR2_EXTERNAL
  - provide the Buidlroot tree to be compliant
  - keep BR2_EXTERNAL private

Et voil?! :-)

> 
> What I'm questioning is really the case where a company makes an
> embedded product, and has used Buildroot to generate a rootfs that
> includes GPL programs, is this company required to distribute Buildroot.

Short answer: no.
Long answer: no, but it is an easy path.

> > I lean toward an explanation in the manual (with proper disclaimer
> > that entice the user to seek legal counsel).
> 
> Fair enough. Care to submit a patch? :-)

I sure can help write and review such a change.

We just have to settle on what we want to put in there: BR2_EXTERNAL:
derived work or not? ;-)

But not before next week, I have to finish my presentations for KR`13
first! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Thomas, Andy, All,

On 2013-09-16 20:01 +0200, Thomas Petazzoni spake thusly:
> On Mon, 16 Sep 2013 17:45:00 +0000, ANDY KENNEDY wrote:
> > >   - on an embedded system, the probability that there is a GPL
> > > program is rather high (eg. busybox, the Linux kernel);
> > 
> > Not always.  There is still a high contingent in the States that use
> > VXWorks, and then a lesser population use a home-grown kernel without
> > every touching GPL code.
> 
> I believe Yann was implicitly saying "embedded Linux system". Buildroot
> is of no-use for VxWorks based systems or other non-Linux operating
> systems.

Yes, of course. :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Mon, 16 Sep 2013 20:15:12 +0200, Yann E. MORIN wrote:

> > Whether Buildroot is part of such scripts or not remains a slightly
> > open question. That the kernel makefiles and scripts, the Busybox
> > makefiles and scripts should remain part of the kernel and Busybox
> > sources is clear. That the tool that is used to orchestrate the overall
> > building process is part of those "script to control compilation and
> > installation" is not so clear-cut from my point of view.
> > 
> > At least so far, I don't think I've seen many companies using
> > Buildroot, on products that include GPL components, providing the
> > source code for Buildroot.
> 
> I never said you *had* to provide the Buildroot tree. I said that
> providing it was *easier* than trying to separately provide build
> instructions for each GPL program on the device.

Right. But are there any companies doing any of above? I don't think
I've ever seen a document, or script, or anything like that that
details the build process of the overall system.

I remain unsure that what Buildroot is doing is part of the scripts
governing the compilation of the GPL components.

> > Right, but it is not necessarily easy to separate within Buildroot the
> > thing that you are ready to distribute (package recipes of open-source
> > programs) from the package recipes of proprietary programs, your root
> > filesystem overlay and so on.
> 
> But your BR2_EXTERNAL will make this really trivial:
>   - do all FLOSS stuff in Buildroot tree
>   - do all proprietary stuff in BR2_EXTERNAL
>   - provide the Buidlroot tree to be compliant
>   - keep BR2_EXTERNAL private
> 
> Et voil?! :-)

What a good selling argument for BR2_EXTERNAL :-)

But it isn't quite sufficient: by your definition, the
Buildroot .config should also be shipped, and it also contains config
options for the company-specific proprietary packages. So the .config
should be "filtered" before being released. Not simple.

> > What I'm questioning is really the case where a company makes an
> > embedded product, and has used Buildroot to generate a rootfs that
> > includes GPL programs, is this company required to distribute Buildroot.
> 
> Short answer: no.
> Long answer: no, but it is an easy path.

Aren't companies going to be afraid of revealing too much of their
secret sauce if they have to disclose their Buildroot tree?

My point is also that nobody has ever cared about enforcing the
Buildroot license, and I sincerely doubt anyone will be willing to do
in the foreseeable future. Since no-one cares about that, is it really
necessary to annoy companies with the fairly fuzzy requirements of the
GPL when applied to a build system?

> We just have to settle on what we want to put in there: BR2_EXTERNAL:
> derived work or not? ;-)

If it's considered as a derived work, then your above idea of using
BR2_EXTERNAL to split things between secret stuff and released stuff no
longer works.

> But not before next week, I have to finish my presentations for KR`13
> first! ;-)

Argh, I should do that too!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Mon, 16 Sep 2013 19:30:46 +0200, Yann E. MORIN wrote:

> >  So,
> > from a certain point of view, the *.mk files contain various variables
> > and then make a FUNCTION call back into the system.  This would negate there
> > being a "derived work" for anyone adding on an IP app to the build system.
> 
> I think you are confused by your "IP app" and the "recipes to build your
> IP app".

By the way, "IP app", or "IP" doesn't mean anything. Why would
"Intellectual Property" refer to something that is
proprietary/non-free? That doesn't make any sense.

Can't we use "proprietary applications", or "private applications"
instead? Intellectual property really has nothing to do with whether
something is closed-source or open-source.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[Arnout Vandecappelle]

On 15/09/13 00:16, Yann E. MORIN wrote:
> Arnout, All,
>
> Once again, I am not a lawyer, and the below is not legal advice.
> Anyone sane would consult a lawyer for a definitive legal counsel.

[snip]

  I'm not going to quote the entire mail, but I agree with Yann's analysis.


[snip]
>>   After thinking about it this much, it looks like the buildroot license is a
>> lot more lenient than I first thought. But of course, I don't really now.
>
> I don't believe you could call the GPLv2 'lenient'. In fact I would call
> it anything but lenient: anything that is a derived work of Buildroot is
> covered by its license.
>
> Note: the target rootfs is *not* a derived work of Buildroot.

  With "lenient" I mean that, even though the GPL forces you to make the 
buildroot source available if you use it to build a GPL'ed program, there 
is in fact nothing compelling you to make the _complete_ buildroot source 
(and .config) available - as long as it produces the same GPL'ed program.

  My intuitive assumption, without thinking about it, was that if you are 
compelled to make the buildroot source available, that also means that 
you're compelled to make your modifications (e.g. additional proprietary 
packages) available. However, our analysis in this thread seems to 
indicate that it doesn't. That's why I said it is more lenient than I 
thought.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Is GPLv2 the right license for Buildroot?

[ANDY KENNEDY]

> -----Original Message-----
> From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com]
> 
> Dear Yann E. MORIN,
> 
> On Mon, 16 Sep 2013 20:15:12 +0200, Yann E. MORIN wrote:
> 
> > > Whether Buildroot is part of such scripts or not remains a slightly
> > > open question. That the kernel makefiles and scripts, the Busybox
> > > makefiles and scripts should remain part of the kernel and Busybox
> > > sources is clear. That the tool that is used to orchestrate the overall
> > > building process is part of those "script to control compilation and
> > > installation" is not so clear-cut from my point of view.
> > >
> > > At least so far, I don't think I've seen many companies using
> > > Buildroot, on products that include GPL components, providing the
> > > source code for Buildroot.
> >
> > I never said you *had* to provide the Buildroot tree. I said that
> > providing it was *easier* than trying to separately provide build
> > instructions for each GPL program on the device.
> 
> Right. But are there any companies doing any of above? I don't think
> I've ever seen a document, or script, or anything like that that
> details the build process of the overall system.
> 
> I remain unsure that what Buildroot is doing is part of the scripts
> governing the compilation of the GPL components.

By reading the original license of Buildroot, I find that the license
does imply that one must provide the BR source tree too.  Though,
I am speaking without the aide of any council on my side.

> 
> > > Right, but it is not necessarily easy to separate within Buildroot the
> > > thing that you are ready to distribute (package recipes of open-source
> > > programs) from the package recipes of proprietary programs, your root
> > > filesystem overlay and so on.
> >
> > But your BR2_EXTERNAL will make this really trivial:
> >   - do all FLOSS stuff in Buildroot tree
> >   - do all proprietary stuff in BR2_EXTERNAL
> >   - provide the Buidlroot tree to be compliant
> >   - keep BR2_EXTERNAL private
> >
> > Et voil?! :-)
> 
> What a good selling argument for BR2_EXTERNAL :-)
> 
> But it isn't quite sufficient: by your definition, the
> Buildroot .config should also be shipped, and it also contains config
> options for the company-specific proprietary packages. So the .config
> should be "filtered" before being released. Not simple.

Hmmm, almost sounds like there should be a separate $(MAKE) -C into the
BR2_EXTERNAL directory for menuconfig.  Then, if BR2_EXTERNAL is set,
parse the file $(BR2_EXTERNAL)/.config as a separate entity.

> 
> > > What I'm questioning is really the case where a company makes an
> > > embedded product, and has used Buildroot to generate a rootfs that
> > > includes GPL programs, is this company required to distribute Buildroot.
> >
> > Short answer: no.
> > Long answer: no, but it is an easy path.
> 
> Aren't companies going to be afraid of revealing too much of their
> secret sauce if they have to disclose their Buildroot tree?
> 
> My point is also that nobody has ever cared about enforcing the
> Buildroot license, and I sincerely doubt anyone will be willing to do
> in the foreseeable future. Since no-one cares about that, is it really
> necessary to annoy companies with the fairly fuzzy requirements of the
> GPL when applied to a build system?

In this case, could we not provide the license to mean something like:

BuildRoot is  provided as a  convenience to companies / individuals that
desire  to build an embedded GNU / Linux system for personal pleasure or
for financial gain.   Though BuildRoot  is licensed under the GPLv2, the
intent of the licenser is to protect the developers of the code from any
third party wishing to seek  legal action in the event of harm done to a
system by the use/misuse of BuildRoot.  Also, the desire of the licenser
is to grant access of the  BuildRoot build system to  use in the  manner
that is most beneficial to the licensee. The expectation of the licensee
is that  you  will  provide the BuildRoot build  system  for all  binary
kernels, root file systems, and any  other GPL code  that you build with
your  binaries as the  "build scripts"  for the related  GPL code.   The
licenser does not desire to seek legal action for the use in any form of
the BuildRoot build system, but asks only that those companies/individu-
als that use BuildRoot to make financial gain respect the  community and
help us to make  BuildRoot better.   If you believe  that BuildRoot is a
build system that can assist you, please PUSH BACK YOUR  IMPROVEMENTS TO
BUILDROOT!!

> 
> > We just have to settle on what we want to put in there: BR2_EXTERNAL:
> > derived work or not? ;-)
> 
> If it's considered as a derived work, then your above idea of using
> BR2_EXTERNAL to split things between secret stuff and released stuff no
> longer works.
> 
> > But not before next week, I have to finish my presentations for KR`13
> > first! ;-)
> 
> Argh, I should do that too!

Yeah, ya'll have fun with that :(.  There is no way for me to attend such
a function (lack of funds, wife, three kids with one on the way, etc).

Andy

[Buildroot] [PATCH 0/3] Support for out-of-tree Buildroot customization

[ANDY KENNEDY]

> -----Original Message-----
> From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com]
> 
> Dear Yann E. MORIN,
> 
> On Mon, 16 Sep 2013 19:30:46 +0200, Yann E. MORIN wrote:
> 
> > >  So,
> > > from a certain point of view, the *.mk files contain various variables
> > > and then make a FUNCTION call back into the system.  This would negate there
> > > being a "derived work" for anyone adding on an IP app to the build system.
> >
> > I think you are confused by your "IP app" and the "recipes to build your
> > IP app".
> 
> By the way, "IP app", or "IP" doesn't mean anything. Why would
> "Intellectual Property" refer to something that is
> proprietary/non-free? That doesn't make any sense.
> 
> Can't we use "proprietary applications", or "private applications"
> instead? Intellectual property really has nothing to do with whether
> something is closed-source or open-source.

Again, that is "common vernacular" in my world.  I'll try to adopt your
words while speaking on your list ;).

Andy

[Buildroot] Is GPLv2 the right license for Buildroot?

[Grant Edwards]

On 2013-09-16, Yann E. MORIN <yann.morin.1998@free.fr> wrote:

> Sorry, but it is very clear to me, and I am no lawyer. And any decent
> company will have a rather decent team of lawyers to help in these types
> of cases.

So no company with less that several hundred employees is "decent"?

-- 
Grant Edwards               grant.b.edwards        Yow! My EARS are GONE!!
                                  at               
                              gmail.com            

[Buildroot] Is GPLv2 the right license for Buildroot?

[Arnout Vandecappelle]

On 16/09/13 19:08, Yann E. MORIN wrote:
> Thomas, All,
>
> On 2013-09-16 18:21 +0200, Thomas Petazzoni spake thusly:
>> >On Sun, 15 Sep 2013 00:16:13 +0200, Yann E. MORIN wrote:
>> >
>>> > >Since Buildroot's license is the GPLv2, BR2_EXTERNAL is covered by the
>>> > >GPLv2.
>> >
>> >Probably going to make waves,
> Probably, yes.;-)
>
>> >but I'm wondering if the GPLv2 is the
>> >right license for Buildroot. I believe it is not very easy to
>> >understand how the terms of the GPL apply to something such as a build
>> >system, and I am not sure that the GPL copyleft requirements are really
>> >benefiting to Buildroot in any way. I am pretty sure that the vast
>> >majority of companies using Buildroot are not really realizing it's
>> >licensed under the GPL and therefore are not complying with the
>> >Buildroot license terms (while they probably do realize that the
>> >kernel, U-Boot, etc. are under the GPL and comply with their terms).
> On the other hand, the GPLv2 only applies at the time of distribution.
> So long as the Buildroot tre is not distributed, there is no reason to
> fear anything.
>
> Now, let's try to make things clear:
>
>    - on an embedded system, the probability that there is a GPL program
>      is rather high (eg. busybox, the Linux kernel);
>
>    - lets assume Buildroot is used to build those programs;
>
>    - the GPL (as applied to_those_  programs, not Buildroot) mandates
>      that the script to control compilation and installation of those
>      programs be made available (section of GPLv2);
>
>    - so the easiest way to comply with those programs' GPL is to
>      distribute the Buildroot tree that was used to build the target
>      filesystem, since it does contain all required recipes (aka the
>      scripts of section 3 of the GPLv2)
>
>    - Buildroot is itself GPLv2, so by distributing the Buildroot tree, a
>      company has to release it under GPLv2.

  I think this states the same as docs/manual/legal-notice.txt already 
does, though a bit easier to follow. That paragraph in the manual also 
clearly states that this is the interpretation of the buildroot developers.

  I think that for the users, things are still simple: just make 
buildroot available. It also takes away a large part of the burden of 
GPL-compliance, because buildroot takes care of downloading the source so 
you can consider that job done (at least according to the spirit of the 
license, and explicitly in GPLv3).

  As to the .config: would it really be an issue for your company that 
the end user can see that there is something called BR2_THE_APP installed 
on the system? Looking back at some buildroot projects I've done, the 
proprietary packages had names like "pir", "h264", "upgrade", "rt_app", 
"system". Like, that's really leaking information...

  And for the really paranoid companies that lock down their filesystem 
sufficiently that users can't read (and thereby discover those top-secret 
application names), I think they will can afford the 15-minute investment 
to grep away the proprietary packages from the .config.


  However, coming back to Thomas's idea of changing the license: the 
buildroot license is not really the issue here. Even if buildroot were 
MIT-licensed, the GPL of the target programs may still require you to 
provide the buildroot source. Since buildroot itself is source-only, 
there is really no big difference between GPL or any other free software 
license.


  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Thomas, All,

On 2013-09-16 20:24 +0200, Thomas Petazzoni spake thusly:
> On Mon, 16 Sep 2013 20:15:12 +0200, Yann E. MORIN wrote:
> > But your BR2_EXTERNAL will make this really trivial:
> >   - do all FLOSS stuff in Buildroot tree
> >   - do all proprietary stuff in BR2_EXTERNAL
> >   - provide the Buidlroot tree to be compliant
> >   - keep BR2_EXTERNAL private
> > 
> > Et voil?! :-)
> 
> What a good selling argument for BR2_EXTERNAL :-)

Hehe! :-)

Also, I would like to reiterate my suggestion the vriable BR2_EXTERNAL
contains a list (space- or colon-sperated, as you wish) of external
trees to descend into, rather than containing a single entry.

This way, one could have this layout:

    buildroot/    <- Buildroot tree with only redistributable changes
    ext-floss/    <- external tree with only FLOSS components
    ext-prop/     <- external tree with weak-copyleft or proprietary
                     components

which would be used as thus:
    make BR2_EXTERNAL="/path/to/ext-floss /path/to/ext-prop"

Then, distribute both buildroot/ and ext-floss/ and you're compliant
without leaking any sensitive stuff.

A side question to this: what if any of the external tree(s) contain a
package already in a previous external tree or in the original Buildroot
tree?

> But it isn't quite sufficient: by your definition, the
> Buildroot .config should also be shipped, and it also contains config
> options for the company-specific proprietary packages. So the .config
> should be "filtered" before being released. Not simple.

I think that is relatively easy: in the Buildroot tree, just run:
    make BR2_EXTERNAL= oldconfig

That will expunge the .config from any variable from the BR2_EXTERNAL
tree. You can then tar up the Buildroot tree, and make that available.
Quite easy, but requires a process that documents this simple extra step
prior to publication.

> > > What I'm questioning is really the case where a company makes an
> > > embedded product, and has used Buildroot to generate a rootfs that
> > > includes GPL programs, is this company required to distribute Buildroot.
> > 
> > Short answer: no.
> > Long answer: no, but it is an easy path.
> 
> Aren't companies going to be afraid of revealing too much of their
> secret sauce if they have to disclose their Buildroot tree?

Depends on what you call "their secret sauce". If you meant "their
prorietary apps packaged in Buildroot" then, curently, yes, since those
would be in the Buildroot tree. But with BR2_EXTERNAL (and the little
oldconfig trick above), that'd be straightforward.

If you meant "the fact that they actually use Buildroot", then I don;t
think how this could be seen as a "secret sauce".

> My point is also that nobody has ever cared about enforcing the
> Buildroot license, and I sincerely doubt anyone will be willing to do
> in the foreseeable future. Since no-one cares about that, is it really
> necessary to annoy companies with the fairly fuzzy requirements of the
> GPL when applied to a build system?
> 
> > We just have to settle on what we want to put in there: BR2_EXTERNAL:
> > derived work or not? ;-)
> 
> If it's considered as a derived work, then your above idea of using
> BR2_EXTERNAL to split things between secret stuff and released stuff no
> longer works.

Well, yes it still stands. The GPL only applies at the time of
distribution. So if BR2_EXTERNAL only contains recipes for
weak-copyleft or non-copyleft packages, then there is no need for the
company to distribute it, since the "official" Buildroot tree is enough
to reproduce the GPL program(s) on the target.

This applies, of course, only if the company decides to publish
Buildroot to catter for section 3 of the GPL of those programs.

If the company decides for another route, then this point is moot
anyway, since there is no distribution at all.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Grant, All,

On 2013-09-16 19:50 +0000, Grant Edwards spake thusly:
> On 2013-09-16, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> 
> > Sorry, but it is very clear to me, and I am no lawyer. And any decent
> > company will have a rather decent team of lawyers to help in these types
> > of cases.
> 
> So no company with less that several hundred employees is "decent"?

Yes, sorry, I've been pushing that one a bit too far.
You've served me right. I do apologise.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 Thomas> So, I believe that we should either:

 Thomas>  (1) Clarify and document how we believe the GPL terms apply to
 Thomas>      Buildroot (this would probably be a long discussion process, in
 Thomas>      which the SFLC should probably participate). When I see the
 Thomas>      discussions around BR2_EXTERNAL where the package .mk files and
 Thomas>      Config.in files may be seen as derivative work, but not the root
 Thomas>      filesystem overlay, or that package .mk files for GPL packages
 Thomas>      should be under the GPL, but not necessarily .mk files for non-GPL
 Thomas>      packages, I believe it is way too complicated for users. To me, it
 Thomas>      seems like complying with the Buildroot license is more
 Thomas>      complicated than using Buildroot itself, which is kind of silly.

We have already tried to clarify it in the user manual:

http://buildroot.net/downloads/manual/manual.html#_complying_with_the_buildroot_license

E.G. Buildroot should be handled just like how you handle the Linux
kernel and Busybox.


 Thomas>  (2) Change the Buildroot license to a non-copyleft license. Of course,
 Thomas>      that requires contacting a lot of people, but maybe not so much:
 Thomas>      over the last 3-4 years, the vast majority of the Buildroot code
 Thomas>      base has been rewritten, and many of the people having worked on
 Thomas>      that are still around today.

It's imho still too many people to be realistic:

git shortlog -s --since='3 years' | wc -l
230


 Thomas> What other build systems are doing:

 Thomas>  * Yocto/OpenEmbeded: bitbake is under GPLv2, the rest (package
 Thomas>    recipes) is under MIT.

 Thomas>  * PTXdist is under GPLv2, but has a small license clarification "Note:
 Thomas>    PTXdist is a build system which generates a distribution for
 Thomas>    embedded systems. This license does only cover the build system
 Thomas>    itself and doesn't affect the license of managed software in any
 Thomas>    way, even if it might be distributed together with the build
 Thomas>    system." I believe it doesn't really clarify completely how the GPL
 Thomas>    applies to a build system.

 Thomas>  * OpenBricks is under GPLv2.

 Thomas>  * OpenWRT is under GPLv2, since it is originally a fork of Buildroot.

 Thomas>  * LTIB is under GPLv2.

So they are basically all in the same boat as us.

-- 
Bye, Peter Korsgaard

[Buildroot] Is GPLv2 the right license for Buildroot?

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

Hi,

 Yann> Now, let's try to make things clear:

 Yann>   - on an embedded system, the probability that there is a GPL program
 Yann>     is rather high (eg. busybox, the Linux kernel);

 Yann>   - lets assume Buildroot is used to build those programs;

 Yann>   - the GPL (as applied to _those_ programs, not Buildroot) mandates
 Yann>     that the script to control compilation and installation of those
 Yann>     programs be made available (section of GPLv2);

 Yann>   - so the easiest way to comply with those programs' GPL is to
 Yann>     distribute the Buildroot tree that was used to build the target
 Yann>     filesystem, since it does contain all required recipes (aka the
 Yann>     scripts of section 3 of the GPLv2)

 Yann>   - Buildroot is itself GPLv2, so by distributing the Buildroot tree, a
 Yann>     company has to release it under GPLv2.

Indeed.

-- 
Bye, Peter Korsgaard

[Buildroot] Is GPLv2 the right license for Buildroot?

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 Thomas> Sure. But in Buildroot, things are more complicated: within Buildroot
 Thomas> *itself* there may be parts that you have to redistribute (i.e package
 Thomas> recipes for GPL programs), and some other parts you may not be willing
 Thomas> to redistribute (i.e package recipes for your own applications, or your
 Thomas> root filesystem overlay). This makes it quite difficult for a company
 Thomas> using Buildroot to easily separate what must be distributed from what
 Thomas> shouldn't be distributed.

I can only speak of personal experience, naturally (and you get to work
with a lot more companies than I do), but I haven't seen this as a big
problem. These things typically don't contain many "secrets".

People basically have 2 options:

- Integrate their changes in the Buildroot tree and gain the advantages
  of Buildroot

- Or keep it seperate (E.G add the stuff outside buildroot) and be able
  to keep it secret.

-- 
Bye, Peter Korsgaard

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Mon, 16 Sep 2013 22:04:20 +0200, Yann E. MORIN wrote:

> Also, I would like to reiterate my suggestion the vriable BR2_EXTERNAL
> contains a list (space- or colon-sperated, as you wish) of external
> trees to descend into, rather than containing a single entry.

Unfortunately, I don't think this is easy to do. Have you seen how the
BR2_EXTERNAL/Config.in gets included in the top-level menu? The way
we're doing this in the current implementation doesn't make it easy to
have several paths in BR2_EXTERNAL.

> A side question to this: what if any of the external tree(s) contain a
> package already in a previous external tree or in the original
> Buildroot tree?

I don't think we want to support overlay-ing packages.

> > But it isn't quite sufficient: by your definition, the
> > Buildroot .config should also be shipped, and it also contains
> > config options for the company-specific proprietary packages. So
> > the .config should be "filtered" before being released. Not simple.
> 
> I think that is relatively easy: in the Buildroot tree, just run:
>     make BR2_EXTERNAL= oldconfig
> 
> That will expunge the .config from any variable from the BR2_EXTERNAL
> tree. You can then tar up the Buildroot tree, and make that available.
> Quite easy, but requires a process that documents this simple extra
> step prior to publication.

Hum, right, makes sense.

> > Aren't companies going to be afraid of revealing too much of their
> > secret sauce if they have to disclose their Buildroot tree?
> 
> Depends on what you call "their secret sauce". If you meant "their
> prorietary apps packaged in Buildroot" then, curently, yes, since
> those would be in the Buildroot tree. But with BR2_EXTERNAL (and the
> little oldconfig trick above), that'd be straightforward.
> 
> If you meant "the fact that they actually use Buildroot", then I don;t
> think how this could be seen as a "secret sauce".

I was talking about their proprietary apps / rootfs overlay, etc. Which
indeed with BR2_EXTERNAL they can keep separate.

> > If it's considered as a derived work, then your above idea of using
> > BR2_EXTERNAL to split things between secret stuff and released
> > stuff no longer works.
> 
> Well, yes it still stands. The GPL only applies at the time of
> distribution. So if BR2_EXTERNAL only contains recipes for
> weak-copyleft or non-copyleft packages,

Yes, and this *if* is very important.

> then there is no need for the
> company to distribute it, since the "official" Buildroot tree is
> enough to reproduce the GPL program(s) on the target.
> 
> This applies, of course, only if the company decides to publish
> Buildroot to catter for section 3 of the GPL of those programs.
> 
> If the company decides for another route, then this point is moot
> anyway, since there is no distribution at all.

Sure.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Is GPLv2 the right license for Buildroot?

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Mon, 16 Sep 2013 23:12:55 +0200, Peter Korsgaard wrote:

> We have already tried to clarify it in the user manual:
> 
> http://buildroot.net/downloads/manual/manual.html#_complying_with_the_buildroot_license
> 
> E.G. Buildroot should be handled just like how you handle the Linux
> kernel and Busybox.

I think it's not really as clear as some other explanations that Yann
gave. Especially that:

 *) Distributing Buildroot as part of the "scripts needed for
 compilation" requirement is not due to the fact that Buildroot is
 under the GPL, but because most likely the embedded Linux system
 produced does contain GPL packages.

 *) That an alternate possibilities to distributing Buildroot is to
 provide a large shell script that runs the same commands that
 Buildroot is doing, but that this is probably much more complicated
 than providing the Buildroot source code.

 *) That package recipes for non-GPL packages can be removed from the
 Buildroot tree before it is distributed (especially the package
 recipes for proprietary applications)

 *) That the .config should be included as well, but that references to
 non-GPL applications can be removed from the .config if necessary.

>  Thomas>  (2) Change the Buildroot license to a non-copyleft license.
>  Thomas> Of course, that requires contacting a lot of people, but
>  Thomas> maybe not so much: over the last 3-4 years, the vast
>  Thomas> majority of the Buildroot code base has been rewritten, and
>  Thomas> many of the people having worked on that are still around
>  Thomas> today.
> 
> It's imho still too many people to be realistic:
> 
> git shortlog -s --since='3 years' | wc -l
> 230

It's not so many people in fact :)


> 
> 
>  Thomas> What other build systems are doing:
> 
>  Thomas>  * Yocto/OpenEmbeded: bitbake is under GPLv2, the rest
>  Thomas> (package recipes) is under MIT.
> 
>  Thomas>  * PTXdist is under GPLv2, but has a small license
>  Thomas> clarification "Note: PTXdist is a build system which
>  Thomas> generates a distribution for embedded systems. This license
>  Thomas> does only cover the build system itself and doesn't affect
>  Thomas> the license of managed software in any way, even if it might
>  Thomas> be distributed together with the build system." I believe it
>  Thomas> doesn't really clarify completely how the GPL applies to a
>  Thomas> build system.
> 
>  Thomas>  * OpenBricks is under GPLv2.
> 
>  Thomas>  * OpenWRT is under GPLv2, since it is originally a fork of
>  Thomas> Buildroot.
> 
>  Thomas>  * LTIB is under GPLv2.
> 
> So they are basically all in the same boat as us.

Yes, but I do believe that no company is actually complying to the GPL
requirements by providing the build system source code. So yes, they
are under GPLv2, now whether that makes sense or not is a different
question :-)

But ok, my question "Is GPLv2 the right license for Buildroot" was
mainly a rhetoric question, I am really not that much interested in
working on a license change process, but I continue to think that
additional clarifications on how the license requirements apply to
Buildroot would be nice.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Is GPLv2 the right license for Buildroot?

[Grant Edwards]

On 2013-09-17, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Yes, but I do believe that no company is actually complying to the
> GPL requirements by providing the build system source code.

We do.  I wrote a shell-script that removes proprietary files/packages
and bundles the resulting buildroot source tree which can then be used
to build all of the open-source stuff.

Nobody's ever _requested_ the sources, but they're available.  

I don't expect that anyone ever will request sources, since without
the proprietary software, the hardware is pretty much useless.  I
doubt any of our customers would be capable of building it even if
they did have the source tree in hand.

-- 
Grant Edwards               grant.b.edwards        Yow! If I had a Q-TIP, I
                                  at               could prevent th' collapse
                              gmail.com            of NEGOTIATIONS!!

[Buildroot] Is GPLv2 the right license for Buildroot?

[Jeremy Rosen]

I'll second that,

when we develop software for a customer building a product we educate them
on their duties WRT open-source software and what they should do to comply.
We also provide them with all the source code they need to comply.

(on a side note, some customers don't want to deal with that and just let
us push upstream all the changes to FOSS so all changes are upstream and
they don't need to provide their changes :P )

But don't overestimate that whole "provide the code" thing. There is 
very little code to provide, 99% of the code our customers use is unmodified
and their only duty is to point you to the original project's URL (that 
usually includes buildroot). The only real modification would be  if the
.config is considered derived work. But for a lot of our customers that would
be the only thing they would need to provide...

    Cordialement

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> On 2013-09-17, Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> wrote:
> 
> > Yes, but I do believe that no company is actually complying to the
> > GPL requirements by providing the build system source code.
> 
> We do.  I wrote a shell-script that removes proprietary
> files/packages
> and bundles the resulting buildroot source tree which can then be
> used
> to build all of the open-source stuff.
> 
> Nobody's ever _requested_ the sources, but they're available.
> 
> I don't expect that anyone ever will request sources, since without
> the proprietary software, the hardware is pretty much useless.  I
> doubt any of our customers would be capable of building it even if
> they did have the source tree in hand.
> 
> --
> Grant Edwards               grant.b.edwards        Yow! If I had a
> Q-TIP, I
>                                   at               could prevent th'
>                                   collapse
>                               gmail.com            of NEGOTIATIONS!!
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] Is GPLv2 the right license for Buildroot?

[Grant Edwards]

On 2013-09-17, Jeremy Rosen <jeremy.rosen@openwide.fr> wrote:

> But don't overestimate that whole "provide the code" thing. There is 
> very little code to provide, 99% of the code our customers use is unmodified
> and their only duty is to point you to the original project's URL (that 
> usually includes buildroot).

I don't think that's true.  If you distribute a binary, you must
provide or offer to provide the actual source code on machine-readable
media.  Legally, you can't just give somebody a URL to a third-party
site where the source code can be obtained.  In practice, that's often
enough to make most people happy, but it's not enough to satisfy the
GPL.

-- 
Grant Edwards               grant.b.edwards        Yow! Of course, you
                                  at               UNDERSTAND about the PLAIDS
                              gmail.com            in the SPIN CYCLE --

[Buildroot] Is GPLv2 the right license for Buildroot?

[Peter Korsgaard]

>>>>> "Grant" == Grant Edwards <grant.b.edwards@gmail.com> writes:

 Grant> On 2013-09-17, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
 >> Yes, but I do believe that no company is actually complying to the
 >> GPL requirements by providing the build system source code.

 Grant> We do.  I wrote a shell-script that removes proprietary files/packages
 Grant> and bundles the resulting buildroot source tree which can then be used
 Grant> to build all of the open-source stuff.

 Grant> Nobody's ever _requested_ the sources, but they're available.  

 Grant> I don't expect that anyone ever will request sources, since without
 Grant> the proprietary software, the hardware is pretty much useless.  I
 Grant> doubt any of our customers would be capable of building it even if
 Grant> they did have the source tree in hand.

Indeed. I've also archived U-Boot / Linux / Buildroot and ensured the
user manual contains the GPL source code offer for our products, but so
far nobody has ever requested it.

-- 
Bye, Peter Korsgaard

[Buildroot] Is GPLv2 the right license for Buildroot?

[Jason Rennie]

I may be missing something here and feel free to call me names if I am, 
also IANAL.

That being said, wouldn't building a system with buildroot be 
essentially the same case to using GCC to compile a program? The final 
libraries you link with (or packages you include in the case of 
buildroot) may be encumbered with all manner of licenses but the license 
of the buildtool doesn't make any difference to the license status of 
the final product.

If you were making changes to buildroot and redistributing those then 
the license would apply but not otherwise.

Jason

On 17/09/2013 4:01 AM, Thomas Petazzoni wrote:
> Dear ANDY KENNEDY,
>
> On Mon, 16 Sep 2013 17:45:00 +0000, ANDY KENNEDY wrote:
>
>>>    - on an embedded system, the probability that there is a GPL
>>> program is rather high (eg. busybox, the Linux kernel);
>> Not always.  There is still a high contingent in the States that use
>> VXWorks, and then a lesser population use a home-grown kernel without
>> every touching GPL code.
> I believe Yann was implicitly saying "embedded Linux system". Buildroot
> is of no-use for VxWorks based systems or other non-Linux operating
> systems.
>
>> I agree with Yann on this point.  I don't see a valid reason to change
>> the license of BuildRoot (even if you wanted to make it a less
>> restrictive license for the main package/*.mk files).  In my current
>> company, I have discussed in great detail exactly what must be done
>> in order to comply with the GPL.  The team I'm on has taken great
>> care to isolate the code into two sections:  Open Sourced code
>> (including MIT license stuff, etc) and IP code.  I have stressed the
>> importance of carefully considering how applications are made, what
>> dependencies we have on the applications, etc.  My company is
>> on-board with (when we get to the point in which we are shipping
>> Linux as the OS on a product) releasing all the non-company code.
> Sure. But in Buildroot, things are more complicated: within Buildroot
> *itself* there may be parts that you have to redistribute (i.e package
> recipes for GPL programs), and some other parts you may not be willing
> to redistribute (i.e package recipes for your own applications, or your
> root filesystem overlay). This makes it quite difficult for a company
> using Buildroot to easily separate what must be distributed from what
> shouldn't be distributed.
>
> Thomas

[Buildroot] Is GPLv2 the right license for Buildroot?

[Jason Rennie]

On 17/09/2013 5:50 AM, Grant Edwards wrote:
> On 2013-09-16, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
>> Sorry, but it is very clear to me, and I am no lawyer. And any decent
>> company will have a rather decent team of lawyers to help in these types
>> of cases.
> So no company with less that several hundred employees is "decent"?
>
I think decent means large in this context rather than morally upright.

[Buildroot] Is GPLv2 the right license for Buildroot?

[Peter Korsgaard]

>>>>> "Jason" == Jason Rennie <jason.rennie@rftechnology.com.au> writes:

 Jason> I may be missing something here and feel free to call me names if I
 Jason> am, also IANAL.

 Jason> That being said, wouldn't building a system with buildroot be
 Jason> essentially the same case to using GCC to compile a program? The final
 Jason> libraries you link with (or packages you include in the case of
 Jason> buildroot) may be encumbered with all manner of licenses but the
 Jason> license of the buildtool doesn't make any difference to the license
 Jason> status of the final product.

 Jason> If you were making changes to buildroot and redistributing those then
 Jason> the license would apply but not otherwise.

Well, yes and no - It depends on the interpretation the 'scripts used to
control compilation' part of section 3:

The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source code
means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to control
compilation and installation of the executable

Again, IANAL.

-- 
Bye, Peter Korsgaard

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Jason, All,

On 2013-09-18 11:50 +1000, Jason Rennie spake thusly:
> I may be missing something here and feel free to call me names if I am, also
> IANAL.
> 
> That being said, wouldn't building a system with buildroot be essentially
> the same case to using GCC to compile a program? The final libraries you
> link with (or packages you include in the case of buildroot) may be
> encumbered with all manner of licenses but the license of the buildtool
> doesn't make any difference to the license status of the final product.

Yes, that's basically true.

But consider this:

  - you use Buildroot to build a GPL package, which ends up on the
    target

  - that package is GPL, so the GPL applies to that package

  - section 3 of the GPL requires you provide the "scripts used to
    control compilation and installation of the executable."

  - So you have two options:
    - come up with the instructions by yourself (by looking at how
      Buildroot does its stuff, for example), but it's not trivial;
    - or provide your Buildroot tree (and .config) since this is the
      easiest. Let's assume that's the way you go.

  - so, since you are distributing Buildroot, you have to abide by its
    license. It happens that this license is the GPL, but it's just
    happenstance that it is the same license as your GPL package, above.

> If you were making changes to buildroot and redistributing those then the
> license would apply but not otherwise.

I believe we never argued otherwise. :-)

What was said was, to comply with the section 3 of the GPLv2 for programs
on the target, the easiest is to provide the Buildroot tree (and .config).

Note that doing so is not enough: you'd still have to provide the sources
of the (L)GPL programs that end up on the target.

Again, IANAL... ;-)

Note: I'm talking about section 3 of the GPLv2. GPLv3 has a similar
section, but I'm not digging it now; I'll let you look at it by
yourself! ;-p

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Is GPLv2 the right license for Buildroot?

[Jason Rennie]

On 19/09/2013 8:09 AM, Yann E. MORIN wrote:
>
> Yes, that's basically true.
>
> But consider this:
>    - you use Buildroot to build a GPL package, which ends up on the
>      target
>
>    - that package is GPL, so the GPL applies to that package
>
>    - section 3 of the GPL requires you provide the "scripts used to
>      control compilation and installation of the executable."
>
>    - So you have two options:
>      - come up with the instructions by yourself (by looking at how
>        Buildroot does its stuff, for example), but it's not trivial;
>      - or provide your Buildroot tree (and .config) since this is the
>        easiest. Let's assume that's the way you go.
>
>    - so, since you are distributing Buildroot, you have to abide by its
>      license. It happens that this license is the GPL, but it's just
>      happenstance that it is the same license as your GPL package, above.
>

Wouldn't the intent of the license suggest that this was talking about 
the makefiles/configure scripts/whatever that you need to actually build 
the executable? Rather than some metabuild script like buildroot? I'm 
guessing the clause is there to stop someone "complying" by giving you a 
bunch of .c and .h files but not providing you a meaningful way to get 
an executeable out the other end.

The understanding you are suggesting would seem to be like requiring 
that the automated build script that runs on a build server of an 
evening (which calls the make files, maybe runs some automated tests 
etc, basically an automated developer task) would need to be included to 
distribute the software.

As I said IANAL and so perhaps that would be required but that would 
seem like a bug in the license at that point and the problem is the 
terms of the license needing review.

Jason

[Buildroot] Is GPLv2 the right license for Buildroot?

[Yann E. MORIN]

Jason, All,

On 2013-09-19 10:25 +1000, Jason Rennie spake thusly:
> On 19/09/2013 8:09 AM, Yann E. MORIN wrote:
> >But consider this:
> >   - you use Buildroot to build a GPL package, which ends up on the
> >     target
> >
> >   - that package is GPL, so the GPL applies to that package
> >
> >   - section 3 of the GPL requires you provide the "scripts used to
> >     control compilation and installation of the executable."
> >
> >   - So you have two options:
> >     - come up with the instructions by yourself (by looking at how
> >       Buildroot does its stuff, for example), but it's not trivial;
> >     - or provide your Buildroot tree (and .config) since this is the
> >       easiest. Let's assume that's the way you go.
> >
> >   - so, since you are distributing Buildroot, you have to abide by its
> >     license. It happens that this license is the GPL, but it's just
> >     happenstance that it is the same license as your GPL package, above.
> >
> 
> Wouldn't the intent of the license suggest that this was talking about the
> makefiles/configure scripts/whatever that you need to actually build the
> executable? Rather than some metabuild script like buildroot? I'm guessing
> the clause is there to stop someone "complying" by giving you a bunch of .c
> and .h files but not providing you a meaningful way to get an executeable
> out the other end.
> 
> The understanding you are suggesting would seem to be like requiring that
> the automated build script that runs on a build server of an evening (which
> calls the make files, maybe runs some automated tests etc, basically an
> automated developer task) would need to be included to distribute the
> software.

No, I never said that [*].
I said doing so would make complying _much easier_.

And since we're speaking on the Buildroot ML, I'm assuming the build
system in question is Buildroot, not some weirdo internal and
proprietary build system.

> As I said IANAL and so perhaps that would be required but that would seem
> like a bug in the license at that point and the problem is the terms of the
> license needing review.

Well, what is important is the _spirit_ of the license, as much as the
_letter_ of the license is.

Now, I believe we've gone as far as our uninformed opinions would get
us, and we won't be able to cover all cases, so we can safely close this
subject for now. ;-)

I'll try to gather all of this thread in a patch against the manual later
next week, and will submit that back here for further discussion. Thanks
all of you for your very valuable inputs! :-)


[*] In fact, I even never said that you had to, or did not have to
    provide it. I just said providing it makes complying very easy.
    YMMV, as they use to say! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'