[Buildroot] [RFC 0/3] Adding software stacks

[Buildroot] [RFC 0/3] Adding software stacks

[Angelo Compagnucci]

Software stacks are a way to define a group of options that should be used together and they are orthogonal to configs: a software stack can indeed be shared among several configs.

Having a stack system like the one envisioned in this patch series make possible to do something like this:

make imx6-sabresd_defconfig; make mesa3d-etnaviv_stack qt5-fb_stack qt5-demos_stack; make

In this way we can declutter redundant configs (ex: imx6-sabresd_qt5_defconfig) and decouple software packages side releated symbols from hardware related defconfigs.


Angelo Compagnucci (3):
  support/kconfig/merge_config.sh: merge also buildroot config files
  Makefile: add handling of software stacks
  stacks: add a bunch of stacks

 Makefile                        | 40 ++++++++++++++++++++++++++++++++++++++--
 stacks/lamp_stack               | 10 ++++++++++
 stacks/mesa3d-etnaviv_stack     |  7 +++++++
 stacks/qt5-demos_stack          |  7 +++++++
 stacks/qt5-fb_stack             | 11 +++++++++++
 support/kconfig/merge_config.sh | 16 +++++++++++++++-
 6 files changed, 88 insertions(+), 3 deletions(-)
 create mode 100644 stacks/lamp_stack
 create mode 100644 stacks/mesa3d-etnaviv_stack
 create mode 100644 stacks/qt5-demos_stack
 create mode 100644 stacks/qt5-fb_stack

-- 
2.7.4

[Buildroot] [RFC 1/3] support/kconfig/merge_config.sh: merge also buildroot config files

[Angelo Compagnucci]

This patch adds a way to merge buildroot config file programmatically.
It adds an option (-b, buildroot mode) to manage buildroot config files.
The buildroot mode changes the way the script looks for configurations
entries using the BR2_ prefix and modify the call to the make command
to be buildroot friendly.

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 support/kconfig/merge_config.sh | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/support/kconfig/merge_config.sh b/support/kconfig/merge_config.sh
index 8a1708b..cb6e439 100755
--- a/support/kconfig/merge_config.sh
+++ b/support/kconfig/merge_config.sh
@@ -29,6 +29,7 @@ trap clean_up HUP INT TERM
 usage() {
 	echo "Usage: $0 [OPTIONS] [CONFIG [...]]"
 	echo "  -h    display this help text"
+	echo "  -b    buildroot mode (searches for BR2_ and uses a custom make command)"
 	echo "  -m    only merge the fragments, do not execute the make command"
 	echo "  -n    use allnoconfig instead of alldefconfig"
 	echo "  -r    list redundant entries when merging fragments"
@@ -39,6 +40,8 @@ MAKE=true
 ALLTARGET=alldefconfig
 WARNREDUN=false
 OUTPUT=.
+BUILDROOT_MODE=FALSE
+CONFIG_PREFIX=CONFIG_
 
 while true; do
 	case $1 in
@@ -71,6 +74,12 @@ while true; do
 		shift 2
 		continue
 		;;
+	"-b")
+		BUILDROOT_MODE=true
+		CONFIG_PREFIX=BR2_
+		shift
+		continue
+		;;
 	*)
 		break
 		;;
@@ -81,7 +90,7 @@ INITFILE=$1
 shift;
 
 MERGE_LIST=$*
-SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
+SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(${CONFIG_PREFIX}[a-zA-Z0-9_]*\)[= ].*/\2/p"
 TMP_FILE=$(mktemp -t .tmp.config.XXXXXXXXXX)
 
 echo "Using $INITFILE as base"
@@ -131,7 +140,12 @@ fi
 # Use the merged file as the starting point for:
 # alldefconfig: Fills in any missing symbols with Kconfig default
 # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
+if [ "$BUILDROOT_MODE" = "false" ]; then
 make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
+else
+cp $TMP_FILE $OUTPUT/.config
+make $OUTPUT_ARG olddefconfig
+fi
 
 
 # Check all specified config values took (might have missed-dependency issues)
-- 
2.7.4

[Buildroot] [RFC 2/3] Makefile: add handling of software stacks

[Angelo Compagnucci]

Software stacks are a new way to express a bundle of configuration
options that should be used togheter.
A software stack is orthogonal to a classic buildroot config: a software
stack indeed could be shared beetween multiple configs.

A software stack could be used with standard buildroot tools, like:

make qt5-fb_stack

The fragment called qt5-fb_stack is merged inside current .config and
everithing is checked to produce a valid configurationqt5-fb_stack is
merged inside current .config and everithing is checked to produce a
valid configuration.

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 Makefile | 40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 02f7cda..27283b8 100644
--- a/Makefile
+++ b/Makefile
@@ -128,7 +128,7 @@ export BR2_VERSION_FULL := $(BR2_VERSION)$(shell $(TOPDIR)/support/scripts/setlo
 
 # List of targets and target patterns for which .config doesn't need to be read in
 noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconfig \
-	defconfig %_defconfig allyesconfig allnoconfig alldefconfig silentoldconfig release \
+	defconfig %_defconfig %_stack allyesconfig allnoconfig alldefconfig silentoldconfig release \
 	randpackageconfig allyespackageconfig allnopackageconfig \
 	print-version olddefconfig distclean manual manual-%
 
@@ -146,7 +146,7 @@ nobuild_targets := source %-source source-check \
 	clean distclean help show-targets graph-depends \
 	%-graph-depends %-show-depends %-show-version \
 	graph-build graph-size list-defconfigs \
-	savedefconfig printvars
+	list-stacks savedefconfig printvars
 ifeq ($(MAKECMDGOALS),)
 BR_BUILDING = y
 else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
@@ -919,6 +919,12 @@ define percent_defconfig
 endef
 $(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_defconfig,$(d))$(sep)))
 
+define percent_stack
+%_stack: $(1)/stacks/%_stack
+	$(TOPDIR)/support/kconfig/merge_config.sh -b -O $(BASE_DIR) $(BR2_CONFIG) $(1)/stacks/$$@
+endef
+$(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_stack,$(d))$(sep)))
+
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
 	@$(COMMON_CONFIG_ENV) $< \
 		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
@@ -1039,6 +1045,7 @@ help:
 	@echo '  graph-depends          - generate graph of the dependency tree'
 	@echo '  graph-size             - generate stats of the filesystem size'
 	@echo '  list-defconfigs        - list all defconfigs (pre-configured minimal systems)'
+	@echo '  list-stacks            - list all stacks (pre-configured group of packages)'
 	@echo
 	@echo 'Miscellaneous:'
 	@echo '  source                 - download all sources needed for offline-build'
@@ -1075,6 +1082,26 @@ define list-defconfigs
 	$${first} || printf "\n"
 endef
 
+# List the stacks files
+# $(1): base directory
+# $(2): br2-external name, empty for bundled
+define list-stacks
+	@first=true; \
+	for stack in $(1)/stacks/*_stack; do \
+		[ -f "$${stack}" ] || continue; \
+		if $${first}; then \
+			if [ "$(2)" ]; then \
+				printf 'External stacks in "$(call qstrip,$(2))":\n'; \
+			else \
+				printf "Built-in stacks:\n"; \
+			fi; \
+			first=false; \
+		fi; \
+		printf "  %-35s - %s\n" ""$${stack##*/}"" "$$(head -n1 $${stack} | sed 's/^##*//' )"; \
+	done; \
+	$${first} || printf "\n"
+endef
+
 # We iterate over BR2_EXTERNAL_NAMES rather than BR2_EXTERNAL_DIRS,
 # because we want to display the name of the br2-external tree.
 .PHONY: list-defconfigs
@@ -1084,6 +1111,15 @@ list-defconfigs:
 		$(call list-defconfigs,$(BR2_EXTERNAL_$(name)_PATH),\
 			$(BR2_EXTERNAL_$(name)_DESC))$(sep))
 
+# We iterate over BR2_EXTERNAL_NAMES rather than BR2_EXTERNAL_DIRS,
+# because we want to display the name of the br2-external tree.
+.PHONY: list-stacks
+list-stacks:
+	$(call list-stacks,$(TOPDIR))
+	$(foreach name,$(BR2_EXTERNAL_NAMES),\
+		$(call list-stacks,$(BR2_EXTERNAL_$(name)_PATH),\
+			$(BR2_EXTERNAL_$(name)_DESC))$(sep))
+
 release: OUT = buildroot-$(BR2_VERSION)
 
 # Create release tarballs. We need to fiddle a bit to add the generated
-- 
2.7.4

[Buildroot] [RFC 3/3] stacks: add a bunch of stacks

[Angelo Compagnucci]

This patch adds a bunch of stacks to thinkering with.

Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
---
 stacks/lamp_stack           | 10 ++++++++++
 stacks/mesa3d-etnaviv_stack |  7 +++++++
 stacks/qt5-demos_stack      |  7 +++++++
 stacks/qt5-fb_stack         | 11 +++++++++++
 4 files changed, 35 insertions(+)
 create mode 100644 stacks/lamp_stack
 create mode 100644 stacks/mesa3d-etnaviv_stack
 create mode 100644 stacks/qt5-demos_stack
 create mode 100644 stacks/qt5-fb_stack

diff --git a/stacks/lamp_stack b/stacks/lamp_stack
new file mode 100644
index 0000000..bc9b0f1
--- /dev/null
+++ b/stacks/lamp_stack
@@ -0,0 +1,10 @@
+# Linux Apache Mysql Php (LAMP stack)
+
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+BR2_PACKAGE_PHP=y
+BR2_PACKAGE_PHP_EXT_DBA=y
+BR2_PACKAGE_PHP_EXT_PDO=y
+BR2_PACKAGE_PHP_EXT_PDO_MYSQL=y
+BR2_PACKAGE_MYSQL=y
+BR2_PACKAGE_ORACLE_MYSQL_SERVER=y
+BR2_PACKAGE_APACHE=y
diff --git a/stacks/mesa3d-etnaviv_stack b/stacks/mesa3d-etnaviv_stack
new file mode 100644
index 0000000..959967c
--- /dev/null
+++ b/stacks/mesa3d-etnaviv_stack
@@ -0,0 +1,7 @@
+# Mesa3D for boards based on Etnaviv driver
+BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+BR2_PACKAGE_MESA3D=y
+BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_ETNAVIV=y
+BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST=y
+BR2_PACKAGE_MESA3D_OPENGL_ES=y
diff --git a/stacks/qt5-demos_stack b/stacks/qt5-demos_stack
new file mode 100644
index 0000000..b00ef34
--- /dev/null
+++ b/stacks/qt5-demos_stack
@@ -0,0 +1,7 @@
+# QT5 demo packages
+BR2_PACKAGE_GLMARK2=y
+BR2_PACKAGE_MESA3D_DEMOS=y
+BR2_PACKAGE_KMSCUBE=y
+BR2_PACKAGE_QT5CINEX=y
+BR2_PACKAGE_QT5CINEX_HD=y
+BR2_PACKAGE_LIBV4L=y
diff --git a/stacks/qt5-fb_stack b/stacks/qt5-fb_stack
new file mode 100644
index 0000000..fcadde8
--- /dev/null
+++ b/stacks/qt5-fb_stack
@@ -0,0 +1,11 @@
+# QT5 stack based on the Linux famebuffer
+BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+BR2_PACKAGE_QT5=y
+BR2_PACKAGE_QT5BASE_GUI=y
+BR2_PACKAGE_QT5BASE_OPENGL=y
+BR2_PACKAGE_QT5BASE_OPENGL_LIB=y
+BR2_PACKAGE_QT5BASE_LINUXFB=y
+BR2_PACKAGE_QT5BASE_FONTCONFIG=y
+BR2_PACKAGE_QT5BASE_GIF=y
+BR2_PACKAGE_QT5BASE_JPEG=y
-- 
2.7.4

[Buildroot] [RFC 0/3] Adding software stacks

[Arnout Vandecappelle]

 Hi Angelo,

On 10-10-17 22:43, Angelo Compagnucci wrote:
> Software stacks are a way to define a group of options that should be used together and they are orthogonal to configs: a software stack can indeed be shared among several configs.

 Wow, you always come with the innovative ideas, don't you?

 We discussed such a feature before, but never came up with an elegant way to do
it, and that was better than just calling merge_config.sh.

> Having a stack system like the one envisioned in this patch series make possible to do something like this:
> 
> make imx6-sabresd_defconfig; make mesa3d-etnaviv_stack qt5-fb_stack qt5-demos_stack; make

 So yeah, is this really better than

./utils/merge_config.sh configs/imx6-sabresd_defconfig
stacks/mesa3d-etnaviv_stack stacks/qt5-fb_stack stacks/qt5-demos_stack; make

? Yes, it is significantly shorter, but you can at least use tab-completion.

 That said, I'm not absolutely opposed to the idea. But I'm not convinced either.

 Adding stacks, on the other hand, seems like a worthwhile endeavour.


> In this way we can declutter redundant configs (ex: imx6-sabresd_qt5_defconfig) and decouple software packages side releated symbols from hardware related defconfigs.

 Not the best example, because imx6-sabresd_qt5_defconfig contains quite a few
board-specific adaptions: a kconfig fragment and a rootfs overlay. And of course
increasing the ext2 rootfs size.

 A better example would be atmel_sama5d4_xplained_dev_defconfig which is purely
a collection of packages.


 Regards,
 Arnout


> 
> 
> Angelo Compagnucci (3):
>   support/kconfig/merge_config.sh: merge also buildroot config files
>   Makefile: add handling of software stacks
>   stacks: add a bunch of stacks
> 
>  Makefile                        | 40 ++++++++++++++++++++++++++++++++++++++--
>  stacks/lamp_stack               | 10 ++++++++++
>  stacks/mesa3d-etnaviv_stack     |  7 +++++++
>  stacks/qt5-demos_stack          |  7 +++++++
>  stacks/qt5-fb_stack             | 11 +++++++++++
>  support/kconfig/merge_config.sh | 16 +++++++++++++++-
>  6 files changed, 88 insertions(+), 3 deletions(-)
>  create mode 100644 stacks/lamp_stack
>  create mode 100644 stacks/mesa3d-etnaviv_stack
>  create mode 100644 stacks/qt5-demos_stack
>  create mode 100644 stacks/qt5-fb_stack
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 1/3] support/kconfig/merge_config.sh: merge also buildroot config files

[Arnout Vandecappelle]

On 10-10-17 22:43, Angelo Compagnucci wrote:
> This patch adds a way to merge buildroot config file programmatically.

 We already do that in test-pkg, without this patch. What are we doing wrong?

> It adds an option (-b, buildroot mode) to manage buildroot config files.

 I wouldn't bother with that option. It's only going to be used for Buildroot
config files (other packages using Kconfig will have a copy of the script
already). So we can just change it directly. And we already have a stack of
changes on the upstream kconfig.

> The buildroot mode changes the way the script looks for configurations
> entries using the BR2_ prefix and modify the call to the make command
> to be buildroot friendly.
> 
> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
[snip]
> @@ -81,7 +90,7 @@ INITFILE=$1
>  shift;
>  
>  MERGE_LIST=$*
> -SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
> +SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(${CONFIG_PREFIX}[a-zA-Z0-9_]*\)[= ].*/\2/p"

 So here you can just hardcode BR2_

>  TMP_FILE=$(mktemp -t .tmp.config.XXXXXXXXXX)
>  
>  echo "Using $INITFILE as base"
> @@ -131,7 +140,12 @@ fi
>  # Use the merged file as the starting point for:
>  # alldefconfig: Fills in any missing symbols with Kconfig default
>  # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
> +if [ "$BUILDROOT_MODE" = "false" ]; then
>  make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
> +else
> +cp $TMP_FILE $OUTPUT/.config
> +make $OUTPUT_ARG olddefconfig

 Why is this needed? alldefconfig works fine, no?

> +fi
>  
>  
>  # Check all specified config values took (might have missed-dependency issues)

 Since the kconfig stuff comes from upstream but is modified, we also maintain
the changes as a stack of patches in support/kconfig/patches. So you should
generate a new patch for this change and add it to the series file. I'm not sure
why we don't use a vendor branch and just merge, but that's the way we do it :-).


 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 2/3] Makefile: add handling of software stacks

[Arnout Vandecappelle]

On 10-10-17 22:43, Angelo Compagnucci wrote:
> Software stacks are a new way to express a bundle of configuration
> options that should be used togheter.

 together

> A software stack is orthogonal to a classic buildroot config: a software
> stack indeed could be shared beetween multiple configs.

 between

> 
> A software stack could be used with standard buildroot tools, like:
> 
> make qt5-fb_stack
> 
> The fragment called qt5-fb_stack is merged inside current .config and
> everithing is checked to produce a valid configurationqt5-fb_stack is

 everything                                       missing space

> merged inside current .config and everithing is checked to produce a

 everything

> valid configuration.
> 
> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> ---
>  Makefile | 40 ++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 38 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 02f7cda..27283b8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -128,7 +128,7 @@ export BR2_VERSION_FULL := $(BR2_VERSION)$(shell $(TOPDIR)/support/scripts/setlo
>  
>  # List of targets and target patterns for which .config doesn't need to be read in
>  noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconfig \
> -	defconfig %_defconfig allyesconfig allnoconfig alldefconfig silentoldconfig release \
> +	defconfig %_defconfig %_stack allyesconfig allnoconfig alldefconfig silentoldconfig release \
>  	randpackageconfig allyespackageconfig allnopackageconfig \
>  	print-version olddefconfig distclean manual manual-%
>  
> @@ -146,7 +146,7 @@ nobuild_targets := source %-source source-check \
>  	clean distclean help show-targets graph-depends \
>  	%-graph-depends %-show-depends %-show-version \
>  	graph-build graph-size list-defconfigs \
> -	savedefconfig printvars
> +	list-stacks savedefconfig printvars
>  ifeq ($(MAKECMDGOALS),)
>  BR_BUILDING = y
>  else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
> @@ -919,6 +919,12 @@ define percent_defconfig
>  endef
>  $(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_defconfig,$(d))$(sep)))
>  
> +define percent_stack
> +%_stack: $(1)/stacks/%_stack
> +	$(TOPDIR)/support/kconfig/merge_config.sh -b -O $(BASE_DIR) $(BR2_CONFIG) $(1)/stacks/$$@
> +endef
> +$(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_stack,$(d))$(sep)))

 This could be refactored more, by adding two arguments to percent_defconfig
(configs and _defconfig for defconfig, and stacks and _stack for stacks). The
macro should then obviously be renamed.

> +
>  savedefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
>  	@$(COMMON_CONFIG_ENV) $< \
>  		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
> @@ -1039,6 +1045,7 @@ help:
>  	@echo '  graph-depends          - generate graph of the dependency tree'
>  	@echo '  graph-size             - generate stats of the filesystem size'
>  	@echo '  list-defconfigs        - list all defconfigs (pre-configured minimal systems)'
> +	@echo '  list-stacks            - list all stacks (pre-configured group of packages)'
>  	@echo
>  	@echo 'Miscellaneous:'
>  	@echo '  source                 - download all sources needed for offline-build'
> @@ -1075,6 +1082,26 @@ define list-defconfigs
>  	$${first} || printf "\n"
>  endef
>  
> +# List the stacks files
> +# $(1): base directory
> +# $(2): br2-external name, empty for bundled
> +define list-stacks
> +	@first=true; \
> +	for stack in $(1)/stacks/*_stack; do \
> +		[ -f "$${stack}" ] || continue; \
> +		if $${first}; then \
> +			if [ "$(2)" ]; then \
> +				printf 'External stacks in "$(call qstrip,$(2))":\n'; \
> +			else \
> +				printf "Built-in stacks:\n"; \
> +			fi; \
> +			first=false; \
> +		fi; \
> +		printf "  %-35s - %s\n" ""$${stack##*/}"" "$$(head -n1 $${stack} | sed 's/^##*//' )"; \
> +	done; \
> +	$${first} || printf "\n"
> +endef

 Same here, can be refactored with list-defconfigs by adding extra arguments.

> +
>  # We iterate over BR2_EXTERNAL_NAMES rather than BR2_EXTERNAL_DIRS,
>  # because we want to display the name of the br2-external tree.
>  .PHONY: list-defconfigs
> @@ -1084,6 +1111,15 @@ list-defconfigs:
>  		$(call list-defconfigs,$(BR2_EXTERNAL_$(name)_PATH),\
>  			$(BR2_EXTERNAL_$(name)_DESC))$(sep))
>  
> +# We iterate over BR2_EXTERNAL_NAMES rather than BR2_EXTERNAL_DIRS,
> +# because we want to display the name of the br2-external tree.
> +.PHONY: list-stacks
> +list-stacks:
> +	$(call list-stacks,$(TOPDIR))
> +	$(foreach name,$(BR2_EXTERNAL_NAMES),\
> +		$(call list-stacks,$(BR2_EXTERNAL_$(name)_PATH),\
> +			$(BR2_EXTERNAL_$(name)_DESC))$(sep))

 This could also be factored into a new macro.

 Regards,
 Arnout

> +
>  release: OUT = buildroot-$(BR2_VERSION)
>  
>  # Create release tarballs. We need to fiddle a bit to add the generated
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 2/3] Makefile: add handling of software stacks

[Arnout Vandecappelle]

On 10-10-17 22:43, Angelo Compagnucci wrote:
> +define percent_stack
> +%_stack: $(1)/stacks/%_stack
> +	$(TOPDIR)/support/kconfig/merge_config.sh -b -O $(BASE_DIR) $(BR2_CONFIG) $(1)/stacks/$$@
> +endef
> +$(eval $(foreach d,$(call reverse,$(TOPDIR) $(BR2_EXTERNAL_DIRS)),$(call percent_stack,$(d))$(sep)))

 I forgot to mention: there's a future problem here. When we finally enable
top-level parallel build, the different stacks are going to overwrite each other
when you call it as "make stack1 stack2 stack3".

 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 3/3] stacks: add a bunch of stacks

[Arnout Vandecappelle]

On 10-10-17 22:43, Angelo Compagnucci wrote:
> This patch adds a bunch of stacks to thinkering with.

 tinkering

> 
> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> ---
>  stacks/lamp_stack           | 10 ++++++++++
>  stacks/mesa3d-etnaviv_stack |  7 +++++++
>  stacks/qt5-demos_stack      |  7 +++++++
>  stacks/qt5-fb_stack         | 11 +++++++++++
>  4 files changed, 35 insertions(+)
>  create mode 100644 stacks/lamp_stack
>  create mode 100644 stacks/mesa3d-etnaviv_stack
>  create mode 100644 stacks/qt5-demos_stack
>  create mode 100644 stacks/qt5-fb_stack
> 
> diff --git a/stacks/lamp_stack b/stacks/lamp_stack
> new file mode 100644
> index 0000000..bc9b0f1
> --- /dev/null
> +++ b/stacks/lamp_stack
> @@ -0,0 +1,10 @@
> +# Linux Apache Mysql Php (LAMP stack)
> +
> +BR2_TOOLCHAIN_BUILDROOT_CXX=y

 Ah, yes, these things... When you start combining defconfigs and stacks, it is
possible that there are conflicting options. So it would be nice to be able to
check that all the options we merge in are really retained. In fact, we already
do that, in test-pkg. So perhaps the best approach is to factor that into a new
script, e.g. utils/stack-config, and tell the user to run that script to combine
the stacks. Then call that script from test-pkg instead of repeating the same code.

 There's another limitation of this thing: if we want to combine it with an
external toolchain, this will no longer make sure that the external toolchain
has C++ enabled. We would basically want to include
BR2_INSTALL_LIBSTDCPP=y
in the stack config, so that the stack-config script will error out if the
external toolchain doesn't have C++. But then, I'm not sure how to keep the
BR2_TOOLCHAIN_BUILDROOT_CXX=y in the stack config because it will always be
thrown away for an external toolchain. Any ideas?


> +BR2_PACKAGE_PHP=y
> +BR2_PACKAGE_PHP_EXT_DBA=y
> +BR2_PACKAGE_PHP_EXT_PDO=y
> +BR2_PACKAGE_PHP_EXT_PDO_MYSQL=y

BR2_PACKAGE_PHP_SAPI_APACHE seems obvious... CGI is the default put probably not
really want you want if you have apache.

 And without BR2_PACKAGE_PHP_EXT_SESSION and BR2_PACKAGE_PHP_EXT_JSON you also
probably won't get very far.

> +BR2_PACKAGE_MYSQL=y
> +BR2_PACKAGE_ORACLE_MYSQL_SERVER=y
> +BR2_PACKAGE_APACHE=y
> diff --git a/stacks/mesa3d-etnaviv_stack b/stacks/mesa3d-etnaviv_stack
> new file mode 100644
> index 0000000..959967c
> --- /dev/null
> +++ b/stacks/mesa3d-etnaviv_stack
> @@ -0,0 +1,7 @@
> +# Mesa3D for boards based on Etnaviv driver

 It's better readable if you leave an empty line, like you did for the LAMP stack.

> +BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
> +BR2_TOOLCHAIN_BUILDROOT_CXX=y
> +BR2_PACKAGE_MESA3D=y
> +BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_ETNAVIV=y
> +BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST=y
> +BR2_PACKAGE_MESA3D_OPENGL_ES=y
> diff --git a/stacks/qt5-demos_stack b/stacks/qt5-demos_stack
> new file mode 100644
> index 0000000..b00ef34
> --- /dev/null
> +++ b/stacks/qt5-demos_stack
> @@ -0,0 +1,7 @@
> +# QT5 demo packages
> +BR2_PACKAGE_GLMARK2=y
> +BR2_PACKAGE_MESA3D_DEMOS=y
> +BR2_PACKAGE_KMSCUBE=y
> +BR2_PACKAGE_QT5CINEX=y
> +BR2_PACKAGE_QT5CINEX_HD=y

 Ha, that's another thing. This one needs qt5-fb_stack, so it would be nice if
it could #include it somehow.

 Regards,
 Arnout

> +BR2_PACKAGE_LIBV4L=y
> diff --git a/stacks/qt5-fb_stack b/stacks/qt5-fb_stack
> new file mode 100644
> index 0000000..fcadde8
> --- /dev/null
> +++ b/stacks/qt5-fb_stack
> @@ -0,0 +1,11 @@
> +# QT5 stack based on the Linux famebuffer
> +BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
> +BR2_TOOLCHAIN_BUILDROOT_CXX=y
> +BR2_PACKAGE_QT5=y
> +BR2_PACKAGE_QT5BASE_GUI=y
> +BR2_PACKAGE_QT5BASE_OPENGL=y
> +BR2_PACKAGE_QT5BASE_OPENGL_LIB=y
> +BR2_PACKAGE_QT5BASE_LINUXFB=y
> +BR2_PACKAGE_QT5BASE_FONTCONFIG=y
> +BR2_PACKAGE_QT5BASE_GIF=y
> +BR2_PACKAGE_QT5BASE_JPEG=y
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 0/3] Adding software stacks

[Angelo Compagnucci]

Dear Arnout Vandecappelle

2017-10-11 23:38 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>  Hi Angelo,
>
> On 10-10-17 22:43, Angelo Compagnucci wrote:
>> Software stacks are a way to define a group of options that should be used together and they are orthogonal to configs: a software stack can indeed be shared among several configs.
>
>  Wow, you always come with the innovative ideas, don't you?

Could you point me where I said it's innovative?

>  We discussed such a feature before, but never came up with an elegant way to do
> it, and that was better than just calling merge_config.sh.

I'm only proposing my POV.

>> Having a stack system like the one envisioned in this patch series make possible to do something like this:
>>
>> make imx6-sabresd_defconfig; make mesa3d-etnaviv_stack qt5-fb_stack qt5-demos_stack; make
>
>  So yeah, is this really better than
>
> ./utils/merge_config.sh configs/imx6-sabresd_defconfig
> stacks/mesa3d-etnaviv_stack stacks/qt5-fb_stack stacks/qt5-demos_stack; make
>
> ? Yes, it is significantly shorter, but you can at least use tab-completion.

Well, the difference is that on the mailing list or the documentation
you can say "there's a defined way in buildroot to do this thing"
instead of "you should merge your configurations manually". Explicit
is better than implicit!

>  That said, I'm not absolutely opposed to the idea. But I'm not convinced either.

It's not a problem for me to have a makefile target or not, I really
want to know if it's an acceptance to the idea and if I should spend
some time to refine it. If it's a no go, please tell me!

>  Adding stacks, on the other hand, seems like a worthwhile endeavour.
>
>
>> In this way we can declutter redundant configs (ex: imx6-sabresd_qt5_defconfig) and decouple software packages side related symbols from hardware related defconfigs.
>
>  Not the best example, because imx6-sabresd_qt5_defconfig contains quite a few
> board-specific adaptions: a kconfig fragment and a rootfs overlay. And of course
> increasing the ext2 rootfs size.
>
>  A better example would be atmel_sama5d4_xplained_dev_defconfig which is purely
> a collection of packages.

It was an example. Obviously for the sama5d4, we can have a stack for
each board variant to be merged with a base one.

>
>
>  Regards,
>  Arnout
>
>
>>
>>
>> Angelo Compagnucci (3):
>>   support/kconfig/merge_config.sh: merge also buildroot config files
>>   Makefile: add handling of software stacks
>>   stacks: add a bunch of stacks
>>
>>  Makefile                        | 40 ++++++++++++++++++++++++++++++++++++++--
>>  stacks/lamp_stack               | 10 ++++++++++
>>  stacks/mesa3d-etnaviv_stack     |  7 +++++++
>>  stacks/qt5-demos_stack          |  7 +++++++
>>  stacks/qt5-fb_stack             | 11 +++++++++++
>>  support/kconfig/merge_config.sh | 16 +++++++++++++++-
>>  6 files changed, 88 insertions(+), 3 deletions(-)
>>  create mode 100644 stacks/lamp_stack
>>  create mode 100644 stacks/mesa3d-etnaviv_stack
>>  create mode 100644 stacks/qt5-demos_stack
>>  create mode 100644 stacks/qt5-fb_stack
>>
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] [RFC 1/3] support/kconfig/merge_config.sh: merge also buildroot config files

[Angelo Compagnucci]

Dear Arnout Vandecappelle,

2017-10-11 23:43 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>
>
> On 10-10-17 22:43, Angelo Compagnucci wrote:
>> This patch adds a way to merge buildroot config file programmatically.
>
>  We already do that in test-pkg, without this patch. What are we doing wrong?

The actual support/kconfig/merge_config.sh looks for CONFIG_ symbols
via a regexp to print it's output and to check for inconsistencies.
Without the correct regexp this is impossible. Probably the right
commit message should be: "support/kconfig/merge_config.sh: check also
buildroot config files"

>> It adds an option (-b, buildroot mode) to manage buildroot config files.
>
>  I wouldn't bother with that option. It's only going to be used for Buildroot
> config files (other packages using Kconfig will have a copy of the script
> already). So we can just change it directly. And we already have a stack of
> changes on the upstream kconfig.
>
>> The buildroot mode changes the way the script looks for configurations
>> entries using the BR2_ prefix and modify the call to the make command
>> to be buildroot friendly.
>>
>> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
> [snip]
>> @@ -81,7 +90,7 @@ INITFILE=$1
>>  shift;
>>
>>  MERGE_LIST=$*
>> -SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(CONFIG_[a-zA-Z0-9_]*\)[= ].*/\2/p"
>> +SED_CONFIG_EXP="s/^\(# \)\{0,1\}\(${CONFIG_PREFIX}[a-zA-Z0-9_]*\)[= ].*/\2/p"
>
>  So here you can just hardcode BR2_

Ok

>
>>  TMP_FILE=$(mktemp -t .tmp.config.XXXXXXXXXX)
>>
>>  echo "Using $INITFILE as base"
>> @@ -131,7 +140,12 @@ fi
>>  # Use the merged file as the starting point for:
>>  # alldefconfig: Fills in any missing symbols with Kconfig default
>>  # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
>> +if [ "$BUILDROOT_MODE" = "false" ]; then
>>  make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
>> +else
>> +cp $TMP_FILE $OUTPUT/.config
>> +make $OUTPUT_ARG olddefconfig
>
>  Why is this needed? alldefconfig works fine, no?

Yes, indeed, this part could be removed.

>
>> +fi
>>
>>
>>  # Check all specified config values took (might have missed-dependency issues)
>
>  Since the kconfig stuff comes from upstream but is modified, we also maintain
> the changes as a stack of patches in support/kconfig/patches. So you should
> generate a new patch for this change and add it to the series file. I'm not sure
> why we don't use a vendor branch and just merge, but that's the way we do it :-).

Ok.

>
>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] [RFC 1/3] support/kconfig/merge_config.sh: merge also buildroot config files

[Arnout Vandecappelle]

On 12-10-17 08:42, Angelo Compagnucci wrote:
> Dear Arnout Vandecappelle,
> 
> 2017-10-11 23:43 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>>
>>
>> On 10-10-17 22:43, Angelo Compagnucci wrote:
>>> This patch adds a way to merge buildroot config file programmatically.
>>
>>  We already do that in test-pkg, without this patch. What are we doing wrong?
> 
> The actual support/kconfig/merge_config.sh looks for CONFIG_ symbols
> via a regexp to print it's output and to check for inconsistencies.
> Without the correct regexp this is impossible. Probably the right
> commit message should be: "support/kconfig/merge_config.sh: check also
> buildroot config files"

 Ah, and in test-pkg we don't need it because we check for that explicitly after
running merge_config.sh.

 But if you start from a full .config (which you do in the example of 'make
foo_defconfig; make bar_stack), isn't it going to give a warning for each and
every symbol defined in the stack?

[snip]
>>> @@ -131,7 +140,12 @@ fi
>>>  # Use the merged file as the starting point for:
>>>  # alldefconfig: Fills in any missing symbols with Kconfig default
>>>  # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
>>> +if [ "$BUILDROOT_MODE" = "false" ]; then
>>>  make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
>>> +else
>>> +cp $TMP_FILE $OUTPUT/.config
>>> +make $OUTPUT_ARG olddefconfig
>>
>>  Why is this needed? alldefconfig works fine, no?
> 
> Yes, indeed, this part could be removed.

 We only added alldefconfig a few months ago, perhaps your initial patch still
predates that.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 0/3] Adding software stacks

[Arnout Vandecappelle]

On 12-10-17 08:13, Angelo Compagnucci wrote:
> Dear Arnout Vandecappelle
> 
> 2017-10-11 23:38 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>>  Hi Angelo,
>>
>> On 10-10-17 22:43, Angelo Compagnucci wrote:
>>> Software stacks are a way to define a group of options that should be used together and they are orthogonal to configs: a software stack can indeed be shared among several configs.
>>
>>  Wow, you always come with the innovative ideas, don't you?
> 
> Could you point me where I said it's innovative?

 You didn't say it, but I thought it was innovative. Sorry, I probably didn't
express myself clearly. When I saw your patch series the first time, I
immediately thought "hey, this is nice" and started composing this answer (but
only this sentence). But then I realized we had discussed something like this
before both on the list and in face-to-face meetings, so I left it to have a
sleep on it before continuing. I realize now that the rest of my mail can come
over pretty critical. With my

 So to clarify (hopefully :-): I really like the idea of adding software stacks
(good name BTW, because I was thinking of "packages of packages" which sounds
awful). I also like the idea of making it very easy for people to add those
stacks together. But only if there is a real added value compared to calling
merge_config.sh directly. Some of my comments in patch 2/3 are meant to go in
that direction.

 Also, I only give my own opinion. This may or may not represent the position of
most of the community.

>>  We discussed such a feature before, but never came up with an elegant way to do
>> it, and that was better than just calling merge_config.sh.
> 
> I'm only proposing my POV.
> 
>>> Having a stack system like the one envisioned in this patch series make possible to do something like this:
>>>
>>> make imx6-sabresd_defconfig; make mesa3d-etnaviv_stack qt5-fb_stack qt5-demos_stack; make
>>
>>  So yeah, is this really better than
>>
>> ./utils/merge_config.sh configs/imx6-sabresd_defconfig
>> stacks/mesa3d-etnaviv_stack stacks/qt5-fb_stack stacks/qt5-demos_stack; make
>>
>> ? Yes, it is significantly shorter, but you can at least use tab-completion.
> 
> Well, the difference is that on the mailing list or the documentation
> you can say "there's a defined way in buildroot to do this thing"
> instead of "you should merge your configurations manually". Explicit
> is better than implicit!

 Very true. But calling merge_config.sh is also a defined way, isn't it?


>>  That said, I'm not absolutely opposed to the idea. But I'm not convinced either.
> 
> It's not a problem for me to have a makefile target or not, I really
> want to know if it's an acceptance to the idea and if I should spend
> some time to refine it. If it's a no go, please tell me!

 I don't want to say you shouldn't spend time on it. Perhaps the best approach
is to start with the stacks themselves and document how to use them, and we can
add make rules or scripts later on.


>>  Adding stacks, on the other hand, seems like a worthwhile endeavour.
>>
>>
>>> In this way we can declutter redundant configs (ex: imx6-sabresd_qt5_defconfig) and decouple software packages side related symbols from hardware related defconfigs.
>>
>>  Not the best example, because imx6-sabresd_qt5_defconfig contains quite a few
>> board-specific adaptions: a kconfig fragment and a rootfs overlay. And of course
>> increasing the ext2 rootfs size.
>>
>>  A better example would be atmel_sama5d4_xplained_dev_defconfig which is purely
>> a collection of packages.
> 
> It was an example. Obviously for the sama5d4, we can have a stack for
> each board variant to be merged with a base one.

 My point is, the "dev" stack (a collection of development/debug tools +
enabling BR2_PTHREAD_DEBUG=y) is a more convincing example of why these stacks
are useful.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [RFC 0/3] Adding software stacks

[Angelo Compagnucci]

Dear Arnout Vandecappelle,

2017-10-13 8:31 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>
>
> On 12-10-17 08:13, Angelo Compagnucci wrote:
>> Dear Arnout Vandecappelle
>>
>> 2017-10-11 23:38 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>>>  Hi Angelo,
>>>
>>> On 10-10-17 22:43, Angelo Compagnucci wrote:
>>>> Software stacks are a way to define a group of options that should be used together and they are orthogonal to configs: a software stack can indeed be shared among several configs.
>>>
>>>  Wow, you always come with the innovative ideas, don't you?
>>
>> Could you point me where I said it's innovative?
>
>  You didn't say it, but I thought it was innovative. Sorry, I probably didn't
> express myself clearly. When I saw your patch series the first time, I
> immediately thought "hey, this is nice" and started composing this answer (but
> only this sentence). But then I realized we had discussed something like this
> before both on the list and in face-to-face meetings, so I left it to have a
> sleep on it before continuing. I realize now that the rest of my mail can come
> over pretty critical. With my

Sorry Arnout for having intended your sentence in a critical way, I
know my patch series could be controversial at first and I was
expecting negative reactions.
So thank you for your appreciation! So let's go!

>  So to clarify (hopefully :-): I really like the idea of adding software stacks
> (good name BTW, because I was thinking of "packages of packages" which sounds
> awful). I also like the idea of making it very easy for people to add those
> stacks together. But only if there is a real added value compared to calling
> merge_config.sh directly. Some of my comments in patch 2/3 are meant to go in
> that direction.

My POV is that the intended way to go should be codified in software
and not only in the manual.


>  Also, I only give my own opinion. This may or may not represent the position of
> most of the community.

Well, of course it's your own, but a really inspiring one!

>>>  We discussed such a feature before, but never came up with an elegant way to do
>>> it, and that was better than just calling merge_config.sh.
>>
>> I'm only proposing my POV.
>>
>>>> Having a stack system like the one envisioned in this patch series make possible to do something like this:
>>>>
>>>> make imx6-sabresd_defconfig; make mesa3d-etnaviv_stack qt5-fb_stack qt5-demos_stack; make
>>>
>>>  So yeah, is this really better than
>>>
>>> ./utils/merge_config.sh configs/imx6-sabresd_defconfig
>>> stacks/mesa3d-etnaviv_stack stacks/qt5-fb_stack stacks/qt5-demos_stack; make
>>>
>>> ? Yes, it is significantly shorter, but you can at least use tab-completion.
>>
>> Well, the difference is that on the mailing list or the documentation
>> you can say "there's a defined way in buildroot to do this thing"
>> instead of "you should merge your configurations manually". Explicit
>> is better than implicit!
>
>  Very true. But calling merge_config.sh is also a defined way, isn't it?

Not documented anywhere:

$ rgrep "merge_config" docs/manual/ || echo nothing found
nothing found

>>>  That said, I'm not absolutely opposed to the idea. But I'm not convinced either.
>>
>> It's not a problem for me to have a makefile target or not, I really
>> want to know if it's an acceptance to the idea and if I should spend
>> some time to refine it. If it's a no go, please tell me!
>
>  I don't want to say you shouldn't spend time on it. Perhaps the best approach
> is to start with the stacks themselves and document how to use them, and we can
> add make rules or scripts later on.

Ok, why not. But if the feature is interesting and can be discussed
more, why not starting right from the begging instead of patching
over?

There a feature I included that I really like: if you do a make
list-stacks the first line of the stack will be printed as a
description of the stack. I think this feature should be here right
from the start instead of having the user looking for help file by
file manually. I was thinking to refine it more, having printed the
first commented lines of a stack with something like:

$ make qt5-fb_stack --help
QT5 stack based on the Linux famebuffer

This stack provides a complete QT5 with OPENGL enabled environment.
Of course you should have included an opengl capable driver stack
like mesa3d-etnaviv_stack

>>>  Adding stacks, on the other hand, seems like a worthwhile endeavour.
>>>
>>>> In this way we can declutter redundant configs (ex: imx6-sabresd_qt5_defconfig) and decouple software packages side related symbols from hardware related defconfigs.
>>>
>>>  Not the best example, because imx6-sabresd_qt5_defconfig contains quite a few
>>> board-specific adaptions: a kconfig fragment and a rootfs overlay. And of course
>>> increasing the ext2 rootfs size.
>>>
>>>  A better example would be atmel_sama5d4_xplained_dev_defconfig which is purely
>>> a collection of packages.
>>
>> It was an example. Obviously for the sama5d4, we can have a stack for
>> each board variant to be merged with a base one.
>
>  My point is, the "dev" stack (a collection of development/debug tools +
> enabling BR2_PTHREAD_DEBUG=y) is a more convincing example of why these stacks
> are useful.

Well yes, a stack could be anything. In my POV stacks should be used
at first to declutter config files from packages configurations
before, then I would very see stacks for each sort of complex
configuration that could be tricky to do right like a lamp stack or a
QT5 one.

Stack could also be used to build products and distribute it outside
buildroot (ex on an internal git of a corporate environment). I
imagine a software products that runs on several boards: having a
stack ready made make easy to port it beetween boards without too much
fuss.

Obviously, merge_confg.sh could do the same thing, but hey, having
included such a feature in buildroot looks cool to me!

>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] [RFC 1/3] support/kconfig/merge_config.sh: merge also buildroot config files

[Angelo Compagnucci]

Dear Arnout Vandecappelle ,

2017-10-12 16:41 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>
>
> On 12-10-17 08:42, Angelo Compagnucci wrote:
>> Dear Arnout Vandecappelle,
>>
>> 2017-10-11 23:43 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>>>
>>>
>>> On 10-10-17 22:43, Angelo Compagnucci wrote:
>>>> This patch adds a way to merge buildroot config file programmatically.
>>>
>>>  We already do that in test-pkg, without this patch. What are we doing wrong?
>>
>> The actual support/kconfig/merge_config.sh looks for CONFIG_ symbols
>> via a regexp to print it's output and to check for inconsistencies.
>> Without the correct regexp this is impossible. Probably the right
>> commit message should be: "support/kconfig/merge_config.sh: check also
>> buildroot config files"
>
>  Ah, and in test-pkg we don't need it because we check for that explicitly after
> running merge_config.sh.
>
>  But if you start from a full .config (which you do in the example of 'make
> foo_defconfig; make bar_stack), isn't it going to give a warning for each and
> every symbol defined in the stack?

No, this is the output:

$ make qemu_arm_versatile_defconfig && make lamp_stack
umask 0022 && make -C /media/angelo/WD/data/BUILDROOT/buildroot
O=/media/angelo/WD/data/BUILDROOT/br_qemu_arm/.
qemu_arm_versatile_defconfig
  GEN     /media/angelo/WD/data/BUILDROOT/br_qemu_arm/Makefile
#
# configuration written to /media/angelo/WD/data/BUILDROOT/br_qemu_arm/.config
#
umask 0022 && make -C /media/angelo/WD/data/BUILDROOT/buildroot
O=/media/angelo/WD/data/BUILDROOT/br_qemu_arm/. lamp_stack
/media/angelo/WD/data/BUILDROOT/buildroot/support/kconfig/merge_config.sh
-b -O /media/angelo/WD/data/BUILDROOT/br_qemu_arm
/media/angelo/WD/data/BUILDROOT/br_qemu_arm/.config
/media/angelo/WD/data/BUILDROOT/buildroot/stacks/lamp_stack
Using /media/angelo/WD/data/BUILDROOT/br_qemu_arm/.config as base
Merging /media/angelo/WD/data/BUILDROOT/buildroot/stacks/lamp_stack
Value of BR2_TOOLCHAIN_BUILDROOT_CXX is redefined by fragment
/media/angelo/WD/data/BUILDROOT/buildroot/stacks/lamp_stack:
Previous value: # BR2_TOOLCHAIN_BUILDROOT_CXX is not set
New value: BR2_TOOLCHAIN_BUILDROOT_CXX=y

Value of BR2_PACKAGE_PHP is redefined by fragment
/media/angelo/WD/data/BUILDROOT/buildroot/stacks/lamp_stack:
Previous value: # BR2_PACKAGE_PHP is not set
New value: BR2_PACKAGE_PHP=y

Value of BR2_PACKAGE_APACHE is redefined by fragment
/media/angelo/WD/data/BUILDROOT/buildroot/stacks/lamp_stack:
Previous value: # BR2_PACKAGE_APACHE is not set
New value: BR2_PACKAGE_APACHE=y

  GEN     /media/angelo/WD/data/BUILDROOT/br_qemu_arm/Makefile
#
# configuration written to /media/angelo/WD/data/BUILDROOT/br_qemu_arm/.config
#
Value requested for BR2_PACKAGE_ZLIB not in final .config
Requested value:  # BR2_PACKAGE_ZLIB is not set
Actual value:     BR2_PACKAGE_ZLIB=y

Value requested for BR2_PACKAGE_EXPAT not in final .config
Requested value:  # BR2_PACKAGE_EXPAT is not set
Actual value:     BR2_PACKAGE_EXPAT=y

Value requested for BR2_PACKAGE_APR not in final .config
Requested value:  # BR2_PACKAGE_APR is not set
Actual value:     BR2_PACKAGE_APR=y

Value requested for BR2_PACKAGE_APR_UTIL not in final .config
Requested value:  # BR2_PACKAGE_APR_UTIL is not set
Actual value:     BR2_PACKAGE_APR_UTIL=y

Value requested for BR2_PACKAGE_NCURSES not in final .config
Requested value:  # BR2_PACKAGE_NCURSES is not set
Actual value:     BR2_PACKAGE_NCURSES=y

Value requested for BR2_PACKAGE_PCRE not in final .config
Requested value:  # BR2_PACKAGE_PCRE is not set
Actual value:     BR2_PACKAGE_PCRE=y

Value requested for BR2_PACKAGE_READLINE not in final .config
Requested value:  # BR2_PACKAGE_READLINE is not set
Actual value:     BR2_PACKAGE_READLINE=y


>
> [snip]
>>>> @@ -131,7 +140,12 @@ fi
>>>>  # Use the merged file as the starting point for:
>>>>  # alldefconfig: Fills in any missing symbols with Kconfig default
>>>>  # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
>>>> +if [ "$BUILDROOT_MODE" = "false" ]; then
>>>>  make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
>>>> +else
>>>> +cp $TMP_FILE $OUTPUT/.config
>>>> +make $OUTPUT_ARG olddefconfig
>>>
>>>  Why is this needed? alldefconfig works fine, no?
>>
>> Yes, indeed, this part could be removed.
>
>  We only added alldefconfig a few months ago, perhaps your initial patch still
> predates that.

Nope, only thought that olddefconfig could be better suited.

>
>  Regards,
>  Arnout
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] [RFC 3/3] stacks: add a bunch of stacks

[Angelo Compagnucci]

Dear Arnout Vandecappelle,

2017-10-12 0:09 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>
>
> On 10-10-17 22:43, Angelo Compagnucci wrote:
>> This patch adds a bunch of stacks to thinkering with.
>
>  tinkering
>
>>
>> Signed-off-by: Angelo Compagnucci <angelo.compagnucci@gmail.com>
>> ---
>>  stacks/lamp_stack           | 10 ++++++++++
>>  stacks/mesa3d-etnaviv_stack |  7 +++++++
>>  stacks/qt5-demos_stack      |  7 +++++++
>>  stacks/qt5-fb_stack         | 11 +++++++++++
>>  4 files changed, 35 insertions(+)
>>  create mode 100644 stacks/lamp_stack
>>  create mode 100644 stacks/mesa3d-etnaviv_stack
>>  create mode 100644 stacks/qt5-demos_stack
>>  create mode 100644 stacks/qt5-fb_stack
>>
>> diff --git a/stacks/lamp_stack b/stacks/lamp_stack
>> new file mode 100644
>> index 0000000..bc9b0f1
>> --- /dev/null
>> +++ b/stacks/lamp_stack
>> @@ -0,0 +1,10 @@
>> +# Linux Apache Mysql Php (LAMP stack)
>> +
>> +BR2_TOOLCHAIN_BUILDROOT_CXX=y
>
>  Ah, yes, these things... When you start combining defconfigs and stacks, it is
> possible that there are conflicting options. So it would be nice to be able to
> check that all the options we merge in are really retained. In fact, we already
> do that, in test-pkg. So perhaps the best approach is to factor that into a new
> script, e.g. utils/stack-config, and tell the user to run that script to combine
> the stacks. Then call that script from test-pkg instead of repeating the same code.

I don't think that combining stacks that have conflicting options
should be a problem if there an easy way for the user to understand if
a symbol is present or not into the final .config and the patch I did
to merge_config.sh goes this way.
I think the main driver here should be to give the user something
prepared and reusable, but not to give a bullet proof solution.
I'm using stacks for some weeks now and I'm now acquainted to check
the output of merge_config to see if some symbol I requested is
missing in the final .config.
Probably we could enforce this but if we enforce, we should also check
that all the permutation of defconfigs and stack are working. I think
that adding a proper disclaimer is fair enough.

>  There's another limitation of this thing: if we want to combine it with an
> external toolchain, this will no longer make sure that the external toolchain
> has C++ enabled. We would basically want to include
> BR2_INSTALL_LIBSTDCPP=y
> in the stack config, so that the stack-config script will error out if the
> external toolchain doesn't have C++. But then, I'm not sure how to keep the
> BR2_TOOLCHAIN_BUILDROOT_CXX=y in the stack config because it will always be
> thrown away for an external toolchain. Any ideas?

I don't think also this should be a main problem. We can add both and
the build system remove the unused one. At least I tried with a
modified qt5-fb_stack and everything works. Of course, this behavior
should be documented.


>> +BR2_PACKAGE_PHP=y
>> +BR2_PACKAGE_PHP_EXT_DBA=y
>> +BR2_PACKAGE_PHP_EXT_PDO=y
>> +BR2_PACKAGE_PHP_EXT_PDO_MYSQL=y
>
> BR2_PACKAGE_PHP_SAPI_APACHE seems obvious... CGI is the default put probably not
> really want you want if you have apache.
>
>  And without BR2_PACKAGE_PHP_EXT_SESSION and BR2_PACKAGE_PHP_EXT_JSON you also
> probably won't get very far.

Yes right, this stack was only an example thrown there to show how a
LAMP stack could be.


>> +BR2_PACKAGE_MYSQL=y
>> +BR2_PACKAGE_ORACLE_MYSQL_SERVER=y
>> +BR2_PACKAGE_APACHE=y
>> diff --git a/stacks/mesa3d-etnaviv_stack b/stacks/mesa3d-etnaviv_stack
>> new file mode 100644
>> index 0000000..959967c
>> --- /dev/null
>> +++ b/stacks/mesa3d-etnaviv_stack
>> @@ -0,0 +1,7 @@
>> +# Mesa3D for boards based on Etnaviv driver
>
>  It's better readable if you leave an empty line, like you did for the LAMP stack.
>
>> +BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
>> +BR2_TOOLCHAIN_BUILDROOT_CXX=y
>> +BR2_PACKAGE_MESA3D=y
>> +BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_ETNAVIV=y
>> +BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST=y
>> +BR2_PACKAGE_MESA3D_OPENGL_ES=y
>> diff --git a/stacks/qt5-demos_stack b/stacks/qt5-demos_stack
>> new file mode 100644
>> index 0000000..b00ef34
>> --- /dev/null
>> +++ b/stacks/qt5-demos_stack
>> @@ -0,0 +1,7 @@
>> +# QT5 demo packages
>> +BR2_PACKAGE_GLMARK2=y
>> +BR2_PACKAGE_MESA3D_DEMOS=y
>> +BR2_PACKAGE_KMSCUBE=y
>> +BR2_PACKAGE_QT5CINEX=y
>> +BR2_PACKAGE_QT5CINEX_HD=y
>
>  Ha, that's another thing. This one needs qt5-fb_stack, so it would be nice if
> it could #include it somehow.

Ah, the inclusion mechanism. I don't think we should enforce such an
inclusion mechanism. Users could mix and match as they want. Probably
for this stack is obvious you should have merged also a Qt5 stack but
for other less obvious dependencies we should probably include some
sort of instructions somewhere.
I think we can add something to the firs lines of comments to
explicate the stack requirementes, something like this:

# QT5 demo packages
#
# Needs: a stack with qt5 and opengl enabled (like qt5-fb_stack)
#

>
>  Regards,
>  Arnout
>
>> +BR2_PACKAGE_LIBV4L=y
>> diff --git a/stacks/qt5-fb_stack b/stacks/qt5-fb_stack
>> new file mode 100644
>> index 0000000..fcadde8
>> --- /dev/null
>> +++ b/stacks/qt5-fb_stack
>> @@ -0,0 +1,11 @@
>> +# QT5 stack based on the Linux famebuffer
>> +BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
>> +BR2_TOOLCHAIN_BUILDROOT_CXX=y
>> +BR2_PACKAGE_QT5=y
>> +BR2_PACKAGE_QT5BASE_GUI=y
>> +BR2_PACKAGE_QT5BASE_OPENGL=y
>> +BR2_PACKAGE_QT5BASE_OPENGL_LIB=y
>> +BR2_PACKAGE_QT5BASE_LINUXFB=y
>> +BR2_PACKAGE_QT5BASE_FONTCONFIG=y
>> +BR2_PACKAGE_QT5BASE_GIF=y
>> +BR2_PACKAGE_QT5BASE_JPEG=y
>>
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo