[Buildroot] [PATCH 0/5 v5] core/pkg-kconfig: handle generated kconfig defconfigs (branch yem/kconfig-gen-defconfig)

[Buildroot] [PATCH 0/5 v5] core/pkg-kconfig: handle generated kconfig defconfigs (branch yem/kconfig-gen-defconfig)

[Yann E. MORIN]

Hello All!

This series is an attempt to support defconfig that are generated with
make rules.

The Linux kernel now has at least one defconfig that is not an actual
file, but a Makefile rule:

    ea4d1a8: powerpc/configs: Replace pseries_le_defconfig with a
             Makefile target using merge_config

Which means that we can no longer just copy defconfig files.

The series was discussed on IRC a while ago, and it was decided I would
just adopt it, clean it and resubmit. Thanks go to Sam for the initial
submission. Thanks! :-)


Changes v4 -> v5:
  - correctly check that the user did provide a defconfig name  (Thomas)

Changes v3 -> v4:
  - do not append the _defconfig suffix in the infra, let the packages
    do that (as it was before)  (Thomas)

Changes v2 -> v3 (Yann):
  - make all core changes in the same patch
  - update the manual
  - misc cleanups
  - misc typoes and rephrasing


Regards,
Yann E. MORIN.


The following changes since commit 88511ccf4a6e0338cbfd8dc985b9ebcda5d0e6b2:

  ltrace: add patch to fix build on sparc (2015-12-22 21:52:11 +0100)

are available in the git repository at:

  git://git.busybox.net/~ymorin/git/buildroot yem/kconfig-gen-defconfig

for you to fetch changes up to 88112e80085a0b2a4f361473c41d08662f1baf04:

  boot/barebox: handle generated defconfigs (2015-12-22 22:14:10 +0100)

----------------------------------------------------------------
Sam bobroff (4):
      core/pkg-kconfig: handle generated defconfigs
      package/linux: handle generated defconfigs
      boot/at91bootstrap3: handle generated defconfigs
      boot/barebox: handle generated defconfigs

Yann E. MORIN (1):
      docs/manual: document the new kconfig-package variable

 boot/at91bootstrap3/at91bootstrap3.mk   | 12 +++++++-----
 boot/barebox/barebox.mk                 | 13 +++++++------
 docs/manual/adding-packages-kconfig.txt | 10 ++++++++++
 linux/linux.mk                          | 13 +++++++------
 package/pkg-kconfig.mk                  | 20 ++++++++++++++++----
 5 files changed, 47 insertions(+), 21 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5 v5] core/pkg-kconfig: handle generated defconfigs

[Yann E. MORIN]

From: Sam bobroff <sam.bobroff@au1.ibm.com>

As of Linux kernel commit ea4d1a8 "powerpc/configs: Replace
pseries_le_defconfig with a Makefile target using merge_config" some
kconfig defconfigs (one so far: "pseries_le_defconfig") can be
generated using "make <defconfig>" and they do not exist on disk as a
single defconfig file.

This causes buildroot's build to fail for those configs with:
'arch/powerpc/configs/pseries_le_defconfig' for 'linux' does not exist

To handle this case and keep the makefile steps as simple as possible,
introduce a new package variable, *_KCONFIG_DEFCONFIG, that can be
used to indicate that a defconfig rule is being used, rather than a
file.

This allows the rule that generates the .config file to use either the
provided file (by copying) or a generated defconfig (by running "make
<defconfig>") as its starting point. merge_config.sh can then be run the
same way in either case, using .config as both input and output.

Note that merge_config.sh is now modifying .config in-place but this
is safe because it uses a temporary copy while making changes.

This patch introduces the new variable but does not make use of it.
Use of the new variable will be introduced in separate patches.

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
[yann.morin.1998 at free.fr:
  - include all the other kconfig-package hunks into this one patch
  - do not transform the 'echo "..."; exit 1' into $(error ...) calls
  - use a make $(if)-block instead of a shell if-block to copy the file
    or run make (like is done to check the kconfig snippets)
  - misc typoes and rephrasing in the commit log
  - do not force the _defconfig suffix in the infra  (Thomas)
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

---
Changes v3 -> v4:
  - do not append the _defconfig suffix in the infra, expect the package
    to provide the rule as-is  (Thomas)
---
 package/pkg-kconfig.mk | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/package/pkg-kconfig.mk b/package/pkg-kconfig.mk
index 44c74a7..eaee572 100644
--- a/package/pkg-kconfig.mk
+++ b/package/pkg-kconfig.mk
@@ -63,8 +63,12 @@ $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES): | $(1)-patch
 # full .config first. We use 'make oldconfig' because this can be safely
 # done even when the package does not support defconfigs.
 $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
+	$$(if $$($(2)_KCONFIG_DEFCONFIG), \
+		$$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) \
+			$$($(2)_KCONFIG_OPTS) $$($(2)_KCONFIG_DEFCONFIG), \
+		cp $$($(2)_KCONFIG_FILE) $$(@))
 	support/kconfig/merge_config.sh -m -O $$(@D) \
-		$$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
+		$$(@) $$($(2)_KCONFIG_FRAGMENT_FILES)
 	$$(Q)yes "" | $$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) \
 		$$($(2)_KCONFIG_OPTS) oldconfig
 
@@ -89,10 +93,14 @@ $$($(2)_TARGET_CONFIGURE): $$($(2)_DIR)/.stamp_kconfig_fixup_done
 # already called above, so we can effectively use this variable.
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
-# FOO_KCONFIG_FILE is required
 ifeq ($$(BR_BUILDING),y)
-ifeq ($$($(2)_KCONFIG_FILE),)
-$$(error Internal error: no value specified for $(2)_KCONFIG_FILE)
+# Either FOO_KCONFIG_FILE or FOO_KCONFIG_DEFCONFIG is required...
+ifeq ($$(or $$($(2)_KCONFIG_FILE),$$($(2)_KCONFIG_DEFCONFIG)),)
+$$(error Internal error: no value specified for $(2)_KCONFIG_FILE or $(2)_KCONFIG_DEFCONFIG)
+endif
+# ... but not both:
+ifneq ($$(and $$($(2)_KCONFIG_FILE),$$($(2)_KCONFIG_DEFCONFIG)),)
+$$(error Internal error: $(2)_KCONFIG_FILE and $(2)_KCONFIG_DEFCONFIG are mutually exclusive but both are defined)
 endif
 endif
 
@@ -160,6 +168,8 @@ $(1)-savedefconfig: $(1)-check-configuration-done
 $(1)-update-config: $(1)-check-configuration-done
 	@$$(if $$($(2)_KCONFIG_FRAGMENT_FILES), \
 		echo "Unable to perform $(1)-update-config when fragment files are set"; exit 1)
+	@$$(if $$($(2)_KCONFIG_DEFCONFIG), \
+		echo "Unable to perform $(1)-update-config when using a defconfig rule"; exit 1)
 	cp -f $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 	touch --reference $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 
@@ -170,6 +180,8 @@ $(1)-update-config: $(1)-check-configuration-done
 $(1)-update-defconfig: $(1)-savedefconfig
 	@$$(if $$($(2)_KCONFIG_FRAGMENT_FILES), \
 		echo "Unable to perform $(1)-update-defconfig when fragment files are set"; exit 1)
+	@$$(if $$($(2)_KCONFIG_DEFCONFIG), \
+		echo "Unable to perform $(1)-update-defconfig when using a defconfig rule"; exit 1)
 	cp -f $$($(2)_DIR)/defconfig $$($(2)_KCONFIG_FILE)
 	touch --reference $$($(2)_DIR)/.config $$($(2)_KCONFIG_FILE)
 
-- 
1.9.1

[Buildroot] [PATCH 2/5 v5] docs/manual: document the new kconfig-package variable

[Yann E. MORIN]

The previous patch introduced the new FOO_KCONFIG_DEFCONFIG variable
to specify a defconfig rule rather than a (def)config file.

Add this to the manual.

Also document the pre-existing FOO_KCONFIG_FILE for which the
explanations were missing altogether.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
 docs/manual/adding-packages-kconfig.txt | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/docs/manual/adding-packages-kconfig.txt b/docs/manual/adding-packages-kconfig.txt
index 17f91e3..3290024 100644
--- a/docs/manual/adding-packages-kconfig.txt
+++ b/docs/manual/adding-packages-kconfig.txt
@@ -46,6 +46,16 @@ This snippet creates the following make targets:
 and ensures that the source configuration file is copied to the build
 directory at the right moment.
 
+There are two options to specify a configuration file to use, either
++FOO_KCONFIG_FILE+ (as in the example, above) or +FOO_KCONFIG_DEFCONFIG+.
+It is mandatory to provide either, but not both:
+
+* +FOO_KCONFIG_FILE+ specifies the path to a defconfig or full-config file
+  to be used to configure the package.
+
+* +FOO_KCONFIG_DEFCONFIG+ specifies the defconfig 'make' rule to call to
+  configure the package.
+
 In addition to these minimally required lines, several optional variables can
 be set to suit the needs of the package under consideration:
 
-- 
1.9.1

[Buildroot] [PATCH 3/5 v5] package/linux: handle generated defconfigs

[Yann E. MORIN]

From: Sam bobroff <sam.bobroff@au1.ibm.com>

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
[yann.morin.1998 at free.fr: move the kconfig-package hunk to the
 corresponding patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

---
Changes v3 -> v4:
  - append the _defconfig suffix  (Thomas)
---
 linux/linux.mk | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index 451e681..06a27eb 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -188,12 +188,10 @@ endef
 LINUX_POST_PATCH_HOOKS += LINUX_APPLY_LOCAL_PATCHES
 
 ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
-KERNEL_SOURCE_CONFIG = $(KERNEL_ARCH_PATH)/configs/$(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))_defconfig
+LINUX_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))_defconfig
 else ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG),y)
-KERNEL_SOURCE_CONFIG = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE))
+LINUX_KCONFIG_FILE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE))
 endif
-
-LINUX_KCONFIG_FILE = $(KERNEL_SOURCE_CONFIG)
 LINUX_KCONFIG_FRAGMENT_FILES = $(call qstrip,$(BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES))
 LINUX_KCONFIG_EDITORS = menuconfig xconfig gconfig nconfig
 LINUX_KCONFIG_OPTS = $(LINUX_MAKE_FLAGS)
@@ -404,13 +402,16 @@ LINUX_POST_INSTALL_TARGET_HOOKS += $(foreach tool,$(LINUX_TOOLS),\
 ifeq ($(BR_BUILDING),y)
 
 ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
-ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG)),)
+# We must use the user-supplied kconfig value, because
+# LINUX_KCONFIG_DEFCONFIG will at least contain the
+# trailing _defconfig
+ifeq ($(qstrip $(BR2_LINUX_KERNEL_DEFCONFIG)),)
 $(error No kernel defconfig name specified, check your BR2_LINUX_KERNEL_DEFCONFIG setting)
 endif
 endif
 
 ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG),y)
-ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE)),)
+ifeq ($(LINUX_KCONFIG_FILE),)
 $(error No kernel configuration file specified, check your BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE setting)
 endif
 endif
-- 
1.9.1

[Buildroot] [PATCH 4/5 v5] boot/at91bootstrap3: handle generated defconfigs

[Yann E. MORIN]

From: Sam bobroff <sam.bobroff@au1.ibm.com>

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
[yann.morin.1998 at free.fr: move the kconfig-package hunk to the
 corresponding patch]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

---
Changes v3 -> v4:
  - append the _defconfig suffix  (Thomas)
---
 boot/at91bootstrap3/at91bootstrap3.mk | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/boot/at91bootstrap3/at91bootstrap3.mk b/boot/at91bootstrap3/at91bootstrap3.mk
index fa67ea6..0f5d3bd 100644
--- a/boot/at91bootstrap3/at91bootstrap3.mk
+++ b/boot/at91bootstrap3/at91bootstrap3.mk
@@ -41,12 +41,11 @@ define AT91BOOTSTRAP3_INSTALL_IMAGES_CMDS
 endef
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
-AT91BOOTSTRAP3_SOURCE_CONFIG = $(AT91BOOTSTRAP3_DIR)/board/*/$(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG))_defconfig
+AT91BOOTSTRAP3_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG))_defconfig
 else ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_CUSTOM_CONFIG),y)
-AT91BOOTSTRAP3_SOURCE_CONFIG = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE))
+AT91BOOTSTRAP3_KCONFIG_FILE = $(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE))
 endif
 
-AT91BOOTSTRAP3_KCONFIG_FILE = $(AT91BOOTSTRAP3_SOURCE_CONFIG)
 AT91BOOTSTRAP3_KCONFIG_EDITORS = menuconfig xconfig gconfig
 AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
 
@@ -55,13 +54,16 @@ AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
 ifeq ($(BR_BUILDING),y)
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
-ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG)),)
+# We must use the user-supplied kconfig value, because
+# AT91BOOTSTRAP3_KCONFIG_DEFCONFIG will at least contain
+# the trailing _defconfig
+ifeq ($(AT91BOOTSTRAP3_KCONFIG_DEFCONFIG),)
 $(error No at91bootstrap3 defconfig name specified, check your BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG setting)
 endif
 endif
 
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_CUSTOM_CONFIG),y)
-ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE)),)
+ifeq ($(AT91BOOTSTRAP3_KCONFIG_FILE),)
 $(error No at91bootstrap3 configuration file specified, check your BR2_TARGET_AT91BOOTSTRAP3_CUSTOM_CONFIG_FILE setting)
 endif
 endif
-- 
1.9.1

[Buildroot] [PATCH 5/5 v5] boot/barebox: handle generated defconfigs

[Yann E. MORIN]

From: Sam bobroff <sam.bobroff@au1.ibm.com>

Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

---
Changes v3 -> v4;
  - append the _defconfig suffix  (Thomas)
---
 boot/barebox/barebox.mk | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/boot/barebox/barebox.mk b/boot/barebox/barebox.mk
index 00e1537..f36ca70 100644
--- a/boot/barebox/barebox.mk
+++ b/boot/barebox/barebox.mk
@@ -56,13 +56,11 @@ BAREBOX_MAKE_FLAGS = ARCH=$(BAREBOX_ARCH) CROSS_COMPILE="$(TARGET_CROSS)"
 BAREBOX_MAKE_ENV = $(TARGET_MAKE_ENV)
 
 ifeq ($(BR2_TARGET_BAREBOX_USE_DEFCONFIG),y)
-BAREBOX_SOURCE_CONFIG = $(BAREBOX_DIR)/arch/$(BAREBOX_ARCH)/configs/$(call qstrip,\
-	$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))_defconfig
+BAREBOX_KCONFIG_DEFCONFIG = $(call qstrip,$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))_defconfig
 else ifeq ($(BR2_TARGET_BAREBOX_USE_CUSTOM_CONFIG),y)
-BAREBOX_SOURCE_CONFIG = $(call qstrip,$(BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE))
+BAREBOX_KCONFIG_FILE = $(call qstrip,$(BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE))
 endif
 
-BAREBOX_KCONFIG_FILE = $(BAREBOX_SOURCE_CONFIG)
 BAREBOX_KCONFIG_FRAGMENT_FILES = $(call qstrip,$(BR2_TARGET_BAREBOX_CONFIG_FRAGMENT_FILES))
 BAREBOX_KCONFIG_EDITORS = menuconfig xconfig gconfig nconfig
 BAREBOX_KCONFIG_OPTS = $(BAREBOX_MAKE_FLAGS)
@@ -111,8 +109,11 @@ endif
 # Checks to give errors that the user can understand
 # Must be before we call to kconfig-package
 ifeq ($(BR2_TARGET_BAREBOX)$(BR_BUILDING),yy)
-ifeq ($(BAREBOX_SOURCE_CONFIG),)
-$(error No Barebox config file. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
+# We must use the user-supplied kconfig value, because
+# BAREBOX_KCONFIG_DEFCONFIG will at least contain the
+# trailing _defconfig
+ifeq ($(or $(BAREBOX_KCONFIG_FILE),$(BAREBOX_KCONFIG_DEFCONFIG)),)
+$(error No Barebox config. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
 endif
 endif
 
-- 
1.9.1

[Buildroot] [PATCH 0/5 v5] core/pkg-kconfig: handle generated kconfig defconfigs (branch yem/kconfig-gen-defconfig)

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Tue, 22 Dec 2015 22:22:00 +0100, Yann E. MORIN wrote:

> Sam bobroff (4):
>       core/pkg-kconfig: handle generated defconfigs
>       package/linux: handle generated defconfigs
>       boot/at91bootstrap3: handle generated defconfigs
>       boot/barebox: handle generated defconfigs

I've applied the series, after adding an additional patch doing the
same change for the uboot package.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com