[Buildroot] [PATCH 1/7 v4] linux: explain why we need host-{flex, bison}

[Buildroot] [PATCH 1/7 v4] linux: explain why we need host-{flex, bison}

[Yann E. MORIN]

The commit that added the dependency on host-{bison,flex} did
so because the pre-generated kconfig parser source files were
removed from the kernel tree, in linux-4.16.

But then, in linux-4.17, the pre-generated dtc parser source
files were in turn removed as well.

So, document the two reasons why they are needed, so we don't
accidentally remove them when we (soon) introduce the kconfig
dependencies.

(Also fix the first assignment to LINUX_DEPENDENCIES to be a
simple assignement, not an append-assignment.)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 linux/linux.mk | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index e0d437427d..d307970065 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -62,7 +62,13 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH))
 LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES))
 
 LINUX_INSTALL_IMAGES = YES
-LINUX_DEPENDENCIES += host-bison host-flex host-kmod
+LINUX_DEPENDENCIES = host-kmod
+
+# Starting with 4.16, the generated kconfig paser code is no longer
+# shipped with the kernel sources, so we need flex and bison.
+# Starting with 4.17, the generated dtc parser code is no longer
+# shipped with the kernel sources, so we need flex and bison.
+LINUX_DEPENDENCIES += host-bison host-flex
 
 # host tools needed for kernel compression
 ifeq ($(BR2_LINUX_KERNEL_LZ4),y)
-- 
2.14.1

[Buildroot] [PATCH 2/7 v4] support/dependencies: check for system-provided bison and flex

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 support/dependencies/check-host-bison-flex.mk | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 support/dependencies/check-host-bison-flex.mk

diff --git a/support/dependencies/check-host-bison-flex.mk b/support/dependencies/check-host-bison-flex.mk
new file mode 100644
index 0000000000..233b6c51cc
--- /dev/null
+++ b/support/dependencies/check-host-bison-flex.mk
@@ -0,0 +1,10 @@
+# If the system lacks bison or flex, add
+# dependencies to suitable host packages
+
+ifeq ($(shell which bison 2>/dev/null),)
+BR2_BISON_HOST_DEPENDENCY = host-bison
+endif
+
+ifeq ($(shell which flex 2>/dev/null),)
+BR2_FLEX_HOST_DEPENDENCY = host-flex
+endif
-- 
2.14.1

[Buildroot] [PATCH 3/7 v4] core/pkg-kconfig: allow dependencies before configurators

[Yann E. MORIN]

Some users of kconfig need some packages to be built before their
kconfig infra be used.

For example, the linux kernel, starting with 4.16, needs flex and bison
to generate the parser code. Furthermore, starting with 4.18, it will
also need the cross-compiler before parsing the kconfig stuff, because
that calls the compiler to check its features.

Currently, this is broken, even the flex/bison ones, even though they
are listed, because there is no way to define dependencie that are
guaranteed before the (visual) configurators. For example:

    $ make distclean
    $ make menuconfig
      --> enable the linux kernel, choose a defconfig, save, exit
    $ make linux-menuconfig
    [...]
      HOSTCC  scripts/basic/fixdep
      HOSTCC  scripts/kconfig/conf.o
      YACC    scripts/kconfig/zconf.tab.c
    /bin/sh: bison: command not found
      LEX     scripts/kconfig/zconf.lex.c
    scripts/Makefile.lib:196: recipe for target 'scripts/kconfig/zconf.tab.c' failed
    make[3]: *** [scripts/kconfig/zconf.tab.c] Error 127
    make[3]: *** Waiting for unfinished jobs....
    /bin/sh: flex: command not found
    scripts/Makefile.lib:188: recipe for target 'scripts/kconfig/zconf.lex.c' failed
    make[3]: *** [scripts/kconfig/zconf.lex.c] Error 127
    Makefile:528: recipe for target 'rpc_defconfig' failed
    make[2]: *** [rpc_defconfig] Error 2
    linux/linux.mk:511: recipe for target '/home/ymorin/dev/buildroot/buildroot/output/build/linux-4.17.11/.config' failed
    make[1]: *** [/home/ymorin/dev/buildroot/buildroot/output/build/linux-4.17.11/.config] Error 2
    Makefile:79: recipe for target '_all' failed
    make: *** [_all] Error 2

So, we introduce a new type of dependencies for kconfig-based packages,
that are guaranteed to be built and installed before the (visual)
configurators are called.

Since those dependencies are phony targets and therefore always out of
date, a normal dependency would cause the .config target to be rebuilt
on each invocation of make. So we use an order-only pre-requisite, like
is done for the patch dependency.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Jan Kundr?t <jan.kundrat@cesnet.cz>
Tested-by: Jan Kundr?t <jan.kundrat@cesnet.cz>
---
 docs/manual/adding-packages-kconfig.txt | 4 ++++
 package/pkg-kconfig.mk                  | 9 +++++++++
 2 files changed, 13 insertions(+)

diff --git a/docs/manual/adding-packages-kconfig.txt b/docs/manual/adding-packages-kconfig.txt
index d4d8630a79..982c1229ea 100644
--- a/docs/manual/adding-packages-kconfig.txt
+++ b/docs/manual/adding-packages-kconfig.txt
@@ -81,3 +81,7 @@ be set to suit the needs of the package under consideration:
   be well suited for all packages that use the standard kconfig
   infrastructure as inherited from the Linux kernel; some packages use
   a derivative of kconfig that use a different location.
+
+* +FOO_KCONFIG_DEPENDENCIES+: the list of packages (most probably, host
+  packages) that need to be built before this package's kconfig is
+  interpreted. Seldom used. By default, empty.
diff --git a/package/pkg-kconfig.mk b/package/pkg-kconfig.mk
index 81bba5220c..d6c95b897e 100644
--- a/package/pkg-kconfig.mk
+++ b/package/pkg-kconfig.mk
@@ -41,6 +41,10 @@ endef
 
 define inner-kconfig-package
 
+# Register the kconfig dependencies as regular dependencies, so that
+# they are also accounted for in the generated graphs.
+$(2)_DEPENDENCIES += $$($(2)_KCONFIG_DEPENDENCIES)
+
 # Call the generic package infrastructure to generate the necessary
 # make targets.
 # Note: this must be done _before_ attempting to use $$($(2)_DIR) in a
@@ -121,6 +125,11 @@ $$($(2)_DIR)/$$($(2)_KCONFIG_DOTCONFIG): $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_F
 # it explicitly. It doesn't hurt to always have it though.
 $$($(2)_DIR)/$$($(2)_KCONFIG_DOTCONFIG): | $(1)-patch
 
+# Some packages may need additional tools to be present by the time their
+# kconfig structure is parsed (e.g. the linux kernel may need to call to
+# the compiler to test its features).
+$$($(2)_DIR)/$$($(2)_KCONFIG_DOTCONFIG): | $$($(2)_KCONFIG_DEPENDENCIES)
+
 # In order to get a usable, consistent configuration, some fixup may be needed.
 # The exact rules are specified by the package .mk file.
 define $(2)_FIXUP_DOT_CONFIG
-- 
2.14.1

[Buildroot] [PATCH 0/7 v4] core/pkg-kconfig: ensure we have necessary tools to run configurators

[Yann E. MORIN]

Hello All!

Recent version of the linux kernel introduced major changes in the
kconfig infrastructure.

Starting with linux-4.16, the kconfig parser code is no longer shipped
with the linux source tree, and mut be generated from the lex/yacc
parser sources.

Similarly, starting with linux-4.18, the kconfig infra allows calling
back to the shell to run various commands, especially calling to the
compiler to test its features (and hide/show options accordingly).

So, we need to ensure the appropriate tools are available before we
attempt to run any of the configurators. Since building host-flex and
host-bison is not entirely free (they both have a few dependencies of
their own), we want to be able to use the system ones if they are
present.

This seven-patch series first documents the reasons for having host-flex
and host-bison as dependencies of the linux package.

Then we add support for detecting if the system provides flex and/or
bison; if either is missing, a conditional dependency is defined for
either.

Third, the kconfig infra is enhanced to accept dependencies needed
before the configurators are called.

The linux packages is then adapted, one step at a time, to use the
conditional flex/bison dependencies for the configurators, and use
the host-{flex,bison} dependencies only for the DTS support.

Then, the toolchain is added as a kconfig dependencies of the linux
kernel.

Eventually, uboot is also fixed to conditionally require flex/bison.

Changes v3 -> v4:
  - also fix uboot

(note: there was no v2.)

Changes v1 -> v3:
  - introduce the KCONFIG_DEPENDENCIES in the kconfig-package infra
    (Thomas)
  - change linux to take advantage of that  (Thomas)


Regards,
Yann E. MORIN.


The following changes since commit f70b13a7cb6eb01e9d78fddcb695b84a22e5699b

  dahdi-tools: unify dependencies comments (2018-08-17 17:07:11 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to ed63f1c812145907ef83c6bf245ad5765c67aded

  boot/uboot: needs host-{flex,bison} (2018-08-17 17:40:03 +0200)


----------------------------------------------------------------
Yann E. MORIN (7):
      linux: explain why we need host-{flex,bison}
      support/dependencies: check for system-provided bison and flex
      core/pkg-kconfig: allow dependencies before configurators
      linux: kconfig may need host-{flex,bison} to build the configurators
      linux: host-{flex,bison} only needed for DTS
      linux: kconfig needs the toolchain
      boot/uboot: needs host-{flex,bison}

 boot/uboot/uboot.mk                           |  3 +++
 docs/manual/adding-packages-kconfig.txt       |  4 ++++
 linux/linux.mk                                | 19 ++++++++++++++++++-
 package/pkg-kconfig.mk                        |  9 +++++++++
 support/dependencies/check-host-bison-flex.mk | 10 ++++++++++
 5 files changed, 44 insertions(+), 1 deletion(-)
 create mode 100644 support/dependencies/check-host-bison-flex.mk

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 4/7 v4] linux: kconfig may need host-{flex, bison} to build the configurators

[Yann E. MORIN]

Rely on the system provided ones if avalable, and only resort to use our
owns if the sytem does not provide them.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Jan Kundr?t <jan.kundrat@cesnet.cz>

---
Notes: I did not carry Jan's Tested-by tag, because the patch is now
using the conditional dependencies, so it would have been a bit
presumptuous of me to carry that tag.

---
Changes v2 -> v3:
  - use the newly-introduced conditional dependencies  (Thomas, Arnout)
---
 linux/linux.mk | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index d307970065..deb5a4ac57 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -64,12 +64,18 @@ LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES))
 LINUX_INSTALL_IMAGES = YES
 LINUX_DEPENDENCIES = host-kmod
 
-# Starting with 4.16, the generated kconfig paser code is no longer
-# shipped with the kernel sources, so we need flex and bison.
 # Starting with 4.17, the generated dtc parser code is no longer
-# shipped with the kernel sources, so we need flex and bison.
+# shipped with the kernel sources, so we need flex and bison. For
+# reproducibility, we use our owns rather than the host ones.
 LINUX_DEPENDENCIES += host-bison host-flex
 
+# Starting with 4.16, the generated kconfig paser code is no longer
+# shipped with the kernel sources, so we need flex and bison, but
+# only if the host does not have them.
+LINUX_KCONFIG_DEPENDENCIES = \
+	$(BR2_BISON_HOST_DEPENDENCY) \
+	$(BR2_FLEX_HOST_DEPENDENCY)
+
 # host tools needed for kernel compression
 ifeq ($(BR2_LINUX_KERNEL_LZ4),y)
 LINUX_DEPENDENCIES += host-lz4
-- 
2.14.1

[Buildroot] [PATCH 5/7 v4] linux: host-{flex, bison} only needed for DTS

[Yann E. MORIN]

host-{flex,bison} are only needed to generate the dtc parser, so we
don't need them if the kernel does not have support for device tree.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 linux/linux.mk | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index deb5a4ac57..d6b0853518 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -64,11 +64,6 @@ LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES))
 LINUX_INSTALL_IMAGES = YES
 LINUX_DEPENDENCIES = host-kmod
 
-# Starting with 4.17, the generated dtc parser code is no longer
-# shipped with the kernel sources, so we need flex and bison. For
-# reproducibility, we use our owns rather than the host ones.
-LINUX_DEPENDENCIES += host-bison host-flex
-
 # Starting with 4.16, the generated kconfig paser code is no longer
 # shipped with the kernel sources, so we need flex and bison, but
 # only if the host does not have them.
@@ -352,6 +347,11 @@ define LINUX_KCONFIG_FIXUP_CMDS
 endef
 
 ifeq ($(BR2_LINUX_KERNEL_DTS_SUPPORT),y)
+# Starting with 4.17, the generated dtc parser code is no longer
+# shipped with the kernel sources, so we need flex and bison. For
+# reproducibility, we use our owns rather than the host ones.
+LINUX_DEPENDENCIES += host-bison host-flex
+
 ifeq ($(BR2_LINUX_KERNEL_DTB_IS_SELF_BUILT),)
 define LINUX_BUILD_DTB
 	$(LINUX_MAKE_ENV) $(MAKE) $(LINUX_MAKE_FLAGS) -C $(@D) $(LINUX_DTBS)
-- 
2.14.1

[Buildroot] [PATCH 6/7 v4] linux: kconfig needs the toolchain

[Yann E. MORIN]

Starting with linux-4.18, the kconfig from the kernel can call
to the compiler to test its capabilities; see:

    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/Kconfig.include

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Jan Kundr?t <jan.kundrat@cesnet.cz>
Tested-by: Jan Kundr?t <jan.kundrat@cesnet.cz>
Reviewed-by: Jan Kundr?t <jan.kundrat@cesnet.cz>

---
Changes v1 -> v2:
  - take advantage of the kconfig-package infra  (Thomas)
---
 linux/linux.mk | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/linux/linux.mk b/linux/linux.mk
index d6b0853518..675e7906a8 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -71,6 +71,11 @@ LINUX_KCONFIG_DEPENDENCIES = \
 	$(BR2_BISON_HOST_DEPENDENCY) \
 	$(BR2_FLEX_HOST_DEPENDENCY)
 
+# Starting with 4.18, the kconfig in the kernel calls the
+# cross-compiler to check its capabilities. So we need the
+# toolchain before we can call the configurators.
+LINUX_KCONFIG_DEPENDENCIES += toolchain
+
 # host tools needed for kernel compression
 ifeq ($(BR2_LINUX_KERNEL_LZ4),y)
 LINUX_DEPENDENCIES += host-lz4
-- 
2.14.1

[Buildroot] [PATCH 7/7 v4] boot/uboot: needs host-{flex,bison}

[Yann E. MORIN]

Recent U-Boot no longer ship the flex/bison generated kconfig parser, as
of commit e91610da7c8a9fe42f3e5a75f06c3d1a0cb5f815 (kconfig: re-sync
with Linux 4.17-rc4).

So, add the conditional kconfig dependencies, as we just did for the
kernel.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 boot/uboot/uboot.mk | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
index 04c759be34..bddafe234d 100644
--- a/boot/uboot/uboot.mk
+++ b/boot/uboot/uboot.mk
@@ -449,5 +449,8 @@ endif # BR2_TARGET_UBOOT && BR_BUILDING
 ifeq ($(BR2_TARGET_UBOOT_BUILD_SYSTEM_LEGACY),y)
 $(eval $(generic-package))
 else ifeq ($(BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG),y)
+UBOOT_KCONFIG_DEPENDENCIES = \
+	$(BR2_BISON_HOST_DEPENDENCY) \
+	$(BR2_FLEX_HOST_DEPENDENCY)
 $(eval $(kconfig-package))
 endif # BR2_TARGET_UBOOT_BUILD_SYSTEM_LEGACY
-- 
2.14.1

[Buildroot] [PATCH 1/7 v4] linux: explain why we need host-{flex, bison}

[Thomas Petazzoni]

Hello,

On Fri, 17 Aug 2018 18:06:47 +0200, Yann E. MORIN wrote:
> The commit that added the dependency on host-{bison,flex} did
> so because the pre-generated kconfig parser source files were
> removed from the kernel tree, in linux-4.16.
> 
> But then, in linux-4.17, the pre-generated dtc parser source
> files were in turn removed as well.
> 
> So, document the two reasons why they are needed, so we don't
> accidentally remove them when we (soon) introduce the kconfig
> dependencies.
> 
> (Also fix the first assignment to LINUX_DEPENDENCIES to be a
> simple assignement, not an append-assignment.)
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  linux/linux.mk | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 2/7 v4] support/dependencies: check for system-provided bison and flex

[Thomas Petazzoni]

Hello,

On Fri, 17 Aug 2018 18:06:48 +0200, Yann E. MORIN wrote:
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  support/dependencies/check-host-bison-flex.mk | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>  create mode 100644 support/dependencies/check-host-bison-flex.mk
> 
> diff --git a/support/dependencies/check-host-bison-flex.mk b/support/dependencies/check-host-bison-flex.mk
> new file mode 100644
> index 0000000000..233b6c51cc
> --- /dev/null
> +++ b/support/dependencies/check-host-bison-flex.mk
> @@ -0,0 +1,10 @@
> +# If the system lacks bison or flex, add
> +# dependencies to suitable host packages
> +
> +ifeq ($(shell which bison 2>/dev/null),)
> +BR2_BISON_HOST_DEPENDENCY = host-bison
> +endif
> +
> +ifeq ($(shell which flex 2>/dev/null),)
> +BR2_FLEX_HOST_DEPENDENCY = host-flex
> +endif

This solution (which differs from what we do for other dependencies,
that are using suitable-host-package) has a drawback: it shows stderr
on the terminal, despite the 2>/dev/null:

$ make
which: no bison in (/usr/local/bin:/usr/bin:/bin:/home/thomas/.rvm/bin:/usr/local/sbin:/usr/sbin:/home/thomas/.rvm/bin:/home/thomas/sys/bin:/home/thomas/.gem/ruby/2.1.0/bin:/home/thomas/.rvm/bin)

I'm not sure why this happens.

Even though it's a bit verbose in code, perhaps we should stick to the
existing model, and introduce check-host-bison.sh check-host-bison.mk
check-host-flex.sh and check-host-flex.mk ?

Long term, maybe all those .mk files are a bit stupid, and we could
simply do the all make logic from dependencies.mk (but that's not for
master).

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/7 v4] core/pkg-kconfig: allow dependencies before configurators

[Thomas Petazzoni]

Hello,

On Fri, 17 Aug 2018 18:06:49 +0200, Yann E. MORIN wrote:
> Some users of kconfig need some packages to be built before their
> kconfig infra be used.
> 
> For example, the linux kernel, starting with 4.16, needs flex and bison
> to generate the parser code. Furthermore, starting with 4.18, it will
> also need the cross-compiler before parsing the kconfig stuff, because
> that calls the compiler to check its features.
> 
> Currently, this is broken, even the flex/bison ones, even though they
> are listed, because there is no way to define dependencie that are
> guaranteed before the (visual) configurators. For example:
> 
>     $ make distclean
>     $ make menuconfig
>       --> enable the linux kernel, choose a defconfig, save, exit  
>     $ make linux-menuconfig
>     [...]
>       HOSTCC  scripts/basic/fixdep
>       HOSTCC  scripts/kconfig/conf.o
>       YACC    scripts/kconfig/zconf.tab.c
>     /bin/sh: bison: command not found
>       LEX     scripts/kconfig/zconf.lex.c
>     scripts/Makefile.lib:196: recipe for target 'scripts/kconfig/zconf.tab.c' failed
>     make[3]: *** [scripts/kconfig/zconf.tab.c] Error 127
>     make[3]: *** Waiting for unfinished jobs....
>     /bin/sh: flex: command not found
>     scripts/Makefile.lib:188: recipe for target 'scripts/kconfig/zconf.lex.c' failed
>     make[3]: *** [scripts/kconfig/zconf.lex.c] Error 127
>     Makefile:528: recipe for target 'rpc_defconfig' failed
>     make[2]: *** [rpc_defconfig] Error 2
>     linux/linux.mk:511: recipe for target '/home/ymorin/dev/buildroot/buildroot/output/build/linux-4.17.11/.config' failed
>     make[1]: *** [/home/ymorin/dev/buildroot/buildroot/output/build/linux-4.17.11/.config] Error 2
>     Makefile:79: recipe for target '_all' failed
>     make: *** [_all] Error 2
> 
> So, we introduce a new type of dependencies for kconfig-based packages,
> that are guaranteed to be built and installed before the (visual)
> configurators are called.
> 
> Since those dependencies are phony targets and therefore always out of
> date, a normal dependency would cause the .config target to be rebuilt
> on each invocation of make. So we use an order-only pre-requisite, like
> is done for the patch dependency.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Jan Kundr?t <jan.kundrat@cesnet.cz>
> Tested-by: Jan Kundr?t <jan.kundrat@cesnet.cz>
> ---
>  docs/manual/adding-packages-kconfig.txt | 4 ++++
>  package/pkg-kconfig.mk                  | 9 +++++++++
>  2 files changed, 13 insertions(+)

Applied to master, thanks. If anybody wonders why this gets into
master: I believe people are going to hit this problem, either because
of the bison or flex dependency, or because of the toolchain
dependency, and this is something we should fix in our upcoming 2018.08
release.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 2/7 v4] support/dependencies: check for system-provided bison and flex

[Thomas Petazzoni]

Hello,

On Sat, 18 Aug 2018 23:17:58 +0200, Thomas Petazzoni wrote:

> This solution (which differs from what we do for other dependencies,
> that are using suitable-host-package) has a drawback: it shows stderr
> on the terminal, despite the 2>/dev/null:
> 
> $ make
> which: no bison in (/usr/local/bin:/usr/bin:/bin:/home/thomas/.rvm/bin:/usr/local/sbin:/usr/sbin:/home/thomas/.rvm/bin:/home/thomas/sys/bin:/home/thomas/.gem/ruby/2.1.0/bin:/home/thomas/.rvm/bin)
> 
> I'm not sure why this happens.

Wait, wait, scratch that. Even without your patch it does that. This
error is displayed because of:

BISON := $(shell which bison || type -p bison)

in package/Makefile.in. I'm not even sure why we have this definition,
we don't use it anywhere, ditto for the FLEX definition.

So basically your patch looks good, we probably need to remove the
useless lines in package/Makefile.in, as a separate patch.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 0/7 v4] core/pkg-kconfig: ensure we have necessary tools to run configurators

[Thomas Petazzoni]

Hello,

On Fri, 17 Aug 2018 18:06:50 +0200, Yann E. MORIN wrote:

> Yann E. MORIN (7):
>       linux: explain why we need host-{flex,bison}
>       support/dependencies: check for system-provided bison and flex
>       core/pkg-kconfig: allow dependencies before configurators
>       linux: kconfig may need host-{flex,bison} to build the configurators
>       linux: host-{flex,bison} only needed for DTS
>       linux: kconfig needs the toolchain
>       boot/uboot: needs host-{flex,bison}

Series applied to master. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 7/7 v4] boot/uboot: needs host-{flex,bison}

[Thomas Petazzoni]

Hello,

On Fri, 17 Aug 2018 18:06:53 +0200, Yann E. MORIN wrote:
> Recent U-Boot no longer ship the flex/bison generated kconfig parser, as
> of commit e91610da7c8a9fe42f3e5a75f06c3d1a0cb5f815 (kconfig: re-sync
> with Linux 4.17-rc4).
> 
> So, add the conditional kconfig dependencies, as we just did for the
> kernel.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> ---
>  boot/uboot/uboot.mk | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
> index 04c759be34..bddafe234d 100644
> --- a/boot/uboot/uboot.mk
> +++ b/boot/uboot/uboot.mk
> @@ -449,5 +449,8 @@ endif # BR2_TARGET_UBOOT && BR_BUILDING
>  ifeq ($(BR2_TARGET_UBOOT_BUILD_SYSTEM_LEGACY),y)
>  $(eval $(generic-package))
>  else ifeq ($(BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG),y)
> +UBOOT_KCONFIG_DEPENDENCIES = \
> +	$(BR2_BISON_HOST_DEPENDENCY) \
> +	$(BR2_FLEX_HOST_DEPENDENCY)
>  $(eval $(kconfig-package))
>  endif # BR2_TARGET_UBOOT_BUILD_SYSTEM_LEGACY

This commit doesn't work well, because even if you chose "legacy build
system", kconfig is used, and it tries to use bison.

Example configuration:

BR2_TARGET_UBOOT=y
BR2_TARGET_UBOOT_BOARDNAME="mvebu_mcbin-88f8040"
BR2_TARGET_UBOOT_CUSTOM_VERSION=y
BR2_TARGET_UBOOT_CUSTOM_VERSION_VALUE="2018.09"

fails with:

>>> uboot 2018.09 Configuring
[...]
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  YACC    scripts/kconfig/zconf.tab.c
/bin/sh: 1: bison: not found
make[3]: *** [scripts/kconfig/zconf.tab.c] Error 127
make[3]: *** Waiting for unfinished jobs....
  LEX     scripts/kconfig/zconf.lex.c
/bin/sh: 1: flex: not found
make[3]: *** [scripts/kconfig/zconf.lex.c] Error 127
make[2]: *** [mvebu_mcbin-88f8040_config] Error 2
make[1]: *** [/home/test/outputs/mainline/build/uboot-2018.09/.stamp_configured] Error 2
make: *** [_all] Error 2

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com