[Buildroot] [PATCHv3 00/20] Toolchain updates: package infra, musl support, Linaro updates, glibc fixes

[Buildroot] [PATCHv3 00/20] Toolchain updates: package infra, musl support, Linaro updates, glibc fixes

[Thomas Petazzoni]

Hello,

Here is a new version of the toolchain fixes and updates. Careful
review of patches 19 and 20 would be appreciated.

Peter, the other patches (before 19 and 20) can be merged even if
there is still ongoing discussion on patches 19 and 20.

Changes since v2:

 * Different order for the patches: the intrusive modification of the
   package infrastructure to support package-based 'make
   external-deps' is pushed at the end of the series, since it's most
   likely to be the commit causing the biggest amount of
   discussion/testing.

 * Two fixes for the internal glibc build logic have been added.

 * The helpers.mk is no longer moved into
   toolchain/toolchain-external/. Even though moving it in this
   location is not causing any problem, it is not very logical, since
   it's also used by the package/glibc/glibc.mk file, part of the
   internal toolchain backend.

 * Some minor cleanup of the helpers.mk file.

Changes since RFCv1:

 * Remove remaining references to the Crosstool-NG backend in the
   source code, noticed by Thomas De Schampheleire.

 * Update the manual after Crosstool-NG backend removal

 * Added Acked-by on the relevant patches.

 * Ensure 'helpers.mk' is properly include from the right place after
   being moved into toolchain-external. Noticed by Luca Ceresoli.

 * Fixed a few typos in comments, noticed by Thomas De Schampheleire.

 * Renamed <pkg>_SOURCE_ADDONS to <pkg>_EXTRA_DOWNLOADS as suggested
   by Luca Ceresoli, and added documentation for this new variable in
   the Buildroot manual.

 * The support for musl-based external toolchains has been integrated
   in this patch set, together with the fix for the hardcoded
   ld-linux-arm.so problem that was noticed in the v1 of the musl
   patch set.

 * Updates for the Linaro ARM and AArch64 toolchains have been added.

 * A fix for the Linaro ARM related bug #6452 has been added.

Thanks,

Thomas

Thomas Petazzoni (20):
  glibc: both eglibc and glibc need host-gawk
  glibc: fix glibc build by creating an empty gnu/stubs.h
  package: add a <pkg>_EXTRA_DOWNLOADS variable
  toolchain: introduce a virtual package
  toolchain-buildroot: convert to the package infrastructure
  toolchain-external: convert to the package infrastructure
  dependencies: remove useless targets
  toolchain-external: conditionalize the installation of libraries
  toolchain: modify the wildcard logic for shared libraries copying
  toolchain: fix the helpers.mk comment
  toolchain: do not check largefile, wchar, IPv6 and locale for glibc
    toolchains
  toolchain-external: add support for musl C library
  toolchain-external: improve help text of some options
  busybox: add patches to fix build with the musl C library
  toolchain-external: update Linaro ARM toolchain
  toolchain-external: update Linaro AArch64 toolchain
  toolchain-external: improve target library copy logic
  toolchain-external: fix Linaro ARM toolchain support
  package: package-based implementation of source, external-deps and
    legal-info
  package: remove useless SHOW_EXTERNAL_DEPS support

 Makefile                                           |  70 ++-----
 docs/manual/adding-packages-generic.txt            |   6 +
 ...01-libbb-use-poll.h-instead-of-sys-poll.h.patch |  32 +++
 ...onfig-include-linux-if_slip.h-instead-of-.patch |  33 ++++
 ...libiproute-use-linux-if_packet.h-instead-.patch |  36 ++++
 package/glibc/glibc.mk                             |  27 ++-
 package/pkg-download.mk                            |  44 +----
 package/pkg-generic.mk                             |  22 ++-
 support/dependencies/dependencies.mk               |   8 -
 toolchain/Config.in                                |   9 +
 toolchain/helpers.mk                               |  46 +++--
 toolchain/toolchain-buildroot.mk                   |   9 -
 .../toolchain-buildroot/toolchain-buildroot.mk     |  17 ++
 toolchain/toolchain-external.mk                    |   4 -
 toolchain/toolchain-external/Config.in             |  61 ++++--
 .../{ext-tool.mk => toolchain-external.mk}         | 220 +++++++++++----------
 toolchain/toolchain/toolchain.mk                   |  18 ++
 17 files changed, 391 insertions(+), 271 deletions(-)
 create mode 100644 package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
 create mode 100644 package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
 create mode 100644 package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch
 delete mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk
 delete mode 100644 toolchain/toolchain-external.mk
 rename toolchain/toolchain-external/{ext-tool.mk => toolchain-external.mk} (82%)
 create mode 100644 toolchain/toolchain/toolchain.mk

-- 
1.8.1.2

[Buildroot] [PATCHv3 01/20] glibc: both eglibc and glibc need host-gawk

[Thomas Petazzoni]

Until now, only the eglibc build was pulling host-gawk as a
dependency, but after more testing, it turns out that the glibc build
also requires host-gawk.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/glibc/glibc.mk | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
index 48ff0f7..a9e89f1 100644
--- a/package/glibc/glibc.mk
+++ b/package/glibc/glibc.mk
@@ -21,12 +21,7 @@ GLIBC_LICENSE_FILES = $(addprefix $(GLIBC_SRC_SUBDIR)/,COPYING COPYING.LIB LICEN
 
 # Before (e)glibc is configured, we must have the first stage
 # cross-compiler and the kernel headers
-GLIBC_DEPENDENCIES = host-gcc-initial linux-headers
-
-# eglibc also needs host-gawk
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT_EGLIBC),y)
-GLIBC_DEPENDENCIES += host-gawk
-endif
+GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-gawk
 
 # Before (e)glibc is built, we must have the second stage cross-compiler
 glibc-build: host-gcc-intermediate
-- 
1.8.1.2

[Buildroot] [PATCHv3 02/20] glibc: fix glibc build by creating an empty gnu/stubs.h

[Thomas Petazzoni]

The stubs.h header is not installed by install-headers, but is needed
for the gcc build. An empty stubs.h will work, as explained in
http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick is
used by Crosstool-NG.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/glibc/glibc.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
index a9e89f1..4d89caf 100644
--- a/package/glibc/glibc.mk
+++ b/package/glibc/glibc.mk
@@ -47,6 +47,17 @@ else ifeq ($(BR2_MIPS_OABI32),y)
 GLIBC_EXTRA_CFLAGS += -mabi=32
 endif
 
+# The stubs.h header is not installed by install-headers, but is
+# needed for the gcc build. An empty stubs.h will work, as explained
+# in http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick
+# is used by Crosstool-NG.
+ifeq ($(BR2_TOOLCHAIN_BUILDROOT_GLIBC),y)
+define GLIBC_ADD_MISSING_STUB_H
+	mkdir -p $(STAGING_DIR)/usr/include/gnu
+	touch $(STAGING_DIR)/usr/include/gnu/stubs.h
+endef
+endif
+
 # Even though we use the autotools-package infrastructure, we have to
 # override the default configure commands for several reasons:
 #
@@ -89,6 +100,7 @@ define GLIBC_CONFIGURE_CMDS
 	cp $(@D)/build/csu/crt1.o $(STAGING_DIR)/usr/lib/
 	cp $(@D)/build/csu/crti.o $(STAGING_DIR)/usr/lib/
 	cp $(@D)/build/csu/crtn.o $(STAGING_DIR)/usr/lib/
+	$(GLIBC_ADD_MISSING_STUB_H)
 	$(TARGET_CROSS)gcc -nostdlib \
 		-nostartfiles -shared -x c /dev/null -o $(STAGING_DIR)/usr/lib/libc.so
 endef
-- 
1.8.1.2

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Thomas Petazzoni]

Converting the external toolchain logic into a package raises a very
special use case that wasn't handled by the package infrastructure:
the Blackfin toolchain is delivered as two tarballs instead of
one. Unfortunately <pkg>_SOURCE only allows to pass one tarball name.

However, we really want both tarballs to be known by the package
infrastructure, so that the normal 'source' and 'external-deps'
mechanism work fine.

In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable,
which allows a package to list other stuff it would like to see
downloaded, but that are otherwise not used by the package
infrastructure itself: it is up to the package to do it by itself.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/adding-packages-generic.txt | 6 ++++++
 package/pkg-generic.mk                  | 1 +
 2 files changed, 7 insertions(+)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index cb067bd..59a2684 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -248,6 +248,12 @@ information is (assuming the package name is +libfoo+) :
     +LIBFOO_SITE=/opt/software/libfoo.tar.gz+ +
     +LIBFOO_SITE=$(TOPDIR)/../src/libfoo/+
 
+* +LIBFOO_EXTRA_DOWNLOADS+ lists a number of additional files that
+  Buildroot should download from +LIBFOO_SITE+ in addition to the main
+  +LIBFOO_SOURCE+ (which usually is a tarball). Buildroot will not do
+  anything with those additional files, except download files: it will
+  be up to the package recipe to use them from +$(DL_DIR)+.
+
 * +LIBFOO_SITE_METHOD+ determines the method used to fetch or copy the
   package source code. In many cases, Buildroot guesses the method
   from the contents of +LIBFOO_SITE+ and setting +LIBFOO_SITE_METHOD+
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index bfc4dc1..a46457c 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -40,6 +40,7 @@ ifeq ($(DL_MODE),DOWNLOAD)
 	fi
 endif
 	$(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$($(PKG)_SOURCE)))
+	$(foreach p,$($(PKG)_EXTRA_DOWNLOADS),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$(p))$(sep))
 	$(foreach p,$($(PKG)_PATCH),\
 		$(if $(findstring ://,$(p)),\
 			$(call DOWNLOAD,$(p)),\
-- 
1.8.1.2

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Thomas Petazzoni]

This commit introduces a virtual package called 'toolchain', located
in 'toolchain/toolchain', which simply depends on
'toolchain-buildroot' or 'toolchain-external' depending on the
selected toolchain backend.

For now, toolchain-buildroot and toolchain-external are still manual
make targets, but the following patches convert those backends to use
the package infrastructure as well.

In addition to this:

 * The main Makefile is modified to always make BASE_TARGETS point to
   this new toolchain virtual package.

 * The main Makefile is changed to include all the toolchain/*/*.mk
   files: the toolchain virtual package, and the toolchain-buildroot
   and toolchain-external directories.

 * The dependency of the toolchain on prepare dirs and dependencies is
   moved to the toolchain virtual package. It is moved as a
   prerequisite of the "toolchain-source" rule to ensure that all
   directories are prepared before we even start extracting the
   toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
 Makefile                         | 19 ++++---------------
 toolchain/toolchain/toolchain.mk | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+), 15 deletions(-)
 create mode 100644 toolchain/toolchain/toolchain.mk

diff --git a/Makefile b/Makefile
index 2fb1168..f266e2d 100644
--- a/Makefile
+++ b/Makefile
@@ -236,11 +236,7 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 #
 ################################################################################
 
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-BASE_TARGETS += toolchain-buildroot
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-BASE_TARGETS += toolchain-external
-endif
+BASE_TARGETS = toolchain
 
 TARGETS:=
 
@@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk
 # We also need the various per-package makefiles, which also add
 # each selected package to TARGETS if that package was selected
 # in the .config file.
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-include toolchain/toolchain-buildroot.mk
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-include toolchain/toolchain-external.mk
-endif
+include toolchain/helpers.mk
+include toolchain/*/*.mk
 
 # Include the package override file if one has been provided in the
 # configuration.
@@ -388,16 +381,12 @@ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
-$(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
-
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
 
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-toolchain: prepare dirs dependencies $(BASE_TARGETS)
-
-world: toolchain $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(TARGETS_ALL)
 
 .PHONY: all world toolchain dirs clean distclean source outputmakefile \
 	legal-info legal-info-prepare legal-info-clean printvars \
diff --git a/toolchain/toolchain/toolchain.mk b/toolchain/toolchain/toolchain.mk
new file mode 100644
index 0000000..44ed629
--- /dev/null
+++ b/toolchain/toolchain/toolchain.mk
@@ -0,0 +1,18 @@
+################################################################################
+#
+# toolchain
+#
+################################################################################
+
+TOOLCHAIN_SOURCE =
+
+ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
+TOOLCHAIN_DEPENDENCIES += toolchain-buildroot
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
+TOOLCHAIN_DEPENDENCIES += toolchain-external
+endif
+
+$(eval $(generic-package))
+
+toolchain-source: prepare dirs dependencies $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
+
-- 
1.8.1.2

[Buildroot] [PATCHv3 05/20] toolchain-buildroot: convert to the package infrastructure

[Thomas Petazzoni]

This commit converts the toolchain-buildroot logic to the package
infrastructure. The package is fairly simple as it only defines
BUILDROOT_LIBC, and depends on host-gcc-final to get the overall
internal toolchain build logic started.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
 toolchain/toolchain-buildroot.mk                     |  9 ---------
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 17 +++++++++++++++++
 2 files changed, 17 insertions(+), 9 deletions(-)
 delete mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk

diff --git a/toolchain/toolchain-buildroot.mk b/toolchain/toolchain-buildroot.mk
deleted file mode 100644
index 3a05800..0000000
--- a/toolchain/toolchain-buildroot.mk
+++ /dev/null
@@ -1,9 +0,0 @@
-# Triggerring the build of the host-gcc-final will automaticaly do the
-# build of binutils, uClibc, kernel headers and all the intermediate
-# gcc steps.
-
-include toolchain/helpers.mk
-
-BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
-
-toolchain-buildroot: host-gcc-final
diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk
new file mode 100644
index 0000000..ced430e
--- /dev/null
+++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk
@@ -0,0 +1,17 @@
+################################################################################
+#
+# toolchain-buildroot
+#
+################################################################################
+
+TOOLCHAIN_BUILDROOT_SOURCE =
+
+BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
+
+# Trigerring the build of the host-gcc-final will automatically do the
+# build of binutils, uClibc, kernel headers and all the intermediate
+# gcc steps.
+
+TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
+
+$(eval $(generic-package))
-- 
1.8.1.2

[Buildroot] [PATCHv3 06/20] toolchain-external: convert to the package infrastructure

[Thomas Petazzoni]

This commit converts the 'toolchain-external' logic to the package
infrastructure.

The TOOLCHAIN_EXTERNAL_DIR variable (which points to where the
toolchain is located) is renamed to TOOLCHAIN_EXTERNAL_INSTALL_DIR,
because the former conflicts with the package infrastructure (which
defines the <pkg>_DIR variable for each package as pointing to its
build directory).

The new _EXTRA_DOWNLOADS mechanism is used for Blackfin toolchains.

The extract, configuration and installation steps are converted inside
the <pkg>_EXTRACT_CMDS, <pkg>_CONFIGURE_CMDS and
<pkg>_INSTALL_STAGING_CMDS.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
---
 toolchain/toolchain-external.mk                    |   4 -
 .../{ext-tool.mk => toolchain-external.mk}         | 154 +++++++++------------
 2 files changed, 69 insertions(+), 89 deletions(-)
 delete mode 100644 toolchain/toolchain-external.mk
 rename toolchain/toolchain-external/{ext-tool.mk => toolchain-external.mk} (86%)

diff --git a/toolchain/toolchain-external.mk b/toolchain/toolchain-external.mk
deleted file mode 100644
index 2f43db7..0000000
--- a/toolchain/toolchain-external.mk
+++ /dev/null
@@ -1,4 +0,0 @@
-# Required includes for the external toolchain backend
-
-include toolchain/helpers.mk
-include toolchain/toolchain-external/ext-tool.mk
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/toolchain-external.mk
similarity index 86%
rename from toolchain/toolchain-external/ext-tool.mk
rename to toolchain/toolchain-external/toolchain-external.mk
index 8cfd1b1..85fc341 100644
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -1,5 +1,11 @@
+################################################################################
 #
-# This file implements the support for external toolchains, i.e
+# toolchain-external
+#
+################################################################################
+
+#
+# This package implements the support for external toolchains, i.e
 # toolchains that have not been produced by Buildroot itself and that
 # Buildroot can download from the Web or that are already available on
 # the system on which Buildroot runs. So far, we have tested this
@@ -14,7 +20,7 @@
 # The basic principle is the following
 #
 #  1. If the toolchain is not pre-installed, download and extract it
-#  in $(TOOLCHAIN_EXTERNAL_DIR).
+#  in $(TOOLCHAIN_EXTERNAL_INSTALL_DIR).
 #
 #  2. For all external toolchains, perform some checks on the
 #  conformity between the toolchain configuration described in the
@@ -105,21 +111,21 @@ endif # ! no threads
 
 TOOLCHAIN_EXTERNAL_PREFIX=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PREFIX))
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
-TOOLCHAIN_EXTERNAL_DIR=$(HOST_DIR)/opt/ext-toolchain
+TOOLCHAIN_EXTERNAL_INSTALL_DIR=$(HOST_DIR)/opt/ext-toolchain
 else
-TOOLCHAIN_EXTERNAL_DIR=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PATH))
+TOOLCHAIN_EXTERNAL_INSTALL_DIR=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PATH))
 endif
 
-ifeq ($(TOOLCHAIN_EXTERNAL_DIR),)
+ifeq ($(TOOLCHAIN_EXTERNAL_INSTALL_DIR),)
 ifneq ($(TOOLCHAIN_EXTERNAL_PREFIX),)
 # if no path set, figure it out from path
 TOOLCHAIN_EXTERNAL_BIN := $(shell dirname $(shell which $(TOOLCHAIN_EXTERNAL_PREFIX)-gcc))
 endif
 else
 ifeq ($(BR2_bfin),y)
-TOOLCHAIN_EXTERNAL_BIN := $(TOOLCHAIN_EXTERNAL_DIR)/$(TOOLCHAIN_EXTERNAL_PREFIX)/bin
+TOOLCHAIN_EXTERNAL_BIN := $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/$(TOOLCHAIN_EXTERNAL_PREFIX)/bin
 else
-TOOLCHAIN_EXTERNAL_BIN := $(TOOLCHAIN_EXTERNAL_DIR)/bin
+TOOLCHAIN_EXTERNAL_BIN := $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/bin
 endif
 endif
 
@@ -203,10 +209,6 @@ TOOLCHAIN_EXTERNAL_CFLAGS += -msoft-float
 TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_SOFTFLOAT=1
 endif
 
-ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
-TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
-endif
-
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201109),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/arm-none-linux-gnueabi/
 TOOLCHAIN_EXTERNAL_SOURCE = arm-2011.09-70-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
@@ -220,15 +222,15 @@ else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV7A_201109),y)
 TOOLCHAIN_EXTERNAL_SITE = http://software-dl.ti.com/sdoemb/sdoemb_public_sw/arago_toolchain/2011_09/exports/
 TOOLCHAIN_EXTERNAL_SOURCE = arago-2011.09-armv7a-linux-gnueabi-sdk.tar.bz2
 define TOOLCHAIN_EXTERNAL_FIXUP_CMDS
-	mv $(@D)/arago-2011.09/armv7a/* $(@D)/
-	rm -rf $(@D)/arago-2011.09/
+	mv $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/armv7a/* $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/
+	rm -rf $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/
 endef
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV5TE_201109),y)
 TOOLCHAIN_EXTERNAL_SITE = http://software-dl.ti.com/sdoemb/sdoemb_public_sw/arago_toolchain/2011_09/exports/
 TOOLCHAIN_EXTERNAL_SOURCE = arago-2011.09-armv5te-linux-gnueabi-sdk.tar.bz2
 define TOOLCHAIN_EXTERNAL_FIXUP_CMDS
-	mv $(@D)/arago-2011.09/armv5te/* $(@D)/
-	rm -rf $(@D)/arago-2011.09/
+	mv $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/armv5te/* $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/
+	rm -rf $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/
 endef
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06),y)
 TOOLCHAIN_EXTERNAL_SITE = https://releases.linaro.org/13.06/components/toolchain/binaries/
@@ -282,17 +284,13 @@ else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_X86_201209),y)
 TOOLCHAIN_EXTERNAL_SITE = https://sourcery.mentor.com/public/gnu_toolchain/i686-pc-linux-gnu/
 TOOLCHAIN_EXTERNAL_SOURCE = ia32-2012.09-62-i686-pc-linux-gnu-i386-linux.tar.bz2
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R1),y)
-TOOLCHAIN_EXTERNAL_SITE_1   = http://blackfin.uclinux.org/gf/download/frsrelease/559/9858/
-TOOLCHAIN_EXTERNAL_SOURCE_1 = blackfin-toolchain-2012R1-RC2.i386.tar.bz2
-TOOLCHAIN_EXTERNAL_SITE_2   = http://blackfin.uclinux.org/gf/download/frsrelease/559/9866/
-TOOLCHAIN_EXTERNAL_SOURCE_2 = blackfin-toolchain-uclibc-full-2012R1-RC2.i386.tar.bz2
-TOOLCHAIN_EXTERNAL_SOURCE   = $(TOOLCHAIN_EXTERNAL_SOURCE_1) $(TOOLCHAIN_EXTERNAL_SOURCE_2)
+TOOLCHAIN_EXTERNAL_SITE = http://downloads.sourceforge.net/projects/adi-toolchain/files/2012R1-BF60X/2012R1-RC2-BF60X/i386/
+TOOLCHAIN_EXTERNAL_SOURCE = blackfin-toolchain-2012R1-RC2.i386.tar.bz2
+TOOLCHAIN_EXTERNAL_EXTRA_DOWNLOADS = blackfin-toolchain-uclibc-full-2012R1-RC2.i386.tar.bz2
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R2),y)
-TOOLCHAIN_EXTERNAL_SITE_1   = http://blackfin.uclinux.org/gf/download/frsrelease/588/10139/
-TOOLCHAIN_EXTERNAL_SOURCE_1 = blackfin-toolchain-2012R2-RC2.i386.tar.bz2
-TOOLCHAIN_EXTERNAL_SITE_2   = http://blackfin.uclinux.org/gf/download/frsrelease/588/10147/
-TOOLCHAIN_EXTERNAL_SOURCE_2 = blackfin-toolchain-uclibc-full-2012R2-RC2.i386.tar.bz2
-TOOLCHAIN_EXTERNAL_SOURCE   = $(TOOLCHAIN_EXTERNAL_SOURCE_1) $(TOOLCHAIN_EXTERNAL_SOURCE_2)
+TOOLCHAIN_EXTERNAL_SITE   = http://downloads.sourceforge.net/project/adi-toolchain/2012R2/2012R2-RC2/i386/
+TOOLCHAIN_EXTERNAL_SOURCE = blackfin-toolchain-2012R2-RC2.i386.tar.bz2
+TOOLCHAIN_EXTERNAL_EXTRA_DOWNLOADS = blackfin-toolchain-uclibc-full-2012R2-RC2.i386.tar.bz2
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEEL_14_3),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sources.buildroot.net/
 TOOLCHAIN_EXTERNAL_SOURCE = lin32-microblazeel-unknown-linux-gnu_14.3_early.tar.xz
@@ -318,53 +316,38 @@ else
 # Custom toolchain
 TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
 TOOLCHAIN_EXTERNAL_SOURCE = $(notdir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
-
-# A value must be set (even if unused), otherwise the
-# $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) rule would override the main
-# $(DL_DIR) rule
-ifeq (,$(TOOLCHAIN_EXTERNAL_SOURCE))
-TOOLCHAIN_EXTERNAL_SOURCE = none
-endif
 endif
 
+TOOLCHAIN_EXTERNAL_INSTALL_STAGING = YES
+
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R1)$(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R2),y)
 # Special handling for Blackfin toolchain, because of the split in two
 # tarballs, and the organization of tarball contents. The tarballs
 # contain ./opt/uClinux/{bfin-uclinux,bfin-linux-uclibc} directories,
 # which themselves contain the toolchain. This is why we strip more
 # components than usual.
-ifeq ($(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R1)$(BR2_TOOLCHAIN_EXTERNAL_BLACKFIN_UCLINUX_2012R2),y)
-$(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1):
-	$(call DOWNLOAD,$(TOOLCHAIN_EXTERNAL_SITE_1:/=)/$(TOOLCHAIN_EXTERNAL_SOURCE_1))
-
-$(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2):
-	$(call DOWNLOAD,$(TOOLCHAIN_EXTERNAL_SITE_2:/=)/$(TOOLCHAIN_EXTERNAL_SOURCE_2))
-
-$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2)
-	mkdir -p $(@D)
-	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_SOURCE_1)) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) | \
-		$(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
-	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_SOURCE_2)) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2) | \
-		$(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
-	$(Q)touch $@
-else
-# Download and extraction of a toolchain
-$(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE):
-	$(call DOWNLOAD,$(TOOLCHAIN_EXTERNAL_SITE)$(TOOLCHAIN_EXTERNAL_SOURCE),$(TOOLCHAIN_EXTERNAL_SOURCE))
-
-$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE)
-	mkdir -p $(@D)
-	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_SOURCE)) $^ | \
-		$(TAR) $(TAR_STRIP_COMPONENTS)=1 --exclude='usr/lib/locale/*' -C $(@D) $(TAR_OPTIONS) -
+define TOOLCHAIN_EXTERNAL_EXTRACT_CMDS
+	mkdir -p $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)
+	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_SOURCE)) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) | \
+		$(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(TOOLCHAIN_EXTERNAL_INSTALL_DIR) $(TAR_OPTIONS) -
+	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_EXTRA_DOWNLOADS)) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_EXTRA_DOWNLOADS) | \
+		$(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(TOOLCHAIN_EXTERNAL_INSTALL_DIR) $(TAR_OPTIONS) -
+endef
+else ifneq ($(TOOLCHAIN_EXTERNAL_SOURCE),)
+# Normal handling of toolchain tarball extraction.
+define TOOLCHAIN_EXTERNAL_EXTRACT_CMDS
+	mkdir -p $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)
+	$(call suitable-extractor,$(TOOLCHAIN_EXTERNAL_SOURCE)) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) | \
+		$(TAR) $(TAR_STRIP_COMPONENTS)=1 --exclude='usr/lib/locale/*' -C $(TOOLCHAIN_EXTERNAL_INSTALL_DIR) $(TAR_OPTIONS) -
 	$(TOOLCHAIN_EXTERNAL_FIXUP_CMDS)
-	$(Q)touch $@
+endef
 endif
 
 # Checks for an already installed toolchain: check the toolchain
 # location, check that it supports sysroot, and then verify that it
 # matches the configuration provided in Buildroot: ABI, C++ support,
 # type of C library and all C library features.
-$(STAMP_DIR)/ext-toolchain-checked: $(TOOLCHAIN_EXTERNAL_DEPENDENCIES)
-	@$(call MESSAGE,"Checking external toolchain settings")
+define TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS
 	$(Q)$(call check_cross_compiler_exists,$(TOOLCHAIN_EXTERNAL_CC))
 	$(Q)LIBC_A_LOCATION=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a)` ; \
 	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:usr/lib(32|64)?/(.*/)?libc\.a::'` ; \
@@ -385,7 +368,7 @@ $(STAMP_DIR)/ext-toolchain-checked: $(TOOLCHAIN_EXTERNAL_DEPENDENCIES)
 	else \
 		$(call check_glibc,$${SYSROOT_DIR}) ; \
 	fi
-	$(Q)touch $@
+endef
 
 # Integration of the toolchain into Buildroot: find the main sysroot
 # and the variant-specific sysroot, then copy the needed libraries to
@@ -435,7 +418,7 @@ $(STAMP_DIR)/ext-toolchain-checked: $(TOOLCHAIN_EXTERNAL_DEPENDENCIES)
 #                       considered when searching libraries for copy
 #                       to the target filesystem.
 
-$(STAMP_DIR)/ext-toolchain-installed: $(STAMP_DIR)/ext-toolchain-checked
+define TOOLCHAIN_EXTERNAL_INSTALL_CORE
 	$(Q)LIBC_A_LOCATION=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a)` ; \
 	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:usr/lib(32|64)?/(.*/)?libc\.a::'` ; \
 	if test -z "$${SYSROOT_DIR}" ; then \
@@ -478,14 +461,15 @@ $(STAMP_DIR)/ext-toolchain-installed: $(STAMP_DIR)/ext-toolchain-checked
 			echo "Could not find gdbserver in external toolchain" ; \
 			exit 1 ; \
 		fi ; \
-	fi ; \
-	touch $@
+	fi
+endef
 
 # Special installation target used on the Blackfin architecture when
 # FDPIC is not the primary binary format being used, but the user has
 # nonetheless requested the installation of the FDPIC libraries to the
 # target filesystem.
-$(STAMP_DIR)/ext-toolchain-bfin-fdpic-shared-installed: $(STAMP_DIR)/ext-toolchain-checked
+ifeq ($(BR2_BFIN_INSTALL_FDPIC_SHARED),y)
+define TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC
 	$(Q)$(call MESSAGE,"Install external toolchain FDPIC libraries to target...") ; \
 	FDPIC_EXTERNAL_CC=$(dir $(TOOLCHAIN_EXTERNAL_CC))/../../bfin-linux-uclibc/bin/bfin-linux-uclibc-gcc ; \
 	FDPIC_LIBC_A_LOCATION=`readlink -f $$(LANG=C $${FDPIC_EXTERNAL_CC} $(TOOLCHAIN_EXTERNAL_CFLAGS) -print-file-name=libc.a)` ; \
@@ -503,8 +487,9 @@ $(STAMP_DIR)/ext-toolchain-bfin-fdpic-shared-installed: $(STAMP_DIR)/ext-toolcha
 	done ; \
 	for libs in $(USR_LIB_EXTERNAL_LIBS); do \
 	        $(call copy_toolchain_lib_root,$${FDPIC_SYSROOT_DIR},$${FDPIC_SUPPORT_LIB_DIR},$${FDPIC_LIB_DIR},$$libs,/usr/lib); \
-	done ; \
-	touch $@
+	done
+endef
+endif
 
 # Special installation target used on the Blackfin architecture when
 # shared FLAT is not the primary format being used, but the user has
@@ -512,36 +497,28 @@ $(STAMP_DIR)/ext-toolchain-bfin-fdpic-shared-installed: $(STAMP_DIR)/ext-toolcha
 # to the target filesystem. The flat libraries are found and linked
 # according to the index in name "libN.so". Index 1 is reserved for
 # the standard C library. Customer libraries can use 4 and above.
-$(STAMP_DIR)/ext-toolchain-bfin-shared-flat-installed: $(STAMP_DIR)/ext-toolchain-checked
+ifeq ($(BR2_BFIN_INSTALL_FLAT_SHARED),y)
+define TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT
 	$(Q)$(call MESSAGE,"Install external toolchain FLAT libraries to target...") ; \
 	FLAT_EXTERNAL_CC=$(dir $(TOOLCHAIN_EXTERNAL_CC))../../bfin-uclinux/bin/bfin-uclinux-gcc ; \
 	FLAT_LIBC_A_LOCATION=`$${FLAT_EXTERNAL_CC} $(TOOLCHAIN_EXTERNAL_CFLAGS) -mid-shared-library -print-file-name=libc`; \
 	if [ -f $${FLAT_LIBC_A_LOCATION} -a ! -h $${FLAT_LIBC_A_LOCATION} ] ; then \
 	        $(INSTALL) -D $${FLAT_LIBC_A_LOCATION} $(TARGET_DIR)/lib/lib1.so; \
-	fi ; \
-	touch $@
-
-TOOLCHAIN_EXTERNAL_INSTALL = $(STAMP_DIR)/ext-toolchain-installed
-
-ifeq ($(BR2_BFIN_INSTALL_FDPIC_SHARED),y)
-TOOLCHAIN_EXTERNAL_INSTALL += $(STAMP_DIR)/ext-toolchain-bfin-fdpic-shared-installed
-endif
-
-ifeq ($(BR2_BFIN_INSTALL_FLAT_SHARED),y)
-TOOLCHAIN_EXTERNAL_INSTALL += $(STAMP_DIR)/ext-toolchain-bfin-shared-flat-installed
+	fi
+endef
 endif
 
 # Build toolchain wrapper for preprocessor, C and C++ compiler and setup
 # symlinks for everything else. Skip gdb symlink when we are building our
 # own gdb to prevent two gdb's in output/host/usr/bin.
-$(HOST_DIR)/usr/bin/ext-toolchain-wrapper: $(TOOLCHAIN_EXTERNAL_INSTALL)
+define TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER
 	$(Q)$(call MESSAGE,"Building ext-toolchain wrapper")
 	mkdir -p $(HOST_DIR)/usr/bin; cd $(HOST_DIR)/usr/bin; \
 	for i in $(TOOLCHAIN_EXTERNAL_CROSS)*; do \
 		base=$${i##*/}; \
 		case "$$base" in \
 		*cc|*cc-*|*++|*++-*|*cpp) \
-			ln -sf $(@F) $$base; \
+			ln -sf ext-toolchain-wrapper $$base; \
 			;; \
 		*gdb|*gdbtui) \
 			if test "$(BR2_PACKAGE_HOST_GDB)" != "y"; then \
@@ -556,11 +533,18 @@ $(HOST_DIR)/usr/bin/ext-toolchain-wrapper: $(TOOLCHAIN_EXTERNAL_INSTALL)
 	# We use --hash-style=both to increase the compatibility of
 	# the generated binary with older platforms
 	$(HOSTCC) $(HOST_CFLAGS) $(TOOLCHAIN_EXTERNAL_WRAPPER_ARGS) -s -Wl,--hash-style=both \
-		toolchain/toolchain-external/ext-toolchain-wrapper.c -o $@
+		toolchain/toolchain-external/ext-toolchain-wrapper.c \
+		-o $(HOST_DIR)/usr/bin/ext-toolchain-wrapper
+endef
 
-toolchain-external: dependencies $(HOST_DIR)/usr/bin/ext-toolchain-wrapper
+# Even though we're installing things in both the staging, the host
+# and the target directory, we do everything within the
+# install-staging step, arbitrarily.
+define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
+	$(TOOLCHAIN_EXTERNAL_INSTALL_CORE)
+	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC)
+	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)
+	$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
+endef
 
-ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
-# download ext toolchain if so configured
-toolchain-external-source: $(addprefix $(DL_DIR)/,$(TOOLCHAIN_EXTERNAL_SOURCE))
-endif
+$(eval $(generic-package))
-- 
1.8.1.2

[Buildroot] [PATCHv3 07/20] dependencies: remove useless targets

[Thomas Petazzoni]

The dependencies-source, dependencies-clean and dependencies-dirclean
targets are not needed, as long as 'dependencies' is not used in the
<pkg>_DEPENDENCIES of a package.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 support/dependencies/dependencies.mk | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
index 97f85e6..155a909 100644
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -33,14 +33,6 @@ dependencies: HOSTCC=$(HOSTCC_NOCCACHE)
 dependencies: HOSTCXX=$(HOSTCXX_NOCCACHE)
 dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ)
 
-dependencies-source:
-
-dependencies-clean:
-	rm -f $(SSTRIP_TARGET)
-
-dependencies-dirclean:
-	true
-
 ################################################################################
 #
 # Toplevel Makefile options
-- 
1.8.1.2

[Buildroot] [PATCHv3 08/20] toolchain-external: conditionalize the installation of libraries

[Thomas Petazzoni]

The external toolchain code makes the assumption that all C libraries
have a ld*.so, libc.so, libcrypt.so, libdl.so, libgcc_s.so, libm.so,
libnsl.so, libresolv.so, libutil.so, and when thread support is
enabled, libpthread.so, etc.

However, this is not the case with the musl C library, which
integrates all the functionalities in a single libc.so file. In
preparation of the support of the musl library, we make the current
value of LIB_EXTERNAL_LIBS conditional to glibc or uClibc.

The addition of additional libraries through
BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS is kept outside the condition, at
the end.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/toolchain-external.mk | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index 85fc341..aaf4994 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -54,8 +54,16 @@
 #  $(HOST_DIR)/usr/bin like for the internal toolchains, and the rest
 #  of Buildroot is handled identical for the 2 toolchain types.
 
-LIB_EXTERNAL_LIBS=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
-LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
+LIB_EXTERNAL_LIBS+=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
+ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
+LIB_EXTERNAL_LIBS+=libpthread.so
+ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
+LIB_EXTERNAL_LIBS+=libthread_db.so
+endif # gdbserver
+endif # ! no threads
+endif
+
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
 LIB_EXTERNAL_LIBS+=libnss_files.so libnss_dns.so
 endif
@@ -64,12 +72,7 @@ ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
 USR_LIB_EXTERNAL_LIBS+=libstdc++.so
 endif
 
-ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
-LIB_EXTERNAL_LIBS+=libpthread.so
-ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
-LIB_EXTERNAL_LIBS+=libthread_db.so
-endif # gdbserver
-endif # ! no threads
+LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
 
 # Details about sysroot directory selection.
 #
-- 
1.8.1.2

[Buildroot] [PATCHv3 09/20] toolchain: modify the wildcard logic for shared libraries copying

[Thomas Petazzoni]

Until now, the copy_toolchain_lib_root function took as argument the
base name of a library (e.g: libm.so), and was assuming that the usual
scheme libm.so.<x> being a symbolic link to the real library was used.

However, with musl based toolchains, the C library is named libc.so
directly, with no symbolic link at all. Therefore, this commit changes
the copy_toolchain_lib_root to move the responsibility of using a
wildcard or not after the library name the caller's responsibility.

So, all the existing LIB_EXTERNAL_LIBS values are modified to have a
.* at the end, so that the behavior is effectively unchanged.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/glibc/glibc.mk                             |  8 ++++----
 toolchain/helpers.mk                               |  2 +-
 toolchain/toolchain-external/toolchain-external.mk | 10 +++++-----
 3 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
index 4d89caf..0dab492 100644
--- a/package/glibc/glibc.mk
+++ b/package/glibc/glibc.mk
@@ -112,12 +112,12 @@ endef
 #
 
 GLIBC_LIBS_LIB = \
-	ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so	   \
-	libnsl.so libpthread.so libresolv.so librt.so libutil.so   \
-	libnss_files.so libnss_dns.so
+	ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.*        \
+	libnsl.so.* libpthread.so.* libresolv.so.* librt.so.* libutil.so.*   \
+	libnss_files.so.* libnss_dns.so.*
 
 ifeq ($(BR2_PACKAGE_GDB_SERVER),y)
-GLIBC_LIBS_LIB += libthread_db.so
+GLIBC_LIBS_LIB += libthread_db.so.*
 endif
 
 define GLIBC_INSTALL_TARGET_CMDS
diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 27b6710..37f2cf9 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -54,7 +54,7 @@ copy_toolchain_lib_root = \
 		$${ARCH_SYSROOT_DIR}/$${ARCH_LIB_DIR} \
 		$${ARCH_SYSROOT_DIR}/usr/$${ARCH_LIB_DIR} \
 		$${SUPPORT_LIB_DIR} ; do \
-		LIBSPATH=`find $${dir} -maxdepth 1 -name "$${LIB}.*" 2>/dev/null` ; \
+		LIBSPATH=`find $${dir} -maxdepth 1 -name "$${LIB}" 2>/dev/null` ; \
 		if test -n "$${LIBSPATH}" ; then \
 			break ; \
 		fi \
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index aaf4994..2bfe64a 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -55,21 +55,21 @@
 #  of Buildroot is handled identical for the 2 toolchain types.
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
-LIB_EXTERNAL_LIBS+=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
+LIB_EXTERNAL_LIBS+=ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
-LIB_EXTERNAL_LIBS+=libpthread.so
+LIB_EXTERNAL_LIBS+=libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
-LIB_EXTERNAL_LIBS+=libthread_db.so
+LIB_EXTERNAL_LIBS+=libthread_db.so.*
 endif # gdbserver
 endif # ! no threads
 endif
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
-LIB_EXTERNAL_LIBS+=libnss_files.so libnss_dns.so
+LIB_EXTERNAL_LIBS+=libnss_files.so.* libnss_dns.so.*
 endif
 
 ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
-USR_LIB_EXTERNAL_LIBS+=libstdc++.so
+USR_LIB_EXTERNAL_LIBS+=libstdc++.so.*
 endif
 
 LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
-- 
1.8.1.2

[Buildroot] [PATCHv3 10/20] toolchain: fix the helpers.mk comment

[Thomas Petazzoni]

This commit fixes various typos and mistakes in the comment at the top
of the helpers.mk file:

 * usefull -> useful

 * The optional stripping of libraries no longer exists, so there's no
   reason to mention it.

 * Indicate that the copy_toolchain_lib_root function is also used by
   the glibc package, not only by the external toolchain logic.

 * Separate more clearly the top comment introducing the entire file,
   from the comment introducing the first function.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/helpers.mk | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 37f2cf9..d9051ae 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -1,9 +1,11 @@
-# This Makefile fragment declares helper functions, usefull to handle
+# This Makefile fragment declares helper functions, useful to handle
 # non- buildroot-built toolchains, eg. purely external toolchains.
-#
-# Copy a toolchain library and its symbolic links from the sysroot
-# directory to the target directory. Also optionaly strips the
-# library.
+
+# The copy_toolchain_lib_root function copies a toolchain library and
+# its symbolic links from the sysroot directory to the target
+# directory. Note that this function is used both the external
+# toolchain logic, and the glibc package, so care must be taken when
+# changing this function.
 #
 # Most toolchains (CodeSourcery ones) have their libraries either in
 # /lib or /usr/lib relative to their ARCH_SYSROOT_DIR, so we search
-- 
1.8.1.2

[Buildroot] [PATCHv3 11/20] toolchain: do not check largefile, wchar, IPv6 and locale for glibc toolchains

[Thomas Petazzoni]

The check_glibc function contained checks to verify that the user had
properly enabled the largefile, IPv6, locale and wchar options, to
match how glibc is configured. This was useful when the support for
glibc external toolchains was introduced, but since then, we added the
BR2_TOOLCHAIN_USES_GLIBC symbol that automatically enables largefile,
IPv6, locale and wchar support when a glibc or eglibc external
toolchain is used. Therefore, many of the check_glibc checks are
useless now, so we can remove these.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/helpers.mk | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index d9051ae..b530a02 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -206,11 +206,7 @@ check_glibc = \
 		echo "Incorrect selection of the C library"; \
 		exit -1; \
 	fi; \
-	$(call check_glibc_feature,BR2_LARGEFILE,Large file support) ;\
-	$(call check_glibc_feature,BR2_INET_IPV6,IPv6 support) ;\
-	$(call check_glibc_feature,BR2_ENABLE_LOCALE,Locale support) ;\
 	$(call check_glibc_feature,BR2_USE_MMU,MMU support) ;\
-	$(call check_glibc_feature,BR2_USE_WCHAR,Wide char support) ;\
 	$(call check_glibc_rpc_feature,$${SYSROOT_DIR})
 
 #
-- 
1.8.1.2

[Buildroot] [PATCHv3 12/20] toolchain-external: add support for musl C library

[Thomas Petazzoni]

This commit adds support for external toolchains based on the musl C
library, as available from http://www.musl-libc.org.

Note that the pre-built musl toolchains available from
http://musl.codu.org/ are not working for the moment, since they lack
sysroot support. However, this problem has been reported to the
maintainer, who has already added sysroot support in his scripts at
https://bitbucket.org/GregorR/musl-cross, and therefore the next
version of the pre-built toolchains should work with Buildroot
out-of-the-box. In the mean time, the musl-cross script must be used
to build the toolchain.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/Config.in                                |  9 +++++++++
 toolchain/helpers.mk                               | 11 +++++++++++
 toolchain/toolchain-external/Config.in             | 12 ++++++++++++
 toolchain/toolchain-external/toolchain-external.mk | 18 +++++++++++++++++-
 4 files changed, 49 insertions(+), 1 deletion(-)

diff --git a/toolchain/Config.in b/toolchain/Config.in
index 0f9a7dd..3980d79 100644
--- a/toolchain/Config.in
+++ b/toolchain/Config.in
@@ -15,6 +15,15 @@ config BR2_TOOLCHAIN_USES_GLIBC
 config BR2_TOOLCHAIN_USES_UCLIBC
 	bool
 
+config BR2_TOOLCHAIN_USES_MUSL
+	bool
+	select BR2_LARGEFILE
+	select BR2_INET_IPV6
+	select BR2_USE_WCHAR
+	select BR2_ENABLE_LOCALE
+	select BR2_TOOLCHAIN_HAS_THREADS
+	select BR2_TOOLCHAIN_HAS_THREADS_DEBUG
+
 choice
 	prompt "Toolchain type"
 	help
diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index b530a02..dfeef76 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -210,6 +210,17 @@ check_glibc = \
 	$(call check_glibc_rpc_feature,$${SYSROOT_DIR})
 
 #
+# Check that the selected C library is really musl
+#
+# $1: sysroot directory
+check_musl = \
+	SYSROOT_DIR="$(strip $1)"; \
+	if test ! -f $${SYSROOT_DIR}/lib/libc.so -o -e $${SYSROOT_DIR}/lib/libm.so ; then \
+		echo "Incorrect selection of the C library" ; \
+		exit -1; \
+	fi
+
+#
 # Check the conformity of Buildroot configuration with regard to the
 # uClibc configuration of the external toolchain, for a particular
 # feature.
diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
index c53577e..89c799e 100644
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -860,6 +860,10 @@ config BR2_TOOLCHAIN_EXTERNAL_UCLIBC
 	bool
 	select BR2_TOOLCHAIN_USES_UCLIBC
 
+config BR2_TOOLCHAIN_EXTERNAL_MUSL
+	bool
+	select BR2_TOOLCHAIN_USES_MUSL
+
 if BR2_TOOLCHAIN_EXTERNAL_CUSTOM
 
 choice
@@ -882,6 +886,13 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC
 	  eglibc configured to exclude key features may cause build failures to
 	  some packages.
 
+config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
+	bool "musl"
+	select BR2_TOOLCHAIN_EXTERNAL_MUSL
+	help
+	  Select this option if your external toolchain uses the
+	  'musl' C library, available from http://www.musl-libc.org/.
+
 endchoice
 
 if BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
@@ -952,6 +963,7 @@ endif # BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
 config BR2_TOOLCHAIN_EXTERNAL_INET_RPC
 	bool "Toolchain has RPC support?"
 	select BR2_TOOLCHAIN_HAS_NATIVE_RPC
+	depends on !BR2_TOOLCHAIN_EXTERNAL_MUSL
 	default y if BR2_TOOLCHAIN_EXTERNAL_GLIBC
 	help
 	  Select this option if your external toolchain supports
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index 2bfe64a..b2266b6 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -68,6 +68,10 @@ ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
 LIB_EXTERNAL_LIBS+=libnss_files.so.* libnss_dns.so.*
 endif
 
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y)
+LIB_EXTERNAL_LIBS += libc.so libgcc_s.so.*
+endif
+
 ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
 USR_LIB_EXTERNAL_LIBS+=libstdc++.so.*
 endif
@@ -353,7 +357,7 @@ endif
 define TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS
 	$(Q)$(call check_cross_compiler_exists,$(TOOLCHAIN_EXTERNAL_CC))
 	$(Q)LIBC_A_LOCATION=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a)` ; \
-	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:usr/lib(32|64)?/(.*/)?libc\.a::'` ; \
+	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:(usr/)?lib(32|64)?/(.*/)?libc\.a::'` ; \
 	if test -z "$${SYSROOT_DIR}" ; then \
 		@echo "External toolchain doesn't support --sysroot. Cannot use." ; \
 		exit 1 ; \
@@ -368,11 +372,23 @@ define TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS
 	fi ; \
 	if test "$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC)" = "y" ; then \
 		$(call check_uclibc,$${SYSROOT_DIR}) ; \
+	elif test "$(BR2_TOOLCHAIN_EXTERNAL_MUSL)" = "y" ; then \
+		$(call check_musl,$${SYSROOT_DIR}) ; \
 	else \
 		$(call check_glibc,$${SYSROOT_DIR}) ; \
 	fi
 endef
 
+# With the musl C library, the libc.so library directly plays the role
+# of the dynamic library loader. We just need to create a symbolic
+# link to libc.so with the appropriate name.
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y)
+define TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
+	ln -sf libc.so $(TARGET_DIR)/lib/ld-musl-$(ARCH).so.1
+endef
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
+endif
+
 # Integration of the toolchain into Buildroot: find the main sysroot
 # and the variant-specific sysroot, then copy the needed libraries to
 # the $(TARGET_DIR) and copy the whole sysroot (libraries and headers)
-- 
1.8.1.2

[Buildroot] [PATCHv3 13/20] toolchain-external: improve help text of some options

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/Config.in | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
index 89c799e..67f4fc7 100644
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -876,15 +876,22 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
 	# For the time being, we assume that all custom external
 	# toolchains have shadow password support.
 	select BR2_TOOLCHAIN_HAS_SHADOW_PASSWORDS
+	help
+	  Select this option if your external toolchain uses the
+	  uClibc C library (available from http://www.uclibc.org/).
 
 config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC
 	bool "glibc/eglibc"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
 	help
-	  Note: eglibc is a variant of glibc that (among other things) can be
-	  configured to exclude some of its features. Using a toolchain with
-	  eglibc configured to exclude key features may cause build failures to
-	  some packages.
+	  Select this option if your external toolchain uses the GNU C
+	  library (available from https://www.gnu.org/software/libc/)
+	  or its variant the eglibc library (http://www.eglibc.org/).
+
+	  Note: eglibc is a variant of glibc that (among other things)
+	  can be configured to exclude some of its features. Using a
+	  toolchain with eglibc configured to exclude key features may
+	  cause build failures to some packages.
 
 config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
 	bool "musl"
-- 
1.8.1.2

[Buildroot] [PATCHv3 14/20] busybox: add patches to fix build with the musl C library

[Thomas Petazzoni]

Those patches have been submitted upstream.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 ...01-libbb-use-poll.h-instead-of-sys-poll.h.patch | 32 +++++++++++++++++++
 ...onfig-include-linux-if_slip.h-instead-of-.patch | 33 ++++++++++++++++++++
 ...libiproute-use-linux-if_packet.h-instead-.patch | 36 ++++++++++++++++++++++
 3 files changed, 101 insertions(+)
 create mode 100644 package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
 create mode 100644 package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
 create mode 100644 package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch

diff --git a/package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch b/package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
new file mode 100644
index 0000000..089d858
--- /dev/null
+++ b/package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
@@ -0,0 +1,32 @@
+From 676452a92d1fc4f85e8d89930d7729197c031026 Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+Date: Sat, 5 Oct 2013 15:51:56 +0200
+Subject: [PATCH 1/3] libbb: use <poll.h> instead of <sys/poll.h>
+
+As the pol() manpage recommends, include the <poll.h> header instead
+of <sys/poll.h>. This allows to get rid of gazillions of warnings when
+building Busybox against the musl C library, which prints a warning
+when the internal header <sys/poll.h> is included directly instead of
+the <poll.h> header.
+
+Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ include/libbb.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/include/libbb.h b/include/libbb.h
+index e520060..f03f454 100644
+--- a/include/libbb.h
++++ b/include/libbb.h
+@@ -37,7 +37,7 @@
+ #include <libgen.h> /* dirname,basename */
+ #undef basename
+ #define basename dont_use_basename
+-#include <sys/poll.h>
++#include <poll.h>
+ #include <sys/ioctl.h>
+ #include <sys/mman.h>
+ #include <sys/socket.h>
+-- 
+1.8.1.2
+
diff --git a/package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch b/package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
new file mode 100644
index 0000000..ed35719
--- /dev/null
+++ b/package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
@@ -0,0 +1,33 @@
+From 5eae213031a3ef88fe8d9d0a387f44264ea635cc Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+Date: Sat, 5 Oct 2013 15:53:16 +0200
+Subject: [PATCH 2/3] network/ifconfig: include <linux/if_slip.h> instead of
+ <net/if_slip.h>
+
+The musl C library doesn't provide the <net/if_slip.h> since userspace
+applications can just as well use the <linux/if_slip.h> kernel header.
+
+This commit fixes the build of the ifconfig applet with the musl C
+library.
+
+Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ networking/ifconfig.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/networking/ifconfig.c b/networking/ifconfig.c
+index 782374b..c90ed6b 100644
+--- a/networking/ifconfig.c
++++ b/networking/ifconfig.c
+@@ -56,7 +56,7 @@
+ #endif
+ 
+ #if ENABLE_FEATURE_IFCONFIG_SLIP
+-# include <net/if_slip.h>
++# include <linux/if_slip.h>
+ #endif
+ 
+ /* I don't know if this is needed for busybox or not.  Anyone? */
+-- 
+1.8.1.2
+
diff --git a/package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch b/package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch
new file mode 100644
index 0000000..7b74656
--- /dev/null
+++ b/package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch
@@ -0,0 +1,36 @@
+From d4fec31889ad660a58dab633c511221feb66e817 Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+Date: Sat, 5 Oct 2013 15:55:06 +0200
+Subject: [PATCH 3/3] networking/libiproute: use <linux/if_packet.h> instead of
+ <net/if_packet.h>
+
+The musl C library doesn't provide the <net/if_packet.h> since the
+corresponding kernel headers <linux/if_packet.h> already provides the
+necessary definitions. Replacing <net/if_packet.h> by
+<linux/if_packet.h> also removes the need to include
+<netpacket/packet.h>
+
+This commit fixes the build of iplink with the musl C library.
+
+Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ networking/libiproute/iplink.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/networking/libiproute/iplink.c b/networking/libiproute/iplink.c
+index bad2017..32ccb1c 100644
+--- a/networking/libiproute/iplink.c
++++ b/networking/libiproute/iplink.c
+@@ -5,8 +5,7 @@
+  * Licensed under GPLv2 or later, see file LICENSE in this source tree.
+  */
+ #include <net/if.h>
+-#include <net/if_packet.h>
+-#include <netpacket/packet.h>
++#include <linux/if_packet.h>
+ #include <netinet/if_ether.h>
+ 
+ #include "ip_common.h"  /* #include "libbb.h" is inside */
+-- 
+1.8.1.2
+
-- 
1.8.1.2

[Buildroot] [PATCHv3 15/20] toolchain-external: update Linaro ARM toolchain

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/Config.in             | 20 ++++++++++----------
 toolchain/toolchain-external/toolchain-external.mk |  6 +++---
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
index 67f4fc7..ecfe17c 100644
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -3,8 +3,8 @@ if BR2_TOOLCHAIN_EXTERNAL
 choice
 	prompt "Toolchain"
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
-	bool "Linaro 2013.08"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09
+	bool "Linaro 2013.09"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -15,7 +15,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.08 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.09 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -24,8 +24,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 
 	  To use this toolchain, you must disable soft float usage.
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
-	bool "Linaro 2013.07"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
+	bool "Linaro 2013.08"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -36,7 +36,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.07 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.08 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -45,8 +45,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
 
 	  To use this toolchain, you must disable soft float usage.
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
-	bool "Linaro 2013.06"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
+	bool "Linaro 2013.07"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -57,7 +57,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.06 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.07 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -816,9 +816,9 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX
 
 config BR2_TOOLCHAIN_EXTERNAL_PREFIX
 	string
+	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09
 	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
-	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201109
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201203
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201305
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index b2266b6..e38ee48 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -239,15 +239,15 @@ define TOOLCHAIN_EXTERNAL_FIXUP_CMDS
 	mv $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/armv5te/* $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/
 	rm -rf $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/
 endef
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06),y)
-TOOLCHAIN_EXTERNAL_SITE = https://releases.linaro.org/13.06/components/toolchain/binaries/
-TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.06_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.07-1_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux.tar.xz
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09),y)
+TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
+TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS201203),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/mips-linux-gnu/
 TOOLCHAIN_EXTERNAL_SOURCE = mips-2012.03-63-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2
-- 
1.8.1.2

[Buildroot] [PATCHv3 16/20] toolchain-external: update Linaro AArch64 toolchain

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/Config.in             | 14 +++++++-------
 toolchain/toolchain-external/toolchain-external.mk |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
index ecfe17c..0d42041 100644
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -730,8 +730,8 @@ config BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_V2
 	  Toolchain for the Microblaze architecture, from
 	  http://wiki.xilinx.com/mb-gnu-tools
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
-	bool "Linaro AArch64 13.08"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09
+	bool "Linaro AArch64 13.09"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -742,8 +742,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
 	  Toolchain for the AArch64 architecture, from
 	  http://www.linaro.org/engineering/armv8/
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
-	bool "Linaro AArch64 13.07"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
+	bool "Linaro AArch64 13.08"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -754,8 +754,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
 	  Toolchain for the AArch64 architecture, from
 	  http://www.linaro.org/engineering/armv8/
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06
-	bool "Linaro AArch64 13.04"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
+	bool "Linaro AArch64 13.07"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -824,9 +824,9 @@ config BR2_TOOLCHAIN_EXTERNAL_PREFIX
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201305
 	default "arm-arago-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV7A_201109
 	default "arm-arago-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV5TE_201109
+	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09
 	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
 	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
-	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06
 	default "microblazeel-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEEL_14_3
 	default "microblazeel-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEEL_V2
 	default "microblaze-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_14_3
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index e38ee48..73a761b 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -310,15 +310,15 @@ TOOLCHAIN_EXTERNAL_SOURCE = lin32-microblaze-unknown-linux-gnu_14.3_early.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_V2),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sources.buildroot.net/
 TOOLCHAIN_EXTERNAL_SOURCE = microblaze-unknown-linux-gnu.tgz
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06),y)
-TOOLCHAIN_EXTERNAL_SITE = https://releases.linaro.org/13.06/components/toolchain/binaries/
-TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.06_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.07-1_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.08_linux.tar.xz
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09),y)
+TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
+TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.09_linux.tar.xz
 else
 # Custom toolchain
 TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
-- 
1.8.1.2

[Buildroot] [PATCHv3 17/20] toolchain-external: improve target library copy logic

[Thomas Petazzoni]

The copy_toolchain_lib_root function is responsible for copying a
given library (and its symbolic link) to the target filesystem. To do
so, it looks for the library in various locations, and then iterates
over the symbolic link all the way to the library, copying them as
needed to the target filesystem.

However, the latest Linaro toolchains bring an interesting use case:
the lib/ directory in the toolchain is organized as follows:

 - ld-linux.so.3 -> arm-linux-gnueabi/ld-2.17...so
 - ld-linux-armhf.so.3 -> arm-linux-gnueabihf/ld-2.17...so
 - arm-linux-gnueabi/
   - all ARMv4T soft float libraries
 - arm-linux-gnueabihf/
   - all ARMv7 hard float libraries

In order to match what we do with all other toolchains, we want all
those libraries and symbolic links to be copied directly under
$(TARGET_DIR)/lib. This commit does that by adjusting the copy logic.

This is part of the fix for bug #6452 (eglibc from Linaro 2013.07 not
copied to target correctly).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/helpers.mk | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index dfeef76..f7e70cd 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -61,21 +61,24 @@ copy_toolchain_lib_root = \
 			break ; \
 		fi \
 	done ; \
+	mkdir -p $(TARGET_DIR)/$${DESTDIR}; \
 	for LIBPATH in $${LIBSPATH} ; do \
-		LIBNAME=`basename $${LIBPATH}`; \
-		LIBDIR=`dirname $${LIBPATH}` ; \
-		while test \! -z "$${LIBNAME}" ; do \
-			LIBPATH=$${LIBDIR}/$${LIBNAME} ; \
+		while true ; do \
+			LIBNAME=`basename $${LIBPATH}`; \
+			LIBDIR=`dirname $${LIBPATH}` ; \
+			LINKTARGET=`readlink $${LIBPATH}` ; \
 			rm -fr $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
-			mkdir -p $(TARGET_DIR)/$${DESTDIR}; \
 			if test -h $${LIBPATH} ; then \
-				cp -d $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/; \
+				ln -sf `basename $${LINKTARGET}` $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME} ; \
 			elif test -f $${LIBPATH}; then \
 				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 			else \
 				exit -1; \
 			fi; \
-			LIBNAME="`readlink $${LIBPATH}`"; \
+			if test -z "$${LINKTARGET}" ; then \
+				break ; \
+			fi ; \
+			LIBPATH="`readlink -f $${LIBPATH}`"; \
 		done; \
 	done; \
  \
-- 
1.8.1.2

[Buildroot] [PATCHv3 18/20] toolchain-external: fix Linaro ARM toolchain support

[Thomas Petazzoni]

This commit fixes bug #6452 (eglibc from Linaro 2013.07 not copied to
target correctly) by:

 * Copying only the relevant library loader to the target on ARMhf
   (i.e ld-linux-armhf.so and not ld.so*). This is needed since Linaro
   toolchains provide two library loaders, one ARMv7 hf, and one ARMv4
   soft-float.

 * Making sure a $(TARGET_DIR)/lib/arm-linux-gnueabihf/ symbolic link
   to $(TARGET_DIR)/lib/ exists, since the dynamic loader of Linaro
   toolchains expects libraries to be found in
   $(TARGET_DIR)/lib/arm-linux-gnueabihf/.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/toolchain-external.mk | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index 73a761b..b5b1ce7 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -55,7 +55,12 @@
 #  of Buildroot is handled identical for the 2 toolchain types.
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
-LIB_EXTERNAL_LIBS+=ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
+LIB_EXTERNAL_LIBS+=libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
+ifeq ($(BR2_ARM_EABIHF),y)
+LIB_EXTERNAL_LIBS+=ld-linux-armhf.so.*
+else
+LIB_EXTERNAL_LIBS+=ld*.so.*
+endif
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
 LIB_EXTERNAL_LIBS+=libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
@@ -216,6 +221,13 @@ TOOLCHAIN_EXTERNAL_CFLAGS += -msoft-float
 TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_SOFTFLOAT=1
 endif
 
+# The Linaro ARMhf toolchain expects the libraries in
+# /lib/arm-linux-gnueabihf, but Buildroot copies them to /lib, so we
+# need to create a symbolic link.
+define TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
+	ln -sf . $(TARGET_DIR)/lib/arm-linux-gnueabihf
+endef
+
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201109),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/arm-none-linux-gnueabi/
 TOOLCHAIN_EXTERNAL_SOURCE = arm-2011.09-70-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
@@ -242,12 +254,15 @@ endef
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.07-1_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS201203),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/mips-linux-gnu/
 TOOLCHAIN_EXTERNAL_SOURCE = mips-2012.03-63-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2
-- 
1.8.1.2

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Thomas Petazzoni]

Until now, the make source, make external-deps and make legal-info
where relying on enumerating the packages by looking at $(TARGETS)
which contains only the target packages and not the host
packages. Thanks to the TARGET_HOST_DEPS and HOST_DEPS variables, it
was doing a two-level resolution of host package dependencies, but it
was not entirely correct since the dependency chain might be deeper
than that: some packages could be missed.

From an idea of Arnout, this patch introduces in each package
additional targets <pkg>-all-source, <pkg>-all-legal-info and
<pkg>-all-external-deps that executes the 'source', 'legal-info' and
'external-deps' targets for this package and all its dependencies, be
they target or host dependencies.

This provides a much cleaner implementation of this mechanism, which
is also more robust.

In order to achieve this, this patch also separates the "package"
targets from other targets: instead of mixing them both in the global
TARGETS variable, the new PACKAGES variable contains the name of all
packages that are enabled in the configuration, while TARGETS is only
used for additional things to be done (target-finalize, etc.). This
separation is needed so that we don't try to use targets (such as
<foo>-all-external-deps) on things that are not packages. Some further
cleanups in this direction are possible, this commit takes a
relatively minimal approach for now.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile               | 53 +++++++++++++-------------------------------------
 package/pkg-generic.mk | 16 ++++++++++++++-
 2 files changed, 28 insertions(+), 41 deletions(-)

diff --git a/Makefile b/Makefile
index f266e2d..702e7d4 100644
--- a/Makefile
+++ b/Makefile
@@ -238,8 +238,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 
 BASE_TARGETS = toolchain
 
-TARGETS:=
-
 # silent mode requested?
 QUIET:=$(if $(findstring s,$(MAKEFLAGS)),-q)
 
@@ -310,9 +308,6 @@ endif
 include package/Makefile.in
 include support/dependencies/dependencies.mk
 
-# We also need the various per-package makefiles, which also add
-# each selected package to TARGETS if that package was selected
-# in the .config file.
 include toolchain/helpers.mk
 include toolchain/*/*.mk
 
@@ -349,34 +344,11 @@ include fs/common.mk
 
 TARGETS+=target-post-image
 
-TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
-TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
-TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
-
-# host-* dependencies have to be handled specially, as those aren't
-# visible in Kconfig and hence not added to a variable like TARGETS.
-# instead, find all the host-* targets listed in each <PKG>_DEPENDENCIES
-# variable for each enabled target.
-# Notice: this only works for newstyle gentargets/autotargets packages
-TARGETS_HOST_DEPS = $(sort $(filter host-%,$(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS))),\
-		$($(dep)))))
-# Host packages can in turn have their own dependencies. Likewise find
-# all the package names listed in the HOST_<PKG>_DEPENDENCIES for each
-# host package found above. Ideally this should be done recursively until
-# no more packages are found, but that's hard to do in make, so limit to
-# 1 level for now.
-HOST_DEPS = $(sort $(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS_HOST_DEPS))),\
-		$($(dep))))
-HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
-
-TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\
-		$(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS))))
-
-# all targets depend on the crosscompiler and it's prerequisites
-$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
+PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))
+PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))
+PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
 
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
@@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-world: $(BASE_TARGETS) $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
 .PHONY: all world toolchain dirs clean distclean source outputmakefile \
 	legal-info legal-info-prepare legal-info-clean printvars \
-	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
-	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
+	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
+	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
+	$(PACKAGES_EXTERNAL_DEPS) \
 	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
@@ -562,10 +535,10 @@ target-post-image:
 toolchain-eclipse-register:
 	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
 
-source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
+source: dirs $(PACKAGES_SOURCE)
 
 external-deps:
-	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
+	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
 
 legal-info-clean:
 	@rm -fr $(LEGAL_INFO_DIR)
@@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
 	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
 
 legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
-		$(TARGETS_LEGAL_INFO)
+		$(PACKAGES_LEGAL_INFO)
 	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
 	@if [ -r $(LEGAL_WARNINGS) ]; then \
 		cat support/legal-info/README.warnings-header \
@@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
 	@rm -f $(LEGAL_WARNINGS)
 
 show-targets:
-	@echo $(TARGETS)
+	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index a46457c..bd6169c 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -424,6 +424,20 @@ endif
 $(1)-show-depends:
 			@echo $$($(2)_DEPENDENCIES)
 
+$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
+
+$(1)-external-deps:
+ifneq ($$($(2)_SOURCE),)
+			@echo $$($(2)_SOURCE)
+endif
+ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
+			@echo $$($(2)_EXTRA_DOWNLOADS)
+endif
+
+$(1)-all-external-deps:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-external-deps) $(1)-external-deps
+
+$(1)-all-legal-info:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-legal-info) $(1)-legal-info
+
 $(1)-uninstall:		$(1)-configure $$($(2)_TARGET_UNINSTALL)
 
 $(1)-clean:		$(1)-uninstall \
@@ -528,7 +542,7 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 # configuration
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
-TARGETS += $(1)
+PACKAGES += $(1)
 PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
 PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
 PACKAGES_USERS += $$($(2)_USERS)$$(sep)
-- 
1.8.1.2

[Buildroot] [PATCHv3 20/20] package: remove useless SHOW_EXTERNAL_DEPS support

[Thomas Petazzoni]

Now that a package-based 'make external-deps' support has been added,
the DL_MODE=SHOW_EXTERNAL_DEPS mechanism is no longer needed, so some
clean up is done in this commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-download.mk | 44 ++------------------------------------------
 package/pkg-generic.mk  |  5 +----
 2 files changed, 3 insertions(+), 46 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 8e4a1ec..208c83c 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -18,9 +18,8 @@ SCP := $(call qstrip,$(BR2_SCP)) $(QUIET)
 SSH := $(call qstrip,$(BR2_SSH)) $(QUIET)
 LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
-# Default spider mode is 'DOWNLOAD'. Other possible values are 'SOURCE_CHECK'
-# used by the _source-check target and 'SHOW_EXTERNAL_DEPS', used by the
-# external-deps target.
+# Default spider mode is 'DOWNLOAD'. The possible value is
+# 'SOURCE_CHECK' used by the _source-check target.
 DL_MODE=DOWNLOAD
 
 # Override BR2_DL_DIR if shell variable defined
@@ -66,11 +65,6 @@ domainseparator=$(if $(1),$(1),/)
 # The SOURCE_CHECK_* helpers are in charge of simply checking that the source
 # is available for download. This can be used to make sure one will be able
 # to get all the sources needed for one's build configuration.
-#
-# The SHOW_EXTERNAL_DEPS_* helpers simply output to the console the names
-# of the files that will be downloaded, or path and revision of the
-# source repositories, producing a list of all the "external dependencies"
-# of a given build configuration.
 ################################################################################
 
 # Try a shallow clone - but that only works if the version is a ref (tag or
@@ -101,11 +95,6 @@ define SOURCE_CHECK_GIT
   $(GIT) ls-remote --heads $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_GIT
-	echo $($(PKG)_SOURCE)
-endef
-
-
 define DOWNLOAD_BZR
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	$(BZR) export $(DL_DIR)/$($(PKG)_SOURCE) $($(PKG)_SITE) -r $($(PKG)_DL_VERSION)
@@ -115,10 +104,6 @@ define SOURCE_CHECK_BZR
 	$(BZR) ls --quiet $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_BZR
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_CVS
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -134,10 +119,6 @@ define SOURCE_CHECK_CVS
 	$(CVS) -d:pserver:anonymous:@$(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) login
 endef
 
-define SHOW_EXTERNAL_DEPS_CVS
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_SVN
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -151,10 +132,6 @@ define SOURCE_CHECK_SVN
   $(SVN) ls $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SVN
-  echo $($(PKG)_SOURCE)
-endef
-
 # SCP URIs should be of the form scp://[user@]host:filepath
 # Note that filepath is relative to the user's home directory, so you may want
 # to prepend the path with a slash: scp://[user@]host:/absolutepath
@@ -167,11 +144,6 @@ define SOURCE_CHECK_SCP
 	$(SSH) $(call domain,$(1),:) ls '$(call notdomain,$(1),:)' > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SCP
-	echo $(2)
-endef
-
-
 define DOWNLOAD_HG
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -188,10 +160,6 @@ define SOURCE_CHECK_HG
   $(HG) incoming --force -l1 $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_HG
-  echo $($(PKG)_SOURCE)
-endef
-
 # Download a file using wget. Only download the file if it doesn't
 # already exist in the download directory. If the download fails,
 # remove the file (because wget -O creates a 0-byte file even if the
@@ -209,10 +177,6 @@ define SOURCE_CHECK_WGET
   $(WGET) --spider '$(call qstrip,$(1))'
 endef
 
-define SHOW_EXTERNAL_DEPS_WGET
-  echo $(2)
-endef
-
 define DOWNLOAD_LOCALFILES
 	test -e $(DL_DIR)/$(2) || \
 		$(LOCALFILES) $(call stripurischeme,$(call qstrip,$(1))) $(DL_DIR)
@@ -222,10 +186,6 @@ define SOURCE_CHECK_LOCALFILES
   test -e $(call stripurischeme,$(call qstrip,$(1)))
 endef
 
-define SHOW_EXTERNAL_DEPS_LOCALFILES
-  echo $(2)
-endef
-
 ################################################################################
 # DOWNLOAD -- Download helper. Will try to download source from:
 # 1) BR2_PRIMARY_SITE if enabled
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index bd6169c..1e09efb 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -71,13 +71,10 @@ $(BUILD_DIR)/%/.stamp_rsynced:
 	rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)
 	$(Q)touch $@
 
-# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
-# packages
+# Handle the SOURCE_CHECK case for rsynced packages
 $(BUILD_DIR)/%/.stamp_rsync_sourced:
 ifeq ($(DL_MODE),SOURCE_CHECK)
 	test -d $(SRCDIR)
-else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS)
-	echo "file://$(SRCDIR)"
 else
 	@true # Nothing to do to source a local package
 endif
-- 
1.8.1.2

[Buildroot] [PATCHv3 01/20] glibc: both eglibc and glibc need host-gawk

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Until now, only the eglibc build was pulling host-gawk as a
 Thomas> dependency, but after more testing, it turns out that the glibc build
 Thomas> also requires host-gawk.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 02/20] glibc: fix glibc build by creating an empty gnu/stubs.h

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The stubs.h header is not installed by install-headers, but is needed
 Thomas> for the gcc build. An empty stubs.h will work, as explained in
 Thomas> http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick is
 Thomas> used by Crosstool-NG.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Converting the external toolchain logic into a package raises a very
 Thomas> special use case that wasn't handled by the package infrastructure:
 Thomas> the Blackfin toolchain is delivered as two tarballs instead of
 Thomas> one. Unfortunately <pkg>_SOURCE only allows to pass one tarball name.

 Thomas> However, we really want both tarballs to be known by the package
 Thomas> infrastructure, so that the normal 'source' and 'external-deps'
 Thomas> mechanism work fine.

 Thomas> In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable,
 Thomas> which allows a package to list other stuff it would like to see
 Thomas> downloaded, but that are otherwise not used by the package
 Thomas> infrastructure itself: it is up to the package to do it by itself.

I think it should be possible to support a space seperated list of
sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without
breaking backwards compatibility, but I won't delay this series because
of it - So committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Tue, 08 Oct 2013 23:32:16 +0200, Peter Korsgaard wrote:

>  Thomas> In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable,
>  Thomas> which allows a package to list other stuff it would like to see
>  Thomas> downloaded, but that are otherwise not used by the package
>  Thomas> infrastructure itself: it is up to the package to do it by itself.
> 
> I think it should be possible to support a space seperated list of
> sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without
> breaking backwards compatibility, but I won't delay this series because
> of it - So committed, thanks.

This has been discussed in a prior version of this patch set. The
problem is that <pkg>_SOURCE is not only about downloading, but also
about extracting.

So, what would <pkg>_SOURCE do? Download all the files listed, but
extract only the first one? This seems like a weird semantic. I think I
prefer the following clear semantic:

 <pkg>_SOURCE is one file, that gets downloaded and then extracted into
 the source directory of the package.

 <pkg>_PATCH is a list of files or URLs, that get downloaded and apply
 as patches into the source directory

 <pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded, and
 nothing else. The package is responsible for doing something with them.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Fabio Porcedda]

Hi Thomas,

On Tue, Oct 8, 2013 at 8:17 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> [...]
>  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>
> -toolchain: prepare dirs dependencies $(BASE_TARGETS)
> -
> -world: toolchain $(TARGETS_ALL)
> +world: $(BASE_TARGETS) $(TARGETS_ALL)

Why do you change that? It seems to me better to use the "toolchain"
target as dependency instead of $(BASE_TARGETS)
IMHO  $(BASE_TARGETS) is an implementation detail of the new toolchain
virtual package so it's better to use inside the new toolchain virtual
package.

> [...]

Best regards
-- 
Fabio Porcedda

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Thomas Petazzoni]

Dear Fabio Porcedda,

On Wed, 9 Oct 2013 10:26:56 +0200, Fabio Porcedda wrote:
> Hi Thomas,
> 
> On Tue, Oct 8, 2013 at 8:17 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > [...]
> >  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
> >
> > -toolchain: prepare dirs dependencies $(BASE_TARGETS)
> > -
> > -world: toolchain $(TARGETS_ALL)
> > +world: $(BASE_TARGETS) $(TARGETS_ALL)
> 
> Why do you change that? It seems to me better to use the "toolchain"
> target as dependency instead of $(BASE_TARGETS)
> IMHO  $(BASE_TARGETS) is an implementation detail of the new toolchain
> virtual package so it's better to use inside the new toolchain virtual
> package.

BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure
why you say BASE_TARGETS is an implementation detail of the new
toolchain virtual package. BASE_TARGETS now never has any other value
than just "toolchain".

Of course, we might want to get rid of BASE_TARGETS later on, but for
the moment, BASE_TARGETS is still used in other places, like when
building the list of packages for which make source or make
external-deps should be executed:

PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))

Ultimately, toolchain will no longer need to be part of BASE_TARGETS as
it will be handled just as a normal package dependency (done by your
patches). But I'm trying to do things step by step, progressively.
Therefore, in the end, I believe the world rule will no even contain
"toolchain" and $(TARGETS_ALL) but just $(PACKAGES), but we can't
change the world in one day :-)

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> I think it should be possible to support a space seperated list of
 >> sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without
 >> breaking backwards compatibility, but I won't delay this series because
 >> of it - So committed, thanks.

 Thomas> This has been discussed in a prior version of this patch set. The
 Thomas> problem is that <pkg>_SOURCE is not only about downloading, but also
 Thomas> about extracting.

 Thomas> So, what would <pkg>_SOURCE do? Download all the files listed,
 Thomas> but extract only the first one? This seems like a weird
 Thomas> semantic. I think I prefer the following clear semantic:

 Thomas>  <pkg>_SOURCE is one file, that gets downloaded and then extracted into
 Thomas>  the source directory of the package.

 Thomas>  <pkg>_PATCH is a list of files or URLs, that get downloaded and apply
 Thomas>  as patches into the source directory

 Thomas>  <pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded,
 Thomas>  and nothing else. The package is responsible for doing
 Thomas>  something with them.

Well, I don't feel strongly about it, but no matter what we do packages
with multple sources will be "special" (Luckily they are very uncommon),
so you always have to do something explicit to use (extract) the extra
sources.

But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than
just saying that only the first element in _SOURCE will be automatically
extracted.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This commit introduces a virtual package called 'toolchain', located
 Thomas> in 'toolchain/toolchain', which simply depends on
 Thomas> 'toolchain-buildroot' or 'toolchain-external' depending on the
 Thomas> selected toolchain backend.

[snip]

 Thomas> @@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk
 Thomas>  # We also need the various per-package makefiles, which also add
 Thomas>  # each selected package to TARGETS if that package was selected
 Thomas>  # in the .config file.
 Thomas> -ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
 Thomas> -include toolchain/toolchain-buildroot.mk
 Thomas> -else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
 Thomas> -include toolchain/toolchain-external.mk
 Thomas> -endif
 Thomas> +include toolchain/helpers.mk
 Thomas> +include toolchain/*/*.mk

That's no good. toolchain/*/*.mk doesn't pick up up
toolchain/toolchain-{buildroot,external}.mk:

>>> toolchain undefined Extracting
>>> toolchain undefined Patching toolchain//toolchain
make: *** No rule to make target `toolchain-buildroot', needed by `toolchain-depends'.  Stop.

Did you mean to simply do 'include toolchain/*.mk' instead of those two lines?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Wed, 09 Oct 2013 12:46:08 +0200, Peter Korsgaard wrote:
> >>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:
> 
>  Thomas> This commit introduces a virtual package called 'toolchain', located
>  Thomas> in 'toolchain/toolchain', which simply depends on
>  Thomas> 'toolchain-buildroot' or 'toolchain-external' depending on the
>  Thomas> selected toolchain backend.
> 
> [snip]
> 
>  Thomas> @@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk
>  Thomas>  # We also need the various per-package makefiles, which also add
>  Thomas>  # each selected package to TARGETS if that package was selected
>  Thomas>  # in the .config file.
>  Thomas> -ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
>  Thomas> -include toolchain/toolchain-buildroot.mk
>  Thomas> -else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
>  Thomas> -include toolchain/toolchain-external.mk
>  Thomas> -endif
>  Thomas> +include toolchain/helpers.mk
>  Thomas> +include toolchain/*/*.mk
> 
> That's no good. toolchain/*/*.mk doesn't pick up up
> toolchain/toolchain-{buildroot,external}.mk:
> 
> >>> toolchain undefined Extracting
> >>> toolchain undefined Patching toolchain//toolchain
> make: *** No rule to make target `toolchain-buildroot', needed by `toolchain-depends'.  Stop.
> 
> Did you mean to simply do 'include toolchain/*.mk' instead of those two lines?

No. The toolchain/toolchain-{buildroot,external}.mk file get removed in
PATCH 5 and PATCH 6 respectively. So the problem is just a bisection
problem: I haven't tested the series after each commit.

Do you want me to respin with this bisection issue fixed?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Wed, 09 Oct 2013 11:29:02 +0200, Peter Korsgaard wrote:

>  Thomas> This has been discussed in a prior version of this patch set. The
>  Thomas> problem is that <pkg>_SOURCE is not only about downloading, but also
>  Thomas> about extracting.
> 
>  Thomas> So, what would <pkg>_SOURCE do? Download all the files listed,
>  Thomas> but extract only the first one? This seems like a weird
>  Thomas> semantic. I think I prefer the following clear semantic:
> 
>  Thomas>  <pkg>_SOURCE is one file, that gets downloaded and then extracted into
>  Thomas>  the source directory of the package.
> 
>  Thomas>  <pkg>_PATCH is a list of files or URLs, that get downloaded and apply
>  Thomas>  as patches into the source directory
> 
>  Thomas>  <pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded,
>  Thomas>  and nothing else. The package is responsible for doing
>  Thomas>  something with them.
> 
> Well, I don't feel strongly about it, but no matter what we do packages
> with multple sources will be "special" (Luckily they are very uncommon),
> so you always have to do something explicit to use (extract) the extra
> sources.

Sure.

> But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than
> just saying that only the first element in _SOURCE will be automatically
> extracted.

Well, I find it cleaner, but if you prefer to have it in _SOURCE, I can
certainly modify the patch to extend the semantic of _SOURCE instead.
Let me know what you decide so that I can respin the patch set if
needed.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner
 >> than just saying that only the first element in _SOURCE will be
 >> automatically extracted.

 Thomas> Well, I find it cleaner, but if you prefer to have it in
 Thomas> _SOURCE, I can certainly modify the patch to extend the
 Thomas> semantic of _SOURCE instead.  Let me know what you decide so
 Thomas> that I can respin the patch set if needed.

Just leave it for now. _EXTRA_DOWNLOADS is ok.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> Did you mean to simply do 'include toolchain/*.mk' instead of those
 >> two lines?

 Thomas> No. The toolchain/toolchain-{buildroot,external}.mk file get
 Thomas> removed in PATCH 5 and PATCH 6 respectively. So the problem is
 Thomas> just a bisection problem: I haven't tested the series after
 Thomas> each commit.

Tsk tsk ;)

 Thomas> Do you want me to respin with this bisection issue fixed?

No, lets just leave it as it is - Thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Fabio Porcedda]

On Wed, Oct 9, 2013 at 11:02 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Fabio Porcedda,
>
> On Wed, 9 Oct 2013 10:26:56 +0200, Fabio Porcedda wrote:
>> Hi Thomas,
>>
>> On Tue, Oct 8, 2013 at 8:17 PM, Thomas Petazzoni
>> <thomas.petazzoni@free-electrons.com> wrote:
>> > [...]
>> >  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>> >
>> > -toolchain: prepare dirs dependencies $(BASE_TARGETS)
>> > -
>> > -world: toolchain $(TARGETS_ALL)
>> > +world: $(BASE_TARGETS) $(TARGETS_ALL)
>>
>> Why do you change that? It seems to me better to use the "toolchain"
>> target as dependency instead of $(BASE_TARGETS)
>> IMHO  $(BASE_TARGETS) is an implementation detail of the new toolchain
>> virtual package so it's better to use inside the new toolchain virtual
>> package.
>
> BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure
> why you say BASE_TARGETS is an implementation detail of the new
> toolchain virtual package. BASE_TARGETS now never has any other value
> than just "toolchain".

Sorry, I've missed that part, now I understand better.

> Of course, we might want to get rid of BASE_TARGETS later on, but for
> the moment, BASE_TARGETS is still used in other places, like when
> building the list of packages for which make source or make
> external-deps should be executed:
>
> PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
> PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))

Ok I understand.

> Ultimately, toolchain will no longer need to be part of BASE_TARGETS as
> it will be handled just as a normal package dependency (done by your
> patches). But I'm trying to do things step by step, progressively.
> Therefore, in the end, I believe the world rule will no even contain
> "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),

Well about the world rule in my patches in the end it contain only the
"target-post-image" because it's the last step, but that's another
story.

> but we can't
> change the world in one day :-)

That's for sure :)

Thanks for these toolchains improvements
-- 
Fabio Porcedda

[Buildroot] [PATCHv3 05/20] toolchain-buildroot: convert to the package infrastructure

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This commit converts the toolchain-buildroot logic to the package
 Thomas> infrastructure. The package is fairly simple as it only defines
 Thomas> BUILDROOT_LIBC, and depends on host-gcc-final to get the overall
 Thomas> internal toolchain build logic started.

It's more like getting it completed, but ok - Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 06/20] toolchain-external: convert to the package infrastructure

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This commit converts the 'toolchain-external' logic to the package
 Thomas> infrastructure.

 Thomas> The TOOLCHAIN_EXTERNAL_DIR variable (which points to where the
 Thomas> toolchain is located) is renamed to TOOLCHAIN_EXTERNAL_INSTALL_DIR,
 Thomas> because the former conflicts with the package infrastructure (which
 Thomas> defines the <pkg>_DIR variable for each package as pointing to its
 Thomas> build directory).

 Thomas> The new _EXTRA_DOWNLOADS mechanism is used for Blackfin toolchains.

 Thomas> The extract, configuration and installation steps are converted inside
 Thomas> the <pkg>_EXTRACT_CMDS, <pkg>_CONFIGURE_CMDS and
 Thomas> <pkg>_INSTALL_STAGING_CMDS.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
 Thomas> Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 07/20] dependencies: remove useless targets

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The dependencies-source, dependencies-clean and dependencies-dirclean
 Thomas> targets are not needed, as long as 'dependencies' is not used in the
 Thomas> <pkg>_DEPENDENCIES of a package.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 08/20] toolchain-external: conditionalize the installation of libraries

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The external toolchain code makes the assumption that all C libraries
 Thomas> have a ld*.so, libc.so, libcrypt.so, libdl.so, libgcc_s.so, libm.so,
 Thomas> libnsl.so, libresolv.so, libutil.so, and when thread support is
 Thomas> enabled, libpthread.so, etc.

 Thomas> However, this is not the case with the musl C library, which
 Thomas> integrates all the functionalities in a single libc.so file. In
 Thomas> preparation of the support of the musl library, we make the current
 Thomas> value of LIB_EXTERNAL_LIBS conditional to glibc or uClibc.

 Thomas> The addition of additional libraries through
 Thomas> BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS is kept outside the condition, at
 Thomas> the end.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 09/20] toolchain: modify the wildcard logic for shared libraries copying

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Until now, the copy_toolchain_lib_root function took as argument the
 Thomas> base name of a library (e.g: libm.so), and was assuming that the usual
 Thomas> scheme libm.so.<x> being a symbolic link to the real library was used.

 Thomas> However, with musl based toolchains, the C library is named libc.so
 Thomas> directly, with no symbolic link at all. Therefore, this commit changes
 Thomas> the copy_toolchain_lib_root to move the responsibility of using a
 Thomas> wildcard or not after the library name the caller's responsibility.

 Thomas> So, all the existing LIB_EXTERNAL_LIBS values are modified to have a
 Thomas> .* at the end, so that the behavior is effectively unchanged.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 03/20] package: add a <pkg>_EXTRA_DOWNLOADS variable

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Wed, 09 Oct 2013 14:31:46 +0200, Peter Korsgaard wrote:

>  >> But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner
>  >> than just saying that only the first element in _SOURCE will be
>  >> automatically extracted.
> 
>  Thomas> Well, I find it cleaner, but if you prefer to have it in
>  Thomas> _SOURCE, I can certainly modify the patch to extend the
>  Thomas> semantic of _SOURCE instead.  Let me know what you decide so
>  Thomas> that I can respin the patch set if needed.
> 
> Just leave it for now. _EXTRA_DOWNLOADS is ok.

I don't mind at all changing this, just let me know if I should do it.
You're mail was not clear about what you really wanted to happen. I
must say I don't have a really strong opinion about this, so either way
is fine with me.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 10/20] toolchain: fix the helpers.mk comment

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This commit fixes various typos and mistakes in the comment at the top
 Thomas> of the helpers.mk file:

 Thomas>  * usefull -> useful

 Thomas>  * The optional stripping of libraries no longer exists, so there's no
 Thomas>    reason to mention it.

 Thomas>  * Indicate that the copy_toolchain_lib_root function is also used by
 Thomas>    the glibc package, not only by the external toolchain logic.

 Thomas>  * Separate more clearly the top comment introducing the entire file,
 Thomas>    from the comment introducing the first function.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
 Thomas> ---
 Thomas>  toolchain/helpers.mk | 12 +++++++-----
 Thomas>  1 file changed, 7 insertions(+), 5 deletions(-)

 Thomas> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
 Thomas> index 37f2cf9..d9051ae 100644
 Thomas> --- a/toolchain/helpers.mk
 Thomas> +++ b/toolchain/helpers.mk
 Thomas> @@ -1,9 +1,11 @@
 Thomas> -# This Makefile fragment declares helper functions, usefull to handle
 Thomas> +# This Makefile fragment declares helper functions, useful to handle
 Thomas>  # non- buildroot-built toolchains, eg. purely external toolchains.

This comment is still not quite true.

I've changed it to simply say:

This Makefile fragment declares toolchain related helper functions.

 Thomas> -#
 Thomas> -# Copy a toolchain library and its symbolic links from the sysroot
 Thomas> -# directory to the target directory. Also optionaly strips the
 Thomas> -# library.
 Thomas> +
 Thomas> +# The copy_toolchain_lib_root function copies a toolchain library and
 Thomas> +# its symbolic links from the sysroot directory to the target
 Thomas> +# directory. Note that this function is used both the external

s/both/by both/

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 11/20] toolchain: do not check largefile, wchar, IPv6 and locale for glibc toolchains

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The check_glibc function contained checks to verify that the user had
 Thomas> properly enabled the largefile, IPv6, locale and wchar options, to
 Thomas> match how glibc is configured. This was useful when the support for
 Thomas> glibc external toolchains was introduced, but since then, we added the
 Thomas> BR2_TOOLCHAIN_USES_GLIBC symbol that automatically enables largefile,
 Thomas> IPv6, locale and wchar support when a glibc or eglibc external
 Thomas> toolchain is used. Therefore, many of the check_glibc checks are
 Thomas> useless now, so we can remove these.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Thomas Petazzoni]

Dear Fabio Porcedda,

On Wed, 9 Oct 2013 14:59:07 +0200, Fabio Porcedda wrote:

> > BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure
> > why you say BASE_TARGETS is an implementation detail of the new
> > toolchain virtual package. BASE_TARGETS now never has any other value
> > than just "toolchain".
> 
> Sorry, I've missed that part, now I understand better.

No problem :)

> > Ultimately, toolchain will no longer need to be part of BASE_TARGETS as
> > it will be handled just as a normal package dependency (done by your
> > patches). But I'm trying to do things step by step, progressively.
> > Therefore, in the end, I believe the world rule will no even contain
> > "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),
> 
> Well about the world rule in my patches in the end it contain only the
> "target-post-image" because it's the last step, but that's another
> story.

Yes, indeed. Seen your patches, I think they look good on principle, I
need to test them/have a more detailed look.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 04/20] toolchain: introduce a virtual package

[Fabio Porcedda]

On Wed, Oct 9, 2013 at 3:31 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Fabio Porcedda,
>
> On Wed, 9 Oct 2013 14:59:07 +0200, Fabio Porcedda wrote:
>
>> > BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure
>> > why you say BASE_TARGETS is an implementation detail of the new
>> > toolchain virtual package. BASE_TARGETS now never has any other value
>> > than just "toolchain".
>>
>> Sorry, I've missed that part, now I understand better.
>
> No problem :)
>
>> > Ultimately, toolchain will no longer need to be part of BASE_TARGETS as
>> > it will be handled just as a normal package dependency (done by your
>> > patches). But I'm trying to do things step by step, progressively.
>> > Therefore, in the end, I believe the world rule will no even contain
>> > "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),
>>
>> Well about the world rule in my patches in the end it contain only the
>> "target-post-image" because it's the last step, but that's another
>> story.
>
> Yes, indeed. Seen your patches, I think they look good on principle, I
> need to test them/have a more detailed look.

About those patch set I'll send a new version rebased over your latest changes.

Best regards
-- 
Fabio Porcedda

[Buildroot] [PATCHv3 10/20] toolchain: fix the helpers.mk comment

[Thomas De Schampheleire]

On Tue, Oct 8, 2013 at 8:17 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> This commit fixes various typos and mistakes in the comment at the top
> of the helpers.mk file:
>
>  * usefull -> useful
>
>  * The optional stripping of libraries no longer exists, so there's no
>    reason to mention it.
>
>  * Indicate that the copy_toolchain_lib_root function is also used by
>    the glibc package, not only by the external toolchain logic.
>
>  * Separate more clearly the top comment introducing the entire file,
>    from the comment introducing the first function.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  toolchain/helpers.mk | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index 37f2cf9..d9051ae 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -1,9 +1,11 @@
> -# This Makefile fragment declares helper functions, usefull to handle
> +# This Makefile fragment declares helper functions, useful to handle
>  # non- buildroot-built toolchains, eg. purely external toolchains.
> -#
> -# Copy a toolchain library and its symbolic links from the sysroot
> -# directory to the target directory. Also optionaly strips the
> -# library.
> +
> +# The copy_toolchain_lib_root function copies a toolchain library and
> +# its symbolic links from the sysroot directory to the target
> +# directory. Note that this function is used both the external

used for

> +# toolchain logic, and the glibc package, so care must be taken when

I think the comma before 'and the' is not needed.

> +# changing this function.
>  #
>  # Most toolchains (CodeSourcery ones) have their libraries either in
>  # /lib or /usr/lib relative to their ARCH_SYSROOT_DIR, so we search
> --


Best regards,
Thomas

[Buildroot] [PATCHv3 12/20] toolchain-external: add support for musl C library

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This commit adds support for external toolchains based on the musl C
 Thomas> library, as available from http://www.musl-libc.org.

 Thomas> Note that the pre-built musl toolchains available from
 Thomas> http://musl.codu.org/ are not working for the moment, since they lack
 Thomas> sysroot support. However, this problem has been reported to the
 Thomas> maintainer, who has already added sysroot support in his scripts at
 Thomas> https://bitbucket.org/GregorR/musl-cross, and therefore the next
 Thomas> version of the pre-built toolchains should work with Buildroot
 Thomas> out-of-the-box. In the mean time, the musl-cross script must be used
 Thomas> to build the toolchain.

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

 Thomas> +# Check that the selected C library is really musl

s/is really/really is/

Committed with that fixed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 13/20] toolchain-external: improve help text of some options

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
 Thomas> ---
 Thomas>  toolchain/toolchain-external/Config.in | 15 +++++++++++----
 Thomas>  1 file changed, 11 insertions(+), 4 deletions(-)

[snip]

 Thomas> +	  Note: eglibc is a variant of glibc that (among other things)
 Thomas> +	  can be configured to exclude some of its features. Using a
 Thomas> +	  toolchain with eglibc configured to exclude key features may
 Thomas> +	  cause build failures to some packages.
 
 Thomas>  config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
 Thomas>  	bool "musl"

No musl help text?

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 05/20] toolchain-buildroot: convert to the package infrastructure

[Thomas De Schampheleire]

Hi Thomas,

On Tue, Oct 8, 2013 at 8:17 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> This commit converts the toolchain-buildroot logic to the package
> infrastructure. The package is fairly simple as it only defines
> BUILDROOT_LIBC, and depends on host-gcc-final to get the overall
> internal toolchain build logic started.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> ---
>  toolchain/toolchain-buildroot.mk                     |  9 ---------
>  toolchain/toolchain-buildroot/toolchain-buildroot.mk | 17 +++++++++++++++++
>  2 files changed, 17 insertions(+), 9 deletions(-)
>  delete mode 100644 toolchain/toolchain-buildroot.mk
>  create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk
>
> diff --git a/toolchain/toolchain-buildroot.mk b/toolchain/toolchain-buildroot.mk
> deleted file mode 100644
> index 3a05800..0000000
> --- a/toolchain/toolchain-buildroot.mk
> +++ /dev/null
> @@ -1,9 +0,0 @@
> -# Triggerring the build of the host-gcc-final will automaticaly do the
> -# build of binutils, uClibc, kernel headers and all the intermediate
> -# gcc steps.
> -
> -include toolchain/helpers.mk
> -
> -BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
> -
> -toolchain-buildroot: host-gcc-final
> diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk
> new file mode 100644
> index 0000000..ced430e
> --- /dev/null
> +++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk
> @@ -0,0 +1,17 @@
> +################################################################################
> +#
> +# toolchain-buildroot
> +#
> +################################################################################
> +
> +TOOLCHAIN_BUILDROOT_SOURCE =
> +
> +BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
> +
> +# Trigerring the build of the host-gcc-final will automatically do the
> +# build of binutils, uClibc, kernel headers and all the intermediate
> +# gcc steps.

It seems you miscorrected 'triggerring' after my comment on the first
version of your patch (and I did not yet have/take time to review v2
and v3). It should be:
Triggering (double g, single r instead of the other way around).

Best regards,
Thomas

[Buildroot] [PATCHv3 13/20] toolchain-external: improve help text of some options

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Wed, 09 Oct 2013 16:03:07 +0200, Peter Korsgaard wrote:
> >>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:
> 
>  Thomas> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>  Thomas> ---
>  Thomas>  toolchain/toolchain-external/Config.in | 15 +++++++++++----
>  Thomas>  1 file changed, 11 insertions(+), 4 deletions(-)
> 
> [snip]
> 
>  Thomas> +	  Note: eglibc is a variant of glibc that (among other things)
>  Thomas> +	  can be configured to exclude some of its features. Using a
>  Thomas> +	  toolchain with eglibc configured to exclude key features may
>  Thomas> +	  cause build failures to some packages.
>  
>  Thomas>  config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
>  Thomas>  	bool "musl"
> 
> No musl help text?

Damn, will send a follow up patch.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 14/20] busybox: add patches to fix build with the musl C library

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Those patches have been submitted upstream.

Committed this and 15, 16, 17, 18 - Thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 05/20] toolchain-buildroot: convert to the package infrastructure

[Peter Korsgaard]

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin@gmail.com> writes:

Hi,

 >> +# Trigerring the build of the host-gcc-final will automatically do the
 >> +# build of binutils, uClibc, kernel headers and all the intermediate
 >> +# gcc steps.

 Thomas> It seems you miscorrected 'triggerring' after my comment on the first
 Thomas> version of your patch (and I did not yet have/take time to review v2
 Thomas> and v3). It should be:
 Thomas> Triggering (double g, single r instead of the other way around).

Ahh yes, will fix - Thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Arnout Vandecappelle]

On 08/10/13 20:17, Thomas Petazzoni wrote:
> Until now, the make source, make external-deps and make legal-info
> where relying on enumerating the packages by looking at $(TARGETS)
> which contains only the target packages and not the host
> packages. Thanks to the TARGET_HOST_DEPS and HOST_DEPS variables, it
> was doing a two-level resolution of host package dependencies, but it
> was not entirely correct since the dependency chain might be deeper
> than that: some packages could be missed.
>
>>From an idea of Arnout, this patch introduces in each package
> additional targets <pkg>-all-source, <pkg>-all-legal-info and
> <pkg>-all-external-deps that executes the 'source', 'legal-info' and
> 'external-deps' targets for this package and all its dependencies, be
> they target or host dependencies.
>
> This provides a much cleaner implementation of this mechanism, which
> is also more robust.
>
> In order to achieve this, this patch also separates the "package"
> targets from other targets: instead of mixing them both in the global
> TARGETS variable, the new PACKAGES variable contains the name of all
> packages that are enabled in the configuration, while TARGETS is only
> used for additional things to be done (target-finalize, etc.). This
> separation is needed so that we don't try to use targets (such as
> <foo>-all-external-deps) on things that are not packages. Some further
> cleanups in this direction are possible, this commit takes a
> relatively minimal approach for now.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>   Makefile               | 53 +++++++++++++-------------------------------------
>   package/pkg-generic.mk | 16 ++++++++++++++-
>   2 files changed, 28 insertions(+), 41 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index f266e2d..702e7d4 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -238,8 +238,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
>
>   BASE_TARGETS = toolchain
>
> -TARGETS:=
> -
>   # silent mode requested?
>   QUIET:=$(if $(findstring s,$(MAKEFLAGS)),-q)
>
> @@ -310,9 +308,6 @@ endif
>   include package/Makefile.in
>   include support/dependencies/dependencies.mk
>

  You could remove this empty line as well.

> -# We also need the various per-package makefiles, which also add
> -# each selected package to TARGETS if that package was selected
> -# in the .config file.
>   include toolchain/helpers.mk
>   include toolchain/*/*.mk
>
> @@ -349,34 +344,11 @@ include fs/common.mk
>
>   TARGETS+=target-post-image
>
> -TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
> -TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
> -TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
> -TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
> -
> -# host-* dependencies have to be handled specially, as those aren't
> -# visible in Kconfig and hence not added to a variable like TARGETS.
> -# instead, find all the host-* targets listed in each <PKG>_DEPENDENCIES
> -# variable for each enabled target.
> -# Notice: this only works for newstyle gentargets/autotargets packages
> -TARGETS_HOST_DEPS = $(sort $(filter host-%,$(foreach dep,\
> -		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS))),\
> -		$($(dep)))))
> -# Host packages can in turn have their own dependencies. Likewise find
> -# all the package names listed in the HOST_<PKG>_DEPENDENCIES for each
> -# host package found above. Ideally this should be done recursively until
> -# no more packages are found, but that's hard to do in make, so limit to
> -# 1 level for now.
> -HOST_DEPS = $(sort $(foreach dep,\
> -		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS_HOST_DEPS))),\
> -		$($(dep))))
> -HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
> -
> -TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\
> -		$(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS))))
> -
> -# all targets depend on the crosscompiler and it's prerequisites
> -$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
> +PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))

  Can you make these match the coding style? PACKAGES_CLEAN = ...

> +PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))

  BASE_TARGETS is just "toolchain", right? And that is already part of 
the dependencies of PACKAGES, right? So why is that still needed here?

> +PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
> +PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))

  The PACKAGES_DIRCLEAN is really redundant, it is only used to define 
them as .PHONY but that is not done for e.g. -build.

> +PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
>
>   dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
>   	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
> @@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
>
>   prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>
> -world: $(BASE_TARGETS) $(TARGETS_ALL)
> +world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
>
>   .PHONY: all world toolchain dirs clean distclean source outputmakefile \
>   	legal-info legal-info-prepare legal-info-clean printvars \
> -	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
> -	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
> +	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
> +	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
> +	$(PACKAGES_EXTERNAL_DEPS) \
>   	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
>   	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
>
> @@ -562,10 +535,10 @@ target-post-image:
>   toolchain-eclipse-register:
>   	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
>
> -source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
> +source: dirs $(PACKAGES_SOURCE)
>
>   external-deps:
> -	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
> +	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
>
>   legal-info-clean:
>   	@rm -fr $(LEGAL_INFO_DIR)
> @@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
>   	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
>
>   legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
> -		$(TARGETS_LEGAL_INFO)
> +		$(PACKAGES_LEGAL_INFO)
>   	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
>   	@if [ -r $(LEGAL_WARNINGS) ]; then \
>   		cat support/legal-info/README.warnings-header \
> @@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
>   	@rm -f $(LEGAL_WARNINGS)
>
>   show-targets:
> -	@echo $(TARGETS)
> +	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
>
>   else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
>
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index a46457c..bd6169c 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -424,6 +424,20 @@ endif
>   $(1)-show-depends:
>   			@echo $$($(2)_DEPENDENCIES)
>
> +$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
> +
> +$(1)-external-deps:
> +ifneq ($$($(2)_SOURCE),)
> +			@echo $$($(2)_SOURCE)
> +endif
> +ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
> +			@echo $$($(2)_EXTRA_DOWNLOADS)
> +endif

  While you're at it, you could add $(2)_PATCH.

  Are the double dollars really needed? They're not used in -all-source 
so why would you use them here...

> +
> +$(1)-all-external-deps:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-external-deps) $(1)-external-deps
> +
> +$(1)-all-legal-info:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-legal-info) $(1)-legal-info
> +
>   $(1)-uninstall:		$(1)-configure $$($(2)_TARGET_UNINSTALL)
>
>   $(1)-clean:		$(1)-uninstall \
> @@ -528,7 +542,7 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
>   # configuration
>   ifeq ($$($$($(2)_KCONFIG_VAR)),y)
>
> -TARGETS += $(1)
> +PACKAGES += $(1)
>   PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
>   PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
>   PACKAGES_USERS += $$($(2)_USERS)$$(sep)


  Shouldn't you do something similar in fs/ ? Otherwise 'make source' 
will not download e.g. mtd.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv3 20/20] package: remove useless SHOW_EXTERNAL_DEPS support

[Arnout Vandecappelle]

On 08/10/13 20:17, Thomas Petazzoni wrote:
> Now that a package-based 'make external-deps' support has been added,
> the DL_MODE=SHOW_EXTERNAL_DEPS mechanism is no longer needed, so some
> clean up is done in this commit.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

  Looks good. Except:

[snip]
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index bd6169c..1e09efb 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -71,13 +71,10 @@ $(BUILD_DIR)/%/.stamp_rsynced:
>   	rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)
>   	$(Q)touch $@
>
> -# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
> -# packages
> +# Handle the SOURCE_CHECK case for rsynced packages
>   $(BUILD_DIR)/%/.stamp_rsync_sourced:
>   ifeq ($(DL_MODE),SOURCE_CHECK)
>   	test -d $(SRCDIR)
> -else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS)
> -	echo "file://$(SRCDIR)"

  This implies that you did the wrong thing in your previous patch. the 
-show-external-deps target should contain a check for _OVERRIDE_SRCDIR.

  Regards,
  Arnout

>   else
>   	@true # Nothing to do to source a local package
>   endif
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Thomas Petazzoni]

Arnout,

Thanks a lot for your review!

On Thu, 10 Oct 2013 19:24:12 +0200, Arnout Vandecappelle wrote:
> > @@ -310,9 +308,6 @@ endif
> >   include package/Makefile.in
> >   include support/dependencies/dependencies.mk
> >
> 
>   You could remove this empty line as well.

Right.

> > -# all targets depend on the crosscompiler and it's prerequisites
> > -$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
> > +PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))
> 
>   Can you make these match the coding style? PACKAGES_CLEAN = ...

Yes, certainly.

> > +PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
> 
>   BASE_TARGETS is just "toolchain", right? And that is already part of 
> the dependencies of PACKAGES, right? So why is that still needed here?

Yes, BASE_TARGETS is just toolchain now. However no, "toolchain" is not
yet part of the dependencies of all packages. This is something the
patch series from Fabio Porcedda is doing, and I don't want to solve
all problems in this patch set :)

The main reason I've kept BASE_TARGETS for now is because I knew Fabio
would be cleaning up that further with his patch set.

> > +PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
> > +PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))
> 
>   The PACKAGES_DIRCLEAN is really redundant, it is only used to define 
> them as .PHONY but that is not done for e.g. -build.

True, will remove.

> > +PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
> >
> >   dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
> >   	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
> > @@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
> >
> >   prepare: $(BUILD_DIR)/buildroot-config/auto.conf
> >
> > -world: $(BASE_TARGETS) $(TARGETS_ALL)
> > +world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
> >
> >   .PHONY: all world toolchain dirs clean distclean source outputmakefile \
> >   	legal-info legal-info-prepare legal-info-clean printvars \
> > -	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
> > -	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
> > +	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
> > +	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
> > +	$(PACKAGES_EXTERNAL_DEPS) \
> >   	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
> >   	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
> >
> > @@ -562,10 +535,10 @@ target-post-image:
> >   toolchain-eclipse-register:
> >   	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
> >
> > -source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
> > +source: dirs $(PACKAGES_SOURCE)
> >
> >   external-deps:
> > -	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
> > +	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
> >
> >   legal-info-clean:
> >   	@rm -fr $(LEGAL_INFO_DIR)
> > @@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
> >   	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
> >
> >   legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
> > -		$(TARGETS_LEGAL_INFO)
> > +		$(PACKAGES_LEGAL_INFO)
> >   	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
> >   	@if [ -r $(LEGAL_WARNINGS) ]; then \
> >   		cat support/legal-info/README.warnings-header \
> > @@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
> >   	@rm -f $(LEGAL_WARNINGS)
> >
> >   show-targets:
> > -	@echo $(TARGETS)
> > +	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
> >
> >   else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
> >
> > diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> > index a46457c..bd6169c 100644
> > --- a/package/pkg-generic.mk
> > +++ b/package/pkg-generic.mk
> > @@ -424,6 +424,20 @@ endif
> >   $(1)-show-depends:
> >   			@echo $$($(2)_DEPENDENCIES)
> >
> > +$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
> > +
> > +$(1)-external-deps:
> > +ifneq ($$($(2)_SOURCE),)
> > +			@echo $$($(2)_SOURCE)
> > +endif
> > +ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
> > +			@echo $$($(2)_EXTRA_DOWNLOADS)
> > +endif
> 
>   While you're at it, you could add $(2)_PATCH.

Ahh, yes, true. I thought about it at some point, and then forgot.

>   Are the double dollars really needed? They're not used in -all-source 
> so why would you use them here...

I'll check that, maybe not.

> > -TARGETS += $(1)
> > +PACKAGES += $(1)
> >   PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
> >   PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
> >   PACKAGES_USERS += $$($(2)_USERS)$$(sep)
> 
>   Shouldn't you do something similar in fs/ ? Otherwise 'make source' 
> will not download e.g. mtd.

That's a weakness of this patch series. I believe I might need to
convert the fs/ stuff to packages (after all they are packages that
have dependencies, and install something in the images/ directory), so
that the legal-info/source/external-deps logic works for them as well,
without doing hacks. What do you think about this?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Arnout Vandecappelle]

On 10/10/13 23:33, Thomas Petazzoni wrote:
>> >   Shouldn't you do something similar in fs/ ? Otherwise 'make source'
>> >will not download e.g. mtd.
> That's a weakness of this patch series. I believe I might need to
> convert the fs/ stuff to packages (after all they are packages that
> have dependencies, and install something in the images/ directory), so
> that the legal-info/source/external-deps logic works for them as well,
> without doing hacks. What do you think about this?

  I don't think it's useful to make them packages, because they are 
really different. Adding the propagation of -all-source etc. to the 
dependencies can easily be done in ROOTFS_TARGET_INTERNAL.

  In a later stage, we could consider to move the (few) common parts into 
some common package/rootfs infrastructure, but the gain is limited I expect.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Fri, 11 Oct 2013 00:38:25 +0200, Arnout Vandecappelle wrote:

>   I don't think it's useful to make them packages, because they are 
> really different. Adding the propagation of -all-source etc. to the 
> dependencies can easily be done in ROOTFS_TARGET_INTERNAL.

Yeah, that's indeed another option. We're already doing it with
$(1)-show-depends.

>   In a later stage, we could consider to move the (few) common parts into 
> some common package/rootfs infrastructure, but the gain is limited I expect.

Note that we have another strange location for a dependency on a
package: the target-root-passwd target, which sets the root password,
might depend on the host-mkpasswd package. So while normally, we
consider all the target-<foo> targets in system/system.mk as useless
for 'make source', 'make external-deps' and so on, it's not really the
case for this one. Except that host-mkpasswd in fact has all its source
code directly in package/mkpasswd/ (we don't download anything), and
it's used purely on the host, so maybe we can ignore this problem?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Fabio Porcedda]

On Thu, Oct 10, 2013 at 11:33 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Arnout,
>
> Thanks a lot for your review!
>
> On Thu, 10 Oct 2013 19:24:12 +0200, Arnout Vandecappelle wrote:
> [...]
>> > +PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
>>
>>   BASE_TARGETS is just "toolchain", right? And that is already part of
>> the dependencies of PACKAGES, right? So why is that still needed here?
>
> Yes, BASE_TARGETS is just toolchain now. However no, "toolchain" is not
> yet part of the dependencies of all packages. This is something the
> patch series from Fabio Porcedda is doing, and I don't want to solve
> all problems in this patch set :)
>
> The main reason I've kept BASE_TARGETS for now is because I knew Fabio
> would be cleaning up that further with his patch set.

Sure, I will send a rebased patch set when this patch set is merged.

> [...]

Regards
-- 
Fabio Porcedda

[Buildroot] [PATCHv3 19/20] package: package-based implementation of source, external-deps and legal-info

[Arnout Vandecappelle]

On 11/10/13 09:20, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
>
> On Fri, 11 Oct 2013 00:38:25 +0200, Arnout Vandecappelle wrote:
>
>>    I don't think it's useful to make them packages, because they are
>> really different. Adding the propagation of -all-source etc. to the
>> dependencies can easily be done in ROOTFS_TARGET_INTERNAL.
>
> Yeah, that's indeed another option. We're already doing it with
> $(1)-show-depends.
>
>>    In a later stage, we could consider to move the (few) common parts into
>> some common package/rootfs infrastructure, but the gain is limited I expect.
>
> Note that we have another strange location for a dependency on a
> package: the target-root-passwd target, which sets the root password,
> might depend on the host-mkpasswd package. So while normally, we
> consider all the target-<foo> targets in system/system.mk as useless
> for 'make source', 'make external-deps' and so on, it's not really the
> case for this one. Except that host-mkpasswd in fact has all its source
> code directly in package/mkpasswd/ (we don't download anything), and
> it's used purely on the host, so maybe we can ignore this problem?

  For now, yes. If we ever refactor the common parts from fs and 
pkg-generic, then we can also include the system stuff in there.

  Regards,
  Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F