[Buildroot] [PATCH 0/6] toolchain-external: copy_toolchain_sysroot improvements

[Buildroot] [PATCH 0/6] toolchain-external: copy_toolchain_sysroot improvements

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Hi,

This toolchain patch series started out to fix the handling of the Cavium Octeon
2 toolchain (last patch) but along the way I fixed a few other things.
Details are found in the commit messages of these patches.

I tested a number of external toolchains and arches available in Buildroot, as
well as the Octeon SDK toolchain.  I think I understand the situation
sufficiently well to get some confidence, but this is the type of
change that is difficult to get 100% certainty about.

Best regards,
Thomas

Thomas De Schampheleire (6):
  toolchain-external: remove stale references to (ARCH_)LIBC_A_LOCATION
  toolchain helpers: clarify rsync excludes in copy_toolchain_sysroot
  toolchain-external: handle ld.so fixups centrally
  toolchain helpers: introduce function relpath_prefix
  toolchain-external: cover multilib toolchains with lib/<variant>
    layout
  DEVELOPERS: update Thomas De Schampheleire

 DEVELOPERS                                         |  9 ++--
 toolchain/helpers.mk                               | 63 +++++++++++++++++-----
 .../toolchain-external/pkg-toolchain-external.mk   | 40 +++++++-------
 .../toolchain-external-codesourcery-aarch64.mk     | 10 ----
 4 files changed, 73 insertions(+), 49 deletions(-)

-- 
2.10.2

[Buildroot] [PATCH 1/6] toolchain-external: remove stale references to (ARCH_)LIBC_A_LOCATION

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The variables LIBC_A_LOCATION and ARCH_LIBC_A_LOCATION were killed in commit
646bd86908f7f995cdc2b95e0be381d97600f905 but the corresponding descriptions
were never removed.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 .../toolchain-external/pkg-toolchain-external.mk   | 30 ++++++++--------------
 1 file changed, 11 insertions(+), 19 deletions(-)

diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
index e0a8b41..6658875 100644
--- a/toolchain/toolchain-external/pkg-toolchain-external.mk
+++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
@@ -347,35 +347,27 @@ endef
 #
 # Variables are defined as follows:
 #
-#  LIBC_A_LOCATION:     location of the libc.a file in the default
-#                       multilib variant (allows to find the main
-#                       sysroot directory)
-#                       Ex: /x-tools/mips-2011.03/mips-linux-gnu/libc/usr/lib/libc.a
-#
-#  SYSROOT_DIR:         the main sysroot directory, deduced from
-#                       LIBC_A_LOCATION by removing the
-#                       usr/lib[32|64]/libc.a part of the path.
+# SYSROOT_DIR:          the main sysroot directory, deduced from the location of
+#                       the libc.a file in the default multilib variant, by
+#                       removing the usr/lib[32|64]/libc.a part of the path.
 #                       Ex: /x-tools/mips-2011.03/mips-linux-gnu/libc/
 #
-# ARCH_LIBC_A_LOCATION: location of the libc.a file in the selected
-#                       multilib variant (taking into account the
-#                       CFLAGS). Allows to find the sysroot of the
-#                       selected multilib variant.
-#                       Ex: /x-tools/mips-2011.03/mips-linux-gnu/libc/mips16/soft-float/el/usr/lib/libc.a
-#
 # ARCH_SYSROOT_DIR:     the sysroot of the selected multilib variant,
-#                       deduced from ARCH_LIBC_A_LOCATION by removing
-#                       usr/lib[32|64]/libc.a at the end of the path.
+#                       deduced from the location of the libc.a file in the
+#                       selected multilib variant (taking into account the
+#                       CFLAGS), by removing usr/lib[32|64]/libc.a at the end
+#                       of the path.
 #                       Ex: /x-tools/mips-2011.03/mips-linux-gnu/libc/mips16/soft-float/el/
 #
 # ARCH_LIB_DIR:         'lib', 'lib32' or 'lib64' depending on where libraries
-#                       are stored. Deduced from ARCH_LIBC_A_LOCATION by
-#                       looking at usr/lib??/libc.a.
+#                       are stored. Deduced from the location of the libc.a file
+#                       in the selected multilib variant, by looking at
+#                       usr/lib??/libc.a.
 #                       Ex: lib
 #
 # ARCH_SUBDIR:          the relative location of the sysroot of the selected
 #                       multilib variant compared to the main sysroot.
-#			Ex: mips16/soft-float/el
+#                       Ex: mips16/soft-float/el
 #
 # SUPPORT_LIB_DIR:      some toolchains, such as recent Linaro toolchains,
 #                       store GCC support libraries (libstdc++,
-- 
2.10.2

[Buildroot] [PATCH 2/6] toolchain helpers: clarify rsync excludes in copy_toolchain_sysroot

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The copy_toolchain_sysroot helper features a complex rsync loop that copies
various directories from the extracted toolchain to the staging directory.
The complexity mainly stems from the fact that we support multilib toolchain
tarballs but only copy one of the multilib variants into staging.

Increase understandability of this logic by explicitly restricting the
rsync excludes to the iteration of the for loop they are relevant for.
Additionally, update the function comment.

Note: all attempts to reduce duplication between both rsync while keeping
things nice and readable failed. One has to be extremely careful regarding
line continuation, indentation, and single vs double quoting. In the end, a
split up rsync seemed most clean.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 toolchain/helpers.mk | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 72e7292..06dc197 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -39,11 +39,16 @@ copy_toolchain_lib_root = \
 # dir. The operation of this function is rendered a little bit
 # complicated by the support for multilib toolchains.
 #
-# We start by copying etc, lib, sbin and usr from the sysroot of the
-# selected architecture variant (as pointed by ARCH_SYSROOT_DIR). This
-# allows to import into the staging directory the C library and
-# companion libraries for the correct architecture variant. We
-# explictly only copy etc, lib, sbin and usr since other directories
+# We start by copying etc, 'lib', sbin, usr and usr/'lib' from the
+# sysroot of the selected architecture variant (as pointed to by
+# ARCH_SYSROOT_DIR). This allows to import into the staging directory
+# the C library and companion libraries for the correct architecture
+# variant. 'lib' may not be literally 'lib' but could be something else,
+# e.g. lib32-fp (as determined by ARCH_LIB_DIR) and we only want to copy
+# that lib directory and no other. When copying usr, we therefore need
+# to be extra careful not to include usr/lib* but we _do_ want to
+# include usr/libexec.
+# We are selective in the directories we copy since other directories
 # might exist for other architecture variants (on Codesourcery
 # toolchain, the sysroot for the default architecture variant contains
 # the armv4t and thumb2 subdirectories, which are the sysroot for the
@@ -90,9 +95,14 @@ copy_toolchain_sysroot = \
 	SUPPORT_LIB_DIR="$(strip $5)" ; \
 	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
 		if [ -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
-			rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
-				--include '/libexec*/' --exclude '/lib*/' \
-				$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
+			if [ "$$i" = "usr" ]; then \
+				rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
+					--include '/libexec*/' --exclude '/lib*/' \
+					$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
+			else \
+				rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
+					$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
+			fi ; \
 		fi ; \
 	done ; \
 	if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
-- 
2.10.2

[Buildroot] [PATCH 3/6] toolchain-external: handle ld.so fixups centrally

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

Normally, the Buildroot toolchain logic copies all required libraries from
the external toolchain to the staging directory, including the dynamic
loader ld-*.so.
There are cases, however, where the dynamic loader is _not_ automatically
copied to staging. This happens when the dynamic loader is not inside
ARCH_LIB_DIR itself (e.g. lib64), but instead resides in 'lib' (assume, of
course, that ARCH_LIB_DIR != 'lib').

Currently, this is fixed in a toolchain-specific fixup, e.g. by recreating a
missing symlink or copying over a missing file. Such toolchain specific
fixups are not very nice.

Moreover, in a subsequent patch, the value of ARCH_LIB_DIR changes for some
toolchains, causing them to have the same problem of a missing dynamic
loader. This used to be the case for older Linaro toolchains with libraries
in 'lib/<tuple>': Buildroot used to set ARCH_LIB_DIR=lib but the mentioned
patch changes it to 'lib/<tuple>' instead. As a result, the files directly
under 'lib/' will no longer be copied. There should be none, but the dynamic
loader is a notable exception.
[Note: support for these older Linaro toolchain has been removed in 2016.11]

Instead, copy over the ld.so file(s)/link(s) from the extracted toolchain
into staging, in the central copy_toolchain_sysroot function. The existing
toolchain logic will then handle the copy of these files from staging to
target.
This means the toolchain-specific fixups can be removed.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 toolchain/helpers.mk                                          | 11 +++++++++++
 .../toolchain-external-codesourcery-aarch64.mk                | 10 ----------
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 06dc197..6720629 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -55,6 +55,11 @@ copy_toolchain_lib_root = \
 # corresponding architecture variants), and we don't want to import
 # them.
 #
+# It is possible that ARCH_LIB_DIR does not contain the dynamic loader
+# (ld*.so or similar) because it (or the main symlink to it) normally
+# resides in /lib while ARCH_LIB_DIR may be something else (e.g. lib64).
+# Therefore, copy the dynamic loader separately.
+#
 # Then, if the selected architecture variant is not the default one
 # (i.e, if SYSROOT_DIR != ARCH_SYSROOT_DIR), then we :
 #
@@ -105,6 +110,12 @@ copy_toolchain_sysroot = \
 			fi ; \
 		fi ; \
 	done ; \
+	if [ -e $${ARCH_SYSROOT_DIR}/lib/ld*.so ]; then \
+		cp -a $${ARCH_SYSROOT_DIR}/lib/ld*.so $(STAGING_DIR)/lib/ ; \
+	fi ; \
+	if [ -e $${ARCH_SYSROOT_DIR}/lib/ld*.so.* ]; then \
+		cp -a $${ARCH_SYSROOT_DIR}/lib/ld*.so.* $(STAGING_DIR)/lib/ ; \
+	fi ; \
 	if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
 		if [ ! -d $${ARCH_SYSROOT_DIR}/usr/include ] ; then \
 			cp -a $${SYSROOT_DIR}/usr/include $(STAGING_DIR)/usr ; \
diff --git a/toolchain/toolchain-external/toolchain-external-codesourcery-aarch64/toolchain-external-codesourcery-aarch64.mk b/toolchain/toolchain-external/toolchain-external-codesourcery-aarch64/toolchain-external-codesourcery-aarch64.mk
index bc58c44..5f4453a 100644
--- a/toolchain/toolchain-external/toolchain-external-codesourcery-aarch64/toolchain-external-codesourcery-aarch64.mk
+++ b/toolchain/toolchain-external/toolchain-external-codesourcery-aarch64/toolchain-external-codesourcery-aarch64.mk
@@ -9,14 +9,4 @@ TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_VERSION = 2014.11-95
 TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_SOURCE = aarch64-amd-$(TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_VERSION)-$(TOOLCHAIN_EXTERNAL_PREFIX)-i686-pc-linux-gnu.tar.bz2
 TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_ACTUAL_SOURCE_TARBALL = aarch64-amd-$(TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_VERSION)-$(TOOLCHAIN_EXTERNAL_PREFIX).src.tar.bz2
 
-define TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_STAGING_FIXUP
-	ln -sf ld-2.20.so $(STAGING_DIR)/lib/ld-linux-aarch64.so.1
-endef
-TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_STAGING_FIXUP
-
-define TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_TARGET_FIXUP
-	ln -sf ld-2.20.so $(TARGET_DIR)/lib/ld-linux-aarch64.so.1
-endef
-TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_POST_INSTALL_TARGET_HOOKS += TOOLCHAIN_EXTERNAL_CODESOURCERY_AARCH64_TARGET_FIXUP
-
 $(eval $(toolchain-external-package))
-- 
2.10.2

[Buildroot] [PATCH 4/6] toolchain helpers: introduce function relpath_prefix

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The helper function copy_toolchain_sysroot has some logic to transform a
path into a number of '../' components based on the depth of that path.

As this same logic will be needed in another place in a subsequent patch,
extract it into a separate helper relpath_prefix.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 toolchain/helpers.mk | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 6720629..ba14b38 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -121,11 +121,7 @@ copy_toolchain_sysroot = \
 			cp -a $${SYSROOT_DIR}/usr/include $(STAGING_DIR)/usr ; \
 		fi ; \
 		mkdir -p `dirname $(STAGING_DIR)/$${ARCH_SUBDIR}` ; \
-		relpath="./" ; \
-		nbslashs=`printf $${ARCH_SUBDIR} | sed 's%[^/]%%g' | wc -c` ; \
-		for slash in `seq 1 $${nbslashs}` ; do \
-			relpath=$${relpath}"../" ; \
-		done ; \
+		relpath="$(call relpath_prefix,$${ARCH_SUBDIR})./" ; \
 		ln -s $${relpath} $(STAGING_DIR)/$${ARCH_SUBDIR} ; \
 		echo "Symlinking $(STAGING_DIR)/$${ARCH_SUBDIR} -> $${relpath}" ; \
 	fi ; \
@@ -424,3 +420,22 @@ check_toolchain_ssp = \
 gen_gdbinit_file = \
 	mkdir -p $(STAGING_DIR)/usr/share/buildroot/ ; \
 	echo "set sysroot $(STAGING_DIR)" > $(STAGING_DIR)/usr/share/buildroot/gdbinit
+
+# Given a path, determine the relative prefix (../) needed to return to the
+# root level. Note that the last component is treated as a file component; use a
+# trailing slash to force treating it as a directory. Examples:
+#     relpath_prefix(lib32) = ""
+#     relpath_prefix(lib32/octeon2) = "../"
+#     relpath_prefix(lib32/octeon2/) = "../../"
+#
+# $1: input path
+define relpath_prefix
+$$( \
+	prefix="" ; \
+	nbslashs=`printf $1 | sed 's%[^/]%%g' | wc -c` ; \
+	for slash in `seq 1 $${nbslashs}` ; do \
+		prefix=$${prefix}"../" ; \
+	done ; \
+	printf "$$prefix" ;\
+)
+endef
-- 
2.10.2

[Buildroot] [PATCH 5/6] toolchain-external: cover multilib toolchains with lib/<variant> layout

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>

The toolchain from the Cavium Octeon SDK has a sysroot layout as follows:

./lib32
./lib32/octeon2
./lib32-fp
./lib64
./lib64/octeon2
./lib64-fp
./usr
./usr/lib
./usr/lib32
./usr/lib32/octeon2
./usr/lib32-fp
./usr/lib64
./usr/lib64/octeon2
./usr/lib64-fp
./usr/bin
./usr/bin32
./usr/bin32-fp
./usr/bin64-fp
./usr/libexec
./usr/libexec32
./usr/libexec32-fp
./usr/libexec64-fp
./usr/sbin
./usr/sbin32
./usr/sbin32-fp
./usr/sbin64-fp
./usr/include
./usr/share
./sbin
./sbin32
./sbin32-fp
./sbin64-fp
./etc
./var

with the following selections:
- lib64          : default
- lib64/octeon2  : -march=octeon2
- lib64-fp       : -march=octeon3
- lib32          : -mabi=n32
- lib32/octeon2  : -mabi=n32 -march=octeon2
- lib32-fp       : -mabi=n32 -march=octeon3

In case of '-mabi=n32 -march=octeon2' (but same is true for n64+octeon2)the
original Buildroot toolchain logic would copy both the libraries in
lib32 as the subdirectory lib32/octeon2, which means that every library is
installed twice (but only one of each is really needed).

While ARCH_LIB_DIR is determined by the location of libc.a, which in this
case is effectively:
    <sysroot>/usr/lib32/octeon2/libc.a
the variable only retains 'lib32' and not 'lib32/octeon2' as expected.

To make Buildroot cope with this style of toolchain layout, we need to adapt
the calculation of ARCH_LIB_DIR to also include the second part.

Certain older Linaro toolchains actually had the same layout. Libraries were
located in lib/<tuple> rather than lib directly. Previously, this was
handled by adding a toolchain-specific fixup that creates a symlink
lib/<tuple> -> lib, but with this patch this would no longer be needed.
Note that one difference with the Octeon case is that these Linaro
toolchains are not actually multilib, i.e. there is just one location with
the libraries and thus there is no problem with duplicated libraries.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 toolchain/helpers.mk                                   |  5 +++--
 toolchain/toolchain-external/pkg-toolchain-external.mk | 10 +++++++---
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index ba14b38..db27f73 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -57,8 +57,8 @@ copy_toolchain_lib_root = \
 #
 # It is possible that ARCH_LIB_DIR does not contain the dynamic loader
 # (ld*.so or similar) because it (or the main symlink to it) normally
-# resides in /lib while ARCH_LIB_DIR may be something else (e.g. lib64).
-# Therefore, copy the dynamic loader separately.
+# resides in /lib while ARCH_LIB_DIR may be something else (e.g. lib64,
+# lib/<tuple>, ...). Therefore, copy the dynamic loader separately.
 #
 # Then, if the selected architecture variant is not the default one
 # (i.e, if SYSROOT_DIR != ARCH_SYSROOT_DIR), then we :
@@ -103,6 +103,7 @@ copy_toolchain_sysroot = \
 			if [ "$$i" = "usr" ]; then \
 				rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
 					--include '/libexec*/' --exclude '/lib*/' \
+					--exclude '/$${ARCH_LIB_DIR}' \
 					$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
 			else \
 				rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
index 6658875..ada767e 100644
--- a/toolchain/toolchain-external/pkg-toolchain-external.mk
+++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
@@ -332,7 +332,7 @@ endef
 # Returns the lib subdirectory for the given compiler + flags (i.e
 # typically lib32 or lib64 for some toolchains)
 define toolchain_find_libdir
-$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?([^/]*)?)/([^/]*/)?libc.a:\2:')
+$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?([^/]*)?(/[^/]*)?)/libc.a:\2:')
 endef
 
 # Returns the location of the libc.a file for the given compiler + flags
@@ -442,14 +442,18 @@ endef
 # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
 # Note: the skeleton package additionally creates lib32->lib or lib64->lib
 # (as appropriate)
+# The relpath handling is needed in case ARCH_LIB_DIR is not a singular path
+# like 'lib32-foo' but a composite one, like 'lib/foo'. In either case, we need
+# to link to the top-level 'lib'.
 #
 # $1: destination directory (TARGET_DIR / STAGING_DIR)
 create_lib_symlinks = \
        $(Q)DESTDIR="$(strip $1)" ; \
        ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
        if [ ! -e "$${DESTDIR}/$${ARCH_LIB_DIR}" -a ! -e "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ]; then \
-               ln -snf lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
-               ln -snf lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
+               relpath="$(call relpath_prefix,$${ARCH_LIB_DIR})" ; \
+               ln -snf $${relpath}lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
+               ln -snf $${relpath}lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
        fi
 
 define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
-- 
2.10.2

[Buildroot] [PATCH 6/6] DEVELOPERS: update Thomas De Schampheleire

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

The originally added entries were based on 'last committer' heuristics, but
I do not particularly feel responsible for them.

Instead update the list with more appropriate entries.

Additionally, update my email address.

Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
---
 DEVELOPERS | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/DEVELOPERS b/DEVELOPERS
index c7f2dd9..1101140 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1476,10 +1476,11 @@ F:	package/fcgiwrap/
 N:	Thomas Davis <sunsetbrew@sunsetbrew.com>
 F:	package/civetweb/
 
-N:	Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>
-F:	package/libpfm4/
-F:	package/mkpasswd/
-F:	package/zeromq/
+N:	Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
+F:	docs/manual/
+F:	support/scripts/size-stats
+F:	support/scripts/size-stats-compare
+F:	toolchain/
 
 N:	Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
 F:	arch/Config.in.arm
-- 
2.10.2

[Buildroot] [PATCH 5/6] toolchain-external: cover multilib toolchains with lib/<variant> layout

[Thomas De Schampheleire]

On Tue, Jan 31, 2017 at 5:03 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
>
> The toolchain from the Cavium Octeon SDK has a sysroot layout as follows:
>
> ./lib32
> ./lib32/octeon2
> ./lib32-fp
> ./lib64
> ./lib64/octeon2
> ./lib64-fp
> ./usr
> ./usr/lib
> ./usr/lib32
> ./usr/lib32/octeon2
> ./usr/lib32-fp
> ./usr/lib64
> ./usr/lib64/octeon2
> ./usr/lib64-fp
> ./usr/bin
> ./usr/bin32
> ./usr/bin32-fp
> ./usr/bin64-fp
> ./usr/libexec
> ./usr/libexec32
> ./usr/libexec32-fp
> ./usr/libexec64-fp
> ./usr/sbin
> ./usr/sbin32
> ./usr/sbin32-fp
> ./usr/sbin64-fp
> ./usr/include
> ./usr/share
> ./sbin
> ./sbin32
> ./sbin32-fp
> ./sbin64-fp
> ./etc
> ./var
>
> with the following selections:
> - lib64          : default
> - lib64/octeon2  : -march=octeon2
> - lib64-fp       : -march=octeon3
> - lib32          : -mabi=n32
> - lib32/octeon2  : -mabi=n32 -march=octeon2
> - lib32-fp       : -mabi=n32 -march=octeon3
>
> In case of '-mabi=n32 -march=octeon2' (but same is true for n64+octeon2)the
> original Buildroot toolchain logic would copy both the libraries in
> lib32 as the subdirectory lib32/octeon2, which means that every library is
> installed twice (but only one of each is really needed).
>
> While ARCH_LIB_DIR is determined by the location of libc.a, which in this
> case is effectively:
>     <sysroot>/usr/lib32/octeon2/libc.a
> the variable only retains 'lib32' and not 'lib32/octeon2' as expected.
>
> To make Buildroot cope with this style of toolchain layout, we need to adapt
> the calculation of ARCH_LIB_DIR to also include the second part.
>
> Certain older Linaro toolchains actually had the same layout. Libraries were
> located in lib/<tuple> rather than lib directly. Previously, this was
> handled by adding a toolchain-specific fixup that creates a symlink
> lib/<tuple> -> lib, but with this patch this would no longer be needed.
> Note that one difference with the Octeon case is that these Linaro
> toolchains are not actually multilib, i.e. there is just one location with
> the libraries and thus there is no problem with duplicated libraries.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  toolchain/helpers.mk                                   |  5 +++--
>  toolchain/toolchain-external/pkg-toolchain-external.mk | 10 +++++++---
>  2 files changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index ba14b38..db27f73 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -57,8 +57,8 @@ copy_toolchain_lib_root = \
>  #
>  # It is possible that ARCH_LIB_DIR does not contain the dynamic loader
>  # (ld*.so or similar) because it (or the main symlink to it) normally
> -# resides in /lib while ARCH_LIB_DIR may be something else (e.g. lib64).
> -# Therefore, copy the dynamic loader separately.
> +# resides in /lib while ARCH_LIB_DIR may be something else (e.g. lib64,
> +# lib/<tuple>, ...). Therefore, copy the dynamic loader separately.
>  #
>  # Then, if the selected architecture variant is not the default one
>  # (i.e, if SYSROOT_DIR != ARCH_SYSROOT_DIR), then we :
> @@ -103,6 +103,7 @@ copy_toolchain_sysroot = \
>                         if [ "$$i" = "usr" ]; then \
>                                 rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
>                                         --include '/libexec*/' --exclude '/lib*/' \
> +                                       --exclude '/$${ARCH_LIB_DIR}' \
>                                         $${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
>                         else \
>                                 rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale'\
> diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
> index 6658875..ada767e 100644
> --- a/toolchain/toolchain-external/pkg-toolchain-external.mk
> +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
> @@ -332,7 +332,7 @@ endef
>  # Returns the lib subdirectory for the given compiler + flags (i.e
>  # typically lib32 or lib64 for some toolchains)
>  define toolchain_find_libdir
> -$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?([^/]*)?)/([^/]*/)?libc.a:\2:')
> +$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?([^/]*)?(/[^/]*)?)/libc.a:\2:')
>  endef
>
>  # Returns the location of the libc.a file for the given compiler + flags
> @@ -442,14 +442,18 @@ endef
>  # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
>  # Note: the skeleton package additionally creates lib32->lib or lib64->lib
>  # (as appropriate)
> +# The relpath handling is needed in case ARCH_LIB_DIR is not a singular path
> +# like 'lib32-foo' but a composite one, like 'lib/foo'. In either case, we need
> +# to link to the top-level 'lib'.
>  #
>  # $1: destination directory (TARGET_DIR / STAGING_DIR)
>  create_lib_symlinks = \
>         $(Q)DESTDIR="$(strip $1)" ; \
>         ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
>         if [ ! -e "$${DESTDIR}/$${ARCH_LIB_DIR}" -a ! -e "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ]; then \
> -               ln -snf lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
> -               ln -snf lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
> +               relpath="$(call relpath_prefix,$${ARCH_LIB_DIR})" ; \
> +               ln -snf $${relpath}lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
> +               ln -snf $${relpath}lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
>         fi
>
>  define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
> --
> 2.10.2


It turns out there is still a problem with this patch: with the Cavium
toolchain, the layout in staging is different than the original,
causing broken symlinks, e.g.

output/staging/usr/lib/librt.so -> ../../../lib32/octeon2/librt.so.1

Here, the destination refers to output/lib32/octeon2/librt.so.1, but
it should instead refer to output/staging/lib/librt.so.1.
This is a result of the fact that we copy stuff from one location to
another, but with this patch it is no longer guaranteed that the
directory depth is the same after the copy. Relative symbolic links
can thus start to fail.

I need to think a bit more about it, but a rough thought would be that
if instead of creating all files in lib and usr/lib and creating links
from ARCH_LIB_DIR to that location, we would create all files in the
same relative location as the original toolchain, and create links in
the other direction. Then the directory depth in buildroot is the same
as in the original toolchain, and links should still work.

Any input is welcome.

Thanks,
Thomas

[Buildroot] [PATCH 6/6] DEVELOPERS: update Thomas De Schampheleire

[Thomas Petazzoni]

Hello,

On Tue, 31 Jan 2017 17:03:43 +0100, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> The originally added entries were based on 'last committer' heuristics, but
> I do not particularly feel responsible for them.

Come on, you don't even maintain the packages you've contributed! :-)

> Instead update the list with more appropriate entries.
> 
> Additionally, update my email address.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  DEVELOPERS | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)

Anyway, applied to master, thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/6] toolchain-external: remove stale references to (ARCH_)LIBC_A_LOCATION

[Thomas Petazzoni]

Hello,

On Tue, 31 Jan 2017 17:03:38 +0100, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> 
> The variables LIBC_A_LOCATION and ARCH_LIBC_A_LOCATION were killed in commit
> 646bd86908f7f995cdc2b95e0be381d97600f905 but the corresponding descriptions
> were never removed.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> ---
>  .../toolchain-external/pkg-toolchain-external.mk   | 30 ++++++++--------------
>  1 file changed, 11 insertions(+), 19 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com