[Buildroot] [Bug 9771] New: toolchain-external misses lib directory when copying sysroot

[Buildroot] [Bug 9771] New: toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

            Bug ID: 9771
           Summary: toolchain-external misses lib directory when copying
                    sysroot
           Product: buildroot
           Version: 2016.11
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned at buildroot.uclibc.org
          Reporter: sir.ferdek+buildroot at gmail.com
                CC: buildroot at uclibc.org
  Target Milestone: ---

I have x86_64 cross-toolchain build using crosstool-ng framework, not too old
commit from master branch, circa half month old. I was unable to use it with
buildroot due to wrong toolchain installation performed by buildroot.

Packages' autoconf scripts complain about unusable compiler, due to missing
libgcc_s.so which is in x86_64-unknown-linux-gnu/sysroot/lib

The lib directory is not copied by buildroot during install phase.

This little fix below allowed me to build my system image, although I would
like to ask you if this is complete - I guess no. The reason I ask is that I
saw very complicated logic somewhere in pkg-toolchain-external.mk and I suspect
that this is the place where it should be fixed...

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 2f73ebb..cab92dd 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -88,7 +88,7 @@ copy_toolchain_sysroot = \
        ARCH_SUBDIR="$(strip $3)"; \
        ARCH_LIB_DIR="$(strip $4)" ; \
        SUPPORT_LIB_DIR="$(strip $5)" ; \
-       for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
+       for i in etc lib $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
                if [ ! -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
                        continue ; \
                fi ; \

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
Your patch shouldn't be necessary because $${ARCH_LIB_DIR} in the loop should
be responsible for copying the contents of lib/.

Can you share a tarball of the toolchain you have built with Crosstool-NG, so
we can see why it doesn't work as expected?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #2 from sir.ferdek+buildroot at gmail.com ---
I hope this is OK:
https://drive.google.com/open?id=0B-dCRyKJ6D3zTTJJU192VTZJLWc

From what I remember while debugging this, $${ARCH_LIB_DIR} is equal to lib64,
while there are still some libraries in lib dir that need to be copied.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #3 from Arnout Vandecappelle <arnout@mind.be> ---
Can you try if http://patchwork.ozlabs.org/patch/725387/ fixes this? You'll
have to apply the preceding 3 patches as well, and there may be a few merge
issues.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #4 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
(In reply to Arnout Vandecappelle from comment #3)
Arnout, what makes you think those patches will fix the issue? From a quick
look at the toolchain, I don't see anything being in <sysroot>/lib/<variant>,
all libraries are in <sysroot>/lib64, which is the standard organization that
we should just support fine today.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
OK, so the issue is that some libraries are in <sysroot>/lib64 (C library) and
some others are in <sysroot>/lib (gcc runtime), which is not something
Buildroot handles I believe.

Buildroot does a:

$ ./bin/x86_64-unknown-linux-gnu-gcc -print-file-name=libc.a
/home/thomas/dl/x86_64-unknown-linux-gnu/bin/../x86_64-unknown-linux-gnu/sysroot/usr/lib/../lib64/libc.a

And therefore decides that ARCH_LIB_DIR is "lib64", and therefore looks only in
<sysroot>/lib64 for libraries and not in <sysroot>/lib.

Is this a recent change in Crosstool-NG ? The old Crosstool-NG toolchain we
have at
http://autobuild.buildroot.net/toolchains/tarballs/x86_64-ctng_locales-linux-gnu.tar.xz
has everything in <sysroot>/lib, with <sysroot>/lib64 being a symbolic link to
<sysroot>/lib.

So, are we looking at a Buildroot bug, or a Crosstool-NG bug?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #6 from sir.ferdek+buildroot at gmail.com ---
(In reply to Thomas Petazzoni from comment #5)
> So, are we looking at a Buildroot bug, or a Crosstool-NG bug?

Looks like this is known and ongoing development. I have found a recent
activity regarding this issue in Crostool-NG.
https://github.com/crosstool-ng/crosstool-ng/pull/639

And then, look at this commit message from 12 days ago (included in merged pull
request):
https://github.com/crosstool-ng/crosstool-ng/pull/639/commits/63b2a19de4b33f304b236850028b6c6f32ce51e2

I guess it might require both Buildroot and CT-NG to synchronize on this
issue... :D

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #7 from Thomas De Schampheleire <patrickdepinguin@gmail.com> ---
In the mentioned ct-ng issue I see they are 'working on' a buildroot patch.
However, my toolchain patches already make a fair amount of changes to the
logic, and it makes no sense that other toolchain patches would now be
developed unless based on top of my changes. Would it be possible to apply my
patches first, then see what are the remaining issues for some toolchains?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at buildroot.uclibc |patrickdepinguin at gmail.com
                   |.org                        |

--- Comment #8 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> ---
I tested with the latest Buildroot (which includes the changes Thomas DS was
mentioning in his comment), and indeed the separation of lib/ (for gcc
libraries) and lib64/ has the consequence that libgcc_s.so is not copied to
staging nor to target.

The proposal to include "lib" in the list of folders from the sysroot to copy
seems sensible to me, perhaps we should add "usr/lib" as well.

Thomas DS, any comment on this ?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #9 from Carlos Santos <unixmania@gmail.com> ---
This bug has been dormant for more that one year. Is it still relevant now that
both buildroot's patch 725387 and ct-NG'a PR 639 were merged?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 9771] toolchain-external misses lib directory when copying sysroot

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=9771

--- Comment #10 from Thomas De Schampheleire <patrickdepinguin@gmail.com> ---
Sorry, it seems I never saw comment #8.

Copying the entire 'lib' from the external toolchain's sysroot into
staging/target will copy too much for some multilib toolchains, e.g. similar to
the octeon toolchain described in http://patchwork.ozlabs.org/patch/725387/ but
with 'lib/octeon2' instead of 'lib32/octeon2'. I think there are also other
toolchains where this will be a problem.

So, if the original problem is still an issue with current Buildroot and
reasonably recent toolchains, then I think a better solution should be found.

-- 
You are receiving this mail because:
You are on the CC list for the bug.