[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

Hi,

I have an issue with the integration of an external toolchain into buildroot. When I do this,
buildroot complains "Incorrect selection of the C library".

Digging for the root case of the problem, I found that SYSROOT_DIR is not calculated correctly
in toolchain/toolchain-external/ext-tool.mk. To fix the problem, I had to replace line 272 of that file:

SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a) |sed -r -e 's:usr/lib/libc\.a::;'` ; \

with

SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a) |sed -r -e 's:lib/libc\.a::;'` ; \

As I'm just new to buildroot, I'm not sure if this is a bug or if this fix is just needed due to an uncommon
packaging of my external toolchain. I'm using the latest buildroot release, i.e. buildroot-2011.11. The
(downstripped) directory structure of my toolchain looks like this:

linux_tools
linux_tools/man
linux_tools/man/man7
linux_tools/man/man1
linux_tools/lib
linux_tools/lib/gcc
linux_tools/lib/gcc/i686-v42-linux
linux_tools/lib/gcc/i686-v42-linux/4.2.3
linux_tools/lib/gcc/i686-v42-linux/4.2.3/install-tools
linux_tools/lib/gcc/i686-v42-linux/4.2.3/install-tools/include
linux_tools/lib/gcc/i686-v42-linux/4.2.3/include
linux_tools/lib/gcc/i686-v42-linux/4.2.3/include/ssp
linux_tools/libexec
linux_tools/libexec/gcc
linux_tools/libexec/gcc/i686-v42-linux
linux_tools/libexec/gcc/i686-v42-linux/4.2.3
linux_tools/libexec/gcc/i686-v42-linux/4.2.3/install-tools
linux_tools/info
linux_tools/bin
linux_tools/i686-v42-linux
linux_tools/i686-v42-linux/lib
linux_tools/i686-v42-linux/lib/ldscripts
linux_tools/i686-v42-linux/lib/gconv
linux_tools/i686-v42-linux/libexec
linux_tools/i686-v42-linux/libexec/getconf
linux_tools/i686-v42-linux/bin
linux_tools/i686-v42-linux/sbin
linux_tools/i686-v42-linux/include
linux_tools/i686-v42-linux/etc
linux_tools/i686-v42-linux/share
linux_tools/share

and my PATH / PREFIX setting in menuconfig is: /someDirectory/linux_tools / $(ARCH)-v42 with $(ARCH) == i686.

Can anyone help in telling me if I found (and fixed? :-)) a bug in buildroot or if something is wrong with my toolchain,
and if so, what's wrong?

Thanks for any help,

Chris


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120201/e882c713/attachment-0001.html>

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Thomas Petazzoni]

Hello Christof,

Le Wed, 1 Feb 2012 09:35:06 +0100,
"Warlich, Christof" <christof.warlich@siemens.com> a ?crit :

> I have an issue with the integration of an external toolchain into buildroot. When I do this,
> buildroot complains "Incorrect selection of the C library".
> 
> Digging for the root case of the problem, I found that SYSROOT_DIR is not calculated correctly
> in toolchain/toolchain-external/ext-tool.mk. To fix the problem, I had to replace line 272 of that file:
> 
> SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a) |sed -r -e 's:usr/lib/libc\.a::;'` ; \
> 
> with
> 
> SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a) |sed -r -e 's:lib/libc\.a::;'` ; \

Interesting. In all external toolchains I have seen until now, the
dynamic version of libc is in lib/, but the static version is in
usr/lib/.

If we just do your change, then it will basically break the support for
all other external toolchains. We need to be a little bit smarter.

> and my PATH / PREFIX setting in menuconfig is: /someDirectory/linux_tools / $(ARCH)-v42 with $(ARCH) == i686.
> 
> Can anyone help in telling me if I found (and fixed? :-)) a bug in buildroot or if something is wrong with my toolchain,
> and if so, what's wrong?

The directory organization of the various external toolchains available
is such that Buildroot does not necessarily support any random
toolchain.

Would it be possible to get your external toolchain, so that I could do
some testing, and integrate it in my set of external toolchains, so
that we keep it working in the future?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

Hi Thomas, 

> Would it be possible to get your external toolchain, so that I could do
> some testing, and integrate it in my set of external toolchains, so
> that we keep it working in the future?

Thanks a lot for the offer, but I doubt that it is worth the effort.

Cheers,

Chris

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Thomas Petazzoni]

Le Wed, 1 Feb 2012 12:20:02 +0100,
"Warlich, Christof" <christof.warlich@siemens.com> a ?crit :

> > Would it be possible to get your external toolchain, so that I could do
> > some testing, and integrate it in my set of external toolchains, so
> > that we keep it working in the future?
> 
> Thanks a lot for the offer, but I doubt that it is worth the effort.

Well, if we can't test it, it's going to be hard to improve Buildroot
to support this type of toolchain. If you have solved your specific
problem, it's great, but it would have been ever better to improve the
mainline Buildroot so that other users in your case would find things
working nicely :)

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

Hi Thomas,

> > Thanks a lot for the offer, but I doubt that it is worth the effort.
 
> Well, if we can't test it, it's going to be hard to improve Buildroot
> to support this type of toolchain. If you have solved your specific
> problem, it's great, but it would have been ever better to improve the
> mainline Buildroot so that other users in your case would find things
> working nicely :)

I fully agree with you, but when I talked about the amount of effort, I
was referring to the effort that it would take me to convince management
to give away that code. I know from earlier experience how much (useless)
energy that may take.

Considering the hard and good work that you guys are offering for free,
all I can do for now is to sincerely apologize. I would however certainly
be willing to discuss and test any more generic patch if you are still
Interested to put further work into the issue despite the unpleasent
circunstances.

Again, sorry for not being as helpful as I would like to be,

Chris
 

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Thomas Petazzoni]

Le Wed, 1 Feb 2012 13:30:01 +0100,
"Warlich, Christof" <christof.warlich@siemens.com> a ?crit :

> I fully agree with you, but when I talked about the amount of effort, I
> was referring to the effort that it would take me to convince management
> to give away that code. I know from earlier experience how much (useless)
> energy that may take.

Ok. Do your management realize that a toolchain is mostly composed of
gcc, binutils, uClibc and gdb, that all of these are distributed under
GPL or LGPL licenses, and therefore you have to keep the source code
open for those components in order to comply with their licenses?

> Considering the hard and good work that you guys are offering for free,
> all I can do for now is to sincerely apologize. I would however certainly
> be willing to discuss and test any more generic patch if you are still
> Interested to put further work into the issue despite the unpleasent
> circunstances.
> 
> Again, sorry for not being as helpful as I would like to be,

Can you try:

	sed -r -e 's:(usr/)?lib/libc\.a::'

This will make the usr/ part at the beginning optional. It seems to do
the job here:

thomas at skate:~$ echo "foo/bar/test/lib/libc.a" | sed -r -e 's:(usr/)?lib/libc\.a::'
foo/bar/test/
thomas at skate:~$ echo "foo/bar/test/usr/lib/libc.a" | sed -r -e 's:(usr/)?lib/libc\.a::'
foo/bar/test/

If you just make that change, does all the rest of the external
toolchain logic works for you?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Thomas De Schampheleire]

Hi,

On Wed, Feb 1, 2012 at 9:35 AM, Warlich, Christof
<christof.warlich@siemens.com> wrote:
> Hi,
>
> I have an issue with the integration of an external toolchain into
> buildroot. When I do this,
> buildroot complains "Incorrect selection of the C library".
>
> Digging for the root case of the problem, I found that SYSROOT_DIR is not
> calculated correctly
> in toolchain/toolchain-external/ext-tool.mk. To fix the problem, I had to
> replace line 272 of that file:
>
> SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC)
> -print-file-name=libc.a) |sed -r -e 's:usr/lib/libc\.a::;'` ; \
>
> with
>
> SYSROOT_DIR=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC)
> -print-file-name=libc.a) |sed -r -e 's:lib/libc\.a::;'` ; \
>
> As I'm just new to buildroot, I'm not sure if this is a bug or if this fix
> is just needed due to an uncommon
> packaging of my external toolchain. I'm using the latest buildroot release,
> i.e. buildroot-2011.11. The
> (downstripped) directory structure of my toolchain looks like this:
>
> linux_tools
> linux_tools/man
> linux_tools/man/man7
> linux_tools/man/man1
> linux_tools/lib
> linux_tools/lib/gcc
> linux_tools/lib/gcc/i686-v42-linux
> linux_tools/lib/gcc/i686-v42-linux/4.2.3
> linux_tools/lib/gcc/i686-v42-linux/4.2.3/install-tools
> linux_tools/lib/gcc/i686-v42-linux/4.2.3/install-tools/include
> linux_tools/lib/gcc/i686-v42-linux/4.2.3/include
> linux_tools/lib/gcc/i686-v42-linux/4.2.3/include/ssp
> linux_tools/libexec
> linux_tools/libexec/gcc
> linux_tools/libexec/gcc/i686-v42-linux
> linux_tools/libexec/gcc/i686-v42-linux/4.2.3
> linux_tools/libexec/gcc/i686-v42-linux/4.2.3/install-tools
> linux_tools/info
> linux_tools/bin
> linux_tools/i686-v42-linux
> linux_tools/i686-v42-linux/lib
> linux_tools/i686-v42-linux/lib/ldscripts
> linux_tools/i686-v42-linux/lib/gconv
> linux_tools/i686-v42-linux/libexec
> linux_tools/i686-v42-linux/libexec/getconf
> linux_tools/i686-v42-linux/bin
> linux_tools/i686-v42-linux/sbin
> linux_tools/i686-v42-linux/include
> linux_tools/i686-v42-linux/etc
> linux_tools/i686-v42-linux/share
> linux_tools/share
>
> and my PATH / PREFIX setting in menuconfig is: /someDirectory/linux_tools /
> $(ARCH)-v42 with $(ARCH) == i686.
>
> Can anyone help in telling me if I found (and fixed? :-)) a bug in buildroot
> or if something is wrong with my toolchain,
> and if so, what's wrong?

How is this toolchain built? Or more importantly, how is it packaged?

Isn't this problem simply caused by the fact that the root of your
packaged toolchain is already inside 'usr/', instead of just above it?

In my case, I build a crosstool-ng toolchain with buildroot. I then
package it by going to output/host and tarring the entire usr/
directory. This way, the toolchain has a structure like (stripped):

usr/lib/gcc/
usr/lib/gcc/powerpc-unknown-linux-gnu/
usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/
usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/include/
usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/include/stdfix.h
...
usr/sbin/
usr/libexec/
usr/libexec/awk/
usr/libexec/awk/pwcat
usr/libexec/awk/grcat
usr/libexec/gcc/
usr/libexec/gcc/powerpc-unknown-linux-gnu/
usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/
usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/
usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/fixinc.sh
usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/mkheaders
...

I think your problem would be fixed if you change the way you package
the toolchain.

Of course, if this toolchain came to you pre-packaged, it may be
better to fix buildroot to find everything automatically.

Best regards,
Thomas

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Grant Edwards]

On 2012-02-01, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 1 Feb 2012 13:30:01 +0100,
> "Warlich, Christof" <christof.warlich@siemens.com> a ??crit :
>
>> I fully agree with you, but when I talked about the amount of effort, I
>> was referring to the effort that it would take me to convince management
>> to give away that code. I know from earlier experience how much (useless)
>> energy that may take.
>
> Ok. Do your management realize that a toolchain is mostly composed of
> gcc, binutils, uClibc and gdb, that all of these are distributed under
> GPL or LGPL licenses, and therefore you have to keep the source code
> open for those components in order to comply with their licenses?

For most of the toolchain, that's only true if they distribute
binaries.  Which they apparently do not do.  As long as they don't
distribute binaries of gcc, binutils, gdb, then they can keep the
sources to themselves can't they?

-- 
Grant Edwards               grant.b.edwards        Yow! I have a VISION!  It's
                                  at               a RANCID double-FISHWICH on
                              gmail.com            an ENRICHED BUN!!

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

> > I fully agree with you, but when I talked about the amount of effort, I
> > was referring to the effort that it would take me to convince management
> > to give away that code. I know from earlier experience how much (useless)
> > energy that may take.

> Ok. Do your management realize that a toolchain is mostly composed of
> gcc, binutils, uClibc and gdb, that all of these are distributed under
> GPL or LGPL licenses, and therefore you have to keep the source code
> open for those components in order to comply with their licenses?

I think they do, as the source _is_ passed to our customer. The legal
department has obvoiusly forced them to do so. But I guess the propriarity
mindset is still so deeply engraved in those heads that they still think
it may be better to restrict the audience as much as (legally) possible,
although the customer in turn would (according to the GPL) be allowed to
pass those sources to the public anyway. Again, I'm really sorry for that.
And anyhow, don't mind, the toolchain would not be of much use without
the maching (patched) linux kernel.

> > Considering the hard and good work that you guys are offering for free,
> > all I can do for now is to sincerely apologize. I would however certainly
> > be willing to discuss and test any more generic patch if you are still
> > Interested to put further work into the issue despite the unpleasent
> > circunstances.
> > 
> > Again, sorry for not being as helpful as I would like to be,

> Can you try:
> 
> 	sed -r -e 's:(usr/)?lib/libc\.a::'
> 
> This will make the usr/ part at the beginning optional. It seems to do
> the job here:
> 
> thomas at skate:~$ echo "foo/bar/test/lib/libc.a" | sed -r -e 's:(usr/)?lib/libc\.a::'
> foo/bar/test/
> thomas at skate:~$ echo "foo/bar/test/usr/lib/libc.a" | sed -r -e 's:(usr/)?lib/libc\.a::'
> foo/bar/test/
> 
> If you just make that change, does all the rest of the external
> Toolchain logic works for you?

Yes, that works like a charm with our toolchain as well.

Thanks again for your generous support,

Chris

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

Hi Thomas,

> How is this toolchain built?

I can't say as I got the toolchain from another department. 

> Or more importantly, how is it packaged?
> 
> Isn't this problem simply caused by the fact that the root of your
> packaged toolchain is already inside 'usr/', instead of just above it?
> 
> In my case, I build a crosstool-ng toolchain with buildroot. I then
> package it by going to output/host and tarring the entire usr/
> directory. This way, the toolchain has a structure like (stripped):
> 
> usr/lib/gcc/
> usr/lib/gcc/powerpc-unknown-linux-gnu/
> usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/
> usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/include/
> usr/lib/gcc/powerpc-unknown-linux-gnu/4.4.3/include/stdfix.h
> ...
> usr/sbin/
> usr/libexec/
> usr/libexec/awk/
> usr/libexec/awk/pwcat
> usr/libexec/awk/grcat
> usr/libexec/gcc/
> usr/libexec/gcc/powerpc-unknown-linux-gnu/
> usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/
> usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/
> usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/fixinc.sh
> usr/libexec/gcc/powerpc-unknown-linux-gnu/4.4.3/install-tools/mkheaders
> ...
> 
> I think your problem would be fixed if you change the way you package
> the toolchain.

Anyhow, the fix suggested by Thomas Petazzoni covers both :-).

> Of course, if this toolchain came to you pre-packaged, it may be
> better to fix buildroot to find everything automatically.

:-)

Cheers,

Chris

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Warlich, Christof]

Hi Grant, 

> For most of the toolchain, that's only true if they distribute
> binaries.  Which they apparently do not do.  As long as they don't
> distribute binaries of gcc, binutils, gdb, then they can keep the
> sources to themselves can't they?

Maybe. I'm so happy for not having to to spend my time as a lawyer :-).

Cheers,

Chris

[Buildroot] External Toolchain: Calculation of SYSROOT_DIR in toolchain/toolchain-external/ext-tool.mk wrong?

[Michael S. Zick]

On Wed February 1 2012, Warlich, Christof wrote:
> And anyhow, don't mind, the toolchain would not be of much use without
> the maching (patched) linux kernel.
> 

Which your legal department __also__ must distribute, its GPLv2

Mike