openembedded with a prebuilt external toolchain

openembedded with a prebuilt external toolchain

[Joey Oravec]

Hi,

I'm trying to use openembedded with an external prebuilt toolchain. The 
docs at http://docs.openembedded.org/usermanual/usermanual.html mark 
this as TODO so maybe we can "do" that in this email chain. :)

Marvell provides a *.tar.gz with gcc and C libraries, which I've 
uncompressed somewhere in my home directory. My goal is to build a root 
filesystem with some assortment of packages, but at this point I'm just 
trying "bitbake nano" which is the very first example in the Getting 
Started. That seems like a logical start to board bring-up.

I'm assuming that I need to add some lines to build/conf/local.conf that 
tell OE to make no attempt at building its own compiler or C library (it 
can't anyway, it doesn't know my chip!) and to call my pre-built one 
instead. It looks like I'd want to add a few lines about TOOLCHAIN_*, 
but that uses a recipe for external toolchain, which assumes that it has 
some meta toolchain, and I'm floundering and getting nowhere. Couldn't 
find any of these magic variables documented in the usermanual so I've 
been digging through recipes to learn any of this.

Is it possible to add a few lines that tell OE the path and filename of 
my compiler? Or is it more complicated than that?

-joey

Re: openembedded with a prebuilt external toolchain

[Tom Rini]

On 02/10/2011 01:24 PM, Joey Oravec wrote:
> Hi,
>
> I'm trying to use openembedded with an external prebuilt toolchain. The
> docs at http://docs.openembedded.org/usermanual/usermanual.html mark
> this as TODO so maybe we can "do" that in this email chain. :)
>
> Marvell provides a *.tar.gz with gcc and C libraries, which I've
> uncompressed somewhere in my home directory. My goal is to build a root
> filesystem with some assortment of packages, but at this point I'm just
> trying "bitbake nano" which is the very first example in the Getting
> Started. That seems like a logical start to board bring-up.
>
> I'm assuming that I need to add some lines to build/conf/local.conf that
> tell OE to make no attempt at building its own compiler or C library (it
> can't anyway, it doesn't know my chip!) and to call my pre-built one
> instead. It looks like I'd want to add a few lines about TOOLCHAIN_*,
> but that uses a recipe for external toolchain, which assumes that it has
> some meta toolchain, and I'm floundering and getting nowhere. Couldn't
> find any of these magic variables documented in the usermanual so I've
> been digging through recipes to learn any of this.
>
> Is it possible to add a few lines that tell OE the path and filename of
> my compiler? Or is it more complicated than that?

We talked about this a little on IRC and I provided the example of how 
to use the CodeSourcery recipes that exist today.  The marvell toolchain 
however will need it's own special love.  To start with, using the CSL 
recipe is roughly:
### CSL public ARM does <arch>-none-linux, so fill in the -none.
#TOOLCHAIN_VENDOR = "-none"
### CSL public ia32 does i686-pc-linux, so fill in the -pc.
#TOOLCHAIN_VENDOR = "-pc"

#### Bring in external-toolchain-csl.bb
#TOOLCHAIN_BRAND = "csl"
#TOOLCHAIN_TYPE = "external"

### Where the CSL version is installed.
#TOOLCHAIN_PATH = "/scratch/ia32-2010.09"
### Where inside the CSL install the target libraries exist.
#TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/i686-pc-linux-gnu"
### Add tools to the PATH automatically.
#PATH_prepend = "${TOOLCHAIN_PATH}/bin:"

Now what you'll want to do is make an external-toolchain-marvell.bb and 
set TOOLCHAIN_BRAND to "marvell" and tweak TOOLCHAIN_VENDOR such that we 
go from what OE normally says TARGET_SYS is to what your tools actually 
say it is.  Then it's a matter of tweaking the contents of 
external-toolchain-marvell.bb to copy the right things out of the 
marvell install and into where OE wants them.  In this regard, the -csl 
recipe may be more instructive than the -generic recipe as the -generic 
recipe (along with external-toolchain.bb) are targeted at toolchains 
exported by OE and thus require little mangling to work.

-- 
Tom Rini
Mentor Graphics Corporation

Re: openembedded with a prebuilt external toolchain

[Joey Oravec]

On 2/10/2011 3:58 PM, Tom Rini wrote:
> We talked about this a little on IRC and I provided the example of how 
> to use the CodeSourcery recipes that exist today.  The marvell 
> toolchain however will need it's own special love.

Yes! Much appreciated!

> Now what you'll want to do is make an external-toolchain-marvell.bb 
> and set TOOLCHAIN_BRAND to "marvell" and tweak TOOLCHAIN_VENDOR such 
> that we go from what OE normally says TARGET_SYS is to what your tools 
> actually say it is.  Then it's a matter of tweaking the contents of 
> external-toolchain-marvell.bb to copy the right things out of the 
> marvell install and into where OE wants them.  In this regard, the 
> -csl recipe may be more instructive than the -generic recipe as the 
> -generic recipe (along with external-toolchain.bb) are targeted at 
> toolchains exported by OE and thus require little mangling to work.

Both the -csl.bb and -generic.bb copy a bunch of stuff to STAGING_DIR in 
do_stage(). Of course in the long-run I need for the recipe to copy C 
libraries, but for now I should be able to leave the body blank. At 
least it'll use my compiler and run to completion. Right?

That's what I did and it seems to be ok. The "bitbake nano" example 
failed, looks like it wasn't linking ncurses. Another attempt with 
"bitbake busybox" worked and I saw some packages in the output folder. 
Looks good so far!

-joey

Re: openembedded with a prebuilt external toolchain

[Tom Rini]

On 02/10/2011 04:22 PM, Joey Oravec wrote:
> On 2/10/2011 3:58 PM, Tom Rini wrote:
>> We talked about this a little on IRC and I provided the example of how
>> to use the CodeSourcery recipes that exist today. The marvell
>> toolchain however will need it's own special love.
>
> Yes! Much appreciated!
>
>> Now what you'll want to do is make an external-toolchain-marvell.bb
>> and set TOOLCHAIN_BRAND to "marvell" and tweak TOOLCHAIN_VENDOR such
>> that we go from what OE normally says TARGET_SYS is to what your tools
>> actually say it is. Then it's a matter of tweaking the contents of
>> external-toolchain-marvell.bb to copy the right things out of the
>> marvell install and into where OE wants them. In this regard, the -csl
>> recipe may be more instructive than the -generic recipe as the
>> -generic recipe (along with external-toolchain.bb) are targeted at
>> toolchains exported by OE and thus require little mangling to work.
>
> Both the -csl.bb and -generic.bb copy a bunch of stuff to STAGING_DIR in
> do_stage(). Of course in the long-run I need for the recipe to copy C
> libraries, but for now I should be able to leave the body blank. At
> least it'll use my compiler and run to completion. Right?

Well, as you saw, it fails in some cases.  This is because, yeah, we're 
telling the toolchain stuff lives ...here now, and it doesn't.

> That's what I did and it seems to be ok. The "bitbake nano" example
> failed, looks like it wasn't linking ncurses. Another attempt with
> "bitbake busybox" worked and I saw some packages in the output folder.
> Looks good so far!

busybox, being self contained is more likely to be OK.

-- 
Tom Rini
Mentor Graphics Corporation

Re: openembedded with a prebuilt external toolchain

[Joey Oravec]

On 2/10/2011 6:35 PM, Tom Rini wrote:
>> Both the -csl.bb and -generic.bb copy a bunch of stuff to STAGING_DIR in
>> do_stage(). Of course in the long-run I need for the recipe to copy C
>> libraries, but for now I should be able to leave the body blank. At
>> least it'll use my compiler and run to completion. Right?
>
> Well, as you saw, it fails in some cases.  This is because, yeah, 
> we're telling the toolchain stuff lives ...here now, and it doesn't.

Thanks for the additional help on chat! This followup is to share my 
results with the mailing list.

Regarding my "bitbake nano" failure, that was a symptom of a sysroot 
issue. An external cross compiler has some default sysroot where it 
looks for libraries. By default it looks like OE passes nothing to 
change this. As a result do_configure() didn't look for libraries in 
staging dir, decided that ncurses didn't exist, and the build failed. 
One solution is to add:

TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"

to your build/conf/local.conf and make sure your compiler's default 
sysroot (libc, etc) gets copied into the staging dir. That way you can 
call *-gcc from somewhere in your path but it'll look for everything 
under OE's staging directory.

-joey

Re: openembedded with a prebuilt external toolchain

[Denys Dmytriyenko]

On Mon, Feb 14, 2011 at 03:14:40PM -0500, Joey Oravec wrote:
> On 2/10/2011 6:35 PM, Tom Rini wrote:
>>> Both the -csl.bb and -generic.bb copy a bunch of stuff to STAGING_DIR in
>>> do_stage(). Of course in the long-run I need for the recipe to copy C
>>> libraries, but for now I should be able to leave the body blank. At
>>> least it'll use my compiler and run to completion. Right?
>>
>> Well, as you saw, it fails in some cases.  This is because, yeah, we're 
>> telling the toolchain stuff lives ...here now, and it doesn't.
>
> Thanks for the additional help on chat! This followup is to share my 
> results with the mailing list.
>
> Regarding my "bitbake nano" failure, that was a symptom of a sysroot issue. 
> An external cross compiler has some default sysroot where it looks for 
> libraries. By default it looks like OE passes nothing to change this. As a 
> result do_configure() didn't look for libraries in staging dir, decided 
> that ncurses didn't exist, and the build failed. One solution is to add:
>
> TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"

That is the default now in conf/bitbake.conf:

http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?h=org.openembedded.dev&id=ebd905685a339814c36bf658a8e58bf8f457f14e

> to your build/conf/local.conf and make sure your compiler's default sysroot 
> (libc, etc) gets copied into the staging dir. That way you can call *-gcc 
> from somewhere in your path but it'll look for everything under OE's 
> staging directory.

-- 
Denys