Exploring LLVM/Clang for OE

Exploring LLVM/Clang for OE

[Muhammad Tauqir Ahmad]

Hi,

I am trying to explore the possibility of using LLVM+Clang for
building OE-core images.
The first objective might be to try to build (poky-tiny's)
core-image-minimal? That is the smallest image we know of so maybe
that's a good starting point?

Right now, I am looking at how to build LLVM+Clang cross-compiler
using bitbake and am trying to adapt the llvm3.3 recipe from meta-oe.
As far as I can tell, it doesn't build any tools (doesn't build Clang)
so I am looking at how to modify it to build Clang as well or maybe
just write a new recipe for clang3.3.

The email is meant as an RFC. I would appreciate any pointers and help
I could get with this.
I would also be interested in finding out if and how much work has
been done to try to use LLVM+Clang for OE in the past. If, e.g.,
someone already has the bitbake recipes to get me started and getting
some basic userspace packages to build with LLVM+Clang, it would be
really helpful and avoid duplicate work.
If someone is already working on this, I could work with them.

Any other general or specific comments would also be helpful.

Thanks.

Re: Exploring LLVM/Clang for OE

[Muhammad Tauqir Ahmad]

According to the link below [1], it seems some works has been done at
least for llvm+clang bitbake recipes, although I couldn't find any
clang recipes when searching the net. If anyone can point me in the
right direction for finding these recipes, that would be great.

[1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html

Muhammad Tauqir Ahmad
--------------------------------------
Software Developer Intern at WindRiver Systems
Candidate for Honours Computer Science
Combinatorics and Optimizations Minor
University of Waterloo


On Tue, Nov 12, 2013 at 8:40 PM, Muhammad Tauqir Ahmad
<mian.muhammad.tauqir.ahmad@gmail.com> wrote:
> Hi,
>
> I am trying to explore the possibility of using LLVM+Clang for
> building OE-core images.
> The first objective might be to try to build (poky-tiny's)
> core-image-minimal? That is the smallest image we know of so maybe
> that's a good starting point?
>
> Right now, I am looking at how to build LLVM+Clang cross-compiler
> using bitbake and am trying to adapt the llvm3.3 recipe from meta-oe.
> As far as I can tell, it doesn't build any tools (doesn't build Clang)
> so I am looking at how to modify it to build Clang as well or maybe
> just write a new recipe for clang3.3.
>
> The email is meant as an RFC. I would appreciate any pointers and help
> I could get with this.
> I would also be interested in finding out if and how much work has
> been done to try to use LLVM+Clang for OE in the past. If, e.g.,
> someone already has the bitbake recipes to get me started and getting
> some basic userspace packages to build with LLVM+Clang, it would be
> really helpful and avoid duplicate work.
> If someone is already working on this, I could work with them.
>
> Any other general or specific comments would also be helpful.
>
> Thanks.

Re: Exploring LLVM/Clang for OE

[Randy MacLeod]

On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
<mian.muhammad.tauqir.ahmad@gmail.com> wrote:
> According to the link below [1], it seems some works has been done at
> least for llvm+clang bitbake recipes, although I couldn't find any
> clang recipes when searching the net. If anyone can point me in the
> right direction for finding these recipes, that would be great.
>
> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html


That would be Behan's work, adding him here.

Behan,
Are your clang recipes from, "Worked on bitbake recipes for LLVM and
clang" mentioned in the March 2013 llvmlinux notes available?

Everyone,
Where should Muhammad stage his work and is there an example of using
an alternative cross-compiler.

../Randy


>
> Muhammad Tauqir Ahmad
> --------------------------------------
> Software Developer Intern at WindRiver Systems
> Candidate for Honours Computer Science
> Combinatorics and Optimizations Minor
> University of Waterloo
>
>
> On Tue, Nov 12, 2013 at 8:40 PM, Muhammad Tauqir Ahmad
> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>> Hi,
>>
>> I am trying to explore the possibility of using LLVM+Clang for
>> building OE-core images.
>> The first objective might be to try to build (poky-tiny's)
>> core-image-minimal? That is the smallest image we know of so maybe
>> that's a good starting point?
>>
>> Right now, I am looking at how to build LLVM+Clang cross-compiler
>> using bitbake and am trying to adapt the llvm3.3 recipe from meta-oe.
>> As far as I can tell, it doesn't build any tools (doesn't build Clang)
>> so I am looking at how to modify it to build Clang as well or maybe
>> just write a new recipe for clang3.3.
>>
>> The email is meant as an RFC. I would appreciate any pointers and help
>> I could get with this.
>> I would also be interested in finding out if and how much work has
>> been done to try to use LLVM+Clang for OE in the past. If, e.g.,
>> someone already has the bitbake recipes to get me started and getting
>> some basic userspace packages to build with LLVM+Clang, it would be
>> really helpful and avoid duplicate work.
>> If someone is already working on this, I could work with them.
>>
>> Any other general or specific comments would also be helpful.
>>
>> Thanks.
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
../Randy/..

Re: Exploring LLVM/Clang for OE

[Otavio Salvador]

On Wed, Nov 13, 2013 at 12:47 PM, Randy MacLeod <rwmacleod@gmail.com> wrote:
> On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>> According to the link below [1], it seems some works has been done at
>> least for llvm+clang bitbake recipes, although I couldn't find any
>> clang recipes when searching the net. If anyone can point me in the
>> right direction for finding these recipes, that would be great.
>>
>> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html
>
>
> That would be Behan's work, adding him here.
>
> Behan,
> Are your clang recipes from, "Worked on bitbake recipes for LLVM and
> clang" mentioned in the March 2013 llvmlinux notes available?
>
> Everyone,
> Where should Muhammad stage his work and is there an example of using
> an alternative cross-compiler.

I'd say a github repository for now. This allow for easy forking,
contribution, provides issue tracking, wiki and so on.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Exploring LLVM/Clang for OE

[Muhammad Tauqir Ahmad]

I couldn't find any up-to-date poky mirrors on github so I pushed my
own poky "mirror"[1]. Once I clean up my code, I will push it to my
repo.

I am also finally getting clang3.3 cross-compiler to compile using a
bitbake recipe quickly put together from the llvm3.3 recipe from
meta-oe.
The next step I guess would be to try to figure how to use it and for
the builds.
Any pointers/directions/examples with/for that would be helpful
especially if someone has already done some work along those lines.

[1]: https://github.com/mtahmed/poky
Muhammad Tauqir Ahmad
--------------------------------------
Software Developer Intern at WindRiver Systems
Candidate for Honours Computer Science
Combinatorics and Optimizations Minor
University of Waterloo


On Wed, Nov 13, 2013 at 11:01 AM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Wed, Nov 13, 2013 at 12:47 PM, Randy MacLeod <rwmacleod@gmail.com> wrote:
>> On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
>> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>>> According to the link below [1], it seems some works has been done at
>>> least for llvm+clang bitbake recipes, although I couldn't find any
>>> clang recipes when searching the net. If anyone can point me in the
>>> right direction for finding these recipes, that would be great.
>>>
>>> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html
>>
>>
>> That would be Behan's work, adding him here.
>>
>> Behan,
>> Are your clang recipes from, "Worked on bitbake recipes for LLVM and
>> clang" mentioned in the March 2013 llvmlinux notes available?
>>
>> Everyone,
>> Where should Muhammad stage his work and is there an example of using
>> an alternative cross-compiler.
>
> I'd say a github repository for now. This allow for easy forking,
> contribution, provides issue tracking, wiki and so on.
>
> --
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Exploring LLVM/Clang for OE

[Muhammad Tauqir Ahmad]

On Sun, Nov 17, 2013 at 12:39 PM, Behan Webster <behanw@gmail.com> wrote:
> On 11/13/13 06:47, Randy MacLeod wrote:
>> On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
>> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>>> According to the link below [1], it seems some works has been done at
>>> least for llvm+clang bitbake recipes, although I couldn't find any
>>> clang recipes when searching the net. If anyone can point me in the
>>> right direction for finding these recipes, that would be great.
>>>
>>> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html
>>
>> That would be Behan's work, adding him here.
>>
>> Behan,
>> Are your clang recipes from, "Worked on bitbake recipes for LLVM and
>> clang" mentioned in the March 2013 llvmlinux notes available?
> Khem Raj's work on recipes for llvm and clang were way better than mine,
> so I just use his now from meta-oe

I only found recipes for llvm in meta-oe. Didn't find any for clang. I
am in the process of writing one for clang and will push it to my
github once it's clean enough.

>
>> Everyone,
>> Where should Muhammad stage his work and is there an example of using
>> an alternative cross-compiler.
> Essentially the idea right now is to provide your own do_compile() which
> does whatever is necessary to use the other compiler. There may be a
> better way to do it, but I haven't found it yet.
>

Currently I am exploring using the secondary toolchain method
(http://www.openembedded.org/wiki/Adding_a_secondary_toolchain) as
suggested by fray (Mark Hatle).
Initially I will pre-compile a clang cross-compiler and use that as
the secondary toolchain and hopefully get as much compiled as possible
with the secondary llvm toolchain.
Then I can move to using a clang recipe to build the secondary
toolchain as well.

> Behan
>
> --
> Behan Webster
> behanw@gmail.com
>

Muhammad Tauqir Ahmad
--------------------------------------
Software Developer Intern at WindRiver Systems
Candidate for Honours Computer Science
Combinatorics and Optimizations Minor
University of Waterloo

Re: Exploring LLVM/Clang for OE

[Muhammad Tauqir Ahmad]

Updates:
- I am able to compile most of the packages using clang except the
kernel, gcc and a few others. The ones that don't compile are
mentioned in the README.md in the github repositories listed below.
- The busybox still doesn't seem to be compiling correctly even with
the patch (although it does successfully compile) because the
clang-built busybox causes a kernel panic.
- elfutils is failing to build with clang despite the few patches that I added.

TODO:
- Need to add the set of patches to the kernel to compile it using clang.
- Investigate compiling the failing packages using clang.
- Investigate busybox runtime failure when built with clang

Any help with any of these TODOs would be helpful.

I am hosting my work on my github account in these two repositories:

https://github.com/mtahmed/poky-clang
https://github.com/mtahmed/meta-tc-llvm

The first is upstream poky clone with additional patches to allow it
to compile using clang.
The second is a secondary toolchain layer that allows llvm to be used
for the secondary toolchain.

Muhammad Tauqir Ahmad
--------------------------------------
Software Developer Intern at WindRiver Systems
Candidate for Honours Computer Science
Combinatorics and Optimizations Minor
University of Waterloo


On Mon, Nov 18, 2013 at 8:35 AM, Muhammad Tauqir Ahmad
<mian.muhammad.tauqir.ahmad@gmail.com> wrote:
> On Sun, Nov 17, 2013 at 12:39 PM, Behan Webster <behanw@gmail.com> wrote:
>> On 11/13/13 06:47, Randy MacLeod wrote:
>>> On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
>>> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>>>> According to the link below [1], it seems some works has been done at
>>>> least for llvm+clang bitbake recipes, although I couldn't find any
>>>> clang recipes when searching the net. If anyone can point me in the
>>>> right direction for finding these recipes, that would be great.
>>>>
>>>> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html
>>>
>>> That would be Behan's work, adding him here.
>>>
>>> Behan,
>>> Are your clang recipes from, "Worked on bitbake recipes for LLVM and
>>> clang" mentioned in the March 2013 llvmlinux notes available?
>> Khem Raj's work on recipes for llvm and clang were way better than mine,
>> so I just use his now from meta-oe
>
> I only found recipes for llvm in meta-oe. Didn't find any for clang. I
> am in the process of writing one for clang and will push it to my
> github once it's clean enough.
>
>>
>>> Everyone,
>>> Where should Muhammad stage his work and is there an example of using
>>> an alternative cross-compiler.
>> Essentially the idea right now is to provide your own do_compile() which
>> does whatever is necessary to use the other compiler. There may be a
>> better way to do it, but I haven't found it yet.
>>
>
> Currently I am exploring using the secondary toolchain method
> (http://www.openembedded.org/wiki/Adding_a_secondary_toolchain) as
> suggested by fray (Mark Hatle).
> Initially I will pre-compile a clang cross-compiler and use that as
> the secondary toolchain and hopefully get as much compiled as possible
> with the secondary llvm toolchain.
> Then I can move to using a clang recipe to build the secondary
> toolchain as well.
>
>> Behan
>>
>> --
>> Behan Webster
>> behanw@gmail.com
>>
>
> Muhammad Tauqir Ahmad
> --------------------------------------
> Software Developer Intern at WindRiver Systems
> Candidate for Honours Computer Science
> Combinatorics and Optimizations Minor
> University of Waterloo

Re: Exploring LLVM/Clang for OE

[Otavio Salvador]

On Wed, Dec 4, 2013 at 2:31 PM, Muhammad Tauqir Ahmad
<mian.muhammad.tauqir.ahmad@gmail.com> wrote:
...
> I am hosting my work on my github account in these two repositories:
>
> https://github.com/mtahmed/poky-clang
> https://github.com/mtahmed/meta-tc-llvm
>
> The first is upstream poky clone with additional patches to allow it
> to compile using clang.
> The second is a secondary toolchain layer that allows llvm to be used
> for the secondary toolchain.

Nice work! It would be nice if you could isolate the ready for merge
patches, from Poky, and send them so the review process can be done
along the finishing of the other TODO items.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: Exploring LLVM/Clang for OE

[Bruce Ashfield]

On Wed, Dec 4, 2013 at 11:31 AM, Muhammad Tauqir Ahmad
<mian.muhammad.tauqir.ahmad@gmail.com> wrote:
> Updates:
> - I am able to compile most of the packages using clang except the
> kernel, gcc and a few others. The ones that don't compile are
> mentioned in the README.md in the github repositories listed below.
> - The busybox still doesn't seem to be compiling correctly even with
> the patch (although it does successfully compile) because the
> clang-built busybox causes a kernel panic.
> - elfutils is failing to build with clang despite the few patches that I added.
>
> TODO:
> - Need to add the set of patches to the kernel to compile it using clang.

This is something that I've already queued up, so it'll be taken care
of in time.

Bruce

> - Investigate compiling the failing packages using clang.
> - Investigate busybox runtime failure when built with clang
>
> Any help with any of these TODOs would be helpful.
>
> I am hosting my work on my github account in these two repositories:
>
> https://github.com/mtahmed/poky-clang
> https://github.com/mtahmed/meta-tc-llvm
>
> The first is upstream poky clone with additional patches to allow it
> to compile using clang.
> The second is a secondary toolchain layer that allows llvm to be used
> for the secondary toolchain.
>
> Muhammad Tauqir Ahmad
> --------------------------------------
> Software Developer Intern at WindRiver Systems
> Candidate for Honours Computer Science
> Combinatorics and Optimizations Minor
> University of Waterloo
>
>
> On Mon, Nov 18, 2013 at 8:35 AM, Muhammad Tauqir Ahmad
> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>> On Sun, Nov 17, 2013 at 12:39 PM, Behan Webster <behanw@gmail.com> wrote:
>>> On 11/13/13 06:47, Randy MacLeod wrote:
>>>> On Tue, Nov 12, 2013 at 8:47 PM, Muhammad Tauqir Ahmad
>>>> <mian.muhammad.tauqir.ahmad@gmail.com> wrote:
>>>>> According to the link below [1], it seems some works has been done at
>>>>> least for llvm+clang bitbake recipes, although I couldn't find any
>>>>> clang recipes when searching the net. If anyone can point me in the
>>>>> right direction for finding these recipes, that would be great.
>>>>>
>>>>> [1]: http://lists.linuxfoundation.org/pipermail/llvmlinux/2013-March/000205.html
>>>>
>>>> That would be Behan's work, adding him here.
>>>>
>>>> Behan,
>>>> Are your clang recipes from, "Worked on bitbake recipes for LLVM and
>>>> clang" mentioned in the March 2013 llvmlinux notes available?
>>> Khem Raj's work on recipes for llvm and clang were way better than mine,
>>> so I just use his now from meta-oe
>>
>> I only found recipes for llvm in meta-oe. Didn't find any for clang. I
>> am in the process of writing one for clang and will push it to my
>> github once it's clean enough.
>>
>>>
>>>> Everyone,
>>>> Where should Muhammad stage his work and is there an example of using
>>>> an alternative cross-compiler.
>>> Essentially the idea right now is to provide your own do_compile() which
>>> does whatever is necessary to use the other compiler. There may be a
>>> better way to do it, but I haven't found it yet.
>>>
>>
>> Currently I am exploring using the secondary toolchain method
>> (http://www.openembedded.org/wiki/Adding_a_secondary_toolchain) as
>> suggested by fray (Mark Hatle).
>> Initially I will pre-compile a clang cross-compiler and use that as
>> the secondary toolchain and hopefully get as much compiled as possible
>> with the secondary llvm toolchain.
>> Then I can move to using a clang recipe to build the secondary
>> toolchain as well.
>>
>>> Behan
>>>
>>> --
>>> Behan Webster
>>> behanw@gmail.com
>>>
>>
>> Muhammad Tauqir Ahmad
>> --------------------------------------
>> Software Developer Intern at WindRiver Systems
>> Candidate for Honours Computer Science
>> Combinatorics and Optimizations Minor
>> University of Waterloo
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"