[meta-clang] LLVM shared library support

[meta-clang] LLVM shared library support

[Martin Kelly]

Hi,

I noticed that meta-clang builds only static LLVM libraries, unlike the 
meta-oe version of LLVM 3.3, which builds shared libraries. In fact, 
this appears to be the LLVM upstream default as well.

What is the reason for the difference? Shared versions of LLVM would be 
useful for reducing the size of programs linked to it and making smaller 
filesystem images.

Thanks,
Martin

Re: [meta-clang] LLVM shared library support

[Khem Raj]

On Fri, Mar 3, 2017 at 11:15 AM, Martin Kelly <mkelly@xevo.com> wrote:
> Hi,
>
> I noticed that meta-clang builds only static LLVM libraries, unlike the
> meta-oe version of LLVM 3.3, which builds shared libraries. In fact, this
> appears to be the LLVM upstream default as well.
>
> What is the reason for the difference? Shared versions of LLVM would be
> useful for reducing the size of programs linked to it and making smaller
> filesystem images.

meta-clang has only provided static compiler thus far, so using shared
or static did not matter as much in fact it could be helping with
faster compile
times that we see with clang. However, there is no binding to keep it static
link. Feel free to send a pull request to change it.

>
> Thanks,
> Martin

Re: [meta-clang] LLVM shared library support

[Khem Raj]

On Fri, Mar 3, 2017 at 12:09 PM, Khem Raj <raj.khem@gmail.com> wrote:
> On Fri, Mar 3, 2017 at 11:15 AM, Martin Kelly <mkelly@xevo.com> wrote:
>> Hi,
>>
>> I noticed that meta-clang builds only static LLVM libraries, unlike the
>> meta-oe version of LLVM 3.3, which builds shared libraries. In fact, this
>> appears to be the LLVM upstream default as well.
>>
>> What is the reason for the difference? Shared versions of LLVM would be
>> useful for reducing the size of programs linked to it and making smaller
>> filesystem images.
>
> meta-clang has only provided static compiler thus far, so using shared
> or static did not matter as much in fact it could be helping with
> faster compile
> times that we see with clang. However, there is no binding to keep it static
> link. Feel free to send a pull request to change it.

I did try this and I see a considerable slowdown in compiler speed. I changed
two things at a time so may be its the other

diff --git a/recipes-devtools/clang/clang_git.bb
b/recipes-devtools/clang/clang_git.bb
index 6c6e66a..7de2be1 100644
--- a/recipes-devtools/clang/clang_git.bb
+++ b/recipes-devtools/clang/clang_git.bb
@@ -73,7 +73,8 @@ EXTRA_OECMAKE_append_class-nativesdk = "\
                -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
 "
 EXTRA_OECMAKE_append_class-target = "\
-               -DLLVM_ENABLE_PIC=False \
+               -DBUILD_SHARED_LIBS=ON \
+               -DLLVM_ENABLE_PIC=ON \
                -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen \
                -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
                -DLLVM_TARGETS_TO_BUILD=${@get_clang_target_arch(bb, d)} \

>
>>
>> Thanks,
>> Martin

Re: [meta-clang] LLVM shared library support

[Martin Kelly]

On 03/09/2017 01:06 PM, Khem Raj wrote:
> On Fri, Mar 3, 2017 at 12:09 PM, Khem Raj <raj.khem@gmail.com> wrote:
>>
>> meta-clang has only provided static compiler thus far, so using shared
>> or static did not matter as much in fact it could be helping with
>> faster compile
>> times that we see with clang. However, there is no binding to keep it static
>> link. Feel free to send a pull request to change it.
>
> I did try this and I see a considerable slowdown in compiler speed. I changed
> two things at a time so may be its the other
>
> diff --git a/recipes-devtools/clang/clang_git.bb
> b/recipes-devtools/clang/clang_git.bb
> index 6c6e66a..7de2be1 100644
> --- a/recipes-devtools/clang/clang_git.bb
> +++ b/recipes-devtools/clang/clang_git.bb
> @@ -73,7 +73,8 @@ EXTRA_OECMAKE_append_class-nativesdk = "\
>                 -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
>  "
>  EXTRA_OECMAKE_append_class-target = "\
> -               -DLLVM_ENABLE_PIC=False \
> +               -DBUILD_SHARED_LIBS=ON \
> +               -DLLVM_ENABLE_PIC=ON \
>                 -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen \
>                 -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
>                 -DLLVM_TARGETS_TO_BUILD=${@get_clang_target_arch(bb, d)} \
>
>>

Those are the exact changes I have been testing with; 
-DLLVM_ENABLE_PIC=ON is certainly necessary to prevent QA errors 
regarding text relocation, so you can't really make the changes in 
isolation. That said, the change is still very useful, as there are some 
programs that use only shared libLLVM.so, and without them you can't 
really use them with bitbake.

I hope to have a pull request for you soon once I iron out the rest of 
the issues.

Re: [meta-clang] LLVM shared library support

[Khem Raj]

On Thu, Mar 9, 2017 at 1:32 PM, Martin Kelly <mkelly@xevo.com> wrote:
> On 03/09/2017 01:06 PM, Khem Raj wrote:
>>
>> On Fri, Mar 3, 2017 at 12:09 PM, Khem Raj <raj.khem@gmail.com> wrote:
>>>
>>>
>>> meta-clang has only provided static compiler thus far, so using shared
>>> or static did not matter as much in fact it could be helping with
>>> faster compile
>>> times that we see with clang. However, there is no binding to keep it
>>> static
>>> link. Feel free to send a pull request to change it.
>>
>>
>> I did try this and I see a considerable slowdown in compiler speed. I
>> changed
>> two things at a time so may be its the other
>>
>> diff --git a/recipes-devtools/clang/clang_git.bb
>> b/recipes-devtools/clang/clang_git.bb
>> index 6c6e66a..7de2be1 100644
>> --- a/recipes-devtools/clang/clang_git.bb
>> +++ b/recipes-devtools/clang/clang_git.bb
>> @@ -73,7 +73,8 @@ EXTRA_OECMAKE_append_class-nativesdk = "\
>>                 -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
>>  "
>>  EXTRA_OECMAKE_append_class-target = "\
>> -               -DLLVM_ENABLE_PIC=False \
>> +               -DBUILD_SHARED_LIBS=ON \
>> +               -DLLVM_ENABLE_PIC=ON \
>>                 -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen \
>>                 -DCLANG_TABLEGEN=${STAGING_BINDIR_NATIVE}/clang-tblgen \
>>                 -DLLVM_TARGETS_TO_BUILD=${@get_clang_target_arch(bb, d)} \
>>
>>>
>
> Those are the exact changes I have been testing with; -DLLVM_ENABLE_PIC=ON
> is certainly necessary to prevent QA errors regarding text relocation, so
> you can't really make the changes in isolation. That said, the change is
> still very useful, as there are some programs that use only shared
> libLLVM.so, and without them you can't really use them with bitbake.
>

slow down is 6x when using clang to compile on target, I dont know if such
a patch would be a good thing from compiler pov.

> I hope to have a pull request for you soon once I iron out the rest of the
> issues.

Re: [meta-clang] LLVM shared library support

[Martin Kelly]

On 03/09/2017 09:20 PM, Khem Raj wrote:
>
> slow down is 6x when using clang to compile on target, I dont know if such
> a patch would be a good thing from compiler pov.
>

I definitely understand the drawbacks of a slowdown. That said, some 
programs (such as the one I'm working on packaging) don't support static 
libLLVM at all, so without such a patch you cannot use them in Yocto, 
which is sad. In addition, using the shared version helps reduce disk 
and memory usage if you have multiple programs using it. These of course 
are always good things in an embedded context.

Would it be possible to somehow split the shared and static versions 
into separate recipes, or somehow trigger the choice from local 
configuration? That could provide the best of both worlds.