[meta-clang] oe-core and meta-clang llvm coexsting?

[meta-clang] oe-core and meta-clang llvm coexsting?

[Martin Kelly]

Hi,

I'm trying to integrate the meta-clang version of LLVM 4.0 (used for 
recipes that need a newer LLVM version) alongside the oe-core version of 
LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and some 
to use LLVM 3.3 and for them not to collide with each other.

Right now, once the meta-clang layer is enabled, the recipes collide 
because they both provide llvm-common.bb. I tried explicitly versioning 
llvm-common, which makes them no longer collide. However, mesa then 
fails to build because the meta-clang version of LLVM installs 
llvm-config into the native common sysroot in 
tmp/sysroots/x86_64-linux/usr/bin/llvm-config . This llvm-config is 
based on LLVM 4.0, so when mesa picks it up, it gets confused and fails 
to build.

I'm trying to figure out the right way to fix this and get around it. Do 
you have any suggestions? I imagine there might be a quick-and-dirty way 
to do this and a nicer, long-term way to do it. I'm interested in 
hearing both.

Thanks,
Martin

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Khem Raj]

[-- Attachment #1.1: Type: text/plain, Size: 1233 bytes --]



On 3/30/17 12:20 PM, Martin Kelly wrote:
> Hi,
> 
> I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
> recipes that need a newer LLVM version) alongside the oe-core version of
> LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and some
> to use LLVM 3.3 and for them not to collide with each other.
> 
> Right now, once the meta-clang layer is enabled, the recipes collide
> because they both provide llvm-common.bb. I tried explicitly versioning
> llvm-common, which makes them no longer collide. However, mesa then
> fails to build because the meta-clang version of LLVM installs
> llvm-config into the native common sysroot in

merging all llvm-common into a single recipe might be good

> tmp/sysroots/x86_64-linux/usr/bin/llvm-config . This llvm-config is
> based on LLVM 4.0, so when mesa picks it up, it gets confused and fails
> to build.
> 

rename llvm-config to something like llvm-config<ver>

> I'm trying to figure out the right way to fix this and get around it. Do
> you have any suggestions? I imagine there might be a quick-and-dirty way
> to do this and a nicer, long-term way to do it. I'm interested in
> hearing both.
> 
> Thanks,
> Martin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Martin Kelly]

On 03/30/2017 12:31 PM, Khem Raj wrote:
>
>
> On 3/30/17 12:20 PM, Martin Kelly wrote:
>> Hi,
>>
>> I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
>> recipes that need a newer LLVM version) alongside the oe-core version of
>> LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and some
>> to use LLVM 3.3 and for them not to collide with each other.
>>
>> Right now, once the meta-clang layer is enabled, the recipes collide
>> because they both provide llvm-common.bb. I tried explicitly versioning
>> llvm-common, which makes them no longer collide. However, mesa then
>> fails to build because the meta-clang version of LLVM installs
>> llvm-config into the native common sysroot in
>
> merging all llvm-common into a single recipe might be good

I agree that this would be a good idea. However, at this point the two 
llvm-configs have diverged and return very different sets of flags, so 
they are basically incompatible. Specifically, the oe-core version of 
LLVM returns a set of flags that are not the same as the native 
CPPFLAGS, CFLAGS, etc, while the meta-clang version of llvm-config 
returns a set of flags that are the exact same as CPPFLAGS, CFLAGS, etc. 
See below for the full flag settings in both versions.

Clearly, using CPPFLAGS, CFLAGS, etc. directly breaks cross-compilation. 
For this reason, I added wrapper logic to return TARGET_CPPFLAGS, 
TARGET_CFLAGS, etc. in the meta-clang script, which allowed it to 
properly cross-compile. However, this wrapper logic, and the set of 
flags that the meta-clang llvm-config returns, is incompatible with what 
mesa expects when it calls llvm-config.

AFAICT, this difference in flags boils down to differences between the 
LLVM autotools-based build and the CMake build. For example, in LLVM 
3.3, the autotools build set LLVM_LDFLAGS like this:

./tools/llvm-config/Makefile:50:	$(Verb) $(ECHO) 
's/@LLVM_LDFLAGS@/$(subst /,\/,$(SUB_LDFLAGS))/' \

and SUB_LDFLAGS is empty:

./tools/llvm-config/Makefile:36:SUB_LDFLAGS :=

As a result, in the autotools-based LLVM 3.3, we get:

#define LLVM_LDFLAGS ""

... while the CMake LLVM 4.0 returns the host LDFLAGS, thus 
necessitating the wrapper logic for proper cross-compilation.

In short, (this is just a guess), it appears that the CMake LLVM build 
doesn't do cross-compilation properly, while the autotools-based build does.

I think that's the problem. Fixing it is harder :). We can always 
version both libraries rather than merging llvm-config, but that's ugly 
and not a great long-term strategy.

Can you think of a cleaner way to do this?

Flags:
-----------------------------------------------------------------------

LLVM 3.3 oe-core BuildVariables.inc, after configure step:

#define LLVM_CPPFLAGS " -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS"
#define LLVM_CFLAGS " -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -fomit-frame-pointer -fPIC"
#define LLVM_LDFLAGS ""
#define LLVM_CXXFLAGS " -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -fomit-frame-pointer 
-fvisibility-inlines-hidden -fno-exceptions -fPIC -Woverloaded-virtual 
-Wcast-qual"
-----------------------------------------------------------------------

LLVM 4.0 meta-clang BuildVariables.inc, after configure step:

-----------------------------------------------------------------------
#define LLVM_CPPFLAGS "  -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS"
#define LLVM_CFLAGS " 
-isystem[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/include 
-O2 -pipe -fPIC -Wall -W -Wno-unused-parameter -Wwrite-strings 
-Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-comment 
-Werror=date-time -ffunction-sections -fdata-sections -DNDEBUG 
-D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS 
-D__STDC_LIMIT_MACROS"
#define LLVM_LDFLAGS " 
-isystem[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/include 
-O2 -pipe -D_GLIBCXX_USE_CXX11_ABI=0 
-L[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/lib 
-L[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/lib 
-Wl,-rpath-link,[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/lib 
-Wl,-rpath-link,[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/lib 
-Wl,-rpath,[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/lib 
-Wl,-rpath,[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/lib -Wl,-O1"
#define LLVM_CXXFLAGS " 
-isystem[snip]/build/qemux86-64/tmp/sysroots/x86_64-linux/usr/include 
-O2 -pipe -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -fvisibility-inlines-hidden 
-Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual 
-Wno-missing-field-initializers -pedantic -Wno-long-long 
-Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment 
-Werror=date-time -std=c++11 -ffunction-sections -fdata-sections 
-DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS"
-----------------------------------------------------------------------

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

On 30 March 2017 at 20:20, Martin Kelly <mkelly@xevo.com> wrote:

> I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
> recipes that need a newer LLVM version) alongside the oe-core version of
> LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and some to
> use LLVM 3.3 and for them not to collide with each other.
>

Not that presumably you mean meta-oe, not oe-core.

Ross

[-- Attachment #2: Type: text/html, Size: 822 bytes --]

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Martin Kelly]

On 03/31/2017 01:51 AM, Burton, Ross wrote:
>
> On 30 March 2017 at 20:20, Martin Kelly <mkelly@xevo.com
> <mailto:mkelly@xevo.com>> wrote:
>
>     I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
>     recipes that need a newer LLVM version) alongside the oe-core
>     version of LLVM 3.3 (used for mesa). I'd like some recipes to use
>     LLVM 4.0 and some to use LLVM 3.3 and for them not to collide with
>     each other.
>
>
> Not that presumably you mean meta-oe, not oe-core.
>
> Ross

Yes, typo :).

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Martin Kelly]

On 03/30/2017 04:58 PM, Martin Kelly wrote:
> On 03/30/2017 12:31 PM, Khem Raj wrote:
>>
>>
>> On 3/30/17 12:20 PM, Martin Kelly wrote:
>>> Hi,
>>>
>>> I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
>>> recipes that need a newer LLVM version) alongside the oe-core version of
>>> LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and some
>>> to use LLVM 3.3 and for them not to collide with each other.
>>>
>>> Right now, once the meta-clang layer is enabled, the recipes collide
>>> because they both provide llvm-common.bb. I tried explicitly versioning
>>> llvm-common, which makes them no longer collide. However, mesa then
>>> fails to build because the meta-clang version of LLVM installs
>>> llvm-config into the native common sysroot in
>>
>> merging all llvm-common into a single recipe might be good
>

I thought more about this issue. Absent some clever solution that allows 
us to unify the wrappers (which is difficult due to the issues I 
mentioned in the last mail), I think we have three routes that could 
work to at least allow meta-clang and meta-oe to exist in the same Yocto 
workspace without breakage:

(a) Instead of installing llvm-config into 
${SYSROOT_DESTDIR}${bindir_crossscripts} -- which causes collisions when 
meta-clang and meta-oe both try to overwrite the same shared file -- 
install into ${D}{bindir}llvm-config, and install the binary llvm-config 
into ${D}${bindir}/llvm-config${PV}. Recipes requiring LLVM will then 
need to depend on either llvm-common3.3 or llvm-common4.0, but at least 
there will be no collision or ambiguity which one gets used.

(b) Use a common wrapper in 
${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config that does what the 
meta-oe current version does; just exec 
${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config${WANT_LLVM_RELEASE}. 
Let the meta-oe LLVM 3.3 install the binary llvm-config into 
${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config3.3 and let the 
meta-clang LLVM 4.0 install its wrapper llvm-config into 
${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config4.0. meta-clang will 
have to use the second wrapper script because it needs to edit 
TARGET_CFLAGS and similar to avoid compiling target binaries using 
native flags (due to the CMake LLVM build which appears not to respect 
cross-compile).

(c) Use a common llvm-config wrapper in 
${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config that does a 
hardcoded check for WANT_LLVM_RELEASE. If the version is 4.0 or greater, 
then edit the TARGET_CFLAGS and similar, otherwise don't.

It seems to me that all these options are undesireable, but (a) is 
probably least so.

Khem and others: which option would you like me to pursue, or do you 
have an alternate suggestion?

Thanks,
Martin

Re: [meta-clang] oe-core and meta-clang llvm coexsting?

[Martin Kelly]

On 03/31/2017 03:53 PM, Martin Kelly wrote:
> On 03/30/2017 04:58 PM, Martin Kelly wrote:
>> On 03/30/2017 12:31 PM, Khem Raj wrote:
>>>
>>>
>>> On 3/30/17 12:20 PM, Martin Kelly wrote:
>>>> Hi,
>>>>
>>>> I'm trying to integrate the meta-clang version of LLVM 4.0 (used for
>>>> recipes that need a newer LLVM version) alongside the oe-core
>>>> version of
>>>> LLVM 3.3 (used for mesa). I'd like some recipes to use LLVM 4.0 and
>>>> some
>>>> to use LLVM 3.3 and for them not to collide with each other.
>>>>
>>>> Right now, once the meta-clang layer is enabled, the recipes collide
>>>> because they both provide llvm-common.bb. I tried explicitly versioning
>>>> llvm-common, which makes them no longer collide. However, mesa then
>>>> fails to build because the meta-clang version of LLVM installs
>>>> llvm-config into the native common sysroot in
>>>
>>> merging all llvm-common into a single recipe might be good
>>
>
> I thought more about this issue. Absent some clever solution that allows
> us to unify the wrappers (which is difficult due to the issues I
> mentioned in the last mail), I think we have three routes that could
> work to at least allow meta-clang and meta-oe to exist in the same Yocto
> workspace without breakage:
>
> (a) Instead of installing llvm-config into
> ${SYSROOT_DESTDIR}${bindir_crossscripts} -- which causes collisions when
> meta-clang and meta-oe both try to overwrite the same shared file --
> install into ${D}{bindir}llvm-config, and install the binary llvm-config
> into ${D}${bindir}/llvm-config${PV}. Recipes requiring LLVM will then
> need to depend on either llvm-common3.3 or llvm-common4.0, but at least
> there will be no collision or ambiguity which one gets used.
>
> (b) Use a common wrapper in
> ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config that does what the
> meta-oe current version does; just exec
> ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config${WANT_LLVM_RELEASE}. Let
> the meta-oe LLVM 3.3 install the binary llvm-config into
> ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config3.3 and let the
> meta-clang LLVM 4.0 install its wrapper llvm-config into
> ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config4.0. meta-clang will
> have to use the second wrapper script because it needs to edit
> TARGET_CFLAGS and similar to avoid compiling target binaries using
> native flags (due to the CMake LLVM build which appears not to respect
> cross-compile).
>
> (c) Use a common llvm-config wrapper in
> ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config that does a
> hardcoded check for WANT_LLVM_RELEASE. If the version is 4.0 or greater,
> then edit the TARGET_CFLAGS and similar, otherwise don't.
>
> It seems to me that all these options are undesireable, but (a) is
> probably least so.
>
> Khem and others: which option would you like me to pursue, or do you
> have an alternate suggestion?
>

(friendly ping)