[Buildroot] clang tool-chain support?

[Buildroot] clang tool-chain support?

[Stas Sergeev]

Hello.

Just a few toolchain-related questions from
novice buildroot user.

- Currently it seems buildroot does not support
the clang toolchain natively, so is there any way
(preferably a documented one) to use the host's
clang tool-chain? (I've only found the way to build
clang as a target package, which is not what I need)

- I know that currently there are not too many
clang-only projects, but I just want to build a few
of them. As their amount may only increase
in the future, are there any plans to get the
native clang support in buildroot?
(not found in buildroot's TODO list)

- buildroot seems to provide yasm instead of nasm.
While I was able to work around yasm bugs and
build the project with it, I wonder why such a choice?
Unless I am mistaken, nasm is active and yasm is
pretty much dead - I've got that impression from
reporting bugs to both projects and getting replies
(and instant fixes) only from nasm.
So any plans to switch to nasm?

[Buildroot] clang tool-chain support?

[Romain Naour]

Hi Stas,

Le 03/09/2019 ? 12:00, Stas Sergeev a ?crit?:
> Hello.
> 
> Just a few toolchain-related questions from
> novice buildroot user.
> 
> - Currently it seems buildroot does not support
> the clang toolchain natively, so is there any way
> (preferably a documented one) to use the host's
> clang tool-chain? (I've only found the way to build
> clang as a target package, which is not what I need)

Indded, Buildroot doesn't support yet clang as cross-toolchain.
Last year Valentin worked on llvm/clang integration into Buildroot to provide
llvm/clang libraries for the target. For example, this allow to enable opencl
support (Clover) in mesa3d stack.

I recently tested clang to build a kernel for aarch64 and x86_64, but we needs
some work to build userspace application.

Also, there are some questions about clang toolchain:

"The long-term goal is to have a complete clang-based toolchain. The usefulness
of this is questionable however." [1]

If clang can be used as a toolchain, it would be great to add it as part of an
Buildroot's internal toolchain. Also, the toolchain external infra should be
able to import clang and it's libraries if present in a pre-built toolchain.
To do so, we have to make sure that clang binaries are relocatable.

[1] https://elinux.org/Buildroot:DeveloperDaysFOSDEM2018#LLVM.2FClang

> 
> - I know that currently there are not too many
> clang-only projects, but I just want to build a few
> of them. As their amount may only increase
> in the future, are there any plans to get the
> native clang support in buildroot?
> (not found in buildroot's TODO list)

Can you share the list of clang only projects you are interested in?

> 
> - buildroot seems to provide yasm instead of nasm.
> While I was able to work around yasm bugs and
> build the project with it, I wonder why such a choice?
> Unless I am mistaken, nasm is active and yasm is
> pretty much dead - I've got that impression from
> reporting bugs to both projects and getting replies
> (and instant fixes) only from nasm.
> So any plans to switch to nasm?

There is a nasm package in Buildroot for the host and the target.
Yasm is also available but less used by other packages.
What is you Buildroot version ?

Best regards,
Romain

> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] clang tool-chain support?

[stsp]

Hi!

03.09.2019 23:58, Romain Naour ?????:
> Hi Stas,
>
> Le 03/09/2019 ? 12:00, Stas Sergeev a ?crit?:
>> Hello.
>>
>> Just a few toolchain-related questions from
>> novice buildroot user.
>>
>> - Currently it seems buildroot does not support
>> the clang toolchain natively, so is there any way
>> (preferably a documented one) to use the host's
>> clang tool-chain? (I've only found the way to build
>> clang as a target package, which is not what I need)
> Indded, Buildroot doesn't support yet clang as cross-toolchain.
> Last year Valentin worked on llvm/clang integration into Buildroot to provide
> llvm/clang libraries for the target.
Do you mean that "clang as a target" is a
pre-requisite for "clang as a cross tool-chain"?
As otherwise I suspect this is what I mentioned
as "which is not what I need". :)

> I recently tested clang to build a kernel for aarch64 and x86_64

But here you talk about "clang as a cross-toolchain",
not a target package? You probably want to say that
the work is ongoing in both directions (for target and
for host), but I am not sure if you mean exactly that.


> Also, there are some questions about clang toolchain:
>
> "The long-term goal is to have a complete clang-based toolchain. The usefulness
> of this is questionable however." [1]
>
> If clang can be used as a toolchain,

I suppose it definitely can, at least for some configurations.
For others it won't be selectable (for example).


>   it would be great to add it as part of an
> Buildroot's internal toolchain. Also, the toolchain external infra should be
> able to import clang and it's libraries if present in a pre-built toolchain.
> To do so, we have to make sure that clang binaries are relocatable.

I think clang understands --sysroot= by default,
so, if I understand you correctly, I just need to
build the clang toolchain manually and put it into
some "root". But its not really documented anywhere,
right? By the way, can something like this (just googled)
be of any help for me with that task?
https://www.embtoolkit.org/
It is said to "generate toolchains", but I wonder if
they are "compatible" with what buildroot expects.


>> - I know that currently there are not too many
>> clang-only projects, but I just want to build a few
>> of them. As their amount may only increase
>> in the future, are there any plans to get the
>> native clang support in buildroot?
>> (not found in buildroot's TODO list)
> Can you share the list of clang only projects you are interested in?

Well, the projects I am interested in, are written
by me, in particular this one:
https://github.com/stsp/fdpp
I am reporting bugs to gcc bugzilla for around 20
years (the above project is new but I have others),
and the average time frame of them hanging
in their bugzilla before being fixed, is 10 years.
It was a big pain in the past, but now clang appeared
and no pain if I just declare the project clang-only.
And in this particular case the above project is absolutely
not portable to gcc, i.e. if not for clang, the project would
simply not exist. I only have just 1 bug report in
the clang bugzilla currently pending for that project,
where gcc produces the right code and clang miscompiles.
Working around 1 bug was acceptable, clang seems
quite stable to me.


>> - buildroot seems to provide yasm instead of nasm.
>> While I was able to work around yasm bugs and
>> build the project with it, I wonder why such a choice?
>> Unless I am mistaken, nasm is active and yasm is
>> pretty much dead - I've got that impression from
>> reporting bugs to both projects and getting replies
>> (and instant fixes) only from nasm.
>> So any plans to switch to nasm?
> There is a nasm package in Buildroot for the host and the target.
> Yasm is also available but less used by other packages.
> What is you Buildroot version ?
Hmm, this is interesting.
buildroot-2019.08
In menuconfig I go to Target packages --> Development tools
and only see yasm and no nasm. Then I do the search '/'
and again, search gives BR2_PACKAGE_YASM for yasm
and nothing for nasm. Of course having yasm in target
packages is not what I want - I need it in the host. But
are you sure nasm is there anywhere? I can't find it via
menuconfig, but maybe I am doing something wrongly?

[Buildroot] clang tool-chain support?

[Romain Naour]

Hi Stas,

Le 04/09/2019 ? 02:28, stsp a ?crit?:
> Hi!
> 
> 03.09.2019 23:58, Romain Naour ?????:
>> Hi Stas,
>>
>> Le 03/09/2019 ? 12:00, Stas Sergeev a ?crit?:
>>> Hello.
>>>
>>> Just a few toolchain-related questions from
>>> novice buildroot user.
>>>
>>> - Currently it seems buildroot does not support
>>> the clang toolchain natively, so is there any way
>>> (preferably a documented one) to use the host's
>>> clang tool-chain? (I've only found the way to build
>>> clang as a target package, which is not what I need)
>> Indded, Buildroot doesn't support yet clang as cross-toolchain.
>> Last year Valentin worked on llvm/clang integration into Buildroot to provide
>> llvm/clang libraries for the target.
> Do you mean that "clang as a target" is a
> pre-requisite for "clang as a cross tool-chain"?

No, but we (with Valentin) focussed on "clang as a target". I mean, provide
clang libraries for the target. Buildroot don't support compilers on the target.

> As otherwise I suspect this is what I mentioned
> as "which is not what I need". :)

Yes I know :)

> 
>> I recently tested clang to build a kernel for aarch64 and x86_64
> 
> But here you talk about "clang as a cross-toolchain",
> not a target package? You probably want to say that
> the work is ongoing in both directions (for target and
> for host), but I am not sure if you mean exactly that.

When you build host-clang package (clang for the host machine), clang compiler
is present in $(HOST_DIR)/bin but it's not used by any Buildroot infra/package.
By hacking the linux package to use clang instead of gcc, I'm able to build the
kernel. But this is not upstreamable.

> 
> 
>> Also, there are some questions about clang toolchain:
>>
>> "The long-term goal is to have a complete clang-based toolchain. The usefulness
>> of this is questionable however." [1]
>>
>> If clang can be used as a toolchain,
> 
> I suppose it definitely can, at least for some configurations.
> For others it won't be selectable (for example).

I mean "If clang can be used as a toolchain by Buildroot".
I believe it can but needs some work especially for building userspace programs.

> 
> 
>> ? it would be great to add it as part of an
>> Buildroot's internal toolchain. Also, the toolchain external infra should be
>> able to import clang and it's libraries if present in a pre-built toolchain.
>> To do so, we have to make sure that clang binaries are relocatable.
> 
> I think clang understands --sysroot= by default,
> so, if I understand you correctly, I just need to
> build the clang toolchain manually and put it into
> some "root". But its not really documented anywhere,
> right?

This is not easy, I'm aware of an issue about gcc files needed to run clang on
the host [1]. It seems clang has an issue with gcc toolchains where the sysroot
has been moved. I think clang should use gcc's search-dirs.

[1] http://lists.busybox.net/pipermail/buildroot/2019-August/256204.html

 By the way, can something like this (just googled)
> be of any help for me with that task?
> https://www.embtoolkit.org/
> It is said to "generate toolchains", but I wonder if
> they are "compatible" with what buildroot expects.

For now, Buildroot can't import a prebuit clang toolchain as external toolchain.

> 
> 
>>> - I know that currently there are not too many
>>> clang-only projects, but I just want to build a few
>>> of them. As their amount may only increase
>>> in the future, are there any plans to get the
>>> native clang support in buildroot?
>>> (not found in buildroot's TODO list)
>> Can you share the list of clang only projects you are interested in?
> 
> Well, the projects I am interested in, are written
> by me, in particular this one:
> https://github.com/stsp/fdpp
> I am reporting bugs to gcc bugzilla for around 20
> years (the above project is new but I have others),
> and the average time frame of them hanging
> in their bugzilla before being fixed, is 10 years.
> It was a big pain in the past, but now clang appeared
> and no pain if I just declare the project clang-only.
> And in this particular case the above project is absolutely
> not portable to gcc, i.e. if not for clang, the project would
> simply not exist. I only have just 1 bug report in
> the clang bugzilla currently pending for that project,
> where gcc produces the right code and clang miscompiles.
> Working around 1 bug was acceptable, clang seems
> quite stable to me.

Well, that's probably true for gcc... sometime we can report a bug without
having a reply for a long time.

At least for Binutils and Glibc this is not the case, we had some feedback from
maintainers when needed :)

> 
> 
>>> - buildroot seems to provide yasm instead of nasm.
>>> While I was able to work around yasm bugs and
>>> build the project with it, I wonder why such a choice?
>>> Unless I am mistaken, nasm is active and yasm is
>>> pretty much dead - I've got that impression from
>>> reporting bugs to both projects and getting replies
>>> (and instant fixes) only from nasm.
>>> So any plans to switch to nasm?
>> There is a nasm package in Buildroot for the host and the target.

Ok I made a mistake here. nasm is a host only package.

>> Yasm is also available but less used by other packages.
>> What is you Buildroot version ?
> Hmm, this is interesting.
> buildroot-2019.08
> In menuconfig I go to Target packages --> Development tools
> and only see yasm and no nasm. Then I do the search '/'
> and again, search gives BR2_PACKAGE_YASM for yasm
> and nothing for nasm. Of course having yasm in target
> packages is not what I want - I need it in the host. But
> are you sure nasm is there anywhere? I can't find it via
> menuconfig, but maybe I am doing something wrongly?

Ok, you mean nasm for the target.
I guess that because nobody contributed to make it a target package.

Yasm is available for the target but no package seems to depend on it.

Feel free to contribute a patch enabling nasm for the target :)

Best regards,
Romain

[Buildroot] clang tool-chain support?

[stsp]

Hi,

04.09.2019 10:38, Romain Naour ?????:
>>> Also, there are some questions about clang toolchain:
>>>
>>> "The long-term goal is to have a complete clang-based toolchain. The usefulness
>>> of this is questionable however." [1]
>>>
>>> If clang can be used as a toolchain,
>> I suppose it definitely can, at least for some configurations.
>> For others it won't be selectable (for example).
> I mean "If clang can be used as a toolchain by Buildroot".
> I believe it can but needs some work especially for building userspace programs.


> For now, Buildroot can't import a prebuit clang toolchain as external toolchain.

OK, I see...
Thanks for the info and detailed explanations
of the problem. I guess the search path differences
is not the only problem here.
Since clang support is not even here:
https://www.elinux.org/Buildroot#Todo_list
I don't expect it to appear any soon. :(


> Well, that's probably true for gcc... sometime we can report a bug without
> having a reply for a long time.
>
> At least for Binutils and Glibc this is not the case, we had some feedback from
> maintainers when needed :)

Indeed, and musl too. :)


> Ok I made a mistake here. nasm is a host only package.

That would be what I need.


> Ok, you mean nasm for the target.
No-no, I meant host.
But do you mean the host packages are not
searchible in the menuconfig? I.e. I searched
with '/' for nasm and found nothing.
So I suppose it just cannot be disabled by menuconfig,
and as such, is not listed there?
yasm on target and nasm on host is a perfect
combination, I just can't find where to enable the
host nasm.

[Buildroot] clang tool-chain support?

[Romain Naour]

Hi Stas,

Le 04/09/2019 ? 21:09, stsp a ?crit?:
> Hi,
> 
> 04.09.2019 10:38, Romain Naour ?????:
>>>> Also, there are some questions about clang toolchain:
>>>>
>>>> "The long-term goal is to have a complete clang-based toolchain. The usefulness
>>>> of this is questionable however." [1]
>>>>
>>>> If clang can be used as a toolchain,
>>> I suppose it definitely can, at least for some configurations.
>>> For others it won't be selectable (for example).
>> I mean "If clang can be used as a toolchain by Buildroot".
>> I believe it can but needs some work especially for building userspace programs.
> 
> 
>> For now, Buildroot can't import a prebuit clang toolchain as external toolchain.
> 
> OK, I see...
> Thanks for the info and detailed explanations
> of the problem. I guess the search path differences
> is not the only problem here.
> Since clang support is not even here:
> https://www.elinux.org/Buildroot#Todo_list
> I don't expect it to appear any soon. :(

Well, the todolist is not very exhaustive. Most of the time, a new package or
functionality is contributed without being listed in the todolist :)

Yesterday I found a new article on lowrisc blog about clang testing for RISC-V
architecture with Buildroot [1].

Also you can fine articles about llvm/clang integration into Buildroot [2] and
kernel build [3].

[1]
https://www.lowrisc.org/blog/2019/07/large-scale-risc-v-llvm-testing-with-buildroot

[2] http://www.linuxembedded.fr/2018/07/llvmclang-integration-into-buildroot
[3]
http://www.linuxembedded.fr/2019/08/my-first-linux-kernel-built-with-clang-compiler/


> 
> 
>> Well, that's probably true for gcc... sometime we can report a bug without
>> having a reply for a long time.
>>
>> At least for Binutils and Glibc this is not the case, we had some feedback from
>> maintainers when needed :)
> 
> Indeed, and musl too. :)

+1 :)

> 
> 
>> Ok I made a mistake here. nasm is a host only package.
> 
> That would be what I need.
> 
> 
>> Ok, you mean nasm for the target.
> No-no, I meant host.
> But do you mean the host packages are not
> searchible in the menuconfig? I.e. I searched
> with '/' for nasm and found nothing.
> So I suppose it just cannot be disabled by menuconfig,
> and as such, is not listed there?
> yasm on target and nasm on host is a perfect
> combination, I just can't find where to enable the
> host nasm.

Ok, actually host nasm exist for the host but the dependency is pulled by
packages that need it at build time. grep for host-nasm in the tree.

Actually there is a discussion about adding a config symbol for all host
packages, see:

https://www.elinux.org/Buildroot:DeveloperDaysFOSDEM2019#.22Config.in.host.22_for_all_host_packages

It's the same for yasm. if you want it for the host, you can't select it by
using menuconfig.

There is an entry "Host utilities" in the menuconfig but the list of host
packages are not complete.

See "17.2.2. Config.in.host file"

http://nightly.buildroot.org/manual.html#_config_files

So, if host-nasm is useful for you and you need to select it from the menuconfig
you can contribute a patch adding the Config.in.host file :)

Best regards,
Romain