Alternative compilers to GCC/Clang

Alternative compilers to GCC/Clang

[Amy Parker]

Hello! My name's Amy. I'm really impressed by the work done to make
Clang (and the LLVM toolchain overall) able to compile the kernel.
Figured I might as well donate my monkey hours to helping make it run
on other compilers as well. I haven't been able to find any that use
the same arguments structure as GCC and Clang (read: you can pass it
in as CC=compilername in your $MAKEOPTS). Any compilers along that
route anyone here has worked with that I could work with?

Best regards,
Amy Parker
(she/her/hers)

Re: Alternative compilers to GCC/Clang

[Willy Tarreau]

Hi Amy,

On Mon, Feb 01, 2021 at 03:31:49PM -0800, Amy Parker wrote:
> Hello! My name's Amy. I'm really impressed by the work done to make
> Clang (and the LLVM toolchain overall) able to compile the kernel.
> Figured I might as well donate my monkey hours to helping make it run
> on other compilers as well. I haven't been able to find any that use
> the same arguments structure as GCC and Clang (read: you can pass it
> in as CC=compilername in your $MAKEOPTS). Any compilers along that
> route anyone here has worked with that I could work with?

If you're interested, you should have a look at TCC (tiny CC) :

     https://repo.or.cz/tinycc.git

It compiles extremely fast, implements some subsets of gcc (a few
attributes for example), but is far from being able to compile a kernel
(at least last time I checked). Its speed makes it very convenient for
development. I made some efforts to make haproxy support it (and provided
some fixes to tcc) as it compiles the whole project in 0.5 second instead
of ~10 seconds with a modern gcc. It could probably compile a kernel in
15-20 seconds if properly supported, and this could be particularly handy
for development and testing.

Regards,
Willy

Re: Alternative compilers to GCC/Clang

[Amy Parker]

On Mon, Feb 1, 2021 at 9:33 PM Willy Tarreau <w@1wt.eu> wrote:
>
> Hi Amy,
>
> On Mon, Feb 01, 2021 at 03:31:49PM -0800, Amy Parker wrote:
> > Hello! My name's Amy. I'm really impressed by the work done to make
> > Clang (and the LLVM toolchain overall) able to compile the kernel.
> > Figured I might as well donate my monkey hours to helping make it run
> > on other compilers as well. I haven't been able to find any that use
> > the same arguments structure as GCC and Clang (read: you can pass it
> > in as CC=compilername in your $MAKEOPTS). Any compilers along that
> > route anyone here has worked with that I could work with?
>
> If you're interested, you should have a look at TCC (tiny CC) :
>
>      https://repo.or.cz/tinycc.git

Thank you for linking this! Wasn't able to find this git repository
for it - only binaries for nonfree operating systems (yuck).

>
> It compiles extremely fast, implements some subsets of gcc (a few
> attributes for example), but is far from being able to compile a kernel

Well, we'll see what I can do with that. :)

> (at least last time I checked). Its speed makes it very convenient for
> development. I made some efforts to make haproxy support it (and provided
> some fixes to tcc) as it compiles the whole project in 0.5 second instead
> of ~10 seconds with a modern gcc. It could probably compile a kernel in
> 15-20 seconds if properly supported, and this could be particularly handy
> for development and testing.

Oh wow, yeah, that would be great. Thank you for pointing this out!
Compiling the kernel in 15-20 seconds instead of the current *couple
minutes* would definitely be great.

>
> Regards,
> Willy

Best regards,
Amy Parker
(she/her/hers)

Re: Alternative compilers to GCC/Clang

[Amy Parker]

On Tue, Feb 2, 2021 at 8:26 AM Amy Parker <enbyamy@gmail.com> wrote:
> > It compiles extremely fast, implements some subsets of gcc (a few
> > attributes for example), but is far from being able to compile a kernel
>
> Well, we'll see what I can do with that. :)

Well, just installed it and tried building the kernel. Fails every file. :)

It's definitely something to work towards - but I don't know if kernel
advancements requiring newer GCC versions will go slow enough to allow
TCC improvements to arise. This isn't just something like with Clang
where a few tweaks to files and to Clang itself did the trick.

Re: Alternative compilers to GCC/Clang

[Borislav Petkov]

On Tue, Feb 02, 2021 at 11:11:32AM -0800, Amy Parker wrote:
> On Tue, Feb 2, 2021 at 8:26 AM Amy Parker <enbyamy@gmail.com> wrote:
> > > It compiles extremely fast, implements some subsets of gcc (a few
> > > attributes for example), but is far from being able to compile a kernel
> >
> > Well, we'll see what I can do with that. :)
> 
> Well, just installed it and tried building the kernel. Fails every file. :)
> 
> It's definitely something to work towards - but I don't know if kernel
> advancements requiring newer GCC versions will go slow enough to allow
> TCC improvements to arise. This isn't just something like with Clang
> where a few tweaks to files and to Clang itself did the trick.

Maybe this'll help you find something to do:

https://www.youtube.com/watch?v=iU0Z0vBKrtQ

Yes, it would be lovely to be able to compile the kernel with tcc but it
is not going to be trivial.

Good luck.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: Alternative compilers to GCC/Clang

[Willy Tarreau]

On Tue, Feb 02, 2021 at 09:19:20PM +0100, Borislav Petkov wrote:
> On Tue, Feb 02, 2021 at 11:11:32AM -0800, Amy Parker wrote:
> > On Tue, Feb 2, 2021 at 8:26 AM Amy Parker <enbyamy@gmail.com> wrote:
> > > > It compiles extremely fast, implements some subsets of gcc (a few
> > > > attributes for example), but is far from being able to compile a kernel
> > >
> > > Well, we'll see what I can do with that. :)
> > 
> > Well, just installed it and tried building the kernel. Fails every file. :)
> > 
> > It's definitely something to work towards - but I don't know if kernel
> > advancements requiring newer GCC versions will go slow enough to allow
> > TCC improvements to arise. This isn't just something like with Clang
> > where a few tweaks to files and to Clang itself did the trick.
> 
> Maybe this'll help you find something to do:
> 
> https://www.youtube.com/watch?v=iU0Z0vBKrtQ
> 
> Yes, it would be lovely to be able to compile the kernel with tcc but it
> is not going to be trivial.

In any case there will always be numerous limitations, but at least being
able to perform the basic build check with limited options could save quite
some time to many developers. Using gcc once the tcc-based "typo check"
passes would already be a nice start. Getting the kernel to boot would
indeed be a huge step forward! In haproxy we can only build with threading
disabled and it works slowly but sufficiently for basic tests and printf-
based debugging. It's convenient for bisecting certain bugs.

Thanks for the video Boris, I wasn't aware of it, definitely interesting!

Willy

Re: Alternative compilers to GCC/Clang

[Borislav Petkov]

+ Micha.

On Tue, Feb 02, 2021 at 10:00:39PM +0100, Willy Tarreau wrote:
> On Tue, Feb 02, 2021 at 09:19:20PM +0100, Borislav Petkov wrote:
> > On Tue, Feb 02, 2021 at 11:11:32AM -0800, Amy Parker wrote:
> > > On Tue, Feb 2, 2021 at 8:26 AM Amy Parker <enbyamy@gmail.com> wrote:
> > > > > It compiles extremely fast, implements some subsets of gcc (a few
> > > > > attributes for example), but is far from being able to compile a kernel
> > > >
> > > > Well, we'll see what I can do with that. :)
> > > 
> > > Well, just installed it and tried building the kernel. Fails every file. :)
> > > 
> > > It's definitely something to work towards - but I don't know if kernel
> > > advancements requiring newer GCC versions will go slow enough to allow
> > > TCC improvements to arise. This isn't just something like with Clang
> > > where a few tweaks to files and to Clang itself did the trick.
> > 
> > Maybe this'll help you find something to do:
> > 
> > https://www.youtube.com/watch?v=iU0Z0vBKrtQ
> > 
> > Yes, it would be lovely to be able to compile the kernel with tcc but it
> > is not going to be trivial.
> 
> In any case there will always be numerous limitations, but at least being
> able to perform the basic build check with limited options could save quite
> some time to many developers. Using gcc once the tcc-based "typo check"
> passes would already be a nice start. Getting the kernel to boot would
> indeed be a huge step forward! In haproxy we can only build with threading
> disabled and it works slowly but sufficiently for basic tests and printf-
> based debugging. It's convenient for bisecting certain bugs.
> 
> Thanks for the video Boris, I wasn't aware of it, definitely interesting!

Yeah, and there's a repo with that stuff:

https://github.com/susematz/linux

and also:

https://github.com/susematz/linux/blob/tcc46/README.tcc

It would be good to start forward-porting and integrating some of the
fixes and even extend tcc to handle some of the gnuisms we're using in
the kernel so that we can build the kernel with it too.

I can imagine having CONFIG_TCC - as long as that doesn't get too
intrusive and get in the way of things - and those who wanna build the
kernel with it, can enable it. For example...

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: Alternative compilers to GCC/Clang

[Willy Tarreau]

On Tue, Feb 02, 2021 at 10:20:48PM +0100, Borislav Petkov wrote:
> It would be good to start forward-porting and integrating some of the
> fixes and even extend tcc to handle some of the gnuisms we're using in
> the kernel so that we can build the kernel with it too.

I agree. And the team is responsive and shows great consideration for
patches.

> I can imagine having CONFIG_TCC - as long as that doesn't get too
> intrusive and get in the way of things - and those who wanna build the
> kernel with it, can enable it. For example...

I like this idea. It's way better than having to implement everything
at once or degrade some code just to make it build. It could be solved
at config time by automatically excluding some features.

It should also be less of a hassle than dealing with many gcc versions
because if we see it as a development speed up tool we can easily accept
that we occasionaly break compatibility with older of its versions and
that those who want to use it just rebuild the latest one (it's trivial
and fast, basically "make" and you're done, not the typical toolchain
experience). You don't care if it doesn't work for one week, you're not
supposed to ship any form of official code built with it anyway. It's
just an aid, and a nice one.

Willy

Re: Alternative compilers to GCC/Clang

[Michael Matz]

Hello,

On Tue, 2 Feb 2021, Borislav Petkov wrote:

> + Micha.

Huh, someone found my video ;-)

> > > > > > attributes for example), but is far from being able to compile 
> > > > > > a kernel

A _current_ kernel maybe :)  Some 4.6 x86-64 kernel in qemu in a certain 
config plus a little patches definitely does work.  Slowly, but usable.  
See the repo Boris mentioned.

> > > > It's definitely something to work towards - but I don't know if kernel
> > > > advancements requiring newer GCC versions will go slow enough to allow
> > > > TCC improvements to arise. This isn't just something like with Clang
> > > > where a few tweaks to files and to Clang itself did the trick.
> > > 
> > > Maybe this'll help you find something to do:
> > > 
> > > https://www.youtube.com/watch?v=iU0Z0vBKrtQ
> > > 
> > > Yes, it would be lovely to be able to compile the kernel with tcc but it
> > > is not going to be trivial.

As tcc is so simple it's actually not too much hassle, the biggest 
roadblocks should be gone; the usage of inline asm in the kernel is ... 
creative ... and hence the single pass nature of TCC and the C-asm 
integration pose some challenges ;)  Also anything that requires inlining 
to remove dead but non-conforming code (like calling undefined functions) 
needs an alternative like macros expanding to zero, instead of a function 
returning zero.  (I even have an limited inliner for tcc, but I didn't 
like it too much)

(My interest was tcc, not kernel development, which is why I never did 
anything with that 4.6 kernel, I wanted to retain a stable and big known 
source base for tcc hackery.  If someone is interested in kernel compiling 
that can change the picture of course; I think I at least remember most of 
the reasons for the kernel patches I had to do to make my tcc hackery 
easier :) ).

> It would be good to start forward-porting and integrating some of the 
> fixes and even extend tcc to handle some of the gnuisms we're using in 
> the kernel so that we can build the kernel with it too.
> 
> I can imagine having CONFIG_TCC - as long as that doesn't get too 
> intrusive and get in the way of things - and those who wanna build the 
> kernel with it, can enable it. For example...


Ciao,
Michael.

Re: Alternative compilers to GCC/Clang

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]

Hi!

> > Hello! My name's Amy. I'm really impressed by the work done to make
> > Clang (and the LLVM toolchain overall) able to compile the kernel.
> > Figured I might as well donate my monkey hours to helping make it run
> > on other compilers as well. I haven't been able to find any that use
> > the same arguments structure as GCC and Clang (read: you can pass it
> > in as CC=compilername in your $MAKEOPTS). Any compilers along that
> > route anyone here has worked with that I could work with?
> 
> If you're interested, you should have a look at TCC (tiny CC) :
> 
>      https://repo.or.cz/tinycc.git
> 
> It compiles extremely fast, implements some subsets of gcc (a few
> attributes for example), but is far from being able to compile a kernel
> (at least last time I checked). Its speed makes it very convenient for
> development. I made some efforts to make haproxy support it (and provided
> some fixes to tcc) as it compiles the whole project in 0.5 second instead
> of ~10 seconds with a modern gcc. It could probably compile a kernel in
> 15-20 seconds if properly supported, and this could be particularly handy
> for development and testing.

For the record, yes, something that compiles kernel fast would be very
very nice.

Best regards,
								Pavel

-- 
http://www.livejournal.com/~pavelmachek

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]