[RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[Sedat Dilek]

This patch is currently not mandatory but a prerequisites for the second one.

Folks from ClangBuiltLinux project like the combination of Clang compiler
and LLD linker from LLVM project to build their Linux kernels.

Sami Tolvanen <samitolvanen@google.com> has a patch for using LD_IS_LLD (see [1]).

Documentation/process/changes.rst says and uses "binutils" that's why I called
it LD_IS_BINUTILS (see [2] and [3]).

The second patch will rename existing LD_VERSION to BINUTILS_VERSION to have
a consistent naming convention like:

1. CC_IS_GCC and GCC_VERSION
2. CC_IS_CLANG and CLANG_VERSION
3. LD_IS_BINUTILS and BINUTILS_VERSION

[1] https://github.com/samitolvanen/linux/commit/61889e01f0ed4f07a9d631f163bba6c6637bfa46
[2] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n34
[3] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n76

Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
---
 init/Kconfig | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/init/Kconfig b/init/Kconfig
index 9e22ee8fbd75..520116efea0f 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -16,9 +16,12 @@ config GCC_VERSION
 	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
 	default 0
 
+config LD_IS_BINUTILS
+	def_bool $(success,$(LD) -v | head -n 1 | grep -q 'GNU ld')
+
 config LD_VERSION
 	int
-	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+	default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh) if LD_IS_BINUTILS
 
 config CC_IS_CLANG
 	def_bool $(success,$(CC) --version | head -n 1 | grep -q clang)
-- 
2.26.1

Re: [RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[Nick Desaulniers]

On Sun, Apr 19, 2020 at 6:19 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> This patch is currently not mandatory but a prerequisites for the second one.
>
> Folks from ClangBuiltLinux project like the combination of Clang compiler
> and LLD linker from LLVM project to build their Linux kernels.
>
> Sami Tolvanen <samitolvanen@google.com> has a patch for using LD_IS_LLD (see [1]).
>
> Documentation/process/changes.rst says and uses "binutils" that's why I called
> it LD_IS_BINUTILS (see [2] and [3]).
>
> The second patch will rename existing LD_VERSION to BINUTILS_VERSION to have
> a consistent naming convention like:
>
> 1. CC_IS_GCC and GCC_VERSION
> 2. CC_IS_CLANG and CLANG_VERSION
> 3. LD_IS_BINUTILS and BINUTILS_VERSION
>
> [1] https://github.com/samitolvanen/linux/commit/61889e01f0ed4f07a9d631f163bba6c6637bfa46
> [2] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n34
> [3] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n76
>
> Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>

Just some background on Sami's patch.  Originally we were using
ld.gold (for LTO for Pixel 3) before moving to ld.lld (for LTO for
Pixel 4 and later).  Not sure if Kconfig would be a better place to
check if gold is used, then warn?  I kind of prefer the distinction
that binutils contains two different linkers, though if no one is
supporting ld.gold, and it doesn't work for the kernel, then maybe
that preference is moot?

> ---
>  init/Kconfig | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/init/Kconfig b/init/Kconfig
> index 9e22ee8fbd75..520116efea0f 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -16,9 +16,12 @@ config GCC_VERSION
>         default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC
>         default 0
>
> +config LD_IS_BINUTILS
> +       def_bool $(success,$(LD) -v | head -n 1 | grep -q 'GNU ld')
> +
>  config LD_VERSION
>         int
> -       default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh)
> +       default $(shell,$(LD) --version | $(srctree)/scripts/ld-version.sh) if LD_IS_BINUTILS
>
>  config CC_IS_CLANG
>         def_bool $(success,$(CC) --version | head -n 1 | grep -q clang)
> --
> 2.26.1
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200419131947.173685-1-sedat.dilek%40gmail.com.



-- 
Thanks,
~Nick Desaulniers

Re: [RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[Masahiro Yamada]

On Fri, Apr 24, 2020 at 3:44 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Sun, Apr 19, 2020 at 6:19 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > This patch is currently not mandatory but a prerequisites for the second one.
> >
> > Folks from ClangBuiltLinux project like the combination of Clang compiler
> > and LLD linker from LLVM project to build their Linux kernels.
> >
> > Sami Tolvanen <samitolvanen@google.com> has a patch for using LD_IS_LLD (see [1]).
> >
> > Documentation/process/changes.rst says and uses "binutils" that's why I called
> > it LD_IS_BINUTILS (see [2] and [3]).
> >
> > The second patch will rename existing LD_VERSION to BINUTILS_VERSION to have
> > a consistent naming convention like:
> >
> > 1. CC_IS_GCC and GCC_VERSION
> > 2. CC_IS_CLANG and CLANG_VERSION
> > 3. LD_IS_BINUTILS and BINUTILS_VERSION
> >
> > [1] https://github.com/samitolvanen/linux/commit/61889e01f0ed4f07a9d631f163bba6c6637bfa46
> > [2] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n34
> > [3] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n76
> >
> > Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
>
> Just some background on Sami's patch.  Originally we were using
> ld.gold (for LTO for Pixel 3) before moving to ld.lld (for LTO for
> Pixel 4 and later).  Not sure if Kconfig would be a better place to
> check if gold is used, then warn?  I kind of prefer the distinction
> that binutils contains two different linkers, though if no one is
> supporting ld.gold, and it doesn't work for the kernel, then maybe
> that preference is moot?


I prefer LD_IS_BFD, like this patch:
https://lore.kernel.org/patchwork/patch/1039719/

We do not need LD_IS_GOLD, though.




-- 
Best Regards
Masahiro Yamada

Re: [RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[Fangrui Song]

On 2020-04-24, Masahiro Yamada wrote:
>On Fri, Apr 24, 2020 at 3:44 AM Nick Desaulniers
><ndesaulniers@google.com> wrote:
>>
>> On Sun, Apr 19, 2020 at 6:19 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>> >
>> > This patch is currently not mandatory but a prerequisites for the second one.
>> >
>> > Folks from ClangBuiltLinux project like the combination of Clang compiler
>> > and LLD linker from LLVM project to build their Linux kernels.
>> >
>> > Sami Tolvanen <samitolvanen@google.com> has a patch for using LD_IS_LLD (see [1]).
>> >
>> > Documentation/process/changes.rst says and uses "binutils" that's why I called
>> > it LD_IS_BINUTILS (see [2] and [3]).
>> >
>> > The second patch will rename existing LD_VERSION to BINUTILS_VERSION to have
>> > a consistent naming convention like:
>> >
>> > 1. CC_IS_GCC and GCC_VERSION
>> > 2. CC_IS_CLANG and CLANG_VERSION
>> > 3. LD_IS_BINUTILS and BINUTILS_VERSION
>> >
>> > [1] https://github.com/samitolvanen/linux/commit/61889e01f0ed4f07a9d631f163bba6c6637bfa46
>> > [2] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n34
>> > [3] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n76
>> >
>> > Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
>>
>> Just some background on Sami's patch.  Originally we were using
>> ld.gold (for LTO for Pixel 3) before moving to ld.lld (for LTO for
>> Pixel 4 and later).  Not sure if Kconfig would be a better place to
>> check if gold is used, then warn?  I kind of prefer the distinction
>> that binutils contains two different linkers, though if no one is
>> supporting ld.gold, and it doesn't work for the kernel, then maybe
>> that preference is moot?
>
>
>I prefer LD_IS_BFD, like this patch:
>https://lore.kernel.org/patchwork/patch/1039719/
>
>We do not need LD_IS_GOLD, though.
>
>-- 
>Best Regards
>Masahiro Yamada

+1 for CONFIG_LD_IS_BFD

Usually GNU ld is also installed as ld.bfd and can be referred to by -fuse-ld=bfd (GCC >= 9, or clang)

The repository binutils-gdb includes two linkers: GNU ld and GNU gold, so CONFIG_LD_IS_BINUTILS would be ambiguous.

Re: [RFC PATCH 1/2] kbuild: add CONFIG_LD_IS_BINUTILS

[Sedat Dilek]

On Fri, Apr 24, 2020 at 7:18 AM Fangrui Song <maskray@google.com> wrote:
>
> On 2020-04-24, Masahiro Yamada wrote:
> >On Fri, Apr 24, 2020 at 3:44 AM Nick Desaulniers
> ><ndesaulniers@google.com> wrote:
> >>
> >> On Sun, Apr 19, 2020 at 6:19 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >> >
> >> > This patch is currently not mandatory but a prerequisites for the second one.
> >> >
> >> > Folks from ClangBuiltLinux project like the combination of Clang compiler
> >> > and LLD linker from LLVM project to build their Linux kernels.
> >> >
> >> > Sami Tolvanen <samitolvanen@google.com> has a patch for using LD_IS_LLD (see [1]).
> >> >
> >> > Documentation/process/changes.rst says and uses "binutils" that's why I called
> >> > it LD_IS_BINUTILS (see [2] and [3]).
> >> >
> >> > The second patch will rename existing LD_VERSION to BINUTILS_VERSION to have
> >> > a consistent naming convention like:
> >> >
> >> > 1. CC_IS_GCC and GCC_VERSION
> >> > 2. CC_IS_CLANG and CLANG_VERSION
> >> > 3. LD_IS_BINUTILS and BINUTILS_VERSION
> >> >
> >> > [1] https://github.com/samitolvanen/linux/commit/61889e01f0ed4f07a9d631f163bba6c6637bfa46
> >> > [2] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n34
> >> > [3] https://git.kernel.org/linus/tree/Documentation/process/changes.rst#n76
> >> >
> >> > Signed-off-by: Sedat Dilek <sedat.dilek@gmail.com>
> >>
> >> Just some background on Sami's patch.  Originally we were using
> >> ld.gold (for LTO for Pixel 3) before moving to ld.lld (for LTO for
> >> Pixel 4 and later).  Not sure if Kconfig would be a better place to
> >> check if gold is used, then warn?  I kind of prefer the distinction
> >> that binutils contains two different linkers, though if no one is
> >> supporting ld.gold, and it doesn't work for the kernel, then maybe
> >> that preference is moot?
> >
> >
> >I prefer LD_IS_BFD, like this patch:
> >https://lore.kernel.org/patchwork/patch/1039719/
> >
> >We do not need LD_IS_GOLD, though.
> >
> >--
> >Best Regards
> >Masahiro Yamada
>
> +1 for CONFIG_LD_IS_BFD
>
> Usually GNU ld is also installed as ld.bfd and can be referred to by -fuse-ld=bfd (GCC >= 9, or clang)
>
> The repository binutils-gdb includes two linkers: GNU ld and GNU gold, so CONFIG_LD_IS_BINUTILS would be ambiguous.

Hi,

The last days I was busy with job-hunting, so this is my prio #1.
Thus I was not very responsive.

And I am sorry to mixup different threads in ClangBuiltLinux and elsewhere.

WOW Huh ***Votings***!

Originally the patchset from Nick had LD_IS_BFD, LD_IS_GOLD and LD_IS_LLD.

As pointed out GOLD is no more suitable to link the kernel and thus
deactivated (AFAICS Thomas Gleixner did this).

Personally, I am OK with ***LD_IS_LD*** because we have now
***LD_VERSION*** which was introduced in Linux v5.7-rc1.
We have the pair LD_IS_LD and LD_VERSION like CC_IS_GCC and GCC_VERSION.
The only thing I would like to be changed is the comment in
***scripts/ld-version.sh*** to mention "GNU/ld (binutils)" if you
decide for LD_IS_LD or not.
But I am OK with LD_IS_BFD.
If you ask people what they come into mind when speaking of "ld" - 99%
of the answers of this people will point to GNU/ld from GNU/binutils.
This is simply a fact in the Linux-kernel world.

I am living in my x86 world and do not test with cross-compilation or
any other archs like ARM or MIPS or whatever.
So, I might not catch all corner-cases.

Some bots like kbuild-bot(?) already sent some warnings on my patchset.

My ***main interest*** is to have good support of ***LLD*** which is
my primary linker.

Why?
(Might be off-topic here in this thread)

"Numbers talk, bullshit walks." (Linus Torvalds)

Please, compile yourself (here: Debian/testing AMD64)...

#1: gcc 9.3 with GNU/ld (binutils)
#2: gcc 9.3 with ld.lld-10
#3: clang-10 and ld.lld-10

I have not the same code-base to compare, but first numbers:

Even the combo of gcc-9.3 and ld.lld-10 produces 5GiB more disc-space
in my linux-git.
The debug binaries and the resulting Debian debug packages are
significantly bigger

$ cd stats

$ cat 5.7.0-rc*/disc-usage.txt
23406   linux
1951    archives/5.7.0-rc1-2-amd64-gcc <--- XXX: gcc-9.3 + ld.lld-10
17958   linux
1365    5.7.0-rc2-1-amd64-clang <--- XXX: LLVM/Clang/LLD
10.0.1-git-92d5c1be9ee93850c0a8903f05f36a23ee835dc2

$ cd archives

$ du -m 5.7.0-rc*/linux-image-*-dbg*_amd64.deb
617     5.7.0-rc1-2-amd64-gcc/linux-image-5.7.0-rc1-2-amd64-gcc-dbg_5.7.0~rc1-2~bullseye+dileks1_amd64.deb
424     5.7.0-rc2-1-amd64-clang/linux-image-5.7.0-rc2-1-amd64-clang-dbg_5.7.0~rc2-1~bullseye+dileks1_amd64.deb

$ du -m 5.7.0-rc*/vmlinux*
603     5.7.0-rc1-2-amd64-gcc/vmlinux
7       5.7.0-rc1-2-amd64-gcc/vmlinux.compressed
597     5.7.0-rc1-2-amd64-gcc/vmlinux.o
409     5.7.0-rc2-1-amd64-clang/vmlinux
7       5.7.0-rc2-1-amd64-clang/vmlinux.compressed
404     5.7.0-rc2-1-amd64-clang/vmlinux.o

As said - not the same code and patch base!

This needs definitely to be investigated.

LLD seems to be - seen from the numbers - be a lot of "smarter".

Have more fun!

Regards and happy first day of Ramadan if you celebrate it,
- Sedat -

P.S.: A build of Linux v5.7-rcX with gcc-9.3 as compiler is much much
faster here.