LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Fangrui Song <maskray@google.com>
To: Serge Guelton <sguelton@redhat.com>, Tom Stellard <tstellar@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	Sylvestre Ledru <sylvestre@mozilla.com>,
	Felix Yan <felixonmars@archlinux.org>
Subject: Re: Very slow clang kernel config ..
Date: Sat, 1 May 2021 16:55:49 -0700
Message-ID: <20210501235549.vugtjeb7dmd5xell@google.com> (raw)
In-Reply-To: <CAHk-=whPoHOa6_gA3-pk=nOzpefZmsSK1an_iByEyhLe+3m-AA@mail.gmail.com>

On 2021-05-01, Linus Torvalds wrote:
>On Sat, May 1, 2021 at 12:58 PM Serge Guelton <sguelton@redhat.com> wrote:
>>
>> Different metrics lead to different choice, then comes the great pleasure of
>> making compromises :-)
>
>Even if that particular compromise might be the right one to do for
>clang and llvm, the point is that the Fedora rule is garbage, and it
>doesn't _allow_ for making any compromises at all.
>
>The Fedora policy is basically "you have to use shared libraries
>whether that makes any sense or not".
>
>As mentioned, I've seen a project bitten by that insane policy.  It's bogus.
>
>            Linus

As a very safe optimization, distributions can consider
-fno-semantic-interposition (only effectful on x86 in GCC and Clang,
already used by some Python packages):
avoid GOT/PLT generating relocation if the referenced symbol is defined
in the same translation unit. See my benchmark below: it makes the built
-fPIC clang slightly faster.

As a slightly aggressive optimization, consider
-DCMAKE_EXE_LINKER_FLAGS=-Wl,-Bsymbolic-functions -DCMAKE_SHARED_LINKER_FLAGS=-Wl,-Bsymbolic-functions.
The performance is comparable to a mostly statically linked PIE clang.  (-shared
-Bsymbolic is very similar to -pie.): function calls within libLLVM.so
or libclang-cpp.so has no extra cost compared with a mostly statically linked PIE clang.

Normally I don't recommend -Bsymbolic because

* it can break C++ semantics about address uniqueness of inline functions,
   type_info (exceptions) when there are multiple definitions in the
   process. I believe LLVM+Clang are not subject to such issues.
   We don't throw LLVM/Clang type exceptions.
* it is not compatible with copy relocations[1]. This is not an issue for -Bsymbolic-functions.

-Bsymbolic-functions should be suitable for LLVM+Clang.



LD=ld.lld -j 40 defconfig;  time 'make vmlinux'

# the compile flags may be very different from the clang builds below.
system gcc
     1050.15s user 192.96s system 3015% cpu   41.219 total
     1055.47s user 196.51s system 3022% cpu   41.424 total

clang (libLLVM*.a libclang*.a); LLVM=1
     1588.35s user 193.02s system 3223% cpu   55.259 total
     1613.59s user 193.22s system 3234% cpu   55.861 total
clang (libLLVM.so libclang-cpp.so); LLVM=1
     1870.07s user 222.86s system 3256% cpu 1:04.26 total
     1863.26s user 220.59s system 3219% cpu 1:04.73 total
     1877.79s user 223.98s system 3233% cpu 1:05.00 total
     1859.32s user 221.96s system 3241% cpu 1:04.20 total
clang (libLLVM.so libclang-cpp.so -fno-semantic-interposition); LLVM=1
     1810.47s user 222.98s system 3288% cpu 1:01.83 total
     1790.46s user 219.65s system 3227% cpu 1:02.27 total
     1796.46s user 220.88s system 3139% cpu 1:04.25 total
     1796.55s user 221.28s system 3215% cpu 1:02.75 total
clang (libLLVM.so libclang-cpp.so -fno-semantic-interposition -Wl,-Bsymbolic); LLVM=1
     1608.75s user 221.39s system 3192% cpu   57.333 total
     1607.85s user 220.60s system 3205% cpu   57.042 total
     1598.64s user 191.21s system 3208% cpu   55.778 total
clang (libLLVM.so libclang-cpp.so -fno-semantic-interposition -Wl,-Bsymbolic-functions); LLVM=1
     1617.35s user 220.54s system 3217% cpu   57.115 total



LLVM's reusable component design causes us some overhead here.  Almost
every cross-TU callable function is moved to a public header and
exported, libLLVM.so and libclang-cpp.so have huge dynamic symbol tables.
-Wl,--gc-sections cannot really eliminate much.


(Last, I guess it is a conscious decision that distributions build all
targets instead of just the host -DLLVM_TARGETS_TO_BUILD=host. This
makes cross compilation easy: a single clang can replace various *-linux-gnu-gcc)


[1]: Even if one design goal of -fPIE is to avoid copy relocations, and
   normally there should be no issue on non-x86, there is an unfortunate
   GCC 5 fallout for x86-64 ("x86-64: Optimize access to globals in PIE with copy reloc").
   I'll omit words here as you can find details on https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
   -Bsymbolic-functions avoids such issues.

  reply index

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-29 21:53 Linus Torvalds
2021-04-30  0:19 ` Nick Desaulniers
2021-04-30  2:22   ` Nick Desaulniers
2021-05-01  0:19     ` Nick Desaulniers
2021-05-01  0:23       ` Nick Desaulniers
2021-05-01  0:25         ` Nick Desaulniers
2021-05-01  0:40           ` Nick Desaulniers
2021-05-01  1:22           ` Linus Torvalds
2021-05-01  1:48             ` Nick Desaulniers
2021-05-01  2:16               ` Fangrui Song
2021-05-01  3:32               ` Tom Stellard
2021-05-01 16:32                 ` Linus Torvalds
2021-05-01 19:57                   ` Serge Guelton
2021-05-01 22:39                     ` Linus Torvalds
2021-05-01 23:55                       ` Fangrui Song [this message]
2021-05-01 21:58                   ` David Laight
2021-05-02  9:31                   ` Adrian Bunk
2021-05-02 11:35                     ` David Laight
2021-05-02 16:12                     ` Linus Torvalds
2021-05-02 16:45                       ` Adrian Bunk
2021-05-02 16:49                         ` Linus Torvalds
2021-05-02 17:55                           ` Adrian Bunk
2021-05-02 17:59                             ` Linus Torvalds
2021-05-02 21:48                               ` Adrian Bunk
2021-05-04 22:02                                 ` Miguel Ojeda
2021-05-05  0:58                                   ` Theodore Ts'o
2021-05-05 17:21                                     ` Miguel Ojeda
2021-05-04 21:32                     ` Miguel Ojeda
2021-05-05 11:05                       ` David Laight
2021-05-05 13:53                         ` Miguel Ojeda
2021-05-05 14:13                           ` David Laight
2021-05-05 16:06                             ` Miguel Ojeda
2021-05-05 16:25                               ` David Laight
2021-05-05 17:55                                 ` Miguel Ojeda
2021-05-03  1:03                   ` Maciej W. Rozycki
2021-05-03 14:38                     ` Theodore Ts'o
2021-05-03 14:54                       ` Theodore Ts'o
2021-05-03 17:14                         ` Maciej W. Rozycki
2021-05-03 16:09                       ` David Laight
2021-05-04 23:04                       ` Greg Stark
2021-05-05  0:55                         ` Theodore Ts'o
2021-05-01 23:37               ` Mike Hommey
2021-05-02  5:19               ` Dan Aloni
2021-05-03 16:48                 ` Tom Stellard
2021-05-03 19:00                   ` Fangrui Song
2021-04-30  0:52 ` Nathan Chancellor
2021-04-30  2:21   ` Nick Desaulniers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210501235549.vugtjeb7dmd5xell@google.com \
    --to=maskray@google.com \
    --cc=clang-built-linux@googlegroups.com \
    --cc=felixonmars@archlinux.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=masahiroy@kernel.org \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=sguelton@redhat.com \
    --cc=sylvestre@mozilla.com \
    --cc=torvalds@linux-foundation.org \
    --cc=tstellar@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git