LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Tom Stellard <tstellar@redhat.com>
Cc: Nick Desaulniers <ndesaulniers@google.com>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	Fangrui Song <maskray@google.com>,
	Serge Guelton <sguelton@redhat.com>,
	Sylvestre Ledru <sylvestre@mozilla.com>
Subject: Re: Very slow clang kernel config ..
Date: Sat, 1 May 2021 09:32:25 -0700
Message-ID: <CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com> (raw)
In-Reply-To: <1c5e05fa-a246-9456-ff4e-287960acb18c@redhat.com>

On Fri, Apr 30, 2021 at 8:33 PM Tom Stellard <tstellar@redhat.com> wrote:
>
> Yes, it's intentional.  Dynamic linking libraries from other packages is
> the Fedora policy[1], and clang and llvm are separate packages (in Fedora).

Side note: I really wish Fedora stopped doing that.

Shared libraries are not a good thing in general. They add a lot of
overhead in this case, but more importantly they also add lots of
unnecessary dependencies and complexity, and almost no shared
libraries are actually version-safe, so it adds absolutely zero
upside.

Yes, it can save on disk use, but unless it's some very core library
used by a lot of things (ie particularly things like GUI libraries
like gnome or Qt or similar), the disk savings are often not all that
big - and disk is cheap. And the memory savings are often actually
negative (again, unless it's some big library that is typically used
by lots of different programs at the same time).

In this case, for example, it's true that a parallel build will be
running possibly hundreds of copies of clang at the same time - and
they'll all share the shared llvm library. But they'd share those same
pages even if it wasn't a shared library, because it's the same
executable! And the dynamic linking will actually cause a lot _less_
sharing because of all the fixups.

We hit this in the subsurface project too. We had a couple of
libraries that *nobody* else used. Literally *nobody*. But the Fedora
policy meant that a Fedora package had to go the extra mile to make
those other libraries be shared libraries, for actual negative gain,
and a much more fragile end result (since those libraries were in no
way compatible across different versions - so it all had to be updated
in lock-step).

I think people have this incorrect picture that "shared libraries are
inherently good". They really really aren't. They cause a lot of
problems, and the advantage really should always be weighed against
those (big) disadvantages.

Pretty much the only case shared libraries really make sense is for
truly standardized system libraries that are everywhere, and are part
of the base distro.

[ Or, for those very rare programs that end up dynamically loading
rare modules at run-time - not at startup - because that's their
extension model. But that's a different kind of "shared library"
entirely, even if ELF makes the technical distinction between
"loadable module" and "shared library" be a somewhat moot point ]

                        Linus

  reply index

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-29 21:53 Linus Torvalds
2021-04-30  0:19 ` Nick Desaulniers
2021-04-30  2:22   ` Nick Desaulniers
2021-05-01  0:19     ` Nick Desaulniers
2021-05-01  0:23       ` Nick Desaulniers
2021-05-01  0:25         ` Nick Desaulniers
2021-05-01  0:40           ` Nick Desaulniers
2021-05-01  1:22           ` Linus Torvalds
2021-05-01  1:48             ` Nick Desaulniers
2021-05-01  2:16               ` Fangrui Song
2021-05-01  3:32               ` Tom Stellard
2021-05-01 16:32                 ` Linus Torvalds [this message]
2021-05-01 19:57                   ` Serge Guelton
2021-05-01 22:39                     ` Linus Torvalds
2021-05-01 23:55                       ` Fangrui Song
2021-05-01 21:58                   ` David Laight
2021-05-02  9:31                   ` Adrian Bunk
2021-05-02 11:35                     ` David Laight
2021-05-02 16:12                     ` Linus Torvalds
2021-05-02 16:45                       ` Adrian Bunk
2021-05-02 16:49                         ` Linus Torvalds
2021-05-02 17:55                           ` Adrian Bunk
2021-05-02 17:59                             ` Linus Torvalds
2021-05-02 21:48                               ` Adrian Bunk
2021-05-04 22:02                                 ` Miguel Ojeda
2021-05-05  0:58                                   ` Theodore Ts'o
2021-05-05 17:21                                     ` Miguel Ojeda
2021-05-04 21:32                     ` Miguel Ojeda
2021-05-05 11:05                       ` David Laight
2021-05-05 13:53                         ` Miguel Ojeda
2021-05-05 14:13                           ` David Laight
2021-05-05 16:06                             ` Miguel Ojeda
2021-05-05 16:25                               ` David Laight
2021-05-05 17:55                                 ` Miguel Ojeda
2021-05-03  1:03                   ` Maciej W. Rozycki
2021-05-03 14:38                     ` Theodore Ts'o
2021-05-03 14:54                       ` Theodore Ts'o
2021-05-03 17:14                         ` Maciej W. Rozycki
2021-05-03 16:09                       ` David Laight
2021-05-04 23:04                       ` Greg Stark
2021-05-05  0:55                         ` Theodore Ts'o
2021-05-01 23:37               ` Mike Hommey
2021-05-02  5:19               ` Dan Aloni
2021-05-03 16:48                 ` Tom Stellard
2021-05-03 19:00                   ` Fangrui Song
2021-04-30  0:52 ` Nathan Chancellor
2021-04-30  2:21   ` Nick Desaulniers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com' \
    --to=torvalds@linux-foundation.org \
    --cc=clang-built-linux@googlegroups.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=masahiroy@kernel.org \
    --cc=maskray@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=sguelton@redhat.com \
    --cc=sylvestre@mozilla.com \
    --cc=tstellar@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git