LKML Archive on lore.kernel.org
 help / color / Atom feed
From: "Maciej W. Rozycki" <macro@orcam.me.uk>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Tom Stellard <tstellar@redhat.com>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	Fangrui Song <maskray@google.com>,
	Serge Guelton <sguelton@redhat.com>,
	Sylvestre Ledru <sylvestre@mozilla.com>
Subject: Re: Very slow clang kernel config ..
Date: Mon, 3 May 2021 03:03:31 +0200 (CEST)
Message-ID: <alpine.DEB.2.21.2105020346520.2587@angie.orcam.me.uk> (raw)
In-Reply-To: <CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com>

On Sat, 1 May 2021, Linus Torvalds wrote:

> > Yes, it's intentional.  Dynamic linking libraries from other packages is
> > the Fedora policy[1], and clang and llvm are separate packages (in Fedora).
> 
> Side note: I really wish Fedora stopped doing that.

 I wish they never stop.

> Shared libraries are not a good thing in general. They add a lot of
> overhead in this case, but more importantly they also add lots of
> unnecessary dependencies and complexity, and almost no shared
> libraries are actually version-safe, so it adds absolutely zero
> upside.

 I agree shared libraries are a tough compromise, but there is an 
important upside.  Let me quote myself from a recent discussion 
<https://lore.kernel.org/linux-mips/alpine.DEB.2.21.2103191500040.21463@angie.orcam.me.uk/>:

"Dynamic shared objects (libraries) were invented in early 1990s for two
reasons:

1. To limit the use of virtual memory.  Memory conservation may not be as
   important nowadays in many applications where vast amounts of RAM are
   available, though of course this does not apply everywhere, and still
   it has to be weighed up whether any waste of resources is justified and
   compensated by a gain elsewhere.

2. To make it easy to replace a piece of code shared among many programs,
   so that you don't have to relink them all (or recompile if sources are
   available) when say an issue is found or a feature is added that is
   transparent to applications (for instance a new protocol or a better
   algorithm).  This still stands very much nowadays.

People went through great efforts to support shared libraries, sacrificed
performance for it even back then when the computing power was much lower
than nowadays.  Support was implemented in Linux for the a.out binary
format even, despite the need to go through horrible hoops to get a.out
shared libraries built.  Some COFF environments were adapted for shared
library support too."

 And the context here is a bug in the linker caused all programs built by 
Golang to be broken WRT FPU handling for the 32-bit MIPS configuration, 
due to a bad ABI annotation causing the wrong per-process FPU mode being 
set up at run time (Golang seemed to have got stuck in early 2000s as far 
the MIPS ABI goes and chose to produce what has been considered legacy 
objects for some 10 years now, and nobody noticed in 10 years or so that 
the GNU linker does not handle legacy MIPS objects correctly anymore).  
This could have been fixed easily by rebuilding the Go runtime, but as it 
happens Google chose not to create a shared Go runtime and all programs 
are linked statically except for libc.

 This had led to a desperate attempt to work the issue around crudely in 
the kernel (which cannot be done in a completely foolproof way, because 
there's missing information) so that Debian does not have to rebuild 2000+ 
packages in a stable distribution, which OTOH is a no-no for them.

 Whether distributions package shared libraries in a reasonable manner is 
another matter, and I've lost hope it will ever happen, at least widely 
(there has been an attempt to address that with a distribution called PLD, 
where the policy used to have it that shared libraries coming from a given 
source package need to go into a separate binary package on their own, so 
that several versions of different SONAMEs each of the same shared library 
package can safely coexist in a single system, but I haven't checked it in 
many years whether the policy has been retained, nor actually ever used 
PLD myself).

  Maciej

  parent reply index

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-29 21:53 Linus Torvalds
2021-04-30  0:19 ` Nick Desaulniers
2021-04-30  2:22   ` Nick Desaulniers
2021-05-01  0:19     ` Nick Desaulniers
2021-05-01  0:23       ` Nick Desaulniers
2021-05-01  0:25         ` Nick Desaulniers
2021-05-01  0:40           ` Nick Desaulniers
2021-05-01  1:22           ` Linus Torvalds
2021-05-01  1:48             ` Nick Desaulniers
2021-05-01  2:16               ` Fangrui Song
2021-05-01  3:32               ` Tom Stellard
2021-05-01 16:32                 ` Linus Torvalds
2021-05-01 19:57                   ` Serge Guelton
2021-05-01 22:39                     ` Linus Torvalds
2021-05-01 23:55                       ` Fangrui Song
2021-05-01 21:58                   ` David Laight
2021-05-02  9:31                   ` Adrian Bunk
2021-05-02 11:35                     ` David Laight
2021-05-02 16:12                     ` Linus Torvalds
2021-05-02 16:45                       ` Adrian Bunk
2021-05-02 16:49                         ` Linus Torvalds
2021-05-02 17:55                           ` Adrian Bunk
2021-05-02 17:59                             ` Linus Torvalds
2021-05-02 21:48                               ` Adrian Bunk
2021-05-04 22:02                                 ` Miguel Ojeda
2021-05-05  0:58                                   ` Theodore Ts'o
2021-05-05 17:21                                     ` Miguel Ojeda
2021-05-04 21:32                     ` Miguel Ojeda
2021-05-05 11:05                       ` David Laight
2021-05-05 13:53                         ` Miguel Ojeda
2021-05-05 14:13                           ` David Laight
2021-05-05 16:06                             ` Miguel Ojeda
2021-05-05 16:25                               ` David Laight
2021-05-05 17:55                                 ` Miguel Ojeda
2021-05-03  1:03                   ` Maciej W. Rozycki [this message]
2021-05-03 14:38                     ` Theodore Ts'o
2021-05-03 14:54                       ` Theodore Ts'o
2021-05-03 17:14                         ` Maciej W. Rozycki
2021-05-03 16:09                       ` David Laight
2021-05-04 23:04                       ` Greg Stark
2021-05-05  0:55                         ` Theodore Ts'o
2021-05-01 23:37               ` Mike Hommey
2021-05-02  5:19               ` Dan Aloni
2021-05-03 16:48                 ` Tom Stellard
2021-05-03 19:00                   ` Fangrui Song
2021-04-30  0:52 ` Nathan Chancellor
2021-04-30  2:21   ` Nick Desaulniers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.21.2105020346520.2587@angie.orcam.me.uk \
    --to=macro@orcam.me.uk \
    --cc=clang-built-linux@googlegroups.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=masahiroy@kernel.org \
    --cc=maskray@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=sguelton@redhat.com \
    --cc=sylvestre@mozilla.com \
    --cc=torvalds@linux-foundation.org \
    --cc=tstellar@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git