[PATCH 0/4] [Jethro/master] Fix prelink support

[PATCH 0/4] [Jethro/master] Fix prelink support

[Mark Hatle]

Fix prelink support for Jethro/Master.

* Fix glibc defect with it's prelink API
* Fix GCC configuration for PowerPC (32-bit)
* Update to newer prelink release (fix IA32, ARM and MIPS)
* Revert image-prelink being disabled

Changes are availabe in the git repository at:

git://git.yoctoproject.org/poky-contrib mhatle/jethro_prelink


Mark Hatle (4):
  glibc: Fix ld.so / prelink interface for
    ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA
  gcc: Update default Power GCC settings to use secure-plt
  prelink: Fix various prelink issues on IA32, ARM, and MIPS.
  Revert "local.conf.sample: Disable image-prelink by default"

 meta/conf/local.conf.sample                        |  4 +-
 ...TYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch | 84 ++++++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.22.bb              |  1 +
 meta/recipes-devtools/gcc/gcc-common.inc           |  5 ++
 meta/recipes-devtools/gcc/gcc-configure-common.inc |  1 +
 meta/recipes-devtools/gcc/gcc-cross-initial.inc    |  1 +
 meta/recipes-devtools/prelink/prelink_git.bb       |  4 +-
 7 files changed, 95 insertions(+), 5 deletions(-)
 create mode 100644 meta/recipes-core/glibc/glibc/0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch

-- 
1.9.3

[PATCH 1/4] glibc: Fix ld.so / prelink interface for ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA

[Mark Hatle]

A bug in glibc 2.22's ld.so interface for the prelink support causes
the displayed values to be incorrect.  The included path fixes this
issue.

   Clear ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA for prelink

   prelink runs ld.so with the environment variable LD_TRACE_PRELINKING
   set to dump the relocation type class from _dl_debug_bindings.  prelink
   has the following relocation type classes:

   where ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA has a conflict with
   RTYPE_CLASS_TLS.

   Since prelink doesn't use ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA, we
   should clear the ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA bit when the
   DL_DEBUG_PRELINK bit is set.

[Yocto #8626]

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 ...TYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch | 84 ++++++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.22.bb              |  1 +
 2 files changed, 85 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc/0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch

diff --git a/meta/recipes-core/glibc/glibc/0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch b/meta/recipes-core/glibc/glibc/0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch
new file mode 100644
index 0000000..3455df1
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch
@@ -0,0 +1,84 @@
+From cadaf1336332ca7bcdfe4a400776e5782a20e26d Mon Sep 17 00:00:00 2001
+From: "H.J. Lu" <hjl.tools@gmail.com>
+Date: Wed, 28 Oct 2015 07:49:44 -0700
+Subject: [PATCH] Keep only ELF_RTYPE_CLASS_{PLT|COPY} bits for prelink
+
+prelink runs ld.so with the environment variable LD_TRACE_PRELINKING
+set to dump the relocation type class from _dl_debug_bindings.  prelink
+has the following relocation type classes:
+
+ #define RTYPE_CLASS_VALID       8
+ #define RTYPE_CLASS_PLT         (8|1)
+ #define RTYPE_CLASS_COPY        (8|2)
+ #define RTYPE_CLASS_TLS         (8|4)
+
+where ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA has a conflict with
+RTYPE_CLASS_TLS.
+
+Since prelink only uses ELF_RTYPE_CLASS_PLT and ELF_RTYPE_CLASS_COPY
+bits, we should clear the other bits when the DL_DEBUG_PRELINK bit is
+set.
+
+	[BZ #19178]
+	* elf/dl-lookup.c (RTYPE_CLASS_VALID): New.
+	(RTYPE_CLASS_PLT): Likewise.
+	(RTYPE_CLASS_COPY): Likewise.
+	(RTYPE_CLASS_TLS): Likewise.
+	(_dl_debug_bindings): Use RTYPE_CLASS_TLS and RTYPE_CLASS_VALID
+	to set relocation type class for DL_DEBUG_PRELINK.  Keep only
+	ELF_RTYPE_CLASS_PLT and ELF_RTYPE_CLASS_COPY bits for
+	DL_DEBUG_PRELINK.
+
+Upstream-Status: submitted (https://sourceware.org/bugzilla/show_bug.cgi?id=19178)
+
+Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
+---
+ elf/dl-lookup.c | 21 +++++++++++++++++++--
+ 1 file changed, 19 insertions(+), 2 deletions(-)
+
+diff --git a/elf/dl-lookup.c b/elf/dl-lookup.c
+index 581fb20..6ae6cc3 100644
+--- a/elf/dl-lookup.c
++++ b/elf/dl-lookup.c
+@@ -1016,6 +1016,18 @@ _dl_debug_bindings (const char *undef_name, struct link_map *undef_map,
+ #ifdef SHARED
+   if (GLRO(dl_debug_mask) & DL_DEBUG_PRELINK)
+     {
++/* ELF_RTYPE_CLASS_XXX must match RTYPE_CLASS_XXX used by prelink with
++   LD_TRACE_PRELINKING.  */
++#define RTYPE_CLASS_VALID	8
++#define RTYPE_CLASS_PLT		(8|1)
++#define RTYPE_CLASS_COPY	(8|2)
++#define RTYPE_CLASS_TLS		(8|4)
++#if ELF_RTYPE_CLASS_PLT != 0 && ELF_RTYPE_CLASS_PLT != 1
++# error ELF_RTYPE_CLASS_PLT must be 0 or 1!
++#endif
++#if ELF_RTYPE_CLASS_COPY != 0 && ELF_RTYPE_CLASS_COPY != 2
++# error ELF_RTYPE_CLASS_COPY must be 0 or 2!
++#endif
+       int conflict = 0;
+       struct sym_val val = { NULL, NULL };
+ 
+@@ -1071,12 +1083,17 @@ _dl_debug_bindings (const char *undef_name, struct link_map *undef_map,
+ 
+       if (value->s)
+ 	{
++	  /* Keep only ELF_RTYPE_CLASS_PLT and ELF_RTYPE_CLASS_COPY
++	     bits since since prelink only uses them.  */
++	  type_class &= ELF_RTYPE_CLASS_PLT | ELF_RTYPE_CLASS_COPY;
+ 	  if (__glibc_unlikely (ELFW(ST_TYPE) (value->s->st_info)
+ 				== STT_TLS))
+-	    type_class = 4;
++	    /* Clear the RTYPE_CLASS_VALID bit in RTYPE_CLASS_TLS.  */
++	    type_class = RTYPE_CLASS_TLS & ~RTYPE_CLASS_VALID;
+ 	  else if (__glibc_unlikely (ELFW(ST_TYPE) (value->s->st_info)
+ 				     == STT_GNU_IFUNC))
+-	    type_class |= 8;
++	    /* Set the RTYPE_CLASS_VALID bit.  */
++	    type_class |= RTYPE_CLASS_VALID;
+ 	}
+ 
+       if (conflict
+-- 
+1.9.3
+
diff --git a/meta/recipes-core/glibc/glibc_2.22.bb b/meta/recipes-core/glibc/glibc_2.22.bb
index 2494ad7..1b23597 100644
--- a/meta/recipes-core/glibc/glibc_2.22.bb
+++ b/meta/recipes-core/glibc/glibc_2.22.bb
@@ -39,6 +39,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
            file://0026-eglibc-dl_debug_mask-is-controlled-by-__OPTION_EGLIB.patch \
            file://0027-eglibc-use-option-groups-Conditionally-exclude-c-tes.patch \
            file://nscd-no-bash.patch \
+           file://0028-Clear-ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA-for-prel.patch \
 "
 
 SRC_URI += "\
-- 
1.9.3

[PATCH 2/4] gcc: Update default Power GCC settings to use secure-plt

[Mark Hatle]

The gcc default, bss-plt, will cause errors when using the prelinker.  All
other distributions that I am aware of are using the the secure-plt.  For an
explanation of the differences, the gcc docs:

  Current PowerPC GCC accepts a `-msecure-plt' option that generates code
  capable of using a newer PLT and GOT layout that has the security
  advantage of no executable section ever needing to be writable and no
  writable section ever being executable. PowerPC ld will generate this
  layout, including stubs to access the PLT, if all input files (including
  startup and static libraries) were compiled with `-msecure-plt'.
  `--bss-plt' forces the old BSS PLT (and GOT layout) which can give
  slightly better performance.

The security of the new PLT and ability to run the prelinker outweigh
any performance penalty.

The secure-plt is enabled by default.  The old bss-plt can be enabled by
selecting 'bssplt' in the DISTRO_FEATURES.

[Yocto #7991]

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-common.inc           | 5 +++++
 meta/recipes-devtools/gcc/gcc-configure-common.inc | 1 +
 meta/recipes-devtools/gcc/gcc-cross-initial.inc    | 1 +
 3 files changed, 7 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-common.inc b/meta/recipes-devtools/gcc/gcc-common.inc
index d63c07f..6f2f224 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -25,6 +25,11 @@ def get_gcc_mips_plt_setting(bb, d):
         return "--with-mips-plt"
     return ""
 
+def get_gcc_ppc_plt_settings(bb, d):
+    if d.getVar('TRANSLATED_TARGET_ARCH', True) in [ 'powerpc' ] and not bb.utils.contains('DISTRO_FEATURES', 'bssplt', True, False, d):
+        return "--enable-secureplt"
+    return ""
+
 def get_long_double_setting(bb, d):
     if d.getVar('TRANSLATED_TARGET_ARCH', True) in [ 'powerpc', 'powerpc64' ] and d.getVar('TCLIBC', True) in [ 'uclibc', 'glibc' ]:
         return "--with-long-double-128"
diff --git a/meta/recipes-devtools/gcc/gcc-configure-common.inc b/meta/recipes-devtools/gcc/gcc-configure-common.inc
index a14be73..cee6f4a 100644
--- a/meta/recipes-devtools/gcc/gcc-configure-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-configure-common.inc
@@ -47,6 +47,7 @@ EXTRA_OECONF = "\
     ${EXTRA_OECONF_GCC_FLOAT} \
     ${EXTRA_OECONF_PATHS} \
     ${@get_gcc_mips_plt_setting(bb, d)} \
+    ${@get_gcc_ppc_plt_settings(bb, d)} \
     ${@get_long_double_setting(bb, d)} \
     ${@get_gcc_multiarch_setting(bb, d)} \
 "
diff --git a/meta/recipes-devtools/gcc/gcc-cross-initial.inc b/meta/recipes-devtools/gcc/gcc-cross-initial.inc
index 7197447..c0fa139 100644
--- a/meta/recipes-devtools/gcc/gcc-cross-initial.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross-initial.inc
@@ -26,6 +26,7 @@ EXTRA_OECONF = "\
     ${EXTRA_OECONF_INITIAL} \
     ${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', '--with-ld=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}ld.bfd', '', d)} \
     ${EXTRA_OECONF_GCC_FLOAT} \
+    ${@get_gcc_ppc_plt_settings(bb, d)} \
 "
 
 EXTRA_OECONF += "--with-native-system-header-dir=${SYSTEMHEADERS}"
-- 
1.9.3

[PATCH 3/4] prelink: Fix various prelink issues on IA32, ARM, and MIPS.

[Mark Hatle]

Fix the following issues:

IA32 / ARM - Resync to glibc-2.22, fix a mismatch w/ glibc's ld.so
MIPS - Ignore the new SHT_MIPS_ABIFLAGS
ARM - Fix missing ARM IFUNC support chunk

Also upstream prelink project no longer has a 'trunk' directory.

[Yocto #8627]
[Yocto #8227]

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/recipes-devtools/prelink/prelink_git.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/prelink/prelink_git.bb b/meta/recipes-devtools/prelink/prelink_git.bb
index 79a5f50..e223ef6 100644
--- a/meta/recipes-devtools/prelink/prelink_git.bb
+++ b/meta/recipes-devtools/prelink/prelink_git.bb
@@ -8,7 +8,7 @@ and executables, so that far fewer relocations need to be resolved at \
 runtime and thus programs come up faster."
 LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=c93c0550bd3173f4504b2cbd8991e50b"
-SRCREV = "cdee5a4dd226cc5e9f30f370067a9031f398ef3c"
+SRCREV = "927979bbd115eeb8a75db3231906ef6aca4c4eb6"
 PV = "1.0+git${SRCPV}"
 
 #
@@ -35,7 +35,7 @@ SRC_URI = "git://git.yoctoproject.org/prelink-cross.git;branch=cross_prelink \
 TARGET_OS_ORIG := "${TARGET_OS}"
 OVERRIDES_append = ":${TARGET_OS_ORIG}"
 
-S = "${WORKDIR}/git/trunk"
+S = "${WORKDIR}/git"
 
 inherit autotools 
 
-- 
1.9.3

[PATCH 4/4] Revert "local.conf.sample: Disable image-prelink by default"

[Mark Hatle]

This reverts commit 6dd28030f323d7106a02ec54ce4e249561ab0836.

Prelink now works properly again.  Re-enable the functionality.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/conf/local.conf.sample | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
index 9d1480e..3ae24ab 100644
--- a/meta/conf/local.conf.sample
+++ b/meta/conf/local.conf.sample
@@ -128,9 +128,7 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
 #   - 'image-swab' to perform host system intrusion detection
 # NOTE: if listing mklibs & prelink both, then make sure mklibs is before prelink
 # NOTE: mklibs also needs to be explicitly enabled for a given image, see local.conf.extended
-# NOTE: image-prelink is currently broken due to problems with the prelinker.  It is advised
-# that you do NOT run the prelinker at this time. 
-USER_CLASSES ?= "buildstats image-mklibs"
+USER_CLASSES ?= "buildstats image-mklibs image-prelink"
 
 
 #
-- 
1.9.3

Re: [PATCH 4/4] Revert "local.conf.sample: Disable image-prelink by default"

[Khem Raj]

On Mon, Nov 2, 2015 at 4:07 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> This reverts commit 6dd28030f323d7106a02ec54ce4e249561ab0836.
>
> Prelink now works properly again.  Re-enable the functionality.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
>  meta/conf/local.conf.sample | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
> index 9d1480e..3ae24ab 100644
> --- a/meta/conf/local.conf.sample
> +++ b/meta/conf/local.conf.sample
> @@ -128,9 +128,7 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
>  #   - 'image-swab' to perform host system intrusion detection
>  # NOTE: if listing mklibs & prelink both, then make sure mklibs is before prelink
>  # NOTE: mklibs also needs to be explicitly enabled for a given image, see local.conf.extended
> -# NOTE: image-prelink is currently broken due to problems with the prelinker.  It is advised
> -# that you do NOT run the prelinker at this time.
> -USER_CLASSES ?= "buildstats image-mklibs"
> +USER_CLASSES ?= "buildstats image-mklibs image-prelink"

With all the discussions around benefits of it recently, I think its
better disabled IMO,

Re: [PATCH 4/4] Revert "local.conf.sample: Disable image-prelink by default"

[Mark Hatle]

On 11/2/15 6:10 PM, Khem Raj wrote:
> On Mon, Nov 2, 2015 at 4:07 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> This reverts commit 6dd28030f323d7106a02ec54ce4e249561ab0836.
>>
>> Prelink now works properly again.  Re-enable the functionality.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>>  meta/conf/local.conf.sample | 4 +---
>>  1 file changed, 1 insertion(+), 3 deletions(-)
>>
>> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
>> index 9d1480e..3ae24ab 100644
>> --- a/meta/conf/local.conf.sample
>> +++ b/meta/conf/local.conf.sample
>> @@ -128,9 +128,7 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>  #   - 'image-swab' to perform host system intrusion detection
>>  # NOTE: if listing mklibs & prelink both, then make sure mklibs is before prelink
>>  # NOTE: mklibs also needs to be explicitly enabled for a given image, see local.conf.extended
>> -# NOTE: image-prelink is currently broken due to problems with the prelinker.  It is advised
>> -# that you do NOT run the prelinker at this time.
>> -USER_CLASSES ?= "buildstats image-mklibs"
>> +USER_CLASSES ?= "buildstats image-mklibs image-prelink"
> 
> With all the discussions around benefits of it recently, I think its
> better disabled IMO,
> 

I disagree.  I've got rough estimate figures for prelink improvement to the system.

core-image-sato w/ systemd enabled:

MIPS64 - n64 - 1.5MB of ram saved
MIPS64 - n32 - 500KB of ram saved
MIPS32 - n32 - 300KB of ram saved

ARM - 560k of ram saved

PPC - 380k of ram saved

IA32 - forgot to write it down, was around 500k.


Testing also showed sysvinit saves even more memory due to COW pages then
systemd.  (I was a bit surprised by that.)  My testing consisted of building a
filesystem and booting it.  Waiting for everything to 'finish' and then
comparing the output of the 'free' command in KB.


As for startup time savings, I don't have specific figures BUT, for a few key
cases the load time of the executable is up to 50% faster.  For instance busybox
on the qemux86-64.  Keep in mind that busybox is pretty simple and uses only a
few libraries.  The more libraries in use, the more prelink will affect load times.

The busybox binary loading and then exiting (LD_BIND_NOW=1 LD_DEBUG=statistics):

(not prelinked with LD_BIND_NOW defined to simulate worst case, all relocations
needed)

       566:
       566:     runtime linker statistics:
       566:       total startup time in dynamic loader: 182539970 cycles
       566:                 time needed for relocation: 40894259 cycles (22.4%)

       566:                      number of relocations: 443
       566:           number of relocations from cache: 3
       566:             number of relative relocations: 1200
       566:                time needed to load objects: 63004343 cycles (34.5%)

       566:
       566:     runtime linker statistics:
       566:                final number of relocations: 445
       566:     final number of relocations from cache: 3

(prelinked, same LD_BIND_NOW defined)

       524:
       524:     runtime linker statistics:
       524:       total startup time in dynamic loader: 32420607 cycles
       524:                 time needed for relocation: 2812608 cycles (8.6%)
       524:                      number of relocations: 4
       524:           number of relocations from cache: 76
       524:             number of relative relocations: 0
       524:                time needed to load objects: 15549810 cycles (47.9%)

       524:
       524:     runtime linker statistics:
       524:                final number of relocations: 6
       524:     final number of relocations from cache: 76

(both of the above were run with a hot cache, multiple times until the system
stabilized and returned results that were fairly consistent)

So the executable load time dropped to 17% (32420607/182539970) of the original
value due to being prelinked.

Re: [PATCH 4/4] Revert "local.conf.sample: Disable image-prelink by default"

[Khem Raj]

On Mon, Nov 2, 2015 at 6:04 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 11/2/15 6:10 PM, Khem Raj wrote:
>> On Mon, Nov 2, 2015 at 4:07 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> This reverts commit 6dd28030f323d7106a02ec54ce4e249561ab0836.
>>>
>>> Prelink now works properly again.  Re-enable the functionality.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>> ---
>>>  meta/conf/local.conf.sample | 4 +---
>>>  1 file changed, 1 insertion(+), 3 deletions(-)
>>>
>>> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
>>> index 9d1480e..3ae24ab 100644
>>> --- a/meta/conf/local.conf.sample
>>> +++ b/meta/conf/local.conf.sample
>>> @@ -128,9 +128,7 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>  #   - 'image-swab' to perform host system intrusion detection
>>>  # NOTE: if listing mklibs & prelink both, then make sure mklibs is before prelink
>>>  # NOTE: mklibs also needs to be explicitly enabled for a given image, see local.conf.extended
>>> -# NOTE: image-prelink is currently broken due to problems with the prelinker.  It is advised
>>> -# that you do NOT run the prelinker at this time.
>>> -USER_CLASSES ?= "buildstats image-mklibs"
>>> +USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>
>> With all the discussions around benefits of it recently, I think its
>> better disabled IMO,
>>
>
> I disagree.  I've got rough estimate figures for prelink improvement to the system.
>

core-image-sato is one image, we do not know if all apps in world
build are prelink friendly, I have seen atleast some packages
misbehave, (net-snmp on mips32) when using applications from meta-oe
and big C++ applications internally, Debugging failures due to prelink
issues is very hard, thats why I think
making it default may not give best experience for users.

> core-image-sato w/ systemd enabled:
>
> MIPS64 - n64 - 1.5MB of ram saved
> MIPS64 - n32 - 500KB of ram saved
> MIPS32 - n32 - 300KB of ram saved
>
> ARM - 560k of ram saved
>
> PPC - 380k of ram saved
>
> IA32 - forgot to write it down, was around 500k.
>
>
> Testing also showed sysvinit saves even more memory due to COW pages then
> systemd.  (I was a bit surprised by that.)  My testing consisted of building a
> filesystem and booting it.  Waiting for everything to 'finish' and then
> comparing the output of the 'free' command in KB.
>
>
> As for startup time savings, I don't have specific figures BUT, for a few key
> cases the load time of the executable is up to 50% faster.  For instance busybox
> on the qemux86-64.  Keep in mind that busybox is pretty simple and uses only a
> few libraries.  The more libraries in use, the more prelink will affect load times.
>
> The busybox binary loading and then exiting (LD_BIND_NOW=1 LD_DEBUG=statistics):
>
> (not prelinked with LD_BIND_NOW defined to simulate worst case, all relocations
> needed)
>
>        566:
>        566:     runtime linker statistics:
>        566:       total startup time in dynamic loader: 182539970 cycles
>        566:                 time needed for relocation: 40894259 cycles (22.4%)
>
>        566:                      number of relocations: 443
>        566:           number of relocations from cache: 3
>        566:             number of relative relocations: 1200
>        566:                time needed to load objects: 63004343 cycles (34.5%)
>
>        566:
>        566:     runtime linker statistics:
>        566:                final number of relocations: 445
>        566:     final number of relocations from cache: 3
>
> (prelinked, same LD_BIND_NOW defined)
>
>        524:
>        524:     runtime linker statistics:
>        524:       total startup time in dynamic loader: 32420607 cycles
>        524:                 time needed for relocation: 2812608 cycles (8.6%)
>        524:                      number of relocations: 4
>        524:           number of relocations from cache: 76
>        524:             number of relative relocations: 0
>        524:                time needed to load objects: 15549810 cycles (47.9%)
>
>        524:
>        524:     runtime linker statistics:
>        524:                final number of relocations: 6
>        524:     final number of relocations from cache: 76
>
> (both of the above were run with a hot cache, multiple times until the system
> stabilized and returned results that were fairly consistent)
>
> So the executable load time dropped to 17% (32420607/182539970) of the original
> value due to being prelinked.