Linux-KBuild Archive on lore.kernel.org
 help / color / Atom feed
From: Andi Kleen <andi@firstfloor.org>
To: linux-kernel@vger.kernel.org
Cc: x86@kernel.org, mmarek@suse.cz, linux-kbuild@vger.kernel.org,
	JBeulich@suse.com, akpm@linux-foundation.org,
	Andi Kleen <ak@linux.intel.com>
Subject: [PATCH 68/74] Kbuild, lto: Add LTO build Documentation
Date: Sat, 18 Aug 2012 19:57:04 -0700
Message-ID: <1345345030-22211-69-git-send-email-andi@firstfloor.org> (raw)
In-Reply-To: <1345345030-22211-1-git-send-email-andi@firstfloor.org>

From: Andi Kleen <ak@linux.intel.com>

Add build documentation for LTO.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 Documentation/lto-build |  115 +++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 115 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/lto-build

diff --git a/Documentation/lto-build b/Documentation/lto-build
new file mode 100644
index 0000000..5da427a
--- /dev/null
+++ b/Documentation/lto-build
@@ -0,0 +1,115 @@
+Link time optimization (LTO) for the Linux kernel
+
+This is an experimental feature which still has various problems.
+
+Link Time Optimization allows the compiler to optimize the complete program
+instead of just each file.  Link Time Optimization was a new feature in gcc 4.6,
+but only really works with gcc 4.7. The kernel LTO build also requires
+the Linux binutils (the normal FSF releases do not work at the moment)
+
+The compiler can inline functions between files and do some other global
+optimizations.  It will also drop unused functions which can make the kernel
+image smaller in some circumstances.  The binary gets somewhat larger.
+In return the resulting kernels (usually) have better performance.
+
+Build time and memory consumption at build time will increase.
+The build time penalty depends on the size of the vmlinux. Reasonable 
+sized vmlinux build about twice as long, much larger monolithic kernels 
+like allyesconfig ~4x as long. Modular kernels are less affected.
+
+Normal "reasonable" builds work with less than 4GB of RAM, but very large
+configurations like allyesconfig can need upto 9GB. 
+
+Issues:
+- Various workarounds in kernel needed for toolchain problem.
+- A few kernel features are currently incompatible with LTO, in particular
+function tracing, because they require special compiler flags for
+specific files, which is not supported in LTO right now.
+- The build is faster with LTO_SLIM enabled, but this still triggers
+problems in some circumstances (currently disabled)
+- Jobserver control for -j does not work correctly for the final
+LTO phase. The makefiles hardcodes -j<number of online cpus>
+
+Configuration:
+- Enable CONFIG_LTO_MENU and then disable CONFIG_LTO_DISABLE.
+This is mainly to not have allyesconfig default to LTO.
+- FUNCTION_TRACER, STACK_TRACER, FUNCTION_GRAPH_TRACER have to disabled
+because they are currently incompatible with LTO.
+- MODVERSIONS have to be disabled because they are not fixed for LTO
+yet.
+
+Requirements:
+- Enough memory: 4GB for a standard build, ~8GB for allyesconfig
+If you are tight on memory and use tmpfs as /tmp define TMPDIR and 
+point it to a directory on disk.  The peak memory usage
+happens single threaded (when lto-wpa merges types), so dialing
+back -j options will not help much.
+
+A 32bit compiler is unlikely to work due to the memory requirements.
+You can however build a kernel targetted at 32bit on a 64bit host.
+
+- Get the Linux binutils from
+http://www.kernel.org/pub/linux/devel/binutils/
+Sorry standard binutils releases don't work
+The kernel build has to use this linker, so if it is installed 
+in a non standard location use LD=... on the make line.
+
+- gcc 4.7 built with plugin ld (--with-plugin-ld) also pointing to the 
+linker from the Linux binutils and LTO 
+
+Example build procedure for the tool chain and kernel. This does not 
+overwrite the standard compiler toolchain on the system. If you already
+have a suitable gcc 4.7+ compiler and linker the toolchain build can
+be skipped (note that a distribution gcc 4.7 is not necessarily
+correctly configured for LTO)
+
+Get the Linux binutils from http://www.kernel.org/pub/linux/devel/binutils/
+The standard binutils do not work at this point!
+
+Unpack binutils
+
+cd binutils-VERSION  (or plain binutils in some versions)
+./configure --prefix=/opt/binutils-VERSION --enable-plugins
+nice -n20 make -j$(getconf _NPROCESSORS_ONLN)
+sudo make install
+sudo ln -sf /opt/binutils-VERSION/bin/ld /usr/local/bin/ld-plugin
+
+Unpack gcc-4.7
+
+mkdir obj-gcc
+# please don't skip this cd. the build will not work correctly in the 
+# source dir, you have to use the separate object dir
+cd obj-gcc
+# make sure to install gmp-devel and mpfr-devel
+# and the 32bit glibc package if you have a multilib system
+# if mpc-devel is not there get it from
+# http://www.multiprecision.org/mpc/download/mpc-0.8.2.tar.gz
+# and install in gcc-4.7*/mpc
+../gcc-4.7*/configure --prefix=/opt/gcc-4.7 --enable-lto \
+--with-plugin-ld=/usr/local/bin/ld-plugin  \
+--disable-nls --enable-languages=c,c++ \
+--disable-libstdcxx-pch
+nice -n20 make -j$(getconf _NPROCESSORS_ONLN) 
+sudo make install-no-fixedincludes
+sudo ln -sf /opt/gcc-4.7/bin/gcc /usr/local/bin/gcc47
+sudo ln -sf /opt/gcc-4.7/bin/gcc-ar /usr/local/bin/gcc-ar47
+
+# get lto tree in linux-lto
+
+mkdir obj-lto
+cd obj-lto
+# copy a suitable kernel config file into .config
+make -C ../linux-lto O=$(pwd)  oldconfig
+./source/scripts/config --disable function_tracer --disable function_graph_tracer \
+			 --disable stack_tracer --enable lto_menu \
+                         --disable lto_disable --disable lto_debug --disable lto_slim
+export TMPDIR=$(pwd) 
+# this lowers memory usage with /tmp=tmpfs
+# note the special ar is only needed if CONFIG_LTO_SLIM is enabled
+# The PATH is that gcc-ar finds a plugin aware ar, if your standard
+# binutils doesn't support that. If the standard ar supports --plugin
+# it is not needed
+PATH=/opt/binutils-VERSION:$PATH nice -n20 make CC=gcc47 LD=ld-plugin AR=gcc-ar47 \
+-j $(getconf _NPROCESSORS_ONLN)
+
+Andi Kleen
-- 
1.7.7.6


  parent reply index

Thread overview: 128+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-08-19  2:55 RFC: Link Time Optimization support for the kernel Andi Kleen
2012-08-19  2:55 ` [PATCH 01/74] Add __visible Andi Kleen
2012-08-19  2:55 ` [PATCH 02/74] sections: Make all standard section identifiers __visible Andi Kleen
2012-08-19  2:55 ` [PATCH 03/74] sections: Make external kallsyms tables __visible Andi Kleen
2012-08-19  7:53   ` Jan Beulich
2012-08-19 15:29     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 04/74] sections: Add __visible to ia64 sections Andi Kleen
2012-08-19  2:56 ` [PATCH 05/74] sections: Add __visible to m68k sections Andi Kleen
2012-08-19  2:56 ` [PATCH 06/74] sections: Add __visible to powerpc sections Andi Kleen
2012-08-19  2:56 ` [PATCH 07/74] sections: Add __visible to sh sections Andi Kleen
2012-08-19  2:56 ` [PATCH 08/74] sections: Add __visible to tile sections Andi Kleen
2012-08-19  2:56 ` [PATCH 09/74] sections: Add __visible to x86 sections Andi Kleen
2012-08-19  2:56 ` [PATCH 10/74] sections: Add __visible to drivers/{base,pci} sections Andi Kleen
2012-08-19  2:56 ` [PATCH 11/74] sections: Add __visible to rapidio sections Andi Kleen
2012-08-19  2:56 ` [PATCH 12/74] sections: Add __visible to jump_label sections Andi Kleen
2012-08-19  2:56 ` [PATCH 13/74] sections: Add __visible to init/* sections Andi Kleen
2012-08-19  2:56 ` [PATCH 14/74] sections: Add __visible to kernel/* sections Andi Kleen
2012-08-19  2:56 ` [PATCH 15/74] sections: Add __visible to kernel/trace/* sections Andi Kleen
2012-08-19  2:56 ` [PATCH 16/74] sections: Add __visible to lib/* sections Andi Kleen
2012-08-19  2:56 ` [PATCH 17/74] lto: Make asmlinkage __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 18/74] lto, wan/sbni: Make inline assembler symbols visible and assembler global Andi Kleen
2012-08-19  2:56 ` [PATCH 19/74] x86, lto: Add missing asmlinkages and __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 20/74] x86, lto: Change dotraplinkage into __visible on 32bit Andi Kleen
2012-09-01 14:44   ` Michal Marek
2012-08-19  2:56 ` [PATCH 21/74] lto: Make lockdep_sys_exit asmlinkage Andi Kleen
2012-08-19  2:56 ` [PATCH 22/74] lto: Change kernel_execve to asmlinkage for all architectures Andi Kleen
2012-08-19  2:56 ` [PATCH 23/74] lto, mutex: Mark __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 24/74] lto: Mark do_exit asmlinkage Andi Kleen
2012-08-19  2:56 ` [PATCH 25/74] x86, lto: Fix sys_call_table type in asm/syscall.h v2 Andi Kleen
2012-08-19  2:56 ` [PATCH 26/74] lto, sound: Fix export symbols for !CONFIG_MODULES Andi Kleen
2012-08-20  8:30   ` Takashi Iwai
2012-08-20  9:45     ` Andi Kleen
2012-08-20  9:53       ` Takashi Iwai
2012-08-19  2:56 ` [PATCH 27/74] lto: Mark EXPORT_SYMBOL symbols __visible Andi Kleen
2012-08-20  6:54   ` Rusty Russell
2012-08-20  9:49     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 28/74] lto: Make ksymtab and kcrctab symbols and __this_module __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 29/74] x86, lto: Make amd.c vide visible Andi Kleen
2012-08-19  2:56 ` [PATCH 30/74] x86, lto: Fix AMD K6 indirect call check Andi Kleen
2012-08-19  2:56 ` [PATCH 31/74] x86, lto: Make various variables used by assembler code __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 32/74] lto, PNP: Fix the inline assembler to use asmlinkage symbols Andi Kleen
2012-08-19  2:56 ` [PATCH 33/74] x86, lto, apm: Make APM data structure used from assembler visible Andi Kleen
2012-08-19  2:56 ` [PATCH 34/74] x86, lto, lguest: Fix C functions used by inline assembler Andi Kleen
2012-08-19  2:56 ` [PATCH 35/74] lto, crypto, aes: mark AES tables __visible Andi Kleen
2012-08-20  8:21   ` Herbert Xu
2012-08-19  2:56 ` [PATCH 36/74] lto, crypto, camelia: Make camelia tables used by assembler __visible Andi Kleen
2012-08-20  8:21   ` Herbert Xu
2012-08-19  2:56 ` [PATCH 37/74] lto, KVM: Don't assume asm statements end up in the same assembler file Andi Kleen
2012-08-19  8:59   ` Avi Kivity
2012-08-19 15:09     ` Andi Kleen
2012-08-19 15:12       ` Avi Kivity
2012-08-19 15:20         ` Andi Kleen
2012-08-19  2:56 ` [PATCH 38/74] lto, watchdog/hpwdt.c: Make assembler label global Andi Kleen
2012-08-22 19:25   ` Wim Van Sebroeck
2012-08-22 20:11     ` Mingarelli, Thomas
2012-08-19  2:56 ` [PATCH 39/74] x86, lto: Mark vdso variables __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 40/74] lto, powerpc: Disable LTO for the powerpc VDSO Andi Kleen
2012-08-19  2:56 ` [PATCH 41/74] x86, lto: Disable LTO for the x86 VDSO Andi Kleen
2012-08-19  2:56 ` [PATCH 42/74] lto, raid: disable LTO for the Altivec RAID code Andi Kleen
2012-08-19  2:56 ` [PATCH 43/74] lto, workaround: Disable LTO for sys_ni to work around alias bugs Andi Kleen
2012-08-19  2:56 ` [PATCH 44/74] lto: Mark functions used by the vsyscall init code visible Andi Kleen
2012-09-01 15:26   ` Michal Marek
2012-09-01 17:16     ` Andi Kleen
2012-09-01 17:26       ` Andi Kleen
2012-08-19  2:56 ` [PATCH 45/74] lto: Mark rwsem functions that can be called from assembler asmlinkage Andi Kleen
2012-08-19  2:56 ` [PATCH 46/74] x86, lto: Disable fancy hweight optimizations for LTO Andi Kleen
2012-08-19  8:28   ` Jan Beulich
2012-08-19 15:15     ` Andi Kleen
2012-08-20 10:57       ` Jan Beulich
2012-08-20 11:18         ` Andi Kleen
2012-08-20 12:38           ` Jan Beulich
2012-08-20  9:15   ` Avi Kivity
2012-08-20  9:42     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 47/74] x86, lto: Fix kprobes " Andi Kleen
2012-08-19  2:56 ` [PATCH 48/74] x86, lto: Use inline assembler instead of global register variable to get sp Andi Kleen
2012-08-19  8:37   ` Jan Beulich
2012-08-19 15:18     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 49/74] x86, lto, paravirt: Add __visible/asmlinkage to xen paravirt ops Andi Kleen
2012-08-19  2:56 ` [PATCH 50/74] x86, lto: Make empty_zero_page __visible for LTO Andi Kleen
2012-08-19  2:56 ` [PATCH 51/74] x86, lto, efi: Mark the efi variable used from assembler __visible Andi Kleen
2012-08-19  2:56 ` [PATCH 52/74] x86, lto, paravirt: Don't rely on local assembler labels Andi Kleen
2012-08-19  8:26   ` Jeremy Fitzhardinge
2012-08-19  2:56 ` [PATCH 53/74] x86, lto, paravirt: Make paravirt thunks global Andi Kleen
2012-08-19  8:27   ` Jeremy Fitzhardinge
2012-08-19 15:25     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 54/74] x86, lto, vdso: Don't duplicate vvar address variables Andi Kleen
2012-08-20 17:47   ` Andrew Lutomirski
2012-08-19  2:56 ` [PATCH 55/74] lto, workaround: Add workaround for initcall reordering Andi Kleen
2012-08-19  8:46   ` Jan Beulich
2012-08-19 15:01     ` Andi Kleen
2012-08-20 11:00       ` Jan Beulich
2012-08-19  2:56 ` [PATCH 56/74] lto, workaround: Add workaround for missing LTO symbols in igb Andi Kleen
2012-08-22  8:43   ` Arnd Bergmann
2012-08-22 12:36     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 57/74] lto, workaround: Add workaround for LTO build problem in pvrusb2-audio Andi Kleen
2012-08-19  2:56 ` [PATCH 58/74] lto, workaround: Work around LTO compiler problem in atheros driver Andi Kleen
2012-08-19  2:56 ` [PATCH 59/74] lto: Handle LTO common symbols in module loader Andi Kleen
2012-08-19  8:53   ` Jan Beulich
2012-08-19 15:23     ` Andi Kleen
2012-08-19  2:56 ` [PATCH 60/74] lto, Kbuild, bloat-o-meter: fix static detection Andi Kleen
2012-08-19  2:56 ` [PATCH 61/74] Kbuild, lto: Drop .number postfixes in modpost Andi Kleen
2012-08-19  2:56 ` [PATCH 62/74] Kbuild, lto: add ld-version and ld-ifversion macros Andi Kleen
2012-08-19  2:56 ` [PATCH 63/74] Kbuild, lto: Print correct info messages for vmlinux link Andi Kleen
2012-08-19  2:57 ` [PATCH 64/74] Kbuild, lto: Add a gcc-ld script to let run gcc as ld Andi Kleen
2012-08-19  2:57 ` [PATCH 65/74] Kbuild, lto: Disable LTO for asm-offsets.c Andi Kleen
2012-08-19  2:57 ` [PATCH 66/74] Kbuild, lto: Handle basic LTO in modpost Andi Kleen
2012-08-19  2:57 ` [PATCH 67/74] Kbuild, lto: Add Link Time Optimization support Andi Kleen
2012-08-19  2:57 ` Andi Kleen [this message]
2012-08-19  2:57 ` [PATCH 69/74] lto: Increase kallsyms max symbol length Andi Kleen
2012-08-19  2:57 ` [PATCH 70/74] Kbuild, lto: Handle longer symbols in kallsyms.c Andi Kleen
2012-08-19  2:57 ` [PATCH 71/74] lto, kprobes: Use KSYM_NAME_LEN to size identifier buffers Andi Kleen
2012-08-21  4:44   ` Ananth N Mavinakayanahalli
2012-08-19  2:57 ` [PATCH 72/74] lto: Mark spinlocks noinline when inline spinlocks are disabled Andi Kleen
2012-08-19  2:57 ` [PATCH 73/74] lto, module: Warn about modules that are not fully LTOed Andi Kleen
2012-08-19  2:57 ` [PATCH 74/74] lto, workaround: Mark do_futex noinline to prevent clobbering ebp Andi Kleen
2012-08-23  0:17   ` H. Peter Anvin
2012-08-23  2:14     ` H. Peter Anvin
2012-08-23  2:29     ` Andi Kleen
2012-08-23  3:14       ` H. Peter Anvin
2012-08-20  7:48 ` RFC: Link Time Optimization support for the kernel Ingo Molnar
2012-08-20 10:10   ` Andi Kleen
2012-08-21  7:49     ` Ingo Molnar
2012-08-21 14:05       ` Don Zickus
2012-08-21 14:26       ` Avi Kivity
2012-08-21 17:02       ` Andi Kleen
2012-08-23 15:02         ` Jan Hubicka
2012-08-22  8:58 ` Arnd Bergmann
2012-08-22 12:35   ` Andi Kleen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1345345030-22211-69-git-send-email-andi@firstfloor.org \
    --to=andi@firstfloor.org \
    --cc=JBeulich@suse.com \
    --cc=ak@linux.intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mmarek@suse.cz \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-KBuild Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kbuild/0 linux-kbuild/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kbuild linux-kbuild/ https://lore.kernel.org/linux-kbuild \
		linux-kbuild@vger.kernel.org
	public-inbox-index linux-kbuild

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kbuild


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git