Re: [patch 00/2] improve .text size on gcc 4.0 and newer compilers

[Linus Torvalds <torvalds@osdl.org>]

On Fri, 6 Jan 2006, Ingo Molnar wrote:
> 
> i frequently validate branches in performance-critical kernel code like 
> the scheduler (and the mutex code ;), via instruction-granularity 
> profiling, driven by a high-frequency (10-100 KHz) NMI interrupt. A bad 
> branch layout shows up pretty clearly in annotated assembly listings:

Yes, but we only do this for routines that we look at anyway.

Also, the profiles can be misleading at times: you often get instructions 
with zero hits, because they always schedule together with another 
instruction. So parsing things and then matching them up (correctly) with 
the source code in order to annotate them is probably pretty nontrivial.

But starting with the code-paths that get literally zero profile events is 
definitely the way to go.

> Especially with 64 or 128 byte L1 cachelines our codepaths are really 
> fragmented and we can easily have 3-4 times of the optimal icache 
> footprint, for a given syscall. We very often have cruft in the hotpath, 
> and we often have functions that belong together ripped apart by things 
> like e.g. __sched annotators. I havent seen many cases of wrongly judged 
> likely/unlikely hints, what happens typically is that there's no 
> annotation and the default compiler guess is wrong.

We don't have likely()/unlikely() that often, and at least in my case it's 
partly because the syntax is a pain (it would probably have been better to 
include the "if ()" part in the syntax - the millions of parenthesis just 
drive me wild).

So yeah, we tend to put likely/unlikely only on really obvious stuff, and 
only on functions where we think about it. So we probably don't get it 
wrong that often.

> the dcache footprint of the kernel is much better, mostly because it's 
> so easy to control it in C. The icache footprint is alot more elusive.  
> (and also alot more critical to execution speed on nontrivial workloads)
> 
> so i think there are two major focus areas to improve our icache 
> footprint:
> 
>  - reduce code size
>  - reduce fragmentation of the codepath
> 
> fortunately both are hard and technically challenging projects

That's an interesting use of "fortunately". I tend to prefer the form 
where it means "fortunately, we can trivially fix this with a two-line 
solution that is obviously correct" ;)

		Linus