Re: [PATCH] x86: Align jump targets to 1 byte boundaries

From: Ingo Molnar <mingo@kernel.org>
To: "H. Peter Anvin" <hpa@zytor.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Jason Low <jason.low2@hp.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Davidlohr Bueso <dave@stgolabs.net>,
	Tim Chen <tim.c.chen@linux.intel.com>,
	Aswin Chandramouleeswaran <aswin@hp.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Borislav Petkov <bp@alien8.de>,
	Andy Lutomirski <luto@amacapital.net>,
	Brian Gerst <brgerst@gmail.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: [PATCH] x86: Align jump targets to 1 byte boundaries
Date: Fri, 10 Apr 2015 19:54:07 +0200	[thread overview]
Message-ID: <20150410175407.GB6563@gmail.com> (raw)
In-Reply-To: <5527CD92.1080901@zytor.com>

* H. Peter Anvin <hpa@zytor.com> wrote:

> On 04/10/2015 05:50 AM, Denys Vlasenko wrote:
> > 
> > However, I'm an -Os guy. Expect -O2 people to disagree :)
> > 
> 
> The problem with -Os is that the compiler will make *any* tradeoffs 
> to save a byte.  It is really designed to squeeze as much code into 
> a fixed-size chunk, e.g. a ROM, as possible.
> 
> We have asked for an -Okernel mode from the gcc folks forever.  It 
> basically would mean "-Os except when really dumb."

Yes, and it appears that not aligning to 16 bytes gives 5.5% size 
savings already - which is a big chunk of the -Os win.

So we might be able to get a "poor man's -Okernel" by not aligning. 
(I'm also looking at GCC options to make loop unrolls less aggressive 
- that's another common source of bloat.)

I strongly suspect it's the silly 'use weird, wildly data-dependent 
instructions just to save a single byte' games are that are killing 
-Os performance in practice.

> As far as the 16-byte alignment, my understanding is not that it is 
> related to the I$ but rather is the decoder datum.

Yeah, but the decoder stops if the prefetch crosses a cache line? So 
it appears to be an interaction of the 16 byte prefetch window and 
cache line boundaries?

Btw., given that much of a real life kernel's instructions execute 
cache-cold, a 5.5% reduction in kernel size could easily speed up 
cache-cold execution by a couple of percent. In the cache-cold case 
the prefetch window size is probably not important at all, what 
determines execution speed is cache miss latency and cache footprint.

[ At least in my simple mental picture of it, which might be wrong ;-) ]

Thanks,

	Ingo