Re: [PATCH UPDATED] percpu: use dynamic percpu allocator as the default percpu allocator

From: Linus Torvalds <torvalds@linux-foundation.org>
To: Ingo Molnar <mingo@elte.hu>
Cc: Christoph Lameter <cl@linux.com>, Tejun Heo <tj@kernel.org>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	rusty@rustcorp.com.au, tglx@linutronix.de, x86@kernel.org,
	linux-kernel@vger.kernel.org, hpa@zytor.com,
	Paul Mundt <lethal@linux-sh.org>,
	rmk@arm.linux.org.uk, starvik@axis.com, ralf@linux-mips.org,
	davem@davemloft.net, cooloney@kernel.org, kyle@mcmartin.ca,
	matthew@wil.cx, grundler@parisc-linux.org, takata@linux-m32r.org,
	benh@kernel.crashing.org, rth@twiddle.net,
	ink@jurassic.park.msu.ru, heiko.carstens@de.ibm.com,
	Nick Piggin <npiggin@suse.de>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: [PATCH UPDATED] percpu: use dynamic percpu allocator as the default percpu allocator
Date: Wed, 1 Apr 2009 12:39:46 -0700 (PDT)	[thread overview]
Message-ID: <alpine.LFD.2.00.0904011235160.4130@localhost.localdomain> (raw)
In-Reply-To: <20090401190113.GA734@elte.hu>

On Wed, 1 Apr 2009, Ingo Molnar wrote:
>  
> The rest (unannotated variables) is to be assumed "access-rarely" or 
> "we-dont-care", by default. This is actually 95% of the global 
> variables.

I'm not at all convinced that this is a good option.

The thing is, things like "read_mostly" or "access_rarely" may talk about 
how we access those individual variables, but you're missing a _huge_ 
chunk of the puzzle if you ignore the _correlations_ of those accesses 
with accesses to other variables.

The thign is, if you have variables 'a' and 'b', and they are always 
accessed together, then it's probably worth it to put them in the same 
cacheline.

And that is true EVEN IF 'b' is basically read-only, but 'a' is always 
written to. Marking 'a' as read-only or read-mostly is actually a bad 
idea, since the only thing it can ever result in is a bigger cache 
footprint.

The fact that the read-only cacheline has "nicer" behavior is totally 
irrelevant - it's still an extra cacheline that we're accessing. And they 
aren't free, even if read-only cachelines that can be shared across CPU's 
are certainly a lot cheaper than ones that bounce.

			Linus