Re: [PATCH] Cpuset: rcu optimization of page alloc hook

[Paul Jackson <pj@sgi.com>]

Eric Dumazet wrote:
> Please do use __read_mostly for new kmem_cache :

Paul Jackson wrote:
> Is there any downside to this?

Andrew Morton wrote:
> There's no downside, really.


Hmmm ... I suspect one possible downside.

I would think we would want to spread the hot spots out, to reduce the
chance of getting two hot spots in the same cache line, and starting a
bidding war for that line.

So my intuition is:
  If read alot but seldom written, mark "__read_mostly".
  If seldom read or written, leave unmarked.

so as to leave plenty of the rarely used (neither read nor written on
kernel hot path code) as "cannon fodder" to fill the rest of the cache
lines favored by the hot data.

This leads me to ask, of any item marked "__read_mostly":

  Is it accessed (for read, presumably) frequently, on a hot path?

If not, then I'd favor (absent actual measurements to the contrary) not
marking it.

By this criteria:

  1) I would -not- mark "struct kmem_cache *cpuset" __read_mostly, as it
     is rarely accessed on -any- code path, much less a hot one.  It is
     ideal cannon fodder.

  2) I -would- (following a private email suggestion of Christoph Lameter)
     mark my recently added "int number_of_cpusets" __read_mostly,
     because it is accessed for every zone considered in the loops
     within^Wbeneath __alloc_pages().

Disclaimer -- none of the above speculation is tempered by the heat of any
actual performance measurements.  Hence, it is worth about as much as my
legal advice.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401