Re: Scaling noise

["Martin J. Bligh" <mbligh@aracnet.com>]

> as CPU speeds climb and the relative penalty for going off the chip for
> any reason climb even faster the hardware overhead of keeping a SMP
> machine coherant becomes more significant. sometimes this can be solved by
> changing the algorithm (the O(1) scheduler for example) but other times
> you just have to accept the costof bouncing a cache line from one CPU to
> another so that you can aquire a lock.
> 
> the advantage of multiple images is that the hardware only needs to
> maintain full speed consistancy between CPU's in one image, a higher level
> (the OS or the partitioning software) can deal with the issues of letting
> one image know what it needs to about what the others are doing.

All true, but multiple images isn't the only way to solve the problem
(I happen to like the idea, but let's go for full disclosure on all
possibilities ;-)). Take one example, the global pagemap_lru_lock in
2.4 ... either we could split the LRU by multiple OS images, or explicitly
split it per-node ... like we did in 2.6 (Andrew did that particular bit
of work IIRC, apologies if not).

The question is whether it's better to split *everything* to start with,
and glue back the global bits ... or start with everything global, and
split off the problematic bits explicitly. To date, we've been taking
the latter approach ... it's probably more pragmatic and evolutionary,
and is certainly a damned sight easier. The former approach is a fascinating
idea (to Larry and I at least, it seems ... now we've stepped over our
previous communication breakdown on one aspect at least), but unproven.
I wanna go play with it ;-)

M.