Re: Scaling noise

[Peter Chubb <peter@chubb.wattle.id.au>]

>>>>> "bill" == bill davidsen <davidsen@tmr.com> writes:

> In article <20030903153901.GB5769@work.bitmover.com>, Larry
> McVoy <lm@bitmover.com> wrote:

Larry> It's really easy to claim that scalability isn't the problem.
Larry> Scaling changes in general cause very minute differences, it's
Larry> just that there are a lot of them.  There is constant pressure
Larry> to scale further and people think it's cool.  You can argue
Larry> you all you want that scaling done right isn't a problem but
Larry> nobody has ever managed to do it right.  I know it's 
Larry> politically incorrect to say this group won't either but there
Larry> is no evidence that they will.

bill> I think that if the problem of a single scheduler which is
bill> "best" at everything proves out of reach, perhaps in 2.7 a
bill> modular scheduler will appear, which will allow the user to
bill> select the Nick+Con+Ingo responsiveness, or the default pretty
bill> good at everything, or the 4kbit affinity mask NUMA on steroids
bill> solution.

Well, as I see it it's not processor but memory scalability that's the
problem right now.  Memories are getting larger (and for NUMA systems,
sparser), and the current linux solutions don't scale particularly
well --- particularly when, for architectures like PPC or IA64, you
need two copies in different formats, one for the hardware to look up,
and one for the OS.

I *do* think that pluggable schedulers are a good idea --- I'd like to
introduce something like the scheduler class mechanism that SVr4 has
(except that I've seen that code, and don't want to get sued by SCO)
to allow different processes to be in different classes in a cleaner
manner than the current FIFO or RR vs OTHER classes.  We should be
able to introduce isochronous, gang, lottery or fairshare schedulers
(etc) at runtime, and then tie processes severally and indivdually to
those schedulers, with a well defined idea of what happens when
scheduler priorities overlap, and well defined APIs to adjust
scheduler parameters.  However, this will require more major
infrastructure changes, and a better separation of dispatcher from
scheduler than in the current one-size-fits-all scheduler.


--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories,   all slightly different.