Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Peter Williams wrote:
>
> This version removes the hard/soft CPU rate caps from the SPA schedulers.
>
> A patch for 2.6.18-rc2 is available at:
>
> <http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.pat
>ch?download>
>
> Very Brief Documentation:
>
> You can select a default scheduler at kernel build time.  If you wish to
> boot with a scheduler other than the default it can be selected at boot
> time by adding:
>
> cpusched=<scheduler>

Any reason dynsched couldn't be merged with plugsched?

> to the boot command line where <scheduler> is one of: ingosched,
> ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs
> or zaphod.  If you don't change the default when you build the kernel
> the default scheduler will be ingosched (which is the normal scheduler).
>
> The scheduler in force on a running system can be determined by the
> contents of:
>
> /proc/scheduler

It may be really great, to allow schedulers perPid parent, thus allowing the 
stacking of different scheduler semantics.  This could aid flexibility a 
lot.

Worth a try, and should be easy to implement.

> Control parameters for the scheduler can be read/set via files in:
>
> /sys/cpusched/<scheduler>/

Thanks for the most important out-of-tree patch that makes 2.6 reasonable.

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Peter Williams]

Al Boldi wrote:
> Peter Williams wrote:
>> This version removes the hard/soft CPU rate caps from the SPA schedulers.
>>
>> A patch for 2.6.18-rc2 is available at:
>>
>> <http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.pat
>> ch?download>
>>
>> Very Brief Documentation:
>>
>> You can select a default scheduler at kernel build time.  If you wish to
>> boot with a scheduler other than the default it can be selected at boot
>> time by adding:
>>
>> cpusched=<scheduler>
> 
> Any reason dynsched couldn't be merged with plugsched?

None that I know of (but I'm not familiar with dynsched).  Patches to 
add it to the mix would be accepted and once in I would try to keep it 
in step with kernel changes.

> 
>> to the boot command line where <scheduler> is one of: ingosched,
>> ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs
>> or zaphod.  If you don't change the default when you build the kernel
>> the default scheduler will be ingosched (which is the normal scheduler).
>>
>> The scheduler in force on a running system can be determined by the
>> contents of:
>>
>> /proc/scheduler
> 
> It may be really great, to allow schedulers perPid parent, thus allowing the 
> stacking of different scheduler semantics.  This could aid flexibility a 
> lot.

I'm don't understand what you mean here.  Could you elaborate?

> 
> Worth a try, and should be easy to implement.
> 
>> Control parameters for the scheduler can be read/set via files in:
>>
>> /sys/cpusched/<scheduler>/
> 
> Thanks for the most important out-of-tree patch that makes 2.6 reasonable.

My pleasure,
Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Peter Williams wrote:
> Al Boldi wrote:
> > Peter Williams wrote:
> >> This version removes the hard/soft CPU rate caps from the SPA
> >> schedulers.
> >>
> >> A patch for 2.6.18-rc2 is available at:
> >>
> >> <http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.
> >>pat ch?download>
> >>
> >> Very Brief Documentation:
> >>
> >> You can select a default scheduler at kernel build time.  If you wish
> >> to boot with a scheduler other than the default it can be selected at
> >> boot time by adding:
> >>
> >> cpusched=<scheduler>
> >
> > Any reason dynsched couldn't be merged with plugsched?
>
> None that I know of (but I'm not familiar with dynsched).  Patches to
> add it to the mix would be accepted and once in I would try to keep it
> in step with kernel changes.

I thought dynsched patches against plugsched, what else is needed?

> >> to the boot command line where <scheduler> is one of: ingosched,
> >> ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs
> >> or zaphod.  If you don't change the default when you build the kernel
> >> the default scheduler will be ingosched (which is the normal
> >> scheduler).
> >>
> >> The scheduler in force on a running system can be determined by the
> >> contents of:
> >>
> >> /proc/scheduler
> >
> > It may be really great, to allow schedulers perPid parent, thus allowing
> > the stacking of different scheduler semantics.  This could aid
> > flexibility a lot.
>
> I'm don't understand what you mean here.  Could you elaborate?

i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.


Thanks!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Peter Williams]

Al Boldi wrote:
> Peter Williams wrote:
>> Al Boldi wrote:
>>> Peter Williams wrote:
>>>> This version removes the hard/soft CPU rate caps from the SPA
>>>> schedulers.
>>>>
>>>> A patch for 2.6.18-rc2 is available at:
>>>>
>>>> <http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.
>>>> pat ch?download>
>>>>
>>>> Very Brief Documentation:
>>>>
>>>> You can select a default scheduler at kernel build time.  If you wish
>>>> to boot with a scheduler other than the default it can be selected at
>>>> boot time by adding:
>>>>
>>>> cpusched=<scheduler>
>>> Any reason dynsched couldn't be merged with plugsched?
>> None that I know of (but I'm not familiar with dynsched).  Patches to
>> add it to the mix would be accepted and once in I would try to keep it
>> in step with kernel changes.
> 
> I thought dynsched patches against plugsched, what else is needed?
>

Hopefully, nothing but it may be necessary to modify the plugsched 
interface if dynsched can't be implemented against it "as is".  E.g. 
both staircase and nicksched needed changes to what was required for 
ingosched and the SPA schedulers.

>>>> to the boot command line where <scheduler> is one of: ingosched,
>>>> ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs
>>>> or zaphod.  If you don't change the default when you build the kernel
>>>> the default scheduler will be ingosched (which is the normal
>>>> scheduler).
>>>>
>>>> The scheduler in force on a running system can be determined by the
>>>> contents of:
>>>>
>>>> /proc/scheduler
>>> It may be really great, to allow schedulers perPid parent, thus allowing
>>> the stacking of different scheduler semantics.  This could aid
>>> flexibility a lot.
>> I'm don't understand what you mean here.  Could you elaborate?
> 
> i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.

It's probably not a good idea to have different schedulers managing the 
same resource.  The way to do different scheduling per process is to use 
the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc. 
(possibly extended) within each scheduler.  On the other hand, on an SMP 
system, having a different scheduler on each run queue (or sub set of 
queues) might be interesting :-).  The schedulers would probably have to 
have a common idea of how the run queue works though and this would 
restrict the choice of schedulers.

I have no intentions (at the moment) of going down this path myself.

However, I am thinking about making it possible to switch between the 
various SPA schedulers on a running system.  A extension to this could 
be to attempt automatic selection of which scheduler to use possibly 
based on which users are logged in.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Peter Williams wrote:
> Al Boldi wrote:
> > Peter Williams wrote:
> >> Al Boldi wrote:
> >>> Peter Williams wrote:
> >>>> This version removes the hard/soft CPU rate caps from the SPA
> >>>> schedulers.
> >>>>
> >>>> A patch for 2.6.18-rc2 is available at:
> >>>>
> >>>> <http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc
> >>>>2. pat ch?download>
> >>>>
> >>>> Very Brief Documentation:
> >>>>
> >>>> You can select a default scheduler at kernel build time.  If you wish
> >>>> to boot with a scheduler other than the default it can be selected at
> >>>> boot time by adding:
> >>>>
> >>>> cpusched=<scheduler>
> >>>
> >>> Any reason dynsched couldn't be merged with plugsched?
> >>
> >> None that I know of (but I'm not familiar with dynsched).  Patches to
> >> add it to the mix would be accepted and once in I would try to keep it
> >> in step with kernel changes.
> >
> > I thought dynsched patches against plugsched, what else is needed?
>
> Hopefully, nothing but it may be necessary to modify the plugsched
> interface if dynsched can't be implemented against it "as is".  E.g.
> both staircase and nicksched needed changes to what was required for
> ingosched and the SPA schedulers.
>
> >>>> to the boot command line where <scheduler> is one of: ingosched,
> >>>> ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr,
> >>>> spa_ebs or zaphod.  If you don't change the default when you build
> >>>> the kernel the default scheduler will be ingosched (which is the
> >>>> normal scheduler).
> >>>>
> >>>> The scheduler in force on a running system can be determined by the
> >>>> contents of:
> >>>>
> >>>> /proc/scheduler
> >>>
> >>> It may be really great, to allow schedulers perPid parent, thus
> >>> allowing the stacking of different scheduler semantics.  This could
> >>> aid flexibility a lot.
> >>
> >> I'm don't understand what you mean here.  Could you elaborate?
> >
> > i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.
>
> It's probably not a good idea to have different schedulers managing the
> same resource.  The way to do different scheduling per process is to use
> the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
> (possibly extended) within each scheduler.  On the other hand, on an SMP
> system, having a different scheduler on each run queue (or sub set of
> queues) might be interesting :-).  

What's wrong with multiple run-queues on UP?

> The schedulers would probably have to
> have a common idea of how the run queue works though and this would
> restrict the choice of schedulers.

Probably.

> I have no intentions (at the moment) of going down this path myself.
>
> However, I am thinking about making it possible to switch between the
> various SPA schedulers on a running system.  A extension to this could
> be to attempt automatic selection of which scheduler to use possibly
> based on which users are logged in.

Thanks a lot!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

On Tue, 25 Jul 2006 21:27:14 +0300, Al Boldi said:
> Peter Williams wrote:

> > It's probably not a good idea to have different schedulers managing the
> > same resource.  The way to do different scheduling per process is to use
> > the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
> > (possibly extended) within each scheduler.  On the other hand, on an SMP
> > system, having a different scheduler on each run queue (or sub set of
> > queues) might be interesting :-).  
> 
> What's wrong with multiple run-queues on UP?

On an SMP system, you can have one CPU doing one class of scheduling (long
timeslice for computational, for example), while another CPU is dedicated
to doing RT scheduling, and so on.  It's not clear to me that "different
classes per CPU" makes any real sense on a UP....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Peter Williams]

Al Boldi wrote:
> Peter Williams wrote:
>> Al Boldi wrote:
>>> Peter Williams wrote:
>>>> Al Boldi wrote:
[bits deleted]
>>>>> It may be really great, to allow schedulers perPid parent, thus
>>>>> allowing the stacking of different scheduler semantics.  This could
>>>>> aid flexibility a lot.
>>>> I'm don't understand what you mean here.  Could you elaborate?
>>> i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.
>> It's probably not a good idea to have different schedulers managing the
>> same resource.  The way to do different scheduling per process is to use
>> the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
>> (possibly extended) within each scheduler.  On the other hand, on an SMP
>> system, having a different scheduler on each run queue (or sub set of
>> queues) might be interesting :-).  
> 
> What's wrong with multiple run-queues on UP?

A really high likelihood of starvation of some tasks.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Valdis.Kletnieks@vt.edu wrote:
> On Tue, 25 Jul 2006 21:27:14 +0300, Al Boldi said:
> > Peter Williams wrote:
> > > It's probably not a good idea to have different schedulers managing
> > > the same resource.  The way to do different scheduling per process is
> > > to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
> > > (possibly extended) within each scheduler.  On the other hand, on an
> > > SMP system, having a different scheduler on each run queue (or sub set
> > > of queues) might be interesting :-).
> >
> > What's wrong with multiple run-queues on UP?
>
> On an SMP system, you can have one CPU doing one class of scheduling (long
> timeslice for computational, for example), while another CPU is dedicated
> to doing RT scheduling, and so on.  It's not clear to me that "different
> classes per CPU" makes any real sense on a UP....

Conceptually there should be no difference between UP and MP.

Think HyperThreading.


Thanks!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Peter Williams wrote:
> Al Boldi wrote:
> > Peter Williams wrote:
> >> Al Boldi wrote:
> >>> Peter Williams wrote:
> >>>> Al Boldi wrote:
>
> [bits deleted]
>
> >>>>> It may be really great, to allow schedulers perPid parent, thus
> >>>>> allowing the stacking of different scheduler semantics.  This could
> >>>>> aid flexibility a lot.
> >>>>
> >>>> I'm don't understand what you mean here.  Could you elaborate?
> >>>
> >>> i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.
> >>
> >> It's probably not a good idea to have different schedulers managing the
> >> same resource.  The way to do different scheduling per process is to
> >> use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
> >> (possibly extended) within each scheduler.  On the other hand, on an
> >> SMP system, having a different scheduler on each run queue (or sub set
> >> of queues) might be interesting :-).
> >
> > What's wrong with multiple run-queues on UP?
>
> A really high likelihood of starvation of some tasks.

Maybe you are thinking of running independent run-queues, in which case it 
would probably be unwise to run multiple RQs on a single CPU.

But I was more thinking of a run-queue of run-queues, with the masterRQ 
scheduling slaveRQs, each RQ possible running its own scheduling semantic.


Thanks!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Peter Williams]

Al Boldi wrote:
> Peter Williams wrote:
>> Al Boldi wrote:
>>> Peter Williams wrote:
>>>> Al Boldi wrote:
>>>>> Peter Williams wrote:
>>>>>> Al Boldi wrote:
>> [bits deleted]
>>
>>>>>>> It may be really great, to allow schedulers perPid parent, thus
>>>>>>> allowing the stacking of different scheduler semantics.  This could
>>>>>>> aid flexibility a lot.
>>>>>> I'm don't understand what you mean here.  Could you elaborate?
>>>>> i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.
>>>> It's probably not a good idea to have different schedulers managing the
>>>> same resource.  The way to do different scheduling per process is to
>>>> use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR, etc.
>>>> (possibly extended) within each scheduler.  On the other hand, on an
>>>> SMP system, having a different scheduler on each run queue (or sub set
>>>> of queues) might be interesting :-).
>>> What's wrong with multiple run-queues on UP?
>> A really high likelihood of starvation of some tasks.
> 
> Maybe you are thinking of running independent run-queues, in which case it 
> would probably be unwise to run multiple RQs on a single CPU.

No.  I'm thinking about different schedulers on a single run queue.  I 
don't think that it's a good idea.

> 
> But I was more thinking of a run-queue of run-queues, with the masterRQ 
> scheduling slaveRQs, each RQ possible running its own scheduling semantic.

I think that you need to think a bit harder about the consequences of 
such a system.  The word "chaos" springs to mind.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Peter Williams wrote:
> Al Boldi wrote:
> >>>>>>> It may be really great, to allow schedulers perPid parent, thus
> >>>>>>> allowing the stacking of different scheduler semantics.  This
> >>>>>>> could aid flexibility a lot.
> >>>>>>
> >>>>>> I'm don't understand what you mean here.  Could you elaborate?
> >>>>>
> >>>>> i.e:  Boot the kernel with spa_no_frills, then start X with spa_ws.
> >>>>
> >>>> It's probably not a good idea to have different schedulers managing
> >>>> the same resource.  The way to do different scheduling per process is
> >>>> to use the scheduling policy mechanism i.e. SCHED_FIFO, SCHED_RR,
> >>>> etc. (possibly extended) within each scheduler.  On the other hand,
> >>>> on an SMP system, having a different scheduler on each run queue (or
> >>>> sub set of queues) might be interesting :-).
> >>>
> >>> What's wrong with multiple run-queues on UP?
> >>
> >> A really high likelihood of starvation of some tasks.
> >
> > Maybe you are thinking of running independent run-queues, in which case
> > it would probably be unwise to run multiple RQs on a single CPU.
>
> No.  I'm thinking about different schedulers on a single run queue.  I
> don't think that it's a good idea.

Running different scheds on a single RQ at the same time on the same resource 
would be rather odd.  That's why independent RQs are necessary even on SMP.  
OTOH, running independent RQs on UP doesn't make much sense, unless there is 
a way to relate them.

> > But I was more thinking of a run-queue of run-queues, with the masterRQ
> > scheduling slaveRQs, each RQ possibly running its own scheduling
> > semantic.
>
> I think that you need to think a bit harder about the consequences of
> such a system.  The word "chaos" springs to mind.

Are you sure?

MultiDimensional RunQueues spring to mind.


Thanks!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

On Wed, 26 Jul 2006 07:45:33 +0300, Al Boldi said:
> Valdis.Kletnieks@vt.edu wrote:

> > On an SMP system, you can have one CPU doing one class of scheduling (long
> > timeslice for computational, for example), while another CPU is dedicated
> > to doing RT scheduling, and so on.  It's not clear to me that "different
> > classes per CPU" makes any real sense on a UP....
> 
> Conceptually there should be no difference between UP and MP.
> 
> Think HyperThreading.

Which is why a UP kernel can schedule on both sides of an HT core.

Yeah, I got it now. ;)

An HT core still *has* "the other instruction stream" it can schedule
differetly.  You can't say "We'll schedule this one this way and that other
one that way" when there *is* no "that other one".

(And if you look at the current code, you'll realize that HT is conceptually
different from both UP *and* MP - go look at the places where the *current*
scheduler is HT-aware, and how that was a big win over when it thought each
HT was a fully capable MP......)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

On Wed, 26 Jul 2006 14:23:03 +0300, Al Boldi said:

> Running different scheds on a single RQ at the same time on the same resource
> would be rather odd.  That's why independent RQs are necessary even on SMP.
> OTOH, running independent RQs on UP doesn't make much sense, unless there is
> a way to relate them.

Exactly..

(But now I'm confused why you said SMP and UP were conceptually the same a
few notes back...)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Al Boldi]

Valdis.Kletnieks@vt.edu wrote:
> On Wed, 26 Jul 2006 14:23:03 +0300, Al Boldi said:
> > Running different scheds on a single RQ at the same time on the same
> > resource would be rather odd.  That's why independent RQs are necessary
> > even on SMP. OTOH, running independent RQs on UP doesn't make much
> > sense, unless there is a way to relate them.
>
> Exactly..
>
> (But now I'm confused why you said SMP and UP were conceptually the same a
> few notes back...)

The important part here is 'unless there is a way to relate them', at which 
point UP and MP should be conceptually the same, while possibly differing in 
the implementation details.

Thanks!

--
Al

Re: [ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

On Wed, 26 Jul 2006 17:04:35 +0300, Al Boldi said:

> The important part here is 'unless there is a way to relate them', at which 
> point UP and MP should be conceptually the same, while possibly differing in 
> the implementation details.

Oh, OK. I think we're actually in agreement, just using different terms for
the same thing - what you were considering multiple related queues, I'd
consider one unified queue with subqueuing of some sort.. ;)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

[ANNOUNCE][RFC] PlugSched-6.4 for 2.6.18-rc2

[Peter Williams]

This version removes the hard/soft CPU rate caps from the SPA schedulers.

A patch for 2.6.18-rc2 is available at:

<http://prdownloads.sourceforge.net/cpuse/plugsched-6.4-for-2.6.18-rc2.patch?download>

Very Brief Documentation:

You can select a default scheduler at kernel build time.  If you wish to
boot with a scheduler other than the default it can be selected at boot
time by adding:

cpusched=<scheduler>

to the boot command line where <scheduler> is one of: ingosched,
ingo_ll, nicksched, staircase, spa_no_frills, spa_ws, spa_svr, spa_ebs
or zaphod.  If you don't change the default when you build the kernel
the default scheduler will be ingosched (which is the normal scheduler).

The scheduler in force on a running system can be determined by the
contents of:

/proc/scheduler

Control parameters for the scheduler can be read/set via files in:

/sys/cpusched/<scheduler>/

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce