Question about group scheduler cpu shares

Question about group scheduler cpu shares

[Gaurav Poothia]

Hi,
The cgroup "cpu" subsystem's weight calculation using "cpu shares" is
fairly clear when all tasks are attached to the leaf cgroups in "cpu"
subsystem.
For example in this cgroup hierarchy with tasks A.B,C and D and
associated cpu shares:

ROOT
|
+ -Group1(3072)
|    |
|   +- A(2048)
|    |
|    +- B(1024)
|
+- Group2(2048)
     |
     +-C(1024)
     |
     +-D(1024)

Group 1 and Group 2 split the CPU 60% to 40% respectively
A and B split Group1's 60% share 2/3rd  to 1/3rd so absolute share of
CPU will be 40% and 20% respectively
C and D split Group2's 40% share evenly so absolute shares or 20% each
So far so good. But in this example all tasks are associated with leaf cgroups.

Say I add a task E to Group1's task list (note that is an interior aka
non-leaf node)
How does the CPU split change between A, B and E.
AFAICT there is no cgroup cpu subsystem knob to weight tasks on an
interior node against the tasks in that node's children

Thanks in advance!
Gaurav

Re: Question about group scheduler cpu shares

[Gaurav Poothia]

+Ingo, Peter


On Thu, Jul 21, 2016 at 12:03 PM, Gaurav Poothia
<gaurav.poothia@gmail.com> wrote:
> Hi,
> The cgroup "cpu" subsystem's weight calculation using "cpu shares" is
> fairly clear when all tasks are attached to the leaf cgroups in "cpu"
> subsystem.
> For example in this cgroup hierarchy with tasks A.B,C and D and
> associated cpu shares:
>
> ROOT
> |
> + -Group1(3072)
> |    |
> |   +- A(2048)
> |    |
> |    +- B(1024)
> |
> +- Group2(2048)
>      |
>      +-C(1024)
>      |
>      +-D(1024)
>
> Group 1 and Group 2 split the CPU 60% to 40% respectively
> A and B split Group1's 60% share 2/3rd  to 1/3rd so absolute share of
> CPU will be 40% and 20% respectively
> C and D split Group2's 40% share evenly so absolute shares or 20% each
> So far so good. But in this example all tasks are associated with leaf cgroups.
>
> Say I add a task E to Group1's task list (note that is an interior aka
> non-leaf node)
> How does the CPU split change between A, B and E.
> AFAICT there is no cgroup cpu subsystem knob to weight tasks on an
> interior node against the tasks in that node's children
>
> Thanks in advance!
> Gaurav



-- 
Kiva.org - Loans That Change Lives

Re: Question about group scheduler cpu shares

[Mike Galbraith]

On Thu, 2016-07-21 at 18:18 -0700, Gaurav Poothia wrote:

> > ROOT
> > > 
> > + -Group1(3072)
> > >    |
> > >   +- A(2048)
> > >    |
> > >    +- B(1024)
> > > 
> > +- Group2(2048)
> >      |
> >      +-C(1024)
> >      |
> >      +-D(1024)
> > 

> > Say I add a task E to Group1's task list (note that is an interior aka
> > non-leaf node)
> > How does the CPU split change between A, B and E.
> > AFAICT there is no cgroup cpu subsystem knob to weight tasks on an
> > interior node against the tasks in that node's children

A, B and E are all entities with a weight, so just plug E into your
graph.  Its weight is determined by nice level, which is what cgroups
should have done instead of inventing shares IMHO.  1024 == nice(0).

	-Mike

Re: Question about group scheduler cpu shares

[Gaurav Poothia]

Thanks Mike.
So if I understand you right - tasks on interior nodes get their
weight from task's nice level and not from some cpu.share setting.
Two followups:
1.What is the function that translates from various nice levels to
weight i.e. nice(0) == 1024 how to translate for other levels
2.How does that work when an interior node has multiple tasks? So if I
added tasks E and F with difference nice levels to Group 1 task list
how would the math work?

Appreciate the help -thanks!

On Thu, Jul 21, 2016 at 7:51 PM, Mike Galbraith
<umgwanakikbuti@gmail.com> wrote:
> On Thu, 2016-07-21 at 18:18 -0700, Gaurav Poothia wrote:
>
>> > ROOT
>> > >
>> > + -Group1(3072)
>> > >    |
>> > >   +- A(2048)
>> > >    |
>> > >    +- B(1024)
>> > >
>> > +- Group2(2048)
>> >      |
>> >      +-C(1024)
>> >      |
>> >      +-D(1024)
>> >
>
>> > Say I add a task E to Group1's task list (note that is an interior aka
>> > non-leaf node)
>> > How does the CPU split change between A, B and E.
>> > AFAICT there is no cgroup cpu subsystem knob to weight tasks on an
>> > interior node against the tasks in that node's children
>
> A, B and E are all entities with a weight, so just plug E into your
> graph.  Its weight is determined by nice level, which is what cgroups
> should have done instead of inventing shares IMHO.  1024 == nice(0).
>
>         -Mike



-- 
Kiva.org - Loans That Change Lives

Re: Question about group scheduler cpu shares

[Mike Galbraith]

On Thu, 2016-07-21 at 20:24 -0700, Gaurav Poothia wrote:

(top posting.. naught naughty;)

> 1.What is the function that translates from various nice levels to
> weight i.e. nice(0) == 1024 how to translate for other levels

If you rummage around in kernel/sched, you'll find this table:

const int sched_prio_to_weight[40] = {
 /* -20 */     88761,     71755,     56483,     46273,     36291,
 /* -15 */     29154,     23254,     18705,     14949,     11916,
 /* -10 */      9548,      7620,      6100,      4904,      3906,
 /*  -5 */      3121,      2501,      1991,      1586,      1277,
 /*   0 */      1024,       820,       655,       526,       423,
 /*   5 */       335,       272,       215,       172,       137,
 /*  10 */       110,        87,        70,        56,        45,
 /*  15 */        36,        29,        23,        18,        15,
};

> 2.How does that work when an interior node has multiple tasks? So if I
> added tasks E and F with difference nice levels to Group 1 task list
> how would the math work?

The math works the same as if you were adding task groups.  An entity
is an entity is an entity...

	-Mike