Re: [ckrm-tech] Re: [PATCH 1/3] CPUMETER: add cpumeter framework to the CPUSETS

[Paul Jackson <pj@sgi.com>]

Takahiro-san, replying to pj:
> >                          |
> >                       CPUSET 1a
> >                       cpus: 2, 3
> >                       cpu_exclusive=0
> >                       meter_cpu=0
> >                       meter_cpu_*
> >                          |
> >             +------------+------------+
> >             |                         |
> >          CPUSET 2a                CPUSET 2b
> >          cpus: 2                  cpus: 3
> >          meter_cpu=0              meter_cpu=0
> >          cpu_exclusive=0          cpu_exclusive=0
> > 
> > Note here that marking C (CPUSET 1) as meter_cpu exposes the meter_cpu_*
> > files in the children of C.
> 
> If we split the cpus {2, 3} into {2} and {3} by creating CPUSET 2a 
> and CPUSET 2b, the guarantee assigned to CPUSET 1a might not be
> satisfied.  For example, the maximum cpu resource usage of tasks 
> in CPUSET 2a should essentially be 50% because tasks in CPUSET 2a
> can only use the half number of cpus. 

Ah, yes, this could be difficult and not worth doing.

It might help if I stated more of what I mean, which I didn't before.

I intended that all tasks in the combination of cpusets 1a, 2a, and 2b
would collectively be allowed what ever percentage of cpu cycles the
meter_cpu_* files in cpuset 1a prescribed.  I did not intend to suggest
any particular balance between these tasks in 1a, 2a and 2b would be
enforced.  In particular, I did not expect for anything like a 50%
split between the tasks in 2a and 2b to be enforced.   For the purposes
of your cpu controller, just treat the entire set of tasks in all
three of these cpusets as one pool, governed by one meter_cpu_*
setting, just as if all these tasks were in cpuset 1a, and as if
cpusets 2a and 2b didn't exist.

This might be easier to do - I don't know.  If this is still hard,
then I guess my dream of allowing metered cpusets to have child
cpusets with a "proper subset" of CPUs (strictly fewer CPUs) is too
hard to do now.  In the abstract, this seems like a serious limitation
to me.  But practically speaking, it might not be a big problem.
I don't know.

If it is still hard, even this way, it may just have to wait, until
someone needs it bad enough to find a way.  If we have layed a good
foundation and allowed for this possibility in our architecture,
then perhaps we will have done all we can do for now.

> Probably allowing nested metered cpusets is too hard both to implement
> and to use.  So far, I woundn't like to implement it.

A wise choice.

> This seems very interesting.  Tasks enclosed by a certain cpuset may
> want to change the behavior of oom_killer.

Yes - I think so too.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401