Re: [PATCH 0/5] SUBCPUSETS: a resource control functionality using CPUSETS

[Magnus Damm <magnus.damm@gmail.com>]

On 9/9/05, KUROSAWA Takahiro <kurosawa@valinux.co.jp> wrote:
> On Thu, 8 Sep 2005 05:02:32 -0700
> Paul Jackson <pj@sgi.com> wrote:
> > One of my passions is to avoid special cases across API boundaries.
> >
> > I am proposing that you don't do subcpusets like this.
> >
> > Consider the following alternative I will call 'cpuset meters'.
> >
> > For each resource named 'R' (cpu and mem, for instance):
> >  * Add a boolean flag 'meter_R' to each cpuset.  If set, that R is
> >    metered, for the tasks in that cpuset or any descendent cpuset.
> >  * If a cpuset is metered, then files named meter_R_guar, meter_R_lim
> >    and meter_R_cur appear in that cpuset to manage R's usage by tasks
> >    in that cpuset and descendents.
> >  * There are no additional rules that restrict the ability to change
> >    various other cpuset properties such as cpus, mems, cpu_exclusive,
> >    mem_exclusive, or notify_on_release, when a cpuset is metered.
> >  * It might be that some (or by design all) resource controllers do
> >    not allow nesting metered cpusets.  I don't know.  But one should
> >    (if one has permission) be able to make child cpusets of a metered
> >    cpuset, just like one can of any other cpuset.
> >  * A metered cpuset might well have cpus or mems that are not the
> >    same as its parent, just like an unmetered cpuset ordinarly does.
> 
> Jackson-san's idea looks good for me because users don't need
> to create special cpusets (parents of subcpusets or subcpusets).
> From the point of users, maybe they wouldn't like to create
> special cpusets.

Yes, from the user POV it must be good to keep the hierarchical model.
Ckrm and cpusets both provide a tree with descendents, children and
parents. This hierarchical model is very nice IMO and provides a
powerful API for the user.

> As for the resource controller that I've posted, it assumes
> that there are groups of tasks that share the same cpumasks/nodemasks,
> and that there are no hierarchy in that groups in order to make
> things easier.  I'll investigate how I can attach the resource
> controller to the cpuset meters.

Subcpusets, compared to cpusets and ckrm, gives the user a flat model.
No hierarchy. Limited functionality compared to the hierachical model.

But what I think is important to keep in mind here is that cpusets and
subcpusets do very different things. If I understand cpusets
correctly, each cpuset may share processors or memory nodes with other
cpusets. One task running on a shared processor may starve other
cpusets using the same processor. This design works well with cpusets,
but for resource controllers that must provide some kind of guarantee,
this starvation is unsuitable.

And we already have an hierarchical alternative: ckrm. But look at the
complexity and the amount of code. I believe that the complexity in
ckrm mainly comes from the hierarchical model.

Maybe it is possible to have an hierarchical model and keep the
framework simple and easy to understand while providing guarantees,
I'm not sure. But until that happens, I'm quite happy with a simple,
limited flat model.

/ magnus