Understanding cgroups

Understanding cgroups

[Tommaso Cucinotta]

Hi,

I'm trying to get some understanding of the current cgroups in-kernel 
implementation
(after having read Documentation/cgroup* and having browsed a bit the code).
To this purpose, I tried to draw the relationships among the involved 
data structures
(I'm limited to its relationship with [real-time] scheduling), and 
obtained this:

   http://retis.sssup.it/~tommaso/cgroups.odg
   http://retis.sssup.it/~tommaso/cgroups.eps

(You can see in the bottom left part of the diagram a little 
"key/legend/pattern" for representing lists).

I might have done mistakes, however the greatest doubts that I have now 
concern the relative
cardinalities of the various associated items. Namely:
a) why doesn't a cgroup object directly point to a css_set one, but to a 
list of them (via cg_cgroup_list elements) ?
     it seems that a cgroup object may be associated to multiple css_set 
objects, which in turn contain vectors of
     cgroup_subsys_state;
b) however, cgroup.subsys[] would point to a single cgroup_subsys_state 
object per subsys_id, so, what is the
     difference between cgroup.subsys[] and css_set.subsys[] ? (or, are 
these all redundant pointers and point
     to the same cgroup_subsys_state objects ?)
c) is css_set.cg_links used to point to (the head of) a list of 
cg_cgroup_link objects, or is it used to link multiple css_set objects 
into a list ? In the latter case, where is the head of the list pointed 
to from ?

Apologies for the newbie questions that I might have posted; FYI, I'm 
trying to set-up RT scheduling groups without using the VFS-based 
cgroups interface.

Thanks in advance, regards (please, reply in cc to my e-mail address).

     Tommaso

-- 
Tommaso Cucinotta, Computer Engineering PhD, Researcher
ReTiS Lab, Scuola Superiore Sant'Anna, Pisa, Italy
Tel +39 050 882 024, Fax +39 050 882 003
http://retis.sssup.it/people/tommaso

Re: Understanding cgroups

[Dhaval Giani]

[Adding a few more people and the containers to the copy]

On Thu, Oct 28, 2010 at 5:44 PM, Tommaso Cucinotta
<tommaso.cucinotta-gAmJrWFzCps@public.gmane.org> wrote:
> Hi,
>
> I'm trying to get some understanding of the current cgroups in-kernel
> implementation
> (after having read Documentation/cgroup* and having browsed a bit the code).
> To this purpose, I tried to draw the relationships among the involved data
> structures
> (I'm limited to its relationship with [real-time] scheduling), and obtained
> this:
>
>  http://retis.sssup.it/~tommaso/cgroups.odg
>  http://retis.sssup.it/~tommaso/cgroups.eps
>
> (You can see in the bottom left part of the diagram a little
> "key/legend/pattern" for representing lists).
>
> I might have done mistakes, however the greatest doubts that I have now
> concern the relative
> cardinalities of the various associated items. Namely:
> a) why doesn't a cgroup object directly point to a css_set one, but to a
> list of them (via cg_cgroup_list elements) ?
>    it seems that a cgroup object may be associated to multiple css_set
> objects, which in turn contain vectors of
>    cgroup_subsys_state;
> b) however, cgroup.subsys[] would point to a single cgroup_subsys_state
> object per subsys_id, so, what is the
>    difference between cgroup.subsys[] and css_set.subsys[] ? (or, are these
> all redundant pointers and point
>    to the same cgroup_subsys_state objects ?)
> c) is css_set.cg_links used to point to (the head of) a list of
> cg_cgroup_link objects, or is it used to link multiple css_set objects into
> a list ? In the latter case, where is the head of the list pointed to from ?
>
> Apologies for the newbie questions that I might have posted; FYI, I'm trying
> to set-up RT scheduling groups without using the VFS-based cgroups
> interface.
>
> Thanks in advance, regards (please, reply in cc to my e-mail address).
>
>    Tommaso
>
> --

Re: Understanding cgroups

[Dhaval Giani]

[Adding a few more people and the containers to the copy]

On Thu, Oct 28, 2010 at 5:44 PM, Tommaso Cucinotta
<tommaso.cucinotta@sssup.it> wrote:
> Hi,
>
> I'm trying to get some understanding of the current cgroups in-kernel
> implementation
> (after having read Documentation/cgroup* and having browsed a bit the code).
> To this purpose, I tried to draw the relationships among the involved data
> structures
> (I'm limited to its relationship with [real-time] scheduling), and obtained
> this:
>
>  http://retis.sssup.it/~tommaso/cgroups.odg
>  http://retis.sssup.it/~tommaso/cgroups.eps
>
> (You can see in the bottom left part of the diagram a little
> "key/legend/pattern" for representing lists).
>
> I might have done mistakes, however the greatest doubts that I have now
> concern the relative
> cardinalities of the various associated items. Namely:
> a) why doesn't a cgroup object directly point to a css_set one, but to a
> list of them (via cg_cgroup_list elements) ?
>    it seems that a cgroup object may be associated to multiple css_set
> objects, which in turn contain vectors of
>    cgroup_subsys_state;
> b) however, cgroup.subsys[] would point to a single cgroup_subsys_state
> object per subsys_id, so, what is the
>    difference between cgroup.subsys[] and css_set.subsys[] ? (or, are these
> all redundant pointers and point
>    to the same cgroup_subsys_state objects ?)
> c) is css_set.cg_links used to point to (the head of) a list of
> cg_cgroup_link objects, or is it used to link multiple css_set objects into
> a list ? In the latter case, where is the head of the list pointed to from ?
>
> Apologies for the newbie questions that I might have posted; FYI, I'm trying
> to set-up RT scheduling groups without using the VFS-based cgroups
> interface.
>
> Thanks in advance, regards (please, reply in cc to my e-mail address).
>
>    Tommaso
>
> --

Re: Understanding cgroups

[Paul Menage]

On Thu, Oct 28, 2010 at 8:44 AM, Tommaso Cucinotta
<tommaso.cucinotta-gAmJrWFzCps@public.gmane.org> wrote:
> a) why doesn't a cgroup object directly point to a css_set one, but to a
> list of them (via cg_cgroup_list elements) ?

Each task is in one cgroup per mounted hierarchy. There's no
requirement that different hierarchies have isomorphic partitions of
tasks (in fact that would somewhat defeat the point of mounting
separate hierarchies). So even though all the tasks in a cgroup share
css pointers for the subsystems attached to that cgroup's hierarchy,
in general they will have different total sets of css pointers. Hence
the multiple css_set objects associated with a cgroup.

>    it seems that a cgroup object may be associated to multiple css_set
> objects, which in turn contain vectors of
>    cgroup_subsys_state;
> b) however, cgroup.subsys[] would point to a single cgroup_subsys_state
> object per subsys_id,

No, it points to a single cgroup_subsys_state for each subsys that's
bound to the cgroup's hierarchy, and NULL for all others.

> so, what is the
>    difference between cgroup.subsys[] and css_set.subsys[] ? (or, are these
> all redundant pointers and point
>    to the same cgroup_subsys_state objects ?)

css_set.subsys is the union of all the (disjoint) sets of subsys
pointers in the cgroups that a task is a member of (one cgroup per
hierarchy).

Yes, the pointers in the css_set are redundant in that they could be
determined by following more pointers, but for efficiency they're all
available directly in the css_set. (Also this typically allows a
single refcount inc/dec in the task fork/exit case)

> c) is css_set.cg_links used to point to (the head of) a list of
> cg_cgroup_link objects,

Yes. It's the head of a list of cg_cgroup_link objects, each of which
has a pointer to this css_set and a pointer to a cgroup of which this
css_set represents membership. (One per hierarchy)

> or is it used to link multiple css_set objects into
> a list ?

In a way this is also true, but only indirectly - the cg_cgroup_link
objects form a two-dimensional doubly-linked list lattice that
represents the many:many relationship between cgroup and css_set. One
dimension's linked lists are headed by css_set objects, the other
dimension is headed by cgroups.

Paul

Re: Understanding cgroups

[Paul Menage]

On Thu, Oct 28, 2010 at 8:44 AM, Tommaso Cucinotta
<tommaso.cucinotta@sssup.it> wrote:
> a) why doesn't a cgroup object directly point to a css_set one, but to a
> list of them (via cg_cgroup_list elements) ?

Each task is in one cgroup per mounted hierarchy. There's no
requirement that different hierarchies have isomorphic partitions of
tasks (in fact that would somewhat defeat the point of mounting
separate hierarchies). So even though all the tasks in a cgroup share
css pointers for the subsystems attached to that cgroup's hierarchy,
in general they will have different total sets of css pointers. Hence
the multiple css_set objects associated with a cgroup.

>    it seems that a cgroup object may be associated to multiple css_set
> objects, which in turn contain vectors of
>    cgroup_subsys_state;
> b) however, cgroup.subsys[] would point to a single cgroup_subsys_state
> object per subsys_id,

No, it points to a single cgroup_subsys_state for each subsys that's
bound to the cgroup's hierarchy, and NULL for all others.

> so, what is the
>    difference between cgroup.subsys[] and css_set.subsys[] ? (or, are these
> all redundant pointers and point
>    to the same cgroup_subsys_state objects ?)

css_set.subsys is the union of all the (disjoint) sets of subsys
pointers in the cgroups that a task is a member of (one cgroup per
hierarchy).

Yes, the pointers in the css_set are redundant in that they could be
determined by following more pointers, but for efficiency they're all
available directly in the css_set. (Also this typically allows a
single refcount inc/dec in the task fork/exit case)

> c) is css_set.cg_links used to point to (the head of) a list of
> cg_cgroup_link objects,

Yes. It's the head of a list of cg_cgroup_link objects, each of which
has a pointer to this css_set and a pointer to a cgroup of which this
css_set represents membership. (One per hierarchy)

> or is it used to link multiple css_set objects into
> a list ?

In a way this is also true, but only indirectly - the cg_cgroup_link
objects form a two-dimensional doubly-linked list lattice that
represents the many:many relationship between cgroup and css_set. One
dimension's linked lists are headed by css_set objects, the other
dimension is headed by cgroups.

Paul