Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

[Matthew Dobson <colpatch@us.ibm.com>]

On Tue, 2004-10-05 at 20:01, Paul Jackson wrote:
> Matthew writes:
> > 
> > If that's all 'exclusive' means then 'exclusive' is a poor choice of
> > terminology.  'Exclusive' sounds like it would exclude all tasks it is
> > possible to exclude from running there (ie: with the exception of
> > certain necessary kernel threads).
> 
> I suspect that my aggressive pushing of mechanism _out_ of the
> kernel has obscured what's going on here.
> 
> The real 'exclusive' use of some set of CPUs and Memory Nodes
> is provided by the workload managers, PBS and LSF.  They fabricate
> this out of the kernel cpuset 'exclusive' property, plus other
> optional user level stuff.
> 
> For instance, one doesn't have to follow Simon's example, and leave the
> classic Unix daemon load running in a cpuset that share resources with
> all other cpusets.  Instead, one can coral this classic Unix load into a
> bootcpuset, administratively, at system boot.  All the kernel mechanisms
> required to support this exist in my current cpuset patch in Andrew's
> tree.
> 
> The kernel cpuset 'mems_exclusive' and 'cpus_exclusive' flags are like
> vitamin precursors.  They are elements out of which the real nutrative
> compound is constructed.  Occassionally, as in Simon's configuration,
> they are actually sufficient in their current state.  Usually, more
> processing is required.  This processing just isn't visible to the
> kernel code.
> 
> Perhaps these flags should be called:
> 	mems_exclusive_precursor
> 	cpus_exclusive_precursor
> ;).

Ok...  So if we could offer the 'real' exclusion that the PBS and LSF
workload managers offer directly, would that suffice?  Meaning, could we
make PBS and LSF work on top of in-kernel mechanisms that offer 'real'
exclusion.  'Real' exclusion defined as isolated groups of CPUs and
memory that the kernel can guarantee will not run other processes?  That
way we can get the job done without having to rely on these external
workload managers, and be able to offer this dynamic partitioning to all
users.  Thoughts?

-Matt