[PATCH 0/5] cpusets: dynamical scheduler domain flags

[PATCH 0/5] cpusets: dynamical scheduler domain flags

[Michael Wang]

From: Michael Wang <wangyun@linux.vnet.ibm.com>

This patch set provide a way for user to dynamically configure the scheduler
domain flags, which usually to be static.

We can do the configuration through cpuset cgroup, new file will be found
under each hierarchy:

sched_smt_domain_flag
	-- appear when CONFIG_SCHED_SMT enabled, stands for the domain flags
	   of cpuset on smt level
	
sched_mc_domain_flag
	-- appear when CONFIG_SCHED_MC enabled, stands for the domain flags
	   of cpuset on mc level

sched_book_domain_flag
	-- appear when CONFIG_SCHED_BOOK enabled, stands for the domain flags
	   of cpuset on book level

sched_cpu_domain_flag
	-- stands for the domain flags of cpuset on cpu level

sched_enable_domain_flag
	-- set to 1 if want to use the dynamical domain flags, domain will
	   be rebuild

Examples:

Currently the domain flags bit are:
#define SD_LOAD_BALANCE         0x0001  /* Do load balancing on this domain. */
#define SD_BALANCE_NEWIDLE      0x0002  /* Balance when about to become idle */
#define SD_BALANCE_EXEC         0x0004  /* Balance on exec */
#define SD_BALANCE_FORK         0x0008  /* Balance on fork, clone */
#define SD_BALANCE_WAKE         0x0010  /* Balance on wakeup */
#define SD_WAKE_AFFINE          0x0020  /* Wake task to waking CPU */
#define SD_PREFER_LOCAL         0x0040  /* Prefer to keep tasks local to this domain */
#define SD_SHARE_CPUPOWER       0x0080  /* Domain members share cpu power */
#define SD_SHARE_PKG_RESOURCES  0x0200  /* Domain members share cpu pkg resources */
#define SD_SERIALIZE            0x0400  /* Only a single load balancing instance */
#define SD_ASYM_PACKING         0x0800  /* Place busy groups earlier in the domain */
#define SD_PREFER_SIBLING       0x1000  /* Prefer to place tasks in a sibling domain */
#define SD_OVERLAP              0x2000  /* sched_domains of this level overlap */
#define SD_NUMA                 0x4000  /* cross-node balancing */

If we want to set SMT domain to be:
	'SD_LOAD_BALANCE | SD_BALANCE_WAKE | SD_OVERLAP'
and other level domain only has SD_LOAD_BALANCE, then we can use:
	echo 0x1011 > /sys/fs/cgroup/cpuset/cpuset.sched_smt_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_mc_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_book_domain_flag
	echo 0x1 > /sys/fs/cgroup/cpuset/cpuset.sched_cpu_domain_flag
	echo 1 > /sys/fs/cgroup/cpuset/cpuset.sched_enable_domain_flag

including:
	cpusets: add basic variables
	cpusets: add functions and code for initialization
	cpusets: enable the dynamical domain flags
	cpusets: add fundamental functions for recording
	cpusets: add the configuration facility
		
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
---
 b/include/linux/sched.h |   22 ++++++++
 b/kernel/cpuset.c       |    7 ++
 b/kernel/sched/core.c   |    2 
 include/linux/sched.h   |   10 ++-
 kernel/cpuset.c         |  131 +++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/sched/core.c     |   10 +++
 6 files changed, 176 insertions(+), 6 deletions(-)

Re: [PATCH 0/5] cpusets: dynamical scheduler domain flags

[Peter Zijlstra]

On Tue, 2012-07-17 at 17:03 +0800, Michael Wang wrote:
> This patch set provide a way for user to dynamically configure the scheduler
> domain flags, which usually to be static. 

NAK.. you don't get to expose all this nonsense in a 'stable' ABI.

You shouldn't need to prod at them to begin with.

Re: [PATCH 0/5] cpusets: dynamical scheduler domain flags

[Michael Wang]

On 07/21/2012 12:42 AM, Peter Zijlstra wrote:
> On Tue, 2012-07-17 at 17:03 +0800, Michael Wang wrote:
>> This patch set provide a way for user to dynamically configure the scheduler
>> domain flags, which usually to be static. 
> 
> NAK.. you don't get to expose all this nonsense in a 'stable' ABI.
> 
> You shouldn't need to prod at them to begin with.

So is that means expose those domain flags to user is a bad idea at all?

Is this caused by that people don't need such feature or this feature
has do harm to kernel's stability?(it could be not...)

Regards,
Michael Wang

> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [PATCH 0/5] cpusets: dynamical scheduler domain flags

[Mike Galbraith]

On Mon, 2012-07-23 at 10:30 +0800, Michael Wang wrote: 
> On 07/21/2012 12:42 AM, Peter Zijlstra wrote:
> > On Tue, 2012-07-17 at 17:03 +0800, Michael Wang wrote:
> >> This patch set provide a way for user to dynamically configure the scheduler
> >> domain flags, which usually to be static. 
> > 
> > NAK.. you don't get to expose all this nonsense in a 'stable' ABI.
> > 
> > You shouldn't need to prod at them to begin with.
> 
> So is that means expose those domain flags to user is a bad idea at all?

You can set/clear flags with scripts now, ie domain flags are already
exposed.. as defined by the running kernel.

SD_SHARE_PKG_RESOURCES is a good flag look at.  What does flipping that
switch do, and what did it stop doing recently?  So yeah, methinks
exporting flags via cpusets is a bad idea.  Not only is existence of any
particular flag volatile, functionality behind it is volatile as well,
so having a button to poke does undefined things.  (not to mention
non-exclusive sets)

-Mike

Re: [PATCH 0/5] cpusets: dynamical scheduler domain flags

[Michael Wang]

On 07/23/2012 12:28 PM, Mike Galbraith wrote:
> On Mon, 2012-07-23 at 10:30 +0800, Michael Wang wrote: 
>> On 07/21/2012 12:42 AM, Peter Zijlstra wrote:
>>> On Tue, 2012-07-17 at 17:03 +0800, Michael Wang wrote:
>>>> This patch set provide a way for user to dynamically configure the scheduler
>>>> domain flags, which usually to be static. 
>>>
>>> NAK.. you don't get to expose all this nonsense in a 'stable' ABI.
>>>
>>> You shouldn't need to prod at them to begin with.
>>
>> So is that means expose those domain flags to user is a bad idea at all?
> 
> You can set/clear flags with scripts now, ie domain flags are already
> exposed.. as defined by the running kernel.
> 
> SD_SHARE_PKG_RESOURCES is a good flag look at.  What does flipping that
> switch do, and what did it stop doing recently?  So yeah, methinks
> exporting flags via cpusets is a bad idea.  Not only is existence of any
> particular flag volatile, functionality behind it is volatile as well,
> so having a button to poke does undefined things.  (not to mention
> non-exclusive sets)
> 

I think I got your and peter's opinion, so we could not make sure the
kernel could still work well if some flags was enabled because their
behave are always changing, and it's impossible to maintain such
volatile feature.

Actually I got this idea after reading:

http://marc.info/?l=linux-kernel&m=130822782111533

But looks like I don't get the point, so what we want is building the
domain according to some system topology designed by producer?

Regards,
Michael Wang

> -Mike
> 
>