[PATCH V2 1/2] sched/fair: Rearrange select_task_rq_fair() to optimize it

[PATCH V2 1/2] sched/fair: Rearrange select_task_rq_fair() to optimize it

[Viresh Kumar]

Rearrange select_task_rq_fair() a bit to avoid executing some
conditional statements in few specific code-paths. That gets rid of the
goto as well.

This shouldn't result in any functional changes.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Rohit Jain <rohit.k.jain@oracle.com>

---
V1->V2:
- Optimize a bit more and get rid of affine_sd variable (Valentin)
- Add unlikely while checking for non-NULL sd and add fast/slow path
  comments (Joel)
- Add tested-by from Rohit.

 kernel/sched/fair.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 54dc31e7ab9b..84fc74ddbd4b 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6613,7 +6613,7 @@ static int wake_cap(struct task_struct *p, int cpu, int prev_cpu)
 static int
 select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags)
 {
-	struct sched_domain *tmp, *affine_sd = NULL, *sd = NULL;
+	struct sched_domain *tmp, *sd = NULL;
 	int cpu = smp_processor_id();
 	int new_cpu = prev_cpu;
 	int want_affine = 0;
@@ -6636,7 +6636,10 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 		 */
 		if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
 		    cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
-			affine_sd = tmp;
+			if (cpu != prev_cpu)
+				new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync);
+
+			sd = NULL; /* Prefer wake_affine over balance flags */
 			break;
 		}
 
@@ -6646,33 +6649,25 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 			break;
 	}
 
-	if (affine_sd) {
-		sd = NULL; /* Prefer wake_affine over balance flags */
-		if (cpu == prev_cpu)
-			goto pick_cpu;
-
-		new_cpu = wake_affine(affine_sd, p, cpu, prev_cpu, sync);
-	}
+	if (unlikely(sd)) {
+		/* Slow path */
 
-	if (sd && !(sd_flag & SD_BALANCE_FORK)) {
 		/*
 		 * We're going to need the task's util for capacity_spare_wake
 		 * in find_idlest_group. Sync it up to prev_cpu's
 		 * last_update_time.
 		 */
-		sync_entity_load_avg(&p->se);
-	}
-
-	if (!sd) {
-pick_cpu:
-		if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
-			new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
+		if (!(sd_flag & SD_BALANCE_FORK))
+			sync_entity_load_avg(&p->se);
 
-			if (want_affine)
-				current->recent_used_cpu = cpu;
-		}
-	} else {
 		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
+	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
+		/* Fast path */
+
+		new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
+
+		if (want_affine)
+			current->recent_used_cpu = cpu;
 	}
 	rcu_read_unlock();
 
-- 
2.15.0.194.g9af6a3dea062

[PATCH 2/2] sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

[Viresh Kumar]

Call sync_entity_load_avg() directly from find_idlest_cpu() instead of
select_task_rq_fair(), as that's where we need to use task's utilization
value. And call sync_entity_load_avg() only after making sure sched
domain spans over one of the allowed CPUs for the task.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 kernel/sched/fair.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 84fc74ddbd4b..5b1b4f91f132 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6199,6 +6199,13 @@ static inline int find_idlest_cpu(struct sched_domain *sd, struct task_struct *p
 	if (!cpumask_intersects(sched_domain_span(sd), &p->cpus_allowed))
 		return prev_cpu;
 
+	/*
+	 * We need task's util for capacity_spare_wake, sync it up to prev_cpu's
+	 * last_update_time.
+	 */
+	if (!(sd_flag & SD_BALANCE_FORK))
+		sync_entity_load_avg(&p->se);
+
 	while (sd) {
 		struct sched_group *group;
 		struct sched_domain *tmp;
@@ -6651,15 +6658,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 
 	if (unlikely(sd)) {
 		/* Slow path */
-
-		/*
-		 * We're going to need the task's util for capacity_spare_wake
-		 * in find_idlest_group. Sync it up to prev_cpu's
-		 * last_update_time.
-		 */
-		if (!(sd_flag & SD_BALANCE_FORK))
-			sync_entity_load_avg(&p->se);
-
 		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
 	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
 		/* Fast path */
-- 
2.15.0.194.g9af6a3dea062

Re: [PATCH V2 1/2] sched/fair: Rearrange select_task_rq_fair() to optimize it

[Valentin Schneider]

Hi,

LGTM. Tiny inline comment but TBH might not be worth it.

FWIW: Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

On 26/04/18 11:30, Viresh Kumar wrote:
> Rearrange select_task_rq_fair() a bit to avoid executing some
> conditional statements in few specific code-paths. That gets rid of the
> goto as well.
> 
> This shouldn't result in any functional changes.
> 
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> Tested-by: Rohit Jain <rohit.k.jain@oracle.com>
> 
> ---
> V1->V2:
> - Optimize a bit more and get rid of affine_sd variable (Valentin)
> - Add unlikely while checking for non-NULL sd and add fast/slow path
>   comments (Joel)
> - Add tested-by from Rohit.
> 
>  kernel/sched/fair.c | 37 ++++++++++++++++---------------------
>  1 file changed, 16 insertions(+), 21 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 54dc31e7ab9b..84fc74ddbd4b 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -6613,7 +6613,7 @@ static int wake_cap(struct task_struct *p, int cpu, int prev_cpu)
>  static int
>  select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags)
>  {
> -	struct sched_domain *tmp, *affine_sd = NULL, *sd = NULL;
> +	struct sched_domain *tmp, *sd = NULL;
>  	int cpu = smp_processor_id();
>  	int new_cpu = prev_cpu;
>  	int want_affine = 0;
> @@ -6636,7 +6636,10 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
>  		 */
>  		if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
>  		    cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
> -			affine_sd = tmp;
> +			if (cpu != prev_cpu)
> +				new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync);

This cpu != prev_cpu check could be folded into wake_affine() to make this
look a little neater, but we might want to keep things as is to avoid a
function call (it's only ever called once so it might get inlined, but AFAIK
that's not guaranteed).

> +
> +			sd = NULL; /* Prefer wake_affine over balance flags */
>  			break;
>  		}
>  
> @@ -6646,33 +6649,25 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
>  			break;
>  	}
>  
> -	if (affine_sd) {
> -		sd = NULL; /* Prefer wake_affine over balance flags */
> -		if (cpu == prev_cpu)
> -			goto pick_cpu;
> -
> -		new_cpu = wake_affine(affine_sd, p, cpu, prev_cpu, sync);
> -	}
> +	if (unlikely(sd)) {
> +		/* Slow path */
>  
> -	if (sd && !(sd_flag & SD_BALANCE_FORK)) {
>  		/*
>  		 * We're going to need the task's util for capacity_spare_wake
>  		 * in find_idlest_group. Sync it up to prev_cpu's
>  		 * last_update_time.
>  		 */
> -		sync_entity_load_avg(&p->se);
> -	}
> -
> -	if (!sd) {
> -pick_cpu:
> -		if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
> -			new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
> +		if (!(sd_flag & SD_BALANCE_FORK))
> +			sync_entity_load_avg(&p->se);
>  
> -			if (want_affine)
> -				current->recent_used_cpu = cpu;
> -		}
> -	} else {
>  		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
> +	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
> +		/* Fast path */
> +
> +		new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
> +
> +		if (want_affine)
> +			current->recent_used_cpu = cpu;
>  	}
>  	rcu_read_unlock();
>  
> 

Re: [PATCH 2/2] sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

[Dietmar Eggemann]

Hi Viresh,

On 04/26/2018 12:30 PM, Viresh Kumar wrote:
> Call sync_entity_load_avg() directly from find_idlest_cpu() instead of
> select_task_rq_fair(), as that's where we need to use task's utilization
> value. And call sync_entity_load_avg() only after making sure sched
> domain spans over one of the allowed CPUs for the task.
> 
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

The patch looks correct to me but we want to have the waking task synced 
against its previous rq also for EAS, i.e. for 
find_energy_efficient_cpu() which will sit next to find_idlest_cpu().

https://marc.info/?l=linux-kernel&m=152302907327168&w=2

The comment on top of the if condition would have to be changed though.

I would suggest we leave the call to sync_entity_load_avg() in the slow 
path of strf() so that we're not forced to call it in 
find_energy_efficient_cpu().

> ---
>   kernel/sched/fair.c | 16 +++++++---------
>   1 file changed, 7 insertions(+), 9 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 84fc74ddbd4b..5b1b4f91f132 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -6199,6 +6199,13 @@ static inline int find_idlest_cpu(struct sched_domain *sd, struct task_struct *p
>   	if (!cpumask_intersects(sched_domain_span(sd), &p->cpus_allowed))
>   		return prev_cpu;
>   
> +	/*
> +	 * We need task's util for capacity_spare_wake, sync it up to prev_cpu's
> +	 * last_update_time.
> +	 */
> +	if (!(sd_flag & SD_BALANCE_FORK))
> +		sync_entity_load_avg(&p->se);
> +
>   	while (sd) {
>   		struct sched_group *group;
>   		struct sched_domain *tmp;
> @@ -6651,15 +6658,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
>   
>   	if (unlikely(sd)) {
>   		/* Slow path */
> -
> -		/*
> -		 * We're going to need the task's util for capacity_spare_wake
> -		 * in find_idlest_group. Sync it up to prev_cpu's
> -		 * last_update_time.
> -		 */
> -		if (!(sd_flag & SD_BALANCE_FORK))
> -			sync_entity_load_avg(&p->se);
> -
>   		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
>   	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
>   		/* Fast path */
> 

Re: [PATCH 2/2] sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

[Viresh Kumar]

Hi Dietmar,

On 27-04-18, 10:30, Dietmar Eggemann wrote:
> I would suggest we leave the call to sync_entity_load_avg() in the slow path
> of strf() so that we're not forced to call it in
> find_energy_efficient_cpu().

Well, that's what I did in the very first attempt:

https://lkml.kernel.org/r/20180425051509.aohopadqw7q5urbd@vireshk-i7

But then I realized that based on the current state of code, its
better to move it to find_idlest_cpu() instead.

I would be fine with sending the above patch instead (if Peter
agrees). Though it should be trivial to move it back to strq() in the
patch where you are adding call to find_energy_efficient_cpu().

I am fine with both the options though.

-- 
viresh

Re: [PATCH 2/2] sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

[Quentin Perret]

Hi Dietmar,

On Friday 27 Apr 2018 at 10:30:39 (+0200), Dietmar Eggemann wrote:
> Hi Viresh,
> 
> On 04/26/2018 12:30 PM, Viresh Kumar wrote:
> > Call sync_entity_load_avg() directly from find_idlest_cpu() instead of
> > select_task_rq_fair(), as that's where we need to use task's utilization
> > value. And call sync_entity_load_avg() only after making sure sched
> > domain spans over one of the allowed CPUs for the task.
> > 
> > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> 
> The patch looks correct to me but we want to have the waking task synced
> against its previous rq also for EAS, i.e. for find_energy_efficient_cpu()
> which will sit next to find_idlest_cpu().
> 
> https://marc.info/?l=linux-kernel&m=152302907327168&w=2
> 
> The comment on top of the if condition would have to be changed though.
> 
> I would suggest we leave the call to sync_entity_load_avg() in the slow path
> of strf() so that we're not forced to call it in
> find_energy_efficient_cpu().

With the new implementation of the overutilization mechanism agreed at
OSPM, I don't think we'll be able to avoid calling sync_entity_load_avg()
from find_energy_efficient_cpu(). The EAS integration within strf()
will have to be reworked, so I'm happy if the sync_entity_load_avg()
call moves from strf() to find_idlest_cpu() in the meantime.

Thanks,
Quentin

> 
> > ---
> >   kernel/sched/fair.c | 16 +++++++---------
> >   1 file changed, 7 insertions(+), 9 deletions(-)
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 84fc74ddbd4b..5b1b4f91f132 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -6199,6 +6199,13 @@ static inline int find_idlest_cpu(struct sched_domain *sd, struct task_struct *p
> >   	if (!cpumask_intersects(sched_domain_span(sd), &p->cpus_allowed))
> >   		return prev_cpu;
> > +	/*
> > +	 * We need task's util for capacity_spare_wake, sync it up to prev_cpu's
> > +	 * last_update_time.
> > +	 */
> > +	if (!(sd_flag & SD_BALANCE_FORK))
> > +		sync_entity_load_avg(&p->se);
> > +
> >   	while (sd) {
> >   		struct sched_group *group;
> >   		struct sched_domain *tmp;
> > @@ -6651,15 +6658,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
> >   	if (unlikely(sd)) {
> >   		/* Slow path */
> > -
> > -		/*
> > -		 * We're going to need the task's util for capacity_spare_wake
> > -		 * in find_idlest_group. Sync it up to prev_cpu's
> > -		 * last_update_time.
> > -		 */
> > -		if (!(sd_flag & SD_BALANCE_FORK))
> > -			sync_entity_load_avg(&p->se);
> > -
> >   		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
> >   	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
> >   		/* Fast path */
> > 
> 

[tip:sched/core] sched/fair: Rearrange select_task_rq_fair() to optimize it

[tip-bot for Viresh Kumar]

Commit-ID:  f1d88b4468188ddcd2620b8d612068faf6662a62
Gitweb:     https://git.kernel.org/tip/f1d88b4468188ddcd2620b8d612068faf6662a62
Author:     Viresh Kumar <viresh.kumar@linaro.org>
AuthorDate: Thu, 26 Apr 2018 16:00:50 +0530
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 4 May 2018 10:00:07 +0200

sched/fair: Rearrange select_task_rq_fair() to optimize it

Rearrange select_task_rq_fair() a bit to avoid executing some
conditional statements in few specific code-paths. That gets rid of the
goto as well.

This shouldn't result in any functional changes.

Tested-by: Rohit Jain <rohit.k.jain@oracle.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Link: http://lkml.kernel.org/r/20831b8d237bf3a20e4e328286f678b425ff04c9.1524738578.git.viresh.kumar@linaro.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/fair.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index e3002e5ada31..4b346f358005 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6613,7 +6613,7 @@ static int wake_cap(struct task_struct *p, int cpu, int prev_cpu)
 static int
 select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags)
 {
-	struct sched_domain *tmp, *affine_sd = NULL, *sd = NULL;
+	struct sched_domain *tmp, *sd = NULL;
 	int cpu = smp_processor_id();
 	int new_cpu = prev_cpu;
 	int want_affine = 0;
@@ -6636,7 +6636,10 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 		 */
 		if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
 		    cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
-			affine_sd = tmp;
+			if (cpu != prev_cpu)
+				new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync);
+
+			sd = NULL; /* Prefer wake_affine over balance flags */
 			break;
 		}
 
@@ -6646,33 +6649,25 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 			break;
 	}
 
-	if (affine_sd) {
-		sd = NULL; /* Prefer wake_affine over balance flags */
-		if (cpu == prev_cpu)
-			goto pick_cpu;
-
-		new_cpu = wake_affine(affine_sd, p, cpu, prev_cpu, sync);
-	}
+	if (unlikely(sd)) {
+		/* Slow path */
 
-	if (sd && !(sd_flag & SD_BALANCE_FORK)) {
 		/*
 		 * We're going to need the task's util for capacity_spare_wake
 		 * in find_idlest_group. Sync it up to prev_cpu's
 		 * last_update_time.
 		 */
-		sync_entity_load_avg(&p->se);
-	}
-
-	if (!sd) {
-pick_cpu:
-		if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
-			new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
+		if (!(sd_flag & SD_BALANCE_FORK))
+			sync_entity_load_avg(&p->se);
 
-			if (want_affine)
-				current->recent_used_cpu = cpu;
-		}
-	} else {
 		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
+	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
+		/* Fast path */
+
+		new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
+
+		if (want_affine)
+			current->recent_used_cpu = cpu;
 	}
 	rcu_read_unlock();
 

[tip:sched/core] sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

[tip-bot for Viresh Kumar]

Commit-ID:  c976a862ba4869c1e75c39b9b8f1e9ebfe90cdfc
Gitweb:     https://git.kernel.org/tip/c976a862ba4869c1e75c39b9b8f1e9ebfe90cdfc
Author:     Viresh Kumar <viresh.kumar@linaro.org>
AuthorDate: Thu, 26 Apr 2018 16:00:51 +0530
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 4 May 2018 10:00:08 +0200

sched/fair: Avoid calling sync_entity_load_avg() unnecessarily

Call sync_entity_load_avg() directly from find_idlest_cpu() instead of
select_task_rq_fair(), as that's where we need to use task's utilization
value. And call sync_entity_load_avg() only after making sure sched
domain spans over one of the allowed CPUs for the task.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Link: http://lkml.kernel.org/r/cd019d1753824c81130eae7b43e2bbcec47cc1ad.1524738578.git.viresh.kumar@linaro.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/sched/fair.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 4b346f358005..1f6a23a5b451 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6199,6 +6199,13 @@ static inline int find_idlest_cpu(struct sched_domain *sd, struct task_struct *p
 	if (!cpumask_intersects(sched_domain_span(sd), &p->cpus_allowed))
 		return prev_cpu;
 
+	/*
+	 * We need task's util for capacity_spare_wake, sync it up to prev_cpu's
+	 * last_update_time.
+	 */
+	if (!(sd_flag & SD_BALANCE_FORK))
+		sync_entity_load_avg(&p->se);
+
 	while (sd) {
 		struct sched_group *group;
 		struct sched_domain *tmp;
@@ -6651,15 +6658,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 
 	if (unlikely(sd)) {
 		/* Slow path */
-
-		/*
-		 * We're going to need the task's util for capacity_spare_wake
-		 * in find_idlest_group. Sync it up to prev_cpu's
-		 * last_update_time.
-		 */
-		if (!(sd_flag & SD_BALANCE_FORK))
-			sync_entity_load_avg(&p->se);
-
 		new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag);
 	} else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */
 		/* Fast path */