Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

[Diwakar Tundlam]

We ran into this at Nvidia. QA filed a bug saying coremark_4pthreads scores lower (as if running on 3 cores) when run shortly after bootup. But later its score increases to expected values on 4 cores.

This patch is relevant to linux-2.6.39 but I noticed this fix is not made in linux-3.0, 3.1, 3.2 or 3.3 also.

Please see commit log for more details of the problem and fix.

Thanks,
  Diwakar.
  Tegra Android Kernel SW Engg.
  NVIDIA.
  Santa Clara, CA

commit d04d7ef0e3f8c70bd6cd5abb2abc0236aa8d1f7c
Author: Diwakar Tundlam <dtundlam@nvidia.com>
Date:   Wed Jan 18 18:58:57 2012 -0800

    scheduler: domain: init next_balance in nohz_idle_balancer with jiffies
    
    The next_balance parameter of nohz_idle_balancer should be initialized
    to jiffies since jiffies itself is initialized to 300 seconds shy of
    overflow. Otherwise, nohz_idle_balancer does not run for the first 5
    mins after bootup.
    
    Signed-off-by: Diwakar Tundlam <dtundlam@nvidia.com>
    Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
    Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
    Reviewed-by: Satya Popuri <spopuri@nvidia.com>

 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c index c5b09f7..506c5da 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -8288,6 +8288,7 @@ void __init sched_init(void)
        atomic_set(&nohz.load_balancer, nr_cpu_ids);
        atomic_set(&nohz.first_pick_cpu, nr_cpu_ids);
        atomic_set(&nohz.second_pick_cpu, nr_cpu_ids);
+       nohz.next_balance = jiffies;
 #endif
        /* May be allocated at isolcpus cmdline parse time */
        if (cpu_isolated_map == NULL)

Re: Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

[Mike Galbraith]

On Mon, 2012-01-30 at 10:14 -0800, Diwakar Tundlam wrote:
> We ran into this at Nvidia. QA filed a bug saying coremark_4pthreads scores lower (as if running on 3 cores) when run shortly after bootup. But later its score increases to expected values on 4 cores.
> 
> This patch is relevant to linux-2.6.39 but I noticed this fix is not made in linux-3.0, 3.1, 3.2 or 3.3 also.
> 
> Please see commit log for more details of the problem and fix.
> 
> Thanks,
>   Diwakar.
>   Tegra Android Kernel SW Engg.
>   NVIDIA.
>   Santa Clara, CA
> 
> commit d04d7ef0e3f8c70bd6cd5abb2abc0236aa8d1f7c
> Author: Diwakar Tundlam <dtundlam@nvidia.com>
> Date:   Wed Jan 18 18:58:57 2012 -0800
> 
>     scheduler: domain: init next_balance in nohz_idle_balancer with jiffies
>     
>     The next_balance parameter of nohz_idle_balancer should be initialized
>     to jiffies since jiffies itself is initialized to 300 seconds shy of
>     overflow. Otherwise, nohz_idle_balancer does not run for the first 5
>     mins after bootup.
>     
>     Signed-off-by: Diwakar Tundlam <dtundlam@nvidia.com>
>     Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
>     Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
>     Reviewed-by: Satya Popuri <spopuri@nvidia.com>
> 
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/sched.c b/kernel/sched.c index c5b09f7..506c5da 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -8288,6 +8288,7 @@ void __init sched_init(void)
>         atomic_set(&nohz.load_balancer, nr_cpu_ids);
>         atomic_set(&nohz.first_pick_cpu, nr_cpu_ids);
>         atomic_set(&nohz.second_pick_cpu, nr_cpu_ids);
> +       nohz.next_balance = jiffies;
>  #endif
>         /* May be allocated at isolcpus cmdline parse time */
>         if (cpu_isolated_map == NULL)

Shouldn't that really be INITIAL_JIFFIES + HZ?

	-Mike

RE: Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

[Diwakar Tundlam]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2644 bytes --]

>> Shouldn't that really be INITIAL_JIFFIES + HZ?
That may work as well. I was modeling the change after similar initialization of the 'next_balance' field in each rq which is done a few lines earlier. And nohz.next_balance follows rq->next_balance.

-----Original Message-----
From: Mike Galbraith [mailto:efault@gmx.de] 
Sent: Monday, January 30, 2012 7:49 PM
To: Diwakar Tundlam
Cc: 'Ingo Molnar'; 'Peter Zijlstra'; 'linux-kernel@vger.kernel.org'; Peter De Schrijver; Antti Miettinen; Matthew Longnecker; Kevin Kranzusch
Subject: Re: Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

On Mon, 2012-01-30 at 10:14 -0800, Diwakar Tundlam wrote:
> We ran into this at Nvidia. QA filed a bug saying coremark_4pthreads scores lower (as if running on 3 cores) when run shortly after bootup. But later its score increases to expected values on 4 cores.
> 
> This patch is relevant to linux-2.6.39 but I noticed this fix is not made in linux-3.0, 3.1, 3.2 or 3.3 also.
> 
> Please see commit log for more details of the problem and fix.
> 
> Thanks,
>   Diwakar.
>   Tegra Android Kernel SW Engg.
>   NVIDIA.
>   Santa Clara, CA
> 
> commit d04d7ef0e3f8c70bd6cd5abb2abc0236aa8d1f7c
> Author: Diwakar Tundlam <dtundlam@nvidia.com>
> Date:   Wed Jan 18 18:58:57 2012 -0800
> 
>     scheduler: domain: init next_balance in nohz_idle_balancer with jiffies
>     
>     The next_balance parameter of nohz_idle_balancer should be initialized
>     to jiffies since jiffies itself is initialized to 300 seconds shy of
>     overflow. Otherwise, nohz_idle_balancer does not run for the first 5
>     mins after bootup.
>     
>     Signed-off-by: Diwakar Tundlam <dtundlam@nvidia.com>
>     Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
>     Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
>     Reviewed-by: Satya Popuri <spopuri@nvidia.com>
> 
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/sched.c b/kernel/sched.c index c5b09f7..506c5da 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -8288,6 +8288,7 @@ void __init sched_init(void)
>         atomic_set(&nohz.load_balancer, nr_cpu_ids);
>         atomic_set(&nohz.first_pick_cpu, nr_cpu_ids);
>         atomic_set(&nohz.second_pick_cpu, nr_cpu_ids);
> +       nohz.next_balance = jiffies;
>  #endif
>         /* May be allocated at isolcpus cmdline parse time */
>         if (cpu_isolated_map == NULL)

Shouldn't that really be INITIAL_JIFFIES + HZ?

	-Mike

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: Fix bug: Scheduler's idle-load-balancer not running in first 5 mins after bootup

[Kamalesh Babulal]

* Diwakar Tundlam <dtundlam@nvidia.com> [2012-01-30 10:14:23]:

> We ran into this at Nvidia. QA filed a bug saying coremark_4pthreads scores lower (as if running on 3 cores) when run shortly after bootup. But later its score increases to expected values on 4 cores.
> 
> This patch is relevant to linux-2.6.39 but I noticed this fix is not made in linux-3.0, 3.1, 3.2 or 3.3 also.
> 
> Please see commit log for more details of the problem and fix.
> 
> Thanks,
>   Diwakar.
>   Tegra Android Kernel SW Engg.
>   NVIDIA.
>   Santa Clara, CA
> 
> commit d04d7ef0e3f8c70bd6cd5abb2abc0236aa8d1f7c
> Author: Diwakar Tundlam <dtundlam@nvidia.com>
> Date:   Wed Jan 18 18:58:57 2012 -0800
> 
>     scheduler: domain: init next_balance in nohz_idle_balancer with jiffies
>     
>     The next_balance parameter of nohz_idle_balancer should be initialized
>     to jiffies since jiffies itself is initialized to 300 seconds shy of
>     overflow. Otherwise, nohz_idle_balancer does not run for the first 5
>     mins after bootup.
>     
>     Signed-off-by: Diwakar Tundlam <dtundlam@nvidia.com>
>     Reviewed-by: Aleksandr Frid <afrid@nvidia.com>
>     Reviewed-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
>     Reviewed-by: Satya Popuri <spopuri@nvidia.com>
> 
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/sched.c b/kernel/sched.c index c5b09f7..506c5da 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c

Can you rebase the patch against tip. kernel/sched* has been
moved to kernel/sched/.

> @@ -8288,6 +8288,7 @@ void __init sched_init(void)
>         atomic_set(&nohz.load_balancer, nr_cpu_ids);
>         atomic_set(&nohz.first_pick_cpu, nr_cpu_ids);
>         atomic_set(&nohz.second_pick_cpu, nr_cpu_ids);

above lines are removed by 0b005cf54eac

> +       nohz.next_balance = jiffies;
>  #endif
>         /* May be allocated at isolcpus cmdline parse time */
>         if (cpu_isolated_map == NULL)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>