[PATCH] sched: Initialize rq->age_stamp on processor start

[PATCH] sched: Initialize rq->age_stamp on processor start

[minyard]

From: Corey Minyard <cminyard@mvista.com>

If the sched_clock time starts at a large value, the kernel will spin
in sched_avg_update for a long time while rq->age_stamp catches up
with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise
that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
to avoid this.

I was seeing long delays on a simulator that didn't start the clock at
zero.  This might also be an issue on reboots on processors that don't
re-initialize the timer to zero on reset, and when using kexec.

Signed-off-by: Corey Minyard <cminyard@mvista.com>
---
 kernel/sched/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b46131e..5be3d4a 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5037,11 +5037,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
 	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		/* fall through */
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6922,6 +6931,7 @@ void __init sched_init(void)
 	init_sched_fair_class();
 
 	scheduler_running = 1;
+	set_cpu_rq_start_time();
 }
 
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
-- 
1.8.3.1

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Mike Galbraith]

CC maintainer improves patch aerodynamics.

On Mon, 2014-03-17 at 19:05 -0500, minyard@acm.org wrote: 
> From: Corey Minyard <cminyard@mvista.com>
> 
> If the sched_clock time starts at a large value, the kernel will spin
> in sched_avg_update for a long time while rq->age_stamp catches up
> with rq->clock.
> 
> The comment in kernel/sched/clock.c says that there is no strict promise
> that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
> to avoid this.
> 
> I was seeing long delays on a simulator that didn't start the clock at
> zero.  This might also be an issue on reboots on processors that don't
> re-initialize the timer to zero on reset, and when using kexec.
> 
> Signed-off-by: Corey Minyard <cminyard@mvista.com>
> ---
>  kernel/sched/core.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index b46131e..5be3d4a 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -5037,11 +5037,20 @@ static struct notifier_block migration_notifier = {
>  	.priority = CPU_PRI_MIGRATION,
>  };
>  
> +static void __cpuinit set_cpu_rq_start_time(void)
> +{
> +	int cpu = smp_processor_id();
> +	struct rq *rq = cpu_rq(cpu);
> +	rq->age_stamp = sched_clock_cpu(cpu);
> +}

rq->age_stamp must lag rq->clock.  See scale_rt_power(), and what
happens when it munches magic timewarp mushrooms.

> +
>  static int sched_cpu_active(struct notifier_block *nfb,
>  				      unsigned long action, void *hcpu)
>  {
>  	switch (action & ~CPU_TASKS_FROZEN) {
>  	case CPU_STARTING:
> +		set_cpu_rq_start_time();
> +		/* fall through */
>  	case CPU_DOWN_FAILED:
>  		set_cpu_active((long)hcpu, true);
>  		return NOTIFY_OK;
> @@ -6922,6 +6931,7 @@ void __init sched_init(void)
>  	init_sched_fair_class();
>  
>  	scheduler_running = 1;
> +	set_cpu_rq_start_time();
>  }
>  
>  #ifdef CONFIG_DEBUG_ATOMIC_SLEEP

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Peter Zijlstra]

On Tue, Mar 18, 2014 at 05:06:26AM +0100, Mike Galbraith wrote:
> CC maintainer improves patch aerodynamics.

hehe, for sure. I have very little time to look at lkml these days and
there's a near 100% chance I'll miss anything that doesn't hit the
inbox.

> On Mon, 2014-03-17 at 19:05 -0500, minyard@acm.org wrote: 
> > From: Corey Minyard <cminyard@mvista.com>
> > 
> > If the sched_clock time starts at a large value, the kernel will spin
> > in sched_avg_update for a long time while rq->age_stamp catches up
> > with rq->clock.
> > 
> > The comment in kernel/sched/clock.c says that there is no strict promise
> > that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
> > to avoid this.
> > 
> > I was seeing long delays on a simulator that didn't start the clock at
> > zero.  This might also be an issue on reboots on processors that don't
> > re-initialize the timer to zero on reset, and when using kexec.
> > 
> > Signed-off-by: Corey Minyard <cminyard@mvista.com>
> > ---
> >  kernel/sched/core.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index b46131e..5be3d4a 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -5037,11 +5037,20 @@ static struct notifier_block migration_notifier = {
> >  	.priority = CPU_PRI_MIGRATION,
> >  };
> >  
> > +static void __cpuinit set_cpu_rq_start_time(void)
> > +{
> > +	int cpu = smp_processor_id();
> > +	struct rq *rq = cpu_rq(cpu);
> > +	rq->age_stamp = sched_clock_cpu(cpu);
> > +}
> 
> rq->age_stamp must lag rq->clock.  See scale_rt_power(), and what
> happens when it munches magic timewarp mushrooms.
> 
> > +
> >  static int sched_cpu_active(struct notifier_block *nfb,
> >  				      unsigned long action, void *hcpu)
> >  {
> >  	switch (action & ~CPU_TASKS_FROZEN) {
> >  	case CPU_STARTING:
> > +		set_cpu_rq_start_time();
> > +		/* fall through */
> >  	case CPU_DOWN_FAILED:
> >  		set_cpu_active((long)hcpu, true);
> >  		return NOTIFY_OK;
> > @@ -6922,6 +6931,7 @@ void __init sched_init(void)
> >  	init_sched_fair_class();
> >  
> >  	scheduler_running = 1;
> > +	set_cpu_rq_start_time();

I would put it one line up; that scheduler_running=1 is the last thing
we should do.

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Mike Galbraith]

On Tue, 2014-03-18 at 09:56 +0100, Peter Zijlstra wrote: 
> On Tue, Mar 18, 2014 at 05:06:26AM +0100, Mike Galbraith wrote:

> > > +static void __cpuinit set_cpu_rq_start_time(void)
> > > +{
> > > +	int cpu = smp_processor_id();
> > > +	struct rq *rq = cpu_rq(cpu);
> > > +	rq->age_stamp = sched_clock_cpu(cpu);
> > > +}
> > 
> > rq->age_stamp must lag rq->clock.  See scale_rt_power(), and what
> > happens when it munches magic timewarp mushrooms.
> > 
> > > +
> > >  static int sched_cpu_active(struct notifier_block *nfb,
> > >  				      unsigned long action, void *hcpu)
> > >  {
> > >  	switch (action & ~CPU_TASKS_FROZEN) {
> > >  	case CPU_STARTING:
> > > +		set_cpu_rq_start_time();
> > > +		/* fall through */
> > >  	case CPU_DOWN_FAILED:
> > >  		set_cpu_active((long)hcpu, true);
> > >  		return NOTIFY_OK;
> > > @@ -6922,6 +6931,7 @@ void __init sched_init(void)
> > >  	init_sched_fair_class();
> > >  
> > >  	scheduler_running = 1;
> > > +	set_cpu_rq_start_time();
> 
> I would put it one line up; that scheduler_running=1 is the last thing
> we should do.

And set clock and age, dazed scale_rt_power() is butt ugly.

-Mike

[PATCH] sched: Initialize rq->age_stamp on processor start

[minyard]

From: Corey Minyard <cminyard@mvista.com>

If the sched_clock time starts at a large value, the kernel will spin
in sched_avg_update for a long time while rq->age_stamp catches up
with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise
that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
to avoid this.

I was seeing long delays on a simulator that didn't start the clock at
zero.  This might also be an issue on reboots on processors that don't
re-initialize the timer to zero on reset, and when using kexec.

Signed-off-by: Corey Minyard <cminyard@mvista.com>
---
 kernel/sched/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 268a45e..6e9570b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5039,11 +5039,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
 	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		/* fall through */
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6923,6 +6932,7 @@ void __init sched_init(void)
 	init_sched_fair_class();
 
 	scheduler_running = 1;
+	set_cpu_rq_start_time();
 }
 
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
-- 
1.8.3.1

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On Tue, May 06, 2014 at 02:50:57PM -0500, minyard@acm.org wrote:
> From: Corey Minyard <cminyard@mvista.com>
> 
> If the sched_clock time starts at a large value, the kernel will spin
> in sched_avg_update for a long time while rq->age_stamp catches up
> with rq->clock.
> 
> The comment in kernel/sched/clock.c says that there is no strict promise
> that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
> to avoid this.
> 
> I was seeing long delays on a simulator that didn't start the clock at
> zero.  This might also be an issue on reboots on processors that don't
> re-initialize the timer to zero on reset, and when using kexec.
> 
> Signed-off-by: Corey Minyard <cminyard@mvista.com>

This patch made my cris-defconfig build fail.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Corey Minyard]

On 05/08/2014 11:13 AM, Peter Zijlstra wrote:
> On Tue, May 06, 2014 at 02:50:57PM -0500, minyard@acm.org wrote:
>> From: Corey Minyard <cminyard@mvista.com>
>>
>> If the sched_clock time starts at a large value, the kernel will spin
>> in sched_avg_update for a long time while rq->age_stamp catches up
>> with rq->clock.
>>
>> The comment in kernel/sched/clock.c says that there is no strict promise
>> that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
>> to avoid this.
>>
>> I was seeing long delays on a simulator that didn't start the clock at
>> zero.  This might also be an issue on reboots on processors that don't
>> re-initialize the timer to zero on reset, and when using kexec.
>>
>> Signed-off-by: Corey Minyard <cminyard@mvista.com>
> This patch made my cris-defconfig build fail.
Dang, I missed propagating a change.  Yes, it's broken.  One more time...

-corey

[PATCH] sched: Initialize rq->age_stamp on processor start

[minyard]

From: Corey Minyard <cminyard@mvista.com>

If the sched_clock time starts at a large value, the kernel will spin
in sched_avg_update for a long time while rq->age_stamp catches up
with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise
that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
to avoid this.

I was seeing long delays on a simulator that didn't start the clock at
zero.  This might also be an issue on reboots on processors that don't
re-initialize the timer to zero on reset, and when using kexec.

Signed-off-by: Corey Minyard <cminyard@mvista.com>
---
 kernel/sched/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 268a45e..80725c9 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5039,11 +5039,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
 	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		/* fall through */
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6919,6 +6928,7 @@ void __init sched_init(void)
 	if (cpu_isolated_map == NULL)
 		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 	idle_thread_set_boot_cpu();
+	set_cpu_rq_start_time();
 #endif
 	init_sched_fair_class();
 
-- 
1.8.3.1

Re: [PATCH] sched: Initialize rq->age_stamp on processor start

[Peter Zijlstra]

[-- Attachment #1: Type: text/plain, Size: 739 bytes --]

On Thu, May 08, 2014 at 01:47:39PM -0500, minyard@acm.org wrote:
> From: Corey Minyard <cminyard@mvista.com>
> 
> If the sched_clock time starts at a large value, the kernel will spin
> in sched_avg_update for a long time while rq->age_stamp catches up
> with rq->clock.
> 
> The comment in kernel/sched/clock.c says that there is no strict promise
> that it starts at zero.  So initialize rq->age_stamp when a cpu starts up
> to avoid this.
> 
> I was seeing long delays on a simulator that didn't start the clock at
> zero.  This might also be an issue on reboots on processors that don't
> re-initialize the timer to zero on reset, and when using kexec.
> 
> Signed-off-by: Corey Minyard <cminyard@mvista.com>

Thanks

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]