[PATCH 1/2] sched/nohz: add debugfs control over sched_tick_max_deferment

[PATCH 1/2] sched/nohz: add debugfs control over sched_tick_max_deferment

[Kevin Hilman]

Allow debugfs override of sched_tick_max_deferment in order to ease
finding/fixing the remaining issues with full nohz.

The value to be written is in jiffies, and -1 means the max deferment
is disabled (scheduler_tick_max_deferment() returns KTIME_MAX.)

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
---
 kernel/sched/core.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5ac63c9..4b1fe3e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2175,6 +2175,8 @@ void scheduler_tick(void)
 }
 
 #ifdef CONFIG_NO_HZ_FULL
+static u32 sched_tick_max_deferment = HZ;
+
 /**
  * scheduler_tick_max_deferment
  *
@@ -2193,13 +2195,25 @@ u64 scheduler_tick_max_deferment(void)
 	struct rq *rq = this_rq();
 	unsigned long next, now = ACCESS_ONCE(jiffies);
 
-	next = rq->last_sched_tick + HZ;
+	if (sched_tick_max_deferment == -1)
+		return KTIME_MAX;
+
+	next = rq->last_sched_tick + sched_tick_max_deferment;
 
 	if (time_before_eq(next, now))
 		return 0;
 
 	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
 }
+
+static __init int sched_nohz_full_init_debug(void)
+{
+	debugfs_create_u32("sched_tick_max_deferment", 0644, NULL,
+			   &sched_tick_max_deferment);
+
+	return 0;
+}
+late_initcall(sched_nohz_full_init_debug);
 #endif
 
 notrace unsigned long get_parent_ip(unsigned long addr)
-- 
1.8.3

[PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Kevin Hilman]

The conversion of the max deferment from usecs to nsecs can easily
overflow on platforms where a long is 32-bits.  To fix, cast the usecs
value to u64 before multiplying by NSECS_PER_USEC.

This was discovered on 32-bit ARM platform when extending the max
deferment value.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 4b1fe3e..3d7c80e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2203,7 +2203,7 @@ u64 scheduler_tick_max_deferment(void)
 	if (time_before_eq(next, now))
 		return 0;
 
-	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
+	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;
 }
 
 static __init int sched_nohz_full_init_debug(void)
-- 
1.8.3

Re: [PATCH 1/2] sched/nohz: add debugfs control over sched_tick_max_deferment

[Kevin Hilman]

Frederic,

On Mon, Sep 16, 2013 at 3:43 PM, Kevin Hilman <khilman@linaro.org> wrote:
> Allow debugfs override of sched_tick_max_deferment in order to ease
> finding/fixing the remaining issues with full nohz.
>
> The value to be written is in jiffies, and -1 means the max deferment
> is disabled (scheduler_tick_max_deferment() returns KTIME_MAX.)
>
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Signed-off-by: Kevin Hilman <khilman@linaro.org>

Any objections to this series for ease of debugging and tracking down
issues with the residual tick?

Kevin

> ---
>  kernel/sched/core.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 5ac63c9..4b1fe3e 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2175,6 +2175,8 @@ void scheduler_tick(void)
>  }
>
>  #ifdef CONFIG_NO_HZ_FULL
> +static u32 sched_tick_max_deferment = HZ;
> +
>  /**
>   * scheduler_tick_max_deferment
>   *
> @@ -2193,13 +2195,25 @@ u64 scheduler_tick_max_deferment(void)
>         struct rq *rq = this_rq();
>         unsigned long next, now = ACCESS_ONCE(jiffies);
>
> -       next = rq->last_sched_tick + HZ;
> +       if (sched_tick_max_deferment == -1)
> +               return KTIME_MAX;
> +
> +       next = rq->last_sched_tick + sched_tick_max_deferment;
>
>         if (time_before_eq(next, now))
>                 return 0;
>
>         return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
>  }
> +
> +static __init int sched_nohz_full_init_debug(void)
> +{
> +       debugfs_create_u32("sched_tick_max_deferment", 0644, NULL,
> +                          &sched_tick_max_deferment);
> +
> +       return 0;
> +}
> +late_initcall(sched_nohz_full_init_debug);
>  #endif
>
>  notrace unsigned long get_parent_ip(unsigned long addr)
> --
> 1.8.3
>

Re: [PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Kevin Hilman]

Frederic Weisbecker <fweisbec@gmail.com> writes:

> On Tue, Dec 17, 2013 at 01:23:08PM -0800, Kevin Hilman wrote:
>> The conversion of the max deferment from usecs to nsecs can easily
>> overflow on platforms where a long is 32-bits.  To fix, cast the usecs
>> value to u64 before multiplying by NSECS_PER_USEC.
>> 
>> This was discovered on 32-bit ARM platform when extending the max
>> deferment value.
>> 
>> Cc: Frederic Weisbecker <fweisbec@gmail.com>
>> Signed-off-by: Kevin Hilman <khilman@linaro.org>
>> ---
>>  kernel/sched/core.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 4b1fe3e69fe4..3d7c80e1c4d9 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -2203,7 +2203,7 @@ u64 scheduler_tick_max_deferment(void)
>>  	if (time_before_eq(next, now))
>>  		return 0;
>>  
>> -	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
>> +	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;
>
> Just to be sure I understand the issue. The problem is that jiffies_to_usecs()
> return an unsigned int which is then multiplied by NSEC_PER_USEC. If the result
> of the mul is too big to be stored in an unsigned int, we overflow and may loose
> some high part of the result. Right?

Correct.

Kevin

Re: [PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Frederic Weisbecker]

On Tue, Dec 17, 2013 at 01:23:08PM -0800, Kevin Hilman wrote:
> The conversion of the max deferment from usecs to nsecs can easily
> overflow on platforms where a long is 32-bits.  To fix, cast the usecs
> value to u64 before multiplying by NSECS_PER_USEC.
> 
> This was discovered on 32-bit ARM platform when extending the max
> deferment value.
> 
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Signed-off-by: Kevin Hilman <khilman@linaro.org>
> ---
>  kernel/sched/core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 4b1fe3e69fe4..3d7c80e1c4d9 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2203,7 +2203,7 @@ u64 scheduler_tick_max_deferment(void)
>  	if (time_before_eq(next, now))
>  		return 0;
>  
> -	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
> +	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;

Just to be sure I understand the issue. The problem is that jiffies_to_usecs()
return an unsigned int which is then multiplied by NSEC_PER_USEC. If the result
of the mul is too big to be stored in an unsigned int, we overflow and may loose
some high part of the result. Right?

>  }
>  
>  static __init int sched_nohz_full_init_debug(void)
> -- 
> 1.8.3
> 

[PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Kevin Hilman]

The conversion of the max deferment from usecs to nsecs can easily
overflow on platforms where a long is 32-bits.  To fix, cast the usecs
value to u64 before multiplying by NSECS_PER_USEC.

This was discovered on 32-bit ARM platform when extending the max
deferment value.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 4b1fe3e69fe4..3d7c80e1c4d9 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2203,7 +2203,7 @@ u64 scheduler_tick_max_deferment(void)
 	if (time_before_eq(next, now))
 		return 0;
 
-	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
+	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;
 }
 
 static __init int sched_nohz_full_init_debug(void)
-- 
1.8.3

Re: [PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Frederic Weisbecker]

On Tue, Jun 18, 2013 at 04:58:29PM -0700, Kevin Hilman wrote:
> The conversion of the max deferment from usecs to nsecs can easily
> overflow on platforms where a long is 32-bits.  To fix, cast the usecs
> value to u64 before multiplying by NSECS_PER_USEC.
> 
> This was discovered on 32-bit ARM platform when extending the max
> deferment value.
> 
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Signed-off-by: Kevin Hilman <khilman@linaro.org>

Right, if we make it tunable we need that patch.

Thanks!

Acked-by: Frederic Weisbecker <fweisbec@gmail.com>

> ---
>  kernel/sched/core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index b5d3f99..b506722 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2765,7 +2765,7 @@ u64 scheduler_tick_max_deferment(void)
>  	if (time_before_eq(next, now))
>  		return 0;
>  
> -	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
> +	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;
>  }
>  #endif
>  
> -- 
> 1.8.3
> 

[PATCH 2/2] sched/nohz: fix overflow error in scheduler_tick_max_deferment()

[Kevin Hilman]

The conversion of the max deferment from usecs to nsecs can easily
overflow on platforms where a long is 32-bits.  To fix, cast the usecs
value to u64 before multiplying by NSECS_PER_USEC.

This was discovered on 32-bit ARM platform when extending the max
deferment value.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
---
 kernel/sched/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b5d3f99..b506722 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2765,7 +2765,7 @@ u64 scheduler_tick_max_deferment(void)
 	if (time_before_eq(next, now))
 		return 0;
 
-	return jiffies_to_usecs(next - now) * NSEC_PER_USEC;
+	return (u64)jiffies_to_usecs(next - now) * NSEC_PER_USEC;
 }
 #endif
 
-- 
1.8.3