* [PATCH] sched: fix/optimise calculation of weight-inverse
@ 2011-05-11 16:03 Stephan Bärwolf
2011-05-11 16:20 ` Ingo Molnar
0 siblings, 1 reply; 5+ messages in thread
From: Stephan Bärwolf @ 2011-05-11 16:03 UTC (permalink / raw)
To: Linus Torvalds
Cc: Nikhil Rao, Peter Zijlstra, Mike Galbraith, Nikunj A. Dadhania,
Srivatsa Vaddagiri, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]
If the inverse loadweight should be zero, function "calc_delta_mine"
calculates the inverse of "lw->weight" (in 32bit integer ops).
This calculation is actually a little bit impure (because it is
inverting something around "lw-weight"+1), especially when
"lw->weight" becomes smaller. (This could explain some aritmetical
issues for small shares...)
The correct inverse would be 1/lw->weight multiplied by
"WMULT_CONST" for fixcomma-scaling it into integers.
(So WMULT_CONST/lw->weight ...)
For safety it is also important to check if division by zero
could happen...
The old, impure algorithm took two divisions for inverting lw->weight,
the new, more exact one only takes one and an additional unlikely-if.
Signed-off-by: Stephan Baerwolf <stephan.baerwolf@tu-ilmenau.de>
---
kernel/sched.c | 12 +++++++++---
1 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/kernel/sched.c b/kernel/sched.c
index 312f8b9..bb55996 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -1307,15 +1307,21 @@ calc_delta_mine(unsigned long delta_exec,
unsigned long weight,
{
u64 tmp;
+ tmp = (u64)delta_exec * weight;
+
+ // actually we would have to trap - division by zero - but we stay
and pretend the limit of the operation...
+ if (unlikely(lw->weight == 0)) {
+ if (unlikely(tmp == ((u64)0))) return (unsigned long)0;
+ else return (unsigned long)LONG_MAX;
+ }
+
if (!lw->inv_weight) {
if (BITS_PER_LONG > 32 && unlikely(lw->weight >= WMULT_CONST))
lw->inv_weight = 1;
else
- lw->inv_weight = 1 + (WMULT_CONST-lw->weight/2)
- / (lw->weight+1);
+ lw->inv_weight = WMULT_CONST / lw->weight;
}
- tmp = (u64)delta_exec * weight;
/*
* Check whether we'd overflow the 64-bit multiplication:
*/
--
1.7.3.4
--
Dipl.-Inf. Stephan Bärwolf
Ilmenau University of Technology, Integrated Communication Systems Group
Phone: +49 (0)3677 69 2821, FAX: +49 (0)3677 69 1614
Email: stephan.baerwolf@tu-ilmenau.de,
Web: http://www.tu-ilmenau.de/iks
[-- Attachment #2: 0001-sched-fix-optimise-calculation-of-weight-inverse.patch --]
[-- Type: text/plain, Size: 1981 bytes --]
>From 46d3b89632aee75643b2ef9ea9ccd3085c631211 Mon Sep 17 00:00:00 2001
From: Stephan Baerwolf <stephan.baerwolf@tu-ilmenau.de>
Date: Wed, 11 May 2011 17:36:26 +0200
Subject: [PATCH] sched: fix/optimise calculation of weight-inverse
If the inverse loadweight should be zero, function "calc_delta_mine"
calculates the inverse of "lw->weight" (in 32bit integer ops).
This calculation is actually a little bit impure (because it is
inverting something around "lw-weight"+1), especially when
"lw->weight" becomes smaller. (This could explain some aritmetical
issues for small shares...)
The correct inverse would be 1/lw->weight multiplied by
"WMULT_CONST" for fixcomma-scaling it into integers.
(So WMULT_CONST/lw->weight ...)
For safety it is also important to check if division by zero
could happen...
The old, impure algorithm took two divisions for inverting lw->weight,
the new, more exact one only takes one and an additional unlikely-if.
Signed-off-by: Stephan Baerwolf <stephan.baerwolf@tu-ilmenau.de>
---
kernel/sched.c | 12 +++++++++---
1 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/kernel/sched.c b/kernel/sched.c
index 312f8b9..bb55996 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -1307,15 +1307,21 @@ calc_delta_mine(unsigned long delta_exec, unsigned long weight,
{
u64 tmp;
+ tmp = (u64)delta_exec * weight;
+
+ // actually we would have to trap - division by zero - but we stay and pretend the limit of the operation...
+ if (unlikely(lw->weight == 0)) {
+ if (unlikely(tmp == ((u64)0))) return (unsigned long)0;
+ else return (unsigned long)LONG_MAX;
+ }
+
if (!lw->inv_weight) {
if (BITS_PER_LONG > 32 && unlikely(lw->weight >= WMULT_CONST))
lw->inv_weight = 1;
else
- lw->inv_weight = 1 + (WMULT_CONST-lw->weight/2)
- / (lw->weight+1);
+ lw->inv_weight = WMULT_CONST / lw->weight;
}
- tmp = (u64)delta_exec * weight;
/*
* Check whether we'd overflow the 64-bit multiplication:
*/
--
1.7.3.4
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] sched: fix/optimise calculation of weight-inverse
2011-05-11 16:03 [PATCH] sched: fix/optimise calculation of weight-inverse Stephan Bärwolf
@ 2011-05-11 16:20 ` Ingo Molnar
2011-05-11 16:43 ` Peter Zijlstra
0 siblings, 1 reply; 5+ messages in thread
From: Ingo Molnar @ 2011-05-11 16:20 UTC (permalink / raw)
To: Stephan Bärwolf
Cc: Linus Torvalds, Nikhil Rao, Peter Zijlstra, Mike Galbraith,
Nikunj A. Dadhania, Srivatsa Vaddagiri, linux-kernel
* Stephan Bärwolf <stephan.baerwolf@tu-ilmenau.de> wrote:
> If the inverse loadweight should be zero, function "calc_delta_mine"
> calculates the inverse of "lw->weight" (in 32bit integer ops).
>
> This calculation is actually a little bit impure (because it is
> inverting something around "lw-weight"+1), especially when
> "lw->weight" becomes smaller. (This could explain some aritmetical
> issues for small shares...)
>
> The correct inverse would be 1/lw->weight multiplied by
> "WMULT_CONST" for fixcomma-scaling it into integers.
> (So WMULT_CONST/lw->weight ...)
>
> For safety it is also important to check if division by zero
> could happen...
>
> The old, impure algorithm took two divisions for inverting lw->weight,
> the new, more exact one only takes one and an additional unlikely-if.
>
> Signed-off-by: Stephan Baerwolf <stephan.baerwolf@tu-ilmenau.de>
> ---
> kernel/sched.c | 12 +++++++++---
> 1 files changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched.c b/kernel/sched.c
> index 312f8b9..bb55996 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -1307,15 +1307,21 @@ calc_delta_mine(unsigned long delta_exec,
> unsigned long weight,
> {
> u64 tmp;
>
> + tmp = (u64)delta_exec * weight;
> +
> + // actually we would have to trap - division by zero - but we stay
> and pretend the limit of the operation...
> + if (unlikely(lw->weight == 0)) {
> + if (unlikely(tmp == ((u64)0))) return (unsigned long)0;
> + else return (unsigned long)LONG_MAX;
Can lw->weight ever be zero here? I dont think so - and if it is then getting a
kernel crash there is preferred to hiding it.
Once we do that your patch becomes a lot simpler.
> + }
> +
> if (!lw->inv_weight) {
> if (BITS_PER_LONG > 32 && unlikely(lw->weight >= WMULT_CONST))
> lw->inv_weight = 1;
> else
> - lw->inv_weight = 1 + (WMULT_CONST-lw->weight/2)
> - / (lw->weight+1);
> + lw->inv_weight = WMULT_CONST / lw->weight;
hm, i definitely think there was a rounding reason for that - but apparently
i'm an idiot who does not add comments to non-obvious code! :-)
Peter, do you remember this?
> }
>
> - tmp = (u64)delta_exec * weight;
I agree that moving this multiplication early in the sequence is better for
micro-performance regardless of the lw->weight optimization you do: it can be
executed in parallel.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sched: fix/optimise calculation of weight-inverse
2011-05-11 16:20 ` Ingo Molnar
@ 2011-05-11 16:43 ` Peter Zijlstra
2011-05-11 17:35 ` Stephan Bärwolf
0 siblings, 1 reply; 5+ messages in thread
From: Peter Zijlstra @ 2011-05-11 16:43 UTC (permalink / raw)
To: Ingo Molnar
Cc: Stephan Bärwolf, Linus Torvalds, Nikhil Rao, Mike Galbraith,
Nikunj A. Dadhania, Srivatsa Vaddagiri, linux-kernel
On Wed, 2011-05-11 at 18:20 +0200, Ingo Molnar wrote:
> > - lw->inv_weight = 1 + (WMULT_CONST-lw->weight/2)
> > - / (lw->weight+1);
> > + lw->inv_weight = WMULT_CONST / lw->weight;
>
> hm, i definitely think there was a rounding reason for that - but apparently
> i'm an idiot who does not add comments to non-obvious code! :-)
I suspect I might be the idiot,
> Peter, do you remember this?
I think what we wanted to do was minimize the error:
err = weight - inv*WMULT_CONST
by adding another term. But we could well have simply made a mess of it
instead.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sched: fix/optimise calculation of weight-inverse
2011-05-11 16:43 ` Peter Zijlstra
@ 2011-05-11 17:35 ` Stephan Bärwolf
2011-05-11 19:49 ` Peter Zijlstra
0 siblings, 1 reply; 5+ messages in thread
From: Stephan Bärwolf @ 2011-05-11 17:35 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Ingo Molnar, Linus Torvalds, Nikhil Rao, Mike Galbraith,
Nikunj A. Dadhania, Srivatsa Vaddagiri, linux-kernel
>I think what we wanted to do was minimize the error:
> err = weight - inv*WMULT_CONST
Hi all,
thanks for your fast response.
I think what you mean is:
err = | WMULT_CONST - inv*weight | --> min
right?
But the following table(s) shows the difference in the inverses:
(assuming WMULT_CONST = 2**32 as on x64)
weight oldway
inv_weigth new inv_weight
1 2147483649 (=1+2**31)
4294967295 [=(2**32)-1]
2
1431655766 2147483647
3 (= WEIGHT_IDLEPRIO) 1073741824 1431655765
15 (= nice 19)
268435456 286331153
1024 (=nice 0)
4190212 4194304
...
...
88761 (=nice -20)
48387 48388
----
weight err
oldway err newway
1 2147483647 (=2**31 - 1)
1
2 1431655764
2
3 (= WEIGHT_IDLEPRIO) 1073741824 1
15 (= nice 19) 268435456 1
1024 (=nice 0)
4190208 0
...
...
88761 (=nice -20) 88789
28
Thus the "err" of the old way can become very large (up to about 2^31 ).
And of course the new error increases with increasing weight, it is
still alway
smaller than the oldway err (because oldway inv converts to newway inv)...
regards Stephan
--
Dipl.-Inf. Stephan Bärwolf
Ilmenau University of Technology, Integrated Communication Systems Group
Phone: +49 (0)3677 69 2821, FAX: +49 (0)3677 69 1614
Email: stephan.baerwolf@tu-ilmenau.de,
Web: http://www.tu-ilmenau.de/iks
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] sched: fix/optimise calculation of weight-inverse
2011-05-11 17:35 ` Stephan Bärwolf
@ 2011-05-11 19:49 ` Peter Zijlstra
0 siblings, 0 replies; 5+ messages in thread
From: Peter Zijlstra @ 2011-05-11 19:49 UTC (permalink / raw)
To: Stephan Bärwolf
Cc: Ingo Molnar, Linus Torvalds, Nikhil Rao, Mike Galbraith,
Nikunj A. Dadhania, Srivatsa Vaddagiri, linux-kernel
On Wed, 2011-05-11 at 19:35 +0200, Stephan Bärwolf wrote:
> Thus the "err" of the old way can become very large (up to about 2^31 ).
> And of course the new error increases with increasing weight, it is
> still alway
> smaller than the oldway err (because oldway inv converts to newway inv)...
Your table got white-space mungled, but I'll take your word for it. I'll
give the patch a spin tomorrow (without the weight==0 bits), since by
brain officially gave up for today ;-)
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2011-05-11 19:46 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-11 16:03 [PATCH] sched: fix/optimise calculation of weight-inverse Stephan Bärwolf
2011-05-11 16:20 ` Ingo Molnar
2011-05-11 16:43 ` Peter Zijlstra
2011-05-11 17:35 ` Stephan Bärwolf
2011-05-11 19:49 ` Peter Zijlstra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.