The 10ms averager in fair.c

The 10ms averager in fair.c

[Uwaysi Bin Kareem]

Hiya. I just had an initial look at fair.c

There seems to be a 10ms averager in there?

You are aware that that means you work on delayed values?

Isn`t that counterintuitive to the principle of sharing?

That means short bursts of cpu-use will be filtered out, and given less  
cpu time.
Starting applications won`t have their cpu-usage before 5ms, which is  
quite a bit on modern machines. Well if you use a linearphase filter, I  
don`t know what kind of averager you use. The best would ofcourse be to  
use a minimalphase gaussian averager. Which might be overkill. Atleast a  
one-pole iir, buf = buf + (-buf + in) * cut)); One pole IIRs also have a  
better frequency response.

When you are working with low-latencies, wouldn`t it be better if such  
things are tuned for target latency. I think few care about latency after  
0.2ms. So say the filter should be set to 0.4ms max.

Why would you want to filter cpu-usage also really?

Peace Be With You.

(please CC me.)

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

Just to illustrate, you have a filter that lasts 10ms, and a cpu process  
that lasts 100uS

Original spike

5 |
4 |
3 |
2 |
1 |
0 |
0ms_______________________10ms
Filtered spike

5
4
3
2
1   .....................
0..                      ..
0ms________________________10ms

Not only is the filtered spike, much lower, but it lasts long beyond the  
100uS spike. (10ms). Why would that be used in something that should  
represent cpu-usage?

Peace Be With You.


On Sun, 30 Sep 2012 13:44:14 +0200, Uwaysi Bin Kareem  
<uwaysi.bin.kareem@paradoxuncreated.com> wrote:

> Hiya. I just had an initial look at fair.c
>
> There seems to be a 10ms averager in there?
>
> You are aware that that means you work on delayed values?
>
> Isn`t that counterintuitive to the principle of sharing?
>
> That means short bursts of cpu-use will be filtered out, and given less  
> cpu time.
> Starting applications won`t have their cpu-usage before 5ms, which is  
> quite a bit on modern machines. Well if you use a linearphase filter, I  
> don`t know what kind of averager you use. The best would ofcourse be to  
> use a minimalphase gaussian averager. Which might be overkill. Atleast a  
> one-pole iir, buf = buf + (-buf + in) * cut)); One pole IIRs also have a  
> better frequency response.
>
> When you are working with low-latencies, wouldn`t it be better if such  
> things are tuned for target latency. I think few care about latency  
> after 0.2ms. So say the filter should be set to 0.4ms max.
>
> Why would you want to filter cpu-usage also really?
>
> Peace Be With You.
>
> (please CC me.)

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

On Mon, 01 Oct 2012 06:06:37 +0200, Mike Galbraith <efault@gmx.de> wrote:

> On Sun, 2012-09-30 at 13:44 +0200, Uwaysi Bin Kareem wrote:
>> Hiya. I just had an initial look at fair.c
>>
>> There seems to be a 10ms averager in there?
>>
>> You are aware that that means you work on delayed values?
>>
>> Isn`t that counterintuitive to the principle of sharing?
>
> Not if you want to be able to use lots of groups, and still do something
> other than in-kernel arithmetic.
>
> -Mike
>

"Use lots of groups"? I don`t even see the point with that. Currently  
doesn`t cfs manipulate nice levels? If you set constant nice levels, and  
remove the filter, things will work more as expected. High nice value,  
should be short slice. That is your "group", for instance "low priority  
stuff".

That a filter filters out the initial cpu spike, only to starve it and  
elevate the priority later, is silly. That is delayed execution.

Now I haven`t looked that close at the whole scheduler yet, but no.. I  
can`t possibly think what a filter does in there, that smears at 100uS  
spike, over 10ms.

Btw, I did set it to 1ns, and it only improved things. So that it should  
have some function seems odd to me.

Are you sure this isn`t just a design-philosophy that was done, without  
much knowledge of filters?

Peace Be With You.

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

This is just too much code for me to do a quick patch on. It really needs  
to be evaulated with concerns to what an optimal scheduler is. That would  
be to operate on actual system load ofcourse, not a filtered system load  
that doesn`t represent what is actually happening on a computer.

What you can do for the time being is just set it to 1nS. If that doesn`t  
negatively impact anything, then you know it is bogus.

Peace Be With You.

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

Ok, so you don`t know. Well, then it looks like what I said, to me. What  
is a 100us load, that gets filtered by a 10ms filter? Nothing.
Or a very very small value that rises and falls over 10ms.  
Load-distribution based on a spike that happened a long time ago (in  
computing terms) seems very odd. You don`t even know the average before  
5ms.

I am going to look at the code, and see if I can make a patch.

Peace Be With You.

On Tue, 02 Oct 2012 04:50:02 +0200, Mike Galbraith <efault@gmx.de> wrote:

> On Mon, 2012-10-01 at 15:24 +0200, Uwaysi Bin Kareem wrote:
>
>> Are you sure this isn`t just a design-philosophy that was done, without
>> much knowledge of filters?
>
> Are you?  I see "I don't see the point", along with "I haven't looked
> closely".  If you look, perhaps you'll see a better way, and can post
> patches.  That's the strength of opensource.
>
> -Mike

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

On Tue, 02 Oct 2012 11:19:15 +0200, Mike Galbraith <efault@gmx.de> wrote:

> On Tue, 2012-10-02 at 08:56 +0200, Uwaysi Bin Kareem wrote:
>
>> What you can do for the time being is just set it to 1nS. If that  
>> doesn`t
>> negatively impact anything, then you know it is bogus.
>
> I already know that there is negative impact.
>
> -Mike
>

You already know? Then please elaborate, what a 10ms smoother is doing in  
a nanosecond resolution scheduler.

Superficially there is no negative impact with Renoise, doom 3, chromium,  
on a shaved+low/latency/low jitter kernel, core 2 duo desktop. Chromium  
actually got faster. Well it felt faster, and that is what I care about.  
You can probably config differently and get more throughput for servers,  
but that does not equate to throughput when talking about displayed frames  
on screen. So please elaborate what you mean by negative impact.

Peace Be With You.

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

On Tue, 02 Oct 2012 13:22:53 +0200, Mike Galbraith <efault@gmx.de> wrote:

> On Tue, 2012-10-02 at 10:07 +0200, Uwaysi Bin Kareem wrote:
>> On Tue, 02 Oct 2012 11:19:15 +0200, Mike Galbraith <efault@gmx.de>  
>> wrote:
>>
>> > On Tue, 2012-10-02 at 08:56 +0200, Uwaysi Bin Kareem wrote:
>> >
>> >> What you can do for the time being is just set it to 1nS. If that
>> >> doesn`t
>> >> negatively impact anything, then you know it is bogus.
>> >
>> > I already know that there is negative impact.
>> >
>> > -Mike
>> >
>>
>> You already know? Then please elaborate, what a 10ms smoother is doing  
>> in
>> a nanosecond resolution scheduler.
>
> I already told you.
>
> I passed a piece of information along that I thought would be of use to
> you.  That's all.  I don't like your tone, and owe you nothing, so have
> a nice day, and goodbye.
>
> -Mike
>

Lol. Well, I will just have to fix this on my own then. Imagine if you  
have a nanosecond cpu burst, and calculating load on a 10ms smoothed  
version of that. I really thought that would be easy to understand. I am  
very suprised to find it there, and very surprised at your attitude. If  
you had something actual to say about this, you would have done it in the  
first post. "A lot of groups" means nothing.

Peace Be With You.

Re: The 10ms averager in fair.c + granularity

[Uwaysi Bin Kareem]

The 10 ms averager is not the only strange thing. Obviously there are some  
good things in this scheduler, since it performs quite well. But I am not  
criticising the good.

But the documentation makes a distinction between desktop and server with  
the "resolution" parameter.
I tried some values and "echo 4500000000 > sched_min_granularity_ns" which  
would be strongly tuned towards server, only reduced jitter in doom 3.  
Which means it is a good value for desktop.

So that doesn`t seem to cohere with what is stated aswell. Anyway.. One  
needs to get really deep into the code to understand why it is so. So you  
know, I don know. I am going to take a look at it, and there probably  
won`t be any patch anytime soon.

Peace Be With You.

Re: The 10ms averager in fair.c + granularity

[Uwaysi Bin Kareem]

Ok at 100hz, granularity seems to work as expected. Actually 1000hz for  
desktop seems to be a myth. I have less jitter with 100hz. Very nice. I  
think jitter is 99.99% eliminated from doom 3 now.

Peace Be With You!

Doom 3 perfect on linux.

[Uwaysi Bin Kareem]

At 100hz, shaved kernel, for lowest jitter, and maximal performance, and  
the following hardwirings in fair.c

unsigned int sysctl_sched_min_granularity =  1158500ULL;
unsigned int normalized_sysctl_sched_min_granularity = 1158500ULL;
unsigned int __read_mostly sysctl_sched_shares_window = 0UL;

I finally got doom 3 to run perfectly.

I just wanted you to know. Any comments to this email please.

An ubuntu distro-kernel with the same philosophy, can also be downloaded  
 from here

http://paradoxuncreated.com/tmp/linux-headers-3.5.4_3.5.4-10.00.Custom_amd64.deb
http://paradoxuncreated.com/tmp/linux-image-3.5.4_3.5.4-10.00.Custom_amd64.deb

Some more info: (not updated yet with the new kernel -  
http://paradoxuncreated.com/Blog/wordpress/?p=2268
(please disregard the high hz here, reducing hz gained better jitter  
levels than higher hz)

Peace Be With You.

RME Fireface UCX in Classcompliant USB-mode, is not working?

[Uwaysi Bin Kareem]

Hiya, does anyone have RME Fireface UCX in Classcompliant USB-mode,  
working?

Peace Be With You.

Re: The 10ms averager in fair.c + granularity

[el es]

Hello, Uwaysi,

Uwaysi Bin Kareem <uwaysi.bin.kareem <at> paradoxuncreated.com> writes:

> 
> Ok at 100hz, granularity seems to work as expected. Actually 1000hz for  
> desktop seems to be a myth. I have less jitter with 100hz. Very nice. I  
> think jitter is 99.99% eliminated from doom 3 now.
> 
> Peace Be With You!
> 

I think for some real credibility you'd have to come up with a 
(synthetic) benchmark that clearly demonstrates this problem...

/This/ seems to be the /real/ problem here: how do you measure generic
jitter? In hard figures, not just 'theoretically'.

And no, the subjective 'I can feel it' doesn't work, as everybody
has a different perception of 'jitter', especially in opengl....

Maybe try to match the 'filter' avg timing to your gfx screen refresh rate?
(15ms for 60Hz, you get the idea?) and tell us how that works?

Lukasz

Re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

Ok I have gained a bit more information on this now. Apparently, the  
filter is there, for HPC loads to exclude scheduler activity itself from  
the scheduler?

Filtering all the processes for this, seems completely unessecary though.  
Depending on what resolution these filters run at, you have 50 filters  
running at resolution X, only to do that, with 50 processes. Why not just  
replace that with a simple, on off, say cpu usage = 0 for the first 1 ms.  
Scheduler activity probably doesn`t last longer than that, atleast with  
preempt on. And with a filter you will have 10ms activity indication after  
the last input. That should just be truncated, and after 1ms things should  
just work on peak values. (not average).

 From thinking about how this filter would improve anything, that seems  
like the better way to do anything like that.

Peace Be With You.

re: The 10ms averager in fair.c

[Uwaysi Bin Kareem]

I also did a quick hack changing some of those values, giving  
non-interrputed audiostream with audioapp alone, at 0.7ms. (on a core2duo  
@ 2.5ghz)
That is actually better than BFS.

Peace Be With You.