how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

I'm trying to test the scheduler latency on a powerpc platform.  It appears that 
a realfeel type of program won't work since you can't program /dev/rtc to 
generated interrupts on powerpc.  Is there anything similar which could be done?

Thanks,

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

On Thu, May 08, 2003 at 06:12:12PM -0400, Chris Friesen wrote:
> I'm trying to test the scheduler latency on a powerpc platform.  It appears 
> that a realfeel type of program won't work since you can't program /dev/rtc 
> to generated interrupts on powerpc.  Is there anything similar which could 
> be done?

Why would you want to use an interrupt? Just count jiffies in sched.c


-- wli

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Davide Libenzi]

On Thu, 8 May 2003, William Lee Irwin III wrote:

> On Thu, May 08, 2003 at 06:12:12PM -0400, Chris Friesen wrote:
> > I'm trying to test the scheduler latency on a powerpc platform.  It appears
> > that a realfeel type of program won't work since you can't program /dev/rtc
> > to generated interrupts on powerpc.  Is there anything similar which could
> > be done?
>
> Why would you want to use an interrupt? Just count jiffies in sched.c

I don't know what he does mean for scheduler latency, but if it is the ctx
switch one something like get_cycles() will be better instead of jiffies.



- Davide

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

On Thu, May 08, 2003 at 06:12:12PM -0400, Chris Friesen wrote:
>>> I'm trying to test the scheduler latency on a powerpc platform.  It appears
>>> that a realfeel type of program won't work since you can't program /dev/rtc
>>> to generated interrupts on powerpc.  Is there anything similar which could
>>> be done?

On Thu, 8 May 2003, William Lee Irwin III wrote:
> > Why would you want to use an interrupt? Just count jiffies in sched.c

On Thu, May 08, 2003 at 05:38:23PM -0700, Davide Libenzi wrote:
> I don't know what he does mean for scheduler latency, but if it is the ctx
> switch one something like get_cycles() will be better instead of jiffies.

True, if you're looking for performance tweaks and not pathologies (which
I was) you'll need something that accurate.


-- wli

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Richard B. Johnson]

On Thu, 8 May 2003, William Lee Irwin III wrote:

> On Thu, May 08, 2003 at 06:12:12PM -0400, Chris Friesen wrote:
> >>> I'm trying to test the scheduler latency on a powerpc platform.  It appears
> >>> that a realfeel type of program won't work since you can't program /dev/rtc
> >>> to generated interrupts on powerpc.  Is there anything similar which could
> >>> be done?
>
> On Thu, 8 May 2003, William Lee Irwin III wrote:
> > > Why would you want to use an interrupt? Just count jiffies in sched.c
>
> On Thu, May 08, 2003 at 05:38:23PM -0700, Davide Libenzi wrote:
> > I don't know what he does mean for scheduler latency, but if it is the ctx
> > switch one something like get_cycles() will be better instead of jiffies.
>
> True, if you're looking for performance tweaks and not pathologies (which
> I was) you'll need something that accurate.
>
>
> -- wli
Does it have a printer port like the Intel machines?
If so, set it up to generate interrupts on the 'event' pin
(paper-out, etc.) and have the ISR parrot the status bits
out the printer-port bits.

Start it  up and put function generator on the event bit.
measure the delay beteen that bin and the data data bit(s)
with a 'scope. This tells you the whole story, the total
time necessary for an ISR to actually do something.



Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

Richard B. Johnson wrote:
> On Thu, 8 May 2003, William Lee Irwin III wrote:
>>>>Why would you want to use an interrupt? Just count jiffies in sched.c

I'm trying to get a feel for the maximum time from an interrupt coming in until 
the userspace handler gets notified.  On intel you can program the hardware to 
generate interupts through /dev/rtc.  The powerpc doesn't seem to support this.

Jiffies are not accurate enough, I am expecting max latencies in the 1-2 ms range.

> Does it have a printer port like the Intel machines?

Unfortunately no.  USB/Firewire/Ethernet on the desktop, ethernet/serial for 
compactPCI.

I want to find an additional programmable interrupt source.  It bites that cheap 
PCs have this, and the powerpc doesn't.

Chris



-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Roland Dreier]

    Chris> I want to find an additional programmable interrupt source.
    Chris> It bites that cheap PCs have this, and the powerpc doesn't.

I don't know which PowerPC CPU you are using, but for example the IBM
4xx series (and all Book E processors) have a "fixed interval timer"
interrupt that is currently not used at all by Linux.  (The
"programmable interval timer" is used to increment jiffies)

On something like the Motorola 74xx, you might be able to use the
something like the performance monitor to generate an exception.

Best,
  Roland

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

On Thu, 8 May 2003, William Lee Irwin III wrote:
>>>>> Why would you want to use an interrupt? Just count jiffies in sched.c

On Thu, May 08, 2003 at 11:52:29PM -0400, Chris Friesen wrote:
> I'm trying to get a feel for the maximum time from an interrupt coming in 
> until the userspace handler gets notified.  On intel you can program the 
> hardware to generate interupts through /dev/rtc.  The powerpc doesn't seem 
> to support this.
> Jiffies are not accurate enough, I am expecting max latencies in the 1-2 ms 
> range.
> Unfortunately no.  USB/Firewire/Ethernet on the desktop, ethernet/serial 
> for compactPCI.
> I want to find an additional programmable interrupt source.  It bites that 
> cheap PCs have this, and the powerpc doesn't.

Try the timebase instead.


-- wli

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

Roland Dreier wrote:
>     Chris> I want to find an additional programmable interrupt source.
>     Chris> It bites that cheap PCs have this, and the powerpc doesn't.

> On something like the Motorola 74xx, you might be able to use the
> something like the performance monitor to generate an exception.

That's the chip.  I'll look into that further--thanks for the suggestion.

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

William Lee Irwin III wrote:

> Try the timebase instead.

The timestamp is not hard to get.  The problem is getting a medium-frequency 
(2KHz or so) hardware interrupt to drive the test.

On intel, you can do this by programming /dev/rtc.  This does not work in powerpc.

Chris



-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

William Lee Irwin III wrote:
>> Try the timebase instead.

On Fri, May 09, 2003 at 02:14:13AM -0400, Chris Friesen wrote:
> The timestamp is not hard to get.  The problem is getting a 
> medium-frequency (2KHz or so) hardware interrupt to drive the test.
> On intel, you can do this by programming /dev/rtc.  This does not work in 
> powerpc.

I don't understand why you're obsessed with interrupts. Just run your
load and spray the scheduler latency stats out /proc/


-- wli

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

William Lee Irwin III wrote:

> I don't understand why you're obsessed with interrupts. Just run your
> load and spray the scheduler latency stats out /proc/

I'm obsessed with interrupts because it gives me a higher sampling rate.

I could set up and itimer for a recurring 10ms timeout and see how much extra I 
waited, but then I can only get 100 samples/sec.

With /dev/rtc (on intel) you can get 20x more samples in the same amount of time.

Chris



-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

On Fri, May 09, 2003 at 02:53:00AM -0400, Chris Friesen wrote:
> I'm obsessed with interrupts because it gives me a higher sampling rate.
> I could set up and itimer for a recurring 10ms timeout and see how much 
> extra I waited, but then I can only get 100 samples/sec.
> With /dev/rtc (on intel) you can get 20x more samples in the same amount of 
> time.

Why not just keep track of it in the scheduler? The statistic is well-
defined in terms of things measurable at context switch and wakeup.
Just stamping processes with when they became runnable and when they
were first run with the timebase and/or TSC and/or whatever would seem
to provide the answer you want for T(x). W(x) is slightly more involved
but can be measured properly in the same way.

I think your stats will be more accurate and serve your own (not mine;
I have _zero_ cause to fish this stuff out myself besides curiostiy)
purposes better if measured in the way I'm suggesting. That said, only
your own purposes really matter for this in the end so if I
misunderstand you by all means generate all the interrupts you want.


-- wli

RE: how to measure scheduler latency on powerpc? realfeel doesn

[Giuliano Pochini]

On 08-May-2003 Chris Friesen wrote:
> 
> I'm trying to test the scheduler latency on a powerpc platform.  It appears that 
> a realfeel type of program won't work since you can't program /dev/rtc to 
> generated interrupts on powerpc.  Is there anything similar which could be done?

If you are using a legacy Mac, you can use the serial port. If you have
a sound chip, you can use that one. All archs have a programmable timer, even
if /dev/rtc doesn't suport it. Try to ask in the linuxppc-dev mailing list.


Bye.

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[mikpe]

William Lee Irwin III writes:
 > On Thu, 8 May 2003, William Lee Irwin III wrote:
 > >>>>> Why would you want to use an interrupt? Just count jiffies in sched.c
 > 
 > On Thu, May 08, 2003 at 11:52:29PM -0400, Chris Friesen wrote:
 > > I'm trying to get a feel for the maximum time from an interrupt coming in 
 > > until the userspace handler gets notified.  On intel you can program the 
 > > hardware to generate interupts through /dev/rtc.  The powerpc doesn't seem 
 > > to support this.
 > > Jiffies are not accurate enough, I am expecting max latencies in the 1-2 ms 
 > > range.
 > > Unfortunately no.  USB/Firewire/Ethernet on the desktop, ethernet/serial 
 > > for compactPCI.
 > > I want to find an additional programmable interrupt source.  It bites that 
 > > cheap PCs have this, and the powerpc doesn't.
 > 
 > Try the timebase instead.

On all Moto PPCs I've checked, the time-base runs at 1/4 the bus clock
rather than at the core clock like we're used to on x86.

I believe many PPCs allow you to program one of the performance counters
to count core clocks. Be advised though that many Moto PPCs in the 750-7410
range have an errata that prevents using performance counter interrupts,
so you'd have to run the counter in non-interrupting mode.

This isn't available to user-space (yet), so you'd also have to hack
the kernel to use this facility.

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[paubert]

On Fri, May 09, 2003 at 02:14:13AM -0400, Chris Friesen wrote:
> William Lee Irwin III wrote:
> 
> >Try the timebase instead.
> 
> The timestamp is not hard to get.  The problem is getting a medium-frequency 
> 
> (2KHz or so) hardware interrupt to drive the test.

Reload the decrementer with a smaller value, and read the decrementer value
at the beginning of every timer_interrupt. Timekeeping only depends on the
timebase on PPC (I wrote most of the code), taking additional decrementer
interrupts does not harm. Actually the simplest and most efficient way
to perform a self IPI on PPC is to write 0 to the decrementer (2 machine
instructions). 

Actually, a way to measure the longest time during which interrupts are masked
would be:

	- when disabling interrupts, set decrementer to zero if they
	were previously enabled.
	- read the decrementer on every timer/external interrupt,
	if the value is negative and too large print the value
	and the point at which the interrupts were reenabled.

That's not perfect (it will generate not too many false positives but
quite some overhead, but it should work).

	Gabriel 

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Robert Love]

On Fri, 2003-05-09 at 03:01, William Lee Irwin III wrote:

> Why not just keep track of it in the scheduler? The statistic is well-
> defined in terms of things measurable at context switch and wakeup.

This would measure context switch latency.  Or something.

By definition, scheduling latency is the time from an interrupt which
wakes the task up until the task is actually running.

Historically, it has been measured by things like realfeel or amlat or
whatever which generate interrupts and wake a waiting task up. You then
measure the latency between the interrupt and when the task actually
runs in user-space.

So Chris can then go run this test under varying loads and see how bad
the latency gets.  I understand his question, but (sorry Chris) I have
no idea of the solution on PPC.

	Robert Love

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[William Lee Irwin III]

On Fri, 2003-05-09 at 03:01, William Lee Irwin III wrote:
>> Why not just keep track of it in the scheduler? The statistic is well-
>> defined in terms of things measurable at context switch and wakeup.

On Fri, May 09, 2003 at 12:47:05PM -0400, Robert Love wrote:
> This would measure context switch latency.  Or something.
> By definition, scheduling latency is the time from an interrupt which
> wakes the task up until the task is actually running.
> Historically, it has been measured by things like realfeel or amlat or
> whatever which generate interrupts and wake a waiting task up. You then
> measure the latency between the interrupt and when the task actually
> runs in user-space.
> So Chris can then go run this test under varying loads and see how bad
> the latency gets.  I understand his question, but (sorry Chris) I have
> no idea of the solution on PPC.

Not at all. Just stamp at wakeup and difference when it runs.


-- wli

Re: how to measure scheduler latency on powerpc? realfeel doesn't work due to /dev/rtc issues

[Chris Friesen]

Well, since time was short I didn't want to hack the kernel. What I ended up
doing was setting up an itimer for 10ms intervals and then measuring how much
longer than 10ms I was asleep for.  While this only collects samples at 100Hz,
it will work with the default kernel.

Thanks for the suggestions,

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com