kernel timers vs network card interrupt

kernel timers vs network card interrupt

[Xinwen - Fu]

Hi, all,
	I'm curious that if a network card interrupt happens at the same
time as the kernel timer expires, what will happen?

	It's said the kernel timer is guaranteed accurate. But if
interrupts are not masked off, the network interrupt also should get
response when a kernel timer expires. So I don't know who will preempt
who.

	Thanks for information!

Xinwen Fu

Re: kernel timers vs network card interrupt

[Richard B. Johnson]

On Wed, 3 Jul 2002, Xinwen - Fu wrote:

> Hi, all,
> 	I'm curious that if a network card interrupt happens at the same
> time as the kernel timer expires, what will happen?
> 
> 	It's said the kernel timer is guaranteed accurate. But if
> interrupts are not masked off, the network interrupt also should get
> response when a kernel timer expires. So I don't know who will preempt
> who.
> 
> 	Thanks for information!
> 
> Xinwen Fu

The highest priority interrupt will get serviced first. It's the timer.
Interrupts are serviced in priority-order. Hardware "remembers" which
ones are pending so none are lost if some driver doesn't do something
stupid.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

                 Windows-2000/Professional isn't.

timer queue is still influenced by network load

[Xinwen - Fu]

Richard,
	I did a few experiments using the example (jiq, I changed jiffies 
to do_gettimeofday() ) from Linux Device
Driver, 2nd version (p196). 

	I have two machines m1 and m2. On m1, I run a timer queue (jiq)
module. Then I download a big file from m1 to m2. The timings are
different between before ftp and during ftp.

----------------------------------------
before ftp
----------------------------------------
time       pid     cpu command
   420590   1       0   0 swapper
   430580   1       0   0 swapper
   440579   1       0   0 swapper
   450579   1       0   0 swapper
   460579   1       0   0 swapper
   470579   1       0   0 swapper
   480579   1       0   0 swapper
   490579   1       0   0 swapper
   500579   1       0   0 swapper

----------------------------------------
during ftp
----------------------------------------
time       pid       cpu command
   370605   1524      0 in.ftpd
   380645   0         0 swapper
   390583   0         0 swapper
   400667   0         0 swapper
   410703   1524      0 in.ftpd
   420679   0         0 swapper
   430634   0         0 swapper
   440624   0         0 swapper
   450648   0         0 swapper
	


It shows that
timer queue is still not accurate. So
the conclusion of " you're guaranteed that the queue will run at the next
clock tick, thus eliminating latency caused by system load" is WRONG!!!

	What is your opinion?

Xinwen Fu


On Wed, 3 Jul 2002, Richard B. Johnson wrote:

> On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> 
> > Hi, all,
> > 	I'm curious that if a network card interrupt happens at the same
> > time as the kernel timer expires, what will happen?
> > 
> > 	It's said the kernel timer is guaranteed accurate. But if
> > interrupts are not masked off, the network interrupt also should get
> > response when a kernel timer expires. So I don't know who will preempt
> > who.
> > 
> > 	Thanks for information!
> > 
> > Xinwen Fu
> 
> The highest priority interrupt will get serviced first. It's the timer.
> Interrupts are serviced in priority-order. Hardware "remembers" which
> ones are pending so none are lost if some driver doesn't do something
> stupid.
> 
> Cheers,
> Dick Johnson
> 
> Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
> 
>                  Windows-2000/Professional isn't.
> 
> 

Re: kernel timers vs network card interrupt

[george anzinger]

"Richard B. Johnson" wrote:
> 
> On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> 
> > Hi, all,
> >       I'm curious that if a network card interrupt happens at the same
> > time as the kernel timer expires, what will happen?
> >
> >       It's said the kernel timer is guaranteed accurate. But if
> > interrupts are not masked off, the network interrupt also should get
> > response when a kernel timer expires. So I don't know who will preempt
> > who.
> >
> >       Thanks for information!
> >
> > Xinwen Fu
> 
> The highest priority interrupt will get serviced first. It's the timer.
> Interrupts are serviced in priority-order. Hardware "remembers" which
> ones are pending so none are lost if some driver doesn't do something
> stupid.

That is true as far as it goes, HOWEVER, timers are serviced
by bottom half code which is run at the end of the
interrupt, WITH THE INTERRUPT SYSTEM ON.  Therefore, timer
servicing can be interrupted by an interrupt and thus be
delayed.
 
-- 
George Anzinger   george@mvista.com
High-res-timers: 
http://sourceforge.net/projects/high-res-timers/
Real time sched:  http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

Re: kernel timers vs network card interrupt

[Xinwen - Fu]

        In fact I want a timer (either in user level or kernel level).
This timer (hope it is a periodic timer) must expire at the interval that
I specify. For example, if I
want that the timer expires at 10ms, it should never be fired at
10.0000000001ms or
9.9999999999ms. That is the key part that I want!

        Have an idea?

        Thanks!

Xinwen Fu


On Thu, 4 Jul 2002, george anzinger wrote:

> "Richard B. Johnson" wrote:
> > 
> > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> > 
> > > Hi, all,
> > >       I'm curious that if a network card interrupt happens at the same
> > > time as the kernel timer expires, what will happen?
> > >
> > >       It's said the kernel timer is guaranteed accurate. But if
> > > interrupts are not masked off, the network interrupt also should get
> > > response when a kernel timer expires. So I don't know who will preempt
> > > who.
> > >
> > >       Thanks for information!
> > >
> > > Xinwen Fu
> > 
> > The highest priority interrupt will get serviced first. It's the timer.
> > Interrupts are serviced in priority-order. Hardware "remembers" which
> > ones are pending so none are lost if some driver doesn't do something
> > stupid.
> 
> That is true as far as it goes, HOWEVER, timers are serviced
> by bottom half code which is run at the end of the
> interrupt, WITH THE INTERRUPT SYSTEM ON.  Therefore, timer
> servicing can be interrupted by an interrupt and thus be
> delayed.
>  
> -- 
> George Anzinger   george@mvista.com
> High-res-timers: 
> http://sourceforge.net/projects/high-res-timers/
> Real time sched:  http://sourceforge.net/projects/rtsched/
> Preemption patch:
> http://www.kernel.org/pub/linux/kernel/people/rml
> 

Re: kernel timers vs network card interrupt

[Mark Hahn]

>         In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I

this sounds *perfect* for /dev/rtc, assuming you can deal with 
its fixed set of periodic rates.

Re: kernel timers vs network card interrupt

[george anzinger]

Xinwen - Fu wrote:
> 
>         In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I
> want that the timer expires at 10ms, it should never be fired at
> 10.0000000001ms or
> 9.9999999999ms. That is the key part that I want!

10 nines!  Lots of luck.  You need to spend a LOT more money
than I have.  Cesium clocks may be able to do this, but not
computers...

But first, please define "fire".  If you mean that the
interrupt is generated at this rate, well we can do maybe  4
or 5 nines.  If, on the other hand you mean "your timer
function gets cpu cycles", I don't think you will find a
machine that can do much better than one or 2 nines.  Even
if the timer is the only interrupt, you still have interrupt
off times and cache indeterminism to contend with.

If the idea is to to "tickle" some hardware with this
signal, you will do better to not involve a computer in the
link.

The utime project had some software that would schedule a
timer tick early and then loop reading the TSC until the
"exact" time.  This still has the problems of interrupts and
cache misses, but it is probably the only way to approach
what you want.  Nothing magic, you just figure the worst
case latency and set your timer to expire early enough to be
ahead of the appointed time.  Then you loop on the TSC
waiting for your exact time.

-g
> 
>         Have an idea?
> 
>         Thanks!
> 
> Xinwen Fu
> 
> On Thu, 4 Jul 2002, george anzinger wrote:
> 
> > "Richard B. Johnson" wrote:
> > >
> > > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> > >
> > > > Hi, all,
> > > >       I'm curious that if a network card interrupt happens at the same
> > > > time as the kernel timer expires, what will happen?
> > > >
> > > >       It's said the kernel timer is guaranteed accurate. But if
> > > > interrupts are not masked off, the network interrupt also should get
> > > > response when a kernel timer expires. So I don't know who will preempt
> > > > who.
> > > >
> > > >       Thanks for information!
> > > >
> > > > Xinwen Fu
> > >
> > > The highest priority interrupt will get serviced first. It's the timer.
> > > Interrupts are serviced in priority-order. Hardware "remembers" which
> > > ones are pending so none are lost if some driver doesn't do something
> > > stupid.
> >
> > That is true as far as it goes, HOWEVER, timers are serviced
> > by bottom half code which is run at the end of the
> > interrupt, WITH THE INTERRUPT SYSTEM ON.  Therefore, timer
> > servicing can be interrupted by an interrupt and thus be
> > delayed.
> >
> > --
> > George Anzinger   george@mvista.com
> > High-res-timers:
> > http://sourceforge.net/projects/high-res-timers/
> > Real time sched:  http://sourceforge.net/projects/rtsched/
> > Preemption patch:
> > http://www.kernel.org/pub/linux/kernel/people/rml
> >

-- 
George Anzinger   george@mvista.com
High-res-timers: 
http://sourceforge.net/projects/high-res-timers/
Real time sched:  http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

Re: kernel timers vs network card interrupt

[Xinwen - Fu]

Hi, 
	Do you know if there are existing tools to test the network driver
latency (between time to start the interrupt handler (irq.c) and time to
start tasklet (bh))?

	Thanks!
Xinwen Fu


On Thu, 4 Jul 2002, george anzinger wrote:

> Xinwen - Fu wrote:
> > 
> >         In fact I want a timer (either in user level or kernel level).
> > This timer (hope it is a periodic timer) must expire at the interval that
> > I specify. For example, if I
> > want that the timer expires at 10ms, it should never be fired at
> > 10.0000000001ms or
> > 9.9999999999ms. That is the key part that I want!
> 
> 10 nines!  Lots of luck.  You need to spend a LOT more money
> than I have.  Cesium clocks may be able to do this, but not
> computers...
> 
> But first, please define "fire".  If you mean that the
> interrupt is generated at this rate, well we can do maybe  4
> or 5 nines.  If, on the other hand you mean "your timer
> function gets cpu cycles", I don't think you will find a
> machine that can do much better than one or 2 nines.  Even
> if the timer is the only interrupt, you still have interrupt
> off times and cache indeterminism to contend with.
> 
> If the idea is to to "tickle" some hardware with this
> signal, you will do better to not involve a computer in the
> link.
> 
> The utime project had some software that would schedule a
> timer tick early and then loop reading the TSC until the
> "exact" time.  This still has the problems of interrupts and
> cache misses, but it is probably the only way to approach
> what you want.  Nothing magic, you just figure the worst
> case latency and set your timer to expire early enough to be
> ahead of the appointed time.  Then you loop on the TSC
> waiting for your exact time.
> 
> -g
> > 
> >         Have an idea?
> > 
> >         Thanks!
> > 
> > Xinwen Fu
> > 
> > On Thu, 4 Jul 2002, george anzinger wrote:
> > 
> > > "Richard B. Johnson" wrote:
> > > >
> > > > On Wed, 3 Jul 2002, Xinwen - Fu wrote:
> > > >
> > > > > Hi, all,
> > > > >       I'm curious that if a network card interrupt happens at the same
> > > > > time as the kernel timer expires, what will happen?
> > > > >
> > > > >       It's said the kernel timer is guaranteed accurate. But if
> > > > > interrupts are not masked off, the network interrupt also should get
> > > > > response when a kernel timer expires. So I don't know who will preempt
> > > > > who.
> > > > >
> > > > >       Thanks for information!
> > > > >
> > > > > Xinwen Fu
> > > >
> > > > The highest priority interrupt will get serviced first. It's the timer.
> > > > Interrupts are serviced in priority-order. Hardware "remembers" which
> > > > ones are pending so none are lost if some driver doesn't do something
> > > > stupid.
> > >
> > > That is true as far as it goes, HOWEVER, timers are serviced
> > > by bottom half code which is run at the end of the
> > > interrupt, WITH THE INTERRUPT SYSTEM ON.  Therefore, timer
> > > servicing can be interrupted by an interrupt and thus be
> > > delayed.
> > >
> > > --
> > > George Anzinger   george@mvista.com
> > > High-res-timers:
> > > http://sourceforge.net/projects/high-res-timers/
> > > Real time sched:  http://sourceforge.net/projects/rtsched/
> > > Preemption patch:
> > > http://www.kernel.org/pub/linux/kernel/people/rml
> > >
> 
> -- 
> George Anzinger   george@mvista.com
> High-res-timers: 
> http://sourceforge.net/projects/high-res-timers/
> Real time sched:  http://sourceforge.net/projects/rtsched/
> Preemption patch:
> http://www.kernel.org/pub/linux/kernel/people/rml
> 

Re: kernel timers vs network card interrupt

[Bill Davidsen]

On Thu, 4 Jul 2002, Xinwen - Fu wrote:

>         In fact I want a timer (either in user level or kernel level).
> This timer (hope it is a periodic timer) must expire at the interval that
> I specify. For example, if I
> want that the timer expires at 10ms, it should never be fired at
> 10.0000000001ms or
> 9.9999999999ms. That is the key part that I want!

If you find a way to do picosecond latency on a PC let us know. I (a)
don't believe you need any such thing, and (b) think this is a troll. Or
is the femtosec? 

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: timer queue is still influenced by network load

[Richard B. Johnson]

On Wed, 3 Jul 2002, Xinwen - Fu wrote:

> Richard,
> 	I did a few experiments using the example (jiq, I changed jiffies 
> to do_gettimeofday() ) from Linux Device
> Driver, 2nd version (p196). 
> 
> 	I have two machines m1 and m2. On m1, I run a timer queue (jiq)
> module. Then I download a big file from m1 to m2. The timings are
> different between before ftp and during ftp.
> 

> 
> 
> It shows that
> timer queue is still not accurate. So
> the conclusion of " you're guaranteed that the queue will run at the next
> clock tick, thus eliminating latency caused by system load" is WRONG!!!
> 
> 	What is your opinion?

Well you are guaranteed that it will run. You just don't know how fast
it will run. The bottom-half code run off the timer-queue is run with
the interrupts enabled. It can get interrupted and it may be interrupted
by network driver code that loops in ISRs, taking a large percentage
of the CPU cycles.

So, I don't think you are measuring what you think you are measuring.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

                 Windows-2000/Professional isn't.