RE: 2.6.1 and irq balancing

["Nakajima, Jun" <jun.nakajima@intel.com>]

I don't see a major problem there. If you look at eth1, it has the
biggest load with order of difference. Interrupts from disk controllers
cannot be major unless you have a lot of disks (tens, hundreds) given a
time period.

>   16:    2978264          0          0          0   IO-APIC-level
> sym53c8xx
>   22:    7838940          0          0          0   IO-APIC-level
eth0
>   48:     916078          0     125150          0   IO-APIC-level
aic79xx
>   49:    1099375          0          0          0   IO-APIC-level
aic79xx
>   54:   51484241        316   50560879        279   IO-APIC-level
eth1

Timer is a bit tricky because it continuously generates interrupts 1000
per sec, and it looks higher as the results. But given a time period,
the timer interrupts should be nothing. 

>    0:  184932542          0    2592511          0    IO-APIC-edge
timer


>             CPU0       CPU1
>    0: 1066522197 1117196193    IO-APIC-edge  timer
>    1:         42         19    IO-APIC-edge  i8042
>    2:          0          0          XT-PIC  cascade
>    5:   23523428   23510845   IO-APIC-level  TLAN
>    8:          0          4    IO-APIC-edge  rtc
>    9:         15         15   IO-APIC-level  sym53c8xx
>   10:    6874323    6809042   IO-APIC-level  sym53c8xx
>   11:    7545802    7509034   IO-APIC-level  ida0
>   14:          8          2    IO-APIC-edge  ide0
> NMI:          0          0
> LOC: 2183867261 2183867237
> ERR:          0
> MIS:          0

The above is generated by the round-robin interrupt distribution
provided by the chipset. It's very easy to do this on P4P systems
(that's basically the initial Ingo's patch), but the performance tends
to be worse if you do that. In fact, we saw better results on PIII
systems if we used the one in the kernel, i.e. irqbalance, rather than
the one by the chipset (round-robin).

Jun

> -----Original Message-----
> From: Ethan Weinstein [mailto:lists@stinkfoot.org]
> Sent: Monday, January 12, 2004 10:50 PM
> To: Nakajima, Jun
> Cc: Ed Tomlinson; linux-kernel@vger.kernel.org;
piggin@cyberone.com.au;
> Kamble, Nitin A
> Subject: Re: 2.6.1 and irq balancing
> 
> Nakajima, Jun wrote:
> 
> >> Admittedly, the machine's load was not high when I took this
sample.
> >> However, creating a great deal of load does not change these
statistics
> >> at all.  Being that there are patches available for 2.4.x kernels
to
> >> fix this, I don't think this at all by design, but what do I know?
=)
> >>
> 
> > 2.6 kernels don't need a patch to it as far as I understand. Are you
> > saying that with significant amount of load, you did not see any
> > distribution of interrupts? Today's threshold in the kernel is high
> > because we found moving around interrupts frequently rather hurt the
> > cache and thus lower the performance compared to "do nothing". Can
you
> > try to create significant load with your network (eth0 and eh1) and
see
> > what happens?
> >
> > Jun
> 
> Here's the situation two days later, I created some brief periods of
> high load on eth1 and I see we have some change:
> 
> 
>             CPU0       CPU1       CPU2       CPU3
>    0:  184932542          0    2592511          0    IO-APIC-edge
timer
>    1:       1875          0          0          0    IO-APIC-edge
i8042
>    2:          0          0          0          0          XT-PIC
cascade
>    3:    3046103          0          0          0    IO-APIC-edge
serial
>    8:          2          0          0          0    IO-APIC-edge  rtc
>    9:          0          0          0          0   IO-APIC-level
acpi
>   14:         76          0          0          0    IO-APIC-edge
ide0
>   16:    2978264          0          0          0   IO-APIC-level
> sym53c8xx
>   22:    7838940          0          0          0   IO-APIC-level
eth0
>   48:     916078          0     125150          0   IO-APIC-level
aic79xx
>   49:    1099375          0          0          0   IO-APIC-level
aic79xx
>   54:   51484241        316   50560879        279   IO-APIC-level
eth1
> NMI:          0          0          0          0
> LOC:  187530735  187530988  187530981  187530986
> ERR:          0
> MIS:          0
> 
> 
> My argument is (see below).  This is an old 2x pentium2 @400, also
> running 2.6, an old Compaq Proliant to be exact.  This machine
obviously
> has no HT, so why the balanced load?
> 
> 
>             CPU0       CPU1
>    0: 1066522197 1117196193    IO-APIC-edge  timer
>    1:         42         19    IO-APIC-edge  i8042
>    2:          0          0          XT-PIC  cascade
>    5:   23523428   23510845   IO-APIC-level  TLAN
>    8:          0          4    IO-APIC-edge  rtc
>    9:         15         15   IO-APIC-level  sym53c8xx
>   10:    6874323    6809042   IO-APIC-level  sym53c8xx
>   11:    7545802    7509034   IO-APIC-level  ida0
>   14:          8          2    IO-APIC-edge  ide0
> NMI:          0          0
> LOC: 2183867261 2183867237
> ERR:          0
> MIS:          0
> 
> 
> 
> Ethan