From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S261506AbUAMH5x (ORCPT ); Tue, 13 Jan 2004 02:57:53 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S261575AbUAMH5x (ORCPT ); Tue, 13 Jan 2004 02:57:53 -0500 Received: from fmr05.intel.com ([134.134.136.6]:13482 "EHLO hermes.jf.intel.com") by vger.kernel.org with ESMTP id S261506AbUAMH53 convert rfc822-to-8bit (ORCPT ); Tue, 13 Jan 2004 02:57:29 -0500 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT X-MimeOLE: Produced By Microsoft Exchange V6.0.6487.1 Subject: RE: 2.6.1 and irq balancing Date: Mon, 12 Jan 2004 23:57:10 -0800 Message-ID: <7F740D512C7C1046AB53446D3720017361882E@scsmsx402.sc.intel.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: 2.6.1 and irq balancing Thread-Index: AcPZoYFSZiCpRi35T5yY7iwmQu3rZwABWnEw From: "Nakajima, Jun" To: "Ethan Weinstein" Cc: "Ed Tomlinson" , , , "Kamble, Nitin A" X-OriginalArrivalTime: 13 Jan 2004 07:57:10.0747 (UTC) FILETIME=[D9284AB0:01C3D9AA] Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org I don't see a major problem there. If you look at eth1, it has the biggest load with order of difference. Interrupts from disk controllers cannot be major unless you have a lot of disks (tens, hundreds) given a time period. > 16: 2978264 0 0 0 IO-APIC-level > sym53c8xx > 22: 7838940 0 0 0 IO-APIC-level eth0 > 48: 916078 0 125150 0 IO-APIC-level aic79xx > 49: 1099375 0 0 0 IO-APIC-level aic79xx > 54: 51484241 316 50560879 279 IO-APIC-level eth1 Timer is a bit tricky because it continuously generates interrupts 1000 per sec, and it looks higher as the results. But given a time period, the timer interrupts should be nothing. > 0: 184932542 0 2592511 0 IO-APIC-edge timer > CPU0 CPU1 > 0: 1066522197 1117196193 IO-APIC-edge timer > 1: 42 19 IO-APIC-edge i8042 > 2: 0 0 XT-PIC cascade > 5: 23523428 23510845 IO-APIC-level TLAN > 8: 0 4 IO-APIC-edge rtc > 9: 15 15 IO-APIC-level sym53c8xx > 10: 6874323 6809042 IO-APIC-level sym53c8xx > 11: 7545802 7509034 IO-APIC-level ida0 > 14: 8 2 IO-APIC-edge ide0 > NMI: 0 0 > LOC: 2183867261 2183867237 > ERR: 0 > MIS: 0 The above is generated by the round-robin interrupt distribution provided by the chipset. It's very easy to do this on P4P systems (that's basically the initial Ingo's patch), but the performance tends to be worse if you do that. In fact, we saw better results on PIII systems if we used the one in the kernel, i.e. irqbalance, rather than the one by the chipset (round-robin). Jun > -----Original Message----- > From: Ethan Weinstein [mailto:lists@stinkfoot.org] > Sent: Monday, January 12, 2004 10:50 PM > To: Nakajima, Jun > Cc: Ed Tomlinson; linux-kernel@vger.kernel.org; piggin@cyberone.com.au; > Kamble, Nitin A > Subject: Re: 2.6.1 and irq balancing > > Nakajima, Jun wrote: > > >> Admittedly, the machine's load was not high when I took this sample. > >> However, creating a great deal of load does not change these statistics > >> at all. Being that there are patches available for 2.4.x kernels to > >> fix this, I don't think this at all by design, but what do I know? =) > >> > > > 2.6 kernels don't need a patch to it as far as I understand. Are you > > saying that with significant amount of load, you did not see any > > distribution of interrupts? Today's threshold in the kernel is high > > because we found moving around interrupts frequently rather hurt the > > cache and thus lower the performance compared to "do nothing". Can you > > try to create significant load with your network (eth0 and eh1) and see > > what happens? > > > > Jun > > Here's the situation two days later, I created some brief periods of > high load on eth1 and I see we have some change: > > > CPU0 CPU1 CPU2 CPU3 > 0: 184932542 0 2592511 0 IO-APIC-edge timer > 1: 1875 0 0 0 IO-APIC-edge i8042 > 2: 0 0 0 0 XT-PIC cascade > 3: 3046103 0 0 0 IO-APIC-edge serial > 8: 2 0 0 0 IO-APIC-edge rtc > 9: 0 0 0 0 IO-APIC-level acpi > 14: 76 0 0 0 IO-APIC-edge ide0 > 16: 2978264 0 0 0 IO-APIC-level > sym53c8xx > 22: 7838940 0 0 0 IO-APIC-level eth0 > 48: 916078 0 125150 0 IO-APIC-level aic79xx > 49: 1099375 0 0 0 IO-APIC-level aic79xx > 54: 51484241 316 50560879 279 IO-APIC-level eth1 > NMI: 0 0 0 0 > LOC: 187530735 187530988 187530981 187530986 > ERR: 0 > MIS: 0 > > > My argument is (see below). This is an old 2x pentium2 @400, also > running 2.6, an old Compaq Proliant to be exact. This machine obviously > has no HT, so why the balanced load? > > > CPU0 CPU1 > 0: 1066522197 1117196193 IO-APIC-edge timer > 1: 42 19 IO-APIC-edge i8042 > 2: 0 0 XT-PIC cascade > 5: 23523428 23510845 IO-APIC-level TLAN > 8: 0 4 IO-APIC-edge rtc > 9: 15 15 IO-APIC-level sym53c8xx > 10: 6874323 6809042 IO-APIC-level sym53c8xx > 11: 7545802 7509034 IO-APIC-level ida0 > 14: 8 2 IO-APIC-edge ide0 > NMI: 0 0 > LOC: 2183867261 2183867237 > ERR: 0 > MIS: 0 > > > > Ethan