From: Arjan van de Ven <arjan@infradead.org>
To: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Mark Lord <lkml@rtr.ca>,
Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@osdl.org>, Ingo Molnar <mingo@elte.hu>,
Linux Kernel <linux-kernel@vger.kernel.org>
Subject: Re: CONFIG_IRQBALANCE for 64-bit x86 ?
Date: Mon, 19 Nov 2007 21:37:27 -0800 [thread overview]
Message-ID: <20071119213727.16d5917b@laptopd505.fenrus.org> (raw)
In-Reply-To: <200711201517.16171.nickpiggin@yahoo.com.au>
On Tue, 20 Nov 2007 15:17:15 +1100
Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> On Tuesday 20 November 2007 15:12, Mark Lord wrote:
> > On 32-bit x86, we have CONFIG_IRQBALANCE available,
> > but not on 64-bit x86. Why not?
because the in-kernel one is actually quite bad.
> > My QuadCore box works very well in 32-bit mode with IRQBALANCE,
> > but responsiveness sucks bigtime when run in 64-bit mode (no
> > IRQBALANCE) during periods of multiple heavy I/O streams (USB flash
> > drives).
please run the userspace irq balancer, see http://www.irqbalance.org
afaik most distros ship that by default anyway.
> > As near as I can tell, when IRQBALANCE is not configured,
> > all I/O device interrupts go to CPU#0.
that depends on your chipset; some chipsets do worse than that.
>
> > I don't think our CPU scheduler takes that into account when
> > assigning tasks to CPUs, so anything sent to CPU0 runs with very
> > high latencies.
> >
> > Or something like that.
> >
> > Why no IRQ_BALANCE in 64-bit mode ?
>
> For that matter, I'd like to know why it has been decided that the
> best place for IRQ balancing is in userspace. It should be in kernel
> IMO, and it would probably allow better power saving, performance,
> fairness, etc. if it were to be integrated with the task balancer as
> well.
actually.... no. IRQ balancing is not a "fast" decision; every time you
move an interrupt around, you end up causing a really a TON of cache
line bounces, and generally really bad performance (esp if you do it
for networking ones, since you destroy the packet reassembly stuff in
the tcp/ip stack).
Instead, what ends up working is if you do high level categories of
interrupt classes and balance within those (so that no 2 networking
irqs are on the same core/package unless you have more nics than cores)
etc. Balancing on a 10 second scale seems to work quite well; no need
to pull that complexity into the kernel....
--
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings,
visit http://www.lesswatts.org
next prev parent reply other threads:[~2007-11-20 5:40 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-11-20 4:12 CONFIG_IRQBALANCE for 64-bit x86 ? Mark Lord
2007-11-20 4:15 ` Ismail Dönmez
2007-11-20 4:17 ` Nick Piggin
2007-11-20 4:29 ` Willy Tarreau
2007-11-20 4:37 ` Adrian Bunk
2007-11-20 5:24 ` Nick Piggin
2007-11-20 5:28 ` H. Peter Anvin
2007-11-20 5:37 ` Arjan van de Ven [this message]
2007-11-20 7:37 ` Nick Piggin
2007-11-20 14:47 ` Arjan van de Ven
2007-11-20 15:43 ` Nick Piggin
2007-11-20 19:07 ` Arjan van de Ven
2007-11-20 20:02 ` Mark Lord
2007-11-20 21:58 ` Arjan van de Ven
2007-11-20 23:17 ` Mark Lord
2007-11-22 7:54 ` Nick Piggin
2007-11-23 13:09 ` Ingo Molnar
2007-11-25 10:03 ` Nick Piggin
2007-11-20 15:47 ` Mark Lord
2007-11-20 15:52 ` Mark Lord
2007-11-20 16:02 ` Arjan van de Ven
2007-11-20 16:10 ` Mark Lord
2007-11-20 18:42 ` Mark Lord
2007-11-20 22:01 ` Ingo Molnar
2007-11-20 23:22 ` Mark Lord
2007-11-20 23:27 ` Ingo Molnar
2007-11-20 23:33 ` H. Peter Anvin
2007-11-20 23:47 ` Ingo Molnar
2007-11-20 23:50 ` H. Peter Anvin
2007-11-21 0:07 ` Ingo Molnar
2007-11-21 0:20 ` H. Peter Anvin
2007-11-21 0:36 ` Ingo Molnar
2007-11-21 0:47 ` H. Peter Anvin
2007-11-21 2:48 ` Jeff Garzik
2007-11-21 2:59 ` H. Peter Anvin
2007-11-20 23:28 ` H. Peter Anvin
2007-11-20 19:17 ` Andi Kleen
2007-11-21 2:22 Walt H
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20071119213727.16d5917b@laptopd505.fenrus.org \
--to=arjan@infradead.org \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lkml@rtr.ca \
--cc=mingo@elte.hu \
--cc=nickpiggin@yahoo.com.au \
--cc=torvalds@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).