Re: Should kirqd work on HT?

From: Jeff Garzik <jgarzik@pobox.com>
To: "Martin J. Bligh" <mbligh@aracnet.com>
Cc: "Pallipadi, Venkatesh" <venkatesh.pallipadi@intel.com>,
	ncunningham@cyclades.com, kwijibo@zianet.com,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: Should kirqd work on HT?
Date: Sun, 20 Feb 2005 20:08:01 -0500	[thread overview]
Message-ID: <42193471.4020402@pobox.com> (raw)
In-Reply-To: <320350000.1108910651@[10.10.2.4]>

Martin J. Bligh wrote:
> --Jeff Garzik <jgarzik@pobox.com> wrote (on Saturday, February 19, 2005 11:30:53 -0500):
> 
> 
>>Pallipadi, Venkatesh wrote:
>>
>>>You are right. Kernel balancer doesn't move around the irqs, unless it
>>>has too many interrupts. The logic is moving around interrupts all the
>>>time will not be good on caches. So, there is a threshold above which
>>>the balancer start moving things around.
>>>
>>>You should see them moving around if you do 'ping -f' or a big 'dd' from
>>>the disk.
>>
>>If kirqd is moving NIC interrupts, it's broken.
>>
>>(and another reason why irqbalanced is preferable)
> 
> 
> Why is it broken to move NIC interrupts? Obviously you don't want to
> rotate them around a lot, but in the interests of fairness to other 
> processes, it seems reasonable to migrate them occasionally (IIRC, kirqd
> rate limits to once a second or something).

This has been explained to you before, search your email archives...

The main problem that has been seen in the field SMP packet ordering, 
but a secondary problem observed is cache misses.  Just NAPI mitigates 
this somewhat (no pun intended).

Overall, kirqd should be avoided except in special situations.  It 
doesn't know about such policy things as network-specific or 
storage-specific irq balancing (and shouldn't).

	Jeff