Re: Virtio-scsi multiqueue irq affinity

From: xuyihang <xuyihang@huawei.com>
To: Ming Lei <ming.lei@redhat.com>, Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Xu <peterx@redhat.com>, Christoph Hellwig <hch@lst.de>,
	Jason Wang <jasowang@redhat.com>,
	Luiz Capitulino <lcapitulino@redhat.com>,
	"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>, <minlei@redhat.com>,
	<liaochang1@huawei.com>
Subject: Re: Virtio-scsi multiqueue irq affinity
Date: Sat, 8 May 2021 15:52:26 +0800	[thread overview]
Message-ID: <0f6c8a5f-ad33-1199-f313-53fe9187a672@huawei.com> (raw)
In-Reply-To: <20190325095011.GA23225@ming.t460p>

在 2019/3/25 17:50, Ming Lei 写道:
> On Mon, Mar 25, 2019 at 09:53:28AM +0100, Thomas Gleixner wrote:
>> Ming,
>>
>> On Mon, 25 Mar 2019, Ming Lei wrote:
>>> On Mon, Mar 25, 2019 at 01:02:13PM +0800, Peter Xu wrote:
>>>> One thing I can think of is the real-time scenario where "isolcpus="
>>>> is provided, then logically we should not allow any isolated CPUs to
>>>> be bound to any of the multi-queue IRQs.  Though Ming Lei and I had a
>>> So far, this behaviour is made by user-space.
>>>
>>> >From my understanding, IRQ subsystem doesn't handle "isolcpus=", even
>>> though the Kconfig help doesn't mention irq affinity affect:
>>>
>>>            Make sure that CPUs running critical tasks are not disturbed by
>>>            any source of "noise" such as unbound workqueues, timers, kthreads...
>>>            Unbound jobs get offloaded to housekeeping CPUs. This is driven by
>>>            the "isolcpus=" boot parameter.
>> isolcpus has no effect on the interupts. That's what 'irqaffinity=' is for.
> Indeed.
>
> irq_default_affinity is built from 'irqaffinity=', however, we don't
> consider irq_default_affinity for managed IRQ affinity.
>
> Looks Peter wants to exclude some CPUs from the spread on managed IRQ.

Hi Ming and Thomas,

We are dealing with a scenario which may need to assign a default 
irqaffinity

for managed IRQ.

Assume we have a full CPU usage RT thread running binded to a specific CPU.

In the mean while, interrupt handler registered by a device which is 
ksoftirqd

may never have a chance to run. (And we don't want to use isolate CPU)

There could be a couple way to deal with this problem:

1. Adjust priority of ksoftirqd or RT thread, so the interrupt handler 
could preempt

RT thread. However, I am not sure whether it could have some side 
effects or not.

2. Adjust interrupt CPU affinity or RT thread affinity. But managed IRQ 
seems

design to forbid user from manipulating interrupt affinity.

It seems managed IRQ is coupled with user side application to me.

Would you share your thoughts about this issue please?

Thanks,

Yihang