From: "liaochang (A)" <liaochang1@huawei.com>
To: Thomas Gleixner <tglx@linutronix.de>,
xuyihang <xuyihang@huawei.com>, "Ming Lei" <ming.lei@redhat.com>
Cc: Peter Xu <peterx@redhat.com>, Christoph Hellwig <hch@lst.de>,
Jason Wang <jasowang@redhat.com>,
Luiz Capitulino <lcapitulino@redhat.com>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Michael S. Tsirkin" <mst@redhat.com>, <minlei@redhat.com>
Subject: Re: Virtio-scsi multiqueue irq affinity
Date: Mon, 10 May 2021 11:19:43 +0800 [thread overview]
Message-ID: <9903df53-8a84-fe89-7ae0-aac8e6d3f42f@huawei.com> (raw)
In-Reply-To: <87zgx5l8ck.ffs@nanos.tec.linutronix.de>
Hi Thomas,
在 2021/5/8 20:26, Thomas Gleixner 写道:
> Yihang,
>
> On Sat, May 08 2021 at 15:52, xuyihang wrote:
>>
>> We are dealing with a scenario which may need to assign a default
>> irqaffinity for managed IRQ.
>>
>> Assume we have a full CPU usage RT thread running binded to a specific
>> CPU.
>>
>> In the mean while, interrupt handler registered by a device which is
>> ksoftirqd may never have a chance to run. (And we don't want to use
>> isolate CPU)
>
> A device cannot register and interrupt handler in ksoftirqd.
I learn the scenario further after communicate with Yihang offline:
1.We have a machine with 36 CPUs,and assign several RT threads to last two CPUs(CPU-34, CPU-35).
2.I/O device driver create single managed irq, the affinity of which includes CPU-34 and CPU-35.
3.Another regular application launch I/O operation at different CPUs with the ones RT threads use,
then CPU-34/35 will receive hardware interrupt and wakeup ksoftirqd to deal with real I/O stuff.
4.Cause the priority and schedule policy of RT thread overwhlem per-cpu ksoftirqd, it looks like
ksoftirqd has no chance to run at CPU-34/35,which leads to I/O processing can't finish at time,
and application get stuck.
>
>> There could be a couple way to deal with this problem:
>>
>> 1. Adjust priority of ksoftirqd or RT thread, so the interrupt handler
>> could preempt
>>
>> RT thread. However, I am not sure whether it could have some side
>> effects or not.
>>
>> 2. Adjust interrupt CPU affinity or RT thread affinity. But managed IRQ
>> seems design to forbid user from manipulating interrupt affinity.
>>
>> It seems managed IRQ is coupled with user side application to me.
>>
>> Would you share your thoughts about this issue please?
>
> Can you please provide a more detailed description of your system?
>
> - Number of CPUs
>
> - Kernel version
> - Is NOHZ full enabled?
> - Any isolation mechanisms enabled, and if so how are they
> configured (e.g. on the kernel command line)?
>
> - Number of queues in the multiqueue device
>
> - Is the RT thread issuing I/O to the multiqueue device?
>
> Thanks,
>
> tglx
> .
>
BR,
Liao Chang
next prev parent reply other threads:[~2021-05-10 3:19 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-18 6:21 Virtio-scsi multiqueue irq affinity Peter Xu
2019-03-23 17:15 ` Thomas Gleixner
2019-03-25 5:02 ` Peter Xu
2019-03-25 7:06 ` Ming Lei
2019-03-25 8:53 ` Thomas Gleixner
2019-03-25 9:43 ` Peter Xu
2019-03-25 13:27 ` Thomas Gleixner
2019-03-25 9:50 ` Ming Lei
2021-05-08 7:52 ` xuyihang
2021-05-08 12:26 ` Thomas Gleixner
2021-05-10 3:19 ` liaochang (A) [this message]
2021-05-10 7:54 ` Thomas Gleixner
2021-05-18 1:37 ` liaochang (A)
2021-05-10 8:48 ` xuyihang
2021-05-10 19:56 ` Thomas Gleixner
2021-05-11 12:38 ` xuyihang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9903df53-8a84-fe89-7ae0-aac8e6d3f42f@huawei.com \
--to=liaochang1@huawei.com \
--cc=hch@lst.de \
--cc=jasowang@redhat.com \
--cc=lcapitulino@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=minlei@redhat.com \
--cc=mst@redhat.com \
--cc=peterx@redhat.com \
--cc=tglx@linutronix.de \
--cc=xuyihang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).