linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: Ming Lei <ming.lei@redhat.com>
Cc: Peter Xu <peterx@redhat.com>, Christoph Hellwig <hch@lst.de>,
	Jason Wang <jasowang@redhat.com>,
	Luiz Capitulino <lcapitulino@redhat.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	minlei@redhat.com
Subject: Re: Virtio-scsi multiqueue irq affinity
Date: Mon, 25 Mar 2019 09:53:28 +0100 (CET)	[thread overview]
Message-ID: <alpine.DEB.2.21.1903250948490.1798@nanos.tec.linutronix.de> (raw)
In-Reply-To: <20190325070616.GA9642@ming.t460p>

Ming,

On Mon, 25 Mar 2019, Ming Lei wrote:
> On Mon, Mar 25, 2019 at 01:02:13PM +0800, Peter Xu wrote:
> > One thing I can think of is the real-time scenario where "isolcpus="
> > is provided, then logically we should not allow any isolated CPUs to
> > be bound to any of the multi-queue IRQs.  Though Ming Lei and I had a
> 
> So far, this behaviour is made by user-space.
> 
> >From my understanding, IRQ subsystem doesn't handle "isolcpus=", even
> though the Kconfig help doesn't mention irq affinity affect:
> 
>           Make sure that CPUs running critical tasks are not disturbed by
>           any source of "noise" such as unbound workqueues, timers, kthreads...
>           Unbound jobs get offloaded to housekeeping CPUs. This is driven by
>           the "isolcpus=" boot parameter.

isolcpus has no effect on the interupts. That's what 'irqaffinity=' is for.

> Yeah, some RT application may exclude 'isolcpus=' from some IRQ's
> affinity via /proc/irq interface, and now it becomes not possible any
> more to do that for managed IRQ.
> 
> > discussion offlist before and Ming explained to me that as long as the
> > isolated CPUs do not generate any IO then there will be no IRQ on
> > those isolated (real-time) CPUs at all.  Can we guarantee that?  Now
> 
> It is only guaranteed for 1:1 mapping.
> 
> blk-mq uses managed IRQ's affinity to setup queue mapping, for example:
> 
> 1) single hardware queue
> - this queue's IRQ affinity includes all CPUs, then the hardware queue's
> IRQ is only fired on one specific CPU for IO submitted from any CPU

Right. We can special case that for single HW queue to honor the default
affinity setting. That's not hard to achieve.
 
> 2) multi hardware queue
> - there are N hardware queues
> - for each hardware queue i(i < N), its IRQ's affinity may include N(i) CPUs,
> then IRQ for this hardware queue i is fired on one specific CPU among N(i).

Correct and that's the sane case where it does not matter much, because if
your task on an isolated CPU does I/O then redirecting it through some
other CPU does not make sense. If it doesn't do I/O it wont be affected by
the dormant queue.

Thanks,

	tglx

  reply	other threads:[~2019-03-25  8:53 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-18  6:21 Virtio-scsi multiqueue irq affinity Peter Xu
2019-03-23 17:15 ` Thomas Gleixner
2019-03-25  5:02   ` Peter Xu
2019-03-25  7:06     ` Ming Lei
2019-03-25  8:53       ` Thomas Gleixner [this message]
2019-03-25  9:43         ` Peter Xu
2019-03-25 13:27           ` Thomas Gleixner
2019-03-25  9:50         ` Ming Lei
2021-05-08  7:52           ` xuyihang
2021-05-08 12:26             ` Thomas Gleixner
2021-05-10  3:19               ` liaochang (A)
2021-05-10  7:54                 ` Thomas Gleixner
2021-05-18  1:37                   ` liaochang (A)
2021-05-10  8:48               ` xuyihang
2021-05-10 19:56                 ` Thomas Gleixner
2021-05-11 12:38                   ` xuyihang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.21.1903250948490.1798@nanos.tec.linutronix.de \
    --to=tglx@linutronix.de \
    --cc=hch@lst.de \
    --cc=jasowang@redhat.com \
    --cc=lcapitulino@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=minlei@redhat.com \
    --cc=mst@redhat.com \
    --cc=peterx@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).