linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sumit Saxena <sumit.saxena@broadcom.com>
To: tglx@linutronix.de
Cc: Ming Lei <ming.lei@redhat.com>, hch@lst.de, linux-kernel@vger.kernel.org
Subject: Affinity managed interrupts vs non-managed interrupts
Date: Tue, 28 Aug 2018 12:17:33 +0530	[thread overview]
Message-ID: <453338ab43d43c0bf24acf1aeba95251@mail.gmail.com> (raw)

Hi Thomas,

We are working on next generation MegaRAID product where requirement is-
to allocate additional 16 MSI-x vectors in addition to number of MSI-x
vectors megaraid_sas driver usually allocates.  MegaRAID adapter supports
128 MSI-x vectors.

To explain the requirement and solution, consider that we have 2 socket
system (each socket having 36 logical CPUs). Current driver will allocate
total 72 MSI-x vectors by calling API- pci_alloc_irq_vectors(with flag-
PCI_IRQ_AFFINITY).  All 72 MSI-x vectors will have affinity across NUMA
nodes and interrupts are affinity managed.

If driver calls- pci_alloc_irq_vectors_affinity() with pre_vectors = 16
and, driver can allocate 16 + 72 MSI-x vectors.
All pre_vectors (16) will be mapped to all available online CPUs but
effective affinity of each vector is to CPU 0. Our requirement is to have
pre_vectors 16 reply queues to be mapped to local NUMA node with effective
CPU should be spread within local node cpu mask. Without changing kernel
code, we can achieve this by driver calling pci_enable_msix_range()
(requesting to allocate 16 + 72 MSI-x vectors) instead of
pci_alloc_irq_vectors() API. If we use pci_enable_msix_range(), it also
requires MSI-x to CPU affinity handled by driver and these interrupts will
be non-managed.

Question is-
Is there any restriction or preference of using
pci_alloc_irq_vectors{/_affinity} vs pci_enable_msix_range in low level
driver?
If driver uses non-managed interrupt, all cases are handled correctly
through irqbalancer. Is there any plan in future to migrate to managed
interrupts entirely or it is a choice based call for driver maintainers?

Thanks,
Sumit

             reply	other threads:[~2018-08-28  6:47 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-28  6:47 Sumit Saxena [this message]
     [not found] <eccc46e12890a1d033d9003837012502@mail.gmail.com>
2018-08-29  8:46 ` Affinity managed interrupts vs non-managed interrupts Ming Lei
2018-08-29 10:46   ` Sumit Saxena
2018-08-30 17:15     ` Kashyap Desai
2018-08-31  6:54     ` Ming Lei
2018-08-31  7:50       ` Kashyap Desai
2018-08-31 20:24         ` Thomas Gleixner
2018-08-31 21:49           ` Kashyap Desai
2018-08-31 22:48             ` Thomas Gleixner
2018-08-31 23:37               ` Kashyap Desai
2018-09-02 12:02                 ` Thomas Gleixner
2018-09-03  5:34                   ` Kashyap Desai
2018-09-03 16:28                     ` Thomas Gleixner
2018-09-04 10:29                       ` Kashyap Desai
2018-09-05  5:46                         ` Dou Liyang
2018-09-05  9:45                           ` Kashyap Desai
2018-09-05 10:38                             ` Thomas Gleixner
2018-09-06 10:14                               ` Dou Liyang
2018-09-06 11:46                                 ` Thomas Gleixner
2018-09-11  9:13                                   ` Christoph Hellwig
2018-09-11  9:38                                     ` Dou Liyang
2018-09-11  9:22               ` Christoph Hellwig
2018-09-03  2:13         ` Ming Lei
2018-09-03  6:10           ` Kashyap Desai
2018-09-03  9:21             ` Ming Lei
2018-09-03  9:50               ` Kashyap Desai
2018-09-11  9:21     ` Christoph Hellwig
2018-09-11  9:54       ` Kashyap Desai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=453338ab43d43c0bf24acf1aeba95251@mail.gmail.com \
    --to=sumit.saxena@broadcom.com \
    --cc=hch@lst.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).