From: Bjorn Helgaas <helgaas@kernel.org> To: Nitesh Narayan Lal <nitesh@redhat.com>, hch@infradead.org Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, frederic@kernel.org, mtosatti@redhat.com, sassmann@redhat.com, jesse.brandeburg@intel.com, lihong.yang@intel.com, jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com, jlelli@redhat.com, bhelgaas@google.com, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, thomas.lendacky@amd.com, jiri@nvidia.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, lgoncalv@redhat.com Subject: Re: [PATCH v4 4/4] PCI: Limit pci_alloc_irq_vectors() to housekeeping CPUs Date: Mon, 28 Sep 2020 16:59:31 -0500 Message-ID: <20200928215931.GA2499944@bjorn-Precision-5520> (raw) In-Reply-To: <20200928183529.471328-5-nitesh@redhat.com> [to: Christoph in case he has comments, since I think he wrote this code] On Mon, Sep 28, 2020 at 02:35:29PM -0400, Nitesh Narayan Lal wrote: > If we have isolated CPUs dedicated for use by real-time tasks, we try to > move IRQs to housekeeping CPUs from the userspace to reduce latency > overhead on the isolated CPUs. > > If we allocate too many IRQ vectors, moving them all to housekeeping CPUs > may exceed per-CPU vector limits. > > When we have isolated CPUs, limit the number of vectors allocated by > pci_alloc_irq_vectors() to the minimum number required by the driver, or > to one per housekeeping CPU if that is larger. > > Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> > --- > drivers/pci/msi.c | 18 ++++++++++++++++++ > 1 file changed, 18 insertions(+) > > diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c > index 30ae4ffda5c1..8c156867803c 100644 > --- a/drivers/pci/msi.c > +++ b/drivers/pci/msi.c > @@ -23,6 +23,7 @@ > #include <linux/slab.h> > #include <linux/irqdomain.h> > #include <linux/of_irq.h> > +#include <linux/sched/isolation.h> > > #include "pci.h" > > @@ -1191,8 +1192,25 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, > struct irq_affinity *affd) > { > struct irq_affinity msi_default_affd = {0}; > + unsigned int hk_cpus; > int nvecs = -ENOSPC; > > + hk_cpus = housekeeping_num_online_cpus(HK_FLAG_MANAGED_IRQ); > + > + /* > + * If we have isolated CPUs for use by real-time tasks, to keep the > + * latency overhead to a minimum, device-specific IRQ vectors are moved > + * to the housekeeping CPUs from the userspace by changing their > + * affinity mask. Limit the vector usage to keep housekeeping CPUs from > + * running out of IRQ vectors. > + */ > + if (hk_cpus < num_online_cpus()) { > + if (hk_cpus < min_vecs) > + max_vecs = min_vecs; > + else if (hk_cpus < max_vecs) > + max_vecs = hk_cpus; > + } > + > if (flags & PCI_IRQ_AFFINITY) { > if (!affd) > affd = &msi_default_affd; > -- > 2.18.2 >
next prev parent reply index Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-28 18:35 [PATCH v4 0/4] isolation: limit msix vectors " Nitesh Narayan Lal 2020-09-28 18:35 ` [PATCH v4 1/4] sched/isolation: API to get number of " Nitesh Narayan Lal 2020-09-28 18:35 ` [PATCH v4 2/4] sched/isolation: Extend nohz_full to isolate managed IRQs Nitesh Narayan Lal 2020-10-23 13:25 ` Peter Zijlstra 2020-10-23 13:29 ` Frederic Weisbecker 2020-10-23 13:57 ` Nitesh Narayan Lal 2020-10-23 13:45 ` Nitesh Narayan Lal 2020-09-28 18:35 ` [PATCH v4 3/4] i40e: Limit msix vectors to housekeeping CPUs Nitesh Narayan Lal 2020-09-28 18:35 ` [PATCH v4 4/4] PCI: Limit pci_alloc_irq_vectors() " Nitesh Narayan Lal 2020-09-28 21:59 ` Bjorn Helgaas [this message] 2020-09-29 17:46 ` Christoph Hellwig 2020-10-16 12:20 ` Peter Zijlstra 2020-10-18 18:14 ` Nitesh Narayan Lal 2020-10-19 11:11 ` Peter Zijlstra 2020-10-19 14:00 ` Marcelo Tosatti 2020-10-19 14:25 ` Nitesh Narayan Lal 2020-10-20 7:30 ` Peter Zijlstra 2020-10-20 13:00 ` Nitesh Narayan Lal 2020-10-20 13:41 ` Peter Zijlstra 2020-10-20 14:39 ` Nitesh Narayan Lal 2020-10-22 17:47 ` Nitesh Narayan Lal 2020-10-23 8:58 ` Peter Zijlstra 2020-10-23 13:10 ` Nitesh Narayan Lal 2020-10-23 21:00 ` Thomas Gleixner 2020-10-26 13:35 ` Nitesh Narayan Lal 2020-10-26 13:57 ` Thomas Gleixner 2020-10-26 17:30 ` Marcelo Tosatti 2020-10-26 19:00 ` Thomas Gleixner 2020-10-26 19:11 ` Marcelo Tosatti 2020-10-26 19:21 ` Jacob Keller 2020-10-26 20:11 ` Thomas Gleixner 2020-10-26 21:11 ` Jacob Keller 2020-10-26 21:50 ` Thomas Gleixner 2020-10-26 22:13 ` Jakub Kicinski 2020-10-26 22:46 ` Thomas Gleixner 2020-10-26 22:52 ` Jacob Keller 2020-10-26 22:22 ` Nitesh Narayan Lal 2020-10-26 22:49 ` Thomas Gleixner 2020-10-26 23:08 ` Jacob Keller 2020-10-27 14:28 ` Thomas Gleixner 2020-10-27 11:47 ` Marcelo Tosatti 2020-10-27 14:43 ` Thomas Gleixner 2020-10-19 14:21 ` Frederic Weisbecker 2020-10-20 14:16 ` Thomas Gleixner 2020-10-20 16:18 ` Nitesh Narayan Lal 2020-10-20 18:07 ` Thomas Gleixner 2020-10-21 20:25 ` Thomas Gleixner 2020-10-21 21:04 ` Nitesh Narayan Lal 2020-10-22 0:02 ` Jakub Kicinski 2020-10-22 0:27 ` Jacob Keller 2020-10-22 8:28 ` Thomas Gleixner 2020-10-22 12:28 ` Marcelo Tosatti 2020-10-22 22:39 ` Thomas Gleixner 2020-10-01 15:49 ` [PATCH v4 0/4] isolation: limit msix vectors " Frederic Weisbecker 2020-10-08 21:40 ` Nitesh Narayan Lal
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200928215931.GA2499944@bjorn-Precision-5520 \ --to=helgaas@kernel.org \ --cc=bhelgaas@google.com \ --cc=dennis.dalessandro@intel.com \ --cc=frederic@kernel.org \ --cc=hch@infradead.org \ --cc=intel-wired-lan@lists.osuosl.org \ --cc=jacob.e.keller@intel.com \ --cc=jeffrey.t.kirsher@intel.com \ --cc=jesse.brandeburg@intel.com \ --cc=jiri@nvidia.com \ --cc=jlelli@redhat.com \ --cc=juri.lelli@redhat.com \ --cc=lgoncalv@redhat.com \ --cc=lihong.yang@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=mike.marciniszyn@intel.com \ --cc=mingo@redhat.com \ --cc=mtosatti@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=nitesh@redhat.com \ --cc=peterz@infradead.org \ --cc=sassmann@redhat.com \ --cc=thomas.lendacky@amd.com \ --cc=vincent.guittot@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-PCI Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \ linux-pci@vger.kernel.org public-inbox-index linux-pci Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci AGPL code for this site: git clone https://public-inbox.org/public-inbox.git