linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nitesh Narayan Lal <nitesh@redhat.com>
To: linux-kernel@vger.kernel.org, frederic@kernel.org,
	mtosatti@redhat.com, sassmann@redhat.com,
	jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com,
	jlelli@redhat.com
Subject: [Patch v1] i40e: limit the msix vectors based on housekeeping CPUs
Date: Mon, 15 Jun 2020 16:21:25 -0400	[thread overview]
Message-ID: <20200615202125.27831-2-nitesh@redhat.com> (raw)
In-Reply-To: <20200615202125.27831-1-nitesh@redhat.com>

In a realtime environment, it is essential to isolate
unwanted IRQs from isolated CPUs to prevent latency overheads.
Creating MSIX vectors only based on the online CPUs could lead
to a potential issue on an RT setup that has several isolated
CPUs but a very few housekeeping CPUs. This is because in these
kinds of setups an attempt to move the IRQs to the limited
housekeeping CPUs from isolated CPUs might fail due to the per
CPU vector limit. This could eventually result in latency spikes
because of the IRQ threads that we fail to move from isolated
CPUs. This patch prevents i40e to add vectors only based on
available online CPUs by using housekeeping_cpumask() to derive
the number of available housekeeping CPUs.

Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
---
 drivers/net/ethernet/intel/i40e/i40e_main.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 5d807c8004f8..9691bececb86 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -5,6 +5,7 @@
 #include <linux/of_net.h>
 #include <linux/pci.h>
 #include <linux/bpf.h>
+#include <linux/sched/isolation.h>
 
 /* Local includes */
 #include "i40e.h"
@@ -10933,11 +10934,13 @@ static int i40e_reserve_msix_vectors(struct i40e_pf *pf, int vectors)
 static int i40e_init_msix(struct i40e_pf *pf)
 {
 	struct i40e_hw *hw = &pf->hw;
+	const struct cpumask *mask;
 	int cpus, extra_vectors;
 	int vectors_left;
 	int v_budget, i;
 	int v_actual;
 	int iwarp_requested = 0;
+	int hk_flags;
 
 	if (!(pf->flags & I40E_FLAG_MSIX_ENABLED))
 		return -ENODEV;
@@ -10968,12 +10971,15 @@ static int i40e_init_msix(struct i40e_pf *pf)
 
 	/* reserve some vectors for the main PF traffic queues. Initially we
 	 * only reserve at most 50% of the available vectors, in the case that
-	 * the number of online CPUs is large. This ensures that we can enable
-	 * extra features as well. Once we've enabled the other features, we
-	 * will use any remaining vectors to reach as close as we can to the
-	 * number of online CPUs.
+	 * the number of online (housekeeping) CPUs is large. This ensures that
+	 * we can enable extra features as well. Once we've enabled the other
+	 * features, we will use any remaining vectors to reach as close as we
+	 * can to the number of online (housekeeping) CPUs.
 	 */
-	cpus = num_online_cpus();
+	hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
+	mask = housekeeping_cpumask(hk_flags);
+	cpus = cpumask_weight(mask);
+
 	pf->num_lan_msix = min_t(int, cpus, vectors_left / 2);
 	vectors_left -= pf->num_lan_msix;
 
-- 
2.18.4


  reply	other threads:[~2020-06-15 20:21 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-15 20:21 [PATCH v1 0/1] limit the i40e " Nitesh Narayan Lal
2020-06-15 20:21 ` Nitesh Narayan Lal [this message]
2020-06-15 20:48   ` [Patch v1] i40e: limit the " Keller, Jacob E
2020-06-15 20:55     ` Nitesh Narayan Lal
2020-06-16  8:03   ` Christoph Hellwig
2020-06-16 17:29     ` Nitesh Narayan Lal
2020-06-26 20:11     ` Nitesh Narayan Lal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200615202125.27831-2-nitesh@redhat.com \
    --to=nitesh@redhat.com \
    --cc=frederic@kernel.org \
    --cc=jacob.e.keller@intel.com \
    --cc=jeffrey.t.kirsher@intel.com \
    --cc=jlelli@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mtosatti@redhat.com \
    --cc=sassmann@redhat.com \
    --subject='Re: [Patch v1] i40e: limit the msix vectors based on housekeeping CPUs' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).