All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
To: intel-wired-lan@lists.osuosl.org
Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>,
	jeffrey.t.kirsher@intel.com, netdev@vger.kernel.org,
	Bhagath Singh Karunakaran <bhagath@kalycito.com>
Subject: [next-queue PATCH v1] igb: Fix limiting the number of queues to number of cpus
Date: Thu,  4 Apr 2019 14:56:32 -0700	[thread overview]
Message-ID: <20190404215632.9881-1-vinicius.gomes@intel.com> (raw)

We have seen some reports[1] of users complaining that they aren't
able to use some queues when their machines have less than 4 cpus.
This affects some TSN workloads, as different traffic classes are
assigned different queues. The current behavior limits the number of
traffic classes that can be reliably handled.

In practice, what is not working, it returns an invalid parameter
error, in hosts with less than 4 cpus is something like this:

$ tc qdisc replace dev IFACE parent root mqprio	\
     	   num_tc 3 map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2	\
	   queues 1@0 1@1 2@2 hw 0

Because changing the default logic of the allocation of queues could
bring other effects, we propose adding a module parameter so expert
users may override that decision.

[1] https://github.com/jeez/iproute2/issues/1

Reported-by: Bhagath Singh Karunakaran <bhagath@kalycito.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
---

A similar fix should also be needed for igc, even if I don't have the
hardware to test, I can produce a patch, if others are able to test.

I am not totally sure that using a module parameter is the best
solution, so, suggestions are welcome.

 drivers/net/ethernet/intel/igb/igb_main.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 32d61d5a2706..87072d47c305 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -247,6 +247,10 @@ static int debug = -1;
 module_param(debug, int, 0);
 MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
 
+static unsigned int num_queues;
+module_param(num_queues, uint, 0);
+MODULE_PARM_DESC(num_queues, "Allocate at maximum this number of queues (0=num_cpus(), default)");
+
 struct igb_reg_info {
 	u32 ofs;
 	char *name;
@@ -3763,7 +3767,13 @@ static void igb_init_queue_configuration(struct igb_adapter *adapter)
 	u32 max_rss_queues;
 
 	max_rss_queues = igb_get_max_rss_queues(adapter);
-	adapter->rss_queues = min_t(u32, max_rss_queues, num_online_cpus());
+
+	if (num_queues > 0)
+		adapter->rss_queues = min_t(u32, max_rss_queues,
+					    num_queues);
+	else
+		adapter->rss_queues = min_t(u32, max_rss_queues,
+					    num_online_cpus());
 
 	igb_set_flag_queue_pairs(adapter, max_rss_queues);
 }
-- 
2.21.0


WARNING: multiple messages have this Message-ID (diff)
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
To: intel-wired-lan@osuosl.org
Subject: [Intel-wired-lan] [next-queue PATCH v1] igb: Fix limiting the number of queues to number of cpus
Date: Thu,  4 Apr 2019 14:56:32 -0700	[thread overview]
Message-ID: <20190404215632.9881-1-vinicius.gomes@intel.com> (raw)

We have seen some reports[1] of users complaining that they aren't
able to use some queues when their machines have less than 4 cpus.
This affects some TSN workloads, as different traffic classes are
assigned different queues. The current behavior limits the number of
traffic classes that can be reliably handled.

In practice, what is not working, it returns an invalid parameter
error, in hosts with less than 4 cpus is something like this:

$ tc qdisc replace dev IFACE parent root mqprio	\
     	   num_tc 3 map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2	\
	   queues 1 at 0 1 at 1 2 at 2 hw 0

Because changing the default logic of the allocation of queues could
bring other effects, we propose adding a module parameter so expert
users may override that decision.

[1] https://github.com/jeez/iproute2/issues/1

Reported-by: Bhagath Singh Karunakaran <bhagath@kalycito.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
---

A similar fix should also be needed for igc, even if I don't have the
hardware to test, I can produce a patch, if others are able to test.

I am not totally sure that using a module parameter is the best
solution, so, suggestions are welcome.

 drivers/net/ethernet/intel/igb/igb_main.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 32d61d5a2706..87072d47c305 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -247,6 +247,10 @@ static int debug = -1;
 module_param(debug, int, 0);
 MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
 
+static unsigned int num_queues;
+module_param(num_queues, uint, 0);
+MODULE_PARM_DESC(num_queues, "Allocate at maximum this number of queues (0=num_cpus(), default)");
+
 struct igb_reg_info {
 	u32 ofs;
 	char *name;
@@ -3763,7 +3767,13 @@ static void igb_init_queue_configuration(struct igb_adapter *adapter)
 	u32 max_rss_queues;
 
 	max_rss_queues = igb_get_max_rss_queues(adapter);
-	adapter->rss_queues = min_t(u32, max_rss_queues, num_online_cpus());
+
+	if (num_queues > 0)
+		adapter->rss_queues = min_t(u32, max_rss_queues,
+					    num_queues);
+	else
+		adapter->rss_queues = min_t(u32, max_rss_queues,
+					    num_online_cpus());
 
 	igb_set_flag_queue_pairs(adapter, max_rss_queues);
 }
-- 
2.21.0


             reply	other threads:[~2019-04-04 21:58 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-04 21:56 Vinicius Costa Gomes [this message]
2019-04-04 21:56 ` [Intel-wired-lan] [next-queue PATCH v1] igb: Fix limiting the number of queues to number of cpus Vinicius Costa Gomes
2019-04-05  0:04 ` Jeff Kirsher
2019-04-05  0:04   ` [Intel-wired-lan] " Jeff Kirsher
2019-04-05  1:32   ` Vinicius Costa Gomes
2019-04-05  1:32     ` [Intel-wired-lan] " Vinicius Costa Gomes
2019-04-18 20:57   ` Vinicius Costa Gomes
2019-04-18 20:57     ` [Intel-wired-lan] " Vinicius Costa Gomes
2019-04-18 21:28     ` Alexander Duyck
2019-04-18 21:28       ` Alexander Duyck
2019-04-18 22:40       ` Vinicius Costa Gomes
2019-04-18 22:40         ` Vinicius Costa Gomes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190404215632.9881-1-vinicius.gomes@intel.com \
    --to=vinicius.gomes@intel.com \
    --cc=bhagath@kalycito.com \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=jeffrey.t.kirsher@intel.com \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.