* [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014
@ 2014-02-21 10:39 Amir Vadai
2014-02-21 10:39 ` [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function Amir Vadai
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Amir Vadai @ 2014-02-21 10:39 UTC (permalink / raw)
To: David S. Miller; +Cc: Or Gerlitz, Yevgeny Petrilin, Amir Vadai, netdev
Hi,
This small patchset has a fix to a bogus usage of
netif_get_num_default_rss_queues() in mlx4_en driver.
Changes from V1:
- Removed affinity_hint patch, to make it a generic instead of mlx specific
Changes from V0:
- Instead of reverting the netif_get_num_default_rss_queues() in mlx4_en,
fixing it to limit the actual number of receive queues instead of limiting
the number of IRQ's.
Patchset was applied and tested against commit: cb6e926 "ipv6:fix checkpatch
errors with assignment in if condition"
Thanks,
Amir
Ido Shamay (2):
net/mlx4: Set number of RX rings in a utility function
net/mlx4: Fix limiting number of IRQ's instead of RSS queues
drivers/net/ethernet/mellanox/mlx4/en_main.c | 15 ++-------------
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 25 +++++++++++++++++++++++++
drivers/net/ethernet/mellanox/mlx4/main.c | 3 +--
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 2 +-
4 files changed, 29 insertions(+), 16 deletions(-)
--
1.8.3.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function
2014-02-21 10:39 [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 Amir Vadai
@ 2014-02-21 10:39 ` Amir Vadai
2014-02-21 11:01 ` David Laight
2014-02-21 10:39 ` [PATCH net-next V2 2/2] net/mlx4: Fix limiting number of IRQ's instead of RSS queues Amir Vadai
2014-02-24 23:38 ` [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 David Miller
2 siblings, 1 reply; 6+ messages in thread
From: Amir Vadai @ 2014-02-21 10:39 UTC (permalink / raw)
To: David S. Miller
Cc: Or Gerlitz, Yevgeny Petrilin, Amir Vadai, netdev, Ido Shamay
From: Ido Shamay <idos@mellanox.com>
mlx4_en_add() is too long.
Moving set number of RX rings to a utiltity function to improve
readability and modulization of the code.
Signed-off-by: Ido Shamay <idos@mellanox.com>
Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx4/en_main.c | 15 ++-------------
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 22 ++++++++++++++++++++++
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 2 +-
3 files changed, 25 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_main.c b/drivers/net/ethernet/mellanox/mlx4/en_main.c
index d357bf5..fa2f6e7 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_main.c
@@ -274,19 +274,8 @@ static void *mlx4_en_add(struct mlx4_dev *dev)
if (mdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_TS)
mlx4_en_init_timestamp(mdev);
- mlx4_foreach_port(i, dev, MLX4_PORT_TYPE_ETH) {
- if (!dev->caps.comp_pool) {
- mdev->profile.prof[i].rx_ring_num =
- rounddown_pow_of_two(max_t(int, MIN_RX_RINGS,
- min_t(int,
- dev->caps.num_comp_vectors,
- DEF_RX_RINGS)));
- } else {
- mdev->profile.prof[i].rx_ring_num = rounddown_pow_of_two(
- min_t(int, dev->caps.comp_pool/
- dev->caps.num_ports - 1 , MAX_MSIX_P_PORT - 1));
- }
- }
+ /* Set default number of RX rings*/
+ mlx4_en_set_num_rx_rings(mdev);
/* Create our own workqueue for reset/multicast tasks
* Note: we cannot use the shared workqueue because of deadlocks caused
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index b0eba6d..61e44ae 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -318,6 +318,28 @@ static void mlx4_en_free_rx_buf(struct mlx4_en_priv *priv,
}
}
+void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev)
+{
+ int i;
+ int num_of_eqs;
+ struct mlx4_dev *dev = mdev->dev;
+
+ mlx4_foreach_port(i, dev, MLX4_PORT_TYPE_ETH) {
+ if (!dev->caps.comp_pool)
+ num_of_eqs = max_t(int, MIN_RX_RINGS,
+ min_t(int,
+ dev->caps.num_comp_vectors,
+ DEF_RX_RINGS));
+ else
+ num_of_eqs = min_t(int, MAX_MSIX_P_PORT,
+ dev->caps.comp_pool/
+ dev->caps.num_ports) - 1;
+
+ mdev->profile.prof[i].rx_ring_num =
+ rounddown_pow_of_two(num_of_eqs);
+ }
+}
+
int mlx4_en_create_rx_ring(struct mlx4_en_priv *priv,
struct mlx4_en_rx_ring **pring,
u32 size, u16 stride, int node)
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 101d636..f97a0d3 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
@@ -738,7 +738,7 @@ int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
int cq, int user_prio);
void mlx4_en_deactivate_tx_ring(struct mlx4_en_priv *priv,
struct mlx4_en_tx_ring *ring);
-
+void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev);
int mlx4_en_create_rx_ring(struct mlx4_en_priv *priv,
struct mlx4_en_rx_ring **pring,
u32 size, u16 stride, int node);
--
1.8.3.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH net-next V2 2/2] net/mlx4: Fix limiting number of IRQ's instead of RSS queues
2014-02-21 10:39 [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 Amir Vadai
2014-02-21 10:39 ` [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function Amir Vadai
@ 2014-02-21 10:39 ` Amir Vadai
2014-02-24 23:38 ` [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 David Miller
2 siblings, 0 replies; 6+ messages in thread
From: Amir Vadai @ 2014-02-21 10:39 UTC (permalink / raw)
To: David S. Miller
Cc: Or Gerlitz, Yevgeny Petrilin, Amir Vadai, netdev, Ido Shamay,
Yuval Mintz
From: Ido Shamay <idos@mellanox.com>
This fix a performance bug introduced by commit 90b1ebe "mlx4: set
maximal number of default RSS queues", which limits the numbers of IRQs
opened by core module.
The limit should be on the number of queues in the indirection table -
rx_rings, and not on the number of IRQ's. Also, limiting on mlx4_core
initialization instead of in mlx4_en, prevented using "ethtool -L" to
utilize all the CPU's, when performance mode is prefered, since limiting
this number to 8 reduces overall packet rate by 15%-50% in multiple TCP
streams applications.
For example, after running ethtool -L <ethx> rx 16
Packet rate
Before the fix 897799
After the fix 1142070
Results were obtained using netperf:
S=200 ; ( for i in $(seq 1 $S) ; do ( \
netperf -H 11.7.13.55 -t TCP_RR -l 30 &) ; \
wait ; done | grep "1 1" | awk '{SUM+=$6} END {print SUM}' )
CC: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Ido Shamay <idos@mellanox.com>
Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++++-
drivers/net/ethernet/mellanox/mlx4/main.c | 3 +--
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 61e44ae..630ec03 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -322,6 +322,7 @@ void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev)
{
int i;
int num_of_eqs;
+ int num_rx_rings;
struct mlx4_dev *dev = mdev->dev;
mlx4_foreach_port(i, dev, MLX4_PORT_TYPE_ETH) {
@@ -335,8 +336,10 @@ void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev)
dev->caps.comp_pool/
dev->caps.num_ports) - 1;
+ num_rx_rings = min_t(int, num_of_eqs,
+ netif_get_num_default_rss_queues());
mdev->profile.prof[i].rx_ring_num =
- rounddown_pow_of_two(num_of_eqs);
+ rounddown_pow_of_two(num_rx_rings);
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index d711158..8c82a6b 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -41,7 +41,6 @@
#include <linux/slab.h>
#include <linux/io-mapping.h>
#include <linux/delay.h>
-#include <linux/netdevice.h>
#include <linux/kmod.h>
#include <linux/mlx4/device.h>
@@ -1974,7 +1973,7 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
struct msix_entry *entries;
int nreq = min_t(int, dev->caps.num_ports *
- min_t(int, netif_get_num_default_rss_queues() + 1,
+ min_t(int, num_online_cpus() + 1,
MAX_MSIX_P_PORT) + MSIX_LEGACY_SZ, MAX_MSIX);
int err;
int i;
--
1.8.3.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* RE: [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function
2014-02-21 10:39 ` [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function Amir Vadai
@ 2014-02-21 11:01 ` David Laight
2014-02-21 11:41 ` Ido Shamai
0 siblings, 1 reply; 6+ messages in thread
From: David Laight @ 2014-02-21 11:01 UTC (permalink / raw)
To: 'Amir Vadai', David S. Miller
Cc: Or Gerlitz, Yevgeny Petrilin, netdev, Ido Shamay
From: Amir Vadai
> From: Ido Shamay <idos@mellanox.com>
>
> mlx4_en_add() is too long.
> Moving set number of RX rings to a utiltity function to improve
> readability and modulization of the code.
...
> --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> @@ -318,6 +318,28 @@ static void mlx4_en_free_rx_buf(struct mlx4_en_priv *priv,
> }
> }
>
> +void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev)
> +{
Should be static.
David
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function
2014-02-21 11:01 ` David Laight
@ 2014-02-21 11:41 ` Ido Shamai
0 siblings, 0 replies; 6+ messages in thread
From: Ido Shamai @ 2014-02-21 11:41 UTC (permalink / raw)
To: David Laight, 'Amir Vadai', David S. Miller
Cc: Or Gerlitz, Yevgeny Petrilin, netdev, Ido Shamay
On 2/21/2014 1:01 PM, David Laight wrote:
> From: Amir Vadai
>> From: Ido Shamay <idos@mellanox.com>
>>
>> mlx4_en_add() is too long.
>> Moving set number of RX rings to a utiltity function to improve
>> readability and modulization of the code.
> ...
>> --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
>> +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
>> @@ -318,6 +318,28 @@ static void mlx4_en_free_rx_buf(struct mlx4_en_priv *priv,
>> }
>> }
>>
>> +void mlx4_en_set_num_rx_rings(struct mlx4_en_dev *mdev)
>> +{
>
> Should be static.
It's not in the same c file.
Function was defined in en_rx.c for better modularization.
It may be in use by other files (ethtool.c) in the future
Ido
>
> David
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014
2014-02-21 10:39 [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 Amir Vadai
2014-02-21 10:39 ` [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function Amir Vadai
2014-02-21 10:39 ` [PATCH net-next V2 2/2] net/mlx4: Fix limiting number of IRQ's instead of RSS queues Amir Vadai
@ 2014-02-24 23:38 ` David Miller
2 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2014-02-24 23:38 UTC (permalink / raw)
To: amirv; +Cc: ogerlitz, yevgenyp, netdev
From: Amir Vadai <amirv@mellanox.com>
Date: Fri, 21 Feb 2014 12:39:16 +0200
> This small patchset has a fix to a bogus usage of
> netif_get_num_default_rss_queues() in mlx4_en driver.
>
> Changes from V1:
> - Removed affinity_hint patch, to make it a generic instead of mlx specific
>
> Changes from V0:
> - Instead of reverting the netif_get_num_default_rss_queues() in mlx4_en,
> fixing it to limit the actual number of receive queues instead of limiting
> the number of IRQ's.
>
> Patchset was applied and tested against commit: cb6e926 "ipv6:fix checkpatch
> errors with assignment in if condition"
Series applied, thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-02-24 23:38 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-21 10:39 [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 Amir Vadai
2014-02-21 10:39 ` [PATCH net-next V2 1/2] net/mlx4: Set number of RX rings in a utility function Amir Vadai
2014-02-21 11:01 ` David Laight
2014-02-21 11:41 ` Ido Shamai
2014-02-21 10:39 ` [PATCH net-next V2 2/2] net/mlx4: Fix limiting number of IRQ's instead of RSS queues Amir Vadai
2014-02-24 23:38 ` [PATCH net-next V2 0/2] net/mlx4: Mellanox driver update 01-01-2014 David Miller
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.