[pull request][net-next 0/8] Mellanox, mlx5 updates 2019-08-22

[pull request][net-next 0/8] Mellanox, mlx5 updates 2019-08-22

[Saeed Mahameed]

Hi Dave,

This series provides some misc updates to mlx5 driver.
For more information please see tag log below.

Please pull and let me know if there is any problem.

Please note that the series starts with a merge of mlx5-next branch,
to resolve and avoid dependency with rdma tree.

Thanks,
Saeed.

---
The following changes since commit dc499cdf79f28a42cef0f1d62fe846090d14701a:

  Merge branch 'mlx5-next' of git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux (2019-08-22 16:32:07 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2019-08-22

for you to fetch changes up to 9f7df106da11f5863e12283700138722bdc46c83:

  net/mlx5e: Support TSO and TX checksum offloads for IP-in-IP tunnels (2019-08-22 16:32:15 -0700)

----------------------------------------------------------------
mlx5-updates-2019-08-22

Misc updates for mlx5e net device driver

1) Maxim and Tariq add the support for LAG TX port affinity distribution
When VF LAG is enabled, VFs netdevs will round-robin the TX affinity
of their tx queues among the different LAG ports.
2) Aya adds the support for ip-in-ip RSS.
3) Marina adds the support for ip-in-ip TX TSO and checksum offloads.
4) Moshe adds a device internal drop counter to mlx5 ethtool stats.

----------------------------------------------------------------
Aya Levin (2):
      net/mlx5e: Change function's position to a more fitting file
      net/mlx5e: Support RSS for IP-in-IP and IPv6 tunneled packets

Erez Alfasi (1):
      net/mlx5e: ethtool, Fix a typo in WOL function names

Marina Varshaver (2):
      net/mlx5e: Improve stateless offload capability check
      net/mlx5e: Support TSO and TX checksum offloads for IP-in-IP tunnels

Maxim Mikityanskiy (1):
      net/mlx5e: Support LAG TX port affinity distribution

Moshe Shemesh (1):
      net/mlx5e: Add device out of buffer counter

Tariq Toukan (1):
      net/mlx5e: Expose new function for TIS destroy loop

 drivers/net/ethernet/mellanox/mlx5/core/en.h       | 18 ++++--
 drivers/net/ethernet/mellanox/mlx5/core/en/fs.h    |  9 +++
 .../net/ethernet/mellanox/mlx5/core/en_ethtool.c   |  8 +--
 drivers/net/ethernet/mellanox/mlx5/core/en_fs.c    | 50 +++++++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c  | 75 ++++++++++++++++------
 drivers/net/ethernet/mellanox/mlx5/core/en_rep.c   |  9 +--
 drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 38 +++++++----
 .../net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c  |  4 +-
 .../ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c |  6 +-
 9 files changed, 163 insertions(+), 54 deletions(-)

[net-next 1/8] net/mlx5e: ethtool, Fix a typo in WOL function names

[Saeed Mahameed]

From: Erez Alfasi <ereza@mellanox.com>

Fix a typo in 'mlx5e_refomrat_wol_mode_mlx5_to_linux' and
'mlx5e_refomrat_wol_mode_linux_to_mlx5' function names:
"refomrat" -> "reformat".

Fixes: 928cfe8745a6 ("net/mlx5e: Wake On LAN support")
Signed-off-by: Erez Alfasi <ereza@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 7347d673f448..c5a9c20d7f00 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -1431,7 +1431,7 @@ static __u32 mlx5e_get_wol_supported(struct mlx5_core_dev *mdev)
 	return ret;
 }
 
-static __u32 mlx5e_refomrat_wol_mode_mlx5_to_linux(u8 mode)
+static __u32 mlx5e_reformat_wol_mode_mlx5_to_linux(u8 mode)
 {
 	__u32 ret = 0;
 
@@ -1459,7 +1459,7 @@ static __u32 mlx5e_refomrat_wol_mode_mlx5_to_linux(u8 mode)
 	return ret;
 }
 
-static u8 mlx5e_refomrat_wol_mode_linux_to_mlx5(__u32 mode)
+static u8 mlx5e_reformat_wol_mode_linux_to_mlx5(__u32 mode)
 {
 	u8 ret = 0;
 
@@ -1505,7 +1505,7 @@ static void mlx5e_get_wol(struct net_device *netdev,
 	if (err)
 		return;
 
-	wol->wolopts = mlx5e_refomrat_wol_mode_mlx5_to_linux(mlx5_wol_mode);
+	wol->wolopts = mlx5e_reformat_wol_mode_mlx5_to_linux(mlx5_wol_mode);
 }
 
 static int mlx5e_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
@@ -1521,7 +1521,7 @@ static int mlx5e_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
 	if (wol->wolopts & ~wol_supported)
 		return -EINVAL;
 
-	mlx5_wol_mode = mlx5e_refomrat_wol_mode_linux_to_mlx5(wol->wolopts);
+	mlx5_wol_mode = mlx5e_reformat_wol_mode_linux_to_mlx5(wol->wolopts);
 
 	return mlx5_set_port_wol(mdev, mlx5_wol_mode);
 }
-- 
2.21.0

[net-next 2/8] net/mlx5e: Expose new function for TIS destroy loop

[Saeed Mahameed]

From: Tariq Toukan <tariqt@mellanox.com>

For better modularity and code sharing.
Function internal change to be introduced in the next patches.

Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h      |  1 +
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 13 +++++++++----
 drivers/net/ethernet/mellanox/mlx5/core/en_rep.c  |  9 +++------
 3 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 446792799125..491c281416d0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -1107,6 +1107,7 @@ int mlx5e_create_tis(struct mlx5_core_dev *mdev, void *in, u32 *tisn);
 void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn);
 
 int mlx5e_create_tises(struct mlx5e_priv *priv);
+void mlx5e_destroy_tises(struct mlx5e_priv *priv);
 int mlx5e_update_nic_rx(struct mlx5e_priv *priv);
 void mlx5e_update_carrier(struct mlx5e_priv *priv);
 int mlx5e_close(struct net_device *netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index fa4bf2d4bcd4..d0cda5181ab4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -3179,6 +3179,14 @@ void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn)
 	mlx5_core_destroy_tis(mdev, tisn);
 }
 
+void mlx5e_destroy_tises(struct mlx5e_priv *priv)
+{
+	int tc;
+
+	for (tc = 0; tc < priv->profile->max_tc; tc++)
+		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+}
+
 int mlx5e_create_tises(struct mlx5e_priv *priv)
 {
 	int err;
@@ -3208,10 +3216,7 @@ int mlx5e_create_tises(struct mlx5e_priv *priv)
 
 static void mlx5e_cleanup_nic_tx(struct mlx5e_priv *priv)
 {
-	int tc;
-
-	for (tc = 0; tc < priv->profile->max_tc; tc++)
-		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+	mlx5e_destroy_tises(priv);
 }
 
 static void mlx5e_build_indir_tir_ctx_common(struct mlx5e_priv *priv,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 3c0d36b2b91c..b94fc3a35e10 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -1618,7 +1618,7 @@ static int mlx5e_init_rep_tx(struct mlx5e_priv *priv)
 {
 	struct mlx5e_rep_priv *rpriv = priv->ppriv;
 	struct mlx5_rep_uplink_priv *uplink_priv;
-	int tc, err;
+	int err;
 
 	err = mlx5e_create_tises(priv);
 	if (err) {
@@ -1654,18 +1654,15 @@ static int mlx5e_init_rep_tx(struct mlx5e_priv *priv)
 tc_esw_cleanup:
 	mlx5e_tc_esw_cleanup(&uplink_priv->tc_ht);
 destroy_tises:
-	for (tc = 0; tc < priv->profile->max_tc; tc++)
-		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+	mlx5e_destroy_tises(priv);
 	return err;
 }
 
 static void mlx5e_cleanup_rep_tx(struct mlx5e_priv *priv)
 {
 	struct mlx5e_rep_priv *rpriv = priv->ppriv;
-	int tc;
 
-	for (tc = 0; tc < priv->profile->max_tc; tc++)
-		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+	mlx5e_destroy_tises(priv);
 
 	if (rpriv->rep->vport == MLX5_VPORT_UPLINK) {
 		/* clean indirect TC block notifications */
-- 
2.21.0

[net-next 3/8] net/mlx5e: Support LAG TX port affinity distribution

[Saeed Mahameed]

From: Maxim Mikityanskiy <maximmi@mellanox.com>

When the VF LAG is in use, round-robin the TX affinity of channels among
the different ports, if supported by the firmware. Create a set of TISes
per port, while doing round-robin of the channels over the different
sets. Let all SQs of a channel share the same set of TISes.

If lag_tx_port_affinity HCA cap bit is supported, num_lag_ports > 1 and
we aren't the LACP owner (PF in the regular use), assign the affinities,
otherwise use tx_affinity == 0 in TIS context to let the FW assign the
affinities itself. The TISes of the LACP owner are mapped only to the
native physical port.

For VFs, the starting port for round-robin is determined by its vhca_id,
because a VF may have only one channel if attached to a single-core VM.

Signed-off-by: Maxim Mikityanskiy <maximmi@mellanox.com>
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  | 11 +++-
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 54 +++++++++++++------
 .../ethernet/mellanox/mlx5/core/ipoib/ipoib.c |  4 +-
 .../mellanox/mlx5/core/ipoib/ipoib_vlan.c     |  6 +--
 4 files changed, 53 insertions(+), 22 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 491c281416d0..e03f973c962f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -163,6 +163,14 @@ enum mlx5e_rq_group {
 #define MLX5E_NUM_RQ_GROUPS(g) (1 + MLX5E_RQ_GROUP_##g)
 };
 
+static inline u8 mlx5e_get_num_lag_ports(struct mlx5_core_dev *mdev)
+{
+	if (mlx5_lag_is_lacp_owner(mdev))
+		return 1;
+
+	return clamp_t(u8, MLX5_CAP_GEN(mdev, num_lag_ports), 1, MLX5_MAX_PORTS);
+}
+
 static inline u16 mlx5_min_rx_wqes(int wq_type, u32 wq_size)
 {
 	switch (wq_type) {
@@ -705,6 +713,7 @@ struct mlx5e_channel {
 	struct net_device         *netdev;
 	__be32                     mkey_be;
 	u8                         num_tc;
+	u8                         lag_port;
 
 	/* XDP_REDIRECT */
 	struct mlx5e_xdpsq         xdpsq;
@@ -818,7 +827,7 @@ struct mlx5e_priv {
 	struct mlx5e_rq            drop_rq;
 
 	struct mlx5e_channels      channels;
-	u32                        tisn[MLX5E_MAX_NUM_TC];
+	u32                        tisn[MLX5_MAX_PORTS][MLX5E_MAX_NUM_TC];
 	struct mlx5e_rqt           indir_rqt;
 	struct mlx5e_tir           indir_tir[MLX5E_NUM_INDIR_TIRS];
 	struct mlx5e_tir           inner_indir_tir[MLX5E_NUM_INDIR_TIRS];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index d0cda5181ab4..b1bc0e601cc2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -1442,7 +1442,7 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
 		return err;
 
 	csp.tis_lst_sz      = 1;
-	csp.tisn            = c->priv->tisn[0]; /* tc = 0 */
+	csp.tisn            = c->priv->tisn[c->lag_port][0]; /* tc = 0 */
 	csp.cqn             = sq->cq.mcq.cqn;
 	csp.wq_ctrl         = &sq->wq_ctrl;
 	csp.min_inline_mode = sq->min_inline_mode;
@@ -1692,7 +1692,7 @@ static int mlx5e_open_sqs(struct mlx5e_channel *c,
 	for (tc = 0; tc < params->num_tc; tc++) {
 		int txq_ix = c->ix + tc * priv->max_nch;
 
-		err = mlx5e_open_txqsq(c, c->priv->tisn[tc], txq_ix,
+		err = mlx5e_open_txqsq(c, c->priv->tisn[c->lag_port][tc], txq_ix,
 				       params, &cparam->sq, &c->sq[tc], tc);
 		if (err)
 			goto err_close_sqs;
@@ -1926,6 +1926,13 @@ static void mlx5e_close_queues(struct mlx5e_channel *c)
 	mlx5e_close_cq(&c->icosq.cq);
 }
 
+static u8 mlx5e_enumerate_lag_port(struct mlx5_core_dev *mdev, int ix)
+{
+	u16 port_aff_bias = mlx5_core_is_pf(mdev) ? 0 : MLX5_CAP_GEN(mdev, vhca_id);
+
+	return (ix + port_aff_bias) % mlx5e_get_num_lag_ports(mdev);
+}
+
 static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
 			      struct mlx5e_params *params,
 			      struct mlx5e_channel_param *cparam,
@@ -1960,6 +1967,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
 	c->xdp      = !!params->xdp_prog;
 	c->stats    = &priv->channel_stats[ix].ch;
 	c->irq_desc = irq_to_desc(irq);
+	c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
 
 	err = mlx5e_alloc_xps_cpumask(c, params);
 	if (err)
@@ -3181,35 +3189,49 @@ void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn)
 
 void mlx5e_destroy_tises(struct mlx5e_priv *priv)
 {
-	int tc;
+	int tc, i;
 
-	for (tc = 0; tc < priv->profile->max_tc; tc++)
-		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+	for (i = 0; i < mlx5e_get_num_lag_ports(priv->mdev); i++)
+		for (tc = 0; tc < priv->profile->max_tc; tc++)
+			mlx5e_destroy_tis(priv->mdev, priv->tisn[i][tc]);
+}
+
+static bool mlx5e_lag_should_assign_affinity(struct mlx5_core_dev *mdev)
+{
+	return MLX5_CAP_GEN(mdev, lag_tx_port_affinity) && mlx5e_get_num_lag_ports(mdev) > 1;
 }
 
 int mlx5e_create_tises(struct mlx5e_priv *priv)
 {
+	int tc, i;
 	int err;
-	int tc;
 
-	for (tc = 0; tc < priv->profile->max_tc; tc++) {
-		u32 in[MLX5_ST_SZ_DW(create_tis_in)] = {};
-		void *tisc;
+	for (i = 0; i < mlx5e_get_num_lag_ports(priv->mdev); i++) {
+		for (tc = 0; tc < priv->profile->max_tc; tc++) {
+			u32 in[MLX5_ST_SZ_DW(create_tis_in)] = {};
+			void *tisc;
 
-		tisc = MLX5_ADDR_OF(create_tis_in, in, ctx);
+			tisc = MLX5_ADDR_OF(create_tis_in, in, ctx);
 
-		MLX5_SET(tisc, tisc, prio, tc << 1);
+			MLX5_SET(tisc, tisc, prio, tc << 1);
 
-		err = mlx5e_create_tis(priv->mdev, in, &priv->tisn[tc]);
-		if (err)
-			goto err_close_tises;
+			if (mlx5e_lag_should_assign_affinity(priv->mdev))
+				MLX5_SET(tisc, tisc, lag_tx_port_affinity, i + 1);
+
+			err = mlx5e_create_tis(priv->mdev, in, &priv->tisn[i][tc]);
+			if (err)
+				goto err_close_tises;
+		}
 	}
 
 	return 0;
 
 err_close_tises:
-	for (tc--; tc >= 0; tc--)
-		mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+	for (; i >= 0; i--) {
+		for (tc--; tc >= 0; tc--)
+			mlx5e_destroy_tis(priv->mdev, priv->tisn[i][tc]);
+		tc = priv->profile->max_tc;
+	}
 
 	return err;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index 1a2560e3bf7c..3ed8ab2d703d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -279,7 +279,7 @@ static int mlx5i_init_tx(struct mlx5e_priv *priv)
 		return err;
 	}
 
-	err = mlx5i_create_tis(priv->mdev, ipriv->qp.qpn, &priv->tisn[0]);
+	err = mlx5i_create_tis(priv->mdev, ipriv->qp.qpn, &priv->tisn[0][0]);
 	if (err) {
 		mlx5_core_warn(priv->mdev, "create tis failed, %d\n", err);
 		goto err_destroy_underlay_qp;
@@ -296,7 +296,7 @@ static void mlx5i_cleanup_tx(struct mlx5e_priv *priv)
 {
 	struct mlx5i_priv *ipriv = priv->ppriv;
 
-	mlx5e_destroy_tis(priv->mdev, priv->tisn[0]);
+	mlx5e_destroy_tis(priv->mdev, priv->tisn[0][0]);
 	mlx5i_destroy_underlay_qp(priv->mdev, &ipriv->qp);
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
index c5a491e22e55..96e64187c089 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
@@ -210,7 +210,7 @@ static int mlx5i_pkey_open(struct net_device *netdev)
 		goto err_unint_underlay_qp;
 	}
 
-	err = mlx5i_create_tis(mdev, ipriv->qp.qpn, &epriv->tisn[0]);
+	err = mlx5i_create_tis(mdev, ipriv->qp.qpn, &epriv->tisn[0][0]);
 	if (err) {
 		mlx5_core_warn(mdev, "create child tis failed, %d\n", err);
 		goto err_remove_rx_uderlay_qp;
@@ -228,7 +228,7 @@ static int mlx5i_pkey_open(struct net_device *netdev)
 	return 0;
 
 err_clear_state_opened_flag:
-	mlx5e_destroy_tis(mdev, epriv->tisn[0]);
+	mlx5e_destroy_tis(mdev, epriv->tisn[0][0]);
 err_remove_rx_uderlay_qp:
 	mlx5_fs_remove_rx_underlay_qpn(mdev, ipriv->qp.qpn);
 err_unint_underlay_qp:
@@ -257,7 +257,7 @@ static int mlx5i_pkey_close(struct net_device *netdev)
 	mlx5i_uninit_underlay_qp(priv);
 	mlx5e_deactivate_priv_channels(priv);
 	mlx5e_close_channels(&priv->channels);
-	mlx5e_destroy_tis(mdev, priv->tisn[0]);
+	mlx5e_destroy_tis(mdev, priv->tisn[0][0]);
 unlock:
 	mutex_unlock(&priv->state_lock);
 	return 0;
-- 
2.21.0

[net-next 4/8] net/mlx5e: Add device out of buffer counter

[Saeed Mahameed]

From: Moshe Shemesh <moshe@mellanox.com>

Added the following packets drop counter:
Device out of buffer - counts packets which were dropped due to full
device internal receive queue.
This counter will be shown on ethtool as a new counter called
dev_out_of_buffer.
The counter is read from FW by command QUERY_VNIC_ENV.

Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 .../ethernet/mellanox/mlx5/core/en_stats.c    | 38 ++++++++++++-------
 1 file changed, 25 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 18e4c162256a..fbf7fe2f2657 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -369,17 +369,27 @@ static void mlx5e_grp_q_update_stats(struct mlx5e_priv *priv)
 }
 
 #define VNIC_ENV_OFF(c) MLX5_BYTE_OFF(query_vnic_env_out, c)
-static const struct counter_desc vnic_env_stats_desc[] = {
+static const struct counter_desc vnic_env_stats_steer_desc[] = {
 	{ "rx_steer_missed_packets",
 		VNIC_ENV_OFF(vport_env.nic_receive_steering_discard) },
 };
 
-#define NUM_VNIC_ENV_COUNTERS		ARRAY_SIZE(vnic_env_stats_desc)
+static const struct counter_desc vnic_env_stats_dev_oob_desc[] = {
+	{ "dev_out_of_buffer",
+		VNIC_ENV_OFF(vport_env.internal_rq_out_of_buffer) },
+};
+
+#define NUM_VNIC_ENV_STEER_COUNTERS(dev) \
+	(MLX5_CAP_GEN(dev, nic_receive_steering_discard) ? \
+	 ARRAY_SIZE(vnic_env_stats_steer_desc) : 0)
+#define NUM_VNIC_ENV_DEV_OOB_COUNTERS(dev) \
+	(MLX5_CAP_GEN(dev, vnic_env_int_rq_oob) ? \
+	 ARRAY_SIZE(vnic_env_stats_dev_oob_desc) : 0)
 
 static int mlx5e_grp_vnic_env_get_num_stats(struct mlx5e_priv *priv)
 {
-	return MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard) ?
-		NUM_VNIC_ENV_COUNTERS : 0;
+	return NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev) +
+		NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev);
 }
 
 static int mlx5e_grp_vnic_env_fill_strings(struct mlx5e_priv *priv, u8 *data,
@@ -387,12 +397,13 @@ static int mlx5e_grp_vnic_env_fill_strings(struct mlx5e_priv *priv, u8 *data,
 {
 	int i;
 
-	if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
-		return idx;
+	for (i = 0; i < NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev); i++)
+		strcpy(data + (idx++) * ETH_GSTRING_LEN,
+		       vnic_env_stats_steer_desc[i].format);
 
-	for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+	for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++)
 		strcpy(data + (idx++) * ETH_GSTRING_LEN,
-		       vnic_env_stats_desc[i].format);
+		       vnic_env_stats_dev_oob_desc[i].format);
 	return idx;
 }
 
@@ -401,12 +412,13 @@ static int mlx5e_grp_vnic_env_fill_stats(struct mlx5e_priv *priv, u64 *data,
 {
 	int i;
 
-	if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
-		return idx;
-
-	for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+	for (i = 0; i < NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev); i++)
 		data[idx++] = MLX5E_READ_CTR64_BE(priv->stats.vnic.query_vnic_env_out,
-						  vnic_env_stats_desc, i);
+						  vnic_env_stats_steer_desc, i);
+
+	for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++)
+		data[idx++] = MLX5E_READ_CTR32_BE(priv->stats.vnic.query_vnic_env_out,
+						  vnic_env_stats_dev_oob_desc, i);
 	return idx;
 }
 
-- 
2.21.0

[net-next 5/8] net/mlx5e: Change function's position to a more fitting file

[Saeed Mahameed]

From: Aya Levin <ayal@mellanox.com>

Move function which indicates whether tunnel inner flow table is
supported from en.h to en_fs.c. It fits better right after tunnel
protocol rules definitions.

Signed-off-by: Aya Levin <ayal@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h    | 6 ------
 drivers/net/ethernet/mellanox/mlx5/core/en/fs.h | 2 ++
 drivers/net/ethernet/mellanox/mlx5/core/en_fs.c | 6 ++++++
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index e03f973c962f..8d76452cacdc 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -1065,12 +1065,6 @@ int mlx5e_modify_sq(struct mlx5_core_dev *mdev, u32 sqn,
 void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq);
 void mlx5e_tx_disable_queue(struct netdev_queue *txq);
 
-static inline bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
-{
-	return (MLX5_CAP_ETH(mdev, tunnel_stateless_gre) &&
-		MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
-}
-
 static inline bool mlx5_tx_swp_supported(struct mlx5_core_dev *mdev)
 {
 	return MLX5_CAP_ETH(mdev, swp) &&
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index ca2161b42c7f..5acd982ff228 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -98,6 +98,8 @@ enum mlx5e_tunnel_types {
 	MLX5E_NUM_TUNNEL_TT,
 };
 
+bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev);
+
 /* L3/L4 traffic type classifier */
 struct mlx5e_ttc_table {
 	struct mlx5e_flow_table  ft;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
index 76cc10e44080..a8340e4fb0b9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
@@ -749,6 +749,12 @@ static struct mlx5e_etype_proto ttc_tunnel_rules[] = {
 	},
 };
 
+bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
+{
+	return (MLX5_CAP_ETH(mdev, tunnel_stateless_gre) &&
+		MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
+}
+
 static u8 mlx5e_etype_to_ipv(u16 ethertype)
 {
 	if (ethertype == ETH_P_IP)
-- 
2.21.0

[net-next 6/8] net/mlx5e: Support RSS for IP-in-IP and IPv6 tunneled packets

[Saeed Mahameed]

From: Aya Levin <ayal@mellanox.com>

Add support for inner header RSS on IP-in-IP and IPv6 tunneled packets.

Add rules to the steering table regarding outer IP header, with
IPv4/6->IP-in-IP. Tunneled packets with protocol numbers: 0x4 (IP-in-IP)
and 0x29 (IPv6) are RSS-ed on the inner IP header.
Separate FW dependencies between flow table inner IP capabilities and
GRE offload support. Allowing this feature even if GRE offload is not
supported.  Tested with multi stream TCP traffic tunneled with IPnIP.
Verified that:
Without this patch, only a single RX ring was processing the traffic.
With this patch, multiple RX rings were processing the traffic.
Verified with and without GRE offload support.

Signed-off-by: Aya Levin <ayal@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en/fs.h   |  4 ++
 .../net/ethernet/mellanox/mlx5/core/en_fs.c   | 46 ++++++++++++++++++-
 2 files changed, 49 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index 5acd982ff228..5aae3a7a5497 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -95,6 +95,10 @@ struct mlx5e_tirc_config {
 enum mlx5e_tunnel_types {
 	MLX5E_TT_IPV4_GRE,
 	MLX5E_TT_IPV6_GRE,
+	MLX5E_TT_IPV4_IPIP,
+	MLX5E_TT_IPV6_IPIP,
+	MLX5E_TT_IPV4_IPV6,
+	MLX5E_TT_IPV6_IPV6,
 	MLX5E_NUM_TUNNEL_TT,
 };
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
index a8340e4fb0b9..b99b17957543 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
@@ -747,11 +747,52 @@ static struct mlx5e_etype_proto ttc_tunnel_rules[] = {
 		.etype = ETH_P_IPV6,
 		.proto = IPPROTO_GRE,
 	},
+	[MLX5E_TT_IPV4_IPIP] = {
+		.etype = ETH_P_IP,
+		.proto = IPPROTO_IPIP,
+	},
+	[MLX5E_TT_IPV6_IPIP] = {
+		.etype = ETH_P_IPV6,
+		.proto = IPPROTO_IPIP,
+	},
+	[MLX5E_TT_IPV4_IPV6] = {
+		.etype = ETH_P_IP,
+		.proto = IPPROTO_IPV6,
+	},
+	[MLX5E_TT_IPV6_IPV6] = {
+		.etype = ETH_P_IPV6,
+		.proto = IPPROTO_IPV6,
+	},
+
 };
 
+static bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type)
+{
+	switch (proto_type) {
+	case IPPROTO_GRE:
+		return MLX5_CAP_ETH(mdev, tunnel_stateless_gre);
+	case IPPROTO_IPIP:
+	case IPPROTO_IPV6:
+		return MLX5_CAP_ETH(mdev, tunnel_stateless_ip_over_ip);
+	default:
+		return false;
+	}
+}
+
+static bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev)
+{
+	int tt;
+
+	for (tt = 0; tt < MLX5E_NUM_TUNNEL_TT; tt++) {
+		if (mlx5e_tunnel_proto_supported(mdev, ttc_tunnel_rules[tt].proto))
+			return true;
+	}
+	return false;
+}
+
 bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
 {
-	return (MLX5_CAP_ETH(mdev, tunnel_stateless_gre) &&
+	return (mlx5e_any_tunnel_proto_supported(mdev) &&
 		MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
 }
 
@@ -844,6 +885,9 @@ static int mlx5e_generate_ttc_table_rules(struct mlx5e_priv *priv,
 	dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
 	dest.ft   = params->inner_ttc->ft.t;
 	for (tt = 0; tt < MLX5E_NUM_TUNNEL_TT; tt++) {
+		if (!mlx5e_tunnel_proto_supported(priv->mdev,
+						  ttc_tunnel_rules[tt].proto))
+			continue;
 		rules[tt] = mlx5e_generate_ttc_rule(priv, ft, &dest,
 						    ttc_tunnel_rules[tt].etype,
 						    ttc_tunnel_rules[tt].proto);
-- 
2.21.0

[net-next 7/8] net/mlx5e: Improve stateless offload capability check

[Saeed Mahameed]

From: Marina Varshaver <marinav@mellanox.com>

Use generic function for checking tunnel stateless offload capability
instead of separate macros.

Signed-off-by: Marina Varshaver <marinav@mellanox.com>
Reviewed-by: Aya Levin <ayal@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en/fs.h   | 3 +++
 drivers/net/ethernet/mellanox/mlx5/core/en_fs.c   | 4 ++--
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 ++--
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index 5aae3a7a5497..68d593074f6c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -238,5 +238,8 @@ void mlx5e_disable_cvlan_filter(struct mlx5e_priv *priv);
 int mlx5e_create_flow_steering(struct mlx5e_priv *priv);
 void mlx5e_destroy_flow_steering(struct mlx5e_priv *priv);
 
+bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type);
+bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev);
+
 #endif /* __MLX5E_FLOW_STEER_H__ */
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
index b99b17957543..15b7f0f1427c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
@@ -766,7 +766,7 @@ static struct mlx5e_etype_proto ttc_tunnel_rules[] = {
 
 };
 
-static bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type)
+bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type)
 {
 	switch (proto_type) {
 	case IPPROTO_GRE:
@@ -779,7 +779,7 @@ static bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_ty
 	}
 }
 
-static bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev)
+bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev)
 {
 	int tt;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index b1bc0e601cc2..1c4f82842df9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -4877,7 +4877,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	netdev->hw_features      |= NETIF_F_HW_VLAN_STAG_TX;
 
 	if (mlx5_vxlan_allowed(mdev->vxlan) || mlx5_geneve_tx_allowed(mdev) ||
-	    MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+	    mlx5e_any_tunnel_proto_supported(mdev)) {
 		netdev->hw_enc_features |= NETIF_F_HW_CSUM;
 		netdev->hw_enc_features |= NETIF_F_TSO;
 		netdev->hw_enc_features |= NETIF_F_TSO6;
@@ -4892,7 +4892,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 		netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM;
 	}
 
-	if (MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+	if (mlx5e_tunnel_proto_supported(mdev, IPPROTO_GRE)) {
 		netdev->hw_features     |= NETIF_F_GSO_GRE |
 					   NETIF_F_GSO_GRE_CSUM;
 		netdev->hw_enc_features |= NETIF_F_GSO_GRE |
-- 
2.21.0

[net-next 8/8] net/mlx5e: Support TSO and TX checksum offloads for IP-in-IP tunnels

[Saeed Mahameed]

From: Marina Varshaver <marinav@mellanox.com>

Add TX offloads support for IP-in-IP tunneled packets by reporting
the needed netdev features.

Signed-off-by: Marina Varshaver <marinav@mellanox.com>
Reviewed-by: Aya Levin <ayal@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 1c4f82842df9..f4a5055dfaff 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -4241,6 +4241,7 @@ static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,
 
 	switch (proto) {
 	case IPPROTO_GRE:
+	case IPPROTO_IPIP:
 		return features;
 	case IPPROTO_UDP:
 		udph = udp_hdr(skb);
@@ -4901,6 +4902,15 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 						NETIF_F_GSO_GRE_CSUM;
 	}
 
+	if (mlx5e_tunnel_proto_supported(mdev, IPPROTO_IPIP)) {
+		netdev->hw_features |= NETIF_F_GSO_IPXIP4 |
+				       NETIF_F_GSO_IPXIP6;
+		netdev->hw_enc_features |= NETIF_F_GSO_IPXIP4 |
+					   NETIF_F_GSO_IPXIP6;
+		netdev->gso_partial_features |= NETIF_F_GSO_IPXIP4 |
+						NETIF_F_GSO_IPXIP6;
+	}
+
 	netdev->hw_features	                 |= NETIF_F_GSO_PARTIAL;
 	netdev->gso_partial_features             |= NETIF_F_GSO_UDP_L4;
 	netdev->hw_features                      |= NETIF_F_GSO_UDP_L4;
-- 
2.21.0

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Jakub Kicinski]

On Thu, 22 Aug 2019 23:35:52 +0000, Saeed Mahameed wrote:
> From: Moshe Shemesh <moshe@mellanox.com>
> 
> Added the following packets drop counter:
> Device out of buffer - counts packets which were dropped due to full
> device internal receive queue.
> This counter will be shown on ethtool as a new counter called
> dev_out_of_buffer.
> The counter is read from FW by command QUERY_VNIC_ENV.
> 
> Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>

Sounds like rx_fifo_errors, no? Doesn't rx_fifo_errors count RX
overruns?

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Saeed Mahameed]

On Thu, 2019-08-22 at 18:33 -0700, Jakub Kicinski wrote:
> On Thu, 22 Aug 2019 23:35:52 +0000, Saeed Mahameed wrote:
> > From: Moshe Shemesh <moshe@mellanox.com>
> > 
> > Added the following packets drop counter:
> > Device out of buffer - counts packets which were dropped due to
> > full
> > device internal receive queue.
> > This counter will be shown on ethtool as a new counter called
> > dev_out_of_buffer.
> > The counter is read from FW by command QUERY_VNIC_ENV.
> > 
> > Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
> > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
> 
> Sounds like rx_fifo_errors, no? Doesn't rx_fifo_errors count RX
> overruns?

No, that is port buffer you are looking for and we got that fully
covered in mlx5. this is different.

This new counter is deep into the HW data path pipeline and it covers
very rare and complex scenarios that got only recently introduced with
swichdev mode and "some" lately added tunnels offloads that are routed
between VFs/PFs.

Normally the HW is lossless once the packet passes port buffers into
the data plane pipeline, let's call that "fast lane", BUT for sriov
configurations with switchdev mode enabled and some special hand
crafted tc tunnel offloads that requires hairpin between VFs/PFs, the
hw might decide to send some traffic to a "service lane" which is still
fast path but unlike the "fast lane" it handles traffic through "HW
internal" receive and send queues (just like we do with hairpin) that
might drop packets. the whole thing is transparent to driver and it is
HW implementation specific.

Thanks,
Saeed.

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Jakub Kicinski]

On Fri, 23 Aug 2019 06:00:45 +0000, Saeed Mahameed wrote:
> On Thu, 2019-08-22 at 18:33 -0700, Jakub Kicinski wrote:
> > On Thu, 22 Aug 2019 23:35:52 +0000, Saeed Mahameed wrote:  
> > > From: Moshe Shemesh <moshe@mellanox.com>
> > > 
> > > Added the following packets drop counter:
> > > Device out of buffer - counts packets which were dropped due to
> > > full
> > > device internal receive queue.
> > > This counter will be shown on ethtool as a new counter called
> > > dev_out_of_buffer.
> > > The counter is read from FW by command QUERY_VNIC_ENV.
> > > 
> > > Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
> > > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>  
> > 
> > Sounds like rx_fifo_errors, no? Doesn't rx_fifo_errors count RX
> > overruns?  
> 
> No, that is port buffer you are looking for and we got that fully
> covered in mlx5. this is different.
> 
> This new counter is deep into the HW data path pipeline and it covers
> very rare and complex scenarios that got only recently introduced with
> swichdev mode and "some" lately added tunnels offloads that are routed
> between VFs/PFs.
> 
> Normally the HW is lossless once the packet passes port buffers into
> the data plane pipeline, let's call that "fast lane", BUT for sriov
> configurations with switchdev mode enabled and some special hand
> crafted tc tunnel offloads that requires hairpin between VFs/PFs, the
> hw might decide to send some traffic to a "service lane" which is still
> fast path but unlike the "fast lane" it handles traffic through "HW
> internal" receive and send queues (just like we do with hairpin) that
> might drop packets. the whole thing is transparent to driver and it is
> HW implementation specific.

I see thanks for the explanation and sorry for the delayed response.
Would it perhaps make sense to indicate the hairpin in the name?
dev_out_of_buffer is quite a generic name, and there seems to be no
doc, nor does the commit message explains it as well as you have..

Re: [pull request][net-next 0/8] Mellanox, mlx5 updates 2019-08-22

[David Miller]

From: Saeed Mahameed <saeedm@mellanox.com>
Date: Thu, 22 Aug 2019 23:35:45 +0000

> This series provides some misc updates to mlx5 driver.
> For more information please see tag log below.
> 
> Please pull and let me know if there is any problem.
> 
> Please note that the series starts with a merge of mlx5-next branch,
> to resolve and avoid dependency with rdma tree.

I agree with Jakub that the new statistic can probably have a more
descriptive name.

Thanks.

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Saeed Mahameed]

On Fri, 2019-08-23 at 11:16 -0700, Jakub Kicinski wrote:
> On Fri, 23 Aug 2019 06:00:45 +0000, Saeed Mahameed wrote:
> > On Thu, 2019-08-22 at 18:33 -0700, Jakub Kicinski wrote:
> > > On Thu, 22 Aug 2019 23:35:52 +0000, Saeed Mahameed wrote:  
> > > > From: Moshe Shemesh <moshe@mellanox.com>
> > > > 
> > > > Added the following packets drop counter:
> > > > Device out of buffer - counts packets which were dropped due to
> > > > full
> > > > device internal receive queue.
> > > > This counter will be shown on ethtool as a new counter called
> > > > dev_out_of_buffer.
> > > > The counter is read from FW by command QUERY_VNIC_ENV.
> > > > 
> > > > Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
> > > > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>  
> > > 
> > > Sounds like rx_fifo_errors, no? Doesn't rx_fifo_errors count RX
> > > overruns?  
> > 
> > No, that is port buffer you are looking for and we got that fully
> > covered in mlx5. this is different.
> > 
> > This new counter is deep into the HW data path pipeline and it
> > covers
> > very rare and complex scenarios that got only recently introduced
> > with
> > swichdev mode and "some" lately added tunnels offloads that are
> > routed
> > between VFs/PFs.
> > 
> > Normally the HW is lossless once the packet passes port buffers
> > into
> > the data plane pipeline, let's call that "fast lane", BUT for sriov
> > configurations with switchdev mode enabled and some special hand
> > crafted tc tunnel offloads that requires hairpin between VFs/PFs,
> > the
> > hw might decide to send some traffic to a "service lane" which is
> > still
> > fast path but unlike the "fast lane" it handles traffic through "HW
> > internal" receive and send queues (just like we do with hairpin)
> > that
> > might drop packets. the whole thing is transparent to driver and it
> > is
> > HW implementation specific.
> 
> I see thanks for the explanation and sorry for the delayed response.
> Would it perhaps make sense to indicate the hairpin in the name?

We had some internal discussion and we couldn't come up with the
perfect name :)

hairpin is just an implementation detail, we don't want to exclusively
bind this counter to hairpin only flows, the problem is not with
hairpin, the actual problem is due to the use of internal RQs, for now
it only happens with "hairpin like" flows, but tomorrow it can happen
with a different scenario but same root cause (the use of internal
RQs), we want to have one counter to count internal drops due to
internal use of internal RQs.

so how about:
dev_internal_rq_oob: Device Internal RQ out of buffer
dev_internal_out_of_res: Device Internal out of resources (more generic
? too generic ?)

Any suggestion that you provide will be more than welcome.

> dev_out_of_buffer is quite a generic name, and there seems to be no
> doc, nor does the commit message explains it as well as you have..

Regarding documentation:
All mlx5 ethool counters are documented here
https://community.mellanox.com/s/article/understanding-mlx5-linux-counters-and-status-parameters

once we decide on the name, will add the new counter to the doc.

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Jakub Kicinski]

On Mon, 26 Aug 2019 20:14:47 +0000, Saeed Mahameed wrote:
> > I see thanks for the explanation and sorry for the delayed response.
> > Would it perhaps make sense to indicate the hairpin in the name?  
> 
> We had some internal discussion and we couldn't come up with the
> perfect name :)
> 
> hairpin is just an implementation detail, we don't want to exclusively
> bind this counter to hairpin only flows, the problem is not with
> hairpin, the actual problem is due to the use of internal RQs, for now
> it only happens with "hairpin like" flows, but tomorrow it can happen
> with a different scenario but same root cause (the use of internal
> RQs), we want to have one counter to count internal drops due to
> internal use of internal RQs.
> 
> so how about:
> dev_internal_rq_oob: Device Internal RQ out of buffer
> dev_internal_out_of_res: Device Internal out of resources (more generic
> ? too generic ?)

Maybe dev_internal_queue_oob? The use of 'internal' is a little
unfortunate, because it may be read as RQ run out of internal buffers.
Rather than special type of queue run out of buffers.
But not knowing the HW I don't really have any great suggestions :(
Either of the above would work as well.

> Any suggestion that you provide will be more than welcome.
> 
> > dev_out_of_buffer is quite a generic name, and there seems to be no
> > doc, nor does the commit message explains it as well as you have..  
> 
> Regarding documentation:
> All mlx5 ethool counters are documented here
> https://community.mellanox.com/s/article/understanding-mlx5-linux-counters-and-status-parameters
> 
> once we decide on the name, will add the new counter to the doc.

I see, thanks!

Re: [net-next 4/8] net/mlx5e: Add device out of buffer counter

[Saeed Mahameed]

On Mon, 2019-08-26 at 13:39 -0700, Jakub Kicinski wrote:
> On Mon, 26 Aug 2019 20:14:47 +0000, Saeed Mahameed wrote:
> > > I see thanks for the explanation and sorry for the delayed
> > > response.
> > > Would it perhaps make sense to indicate the hairpin in the
> > > name?  
> > 
> > We had some internal discussion and we couldn't come up with the
> > perfect name :)
> > 
> > hairpin is just an implementation detail, we don't want to
> > exclusively
> > bind this counter to hairpin only flows, the problem is not with
> > hairpin, the actual problem is due to the use of internal RQs, for
> > now
> > it only happens with "hairpin like" flows, but tomorrow it can
> > happen
> > with a different scenario but same root cause (the use of internal
> > RQs), we want to have one counter to count internal drops due to
> > internal use of internal RQs.
> > 
> > so how about:
> > dev_internal_rq_oob: Device Internal RQ out of buffer
> > dev_internal_out_of_res: Device Internal out of resources (more
> > generic
> > ? too generic ?)
> 
> Maybe dev_internal_queue_oob? The use of 'internal' is a little
> unfortunate, because it may be read as RQ run out of internal
> buffers.
> Rather than special type of queue run out of buffers.
> But not knowing the HW I don't really have any great suggestions :(
> Either of the above would work as well.
> 

True, even our HW architects didn't know how to call it, since sticking
to a name now that might be deprecated in a future HW is what we are
trying to avoid. a generic name is preferable.

I like dev_internal_queue_oob, will take it with the team and send v2tomorrow. 

thanks Jakub for the support.


> > Any suggestion that you provide will be more than welcome.
> > 
> > > dev_out_of_buffer is quite a generic name, and there seems to be
> > > no
> > > doc, nor does the commit message explains it as well as you
> > > have..  
> > 
> > Regarding documentation:
> > All mlx5 ethool counters are documented here
> > https://community.mellanox.com/s/article/understanding-mlx5-linux-counters-and-status-parameters
> > 
> > once we decide on the name, will add the new counter to the doc.
> 
> I see, thanks!

I will add a link to this file in 
https://www.kernel.org/doc/html/latest/networking/device_drivers/mellanox/mlx5.html?highlight=mlx5