* [pull request][net-next 00/13] Mellanox, BlueField SmartNIC
@ 2019-02-16 1:34 Saeed Mahameed
2019-02-16 1:34 ` [net-next 01/13] net/mlx5: Correctly set LAG mode for ECPF Saeed Mahameed
` (13 more replies)
0 siblings, 14 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Saeed Mahameed
Hi Dave,
This series adds the support for Melanox BlueField SmartNIC.
For more information please see tag log below.
Please note the merge commit of mlx5-next at the base of the pull request:
259fae5a2cff ("Merge branch 'mlx5-next' of git://git.kernel.org/.../mellanox/linux")
Please pull and let me know if there is any problem.
Thanks,
Saeed.
---
The following changes since commit 259fae5a2cff72e19f82094fb73e2149f8d64396:
Merge branch 'mlx5-next' of git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux (2019-02-15 16:45:31 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2019-02-15
for you to fetch changes up to c96692fb8f3d0a161c6892ab9cc51a5e9992ccf2:
net/mlx5: E-Switch, Allow transition to offloads mode for ECPF (2019-02-15 17:25:58 -0800)
----------------------------------------------------------------
Support Mellanox BlueField SmartNIC (mlx5-updates-2019-02-15)
Bodong Wang says,
BlueField device is a multi-core ARM processor in a highly integrated
system on chip coupled with the ConnectX interconnect controller.
BlueField device can be presented in one out of two modes:
- SEPARATED_HOST: ARM processors as a separated and orthogonal host
like any other external host in the multi-host virtualization model.
- EMBEDDED_CPU: ARM processors as Embedded CPU (EC) and part of the
external hosts virtualization model.
While existing driver already supports the device on separated_host
mode, this patch series focus on the functionalities of embedded_cpu
mode.
On embedded_cpu mode, BlueField device exposes regular network
controller PCI function in the BlueField host(e.g, x86). However, a
separate PCI function called Embedded CPU Physical Function(ECPF) is
also added to the ARM host side, where standard Linux distributions is
able to run on the ARM cores. Depends on the NV configuration from
firmware, ECPF can be the e-switch manager and firmware pages supplier.
If ECPF is configured as e-switch manager and page supplier, it will
take over the responsibilities from the PF on BlueField host includes:
- Owns, controls and manages all e-switch parts, and takes e-switch
traffic by default. It also should perform ENABLE_HCA for the host
PF just like a PF does for its VFs.
- Provides and manages the ICM host memory required for the HCA to
store various contexts for itself, the PF and VFs belong the
e-switch it manages.
The PF on BlueField host side is still responsible for:
- Control its own permanent MAC.
- PCI and SRIOV configurations and perform ENABLE_HCA for its VFs.
The ECPF can also retrieve information about the external host it
controls, like host identifier, PCI BDF and number of virtual functions.
As these parameters may be changed dynamically, an event will be triggered
to the driver on ECPF side.
----------------------------------------------------------------
Bodong Wang (13):
net/mlx5: Correctly set LAG mode for ECPF
net/mlx5: E-Switch, Properly refer to the esw manager vport
net/mlx5: E-Switch, Properly refer to host PF vport as other vport
net/mlx5: E-Switch, Refactor offloads flow steering init/cleanup
net/mlx5: E-Switch, Split VF and special vports for offloads mode
net/mlx5: E-Switch, Use getter and iterator to access vport/rep
net/mlx5: E-Switch, Add state to eswitch vport representors
net/mlx5: E-Switch, Support load/unload reps of specific vport types
net/mlx5: E-Switch, Centralize repersentor reg/unreg to eswitch driver
net/mlx5: E-Switch, Assign a different position for uplink rep and vport
net/mlx5: E-Switch, Consider ECPF vport depends on eswitch ownership
net/mlx5: E-Switch, Load/unload VF reps according to event from host PF
net/mlx5: E-Switch, Allow transition to offloads mode for ECPF
drivers/infiniband/hw/mlx5/ib_rep.c | 20 +-
drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 25 +-
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 175 +++++---
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 65 ++-
.../ethernet/mellanox/mlx5/core/eswitch_offloads.c | 470 +++++++++++++++++----
drivers/net/ethernet/mellanox/mlx5/core/lag.c | 5 +
drivers/net/ethernet/mellanox/mlx5/core/vport.c | 10 +-
include/linux/mlx5/driver.h | 5 +
include/linux/mlx5/eswitch.h | 19 +-
include/linux/mlx5/mlx5_ifc.h | 3 +-
include/linux/mlx5/vport.h | 20 +-
11 files changed, 617 insertions(+), 200 deletions(-)
^ permalink raw reply [flat|nested] 15+ messages in thread
* [net-next 01/13] net/mlx5: Correctly set LAG mode for ECPF
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 02/13] net/mlx5: E-Switch, Properly refer to the esw manager vport Saeed Mahameed
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
When bonding is added, driver assumes that it's RoCE LAG if no VF is
enabled. This is not enough for ECPF as the VF is enabled in host PF
side. LAG should only choose RoCE mode when both slave devices meet
conditions below:
1. E-Switch offloads mode is NONE.
2. No VF is enabled.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/lag.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag.c
index 2d223385dc81..04c5aca7f8c5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag.c
@@ -343,6 +343,11 @@ static void mlx5_do_bond(struct mlx5_lag *ldev)
roce_lag = !mlx5_sriov_is_enabled(dev0) &&
!mlx5_sriov_is_enabled(dev1);
+#ifdef CONFIG_MLX5_ESWITCH
+ roce_lag &= dev0->priv.eswitch->mode == SRIOV_NONE &&
+ dev1->priv.eswitch->mode == SRIOV_NONE;
+#endif
+
if (roce_lag)
mlx5_lag_remove_ib_devices(ldev);
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 02/13] net/mlx5: E-Switch, Properly refer to the esw manager vport
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
2019-02-16 1:34 ` [net-next 01/13] net/mlx5: Correctly set LAG mode for ECPF Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 03/13] net/mlx5: E-Switch, Properly refer to host PF vport as other vport Saeed Mahameed
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Bodong Wang, Eli Cohen, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
In SmartNIC mode, the eswitch manager is not necessarily the PF
(vport 0). Use a helper function to get the correct eswitch manager
vport number and cache on the eswitch instance for fast reference.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Eli Cohen <eli@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 35 ++++++++++++-------
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 10 ++++++
.../mellanox/mlx5/core/eswitch_offloads.c | 7 ++--
include/linux/mlx5/vport.h | 2 ++
4 files changed, 39 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 05830696abd8..9c622749dbde 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -378,16 +378,16 @@ static int esw_add_uc_addr(struct mlx5_eswitch *esw, struct vport_addr *vaddr)
u16 vport = vaddr->vport;
int err;
- /* Skip mlx5_mpfs_add_mac for PFs,
- * it is already done by the PF netdev in mlx5e_execute_l2_action
+ /* Skip mlx5_mpfs_add_mac for eswitch_managers,
+ * it is already done by its netdev in mlx5e_execute_l2_action
*/
- if (!vport)
+ if (esw->manager_vport == vport)
goto fdb_add;
err = mlx5_mpfs_add_mac(esw->dev, mac);
if (err) {
esw_warn(esw->dev,
- "Failed to add L2 table mac(%pM) for vport(%d), err(%d)\n",
+ "Failed to add L2 table mac(%pM) for vport(0x%x), err(%d)\n",
mac, vport, err);
return err;
}
@@ -410,10 +410,10 @@ static int esw_del_uc_addr(struct mlx5_eswitch *esw, struct vport_addr *vaddr)
u16 vport = vaddr->vport;
int err = 0;
- /* Skip mlx5_mpfs_del_mac for PFs,
- * it is already done by the PF netdev in mlx5e_execute_l2_action
+ /* Skip mlx5_mpfs_del_mac for eswitch managerss,
+ * it is already done by its netdev in mlx5e_execute_l2_action
*/
- if (!vport || !vaddr->mpfs)
+ if (!vaddr->mpfs || esw->manager_vport == vport)
goto fdb_del;
err = mlx5_mpfs_del_mac(esw->dev, mac);
@@ -1457,15 +1457,22 @@ static void esw_apply_vport_conf(struct mlx5_eswitch *esw,
{
int vport_num = vport->vport;
- if (!vport_num)
+ if (esw->manager_vport == vport_num)
return;
mlx5_modify_vport_admin_state(esw->dev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
vport_num,
vport->info.link_state);
- mlx5_modify_nic_vport_mac_address(esw->dev, vport_num, vport->info.mac);
- mlx5_modify_nic_vport_node_guid(esw->dev, vport_num, vport->info.node_guid);
+
+ /* Host PF has its own mac/guid. */
+ if (vport_num) {
+ mlx5_modify_nic_vport_mac_address(esw->dev, vport_num,
+ vport->info.mac);
+ mlx5_modify_nic_vport_node_guid(esw->dev, vport_num,
+ vport->info.node_guid);
+ }
+
modify_esw_vport_cvlan(esw->dev, vport_num, vport->info.vlan, vport->info.qos,
(vport->info.vlan || vport->info.qos));
@@ -1537,8 +1544,11 @@ static void esw_enable_vport(struct mlx5_eswitch *esw, int vport_num,
vport->enabled_events = enable_events;
vport->enabled = true;
- /* only PF is trusted by default */
- if (!vport_num)
+ /* Esw manager is trusted by default. Host PF (vport 0) is trusted as well
+ * in smartNIC as it's a vport group manager.
+ */
+ if (esw->manager_vport == vport_num ||
+ (!vport_num && mlx5_core_is_ecpf(esw->dev)))
vport->info.trusted = true;
esw_vport_change_handle_locked(vport);
@@ -1733,6 +1743,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
return -ENOMEM;
esw->dev = dev;
+ esw->manager_vport = mlx5_eswitch_manager_vport(dev);
esw->work_queue = create_singlethread_workqueue("mlx5_esw_wq");
if (!esw->work_queue) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 0a3eee8746c1..959a9e28d08f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -38,6 +38,7 @@
#include <net/devlink.h>
#include <linux/mlx5/device.h>
#include <linux/mlx5/eswitch.h>
+#include <linux/mlx5/vport.h>
#include <linux/mlx5/fs.h>
#include "lib/mpfs.h"
@@ -204,6 +205,7 @@ struct mlx5_eswitch {
struct mlx5_esw_offload offloads;
int mode;
int nvports;
+ u16 manager_vport;
};
void esw_offloads_cleanup(struct mlx5_eswitch *esw, int nvports);
@@ -363,6 +365,14 @@ bool mlx5_esw_lag_prereq(struct mlx5_core_dev *dev0,
#define esw_debug(dev, format, ...) \
mlx5_core_dbg_mask(dev, MLX5_DEBUG_ESWITCH_MASK, format, ##__VA_ARGS__)
+
+/* The returned number is valid only when the dev is eswitch manager. */
+static inline u16 mlx5_eswitch_manager_vport(struct mlx5_core_dev *dev)
+{
+ return mlx5_core_is_ecpf_esw_manager(dev) ?
+ MLX5_VPORT_ECPF : MLX5_VPORT_PF;
+}
+
#else /* CONFIG_MLX5_ESWITCH */
/* eswitch API stubs */
static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 9128b45f3f37..af2c44d31357 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -522,7 +522,8 @@ mlx5_eswitch_add_send_to_vport_rule(struct mlx5_eswitch *esw, int vport, u32 sqn
misc = MLX5_ADDR_OF(fte_match_param, spec->match_value, misc_parameters);
MLX5_SET(fte_match_set_misc, misc, source_sqn, sqn);
- MLX5_SET(fte_match_set_misc, misc, source_port, 0x0); /* source vport is 0 */
+ /* source vport is the esw manager */
+ MLX5_SET(fte_match_set_misc, misc, source_port, esw->manager_vport);
misc = MLX5_ADDR_OF(fte_match_param, spec->match_criteria, misc_parameters);
MLX5_SET_TO_ONES(fte_match_set_misc, misc, source_sqn);
@@ -567,7 +568,7 @@ static void peer_miss_rules_setup(struct mlx5_core_dev *peer_dev,
source_eswitch_owner_vhca_id);
dest->type = MLX5_FLOW_DESTINATION_TYPE_VPORT;
- dest->vport.num = 0;
+ dest->vport.num = peer_dev->priv.eswitch->manager_vport;
dest->vport.vhca_id = MLX5_CAP_GEN(peer_dev, vhca_id);
dest->vport.flags |= MLX5_FLOW_DEST_VPORT_VHCA_ID;
}
@@ -666,7 +667,7 @@ static int esw_add_fdb_miss_rule(struct mlx5_eswitch *esw)
dmac_c[0] = 0x01;
dest.type = MLX5_FLOW_DESTINATION_TYPE_VPORT;
- dest.vport.num = 0;
+ dest.vport.num = esw->manager_vport;
flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
flow_rule = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb, spec,
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index 3bc05449ac39..b67bcc95ab5d 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -52,6 +52,8 @@ enum {
};
enum {
+ MLX5_VPORT_PF = 0x0,
+ MLX5_VPORT_ECPF = 0xfffe,
MLX5_VPORT_UPLINK = 0xffff
};
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 03/13] net/mlx5: E-Switch, Properly refer to host PF vport as other vport
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
2019-02-16 1:34 ` [net-next 01/13] net/mlx5: Correctly set LAG mode for ECPF Saeed Mahameed
2019-02-16 1:34 ` [net-next 02/13] net/mlx5: E-Switch, Properly refer to the esw manager vport Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 04/13] net/mlx5: E-Switch, Refactor offloads flow steering init/cleanup Saeed Mahameed
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Eli Cohen, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
Commands referring to vports use the following scheme:
1. When referring to my own vport, put 0 in vport and 0 in other_vport.
2. When referring to another vport, put the vport number of the
referred vport and put 1 in other_vport. It was assumed that driver
is accessing other vport when vport number is greater than 0.
With the above scheme, the case that ECPF eswitch manager is trying
to access host PF vport will fall over with scheme 1 as the vport
number is 0. This is apparently wrong as driver is trying to refer
other vport.
As such usage can only happen in the eswitch context, change relevant
functions to provide other vport input properly.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Eli Cohen <eli@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 6 ++++--
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 13 ++++++-------
drivers/net/ethernet/mellanox/mlx5/core/vport.c | 10 ++++------
include/linux/mlx5/vport.h | 4 ++--
4 files changed, 16 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 685f1975be58..f84889bbe2a0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -1083,7 +1083,8 @@ static int mlx5e_vf_rep_open(struct net_device *dev)
if (!mlx5_modify_vport_admin_state(priv->mdev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
- rep->vport, MLX5_VPORT_ADMIN_STATE_UP))
+ rep->vport, 1,
+ MLX5_VPORT_ADMIN_STATE_UP))
netif_carrier_on(dev);
unlock:
@@ -1101,7 +1102,8 @@ static int mlx5e_vf_rep_close(struct net_device *dev)
mutex_lock(&priv->state_lock);
mlx5_modify_vport_admin_state(priv->mdev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
- rep->vport, MLX5_VPORT_ADMIN_STATE_DOWN);
+ rep->vport, 1,
+ MLX5_VPORT_ADMIN_STATE_DOWN);
ret = mlx5e_close_locked(dev);
mutex_unlock(&priv->state_lock);
return ret;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 9c622749dbde..648c743cc947 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1462,7 +1462,7 @@ static void esw_apply_vport_conf(struct mlx5_eswitch *esw,
mlx5_modify_vport_admin_state(esw->dev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
- vport_num,
+ vport_num, 1,
vport->info.link_state);
/* Host PF has its own mac/guid. */
@@ -1581,10 +1581,10 @@ static void esw_disable_vport(struct mlx5_eswitch *esw, int vport_num)
esw_vport_change_handle_locked(vport);
vport->enabled_events = 0;
esw_vport_disable_qos(esw, vport_num);
- if (vport_num && esw->mode == SRIOV_LEGACY) {
+ if (esw->mode == SRIOV_LEGACY) {
mlx5_modify_vport_admin_state(esw->dev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
- vport_num,
+ vport_num, 1,
MLX5_VPORT_ADMIN_STATE_DOWN);
esw_vport_disable_egress_acl(esw, vport);
esw_vport_disable_ingress_acl(esw, vport);
@@ -1875,7 +1875,7 @@ int mlx5_eswitch_set_vport_state(struct mlx5_eswitch *esw,
err = mlx5_modify_vport_admin_state(esw->dev,
MLX5_VPORT_STATE_OP_MOD_ESW_VPORT,
- vport, link_state);
+ vport, 1, link_state);
if (err) {
mlx5_core_warn(esw->dev,
"Failed to set vport %d link state, err = %d",
@@ -2137,7 +2137,7 @@ static int mlx5_eswitch_query_vport_drop_stats(struct mlx5_core_dev *dev,
!MLX5_CAP_GEN(dev, transmit_discard_vport_down))
return 0;
- err = mlx5_query_vport_down_stats(dev, vport_idx,
+ err = mlx5_query_vport_down_stats(dev, vport_idx, 1,
&rx_discard_vport_down,
&tx_discard_vport_down);
if (err)
@@ -2174,8 +2174,7 @@ int mlx5_eswitch_get_vport_stats(struct mlx5_eswitch *esw,
MLX5_CMD_OP_QUERY_VPORT_COUNTER);
MLX5_SET(query_vport_counter_in, in, op_mod, 0);
MLX5_SET(query_vport_counter_in, in, vport_number, vport);
- if (vport)
- MLX5_SET(query_vport_counter_in, in, other_vport, 1);
+ MLX5_SET(query_vport_counter_in, in, other_vport, 1);
memset(out, 0, outlen);
err = mlx5_cmd_exec(esw->dev, in, sizeof(in), out, outlen);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c
index 9a928eb48522..ef95feca9961 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c
@@ -64,7 +64,7 @@ u8 mlx5_query_vport_state(struct mlx5_core_dev *mdev, u8 opmod, u16 vport)
}
int mlx5_modify_vport_admin_state(struct mlx5_core_dev *mdev, u8 opmod,
- u16 vport, u8 state)
+ u16 vport, u8 other_vport, u8 state)
{
u32 in[MLX5_ST_SZ_DW(modify_vport_state_in)] = {0};
u32 out[MLX5_ST_SZ_DW(modify_vport_state_out)] = {0};
@@ -73,8 +73,7 @@ int mlx5_modify_vport_admin_state(struct mlx5_core_dev *mdev, u8 opmod,
MLX5_CMD_OP_MODIFY_VPORT_STATE);
MLX5_SET(modify_vport_state_in, in, op_mod, opmod);
MLX5_SET(modify_vport_state_in, in, vport_number, vport);
- if (vport)
- MLX5_SET(modify_vport_state_in, in, other_vport, 1);
+ MLX5_SET(modify_vport_state_in, in, other_vport, other_vport);
MLX5_SET(modify_vport_state_in, in, admin_state, state);
return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
@@ -1057,7 +1056,7 @@ int mlx5_core_query_vport_counter(struct mlx5_core_dev *dev, u8 other_vport,
EXPORT_SYMBOL_GPL(mlx5_core_query_vport_counter);
int mlx5_query_vport_down_stats(struct mlx5_core_dev *mdev, u16 vport,
- u64 *rx_discard_vport_down,
+ u8 other_vport, u64 *rx_discard_vport_down,
u64 *tx_discard_vport_down)
{
u32 out[MLX5_ST_SZ_DW(query_vnic_env_out)] = {0};
@@ -1068,8 +1067,7 @@ int mlx5_query_vport_down_stats(struct mlx5_core_dev *mdev, u16 vport,
MLX5_CMD_OP_QUERY_VNIC_ENV);
MLX5_SET(query_vnic_env_in, in, op_mod, 0);
MLX5_SET(query_vnic_env_in, in, vport_number, vport);
- if (vport)
- MLX5_SET(query_vnic_env_in, in, other_vport, 1);
+ MLX5_SET(query_vnic_env_in, in, other_vport, other_vport);
err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
if (err)
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index b67bcc95ab5d..b7edcb1dadd8 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -59,7 +59,7 @@ enum {
u8 mlx5_query_vport_state(struct mlx5_core_dev *mdev, u8 opmod, u16 vport);
int mlx5_modify_vport_admin_state(struct mlx5_core_dev *mdev, u8 opmod,
- u16 vport, u8 state);
+ u16 vport, u8 other_vport, u8 state);
int mlx5_query_nic_vport_mac_address(struct mlx5_core_dev *mdev,
u16 vport, u8 *addr);
int mlx5_query_nic_vport_min_inline(struct mlx5_core_dev *mdev,
@@ -121,7 +121,7 @@ int mlx5_modify_nic_vport_vlans(struct mlx5_core_dev *dev,
int mlx5_nic_vport_enable_roce(struct mlx5_core_dev *mdev);
int mlx5_nic_vport_disable_roce(struct mlx5_core_dev *mdev);
int mlx5_query_vport_down_stats(struct mlx5_core_dev *mdev, u16 vport,
- u64 *rx_discard_vport_down,
+ u8 other_vport, u64 *rx_discard_vport_down,
u64 *tx_discard_vport_down);
int mlx5_core_query_vport_counter(struct mlx5_core_dev *dev, u8 other_vport,
int vf, u8 port_num, void *out,
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 04/13] net/mlx5: E-Switch, Refactor offloads flow steering init/cleanup
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (2 preceding siblings ...)
2019-02-16 1:34 ` [net-next 03/13] net/mlx5: E-Switch, Properly refer to host PF vport as other vport Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 05/13] net/mlx5: E-Switch, Split VF and special vports for offloads mode Saeed Mahameed
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
E-switch offloads mode initialize/cleanup multiple steering related
entities (flow table/group). Refactor these operations to internal
helper functions for better block design.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../mellanox/mlx5/core/eswitch_offloads.c | 43 +++++++++++++------
1 file changed, 31 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index af2c44d31357..19969d487a01 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1404,7 +1404,7 @@ static void esw_offloads_devcom_cleanup(struct mlx5_eswitch *esw)
mlx5_devcom_unregister_component(devcom, MLX5_DEVCOM_ESW_OFFLOADS);
}
-int esw_offloads_init(struct mlx5_eswitch *esw, int nvports)
+static int esw_offloads_steering_init(struct mlx5_eswitch *esw, int nvports)
{
int err;
@@ -1422,16 +1422,8 @@ int esw_offloads_init(struct mlx5_eswitch *esw, int nvports)
if (err)
goto create_fg_err;
- err = esw_offloads_load_reps(esw, nvports);
- if (err)
- goto err_reps;
-
- esw_offloads_devcom_init(esw);
return 0;
-err_reps:
- esw_destroy_vport_rx_group(esw);
-
create_fg_err:
esw_destroy_offloads_table(esw);
@@ -1441,6 +1433,35 @@ int esw_offloads_init(struct mlx5_eswitch *esw, int nvports)
return err;
}
+static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw)
+{
+ esw_destroy_vport_rx_group(esw);
+ esw_destroy_offloads_table(esw);
+ esw_destroy_offloads_fdb_tables(esw);
+}
+
+int esw_offloads_init(struct mlx5_eswitch *esw, int nvports)
+{
+ int err;
+
+ mutex_init(&esw->fdb_table.offloads.fdb_prio_lock);
+
+ err = esw_offloads_steering_init(esw, nvports);
+ if (err)
+ return err;
+
+ err = esw_offloads_load_reps(esw, nvports);
+ if (err)
+ goto err_reps;
+
+ esw_offloads_devcom_init(esw);
+ return 0;
+
+err_reps:
+ esw_offloads_steering_cleanup(esw);
+ return err;
+}
+
static int esw_offloads_stop(struct mlx5_eswitch *esw,
struct netlink_ext_ack *extack)
{
@@ -1464,9 +1485,7 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw, int nvports)
{
esw_offloads_devcom_cleanup(esw);
esw_offloads_unload_reps(esw, nvports);
- esw_destroy_vport_rx_group(esw);
- esw_destroy_offloads_table(esw);
- esw_destroy_offloads_fdb_tables(esw);
+ esw_offloads_steering_cleanup(esw);
}
static int esw_mode_from_devlink(u16 mode, u16 *mlx5_mode)
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 05/13] net/mlx5: E-Switch, Split VF and special vports for offloads mode
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (3 preceding siblings ...)
2019-02-16 1:34 ` [net-next 04/13] net/mlx5: E-Switch, Refactor offloads flow steering init/cleanup Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 06/13] net/mlx5: E-Switch, Use getter and iterator to access vport/rep Saeed Mahameed
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
When driver is entering offloads mode, there are two major tasks to
do: initialize flow steering and create representors. Flow steering
should make sure enough flow table/group spaces are reserved for all
reps. Representors will be created in a group, all or none.
With the introduction of ECPF, flow steering should still reserve the
same spaces. But, the representors are not always loaded/unloaded in a
single piece. Once ECPF is in offloads mode, it will get the number
of VF changing event from host PF. In such scenario, only the VF reps
should be loaded/unloaded, not the reps for special vports (such as
the uplink vport).
Thus, when entering offloads mode, driver should specify the total
number of reps, and the number of VF reps separately. When leaving
offloads mode, the cleanup should use the information self-contained
in eswitch such as number of VFs.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 7 +--
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 5 +-
.../mellanox/mlx5/core/eswitch_offloads.c | 57 +++++++++++++------
include/linux/mlx5/vport.h | 1 +
4 files changed, 48 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 648c743cc947..be6c2931d2a0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1641,7 +1641,8 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
} else {
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_ETH);
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
- err = esw_offloads_init(esw, nvfs + MLX5_SPECIAL_VPORTS);
+ err = esw_offloads_init(esw, nvfs,
+ nvfs + MLX5_SPECIAL_VPORTS);
}
if (err)
@@ -1683,7 +1684,6 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw)
{
struct esw_mc_addr *mc_promisc;
int old_mode;
- int nvports;
int i;
if (!ESW_ALLOWED(esw) || esw->mode == SRIOV_NONE)
@@ -1693,7 +1693,6 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw)
esw->enabled_vports, esw->mode);
mc_promisc = &esw->mc_promisc;
- nvports = esw->enabled_vports;
if (esw->mode == SRIOV_LEGACY)
mlx5_eq_notifier_unregister(esw->dev, &esw->nb);
@@ -1709,7 +1708,7 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw)
if (esw->mode == SRIOV_LEGACY)
esw_destroy_legacy_fdb_table(esw);
else if (esw->mode == SRIOV_OFFLOADS)
- esw_offloads_cleanup(esw, nvports);
+ esw_offloads_cleanup(esw);
old_mode = esw->mode;
esw->mode = SRIOV_NONE;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 959a9e28d08f..fd845e6c44d5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -208,8 +208,9 @@ struct mlx5_eswitch {
u16 manager_vport;
};
-void esw_offloads_cleanup(struct mlx5_eswitch *esw, int nvports);
-int esw_offloads_init(struct mlx5_eswitch *esw, int nvports);
+void esw_offloads_cleanup(struct mlx5_eswitch *esw);
+int esw_offloads_init(struct mlx5_eswitch *esw, int vf_nvports,
+ int total_nvports);
void esw_offloads_cleanup_reps(struct mlx5_eswitch *esw);
int esw_offloads_init_reps(struct mlx5_eswitch *esw);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 19969d487a01..14f7ad67cfe4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -54,6 +54,8 @@ enum {
#define fdb_prio_table(esw, chain, prio, level) \
(esw)->fdb_table.offloads.fdb_prio[(chain)][(prio)][(level)]
+#define UPLINK_REP_INDEX 0
+
static struct mlx5_flow_table *
esw_get_prio_table(struct mlx5_eswitch *esw, u32 chain, u16 prio, int level);
static void
@@ -1239,19 +1241,28 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
return 0;
}
+static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw,
+ struct mlx5_eswitch_rep *rep, u8 rep_type)
+{
+ if (!rep->rep_if[rep_type].valid)
+ return;
+
+ rep->rep_if[rep_type].unload(rep);
+}
+
static void esw_offloads_unload_reps_type(struct mlx5_eswitch *esw, int nvports,
u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
int vport;
- for (vport = nvports - 1; vport >= 0; vport--) {
+ for (vport = nvports; vport >= MLX5_VPORT_FIRST_VF; vport--) {
rep = &esw->offloads.vport_reps[vport];
- if (!rep->rep_if[rep_type].valid)
- continue;
-
- rep->rep_if[rep_type].unload(rep);
+ __esw_offloads_unload_rep(esw, rep, rep_type);
}
+
+ rep = &esw->offloads.vport_reps[UPLINK_REP_INDEX];
+ __esw_offloads_unload_rep(esw, rep, rep_type);
}
static void esw_offloads_unload_reps(struct mlx5_eswitch *esw, int nvports)
@@ -1262,6 +1273,15 @@ static void esw_offloads_unload_reps(struct mlx5_eswitch *esw, int nvports)
esw_offloads_unload_reps_type(esw, nvports, rep_type);
}
+static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
+ struct mlx5_eswitch_rep *rep, u8 rep_type)
+{
+ if (!rep->rep_if[rep_type].valid)
+ return 0;
+
+ return rep->rep_if[rep_type].load(esw->dev, rep);
+}
+
static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
u8 rep_type)
{
@@ -1269,12 +1289,14 @@ static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
int vport;
int err;
- for (vport = 0; vport < nvports; vport++) {
- rep = &esw->offloads.vport_reps[vport];
- if (!rep->rep_if[rep_type].valid)
- continue;
+ rep = &esw->offloads.vport_reps[UPLINK_REP_INDEX];
+ err = __esw_offloads_load_rep(esw, rep, rep_type);
+ if (err)
+ goto out;
- err = rep->rep_if[rep_type].load(esw->dev, rep);
+ for (vport = MLX5_VPORT_FIRST_VF; vport <= nvports; vport++) {
+ rep = &esw->offloads.vport_reps[vport];
+ err = __esw_offloads_load_rep(esw, rep, rep_type);
if (err)
goto err_reps;
}
@@ -1283,6 +1305,7 @@ static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
err_reps:
esw_offloads_unload_reps_type(esw, vport, rep_type);
+out:
return err;
}
@@ -1440,17 +1463,18 @@ static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw)
esw_destroy_offloads_fdb_tables(esw);
}
-int esw_offloads_init(struct mlx5_eswitch *esw, int nvports)
+int esw_offloads_init(struct mlx5_eswitch *esw, int vf_nvports,
+ int total_nvports)
{
int err;
mutex_init(&esw->fdb_table.offloads.fdb_prio_lock);
- err = esw_offloads_steering_init(esw, nvports);
+ err = esw_offloads_steering_init(esw, total_nvports);
if (err)
return err;
- err = esw_offloads_load_reps(esw, nvports);
+ err = esw_offloads_load_reps(esw, vf_nvports);
if (err)
goto err_reps;
@@ -1481,10 +1505,12 @@ static int esw_offloads_stop(struct mlx5_eswitch *esw,
return err;
}
-void esw_offloads_cleanup(struct mlx5_eswitch *esw, int nvports)
+void esw_offloads_cleanup(struct mlx5_eswitch *esw)
{
+ u16 num_vfs = esw->dev->priv.sriov.num_vfs;
+
esw_offloads_devcom_cleanup(esw);
- esw_offloads_unload_reps(esw, nvports);
+ esw_offloads_unload_reps(esw, num_vfs);
esw_offloads_steering_cleanup(esw);
}
@@ -1822,7 +1848,6 @@ EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_rep);
void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type)
{
-#define UPLINK_REP_INDEX 0
struct mlx5_esw_offload *offloads = &esw->offloads;
struct mlx5_eswitch_rep *rep;
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index b7edcb1dadd8..755aeea19e1c 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -53,6 +53,7 @@ enum {
enum {
MLX5_VPORT_PF = 0x0,
+ MLX5_VPORT_FIRST_VF = 0x1,
MLX5_VPORT_ECPF = 0xfffe,
MLX5_VPORT_UPLINK = 0xffff
};
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 06/13] net/mlx5: E-Switch, Use getter and iterator to access vport/rep
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (4 preceding siblings ...)
2019-02-16 1:34 ` [net-next 05/13] net/mlx5: E-Switch, Split VF and special vports for offloads mode Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 07/13] net/mlx5: E-Switch, Add state to eswitch vport representors Saeed Mahameed
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Saeed Mahameed, Or Gerlitz
From: Bodong Wang <bodong@mellanox.com>
With only PF and VF, it is sufficient to have the vport/rep array
index as the vport number. This is because PF and VF vports numbers
are consecutive serial numbers. In downstream patches with
introducing of ECPF and UPLINK vports, it's not consecutive any more.
Use getter to get specific vport/rep, and use iterator to traversal
a list of vport/rep. This hides the translation between array index
and vport number, and provides flexibility of using different
translation mechanism in the future.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Suggested-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 94 ++++++++++++-------
.../mellanox/mlx5/core/eswitch_offloads.c | 75 ++++++++++-----
2 files changed, 113 insertions(+), 56 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index be6c2931d2a0..d1454f18c0a7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -68,6 +68,26 @@ enum {
MC_ADDR_CHANGE | \
PROMISC_CHANGE)
+/* The vport getter/iterator are only valid after esw->total_vports
+ * and vport->vport are initialized in mlx5_eswitch_init.
+ */
+#define mlx5_esw_for_all_vports(esw, i, vport) \
+ for ((i) = MLX5_VPORT_PF; \
+ (vport) = &(esw)->vports[i], \
+ (i) < (esw)->total_vports; (i)++)
+
+#define mlx5_esw_for_each_vf_vport(esw, i, vport, nvfs) \
+ for ((i) = MLX5_VPORT_FIRST_VF; \
+ (vport) = &(esw)->vports[i], \
+ (i) <= (nvfs); (i)++)
+
+static struct mlx5_vport *mlx5_eswitch_get_vport(struct mlx5_eswitch *esw,
+ u16 vport_num)
+{
+ WARN_ON(vport_num > esw->total_vports - 1);
+ return &esw->vports[vport_num];
+}
+
static int arm_vport_context_events_cmd(struct mlx5_core_dev *dev, u16 vport,
u32 events_mask)
{
@@ -436,17 +456,18 @@ static void update_allmulti_vports(struct mlx5_eswitch *esw,
struct esw_mc_addr *esw_mc)
{
u8 *mac = vaddr->node.addr;
- u16 vport_idx = 0;
+ struct mlx5_vport *vport;
+ u16 i, vport_num;
- for (vport_idx = 0; vport_idx < esw->total_vports; vport_idx++) {
- struct mlx5_vport *vport = &esw->vports[vport_idx];
+ mlx5_esw_for_all_vports(esw, i, vport) {
struct hlist_head *vport_hash = vport->mc_list;
struct vport_addr *iter_vaddr =
l2addr_hash_find(vport_hash,
mac,
struct vport_addr);
+ vport_num = vport->vport;
if (IS_ERR_OR_NULL(vport->allmulti_rule) ||
- vaddr->vport == vport_idx)
+ vaddr->vport == vport_num)
continue;
switch (vaddr->action) {
case MLX5_ACTION_ADD:
@@ -458,14 +479,14 @@ static void update_allmulti_vports(struct mlx5_eswitch *esw,
if (!iter_vaddr) {
esw_warn(esw->dev,
"ALL-MULTI: Failed to add MAC(%pM) to vport[%d] DB\n",
- mac, vport_idx);
+ mac, vport_num);
continue;
}
- iter_vaddr->vport = vport_idx;
+ iter_vaddr->vport = vport_num;
iter_vaddr->flow_rule =
esw_fdb_set_vport_rule(esw,
mac,
- vport_idx);
+ vport_num);
iter_vaddr->mc_promisc = true;
break;
case MLX5_ACTION_DEL:
@@ -564,7 +585,7 @@ static int esw_del_mc_addr(struct mlx5_eswitch *esw, struct vport_addr *vaddr)
static void esw_apply_vport_addr_list(struct mlx5_eswitch *esw,
u16 vport_num, int list_type)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
bool is_uc = list_type == MLX5_NVPRT_LIST_TYPE_UC;
vport_addr_action vport_addr_add;
vport_addr_action vport_addr_del;
@@ -599,7 +620,7 @@ static void esw_apply_vport_addr_list(struct mlx5_eswitch *esw,
static void esw_update_vport_addr_list(struct mlx5_eswitch *esw,
u16 vport_num, int list_type)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
bool is_uc = list_type == MLX5_NVPRT_LIST_TYPE_UC;
u8 (*mac_list)[ETH_ALEN];
struct l2addr_node *node;
@@ -686,7 +707,7 @@ static void esw_update_vport_addr_list(struct mlx5_eswitch *esw,
*/
static void esw_update_vport_mc_promisc(struct mlx5_eswitch *esw, u16 vport_num)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
struct l2addr_node *node;
struct vport_addr *addr;
struct hlist_head *hash;
@@ -722,8 +743,8 @@ static void esw_update_vport_mc_promisc(struct mlx5_eswitch *esw, u16 vport_num)
static void esw_apply_vport_rx_mode(struct mlx5_eswitch *esw, u16 vport_num,
bool promisc, bool mc_promisc)
{
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
struct esw_mc_addr *allmulti_addr = &esw->mc_promisc;
- struct mlx5_vport *vport = &esw->vports[vport_num];
if (IS_ERR_OR_NULL(vport->allmulti_rule) != mc_promisc)
goto promisc;
@@ -764,7 +785,7 @@ static void esw_apply_vport_rx_mode(struct mlx5_eswitch *esw, u16 vport_num,
/* Sync vport rx mode from vport context */
static void esw_update_vport_rx_mode(struct mlx5_eswitch *esw, u16 vport_num)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
int promisc_all = 0;
int promisc_uc = 0;
int promisc_mc = 0;
@@ -1341,8 +1362,8 @@ static void esw_destroy_tsar(struct mlx5_eswitch *esw)
static int esw_vport_enable_qos(struct mlx5_eswitch *esw, int vport_num,
u32 initial_max_rate, u32 initial_bw_share)
{
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
u32 sched_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {0};
- struct mlx5_vport *vport = &esw->vports[vport_num];
struct mlx5_core_dev *dev = esw->dev;
void *vport_elem;
int err = 0;
@@ -1381,7 +1402,7 @@ static int esw_vport_enable_qos(struct mlx5_eswitch *esw, int vport_num,
static void esw_vport_disable_qos(struct mlx5_eswitch *esw, int vport_num)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
int err = 0;
if (!vport->qos.enabled)
@@ -1400,8 +1421,8 @@ static void esw_vport_disable_qos(struct mlx5_eswitch *esw, int vport_num)
static int esw_vport_qos_config(struct mlx5_eswitch *esw, int vport_num,
u32 max_rate, u32 bw_share)
{
+ struct mlx5_vport *vport = mlx5_eswitch_get_vport(esw, vport_num);
u32 sched_ctx[MLX5_ST_SZ_DW(scheduling_context)] = {0};
- struct mlx5_vport *vport = &esw->vports[vport_num];
struct mlx5_core_dev *dev = esw->dev;
void *vport_elem;
u32 bitmask = 0;
@@ -1518,10 +1539,10 @@ static void esw_vport_destroy_drop_counters(struct mlx5_vport *vport)
mlx5_fc_destroy(dev, vport->egress.drop_counter);
}
-static void esw_enable_vport(struct mlx5_eswitch *esw, int vport_num,
+static void esw_enable_vport(struct mlx5_eswitch *esw, struct mlx5_vport *vport,
int enable_events)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ u16 vport_num = vport->vport;
mutex_lock(&esw->state_lock);
WARN_ON(vport->enabled);
@@ -1558,9 +1579,10 @@ static void esw_enable_vport(struct mlx5_eswitch *esw, int vport_num,
mutex_unlock(&esw->state_lock);
}
-static void esw_disable_vport(struct mlx5_eswitch *esw, int vport_num)
+static void esw_disable_vport(struct mlx5_eswitch *esw,
+ struct mlx5_vport *vport)
{
- struct mlx5_vport *vport = &esw->vports[vport_num];
+ u16 vport_num = vport->vport;
if (!vport->enabled)
return;
@@ -1603,7 +1625,7 @@ static int eswitch_vport_event(struct notifier_block *nb,
u16 vport_num;
vport_num = be16_to_cpu(eqe->data.vport_change.vport_num);
- vport = &esw->vports[vport_num];
+ vport = mlx5_eswitch_get_vport(esw, vport_num);
if (vport->enabled)
queue_work(esw->work_queue, &vport->vport_change_handler);
@@ -1615,6 +1637,7 @@ static int eswitch_vport_event(struct notifier_block *nb,
int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
{
+ struct mlx5_vport *vport;
int err;
int i, enabled_events;
@@ -1657,8 +1680,14 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
* 2. FDB/Eswitch is programmed by user space tools
*/
enabled_events = (mode == SRIOV_LEGACY) ? SRIOV_VPORT_EVENTS : 0;
- for (i = 0; i <= nvfs; i++)
- esw_enable_vport(esw, i, enabled_events);
+
+ /* Enable PF vport */
+ vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_PF);
+ esw_enable_vport(esw, vport, enabled_events);
+
+ /* Enable VF vports */
+ mlx5_esw_for_each_vf_vport(esw, i, vport, nvfs)
+ esw_enable_vport(esw, vport, enabled_events);
if (mode == SRIOV_LEGACY) {
MLX5_NB_INIT(&esw->nb, eswitch_vport_event, NIC_VPORT_CHANGE);
@@ -1683,6 +1712,7 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw)
{
struct esw_mc_addr *mc_promisc;
+ struct mlx5_vport *vport;
int old_mode;
int i;
@@ -1697,8 +1727,8 @@ void mlx5_eswitch_disable_sriov(struct mlx5_eswitch *esw)
if (esw->mode == SRIOV_LEGACY)
mlx5_eq_notifier_unregister(esw->dev, &esw->nb);
- for (i = 0; i < esw->total_vports; i++)
- esw_disable_vport(esw, i);
+ mlx5_esw_for_all_vports(esw, i, vport)
+ esw_disable_vport(esw, vport);
if (mc_promisc && mc_promisc->uplink_rule)
mlx5_del_flow_rules(mc_promisc->uplink_rule);
@@ -1725,6 +1755,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
{
int total_vports = MLX5_TOTAL_VPORTS(dev);
struct mlx5_eswitch *esw;
+ struct mlx5_vport *vport;
int vport_num;
int err;
@@ -1757,6 +1788,8 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
goto abort;
}
+ esw->total_vports = total_vports;
+
err = esw_offloads_init_reps(esw);
if (err)
goto abort;
@@ -1765,9 +1798,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
hash_init(esw->offloads.mod_hdr_tbl);
mutex_init(&esw->state_lock);
- for (vport_num = 0; vport_num < total_vports; vport_num++) {
- struct mlx5_vport *vport = &esw->vports[vport_num];
-
+ mlx5_esw_for_all_vports(esw, vport_num, vport) {
vport->vport = vport_num;
vport->info.link_state = MLX5_VPORT_ADMIN_STATE_AUTO;
vport->dev = dev;
@@ -1775,7 +1806,6 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
esw_vport_change_handler);
}
- esw->total_vports = total_vports;
esw->enabled_vports = 0;
esw->mode = SRIOV_NONE;
esw->offloads.inline_mode = MLX5_INLINE_MODE_NONE;
@@ -2017,8 +2047,7 @@ static u32 calculate_vports_min_rate_divider(struct mlx5_eswitch *esw)
u32 max_guarantee = 0;
int i;
- for (i = 0; i < esw->total_vports; i++) {
- evport = &esw->vports[i];
+ mlx5_esw_for_all_vports(esw, i, evport) {
if (!evport->enabled || evport->info.min_rate < max_guarantee)
continue;
max_guarantee = evport->info.min_rate;
@@ -2037,8 +2066,7 @@ static int normalize_vports_min_rate(struct mlx5_eswitch *esw, u32 divider)
int err;
int i;
- for (i = 0; i < esw->total_vports; i++) {
- evport = &esw->vports[i];
+ mlx5_esw_for_all_vports(esw, i, evport) {
if (!evport->enabled)
continue;
vport_min_rate = evport->info.min_rate;
@@ -2053,7 +2081,7 @@ static int normalize_vports_min_rate(struct mlx5_eswitch *esw, u32 divider)
if (bw_share == evport->qos.bw_share)
continue;
- err = esw_vport_qos_config(esw, i, vport_max_rate,
+ err = esw_vport_qos_config(esw, evport->vport, vport_max_rate,
bw_share);
if (!err)
evport->qos.bw_share = bw_share;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 14f7ad67cfe4..4979c7ee0ad7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -56,6 +56,44 @@ enum {
#define UPLINK_REP_INDEX 0
+/* The rep getter/iterator are only valid after esw->total_vports
+ * and vport->vport are initialized in mlx5_eswitch_init.
+ */
+#define mlx5_esw_for_all_reps(esw, i, rep) \
+ for ((i) = MLX5_VPORT_PF; \
+ (rep) = &(esw)->offloads.vport_reps[i], \
+ (i) < (esw)->total_vports; (i)++)
+
+#define mlx5_esw_for_each_vf_rep(esw, i, rep, nvfs) \
+ for ((i) = MLX5_VPORT_FIRST_VF; \
+ (rep) = &(esw)->offloads.vport_reps[i], \
+ (i) <= (nvfs); (i)++)
+
+#define mlx5_esw_for_each_vf_rep_reverse(esw, i, rep, nvfs) \
+ for ((i) = (nvfs); \
+ (rep) = &(esw)->offloads.vport_reps[i], \
+ (i) >= MLX5_VPORT_FIRST_VF; (i)--)
+
+#define mlx5_esw_for_each_vf_vport(esw, vport, nvfs) \
+ for ((vport) = MLX5_VPORT_FIRST_VF; \
+ (vport) <= (nvfs); (vport)++)
+
+#define mlx5_esw_for_each_vf_vport_reverse(esw, vport, nvfs) \
+ for ((vport) = (nvfs); \
+ (vport) >= MLX5_VPORT_FIRST_VF; (vport)--)
+
+static struct mlx5_eswitch_rep *mlx5_eswitch_get_rep(struct mlx5_eswitch *esw,
+ u16 vport_num)
+{
+ u16 idx = vport_num;
+
+ if (vport_num == MLX5_VPORT_UPLINK)
+ idx = UPLINK_REP_INDEX;
+
+ WARN_ON(idx > esw->total_vports - 1);
+ return &esw->offloads.vport_reps[idx];
+}
+
static struct mlx5_flow_table *
esw_get_prio_table(struct mlx5_eswitch *esw, u32 chain, u16 prio, int level);
static void
@@ -604,7 +642,7 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
misc = MLX5_ADDR_OF(fte_match_param, spec->match_value,
misc_parameters);
- for (i = 1; i < nvports; i++) {
+ mlx5_esw_for_each_vf_vport(esw, i, mlx5_core_max_vfs(esw->dev)) {
MLX5_SET(fte_match_set_misc, misc, source_port, i);
flow = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb,
spec, &flow_act, &dest, 1);
@@ -622,7 +660,8 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
return 0;
add_flow_err:
- for (i--; i > 0; i--)
+ nvports = --i;
+ mlx5_esw_for_each_vf_vport_reverse(esw, i, nvports)
mlx5_del_flow_rules(flows[i]);
kvfree(flows);
alloc_flows_err:
@@ -637,7 +676,7 @@ static void esw_del_fdb_peer_miss_rules(struct mlx5_eswitch *esw)
flows = esw->fdb_table.offloads.peer_miss_rules;
- for (i = 1; i < esw->total_vports; i++)
+ mlx5_esw_for_each_vf_vport_reverse(esw, i, mlx5_core_max_vfs(esw->dev))
mlx5_del_flow_rules(flows[i]);
kvfree(flows);
@@ -1229,9 +1268,7 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
offloads = &esw->offloads;
mlx5_query_nic_vport_mac_address(dev, 0, hw_id);
- for (vport = 0; vport < total_vfs; vport++) {
- rep = &offloads->vport_reps[vport];
-
+ mlx5_esw_for_all_reps(esw, vport, rep) {
rep->vport = vport;
ether_addr_copy(rep->hw_id, hw_id);
}
@@ -1256,12 +1293,10 @@ static void esw_offloads_unload_reps_type(struct mlx5_eswitch *esw, int nvports,
struct mlx5_eswitch_rep *rep;
int vport;
- for (vport = nvports; vport >= MLX5_VPORT_FIRST_VF; vport--) {
- rep = &esw->offloads.vport_reps[vport];
+ mlx5_esw_for_each_vf_rep_reverse(esw, vport, rep, nvports)
__esw_offloads_unload_rep(esw, rep, rep_type);
- }
- rep = &esw->offloads.vport_reps[UPLINK_REP_INDEX];
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
__esw_offloads_unload_rep(esw, rep, rep_type);
}
@@ -1289,13 +1324,12 @@ static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
int vport;
int err;
- rep = &esw->offloads.vport_reps[UPLINK_REP_INDEX];
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
err = __esw_offloads_load_rep(esw, rep, rep_type);
if (err)
goto out;
- for (vport = MLX5_VPORT_FIRST_VF; vport <= nvports; vport++) {
- rep = &esw->offloads.vport_reps[vport];
+ mlx5_esw_for_each_vf_rep(esw, vport, rep, nvports) {
err = __esw_offloads_load_rep(esw, rep, rep_type);
if (err)
goto err_reps;
@@ -1304,7 +1338,7 @@ static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
return 0;
err_reps:
- esw_offloads_unload_reps_type(esw, vport, rep_type);
+ esw_offloads_unload_reps_type(esw, --vport, rep_type);
out:
return err;
}
@@ -1848,10 +1882,9 @@ EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_rep);
void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type)
{
- struct mlx5_esw_offload *offloads = &esw->offloads;
struct mlx5_eswitch_rep *rep;
- rep = &offloads->vport_reps[UPLINK_REP_INDEX];
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
return rep->rep_if[rep_type].priv;
}
@@ -1859,13 +1892,9 @@ void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw,
int vport,
u8 rep_type)
{
- struct mlx5_esw_offload *offloads = &esw->offloads;
struct mlx5_eswitch_rep *rep;
- if (vport == MLX5_VPORT_UPLINK)
- vport = UPLINK_REP_INDEX;
-
- rep = &offloads->vport_reps[vport];
+ rep = mlx5_eswitch_get_rep(esw, vport);
if (rep->rep_if[rep_type].valid &&
rep->rep_if[rep_type].get_proto_dev)
@@ -1876,13 +1905,13 @@ EXPORT_SYMBOL(mlx5_eswitch_get_proto_dev);
void *mlx5_eswitch_uplink_get_proto_dev(struct mlx5_eswitch *esw, u8 rep_type)
{
- return mlx5_eswitch_get_proto_dev(esw, UPLINK_REP_INDEX, rep_type);
+ return mlx5_eswitch_get_proto_dev(esw, MLX5_VPORT_UPLINK, rep_type);
}
EXPORT_SYMBOL(mlx5_eswitch_uplink_get_proto_dev);
struct mlx5_eswitch_rep *mlx5_eswitch_vport_rep(struct mlx5_eswitch *esw,
int vport)
{
- return &esw->offloads.vport_reps[vport];
+ return mlx5_eswitch_get_rep(esw, vport);
}
EXPORT_SYMBOL(mlx5_eswitch_vport_rep);
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 07/13] net/mlx5: E-Switch, Add state to eswitch vport representors
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (5 preceding siblings ...)
2019-02-16 1:34 ` [net-next 06/13] net/mlx5: E-Switch, Use getter and iterator to access vport/rep Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 08/13] net/mlx5: E-Switch, Support load/unload reps of specific vport types Saeed Mahameed
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Bodong Wang, Or Gerlitz, Mark Bloch, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
Currently the eswitch vport reps have a valid indicator, which is
set on register and unset on unregister. However, a rep can be loaded
or not loaded when doing unregister, current driver checks if the
vport of that rep is enabled as a flag to imply the rep is loaded.
However, for ECPF, this is not valid as the host PF will enable the
vports for its VFs instead.
Add three states: {unregistered, registered, loaded}, with the
following state changes across different operations:
create: (none) -> unregistered
reg: unregistered -> registered
load: registered -> loaded
unload: loaded -> registered
unreg: registered -> unregistered
Note that the state shall only be updated inside eswitch driver rather
than individual drivers such as ETH or IB.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Suggested-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../mellanox/mlx5/core/eswitch_offloads.c | 31 +++++++++++++------
include/linux/mlx5/eswitch.h | 8 ++++-
2 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 4979c7ee0ad7..c6c9dad69ba8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -364,7 +364,7 @@ static int esw_set_global_vlan_pop(struct mlx5_eswitch *esw, u8 val)
esw_debug(esw->dev, "%s applying global %s policy\n", __func__, val ? "pop" : "none");
for (vf_vport = 1; vf_vport < esw->enabled_vports; vf_vport++) {
rep = &esw->offloads.vport_reps[vf_vport];
- if (!rep->rep_if[REP_ETH].valid)
+ if (rep->rep_if[REP_ETH].state != REP_LOADED)
continue;
err = __mlx5_eswitch_set_vport_vlan(esw, rep->vport, 0, 0, val);
@@ -1256,7 +1256,7 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
struct mlx5_core_dev *dev = esw->dev;
struct mlx5_esw_offload *offloads;
struct mlx5_eswitch_rep *rep;
- u8 hw_id[ETH_ALEN];
+ u8 hw_id[ETH_ALEN], rep_type;
int vport;
esw->offloads.vport_reps = kcalloc(total_vfs,
@@ -1271,6 +1271,9 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
mlx5_esw_for_all_reps(esw, vport, rep) {
rep->vport = vport;
ether_addr_copy(rep->hw_id, hw_id);
+
+ for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++)
+ rep->rep_if[rep_type].state = REP_UNREGISTERED;
}
offloads->vport_reps[0].vport = MLX5_VPORT_UPLINK;
@@ -1281,10 +1284,11 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw,
struct mlx5_eswitch_rep *rep, u8 rep_type)
{
- if (!rep->rep_if[rep_type].valid)
+ if (rep->rep_if[rep_type].state != REP_LOADED)
return;
rep->rep_if[rep_type].unload(rep);
+ rep->rep_if[rep_type].state = REP_REGISTERED;
}
static void esw_offloads_unload_reps_type(struct mlx5_eswitch *esw, int nvports,
@@ -1311,10 +1315,18 @@ static void esw_offloads_unload_reps(struct mlx5_eswitch *esw, int nvports)
static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
struct mlx5_eswitch_rep *rep, u8 rep_type)
{
- if (!rep->rep_if[rep_type].valid)
+ int err = 0;
+
+ if (rep->rep_if[rep_type].state != REP_REGISTERED)
return 0;
- return rep->rep_if[rep_type].load(esw->dev, rep);
+ err = rep->rep_if[rep_type].load(esw->dev, rep);
+ if (err)
+ return err;
+
+ rep->rep_if[rep_type].state = REP_LOADED;
+
+ return 0;
}
static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
@@ -1861,7 +1873,7 @@ void mlx5_eswitch_register_vport_rep(struct mlx5_eswitch *esw,
rep_if->get_proto_dev = __rep_if->get_proto_dev;
rep_if->priv = __rep_if->priv;
- rep_if->valid = true;
+ rep_if->state = REP_REGISTERED;
}
EXPORT_SYMBOL(mlx5_eswitch_register_vport_rep);
@@ -1873,10 +1885,11 @@ void mlx5_eswitch_unregister_vport_rep(struct mlx5_eswitch *esw,
rep = &offloads->vport_reps[vport_index];
- if (esw->mode == SRIOV_OFFLOADS && esw->vports[vport_index].enabled)
+ if (esw->mode == SRIOV_OFFLOADS &&
+ rep->rep_if[rep_type].state == REP_LOADED)
rep->rep_if[rep_type].unload(rep);
- rep->rep_if[rep_type].valid = false;
+ rep->rep_if[rep_type].state = REP_UNREGISTERED;
}
EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_rep);
@@ -1896,7 +1909,7 @@ void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw,
rep = mlx5_eswitch_get_rep(esw, vport);
- if (rep->rep_if[rep_type].valid &&
+ if (rep->rep_if[rep_type].state == REP_LOADED &&
rep->rep_if[rep_type].get_proto_dev)
return rep->rep_if[rep_type].get_proto_dev(rep);
return NULL;
diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
index fab5121ffb8f..e3dbc1bc0917 100644
--- a/include/linux/mlx5/eswitch.h
+++ b/include/linux/mlx5/eswitch.h
@@ -22,6 +22,12 @@ enum {
NUM_REP_TYPES,
};
+enum {
+ REP_UNREGISTERED,
+ REP_REGISTERED,
+ REP_LOADED,
+};
+
struct mlx5_eswitch_rep;
struct mlx5_eswitch_rep_if {
int (*load)(struct mlx5_core_dev *dev,
@@ -29,7 +35,7 @@ struct mlx5_eswitch_rep_if {
void (*unload)(struct mlx5_eswitch_rep *rep);
void *(*get_proto_dev)(struct mlx5_eswitch_rep *rep);
void *priv;
- bool valid;
+ u8 state;
};
struct mlx5_eswitch_rep {
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 08/13] net/mlx5: E-Switch, Support load/unload reps of specific vport types
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (6 preceding siblings ...)
2019-02-16 1:34 ` [net-next 07/13] net/mlx5: E-Switch, Add state to eswitch vport representors Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 09/13] net/mlx5: E-Switch, Centralize repersentor reg/unreg to eswitch driver Saeed Mahameed
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
Currently the driver loads and unloads all reps in an unbreakable
group. However, with ECPF, the reps of special vports such as uplink
and host PF should always be loaded in switchdev mode where the reps
for VFs will be loaded on-demand and unloaded on no-demand. This is
a pre-step for that change.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../mellanox/mlx5/core/eswitch_offloads.c | 109 ++++++++++++++----
1 file changed, 87 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index c6c9dad69ba8..7131d41796fb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1291,25 +1291,47 @@ static void __esw_offloads_unload_rep(struct mlx5_eswitch *esw,
rep->rep_if[rep_type].state = REP_REGISTERED;
}
-static void esw_offloads_unload_reps_type(struct mlx5_eswitch *esw, int nvports,
- u8 rep_type)
+static void __unload_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
- int vport;
-
- mlx5_esw_for_each_vf_rep_reverse(esw, vport, rep, nvports)
- __esw_offloads_unload_rep(esw, rep, rep_type);
rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
__esw_offloads_unload_rep(esw, rep, rep_type);
}
-static void esw_offloads_unload_reps(struct mlx5_eswitch *esw, int nvports)
+static void __unload_reps_vf_vport(struct mlx5_eswitch *esw, int nvports,
+ u8 rep_type)
+{
+ struct mlx5_eswitch_rep *rep;
+ int i;
+
+ mlx5_esw_for_each_vf_rep_reverse(esw, i, rep, nvports)
+ __esw_offloads_unload_rep(esw, rep, rep_type);
+}
+
+static void esw_offloads_unload_vf_reps(struct mlx5_eswitch *esw, int nvports)
+{
+ u8 rep_type = NUM_REP_TYPES;
+
+ while (rep_type-- > 0)
+ __unload_reps_vf_vport(esw, nvports, rep_type);
+}
+
+static void __unload_reps_all_vport(struct mlx5_eswitch *esw, int nvports,
+ u8 rep_type)
+{
+ __unload_reps_vf_vport(esw, nvports, rep_type);
+
+ /* Special vports must be the last to unload. */
+ __unload_reps_special_vport(esw, rep_type);
+}
+
+static void esw_offloads_unload_all_reps(struct mlx5_eswitch *esw, int nvports)
{
u8 rep_type = NUM_REP_TYPES;
while (rep_type-- > 0)
- esw_offloads_unload_reps_type(esw, nvports, rep_type);
+ __unload_reps_all_vport(esw, nvports, rep_type);
}
static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
@@ -1329,39 +1351,82 @@ static int __esw_offloads_load_rep(struct mlx5_eswitch *esw,
return 0;
}
-static int esw_offloads_load_reps_type(struct mlx5_eswitch *esw, int nvports,
- u8 rep_type)
+static int __load_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
- int vport;
int err;
rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
err = __esw_offloads_load_rep(esw, rep, rep_type);
- if (err)
- goto out;
+ return err;
+}
- mlx5_esw_for_each_vf_rep(esw, vport, rep, nvports) {
+static int __load_reps_vf_vport(struct mlx5_eswitch *esw, int nvports,
+ u8 rep_type)
+{
+ struct mlx5_eswitch_rep *rep;
+ int err, i;
+
+ mlx5_esw_for_each_vf_rep(esw, i, rep, nvports) {
err = __esw_offloads_load_rep(esw, rep, rep_type);
if (err)
- goto err_reps;
+ goto err_vf;
}
return 0;
+err_vf:
+ __unload_reps_vf_vport(esw, --i, rep_type);
+ return err;
+}
+
+static int esw_offloads_load_vf_reps(struct mlx5_eswitch *esw, int nvports)
+{
+ u8 rep_type = 0;
+ int err;
+
+ for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++) {
+ err = __load_reps_vf_vport(esw, nvports, rep_type);
+ if (err)
+ goto err_reps;
+ }
+
+ return err;
+
err_reps:
- esw_offloads_unload_reps_type(esw, --vport, rep_type);
-out:
+ while (rep_type-- > 0)
+ __unload_reps_vf_vport(esw, nvports, rep_type);
+ return err;
+}
+
+static int __load_reps_all_vport(struct mlx5_eswitch *esw, int nvports,
+ u8 rep_type)
+{
+ int err;
+
+ /* Special vports must be loaded first. */
+ err = __load_reps_special_vport(esw, rep_type);
+ if (err)
+ return err;
+
+ err = __load_reps_vf_vport(esw, nvports, rep_type);
+ if (err)
+ goto err_vfs;
+
+ return 0;
+
+err_vfs:
+ __unload_reps_special_vport(esw, rep_type);
return err;
}
-static int esw_offloads_load_reps(struct mlx5_eswitch *esw, int nvports)
+static int esw_offloads_load_all_reps(struct mlx5_eswitch *esw, int nvports)
{
u8 rep_type = 0;
int err;
for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++) {
- err = esw_offloads_load_reps_type(esw, nvports, rep_type);
+ err = __load_reps_all_vport(esw, nvports, rep_type);
if (err)
goto err_reps;
}
@@ -1370,7 +1435,7 @@ static int esw_offloads_load_reps(struct mlx5_eswitch *esw, int nvports)
err_reps:
while (rep_type-- > 0)
- esw_offloads_unload_reps_type(esw, nvports, rep_type);
+ __unload_reps_all_vport(esw, nvports, rep_type);
return err;
}
@@ -1520,7 +1585,7 @@ int esw_offloads_init(struct mlx5_eswitch *esw, int vf_nvports,
if (err)
return err;
- err = esw_offloads_load_reps(esw, vf_nvports);
+ err = esw_offloads_load_all_reps(esw, vf_nvports);
if (err)
goto err_reps;
@@ -1556,7 +1621,7 @@ void esw_offloads_cleanup(struct mlx5_eswitch *esw)
u16 num_vfs = esw->dev->priv.sriov.num_vfs;
esw_offloads_devcom_cleanup(esw);
- esw_offloads_unload_reps(esw, num_vfs);
+ esw_offloads_unload_all_reps(esw, num_vfs);
esw_offloads_steering_cleanup(esw);
}
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 09/13] net/mlx5: E-Switch, Centralize repersentor reg/unreg to eswitch driver
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (7 preceding siblings ...)
2019-02-16 1:34 ` [net-next 08/13] net/mlx5: E-Switch, Support load/unload reps of specific vport types Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 10/13] net/mlx5: E-Switch, Assign a different position for uplink rep and vport Saeed Mahameed
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
Eswitch has two users: IB and ETH. They both register repersentors
when mlx5 interface is added, and unregister the repersentors when
mlx5 interface is removed. Ideally, each driver should only deal with
the entities which are unique to itself. However, current IB and ETH
drivers have to perform the following eswitch operations:
1. When registering, specify how many vports to register. This number
is the same for both drivers which is the total available vport
numbers.
2. When unregistering, specify the number of registered vports to do
unregister. Also, unload the repersentors which are already loaded.
It's unnecessary for eswitch driver to hands out the control of above
operations to individual driver users, as they're not unique to each
driver. Instead, such operations should be centralized to eswitch
driver. This consolidates eswitch control flow, and simplified IB and
ETH driver.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/infiniband/hw/mlx5/ib_rep.c | 20 +++------
.../net/ethernet/mellanox/mlx5/core/en_rep.c | 19 +++-----
.../mellanox/mlx5/core/eswitch_offloads.c | 45 +++++++++----------
include/linux/mlx5/eswitch.h | 11 ++---
4 files changed, 39 insertions(+), 56 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
index 99cae9a10195..4700cffb5a00 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.c
+++ b/drivers/infiniband/hw/mlx5/ib_rep.c
@@ -95,26 +95,20 @@ static void *mlx5_ib_vport_get_proto_dev(struct mlx5_eswitch_rep *rep)
void mlx5_ib_register_vport_reps(struct mlx5_core_dev *mdev)
{
struct mlx5_eswitch *esw = mdev->priv.eswitch;
- int total_vports = MLX5_TOTAL_VPORTS(mdev);
struct mlx5_eswitch_rep_if rep_if = {};
- int vport;
-
- for (vport = 0; vport < total_vports; vport++) {
- rep_if.load = mlx5_ib_vport_rep_load;
- rep_if.unload = mlx5_ib_vport_rep_unload;
- rep_if.get_proto_dev = mlx5_ib_vport_get_proto_dev;
- mlx5_eswitch_register_vport_rep(esw, vport, &rep_if, REP_IB);
- }
+
+ rep_if.load = mlx5_ib_vport_rep_load;
+ rep_if.unload = mlx5_ib_vport_rep_unload;
+ rep_if.get_proto_dev = mlx5_ib_vport_get_proto_dev;
+
+ mlx5_eswitch_register_vport_reps(esw, &rep_if, REP_IB);
}
void mlx5_ib_unregister_vport_reps(struct mlx5_core_dev *mdev)
{
struct mlx5_eswitch *esw = mdev->priv.eswitch;
- int total_vports = MLX5_TOTAL_VPORTS(mdev);
- int vport;
- for (vport = total_vports - 1; vport >= 0; vport--)
- mlx5_eswitch_unregister_vport_rep(esw, vport, REP_IB);
+ mlx5_eswitch_unregister_vport_reps(esw, REP_IB);
}
u8 mlx5_ib_eswitch_mode(struct mlx5_eswitch *esw)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index f84889bbe2a0..287d48e5b073 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -1798,25 +1798,18 @@ static void *mlx5e_vport_rep_get_proto_dev(struct mlx5_eswitch_rep *rep)
void mlx5e_rep_register_vport_reps(struct mlx5_core_dev *mdev)
{
struct mlx5_eswitch *esw = mdev->priv.eswitch;
- int total_vfs = MLX5_TOTAL_VPORTS(mdev);
- int vport;
+ struct mlx5_eswitch_rep_if rep_if = {};
- for (vport = 0; vport < total_vfs; vport++) {
- struct mlx5_eswitch_rep_if rep_if = {};
+ rep_if.load = mlx5e_vport_rep_load;
+ rep_if.unload = mlx5e_vport_rep_unload;
+ rep_if.get_proto_dev = mlx5e_vport_rep_get_proto_dev;
- rep_if.load = mlx5e_vport_rep_load;
- rep_if.unload = mlx5e_vport_rep_unload;
- rep_if.get_proto_dev = mlx5e_vport_rep_get_proto_dev;
- mlx5_eswitch_register_vport_rep(esw, vport, &rep_if, REP_ETH);
- }
+ mlx5_eswitch_register_vport_reps(esw, &rep_if, REP_ETH);
}
void mlx5e_rep_unregister_vport_reps(struct mlx5_core_dev *mdev)
{
struct mlx5_eswitch *esw = mdev->priv.eswitch;
- int total_vfs = MLX5_TOTAL_VPORTS(mdev);
- int vport;
- for (vport = total_vfs - 1; vport >= 0; vport--)
- mlx5_eswitch_unregister_vport_rep(esw, vport, REP_ETH);
+ mlx5_eswitch_unregister_vport_reps(esw, REP_ETH);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 7131d41796fb..b702b56c457e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1923,40 +1923,39 @@ int mlx5_devlink_eswitch_encap_mode_get(struct devlink *devlink, u8 *encap)
return 0;
}
-void mlx5_eswitch_register_vport_rep(struct mlx5_eswitch *esw,
- int vport_index,
- struct mlx5_eswitch_rep_if *__rep_if,
- u8 rep_type)
+void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
+ struct mlx5_eswitch_rep_if *__rep_if,
+ u8 rep_type)
{
- struct mlx5_esw_offload *offloads = &esw->offloads;
struct mlx5_eswitch_rep_if *rep_if;
+ struct mlx5_eswitch_rep *rep;
+ int i;
- rep_if = &offloads->vport_reps[vport_index].rep_if[rep_type];
-
- rep_if->load = __rep_if->load;
- rep_if->unload = __rep_if->unload;
- rep_if->get_proto_dev = __rep_if->get_proto_dev;
- rep_if->priv = __rep_if->priv;
+ mlx5_esw_for_all_reps(esw, i, rep) {
+ rep_if = &rep->rep_if[rep_type];
+ rep_if->load = __rep_if->load;
+ rep_if->unload = __rep_if->unload;
+ rep_if->get_proto_dev = __rep_if->get_proto_dev;
+ rep_if->priv = __rep_if->priv;
- rep_if->state = REP_REGISTERED;
+ rep_if->state = REP_REGISTERED;
+ }
}
-EXPORT_SYMBOL(mlx5_eswitch_register_vport_rep);
+EXPORT_SYMBOL(mlx5_eswitch_register_vport_reps);
-void mlx5_eswitch_unregister_vport_rep(struct mlx5_eswitch *esw,
- int vport_index, u8 rep_type)
+void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_type)
{
- struct mlx5_esw_offload *offloads = &esw->offloads;
+ u16 max_vf = mlx5_core_max_vfs(esw->dev);
struct mlx5_eswitch_rep *rep;
+ int i;
- rep = &offloads->vport_reps[vport_index];
-
- if (esw->mode == SRIOV_OFFLOADS &&
- rep->rep_if[rep_type].state == REP_LOADED)
- rep->rep_if[rep_type].unload(rep);
+ if (esw->mode == SRIOV_OFFLOADS)
+ __unload_reps_all_vport(esw, max_vf, rep_type);
- rep->rep_if[rep_type].state = REP_UNREGISTERED;
+ mlx5_esw_for_all_reps(esw, i, rep)
+ rep->rep_if[rep_type].state = REP_UNREGISTERED;
}
-EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_rep);
+EXPORT_SYMBOL(mlx5_eswitch_unregister_vport_reps);
void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type)
{
diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
index e3dbc1bc0917..96d8435421de 100644
--- a/include/linux/mlx5/eswitch.h
+++ b/include/linux/mlx5/eswitch.h
@@ -46,13 +46,10 @@ struct mlx5_eswitch_rep {
u32 vlan_refcount;
};
-void mlx5_eswitch_register_vport_rep(struct mlx5_eswitch *esw,
- int vport_index,
- struct mlx5_eswitch_rep_if *rep_if,
- u8 rep_type);
-void mlx5_eswitch_unregister_vport_rep(struct mlx5_eswitch *esw,
- int vport_index,
- u8 rep_type);
+void mlx5_eswitch_register_vport_reps(struct mlx5_eswitch *esw,
+ struct mlx5_eswitch_rep_if *rep_if,
+ u8 rep_type);
+void mlx5_eswitch_unregister_vport_reps(struct mlx5_eswitch *esw, u8 rep_type);
void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw,
int vport,
u8 rep_type);
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 10/13] net/mlx5: E-Switch, Assign a different position for uplink rep and vport
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (8 preceding siblings ...)
2019-02-16 1:34 ` [net-next 09/13] net/mlx5: E-Switch, Centralize repersentor reg/unreg to eswitch driver Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 11/13] net/mlx5: E-Switch, Consider ECPF vport depends on eswitch ownership Saeed Mahameed
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Bodong Wang, Eli Cohen, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
In offloads mode, the current implementation puts the uplink
representor at index zero of the vport reps array. It is not "natural"
to place it at index 0 since we want to put the representor for vport
0 at index 0 with the introduction of SmartNIC. A separate patch will
handle the case whether a rep is needed for vport 0 (PF vport).
So, we want to have a different placeholder for uplink vport and
representor. It was placed at the end of vport and rep array. Since
vport number can no longer act as an index into the vport or
representors arrays, use functions to map vport numbers to indices
when accessing the vports or representors arrays, and vice versa.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Eli Cohen <eli@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 11 +++++----
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 24 +++++++++++++++++++
.../mellanox/mlx5/core/eswitch_offloads.c | 11 ++-------
include/linux/mlx5/vport.h | 11 ++++++---
4 files changed, 40 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index d1454f18c0a7..bb7f72467df9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -84,8 +84,10 @@ enum {
static struct mlx5_vport *mlx5_eswitch_get_vport(struct mlx5_eswitch *esw,
u16 vport_num)
{
+ u16 idx = mlx5_eswitch_vport_num_to_index(esw, vport_num);
+
WARN_ON(vport_num > esw->total_vports - 1);
- return &esw->vports[vport_num];
+ return &esw->vports[idx];
}
static int arm_vport_context_events_cmd(struct mlx5_core_dev *dev, u16 vport,
@@ -1756,8 +1758,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
int total_vports = MLX5_TOTAL_VPORTS(dev);
struct mlx5_eswitch *esw;
struct mlx5_vport *vport;
- int vport_num;
- int err;
+ int err, i;
if (!MLX5_VPORT_MANAGER(dev))
return 0;
@@ -1798,8 +1799,8 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
hash_init(esw->offloads.mod_hdr_tbl);
mutex_init(&esw->state_lock);
- mlx5_esw_for_all_vports(esw, vport_num, vport) {
- vport->vport = vport_num;
+ mlx5_esw_for_all_vports(esw, i, vport) {
+ vport->vport = mlx5_eswitch_index_to_vport_num(esw, i);
vport->info.link_state = MLX5_VPORT_ADMIN_STATE_AUTO;
vport->dev = dev;
INIT_WORK(&vport->vport_change_handler,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index fd845e6c44d5..2951c1296c3e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -374,6 +374,30 @@ static inline u16 mlx5_eswitch_manager_vport(struct mlx5_core_dev *dev)
MLX5_VPORT_ECPF : MLX5_VPORT_PF;
}
+static inline int mlx5_eswitch_uplink_idx(struct mlx5_eswitch *esw)
+{
+ /* Uplink always locate at the last element of the array.*/
+ return esw->total_vports - 1;
+}
+
+static inline int mlx5_eswitch_vport_num_to_index(struct mlx5_eswitch *esw,
+ u16 vport_num)
+{
+ if (vport_num == MLX5_VPORT_UPLINK)
+ return mlx5_eswitch_uplink_idx(esw);
+
+ return vport_num;
+}
+
+static inline int mlx5_eswitch_index_to_vport_num(struct mlx5_eswitch *esw,
+ int index)
+{
+ if (index == mlx5_eswitch_uplink_idx(esw))
+ return MLX5_VPORT_UPLINK;
+
+ return index;
+}
+
#else /* CONFIG_MLX5_ESWITCH */
/* eswitch API stubs */
static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index b702b56c457e..e787e9212174 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -85,10 +85,7 @@ enum {
static struct mlx5_eswitch_rep *mlx5_eswitch_get_rep(struct mlx5_eswitch *esw,
u16 vport_num)
{
- u16 idx = vport_num;
-
- if (vport_num == MLX5_VPORT_UPLINK)
- idx = UPLINK_REP_INDEX;
+ u16 idx = mlx5_eswitch_vport_num_to_index(esw, vport_num);
WARN_ON(idx > esw->total_vports - 1);
return &esw->offloads.vport_reps[idx];
@@ -1254,7 +1251,6 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
{
int total_vfs = MLX5_TOTAL_VPORTS(esw->dev);
struct mlx5_core_dev *dev = esw->dev;
- struct mlx5_esw_offload *offloads;
struct mlx5_eswitch_rep *rep;
u8 hw_id[ETH_ALEN], rep_type;
int vport;
@@ -1265,19 +1261,16 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
if (!esw->offloads.vport_reps)
return -ENOMEM;
- offloads = &esw->offloads;
mlx5_query_nic_vport_mac_address(dev, 0, hw_id);
mlx5_esw_for_all_reps(esw, vport, rep) {
- rep->vport = vport;
+ rep->vport = mlx5_eswitch_index_to_vport_num(esw, vport);
ether_addr_copy(rep->hw_id, hw_id);
for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++)
rep->rep_if[rep_type].state = REP_UNREGISTERED;
}
- offloads->vport_reps[0].vport = MLX5_VPORT_UPLINK;
-
return 0;
}
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index 755aeea19e1c..134248c02786 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -36,9 +36,14 @@
#include <linux/mlx5/driver.h>
#include <linux/mlx5/device.h>
-#define MLX5_VPORT_PF_PLACEHOLDER (1u)
-#define MLX5_SPECIAL_VPORTS (MLX5_VPORT_PF_PLACEHOLDER)
-#define MLX5_TOTAL_VPORTS(mdev) (MLX5_SPECIAL_VPORTS + mlx5_core_max_vfs(mdev))
+#define MLX5_VPORT_PF_PLACEHOLDER (1u)
+#define MLX5_VPORT_UPLINK_PLACEHOLDER (1u)
+
+#define MLX5_SPECIAL_VPORTS (MLX5_VPORT_PF_PLACEHOLDER + \
+ MLX5_VPORT_UPLINK_PLACEHOLDER)
+
+#define MLX5_TOTAL_VPORTS(mdev) (MLX5_SPECIAL_VPORTS + \
+ mlx5_core_max_vfs(mdev))
#define MLX5_VPORT_MANAGER(mdev) \
(MLX5_CAP_GEN(mdev, vport_group_manager) && \
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 11/13] net/mlx5: E-Switch, Consider ECPF vport depends on eswitch ownership
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (9 preceding siblings ...)
2019-02-16 1:34 ` [net-next 10/13] net/mlx5: E-Switch, Assign a different position for uplink rep and vport Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 12/13] net/mlx5: E-Switch, Load/unload VF reps according to event from host PF Saeed Mahameed
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
ECPF connects to the eswitch through vport 0xfffe. ECPF may or may
not be the eswitch manager depending on firmware configuration.
1. If ECPF is eswitch manager: ECPF will take over the eswitch manager
responsibility. A rep of the host PF shall be created at the ECPF
side for the eswitch manager to control.
2. If ECPF is not eswitch manager: host PF will be the eswitch manager,
ECPF acts similar as a VF to the host PF. Host PF will be aware
of the ECPF vport presence and control it's rep.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 8 +-
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 15 ++++
.../mellanox/mlx5/core/eswitch_offloads.c | 79 ++++++++++++++++++-
include/linux/mlx5/driver.h | 5 ++
include/linux/mlx5/mlx5_ifc.h | 3 +-
include/linux/mlx5/vport.h | 8 +-
6 files changed, 110 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index bb7f72467df9..d2ab1ee19b2a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1667,7 +1667,7 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_ETH);
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
err = esw_offloads_init(esw, nvfs,
- nvfs + MLX5_SPECIAL_VPORTS);
+ nvfs + MLX5_SPECIAL_VPORTS(esw->dev));
}
if (err)
@@ -1687,6 +1687,12 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_PF);
esw_enable_vport(esw, vport, enabled_events);
+ /* Enable ECPF vports */
+ if (mlx5_ecpf_vport_exists(esw->dev)) {
+ vport = mlx5_eswitch_get_vport(esw, MLX5_VPORT_ECPF);
+ esw_enable_vport(esw, vport, enabled_events);
+ }
+
/* Enable VF vports */
mlx5_esw_for_each_vf_vport(esw, i, vport, nvfs)
esw_enable_vport(esw, vport, enabled_events);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 2951c1296c3e..2baa0d71380c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -380,9 +380,20 @@ static inline int mlx5_eswitch_uplink_idx(struct mlx5_eswitch *esw)
return esw->total_vports - 1;
}
+static inline int mlx5_eswitch_ecpf_idx(struct mlx5_eswitch *esw)
+{
+ return esw->total_vports - 2;
+}
+
static inline int mlx5_eswitch_vport_num_to_index(struct mlx5_eswitch *esw,
u16 vport_num)
{
+ if (vport_num == MLX5_VPORT_ECPF) {
+ if (!mlx5_ecpf_vport_exists(esw->dev))
+ esw_warn(esw->dev, "ECPF vport doesn't exist!\n");
+ return mlx5_eswitch_ecpf_idx(esw);
+ }
+
if (vport_num == MLX5_VPORT_UPLINK)
return mlx5_eswitch_uplink_idx(esw);
@@ -392,6 +403,10 @@ static inline int mlx5_eswitch_vport_num_to_index(struct mlx5_eswitch *esw,
static inline int mlx5_eswitch_index_to_vport_num(struct mlx5_eswitch *esw,
int index)
{
+ if (index == mlx5_eswitch_ecpf_idx(esw) &&
+ mlx5_ecpf_vport_exists(esw->dev))
+ return MLX5_VPORT_ECPF;
+
if (index == mlx5_eswitch_uplink_idx(esw))
return MLX5_VPORT_UPLINK;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index e787e9212174..84a33f8e3350 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -639,14 +639,35 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
misc = MLX5_ADDR_OF(fte_match_param, spec->match_value,
misc_parameters);
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ MLX5_SET(fte_match_set_misc, misc, source_port, MLX5_VPORT_PF);
+ flow = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb,
+ spec, &flow_act, &dest, 1);
+ if (IS_ERR(flow)) {
+ err = PTR_ERR(flow);
+ goto add_pf_flow_err;
+ }
+ flows[MLX5_VPORT_PF] = flow;
+ }
+
+ if (mlx5_ecpf_vport_exists(esw->dev)) {
+ MLX5_SET(fte_match_set_misc, misc, source_port, MLX5_VPORT_ECPF);
+ flow = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb,
+ spec, &flow_act, &dest, 1);
+ if (IS_ERR(flow)) {
+ err = PTR_ERR(flow);
+ goto add_ecpf_flow_err;
+ }
+ flows[mlx5_eswitch_ecpf_idx(esw)] = flow;
+ }
+
mlx5_esw_for_each_vf_vport(esw, i, mlx5_core_max_vfs(esw->dev)) {
MLX5_SET(fte_match_set_misc, misc, source_port, i);
flow = mlx5_add_flow_rules(esw->fdb_table.offloads.slow_fdb,
spec, &flow_act, &dest, 1);
if (IS_ERR(flow)) {
err = PTR_ERR(flow);
- esw_warn(esw->dev, "FDB: Failed to add peer miss flow rule err %d\n", err);
- goto add_flow_err;
+ goto add_vf_flow_err;
}
flows[i] = flow;
}
@@ -656,10 +677,18 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
kvfree(spec);
return 0;
-add_flow_err:
+add_vf_flow_err:
nvports = --i;
mlx5_esw_for_each_vf_vport_reverse(esw, i, nvports)
mlx5_del_flow_rules(flows[i]);
+
+ if (mlx5_ecpf_vport_exists(esw->dev))
+ mlx5_del_flow_rules(flows[mlx5_eswitch_ecpf_idx(esw)]);
+add_ecpf_flow_err:
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev))
+ mlx5_del_flow_rules(flows[MLX5_VPORT_PF]);
+add_pf_flow_err:
+ esw_warn(esw->dev, "FDB: Failed to add peer miss flow rule err %d\n", err);
kvfree(flows);
alloc_flows_err:
kvfree(spec);
@@ -676,6 +705,12 @@ static void esw_del_fdb_peer_miss_rules(struct mlx5_eswitch *esw)
mlx5_esw_for_each_vf_vport_reverse(esw, i, mlx5_core_max_vfs(esw->dev))
mlx5_del_flow_rules(flows[i]);
+ if (mlx5_ecpf_vport_exists(esw->dev))
+ mlx5_del_flow_rules(flows[mlx5_eswitch_ecpf_idx(esw)]);
+
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev))
+ mlx5_del_flow_rules(flows[MLX5_VPORT_PF]);
+
kvfree(flows);
}
@@ -1288,6 +1323,16 @@ static void __unload_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
{
struct mlx5_eswitch_rep *rep;
+ if (mlx5_ecpf_vport_exists(esw->dev)) {
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_ECPF);
+ __esw_offloads_unload_rep(esw, rep, rep_type);
+ }
+
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_PF);
+ __esw_offloads_unload_rep(esw, rep, rep_type);
+ }
+
rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
__esw_offloads_unload_rep(esw, rep, rep_type);
}
@@ -1351,6 +1396,34 @@ static int __load_reps_special_vport(struct mlx5_eswitch *esw, u8 rep_type)
rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
err = __esw_offloads_load_rep(esw, rep, rep_type);
+ if (err)
+ return err;
+
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_PF);
+ err = __esw_offloads_load_rep(esw, rep, rep_type);
+ if (err)
+ goto err_pf;
+ }
+
+ if (mlx5_ecpf_vport_exists(esw->dev)) {
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_ECPF);
+ err = __esw_offloads_load_rep(esw, rep, rep_type);
+ if (err)
+ goto err_ecpf;
+ }
+
+ return 0;
+
+err_ecpf:
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_PF);
+ __esw_offloads_unload_rep(esw, rep, rep_type);
+ }
+
+err_pf:
+ rep = mlx5_eswitch_get_rep(esw, MLX5_VPORT_UPLINK);
+ __esw_offloads_unload_rep(esw, rep, rep_type);
return err;
}
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index c5454f985e1d..c2de50f02b33 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -1088,6 +1088,11 @@ static inline bool mlx5_core_is_ecpf_esw_manager(struct mlx5_core_dev *dev)
return dev->caps.embedded_cpu && MLX5_CAP_GEN(dev, eswitch_manager);
}
+static inline bool mlx5_ecpf_vport_exists(struct mlx5_core_dev *dev)
+{
+ return mlx5_core_is_pf(dev) && MLX5_CAP_ESW(dev, ecpf_vport_exists);
+}
+
#define MLX5_HOST_PF_MAX_VFS (127u)
static inline u16 mlx5_core_max_vfs(struct mlx5_core_dev *dev)
{
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 5decffe565fb..b7bb774b57b0 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -631,7 +631,8 @@ struct mlx5_ifc_e_switch_cap_bits {
u8 vport_svlan_insert[0x1];
u8 vport_cvlan_insert_if_not_exist[0x1];
u8 vport_cvlan_insert_overwrite[0x1];
- u8 reserved_at_5[0x17];
+ u8 reserved_at_5[0x16];
+ u8 ecpf_vport_exists[0x1];
u8 counter_eswitch_affinity[0x1];
u8 merged_eswitch[0x1];
u8 nic_vport_node_guid_modify[0x1];
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index 134248c02786..0eef548b9946 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -38,11 +38,13 @@
#define MLX5_VPORT_PF_PLACEHOLDER (1u)
#define MLX5_VPORT_UPLINK_PLACEHOLDER (1u)
+#define MLX5_VPORT_ECPF_PLACEHOLDER(mdev) (mlx5_ecpf_vport_exists(mdev))
-#define MLX5_SPECIAL_VPORTS (MLX5_VPORT_PF_PLACEHOLDER + \
- MLX5_VPORT_UPLINK_PLACEHOLDER)
+#define MLX5_SPECIAL_VPORTS(mdev) (MLX5_VPORT_PF_PLACEHOLDER + \
+ MLX5_VPORT_UPLINK_PLACEHOLDER + \
+ MLX5_VPORT_ECPF_PLACEHOLDER(mdev))
-#define MLX5_TOTAL_VPORTS(mdev) (MLX5_SPECIAL_VPORTS + \
+#define MLX5_TOTAL_VPORTS(mdev) (MLX5_SPECIAL_VPORTS(mdev) + \
mlx5_core_max_vfs(mdev))
#define MLX5_VPORT_MANAGER(mdev) \
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 12/13] net/mlx5: E-Switch, Load/unload VF reps according to event from host PF
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (10 preceding siblings ...)
2019-02-16 1:34 ` [net-next 11/13] net/mlx5: E-Switch, Consider ECPF vport depends on eswitch ownership Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 1:34 ` [net-next 13/13] net/mlx5: E-Switch, Allow transition to offloads mode for ECPF Saeed Mahameed
2019-02-16 20:11 ` [pull request][net-next 00/13] Mellanox, BlueField SmartNIC David Miller
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
When host PF changes the number of VFs, the ECPF esw driver will get
a FW event. It should query the number of VFs enabled by host PF and
update the VF reps accordingly. Note that host PF can't change the
number of VFs dynamically, it has to reset the number of VFs to 0
before changing to a new positive number.
The host event is registered when driver is moving to switchdev mode,
and it's the last step to do in esw_offloads_init. It's unregistered
and the work queue is flushed when driver quits from switchdev mode.
In this way, the host event and devlink command are serialized.
When driver is enabling switchdev mode, pay attention to the following
two facts:
1. Host PF must not have VF initialized as the flow table in ECPF has
ENCAP enabled as default. Such flow table can't be created with
existing initialized VFs.
2. ECPF doesn't know how many VFs the host PF will enable, ECPF
offloads flow steering shall create the flow table/groups based on
the max number of VFs possibly supported by host PF.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 17 ++++-
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 11 +++
.../mellanox/mlx5/core/eswitch_offloads.c | 71 ++++++++++++++++++-
3 files changed, 96 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index d2ab1ee19b2a..e18af31336e6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -39,6 +39,7 @@
#include "lib/eq.h"
#include "eswitch.h"
#include "fs_core.h"
+#include "ecpf.h"
enum {
MLX5_ACTION_NONE = 0,
@@ -1639,6 +1640,7 @@ static int eswitch_vport_event(struct notifier_block *nb,
int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
{
+ int vf_nvports = 0, total_nvports = 0;
struct mlx5_vport *vport;
int err;
int i, enabled_events;
@@ -1657,6 +1659,18 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
esw_info(esw->dev, "E-Switch enable SRIOV: nvfs(%d) mode (%d)\n", nvfs, mode);
+ if (mode == SRIOV_OFFLOADS) {
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ err = mlx5_query_host_params_num_vfs(esw->dev, &vf_nvports);
+ if (err)
+ return err;
+ total_nvports = esw->total_vports;
+ } else {
+ vf_nvports = nvfs;
+ total_nvports = nvfs + MLX5_SPECIAL_VPORTS(esw->dev);
+ }
+ }
+
esw->mode = mode;
mlx5_lag_update(esw->dev);
@@ -1666,8 +1680,7 @@ int mlx5_eswitch_enable_sriov(struct mlx5_eswitch *esw, int nvfs, int mode)
} else {
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_ETH);
mlx5_reload_interface(esw->dev, MLX5_INTERFACE_PROTOCOL_IB);
- err = esw_offloads_init(esw, nvfs,
- nvfs + MLX5_SPECIAL_VPORTS(esw->dev));
+ err = esw_offloads_init(esw, vf_nvports, total_nvports);
}
if (err)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 2baa0d71380c..af5581a57e56 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -182,6 +182,16 @@ struct esw_mc_addr { /* SRIOV only */
u32 refcnt;
};
+struct mlx5_host_work {
+ struct work_struct work;
+ struct mlx5_eswitch *esw;
+};
+
+struct mlx5_host_info {
+ struct mlx5_nb nb;
+ u16 num_vfs;
+};
+
struct mlx5_eswitch {
struct mlx5_core_dev *dev;
struct mlx5_nb nb;
@@ -206,6 +216,7 @@ struct mlx5_eswitch {
int mode;
int nvports;
u16 manager_vport;
+ struct mlx5_host_info host_info;
};
void esw_offloads_cleanup(struct mlx5_eswitch *esw);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 84a33f8e3350..91c4095ac79e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -40,6 +40,8 @@
#include "en.h"
#include "fs_core.h"
#include "lib/devcom.h"
+#include "ecpf.h"
+#include "lib/eq.h"
enum {
FDB_FAST_PATH = 0,
@@ -1640,6 +1642,57 @@ static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw)
esw_destroy_offloads_fdb_tables(esw);
}
+static void esw_host_params_event_handler(struct work_struct *work)
+{
+ struct mlx5_host_work *host_work;
+ struct mlx5_eswitch *esw;
+ int err, num_vf = 0;
+
+ host_work = container_of(work, struct mlx5_host_work, work);
+ esw = host_work->esw;
+
+ err = mlx5_query_host_params_num_vfs(esw->dev, &num_vf);
+ if (err || num_vf == esw->host_info.num_vfs)
+ goto out;
+
+ /* Number of VFs can only change from "0 to x" or "x to 0". */
+ if (esw->host_info.num_vfs > 0) {
+ esw_offloads_unload_vf_reps(esw, esw->host_info.num_vfs);
+ } else {
+ err = esw_offloads_load_vf_reps(esw, num_vf);
+
+ if (err)
+ goto out;
+ }
+
+ esw->host_info.num_vfs = num_vf;
+
+out:
+ kfree(host_work);
+}
+
+static int esw_host_params_event(struct notifier_block *nb,
+ unsigned long type, void *data)
+{
+ struct mlx5_host_work *host_work;
+ struct mlx5_host_info *host_info;
+ struct mlx5_eswitch *esw;
+
+ host_work = kzalloc(sizeof(*host_work), GFP_ATOMIC);
+ if (!host_work)
+ return NOTIFY_DONE;
+
+ host_info = mlx5_nb_cof(nb, struct mlx5_host_info, nb);
+ esw = container_of(host_info, struct mlx5_eswitch, host_info);
+
+ host_work->esw = esw;
+
+ INIT_WORK(&host_work->work, esw_host_params_event_handler);
+ queue_work(esw->work_queue, &host_work->work);
+
+ return NOTIFY_OK;
+}
+
int esw_offloads_init(struct mlx5_eswitch *esw, int vf_nvports,
int total_nvports)
{
@@ -1656,6 +1709,14 @@ int esw_offloads_init(struct mlx5_eswitch *esw, int vf_nvports,
goto err_reps;
esw_offloads_devcom_init(esw);
+
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ MLX5_NB_INIT(&esw->host_info.nb, esw_host_params_event,
+ HOST_PARAMS_CHANGE);
+ mlx5_eq_notifier_register(esw->dev, &esw->host_info.nb);
+ esw->host_info.num_vfs = vf_nvports;
+ }
+
return 0;
err_reps:
@@ -1684,7 +1745,15 @@ static int esw_offloads_stop(struct mlx5_eswitch *esw,
void esw_offloads_cleanup(struct mlx5_eswitch *esw)
{
- u16 num_vfs = esw->dev->priv.sriov.num_vfs;
+ u16 num_vfs;
+
+ if (mlx5_core_is_ecpf_esw_manager(esw->dev)) {
+ mlx5_eq_notifier_unregister(esw->dev, &esw->host_info.nb);
+ flush_workqueue(esw->work_queue);
+ num_vfs = esw->host_info.num_vfs;
+ } else {
+ num_vfs = esw->dev->priv.sriov.num_vfs;
+ }
esw_offloads_devcom_cleanup(esw);
esw_offloads_unload_all_reps(esw, num_vfs);
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [net-next 13/13] net/mlx5: E-Switch, Allow transition to offloads mode for ECPF
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (11 preceding siblings ...)
2019-02-16 1:34 ` [net-next 12/13] net/mlx5: E-Switch, Load/unload VF reps according to event from host PF Saeed Mahameed
@ 2019-02-16 1:34 ` Saeed Mahameed
2019-02-16 20:11 ` [pull request][net-next 00/13] Mellanox, BlueField SmartNIC David Miller
13 siblings, 0 replies; 15+ messages in thread
From: Saeed Mahameed @ 2019-02-16 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Bodong Wang, Or Gerlitz, Saeed Mahameed
From: Bodong Wang <bodong@mellanox.com>
Currently, the e-switch driver requires going to legacy mode before
changing to the offloads mode. This makes sense for regular case as
the legacy mode is done by creating VFs.
However, it's problematic when ECPF is the eswitch manager. In such
case, ECPF will control the vports on peer host including the peer
PF and VFs. But ECPF doesn't need and shall not create VFs as the
VFs are created in the peer PF host.
Grant ECPF the ability to change from none to the offloads mode. Note
that currently the only way to go back to none mode is by unloading
the ECPF driver.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 91c4095ac79e..f2260391be5b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1250,7 +1250,8 @@ static int esw_offloads_start(struct mlx5_eswitch *esw,
{
int err, err1, num_vfs = esw->dev->priv.sriov.num_vfs;
- if (esw->mode != SRIOV_LEGACY) {
+ if (esw->mode != SRIOV_LEGACY &&
+ !mlx5_core_is_ecpf_esw_manager(esw->dev)) {
NL_SET_ERR_MSG_MOD(extack,
"Can't set offloads mode, SRIOV legacy not enabled");
return -EINVAL;
@@ -1846,7 +1847,8 @@ static int mlx5_devlink_eswitch_check(struct devlink *devlink)
if(!MLX5_ESWITCH_MANAGER(dev))
return -EPERM;
- if (dev->priv.eswitch->mode == SRIOV_NONE)
+ if (dev->priv.eswitch->mode == SRIOV_NONE &&
+ !mlx5_core_is_ecpf_esw_manager(dev))
return -EOPNOTSUPP;
return 0;
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [pull request][net-next 00/13] Mellanox, BlueField SmartNIC
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
` (12 preceding siblings ...)
2019-02-16 1:34 ` [net-next 13/13] net/mlx5: E-Switch, Allow transition to offloads mode for ECPF Saeed Mahameed
@ 2019-02-16 20:11 ` David Miller
13 siblings, 0 replies; 15+ messages in thread
From: David Miller @ 2019-02-16 20:11 UTC (permalink / raw)
To: saeedm; +Cc: netdev
From: Saeed Mahameed <saeedm@mellanox.com>
Date: Fri, 15 Feb 2019 17:34:39 -0800
> This series adds the support for Melanox BlueField SmartNIC.
> For more information please see tag log below.
>
> Please note the merge commit of mlx5-next at the base of the pull request:
> 259fae5a2cff ("Merge branch 'mlx5-next' of git://git.kernel.org/.../mellanox/linux")
>
> Please pull and let me know if there is any problem.
Pulled, thanks for all of the contextual info, it really helps.
I'll push this out after my build tests complete.
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2019-02-16 20:13 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-16 1:34 [pull request][net-next 00/13] Mellanox, BlueField SmartNIC Saeed Mahameed
2019-02-16 1:34 ` [net-next 01/13] net/mlx5: Correctly set LAG mode for ECPF Saeed Mahameed
2019-02-16 1:34 ` [net-next 02/13] net/mlx5: E-Switch, Properly refer to the esw manager vport Saeed Mahameed
2019-02-16 1:34 ` [net-next 03/13] net/mlx5: E-Switch, Properly refer to host PF vport as other vport Saeed Mahameed
2019-02-16 1:34 ` [net-next 04/13] net/mlx5: E-Switch, Refactor offloads flow steering init/cleanup Saeed Mahameed
2019-02-16 1:34 ` [net-next 05/13] net/mlx5: E-Switch, Split VF and special vports for offloads mode Saeed Mahameed
2019-02-16 1:34 ` [net-next 06/13] net/mlx5: E-Switch, Use getter and iterator to access vport/rep Saeed Mahameed
2019-02-16 1:34 ` [net-next 07/13] net/mlx5: E-Switch, Add state to eswitch vport representors Saeed Mahameed
2019-02-16 1:34 ` [net-next 08/13] net/mlx5: E-Switch, Support load/unload reps of specific vport types Saeed Mahameed
2019-02-16 1:34 ` [net-next 09/13] net/mlx5: E-Switch, Centralize repersentor reg/unreg to eswitch driver Saeed Mahameed
2019-02-16 1:34 ` [net-next 10/13] net/mlx5: E-Switch, Assign a different position for uplink rep and vport Saeed Mahameed
2019-02-16 1:34 ` [net-next 11/13] net/mlx5: E-Switch, Consider ECPF vport depends on eswitch ownership Saeed Mahameed
2019-02-16 1:34 ` [net-next 12/13] net/mlx5: E-Switch, Load/unload VF reps according to event from host PF Saeed Mahameed
2019-02-16 1:34 ` [net-next 13/13] net/mlx5: E-Switch, Allow transition to offloads mode for ECPF Saeed Mahameed
2019-02-16 20:11 ` [pull request][net-next 00/13] Mellanox, BlueField SmartNIC David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).