* [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support
@ 2019-04-14 21:12 Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name Ori Kam
` (10 more replies)
0 siblings, 11 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Currently MLX5 PMD supports 3 flow engines:
Verbs, Direct Verbs and TCF. The first two engines are for Nic steering
while the TCF is for E-Switch steering.
This series add E-Switch steering support also for the DV engine.
In order to support the new capability there should be support from
both the RDMA and from the NIC.
Ori Kam (9):
net/mlx5: fix translate vport function name
net/mlx5: fix menson compilation with Direct Rules
net/mlx5: add Direct Rules configuration support
net/mlx5: add validation for Direct Rule E-Switch
net/mlx5: add port ID item to Direct Verbs
net/mlx5: add transfer attribute to matcher
net/mlx5: add port ID action to Direct Verbs
net/mlx5: add Forward Database table type
net/mlx5: add drop action to Direct Verbs E-Switch
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 4 +
drivers/net/mlx5/mlx5.c | 61 +++-
drivers/net/mlx5/mlx5.h | 17 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 42 +++
drivers/net/mlx5/mlx5_ethdev.c | 39 +++
drivers/net/mlx5/mlx5_flow.c | 3 +-
drivers/net/mlx5/mlx5_flow.h | 19 ++
drivers/net/mlx5/mlx5_flow_dv.c | 597 +++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 26 ++
drivers/net/mlx5/mlx5_glue.h | 2 +
drivers/net/mlx5/mlx5_prm.h | 328 +++++++++++++++++++++
12 files changed, 1071 insertions(+), 72 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-16 23:47 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules Ori Kam
` (9 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Modify the translate vport function to match other translate items
naming convestions.
Fixes: 0fe3f18f78d8 ("net/mlx5: add source vport match to the ingress rules")
Cc: viacheslavo@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 3862b26..7b582f0 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3087,8 +3087,8 @@ struct field_modify_info modify_tcp[] = {
* Mask
*/
static void
-flow_dv_translate_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
{
void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
@@ -3492,10 +3492,10 @@ struct field_modify_info modify_tcp[] = {
* Add matching on source vport index only
* for ingress rules in E-Switch configurations.
*/
- flow_dv_translate_source_vport(matcher.mask.buf,
- dev_flow->dv.value.buf,
- priv->vport_id,
- 0xffff);
+ flow_dv_translate_item_source_vport(matcher.mask.buf,
+ dev_flow->dv.value.buf,
+ priv->vport_id,
+ 0xffff);
}
for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) {
int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-17 0:01 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support Ori Kam
` (8 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
The meson build was missing the define for Direct Rules.
Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
Cc: orika@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/meson.build | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a4c684e..0037e15 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -111,6 +111,8 @@ if build
'mlx5dv_devx_obj_create' ],
[ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
+ [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-17 1:42 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
` (7 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit provides the basic configuration needed in order to
support Direct Rules eswitch.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 2 +
drivers/net/mlx5/mlx5.c | 52 +++++-
drivers/net/mlx5/mlx5.h | 12 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 42 +++++
drivers/net/mlx5/mlx5_flow.c | 2 +-
drivers/net/mlx5/mlx5_prm.h | 328 ++++++++++++++++++++++++++++++++++++++
7 files changed, 437 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 93bc869..2b72a33 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
enum MLX5DV_DR_NS_TYPE_TERMINATING \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR_ESWITCH \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 0037e15..9dfd28d 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -113,6 +113,8 @@ if build
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
[ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
'MLX5DV_DR_NS_TYPE_TERMINATING' ],
+ [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 9ff50df..938ba1c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -101,6 +101,9 @@
/* Allow L3 VXLAN flow creation. */
#define MLX5_L3_VXLAN_EN "l3_vxlan_en"
+/* Activate DV eswitch flow steering. */
+#define MLX5_DV_ESWITCH_EN "dv_eswitch_en"
+
/* Activate DV flow steering. */
#define MLX5_DV_FLOW_EN "dv_flow_en"
@@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
}
pthread_mutex_init(&sh->dv_mutex, NULL);
sh->tx_ns = ns;
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (priv->config.dv_eswitch_en) {
+ ns = mlx5_glue->dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
+ if (!ns) {
+ DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ sh->fdb_ns = ns;
+ }
+#endif
sh->dv_refcnt++;
priv->dr_shared = 1;
return 0;
@@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
return err;
#else
(void)priv;
@@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
+#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
(void)priv;
@@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
config->l3_vxlan_en = !!tmp;
} else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
config->vf_nl_en = !!tmp;
+ } else if (strcmp(MLX5_DV_ESWITCH_EN, key) == 0) {
+ config->dv_eswitch_en = !!tmp;
} else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
config->dv_flow_en = !!tmp;
} else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
@@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
MLX5_RX_VEC_EN,
MLX5_L3_VXLAN_EN,
MLX5_VF_NL_EN,
+ MLX5_DV_ESWITCH_EN,
MLX5_DV_FLOW_EN,
MLX5_MR_EXT_MEMSEG_EN,
MLX5_REPRESENTOR,
@@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
- if (config.dv_flow_en) {
- err = mlx5_alloc_shared_dr(priv);
- if (err)
- goto error;
- }
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
@@ -1484,8 +1507,26 @@ struct mlx5_dev_spawn_data {
* Verbs context returned by ibv_open_device().
*/
mlx5_link_update(eth_dev, 0);
+#ifdef HAVE_IBV_DEVX_OBJ
+ err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
+ if (err) {
+ err = -err;
+ goto error;
+ }
+#endif
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (!config.hca_attr.eswitch_manager)
+ config.dv_eswitch_en = 0;
+#else
+ config.dv_eswitch_en = 0;
+#endif
/* Store device configuration on private structure. */
priv->config = config;
+ if (config.dv_flow_en) {
+ err = mlx5_alloc_shared_dr(priv);
+ if (err)
+ goto error;
+ }
/* Supported Verbs flow priority number detection. */
err = mlx5_flow_discover_priorities(eth_dev);
if (err < 0) {
@@ -1876,6 +1917,7 @@ struct mlx5_dev_spawn_data {
.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
.min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
},
+ .dv_eswitch_en = 1,
};
/* Device specific configuration. */
switch (pci_dev->id.device_id) {
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 14c7f3c..33a4127 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
int id; /* Flow counter ID */
};
+/* HCA attributes. */
+struct mlx5_hca_attr {
+ uint32_t eswitch_manager:1;
+};
+
/* Flow list . */
TAILQ_HEAD(mlx5_flows, rte_flow);
@@ -171,6 +176,7 @@ struct mlx5_dev_config {
/* Whether memseg should be extended for MR creation. */
unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
+ unsigned int dv_eswitch_en:1; /* Enable eswitch DV flow. */
unsigned int dv_flow_en:1; /* Enable DV flow. */
unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
unsigned int devx:1; /* Whether devx interface is available or not. */
@@ -192,6 +198,7 @@ struct mlx5_dev_config {
int txqs_inline; /* Queue number threshold for inlining. */
int txqs_vec; /* Queue number threshold for vectorized Tx. */
int inline_max_packet_sz; /* Max packet size for inlining. */
+ struct mlx5_hca_attr hca_attr; /* HCA attributes. */
};
/**
@@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
};
#define MLX5_MAX_TABLES 1024
+#define MLX5_MAX_TABLES_FDB 32
#define MLX5_GROUP_FACTOR 1
/*
@@ -260,6 +268,8 @@ struct mlx5_ibv_shared {
/* Shared DV/DR flow data section. */
pthread_mutex_t dv_mutex; /* DV context mutex. */
uint32_t dv_refcnt; /* DV/DR data reference counter. */
+ void *fdb_ns; /* FDB Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
void *rx_ns; /* RX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
/* RX Direct Rules tables. */
@@ -539,4 +549,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct ibv_context *ctx,
int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set *dcx,
int clear,
uint64_t *pkts, uint64_t *bytes);
+int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr);
#endif /* RTE_PMD_MLX5_H_ */
diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c b/drivers/net/mlx5/mlx5_devx_cmds.c
index a9dff58..3caea41 100644
--- a/drivers/net/mlx5/mlx5_devx_cmds.c
+++ b/drivers/net/mlx5/mlx5_devx_cmds.c
@@ -105,3 +105,45 @@ int mlx5_devx_cmd_flow_counter_free(struct mlx5dv_devx_obj *obj)
*bytes = MLX5_GET64(traffic_counter, stats, octets);
return 0;
}
+
+/**
+ * Query HCA attributes.
+ *
+ * @param[in] ctx
+ * ibv contexts returned from mlx5dv_open_device.
+ * @param[out] attr
+ * Attributes device values.
+ *
+ * @return
+ * 0 on success, a negative value otherwise.
+ */
+int
+mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
+ void *hcattr;
+ int status, syndrome, rc;
+
+ MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
+ MLX5_SET(query_hca_cap_in, in, op_mod,
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
+ MLX5_HCA_CAP_OPMOD_GET_CUR);
+
+ rc = mlx5_glue->devx_general_cmd(ctx,
+ in, sizeof(in), out, sizeof(out));
+ if (rc)
+ return rc;
+ status = MLX5_GET(query_hca_cap_out, out, status);
+ syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
+ if (status) {
+ DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
+ "status %x, syndrome = %x",
+ status, syndrome);
+ return -1;
+ }
+ hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
+ attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);
+ return 0;
+}
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index a0683ee..83abc14 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
struct mlx5_priv *priv = dev->data->dev_private;
enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
- if (attr->transfer)
+ if (attr->transfer && !priv->config.dv_eswitch_en)
type = MLX5_FLOW_TYPE_TCF;
else
type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index b15266f..b25d4e8 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -529,6 +529,7 @@ enum {
};
enum {
+ MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
};
@@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
u8 flow_counter_id[0x20];
};
+enum {
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
+ MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
+};
+
+enum {
+ MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
+ MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
+};
+
+struct mlx5_ifc_cmd_hca_cap_bits {
+ u8 reserved_at_0[0x30];
+ u8 vhca_id[0x10];
+ u8 reserved_at_40[0x40];
+ u8 log_max_srq_sz[0x8];
+ u8 log_max_qp_sz[0x8];
+ u8 reserved_at_90[0xb];
+ u8 log_max_qp[0x5];
+ u8 reserved_at_a0[0xb];
+ u8 log_max_srq[0x5];
+ u8 reserved_at_b0[0x10];
+ u8 reserved_at_c0[0x8];
+ u8 log_max_cq_sz[0x8];
+ u8 reserved_at_d0[0xb];
+ u8 log_max_cq[0x5];
+ u8 log_max_eq_sz[0x8];
+ u8 reserved_at_e8[0x2];
+ u8 log_max_mkey[0x6];
+ u8 reserved_at_f0[0x8];
+ u8 dump_fill_mkey[0x1];
+ u8 reserved_at_f9[0x3];
+ u8 log_max_eq[0x4];
+ u8 max_indirection[0x8];
+ u8 fixed_buffer_size[0x1];
+ u8 log_max_mrw_sz[0x7];
+ u8 force_teardown[0x1];
+ u8 reserved_at_111[0x1];
+ u8 log_max_bsf_list_size[0x6];
+ u8 umr_extended_translation_offset[0x1];
+ u8 null_mkey[0x1];
+ u8 log_max_klm_list_size[0x6];
+ u8 reserved_at_120[0xa];
+ u8 log_max_ra_req_dc[0x6];
+ u8 reserved_at_130[0xa];
+ u8 log_max_ra_res_dc[0x6];
+ u8 reserved_at_140[0xa];
+ u8 log_max_ra_req_qp[0x6];
+ u8 reserved_at_150[0xa];
+ u8 log_max_ra_res_qp[0x6];
+ u8 end_pad[0x1];
+ u8 cc_query_allowed[0x1];
+ u8 cc_modify_allowed[0x1];
+ u8 start_pad[0x1];
+ u8 cache_line_128byte[0x1];
+ u8 reserved_at_165[0xa];
+ u8 qcam_reg[0x1];
+ u8 gid_table_size[0x10];
+ u8 out_of_seq_cnt[0x1];
+ u8 vport_counters[0x1];
+ u8 retransmission_q_counters[0x1];
+ u8 debug[0x1];
+ u8 modify_rq_counter_set_id[0x1];
+ u8 rq_delay_drop[0x1];
+ u8 max_qp_cnt[0xa];
+ u8 pkey_table_size[0x10];
+ u8 vport_group_manager[0x1];
+ u8 vhca_group_manager[0x1];
+ u8 ib_virt[0x1];
+ u8 eth_virt[0x1];
+ u8 vnic_env_queue_counters[0x1];
+ u8 ets[0x1];
+ u8 nic_flow_table[0x1];
+ u8 eswitch_manager[0x1];
+ u8 device_memory[0x1];
+ u8 mcam_reg[0x1];
+ u8 pcam_reg[0x1];
+ u8 local_ca_ack_delay[0x5];
+ u8 port_module_event[0x1];
+ u8 enhanced_error_q_counters[0x1];
+ u8 ports_check[0x1];
+ u8 reserved_at_1b3[0x1];
+ u8 disable_link_up[0x1];
+ u8 beacon_led[0x1];
+ u8 port_type[0x2];
+ u8 num_ports[0x8];
+ u8 reserved_at_1c0[0x1];
+ u8 pps[0x1];
+ u8 pps_modify[0x1];
+ u8 log_max_msg[0x5];
+ u8 reserved_at_1c8[0x4];
+ u8 max_tc[0x4];
+ u8 temp_warn_event[0x1];
+ u8 dcbx[0x1];
+ u8 general_notification_event[0x1];
+ u8 reserved_at_1d3[0x2];
+ u8 fpga[0x1];
+ u8 rol_s[0x1];
+ u8 rol_g[0x1];
+ u8 reserved_at_1d8[0x1];
+ u8 wol_s[0x1];
+ u8 wol_g[0x1];
+ u8 wol_a[0x1];
+ u8 wol_b[0x1];
+ u8 wol_m[0x1];
+ u8 wol_u[0x1];
+ u8 wol_p[0x1];
+ u8 stat_rate_support[0x10];
+ u8 reserved_at_1f0[0xc];
+ u8 cqe_version[0x4];
+ u8 compact_address_vector[0x1];
+ u8 striding_rq[0x1];
+ u8 reserved_at_202[0x1];
+ u8 ipoib_enhanced_offloads[0x1];
+ u8 ipoib_basic_offloads[0x1];
+ u8 reserved_at_205[0x1];
+ u8 repeated_block_disabled[0x1];
+ u8 umr_modify_entity_size_disabled[0x1];
+ u8 umr_modify_atomic_disabled[0x1];
+ u8 umr_indirect_mkey_disabled[0x1];
+ u8 umr_fence[0x2];
+ u8 reserved_at_20c[0x3];
+ u8 drain_sigerr[0x1];
+ u8 cmdif_checksum[0x2];
+ u8 sigerr_cqe[0x1];
+ u8 reserved_at_213[0x1];
+ u8 wq_signature[0x1];
+ u8 sctr_data_cqe[0x1];
+ u8 reserved_at_216[0x1];
+ u8 sho[0x1];
+ u8 tph[0x1];
+ u8 rf[0x1];
+ u8 dct[0x1];
+ u8 qos[0x1];
+ u8 eth_net_offloads[0x1];
+ u8 roce[0x1];
+ u8 atomic[0x1];
+ u8 reserved_at_21f[0x1];
+ u8 cq_oi[0x1];
+ u8 cq_resize[0x1];
+ u8 cq_moderation[0x1];
+ u8 reserved_at_223[0x3];
+ u8 cq_eq_remap[0x1];
+ u8 pg[0x1];
+ u8 block_lb_mc[0x1];
+ u8 reserved_at_229[0x1];
+ u8 scqe_break_moderation[0x1];
+ u8 cq_period_start_from_cqe[0x1];
+ u8 cd[0x1];
+ u8 reserved_at_22d[0x1];
+ u8 apm[0x1];
+ u8 vector_calc[0x1];
+ u8 umr_ptr_rlky[0x1];
+ u8 imaicl[0x1];
+ u8 reserved_at_232[0x4];
+ u8 qkv[0x1];
+ u8 pkv[0x1];
+ u8 set_deth_sqpn[0x1];
+ u8 reserved_at_239[0x3];
+ u8 xrc[0x1];
+ u8 ud[0x1];
+ u8 uc[0x1];
+ u8 rc[0x1];
+ u8 uar_4k[0x1];
+ u8 reserved_at_241[0x9];
+ u8 uar_sz[0x6];
+ u8 reserved_at_250[0x8];
+ u8 log_pg_sz[0x8];
+ u8 bf[0x1];
+ u8 driver_version[0x1];
+ u8 pad_tx_eth_packet[0x1];
+ u8 reserved_at_263[0x8];
+ u8 log_bf_reg_size[0x5];
+ u8 reserved_at_270[0xb];
+ u8 lag_master[0x1];
+ u8 num_lag_ports[0x4];
+ u8 reserved_at_280[0x10];
+ u8 max_wqe_sz_sq[0x10];
+ u8 reserved_at_2a0[0x10];
+ u8 max_wqe_sz_rq[0x10];
+ u8 max_flow_counter_31_16[0x10];
+ u8 max_wqe_sz_sq_dc[0x10];
+ u8 reserved_at_2e0[0x7];
+ u8 max_qp_mcg[0x19];
+ u8 reserved_at_300[0x10];
+ u8 flow_counter_bulk_alloc[0x08];
+ u8 log_max_mcg[0x8];
+ u8 reserved_at_320[0x3];
+ u8 log_max_transport_domain[0x5];
+ u8 reserved_at_328[0x3];
+ u8 log_max_pd[0x5];
+ u8 reserved_at_330[0xb];
+ u8 log_max_xrcd[0x5];
+ u8 nic_receive_steering_discard[0x1];
+ u8 receive_discard_vport_down[0x1];
+ u8 transmit_discard_vport_down[0x1];
+ u8 reserved_at_343[0x5];
+ u8 log_max_flow_counter_bulk[0x8];
+ u8 max_flow_counter_15_0[0x10];
+ u8 reserved_at_360[0x3];
+ u8 log_max_rq[0x5];
+ u8 reserved_at_368[0x3];
+ u8 log_max_sq[0x5];
+ u8 reserved_at_370[0x3];
+ u8 log_max_tir[0x5];
+ u8 reserved_at_378[0x3];
+ u8 log_max_tis[0x5];
+ u8 basic_cyclic_rcv_wqe[0x1];
+ u8 reserved_at_381[0x2];
+ u8 log_max_rmp[0x5];
+ u8 reserved_at_388[0x3];
+ u8 log_max_rqt[0x5];
+ u8 reserved_at_390[0x3];
+ u8 log_max_rqt_size[0x5];
+ u8 reserved_at_398[0x3];
+ u8 log_max_tis_per_sq[0x5];
+ u8 ext_stride_num_range[0x1];
+ u8 reserved_at_3a1[0x2];
+ u8 log_max_stride_sz_rq[0x5];
+ u8 reserved_at_3a8[0x3];
+ u8 log_min_stride_sz_rq[0x5];
+ u8 reserved_at_3b0[0x3];
+ u8 log_max_stride_sz_sq[0x5];
+ u8 reserved_at_3b8[0x3];
+ u8 log_min_stride_sz_sq[0x5];
+ u8 hairpin[0x1];
+ u8 reserved_at_3c1[0x2];
+ u8 log_max_hairpin_queues[0x5];
+ u8 reserved_at_3c8[0x3];
+ u8 log_max_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3d0[0x3];
+ u8 log_max_hairpin_num_packets[0x5];
+ u8 reserved_at_3d8[0x3];
+ u8 log_max_wq_sz[0x5];
+ u8 nic_vport_change_event[0x1];
+ u8 disable_local_lb_uc[0x1];
+ u8 disable_local_lb_mc[0x1];
+ u8 log_min_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3e8[0x3];
+ u8 log_max_vlan_list[0x5];
+ u8 reserved_at_3f0[0x3];
+ u8 log_max_current_mc_list[0x5];
+ u8 reserved_at_3f8[0x3];
+ u8 log_max_current_uc_list[0x5];
+ u8 general_obj_types[0x40];
+ u8 reserved_at_440[0x20];
+ u8 reserved_at_460[0x10];
+ u8 max_num_eqs[0x10];
+ u8 reserved_at_480[0x3];
+ u8 log_max_l2_table[0x5];
+ u8 reserved_at_488[0x8];
+ u8 log_uar_page_sz[0x10];
+ u8 reserved_at_4a0[0x20];
+ u8 device_frequency_mhz[0x20];
+ u8 device_frequency_khz[0x20];
+ u8 reserved_at_500[0x20];
+ u8 num_of_uars_per_page[0x20];
+ u8 flex_parser_protocols[0x20];
+ u8 reserved_at_560[0x20];
+ u8 reserved_at_580[0x3c];
+ u8 mini_cqe_resp_stride_index[0x1];
+ u8 cqe_128_always[0x1];
+ u8 cqe_compression_128[0x1];
+ u8 cqe_compression[0x1];
+ u8 cqe_compression_timeout[0x10];
+ u8 cqe_compression_max_num[0x10];
+ u8 reserved_at_5e0[0x10];
+ u8 tag_matching[0x1];
+ u8 rndv_offload_rc[0x1];
+ u8 rndv_offload_dc[0x1];
+ u8 log_tag_matching_list_sz[0x5];
+ u8 reserved_at_5f8[0x3];
+ u8 log_max_xrq[0x5];
+ u8 affiliate_nic_vport_criteria[0x8];
+ u8 native_port_num[0x8];
+ u8 num_vhca_ports[0x8];
+ u8 reserved_at_618[0x6];
+ u8 sw_owner_id[0x1];
+ u8 reserved_at_61f[0x1e1];
+};
+
+struct mlx5_ifc_qos_cap_bits {
+ u8 packet_pacing[0x1];
+ u8 esw_scheduling[0x1];
+ u8 esw_bw_share[0x1];
+ u8 esw_rate_limit[0x1];
+ u8 reserved_at_4[0x1];
+ u8 packet_pacing_burst_bound[0x1];
+ u8 packet_pacing_typical_size[0x1];
+ u8 flow_meter_srtcm[0x1];
+ u8 reserved_at_8[0x8];
+ u8 log_max_flow_meter[0x8];
+ u8 flow_meter_reg_id[0x8];
+ u8 reserved_at_25[0x20];
+ u8 packet_pacing_max_rate[0x20];
+ u8 packet_pacing_min_rate[0x20];
+ u8 reserved_at_80[0x10];
+ u8 packet_pacing_rate_table_size[0x10];
+ u8 esw_element_type[0x10];
+ u8 esw_tsar_type[0x10];
+ u8 reserved_at_c0[0x10];
+ u8 max_qos_para_vport[0x10];
+ u8 max_tsar_bw_share[0x20];
+ u8 reserved_at_100[0x6e8];
+};
+
+union mlx5_ifc_hca_cap_union_bits {
+ struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
+ struct mlx5_ifc_qos_cap_bits qos_cap;
+ u8 reserved_at_0[0x8000];
+};
+
+struct mlx5_ifc_query_hca_cap_out_bits {
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
+ union mlx5_ifc_hca_cap_union_bits capability;
+};
+
+struct mlx5_ifc_query_hca_cap_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 reserved_at_40[0x40];
+};
+
/* CQE format mask. */
#define MLX5E_CQE_FORMAT_MASK 0xc
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (2 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-17 23:59 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
` (6 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Add validation logic for E-Switch using Direct Rules.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_ethdev.c | 39 +++++++
drivers/net/mlx5/mlx5_flow.h | 5 +
drivers/net/mlx5/mlx5_flow_dv.c | 252 ++++++++++++++++++++++++++++++++++++++--
4 files changed, 287 insertions(+), 11 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 33a4127..8d63575 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -412,6 +412,8 @@ int mlx5_ibv_device_to_pci_addr(const struct ibv_device *device,
unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
uint16_t *port_list,
unsigned int port_list_n);
+int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
+ uint16_t *es_port_id);
int mlx5_sysfs_switch_info(unsigned int ifindex,
struct mlx5_switch_info *info);
bool mlx5_translate_port_name(const char *port_name_in,
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 3992918..c821772 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -1376,6 +1376,45 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size)
}
/**
+ * Get the e-switch domain id this port belongs to.
+ *
+ * @param[in] port
+ * Device port id.
+ * @param[out] es_domain_id
+ * e-switch domain id.
+ * @param[out] es_port_id
+ * The port id of the port in the switch.
+ *
+ * @return
+ * 0 on success, Negative error otherwise.
+ */
+int
+mlx5_port_to_eswitch_info(uint16_t port,
+ uint16_t *es_domain_id, uint16_t *es_port_id)
+{
+ struct rte_eth_dev *dev;
+ struct mlx5_priv *priv;
+
+ if (port >= RTE_MAX_ETHPORTS)
+ return -EINVAL;
+ dev = &rte_eth_devices[port];
+ if (dev == NULL)
+ return -ENODEV;
+ if (!dev->device ||
+ !dev->device->driver ||
+ strcmp(dev->device->driver->name, MLX5_DRIVER_NAME))
+ return -EINVAL;
+ priv = dev->data->dev_private;
+ if (!(priv->representor || priv->master))
+ return -EINVAL;
+ if (es_domain_id)
+ *es_domain_id = priv->domain_id;
+ if (es_port_id)
+ *es_port_id = priv->vport_id;
+ return 0;
+}
+
+/**
* Get switch information associated with network interface.
*
* @param ifindex
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9f47fd4..85954c2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -48,6 +48,7 @@
/* General pattern items bits. */
#define MLX5_FLOW_ITEM_METADATA (1u << 16)
+#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
/* Outer Masks. */
#define MLX5_FLOW_LAYER_OUTER_L3 \
@@ -118,6 +119,10 @@
(MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
+#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
+ MLX5_FLOW_ACTION_JUMP)
+
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
MLX5_FLOW_ACTION_RAW_ENCAP)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 7b582f0..fedc6cb 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -613,6 +613,92 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate vport item.
+ *
+ * @param[in] dev
+ * Pointer to the rte_eth_dev structure.
+ * @param[in] item
+ * Item specification.
+ * @param[in] attr
+ * Attributes of flow that includes this item.
+ * @param[in] item_flags
+ * Bit-fields that holds the items detected until now.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_item *item,
+ const struct rte_flow_attr *attr,
+ uint64_t item_flags,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_port_id *spec = item->spec;
+ const struct rte_flow_item_port_id *mask = item->mask;
+ const struct rte_flow_item_port_id switch_mask = {
+ .id = 0xffffffff,
+ };
+ uint16_t esw_domain_id;
+ uint16_t item_port_esw_domain_id;
+ uint16_t item_port_esw_port_id;
+ uint16_t port;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ NULL,
+ "match on port id is valid for"
+ " eswitch only");
+ if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "multiple source vport are not"
+ " supported");
+ if (!mask)
+ mask = &switch_mask;
+ if (mask->id && mask->id != 0xffffffff)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK,
+ mask,
+ "no support for partial mask on"
+ " \"id\" field");
+ ret = mlx5_flow_item_acceptable
+ (item, (const uint8_t *)mask,
+ (const uint8_t *)&rte_flow_item_port_id_mask,
+ sizeof(struct rte_flow_item_port_id),
+ error);
+ if (ret)
+ return ret;
+ if (!spec)
+ return 0;
+ port = mask->id ? spec->id : 0;
+ ret = mlx5_port_to_eswitch_info(port, &item_port_esw_domain_id,
+ &item_port_esw_port_id);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "failed to obtain eswitch info for"
+ " port");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain eswitch info");
+ if (item_port_esw_domain_id != esw_domain_id)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "cannot match on a port from a"
+ " different eswitch");
+ return 0;
+}
+
+/**
* Validate count action.
*
* @param[in] dev
@@ -622,6 +708,7 @@ struct field_modify_info modify_tcp[] = {
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
+ * w
*/
static int
flow_dv_validate_action_count(struct rte_eth_dev *dev,
@@ -676,7 +763,7 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ACTION, NULL,
"can only have a single encap or"
" decap action in a flow");
- if (attr->ingress)
+ if (!attr->transfer && attr->ingress)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -761,7 +848,8 @@ struct field_modify_info modify_tcp[] = {
"can only have a single encap"
" action in a flow");
/* encap without preceding decap is not supported for ingress */
- if (attr->ingress && !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
+ if (!attr->transfer && attr->ingress &&
+ !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -1561,6 +1649,77 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
+/*
+ * Validate the port_id action.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action_flags
+ * Bit-fields that holds the actions detected until now.
+ * @param[in] action
+ * Port_id RTE action structure.
+ * @param[in] attr
+ * Attributes of flow that includes this action.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
+ uint64_t action_flags,
+ const struct rte_flow_action *action,
+ const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_port_id *port_id;
+ uint16_t port;
+ uint16_t esw_domain_id;
+ uint16_t act_port_domain_id;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "port id action is valid in transfer"
+ " mode only");
+ if (!action || !action->conf)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL,
+ "port id action parameters must be"
+ " specified");
+ if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
+ MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "can have only one fate actions in"
+ " a flow");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain eswitch info");
+ port_id = action->conf;
+ port = port_id->original ? dev->data->port_id : port_id->id;
+ ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
+ if (ret)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, port_id,
+ "failed to obtain eswitch port-id for port");
+ if (act_port_domain_id != esw_domain_id)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "port does not belong to"
+ " eswitch being configured");
+ return 0;
+}
/**
* Find existing modify-header resource or create and register a new one.
@@ -1759,11 +1918,34 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
NULL,
"priority out of range");
- if (attributes->transfer)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
- NULL,
- "transfer is not supported");
+ if (attributes->transfer) {
+ if (!priv->config.dv_eswitch_en)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "eswitch dr is not supported");
+ if (!(priv->representor || priv->master))
+ return rte_flow_error_set
+ (error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "eswitch configurationd can only be"
+ " done by a master or a representor"
+ " device");
+ if (attributes->egress)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR_EGRESS,
+ attributes, "egress is not supported");
+ if (attributes->group >= MLX5_MAX_TABLES_FDB)
+ return rte_flow_error_set
+ (error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
+ NULL,
+ "group must be smaller than "
+ RTE_STR(MLX5_MAX_FDB_TABLES));
+ }
if (!(attributes->egress ^ attributes->ingress))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR, NULL,
@@ -1812,6 +1994,13 @@ struct field_modify_info modify_tcp[] = {
switch (items->type) {
case RTE_FLOW_ITEM_TYPE_VOID:
break;
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ ret = flow_dv_validate_item_port_id
+ (dev, items, attr, item_flags, error);
+ if (ret < 0)
+ return ret;
+ item_flags |= MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
ret = mlx5_flow_validate_item_eth(items, item_flags,
error);
@@ -1943,6 +2132,17 @@ struct field_modify_info modify_tcp[] = {
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ ret = flow_dv_validate_action_port_id(dev,
+ action_flags,
+ actions,
+ attr,
+ error);
+ if (ret)
+ return ret;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ ++actions_n;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
ret = mlx5_flow_validate_action_flag(action_flags,
attr, error);
@@ -2133,10 +2333,40 @@ struct field_modify_info modify_tcp[] = {
"action not supported");
}
}
- if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "no fate action is found");
+ /* Eswitch has few restrictions on using items and actions */
+ if (attr->transfer) {
+ if (action_flags & MLX5_FLOW_ACTION_FLAG)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action FLAG");
+ if (action_flags & MLX5_FLOW_ACTION_MARK)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action MARK");
+ if (action_flags & MLX5_FLOW_ACTION_QUEUE)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action QUEUE");
+ if (action_flags & MLX5_FLOW_ACTION_RSS)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action RSS");
+ if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ } else {
+ if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ }
return 0;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (3 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-18 0:19 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
` (5 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Adds the port ID item to the DV steering code.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 86 ++++++++++++++++++++++++++++++-----------
1 file changed, 63 insertions(+), 23 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index fedc6cb..e66ee34 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3095,6 +3095,64 @@ struct field_modify_info modify_tcp[] = {
}
}
+/**
+ * Add source vport match to the specified matcher.
+ *
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] port
+ * Source vport value to match
+ * @param[in] mask
+ * Mask
+ */
+static void
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
+{
+ void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
+ void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
+
+ MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
+ MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
+}
+
+/**
+ * Translate port-id item to eswitch match on port-id.
+ *
+ * @param[in] dev
+ * The devich to configure through.
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] item
+ * Flow pattern to translate.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise.
+ */
+static int
+flow_dv_eswitch_translate_item_port_id(struct rte_eth_dev *dev,
+ void *matcher, void *key,
+ const struct rte_flow_item *item)
+{
+ const struct rte_flow_item_port_id *pid_m = item ? item->mask : NULL;
+ const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
+ uint16_t mask, val, id;
+ int ret;
+
+ mask = pid_m ? pid_m->id : 0xffff;
+ id = pid_v ? pid_v->id : dev->data->port_id;
+ id = mask ? id : dev->data->port_id;
+ ret = mlx5_port_to_eswitch_info(id, NULL, &val);
+ if (ret)
+ return ret;
+ flow_dv_translate_item_source_vport(matcher, key, val, mask);
+ return 0;
+}
+
static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
#define HEADER_IS_ZERO(match_criteria, headers) \
@@ -3305,29 +3363,6 @@ struct field_modify_info modify_tcp[] = {
}
/**
- * Add source vport match to the specified matcher.
- *
- * @param[in, out] matcher
- * Flow matcher.
- * @param[in, out] key
- * Flow matcher value.
- * @param[in] port
- * Source vport value to match
- * @param[in] mask
- * Mask
- */
-static void
-flow_dv_translate_item_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
-{
- void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
- void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
-
- MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
- MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
-}
-
-/**
* Find existing tag resource or create and register a new one.
*
* @param dev[in, out]
@@ -3733,6 +3768,11 @@ struct field_modify_info modify_tcp[] = {
void *match_value = dev_flow->dv.value.buf;
switch (items->type) {
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ flow_dv_eswitch_translate_item_port_id
+ (dev, match_mask, match_value, items);
+ last_item = MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
flow_dv_translate_item_eth(match_mask, match_value,
items, tunnel);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (4 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-18 0:38 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs Ori Kam
` (4 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
In current implementation the DV steering supported only NIC steering.
This commit adds the transfer attribute in order to create a matcher
on the FDB tabels.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 2 ++
drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
3 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 83abc14..9fd5096 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
flow->ingress = attr->ingress;
+ flow->transfer = attr->transfer;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 85954c2..9d72024 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -382,6 +383,7 @@ struct rte_flow {
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
uint8_t ingress; /**< 1 if the flow is ingress. */
uint32_t group; /**< The group index. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index e66ee34..b4ca9ca 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3203,6 +3203,8 @@ struct field_modify_info modify_tcp[] = {
* Table id to use.
* @param[in] egress
* Direction of the table.
+ * @param[in] transfer
+ * E-Switch or Nic flow..
* @param[out] error
* pointer to error structure.
*
@@ -3212,6 +3214,7 @@ struct field_modify_info modify_tcp[] = {
static struct mlx5_flow_tbl_resource *
flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
uint32_t table_id, uint8_t egress,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_priv *priv = dev->data->dev_private;
@@ -3219,7 +3222,12 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow_tbl_resource *tbl;
#ifdef HAVE_MLX5DV_DR
- if (egress) {
+ if (transfer) {
+ tbl = &sh->fdb_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (sh->fdb_ns, table_id);
+ } else if (egress) {
tbl = &sh->tx_tbl[table_id];
if (!tbl->obj)
tbl->obj = mlx5_glue->dr_create_flow_tbl
@@ -3241,7 +3249,9 @@ struct field_modify_info modify_tcp[] = {
#else
(void)error;
(void)tbl;
- if (egress)
+ if (transfer)
+ return &sh->fdb_tbl[table_id];
+ else if (egress)
return &sh->tx_tbl[table_id];
else
return &sh->rx_tbl[table_id];
@@ -3306,6 +3316,7 @@ struct field_modify_info modify_tcp[] = {
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
matcher->group == cache_matcher->group &&
+ matcher->transfer == cache_matcher->transfer &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -3327,7 +3338,8 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
- matcher->egress, error);
+ matcher->egress, matcher->transfer,
+ error);
if (!tbl) {
rte_free(cache_matcher);
return rte_flow_error_set(error, ENOMEM,
@@ -3654,7 +3666,8 @@ struct field_modify_info modify_tcp[] = {
jump_data = action->conf;
tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
MLX5_GROUP_FACTOR,
- attr->egress, error);
+ attr->egress,
+ attr->transfer, error);
if (!tbl)
return rte_flow_error_set
(error, errno,
@@ -3882,6 +3895,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority);
matcher.egress = attr->egress;
matcher.group = attr->group;
+ matcher.transfer = attr->transfer;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (5 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-18 0:59 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type Ori Kam
` (3 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commits adds the port ID action to DV steering.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 12 ++++
drivers/net/mlx5/mlx5_flow_dv.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/net/mlx5/mlx5_glue.c | 14 ++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 178 insertions(+)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 8d63575..016984d 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -281,6 +281,8 @@ struct mlx5_ibv_shared {
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ LIST_HEAD(port_id_action_list, mlx5_flow_dv_port_id_action_resource)
+ port_id_action_list; /* List of port ID actions. */
/* Shared interrupt handler section. */
pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
uint32_t intr_cnt; /* Interrupt handler reference counter. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9d72024..c419e6b 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
};
+/* Port ID resource structure. */
+struct mlx5_flow_dv_port_id_action_resource {
+ LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t port_id; /**< Port ID value. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -289,6 +299,8 @@ struct mlx5_flow_dv {
struct ibv_flow *flow; /**< Installed flow. */
struct mlx5_flow_dv_jump_tbl_resource *jump;
/**< Pointer to the jump action resource. */
+ struct mlx5_flow_dv_port_id_action_resource *port_id_action;
+ /**< Pointer to port ID action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b4ca9ca..f4b7f06 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1058,6 +1058,70 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table port ID resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to port ID action resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_port_id_action_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_port_id_action_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_ibv_shared *sh = priv->sh;
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
+ if (resource->port_id == cache_resource->port_id) {
+ DRV_LOG(DEBUG, "port id action resource resource %p: "
+ "refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.port_id_action = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new port id action resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_vport(priv->sh->fdb_ns,
+ resource->port_id);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
+ dev_flow->dv.port_id_action = cache_resource;
+ DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -3467,6 +3531,44 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Translate port ID action to vport.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action
+ * Pointer to the port ID action.
+ * @param[out] dst_port_id
+ * The target port ID.
+ * @param[out] error
+ * Pointer to the error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_action *action,
+ uint32_t *dst_port_id,
+ struct rte_flow_error *error)
+{
+ uint32_t port;
+ uint16_t port_id;
+ int ret;
+ const struct rte_flow_action_port_id *conf =
+ (const struct rte_flow_action_port_id *)action->conf;
+
+ port = conf->original ? dev->data->port_id : conf->id;
+ ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "No eswitch info was found for port");
+ *dst_port_id = port_id;
+ return 0;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -3524,10 +3626,24 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action_jump *jump_data;
struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
struct mlx5_flow_tbl_resource *tbl;
+ uint32_t port_id = 0;
+ struct mlx5_flow_dv_port_id_action_resource port_id_resource;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ if (flow_dv_translate_action_port_id(dev, action,
+ &port_id, error))
+ return -rte_errno;
+ port_id_resource.port_id = port_id;
+ if (flow_dv_port_id_action_resource_register
+ (dev, &port_id_resource, dev_flow, error))
+ return -rte_errno;
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.port_id_action->action;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
@@ -4131,6 +4247,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release port ID action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource =
+ flow->dv.port_id_action;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "port id action resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Remove the flow from the NIC but keeps it in memory.
*
* @param[in] dev
@@ -4197,6 +4344,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_modify_hdr_resource_release(dev_flow);
if (dev_flow->dv.jump)
flow_dv_jump_tbl_resource_release(dev_flow);
+ if (dev_flow->dv.port_id_action)
+ flow_dv_port_id_action_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index a508faa..117190f 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -382,6 +382,18 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_dest_vport(ns, vport);
+#else
+ (void)ns;
+ (void)vport;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -847,6 +859,8 @@
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
.dr_create_flow_action_dest_flow_tbl =
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
+ .dr_create_flow_action_dest_vport =
+ mlx5_glue_dr_create_flow_action_dest_vport,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 058e9b1..26f5cb3 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -146,6 +146,7 @@ struct mlx5_glue {
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
+ void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (6 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-18 1:16 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
` (2 subsequent siblings)
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Actions like encap/decap, modify header require setting the flow table
type. Until now we supported only Nic RX and Nic TX, this commits adds
the support for FDB table type for those actions.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 54 ++++++++++++++++++++++++++++++-----------
1 file changed, 40 insertions(+), 14 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index f4b7f06..d4dc439 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -40,6 +40,10 @@
#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
#endif
+#ifndef HAVE_MLX5DV_DR_ESWITCH
+#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
+#endif
+
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -943,7 +947,9 @@ struct field_modify_info modify_tcp[] = {
struct mlx5dv_dr_ns *ns;
resource->flags = flow->group ? 0 : 1;
- if (flow->ingress)
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
ns = sh->rx_ns;
else
ns = sh->tx_ns;
@@ -1364,6 +1370,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to action structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1374,6 +1382,7 @@ struct field_modify_info modify_tcp[] = {
flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
const struct rte_flow_action *action,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
const struct rte_flow_item *encap_data;
@@ -1384,6 +1393,8 @@ struct field_modify_info modify_tcp[] = {
.ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
};
+ if (transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
raw_encap_data =
(const struct rte_flow_action_raw_encap *)action->conf;
@@ -1416,6 +1427,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to rte_eth_dev structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1425,6 +1438,7 @@ struct field_modify_info modify_tcp[] = {
static int
flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_flow_dv_encap_decap_resource res = {
@@ -1434,6 +1448,8 @@ struct field_modify_info modify_tcp[] = {
.ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
};
+ if (transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
return rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -1474,8 +1490,11 @@ struct field_modify_info modify_tcp[] = {
res.reformat_type = attr->egress ?
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL :
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
- res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
- MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
+ else
+ res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
return rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -1810,11 +1829,14 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_ibv_shared *sh = priv->sh;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns;
- struct mlx5dv_dr_ns *ns =
- resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
- sh->tx_ns : sh->rx_ns;
-
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
+ ns = sh->tx_ns;
+ else
+ ns = sh->rx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -3615,6 +3637,8 @@ struct field_modify_info modify_tcp[] = {
union flow_dv_attr flow_attr = { .attr = 0 };
struct mlx5_flow_dv_tag_resource tag_resource;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
for (; !actions_end ; actions++) {
@@ -3720,7 +3744,9 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow, error))
+ dev_flow,
+ attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3732,6 +3758,7 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
if (flow_dv_create_action_l2_decap(dev, dev_flow,
+ attr->transfer,
error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
@@ -3751,9 +3778,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
- if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_encap
+ (dev, actions, dev_flow, attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3768,9 +3795,8 @@ struct field_modify_info modify_tcp[] = {
}
/* Handle decap only if it isn't followed by encap. */
if (action->type != RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
- if (flow_dv_create_action_l2_decap(dev,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_decap
+ (dev, dev_flow, attr->transfer, error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (7 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type Ori Kam
@ 2019-04-14 21:12 ` Ori Kam
2019-04-18 1:28 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
10 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-14 21:12 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit adds support for drop action when creating E-Switch flow
using DV.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
drivers/net/mlx5/mlx5.c | 9 +++++++++
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_flow_dv.c | 26 ++++++++++++++++----------
drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 39 insertions(+), 10 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 938ba1c..e3c6d24 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
goto error;
}
sh->fdb_ns = ns;
+ sh->drop_action = mlx5_glue->dr_create_flow_action_drop();
}
#endif
sh->dv_refcnt++;
@@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->drop_action) {
+ mlx5_glue->destroy_flow_action(sh->drop_action);
+ sh->drop_action = NULL;
+ }
return err;
#else
(void)priv;
@@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->drop_action) {
+ mlx5_glue->destroy_flow_action(sh->drop_action);
+ sh->drop_action = NULL;
+ }
#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 016984d..fd85b9d 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -275,6 +275,7 @@ struct mlx5_ibv_shared {
/* RX Direct Rules tables. */
void *tx_ns; /* TX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ void *drop_action; /* Pointer to drop action. */
/* TX Direct Rules tables/ */
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index d4dc439..4a8e894 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4062,6 +4062,7 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_flow_dv *dv;
struct mlx5_flow *dev_flow;
+ struct mlx5_priv *priv = dev->data->dev_private;
int n;
int err;
@@ -4069,17 +4070,22 @@ struct field_modify_info modify_tcp[] = {
dv = &dev_flow->dv;
n = dv->actions_n;
if (flow->actions & MLX5_FLOW_ACTION_DROP) {
- dv->hrxq = mlx5_hrxq_drop_new(dev);
- if (!dv->hrxq) {
- rte_flow_error_set
- (error, errno,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
- "cannot get drop hash queue");
- goto error;
+ if (flow->transfer)
+ dv->actions[n++] = priv->sh->drop_action;
+ else {
+ dv->hrxq = mlx5_hrxq_drop_new(dev);
+ if (!dv->hrxq) {
+ rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "cannot get drop hash queue");
+ goto error;
+ }
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
- dv->actions[n++] =
- mlx5_glue->dv_create_flow_action_dest_ibv_qp
- (dv->hrxq->qp);
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 117190f..b32cd09c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -394,6 +394,16 @@
}
static void *
+mlx5_glue_dr_create_flow_action_drop(void)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_drop();
+#else
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -861,6 +871,8 @@
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_action_dest_vport =
mlx5_glue_dr_create_flow_action_dest_vport,
+ .dr_create_flow_action_drop =
+ mlx5_glue_dr_create_flow_action_drop,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 26f5cb3..1d06583 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -147,6 +147,7 @@ struct mlx5_glue {
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
+ void *(*dr_create_flow_action_drop)();
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name
2019-04-14 21:12 ` [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name Ori Kam
@ 2019-04-16 23:47 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-16 23:47 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:29PM +0000, Ori Kam wrote:
> Modify the translate vport function to match other translate items
> naming convestions.
>
> Fixes: 0fe3f18f78d8 ("net/mlx5: add source vport match to the ingress rules")
> Cc: viacheslavo@mellanox.com
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Small suggestion. How about flow_dv_translate_item_src_vport()? Looks quite long
name..
Either you take it or not,
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> drivers/net/mlx5/mlx5_flow_dv.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 3862b26..7b582f0 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3087,8 +3087,8 @@ struct field_modify_info modify_tcp[] = {
> * Mask
> */
> static void
> -flow_dv_translate_source_vport(void *matcher, void *key,
> - int16_t port, uint16_t mask)
> +flow_dv_translate_item_source_vport(void *matcher, void *key,
> + int16_t port, uint16_t mask)
> {
> void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> @@ -3492,10 +3492,10 @@ struct field_modify_info modify_tcp[] = {
> * Add matching on source vport index only
> * for ingress rules in E-Switch configurations.
> */
> - flow_dv_translate_source_vport(matcher.mask.buf,
> - dev_flow->dv.value.buf,
> - priv->vport_id,
> - 0xffff);
> + flow_dv_translate_item_source_vport(matcher.mask.buf,
> + dev_flow->dv.value.buf,
> + priv->vport_id,
> + 0xffff);
> }
> for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) {
> int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
2019-04-14 21:12 ` [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules Ori Kam
@ 2019-04-17 0:01 ` Yongseok Koh
2019-04-17 0:34 ` Yongseok Koh
2019-04-17 5:18 ` Ori Kam
0 siblings, 2 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-17 0:01 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:30PM +0000, Ori Kam wrote:
> The meson build was missing the define for Direct Rules.
>
> Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
> Cc: orika@mellanox.com
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/meson.build | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> index a4c684e..0037e15 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -111,6 +111,8 @@ if build
> 'mlx5dv_devx_obj_create' ],
> [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> + 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
Better to change the inconsistent naming?
HAVE_MLX5DV_DR -> HAVE_IBV_FLOW_DV_DIRECT_RULES
And I want it to be at a similar location to Makefile. Let's put it next to
HAVE_IBV_FLOW_DV_SUPPORT.
thanks
Yongseok
> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> 'SUPPORTED_40000baseKR4_Full' ],
> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
2019-04-17 0:01 ` Yongseok Koh
@ 2019-04-17 0:34 ` Yongseok Koh
2019-04-17 5:18 ` Ori Kam
2019-04-17 5:18 ` Ori Kam
1 sibling, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-17 0:34 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
> On Apr 16, 2019, at 5:01 PM, Yongseok Koh <yskoh@mellanox.com> wrote:
>
> On Sun, Apr 14, 2019 at 09:12:30PM +0000, Ori Kam wrote:
>> The meson build was missing the define for Direct Rules.
And a typo in the title.
menson -> meson
Also would be better to use 'meson build' instead of 'meson compilation'
"net/mlx5: fix meson build for Direct Rules"
>>
>> Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
>> Cc: orika@mellanox.com
>>
>> Signed-off-by: Ori Kam <orika@mellanox.com>
>> ---
>> drivers/net/mlx5/meson.build | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
>> index a4c684e..0037e15 100644
>> --- a/drivers/net/mlx5/meson.build
>> +++ b/drivers/net/mlx5/meson.build
>> @@ -111,6 +111,8 @@ if build
>> 'mlx5dv_devx_obj_create' ],
>> [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
>> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
>> + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
>> + 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
>
> Better to change the inconsistent naming?
> HAVE_MLX5DV_DR -> HAVE_IBV_FLOW_DV_DIRECT_RULES
>
> And I want it to be at a similar location to Makefile. Let's put it next to
> HAVE_IBV_FLOW_DV_SUPPORT.
>
> thanks
> Yongseok
>
>> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
>> 'SUPPORTED_40000baseKR4_Full' ],
>> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
>> --
>> 1.8.3.1
>>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support
2019-04-14 21:12 ` [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support Ori Kam
@ 2019-04-17 1:42 ` Yongseok Koh
2019-04-17 6:19 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-17 1:42 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:31PM +0000, Ori Kam wrote:
> This commit provides the basic configuration needed in order to
> support Direct Rules eswitch.
What do you mean my "Direct Rules eswitch"? What is the official name of it?
E-Switch is in HCA and DR is use by library? Then, shouldn't it be "E-Switch
with Direct Rules"? Please correct it appropriately.
And I can see many of 'eswitch' in commit log or comment in the code. Please
correct all of them as well.
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
The title is "net/mlx5: add Direct Rules configuration support"
Shouldn't it have the word, "E-Switch"?
And it seems to have more than "configuration"?
> drivers/net/mlx5/Makefile | 5 +
> drivers/net/mlx5/meson.build | 2 +
> drivers/net/mlx5/mlx5.c | 52 +++++-
> drivers/net/mlx5/mlx5.h | 12 ++
> drivers/net/mlx5/mlx5_devx_cmds.c | 42 +++++
> drivers/net/mlx5/mlx5_flow.c | 2 +-
> drivers/net/mlx5/mlx5_prm.h | 328 ++++++++++++++++++++++++++++++++++++++
> 7 files changed, 437 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
> index 93bc869..2b72a33 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
> enum MLX5DV_DR_NS_TYPE_TERMINATING \
> $(AUTOCONF_OUTPUT)
> $Q sh -- '$<' '$@' \
> + HAVE_MLX5DV_DR_ESWITCH \
> + infiniband/mlx5dv.h \
> + enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
> + $(AUTOCONF_OUTPUT)
> + $Q sh -- '$<' '$@' \
Should start from HAVE_IBV_FLOW_
How about HAVE_IBV_FLOW_DV_ESW_DIRECT_RULES?
> HAVE_IBV_DEVX_OBJ \
> infiniband/mlx5dv.h \
> func mlx5dv_devx_obj_create \
> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> index 0037e15..9dfd28d 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -113,6 +113,8 @@ if build
> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
> + [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
> + 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
Same here.
> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> 'SUPPORTED_40000baseKR4_Full' ],
> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index 9ff50df..938ba1c 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -101,6 +101,9 @@
> /* Allow L3 VXLAN flow creation. */
> #define MLX5_L3_VXLAN_EN "l3_vxlan_en"
>
> +/* Activate DV eswitch flow steering. */
> +#define MLX5_DV_ESWITCH_EN "dv_eswitch_en"
> +
We can set a rule to use 'esw'/'ESW' in the code just like dv/tcf/verbs and so on?
Or, what's the difference between E-Switch and FDB?
> /* Activate DV flow steering. */
> #define MLX5_DV_FLOW_EN "dv_flow_en"
>
> @@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
> }
> pthread_mutex_init(&sh->dv_mutex, NULL);
> sh->tx_ns = ns;
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (priv->config.dv_eswitch_en) {
> + ns = mlx5_glue->dr_create_ns(sh->ctx,
> + MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
> + if (!ns) {
> + DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> + sh->fdb_ns = ns;
> + }
> +#endif
> sh->dv_refcnt++;
> priv->dr_shared = 1;
> return 0;
> @@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
> mlx5dv_dr_destroy_ns(sh->tx_ns);
> sh->tx_ns = NULL;
> }
> + if (sh->fdb_ns) {
> + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> + sh->fdb_ns = NULL;
> + }
> return err;
> #else
> (void)priv;
> @@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
> mlx5dv_dr_destroy_ns(sh->tx_ns);
> sh->tx_ns = NULL;
> }
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (sh->fdb_ns) {
> + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> + sh->fdb_ns = NULL;
> + }
> +#endif
> pthread_mutex_destroy(&sh->dv_mutex);
> #else
> (void)priv;
> @@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
> config->l3_vxlan_en = !!tmp;
> } else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
> config->vf_nl_en = !!tmp;
> + } else if (strcmp(MLX5_DV_ESWITCH_EN, key) == 0) {
> + config->dv_eswitch_en = !!tmp;
Do we really need to make it configurable? What is the purpose of doing that? If
esw dr isn't supported, it can fall back to tcf but, if supported, why not using
it? We still have dv_flow_en. If dv_flow_en is disabled, we should disable dv
esw too. But we need not configure the two individually. Thoughts?
> } else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
> config->dv_flow_en = !!tmp;
> } else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
> @@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
> MLX5_RX_VEC_EN,
> MLX5_L3_VXLAN_EN,
> MLX5_VF_NL_EN,
> + MLX5_DV_ESWITCH_EN,
> MLX5_DV_FLOW_EN,
> MLX5_MR_EXT_MEMSEG_EN,
> MLX5_REPRESENTOR,
> @@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
> priv->tcf_context = NULL;
> }
> }
> - if (config.dv_flow_en) {
> - err = mlx5_alloc_shared_dr(priv);
> - if (err)
> - goto error;
> - }
> TAILQ_INIT(&priv->flows);
> TAILQ_INIT(&priv->ctrl_flows);
> /* Hint libmlx5 to use PMD allocator for data plane resources */
> @@ -1484,8 +1507,26 @@ struct mlx5_dev_spawn_data {
> * Verbs context returned by ibv_open_device().
> */
> mlx5_link_update(eth_dev, 0);
> +#ifdef HAVE_IBV_DEVX_OBJ
> + err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
> + if (err) {
> + err = -err;
> + goto error;
> + }
> +#endif
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (!config.hca_attr.eswitch_manager)
> + config.dv_eswitch_en = 0;
> +#else
> + config.dv_eswitch_en = 0;
> +#endif
> /* Store device configuration on private structure. */
> priv->config = config;
> + if (config.dv_flow_en) {
> + err = mlx5_alloc_shared_dr(priv);
> + if (err)
> + goto error;
> + }
> /* Supported Verbs flow priority number detection. */
> err = mlx5_flow_discover_priorities(eth_dev);
> if (err < 0) {
> @@ -1876,6 +1917,7 @@ struct mlx5_dev_spawn_data {
> .max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
> .min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
> },
> + .dv_eswitch_en = 1,
> };
> /* Device specific configuration. */
> switch (pci_dev->id.device_id) {
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 14c7f3c..33a4127 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
> int id; /* Flow counter ID */
> };
>
> +/* HCA attributes. */
> +struct mlx5_hca_attr {
> + uint32_t eswitch_manager:1;
> +};
> +
> /* Flow list . */
> TAILQ_HEAD(mlx5_flows, rte_flow);
>
> @@ -171,6 +176,7 @@ struct mlx5_dev_config {
> /* Whether memseg should be extended for MR creation. */
> unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
> unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
> + unsigned int dv_eswitch_en:1; /* Enable eswitch DV flow. */
> unsigned int dv_flow_en:1; /* Enable DV flow. */
> unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
> unsigned int devx:1; /* Whether devx interface is available or not. */
> @@ -192,6 +198,7 @@ struct mlx5_dev_config {
> int txqs_inline; /* Queue number threshold for inlining. */
> int txqs_vec; /* Queue number threshold for vectorized Tx. */
> int inline_max_packet_sz; /* Max packet size for inlining. */
> + struct mlx5_hca_attr hca_attr; /* HCA attributes. */
> };
>
> /**
> @@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
> };
>
> #define MLX5_MAX_TABLES 1024
> +#define MLX5_MAX_TABLES_FDB 32
> #define MLX5_GROUP_FACTOR 1
>
> /*
> @@ -260,6 +268,8 @@ struct mlx5_ibv_shared {
> /* Shared DV/DR flow data section. */
> pthread_mutex_t dv_mutex; /* DV context mutex. */
> uint32_t dv_refcnt; /* DV/DR data reference counter. */
> + void *fdb_ns; /* FDB Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
> void *rx_ns; /* RX Direct Rules name space handle. */
> struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
> /* RX Direct Rules tables. */
> @@ -539,4 +549,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct ibv_context *ctx,
> int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set *dcx,
> int clear,
> uint64_t *pkts, uint64_t *bytes);
> +int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> + struct mlx5_hca_attr *attr);
> #endif /* RTE_PMD_MLX5_H_ */
> diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c b/drivers/net/mlx5/mlx5_devx_cmds.c
> index a9dff58..3caea41 100644
> --- a/drivers/net/mlx5/mlx5_devx_cmds.c
> +++ b/drivers/net/mlx5/mlx5_devx_cmds.c
> @@ -105,3 +105,45 @@ int mlx5_devx_cmd_flow_counter_free(struct mlx5dv_devx_obj *obj)
> *bytes = MLX5_GET64(traffic_counter, stats, octets);
> return 0;
> }
> +
> +/**
> + * Query HCA attributes.
Need to be more informative. What to query here? Please specify in detail.
> + *
> + * @param[in] ctx
> + * ibv contexts returned from mlx5dv_open_device.
> + * @param[out] attr
> + * Attributes device values.
> + *
> + * @return
> + * 0 on success, a negative value otherwise.
> + */
> +int
> +mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> + struct mlx5_hca_attr *attr)
> +{
> + uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
> + uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
> + void *hcattr;
> + int status, syndrome, rc;
> +
> + MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
> + MLX5_SET(query_hca_cap_in, in, op_mod,
> + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
> + MLX5_HCA_CAP_OPMOD_GET_CUR);
> +
> + rc = mlx5_glue->devx_general_cmd(ctx,
> + in, sizeof(in), out, sizeof(out));
> + if (rc)
> + return rc;
> + status = MLX5_GET(query_hca_cap_out, out, status);
> + syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
> + if (status) {
> + DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
> + "status %x, syndrome = %x",
> + status, syndrome);
> + return -1;
> + }
> + hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
> + attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);
> + return 0;
> +}
> diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> index a0683ee..83abc14 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
> struct mlx5_priv *priv = dev->data->dev_private;
> enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
>
> - if (attr->transfer)
> + if (attr->transfer && !priv->config.dv_eswitch_en)
To make sure it works as intended, a critical precondition MUST be met.
"If dv_flow_en is set, dv_eswitch_en is also set."
Think about a case where
attr->transfer is set
dv_eswtich_en is set
dv_flow_en is unset
MLX5_FLOW_TYPE_VERBS can't handle 'transfer' case, can it?
> type = MLX5_FLOW_TYPE_TCF;
> else
> type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
> diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
> index b15266f..b25d4e8 100644
> --- a/drivers/net/mlx5/mlx5_prm.h
> +++ b/drivers/net/mlx5/mlx5_prm.h
> @@ -529,6 +529,7 @@ enum {
> };
>
> enum {
> + MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
> MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
> MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
> };
> @@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
> u8 flow_counter_id[0x20];
> };
>
Please fix all the indentation violation from here.
> +enum {
> + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
> + MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
> +};
> +
> +enum {
> + MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
> + MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
> +};
> +
> +struct mlx5_ifc_cmd_hca_cap_bits {
> + u8 reserved_at_0[0x30];
> + u8 vhca_id[0x10];
> + u8 reserved_at_40[0x40];
> + u8 log_max_srq_sz[0x8];
> + u8 log_max_qp_sz[0x8];
> + u8 reserved_at_90[0xb];
> + u8 log_max_qp[0x5];
> + u8 reserved_at_a0[0xb];
> + u8 log_max_srq[0x5];
> + u8 reserved_at_b0[0x10];
> + u8 reserved_at_c0[0x8];
> + u8 log_max_cq_sz[0x8];
> + u8 reserved_at_d0[0xb];
> + u8 log_max_cq[0x5];
> + u8 log_max_eq_sz[0x8];
> + u8 reserved_at_e8[0x2];
> + u8 log_max_mkey[0x6];
> + u8 reserved_at_f0[0x8];
> + u8 dump_fill_mkey[0x1];
> + u8 reserved_at_f9[0x3];
> + u8 log_max_eq[0x4];
> + u8 max_indirection[0x8];
> + u8 fixed_buffer_size[0x1];
> + u8 log_max_mrw_sz[0x7];
> + u8 force_teardown[0x1];
> + u8 reserved_at_111[0x1];
> + u8 log_max_bsf_list_size[0x6];
> + u8 umr_extended_translation_offset[0x1];
> + u8 null_mkey[0x1];
> + u8 log_max_klm_list_size[0x6];
> + u8 reserved_at_120[0xa];
> + u8 log_max_ra_req_dc[0x6];
> + u8 reserved_at_130[0xa];
> + u8 log_max_ra_res_dc[0x6];
> + u8 reserved_at_140[0xa];
> + u8 log_max_ra_req_qp[0x6];
> + u8 reserved_at_150[0xa];
> + u8 log_max_ra_res_qp[0x6];
> + u8 end_pad[0x1];
> + u8 cc_query_allowed[0x1];
> + u8 cc_modify_allowed[0x1];
> + u8 start_pad[0x1];
> + u8 cache_line_128byte[0x1];
> + u8 reserved_at_165[0xa];
> + u8 qcam_reg[0x1];
> + u8 gid_table_size[0x10];
> + u8 out_of_seq_cnt[0x1];
> + u8 vport_counters[0x1];
> + u8 retransmission_q_counters[0x1];
> + u8 debug[0x1];
> + u8 modify_rq_counter_set_id[0x1];
> + u8 rq_delay_drop[0x1];
> + u8 max_qp_cnt[0xa];
> + u8 pkey_table_size[0x10];
> + u8 vport_group_manager[0x1];
> + u8 vhca_group_manager[0x1];
> + u8 ib_virt[0x1];
> + u8 eth_virt[0x1];
> + u8 vnic_env_queue_counters[0x1];
> + u8 ets[0x1];
> + u8 nic_flow_table[0x1];
> + u8 eswitch_manager[0x1];
> + u8 device_memory[0x1];
> + u8 mcam_reg[0x1];
> + u8 pcam_reg[0x1];
> + u8 local_ca_ack_delay[0x5];
> + u8 port_module_event[0x1];
> + u8 enhanced_error_q_counters[0x1];
> + u8 ports_check[0x1];
> + u8 reserved_at_1b3[0x1];
> + u8 disable_link_up[0x1];
> + u8 beacon_led[0x1];
> + u8 port_type[0x2];
> + u8 num_ports[0x8];
> + u8 reserved_at_1c0[0x1];
> + u8 pps[0x1];
> + u8 pps_modify[0x1];
> + u8 log_max_msg[0x5];
> + u8 reserved_at_1c8[0x4];
> + u8 max_tc[0x4];
> + u8 temp_warn_event[0x1];
> + u8 dcbx[0x1];
> + u8 general_notification_event[0x1];
> + u8 reserved_at_1d3[0x2];
> + u8 fpga[0x1];
> + u8 rol_s[0x1];
> + u8 rol_g[0x1];
> + u8 reserved_at_1d8[0x1];
> + u8 wol_s[0x1];
> + u8 wol_g[0x1];
> + u8 wol_a[0x1];
> + u8 wol_b[0x1];
> + u8 wol_m[0x1];
> + u8 wol_u[0x1];
> + u8 wol_p[0x1];
> + u8 stat_rate_support[0x10];
> + u8 reserved_at_1f0[0xc];
> + u8 cqe_version[0x4];
> + u8 compact_address_vector[0x1];
> + u8 striding_rq[0x1];
> + u8 reserved_at_202[0x1];
> + u8 ipoib_enhanced_offloads[0x1];
> + u8 ipoib_basic_offloads[0x1];
> + u8 reserved_at_205[0x1];
> + u8 repeated_block_disabled[0x1];
> + u8 umr_modify_entity_size_disabled[0x1];
> + u8 umr_modify_atomic_disabled[0x1];
> + u8 umr_indirect_mkey_disabled[0x1];
> + u8 umr_fence[0x2];
> + u8 reserved_at_20c[0x3];
> + u8 drain_sigerr[0x1];
> + u8 cmdif_checksum[0x2];
> + u8 sigerr_cqe[0x1];
> + u8 reserved_at_213[0x1];
> + u8 wq_signature[0x1];
> + u8 sctr_data_cqe[0x1];
> + u8 reserved_at_216[0x1];
> + u8 sho[0x1];
> + u8 tph[0x1];
> + u8 rf[0x1];
> + u8 dct[0x1];
> + u8 qos[0x1];
> + u8 eth_net_offloads[0x1];
> + u8 roce[0x1];
> + u8 atomic[0x1];
> + u8 reserved_at_21f[0x1];
> + u8 cq_oi[0x1];
> + u8 cq_resize[0x1];
> + u8 cq_moderation[0x1];
> + u8 reserved_at_223[0x3];
> + u8 cq_eq_remap[0x1];
> + u8 pg[0x1];
> + u8 block_lb_mc[0x1];
> + u8 reserved_at_229[0x1];
> + u8 scqe_break_moderation[0x1];
> + u8 cq_period_start_from_cqe[0x1];
> + u8 cd[0x1];
> + u8 reserved_at_22d[0x1];
> + u8 apm[0x1];
> + u8 vector_calc[0x1];
> + u8 umr_ptr_rlky[0x1];
> + u8 imaicl[0x1];
> + u8 reserved_at_232[0x4];
> + u8 qkv[0x1];
> + u8 pkv[0x1];
> + u8 set_deth_sqpn[0x1];
> + u8 reserved_at_239[0x3];
> + u8 xrc[0x1];
> + u8 ud[0x1];
> + u8 uc[0x1];
> + u8 rc[0x1];
> + u8 uar_4k[0x1];
> + u8 reserved_at_241[0x9];
> + u8 uar_sz[0x6];
> + u8 reserved_at_250[0x8];
> + u8 log_pg_sz[0x8];
> + u8 bf[0x1];
> + u8 driver_version[0x1];
> + u8 pad_tx_eth_packet[0x1];
> + u8 reserved_at_263[0x8];
> + u8 log_bf_reg_size[0x5];
> + u8 reserved_at_270[0xb];
> + u8 lag_master[0x1];
> + u8 num_lag_ports[0x4];
> + u8 reserved_at_280[0x10];
> + u8 max_wqe_sz_sq[0x10];
> + u8 reserved_at_2a0[0x10];
> + u8 max_wqe_sz_rq[0x10];
> + u8 max_flow_counter_31_16[0x10];
> + u8 max_wqe_sz_sq_dc[0x10];
> + u8 reserved_at_2e0[0x7];
> + u8 max_qp_mcg[0x19];
> + u8 reserved_at_300[0x10];
> + u8 flow_counter_bulk_alloc[0x08];
> + u8 log_max_mcg[0x8];
> + u8 reserved_at_320[0x3];
> + u8 log_max_transport_domain[0x5];
> + u8 reserved_at_328[0x3];
> + u8 log_max_pd[0x5];
> + u8 reserved_at_330[0xb];
> + u8 log_max_xrcd[0x5];
> + u8 nic_receive_steering_discard[0x1];
> + u8 receive_discard_vport_down[0x1];
> + u8 transmit_discard_vport_down[0x1];
> + u8 reserved_at_343[0x5];
> + u8 log_max_flow_counter_bulk[0x8];
> + u8 max_flow_counter_15_0[0x10];
> + u8 reserved_at_360[0x3];
> + u8 log_max_rq[0x5];
> + u8 reserved_at_368[0x3];
> + u8 log_max_sq[0x5];
> + u8 reserved_at_370[0x3];
> + u8 log_max_tir[0x5];
> + u8 reserved_at_378[0x3];
> + u8 log_max_tis[0x5];
> + u8 basic_cyclic_rcv_wqe[0x1];
> + u8 reserved_at_381[0x2];
> + u8 log_max_rmp[0x5];
> + u8 reserved_at_388[0x3];
> + u8 log_max_rqt[0x5];
> + u8 reserved_at_390[0x3];
> + u8 log_max_rqt_size[0x5];
> + u8 reserved_at_398[0x3];
> + u8 log_max_tis_per_sq[0x5];
> + u8 ext_stride_num_range[0x1];
> + u8 reserved_at_3a1[0x2];
> + u8 log_max_stride_sz_rq[0x5];
> + u8 reserved_at_3a8[0x3];
> + u8 log_min_stride_sz_rq[0x5];
> + u8 reserved_at_3b0[0x3];
> + u8 log_max_stride_sz_sq[0x5];
> + u8 reserved_at_3b8[0x3];
> + u8 log_min_stride_sz_sq[0x5];
> + u8 hairpin[0x1];
> + u8 reserved_at_3c1[0x2];
> + u8 log_max_hairpin_queues[0x5];
> + u8 reserved_at_3c8[0x3];
> + u8 log_max_hairpin_wq_data_sz[0x5];
> + u8 reserved_at_3d0[0x3];
> + u8 log_max_hairpin_num_packets[0x5];
> + u8 reserved_at_3d8[0x3];
> + u8 log_max_wq_sz[0x5];
> + u8 nic_vport_change_event[0x1];
> + u8 disable_local_lb_uc[0x1];
> + u8 disable_local_lb_mc[0x1];
> + u8 log_min_hairpin_wq_data_sz[0x5];
> + u8 reserved_at_3e8[0x3];
> + u8 log_max_vlan_list[0x5];
> + u8 reserved_at_3f0[0x3];
> + u8 log_max_current_mc_list[0x5];
> + u8 reserved_at_3f8[0x3];
> + u8 log_max_current_uc_list[0x5];
> + u8 general_obj_types[0x40];
> + u8 reserved_at_440[0x20];
> + u8 reserved_at_460[0x10];
> + u8 max_num_eqs[0x10];
> + u8 reserved_at_480[0x3];
> + u8 log_max_l2_table[0x5];
> + u8 reserved_at_488[0x8];
> + u8 log_uar_page_sz[0x10];
> + u8 reserved_at_4a0[0x20];
> + u8 device_frequency_mhz[0x20];
> + u8 device_frequency_khz[0x20];
> + u8 reserved_at_500[0x20];
> + u8 num_of_uars_per_page[0x20];
> + u8 flex_parser_protocols[0x20];
> + u8 reserved_at_560[0x20];
> + u8 reserved_at_580[0x3c];
> + u8 mini_cqe_resp_stride_index[0x1];
> + u8 cqe_128_always[0x1];
> + u8 cqe_compression_128[0x1];
> + u8 cqe_compression[0x1];
> + u8 cqe_compression_timeout[0x10];
> + u8 cqe_compression_max_num[0x10];
> + u8 reserved_at_5e0[0x10];
> + u8 tag_matching[0x1];
> + u8 rndv_offload_rc[0x1];
> + u8 rndv_offload_dc[0x1];
> + u8 log_tag_matching_list_sz[0x5];
> + u8 reserved_at_5f8[0x3];
> + u8 log_max_xrq[0x5];
> + u8 affiliate_nic_vport_criteria[0x8];
> + u8 native_port_num[0x8];
> + u8 num_vhca_ports[0x8];
> + u8 reserved_at_618[0x6];
> + u8 sw_owner_id[0x1];
> + u8 reserved_at_61f[0x1e1];
> +};
> +
> +struct mlx5_ifc_qos_cap_bits {
> + u8 packet_pacing[0x1];
> + u8 esw_scheduling[0x1];
> + u8 esw_bw_share[0x1];
> + u8 esw_rate_limit[0x1];
> + u8 reserved_at_4[0x1];
> + u8 packet_pacing_burst_bound[0x1];
> + u8 packet_pacing_typical_size[0x1];
> + u8 flow_meter_srtcm[0x1];
> + u8 reserved_at_8[0x8];
> + u8 log_max_flow_meter[0x8];
> + u8 flow_meter_reg_id[0x8];
> + u8 reserved_at_25[0x20];
> + u8 packet_pacing_max_rate[0x20];
> + u8 packet_pacing_min_rate[0x20];
> + u8 reserved_at_80[0x10];
> + u8 packet_pacing_rate_table_size[0x10];
> + u8 esw_element_type[0x10];
> + u8 esw_tsar_type[0x10];
> + u8 reserved_at_c0[0x10];
> + u8 max_qos_para_vport[0x10];
> + u8 max_tsar_bw_share[0x20];
> + u8 reserved_at_100[0x6e8];
> +};
> +
> +union mlx5_ifc_hca_cap_union_bits {
> + struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
> + struct mlx5_ifc_qos_cap_bits qos_cap;
> + u8 reserved_at_0[0x8000];
> +};
> +
> +struct mlx5_ifc_query_hca_cap_out_bits {
> + u8 status[0x8];
> + u8 reserved_at_8[0x18];
> + u8 syndrome[0x20];
> + u8 reserved_at_40[0x40];
> + union mlx5_ifc_hca_cap_union_bits capability;
> +};
> +
> +struct mlx5_ifc_query_hca_cap_in_bits {
> + u8 opcode[0x10];
> + u8 reserved_at_10[0x10];
> + u8 reserved_at_20[0x10];
> + u8 op_mod[0x10];
> + u8 reserved_at_40[0x40];
> +};
> +
> /* CQE format mask. */
> #define MLX5E_CQE_FORMAT_MASK 0xc
>
> --
> 1.8.3.1
Thanks
Yongseok
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
2019-04-17 0:01 ` Yongseok Koh
2019-04-17 0:34 ` Yongseok Koh
@ 2019-04-17 5:18 ` Ori Kam
1 sibling, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-17 5:18 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
PSB
Thanks,
Ori
> -----Original Message-----
> From: Yongseok Koh
> Sent: Wednesday, April 17, 2019 3:01 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
>
> On Sun, Apr 14, 2019 at 09:12:30PM +0000, Ori Kam wrote:
> > The meson build was missing the define for Direct Rules.
> >
> > Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
> > Cc: orika@mellanox.com
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/meson.build | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> > index a4c684e..0037e15 100644
> > --- a/drivers/net/mlx5/meson.build
> > +++ b/drivers/net/mlx5/meson.build
> > @@ -111,6 +111,8 @@ if build
> > 'mlx5dv_devx_obj_create' ],
> > [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
> > 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> > + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> > + 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
>
> Better to change the inconsistent naming?
> HAVE_MLX5DV_DR -> HAVE_IBV_FLOW_DV_DIRECT_RULES
>
> And I want it to be at a similar location to Makefile. Let's put it next to
> HAVE_IBV_FLOW_DV_SUPPORT.
>
The name consist with the one in Makefile.
This patch is only fixing missing define from the meson build.
Also the name of the define for E-Switch is in the same format.
> thanks
> Yongseok
>
> > [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> > 'SUPPORTED_40000baseKR4_Full' ],
> > [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules
2019-04-17 0:34 ` Yongseok Koh
@ 2019-04-17 5:18 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-17 5:18 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
> -----Original Message-----
> From: Yongseok Koh
> Sent: Wednesday, April 17, 2019 3:34 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with
> Direct Rules
>
>
> > On Apr 16, 2019, at 5:01 PM, Yongseok Koh <yskoh@mellanox.com> wrote:
> >
> > On Sun, Apr 14, 2019 at 09:12:30PM +0000, Ori Kam wrote:
> >> The meson build was missing the define for Direct Rules.
>
> And a typo in the title.
> menson -> meson
>
> Also would be better to use 'meson build' instead of 'meson compilation'
>
> "net/mlx5: fix meson build for Direct Rules"
>
O.K.
> >>
> >> Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
> >> Cc: orika@mellanox.com
> >>
> >> Signed-off-by: Ori Kam <orika@mellanox.com>
> >> ---
> >> drivers/net/mlx5/meson.build | 2 ++
> >> 1 file changed, 2 insertions(+)
> >>
> >> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> >> index a4c684e..0037e15 100644
> >> --- a/drivers/net/mlx5/meson.build
> >> +++ b/drivers/net/mlx5/meson.build
> >> @@ -111,6 +111,8 @@ if build
> >> 'mlx5dv_devx_obj_create' ],
> >> [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
> >> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> >> + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> >> + 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
> >
> > Better to change the inconsistent naming?
> > HAVE_MLX5DV_DR -> HAVE_IBV_FLOW_DV_DIRECT_RULES
> >
> > And I want it to be at a similar location to Makefile. Let's put it next to
> > HAVE_IBV_FLOW_DV_SUPPORT.
> >
> > thanks
> > Yongseok
> >
> >> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> >> 'SUPPORTED_40000baseKR4_Full' ],
> >> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> >> --
> >> 1.8.3.1
> >>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support
2019-04-17 1:42 ` Yongseok Koh
@ 2019-04-17 6:19 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-17 6:19 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Koh,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Wednesday, April 17, 2019 4:42 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 3/9] net/mlx5: add Direct Rules configuration support
>
> On Sun, Apr 14, 2019 at 09:12:31PM +0000, Ori Kam wrote:
> > This commit provides the basic configuration needed in order to
> > support Direct Rules eswitch.
>
> What do you mean my "Direct Rules eswitch"? What is the official name of it?
> E-Switch is in HCA and DR is use by library? Then, shouldn't it be "E-Switch
> with Direct Rules"? Please correct it appropriately.
>
Will fix.
> And I can see many of 'eswitch' in commit log or comment in the code. Please
> correct all of them as well.
>
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
>
> The title is "net/mlx5: add Direct Rules configuration support"
> Shouldn't it have the word, "E-Switch"?
>
> And it seems to have more than "configuration"?
>
> > drivers/net/mlx5/Makefile | 5 +
> > drivers/net/mlx5/meson.build | 2 +
> > drivers/net/mlx5/mlx5.c | 52 +++++-
> > drivers/net/mlx5/mlx5.h | 12 ++
> > drivers/net/mlx5/mlx5_devx_cmds.c | 42 +++++
> > drivers/net/mlx5/mlx5_flow.c | 2 +-
> > drivers/net/mlx5/mlx5_prm.h | 328
> ++++++++++++++++++++++++++++++++++++++
> > 7 files changed, 437 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
> > index 93bc869..2b72a33 100644
> > --- a/drivers/net/mlx5/Makefile
> > +++ b/drivers/net/mlx5/Makefile
> > @@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-
> config-h.sh
> > enum MLX5DV_DR_NS_TYPE_TERMINATING \
> > $(AUTOCONF_OUTPUT)
> > $Q sh -- '$<' '$@' \
> > + HAVE_MLX5DV_DR_ESWITCH \
> > + infiniband/mlx5dv.h \
> > + enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
> > + $(AUTOCONF_OUTPUT)
> > + $Q sh -- '$<' '$@' \
>
> Should start from HAVE_IBV_FLOW_
> How about HAVE_IBV_FLOW_DV_ESW_DIRECT_RULES?
>
Like stated in previous patch. The Nic DR define is of this format.
> > HAVE_IBV_DEVX_OBJ \
> > infiniband/mlx5dv.h \
> > func mlx5dv_devx_obj_create \
> > diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> > index 0037e15..9dfd28d 100644
> > --- a/drivers/net/mlx5/meson.build
> > +++ b/drivers/net/mlx5/meson.build
> > @@ -113,6 +113,8 @@ if build
> > 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> > [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> > 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
> > + [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
> > + 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
>
> Same here.
Same comment as before.
>
> > [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> > 'SUPPORTED_40000baseKR4_Full' ],
> > [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> > index 9ff50df..938ba1c 100644
> > --- a/drivers/net/mlx5/mlx5.c
> > +++ b/drivers/net/mlx5/mlx5.c
> > @@ -101,6 +101,9 @@
> > /* Allow L3 VXLAN flow creation. */
> > #define MLX5_L3_VXLAN_EN "l3_vxlan_en"
> >
> > +/* Activate DV eswitch flow steering. */
> > +#define MLX5_DV_ESWITCH_EN "dv_eswitch_en"
> > +
>
> We can set a rule to use 'esw'/'ESW' in the code just like dv/tcf/verbs and so
> on?
> Or, what's the difference between E-Switch and FDB?
I'm not sure I understand your comment.
E-Switch rules are located in the device FDB,
We can ether create E-Switch rules using tcf or using DR which are part of dv steering.
This parameters is used to select if the E-Switch rules engine will be the tcf or dv.
>
> > /* Activate DV flow steering. */
> > #define MLX5_DV_FLOW_EN "dv_flow_en"
> >
> > @@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
> > }
> > pthread_mutex_init(&sh->dv_mutex, NULL);
> > sh->tx_ns = ns;
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + if (priv->config.dv_eswitch_en) {
> > + ns = mlx5_glue->dr_create_ns(sh->ctx,
> > +
> MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
> > + if (!ns) {
> > + DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
> > + err = errno;
> > + goto error;
> > + }
> > + sh->fdb_ns = ns;
> > + }
> > +#endif
> > sh->dv_refcnt++;
> > priv->dr_shared = 1;
> > return 0;
> > @@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
> > mlx5dv_dr_destroy_ns(sh->tx_ns);
> > sh->tx_ns = NULL;
> > }
> > + if (sh->fdb_ns) {
> > + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > + sh->fdb_ns = NULL;
> > + }
> > return err;
> > #else
> > (void)priv;
> > @@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
> > mlx5dv_dr_destroy_ns(sh->tx_ns);
> > sh->tx_ns = NULL;
> > }
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + if (sh->fdb_ns) {
> > + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > + sh->fdb_ns = NULL;
> > + }
> > +#endif
> > pthread_mutex_destroy(&sh->dv_mutex);
> > #else
> > (void)priv;
> > @@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
> > config->l3_vxlan_en = !!tmp;
> > } else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
> > config->vf_nl_en = !!tmp;
> > + } else if (strcmp(MLX5_DV_ESWITCH_EN, key) == 0) {
> > + config->dv_eswitch_en = !!tmp;
>
> Do we really need to make it configurable? What is the purpose of doing that?
> If
> esw dr isn't supported, it can fall back to tcf but, if supported, why not using
> it? We still have dv_flow_en. If dv_flow_en is disabled, we should disable dv
> esw too. But we need not configure the two individually. Thoughts?
>
I agree and this is the default value, but since there are some basic initializations that are done for the E-Switch,
for example opening the name space, I can see that in some cases the user would like do disable this option.
> > } else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
> > config->dv_flow_en = !!tmp;
> > } else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
> > @@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
> > MLX5_RX_VEC_EN,
> > MLX5_L3_VXLAN_EN,
> > MLX5_VF_NL_EN,
> > + MLX5_DV_ESWITCH_EN,
> > MLX5_DV_FLOW_EN,
> > MLX5_MR_EXT_MEMSEG_EN,
> > MLX5_REPRESENTOR,
> > @@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
> > priv->tcf_context = NULL;
> > }
> > }
> > - if (config.dv_flow_en) {
> > - err = mlx5_alloc_shared_dr(priv);
> > - if (err)
> > - goto error;
> > - }
> > TAILQ_INIT(&priv->flows);
> > TAILQ_INIT(&priv->ctrl_flows);
> > /* Hint libmlx5 to use PMD allocator for data plane resources */
> > @@ -1484,8 +1507,26 @@ struct mlx5_dev_spawn_data {
> > * Verbs context returned by ibv_open_device().
> > */
> > mlx5_link_update(eth_dev, 0);
> > +#ifdef HAVE_IBV_DEVX_OBJ
> > + err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
> > + if (err) {
> > + err = -err;
> > + goto error;
> > + }
> > +#endif
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + if (!config.hca_attr.eswitch_manager)
> > + config.dv_eswitch_en = 0;
> > +#else
> > + config.dv_eswitch_en = 0;
> > +#endif
> > /* Store device configuration on private structure. */
> > priv->config = config;
> > + if (config.dv_flow_en) {
> > + err = mlx5_alloc_shared_dr(priv);
> > + if (err)
> > + goto error;
> > + }
> > /* Supported Verbs flow priority number detection. */
> > err = mlx5_flow_discover_priorities(eth_dev);
> > if (err < 0) {
> > @@ -1876,6 +1917,7 @@ struct mlx5_dev_spawn_data {
> > .max_memcpy_len =
> MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
> > .min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
> > },
> > + .dv_eswitch_en = 1,
> > };
> > /* Device specific configuration. */
> > switch (pci_dev->id.device_id) {
> > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> > index 14c7f3c..33a4127 100644
> > --- a/drivers/net/mlx5/mlx5.h
> > +++ b/drivers/net/mlx5/mlx5.h
> > @@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
> > int id; /* Flow counter ID */
> > };
> >
> > +/* HCA attributes. */
> > +struct mlx5_hca_attr {
> > + uint32_t eswitch_manager:1;
> > +};
> > +
> > /* Flow list . */
> > TAILQ_HEAD(mlx5_flows, rte_flow);
> >
> > @@ -171,6 +176,7 @@ struct mlx5_dev_config {
> > /* Whether memseg should be extended for MR creation. */
> > unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
> > unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
> > + unsigned int dv_eswitch_en:1; /* Enable eswitch DV flow. */
> > unsigned int dv_flow_en:1; /* Enable DV flow. */
> > unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
> > unsigned int devx:1; /* Whether devx interface is available or not. */
> > @@ -192,6 +198,7 @@ struct mlx5_dev_config {
> > int txqs_inline; /* Queue number threshold for inlining. */
> > int txqs_vec; /* Queue number threshold for vectorized Tx. */
> > int inline_max_packet_sz; /* Max packet size for inlining. */
> > + struct mlx5_hca_attr hca_attr; /* HCA attributes. */
> > };
> >
> > /**
> > @@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
> > };
> >
> > #define MLX5_MAX_TABLES 1024
> > +#define MLX5_MAX_TABLES_FDB 32
> > #define MLX5_GROUP_FACTOR 1
> >
> > /*
> > @@ -260,6 +268,8 @@ struct mlx5_ibv_shared {
> > /* Shared DV/DR flow data section. */
> > pthread_mutex_t dv_mutex; /* DV context mutex. */
> > uint32_t dv_refcnt; /* DV/DR data reference counter. */
> > + void *fdb_ns; /* FDB Direct Rules name space handle. */
> > + struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
> > void *rx_ns; /* RX Direct Rules name space handle. */
> > struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
> > /* RX Direct Rules tables. */
> > @@ -539,4 +549,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct
> ibv_context *ctx,
> > int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set
> *dcx,
> > int clear,
> > uint64_t *pkts, uint64_t *bytes);
> > +int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> > + struct mlx5_hca_attr *attr);
> > #endif /* RTE_PMD_MLX5_H_ */
> > diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c
> b/drivers/net/mlx5/mlx5_devx_cmds.c
> > index a9dff58..3caea41 100644
> > --- a/drivers/net/mlx5/mlx5_devx_cmds.c
> > +++ b/drivers/net/mlx5/mlx5_devx_cmds.c
> > @@ -105,3 +105,45 @@ int mlx5_devx_cmd_flow_counter_free(struct
> mlx5dv_devx_obj *obj)
> > *bytes = MLX5_GET64(traffic_counter, stats, octets);
> > return 0;
> > }
> > +
> > +/**
> > + * Query HCA attributes.
>
> Need to be more informative. What to query here? Please specify in detail.
>
O.K.
> > + *
> > + * @param[in] ctx
> > + * ibv contexts returned from mlx5dv_open_device.
> > + * @param[out] attr
> > + * Attributes device values.
> > + *
> > + * @return
> > + * 0 on success, a negative value otherwise.
> > + */
> > +int
> > +mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> > + struct mlx5_hca_attr *attr)
> > +{
> > + uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
> > + uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
> > + void *hcattr;
> > + int status, syndrome, rc;
> > +
> > + MLX5_SET(query_hca_cap_in, in, opcode,
> MLX5_CMD_OP_QUERY_HCA_CAP);
> > + MLX5_SET(query_hca_cap_in, in, op_mod,
> > + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
> > + MLX5_HCA_CAP_OPMOD_GET_CUR);
> > +
> > + rc = mlx5_glue->devx_general_cmd(ctx,
> > + in, sizeof(in), out, sizeof(out));
> > + if (rc)
> > + return rc;
> > + status = MLX5_GET(query_hca_cap_out, out, status);
> > + syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
> > + if (status) {
> > + DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
> > + "status %x, syndrome = %x",
> > + status, syndrome);
> > + return -1;
> > + }
> > + hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
> > + attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr,
> eswitch_manager);
> > + return 0;
> > +}
> > diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> > index a0683ee..83abc14 100644
> > --- a/drivers/net/mlx5/mlx5_flow.c
> > +++ b/drivers/net/mlx5/mlx5_flow.c
> > @@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct
> rte_eth_dev *dev, int32_t priority,
> > struct mlx5_priv *priv = dev->data->dev_private;
> > enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
> >
> > - if (attr->transfer)
> > + if (attr->transfer && !priv->config.dv_eswitch_en)
>
> To make sure it works as intended, a critical precondition MUST be met.
> "If dv_flow_en is set, dv_eswitch_en is also set."
>
> Think about a case where
> attr->transfer is set
> dv_eswtich_en is set
> dv_flow_en is unset
>
> MLX5_FLOW_TYPE_VERBS can't handle 'transfer' case, can it?
>
If (dv_flow_en == 0 ) then dv_switch_en = 0
> > type = MLX5_FLOW_TYPE_TCF;
> > else
> > type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
> > diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
> > index b15266f..b25d4e8 100644
> > --- a/drivers/net/mlx5/mlx5_prm.h
> > +++ b/drivers/net/mlx5/mlx5_prm.h
> > @@ -529,6 +529,7 @@ enum {
> > };
> >
> > enum {
> > + MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
> > MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
> > MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
> > };
> > @@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
> > u8 flow_counter_id[0x20];
> > };
> >
>
> Please fix all the indentation violation from here.
>
It is the same indentation as all other cmd structures.
It is also a copy from the kernel and is based on automatic generation of the PRM.
> > +enum {
> > + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
> > + MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
> > +};
> > +
> > +enum {
> > + MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
> > + MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
> > +};
> > +
> > +struct mlx5_ifc_cmd_hca_cap_bits {
> > + u8 reserved_at_0[0x30];
> > + u8 vhca_id[0x10];
> > + u8 reserved_at_40[0x40];
> > + u8 log_max_srq_sz[0x8];
> > + u8 log_max_qp_sz[0x8];
> > + u8 reserved_at_90[0xb];
> > + u8 log_max_qp[0x5];
> > + u8 reserved_at_a0[0xb];
> > + u8 log_max_srq[0x5];
> > + u8 reserved_at_b0[0x10];
> > + u8 reserved_at_c0[0x8];
> > + u8 log_max_cq_sz[0x8];
> > + u8 reserved_at_d0[0xb];
> > + u8 log_max_cq[0x5];
> > + u8 log_max_eq_sz[0x8];
> > + u8 reserved_at_e8[0x2];
> > + u8 log_max_mkey[0x6];
> > + u8 reserved_at_f0[0x8];
> > + u8 dump_fill_mkey[0x1];
> > + u8 reserved_at_f9[0x3];
> > + u8 log_max_eq[0x4];
> > + u8 max_indirection[0x8];
> > + u8 fixed_buffer_size[0x1];
> > + u8 log_max_mrw_sz[0x7];
> > + u8 force_teardown[0x1];
> > + u8 reserved_at_111[0x1];
> > + u8 log_max_bsf_list_size[0x6];
> > + u8 umr_extended_translation_offset[0x1];
> > + u8 null_mkey[0x1];
> > + u8 log_max_klm_list_size[0x6];
> > + u8 reserved_at_120[0xa];
> > + u8 log_max_ra_req_dc[0x6];
> > + u8 reserved_at_130[0xa];
> > + u8 log_max_ra_res_dc[0x6];
> > + u8 reserved_at_140[0xa];
> > + u8 log_max_ra_req_qp[0x6];
> > + u8 reserved_at_150[0xa];
> > + u8 log_max_ra_res_qp[0x6];
> > + u8 end_pad[0x1];
> > + u8 cc_query_allowed[0x1];
> > + u8 cc_modify_allowed[0x1];
> > + u8 start_pad[0x1];
> > + u8 cache_line_128byte[0x1];
> > + u8 reserved_at_165[0xa];
> > + u8 qcam_reg[0x1];
> > + u8 gid_table_size[0x10];
> > + u8 out_of_seq_cnt[0x1];
> > + u8 vport_counters[0x1];
> > + u8 retransmission_q_counters[0x1];
> > + u8 debug[0x1];
> > + u8 modify_rq_counter_set_id[0x1];
> > + u8 rq_delay_drop[0x1];
> > + u8 max_qp_cnt[0xa];
> > + u8 pkey_table_size[0x10];
> > + u8 vport_group_manager[0x1];
> > + u8 vhca_group_manager[0x1];
> > + u8 ib_virt[0x1];
> > + u8 eth_virt[0x1];
> > + u8 vnic_env_queue_counters[0x1];
> > + u8 ets[0x1];
> > + u8 nic_flow_table[0x1];
> > + u8 eswitch_manager[0x1];
> > + u8 device_memory[0x1];
> > + u8 mcam_reg[0x1];
> > + u8 pcam_reg[0x1];
> > + u8 local_ca_ack_delay[0x5];
> > + u8 port_module_event[0x1];
> > + u8 enhanced_error_q_counters[0x1];
> > + u8 ports_check[0x1];
> > + u8 reserved_at_1b3[0x1];
> > + u8 disable_link_up[0x1];
> > + u8 beacon_led[0x1];
> > + u8 port_type[0x2];
> > + u8 num_ports[0x8];
> > + u8 reserved_at_1c0[0x1];
> > + u8 pps[0x1];
> > + u8 pps_modify[0x1];
> > + u8 log_max_msg[0x5];
> > + u8 reserved_at_1c8[0x4];
> > + u8 max_tc[0x4];
> > + u8 temp_warn_event[0x1];
> > + u8 dcbx[0x1];
> > + u8 general_notification_event[0x1];
> > + u8 reserved_at_1d3[0x2];
> > + u8 fpga[0x1];
> > + u8 rol_s[0x1];
> > + u8 rol_g[0x1];
> > + u8 reserved_at_1d8[0x1];
> > + u8 wol_s[0x1];
> > + u8 wol_g[0x1];
> > + u8 wol_a[0x1];
> > + u8 wol_b[0x1];
> > + u8 wol_m[0x1];
> > + u8 wol_u[0x1];
> > + u8 wol_p[0x1];
> > + u8 stat_rate_support[0x10];
> > + u8 reserved_at_1f0[0xc];
> > + u8 cqe_version[0x4];
> > + u8 compact_address_vector[0x1];
> > + u8 striding_rq[0x1];
> > + u8 reserved_at_202[0x1];
> > + u8 ipoib_enhanced_offloads[0x1];
> > + u8 ipoib_basic_offloads[0x1];
> > + u8 reserved_at_205[0x1];
> > + u8 repeated_block_disabled[0x1];
> > + u8 umr_modify_entity_size_disabled[0x1];
> > + u8 umr_modify_atomic_disabled[0x1];
> > + u8 umr_indirect_mkey_disabled[0x1];
> > + u8 umr_fence[0x2];
> > + u8 reserved_at_20c[0x3];
> > + u8 drain_sigerr[0x1];
> > + u8 cmdif_checksum[0x2];
> > + u8 sigerr_cqe[0x1];
> > + u8 reserved_at_213[0x1];
> > + u8 wq_signature[0x1];
> > + u8 sctr_data_cqe[0x1];
> > + u8 reserved_at_216[0x1];
> > + u8 sho[0x1];
> > + u8 tph[0x1];
> > + u8 rf[0x1];
> > + u8 dct[0x1];
> > + u8 qos[0x1];
> > + u8 eth_net_offloads[0x1];
> > + u8 roce[0x1];
> > + u8 atomic[0x1];
> > + u8 reserved_at_21f[0x1];
> > + u8 cq_oi[0x1];
> > + u8 cq_resize[0x1];
> > + u8 cq_moderation[0x1];
> > + u8 reserved_at_223[0x3];
> > + u8 cq_eq_remap[0x1];
> > + u8 pg[0x1];
> > + u8 block_lb_mc[0x1];
> > + u8 reserved_at_229[0x1];
> > + u8 scqe_break_moderation[0x1];
> > + u8 cq_period_start_from_cqe[0x1];
> > + u8 cd[0x1];
> > + u8 reserved_at_22d[0x1];
> > + u8 apm[0x1];
> > + u8 vector_calc[0x1];
> > + u8 umr_ptr_rlky[0x1];
> > + u8 imaicl[0x1];
> > + u8 reserved_at_232[0x4];
> > + u8 qkv[0x1];
> > + u8 pkv[0x1];
> > + u8 set_deth_sqpn[0x1];
> > + u8 reserved_at_239[0x3];
> > + u8 xrc[0x1];
> > + u8 ud[0x1];
> > + u8 uc[0x1];
> > + u8 rc[0x1];
> > + u8 uar_4k[0x1];
> > + u8 reserved_at_241[0x9];
> > + u8 uar_sz[0x6];
> > + u8 reserved_at_250[0x8];
> > + u8 log_pg_sz[0x8];
> > + u8 bf[0x1];
> > + u8 driver_version[0x1];
> > + u8 pad_tx_eth_packet[0x1];
> > + u8 reserved_at_263[0x8];
> > + u8 log_bf_reg_size[0x5];
> > + u8 reserved_at_270[0xb];
> > + u8 lag_master[0x1];
> > + u8 num_lag_ports[0x4];
> > + u8 reserved_at_280[0x10];
> > + u8 max_wqe_sz_sq[0x10];
> > + u8 reserved_at_2a0[0x10];
> > + u8 max_wqe_sz_rq[0x10];
> > + u8 max_flow_counter_31_16[0x10];
> > + u8 max_wqe_sz_sq_dc[0x10];
> > + u8 reserved_at_2e0[0x7];
> > + u8 max_qp_mcg[0x19];
> > + u8 reserved_at_300[0x10];
> > + u8 flow_counter_bulk_alloc[0x08];
> > + u8 log_max_mcg[0x8];
> > + u8 reserved_at_320[0x3];
> > + u8 log_max_transport_domain[0x5];
> > + u8 reserved_at_328[0x3];
> > + u8 log_max_pd[0x5];
> > + u8 reserved_at_330[0xb];
> > + u8 log_max_xrcd[0x5];
> > + u8 nic_receive_steering_discard[0x1];
> > + u8 receive_discard_vport_down[0x1];
> > + u8 transmit_discard_vport_down[0x1];
> > + u8 reserved_at_343[0x5];
> > + u8 log_max_flow_counter_bulk[0x8];
> > + u8 max_flow_counter_15_0[0x10];
> > + u8 reserved_at_360[0x3];
> > + u8 log_max_rq[0x5];
> > + u8 reserved_at_368[0x3];
> > + u8 log_max_sq[0x5];
> > + u8 reserved_at_370[0x3];
> > + u8 log_max_tir[0x5];
> > + u8 reserved_at_378[0x3];
> > + u8 log_max_tis[0x5];
> > + u8 basic_cyclic_rcv_wqe[0x1];
> > + u8 reserved_at_381[0x2];
> > + u8 log_max_rmp[0x5];
> > + u8 reserved_at_388[0x3];
> > + u8 log_max_rqt[0x5];
> > + u8 reserved_at_390[0x3];
> > + u8 log_max_rqt_size[0x5];
> > + u8 reserved_at_398[0x3];
> > + u8 log_max_tis_per_sq[0x5];
> > + u8 ext_stride_num_range[0x1];
> > + u8 reserved_at_3a1[0x2];
> > + u8 log_max_stride_sz_rq[0x5];
> > + u8 reserved_at_3a8[0x3];
> > + u8 log_min_stride_sz_rq[0x5];
> > + u8 reserved_at_3b0[0x3];
> > + u8 log_max_stride_sz_sq[0x5];
> > + u8 reserved_at_3b8[0x3];
> > + u8 log_min_stride_sz_sq[0x5];
> > + u8 hairpin[0x1];
> > + u8 reserved_at_3c1[0x2];
> > + u8 log_max_hairpin_queues[0x5];
> > + u8 reserved_at_3c8[0x3];
> > + u8 log_max_hairpin_wq_data_sz[0x5];
> > + u8 reserved_at_3d0[0x3];
> > + u8 log_max_hairpin_num_packets[0x5];
> > + u8 reserved_at_3d8[0x3];
> > + u8 log_max_wq_sz[0x5];
> > + u8 nic_vport_change_event[0x1];
> > + u8 disable_local_lb_uc[0x1];
> > + u8 disable_local_lb_mc[0x1];
> > + u8 log_min_hairpin_wq_data_sz[0x5];
> > + u8 reserved_at_3e8[0x3];
> > + u8 log_max_vlan_list[0x5];
> > + u8 reserved_at_3f0[0x3];
> > + u8 log_max_current_mc_list[0x5];
> > + u8 reserved_at_3f8[0x3];
> > + u8 log_max_current_uc_list[0x5];
> > + u8 general_obj_types[0x40];
> > + u8 reserved_at_440[0x20];
> > + u8 reserved_at_460[0x10];
> > + u8 max_num_eqs[0x10];
> > + u8 reserved_at_480[0x3];
> > + u8 log_max_l2_table[0x5];
> > + u8 reserved_at_488[0x8];
> > + u8 log_uar_page_sz[0x10];
> > + u8 reserved_at_4a0[0x20];
> > + u8 device_frequency_mhz[0x20];
> > + u8 device_frequency_khz[0x20];
> > + u8 reserved_at_500[0x20];
> > + u8 num_of_uars_per_page[0x20];
> > + u8 flex_parser_protocols[0x20];
> > + u8 reserved_at_560[0x20];
> > + u8 reserved_at_580[0x3c];
> > + u8 mini_cqe_resp_stride_index[0x1];
> > + u8 cqe_128_always[0x1];
> > + u8 cqe_compression_128[0x1];
> > + u8 cqe_compression[0x1];
> > + u8 cqe_compression_timeout[0x10];
> > + u8 cqe_compression_max_num[0x10];
> > + u8 reserved_at_5e0[0x10];
> > + u8 tag_matching[0x1];
> > + u8 rndv_offload_rc[0x1];
> > + u8 rndv_offload_dc[0x1];
> > + u8 log_tag_matching_list_sz[0x5];
> > + u8 reserved_at_5f8[0x3];
> > + u8 log_max_xrq[0x5];
> > + u8 affiliate_nic_vport_criteria[0x8];
> > + u8 native_port_num[0x8];
> > + u8 num_vhca_ports[0x8];
> > + u8 reserved_at_618[0x6];
> > + u8 sw_owner_id[0x1];
> > + u8 reserved_at_61f[0x1e1];
> > +};
> > +
> > +struct mlx5_ifc_qos_cap_bits {
> > + u8 packet_pacing[0x1];
> > + u8 esw_scheduling[0x1];
> > + u8 esw_bw_share[0x1];
> > + u8 esw_rate_limit[0x1];
> > + u8 reserved_at_4[0x1];
> > + u8 packet_pacing_burst_bound[0x1];
> > + u8 packet_pacing_typical_size[0x1];
> > + u8 flow_meter_srtcm[0x1];
> > + u8 reserved_at_8[0x8];
> > + u8 log_max_flow_meter[0x8];
> > + u8 flow_meter_reg_id[0x8];
> > + u8 reserved_at_25[0x20];
> > + u8 packet_pacing_max_rate[0x20];
> > + u8 packet_pacing_min_rate[0x20];
> > + u8 reserved_at_80[0x10];
> > + u8 packet_pacing_rate_table_size[0x10];
> > + u8 esw_element_type[0x10];
> > + u8 esw_tsar_type[0x10];
> > + u8 reserved_at_c0[0x10];
> > + u8 max_qos_para_vport[0x10];
> > + u8 max_tsar_bw_share[0x20];
> > + u8 reserved_at_100[0x6e8];
> > +};
> > +
> > +union mlx5_ifc_hca_cap_union_bits {
> > + struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
> > + struct mlx5_ifc_qos_cap_bits qos_cap;
> > + u8 reserved_at_0[0x8000];
> > +};
> > +
> > +struct mlx5_ifc_query_hca_cap_out_bits {
> > + u8 status[0x8];
> > + u8 reserved_at_8[0x18];
> > + u8 syndrome[0x20];
> > + u8 reserved_at_40[0x40];
> > + union mlx5_ifc_hca_cap_union_bits capability;
> > +};
> > +
> > +struct mlx5_ifc_query_hca_cap_in_bits {
> > + u8 opcode[0x10];
> > + u8 reserved_at_10[0x10];
> > + u8 reserved_at_20[0x10];
> > + u8 op_mod[0x10];
> > + u8 reserved_at_40[0x40];
> > +};
> > +
> > /* CQE format mask. */
> > #define MLX5E_CQE_FORMAT_MASK 0xc
> >
> > --
> > 1.8.3.1
>
> Thanks
> Yongseok
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-14 21:12 ` [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
@ 2019-04-17 23:59 ` Yongseok Koh
2019-04-18 4:40 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-17 23:59 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:32PM +0000, Ori Kam wrote:
> Add validation logic for E-Switch using Direct Rules.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_ethdev.c | 39 +++++++
> drivers/net/mlx5/mlx5_flow.h | 5 +
> drivers/net/mlx5/mlx5_flow_dv.c | 252 ++++++++++++++++++++++++++++++++++++++--
> 4 files changed, 287 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 33a4127..8d63575 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -412,6 +412,8 @@ int mlx5_ibv_device_to_pci_addr(const struct ibv_device *device,
> unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
> uint16_t *port_list,
> unsigned int port_list_n);
> +int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
> + uint16_t *es_port_id);
> int mlx5_sysfs_switch_info(unsigned int ifindex,
> struct mlx5_switch_info *info);
> bool mlx5_translate_port_name(const char *port_name_in,
> diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
> index 3992918..c821772 100644
> --- a/drivers/net/mlx5/mlx5_ethdev.c
> +++ b/drivers/net/mlx5/mlx5_ethdev.c
> @@ -1376,6 +1376,45 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size)
> }
>
> /**
> + * Get the e-switch domain id this port belongs to.
E-Switch
> + *
> + * @param[in] port
> + * Device port id.
> + * @param[out] es_domain_id
> + * e-switch domain id.
E-Switch
Please correct in the entire patchset.
> + * @param[out] es_port_id
> + * The port id of the port in the switch.
> + *
> + * @return
> + * 0 on success, Negative error otherwise.
From looking at the use-cases below, rte_errno must be set.
> + */
> +int
> +mlx5_port_to_eswitch_info(uint16_t port,
> + uint16_t *es_domain_id, uint16_t *es_port_id)
> +{
> + struct rte_eth_dev *dev;
> + struct mlx5_priv *priv;
> +
> + if (port >= RTE_MAX_ETHPORTS)
> + return -EINVAL;
> + dev = &rte_eth_devices[port];
> + if (dev == NULL)
> + return -ENODEV;
dev is an l-value, it cannot be null.
The above two checks can be replaced with rte_eth_dev_is_valid_port().
> + if (!dev->device ||
> + !dev->device->driver ||
> + strcmp(dev->device->driver->name, MLX5_DRIVER_NAME))
> + return -EINVAL;
Looks too paranoid. The function is just PMD-internal.
> + priv = dev->data->dev_private;
> + if (!(priv->representor || priv->master))
> + return -EINVAL;
> + if (es_domain_id)
> + *es_domain_id = priv->domain_id;
> + if (es_port_id)
> + *es_port_id = priv->vport_id;
It is okay for now but we need to define a new struct like esw_info next time.
This info should be grouped in the priv.
> + return 0;
> +}
> +
> +/**
> * Get switch information associated with network interface.
> *
> * @param ifindex
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 9f47fd4..85954c2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -48,6 +48,7 @@
>
> /* General pattern items bits. */
> #define MLX5_FLOW_ITEM_METADATA (1u << 16)
> +#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
>
> /* Outer Masks. */
> #define MLX5_FLOW_LAYER_OUTER_L3 \
> @@ -118,6 +119,10 @@
> (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
> MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
>
> +#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
> + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
> + MLX5_FLOW_ACTION_JUMP)
> +
> #define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
> MLX5_FLOW_ACTION_NVGRE_ENCAP | \
> MLX5_FLOW_ACTION_RAW_ENCAP)
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 7b582f0..fedc6cb 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -613,6 +613,92 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Validate vport item.
> + *
> + * @param[in] dev
> + * Pointer to the rte_eth_dev structure.
> + * @param[in] item
> + * Item specification.
> + * @param[in] attr
> + * Attributes of flow that includes this item.
> + * @param[in] item_flags
> + * Bit-fields that holds the items detected until now.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
> + const struct rte_flow_item *item,
> + const struct rte_flow_attr *attr,
> + uint64_t item_flags,
> + struct rte_flow_error *error)
> +{
> + const struct rte_flow_item_port_id *spec = item->spec;
> + const struct rte_flow_item_port_id *mask = item->mask;
> + const struct rte_flow_item_port_id switch_mask = {
> + .id = 0xffffffff,
> + };
> + uint16_t esw_domain_id;
> + uint16_t item_port_esw_domain_id;
> + uint16_t item_port_esw_port_id;
> + uint16_t port;
> + int ret;
> +
> + if (!attr->transfer)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + NULL,
> + "match on port id is valid for"
> + " eswitch only");
Need to mention about 'transfer' flag here instead of esw. BTW, is it okay to
speak in PMD specific language for error messages? Even if so, 'eswitch' should
be fixed. Please re-visit all the error messages again.
> + if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ITEM, item,
> + "multiple source vport are not"
> + " supported");
Same here. 'vport' doesn't look appropriate.
> + if (!mask)
> + mask = &switch_mask;
> + if (mask->id && mask->id != 0xffffffff)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ITEM_MASK,
> + mask,
> + "no support for partial mask on"
> + " \"id\" field");
> + ret = mlx5_flow_item_acceptable
> + (item, (const uint8_t *)mask,
> + (const uint8_t *)&rte_flow_item_port_id_mask,
> + sizeof(struct rte_flow_item_port_id),
> + error);
> + if (ret)
> + return ret;
> + if (!spec)
> + return 0;
> + port = mask->id ? spec->id : 0;
Non-masked value means 'any' value. Is it correct to set port 0 in
this case?
> + ret = mlx5_port_to_eswitch_info(port, &item_port_esw_domain_id,
> + &item_port_esw_port_id);
item_port_esw_port_id is of no use here;
> + if (ret)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> + "failed to obtain eswitch info for"
> + " port");
> + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> + &esw_domain_id, NULL);
> + if (ret < 0)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "failed to obtain eswitch info");
> + if (item_port_esw_domain_id != esw_domain_id)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> + "cannot match on a port from a"
> + " different eswitch");
> + return 0;
> +}
> +
> +/**
> * Validate count action.
> *
> * @param[in] dev
> @@ -622,6 +708,7 @@ struct field_modify_info modify_tcp[] = {
> *
> * @return
> * 0 on success, a negative errno value otherwise and rte_errno is set.
> + * w
What is this change?
> */
> static int
> flow_dv_validate_action_count(struct rte_eth_dev *dev,
> @@ -676,7 +763,7 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> "can only have a single encap or"
> " decap action in a flow");
> - if (attr->ingress)
> + if (!attr->transfer && attr->ingress)
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> NULL,
> @@ -761,7 +848,8 @@ struct field_modify_info modify_tcp[] = {
> "can only have a single encap"
> " action in a flow");
> /* encap without preceding decap is not supported for ingress */
> - if (attr->ingress && !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
> + if (!attr->transfer && attr->ingress &&
> + !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> NULL,
> @@ -1561,6 +1649,77 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
>
> +/*
> + * Validate the port_id action.
> + *
> + * @param[in] dev
> + * Pointer to rte_eth_dev structure.
> + * @param[in] action_flags
> + * Bit-fields that holds the actions detected until now.
> + * @param[in] action
> + * Port_id RTE action structure.
> + * @param[in] attr
> + * Attributes of flow that includes this action.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
> + uint64_t action_flags,
> + const struct rte_flow_action *action,
> + const struct rte_flow_attr *attr,
> + struct rte_flow_error *error)
> +{
> + const struct rte_flow_action_port_id *port_id;
> + uint16_t port;
> + uint16_t esw_domain_id;
> + uint16_t act_port_domain_id;
> + int ret;
> +
> + if (!attr->transfer)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "port id action is valid in transfer"
> + " mode only");
> + if (!action || !action->conf)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> + NULL,
> + "port id action parameters must be"
> + " specified");
> + if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
> + MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> + "can have only one fate actions in"
> + " a flow");
> + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> + &esw_domain_id, NULL);
> + if (ret < 0)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "failed to obtain eswitch info");
> + port_id = action->conf;
> + port = port_id->original ? dev->data->port_id : port_id->id;
> + ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
> + if (ret)
> + return rte_flow_error_set
> + (error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION_CONF, port_id,
> + "failed to obtain eswitch port-id for port");
> + if (act_port_domain_id != esw_domain_id)
> + return rte_flow_error_set
> + (error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> + "port does not belong to"
> + " eswitch being configured");
> + return 0;
> +}
>
> /**
> * Find existing modify-header resource or create and register a new one.
> @@ -1759,11 +1918,34 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
> NULL,
> "priority out of range");
> - if (attributes->transfer)
> - return rte_flow_error_set(error, ENOTSUP,
> - RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> - NULL,
> - "transfer is not supported");
> + if (attributes->transfer) {
> + if (!priv->config.dv_eswitch_en)
> + return rte_flow_error_set
> + (error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "eswitch dr is not supported");
If you open a parenthesis in a new line, you should indent by a tab.
> + if (!(priv->representor || priv->master))
> + return rte_flow_error_set
> + (error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "eswitch configurationd can only be"
> + " done by a master or a representor"
> + " device");
> + if (attributes->egress)
> + return rte_flow_error_set
> + (error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS,
> + attributes, "egress is not supported");
> + if (attributes->group >= MLX5_MAX_TABLES_FDB)
> + return rte_flow_error_set
> + (error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> + NULL,
> + "group must be smaller than "
> + RTE_STR(MLX5_MAX_FDB_TABLES));
> + }
> if (!(attributes->egress ^ attributes->ingress))
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR, NULL,
> @@ -1812,6 +1994,13 @@ struct field_modify_info modify_tcp[] = {
> switch (items->type) {
> case RTE_FLOW_ITEM_TYPE_VOID:
> break;
> + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> + ret = flow_dv_validate_item_port_id
> + (dev, items, attr, item_flags, error);
> + if (ret < 0)
> + return ret;
> + item_flags |= MLX5_FLOW_ITEM_PORT_ID;
> + break;
Shouldn't it use last_item?
> case RTE_FLOW_ITEM_TYPE_ETH:
> ret = mlx5_flow_validate_item_eth(items, item_flags,
> error);
> @@ -1943,6 +2132,17 @@ struct field_modify_info modify_tcp[] = {
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> + ret = flow_dv_validate_action_port_id(dev,
> + action_flags,
> + actions,
> + attr,
> + error);
> + if (ret)
> + return ret;
> + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> + ++actions_n;
> + break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> ret = mlx5_flow_validate_action_flag(action_flags,
> attr, error);
> @@ -2133,10 +2333,40 @@ struct field_modify_info modify_tcp[] = {
> "action not supported");
> }
> }
> - if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
> - return rte_flow_error_set(error, EINVAL,
> - RTE_FLOW_ERROR_TYPE_ACTION, actions,
> - "no fate action is found");
> + /* Eswitch has few restrictions on using items and actions */
> + if (attr->transfer) {
> + if (action_flags & MLX5_FLOW_ACTION_FLAG)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action FLAG");
> + if (action_flags & MLX5_FLOW_ACTION_MARK)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action MARK");
> + if (action_flags & MLX5_FLOW_ACTION_QUEUE)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action QUEUE");
> + if (action_flags & MLX5_FLOW_ACTION_RSS)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action RSS");
> + if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + actions,
> + "no fate action is found");
> + } else {
> + if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + actions,
> + "no fate action is found");
> + }
> return 0;
> }
>
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-14 21:12 ` [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
@ 2019-04-18 0:19 ` Yongseok Koh
2019-04-18 4:43 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 0:19 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:33PM +0000, Ori Kam wrote:
> Adds the port ID item to the DV steering code.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 86 ++++++++++++++++++++++++++++++-----------
> 1 file changed, 63 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index fedc6cb..e66ee34 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3095,6 +3095,64 @@ struct field_modify_info modify_tcp[] = {
> }
> }
>
> +/**
> + * Add source vport match to the specified matcher.
> + *
> + * @param[in, out] matcher
> + * Flow matcher.
> + * @param[in, out] key
> + * Flow matcher value.
> + * @param[in] port
> + * Source vport value to match
> + * @param[in] mask
> + * Mask
> + */
> +static void
> +flow_dv_translate_item_source_vport(void *matcher, void *key,
> + int16_t port, uint16_t mask)
> +{
> + void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> + void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> +
> + MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> + MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> +}
> +
> +/**
> + * Translate port-id item to eswitch match on port-id.
> + *
> + * @param[in] dev
> + * The devich to configure through.
> + * @param[in, out] matcher
> + * Flow matcher.
> + * @param[in, out] key
> + * Flow matcher value.
> + * @param[in] item
> + * Flow pattern to translate.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise.
> + */
> +static int
> +flow_dv_eswitch_translate_item_port_id(struct rte_eth_dev *dev,
Why did you put _eswitch in the function name unlike
flow_dv_validate_item_port_id()? It sounds awkward.
> + void *matcher, void *key,
> + const struct rte_flow_item *item)
> +{
> + const struct rte_flow_item_port_id *pid_m = item ? item->mask : NULL;
> + const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
> + uint16_t mask, val, id;
> + int ret;
> +
> + mask = pid_m ? pid_m->id : 0xffff;
> + id = pid_v ? pid_v->id : dev->data->port_id;
> + id = mask ? id : dev->data->port_id;
Again, doesn't zero mask mean 'any' value?
Thanks,
Yongseok
> + ret = mlx5_port_to_eswitch_info(id, NULL, &val);
> + if (ret)
> + return ret;
> + flow_dv_translate_item_source_vport(matcher, key, val, mask);
> + return 0;
> +}
> +
> static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
>
> #define HEADER_IS_ZERO(match_criteria, headers) \
> @@ -3305,29 +3363,6 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> - * Add source vport match to the specified matcher.
> - *
> - * @param[in, out] matcher
> - * Flow matcher.
> - * @param[in, out] key
> - * Flow matcher value.
> - * @param[in] port
> - * Source vport value to match
> - * @param[in] mask
> - * Mask
> - */
> -static void
> -flow_dv_translate_item_source_vport(void *matcher, void *key,
> - int16_t port, uint16_t mask)
> -{
> - void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> - void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> -
> - MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> - MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> -}
> -
> -/**
> * Find existing tag resource or create and register a new one.
> *
> * @param dev[in, out]
> @@ -3733,6 +3768,11 @@ struct field_modify_info modify_tcp[] = {
> void *match_value = dev_flow->dv.value.buf;
>
> switch (items->type) {
> + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> + flow_dv_eswitch_translate_item_port_id
> + (dev, match_mask, match_value, items);
> + last_item = MLX5_FLOW_ITEM_PORT_ID;
> + break;
> case RTE_FLOW_ITEM_TYPE_ETH:
> flow_dv_translate_item_eth(match_mask, match_value,
> items, tunnel);
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher
2019-04-14 21:12 ` [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
@ 2019-04-18 0:38 ` Yongseok Koh
2019-04-18 4:57 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 0:38 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:34PM +0000, Ori Kam wrote:
> In current implementation the DV steering supported only NIC steering.
> This commit adds the transfer attribute in order to create a matcher
> on the FDB tabels.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 2 ++
> drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
> 3 files changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> index 83abc14..9fd5096 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
> flow = rte_calloc(__func__, 1, flow_size, 0);
> flow->drv_type = flow_get_drv_type(dev, attr);
> flow->ingress = attr->ingress;
> + flow->transfer = attr->transfer;
> assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
> flow->drv_type < MLX5_FLOW_TYPE_MAX);
> flow->queue = (void *)(flow + 1);
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 85954c2..9d72024 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
> uint16_t crc; /**< CRC of key. */
> uint16_t priority; /**< Priority of matcher. */
> uint8_t egress; /**< Egress matcher. */
> + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
egress and transfer can be bit fields? Those come from rte_flow_attr but I don't
understand why it needs to be uint8_t individually.
> uint32_t group; /**< The matcher group. */
> struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
> };
> @@ -382,6 +383,7 @@ struct rte_flow {
> struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
> uint8_t ingress; /**< 1 if the flow is ingress. */
> uint32_t group; /**< The group index. */
> + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
Bit-field?
Just out of curiosity, flow->ingress vs matcher->egress, why?
rte_flow_attr has both ingress and egress because a flow can be applied for both
directions. But, in mlx5 PMD, it looks exclusive - !ingress == egress vice
versa. Then, I don't understand why one has ingress and the other one has
egress even wasting bits.
> };
>
> typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index e66ee34..b4ca9ca 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3203,6 +3203,8 @@ struct field_modify_info modify_tcp[] = {
> * Table id to use.
> * @param[in] egress
> * Direction of the table.
> + * @param[in] transfer
> + * E-Switch or Nic flow..
Redundant periods (..)
Nic -> NIC
Thanks,
Yongseok
> * @param[out] error
> * pointer to error structure.
> *
> @@ -3212,6 +3214,7 @@ struct field_modify_info modify_tcp[] = {
> static struct mlx5_flow_tbl_resource *
> flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> uint32_t table_id, uint8_t egress,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> struct mlx5_priv *priv = dev->data->dev_private;
> @@ -3219,7 +3222,12 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_flow_tbl_resource *tbl;
>
> #ifdef HAVE_MLX5DV_DR
> - if (egress) {
> + if (transfer) {
> + tbl = &sh->fdb_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (sh->fdb_ns, table_id);
> + } else if (egress) {
> tbl = &sh->tx_tbl[table_id];
> if (!tbl->obj)
> tbl->obj = mlx5_glue->dr_create_flow_tbl
> @@ -3241,7 +3249,9 @@ struct field_modify_info modify_tcp[] = {
> #else
> (void)error;
> (void)tbl;
> - if (egress)
> + if (transfer)
> + return &sh->fdb_tbl[table_id];
> + else if (egress)
> return &sh->tx_tbl[table_id];
> else
> return &sh->rx_tbl[table_id];
> @@ -3306,6 +3316,7 @@ struct field_modify_info modify_tcp[] = {
> matcher->priority == cache_matcher->priority &&
> matcher->egress == cache_matcher->egress &&
> matcher->group == cache_matcher->group &&
> + matcher->transfer == cache_matcher->transfer &&
> !memcmp((const void *)matcher->mask.buf,
> (const void *)cache_matcher->mask.buf,
> cache_matcher->mask.size)) {
> @@ -3327,7 +3338,8 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
> - matcher->egress, error);
> + matcher->egress, matcher->transfer,
> + error);
> if (!tbl) {
> rte_free(cache_matcher);
> return rte_flow_error_set(error, ENOMEM,
> @@ -3654,7 +3666,8 @@ struct field_modify_info modify_tcp[] = {
> jump_data = action->conf;
> tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
> MLX5_GROUP_FACTOR,
> - attr->egress, error);
> + attr->egress,
> + attr->transfer, error);
> if (!tbl)
> return rte_flow_error_set
> (error, errno,
> @@ -3882,6 +3895,7 @@ struct field_modify_info modify_tcp[] = {
> matcher.priority);
> matcher.egress = attr->egress;
> matcher.group = attr->group;
> + matcher.transfer = attr->transfer;
> if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
> return -rte_errno;
> return 0;
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs
2019-04-14 21:12 ` [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs Ori Kam
@ 2019-04-18 0:59 ` Yongseok Koh
2019-04-18 5:06 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 0:59 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:35PM +0000, Ori Kam wrote:
> This commits adds the port ID action to DV steering.
Poor title and commit message.
Don't you add translation of the action?
Thanks,
Yongseok
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 12 ++++
> drivers/net/mlx5/mlx5_flow_dv.c | 149 ++++++++++++++++++++++++++++++++++++++++
> drivers/net/mlx5/mlx5_glue.c | 14 ++++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 178 insertions(+)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 8d63575..016984d 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -281,6 +281,8 @@ struct mlx5_ibv_shared {
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
> LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
> + LIST_HEAD(port_id_action_list, mlx5_flow_dv_port_id_action_resource)
> + port_id_action_list; /* List of port ID actions. */
> /* Shared interrupt handler section. */
> pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
> uint32_t intr_cnt; /* Interrupt handler reference counter. */
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 9d72024..c419e6b 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
> struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
> };
>
> +/* Port ID resource structure. */
> +struct mlx5_flow_dv_port_id_action_resource {
> + LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t port_id; /**< Port ID value. */
> +};
> +
> /*
> * Max number of actions per DV flow.
> * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
> @@ -289,6 +299,8 @@ struct mlx5_flow_dv {
> struct ibv_flow *flow; /**< Installed flow. */
> struct mlx5_flow_dv_jump_tbl_resource *jump;
> /**< Pointer to the jump action resource. */
> + struct mlx5_flow_dv_port_id_action_resource *port_id_action;
> + /**< Pointer to port ID action resource. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index b4ca9ca..f4b7f06 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -1058,6 +1058,70 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Find existing table port ID resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to port ID action resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_port_id_action_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_port_id_action_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_ibv_shared *sh = priv->sh;
> + struct mlx5_flow_dv_port_id_action_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
> + if (resource->port_id == cache_resource->port_id) {
> + DRV_LOG(DEBUG, "port id action resource resource %p: "
> + "refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->dv.port_id_action = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new port id action resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource memory");
> + *cache_resource = *resource;
> + cache_resource->action =
> + mlx5_glue->dr_create_flow_action_dest_vport(priv->sh->fdb_ns,
> + resource->port_id);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
> + dev_flow->dv.port_id_action = cache_resource;
> + DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> * Get the size of specific rte_flow_item_type
> *
> * @param[in] item_type
> @@ -3467,6 +3531,44 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Translate port ID action to vport.
> + *
> + * @param[in] dev
> + * Pointer to rte_eth_dev structure.
> + * @param[in] action
> + * Pointer to the port ID action.
> + * @param[out] dst_port_id
> + * The target port ID.
> + * @param[out] error
> + * Pointer to the error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
> + const struct rte_flow_action *action,
> + uint32_t *dst_port_id,
> + struct rte_flow_error *error)
> +{
> + uint32_t port;
> + uint16_t port_id;
> + int ret;
> + const struct rte_flow_action_port_id *conf =
> + (const struct rte_flow_action_port_id *)action->conf;
> +
> + port = conf->original ? dev->data->port_id : conf->id;
> + ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
> + if (ret)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "No eswitch info was found for port");
> + *dst_port_id = port_id;
> + return 0;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -3524,10 +3626,24 @@ struct field_modify_info modify_tcp[] = {
> const struct rte_flow_action_jump *jump_data;
> struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
> struct mlx5_flow_tbl_resource *tbl;
> + uint32_t port_id = 0;
> + struct mlx5_flow_dv_port_id_action_resource port_id_resource;
>
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> + if (flow_dv_translate_action_port_id(dev, action,
> + &port_id, error))
> + return -rte_errno;
> + port_id_resource.port_id = port_id;
> + if (flow_dv_port_id_action_resource_register
> + (dev, &port_id_resource, dev_flow, error))
> + return -rte_errno;
> + dev_flow->dv.actions[actions_n++] =
> + dev_flow->dv.port_id_action->action;
> + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> + break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> tag_resource.tag =
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> @@ -4131,6 +4247,37 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Release port ID action resource.
> + *
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
> +{
> + struct mlx5_flow_dv_port_id_action_resource *cache_resource =
> + flow->dv.port_id_action;
> +
> + assert(cache_resource->action);
> + DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action
> + (cache_resource->action));
> + LIST_REMOVE(cache_resource, next);
> + rte_free(cache_resource);
> + DRV_LOG(DEBUG, "port id action resource %p: removed",
> + (void *)cache_resource);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Remove the flow from the NIC but keeps it in memory.
> *
> * @param[in] dev
> @@ -4197,6 +4344,8 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_modify_hdr_resource_release(dev_flow);
> if (dev_flow->dv.jump)
> flow_dv_jump_tbl_resource_release(dev_flow);
> + if (dev_flow->dv.port_id_action)
> + flow_dv_port_id_action_resource_release(dev_flow);
> rte_free(dev_flow);
> }
> }
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index a508faa..117190f 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -382,6 +382,18 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
> +{
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + return mlx5dv_dr_create_action_dest_vport(ns, vport);
> +#else
> + (void)ns;
> + (void)vport;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> {
> #ifdef HAVE_MLX5DV_DR
> @@ -847,6 +859,8 @@
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> .dr_create_flow_action_dest_flow_tbl =
> mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> + .dr_create_flow_action_dest_vport =
> + mlx5_glue_dr_create_flow_action_dest_vport,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns,
> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> index 058e9b1..26f5cb3 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -146,6 +146,7 @@ struct mlx5_glue {
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> + void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type
2019-04-14 21:12 ` [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type Ori Kam
@ 2019-04-18 1:16 ` Yongseok Koh
2019-04-18 5:13 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 1:16 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:36PM +0000, Ori Kam wrote:
> Actions like encap/decap, modify header require setting the flow table
> type. Until now we supported only Nic RX and Nic TX, this commits adds
> the support for FDB table type for those actions.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 54 ++++++++++++++++++++++++++++++-----------
> 1 file changed, 40 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index f4b7f06..d4dc439 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -40,6 +40,10 @@
> #define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
> #endif
>
> +#ifndef HAVE_MLX5DV_DR_ESWITCH
> +#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
> +#endif
> +
> union flow_dv_attr {
> struct {
> uint32_t valid:1;
> @@ -943,7 +947,9 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5dv_dr_ns *ns;
>
> resource->flags = flow->group ? 0 : 1;
> - if (flow->ingress)
> + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> + ns = sh->fdb_ns;
> + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
> ns = sh->rx_ns;
> else
> ns = sh->tx_ns;
> @@ -1364,6 +1370,8 @@ struct field_modify_info modify_tcp[] = {
> * Pointer to action structure.
> * @param[in, out] dev_flow
> * Pointer to the mlx5_flow.
> + * @param[in] transfer
> + * Mark if the flow is E-Switch flow.
> * @param[out] error
> * Pointer to the error structure.
> *
> @@ -1374,6 +1382,7 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
> const struct rte_flow_action *action,
> struct mlx5_flow *dev_flow,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> const struct rte_flow_item *encap_data;
> @@ -1384,6 +1393,8 @@ struct field_modify_info modify_tcp[] = {
> .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
> };
>
> + if (transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
Why overwrite?
You could've done it above:
struct mlx5_flow_dv_encap_decap_resource res = {
.reformat_type =
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL,
.ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
};
Same for the rest.
> if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> raw_encap_data =
> (const struct rte_flow_action_raw_encap *)action->conf;
> @@ -1416,6 +1427,8 @@ struct field_modify_info modify_tcp[] = {
> * Pointer to rte_eth_dev structure.
> * @param[in, out] dev_flow
> * Pointer to the mlx5_flow.
> + * @param[in] transfer
> + * Mark if the flow is E-Switch flow.
> * @param[out] error
> * Pointer to the error structure.
> *
> @@ -1425,6 +1438,7 @@ struct field_modify_info modify_tcp[] = {
> static int
> flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
> struct mlx5_flow *dev_flow,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> struct mlx5_flow_dv_encap_decap_resource res = {
> @@ -1434,6 +1448,8 @@ struct field_modify_info modify_tcp[] = {
> .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
> };
>
> + if (transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
> return rte_flow_error_set(error, EINVAL,
> RTE_FLOW_ERROR_TYPE_ACTION,
> @@ -1474,8 +1490,11 @@ struct field_modify_info modify_tcp[] = {
> res.reformat_type = attr->egress ?
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL :
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
> - res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> - MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> + if (attr->transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> + else
> + res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> + MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
> return rte_flow_error_set(error, EINVAL,
> RTE_FLOW_ERROR_TYPE_ACTION,
> @@ -1810,11 +1829,14 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_ibv_shared *sh = priv->sh;
> struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
> + struct mlx5dv_dr_ns *ns;
>
> - struct mlx5dv_dr_ns *ns =
> - resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
> - sh->tx_ns : sh->rx_ns;
> -
> + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> + ns = sh->fdb_ns;
> + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
> + ns = sh->tx_ns;
> + else
> + ns = sh->rx_ns;
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
> if (resource->ft_type == cache_resource->ft_type &&
> @@ -3615,6 +3637,8 @@ struct field_modify_info modify_tcp[] = {
> union flow_dv_attr flow_attr = { .attr = 0 };
> struct mlx5_flow_dv_tag_resource tag_resource;
>
> + if (attr->transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
This res has nothing to do with encap/decap but it is
mlx5_flow_dv_modify_hdr_resource.
Thanks,
Yongseok
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1;
> for (; !actions_end ; actions++) {
> @@ -3720,7 +3744,9 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
> case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
> if (flow_dv_create_action_l2_encap(dev, actions,
> - dev_flow, error))
> + dev_flow,
> + attr->transfer,
> + error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> @@ -3732,6 +3758,7 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
> case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> + attr->transfer,
> error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> @@ -3751,9 +3778,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow->dv.encap_decap->verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> - if (flow_dv_create_action_l2_encap(dev, actions,
> - dev_flow,
> - error))
> + if (flow_dv_create_action_l2_encap
> + (dev, actions, dev_flow, attr->transfer,
> + error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> @@ -3768,9 +3795,8 @@ struct field_modify_info modify_tcp[] = {
> }
> /* Handle decap only if it isn't followed by encap. */
> if (action->type != RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> - if (flow_dv_create_action_l2_decap(dev,
> - dev_flow,
> - error))
> + if (flow_dv_create_action_l2_decap
> + (dev, dev_flow, attr->transfer, error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-14 21:12 ` [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
@ 2019-04-18 1:28 ` Yongseok Koh
2019-04-18 5:15 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 1:28 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Sun, Apr 14, 2019 at 09:12:37PM +0000, Ori Kam wrote:
> This commit adds support for drop action when creating E-Switch flow
> using DV.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.c | 9 +++++++++
> drivers/net/mlx5/mlx5.h | 1 +
> drivers/net/mlx5/mlx5_flow_dv.c | 26 ++++++++++++++++----------
> drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 39 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index 938ba1c..e3c6d24 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> goto error;
> }
> sh->fdb_ns = ns;
> + sh->drop_action = mlx5_glue->dr_create_flow_action_drop();
> }
> #endif
> sh->dv_refcnt++;
> @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->drop_action) {
> + mlx5_glue->destroy_flow_action(sh->drop_action);
> + sh->drop_action = NULL;
> + }
> return err;
> #else
> (void)priv;
> @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->drop_action) {
> + mlx5_glue->destroy_flow_action(sh->drop_action);
> + sh->drop_action = NULL;
> + }
> #endif
> pthread_mutex_destroy(&sh->dv_mutex);
> #else
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 016984d..fd85b9d 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -275,6 +275,7 @@ struct mlx5_ibv_shared {
> /* RX Direct Rules tables. */
> void *tx_ns; /* TX Direct Rules name space handle. */
> struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> + void *drop_action; /* Pointer to drop action. */
Better to mention esw here?
E.g. esw_drop_action.
> /* TX Direct Rules tables/ */
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index d4dc439..4a8e894 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -4062,6 +4062,7 @@ struct field_modify_info modify_tcp[] = {
> {
> struct mlx5_flow_dv *dv;
> struct mlx5_flow *dev_flow;
> + struct mlx5_priv *priv = dev->data->dev_private;
> int n;
> int err;
>
> @@ -4069,17 +4070,22 @@ struct field_modify_info modify_tcp[] = {
> dv = &dev_flow->dv;
> n = dv->actions_n;
> if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> - dv->hrxq = mlx5_hrxq_drop_new(dev);
> - if (!dv->hrxq) {
> - rte_flow_error_set
> - (error, errno,
> - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> - "cannot get drop hash queue");
> - goto error;
> + if (flow->transfer)
> + dv->actions[n++] = priv->sh->drop_action;
> + else {
> + dv->hrxq = mlx5_hrxq_drop_new(dev);
> + if (!dv->hrxq) {
> + rte_flow_error_set
> + (error, errno,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "cannot get drop hash queue");
> + goto error;
> + }
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
> }
> - dv->actions[n++] =
> - mlx5_glue->dv_create_flow_action_dest_ibv_qp
> - (dv->hrxq->qp);
This seems to be conflicting to your previous fix, which is:
"net/mlx5: fix release of jump to queue action"
Please rebase.
Thanks,
Yongseok
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index 117190f..b32cd09c 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -394,6 +394,16 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_drop(void)
> +{
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + return mlx5dv_dr_create_action_drop();
> +#else
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> {
> #ifdef HAVE_MLX5DV_DR
> @@ -861,6 +871,8 @@
> mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> .dr_create_flow_action_dest_vport =
> mlx5_glue_dr_create_flow_action_dest_vport,
> + .dr_create_flow_action_drop =
> + mlx5_glue_dr_create_flow_action_drop,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns,
> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> index 26f5cb3..1d06583 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -147,6 +147,7 @@ struct mlx5_glue {
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> + void *(*dr_create_flow_action_drop)();
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-17 23:59 ` Yongseok Koh
@ 2019-04-18 4:40 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 4:40 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Koh,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 3:00 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch
>
> On Sun, Apr 14, 2019 at 09:12:32PM +0000, Ori Kam wrote:
> > Add validation logic for E-Switch using Direct Rules.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5.h | 2 +
> > drivers/net/mlx5/mlx5_ethdev.c | 39 +++++++
> > drivers/net/mlx5/mlx5_flow.h | 5 +
> > drivers/net/mlx5/mlx5_flow_dv.c | 252
> ++++++++++++++++++++++++++++++++++++++--
> > 4 files changed, 287 insertions(+), 11 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> > index 33a4127..8d63575 100644
> > --- a/drivers/net/mlx5/mlx5.h
> > +++ b/drivers/net/mlx5/mlx5.h
> > @@ -412,6 +412,8 @@ int mlx5_ibv_device_to_pci_addr(const struct
> ibv_device *device,
> > unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
> > uint16_t *port_list,
> > unsigned int port_list_n);
> > +int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
> > + uint16_t *es_port_id);
> > int mlx5_sysfs_switch_info(unsigned int ifindex,
> > struct mlx5_switch_info *info);
> > bool mlx5_translate_port_name(const char *port_name_in,
> > diff --git a/drivers/net/mlx5/mlx5_ethdev.c
> b/drivers/net/mlx5/mlx5_ethdev.c
> > index 3992918..c821772 100644
> > --- a/drivers/net/mlx5/mlx5_ethdev.c
> > +++ b/drivers/net/mlx5/mlx5_ethdev.c
> > @@ -1376,6 +1376,45 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev,
> char *fw_ver, size_t fw_size)
> > }
> >
> > /**
> > + * Get the e-switch domain id this port belongs to.
>
> E-Switch
Will fix.
>
> > + *
> > + * @param[in] port
> > + * Device port id.
> > + * @param[out] es_domain_id
> > + * e-switch domain id.
>
> E-Switch
> Please correct in the entire patchset.
>
> > + * @param[out] es_port_id
> > + * The port id of the port in the switch.
> > + *
> > + * @return
> > + * 0 on success, Negative error otherwise.
>
> From looking at the use-cases below, rte_errno must be set.
>
Will fix.
> > + */
> > +int
> > +mlx5_port_to_eswitch_info(uint16_t port,
> > + uint16_t *es_domain_id, uint16_t *es_port_id)
> > +{
> > + struct rte_eth_dev *dev;
> > + struct mlx5_priv *priv;
> > +
> > + if (port >= RTE_MAX_ETHPORTS)
> > + return -EINVAL;
> > + dev = &rte_eth_devices[port];
> > + if (dev == NULL)
> > + return -ENODEV;
>
> dev is an l-value, it cannot be null.
> The above two checks can be replaced with rte_eth_dev_is_valid_port().
>
Will fix.
> > + if (!dev->device ||
> > + !dev->device->driver ||
> > + strcmp(dev->device->driver->name, MLX5_DRIVER_NAME))
> > + return -EINVAL;
>
> Looks too paranoid. The function is just PMD-internal.
>
Will delete.
> > + priv = dev->data->dev_private;
> > + if (!(priv->representor || priv->master))
> > + return -EINVAL;
> > + if (es_domain_id)
> > + *es_domain_id = priv->domain_id;
> > + if (es_port_id)
> > + *es_port_id = priv->vport_id;
>
> It is okay for now but we need to define a new struct like esw_info next time.
> This info should be grouped in the priv.
>
I think I need a new member to mark it is pf or vd, can we in this patch set add it as is?
I will create a patch to move all of the E-Switch variables to dedicated structure for next version.
> > + return 0;
> > +}
> > +
> > +/**
> > * Get switch information associated with network interface.
> > *
> > * @param ifindex
> > diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> > index 9f47fd4..85954c2 100644
> > --- a/drivers/net/mlx5/mlx5_flow.h
> > +++ b/drivers/net/mlx5/mlx5_flow.h
> > @@ -48,6 +48,7 @@
> >
> > /* General pattern items bits. */
> > #define MLX5_FLOW_ITEM_METADATA (1u << 16)
> > +#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
> >
> > /* Outer Masks. */
> > #define MLX5_FLOW_LAYER_OUTER_L3 \
> > @@ -118,6 +119,10 @@
> > (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
> > MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
> >
> > +#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
> > + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
> > + MLX5_FLOW_ACTION_JUMP)
> > +
> > #define MLX5_FLOW_ENCAP_ACTIONS
> (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
> > MLX5_FLOW_ACTION_NVGRE_ENCAP | \
> > MLX5_FLOW_ACTION_RAW_ENCAP)
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index 7b582f0..fedc6cb 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -613,6 +613,92 @@ struct field_modify_info modify_tcp[] = {
> > }
> >
> > /**
> > + * Validate vport item.
> > + *
> > + * @param[in] dev
> > + * Pointer to the rte_eth_dev structure.
> > + * @param[in] item
> > + * Item specification.
> > + * @param[in] attr
> > + * Attributes of flow that includes this item.
> > + * @param[in] item_flags
> > + * Bit-fields that holds the items detected until now.
> > + * @param[out] error
> > + * Pointer to error structure.
> > + *
> > + * @return
> > + * 0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +static int
> > +flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
> > + const struct rte_flow_item *item,
> > + const struct rte_flow_attr *attr,
> > + uint64_t item_flags,
> > + struct rte_flow_error *error)
> > +{
> > + const struct rte_flow_item_port_id *spec = item->spec;
> > + const struct rte_flow_item_port_id *mask = item->mask;
> > + const struct rte_flow_item_port_id switch_mask = {
> > + .id = 0xffffffff,
> > + };
> > + uint16_t esw_domain_id;
> > + uint16_t item_port_esw_domain_id;
> > + uint16_t item_port_esw_port_id;
> > + uint16_t port;
> > + int ret;
> > +
> > + if (!attr->transfer)
> > + return rte_flow_error_set(error, EINVAL,
> > + RTE_FLOW_ERROR_TYPE_ITEM,
> > + NULL,
> > + "match on port id is valid for"
> > + " eswitch only");
>
> Need to mention about 'transfer' flag here instead of esw. BTW, is it okay to
> speak in PMD specific language for error messages? Even if so, 'eswitch' should
> be fixed. Please re-visit all the error messages again.
>
I think the correct message should be that the eswitch should be replaced by E-Switch
Other option is to say something like this "matching on port id must have the transfer attribute"
Which do you prefer?
> > + if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
> > + return rte_flow_error_set(error, ENOTSUP,
> > + RTE_FLOW_ERROR_TYPE_ITEM,
> item,
> > + "multiple source vport are not"
> > + " supported");
>
> Same here. 'vport' doesn't look appropriate.
>
Will change from vport to ports.
> > + if (!mask)
> > + mask = &switch_mask;
> > + if (mask->id && mask->id != 0xffffffff)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ITEM_MASK,
> > + mask,
> > + "no support for partial mask on"
> > + " \"id\" field");
> > + ret = mlx5_flow_item_acceptable
> > + (item, (const uint8_t *)mask,
> > + (const uint8_t
> *)&rte_flow_item_port_id_mask,
> > + sizeof(struct rte_flow_item_port_id),
> > + error);
> > + if (ret)
> > + return ret;
> > + if (!spec)
> > + return 0;
> > + port = mask->id ? spec->id : 0;
>
> Non-masked value means 'any' value. Is it correct to set port 0 in
> this case?
>
Will check again.
> > + ret = mlx5_port_to_eswitch_info(port, &item_port_esw_domain_id,
> > + &item_port_esw_port_id);
>
> item_port_esw_port_id is of no use here;
>
Will remove the variable.
> > + if (ret)
> > + return rte_flow_error_set(error, -ret,
> > +
> RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> > + "failed to obtain eswitch info for"
> > + " port");
> > + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> > + &esw_domain_id, NULL);
> > + if (ret < 0)
> > + return rte_flow_error_set(error, -ret,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "failed to obtain eswitch info");
> > + if (item_port_esw_domain_id != esw_domain_id)
> > + return rte_flow_error_set(error, -ret,
> > +
> RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> > + "cannot match on a port from a"
> > + " different eswitch");
> > + return 0;
> > +}
> > +
> > +/**
> > * Validate count action.
> > *
> > * @param[in] dev
> > @@ -622,6 +708,7 @@ struct field_modify_info modify_tcp[] = {
> > *
> > * @return
> > * 0 on success, a negative errno value otherwise and rte_errno is set.
> > + * w
>
> What is this change?
>
Will fix
> > */
> > static int
> > flow_dv_validate_action_count(struct rte_eth_dev *dev,
> > @@ -676,7 +763,7 @@ struct field_modify_info modify_tcp[] = {
> > RTE_FLOW_ERROR_TYPE_ACTION,
> NULL,
> > "can only have a single encap or"
> > " decap action in a flow");
> > - if (attr->ingress)
> > + if (!attr->transfer && attr->ingress)
> > return rte_flow_error_set(error, ENOTSUP,
> >
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> > NULL,
> > @@ -761,7 +848,8 @@ struct field_modify_info modify_tcp[] = {
> > "can only have a single encap"
> > " action in a flow");
> > /* encap without preceding decap is not supported for ingress */
> > - if (attr->ingress && !(action_flags &
> MLX5_FLOW_ACTION_RAW_DECAP))
> > + if (!attr->transfer && attr->ingress &&
> > + !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
> > return rte_flow_error_set(error, ENOTSUP,
> >
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> > NULL,
> > @@ -1561,6 +1649,77 @@ struct field_modify_info modify_tcp[] = {
> > return 0;
> > }
> >
> > +/*
> > + * Validate the port_id action.
> > + *
> > + * @param[in] dev
> > + * Pointer to rte_eth_dev structure.
> > + * @param[in] action_flags
> > + * Bit-fields that holds the actions detected until now.
> > + * @param[in] action
> > + * Port_id RTE action structure.
> > + * @param[in] attr
> > + * Attributes of flow that includes this action.
> > + * @param[out] error
> > + * Pointer to error structure.
> > + *
> > + * @return
> > + * 0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +static int
> > +flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
> > + uint64_t action_flags,
> > + const struct rte_flow_action *action,
> > + const struct rte_flow_attr *attr,
> > + struct rte_flow_error *error)
> > +{
> > + const struct rte_flow_action_port_id *port_id;
> > + uint16_t port;
> > + uint16_t esw_domain_id;
> > + uint16_t act_port_domain_id;
> > + int ret;
> > +
> > + if (!attr->transfer)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "port id action is valid in transfer"
> > + " mode only");
> > + if (!action || !action->conf)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> > + NULL,
> > + "port id action parameters must be"
> > + " specified");
> > + if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
> > + MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> > + return rte_flow_error_set(error, EINVAL,
> > + RTE_FLOW_ERROR_TYPE_ACTION,
> NULL,
> > + "can have only one fate actions in"
> > + " a flow");
> > + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> > + &esw_domain_id, NULL);
> > + if (ret < 0)
> > + return rte_flow_error_set(error, -ret,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "failed to obtain eswitch info");
> > + port_id = action->conf;
> > + port = port_id->original ? dev->data->port_id : port_id->id;
> > + ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
> > + if (ret)
> > + return rte_flow_error_set
> > + (error, -ret,
> > + RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> port_id,
> > + "failed to obtain eswitch port-id for port");
> > + if (act_port_domain_id != esw_domain_id)
> > + return rte_flow_error_set
> > + (error, -ret,
> > + RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> > + "port does not belong to"
> > + " eswitch being configured");
> > + return 0;
> > +}
> >
> > /**
> > * Find existing modify-header resource or create and register a new one.
> > @@ -1759,11 +1918,34 @@ struct field_modify_info modify_tcp[] = {
> >
> RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
> > NULL,
> > "priority out of range");
> > - if (attributes->transfer)
> > - return rte_flow_error_set(error, ENOTSUP,
> > -
> RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> > - NULL,
> > - "transfer is not supported");
> > + if (attributes->transfer) {
> > + if (!priv->config.dv_eswitch_en)
> > + return rte_flow_error_set
> > + (error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "eswitch dr is not supported");
>
> If you open a parenthesis in a new line, you should indent by a tab.
>
Missed that, fill fix.
> > + if (!(priv->representor || priv->master))
> > + return rte_flow_error_set
> > + (error, EINVAL,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "eswitch configurationd can only be"
> > + " done by a master or a representor"
> > + " device");
> > + if (attributes->egress)
> > + return rte_flow_error_set
> > + (error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ATTR_EGRESS,
> > + attributes, "egress is not supported");
> > + if (attributes->group >= MLX5_MAX_TABLES_FDB)
> > + return rte_flow_error_set
> > + (error, EINVAL,
> > +
> RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> > + NULL,
> > + "group must be smaller than "
> > + RTE_STR(MLX5_MAX_FDB_TABLES));
> > + }
> > if (!(attributes->egress ^ attributes->ingress))
> > return rte_flow_error_set(error, ENOTSUP,
> > RTE_FLOW_ERROR_TYPE_ATTR,
> NULL,
> > @@ -1812,6 +1994,13 @@ struct field_modify_info modify_tcp[] = {
> > switch (items->type) {
> > case RTE_FLOW_ITEM_TYPE_VOID:
> > break;
> > + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> > + ret = flow_dv_validate_item_port_id
> > + (dev, items, attr, item_flags, error);
> > + if (ret < 0)
> > + return ret;
> > + item_flags |= MLX5_FLOW_ITEM_PORT_ID;
> > + break;
>
> Shouldn't it use last_item?
>
Good question, should the port id be in some specific order?
We can force it to be first or last the which makes sense.
I will change to last_item.
> > case RTE_FLOW_ITEM_TYPE_ETH:
> > ret = mlx5_flow_validate_item_eth(items, item_flags,
> > error);
> > @@ -1943,6 +2132,17 @@ struct field_modify_info modify_tcp[] = {
> > switch (actions->type) {
> > case RTE_FLOW_ACTION_TYPE_VOID:
> > break;
> > + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> > + ret = flow_dv_validate_action_port_id(dev,
> > + action_flags,
> > + actions,
> > + attr,
> > + error);
> > + if (ret)
> > + return ret;
> > + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> > + ++actions_n;
> > + break;
> > case RTE_FLOW_ACTION_TYPE_FLAG:
> > ret = mlx5_flow_validate_action_flag(action_flags,
> > attr, error);
> > @@ -2133,10 +2333,40 @@ struct field_modify_info modify_tcp[] = {
> > "action not supported");
> > }
> > }
> > - if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
> > - return rte_flow_error_set(error, EINVAL,
> > - RTE_FLOW_ERROR_TYPE_ACTION,
> actions,
> > - "no fate action is found");
> > + /* Eswitch has few restrictions on using items and actions */
> > + if (attr->transfer) {
> > + if (action_flags & MLX5_FLOW_ACTION_FLAG)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + NULL,
> > + "unsupported action FLAG");
> > + if (action_flags & MLX5_FLOW_ACTION_MARK)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + NULL,
> > + "unsupported action MARK");
> > + if (action_flags & MLX5_FLOW_ACTION_QUEUE)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + NULL,
> > + "unsupported action
> QUEUE");
> > + if (action_flags & MLX5_FLOW_ACTION_RSS)
> > + return rte_flow_error_set(error, ENOTSUP,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + NULL,
> > + "unsupported action RSS");
> > + if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> > + return rte_flow_error_set(error, EINVAL,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + actions,
> > + "no fate action is found");
> > + } else {
> > + if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr-
> >ingress)
> > + return rte_flow_error_set(error, EINVAL,
> > +
> RTE_FLOW_ERROR_TYPE_ACTION,
> > + actions,
> > + "no fate action is found");
> > + }
> > return 0;
> > }
> >
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-18 0:19 ` Yongseok Koh
@ 2019-04-18 4:43 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 4:43 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Yongseok,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 3:20 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs
>
> On Sun, Apr 14, 2019 at 09:12:33PM +0000, Ori Kam wrote:
> > Adds the port ID item to the DV steering code.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5_flow_dv.c | 86 ++++++++++++++++++++++++++++++--
> ---------
> > 1 file changed, 63 insertions(+), 23 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index fedc6cb..e66ee34 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -3095,6 +3095,64 @@ struct field_modify_info modify_tcp[] = {
> > }
> > }
> >
> > +/**
> > + * Add source vport match to the specified matcher.
> > + *
> > + * @param[in, out] matcher
> > + * Flow matcher.
> > + * @param[in, out] key
> > + * Flow matcher value.
> > + * @param[in] port
> > + * Source vport value to match
> > + * @param[in] mask
> > + * Mask
> > + */
> > +static void
> > +flow_dv_translate_item_source_vport(void *matcher, void *key,
> > + int16_t port, uint16_t mask)
> > +{
> > + void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher,
> misc_parameters);
> > + void *misc_v = MLX5_ADDR_OF(fte_match_param, key,
> misc_parameters);
> > +
> > + MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> > + MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> > +}
> > +
> > +/**
> > + * Translate port-id item to eswitch match on port-id.
> > + *
> > + * @param[in] dev
> > + * The devich to configure through.
> > + * @param[in, out] matcher
> > + * Flow matcher.
> > + * @param[in, out] key
> > + * Flow matcher value.
> > + * @param[in] item
> > + * Flow pattern to translate.
> > + *
> > + * @return
> > + * 0 on success, a negative errno value otherwise.
> > + */
> > +static int
> > +flow_dv_eswitch_translate_item_port_id(struct rte_eth_dev *dev,
>
> Why did you put _eswitch in the function name unlike
> flow_dv_validate_item_port_id()? It sounds awkward.
>
Right will fix.
> > + void *matcher, void *key,
> > + const struct rte_flow_item *item)
> > +{
> > + const struct rte_flow_item_port_id *pid_m = item ? item->mask :
> NULL;
> > + const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
> > + uint16_t mask, val, id;
> > + int ret;
> > +
> > + mask = pid_m ? pid_m->id : 0xffff;
> > + id = pid_v ? pid_v->id : dev->data->port_id;
> > + id = mask ? id : dev->data->port_id;
>
> Again, doesn't zero mask mean 'any' value?
>
Will double check. In any case I think we can agree that it doesn't make sense to
Have rules for different flows. So even if the Nic allows we can disable this feature.
What do you think?
> Thanks,
> Yongseok
>
> > + ret = mlx5_port_to_eswitch_info(id, NULL, &val);
> > + if (ret)
> > + return ret;
> > + flow_dv_translate_item_source_vport(matcher, key, val, mask);
> > + return 0;
> > +}
> > +
> > static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
> >
> > #define HEADER_IS_ZERO(match_criteria, headers)
> \
> > @@ -3305,29 +3363,6 @@ struct field_modify_info modify_tcp[] = {
> > }
> >
> > /**
> > - * Add source vport match to the specified matcher.
> > - *
> > - * @param[in, out] matcher
> > - * Flow matcher.
> > - * @param[in, out] key
> > - * Flow matcher value.
> > - * @param[in] port
> > - * Source vport value to match
> > - * @param[in] mask
> > - * Mask
> > - */
> > -static void
> > -flow_dv_translate_item_source_vport(void *matcher, void *key,
> > - int16_t port, uint16_t mask)
> > -{
> > - void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher,
> misc_parameters);
> > - void *misc_v = MLX5_ADDR_OF(fte_match_param, key,
> misc_parameters);
> > -
> > - MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> > - MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> > -}
> > -
> > -/**
> > * Find existing tag resource or create and register a new one.
> > *
> > * @param dev[in, out]
> > @@ -3733,6 +3768,11 @@ struct field_modify_info modify_tcp[] = {
> > void *match_value = dev_flow->dv.value.buf;
> >
> > switch (items->type) {
> > + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> > + flow_dv_eswitch_translate_item_port_id
> > + (dev, match_mask, match_value,
> items);
> > + last_item = MLX5_FLOW_ITEM_PORT_ID;
> > + break;
> > case RTE_FLOW_ITEM_TYPE_ETH:
> > flow_dv_translate_item_eth(match_mask,
> match_value,
> > items, tunnel);
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher
2019-04-18 0:38 ` Yongseok Koh
@ 2019-04-18 4:57 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 4:57 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Yongseok,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 3:39 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 6/9] net/mlx5: add transfer attribute to matcher
>
> On Sun, Apr 14, 2019 at 09:12:34PM +0000, Ori Kam wrote:
> > In current implementation the DV steering supported only NIC steering.
> > This commit adds the transfer attribute in order to create a matcher
> > on the FDB tabels.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5_flow.c | 1 +
> > drivers/net/mlx5/mlx5_flow.h | 2 ++
> > drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
> > 3 files changed, 21 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> > index 83abc14..9fd5096 100644
> > --- a/drivers/net/mlx5/mlx5_flow.c
> > +++ b/drivers/net/mlx5/mlx5_flow.c
> > @@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct
> rte_eth_dev *dev, int32_t priority,
> > flow = rte_calloc(__func__, 1, flow_size, 0);
> > flow->drv_type = flow_get_drv_type(dev, attr);
> > flow->ingress = attr->ingress;
> > + flow->transfer = attr->transfer;
> > assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
> > flow->drv_type < MLX5_FLOW_TYPE_MAX);
> > flow->queue = (void *)(flow + 1);
> > diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> > index 85954c2..9d72024 100644
> > --- a/drivers/net/mlx5/mlx5_flow.h
> > +++ b/drivers/net/mlx5/mlx5_flow.h
> > @@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
> > uint16_t crc; /**< CRC of key. */
> > uint16_t priority; /**< Priority of matcher. */
> > uint8_t egress; /**< Egress matcher. */
> > + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
>
> egress and transfer can be bit fields? Those come from rte_flow_attr but I don't
> understand why it needs to be uint8_t individually.
>
You are correct but since the egress was already defined like this I didn't want to add my code differently.
I don't care to change the new code to bit mask and in later patch (after this release) modify also the egress;
> > uint32_t group; /**< The matcher group. */
> > struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
> > };
> > @@ -382,6 +383,7 @@ struct rte_flow {
> > struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
> > uint8_t ingress; /**< 1 if the flow is ingress. */
> > uint32_t group; /**< The group index. */
> > + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
>
> Bit-field?
>
Same as above comment to keep with the ingress variable.
> Just out of curiosity, flow->ingress vs matcher->egress, why?
> rte_flow_attr has both ingress and egress because a flow can be applied for
> both
> directions. But, in mlx5 PMD, it looks exclusive - !ingress == egress vice
> versa. Then, I don't understand why one has ingress and the other one has
> egress even wasting bits.
>
First I agree it is confusing. But it doesn't waste bits since in any case we must store a bit in each structure.
In any case I think I also should put it in my future commites.
> > };
> >
> > typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index e66ee34..b4ca9ca 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -3203,6 +3203,8 @@ struct field_modify_info modify_tcp[] = {
> > * Table id to use.
> > * @param[in] egress
> > * Direction of the table.
> > + * @param[in] transfer
> > + * E-Switch or Nic flow..
>
> Redundant periods (..)
> Nic -> NIC
>
Will fix.
>
> Thanks,
> Yongseok
>
> > * @param[out] error
> > * pointer to error structure.
> > *
> > @@ -3212,6 +3214,7 @@ struct field_modify_info modify_tcp[] = {
> > static struct mlx5_flow_tbl_resource *
> > flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> > uint32_t table_id, uint8_t egress,
> > + uint8_t transfer,
> > struct rte_flow_error *error)
> > {
> > struct mlx5_priv *priv = dev->data->dev_private;
> > @@ -3219,7 +3222,12 @@ struct field_modify_info modify_tcp[] = {
> > struct mlx5_flow_tbl_resource *tbl;
> >
> > #ifdef HAVE_MLX5DV_DR
> > - if (egress) {
> > + if (transfer) {
> > + tbl = &sh->fdb_tbl[table_id];
> > + if (!tbl->obj)
> > + tbl->obj = mlx5_glue->dr_create_flow_tbl
> > + (sh->fdb_ns, table_id);
> > + } else if (egress) {
> > tbl = &sh->tx_tbl[table_id];
> > if (!tbl->obj)
> > tbl->obj = mlx5_glue->dr_create_flow_tbl
> > @@ -3241,7 +3249,9 @@ struct field_modify_info modify_tcp[] = {
> > #else
> > (void)error;
> > (void)tbl;
> > - if (egress)
> > + if (transfer)
> > + return &sh->fdb_tbl[table_id];
> > + else if (egress)
> > return &sh->tx_tbl[table_id];
> > else
> > return &sh->rx_tbl[table_id];
> > @@ -3306,6 +3316,7 @@ struct field_modify_info modify_tcp[] = {
> > matcher->priority == cache_matcher->priority &&
> > matcher->egress == cache_matcher->egress &&
> > matcher->group == cache_matcher->group &&
> > + matcher->transfer == cache_matcher->transfer &&
> > !memcmp((const void *)matcher->mask.buf,
> > (const void *)cache_matcher->mask.buf,
> > cache_matcher->mask.size)) {
> > @@ -3327,7 +3338,8 @@ struct field_modify_info modify_tcp[] = {
> >
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > "cannot allocate matcher memory");
> > tbl = flow_dv_tbl_resource_get(dev, matcher->group *
> MLX5_GROUP_FACTOR,
> > - matcher->egress, error);
> > + matcher->egress, matcher->transfer,
> > + error);
> > if (!tbl) {
> > rte_free(cache_matcher);
> > return rte_flow_error_set(error, ENOMEM,
> > @@ -3654,7 +3666,8 @@ struct field_modify_info modify_tcp[] = {
> > jump_data = action->conf;
> > tbl = flow_dv_tbl_resource_get(dev, jump_data->group
> *
> > MLX5_GROUP_FACTOR,
> > - attr->egress, error);
> > + attr->egress,
> > + attr->transfer, error);
> > if (!tbl)
> > return rte_flow_error_set
> > (error, errno,
> > @@ -3882,6 +3895,7 @@ struct field_modify_info modify_tcp[] = {
> > matcher.priority);
> > matcher.egress = attr->egress;
> > matcher.group = attr->group;
> > + matcher.transfer = attr->transfer;
> > if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
> > return -rte_errno;
> > return 0;
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs
2019-04-18 0:59 ` Yongseok Koh
@ 2019-04-18 5:06 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 5:06 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Yongseok,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 4:00 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs
>
> On Sun, Apr 14, 2019 at 09:12:35PM +0000, Ori Kam wrote:
> > This commits adds the port ID action to DV steering.
>
> Poor title and commit message.
> Don't you add translation of the action?
>
Yes I add translation function but this is internal. the meaning of adding translation is adding
the command. I thought that we shouldn't use PMD wording.
How about this: net/mlx5: add E-Switch port_id action to Direct Verbs
Problem it is too long.
Regarding the message, what about:
This commit adds the E-Switch port_id action to DV which enables E-Switch routing to a destination port
> Thanks,
> Yongseok
>
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5.h | 2 +
> > drivers/net/mlx5/mlx5_flow.h | 12 ++++
> > drivers/net/mlx5/mlx5_flow_dv.c | 149
> ++++++++++++++++++++++++++++++++++++++++
> > drivers/net/mlx5/mlx5_glue.c | 14 ++++
> > drivers/net/mlx5/mlx5_glue.h | 1 +
> > 5 files changed, 178 insertions(+)
> >
> > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> > index 8d63575..016984d 100644
> > --- a/drivers/net/mlx5/mlx5.h
> > +++ b/drivers/net/mlx5/mlx5.h
> > @@ -281,6 +281,8 @@ struct mlx5_ibv_shared {
> > LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource)
> modify_cmds;
> > LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> > LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
> > + LIST_HEAD(port_id_action_list,
> mlx5_flow_dv_port_id_action_resource)
> > + port_id_action_list; /* List of port ID actions. */
> > /* Shared interrupt handler section. */
> > pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
> > uint32_t intr_cnt; /* Interrupt handler reference counter. */
> > diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> > index 9d72024..c419e6b 100644
> > --- a/drivers/net/mlx5/mlx5_flow.h
> > +++ b/drivers/net/mlx5/mlx5_flow.h
> > @@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
> > struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
> > };
> >
> > +/* Port ID resource structure. */
> > +struct mlx5_flow_dv_port_id_action_resource {
> > + LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
> > + /* Pointer to next element. */
> > + rte_atomic32_t refcnt; /**< Reference counter. */
> > + void *action;
> > + /**< Verbs tag action object. */
> > + uint32_t port_id; /**< Port ID value. */
> > +};
> > +
> > /*
> > * Max number of actions per DV flow.
> > * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
> > @@ -289,6 +299,8 @@ struct mlx5_flow_dv {
> > struct ibv_flow *flow; /**< Installed flow. */
> > struct mlx5_flow_dv_jump_tbl_resource *jump;
> > /**< Pointer to the jump action resource. */
> > + struct mlx5_flow_dv_port_id_action_resource *port_id_action;
> > + /**< Pointer to port ID action resource. */
> > #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> > void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> > /**< Action list. */
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index b4ca9ca..f4b7f06 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -1058,6 +1058,70 @@ struct field_modify_info modify_tcp[] = {
> > }
> >
> > /**
> > + * Find existing table port ID resource or create and register a new one.
> > + *
> > + * @param dev[in, out]
> > + * Pointer to rte_eth_dev structure.
> > + * @param[in, out] resource
> > + * Pointer to port ID action resource.
> > + * @parm[in, out] dev_flow
> > + * Pointer to the dev_flow.
> > + * @param[out] error
> > + * pointer to error structure.
> > + *
> > + * @return
> > + * 0 on success otherwise -errno and errno is set.
> > + */
> > +static int
> > +flow_dv_port_id_action_resource_register
> > + (struct rte_eth_dev *dev,
> > + struct mlx5_flow_dv_port_id_action_resource
> *resource,
> > + struct mlx5_flow *dev_flow,
> > + struct rte_flow_error *error)
> > +{
> > + struct mlx5_priv *priv = dev->data->dev_private;
> > + struct mlx5_ibv_shared *sh = priv->sh;
> > + struct mlx5_flow_dv_port_id_action_resource *cache_resource;
> > +
> > + /* Lookup a matching resource from cache. */
> > + LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
> > + if (resource->port_id == cache_resource->port_id) {
> > + DRV_LOG(DEBUG, "port id action resource resource
> %p: "
> > + "refcnt %d++",
> > + (void *)cache_resource,
> > + rte_atomic32_read(&cache_resource-
> >refcnt));
> > + rte_atomic32_inc(&cache_resource->refcnt);
> > + dev_flow->dv.port_id_action = cache_resource;
> > + return 0;
> > + }
> > + }
> > + /* Register new port id action resource. */
> > + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> > + if (!cache_resource)
> > + return rte_flow_error_set(error, ENOMEM,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > + "cannot allocate resource memory");
> > + *cache_resource = *resource;
> > + cache_resource->action =
> > + mlx5_glue->dr_create_flow_action_dest_vport(priv->sh-
> >fdb_ns,
> > + resource->port_id);
> > + if (!cache_resource->action) {
> > + rte_free(cache_resource);
> > + return rte_flow_error_set(error, ENOMEM,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL, "cannot create action");
> > + }
> > + rte_atomic32_init(&cache_resource->refcnt);
> > + rte_atomic32_inc(&cache_resource->refcnt);
> > + LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
> > + dev_flow->dv.port_id_action = cache_resource;
> > + DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
> > + (void *)cache_resource,
> > + rte_atomic32_read(&cache_resource->refcnt));
> > + return 0;
> > +}
> > +
> > +/**
> > * Get the size of specific rte_flow_item_type
> > *
> > * @param[in] item_type
> > @@ -3467,6 +3531,44 @@ struct field_modify_info modify_tcp[] = {
> > }
> >
> > /**
> > + * Translate port ID action to vport.
> > + *
> > + * @param[in] dev
> > + * Pointer to rte_eth_dev structure.
> > + * @param[in] action
> > + * Pointer to the port ID action.
> > + * @param[out] dst_port_id
> > + * The target port ID.
> > + * @param[out] error
> > + * Pointer to the error structure.
> > + *
> > + * @return
> > + * 0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +static int
> > +flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
> > + const struct rte_flow_action *action,
> > + uint32_t *dst_port_id,
> > + struct rte_flow_error *error)
> > +{
> > + uint32_t port;
> > + uint16_t port_id;
> > + int ret;
> > + const struct rte_flow_action_port_id *conf =
> > + (const struct rte_flow_action_port_id *)action->conf;
> > +
> > + port = conf->original ? dev->data->port_id : conf->id;
> > + ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
> > + if (ret)
> > + return rte_flow_error_set(error, -ret,
> > + RTE_FLOW_ERROR_TYPE_ACTION,
> > + NULL,
> > + "No eswitch info was found for port");
> > + *dst_port_id = port_id;
> > + return 0;
> > +}
> > +
> > +/**
> > * Fill the flow with DV spec.
> > *
> > * @param[in] dev
> > @@ -3524,10 +3626,24 @@ struct field_modify_info modify_tcp[] = {
> > const struct rte_flow_action_jump *jump_data;
> > struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
> > struct mlx5_flow_tbl_resource *tbl;
> > + uint32_t port_id = 0;
> > + struct mlx5_flow_dv_port_id_action_resource
> port_id_resource;
> >
> > switch (actions->type) {
> > case RTE_FLOW_ACTION_TYPE_VOID:
> > break;
> > + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> > + if (flow_dv_translate_action_port_id(dev, action,
> > + &port_id, error))
> > + return -rte_errno;
> > + port_id_resource.port_id = port_id;
> > + if (flow_dv_port_id_action_resource_register
> > + (dev, &port_id_resource, dev_flow, error))
> > + return -rte_errno;
> > + dev_flow->dv.actions[actions_n++] =
> > + dev_flow->dv.port_id_action->action;
> > + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> > + break;
> > case RTE_FLOW_ACTION_TYPE_FLAG:
> > tag_resource.tag =
> >
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> > @@ -4131,6 +4247,37 @@ struct field_modify_info modify_tcp[] = {
> > }
> >
> > /**
> > + * Release port ID action resource.
> > + *
> > + * @param flow
> > + * Pointer to mlx5_flow.
> > + *
> > + * @return
> > + * 1 while a reference on it exists, 0 when freed.
> > + */
> > +static int
> > +flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
> > +{
> > + struct mlx5_flow_dv_port_id_action_resource *cache_resource =
> > + flow->dv.port_id_action;
> > +
> > + assert(cache_resource->action);
> > + DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
> > + (void *)cache_resource,
> > + rte_atomic32_read(&cache_resource->refcnt));
> > + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
> > + claim_zero(mlx5_glue->destroy_flow_action
> > + (cache_resource->action));
> > + LIST_REMOVE(cache_resource, next);
> > + rte_free(cache_resource);
> > + DRV_LOG(DEBUG, "port id action resource %p: removed",
> > + (void *)cache_resource);
> > + return 0;
> > + }
> > + return 1;
> > +}
> > +
> > +/**
> > * Remove the flow from the NIC but keeps it in memory.
> > *
> > * @param[in] dev
> > @@ -4197,6 +4344,8 @@ struct field_modify_info modify_tcp[] = {
> > flow_dv_modify_hdr_resource_release(dev_flow);
> > if (dev_flow->dv.jump)
> > flow_dv_jump_tbl_resource_release(dev_flow);
> > + if (dev_flow->dv.port_id_action)
> > + flow_dv_port_id_action_resource_release(dev_flow);
> > rte_free(dev_flow);
> > }
> > }
> > diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> > index a508faa..117190f 100644
> > --- a/drivers/net/mlx5/mlx5_glue.c
> > +++ b/drivers/net/mlx5/mlx5_glue.c
> > @@ -382,6 +382,18 @@
> > }
> >
> > static void *
> > +mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
> > +{
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + return mlx5dv_dr_create_action_dest_vport(ns, vport);
> > +#else
> > + (void)ns;
> > + (void)vport;
> > + return NULL;
> > +#endif
> > +}
> > +
> > +static void *
> > mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> > {
> > #ifdef HAVE_MLX5DV_DR
> > @@ -847,6 +859,8 @@
> > .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> > .dr_create_flow_action_dest_flow_tbl =
> > mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> > + .dr_create_flow_action_dest_vport =
> > + mlx5_glue_dr_create_flow_action_dest_vport,
> > .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> > .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> > .dr_create_ns = mlx5_glue_dr_create_ns,
> > diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> > index 058e9b1..26f5cb3 100644
> > --- a/drivers/net/mlx5/mlx5_glue.h
> > +++ b/drivers/net/mlx5/mlx5_glue.h
> > @@ -146,6 +146,7 @@ struct mlx5_glue {
> > const char *(*port_state_str)(enum ibv_port_state port_state);
> > struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> > void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> > + void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> > void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> > int (*dr_destroy_flow_tbl)(void *tbl);
> > void *(*dr_create_ns)(struct ibv_context *ctx,
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type
2019-04-18 1:16 ` Yongseok Koh
@ 2019-04-18 5:13 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 5:13 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Yongseok,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 4:16 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 8/9] net/mlx5: add Forward Database table type
>
> On Sun, Apr 14, 2019 at 09:12:36PM +0000, Ori Kam wrote:
> > Actions like encap/decap, modify header require setting the flow table
> > type. Until now we supported only Nic RX and Nic TX, this commits adds
> > the support for FDB table type for those actions.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5_flow_dv.c | 54 ++++++++++++++++++++++++++++++--
> ---------
> > 1 file changed, 40 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index f4b7f06..d4dc439 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -40,6 +40,10 @@
> > #define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
> > #endif
> >
> > +#ifndef HAVE_MLX5DV_DR_ESWITCH
> > +#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
> > +#endif
> > +
> > union flow_dv_attr {
> > struct {
> > uint32_t valid:1;
> > @@ -943,7 +947,9 @@ struct field_modify_info modify_tcp[] = {
> > struct mlx5dv_dr_ns *ns;
> >
> > resource->flags = flow->group ? 0 : 1;
> > - if (flow->ingress)
> > + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> > + ns = sh->fdb_ns;
> > + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
> > ns = sh->rx_ns;
> > else
> > ns = sh->tx_ns;
> > @@ -1364,6 +1370,8 @@ struct field_modify_info modify_tcp[] = {
> > * Pointer to action structure.
> > * @param[in, out] dev_flow
> > * Pointer to the mlx5_flow.
> > + * @param[in] transfer
> > + * Mark if the flow is E-Switch flow.
> > * @param[out] error
> > * Pointer to the error structure.
> > *
> > @@ -1374,6 +1382,7 @@ struct field_modify_info modify_tcp[] = {
> > flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
> > const struct rte_flow_action *action,
> > struct mlx5_flow *dev_flow,
> > + uint8_t transfer,
> > struct rte_flow_error *error)
> > {
> > const struct rte_flow_item *encap_data;
> > @@ -1384,6 +1393,8 @@ struct field_modify_info modify_tcp[] = {
> > .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
> > };
> >
> > + if (transfer)
> > + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
>
> Why overwrite?
> You could've done it above:
>
> struct mlx5_flow_dv_encap_decap_resource res = {
> .reformat_type =
>
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNN
> EL,
> .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
> MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
> };
>
> Same for the rest.
>
Will fix.
> > if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> > raw_encap_data =
> > (const struct rte_flow_action_raw_encap *)action-
> >conf;
> > @@ -1416,6 +1427,8 @@ struct field_modify_info modify_tcp[] = {
> > * Pointer to rte_eth_dev structure.
> > * @param[in, out] dev_flow
> > * Pointer to the mlx5_flow.
> > + * @param[in] transfer
> > + * Mark if the flow is E-Switch flow.
> > * @param[out] error
> > * Pointer to the error structure.
> > *
> > @@ -1425,6 +1438,7 @@ struct field_modify_info modify_tcp[] = {
> > static int
> > flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
> > struct mlx5_flow *dev_flow,
> > + uint8_t transfer,
> > struct rte_flow_error *error)
> > {
> > struct mlx5_flow_dv_encap_decap_resource res = {
> > @@ -1434,6 +1448,8 @@ struct field_modify_info modify_tcp[] = {
> > .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
> > };
> >
> > + if (transfer)
> > + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> > if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow,
> error))
> > return rte_flow_error_set(error, EINVAL,
> > RTE_FLOW_ERROR_TYPE_ACTION,
> > @@ -1474,8 +1490,11 @@ struct field_modify_info modify_tcp[] = {
> > res.reformat_type = attr->egress ?
> >
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNN
> EL :
> >
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_
> L2;
> > - res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> > - MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> > + if (attr->transfer)
> > + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> > + else
> > + res.ft_type = attr->egress ?
> MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> > +
> MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> > if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow,
> error))
> > return rte_flow_error_set(error, EINVAL,
> > RTE_FLOW_ERROR_TYPE_ACTION,
> > @@ -1810,11 +1829,14 @@ struct field_modify_info modify_tcp[] = {
> > struct mlx5_priv *priv = dev->data->dev_private;
> > struct mlx5_ibv_shared *sh = priv->sh;
> > struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
> > + struct mlx5dv_dr_ns *ns;
> >
> > - struct mlx5dv_dr_ns *ns =
> > - resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
> > - sh->tx_ns : sh->rx_ns;
> > -
> > + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> > + ns = sh->fdb_ns;
> > + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
> > + ns = sh->tx_ns;
> > + else
> > + ns = sh->rx_ns;
> > /* Lookup a matching resource from cache. */
> > LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
> > if (resource->ft_type == cache_resource->ft_type &&
> > @@ -3615,6 +3637,8 @@ struct field_modify_info modify_tcp[] = {
> > union flow_dv_attr flow_attr = { .attr = 0 };
> > struct mlx5_flow_dv_tag_resource tag_resource;
> >
> > + if (attr->transfer)
> > + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
>
> This res has nothing to do with encap/decap but it is
> mlx5_flow_dv_modify_hdr_resource.
>
Yes like I wrote in the commit, I'm adding the table support for all actions
including the modify.
> Thanks,
> Yongseok
>
> > if (priority == MLX5_FLOW_PRIO_RSVD)
> > priority = priv->config.flow_prio - 1;
> > for (; !actions_end ; actions++) {
> > @@ -3720,7 +3744,9 @@ struct field_modify_info modify_tcp[] = {
> > case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
> > case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
> > if (flow_dv_create_action_l2_encap(dev, actions,
> > - dev_flow, error))
> > + dev_flow,
> > + attr->transfer,
> > + error))
> > return -rte_errno;
> > dev_flow->dv.actions[actions_n++] =
> > dev_flow->dv.encap_decap->verbs_action;
> > @@ -3732,6 +3758,7 @@ struct field_modify_info modify_tcp[] = {
> > case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
> > case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
> > if (flow_dv_create_action_l2_decap(dev, dev_flow,
> > + attr->transfer,
> > error))
> > return -rte_errno;
> > dev_flow->dv.actions[actions_n++] =
> > @@ -3751,9 +3778,9 @@ struct field_modify_info modify_tcp[] = {
> > dev_flow->dv.encap_decap-
> >verbs_action;
> > } else {
> > /* Handle encap without preceding decap. */
> > - if (flow_dv_create_action_l2_encap(dev,
> actions,
> > - dev_flow,
> > - error))
> > + if (flow_dv_create_action_l2_encap
> > + (dev, actions, dev_flow, attr->transfer,
> > + error))
> > return -rte_errno;
> > dev_flow->dv.actions[actions_n++] =
> > dev_flow->dv.encap_decap-
> >verbs_action;
> > @@ -3768,9 +3795,8 @@ struct field_modify_info modify_tcp[] = {
> > }
> > /* Handle decap only if it isn't followed by encap. */
> > if (action->type !=
> RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> > - if (flow_dv_create_action_l2_decap(dev,
> > - dev_flow,
> > - error))
> > + if (flow_dv_create_action_l2_decap
> > + (dev, dev_flow, attr->transfer, error))
> > return -rte_errno;
> > dev_flow->dv.actions[actions_n++] =
> > dev_flow->dv.encap_decap-
> >verbs_action;
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 1:28 ` Yongseok Koh
@ 2019-04-18 5:15 ` Ori Kam
0 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 5:15 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Yongseok,
PSB
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 4:28 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
>
> On Sun, Apr 14, 2019 at 09:12:37PM +0000, Ori Kam wrote:
> > This commit adds support for drop action when creating E-Switch flow
> > using DV.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5.c | 9 +++++++++
> > drivers/net/mlx5/mlx5.h | 1 +
> > drivers/net/mlx5/mlx5_flow_dv.c | 26 ++++++++++++++++----------
> > drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> > drivers/net/mlx5/mlx5_glue.h | 1 +
> > 5 files changed, 39 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> > index 938ba1c..e3c6d24 100644
> > --- a/drivers/net/mlx5/mlx5.c
> > +++ b/drivers/net/mlx5/mlx5.c
> > @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> > goto error;
> > }
> > sh->fdb_ns = ns;
> > + sh->drop_action = mlx5_glue->dr_create_flow_action_drop();
> > }
> > #endif
> > sh->dv_refcnt++;
> > @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > sh->fdb_ns = NULL;
> > }
> > + if (sh->drop_action) {
> > + mlx5_glue->destroy_flow_action(sh->drop_action);
> > + sh->drop_action = NULL;
> > + }
> > return err;
> > #else
> > (void)priv;
> > @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > sh->fdb_ns = NULL;
> > }
> > + if (sh->drop_action) {
> > + mlx5_glue->destroy_flow_action(sh->drop_action);
> > + sh->drop_action = NULL;
> > + }
> > #endif
> > pthread_mutex_destroy(&sh->dv_mutex);
> > #else
> > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> > index 016984d..fd85b9d 100644
> > --- a/drivers/net/mlx5/mlx5.h
> > +++ b/drivers/net/mlx5/mlx5.h
> > @@ -275,6 +275,7 @@ struct mlx5_ibv_shared {
> > /* RX Direct Rules tables. */
> > void *tx_ns; /* TX Direct Rules name space handle. */
> > struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> > + void *drop_action; /* Pointer to drop action. */
>
> Better to mention esw here?
> E.g. esw_drop_action.
>
Sure.
> > /* TX Direct Rules tables/ */
> > LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> > LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index d4dc439..4a8e894 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -4062,6 +4062,7 @@ struct field_modify_info modify_tcp[] = {
> > {
> > struct mlx5_flow_dv *dv;
> > struct mlx5_flow *dev_flow;
> > + struct mlx5_priv *priv = dev->data->dev_private;
> > int n;
> > int err;
> >
> > @@ -4069,17 +4070,22 @@ struct field_modify_info modify_tcp[] = {
> > dv = &dev_flow->dv;
> > n = dv->actions_n;
> > if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> > - dv->hrxq = mlx5_hrxq_drop_new(dev);
> > - if (!dv->hrxq) {
> > - rte_flow_error_set
> > - (error, errno,
> > -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > - "cannot get drop hash queue");
> > - goto error;
> > + if (flow->transfer)
> > + dv->actions[n++] = priv->sh->drop_action;
> > + else {
> > + dv->hrxq = mlx5_hrxq_drop_new(dev);
> > + if (!dv->hrxq) {
> > + rte_flow_error_set
> > + (error, errno,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "cannot get drop hash
> queue");
> > + goto error;
> > + }
> > + dv->actions[n++] =
> > + mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> > + (dv->hrxq->qp);
> > }
> > - dv->actions[n++] =
> > - mlx5_glue-
> >dv_create_flow_action_dest_ibv_qp
> > - (dv->hrxq->qp);
>
> This seems to be conflicting to your previous fix, which is:
> "net/mlx5: fix release of jump to queue action"
>
> Please rebase.
>
Sure.
> Thanks,
> Yongseok
>
> > } else if (flow->actions &
> > (MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)) {
> > struct mlx5_hrxq *hrxq;
> > diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> > index 117190f..b32cd09c 100644
> > --- a/drivers/net/mlx5/mlx5_glue.c
> > +++ b/drivers/net/mlx5/mlx5_glue.c
> > @@ -394,6 +394,16 @@
> > }
> >
> > static void *
> > +mlx5_glue_dr_create_flow_action_drop(void)
> > +{
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + return mlx5dv_dr_create_action_drop();
> > +#else
> > + return NULL;
> > +#endif
> > +}
> > +
> > +static void *
> > mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> > {
> > #ifdef HAVE_MLX5DV_DR
> > @@ -861,6 +871,8 @@
> > mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> > .dr_create_flow_action_dest_vport =
> > mlx5_glue_dr_create_flow_action_dest_vport,
> > + .dr_create_flow_action_drop =
> > + mlx5_glue_dr_create_flow_action_drop,
> > .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> > .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> > .dr_create_ns = mlx5_glue_dr_create_ns,
> > diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> > index 26f5cb3..1d06583 100644
> > --- a/drivers/net/mlx5/mlx5_glue.h
> > +++ b/drivers/net/mlx5/mlx5_glue.h
> > @@ -147,6 +147,7 @@ struct mlx5_glue {
> > struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> > void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> > void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> > + void *(*dr_create_flow_action_drop)();
> > void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> > int (*dr_destroy_flow_tbl)(void *tbl);
> > void *(*dr_create_ns)(struct ibv_context *ctx,
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (8 preceding siblings ...)
2019-04-14 21:12 ` [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name Ori Kam
` (8 more replies)
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
10 siblings, 9 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Currently MLX5 PMD supports 3 flow engines:
Verbs, Direct Verbs and TCF. The first two engines are for Nic steering
while the TCF is for E-Switch steering.
This series add E-Switch steering support also for the DV engine.
In order to support the new capability there should be support from
both the RDMA and from the NIC.
V2:
* Address ML comments
Ori Kam (9):
net/mlx5: fix translate vport function name
net/mlx5: fix meson build for Direct Rules
net/mlx5: add Direct Rules E-Switch support
net/mlx5: add validation for Direct Rule E-Switch
net/mlx5: add port ID item to Direct Verbs
net/mlx5: add transfer attribute to matcher
net/mlx5: add E-Switch port ID action to Direct Verbs
net/mlx5: add Forward Database table type
net/mlx5: add drop action to Direct Verbs E-Switch
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 4 +
drivers/net/mlx5/mlx5.c | 62 +++-
drivers/net/mlx5/mlx5.h | 17 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 44 +++
drivers/net/mlx5/mlx5_ethdev.c | 41 +++
drivers/net/mlx5/mlx5_flow.c | 3 +-
drivers/net/mlx5/mlx5_flow.h | 19 ++
drivers/net/mlx5/mlx5_flow_dv.c | 585 +++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 26 ++
drivers/net/mlx5/mlx5_glue.h | 2 +
drivers/net/mlx5/mlx5_prm.h | 328 +++++++++++++++++++++
12 files changed, 1065 insertions(+), 71 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:06 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
` (7 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Modify the translate vport function to match other translate items
naming convestions.
Fixes: 0fe3f18f78d8 ("net/mlx5: add source vport match to the ingress rules")
Cc: viacheslavo@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5_flow_dv.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 04ab3d6..1e25e0b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3087,8 +3087,8 @@ struct field_modify_info modify_tcp[] = {
* Mask
*/
static void
-flow_dv_translate_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
{
void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
@@ -3492,10 +3492,10 @@ struct field_modify_info modify_tcp[] = {
* Add matching on source vport index only
* for ingress rules in E-Switch configurations.
*/
- flow_dv_translate_source_vport(matcher.mask.buf,
- dev_flow->dv.value.buf,
- priv->vport_id,
- 0xffff);
+ flow_dv_translate_item_source_vport(matcher.mask.buf,
+ dev_flow->dv.value.buf,
+ priv->vport_id,
+ 0xffff);
}
for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) {
int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:09 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
` (6 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
The meson build was missing the define for Direct Rules.
Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
Cc: orika@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/meson.build | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a4c684e..0037e15 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -111,6 +111,8 @@ if build
'mlx5dv_devx_obj_create' ],
[ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
+ [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:11 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
` (5 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit checks the for DR E-Switch support.
The support is based on both Device and Kernel.
This commit also enables the user to manualy disable this this feature.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 2 +
drivers/net/mlx5/mlx5.c | 53 +++++-
drivers/net/mlx5/mlx5.h | 12 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 44 +++++
drivers/net/mlx5/mlx5_flow.c | 2 +-
drivers/net/mlx5/mlx5_prm.h | 328 ++++++++++++++++++++++++++++++++++++++
7 files changed, 440 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 93bc869..2b72a33 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
enum MLX5DV_DR_NS_TYPE_TERMINATING \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR_ESWITCH \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 0037e15..9dfd28d 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -113,6 +113,8 @@ if build
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
[ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
'MLX5DV_DR_NS_TYPE_TERMINATING' ],
+ [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 9ff50df..ff24e1d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -101,6 +101,9 @@
/* Allow L3 VXLAN flow creation. */
#define MLX5_L3_VXLAN_EN "l3_vxlan_en"
+/* Activate DV E-Switch flow steering. */
+#define MLX5_DV_ESW_EN "dv_esw_en"
+
/* Activate DV flow steering. */
#define MLX5_DV_FLOW_EN "dv_flow_en"
@@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
}
pthread_mutex_init(&sh->dv_mutex, NULL);
sh->tx_ns = ns;
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (priv->config.dv_esw_en) {
+ ns = mlx5_glue->dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
+ if (!ns) {
+ DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ sh->fdb_ns = ns;
+ }
+#endif
sh->dv_refcnt++;
priv->dr_shared = 1;
return 0;
@@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
return err;
#else
(void)priv;
@@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
+#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
(void)priv;
@@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
config->l3_vxlan_en = !!tmp;
} else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
config->vf_nl_en = !!tmp;
+ } else if (strcmp(MLX5_DV_ESW_EN, key) == 0) {
+ config->dv_esw_en = !!tmp;
} else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
config->dv_flow_en = !!tmp;
} else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
@@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
MLX5_RX_VEC_EN,
MLX5_L3_VXLAN_EN,
MLX5_VF_NL_EN,
+ MLX5_DV_ESW_EN,
MLX5_DV_FLOW_EN,
MLX5_MR_EXT_MEMSEG_EN,
MLX5_REPRESENTOR,
@@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
- if (config.dv_flow_en) {
- err = mlx5_alloc_shared_dr(priv);
- if (err)
- goto error;
- }
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
@@ -1484,8 +1507,27 @@ struct mlx5_dev_spawn_data {
* Verbs context returned by ibv_open_device().
*/
mlx5_link_update(eth_dev, 0);
+#ifdef HAVE_IBV_DEVX_OBJ
+ err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
+ if (err) {
+ err = -err;
+ goto error;
+ }
+#endif
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (!(config.hca_attr.eswitch_manager && config.dv_flow_en &&
+ (switch_info->representor || switch_info->master)))
+ config.dv_esw_en = 0;
+#else
+ config.dv_esw_en = 0;
+#endif
/* Store device configuration on private structure. */
priv->config = config;
+ if (config.dv_flow_en) {
+ err = mlx5_alloc_shared_dr(priv);
+ if (err)
+ goto error;
+ }
/* Supported Verbs flow priority number detection. */
err = mlx5_flow_discover_priorities(eth_dev);
if (err < 0) {
@@ -1876,6 +1918,7 @@ struct mlx5_dev_spawn_data {
.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
.min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
},
+ .dv_esw_en = 1,
};
/* Device specific configuration. */
switch (pci_dev->id.device_id) {
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 14c7f3c..b9946f6 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
int id; /* Flow counter ID */
};
+/* HCA attributes. */
+struct mlx5_hca_attr {
+ uint32_t eswitch_manager:1;
+};
+
/* Flow list . */
TAILQ_HEAD(mlx5_flows, rte_flow);
@@ -171,6 +176,7 @@ struct mlx5_dev_config {
/* Whether memseg should be extended for MR creation. */
unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
+ unsigned int dv_esw_en:1; /* Enable E-Switch DV flow. */
unsigned int dv_flow_en:1; /* Enable DV flow. */
unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
unsigned int devx:1; /* Whether devx interface is available or not. */
@@ -192,6 +198,7 @@ struct mlx5_dev_config {
int txqs_inline; /* Queue number threshold for inlining. */
int txqs_vec; /* Queue number threshold for vectorized Tx. */
int inline_max_packet_sz; /* Max packet size for inlining. */
+ struct mlx5_hca_attr hca_attr; /* HCA attributes. */
};
/**
@@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
};
#define MLX5_MAX_TABLES 1024
+#define MLX5_MAX_TABLES_FDB 32
#define MLX5_GROUP_FACTOR 1
/*
@@ -260,6 +268,8 @@ struct mlx5_ibv_shared {
/* Shared DV/DR flow data section. */
pthread_mutex_t dv_mutex; /* DV context mutex. */
uint32_t dv_refcnt; /* DV/DR data reference counter. */
+ void *fdb_ns; /* FDB Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
void *rx_ns; /* RX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
/* RX Direct Rules tables. */
@@ -539,4 +549,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct ibv_context *ctx,
int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set *dcx,
int clear,
uint64_t *pkts, uint64_t *bytes);
+int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr);
#endif /* RTE_PMD_MLX5_H_ */
diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c b/drivers/net/mlx5/mlx5_devx_cmds.c
index a9dff58..e5776c4 100644
--- a/drivers/net/mlx5/mlx5_devx_cmds.c
+++ b/drivers/net/mlx5/mlx5_devx_cmds.c
@@ -105,3 +105,47 @@ int mlx5_devx_cmd_flow_counter_free(struct mlx5dv_devx_obj *obj)
*bytes = MLX5_GET64(traffic_counter, stats, octets);
return 0;
}
+
+/**
+ * Query HCA attributes.
+ * Using those attributes we can check on run time if the device
+ * is having the required capabilities.
+ *
+ * @param[in] ctx
+ * ibv contexts returned from mlx5dv_open_device.
+ * @param[out] attr
+ * Attributes device values.
+ *
+ * @return
+ * 0 on success, a negative value otherwise.
+ */
+int
+mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
+ void *hcattr;
+ int status, syndrome, rc;
+
+ MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
+ MLX5_SET(query_hca_cap_in, in, op_mod,
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
+ MLX5_HCA_CAP_OPMOD_GET_CUR);
+
+ rc = mlx5_glue->devx_general_cmd(ctx,
+ in, sizeof(in), out, sizeof(out));
+ if (rc)
+ return rc;
+ status = MLX5_GET(query_hca_cap_out, out, status);
+ syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
+ if (status) {
+ DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
+ "status %x, syndrome = %x",
+ status, syndrome);
+ return -1;
+ }
+ hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
+ attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);
+ return 0;
+}
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index a0683ee..b1effda 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
struct mlx5_priv *priv = dev->data->dev_private;
enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
- if (attr->transfer)
+ if (attr->transfer && !priv->config.dv_esw_en)
type = MLX5_FLOW_TYPE_TCF;
else
type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index b15266f..8c42380 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -529,6 +529,7 @@ enum {
};
enum {
+ MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
};
@@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
u8 flow_counter_id[0x20];
};
+enum {
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
+ MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
+};
+
+enum {
+ MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
+ MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
+};
+
+struct mlx5_ifc_cmd_hca_cap_bits {
+ u8 reserved_at_0[0x30];
+ u8 vhca_id[0x10];
+ u8 reserved_at_40[0x40];
+ u8 log_max_srq_sz[0x8];
+ u8 log_max_qp_sz[0x8];
+ u8 reserved_at_90[0xb];
+ u8 log_max_qp[0x5];
+ u8 reserved_at_a0[0xb];
+ u8 log_max_srq[0x5];
+ u8 reserved_at_b0[0x10];
+ u8 reserved_at_c0[0x8];
+ u8 log_max_cq_sz[0x8];
+ u8 reserved_at_d0[0xb];
+ u8 log_max_cq[0x5];
+ u8 log_max_eq_sz[0x8];
+ u8 reserved_at_e8[0x2];
+ u8 log_max_mkey[0x6];
+ u8 reserved_at_f0[0x8];
+ u8 dump_fill_mkey[0x1];
+ u8 reserved_at_f9[0x3];
+ u8 log_max_eq[0x4];
+ u8 max_indirection[0x8];
+ u8 fixed_buffer_size[0x1];
+ u8 log_max_mrw_sz[0x7];
+ u8 force_teardown[0x1];
+ u8 reserved_at_111[0x1];
+ u8 log_max_bsf_list_size[0x6];
+ u8 umr_extended_translation_offset[0x1];
+ u8 null_mkey[0x1];
+ u8 log_max_klm_list_size[0x6];
+ u8 reserved_at_120[0xa];
+ u8 log_max_ra_req_dc[0x6];
+ u8 reserved_at_130[0xa];
+ u8 log_max_ra_res_dc[0x6];
+ u8 reserved_at_140[0xa];
+ u8 log_max_ra_req_qp[0x6];
+ u8 reserved_at_150[0xa];
+ u8 log_max_ra_res_qp[0x6];
+ u8 end_pad[0x1];
+ u8 cc_query_allowed[0x1];
+ u8 cc_modify_allowed[0x1];
+ u8 start_pad[0x1];
+ u8 cache_line_128byte[0x1];
+ u8 reserved_at_165[0xa];
+ u8 qcam_reg[0x1];
+ u8 gid_table_size[0x10];
+ u8 out_of_seq_cnt[0x1];
+ u8 vport_counters[0x1];
+ u8 retransmission_q_counters[0x1];
+ u8 debug[0x1];
+ u8 modify_rq_counter_set_id[0x1];
+ u8 rq_delay_drop[0x1];
+ u8 max_qp_cnt[0xa];
+ u8 pkey_table_size[0x10];
+ u8 vport_group_manager[0x1];
+ u8 vhca_group_manager[0x1];
+ u8 ib_virt[0x1];
+ u8 eth_virt[0x1];
+ u8 vnic_env_queue_counters[0x1];
+ u8 ets[0x1];
+ u8 nic_flow_table[0x1];
+ u8 eswitch_manager[0x1];
+ u8 device_memory[0x1];
+ u8 mcam_reg[0x1];
+ u8 pcam_reg[0x1];
+ u8 local_ca_ack_delay[0x5];
+ u8 port_module_event[0x1];
+ u8 enhanced_error_q_counters[0x1];
+ u8 ports_check[0x1];
+ u8 reserved_at_1b3[0x1];
+ u8 disable_link_up[0x1];
+ u8 beacon_led[0x1];
+ u8 port_type[0x2];
+ u8 num_ports[0x8];
+ u8 reserved_at_1c0[0x1];
+ u8 pps[0x1];
+ u8 pps_modify[0x1];
+ u8 log_max_msg[0x5];
+ u8 reserved_at_1c8[0x4];
+ u8 max_tc[0x4];
+ u8 temp_warn_event[0x1];
+ u8 dcbx[0x1];
+ u8 general_notification_event[0x1];
+ u8 reserved_at_1d3[0x2];
+ u8 fpga[0x1];
+ u8 rol_s[0x1];
+ u8 rol_g[0x1];
+ u8 reserved_at_1d8[0x1];
+ u8 wol_s[0x1];
+ u8 wol_g[0x1];
+ u8 wol_a[0x1];
+ u8 wol_b[0x1];
+ u8 wol_m[0x1];
+ u8 wol_u[0x1];
+ u8 wol_p[0x1];
+ u8 stat_rate_support[0x10];
+ u8 reserved_at_1f0[0xc];
+ u8 cqe_version[0x4];
+ u8 compact_address_vector[0x1];
+ u8 striding_rq[0x1];
+ u8 reserved_at_202[0x1];
+ u8 ipoib_enhanced_offloads[0x1];
+ u8 ipoib_basic_offloads[0x1];
+ u8 reserved_at_205[0x1];
+ u8 repeated_block_disabled[0x1];
+ u8 umr_modify_entity_size_disabled[0x1];
+ u8 umr_modify_atomic_disabled[0x1];
+ u8 umr_indirect_mkey_disabled[0x1];
+ u8 umr_fence[0x2];
+ u8 reserved_at_20c[0x3];
+ u8 drain_sigerr[0x1];
+ u8 cmdif_checksum[0x2];
+ u8 sigerr_cqe[0x1];
+ u8 reserved_at_213[0x1];
+ u8 wq_signature[0x1];
+ u8 sctr_data_cqe[0x1];
+ u8 reserved_at_216[0x1];
+ u8 sho[0x1];
+ u8 tph[0x1];
+ u8 rf[0x1];
+ u8 dct[0x1];
+ u8 qos[0x1];
+ u8 eth_net_offloads[0x1];
+ u8 roce[0x1];
+ u8 atomic[0x1];
+ u8 reserved_at_21f[0x1];
+ u8 cq_oi[0x1];
+ u8 cq_resize[0x1];
+ u8 cq_moderation[0x1];
+ u8 reserved_at_223[0x3];
+ u8 cq_eq_remap[0x1];
+ u8 pg[0x1];
+ u8 block_lb_mc[0x1];
+ u8 reserved_at_229[0x1];
+ u8 scqe_break_moderation[0x1];
+ u8 cq_period_start_from_cqe[0x1];
+ u8 cd[0x1];
+ u8 reserved_at_22d[0x1];
+ u8 apm[0x1];
+ u8 vector_calc[0x1];
+ u8 umr_ptr_rlky[0x1];
+ u8 imaicl[0x1];
+ u8 reserved_at_232[0x4];
+ u8 qkv[0x1];
+ u8 pkv[0x1];
+ u8 set_deth_sqpn[0x1];
+ u8 reserved_at_239[0x3];
+ u8 xrc[0x1];
+ u8 ud[0x1];
+ u8 uc[0x1];
+ u8 rc[0x1];
+ u8 uar_4k[0x1];
+ u8 reserved_at_241[0x9];
+ u8 uar_sz[0x6];
+ u8 reserved_at_250[0x8];
+ u8 log_pg_sz[0x8];
+ u8 bf[0x1];
+ u8 driver_version[0x1];
+ u8 pad_tx_eth_packet[0x1];
+ u8 reserved_at_263[0x8];
+ u8 log_bf_reg_size[0x5];
+ u8 reserved_at_270[0xb];
+ u8 lag_master[0x1];
+ u8 num_lag_ports[0x4];
+ u8 reserved_at_280[0x10];
+ u8 max_wqe_sz_sq[0x10];
+ u8 reserved_at_2a0[0x10];
+ u8 max_wqe_sz_rq[0x10];
+ u8 max_flow_counter_31_16[0x10];
+ u8 max_wqe_sz_sq_dc[0x10];
+ u8 reserved_at_2e0[0x7];
+ u8 max_qp_mcg[0x19];
+ u8 reserved_at_300[0x10];
+ u8 flow_counter_bulk_alloc[0x08];
+ u8 log_max_mcg[0x8];
+ u8 reserved_at_320[0x3];
+ u8 log_max_transport_domain[0x5];
+ u8 reserved_at_328[0x3];
+ u8 log_max_pd[0x5];
+ u8 reserved_at_330[0xb];
+ u8 log_max_xrcd[0x5];
+ u8 nic_receive_steering_discard[0x1];
+ u8 receive_discard_vport_down[0x1];
+ u8 transmit_discard_vport_down[0x1];
+ u8 reserved_at_343[0x5];
+ u8 log_max_flow_counter_bulk[0x8];
+ u8 max_flow_counter_15_0[0x10];
+ u8 reserved_at_360[0x3];
+ u8 log_max_rq[0x5];
+ u8 reserved_at_368[0x3];
+ u8 log_max_sq[0x5];
+ u8 reserved_at_370[0x3];
+ u8 log_max_tir[0x5];
+ u8 reserved_at_378[0x3];
+ u8 log_max_tis[0x5];
+ u8 basic_cyclic_rcv_wqe[0x1];
+ u8 reserved_at_381[0x2];
+ u8 log_max_rmp[0x5];
+ u8 reserved_at_388[0x3];
+ u8 log_max_rqt[0x5];
+ u8 reserved_at_390[0x3];
+ u8 log_max_rqt_size[0x5];
+ u8 reserved_at_398[0x3];
+ u8 log_max_tis_per_sq[0x5];
+ u8 ext_stride_num_range[0x1];
+ u8 reserved_at_3a1[0x2];
+ u8 log_max_stride_sz_rq[0x5];
+ u8 reserved_at_3a8[0x3];
+ u8 log_min_stride_sz_rq[0x5];
+ u8 reserved_at_3b0[0x3];
+ u8 log_max_stride_sz_sq[0x5];
+ u8 reserved_at_3b8[0x3];
+ u8 log_min_stride_sz_sq[0x5];
+ u8 hairpin[0x1];
+ u8 reserved_at_3c1[0x2];
+ u8 log_max_hairpin_queues[0x5];
+ u8 reserved_at_3c8[0x3];
+ u8 log_max_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3d0[0x3];
+ u8 log_max_hairpin_num_packets[0x5];
+ u8 reserved_at_3d8[0x3];
+ u8 log_max_wq_sz[0x5];
+ u8 nic_vport_change_event[0x1];
+ u8 disable_local_lb_uc[0x1];
+ u8 disable_local_lb_mc[0x1];
+ u8 log_min_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3e8[0x3];
+ u8 log_max_vlan_list[0x5];
+ u8 reserved_at_3f0[0x3];
+ u8 log_max_current_mc_list[0x5];
+ u8 reserved_at_3f8[0x3];
+ u8 log_max_current_uc_list[0x5];
+ u8 general_obj_types[0x40];
+ u8 reserved_at_440[0x20];
+ u8 reserved_at_460[0x10];
+ u8 max_num_eqs[0x10];
+ u8 reserved_at_480[0x3];
+ u8 log_max_l2_table[0x5];
+ u8 reserved_at_488[0x8];
+ u8 log_uar_page_sz[0x10];
+ u8 reserved_at_4a0[0x20];
+ u8 device_frequency_mhz[0x20];
+ u8 device_frequency_khz[0x20];
+ u8 reserved_at_500[0x20];
+ u8 num_of_uars_per_page[0x20];
+ u8 flex_parser_protocols[0x20];
+ u8 reserved_at_560[0x20];
+ u8 reserved_at_580[0x3c];
+ u8 mini_cqe_resp_stride_index[0x1];
+ u8 cqe_128_always[0x1];
+ u8 cqe_compression_128[0x1];
+ u8 cqe_compression[0x1];
+ u8 cqe_compression_timeout[0x10];
+ u8 cqe_compression_max_num[0x10];
+ u8 reserved_at_5e0[0x10];
+ u8 tag_matching[0x1];
+ u8 rndv_offload_rc[0x1];
+ u8 rndv_offload_dc[0x1];
+ u8 log_tag_matching_list_sz[0x5];
+ u8 reserved_at_5f8[0x3];
+ u8 log_max_xrq[0x5];
+ u8 affiliate_nic_vport_criteria[0x8];
+ u8 native_port_num[0x8];
+ u8 num_vhca_ports[0x8];
+ u8 reserved_at_618[0x6];
+ u8 sw_owner_id[0x1];
+ u8 reserved_at_61f[0x1e1];
+};
+
+struct mlx5_ifc_qos_cap_bits {
+ u8 packet_pacing[0x1];
+ u8 esw_scheduling[0x1];
+ u8 esw_bw_share[0x1];
+ u8 esw_rate_limit[0x1];
+ u8 reserved_at_4[0x1];
+ u8 packet_pacing_burst_bound[0x1];
+ u8 packet_pacing_typical_size[0x1];
+ u8 flow_meter_srtcm[0x1];
+ u8 reserved_at_8[0x8];
+ u8 log_max_flow_meter[0x8];
+ u8 flow_meter_reg_id[0x8];
+ u8 reserved_at_25[0x20];
+ u8 packet_pacing_max_rate[0x20];
+ u8 packet_pacing_min_rate[0x20];
+ u8 reserved_at_80[0x10];
+ u8 packet_pacing_rate_table_size[0x10];
+ u8 esw_element_type[0x10];
+ u8 esw_tsar_type[0x10];
+ u8 reserved_at_c0[0x10];
+ u8 max_qos_para_vport[0x10];
+ u8 max_tsar_bw_share[0x20];
+ u8 reserved_at_100[0x6e8];
+};
+
+union mlx5_ifc_hca_cap_union_bits {
+ struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
+ struct mlx5_ifc_qos_cap_bits qos_cap;
+ u8 reserved_at_0[0x8000];
+};
+
+struct mlx5_ifc_query_hca_cap_out_bits {
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
+ union mlx5_ifc_hca_cap_union_bits capability;
+};
+
+struct mlx5_ifc_query_hca_cap_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 reserved_at_40[0x40];
+};
+
/* CQE format mask. */
#define MLX5E_CQE_FORMAT_MASK 0xc
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (2 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:16 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
` (4 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Add validation logic for E-Switch using Direct Rules.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_ethdev.c | 41 +++++++
drivers/net/mlx5/mlx5_flow.h | 5 +
drivers/net/mlx5/mlx5_flow_dv.c | 243 ++++++++++++++++++++++++++++++++++++++--
4 files changed, 280 insertions(+), 11 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index b9946f6..dd22bb6 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -412,6 +412,8 @@ int mlx5_ibv_device_to_pci_addr(const struct ibv_device *device,
unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
uint16_t *port_list,
unsigned int port_list_n);
+int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
+ uint16_t *es_port_id);
int mlx5_sysfs_switch_info(unsigned int ifindex,
struct mlx5_switch_info *info);
bool mlx5_translate_port_name(const char *port_name_in,
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 3992918..695440a 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -1376,6 +1376,47 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size)
}
/**
+ * Get the E-Switch domain id this port belongs to.
+ *
+ * @param[in] port
+ * Device port id.
+ * @param[out] es_domain_id
+ * E-Switch domain id.
+ * @param[out] es_port_id
+ * The port id of the port in the E-Switch.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_port_to_eswitch_info(uint16_t port,
+ uint16_t *es_domain_id, uint16_t *es_port_id)
+{
+ struct rte_eth_dev *dev;
+ struct mlx5_priv *priv;
+
+ if (port >= RTE_MAX_ETHPORTS) {
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ if (!rte_eth_dev_is_valid_port(port)) {
+ rte_errno = ENODEV;
+ return -rte_errno;
+ }
+ dev = &rte_eth_devices[port];
+ priv = dev->data->dev_private;
+ if (!(priv->representor || priv->master)) {
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ if (es_domain_id)
+ *es_domain_id = priv->domain_id;
+ if (es_port_id)
+ *es_port_id = priv->vport_id;
+ return 0;
+}
+
+/**
* Get switch information associated with network interface.
*
* @param ifindex
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9f47fd4..85954c2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -48,6 +48,7 @@
/* General pattern items bits. */
#define MLX5_FLOW_ITEM_METADATA (1u << 16)
+#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
/* Outer Masks. */
#define MLX5_FLOW_LAYER_OUTER_L3 \
@@ -118,6 +119,10 @@
(MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
+#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
+ MLX5_FLOW_ACTION_JUMP)
+
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
MLX5_FLOW_ACTION_RAW_ENCAP)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 1e25e0b..b819359 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -613,6 +613,89 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate vport item.
+ *
+ * @param[in] dev
+ * Pointer to the rte_eth_dev structure.
+ * @param[in] item
+ * Item specification.
+ * @param[in] attr
+ * Attributes of flow that includes this item.
+ * @param[in] item_flags
+ * Bit-fields that holds the items detected until now.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_item *item,
+ const struct rte_flow_attr *attr,
+ uint64_t item_flags,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_port_id *spec = item->spec;
+ const struct rte_flow_item_port_id *mask = item->mask;
+ const struct rte_flow_item_port_id switch_mask = {
+ .id = 0xffffffff,
+ };
+ uint16_t esw_domain_id;
+ uint16_t item_port_esw_domain_id;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ NULL,
+ "match on port id is valid only"
+ " when transfer flag is enabled");
+ if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "multiple source ports are not"
+ " supported");
+ if (!mask)
+ mask = &switch_mask;
+ if (mask->id != 0xffffffff)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK,
+ mask,
+ "no support for partial mask on"
+ " \"id\" field");
+ ret = mlx5_flow_item_acceptable
+ (item, (const uint8_t *)mask,
+ (const uint8_t *)&rte_flow_item_port_id_mask,
+ sizeof(struct rte_flow_item_port_id),
+ error);
+ if (ret)
+ return ret;
+ if (!spec)
+ return 0;
+ ret = mlx5_port_to_eswitch_info(spec->id, &item_port_esw_domain_id,
+ NULL);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "failed to obtain E-Switch info for"
+ " port");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain E-Switch info");
+ if (item_port_esw_domain_id != esw_domain_id)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "cannot match on a port from a"
+ " different E-Switch");
+ return 0;
+}
+
+/**
* Validate count action.
*
* @param[in] dev
@@ -676,7 +759,7 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ACTION, NULL,
"can only have a single encap or"
" decap action in a flow");
- if (attr->ingress)
+ if (!attr->transfer && attr->ingress)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -761,7 +844,8 @@ struct field_modify_info modify_tcp[] = {
"can only have a single encap"
" action in a flow");
/* encap without preceding decap is not supported for ingress */
- if (attr->ingress && !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
+ if (!attr->transfer && attr->ingress &&
+ !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -1561,6 +1645,77 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
+/*
+ * Validate the port_id action.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action_flags
+ * Bit-fields that holds the actions detected until now.
+ * @param[in] action
+ * Port_id RTE action structure.
+ * @param[in] attr
+ * Attributes of flow that includes this action.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
+ uint64_t action_flags,
+ const struct rte_flow_action *action,
+ const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_port_id *port_id;
+ uint16_t port;
+ uint16_t esw_domain_id;
+ uint16_t act_port_domain_id;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "port id action is valid in transfer"
+ " mode only");
+ if (!action || !action->conf)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL,
+ "port id action parameters must be"
+ " specified");
+ if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
+ MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "can have only one fate actions in"
+ " a flow");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain E-Switch info");
+ port_id = action->conf;
+ port = port_id->original ? dev->data->port_id : port_id->id;
+ ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
+ if (ret)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, port_id,
+ "failed to obtain E-Switch port id for port");
+ if (act_port_domain_id != esw_domain_id)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "port does not belong to"
+ " E-Switch being configured");
+ return 0;
+}
/**
* Find existing modify-header resource or create and register a new one.
@@ -1759,11 +1914,29 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
NULL,
"priority out of range");
- if (attributes->transfer)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
- NULL,
- "transfer is not supported");
+ if (attributes->transfer) {
+ if (!priv->config.dv_esw_en)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "E-Switch dr is not supported");
+ if (!(priv->representor || priv->master))
+ return rte_flow_error_set
+ (error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "E-Switch configurationd can only be"
+ " done by a master or a representor device");
+ if (attributes->egress)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, attributes,
+ "egress is not supported");
+ if (attributes->group >= MLX5_MAX_TABLES_FDB)
+ return rte_flow_error_set
+ (error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
+ NULL, "group must be smaller than "
+ RTE_STR(MLX5_MAX_FDB_TABLES));
+ }
if (!(attributes->egress ^ attributes->ingress))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR, NULL,
@@ -1812,6 +1985,13 @@ struct field_modify_info modify_tcp[] = {
switch (items->type) {
case RTE_FLOW_ITEM_TYPE_VOID:
break;
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ ret = flow_dv_validate_item_port_id
+ (dev, items, attr, item_flags, error);
+ if (ret < 0)
+ return ret;
+ last_item |= MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
ret = mlx5_flow_validate_item_eth(items, item_flags,
error);
@@ -1943,6 +2123,17 @@ struct field_modify_info modify_tcp[] = {
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ ret = flow_dv_validate_action_port_id(dev,
+ action_flags,
+ actions,
+ attr,
+ error);
+ if (ret)
+ return ret;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ ++actions_n;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
ret = mlx5_flow_validate_action_flag(action_flags,
attr, error);
@@ -2133,10 +2324,40 @@ struct field_modify_info modify_tcp[] = {
"action not supported");
}
}
- if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "no fate action is found");
+ /* Eswitch has few restrictions on using items and actions */
+ if (attr->transfer) {
+ if (action_flags & MLX5_FLOW_ACTION_FLAG)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action FLAG");
+ if (action_flags & MLX5_FLOW_ACTION_MARK)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action MARK");
+ if (action_flags & MLX5_FLOW_ACTION_QUEUE)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action QUEUE");
+ if (action_flags & MLX5_FLOW_ACTION_RSS)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action RSS");
+ if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ } else {
+ if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ }
return 0;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (3 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:17 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
` (3 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Adds the port ID item to the DV steering code.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5_flow_dv.c | 84 ++++++++++++++++++++++++++++++-----------
1 file changed, 61 insertions(+), 23 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b819359..e3d9aa2 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3086,6 +3086,62 @@ struct field_modify_info modify_tcp[] = {
}
}
+/**
+ * Add source vport match to the specified matcher.
+ *
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] port
+ * Source vport value to match
+ * @param[in] mask
+ * Mask
+ */
+static void
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
+{
+ void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
+ void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
+
+ MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
+ MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
+}
+
+/**
+ * Translate port-id item to eswitch match on port-id.
+ *
+ * @param[in] dev
+ * The devich to configure through.
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] item
+ * Flow pattern to translate.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise.
+ */
+static int
+flow_dv_translate_item_port_id(struct rte_eth_dev *dev, void *matcher,
+ void *key, const struct rte_flow_item *item)
+{
+ const struct rte_flow_item_port_id *pid_m = item ? item->mask : NULL;
+ const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
+ uint16_t mask, val, id;
+ int ret;
+
+ mask = pid_m ? pid_m->id : 0xffff;
+ id = pid_v ? pid_v->id : dev->data->port_id;
+ ret = mlx5_port_to_eswitch_info(id, NULL, &val);
+ if (ret)
+ return ret;
+ flow_dv_translate_item_source_vport(matcher, key, val, mask);
+ return 0;
+}
+
static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
#define HEADER_IS_ZERO(match_criteria, headers) \
@@ -3296,29 +3352,6 @@ struct field_modify_info modify_tcp[] = {
}
/**
- * Add source vport match to the specified matcher.
- *
- * @param[in, out] matcher
- * Flow matcher.
- * @param[in, out] key
- * Flow matcher value.
- * @param[in] port
- * Source vport value to match
- * @param[in] mask
- * Mask
- */
-static void
-flow_dv_translate_item_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
-{
- void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
- void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
-
- MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
- MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
-}
-
-/**
* Find existing tag resource or create and register a new one.
*
* @param dev[in, out]
@@ -3724,6 +3757,11 @@ struct field_modify_info modify_tcp[] = {
void *match_value = dev_flow->dv.value.buf;
switch (items->type) {
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ flow_dv_translate_item_port_id(dev, match_mask,
+ match_value, items);
+ last_item = MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
flow_dv_translate_item_eth(match_mask, match_value,
items, tunnel);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (4 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:19 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
` (2 subsequent siblings)
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
In current implementation the DV steering supported only NIC steering.
This commit adds the transfer attribute in order to create a matcher
on the FDB tabels.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 2 ++
drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
3 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index b1effda..8afb966 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
flow->ingress = attr->ingress;
+ flow->transfer = attr->transfer;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 85954c2..9d72024 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -382,6 +383,7 @@ struct rte_flow {
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
uint8_t ingress; /**< 1 if the flow is ingress. */
uint32_t group; /**< The group index. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index e3d9aa2..182abb3 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3192,6 +3192,8 @@ struct field_modify_info modify_tcp[] = {
* Table id to use.
* @param[in] egress
* Direction of the table.
+ * @param[in] transfer
+ * E-Switch or NIC flow.
* @param[out] error
* pointer to error structure.
*
@@ -3201,6 +3203,7 @@ struct field_modify_info modify_tcp[] = {
static struct mlx5_flow_tbl_resource *
flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
uint32_t table_id, uint8_t egress,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_priv *priv = dev->data->dev_private;
@@ -3208,7 +3211,12 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow_tbl_resource *tbl;
#ifdef HAVE_MLX5DV_DR
- if (egress) {
+ if (transfer) {
+ tbl = &sh->fdb_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (sh->fdb_ns, table_id);
+ } else if (egress) {
tbl = &sh->tx_tbl[table_id];
if (!tbl->obj)
tbl->obj = mlx5_glue->dr_create_flow_tbl
@@ -3230,7 +3238,9 @@ struct field_modify_info modify_tcp[] = {
#else
(void)error;
(void)tbl;
- if (egress)
+ if (transfer)
+ return &sh->fdb_tbl[table_id];
+ else if (egress)
return &sh->tx_tbl[table_id];
else
return &sh->rx_tbl[table_id];
@@ -3295,6 +3305,7 @@ struct field_modify_info modify_tcp[] = {
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
matcher->group == cache_matcher->group &&
+ matcher->transfer == cache_matcher->transfer &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -3316,7 +3327,8 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
- matcher->egress, error);
+ matcher->egress, matcher->transfer,
+ error);
if (!tbl) {
rte_free(cache_matcher);
return rte_flow_error_set(error, ENOMEM,
@@ -3643,7 +3655,8 @@ struct field_modify_info modify_tcp[] = {
jump_data = action->conf;
tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
MLX5_GROUP_FACTOR,
- attr->egress, error);
+ attr->egress,
+ attr->transfer, error);
if (!tbl)
return rte_flow_error_set
(error, errno,
@@ -3871,6 +3884,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority);
matcher.egress = attr->egress;
matcher.group = attr->group;
+ matcher.transfer = attr->transfer;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (5 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:19 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commits adds matching on source port, using DV API.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 12 ++++
drivers/net/mlx5/mlx5_flow_dv.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/net/mlx5/mlx5_glue.c | 14 ++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 178 insertions(+)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index dd22bb6..4b6dbc2 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -281,6 +281,8 @@ struct mlx5_ibv_shared {
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ LIST_HEAD(port_id_action_list, mlx5_flow_dv_port_id_action_resource)
+ port_id_action_list; /* List of port ID actions. */
/* Shared interrupt handler section. */
pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
uint32_t intr_cnt; /* Interrupt handler reference counter. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9d72024..c419e6b 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
};
+/* Port ID resource structure. */
+struct mlx5_flow_dv_port_id_action_resource {
+ LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t port_id; /**< Port ID value. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -289,6 +299,8 @@ struct mlx5_flow_dv {
struct ibv_flow *flow; /**< Installed flow. */
struct mlx5_flow_dv_jump_tbl_resource *jump;
/**< Pointer to the jump action resource. */
+ struct mlx5_flow_dv_port_id_action_resource *port_id_action;
+ /**< Pointer to port ID action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 182abb3..9c5826c 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1054,6 +1054,70 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table port ID resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to port ID action resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_port_id_action_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_port_id_action_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_ibv_shared *sh = priv->sh;
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
+ if (resource->port_id == cache_resource->port_id) {
+ DRV_LOG(DEBUG, "port id action resource resource %p: "
+ "refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.port_id_action = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new port id action resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_vport(priv->sh->fdb_ns,
+ resource->port_id);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
+ dev_flow->dv.port_id_action = cache_resource;
+ DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -3456,6 +3520,44 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Translate port ID action to vport.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action
+ * Pointer to the port ID action.
+ * @param[out] dst_port_id
+ * The target port ID.
+ * @param[out] error
+ * Pointer to the error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_action *action,
+ uint32_t *dst_port_id,
+ struct rte_flow_error *error)
+{
+ uint32_t port;
+ uint16_t port_id;
+ int ret;
+ const struct rte_flow_action_port_id *conf =
+ (const struct rte_flow_action_port_id *)action->conf;
+
+ port = conf->original ? dev->data->port_id : conf->id;
+ ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "No eswitch info was found for port");
+ *dst_port_id = port_id;
+ return 0;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -3513,10 +3615,24 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action_jump *jump_data;
struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
struct mlx5_flow_tbl_resource *tbl;
+ uint32_t port_id = 0;
+ struct mlx5_flow_dv_port_id_action_resource port_id_resource;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ if (flow_dv_translate_action_port_id(dev, action,
+ &port_id, error))
+ return -rte_errno;
+ port_id_resource.port_id = port_id;
+ if (flow_dv_port_id_action_resource_register
+ (dev, &port_id_resource, dev_flow, error))
+ return -rte_errno;
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.port_id_action->action;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
@@ -4116,6 +4232,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release port ID action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource =
+ flow->dv.port_id_action;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "port id action resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Remove the flow from the NIC but keeps it in memory.
*
* @param[in] dev
@@ -4182,6 +4329,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_modify_hdr_resource_release(dev_flow);
if (dev_flow->dv.jump)
flow_dv_jump_tbl_resource_release(dev_flow);
+ if (dev_flow->dv.port_id_action)
+ flow_dv_port_id_action_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index a508faa..117190f 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -382,6 +382,18 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_dest_vport(ns, vport);
+#else
+ (void)ns;
+ (void)vport;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -847,6 +859,8 @@
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
.dr_create_flow_action_dest_flow_tbl =
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
+ .dr_create_flow_action_dest_vport =
+ mlx5_glue_dr_create_flow_action_dest_vport,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 058e9b1..26f5cb3 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -146,6 +146,7 @@ struct mlx5_glue {
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
+ void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (6 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:21 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Actions like encap/decap, modify header require setting the flow table
type. Until now we supported only Nic RX and Nic TX, this commits adds
the support for FDB table type for those actions.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5_flow_dv.c | 56 +++++++++++++++++++++++++++++------------
1 file changed, 40 insertions(+), 16 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 9c5826c..5997182 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -40,6 +40,10 @@
#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
#endif
+#ifndef HAVE_MLX5DV_DR_ESWITCH
+#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
+#endif
+
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -939,7 +943,9 @@ struct field_modify_info modify_tcp[] = {
struct mlx5dv_dr_ns *ns;
resource->flags = flow->group ? 0 : 1;
- if (flow->ingress)
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
ns = sh->rx_ns;
else
ns = sh->tx_ns;
@@ -1360,6 +1366,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to action structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1370,6 +1378,7 @@ struct field_modify_info modify_tcp[] = {
flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
const struct rte_flow_action *action,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
const struct rte_flow_item *encap_data;
@@ -1377,7 +1386,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow_dv_encap_decap_resource res = {
.reformat_type =
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL,
- .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
+ .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
};
if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
@@ -1412,6 +1422,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to rte_eth_dev structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1421,13 +1433,15 @@ struct field_modify_info modify_tcp[] = {
static int
flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_flow_dv_encap_decap_resource res = {
.size = 0,
.reformat_type =
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2,
- .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
+ .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
};
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
@@ -1470,8 +1484,11 @@ struct field_modify_info modify_tcp[] = {
res.reformat_type = attr->egress ?
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL :
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
- res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
- MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
+ else
+ res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
return rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -1806,11 +1823,14 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_ibv_shared *sh = priv->sh;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns;
- struct mlx5dv_dr_ns *ns =
- resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
- sh->tx_ns : sh->rx_ns;
-
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
+ ns = sh->tx_ns;
+ else
+ ns = sh->rx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -3604,6 +3624,8 @@ struct field_modify_info modify_tcp[] = {
union flow_dv_attr flow_attr = { .attr = 0 };
struct mlx5_flow_dv_tag_resource tag_resource;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
for (; !actions_end ; actions++) {
@@ -3709,7 +3731,9 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow, error))
+ dev_flow,
+ attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3721,6 +3745,7 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
if (flow_dv_create_action_l2_decap(dev, dev_flow,
+ attr->transfer,
error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
@@ -3740,9 +3765,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
- if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_encap
+ (dev, actions, dev_flow, attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3757,9 +3782,8 @@ struct field_modify_info modify_tcp[] = {
}
/* Handle decap only if it isn't followed by encap. */
if (action->type != RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
- if (flow_dv_create_action_l2_decap(dev,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_decap
+ (dev, dev_flow, attr->transfer, error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (7 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type Ori Kam
@ 2019-04-18 11:28 ` Ori Kam
2019-04-18 12:28 ` Yongseok Koh
8 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 11:28 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit adds support for drop action when creating E-Switch flow
using DV.
Signed-off-by: Ori Kam <orika@mellanox.com>
---
v2:
* Address ML comments.
---
drivers/net/mlx5/mlx5.c | 9 +++++++++
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_flow_dv.c | 23 ++++++++++++++++-------
drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 39 insertions(+), 7 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index ff24e1d..65c69af 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
goto error;
}
sh->fdb_ns = ns;
+ sh->esw_drop_action = mlx5_glue->dr_create_flow_action_drop();
}
#endif
sh->dv_refcnt++;
@@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->esw_drop_action) {
+ mlx5_glue->destroy_flow_action(sh->esw_drop_action);
+ sh->esw_drop_action = NULL;
+ }
return err;
#else
(void)priv;
@@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->esw_drop_action) {
+ mlx5_glue->destroy_flow_action(sh->esw_drop_action);
+ sh->esw_drop_action = NULL;
+ }
#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 4b6dbc2..a736d57 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -275,6 +275,7 @@ struct mlx5_ibv_shared {
/* RX Direct Rules tables. */
void *tx_ns; /* TX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
/* TX Direct Rules tables/ */
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 5997182..5ec8caf 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_flow_dv *dv;
struct mlx5_flow *dev_flow;
+ struct mlx5_priv *priv = dev->data->dev_private;
int n;
int err;
@@ -4056,13 +4057,21 @@ struct field_modify_info modify_tcp[] = {
dv = &dev_flow->dv;
n = dv->actions_n;
if (flow->actions & MLX5_FLOW_ACTION_DROP) {
- dv->hrxq = mlx5_hrxq_drop_new(dev);
- if (!dv->hrxq) {
- rte_flow_error_set
- (error, errno,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
- "cannot get drop hash queue");
- goto error;
+ if (flow->transfer)
+ dv->actions[n++] = priv->sh->esw_drop_action;
+ else {
+ dv->hrxq = mlx5_hrxq_drop_new(dev);
+ if (!dv->hrxq) {
+ rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "cannot get drop hash queue");
+ goto error;
+ }
+ dv->actions[n++] =
+ mlx5_glue->dv_create_flow_action_dest_ibv_qp
+ (dv->hrxq->qp);
}
dv->actions[n++] = dv->hrxq->action;
} else if (flow->actions &
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 117190f..b32cd09c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -394,6 +394,16 @@
}
static void *
+mlx5_glue_dr_create_flow_action_drop(void)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_drop();
+#else
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -861,6 +871,8 @@
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_action_dest_vport =
mlx5_glue_dr_create_flow_action_dest_vport,
+ .dr_create_flow_action_drop =
+ mlx5_glue_dr_create_flow_action_drop,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 26f5cb3..1d06583 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -147,6 +147,7 @@ struct mlx5_glue {
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
+ void *(*dr_create_flow_action_drop)();
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name Ori Kam
@ 2019-04-18 12:06 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:06 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:42AM +0000, Ori Kam wrote:
> Modify the translate vport function to match other translate items
> naming convestions.
>
> Fixes: 0fe3f18f78d8 ("net/mlx5: add source vport match to the ingress rules")
> Cc: viacheslavo@mellanox.com
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
You were supposed to change the name to flow_dv_translate_item_src_vport, but
that's fine.
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> ---
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 04ab3d6..1e25e0b 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3087,8 +3087,8 @@ struct field_modify_info modify_tcp[] = {
> * Mask
> */
> static void
> -flow_dv_translate_source_vport(void *matcher, void *key,
> - int16_t port, uint16_t mask)
> +flow_dv_translate_item_source_vport(void *matcher, void *key,
> + int16_t port, uint16_t mask)
> {
> void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> @@ -3492,10 +3492,10 @@ struct field_modify_info modify_tcp[] = {
> * Add matching on source vport index only
> * for ingress rules in E-Switch configurations.
> */
> - flow_dv_translate_source_vport(matcher.mask.buf,
> - dev_flow->dv.value.buf,
> - priv->vport_id,
> - 0xffff);
> + flow_dv_translate_item_source_vport(matcher.mask.buf,
> + dev_flow->dv.value.buf,
> + priv->vport_id,
> + 0xffff);
> }
> for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) {
> int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
@ 2019-04-18 12:09 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:09 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:43AM +0000, Ori Kam wrote:
> The meson build was missing the define for Direct Rules.
>
> Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
> Cc: orika@mellanox.com
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> ---
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/meson.build | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> index a4c684e..0037e15 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -111,6 +111,8 @@ if build
> 'mlx5dv_devx_obj_create' ],
> [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> + 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> 'SUPPORTED_40000baseKR4_Full' ],
> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
@ 2019-04-18 12:11 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:11 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:44AM +0000, Ori Kam wrote:
> This commit checks the for DR E-Switch support.
> The support is based on both Device and Kernel.
> This commit also enables the user to manualy disable this this feature.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Only one minor nit below.
With the fix,
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/Makefile | 5 +
> drivers/net/mlx5/meson.build | 2 +
> drivers/net/mlx5/mlx5.c | 53 +++++-
> drivers/net/mlx5/mlx5.h | 12 ++
> drivers/net/mlx5/mlx5_devx_cmds.c | 44 +++++
> drivers/net/mlx5/mlx5_flow.c | 2 +-
> drivers/net/mlx5/mlx5_prm.h | 328 ++++++++++++++++++++++++++++++++++++++
> 7 files changed, 440 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
> index 93bc869..2b72a33 100644
> --- a/drivers/net/mlx5/Makefile
> +++ b/drivers/net/mlx5/Makefile
> @@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
> enum MLX5DV_DR_NS_TYPE_TERMINATING \
> $(AUTOCONF_OUTPUT)
> $Q sh -- '$<' '$@' \
> + HAVE_MLX5DV_DR_ESWITCH \
> + infiniband/mlx5dv.h \
> + enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
> + $(AUTOCONF_OUTPUT)
> + $Q sh -- '$<' '$@' \
> HAVE_IBV_DEVX_OBJ \
> infiniband/mlx5dv.h \
> func mlx5dv_devx_obj_create \
> diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
> index 0037e15..9dfd28d 100644
> --- a/drivers/net/mlx5/meson.build
> +++ b/drivers/net/mlx5/meson.build
> @@ -113,6 +113,8 @@ if build
> 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
> [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
> 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
> + [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
> + 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
> [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
> 'SUPPORTED_40000baseKR4_Full' ],
> [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index 9ff50df..ff24e1d 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -101,6 +101,9 @@
> /* Allow L3 VXLAN flow creation. */
> #define MLX5_L3_VXLAN_EN "l3_vxlan_en"
>
> +/* Activate DV E-Switch flow steering. */
> +#define MLX5_DV_ESW_EN "dv_esw_en"
> +
> /* Activate DV flow steering. */
> #define MLX5_DV_FLOW_EN "dv_flow_en"
>
> @@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
> }
> pthread_mutex_init(&sh->dv_mutex, NULL);
> sh->tx_ns = ns;
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (priv->config.dv_esw_en) {
> + ns = mlx5_glue->dr_create_ns(sh->ctx,
> + MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
> + if (!ns) {
> + DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
> + err = errno;
> + goto error;
> + }
> + sh->fdb_ns = ns;
> + }
> +#endif
> sh->dv_refcnt++;
> priv->dr_shared = 1;
> return 0;
> @@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
> mlx5dv_dr_destroy_ns(sh->tx_ns);
> sh->tx_ns = NULL;
> }
> + if (sh->fdb_ns) {
> + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> + sh->fdb_ns = NULL;
> + }
> return err;
> #else
> (void)priv;
> @@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
> mlx5dv_dr_destroy_ns(sh->tx_ns);
> sh->tx_ns = NULL;
> }
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (sh->fdb_ns) {
> + mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> + sh->fdb_ns = NULL;
> + }
> +#endif
> pthread_mutex_destroy(&sh->dv_mutex);
> #else
> (void)priv;
> @@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
> config->l3_vxlan_en = !!tmp;
> } else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
> config->vf_nl_en = !!tmp;
> + } else if (strcmp(MLX5_DV_ESW_EN, key) == 0) {
> + config->dv_esw_en = !!tmp;
> } else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
> config->dv_flow_en = !!tmp;
> } else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
> @@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
> MLX5_RX_VEC_EN,
> MLX5_L3_VXLAN_EN,
> MLX5_VF_NL_EN,
> + MLX5_DV_ESW_EN,
> MLX5_DV_FLOW_EN,
> MLX5_MR_EXT_MEMSEG_EN,
> MLX5_REPRESENTOR,
> @@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
> priv->tcf_context = NULL;
> }
> }
> - if (config.dv_flow_en) {
> - err = mlx5_alloc_shared_dr(priv);
> - if (err)
> - goto error;
> - }
> TAILQ_INIT(&priv->flows);
> TAILQ_INIT(&priv->ctrl_flows);
> /* Hint libmlx5 to use PMD allocator for data plane resources */
> @@ -1484,8 +1507,27 @@ struct mlx5_dev_spawn_data {
> * Verbs context returned by ibv_open_device().
> */
> mlx5_link_update(eth_dev, 0);
> +#ifdef HAVE_IBV_DEVX_OBJ
> + err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
> + if (err) {
> + err = -err;
> + goto error;
> + }
> +#endif
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + if (!(config.hca_attr.eswitch_manager && config.dv_flow_en &&
> + (switch_info->representor || switch_info->master)))
> + config.dv_esw_en = 0;
> +#else
> + config.dv_esw_en = 0;
> +#endif
> /* Store device configuration on private structure. */
> priv->config = config;
> + if (config.dv_flow_en) {
> + err = mlx5_alloc_shared_dr(priv);
> + if (err)
> + goto error;
> + }
> /* Supported Verbs flow priority number detection. */
> err = mlx5_flow_discover_priorities(eth_dev);
> if (err < 0) {
> @@ -1876,6 +1918,7 @@ struct mlx5_dev_spawn_data {
> .max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
> .min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
> },
> + .dv_esw_en = 1,
> };
> /* Device specific configuration. */
> switch (pci_dev->id.device_id) {
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 14c7f3c..b9946f6 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
> int id; /* Flow counter ID */
> };
>
> +/* HCA attributes. */
> +struct mlx5_hca_attr {
> + uint32_t eswitch_manager:1;
> +};
> +
> /* Flow list . */
> TAILQ_HEAD(mlx5_flows, rte_flow);
>
> @@ -171,6 +176,7 @@ struct mlx5_dev_config {
> /* Whether memseg should be extended for MR creation. */
> unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
> unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
> + unsigned int dv_esw_en:1; /* Enable E-Switch DV flow. */
> unsigned int dv_flow_en:1; /* Enable DV flow. */
> unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
> unsigned int devx:1; /* Whether devx interface is available or not. */
> @@ -192,6 +198,7 @@ struct mlx5_dev_config {
> int txqs_inline; /* Queue number threshold for inlining. */
> int txqs_vec; /* Queue number threshold for vectorized Tx. */
> int inline_max_packet_sz; /* Max packet size for inlining. */
> + struct mlx5_hca_attr hca_attr; /* HCA attributes. */
> };
>
> /**
> @@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
> };
>
> #define MLX5_MAX_TABLES 1024
> +#define MLX5_MAX_TABLES_FDB 32
> #define MLX5_GROUP_FACTOR 1
>
> /*
> @@ -260,6 +268,8 @@ struct mlx5_ibv_shared {
> /* Shared DV/DR flow data section. */
> pthread_mutex_t dv_mutex; /* DV context mutex. */
> uint32_t dv_refcnt; /* DV/DR data reference counter. */
> + void *fdb_ns; /* FDB Direct Rules name space handle. */
> + struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
Please add a comment for fdb_tbl.
> void *rx_ns; /* RX Direct Rules name space handle. */
> struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
> /* RX Direct Rules tables. */
> @@ -539,4 +549,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct ibv_context *ctx,
> int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set *dcx,
> int clear,
> uint64_t *pkts, uint64_t *bytes);
> +int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> + struct mlx5_hca_attr *attr);
> #endif /* RTE_PMD_MLX5_H_ */
> diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c b/drivers/net/mlx5/mlx5_devx_cmds.c
> index a9dff58..e5776c4 100644
> --- a/drivers/net/mlx5/mlx5_devx_cmds.c
> +++ b/drivers/net/mlx5/mlx5_devx_cmds.c
> @@ -105,3 +105,47 @@ int mlx5_devx_cmd_flow_counter_free(struct mlx5dv_devx_obj *obj)
> *bytes = MLX5_GET64(traffic_counter, stats, octets);
> return 0;
> }
> +
> +/**
> + * Query HCA attributes.
> + * Using those attributes we can check on run time if the device
> + * is having the required capabilities.
> + *
> + * @param[in] ctx
> + * ibv contexts returned from mlx5dv_open_device.
> + * @param[out] attr
> + * Attributes device values.
> + *
> + * @return
> + * 0 on success, a negative value otherwise.
> + */
> +int
> +mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
> + struct mlx5_hca_attr *attr)
> +{
> + uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
> + uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
> + void *hcattr;
> + int status, syndrome, rc;
> +
> + MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
> + MLX5_SET(query_hca_cap_in, in, op_mod,
> + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
> + MLX5_HCA_CAP_OPMOD_GET_CUR);
> +
> + rc = mlx5_glue->devx_general_cmd(ctx,
> + in, sizeof(in), out, sizeof(out));
> + if (rc)
> + return rc;
> + status = MLX5_GET(query_hca_cap_out, out, status);
> + syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
> + if (status) {
> + DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
> + "status %x, syndrome = %x",
> + status, syndrome);
> + return -1;
> + }
> + hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
> + attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);
> + return 0;
> +}
> diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> index a0683ee..b1effda 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
> struct mlx5_priv *priv = dev->data->dev_private;
> enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
>
> - if (attr->transfer)
> + if (attr->transfer && !priv->config.dv_esw_en)
> type = MLX5_FLOW_TYPE_TCF;
> else
> type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
> diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
> index b15266f..8c42380 100644
> --- a/drivers/net/mlx5/mlx5_prm.h
> +++ b/drivers/net/mlx5/mlx5_prm.h
> @@ -529,6 +529,7 @@ enum {
> };
>
> enum {
> + MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
> MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
> MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
> };
> @@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
> u8 flow_counter_id[0x20];
> };
>
> +enum {
> + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
> + MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
> +};
> +
> +enum {
> + MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
> + MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
> +};
> +
> +struct mlx5_ifc_cmd_hca_cap_bits {
> + u8 reserved_at_0[0x30];
> + u8 vhca_id[0x10];
> + u8 reserved_at_40[0x40];
> + u8 log_max_srq_sz[0x8];
> + u8 log_max_qp_sz[0x8];
> + u8 reserved_at_90[0xb];
> + u8 log_max_qp[0x5];
> + u8 reserved_at_a0[0xb];
> + u8 log_max_srq[0x5];
> + u8 reserved_at_b0[0x10];
> + u8 reserved_at_c0[0x8];
> + u8 log_max_cq_sz[0x8];
> + u8 reserved_at_d0[0xb];
> + u8 log_max_cq[0x5];
> + u8 log_max_eq_sz[0x8];
> + u8 reserved_at_e8[0x2];
> + u8 log_max_mkey[0x6];
> + u8 reserved_at_f0[0x8];
> + u8 dump_fill_mkey[0x1];
> + u8 reserved_at_f9[0x3];
> + u8 log_max_eq[0x4];
> + u8 max_indirection[0x8];
> + u8 fixed_buffer_size[0x1];
> + u8 log_max_mrw_sz[0x7];
> + u8 force_teardown[0x1];
> + u8 reserved_at_111[0x1];
> + u8 log_max_bsf_list_size[0x6];
> + u8 umr_extended_translation_offset[0x1];
> + u8 null_mkey[0x1];
> + u8 log_max_klm_list_size[0x6];
> + u8 reserved_at_120[0xa];
> + u8 log_max_ra_req_dc[0x6];
> + u8 reserved_at_130[0xa];
> + u8 log_max_ra_res_dc[0x6];
> + u8 reserved_at_140[0xa];
> + u8 log_max_ra_req_qp[0x6];
> + u8 reserved_at_150[0xa];
> + u8 log_max_ra_res_qp[0x6];
> + u8 end_pad[0x1];
> + u8 cc_query_allowed[0x1];
> + u8 cc_modify_allowed[0x1];
> + u8 start_pad[0x1];
> + u8 cache_line_128byte[0x1];
> + u8 reserved_at_165[0xa];
> + u8 qcam_reg[0x1];
> + u8 gid_table_size[0x10];
> + u8 out_of_seq_cnt[0x1];
> + u8 vport_counters[0x1];
> + u8 retransmission_q_counters[0x1];
> + u8 debug[0x1];
> + u8 modify_rq_counter_set_id[0x1];
> + u8 rq_delay_drop[0x1];
> + u8 max_qp_cnt[0xa];
> + u8 pkey_table_size[0x10];
> + u8 vport_group_manager[0x1];
> + u8 vhca_group_manager[0x1];
> + u8 ib_virt[0x1];
> + u8 eth_virt[0x1];
> + u8 vnic_env_queue_counters[0x1];
> + u8 ets[0x1];
> + u8 nic_flow_table[0x1];
> + u8 eswitch_manager[0x1];
> + u8 device_memory[0x1];
> + u8 mcam_reg[0x1];
> + u8 pcam_reg[0x1];
> + u8 local_ca_ack_delay[0x5];
> + u8 port_module_event[0x1];
> + u8 enhanced_error_q_counters[0x1];
> + u8 ports_check[0x1];
> + u8 reserved_at_1b3[0x1];
> + u8 disable_link_up[0x1];
> + u8 beacon_led[0x1];
> + u8 port_type[0x2];
> + u8 num_ports[0x8];
> + u8 reserved_at_1c0[0x1];
> + u8 pps[0x1];
> + u8 pps_modify[0x1];
> + u8 log_max_msg[0x5];
> + u8 reserved_at_1c8[0x4];
> + u8 max_tc[0x4];
> + u8 temp_warn_event[0x1];
> + u8 dcbx[0x1];
> + u8 general_notification_event[0x1];
> + u8 reserved_at_1d3[0x2];
> + u8 fpga[0x1];
> + u8 rol_s[0x1];
> + u8 rol_g[0x1];
> + u8 reserved_at_1d8[0x1];
> + u8 wol_s[0x1];
> + u8 wol_g[0x1];
> + u8 wol_a[0x1];
> + u8 wol_b[0x1];
> + u8 wol_m[0x1];
> + u8 wol_u[0x1];
> + u8 wol_p[0x1];
> + u8 stat_rate_support[0x10];
> + u8 reserved_at_1f0[0xc];
> + u8 cqe_version[0x4];
> + u8 compact_address_vector[0x1];
> + u8 striding_rq[0x1];
> + u8 reserved_at_202[0x1];
> + u8 ipoib_enhanced_offloads[0x1];
> + u8 ipoib_basic_offloads[0x1];
> + u8 reserved_at_205[0x1];
> + u8 repeated_block_disabled[0x1];
> + u8 umr_modify_entity_size_disabled[0x1];
> + u8 umr_modify_atomic_disabled[0x1];
> + u8 umr_indirect_mkey_disabled[0x1];
> + u8 umr_fence[0x2];
> + u8 reserved_at_20c[0x3];
> + u8 drain_sigerr[0x1];
> + u8 cmdif_checksum[0x2];
> + u8 sigerr_cqe[0x1];
> + u8 reserved_at_213[0x1];
> + u8 wq_signature[0x1];
> + u8 sctr_data_cqe[0x1];
> + u8 reserved_at_216[0x1];
> + u8 sho[0x1];
> + u8 tph[0x1];
> + u8 rf[0x1];
> + u8 dct[0x1];
> + u8 qos[0x1];
> + u8 eth_net_offloads[0x1];
> + u8 roce[0x1];
> + u8 atomic[0x1];
> + u8 reserved_at_21f[0x1];
> + u8 cq_oi[0x1];
> + u8 cq_resize[0x1];
> + u8 cq_moderation[0x1];
> + u8 reserved_at_223[0x3];
> + u8 cq_eq_remap[0x1];
> + u8 pg[0x1];
> + u8 block_lb_mc[0x1];
> + u8 reserved_at_229[0x1];
> + u8 scqe_break_moderation[0x1];
> + u8 cq_period_start_from_cqe[0x1];
> + u8 cd[0x1];
> + u8 reserved_at_22d[0x1];
> + u8 apm[0x1];
> + u8 vector_calc[0x1];
> + u8 umr_ptr_rlky[0x1];
> + u8 imaicl[0x1];
> + u8 reserved_at_232[0x4];
> + u8 qkv[0x1];
> + u8 pkv[0x1];
> + u8 set_deth_sqpn[0x1];
> + u8 reserved_at_239[0x3];
> + u8 xrc[0x1];
> + u8 ud[0x1];
> + u8 uc[0x1];
> + u8 rc[0x1];
> + u8 uar_4k[0x1];
> + u8 reserved_at_241[0x9];
> + u8 uar_sz[0x6];
> + u8 reserved_at_250[0x8];
> + u8 log_pg_sz[0x8];
> + u8 bf[0x1];
> + u8 driver_version[0x1];
> + u8 pad_tx_eth_packet[0x1];
> + u8 reserved_at_263[0x8];
> + u8 log_bf_reg_size[0x5];
> + u8 reserved_at_270[0xb];
> + u8 lag_master[0x1];
> + u8 num_lag_ports[0x4];
> + u8 reserved_at_280[0x10];
> + u8 max_wqe_sz_sq[0x10];
> + u8 reserved_at_2a0[0x10];
> + u8 max_wqe_sz_rq[0x10];
> + u8 max_flow_counter_31_16[0x10];
> + u8 max_wqe_sz_sq_dc[0x10];
> + u8 reserved_at_2e0[0x7];
> + u8 max_qp_mcg[0x19];
> + u8 reserved_at_300[0x10];
> + u8 flow_counter_bulk_alloc[0x08];
> + u8 log_max_mcg[0x8];
> + u8 reserved_at_320[0x3];
> + u8 log_max_transport_domain[0x5];
> + u8 reserved_at_328[0x3];
> + u8 log_max_pd[0x5];
> + u8 reserved_at_330[0xb];
> + u8 log_max_xrcd[0x5];
> + u8 nic_receive_steering_discard[0x1];
> + u8 receive_discard_vport_down[0x1];
> + u8 transmit_discard_vport_down[0x1];
> + u8 reserved_at_343[0x5];
> + u8 log_max_flow_counter_bulk[0x8];
> + u8 max_flow_counter_15_0[0x10];
> + u8 reserved_at_360[0x3];
> + u8 log_max_rq[0x5];
> + u8 reserved_at_368[0x3];
> + u8 log_max_sq[0x5];
> + u8 reserved_at_370[0x3];
> + u8 log_max_tir[0x5];
> + u8 reserved_at_378[0x3];
> + u8 log_max_tis[0x5];
> + u8 basic_cyclic_rcv_wqe[0x1];
> + u8 reserved_at_381[0x2];
> + u8 log_max_rmp[0x5];
> + u8 reserved_at_388[0x3];
> + u8 log_max_rqt[0x5];
> + u8 reserved_at_390[0x3];
> + u8 log_max_rqt_size[0x5];
> + u8 reserved_at_398[0x3];
> + u8 log_max_tis_per_sq[0x5];
> + u8 ext_stride_num_range[0x1];
> + u8 reserved_at_3a1[0x2];
> + u8 log_max_stride_sz_rq[0x5];
> + u8 reserved_at_3a8[0x3];
> + u8 log_min_stride_sz_rq[0x5];
> + u8 reserved_at_3b0[0x3];
> + u8 log_max_stride_sz_sq[0x5];
> + u8 reserved_at_3b8[0x3];
> + u8 log_min_stride_sz_sq[0x5];
> + u8 hairpin[0x1];
> + u8 reserved_at_3c1[0x2];
> + u8 log_max_hairpin_queues[0x5];
> + u8 reserved_at_3c8[0x3];
> + u8 log_max_hairpin_wq_data_sz[0x5];
> + u8 reserved_at_3d0[0x3];
> + u8 log_max_hairpin_num_packets[0x5];
> + u8 reserved_at_3d8[0x3];
> + u8 log_max_wq_sz[0x5];
> + u8 nic_vport_change_event[0x1];
> + u8 disable_local_lb_uc[0x1];
> + u8 disable_local_lb_mc[0x1];
> + u8 log_min_hairpin_wq_data_sz[0x5];
> + u8 reserved_at_3e8[0x3];
> + u8 log_max_vlan_list[0x5];
> + u8 reserved_at_3f0[0x3];
> + u8 log_max_current_mc_list[0x5];
> + u8 reserved_at_3f8[0x3];
> + u8 log_max_current_uc_list[0x5];
> + u8 general_obj_types[0x40];
> + u8 reserved_at_440[0x20];
> + u8 reserved_at_460[0x10];
> + u8 max_num_eqs[0x10];
> + u8 reserved_at_480[0x3];
> + u8 log_max_l2_table[0x5];
> + u8 reserved_at_488[0x8];
> + u8 log_uar_page_sz[0x10];
> + u8 reserved_at_4a0[0x20];
> + u8 device_frequency_mhz[0x20];
> + u8 device_frequency_khz[0x20];
> + u8 reserved_at_500[0x20];
> + u8 num_of_uars_per_page[0x20];
> + u8 flex_parser_protocols[0x20];
> + u8 reserved_at_560[0x20];
> + u8 reserved_at_580[0x3c];
> + u8 mini_cqe_resp_stride_index[0x1];
> + u8 cqe_128_always[0x1];
> + u8 cqe_compression_128[0x1];
> + u8 cqe_compression[0x1];
> + u8 cqe_compression_timeout[0x10];
> + u8 cqe_compression_max_num[0x10];
> + u8 reserved_at_5e0[0x10];
> + u8 tag_matching[0x1];
> + u8 rndv_offload_rc[0x1];
> + u8 rndv_offload_dc[0x1];
> + u8 log_tag_matching_list_sz[0x5];
> + u8 reserved_at_5f8[0x3];
> + u8 log_max_xrq[0x5];
> + u8 affiliate_nic_vport_criteria[0x8];
> + u8 native_port_num[0x8];
> + u8 num_vhca_ports[0x8];
> + u8 reserved_at_618[0x6];
> + u8 sw_owner_id[0x1];
> + u8 reserved_at_61f[0x1e1];
> +};
> +
> +struct mlx5_ifc_qos_cap_bits {
> + u8 packet_pacing[0x1];
> + u8 esw_scheduling[0x1];
> + u8 esw_bw_share[0x1];
> + u8 esw_rate_limit[0x1];
> + u8 reserved_at_4[0x1];
> + u8 packet_pacing_burst_bound[0x1];
> + u8 packet_pacing_typical_size[0x1];
> + u8 flow_meter_srtcm[0x1];
> + u8 reserved_at_8[0x8];
> + u8 log_max_flow_meter[0x8];
> + u8 flow_meter_reg_id[0x8];
> + u8 reserved_at_25[0x20];
> + u8 packet_pacing_max_rate[0x20];
> + u8 packet_pacing_min_rate[0x20];
> + u8 reserved_at_80[0x10];
> + u8 packet_pacing_rate_table_size[0x10];
> + u8 esw_element_type[0x10];
> + u8 esw_tsar_type[0x10];
> + u8 reserved_at_c0[0x10];
> + u8 max_qos_para_vport[0x10];
> + u8 max_tsar_bw_share[0x20];
> + u8 reserved_at_100[0x6e8];
> +};
> +
> +union mlx5_ifc_hca_cap_union_bits {
> + struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
> + struct mlx5_ifc_qos_cap_bits qos_cap;
> + u8 reserved_at_0[0x8000];
> +};
> +
> +struct mlx5_ifc_query_hca_cap_out_bits {
> + u8 status[0x8];
> + u8 reserved_at_8[0x18];
> + u8 syndrome[0x20];
> + u8 reserved_at_40[0x40];
> + union mlx5_ifc_hca_cap_union_bits capability;
> +};
> +
> +struct mlx5_ifc_query_hca_cap_in_bits {
> + u8 opcode[0x10];
> + u8 reserved_at_10[0x10];
> + u8 reserved_at_20[0x10];
> + u8 op_mod[0x10];
> + u8 reserved_at_40[0x40];
> +};
> +
> /* CQE format mask. */
> #define MLX5E_CQE_FORMAT_MASK 0xc
>
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
@ 2019-04-18 12:16 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:16 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:45AM +0000, Ori Kam wrote:
> Add validation logic for E-Switch using Direct Rules.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_ethdev.c | 41 +++++++
> drivers/net/mlx5/mlx5_flow.h | 5 +
> drivers/net/mlx5/mlx5_flow_dv.c | 243 ++++++++++++++++++++++++++++++++++++++--
> 4 files changed, 280 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index b9946f6..dd22bb6 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -412,6 +412,8 @@ int mlx5_ibv_device_to_pci_addr(const struct ibv_device *device,
> unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
> uint16_t *port_list,
> unsigned int port_list_n);
> +int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
> + uint16_t *es_port_id);
> int mlx5_sysfs_switch_info(unsigned int ifindex,
> struct mlx5_switch_info *info);
> bool mlx5_translate_port_name(const char *port_name_in,
> diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
> index 3992918..695440a 100644
> --- a/drivers/net/mlx5/mlx5_ethdev.c
> +++ b/drivers/net/mlx5/mlx5_ethdev.c
> @@ -1376,6 +1376,47 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size)
> }
>
> /**
> + * Get the E-Switch domain id this port belongs to.
> + *
> + * @param[in] port
> + * Device port id.
> + * @param[out] es_domain_id
> + * E-Switch domain id.
> + * @param[out] es_port_id
> + * The port id of the port in the E-Switch.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +int
> +mlx5_port_to_eswitch_info(uint16_t port,
> + uint16_t *es_domain_id, uint16_t *es_port_id)
> +{
> + struct rte_eth_dev *dev;
> + struct mlx5_priv *priv;
> +
> + if (port >= RTE_MAX_ETHPORTS) {
> + rte_errno = EINVAL;
> + return -rte_errno;
> + }
> + if (!rte_eth_dev_is_valid_port(port)) {
> + rte_errno = ENODEV;
> + return -rte_errno;
> + }
> + dev = &rte_eth_devices[port];
> + priv = dev->data->dev_private;
> + if (!(priv->representor || priv->master)) {
> + rte_errno = EINVAL;
> + return -rte_errno;
> + }
> + if (es_domain_id)
> + *es_domain_id = priv->domain_id;
> + if (es_port_id)
> + *es_port_id = priv->vport_id;
> + return 0;
> +}
> +
> +/**
> * Get switch information associated with network interface.
> *
> * @param ifindex
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 9f47fd4..85954c2 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -48,6 +48,7 @@
>
> /* General pattern items bits. */
> #define MLX5_FLOW_ITEM_METADATA (1u << 16)
> +#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
>
> /* Outer Masks. */
> #define MLX5_FLOW_LAYER_OUTER_L3 \
> @@ -118,6 +119,10 @@
> (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
> MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
>
> +#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
> + (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
> + MLX5_FLOW_ACTION_JUMP)
> +
> #define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
> MLX5_FLOW_ACTION_NVGRE_ENCAP | \
> MLX5_FLOW_ACTION_RAW_ENCAP)
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 1e25e0b..b819359 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -613,6 +613,89 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Validate vport item.
> + *
> + * @param[in] dev
> + * Pointer to the rte_eth_dev structure.
> + * @param[in] item
> + * Item specification.
> + * @param[in] attr
> + * Attributes of flow that includes this item.
> + * @param[in] item_flags
> + * Bit-fields that holds the items detected until now.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
> + const struct rte_flow_item *item,
> + const struct rte_flow_attr *attr,
> + uint64_t item_flags,
> + struct rte_flow_error *error)
> +{
> + const struct rte_flow_item_port_id *spec = item->spec;
> + const struct rte_flow_item_port_id *mask = item->mask;
> + const struct rte_flow_item_port_id switch_mask = {
> + .id = 0xffffffff,
> + };
> + uint16_t esw_domain_id;
> + uint16_t item_port_esw_domain_id;
> + int ret;
> +
> + if (!attr->transfer)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + NULL,
> + "match on port id is valid only"
> + " when transfer flag is enabled");
> + if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ITEM, item,
> + "multiple source ports are not"
> + " supported");
> + if (!mask)
> + mask = &switch_mask;
> + if (mask->id != 0xffffffff)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ITEM_MASK,
> + mask,
> + "no support for partial mask on"
> + " \"id\" field");
> + ret = mlx5_flow_item_acceptable
> + (item, (const uint8_t *)mask,
> + (const uint8_t *)&rte_flow_item_port_id_mask,
> + sizeof(struct rte_flow_item_port_id),
> + error);
> + if (ret)
> + return ret;
> + if (!spec)
> + return 0;
> + ret = mlx5_port_to_eswitch_info(spec->id, &item_port_esw_domain_id,
> + NULL);
> + if (ret)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> + "failed to obtain E-Switch info for"
> + " port");
> + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> + &esw_domain_id, NULL);
> + if (ret < 0)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "failed to obtain E-Switch info");
> + if (item_port_esw_domain_id != esw_domain_id)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
> + "cannot match on a port from a"
> + " different E-Switch");
> + return 0;
> +}
> +
> +/**
> * Validate count action.
> *
> * @param[in] dev
> @@ -676,7 +759,7 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> "can only have a single encap or"
> " decap action in a flow");
> - if (attr->ingress)
> + if (!attr->transfer && attr->ingress)
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> NULL,
> @@ -761,7 +844,8 @@ struct field_modify_info modify_tcp[] = {
> "can only have a single encap"
> " action in a flow");
> /* encap without preceding decap is not supported for ingress */
> - if (attr->ingress && !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
> + if (!attr->transfer && attr->ingress &&
> + !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
> NULL,
> @@ -1561,6 +1645,77 @@ struct field_modify_info modify_tcp[] = {
> return 0;
> }
>
> +/*
> + * Validate the port_id action.
> + *
> + * @param[in] dev
> + * Pointer to rte_eth_dev structure.
> + * @param[in] action_flags
> + * Bit-fields that holds the actions detected until now.
> + * @param[in] action
> + * Port_id RTE action structure.
> + * @param[in] attr
> + * Attributes of flow that includes this action.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
> + uint64_t action_flags,
> + const struct rte_flow_action *action,
> + const struct rte_flow_attr *attr,
> + struct rte_flow_error *error)
> +{
> + const struct rte_flow_action_port_id *port_id;
> + uint16_t port;
> + uint16_t esw_domain_id;
> + uint16_t act_port_domain_id;
> + int ret;
> +
> + if (!attr->transfer)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "port id action is valid in transfer"
> + " mode only");
> + if (!action || !action->conf)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION_CONF,
> + NULL,
> + "port id action parameters must be"
> + " specified");
> + if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
> + MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> + "can have only one fate actions in"
> + " a flow");
> + ret = mlx5_port_to_eswitch_info(dev->data->port_id,
> + &esw_domain_id, NULL);
> + if (ret < 0)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "failed to obtain E-Switch info");
> + port_id = action->conf;
> + port = port_id->original ? dev->data->port_id : port_id->id;
> + ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
> + if (ret)
> + return rte_flow_error_set
> + (error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION_CONF, port_id,
> + "failed to obtain E-Switch port id for port");
> + if (act_port_domain_id != esw_domain_id)
> + return rte_flow_error_set
> + (error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION, NULL,
> + "port does not belong to"
> + " E-Switch being configured");
> + return 0;
> +}
>
> /**
> * Find existing modify-header resource or create and register a new one.
> @@ -1759,11 +1914,29 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
> NULL,
> "priority out of range");
> - if (attributes->transfer)
> - return rte_flow_error_set(error, ENOTSUP,
> - RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> - NULL,
> - "transfer is not supported");
> + if (attributes->transfer) {
> + if (!priv->config.dv_esw_en)
> + return rte_flow_error_set
> + (error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "E-Switch dr is not supported");
> + if (!(priv->representor || priv->master))
> + return rte_flow_error_set
> + (error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "E-Switch configurationd can only be"
> + " done by a master or a representor device");
> + if (attributes->egress)
> + return rte_flow_error_set
> + (error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, attributes,
> + "egress is not supported");
> + if (attributes->group >= MLX5_MAX_TABLES_FDB)
> + return rte_flow_error_set
> + (error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
> + NULL, "group must be smaller than "
> + RTE_STR(MLX5_MAX_FDB_TABLES));
> + }
> if (!(attributes->egress ^ attributes->ingress))
> return rte_flow_error_set(error, ENOTSUP,
> RTE_FLOW_ERROR_TYPE_ATTR, NULL,
> @@ -1812,6 +1985,13 @@ struct field_modify_info modify_tcp[] = {
> switch (items->type) {
> case RTE_FLOW_ITEM_TYPE_VOID:
> break;
> + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> + ret = flow_dv_validate_item_port_id
> + (dev, items, attr, item_flags, error);
> + if (ret < 0)
> + return ret;
> + last_item |= MLX5_FLOW_ITEM_PORT_ID;
> + break;
> case RTE_FLOW_ITEM_TYPE_ETH:
> ret = mlx5_flow_validate_item_eth(items, item_flags,
> error);
> @@ -1943,6 +2123,17 @@ struct field_modify_info modify_tcp[] = {
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> + ret = flow_dv_validate_action_port_id(dev,
> + action_flags,
> + actions,
> + attr,
> + error);
> + if (ret)
> + return ret;
> + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> + ++actions_n;
> + break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> ret = mlx5_flow_validate_action_flag(action_flags,
> attr, error);
> @@ -2133,10 +2324,40 @@ struct field_modify_info modify_tcp[] = {
> "action not supported");
> }
> }
> - if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
> - return rte_flow_error_set(error, EINVAL,
> - RTE_FLOW_ERROR_TYPE_ACTION, actions,
> - "no fate action is found");
> + /* Eswitch has few restrictions on using items and actions */
> + if (attr->transfer) {
> + if (action_flags & MLX5_FLOW_ACTION_FLAG)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action FLAG");
> + if (action_flags & MLX5_FLOW_ACTION_MARK)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action MARK");
> + if (action_flags & MLX5_FLOW_ACTION_QUEUE)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action QUEUE");
> + if (action_flags & MLX5_FLOW_ACTION_RSS)
> + return rte_flow_error_set(error, ENOTSUP,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "unsupported action RSS");
> + if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + actions,
> + "no fate action is found");
> + } else {
> + if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
> + return rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + actions,
> + "no fate action is found");
> + }
> return 0;
> }
>
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
@ 2019-04-18 12:17 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:17 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:46AM +0000, Ori Kam wrote:
> Adds the port ID item to the DV steering code.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 84 ++++++++++++++++++++++++++++++-----------
> 1 file changed, 61 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index b819359..e3d9aa2 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3086,6 +3086,62 @@ struct field_modify_info modify_tcp[] = {
> }
> }
>
> +/**
> + * Add source vport match to the specified matcher.
> + *
> + * @param[in, out] matcher
> + * Flow matcher.
> + * @param[in, out] key
> + * Flow matcher value.
> + * @param[in] port
> + * Source vport value to match
> + * @param[in] mask
> + * Mask
> + */
> +static void
> +flow_dv_translate_item_source_vport(void *matcher, void *key,
> + int16_t port, uint16_t mask)
> +{
> + void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> + void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> +
> + MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> + MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> +}
> +
> +/**
> + * Translate port-id item to eswitch match on port-id.
> + *
> + * @param[in] dev
> + * The devich to configure through.
> + * @param[in, out] matcher
> + * Flow matcher.
> + * @param[in, out] key
> + * Flow matcher value.
> + * @param[in] item
> + * Flow pattern to translate.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise.
> + */
> +static int
> +flow_dv_translate_item_port_id(struct rte_eth_dev *dev, void *matcher,
> + void *key, const struct rte_flow_item *item)
> +{
> + const struct rte_flow_item_port_id *pid_m = item ? item->mask : NULL;
> + const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
> + uint16_t mask, val, id;
> + int ret;
> +
> + mask = pid_m ? pid_m->id : 0xffff;
> + id = pid_v ? pid_v->id : dev->data->port_id;
> + ret = mlx5_port_to_eswitch_info(id, NULL, &val);
> + if (ret)
> + return ret;
> + flow_dv_translate_item_source_vport(matcher, key, val, mask);
> + return 0;
> +}
> +
> static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
>
> #define HEADER_IS_ZERO(match_criteria, headers) \
> @@ -3296,29 +3352,6 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> - * Add source vport match to the specified matcher.
> - *
> - * @param[in, out] matcher
> - * Flow matcher.
> - * @param[in, out] key
> - * Flow matcher value.
> - * @param[in] port
> - * Source vport value to match
> - * @param[in] mask
> - * Mask
> - */
> -static void
> -flow_dv_translate_item_source_vport(void *matcher, void *key,
> - int16_t port, uint16_t mask)
> -{
> - void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
> - void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
> -
> - MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
> - MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
> -}
> -
> -/**
> * Find existing tag resource or create and register a new one.
> *
> * @param dev[in, out]
> @@ -3724,6 +3757,11 @@ struct field_modify_info modify_tcp[] = {
> void *match_value = dev_flow->dv.value.buf;
>
> switch (items->type) {
> + case RTE_FLOW_ITEM_TYPE_PORT_ID:
> + flow_dv_translate_item_port_id(dev, match_mask,
> + match_value, items);
> + last_item = MLX5_FLOW_ITEM_PORT_ID;
> + break;
> case RTE_FLOW_ITEM_TYPE_ETH:
> flow_dv_translate_item_eth(match_mask, match_value,
> items, tunnel);
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
@ 2019-04-18 12:19 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:19 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:47AM +0000, Ori Kam wrote:
> In current implementation the DV steering supported only NIC steering.
> This commit adds the transfer attribute in order to create a matcher
> on the FDB tabels.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5_flow.c | 1 +
> drivers/net/mlx5/mlx5_flow.h | 2 ++
> drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
> 3 files changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
> index b1effda..8afb966 100644
> --- a/drivers/net/mlx5/mlx5_flow.c
> +++ b/drivers/net/mlx5/mlx5_flow.c
> @@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
> flow = rte_calloc(__func__, 1, flow_size, 0);
> flow->drv_type = flow_get_drv_type(dev, attr);
> flow->ingress = attr->ingress;
> + flow->transfer = attr->transfer;
> assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
> flow->drv_type < MLX5_FLOW_TYPE_MAX);
> flow->queue = (void *)(flow + 1);
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 85954c2..9d72024 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
> uint16_t crc; /**< CRC of key. */
> uint16_t priority; /**< Priority of matcher. */
> uint8_t egress; /**< Egress matcher. */
> + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
> uint32_t group; /**< The matcher group. */
> struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
> };
> @@ -382,6 +383,7 @@ struct rte_flow {
> struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
> uint8_t ingress; /**< 1 if the flow is ingress. */
> uint32_t group; /**< The group index. */
> + uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
> };
>
> typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index e3d9aa2..182abb3 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -3192,6 +3192,8 @@ struct field_modify_info modify_tcp[] = {
> * Table id to use.
> * @param[in] egress
> * Direction of the table.
> + * @param[in] transfer
> + * E-Switch or NIC flow.
> * @param[out] error
> * pointer to error structure.
> *
> @@ -3201,6 +3203,7 @@ struct field_modify_info modify_tcp[] = {
> static struct mlx5_flow_tbl_resource *
> flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
> uint32_t table_id, uint8_t egress,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> struct mlx5_priv *priv = dev->data->dev_private;
> @@ -3208,7 +3211,12 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_flow_tbl_resource *tbl;
>
> #ifdef HAVE_MLX5DV_DR
> - if (egress) {
> + if (transfer) {
> + tbl = &sh->fdb_tbl[table_id];
> + if (!tbl->obj)
> + tbl->obj = mlx5_glue->dr_create_flow_tbl
> + (sh->fdb_ns, table_id);
> + } else if (egress) {
> tbl = &sh->tx_tbl[table_id];
> if (!tbl->obj)
> tbl->obj = mlx5_glue->dr_create_flow_tbl
> @@ -3230,7 +3238,9 @@ struct field_modify_info modify_tcp[] = {
> #else
> (void)error;
> (void)tbl;
> - if (egress)
> + if (transfer)
> + return &sh->fdb_tbl[table_id];
> + else if (egress)
> return &sh->tx_tbl[table_id];
> else
> return &sh->rx_tbl[table_id];
> @@ -3295,6 +3305,7 @@ struct field_modify_info modify_tcp[] = {
> matcher->priority == cache_matcher->priority &&
> matcher->egress == cache_matcher->egress &&
> matcher->group == cache_matcher->group &&
> + matcher->transfer == cache_matcher->transfer &&
> !memcmp((const void *)matcher->mask.buf,
> (const void *)cache_matcher->mask.buf,
> cache_matcher->mask.size)) {
> @@ -3316,7 +3327,8 @@ struct field_modify_info modify_tcp[] = {
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> "cannot allocate matcher memory");
> tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
> - matcher->egress, error);
> + matcher->egress, matcher->transfer,
> + error);
> if (!tbl) {
> rte_free(cache_matcher);
> return rte_flow_error_set(error, ENOMEM,
> @@ -3643,7 +3655,8 @@ struct field_modify_info modify_tcp[] = {
> jump_data = action->conf;
> tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
> MLX5_GROUP_FACTOR,
> - attr->egress, error);
> + attr->egress,
> + attr->transfer, error);
> if (!tbl)
> return rte_flow_error_set
> (error, errno,
> @@ -3871,6 +3884,7 @@ struct field_modify_info modify_tcp[] = {
> matcher.priority);
> matcher.egress = attr->egress;
> matcher.group = attr->group;
> + matcher.transfer = attr->transfer;
> if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
> return -rte_errno;
> return 0;
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
@ 2019-04-18 12:19 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:19 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:48AM +0000, Ori Kam wrote:
> This commits adds matching on source port, using DV API.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5.h | 2 +
> drivers/net/mlx5/mlx5_flow.h | 12 ++++
> drivers/net/mlx5/mlx5_flow_dv.c | 149 ++++++++++++++++++++++++++++++++++++++++
> drivers/net/mlx5/mlx5_glue.c | 14 ++++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 178 insertions(+)
>
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index dd22bb6..4b6dbc2 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -281,6 +281,8 @@ struct mlx5_ibv_shared {
> LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
> LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
> LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
> + LIST_HEAD(port_id_action_list, mlx5_flow_dv_port_id_action_resource)
> + port_id_action_list; /* List of port ID actions. */
> /* Shared interrupt handler section. */
> pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
> uint32_t intr_cnt; /* Interrupt handler reference counter. */
> diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
> index 9d72024..c419e6b 100644
> --- a/drivers/net/mlx5/mlx5_flow.h
> +++ b/drivers/net/mlx5/mlx5_flow.h
> @@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
> struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
> };
>
> +/* Port ID resource structure. */
> +struct mlx5_flow_dv_port_id_action_resource {
> + LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
> + /* Pointer to next element. */
> + rte_atomic32_t refcnt; /**< Reference counter. */
> + void *action;
> + /**< Verbs tag action object. */
> + uint32_t port_id; /**< Port ID value. */
> +};
> +
> /*
> * Max number of actions per DV flow.
> * See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
> @@ -289,6 +299,8 @@ struct mlx5_flow_dv {
> struct ibv_flow *flow; /**< Installed flow. */
> struct mlx5_flow_dv_jump_tbl_resource *jump;
> /**< Pointer to the jump action resource. */
> + struct mlx5_flow_dv_port_id_action_resource *port_id_action;
> + /**< Pointer to port ID action resource. */
> #ifdef HAVE_IBV_FLOW_DV_SUPPORT
> void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
> /**< Action list. */
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 182abb3..9c5826c 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -1054,6 +1054,70 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Find existing table port ID resource or create and register a new one.
> + *
> + * @param dev[in, out]
> + * Pointer to rte_eth_dev structure.
> + * @param[in, out] resource
> + * Pointer to port ID action resource.
> + * @parm[in, out] dev_flow
> + * Pointer to the dev_flow.
> + * @param[out] error
> + * pointer to error structure.
> + *
> + * @return
> + * 0 on success otherwise -errno and errno is set.
> + */
> +static int
> +flow_dv_port_id_action_resource_register
> + (struct rte_eth_dev *dev,
> + struct mlx5_flow_dv_port_id_action_resource *resource,
> + struct mlx5_flow *dev_flow,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct mlx5_ibv_shared *sh = priv->sh;
> + struct mlx5_flow_dv_port_id_action_resource *cache_resource;
> +
> + /* Lookup a matching resource from cache. */
> + LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
> + if (resource->port_id == cache_resource->port_id) {
> + DRV_LOG(DEBUG, "port id action resource resource %p: "
> + "refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + rte_atomic32_inc(&cache_resource->refcnt);
> + dev_flow->dv.port_id_action = cache_resource;
> + return 0;
> + }
> + }
> + /* Register new port id action resource. */
> + cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
> + if (!cache_resource)
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> + "cannot allocate resource memory");
> + *cache_resource = *resource;
> + cache_resource->action =
> + mlx5_glue->dr_create_flow_action_dest_vport(priv->sh->fdb_ns,
> + resource->port_id);
> + if (!cache_resource->action) {
> + rte_free(cache_resource);
> + return rte_flow_error_set(error, ENOMEM,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL, "cannot create action");
> + }
> + rte_atomic32_init(&cache_resource->refcnt);
> + rte_atomic32_inc(&cache_resource->refcnt);
> + LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
> + dev_flow->dv.port_id_action = cache_resource;
> + DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + return 0;
> +}
> +
> +/**
> * Get the size of specific rte_flow_item_type
> *
> * @param[in] item_type
> @@ -3456,6 +3520,44 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Translate port ID action to vport.
> + *
> + * @param[in] dev
> + * Pointer to rte_eth_dev structure.
> + * @param[in] action
> + * Pointer to the port ID action.
> + * @param[out] dst_port_id
> + * The target port ID.
> + * @param[out] error
> + * Pointer to the error structure.
> + *
> + * @return
> + * 0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +static int
> +flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
> + const struct rte_flow_action *action,
> + uint32_t *dst_port_id,
> + struct rte_flow_error *error)
> +{
> + uint32_t port;
> + uint16_t port_id;
> + int ret;
> + const struct rte_flow_action_port_id *conf =
> + (const struct rte_flow_action_port_id *)action->conf;
> +
> + port = conf->original ? dev->data->port_id : conf->id;
> + ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
> + if (ret)
> + return rte_flow_error_set(error, -ret,
> + RTE_FLOW_ERROR_TYPE_ACTION,
> + NULL,
> + "No eswitch info was found for port");
> + *dst_port_id = port_id;
> + return 0;
> +}
> +
> +/**
> * Fill the flow with DV spec.
> *
> * @param[in] dev
> @@ -3513,10 +3615,24 @@ struct field_modify_info modify_tcp[] = {
> const struct rte_flow_action_jump *jump_data;
> struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
> struct mlx5_flow_tbl_resource *tbl;
> + uint32_t port_id = 0;
> + struct mlx5_flow_dv_port_id_action_resource port_id_resource;
>
> switch (actions->type) {
> case RTE_FLOW_ACTION_TYPE_VOID:
> break;
> + case RTE_FLOW_ACTION_TYPE_PORT_ID:
> + if (flow_dv_translate_action_port_id(dev, action,
> + &port_id, error))
> + return -rte_errno;
> + port_id_resource.port_id = port_id;
> + if (flow_dv_port_id_action_resource_register
> + (dev, &port_id_resource, dev_flow, error))
> + return -rte_errno;
> + dev_flow->dv.actions[actions_n++] =
> + dev_flow->dv.port_id_action->action;
> + action_flags |= MLX5_FLOW_ACTION_PORT_ID;
> + break;
> case RTE_FLOW_ACTION_TYPE_FLAG:
> tag_resource.tag =
> mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
> @@ -4116,6 +4232,37 @@ struct field_modify_info modify_tcp[] = {
> }
>
> /**
> + * Release port ID action resource.
> + *
> + * @param flow
> + * Pointer to mlx5_flow.
> + *
> + * @return
> + * 1 while a reference on it exists, 0 when freed.
> + */
> +static int
> +flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
> +{
> + struct mlx5_flow_dv_port_id_action_resource *cache_resource =
> + flow->dv.port_id_action;
> +
> + assert(cache_resource->action);
> + DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
> + (void *)cache_resource,
> + rte_atomic32_read(&cache_resource->refcnt));
> + if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
> + claim_zero(mlx5_glue->destroy_flow_action
> + (cache_resource->action));
> + LIST_REMOVE(cache_resource, next);
> + rte_free(cache_resource);
> + DRV_LOG(DEBUG, "port id action resource %p: removed",
> + (void *)cache_resource);
> + return 0;
> + }
> + return 1;
> +}
> +
> +/**
> * Remove the flow from the NIC but keeps it in memory.
> *
> * @param[in] dev
> @@ -4182,6 +4329,8 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_modify_hdr_resource_release(dev_flow);
> if (dev_flow->dv.jump)
> flow_dv_jump_tbl_resource_release(dev_flow);
> + if (dev_flow->dv.port_id_action)
> + flow_dv_port_id_action_resource_release(dev_flow);
> rte_free(dev_flow);
> }
> }
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index a508faa..117190f 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -382,6 +382,18 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
> +{
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + return mlx5dv_dr_create_action_dest_vport(ns, vport);
> +#else
> + (void)ns;
> + (void)vport;
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> {
> #ifdef HAVE_MLX5DV_DR
> @@ -847,6 +859,8 @@
> .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
> .dr_create_flow_action_dest_flow_tbl =
> mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> + .dr_create_flow_action_dest_vport =
> + mlx5_glue_dr_create_flow_action_dest_vport,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns,
> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> index 058e9b1..26f5cb3 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -146,6 +146,7 @@ struct mlx5_glue {
> const char *(*port_state_str)(enum ibv_port_state port_state);
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> + void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type Ori Kam
@ 2019-04-18 12:21 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:21 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:49AM +0000, Ori Kam wrote:
> Actions like encap/decap, modify header require setting the flow table
> type. Until now we supported only Nic RX and Nic TX, this commits adds
> the support for FDB table type for those actions.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
Acked-by: Yongseok Koh <yskoh@mellanox.com>
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5_flow_dv.c | 56 +++++++++++++++++++++++++++++------------
> 1 file changed, 40 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 9c5826c..5997182 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -40,6 +40,10 @@
> #define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
> #endif
>
> +#ifndef HAVE_MLX5DV_DR_ESWITCH
> +#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
> +#endif
> +
> union flow_dv_attr {
> struct {
> uint32_t valid:1;
> @@ -939,7 +943,9 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5dv_dr_ns *ns;
>
> resource->flags = flow->group ? 0 : 1;
> - if (flow->ingress)
> + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> + ns = sh->fdb_ns;
> + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
> ns = sh->rx_ns;
> else
> ns = sh->tx_ns;
> @@ -1360,6 +1366,8 @@ struct field_modify_info modify_tcp[] = {
> * Pointer to action structure.
> * @param[in, out] dev_flow
> * Pointer to the mlx5_flow.
> + * @param[in] transfer
> + * Mark if the flow is E-Switch flow.
> * @param[out] error
> * Pointer to the error structure.
> *
> @@ -1370,6 +1378,7 @@ struct field_modify_info modify_tcp[] = {
> flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
> const struct rte_flow_action *action,
> struct mlx5_flow *dev_flow,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> const struct rte_flow_item *encap_data;
> @@ -1377,7 +1386,8 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_flow_dv_encap_decap_resource res = {
> .reformat_type =
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL,
> - .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
> + .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
> + MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
> };
>
> if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> @@ -1412,6 +1422,8 @@ struct field_modify_info modify_tcp[] = {
> * Pointer to rte_eth_dev structure.
> * @param[in, out] dev_flow
> * Pointer to the mlx5_flow.
> + * @param[in] transfer
> + * Mark if the flow is E-Switch flow.
> * @param[out] error
> * Pointer to the error structure.
> *
> @@ -1421,13 +1433,15 @@ struct field_modify_info modify_tcp[] = {
> static int
> flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
> struct mlx5_flow *dev_flow,
> + uint8_t transfer,
> struct rte_flow_error *error)
> {
> struct mlx5_flow_dv_encap_decap_resource res = {
> .size = 0,
> .reformat_type =
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2,
> - .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
> + .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
> + MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
> };
>
> if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
> @@ -1470,8 +1484,11 @@ struct field_modify_info modify_tcp[] = {
> res.reformat_type = attr->egress ?
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL :
> MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
> - res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> - MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> + if (attr->transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> + else
> + res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
> + MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
> if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
> return rte_flow_error_set(error, EINVAL,
> RTE_FLOW_ERROR_TYPE_ACTION,
> @@ -1806,11 +1823,14 @@ struct field_modify_info modify_tcp[] = {
> struct mlx5_priv *priv = dev->data->dev_private;
> struct mlx5_ibv_shared *sh = priv->sh;
> struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
> + struct mlx5dv_dr_ns *ns;
>
> - struct mlx5dv_dr_ns *ns =
> - resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
> - sh->tx_ns : sh->rx_ns;
> -
> + if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
> + ns = sh->fdb_ns;
> + else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
> + ns = sh->tx_ns;
> + else
> + ns = sh->rx_ns;
> /* Lookup a matching resource from cache. */
> LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
> if (resource->ft_type == cache_resource->ft_type &&
> @@ -3604,6 +3624,8 @@ struct field_modify_info modify_tcp[] = {
> union flow_dv_attr flow_attr = { .attr = 0 };
> struct mlx5_flow_dv_tag_resource tag_resource;
>
> + if (attr->transfer)
> + res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
> if (priority == MLX5_FLOW_PRIO_RSVD)
> priority = priv->config.flow_prio - 1;
> for (; !actions_end ; actions++) {
> @@ -3709,7 +3731,9 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
> case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
> if (flow_dv_create_action_l2_encap(dev, actions,
> - dev_flow, error))
> + dev_flow,
> + attr->transfer,
> + error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> @@ -3721,6 +3745,7 @@ struct field_modify_info modify_tcp[] = {
> case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
> case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
> if (flow_dv_create_action_l2_decap(dev, dev_flow,
> + attr->transfer,
> error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> @@ -3740,9 +3765,9 @@ struct field_modify_info modify_tcp[] = {
> dev_flow->dv.encap_decap->verbs_action;
> } else {
> /* Handle encap without preceding decap. */
> - if (flow_dv_create_action_l2_encap(dev, actions,
> - dev_flow,
> - error))
> + if (flow_dv_create_action_l2_encap
> + (dev, actions, dev_flow, attr->transfer,
> + error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> @@ -3757,9 +3782,8 @@ struct field_modify_info modify_tcp[] = {
> }
> /* Handle decap only if it isn't followed by encap. */
> if (action->type != RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
> - if (flow_dv_create_action_l2_decap(dev,
> - dev_flow,
> - error))
> + if (flow_dv_create_action_l2_decap
> + (dev, dev_flow, attr->transfer, error))
> return -rte_errno;
> dev_flow->dv.actions[actions_n++] =
> dev_flow->dv.encap_decap->verbs_action;
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
@ 2019-04-18 12:28 ` Yongseok Koh
0 siblings, 0 replies; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 12:28 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
On Thu, Apr 18, 2019 at 11:28:50AM +0000, Ori Kam wrote:
> This commit adds support for drop action when creating E-Switch flow
> using DV.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> ---
> v2:
> * Address ML comments.
> ---
> drivers/net/mlx5/mlx5.c | 9 +++++++++
> drivers/net/mlx5/mlx5.h | 1 +
> drivers/net/mlx5/mlx5_flow_dv.c | 23 ++++++++++++++++-------
> drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 39 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index ff24e1d..65c69af 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> goto error;
> }
> sh->fdb_ns = ns;
> + sh->esw_drop_action = mlx5_glue->dr_create_flow_action_drop();
> }
> #endif
> sh->dv_refcnt++;
> @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->esw_drop_action) {
> + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> + sh->esw_drop_action = NULL;
> + }
> return err;
> #else
> (void)priv;
> @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->esw_drop_action) {
> + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> + sh->esw_drop_action = NULL;
> + }
> #endif
> pthread_mutex_destroy(&sh->dv_mutex);
> #else
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 4b6dbc2..a736d57 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -275,6 +275,7 @@ struct mlx5_ibv_shared {
> /* RX Direct Rules tables. */
> void *tx_ns; /* TX Direct Rules name space handle. */
> struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> + void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
> /* TX Direct Rules tables/ */
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 5997182..5ec8caf 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = {
> {
> struct mlx5_flow_dv *dv;
> struct mlx5_flow *dev_flow;
> + struct mlx5_priv *priv = dev->data->dev_private;
> int n;
> int err;
>
> @@ -4056,13 +4057,21 @@ struct field_modify_info modify_tcp[] = {
> dv = &dev_flow->dv;
> n = dv->actions_n;
> if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> - dv->hrxq = mlx5_hrxq_drop_new(dev);
> - if (!dv->hrxq) {
> - rte_flow_error_set
> - (error, errno,
> - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> - "cannot get drop hash queue");
> - goto error;
> + if (flow->transfer)
> + dv->actions[n++] = priv->sh->esw_drop_action;
> + else {
> + dv->hrxq = mlx5_hrxq_drop_new(dev);
> + if (!dv->hrxq) {
> + rte_flow_error_set
> + (error, errno,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "cannot get drop hash queue");
> + goto error;
> + }
> + dv->actions[n++] =
> + mlx5_glue->dv_create_flow_action_dest_ibv_qp
> + (dv->hrxq->qp);
Looks like a bug when rebasing it.
dv->actions[] is set twice. You should remove this line and move the below one
to this place. Right?
Thanks,
Yongseok
> }
> dv->actions[n++] = dv->hrxq->action;
> } else if (flow->actions &
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index 117190f..b32cd09c 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -394,6 +394,16 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_drop(void)
> +{
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + return mlx5dv_dr_create_action_drop();
> +#else
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> {
> #ifdef HAVE_MLX5DV_DR
> @@ -861,6 +871,8 @@
> mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> .dr_create_flow_action_dest_vport =
> mlx5_glue_dr_create_flow_action_dest_vport,
> + .dr_create_flow_action_drop =
> + mlx5_glue_dr_create_flow_action_drop,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns,
> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> index 26f5cb3..1d06583 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -147,6 +147,7 @@ struct mlx5_glue {
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> + void *(*dr_create_flow_action_drop)();
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (9 preceding siblings ...)
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
@ 2019-04-18 13:15 ` Ori Kam
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 1/9] net/mlx5: fix translate vport function name Ori Kam
` (9 more replies)
10 siblings, 10 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:15 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Currently MLX5 PMD supports 3 flow engines:
Verbs, Direct Verbs and TCF. The first two engines are for Nic steering
while the TCF is for E-Switch steering.
This series add E-Switch steering support also for the DV engine.
In order to support the new capability there should be support from
both the RDMA and from the NIC.
v3:
* Small nit in patch 03
* Fix bug in patch 09
v2:
* Address ML comments
Ori Kam (9):
net/mlx5: fix translate vport function name
net/mlx5: fix meson build for Direct Rules
net/mlx5: add Direct Rules E-Switch support
net/mlx5: add validation for Direct Rule E-Switch
net/mlx5: add port ID item to Direct Verbs
net/mlx5: add transfer attribute to matcher
net/mlx5: add E-Switch port ID action to Direct Verbs
net/mlx5: add Forward Database table type
net/mlx5: add drop action to Direct Verbs E-Switch
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 4 +
drivers/net/mlx5/mlx5.c | 62 +++-
drivers/net/mlx5/mlx5.h | 18 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 44 +++
drivers/net/mlx5/mlx5_ethdev.c | 41 +++
drivers/net/mlx5/mlx5_flow.c | 3 +-
drivers/net/mlx5/mlx5_flow.h | 19 ++
drivers/net/mlx5/mlx5_flow_dv.c | 584 +++++++++++++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_glue.c | 26 ++
drivers/net/mlx5/mlx5_glue.h | 2 +
drivers/net/mlx5/mlx5_prm.h | 328 +++++++++++++++++++++
12 files changed, 1064 insertions(+), 72 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 1/9] net/mlx5: fix translate vport function name
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
@ 2019-04-18 13:15 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
` (8 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:15 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Modify the translate vport function to match other translate items
naming convestions.
Fixes: 0fe3f18f78d8 ("net/mlx5: add source vport match to the ingress rules")
Cc: viacheslavo@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 04ab3d6..1e25e0b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3087,8 +3087,8 @@ struct field_modify_info modify_tcp[] = {
* Mask
*/
static void
-flow_dv_translate_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
{
void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
@@ -3492,10 +3492,10 @@ struct field_modify_info modify_tcp[] = {
* Add matching on source vport index only
* for ingress rules in E-Switch configurations.
*/
- flow_dv_translate_source_vport(matcher.mask.buf,
- dev_flow->dv.value.buf,
- priv->vport_id,
- 0xffff);
+ flow_dv_translate_item_source_vport(matcher.mask.buf,
+ dev_flow->dv.value.buf,
+ priv->vport_id,
+ 0xffff);
}
for (; items->type != RTE_FLOW_ITEM_TYPE_END; items++) {
int tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 2/9] net/mlx5: fix meson build for Direct Rules
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 1/9] net/mlx5: fix translate vport function name Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
` (7 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
The meson build was missing the define for Direct Rules.
Fixes: 4f84a19779ca ("net/mlx5: add Direct Rules API")
Cc: orika@mellanox.com
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/meson.build | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index a4c684e..0037e15 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -111,6 +111,8 @@ if build
'mlx5dv_devx_obj_create' ],
[ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h',
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
+ [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_TYPE_TERMINATING' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 3/9] net/mlx5: add Direct Rules E-Switch support
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
` (6 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit checks the for DR E-Switch support.
The support is based on both Device and Kernel.
This commit also enables the user to manualy disable this this feature.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
v3:
* Add comment to fdb_tbl
---
drivers/net/mlx5/Makefile | 5 +
drivers/net/mlx5/meson.build | 2 +
drivers/net/mlx5/mlx5.c | 53 +++++-
drivers/net/mlx5/mlx5.h | 13 ++
drivers/net/mlx5/mlx5_devx_cmds.c | 44 +++++
drivers/net/mlx5/mlx5_flow.c | 2 +-
drivers/net/mlx5/mlx5_prm.h | 328 ++++++++++++++++++++++++++++++++++++++
7 files changed, 441 insertions(+), 6 deletions(-)
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index 93bc869..2b72a33 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -161,6 +161,11 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
enum MLX5DV_DR_NS_TYPE_TERMINATING \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
+ HAVE_MLX5DV_DR_ESWITCH \
+ infiniband/mlx5dv.h \
+ enum MLX5DV_DR_NS_DOMAIN_FDB_BYPASS \
+ $(AUTOCONF_OUTPUT)
+ $Q sh -- '$<' '$@' \
HAVE_IBV_DEVX_OBJ \
infiniband/mlx5dv.h \
func mlx5dv_devx_obj_create \
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 0037e15..9dfd28d 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -113,6 +113,8 @@ if build
'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ],
[ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h',
'MLX5DV_DR_NS_TYPE_TERMINATING' ],
+ [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h',
+ 'MLX5DV_DR_NS_DOMAIN_FDB_BYPASS' ],
[ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h',
'SUPPORTED_40000baseKR4_Full' ],
[ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 9ff50df..ff24e1d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -101,6 +101,9 @@
/* Allow L3 VXLAN flow creation. */
#define MLX5_L3_VXLAN_EN "l3_vxlan_en"
+/* Activate DV E-Switch flow steering. */
+#define MLX5_DV_ESW_EN "dv_esw_en"
+
/* Activate DV flow steering. */
#define MLX5_DV_FLOW_EN "dv_flow_en"
@@ -344,6 +347,18 @@ struct mlx5_dev_spawn_data {
}
pthread_mutex_init(&sh->dv_mutex, NULL);
sh->tx_ns = ns;
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (priv->config.dv_esw_en) {
+ ns = mlx5_glue->dr_create_ns(sh->ctx,
+ MLX5DV_DR_NS_DOMAIN_FDB_BYPASS);
+ if (!ns) {
+ DRV_LOG(ERR, "FDB mlx5dv_dr_create_ns failed");
+ err = errno;
+ goto error;
+ }
+ sh->fdb_ns = ns;
+ }
+#endif
sh->dv_refcnt++;
priv->dr_shared = 1;
return 0;
@@ -358,6 +373,10 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
return err;
#else
(void)priv;
@@ -393,6 +412,12 @@ struct mlx5_dev_spawn_data {
mlx5dv_dr_destroy_ns(sh->tx_ns);
sh->tx_ns = NULL;
}
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (sh->fdb_ns) {
+ mlx5_glue->dr_destroy_ns(sh->fdb_ns);
+ sh->fdb_ns = NULL;
+ }
+#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
(void)priv;
@@ -861,6 +886,8 @@ struct mlx5_dev_spawn_data {
config->l3_vxlan_en = !!tmp;
} else if (strcmp(MLX5_VF_NL_EN, key) == 0) {
config->vf_nl_en = !!tmp;
+ } else if (strcmp(MLX5_DV_ESW_EN, key) == 0) {
+ config->dv_esw_en = !!tmp;
} else if (strcmp(MLX5_DV_FLOW_EN, key) == 0) {
config->dv_flow_en = !!tmp;
} else if (strcmp(MLX5_MR_EXT_MEMSEG_EN, key) == 0) {
@@ -905,6 +932,7 @@ struct mlx5_dev_spawn_data {
MLX5_RX_VEC_EN,
MLX5_L3_VXLAN_EN,
MLX5_VF_NL_EN,
+ MLX5_DV_ESW_EN,
MLX5_DV_FLOW_EN,
MLX5_MR_EXT_MEMSEG_EN,
MLX5_REPRESENTOR,
@@ -1458,11 +1486,6 @@ struct mlx5_dev_spawn_data {
priv->tcf_context = NULL;
}
}
- if (config.dv_flow_en) {
- err = mlx5_alloc_shared_dr(priv);
- if (err)
- goto error;
- }
TAILQ_INIT(&priv->flows);
TAILQ_INIT(&priv->ctrl_flows);
/* Hint libmlx5 to use PMD allocator for data plane resources */
@@ -1484,8 +1507,27 @@ struct mlx5_dev_spawn_data {
* Verbs context returned by ibv_open_device().
*/
mlx5_link_update(eth_dev, 0);
+#ifdef HAVE_IBV_DEVX_OBJ
+ err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config.hca_attr);
+ if (err) {
+ err = -err;
+ goto error;
+ }
+#endif
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ if (!(config.hca_attr.eswitch_manager && config.dv_flow_en &&
+ (switch_info->representor || switch_info->master)))
+ config.dv_esw_en = 0;
+#else
+ config.dv_esw_en = 0;
+#endif
/* Store device configuration on private structure. */
priv->config = config;
+ if (config.dv_flow_en) {
+ err = mlx5_alloc_shared_dr(priv);
+ if (err)
+ goto error;
+ }
/* Supported Verbs flow priority number detection. */
err = mlx5_flow_discover_priorities(eth_dev);
if (err < 0) {
@@ -1876,6 +1918,7 @@ struct mlx5_dev_spawn_data {
.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN,
.min_rxqs_num = MLX5_MPRQ_MIN_RXQS,
},
+ .dv_esw_en = 1,
};
/* Device specific configuration. */
switch (pci_dev->id.device_id) {
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 14c7f3c..f20e589 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -138,6 +138,11 @@ struct mlx5_devx_counter_set {
int id; /* Flow counter ID */
};
+/* HCA attributes. */
+struct mlx5_hca_attr {
+ uint32_t eswitch_manager:1;
+};
+
/* Flow list . */
TAILQ_HEAD(mlx5_flows, rte_flow);
@@ -171,6 +176,7 @@ struct mlx5_dev_config {
/* Whether memseg should be extended for MR creation. */
unsigned int l3_vxlan_en:1; /* Enable L3 VXLAN flow creation. */
unsigned int vf_nl_en:1; /* Enable Netlink requests in VF mode. */
+ unsigned int dv_esw_en:1; /* Enable E-Switch DV flow. */
unsigned int dv_flow_en:1; /* Enable DV flow. */
unsigned int swp:1; /* Tx generic tunnel checksum and TSO offload. */
unsigned int devx:1; /* Whether devx interface is available or not. */
@@ -192,6 +198,7 @@ struct mlx5_dev_config {
int txqs_inline; /* Queue number threshold for inlining. */
int txqs_vec; /* Queue number threshold for vectorized Tx. */
int inline_max_packet_sz; /* Max packet size for inlining. */
+ struct mlx5_hca_attr hca_attr; /* HCA attributes. */
};
/**
@@ -241,6 +248,7 @@ struct mlx5_flow_tbl_resource {
};
#define MLX5_MAX_TABLES 1024
+#define MLX5_MAX_TABLES_FDB 32
#define MLX5_GROUP_FACTOR 1
/*
@@ -260,6 +268,9 @@ struct mlx5_ibv_shared {
/* Shared DV/DR flow data section. */
pthread_mutex_t dv_mutex; /* DV context mutex. */
uint32_t dv_refcnt; /* DV/DR data reference counter. */
+ void *fdb_ns; /* FDB Direct Rules name space handle. */
+ struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB];
+ /* FDB Direct Rules tables. */
void *rx_ns; /* RX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES];
/* RX Direct Rules tables. */
@@ -539,4 +550,6 @@ int mlx5_devx_cmd_flow_counter_alloc(struct ibv_context *ctx,
int mlx5_devx_cmd_flow_counter_query(struct mlx5_devx_counter_set *dcx,
int clear,
uint64_t *pkts, uint64_t *bytes);
+int mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr);
#endif /* RTE_PMD_MLX5_H_ */
diff --git a/drivers/net/mlx5/mlx5_devx_cmds.c b/drivers/net/mlx5/mlx5_devx_cmds.c
index a9dff58..e5776c4 100644
--- a/drivers/net/mlx5/mlx5_devx_cmds.c
+++ b/drivers/net/mlx5/mlx5_devx_cmds.c
@@ -105,3 +105,47 @@ int mlx5_devx_cmd_flow_counter_free(struct mlx5dv_devx_obj *obj)
*bytes = MLX5_GET64(traffic_counter, stats, octets);
return 0;
}
+
+/**
+ * Query HCA attributes.
+ * Using those attributes we can check on run time if the device
+ * is having the required capabilities.
+ *
+ * @param[in] ctx
+ * ibv contexts returned from mlx5dv_open_device.
+ * @param[out] attr
+ * Attributes device values.
+ *
+ * @return
+ * 0 on success, a negative value otherwise.
+ */
+int
+mlx5_devx_cmd_query_hca_attr(struct ibv_context *ctx,
+ struct mlx5_hca_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
+ void *hcattr;
+ int status, syndrome, rc;
+
+ MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
+ MLX5_SET(query_hca_cap_in, in, op_mod,
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE |
+ MLX5_HCA_CAP_OPMOD_GET_CUR);
+
+ rc = mlx5_glue->devx_general_cmd(ctx,
+ in, sizeof(in), out, sizeof(out));
+ if (rc)
+ return rc;
+ status = MLX5_GET(query_hca_cap_out, out, status);
+ syndrome = MLX5_GET(query_hca_cap_out, out, syndrome);
+ if (status) {
+ DRV_LOG(DEBUG, "Failed to query devx HCA capabilities, "
+ "status %x, syndrome = %x",
+ status, syndrome);
+ return -1;
+ }
+ hcattr = MLX5_ADDR_OF(query_hca_cap_out, out, capability);
+ attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);
+ return 0;
+}
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index a0683ee..b1effda 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -1784,7 +1784,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
struct mlx5_priv *priv = dev->data->dev_private;
enum mlx5_flow_drv_type type = MLX5_FLOW_TYPE_MAX;
- if (attr->transfer)
+ if (attr->transfer && !priv->config.dv_esw_en)
type = MLX5_FLOW_TYPE_TCF;
else
type = priv->config.dv_flow_en ? MLX5_FLOW_TYPE_DV :
diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h
index b15266f..8c42380 100644
--- a/drivers/net/mlx5/mlx5_prm.h
+++ b/drivers/net/mlx5/mlx5_prm.h
@@ -529,6 +529,7 @@ enum {
};
enum {
+ MLX5_CMD_OP_QUERY_HCA_CAP = 0x100,
MLX5_CMD_OP_ALLOC_FLOW_COUNTER = 0x939,
MLX5_CMD_OP_QUERY_FLOW_COUNTER = 0x93b,
};
@@ -591,6 +592,333 @@ struct mlx5_ifc_query_flow_counter_in_bits {
u8 flow_counter_id[0x20];
};
+enum {
+ MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
+ MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP = 0xc << 1,
+};
+
+enum {
+ MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
+ MLX5_HCA_CAP_OPMOD_GET_CUR = 1,
+};
+
+struct mlx5_ifc_cmd_hca_cap_bits {
+ u8 reserved_at_0[0x30];
+ u8 vhca_id[0x10];
+ u8 reserved_at_40[0x40];
+ u8 log_max_srq_sz[0x8];
+ u8 log_max_qp_sz[0x8];
+ u8 reserved_at_90[0xb];
+ u8 log_max_qp[0x5];
+ u8 reserved_at_a0[0xb];
+ u8 log_max_srq[0x5];
+ u8 reserved_at_b0[0x10];
+ u8 reserved_at_c0[0x8];
+ u8 log_max_cq_sz[0x8];
+ u8 reserved_at_d0[0xb];
+ u8 log_max_cq[0x5];
+ u8 log_max_eq_sz[0x8];
+ u8 reserved_at_e8[0x2];
+ u8 log_max_mkey[0x6];
+ u8 reserved_at_f0[0x8];
+ u8 dump_fill_mkey[0x1];
+ u8 reserved_at_f9[0x3];
+ u8 log_max_eq[0x4];
+ u8 max_indirection[0x8];
+ u8 fixed_buffer_size[0x1];
+ u8 log_max_mrw_sz[0x7];
+ u8 force_teardown[0x1];
+ u8 reserved_at_111[0x1];
+ u8 log_max_bsf_list_size[0x6];
+ u8 umr_extended_translation_offset[0x1];
+ u8 null_mkey[0x1];
+ u8 log_max_klm_list_size[0x6];
+ u8 reserved_at_120[0xa];
+ u8 log_max_ra_req_dc[0x6];
+ u8 reserved_at_130[0xa];
+ u8 log_max_ra_res_dc[0x6];
+ u8 reserved_at_140[0xa];
+ u8 log_max_ra_req_qp[0x6];
+ u8 reserved_at_150[0xa];
+ u8 log_max_ra_res_qp[0x6];
+ u8 end_pad[0x1];
+ u8 cc_query_allowed[0x1];
+ u8 cc_modify_allowed[0x1];
+ u8 start_pad[0x1];
+ u8 cache_line_128byte[0x1];
+ u8 reserved_at_165[0xa];
+ u8 qcam_reg[0x1];
+ u8 gid_table_size[0x10];
+ u8 out_of_seq_cnt[0x1];
+ u8 vport_counters[0x1];
+ u8 retransmission_q_counters[0x1];
+ u8 debug[0x1];
+ u8 modify_rq_counter_set_id[0x1];
+ u8 rq_delay_drop[0x1];
+ u8 max_qp_cnt[0xa];
+ u8 pkey_table_size[0x10];
+ u8 vport_group_manager[0x1];
+ u8 vhca_group_manager[0x1];
+ u8 ib_virt[0x1];
+ u8 eth_virt[0x1];
+ u8 vnic_env_queue_counters[0x1];
+ u8 ets[0x1];
+ u8 nic_flow_table[0x1];
+ u8 eswitch_manager[0x1];
+ u8 device_memory[0x1];
+ u8 mcam_reg[0x1];
+ u8 pcam_reg[0x1];
+ u8 local_ca_ack_delay[0x5];
+ u8 port_module_event[0x1];
+ u8 enhanced_error_q_counters[0x1];
+ u8 ports_check[0x1];
+ u8 reserved_at_1b3[0x1];
+ u8 disable_link_up[0x1];
+ u8 beacon_led[0x1];
+ u8 port_type[0x2];
+ u8 num_ports[0x8];
+ u8 reserved_at_1c0[0x1];
+ u8 pps[0x1];
+ u8 pps_modify[0x1];
+ u8 log_max_msg[0x5];
+ u8 reserved_at_1c8[0x4];
+ u8 max_tc[0x4];
+ u8 temp_warn_event[0x1];
+ u8 dcbx[0x1];
+ u8 general_notification_event[0x1];
+ u8 reserved_at_1d3[0x2];
+ u8 fpga[0x1];
+ u8 rol_s[0x1];
+ u8 rol_g[0x1];
+ u8 reserved_at_1d8[0x1];
+ u8 wol_s[0x1];
+ u8 wol_g[0x1];
+ u8 wol_a[0x1];
+ u8 wol_b[0x1];
+ u8 wol_m[0x1];
+ u8 wol_u[0x1];
+ u8 wol_p[0x1];
+ u8 stat_rate_support[0x10];
+ u8 reserved_at_1f0[0xc];
+ u8 cqe_version[0x4];
+ u8 compact_address_vector[0x1];
+ u8 striding_rq[0x1];
+ u8 reserved_at_202[0x1];
+ u8 ipoib_enhanced_offloads[0x1];
+ u8 ipoib_basic_offloads[0x1];
+ u8 reserved_at_205[0x1];
+ u8 repeated_block_disabled[0x1];
+ u8 umr_modify_entity_size_disabled[0x1];
+ u8 umr_modify_atomic_disabled[0x1];
+ u8 umr_indirect_mkey_disabled[0x1];
+ u8 umr_fence[0x2];
+ u8 reserved_at_20c[0x3];
+ u8 drain_sigerr[0x1];
+ u8 cmdif_checksum[0x2];
+ u8 sigerr_cqe[0x1];
+ u8 reserved_at_213[0x1];
+ u8 wq_signature[0x1];
+ u8 sctr_data_cqe[0x1];
+ u8 reserved_at_216[0x1];
+ u8 sho[0x1];
+ u8 tph[0x1];
+ u8 rf[0x1];
+ u8 dct[0x1];
+ u8 qos[0x1];
+ u8 eth_net_offloads[0x1];
+ u8 roce[0x1];
+ u8 atomic[0x1];
+ u8 reserved_at_21f[0x1];
+ u8 cq_oi[0x1];
+ u8 cq_resize[0x1];
+ u8 cq_moderation[0x1];
+ u8 reserved_at_223[0x3];
+ u8 cq_eq_remap[0x1];
+ u8 pg[0x1];
+ u8 block_lb_mc[0x1];
+ u8 reserved_at_229[0x1];
+ u8 scqe_break_moderation[0x1];
+ u8 cq_period_start_from_cqe[0x1];
+ u8 cd[0x1];
+ u8 reserved_at_22d[0x1];
+ u8 apm[0x1];
+ u8 vector_calc[0x1];
+ u8 umr_ptr_rlky[0x1];
+ u8 imaicl[0x1];
+ u8 reserved_at_232[0x4];
+ u8 qkv[0x1];
+ u8 pkv[0x1];
+ u8 set_deth_sqpn[0x1];
+ u8 reserved_at_239[0x3];
+ u8 xrc[0x1];
+ u8 ud[0x1];
+ u8 uc[0x1];
+ u8 rc[0x1];
+ u8 uar_4k[0x1];
+ u8 reserved_at_241[0x9];
+ u8 uar_sz[0x6];
+ u8 reserved_at_250[0x8];
+ u8 log_pg_sz[0x8];
+ u8 bf[0x1];
+ u8 driver_version[0x1];
+ u8 pad_tx_eth_packet[0x1];
+ u8 reserved_at_263[0x8];
+ u8 log_bf_reg_size[0x5];
+ u8 reserved_at_270[0xb];
+ u8 lag_master[0x1];
+ u8 num_lag_ports[0x4];
+ u8 reserved_at_280[0x10];
+ u8 max_wqe_sz_sq[0x10];
+ u8 reserved_at_2a0[0x10];
+ u8 max_wqe_sz_rq[0x10];
+ u8 max_flow_counter_31_16[0x10];
+ u8 max_wqe_sz_sq_dc[0x10];
+ u8 reserved_at_2e0[0x7];
+ u8 max_qp_mcg[0x19];
+ u8 reserved_at_300[0x10];
+ u8 flow_counter_bulk_alloc[0x08];
+ u8 log_max_mcg[0x8];
+ u8 reserved_at_320[0x3];
+ u8 log_max_transport_domain[0x5];
+ u8 reserved_at_328[0x3];
+ u8 log_max_pd[0x5];
+ u8 reserved_at_330[0xb];
+ u8 log_max_xrcd[0x5];
+ u8 nic_receive_steering_discard[0x1];
+ u8 receive_discard_vport_down[0x1];
+ u8 transmit_discard_vport_down[0x1];
+ u8 reserved_at_343[0x5];
+ u8 log_max_flow_counter_bulk[0x8];
+ u8 max_flow_counter_15_0[0x10];
+ u8 reserved_at_360[0x3];
+ u8 log_max_rq[0x5];
+ u8 reserved_at_368[0x3];
+ u8 log_max_sq[0x5];
+ u8 reserved_at_370[0x3];
+ u8 log_max_tir[0x5];
+ u8 reserved_at_378[0x3];
+ u8 log_max_tis[0x5];
+ u8 basic_cyclic_rcv_wqe[0x1];
+ u8 reserved_at_381[0x2];
+ u8 log_max_rmp[0x5];
+ u8 reserved_at_388[0x3];
+ u8 log_max_rqt[0x5];
+ u8 reserved_at_390[0x3];
+ u8 log_max_rqt_size[0x5];
+ u8 reserved_at_398[0x3];
+ u8 log_max_tis_per_sq[0x5];
+ u8 ext_stride_num_range[0x1];
+ u8 reserved_at_3a1[0x2];
+ u8 log_max_stride_sz_rq[0x5];
+ u8 reserved_at_3a8[0x3];
+ u8 log_min_stride_sz_rq[0x5];
+ u8 reserved_at_3b0[0x3];
+ u8 log_max_stride_sz_sq[0x5];
+ u8 reserved_at_3b8[0x3];
+ u8 log_min_stride_sz_sq[0x5];
+ u8 hairpin[0x1];
+ u8 reserved_at_3c1[0x2];
+ u8 log_max_hairpin_queues[0x5];
+ u8 reserved_at_3c8[0x3];
+ u8 log_max_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3d0[0x3];
+ u8 log_max_hairpin_num_packets[0x5];
+ u8 reserved_at_3d8[0x3];
+ u8 log_max_wq_sz[0x5];
+ u8 nic_vport_change_event[0x1];
+ u8 disable_local_lb_uc[0x1];
+ u8 disable_local_lb_mc[0x1];
+ u8 log_min_hairpin_wq_data_sz[0x5];
+ u8 reserved_at_3e8[0x3];
+ u8 log_max_vlan_list[0x5];
+ u8 reserved_at_3f0[0x3];
+ u8 log_max_current_mc_list[0x5];
+ u8 reserved_at_3f8[0x3];
+ u8 log_max_current_uc_list[0x5];
+ u8 general_obj_types[0x40];
+ u8 reserved_at_440[0x20];
+ u8 reserved_at_460[0x10];
+ u8 max_num_eqs[0x10];
+ u8 reserved_at_480[0x3];
+ u8 log_max_l2_table[0x5];
+ u8 reserved_at_488[0x8];
+ u8 log_uar_page_sz[0x10];
+ u8 reserved_at_4a0[0x20];
+ u8 device_frequency_mhz[0x20];
+ u8 device_frequency_khz[0x20];
+ u8 reserved_at_500[0x20];
+ u8 num_of_uars_per_page[0x20];
+ u8 flex_parser_protocols[0x20];
+ u8 reserved_at_560[0x20];
+ u8 reserved_at_580[0x3c];
+ u8 mini_cqe_resp_stride_index[0x1];
+ u8 cqe_128_always[0x1];
+ u8 cqe_compression_128[0x1];
+ u8 cqe_compression[0x1];
+ u8 cqe_compression_timeout[0x10];
+ u8 cqe_compression_max_num[0x10];
+ u8 reserved_at_5e0[0x10];
+ u8 tag_matching[0x1];
+ u8 rndv_offload_rc[0x1];
+ u8 rndv_offload_dc[0x1];
+ u8 log_tag_matching_list_sz[0x5];
+ u8 reserved_at_5f8[0x3];
+ u8 log_max_xrq[0x5];
+ u8 affiliate_nic_vport_criteria[0x8];
+ u8 native_port_num[0x8];
+ u8 num_vhca_ports[0x8];
+ u8 reserved_at_618[0x6];
+ u8 sw_owner_id[0x1];
+ u8 reserved_at_61f[0x1e1];
+};
+
+struct mlx5_ifc_qos_cap_bits {
+ u8 packet_pacing[0x1];
+ u8 esw_scheduling[0x1];
+ u8 esw_bw_share[0x1];
+ u8 esw_rate_limit[0x1];
+ u8 reserved_at_4[0x1];
+ u8 packet_pacing_burst_bound[0x1];
+ u8 packet_pacing_typical_size[0x1];
+ u8 flow_meter_srtcm[0x1];
+ u8 reserved_at_8[0x8];
+ u8 log_max_flow_meter[0x8];
+ u8 flow_meter_reg_id[0x8];
+ u8 reserved_at_25[0x20];
+ u8 packet_pacing_max_rate[0x20];
+ u8 packet_pacing_min_rate[0x20];
+ u8 reserved_at_80[0x10];
+ u8 packet_pacing_rate_table_size[0x10];
+ u8 esw_element_type[0x10];
+ u8 esw_tsar_type[0x10];
+ u8 reserved_at_c0[0x10];
+ u8 max_qos_para_vport[0x10];
+ u8 max_tsar_bw_share[0x20];
+ u8 reserved_at_100[0x6e8];
+};
+
+union mlx5_ifc_hca_cap_union_bits {
+ struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
+ struct mlx5_ifc_qos_cap_bits qos_cap;
+ u8 reserved_at_0[0x8000];
+};
+
+struct mlx5_ifc_query_hca_cap_out_bits {
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
+ union mlx5_ifc_hca_cap_union_bits capability;
+};
+
+struct mlx5_ifc_query_hca_cap_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 reserved_at_40[0x40];
+};
+
/* CQE format mask. */
#define MLX5E_CQE_FORMAT_MASK 0xc
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 4/9] net/mlx5: add validation for Direct Rule E-Switch
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (2 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
` (5 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Add validation logic for E-Switch using Direct Rules.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_ethdev.c | 41 +++++++
drivers/net/mlx5/mlx5_flow.h | 5 +
drivers/net/mlx5/mlx5_flow_dv.c | 243 ++++++++++++++++++++++++++++++++++++++--
4 files changed, 280 insertions(+), 11 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index f20e589..5e70856 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -413,6 +413,8 @@ int mlx5_ibv_device_to_pci_addr(const struct ibv_device *device,
unsigned int mlx5_dev_to_port_id(const struct rte_device *dev,
uint16_t *port_list,
unsigned int port_list_n);
+int mlx5_port_to_eswitch_info(uint16_t port, uint16_t *es_domain_id,
+ uint16_t *es_port_id);
int mlx5_sysfs_switch_info(unsigned int ifindex,
struct mlx5_switch_info *info);
bool mlx5_translate_port_name(const char *port_name_in,
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 3992918..695440a 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -1376,6 +1376,47 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size)
}
/**
+ * Get the E-Switch domain id this port belongs to.
+ *
+ * @param[in] port
+ * Device port id.
+ * @param[out] es_domain_id
+ * E-Switch domain id.
+ * @param[out] es_port_id
+ * The port id of the port in the E-Switch.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_port_to_eswitch_info(uint16_t port,
+ uint16_t *es_domain_id, uint16_t *es_port_id)
+{
+ struct rte_eth_dev *dev;
+ struct mlx5_priv *priv;
+
+ if (port >= RTE_MAX_ETHPORTS) {
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ if (!rte_eth_dev_is_valid_port(port)) {
+ rte_errno = ENODEV;
+ return -rte_errno;
+ }
+ dev = &rte_eth_devices[port];
+ priv = dev->data->dev_private;
+ if (!(priv->representor || priv->master)) {
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ if (es_domain_id)
+ *es_domain_id = priv->domain_id;
+ if (es_port_id)
+ *es_port_id = priv->vport_id;
+ return 0;
+}
+
+/**
* Get switch information associated with network interface.
*
* @param ifindex
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9f47fd4..85954c2 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -48,6 +48,7 @@
/* General pattern items bits. */
#define MLX5_FLOW_ITEM_METADATA (1u << 16)
+#define MLX5_FLOW_ITEM_PORT_ID (1u << 17)
/* Outer Masks. */
#define MLX5_FLOW_LAYER_OUTER_L3 \
@@ -118,6 +119,10 @@
(MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP)
+#define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
+ (MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
+ MLX5_FLOW_ACTION_JUMP)
+
#define MLX5_FLOW_ENCAP_ACTIONS (MLX5_FLOW_ACTION_VXLAN_ENCAP | \
MLX5_FLOW_ACTION_NVGRE_ENCAP | \
MLX5_FLOW_ACTION_RAW_ENCAP)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 1e25e0b..b819359 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -613,6 +613,89 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Validate vport item.
+ *
+ * @param[in] dev
+ * Pointer to the rte_eth_dev structure.
+ * @param[in] item
+ * Item specification.
+ * @param[in] attr
+ * Attributes of flow that includes this item.
+ * @param[in] item_flags
+ * Bit-fields that holds the items detected until now.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_item_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_item *item,
+ const struct rte_flow_attr *attr,
+ uint64_t item_flags,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_item_port_id *spec = item->spec;
+ const struct rte_flow_item_port_id *mask = item->mask;
+ const struct rte_flow_item_port_id switch_mask = {
+ .id = 0xffffffff,
+ };
+ uint16_t esw_domain_id;
+ uint16_t item_port_esw_domain_id;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ NULL,
+ "match on port id is valid only"
+ " when transfer flag is enabled");
+ if (item_flags & MLX5_FLOW_ITEM_PORT_ID)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM, item,
+ "multiple source ports are not"
+ " supported");
+ if (!mask)
+ mask = &switch_mask;
+ if (mask->id != 0xffffffff)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ITEM_MASK,
+ mask,
+ "no support for partial mask on"
+ " \"id\" field");
+ ret = mlx5_flow_item_acceptable
+ (item, (const uint8_t *)mask,
+ (const uint8_t *)&rte_flow_item_port_id_mask,
+ sizeof(struct rte_flow_item_port_id),
+ error);
+ if (ret)
+ return ret;
+ if (!spec)
+ return 0;
+ ret = mlx5_port_to_eswitch_info(spec->id, &item_port_esw_domain_id,
+ NULL);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "failed to obtain E-Switch info for"
+ " port");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain E-Switch info");
+ if (item_port_esw_domain_id != esw_domain_id)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ITEM_SPEC, spec,
+ "cannot match on a port from a"
+ " different E-Switch");
+ return 0;
+}
+
+/**
* Validate count action.
*
* @param[in] dev
@@ -676,7 +759,7 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ACTION, NULL,
"can only have a single encap or"
" decap action in a flow");
- if (attr->ingress)
+ if (!attr->transfer && attr->ingress)
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -761,7 +844,8 @@ struct field_modify_info modify_tcp[] = {
"can only have a single encap"
" action in a flow");
/* encap without preceding decap is not supported for ingress */
- if (attr->ingress && !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
+ if (!attr->transfer && attr->ingress &&
+ !(action_flags & MLX5_FLOW_ACTION_RAW_DECAP))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
NULL,
@@ -1561,6 +1645,77 @@ struct field_modify_info modify_tcp[] = {
return 0;
}
+/*
+ * Validate the port_id action.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action_flags
+ * Bit-fields that holds the actions detected until now.
+ * @param[in] action
+ * Port_id RTE action structure.
+ * @param[in] attr
+ * Attributes of flow that includes this action.
+ * @param[out] error
+ * Pointer to error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_validate_action_port_id(struct rte_eth_dev *dev,
+ uint64_t action_flags,
+ const struct rte_flow_action *action,
+ const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ const struct rte_flow_action_port_id *port_id;
+ uint16_t port;
+ uint16_t esw_domain_id;
+ uint16_t act_port_domain_id;
+ int ret;
+
+ if (!attr->transfer)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "port id action is valid in transfer"
+ " mode only");
+ if (!action || !action->conf)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF,
+ NULL,
+ "port id action parameters must be"
+ " specified");
+ if (action_flags & (MLX5_FLOW_FATE_ACTIONS |
+ MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "can have only one fate actions in"
+ " a flow");
+ ret = mlx5_port_to_eswitch_info(dev->data->port_id,
+ &esw_domain_id, NULL);
+ if (ret < 0)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "failed to obtain E-Switch info");
+ port_id = action->conf;
+ port = port_id->original ? dev->data->port_id : port_id->id;
+ ret = mlx5_port_to_eswitch_info(port, &act_port_domain_id, NULL);
+ if (ret)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION_CONF, port_id,
+ "failed to obtain E-Switch port id for port");
+ if (act_port_domain_id != esw_domain_id)
+ return rte_flow_error_set
+ (error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION, NULL,
+ "port does not belong to"
+ " E-Switch being configured");
+ return 0;
+}
/**
* Find existing modify-header resource or create and register a new one.
@@ -1759,11 +1914,29 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
NULL,
"priority out of range");
- if (attributes->transfer)
- return rte_flow_error_set(error, ENOTSUP,
- RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
- NULL,
- "transfer is not supported");
+ if (attributes->transfer) {
+ if (!priv->config.dv_esw_en)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "E-Switch dr is not supported");
+ if (!(priv->representor || priv->master))
+ return rte_flow_error_set
+ (error, EINVAL, RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "E-Switch configurationd can only be"
+ " done by a master or a representor device");
+ if (attributes->egress)
+ return rte_flow_error_set
+ (error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ATTR_EGRESS, attributes,
+ "egress is not supported");
+ if (attributes->group >= MLX5_MAX_TABLES_FDB)
+ return rte_flow_error_set
+ (error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
+ NULL, "group must be smaller than "
+ RTE_STR(MLX5_MAX_FDB_TABLES));
+ }
if (!(attributes->egress ^ attributes->ingress))
return rte_flow_error_set(error, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ATTR, NULL,
@@ -1812,6 +1985,13 @@ struct field_modify_info modify_tcp[] = {
switch (items->type) {
case RTE_FLOW_ITEM_TYPE_VOID:
break;
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ ret = flow_dv_validate_item_port_id
+ (dev, items, attr, item_flags, error);
+ if (ret < 0)
+ return ret;
+ last_item |= MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
ret = mlx5_flow_validate_item_eth(items, item_flags,
error);
@@ -1943,6 +2123,17 @@ struct field_modify_info modify_tcp[] = {
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ ret = flow_dv_validate_action_port_id(dev,
+ action_flags,
+ actions,
+ attr,
+ error);
+ if (ret)
+ return ret;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ ++actions_n;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
ret = mlx5_flow_validate_action_flag(action_flags,
attr, error);
@@ -2133,10 +2324,40 @@ struct field_modify_info modify_tcp[] = {
"action not supported");
}
}
- if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
- return rte_flow_error_set(error, EINVAL,
- RTE_FLOW_ERROR_TYPE_ACTION, actions,
- "no fate action is found");
+ /* Eswitch has few restrictions on using items and actions */
+ if (attr->transfer) {
+ if (action_flags & MLX5_FLOW_ACTION_FLAG)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action FLAG");
+ if (action_flags & MLX5_FLOW_ACTION_MARK)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action MARK");
+ if (action_flags & MLX5_FLOW_ACTION_QUEUE)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action QUEUE");
+ if (action_flags & MLX5_FLOW_ACTION_RSS)
+ return rte_flow_error_set(error, ENOTSUP,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "unsupported action RSS");
+ if (!(action_flags & MLX5_FLOW_FATE_ESWITCH_ACTIONS))
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ } else {
+ if (!(action_flags & MLX5_FLOW_FATE_ACTIONS) && attr->ingress)
+ return rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ actions,
+ "no fate action is found");
+ }
return 0;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 5/9] net/mlx5: add port ID item to Direct Verbs
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (3 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
` (4 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Adds the port ID item to the DV steering code.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 84 ++++++++++++++++++++++++++++++-----------
1 file changed, 61 insertions(+), 23 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b819359..e3d9aa2 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3086,6 +3086,62 @@ struct field_modify_info modify_tcp[] = {
}
}
+/**
+ * Add source vport match to the specified matcher.
+ *
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] port
+ * Source vport value to match
+ * @param[in] mask
+ * Mask
+ */
+static void
+flow_dv_translate_item_source_vport(void *matcher, void *key,
+ int16_t port, uint16_t mask)
+{
+ void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
+ void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
+
+ MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
+ MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
+}
+
+/**
+ * Translate port-id item to eswitch match on port-id.
+ *
+ * @param[in] dev
+ * The devich to configure through.
+ * @param[in, out] matcher
+ * Flow matcher.
+ * @param[in, out] key
+ * Flow matcher value.
+ * @param[in] item
+ * Flow pattern to translate.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise.
+ */
+static int
+flow_dv_translate_item_port_id(struct rte_eth_dev *dev, void *matcher,
+ void *key, const struct rte_flow_item *item)
+{
+ const struct rte_flow_item_port_id *pid_m = item ? item->mask : NULL;
+ const struct rte_flow_item_port_id *pid_v = item ? item->spec : NULL;
+ uint16_t mask, val, id;
+ int ret;
+
+ mask = pid_m ? pid_m->id : 0xffff;
+ id = pid_v ? pid_v->id : dev->data->port_id;
+ ret = mlx5_port_to_eswitch_info(id, NULL, &val);
+ if (ret)
+ return ret;
+ flow_dv_translate_item_source_vport(matcher, key, val, mask);
+ return 0;
+}
+
static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 };
#define HEADER_IS_ZERO(match_criteria, headers) \
@@ -3296,29 +3352,6 @@ struct field_modify_info modify_tcp[] = {
}
/**
- * Add source vport match to the specified matcher.
- *
- * @param[in, out] matcher
- * Flow matcher.
- * @param[in, out] key
- * Flow matcher value.
- * @param[in] port
- * Source vport value to match
- * @param[in] mask
- * Mask
- */
-static void
-flow_dv_translate_item_source_vport(void *matcher, void *key,
- int16_t port, uint16_t mask)
-{
- void *misc_m = MLX5_ADDR_OF(fte_match_param, matcher, misc_parameters);
- void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
-
- MLX5_SET(fte_match_set_misc, misc_m, source_port, mask);
- MLX5_SET(fte_match_set_misc, misc_v, source_port, port);
-}
-
-/**
* Find existing tag resource or create and register a new one.
*
* @param dev[in, out]
@@ -3724,6 +3757,11 @@ struct field_modify_info modify_tcp[] = {
void *match_value = dev_flow->dv.value.buf;
switch (items->type) {
+ case RTE_FLOW_ITEM_TYPE_PORT_ID:
+ flow_dv_translate_item_port_id(dev, match_mask,
+ match_value, items);
+ last_item = MLX5_FLOW_ITEM_PORT_ID;
+ break;
case RTE_FLOW_ITEM_TYPE_ETH:
flow_dv_translate_item_eth(match_mask, match_value,
items, tunnel);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 6/9] net/mlx5: add transfer attribute to matcher
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (4 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
` (3 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
In current implementation the DV steering supported only NIC steering.
This commit adds the transfer attribute in order to create a matcher
on the FDB tabels.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5_flow.c | 1 +
drivers/net/mlx5/mlx5_flow.h | 2 ++
drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++++++----
3 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index b1effda..8afb966 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2095,6 +2095,7 @@ uint32_t mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
flow = rte_calloc(__func__, 1, flow_size, 0);
flow->drv_type = flow_get_drv_type(dev, attr);
flow->ingress = attr->ingress;
+ flow->transfer = attr->transfer;
assert(flow->drv_type > MLX5_FLOW_TYPE_MIN &&
flow->drv_type < MLX5_FLOW_TYPE_MAX);
flow->queue = (void *)(flow + 1);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 85954c2..9d72024 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -210,6 +210,7 @@ struct mlx5_flow_dv_matcher {
uint16_t crc; /**< CRC of key. */
uint16_t priority; /**< Priority of matcher. */
uint8_t egress; /**< Egress matcher. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
uint32_t group; /**< The matcher group. */
struct mlx5_flow_dv_match_params mask; /**< Matcher mask. */
};
@@ -382,6 +383,7 @@ struct rte_flow {
struct mlx5_fdir *fdir; /**< Pointer to associated FDIR if any. */
uint8_t ingress; /**< 1 if the flow is ingress. */
uint32_t group; /**< The group index. */
+ uint8_t transfer; /**< 1 if the flow is E-Switch flow. */
};
typedef int (*mlx5_flow_validate_t)(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index e3d9aa2..182abb3 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -3192,6 +3192,8 @@ struct field_modify_info modify_tcp[] = {
* Table id to use.
* @param[in] egress
* Direction of the table.
+ * @param[in] transfer
+ * E-Switch or NIC flow.
* @param[out] error
* pointer to error structure.
*
@@ -3201,6 +3203,7 @@ struct field_modify_info modify_tcp[] = {
static struct mlx5_flow_tbl_resource *
flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
uint32_t table_id, uint8_t egress,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_priv *priv = dev->data->dev_private;
@@ -3208,7 +3211,12 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow_tbl_resource *tbl;
#ifdef HAVE_MLX5DV_DR
- if (egress) {
+ if (transfer) {
+ tbl = &sh->fdb_tbl[table_id];
+ if (!tbl->obj)
+ tbl->obj = mlx5_glue->dr_create_flow_tbl
+ (sh->fdb_ns, table_id);
+ } else if (egress) {
tbl = &sh->tx_tbl[table_id];
if (!tbl->obj)
tbl->obj = mlx5_glue->dr_create_flow_tbl
@@ -3230,7 +3238,9 @@ struct field_modify_info modify_tcp[] = {
#else
(void)error;
(void)tbl;
- if (egress)
+ if (transfer)
+ return &sh->fdb_tbl[table_id];
+ else if (egress)
return &sh->tx_tbl[table_id];
else
return &sh->rx_tbl[table_id];
@@ -3295,6 +3305,7 @@ struct field_modify_info modify_tcp[] = {
matcher->priority == cache_matcher->priority &&
matcher->egress == cache_matcher->egress &&
matcher->group == cache_matcher->group &&
+ matcher->transfer == cache_matcher->transfer &&
!memcmp((const void *)matcher->mask.buf,
(const void *)cache_matcher->mask.buf,
cache_matcher->mask.size)) {
@@ -3316,7 +3327,8 @@ struct field_modify_info modify_tcp[] = {
RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
"cannot allocate matcher memory");
tbl = flow_dv_tbl_resource_get(dev, matcher->group * MLX5_GROUP_FACTOR,
- matcher->egress, error);
+ matcher->egress, matcher->transfer,
+ error);
if (!tbl) {
rte_free(cache_matcher);
return rte_flow_error_set(error, ENOMEM,
@@ -3643,7 +3655,8 @@ struct field_modify_info modify_tcp[] = {
jump_data = action->conf;
tbl = flow_dv_tbl_resource_get(dev, jump_data->group *
MLX5_GROUP_FACTOR,
- attr->egress, error);
+ attr->egress,
+ attr->transfer, error);
if (!tbl)
return rte_flow_error_set
(error, errno,
@@ -3871,6 +3884,7 @@ struct field_modify_info modify_tcp[] = {
matcher.priority);
matcher.egress = attr->egress;
matcher.group = attr->group;
+ matcher.transfer = attr->transfer;
if (flow_dv_matcher_register(dev, &matcher, dev_flow, error))
return -rte_errno;
return 0;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (5 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 8/9] net/mlx5: add Forward Database table type Ori Kam
` (2 subsequent siblings)
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commits adds matching on source port, using DV API.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5.h | 2 +
drivers/net/mlx5/mlx5_flow.h | 12 ++++
drivers/net/mlx5/mlx5_flow_dv.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/net/mlx5/mlx5_glue.c | 14 ++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 178 insertions(+)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 5e70856..907d5c3 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -282,6 +282,8 @@ struct mlx5_ibv_shared {
LIST_HEAD(modify_cmd, mlx5_flow_dv_modify_hdr_resource) modify_cmds;
LIST_HEAD(tag, mlx5_flow_dv_tag_resource) tags;
LIST_HEAD(jump, mlx5_flow_dv_jump_tbl_resource) jump_tbl;
+ LIST_HEAD(port_id_action_list, mlx5_flow_dv_port_id_action_resource)
+ port_id_action_list; /* List of port ID actions. */
/* Shared interrupt handler section. */
pthread_mutex_t intr_mutex; /* Interrupt config mutex. */
uint32_t intr_cnt; /* Interrupt handler reference counter. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 9d72024..c419e6b 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -267,6 +267,16 @@ struct mlx5_flow_dv_jump_tbl_resource {
struct mlx5_flow_tbl_resource *tbl; /**< The target table. */
};
+/* Port ID resource structure. */
+struct mlx5_flow_dv_port_id_action_resource {
+ LIST_ENTRY(mlx5_flow_dv_port_id_action_resource) next;
+ /* Pointer to next element. */
+ rte_atomic32_t refcnt; /**< Reference counter. */
+ void *action;
+ /**< Verbs tag action object. */
+ uint32_t port_id; /**< Port ID value. */
+};
+
/*
* Max number of actions per DV flow.
* See CREATE_FLOW_MAX_FLOW_ACTIONS_SUPPORTED
@@ -289,6 +299,8 @@ struct mlx5_flow_dv {
struct ibv_flow *flow; /**< Installed flow. */
struct mlx5_flow_dv_jump_tbl_resource *jump;
/**< Pointer to the jump action resource. */
+ struct mlx5_flow_dv_port_id_action_resource *port_id_action;
+ /**< Pointer to port ID action resource. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
void *actions[MLX5_DV_MAX_NUMBER_OF_ACTIONS];
/**< Action list. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 182abb3..9c5826c 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1054,6 +1054,70 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Find existing table port ID resource or create and register a new one.
+ *
+ * @param dev[in, out]
+ * Pointer to rte_eth_dev structure.
+ * @param[in, out] resource
+ * Pointer to port ID action resource.
+ * @parm[in, out] dev_flow
+ * Pointer to the dev_flow.
+ * @param[out] error
+ * pointer to error structure.
+ *
+ * @return
+ * 0 on success otherwise -errno and errno is set.
+ */
+static int
+flow_dv_port_id_action_resource_register
+ (struct rte_eth_dev *dev,
+ struct mlx5_flow_dv_port_id_action_resource *resource,
+ struct mlx5_flow *dev_flow,
+ struct rte_flow_error *error)
+{
+ struct mlx5_priv *priv = dev->data->dev_private;
+ struct mlx5_ibv_shared *sh = priv->sh;
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource;
+
+ /* Lookup a matching resource from cache. */
+ LIST_FOREACH(cache_resource, &sh->port_id_action_list, next) {
+ if (resource->port_id == cache_resource->port_id) {
+ DRV_LOG(DEBUG, "port id action resource resource %p: "
+ "refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ rte_atomic32_inc(&cache_resource->refcnt);
+ dev_flow->dv.port_id_action = cache_resource;
+ return 0;
+ }
+ }
+ /* Register new port id action resource. */
+ cache_resource = rte_calloc(__func__, 1, sizeof(*cache_resource), 0);
+ if (!cache_resource)
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+ "cannot allocate resource memory");
+ *cache_resource = *resource;
+ cache_resource->action =
+ mlx5_glue->dr_create_flow_action_dest_vport(priv->sh->fdb_ns,
+ resource->port_id);
+ if (!cache_resource->action) {
+ rte_free(cache_resource);
+ return rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL, "cannot create action");
+ }
+ rte_atomic32_init(&cache_resource->refcnt);
+ rte_atomic32_inc(&cache_resource->refcnt);
+ LIST_INSERT_HEAD(&sh->port_id_action_list, cache_resource, next);
+ dev_flow->dv.port_id_action = cache_resource;
+ DRV_LOG(DEBUG, "new port id action resource %p: refcnt %d++",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ return 0;
+}
+
+/**
* Get the size of specific rte_flow_item_type
*
* @param[in] item_type
@@ -3456,6 +3520,44 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Translate port ID action to vport.
+ *
+ * @param[in] dev
+ * Pointer to rte_eth_dev structure.
+ * @param[in] action
+ * Pointer to the port ID action.
+ * @param[out] dst_port_id
+ * The target port ID.
+ * @param[out] error
+ * Pointer to the error structure.
+ *
+ * @return
+ * 0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+flow_dv_translate_action_port_id(struct rte_eth_dev *dev,
+ const struct rte_flow_action *action,
+ uint32_t *dst_port_id,
+ struct rte_flow_error *error)
+{
+ uint32_t port;
+ uint16_t port_id;
+ int ret;
+ const struct rte_flow_action_port_id *conf =
+ (const struct rte_flow_action_port_id *)action->conf;
+
+ port = conf->original ? dev->data->port_id : conf->id;
+ ret = mlx5_port_to_eswitch_info(port, NULL, &port_id);
+ if (ret)
+ return rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_ACTION,
+ NULL,
+ "No eswitch info was found for port");
+ *dst_port_id = port_id;
+ return 0;
+}
+
+/**
* Fill the flow with DV spec.
*
* @param[in] dev
@@ -3513,10 +3615,24 @@ struct field_modify_info modify_tcp[] = {
const struct rte_flow_action_jump *jump_data;
struct mlx5_flow_dv_jump_tbl_resource jump_tbl_resource;
struct mlx5_flow_tbl_resource *tbl;
+ uint32_t port_id = 0;
+ struct mlx5_flow_dv_port_id_action_resource port_id_resource;
switch (actions->type) {
case RTE_FLOW_ACTION_TYPE_VOID:
break;
+ case RTE_FLOW_ACTION_TYPE_PORT_ID:
+ if (flow_dv_translate_action_port_id(dev, action,
+ &port_id, error))
+ return -rte_errno;
+ port_id_resource.port_id = port_id;
+ if (flow_dv_port_id_action_resource_register
+ (dev, &port_id_resource, dev_flow, error))
+ return -rte_errno;
+ dev_flow->dv.actions[actions_n++] =
+ dev_flow->dv.port_id_action->action;
+ action_flags |= MLX5_FLOW_ACTION_PORT_ID;
+ break;
case RTE_FLOW_ACTION_TYPE_FLAG:
tag_resource.tag =
mlx5_flow_mark_set(MLX5_FLOW_MARK_DEFAULT);
@@ -4116,6 +4232,37 @@ struct field_modify_info modify_tcp[] = {
}
/**
+ * Release port ID action resource.
+ *
+ * @param flow
+ * Pointer to mlx5_flow.
+ *
+ * @return
+ * 1 while a reference on it exists, 0 when freed.
+ */
+static int
+flow_dv_port_id_action_resource_release(struct mlx5_flow *flow)
+{
+ struct mlx5_flow_dv_port_id_action_resource *cache_resource =
+ flow->dv.port_id_action;
+
+ assert(cache_resource->action);
+ DRV_LOG(DEBUG, "port ID action resource %p: refcnt %d--",
+ (void *)cache_resource,
+ rte_atomic32_read(&cache_resource->refcnt));
+ if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
+ claim_zero(mlx5_glue->destroy_flow_action
+ (cache_resource->action));
+ LIST_REMOVE(cache_resource, next);
+ rte_free(cache_resource);
+ DRV_LOG(DEBUG, "port id action resource %p: removed",
+ (void *)cache_resource);
+ return 0;
+ }
+ return 1;
+}
+
+/**
* Remove the flow from the NIC but keeps it in memory.
*
* @param[in] dev
@@ -4182,6 +4329,8 @@ struct field_modify_info modify_tcp[] = {
flow_dv_modify_hdr_resource_release(dev_flow);
if (dev_flow->dv.jump)
flow_dv_jump_tbl_resource_release(dev_flow);
+ if (dev_flow->dv.port_id_action)
+ flow_dv_port_id_action_resource_release(dev_flow);
rte_free(dev_flow);
}
}
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index a508faa..117190f 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -382,6 +382,18 @@
}
static void *
+mlx5_glue_dr_create_flow_action_dest_vport(void *ns, uint32_t vport)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_dest_vport(ns, vport);
+#else
+ (void)ns;
+ (void)vport;
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -847,6 +859,8 @@
.cq_ex_to_cq = mlx5_glue_cq_ex_to_cq,
.dr_create_flow_action_dest_flow_tbl =
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
+ .dr_create_flow_action_dest_vport =
+ mlx5_glue_dr_create_flow_action_dest_vport,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 058e9b1..26f5cb3 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -146,6 +146,7 @@ struct mlx5_glue {
const char *(*port_state_str)(enum ibv_port_state port_state);
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
+ void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 8/9] net/mlx5: add Forward Database table type
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (6 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
2019-04-18 18:55 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Shahaf Shuler
9 siblings, 0 replies; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
Actions like encap/decap, modify header require setting the flow table
type. Until now we supported only Nic RX and Nic TX, this commits adds
the support for FDB table type for those actions.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5_flow_dv.c | 56 +++++++++++++++++++++++++++++------------
1 file changed, 40 insertions(+), 16 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 9c5826c..5997182 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -40,6 +40,10 @@
#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
#endif
+#ifndef HAVE_MLX5DV_DR_ESWITCH
+#define MLX5DV_FLOW_TABLE_TYPE_FDB 0
+#endif
+
union flow_dv_attr {
struct {
uint32_t valid:1;
@@ -939,7 +943,9 @@ struct field_modify_info modify_tcp[] = {
struct mlx5dv_dr_ns *ns;
resource->flags = flow->group ? 0 : 1;
- if (flow->ingress)
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_RX)
ns = sh->rx_ns;
else
ns = sh->tx_ns;
@@ -1360,6 +1366,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to action structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1370,6 +1378,7 @@ struct field_modify_info modify_tcp[] = {
flow_dv_create_action_l2_encap(struct rte_eth_dev *dev,
const struct rte_flow_action *action,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
const struct rte_flow_item *encap_data;
@@ -1377,7 +1386,8 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_flow_dv_encap_decap_resource res = {
.reformat_type =
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL,
- .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
+ .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_TX,
};
if (action->type == RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
@@ -1412,6 +1422,8 @@ struct field_modify_info modify_tcp[] = {
* Pointer to rte_eth_dev structure.
* @param[in, out] dev_flow
* Pointer to the mlx5_flow.
+ * @param[in] transfer
+ * Mark if the flow is E-Switch flow.
* @param[out] error
* Pointer to the error structure.
*
@@ -1421,13 +1433,15 @@ struct field_modify_info modify_tcp[] = {
static int
flow_dv_create_action_l2_decap(struct rte_eth_dev *dev,
struct mlx5_flow *dev_flow,
+ uint8_t transfer,
struct rte_flow_error *error)
{
struct mlx5_flow_dv_encap_decap_resource res = {
.size = 0,
.reformat_type =
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2,
- .ft_type = MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
+ .ft_type = transfer ? MLX5DV_FLOW_TABLE_TYPE_FDB :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_RX,
};
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
@@ -1470,8 +1484,11 @@ struct field_modify_info modify_tcp[] = {
res.reformat_type = attr->egress ?
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL :
MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
- res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
- MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
+ else
+ res.ft_type = attr->egress ? MLX5DV_FLOW_TABLE_TYPE_NIC_TX :
+ MLX5DV_FLOW_TABLE_TYPE_NIC_RX;
if (flow_dv_encap_decap_resource_register(dev, &res, dev_flow, error))
return rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ACTION,
@@ -1806,11 +1823,14 @@ struct field_modify_info modify_tcp[] = {
struct mlx5_priv *priv = dev->data->dev_private;
struct mlx5_ibv_shared *sh = priv->sh;
struct mlx5_flow_dv_modify_hdr_resource *cache_resource;
+ struct mlx5dv_dr_ns *ns;
- struct mlx5dv_dr_ns *ns =
- resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX ?
- sh->tx_ns : sh->rx_ns;
-
+ if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB)
+ ns = sh->fdb_ns;
+ else if (resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_NIC_TX)
+ ns = sh->tx_ns;
+ else
+ ns = sh->rx_ns;
/* Lookup a matching resource from cache. */
LIST_FOREACH(cache_resource, &sh->modify_cmds, next) {
if (resource->ft_type == cache_resource->ft_type &&
@@ -3604,6 +3624,8 @@ struct field_modify_info modify_tcp[] = {
union flow_dv_attr flow_attr = { .attr = 0 };
struct mlx5_flow_dv_tag_resource tag_resource;
+ if (attr->transfer)
+ res.ft_type = MLX5DV_FLOW_TABLE_TYPE_FDB;
if (priority == MLX5_FLOW_PRIO_RSVD)
priority = priv->config.flow_prio - 1;
for (; !actions_end ; actions++) {
@@ -3709,7 +3731,9 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_ENCAP:
if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow, error))
+ dev_flow,
+ attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3721,6 +3745,7 @@ struct field_modify_info modify_tcp[] = {
case RTE_FLOW_ACTION_TYPE_VXLAN_DECAP:
case RTE_FLOW_ACTION_TYPE_NVGRE_DECAP:
if (flow_dv_create_action_l2_decap(dev, dev_flow,
+ attr->transfer,
error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
@@ -3740,9 +3765,9 @@ struct field_modify_info modify_tcp[] = {
dev_flow->dv.encap_decap->verbs_action;
} else {
/* Handle encap without preceding decap. */
- if (flow_dv_create_action_l2_encap(dev, actions,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_encap
+ (dev, actions, dev_flow, attr->transfer,
+ error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
@@ -3757,9 +3782,8 @@ struct field_modify_info modify_tcp[] = {
}
/* Handle decap only if it isn't followed by encap. */
if (action->type != RTE_FLOW_ACTION_TYPE_RAW_ENCAP) {
- if (flow_dv_create_action_l2_decap(dev,
- dev_flow,
- error))
+ if (flow_dv_create_action_l2_decap
+ (dev, dev_flow, attr->transfer, error))
return -rte_errno;
dev_flow->dv.actions[actions_n++] =
dev_flow->dv.encap_decap->verbs_action;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (7 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 8/9] net/mlx5: add Forward Database table type Ori Kam
@ 2019-04-18 13:16 ` Ori Kam
2019-04-18 13:23 ` Yongseok Koh
2019-04-18 18:55 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Shahaf Shuler
9 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:16 UTC (permalink / raw)
To: yskoh, shahafs, matan, viacheslavo, motih; +Cc: dev, orika
This commit adds support for drop action when creating E-Switch flow
using DV.
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
---
drivers/net/mlx5/mlx5.c | 9 +++++++++
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++--------
drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
drivers/net/mlx5/mlx5_glue.h | 1 +
5 files changed, 37 insertions(+), 8 deletions(-)
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index ff24e1d..65c69af 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
goto error;
}
sh->fdb_ns = ns;
+ sh->esw_drop_action = mlx5_glue->dr_create_flow_action_drop();
}
#endif
sh->dv_refcnt++;
@@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->esw_drop_action) {
+ mlx5_glue->destroy_flow_action(sh->esw_drop_action);
+ sh->esw_drop_action = NULL;
+ }
return err;
#else
(void)priv;
@@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
mlx5_glue->dr_destroy_ns(sh->fdb_ns);
sh->fdb_ns = NULL;
}
+ if (sh->esw_drop_action) {
+ mlx5_glue->destroy_flow_action(sh->esw_drop_action);
+ sh->esw_drop_action = NULL;
+ }
#endif
pthread_mutex_destroy(&sh->dv_mutex);
#else
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 907d5c3..443625e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -276,6 +276,7 @@ struct mlx5_ibv_shared {
/* RX Direct Rules tables. */
void *tx_ns; /* TX Direct Rules name space handle. */
struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
+ void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
/* TX Direct Rules tables/ */
LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 5997182..4352d48 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = {
{
struct mlx5_flow_dv *dv;
struct mlx5_flow *dev_flow;
+ struct mlx5_priv *priv = dev->data->dev_private;
int n;
int err;
@@ -4056,15 +4057,20 @@ struct field_modify_info modify_tcp[] = {
dv = &dev_flow->dv;
n = dv->actions_n;
if (flow->actions & MLX5_FLOW_ACTION_DROP) {
- dv->hrxq = mlx5_hrxq_drop_new(dev);
- if (!dv->hrxq) {
- rte_flow_error_set
- (error, errno,
- RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
- "cannot get drop hash queue");
- goto error;
+ if (flow->transfer)
+ dv->actions[n++] = priv->sh->esw_drop_action;
+ else {
+ dv->hrxq = mlx5_hrxq_drop_new(dev);
+ if (!dv->hrxq) {
+ rte_flow_error_set
+ (error, errno,
+ RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
+ NULL,
+ "cannot get drop hash queue");
+ goto error;
+ }
+ dv->actions[n++] = dv->hrxq->action;
}
- dv->actions[n++] = dv->hrxq->action;
} else if (flow->actions &
(MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
struct mlx5_hrxq *hrxq;
diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
index 117190f..b32cd09c 100644
--- a/drivers/net/mlx5/mlx5_glue.c
+++ b/drivers/net/mlx5/mlx5_glue.c
@@ -394,6 +394,16 @@
}
static void *
+mlx5_glue_dr_create_flow_action_drop(void)
+{
+#ifdef HAVE_MLX5DV_DR_ESWITCH
+ return mlx5dv_dr_create_action_drop();
+#else
+ return NULL;
+#endif
+}
+
+static void *
mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
{
#ifdef HAVE_MLX5DV_DR
@@ -861,6 +871,8 @@
mlx5_glue_dr_create_flow_action_dest_flow_tbl,
.dr_create_flow_action_dest_vport =
mlx5_glue_dr_create_flow_action_dest_vport,
+ .dr_create_flow_action_drop =
+ mlx5_glue_dr_create_flow_action_drop,
.dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
.dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
.dr_create_ns = mlx5_glue_dr_create_ns,
diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
index 26f5cb3..1d06583 100644
--- a/drivers/net/mlx5/mlx5_glue.h
+++ b/drivers/net/mlx5/mlx5_glue.h
@@ -147,6 +147,7 @@ struct mlx5_glue {
struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
+ void *(*dr_create_flow_action_drop)();
void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
int (*dr_destroy_flow_tbl)(void *tbl);
void *(*dr_create_ns)(struct ibv_context *ctx,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
@ 2019-04-18 13:23 ` Yongseok Koh
2019-04-18 13:47 ` Ori Kam
0 siblings, 1 reply; 62+ messages in thread
From: Yongseok Koh @ 2019-04-18 13:23 UTC (permalink / raw)
To: Ori Kam; +Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
> On Apr 18, 2019, at 6:16 AM, Ori Kam <orika@mellanox.com> wrote:
>
> This commit adds support for drop action when creating E-Switch flow
> using DV.
>
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Yongseok Koh <yskoh@mellanox.com>
> ---
> drivers/net/mlx5/mlx5.c | 9 +++++++++
> drivers/net/mlx5/mlx5.h | 1 +
> drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++--------
> drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> drivers/net/mlx5/mlx5_glue.h | 1 +
> 5 files changed, 37 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> index ff24e1d..65c69af 100644
> --- a/drivers/net/mlx5/mlx5.c
> +++ b/drivers/net/mlx5/mlx5.c
> @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> goto error;
> }
> sh->fdb_ns = ns;
> + sh->esw_drop_action = mlx5_glue->dr_create_flow_action_drop();
> }
> #endif
> sh->dv_refcnt++;
> @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->esw_drop_action) {
> + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> + sh->esw_drop_action = NULL;
> + }
> return err;
> #else
> (void)priv;
> @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> sh->fdb_ns = NULL;
> }
> + if (sh->esw_drop_action) {
> + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> + sh->esw_drop_action = NULL;
> + }
> #endif
> pthread_mutex_destroy(&sh->dv_mutex);
> #else
> diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> index 907d5c3..443625e 100644
> --- a/drivers/net/mlx5/mlx5.h
> +++ b/drivers/net/mlx5/mlx5.h
> @@ -276,6 +276,7 @@ struct mlx5_ibv_shared {
> /* RX Direct Rules tables. */
> void *tx_ns; /* TX Direct Rules name space handle. */
> struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> + void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
> /* TX Direct Rules tables/ */
> LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource) encaps_decaps;
> diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
> index 5997182..4352d48 100644
> --- a/drivers/net/mlx5/mlx5_flow_dv.c
> +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> @@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = {
> {
> struct mlx5_flow_dv *dv;
> struct mlx5_flow *dev_flow;
> + struct mlx5_priv *priv = dev->data->dev_private;
> int n;
> int err;
>
> @@ -4056,15 +4057,20 @@ struct field_modify_info modify_tcp[] = {
> dv = &dev_flow->dv;
> n = dv->actions_n;
> if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> - dv->hrxq = mlx5_hrxq_drop_new(dev);
> - if (!dv->hrxq) {
> - rte_flow_error_set
> - (error, errno,
> - RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> - "cannot get drop hash queue");
> - goto error;
> + if (flow->transfer)
> + dv->actions[n++] = priv->sh->esw_drop_action;
> + else {
Please fix the coding style issue.
http://mails.dpdk.org/archives/test-report/2019-April/080992.html
> + dv->hrxq = mlx5_hrxq_drop_new(dev);
> + if (!dv->hrxq) {
> + rte_flow_error_set
> + (error, errno,
> + RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> + NULL,
> + "cannot get drop hash queue");
> + goto error;
> + }
> + dv->actions[n++] = dv->hrxq->action;
> }
> - dv->actions[n++] = dv->hrxq->action;
> } else if (flow->actions &
> (MLX5_FLOW_ACTION_QUEUE | MLX5_FLOW_ACTION_RSS)) {
> struct mlx5_hrxq *hrxq;
> diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> index 117190f..b32cd09c 100644
> --- a/drivers/net/mlx5/mlx5_glue.c
> +++ b/drivers/net/mlx5/mlx5_glue.c
> @@ -394,6 +394,16 @@
> }
>
> static void *
> +mlx5_glue_dr_create_flow_action_drop(void)
> +{
> +#ifdef HAVE_MLX5DV_DR_ESWITCH
> + return mlx5dv_dr_create_action_drop();
> +#else
> + return NULL;
> +#endif
> +}
> +
> +static void *
> mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> {
> #ifdef HAVE_MLX5DV_DR
> @@ -861,6 +871,8 @@
> mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> .dr_create_flow_action_dest_vport =
> mlx5_glue_dr_create_flow_action_dest_vport,
> + .dr_create_flow_action_drop =
> + mlx5_glue_dr_create_flow_action_drop,
> .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> .dr_create_ns = mlx5_glue_dr_create_ns,
> diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> index 26f5cb3..1d06583 100644
> --- a/drivers/net/mlx5/mlx5_glue.h
> +++ b/drivers/net/mlx5/mlx5_glue.h
> @@ -147,6 +147,7 @@ struct mlx5_glue {
> struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> + void *(*dr_create_flow_action_drop)();
> void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> int (*dr_destroy_flow_tbl)(void *tbl);
> void *(*dr_create_ns)(struct ibv_context *ctx,
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 13:23 ` Yongseok Koh
@ 2019-04-18 13:47 ` Ori Kam
2019-04-18 18:14 ` Shahaf Shuler
0 siblings, 1 reply; 62+ messages in thread
From: Ori Kam @ 2019-04-18 13:47 UTC (permalink / raw)
To: Yongseok Koh
Cc: Shahaf Shuler, Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Hi Shahaf,
Can you please fix it when you take it?
Thanks,
Ori
> -----Original Message-----
> From: Yongseok Koh
> Sent: Thursday, April 18, 2019 4:23 PM
> To: Ori Kam <orika@mellanox.com>
> Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Moti
> Haimovsky <motih@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
>
> > On Apr 18, 2019, at 6:16 AM, Ori Kam <orika@mellanox.com> wrote:
> >
> > This commit adds support for drop action when creating E-Switch flow
> > using DV.
> >
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > Acked-by: Yongseok Koh <yskoh@mellanox.com>
> > ---
> > drivers/net/mlx5/mlx5.c | 9 +++++++++
> > drivers/net/mlx5/mlx5.h | 1 +
> > drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++--------
> > drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> > drivers/net/mlx5/mlx5_glue.h | 1 +
> > 5 files changed, 37 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
> > index ff24e1d..65c69af 100644
> > --- a/drivers/net/mlx5/mlx5.c
> > +++ b/drivers/net/mlx5/mlx5.c
> > @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> > goto error;
> > }
> > sh->fdb_ns = ns;
> > + sh->esw_drop_action = mlx5_glue-
> >dr_create_flow_action_drop();
> > }
> > #endif
> > sh->dv_refcnt++;
> > @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > sh->fdb_ns = NULL;
> > }
> > + if (sh->esw_drop_action) {
> > + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> > + sh->esw_drop_action = NULL;
> > + }
> > return err;
> > #else
> > (void)priv;
> > @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > sh->fdb_ns = NULL;
> > }
> > + if (sh->esw_drop_action) {
> > + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> > + sh->esw_drop_action = NULL;
> > + }
> > #endif
> > pthread_mutex_destroy(&sh->dv_mutex);
> > #else
> > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
> > index 907d5c3..443625e 100644
> > --- a/drivers/net/mlx5/mlx5.h
> > +++ b/drivers/net/mlx5/mlx5.h
> > @@ -276,6 +276,7 @@ struct mlx5_ibv_shared {
> > /* RX Direct Rules tables. */
> > void *tx_ns; /* TX Direct Rules name space handle. */
> > struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> > + void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
> > /* TX Direct Rules tables/ */
> > LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> > LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> encaps_decaps;
> > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> b/drivers/net/mlx5/mlx5_flow_dv.c
> > index 5997182..4352d48 100644
> > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > @@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = {
> > {
> > struct mlx5_flow_dv *dv;
> > struct mlx5_flow *dev_flow;
> > + struct mlx5_priv *priv = dev->data->dev_private;
> > int n;
> > int err;
> >
> > @@ -4056,15 +4057,20 @@ struct field_modify_info modify_tcp[] = {
> > dv = &dev_flow->dv;
> > n = dv->actions_n;
> > if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> > - dv->hrxq = mlx5_hrxq_drop_new(dev);
> > - if (!dv->hrxq) {
> > - rte_flow_error_set
> > - (error, errno,
> > -
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > - "cannot get drop hash queue");
> > - goto error;
> > + if (flow->transfer)
> > + dv->actions[n++] = priv->sh->esw_drop_action;
> > + else {
>
> Please fix the coding style issue.
> http://mails.dpdk.org/archives/test-report/2019-April/080992.html
>
> > + dv->hrxq = mlx5_hrxq_drop_new(dev);
> > + if (!dv->hrxq) {
> > + rte_flow_error_set
> > + (error, errno,
> > +
> RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > + NULL,
> > + "cannot get drop hash
> queue");
> > + goto error;
> > + }
> > + dv->actions[n++] = dv->hrxq->action;
> > }
> > - dv->actions[n++] = dv->hrxq->action;
> > } else if (flow->actions &
> > (MLX5_FLOW_ACTION_QUEUE |
> MLX5_FLOW_ACTION_RSS)) {
> > struct mlx5_hrxq *hrxq;
> > diff --git a/drivers/net/mlx5/mlx5_glue.c b/drivers/net/mlx5/mlx5_glue.c
> > index 117190f..b32cd09c 100644
> > --- a/drivers/net/mlx5/mlx5_glue.c
> > +++ b/drivers/net/mlx5/mlx5_glue.c
> > @@ -394,6 +394,16 @@
> > }
> >
> > static void *
> > +mlx5_glue_dr_create_flow_action_drop(void)
> > +{
> > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > + return mlx5dv_dr_create_action_drop();
> > +#else
> > + return NULL;
> > +#endif
> > +}
> > +
> > +static void *
> > mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level)
> > {
> > #ifdef HAVE_MLX5DV_DR
> > @@ -861,6 +871,8 @@
> > mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> > .dr_create_flow_action_dest_vport =
> > mlx5_glue_dr_create_flow_action_dest_vport,
> > + .dr_create_flow_action_drop =
> > + mlx5_glue_dr_create_flow_action_drop,
> > .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> > .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> > .dr_create_ns = mlx5_glue_dr_create_ns,
> > diff --git a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h
> > index 26f5cb3..1d06583 100644
> > --- a/drivers/net/mlx5/mlx5_glue.h
> > +++ b/drivers/net/mlx5/mlx5_glue.h
> > @@ -147,6 +147,7 @@ struct mlx5_glue {
> > struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> > void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> > void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t vport);
> > + void *(*dr_create_flow_action_drop)();
> > void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> > int (*dr_destroy_flow_tbl)(void *tbl);
> > void *(*dr_create_ns)(struct ibv_context *ctx,
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch
2019-04-18 13:47 ` Ori Kam
@ 2019-04-18 18:14 ` Shahaf Shuler
0 siblings, 0 replies; 62+ messages in thread
From: Shahaf Shuler @ 2019-04-18 18:14 UTC (permalink / raw)
To: Ori Kam, Yongseok Koh; +Cc: Matan Azrad, Slava Ovsiienko, Moti Haimovsky, dev
Thursday, April 18, 2019 4:48 PM, Ori Kam:
> Subject: RE: [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-
> Switch
>
> Hi Shahaf,
>
> Can you please fix it when you take it?
Yes took care.
>
> Thanks,
> Ori
>
> > -----Original Message-----
> > From: Yongseok Koh
> > Sent: Thursday, April 18, 2019 4:23 PM
> > To: Ori Kam <orika@mellanox.com>
> > Cc: Shahaf Shuler <shahafs@mellanox.com>; Matan Azrad
> > <matan@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>;
> Moti
> > Haimovsky <motih@mellanox.com>; dev@dpdk.org
> > Subject: Re: [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs
> > E-Switch
> >
> > > On Apr 18, 2019, at 6:16 AM, Ori Kam <orika@mellanox.com> wrote:
> > >
> > > This commit adds support for drop action when creating E-Switch flow
> > > using DV.
> > >
> > > Signed-off-by: Ori Kam <orika@mellanox.com>
> > > Acked-by: Yongseok Koh <yskoh@mellanox.com>
> > > ---
> > > drivers/net/mlx5/mlx5.c | 9 +++++++++
> > > drivers/net/mlx5/mlx5.h | 1 +
> > > drivers/net/mlx5/mlx5_flow_dv.c | 22 ++++++++++++++--------
> > > drivers/net/mlx5/mlx5_glue.c | 12 ++++++++++++
> > > drivers/net/mlx5/mlx5_glue.h | 1 +
> > > 5 files changed, 37 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index
> > > ff24e1d..65c69af 100644
> > > --- a/drivers/net/mlx5/mlx5.c
> > > +++ b/drivers/net/mlx5/mlx5.c
> > > @@ -357,6 +357,7 @@ struct mlx5_dev_spawn_data {
> > > goto error;
> > > }
> > > sh->fdb_ns = ns;
> > > + sh->esw_drop_action = mlx5_glue-
> > >dr_create_flow_action_drop();
> > > }
> > > #endif
> > > sh->dv_refcnt++;
> > > @@ -377,6 +378,10 @@ struct mlx5_dev_spawn_data {
> > > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > > sh->fdb_ns = NULL;
> > > }
> > > + if (sh->esw_drop_action) {
> > > + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> > > + sh->esw_drop_action = NULL;
> > > + }
> > > return err;
> > > #else
> > > (void)priv;
> > > @@ -417,6 +422,10 @@ struct mlx5_dev_spawn_data {
> > > mlx5_glue->dr_destroy_ns(sh->fdb_ns);
> > > sh->fdb_ns = NULL;
> > > }
> > > + if (sh->esw_drop_action) {
> > > + mlx5_glue->destroy_flow_action(sh->esw_drop_action);
> > > + sh->esw_drop_action = NULL;
> > > + }
> > > #endif
> > > pthread_mutex_destroy(&sh->dv_mutex);
> > > #else
> > > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index
> > > 907d5c3..443625e 100644
> > > --- a/drivers/net/mlx5/mlx5.h
> > > +++ b/drivers/net/mlx5/mlx5.h
> > > @@ -276,6 +276,7 @@ struct mlx5_ibv_shared {
> > > /* RX Direct Rules tables. */
> > > void *tx_ns; /* TX Direct Rules name space handle. */
> > > struct mlx5_flow_tbl_resource tx_tbl[MLX5_MAX_TABLES];
> > > + void *esw_drop_action; /* Pointer to DR E-Switch drop action. */
> > > /* TX Direct Rules tables/ */
> > > LIST_HEAD(matchers, mlx5_flow_dv_matcher) matchers;
> > > LIST_HEAD(encap_decap, mlx5_flow_dv_encap_decap_resource)
> > encaps_decaps;
> > > diff --git a/drivers/net/mlx5/mlx5_flow_dv.c
> > b/drivers/net/mlx5/mlx5_flow_dv.c
> > > index 5997182..4352d48 100644
> > > --- a/drivers/net/mlx5/mlx5_flow_dv.c
> > > +++ b/drivers/net/mlx5/mlx5_flow_dv.c
> > > @@ -4049,6 +4049,7 @@ struct field_modify_info modify_tcp[] = { {
> > > struct mlx5_flow_dv *dv;
> > > struct mlx5_flow *dev_flow;
> > > + struct mlx5_priv *priv = dev->data->dev_private;
> > > int n;
> > > int err;
> > >
> > > @@ -4056,15 +4057,20 @@ struct field_modify_info modify_tcp[] = {
> > > dv = &dev_flow->dv;
> > > n = dv->actions_n;
> > > if (flow->actions & MLX5_FLOW_ACTION_DROP) {
> > > - dv->hrxq = mlx5_hrxq_drop_new(dev);
> > > - if (!dv->hrxq) {
> > > - rte_flow_error_set
> > > - (error, errno,
> > > -
> > RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
> > > - "cannot get drop hash queue");
> > > - goto error;
> > > + if (flow->transfer)
> > > + dv->actions[n++] = priv->sh-
> >esw_drop_action;
> > > + else {
> >
> > Please fix the coding style issue.
> > http://mails.dpdk.org/archives/test-report/2019-April/080992.html
> >
> > > + dv->hrxq = mlx5_hrxq_drop_new(dev);
> > > + if (!dv->hrxq) {
> > > + rte_flow_error_set
> > > + (error, errno,
> > > +
> > RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
> > > + NULL,
> > > + "cannot get drop hash
> > queue");
> > > + goto error;
> > > + }
> > > + dv->actions[n++] = dv->hrxq->action;
> > > }
> > > - dv->actions[n++] = dv->hrxq->action;
> > > } else if (flow->actions &
> > > (MLX5_FLOW_ACTION_QUEUE |
> > MLX5_FLOW_ACTION_RSS)) {
> > > struct mlx5_hrxq *hrxq;
> > > diff --git a/drivers/net/mlx5/mlx5_glue.c
> > > b/drivers/net/mlx5/mlx5_glue.c index 117190f..b32cd09c 100644
> > > --- a/drivers/net/mlx5/mlx5_glue.c
> > > +++ b/drivers/net/mlx5/mlx5_glue.c
> > > @@ -394,6 +394,16 @@
> > > }
> > >
> > > static void *
> > > +mlx5_glue_dr_create_flow_action_drop(void)
> > > +{
> > > +#ifdef HAVE_MLX5DV_DR_ESWITCH
> > > + return mlx5dv_dr_create_action_drop(); #else
> > > + return NULL;
> > > +#endif
> > > +}
> > > +
> > > +static void *
> > > mlx5_glue_dr_create_flow_tbl(void *ns, uint32_t level) { #ifdef
> > > HAVE_MLX5DV_DR @@ -861,6 +871,8 @@
> > > mlx5_glue_dr_create_flow_action_dest_flow_tbl,
> > > .dr_create_flow_action_dest_vport =
> > > mlx5_glue_dr_create_flow_action_dest_vport,
> > > + .dr_create_flow_action_drop =
> > > + mlx5_glue_dr_create_flow_action_drop,
> > > .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl,
> > > .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl,
> > > .dr_create_ns = mlx5_glue_dr_create_ns, diff --git
> > > a/drivers/net/mlx5/mlx5_glue.h b/drivers/net/mlx5/mlx5_glue.h index
> > > 26f5cb3..1d06583 100644
> > > --- a/drivers/net/mlx5/mlx5_glue.h
> > > +++ b/drivers/net/mlx5/mlx5_glue.h
> > > @@ -147,6 +147,7 @@ struct mlx5_glue {
> > > struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
> > > void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl);
> > > void *(*dr_create_flow_action_dest_vport)(void *ns, uint32_t
> > > vport);
> > > + void *(*dr_create_flow_action_drop)();
> > > void *(*dr_create_flow_tbl)(void *ns, uint32_t level);
> > > int (*dr_destroy_flow_tbl)(void *tbl);
> > > void *(*dr_create_ns)(struct ibv_context *ctx,
> > > --
> > > 1.8.3.1
> > >
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
` (8 preceding siblings ...)
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
@ 2019-04-18 18:55 ` Shahaf Shuler
9 siblings, 0 replies; 62+ messages in thread
From: Shahaf Shuler @ 2019-04-18 18:55 UTC (permalink / raw)
To: Ori Kam, Yongseok Koh, Matan Azrad, Slava Ovsiienko, Moti Haimovsky
Cc: dev, Ori Kam
Thursday, April 18, 2019 4:16 PM, Ori Kam:
> Subject: [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch
> support
>
> Currently MLX5 PMD supports 3 flow engines:
> Verbs, Direct Verbs and TCF. The first two engines are for Nic steering while
> the TCF is for E-Switch steering.
>
> This series add E-Switch steering support also for the DV engine.
>
> In order to support the new capability there should be support from both the
> RDMA and from the NIC.
>
> v3:
> * Small nit in patch 03
> * Fix bug in patch 09
>
> v2:
> * Address ML comments
Applied to next-net-mlx, thanks.
^ permalink raw reply [flat|nested] 62+ messages in thread
end of thread, other threads:[~2019-04-18 18:55 UTC | newest]
Thread overview: 62+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-14 21:12 [dpdk-dev] [PATCH 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-16 23:47 ` Yongseok Koh
2019-04-14 21:12 ` [dpdk-dev] [PATCH 2/9] net/mlx5: fix menson compilation with Direct Rules Ori Kam
2019-04-17 0:01 ` Yongseok Koh
2019-04-17 0:34 ` Yongseok Koh
2019-04-17 5:18 ` Ori Kam
2019-04-17 5:18 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 3/9] net/mlx5: add Direct Rules configuration support Ori Kam
2019-04-17 1:42 ` Yongseok Koh
2019-04-17 6:19 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
2019-04-17 23:59 ` Yongseok Koh
2019-04-18 4:40 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
2019-04-18 0:19 ` Yongseok Koh
2019-04-18 4:43 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
2019-04-18 0:38 ` Yongseok Koh
2019-04-18 4:57 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 7/9] net/mlx5: add port ID action to Direct Verbs Ori Kam
2019-04-18 0:59 ` Yongseok Koh
2019-04-18 5:06 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 8/9] net/mlx5: add Forward Database table type Ori Kam
2019-04-18 1:16 ` Yongseok Koh
2019-04-18 5:13 ` Ori Kam
2019-04-14 21:12 ` [dpdk-dev] [PATCH 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
2019-04-18 1:28 ` Yongseok Koh
2019-04-18 5:15 ` Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-18 12:06 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
2019-04-18 12:09 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
2019-04-18 12:11 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
2019-04-18 12:16 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
2019-04-18 12:17 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
2019-04-18 12:19 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
2019-04-18 12:19 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 8/9] net/mlx5: add Forward Database table type Ori Kam
2019-04-18 12:21 ` Yongseok Koh
2019-04-18 11:28 ` [dpdk-dev] [PATCH v2 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
2019-04-18 12:28 ` Yongseok Koh
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Ori Kam
2019-04-18 13:15 ` [dpdk-dev] [PATCH v3 1/9] net/mlx5: fix translate vport function name Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 2/9] net/mlx5: fix meson build for Direct Rules Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 3/9] net/mlx5: add Direct Rules E-Switch support Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 4/9] net/mlx5: add validation for Direct Rule E-Switch Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 5/9] net/mlx5: add port ID item to Direct Verbs Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 6/9] net/mlx5: add transfer attribute to matcher Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 7/9] net/mlx5: add E-Switch port ID action to Direct Verbs Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 8/9] net/mlx5: add Forward Database table type Ori Kam
2019-04-18 13:16 ` [dpdk-dev] [PATCH v3 9/9] net/mlx5: add drop action to Direct Verbs E-Switch Ori Kam
2019-04-18 13:23 ` Yongseok Koh
2019-04-18 13:47 ` Ori Kam
2019-04-18 18:14 ` Shahaf Shuler
2019-04-18 18:55 ` [dpdk-dev] [PATCH v3 0/9] net/mlx5: add Direct Verbs E-Switch support Shahaf Shuler
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.