* [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor
@ 2019-08-29 2:34 Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
` (8 more replies)
0 siblings, 9 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev, Leyi Rong
This patchset enable Rx flexible descriptor for ice PMD
in both normal path and vector path.
Depends on shared code update patchset.
Haiyue Wang (3):
net/ice: add Rx flex descriptor definition
net/ice: handle the Rx flex descriptor
net/ice: add protocol extraction support for per Rx queue
Leyi Rong (1):
net/ice: switch to Rx flexible descriptor in AVX path
Wenzhuo Lu (2):
net/ice: support more ptype
net/ice: switch to flexible descriptor in SSE path
config/common_base | 1 +
doc/guides/nics/ice.rst | 107 ++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 330 ++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 348 +++++++++++++++-----------
drivers/net/ice/ice_rxtx.h | 6 +
drivers/net/ice/ice_rxtx_vec_avx2.c | 232 ++++++++---------
drivers/net/ice/ice_rxtx_vec_common.h | 5 +
drivers/net/ice/ice_rxtx_vec_sse.c | 243 +++++++++---------
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 156 ++++++++++++
12 files changed, 1054 insertions(+), 383 deletions(-)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
--
2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
` (7 subsequent siblings)
8 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The Rx flex descriptor has 16B and 32B size, with different
field definitions compared to legacy type.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index e9214110c..64e891875 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,8 +21,10 @@
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
#define ice_rx_desc ice_16byte_rx_desc
+#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
#define ice_rx_desc ice_32byte_rx_desc
+#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
#define ICE_SUPPORT_CHAIN_NUM 5
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 2/6] net/ice: handle the Rx flex descriptor
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
` (6 subsequent siblings)
8 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Set the RXDID with flex descriptor type by default, change the rx
function to support new descriptor.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 262 +++++++++++++++++--------------------
1 file changed, 121 insertions(+), 141 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 81af81441..3b4ccf151 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,7 +13,6 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
-#define ICE_RX_ERR_BITS 0x3f
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -25,18 +24,9 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
enum ice_status err;
uint16_t buf_size, len;
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ uint32_t rxdid = ICE_RXDID_COMMS_GENERIC;
uint32_t regval;
- /**
- * The kernel driver uses flex descriptor. It sets the register
- * to flex descriptor mode.
- * DPDK uses legacy descriptor. It should set the register back
- * to the default value, then uses legacy descriptor mode.
- */
- regval = (0x01 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
- QRXFLXP_CNTXT_RXDID_PRIO_M;
- ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
-
/* Set buffer size as the head split is disabled. */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
RTE_PKTMBUF_HEADROOM);
@@ -94,6 +84,21 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile ID;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
+
err = ice_clear_rxq_ctx(hw, rxq->reg_idx);
if (err) {
PMD_DRV_LOG(ERR, "Failed to clear Lan Rx queue (%u) context",
@@ -127,7 +132,6 @@ static int
ice_alloc_rx_queue_mbufs(struct ice_rx_queue *rxq)
{
struct ice_rx_entry *rxe = rxq->sw_ring;
- uint64_t dma_addr;
uint16_t i;
for (i = 0; i < rxq->nb_rx_desc; i++) {
@@ -145,11 +149,9 @@ ice_alloc_rx_queue_mbufs(struct ice_rx_queue *rxq)
mbuf->nb_segs = 1;
mbuf->port = rxq->port_id;
- dma_addr =
- rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
-
rxd = &rxq->rx_ring[i];
- rxd->read.pkt_addr = dma_addr;
+ rxd->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
rxd->read.hdr_addr = 0;
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
rxd->read.rsvd1 = 0;
@@ -961,16 +963,15 @@ uint32_t
ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
#define ICE_RXQ_SCAN_INTERVAL 4
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq;
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
- ((rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S) &
- (1 << ICE_RX_DESC_STATUS_DD_S)) {
+ rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
/**
* Check the DD bit of a rx descriptor of each 4 in a group,
* to avoid checking too frequently and downgrading performance
@@ -979,79 +980,77 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = (volatile union ice_rx_flex_desc *)
+ &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
return desc;
}
-/* Translate the rx descriptor status to pkt flags */
-static inline uint64_t
-ice_rxd_status_to_pkt_flags(uint64_t qword)
-{
- uint64_t flags;
-
- /* Check if RSS_HASH */
- flags = (((qword >> ICE_RX_DESC_STATUS_FLTSTAT_S) &
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ==
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ? PKT_RX_RSS_HASH : 0;
-
- return flags;
-}
+#define ICE_RX_FLEX_ERR0_BITS \
+ ((1 << ICE_RX_FLEX_DESC_STATUS0_HBO_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_RXE_S))
/* Rx L3/L4 checksum */
static inline uint64_t
-ice_rxd_error_to_pkt_flags(uint64_t qword)
+ice_rxd_error_to_pkt_flags(uint16_t stat_err0)
{
uint64_t flags = 0;
- uint64_t error_bits = (qword >> ICE_RXD_QW1_ERROR_S);
- if (likely((error_bits & ICE_RX_ERR_BITS) == 0)) {
+ /* check if HW has decoded the packet and checksum */
+ if (unlikely(!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return 0;
+
+ if (likely(!(stat_err0 & ICE_RX_FLEX_ERR0_BITS))) {
flags |= (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD);
return flags;
}
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_IPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
flags |= PKT_RX_IP_CKSUM_BAD;
else
flags |= PKT_RX_IP_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_L4E_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
flags |= PKT_RX_L4_CKSUM_BAD;
else
flags |= PKT_RX_L4_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_EIPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
flags |= PKT_RX_EIP_CKSUM_BAD;
return flags;
}
static inline void
-ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
+ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
{
- if (rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- (1 << ICE_RX_DESC_STATUS_L2TAG1P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S)) {
mb->ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED;
mb->vlan_tci =
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1);
+ rte_le_to_cpu_16(rxdp->wb.l2tag1);
PMD_RX_LOG(DEBUG, "Descriptor l2tag1: %u",
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1));
+ rte_le_to_cpu_16(rxdp->wb.l2tag1));
} else {
mb->vlan_tci = 0;
}
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
- if (rte_le_to_cpu_16(rxdp->wb.qword2.ext_status) &
- (1 << ICE_RX_DESC_EXT_STATUS_L2TAG2P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error1) &
+ (1 << ICE_RX_FLEX_DESC_STATUS1_L2TAG2P_S)) {
mb->ol_flags |= PKT_RX_QINQ_STRIPPED | PKT_RX_QINQ |
PKT_RX_VLAN_STRIPPED | PKT_RX_VLAN;
mb->vlan_tci_outer = mb->vlan_tci;
- mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2);
+ mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd);
PMD_RX_LOG(DEBUG, "Descriptor l2tag2_1: %u, l2tag2_2: %u",
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_1),
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2));
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_1st),
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd));
} else {
mb->vlan_tci_outer = 0;
}
@@ -1060,6 +1059,21 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+static inline void
+ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
+ volatile union ice_rx_flex_desc *rxdp)
+{
+ volatile struct ice_32b_rx_flex_desc_comms *desc =
+ (volatile struct ice_32b_rx_flex_desc_comms *)rxdp;
+ uint16_t stat_err;
+
+ stat_err = rte_le_to_cpu_16(desc->status_error0);
+ if (likely(stat_err & (1 << ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S))) {
+ mb->ol_flags |= PKT_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
+ }
+}
+
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
#define ICE_LOOK_AHEAD 8
#if (ICE_LOOK_AHEAD != 8)
@@ -1068,25 +1082,23 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
static inline int
ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
+ uint16_t stat_err0;
uint16_t pkt_len;
- uint64_t qword1;
- uint32_t rx_status;
int32_t s[ICE_LOOK_AHEAD], nb_dd;
int32_t i, j, nb_rx = 0;
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S;
+ stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Make sure there is at least 1 packet to receive */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/**
@@ -1096,42 +1108,31 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
for (i = 0; i < ICE_RX_MAX_BURST; i += ICE_LOOK_AHEAD,
rxdp += ICE_LOOK_AHEAD, rxep += ICE_LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--) {
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- s[j] = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
- }
+ for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--)
+ s[j] = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
rte_smp_rmb();
/* Compute how many status bits were set */
for (j = 0, nb_dd = 0; j < ICE_LOOK_AHEAD; j++)
- nb_dd += s[j] & (1 << ICE_RX_DESC_STATUS_DD_S);
+ nb_dd += s[j] & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S);
nb_rx += nb_dd;
/* Translate descriptor info to mbuf parameters */
for (j = 0; j < nb_dd; j++) {
mb = rxep[j].mbuf;
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- pkt_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ pkt_len = (rte_le_to_cpu_16(rxdp[j].wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
mb->data_len = pkt_len;
mb->pkt_len = pkt_len;
mb->ol_flags = 0;
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- mb->hash.rss =
- rte_le_to_cpu_32(
- rxdp[j].wb.qword0.hi_dword.rss);
- mb->packet_type = ptype_tbl[(uint8_t)(
- (qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
+ stat_err0 = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
+ pkt_flags = ice_rxd_error_to_pkt_flags(stat_err0);
+ mb->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxdp[j].wb.ptype_flex_flags0)];
ice_rxd_to_vlan_tci(mb, &rxdp[j]);
+ ice_rxd_to_pkt_fields(mb, &rxdp[j]);
mb->ol_flags |= pkt_flags;
}
@@ -1180,7 +1181,6 @@ ice_rx_alloc_bufs(struct ice_rx_queue *rxq)
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
uint16_t alloc_idx, i;
- uint64_t dma_addr;
int diag;
/* Allocate buffers in bulk */
@@ -1206,9 +1206,10 @@ ice_rx_alloc_bufs(struct ice_rx_queue *rxq)
mb->data_off = RTE_PKTMBUF_HEADROOM;
mb->nb_segs = 1;
mb->port = rxq->port_id;
- dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(mb));
+
+ rxdp[i].read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mb));
rxdp[i].read.hdr_addr = 0;
- rxdp[i].read.pkt_addr = dma_addr;
}
/* Update rx tail regsiter */
@@ -1312,8 +1313,8 @@ ice_recv_scattered_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *first_seg = rxq->pkt_first_seg;
@@ -1324,21 +1325,17 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
- uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint16_t rx_stat_err0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1371,20 +1368,16 @@ ice_recv_scattered_pkts(void *rx_queue,
rxm = rxe->mbuf;
rxe->mbuf = nmb;
- dma_addr =
- rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
/* Set data buffer address and data length of the mbuf */
+ rxdp->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
rxdp->read.hdr_addr = 0;
- rxdp->read.pkt_addr = dma_addr;
- rx_packet_len = (qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S;
+
+ rx_packet_len = rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
rxm->data_len = rx_packet_len;
rxm->data_off = RTE_PKTMBUF_HEADROOM;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
/**
* If this is the first buffer of the received packet, set the
@@ -1410,7 +1403,7 @@ ice_recv_scattered_pkts(void *rx_queue,
* update the pointer to the last mbuf of the current scattered
* packet and continue to parse the RX ring.
*/
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_EOF_S))) {
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_EOF_S))) {
last_seg = rxm;
continue;
}
@@ -1442,13 +1435,11 @@ ice_recv_scattered_pkts(void *rx_queue,
first_seg->port = rxq->port_id;
first_seg->ol_flags = 0;
-
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- first_seg->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
-
+ first_seg->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(first_seg, &rxd);
+ ice_rxd_to_pkt_fields(first_seg, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
first_seg->ol_flags |= pkt_flags;
/* Prefetch data of first segment, if configured to do so. */
rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr,
@@ -1538,9 +1529,8 @@ ice_dev_supported_ptypes_get(struct rte_eth_dev *dev)
int
ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq = rx_queue;
- volatile uint64_t *status;
- uint64_t mask;
uint32_t desc;
if (unlikely(offset >= rxq->nb_rx_desc))
@@ -1553,10 +1543,9 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- status = &rxq->rx_ring[desc].wb.qword1.status_error_len;
- mask = rte_cpu_to_le_64((1ULL << ICE_RX_DESC_STATUS_DD_S) <<
- ICE_RXD_QW1_STATUS_S);
- if (*status & mask)
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
return RTE_ETH_RX_DESC_AVAIL;
@@ -1642,8 +1631,8 @@ ice_recv_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *nmb; /* new allocated mbuf */
@@ -1652,21 +1641,17 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
- uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint16_t rx_stat_err0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1685,19 +1670,18 @@ ice_recv_pkts(void *rx_queue,
rx_id = 0;
rxm = rxe->mbuf;
rxe->mbuf = nmb;
- dma_addr =
- rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
/**
* fill the read format of descriptor with physic address in
* new allocated mbuf: nmb
*/
+ rxdp->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
rxdp->read.hdr_addr = 0;
- rxdp->read.pkt_addr = dma_addr;
/* calculate rx_packet_len of the received pkt */
- rx_packet_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ rx_packet_len = (rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
/* fill old mbuf with received descriptor: rxd */
rxm->data_off = RTE_PKTMBUF_HEADROOM;
@@ -1707,15 +1691,11 @@ ice_recv_pkts(void *rx_queue,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- rxm->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
+ rxm->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(rxm, &rxd);
+ ice_rxd_to_pkt_fields(rxm, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
rxm->ol_flags |= pkt_flags;
/* copy old mbuf to rx_pkts */
rx_pkts[nb_rx++] = rxm;
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 3/6] net/ice: add protocol extraction support for per Rx queue
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 4/6] net/ice: support more ptype Leyi Rong
` (5 subsequent siblings)
8 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The ice has the feature to extract protocol fields into flex descriptor
by programming per queue. Currently, the ice PMD will put the protocol
fields into rte_mbuf::udata64 with different type format. Application
can access the protocol fields quickly.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
config/common_base | 1 +
doc/guides/nics/ice.rst | 107 +++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 330 ++++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 75 ++++++
drivers/net/ice/ice_rxtx.h | 4 +
drivers/net/ice/ice_rxtx_vec_common.h | 5 +
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 156 ++++++++++++
10 files changed, 687 insertions(+)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
diff --git a/config/common_base b/config/common_base
index 8ef75c203..9c876fdbe 100644
--- a/config/common_base
+++ b/config/common_base
@@ -324,6 +324,7 @@ CONFIG_RTE_LIBRTE_ICE_DEBUG_TX=n
CONFIG_RTE_LIBRTE_ICE_DEBUG_TX_FREE=n
CONFIG_RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC=y
CONFIG_RTE_LIBRTE_ICE_16BYTE_RX_DESC=n
+CONFIG_RTE_LIBRTE_ICE_PROTO_XTR=y
# Compile burst-oriented IAVF PMD driver
#
diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
index 03819d29f..185601f21 100644
--- a/doc/guides/nics/ice.rst
+++ b/doc/guides/nics/ice.rst
@@ -46,6 +46,12 @@ Please note that enabling debugging options may affect system performance.
Toggle to use a 16-byte RX descriptor, by default the RX descriptor is 32 byte.
+- ``CONFIG_RTE_LIBRTE_ICE_PROTO_XTR`` (default ``y``)
+
+ Toggle to enable ``Protocol extraction for per queue``, it can't be enabled
+ with ``CONFIG_RTE_LIBRTE_ICE_16BYTE_RX_DESC`` at the same time. And it will
+ modify the ``rte_mbuf::udata64``.
+
Runtime Config Options
~~~~~~~~~~~~~~~~~~~~~~
@@ -61,6 +67,107 @@ Runtime Config Options
NOTE: In Safe mode, only very limited features are available, features like RSS,
checksum, fdir, tunneling ... are all disabled.
+- ``Protocol extraction for per queue``
+
+ Configure the RX queues to do protocol extraction into ``rte_mbuf::udata64``
+ for protocol handling acceleration, like checking the TCP SYN packets quickly.
+
+ The argument format is::
+
+ -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
+ -w 18:00.0,proto_xtr=<protocol>
+
+ Queues are grouped by ``(`` and ``)`` within the group. The ``-`` character
+ is used as a range separator and ``,`` is used as a single number separator.
+ The grouping ``()`` can be omitted for single element group. If no queues are
+ specified, PMD will use this protocol extraction type for all queues.
+
+ Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-13 are
+ VLAN extraction, other queues run with no protocol extraction.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-23 are
+ IPv6 extraction, other queues use the default VLAN extraction.
+
+ The extraction will be copied into the lower 32 bit of ``rte_mbuf::udata64``.
+
+ .. table:: Protocol extraction : ``vlan``
+
+ +----------------------------+----------------------------+
+ | VLAN2 | VLAN1 |
+ +======+===+=================+======+===+=================+
+ | PCP | D | VID | PCP | D | VID |
+ +------+---+-----------------+------+---+-----------------+
+
+ VLAN1 - single or EVLAN (first for QinQ).
+
+ VLAN2 - C-VLAN (second for QinQ).
+
+ .. table:: Protocol extraction : ``ipv4``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +======+=======+=============+==============+=============+
+ | Ver |Hdr Len| ToS | TTL | Protocol |
+ +------+-------+-------------+--------------+-------------+
+
+ IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
+
+ IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
+
+ .. table:: Protocol extraction : ``ipv6``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+=============+==============+
+ | Ver |Traffic class| Flow | Next Header | Hop Limit |
+ +-----+-------------+--------+-------------+--------------+
+
+ IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``ipv6_flow``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+============================+
+ | Ver |Traffic class| Flow Label |
+ +-----+-------------+-------------------------------------+
+
+ IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``tcp``
+
+ +----------------------------+----------------------------+
+ | TCPHDR2 | TCPHDR1 |
+ +============================+======+======+==============+
+ | Reserved |Offset| RSV | Flags |
+ +----------------------------+------+------+--------------+
+
+ TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
+
+ TCPHDR2 - Reserved
+
+ Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the protocol
+ extraction, do not use ``rte_mbuf::udata64`` directly.
+
+ The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
+ access the protocol extraction result in ``struct rte_mbuf``.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile
index ae53c2646..4a279f196 100644
--- a/drivers/net/ice/Makefile
+++ b/drivers/net/ice/Makefile
@@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1)
endif
SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
+
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 63997fdfb..68ab5a2bf 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -19,9 +19,13 @@
/* devargs */
#define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
+#define ICE_PROTO_XTR_ARG "proto_xtr"
static const char * const ice_valid_args[] = {
ICE_SAFE_MODE_SUPPORT_ARG,
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ ICE_PROTO_XTR_ARG,
+#endif
NULL
};
@@ -257,6 +261,307 @@ ice_init_controlq_parameter(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ;
}
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+
+static int
+lookup_proto_xtr_type(const char *xtr_name)
+{
+ static struct {
+ const char *name;
+ enum proto_xtr_type type;
+ } xtr_type_map[] = {
+ { "vlan", PROTO_XTR_VLAN },
+ { "ipv4", PROTO_XTR_IPV4 },
+ { "ipv6", PROTO_XTR_IPV6 },
+ { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
+ { "tcp", PROTO_XTR_TCP },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
+ if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
+ return xtr_type_map[i].type;
+ }
+
+ return -1;
+}
+
+/*
+ * Parse elem, the elem could be single number/range or '(' ')' group
+ * 1) A single number elem, it's just a simple digit. e.g. 9
+ * 2) A single range elem, two digits with a '-' between. e.g. 2-6
+ * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
+ * Within group elem, '-' used for a range separator;
+ * ',' used for a single number.
+ */
+static int
+parse_queue_set(const char *input, int xtr_type, struct ice_pf *pf)
+{
+ const char *str = input;
+ char *end = NULL;
+ uint32_t min, max;
+ uint32_t idx;
+
+ while (isblank(*str))
+ str++;
+
+ if ((!isdigit(*str) && *str != '(') || (*str == '\0'))
+ return -1;
+
+ /* process single number or single range of number */
+ if (*str != '(') {
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ while (isblank(*end))
+ end++;
+
+ min = idx;
+ max = idx;
+
+ /* process single <number>-<number> */
+ if (*end == '-') {
+ end++;
+ while (isblank(*end))
+ end++;
+ if (!isdigit(*end))
+ return -1;
+
+ errno = 0;
+ idx = strtoul(end, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ max = idx;
+ while (isblank(*end))
+ end++;
+ }
+
+ if (*end != ':')
+ return -1;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ pf->proto_xtr_table[idx] = xtr_type;
+
+ return 0;
+ }
+
+ /* process set within bracket */
+ str++;
+ while (isblank(*str))
+ str++;
+ if (*str == '\0')
+ return -1;
+
+ min = pf->lan_nb_qps;
+ do {
+ /* go ahead to the first digit */
+ while (isblank(*str))
+ str++;
+ if (!isdigit(*str))
+ return -1;
+
+ /* get the digit value */
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ /* go ahead to separator '-',',' and ')' */
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ if (min == pf->lan_nb_qps)
+ min = idx;
+ else /* avoid continuous '-' */
+ return -1;
+ } else if (*end == ',' || *end == ')') {
+ max = idx;
+ if (min == pf->lan_nb_qps)
+ min = idx;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ pf->proto_xtr_table[idx] = xtr_type;
+
+ min = pf->lan_nb_qps;
+ } else {
+ return -1;
+ }
+
+ str = end + 1;
+ } while (*end != ')' && *end != '\0');
+
+ return 0;
+}
+
+static int
+parse_queue_proto_xtr(const char *queues, struct ice_pf *pf)
+{
+ const char *queue_start;
+ uint32_t idx;
+ int xtr_type;
+ char xtr_name[32];
+
+ while (isblank(*queues))
+ queues++;
+
+ if (*queues != '[') {
+ xtr_type = lookup_proto_xtr_type(queues);
+ if (xtr_type < 0)
+ return -1;
+
+ pf->proto_xtr = xtr_type;
+ return 0;
+ }
+
+ if (pf->proto_xtr_table == NULL) {
+ PMD_DRV_LOG(ERR,
+ "Can't support protocol extraction table for no memory");
+ return -1;
+ }
+
+ queues++;
+ do {
+ while (isblank(*queues))
+ queues++;
+ if (*queues == '\0')
+ return -1;
+
+ queue_start = queues;
+
+ /* go across a complete bracket */
+ if (*queue_start == '(') {
+ queues += strcspn(queues, ")");
+ if (*queues != ')')
+ return -1;
+ }
+
+ /* scan the separator ':' */
+ queues += strcspn(queues, ":");
+ if (*queues++ != ':')
+ return -1;
+ while (isblank(*queues))
+ queues++;
+
+ for (idx = 0; ; idx++) {
+ if (isblank(queues[idx]) ||
+ queues[idx] == ',' ||
+ queues[idx] == ']' ||
+ queues[idx] == '\0')
+ break;
+
+ if (idx > sizeof(xtr_name) - 2)
+ return -1;
+
+ xtr_name[idx] = queues[idx];
+ }
+ xtr_name[idx] = '\0';
+ xtr_type = lookup_proto_xtr_type(xtr_name);
+ if (xtr_type < 0)
+ return -1;
+
+ queues += idx;
+
+ while (isblank(*queues) || *queues == ',' || *queues == ']')
+ queues++;
+
+ if (parse_queue_set(queue_start, xtr_type, pf) < 0)
+ return -1;
+ } while (*queues != '\0');
+
+ return 0;
+}
+
+static int
+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
+ void *extra_args)
+{
+ struct ice_pf *pf = extra_args;
+
+ if (value == NULL || extra_args == NULL)
+ return -EINVAL;
+
+ if (parse_queue_proto_xtr(value, pf) < 0) {
+ PMD_DRV_LOG(ERR,
+ "The protocol extraction parameter is wrong : '%s'",
+ value);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+ice_parse_proto_xtr_devarg(struct rte_devargs *devargs, struct ice_pf *pf)
+{
+ struct rte_kvargs *kvlist;
+
+ pf->proto_xtr = PROTO_XTR_NONE;
+ if (devargs == NULL)
+ return;
+
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL)
+ return;
+
+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
+ handle_proto_xtr_arg, pf);
+
+ rte_kvargs_free(kvlist);
+}
+
+static bool
+ice_proto_xtr_support(struct ice_hw *hw)
+{
+#define FLX_REG(val, fld, idx) \
+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
+ static struct {
+ uint32_t rxdid;
+ uint16_t protid_0;
+ uint16_t protid_1;
+ } xtr_sets[] = {
+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
+ ICE_PROT_IPV4_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
+ uint32_t rxdid = xtr_sets[i].rxdid;
+ uint32_t v;
+
+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+
+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+ }
+
+ return true;
+}
+
+#endif
+
static int
ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
uint32_t num)
@@ -1088,6 +1393,21 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
pf->lan_nb_qps = pf->lan_nb_qp_max;
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ if (ice_proto_xtr_support(hw)) {
+ pf->proto_xtr_table = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
+ if (pf->proto_xtr_table == NULL)
+ PMD_DRV_LOG(ERR, "No memory for protocol extraction table");
+
+ ice_parse_proto_xtr_devarg(dev->device->devargs, pf);
+ } else {
+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
+
+ pf->proto_xtr = PROTO_XTR_NONE;
+ pf->proto_xtr_table = NULL;
+ }
+#endif
+
return 0;
}
@@ -1556,6 +1876,9 @@ ice_dev_init(struct rte_eth_dev *dev)
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ rte_free(pf->proto_xtr_table);
+#endif
return ret;
}
@@ -1681,6 +2004,10 @@ ice_dev_close(struct rte_eth_dev *dev)
rte_free(hw->port_info);
hw->port_info = NULL;
ice_shutdown_all_ctrlq(hw);
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ rte_free(pf->proto_xtr_table);
+ pf->proto_xtr_table = NULL;
+#endif
}
static int
@@ -3807,6 +4134,9 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ice,
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
+#endif
ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
RTE_INIT(ice_init_log)
diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
index c4444f903..19fe277e3 100644
--- a/drivers/net/ice/ice_ethdev.h
+++ b/drivers/net/ice/ice_ethdev.h
@@ -263,6 +263,10 @@ struct ice_pf {
uint16_t lan_nb_qp_max;
uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
uint16_t base_queue; /* The base queue pairs index in the device */
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
+ uint8_t *proto_xtr_table; /* Per queue for lan_nb_qps size */
+#endif
struct ice_hw_port_stats stats_offset;
struct ice_hw_port_stats stats;
/* internal packet statistics, it should be excluded from the total */
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 3b4ccf151..69913b545 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,6 +13,40 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+
+static inline uint8_t
+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
+{
+ static uint8_t xtr_map[] = {
+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
+ };
+
+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
+}
+
+static inline uint8_t
+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
+{
+ static uint8_t rxdid_map[] = {
+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
+ };
+ uint8_t rxdid;
+
+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
+
+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
+}
+
+#endif
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -84,6 +118,13 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
+
+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
+ rxq->port_id, rxq->queue_id, rxdid);
+#endif
+
/* Enable Flexible Descriptors in the queue context which
* allows this driver to select a specific receive descriptor format
*/
@@ -639,6 +680,13 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
rxq->vsi = vsi;
rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ rxq->proto_xtr = pf->proto_xtr_table != NULL ?
+ pf->proto_xtr_table[queue_idx] : PROTO_XTR_NONE;
+ if (rxq->proto_xtr == PROTO_XTR_NONE)
+ rxq->proto_xtr = pf->proto_xtr;
+#endif
+
/* Allocate the maximun number of RX ring hardware descriptor. */
len = ICE_MAX_RING_DESC;
@@ -1059,6 +1107,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+#define ICE_RX_PROTO_XTR_VALID \
+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+
static inline void
ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
volatile union ice_rx_flex_desc *rxdp)
@@ -1072,6 +1124,29 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
mb->ol_flags |= PKT_RX_RSS_HASH;
mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
}
+
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
+#error "RTE_LIBRTE_ICE_16BYTE_RX_DESC must be disabled"
+#endif
+ init_proto_xtr_flds(mb);
+
+ stat_err = rte_le_to_cpu_16(desc->status_error1);
+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
+ xtr->u.raw.data0 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+ xtr->u.raw.data1 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
+
+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
+ xtr->magic = PROTO_XTR_MAGIC_ID;
+ }
+#endif
}
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index 64e891875..6f4726854 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -5,6 +5,7 @@
#ifndef _ICE_RXTX_H_
#define _ICE_RXTX_H_
+#include "rte_pmd_ice.h"
#include "ice_ethdev.h"
#define ICE_ALIGN_RING_DESC 32
@@ -78,6 +79,9 @@ struct ice_rx_queue {
uint16_t max_pkt_len; /* Maximum packet length */
bool q_set; /* indicate if rx queue has been configured */
bool rx_deferred_start; /* don't start this queue in dev start */
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
+#endif
ice_rx_release_mbufs_t rx_rel_mbufs;
};
diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
index c5f0d564f..ebdd714f8 100644
--- a/drivers/net/ice/ice_rxtx_vec_common.h
+++ b/drivers/net/ice/ice_rxtx_vec_common.h
@@ -234,6 +234,11 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
if (rxq->nb_rx_desc % rxq->rx_free_thresh)
return -1;
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+ if (rxq->proto_xtr != PROTO_XTR_NONE)
+ return -1;
+#endif
+
return 0;
}
diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
index 36b4b3c85..6828170a9 100644
--- a/drivers/net/ice/meson.build
+++ b/drivers/net/ice/meson.build
@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
endif
endif
+
+install_headers('rte_pmd_ice.h')
diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
new file mode 100644
index 000000000..2bd73dd3d
--- /dev/null
+++ b/drivers/net/ice/rte_pmd_ice.h
@@ -0,0 +1,156 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#ifndef _RTE_PMD_ICE_H_
+#define _RTE_PMD_ICE_H_
+
+#include <stdio.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum proto_xtr_type {
+ PROTO_XTR_NONE,
+ PROTO_XTR_VLAN,
+ PROTO_XTR_IPV4,
+ PROTO_XTR_IPV6,
+ PROTO_XTR_IPV6_FLOW,
+ PROTO_XTR_TCP,
+};
+
+struct proto_xtr_flds {
+ union {
+ struct {
+ uint16_t data0;
+ uint16_t data1;
+ } raw;
+ struct {
+ uint16_t stag_vid:12,
+ stag_dei:1,
+ stag_pcp:3;
+ uint16_t ctag_vid:12,
+ ctag_dei:1,
+ ctag_pcp:3;
+ } vlan;
+ struct {
+ uint16_t protocol:8,
+ ttl:8;
+ uint16_t tos:8,
+ ihl:4,
+ version:4;
+ } ipv4;
+ struct {
+ uint16_t hoplimit:8,
+ nexthdr:8;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6;
+ struct {
+ uint16_t flowlo16;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6_flow;
+ struct {
+ uint16_t fin:1,
+ syn:1,
+ rst:1,
+ psh:1,
+ ack:1,
+ urg:1,
+ ece:1,
+ cwr:1,
+ res1:4,
+ doff:4;
+ uint16_t rsvd;
+ } tcp;
+ } u;
+
+ uint16_t rsvd;
+
+ uint8_t type;
+
+#define PROTO_XTR_MAGIC_ID 0xCE
+ uint8_t magic;
+};
+
+#ifdef RTE_LIBRTE_ICE_PROTO_XTR
+
+static inline void
+init_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ mb->udata64 = 0;
+}
+
+static inline struct proto_xtr_flds *
+get_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
+
+ return (struct proto_xtr_flds *)&mb->udata64;
+}
+
+static inline void
+dump_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
+ return;
+
+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
+ xtr->u.raw.data0, xtr->u.raw.data1);
+
+ if (xtr->type == PROTO_XTR_VLAN)
+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
+ xtr->u.vlan.stag_pcp,
+ xtr->u.vlan.stag_dei,
+ xtr->u.vlan.stag_vid,
+ xtr->u.vlan.ctag_pcp,
+ xtr->u.vlan.ctag_dei,
+ xtr->u.vlan.ctag_vid);
+ else if (xtr->type == PROTO_XTR_IPV4)
+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
+ xtr->u.ipv4.version,
+ xtr->u.ipv4.ihl,
+ xtr->u.ipv4.tos,
+ xtr->u.ipv4.ttl,
+ xtr->u.ipv4.protocol);
+ else if (xtr->type == PROTO_XTR_IPV6)
+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
+ xtr->u.ipv6.version,
+ xtr->u.ipv6.tc,
+ xtr->u.ipv6.flowhi4,
+ xtr->u.ipv6.nexthdr,
+ xtr->u.ipv6.hoplimit);
+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
+ xtr->u.ipv6_flow.version,
+ xtr->u.ipv6_flow.tc,
+ xtr->u.ipv6_flow.flowhi4,
+ xtr->u.ipv6_flow.flowlo16);
+ else if (xtr->type == PROTO_XTR_TCP)
+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
+ xtr->u.tcp.doff,
+ xtr->u.tcp.cwr ? "C" : "",
+ xtr->u.tcp.ece ? "E" : "",
+ xtr->u.tcp.urg ? "U" : "",
+ xtr->u.tcp.ack ? "A" : "",
+ xtr->u.tcp.psh ? "P" : "",
+ xtr->u.tcp.rst ? "R" : "",
+ xtr->u.tcp.syn ? "S" : "",
+ xtr->u.tcp.fin ? "F" : "");
+}
+
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_PMD_ICE_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 4/6] net/ice: support more ptype
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (2 preceding siblings ...)
2019-08-29 2:34 ` [dpdk-dev] [PATCH 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
` (4 subsequent siblings)
8 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
More protocol types are supported by HW.
Add them.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 69913b545..b815ade0f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -3121,6 +3121,17 @@ ice_get_default_pkt_type(uint16_t ptype)
RTE_PTYPE_TUNNEL_GTPU,
[267] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN |
RTE_PTYPE_TUNNEL_GTPU,
+ /* [268] - [271] reserved */
+ /* Some protocols are not supported by API, Like, VRRP, OSPF.
+ * Just report them as L2 or L3 packets.
+ */
+ [272] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ [273] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ [274] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ [275] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ [276] = RTE_PTYPE_L2_ETHER,
+ /* [277] reserved */
+ [278] = RTE_PTYPE_L2_ETHER,
/* All others reserved */
};
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 5/6] net/ice: switch to flexible descriptor in SSE path
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (3 preceding siblings ...)
2019-08-29 2:34 ` [dpdk-dev] [PATCH 4/6] net/ice: support more ptype Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
` (3 subsequent siblings)
8 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
With this path, the flexible descriptor is supported
in SSE path. And the legacy descriptor is not supported.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx_vec_sse.c | 243 ++++++++++++++---------------
1 file changed, 115 insertions(+), 128 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index 967a7b16b..aea00ecd0 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -15,14 +15,14 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
struct rte_mbuf *mb0, *mb1;
__m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -88,93 +88,96 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer);
__m128i rearm0, rearm1, rearm2, rearm3;
- __m128i vlan0, vlan1, rss, l3_l4e;
+ __m128i tmp_desc, flags, rss, vlan;
- /* mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication.
+ /* mask everything except checksum, RSS and VLAN flags.
+ * bit6:4 for checksum.
+ * bit12 for RSS indication.
+ * bit13 for VLAN indication.
*/
- const __m128i rss_vlan_msk = _mm_set_epi32(0x1c03804, 0x1c03804,
- 0x1c03804, 0x1c03804);
+ const __m128i desc_mask = _mm_set_epi32(0x3070, 0x3070,
+ 0x3070, 0x3070);
- const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD);
- /* map rss and vlan type to rss hash and vlan flag */
- const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- 0, 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
- 0, 0, 0, 0);
-
- const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH, 0, 0,
- 0, 0, PKT_RX_FDIR, 0);
-
- const __m128i l3_l4e_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ /* map the checksum, rss and vlan fields to the checksum, rss
+ * and vlan flag
+ */
+ const __m128i cksum_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
- vlan0 = _mm_unpackhi_epi32(descs[0], descs[1]);
- vlan1 = _mm_unpackhi_epi32(descs[2], descs[3]);
- vlan0 = _mm_unpacklo_epi64(vlan0, vlan1);
-
- vlan1 = _mm_and_si128(vlan0, rss_vlan_msk);
- vlan0 = _mm_shuffle_epi8(vlan_flags, vlan1);
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
- rss = _mm_srli_epi32(vlan1, 11);
- rss = _mm_shuffle_epi8(rss_flags, rss);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0);
- l3_l4e = _mm_srli_epi32(vlan1, 22);
- l3_l4e = _mm_shuffle_epi8(l3_l4e_flags, l3_l4e);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0);
+
+ /* merge 4 descriptors */
+ flags = _mm_unpackhi_epi32(descs[0], descs[1]);
+ tmp_desc = _mm_unpackhi_epi32(descs[2], descs[3]);
+ tmp_desc = _mm_unpacklo_epi64(flags, tmp_desc);
+ tmp_desc = _mm_and_si128(flags, desc_mask);
+
+ /* checksum flags */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 4);
+ flags = _mm_shuffle_epi8(cksum_flags, tmp_desc);
/* then we shift left 1 bit */
- l3_l4e = _mm_slli_epi32(l3_l4e, 1);
- /* we need to mask out the reduntant bits */
- l3_l4e = _mm_and_si128(l3_l4e, cksum_mask);
+ flags = _mm_slli_epi32(flags, 1);
+ /* we need to mask out the reduntant bits introduced by RSS or
+ * VLAN fields.
+ */
+ flags = _mm_and_si128(flags, cksum_mask);
+
+ /* RSS, VLAN flag */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 8);
+ rss = _mm_shuffle_epi8(rss_flags, tmp_desc);
+ vlan = _mm_shuffle_epi8(vlan_flags, tmp_desc);
- vlan0 = _mm_or_si128(vlan0, rss);
- vlan0 = _mm_or_si128(vlan0, l3_l4e);
+ /* merge the flags */
+ flags = _mm_or_si128(flags, rss);
+ flags = _mm_or_si128(flags, vlan);
/**
* At this point, we have the 4 sets of flags in the low 16-bits
- * of each 32-bit value in vlan0.
+ * of each 32-bit value in flags.
* We want to extract these, and merge them with the mbuf init data
* so we can do a single 16-byte write to the mbuf to set the flags
* and all the other initialization fields. Extracting the
* appropriate flags means that we have to do a shift and blend for
* each mbuf before we do the write.
*/
- rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 8), 0x10);
- rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 4), 0x10);
- rearm2 = _mm_blend_epi16(mbuf_init, vlan0, 0x10);
- rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10);
+ rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x10);
+ rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x10);
+ rearm2 = _mm_blend_epi16(mbuf_init, flags, 0x10);
+ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x10);
/* write the rearm data and the olflags in one write */
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
@@ -187,22 +190,24 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
_mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3);
}
-#define PKTLEN_SHIFT 10
-
static inline void
ice_rx_desc_to_ptype_v(__m128i descs[4], struct rte_mbuf **rx_pkts,
uint32_t *ptype_tbl)
{
- __m128i ptype0 = _mm_unpackhi_epi64(descs[0], descs[1]);
- __m128i ptype1 = _mm_unpackhi_epi64(descs[2], descs[3]);
-
- ptype0 = _mm_srli_epi64(ptype0, 30);
- ptype1 = _mm_srli_epi64(ptype1, 30);
-
- rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 0)];
- rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 8)];
- rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 0)];
- rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 8)];
+ const __m128i ptype_mask = _mm_set_epi16(0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M);
+ __m128i ptype_01 = _mm_unpacklo_epi32(descs[0], descs[1]);
+ __m128i ptype_23 = _mm_unpacklo_epi32(descs[2], descs[3]);
+ __m128i ptype_all = _mm_unpacklo_epi64(ptype_01, ptype_23);
+
+ ptype_all = _mm_and_si128(ptype_all, ptype_mask);
+
+ rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
}
/**
@@ -215,21 +220,39 @@ static inline uint16_t
_ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
uint64_t var;
- __m128i shuf_msk;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
-
__m128i crc_adjust = _mm_set_epi16
- (0, 0, 0, /* ignore non-length fields */
+ (0, 0, 0, /* ignore non-length fields */
-rxq->crc_len, /* sub crc on data_len */
0, /* ignore high-16bits of pkt_len */
-rxq->crc_len, /* sub crc on pkt_len */
- 0, 0 /* ignore pkt_type field */
+ 0, 0 /* ignore pkt_type field */
);
+ const __m128i zero = _mm_setzero_si128();
+ /* mask to shuffle from desc. to mbuf */
+ const __m128i shuf_msk = _mm_set_epi8
+ (0xFF, 0xFF, 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, low 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /* pkt_type set as unknown */
+ );
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+
/**
* compile-time check the above crc_adjust layout is correct.
* NOTE: the first field (lowest address) is given last in set_epi16
@@ -239,7 +262,13 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
- __m128i dd_check, eop_check;
+
+ /* 4 packets DD mask */
+ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ /* 4 packets EOP mask */
+ const __m128i eop_check = _mm_set_epi64x(0x0000000200000002LL,
+ 0x0000000200000002LL);
/* nb_pkts shall be less equal than ICE_MAX_RX_BURST */
nb_pkts = RTE_MIN(nb_pkts, ICE_MAX_RX_BURST);
@@ -250,7 +279,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = rxq->rx_ring + rxq->rx_tail;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
@@ -263,26 +292,10 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
- /* 4 packets DD mask */
- dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL);
-
- /* 4 packets EOP mask */
- eop_check = _mm_set_epi64x(0x0000000200000002LL, 0x0000000200000002LL);
-
- /* mask to shuffle from desc. to mbuf */
- shuf_msk = _mm_set_epi8
- (7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
- );
/**
* Compile-time verify the shuffle mask
* NOTE: some field positions already verified above, but duplicated
@@ -315,7 +328,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
rxdp += ICE_DESCS_PER_LOOP) {
__m128i descs[ICE_DESCS_PER_LOOP];
__m128i pkt_mb1, pkt_mb2, pkt_mb3, pkt_mb4;
- __m128i zero, staterr, sterr_tmp1, sterr_tmp2;
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
/* 2 64 bit or 4 32 bit mbuf pointers in one XMM reg. */
__m128i mbp1;
#if defined(RTE_ARCH_X86_64)
@@ -359,14 +372,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* avoid compiler reorder optimization */
rte_compiler_barrier();
- /* pkt 3,4 shift the pktlen field to be 16-bit aligned*/
- const __m128i len3 = _mm_slli_epi32(descs[3], PKTLEN_SHIFT);
- const __m128i len2 = _mm_slli_epi32(descs[2], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[3] = _mm_blend_epi16(descs[3], len3, 0x80);
- descs[2] = _mm_blend_epi16(descs[2], len2, 0x80);
-
/* D.1 pkt 3,4 convert format from desc to pktmbuf */
pkt_mb4 = _mm_shuffle_epi8(descs[3], shuf_msk);
pkt_mb3 = _mm_shuffle_epi8(descs[2], shuf_msk);
@@ -382,20 +387,11 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
pkt_mb4 = _mm_add_epi16(pkt_mb4, crc_adjust);
pkt_mb3 = _mm_add_epi16(pkt_mb3, crc_adjust);
- /* pkt 1,2 shift the pktlen field to be 16-bit aligned*/
- const __m128i len1 = _mm_slli_epi32(descs[1], PKTLEN_SHIFT);
- const __m128i len0 = _mm_slli_epi32(descs[0], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[1] = _mm_blend_epi16(descs[1], len1, 0x80);
- descs[0] = _mm_blend_epi16(descs[0], len0, 0x80);
-
/* D.1 pkt 1,2 convert format from desc to pktmbuf */
pkt_mb2 = _mm_shuffle_epi8(descs[1], shuf_msk);
pkt_mb1 = _mm_shuffle_epi8(descs[0], shuf_msk);
/* C.2 get 4 pkts staterr value */
- zero = _mm_xor_si128(dd_check, dd_check);
staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
/* D.3 copy final 3,4 data to rx_pkts */
@@ -412,15 +408,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* C* extract and record EOP bit */
if (split_packet) {
- __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0x04, 0x0C,
- 0x00, 0x08);
-
/* and with mask to extract bits, flipping 1-0 */
__m128i eop_bits = _mm_andnot_si128(staterr, eop_check);
/* the staterr values are not in order, as the count
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (4 preceding siblings ...)
2019-08-29 2:34 ` [dpdk-dev] [PATCH 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
@ 2019-08-29 2:34 ` Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
` (2 subsequent siblings)
8 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 2:34 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev, Leyi Rong
Switch to Rx flexible descriptor format instead of legacy
descriptor format.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
---
drivers/net/ice/ice_rxtx_vec_avx2.c | 232 ++++++++++++++--------------
1 file changed, 118 insertions(+), 114 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 5ce29c2a2..158f17d80 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -132,8 +132,6 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
ICE_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
}
-#define PKTLEN_SHIFT 10
-
static inline uint16_t
_ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
@@ -144,7 +142,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp =
+ (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
@@ -161,8 +160,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/* constants used in processing loop */
@@ -193,21 +192,23 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i shuf_msk =
_mm256_set_epi8
(/* first descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF, /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF, /*pkt_type set as unknown */
/* second descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /*pkt_type set as unknown */
);
/**
* compile-time check the above crc and shuffle layout is correct.
@@ -225,68 +226,76 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Status/Error flag masks */
/**
- * mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication. Bits 3-5 of error
- * field (bits 22-24) are for IP/L4 checksum errors
+ * mask everything except Checksum Reports, RSS indication
+ * and VLAN indication.
+ * bit6:4 for IP/L4 checksum errors.
+ * bit12 is for RSS indication.
+ * bit13 is for VLAN indication.
*/
const __m256i flags_mask =
- _mm256_set1_epi32((1 << 2) | (1 << 11) |
- (3 << 12) | (7 << 22));
- /**
- * data to be shuffled by result of flag mask. If VLAN bit is set,
- * (bit 2), then position 4 in this array will be used in the
- * destination
- */
- const __m256i vlan_flags_shuf =
- _mm256_set_epi32(0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0,
- 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0);
- /**
- * data to be shuffled by result of flag mask, shifted down 11.
- * If RSS/FDIR bits are set, shuffle moves appropriate flags in
- * place.
- */
- const __m256i rss_flags_shuf =
- _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0,/* end up 128-bits */
- 0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0);
-
+ _mm256_set1_epi32((7 << 4) | (1 << 12) | (1 << 13));
/**
- * data to be shuffled by the result of the flags mask shifted by 22
+ * data to be shuffled by the result of the flags mask shifted by 4
* bits. This gives use the l3_l4 flags.
*/
const __m256i l3_l4_flags_shuf = _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1,
/* second 128-bits */
0, 0, 0, 0, 0, 0, 0, 0,
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
const __m256i cksum_mask =
_mm256_set1_epi32(PKT_RX_IP_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD |
PKT_RX_L4_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_EIP_CKSUM_BAD);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If RSS bit is set, shuffle moves appropriate flags in place.
+ */
+ const __m256i rss_flags_shuf =
+ _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If VLAN bit is set(bit 13), shuffle moves appropriate flags in place.
+ */
+ const __m256i vlan_flags_shuf =
+ _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0);
RTE_SET_USED(avx_aligned); /* for 32B descriptors we don't use this */
@@ -369,73 +378,66 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
}
/**
- * convert descriptors 4-7 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 4-7 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len6_7 = _mm256_slli_epi32(raw_desc6_7,
- PKTLEN_SHIFT);
- const __m256i len4_5 = _mm256_slli_epi32(raw_desc4_5,
- PKTLEN_SHIFT);
- const __m256i desc6_7 = _mm256_blend_epi16(raw_desc6_7,
- len6_7, 0x80);
- const __m256i desc4_5 = _mm256_blend_epi16(raw_desc4_5,
- len4_5, 0x80);
- __m256i mb6_7 = _mm256_shuffle_epi8(desc6_7, shuf_msk);
- __m256i mb4_5 = _mm256_shuffle_epi8(desc4_5, shuf_msk);
+ __m256i mb6_7 = _mm256_shuffle_epi8(raw_desc6_7, shuf_msk);
+ __m256i mb4_5 = _mm256_shuffle_epi8(raw_desc4_5, shuf_msk);
mb6_7 = _mm256_add_epi16(mb6_7, crc_adjust);
mb4_5 = _mm256_add_epi16(mb4_5, crc_adjust);
/**
- * to get packet types, shift 64-bit values down 30 bits
- * and so ptype is in lower 8-bits in each
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
*/
- const __m256i ptypes6_7 = _mm256_srli_epi64(desc6_7, 30);
- const __m256i ptypes4_5 = _mm256_srli_epi64(desc4_5, 30);
- const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 24);
- const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 8);
- const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 24);
- const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 8);
+ const __m256i ptype_mask =
+ _mm256_set1_epi16(ICE_RX_FLEX_DESC_PTYPE_M);
+ const __m256i ptypes6_7 =
+ _mm256_and_si256(raw_desc6_7, ptype_mask);
+ const __m256i ptypes4_5 =
+ _mm256_and_si256(raw_desc4_5, ptype_mask);
+ const uint16_t ptype7 = _mm256_extract_epi16(ptypes6_7, 9);
+ const uint16_t ptype6 = _mm256_extract_epi16(ptypes6_7, 1);
+ const uint16_t ptype5 = _mm256_extract_epi16(ptypes4_5, 9);
+ const uint16_t ptype4 = _mm256_extract_epi16(ptypes4_5, 1);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype7], 4);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype6], 0);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype5], 4);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype4], 0);
/* merge the status bits into one register */
- const __m256i status4_7 = _mm256_unpackhi_epi32(desc6_7,
- desc4_5);
+ const __m256i status4_7 = _mm256_unpackhi_epi32(raw_desc6_7,
+ raw_desc4_5);
/**
- * convert descriptors 0-3 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 0-3 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len2_3 = _mm256_slli_epi32(raw_desc2_3,
- PKTLEN_SHIFT);
- const __m256i len0_1 = _mm256_slli_epi32(raw_desc0_1,
- PKTLEN_SHIFT);
- const __m256i desc2_3 = _mm256_blend_epi16(raw_desc2_3,
- len2_3, 0x80);
- const __m256i desc0_1 = _mm256_blend_epi16(raw_desc0_1,
- len0_1, 0x80);
- __m256i mb2_3 = _mm256_shuffle_epi8(desc2_3, shuf_msk);
- __m256i mb0_1 = _mm256_shuffle_epi8(desc0_1, shuf_msk);
+ __m256i mb2_3 = _mm256_shuffle_epi8(raw_desc2_3, shuf_msk);
+ __m256i mb0_1 = _mm256_shuffle_epi8(raw_desc0_1, shuf_msk);
mb2_3 = _mm256_add_epi16(mb2_3, crc_adjust);
mb0_1 = _mm256_add_epi16(mb0_1, crc_adjust);
- /* get the packet types */
- const __m256i ptypes2_3 = _mm256_srli_epi64(desc2_3, 30);
- const __m256i ptypes0_1 = _mm256_srli_epi64(desc0_1, 30);
- const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 24);
- const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 8);
- const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 24);
- const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 8);
+ /**
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
+ */
+ const __m256i ptypes2_3 =
+ _mm256_and_si256(raw_desc2_3, ptype_mask);
+ const __m256i ptypes0_1 =
+ _mm256_and_si256(raw_desc0_1, ptype_mask);
+ const uint16_t ptype3 = _mm256_extract_epi16(ptypes2_3, 9);
+ const uint16_t ptype2 = _mm256_extract_epi16(ptypes2_3, 1);
+ const uint16_t ptype1 = _mm256_extract_epi16(ptypes0_1, 9);
+ const uint16_t ptype0 = _mm256_extract_epi16(ptypes0_1, 1);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype3], 4);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype2], 0);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype1], 4);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype0], 0);
/* merge the status bits into one register */
- const __m256i status0_3 = _mm256_unpackhi_epi32(desc2_3,
- desc0_1);
+ const __m256i status0_3 = _mm256_unpackhi_epi32(raw_desc2_3,
+ raw_desc0_1);
/**
* take the two sets of status bits and merge to one
@@ -450,20 +452,22 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* get only flag/error bits we want */
const __m256i flag_bits =
_mm256_and_si256(status0_7, flags_mask);
- /* set vlan and rss flags */
- const __m256i vlan_flags =
- _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
- const __m256i rss_flags =
- _mm256_shuffle_epi8(rss_flags_shuf,
- _mm256_srli_epi32(flag_bits, 11));
/**
* l3_l4_error flags, shuffle, then shift to correct adjustment
* of flags in flags_shuf, and finally mask out extra bits
*/
__m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
- _mm256_srli_epi32(flag_bits, 22));
+ _mm256_srli_epi32(flag_bits, 4));
l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
+ /* set rss and vlan flags */
+ const __m256i rss_vlan_flag_bits =
+ _mm256_srli_epi32(flag_bits, 12);
+ const __m256i rss_flags =
+ _mm256_shuffle_epi8(rss_flags_shuf, rss_vlan_flag_bits);
+ const __m256i vlan_flags =
+ _mm256_shuffle_epi8(vlan_flags_shuf,
+ rss_vlan_flag_bits);
/* merge flags */
const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor
2019-08-29 2:34 ` [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
` (5 more replies)
0 siblings, 6 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev, Leyi Rong
This patchset enable Rx flexible descriptor for ice PMD
in both normal path and vector path.
Depends on shared code update patchset.
---
v2:
Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
when handling Rx packets.
Haiyue Wang (3):
net/ice: add Rx flex descriptor definition
net/ice: handle the Rx flex descriptor
net/ice: add protocol extraction support for per Rx queue
Leyi Rong (1):
net/ice: switch to Rx flexible descriptor in AVX path
Wenzhuo Lu (2):
net/ice: support more ptype
net/ice: switch to flexible descriptor in SSE path
doc/guides/nics/ice.rst | 101 ++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 316 ++++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 2 +
drivers/net/ice/ice_rxtx.c | 310 +++++++++++++++----------
drivers/net/ice/ice_rxtx.h | 4 +
drivers/net/ice/ice_rxtx_vec_avx2.c | 232 +++++++++----------
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/ice_rxtx_vec_sse.c | 243 ++++++++++----------
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
11 files changed, 1001 insertions(+), 367 deletions(-)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
--
2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 1/6] net/ice: add Rx flex descriptor definition
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
` (4 subsequent siblings)
5 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The Rx flex descriptor has 16B and 32B size, with different
field definitions compared to legacy type.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index e9214110c..64e891875 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,8 +21,10 @@
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
#define ice_rx_desc ice_16byte_rx_desc
+#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
#define ice_rx_desc ice_32byte_rx_desc
+#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
#define ICE_SUPPORT_CHAIN_NUM 5
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 2/6] net/ice: handle the Rx flex descriptor
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
` (3 subsequent siblings)
5 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Set the RXDID with flex descriptor type by default, change the Rx
function to support new descriptor handling.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 236 +++++++++++++++++--------------------
1 file changed, 111 insertions(+), 125 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 0282b5375..d2e36853f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,7 +13,6 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
-#define ICE_RX_ERR_BITS 0x3f
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -25,18 +24,9 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
enum ice_status err;
uint16_t buf_size, len;
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ uint32_t rxdid = ICE_RXDID_COMMS_GENERIC;
uint32_t regval;
- /**
- * The kernel driver uses flex descriptor. It sets the register
- * to flex descriptor mode.
- * DPDK uses legacy descriptor. It should set the register back
- * to the default value, then uses legacy descriptor mode.
- */
- regval = (0x01 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
- QRXFLXP_CNTXT_RXDID_PRIO_M;
- ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
-
/* Set buffer size as the head split is disabled. */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
RTE_PKTMBUF_HEADROOM);
@@ -94,6 +84,21 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile ID;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
+
err = ice_clear_rxq_ctx(hw, rxq->reg_idx);
if (err) {
PMD_DRV_LOG(ERR, "Failed to clear Lan Rx queue (%u) context",
@@ -961,16 +966,15 @@ uint32_t
ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
#define ICE_RXQ_SCAN_INTERVAL 4
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq;
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
- ((rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S) &
- (1 << ICE_RX_DESC_STATUS_DD_S)) {
+ rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
/**
* Check the DD bit of a rx descriptor of each 4 in a group,
* to avoid checking too frequently and downgrading performance
@@ -979,79 +983,77 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = (volatile union ice_rx_flex_desc *)
+ &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
return desc;
}
-/* Translate the rx descriptor status to pkt flags */
-static inline uint64_t
-ice_rxd_status_to_pkt_flags(uint64_t qword)
-{
- uint64_t flags;
-
- /* Check if RSS_HASH */
- flags = (((qword >> ICE_RX_DESC_STATUS_FLTSTAT_S) &
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ==
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ? PKT_RX_RSS_HASH : 0;
-
- return flags;
-}
+#define ICE_RX_FLEX_ERR0_BITS \
+ ((1 << ICE_RX_FLEX_DESC_STATUS0_HBO_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_RXE_S))
/* Rx L3/L4 checksum */
static inline uint64_t
-ice_rxd_error_to_pkt_flags(uint64_t qword)
+ice_rxd_error_to_pkt_flags(uint16_t stat_err0)
{
uint64_t flags = 0;
- uint64_t error_bits = (qword >> ICE_RXD_QW1_ERROR_S);
- if (likely((error_bits & ICE_RX_ERR_BITS) == 0)) {
+ /* check if HW has decoded the packet and checksum */
+ if (unlikely(!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return 0;
+
+ if (likely(!(stat_err0 & ICE_RX_FLEX_ERR0_BITS))) {
flags |= (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD);
return flags;
}
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_IPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
flags |= PKT_RX_IP_CKSUM_BAD;
else
flags |= PKT_RX_IP_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_L4E_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
flags |= PKT_RX_L4_CKSUM_BAD;
else
flags |= PKT_RX_L4_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_EIPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
flags |= PKT_RX_EIP_CKSUM_BAD;
return flags;
}
static inline void
-ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
+ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
{
- if (rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- (1 << ICE_RX_DESC_STATUS_L2TAG1P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S)) {
mb->ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED;
mb->vlan_tci =
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1);
+ rte_le_to_cpu_16(rxdp->wb.l2tag1);
PMD_RX_LOG(DEBUG, "Descriptor l2tag1: %u",
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1));
+ rte_le_to_cpu_16(rxdp->wb.l2tag1));
} else {
mb->vlan_tci = 0;
}
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
- if (rte_le_to_cpu_16(rxdp->wb.qword2.ext_status) &
- (1 << ICE_RX_DESC_EXT_STATUS_L2TAG2P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error1) &
+ (1 << ICE_RX_FLEX_DESC_STATUS1_L2TAG2P_S)) {
mb->ol_flags |= PKT_RX_QINQ_STRIPPED | PKT_RX_QINQ |
PKT_RX_VLAN_STRIPPED | PKT_RX_VLAN;
mb->vlan_tci_outer = mb->vlan_tci;
- mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2);
+ mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd);
PMD_RX_LOG(DEBUG, "Descriptor l2tag2_1: %u, l2tag2_2: %u",
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_1),
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2));
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_1st),
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd));
} else {
mb->vlan_tci_outer = 0;
}
@@ -1060,6 +1062,21 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+static inline void
+ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
+ volatile union ice_rx_flex_desc *rxdp)
+{
+ volatile struct ice_32b_rx_flex_desc_comms *desc =
+ (volatile struct ice_32b_rx_flex_desc_comms *)rxdp;
+ uint16_t stat_err;
+
+ stat_err = rte_le_to_cpu_16(desc->status_error0);
+ if (likely(stat_err & (1 << ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S))) {
+ mb->ol_flags |= PKT_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
+ }
+}
+
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
#define ICE_LOOK_AHEAD 8
#if (ICE_LOOK_AHEAD != 8)
@@ -1068,25 +1085,23 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
static inline int
ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
+ uint16_t stat_err0;
uint16_t pkt_len;
- uint64_t qword1;
- uint32_t rx_status;
int32_t s[ICE_LOOK_AHEAD], nb_dd;
int32_t i, j, nb_rx = 0;
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S;
+ stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Make sure there is at least 1 packet to receive */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/**
@@ -1096,42 +1111,31 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
for (i = 0; i < ICE_RX_MAX_BURST; i += ICE_LOOK_AHEAD,
rxdp += ICE_LOOK_AHEAD, rxep += ICE_LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--) {
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- s[j] = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
- }
+ for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--)
+ s[j] = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
rte_smp_rmb();
/* Compute how many status bits were set */
for (j = 0, nb_dd = 0; j < ICE_LOOK_AHEAD; j++)
- nb_dd += s[j] & (1 << ICE_RX_DESC_STATUS_DD_S);
+ nb_dd += s[j] & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S);
nb_rx += nb_dd;
/* Translate descriptor info to mbuf parameters */
for (j = 0; j < nb_dd; j++) {
mb = rxep[j].mbuf;
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- pkt_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ pkt_len = (rte_le_to_cpu_16(rxdp[j].wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
mb->data_len = pkt_len;
mb->pkt_len = pkt_len;
mb->ol_flags = 0;
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- mb->hash.rss =
- rte_le_to_cpu_32(
- rxdp[j].wb.qword0.hi_dword.rss);
- mb->packet_type = ptype_tbl[(uint8_t)(
- (qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
+ stat_err0 = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
+ pkt_flags = ice_rxd_error_to_pkt_flags(stat_err0);
+ mb->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxdp[j].wb.ptype_flex_flags0)];
ice_rxd_to_vlan_tci(mb, &rxdp[j]);
+ ice_rxd_to_pkt_fields(mb, &rxdp[j]);
mb->ol_flags |= pkt_flags;
}
@@ -1312,8 +1316,8 @@ ice_recv_scattered_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *first_seg = rxq->pkt_first_seg;
@@ -1324,21 +1328,18 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1377,14 +1378,10 @@ ice_recv_scattered_pkts(void *rx_queue,
/* Set data buffer address and data length of the mbuf */
rxdp->read.hdr_addr = 0;
rxdp->read.pkt_addr = dma_addr;
- rx_packet_len = (qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S;
+ rx_packet_len = rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
rxm->data_len = rx_packet_len;
rxm->data_off = RTE_PKTMBUF_HEADROOM;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
/**
* If this is the first buffer of the received packet, set the
@@ -1410,7 +1407,7 @@ ice_recv_scattered_pkts(void *rx_queue,
* update the pointer to the last mbuf of the current scattered
* packet and continue to parse the RX ring.
*/
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_EOF_S))) {
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_EOF_S))) {
last_seg = rxm;
continue;
}
@@ -1442,13 +1439,11 @@ ice_recv_scattered_pkts(void *rx_queue,
first_seg->port = rxq->port_id;
first_seg->ol_flags = 0;
-
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- first_seg->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
-
+ first_seg->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(first_seg, &rxd);
+ ice_rxd_to_pkt_fields(first_seg, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
first_seg->ol_flags |= pkt_flags;
/* Prefetch data of first segment, if configured to do so. */
rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr,
@@ -1538,9 +1533,8 @@ ice_dev_supported_ptypes_get(struct rte_eth_dev *dev)
int
ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq = rx_queue;
- volatile uint64_t *status;
- uint64_t mask;
uint32_t desc;
if (unlikely(offset >= rxq->nb_rx_desc))
@@ -1553,10 +1547,9 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- status = &rxq->rx_ring[desc].wb.qword1.status_error_len;
- mask = rte_cpu_to_le_64((1ULL << ICE_RX_DESC_STATUS_DD_S) <<
- ICE_RXD_QW1_STATUS_S);
- if (*status & mask)
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
return RTE_ETH_RX_DESC_AVAIL;
@@ -1642,8 +1635,8 @@ ice_recv_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *nmb; /* new allocated mbuf */
@@ -1652,21 +1645,18 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1696,8 +1686,8 @@ ice_recv_pkts(void *rx_queue,
rxdp->read.pkt_addr = dma_addr;
/* calculate rx_packet_len of the received pkt */
- rx_packet_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ rx_packet_len = (rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
/* fill old mbuf with received descriptor: rxd */
rxm->data_off = RTE_PKTMBUF_HEADROOM;
@@ -1707,15 +1697,11 @@ ice_recv_pkts(void *rx_queue,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- rxm->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
+ rxm->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(rxm, &rxd);
+ ice_rxd_to_pkt_fields(rxm, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
rxm->ol_flags |= pkt_flags;
/* copy old mbuf to rx_pkts */
rx_pkts[nb_rx++] = rxm;
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 3/6] net/ice: add protocol extraction support for per Rx queue
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 4/6] net/ice: support more ptype Leyi Rong
` (2 subsequent siblings)
5 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The ice has the feature to extract protocol fields into flex descriptor
by programming per queue. Currently, the ice PMD will put the protocol
fields into rte_mbuf::udata64 with different type format. Application
can access the protocol fields quickly.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
doc/guides/nics/ice.rst | 101 ++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 316 ++++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 2 +
drivers/net/ice/ice_rxtx.c | 63 +++++
drivers/net/ice/ice_rxtx.h | 2 +
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
9 files changed, 644 insertions(+)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
index 03819d29f..8a6f60e71 100644
--- a/doc/guides/nics/ice.rst
+++ b/doc/guides/nics/ice.rst
@@ -61,6 +61,107 @@ Runtime Config Options
NOTE: In Safe mode, only very limited features are available, features like RSS,
checksum, fdir, tunneling ... are all disabled.
+- ``Protocol extraction for per queue``
+
+ Configure the RX queues to do protocol extraction into ``rte_mbuf::udata64``
+ for protocol handling acceleration, like checking the TCP SYN packets quickly.
+
+ The argument format is::
+
+ -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
+ -w 18:00.0,proto_xtr=<protocol>
+
+ Queues are grouped by ``(`` and ``)`` within the group. The ``-`` character
+ is used as a range separator and ``,`` is used as a single number separator.
+ The grouping ``()`` can be omitted for single element group. If no queues are
+ specified, PMD will use this protocol extraction type for all queues.
+
+ Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-13 are
+ VLAN extraction, other queues run with no protocol extraction.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-23 are
+ IPv6 extraction, other queues use the default VLAN extraction.
+
+ The extraction will be copied into the lower 32 bit of ``rte_mbuf::udata64``.
+
+ .. table:: Protocol extraction : ``vlan``
+
+ +----------------------------+----------------------------+
+ | VLAN2 | VLAN1 |
+ +======+===+=================+======+===+=================+
+ | PCP | D | VID | PCP | D | VID |
+ +------+---+-----------------+------+---+-----------------+
+
+ VLAN1 - single or EVLAN (first for QinQ).
+
+ VLAN2 - C-VLAN (second for QinQ).
+
+ .. table:: Protocol extraction : ``ipv4``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +======+=======+=============+==============+=============+
+ | Ver |Hdr Len| ToS | TTL | Protocol |
+ +------+-------+-------------+--------------+-------------+
+
+ IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
+
+ IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
+
+ .. table:: Protocol extraction : ``ipv6``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+=============+==============+
+ | Ver |Traffic class| Flow | Next Header | Hop Limit |
+ +-----+-------------+--------+-------------+--------------+
+
+ IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``ipv6_flow``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+============================+
+ | Ver |Traffic class| Flow Label |
+ +-----+-------------+-------------------------------------+
+
+ IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``tcp``
+
+ +----------------------------+----------------------------+
+ | TCPHDR2 | TCPHDR1 |
+ +============================+======+======+==============+
+ | Reserved |Offset| RSV | Flags |
+ +----------------------------+------+------+--------------+
+
+ TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
+
+ TCPHDR2 - Reserved
+
+ Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the protocol
+ extraction, do not use ``rte_mbuf::udata64`` directly.
+
+ The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
+ access the protocol extraction result in ``struct rte_mbuf``.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile
index ae53c2646..4a279f196 100644
--- a/drivers/net/ice/Makefile
+++ b/drivers/net/ice/Makefile
@@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1)
endif
SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
+
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..9d3e76588 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -19,9 +19,11 @@
/* devargs */
#define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
+#define ICE_PROTO_XTR_ARG "proto_xtr"
static const char * const ice_valid_args[] = {
ICE_SAFE_MODE_SUPPORT_ARG,
+ ICE_PROTO_XTR_ARG,
NULL
};
@@ -257,6 +259,303 @@ ice_init_controlq_parameter(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ;
}
+static int
+lookup_proto_xtr_type(const char *xtr_name)
+{
+ static struct {
+ const char *name;
+ enum proto_xtr_type type;
+ } xtr_type_map[] = {
+ { "vlan", PROTO_XTR_VLAN },
+ { "ipv4", PROTO_XTR_IPV4 },
+ { "ipv6", PROTO_XTR_IPV6 },
+ { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
+ { "tcp", PROTO_XTR_TCP },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
+ if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
+ return xtr_type_map[i].type;
+ }
+
+ return -1;
+}
+
+/*
+ * Parse elem, the elem could be single number/range or '(' ')' group
+ * 1) A single number elem, it's just a simple digit. e.g. 9
+ * 2) A single range elem, two digits with a '-' between. e.g. 2-6
+ * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
+ * Within group elem, '-' used for a range separator;
+ * ',' used for a single number.
+ */
+static int
+parse_queue_set(const char *input, int xtr_type, struct ice_pf *pf)
+{
+ const char *str = input;
+ char *end = NULL;
+ uint32_t min, max;
+ uint32_t idx;
+
+ while (isblank(*str))
+ str++;
+
+ if ((!isdigit(*str) && *str != '(') || (*str == '\0'))
+ return -1;
+
+ /* process single number or single range of number */
+ if (*str != '(') {
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ while (isblank(*end))
+ end++;
+
+ min = idx;
+ max = idx;
+
+ /* process single <number>-<number> */
+ if (*end == '-') {
+ end++;
+ while (isblank(*end))
+ end++;
+ if (!isdigit(*end))
+ return -1;
+
+ errno = 0;
+ idx = strtoul(end, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ max = idx;
+ while (isblank(*end))
+ end++;
+ }
+
+ if (*end != ':')
+ return -1;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ pf->proto_xtr_table[idx] = xtr_type;
+
+ return 0;
+ }
+
+ /* process set within bracket */
+ str++;
+ while (isblank(*str))
+ str++;
+ if (*str == '\0')
+ return -1;
+
+ min = pf->lan_nb_qps;
+ do {
+ /* go ahead to the first digit */
+ while (isblank(*str))
+ str++;
+ if (!isdigit(*str))
+ return -1;
+
+ /* get the digit value */
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= pf->lan_nb_qps)
+ return -1;
+
+ /* go ahead to separator '-',',' and ')' */
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ if (min == pf->lan_nb_qps)
+ min = idx;
+ else /* avoid continuous '-' */
+ return -1;
+ } else if (*end == ',' || *end == ')') {
+ max = idx;
+ if (min == pf->lan_nb_qps)
+ min = idx;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ pf->proto_xtr_table[idx] = xtr_type;
+
+ min = pf->lan_nb_qps;
+ } else {
+ return -1;
+ }
+
+ str = end + 1;
+ } while (*end != ')' && *end != '\0');
+
+ return 0;
+}
+
+static int
+parse_queue_proto_xtr(const char *queues, struct ice_pf *pf)
+{
+ const char *queue_start;
+ uint32_t idx;
+ int xtr_type;
+ char xtr_name[32];
+
+ while (isblank(*queues))
+ queues++;
+
+ if (*queues != '[') {
+ xtr_type = lookup_proto_xtr_type(queues);
+ if (xtr_type < 0)
+ return -1;
+
+ pf->proto_xtr = xtr_type;
+ return 0;
+ }
+
+ if (pf->proto_xtr_table == NULL) {
+ PMD_DRV_LOG(ERR,
+ "Can't support protocol extraction table for no memory");
+ return -1;
+ }
+
+ queues++;
+ do {
+ while (isblank(*queues))
+ queues++;
+ if (*queues == '\0')
+ return -1;
+
+ queue_start = queues;
+
+ /* go across a complete bracket */
+ if (*queue_start == '(') {
+ queues += strcspn(queues, ")");
+ if (*queues != ')')
+ return -1;
+ }
+
+ /* scan the separator ':' */
+ queues += strcspn(queues, ":");
+ if (*queues++ != ':')
+ return -1;
+ while (isblank(*queues))
+ queues++;
+
+ for (idx = 0; ; idx++) {
+ if (isblank(queues[idx]) ||
+ queues[idx] == ',' ||
+ queues[idx] == ']' ||
+ queues[idx] == '\0')
+ break;
+
+ if (idx > sizeof(xtr_name) - 2)
+ return -1;
+
+ xtr_name[idx] = queues[idx];
+ }
+ xtr_name[idx] = '\0';
+ xtr_type = lookup_proto_xtr_type(xtr_name);
+ if (xtr_type < 0)
+ return -1;
+
+ queues += idx;
+
+ while (isblank(*queues) || *queues == ',' || *queues == ']')
+ queues++;
+
+ if (parse_queue_set(queue_start, xtr_type, pf) < 0)
+ return -1;
+ } while (*queues != '\0');
+
+ return 0;
+}
+
+static int
+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
+ void *extra_args)
+{
+ struct ice_pf *pf = extra_args;
+
+ if (value == NULL || extra_args == NULL)
+ return -EINVAL;
+
+ if (parse_queue_proto_xtr(value, pf) < 0) {
+ PMD_DRV_LOG(ERR,
+ "The protocol extraction parameter is wrong : '%s'",
+ value);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+ice_parse_proto_xtr_devarg(struct rte_devargs *devargs, struct ice_pf *pf)
+{
+ struct rte_kvargs *kvlist;
+
+ pf->proto_xtr = PROTO_XTR_NONE;
+ if (devargs == NULL)
+ return;
+
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL)
+ return;
+
+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
+ handle_proto_xtr_arg, pf);
+
+ rte_kvargs_free(kvlist);
+}
+
+static bool
+ice_proto_xtr_support(struct ice_hw *hw)
+{
+#define FLX_REG(val, fld, idx) \
+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
+ static struct {
+ uint32_t rxdid;
+ uint16_t protid_0;
+ uint16_t protid_1;
+ } xtr_sets[] = {
+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
+ ICE_PROT_IPV4_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
+ uint32_t rxdid = xtr_sets[i].rxdid;
+ uint32_t v;
+
+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+
+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+ }
+
+ return true;
+}
+
static int
ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
uint32_t num)
@@ -1088,6 +1387,19 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
pf->lan_nb_qps = pf->lan_nb_qp_max;
+ if (ice_proto_xtr_support(hw)) {
+ pf->proto_xtr_table = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
+ if (pf->proto_xtr_table == NULL)
+ PMD_DRV_LOG(ERR, "No memory for protocol extraction table");
+
+ ice_parse_proto_xtr_devarg(dev->device->devargs, pf);
+ } else {
+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
+
+ pf->proto_xtr = PROTO_XTR_NONE;
+ pf->proto_xtr_table = NULL;
+ }
+
return 0;
}
@@ -1547,6 +1859,7 @@ ice_dev_init(struct rte_eth_dev *dev)
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr_table);
return ret;
}
@@ -1672,6 +1985,8 @@ ice_dev_close(struct rte_eth_dev *dev)
rte_free(hw->port_info);
hw->port_info = NULL;
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr_table);
+ pf->proto_xtr_table = NULL;
}
static int
@@ -3795,6 +4110,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ice,
+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
RTE_INIT(ice_init_log)
diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
index f569da833..2a0bd4dba 100644
--- a/drivers/net/ice/ice_ethdev.h
+++ b/drivers/net/ice/ice_ethdev.h
@@ -263,6 +263,8 @@ struct ice_pf {
uint16_t lan_nb_qp_max;
uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
uint16_t base_queue; /* The base queue pairs index in the device */
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
+ uint8_t *proto_xtr_table; /* Per queue for lan_nb_qps size */
struct ice_hw_port_stats stats_offset;
struct ice_hw_port_stats stats;
/* internal packet statistics, it should be excluded from the total */
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index d2e36853f..ade755c2f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,6 +13,36 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+static inline uint8_t
+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
+{
+ static uint8_t xtr_map[] = {
+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
+ };
+
+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
+}
+
+static inline uint8_t
+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
+{
+ static uint8_t rxdid_map[] = {
+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
+ };
+ uint8_t rxdid;
+
+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
+
+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
+}
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -84,6 +114,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
+
+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
+ rxq->port_id, rxq->queue_id, rxdid);
+
/* Enable Flexible Descriptors in the queue context which
* allows this driver to select a specific receive descriptor format
*/
@@ -641,6 +676,10 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
rxq->drop_en = rx_conf->rx_drop_en;
rxq->vsi = vsi;
rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->proto_xtr = pf->proto_xtr_table != NULL ?
+ pf->proto_xtr_table[queue_idx] : PROTO_XTR_NONE;
+ if (rxq->proto_xtr == PROTO_XTR_NONE)
+ rxq->proto_xtr = pf->proto_xtr;
/* Allocate the maximun number of RX ring hardware descriptor. */
len = ICE_MAX_RING_DESC;
@@ -1062,6 +1101,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+#define ICE_RX_PROTO_XTR_VALID \
+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+
static inline void
ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
volatile union ice_rx_flex_desc *rxdp)
@@ -1075,6 +1118,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
mb->ol_flags |= PKT_RX_RSS_HASH;
mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
}
+
+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
+ init_proto_xtr_flds(mb);
+
+ stat_err = rte_le_to_cpu_16(desc->status_error1);
+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
+ xtr->u.raw.data0 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+ xtr->u.raw.data1 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
+
+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
+ xtr->magic = PROTO_XTR_MAGIC_ID;
+ }
+#endif
}
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index 64e891875..de16637f3 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -5,6 +5,7 @@
#ifndef _ICE_RXTX_H_
#define _ICE_RXTX_H_
+#include "rte_pmd_ice.h"
#include "ice_ethdev.h"
#define ICE_ALIGN_RING_DESC 32
@@ -78,6 +79,7 @@ struct ice_rx_queue {
uint16_t max_pkt_len; /* Maximum packet length */
bool q_set; /* indicate if rx queue has been configured */
bool rx_deferred_start; /* don't start this queue in dev start */
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
ice_rx_release_mbufs_t rx_rel_mbufs;
};
diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
index c5f0d564f..080ca4175 100644
--- a/drivers/net/ice/ice_rxtx_vec_common.h
+++ b/drivers/net/ice/ice_rxtx_vec_common.h
@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
if (rxq->nb_rx_desc % rxq->rx_free_thresh)
return -1;
+ if (rxq->proto_xtr != PROTO_XTR_NONE)
+ return -1;
+
return 0;
}
diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
index 36b4b3c85..6828170a9 100644
--- a/drivers/net/ice/meson.build
+++ b/drivers/net/ice/meson.build
@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
endif
endif
+
+install_headers('rte_pmd_ice.h')
diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
new file mode 100644
index 000000000..719487e1e
--- /dev/null
+++ b/drivers/net/ice/rte_pmd_ice.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#ifndef _RTE_PMD_ICE_H_
+#define _RTE_PMD_ICE_H_
+
+#include <stdio.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum proto_xtr_type {
+ PROTO_XTR_NONE,
+ PROTO_XTR_VLAN,
+ PROTO_XTR_IPV4,
+ PROTO_XTR_IPV6,
+ PROTO_XTR_IPV6_FLOW,
+ PROTO_XTR_TCP,
+};
+
+struct proto_xtr_flds {
+ union {
+ struct {
+ uint16_t data0;
+ uint16_t data1;
+ } raw;
+ struct {
+ uint16_t stag_vid:12,
+ stag_dei:1,
+ stag_pcp:3;
+ uint16_t ctag_vid:12,
+ ctag_dei:1,
+ ctag_pcp:3;
+ } vlan;
+ struct {
+ uint16_t protocol:8,
+ ttl:8;
+ uint16_t tos:8,
+ ihl:4,
+ version:4;
+ } ipv4;
+ struct {
+ uint16_t hoplimit:8,
+ nexthdr:8;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6;
+ struct {
+ uint16_t flowlo16;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6_flow;
+ struct {
+ uint16_t fin:1,
+ syn:1,
+ rst:1,
+ psh:1,
+ ack:1,
+ urg:1,
+ ece:1,
+ cwr:1,
+ res1:4,
+ doff:4;
+ uint16_t rsvd;
+ } tcp;
+ } u;
+
+ uint16_t rsvd;
+
+ uint8_t type;
+
+#define PROTO_XTR_MAGIC_ID 0xCE
+ uint8_t magic;
+};
+
+static inline void
+init_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ mb->udata64 = 0;
+}
+
+static inline struct proto_xtr_flds *
+get_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
+
+ return (struct proto_xtr_flds *)&mb->udata64;
+}
+
+static inline void
+dump_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
+ return;
+
+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
+ xtr->u.raw.data0, xtr->u.raw.data1);
+
+ if (xtr->type == PROTO_XTR_VLAN)
+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
+ xtr->u.vlan.stag_pcp,
+ xtr->u.vlan.stag_dei,
+ xtr->u.vlan.stag_vid,
+ xtr->u.vlan.ctag_pcp,
+ xtr->u.vlan.ctag_dei,
+ xtr->u.vlan.ctag_vid);
+ else if (xtr->type == PROTO_XTR_IPV4)
+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
+ xtr->u.ipv4.version,
+ xtr->u.ipv4.ihl,
+ xtr->u.ipv4.tos,
+ xtr->u.ipv4.ttl,
+ xtr->u.ipv4.protocol);
+ else if (xtr->type == PROTO_XTR_IPV6)
+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
+ xtr->u.ipv6.version,
+ xtr->u.ipv6.tc,
+ xtr->u.ipv6.flowhi4,
+ xtr->u.ipv6.nexthdr,
+ xtr->u.ipv6.hoplimit);
+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
+ xtr->u.ipv6_flow.version,
+ xtr->u.ipv6_flow.tc,
+ xtr->u.ipv6_flow.flowhi4,
+ xtr->u.ipv6_flow.flowlo16);
+ else if (xtr->type == PROTO_XTR_TCP)
+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
+ xtr->u.tcp.doff,
+ xtr->u.tcp.cwr ? "C" : "",
+ xtr->u.tcp.ece ? "E" : "",
+ xtr->u.tcp.urg ? "U" : "",
+ xtr->u.tcp.ack ? "A" : "",
+ xtr->u.tcp.psh ? "P" : "",
+ xtr->u.tcp.rst ? "R" : "",
+ xtr->u.tcp.syn ? "S" : "",
+ xtr->u.tcp.fin ? "F" : "");
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_PMD_ICE_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 4/6] net/ice: support more ptype
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
` (2 preceding siblings ...)
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
5 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
More protocol types are supported by HW.
Add them.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index ade755c2f..75def755a 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -3093,6 +3093,17 @@ ice_get_default_pkt_type(uint16_t ptype)
RTE_PTYPE_TUNNEL_GTPU,
[267] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN |
RTE_PTYPE_TUNNEL_GTPU,
+ /* [268] - [271] reserved */
+ /* Some protocols are not supported by API, Like, VRRP, OSPF.
+ * Just report them as L2 or L3 packets.
+ */
+ [272] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ [273] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ [274] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ [275] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ [276] = RTE_PTYPE_L2_ETHER,
+ /* [277] reserved */
+ [278] = RTE_PTYPE_L2_ETHER,
/* All others reserved */
};
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 5/6] net/ice: switch to flexible descriptor in SSE path
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
` (3 preceding siblings ...)
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 4/6] net/ice: support more ptype Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
5 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
With this path, the flexible descriptor is supported
in SSE path. And the legacy descriptor is not supported.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx_vec_sse.c | 243 ++++++++++++++---------------
1 file changed, 115 insertions(+), 128 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index 967a7b16b..aea00ecd0 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -15,14 +15,14 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
struct rte_mbuf *mb0, *mb1;
__m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -88,93 +88,96 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer);
__m128i rearm0, rearm1, rearm2, rearm3;
- __m128i vlan0, vlan1, rss, l3_l4e;
+ __m128i tmp_desc, flags, rss, vlan;
- /* mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication.
+ /* mask everything except checksum, RSS and VLAN flags.
+ * bit6:4 for checksum.
+ * bit12 for RSS indication.
+ * bit13 for VLAN indication.
*/
- const __m128i rss_vlan_msk = _mm_set_epi32(0x1c03804, 0x1c03804,
- 0x1c03804, 0x1c03804);
+ const __m128i desc_mask = _mm_set_epi32(0x3070, 0x3070,
+ 0x3070, 0x3070);
- const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD);
- /* map rss and vlan type to rss hash and vlan flag */
- const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- 0, 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
- 0, 0, 0, 0);
-
- const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH, 0, 0,
- 0, 0, PKT_RX_FDIR, 0);
-
- const __m128i l3_l4e_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ /* map the checksum, rss and vlan fields to the checksum, rss
+ * and vlan flag
+ */
+ const __m128i cksum_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
- vlan0 = _mm_unpackhi_epi32(descs[0], descs[1]);
- vlan1 = _mm_unpackhi_epi32(descs[2], descs[3]);
- vlan0 = _mm_unpacklo_epi64(vlan0, vlan1);
-
- vlan1 = _mm_and_si128(vlan0, rss_vlan_msk);
- vlan0 = _mm_shuffle_epi8(vlan_flags, vlan1);
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
- rss = _mm_srli_epi32(vlan1, 11);
- rss = _mm_shuffle_epi8(rss_flags, rss);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0);
- l3_l4e = _mm_srli_epi32(vlan1, 22);
- l3_l4e = _mm_shuffle_epi8(l3_l4e_flags, l3_l4e);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0);
+
+ /* merge 4 descriptors */
+ flags = _mm_unpackhi_epi32(descs[0], descs[1]);
+ tmp_desc = _mm_unpackhi_epi32(descs[2], descs[3]);
+ tmp_desc = _mm_unpacklo_epi64(flags, tmp_desc);
+ tmp_desc = _mm_and_si128(flags, desc_mask);
+
+ /* checksum flags */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 4);
+ flags = _mm_shuffle_epi8(cksum_flags, tmp_desc);
/* then we shift left 1 bit */
- l3_l4e = _mm_slli_epi32(l3_l4e, 1);
- /* we need to mask out the reduntant bits */
- l3_l4e = _mm_and_si128(l3_l4e, cksum_mask);
+ flags = _mm_slli_epi32(flags, 1);
+ /* we need to mask out the reduntant bits introduced by RSS or
+ * VLAN fields.
+ */
+ flags = _mm_and_si128(flags, cksum_mask);
+
+ /* RSS, VLAN flag */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 8);
+ rss = _mm_shuffle_epi8(rss_flags, tmp_desc);
+ vlan = _mm_shuffle_epi8(vlan_flags, tmp_desc);
- vlan0 = _mm_or_si128(vlan0, rss);
- vlan0 = _mm_or_si128(vlan0, l3_l4e);
+ /* merge the flags */
+ flags = _mm_or_si128(flags, rss);
+ flags = _mm_or_si128(flags, vlan);
/**
* At this point, we have the 4 sets of flags in the low 16-bits
- * of each 32-bit value in vlan0.
+ * of each 32-bit value in flags.
* We want to extract these, and merge them with the mbuf init data
* so we can do a single 16-byte write to the mbuf to set the flags
* and all the other initialization fields. Extracting the
* appropriate flags means that we have to do a shift and blend for
* each mbuf before we do the write.
*/
- rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 8), 0x10);
- rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 4), 0x10);
- rearm2 = _mm_blend_epi16(mbuf_init, vlan0, 0x10);
- rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10);
+ rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x10);
+ rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x10);
+ rearm2 = _mm_blend_epi16(mbuf_init, flags, 0x10);
+ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x10);
/* write the rearm data and the olflags in one write */
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
@@ -187,22 +190,24 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
_mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3);
}
-#define PKTLEN_SHIFT 10
-
static inline void
ice_rx_desc_to_ptype_v(__m128i descs[4], struct rte_mbuf **rx_pkts,
uint32_t *ptype_tbl)
{
- __m128i ptype0 = _mm_unpackhi_epi64(descs[0], descs[1]);
- __m128i ptype1 = _mm_unpackhi_epi64(descs[2], descs[3]);
-
- ptype0 = _mm_srli_epi64(ptype0, 30);
- ptype1 = _mm_srli_epi64(ptype1, 30);
-
- rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 0)];
- rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 8)];
- rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 0)];
- rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 8)];
+ const __m128i ptype_mask = _mm_set_epi16(0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M);
+ __m128i ptype_01 = _mm_unpacklo_epi32(descs[0], descs[1]);
+ __m128i ptype_23 = _mm_unpacklo_epi32(descs[2], descs[3]);
+ __m128i ptype_all = _mm_unpacklo_epi64(ptype_01, ptype_23);
+
+ ptype_all = _mm_and_si128(ptype_all, ptype_mask);
+
+ rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
}
/**
@@ -215,21 +220,39 @@ static inline uint16_t
_ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
uint64_t var;
- __m128i shuf_msk;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
-
__m128i crc_adjust = _mm_set_epi16
- (0, 0, 0, /* ignore non-length fields */
+ (0, 0, 0, /* ignore non-length fields */
-rxq->crc_len, /* sub crc on data_len */
0, /* ignore high-16bits of pkt_len */
-rxq->crc_len, /* sub crc on pkt_len */
- 0, 0 /* ignore pkt_type field */
+ 0, 0 /* ignore pkt_type field */
);
+ const __m128i zero = _mm_setzero_si128();
+ /* mask to shuffle from desc. to mbuf */
+ const __m128i shuf_msk = _mm_set_epi8
+ (0xFF, 0xFF, 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, low 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /* pkt_type set as unknown */
+ );
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+
/**
* compile-time check the above crc_adjust layout is correct.
* NOTE: the first field (lowest address) is given last in set_epi16
@@ -239,7 +262,13 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
- __m128i dd_check, eop_check;
+
+ /* 4 packets DD mask */
+ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ /* 4 packets EOP mask */
+ const __m128i eop_check = _mm_set_epi64x(0x0000000200000002LL,
+ 0x0000000200000002LL);
/* nb_pkts shall be less equal than ICE_MAX_RX_BURST */
nb_pkts = RTE_MIN(nb_pkts, ICE_MAX_RX_BURST);
@@ -250,7 +279,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = rxq->rx_ring + rxq->rx_tail;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
@@ -263,26 +292,10 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
- /* 4 packets DD mask */
- dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL);
-
- /* 4 packets EOP mask */
- eop_check = _mm_set_epi64x(0x0000000200000002LL, 0x0000000200000002LL);
-
- /* mask to shuffle from desc. to mbuf */
- shuf_msk = _mm_set_epi8
- (7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
- );
/**
* Compile-time verify the shuffle mask
* NOTE: some field positions already verified above, but duplicated
@@ -315,7 +328,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
rxdp += ICE_DESCS_PER_LOOP) {
__m128i descs[ICE_DESCS_PER_LOOP];
__m128i pkt_mb1, pkt_mb2, pkt_mb3, pkt_mb4;
- __m128i zero, staterr, sterr_tmp1, sterr_tmp2;
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
/* 2 64 bit or 4 32 bit mbuf pointers in one XMM reg. */
__m128i mbp1;
#if defined(RTE_ARCH_X86_64)
@@ -359,14 +372,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* avoid compiler reorder optimization */
rte_compiler_barrier();
- /* pkt 3,4 shift the pktlen field to be 16-bit aligned*/
- const __m128i len3 = _mm_slli_epi32(descs[3], PKTLEN_SHIFT);
- const __m128i len2 = _mm_slli_epi32(descs[2], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[3] = _mm_blend_epi16(descs[3], len3, 0x80);
- descs[2] = _mm_blend_epi16(descs[2], len2, 0x80);
-
/* D.1 pkt 3,4 convert format from desc to pktmbuf */
pkt_mb4 = _mm_shuffle_epi8(descs[3], shuf_msk);
pkt_mb3 = _mm_shuffle_epi8(descs[2], shuf_msk);
@@ -382,20 +387,11 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
pkt_mb4 = _mm_add_epi16(pkt_mb4, crc_adjust);
pkt_mb3 = _mm_add_epi16(pkt_mb3, crc_adjust);
- /* pkt 1,2 shift the pktlen field to be 16-bit aligned*/
- const __m128i len1 = _mm_slli_epi32(descs[1], PKTLEN_SHIFT);
- const __m128i len0 = _mm_slli_epi32(descs[0], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[1] = _mm_blend_epi16(descs[1], len1, 0x80);
- descs[0] = _mm_blend_epi16(descs[0], len0, 0x80);
-
/* D.1 pkt 1,2 convert format from desc to pktmbuf */
pkt_mb2 = _mm_shuffle_epi8(descs[1], shuf_msk);
pkt_mb1 = _mm_shuffle_epi8(descs[0], shuf_msk);
/* C.2 get 4 pkts staterr value */
- zero = _mm_xor_si128(dd_check, dd_check);
staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
/* D.3 copy final 3,4 data to rx_pkts */
@@ -412,15 +408,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* C* extract and record EOP bit */
if (split_packet) {
- __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0x04, 0x0C,
- 0x00, 0x08);
-
/* and with mask to extract bits, flipping 1-0 */
__m128i eop_bits = _mm_andnot_si128(staterr, eop_check);
/* the staterr values are not in order, as the count
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
` (4 preceding siblings ...)
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
@ 2019-08-29 8:04 ` Leyi Rong
2019-08-29 23:31 ` Zhang, Qi Z
5 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-08-29 8:04 UTC (permalink / raw)
To: qi.z.zhang, xiaolong.ye, haiyue.wang, wenzhuo.lu; +Cc: dev, Leyi Rong
Switch to Rx flexible descriptor format instead of legacy
descriptor format.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
---
drivers/net/ice/ice_rxtx_vec_avx2.c | 232 ++++++++++++++--------------
1 file changed, 118 insertions(+), 114 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 5ce29c2a2..158f17d80 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -132,8 +132,6 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
ICE_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
}
-#define PKTLEN_SHIFT 10
-
static inline uint16_t
_ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
@@ -144,7 +142,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp =
+ (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
@@ -161,8 +160,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/* constants used in processing loop */
@@ -193,21 +192,23 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i shuf_msk =
_mm256_set_epi8
(/* first descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF, /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF, /*pkt_type set as unknown */
/* second descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /*pkt_type set as unknown */
);
/**
* compile-time check the above crc and shuffle layout is correct.
@@ -225,68 +226,76 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Status/Error flag masks */
/**
- * mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication. Bits 3-5 of error
- * field (bits 22-24) are for IP/L4 checksum errors
+ * mask everything except Checksum Reports, RSS indication
+ * and VLAN indication.
+ * bit6:4 for IP/L4 checksum errors.
+ * bit12 is for RSS indication.
+ * bit13 is for VLAN indication.
*/
const __m256i flags_mask =
- _mm256_set1_epi32((1 << 2) | (1 << 11) |
- (3 << 12) | (7 << 22));
- /**
- * data to be shuffled by result of flag mask. If VLAN bit is set,
- * (bit 2), then position 4 in this array will be used in the
- * destination
- */
- const __m256i vlan_flags_shuf =
- _mm256_set_epi32(0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0,
- 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0);
- /**
- * data to be shuffled by result of flag mask, shifted down 11.
- * If RSS/FDIR bits are set, shuffle moves appropriate flags in
- * place.
- */
- const __m256i rss_flags_shuf =
- _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0,/* end up 128-bits */
- 0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0);
-
+ _mm256_set1_epi32((7 << 4) | (1 << 12) | (1 << 13));
/**
- * data to be shuffled by the result of the flags mask shifted by 22
+ * data to be shuffled by the result of the flags mask shifted by 4
* bits. This gives use the l3_l4 flags.
*/
const __m256i l3_l4_flags_shuf = _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1,
/* second 128-bits */
0, 0, 0, 0, 0, 0, 0, 0,
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
const __m256i cksum_mask =
_mm256_set1_epi32(PKT_RX_IP_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD |
PKT_RX_L4_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_EIP_CKSUM_BAD);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If RSS bit is set, shuffle moves appropriate flags in place.
+ */
+ const __m256i rss_flags_shuf =
+ _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH, 0, PKT_RX_RSS_HASH, 0);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If VLAN bit is set(bit 13), shuffle moves appropriate flags in place.
+ */
+ const __m256i vlan_flags_shuf =
+ _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0, 0);
RTE_SET_USED(avx_aligned); /* for 32B descriptors we don't use this */
@@ -369,73 +378,66 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
}
/**
- * convert descriptors 4-7 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 4-7 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len6_7 = _mm256_slli_epi32(raw_desc6_7,
- PKTLEN_SHIFT);
- const __m256i len4_5 = _mm256_slli_epi32(raw_desc4_5,
- PKTLEN_SHIFT);
- const __m256i desc6_7 = _mm256_blend_epi16(raw_desc6_7,
- len6_7, 0x80);
- const __m256i desc4_5 = _mm256_blend_epi16(raw_desc4_5,
- len4_5, 0x80);
- __m256i mb6_7 = _mm256_shuffle_epi8(desc6_7, shuf_msk);
- __m256i mb4_5 = _mm256_shuffle_epi8(desc4_5, shuf_msk);
+ __m256i mb6_7 = _mm256_shuffle_epi8(raw_desc6_7, shuf_msk);
+ __m256i mb4_5 = _mm256_shuffle_epi8(raw_desc4_5, shuf_msk);
mb6_7 = _mm256_add_epi16(mb6_7, crc_adjust);
mb4_5 = _mm256_add_epi16(mb4_5, crc_adjust);
/**
- * to get packet types, shift 64-bit values down 30 bits
- * and so ptype is in lower 8-bits in each
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
*/
- const __m256i ptypes6_7 = _mm256_srli_epi64(desc6_7, 30);
- const __m256i ptypes4_5 = _mm256_srli_epi64(desc4_5, 30);
- const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 24);
- const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 8);
- const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 24);
- const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 8);
+ const __m256i ptype_mask =
+ _mm256_set1_epi16(ICE_RX_FLEX_DESC_PTYPE_M);
+ const __m256i ptypes6_7 =
+ _mm256_and_si256(raw_desc6_7, ptype_mask);
+ const __m256i ptypes4_5 =
+ _mm256_and_si256(raw_desc4_5, ptype_mask);
+ const uint16_t ptype7 = _mm256_extract_epi16(ptypes6_7, 9);
+ const uint16_t ptype6 = _mm256_extract_epi16(ptypes6_7, 1);
+ const uint16_t ptype5 = _mm256_extract_epi16(ptypes4_5, 9);
+ const uint16_t ptype4 = _mm256_extract_epi16(ptypes4_5, 1);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype7], 4);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype6], 0);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype5], 4);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype4], 0);
/* merge the status bits into one register */
- const __m256i status4_7 = _mm256_unpackhi_epi32(desc6_7,
- desc4_5);
+ const __m256i status4_7 = _mm256_unpackhi_epi32(raw_desc6_7,
+ raw_desc4_5);
/**
- * convert descriptors 0-3 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 0-3 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len2_3 = _mm256_slli_epi32(raw_desc2_3,
- PKTLEN_SHIFT);
- const __m256i len0_1 = _mm256_slli_epi32(raw_desc0_1,
- PKTLEN_SHIFT);
- const __m256i desc2_3 = _mm256_blend_epi16(raw_desc2_3,
- len2_3, 0x80);
- const __m256i desc0_1 = _mm256_blend_epi16(raw_desc0_1,
- len0_1, 0x80);
- __m256i mb2_3 = _mm256_shuffle_epi8(desc2_3, shuf_msk);
- __m256i mb0_1 = _mm256_shuffle_epi8(desc0_1, shuf_msk);
+ __m256i mb2_3 = _mm256_shuffle_epi8(raw_desc2_3, shuf_msk);
+ __m256i mb0_1 = _mm256_shuffle_epi8(raw_desc0_1, shuf_msk);
mb2_3 = _mm256_add_epi16(mb2_3, crc_adjust);
mb0_1 = _mm256_add_epi16(mb0_1, crc_adjust);
- /* get the packet types */
- const __m256i ptypes2_3 = _mm256_srli_epi64(desc2_3, 30);
- const __m256i ptypes0_1 = _mm256_srli_epi64(desc0_1, 30);
- const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 24);
- const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 8);
- const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 24);
- const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 8);
+ /**
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
+ */
+ const __m256i ptypes2_3 =
+ _mm256_and_si256(raw_desc2_3, ptype_mask);
+ const __m256i ptypes0_1 =
+ _mm256_and_si256(raw_desc0_1, ptype_mask);
+ const uint16_t ptype3 = _mm256_extract_epi16(ptypes2_3, 9);
+ const uint16_t ptype2 = _mm256_extract_epi16(ptypes2_3, 1);
+ const uint16_t ptype1 = _mm256_extract_epi16(ptypes0_1, 9);
+ const uint16_t ptype0 = _mm256_extract_epi16(ptypes0_1, 1);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype3], 4);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype2], 0);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype1], 4);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype0], 0);
/* merge the status bits into one register */
- const __m256i status0_3 = _mm256_unpackhi_epi32(desc2_3,
- desc0_1);
+ const __m256i status0_3 = _mm256_unpackhi_epi32(raw_desc2_3,
+ raw_desc0_1);
/**
* take the two sets of status bits and merge to one
@@ -450,20 +452,22 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* get only flag/error bits we want */
const __m256i flag_bits =
_mm256_and_si256(status0_7, flags_mask);
- /* set vlan and rss flags */
- const __m256i vlan_flags =
- _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
- const __m256i rss_flags =
- _mm256_shuffle_epi8(rss_flags_shuf,
- _mm256_srli_epi32(flag_bits, 11));
/**
* l3_l4_error flags, shuffle, then shift to correct adjustment
* of flags in flags_shuf, and finally mask out extra bits
*/
__m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
- _mm256_srli_epi32(flag_bits, 22));
+ _mm256_srli_epi32(flag_bits, 4));
l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
+ /* set rss and vlan flags */
+ const __m256i rss_vlan_flag_bits =
+ _mm256_srli_epi32(flag_bits, 12);
+ const __m256i rss_flags =
+ _mm256_shuffle_epi8(rss_flags_shuf, rss_vlan_flag_bits);
+ const __m256i vlan_flags =
+ _mm256_shuffle_epi8(vlan_flags_shuf,
+ rss_vlan_flag_bits);
/* merge flags */
const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
@ 2019-08-29 23:31 ` Zhang, Qi Z
2019-08-30 1:05 ` Wang, Haiyue
2019-08-30 6:17 ` Rong, Leyi
0 siblings, 2 replies; 54+ messages in thread
From: Zhang, Qi Z @ 2019-08-29 23:31 UTC (permalink / raw)
To: Rong, Leyi, Ye, Xiaolong, Wang, Haiyue, Lu, Wenzhuo; +Cc: dev
> -----Original Message-----
> From: Rong, Leyi
> Sent: Thursday, August 29, 2019 4:05 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> <xiaolong.ye@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu,
> Wenzhuo <wenzhuo.lu@intel.com>
> Cc: dev@dpdk.org; Rong, Leyi <leyi.rong@intel.com>
> Subject: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
>
> Switch to Rx flexible descriptor format instead of legacy descriptor format.
>
> Signed-off-by: Leyi Rong <leyi.rong@intel.com>
> ---
> drivers/net/ice/ice_rxtx_vec_avx2.c | 232 ++++++++++++++--------------
> 1 file changed, 118 insertions(+), 114 deletions(-)
>
> diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c
> b/drivers/net/ice/ice_rxtx_vec_avx2.c
> index 5ce29c2a2..158f17d80 100644
> --- a/drivers/net/ice/ice_rxtx_vec_avx2.c
> +++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
> @@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq) {
> int i;
> uint16_t rx_id;
> - volatile union ice_rx_desc *rxdp;
> + volatile union ice_rx_flex_desc *rxdp;
> struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
>
> - rxdp = rxq->rx_ring + rxq->rxrearm_start;
> + rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
Since after this patch, all data paths (normal, sse, avx2) are moved to flex desc,
Ice_rx_desc is not used anymore, so can replace all of them with ice_rx_flex_desc,
then above convention can be avoid.
<.......>
> * take the two sets of status bits and merge to one @@ -450,20
> +452,22 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct
> rte_mbuf **rx_pkts,
> /* get only flag/error bits we want */
> const __m256i flag_bits =
> _mm256_and_si256(status0_7, flags_mask);
> - /* set vlan and rss flags */
> - const __m256i vlan_flags =
> - _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
> - const __m256i rss_flags =
> - _mm256_shuffle_epi8(rss_flags_shuf,
> - _mm256_srli_epi32(flag_bits, 11));
> /**
> * l3_l4_error flags, shuffle, then shift to correct adjustment
> * of flags in flags_shuf, and finally mask out extra bits
> */
> __m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
> - _mm256_srli_epi32(flag_bits, 22));
> + _mm256_srli_epi32(flag_bits, 4));
> l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
> l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
> + /* set rss and vlan flags */
> + const __m256i rss_vlan_flag_bits =
> + _mm256_srli_epi32(flag_bits, 12);
> + const __m256i rss_flags =
> + _mm256_shuffle_epi8(rss_flags_shuf, rss_vlan_flag_bits);
> + const __m256i vlan_flags =
> + _mm256_shuffle_epi8(vlan_flags_shuf,
> + rss_vlan_flag_bits);
Seems we can "or" rss_flags_shuf and vlan_flags_shuf, so just need to do one shuffle here to save some CPU cycles?
>
> /* merge flags */
> const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
> --
> 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-29 23:31 ` Zhang, Qi Z
@ 2019-08-30 1:05 ` Wang, Haiyue
2019-08-30 1:06 ` Zhang, Qi Z
2019-08-30 6:17 ` Rong, Leyi
1 sibling, 1 reply; 54+ messages in thread
From: Wang, Haiyue @ 2019-08-30 1:05 UTC (permalink / raw)
To: Zhang, Qi Z, Rong, Leyi, Ye, Xiaolong, Lu, Wenzhuo; +Cc: dev
> -----Original Message-----
> From: Zhang, Qi Z
> Sent: Friday, August 30, 2019 07:32
> To: Rong, Leyi <leyi.rong@intel.com>; Ye, Xiaolong <xiaolong.ye@intel.com>; Wang, Haiyue
> <haiyue.wang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
>
>
>
> > -----Original Message-----
> > From: Rong, Leyi
> > Sent: Thursday, August 29, 2019 4:05 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> > <xiaolong.ye@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu,
> > Wenzhuo <wenzhuo.lu@intel.com>
> > Cc: dev@dpdk.org; Rong, Leyi <leyi.rong@intel.com>
> > Subject: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
> >
> > Switch to Rx flexible descriptor format instead of legacy descriptor format.
> >
> > Signed-off-by: Leyi Rong <leyi.rong@intel.com>
> > ---
> > drivers/net/ice/ice_rxtx_vec_avx2.c | 232 ++++++++++++++--------------
> > 1 file changed, 118 insertions(+), 114 deletions(-)
> >
> > diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c
> > b/drivers/net/ice/ice_rxtx_vec_avx2.c
> > index 5ce29c2a2..158f17d80 100644
> > --- a/drivers/net/ice/ice_rxtx_vec_avx2.c
> > +++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
> > @@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq) {
> > int i;
> > uint16_t rx_id;
> > - volatile union ice_rx_desc *rxdp;
> > + volatile union ice_rx_flex_desc *rxdp;
> > struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
> >
> > - rxdp = rxq->rx_ring + rxq->rxrearm_start;
> > + rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
>
> Since after this patch, all data paths (normal, sse, avx2) are moved to flex desc,
> Ice_rx_desc is not used anymore, so can replace all of them with ice_rx_flex_desc,
> then above convention can be avoid.
>
I think we can submit another minimal patch set to clean up ice_rx_desc
and fix this kind of convention. Since for developing in parallel, Reyi
and I have to use this kind of convention. :)
> <.......>
>
> > * take the two sets of status bits and merge to one @@ -450,20
>
> >
> > /* merge flags */
> > const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
> > --
> > 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-30 1:05 ` Wang, Haiyue
@ 2019-08-30 1:06 ` Zhang, Qi Z
0 siblings, 0 replies; 54+ messages in thread
From: Zhang, Qi Z @ 2019-08-30 1:06 UTC (permalink / raw)
To: Wang, Haiyue, Rong, Leyi, Ye, Xiaolong, Lu, Wenzhuo; +Cc: dev
> -----Original Message-----
> From: Wang, Haiyue
> Sent: Friday, August 30, 2019 9:06 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Rong, Leyi <leyi.rong@intel.com>;
> Ye, Xiaolong <xiaolong.ye@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX
> path
>
> > -----Original Message-----
> > From: Zhang, Qi Z
> > Sent: Friday, August 30, 2019 07:32
> > To: Rong, Leyi <leyi.rong@intel.com>; Ye, Xiaolong
> > <xiaolong.ye@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu,
> > Wenzhuo <wenzhuo.lu@intel.com>
> > Cc: dev@dpdk.org
> > Subject: RE: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor
> > in AVX path
> >
> >
> >
> > > -----Original Message-----
> > > From: Rong, Leyi
> > > Sent: Thursday, August 29, 2019 4:05 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> > > <xiaolong.ye@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu,
> > > Wenzhuo <wenzhuo.lu@intel.com>
> > > Cc: dev@dpdk.org; Rong, Leyi <leyi.rong@intel.com>
> > > Subject: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in
> > > AVX path
> > >
> > > Switch to Rx flexible descriptor format instead of legacy descriptor
> format.
> > >
> > > Signed-off-by: Leyi Rong <leyi.rong@intel.com>
> > > ---
> > > drivers/net/ice/ice_rxtx_vec_avx2.c | 232
> > > ++++++++++++++--------------
> > > 1 file changed, 118 insertions(+), 114 deletions(-)
> > >
> > > diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c
> > > b/drivers/net/ice/ice_rxtx_vec_avx2.c
> > > index 5ce29c2a2..158f17d80 100644
> > > --- a/drivers/net/ice/ice_rxtx_vec_avx2.c
> > > +++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
> > > @@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq) {
> > > int i;
> > > uint16_t rx_id;
> > > - volatile union ice_rx_desc *rxdp;
> > > + volatile union ice_rx_flex_desc *rxdp;
> > > struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
> > >
> > > - rxdp = rxq->rx_ring + rxq->rxrearm_start;
> > > + rxdp = (union ice_rx_flex_desc *)rxq->rx_ring +
> > > +rxq->rxrearm_start;
> >
> > Since after this patch, all data paths (normal, sse, avx2) are moved
> > to flex desc, Ice_rx_desc is not used anymore, so can replace all of
> > them with ice_rx_flex_desc, then above convention can be avoid.
> >
>
> I think we can submit another minimal patch set to clean up ice_rx_desc and
> fix this kind of convention. Since for developing in parallel, Reyi and I have to
> use this kind of convention. :)
+1
>
> > <.......>
> >
> > > * take the two sets of status bits and merge to one @@
> -450,20
> >
> > >
> > > /* merge flags */
> > > const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
> > > --
> > > 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-08-29 23:31 ` Zhang, Qi Z
2019-08-30 1:05 ` Wang, Haiyue
@ 2019-08-30 6:17 ` Rong, Leyi
1 sibling, 0 replies; 54+ messages in thread
From: Rong, Leyi @ 2019-08-30 6:17 UTC (permalink / raw)
To: Zhang, Qi Z, Ye, Xiaolong, Wang, Haiyue, Lu, Wenzhuo; +Cc: dev
> -----Original Message-----
> From: Zhang, Qi Z
> Sent: Friday, August 30, 2019 7:32 AM
> To: Rong, Leyi <leyi.rong@intel.com>; Ye, Xiaolong <xiaolong.ye@intel.com>;
> Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path
>
>
> > * take the two sets of status bits and merge to one @@ -
> 450,20
> > +452,22 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq,
> > +struct
> > rte_mbuf **rx_pkts,
> > /* get only flag/error bits we want */
> > const __m256i flag_bits =
> > _mm256_and_si256(status0_7, flags_mask);
> > - /* set vlan and rss flags */
> > - const __m256i vlan_flags =
> > - _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
> > - const __m256i rss_flags =
> > - _mm256_shuffle_epi8(rss_flags_shuf,
> > - _mm256_srli_epi32(flag_bits, 11));
> > /**
> > * l3_l4_error flags, shuffle, then shift to correct adjustment
> > * of flags in flags_shuf, and finally mask out extra bits
> > */
> > __m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
> > - _mm256_srli_epi32(flag_bits, 22));
> > + _mm256_srli_epi32(flag_bits, 4));
> > l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
> > l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
> > + /* set rss and vlan flags */
> > + const __m256i rss_vlan_flag_bits =
> > + _mm256_srli_epi32(flag_bits, 12);
> > + const __m256i rss_flags =
> > + _mm256_shuffle_epi8(rss_flags_shuf,
> rss_vlan_flag_bits);
> > + const __m256i vlan_flags =
> > + _mm256_shuffle_epi8(vlan_flags_shuf,
> > + rss_vlan_flag_bits);
>
> Seems we can "or" rss_flags_shuf and vlan_flags_shuf, so just need to do one
> shuffle here to save some CPU cycles?
>
That's make sense literally, will do some benchmarking test for this adjustment:).
> >
> > /* merge flags */
> > const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
> > --
> > 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (5 preceding siblings ...)
2019-08-29 2:34 ` [dpdk-dev] [PATCH 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition Leyi Rong
` (4 more replies)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
8 siblings, 5 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
This patchset enable Rx flexible descriptor for ice PMD
in both normal path and vector path.
Depends on shared code update patchset.
---
v3:
- Parse the ‘proto_xtr’ devargs before CVL HW initialization and
save it for later accessing.
- Merge shuffle ops on vlan and rss flag on vector path.
v2:
- Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
when handling Rx packets.
Haiyue Wang (3):
net/ice: add Rx flex descriptor definition
net/ice: handle the Rx flex descriptor
net/ice: add protocol extraction support for per Rx queue
Leyi Rong (1):
net/ice: switch to Rx flexible descriptor in AVX path
Wenzhuo Lu (1):
net/ice: switch to flexible descriptor in SSE path
doc/guides/nics/ice.rst | 101 +++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 312 ++++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 5 +
drivers/net/ice/ice_rxtx.c | 297 +++++++++++++-----------
drivers/net/ice/ice_rxtx.h | 4 +
drivers/net/ice/ice_rxtx_vec_avx2.c | 224 +++++++++---------
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/ice_rxtx_vec_sse.c | 239 +++++++++-----------
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
11 files changed, 973 insertions(+), 369 deletions(-)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
--
2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
2019-09-18 21:56 ` Ye Xiaolong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 2/5] net/ice: handle the Rx flex descriptor Leyi Rong
` (3 subsequent siblings)
4 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The Rx flex descriptor has 16B and 32B size, with different
field definitions compared to legacy type.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index e9214110c..64e891875 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,8 +21,10 @@
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
#define ice_rx_desc ice_16byte_rx_desc
+#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
#define ice_rx_desc ice_32byte_rx_desc
+#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
#define ICE_SUPPORT_CHAIN_NUM 5
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 2/5] net/ice: handle the Rx flex descriptor
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue Leyi Rong
` (2 subsequent siblings)
4 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Set the RXDID with flex descriptor type by default, change the Rx
function to support new descriptor handling.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 236 +++++++++++++++++--------------------
1 file changed, 111 insertions(+), 125 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 0282b5375..d2e36853f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,7 +13,6 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
-#define ICE_RX_ERR_BITS 0x3f
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -25,18 +24,9 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
enum ice_status err;
uint16_t buf_size, len;
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ uint32_t rxdid = ICE_RXDID_COMMS_GENERIC;
uint32_t regval;
- /**
- * The kernel driver uses flex descriptor. It sets the register
- * to flex descriptor mode.
- * DPDK uses legacy descriptor. It should set the register back
- * to the default value, then uses legacy descriptor mode.
- */
- regval = (0x01 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
- QRXFLXP_CNTXT_RXDID_PRIO_M;
- ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
-
/* Set buffer size as the head split is disabled. */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
RTE_PKTMBUF_HEADROOM);
@@ -94,6 +84,21 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile ID;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
+
err = ice_clear_rxq_ctx(hw, rxq->reg_idx);
if (err) {
PMD_DRV_LOG(ERR, "Failed to clear Lan Rx queue (%u) context",
@@ -961,16 +966,15 @@ uint32_t
ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
#define ICE_RXQ_SCAN_INTERVAL 4
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq;
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
- ((rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S) &
- (1 << ICE_RX_DESC_STATUS_DD_S)) {
+ rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
/**
* Check the DD bit of a rx descriptor of each 4 in a group,
* to avoid checking too frequently and downgrading performance
@@ -979,79 +983,77 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = (volatile union ice_rx_flex_desc *)
+ &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
return desc;
}
-/* Translate the rx descriptor status to pkt flags */
-static inline uint64_t
-ice_rxd_status_to_pkt_flags(uint64_t qword)
-{
- uint64_t flags;
-
- /* Check if RSS_HASH */
- flags = (((qword >> ICE_RX_DESC_STATUS_FLTSTAT_S) &
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ==
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ? PKT_RX_RSS_HASH : 0;
-
- return flags;
-}
+#define ICE_RX_FLEX_ERR0_BITS \
+ ((1 << ICE_RX_FLEX_DESC_STATUS0_HBO_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_RXE_S))
/* Rx L3/L4 checksum */
static inline uint64_t
-ice_rxd_error_to_pkt_flags(uint64_t qword)
+ice_rxd_error_to_pkt_flags(uint16_t stat_err0)
{
uint64_t flags = 0;
- uint64_t error_bits = (qword >> ICE_RXD_QW1_ERROR_S);
- if (likely((error_bits & ICE_RX_ERR_BITS) == 0)) {
+ /* check if HW has decoded the packet and checksum */
+ if (unlikely(!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return 0;
+
+ if (likely(!(stat_err0 & ICE_RX_FLEX_ERR0_BITS))) {
flags |= (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD);
return flags;
}
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_IPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
flags |= PKT_RX_IP_CKSUM_BAD;
else
flags |= PKT_RX_IP_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_L4E_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
flags |= PKT_RX_L4_CKSUM_BAD;
else
flags |= PKT_RX_L4_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_EIPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
flags |= PKT_RX_EIP_CKSUM_BAD;
return flags;
}
static inline void
-ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
+ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
{
- if (rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- (1 << ICE_RX_DESC_STATUS_L2TAG1P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S)) {
mb->ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED;
mb->vlan_tci =
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1);
+ rte_le_to_cpu_16(rxdp->wb.l2tag1);
PMD_RX_LOG(DEBUG, "Descriptor l2tag1: %u",
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1));
+ rte_le_to_cpu_16(rxdp->wb.l2tag1));
} else {
mb->vlan_tci = 0;
}
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
- if (rte_le_to_cpu_16(rxdp->wb.qword2.ext_status) &
- (1 << ICE_RX_DESC_EXT_STATUS_L2TAG2P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error1) &
+ (1 << ICE_RX_FLEX_DESC_STATUS1_L2TAG2P_S)) {
mb->ol_flags |= PKT_RX_QINQ_STRIPPED | PKT_RX_QINQ |
PKT_RX_VLAN_STRIPPED | PKT_RX_VLAN;
mb->vlan_tci_outer = mb->vlan_tci;
- mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2);
+ mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd);
PMD_RX_LOG(DEBUG, "Descriptor l2tag2_1: %u, l2tag2_2: %u",
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_1),
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2));
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_1st),
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd));
} else {
mb->vlan_tci_outer = 0;
}
@@ -1060,6 +1062,21 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+static inline void
+ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
+ volatile union ice_rx_flex_desc *rxdp)
+{
+ volatile struct ice_32b_rx_flex_desc_comms *desc =
+ (volatile struct ice_32b_rx_flex_desc_comms *)rxdp;
+ uint16_t stat_err;
+
+ stat_err = rte_le_to_cpu_16(desc->status_error0);
+ if (likely(stat_err & (1 << ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S))) {
+ mb->ol_flags |= PKT_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
+ }
+}
+
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
#define ICE_LOOK_AHEAD 8
#if (ICE_LOOK_AHEAD != 8)
@@ -1068,25 +1085,23 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
static inline int
ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
+ uint16_t stat_err0;
uint16_t pkt_len;
- uint64_t qword1;
- uint32_t rx_status;
int32_t s[ICE_LOOK_AHEAD], nb_dd;
int32_t i, j, nb_rx = 0;
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S;
+ stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Make sure there is at least 1 packet to receive */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/**
@@ -1096,42 +1111,31 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
for (i = 0; i < ICE_RX_MAX_BURST; i += ICE_LOOK_AHEAD,
rxdp += ICE_LOOK_AHEAD, rxep += ICE_LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--) {
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- s[j] = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
- }
+ for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--)
+ s[j] = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
rte_smp_rmb();
/* Compute how many status bits were set */
for (j = 0, nb_dd = 0; j < ICE_LOOK_AHEAD; j++)
- nb_dd += s[j] & (1 << ICE_RX_DESC_STATUS_DD_S);
+ nb_dd += s[j] & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S);
nb_rx += nb_dd;
/* Translate descriptor info to mbuf parameters */
for (j = 0; j < nb_dd; j++) {
mb = rxep[j].mbuf;
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- pkt_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ pkt_len = (rte_le_to_cpu_16(rxdp[j].wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
mb->data_len = pkt_len;
mb->pkt_len = pkt_len;
mb->ol_flags = 0;
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- mb->hash.rss =
- rte_le_to_cpu_32(
- rxdp[j].wb.qword0.hi_dword.rss);
- mb->packet_type = ptype_tbl[(uint8_t)(
- (qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
+ stat_err0 = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
+ pkt_flags = ice_rxd_error_to_pkt_flags(stat_err0);
+ mb->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxdp[j].wb.ptype_flex_flags0)];
ice_rxd_to_vlan_tci(mb, &rxdp[j]);
+ ice_rxd_to_pkt_fields(mb, &rxdp[j]);
mb->ol_flags |= pkt_flags;
}
@@ -1312,8 +1316,8 @@ ice_recv_scattered_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *first_seg = rxq->pkt_first_seg;
@@ -1324,21 +1328,18 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1377,14 +1378,10 @@ ice_recv_scattered_pkts(void *rx_queue,
/* Set data buffer address and data length of the mbuf */
rxdp->read.hdr_addr = 0;
rxdp->read.pkt_addr = dma_addr;
- rx_packet_len = (qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S;
+ rx_packet_len = rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
rxm->data_len = rx_packet_len;
rxm->data_off = RTE_PKTMBUF_HEADROOM;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
/**
* If this is the first buffer of the received packet, set the
@@ -1410,7 +1407,7 @@ ice_recv_scattered_pkts(void *rx_queue,
* update the pointer to the last mbuf of the current scattered
* packet and continue to parse the RX ring.
*/
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_EOF_S))) {
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_EOF_S))) {
last_seg = rxm;
continue;
}
@@ -1442,13 +1439,11 @@ ice_recv_scattered_pkts(void *rx_queue,
first_seg->port = rxq->port_id;
first_seg->ol_flags = 0;
-
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- first_seg->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
-
+ first_seg->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(first_seg, &rxd);
+ ice_rxd_to_pkt_fields(first_seg, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
first_seg->ol_flags |= pkt_flags;
/* Prefetch data of first segment, if configured to do so. */
rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr,
@@ -1538,9 +1533,8 @@ ice_dev_supported_ptypes_get(struct rte_eth_dev *dev)
int
ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq = rx_queue;
- volatile uint64_t *status;
- uint64_t mask;
uint32_t desc;
if (unlikely(offset >= rxq->nb_rx_desc))
@@ -1553,10 +1547,9 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- status = &rxq->rx_ring[desc].wb.qword1.status_error_len;
- mask = rte_cpu_to_le_64((1ULL << ICE_RX_DESC_STATUS_DD_S) <<
- ICE_RXD_QW1_STATUS_S);
- if (*status & mask)
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
return RTE_ETH_RX_DESC_AVAIL;
@@ -1642,8 +1635,8 @@ ice_recv_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *nmb; /* new allocated mbuf */
@@ -1652,21 +1645,18 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1696,8 +1686,8 @@ ice_recv_pkts(void *rx_queue,
rxdp->read.pkt_addr = dma_addr;
/* calculate rx_packet_len of the received pkt */
- rx_packet_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ rx_packet_len = (rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
/* fill old mbuf with received descriptor: rxd */
rxm->data_off = RTE_PKTMBUF_HEADROOM;
@@ -1707,15 +1697,11 @@ ice_recv_pkts(void *rx_queue,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- rxm->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
+ rxm->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(rxm, &rxd);
+ ice_rxd_to_pkt_fields(rxm, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
rxm->ol_flags |= pkt_flags;
/* copy old mbuf to rx_pkts */
rx_pkts[nb_rx++] = rxm;
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 2/5] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
2019-09-18 23:30 ` Ye Xiaolong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 4/5] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 5/5] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
4 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The ice has the feature to extract protocol fields into flex descriptor
by programming per queue. Currently, the ice PMD will put the protocol
fields into rte_mbuf::udata64 with different type format. Application
can access the protocol fields quickly.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
doc/guides/nics/ice.rst | 101 +++++++++
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 312 ++++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 5 +
drivers/net/ice/ice_rxtx.c | 61 +++++
drivers/net/ice/ice_rxtx.h | 2 +
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
9 files changed, 641 insertions(+)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
index 03819d29f..8a6f60e71 100644
--- a/doc/guides/nics/ice.rst
+++ b/doc/guides/nics/ice.rst
@@ -61,6 +61,107 @@ Runtime Config Options
NOTE: In Safe mode, only very limited features are available, features like RSS,
checksum, fdir, tunneling ... are all disabled.
+- ``Protocol extraction for per queue``
+
+ Configure the RX queues to do protocol extraction into ``rte_mbuf::udata64``
+ for protocol handling acceleration, like checking the TCP SYN packets quickly.
+
+ The argument format is::
+
+ -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
+ -w 18:00.0,proto_xtr=<protocol>
+
+ Queues are grouped by ``(`` and ``)`` within the group. The ``-`` character
+ is used as a range separator and ``,`` is used as a single number separator.
+ The grouping ``()`` can be omitted for single element group. If no queues are
+ specified, PMD will use this protocol extraction type for all queues.
+
+ Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-13 are
+ VLAN extraction, other queues run with no protocol extraction.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-23 are
+ IPv6 extraction, other queues use the default VLAN extraction.
+
+ The extraction will be copied into the lower 32 bit of ``rte_mbuf::udata64``.
+
+ .. table:: Protocol extraction : ``vlan``
+
+ +----------------------------+----------------------------+
+ | VLAN2 | VLAN1 |
+ +======+===+=================+======+===+=================+
+ | PCP | D | VID | PCP | D | VID |
+ +------+---+-----------------+------+---+-----------------+
+
+ VLAN1 - single or EVLAN (first for QinQ).
+
+ VLAN2 - C-VLAN (second for QinQ).
+
+ .. table:: Protocol extraction : ``ipv4``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +======+=======+=============+==============+=============+
+ | Ver |Hdr Len| ToS | TTL | Protocol |
+ +------+-------+-------------+--------------+-------------+
+
+ IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
+
+ IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
+
+ .. table:: Protocol extraction : ``ipv6``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+=============+==============+
+ | Ver |Traffic class| Flow | Next Header | Hop Limit |
+ +-----+-------------+--------+-------------+--------------+
+
+ IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``ipv6_flow``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+============================+
+ | Ver |Traffic class| Flow Label |
+ +-----+-------------+-------------------------------------+
+
+ IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``tcp``
+
+ +----------------------------+----------------------------+
+ | TCPHDR2 | TCPHDR1 |
+ +============================+======+======+==============+
+ | Reserved |Offset| RSV | Flags |
+ +----------------------------+------+------+--------------+
+
+ TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
+
+ TCPHDR2 - Reserved
+
+ Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the protocol
+ extraction, do not use ``rte_mbuf::udata64`` directly.
+
+ The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
+ access the protocol extraction result in ``struct rte_mbuf``.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile
index ae53c2646..4a279f196 100644
--- a/drivers/net/ice/Makefile
+++ b/drivers/net/ice/Makefile
@@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1)
endif
SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
+
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..83b6e596e 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -19,9 +19,11 @@
/* devargs */
#define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
+#define ICE_PROTO_XTR_ARG "proto_xtr"
static const char * const ice_valid_args[] = {
ICE_SAFE_MODE_SUPPORT_ARG,
+ ICE_PROTO_XTR_ARG,
NULL
};
@@ -257,6 +259,293 @@ ice_init_controlq_parameter(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ;
}
+static int
+lookup_proto_xtr_type(const char *xtr_name)
+{
+ static struct {
+ const char *name;
+ enum proto_xtr_type type;
+ } xtr_type_map[] = {
+ { "vlan", PROTO_XTR_VLAN },
+ { "ipv4", PROTO_XTR_IPV4 },
+ { "ipv6", PROTO_XTR_IPV6 },
+ { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
+ { "tcp", PROTO_XTR_TCP },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
+ if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
+ return xtr_type_map[i].type;
+ }
+
+ return -1;
+}
+
+/*
+ * Parse elem, the elem could be single number/range or '(' ')' group
+ * 1) A single number elem, it's just a simple digit. e.g. 9
+ * 2) A single range elem, two digits with a '-' between. e.g. 2-6
+ * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
+ * Within group elem, '-' used for a range separator;
+ * ',' used for a single number.
+ */
+static int
+parse_queue_set(const char *input, int xtr_type, struct ice_devargs *devargs)
+{
+ const char *str = input;
+ char *end = NULL;
+ uint32_t min, max;
+ uint32_t idx;
+
+ while (isblank(*str))
+ str++;
+
+ if ((!isdigit(*str) && *str != '(') || (*str == '\0'))
+ return -1;
+
+ /* process single number or single range of number */
+ if (*str != '(') {
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ while (isblank(*end))
+ end++;
+
+ min = idx;
+ max = idx;
+
+ /* process single <number>-<number> */
+ if (*end == '-') {
+ end++;
+ while (isblank(*end))
+ end++;
+ if (!isdigit(*end))
+ return -1;
+
+ errno = 0;
+ idx = strtoul(end, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ max = idx;
+ while (isblank(*end))
+ end++;
+ }
+
+ if (*end != ':')
+ return -1;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ return 0;
+ }
+
+ /* process set within bracket */
+ str++;
+ while (isblank(*str))
+ str++;
+ if (*str == '\0')
+ return -1;
+
+ min = ICE_MAX_QUEUE_NUM;
+ do {
+ /* go ahead to the first digit */
+ while (isblank(*str))
+ str++;
+ if (!isdigit(*str))
+ return -1;
+
+ /* get the digit value */
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ /* go ahead to separator '-',',' and ')' */
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+ else /* avoid continuous '-' */
+ return -1;
+ } else if (*end == ',' || *end == ')') {
+ max = idx;
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ min = ICE_MAX_QUEUE_NUM;
+ } else {
+ return -1;
+ }
+
+ str = end + 1;
+ } while (*end != ')' && *end != '\0');
+
+ return 0;
+}
+
+static int
+parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
+{
+ const char *queue_start;
+ uint32_t idx;
+ int xtr_type;
+ char xtr_name[32];
+
+ while (isblank(*queues))
+ queues++;
+
+ if (*queues != '[') {
+ xtr_type = lookup_proto_xtr_type(queues);
+ if (xtr_type < 0)
+ return -1;
+
+ devargs->proto_xtr_dflt = xtr_type;
+ return 0;
+ }
+
+ queues++;
+ do {
+ while (isblank(*queues))
+ queues++;
+ if (*queues == '\0')
+ return -1;
+
+ queue_start = queues;
+
+ /* go across a complete bracket */
+ if (*queue_start == '(') {
+ queues += strcspn(queues, ")");
+ if (*queues != ')')
+ return -1;
+ }
+
+ /* scan the separator ':' */
+ queues += strcspn(queues, ":");
+ if (*queues++ != ':')
+ return -1;
+ while (isblank(*queues))
+ queues++;
+
+ for (idx = 0; ; idx++) {
+ if (isblank(queues[idx]) ||
+ queues[idx] == ',' ||
+ queues[idx] == ']' ||
+ queues[idx] == '\0')
+ break;
+
+ if (idx > sizeof(xtr_name) - 2)
+ return -1;
+
+ xtr_name[idx] = queues[idx];
+ }
+ xtr_name[idx] = '\0';
+ xtr_type = lookup_proto_xtr_type(xtr_name);
+ if (xtr_type < 0)
+ return -1;
+
+ queues += idx;
+
+ while (isblank(*queues) || *queues == ',' || *queues == ']')
+ queues++;
+
+ if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
+ return -1;
+ } while (*queues != '\0');
+
+ return 0;
+}
+
+static int
+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
+ void *extra_args)
+{
+ struct ice_devargs *devargs = extra_args;
+
+ if (value == NULL || extra_args == NULL)
+ return -EINVAL;
+
+ if (parse_queue_proto_xtr(value, devargs) < 0) {
+ PMD_DRV_LOG(ERR,
+ "The protocol extraction parameter is wrong : '%s'",
+ value);
+ return -1;
+ }
+
+ return 0;
+}
+
+static void
+ice_parse_proto_xtr_devarg(struct rte_kvargs *kvlist,
+ struct ice_devargs *devargs)
+{
+ int i;
+
+ devargs->proto_xtr_dflt = PROTO_XTR_NONE;
+
+ for (i = 0; i < ICE_MAX_QUEUE_NUM; i++)
+ devargs->proto_xtr[i] = PROTO_XTR_NONE;
+
+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
+ handle_proto_xtr_arg, devargs);
+}
+
+static bool
+ice_proto_xtr_support(struct ice_hw *hw)
+{
+#define FLX_REG(val, fld, idx) \
+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
+ static struct {
+ uint32_t rxdid;
+ uint16_t protid_0;
+ uint16_t protid_1;
+ } xtr_sets[] = {
+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
+ ICE_PROT_IPV4_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
+ uint32_t rxdid = xtr_sets[i].rxdid;
+ uint32_t v;
+
+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+
+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+ }
+
+ return true;
+}
+
static int
ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
uint32_t num)
@@ -1079,6 +1368,8 @@ ice_interrupt_handler(void *param)
static int
ice_pf_sw_init(struct rte_eth_dev *dev)
{
+ struct ice_adapter *ad =
+ ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct ice_hw *hw = ICE_PF_TO_HW(pf);
@@ -1088,6 +1379,21 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
pf->lan_nb_qps = pf->lan_nb_qp_max;
+ if (ice_proto_xtr_support(hw))
+ pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
+
+ if (pf->proto_xtr != NULL) {
+ uint16_t i;
+
+ for (i = 0; i < pf->lan_nb_qps; i++)
+ pf->proto_xtr[i] =
+ ad->devargs.proto_xtr[i] != PROTO_XTR_NONE ?
+ ad->devargs.proto_xtr[i] :
+ ad->devargs.proto_xtr_dflt;
+ } else {
+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
+ }
+
return 0;
}
@@ -1378,6 +1684,8 @@ static int ice_parse_devargs(struct rte_eth_dev *dev)
return -EINVAL;
}
+ ice_parse_proto_xtr_devarg(kvlist, &ad->devargs);
+
ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
&parse_bool, &ad->devargs.safe_mode_support);
@@ -1547,6 +1855,7 @@ ice_dev_init(struct rte_eth_dev *dev)
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
return ret;
}
@@ -1672,6 +1981,8 @@ ice_dev_close(struct rte_eth_dev *dev)
rte_free(hw->port_info);
hw->port_info = NULL;
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
+ pf->proto_xtr = NULL;
}
static int
@@ -3795,6 +4106,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ice,
+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
RTE_INIT(ice_init_log)
diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
index f569da833..e58192104 100644
--- a/drivers/net/ice/ice_ethdev.h
+++ b/drivers/net/ice/ice_ethdev.h
@@ -263,6 +263,7 @@ struct ice_pf {
uint16_t lan_nb_qp_max;
uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
uint16_t base_queue; /* The base queue pairs index in the device */
+ uint8_t *proto_xtr; /* Protocol extraction type for all queues */
struct ice_hw_port_stats stats_offset;
struct ice_hw_port_stats stats;
/* internal packet statistics, it should be excluded from the total */
@@ -273,11 +274,15 @@ struct ice_pf {
struct ice_flow_list flow_list;
};
+#define ICE_MAX_QUEUE_NUM 2048
+
/**
* Cache devargs parse result.
*/
struct ice_devargs {
int safe_mode_support;
+ uint8_t proto_xtr_dflt;
+ uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
};
/**
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index d2e36853f..e28310b96 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,6 +13,36 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+static inline uint8_t
+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
+{
+ static uint8_t xtr_map[] = {
+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
+ };
+
+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
+}
+
+static inline uint8_t
+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
+{
+ static uint8_t rxdid_map[] = {
+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
+ };
+ uint8_t rxdid;
+
+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
+
+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
+}
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -84,6 +114,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
+
+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
+ rxq->port_id, rxq->queue_id, rxdid);
+
/* Enable Flexible Descriptors in the queue context which
* allows this driver to select a specific receive descriptor format
*/
@@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
rxq->drop_en = rx_conf->rx_drop_en;
rxq->vsi = vsi;
rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->proto_xtr = pf->proto_xtr != NULL ?
+ pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
/* Allocate the maximun number of RX ring hardware descriptor. */
len = ICE_MAX_RING_DESC;
@@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+#define ICE_RX_PROTO_XTR_VALID \
+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+
static inline void
ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
volatile union ice_rx_flex_desc *rxdp)
@@ -1075,6 +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
mb->ol_flags |= PKT_RX_RSS_HASH;
mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
}
+
+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
+ init_proto_xtr_flds(mb);
+
+ stat_err = rte_le_to_cpu_16(desc->status_error1);
+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
+ xtr->u.raw.data0 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+ xtr->u.raw.data1 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
+
+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
+ xtr->magic = PROTO_XTR_MAGIC_ID;
+ }
+#endif
}
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index 64e891875..de16637f3 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -5,6 +5,7 @@
#ifndef _ICE_RXTX_H_
#define _ICE_RXTX_H_
+#include "rte_pmd_ice.h"
#include "ice_ethdev.h"
#define ICE_ALIGN_RING_DESC 32
@@ -78,6 +79,7 @@ struct ice_rx_queue {
uint16_t max_pkt_len; /* Maximum packet length */
bool q_set; /* indicate if rx queue has been configured */
bool rx_deferred_start; /* don't start this queue in dev start */
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
ice_rx_release_mbufs_t rx_rel_mbufs;
};
diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
index c5f0d564f..080ca4175 100644
--- a/drivers/net/ice/ice_rxtx_vec_common.h
+++ b/drivers/net/ice/ice_rxtx_vec_common.h
@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
if (rxq->nb_rx_desc % rxq->rx_free_thresh)
return -1;
+ if (rxq->proto_xtr != PROTO_XTR_NONE)
+ return -1;
+
return 0;
}
diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
index 36b4b3c85..6828170a9 100644
--- a/drivers/net/ice/meson.build
+++ b/drivers/net/ice/meson.build
@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
endif
endif
+
+install_headers('rte_pmd_ice.h')
diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
new file mode 100644
index 000000000..719487e1e
--- /dev/null
+++ b/drivers/net/ice/rte_pmd_ice.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#ifndef _RTE_PMD_ICE_H_
+#define _RTE_PMD_ICE_H_
+
+#include <stdio.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum proto_xtr_type {
+ PROTO_XTR_NONE,
+ PROTO_XTR_VLAN,
+ PROTO_XTR_IPV4,
+ PROTO_XTR_IPV6,
+ PROTO_XTR_IPV6_FLOW,
+ PROTO_XTR_TCP,
+};
+
+struct proto_xtr_flds {
+ union {
+ struct {
+ uint16_t data0;
+ uint16_t data1;
+ } raw;
+ struct {
+ uint16_t stag_vid:12,
+ stag_dei:1,
+ stag_pcp:3;
+ uint16_t ctag_vid:12,
+ ctag_dei:1,
+ ctag_pcp:3;
+ } vlan;
+ struct {
+ uint16_t protocol:8,
+ ttl:8;
+ uint16_t tos:8,
+ ihl:4,
+ version:4;
+ } ipv4;
+ struct {
+ uint16_t hoplimit:8,
+ nexthdr:8;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6;
+ struct {
+ uint16_t flowlo16;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6_flow;
+ struct {
+ uint16_t fin:1,
+ syn:1,
+ rst:1,
+ psh:1,
+ ack:1,
+ urg:1,
+ ece:1,
+ cwr:1,
+ res1:4,
+ doff:4;
+ uint16_t rsvd;
+ } tcp;
+ } u;
+
+ uint16_t rsvd;
+
+ uint8_t type;
+
+#define PROTO_XTR_MAGIC_ID 0xCE
+ uint8_t magic;
+};
+
+static inline void
+init_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ mb->udata64 = 0;
+}
+
+static inline struct proto_xtr_flds *
+get_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
+
+ return (struct proto_xtr_flds *)&mb->udata64;
+}
+
+static inline void
+dump_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
+ return;
+
+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
+ xtr->u.raw.data0, xtr->u.raw.data1);
+
+ if (xtr->type == PROTO_XTR_VLAN)
+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
+ xtr->u.vlan.stag_pcp,
+ xtr->u.vlan.stag_dei,
+ xtr->u.vlan.stag_vid,
+ xtr->u.vlan.ctag_pcp,
+ xtr->u.vlan.ctag_dei,
+ xtr->u.vlan.ctag_vid);
+ else if (xtr->type == PROTO_XTR_IPV4)
+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
+ xtr->u.ipv4.version,
+ xtr->u.ipv4.ihl,
+ xtr->u.ipv4.tos,
+ xtr->u.ipv4.ttl,
+ xtr->u.ipv4.protocol);
+ else if (xtr->type == PROTO_XTR_IPV6)
+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
+ xtr->u.ipv6.version,
+ xtr->u.ipv6.tc,
+ xtr->u.ipv6.flowhi4,
+ xtr->u.ipv6.nexthdr,
+ xtr->u.ipv6.hoplimit);
+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
+ xtr->u.ipv6_flow.version,
+ xtr->u.ipv6_flow.tc,
+ xtr->u.ipv6_flow.flowhi4,
+ xtr->u.ipv6_flow.flowlo16);
+ else if (xtr->type == PROTO_XTR_TCP)
+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
+ xtr->u.tcp.doff,
+ xtr->u.tcp.cwr ? "C" : "",
+ xtr->u.tcp.ece ? "E" : "",
+ xtr->u.tcp.urg ? "U" : "",
+ xtr->u.tcp.ack ? "A" : "",
+ xtr->u.tcp.psh ? "P" : "",
+ xtr->u.tcp.rst ? "R" : "",
+ xtr->u.tcp.syn ? "S" : "",
+ xtr->u.tcp.fin ? "F" : "");
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_PMD_ICE_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 4/5] net/ice: switch to flexible descriptor in SSE path
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
` (2 preceding siblings ...)
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 5/5] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
4 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
With this path, the flexible descriptor is supported
in SSE path. And the legacy descriptor is not supported.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx_vec_sse.c | 239 +++++++++++++----------------
1 file changed, 110 insertions(+), 129 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index 967a7b16b..dafcb081a 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -15,14 +15,14 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
struct rte_mbuf *mb0, *mb1;
__m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -88,93 +88,90 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer);
__m128i rearm0, rearm1, rearm2, rearm3;
- __m128i vlan0, vlan1, rss, l3_l4e;
+ __m128i tmp_desc, flags, rss_vlan;
- /* mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication.
+ /* mask everything except checksum, RSS and VLAN flags.
+ * bit6:4 for checksum.
+ * bit12 for RSS indication.
+ * bit13 for VLAN indication.
*/
- const __m128i rss_vlan_msk = _mm_set_epi32(0x1c03804, 0x1c03804,
- 0x1c03804, 0x1c03804);
+ const __m128i desc_mask = _mm_set_epi32(0x3070, 0x3070,
+ 0x3070, 0x3070);
- const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD);
- /* map rss and vlan type to rss hash and vlan flag */
- const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- 0, 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
- 0, 0, 0, 0);
-
- const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH, 0, 0,
- 0, 0, PKT_RX_FDIR, 0);
-
- const __m128i l3_l4e_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ /* map the checksum, rss and vlan fields to the checksum, rss
+ * and vlan flag
+ */
+ const __m128i cksum_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
- vlan0 = _mm_unpackhi_epi32(descs[0], descs[1]);
- vlan1 = _mm_unpackhi_epi32(descs[2], descs[3]);
- vlan0 = _mm_unpacklo_epi64(vlan0, vlan1);
-
- vlan1 = _mm_and_si128(vlan0, rss_vlan_msk);
- vlan0 = _mm_shuffle_epi8(vlan_flags, vlan1);
-
- rss = _mm_srli_epi32(vlan1, 11);
- rss = _mm_shuffle_epi8(rss_flags, rss);
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
- l3_l4e = _mm_srli_epi32(vlan1, 22);
- l3_l4e = _mm_shuffle_epi8(l3_l4e_flags, l3_l4e);
+ const __m128i rss_vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
+
+ /* merge 4 descriptors */
+ flags = _mm_unpackhi_epi32(descs[0], descs[1]);
+ tmp_desc = _mm_unpackhi_epi32(descs[2], descs[3]);
+ tmp_desc = _mm_unpacklo_epi64(flags, tmp_desc);
+ tmp_desc = _mm_and_si128(flags, desc_mask);
+
+ /* checksum flags */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 4);
+ flags = _mm_shuffle_epi8(cksum_flags, tmp_desc);
/* then we shift left 1 bit */
- l3_l4e = _mm_slli_epi32(l3_l4e, 1);
- /* we need to mask out the reduntant bits */
- l3_l4e = _mm_and_si128(l3_l4e, cksum_mask);
+ flags = _mm_slli_epi32(flags, 1);
+ /* we need to mask out the reduntant bits introduced by RSS or
+ * VLAN fields.
+ */
+ flags = _mm_and_si128(flags, cksum_mask);
- vlan0 = _mm_or_si128(vlan0, rss);
- vlan0 = _mm_or_si128(vlan0, l3_l4e);
+ /* RSS, VLAN flag */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 8);
+ rss_vlan = _mm_shuffle_epi8(rss_vlan_flags, tmp_desc);
+
+ /* merge the flags */
+ flags = _mm_or_si128(flags, rss_vlan);
/**
* At this point, we have the 4 sets of flags in the low 16-bits
- * of each 32-bit value in vlan0.
+ * of each 32-bit value in flags.
* We want to extract these, and merge them with the mbuf init data
* so we can do a single 16-byte write to the mbuf to set the flags
* and all the other initialization fields. Extracting the
* appropriate flags means that we have to do a shift and blend for
* each mbuf before we do the write.
*/
- rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 8), 0x10);
- rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 4), 0x10);
- rearm2 = _mm_blend_epi16(mbuf_init, vlan0, 0x10);
- rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10);
+ rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x10);
+ rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x10);
+ rearm2 = _mm_blend_epi16(mbuf_init, flags, 0x10);
+ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x10);
/* write the rearm data and the olflags in one write */
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
@@ -187,22 +184,24 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
_mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3);
}
-#define PKTLEN_SHIFT 10
-
static inline void
ice_rx_desc_to_ptype_v(__m128i descs[4], struct rte_mbuf **rx_pkts,
uint32_t *ptype_tbl)
{
- __m128i ptype0 = _mm_unpackhi_epi64(descs[0], descs[1]);
- __m128i ptype1 = _mm_unpackhi_epi64(descs[2], descs[3]);
-
- ptype0 = _mm_srli_epi64(ptype0, 30);
- ptype1 = _mm_srli_epi64(ptype1, 30);
-
- rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 0)];
- rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 8)];
- rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 0)];
- rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 8)];
+ const __m128i ptype_mask = _mm_set_epi16(0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M);
+ __m128i ptype_01 = _mm_unpacklo_epi32(descs[0], descs[1]);
+ __m128i ptype_23 = _mm_unpacklo_epi32(descs[2], descs[3]);
+ __m128i ptype_all = _mm_unpacklo_epi64(ptype_01, ptype_23);
+
+ ptype_all = _mm_and_si128(ptype_all, ptype_mask);
+
+ rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
}
/**
@@ -215,21 +214,39 @@ static inline uint16_t
_ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
uint64_t var;
- __m128i shuf_msk;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
-
__m128i crc_adjust = _mm_set_epi16
- (0, 0, 0, /* ignore non-length fields */
+ (0, 0, 0, /* ignore non-length fields */
-rxq->crc_len, /* sub crc on data_len */
0, /* ignore high-16bits of pkt_len */
-rxq->crc_len, /* sub crc on pkt_len */
- 0, 0 /* ignore pkt_type field */
+ 0, 0 /* ignore pkt_type field */
);
+ const __m128i zero = _mm_setzero_si128();
+ /* mask to shuffle from desc. to mbuf */
+ const __m128i shuf_msk = _mm_set_epi8
+ (0xFF, 0xFF, 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, low 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /* pkt_type set as unknown */
+ );
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+
/**
* compile-time check the above crc_adjust layout is correct.
* NOTE: the first field (lowest address) is given last in set_epi16
@@ -239,7 +256,13 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
- __m128i dd_check, eop_check;
+
+ /* 4 packets DD mask */
+ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ /* 4 packets EOP mask */
+ const __m128i eop_check = _mm_set_epi64x(0x0000000200000002LL,
+ 0x0000000200000002LL);
/* nb_pkts shall be less equal than ICE_MAX_RX_BURST */
nb_pkts = RTE_MIN(nb_pkts, ICE_MAX_RX_BURST);
@@ -250,7 +273,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = rxq->rx_ring + rxq->rx_tail;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
@@ -263,26 +286,10 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
- /* 4 packets DD mask */
- dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL);
-
- /* 4 packets EOP mask */
- eop_check = _mm_set_epi64x(0x0000000200000002LL, 0x0000000200000002LL);
-
- /* mask to shuffle from desc. to mbuf */
- shuf_msk = _mm_set_epi8
- (7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
- );
/**
* Compile-time verify the shuffle mask
* NOTE: some field positions already verified above, but duplicated
@@ -315,7 +322,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
rxdp += ICE_DESCS_PER_LOOP) {
__m128i descs[ICE_DESCS_PER_LOOP];
__m128i pkt_mb1, pkt_mb2, pkt_mb3, pkt_mb4;
- __m128i zero, staterr, sterr_tmp1, sterr_tmp2;
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
/* 2 64 bit or 4 32 bit mbuf pointers in one XMM reg. */
__m128i mbp1;
#if defined(RTE_ARCH_X86_64)
@@ -359,14 +366,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* avoid compiler reorder optimization */
rte_compiler_barrier();
- /* pkt 3,4 shift the pktlen field to be 16-bit aligned*/
- const __m128i len3 = _mm_slli_epi32(descs[3], PKTLEN_SHIFT);
- const __m128i len2 = _mm_slli_epi32(descs[2], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[3] = _mm_blend_epi16(descs[3], len3, 0x80);
- descs[2] = _mm_blend_epi16(descs[2], len2, 0x80);
-
/* D.1 pkt 3,4 convert format from desc to pktmbuf */
pkt_mb4 = _mm_shuffle_epi8(descs[3], shuf_msk);
pkt_mb3 = _mm_shuffle_epi8(descs[2], shuf_msk);
@@ -382,20 +381,11 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
pkt_mb4 = _mm_add_epi16(pkt_mb4, crc_adjust);
pkt_mb3 = _mm_add_epi16(pkt_mb3, crc_adjust);
- /* pkt 1,2 shift the pktlen field to be 16-bit aligned*/
- const __m128i len1 = _mm_slli_epi32(descs[1], PKTLEN_SHIFT);
- const __m128i len0 = _mm_slli_epi32(descs[0], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[1] = _mm_blend_epi16(descs[1], len1, 0x80);
- descs[0] = _mm_blend_epi16(descs[0], len0, 0x80);
-
/* D.1 pkt 1,2 convert format from desc to pktmbuf */
pkt_mb2 = _mm_shuffle_epi8(descs[1], shuf_msk);
pkt_mb1 = _mm_shuffle_epi8(descs[0], shuf_msk);
/* C.2 get 4 pkts staterr value */
- zero = _mm_xor_si128(dd_check, dd_check);
staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
/* D.3 copy final 3,4 data to rx_pkts */
@@ -412,15 +402,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* C* extract and record EOP bit */
if (split_packet) {
- __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0x04, 0x0C,
- 0x00, 0x08);
-
/* and with mask to extract bits, flipping 1-0 */
__m128i eop_bits = _mm_andnot_si128(staterr, eop_check);
/* the staterr values are not in order, as the count
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v3 5/5] net/ice: switch to Rx flexible descriptor in AVX path
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
` (3 preceding siblings ...)
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 4/5] net/ice: switch to flexible descriptor in SSE path Leyi Rong
@ 2019-09-17 8:53 ` Leyi Rong
4 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-17 8:53 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
Switch to Rx flexible descriptor format instead of legacy
descriptor format.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
---
drivers/net/ice/ice_rxtx_vec_avx2.c | 224 ++++++++++++++--------------
1 file changed, 109 insertions(+), 115 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 5ce29c2a2..46776fa12 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -132,8 +132,6 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
ICE_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
}
-#define PKTLEN_SHIFT 10
-
static inline uint16_t
_ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
@@ -144,7 +142,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp =
+ (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
@@ -161,8 +160,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/* constants used in processing loop */
@@ -193,21 +192,23 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i shuf_msk =
_mm256_set_epi8
(/* first descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF, /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF, /*pkt_type set as unknown */
/* second descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /*pkt_type set as unknown */
);
/**
* compile-time check the above crc and shuffle layout is correct.
@@ -225,68 +226,68 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Status/Error flag masks */
/**
- * mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication. Bits 3-5 of error
- * field (bits 22-24) are for IP/L4 checksum errors
+ * mask everything except Checksum Reports, RSS indication
+ * and VLAN indication.
+ * bit6:4 for IP/L4 checksum errors.
+ * bit12 is for RSS indication.
+ * bit13 is for VLAN indication.
*/
const __m256i flags_mask =
- _mm256_set1_epi32((1 << 2) | (1 << 11) |
- (3 << 12) | (7 << 22));
- /**
- * data to be shuffled by result of flag mask. If VLAN bit is set,
- * (bit 2), then position 4 in this array will be used in the
- * destination
- */
- const __m256i vlan_flags_shuf =
- _mm256_set_epi32(0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0,
- 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0);
- /**
- * data to be shuffled by result of flag mask, shifted down 11.
- * If RSS/FDIR bits are set, shuffle moves appropriate flags in
- * place.
- */
- const __m256i rss_flags_shuf =
- _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0,/* end up 128-bits */
- 0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0);
-
+ _mm256_set1_epi32((7 << 4) | (1 << 12) | (1 << 13));
/**
- * data to be shuffled by the result of the flags mask shifted by 22
+ * data to be shuffled by the result of the flags mask shifted by 4
* bits. This gives use the l3_l4 flags.
*/
const __m256i l3_l4_flags_shuf = _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1,
/* second 128-bits */
0, 0, 0, 0, 0, 0, 0, 0,
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
const __m256i cksum_mask =
_mm256_set1_epi32(PKT_RX_IP_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD |
PKT_RX_L4_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_EIP_CKSUM_BAD);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If RSS(bit12)/VLAN(bit13) are set,
+ * shuffle moves appropriate flags in place.
+ */
+ const __m256i rss_vlan_flags_shuf = _mm256_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
RTE_SET_USED(avx_aligned); /* for 32B descriptors we don't use this */
@@ -369,73 +370,66 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
}
/**
- * convert descriptors 4-7 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 4-7 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len6_7 = _mm256_slli_epi32(raw_desc6_7,
- PKTLEN_SHIFT);
- const __m256i len4_5 = _mm256_slli_epi32(raw_desc4_5,
- PKTLEN_SHIFT);
- const __m256i desc6_7 = _mm256_blend_epi16(raw_desc6_7,
- len6_7, 0x80);
- const __m256i desc4_5 = _mm256_blend_epi16(raw_desc4_5,
- len4_5, 0x80);
- __m256i mb6_7 = _mm256_shuffle_epi8(desc6_7, shuf_msk);
- __m256i mb4_5 = _mm256_shuffle_epi8(desc4_5, shuf_msk);
+ __m256i mb6_7 = _mm256_shuffle_epi8(raw_desc6_7, shuf_msk);
+ __m256i mb4_5 = _mm256_shuffle_epi8(raw_desc4_5, shuf_msk);
mb6_7 = _mm256_add_epi16(mb6_7, crc_adjust);
mb4_5 = _mm256_add_epi16(mb4_5, crc_adjust);
/**
- * to get packet types, shift 64-bit values down 30 bits
- * and so ptype is in lower 8-bits in each
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
*/
- const __m256i ptypes6_7 = _mm256_srli_epi64(desc6_7, 30);
- const __m256i ptypes4_5 = _mm256_srli_epi64(desc4_5, 30);
- const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 24);
- const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 8);
- const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 24);
- const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 8);
+ const __m256i ptype_mask =
+ _mm256_set1_epi16(ICE_RX_FLEX_DESC_PTYPE_M);
+ const __m256i ptypes6_7 =
+ _mm256_and_si256(raw_desc6_7, ptype_mask);
+ const __m256i ptypes4_5 =
+ _mm256_and_si256(raw_desc4_5, ptype_mask);
+ const uint16_t ptype7 = _mm256_extract_epi16(ptypes6_7, 9);
+ const uint16_t ptype6 = _mm256_extract_epi16(ptypes6_7, 1);
+ const uint16_t ptype5 = _mm256_extract_epi16(ptypes4_5, 9);
+ const uint16_t ptype4 = _mm256_extract_epi16(ptypes4_5, 1);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype7], 4);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype6], 0);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype5], 4);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype4], 0);
/* merge the status bits into one register */
- const __m256i status4_7 = _mm256_unpackhi_epi32(desc6_7,
- desc4_5);
+ const __m256i status4_7 = _mm256_unpackhi_epi32(raw_desc6_7,
+ raw_desc4_5);
/**
- * convert descriptors 0-3 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 0-3 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len2_3 = _mm256_slli_epi32(raw_desc2_3,
- PKTLEN_SHIFT);
- const __m256i len0_1 = _mm256_slli_epi32(raw_desc0_1,
- PKTLEN_SHIFT);
- const __m256i desc2_3 = _mm256_blend_epi16(raw_desc2_3,
- len2_3, 0x80);
- const __m256i desc0_1 = _mm256_blend_epi16(raw_desc0_1,
- len0_1, 0x80);
- __m256i mb2_3 = _mm256_shuffle_epi8(desc2_3, shuf_msk);
- __m256i mb0_1 = _mm256_shuffle_epi8(desc0_1, shuf_msk);
+ __m256i mb2_3 = _mm256_shuffle_epi8(raw_desc2_3, shuf_msk);
+ __m256i mb0_1 = _mm256_shuffle_epi8(raw_desc0_1, shuf_msk);
mb2_3 = _mm256_add_epi16(mb2_3, crc_adjust);
mb0_1 = _mm256_add_epi16(mb0_1, crc_adjust);
- /* get the packet types */
- const __m256i ptypes2_3 = _mm256_srli_epi64(desc2_3, 30);
- const __m256i ptypes0_1 = _mm256_srli_epi64(desc0_1, 30);
- const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 24);
- const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 8);
- const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 24);
- const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 8);
+ /**
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
+ */
+ const __m256i ptypes2_3 =
+ _mm256_and_si256(raw_desc2_3, ptype_mask);
+ const __m256i ptypes0_1 =
+ _mm256_and_si256(raw_desc0_1, ptype_mask);
+ const uint16_t ptype3 = _mm256_extract_epi16(ptypes2_3, 9);
+ const uint16_t ptype2 = _mm256_extract_epi16(ptypes2_3, 1);
+ const uint16_t ptype1 = _mm256_extract_epi16(ptypes0_1, 9);
+ const uint16_t ptype0 = _mm256_extract_epi16(ptypes0_1, 1);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype3], 4);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype2], 0);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype1], 4);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype0], 0);
/* merge the status bits into one register */
- const __m256i status0_3 = _mm256_unpackhi_epi32(desc2_3,
- desc0_1);
+ const __m256i status0_3 = _mm256_unpackhi_epi32(raw_desc2_3,
+ raw_desc0_1);
/**
* take the two sets of status bits and merge to one
@@ -450,24 +444,24 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* get only flag/error bits we want */
const __m256i flag_bits =
_mm256_and_si256(status0_7, flags_mask);
- /* set vlan and rss flags */
- const __m256i vlan_flags =
- _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
- const __m256i rss_flags =
- _mm256_shuffle_epi8(rss_flags_shuf,
- _mm256_srli_epi32(flag_bits, 11));
/**
* l3_l4_error flags, shuffle, then shift to correct adjustment
* of flags in flags_shuf, and finally mask out extra bits
*/
__m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
- _mm256_srli_epi32(flag_bits, 22));
+ _mm256_srli_epi32(flag_bits, 4));
l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
+ /* set rss and vlan flags */
+ const __m256i rss_vlan_flag_bits =
+ _mm256_srli_epi32(flag_bits, 12);
+ const __m256i rss_vlan_flags =
+ _mm256_shuffle_epi8(rss_vlan_flags_shuf,
+ rss_vlan_flag_bits);
/* merge flags */
const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
- _mm256_or_si256(rss_flags, vlan_flags));
+ rss_vlan_flags);
/**
* At this point, we have the 8 sets of flags in the low 16-bits
* of each 32-bit value in vlan0.
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-09-18 21:56 ` Ye Xiaolong
0 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-18 21:56 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/17, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
>The Rx flex descriptor has 16B and 32B size, with different
>field definitions compared to legacy type.
>
>Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
>---
> drivers/net/ice/ice_rxtx.h | 2 ++
> 1 file changed, 2 insertions(+)
>
>diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
>index e9214110c..64e891875 100644
>--- a/drivers/net/ice/ice_rxtx.h
>+++ b/drivers/net/ice/ice_rxtx.h
>@@ -21,8 +21,10 @@
>
> #ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
> #define ice_rx_desc ice_16byte_rx_desc
>+#define ice_rx_flex_desc ice_16b_rx_flex_desc
> #else
> #define ice_rx_desc ice_32byte_rx_desc
>+#define ice_rx_flex_desc ice_32b_rx_flex_desc
> #endif
>
> #define ICE_SUPPORT_CHAIN_NUM 5
>--
>2.17.1
>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-18 23:30 ` Ye Xiaolong
2019-09-19 1:36 ` Wang, Haiyue
2019-09-19 1:44 ` Wang, Haiyue
0 siblings, 2 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-18 23:30 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/17, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
>The ice has the feature to extract protocol fields into flex descriptor
>by programming per queue. Currently, the ice PMD will put the protocol
>fields into rte_mbuf::udata64 with different type format. Application
>can access the protocol fields quickly.
>
[snip]
>+static int
>+parse_queue_set(const char *input, int xtr_type, struct ice_devargs *devargs)
>+{
>+ const char *str = input;
>+ char *end = NULL;
>+ uint32_t min, max;
>+ uint32_t idx;
>+
>+ while (isblank(*str))
>+ str++;
>+
>+ if ((!isdigit(*str) && *str != '(') || (*str == '\0'))
Minor nit, (*str == '\0') seems redundant here, no?
>+ return -1;
>+
>+ /* process single number or single range of number */
>+ if (*str != '(') {
>+ errno = 0;
>+ idx = strtoul(str, &end, 10);
>+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
>+ return -1;
>+
>+ while (isblank(*end))
>+ end++;
>+
>+ min = idx;
>+ max = idx;
>+
>+ /* process single <number>-<number> */
>+ if (*end == '-') {
>+ end++;
>+ while (isblank(*end))
>+ end++;
>+ if (!isdigit(*end))
>+ return -1;
>+
>+ errno = 0;
>+ idx = strtoul(end, &end, 10);
>+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
>+ return -1;
>+
>+ max = idx;
>+ while (isblank(*end))
>+ end++;
>+ }
>+
>+ if (*end != ':')
>+ return -1;
>+
>+ for (idx = RTE_MIN(min, max);
>+ idx <= RTE_MAX(min, max); idx++)
>+ devargs->proto_xtr[idx] = xtr_type;
>+
>+ return 0;
>+ }
>+
>+ /* process set within bracket */
>+ str++;
>+ while (isblank(*str))
>+ str++;
>+ if (*str == '\0')
>+ return -1;
>+
>+ min = ICE_MAX_QUEUE_NUM;
>+ do {
>+ /* go ahead to the first digit */
>+ while (isblank(*str))
>+ str++;
>+ if (!isdigit(*str))
>+ return -1;
>+
>+ /* get the digit value */
>+ errno = 0;
>+ idx = strtoul(str, &end, 10);
>+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
>+ return -1;
>+
>+ /* go ahead to separator '-',',' and ')' */
>+ while (isblank(*end))
>+ end++;
>+ if (*end == '-') {
>+ if (min == ICE_MAX_QUEUE_NUM)
>+ min = idx;
>+ else /* avoid continuous '-' */
>+ return -1;
>+ } else if (*end == ',' || *end == ')') {
>+ max = idx;
>+ if (min == ICE_MAX_QUEUE_NUM)
>+ min = idx;
>+
>+ for (idx = RTE_MIN(min, max);
>+ idx <= RTE_MAX(min, max); idx++)
>+ devargs->proto_xtr[idx] = xtr_type;
>+
>+ min = ICE_MAX_QUEUE_NUM;
>+ } else {
>+ return -1;
>+ }
>+
>+ str = end + 1;
>+ } while (*end != ')' && *end != '\0');
>+
>+ return 0;
>+}
>+
>+static int
>+parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
>+{
>+ const char *queue_start;
>+ uint32_t idx;
>+ int xtr_type;
>+ char xtr_name[32];
>+
>+ while (isblank(*queues))
>+ queues++;
>+
>+ if (*queues != '[') {
>+ xtr_type = lookup_proto_xtr_type(queues);
>+ if (xtr_type < 0)
>+ return -1;
>+
>+ devargs->proto_xtr_dflt = xtr_type;
If we memset(devargs->proto_xtr, xtr_type, ICE_MAX_QUEUE_NUM) here, seems
we don't need proto_xtr_dflt.
>+ return 0;
>+ }
>+
>+ queues++;
>+ do {
>+ while (isblank(*queues))
>+ queues++;
>+ if (*queues == '\0')
>+ return -1;
>+
>+ queue_start = queues;
>+
>+ /* go across a complete bracket */
>+ if (*queue_start == '(') {
>+ queues += strcspn(queues, ")");
>+ if (*queues != ')')
>+ return -1;
>+ }
>+
>+ /* scan the separator ':' */
>+ queues += strcspn(queues, ":");
>+ if (*queues++ != ':')
>+ return -1;
>+ while (isblank(*queues))
>+ queues++;
>+
>+ for (idx = 0; ; idx++) {
>+ if (isblank(queues[idx]) ||
>+ queues[idx] == ',' ||
>+ queues[idx] == ']' ||
>+ queues[idx] == '\0')
>+ break;
>+
>+ if (idx > sizeof(xtr_name) - 2)
>+ return -1;
>+
>+ xtr_name[idx] = queues[idx];
>+ }
>+ xtr_name[idx] = '\0';
>+ xtr_type = lookup_proto_xtr_type(xtr_name);
>+ if (xtr_type < 0)
>+ return -1;
>+
>+ queues += idx;
>+
>+ while (isblank(*queues) || *queues == ',' || *queues == ']')
>+ queues++;
>+
>+ if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
>+ return -1;
>+ } while (*queues != '\0');
>+
>+ return 0;
>+}
>+
>+static int
>+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
>+ void *extra_args)
>+{
>+ struct ice_devargs *devargs = extra_args;
>+
>+ if (value == NULL || extra_args == NULL)
>+ return -EINVAL;
>+
>+ if (parse_queue_proto_xtr(value, devargs) < 0) {
>+ PMD_DRV_LOG(ERR,
>+ "The protocol extraction parameter is wrong : '%s'",
>+ value);
>+ return -1;
>+ }
>+
>+ return 0;
>+}
>+
>+static void
>+ice_parse_proto_xtr_devarg(struct rte_kvargs *kvlist,
>+ struct ice_devargs *devargs)
>+{
>+ int i;
>+
>+ devargs->proto_xtr_dflt = PROTO_XTR_NONE;
>+
>+ for (i = 0; i < ICE_MAX_QUEUE_NUM; i++)
>+ devargs->proto_xtr[i] = PROTO_XTR_NONE;
memset(devargs->proto_xtr, PROTO_XTR_NONE, ICE_MAX_QUEUE_NUM) ?
>+
>+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
>+ handle_proto_xtr_arg, devargs);
Do we need to check ret of rte_kvargs_process here and change ice_parse_proto_xtr_devarg
to return int?
>+}
>+
>+static bool
>+ice_proto_xtr_support(struct ice_hw *hw)
>+{
>+#define FLX_REG(val, fld, idx) \
>+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
>+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
>+ static struct {
>+ uint32_t rxdid;
>+ uint16_t protid_0;
>+ uint16_t protid_1;
>+ } xtr_sets[] = {
>+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
>+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
>+ ICE_PROT_IPV4_OF_OR_S },
>+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
>+ ICE_PROT_IPV6_OF_OR_S },
>+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
>+ ICE_PROT_IPV6_OF_OR_S },
>+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
>+ };
>+ uint32_t i;
>+
>+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
>+ uint32_t rxdid = xtr_sets[i].rxdid;
>+ uint32_t v;
>+
>+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
>+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
>+
>+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
>+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
>+ return false;
>+ }
>+
>+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
>+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
>+
>+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
>+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
>+ return false;
>+ }
>+ }
>+
>+ return true;
>+}
>+
> static int
> ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
> uint32_t num)
>@@ -1079,6 +1368,8 @@ ice_interrupt_handler(void *param)
> static int
> ice_pf_sw_init(struct rte_eth_dev *dev)
> {
>+ struct ice_adapter *ad =
>+ ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
> struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> struct ice_hw *hw = ICE_PF_TO_HW(pf);
>
>@@ -1088,6 +1379,21 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
>
> pf->lan_nb_qps = pf->lan_nb_qp_max;
>
>+ if (ice_proto_xtr_support(hw))
>+ pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
>+
>+ if (pf->proto_xtr != NULL) {
>+ uint16_t i;
>+
>+ for (i = 0; i < pf->lan_nb_qps; i++)
>+ pf->proto_xtr[i] =
>+ ad->devargs.proto_xtr[i] != PROTO_XTR_NONE ?
>+ ad->devargs.proto_xtr[i] :
>+ ad->devargs.proto_xtr_dflt;
>+ } else {
>+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
>+ }
>+
> return 0;
> }
>
>@@ -1378,6 +1684,8 @@ static int ice_parse_devargs(struct rte_eth_dev *dev)
> return -EINVAL;
> }
>
>+ ice_parse_proto_xtr_devarg(kvlist, &ad->devargs);
>+
> ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
> &parse_bool, &ad->devargs.safe_mode_support);
>
>@@ -1547,6 +1855,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> ice_sched_cleanup_all(hw);
> rte_free(hw->port_info);
> ice_shutdown_all_ctrlq(hw);
>+ rte_free(pf->proto_xtr);
>
> return ret;
> }
>@@ -1672,6 +1981,8 @@ ice_dev_close(struct rte_eth_dev *dev)
> rte_free(hw->port_info);
> hw->port_info = NULL;
> ice_shutdown_all_ctrlq(hw);
>+ rte_free(pf->proto_xtr);
>+ pf->proto_xtr = NULL;
> }
>
> static int
>@@ -3795,6 +4106,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
> RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
> RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
> RTE_PMD_REGISTER_PARAM_STRING(net_ice,
>+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
> ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
>
> RTE_INIT(ice_init_log)
>diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
>index f569da833..e58192104 100644
>--- a/drivers/net/ice/ice_ethdev.h
>+++ b/drivers/net/ice/ice_ethdev.h
>@@ -263,6 +263,7 @@ struct ice_pf {
> uint16_t lan_nb_qp_max;
> uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
> uint16_t base_queue; /* The base queue pairs index in the device */
>+ uint8_t *proto_xtr; /* Protocol extraction type for all queues */
> struct ice_hw_port_stats stats_offset;
> struct ice_hw_port_stats stats;
> /* internal packet statistics, it should be excluded from the total */
>@@ -273,11 +274,15 @@ struct ice_pf {
> struct ice_flow_list flow_list;
> };
>
>+#define ICE_MAX_QUEUE_NUM 2048
>+
> /**
> * Cache devargs parse result.
> */
> struct ice_devargs {
> int safe_mode_support;
>+ uint8_t proto_xtr_dflt;
>+ uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
> };
>
> /**
>diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
>index d2e36853f..e28310b96 100644
>--- a/drivers/net/ice/ice_rxtx.c
>+++ b/drivers/net/ice/ice_rxtx.c
>@@ -13,6 +13,36 @@
> PKT_TX_TCP_SEG | \
> PKT_TX_OUTER_IP_CKSUM)
>
>+static inline uint8_t
>+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
>+{
>+ static uint8_t xtr_map[] = {
>+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
>+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
>+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
>+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
>+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
>+ };
>+
>+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
>+}
>+
>+static inline uint8_t
>+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
>+{
>+ static uint8_t rxdid_map[] = {
>+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
>+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
>+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
>+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
>+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
>+ };
>+ uint8_t rxdid;
>+
>+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
>+
>+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
>+}
>
> static enum ice_status
> ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
>@@ -84,6 +114,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
> rx_ctx.showiv = 0;
> rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
>
>+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
>+
>+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
>+ rxq->port_id, rxq->queue_id, rxdid);
>+
> /* Enable Flexible Descriptors in the queue context which
> * allows this driver to select a specific receive descriptor format
> */
>@@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
> rxq->drop_en = rx_conf->rx_drop_en;
> rxq->vsi = vsi;
> rxq->rx_deferred_start = rx_conf->rx_deferred_start;
>+ rxq->proto_xtr = pf->proto_xtr != NULL ?
>+ pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
>
> /* Allocate the maximun number of RX ring hardware descriptor. */
> len = ICE_MAX_RING_DESC;
>@@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
> mb->vlan_tci, mb->vlan_tci_outer);
> }
>
>+#define ICE_RX_PROTO_XTR_VALID \
>+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
>+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
>+
> static inline void
> ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> volatile union ice_rx_flex_desc *rxdp)
>@@ -1075,6 +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> mb->ol_flags |= PKT_RX_RSS_HASH;
> mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
> }
>+
>+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
>+ init_proto_xtr_flds(mb);
>+
>+ stat_err = rte_le_to_cpu_16(desc->status_error1);
>+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
>+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
>+
>+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
>+ xtr->u.raw.data0 =
>+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
>+
>+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
>+ xtr->u.raw.data1 =
>+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
>+
>+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
>+ xtr->magic = PROTO_XTR_MAGIC_ID;
>+ }
>+#endif
> }
>
> #ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
>diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
>index 64e891875..de16637f3 100644
>--- a/drivers/net/ice/ice_rxtx.h
>+++ b/drivers/net/ice/ice_rxtx.h
>@@ -5,6 +5,7 @@
> #ifndef _ICE_RXTX_H_
> #define _ICE_RXTX_H_
>
>+#include "rte_pmd_ice.h"
> #include "ice_ethdev.h"
>
> #define ICE_ALIGN_RING_DESC 32
>@@ -78,6 +79,7 @@ struct ice_rx_queue {
> uint16_t max_pkt_len; /* Maximum packet length */
> bool q_set; /* indicate if rx queue has been configured */
> bool rx_deferred_start; /* don't start this queue in dev start */
>+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
> ice_rx_release_mbufs_t rx_rel_mbufs;
> };
>
>diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
>index c5f0d564f..080ca4175 100644
>--- a/drivers/net/ice/ice_rxtx_vec_common.h
>+++ b/drivers/net/ice/ice_rxtx_vec_common.h
>@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
> if (rxq->nb_rx_desc % rxq->rx_free_thresh)
> return -1;
>
>+ if (rxq->proto_xtr != PROTO_XTR_NONE)
>+ return -1;
>+
> return 0;
> }
>
>diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
>index 36b4b3c85..6828170a9 100644
>--- a/drivers/net/ice/meson.build
>+++ b/drivers/net/ice/meson.build
>@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
> objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
> endif
> endif
>+
>+install_headers('rte_pmd_ice.h')
>diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
>new file mode 100644
>index 000000000..719487e1e
>--- /dev/null
>+++ b/drivers/net/ice/rte_pmd_ice.h
>@@ -0,0 +1,152 @@
>+/* SPDX-License-Identifier: BSD-3-Clause
>+ * Copyright(c) 2019 Intel Corporation
>+ */
>+
>+#ifndef _RTE_PMD_ICE_H_
>+#define _RTE_PMD_ICE_H_
>+
>+#include <stdio.h>
>+#include <rte_mbuf.h>
>+#include <rte_ethdev.h>
>+
>+#ifdef __cplusplus
>+extern "C" {
>+#endif
>+
>+enum proto_xtr_type {
>+ PROTO_XTR_NONE,
>+ PROTO_XTR_VLAN,
>+ PROTO_XTR_IPV4,
>+ PROTO_XTR_IPV6,
>+ PROTO_XTR_IPV6_FLOW,
>+ PROTO_XTR_TCP,
>+};
>+
>+struct proto_xtr_flds {
>+ union {
>+ struct {
>+ uint16_t data0;
>+ uint16_t data1;
>+ } raw;
>+ struct {
>+ uint16_t stag_vid:12,
>+ stag_dei:1,
>+ stag_pcp:3;
>+ uint16_t ctag_vid:12,
>+ ctag_dei:1,
>+ ctag_pcp:3;
>+ } vlan;
>+ struct {
>+ uint16_t protocol:8,
>+ ttl:8;
>+ uint16_t tos:8,
>+ ihl:4,
>+ version:4;
>+ } ipv4;
>+ struct {
>+ uint16_t hoplimit:8,
>+ nexthdr:8;
>+ uint16_t flowhi4:4,
>+ tc:8,
>+ version:4;
>+ } ipv6;
>+ struct {
>+ uint16_t flowlo16;
>+ uint16_t flowhi4:4,
>+ tc:8,
>+ version:4;
>+ } ipv6_flow;
>+ struct {
>+ uint16_t fin:1,
>+ syn:1,
>+ rst:1,
>+ psh:1,
>+ ack:1,
>+ urg:1,
>+ ece:1,
>+ cwr:1,
>+ res1:4,
>+ doff:4;
>+ uint16_t rsvd;
>+ } tcp;
>+ } u;
>+
>+ uint16_t rsvd;
>+
>+ uint8_t type;
>+
>+#define PROTO_XTR_MAGIC_ID 0xCE
>+ uint8_t magic;
>+};
>+
>+static inline void
>+init_proto_xtr_flds(struct rte_mbuf *mb)
>+{
>+ mb->udata64 = 0;
>+}
>+
>+static inline struct proto_xtr_flds *
>+get_proto_xtr_flds(struct rte_mbuf *mb)
>+{
>+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
>+
>+ return (struct proto_xtr_flds *)&mb->udata64;
>+}
>+
>+static inline void
>+dump_proto_xtr_flds(struct rte_mbuf *mb)
>+{
>+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
>+
>+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
>+ return;
>+
>+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
>+ xtr->u.raw.data0, xtr->u.raw.data1);
>+
>+ if (xtr->type == PROTO_XTR_VLAN)
>+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
>+ xtr->u.vlan.stag_pcp,
>+ xtr->u.vlan.stag_dei,
>+ xtr->u.vlan.stag_vid,
>+ xtr->u.vlan.ctag_pcp,
>+ xtr->u.vlan.ctag_dei,
>+ xtr->u.vlan.ctag_vid);
>+ else if (xtr->type == PROTO_XTR_IPV4)
>+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
>+ xtr->u.ipv4.version,
>+ xtr->u.ipv4.ihl,
>+ xtr->u.ipv4.tos,
>+ xtr->u.ipv4.ttl,
>+ xtr->u.ipv4.protocol);
>+ else if (xtr->type == PROTO_XTR_IPV6)
>+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
>+ xtr->u.ipv6.version,
>+ xtr->u.ipv6.tc,
>+ xtr->u.ipv6.flowhi4,
>+ xtr->u.ipv6.nexthdr,
>+ xtr->u.ipv6.hoplimit);
>+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
>+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
>+ xtr->u.ipv6_flow.version,
>+ xtr->u.ipv6_flow.tc,
>+ xtr->u.ipv6_flow.flowhi4,
>+ xtr->u.ipv6_flow.flowlo16);
>+ else if (xtr->type == PROTO_XTR_TCP)
>+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
>+ xtr->u.tcp.doff,
>+ xtr->u.tcp.cwr ? "C" : "",
>+ xtr->u.tcp.ece ? "E" : "",
>+ xtr->u.tcp.urg ? "U" : "",
>+ xtr->u.tcp.ack ? "A" : "",
>+ xtr->u.tcp.psh ? "P" : "",
>+ xtr->u.tcp.rst ? "R" : "",
>+ xtr->u.tcp.syn ? "S" : "",
>+ xtr->u.tcp.fin ? "F" : "");
>+}
>+
>+#ifdef __cplusplus
>+}
>+#endif
>+
>+#endif /* _RTE_PMD_ICE_H_ */
>--
>2.17.1
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
2019-09-18 23:30 ` Ye Xiaolong
@ 2019-09-19 1:36 ` Wang, Haiyue
2019-09-19 1:44 ` Wang, Haiyue
1 sibling, 0 replies; 54+ messages in thread
From: Wang, Haiyue @ 2019-09-19 1:36 UTC (permalink / raw)
To: Ye, Xiaolong, Rong, Leyi; +Cc: Lu, Wenzhuo, Zhang, Qi Z, dev
> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Thursday, September 19, 2019 07:30
> To: Rong, Leyi <leyi.rong@intel.com>
> Cc: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; dev@dpdk.org
> Subject: Re: [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
>
> On 09/17, Leyi Rong wrote:
> >From: Haiyue Wang <haiyue.wang@intel.com>
> >
> >The ice has the feature to extract protocol fields into flex descriptor
> >by programming per queue. Currently, the ice PMD will put the protocol
> >fields into rte_mbuf::udata64 with different type format. Application
> >can access the protocol fields quickly.
> >
>
> [snip]
>
> >+static int
> >+parse_queue_set(const char *input, int xtr_type, struct ice_devargs *devargs)
> >+{
> >+ const char *str = input;
> >+ char *end = NULL;
> >+ uint32_t min, max;
> >+ uint32_t idx;
> >+
> >+ while (isblank(*str))
> >+ str++;
> >+
> >+ if ((!isdigit(*str) && *str != '(') || (*str == '\0'))
>
> Minor nit, (*str == '\0') seems redundant here, no?
>
Yes! Will make it clean.
> >+ return -1;
> >+
> >+ /* process single number or single range of number */
> >+ if (*str != '(') {
> >+ errno = 0;
> >+ idx = strtoul(str, &end, 10);
> >+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> >+ return -1;
> >+
> >+ while (isblank(*end))
> >+ end++;
> >+
> >+ min = idx;
> >+ max = idx;
> >+
> >+ /* process single <number>-<number> */
> >+ if (*end == '-') {
> >+ end++;
> >+ while (isblank(*end))
> >+ end++;
> >+ if (!isdigit(*end))
> >+ return -1;
> >+
> >+ errno = 0;
> >+ idx = strtoul(end, &end, 10);
> >+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> >+ return -1;
> >+
> >+ max = idx;
> >+ while (isblank(*end))
> >+ end++;
> >+ }
> >+
> >+ if (*end != ':')
> >+ return -1;
> >+
> >+ for (idx = RTE_MIN(min, max);
> >+ idx <= RTE_MAX(min, max); idx++)
> >+ devargs->proto_xtr[idx] = xtr_type;
> >+
> >+ return 0;
> >+ }
> >+
> >+ /* process set within bracket */
> >+ str++;
> >+ while (isblank(*str))
> >+ str++;
> >+ if (*str == '\0')
> >+ return -1;
> >+
> >+ min = ICE_MAX_QUEUE_NUM;
> >+ do {
> >+ /* go ahead to the first digit */
> >+ while (isblank(*str))
> >+ str++;
> >+ if (!isdigit(*str))
> >+ return -1;
> >+
> >+ /* get the digit value */
> >+ errno = 0;
> >+ idx = strtoul(str, &end, 10);
> >+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> >+ return -1;
> >+
> >+ /* go ahead to separator '-',',' and ')' */
> >+ while (isblank(*end))
> >+ end++;
> >+ if (*end == '-') {
> >+ if (min == ICE_MAX_QUEUE_NUM)
> >+ min = idx;
> >+ else /* avoid continuous '-' */
> >+ return -1;
> >+ } else if (*end == ',' || *end == ')') {
> >+ max = idx;
> >+ if (min == ICE_MAX_QUEUE_NUM)
> >+ min = idx;
> >+
> >+ for (idx = RTE_MIN(min, max);
> >+ idx <= RTE_MAX(min, max); idx++)
> >+ devargs->proto_xtr[idx] = xtr_type;
> >+
> >+ min = ICE_MAX_QUEUE_NUM;
> >+ } else {
> >+ return -1;
> >+ }
> >+
> >+ str = end + 1;
> >+ } while (*end != ')' && *end != '\0');
> >+
> >+ return 0;
> >+}
> >+
> >+static int
> >+parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
> >+{
> >+ const char *queue_start;
> >+ uint32_t idx;
> >+ int xtr_type;
> >+ char xtr_name[32];
> >+
> >+ while (isblank(*queues))
> >+ queues++;
> >+
> >+ if (*queues != '[') {
> >+ xtr_type = lookup_proto_xtr_type(queues);
> >+ if (xtr_type < 0)
> >+ return -1;
> >+
> >+ devargs->proto_xtr_dflt = xtr_type;
>
> If we memset(devargs->proto_xtr, xtr_type, ICE_MAX_QUEUE_NUM) here, seems
> we don't need proto_xtr_dflt.
+1, good catch.
>
> >+ return 0;
> >+ }
> >+
> >+ queues++;
> >+ do {
> >+ while (isblank(*queues))
> >+ queues++;
> >+ if (*queues == '\0')
> >+ return -1;
> >+
> >+ queue_start = queues;
> >+
> >+ /* go across a complete bracket */
> >+ if (*queue_start == '(') {
> >+ queues += strcspn(queues, ")");
> >+ if (*queues != ')')
> >+ return -1;
> >+ }
> >+
> >+ /* scan the separator ':' */
> >+ queues += strcspn(queues, ":");
> >+ if (*queues++ != ':')
> >+ return -1;
> >+ while (isblank(*queues))
> >+ queues++;
> >+
> >+ for (idx = 0; ; idx++) {
> >+ if (isblank(queues[idx]) ||
> >+ queues[idx] == ',' ||
> >+ queues[idx] == ']' ||
> >+ queues[idx] == '\0')
> >+ break;
> >+
> >+ if (idx > sizeof(xtr_name) - 2)
> >+ return -1;
> >+
> >+ xtr_name[idx] = queues[idx];
> >+ }
> >+ xtr_name[idx] = '\0';
> >+ xtr_type = lookup_proto_xtr_type(xtr_name);
> >+ if (xtr_type < 0)
> >+ return -1;
> >+
> >+ queues += idx;
> >+
> >+ while (isblank(*queues) || *queues == ',' || *queues == ']')
> >+ queues++;
> >+
> >+ if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
> >+ return -1;
> >+ } while (*queues != '\0');
> >+
> >+ return 0;
> >+}
> >+
> >+static int
> >+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
> >+ void *extra_args)
> >+{
> >+ struct ice_devargs *devargs = extra_args;
> >+
> >+ if (value == NULL || extra_args == NULL)
> >+ return -EINVAL;
> >+
> >+ if (parse_queue_proto_xtr(value, devargs) < 0) {
> >+ PMD_DRV_LOG(ERR,
> >+ "The protocol extraction parameter is wrong : '%s'",
> >+ value);
> >+ return -1;
> >+ }
> >+
> >+ return 0;
> >+}
> >+
> >+static void
> >+ice_parse_proto_xtr_devarg(struct rte_kvargs *kvlist,
> >+ struct ice_devargs *devargs)
> >+{
> >+ int i;
> >+
> >+ devargs->proto_xtr_dflt = PROTO_XTR_NONE;
> >+
> >+ for (i = 0; i < ICE_MAX_QUEUE_NUM; i++)
> >+ devargs->proto_xtr[i] = PROTO_XTR_NONE;
>
> memset(devargs->proto_xtr, PROTO_XTR_NONE, ICE_MAX_QUEUE_NUM) ?
>
Yes.
> >+
> >+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
> >+ handle_proto_xtr_arg, devargs);
>
> Do we need to check ret of rte_kvargs_process here and change ice_parse_proto_xtr_devarg
> to return int?
>
> >+}
> >+
> >+static bool
> >+ice_proto_xtr_support(struct ice_hw *hw)
> >+{
> >+#define FLX_REG(val, fld, idx) \
> >+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
> >+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
> >+ static struct {
> >+ uint32_t rxdid;
> >+ uint16_t protid_0;
> >+ uint16_t protid_1;
> >+ } xtr_sets[] = {
> >+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
> >+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
> >+ ICE_PROT_IPV4_OF_OR_S },
> >+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
> >+ ICE_PROT_IPV6_OF_OR_S },
> >+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
> >+ ICE_PROT_IPV6_OF_OR_S },
> >+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
> >+ };
> >+ uint32_t i;
> >+
> >+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
> >+ uint32_t rxdid = xtr_sets[i].rxdid;
> >+ uint32_t v;
> >+
> >+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
> >+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
> >+
> >+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
> >+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
> >+ return false;
> >+ }
> >+
> >+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
> >+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
> >+
> >+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
> >+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
> >+ return false;
> >+ }
> >+ }
> >+
> >+ return true;
> >+}
> >+
> > static int
> > ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
> > uint32_t num)
> >@@ -1079,6 +1368,8 @@ ice_interrupt_handler(void *param)
> > static int
> > ice_pf_sw_init(struct rte_eth_dev *dev)
> > {
> >+ struct ice_adapter *ad =
> >+ ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
> > struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> > struct ice_hw *hw = ICE_PF_TO_HW(pf);
> >
> >@@ -1088,6 +1379,21 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
> >
> > pf->lan_nb_qps = pf->lan_nb_qp_max;
> >
> >+ if (ice_proto_xtr_support(hw))
> >+ pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
> >+
> >+ if (pf->proto_xtr != NULL) {
> >+ uint16_t i;
> >+
> >+ for (i = 0; i < pf->lan_nb_qps; i++)
> >+ pf->proto_xtr[i] =
> >+ ad->devargs.proto_xtr[i] != PROTO_XTR_NONE ?
> >+ ad->devargs.proto_xtr[i] :
> >+ ad->devargs.proto_xtr_dflt;
> >+ } else {
> >+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
> >+ }
> >+
> > return 0;
> > }
> >
> >@@ -1378,6 +1684,8 @@ static int ice_parse_devargs(struct rte_eth_dev *dev)
> > return -EINVAL;
> > }
> >
> >+ ice_parse_proto_xtr_devarg(kvlist, &ad->devargs);
> >+
> > ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
> > &parse_bool, &ad->devargs.safe_mode_support);
> >
> >@@ -1547,6 +1855,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> > ice_sched_cleanup_all(hw);
> > rte_free(hw->port_info);
> > ice_shutdown_all_ctrlq(hw);
> >+ rte_free(pf->proto_xtr);
> >
> > return ret;
> > }
> >@@ -1672,6 +1981,8 @@ ice_dev_close(struct rte_eth_dev *dev)
> > rte_free(hw->port_info);
> > hw->port_info = NULL;
> > ice_shutdown_all_ctrlq(hw);
> >+ rte_free(pf->proto_xtr);
> >+ pf->proto_xtr = NULL;
> > }
> >
> > static int
> >@@ -3795,6 +4106,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
> > RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
> > RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
> > RTE_PMD_REGISTER_PARAM_STRING(net_ice,
> >+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
> > ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
> >
> > RTE_INIT(ice_init_log)
> >diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
> >index f569da833..e58192104 100644
> >--- a/drivers/net/ice/ice_ethdev.h
> >+++ b/drivers/net/ice/ice_ethdev.h
> >@@ -263,6 +263,7 @@ struct ice_pf {
> > uint16_t lan_nb_qp_max;
> > uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
> > uint16_t base_queue; /* The base queue pairs index in the device */
> >+ uint8_t *proto_xtr; /* Protocol extraction type for all queues */
> > struct ice_hw_port_stats stats_offset;
> > struct ice_hw_port_stats stats;
> > /* internal packet statistics, it should be excluded from the total */
> >@@ -273,11 +274,15 @@ struct ice_pf {
> > struct ice_flow_list flow_list;
> > };
> >
> >+#define ICE_MAX_QUEUE_NUM 2048
> >+
> > /**
> > * Cache devargs parse result.
> > */
> > struct ice_devargs {
> > int safe_mode_support;
> >+ uint8_t proto_xtr_dflt;
> >+ uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
> > };
> >
> > /**
> >diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
> >index d2e36853f..e28310b96 100644
> >--- a/drivers/net/ice/ice_rxtx.c
> >+++ b/drivers/net/ice/ice_rxtx.c
> >@@ -13,6 +13,36 @@
> > PKT_TX_TCP_SEG | \
> > PKT_TX_OUTER_IP_CKSUM)
> >
> >+static inline uint8_t
> >+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
> >+{
> >+ static uint8_t xtr_map[] = {
> >+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
> >+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
> >+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
> >+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
> >+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
> >+ };
> >+
> >+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
> >+}
> >+
> >+static inline uint8_t
> >+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
> >+{
> >+ static uint8_t rxdid_map[] = {
> >+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
> >+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
> >+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
> >+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> >+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
> >+ };
> >+ uint8_t rxdid;
> >+
> >+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
> >+
> >+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
> >+}
> >
> > static enum ice_status
> > ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
> >@@ -84,6 +114,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
> > rx_ctx.showiv = 0;
> > rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
> >
> >+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
> >+
> >+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
> >+ rxq->port_id, rxq->queue_id, rxdid);
> >+
> > /* Enable Flexible Descriptors in the queue context which
> > * allows this driver to select a specific receive descriptor format
> > */
> >@@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
> > rxq->drop_en = rx_conf->rx_drop_en;
> > rxq->vsi = vsi;
> > rxq->rx_deferred_start = rx_conf->rx_deferred_start;
> >+ rxq->proto_xtr = pf->proto_xtr != NULL ?
> >+ pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
> >
> > /* Allocate the maximun number of RX ring hardware descriptor. */
> > len = ICE_MAX_RING_DESC;
> >@@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
> > mb->vlan_tci, mb->vlan_tci_outer);
> > }
> >
> >+#define ICE_RX_PROTO_XTR_VALID \
> >+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
> >+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> >+
> > static inline void
> > ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> > volatile union ice_rx_flex_desc *rxdp)
> >@@ -1075,6 +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> > mb->ol_flags |= PKT_RX_RSS_HASH;
> > mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
> > }
> >+
> >+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
> >+ init_proto_xtr_flds(mb);
> >+
> >+ stat_err = rte_le_to_cpu_16(desc->status_error1);
> >+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
> >+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> >+
> >+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
> >+ xtr->u.raw.data0 =
> >+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
> >+
> >+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> >+ xtr->u.raw.data1 =
> >+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
> >+
> >+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
> >+ xtr->magic = PROTO_XTR_MAGIC_ID;
> >+ }
> >+#endif
> > }
> >
> > #ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
> >diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
> >index 64e891875..de16637f3 100644
> >--- a/drivers/net/ice/ice_rxtx.h
> >+++ b/drivers/net/ice/ice_rxtx.h
> >@@ -5,6 +5,7 @@
> > #ifndef _ICE_RXTX_H_
> > #define _ICE_RXTX_H_
> >
> >+#include "rte_pmd_ice.h"
> > #include "ice_ethdev.h"
> >
> > #define ICE_ALIGN_RING_DESC 32
> >@@ -78,6 +79,7 @@ struct ice_rx_queue {
> > uint16_t max_pkt_len; /* Maximum packet length */
> > bool q_set; /* indicate if rx queue has been configured */
> > bool rx_deferred_start; /* don't start this queue in dev start */
> >+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
> > ice_rx_release_mbufs_t rx_rel_mbufs;
> > };
> >
> >diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
> >index c5f0d564f..080ca4175 100644
> >--- a/drivers/net/ice/ice_rxtx_vec_common.h
> >+++ b/drivers/net/ice/ice_rxtx_vec_common.h
> >@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
> > if (rxq->nb_rx_desc % rxq->rx_free_thresh)
> > return -1;
> >
> >+ if (rxq->proto_xtr != PROTO_XTR_NONE)
> >+ return -1;
> >+
> > return 0;
> > }
> >
> >diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
> >index 36b4b3c85..6828170a9 100644
> >--- a/drivers/net/ice/meson.build
> >+++ b/drivers/net/ice/meson.build
> >@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
> > objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
> > endif
> > endif
> >+
> >+install_headers('rte_pmd_ice.h')
> >diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
> >new file mode 100644
> >index 000000000..719487e1e
> >--- /dev/null
> >+++ b/drivers/net/ice/rte_pmd_ice.h
> >@@ -0,0 +1,152 @@
> >+/* SPDX-License-Identifier: BSD-3-Clause
> >+ * Copyright(c) 2019 Intel Corporation
> >+ */
> >+
> >+#ifndef _RTE_PMD_ICE_H_
> >+#define _RTE_PMD_ICE_H_
> >+
> >+#include <stdio.h>
> >+#include <rte_mbuf.h>
> >+#include <rte_ethdev.h>
> >+
> >+#ifdef __cplusplus
> >+extern "C" {
> >+#endif
> >+
> >+enum proto_xtr_type {
> >+ PROTO_XTR_NONE,
> >+ PROTO_XTR_VLAN,
> >+ PROTO_XTR_IPV4,
> >+ PROTO_XTR_IPV6,
> >+ PROTO_XTR_IPV6_FLOW,
> >+ PROTO_XTR_TCP,
> >+};
> >+
> >+struct proto_xtr_flds {
> >+ union {
> >+ struct {
> >+ uint16_t data0;
> >+ uint16_t data1;
> >+ } raw;
> >+ struct {
> >+ uint16_t stag_vid:12,
> >+ stag_dei:1,
> >+ stag_pcp:3;
> >+ uint16_t ctag_vid:12,
> >+ ctag_dei:1,
> >+ ctag_pcp:3;
> >+ } vlan;
> >+ struct {
> >+ uint16_t protocol:8,
> >+ ttl:8;
> >+ uint16_t tos:8,
> >+ ihl:4,
> >+ version:4;
> >+ } ipv4;
> >+ struct {
> >+ uint16_t hoplimit:8,
> >+ nexthdr:8;
> >+ uint16_t flowhi4:4,
> >+ tc:8,
> >+ version:4;
> >+ } ipv6;
> >+ struct {
> >+ uint16_t flowlo16;
> >+ uint16_t flowhi4:4,
> >+ tc:8,
> >+ version:4;
> >+ } ipv6_flow;
> >+ struct {
> >+ uint16_t fin:1,
> >+ syn:1,
> >+ rst:1,
> >+ psh:1,
> >+ ack:1,
> >+ urg:1,
> >+ ece:1,
> >+ cwr:1,
> >+ res1:4,
> >+ doff:4;
> >+ uint16_t rsvd;
> >+ } tcp;
> >+ } u;
> >+
> >+ uint16_t rsvd;
> >+
> >+ uint8_t type;
> >+
> >+#define PROTO_XTR_MAGIC_ID 0xCE
> >+ uint8_t magic;
> >+};
> >+
> >+static inline void
> >+init_proto_xtr_flds(struct rte_mbuf *mb)
> >+{
> >+ mb->udata64 = 0;
> >+}
> >+
> >+static inline struct proto_xtr_flds *
> >+get_proto_xtr_flds(struct rte_mbuf *mb)
> >+{
> >+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
> >+
> >+ return (struct proto_xtr_flds *)&mb->udata64;
> >+}
> >+
> >+static inline void
> >+dump_proto_xtr_flds(struct rte_mbuf *mb)
> >+{
> >+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> >+
> >+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
> >+ return;
> >+
> >+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
> >+ xtr->u.raw.data0, xtr->u.raw.data1);
> >+
> >+ if (xtr->type == PROTO_XTR_VLAN)
> >+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
> >+ xtr->u.vlan.stag_pcp,
> >+ xtr->u.vlan.stag_dei,
> >+ xtr->u.vlan.stag_vid,
> >+ xtr->u.vlan.ctag_pcp,
> >+ xtr->u.vlan.ctag_dei,
> >+ xtr->u.vlan.ctag_vid);
> >+ else if (xtr->type == PROTO_XTR_IPV4)
> >+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
> >+ xtr->u.ipv4.version,
> >+ xtr->u.ipv4.ihl,
> >+ xtr->u.ipv4.tos,
> >+ xtr->u.ipv4.ttl,
> >+ xtr->u.ipv4.protocol);
> >+ else if (xtr->type == PROTO_XTR_IPV6)
> >+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
> >+ xtr->u.ipv6.version,
> >+ xtr->u.ipv6.tc,
> >+ xtr->u.ipv6.flowhi4,
> >+ xtr->u.ipv6.nexthdr,
> >+ xtr->u.ipv6.hoplimit);
> >+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
> >+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
> >+ xtr->u.ipv6_flow.version,
> >+ xtr->u.ipv6_flow.tc,
> >+ xtr->u.ipv6_flow.flowhi4,
> >+ xtr->u.ipv6_flow.flowlo16);
> >+ else if (xtr->type == PROTO_XTR_TCP)
> >+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
> >+ xtr->u.tcp.doff,
> >+ xtr->u.tcp.cwr ? "C" : "",
> >+ xtr->u.tcp.ece ? "E" : "",
> >+ xtr->u.tcp.urg ? "U" : "",
> >+ xtr->u.tcp.ack ? "A" : "",
> >+ xtr->u.tcp.psh ? "P" : "",
> >+ xtr->u.tcp.rst ? "R" : "",
> >+ xtr->u.tcp.syn ? "S" : "",
> >+ xtr->u.tcp.fin ? "F" : "");
> >+}
> >+
> >+#ifdef __cplusplus
> >+}
> >+#endif
> >+
> >+#endif /* _RTE_PMD_ICE_H_ */
> >--
> >2.17.1
> >
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
2019-09-18 23:30 ` Ye Xiaolong
2019-09-19 1:36 ` Wang, Haiyue
@ 2019-09-19 1:44 ` Wang, Haiyue
1 sibling, 0 replies; 54+ messages in thread
From: Wang, Haiyue @ 2019-09-19 1:44 UTC (permalink / raw)
To: Ye, Xiaolong, Rong, Leyi; +Cc: Lu, Wenzhuo, Zhang, Qi Z, dev
> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Thursday, September 19, 2019 07:30
> To: Rong, Leyi <leyi.rong@intel.com>
> Cc: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; dev@dpdk.org
> Subject: Re: [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue
>
> On 09/17, Leyi Rong wrote:
> >From: Haiyue Wang <haiyue.wang@intel.com>
> >
> >The ice has the feature to extract protocol fields into flex descriptor
> >by programming per queue. Currently, the ice PMD will put the protocol
> >fields into rte_mbuf::udata64 with different type format. Application
> >can access the protocol fields quickly.
> >
[snip]
> >+
> >+ rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
> >+ handle_proto_xtr_arg, devargs);
>
> Do we need to check ret of rte_kvargs_process here and change ice_parse_proto_xtr_devarg
> to return int?
>
May be return error is a good idea, if the application wants to use it. I just
give a graceful warning message. Will change it, thanks!
PMD_DRV_LOG(ERR,
"The protocol extraction parameter is wrong : '%s'",
value);
return -1;
> >--
> >2.17.1
> >
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (6 preceding siblings ...)
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
` (6 more replies)
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
8 siblings, 7 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
This patchset enable Rx flexible descriptor for ice PMD
in both normal path and vector path.
Depends on shared code update patchset.
---
v4:
- Handle the ‘proto_xtr’ with error processing, and remove the
Redundant default type variable.
- Add flex descriptor and protocol extraction release note.
- Clean up the legacy descriptor.
v3:
- Parse the ‘proto_xtr’ devargs before CVL HW initialization and
save it for later accessing.
- Merge shuffle ops on vlan and rss flag on vector path.
v2:
- Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
when handling Rx packets.
Haiyue Wang (4):
net/ice: add Rx flex descriptor definition
net/ice: handle the Rx flex descriptor
net/ice: add protocol extraction support for per Rx queue
net/ice: remove Rx legacy descriptor definition
Leyi Rong (1):
net/ice: switch to Rx flexible descriptor in AVX path
Wenzhuo Lu (1):
net/ice: switch to flexible descriptor in SSE path
doc/guides/nics/ice.rst | 101 +++++++++
doc/guides/rel_notes/release_19_11.rst | 4 +
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 298 +++++++++++++-----------
drivers/net/ice/ice_rxtx.h | 8 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 221 +++++++++---------
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/ice_rxtx_vec_sse.c | 235 +++++++++----------
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
12 files changed, 962 insertions(+), 370 deletions(-)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
--
2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 1/6] net/ice: add Rx flex descriptor definition
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
` (5 subsequent siblings)
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The Rx flex descriptor has 16B and 32B size, with different
field definitions compared to legacy type.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index e9214110c..64e891875 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,8 +21,10 @@
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
#define ice_rx_desc ice_16byte_rx_desc
+#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
#define ice_rx_desc ice_32byte_rx_desc
+#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
#define ICE_SUPPORT_CHAIN_NUM 5
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-23 11:05 ` Ye Xiaolong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
` (4 subsequent siblings)
6 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Set the RXDID with flex descriptor type by default, change the Rx
function to support new descriptor handling.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 236 +++++++++++++++++--------------------
1 file changed, 111 insertions(+), 125 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 0282b5375..d2e36853f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,7 +13,6 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
-#define ICE_RX_ERR_BITS 0x3f
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -25,18 +24,9 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
enum ice_status err;
uint16_t buf_size, len;
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ uint32_t rxdid = ICE_RXDID_COMMS_GENERIC;
uint32_t regval;
- /**
- * The kernel driver uses flex descriptor. It sets the register
- * to flex descriptor mode.
- * DPDK uses legacy descriptor. It should set the register back
- * to the default value, then uses legacy descriptor mode.
- */
- regval = (0x01 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
- QRXFLXP_CNTXT_RXDID_PRIO_M;
- ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
-
/* Set buffer size as the head split is disabled. */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
RTE_PKTMBUF_HEADROOM);
@@ -94,6 +84,21 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile ID;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
+
err = ice_clear_rxq_ctx(hw, rxq->reg_idx);
if (err) {
PMD_DRV_LOG(ERR, "Failed to clear Lan Rx queue (%u) context",
@@ -961,16 +966,15 @@ uint32_t
ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
#define ICE_RXQ_SCAN_INTERVAL 4
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq;
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
- ((rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S) &
- (1 << ICE_RX_DESC_STATUS_DD_S)) {
+ rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
/**
* Check the DD bit of a rx descriptor of each 4 in a group,
* to avoid checking too frequently and downgrading performance
@@ -979,79 +983,77 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = (volatile union ice_rx_flex_desc *)
+ &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
return desc;
}
-/* Translate the rx descriptor status to pkt flags */
-static inline uint64_t
-ice_rxd_status_to_pkt_flags(uint64_t qword)
-{
- uint64_t flags;
-
- /* Check if RSS_HASH */
- flags = (((qword >> ICE_RX_DESC_STATUS_FLTSTAT_S) &
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ==
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ? PKT_RX_RSS_HASH : 0;
-
- return flags;
-}
+#define ICE_RX_FLEX_ERR0_BITS \
+ ((1 << ICE_RX_FLEX_DESC_STATUS0_HBO_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_RXE_S))
/* Rx L3/L4 checksum */
static inline uint64_t
-ice_rxd_error_to_pkt_flags(uint64_t qword)
+ice_rxd_error_to_pkt_flags(uint16_t stat_err0)
{
uint64_t flags = 0;
- uint64_t error_bits = (qword >> ICE_RXD_QW1_ERROR_S);
- if (likely((error_bits & ICE_RX_ERR_BITS) == 0)) {
+ /* check if HW has decoded the packet and checksum */
+ if (unlikely(!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return 0;
+
+ if (likely(!(stat_err0 & ICE_RX_FLEX_ERR0_BITS))) {
flags |= (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD);
return flags;
}
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_IPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
flags |= PKT_RX_IP_CKSUM_BAD;
else
flags |= PKT_RX_IP_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_L4E_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
flags |= PKT_RX_L4_CKSUM_BAD;
else
flags |= PKT_RX_L4_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_EIPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
flags |= PKT_RX_EIP_CKSUM_BAD;
return flags;
}
static inline void
-ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
+ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
{
- if (rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- (1 << ICE_RX_DESC_STATUS_L2TAG1P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S)) {
mb->ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED;
mb->vlan_tci =
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1);
+ rte_le_to_cpu_16(rxdp->wb.l2tag1);
PMD_RX_LOG(DEBUG, "Descriptor l2tag1: %u",
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1));
+ rte_le_to_cpu_16(rxdp->wb.l2tag1));
} else {
mb->vlan_tci = 0;
}
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
- if (rte_le_to_cpu_16(rxdp->wb.qword2.ext_status) &
- (1 << ICE_RX_DESC_EXT_STATUS_L2TAG2P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error1) &
+ (1 << ICE_RX_FLEX_DESC_STATUS1_L2TAG2P_S)) {
mb->ol_flags |= PKT_RX_QINQ_STRIPPED | PKT_RX_QINQ |
PKT_RX_VLAN_STRIPPED | PKT_RX_VLAN;
mb->vlan_tci_outer = mb->vlan_tci;
- mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2);
+ mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd);
PMD_RX_LOG(DEBUG, "Descriptor l2tag2_1: %u, l2tag2_2: %u",
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_1),
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2));
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_1st),
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd));
} else {
mb->vlan_tci_outer = 0;
}
@@ -1060,6 +1062,21 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+static inline void
+ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
+ volatile union ice_rx_flex_desc *rxdp)
+{
+ volatile struct ice_32b_rx_flex_desc_comms *desc =
+ (volatile struct ice_32b_rx_flex_desc_comms *)rxdp;
+ uint16_t stat_err;
+
+ stat_err = rte_le_to_cpu_16(desc->status_error0);
+ if (likely(stat_err & (1 << ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S))) {
+ mb->ol_flags |= PKT_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
+ }
+}
+
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
#define ICE_LOOK_AHEAD 8
#if (ICE_LOOK_AHEAD != 8)
@@ -1068,25 +1085,23 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
static inline int
ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
+ uint16_t stat_err0;
uint16_t pkt_len;
- uint64_t qword1;
- uint32_t rx_status;
int32_t s[ICE_LOOK_AHEAD], nb_dd;
int32_t i, j, nb_rx = 0;
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S;
+ stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Make sure there is at least 1 packet to receive */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/**
@@ -1096,42 +1111,31 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
for (i = 0; i < ICE_RX_MAX_BURST; i += ICE_LOOK_AHEAD,
rxdp += ICE_LOOK_AHEAD, rxep += ICE_LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--) {
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- s[j] = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
- }
+ for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--)
+ s[j] = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
rte_smp_rmb();
/* Compute how many status bits were set */
for (j = 0, nb_dd = 0; j < ICE_LOOK_AHEAD; j++)
- nb_dd += s[j] & (1 << ICE_RX_DESC_STATUS_DD_S);
+ nb_dd += s[j] & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S);
nb_rx += nb_dd;
/* Translate descriptor info to mbuf parameters */
for (j = 0; j < nb_dd; j++) {
mb = rxep[j].mbuf;
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- pkt_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ pkt_len = (rte_le_to_cpu_16(rxdp[j].wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
mb->data_len = pkt_len;
mb->pkt_len = pkt_len;
mb->ol_flags = 0;
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- mb->hash.rss =
- rte_le_to_cpu_32(
- rxdp[j].wb.qword0.hi_dword.rss);
- mb->packet_type = ptype_tbl[(uint8_t)(
- (qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
+ stat_err0 = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
+ pkt_flags = ice_rxd_error_to_pkt_flags(stat_err0);
+ mb->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxdp[j].wb.ptype_flex_flags0)];
ice_rxd_to_vlan_tci(mb, &rxdp[j]);
+ ice_rxd_to_pkt_fields(mb, &rxdp[j]);
mb->ol_flags |= pkt_flags;
}
@@ -1312,8 +1316,8 @@ ice_recv_scattered_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *first_seg = rxq->pkt_first_seg;
@@ -1324,21 +1328,18 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1377,14 +1378,10 @@ ice_recv_scattered_pkts(void *rx_queue,
/* Set data buffer address and data length of the mbuf */
rxdp->read.hdr_addr = 0;
rxdp->read.pkt_addr = dma_addr;
- rx_packet_len = (qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S;
+ rx_packet_len = rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
rxm->data_len = rx_packet_len;
rxm->data_off = RTE_PKTMBUF_HEADROOM;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
/**
* If this is the first buffer of the received packet, set the
@@ -1410,7 +1407,7 @@ ice_recv_scattered_pkts(void *rx_queue,
* update the pointer to the last mbuf of the current scattered
* packet and continue to parse the RX ring.
*/
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_EOF_S))) {
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_EOF_S))) {
last_seg = rxm;
continue;
}
@@ -1442,13 +1439,11 @@ ice_recv_scattered_pkts(void *rx_queue,
first_seg->port = rxq->port_id;
first_seg->ol_flags = 0;
-
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- first_seg->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
-
+ first_seg->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(first_seg, &rxd);
+ ice_rxd_to_pkt_fields(first_seg, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
first_seg->ol_flags |= pkt_flags;
/* Prefetch data of first segment, if configured to do so. */
rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr,
@@ -1538,9 +1533,8 @@ ice_dev_supported_ptypes_get(struct rte_eth_dev *dev)
int
ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq = rx_queue;
- volatile uint64_t *status;
- uint64_t mask;
uint32_t desc;
if (unlikely(offset >= rxq->nb_rx_desc))
@@ -1553,10 +1547,9 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- status = &rxq->rx_ring[desc].wb.qword1.status_error_len;
- mask = rte_cpu_to_le_64((1ULL << ICE_RX_DESC_STATUS_DD_S) <<
- ICE_RXD_QW1_STATUS_S);
- if (*status & mask)
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
return RTE_ETH_RX_DESC_AVAIL;
@@ -1642,8 +1635,8 @@ ice_recv_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *nmb; /* new allocated mbuf */
@@ -1652,21 +1645,18 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1696,8 +1686,8 @@ ice_recv_pkts(void *rx_queue,
rxdp->read.pkt_addr = dma_addr;
/* calculate rx_packet_len of the received pkt */
- rx_packet_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ rx_packet_len = (rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
/* fill old mbuf with received descriptor: rxd */
rxm->data_off = RTE_PKTMBUF_HEADROOM;
@@ -1707,15 +1697,11 @@ ice_recv_pkts(void *rx_queue,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- rxm->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
+ rxm->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(rxm, &rxd);
+ ice_rxd_to_pkt_fields(rxm, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
rxm->ol_flags |= pkt_flags;
/* copy old mbuf to rx_pkts */
rx_pkts[nb_rx++] = rxm;
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-23 3:25 ` Yang, Qiming
2019-09-23 14:24 ` Ye Xiaolong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
` (3 subsequent siblings)
6 siblings, 2 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The ice has the feature to extract protocol fields into flex descriptor
by programming per queue. Currently, the ice PMD will put the protocol
fields into rte_mbuf::udata64 with different type format. Application
can access the protocol fields quickly.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
doc/guides/nics/ice.rst | 101 +++++++++
doc/guides/rel_notes/release_19_11.rst | 4 +
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 61 +++++
drivers/net/ice/ice_rxtx.h | 2 +
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
10 files changed, 633 insertions(+)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
index 03819d29f..8a6f60e71 100644
--- a/doc/guides/nics/ice.rst
+++ b/doc/guides/nics/ice.rst
@@ -61,6 +61,107 @@ Runtime Config Options
NOTE: In Safe mode, only very limited features are available, features like RSS,
checksum, fdir, tunneling ... are all disabled.
+- ``Protocol extraction for per queue``
+
+ Configure the RX queues to do protocol extraction into ``rte_mbuf::udata64``
+ for protocol handling acceleration, like checking the TCP SYN packets quickly.
+
+ The argument format is::
+
+ -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
+ -w 18:00.0,proto_xtr=<protocol>
+
+ Queues are grouped by ``(`` and ``)`` within the group. The ``-`` character
+ is used as a range separator and ``,`` is used as a single number separator.
+ The grouping ``()`` can be omitted for single element group. If no queues are
+ specified, PMD will use this protocol extraction type for all queues.
+
+ Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-13 are
+ VLAN extraction, other queues run with no protocol extraction.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-23 are
+ IPv6 extraction, other queues use the default VLAN extraction.
+
+ The extraction will be copied into the lower 32 bit of ``rte_mbuf::udata64``.
+
+ .. table:: Protocol extraction : ``vlan``
+
+ +----------------------------+----------------------------+
+ | VLAN2 | VLAN1 |
+ +======+===+=================+======+===+=================+
+ | PCP | D | VID | PCP | D | VID |
+ +------+---+-----------------+------+---+-----------------+
+
+ VLAN1 - single or EVLAN (first for QinQ).
+
+ VLAN2 - C-VLAN (second for QinQ).
+
+ .. table:: Protocol extraction : ``ipv4``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +======+=======+=============+==============+=============+
+ | Ver |Hdr Len| ToS | TTL | Protocol |
+ +------+-------+-------------+--------------+-------------+
+
+ IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
+
+ IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
+
+ .. table:: Protocol extraction : ``ipv6``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+=============+==============+
+ | Ver |Traffic class| Flow | Next Header | Hop Limit |
+ +-----+-------------+--------+-------------+--------------+
+
+ IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``ipv6_flow``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+============================+
+ | Ver |Traffic class| Flow Label |
+ +-----+-------------+-------------------------------------+
+
+ IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``tcp``
+
+ +----------------------------+----------------------------+
+ | TCPHDR2 | TCPHDR1 |
+ +============================+======+======+==============+
+ | Reserved |Offset| RSV | Flags |
+ +----------------------------+------+------+--------------+
+
+ TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
+
+ TCPHDR2 - Reserved
+
+ Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the protocol
+ extraction, do not use ``rte_mbuf::udata64`` directly.
+
+ The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
+ access the protocol extraction result in ``struct rte_mbuf``.
+
Driver compilation and testing
------------------------------
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 8490d897c..382806229 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -21,6 +21,10 @@ DPDK Release 19.11
xdg-open build/doc/html/guides/rel_notes/release_19_11.html
+* **Updated the ICE driver.**
+
+ * Added support for handling Receive Flex Descriptor.
+ * Added support for protocol extraction on per Rx queue.
New Features
------------
diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile
index ae53c2646..4a279f196 100644
--- a/drivers/net/ice/Makefile
+++ b/drivers/net/ice/Makefile
@@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1)
endif
SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
+
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..7c74b6169 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -19,9 +19,11 @@
/* devargs */
#define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
+#define ICE_PROTO_XTR_ARG "proto_xtr"
static const char * const ice_valid_args[] = {
ICE_SAFE_MODE_SUPPORT_ARG,
+ ICE_PROTO_XTR_ARG,
NULL
};
@@ -257,6 +259,280 @@ ice_init_controlq_parameter(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ;
}
+static int
+lookup_proto_xtr_type(const char *xtr_name)
+{
+ static struct {
+ const char *name;
+ enum proto_xtr_type type;
+ } xtr_type_map[] = {
+ { "vlan", PROTO_XTR_VLAN },
+ { "ipv4", PROTO_XTR_IPV4 },
+ { "ipv6", PROTO_XTR_IPV6 },
+ { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
+ { "tcp", PROTO_XTR_TCP },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
+ if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
+ return xtr_type_map[i].type;
+ }
+
+ return -1;
+}
+
+/*
+ * Parse elem, the elem could be single number/range or '(' ')' group
+ * 1) A single number elem, it's just a simple digit. e.g. 9
+ * 2) A single range elem, two digits with a '-' between. e.g. 2-6
+ * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
+ * Within group elem, '-' used for a range separator;
+ * ',' used for a single number.
+ */
+static int
+parse_queue_set(const char *input, int xtr_type, struct ice_devargs *devargs)
+{
+ const char *str = input;
+ char *end = NULL;
+ uint32_t min, max;
+ uint32_t idx;
+
+ while (isblank(*str))
+ str++;
+
+ if (!isdigit(*str) && *str != '(')
+ return -1;
+
+ /* process single number or single range of number */
+ if (*str != '(') {
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ while (isblank(*end))
+ end++;
+
+ min = idx;
+ max = idx;
+
+ /* process single <number>-<number> */
+ if (*end == '-') {
+ end++;
+ while (isblank(*end))
+ end++;
+ if (!isdigit(*end))
+ return -1;
+
+ errno = 0;
+ idx = strtoul(end, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ max = idx;
+ while (isblank(*end))
+ end++;
+ }
+
+ if (*end != ':')
+ return -1;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ return 0;
+ }
+
+ /* process set within bracket */
+ str++;
+ while (isblank(*str))
+ str++;
+ if (*str == '\0')
+ return -1;
+
+ min = ICE_MAX_QUEUE_NUM;
+ do {
+ /* go ahead to the first digit */
+ while (isblank(*str))
+ str++;
+ if (!isdigit(*str))
+ return -1;
+
+ /* get the digit value */
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ /* go ahead to separator '-',',' and ')' */
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+ else /* avoid continuous '-' */
+ return -1;
+ } else if (*end == ',' || *end == ')') {
+ max = idx;
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ min = ICE_MAX_QUEUE_NUM;
+ } else {
+ return -1;
+ }
+
+ str = end + 1;
+ } while (*end != ')' && *end != '\0');
+
+ return 0;
+}
+
+static int
+parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
+{
+ const char *queue_start;
+ uint32_t idx;
+ int xtr_type;
+ char xtr_name[32];
+
+ while (isblank(*queues))
+ queues++;
+
+ if (*queues != '[') {
+ xtr_type = lookup_proto_xtr_type(queues);
+ if (xtr_type < 0)
+ return -1;
+
+ memset(devargs->proto_xtr, xtr_type,
+ sizeof(devargs->proto_xtr));
+
+ return 0;
+ }
+
+ queues++;
+ do {
+ while (isblank(*queues))
+ queues++;
+ if (*queues == '\0')
+ return -1;
+
+ queue_start = queues;
+
+ /* go across a complete bracket */
+ if (*queue_start == '(') {
+ queues += strcspn(queues, ")");
+ if (*queues != ')')
+ return -1;
+ }
+
+ /* scan the separator ':' */
+ queues += strcspn(queues, ":");
+ if (*queues++ != ':')
+ return -1;
+ while (isblank(*queues))
+ queues++;
+
+ for (idx = 0; ; idx++) {
+ if (isblank(queues[idx]) ||
+ queues[idx] == ',' ||
+ queues[idx] == ']' ||
+ queues[idx] == '\0')
+ break;
+
+ if (idx > sizeof(xtr_name) - 2)
+ return -1;
+
+ xtr_name[idx] = queues[idx];
+ }
+ xtr_name[idx] = '\0';
+ xtr_type = lookup_proto_xtr_type(xtr_name);
+ if (xtr_type < 0)
+ return -1;
+
+ queues += idx;
+
+ while (isblank(*queues) || *queues == ',' || *queues == ']')
+ queues++;
+
+ if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
+ return -1;
+ } while (*queues != '\0');
+
+ return 0;
+}
+
+static int
+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
+ void *extra_args)
+{
+ struct ice_devargs *devargs = extra_args;
+
+ if (value == NULL || extra_args == NULL)
+ return -EINVAL;
+
+ if (parse_queue_proto_xtr(value, devargs) < 0) {
+ PMD_DRV_LOG(ERR,
+ "The protocol extraction parameter is wrong : '%s'",
+ value);
+ return -1;
+ }
+
+ return 0;
+}
+
+static bool
+ice_proto_xtr_support(struct ice_hw *hw)
+{
+#define FLX_REG(val, fld, idx) \
+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
+ static struct {
+ uint32_t rxdid;
+ uint16_t protid_0;
+ uint16_t protid_1;
+ } xtr_sets[] = {
+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
+ ICE_PROT_IPV4_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
+ uint32_t rxdid = xtr_sets[i].rxdid;
+ uint32_t v;
+
+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+
+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+ }
+
+ return true;
+}
+
static int
ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
uint32_t num)
@@ -1079,6 +1355,8 @@ ice_interrupt_handler(void *param)
static int
ice_pf_sw_init(struct rte_eth_dev *dev)
{
+ struct ice_adapter *ad =
+ ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct ice_hw *hw = ICE_PF_TO_HW(pf);
@@ -1088,6 +1366,16 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
pf->lan_nb_qps = pf->lan_nb_qp_max;
+ if (ice_proto_xtr_support(hw))
+ pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
+
+ if (pf->proto_xtr != NULL)
+ rte_memcpy(pf->proto_xtr, ad->devargs.proto_xtr,
+ RTE_MIN((size_t)pf->lan_nb_qps,
+ sizeof(ad->devargs.proto_xtr)));
+ else
+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
+
return 0;
}
@@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev *dev)
return -EINVAL;
}
+ memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
+ sizeof(ad->devargs.proto_xtr));
+
+ ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
+ &handle_proto_xtr_arg, &ad->devargs);
+ if (ret)
+ goto bail;
+
ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
&parse_bool, &ad->devargs.safe_mode_support);
+bail:
rte_kvargs_free(kvlist);
return ret;
}
@@ -1547,6 +1844,7 @@ ice_dev_init(struct rte_eth_dev *dev)
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
return ret;
}
@@ -1672,6 +1970,8 @@ ice_dev_close(struct rte_eth_dev *dev)
rte_free(hw->port_info);
hw->port_info = NULL;
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
+ pf->proto_xtr = NULL;
}
static int
@@ -3795,6 +4095,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ice,
+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
RTE_INIT(ice_init_log)
diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
index f569da833..adbb66322 100644
--- a/drivers/net/ice/ice_ethdev.h
+++ b/drivers/net/ice/ice_ethdev.h
@@ -263,6 +263,7 @@ struct ice_pf {
uint16_t lan_nb_qp_max;
uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
uint16_t base_queue; /* The base queue pairs index in the device */
+ uint8_t *proto_xtr; /* Protocol extraction type for all queues */
struct ice_hw_port_stats stats_offset;
struct ice_hw_port_stats stats;
/* internal packet statistics, it should be excluded from the total */
@@ -273,11 +274,14 @@ struct ice_pf {
struct ice_flow_list flow_list;
};
+#define ICE_MAX_QUEUE_NUM 2048
+
/**
* Cache devargs parse result.
*/
struct ice_devargs {
int safe_mode_support;
+ uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
};
/**
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index d2e36853f..e28310b96 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,6 +13,36 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+static inline uint8_t
+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
+{
+ static uint8_t xtr_map[] = {
+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
+ };
+
+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
+}
+
+static inline uint8_t
+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
+{
+ static uint8_t rxdid_map[] = {
+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
+ };
+ uint8_t rxdid;
+
+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
+
+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
+}
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -84,6 +114,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
+
+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
+ rxq->port_id, rxq->queue_id, rxdid);
+
/* Enable Flexible Descriptors in the queue context which
* allows this driver to select a specific receive descriptor format
*/
@@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
rxq->drop_en = rx_conf->rx_drop_en;
rxq->vsi = vsi;
rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->proto_xtr = pf->proto_xtr != NULL ?
+ pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
/* Allocate the maximun number of RX ring hardware descriptor. */
len = ICE_MAX_RING_DESC;
@@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+#define ICE_RX_PROTO_XTR_VALID \
+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+
static inline void
ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
volatile union ice_rx_flex_desc *rxdp)
@@ -1075,6 +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
mb->ol_flags |= PKT_RX_RSS_HASH;
mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
}
+
+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
+ init_proto_xtr_flds(mb);
+
+ stat_err = rte_le_to_cpu_16(desc->status_error1);
+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
+ xtr->u.raw.data0 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+ xtr->u.raw.data1 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
+
+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
+ xtr->magic = PROTO_XTR_MAGIC_ID;
+ }
+#endif
}
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index 64e891875..de16637f3 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -5,6 +5,7 @@
#ifndef _ICE_RXTX_H_
#define _ICE_RXTX_H_
+#include "rte_pmd_ice.h"
#include "ice_ethdev.h"
#define ICE_ALIGN_RING_DESC 32
@@ -78,6 +79,7 @@ struct ice_rx_queue {
uint16_t max_pkt_len; /* Maximum packet length */
bool q_set; /* indicate if rx queue has been configured */
bool rx_deferred_start; /* don't start this queue in dev start */
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
ice_rx_release_mbufs_t rx_rel_mbufs;
};
diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
index c5f0d564f..080ca4175 100644
--- a/drivers/net/ice/ice_rxtx_vec_common.h
+++ b/drivers/net/ice/ice_rxtx_vec_common.h
@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
if (rxq->nb_rx_desc % rxq->rx_free_thresh)
return -1;
+ if (rxq->proto_xtr != PROTO_XTR_NONE)
+ return -1;
+
return 0;
}
diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
index 36b4b3c85..6828170a9 100644
--- a/drivers/net/ice/meson.build
+++ b/drivers/net/ice/meson.build
@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
endif
endif
+
+install_headers('rte_pmd_ice.h')
diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
new file mode 100644
index 000000000..719487e1e
--- /dev/null
+++ b/drivers/net/ice/rte_pmd_ice.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#ifndef _RTE_PMD_ICE_H_
+#define _RTE_PMD_ICE_H_
+
+#include <stdio.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum proto_xtr_type {
+ PROTO_XTR_NONE,
+ PROTO_XTR_VLAN,
+ PROTO_XTR_IPV4,
+ PROTO_XTR_IPV6,
+ PROTO_XTR_IPV6_FLOW,
+ PROTO_XTR_TCP,
+};
+
+struct proto_xtr_flds {
+ union {
+ struct {
+ uint16_t data0;
+ uint16_t data1;
+ } raw;
+ struct {
+ uint16_t stag_vid:12,
+ stag_dei:1,
+ stag_pcp:3;
+ uint16_t ctag_vid:12,
+ ctag_dei:1,
+ ctag_pcp:3;
+ } vlan;
+ struct {
+ uint16_t protocol:8,
+ ttl:8;
+ uint16_t tos:8,
+ ihl:4,
+ version:4;
+ } ipv4;
+ struct {
+ uint16_t hoplimit:8,
+ nexthdr:8;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6;
+ struct {
+ uint16_t flowlo16;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6_flow;
+ struct {
+ uint16_t fin:1,
+ syn:1,
+ rst:1,
+ psh:1,
+ ack:1,
+ urg:1,
+ ece:1,
+ cwr:1,
+ res1:4,
+ doff:4;
+ uint16_t rsvd;
+ } tcp;
+ } u;
+
+ uint16_t rsvd;
+
+ uint8_t type;
+
+#define PROTO_XTR_MAGIC_ID 0xCE
+ uint8_t magic;
+};
+
+static inline void
+init_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ mb->udata64 = 0;
+}
+
+static inline struct proto_xtr_flds *
+get_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
+
+ return (struct proto_xtr_flds *)&mb->udata64;
+}
+
+static inline void
+dump_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
+ return;
+
+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
+ xtr->u.raw.data0, xtr->u.raw.data1);
+
+ if (xtr->type == PROTO_XTR_VLAN)
+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
+ xtr->u.vlan.stag_pcp,
+ xtr->u.vlan.stag_dei,
+ xtr->u.vlan.stag_vid,
+ xtr->u.vlan.ctag_pcp,
+ xtr->u.vlan.ctag_dei,
+ xtr->u.vlan.ctag_vid);
+ else if (xtr->type == PROTO_XTR_IPV4)
+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
+ xtr->u.ipv4.version,
+ xtr->u.ipv4.ihl,
+ xtr->u.ipv4.tos,
+ xtr->u.ipv4.ttl,
+ xtr->u.ipv4.protocol);
+ else if (xtr->type == PROTO_XTR_IPV6)
+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
+ xtr->u.ipv6.version,
+ xtr->u.ipv6.tc,
+ xtr->u.ipv6.flowhi4,
+ xtr->u.ipv6.nexthdr,
+ xtr->u.ipv6.hoplimit);
+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
+ xtr->u.ipv6_flow.version,
+ xtr->u.ipv6_flow.tc,
+ xtr->u.ipv6_flow.flowhi4,
+ xtr->u.ipv6_flow.flowlo16);
+ else if (xtr->type == PROTO_XTR_TCP)
+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
+ xtr->u.tcp.doff,
+ xtr->u.tcp.cwr ? "C" : "",
+ xtr->u.tcp.ece ? "E" : "",
+ xtr->u.tcp.urg ? "U" : "",
+ xtr->u.tcp.ack ? "A" : "",
+ xtr->u.tcp.psh ? "P" : "",
+ xtr->u.tcp.rst ? "R" : "",
+ xtr->u.tcp.syn ? "S" : "",
+ xtr->u.tcp.fin ? "F" : "");
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_PMD_ICE_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 4/6] net/ice: switch to flexible descriptor in SSE path
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
` (2 preceding siblings ...)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
` (2 subsequent siblings)
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
With this path, the flexible descriptor is supported
in SSE path. And the legacy descriptor is not supported.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx_vec_sse.c | 239 +++++++++++++----------------
1 file changed, 110 insertions(+), 129 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index 967a7b16b..dafcb081a 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -15,14 +15,14 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
struct rte_mbuf *mb0, *mb1;
__m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -88,93 +88,90 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer);
__m128i rearm0, rearm1, rearm2, rearm3;
- __m128i vlan0, vlan1, rss, l3_l4e;
+ __m128i tmp_desc, flags, rss_vlan;
- /* mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication.
+ /* mask everything except checksum, RSS and VLAN flags.
+ * bit6:4 for checksum.
+ * bit12 for RSS indication.
+ * bit13 for VLAN indication.
*/
- const __m128i rss_vlan_msk = _mm_set_epi32(0x1c03804, 0x1c03804,
- 0x1c03804, 0x1c03804);
+ const __m128i desc_mask = _mm_set_epi32(0x3070, 0x3070,
+ 0x3070, 0x3070);
- const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD);
- /* map rss and vlan type to rss hash and vlan flag */
- const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- 0, 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
- 0, 0, 0, 0);
-
- const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH, 0, 0,
- 0, 0, PKT_RX_FDIR, 0);
-
- const __m128i l3_l4e_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ /* map the checksum, rss and vlan fields to the checksum, rss
+ * and vlan flag
+ */
+ const __m128i cksum_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
- vlan0 = _mm_unpackhi_epi32(descs[0], descs[1]);
- vlan1 = _mm_unpackhi_epi32(descs[2], descs[3]);
- vlan0 = _mm_unpacklo_epi64(vlan0, vlan1);
-
- vlan1 = _mm_and_si128(vlan0, rss_vlan_msk);
- vlan0 = _mm_shuffle_epi8(vlan_flags, vlan1);
-
- rss = _mm_srli_epi32(vlan1, 11);
- rss = _mm_shuffle_epi8(rss_flags, rss);
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
- l3_l4e = _mm_srli_epi32(vlan1, 22);
- l3_l4e = _mm_shuffle_epi8(l3_l4e_flags, l3_l4e);
+ const __m128i rss_vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
+
+ /* merge 4 descriptors */
+ flags = _mm_unpackhi_epi32(descs[0], descs[1]);
+ tmp_desc = _mm_unpackhi_epi32(descs[2], descs[3]);
+ tmp_desc = _mm_unpacklo_epi64(flags, tmp_desc);
+ tmp_desc = _mm_and_si128(flags, desc_mask);
+
+ /* checksum flags */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 4);
+ flags = _mm_shuffle_epi8(cksum_flags, tmp_desc);
/* then we shift left 1 bit */
- l3_l4e = _mm_slli_epi32(l3_l4e, 1);
- /* we need to mask out the reduntant bits */
- l3_l4e = _mm_and_si128(l3_l4e, cksum_mask);
+ flags = _mm_slli_epi32(flags, 1);
+ /* we need to mask out the reduntant bits introduced by RSS or
+ * VLAN fields.
+ */
+ flags = _mm_and_si128(flags, cksum_mask);
- vlan0 = _mm_or_si128(vlan0, rss);
- vlan0 = _mm_or_si128(vlan0, l3_l4e);
+ /* RSS, VLAN flag */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 8);
+ rss_vlan = _mm_shuffle_epi8(rss_vlan_flags, tmp_desc);
+
+ /* merge the flags */
+ flags = _mm_or_si128(flags, rss_vlan);
/**
* At this point, we have the 4 sets of flags in the low 16-bits
- * of each 32-bit value in vlan0.
+ * of each 32-bit value in flags.
* We want to extract these, and merge them with the mbuf init data
* so we can do a single 16-byte write to the mbuf to set the flags
* and all the other initialization fields. Extracting the
* appropriate flags means that we have to do a shift and blend for
* each mbuf before we do the write.
*/
- rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 8), 0x10);
- rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 4), 0x10);
- rearm2 = _mm_blend_epi16(mbuf_init, vlan0, 0x10);
- rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10);
+ rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x10);
+ rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x10);
+ rearm2 = _mm_blend_epi16(mbuf_init, flags, 0x10);
+ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x10);
/* write the rearm data and the olflags in one write */
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
@@ -187,22 +184,24 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
_mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3);
}
-#define PKTLEN_SHIFT 10
-
static inline void
ice_rx_desc_to_ptype_v(__m128i descs[4], struct rte_mbuf **rx_pkts,
uint32_t *ptype_tbl)
{
- __m128i ptype0 = _mm_unpackhi_epi64(descs[0], descs[1]);
- __m128i ptype1 = _mm_unpackhi_epi64(descs[2], descs[3]);
-
- ptype0 = _mm_srli_epi64(ptype0, 30);
- ptype1 = _mm_srli_epi64(ptype1, 30);
-
- rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 0)];
- rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 8)];
- rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 0)];
- rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 8)];
+ const __m128i ptype_mask = _mm_set_epi16(0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M);
+ __m128i ptype_01 = _mm_unpacklo_epi32(descs[0], descs[1]);
+ __m128i ptype_23 = _mm_unpacklo_epi32(descs[2], descs[3]);
+ __m128i ptype_all = _mm_unpacklo_epi64(ptype_01, ptype_23);
+
+ ptype_all = _mm_and_si128(ptype_all, ptype_mask);
+
+ rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
}
/**
@@ -215,21 +214,39 @@ static inline uint16_t
_ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
uint64_t var;
- __m128i shuf_msk;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
-
__m128i crc_adjust = _mm_set_epi16
- (0, 0, 0, /* ignore non-length fields */
+ (0, 0, 0, /* ignore non-length fields */
-rxq->crc_len, /* sub crc on data_len */
0, /* ignore high-16bits of pkt_len */
-rxq->crc_len, /* sub crc on pkt_len */
- 0, 0 /* ignore pkt_type field */
+ 0, 0 /* ignore pkt_type field */
);
+ const __m128i zero = _mm_setzero_si128();
+ /* mask to shuffle from desc. to mbuf */
+ const __m128i shuf_msk = _mm_set_epi8
+ (0xFF, 0xFF, 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, low 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /* pkt_type set as unknown */
+ );
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+
/**
* compile-time check the above crc_adjust layout is correct.
* NOTE: the first field (lowest address) is given last in set_epi16
@@ -239,7 +256,13 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
- __m128i dd_check, eop_check;
+
+ /* 4 packets DD mask */
+ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ /* 4 packets EOP mask */
+ const __m128i eop_check = _mm_set_epi64x(0x0000000200000002LL,
+ 0x0000000200000002LL);
/* nb_pkts shall be less equal than ICE_MAX_RX_BURST */
nb_pkts = RTE_MIN(nb_pkts, ICE_MAX_RX_BURST);
@@ -250,7 +273,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = rxq->rx_ring + rxq->rx_tail;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
@@ -263,26 +286,10 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
- /* 4 packets DD mask */
- dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL);
-
- /* 4 packets EOP mask */
- eop_check = _mm_set_epi64x(0x0000000200000002LL, 0x0000000200000002LL);
-
- /* mask to shuffle from desc. to mbuf */
- shuf_msk = _mm_set_epi8
- (7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
- );
/**
* Compile-time verify the shuffle mask
* NOTE: some field positions already verified above, but duplicated
@@ -315,7 +322,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
rxdp += ICE_DESCS_PER_LOOP) {
__m128i descs[ICE_DESCS_PER_LOOP];
__m128i pkt_mb1, pkt_mb2, pkt_mb3, pkt_mb4;
- __m128i zero, staterr, sterr_tmp1, sterr_tmp2;
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
/* 2 64 bit or 4 32 bit mbuf pointers in one XMM reg. */
__m128i mbp1;
#if defined(RTE_ARCH_X86_64)
@@ -359,14 +366,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* avoid compiler reorder optimization */
rte_compiler_barrier();
- /* pkt 3,4 shift the pktlen field to be 16-bit aligned*/
- const __m128i len3 = _mm_slli_epi32(descs[3], PKTLEN_SHIFT);
- const __m128i len2 = _mm_slli_epi32(descs[2], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[3] = _mm_blend_epi16(descs[3], len3, 0x80);
- descs[2] = _mm_blend_epi16(descs[2], len2, 0x80);
-
/* D.1 pkt 3,4 convert format from desc to pktmbuf */
pkt_mb4 = _mm_shuffle_epi8(descs[3], shuf_msk);
pkt_mb3 = _mm_shuffle_epi8(descs[2], shuf_msk);
@@ -382,20 +381,11 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
pkt_mb4 = _mm_add_epi16(pkt_mb4, crc_adjust);
pkt_mb3 = _mm_add_epi16(pkt_mb3, crc_adjust);
- /* pkt 1,2 shift the pktlen field to be 16-bit aligned*/
- const __m128i len1 = _mm_slli_epi32(descs[1], PKTLEN_SHIFT);
- const __m128i len0 = _mm_slli_epi32(descs[0], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[1] = _mm_blend_epi16(descs[1], len1, 0x80);
- descs[0] = _mm_blend_epi16(descs[0], len0, 0x80);
-
/* D.1 pkt 1,2 convert format from desc to pktmbuf */
pkt_mb2 = _mm_shuffle_epi8(descs[1], shuf_msk);
pkt_mb1 = _mm_shuffle_epi8(descs[0], shuf_msk);
/* C.2 get 4 pkts staterr value */
- zero = _mm_xor_si128(dd_check, dd_check);
staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
/* D.3 copy final 3,4 data to rx_pkts */
@@ -412,15 +402,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* C* extract and record EOP bit */
if (split_packet) {
- __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0x04, 0x0C,
- 0x00, 0x08);
-
/* and with mask to extract bits, flipping 1-0 */
__m128i eop_bits = _mm_andnot_si128(staterr, eop_check);
/* the staterr values are not in order, as the count
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 5/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
` (3 preceding siblings ...)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
2019-09-19 6:38 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Zhang, Qi Z
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
Switch to Rx flexible descriptor format instead of legacy
descriptor format.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
---
drivers/net/ice/ice_rxtx_vec_avx2.c | 224 ++++++++++++++--------------
1 file changed, 109 insertions(+), 115 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 5ce29c2a2..46776fa12 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -132,8 +132,6 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
ICE_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
}
-#define PKTLEN_SHIFT 10
-
static inline uint16_t
_ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
@@ -144,7 +142,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp =
+ (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
@@ -161,8 +160,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/* constants used in processing loop */
@@ -193,21 +192,23 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i shuf_msk =
_mm256_set_epi8
(/* first descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF, /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF, /*pkt_type set as unknown */
/* second descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /*pkt_type set as unknown */
);
/**
* compile-time check the above crc and shuffle layout is correct.
@@ -225,68 +226,68 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Status/Error flag masks */
/**
- * mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication. Bits 3-5 of error
- * field (bits 22-24) are for IP/L4 checksum errors
+ * mask everything except Checksum Reports, RSS indication
+ * and VLAN indication.
+ * bit6:4 for IP/L4 checksum errors.
+ * bit12 is for RSS indication.
+ * bit13 is for VLAN indication.
*/
const __m256i flags_mask =
- _mm256_set1_epi32((1 << 2) | (1 << 11) |
- (3 << 12) | (7 << 22));
- /**
- * data to be shuffled by result of flag mask. If VLAN bit is set,
- * (bit 2), then position 4 in this array will be used in the
- * destination
- */
- const __m256i vlan_flags_shuf =
- _mm256_set_epi32(0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0,
- 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0);
- /**
- * data to be shuffled by result of flag mask, shifted down 11.
- * If RSS/FDIR bits are set, shuffle moves appropriate flags in
- * place.
- */
- const __m256i rss_flags_shuf =
- _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0,/* end up 128-bits */
- 0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0);
-
+ _mm256_set1_epi32((7 << 4) | (1 << 12) | (1 << 13));
/**
- * data to be shuffled by the result of the flags mask shifted by 22
+ * data to be shuffled by the result of the flags mask shifted by 4
* bits. This gives use the l3_l4 flags.
*/
const __m256i l3_l4_flags_shuf = _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1,
/* second 128-bits */
0, 0, 0, 0, 0, 0, 0, 0,
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
const __m256i cksum_mask =
_mm256_set1_epi32(PKT_RX_IP_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD |
PKT_RX_L4_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_EIP_CKSUM_BAD);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If RSS(bit12)/VLAN(bit13) are set,
+ * shuffle moves appropriate flags in place.
+ */
+ const __m256i rss_vlan_flags_shuf = _mm256_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
RTE_SET_USED(avx_aligned); /* for 32B descriptors we don't use this */
@@ -369,73 +370,66 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
}
/**
- * convert descriptors 4-7 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 4-7 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len6_7 = _mm256_slli_epi32(raw_desc6_7,
- PKTLEN_SHIFT);
- const __m256i len4_5 = _mm256_slli_epi32(raw_desc4_5,
- PKTLEN_SHIFT);
- const __m256i desc6_7 = _mm256_blend_epi16(raw_desc6_7,
- len6_7, 0x80);
- const __m256i desc4_5 = _mm256_blend_epi16(raw_desc4_5,
- len4_5, 0x80);
- __m256i mb6_7 = _mm256_shuffle_epi8(desc6_7, shuf_msk);
- __m256i mb4_5 = _mm256_shuffle_epi8(desc4_5, shuf_msk);
+ __m256i mb6_7 = _mm256_shuffle_epi8(raw_desc6_7, shuf_msk);
+ __m256i mb4_5 = _mm256_shuffle_epi8(raw_desc4_5, shuf_msk);
mb6_7 = _mm256_add_epi16(mb6_7, crc_adjust);
mb4_5 = _mm256_add_epi16(mb4_5, crc_adjust);
/**
- * to get packet types, shift 64-bit values down 30 bits
- * and so ptype is in lower 8-bits in each
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
*/
- const __m256i ptypes6_7 = _mm256_srli_epi64(desc6_7, 30);
- const __m256i ptypes4_5 = _mm256_srli_epi64(desc4_5, 30);
- const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 24);
- const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 8);
- const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 24);
- const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 8);
+ const __m256i ptype_mask =
+ _mm256_set1_epi16(ICE_RX_FLEX_DESC_PTYPE_M);
+ const __m256i ptypes6_7 =
+ _mm256_and_si256(raw_desc6_7, ptype_mask);
+ const __m256i ptypes4_5 =
+ _mm256_and_si256(raw_desc4_5, ptype_mask);
+ const uint16_t ptype7 = _mm256_extract_epi16(ptypes6_7, 9);
+ const uint16_t ptype6 = _mm256_extract_epi16(ptypes6_7, 1);
+ const uint16_t ptype5 = _mm256_extract_epi16(ptypes4_5, 9);
+ const uint16_t ptype4 = _mm256_extract_epi16(ptypes4_5, 1);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype7], 4);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype6], 0);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype5], 4);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype4], 0);
/* merge the status bits into one register */
- const __m256i status4_7 = _mm256_unpackhi_epi32(desc6_7,
- desc4_5);
+ const __m256i status4_7 = _mm256_unpackhi_epi32(raw_desc6_7,
+ raw_desc4_5);
/**
- * convert descriptors 0-3 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 0-3 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len2_3 = _mm256_slli_epi32(raw_desc2_3,
- PKTLEN_SHIFT);
- const __m256i len0_1 = _mm256_slli_epi32(raw_desc0_1,
- PKTLEN_SHIFT);
- const __m256i desc2_3 = _mm256_blend_epi16(raw_desc2_3,
- len2_3, 0x80);
- const __m256i desc0_1 = _mm256_blend_epi16(raw_desc0_1,
- len0_1, 0x80);
- __m256i mb2_3 = _mm256_shuffle_epi8(desc2_3, shuf_msk);
- __m256i mb0_1 = _mm256_shuffle_epi8(desc0_1, shuf_msk);
+ __m256i mb2_3 = _mm256_shuffle_epi8(raw_desc2_3, shuf_msk);
+ __m256i mb0_1 = _mm256_shuffle_epi8(raw_desc0_1, shuf_msk);
mb2_3 = _mm256_add_epi16(mb2_3, crc_adjust);
mb0_1 = _mm256_add_epi16(mb0_1, crc_adjust);
- /* get the packet types */
- const __m256i ptypes2_3 = _mm256_srli_epi64(desc2_3, 30);
- const __m256i ptypes0_1 = _mm256_srli_epi64(desc0_1, 30);
- const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 24);
- const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 8);
- const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 24);
- const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 8);
+ /**
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
+ */
+ const __m256i ptypes2_3 =
+ _mm256_and_si256(raw_desc2_3, ptype_mask);
+ const __m256i ptypes0_1 =
+ _mm256_and_si256(raw_desc0_1, ptype_mask);
+ const uint16_t ptype3 = _mm256_extract_epi16(ptypes2_3, 9);
+ const uint16_t ptype2 = _mm256_extract_epi16(ptypes2_3, 1);
+ const uint16_t ptype1 = _mm256_extract_epi16(ptypes0_1, 9);
+ const uint16_t ptype0 = _mm256_extract_epi16(ptypes0_1, 1);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype3], 4);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype2], 0);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype1], 4);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype0], 0);
/* merge the status bits into one register */
- const __m256i status0_3 = _mm256_unpackhi_epi32(desc2_3,
- desc0_1);
+ const __m256i status0_3 = _mm256_unpackhi_epi32(raw_desc2_3,
+ raw_desc0_1);
/**
* take the two sets of status bits and merge to one
@@ -450,24 +444,24 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* get only flag/error bits we want */
const __m256i flag_bits =
_mm256_and_si256(status0_7, flags_mask);
- /* set vlan and rss flags */
- const __m256i vlan_flags =
- _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
- const __m256i rss_flags =
- _mm256_shuffle_epi8(rss_flags_shuf,
- _mm256_srli_epi32(flag_bits, 11));
/**
* l3_l4_error flags, shuffle, then shift to correct adjustment
* of flags in flags_shuf, and finally mask out extra bits
*/
__m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
- _mm256_srli_epi32(flag_bits, 22));
+ _mm256_srli_epi32(flag_bits, 4));
l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
+ /* set rss and vlan flags */
+ const __m256i rss_vlan_flag_bits =
+ _mm256_srli_epi32(flag_bits, 12);
+ const __m256i rss_vlan_flags =
+ _mm256_shuffle_epi8(rss_vlan_flags_shuf,
+ rss_vlan_flag_bits);
/* merge flags */
const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
- _mm256_or_si256(rss_flags, vlan_flags));
+ rss_vlan_flags);
/**
* At this point, we have the 8 sets of flags in the low 16-bits
* of each 32-bit value in vlan0.
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
` (4 preceding siblings ...)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
@ 2019-09-19 6:25 ` Leyi Rong
2019-09-23 14:31 ` Ye Xiaolong
2019-09-19 6:38 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Zhang, Qi Z
6 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-09-19 6:25 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Since now the ice PMD only handles Rx Flex descriptor, so remove the
legacy descriptor definition.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 25 ++++++++++++-------------
drivers/net/ice/ice_rxtx.h | 4 +---
drivers/net/ice/ice_rxtx_vec_avx2.c | 5 ++---
drivers/net/ice/ice_rxtx_vec_sse.c | 4 ++--
4 files changed, 17 insertions(+), 21 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index e28310b96..40186131f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -171,7 +171,7 @@ ice_alloc_rx_queue_mbufs(struct ice_rx_queue *rxq)
uint16_t i;
for (i = 0; i < rxq->nb_rx_desc; i++) {
- volatile union ice_rx_desc *rxd;
+ volatile union ice_rx_flex_desc *rxd;
struct rte_mbuf *mbuf = rte_mbuf_raw_alloc(rxq->mp);
if (unlikely(!mbuf)) {
@@ -346,7 +346,7 @@ ice_reset_rx_queue(struct ice_rx_queue *rxq)
#endif /* RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC */
len = rxq->nb_rx_desc;
- for (i = 0; i < len * sizeof(union ice_rx_desc); i++)
+ for (i = 0; i < len * sizeof(union ice_rx_flex_desc); i++)
((volatile char *)rxq->rx_ring)[i] = 0;
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
@@ -691,7 +691,7 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
#endif
/* Allocate the maximum number of RX ring hardware descriptor. */
- ring_size = sizeof(union ice_rx_desc) * len;
+ ring_size = sizeof(union ice_rx_flex_desc) * len;
ring_size = RTE_ALIGN(ring_size, ICE_DMA_MEM_ALIGN);
rz = rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx,
ring_size, ICE_RING_BASE_ALIGN,
@@ -1008,7 +1008,7 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
+ rxdp = &rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
rte_le_to_cpu_16(rxdp->wb.status_error0) &
(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
@@ -1020,8 +1020,7 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = (volatile union ice_rx_flex_desc *)
- &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
@@ -1156,7 +1155,7 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
+ rxdp = &rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
@@ -1241,7 +1240,7 @@ ice_rx_fill_from_stage(struct ice_rx_queue *rxq,
static inline int
ice_rx_alloc_bufs(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
uint16_t alloc_idx, i;
@@ -1376,7 +1375,7 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_pkts)
{
struct ice_rx_queue *rxq = rx_queue;
- volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
+ volatile union ice_rx_flex_desc *rx_ring = rxq->rx_ring;
volatile union ice_rx_flex_desc *rxdp;
union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
@@ -1396,7 +1395,7 @@ ice_recv_scattered_pkts(void *rx_queue,
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rxdp = &rx_ring[rx_id];
rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
@@ -1608,7 +1607,7 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ rxdp = &rxq->rx_ring[desc];
if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
@@ -1695,7 +1694,7 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_pkts)
{
struct ice_rx_queue *rxq = rx_queue;
- volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
+ volatile union ice_rx_flex_desc *rx_ring = rxq->rx_ring;
volatile union ice_rx_flex_desc *rxdp;
union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
@@ -1713,7 +1712,7 @@ ice_recv_pkts(void *rx_queue,
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rxdp = &rx_ring[rx_id];
rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index de16637f3..25b3822df 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,10 +21,8 @@
#define ICE_CHK_Q_ENA_INTERVAL_US 100
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
-#define ice_rx_desc ice_16byte_rx_desc
#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
-#define ice_rx_desc ice_32byte_rx_desc
#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
@@ -48,7 +46,7 @@ struct ice_rx_entry {
struct ice_rx_queue {
struct rte_mempool *mp; /* mbuf pool to populate RX ring */
- volatile union ice_rx_desc *rx_ring;/* RX ring virtual address */
+ volatile union ice_rx_flex_desc *rx_ring;/* RX ring virtual address */
rte_iova_t rx_ring_dma; /* RX ring DMA address */
struct ice_rx_entry *sw_ring; /* address of RX soft ring */
uint16_t nb_rx_desc; /* number of RX descriptors */
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 46776fa12..f32222bb4 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -18,7 +18,7 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -142,8 +142,7 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_flex_desc *rxdp =
- (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index dafcb081a..2ae9370f4 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -22,7 +22,7 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -273,7 +273,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
+ rxdp = rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
` (5 preceding siblings ...)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
@ 2019-09-19 6:38 ` Zhang, Qi Z
6 siblings, 0 replies; 54+ messages in thread
From: Zhang, Qi Z @ 2019-09-19 6:38 UTC (permalink / raw)
To: Rong, Leyi, Wang, Haiyue, Lu, Wenzhuo, Ye, Xiaolong; +Cc: dev
> -----Original Message-----
> From: Rong, Leyi
> Sent: Thursday, September 19, 2019 2:26 PM
> To: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> <xiaolong.ye@intel.com>
> Cc: dev@dpdk.org; Rong, Leyi <leyi.rong@intel.com>
> Subject: [PATCH v4 0/6] enable Rx flexible descriptor
>
> This patchset enable Rx flexible descriptor for ice PMD in both normal path and
> vector path.
> Depends on shared code update patchset.
>
> ---
> v4:
> - Handle the ‘proto_xtr’ with error processing, and remove the
> Redundant default type variable.
> - Add flex descriptor and protocol extraction release note.
> - Clean up the legacy descriptor.
>
> v3:
> - Parse the ‘proto_xtr’ devargs before CVL HW initialization and
> save it for later accessing.
> - Merge shuffle ops on vlan and rss flag on vector path.
>
> v2:
> - Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
> RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
> when handling Rx packets.
>
> Haiyue Wang (4):
> net/ice: add Rx flex descriptor definition
> net/ice: handle the Rx flex descriptor
> net/ice: add protocol extraction support for per Rx queue
> net/ice: remove Rx legacy descriptor definition
>
> Leyi Rong (1):
> net/ice: switch to Rx flexible descriptor in AVX path
>
> Wenzhuo Lu (1):
> net/ice: switch to flexible descriptor in SSE path
>
> doc/guides/nics/ice.rst | 101 +++++++++
> doc/guides/rel_notes/release_19_11.rst | 4 +
> drivers/net/ice/Makefile | 3 +
> drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
> drivers/net/ice/ice_ethdev.h | 4 +
> drivers/net/ice/ice_rxtx.c | 298 +++++++++++++-----------
> drivers/net/ice/ice_rxtx.h | 8 +-
> drivers/net/ice/ice_rxtx_vec_avx2.c | 221 +++++++++---------
> drivers/net/ice/ice_rxtx_vec_common.h | 3 +
> drivers/net/ice/ice_rxtx_vec_sse.c | 235 +++++++++----------
> drivers/net/ice/meson.build | 2 +
> drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
> 12 files changed, 962 insertions(+), 370 deletions(-) create mode 100644
> drivers/net/ice/rte_pmd_ice.h
>
> --
> 2.17.1
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-23 3:25 ` Yang, Qiming
2019-09-23 3:34 ` Wang, Haiyue
2019-09-23 8:29 ` Ye Xiaolong
2019-09-23 14:24 ` Ye Xiaolong
1 sibling, 2 replies; 54+ messages in thread
From: Yang, Qiming @ 2019-09-23 3:25 UTC (permalink / raw)
To: Rong, Leyi, Wang, Haiyue, Lu, Wenzhuo, Zhang, Qi Z, Ye, Xiaolong; +Cc: dev
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Leyi Rong
> Sent: Thursday, September 19, 2019 2:26 PM
> To: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> <xiaolong.ye@intel.com>
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support
> for per Rx queue
>
> From: Haiyue Wang <haiyue.wang@intel.com>
>
> The ice has the feature to extract protocol fields into flex descriptor by
> programming per queue. Currently, the ice PMD will put the protocol fields
> into rte_mbuf::udata64 with different type format. Application can access
> the protocol fields quickly.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
> doc/guides/nics/ice.rst | 101 +++++++++
> doc/guides/rel_notes/release_19_11.rst | 4 +
> drivers/net/ice/Makefile | 3 +
> drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
> drivers/net/ice/ice_ethdev.h | 4 +
> drivers/net/ice/ice_rxtx.c | 61 +++++
> drivers/net/ice/ice_rxtx.h | 2 +
> drivers/net/ice/ice_rxtx_vec_common.h | 3 +
> drivers/net/ice/meson.build | 2 +
> drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
> 10 files changed, 633 insertions(+)
> create mode 100644 drivers/net/ice/rte_pmd_ice.h
>
> diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst index
> 03819d29f..8a6f60e71 100644
> --- a/doc/guides/nics/ice.rst
> +++ b/doc/guides/nics/ice.rst
> @@ -61,6 +61,107 @@ Runtime Config Options
> NOTE: In Safe mode, only very limited features are available, features like
> RSS,
> checksum, fdir, tunneling ... are all disabled.
>
> +- ``Protocol extraction for per queue``
> +
> + Configure the RX queues to do protocol extraction into
> + ``rte_mbuf::udata64`` for protocol handling acceleration, like checking the
> TCP SYN packets quickly.
> +
> + The argument format is::
> +
> + -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
> + -w 18:00.0,proto_xtr=<protocol>
> +
> + Queues are grouped by ``(`` and ``)`` within the group. The ``-``
> + character is used as a range separator and ``,`` is used as a single number
> separator.
> + The grouping ``()`` can be omitted for single element group. If no
> + queues are specified, PMD will use this protocol extraction type for all
> queues.
> +
> + Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
> +
> + .. code-block:: console
> +
> + testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
> +
> + This setting means queues 1, 2-3, 8-9 are TCP extraction, queues
> + 10-13 are VLAN extraction, other queues run with no protocol extraction.
> +
> + .. code-block:: console
> +
> + testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
> +
> + This setting means queues 1, 2-3, 8-9 are TCP extraction, queues
> + 10-23 are
> + IPv6 extraction, other queues use the default VLAN extraction.
> +
> + The extraction will be copied into the lower 32 bit of
> ``rte_mbuf::udata64``.
> +
> + .. table:: Protocol extraction : ``vlan``
> +
> + +----------------------------+----------------------------+
> + | VLAN2 | VLAN1 |
> + +======+===+=================+======+===+=================+
> + | PCP | D | VID | PCP | D | VID |
> + +------+---+-----------------+------+---+-----------------+
> +
> + VLAN1 - single or EVLAN (first for QinQ).
> +
> + VLAN2 - C-VLAN (second for QinQ).
> +
> + .. table:: Protocol extraction : ``ipv4``
> +
> + +----------------------------+----------------------------+
> + | IPHDR2 | IPHDR1 |
> + +======+=======+=============+==============+=============+
> + | Ver |Hdr Len| ToS | TTL | Protocol |
> + +------+-------+-------------+--------------+-------------+
> +
> + IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
> +
> + IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
> +
> + .. table:: Protocol extraction : ``ipv6``
> +
> + +----------------------------+----------------------------+
> + | IPHDR2 | IPHDR1 |
> + +=====+=============+========+=============+==============+
> + | Ver |Traffic class| Flow | Next Header | Hop Limit |
> + +-----+-------------+--------+-------------+--------------+
> +
> + IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
> +
> + IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits
> + of "Flow Label" fields.
> +
> + .. table:: Protocol extraction : ``ipv6_flow``
> +
> + +----------------------------+----------------------------+
> + | IPHDR2 | IPHDR1 |
> + +=====+=============+========+============================+
> + | Ver |Traffic class| Flow Label |
> + +-----+-------------+-------------------------------------+
> +
> + IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
> +
> + IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits
> + of "Flow Label" fields.
> +
> + .. table:: Protocol extraction : ``tcp``
> +
> + +----------------------------+----------------------------+
> + | TCPHDR2 | TCPHDR1 |
> + +============================+======+======+==============+
> + | Reserved |Offset| RSV | Flags |
> + +----------------------------+------+------+--------------+
> +
> + TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
> +
> + TCPHDR2 - Reserved
> +
> + Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the
> + protocol extraction, do not use ``rte_mbuf::udata64`` directly.
> +
> + The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
> + access the protocol extraction result in ``struct rte_mbuf``.
> +
> Driver compilation and testing
> ------------------------------
>
> diff --git a/doc/guides/rel_notes/release_19_11.rst
> b/doc/guides/rel_notes/release_19_11.rst
> index 8490d897c..382806229 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -21,6 +21,10 @@ DPDK Release 19.11
>
> xdg-open build/doc/html/guides/rel_notes/release_19_11.html
>
> +* **Updated the ICE driver.**
> +
> + * Added support for handling Receive Flex Descriptor.
> + * Added support for protocol extraction on per Rx queue.
>
> New Features
> ------------
> diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile index
> ae53c2646..4a279f196 100644
> --- a/drivers/net/ice/Makefile
> +++ b/drivers/net/ice/Makefile
> @@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1) endif
> SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
>
> +# install this header file
> +SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
> +
> include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index
> 44a14cb8a..7c74b6169 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -19,9 +19,11 @@
>
> /* devargs */
> #define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
> +#define ICE_PROTO_XTR_ARG "proto_xtr"
>
> static const char * const ice_valid_args[] = {
> ICE_SAFE_MODE_SUPPORT_ARG,
> + ICE_PROTO_XTR_ARG,
> NULL
> };
>
> @@ -257,6 +259,280 @@ ice_init_controlq_parameter(struct ice_hw *hw)
> hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ; }
>
> +static int
> +lookup_proto_xtr_type(const char *xtr_name) {
> + static struct {
> + const char *name;
> + enum proto_xtr_type type;
> + } xtr_type_map[] = {
> + { "vlan", PROTO_XTR_VLAN },
> + { "ipv4", PROTO_XTR_IPV4 },
> + { "ipv6", PROTO_XTR_IPV6 },
> + { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
> + { "tcp", PROTO_XTR_TCP },
> + };
> + uint32_t i;
> +
> + for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
> + if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
> + return xtr_type_map[i].type;
> + }
> +
> + return -1;
> +}
> +
> +/*
> + * Parse elem, the elem could be single number/range or '(' ')' group
> + * 1) A single number elem, it's just a simple digit. e.g. 9
> + * 2) A single range elem, two digits with a '-' between. e.g. 2-6
> + * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
> + * Within group elem, '-' used for a range separator;
> + * ',' used for a single number.
> + */
> +static int
> +parse_queue_set(const char *input, int xtr_type, struct ice_devargs
> +*devargs) {
> + const char *str = input;
> + char *end = NULL;
> + uint32_t min, max;
> + uint32_t idx;
> +
> + while (isblank(*str))
> + str++;
> +
> + if (!isdigit(*str) && *str != '(')
> + return -1;
> +
> + /* process single number or single range of number */
> + if (*str != '(') {
> + errno = 0;
> + idx = strtoul(str, &end, 10);
> + if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> + return -1;
> +
> + while (isblank(*end))
> + end++;
> +
> + min = idx;
> + max = idx;
> +
> + /* process single <number>-<number> */
> + if (*end == '-') {
> + end++;
> + while (isblank(*end))
> + end++;
> + if (!isdigit(*end))
> + return -1;
> +
> + errno = 0;
> + idx = strtoul(end, &end, 10);
> + if (errno || end == NULL || idx >=
> ICE_MAX_QUEUE_NUM)
> + return -1;
> +
> + max = idx;
> + while (isblank(*end))
> + end++;
> + }
> +
> + if (*end != ':')
> + return -1;
> +
> + for (idx = RTE_MIN(min, max);
> + idx <= RTE_MAX(min, max); idx++)
> + devargs->proto_xtr[idx] = xtr_type;
> +
> + return 0;
> + }
> +
> + /* process set within bracket */
> + str++;
> + while (isblank(*str))
> + str++;
> + if (*str == '\0')
> + return -1;
> +
> + min = ICE_MAX_QUEUE_NUM;
> + do {
> + /* go ahead to the first digit */
> + while (isblank(*str))
> + str++;
> + if (!isdigit(*str))
> + return -1;
> +
> + /* get the digit value */
> + errno = 0;
> + idx = strtoul(str, &end, 10);
> + if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> + return -1;
> +
> + /* go ahead to separator '-',',' and ')' */
> + while (isblank(*end))
> + end++;
> + if (*end == '-') {
> + if (min == ICE_MAX_QUEUE_NUM)
> + min = idx;
> + else /* avoid continuous '-' */
> + return -1;
> + } else if (*end == ',' || *end == ')') {
> + max = idx;
> + if (min == ICE_MAX_QUEUE_NUM)
> + min = idx;
> +
> + for (idx = RTE_MIN(min, max);
> + idx <= RTE_MAX(min, max); idx++)
> + devargs->proto_xtr[idx] = xtr_type;
> +
> + min = ICE_MAX_QUEUE_NUM;
> + } else {
> + return -1;
> + }
> +
> + str = end + 1;
> + } while (*end != ')' && *end != '\0');
> +
> + return 0;
> +}
> +
> +static int
> +parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
> +{
> + const char *queue_start;
> + uint32_t idx;
> + int xtr_type;
> + char xtr_name[32];
> +
> + while (isblank(*queues))
> + queues++;
> +
> + if (*queues != '[') {
> + xtr_type = lookup_proto_xtr_type(queues);
> + if (xtr_type < 0)
> + return -1;
> +
> + memset(devargs->proto_xtr, xtr_type,
> + sizeof(devargs->proto_xtr));
> +
> + return 0;
> + }
> +
> + queues++;
> + do {
> + while (isblank(*queues))
> + queues++;
> + if (*queues == '\0')
> + return -1;
> +
> + queue_start = queues;
> +
> + /* go across a complete bracket */
> + if (*queue_start == '(') {
> + queues += strcspn(queues, ")");
> + if (*queues != ')')
> + return -1;
> + }
> +
> + /* scan the separator ':' */
> + queues += strcspn(queues, ":");
> + if (*queues++ != ':')
> + return -1;
> + while (isblank(*queues))
> + queues++;
> +
> + for (idx = 0; ; idx++) {
> + if (isblank(queues[idx]) ||
> + queues[idx] == ',' ||
> + queues[idx] == ']' ||
> + queues[idx] == '\0')
> + break;
> +
> + if (idx > sizeof(xtr_name) - 2)
> + return -1;
> +
> + xtr_name[idx] = queues[idx];
> + }
> + xtr_name[idx] = '\0';
> + xtr_type = lookup_proto_xtr_type(xtr_name);
> + if (xtr_type < 0)
> + return -1;
> +
> + queues += idx;
> +
> + while (isblank(*queues) || *queues == ',' || *queues == ']')
> + queues++;
> +
> + if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
> + return -1;
> + } while (*queues != '\0');
> +
> + return 0;
> +}
> +
> +static int
> +handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
> + void *extra_args)
> +{
> + struct ice_devargs *devargs = extra_args;
> +
> + if (value == NULL || extra_args == NULL)
> + return -EINVAL;
> +
> + if (parse_queue_proto_xtr(value, devargs) < 0) {
> + PMD_DRV_LOG(ERR,
> + "The protocol extraction parameter is wrong : '%s'",
> + value);
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static bool
> +ice_proto_xtr_support(struct ice_hw *hw) { #define FLX_REG(val, fld,
> +idx) \
> + (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
> + GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
> + static struct {
> + uint32_t rxdid;
> + uint16_t protid_0;
> + uint16_t protid_1;
> + } xtr_sets[] = {
> + { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O,
> ICE_PROT_VLAN_O },
> + { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
> + ICE_PROT_IPV4_OF_OR_S },
> + { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
> + ICE_PROT_IPV6_OF_OR_S },
> + { ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> ICE_PROT_IPV6_OF_OR_S,
> + ICE_PROT_IPV6_OF_OR_S },
> + { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL,
> ICE_PROT_ID_INVAL },
> + };
> + uint32_t i;
> +
> + for (i = 0; i < RTE_DIM(xtr_sets); i++) {
> + uint32_t rxdid = xtr_sets[i].rxdid;
> + uint32_t v;
> +
> + if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
> + v = ICE_READ_REG(hw,
> GLFLXP_RXDID_FLX_WRD_4(rxdid));
> +
> + if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0
> ||
> + FLX_REG(v, RXDID_OPCODE, 4) !=
> ICE_RX_OPC_EXTRACT)
> + return false;
> + }
> +
> + if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
> + v = ICE_READ_REG(hw,
> GLFLXP_RXDID_FLX_WRD_5(rxdid));
> +
> + if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1
> ||
> + FLX_REG(v, RXDID_OPCODE, 5) !=
> ICE_RX_OPC_EXTRACT)
> + return false;
> + }
> + }
> +
> + return true;
> +}
> +
> static int
> ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
> uint32_t num)
> @@ -1079,6 +1355,8 @@ ice_interrupt_handler(void *param) static int
> ice_pf_sw_init(struct rte_eth_dev *dev) {
> + struct ice_adapter *ad =
> + ICE_DEV_PRIVATE_TO_ADAPTER(dev->data-
> >dev_private);
> struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> struct ice_hw *hw = ICE_PF_TO_HW(pf);
>
> @@ -1088,6 +1366,16 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
>
> pf->lan_nb_qps = pf->lan_nb_qp_max;
>
> + if (ice_proto_xtr_support(hw))
> + pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
> +
> + if (pf->proto_xtr != NULL)
> + rte_memcpy(pf->proto_xtr, ad->devargs.proto_xtr,
> + RTE_MIN((size_t)pf->lan_nb_qps,
> + sizeof(ad->devargs.proto_xtr)));
> + else
> + PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
> +
> return 0;
> }
>
> @@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev
> *dev)
> return -EINVAL;
> }
>
> + memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
> + sizeof(ad->devargs.proto_xtr));
> +
> + ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
> + &handle_proto_xtr_arg, &ad->devargs);
> + if (ret)
> + goto bail;
> +
Why is bail?
> ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
> &parse_bool, &ad-
> >devargs.safe_mode_support);
>
> +bail:
> rte_kvargs_free(kvlist);
> return ret;
> }
> @@ -1547,6 +1844,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> ice_sched_cleanup_all(hw);
> rte_free(hw->port_info);
> ice_shutdown_all_ctrlq(hw);
> + rte_free(pf->proto_xtr);
>
> return ret;
> }
> @@ -1672,6 +1970,8 @@ ice_dev_close(struct rte_eth_dev *dev)
> rte_free(hw->port_info);
> hw->port_info = NULL;
> ice_shutdown_all_ctrlq(hw);
> + rte_free(pf->proto_xtr);
> + pf->proto_xtr = NULL;
> }
>
> static int
> @@ -3795,6 +4095,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
> RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
> RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-
> pci"); RTE_PMD_REGISTER_PARAM_STRING(net_ice,
> + ICE_PROTO_XTR_ARG
> "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
> ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
>
> RTE_INIT(ice_init_log)
> diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h index
> f569da833..adbb66322 100644
> --- a/drivers/net/ice/ice_ethdev.h
> +++ b/drivers/net/ice/ice_ethdev.h
> @@ -263,6 +263,7 @@ struct ice_pf {
> uint16_t lan_nb_qp_max;
> uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
> uint16_t base_queue; /* The base queue pairs index in the device
> */
> + uint8_t *proto_xtr; /* Protocol extraction type for all queues */
> struct ice_hw_port_stats stats_offset;
> struct ice_hw_port_stats stats;
> /* internal packet statistics, it should be excluded from the total */
> @@ -273,11 +274,14 @@ struct ice_pf {
> struct ice_flow_list flow_list;
> };
>
> +#define ICE_MAX_QUEUE_NUM 2048
> +
> /**
> * Cache devargs parse result.
> */
> struct ice_devargs {
> int safe_mode_support;
> + uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
> };
>
> /**
> diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c index
> d2e36853f..e28310b96 100644
> --- a/drivers/net/ice/ice_rxtx.c
> +++ b/drivers/net/ice/ice_rxtx.c
> @@ -13,6 +13,36 @@
> PKT_TX_TCP_SEG | \
> PKT_TX_OUTER_IP_CKSUM)
>
> +static inline uint8_t
> +ice_rxdid_to_proto_xtr_type(uint8_t rxdid) {
> + static uint8_t xtr_map[] = {
> + [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
> + [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
> + [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
> + [ICE_RXDID_COMMS_AUX_IPV6_FLOW] =
> PROTO_XTR_IPV6_FLOW,
> + [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
> + };
> +
> + return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] :
> PROTO_XTR_NONE; }
> +
> +static inline uint8_t
> +ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye) {
> + static uint8_t rxdid_map[] = {
> + [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
> + [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
> + [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
> + [PROTO_XTR_IPV6_FLOW] =
> ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> + [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
> + };
> + uint8_t rxdid;
> +
> + rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
> +
> + return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC; }
>
> static enum ice_status
> ice_program_hw_rx_queue(struct ice_rx_queue *rxq) @@ -84,6 +114,11
> @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
> rx_ctx.showiv = 0;
> rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
>
> + rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
> +
> + PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with
> RXDID : %u",
> + rxq->port_id, rxq->queue_id, rxdid);
> +
> /* Enable Flexible Descriptors in the queue context which
> * allows this driver to select a specific receive descriptor format
> */
> @@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
> rxq->drop_en = rx_conf->rx_drop_en;
> rxq->vsi = vsi;
> rxq->rx_deferred_start = rx_conf->rx_deferred_start;
> + rxq->proto_xtr = pf->proto_xtr != NULL ?
> + pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
>
> /* Allocate the maximun number of RX ring hardware descriptor. */
> len = ICE_MAX_RING_DESC;
> @@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile
> union ice_rx_flex_desc *rxdp)
> mb->vlan_tci, mb->vlan_tci_outer); }
>
> +#define ICE_RX_PROTO_XTR_VALID \
> + ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
> + (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> +
> static inline void
> ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> volatile union ice_rx_flex_desc *rxdp) @@ -1075,6
> +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> mb->ol_flags |= PKT_RX_RSS_HASH;
> mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
> }
> +
> +#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
> + init_proto_xtr_flds(mb);
> +
> + stat_err = rte_le_to_cpu_16(desc->status_error1);
> + if (stat_err & ICE_RX_PROTO_XTR_VALID) {
> + struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> +
> + if (stat_err & (1 <<
> ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
> + xtr->u.raw.data0 =
> + rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
> +
> + if (stat_err & (1 <<
> ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> + xtr->u.raw.data1 =
> + rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
> +
> + xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
> + xtr->magic = PROTO_XTR_MAGIC_ID;
> + }
> +#endif
> }
>
> #ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
> diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h index
> 64e891875..de16637f3 100644
> --- a/drivers/net/ice/ice_rxtx.h
> +++ b/drivers/net/ice/ice_rxtx.h
> @@ -5,6 +5,7 @@
> #ifndef _ICE_RXTX_H_
> #define _ICE_RXTX_H_
>
> +#include "rte_pmd_ice.h"
> #include "ice_ethdev.h"
>
> #define ICE_ALIGN_RING_DESC 32
> @@ -78,6 +79,7 @@ struct ice_rx_queue {
> uint16_t max_pkt_len; /* Maximum packet length */
> bool q_set; /* indicate if rx queue has been configured */
> bool rx_deferred_start; /* don't start this queue in dev start */
> + uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
> ice_rx_release_mbufs_t rx_rel_mbufs;
> };
>
> diff --git a/drivers/net/ice/ice_rxtx_vec_common.h
> b/drivers/net/ice/ice_rxtx_vec_common.h
> index c5f0d564f..080ca4175 100644
> --- a/drivers/net/ice/ice_rxtx_vec_common.h
> +++ b/drivers/net/ice/ice_rxtx_vec_common.h
> @@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
> if (rxq->nb_rx_desc % rxq->rx_free_thresh)
> return -1;
>
> + if (rxq->proto_xtr != PROTO_XTR_NONE)
> + return -1;
> +
> return 0;
> }
>
> diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build index
> 36b4b3c85..6828170a9 100644
> --- a/drivers/net/ice/meson.build
> +++ b/drivers/net/ice/meson.build
> @@ -34,3 +34,5 @@ if arch_subdir == 'x86'
> objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
> endif
> endif
> +
> +install_headers('rte_pmd_ice.h')
> diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
> new file mode 100644 index 000000000..719487e1e
> --- /dev/null
> +++ b/drivers/net/ice/rte_pmd_ice.h
> @@ -0,0 +1,152 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2019 Intel Corporation
> + */
> +
> +#ifndef _RTE_PMD_ICE_H_
> +#define _RTE_PMD_ICE_H_
> +
> +#include <stdio.h>
> +#include <rte_mbuf.h>
> +#include <rte_ethdev.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +enum proto_xtr_type {
> + PROTO_XTR_NONE,
> + PROTO_XTR_VLAN,
> + PROTO_XTR_IPV4,
> + PROTO_XTR_IPV6,
> + PROTO_XTR_IPV6_FLOW,
> + PROTO_XTR_TCP,
> +};
> +
> +struct proto_xtr_flds {
> + union {
> + struct {
> + uint16_t data0;
> + uint16_t data1;
> + } raw;
> + struct {
> + uint16_t stag_vid:12,
> + stag_dei:1,
> + stag_pcp:3;
> + uint16_t ctag_vid:12,
> + ctag_dei:1,
> + ctag_pcp:3;
> + } vlan;
> + struct {
> + uint16_t protocol:8,
> + ttl:8;
> + uint16_t tos:8,
> + ihl:4,
> + version:4;
> + } ipv4;
> + struct {
> + uint16_t hoplimit:8,
> + nexthdr:8;
> + uint16_t flowhi4:4,
> + tc:8,
> + version:4;
> + } ipv6;
> + struct {
> + uint16_t flowlo16;
> + uint16_t flowhi4:4,
> + tc:8,
> + version:4;
> + } ipv6_flow;
> + struct {
> + uint16_t fin:1,
> + syn:1,
> + rst:1,
> + psh:1,
> + ack:1,
> + urg:1,
> + ece:1,
> + cwr:1,
> + res1:4,
> + doff:4;
> + uint16_t rsvd;
> + } tcp;
> + } u;
> +
> + uint16_t rsvd;
> +
> + uint8_t type;
> +
> +#define PROTO_XTR_MAGIC_ID 0xCE
> + uint8_t magic;
> +};
> +
> +static inline void
> +init_proto_xtr_flds(struct rte_mbuf *mb) {
> + mb->udata64 = 0;
> +}
> +
> +static inline struct proto_xtr_flds *
> +get_proto_xtr_flds(struct rte_mbuf *mb) {
> + RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb-
> >udata64));
> +
> + return (struct proto_xtr_flds *)&mb->udata64; }
> +
> +static inline void
> +dump_proto_xtr_flds(struct rte_mbuf *mb) {
> + struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> +
> + if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type ==
> PROTO_XTR_NONE)
> + return;
> +
> + printf(" - Protocol Extraction:[0x%04x:0x%04x],",
> + xtr->u.raw.data0, xtr->u.raw.data1);
> +
> + if (xtr->type == PROTO_XTR_VLAN)
> + printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
> + xtr->u.vlan.stag_pcp,
> + xtr->u.vlan.stag_dei,
> + xtr->u.vlan.stag_vid,
> + xtr->u.vlan.ctag_pcp,
> + xtr->u.vlan.ctag_dei,
> + xtr->u.vlan.ctag_vid);
> + else if (xtr->type == PROTO_XTR_IPV4)
> + printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
> + xtr->u.ipv4.version,
> + xtr->u.ipv4.ihl,
> + xtr->u.ipv4.tos,
> + xtr->u.ipv4.ttl,
> + xtr->u.ipv4.protocol);
> + else if (xtr->type == PROTO_XTR_IPV6)
> +
> printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
> + xtr->u.ipv6.version,
> + xtr->u.ipv6.tc,
> + xtr->u.ipv6.flowhi4,
> + xtr->u.ipv6.nexthdr,
> + xtr->u.ipv6.hoplimit);
> + else if (xtr->type == PROTO_XTR_IPV6_FLOW)
> + printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
> + xtr->u.ipv6_flow.version,
> + xtr->u.ipv6_flow.tc,
> + xtr->u.ipv6_flow.flowhi4,
> + xtr->u.ipv6_flow.flowlo16);
> + else if (xtr->type == PROTO_XTR_TCP)
> + printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
> + xtr->u.tcp.doff,
> + xtr->u.tcp.cwr ? "C" : "",
> + xtr->u.tcp.ece ? "E" : "",
> + xtr->u.tcp.urg ? "U" : "",
> + xtr->u.tcp.ack ? "A" : "",
> + xtr->u.tcp.psh ? "P" : "",
> + xtr->u.tcp.rst ? "R" : "",
> + xtr->u.tcp.syn ? "S" : "",
> + xtr->u.tcp.fin ? "F" : "");
> +}
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* _RTE_PMD_ICE_H_ */
> --
> 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-23 3:25 ` Yang, Qiming
@ 2019-09-23 3:34 ` Wang, Haiyue
2019-09-23 8:29 ` Ye Xiaolong
1 sibling, 0 replies; 54+ messages in thread
From: Wang, Haiyue @ 2019-09-23 3:34 UTC (permalink / raw)
To: Yang, Qiming, Rong, Leyi, Lu, Wenzhuo, Zhang, Qi Z, Ye, Xiaolong; +Cc: dev
> -----Original Message-----
> From: Yang, Qiming
> Sent: Monday, September 23, 2019 11:26
> To: Rong, Leyi <leyi.rong@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong <xiaolong.ye@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
>
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Leyi Rong
> > Sent: Thursday, September 19, 2019 2:26 PM
> > To: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo
> > <wenzhuo.lu@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; Ye, Xiaolong
> > <xiaolong.ye@intel.com>
> > Cc: dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support
> > for per Rx queue
> >
> > From: Haiyue Wang <haiyue.wang@intel.com>
> >
> > The ice has the feature to extract protocol fields into flex descriptor by
> > programming per queue. Currently, the ice PMD will put the protocol fields
> > into rte_mbuf::udata64 with different type format. Application can access
> > the protocol fields quickly.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > ---
> > doc/guides/nics/ice.rst | 101 +++++++++
> > doc/guides/rel_notes/release_19_11.rst | 4 +
> > drivers/net/ice/Makefile | 3 +
> > drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
> > drivers/net/ice/ice_ethdev.h | 4 +
> > drivers/net/ice/ice_rxtx.c | 61 +++++
> > drivers/net/ice/ice_rxtx.h | 2 +
> > drivers/net/ice/ice_rxtx_vec_common.h | 3 +
> > drivers/net/ice/meson.build | 2 +
> > drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
> > 10 files changed, 633 insertions(+)
> > create mode 100644 drivers/net/ice/rte_pmd_ice.h
> >
> > diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst index
> > 03819d29f..8a6f60e71 100644
> > --- a/doc/guides/nics/ice.rst
> > +++ b/doc/guides/nics/ice.rst
> > @@ -61,6 +61,107 @@ Runtime Config Options
> > NOTE: In Safe mode, only very limited features are available, features like
> > RSS,
> > checksum, fdir, tunneling ... are all disabled.
> >
> > +- ``Protocol extraction for per queue``
> > +
> > + Configure the RX queues to do protocol extraction into
> > + ``rte_mbuf::udata64`` for protocol handling acceleration, like checking the
> > TCP SYN packets quickly.
> > +
> > + The argument format is::
> > +
> > + -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
> > + -w 18:00.0,proto_xtr=<protocol>
> > +
> > + Queues are grouped by ``(`` and ``)`` within the group. The ``-``
> > + character is used as a range separator and ``,`` is used as a single number
> > separator.
> > + The grouping ``()`` can be omitted for single element group. If no
> > + queues are specified, PMD will use this protocol extraction type for all
> > queues.
> > +
> > + Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
> > +
> > + .. code-block:: console
> > +
> > + testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
> > +
> > + This setting means queues 1, 2-3, 8-9 are TCP extraction, queues
> > + 10-13 are VLAN extraction, other queues run with no protocol extraction.
> > +
> > + .. code-block:: console
> > +
> > + testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
> > +
> > + This setting means queues 1, 2-3, 8-9 are TCP extraction, queues
> > + 10-23 are
> > + IPv6 extraction, other queues use the default VLAN extraction.
> > +
> > + The extraction will be copied into the lower 32 bit of
> > ``rte_mbuf::udata64``.
> > +
> > + .. table:: Protocol extraction : ``vlan``
> > +
> > + +----------------------------+----------------------------+
> > + | VLAN2 | VLAN1 |
> > + +======+===+=================+======+===+=================+
> > + | PCP | D | VID | PCP | D | VID |
> > + +------+---+-----------------+------+---+-----------------+
> > +
> > + VLAN1 - single or EVLAN (first for QinQ).
> > +
> > + VLAN2 - C-VLAN (second for QinQ).
> > +
> > + .. table:: Protocol extraction : ``ipv4``
> > +
> > + +----------------------------+----------------------------+
> > + | IPHDR2 | IPHDR1 |
> > + +======+=======+=============+==============+=============+
> > + | Ver |Hdr Len| ToS | TTL | Protocol |
> > + +------+-------+-------------+--------------+-------------+
> > +
> > + IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
> > +
> > + IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
> > +
> > + .. table:: Protocol extraction : ``ipv6``
> > +
> > + +----------------------------+----------------------------+
> > + | IPHDR2 | IPHDR1 |
> > + +=====+=============+========+=============+==============+
> > + | Ver |Traffic class| Flow | Next Header | Hop Limit |
> > + +-----+-------------+--------+-------------+--------------+
> > +
> > + IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
> > +
> > + IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits
> > + of "Flow Label" fields.
> > +
> > + .. table:: Protocol extraction : ``ipv6_flow``
> > +
> > + +----------------------------+----------------------------+
> > + | IPHDR2 | IPHDR1 |
> > + +=====+=============+========+============================+
> > + | Ver |Traffic class| Flow Label |
> > + +-----+-------------+-------------------------------------+
> > +
> > + IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
> > +
> > + IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits
> > + of "Flow Label" fields.
> > +
> > + .. table:: Protocol extraction : ``tcp``
> > +
> > + +----------------------------+----------------------------+
> > + | TCPHDR2 | TCPHDR1 |
> > + +============================+======+======+==============+
> > + | Reserved |Offset| RSV | Flags |
> > + +----------------------------+------+------+--------------+
> > +
> > + TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
> > +
> > + TCPHDR2 - Reserved
> > +
> > + Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the
> > + protocol extraction, do not use ``rte_mbuf::udata64`` directly.
> > +
> > + The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
> > + access the protocol extraction result in ``struct rte_mbuf``.
> > +
> > Driver compilation and testing
> > ------------------------------
> >
> > diff --git a/doc/guides/rel_notes/release_19_11.rst
> > b/doc/guides/rel_notes/release_19_11.rst
> > index 8490d897c..382806229 100644
> > --- a/doc/guides/rel_notes/release_19_11.rst
> > +++ b/doc/guides/rel_notes/release_19_11.rst
> > @@ -21,6 +21,10 @@ DPDK Release 19.11
> >
> > xdg-open build/doc/html/guides/rel_notes/release_19_11.html
> >
> > +* **Updated the ICE driver.**
> > +
> > + * Added support for handling Receive Flex Descriptor.
> > + * Added support for protocol extraction on per Rx queue.
> >
> > New Features
> > ------------
> > diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile index
> > ae53c2646..4a279f196 100644
> > --- a/drivers/net/ice/Makefile
> > +++ b/drivers/net/ice/Makefile
> > @@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1) endif
> > SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
> >
> > +# install this header file
> > +SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
> > +
> > include $(RTE_SDK)/mk/rte.lib.mk
> > diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index
> > 44a14cb8a..7c74b6169 100644
> > --- a/drivers/net/ice/ice_ethdev.c
> > +++ b/drivers/net/ice/ice_ethdev.c
> > @@ -19,9 +19,11 @@
> >
> > /* devargs */
> > #define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
> > +#define ICE_PROTO_XTR_ARG "proto_xtr"
> >
> > static const char * const ice_valid_args[] = {
> > ICE_SAFE_MODE_SUPPORT_ARG,
> > + ICE_PROTO_XTR_ARG,
> > NULL
> > };
> >
> > @@ -257,6 +259,280 @@ ice_init_controlq_parameter(struct ice_hw *hw)
> > hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ; }
> >
> > +static int
> > +lookup_proto_xtr_type(const char *xtr_name) {
> > + static struct {
> > + const char *name;
> > + enum proto_xtr_type type;
> > + } xtr_type_map[] = {
> > + { "vlan", PROTO_XTR_VLAN },
> > + { "ipv4", PROTO_XTR_IPV4 },
> > + { "ipv6", PROTO_XTR_IPV6 },
> > + { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
> > + { "tcp", PROTO_XTR_TCP },
> > + };
> > + uint32_t i;
> > +
> > + for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
> > + if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
> > + return xtr_type_map[i].type;
> > + }
> > +
> > + return -1;
> > +}
> > +
> > +/*
> > + * Parse elem, the elem could be single number/range or '(' ')' group
> > + * 1) A single number elem, it's just a simple digit. e.g. 9
> > + * 2) A single range elem, two digits with a '-' between. e.g. 2-6
> > + * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
> > + * Within group elem, '-' used for a range separator;
> > + * ',' used for a single number.
> > + */
> > +static int
> > +parse_queue_set(const char *input, int xtr_type, struct ice_devargs
> > +*devargs) {
> > + const char *str = input;
> > + char *end = NULL;
> > + uint32_t min, max;
> > + uint32_t idx;
> > +
> > + while (isblank(*str))
> > + str++;
> > +
> > + if (!isdigit(*str) && *str != '(')
> > + return -1;
> > +
> > + /* process single number or single range of number */
> > + if (*str != '(') {
> > + errno = 0;
> > + idx = strtoul(str, &end, 10);
> > + if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> > + return -1;
> > +
> > + while (isblank(*end))
> > + end++;
> > +
> > + min = idx;
> > + max = idx;
> > +
> > + /* process single <number>-<number> */
> > + if (*end == '-') {
> > + end++;
> > + while (isblank(*end))
> > + end++;
> > + if (!isdigit(*end))
> > + return -1;
> > +
> > + errno = 0;
> > + idx = strtoul(end, &end, 10);
> > + if (errno || end == NULL || idx >=
> > ICE_MAX_QUEUE_NUM)
> > + return -1;
> > +
> > + max = idx;
> > + while (isblank(*end))
> > + end++;
> > + }
> > +
> > + if (*end != ':')
> > + return -1;
> > +
> > + for (idx = RTE_MIN(min, max);
> > + idx <= RTE_MAX(min, max); idx++)
> > + devargs->proto_xtr[idx] = xtr_type;
> > +
> > + return 0;
> > + }
> > +
> > + /* process set within bracket */
> > + str++;
> > + while (isblank(*str))
> > + str++;
> > + if (*str == '\0')
> > + return -1;
> > +
> > + min = ICE_MAX_QUEUE_NUM;
> > + do {
> > + /* go ahead to the first digit */
> > + while (isblank(*str))
> > + str++;
> > + if (!isdigit(*str))
> > + return -1;
> > +
> > + /* get the digit value */
> > + errno = 0;
> > + idx = strtoul(str, &end, 10);
> > + if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
> > + return -1;
> > +
> > + /* go ahead to separator '-',',' and ')' */
> > + while (isblank(*end))
> > + end++;
> > + if (*end == '-') {
> > + if (min == ICE_MAX_QUEUE_NUM)
> > + min = idx;
> > + else /* avoid continuous '-' */
> > + return -1;
> > + } else if (*end == ',' || *end == ')') {
> > + max = idx;
> > + if (min == ICE_MAX_QUEUE_NUM)
> > + min = idx;
> > +
> > + for (idx = RTE_MIN(min, max);
> > + idx <= RTE_MAX(min, max); idx++)
> > + devargs->proto_xtr[idx] = xtr_type;
> > +
> > + min = ICE_MAX_QUEUE_NUM;
> > + } else {
> > + return -1;
> > + }
> > +
> > + str = end + 1;
> > + } while (*end != ')' && *end != '\0');
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > +parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
> > +{
> > + const char *queue_start;
> > + uint32_t idx;
> > + int xtr_type;
> > + char xtr_name[32];
> > +
> > + while (isblank(*queues))
> > + queues++;
> > +
> > + if (*queues != '[') {
> > + xtr_type = lookup_proto_xtr_type(queues);
> > + if (xtr_type < 0)
> > + return -1;
> > +
> > + memset(devargs->proto_xtr, xtr_type,
> > + sizeof(devargs->proto_xtr));
> > +
> > + return 0;
> > + }
> > +
> > + queues++;
> > + do {
> > + while (isblank(*queues))
> > + queues++;
> > + if (*queues == '\0')
> > + return -1;
> > +
> > + queue_start = queues;
> > +
> > + /* go across a complete bracket */
> > + if (*queue_start == '(') {
> > + queues += strcspn(queues, ")");
> > + if (*queues != ')')
> > + return -1;
> > + }
> > +
> > + /* scan the separator ':' */
> > + queues += strcspn(queues, ":");
> > + if (*queues++ != ':')
> > + return -1;
> > + while (isblank(*queues))
> > + queues++;
> > +
> > + for (idx = 0; ; idx++) {
> > + if (isblank(queues[idx]) ||
> > + queues[idx] == ',' ||
> > + queues[idx] == ']' ||
> > + queues[idx] == '\0')
> > + break;
> > +
> > + if (idx > sizeof(xtr_name) - 2)
> > + return -1;
> > +
> > + xtr_name[idx] = queues[idx];
> > + }
> > + xtr_name[idx] = '\0';
> > + xtr_type = lookup_proto_xtr_type(xtr_name);
> > + if (xtr_type < 0)
> > + return -1;
> > +
> > + queues += idx;
> > +
> > + while (isblank(*queues) || *queues == ',' || *queues == ']')
> > + queues++;
> > +
> > + if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
> > + return -1;
> > + } while (*queues != '\0');
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > +handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
> > + void *extra_args)
> > +{
> > + struct ice_devargs *devargs = extra_args;
> > +
> > + if (value == NULL || extra_args == NULL)
> > + return -EINVAL;
> > +
> > + if (parse_queue_proto_xtr(value, devargs) < 0) {
> > + PMD_DRV_LOG(ERR,
> > + "The protocol extraction parameter is wrong : '%s'",
> > + value);
> > + return -1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static bool
> > +ice_proto_xtr_support(struct ice_hw *hw) { #define FLX_REG(val, fld,
> > +idx) \
> > + (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
> > + GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
> > + static struct {
> > + uint32_t rxdid;
> > + uint16_t protid_0;
> > + uint16_t protid_1;
> > + } xtr_sets[] = {
> > + { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O,
> > ICE_PROT_VLAN_O },
> > + { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
> > + ICE_PROT_IPV4_OF_OR_S },
> > + { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
> > + ICE_PROT_IPV6_OF_OR_S },
> > + { ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> > ICE_PROT_IPV6_OF_OR_S,
> > + ICE_PROT_IPV6_OF_OR_S },
> > + { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL,
> > ICE_PROT_ID_INVAL },
> > + };
> > + uint32_t i;
> > +
> > + for (i = 0; i < RTE_DIM(xtr_sets); i++) {
> > + uint32_t rxdid = xtr_sets[i].rxdid;
> > + uint32_t v;
> > +
> > + if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
> > + v = ICE_READ_REG(hw,
> > GLFLXP_RXDID_FLX_WRD_4(rxdid));
> > +
> > + if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0
> > ||
> > + FLX_REG(v, RXDID_OPCODE, 4) !=
> > ICE_RX_OPC_EXTRACT)
> > + return false;
> > + }
> > +
> > + if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
> > + v = ICE_READ_REG(hw,
> > GLFLXP_RXDID_FLX_WRD_5(rxdid));
> > +
> > + if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1
> > ||
> > + FLX_REG(v, RXDID_OPCODE, 5) !=
> > ICE_RX_OPC_EXTRACT)
> > + return false;
> > + }
> > + }
> > +
> > + return true;
> > +}
> > +
> > static int
> > ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
> > uint32_t num)
> > @@ -1079,6 +1355,8 @@ ice_interrupt_handler(void *param) static int
> > ice_pf_sw_init(struct rte_eth_dev *dev) {
> > + struct ice_adapter *ad =
> > + ICE_DEV_PRIVATE_TO_ADAPTER(dev->data-
> > >dev_private);
> > struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
> > struct ice_hw *hw = ICE_PF_TO_HW(pf);
> >
> > @@ -1088,6 +1366,16 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
> >
> > pf->lan_nb_qps = pf->lan_nb_qp_max;
> >
> > + if (ice_proto_xtr_support(hw))
> > + pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
> > +
> > + if (pf->proto_xtr != NULL)
> > + rte_memcpy(pf->proto_xtr, ad->devargs.proto_xtr,
> > + RTE_MIN((size_t)pf->lan_nb_qps,
> > + sizeof(ad->devargs.proto_xtr)));
> > + else
> > + PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
> > +
> > return 0;
> > }
> >
> > @@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev
> > *dev)
> > return -EINVAL;
> > }
> >
> > + memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
> > + sizeof(ad->devargs.proto_xtr));
> > +
> > + ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
> > + &handle_proto_xtr_arg, &ad->devargs);
> > + if (ret)
> > + goto bail;
> > +
>
> Why is bail?
>
free & ret, since 'ret' is used, 'bail' name from kernel. :)
> > ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
> > &parse_bool, &ad-
> > >devargs.safe_mode_support);
> >
> > +bail:
> > rte_kvargs_free(kvlist);
> > return ret;
> > }
> > @@ -1547,6 +1844,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> > ice_sched_cleanup_all(hw);
> > rte_free(hw->port_info);
> > ice_shutdown_all_ctrlq(hw);
> > + rte_free(pf->proto_xtr);
> >
> > return ret;
> > }
> > @@ -1672,6 +1970,8 @@ ice_dev_close(struct rte_eth_dev *dev)
> > rte_free(hw->port_info);
> > hw->port_info = NULL;
> > ice_shutdown_all_ctrlq(hw);
> > + rte_free(pf->proto_xtr);
> > + pf->proto_xtr = NULL;
> > }
> >
> > static int
> > @@ -3795,6 +4095,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
> > RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
> > RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-
> > pci"); RTE_PMD_REGISTER_PARAM_STRING(net_ice,
> > + ICE_PROTO_XTR_ARG
> > "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
> > ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
> >
> > RTE_INIT(ice_init_log)
> > diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h index
> > f569da833..adbb66322 100644
> > --- a/drivers/net/ice/ice_ethdev.h
> > +++ b/drivers/net/ice/ice_ethdev.h
> > @@ -263,6 +263,7 @@ struct ice_pf {
> > uint16_t lan_nb_qp_max;
> > uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
> > uint16_t base_queue; /* The base queue pairs index in the device
> > */
> > + uint8_t *proto_xtr; /* Protocol extraction type for all queues */
> > struct ice_hw_port_stats stats_offset;
> > struct ice_hw_port_stats stats;
> > /* internal packet statistics, it should be excluded from the total */
> > @@ -273,11 +274,14 @@ struct ice_pf {
> > struct ice_flow_list flow_list;
> > };
> >
> > +#define ICE_MAX_QUEUE_NUM 2048
> > +
> > /**
> > * Cache devargs parse result.
> > */
> > struct ice_devargs {
> > int safe_mode_support;
> > + uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
> > };
> >
> > /**
> > diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c index
> > d2e36853f..e28310b96 100644
> > --- a/drivers/net/ice/ice_rxtx.c
> > +++ b/drivers/net/ice/ice_rxtx.c
> > @@ -13,6 +13,36 @@
> > PKT_TX_TCP_SEG | \
> > PKT_TX_OUTER_IP_CKSUM)
> >
> > +static inline uint8_t
> > +ice_rxdid_to_proto_xtr_type(uint8_t rxdid) {
> > + static uint8_t xtr_map[] = {
> > + [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
> > + [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
> > + [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
> > + [ICE_RXDID_COMMS_AUX_IPV6_FLOW] =
> > PROTO_XTR_IPV6_FLOW,
> > + [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
> > + };
> > +
> > + return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] :
> > PROTO_XTR_NONE; }
> > +
> > +static inline uint8_t
> > +ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye) {
> > + static uint8_t rxdid_map[] = {
> > + [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
> > + [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
> > + [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
> > + [PROTO_XTR_IPV6_FLOW] =
> > ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> > + [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
> > + };
> > + uint8_t rxdid;
> > +
> > + rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
> > +
> > + return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC; }
> >
> > static enum ice_status
> > ice_program_hw_rx_queue(struct ice_rx_queue *rxq) @@ -84,6 +114,11
> > @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
> > rx_ctx.showiv = 0;
> > rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
> >
> > + rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
> > +
> > + PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with
> > RXDID : %u",
> > + rxq->port_id, rxq->queue_id, rxdid);
> > +
> > /* Enable Flexible Descriptors in the queue context which
> > * allows this driver to select a specific receive descriptor format
> > */
> > @@ -641,6 +676,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
> > rxq->drop_en = rx_conf->rx_drop_en;
> > rxq->vsi = vsi;
> > rxq->rx_deferred_start = rx_conf->rx_deferred_start;
> > + rxq->proto_xtr = pf->proto_xtr != NULL ?
> > + pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
> >
> > /* Allocate the maximun number of RX ring hardware descriptor. */
> > len = ICE_MAX_RING_DESC;
> > @@ -1062,6 +1099,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile
> > union ice_rx_flex_desc *rxdp)
> > mb->vlan_tci, mb->vlan_tci_outer); }
> >
> > +#define ICE_RX_PROTO_XTR_VALID \
> > + ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
> > + (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> > +
> > static inline void
> > ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> > volatile union ice_rx_flex_desc *rxdp) @@ -1075,6
> > +1116,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
> > mb->ol_flags |= PKT_RX_RSS_HASH;
> > mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
> > }
> > +
> > +#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
> > + init_proto_xtr_flds(mb);
> > +
> > + stat_err = rte_le_to_cpu_16(desc->status_error1);
> > + if (stat_err & ICE_RX_PROTO_XTR_VALID) {
> > + struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> > +
> > + if (stat_err & (1 <<
> > ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
> > + xtr->u.raw.data0 =
> > + rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
> > +
> > + if (stat_err & (1 <<
> > ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
> > + xtr->u.raw.data1 =
> > + rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
> > +
> > + xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
> > + xtr->magic = PROTO_XTR_MAGIC_ID;
> > + }
> > +#endif
> > }
> >
> > #ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
> > diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h index
> > 64e891875..de16637f3 100644
> > --- a/drivers/net/ice/ice_rxtx.h
> > +++ b/drivers/net/ice/ice_rxtx.h
> > @@ -5,6 +5,7 @@
> > #ifndef _ICE_RXTX_H_
> > #define _ICE_RXTX_H_
> >
> > +#include "rte_pmd_ice.h"
> > #include "ice_ethdev.h"
> >
> > #define ICE_ALIGN_RING_DESC 32
> > @@ -78,6 +79,7 @@ struct ice_rx_queue {
> > uint16_t max_pkt_len; /* Maximum packet length */
> > bool q_set; /* indicate if rx queue has been configured */
> > bool rx_deferred_start; /* don't start this queue in dev start */
> > + uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
> > ice_rx_release_mbufs_t rx_rel_mbufs;
> > };
> >
> > diff --git a/drivers/net/ice/ice_rxtx_vec_common.h
> > b/drivers/net/ice/ice_rxtx_vec_common.h
> > index c5f0d564f..080ca4175 100644
> > --- a/drivers/net/ice/ice_rxtx_vec_common.h
> > +++ b/drivers/net/ice/ice_rxtx_vec_common.h
> > @@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
> > if (rxq->nb_rx_desc % rxq->rx_free_thresh)
> > return -1;
> >
> > + if (rxq->proto_xtr != PROTO_XTR_NONE)
> > + return -1;
> > +
> > return 0;
> > }
> >
> > diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build index
> > 36b4b3c85..6828170a9 100644
> > --- a/drivers/net/ice/meson.build
> > +++ b/drivers/net/ice/meson.build
> > @@ -34,3 +34,5 @@ if arch_subdir == 'x86'
> > objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
> > endif
> > endif
> > +
> > +install_headers('rte_pmd_ice.h')
> > diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
> > new file mode 100644 index 000000000..719487e1e
> > --- /dev/null
> > +++ b/drivers/net/ice/rte_pmd_ice.h
> > @@ -0,0 +1,152 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2019 Intel Corporation
> > + */
> > +
> > +#ifndef _RTE_PMD_ICE_H_
> > +#define _RTE_PMD_ICE_H_
> > +
> > +#include <stdio.h>
> > +#include <rte_mbuf.h>
> > +#include <rte_ethdev.h>
> > +
> > +#ifdef __cplusplus
> > +extern "C" {
> > +#endif
> > +
> > +enum proto_xtr_type {
> > + PROTO_XTR_NONE,
> > + PROTO_XTR_VLAN,
> > + PROTO_XTR_IPV4,
> > + PROTO_XTR_IPV6,
> > + PROTO_XTR_IPV6_FLOW,
> > + PROTO_XTR_TCP,
> > +};
> > +
> > +struct proto_xtr_flds {
> > + union {
> > + struct {
> > + uint16_t data0;
> > + uint16_t data1;
> > + } raw;
> > + struct {
> > + uint16_t stag_vid:12,
> > + stag_dei:1,
> > + stag_pcp:3;
> > + uint16_t ctag_vid:12,
> > + ctag_dei:1,
> > + ctag_pcp:3;
> > + } vlan;
> > + struct {
> > + uint16_t protocol:8,
> > + ttl:8;
> > + uint16_t tos:8,
> > + ihl:4,
> > + version:4;
> > + } ipv4;
> > + struct {
> > + uint16_t hoplimit:8,
> > + nexthdr:8;
> > + uint16_t flowhi4:4,
> > + tc:8,
> > + version:4;
> > + } ipv6;
> > + struct {
> > + uint16_t flowlo16;
> > + uint16_t flowhi4:4,
> > + tc:8,
> > + version:4;
> > + } ipv6_flow;
> > + struct {
> > + uint16_t fin:1,
> > + syn:1,
> > + rst:1,
> > + psh:1,
> > + ack:1,
> > + urg:1,
> > + ece:1,
> > + cwr:1,
> > + res1:4,
> > + doff:4;
> > + uint16_t rsvd;
> > + } tcp;
> > + } u;
> > +
> > + uint16_t rsvd;
> > +
> > + uint8_t type;
> > +
> > +#define PROTO_XTR_MAGIC_ID 0xCE
> > + uint8_t magic;
> > +};
> > +
> > +static inline void
> > +init_proto_xtr_flds(struct rte_mbuf *mb) {
> > + mb->udata64 = 0;
> > +}
> > +
> > +static inline struct proto_xtr_flds *
> > +get_proto_xtr_flds(struct rte_mbuf *mb) {
> > + RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb-
> > >udata64));
> > +
> > + return (struct proto_xtr_flds *)&mb->udata64; }
> > +
> > +static inline void
> > +dump_proto_xtr_flds(struct rte_mbuf *mb) {
> > + struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
> > +
> > + if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type ==
> > PROTO_XTR_NONE)
> > + return;
> > +
> > + printf(" - Protocol Extraction:[0x%04x:0x%04x],",
> > + xtr->u.raw.data0, xtr->u.raw.data1);
> > +
> > + if (xtr->type == PROTO_XTR_VLAN)
> > + printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
> > + xtr->u.vlan.stag_pcp,
> > + xtr->u.vlan.stag_dei,
> > + xtr->u.vlan.stag_vid,
> > + xtr->u.vlan.ctag_pcp,
> > + xtr->u.vlan.ctag_dei,
> > + xtr->u.vlan.ctag_vid);
> > + else if (xtr->type == PROTO_XTR_IPV4)
> > + printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
> > + xtr->u.ipv4.version,
> > + xtr->u.ipv4.ihl,
> > + xtr->u.ipv4.tos,
> > + xtr->u.ipv4.ttl,
> > + xtr->u.ipv4.protocol);
> > + else if (xtr->type == PROTO_XTR_IPV6)
> > +
> > printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
> > + xtr->u.ipv6.version,
> > + xtr->u.ipv6.tc,
> > + xtr->u.ipv6.flowhi4,
> > + xtr->u.ipv6.nexthdr,
> > + xtr->u.ipv6.hoplimit);
> > + else if (xtr->type == PROTO_XTR_IPV6_FLOW)
> > + printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
> > + xtr->u.ipv6_flow.version,
> > + xtr->u.ipv6_flow.tc,
> > + xtr->u.ipv6_flow.flowhi4,
> > + xtr->u.ipv6_flow.flowlo16);
> > + else if (xtr->type == PROTO_XTR_TCP)
> > + printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
> > + xtr->u.tcp.doff,
> > + xtr->u.tcp.cwr ? "C" : "",
> > + xtr->u.tcp.ece ? "E" : "",
> > + xtr->u.tcp.urg ? "U" : "",
> > + xtr->u.tcp.ack ? "A" : "",
> > + xtr->u.tcp.psh ? "P" : "",
> > + xtr->u.tcp.rst ? "R" : "",
> > + xtr->u.tcp.syn ? "S" : "",
> > + xtr->u.tcp.fin ? "F" : "");
> > +}
> > +
> > +#ifdef __cplusplus
> > +}
> > +#endif
> > +
> > +#endif /* _RTE_PMD_ICE_H_ */
> > --
> > 2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-23 3:25 ` Yang, Qiming
2019-09-23 3:34 ` Wang, Haiyue
@ 2019-09-23 8:29 ` Ye Xiaolong
2019-09-23 11:03 ` Wang, Haiyue
1 sibling, 1 reply; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-23 8:29 UTC (permalink / raw)
To: Yang, Qiming; +Cc: Rong, Leyi, Wang, Haiyue, Lu, Wenzhuo, Zhang, Qi Z, dev
On 09/23, Yang, Qiming wrote:
>
>
[snip]
>> @@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev
>> *dev)
>> return -EINVAL;
>> }
>>
>> + memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
>> + sizeof(ad->devargs.proto_xtr));
>> +
>> + ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
>> + &handle_proto_xtr_arg, &ad->devargs);
>> + if (ret)
>> + goto bail;
>> +
>
>Why is bail?
>
Minor nit, better to snip unrelated context for one single comment, otherwise
it's quite difficult to find it. :)
Thanks,
Xiaolong
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-23 8:29 ` Ye Xiaolong
@ 2019-09-23 11:03 ` Wang, Haiyue
0 siblings, 0 replies; 54+ messages in thread
From: Wang, Haiyue @ 2019-09-23 11:03 UTC (permalink / raw)
To: Ye, Xiaolong, Yang, Qiming; +Cc: Rong, Leyi, Lu, Wenzhuo, Zhang, Qi Z, dev
> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Monday, September 23, 2019 16:30
> To: Yang, Qiming <qiming.yang@intel.com>
> Cc: Rong, Leyi <leyi.rong@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo
> <wenzhuo.lu@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
>
> On 09/23, Yang, Qiming wrote:
> >
> >
>
> [snip]
>
> >> @@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev
> >> *dev)
> >> return -EINVAL;
> >> }
> >>
> >> + memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
> >> + sizeof(ad->devargs.proto_xtr));
> >> +
> >> + ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
> >> + &handle_proto_xtr_arg, &ad->devargs);
> >> + if (ret)
> >> + goto bail;
> >> +
> >
> >Why is bail?
> >
>
> Minor nit, better to snip unrelated context for one single comment, otherwise
> it's quite difficult to find it. :)
>
+1, I checked many lines very carefully to avoid miss something. :)
> Thanks,
> Xiaolong
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-09-23 11:05 ` Ye Xiaolong
0 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-23 11:05 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/19, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
>Set the RXDID with flex descriptor type by default, change the Rx
>function to support new descriptor handling.
>
>Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
>---
> drivers/net/ice/ice_rxtx.c | 236 +++++++++++++++++--------------------
> 1 file changed, 111 insertions(+), 125 deletions(-)
>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-09-23 3:25 ` Yang, Qiming
@ 2019-09-23 14:24 ` Ye Xiaolong
2019-09-23 15:00 ` Wang, Haiyue
1 sibling, 1 reply; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-23 14:24 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/19, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
[snip]
>+static inline uint8_t
>+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
>+{
>+ static uint8_t xtr_map[] = {
>+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
>+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
>+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
>+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
>+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
>+ };
>+
>+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
>+}
>+
>+static inline uint8_t
>+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
s/xtr_tpye/xtr_type/
>+{
>+ static uint8_t rxdid_map[] = {
>+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
>+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
>+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
>+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
>+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
>+ };
>+ uint8_t rxdid;
>+
>+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
>+
>+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
Maybe just
return xtr_type < RTE_DIM(rxdid_map) ?
rxdid_map[xtr_type] : ICE_RXDID_COMMS_GENERIC;
as previous ice_rxdid_to_proto_xtr_type.
Thanks,
Xiaolong
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
@ 2019-09-23 14:31 ` Ye Xiaolong
0 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-23 14:31 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/19, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
>Since now the ice PMD only handles Rx Flex descriptor, so remove the
>legacy descriptor definition.
>
>Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
>---
> drivers/net/ice/ice_rxtx.c | 25 ++++++++++++-------------
> drivers/net/ice/ice_rxtx.h | 4 +---
> drivers/net/ice/ice_rxtx_vec_avx2.c | 5 ++---
> drivers/net/ice/ice_rxtx_vec_sse.c | 4 ++--
> 4 files changed, 17 insertions(+), 21 deletions(-)
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-23 14:24 ` Ye Xiaolong
@ 2019-09-23 15:00 ` Wang, Haiyue
2019-09-23 15:55 ` Ye Xiaolong
0 siblings, 1 reply; 54+ messages in thread
From: Wang, Haiyue @ 2019-09-23 15:00 UTC (permalink / raw)
To: Ye, Xiaolong, Rong, Leyi; +Cc: Lu, Wenzhuo, Zhang, Qi Z, dev
> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Monday, September 23, 2019 22:24
> To: Rong, Leyi <leyi.rong@intel.com>
> Cc: Wang, Haiyue <haiyue.wang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; dev@dpdk.org
> Subject: Re: [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
>
> On 09/19, Leyi Rong wrote:
> >From: Haiyue Wang <haiyue.wang@intel.com>
> >
>
> [snip]
>
> >+static inline uint8_t
> >+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
> >+{
> >+ static uint8_t xtr_map[] = {
> >+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
> >+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
> >+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
> >+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
> >+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
> >+ };
> >+
> >+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
> >+}
> >+
> >+static inline uint8_t
> >+ice_proto_xtr_type_to_rxdid(uint8_t xtr_tpye)
>
> s/xtr_tpye/xtr_type/
>
Got it!
> >+{
> >+ static uint8_t rxdid_map[] = {
Add this line:
[PROTO_XTR_NONE] = ICE_RXDID_COMMS_GENERIC,
> >+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
> >+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
> >+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
> >+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
> >+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
> >+ };
> >+ uint8_t rxdid;
> >+
> >+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
> >+
> >+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
>
> Maybe just
>
> return xtr_type < RTE_DIM(rxdid_map) ?
> rxdid_map[xtr_type] : ICE_RXDID_COMMS_GENERIC;
>
Then this, will be updated in next version.
> as previous ice_rxdid_to_proto_xtr_type.
>
>
> Thanks,
> Xiaolong
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-23 15:00 ` Wang, Haiyue
@ 2019-09-23 15:55 ` Ye Xiaolong
0 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-23 15:55 UTC (permalink / raw)
To: Wang, Haiyue; +Cc: Rong, Leyi, Lu, Wenzhuo, Zhang, Qi Z, dev
On 09/23, Wang, Haiyue wrote:
>> >+{
>> >+ static uint8_t rxdid_map[] = {
>Add this line:
> [PROTO_XTR_NONE] = ICE_RXDID_COMMS_GENERIC,
This makes sense.
>> >+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
>> >+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
>> >+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
>> >+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
>> >+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
>> >+ };
>> >+ uint8_t rxdid;
>> >+
>> >+ rxdid = xtr_tpye < RTE_DIM(rxdid_map) ? rxdid_map[xtr_tpye] : 0;
>> >+
>> >+ return rxdid != 0 ? rxdid : ICE_RXDID_COMMS_GENERIC;
>>
>> Maybe just
>>
>> return xtr_type < RTE_DIM(rxdid_map) ?
>> rxdid_map[xtr_type] : ICE_RXDID_COMMS_GENERIC;
>>
>
>Then this, will be updated in next version.
>
>> as previous ice_rxdid_to_proto_xtr_type.
>>
>>
>> Thanks,
>> Xiaolong
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 0/6] enable Rx flexible descriptor
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
` (7 preceding siblings ...)
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
` (6 more replies)
8 siblings, 7 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
This patchset enable Rx flexible descriptor for ice PMD
in both normal path and vector path.
Depends on shared code update patchset.
---
v5:
- Fix some typo, and enhance the array index access handling.
- Move update for release note to `New Features` section.
v4:
- Handle the ‘proto_xtr’ with error processing, and remove the
Redundant default type variable.
- Add flex descriptor and protocol extraction release note.
- Clean up the legacy descriptor.
v3:
- Parse the ‘proto_xtr’ devargs before CVL HW initialization and
save it for later accessing.
- Merge shuffle ops on vlan and rss flag on vector path.
v2:
- Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
when handling Rx packets.
Haiyue Wang (4):
net/ice: add Rx flex descriptor definition
net/ice: handle the Rx flex descriptor
net/ice: add protocol extraction support for per Rx queue
net/ice: remove Rx legacy descriptor definition
Leyi Rong (1):
net/ice: switch to Rx flexible descriptor in AVX path
Wenzhuo Lu (1):
net/ice: switch to flexible descriptor in SSE path
doc/guides/nics/ice.rst | 101 +++++++++
doc/guides/rel_notes/release_19_11.rst | 4 +
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 297 +++++++++++++-----------
drivers/net/ice/ice_rxtx.h | 8 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 221 +++++++++---------
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/ice_rxtx_vec_sse.c | 235 +++++++++----------
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
12 files changed, 961 insertions(+), 370 deletions(-)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
--
2.17.1
^ permalink raw reply [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 1/6] net/ice: add Rx flex descriptor definition
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
` (5 subsequent siblings)
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The Rx flex descriptor has 16B and 32B size, with different
field definitions compared to legacy type.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index e9214110c..64e891875 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,8 +21,10 @@
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
#define ice_rx_desc ice_16byte_rx_desc
+#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
#define ice_rx_desc ice_32byte_rx_desc
+#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
#define ICE_SUPPORT_CHAIN_NUM 5
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 2/6] net/ice: handle the Rx flex descriptor
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
` (4 subsequent siblings)
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Set the RXDID with flex descriptor type by default, change the Rx
function to support new descriptor handling.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 236 +++++++++++++++++--------------------
1 file changed, 111 insertions(+), 125 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 0282b5375..d2e36853f 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,7 +13,6 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
-#define ICE_RX_ERR_BITS 0x3f
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -25,18 +24,9 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
enum ice_status err;
uint16_t buf_size, len;
struct rte_eth_rxmode *rxmode = &dev->data->dev_conf.rxmode;
+ uint32_t rxdid = ICE_RXDID_COMMS_GENERIC;
uint32_t regval;
- /**
- * The kernel driver uses flex descriptor. It sets the register
- * to flex descriptor mode.
- * DPDK uses legacy descriptor. It should set the register back
- * to the default value, then uses legacy descriptor mode.
- */
- regval = (0x01 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
- QRXFLXP_CNTXT_RXDID_PRIO_M;
- ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
-
/* Set buffer size as the head split is disabled. */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
RTE_PKTMBUF_HEADROOM);
@@ -94,6 +84,21 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile ID;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ ICE_WRITE_REG(hw, QRXFLXP_CNTXT(rxq->reg_idx), regval);
+
err = ice_clear_rxq_ctx(hw, rxq->reg_idx);
if (err) {
PMD_DRV_LOG(ERR, "Failed to clear Lan Rx queue (%u) context",
@@ -961,16 +966,15 @@ uint32_t
ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
#define ICE_RXQ_SCAN_INTERVAL 4
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq;
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
- ((rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S) &
- (1 << ICE_RX_DESC_STATUS_DD_S)) {
+ rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
/**
* Check the DD bit of a rx descriptor of each 4 in a group,
* to avoid checking too frequently and downgrading performance
@@ -979,79 +983,77 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = (volatile union ice_rx_flex_desc *)
+ &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
return desc;
}
-/* Translate the rx descriptor status to pkt flags */
-static inline uint64_t
-ice_rxd_status_to_pkt_flags(uint64_t qword)
-{
- uint64_t flags;
-
- /* Check if RSS_HASH */
- flags = (((qword >> ICE_RX_DESC_STATUS_FLTSTAT_S) &
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ==
- ICE_RX_DESC_FLTSTAT_RSS_HASH) ? PKT_RX_RSS_HASH : 0;
-
- return flags;
-}
+#define ICE_RX_FLEX_ERR0_BITS \
+ ((1 << ICE_RX_FLEX_DESC_STATUS0_HBO_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS0_RXE_S))
/* Rx L3/L4 checksum */
static inline uint64_t
-ice_rxd_error_to_pkt_flags(uint64_t qword)
+ice_rxd_error_to_pkt_flags(uint16_t stat_err0)
{
uint64_t flags = 0;
- uint64_t error_bits = (qword >> ICE_RXD_QW1_ERROR_S);
- if (likely((error_bits & ICE_RX_ERR_BITS) == 0)) {
+ /* check if HW has decoded the packet and checksum */
+ if (unlikely(!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return 0;
+
+ if (likely(!(stat_err0 & ICE_RX_FLEX_ERR0_BITS))) {
flags |= (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD);
return flags;
}
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_IPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
flags |= PKT_RX_IP_CKSUM_BAD;
else
flags |= PKT_RX_IP_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_L4E_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
flags |= PKT_RX_L4_CKSUM_BAD;
else
flags |= PKT_RX_L4_CKSUM_GOOD;
- if (unlikely(error_bits & (1 << ICE_RX_DESC_ERROR_EIPE_S)))
+ if (unlikely(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
flags |= PKT_RX_EIP_CKSUM_BAD;
return flags;
}
static inline void
-ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
+ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
{
- if (rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len) &
- (1 << ICE_RX_DESC_STATUS_L2TAG1P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S)) {
mb->ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED;
mb->vlan_tci =
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1);
+ rte_le_to_cpu_16(rxdp->wb.l2tag1);
PMD_RX_LOG(DEBUG, "Descriptor l2tag1: %u",
- rte_le_to_cpu_16(rxdp->wb.qword0.lo_dword.l2tag1));
+ rte_le_to_cpu_16(rxdp->wb.l2tag1));
} else {
mb->vlan_tci = 0;
}
#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
- if (rte_le_to_cpu_16(rxdp->wb.qword2.ext_status) &
- (1 << ICE_RX_DESC_EXT_STATUS_L2TAG2P_S)) {
+ if (rte_le_to_cpu_16(rxdp->wb.status_error1) &
+ (1 << ICE_RX_FLEX_DESC_STATUS1_L2TAG2P_S)) {
mb->ol_flags |= PKT_RX_QINQ_STRIPPED | PKT_RX_QINQ |
PKT_RX_VLAN_STRIPPED | PKT_RX_VLAN;
mb->vlan_tci_outer = mb->vlan_tci;
- mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2);
+ mb->vlan_tci = rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd);
PMD_RX_LOG(DEBUG, "Descriptor l2tag2_1: %u, l2tag2_2: %u",
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_1),
- rte_le_to_cpu_16(rxdp->wb.qword2.l2tag2_2));
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_1st),
+ rte_le_to_cpu_16(rxdp->wb.l2tag2_2nd));
} else {
mb->vlan_tci_outer = 0;
}
@@ -1060,6 +1062,21 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+static inline void
+ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
+ volatile union ice_rx_flex_desc *rxdp)
+{
+ volatile struct ice_32b_rx_flex_desc_comms *desc =
+ (volatile struct ice_32b_rx_flex_desc_comms *)rxdp;
+ uint16_t stat_err;
+
+ stat_err = rte_le_to_cpu_16(desc->status_error0);
+ if (likely(stat_err & (1 << ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S))) {
+ mb->ol_flags |= PKT_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
+ }
+}
+
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
#define ICE_LOOK_AHEAD 8
#if (ICE_LOOK_AHEAD != 8)
@@ -1068,25 +1085,23 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_desc *rxdp)
static inline int
ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
+ uint16_t stat_err0;
uint16_t pkt_len;
- uint64_t qword1;
- uint32_t rx_status;
int32_t s[ICE_LOOK_AHEAD], nb_dd;
int32_t i, j, nb_rx = 0;
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = &rxq->rx_ring[rxq->rx_tail];
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >> ICE_RXD_QW1_STATUS_S;
+ stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Make sure there is at least 1 packet to receive */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/**
@@ -1096,42 +1111,31 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
for (i = 0; i < ICE_RX_MAX_BURST; i += ICE_LOOK_AHEAD,
rxdp += ICE_LOOK_AHEAD, rxep += ICE_LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--) {
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- s[j] = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
- }
+ for (j = ICE_LOOK_AHEAD - 1; j >= 0; j--)
+ s[j] = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
rte_smp_rmb();
/* Compute how many status bits were set */
for (j = 0, nb_dd = 0; j < ICE_LOOK_AHEAD; j++)
- nb_dd += s[j] & (1 << ICE_RX_DESC_STATUS_DD_S);
+ nb_dd += s[j] & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S);
nb_rx += nb_dd;
/* Translate descriptor info to mbuf parameters */
for (j = 0; j < nb_dd; j++) {
mb = rxep[j].mbuf;
- qword1 = rte_le_to_cpu_64(
- rxdp[j].wb.qword1.status_error_len);
- pkt_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ pkt_len = (rte_le_to_cpu_16(rxdp[j].wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
mb->data_len = pkt_len;
mb->pkt_len = pkt_len;
mb->ol_flags = 0;
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- mb->hash.rss =
- rte_le_to_cpu_32(
- rxdp[j].wb.qword0.hi_dword.rss);
- mb->packet_type = ptype_tbl[(uint8_t)(
- (qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
+ stat_err0 = rte_le_to_cpu_16(rxdp[j].wb.status_error0);
+ pkt_flags = ice_rxd_error_to_pkt_flags(stat_err0);
+ mb->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxdp[j].wb.ptype_flex_flags0)];
ice_rxd_to_vlan_tci(mb, &rxdp[j]);
+ ice_rxd_to_pkt_fields(mb, &rxdp[j]);
mb->ol_flags |= pkt_flags;
}
@@ -1312,8 +1316,8 @@ ice_recv_scattered_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *first_seg = rxq->pkt_first_seg;
@@ -1324,21 +1328,18 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1377,14 +1378,10 @@ ice_recv_scattered_pkts(void *rx_queue,
/* Set data buffer address and data length of the mbuf */
rxdp->read.hdr_addr = 0;
rxdp->read.pkt_addr = dma_addr;
- rx_packet_len = (qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S;
+ rx_packet_len = rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
rxm->data_len = rx_packet_len;
rxm->data_off = RTE_PKTMBUF_HEADROOM;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
/**
* If this is the first buffer of the received packet, set the
@@ -1410,7 +1407,7 @@ ice_recv_scattered_pkts(void *rx_queue,
* update the pointer to the last mbuf of the current scattered
* packet and continue to parse the RX ring.
*/
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_EOF_S))) {
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_EOF_S))) {
last_seg = rxm;
continue;
}
@@ -1442,13 +1439,11 @@ ice_recv_scattered_pkts(void *rx_queue,
first_seg->port = rxq->port_id;
first_seg->ol_flags = 0;
-
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- first_seg->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
-
+ first_seg->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(first_seg, &rxd);
+ ice_rxd_to_pkt_fields(first_seg, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
first_seg->ol_flags |= pkt_flags;
/* Prefetch data of first segment, if configured to do so. */
rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr,
@@ -1538,9 +1533,8 @@ ice_dev_supported_ptypes_get(struct rte_eth_dev *dev)
int
ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
{
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_queue *rxq = rx_queue;
- volatile uint64_t *status;
- uint64_t mask;
uint32_t desc;
if (unlikely(offset >= rxq->nb_rx_desc))
@@ -1553,10 +1547,9 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- status = &rxq->rx_ring[desc].wb.qword1.status_error_len;
- mask = rte_cpu_to_le_64((1ULL << ICE_RX_DESC_STATUS_DD_S) <<
- ICE_RXD_QW1_STATUS_S);
- if (*status & mask)
+ rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
+ (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
return RTE_ETH_RX_DESC_AVAIL;
@@ -1642,8 +1635,8 @@ ice_recv_pkts(void *rx_queue,
{
struct ice_rx_queue *rxq = rx_queue;
volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
- volatile union ice_rx_desc *rxdp;
- union ice_rx_desc rxd;
+ volatile union ice_rx_flex_desc *rxdp;
+ union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
struct ice_rx_entry *rxe;
struct rte_mbuf *nmb; /* new allocated mbuf */
@@ -1652,21 +1645,18 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_rx = 0;
uint16_t nb_hold = 0;
uint16_t rx_packet_len;
- uint32_t rx_status;
- uint64_t qword1;
+ uint16_t rx_stat_err0;
uint64_t dma_addr;
- uint64_t pkt_flags = 0;
+ uint64_t pkt_flags;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = &rx_ring[rx_id];
- qword1 = rte_le_to_cpu_64(rxdp->wb.qword1.status_error_len);
- rx_status = (qword1 & ICE_RXD_QW1_STATUS_M) >>
- ICE_RXD_QW1_STATUS_S;
+ rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
- if (!(rx_status & (1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rx_stat_err0 & (1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
break;
/* allocate mbuf */
@@ -1696,8 +1686,8 @@ ice_recv_pkts(void *rx_queue,
rxdp->read.pkt_addr = dma_addr;
/* calculate rx_packet_len of the received pkt */
- rx_packet_len = ((qword1 & ICE_RXD_QW1_LEN_PBUF_M) >>
- ICE_RXD_QW1_LEN_PBUF_S) - rxq->crc_len;
+ rx_packet_len = (rte_le_to_cpu_16(rxd.wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M) - rxq->crc_len;
/* fill old mbuf with received descriptor: rxd */
rxm->data_off = RTE_PKTMBUF_HEADROOM;
@@ -1707,15 +1697,11 @@ ice_recv_pkts(void *rx_queue,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
- ice_rxd_to_vlan_tci(rxm, rxdp);
- rxm->packet_type = ptype_tbl[(uint8_t)((qword1 &
- ICE_RXD_QW1_PTYPE_M) >>
- ICE_RXD_QW1_PTYPE_S)];
- pkt_flags = ice_rxd_status_to_pkt_flags(qword1);
- pkt_flags |= ice_rxd_error_to_pkt_flags(qword1);
- if (pkt_flags & PKT_RX_RSS_HASH)
- rxm->hash.rss =
- rte_le_to_cpu_32(rxd.wb.qword0.hi_dword.rss);
+ rxm->packet_type = ptype_tbl[ICE_RX_FLEX_DESC_PTYPE_M &
+ rte_le_to_cpu_16(rxd.wb.ptype_flex_flags0)];
+ ice_rxd_to_vlan_tci(rxm, &rxd);
+ ice_rxd_to_pkt_fields(rxm, &rxd);
+ pkt_flags = ice_rxd_error_to_pkt_flags(rx_stat_err0);
rxm->ol_flags |= pkt_flags;
/* copy old mbuf to rx_pkts */
rx_pkts[nb_rx++] = rxm;
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 9:02 ` Ye Xiaolong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
` (3 subsequent siblings)
6 siblings, 1 reply; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
The ice has the feature to extract protocol fields into flex descriptor
by programming per queue. Currently, the ice PMD will put the protocol
fields into rte_mbuf::udata64 with different type format. Application
can access the protocol fields quickly.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
doc/guides/nics/ice.rst | 101 +++++++++
doc/guides/rel_notes/release_19_11.rst | 4 +
drivers/net/ice/Makefile | 3 +
drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
drivers/net/ice/ice_ethdev.h | 4 +
drivers/net/ice/ice_rxtx.c | 60 +++++
drivers/net/ice/ice_rxtx.h | 2 +
drivers/net/ice/ice_rxtx_vec_common.h | 3 +
drivers/net/ice/meson.build | 2 +
drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
10 files changed, 632 insertions(+)
create mode 100644 drivers/net/ice/rte_pmd_ice.h
diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst
index 03819d29f..8a6f60e71 100644
--- a/doc/guides/nics/ice.rst
+++ b/doc/guides/nics/ice.rst
@@ -61,6 +61,107 @@ Runtime Config Options
NOTE: In Safe mode, only very limited features are available, features like RSS,
checksum, fdir, tunneling ... are all disabled.
+- ``Protocol extraction for per queue``
+
+ Configure the RX queues to do protocol extraction into ``rte_mbuf::udata64``
+ for protocol handling acceleration, like checking the TCP SYN packets quickly.
+
+ The argument format is::
+
+ -w 18:00.0,proto_xtr=<queues:protocol>[<queues:protocol>...]
+ -w 18:00.0,proto_xtr=<protocol>
+
+ Queues are grouped by ``(`` and ``)`` within the group. The ``-`` character
+ is used as a range separator and ``,`` is used as a single number separator.
+ The grouping ``()`` can be omitted for single element group. If no queues are
+ specified, PMD will use this protocol extraction type for all queues.
+
+ Protocol is : ``vlan, ipv4, ipv6, ipv6_flow, tcp``.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr='[(1,2-3,8-9):tcp,10-13:vlan]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-13 are
+ VLAN extraction, other queues run with no protocol extraction.
+
+ .. code-block:: console
+
+ testpmd -w 18:00.0,proto_xtr=vlan,proto_xtr='[(1,2-3,8-9):tcp,10-23:ipv6]'
+
+ This setting means queues 1, 2-3, 8-9 are TCP extraction, queues 10-23 are
+ IPv6 extraction, other queues use the default VLAN extraction.
+
+ The extraction will be copied into the lower 32 bit of ``rte_mbuf::udata64``.
+
+ .. table:: Protocol extraction : ``vlan``
+
+ +----------------------------+----------------------------+
+ | VLAN2 | VLAN1 |
+ +======+===+=================+======+===+=================+
+ | PCP | D | VID | PCP | D | VID |
+ +------+---+-----------------+------+---+-----------------+
+
+ VLAN1 - single or EVLAN (first for QinQ).
+
+ VLAN2 - C-VLAN (second for QinQ).
+
+ .. table:: Protocol extraction : ``ipv4``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +======+=======+=============+==============+=============+
+ | Ver |Hdr Len| ToS | TTL | Protocol |
+ +------+-------+-------------+--------------+-------------+
+
+ IPHDR1 - IPv4 header word 4, "TTL" and "Protocol" fields.
+
+ IPHDR2 - IPv4 header word 0, "Ver", "Hdr Len" and "Type of Service" fields.
+
+ .. table:: Protocol extraction : ``ipv6``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+=============+==============+
+ | Ver |Traffic class| Flow | Next Header | Hop Limit |
+ +-----+-------------+--------+-------------+--------------+
+
+ IPHDR1 - IPv6 header word 3, "Next Header" and "Hop Limit" fields.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``ipv6_flow``
+
+ +----------------------------+----------------------------+
+ | IPHDR2 | IPHDR1 |
+ +=====+=============+========+============================+
+ | Ver |Traffic class| Flow Label |
+ +-----+-------------+-------------------------------------+
+
+ IPHDR1 - IPv6 header word 1, 16 low bits of the "Flow Label" field.
+
+ IPHDR2 - IPv6 header word 0, "Ver", "Traffic class" and high 4 bits of
+ "Flow Label" fields.
+
+ .. table:: Protocol extraction : ``tcp``
+
+ +----------------------------+----------------------------+
+ | TCPHDR2 | TCPHDR1 |
+ +============================+======+======+==============+
+ | Reserved |Offset| RSV | Flags |
+ +----------------------------+------+------+--------------+
+
+ TCPHDR1 - TCP header word 6, "Data Offset" and "Flags" fields.
+
+ TCPHDR2 - Reserved
+
+ Use ``get_proto_xtr_flds(struct rte_mbuf *mb)`` to access the protocol
+ extraction, do not use ``rte_mbuf::udata64`` directly.
+
+ The ``dump_proto_xtr_flds(struct rte_mbuf *mb)`` routine shows how to
+ access the protocol extraction result in ``struct rte_mbuf``.
+
Driver compilation and testing
------------------------------
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index 8490d897c..479726285 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -56,6 +56,10 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Updated the ICE driver.**
+
+ * Added support for handling Receive Flex Descriptor.
+ * Added support for protocol extraction on per Rx queue.
Removed Items
-------------
diff --git a/drivers/net/ice/Makefile b/drivers/net/ice/Makefile
index ae53c2646..4a279f196 100644
--- a/drivers/net/ice/Makefile
+++ b/drivers/net/ice/Makefile
@@ -82,4 +82,7 @@ ifeq ($(CC_AVX2_SUPPORT), 1)
endif
SRCS-$(CONFIG_RTE_LIBRTE_ICE_PMD) += ice_generic_flow.c
+# install this header file
+SYMLINK-$(CONFIG_RTE_LIBRTE_ICE_PMD)-include := rte_pmd_ice.h
+
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 44a14cb8a..7c74b6169 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -19,9 +19,11 @@
/* devargs */
#define ICE_SAFE_MODE_SUPPORT_ARG "safe-mode-support"
+#define ICE_PROTO_XTR_ARG "proto_xtr"
static const char * const ice_valid_args[] = {
ICE_SAFE_MODE_SUPPORT_ARG,
+ ICE_PROTO_XTR_ARG,
NULL
};
@@ -257,6 +259,280 @@ ice_init_controlq_parameter(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MAILBOXQ_BUF_SZ;
}
+static int
+lookup_proto_xtr_type(const char *xtr_name)
+{
+ static struct {
+ const char *name;
+ enum proto_xtr_type type;
+ } xtr_type_map[] = {
+ { "vlan", PROTO_XTR_VLAN },
+ { "ipv4", PROTO_XTR_IPV4 },
+ { "ipv6", PROTO_XTR_IPV6 },
+ { "ipv6_flow", PROTO_XTR_IPV6_FLOW },
+ { "tcp", PROTO_XTR_TCP },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_type_map); i++) {
+ if (strcmp(xtr_name, xtr_type_map[i].name) == 0)
+ return xtr_type_map[i].type;
+ }
+
+ return -1;
+}
+
+/*
+ * Parse elem, the elem could be single number/range or '(' ')' group
+ * 1) A single number elem, it's just a simple digit. e.g. 9
+ * 2) A single range elem, two digits with a '-' between. e.g. 2-6
+ * 3) A group elem, combines multiple 1) or 2) with '( )'. e.g (0,2-4,6)
+ * Within group elem, '-' used for a range separator;
+ * ',' used for a single number.
+ */
+static int
+parse_queue_set(const char *input, int xtr_type, struct ice_devargs *devargs)
+{
+ const char *str = input;
+ char *end = NULL;
+ uint32_t min, max;
+ uint32_t idx;
+
+ while (isblank(*str))
+ str++;
+
+ if (!isdigit(*str) && *str != '(')
+ return -1;
+
+ /* process single number or single range of number */
+ if (*str != '(') {
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ while (isblank(*end))
+ end++;
+
+ min = idx;
+ max = idx;
+
+ /* process single <number>-<number> */
+ if (*end == '-') {
+ end++;
+ while (isblank(*end))
+ end++;
+ if (!isdigit(*end))
+ return -1;
+
+ errno = 0;
+ idx = strtoul(end, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ max = idx;
+ while (isblank(*end))
+ end++;
+ }
+
+ if (*end != ':')
+ return -1;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ return 0;
+ }
+
+ /* process set within bracket */
+ str++;
+ while (isblank(*str))
+ str++;
+ if (*str == '\0')
+ return -1;
+
+ min = ICE_MAX_QUEUE_NUM;
+ do {
+ /* go ahead to the first digit */
+ while (isblank(*str))
+ str++;
+ if (!isdigit(*str))
+ return -1;
+
+ /* get the digit value */
+ errno = 0;
+ idx = strtoul(str, &end, 10);
+ if (errno || end == NULL || idx >= ICE_MAX_QUEUE_NUM)
+ return -1;
+
+ /* go ahead to separator '-',',' and ')' */
+ while (isblank(*end))
+ end++;
+ if (*end == '-') {
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+ else /* avoid continuous '-' */
+ return -1;
+ } else if (*end == ',' || *end == ')') {
+ max = idx;
+ if (min == ICE_MAX_QUEUE_NUM)
+ min = idx;
+
+ for (idx = RTE_MIN(min, max);
+ idx <= RTE_MAX(min, max); idx++)
+ devargs->proto_xtr[idx] = xtr_type;
+
+ min = ICE_MAX_QUEUE_NUM;
+ } else {
+ return -1;
+ }
+
+ str = end + 1;
+ } while (*end != ')' && *end != '\0');
+
+ return 0;
+}
+
+static int
+parse_queue_proto_xtr(const char *queues, struct ice_devargs *devargs)
+{
+ const char *queue_start;
+ uint32_t idx;
+ int xtr_type;
+ char xtr_name[32];
+
+ while (isblank(*queues))
+ queues++;
+
+ if (*queues != '[') {
+ xtr_type = lookup_proto_xtr_type(queues);
+ if (xtr_type < 0)
+ return -1;
+
+ memset(devargs->proto_xtr, xtr_type,
+ sizeof(devargs->proto_xtr));
+
+ return 0;
+ }
+
+ queues++;
+ do {
+ while (isblank(*queues))
+ queues++;
+ if (*queues == '\0')
+ return -1;
+
+ queue_start = queues;
+
+ /* go across a complete bracket */
+ if (*queue_start == '(') {
+ queues += strcspn(queues, ")");
+ if (*queues != ')')
+ return -1;
+ }
+
+ /* scan the separator ':' */
+ queues += strcspn(queues, ":");
+ if (*queues++ != ':')
+ return -1;
+ while (isblank(*queues))
+ queues++;
+
+ for (idx = 0; ; idx++) {
+ if (isblank(queues[idx]) ||
+ queues[idx] == ',' ||
+ queues[idx] == ']' ||
+ queues[idx] == '\0')
+ break;
+
+ if (idx > sizeof(xtr_name) - 2)
+ return -1;
+
+ xtr_name[idx] = queues[idx];
+ }
+ xtr_name[idx] = '\0';
+ xtr_type = lookup_proto_xtr_type(xtr_name);
+ if (xtr_type < 0)
+ return -1;
+
+ queues += idx;
+
+ while (isblank(*queues) || *queues == ',' || *queues == ']')
+ queues++;
+
+ if (parse_queue_set(queue_start, xtr_type, devargs) < 0)
+ return -1;
+ } while (*queues != '\0');
+
+ return 0;
+}
+
+static int
+handle_proto_xtr_arg(__rte_unused const char *key, const char *value,
+ void *extra_args)
+{
+ struct ice_devargs *devargs = extra_args;
+
+ if (value == NULL || extra_args == NULL)
+ return -EINVAL;
+
+ if (parse_queue_proto_xtr(value, devargs) < 0) {
+ PMD_DRV_LOG(ERR,
+ "The protocol extraction parameter is wrong : '%s'",
+ value);
+ return -1;
+ }
+
+ return 0;
+}
+
+static bool
+ice_proto_xtr_support(struct ice_hw *hw)
+{
+#define FLX_REG(val, fld, idx) \
+ (((val) & GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_M) >> \
+ GLFLXP_RXDID_FLX_WRD_##idx##_##fld##_S)
+ static struct {
+ uint32_t rxdid;
+ uint16_t protid_0;
+ uint16_t protid_1;
+ } xtr_sets[] = {
+ { ICE_RXDID_COMMS_AUX_VLAN, ICE_PROT_EVLAN_O, ICE_PROT_VLAN_O },
+ { ICE_RXDID_COMMS_AUX_IPV4, ICE_PROT_IPV4_OF_OR_S,
+ ICE_PROT_IPV4_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_IPV6_FLOW, ICE_PROT_IPV6_OF_OR_S,
+ ICE_PROT_IPV6_OF_OR_S },
+ { ICE_RXDID_COMMS_AUX_TCP, ICE_PROT_TCP_IL, ICE_PROT_ID_INVAL },
+ };
+ uint32_t i;
+
+ for (i = 0; i < RTE_DIM(xtr_sets); i++) {
+ uint32_t rxdid = xtr_sets[i].rxdid;
+ uint32_t v;
+
+ if (xtr_sets[i].protid_0 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_4(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 4) != xtr_sets[i].protid_0 ||
+ FLX_REG(v, RXDID_OPCODE, 4) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+
+ if (xtr_sets[i].protid_1 != ICE_PROT_ID_INVAL) {
+ v = ICE_READ_REG(hw, GLFLXP_RXDID_FLX_WRD_5(rxdid));
+
+ if (FLX_REG(v, PROT_MDID, 5) != xtr_sets[i].protid_1 ||
+ FLX_REG(v, RXDID_OPCODE, 5) != ICE_RX_OPC_EXTRACT)
+ return false;
+ }
+ }
+
+ return true;
+}
+
static int
ice_res_pool_init(struct ice_res_pool_info *pool, uint32_t base,
uint32_t num)
@@ -1079,6 +1355,8 @@ ice_interrupt_handler(void *param)
static int
ice_pf_sw_init(struct rte_eth_dev *dev)
{
+ struct ice_adapter *ad =
+ ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private);
struct ice_pf *pf = ICE_DEV_PRIVATE_TO_PF(dev->data->dev_private);
struct ice_hw *hw = ICE_PF_TO_HW(pf);
@@ -1088,6 +1366,16 @@ ice_pf_sw_init(struct rte_eth_dev *dev)
pf->lan_nb_qps = pf->lan_nb_qp_max;
+ if (ice_proto_xtr_support(hw))
+ pf->proto_xtr = rte_zmalloc(NULL, pf->lan_nb_qps, 0);
+
+ if (pf->proto_xtr != NULL)
+ rte_memcpy(pf->proto_xtr, ad->devargs.proto_xtr,
+ RTE_MIN((size_t)pf->lan_nb_qps,
+ sizeof(ad->devargs.proto_xtr)));
+ else
+ PMD_DRV_LOG(NOTICE, "Protocol extraction is disabled");
+
return 0;
}
@@ -1378,9 +1666,18 @@ static int ice_parse_devargs(struct rte_eth_dev *dev)
return -EINVAL;
}
+ memset(ad->devargs.proto_xtr, PROTO_XTR_NONE,
+ sizeof(ad->devargs.proto_xtr));
+
+ ret = rte_kvargs_process(kvlist, ICE_PROTO_XTR_ARG,
+ &handle_proto_xtr_arg, &ad->devargs);
+ if (ret)
+ goto bail;
+
ret = rte_kvargs_process(kvlist, ICE_SAFE_MODE_SUPPORT_ARG,
&parse_bool, &ad->devargs.safe_mode_support);
+bail:
rte_kvargs_free(kvlist);
return ret;
}
@@ -1547,6 +1844,7 @@ ice_dev_init(struct rte_eth_dev *dev)
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
return ret;
}
@@ -1672,6 +1970,8 @@ ice_dev_close(struct rte_eth_dev *dev)
rte_free(hw->port_info);
hw->port_info = NULL;
ice_shutdown_all_ctrlq(hw);
+ rte_free(pf->proto_xtr);
+ pf->proto_xtr = NULL;
}
static int
@@ -3795,6 +4095,7 @@ RTE_PMD_REGISTER_PCI(net_ice, rte_ice_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ice, pci_id_ice_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ice,
+ ICE_PROTO_XTR_ARG "=[queue:]<vlan|ipv4|ipv6|ipv6_flow|tcp>"
ICE_SAFE_MODE_SUPPORT_ARG "=<0|1>");
RTE_INIT(ice_init_log)
diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h
index f569da833..adbb66322 100644
--- a/drivers/net/ice/ice_ethdev.h
+++ b/drivers/net/ice/ice_ethdev.h
@@ -263,6 +263,7 @@ struct ice_pf {
uint16_t lan_nb_qp_max;
uint16_t lan_nb_qps; /* The number of queue pairs of LAN */
uint16_t base_queue; /* The base queue pairs index in the device */
+ uint8_t *proto_xtr; /* Protocol extraction type for all queues */
struct ice_hw_port_stats stats_offset;
struct ice_hw_port_stats stats;
/* internal packet statistics, it should be excluded from the total */
@@ -273,11 +274,14 @@ struct ice_pf {
struct ice_flow_list flow_list;
};
+#define ICE_MAX_QUEUE_NUM 2048
+
/**
* Cache devargs parse result.
*/
struct ice_devargs {
int safe_mode_support;
+ uint8_t proto_xtr[ICE_MAX_QUEUE_NUM];
};
/**
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index d2e36853f..66487b990 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -13,6 +13,35 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+static inline uint8_t
+ice_rxdid_to_proto_xtr_type(uint8_t rxdid)
+{
+ static uint8_t xtr_map[] = {
+ [ICE_RXDID_COMMS_AUX_VLAN] = PROTO_XTR_VLAN,
+ [ICE_RXDID_COMMS_AUX_IPV4] = PROTO_XTR_IPV4,
+ [ICE_RXDID_COMMS_AUX_IPV6] = PROTO_XTR_IPV6,
+ [ICE_RXDID_COMMS_AUX_IPV6_FLOW] = PROTO_XTR_IPV6_FLOW,
+ [ICE_RXDID_COMMS_AUX_TCP] = PROTO_XTR_TCP,
+ };
+
+ return rxdid < RTE_DIM(xtr_map) ? xtr_map[rxdid] : PROTO_XTR_NONE;
+}
+
+static inline uint8_t
+ice_proto_xtr_type_to_rxdid(uint8_t xtr_type)
+{
+ static uint8_t rxdid_map[] = {
+ [PROTO_XTR_NONE] = ICE_RXDID_COMMS_GENERIC,
+ [PROTO_XTR_VLAN] = ICE_RXDID_COMMS_AUX_VLAN,
+ [PROTO_XTR_IPV4] = ICE_RXDID_COMMS_AUX_IPV4,
+ [PROTO_XTR_IPV6] = ICE_RXDID_COMMS_AUX_IPV6,
+ [PROTO_XTR_IPV6_FLOW] = ICE_RXDID_COMMS_AUX_IPV6_FLOW,
+ [PROTO_XTR_TCP] = ICE_RXDID_COMMS_AUX_TCP,
+ };
+
+ return xtr_type < RTE_DIM(rxdid_map) ?
+ rxdid_map[xtr_type] : ICE_RXDID_COMMS_GENERIC;
+}
static enum ice_status
ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
@@ -84,6 +113,11 @@ ice_program_hw_rx_queue(struct ice_rx_queue *rxq)
rx_ctx.showiv = 0;
rx_ctx.crcstrip = (rxq->crc_len == 0) ? 1 : 0;
+ rxdid = ice_proto_xtr_type_to_rxdid(rxq->proto_xtr);
+
+ PMD_DRV_LOG(DEBUG, "Port (%u) - Rx queue (%u) is set with RXDID : %u",
+ rxq->port_id, rxq->queue_id, rxdid);
+
/* Enable Flexible Descriptors in the queue context which
* allows this driver to select a specific receive descriptor format
*/
@@ -641,6 +675,8 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
rxq->drop_en = rx_conf->rx_drop_en;
rxq->vsi = vsi;
rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->proto_xtr = pf->proto_xtr != NULL ?
+ pf->proto_xtr[queue_idx] : PROTO_XTR_NONE;
/* Allocate the maximun number of RX ring hardware descriptor. */
len = ICE_MAX_RING_DESC;
@@ -1062,6 +1098,10 @@ ice_rxd_to_vlan_tci(struct rte_mbuf *mb, volatile union ice_rx_flex_desc *rxdp)
mb->vlan_tci, mb->vlan_tci_outer);
}
+#define ICE_RX_PROTO_XTR_VALID \
+ ((1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S) | \
+ (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+
static inline void
ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
volatile union ice_rx_flex_desc *rxdp)
@@ -1075,6 +1115,26 @@ ice_rxd_to_pkt_fields(struct rte_mbuf *mb,
mb->ol_flags |= PKT_RX_RSS_HASH;
mb->hash.rss = rte_le_to_cpu_32(desc->rss_hash);
}
+
+#ifndef RTE_LIBRTE_ICE_16BYTE_RX_DESC
+ init_proto_xtr_flds(mb);
+
+ stat_err = rte_le_to_cpu_16(desc->status_error1);
+ if (stat_err & ICE_RX_PROTO_XTR_VALID) {
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S))
+ xtr->u.raw.data0 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux0);
+
+ if (stat_err & (1 << ICE_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S))
+ xtr->u.raw.data1 =
+ rte_le_to_cpu_16(desc->flex_ts.flex.aux1);
+
+ xtr->type = ice_rxdid_to_proto_xtr_type(desc->rxdid);
+ xtr->magic = PROTO_XTR_MAGIC_ID;
+ }
+#endif
}
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index 64e891875..de16637f3 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -5,6 +5,7 @@
#ifndef _ICE_RXTX_H_
#define _ICE_RXTX_H_
+#include "rte_pmd_ice.h"
#include "ice_ethdev.h"
#define ICE_ALIGN_RING_DESC 32
@@ -78,6 +79,7 @@ struct ice_rx_queue {
uint16_t max_pkt_len; /* Maximum packet length */
bool q_set; /* indicate if rx queue has been configured */
bool rx_deferred_start; /* don't start this queue in dev start */
+ uint8_t proto_xtr; /* Protocol extraction from flexible descriptor */
ice_rx_release_mbufs_t rx_rel_mbufs;
};
diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h
index c5f0d564f..080ca4175 100644
--- a/drivers/net/ice/ice_rxtx_vec_common.h
+++ b/drivers/net/ice/ice_rxtx_vec_common.h
@@ -234,6 +234,9 @@ ice_rx_vec_queue_default(struct ice_rx_queue *rxq)
if (rxq->nb_rx_desc % rxq->rx_free_thresh)
return -1;
+ if (rxq->proto_xtr != PROTO_XTR_NONE)
+ return -1;
+
return 0;
}
diff --git a/drivers/net/ice/meson.build b/drivers/net/ice/meson.build
index 36b4b3c85..6828170a9 100644
--- a/drivers/net/ice/meson.build
+++ b/drivers/net/ice/meson.build
@@ -34,3 +34,5 @@ if arch_subdir == 'x86'
objs += ice_avx2_lib.extract_objects('ice_rxtx_vec_avx2.c')
endif
endif
+
+install_headers('rte_pmd_ice.h')
diff --git a/drivers/net/ice/rte_pmd_ice.h b/drivers/net/ice/rte_pmd_ice.h
new file mode 100644
index 000000000..719487e1e
--- /dev/null
+++ b/drivers/net/ice/rte_pmd_ice.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2019 Intel Corporation
+ */
+
+#ifndef _RTE_PMD_ICE_H_
+#define _RTE_PMD_ICE_H_
+
+#include <stdio.h>
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum proto_xtr_type {
+ PROTO_XTR_NONE,
+ PROTO_XTR_VLAN,
+ PROTO_XTR_IPV4,
+ PROTO_XTR_IPV6,
+ PROTO_XTR_IPV6_FLOW,
+ PROTO_XTR_TCP,
+};
+
+struct proto_xtr_flds {
+ union {
+ struct {
+ uint16_t data0;
+ uint16_t data1;
+ } raw;
+ struct {
+ uint16_t stag_vid:12,
+ stag_dei:1,
+ stag_pcp:3;
+ uint16_t ctag_vid:12,
+ ctag_dei:1,
+ ctag_pcp:3;
+ } vlan;
+ struct {
+ uint16_t protocol:8,
+ ttl:8;
+ uint16_t tos:8,
+ ihl:4,
+ version:4;
+ } ipv4;
+ struct {
+ uint16_t hoplimit:8,
+ nexthdr:8;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6;
+ struct {
+ uint16_t flowlo16;
+ uint16_t flowhi4:4,
+ tc:8,
+ version:4;
+ } ipv6_flow;
+ struct {
+ uint16_t fin:1,
+ syn:1,
+ rst:1,
+ psh:1,
+ ack:1,
+ urg:1,
+ ece:1,
+ cwr:1,
+ res1:4,
+ doff:4;
+ uint16_t rsvd;
+ } tcp;
+ } u;
+
+ uint16_t rsvd;
+
+ uint8_t type;
+
+#define PROTO_XTR_MAGIC_ID 0xCE
+ uint8_t magic;
+};
+
+static inline void
+init_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ mb->udata64 = 0;
+}
+
+static inline struct proto_xtr_flds *
+get_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ RTE_BUILD_BUG_ON(sizeof(struct proto_xtr_flds) > sizeof(mb->udata64));
+
+ return (struct proto_xtr_flds *)&mb->udata64;
+}
+
+static inline void
+dump_proto_xtr_flds(struct rte_mbuf *mb)
+{
+ struct proto_xtr_flds *xtr = get_proto_xtr_flds(mb);
+
+ if (xtr->magic != PROTO_XTR_MAGIC_ID || xtr->type == PROTO_XTR_NONE)
+ return;
+
+ printf(" - Protocol Extraction:[0x%04x:0x%04x],",
+ xtr->u.raw.data0, xtr->u.raw.data1);
+
+ if (xtr->type == PROTO_XTR_VLAN)
+ printf("vlan,stag=%u:%u:%u,ctag=%u:%u:%u ",
+ xtr->u.vlan.stag_pcp,
+ xtr->u.vlan.stag_dei,
+ xtr->u.vlan.stag_vid,
+ xtr->u.vlan.ctag_pcp,
+ xtr->u.vlan.ctag_dei,
+ xtr->u.vlan.ctag_vid);
+ else if (xtr->type == PROTO_XTR_IPV4)
+ printf("ipv4,ver=%u,hdrlen=%u,tos=%u,ttl=%u,proto=%u ",
+ xtr->u.ipv4.version,
+ xtr->u.ipv4.ihl,
+ xtr->u.ipv4.tos,
+ xtr->u.ipv4.ttl,
+ xtr->u.ipv4.protocol);
+ else if (xtr->type == PROTO_XTR_IPV6)
+ printf("ipv6,ver=%u,tc=%u,flow_hi4=0x%x,nexthdr=%u,hoplimit=%u ",
+ xtr->u.ipv6.version,
+ xtr->u.ipv6.tc,
+ xtr->u.ipv6.flowhi4,
+ xtr->u.ipv6.nexthdr,
+ xtr->u.ipv6.hoplimit);
+ else if (xtr->type == PROTO_XTR_IPV6_FLOW)
+ printf("ipv6_flow,ver=%u,tc=%u,flow=0x%x%04x ",
+ xtr->u.ipv6_flow.version,
+ xtr->u.ipv6_flow.tc,
+ xtr->u.ipv6_flow.flowhi4,
+ xtr->u.ipv6_flow.flowlo16);
+ else if (xtr->type == PROTO_XTR_TCP)
+ printf("tcp,doff=%u,flags=%s%s%s%s%s%s%s%s ",
+ xtr->u.tcp.doff,
+ xtr->u.tcp.cwr ? "C" : "",
+ xtr->u.tcp.ece ? "E" : "",
+ xtr->u.tcp.urg ? "U" : "",
+ xtr->u.tcp.ack ? "A" : "",
+ xtr->u.tcp.psh ? "P" : "",
+ xtr->u.tcp.rst ? "R" : "",
+ xtr->u.tcp.syn ? "S" : "",
+ xtr->u.tcp.fin ? "F" : "");
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_PMD_ICE_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 4/6] net/ice: switch to flexible descriptor in SSE path
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
` (2 preceding siblings ...)
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
` (2 subsequent siblings)
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Wenzhuo Lu <wenzhuo.lu@intel.com>
With this path, the flexible descriptor is supported
in SSE path. And the legacy descriptor is not supported.
Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
drivers/net/ice/ice_rxtx_vec_sse.c | 239 +++++++++++++----------------
1 file changed, 110 insertions(+), 129 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index 967a7b16b..dafcb081a 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -15,14 +15,14 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
struct rte_mbuf *mb0, *mb1;
__m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -88,93 +88,90 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer);
__m128i rearm0, rearm1, rearm2, rearm3;
- __m128i vlan0, vlan1, rss, l3_l4e;
+ __m128i tmp_desc, flags, rss_vlan;
- /* mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication.
+ /* mask everything except checksum, RSS and VLAN flags.
+ * bit6:4 for checksum.
+ * bit12 for RSS indication.
+ * bit13 for VLAN indication.
*/
- const __m128i rss_vlan_msk = _mm_set_epi32(0x1c03804, 0x1c03804,
- 0x1c03804, 0x1c03804);
+ const __m128i desc_mask = _mm_set_epi32(0x3070, 0x3070,
+ 0x3070, 0x3070);
- const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ const __m128i cksum_mask = _mm_set_epi32(PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD,
- PKT_RX_IP_CKSUM_GOOD |
- PKT_RX_IP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_GOOD |
- PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_MASK |
+ PKT_RX_L4_CKSUM_MASK |
PKT_RX_EIP_CKSUM_BAD);
- /* map rss and vlan type to rss hash and vlan flag */
- const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- 0, 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
- 0, 0, 0, 0);
-
- const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
- 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH, 0, 0,
- 0, 0, PKT_RX_FDIR, 0);
-
- const __m128i l3_l4e_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ /* map the checksum, rss and vlan fields to the checksum, rss
+ * and vlan flag
+ */
+ const __m128i cksum_flags = _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
- vlan0 = _mm_unpackhi_epi32(descs[0], descs[1]);
- vlan1 = _mm_unpackhi_epi32(descs[2], descs[3]);
- vlan0 = _mm_unpacklo_epi64(vlan0, vlan1);
-
- vlan1 = _mm_and_si128(vlan0, rss_vlan_msk);
- vlan0 = _mm_shuffle_epi8(vlan_flags, vlan1);
-
- rss = _mm_srli_epi32(vlan1, 11);
- rss = _mm_shuffle_epi8(rss_flags, rss);
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
- l3_l4e = _mm_srli_epi32(vlan1, 22);
- l3_l4e = _mm_shuffle_epi8(l3_l4e_flags, l3_l4e);
+ const __m128i rss_vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
+
+ /* merge 4 descriptors */
+ flags = _mm_unpackhi_epi32(descs[0], descs[1]);
+ tmp_desc = _mm_unpackhi_epi32(descs[2], descs[3]);
+ tmp_desc = _mm_unpacklo_epi64(flags, tmp_desc);
+ tmp_desc = _mm_and_si128(flags, desc_mask);
+
+ /* checksum flags */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 4);
+ flags = _mm_shuffle_epi8(cksum_flags, tmp_desc);
/* then we shift left 1 bit */
- l3_l4e = _mm_slli_epi32(l3_l4e, 1);
- /* we need to mask out the reduntant bits */
- l3_l4e = _mm_and_si128(l3_l4e, cksum_mask);
+ flags = _mm_slli_epi32(flags, 1);
+ /* we need to mask out the reduntant bits introduced by RSS or
+ * VLAN fields.
+ */
+ flags = _mm_and_si128(flags, cksum_mask);
- vlan0 = _mm_or_si128(vlan0, rss);
- vlan0 = _mm_or_si128(vlan0, l3_l4e);
+ /* RSS, VLAN flag */
+ tmp_desc = _mm_srli_epi32(tmp_desc, 8);
+ rss_vlan = _mm_shuffle_epi8(rss_vlan_flags, tmp_desc);
+
+ /* merge the flags */
+ flags = _mm_or_si128(flags, rss_vlan);
/**
* At this point, we have the 4 sets of flags in the low 16-bits
- * of each 32-bit value in vlan0.
+ * of each 32-bit value in flags.
* We want to extract these, and merge them with the mbuf init data
* so we can do a single 16-byte write to the mbuf to set the flags
* and all the other initialization fields. Extracting the
* appropriate flags means that we have to do a shift and blend for
* each mbuf before we do the write.
*/
- rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 8), 0x10);
- rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vlan0, 4), 0x10);
- rearm2 = _mm_blend_epi16(mbuf_init, vlan0, 0x10);
- rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10);
+ rearm0 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x10);
+ rearm1 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x10);
+ rearm2 = _mm_blend_epi16(mbuf_init, flags, 0x10);
+ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x10);
/* write the rearm data and the olflags in one write */
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
@@ -187,22 +184,24 @@ ice_rx_desc_to_olflags_v(struct ice_rx_queue *rxq, __m128i descs[4],
_mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3);
}
-#define PKTLEN_SHIFT 10
-
static inline void
ice_rx_desc_to_ptype_v(__m128i descs[4], struct rte_mbuf **rx_pkts,
uint32_t *ptype_tbl)
{
- __m128i ptype0 = _mm_unpackhi_epi64(descs[0], descs[1]);
- __m128i ptype1 = _mm_unpackhi_epi64(descs[2], descs[3]);
-
- ptype0 = _mm_srli_epi64(ptype0, 30);
- ptype1 = _mm_srli_epi64(ptype1, 30);
-
- rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 0)];
- rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi8(ptype0, 8)];
- rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 0)];
- rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi8(ptype1, 8)];
+ const __m128i ptype_mask = _mm_set_epi16(0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M,
+ 0, ICE_RX_FLEX_DESC_PTYPE_M);
+ __m128i ptype_01 = _mm_unpacklo_epi32(descs[0], descs[1]);
+ __m128i ptype_23 = _mm_unpacklo_epi32(descs[2], descs[3]);
+ __m128i ptype_all = _mm_unpacklo_epi64(ptype_01, ptype_23);
+
+ ptype_all = _mm_and_si128(ptype_all, ptype_mask);
+
+ rx_pkts[0]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ rx_pkts[3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
}
/**
@@ -215,21 +214,39 @@ static inline uint16_t
_ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
uint64_t var;
- __m128i shuf_msk;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
-
__m128i crc_adjust = _mm_set_epi16
- (0, 0, 0, /* ignore non-length fields */
+ (0, 0, 0, /* ignore non-length fields */
-rxq->crc_len, /* sub crc on data_len */
0, /* ignore high-16bits of pkt_len */
-rxq->crc_len, /* sub crc on pkt_len */
- 0, 0 /* ignore pkt_type field */
+ 0, 0 /* ignore pkt_type field */
);
+ const __m128i zero = _mm_setzero_si128();
+ /* mask to shuffle from desc. to mbuf */
+ const __m128i shuf_msk = _mm_set_epi8
+ (0xFF, 0xFF, 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, low 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /* pkt_type set as unknown */
+ );
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+
/**
* compile-time check the above crc_adjust layout is correct.
* NOTE: the first field (lowest address) is given last in set_epi16
@@ -239,7 +256,13 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
- __m128i dd_check, eop_check;
+
+ /* 4 packets DD mask */
+ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ /* 4 packets EOP mask */
+ const __m128i eop_check = _mm_set_epi64x(0x0000000200000002LL,
+ 0x0000000200000002LL);
/* nb_pkts shall be less equal than ICE_MAX_RX_BURST */
nb_pkts = RTE_MIN(nb_pkts, ICE_MAX_RX_BURST);
@@ -250,7 +273,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = rxq->rx_ring + rxq->rx_tail;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
@@ -263,26 +286,10 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
- /* 4 packets DD mask */
- dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL);
-
- /* 4 packets EOP mask */
- eop_check = _mm_set_epi64x(0x0000000200000002LL, 0x0000000200000002LL);
-
- /* mask to shuffle from desc. to mbuf */
- shuf_msk = _mm_set_epi8
- (7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
- );
/**
* Compile-time verify the shuffle mask
* NOTE: some field positions already verified above, but duplicated
@@ -315,7 +322,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
rxdp += ICE_DESCS_PER_LOOP) {
__m128i descs[ICE_DESCS_PER_LOOP];
__m128i pkt_mb1, pkt_mb2, pkt_mb3, pkt_mb4;
- __m128i zero, staterr, sterr_tmp1, sterr_tmp2;
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
/* 2 64 bit or 4 32 bit mbuf pointers in one XMM reg. */
__m128i mbp1;
#if defined(RTE_ARCH_X86_64)
@@ -359,14 +366,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* avoid compiler reorder optimization */
rte_compiler_barrier();
- /* pkt 3,4 shift the pktlen field to be 16-bit aligned*/
- const __m128i len3 = _mm_slli_epi32(descs[3], PKTLEN_SHIFT);
- const __m128i len2 = _mm_slli_epi32(descs[2], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[3] = _mm_blend_epi16(descs[3], len3, 0x80);
- descs[2] = _mm_blend_epi16(descs[2], len2, 0x80);
-
/* D.1 pkt 3,4 convert format from desc to pktmbuf */
pkt_mb4 = _mm_shuffle_epi8(descs[3], shuf_msk);
pkt_mb3 = _mm_shuffle_epi8(descs[2], shuf_msk);
@@ -382,20 +381,11 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
pkt_mb4 = _mm_add_epi16(pkt_mb4, crc_adjust);
pkt_mb3 = _mm_add_epi16(pkt_mb3, crc_adjust);
- /* pkt 1,2 shift the pktlen field to be 16-bit aligned*/
- const __m128i len1 = _mm_slli_epi32(descs[1], PKTLEN_SHIFT);
- const __m128i len0 = _mm_slli_epi32(descs[0], PKTLEN_SHIFT);
-
- /* merge the now-aligned packet length fields back in */
- descs[1] = _mm_blend_epi16(descs[1], len1, 0x80);
- descs[0] = _mm_blend_epi16(descs[0], len0, 0x80);
-
/* D.1 pkt 1,2 convert format from desc to pktmbuf */
pkt_mb2 = _mm_shuffle_epi8(descs[1], shuf_msk);
pkt_mb1 = _mm_shuffle_epi8(descs[0], shuf_msk);
/* C.2 get 4 pkts staterr value */
- zero = _mm_xor_si128(dd_check, dd_check);
staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
/* D.3 copy final 3,4 data to rx_pkts */
@@ -412,15 +402,6 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* C* extract and record EOP bit */
if (split_packet) {
- __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0xFF, 0xFF,
- 0x04, 0x0C,
- 0x00, 0x08);
-
/* and with mask to extract bits, flipping 1-0 */
__m128i eop_bits = _mm_andnot_si128(staterr, eop_check);
/* the staterr values are not in order, as the count
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 5/6] net/ice: switch to Rx flexible descriptor in AVX path
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
` (3 preceding siblings ...)
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
2019-09-24 9:05 ` [dpdk-dev] [PATCH v5 0/6] enable Rx flexible descriptor Ye Xiaolong
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev, Leyi Rong
Switch to Rx flexible descriptor format instead of legacy
descriptor format.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
---
drivers/net/ice/ice_rxtx_vec_avx2.c | 224 ++++++++++++++--------------
1 file changed, 109 insertions(+), 115 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 5ce29c2a2..46776fa12 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -15,10 +15,10 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
{
int i;
uint16_t rx_id;
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -132,8 +132,6 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
ICE_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
}
-#define PKTLEN_SHIFT 10
-
static inline uint16_t
_ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts, uint8_t *split_packet)
@@ -144,7 +142,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp =
+ (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
@@ -161,8 +160,8 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Before we start moving massive data around, check to see if
* there is actually a packet available
*/
- if (!(rxdp->wb.qword1.status_error_len &
- rte_cpu_to_le_32(1 << ICE_RX_DESC_STATUS_DD_S)))
+ if (!(rxdp->wb.status_error0 &
+ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)))
return 0;
/* constants used in processing loop */
@@ -193,21 +192,23 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i shuf_msk =
_mm256_set_epi8
(/* first descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF, /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF, /*pkt_type set as unknown */
/* second descriptor */
- 7, 6, 5, 4, /* octet 4~7, 32bits rss */
- 3, 2, /* octet 2~3, low 16 bits vlan_macip */
- 15, 14, /* octet 15~14, 16 bits data_len */
- 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */
- 15, 14, /* octet 15~14, low 16 bits pkt_len */
- 0xFF, 0xFF, /* pkt_type set as unknown */
- 0xFF, 0xFF /*pkt_type set as unknown */
+ 0xFF, 0xFF,
+ 0xFF, 0xFF, /* rss not supported */
+ 11, 10, /* octet 10~11, 16 bits vlan_macip */
+ 5, 4, /* octet 4~5, 16 bits data_len */
+ 0xFF, 0xFF, /* skip hi 16 bits pkt_len, zero out */
+ 5, 4, /* octet 4~5, 16 bits pkt_len */
+ 0xFF, 0xFF, /* pkt_type set as unknown */
+ 0xFF, 0xFF /*pkt_type set as unknown */
);
/**
* compile-time check the above crc and shuffle layout is correct.
@@ -225,68 +226,68 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Status/Error flag masks */
/**
- * mask everything except RSS, flow director and VLAN flags
- * bit2 is for VLAN tag, bit11 for flow director indication
- * bit13:12 for RSS indication. Bits 3-5 of error
- * field (bits 22-24) are for IP/L4 checksum errors
+ * mask everything except Checksum Reports, RSS indication
+ * and VLAN indication.
+ * bit6:4 for IP/L4 checksum errors.
+ * bit12 is for RSS indication.
+ * bit13 is for VLAN indication.
*/
const __m256i flags_mask =
- _mm256_set1_epi32((1 << 2) | (1 << 11) |
- (3 << 12) | (7 << 22));
- /**
- * data to be shuffled by result of flag mask. If VLAN bit is set,
- * (bit 2), then position 4 in this array will be used in the
- * destination
- */
- const __m256i vlan_flags_shuf =
- _mm256_set_epi32(0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0,
- 0, 0, PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED, 0);
- /**
- * data to be shuffled by result of flag mask, shifted down 11.
- * If RSS/FDIR bits are set, shuffle moves appropriate flags in
- * place.
- */
- const __m256i rss_flags_shuf =
- _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0,/* end up 128-bits */
- 0, 0, 0, 0, 0, 0, 0, 0,
- PKT_RX_RSS_HASH | PKT_RX_FDIR, PKT_RX_RSS_HASH,
- 0, 0, 0, 0, PKT_RX_FDIR, 0);
-
+ _mm256_set1_epi32((7 << 4) | (1 << 12) | (1 << 13));
/**
- * data to be shuffled by the result of the flags mask shifted by 22
+ * data to be shuffled by the result of the flags mask shifted by 4
* bits. This gives use the l3_l4 flags.
*/
const __m256i l3_l4_flags_shuf = _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
/* shift right 1 bit to make sure it not exceed 255 */
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1,
/* second 128-bits */
0, 0, 0, 0, 0, 0, 0, 0,
(PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD |
- PKT_RX_L4_CKSUM_BAD) >> 1,
- (PKT_RX_EIP_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_EIP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_EIP_CKSUM_BAD | PKT_RX_L4_CKSUM_GOOD |
+ PKT_RX_IP_CKSUM_GOOD) >> 1,
(PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_BAD) >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD) >> 1,
- PKT_RX_IP_CKSUM_BAD >> 1,
- (PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD) >> 1);
-
+ (PKT_RX_L4_CKSUM_BAD | PKT_RX_IP_CKSUM_GOOD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD) >> 1,
+ (PKT_RX_L4_CKSUM_GOOD | PKT_RX_IP_CKSUM_GOOD) >> 1);
const __m256i cksum_mask =
_mm256_set1_epi32(PKT_RX_IP_CKSUM_GOOD | PKT_RX_IP_CKSUM_BAD |
PKT_RX_L4_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD |
PKT_RX_EIP_CKSUM_BAD);
+ /**
+ * data to be shuffled by result of flag mask, shifted down 12.
+ * If RSS(bit12)/VLAN(bit13) are set,
+ * shuffle moves appropriate flags in place.
+ */
+ const __m256i rss_vlan_flags_shuf = _mm256_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0,
+ /* end up 128-bits */
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, 0,
+ PKT_RX_RSS_HASH | PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED,
+ PKT_RX_RSS_HASH, 0);
RTE_SET_USED(avx_aligned); /* for 32B descriptors we don't use this */
@@ -369,73 +370,66 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
}
/**
- * convert descriptors 4-7 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 4-7 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len6_7 = _mm256_slli_epi32(raw_desc6_7,
- PKTLEN_SHIFT);
- const __m256i len4_5 = _mm256_slli_epi32(raw_desc4_5,
- PKTLEN_SHIFT);
- const __m256i desc6_7 = _mm256_blend_epi16(raw_desc6_7,
- len6_7, 0x80);
- const __m256i desc4_5 = _mm256_blend_epi16(raw_desc4_5,
- len4_5, 0x80);
- __m256i mb6_7 = _mm256_shuffle_epi8(desc6_7, shuf_msk);
- __m256i mb4_5 = _mm256_shuffle_epi8(desc4_5, shuf_msk);
+ __m256i mb6_7 = _mm256_shuffle_epi8(raw_desc6_7, shuf_msk);
+ __m256i mb4_5 = _mm256_shuffle_epi8(raw_desc4_5, shuf_msk);
mb6_7 = _mm256_add_epi16(mb6_7, crc_adjust);
mb4_5 = _mm256_add_epi16(mb4_5, crc_adjust);
/**
- * to get packet types, shift 64-bit values down 30 bits
- * and so ptype is in lower 8-bits in each
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
*/
- const __m256i ptypes6_7 = _mm256_srli_epi64(desc6_7, 30);
- const __m256i ptypes4_5 = _mm256_srli_epi64(desc4_5, 30);
- const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 24);
- const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 8);
- const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 24);
- const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 8);
+ const __m256i ptype_mask =
+ _mm256_set1_epi16(ICE_RX_FLEX_DESC_PTYPE_M);
+ const __m256i ptypes6_7 =
+ _mm256_and_si256(raw_desc6_7, ptype_mask);
+ const __m256i ptypes4_5 =
+ _mm256_and_si256(raw_desc4_5, ptype_mask);
+ const uint16_t ptype7 = _mm256_extract_epi16(ptypes6_7, 9);
+ const uint16_t ptype6 = _mm256_extract_epi16(ptypes6_7, 1);
+ const uint16_t ptype5 = _mm256_extract_epi16(ptypes4_5, 9);
+ const uint16_t ptype4 = _mm256_extract_epi16(ptypes4_5, 1);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype7], 4);
mb6_7 = _mm256_insert_epi32(mb6_7, ptype_tbl[ptype6], 0);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype5], 4);
mb4_5 = _mm256_insert_epi32(mb4_5, ptype_tbl[ptype4], 0);
/* merge the status bits into one register */
- const __m256i status4_7 = _mm256_unpackhi_epi32(desc6_7,
- desc4_5);
+ const __m256i status4_7 = _mm256_unpackhi_epi32(raw_desc6_7,
+ raw_desc4_5);
/**
- * convert descriptors 0-3 into mbufs, adjusting length and
- * re-arranging fields. Then write into the mbuf
+ * convert descriptors 0-3 into mbufs, re-arrange fields.
+ * Then write into the mbuf.
*/
- const __m256i len2_3 = _mm256_slli_epi32(raw_desc2_3,
- PKTLEN_SHIFT);
- const __m256i len0_1 = _mm256_slli_epi32(raw_desc0_1,
- PKTLEN_SHIFT);
- const __m256i desc2_3 = _mm256_blend_epi16(raw_desc2_3,
- len2_3, 0x80);
- const __m256i desc0_1 = _mm256_blend_epi16(raw_desc0_1,
- len0_1, 0x80);
- __m256i mb2_3 = _mm256_shuffle_epi8(desc2_3, shuf_msk);
- __m256i mb0_1 = _mm256_shuffle_epi8(desc0_1, shuf_msk);
+ __m256i mb2_3 = _mm256_shuffle_epi8(raw_desc2_3, shuf_msk);
+ __m256i mb0_1 = _mm256_shuffle_epi8(raw_desc0_1, shuf_msk);
mb2_3 = _mm256_add_epi16(mb2_3, crc_adjust);
mb0_1 = _mm256_add_epi16(mb0_1, crc_adjust);
- /* get the packet types */
- const __m256i ptypes2_3 = _mm256_srli_epi64(desc2_3, 30);
- const __m256i ptypes0_1 = _mm256_srli_epi64(desc0_1, 30);
- const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 24);
- const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 8);
- const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 24);
- const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 8);
+ /**
+ * to get packet types, ptype is located in bit16-25
+ * of each 128bits
+ */
+ const __m256i ptypes2_3 =
+ _mm256_and_si256(raw_desc2_3, ptype_mask);
+ const __m256i ptypes0_1 =
+ _mm256_and_si256(raw_desc0_1, ptype_mask);
+ const uint16_t ptype3 = _mm256_extract_epi16(ptypes2_3, 9);
+ const uint16_t ptype2 = _mm256_extract_epi16(ptypes2_3, 1);
+ const uint16_t ptype1 = _mm256_extract_epi16(ptypes0_1, 9);
+ const uint16_t ptype0 = _mm256_extract_epi16(ptypes0_1, 1);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype3], 4);
mb2_3 = _mm256_insert_epi32(mb2_3, ptype_tbl[ptype2], 0);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype1], 4);
mb0_1 = _mm256_insert_epi32(mb0_1, ptype_tbl[ptype0], 0);
/* merge the status bits into one register */
- const __m256i status0_3 = _mm256_unpackhi_epi32(desc2_3,
- desc0_1);
+ const __m256i status0_3 = _mm256_unpackhi_epi32(raw_desc2_3,
+ raw_desc0_1);
/**
* take the two sets of status bits and merge to one
@@ -450,24 +444,24 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* get only flag/error bits we want */
const __m256i flag_bits =
_mm256_and_si256(status0_7, flags_mask);
- /* set vlan and rss flags */
- const __m256i vlan_flags =
- _mm256_shuffle_epi8(vlan_flags_shuf, flag_bits);
- const __m256i rss_flags =
- _mm256_shuffle_epi8(rss_flags_shuf,
- _mm256_srli_epi32(flag_bits, 11));
/**
* l3_l4_error flags, shuffle, then shift to correct adjustment
* of flags in flags_shuf, and finally mask out extra bits
*/
__m256i l3_l4_flags = _mm256_shuffle_epi8(l3_l4_flags_shuf,
- _mm256_srli_epi32(flag_bits, 22));
+ _mm256_srli_epi32(flag_bits, 4));
l3_l4_flags = _mm256_slli_epi32(l3_l4_flags, 1);
l3_l4_flags = _mm256_and_si256(l3_l4_flags, cksum_mask);
+ /* set rss and vlan flags */
+ const __m256i rss_vlan_flag_bits =
+ _mm256_srli_epi32(flag_bits, 12);
+ const __m256i rss_vlan_flags =
+ _mm256_shuffle_epi8(rss_vlan_flags_shuf,
+ rss_vlan_flag_bits);
/* merge flags */
const __m256i mbuf_flags = _mm256_or_si256(l3_l4_flags,
- _mm256_or_si256(rss_flags, vlan_flags));
+ rss_vlan_flags);
/**
* At this point, we have the 8 sets of flags in the low 16-bits
* of each 32-bit value in vlan0.
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [dpdk-dev] [PATCH v5 6/6] net/ice: remove Rx legacy descriptor definition
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
` (4 preceding siblings ...)
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
@ 2019-09-24 2:38 ` Leyi Rong
2019-09-24 9:05 ` [dpdk-dev] [PATCH v5 0/6] enable Rx flexible descriptor Ye Xiaolong
6 siblings, 0 replies; 54+ messages in thread
From: Leyi Rong @ 2019-09-24 2:38 UTC (permalink / raw)
To: haiyue.wang, wenzhuo.lu, qi.z.zhang, xiaolong.ye; +Cc: dev
From: Haiyue Wang <haiyue.wang@intel.com>
Since now the ice PMD only handles Rx Flex descriptor, so remove the
legacy descriptor definition.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
drivers/net/ice/ice_rxtx.c | 25 ++++++++++++-------------
drivers/net/ice/ice_rxtx.h | 4 +---
drivers/net/ice/ice_rxtx_vec_avx2.c | 5 ++---
drivers/net/ice/ice_rxtx_vec_sse.c | 4 ++--
4 files changed, 17 insertions(+), 21 deletions(-)
diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c
index 66487b990..df117fec5 100644
--- a/drivers/net/ice/ice_rxtx.c
+++ b/drivers/net/ice/ice_rxtx.c
@@ -170,7 +170,7 @@ ice_alloc_rx_queue_mbufs(struct ice_rx_queue *rxq)
uint16_t i;
for (i = 0; i < rxq->nb_rx_desc; i++) {
- volatile union ice_rx_desc *rxd;
+ volatile union ice_rx_flex_desc *rxd;
struct rte_mbuf *mbuf = rte_mbuf_raw_alloc(rxq->mp);
if (unlikely(!mbuf)) {
@@ -345,7 +345,7 @@ ice_reset_rx_queue(struct ice_rx_queue *rxq)
#endif /* RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC */
len = rxq->nb_rx_desc;
- for (i = 0; i < len * sizeof(union ice_rx_desc); i++)
+ for (i = 0; i < len * sizeof(union ice_rx_flex_desc); i++)
((volatile char *)rxq->rx_ring)[i] = 0;
#ifdef RTE_LIBRTE_ICE_RX_ALLOW_BULK_ALLOC
@@ -690,7 +690,7 @@ ice_rx_queue_setup(struct rte_eth_dev *dev,
#endif
/* Allocate the maximum number of RX ring hardware descriptor. */
- ring_size = sizeof(union ice_rx_desc) * len;
+ ring_size = sizeof(union ice_rx_flex_desc) * len;
ring_size = RTE_ALIGN(ring_size, ICE_DMA_MEM_ALIGN);
rz = rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx,
ring_size, ICE_RING_BASE_ALIGN,
@@ -1007,7 +1007,7 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
uint16_t desc = 0;
rxq = dev->data->rx_queues[rx_queue_id];
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
+ rxdp = &rxq->rx_ring[rxq->rx_tail];
while ((desc < rxq->nb_rx_desc) &&
rte_le_to_cpu_16(rxdp->wb.status_error0) &
(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S)) {
@@ -1019,8 +1019,7 @@ ice_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
desc += ICE_RXQ_SCAN_INTERVAL;
rxdp += ICE_RXQ_SCAN_INTERVAL;
if (rxq->rx_tail + desc >= rxq->nb_rx_desc)
- rxdp = (volatile union ice_rx_flex_desc *)
- &(rxq->rx_ring[rxq->rx_tail +
+ rxdp = &(rxq->rx_ring[rxq->rx_tail +
desc - rxq->nb_rx_desc]);
}
@@ -1155,7 +1154,7 @@ ice_rx_scan_hw_ring(struct ice_rx_queue *rxq)
uint64_t pkt_flags = 0;
uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[rxq->rx_tail];
+ rxdp = &rxq->rx_ring[rxq->rx_tail];
rxep = &rxq->sw_ring[rxq->rx_tail];
stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
@@ -1240,7 +1239,7 @@ ice_rx_fill_from_stage(struct ice_rx_queue *rxq,
static inline int
ice_rx_alloc_bufs(struct ice_rx_queue *rxq)
{
- volatile union ice_rx_desc *rxdp;
+ volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep;
struct rte_mbuf *mb;
uint16_t alloc_idx, i;
@@ -1375,7 +1374,7 @@ ice_recv_scattered_pkts(void *rx_queue,
uint16_t nb_pkts)
{
struct ice_rx_queue *rxq = rx_queue;
- volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
+ volatile union ice_rx_flex_desc *rx_ring = rxq->rx_ring;
volatile union ice_rx_flex_desc *rxdp;
union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
@@ -1395,7 +1394,7 @@ ice_recv_scattered_pkts(void *rx_queue,
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rxdp = &rx_ring[rx_id];
rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
@@ -1607,7 +1606,7 @@ ice_rx_descriptor_status(void *rx_queue, uint16_t offset)
if (desc >= rxq->nb_rx_desc)
desc -= rxq->nb_rx_desc;
- rxdp = (volatile union ice_rx_flex_desc *)&rxq->rx_ring[desc];
+ rxdp = &rxq->rx_ring[desc];
if (rte_le_to_cpu_16(rxdp->wb.status_error0) &
(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))
return RTE_ETH_RX_DESC_DONE;
@@ -1694,7 +1693,7 @@ ice_recv_pkts(void *rx_queue,
uint16_t nb_pkts)
{
struct ice_rx_queue *rxq = rx_queue;
- volatile union ice_rx_desc *rx_ring = rxq->rx_ring;
+ volatile union ice_rx_flex_desc *rx_ring = rxq->rx_ring;
volatile union ice_rx_flex_desc *rxdp;
union ice_rx_flex_desc rxd;
struct ice_rx_entry *sw_ring = rxq->sw_ring;
@@ -1712,7 +1711,7 @@ ice_recv_pkts(void *rx_queue,
struct rte_eth_dev *dev;
while (nb_rx < nb_pkts) {
- rxdp = (volatile union ice_rx_flex_desc *)&rx_ring[rx_id];
+ rxdp = &rx_ring[rx_id];
rx_stat_err0 = rte_le_to_cpu_16(rxdp->wb.status_error0);
/* Check the DD bit first */
diff --git a/drivers/net/ice/ice_rxtx.h b/drivers/net/ice/ice_rxtx.h
index de16637f3..25b3822df 100644
--- a/drivers/net/ice/ice_rxtx.h
+++ b/drivers/net/ice/ice_rxtx.h
@@ -21,10 +21,8 @@
#define ICE_CHK_Q_ENA_INTERVAL_US 100
#ifdef RTE_LIBRTE_ICE_16BYTE_RX_DESC
-#define ice_rx_desc ice_16byte_rx_desc
#define ice_rx_flex_desc ice_16b_rx_flex_desc
#else
-#define ice_rx_desc ice_32byte_rx_desc
#define ice_rx_flex_desc ice_32b_rx_flex_desc
#endif
@@ -48,7 +46,7 @@ struct ice_rx_entry {
struct ice_rx_queue {
struct rte_mempool *mp; /* mbuf pool to populate RX ring */
- volatile union ice_rx_desc *rx_ring;/* RX ring virtual address */
+ volatile union ice_rx_flex_desc *rx_ring;/* RX ring virtual address */
rte_iova_t rx_ring_dma; /* RX ring DMA address */
struct ice_rx_entry *sw_ring; /* address of RX soft ring */
uint16_t nb_rx_desc; /* number of RX descriptors */
diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c
index 46776fa12..f32222bb4 100644
--- a/drivers/net/ice/ice_rxtx_vec_avx2.c
+++ b/drivers/net/ice/ice_rxtx_vec_avx2.c
@@ -18,7 +18,7 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
volatile union ice_rx_flex_desc *rxdp;
struct ice_rx_entry *rxep = &rxq->sw_ring[rxq->rxrearm_start];
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -142,8 +142,7 @@ _ice_recv_raw_pkts_vec_avx2(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
const __m256i mbuf_init = _mm256_set_epi64x(0, 0,
0, rxq->mbuf_initializer);
struct ice_rx_entry *sw_ring = &rxq->sw_ring[rxq->rx_tail];
- volatile union ice_rx_flex_desc *rxdp =
- (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
+ volatile union ice_rx_flex_desc *rxdp = rxq->rx_ring + rxq->rx_tail;
const int avx_aligned = ((rxq->rx_tail & 1) == 0);
rte_prefetch0(rxdp);
diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c
index dafcb081a..2ae9370f4 100644
--- a/drivers/net/ice/ice_rxtx_vec_sse.c
+++ b/drivers/net/ice/ice_rxtx_vec_sse.c
@@ -22,7 +22,7 @@ ice_rxq_rearm(struct ice_rx_queue *rxq)
RTE_PKTMBUF_HEADROOM);
__m128i dma_addr0, dma_addr1;
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rxrearm_start;
+ rxdp = rxq->rx_ring + rxq->rxrearm_start;
/* Pull 'n' more MBUFs into the software ring */
if (rte_mempool_get_bulk(rxq->mp,
@@ -273,7 +273,7 @@ _ice_recv_raw_pkts_vec(struct ice_rx_queue *rxq, struct rte_mbuf **rx_pkts,
/* Just the act of getting into the function from the application is
* going to cost about 7 cycles
*/
- rxdp = (union ice_rx_flex_desc *)rxq->rx_ring + rxq->rx_tail;
+ rxdp = rxq->rx_ring + rxq->rx_tail;
rte_prefetch0(rxdp);
--
2.17.1
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
@ 2019-09-24 9:02 ` Ye Xiaolong
0 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-24 9:02 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/24, Leyi Rong wrote:
>From: Haiyue Wang <haiyue.wang@intel.com>
>
>The ice has the feature to extract protocol fields into flex descriptor
>by programming per queue. Currently, the ice PMD will put the protocol
>fields into rte_mbuf::udata64 with different type format. Application
>can access the protocol fields quickly.
>
>Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
>---
> doc/guides/nics/ice.rst | 101 +++++++++
> doc/guides/rel_notes/release_19_11.rst | 4 +
> drivers/net/ice/Makefile | 3 +
> drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
> drivers/net/ice/ice_ethdev.h | 4 +
> drivers/net/ice/ice_rxtx.c | 60 +++++
> drivers/net/ice/ice_rxtx.h | 2 +
> drivers/net/ice/ice_rxtx_vec_common.h | 3 +
> drivers/net/ice/meson.build | 2 +
> drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
> 10 files changed, 632 insertions(+)
> create mode 100644 drivers/net/ice/rte_pmd_ice.h
>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [dpdk-dev] [PATCH v5 0/6] enable Rx flexible descriptor
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
` (5 preceding siblings ...)
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
@ 2019-09-24 9:05 ` Ye Xiaolong
6 siblings, 0 replies; 54+ messages in thread
From: Ye Xiaolong @ 2019-09-24 9:05 UTC (permalink / raw)
To: Leyi Rong; +Cc: haiyue.wang, wenzhuo.lu, qi.z.zhang, dev
On 09/24, Leyi Rong wrote:
>This patchset enable Rx flexible descriptor for ice PMD
>in both normal path and vector path.
>Depends on shared code update patchset.
>
>---
>v5:
>- Fix some typo, and enhance the array index access handling.
>- Move update for release note to `New Features` section.
>
>v4:
>- Handle the ‘proto_xtr’ with error processing, and remove the
> Redundant default type variable.
>- Add flex descriptor and protocol extraction release note.
>- Clean up the legacy descriptor.
>
>v3:
>- Parse the ‘proto_xtr’ devargs before CVL HW initialization and
> save it for later accessing.
>- Merge shuffle ops on vlan and rss flag on vector path.
>
>v2:
>- Remove the CONFIG_RTE_LIBRTE_ICE_PROTO_XTR definition, and use the
> RTE_LIBRTE_ICE_16BYTE_RX_DESC to control the protocol extraction
> when handling Rx packets.
>
>Haiyue Wang (4):
> net/ice: add Rx flex descriptor definition
> net/ice: handle the Rx flex descriptor
> net/ice: add protocol extraction support for per Rx queue
> net/ice: remove Rx legacy descriptor definition
>
>Leyi Rong (1):
> net/ice: switch to Rx flexible descriptor in AVX path
>
>Wenzhuo Lu (1):
> net/ice: switch to flexible descriptor in SSE path
>
> doc/guides/nics/ice.rst | 101 +++++++++
> doc/guides/rel_notes/release_19_11.rst | 4 +
> drivers/net/ice/Makefile | 3 +
> drivers/net/ice/ice_ethdev.c | 301 +++++++++++++++++++++++++
> drivers/net/ice/ice_ethdev.h | 4 +
> drivers/net/ice/ice_rxtx.c | 297 +++++++++++++-----------
> drivers/net/ice/ice_rxtx.h | 8 +-
> drivers/net/ice/ice_rxtx_vec_avx2.c | 221 +++++++++---------
> drivers/net/ice/ice_rxtx_vec_common.h | 3 +
> drivers/net/ice/ice_rxtx_vec_sse.c | 235 +++++++++----------
> drivers/net/ice/meson.build | 2 +
> drivers/net/ice/rte_pmd_ice.h | 152 +++++++++++++
> 12 files changed, 961 insertions(+), 370 deletions(-)
> create mode 100644 drivers/net/ice/rte_pmd_ice.h
>
>--
>2.17.1
>
Series applied to dpdk-next-net-intel.
^ permalink raw reply [flat|nested] 54+ messages in thread
end of thread, other threads:[~2019-09-24 9:07 UTC | newest]
Thread overview: 54+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-29 2:34 [dpdk-dev] [PATCH 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 0/6] enable Rx flexible descriptor Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 4/6] net/ice: support more ptype Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-08-29 8:04 ` [dpdk-dev] [PATCH v2 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
2019-08-29 23:31 ` Zhang, Qi Z
2019-08-30 1:05 ` Wang, Haiyue
2019-08-30 1:06 ` Zhang, Qi Z
2019-08-30 6:17 ` Rong, Leyi
2019-08-29 2:34 ` [dpdk-dev] [PATCH 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 4/6] net/ice: support more ptype Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 5/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-08-29 2:34 ` [dpdk-dev] [PATCH 6/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 0/5] enable Rx flexible descriptor Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 1/5] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-18 21:56 ` Ye Xiaolong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 2/5] net/ice: handle the Rx flex descriptor Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 3/5] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-09-18 23:30 ` Ye Xiaolong
2019-09-19 1:36 ` Wang, Haiyue
2019-09-19 1:44 ` Wang, Haiyue
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 4/5] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-09-17 8:53 ` [dpdk-dev] [PATCH v3 5/5] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
2019-09-23 11:05 ` Ye Xiaolong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-09-23 3:25 ` Yang, Qiming
2019-09-23 3:34 ` Wang, Haiyue
2019-09-23 8:29 ` Ye Xiaolong
2019-09-23 11:03 ` Wang, Haiyue
2019-09-23 14:24 ` Ye Xiaolong
2019-09-23 15:00 ` Wang, Haiyue
2019-09-23 15:55 ` Ye Xiaolong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
2019-09-19 6:25 ` [dpdk-dev] [PATCH v4 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
2019-09-23 14:31 ` Ye Xiaolong
2019-09-19 6:38 ` [dpdk-dev] [PATCH v4 0/6] enable Rx flexible descriptor Zhang, Qi Z
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 " Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 1/6] net/ice: add Rx flex descriptor definition Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 2/6] net/ice: handle the Rx flex descriptor Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 3/6] net/ice: add protocol extraction support for per Rx queue Leyi Rong
2019-09-24 9:02 ` Ye Xiaolong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 4/6] net/ice: switch to flexible descriptor in SSE path Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 5/6] net/ice: switch to Rx flexible descriptor in AVX path Leyi Rong
2019-09-24 2:38 ` [dpdk-dev] [PATCH v5 6/6] net/ice: remove Rx legacy descriptor definition Leyi Rong
2019-09-24 9:05 ` [dpdk-dev] [PATCH v5 0/6] enable Rx flexible descriptor Ye Xiaolong
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.