[RFC PATCH 0/3] fm10k: enable FTAG based forwarding

[RFC PATCH 0/3] fm10k: enable FTAG based forwarding

[Wang Xiao W]

This is a RFC patch set for FTAG based forwarding feature of RRC.

Wang Xiao W (3):
  fm10k: enable FTAG based forwarding
  fm10k: add a unit test for FTAG based forwarding
  doc: add introduction for fm10k FTAG based forwarding

 app/test/Makefile                  |   1 +
 app/test/test_fm10k_ftag.c         | 253 +++++++++++++++++++++++++++++++++++++
 config/common_bsdapp               |   1 +
 config/common_linuxapp             |   1 +
 doc/guides/nics/fm10k.rst          |  13 ++
 drivers/net/fm10k/fm10k_ethdev.c   |   5 +
 drivers/net/fm10k/fm10k_rxtx.c     |  17 +++
 drivers/net/fm10k/fm10k_rxtx_vec.c |   9 ++
 8 files changed, 300 insertions(+)
 create mode 100644 app/test/test_fm10k_ftag.c

-- 
1.9.3

[RFC PATCH 1/3] fm10k: enable FTAG based forwarding

[Wang Xiao W]

This patch enables reading sglort info into mbuf for RX and inserting
an FTAG at the beginning of the packet for TX. The vlan_tci_outer field
selected from rte_mbuf structure for sglort is not used in fm10k now.
In FTAG based forwarding mode, the switch will forward packets according
to glort info in FTAG rather than mac and vlan table.

To activate this feature, user needs to turn CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD
to y in common_linuxapp or common_bsdapp. Currently this feature is supported
only on PF.

Signed-off-by: Wang Xiao W <xiao.w.wang@intel.com>
---
 config/common_bsdapp               |  1 +
 config/common_linuxapp             |  1 +
 drivers/net/fm10k/fm10k_ethdev.c   |  5 +++++
 drivers/net/fm10k/fm10k_rxtx.c     | 17 +++++++++++++++++
 drivers/net/fm10k/fm10k_rxtx_vec.c |  9 +++++++++
 5 files changed, 33 insertions(+)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index ed7c31c..451f81a 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -208,6 +208,7 @@ CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX=n
 CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX_FREE=n
 CONFIG_RTE_LIBRTE_FM10K_DEBUG_DRIVER=n
 CONFIG_RTE_LIBRTE_FM10K_RX_OLFLAGS_ENABLE=y
+CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD=n
 
 #
 # Compile burst-oriented Mellanox ConnectX-3 (MLX4) PMD
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 74bc515..c928bce 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -207,6 +207,7 @@ CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX_FREE=n
 CONFIG_RTE_LIBRTE_FM10K_DEBUG_DRIVER=n
 CONFIG_RTE_LIBRTE_FM10K_RX_OLFLAGS_ENABLE=y
 CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR=y
+CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD=n
 
 #
 # Compile burst-oriented Mellanox ConnectX-3 (MLX4) PMD
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index e4aed94..d5c376a 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -668,6 +668,11 @@ fm10k_dev_tx_init(struct rte_eth_dev *dev)
 			PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
 			return -1;
 		}
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+		/* enable use of FTAG bit in Tx descriptor, register is RO for VF */
+		if (hw->mac.type == fm10k_mac_pf)
+			FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i), FM10K_PFVTCTL_FTAG_DESC_ENABLE);
+#endif
 
 		/* set location and size for descriptor ring */
 		FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
diff --git a/drivers/net/fm10k/fm10k_rxtx.c b/drivers/net/fm10k/fm10k_rxtx.c
index e958865..f87987d 100644
--- a/drivers/net/fm10k/fm10k_rxtx.c
+++ b/drivers/net/fm10k/fm10k_rxtx.c
@@ -152,6 +152,13 @@ fm10k_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 		 */
 		mbuf->ol_flags |= PKT_RX_VLAN_PKT;
 		mbuf->vlan_tci = desc.w.vlan;
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+		/**
+		 * mbuf->vlan_tci_outer is an idle field in fm10k driver,
+		 * so it can be selected to store sglort value.
+		 */
+		mbuf->vlan_tci_outer = rte_le_to_cpu_16(desc.w.sglort);
+#endif
 
 		rx_pkts[count] = mbuf;
 		if (++next_dd == q->nb_desc) {
@@ -307,6 +314,13 @@ fm10k_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 		 */
 		mbuf->ol_flags |= PKT_RX_VLAN_PKT;
 		first_seg->vlan_tci = desc.w.vlan;
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+		/**
+		 * mbuf->vlan_tci_outer is an idle field in fm10k driver,
+		 * so it can be selected to store sglort value.
+		 */
+		first_seg->vlan_tci_outer = rte_le_to_cpu_16(desc.w.sglort);
+#endif
 
 		/* Prefetch data of first segment, if configured to do so. */
 		rte_packet_prefetch((char *)first_seg->buf_addr +
@@ -432,6 +446,9 @@ static inline void tx_xmit_pkt(struct fm10k_tx_queue *q, struct rte_mbuf *mb)
 	q->nb_free -= mb->nb_segs;
 
 	q->hw_ring[q->next_free].flags = 0;
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+	q->hw_ring[q->next_free].flags |= FM10K_TXD_FLAG_FTAG;
+#endif
 	/* set checksum flags on first descriptor of packet. SCTP checksum
 	 * offload is not supported, but we do not explicitly check for this
 	 * case in favor of greatly simplified processing. */
diff --git a/drivers/net/fm10k/fm10k_rxtx_vec.c b/drivers/net/fm10k/fm10k_rxtx_vec.c
index 2a57eef..0b0f2e3 100644
--- a/drivers/net/fm10k/fm10k_rxtx_vec.c
+++ b/drivers/net/fm10k/fm10k_rxtx_vec.c
@@ -198,7 +198,12 @@ fm10k_rx_vec_condition_check(struct rte_eth_dev *dev)
 	    rxmode->header_split == 1)
 		return -1;
 
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+	return -1;
+#else
 	return 0;
+#endif
+
 #else
 	RTE_SET_USED(dev);
 	return -1;
@@ -648,7 +653,11 @@ fm10k_tx_vec_condition_check(struct fm10k_tx_queue *txq)
 	if ((txq->txq_flags & FM10K_SIMPLE_TX_FLAG) != FM10K_SIMPLE_TX_FLAG)
 		return -1;
 
+#ifdef RTE_LIBRTE_FM10K_FTAG_FWD
+	return -1;
+#else
 	return 0;
+#endif
 }
 
 static inline void
-- 
1.9.3

[RFC PATCH 2/3] fm10k: add a unit test for FTAG based forwarding

[Wang Xiao W]

This patch adds a unit test case for FTAG functional test. Before running
the test, set PORT0_GLORT and PORT1_GLORT environment variables, and ensure
two fm10k ports are used for dpdk, glort info for each port can be shown in
TestPoint. In the unit test, a packet will be forwarded to the target port by
the switch without changing the destination mac address.

Signed-off-by: Wang Xiao W <xiao.w.wang@intel.com>
---
 app/test/Makefile          |   1 +
 app/test/test_fm10k_ftag.c | 253 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 254 insertions(+)
 create mode 100644 app/test/test_fm10k_ftag.c

diff --git a/app/test/Makefile b/app/test/Makefile
index ec33e1a..d72be8d 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -57,6 +57,7 @@ SRCS-y += test_memzone.c
 SRCS-y += test_ring.c
 SRCS-y += test_ring_perf.c
 SRCS-y += test_pmd_perf.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD) += test_fm10k_ftag.c
 
 ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y)
 SRCS-y += test_table.c
diff --git a/app/test/test_fm10k_ftag.c b/app/test/test_fm10k_ftag.c
new file mode 100644
index 0000000..325a652
--- /dev/null
+++ b/app/test/test_fm10k_ftag.c
@@ -0,0 +1,253 @@
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Intel Corporation nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include "test.h"
+
+#define RX_RING_SIZE 128
+#define TX_RING_SIZE 512
+
+#define NUM_MBUFS 8191
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+struct fm10k_ftag {
+	uint16_t swpri_type_user;
+	uint16_t vlan;
+	uint16_t sglort;
+	uint16_t dglort;
+};
+
+static const struct rte_eth_conf port_conf_default = {
+	.rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN }
+};
+
+/*
+ * Initializes a given port using global settings and with the RX buffers
+ * coming from the mbuf_pool passed as a parameter.
+ */
+static inline int
+port_init(uint8_t port, struct rte_mempool *mbuf_pool)
+{
+	struct rte_eth_conf port_conf = port_conf_default;
+	const uint16_t rx_rings = 1, tx_rings = 1;
+	int retval;
+	uint16_t q;
+
+	if (port >= rte_eth_dev_count())
+		return -1;
+
+	/* Configure the Ethernet device. */
+	retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf);
+	if (retval != 0)
+		return retval;
+
+	/* Allocate and set up 1 RX queue per Ethernet port. */
+	for (q = 0; q < rx_rings; q++) {
+		retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,
+				rte_eth_dev_socket_id(port), NULL, mbuf_pool);
+		if (retval < 0)
+			return retval;
+	}
+
+	/* Allocate and set up 1 TX queue per Ethernet port. */
+	for (q = 0; q < tx_rings; q++) {
+		retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,
+				rte_eth_dev_socket_id(port), NULL);
+		if (retval < 0)
+			return retval;
+	}
+
+	/* Start the Ethernet port. */
+	retval = rte_eth_dev_start(port);
+	if (retval < 0)
+		return retval;
+
+	/* Display the port MAC address. */
+	struct ether_addr addr;
+	rte_eth_macaddr_get(port, &addr);
+	printf("Port %u MAC: %02" PRIx8 " %02" PRIx8 " %02" PRIx8
+			   " %02" PRIx8 " %02" PRIx8 " %02" PRIx8 "\n",
+			(unsigned)port,
+			addr.addr_bytes[0], addr.addr_bytes[1],
+			addr.addr_bytes[2], addr.addr_bytes[3],
+			addr.addr_bytes[4], addr.addr_bytes[5]);
+
+	return 0;
+}
+
+static int set_glort_value(const char *str, uint16_t *glort)
+{
+	const char *glort_str;
+	char *end = NULL;
+	glort_str = getenv(str);
+	if (glort_str == NULL) {
+		printf("Please set environment value %s first\n", str);
+		return -1;
+	}
+	*glort = (uint16_t)strtoul(glort_str, &end, 16);
+	if ((glort_str[0] == '\0') || (end == NULL) || (*end != '\0')) {
+		printf("Glort value is not valid\n");
+		return -1;
+	}
+	return 0;
+}
+
+static int test_ftag_rxtx(void)
+{
+	uint8_t port = 0;
+	uint16_t glort[2];
+
+	struct rte_mbuf *bufs[BURST_SIZE];
+	uint16_t nb_rx, nb_tx, i;
+	struct fm10k_ftag *ftag_addr;
+	int ret = 0;
+
+	/* Get the glort value of the two ports */
+	if ((set_glort_value("PORT0_GLORT", &glort[0]) < 0) ||
+			(set_glort_value("PORT1_GLORT", &glort[1]) < 0))
+		return -1;
+
+	/* Receive packets coming from EPL on any of the two ports */
+	printf("Please send some packets from Ethernet port to one PEP\n");
+	do {
+		port ^= 1;
+		nb_rx = rte_eth_rx_burst(port, 0, bufs, BURST_SIZE);
+	} while (nb_rx <= 0);
+	printf("Receive %d packets on port %d\n", nb_rx, port);
+
+	/* Check sglort value on mbuf->vlan_tci_outer. */
+	for (i = 0; i < nb_rx; i++) {
+		if (bufs[i]->vlan_tci_outer == 0) {
+			printf("Find a packet with sglort 0\n");
+			return -1;
+		}
+	}
+	printf("test for FTAG RX passed\n");
+
+	/* Put an FTAG header on each of the packets received */
+	for (i = 0; i < nb_rx; i++) {
+		ftag_addr = (struct fm10k_ftag *)rte_pktmbuf_prepend(bufs[i],
+						sizeof(struct fm10k_ftag));
+		ftag_addr->swpri_type_user = 0;
+		ftag_addr->vlan = 0;
+		ftag_addr->dglort = rte_cpu_to_be_16(glort[port ^ 1]);
+		ftag_addr->sglort = rte_cpu_to_be_16(glort[port]);
+	}
+
+	/* Send packets to the other port by method of FTAG based forwarding */
+	nb_tx = rte_eth_tx_burst(port, 0, bufs, nb_rx);
+	if (nb_tx <= 0) {
+		printf("Can not send out packets with FTAG\n");
+		return -1;
+	}
+	if (unlikely(nb_tx < nb_rx)) {
+		do {
+			rte_pktmbuf_free(bufs[nb_tx]);
+		} while (++nb_tx < nb_rx);
+	}
+	printf("Send out %d packets with FTAG on port %d\n", nb_tx, port);
+
+	/* Wait enough time for a burst of packets forwarding */
+	rte_delay_us(100);
+
+	nb_rx = rte_eth_rx_burst(port ^ 1, 0, bufs, BURST_SIZE);
+	printf("Receive %d packets on port %d\n", nb_rx, port ^ 1);
+	if (nb_rx < nb_tx) {
+		printf("Packet loss happens in FTAG TX test\n");
+		ret = -1;
+	}
+
+	/* check if the sglort value is right on the other port */
+	for (i = 0; i < nb_rx; i++)	{
+		if (bufs[i]->vlan_tci_outer != glort[port]) {
+			printf("sglort of the received packet is not right\n");
+			ret = -1;
+			break;
+		}
+	}
+	for (i = 0; i < nb_rx; i++)
+		rte_pktmbuf_free(bufs[i]);
+
+	if (!ret)
+		printf("test for FTAG TX passed\n");
+	return ret;
+}
+
+static int
+test_fm10k_ftag(void)
+{
+	uint16_t nb_ports;
+	uint16_t portid;
+	struct rte_mempool *mbuf_pool;
+
+	nb_ports = rte_eth_dev_count();
+	if (nb_ports != 2) {
+		printf("2 ports needed for fm10k ftag based forwarding test\n");
+		return -1;
+	}
+
+	/* Creates a new mempool in memory to hold the mbufs. */
+	mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUFS * nb_ports,
+		MBUF_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
+
+	if (mbuf_pool == NULL)
+		rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n");
+
+	/* Initialize all ports. */
+	for (portid = 0; portid < nb_ports; portid++)
+		if (port_init(portid, mbuf_pool) != 0)
+			rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8 "\n",
+					portid);
+
+	if (test_ftag_rxtx() < 0)
+		return -1;
+
+	/* port tear down */
+	for (portid = 0; portid < nb_ports; portid++)
+		rte_eth_dev_stop(portid);
+
+	return 0;
+}
+
+static struct test_command fm10k_ftag_cmd = {
+	.command = "fm10k_ftag_autotest",
+	.callback = test_fm10k_ftag,
+};
+REGISTER_TEST_COMMAND(fm10k_ftag_cmd);
-- 
1.9.3

[RFC PATCH 3/3] doc: add introduction for fm10k FTAG based forwarding

[Wang Xiao W]

Add a brief introduction on FTAG, describes what's FTAG and how it works
in forwarding, introduction on how to run fm10k with FTAG is also included.

Signed-off-by: Wang Xiao W <xiao.w.wang@intel.com>
---
 doc/guides/nics/fm10k.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
index 4206b7f..d82bf41 100644
--- a/doc/guides/nics/fm10k.rst
+++ b/doc/guides/nics/fm10k.rst
@@ -34,6 +34,19 @@ FM10K Poll Mode Driver
 The FM10K poll mode driver library provides support for the Intel FM10000
 (FM10K) family of 40GbE/100GbE adapters.
 
+FTAG Based Forwarding of FM10K
+------------------------------
+FTAG Based Forwarding is a unique feature of FM10K. The FM10K family of NICs
+support the addition of a Fabric Tag (FTAG) to carry special information.
+The FTAG is placed at the beginning of the frame, it contains information such
+as where the packet comes from and goes, the vlan tag. In FTAG based forwarding
+mode, the switch logic forwards packets according to glort (global resource tag)
+information, other than the mac and vlan table. Now this feature works only on
+PF.
+
+To enable this feature, turn CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD to y in the
+configuration file. A unit test case fm10k_ftag_autotest is for reference, it shows
+how to read sglort info on RX and how to make an FTAG on TX.
 
 Limitations
 -----------
-- 
1.9.3

Re: [RFC PATCH 3/3] doc: add introduction for fm10k FTAG based forwarding

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wang Xiao W
> Sent: Tuesday, January 5, 2016 12:32 PM
> To: Chen, Jing D; Richardson, Bruce
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [RFC PATCH 3/3] doc: add introduction for fm10k FTAG
> based forwarding
> 
> Add a brief introduction on FTAG, describes what's FTAG and how it works
> in forwarding, introduction on how to run fm10k with FTAG is also
> included.
> 
> Signed-off-by: Wang Xiao W <xiao.w.wang@intel.com>
> ---
>  doc/guides/nics/fm10k.rst | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst index
> 4206b7f..d82bf41 100644
> --- a/doc/guides/nics/fm10k.rst
> +++ b/doc/guides/nics/fm10k.rst
> @@ -34,6 +34,19 @@ FM10K Poll Mode Driver  The FM10K poll mode driver
> library provides support for the Intel FM10000
>  (FM10K) family of 40GbE/100GbE adapters.
> 

Hi,

Some very minor comments.


> +FTAG Based Forwarding of FM10K
> +------------------------------

The Documentation Guidelines say to put a newline after section headers.

> +FTAG Based Forwarding is a unique feature of FM10K. The FM10K family of
> +NICs support the addition of a Fabric Tag (FTAG) to carry special
> information.
> +The FTAG is placed at the beginning of the frame, it contains
> +information such as where the packet comes from and goes, the vlan tag.

s/the vlan tag/and the vlan tag


> +In FTAG based forwarding mode, the switch logic forwards packets
> +according to glort (global resource tag) information, other than the

s/other/rather


> +mac and vlan table. Now this feature works only on PF.

s/Now/Currently


> +
> +To enable this feature, turn CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD to y in

In general variable and config names should be in fixed width quotes:

``CONFIG_RTE_LIBRTE_FM10K_FTAG_FWD``


> +the configuration file. A unit test case fm10k_ftag_autotest is for

s/for/provided for

John.
--