[PATCH 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[PATCH 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

This series of patches fix compilation and enable CXGBE poll mode driver for
FreeBSD.  The first patch fixes a limitation of nic_uio that only binds to
devices present in rte_pci_dev_ids.h.  The second patch does the actual
compilation fix and enabling of CXGBE PMD for FreeBSD.  The last patch updates
cxgbe documentation to reflect the FreeBSD support for CXGBE PMD.

Rahul Lakkireddy (3):
  nic_uio: Fix to allow any device to be bound to nic_uio
  cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD
  doc: Update documentation to reflect FreeBSD support for CXGBE PMD

 config/common_bsdapp                    |   2 +-
 doc/guides/nics/cxgbe.rst               | 280 ++++++++++++++++++++++++++++++--
 drivers/net/cxgbe/base/common.h         |   2 +-
 drivers/net/cxgbe/base/t4_hw.c          |   6 +-
 drivers/net/cxgbe/cxgbe_compat.h        |  12 +-
 drivers/net/cxgbe/cxgbe_main.c          |   4 +-
 drivers/net/cxgbe/sge.c                 |  10 +-
 lib/librte_eal/bsdapp/nic_uio/nic_uio.c |  46 ++----
 8 files changed, 297 insertions(+), 65 deletions(-)

-- 
2.4.1

[PATCH 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[Rahul Lakkireddy]

nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
bind the devices to nic_uio.  However, it's better to remove this whitelist of
pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow binding
any device to nic_uio.

Suggested-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
 lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 46 +++++++++------------------------
 1 file changed, 12 insertions(+), 34 deletions(-)

diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
index 2354e84..eca53de 100644
--- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
+++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
@@ -106,24 +106,6 @@ struct pci_bdf {
 	uint32_t function;
 };
 
-
-#define RTE_PCI_DEV_ID_DECL_EM(vend, dev)      {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGB(vend, dev)     {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGBVF(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBE(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBEVF(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40E(vend, dev)    {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40EVF(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VIRTIO(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VMXNET3(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_ENIC(vend, dev)    {vend, dev},
-
-const struct device devices[] = {
-#include <rte_pci_dev_ids.h>
-};
-#define NUM_DEVICES (sizeof(devices)/sizeof(devices[0]))
-
-
 static devclass_t nic_uio_devclass;
 
 DEFINE_CLASS_0(nic_uio, nic_uio_driver, nic_uio_methods, sizeof(struct nic_uio_softc));
@@ -195,10 +177,9 @@ nic_uio_probe (device_t dev)
 {
 	int i;
 
-	for (i = 0; i < NUM_DEVICES; i++)
-		if (pci_get_vendor(dev) == devices[i].vend &&
-			pci_get_device(dev) == devices[i].dev) {
-
+	for (i = 0; i < num_detached; i++)
+		if (pci_get_vendor(dev) == pci_get_vendor(detached_devices[i]) &&
+		    pci_get_device(dev) == pci_get_device(detached_devices[i])) {
 			device_set_desc(dev, "Intel(R) DPDK PCI Device");
 			return BUS_PROBE_SPECIFIC;
 		}
@@ -256,7 +237,6 @@ static void
 nic_uio_load(void)
 {
 	uint32_t bus, device, function;
-	int i;
 	device_t dev;
 	char bdf_str[256];
 	char *token, *remaining;
@@ -295,17 +275,15 @@ nic_uio_load(void)
 		if (dev == NULL)
 			continue;
 
-		for (i = 0; i < NUM_DEVICES; i++)
-			if (pci_get_vendor(dev) == devices[i].vend &&
-					pci_get_device(dev) == devices[i].dev) {
-						if (num_detached < MAX_DETACHED_DEVICES) {
-							printf("nic_uio_load: detaching and storing dev=%p\n", dev);
-							detached_devices[num_detached++] = dev;
-						} else
-							printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
-								MAX_DETACHED_DEVICES, dev);
-						device_detach(dev);
-			}
+		if (num_detached < MAX_DETACHED_DEVICES) {
+			printf("nic_uio_load: detaching and storing dev=%p\n",
+			       dev);
+			detached_devices[num_detached++] = dev;
+		} else {
+			printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
+			       MAX_DETACHED_DEVICES, dev);
+		}
+		device_detach(dev);
 	}
 }
 
-- 
2.4.1

[PATCH 2/3] cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD

[Rahul Lakkireddy]

Fix "MACRO redefined" and "function redefined" compilation errors in FreeBSD
by adding CXGBE prefix to them.  Also remove reference to a linux header
linux/if_ether.h and use DPDK macros directly.  Finally, enable CXGBE PMD for
FreeBSD.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
 config/common_bsdapp             |  2 +-
 drivers/net/cxgbe/base/common.h  |  2 +-
 drivers/net/cxgbe/base/t4_hw.c   |  6 +++---
 drivers/net/cxgbe/cxgbe_compat.h | 12 ++++++------
 drivers/net/cxgbe/cxgbe_main.c   |  4 ++--
 drivers/net/cxgbe/sge.c          | 10 +++++-----
 6 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index 4e505bf..e96e4a5 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -217,7 +217,7 @@ CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
 #
 # Compile burst-oriented Chelsio Terminator 10GbE/40GbE (CXGBE) PMD
 #
-CONFIG_RTE_LIBRTE_CXGBE_PMD=n
+CONFIG_RTE_LIBRTE_CXGBE_PMD=y
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX=n
diff --git a/drivers/net/cxgbe/base/common.h b/drivers/net/cxgbe/base/common.h
index 6ddc7d4..cf2e82d 100644
--- a/drivers/net/cxgbe/base/common.h
+++ b/drivers/net/cxgbe/base/common.h
@@ -43,7 +43,7 @@
 extern "C" {
 #endif
 
-#define PAGE_SIZE RTE_PGSIZE_4K
+#define CXGBE_PAGE_SIZE RTE_PGSIZE_4K
 
 enum {
 	MAX_NPORTS     = 4,     /* max # of ports */
diff --git a/drivers/net/cxgbe/base/t4_hw.c b/drivers/net/cxgbe/base/t4_hw.c
index c57200e..884d2cf 100644
--- a/drivers/net/cxgbe/base/t4_hw.c
+++ b/drivers/net/cxgbe/base/t4_hw.c
@@ -127,7 +127,7 @@ void t4_load_mtus(struct adapter *adap, const unsigned short *mtus,
 
 	for (i = 0; i < NMTUS; ++i) {
 		unsigned int mtu = mtus[i];
-		unsigned int log2 = fls(mtu);
+		unsigned int log2 = cxgbe_fls(mtu);
 
 		if (!(mtu & ((1 << log2) >> 2)))     /* round */
 			log2--;
@@ -1545,11 +1545,11 @@ int t4_fixup_host_params_compat(struct adapter *adap,
 				unsigned int cache_line_size,
 				enum chip_type chip_compat)
 {
-	unsigned int page_shift = fls(page_size) - 1;
+	unsigned int page_shift = cxgbe_fls(page_size) - 1;
 	unsigned int sge_hps = page_shift - 10;
 	unsigned int stat_len = cache_line_size > 64 ? 128 : 64;
 	unsigned int fl_align = cache_line_size < 32 ? 32 : cache_line_size;
-	unsigned int fl_align_log = fls(fl_align) - 1;
+	unsigned int fl_align_log = cxgbe_fls(fl_align) - 1;
 
 	t4_write_reg(adap, A_SGE_HOST_PAGE_SIZE,
 		     V_HOSTPAGESIZEPF0(sge_hps) |
diff --git a/drivers/net/cxgbe/cxgbe_compat.h b/drivers/net/cxgbe/cxgbe_compat.h
index 3b871ee..e68f8f5 100644
--- a/drivers/net/cxgbe/cxgbe_compat.h
+++ b/drivers/net/cxgbe/cxgbe_compat.h
@@ -119,8 +119,8 @@
 #define L1_CACHE_BYTES  BIT(L1_CACHE_SHIFT)
 
 #define PAGE_SHIFT  12
-#define ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
-#define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a)))
+#define CXGBE_ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define PTR_ALIGN(p, a) ((typeof(p))CXGBE_ALIGN((unsigned long)(p), (a)))
 
 #define VLAN_HLEN 4
 
@@ -178,7 +178,7 @@ typedef uint64_t  dma_addr_t;
 /*
  * round up val _p to a power of 2 size _s
  */
-#define roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
+#define cxgbe_roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
 
 #undef container_of
 #define container_of(ptr, type, member) ({ \
@@ -214,13 +214,13 @@ static inline uint8_t hweight32(uint32_t word32)
 } /* weight32 */
 
 /**
- * fls - find last (most-significant) bit set
+ * cxgbe_fls - find last (most-significant) bit set
  * @x: the word to search
  *
  * This is defined the same way as ffs.
- * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ * Note cxgbe_fls(0) = 0, cxgbe_fls(1) = 1, cxgbe_fls(0x80000000) = 32.
  */
-static inline int fls(int x)
+static inline int cxgbe_fls(int x)
 {
 	return x ? sizeof(x) * 8 - __builtin_clz(x) : 0;
 }
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c
index b879820..3755444 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -411,7 +411,7 @@ static int adap_init0_tweaks(struct adapter *adapter)
 	 * Line Size, etc.  The firmware default is for a 4KB Page Size and
 	 * 64B Cache Line Size ...
 	 */
-	t4_fixup_host_params_compat(adapter, PAGE_SIZE, L1_CACHE_BYTES,
+	t4_fixup_host_params_compat(adapter, CXGBE_PAGE_SIZE, L1_CACHE_BYTES,
 				    T5_LAST_REV);
 
 	/*
@@ -1100,7 +1100,7 @@ int cxgbe_probe(struct adapter *adapter)
 		qpp = 1 << ((t4_read_reg(adapter,
 				A_SGE_EGRESS_QUEUES_PER_PAGE_PF) >> s_qpp)
 				& M_QUEUESPERPAGEPF0);
-		num_seg = PAGE_SIZE / UDBS_SEG_SIZE;
+		num_seg = CXGBE_PAGE_SIZE / UDBS_SEG_SIZE;
 		if (qpp > num_seg)
 			dev_warn(adapter, "Incorrect SGE EGRESS QUEUES_PER_PAGE configuration, continuing in debug mode\n");
 
diff --git a/drivers/net/cxgbe/sge.c b/drivers/net/cxgbe/sge.c
index 1a10278..d570d33 100644
--- a/drivers/net/cxgbe/sge.c
+++ b/drivers/net/cxgbe/sge.c
@@ -31,7 +31,6 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#include <linux/if_ether.h>
 #include <sys/queue.h>
 #include <stdio.h>
 #include <errno.h>
@@ -98,7 +97,8 @@ static inline unsigned int fl_mtu_bufsize(struct adapter *adapter,
 {
 	struct sge *s = &adapter->sge;
 
-	return ALIGN(s->pktshift + ETH_HLEN + VLAN_HLEN + mtu, s->fl_align);
+	return CXGBE_ALIGN(s->pktshift + ETHER_HDR_LEN + VLAN_HLEN + mtu,
+			   s->fl_align);
 }
 
 #define FL_MTU_SMALL_BUFSIZE(adapter) fl_mtu_bufsize(adapter, FL_MTU_SMALL)
@@ -1578,7 +1578,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 	unsigned int nb_refill;
 
 	/* Size needs to be multiple of 16, including status entry. */
-	iq->size = roundup(iq->size, 16);
+	iq->size = cxgbe_roundup(iq->size, 16);
 
 	snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 		 eth_dev->driver->pci_drv.name, fwevtq ? "fwq_ring" : "rx_ring",
@@ -1630,7 +1630,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 		 */
 		if (fl->size < s->fl_starve_thres - 1 + 2 * 8)
 			fl->size = s->fl_starve_thres - 1 + 2 * 8;
-		fl->size = roundup(fl->size, 8);
+		fl->size = cxgbe_roundup(fl->size, 8);
 
 		snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 			 eth_dev->driver->pci_drv.name,
@@ -2064,7 +2064,7 @@ static int t4_sge_init_soft(struct adapter *adap)
 	 * The Page Size Buffer must be exactly equal to our Page Size and the
 	 * Large Page Size Buffer should be 0 (per above) or a power of 2.
 	 */
-	if (fl_small_pg != PAGE_SIZE ||
+	if (fl_small_pg != CXGBE_PAGE_SIZE ||
 	    (fl_large_pg & (fl_large_pg - 1)) != 0) {
 		dev_err(adap, "bad SGE FL page buffer sizes [%d, %d]\n",
 			fl_small_pg, fl_large_pg);
-- 
2.4.1

[PATCH 3/3] doc: Update documentation to reflect FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

Update cxgbe documentation to include support for FreeBSD:
1. Add instructions on how to compile CXGBE PMD in FreeBSD.
2. Add instructions on how to flash firmware image on Chelsio T5 cards in
   FreeBSD.
3. Add sample application usage for FreeBSD.
4. Add an extra step to reload kernel module in Linux in order for the new
   firmware to be loaded.
5. Typo fixes.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
 doc/guides/nics/cxgbe.rst | 280 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 267 insertions(+), 13 deletions(-)

diff --git a/doc/guides/nics/cxgbe.rst b/doc/guides/nics/cxgbe.rst
index 7557db0..148cd25 100644
--- a/doc/guides/nics/cxgbe.rst
+++ b/doc/guides/nics/cxgbe.rst
@@ -32,7 +32,8 @@ CXGBE Poll Mode Driver
 ======================
 
 The CXGBE PMD (**librte_pmd_cxgbe**) provides poll mode driver support
-for **Chelsio T5** 10/40 Gbps family of adapters.
+for **Chelsio T5** 10/40 Gbps family of adapters. CXGBE PMD has support
+for the latest Linux and FreeBSD operating systems.
 
 More information can be found at `Chelsio Communications Official Website
 <http://www.chelsio.com>`_.
@@ -73,10 +74,20 @@ Prerequisites
 
 - Requires firmware version **1.13.32.0** and higher. Visit
   `Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware
-  bundled with the latest Chelsio Unified Wire package. Installing and loading
-  the latest cxgb4 kernel driver from the Chelsio Unified Wire package should
-  get you the latest firmware. More information can be obtained from the User
-  Guide that is bundled with the Chelsio Unified Wire package.
+  bundled with the latest Chelsio Unified Wire package.
+
+  For Linux, installing and loading the latest cxgb4 kernel driver from the
+  Chelsio Unified Wire package should get you the latest firmware. More
+  information can be obtained from the User Guide that is bundled with the
+  Chelsio Unified Wire package.
+
+  For FreeBSD, the latest firmware obtained from the Chelsio Unified Wire
+  package must be manually flashed via cxgbetool available in FreeBSD source
+  repository.
+
+  Instructions on how to manually flash the firmware are given in section
+  :ref:`linux-installation` for Linux and section :ref:`freebsd-installation`
+  for FreeBSD.
 
 Pre-Installation Configuration
 ------------------------------
@@ -111,6 +122,8 @@ enabling debugging options may affect system performance.
 
   Toggle display of receiving data path run-time check messages.
 
+.. _driver-compilation:
+
 Driver Compilation
 ~~~~~~~~~~~~~~~~~~
 
@@ -122,11 +135,24 @@ command:
    cd <DPDK-source-directory>
    make config T=x86_64-native-linuxapp-gcc install
 
-Installation
-------------
+To compile CXGBE PMD for FreeBSD x86_64 clang target, run the following "gmake"
+command:
 
-Steps to manually install the latest firmware from the downloaded source files
-are as follows:
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   gmake config T=x86_64-native-bsdapp-clang install
+
+Linux
+-----
+
+.. _linux-installation:
+
+Linux Installation
+~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for Linux operating system are as follows:
 
 #. Load the kernel module:
 
@@ -166,6 +192,13 @@ are as follows:
 
       cxgbtool p1p1 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
 
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      modprobe -r cxgb4
+      modprobe cxgb4
+
 #. Verify with ethtool:
 
    .. code-block:: console
@@ -179,10 +212,17 @@ are as follows:
       firmware-version: 1.13.32.0, TP 0.1.4.8
 
 Sample Application Notes
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~
 
 This section demonstrates how to launch **testpmd** with Chelsio T5
-devices managed by librte_pmd_cxgbe.
+devices managed by librte_pmd_cxgbe in Linux operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
 
 #. Load the kernel module:
 
@@ -253,7 +293,7 @@ devices managed by librte_pmd_cxgbe.
 
    .. note::
 
-      CXGBE PMD currently only supports the binding of PF4 for Chelsio T5 NICs.
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
 
 #. Start testpmd with basic parameters:
 
@@ -271,7 +311,221 @@ devices managed by librte_pmd_cxgbe.
       EAL:   PCI memory mapped at 0x7fd7c0200000
       EAL:   PCI memory mapped at 0x7fd77cdfd000
       EAL:   PCI memory mapped at 0x7fd7c10b7000
-      PMD: rte_cxgbe_pmd: fw: 1.13.33.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
+      Interactive-mode selected
+      Configuring Port 0 (socket 0)
+      Port 0: 00:07:43:2D:EA:C0
+      Configuring Port 1 (socket 0)
+      Port 1: 00:07:43:2D:EA:C8
+      Checking link statuses...
+      PMD: rte_cxgbe_pmd: Port0: passive DA port module inserted
+      PMD: rte_cxgbe_pmd: Port1: passive DA port module inserted
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+.. note::
+
+   Flow control pause TX/RX is disabled by default and can be enabled via
+   testpmd as follows:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 0
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 1
+
+   To disable again, use:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 0
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 1
+
+FreeBSD
+-------
+
+.. _freebsd-installation:
+
+FreeBSD Installation
+~~~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for FreeBSD operating system are as follows:
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Use dmesg to get the t5nex instance assigned to the Chelsio card:
+
+   .. code-block:: console
+
+      dmesg | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0: <Chelsio T520-CR> irq 16 at device 0.4 on pci2
+      cxl0: <port 0> on t5nex0
+      cxl1: <port 1> on t5nex0
+      t5nex0: PCIe x8, 2 ports, 14 MSI-X interrupts, 31 eq, 13 iq
+
+   In the example above, a Chelsio T520-CR card is bound to a t5nex0 instance.
+
+#. Install cxgbetool from FreeBSD source repository:
+
+   .. code-block:: console
+
+      cd <path_to_FreeBSD_source>/tools/tools/cxgbetool/
+      make && make install
+
+#. Use cxgbetool to load the firmware image onto the card:
+
+   .. code-block:: console
+
+      cxgbetool t5nex0 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
+
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+      kldload if_cxgbe
+
+#. Verify with sysctl:
+
+   .. code-block:: console
+
+      sysctl -a | grep "t5nex" | grep "firmware"
+
+   Example output:
+
+   .. code-block:: console
+
+      dev.t5nex.0.firmware_version: 1.13.32.0
+
+Sample Application Notes
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+This section demonstrates how to launch **testpmd** with Chelsio T5
+devices managed by librte_pmd_cxgbe in FreeBSD operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
+
+#. Copy the contigmem kernel module to /boot/kernel directory:
+
+   .. code-block:: console
+
+      cp x86_64-native-bsdapp-clang/kmod/contigmem.ko /boot/kernel/
+
+#. Add the following lines to /boot/loader.conf:
+
+   .. code-block:: console
+
+      # reserve 2 x 1G blocks of contiguous memory using contigmem driver
+      hw.contigmem.num_buffers=2
+      hw.contigmem.buffer_size=1073741824
+      # load contigmem module during boot process
+      contigmem_load="YES"
+
+   The above lines load the contigmem kernel module during boot process and
+   allocate 2 x 1G blocks of contiguous memory to be used for DPDK later on.
+   This is to avoid issues with potential memory fragmentation during later
+   system up time, which may result in failure of allocating the contiguous
+   memory required for the contigmem kernel module.
+
+#. Restart the system and ensure the contigmem module is loaded successfully:
+
+   .. code-block:: console
+
+      reboot
+      kldstat | grep "contigmem"
+
+   Example output:
+
+   .. code-block:: console
+
+      2    1 0xffffffff817f1000 3118     contigmem.ko
+
+#. Repeat step 1 to ensure that you are in the DPDK source directory.
+
+#. Load the cxgbe kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Get the PCI bus addresses of the interfaces bound to t5nex driver:
+
+   .. code-block:: console
+
+      pciconf -l | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0@pci0:2:0:4: class=0x020000 card=0x00001425 chip=0x54011425 rev=0x00
+
+   In the above example, the t5nex0 is bound to 2:0:4 bus address.
+
+   .. note::
+
+      Both the interfaces of a Chelsio T5 2-port adapter are bound to the
+      same PCI bus address.
+
+#. Unload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+
+#. Set the PCI bus addresses to hw.nic_uio.bdfs kernel environment parameter:
+
+   .. code-block:: console
+
+      kenv hw.nic_uio.bdfs="2:0:4"
+
+   This automatically binds 2:0:4 to nic_uio kernel driver when it is loaded in
+   the next step.
+
+   .. note::
+
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
+
+#. Load nic_uio kernel driver:
+
+   .. code-block:: console
+
+      kldload ./x86_64-native-bsdapp-clang/kmod/nic_uio.ko
+
+#. Start testpmd with basic parameters:
+
+   .. code-block:: console
+
+      ./x86_64-native-bsdapp-clang/app/testpmd -c 0xf -n 4 -w 0000:02:00.4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:02:00.4 on NUMA socket 0
+      EAL:   probe driver: 1425:5401 rte_cxgbe_pmd
+      EAL:   PCI memory mapped at 0x8007ec000
+      EAL:   PCI memory mapped at 0x842800000
+      EAL:   PCI memory mapped at 0x80086c000
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
       PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
       Interactive-mode selected
       Configuring Port 0 (socket 0)
-- 
2.4.1

Re: [PATCH 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[Thomas Monjalon]

2015-07-17 23:50, Rahul Lakkireddy:
> nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
> bind the devices to nic_uio.  However, it's better to remove this whitelist of
> pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow binding
> any device to nic_uio.
[...]
> -	for (i = 0; i < NUM_DEVICES; i++)
> -		if (pci_get_vendor(dev) == devices[i].vend &&
> -			pci_get_device(dev) == devices[i].dev) {
> -
> +	for (i = 0; i < num_detached; i++)
> +		if (pci_get_vendor(dev) == pci_get_vendor(detached_devices[i]) &&
> +		    pci_get_device(dev) == pci_get_device(detached_devices[i])) {
>  			device_set_desc(dev, "Intel(R) DPDK PCI Device");

This is an old name. You can now rename "Intel(R) DPDK" to "DPDK".

[PATCH v2 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

This series of patches fix compilation and enable CXGBE poll mode driver for
FreeBSD.  The first patch fixes a limitation of nic_uio that only binds to
devices present in rte_pci_dev_ids.h.  The second patch does the actual
compilation fix and enabling of CXGBE PMD for FreeBSD.  The last patch updates
cxgbe documentation to reflect the FreeBSD support for CXGBE PMD.

v2:
- Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.

Rahul Lakkireddy (3):
  nic_uio: Fix to allow any device to be bound to nic_uio
  cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD
  doc: Update documentation to reflect FreeBSD support for CXGBE PMD

 config/common_bsdapp                    |   2 +-
 doc/guides/nics/cxgbe.rst               | 280 ++++++++++++++++++++++++++++++--
 drivers/net/cxgbe/base/common.h         |   2 +-
 drivers/net/cxgbe/base/t4_hw.c          |   6 +-
 drivers/net/cxgbe/cxgbe_compat.h        |  12 +-
 drivers/net/cxgbe/cxgbe_main.c          |   4 +-
 drivers/net/cxgbe/sge.c                 |  10 +-
 lib/librte_eal/bsdapp/nic_uio/nic_uio.c |  48 ++----
 8 files changed, 298 insertions(+), 66 deletions(-)

-- 
2.4.1

[PATCH v2 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[Rahul Lakkireddy]

nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
bind the devices to nic_uio.  However, it's better to remove this whitelist of
pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow binding
any device to nic_uio.

Suggested-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v2:
- Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.

 lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 48 +++++++++------------------------
 1 file changed, 13 insertions(+), 35 deletions(-)

diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
index 2354e84..f868dc8 100644
--- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
+++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
@@ -106,24 +106,6 @@ struct pci_bdf {
 	uint32_t function;
 };
 
-
-#define RTE_PCI_DEV_ID_DECL_EM(vend, dev)      {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGB(vend, dev)     {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGBVF(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBE(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBEVF(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40E(vend, dev)    {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40EVF(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VIRTIO(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VMXNET3(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_ENIC(vend, dev)    {vend, dev},
-
-const struct device devices[] = {
-#include <rte_pci_dev_ids.h>
-};
-#define NUM_DEVICES (sizeof(devices)/sizeof(devices[0]))
-
-
 static devclass_t nic_uio_devclass;
 
 DEFINE_CLASS_0(nic_uio, nic_uio_driver, nic_uio_methods, sizeof(struct nic_uio_softc));
@@ -195,11 +177,10 @@ nic_uio_probe (device_t dev)
 {
 	int i;
 
-	for (i = 0; i < NUM_DEVICES; i++)
-		if (pci_get_vendor(dev) == devices[i].vend &&
-			pci_get_device(dev) == devices[i].dev) {
-
-			device_set_desc(dev, "Intel(R) DPDK PCI Device");
+	for (i = 0; i < num_detached; i++)
+		if (pci_get_vendor(dev) == pci_get_vendor(detached_devices[i]) &&
+		    pci_get_device(dev) == pci_get_device(detached_devices[i])) {
+			device_set_desc(dev, "DPDK PCI Device");
 			return BUS_PROBE_SPECIFIC;
 		}
 
@@ -256,7 +237,6 @@ static void
 nic_uio_load(void)
 {
 	uint32_t bus, device, function;
-	int i;
 	device_t dev;
 	char bdf_str[256];
 	char *token, *remaining;
@@ -295,17 +275,15 @@ nic_uio_load(void)
 		if (dev == NULL)
 			continue;
 
-		for (i = 0; i < NUM_DEVICES; i++)
-			if (pci_get_vendor(dev) == devices[i].vend &&
-					pci_get_device(dev) == devices[i].dev) {
-						if (num_detached < MAX_DETACHED_DEVICES) {
-							printf("nic_uio_load: detaching and storing dev=%p\n", dev);
-							detached_devices[num_detached++] = dev;
-						} else
-							printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
-								MAX_DETACHED_DEVICES, dev);
-						device_detach(dev);
-			}
+		if (num_detached < MAX_DETACHED_DEVICES) {
+			printf("nic_uio_load: detaching and storing dev=%p\n",
+			       dev);
+			detached_devices[num_detached++] = dev;
+		} else {
+			printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
+			       MAX_DETACHED_DEVICES, dev);
+		}
+		device_detach(dev);
 	}
 }
 
-- 
2.4.1

[PATCH v2 2/3] cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD

[Rahul Lakkireddy]

Fix "MACRO redefined" and "function redefined" compilation errors in FreeBSD
by adding CXGBE prefix to them.  Also remove reference to a linux header
linux/if_ether.h and use DPDK macros directly.  Finally, enable CXGBE PMD for
FreeBSD.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v2:
- No changes.

 config/common_bsdapp             |  2 +-
 drivers/net/cxgbe/base/common.h  |  2 +-
 drivers/net/cxgbe/base/t4_hw.c   |  6 +++---
 drivers/net/cxgbe/cxgbe_compat.h | 12 ++++++------
 drivers/net/cxgbe/cxgbe_main.c   |  4 ++--
 drivers/net/cxgbe/sge.c          | 10 +++++-----
 6 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index 4e505bf..e96e4a5 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -217,7 +217,7 @@ CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
 #
 # Compile burst-oriented Chelsio Terminator 10GbE/40GbE (CXGBE) PMD
 #
-CONFIG_RTE_LIBRTE_CXGBE_PMD=n
+CONFIG_RTE_LIBRTE_CXGBE_PMD=y
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX=n
diff --git a/drivers/net/cxgbe/base/common.h b/drivers/net/cxgbe/base/common.h
index 6ddc7d4..cf2e82d 100644
--- a/drivers/net/cxgbe/base/common.h
+++ b/drivers/net/cxgbe/base/common.h
@@ -43,7 +43,7 @@
 extern "C" {
 #endif
 
-#define PAGE_SIZE RTE_PGSIZE_4K
+#define CXGBE_PAGE_SIZE RTE_PGSIZE_4K
 
 enum {
 	MAX_NPORTS     = 4,     /* max # of ports */
diff --git a/drivers/net/cxgbe/base/t4_hw.c b/drivers/net/cxgbe/base/t4_hw.c
index c57200e..884d2cf 100644
--- a/drivers/net/cxgbe/base/t4_hw.c
+++ b/drivers/net/cxgbe/base/t4_hw.c
@@ -127,7 +127,7 @@ void t4_load_mtus(struct adapter *adap, const unsigned short *mtus,
 
 	for (i = 0; i < NMTUS; ++i) {
 		unsigned int mtu = mtus[i];
-		unsigned int log2 = fls(mtu);
+		unsigned int log2 = cxgbe_fls(mtu);
 
 		if (!(mtu & ((1 << log2) >> 2)))     /* round */
 			log2--;
@@ -1545,11 +1545,11 @@ int t4_fixup_host_params_compat(struct adapter *adap,
 				unsigned int cache_line_size,
 				enum chip_type chip_compat)
 {
-	unsigned int page_shift = fls(page_size) - 1;
+	unsigned int page_shift = cxgbe_fls(page_size) - 1;
 	unsigned int sge_hps = page_shift - 10;
 	unsigned int stat_len = cache_line_size > 64 ? 128 : 64;
 	unsigned int fl_align = cache_line_size < 32 ? 32 : cache_line_size;
-	unsigned int fl_align_log = fls(fl_align) - 1;
+	unsigned int fl_align_log = cxgbe_fls(fl_align) - 1;
 
 	t4_write_reg(adap, A_SGE_HOST_PAGE_SIZE,
 		     V_HOSTPAGESIZEPF0(sge_hps) |
diff --git a/drivers/net/cxgbe/cxgbe_compat.h b/drivers/net/cxgbe/cxgbe_compat.h
index 3b871ee..e68f8f5 100644
--- a/drivers/net/cxgbe/cxgbe_compat.h
+++ b/drivers/net/cxgbe/cxgbe_compat.h
@@ -119,8 +119,8 @@
 #define L1_CACHE_BYTES  BIT(L1_CACHE_SHIFT)
 
 #define PAGE_SHIFT  12
-#define ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
-#define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a)))
+#define CXGBE_ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define PTR_ALIGN(p, a) ((typeof(p))CXGBE_ALIGN((unsigned long)(p), (a)))
 
 #define VLAN_HLEN 4
 
@@ -178,7 +178,7 @@ typedef uint64_t  dma_addr_t;
 /*
  * round up val _p to a power of 2 size _s
  */
-#define roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
+#define cxgbe_roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
 
 #undef container_of
 #define container_of(ptr, type, member) ({ \
@@ -214,13 +214,13 @@ static inline uint8_t hweight32(uint32_t word32)
 } /* weight32 */
 
 /**
- * fls - find last (most-significant) bit set
+ * cxgbe_fls - find last (most-significant) bit set
  * @x: the word to search
  *
  * This is defined the same way as ffs.
- * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ * Note cxgbe_fls(0) = 0, cxgbe_fls(1) = 1, cxgbe_fls(0x80000000) = 32.
  */
-static inline int fls(int x)
+static inline int cxgbe_fls(int x)
 {
 	return x ? sizeof(x) * 8 - __builtin_clz(x) : 0;
 }
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c
index b879820..3755444 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -411,7 +411,7 @@ static int adap_init0_tweaks(struct adapter *adapter)
 	 * Line Size, etc.  The firmware default is for a 4KB Page Size and
 	 * 64B Cache Line Size ...
 	 */
-	t4_fixup_host_params_compat(adapter, PAGE_SIZE, L1_CACHE_BYTES,
+	t4_fixup_host_params_compat(adapter, CXGBE_PAGE_SIZE, L1_CACHE_BYTES,
 				    T5_LAST_REV);
 
 	/*
@@ -1100,7 +1100,7 @@ int cxgbe_probe(struct adapter *adapter)
 		qpp = 1 << ((t4_read_reg(adapter,
 				A_SGE_EGRESS_QUEUES_PER_PAGE_PF) >> s_qpp)
 				& M_QUEUESPERPAGEPF0);
-		num_seg = PAGE_SIZE / UDBS_SEG_SIZE;
+		num_seg = CXGBE_PAGE_SIZE / UDBS_SEG_SIZE;
 		if (qpp > num_seg)
 			dev_warn(adapter, "Incorrect SGE EGRESS QUEUES_PER_PAGE configuration, continuing in debug mode\n");
 
diff --git a/drivers/net/cxgbe/sge.c b/drivers/net/cxgbe/sge.c
index 1a10278..d570d33 100644
--- a/drivers/net/cxgbe/sge.c
+++ b/drivers/net/cxgbe/sge.c
@@ -31,7 +31,6 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#include <linux/if_ether.h>
 #include <sys/queue.h>
 #include <stdio.h>
 #include <errno.h>
@@ -98,7 +97,8 @@ static inline unsigned int fl_mtu_bufsize(struct adapter *adapter,
 {
 	struct sge *s = &adapter->sge;
 
-	return ALIGN(s->pktshift + ETH_HLEN + VLAN_HLEN + mtu, s->fl_align);
+	return CXGBE_ALIGN(s->pktshift + ETHER_HDR_LEN + VLAN_HLEN + mtu,
+			   s->fl_align);
 }
 
 #define FL_MTU_SMALL_BUFSIZE(adapter) fl_mtu_bufsize(adapter, FL_MTU_SMALL)
@@ -1578,7 +1578,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 	unsigned int nb_refill;
 
 	/* Size needs to be multiple of 16, including status entry. */
-	iq->size = roundup(iq->size, 16);
+	iq->size = cxgbe_roundup(iq->size, 16);
 
 	snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 		 eth_dev->driver->pci_drv.name, fwevtq ? "fwq_ring" : "rx_ring",
@@ -1630,7 +1630,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 		 */
 		if (fl->size < s->fl_starve_thres - 1 + 2 * 8)
 			fl->size = s->fl_starve_thres - 1 + 2 * 8;
-		fl->size = roundup(fl->size, 8);
+		fl->size = cxgbe_roundup(fl->size, 8);
 
 		snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 			 eth_dev->driver->pci_drv.name,
@@ -2064,7 +2064,7 @@ static int t4_sge_init_soft(struct adapter *adap)
 	 * The Page Size Buffer must be exactly equal to our Page Size and the
 	 * Large Page Size Buffer should be 0 (per above) or a power of 2.
 	 */
-	if (fl_small_pg != PAGE_SIZE ||
+	if (fl_small_pg != CXGBE_PAGE_SIZE ||
 	    (fl_large_pg & (fl_large_pg - 1)) != 0) {
 		dev_err(adap, "bad SGE FL page buffer sizes [%d, %d]\n",
 			fl_small_pg, fl_large_pg);
-- 
2.4.1

[PATCH v2 3/3] doc: Update documentation to reflect FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

Update cxgbe documentation to include support for FreeBSD:
1. Add instructions on how to compile CXGBE PMD in FreeBSD.
2. Add instructions on how to flash firmware image on Chelsio T5 cards in
   FreeBSD.
3. Add sample application usage for FreeBSD.
4. Add an extra step to reload kernel module in Linux in order for the new
   firmware to be loaded.
5. Typo fixes.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v2:
- No changes.

 doc/guides/nics/cxgbe.rst | 280 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 267 insertions(+), 13 deletions(-)

diff --git a/doc/guides/nics/cxgbe.rst b/doc/guides/nics/cxgbe.rst
index 7557db0..148cd25 100644
--- a/doc/guides/nics/cxgbe.rst
+++ b/doc/guides/nics/cxgbe.rst
@@ -32,7 +32,8 @@ CXGBE Poll Mode Driver
 ======================
 
 The CXGBE PMD (**librte_pmd_cxgbe**) provides poll mode driver support
-for **Chelsio T5** 10/40 Gbps family of adapters.
+for **Chelsio T5** 10/40 Gbps family of adapters. CXGBE PMD has support
+for the latest Linux and FreeBSD operating systems.
 
 More information can be found at `Chelsio Communications Official Website
 <http://www.chelsio.com>`_.
@@ -73,10 +74,20 @@ Prerequisites
 
 - Requires firmware version **1.13.32.0** and higher. Visit
   `Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware
-  bundled with the latest Chelsio Unified Wire package. Installing and loading
-  the latest cxgb4 kernel driver from the Chelsio Unified Wire package should
-  get you the latest firmware. More information can be obtained from the User
-  Guide that is bundled with the Chelsio Unified Wire package.
+  bundled with the latest Chelsio Unified Wire package.
+
+  For Linux, installing and loading the latest cxgb4 kernel driver from the
+  Chelsio Unified Wire package should get you the latest firmware. More
+  information can be obtained from the User Guide that is bundled with the
+  Chelsio Unified Wire package.
+
+  For FreeBSD, the latest firmware obtained from the Chelsio Unified Wire
+  package must be manually flashed via cxgbetool available in FreeBSD source
+  repository.
+
+  Instructions on how to manually flash the firmware are given in section
+  :ref:`linux-installation` for Linux and section :ref:`freebsd-installation`
+  for FreeBSD.
 
 Pre-Installation Configuration
 ------------------------------
@@ -111,6 +122,8 @@ enabling debugging options may affect system performance.
 
   Toggle display of receiving data path run-time check messages.
 
+.. _driver-compilation:
+
 Driver Compilation
 ~~~~~~~~~~~~~~~~~~
 
@@ -122,11 +135,24 @@ command:
    cd <DPDK-source-directory>
    make config T=x86_64-native-linuxapp-gcc install
 
-Installation
-------------
+To compile CXGBE PMD for FreeBSD x86_64 clang target, run the following "gmake"
+command:
 
-Steps to manually install the latest firmware from the downloaded source files
-are as follows:
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   gmake config T=x86_64-native-bsdapp-clang install
+
+Linux
+-----
+
+.. _linux-installation:
+
+Linux Installation
+~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for Linux operating system are as follows:
 
 #. Load the kernel module:
 
@@ -166,6 +192,13 @@ are as follows:
 
       cxgbtool p1p1 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
 
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      modprobe -r cxgb4
+      modprobe cxgb4
+
 #. Verify with ethtool:
 
    .. code-block:: console
@@ -179,10 +212,17 @@ are as follows:
       firmware-version: 1.13.32.0, TP 0.1.4.8
 
 Sample Application Notes
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~
 
 This section demonstrates how to launch **testpmd** with Chelsio T5
-devices managed by librte_pmd_cxgbe.
+devices managed by librte_pmd_cxgbe in Linux operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
 
 #. Load the kernel module:
 
@@ -253,7 +293,7 @@ devices managed by librte_pmd_cxgbe.
 
    .. note::
 
-      CXGBE PMD currently only supports the binding of PF4 for Chelsio T5 NICs.
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
 
 #. Start testpmd with basic parameters:
 
@@ -271,7 +311,221 @@ devices managed by librte_pmd_cxgbe.
       EAL:   PCI memory mapped at 0x7fd7c0200000
       EAL:   PCI memory mapped at 0x7fd77cdfd000
       EAL:   PCI memory mapped at 0x7fd7c10b7000
-      PMD: rte_cxgbe_pmd: fw: 1.13.33.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
+      Interactive-mode selected
+      Configuring Port 0 (socket 0)
+      Port 0: 00:07:43:2D:EA:C0
+      Configuring Port 1 (socket 0)
+      Port 1: 00:07:43:2D:EA:C8
+      Checking link statuses...
+      PMD: rte_cxgbe_pmd: Port0: passive DA port module inserted
+      PMD: rte_cxgbe_pmd: Port1: passive DA port module inserted
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+.. note::
+
+   Flow control pause TX/RX is disabled by default and can be enabled via
+   testpmd as follows:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 0
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 1
+
+   To disable again, use:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 0
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 1
+
+FreeBSD
+-------
+
+.. _freebsd-installation:
+
+FreeBSD Installation
+~~~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for FreeBSD operating system are as follows:
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Use dmesg to get the t5nex instance assigned to the Chelsio card:
+
+   .. code-block:: console
+
+      dmesg | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0: <Chelsio T520-CR> irq 16 at device 0.4 on pci2
+      cxl0: <port 0> on t5nex0
+      cxl1: <port 1> on t5nex0
+      t5nex0: PCIe x8, 2 ports, 14 MSI-X interrupts, 31 eq, 13 iq
+
+   In the example above, a Chelsio T520-CR card is bound to a t5nex0 instance.
+
+#. Install cxgbetool from FreeBSD source repository:
+
+   .. code-block:: console
+
+      cd <path_to_FreeBSD_source>/tools/tools/cxgbetool/
+      make && make install
+
+#. Use cxgbetool to load the firmware image onto the card:
+
+   .. code-block:: console
+
+      cxgbetool t5nex0 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
+
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+      kldload if_cxgbe
+
+#. Verify with sysctl:
+
+   .. code-block:: console
+
+      sysctl -a | grep "t5nex" | grep "firmware"
+
+   Example output:
+
+   .. code-block:: console
+
+      dev.t5nex.0.firmware_version: 1.13.32.0
+
+Sample Application Notes
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+This section demonstrates how to launch **testpmd** with Chelsio T5
+devices managed by librte_pmd_cxgbe in FreeBSD operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
+
+#. Copy the contigmem kernel module to /boot/kernel directory:
+
+   .. code-block:: console
+
+      cp x86_64-native-bsdapp-clang/kmod/contigmem.ko /boot/kernel/
+
+#. Add the following lines to /boot/loader.conf:
+
+   .. code-block:: console
+
+      # reserve 2 x 1G blocks of contiguous memory using contigmem driver
+      hw.contigmem.num_buffers=2
+      hw.contigmem.buffer_size=1073741824
+      # load contigmem module during boot process
+      contigmem_load="YES"
+
+   The above lines load the contigmem kernel module during boot process and
+   allocate 2 x 1G blocks of contiguous memory to be used for DPDK later on.
+   This is to avoid issues with potential memory fragmentation during later
+   system up time, which may result in failure of allocating the contiguous
+   memory required for the contigmem kernel module.
+
+#. Restart the system and ensure the contigmem module is loaded successfully:
+
+   .. code-block:: console
+
+      reboot
+      kldstat | grep "contigmem"
+
+   Example output:
+
+   .. code-block:: console
+
+      2    1 0xffffffff817f1000 3118     contigmem.ko
+
+#. Repeat step 1 to ensure that you are in the DPDK source directory.
+
+#. Load the cxgbe kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Get the PCI bus addresses of the interfaces bound to t5nex driver:
+
+   .. code-block:: console
+
+      pciconf -l | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0@pci0:2:0:4: class=0x020000 card=0x00001425 chip=0x54011425 rev=0x00
+
+   In the above example, the t5nex0 is bound to 2:0:4 bus address.
+
+   .. note::
+
+      Both the interfaces of a Chelsio T5 2-port adapter are bound to the
+      same PCI bus address.
+
+#. Unload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+
+#. Set the PCI bus addresses to hw.nic_uio.bdfs kernel environment parameter:
+
+   .. code-block:: console
+
+      kenv hw.nic_uio.bdfs="2:0:4"
+
+   This automatically binds 2:0:4 to nic_uio kernel driver when it is loaded in
+   the next step.
+
+   .. note::
+
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
+
+#. Load nic_uio kernel driver:
+
+   .. code-block:: console
+
+      kldload ./x86_64-native-bsdapp-clang/kmod/nic_uio.ko
+
+#. Start testpmd with basic parameters:
+
+   .. code-block:: console
+
+      ./x86_64-native-bsdapp-clang/app/testpmd -c 0xf -n 4 -w 0000:02:00.4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:02:00.4 on NUMA socket 0
+      EAL:   probe driver: 1425:5401 rte_cxgbe_pmd
+      EAL:   PCI memory mapped at 0x8007ec000
+      EAL:   PCI memory mapped at 0x842800000
+      EAL:   PCI memory mapped at 0x80086c000
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
       PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
       Interactive-mode selected
       Configuring Port 0 (socket 0)
-- 
2.4.1

Re: [PATCH v2 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[David Marchand]

Hello Rahul,

On Mon, Jul 20, 2015 at 8:41 AM, Rahul Lakkireddy <
rahul.lakkireddy@chelsio.com> wrote:

> nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
> bind the devices to nic_uio.  However, it's better to remove this
> whitelist of
> pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow
> binding
> any device to nic_uio.
>
> Suggested-by: David Marchand <david.marchand@6wind.com>
> Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
> Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
>

Hum, what bothers me is that you do not rely on the same criteria to
re-attach the devices to nic_uio.
See below.


>  lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 48
> +++++++++------------------------
>  1 file changed, 13 insertions(+), 35 deletions(-)
>
> diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> index 2354e84..f868dc8 100644
> --- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> +++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> [snip]
> @@ -195,11 +177,10 @@ nic_uio_probe (device_t dev)
>  {
>         int i;
>
> -       for (i = 0; i < NUM_DEVICES; i++)
> -               if (pci_get_vendor(dev) == devices[i].vend &&
> -                       pci_get_device(dev) == devices[i].dev) {
> -
> -                       device_set_desc(dev, "Intel(R) DPDK PCI Device");
> +       for (i = 0; i < num_detached; i++)
> +               if (pci_get_vendor(dev) ==
> pci_get_vendor(detached_devices[i]) &&
> +                   pci_get_device(dev) ==
> pci_get_device(detached_devices[i])) {
> +                       device_set_desc(dev, "DPDK PCI Device");
>                         return BUS_PROBE_SPECIFIC;
>                 }
>
>
When going through the probe stuff, the device vendor and type are used as
the matching criteria.

@@ -256,7 +237,6 @@ static void
>  nic_uio_load(void)
>  {
>         uint32_t bus, device, function;
> -       int i;
>         device_t dev;
>         char bdf_str[256];
>         char *token, *remaining;
> @@ -295,17 +275,15 @@ nic_uio_load(void)
>                 if (dev == NULL)
>                         continue;
>
> -               for (i = 0; i < NUM_DEVICES; i++)
> -                       if (pci_get_vendor(dev) == devices[i].vend &&
> -                                       pci_get_device(dev) ==
> devices[i].dev) {
> -                                               if (num_detached <
> MAX_DETACHED_DEVICES) {
> -
>  printf("nic_uio_load: detaching and storing dev=%p\n", dev);
> -
>  detached_devices[num_detached++] = dev;
> -                                               } else
> -
>  printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be
> reattached\n",
> -
>  MAX_DETACHED_DEVICES, dev);
> -                                               device_detach(dev);
> -                       }
> +               if (num_detached < MAX_DETACHED_DEVICES) {
> +                       printf("nic_uio_load: detaching and storing
> dev=%p\n",
> +                              dev);
> +                       detached_devices[num_detached++] = dev;
> +               } else {
> +                       printf("nic_uio_load: reached
> MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
> +                              MAX_DETACHED_DEVICES, dev);
> +               }
> +               device_detach(dev);
>         }
>  }
>

But here at init time, the bdfs informations are used to detach the pci
devices.

I would say this is safer we have the same criteria in both cases.
I think that the pci addresses are the best criteria since this is what the
user gives.
Don't we have them in the dev pointer ?



Btw, with this change, we would then be limited to MAX_DETACHED_DEVICES
devices even if 128 pci devices looks quite big enough to me.
This part could be reworked (later).


-- 
David Marchand

Re: [PATCH v2 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[Rahul Lakkireddy]

Hi David,

On Mon, Jul 20, 2015 at 09:43:57 +0200, David Marchand wrote:
>    Hum, what bothers me is that you do not rely on the same criteria to
>    re-attach the devices to nic_uio.
>    See below.
> 
>       lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 48
>      +++++++++------------------------
>       1 file changed, 13 insertions(+), 35 deletions(-)
> 
>      diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
>      b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
>      index 2354e84..f868dc8 100644
>      --- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
>      +++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
>      [snip]
>      @@ -195,11 +177,10 @@ nic_uio_probe (device_t dev)
>       {
>              int i;
> 
>      -       for (i = 0; i < NUM_DEVICES; i++)
>      -               if (pci_get_vendor(dev) == devices[i].vend &&
>      -                       pci_get_device(dev) == devices[i].dev) {
>      -
>      -                       device_set_desc(dev, "Intel(R) DPDK PCI
>      Device");
>      +       for (i = 0; i < num_detached; i++)
>      +               if (pci_get_vendor(dev) ==
>      pci_get_vendor(detached_devices[i]) &&
>      +                   pci_get_device(dev) ==
>      pci_get_device(detached_devices[i])) {
>      +                       device_set_desc(dev, "DPDK PCI Device");
>                              return BUS_PROBE_SPECIFIC;
>                      }
> 
>    When going through the probe stuff, the device vendor and type are used as
>    the matching criteria.
> 
>      @@ -256,7 +237,6 @@ static void
>       nic_uio_load(void)
>       {
>              uint32_t bus, device, function;
>      -       int i;
>              device_t dev;
>              char bdf_str[256];
>              char *token, *remaining;
>      @@ -295,17 +275,15 @@ nic_uio_load(void)
>                      if (dev == NULL)
>                              continue;
> 
>      -               for (i = 0; i < NUM_DEVICES; i++)
>      -                       if (pci_get_vendor(dev) == devices[i].vend &&
>      -                                       pci_get_device(dev) ==
>      devices[i].dev) {
>      -                                               if (num_detached <
>      MAX_DETACHED_DEVICES) {
>      -                                                     
>       printf("nic_uio_load: detaching and storing dev=%p\n", dev);
>      -                                                     
>       detached_devices[num_detached++] = dev;
>      -                                               } else
>      -                                                     
>       printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be
>      reattached\n",
>      -                                                             
>       MAX_DETACHED_DEVICES, dev);
>      -                                               device_detach(dev);
>      -                       }
>      +               if (num_detached < MAX_DETACHED_DEVICES) {
>      +                       printf("nic_uio_load: detaching and storing
>      dev=%p\n",
>      +                              dev);
>      +                       detached_devices[num_detached++] = dev;
>      +               } else {
>      +                       printf("nic_uio_load: reached
>      MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
>      +                              MAX_DETACHED_DEVICES, dev);
>      +               }
>      +               device_detach(dev);
>              }
>       }
> 
>    But here at init time, the bdfs informations are used to detach the pci
>    devices.
>    I would say this is safer we have the same criteria in both cases.
>    I think that the pci addresses are the best criteria since this is what
>    the user gives.
>    Don't we have them in the dev pointer ?

It looks like we can get them via pci_get_bus(), pci_get_slot(), and
pci_get_function().  Will add check for these 3 info instead of vendor
and device in probe to make it consistent.

> 
>    Btw, with this change, we would then be limited to MAX_DETACHED_DEVICES
>    devices even if 128 pci devices looks quite big enough to me.
>    This part could be reworked (later).
>    --
>    David Marchand

Thanks,
Rahul

[PATCH v3 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

This series of patches fix compilation and enable CXGBE poll mode driver for
FreeBSD.  The first patch fixes a limitation of nic_uio that only binds to
devices present in rte_pci_dev_ids.h.  The second patch does the actual
compilation fix and enabling of CXGBE PMD for FreeBSD.  The last patch updates
cxgbe documentation to reflect the FreeBSD support for CXGBE PMD.

v3:
- Use checks for bus, slot, and function info instead of vendor and device
  during nic_uio probe for better consistency.

v2:
- Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.

Rahul Lakkireddy (3):
  nic_uio: Fix to allow any device to be bound to nic_uio
  cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD
  doc: Update documentation to reflect FreeBSD support for CXGBE PMD

 config/common_bsdapp                    |   2 +-
 doc/guides/nics/cxgbe.rst               | 280 ++++++++++++++++++++++++++++++--
 drivers/net/cxgbe/base/common.h         |   2 +-
 drivers/net/cxgbe/base/t4_hw.c          |   6 +-
 drivers/net/cxgbe/cxgbe_compat.h        |  12 +-
 drivers/net/cxgbe/cxgbe_main.c          |   4 +-
 drivers/net/cxgbe/sge.c                 |  10 +-
 lib/librte_eal/bsdapp/nic_uio/nic_uio.c |  54 ++----
 8 files changed, 303 insertions(+), 67 deletions(-)

-- 
2.4.1

[PATCH v3 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[Rahul Lakkireddy]

nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
bind the devices to nic_uio.  However, it's better to remove this whitelist of
pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow binding
any device to nic_uio.

Suggested-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v3:
- Use checks for bus, slot, and function info instead of vendor and device
  during nic_uio probe for better consistency.

v2:
- Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.

 lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 54 +++++++++++----------------------
 1 file changed, 18 insertions(+), 36 deletions(-)

diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
index 2354e84..99a4975 100644
--- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
+++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
@@ -106,24 +106,6 @@ struct pci_bdf {
 	uint32_t function;
 };
 
-
-#define RTE_PCI_DEV_ID_DECL_EM(vend, dev)      {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGB(vend, dev)     {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IGBVF(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBE(vend, dev)   {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_IXGBEVF(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40E(vend, dev)    {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_I40EVF(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VIRTIO(vend, dev)  {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_VMXNET3(vend, dev) {vend, dev},
-#define RTE_PCI_DEV_ID_DECL_ENIC(vend, dev)    {vend, dev},
-
-const struct device devices[] = {
-#include <rte_pci_dev_ids.h>
-};
-#define NUM_DEVICES (sizeof(devices)/sizeof(devices[0]))
-
-
 static devclass_t nic_uio_devclass;
 
 DEFINE_CLASS_0(nic_uio, nic_uio_driver, nic_uio_methods, sizeof(struct nic_uio_softc));
@@ -194,12 +176,15 @@ static int
 nic_uio_probe (device_t dev)
 {
 	int i;
-
-	for (i = 0; i < NUM_DEVICES; i++)
-		if (pci_get_vendor(dev) == devices[i].vend &&
-			pci_get_device(dev) == devices[i].dev) {
-
-			device_set_desc(dev, "Intel(R) DPDK PCI Device");
+	unsigned int bus = pci_get_bus(dev);
+	unsigned int device = pci_get_slot(dev);
+	unsigned int function = pci_get_function(dev);
+
+	for (i = 0; i < num_detached; i++)
+		if (bus == pci_get_bus(detached_devices[i]) &&
+		    device == pci_get_slot(detached_devices[i]) &&
+		    function == pci_get_function(detached_devices[i])) {
+			device_set_desc(dev, "DPDK PCI Device");
 			return BUS_PROBE_SPECIFIC;
 		}
 
@@ -256,7 +241,6 @@ static void
 nic_uio_load(void)
 {
 	uint32_t bus, device, function;
-	int i;
 	device_t dev;
 	char bdf_str[256];
 	char *token, *remaining;
@@ -295,17 +279,15 @@ nic_uio_load(void)
 		if (dev == NULL)
 			continue;
 
-		for (i = 0; i < NUM_DEVICES; i++)
-			if (pci_get_vendor(dev) == devices[i].vend &&
-					pci_get_device(dev) == devices[i].dev) {
-						if (num_detached < MAX_DETACHED_DEVICES) {
-							printf("nic_uio_load: detaching and storing dev=%p\n", dev);
-							detached_devices[num_detached++] = dev;
-						} else
-							printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
-								MAX_DETACHED_DEVICES, dev);
-						device_detach(dev);
-			}
+		if (num_detached < MAX_DETACHED_DEVICES) {
+			printf("nic_uio_load: detaching and storing dev=%p\n",
+			       dev);
+			detached_devices[num_detached++] = dev;
+		} else {
+			printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be reattached\n",
+			       MAX_DETACHED_DEVICES, dev);
+		}
+		device_detach(dev);
 	}
 }
 
-- 
2.4.1

[PATCH v3 2/3] cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD

[Rahul Lakkireddy]

Fix "MACRO redefined" and "function redefined" compilation errors in FreeBSD
by adding CXGBE prefix to them.  Also remove reference to a linux header
linux/if_ether.h and use DPDK macros directly.  Finally, enable CXGBE PMD for
FreeBSD.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v3:
- No changes.

v2:
- No changes.

 config/common_bsdapp             |  2 +-
 drivers/net/cxgbe/base/common.h  |  2 +-
 drivers/net/cxgbe/base/t4_hw.c   |  6 +++---
 drivers/net/cxgbe/cxgbe_compat.h | 12 ++++++------
 drivers/net/cxgbe/cxgbe_main.c   |  4 ++--
 drivers/net/cxgbe/sge.c          | 10 +++++-----
 6 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/config/common_bsdapp b/config/common_bsdapp
index 4e505bf..e96e4a5 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -217,7 +217,7 @@ CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
 #
 # Compile burst-oriented Chelsio Terminator 10GbE/40GbE (CXGBE) PMD
 #
-CONFIG_RTE_LIBRTE_CXGBE_PMD=n
+CONFIG_RTE_LIBRTE_CXGBE_PMD=y
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG=n
 CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX=n
diff --git a/drivers/net/cxgbe/base/common.h b/drivers/net/cxgbe/base/common.h
index 6ddc7d4..cf2e82d 100644
--- a/drivers/net/cxgbe/base/common.h
+++ b/drivers/net/cxgbe/base/common.h
@@ -43,7 +43,7 @@
 extern "C" {
 #endif
 
-#define PAGE_SIZE RTE_PGSIZE_4K
+#define CXGBE_PAGE_SIZE RTE_PGSIZE_4K
 
 enum {
 	MAX_NPORTS     = 4,     /* max # of ports */
diff --git a/drivers/net/cxgbe/base/t4_hw.c b/drivers/net/cxgbe/base/t4_hw.c
index c57200e..884d2cf 100644
--- a/drivers/net/cxgbe/base/t4_hw.c
+++ b/drivers/net/cxgbe/base/t4_hw.c
@@ -127,7 +127,7 @@ void t4_load_mtus(struct adapter *adap, const unsigned short *mtus,
 
 	for (i = 0; i < NMTUS; ++i) {
 		unsigned int mtu = mtus[i];
-		unsigned int log2 = fls(mtu);
+		unsigned int log2 = cxgbe_fls(mtu);
 
 		if (!(mtu & ((1 << log2) >> 2)))     /* round */
 			log2--;
@@ -1545,11 +1545,11 @@ int t4_fixup_host_params_compat(struct adapter *adap,
 				unsigned int cache_line_size,
 				enum chip_type chip_compat)
 {
-	unsigned int page_shift = fls(page_size) - 1;
+	unsigned int page_shift = cxgbe_fls(page_size) - 1;
 	unsigned int sge_hps = page_shift - 10;
 	unsigned int stat_len = cache_line_size > 64 ? 128 : 64;
 	unsigned int fl_align = cache_line_size < 32 ? 32 : cache_line_size;
-	unsigned int fl_align_log = fls(fl_align) - 1;
+	unsigned int fl_align_log = cxgbe_fls(fl_align) - 1;
 
 	t4_write_reg(adap, A_SGE_HOST_PAGE_SIZE,
 		     V_HOSTPAGESIZEPF0(sge_hps) |
diff --git a/drivers/net/cxgbe/cxgbe_compat.h b/drivers/net/cxgbe/cxgbe_compat.h
index 3b871ee..e68f8f5 100644
--- a/drivers/net/cxgbe/cxgbe_compat.h
+++ b/drivers/net/cxgbe/cxgbe_compat.h
@@ -119,8 +119,8 @@
 #define L1_CACHE_BYTES  BIT(L1_CACHE_SHIFT)
 
 #define PAGE_SHIFT  12
-#define ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
-#define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a)))
+#define CXGBE_ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define PTR_ALIGN(p, a) ((typeof(p))CXGBE_ALIGN((unsigned long)(p), (a)))
 
 #define VLAN_HLEN 4
 
@@ -178,7 +178,7 @@ typedef uint64_t  dma_addr_t;
 /*
  * round up val _p to a power of 2 size _s
  */
-#define roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
+#define cxgbe_roundup(_p, _s) (((unsigned long)(_p) + (_s - 1)) & ~(_s - 1))
 
 #undef container_of
 #define container_of(ptr, type, member) ({ \
@@ -214,13 +214,13 @@ static inline uint8_t hweight32(uint32_t word32)
 } /* weight32 */
 
 /**
- * fls - find last (most-significant) bit set
+ * cxgbe_fls - find last (most-significant) bit set
  * @x: the word to search
  *
  * This is defined the same way as ffs.
- * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ * Note cxgbe_fls(0) = 0, cxgbe_fls(1) = 1, cxgbe_fls(0x80000000) = 32.
  */
-static inline int fls(int x)
+static inline int cxgbe_fls(int x)
 {
 	return x ? sizeof(x) * 8 - __builtin_clz(x) : 0;
 }
diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c
index b879820..3755444 100644
--- a/drivers/net/cxgbe/cxgbe_main.c
+++ b/drivers/net/cxgbe/cxgbe_main.c
@@ -411,7 +411,7 @@ static int adap_init0_tweaks(struct adapter *adapter)
 	 * Line Size, etc.  The firmware default is for a 4KB Page Size and
 	 * 64B Cache Line Size ...
 	 */
-	t4_fixup_host_params_compat(adapter, PAGE_SIZE, L1_CACHE_BYTES,
+	t4_fixup_host_params_compat(adapter, CXGBE_PAGE_SIZE, L1_CACHE_BYTES,
 				    T5_LAST_REV);
 
 	/*
@@ -1100,7 +1100,7 @@ int cxgbe_probe(struct adapter *adapter)
 		qpp = 1 << ((t4_read_reg(adapter,
 				A_SGE_EGRESS_QUEUES_PER_PAGE_PF) >> s_qpp)
 				& M_QUEUESPERPAGEPF0);
-		num_seg = PAGE_SIZE / UDBS_SEG_SIZE;
+		num_seg = CXGBE_PAGE_SIZE / UDBS_SEG_SIZE;
 		if (qpp > num_seg)
 			dev_warn(adapter, "Incorrect SGE EGRESS QUEUES_PER_PAGE configuration, continuing in debug mode\n");
 
diff --git a/drivers/net/cxgbe/sge.c b/drivers/net/cxgbe/sge.c
index 1a10278..d570d33 100644
--- a/drivers/net/cxgbe/sge.c
+++ b/drivers/net/cxgbe/sge.c
@@ -31,7 +31,6 @@
  *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#include <linux/if_ether.h>
 #include <sys/queue.h>
 #include <stdio.h>
 #include <errno.h>
@@ -98,7 +97,8 @@ static inline unsigned int fl_mtu_bufsize(struct adapter *adapter,
 {
 	struct sge *s = &adapter->sge;
 
-	return ALIGN(s->pktshift + ETH_HLEN + VLAN_HLEN + mtu, s->fl_align);
+	return CXGBE_ALIGN(s->pktshift + ETHER_HDR_LEN + VLAN_HLEN + mtu,
+			   s->fl_align);
 }
 
 #define FL_MTU_SMALL_BUFSIZE(adapter) fl_mtu_bufsize(adapter, FL_MTU_SMALL)
@@ -1578,7 +1578,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 	unsigned int nb_refill;
 
 	/* Size needs to be multiple of 16, including status entry. */
-	iq->size = roundup(iq->size, 16);
+	iq->size = cxgbe_roundup(iq->size, 16);
 
 	snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 		 eth_dev->driver->pci_drv.name, fwevtq ? "fwq_ring" : "rx_ring",
@@ -1630,7 +1630,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
 		 */
 		if (fl->size < s->fl_starve_thres - 1 + 2 * 8)
 			fl->size = s->fl_starve_thres - 1 + 2 * 8;
-		fl->size = roundup(fl->size, 8);
+		fl->size = cxgbe_roundup(fl->size, 8);
 
 		snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d",
 			 eth_dev->driver->pci_drv.name,
@@ -2064,7 +2064,7 @@ static int t4_sge_init_soft(struct adapter *adap)
 	 * The Page Size Buffer must be exactly equal to our Page Size and the
 	 * Large Page Size Buffer should be 0 (per above) or a power of 2.
 	 */
-	if (fl_small_pg != PAGE_SIZE ||
+	if (fl_small_pg != CXGBE_PAGE_SIZE ||
 	    (fl_large_pg & (fl_large_pg - 1)) != 0) {
 		dev_err(adap, "bad SGE FL page buffer sizes [%d, %d]\n",
 			fl_small_pg, fl_large_pg);
-- 
2.4.1

[PATCH v3 3/3] doc: Update documentation to reflect FreeBSD support for CXGBE PMD

[Rahul Lakkireddy]

Update cxgbe documentation to include support for FreeBSD:
1. Add instructions on how to compile CXGBE PMD in FreeBSD.
2. Add instructions on how to flash firmware image on Chelsio T5 cards in
   FreeBSD.
3. Add sample application usage for FreeBSD.
4. Add an extra step to reload kernel module in Linux in order for the new
   firmware to be loaded.
5. Typo fixes.

Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
---
v3:
- No changes.

v2:
- No changes.

 doc/guides/nics/cxgbe.rst | 280 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 267 insertions(+), 13 deletions(-)

diff --git a/doc/guides/nics/cxgbe.rst b/doc/guides/nics/cxgbe.rst
index 7557db0..148cd25 100644
--- a/doc/guides/nics/cxgbe.rst
+++ b/doc/guides/nics/cxgbe.rst
@@ -32,7 +32,8 @@ CXGBE Poll Mode Driver
 ======================
 
 The CXGBE PMD (**librte_pmd_cxgbe**) provides poll mode driver support
-for **Chelsio T5** 10/40 Gbps family of adapters.
+for **Chelsio T5** 10/40 Gbps family of adapters. CXGBE PMD has support
+for the latest Linux and FreeBSD operating systems.
 
 More information can be found at `Chelsio Communications Official Website
 <http://www.chelsio.com>`_.
@@ -73,10 +74,20 @@ Prerequisites
 
 - Requires firmware version **1.13.32.0** and higher. Visit
   `Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware
-  bundled with the latest Chelsio Unified Wire package. Installing and loading
-  the latest cxgb4 kernel driver from the Chelsio Unified Wire package should
-  get you the latest firmware. More information can be obtained from the User
-  Guide that is bundled with the Chelsio Unified Wire package.
+  bundled with the latest Chelsio Unified Wire package.
+
+  For Linux, installing and loading the latest cxgb4 kernel driver from the
+  Chelsio Unified Wire package should get you the latest firmware. More
+  information can be obtained from the User Guide that is bundled with the
+  Chelsio Unified Wire package.
+
+  For FreeBSD, the latest firmware obtained from the Chelsio Unified Wire
+  package must be manually flashed via cxgbetool available in FreeBSD source
+  repository.
+
+  Instructions on how to manually flash the firmware are given in section
+  :ref:`linux-installation` for Linux and section :ref:`freebsd-installation`
+  for FreeBSD.
 
 Pre-Installation Configuration
 ------------------------------
@@ -111,6 +122,8 @@ enabling debugging options may affect system performance.
 
   Toggle display of receiving data path run-time check messages.
 
+.. _driver-compilation:
+
 Driver Compilation
 ~~~~~~~~~~~~~~~~~~
 
@@ -122,11 +135,24 @@ command:
    cd <DPDK-source-directory>
    make config T=x86_64-native-linuxapp-gcc install
 
-Installation
-------------
+To compile CXGBE PMD for FreeBSD x86_64 clang target, run the following "gmake"
+command:
 
-Steps to manually install the latest firmware from the downloaded source files
-are as follows:
+.. code-block:: console
+
+   cd <DPDK-source-directory>
+   gmake config T=x86_64-native-bsdapp-clang install
+
+Linux
+-----
+
+.. _linux-installation:
+
+Linux Installation
+~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for Linux operating system are as follows:
 
 #. Load the kernel module:
 
@@ -166,6 +192,13 @@ are as follows:
 
       cxgbtool p1p1 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
 
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      modprobe -r cxgb4
+      modprobe cxgb4
+
 #. Verify with ethtool:
 
    .. code-block:: console
@@ -179,10 +212,17 @@ are as follows:
       firmware-version: 1.13.32.0, TP 0.1.4.8
 
 Sample Application Notes
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~
 
 This section demonstrates how to launch **testpmd** with Chelsio T5
-devices managed by librte_pmd_cxgbe.
+devices managed by librte_pmd_cxgbe in Linux operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
 
 #. Load the kernel module:
 
@@ -253,7 +293,7 @@ devices managed by librte_pmd_cxgbe.
 
    .. note::
 
-      CXGBE PMD currently only supports the binding of PF4 for Chelsio T5 NICs.
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
 
 #. Start testpmd with basic parameters:
 
@@ -271,7 +311,221 @@ devices managed by librte_pmd_cxgbe.
       EAL:   PCI memory mapped at 0x7fd7c0200000
       EAL:   PCI memory mapped at 0x7fd77cdfd000
       EAL:   PCI memory mapped at 0x7fd7c10b7000
-      PMD: rte_cxgbe_pmd: fw: 1.13.33.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
+      PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
+      Interactive-mode selected
+      Configuring Port 0 (socket 0)
+      Port 0: 00:07:43:2D:EA:C0
+      Configuring Port 1 (socket 0)
+      Port 1: 00:07:43:2D:EA:C8
+      Checking link statuses...
+      PMD: rte_cxgbe_pmd: Port0: passive DA port module inserted
+      PMD: rte_cxgbe_pmd: Port1: passive DA port module inserted
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Port 1 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
+
+.. note::
+
+   Flow control pause TX/RX is disabled by default and can be enabled via
+   testpmd as follows:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 0
+      testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 1
+
+   To disable again, use:
+
+   .. code-block:: console
+
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 0
+      testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 1
+
+FreeBSD
+-------
+
+.. _freebsd-installation:
+
+FreeBSD Installation
+~~~~~~~~~~~~~~~~~~~~
+
+Steps to manually install the latest firmware from the downloaded Chelsio
+Unified Wire package for FreeBSD operating system are as follows:
+
+#. Load the kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Use dmesg to get the t5nex instance assigned to the Chelsio card:
+
+   .. code-block:: console
+
+      dmesg | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0: <Chelsio T520-CR> irq 16 at device 0.4 on pci2
+      cxl0: <port 0> on t5nex0
+      cxl1: <port 1> on t5nex0
+      t5nex0: PCIe x8, 2 ports, 14 MSI-X interrupts, 31 eq, 13 iq
+
+   In the example above, a Chelsio T520-CR card is bound to a t5nex0 instance.
+
+#. Install cxgbetool from FreeBSD source repository:
+
+   .. code-block:: console
+
+      cd <path_to_FreeBSD_source>/tools/tools/cxgbetool/
+      make && make install
+
+#. Use cxgbetool to load the firmware image onto the card:
+
+   .. code-block:: console
+
+      cxgbetool t5nex0 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin
+
+#. Unload and reload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+      kldload if_cxgbe
+
+#. Verify with sysctl:
+
+   .. code-block:: console
+
+      sysctl -a | grep "t5nex" | grep "firmware"
+
+   Example output:
+
+   .. code-block:: console
+
+      dev.t5nex.0.firmware_version: 1.13.32.0
+
+Sample Application Notes
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+This section demonstrates how to launch **testpmd** with Chelsio T5
+devices managed by librte_pmd_cxgbe in FreeBSD operating system.
+
+#. Change to DPDK source directory where the target has been compiled in
+   section :ref:`driver-compilation`:
+
+   .. code-block:: console
+
+      cd <DPDK-source-directory>
+
+#. Copy the contigmem kernel module to /boot/kernel directory:
+
+   .. code-block:: console
+
+      cp x86_64-native-bsdapp-clang/kmod/contigmem.ko /boot/kernel/
+
+#. Add the following lines to /boot/loader.conf:
+
+   .. code-block:: console
+
+      # reserve 2 x 1G blocks of contiguous memory using contigmem driver
+      hw.contigmem.num_buffers=2
+      hw.contigmem.buffer_size=1073741824
+      # load contigmem module during boot process
+      contigmem_load="YES"
+
+   The above lines load the contigmem kernel module during boot process and
+   allocate 2 x 1G blocks of contiguous memory to be used for DPDK later on.
+   This is to avoid issues with potential memory fragmentation during later
+   system up time, which may result in failure of allocating the contiguous
+   memory required for the contigmem kernel module.
+
+#. Restart the system and ensure the contigmem module is loaded successfully:
+
+   .. code-block:: console
+
+      reboot
+      kldstat | grep "contigmem"
+
+   Example output:
+
+   .. code-block:: console
+
+      2    1 0xffffffff817f1000 3118     contigmem.ko
+
+#. Repeat step 1 to ensure that you are in the DPDK source directory.
+
+#. Load the cxgbe kernel module:
+
+   .. code-block:: console
+
+      kldload if_cxgbe
+
+#. Get the PCI bus addresses of the interfaces bound to t5nex driver:
+
+   .. code-block:: console
+
+      pciconf -l | grep "t5nex"
+
+   Example output:
+
+   .. code-block:: console
+
+      t5nex0@pci0:2:0:4: class=0x020000 card=0x00001425 chip=0x54011425 rev=0x00
+
+   In the above example, the t5nex0 is bound to 2:0:4 bus address.
+
+   .. note::
+
+      Both the interfaces of a Chelsio T5 2-port adapter are bound to the
+      same PCI bus address.
+
+#. Unload the kernel module:
+
+   .. code-block:: console
+
+      kldunload if_cxgbe
+
+#. Set the PCI bus addresses to hw.nic_uio.bdfs kernel environment parameter:
+
+   .. code-block:: console
+
+      kenv hw.nic_uio.bdfs="2:0:4"
+
+   This automatically binds 2:0:4 to nic_uio kernel driver when it is loaded in
+   the next step.
+
+   .. note::
+
+      Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs.
+
+#. Load nic_uio kernel driver:
+
+   .. code-block:: console
+
+      kldload ./x86_64-native-bsdapp-clang/kmod/nic_uio.ko
+
+#. Start testpmd with basic parameters:
+
+   .. code-block:: console
+
+      ./x86_64-native-bsdapp-clang/app/testpmd -c 0xf -n 4 -w 0000:02:00.4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:02:00.4 on NUMA socket 0
+      EAL:   probe driver: 1425:5401 rte_cxgbe_pmd
+      EAL:   PCI memory mapped at 0x8007ec000
+      EAL:   PCI memory mapped at 0x842800000
+      EAL:   PCI memory mapped at 0x80086c000
+      PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8
       PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
       Interactive-mode selected
       Configuring Port 0 (socket 0)
-- 
2.4.1

Re: [PATCH v3 1/3] nic_uio: Fix to allow any device to be bound to nic_uio

[David Marchand]

On Mon, Jul 20, 2015 at 7:31 PM, Rahul Lakkireddy <
rahul.lakkireddy@chelsio.com> wrote:

> nic_uio requires the pci ids to be present in rte_pci_dev_ids.h in order to
> bind the devices to nic_uio.  However, it's better to remove this
> whitelist of
> pci ids, and instead rely on hw.nic_uio.bdfs kenv parameter to allow
> binding
> any device to nic_uio.
>
> Suggested-by: David Marchand <david.marchand@6wind.com>
> Signed-off-by: Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>
> Signed-off-by: Kumar Sanghvi <kumaras@chelsio.com>
> ---
> v3:
> - Use checks for bus, slot, and function info instead of vendor and device
>   during nic_uio probe for better consistency.
>

Looks good to me.
Acked-by: David Marchand <david.marchand@6wind.com>

Thanks Rahul.

-- 
David Marchand

Re: [PATCH v3 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[Bruce Richardson]

On Mon, Jul 20, 2015 at 11:01:34PM +0530, Rahul Lakkireddy wrote:
> This series of patches fix compilation and enable CXGBE poll mode driver for
> FreeBSD.  The first patch fixes a limitation of nic_uio that only binds to
> devices present in rte_pci_dev_ids.h.  The second patch does the actual
> compilation fix and enabling of CXGBE PMD for FreeBSD.  The last patch updates
> cxgbe documentation to reflect the FreeBSD support for CXGBE PMD.
> 
> v3:
> - Use checks for bus, slot, and function info instead of vendor and device
>   during nic_uio probe for better consistency.
> 
> v2:
> - Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.
> 
> Rahul Lakkireddy (3):
>   nic_uio: Fix to allow any device to be bound to nic_uio
>   cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD
>   doc: Update documentation to reflect FreeBSD support for CXGBE PMD
>

Compilation testing with the patches on my FreeBSD system is all ok, and unbinding
of Intel 10G NICs works ok as before for me.

Thanks for the patches and the new FreeBSD hardware support!

Series Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH v3 0/3] cxgbe: Fix compilation and enable FreeBSD support for CXGBE PMD

[Thomas Monjalon]

2015-07-21 10:40, Bruce Richardson:
> On Mon, Jul 20, 2015 at 11:01:34PM +0530, Rahul Lakkireddy wrote:
> > This series of patches fix compilation and enable CXGBE poll mode driver for
> > FreeBSD.  The first patch fixes a limitation of nic_uio that only binds to
> > devices present in rte_pci_dev_ids.h.  The second patch does the actual
> > compilation fix and enabling of CXGBE PMD for FreeBSD.  The last patch updates
> > cxgbe documentation to reflect the FreeBSD support for CXGBE PMD.
> > 
> > v3:
> > - Use checks for bus, slot, and function info instead of vendor and device
> >   during nic_uio probe for better consistency.
> > 
> > v2:
> > - Replace "Intel(R) DPDK" with just "DPDK" in device description for nic_uio.
> > 
> > Rahul Lakkireddy (3):
> >   nic_uio: Fix to allow any device to be bound to nic_uio
> >   cxgbe: Enable and fix FreeBSD compilation for CXGBE PMD
> >   doc: Update documentation to reflect FreeBSD support for CXGBE PMD
> >
> 
> Compilation testing with the patches on my FreeBSD system is all ok, and unbinding
> of Intel 10G NICs works ok as before for me.
> 
> Thanks for the patches and the new FreeBSD hardware support!
> 
> Series Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied, thanks