From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rasesh Mody Subject: [PATCH v5 02/10] qede: Add documentation Date: Thu, 31 Mar 2016 19:15:19 -0700 Message-ID: <1459476927-21387-3-git-send-email-rasesh.mody@qlogic.com> References: <1459476927-21387-1-git-send-email-rasesh.mody@qlogic.com> Mime-Version: 1.0 Content-Type: text/plain Cc: , , , , , Rasesh Mody To: , Return-path: Received: from mx0b-0016ce01.pphosted.com (mx0a-0016ce01.pphosted.com [67.231.148.157]) by dpdk.org (Postfix) with ESMTP id AB25D9AA6 for ; Fri, 1 Apr 2016 04:15:44 +0200 (CEST) In-Reply-To: <1459476927-21387-1-git-send-email-rasesh.mody@qlogic.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Harish Patil Signed-off-by: Rasesh Mody Signed-off-by: Sony Chacko --- doc/guides/nics/index.rst | 1 + doc/guides/nics/overview.rst | 78 +++++----- doc/guides/nics/qede.rst | 340 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 380 insertions(+), 39 deletions(-) create mode 100644 doc/guides/nics/qede.rst diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 769f677..d67f056 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -53,6 +53,7 @@ Network Interface Controller Drivers vhost vmxnet3 pcap_ring + qede **Figures** diff --git a/doc/guides/nics/overview.rst b/doc/guides/nics/overview.rst index 542479a..8798f3f 100644 --- a/doc/guides/nics/overview.rst +++ b/doc/guides/nics/overview.rst @@ -74,39 +74,39 @@ Most of these differences are summarized below. .. table:: Features availability in networking drivers - ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = - Feature a b b b c e e e i i i i i i i i i i f f m m m n n p r s v v v v x - f n n o x 1 n n 4 4 4 4 g g x x x x m m l l p f u c i z h i i m e - p x x n g 0 a i 0 0 0 0 b b g g g g 1 1 x x i p l a n e o r r x n - a 2 2 d b 0 c e e e e v b b b b 0 0 4 5 p l p g d s t t n v - c x x i e 0 . v v f e e e e k k e a t i i e i - k v n . f f . v v . t o o t r - e f g . . . f f . a . 3 t - t v v v v v 2 v - e e e e e e - c c c c c c - ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = - link status X X X X X - link status event X X X - queue status event X + ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = + Feature a b b b c e e e i i i i i i i i i i f f m m m n n p q q r s v v v v x + f n n o x 1 n n 4 4 4 4 g g x x x x m m l l p f u c e e i z h i i m e + p x x n g 0 a i 0 0 0 0 b b g g g g 1 1 x x i p l a d d n e o r r x n + a 2 2 d b 0 c e e e e v b b b b 0 0 4 5 p l p e e g d s t t n v + c x x i e 0 . v v f e e e e k k e v a t i i e i + k v n . f f . v v . f t o o t r + e f g . . . f f . a . 3 t + t v v v v v 2 v + e e e e e e + c c c c c c + ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = + link status X X X X X X X + link status event X X X X X + queue status event X Rx interrupt X X X X - queue start/stop X X X X X X X + queue start/stop X X X X X X X MTU update X X - jumbo frame X X X X X X - scattered Rx X X X X X X X + jumbo frame X X X X X X X X + scattered Rx X X X X X X X LRO TSO X X X X X X - promiscuous mode X X X X X X - allmulticast mode X X X X X X - unicast MAC filter X X X X - multicast MAC filter X X X X - RSS hash X X X X X X + promiscuous mode X X X X X X X X + allmulticast mode X X X X X X X X + unicast MAC filter X X X X X X + multicast MAC filter X X X X X X + RSS hash X X X X X X X X RSS key update X X X X X RSS reta update X X X X X VMDq X X - SR-IOV X X X + SR-IOV X X X X DCB X X - VLAN filter X X X X + VLAN filter X X X X X X ethertype filter X X n-tuple filter SYN filter @@ -114,35 +114,35 @@ Most of these differences are summarized below. flexible filter hash filter X X X X flow director X X - flow control X X X + flow control X X X X X rate limitation traffic mirroring X X - CRC offload X X X X - VLAN offload X X X X + CRC offload X X X X X X + VLAN offload X X X X X X QinQ offload X X L3 checksum offload X X X X L4 checksum offload X X X X inner L3 checksum X X X inner L4 checksum X X X - packet type parsing X X X + packet type parsing X X X X X timesync X X - basic stats X X X X X X X X + basic stats X X X X X X X X X X extended stats X X X X X - stats per queue X X + stats per queue X X X X EEPROM dump registers dump - multiprocess aware X X X X + multiprocess aware X X X X X X BSD nic_uio X X X X X - Linux UIO X X X X X X - Linux VFIO X X X X X - other kdrv X + Linux UIO X X X X X X X X + Linux VFIO X X X X X X X + other kdrv X ARMv7 ARMv8 Power8 TILE-Gx - x86-32 X X X X X X X - x86-64 X X X X X X X X - usage doc X X + x86-32 X X X X X X X + x86-64 X X X X X X X X X X + usage doc X X X X design doc perf doc - ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = + ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst new file mode 100644 index 0000000..67d23e9 --- /dev/null +++ b/doc/guides/nics/qede.rst @@ -0,0 +1,340 @@ +.. BSD LICENSE + Copyright (c) 2016 QLogic Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of QLogic Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +QEDE Poll Mode Driver +====================== + +The QEDE poll mode driver library (**librte_pmd_qede**) implements support +for **QLogic FastLinQ QL4xxxx 25G/40G CNA** family of adapters as well +as their virtual functions (VF) in SR-IOV context. It is supported on +several standard Linux distros like RHEL7.x and SLES12.x OS. +It is compile-tested under FreeBSD OS. + +More information can be found at `QLogic Corporation's Official Website +`_. + +Supported Features +------------------ + +- Unicast/Multicast filtering +- Promiscuous mode +- Allmulti mode +- Port hardware statistics +- Jumbo frames (using single buffer) +- VLAN offload - Filtering and stripping +- Stateless checksum offloads (IPv4/TCP/UDP) +- Multiple Rx/Tx queues (queue-pairs) +- RSS (with default table/key) +- TSS +- Multiple MAC address +- Default pause flow control +- SR-IOV VF + +Non-supported Features +---------------------- + +- Scatter-Gather Rx/Tx frames +- User configurable RETA table/key +- Unequal number of Rx/Tx queues +- MTU change (dynamic) +- SR-IOV PF +- Tunneling offloads +- Ungraceful recovery + +Supported QLogic NICs +--------------------- + +- QLogic FastLinQ QL4xxxx 25G/40G CNAs + +Prerequisites +------------- + +- Requires storm firmware version **8.7.x.** and management + firmware version **8.7.x and higher**. Storm firmware may be available + under /usr/lib/firmware/qed/ in certain newer Linux + distributions (e.g. qed_init_values_zipped-8.7.8.0.bin). + +- If the required firmware files are not available then visit + `QLogic Driver Download Center ` + +- This driver relies on external zlib library (-lz) for uncompressing + the firmware file. + +Performance note +~~~~~~~~~~~~~~~~ + +- For better performance, it is recommended to use 8K or 16K RX/TX rings + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +The following options can be modified in the ``.config`` file. Please note that +enabling debugging options may affect system performance. + +- ``CONFIG_RTE_LIBRTE_QEDE_PMD`` (default **y**) + + Toggle compilation of QEDE PMD driver. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO`` (default **n**) + + Toggle display of generic debugging messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_ECORE`` (default **n**) + + Toggle display of ecore related messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **n**) + + Toggle display of transmit fast path run-time messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **n**) + + Toggle display of receive fast path run-time messages. + +- ``CONFIG_RTE_LIBRTE_QEDE_RX_COAL_US`` (default **24**) + + Change Rx interrupt coalescing timer (in us). + +- ``CONFIG_RTE_LIBRTE_QEDE_TX_COAL_US`` (default **48**) + + Change Tx interrupt coalescing timer (in us). + +- ``CONFIG_RTE_LIBRTE_QEDE_TX_SWITCHING`` (default **y**) + + Toggle Tx switching + +- ``CONFIG_RTE_LIBRTE_QEDE_FW`` (default **n**) + + Path of firmware file (overrides default location) + +Driver Compilation +~~~~~~~~~~~~~~~~~~ + +To compile QEDE PMD for Linux x86_64 gcc target, run the following "make" +command:: + + cd + make config T=x86_64-native-linuxapp-gcc install + +To compile QEDE PMD for Linux x86_64 clang target, run the following "make" +command:: + + cd + make config T=x86_64-native-linuxapp-clang install + +To compile QEDE PMD for FreeBSD x86_64 clang target, run the following "gmake" +command:: + + cd + gmake config T=x86_64-native-bsdapp-clang install + +To compile QEDE PMD for FreeBSD x86_64 gcc target, run the following "gmake" +command:: + + cd + gmake config T=x86_64-native-bsdapp-gcc install -Wl,-rpath=\ + /usr/local/lib/gcc48 CC=gcc48 + + +Sample Application Notes +~~~~~~~~~~~~~~~~~~~~~~~~ + +This section demonstrates how to launch ``testpmd`` with QLogic 579xx +devices managed by ``librte_pmd_qede`` in Linux operating system. + +#. Request huge pages: + + .. code-block:: console + + echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/ \ + nr_hugepages + +#. Load ``igb_uio`` or ``vfio-pci`` driver: + + .. code-block:: console + + insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + + or + + .. code-block:: console + + modprobe vfio-pci + +#. Bind the QLogic 579xx adapters to ``igb_uio`` or ``vfio-pci`` loaded in the + previous step:: + + ./tools/dpdk_nic_bind.py --bind igb_uio 0000:84:00.0 0000:84:00.1 \ + 0000:84:00.2 0000:84:00.3 + + or + + Setup VFIO permissions for regular users and then bind to ``vfio-pci``: + + .. code-block:: console + + sudo chmod a+x /dev/vfio + + sudo chmod 0666 /dev/vfio/* + + ./tools/dpdk_nic_bind.py --bind vfio-pci 0000:84:00.0 0000:84:00.1 \ + 0000:84:00.2 0000:84:00.3 + +#. Start ``testpmd`` with basic parameters: + + .. code-block:: console + + testpmd -c 0xff1 -n 4 -- -i --nb-cores=8 --portmask=0xf --rxd=4096 \ + --txd=4096 --txfreet=4068 --enable-rx-cksum --rxq=4 --txq=4 \ + --rss-ip --rss-udp + + [...] + + EAL: PCI device 0000:84:00.0 on NUMA socket 1 + EAL: probe driver: 1077:1634 rte_qede_pmd + EAL: Not managed by a supported kernel driver, skipped + EAL: PCI device 0000:84:00.1 on NUMA socket 1 + EAL: probe driver: 1077:1634 rte_qede_pmd + EAL: Not managed by a supported kernel driver, skipped + EAL: PCI device 0000:88:00.0 on NUMA socket 1 + EAL: probe driver: 1077:1656 rte_qede_pmd + EAL: PCI memory mapped at 0x7f738b200000 + EAL: PCI memory mapped at 0x7f738b280000 + EAL: PCI memory mapped at 0x7f738b300000 + PMD: Chip details : BB1 + PMD: Driver version : QEDE PMD 8.7.9.0_1.0.0 + PMD: Firmware version : 8.7.7.0 + PMD: Management firmware version : 8.7.8.0 + PMD: Firmware file : /lib/firmware/qed/qed_init_values_zipped-8.7.7.0.bin + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_common_dev_init:macaddr \ + 00:0e:1e:d2:09:9c + [...] + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 0 num_desc 4096 \ + tx_free_thresh 4068 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 1 num_desc 4096 \ + tx_free_thresh 4068 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 2 num_desc 4096 \ + tx_free_thresh 4068 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 3 num_desc 4096 \ + tx_free_thresh 4068 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 0 num_desc 4096 \ + rx_buf_size=2148 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 1 num_desc 4096 \ + rx_buf_size=2148 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 2 num_desc 4096 \ + rx_buf_size=2148 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 3 num_desc 4096 \ + rx_buf_size=2148 socket 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:port 0 + [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:link status: down + [...] + Checking link statuses... + Port 0 Link Up - speed 25000 Mbps - full-duplex + Port 1 Link Up - speed 25000 Mbps - full-duplex + Port 2 Link Up - speed 25000 Mbps - full-duplex + Port 3 Link Up - speed 25000 Mbps - full-duplex + Done + testpmd> + + +SR-IOV: Prerequisites and Sample Application Notes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This section provides instructions to configure SR-IOV with Linux OS. + +**Note**: QEDE PMD will be used to bind to SR-IOV VF device and + Linux native kernel driver (QEDE) will function as SR-IOV PF driver. + +#. Verify SR-IOV and ARI capability is enabled on the adapter using ``lspci``: + + .. code-block:: console + + lspci -s -vvv + + Example output: + + .. code-block:: console + + [...] + Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI) + [...] + Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV) + [...] + Kernel driver in use: igb_uio + +#. Load the kernel module: + + .. code-block:: console + + modprobe qede + + Example output: + + .. code-block:: console + + systemd-udevd[4848]: renamed network interface eth0 to ens5f0 + systemd-udevd[4848]: renamed network interface eth1 to ens5f1 + +#. Bring up the PF ports: + + .. code-block:: console + + ifconfig ens5f0 up + ifconfig ens5f1 up + +#. Create VF device(s): + + Echo the number of VFs to be created into "sriov_numvfs" sysfs entry + of the parent PF. + + Example output: + + .. code-block:: console + + echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs + + +#. Assign VF MAC address: + + Assign MAC address to the VF using iproute2 utility. The syntax is:: + ip link set vf mac + + Example output: + + .. code-block:: console + + ip link set ens5f0 vf 0 mac 52:54:00:2f:9d:e8 + + +#. PCI Passthrough: + + The VF devices may be passed through to the guest VM using virt-manager or + virsh. QEDE PMD should be used to bind the VF devices in the guest VM + using the instructions outlined in the Application notes above. -- 1.7.10.3