[PATCH 0/3] doc: documentation update for mlx4 PMD

[PATCH 0/3] doc: documentation update for mlx4 PMD

[Moti Haimovsky]

The following three commits update the mlx4 documentation to reflect
the modifications done to the PMD and the new features added to it.

Moti Haimovsky (3):
  doc: fix remove mlx4 Tx inline compilation option
  doc: fix mlx4 drop inline receive support
  doc: update mlx4 documentation

 doc/guides/nics/mlx4.rst               | 114 ++++++++++++++++-----------------
 doc/guides/rel_notes/release_17_11.rst |  26 ++++++++
 2 files changed, 80 insertions(+), 60 deletions(-)

-- 
1.8.3.1

[PATCH 1/3] doc: fix remove mlx4 Tx inline compilation option

[Moti Haimovsky]

This patch removes the stale CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE compilation
option from the documentation.

Fixes: 586db080585b ("net/mlx4: remove Tx inline compilation option")

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
---
 doc/guides/nics/mlx4.rst | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
index 5c3fb76..be1ba16 100644
--- a/doc/guides/nics/mlx4.rst
+++ b/doc/guides/nics/mlx4.rst
@@ -107,11 +107,6 @@ These options can be modified in the ``.config`` file.
   to abort with harmless debugging messages as a workaround.
   Relevant only when CONFIG_RTE_LIBRTE_MLX4_DEBUG is enabled.
 
-- ``CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE`` (default **0**)
-
-  Amount of data to be inlined during TX operations. Improves latency but
-  lowers throughput.
-
 - ``CONFIG_RTE_LIBRTE_MLX4_TX_MP_CACHE`` (default **8**)
 
   Maximum number of cached memory pools (MPs) per TX queue. Each MP from
-- 
1.8.3.1

[PATCH 2/3] doc: fix mlx4 drop inline receive support

[Moti Haimovsky]

This commit removes the stale MLX4_INLINE_RECV_SIZE environment
variable from the documentation.

Fixes: 056eaf2e6d55 ("net/mlx4: drop inline receive support")

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
---
 doc/guides/nics/mlx4.rst | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
index be1ba16..7c43177 100644
--- a/doc/guides/nics/mlx4.rst
+++ b/doc/guides/nics/mlx4.rst
@@ -115,15 +115,6 @@ These options can be modified in the ``.config`` file.
 
   This value is always 1 for RX queues since they use a single MP.
 
-Environment variables
-~~~~~~~~~~~~~~~~~~~~~
-
-- ``MLX4_INLINE_RECV_SIZE``
-
-  A nonzero value enables inline receive for packets up to that size. May
-  significantly improve performance in some cases but lower it in
-  others. Requires careful testing.
-
 Run-time configuration
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -325,10 +316,7 @@ Performance tuning
 
         cat /sys/module/mlx4_core/parameters/log_num_mgm_entry_size
 
-2. Use environment variable MLX4_INLINE_RECV_SIZE=64 to get maximum
-   performance for 64B messages.
-
-3. Use the CPU near local NUMA node to which the PCIe adapter is connected,
+2. Use the CPU near local NUMA node to which the PCIe adapter is connected,
    for better performance. For VMs, verify that the right CPU
    and NUMA node are pinned according to the above. Run:
 
@@ -338,19 +326,19 @@ Performance tuning
 
    to identify the NUMA node to which the PCIe adapter is connected.
 
-4. If more than one adapter is used, and root complex capabilities allow
+3. If more than one adapter is used, and root complex capabilities allow
    to put both adapters on the same NUMA node without PCI bandwidth degradation,
    it is recommended to locate both adapters on the same NUMA node.
    This in order to forward packets from one to the other without
    NUMA performance penalty.
 
-5. Disable pause frames:
+4. Disable pause frames:
 
    .. code-block:: console
 
         ethtool -A <netdev> rx off tx off
 
-6. Verify IO non-posted prefetch is disabled by default. This can be checked
+5. Verify IO non-posted prefetch is disabled by default. This can be checked
    via the BIOS configuration. Please contact you server provider for more
    information about the settings.
 
-- 
1.8.3.1

[PATCH 3/3] doc: update mlx4 documentation

[Moti Haimovsky]

This updates mlx4 documentation and DPDK release notes
to reflect the PMD support for rdma-core from linux-rdma.

- PMD is now freed from Mellanox OFED and now only depends on the
  public rdma-core package (v15 and above) instead.
  (see https://github.com/linux-rdma/rdma-core/releases)
  This PMD should run under Linux v4.14 and above.
- In case any of the above requirements can't be satisfied,
  Mellanox OFED v4.2 and above also provide an updated rdma-core
  as well back-ported kernel modules for most Linux distributions
  and previous Linux versions.
  (see http://www.mellanox.com/page/products_dyn?product_family=26).

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
---
 doc/guides/nics/mlx4.rst               | 89 +++++++++++++++++++---------------
 doc/guides/rel_notes/release_17_11.rst | 26 ++++++++++
 2 files changed, 76 insertions(+), 39 deletions(-)

diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
index 7c43177..22341b9 100644
--- a/doc/guides/nics/mlx4.rst
+++ b/doc/guides/nics/mlx4.rst
@@ -74,13 +74,6 @@ long as they share the same MAC address.
 
 Compiling librte_pmd_mlx4 causes DPDK to be linked against libibverbs.
 
-Features
---------
-
-- Multi arch support: x86_64 and POWER8.
-- Link state information is provided.
-- RX interrupts.
-
 Configuration
 -------------
 
@@ -160,7 +153,7 @@ This driver relies on external libraries and kernel drivers for resources
 allocations and initialization. The following dependencies are not part of
 DPDK and must be installed separately:
 
-- **libibverbs**
+- **libibverbs** (provided by rdma-core package)
 
   User space verbs framework used by librte_pmd_mlx4. This library provides
   a generic interface between the kernel and low-level user space drivers
@@ -170,7 +163,7 @@ DPDK and must be installed separately:
   resources allocations) to be managed by the kernel and fast operations to
   never leave user space.
 
-- **libmlx4**
+- **libmlx4** (provided by rdma-core package)
 
   Low-level user space driver library for Mellanox ConnectX-3 devices,
   it is automatically loaded by libibverbs.
@@ -178,7 +171,7 @@ DPDK and must be installed separately:
   This library basically implements send/receive calls to the hardware
   queues.
 
-- **Kernel modules** (mlnx-ofed-kernel)
+- **Kernel modules**
 
   They provide the kernel-side verbs API and low level device drivers that
   manage actual hardware initialization and resources sharing with user
@@ -204,24 +197,27 @@ DPDK and must be installed separately:
    Both libraries are BSD and GPL licensed. Linux kernel modules are GPL
    licensed.
 
-Currently supported by DPDK:
+Depending on system constraints and user preferences either RDMA core library
+with a recent enough Linux kernel release (recommended) or Mellanox OFED,
+which provides compatibility with older releases.
+
+Current RDMA core package and Linux kernel (recommended)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- Minimal Linux kernel version: 4.14.
+- Minimal RDMA core version: v15 (see `RDMA core installation documentation`_).
 
-- Mellanox OFED **4.1**.
-- Firmware version **2.36.5000** and above.
+.. _`RDMA core installation documentation`: https://raw.githubusercontent.com/linux-rdma/rdma-core/master/README.md
 
-Getting Mellanox OFED
-~~~~~~~~~~~~~~~~~~~~~
+.. _Mellanox_OFED_as_a_fallback:
 
-While these libraries and kernel modules are available on OpenFabrics
-Alliance's `website <https://www.openfabrics.org/>`_ and provided by package
-managers on most distributions, this PMD requires Ethernet extensions that
-may not be supported at the moment (this is a work in progress).
+Mellanox OFED as a fallback
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-`Mellanox OFED
-<http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers>`_
-includes the necessary support and should be used in the meantime. For DPDK,
-only libibverbs, libmlx4, mlnx-ofed-kernel packages and firmware updates are
-required from that distribution.
+- `Mellanox OFED`_ version: **4.2**.
+- firmware version: **2.42.5000** and above.
+
+.. _`Mellanox OFED`: http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers
 
 .. note::
 
@@ -229,15 +225,10 @@ required from that distribution.
    this DPDK release was developed and tested against is strongly
    recommended. Please check the `prerequisites`_.
 
-Supported NICs
---------------
+Installing Mellanox OFED
+^^^^^^^^^^^^^^^^^^^^^^^^
 
-* Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G)
-
-Quick Start Guide
------------------
-
-1. Download latest Mellanox OFED. For more info check the  `prerequisites`_.
+1. Download latest Mellanox OFED.
 
 2. Install the required libraries and kernel modules either by installing
    only the required set, or by installing the entire Mellanox OFED:
@@ -246,19 +237,19 @@ Quick Start Guide
 
    .. code-block:: console
 
-        ./mlnxofedinstall
+        ./mlnxofedinstall --dpdk --upstream-libs
 
    For SR-IOV hypervisors use:
 
    .. code-block:: console
 
-        ./mlnxofedinstall --enable-sriov -hypervisor
+        ./mlnxofedinstall --dpdk --upstream-libs --enable-sriov --hypervisor
 
    For SR-IOV virtual machine use:
 
    .. code-block:: console
 
-        ./mlnxofedinstall --guest
+        ./mlnxofedinstall --dpdk --upstream-libs --guest
 
 3. Verify the firmware is the correct one:
 
@@ -272,7 +263,19 @@ Quick Start Guide
 
         connectx_port_config
 
-   Or in the manual way:
+5. Continue with :ref:`section 2 of the Quick Start Guide <QSG_2>`.
+
+Supported NICs
+--------------
+
+* Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G)
+
+.. _qsg:
+
+Quick Start Guide
+-----------------
+
+1. Set all ports links to Ethernet
 
    .. code-block:: console
 
@@ -280,7 +283,15 @@ Quick Start Guide
         echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port0"
         echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port1"
 
-5. In case of bare metal or hypervisor, configure optimized steering mode
+   .. note::
+
+        If using Mellanox OFED one can permanently set the port link
+        to Ethernet using connectx_port_config tool provided by it.
+        :ref:`Mellanox_OFED_as_a_fallback`:
+
+.. _QSG_2:
+
+2. In case of bare metal or hypervisor, configure optimized steering mode
    by adding the following line to ``/etc/modprobe.d/mlx4_core.conf``:
 
    .. code-block:: console
@@ -292,7 +303,7 @@ Quick Start Guide
         If VLAN filtering is used, set log_num_mgm_entry_size=-1.
         Performance degradation can occur on this case.
 
-6. Restart the driver:
+3. Restart the driver:
 
    .. code-block:: console
 
@@ -304,7 +315,7 @@ Quick Start Guide
 
         service openibd restart
 
-7. Compile DPDK and you are ready to go. See instructions on
+4. Compile DPDK and you are ready to go. See instructions on
    :ref:`Development Kit Build System <Development_Kit_Build_System>`
 
 Performance tuning
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index e6e4407..c7d8826 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -59,6 +59,32 @@ New Features
   "Network Interface Controller Drivers" document for more details on this new
   driver.
 
+* **Updated mlx4 driver.**
+
+  Updated the mlx4 driver including the following changes:
+
+   * Isolated mode (rte_flow) can now be enabled anytime, not only during
+     initial device configuration.
+   * Flow rules now support up to 4096 priority levels usable at will by
+     applications.
+   * Enhanced error message to help debugging invalid/unsupported flow rules.
+   * Flow rules matching all multicast and promiscuous traffic are now allowed.
+   * No more software restrictions on flow rules with the RSS action, their
+     configuration is much more flexible.
+   * Significantly reduced memory footprint for Rx and Tx queue objects.
+   * While supported, UDP RSS is temporarily disabled due to a remaining issue
+     with its support in the Linux kernel.
+   * The new RSS implementation does not automatically spread traffic according
+     to the inner packet of VXLAN frames anymore, only the outer one (like
+     other PMDs).
+   * Partial (Tx only) support for secondary processes was broken and had to be
+     removed.
+   * Refactored driver to get rid of dependency on the components provided by
+     Mellanox OFED and instead rely on the current and public rdma-core
+     package and Linux version from now on.
+   * Removed compile-time limitation on number of device instances the PMD
+     can support.
+
 * **Updated mlx5 driver.**
 
   Updated the mlx5 driver including the following changes:
-- 
1.8.3.1

Re: [PATCH 0/3] doc: documentation update for mlx4 PMD

[Adrien Mazarguil]

On Thu, Nov 09, 2017 at 07:30:53PM +0200, Moti Haimovsky wrote:
> The following three commits update the mlx4 documentation to reflect
> the modifications done to the PMD and the new features added to it.
> 
> Moti Haimovsky (3):
>   doc: fix remove mlx4 Tx inline compilation option
>   doc: fix mlx4 drop inline receive support
>   doc: update mlx4 documentation
> 
>  doc/guides/nics/mlx4.rst               | 114 ++++++++++++++++-----------------
>  doc/guides/rel_notes/release_17_11.rst |  26 ++++++++
>  2 files changed, 80 insertions(+), 60 deletions(-)

For the series,

Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>

Thanks.

-- 
Adrien Mazarguil
6WIND

Re: [PATCH 0/3] doc: documentation update for mlx4 PMD

[Thomas Monjalon]

09/11/2017 18:37, Adrien Mazarguil:
> On Thu, Nov 09, 2017 at 07:30:53PM +0200, Moti Haimovsky wrote:
> > The following three commits update the mlx4 documentation to reflect
> > the modifications done to the PMD and the new features added to it.
> > 
> > Moti Haimovsky (3):
> >   doc: fix remove mlx4 Tx inline compilation option
> >   doc: fix mlx4 drop inline receive support
> >   doc: update mlx4 documentation
> > 
> >  doc/guides/nics/mlx4.rst               | 114 ++++++++++++++++-----------------
> >  doc/guides/rel_notes/release_17_11.rst |  26 ++++++++
> >  2 files changed, 80 insertions(+), 60 deletions(-)
> 
> For the series,
> 
> Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>

Applied, thanks