Re: [dpdk-dev] [PATCH v6 1/8] net/dpaa: add support for fmlib in dpdk

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 9/3/2020 4:24 AM, Hemant Agrawal wrote:
> Hi Ferruh,
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Wednesday, September 2, 2020 7:02 PM
>>>> On 9/1/2020 1:36 PM, Hemant Agrawal wrote:
>>>>> DPAA platorm MAC interface is known as FMAN i.e. Frame Manager.
>>>>> There are two ways to control it.
>>>>> 1. Statically configure the queues and classification rules before
>>>>> the start of the application using FMC tool.
>>>>> 2. Dynamically configure it within application by making API calls
>>>>> of fmlib.
>>>>>
>>>>> The fmlib or Frame Manager library provides an API on top of the
>>>>> Frame Manager driver ioctl calls, that provides a user space
>>>>> application with a simple way to configure driver parameters and PCD
>>>>> (parse - classify - distribute) rules.
>>>>>
>>>>> This patch integrates the base fmlib so that various queue config,
>>>>> RSS and classification related features can be supported on DPAA
>> platform.
>>>>>
>>>>> Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
>>>>> Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>>>>> ---
>>>>>  doc/guides/nics/dpaa.rst                  |   52 +-
>>>>>  doc/guides/platform/dpaa.rst              |   21 +-
>>>>>  drivers/net/dpaa/fmlib/dpaa_integration.h |   50 +
>>>>>  drivers/net/dpaa/fmlib/fm_ext.h           |  463 ++
>>>>>  drivers/net/dpaa/fmlib/fm_lib.c           |  561 ++
>>>>>  drivers/net/dpaa/fmlib/fm_pcd_ext.h       | 5787
>>>> +++++++++++++++++++++
>>>>>  drivers/net/dpaa/fmlib/fm_port_ext.h      | 3350 ++++++++++++
>>>>>  drivers/net/dpaa/fmlib/ncsw_ext.h         |  158 +
>>>>>  drivers/net/dpaa/fmlib/net_ext.h          |  411 ++
>>>>>  drivers/net/dpaa/meson.build              |    3 +-
>>>>>  10 files changed, 10849 insertions(+), 7 deletions(-)  create mode
>>>>> 100644 drivers/net/dpaa/fmlib/dpaa_integration.h
>>>>>  create mode 100644 drivers/net/dpaa/fmlib/fm_ext.h  create mode
>>>>> 100644 drivers/net/dpaa/fmlib/fm_lib.c  create mode 100644
>>>>> drivers/net/dpaa/fmlib/fm_pcd_ext.h
>>>>>  create mode 100644 drivers/net/dpaa/fmlib/fm_port_ext.h
>>>>>  create mode 100644 drivers/net/dpaa/fmlib/ncsw_ext.h  create mode
>>>>> 100644 drivers/net/dpaa/fmlib/net_ext.h
>>>>>
>>>>> diff --git a/doc/guides/nics/dpaa.rst b/doc/guides/nics/dpaa.rst
>>>>> index
>>>>> 17839a920..7e6010471 100644
>>>>> --- a/doc/guides/nics/dpaa.rst
>>>>> +++ b/doc/guides/nics/dpaa.rst
>>>>> @@ -1,5 +1,5 @@
>>>>>  ..  SPDX-License-Identifier: BSD-3-Clause
>>>>> -    Copyright 2017 NXP
>>>>> +    Copyright 2017,2020 NXP
>>>>>
>>>>>
>>>>>  DPAA Poll Mode Driver
>>>>> @@ -21,6 +21,7 @@ Contents summary
>>>>>
>>>>>  - DPAA overview
>>>>>  - DPAA driver architecture overview
>>>>> +- FMAN configuration tools and library
>>>>>
>>>>>  .. _dpaa_overview:
>>>>>
>>>>> @@ -285,6 +286,55 @@ for details.
>>>>>        Done
>>>>>        testpmd>
>>>>>
>>>>> +FMAN Config
>>>>> +-----------
>>>>> +
>>>>> +Frame Manager is also responsible for parser, classify and
>>>>> +distribute functionality in the DPAA.
>>>>> +
>>>>> +   FMAN supports:
>>>>> +   Packet parsing at wire speed. It supports standard protocols parsing
>> and
>>>>> +   identification by HW
>>>> (VLAN/IP/UDP/TCP/SCTP/PPPoE/PPP/MPLS/GRE/IPSec).
>>>>> +   It supports non-standard UDF header parsing for custom protocols.
>>>>> +   Classification / Distribution: Coarse classification based on
>>>>> + Key
>>>> generation
>>>>> +   Hash and exact match lookup
>>>>> +
>>>>> +FMC - FMAN Configuration Tool
>>>>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>> +   This tool is available in User Space. The tool is used to configure FMAN
>>>>> +   Physical (MAC) or Ephemeral (OH)ports for Parse/Classify/distribute.
>>>>> +   The PCDs can be hash based where a set of fields are key input for
>> hash
>>>>> +   generation within FMAN keygen. The hash value is used to
>>>>> +generate a
>>>> FQID for
>>>>> +   frame. There is a provision to setup exact match lookup too where
>> field
>>>>> +   values within a packet drives corresponding FQID.
>>>>> +   Currently it works on XML file inputs.
>>>>> +
>>>>> +   Limitations:
>>>>> +   1.For Dynamic Configuration change, currently no support is available.
>>>>> +   E.g. enable/disable a port, a operator (set of VLANs and associate
>> rules).
>>>>> +
>>>>> +   2.During FMC configuration, port for which policy is being configured
>> is
>>>>> +   brought down and the policy is flushed on port before new policy
>>>>> + is
>>>> updated
>>>>> +   for the port. Support is required to add/append/delete etc.
>>>>> +
>>>>> +   3.FMC, being a separate user-space application, needs to be
>>>>> + invoked
>>>> from
>>>>> +   Shell.
>>>>> +
>>>>> +
>>>>> +   The details can be found in FMC Doc at:
>>>>> +   `Frame Mnager Configuration Tool
>>>>
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
>>>> w.nxp.com%2Fdocs%2Fen%2Fapplication-
>>>>
>> note%2FAN4760.pdf&amp;data=02%7C01%7Chemant.agrawal%40nxp.com%
>>>>
>> 7Cdb7eda614491479976c508d84e8e8c42%7C686ea1d3bc2b4c6fa92cd99c5c30
>>>>
>> 1635%7C0%7C1%7C637345721434145756&amp;sdata=ynJeLWqw0hPTya5IRt
>>>> GviovNA7mAGIZ8k3XSFcZyLP4%3D&amp;reserved=0>`_.
>>>>> +
>>>>> +FMLIB
>>>>> +~~~~~
>>>>> +   The Frame Manager library provides an API on top of the Frame
>>>> Manager driver
>>>>> +   ioctl calls, that provides a user space application with a simple way to
>>>>> +   configure driver parameters and PCD (parse - classify - distribute)
>> rules.
>>>>> +
>>>>> +   This is an alternate to the FMC based configuration. This libray
>> provides
>>>>> +   direct ioctl based interfaces for FMAN configuration as used by
>>>>> + the FMC
>>>> tool
>>>>> +   as well. This helps in overcoming the main limitaiton of FMC - i.e. lack
>>>>> +   of dynamic configuration.
>>>>> +
>>>>>  Limitations
>>>>>  -----------
>>>>>
>>>>> diff --git a/doc/guides/platform/dpaa.rst
>>>>> b/doc/guides/platform/dpaa.rst index 6005f2221..20a0e3932 100644
>>>>> --- a/doc/guides/platform/dpaa.rst
>>>>> +++ b/doc/guides/platform/dpaa.rst
>>>>> @@ -58,17 +58,28 @@ compatible board:
>>>>>
>>>>>  4. **FMC Tool**
>>>>>
>>>>> -   Before any DPDK application can be executed, the Frame Manager
>>>> Configuration
>>>>> -   Tool (FMC) need to be executed to set the configurations of the
>> queues.
>>>> This
>>>>> +   If one is planning to use more than 1 Recv queue and hardware
>>>> capability to
>>>>> +   parse, classify and distribute the packets, the Frame Manager
>>>> Configuration
>>>>> +   Tool (FMC) need to be executed to set the configurations of the
>>>>> + queues
>>>> before
>>>>> +   running the DPAA based DPDK application. This setting is persistent,
>> the
>>>>> +   configuration will remain in the hardware till it is
>>>>> + re-configured. This
>>>>>     includes the queue state, RSS and other policies.
>>>>>     This tool can be obtained from `NXP (Freescale) Public Git
>>>>> Repository
>>>>
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fso
>>>> ur
>>>> ce.codeaurora.org%2Fexternal%2Fqoriq%2Fqoriq-
>>>>
>> components%2Ffmc&amp;data=02%7C01%7Chemant.agrawal%40nxp.com%
>>>>
>> 7Cdb7eda614491479976c508d84e8e8c42%7C686ea1d3bc2b4c6fa92cd99c5c30
>>>>
>> 1635%7C0%7C1%7C637345721434145756&amp;sdata=pwOJGxY%2BGTkcpdk
>>>> O%2BKwoVejywmOeGsWbudo9OPSDqPU%3D&amp;reserved=0>`_.
>>>>>
>>>>>     This tool needs configuration files which are available in the
>>>>>     :ref:`DPDK Extra Scripts <extra_scripts>`, described below for
>>>>> DPDK
>>>> usages.
>>>>>
>>>>> -As an alternative method, DPAA PMD can also be executed using
>>>>> images provided -as part of SDK from NXP. The SDK includes all the
>>>>> above prerequisites necessary -to bring up a DPAA board.
>>>>> +   Note that DPAA PMD can also be executed using images provided
>>>>> +   as part of SDK from NXP. The SDK includes all the above prerequisites
>>>>> +   necessary (i.e. fmc tool) to bring up a DPAA board.
>>>>> +
>>>>> +   As an alternate method, DPAA PMDs starting from DPDK 20.11 also
>>>> support the
>>>>> +   fmlib library integration. The driver will detect about any existing FMC
>>>>> +   based config (if /tmp/fmc.bin is present). DPAA FMD will be used
>>>>> + only if
>>>> no
>>>>> +   previous fmc config is existing.
>>>>> +
>>>>> +   Note that fmlib based integratin rely on underlying fmd driver in
>> kernel,
>>>>> +   which is available as part of NXP kernel or NXP SDK.
>>>>
>>>> Hi Hemant,
>>>>
>>>> Thanks for the documentation.
>>>>
>>>> Can it be possible to give a little more detail related to the 'fmd'
>>>> module, like its folder etc..
>>>>
>>>> Another thing is with current patch 'dpaa' driver becomes completely
>>>> dependent to the 'fmd' kernel module [1] and PMD won't run without
>>>> it, was the dependency always there indirectly?
>>>
>>> [Hemant]  I thought, I was clear in my documentation.
>>> " This libray provides direct ioctl based interfaces for FMAN configuration as
>> used by the FMC tool  as well"
>>> FMD always exist in the kernel for DPAA.  At present FMC is using it via
>> IOCTL calls and FMC is indepdent tools with limitations for DPDK.
>>> Now, we have directly used those ioctl calls via fmlib in DPDK.
>>>
>>>> And does it make sense to support no 'fmd' kernel module case, as
>>>> done previously?
>>> [Hemant] FMD is already part of DPAA supported kernels.
>>
>>
>> OK, so what I understand is 'fmd' module was already an indirect
>> dependency and there is no point to support 'dpaa' for no 'fmd' available
>> case.
>>
>> Can you just give the location of the 'fmd' module, in case someone would
>> like to check the corresponding code for an ioctl call, it may help.
> 
> [Hemant] Thanks for the suggestion. 
>  Yes, I will do that in next version. 
> 
> FYI - the location is as follows: 
> https://source.codeaurora.org/external/qoriq/qoriq-components/linux/tree/drivers/net/ethernet/freescale/sdk_fman?h=linux-4.19-rt
> 
> Please let me know if you have any more comments on the patch. 
> 

Rest looks good to me, thanks.
Only there are below checkpatch warnings, if you will send new version can you
please check them too:

### net/dpaa: add support for fmlib in dpdk







WARNING:TYPO_SPELLING: 'libray' may be misspelled - perhaps 'library'?



#91: FILE: doc/guides/nics/dpaa.rst:333:



+   This is an alternate to the FMC based configuration. This libray provides







WARNING:REPEATED_WORD: Possible repeated word: 'default'



#1627: FILE: drivers/net/dpaa/fmlib/fm_pcd_ext.h:393:



+ *               By default default values are 0.







WARNING:REPEATED_WORD: Possible repeated word: 'if'
#3785: FILE: drivers/net/dpaa/fmlib/fm_pcd_ext.h:2551:
+               /**< 0-7, Relevant only if if update_type =


WARNING:REPEATED_WORD: Possible repeated word: 'default'
#5498: FILE: drivers/net/dpaa/fmlib/fm_pcd_ext.h:4264:
+ *               By default default values are 0.