[dpdk-dev] [PATCH v4 14/15] doc: add event mode support to ipsec-secgw

[Lukasz Bartosik <lbartosik@marvell.com>]

Document addition of event mode support
to ipsec-secgw application.

Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
---
 doc/guides/sample_app_ug/ipsec_secgw.rst | 138 ++++++++++++++++++++++++++-----
 1 file changed, 117 insertions(+), 21 deletions(-)

diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst
index 5ec9b1e..fddd88c 100644
--- a/doc/guides/sample_app_ug/ipsec_secgw.rst
+++ b/doc/guides/sample_app_ug/ipsec_secgw.rst
@@ -1,5 +1,6 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright(c) 2016-2017 Intel Corporation.
+    Copyright (C) 2020 Marvell International Ltd.
 
 IPsec Security Gateway Sample Application
 =========================================
@@ -61,6 +62,44 @@ The Path for the IPsec Outbound traffic is:
 *  Routing.
 *  Write packet to port.
 
+The application supports two modes of operation: poll mode and event mode.
+
+* In the poll mode a core receives packets from statically configured list
+  of eth ports and eth ports' queues.
+
+* In the event mode a core receives packets as events. After packet processing
+  is done core submits them back as events to an event device. This enables
+  multicore scaling and HW assisted scheduling by making use of the event device
+  capabilities. The event mode configuration is predefined. All packets reaching
+  given eth port will arrive at the same event queue. All event queues are mapped
+  to all event ports. This allows all cores to receive traffic from all ports.
+  Since the underlying event device might have varying capabilities, the worker
+  threads can be drafted differently to maximize performance. For example, if an
+  event device - eth device pair has Tx internal port, then application can call
+  rte_event_eth_tx_adapter_enqueue() instead of regular rte_event_enqueue_burst().
+  So a thread which assumes that the device pair has internal port will not be the
+  right solution for another pair. The infrastructure added for the event mode aims
+  to help application to have multiple worker threads by maximizing performance from
+  every type of event device without affecting existing paths/use cases. The worker
+  to be used will be determined by the operating conditions and the underlying device
+  capabilities. **Currently the application provides non-burst, internal port worker
+  threads and supports inline protocol only.** It also provides infrastructure for
+  non-internal port however does not define any worker threads.
+
+Additionally the event mode introduces two submodes of processing packets:
+
+* Driver submode: This submode has bare minimum changes in the application to support
+  IPsec. There are no lookups, no routing done in the application. And for inline
+  protocol use case, the worker thread resembles l2fwd worker thread as the IPsec
+  processing is done entirely in HW. This mode can be used to benchmark the raw
+  performance of the HW. The driver submode is selected with --single-sa option
+  (used also by poll mode). When --single-sa option is used in conjution with event
+  mode then index passed to --single-sa is ignored.
+
+* App submode: This submode has all the features currently implemented with the
+  application (non librte_ipsec path). All the lookups, routing follows existing
+  methods and report numbers that can be compared against regular poll mode
+  benchmark numbers.
 
 Constraints
 -----------
@@ -94,13 +133,18 @@ The application has a number of command line options::
                         -p PORTMASK -P -u PORTMASK -j FRAMESIZE
                         -l -w REPLAY_WINOW_SIZE -e -a
                         -c SAD_CACHE_SIZE
+                        -s NUMBER_OF_MBUFS_IN_PACKET_POOL
+                        -f CONFIG_FILE_PATH
                         --config (port,queue,lcore)[,(port,queue,lcore]
                         --single-sa SAIDX
+                        --cryptodev_mask MASK
+                        --transfer-mode MODE
+                        --event-schedule-type TYPE
                         --rxoffload MASK
                         --txoffload MASK
-                        --mtu MTU
                         --reassemble NUM
-                        -f CONFIG_FILE_PATH
+                        --mtu MTU
+                        --frag-ttl FRAG_TTL_NS
 
 Where:
 
@@ -138,12 +182,38 @@ Where:
     Zero value disables cache.
     Default value: 128.
 
-*   ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues
-    from which ports are mapped to which cores.
+*   ``-s``: sets number of mbufs in packet pool, if not provided number of mbufs
+    will be calculated based on number of cores, eth ports and crypto queues.
+
+*   ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
+    configuration items for running the application (See Configuration file
+    syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
+    **ONLY** the UNIX format configuration file is accepted.
+
+*   ``--config (port,queue,lcore)[,(port,queue,lcore)]``: in poll mode determines
+    which queues from which ports are mapped to which cores. In event mode this
+    is required for eth ports initialization only. Afterwards packets are dynamically
+    scheduled to cores by HW.
+
+*   ``--single-sa SAIDX``: in poll mode use a single SA for outbound traffic,
+    bypassing the SP on both Inbound and Outbound. This option is meant for
+    debugging/performance purposes. In event mode selects driver submode, SA index
+    value is ignored.
 
-*   ``--single-sa SAIDX``: use a single SA for outbound traffic, bypassing the SP
-    on both Inbound and Outbound. This option is meant for debugging/performance
-    purposes.
+*   ``--cryptodev_mask MASK``: hexadecimal bitmask of the crypto devices
+    to configure.
+
+*   ``--transfer-mode MODE``: sets operating mode of the application
+    "poll"  : packet transfer via polling (default)
+    "event" : Packet transfer via event device
+
+*   ``--event-schedule-type TYPE``: queue schedule type, applies only when
+    --transfer-mode is set to event.
+    "ordered"  : Ordered (default)
+    "atomic"   : Atomic
+    "parallel" : Parallel
+    When --event-schedule-type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event
+    device will ensure the ordering. Ordering will be lost when tried in PARALLEL.
 
 *   ``--rxoffload MASK``: RX HW offload capabilities to enable/use on this port
     (bitmask of DEV_RX_OFFLOAD_* values). It is an optional parameter and
@@ -155,6 +225,10 @@ Where:
     allows user to disable some of the TX HW offload capabilities.
     By default all HW TX offloads are enabled.
 
+*   ``--reassemble NUM``: max number of entries in reassemble fragment table.
+    Zero value disables reassembly functionality.
+    Default value: 0.
+
 *   ``--mtu MTU``: MTU value (in bytes) on all attached ethernet ports.
     Outgoing packets with length bigger then MTU will be fragmented.
     Incoming packets with length bigger then MTU will be discarded.
@@ -167,26 +241,17 @@ Where:
     Should be lower for low number of reassembly buckets.
     Valid values: from 1 ns to 10 s. Default value: 10000000 (10 s).
 
-*   ``--reassemble NUM``: max number of entries in reassemble fragment table.
-    Zero value disables reassembly functionality.
-    Default value: 0.
-
-*   ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all
-    configuration items for running the application (See Configuration file
-    syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified.
-    **ONLY** the UNIX format configuration file is accepted.
-
 
 The mapping of lcores to port/queues is similar to other l3fwd applications.
 
-For example, given the following command line::
+For example, given the following command line to run application in poll mode::
 
     ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048       \
-           --vdev "crypto_null" -- -p 0xf -P -u 0x3      \
+           --vdev "crypto_null" -- -p 0xf -P -u 0x3             \
            --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)"       \
-           -f /path/to/config_file                              \
+           -f /path/to/config_file --transfer-mode poll         \
 
-where each options means:
+where each option means:
 
 *   The ``-l`` option enables cores 20 and 21.
 
@@ -200,7 +265,7 @@ where each options means:
 
 *   The ``-P`` option enables promiscuous mode.
 
-*   The ``-u`` option sets ports 1 and 2 as unprotected, leaving 2 and 3 as protected.
+*   The ``-u`` option sets ports 0 and 1 as unprotected, leaving 2 and 3 as protected.
 
 *   The ``--config`` option enables one queue per port with the following mapping:
 
@@ -228,6 +293,37 @@ where each options means:
     **note** the parser only accepts UNIX format text file. Other formats
     such as DOS/MAC format will cause a parse error.
 
+*   The ``--transfer-mode`` option selects poll mode for processing packets.
+
+Similarly for example, given the following command line to run application in
+event app mode::
+
+    ./build/ipsec-secgw -c 0x3 -- -P -p 0x3 -u 0x1       \
+           --config "(1,0,0),(0,0,1)"                    \
+           -f /path/to/config_file --transfer-mode event \
+           --event-schedule-type parallel                \
+
+where each option means:
+
+*   The ``-c`` option selects cores 0 and 1 to run on.
+
+*   The ``-P`` option enables promiscuous mode.
+
+*   The ``-p`` option enables ports (detected) 0 and 1.
+
+*   The ``-u`` option sets ports 0 as unprotected, leaving 1 as protected.
+
+*   The ``--config`` option provides configuration for eth ports initialization
+    only. Afterwards packets are dynamically scheduled to cores by HW.
+
+*   The ``-f /path/to/config_file`` option has the same behavior as in poll
+    mode example.
+
+*   The ``--transfer-mode`` option selects event mode for processing packets.
+
+*   The ``--event-schedule-type`` option selects parallel ordering of event queues.
+
+
 Refer to the *DPDK Getting Started Guide* for general information on running
 applications and the Environment Abstraction Layer (EAL) options.
 
-- 
2.7.4