Re: [PATCH v5 2/2] ethdev: add traffic management API

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Fri, 9 Jun 2017 17:51:15 +0100
> From: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> To: dev@dpdk.org
> CC: thomas@monjalon.net, jerin.jacob@caviumnetworks.com,
>  balasubramanian.manoharan@cavium.com, hemant.agrawal@nxp.com,
>  shreyansh.jain@nxp.com, jasvinder.singh@intel.com, wenzhuo.lu@intel.com
> Subject: [PATCH v5 2/2] ethdev: add traffic management API
> X-Mailer: git-send-email 2.7.4
> 
> This patch introduces the generic ethdev API for the traffic manager
> capability, which includes: hierarchical scheduling, traffic shaping,
> congestion management, packet marking.
> 
> Main features:
> - Exposed as ethdev plugin capability (similar to rte_flow)
> - Capability query API per port, per level and per node
> - Scheduling algorithms: Strict Priority (SP), Weighed Fair Queuing (WFQ)
> - Traffic shaping: single/dual rate, private (per node) and shared (by
>   multiple nodes) shapers
> - Congestion management for hierarchy leaf nodes: algorithms of tail drop,
>   head drop, WRED; private (per node) and shared (by multiple nodes) WRED
>   contexts
> - Packet marking: IEEE 802.1q (VLAN DEI), IETF RFC 3168 (IPv4/IPv6 ECN for
>   TCP and SCTP), IETF RFC 2597 (IPv4 / IPv6 DSCP)
> 
> Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Acked-by: Balasubramanian.Manoharan <balasubramanian.manoharan@caviumnetworks.com>
> Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
> ---
> Changes in v5:
> - Implemented feedback from Jerin [8]
> 	- Add level parameter to node add API function
> 	- Doxygen: fixed comments applicable to field below/before
> 	- Doxygen: added missing @see
> 	- Doxygen: fixed hooks in doc/api/doxy-api-index.md
> 	- Doxygen: fixed table rendering
> 	- Added copyright on API header file from Cavium and NXP to
> 	  existing Intel copyright
> 	- MANTAINERS: added next-tm tree
> - Added V4 ACKs from Jerin, Bala and Hemant
> 
> Changes in v4:
> - Implemented feedback from Hemant [6]
> 	- Capability API: Reworked the port, level and node capability API
> 	  data structure to remove confusion due to "summary across all
> 	  nodes" approach, which made it unclear whether a particular
> 	  capability is supported by all nodes or by at least one node.
> 	- Capability API: Added flags for "all nodes have identical
> 	  capability set"
> 	- Suspended state: documented the required behavior in Doxygen
> 	  description
> - Implemented feedback from Jerin [7]
> 	- Node add: added level parameter (see new API function:
> 	  rte_tm_node_add_check_level())
> 	- RTE_TM_ETH_FRAMING_OVERHEAD, RTE_TM_ETH_FRAMING_OVERHEAD_FCS:
> 	  documented their usage in their Doxygen description
> 	- Capability API: for each function, mention the related
> 	  capability field (Doxygen @see)
> 	- stats_mask, capability_mask: document the enum flags used to
> 	  build each mask (Doxygen @see)
> 	- Rename rte_tm_get_leaf_nodes() to
> 	  rte_tm_get_number_of_leaf_nodes()
> 	- Doxygen: add @param[in, out] to the description of all API funcs
> 	- Doxygen: fix hooks in doc/api/doxy-api-index.md
> - Rename rte_tm_hierarchy_set() to rte_tm_hierarchy_commit(), improved
>   Doxygen description
> - Node add, node delete: improved Doxygen description
> - Fixed incorrect design assumption that packet-based weight mode for WFQ
>   is identical to WRR. As result, removed all references to WRR support.
>   Renamed the "scheduling mode" node parameters to "wfq_weight_mode".
> 
> Changes in v3:
> - Implemented feedback from Jerin [5]
> - Changed naming convention: scheddev -> tm
> - Improvements on the capability API:
> 	- Specification of marking capabilities per color
> 	- WFQ/WRR groups: sp_n_children_max ->
> 	  wfq_wrr_n_children_per_group_max, added wfq_wrr_n_groups_max,
> 	  improved description of both, improved description of
> 	  wfq_wrr_weight_max
> 	- Dynamic updates: added KEEP_LEVEL and CHANGE_LEVEL for parent
> 	  update
> - Enforced/documented restrictions for root node (node_add() and
>   update())
> - Enforced/documented shaper profile restrictions on PIR: PIR != 0,
>   PIR >= CIR
> - Turned repetitive code in rte_tm.c into macro
> - Removed dependency on rte_red.h file (added RED params to rte_tm.h)
> - Color: removed "e_" from color names enum
> - Fixed small Doxygen style issues
> 
> Changes in v2:
> - Implemented feedback from Hemant [4]
> - Improvements on the capability API
> 	- Added capability API for hierarchy level
> 	- Merged stats capability into the capability API
> 	- Added dynamic updates
> 	- Added non-leaf/leaf union to the node capability structure
> 	- Renamed sp_priority_min to sp_n_priorities_max, added
> 	  clarifications
> 	- Fixed description for sp_n_children_max
> - Clarified and enforced rule on node ID range for leaf and non-leaf nodes
> 	- Added API functions to get node type (i.e. leaf/non-leaf):
> 	  get_leaf_nodes(), node_type_get()
> - Added clarification for the root node: its creation, parent, role
> 	- Macro NODE_ID_NULL as root node's parent
> 	- Description of the node_add() and node_parent_update() API funcs
> - Added clarification for the first time add vs. subsequent updates rule
> 	- Cleaned up the description for the node_add() function
> - Statistics API improvements
> 	- Merged stats capability into the capability API
> 	- Added API function node_stats_update()
> 	- Added more stats per packet color
> - Added more error types
> - Fixed small Doxygen style issues
> 
> Changes in v1 (since RFC [1]):
> - Implemented as ethdev plugin (similar to rte_flow) as opposed to more
>   monolithic additions to ethdev itself
> - Implemented feedback from Jerin [2] and Hemant [3]. Implemented all the
>   suggested items with only one exception, see the long list below,
>   hopefully nothing was forgotten.
>     - The item not done (hopefully for a good reason): driver-generated
>       object IDs. IMO the choice to have application-generated object IDs
>       adds marginal complexity to the driver (search ID function
>       required), but it provides huge simplification for the application.
>       The app does not need to worry about building & managing tree-like
>       structure for storing driver-generated object IDs, the app can use
>       its own convention for node IDs depending on the specific hierarchy
>       that it needs. Trivial example: identify all level-2 nodes with IDs
>       like 100, 200, 300, … and the level-3 nodes based on their level-2
>       parents: 110, 120, 130, 140, …, 210, 220, 230, 240, …, 310, 320,
>       330, … and level-4 nodes based on their level-3 parents: 111, 112,
>       113, 114, …, 121, 122, 123, 124, …). Moreover, see the change log
>       for the other related simplification that was implemented: leaf
>       nodes now have predefined IDs that are the same with their Ethernet
>       TX queue ID ( therefore no translation is required for leaf nodes).
> - Capability API. Done per port and per node as well.
> - Dual rate shapers
> - Added configuration of private shaper (per node) directly from the
>   shaper profile as part of node API (no shaper ID needed for private
>   shapers), while the shared shapers are configured outside of the node
>   API using shaper profile and communicated to the node using shared
>   shaper ID. So there is no configuration overhead for shared shapers if
>   the app does not use any of them.
> - Leaf nodes now have predefined IDs that are the same with their Ethernet
>   TX queue ID (therefore no translation is required for leaf nodes). This
>   is also used to differentiate between a leaf node and a non-leaf node.
> - Domain-specific errors to give a precise indication of the error cause
>   (same as done by rte_flow)
> - Packet marking API
> - Packet length optional adjustment for shapers, positive (e.g. for adding
>   Ethernet framing overhead of 20 bytes) or negative (e.g. for rate
>   limiting based on IP packet bytes)
> 
> [1] RFC: http://dpdk.org/ml/archives/dev/2016-November/050956.html
> [2] Jerin’s feedback on RFC: http://www.dpdk.org/ml/archives/dev/2017-January/054484.html
> [3] Hemant’s feedback on RFC: http://www.dpdk.org/ml/archives/dev/2017-January/054866.html
> [4] Hemant's feedback on v1: http://www.dpdk.org/ml/archives/dev/2017-February/058033.html
> [5] Jerin's feedback on v1: http://www.dpdk.org/ml/archives/dev/2017-March/058895.html
> [6] Hemant's feedback on v3: http://www.dpdk.org/ml/archives/dev/2017-March/062354.html
> [7] Jerin's feedback on v3: http://www.dpdk.org/ml/archives/dev/2017-April/063429.html
> [8] Jerin's feedback on v4: http://www.dpdk.org/ml/archives/dev/2017-May/066932.html
> 
> 
>  MAINTAINERS                            |    5 +
>  lib/librte_ether/Makefile              |    5 +-
>  lib/librte_ether/rte_ether_version.map |   30 +
>  lib/librte_ether/rte_tm.c              |  438 ++++++++
>  lib/librte_ether/rte_tm.h              | 1899 ++++++++++++++++++++++++++++++++
>  lib/librte_ether/rte_tm_driver.h       |  366 ++++++
>  6 files changed, 2742 insertions(+), 1 deletion(-)
>  create mode 100644 lib/librte_ether/rte_tm.c
>  create mode 100644 lib/librte_ether/rte_tm.h
>  create mode 100644 lib/librte_ether/rte_tm_driver.h

Please update the the missing the doxygen hooks in doc/api/doxy-api-index.md.

If it makes sense, then add (@see) for the exact capability field for following
functions.

rte_tm_node_wred_context_update
rte_tm_node_shared_wred_context_update
rte_tm_node_shaper_update
rte_tm_node_shared_shaper_update