From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Kavanagh Subject: [PATCH v5 0/6] Support TCP/IPv4, VxLAN and GRE GSO in DPDK Date: Thu, 28 Sep 2017 23:13:47 +0100 Message-ID: <1506636833-25851-1-git-send-email-mark.b.kavanagh@intel.com> References: <1505806379-71355-1-git-send-email-jiayu.hu@intel.com> Cc: jiayu.hu@intel.com, jianfeng.tan@intel.com, konstantin.ananyev@intel.com, ferruh.yigit@intel.com, thomas@monjalon.net, Mark Kavanagh To: dev@dpdk.org Return-path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 471E6107A for ; Fri, 29 Sep 2017 00:14:30 +0200 (CEST) In-Reply-To: <1505806379-71355-1-git-send-email-jiayu.hu@intel.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Generic Segmentation Offload (GSO) is a SW technique to split large packets into small ones. Akin to TSO, GSO enables applications to operate on large packets, thus reducing per-packet processing overhead. To enable more flexibility to applications, DPDK GSO is implemented as a standalone library. Applications explicitly use the GSO library to segment packets. This patch adds GSO support to DPDK for specific packet types: specifically, TCP/IPv4, VxLAN, and GRE. The first patch introduces the GSO API framework. The second patch adds GSO support for TCP/IPv4 packets (containing an optional VLAN tag). The third patch adds GSO support for VxLAN packets that contain outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or outer VLAN tags). The fourth patch adds GSO support for GRE packets that contain outer IPv4, and inner TCP/IPv4 headers (with optional outer VLAN tag). The fifth patch in the series enables TCP/IPv4, VxLAN, and GRE GSO in testpmd's checksum forwarding engine. The final patch in the series adds GSO documentation to the programmer's guide. The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using iperf. Setup for the test is described as follows: a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same machine, together physically. b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding engine with "retry". c. Select IP and TCP HW checksum calculation for P0; select TCP HW checksum calculation for vhost-user port. d. Launch a VM with csum and tso offloading enabled. e. Run iperf-client on virtio-net port in the VM to send TCP packets. With enabling csum and tso, the VM can send large TCP/IPv4 packets (mss is up to 64KB). f. P1 is assigned to linux kernel and enabled kernel GRO. Run iperf-server on P1. We conduct three iperf tests: test-1: enable GSO for P0 in testpmd, and set max GSO segment length to 1518B. Run two iperf-client in the VM. test-2: enable TSO for P0 in testpmd, and set TSO segsz to 1518B. Run two iperf-client in the VM. test-3: disable GSO and TSO in testpmd. Run two iperf-client in the VM. Throughput of the above three tests: test-1: 9.4Gbps test-2: 9.5Gbps test-3: 3Mbps Unlike TCP packets, VMs can't send large VxLAN or GRE packets. The max length of tunnelled packets from VMs is 1514B. So current experiment method can't be used to measure VxLAN and GRE GSO performance, but simply test the functionality via setting small GSO segment length (e.g. 500B). To test VxLAN GSO functionality, we use the following setup: a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same machine, together physically. b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding engine with "retry". c. Testpmd commands: - csum parse-tunnel on "P0" - csum parse-tunnel on "vhost-user port" - csum set outer-ip hw "P0" - csum set ip hw "P0" - csum set tcp hw "P0" - csum set tcp hw "vhost-user port" - set port "P0" gso on - set gso segsz 500 d. Launch a VM with csum and tso offloading enabled. e. Create a vxlan port for the virtio-net port in the VM. Run iperf-client on the VxLAN port, so TCP packets are VxLAN encapsulated. However, the max packet length is 1514B. f. P1 is assigned to linux kernel and disabled kernel GRO. Similarly, create a VxLAN port for P1, and run iperf-server on the VxLAN port. In testpmd, we can see the length of all packets sent from P0 is smaller than or equal to 500B. Additionally, the packets arriving in P1 is encapsulated and is smaller than or equal to 500B. Change log ========== v5: - add GSO section to the programmer's guide. - use MF or (previously 'and') offset to check if a packet is IP fragmented. - move 'update_header' helper functions to gso_common.h. - move txp/ipv4 'update_header' function to gso_tcp4.c. - move tunnel 'update_header' function to gso_tunnel_tcp4.c. - add offset parameter to 'update_header' functions. - combine GRE and VxLAN tunnel header update functions into a single function. - correct typos and errors in comments/commit messages. v4: - use ol_flags instead of packet_type to decide which segmentation function to use. - use MF and offset to check if a packet is IP fragmented, instead of using DF. - remove ETHER_CRC_LEN from gso segment payload length calculation. - refactor internal header update and other functions. - remove RTE_GSO_IPID_INCREASE. - add some of GSO documents. - set the default GSO length to 1514 and fill PKT_TX_TCP_SEG for the packets sent from GSO-enabled ports in testpmd. v3: - support all IPv4 header flags, including RTE_PTYPE_(INNER_)L3_IPV4, RTE_PTYPE_(INNER_)L3_IPV4_EXT and RTE_PTYPE_(INNER_)L3_IPV4_EXT_ UNKNOWN. - fill mbuf->packet_type instead of using rte_net_get_ptype() in csumonly.c, since rte_net_get_ptype() doesn't support vxlan. - store the input packet into pkts_out inside gso_tcp4_segment() and gso_tunnel_tcp4_segment() instead of rte_gso_segment(), when no GSO is performed. - add missing incldues. - optimize file names, function names and function description. - fix one bug in testpmd. v2: - merge data segments whose data_len is less than mss into a large data segment in gso_do_segment(). - use mbuf->packet_type/l2_len/l3_len etc. instead of parsing the packet header in rte_gso_segment(). - provide IP id macros for applications to select fixed or incremental IP ids. - change the defination of gso_types in struct rte_gso_ctx. - replace rte_pktmbuf_detach() with rte_pktmbuf_free(). - refactor gso_update_pkt_headers(). - change the return value of rte_gso_segment(). - remove parameter checks in rte_gso_segment(). - use rte_net_get_ptype() in app/test-pmd/csumonly.c to fill mbuf->packet_type. - add a new GSO command in testpmd to show GSO configuration for ports. - misc: fix typo and optimize function description. Jiayu Hu (3): gso: add Generic Segmentation Offload API framework gso: add TCP/IPv4 GSO support app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO Mark Kavanagh (3): gso: add VxLAN GSO support gso: add GRE GSO support doc: add GSO programmer's guide MAINTAINERS | 6 + app/test-pmd/cmdline.c | 178 ++++++++ app/test-pmd/config.c | 24 ++ app/test-pmd/csumonly.c | 69 ++- app/test-pmd/testpmd.c | 13 + app/test-pmd/testpmd.h | 10 + config/common_base | 5 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + .../generic_segmentation_offload_lib.rst | 256 +++++++++++ .../prog_guide/img/gso-output-segment-format.svg | 313 ++++++++++++++ doc/guides/prog_guide/img/gso-three-seg-mbuf.svg | 477 +++++++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_17_11.rst | 19 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 46 ++ lib/Makefile | 2 + lib/librte_eal/common/include/rte_log.h | 1 + lib/librte_gso/Makefile | 52 +++ lib/librte_gso/gso_common.c | 153 +++++++ lib/librte_gso/gso_common.h | 171 ++++++++ lib/librte_gso/gso_tcp4.c | 106 +++++ lib/librte_gso/gso_tcp4.h | 74 ++++ lib/librte_gso/gso_tunnel_tcp4.c | 129 ++++++ lib/librte_gso/gso_tunnel_tcp4.h | 75 ++++ lib/librte_gso/rte_gso.c | 107 +++++ lib/librte_gso/rte_gso.h | 145 +++++++ lib/librte_gso/rte_gso_version.map | 7 + mk/rte.app.mk | 1 + 28 files changed, 2437 insertions(+), 5 deletions(-) create mode 100644 doc/guides/prog_guide/generic_segmentation_offload_lib.rst create mode 100644 doc/guides/prog_guide/img/gso-output-segment-format.svg create mode 100644 doc/guides/prog_guide/img/gso-three-seg-mbuf.svg create mode 100644 lib/librte_gso/Makefile create mode 100644 lib/librte_gso/gso_common.c create mode 100644 lib/librte_gso/gso_common.h create mode 100644 lib/librte_gso/gso_tcp4.c create mode 100644 lib/librte_gso/gso_tcp4.h create mode 100644 lib/librte_gso/gso_tunnel_tcp4.c create mode 100644 lib/librte_gso/gso_tunnel_tcp4.h create mode 100644 lib/librte_gso/rte_gso.c create mode 100644 lib/librte_gso/rte_gso.h create mode 100644 lib/librte_gso/rte_gso_version.map -- 1.9.3