From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexei Starovoitov Subject: Re: [PATCH bpf-next 00/11] AF_XDP zero-copy support for i40e Date: Wed, 29 Aug 2018 12:39:55 -0700 Message-ID: <20180829193953.wejfzvgxlime3bpw@ast-mbp.dhcp.thefacebook.com> References: <20180828124435.30578-1-bjorn.topel@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: magnus.karlsson@intel.com, magnus.karlsson@gmail.com, alexander.h.duyck@intel.com, alexander.duyck@gmail.com, ast@kernel.org, brouer@redhat.com, daniel@iogearbox.net, netdev@vger.kernel.org, jesse.brandeburg@intel.com, anjali.singhai@intel.com, peter.waskiewicz.jr@intel.com, =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , michael.lundkvist@ericsson.com, willemdebruijn.kernel@gmail.com, john.fastabend@gmail.com, jakub.kicinski@netronome.com, neerav.parikh@intel.com, mykyta.iziumtsev@linaro.org, francois.ozog@linaro.org, ilias.apalodimas@linaro.org, brian.brooks@linaro.org, u9012063@gmail.com, pavel@fastnetmon.com, qi.z.zhang@intel.com To: =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= Return-path: Received: from mail-ed1-f65.google.com ([209.85.208.65]:34889 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727692AbeH2Xi1 (ORCPT ); Wed, 29 Aug 2018 19:38:27 -0400 Received: by mail-ed1-f65.google.com with SMTP id y20-v6so4801196edq.2 for ; Wed, 29 Aug 2018 12:40:03 -0700 (PDT) Content-Disposition: inline In-Reply-To: <20180828124435.30578-1-bjorn.topel@gmail.com> Sender: netdev-owner@vger.kernel.org List-ID: On Tue, Aug 28, 2018 at 02:44:24PM +0200, Björn Töpel wrote: > From: Björn Töpel > > This patch set introduces zero-copy AF_XDP support for Intel's i40e > driver. In the first preparatory patch we also add support for > XDP_REDIRECT for zero-copy allocated frames so that XDP programs can > redirect them. This was a ToDo from the first AF_XDP zero-copy patch > set from early June. Special thanks to Alex Duyck and Jesper Dangaard > Brouer for reviewing earlier versions of this patch set. > > The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note > that in the interest of time, to get an AF_XDP zero-copy implementation > out there for people to try, some code paths have been copied from the > XDP path to the zero-copy path. It is out goal to merge the two paths > in later patch sets. > > In contrast to the implementation from beginning of June, this patch > set does not require any extra HW queues for AF_XDP zero-copy > TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and > AF_XDP zero-copy TX. > > Jeff, given that most of changes are in i40e, it is up to you how you > would like to route these patches. The set is tagged bpf-next, but > if taking it via the Intel driver tree is easier, let us know. > > We have run some benchmarks on a dual socket system with two Broadwell > E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 > cores which gives a total of 28, but only two cores are used in these > experiments. One for TR/RX and one for the user space application. The > memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is > 8192MB and with 8 of those DIMMs in the system we have 64 GB of total > memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The > NIC is Intel I40E 40Gbit/s using the i40e driver. > > Below are the results in Mpps of the I40E NIC benchmark runs for 64 > and 1500 byte packets, generated by a commercial packet generator HW > outputing packets at full 40 Gbit/s line rate. The results are with > retpoline and all other spectre and meltdown fixes, so these results > are not comparable to the ones from the zero-copy patch set in June. > > AF_XDP performance 64 byte packets. > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.6 8.2 15.0 > txpush 2.2 - 21.9 > l2fwd 1.7 2.3 11.3 > > AF_XDP performance 1500 byte packets: > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.0 3.3 3.3 > l2fwd 1.3 1.7 3.1 > > XDP performance on our system as a base line: > > 64 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 18.4M 0 > > 1500 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 3.3M 0 > > The structure of the patch set is as follows: > > Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames > Patches 2-4: Preparatory patches to common xsk and net code > Patches 5-7: Preparatory patches to i40e driver code for RX > Patch 8: i40e zero-copy support for RX > Patch 9: Preparatory patch to i40e driver code for TX > Patch 10: i40e zero-copy support for TX > Patch 11: Add flags to sample application to force zero-copy/copy mode > > We based this patch set on bpf-next commit 050cdc6c9501 ("Merge > git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") > > > Magnus & Björn > > Björn Töpel (8): > xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY > xdp: export xdp_rxq_info_unreg_mem_model > xsk: expose xdp_umem_get_{data,dma} to drivers > i40e: added queue pair disable/enable functions > i40e: refactor Rx path for re-use > i40e: move common Rx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Rx support > samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock > > Magnus Karlsson (3): > net: add napi_if_scheduled_mark_missed > i40e: move common Tx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Tx support Applied to bpf-next. Thanks a lot to the authors, code reviewers and testers. I hope all the issues brought up during code review can be quickly addressed in the follow up patches and zerocopy feature in i40e and other drivers will be mainline ready for the next merge window.