From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FB82C433B4 for ; Mon, 19 Apr 2021 06:56:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 740606101D for ; Mon, 19 Apr 2021 06:56:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233418AbhDSG44 (ORCPT ); Mon, 19 Apr 2021 02:56:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:46025 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231831AbhDSG4z (ORCPT ); Mon, 19 Apr 2021 02:56:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618815357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CfFgAP3oaXw9DBMJywNjiJxkc/lfcdS2wTAzSi6+ro8=; b=UGlsAJ9S0XYGimHLvecSY/yd8fWIZM8LveyFv7HnAgnQURpsHfkHR3hc3aQNLbK2txMgQ2 9ZHMILrxe/kUQAIsJyEwXpiHPxN08QjXksgIpwtTGhQ55VH7DFUpjq1yDniJ0ZCcO/DJl9 iTVWtMBSaudJf0ZLlwOtlPjctBmC7+o= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-396-VUfwwlIQMVWR4RdHSzny6w-1; Mon, 19 Apr 2021 02:55:54 -0400 X-MC-Unique: VUfwwlIQMVWR4RdHSzny6w-1 Received: by mail-ej1-f70.google.com with SMTP id ji8-20020a1709079808b029037c921a9ea0so3274001ejc.9 for ; Sun, 18 Apr 2021 23:55:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=CfFgAP3oaXw9DBMJywNjiJxkc/lfcdS2wTAzSi6+ro8=; b=uNnJ35Hn4rwYtDzL6v0Etg2DVBcui+D7oP3i6QqsoEqKrnfVumOvHviJbwQGtHZd07 we72hy3YGatgdFigVbl99O/nDLJKMH8Tbgl5q44VxGgl8dVH7csrDsoKvn+mp3OxHz0R eOTqF+W3NH8XpNHhFFT8yVC4Pyy+1/Of+Z+94jrFuQep+16w2Mm+ywKJWepm018uz5tl Yxji88wHVf5boWYgLu76oD0zc/TYIXSF4Ec/MBTm0g0VtG+i6HykmOQw3EsvHLpGUrux /89zNkQRCh24k2eMJLgifACHAtsas7NeperqfDkxaRkficX6kvlj7PfTH0dCQ09hxlgD iAFg== X-Gm-Message-State: AOAM530FIat8nJMc2melfvaY4b3atyDwQrPZ2opJ7pDUDB5132gZQgLb fw3wDX1siVXEouGemkg7v9iJKqI4oGMc0oZ3H++1mjlJZ/X4woYwwah4dSMAaBg3uFTcaNq1XP6 6KxrLHLRUZ+3P X-Received: by 2002:a17:906:9b2:: with SMTP id q18mr20434081eje.147.1618815353491; Sun, 18 Apr 2021 23:55:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxcn890CBV/YNi8GpOhoDwYaWgjCDq79VbJ5vYMQq42Tf7E/7us2+m+jkJL+50T0u4LSEMJwg== X-Received: by 2002:a17:906:9b2:: with SMTP id q18mr20434070eje.147.1618815353329; Sun, 18 Apr 2021 23:55:53 -0700 (PDT) Received: from localhost ([151.66.28.185]) by smtp.gmail.com with ESMTPSA id s3sm12039574edw.66.2021.04.18.23.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 18 Apr 2021 23:55:52 -0700 (PDT) Date: Mon, 19 Apr 2021 08:55:49 +0200 From: Lorenzo Bianconi To: Magnus Karlsson Cc: Jesper Dangaard Brouer , Lorenzo Bianconi , bpf , Network Development , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , shayagr@amazon.com, sameehj@amazon.com, John Fastabend , David Ahern , Eelco Chaudron , Jason Wang , Alexander Duyck , Saeed Mahameed , "Fijalkowski, Maciej" , Tirthendu Subject: Re: [PATCH v8 bpf-next 00/14] mvneta: introduce XDP multi-buffer support Message-ID: References: <20210418181801.17166935@carbon> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="zJQ7ehSoYb8/vkCh" Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org --zJQ7ehSoYb8/vkCh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > On Sun, Apr 18, 2021 at 6:18 PM Jesper Dangaard Brouer > wrote: > > > > On Fri, 16 Apr 2021 16:27:18 +0200 > > Magnus Karlsson wrote: > > > > > On Thu, Apr 8, 2021 at 2:51 PM Lorenzo Bianconi = wrote: > > > > > > > > This series introduce XDP multi-buffer support. The mvneta driver is > > > > the first to support these new "non-linear" xdp_{buff,frame}. Revie= wers > > > > please focus on how these new types of xdp_{buff,frame} packets > > > > traverse the different layers and the layout design. It is on purpo= se > > > > that BPF-helpers are kept simple, as we don't want to expose the > > > > internal layout to allow later changes. > > > > > > > > For now, to keep the design simple and to maintain performance, the= XDP > > > > BPF-prog (still) only have access to the first-buffer. It is left f= or > > > > later (another patchset) to add payload access across multiple buff= ers. > > > > This patchset should still allow for these future extensions. The g= oal > > > > is to lift the XDP MTU restriction that comes with XDP, but maintain > > > > same performance as before. > > [...] > > > > > > > > [0] https://netdevconf.info/0x14/session.html?talk-the-path-to-tcp-= 4k-mtu-and-rx-zerocopy > > > > [1] https://github.com/xdp-project/xdp-project/blob/master/areas/co= re/xdp-multi-buffer01-design.org > > > > [2] https://netdevconf.info/0x14/session.html?tutorial-add-XDP-supp= ort-to-a-NIC-driver (XDPmulti-buffers section) > > > > > > Took your patches for a test run with the AF_XDP sample xdpsock on an > > > i40e card and the throughput degradation is between 2 to 6% depending > > > on the setup and microbenchmark within xdpsock that is executed. And > > > this is without sending any multi frame packets. Just single frame > > > ones. Tirtha made changes to the i40e driver to support this new > > > interface so that is being included in the measurements. > > > > Could you please share Tirtha's i40e support patch with me? >=20 > We will post them on the list as an RFC. Tirtha also added AF_XDP > multi-frame support on top of Lorenzo's patches so we will send that > one out as well. Will also rerun my experiments, properly document > them and send out just to be sure that I did not make any mistake. ack, very cool, thx >=20 > Just note that I would really like for the multi-frame support to get > in. I have lost count on how many people that have asked for it to be > added to XDP and AF_XDP. So please check our implementation and > improve it so we can get the overhead down to where we want it to be. sure, I will do. Regards, Lorenzo >=20 > Thanks: Magnus >=20 > > I would like to reproduce these results in my testlab, in-order to > > figure out where the throughput degradation comes from. > > > > > What performance do you see with the mvneta card? How much are we > > > willing to pay for this feature when it is not being used or can we in > > > some way selectively turn it on only when needed? > > > > Well, as Daniel says performance wise we require close to /zero/ > > additional overhead, especially as you state this happens when sending > > a single frame, which is a base case that we must not slowdown. > > > > -- > > Best regards, > > Jesper Dangaard Brouer > > MSc.CS, Principal Kernel Engineer at Red Hat > > LinkedIn: http://www.linkedin.com/in/brouer > > >=20 --zJQ7ehSoYb8/vkCh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTquNwa3Txd3rGGn7Y6cBh0uS2trAUCYH0pcwAKCRA6cBh0uS2t rLg6AQC4guHiZReHqLkzgFeVNP3vQpWAKZzxHZ6EIgR8I6Ba+AEA0uGSd14znwF8 DySXiO48RWgBbJeoVDf6wSagKTGRCA0= =rFYd -----END PGP SIGNATURE----- --zJQ7ehSoYb8/vkCh--