From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0EADC2D0A3 for ; Wed, 4 Nov 2020 23:33:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 81A4E20825 for ; Wed, 4 Nov 2020 23:33:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604532803; bh=pLgzlZcEoMWv2J4Mk1Xd7jOAN5YoKXgl49GtlZW2QNo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-ID:From; b=XifqxkYFygDXhDDTz40sSpkik4+8ZL0dYC/ju7pVi3KtBBeTlE6K6gP/sPy1cPyiL E3MdVzXmWSJJe9tIeN06lKOgd6XrEn56V9CsipKBWrJxTK0vqz3zPMN6oeg9xD4rBX tycMtA/K4uHI5+HkwCzEqBVEoKPHjlSkSie886nI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732604AbgKDXdX (ORCPT ); Wed, 4 Nov 2020 18:33:23 -0500 Received: from mail.kernel.org ([198.145.29.99]:35504 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731539AbgKDXdW (ORCPT ); Wed, 4 Nov 2020 18:33:22 -0500 Received: from kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com (unknown [163.114.132.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5AEA22074B; Wed, 4 Nov 2020 23:33:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604532802; bh=pLgzlZcEoMWv2J4Mk1Xd7jOAN5YoKXgl49GtlZW2QNo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=JmQgCSoSsXOhiz82qHrpYSpMT86LMGk98P+q/NWFve+aj9dPapUGXW2jZtVpjbCl/ lBnZXOwQS4ULyHLXRG0cdVMt+bbLtxZ5Ttcj9SIMxFSEEBYU/jrGwVB/hTcaLzotjg +b2z5paNfpiUwSq5ORfkMRu/7q+Vm9ROOGUecoZE= Date: Wed, 4 Nov 2020 15:33:20 -0800 From: Jakub Kicinski To: Magnus Karlsson Cc: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com, bpf@vger.kernel.org, jeffrey.t.kirsher@intel.com, anthony.l.nguyen@intel.com, maciej.fijalkowski@intel.com, maciejromanfijalkowski@gmail.com, intel-wired-lan@lists.osuosl.org Subject: Re: [PATCH bpf-next 1/6] i40e: introduce lazy Tx completions for AF_XDP zero-copy Message-ID: <20201104153320.66cecba8@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> In-Reply-To: <1604498942-24274-2-git-send-email-magnus.karlsson@gmail.com> References: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> <1604498942-24274-2-git-send-email-magnus.karlsson@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Wed, 4 Nov 2020 15:08:57 +0100 Magnus Karlsson wrote: > From: Magnus Karlsson > > Introduce lazy Tx completions when a queue is used for AF_XDP > zero-copy. In the current design, each time we get into the NAPI poll > loop we try to complete as many Tx packets as possible from the > NIC. This is performed by reading the head pointer register in the NIC > that tells us how many packets have been completed. Reading this > register is expensive as it is across PCIe, so let us try to limit the > number of times it is read by only completing Tx packets to user-space > when the number of available descriptors in the Tx HW ring is below > some threshold. This will decrease the number of reads issued to the > NIC and improves performance with 1.5% - 2% for the l2fwd xdpsock > microbenchmark. > > The threshold is set to the minimum possible size that the HW ring can > have. This so that we do not run into a scenario where the threshold > is higher than the configured number of descriptors in the HW ring. > > Signed-off-by: Magnus Karlsson I feel like this needs a big fat warning somewhere. It's perfectly fine to never complete TCP packets, but AF_XDP could be used to implement protocols in user space. What if someone wants to implement something like TSQ?