From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56673C55178 for ; Thu, 5 Nov 2020 15:45:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BEB220867 for ; Thu, 5 Nov 2020 15:45:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604591115; bh=RYfcojP3xQPdgNO24dCEuSYbe3kFwBI9UaElG668AGk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-ID:From; b=rVAHyzFEQeD9PrY0BiZgIOEk9Hibi6etzxQ/9/5TDYEqpINBCsySch3iS+tAYEcZv l8oGFBe09CRdaR8U2R48WQ0tSCktMir7NrdaAO9HmizUYPUD8QY6503NvXUqFTBGZO pADqEjTCLJzjI5f/S9eU1k9THX/SDxm6N6/GYeLs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731309AbgKEPpO (ORCPT ); Thu, 5 Nov 2020 10:45:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:34542 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731202AbgKEPpO (ORCPT ); Thu, 5 Nov 2020 10:45:14 -0500 Received: from kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com (unknown [163.114.132.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9FF3B206FA; Thu, 5 Nov 2020 15:45:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604591113; bh=RYfcojP3xQPdgNO24dCEuSYbe3kFwBI9UaElG668AGk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=WUwkmm5MK3MfjMJKe/ZGRT7ERL++vJOrKm12aDuXQEi30a6pVcfAU6unoYTvoEOlF I+zYBqAGCEYm9vIR+7awtLUxv0p75nKTNY/qrgObV4Ns3gOD1775K0cM1XKiYZM6bd P6Pq35Qn0c69LObw3Lo2vkRNCDgGpXO27/tg2Q84= Date: Thu, 5 Nov 2020 07:45:11 -0800 From: Jakub Kicinski To: Magnus Karlsson Cc: "Karlsson, Magnus" , =?UTF-8?B?QmrDtnJuIFQ=?= =?UTF-8?B?w7ZwZWw=?= , Alexei Starovoitov , Daniel Borkmann , Network Development , Jonathan Lemon , bpf , jeffrey.t.kirsher@intel.com, anthony.l.nguyen@intel.com, "Fijalkowski, Maciej" , Maciej Fijalkowski , intel-wired-lan Subject: Re: [PATCH bpf-next 1/6] i40e: introduce lazy Tx completions for AF_XDP zero-copy Message-ID: <20201105074511.6935e8b7@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> In-Reply-To: References: <1604498942-24274-1-git-send-email-magnus.karlsson@gmail.com> <1604498942-24274-2-git-send-email-magnus.karlsson@gmail.com> <20201104153320.66cecba8@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Thu, 5 Nov 2020 15:17:50 +0100 Magnus Karlsson wrote: > > I feel like this needs a big fat warning somewhere. > > > > It's perfectly fine to never complete TCP packets, but AF_XDP could be > > used to implement protocols in user space. What if someone wants to > > implement something like TSQ? > > I might misunderstand you, but with TSQ here (for something that > bypasses qdisk and any buffering and just goes straight to the driver) > you mean the ability to have just a few buffers outstanding and > continuously reuse these? If so, that is likely best achieved by > setting a low Tx queue size on the NIC. Note that even without this > patch, completions could be delayed. Though this patch makes that the > normal case. In any way, I think this calls for some improved > documentation. TSQ tries to limit the amount of data the TCP stack queues into TC/sched and drivers. Say 1MB ~ 16 GSO frames. It will not queue more data until some of the transfer is reported as completed. IIUC you're allowing up to 64 descriptors to linger without reporting back that the transfer is done. That means that user space implementing a scheme similar to TSQ may see its transfers stalled.