From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 825F8C4332F for ; Wed, 23 Nov 2022 19:14:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239801AbiKWTOi (ORCPT ); Wed, 23 Nov 2022 14:14:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239796AbiKWTOh (ORCPT ); Wed, 23 Nov 2022 14:14:37 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABF77A8152; Wed, 23 Nov 2022 11:14:36 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 200BECE264A; Wed, 23 Nov 2022 19:14:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 929C3C433C1; Wed, 23 Nov 2022 19:14:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669230873; bh=KbrG34cmabGhC68gT1YoUcErf4n0xzOwCvNVvok0PiI=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=FQ3vf9Yt/Tqn291D7wwpELSG8JKOA34K8tluIpaiJn5Mqb2sX9jb7Dy0WZmPtFAm9 ggumioiygzhDro/VU74pJOJInsUJXa9LeTDt25ipb92OLGeEQxZIL8tJm021oclbS+ +iT7d9OCNwS5e4UuNYq/IEoo0IXSjaQIwr4RLQT+HXwzE/jTNU/PS1grcY6NJ5/Zxi YpBf2YB1cZHkwtl7wwP3TzNVknTtC6dtsvWDnAnRNy9UYnhdq7QhUTZq/7NqCPXHWN q+/EX7kviPpHFA7vXjHdGM/6I4/WHLQpSuF27YqxpdtOSq+LDxbGR5HVQH/n9FZwcQ inkC8LYvJUpVg== Date: Wed, 23 Nov 2022 11:14:31 -0800 From: Jakub Kicinski To: Stanislav Fomichev Cc: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com, jolsa@kernel.org, Tariq Toukan , David Ahern , Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org Subject: Re: [xdp-hints] [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff Message-ID: <20221123111431.7b54668e@kernel.org> In-Reply-To: References: <20221121182552.2152891-1-sdf@google.com> <20221121182552.2152891-7-sdf@google.com> <874jupviyc.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Wed, 23 Nov 2022 10:26:41 -0800 Stanislav Fomichev wrote: > > This embedding trick works for drivers that put xdp_buff on the stack, > > but mlx5 supports XSK zerocopy, which uses the xsk_buff_pool for > > allocating them. This makes it a bit awkward to do the same thing there; > > and since it's probably going to be fairly common to do something like > > this, how about we just add a 'void *drv_priv' pointer to struct > > xdp_buff that the drivers can use? The xdp_buff already takes up a full > > cache line anyway, so any data stuffed after it will spill over to a new > > one; so I don't think there's much difference performance-wise. > > I guess the alternative is to extend xsk_buff_pool with some new > argument for xdp_buff tailroom? (so it can kmalloc(sizeof(xdp_buff) + > xdp_buff_tailroom)) > But it seems messy because there is no way of knowing what the target > device's tailroom is, so it has to be a user setting :-/ > I've started with a priv pointer in xdp_buff initially, it seems fine > to go back. I'll probably convert veth/mlx4 to the same mode as well > to avoid having different approaches in different places.. Can we not do this please? Add 16B of "private driver space" after the xdp_buff in xdp_buff_xsk (we have 16B to full cacheline), the drivers decide how they use it. Drivers can do BUILD_BUG_ON() for their expected size and cast that to whatever struct they want. This is how various offloads work, the variable size tailroom would be an over design IMO. And this way non XSK paths can keep its normal typing. > > I'll send my patch to add support to mlx5 (using the drv_priv pointer > > approach) separately. > > Saw them, thanks! Will include them in v3+.