From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 572A3C43217 for ; Fri, 25 Nov 2022 00:38:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229436AbiKYAh7 (ORCPT ); Thu, 24 Nov 2022 19:37:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229480AbiKYAhz (ORCPT ); Thu, 24 Nov 2022 19:37:55 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3389382BC6 for ; Thu, 24 Nov 2022 16:36:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669336616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bVrOkDrmMGpa5h7AIlaFraeH3E0rHqCXhdxo/iWpc3M=; b=XBFVp4HnA/btmGL5SXFWdcy+bQD2QbqbCfEb9vcdJ/Rv5d8f+dwIcFhjak3SxKK/KqdZu7 tnakvykpxgYyWDypu/tx9Fvch4foQxNLLz2P9pjaIu492VN4WiPaHFDFctsvLdp2w7bzSF XyT312brCz5+PPwaoeVn7CUAu05Xz/M= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-633-LKowYuPgOsuzyL_Bz180KA-1; Thu, 24 Nov 2022 19:36:55 -0500 X-MC-Unique: LKowYuPgOsuzyL_Bz180KA-1 Received: by mail-ej1-f72.google.com with SMTP id sa20-20020a1709076d1400b007bbe8699c2eso661369ejc.6 for ; Thu, 24 Nov 2022 16:36:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bVrOkDrmMGpa5h7AIlaFraeH3E0rHqCXhdxo/iWpc3M=; b=K3dQ9/0eq2q39JBB0BbIj6Z73Avt6F5MhvAR2smUQQiIGpT+urQxzj6k9NBRoWkq80 NxHfwl44I5s8Lhj1lXC/1lY8QGCISZea/3su4LQFMVlP2+X0uZZBaMxlRzAeFVaZUfPD I1cmdieXiCPO082tZhvGenikLYPbuG6aNSljHPxWHW0p24FQI3hhMlOQ9sP1sTM1vpna WlE/SMypTkeQgkOxF/nI6ejMcbmZub2MH/qvYFAKkV0IJkakXNW5+4xqd53HnhXx1zd2 PIqiv8Lnsr8zMjCL1FyfR76QyL3zY9du9/PIv60LmW7Wz2ZMVG0+4Y3Oz4lbiOkWxbGl ojFw== X-Gm-Message-State: ANoB5pkirChchHUpHctwybbDGv25ILEgXz5OKnRaK2IiaH4cDgMVJoRj S2z36ADTGPgRoWxD9us+2tjYH16wsOOpdnF2Sq/edDe2hSdYlbVnlW6d4RmlJW41DN84QZz9MeZ 1yU4sBOZfibZI X-Received: by 2002:a17:906:810:b0:7ba:5b85:994f with SMTP id e16-20020a170906081000b007ba5b85994fmr7093371ejd.510.1669336613671; Thu, 24 Nov 2022 16:36:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf5kN53QWAHI3CajHtXDJoe0dz9z+qfwYwqCLE3CqwTYapDEN2akUb+MJf0gMCXGfUJU4PLHlQ== X-Received: by 2002:a17:906:810:b0:7ba:5b85:994f with SMTP id e16-20020a170906081000b007ba5b85994fmr7093337ejd.510.1669336613264; Thu, 24 Nov 2022 16:36:53 -0800 (PST) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id i10-20020a05640242ca00b00467c3cbab6fsm1115299edc.77.2022.11.24.16.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 16:36:52 -0800 (PST) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id D11337EB6D8; Fri, 25 Nov 2022 01:36:51 +0100 (CET) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Maciej Fijalkowski Cc: Jakub Kicinski , sdf@google.com, bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com, jolsa@kernel.org, Tariq Toukan , David Ahern , Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org Subject: Re: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff In-Reply-To: References: <20221121182552.2152891-1-sdf@google.com> <20221121182552.2152891-7-sdf@google.com> <874jupviyc.fsf@toke.dk> <20221123111431.7b54668e@kernel.org> <871qptuyie.fsf@toke.dk> <20221123174746.418920e5@kernel.org> <87edts2z8n.fsf@toke.dk> X-Clacks-Overhead: GNU Terry Pratchett Date: Fri, 25 Nov 2022 01:36:51 +0100 Message-ID: <874jun3m58.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Maciej Fijalkowski writes: > On Thu, Nov 24, 2022 at 04:17:01PM +0100, Maciej Fijalkowski wrote: >> On Thu, Nov 24, 2022 at 03:39:20PM +0100, Toke H=C3=B8iland-J=C3=B8rgens= en wrote: >> > Jakub Kicinski writes: >> >=20 >> > > On Wed, 23 Nov 2022 22:55:21 +0100 Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >> > >> > Good idea, prototyped below, lmk if it that's not what you had in= mind. >> > >> > >> > >> > struct xdp_buff_xsk { >> > >> > struct xdp_buff xdp; /* 0 56 = */ >> > >> > u8 cb[16]; /* 56 16 = */ >> > >> > /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */=20= =20 >> > >>=20 >> > >> As pahole helpfully says here, xdp_buff is actually only 8 bytes fr= om >> > >> being a full cache line. I thought about adding a 'cb' field like t= his >> > >> to xdp_buff itself, but figured that since there's only room for a >> > >> single pointer, why not just add that and let the driver point it to >> > >> where it wants to store the extra context data? >> > > >> > > What if the driver wants to store multiple pointers or an integer or >> > > whatever else? The single pointer seems quite arbitrary and not >> > > strictly necessary. >> >=20 >> > Well, then you allocate a separate struct and point to that? Like I did >> > in mlx5: >> >=20 >> >=20 >> > + struct mlx5_xdp_ctx mlctx =3D { .cqe =3D cqe, .rq =3D rq }; >> > + struct xdp_buff xdp =3D { .drv_priv =3D &mlctx }; >> >=20 >> > but yeah, this does give an extra pointer deref on access. I'm not >> > really opposed to the cb field either, I just think it's a bit odd to >> > put it in struct xdp_buff_xsk; that basically requires the driver to >> > keep the layouts in sync. >> >=20 >> > Instead, why not but a cb field into xdp_buff itself so it can be used >> > for both the XSK and the non-XSK paths? Then the driver can just >> > typecast the xdp_buff into its own struct that has whatever data it >> > wants in place of the cb field? >>=20 >> Why can't you simply have a pointer to xdp_buff in driver specific >> xdp_buff container which would point to xdp_buff that is stack based (or >> whatever else memory that will back it up - I am about to push a change >> that makes ice driver embed xdp_buff within a struct that represents Rx >> ring) for XDP path and for ZC the pointer to xdp_buff that you get from >> xsk_buff_pool ? This would satisfy both sides I believe and would let us >> keep the same container struct. >>=20 >> struct mlx4_xdp_buff { >> struct xdp_buff *xdp; >> struct mlx4_cqe *cqe; >> struct mlx4_en_dev *mdev; >> struct mlx4_en_rx_ring *ring; >> struct net_device *dev; >> }; > > Nah this won't work from kfunc POV, probably no way to retrieve the > mlx4_xdp_buff based on xdp_buff ptr that needs to be used as an arg. > > Sorry I'll think more about it, in the meantime let's hear more voices > whether we should keep Stan's original approach + modify xdp_buff_xsk or > go with Toke's proposal. OK, so I played around with the mlx5 code a bit more, and I think the "wrapping struct + cb area" can be made to work without too many ugly casts; I'll send an updated version of the mlx5 patches with this incorporated tomorrow, after I've run some tests... -Toke