Re: [PATCH net-next 28/33] mlx5: rx queue setup time determine frame_sz for XDP

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: sameehj@amazon.com, "Tariq Toukan" <tariqt@mellanox.com>,
	"Saeed Mahameed" <saeedm@mellanox.com>,
	"Network Development" <netdev@vger.kernel.org>,
	bpf <bpf@vger.kernel.org>,
	zorik@amazon.com, "Arthur Kiyanovski" <akiyano@amazon.com>,
	gtzalik@amazon.com, "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"Daniel Borkmann" <borkmann@iogearbox.net>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Alexander Duyck" <alexander.duyck@gmail.com>,
	"Jeff Kirsher" <jeffrey.t.kirsher@intel.com>,
	"David Ahern" <dsahern@gmail.com>,
	"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
	"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
	"Lorenzo Bianconi" <lorenzo@kernel.org>,
	"Steffen Klassert" <steffen.klassert@secunet.com>
Subject: Re: [PATCH net-next 28/33] mlx5: rx queue setup time determine frame_sz for XDP
Date: Fri, 24 Apr 2020 17:58:39 -0700	[thread overview]
Message-ID: <CAADnVQ+hmYAJP9W_nwY+21OODxNpbxb8W-6CZaro8eL-fG4mog@mail.gmail.com> (raw)
In-Reply-To: <158757178332.1370371.3518949026344543513.stgit@firesoul>

On Wed, Apr 22, 2020 at 9:09 AM Jesper Dangaard Brouer
<brouer@redhat.com> wrote:
>
> The mlx5 driver have multiple memory models, which are also changed
> according to whether a XDP bpf_prog is attached.
>
> The 'rx_striding_rq' setting is adjusted via ethtool priv-flags e.g.:
>  # ethtool --set-priv-flags mlx5p2 rx_striding_rq off
>
> On the general case with 4K page_size and regular MTU packet, then
> the frame_sz is 2048 and 4096 when XDP is enabled, in both modes.
>
> The info on the given frame size is stored differently depending on the
> RQ-mode and encoded in a union in struct mlx5e_rq union wqe/mpwqe.
> In rx striding mode rq->mpwqe.log_stride_sz is either 11 or 12, which
> corresponds to 2048 or 4096 (MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ).
> In non-striding mode (MLX5_WQ_TYPE_CYCLIC) the frag_stride is stored
> in rq->wqe.info.arr[0].frag_stride, for the first fragment, which is
> what the XDP case cares about.
>
> To reduce effect on fast-path, this patch determine the frame_sz at
> setup time, to avoid determining the memory model runtime. Variable
> is named first_frame_sz to make it clear that this is only the frame
> size of the first fragment.
>
> This mlx5 driver does a DMA-sync on XDP_TX action, but grow is safe
> as it have done a DMA-map on the entire PAGE_SIZE. The driver also
> already does a XDP length check against sq->hw_mtu on the possible
> XDP xmit paths mlx5e_xmit_xdp_frame() + mlx5e_xmit_xdp_frame_mpwqe().
>
> Cc: Tariq Toukan <tariqt@mellanox.com>
> Cc: Saeed Mahameed <saeedm@mellanox.com>
> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
> ---
>  drivers/net/ethernet/mellanox/mlx5/core/en.h      |    1 +
>  drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c  |    1 +
>  drivers/net/ethernet/mellanox/mlx5/core/en_main.c |    6 ++++++
>  3 files changed, 8 insertions(+)

Hey mellanox folks,

you had an active discussion regarding mlx5 changes earlier.
Were your concerns resolved ?
If so, could you please ack.