Re: [net-next 10/11] net/mlx5e: kTLS, Add kTLS RX resync support

[Saeed Mahameed <saeedm@mellanox.com>]

On Fri, 2020-05-29 at 13:16 -0700, Jakub Kicinski wrote:
> On Fri, 29 May 2020 12:46:40 -0700 Saeed Mahameed wrote:
> >  /* Re-sync */
> > +static struct mlx5_wqe_ctrl_seg *
> > +resync_post_get_progress_params(struct mlx5e_icosq *sq,
> > +				struct mlx5e_ktls_offload_context_rx
> > *priv_rx)
> > +{
> > +	struct mlx5e_ktls_rx_resync_ctx *resync = &priv_rx->resync;
> > +	struct mlx5e_get_tls_progress_params_wqe *wqe;
> > +	struct mlx5_wqe_ctrl_seg *cseg;
> > +	struct mlx5_seg_get_psv *psv;
> > +	u16 pi;
> > +
> > +	dma_sync_single_for_device(resync->priv->mdev->device,
> > +				   resync->dma_addr,
> > +				   PROGRESS_PARAMS_PADDED_SIZE,
> > +				   DMA_FROM_DEVICE);
> > +	BUILD_BUG_ON(MLX5E_KTLS_GET_PROGRESS_WQEBBS != 1);
> > +	if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc,
> > 1)))
> > +		return ERR_PTR(-ENOSPC);
> 
> I thought you said that resync requests are guaranteed to never fail?
> 

I didn't say that :), maybe tariq did say this before my review,
but basically with the current mlx5 arch, it is impossible to guarantee
this unless we open 1 service queue per ktls offloads and that is going
to be an overkill!

This is a rare corner case anyway, where more than 1k tcp connections
sharing the same RX ring will request resync at the same exact moment. 

> > +	pi = mlx5e_icosq_get_next_pi(sq, 1);
> > +	wqe = MLX5E_TLS_FETCH_GET_PROGRESS_PARAMS_WQE(sq, pi);
> > +
> > +#define GET_PSV_DS_CNT (DIV_ROUND_UP(sizeof(*wqe),
> > MLX5_SEND_WQE_DS))
> > +
> > +	cseg = &wqe->ctrl;
> > +	cseg->opmod_idx_opcode =
> > +		cpu_to_be32((sq->pc << 8) | MLX5_OPCODE_GET_PSV |
> > +			    (MLX5_OPC_MOD_TLS_TIR_PROGRESS_PARAMS <<
> > 24));
> > +	cseg->qpn_ds =
> > +		cpu_to_be32((sq->sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
> > GET_PSV_DS_CNT);
> > +
> > +	psv = &wqe->psv;
> > +	psv->num_psv      = 1 << 4;
> > +	psv->l_key        = sq->channel->mkey_be;
> > +	psv->psv_index[0] = cpu_to_be32(priv_rx->tirn);
> > +	psv->va           = cpu_to_be64(priv_rx->resync.dma_addr);
> > +
> > +	icosq_fill_wi(sq, pi, MLX5E_ICOSQ_WQE_GET_PSV_TLS, 1, priv_rx);
> > +	sq->pc++;
> > +
> > +	return cseg;
> > +}