[PATCH rdma-next 0/2] Add support to SQD2RTS transition

[PATCH rdma-next 0/2] Add support to SQD2RTS transition

[Leon Romanovsky]

From: Leon Romanovsky <leonro@nvidia.com>

The IBTA specification allows SQD2RTS QP transition, however the mlx5_ib
driver didn't have needed implementation. 

This patchset from Sergey adds the needed functionality for this modify QP
transition state and adds extra flag to mark supported kernel.

Thanks

Sergey Gorenko (2):
  RDMA/mlx5: Support SQD2RTS for modify QP
  RDMA/mlx5: Add SQD2RTS bit to the alloc ucontext response

 drivers/infiniband/hw/mlx5/main.c |  4 ++++
 drivers/infiniband/hw/mlx5/qp.c   | 12 ++++++++++++
 drivers/infiniband/hw/mlx5/qpc.c  |  6 ++++++
 include/uapi/rdma/mlx5-abi.h      |  1 +
 4 files changed, 23 insertions(+)

-- 
2.31.1

[PATCH rdma-next 1/2] RDMA/mlx5: Support SQD2RTS for modify QP

[Leon Romanovsky]

From: Sergey Gorenko <sergeygo@nvidia.com>

The transition of the QP state from SQD to RTS is allowed by the IB
specification. The hardware also supports that, but it is not
implemented in mlx5_ib.

This commit adds SQD2RTS command to the modify QP in mlx5_ib to support
the missing feature. The feature is required by the signature pipelining
API that will be added to rdma-core.

Reviewed-by: Evgenii Kochetov <evgeniik@nvidia.com>
Signed-off-by: Sergey Gorenko <sergeygo@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c  | 12 ++++++++++++
 drivers/infiniband/hw/mlx5/qpc.c |  6 ++++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 11887f8bd75c..4f11c6713284 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -3455,6 +3455,17 @@ static enum mlx5_qp_optpar opt_mask[MLX5_QP_NUM_STATE][MLX5_QP_NUM_STATE][MLX5_Q
 					   MLX5_QP_OPTPAR_RRE,
 		},
 	},
+	[MLX5_QP_STATE_SQD] = {
+		[MLX5_QP_STATE_RTS] = {
+			[MLX5_QP_ST_UD] = MLX5_QP_OPTPAR_Q_KEY,
+			[MLX5_QP_ST_MLX] = MLX5_QP_OPTPAR_Q_KEY,
+			[MLX5_QP_ST_UC] = MLX5_QP_OPTPAR_RWE,
+			[MLX5_QP_ST_RC] = MLX5_QP_OPTPAR_RNR_TIMEOUT	|
+					  MLX5_QP_OPTPAR_RWE		|
+					  MLX5_QP_OPTPAR_RAE		|
+					  MLX5_QP_OPTPAR_RRE,
+		},
+	},
 };
 
 static int ib_nr_to_mlx5_nr(int ib_mask)
@@ -3850,6 +3861,7 @@ static int __mlx5_ib_modify_qp(struct ib_qp *ibqp,
 		[MLX5_QP_STATE_SQD] = {
 			[MLX5_QP_STATE_RST]	= MLX5_CMD_OP_2RST_QP,
 			[MLX5_QP_STATE_ERR]	= MLX5_CMD_OP_2ERR_QP,
+			[MLX5_QP_STATE_RTS]	= MLX5_CMD_OP_SQD_RTS_QP,
 		},
 		[MLX5_QP_STATE_SQER] = {
 			[MLX5_QP_STATE_RST]	= MLX5_CMD_OP_2RST_QP,
diff --git a/drivers/infiniband/hw/mlx5/qpc.c b/drivers/infiniband/hw/mlx5/qpc.c
index c683d7000168..8844eacf2380 100644
--- a/drivers/infiniband/hw/mlx5/qpc.c
+++ b/drivers/infiniband/hw/mlx5/qpc.c
@@ -441,6 +441,12 @@ static int modify_qp_mbox_alloc(struct mlx5_core_dev *dev, u16 opcode, int qpn,
 		MOD_QP_IN_SET_QPC(sqerr2rts_qp, mbox->in, opcode, qpn,
 				  opt_param_mask, qpc, uid);
 		break;
+	case MLX5_CMD_OP_SQD_RTS_QP:
+		if (MBOX_ALLOC(mbox, sqd2rts_qp))
+			return -ENOMEM;
+		MOD_QP_IN_SET_QPC(sqd2rts_qp, mbox->in, opcode, qpn,
+				  opt_param_mask, qpc, uid);
+		break;
 	case MLX5_CMD_OP_INIT2INIT_QP:
 		if (MBOX_ALLOC(mbox, init2init_qp))
 			return -ENOMEM;
-- 
2.31.1

[PATCH rdma-next 2/2] RDMA/mlx5: Add SQD2RTS bit to the alloc ucontext response

[Leon Romanovsky]

From: Sergey Gorenko <sergeygo@nvidia.com>

The new bit in the comp_mask is needed to mark that kernel supports
SQD2RTS transition for the modify QP command.

Reviewed-by: Evgenii Kochetov <evgeniik@nvidia.com>
Signed-off-by: Sergey Gorenko <sergeygo@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c | 4 ++++
 include/uapi/rdma/mlx5-abi.h      | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 6d1dd09a4388..312aa731860d 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1817,6 +1817,10 @@ static int set_ucontext_resp(struct ib_ucontext *uctx,
 		resp->comp_mask |= MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_ECE;
 
 	resp->num_dyn_bfregs = bfregi->num_dyn_bfregs;
+
+	if (MLX5_CAP_GEN(dev->mdev, drain_sigerr))
+		resp->comp_mask |= MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_SQD2RTS;
+
 	return 0;
 }
 
diff --git a/include/uapi/rdma/mlx5-abi.h b/include/uapi/rdma/mlx5-abi.h
index 27905a0268c9..995faf8f44bd 100644
--- a/include/uapi/rdma/mlx5-abi.h
+++ b/include/uapi/rdma/mlx5-abi.h
@@ -101,6 +101,7 @@ enum mlx5_ib_alloc_ucontext_resp_mask {
 	MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_CORE_CLOCK_OFFSET = 1UL << 0,
 	MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DUMP_FILL_MKEY    = 1UL << 1,
 	MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_ECE               = 1UL << 2,
+	MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_SQD2RTS           = 1UL << 3,
 };
 
 enum mlx5_user_cmds_supp_uhw {
-- 
2.31.1

Re: [PATCH rdma-next 0/2] Add support to SQD2RTS transition

[Jason Gunthorpe]

On Mon, May 10, 2021 at 01:23:31PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> The IBTA specification allows SQD2RTS QP transition, however the mlx5_ib
> driver didn't have needed implementation. 
> 
> This patchset from Sergey adds the needed functionality for this modify QP
> transition state and adds extra flag to mark supported kernel.
> 
> Thanks
> 
> Sergey Gorenko (2):
>   RDMA/mlx5: Support SQD2RTS for modify QP
>   RDMA/mlx5: Add SQD2RTS bit to the alloc ucontext response

Applied to for-next, thanks

Jason