* [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs
@ 2019-08-19 12:08 Leon Romanovsky
2019-08-19 12:08 ` [PATCH mlx5-next v3 1/3] net/mlx5: Set ODP capabilities for DC transport to max Leon Romanovsky
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Leon Romanovsky @ 2019-08-19 12:08 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
From: Leon Romanovsky <leonro@mellanox.com>
Changelog
v3:
* Rewrote patches to expose through DEVX without need to change mlx5-abi.h at all.
v2: https://lore.kernel.org/linux-rdma/20190806074807.9111-1-leon@kernel.org
* Fixed reserved_* field wrong name (Saeed M.)
* Split first patch to two patches, one for mlx5-next and one for rdma-next. (Saeed M.)
v1: https://lore.kernel.org/linux-rdma/20190804100048.32671-1-leon@kernel.org
* Fixed alignment to u64 in mlx5-abi.h (Gal P.)
v0: https://lore.kernel.org/linux-rdma/20190801122139.25224-1-leon@kernel.org
---------------------------------------------------------------------------------
From Michael,
The series adds support for on-demand paging for DC transport.
As DC is mlx-only transport, the capabilities are exposed
to the user using DEVX objects and later on through mlx5dv_query_device.
Thanks
Michael Guralnik (3):
net/mlx5: Set ODP capabilities for DC transport to max
IB/mlx5: Remove check of FW capabilities in ODP page fault handling
IB/mlx5: Add page fault handler for DC initiator WQE
drivers/infiniband/hw/mlx5/odp.c | 51 ++-----------------
.../net/ethernet/mellanox/mlx5/core/main.c | 6 +++
include/linux/mlx5/mlx5_ifc.h | 4 +-
3 files changed, 12 insertions(+), 49 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH mlx5-next v3 1/3] net/mlx5: Set ODP capabilities for DC transport to max
2019-08-19 12:08 [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Leon Romanovsky
@ 2019-08-19 12:08 ` Leon Romanovsky
2019-08-19 12:08 ` [PATCH rdma-next v3 2/3] IB/mlx5: Remove check of FW capabilities in ODP page fault handling Leon Romanovsky
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2019-08-19 12:08 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
From: Michael Guralnik <michaelgur@mellanox.com>
In mlx5_core initialization, query max ODP capabilities for DC transport
from FW and set as current capabilities.
Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/main.c | 6 ++++++
include/linux/mlx5/mlx5_ifc.h | 4 +++-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index fa0e991f1983..7f70ecb1db6d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -495,6 +495,12 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
ODP_CAP_SET_MAX(dev, xrc_odp_caps.write);
ODP_CAP_SET_MAX(dev, xrc_odp_caps.read);
ODP_CAP_SET_MAX(dev, xrc_odp_caps.atomic);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.srq_receive);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.send);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.receive);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.write);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.read);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.atomic);
if (do_set)
err = set_caps(dev, set_ctx, set_sz,
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index ab6ae723aae6..f037f8d5970e 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -946,7 +946,9 @@ struct mlx5_ifc_odp_cap_bits {
struct mlx5_ifc_odp_per_transport_service_cap_bits xrc_odp_caps;
- u8 reserved_at_100[0x700];
+ struct mlx5_ifc_odp_per_transport_service_cap_bits dc_odp_caps;
+
+ u8 reserved_at_120[0x6E0];
};
struct mlx5_ifc_calc_op {
--
2.20.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH rdma-next v3 2/3] IB/mlx5: Remove check of FW capabilities in ODP page fault handling
2019-08-19 12:08 [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Leon Romanovsky
2019-08-19 12:08 ` [PATCH mlx5-next v3 1/3] net/mlx5: Set ODP capabilities for DC transport to max Leon Romanovsky
@ 2019-08-19 12:08 ` Leon Romanovsky
2019-08-19 12:08 ` [PATCH rdma-next v3 3/3] IB/mlx5: Add page fault handler for DC initiator WQE Leon Romanovsky
2019-08-27 15:51 ` [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Jason Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2019-08-19 12:08 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
From: Michael Guralnik <michaelgur@mellanox.com>
As page fault handling is initiated by FW, there is no need to check that
the ODP supports the operation and transport.
Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx5/odp.c | 48 +-------------------------------
1 file changed, 1 insertion(+), 47 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 8b155a1f0b38..e7a4ea979209 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -986,17 +986,6 @@ static int pagefault_data_segments(struct mlx5_ib_dev *dev,
return ret < 0 ? ret : npages;
}
-static const u32 mlx5_ib_odp_opcode_cap[] = {
- [MLX5_OPCODE_SEND] = IB_ODP_SUPPORT_SEND,
- [MLX5_OPCODE_SEND_IMM] = IB_ODP_SUPPORT_SEND,
- [MLX5_OPCODE_SEND_INVAL] = IB_ODP_SUPPORT_SEND,
- [MLX5_OPCODE_RDMA_WRITE] = IB_ODP_SUPPORT_WRITE,
- [MLX5_OPCODE_RDMA_WRITE_IMM] = IB_ODP_SUPPORT_WRITE,
- [MLX5_OPCODE_RDMA_READ] = IB_ODP_SUPPORT_READ,
- [MLX5_OPCODE_ATOMIC_CS] = IB_ODP_SUPPORT_ATOMIC,
- [MLX5_OPCODE_ATOMIC_FA] = IB_ODP_SUPPORT_ATOMIC,
-};
-
/*
* Parse initiator WQE. Advances the wqe pointer to point at the
* scatter-gather list, and set wqe_end to the end of the WQE.
@@ -1007,7 +996,6 @@ static int mlx5_ib_mr_initiator_pfault_handler(
{
struct mlx5_wqe_ctrl_seg *ctrl = *wqe;
u16 wqe_index = pfault->wqe.wqe_index;
- u32 transport_caps;
struct mlx5_base_av *av;
unsigned ds, opcode;
u32 qpn = qp->trans_qp.base.mqp.qpn;
@@ -1031,29 +1019,8 @@ static int mlx5_ib_mr_initiator_pfault_handler(
opcode = be32_to_cpu(ctrl->opmod_idx_opcode) &
MLX5_WQE_CTRL_OPCODE_MASK;
- switch (qp->ibqp.qp_type) {
- case IB_QPT_XRC_INI:
+ if (qp->ibqp.qp_type == IB_QPT_XRC_INI)
*wqe += sizeof(struct mlx5_wqe_xrc_seg);
- transport_caps = dev->odp_caps.per_transport_caps.xrc_odp_caps;
- break;
- case IB_QPT_RC:
- transport_caps = dev->odp_caps.per_transport_caps.rc_odp_caps;
- break;
- case IB_QPT_UD:
- transport_caps = dev->odp_caps.per_transport_caps.ud_odp_caps;
- break;
- default:
- mlx5_ib_err(dev, "ODP fault on QP of an unsupported transport 0x%x\n",
- qp->ibqp.qp_type);
- return -EFAULT;
- }
-
- if (unlikely(opcode >= ARRAY_SIZE(mlx5_ib_odp_opcode_cap) ||
- !(transport_caps & mlx5_ib_odp_opcode_cap[opcode]))) {
- mlx5_ib_err(dev, "ODP fault on QP of an unsupported opcode 0x%x\n",
- opcode);
- return -EFAULT;
- }
if (qp->ibqp.qp_type == IB_QPT_UD) {
av = *wqe;
@@ -1118,19 +1085,6 @@ static int mlx5_ib_mr_responder_pfault_handler_rq(struct mlx5_ib_dev *dev,
return -EFAULT;
}
- switch (qp->ibqp.qp_type) {
- case IB_QPT_RC:
- if (!(dev->odp_caps.per_transport_caps.rc_odp_caps &
- IB_ODP_SUPPORT_RECV))
- goto invalid_transport_or_opcode;
- break;
- default:
-invalid_transport_or_opcode:
- mlx5_ib_err(dev, "ODP fault on QP of an unsupported transport. transport: 0x%x\n",
- qp->ibqp.qp_type);
- return -EFAULT;
- }
-
*wqe_end = wqe + wqe_size;
return 0;
--
2.20.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH rdma-next v3 3/3] IB/mlx5: Add page fault handler for DC initiator WQE
2019-08-19 12:08 [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Leon Romanovsky
2019-08-19 12:08 ` [PATCH mlx5-next v3 1/3] net/mlx5: Set ODP capabilities for DC transport to max Leon Romanovsky
2019-08-19 12:08 ` [PATCH rdma-next v3 2/3] IB/mlx5: Remove check of FW capabilities in ODP page fault handling Leon Romanovsky
@ 2019-08-19 12:08 ` Leon Romanovsky
2019-08-27 15:51 ` [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Jason Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2019-08-19 12:08 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
From: Michael Guralnik <michaelgur@mellanox.com>
Parsing DC initiator WQEs upon page fault requires skipping an address
vector segment, as in UD WQEs.
Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx5/odp.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index e7a4ea979209..e6903e90aaf1 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -1022,7 +1022,8 @@ static int mlx5_ib_mr_initiator_pfault_handler(
if (qp->ibqp.qp_type == IB_QPT_XRC_INI)
*wqe += sizeof(struct mlx5_wqe_xrc_seg);
- if (qp->ibqp.qp_type == IB_QPT_UD) {
+ if (qp->ibqp.qp_type == IB_QPT_UD ||
+ qp->qp_sub_type == MLX5_IB_QPT_DCI) {
av = *wqe;
if (av->dqp_dct & cpu_to_be32(MLX5_EXTENDED_UD_AV))
*wqe += sizeof(struct mlx5_av);
--
2.20.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs
2019-08-19 12:08 [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Leon Romanovsky
` (2 preceding siblings ...)
2019-08-19 12:08 ` [PATCH rdma-next v3 3/3] IB/mlx5: Add page fault handler for DC initiator WQE Leon Romanovsky
@ 2019-08-27 15:51 ` Jason Gunthorpe
2019-08-28 7:06 ` Leon Romanovsky
3 siblings, 1 reply; 7+ messages in thread
From: Jason Gunthorpe @ 2019-08-27 15:51 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Leon Romanovsky, RDMA mailing list,
Michael Guralnik, Saeed Mahameed, linux-netdev
On Mon, Aug 19, 2019 at 03:08:12PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> Changelog
> v3:
> * Rewrote patches to expose through DEVX without need to change mlx5-abi.h at all.
> v2: https://lore.kernel.org/linux-rdma/20190806074807.9111-1-leon@kernel.org
> * Fixed reserved_* field wrong name (Saeed M.)
> * Split first patch to two patches, one for mlx5-next and one for rdma-next. (Saeed M.)
> v1: https://lore.kernel.org/linux-rdma/20190804100048.32671-1-leon@kernel.org
> * Fixed alignment to u64 in mlx5-abi.h (Gal P.)
> v0: https://lore.kernel.org/linux-rdma/20190801122139.25224-1-leon@kernel.org
>
> >From Michael,
>
> The series adds support for on-demand paging for DC transport.
>
> As DC is mlx-only transport, the capabilities are exposed
> to the user using DEVX objects and later on through mlx5dv_query_device.
>
> Thanks
>
> Michael Guralnik (3):
> net/mlx5: Set ODP capabilities for DC transport to max
> IB/mlx5: Remove check of FW capabilities in ODP page fault handling
> IB/mlx5: Add page fault handler for DC initiator WQE
This seems fine, can you put the commit on the shared branch?
Thanks,
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs
2019-08-27 15:51 ` [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Jason Gunthorpe
@ 2019-08-28 7:06 ` Leon Romanovsky
2019-08-28 15:18 ` Jason Gunthorpe
0 siblings, 1 reply; 7+ messages in thread
From: Leon Romanovsky @ 2019-08-28 7:06 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
On Tue, Aug 27, 2019 at 12:51:40PM -0300, Jason Gunthorpe wrote:
> On Mon, Aug 19, 2019 at 03:08:12PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > Changelog
> > v3:
> > * Rewrote patches to expose through DEVX without need to change mlx5-abi.h at all.
> > v2: https://lore.kernel.org/linux-rdma/20190806074807.9111-1-leon@kernel.org
> > * Fixed reserved_* field wrong name (Saeed M.)
> > * Split first patch to two patches, one for mlx5-next and one for rdma-next. (Saeed M.)
> > v1: https://lore.kernel.org/linux-rdma/20190804100048.32671-1-leon@kernel.org
> > * Fixed alignment to u64 in mlx5-abi.h (Gal P.)
> > v0: https://lore.kernel.org/linux-rdma/20190801122139.25224-1-leon@kernel.org
> >
> > >From Michael,
> >
> > The series adds support for on-demand paging for DC transport.
> >
> > As DC is mlx-only transport, the capabilities are exposed
> > to the user using DEVX objects and later on through mlx5dv_query_device.
> >
> > Thanks
> >
> > Michael Guralnik (3):
> > net/mlx5: Set ODP capabilities for DC transport to max
> > IB/mlx5: Remove check of FW capabilities in ODP page fault handling
> > IB/mlx5: Add page fault handler for DC initiator WQE
>
> This seems fine, can you put the commit on the shared branch?
Thanks, applied to mlx5-next
00679b631edd net/mlx5: Set ODP capabilities for DC transport to max
>
> Thanks,
> Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs
2019-08-28 7:06 ` Leon Romanovsky
@ 2019-08-28 15:18 ` Jason Gunthorpe
0 siblings, 0 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2019-08-28 15:18 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, RDMA mailing list, Michael Guralnik,
Saeed Mahameed, linux-netdev
On Wed, Aug 28, 2019 at 10:06:20AM +0300, Leon Romanovsky wrote:
> On Tue, Aug 27, 2019 at 12:51:40PM -0300, Jason Gunthorpe wrote:
> > On Mon, Aug 19, 2019 at 03:08:12PM +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > Changelog
> > > v3:
> > > * Rewrote patches to expose through DEVX without need to change mlx5-abi.h at all.
> > > v2: https://lore.kernel.org/linux-rdma/20190806074807.9111-1-leon@kernel.org
> > > * Fixed reserved_* field wrong name (Saeed M.)
> > > * Split first patch to two patches, one for mlx5-next and one for rdma-next. (Saeed M.)
> > > v1: https://lore.kernel.org/linux-rdma/20190804100048.32671-1-leon@kernel.org
> > > * Fixed alignment to u64 in mlx5-abi.h (Gal P.)
> > > v0: https://lore.kernel.org/linux-rdma/20190801122139.25224-1-leon@kernel.org
> > >
> > > >From Michael,
> > >
> > > The series adds support for on-demand paging for DC transport.
> > >
> > > As DC is mlx-only transport, the capabilities are exposed
> > > to the user using DEVX objects and later on through mlx5dv_query_device.
> > >
> > > Thanks
> > >
> > > Michael Guralnik (3):
> > > net/mlx5: Set ODP capabilities for DC transport to max
> > > IB/mlx5: Remove check of FW capabilities in ODP page fault handling
> > > IB/mlx5: Add page fault handler for DC initiator WQE
> >
> > This seems fine, can you put the commit on the shared branch?
>
> Thanks, applied to mlx5-next
> 00679b631edd net/mlx5: Set ODP capabilities for DC transport to max
Done, thanks
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-08-28 15:18 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-19 12:08 [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Leon Romanovsky
2019-08-19 12:08 ` [PATCH mlx5-next v3 1/3] net/mlx5: Set ODP capabilities for DC transport to max Leon Romanovsky
2019-08-19 12:08 ` [PATCH rdma-next v3 2/3] IB/mlx5: Remove check of FW capabilities in ODP page fault handling Leon Romanovsky
2019-08-19 12:08 ` [PATCH rdma-next v3 3/3] IB/mlx5: Add page fault handler for DC initiator WQE Leon Romanovsky
2019-08-27 15:51 ` [PATCH rdma-next v3 0/3] ODP support for mlx5 DC QPs Jason Gunthorpe
2019-08-28 7:06 ` Leon Romanovsky
2019-08-28 15:18 ` Jason Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).