From: Leon Romanovsky <leonro@mellanox.com>
To: "David S . Miller" <davem@davemloft.net>,
Santosh Shilimkar <santosh.shilimkar@oracle.com>
Cc: Doug Ledford <dledford@redhat.com>,
Jason Gunthorpe <jgg@mellanox.com>,
RDMA mailing list <linux-rdma@vger.kernel.org>,
Hans Westgaard Ry <hans.westgaard.ry@oracle.com>,
Moni Shoua <monis@mellanox.com>,
linux-netdev <netdev@vger.kernel.org>
Subject: Re: [PATCH mlx5-next 00/10] Use ODP MRs for kernel ULPs
Date: Thu, 16 Jan 2020 06:59:29 +0000 [thread overview]
Message-ID: <20200116065926.GD76932@unreal> (raw)
In-Reply-To: <20200115124340.79108-1-leon@kernel.org>
On Wed, Jan 15, 2020 at 02:43:30PM +0200, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> Hi,
>
> The following series extends MR creation routines to allow creation of
> user MRs through kernel ULPs as a proxy. The immediate use case is to
> allow RDS to work over FS-DAX, which requires ODP (on-demand-paging)
> MRs to be created and such MRs were not possible to create prior this
> series.
>
> The first part of this patchset extends RDMA to have special verb
> ib_reg_user_mr(). The common use case that uses this function is a userspace
> application that allocates memory for HCA access but the responsibility
> to register the memory at the HCA is on an kernel ULP. This ULP that acts
> as an agent for the userspace application.
>
> The second part provides advise MR functionality for ULPs. This is
> integral part of ODP flows and used to trigger pagefaults in advance
> to prepare memory before running working set.
>
> The third part is actual user of those in-kernel APIs.
>
> Thanks
>
> Hans Westgaard Ry (3):
> net/rds: Detect need of On-Demand-Paging memory registration
> net/rds: Handle ODP mr registration/unregistration
> net/rds: Use prefetch for On-Demand-Paging MR
>
> Jason Gunthorpe (1):
> RDMA/mlx5: Fix handling of IOVA != user_va in ODP paths
>
> Leon Romanovsky (1):
> RDMA/mlx5: Don't fake udata for kernel path
>
> Moni Shoua (5):
> IB: Allow calls to ib_umem_get from kernel ULPs
> IB/core: Introduce ib_reg_user_mr
> IB/core: Add interface to advise_mr for kernel users
> IB/mlx5: Add ODP WQE handlers for kernel QPs
> IB/mlx5: Mask out unsupported ODP capabilities for kernel QPs
>
> drivers/infiniband/core/umem.c | 27 +--
> drivers/infiniband/core/umem_odp.c | 29 +--
> drivers/infiniband/core/verbs.c | 41 +++++
> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 12 +-
> drivers/infiniband/hw/cxgb4/mem.c | 2 +-
> drivers/infiniband/hw/efa/efa_verbs.c | 4 +-
> drivers/infiniband/hw/hns/hns_roce_cq.c | 2 +-
> drivers/infiniband/hw/hns/hns_roce_db.c | 3 +-
> drivers/infiniband/hw/hns/hns_roce_mr.c | 4 +-
> drivers/infiniband/hw/hns/hns_roce_qp.c | 2 +-
> drivers/infiniband/hw/hns/hns_roce_srq.c | 5 +-
> drivers/infiniband/hw/i40iw/i40iw_verbs.c | 5 +-
> drivers/infiniband/hw/mlx4/cq.c | 2 +-
> drivers/infiniband/hw/mlx4/doorbell.c | 3 +-
> drivers/infiniband/hw/mlx4/mr.c | 8 +-
> drivers/infiniband/hw/mlx4/qp.c | 5 +-
> drivers/infiniband/hw/mlx4/srq.c | 3 +-
> drivers/infiniband/hw/mlx5/cq.c | 6 +-
> drivers/infiniband/hw/mlx5/devx.c | 2 +-
> drivers/infiniband/hw/mlx5/doorbell.c | 3 +-
> drivers/infiniband/hw/mlx5/main.c | 51 ++++--
> drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +-
> drivers/infiniband/hw/mlx5/mr.c | 20 +--
> drivers/infiniband/hw/mlx5/odp.c | 33 ++--
> drivers/infiniband/hw/mlx5/qp.c | 167 +++++++++++-------
> drivers/infiniband/hw/mlx5/srq.c | 2 +-
> drivers/infiniband/hw/mthca/mthca_provider.c | 2 +-
> drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 2 +-
> drivers/infiniband/hw/qedr/verbs.c | 9 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c | 2 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c | 2 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c | 7 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_srq.c | 2 +-
> drivers/infiniband/sw/rdmavt/mr.c | 2 +-
> drivers/infiniband/sw/rxe/rxe_mr.c | 2 +-
> include/rdma/ib_umem.h | 4 +-
> include/rdma/ib_umem_odp.h | 6 +-
> include/rdma/ib_verbs.h | 9 +
> net/rds/ib.c | 7 +
> net/rds/ib.h | 3 +-
> net/rds/ib_mr.h | 7 +-
> net/rds/ib_rdma.c | 83 ++++++++-
> net/rds/ib_send.c | 44 +++--
> net/rds/rdma.c | 156 +++++++++++-----
> net/rds/rds.h | 13 +-
> 45 files changed, 559 insertions(+), 256 deletions(-)
Thanks Santosh for your review.
David,
Is it ok to route those patches through RDMA tree given the fact that
we are touching a lot of files in drivers/infiniband/* ?
There is no conflict between netdev and RDMA versions of RDS, but to be
on safe side, I'll put all this code to mlx5-next tree.
Thanks
>
> --
> 2.20.1
>
next prev parent reply other threads:[~2020-01-16 6:59 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-15 12:43 [PATCH mlx5-next 00/10] Use ODP MRs for kernel ULPs Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 01/10] IB: Allow calls to ib_umem_get from " Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 02/10] IB/core: Introduce ib_reg_user_mr Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 03/10] IB/core: Add interface to advise_mr for kernel users Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 04/10] IB/mlx5: Add ODP WQE handlers for kernel QPs Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 05/10] RDMA/mlx5: Don't fake udata for kernel path Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 06/10] IB/mlx5: Mask out unsupported ODP capabilities for kernel QPs Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 07/10] RDMA/mlx5: Fix handling of IOVA != user_va in ODP paths Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 08/10] net/rds: Detect need of On-Demand-Paging memory registration Leon Romanovsky
2020-01-15 21:42 ` santosh.shilimkar
2020-01-15 12:43 ` [PATCH mlx5-next 09/10] net/rds: Handle ODP mr registration/unregistration Leon Romanovsky
2020-01-15 21:51 ` santosh.shilimkar
2020-01-16 7:11 ` Leon Romanovsky
2020-01-16 7:22 ` santosh.shilimkar
2020-01-18 10:19 ` Leon Romanovsky
2020-01-15 12:43 ` [PATCH mlx5-next 10/10] net/rds: Use prefetch for On-Demand-Paging MR Leon Romanovsky
2020-01-15 21:43 ` santosh.shilimkar
2020-01-16 6:59 ` Leon Romanovsky [this message]
2020-01-16 13:57 ` [PATCH mlx5-next 00/10] Use ODP MRs for kernel ULPs Jason Gunthorpe
2020-01-16 14:04 ` Leon Romanovsky
2020-01-16 19:34 ` santosh.shilimkar
2020-01-17 14:12 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200116065926.GD76932@unreal \
--to=leonro@mellanox.com \
--cc=davem@davemloft.net \
--cc=dledford@redhat.com \
--cc=hans.westgaard.ry@oracle.com \
--cc=jgg@mellanox.com \
--cc=linux-rdma@vger.kernel.org \
--cc=monis@mellanox.com \
--cc=netdev@vger.kernel.org \
--cc=santosh.shilimkar@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).