All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Christian König" <christian.koenig@amd.com>,
	dri-devel@lists.freedesktop.org,
	"Leon Romanovsky" <leon@kernel.org>,
	linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org,
	linux-rdma@vger.kernel.org, netdev@vger.kernel.org,
	"Saeed Mahameed" <saeedm@nvidia.com>,
	"Sumit Semwal" <sumit.semwal@linaro.org>
Cc: Kamal Heib <kamalheib1@gmail.com>,
	Mohammad Kabat <mohammadkab@nvidia.com>
Subject: [PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS
Date: Thu,  1 Sep 2022 11:20:52 -0300	[thread overview]
Message-ID: <0-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com> (raw)

This series adds support for DMABUF when creating a devx umem. devx umems
are quite similar to MR's execpt they cannot be revoked, so this uses the
dmabuf pinned memory flow. Several mlx5dv flows require umem and cannot
work with MR.

The intended use case is primarily for P2P transfers using dmabuf as a
handle to the underlying PCI BAR memory from the exporter. When a PCI
switch is present the P2P transfers can bypass the host bridge completely
and go directly through the switch. ATS allows this bypass to function in
more cases as translated TLPs issued after an ATS query allows the request
redirect setting to be bypassed in the switch.

Have mlx5 automatically use ATS in places where it makes sense.

Jason Gunthorpe (4):
  net/mlx5: Add IFC bits for mkey ATS
  RDMA/core: Add UVERBS_ATTR_RAW_FD
  RDMA/mlx5: Add support for dmabuf to devx umem
  RDMA/mlx5: Enable ATS support for MRs and umems

 drivers/infiniband/core/uverbs_ioctl.c   |  8 ++++
 drivers/infiniband/hw/mlx5/devx.c        | 55 +++++++++++++++++-------
 drivers/infiniband/hw/mlx5/mlx5_ib.h     | 36 ++++++++++++++++
 drivers/infiniband/hw/mlx5/mr.c          |  5 ++-
 include/linux/mlx5/mlx5_ifc.h            | 11 +++--
 include/rdma/uverbs_ioctl.h              | 13 ++++++
 include/uapi/rdma/mlx5_user_ioctl_cmds.h |  1 +
 7 files changed, 109 insertions(+), 20 deletions(-)


base-commit: b90cb1053190353cc30f0fef0ef1f378ccc063c5
-- 
2.37.2


WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Christian König" <christian.koenig@amd.com>,
	dri-devel@lists.freedesktop.org,
	"Leon Romanovsky" <leon@kernel.org>,
	linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org,
	linux-rdma@vger.kernel.org, netdev@vger.kernel.org,
	"Saeed Mahameed" <saeedm@nvidia.com>,
	"Sumit Semwal" <sumit.semwal@linaro.org>
Cc: Mohammad Kabat <mohammadkab@nvidia.com>,
	Kamal Heib <kamalheib1@gmail.com>
Subject: [PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS
Date: Thu,  1 Sep 2022 11:20:52 -0300	[thread overview]
Message-ID: <0-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com> (raw)

This series adds support for DMABUF when creating a devx umem. devx umems
are quite similar to MR's execpt they cannot be revoked, so this uses the
dmabuf pinned memory flow. Several mlx5dv flows require umem and cannot
work with MR.

The intended use case is primarily for P2P transfers using dmabuf as a
handle to the underlying PCI BAR memory from the exporter. When a PCI
switch is present the P2P transfers can bypass the host bridge completely
and go directly through the switch. ATS allows this bypass to function in
more cases as translated TLPs issued after an ATS query allows the request
redirect setting to be bypassed in the switch.

Have mlx5 automatically use ATS in places where it makes sense.

Jason Gunthorpe (4):
  net/mlx5: Add IFC bits for mkey ATS
  RDMA/core: Add UVERBS_ATTR_RAW_FD
  RDMA/mlx5: Add support for dmabuf to devx umem
  RDMA/mlx5: Enable ATS support for MRs and umems

 drivers/infiniband/core/uverbs_ioctl.c   |  8 ++++
 drivers/infiniband/hw/mlx5/devx.c        | 55 +++++++++++++++++-------
 drivers/infiniband/hw/mlx5/mlx5_ib.h     | 36 ++++++++++++++++
 drivers/infiniband/hw/mlx5/mr.c          |  5 ++-
 include/linux/mlx5/mlx5_ifc.h            | 11 +++--
 include/rdma/uverbs_ioctl.h              | 13 ++++++
 include/uapi/rdma/mlx5_user_ioctl_cmds.h |  1 +
 7 files changed, 109 insertions(+), 20 deletions(-)


base-commit: b90cb1053190353cc30f0fef0ef1f378ccc063c5
-- 
2.37.2


             reply	other threads:[~2022-09-01 14:21 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-01 14:20 Jason Gunthorpe [this message]
2022-09-01 14:20 ` [PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS Jason Gunthorpe
2022-09-01 14:20 ` [PATCH 1/4] net/mlx5: Add IFC bits for mkey ATS Jason Gunthorpe
2022-09-01 14:20   ` Jason Gunthorpe
2022-09-01 14:20 ` [PATCH 2/4] RDMA/core: Add UVERBS_ATTR_RAW_FD Jason Gunthorpe
2022-09-01 14:20   ` Jason Gunthorpe
2022-09-01 14:20 ` [PATCH 3/4] RDMA/mlx5: Add support for dmabuf to devx umem Jason Gunthorpe
2022-09-01 14:20   ` Jason Gunthorpe
2022-09-01 14:20 ` [PATCH 4/4] RDMA/mlx5: Enable ATS support for MRs and umems Jason Gunthorpe
2022-09-01 14:20   ` Jason Gunthorpe
2022-09-26 17:51 ` [PATCH 0/4] RDMA/mlx5: Support DMABUF in umems and enable ATS Jason Gunthorpe
2022-09-26 17:51   ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0-v1-bd147097458e+ede-umem_dmabuf_jgg@nvidia.com \
    --to=jgg@nvidia.com \
    --cc=christian.koenig@amd.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=kamalheib1@gmail.com \
    --cc=leon@kernel.org \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mohammadkab@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@nvidia.com \
    --cc=sumit.semwal@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.