From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Stephen Bates" <sbates@raithlin.com>, "Christoph Hellwig" <hch@lst.de>, "Dan Williams" <dan.j.williams@intel.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Christian König" <christian.koenig@amd.com>, "John Hubbard" <jhubbard@nvidia.com>, "Don Dutile" <ddutile@redhat.com>, "Matthew Wilcox" <willy@infradead.org>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Ira Weiny" <ira.weiny@intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Jason Gunthorpe" <jgg@nvidia.com>, "Max Gurtovoy" <mgurtovoy@nvidia.com> Subject: [PATCH v6 11/21] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Date: Thu, 7 Apr 2022 09:47:07 -0600 [thread overview] Message-ID: <20220407154717.7695-12-logang@deltatee.com> (raw) In-Reply-To: <20220407154717.7695-1-logang@deltatee.com> Introduce the helper function ib_dma_pci_p2p_dma_supported() to check if a given ib_device can be used in P2PDMA transfers. This ensures the ib_device is not using virt_dma and also that the underlying dma_device supports P2PDMA. Use the new helper in nvme-rdma to replace the existing check for ib_uses_virt_dma(). Adding the dma_pci_p2pdma_supported() check allows switching away from pci_p2pdma_[un]map_sg(). Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Max Gurtovoy <mgurtovoy@nvidia.com> --- drivers/nvme/target/rdma.c | 2 +- include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 2fab0b219b25..12258f87ccc8 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -415,7 +415,7 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) goto out_free_rsp; - if (!ib_uses_virt_dma(ndev->device)) + if (ib_dma_pci_p2p_dma_supported(ndev->device)) r->req.p2p_client = &ndev->device->dev; r->send_sge.length = sizeof(*r->req.cqe); r->send_sge.lkey = ndev->pd->local_dma_lkey; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 69d883f7fb41..79609ab73014 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4003,6 +4003,17 @@ static inline bool ib_uses_virt_dma(struct ib_device *dev) return IS_ENABLED(CONFIG_INFINIBAND_VIRT_DMA) && !dev->dma_device; } +/* + * Check if a IB device's underlying DMA mapping supports P2PDMA transfers. + */ +static inline bool ib_dma_pci_p2p_dma_supported(struct ib_device *dev) +{ + if (ib_uses_virt_dma(dev)) + return false; + + return dma_pci_p2pdma_supported(dev->dma_device); +} + /** * ib_dma_mapping_error - check a DMA addr for error * @dev: The device for which the dma_addr was created -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Stephen Bates" <sbates@raithlin.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Ira Weiny" <ira.weiny@intel.com>, "Christoph Hellwig" <hch@lst.de>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Matthew Wilcox" <willy@infradead.org>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Jason Gunthorpe" <jgg@nvidia.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "John Hubbard" <jhubbard@nvidia.com>, "Dan Williams" <dan.j.williams@intel.com>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Max Gurtovoy" <mgurtovoy@nvidia.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Robin Murphy" <robin.murphy@arm.com>, "Christian König" <christian.koenig@amd.com> Subject: [PATCH v6 11/21] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Date: Thu, 7 Apr 2022 09:47:07 -0600 [thread overview] Message-ID: <20220407154717.7695-12-logang@deltatee.com> (raw) In-Reply-To: <20220407154717.7695-1-logang@deltatee.com> Introduce the helper function ib_dma_pci_p2p_dma_supported() to check if a given ib_device can be used in P2PDMA transfers. This ensures the ib_device is not using virt_dma and also that the underlying dma_device supports P2PDMA. Use the new helper in nvme-rdma to replace the existing check for ib_uses_virt_dma(). Adding the dma_pci_p2pdma_supported() check allows switching away from pci_p2pdma_[un]map_sg(). Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Max Gurtovoy <mgurtovoy@nvidia.com> --- drivers/nvme/target/rdma.c | 2 +- include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 2fab0b219b25..12258f87ccc8 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -415,7 +415,7 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) goto out_free_rsp; - if (!ib_uses_virt_dma(ndev->device)) + if (ib_dma_pci_p2p_dma_supported(ndev->device)) r->req.p2p_client = &ndev->device->dev; r->send_sge.length = sizeof(*r->req.cqe); r->send_sge.lkey = ndev->pd->local_dma_lkey; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 69d883f7fb41..79609ab73014 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4003,6 +4003,17 @@ static inline bool ib_uses_virt_dma(struct ib_device *dev) return IS_ENABLED(CONFIG_INFINIBAND_VIRT_DMA) && !dev->dma_device; } +/* + * Check if a IB device's underlying DMA mapping supports P2PDMA transfers. + */ +static inline bool ib_dma_pci_p2p_dma_supported(struct ib_device *dev) +{ + if (ib_uses_virt_dma(dev)) + return false; + + return dma_pci_p2pdma_supported(dev->dma_device); +} + /** * ib_dma_mapping_error - check a DMA addr for error * @dev: The device for which the dma_addr was created -- 2.30.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2022-04-07 17:28 UTC|newest] Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-04-07 15:46 [PATCH v6 00/21] Userspace P2PDMA with O_DIRECT NVMe devices Logan Gunthorpe 2022-04-07 15:46 ` Logan Gunthorpe 2022-04-07 15:46 ` [PATCH v6 01/21] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL Logan Gunthorpe 2022-04-07 15:46 ` Logan Gunthorpe 2022-04-07 15:46 ` [PATCH v6 02/21] PCI/P2PDMA: Attempt to set map_type if it has not been set Logan Gunthorpe 2022-04-07 15:46 ` Logan Gunthorpe 2022-04-07 15:46 ` [PATCH v6 03/21] PCI/P2PDMA: Expose pci_p2pdma_map_type() Logan Gunthorpe 2022-04-07 15:46 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 04/21] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 05/21] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 06/21] dma-direct: support PCI P2PDMA pages in dma-direct map_sg Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 07/21] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 08/21] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 09/21] nvme-pci: check DMA ops when indicating support for PCI P2PDMA Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 10/21] nvme-pci: convert to using dma_map_sgtable() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe [this message] 2022-04-07 15:47 ` [PATCH v6 11/21] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 12/21] RDMA/rw: drop pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 13/21] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 14/21] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 15/21] iov_iter: introduce iov_iter_get_pages_[alloc_]flags() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 16/21] block: add check when merging zone device pages Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 17/21] lib/scatterlist: " Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 18/21] block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 19/21] block: set FOLL_PCI_P2PDMA in bio_map_user_iov() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 20/21] PCI/P2PDMA: Introduce pci_mmap_p2pmem() Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-05-27 12:55 ` Jason Gunthorpe 2022-05-27 12:55 ` Jason Gunthorpe 2022-05-27 15:35 ` Logan Gunthorpe 2022-05-27 15:35 ` Logan Gunthorpe 2022-05-27 19:03 ` Jason Gunthorpe 2022-05-27 19:03 ` Jason Gunthorpe 2022-05-27 22:41 ` Logan Gunthorpe 2022-05-27 22:41 ` Logan Gunthorpe 2022-06-02 0:00 ` Jason Gunthorpe 2022-06-02 0:00 ` Jason Gunthorpe 2022-06-02 16:16 ` Logan Gunthorpe 2022-06-02 16:16 ` Logan Gunthorpe 2022-06-02 16:30 ` Jason Gunthorpe 2022-06-02 16:30 ` Jason Gunthorpe 2022-06-02 16:45 ` Logan Gunthorpe 2022-06-02 16:45 ` Logan Gunthorpe 2022-06-02 17:18 ` Jason Gunthorpe 2022-06-02 17:18 ` Jason Gunthorpe 2022-06-02 16:49 ` Logan Gunthorpe 2022-06-02 16:49 ` Logan Gunthorpe 2022-06-02 17:28 ` Jason Gunthorpe 2022-06-02 17:28 ` Jason Gunthorpe 2022-06-02 17:49 ` Logan Gunthorpe 2022-06-02 17:49 ` Logan Gunthorpe 2022-04-07 15:47 ` [PATCH v6 21/21] nvme-pci: allow mmaping the CMB in userspace Logan Gunthorpe 2022-04-07 15:47 ` Logan Gunthorpe 2022-05-16 22:31 ` [PATCH v6 00/21] Userspace P2PDMA with O_DIRECT NVMe devices Chaitanya Kulkarni 2022-05-16 22:31 ` Chaitanya Kulkarni via iommu 2022-05-16 22:35 ` Logan Gunthorpe 2022-05-16 22:35 ` Logan Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220407154717.7695-12-logang@deltatee.com \ --to=logang@deltatee.com \ --cc=andrzej.jakowski@intel.com \ --cc=christian.koenig@amd.com \ --cc=ckulkarnilinux@gmail.com \ --cc=dan.j.williams@intel.com \ --cc=daniel.vetter@ffwll.ch \ --cc=dave.b.minturn@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=ddutile@redhat.com \ --cc=hch@lst.de \ --cc=helgaas@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=ira.weiny@intel.com \ --cc=jason@jlekstrand.net \ --cc=jgg@nvidia.com \ --cc=jgg@ziepe.ca \ --cc=jhubbard@nvidia.com \ --cc=jianxin.xiong@intel.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=martin.oliveira@eideticom.com \ --cc=mgurtovoy@nvidia.com \ --cc=rcampbell@nvidia.com \ --cc=robin.murphy@arm.com \ --cc=sbates@raithlin.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.