From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Stephen Bates" <sbates@raithlin.com>, "Christoph Hellwig" <hch@lst.de>, "Dan Williams" <dan.j.williams@intel.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Christian König" <christian.koenig@amd.com>, "John Hubbard" <jhubbard@nvidia.com>, "Don Dutile" <ddutile@redhat.com>, "Matthew Wilcox" <willy@infradead.org>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Ira Weiny" <ira.weiny@intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Bjorn Helgaas" <bhelgaas@google.com> Subject: [PATCH v5 06/24] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Date: Thu, 27 Jan 2022 17:25:56 -0700 [thread overview] Message-ID: <20220128002614.6136-7-logang@deltatee.com> (raw) In-Reply-To: <20220128002614.6136-1-logang@deltatee.com> Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() implementations. It takes an scatterlist segment that must point to a pci_p2pdma struct page and will map it if the mapping requires a bus address. The return value indicates whether the mapping required a bus address or whether the caller still needs to map the segment normally. If the segment should not be mapped, -EREMOTEIO is returned. This helper uses a state structure to track the changes to the pgmap across calls and avoid needing to lookup into the xarray for every page. Also add pci_p2pdma_map_bus_segment() which is useful for IOMMU dma_map_sg() implementations where the sg segment containing the page differs from the sg segment containing the DMA address. Prototypes for these helpers are added to dma-map-ops.h as they are only useful to dma map implementations and don't need to pollute the public pci-p2pdma header. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> --- drivers/pci/p2pdma.c | 59 +++++++++++++++++++++++++++++++++++++ include/linux/dma-map-ops.h | 21 +++++++++++++ 2 files changed, 80 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 751d682e66ae..2b2cf00664e7 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -945,6 +945,65 @@ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, } EXPORT_SYMBOL_GPL(pci_p2pdma_unmap_sg_attrs); +/** + * pci_p2pdma_map_segment - map an sg segment determining the mapping type + * @state: State structure that should be declared outside of the for_each_sg() + * loop and initialized to zero. + * @dev: DMA device that's doing the mapping operation + * @sg: scatterlist segment to map + * + * This is a helper to be used by non-IOMMU dma_map_sg() implementations where + * the sg segment is the same for the page_link and the dma_address. + * + * Attempt to map a single segment in an SGL with the PCI bus address. + * The segment must point to a PCI P2PDMA page and thus must be + * wrapped in a is_pci_p2pdma_page(sg_page(sg)) check. + * + * Returns the type of mapping used and maps the page if the type is + * PCI_P2PDMA_MAP_BUS_ADDR. + */ +enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg) +{ + if (state->pgmap != sg_page(sg)->pgmap) { + state->pgmap = sg_page(sg)->pgmap; + state->map = pci_p2pdma_map_type(state->pgmap, dev); + state->bus_off = to_p2p_pgmap(state->pgmap)->bus_offset; + } + + if (state->map == PCI_P2PDMA_MAP_BUS_ADDR) { + sg->dma_address = sg_phys(sg) + state->bus_off; + sg_dma_len(sg) = sg->length; + sg_dma_mark_bus_address(sg); + } + + return state->map; +} + +/** + * pci_p2pdma_map_bus_segment - map an sg segment pre determined to + * be mapped with PCI_P2PDMA_MAP_BUS_ADDR + * @pg_sg: scatterlist segment with the page to map + * @dma_sg: scatterlist segment to assign a DMA address to + * + * This is a helper for iommu dma_map_sg() implementations when the + * segment for the DMA address differs from the segment containing the + * source page. + * + * pci_p2pdma_map_type() must have already been called on the pg_sg and + * returned PCI_P2PDMA_MAP_BUS_ADDR. + */ +void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg) +{ + struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(sg_page(pg_sg)->pgmap); + + dma_sg->dma_address = sg_phys(pg_sg) + pgmap->bus_offset; + sg_dma_len(dma_sg) = pg_sg->length; + sg_dma_mark_bus_address(dma_sg); +} + /** * pci_p2pdma_enable_store - parse a configfs/sysfs attribute store * to enable p2pdma diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index d693a0e33bac..752f91e5eb5d 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -413,15 +413,36 @@ enum pci_p2pdma_map_type { PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, }; +struct pci_p2pdma_map_state { + struct dev_pagemap *pgmap; + int map; + u64 bus_off; +}; + #ifdef CONFIG_PCI_P2PDMA enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap, struct device *dev); +enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg); +void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg); #else /* CONFIG_PCI_P2PDMA */ static inline enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap, struct device *dev) { return PCI_P2PDMA_MAP_NOT_SUPPORTED; } +static inline enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg) +{ + return PCI_P2PDMA_MAP_NOT_SUPPORTED; +} +static inline void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg) +{ +} #endif /* CONFIG_PCI_P2PDMA */ #endif /* _LINUX_DMA_MAP_OPS_H */ -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Stephen Bates" <sbates@raithlin.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Ira Weiny" <ira.weiny@intel.com>, "Christoph Hellwig" <hch@lst.de>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Matthew Wilcox" <willy@infradead.org>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Ralph Campbell" <rcampbell@nvidia.com>, "John Hubbard" <jhubbard@nvidia.com>, "Bjorn Helgaas" <bhelgaas@google.com>, "Dan Williams" <dan.j.williams@intel.com>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Robin Murphy" <robin.murphy@arm.com>, "Christian König" <christian.koenig@amd.com> Subject: [PATCH v5 06/24] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Date: Thu, 27 Jan 2022 17:25:56 -0700 [thread overview] Message-ID: <20220128002614.6136-7-logang@deltatee.com> (raw) In-Reply-To: <20220128002614.6136-1-logang@deltatee.com> Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() implementations. It takes an scatterlist segment that must point to a pci_p2pdma struct page and will map it if the mapping requires a bus address. The return value indicates whether the mapping required a bus address or whether the caller still needs to map the segment normally. If the segment should not be mapped, -EREMOTEIO is returned. This helper uses a state structure to track the changes to the pgmap across calls and avoid needing to lookup into the xarray for every page. Also add pci_p2pdma_map_bus_segment() which is useful for IOMMU dma_map_sg() implementations where the sg segment containing the page differs from the sg segment containing the DMA address. Prototypes for these helpers are added to dma-map-ops.h as they are only useful to dma map implementations and don't need to pollute the public pci-p2pdma header. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> --- drivers/pci/p2pdma.c | 59 +++++++++++++++++++++++++++++++++++++ include/linux/dma-map-ops.h | 21 +++++++++++++ 2 files changed, 80 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 751d682e66ae..2b2cf00664e7 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -945,6 +945,65 @@ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, } EXPORT_SYMBOL_GPL(pci_p2pdma_unmap_sg_attrs); +/** + * pci_p2pdma_map_segment - map an sg segment determining the mapping type + * @state: State structure that should be declared outside of the for_each_sg() + * loop and initialized to zero. + * @dev: DMA device that's doing the mapping operation + * @sg: scatterlist segment to map + * + * This is a helper to be used by non-IOMMU dma_map_sg() implementations where + * the sg segment is the same for the page_link and the dma_address. + * + * Attempt to map a single segment in an SGL with the PCI bus address. + * The segment must point to a PCI P2PDMA page and thus must be + * wrapped in a is_pci_p2pdma_page(sg_page(sg)) check. + * + * Returns the type of mapping used and maps the page if the type is + * PCI_P2PDMA_MAP_BUS_ADDR. + */ +enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg) +{ + if (state->pgmap != sg_page(sg)->pgmap) { + state->pgmap = sg_page(sg)->pgmap; + state->map = pci_p2pdma_map_type(state->pgmap, dev); + state->bus_off = to_p2p_pgmap(state->pgmap)->bus_offset; + } + + if (state->map == PCI_P2PDMA_MAP_BUS_ADDR) { + sg->dma_address = sg_phys(sg) + state->bus_off; + sg_dma_len(sg) = sg->length; + sg_dma_mark_bus_address(sg); + } + + return state->map; +} + +/** + * pci_p2pdma_map_bus_segment - map an sg segment pre determined to + * be mapped with PCI_P2PDMA_MAP_BUS_ADDR + * @pg_sg: scatterlist segment with the page to map + * @dma_sg: scatterlist segment to assign a DMA address to + * + * This is a helper for iommu dma_map_sg() implementations when the + * segment for the DMA address differs from the segment containing the + * source page. + * + * pci_p2pdma_map_type() must have already been called on the pg_sg and + * returned PCI_P2PDMA_MAP_BUS_ADDR. + */ +void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg) +{ + struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(sg_page(pg_sg)->pgmap); + + dma_sg->dma_address = sg_phys(pg_sg) + pgmap->bus_offset; + sg_dma_len(dma_sg) = pg_sg->length; + sg_dma_mark_bus_address(dma_sg); +} + /** * pci_p2pdma_enable_store - parse a configfs/sysfs attribute store * to enable p2pdma diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index d693a0e33bac..752f91e5eb5d 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -413,15 +413,36 @@ enum pci_p2pdma_map_type { PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, }; +struct pci_p2pdma_map_state { + struct dev_pagemap *pgmap; + int map; + u64 bus_off; +}; + #ifdef CONFIG_PCI_P2PDMA enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap, struct device *dev); +enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg); +void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg); #else /* CONFIG_PCI_P2PDMA */ static inline enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap, struct device *dev) { return PCI_P2PDMA_MAP_NOT_SUPPORTED; } +static inline enum pci_p2pdma_map_type +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, + struct scatterlist *sg) +{ + return PCI_P2PDMA_MAP_NOT_SUPPORTED; +} +static inline void pci_p2pdma_map_bus_segment(struct scatterlist *pg_sg, + struct scatterlist *dma_sg) +{ +} #endif /* CONFIG_PCI_P2PDMA */ #endif /* _LINUX_DMA_MAP_OPS_H */ -- 2.30.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2022-01-28 0:26 UTC|newest] Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-01-28 0:25 [PATCH v5 00/24] Userspace P2PDMA with O_DIRECT NVMe devices Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 01/24] ext4/xfs: add page refcount helper Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 9:26 ` Chaitanya Kulkarni 2022-01-28 9:26 ` Chaitanya Kulkarni via iommu 2022-01-28 0:25 ` [PATCH v5 02/24] mm: remove extra ZONE_DEVICE struct page refcount Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 14:21 ` Jason Gunthorpe 2022-01-28 14:21 ` Jason Gunthorpe 2022-01-28 16:51 ` Logan Gunthorpe 2022-01-28 16:51 ` Logan Gunthorpe 2022-01-28 17:16 ` Ralph Campbell 2022-01-28 17:16 ` Ralph Campbell via iommu 2022-01-28 0:25 ` [PATCH v5 03/24] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 04/24] PCI/P2PDMA: Attempt to set map_type if it has not been set Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 05/24] PCI/P2PDMA: Expose pci_p2pdma_map_type() Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe [this message] 2022-01-28 0:25 ` [PATCH v5 06/24] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 07/24] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 08/24] dma-direct: support PCI P2PDMA pages in dma-direct map_sg Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-02-01 20:53 ` Jonathan Derrick 2022-02-01 20:53 ` Jonathan Derrick 2022-02-01 20:57 ` Logan Gunthorpe 2022-02-01 20:57 ` Logan Gunthorpe 2022-01-28 0:25 ` [PATCH v5 09/24] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support Logan Gunthorpe 2022-01-28 0:25 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 10/24] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 11/24] nvme-pci: check DMA ops when indicating support for PCI P2PDMA Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 12/24] nvme-pci: convert to using dma_map_sgtable() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 9:13 ` Chaitanya Kulkarni 2022-01-28 9:13 ` Chaitanya Kulkarni via iommu 2022-01-28 0:26 ` [PATCH v5 13/24] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 14/24] RDMA/rw: drop pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 15/24] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 16/24] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 17/24] iov_iter: introduce iov_iter_get_pages_[alloc_]flags() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 18/24] block: add check when merging zone device pages Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 19/24] lib/scatterlist: " Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 20/24] block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 21/24] block: set FOLL_PCI_P2PDMA in bio_map_user_iov() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 22/24] mm: use custom page_free for P2PDMA pages Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 14:22 ` Jason Gunthorpe 2022-01-28 14:22 ` Jason Gunthorpe 2022-01-28 16:52 ` Logan Gunthorpe 2022-01-28 16:52 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 23/24] PCI/P2PDMA: Introduce pci_mmap_p2pmem() Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-28 0:26 ` [PATCH v5 24/24] nvme-pci: allow mmaping the CMB in userspace Logan Gunthorpe 2022-01-28 0:26 ` Logan Gunthorpe 2022-01-31 18:56 ` [PATCH v5 00/24] Userspace P2PDMA with O_DIRECT NVMe devices Jonathan Derrick 2022-01-31 18:56 ` Jonathan Derrick 2022-01-31 19:00 ` Logan Gunthorpe 2022-01-31 19:00 ` Logan Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220128002614.6136-7-logang@deltatee.com \ --to=logang@deltatee.com \ --cc=andrzej.jakowski@intel.com \ --cc=bhelgaas@google.com \ --cc=christian.koenig@amd.com \ --cc=ckulkarnilinux@gmail.com \ --cc=dan.j.williams@intel.com \ --cc=daniel.vetter@ffwll.ch \ --cc=dave.b.minturn@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=ddutile@redhat.com \ --cc=hch@lst.de \ --cc=helgaas@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=ira.weiny@intel.com \ --cc=jason@jlekstrand.net \ --cc=jgg@ziepe.ca \ --cc=jhubbard@nvidia.com \ --cc=jianxin.xiong@intel.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=martin.oliveira@eideticom.com \ --cc=rcampbell@nvidia.com \ --cc=robin.murphy@arm.com \ --cc=sbates@raithlin.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.