From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Stephen Bates" <sbates@raithlin.com>, "Christoph Hellwig" <hch@lst.de>, "Dan Williams" <dan.j.williams@intel.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Christian König" <christian.koenig@amd.com>, "John Hubbard" <jhubbard@nvidia.com>, "Don Dutile" <ddutile@redhat.com>, "Matthew Wilcox" <willy@infradead.org>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Ira Weiny" <ira.weiny@intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "Logan Gunthorpe" <logang@deltatee.com> Subject: [PATCH v7 14/21] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Date: Wed, 15 Jun 2022 10:12:26 -0600 [thread overview] Message-ID: <20220615161233.17527-15-logang@deltatee.com> (raw) In-Reply-To: <20220615161233.17527-1-logang@deltatee.com> GUP Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to allow obtaining P2PDMA pages. If GUP is called without the flag and a P2PDMA page is found, it will return an error. FOLL_PCI_P2PDMA cannot be set if FOLL_LONGTERM is set. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- include/linux/mm.h | 1 + mm/gup.c | 22 +++++++++++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index bc8f326be0ce..0bcb54ea503c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2941,6 +2941,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ #define FOLL_PIN 0x40000 /* pages must be released via unpin_user_page */ #define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ +#define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ /* * FOLL_PIN and FOLL_LONGTERM may be used in various combinations with each diff --git a/mm/gup.c b/mm/gup.c index 551264407624..f15f01d06a09 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -564,6 +564,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, goto out; } + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) { + page = ERR_PTR(-EREMOTEIO); + goto out; + } + VM_BUG_ON_PAGE((flags & FOLL_PIN) && PageAnon(page) && !PageAnonExclusive(page), page); @@ -994,6 +1000,9 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma)) return -EOPNOTSUPP; + if ((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA)) + return -EOPNOTSUPP; + if (vma_is_secretmem(vma)) return -EFAULT; @@ -2289,6 +2298,10 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) + goto pte_unmap; + folio = try_grab_folio(page, 1, flags); if (!folio) goto pte_unmap; @@ -2368,6 +2381,12 @@ static int __gup_device_huge(unsigned long pfn, unsigned long addr, undo_dev_pagemap(nr, nr_start, flags, pages); break; } + + if (!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { + undo_dev_pagemap(nr, nr_start, flags, pages); + break; + } + SetPageReferenced(page); pages[*nr] = page; if (unlikely(!try_grab_page(page, flags))) { @@ -2856,7 +2875,8 @@ static int internal_get_user_pages_fast(unsigned long start, if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN | FOLL_GET | - FOLL_FAST_ONLY | FOLL_NOFAULT))) + FOLL_FAST_ONLY | FOLL_NOFAULT | + FOLL_PCI_P2PDMA))) return -EINVAL; if (gup_flags & FOLL_PIN) -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Minturn Dave B" <dave.b.minturn@intel.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "John Hubbard" <jhubbard@nvidia.com>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Matthew Wilcox" <willy@infradead.org>, "Christian König" <christian.koenig@amd.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Logan Gunthorpe" <logang@deltatee.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Bjorn Helgaas" <helgaas@kernel.org>, "Dan Williams" <dan.j.williams@intel.com>, "Stephen Bates" <sbates@raithlin.com>, "Ira Weiny" <ira.weiny@intel.com>, "Christoph Hellwig" <hch@lst.de>, "Xiong Jianxin" <jianxin.xiong@intel.com> Subject: [PATCH v7 14/21] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Date: Wed, 15 Jun 2022 10:12:26 -0600 [thread overview] Message-ID: <20220615161233.17527-15-logang@deltatee.com> (raw) In-Reply-To: <20220615161233.17527-1-logang@deltatee.com> GUP Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to allow obtaining P2PDMA pages. If GUP is called without the flag and a P2PDMA page is found, it will return an error. FOLL_PCI_P2PDMA cannot be set if FOLL_LONGTERM is set. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- include/linux/mm.h | 1 + mm/gup.c | 22 +++++++++++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index bc8f326be0ce..0bcb54ea503c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2941,6 +2941,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ #define FOLL_PIN 0x40000 /* pages must be released via unpin_user_page */ #define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ +#define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ /* * FOLL_PIN and FOLL_LONGTERM may be used in various combinations with each diff --git a/mm/gup.c b/mm/gup.c index 551264407624..f15f01d06a09 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -564,6 +564,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, goto out; } + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) { + page = ERR_PTR(-EREMOTEIO); + goto out; + } + VM_BUG_ON_PAGE((flags & FOLL_PIN) && PageAnon(page) && !PageAnonExclusive(page), page); @@ -994,6 +1000,9 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma)) return -EOPNOTSUPP; + if ((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA)) + return -EOPNOTSUPP; + if (vma_is_secretmem(vma)) return -EFAULT; @@ -2289,6 +2298,10 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) + goto pte_unmap; + folio = try_grab_folio(page, 1, flags); if (!folio) goto pte_unmap; @@ -2368,6 +2381,12 @@ static int __gup_device_huge(unsigned long pfn, unsigned long addr, undo_dev_pagemap(nr, nr_start, flags, pages); break; } + + if (!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { + undo_dev_pagemap(nr, nr_start, flags, pages); + break; + } + SetPageReferenced(page); pages[*nr] = page; if (unlikely(!try_grab_page(page, flags))) { @@ -2856,7 +2875,8 @@ static int internal_get_user_pages_fast(unsigned long start, if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN | FOLL_GET | - FOLL_FAST_ONLY | FOLL_NOFAULT))) + FOLL_FAST_ONLY | FOLL_NOFAULT | + FOLL_PCI_P2PDMA))) return -EINVAL; if (gup_flags & FOLL_PIN) -- 2.30.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2022-06-15 16:13 UTC|newest] Thread overview: 141+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-06-15 16:12 [PATCH v7 00/21] Userspace P2PDMA with O_DIRECT NVMe devices Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 01/21] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:33 ` Christoph Hellwig 2022-06-29 6:33 ` Christoph Hellwig 2022-06-29 9:05 ` Robin Murphy 2022-06-29 9:05 ` Robin Murphy 2022-06-29 15:39 ` Logan Gunthorpe 2022-06-29 15:39 ` Logan Gunthorpe 2022-06-29 18:02 ` Robin Murphy 2022-06-29 18:02 ` Robin Murphy 2022-06-29 18:24 ` Logan Gunthorpe 2022-06-29 18:24 ` Logan Gunthorpe 2022-07-04 15:08 ` Robin Murphy 2022-07-04 15:08 ` Robin Murphy 2022-06-15 16:12 ` [PATCH v7 02/21] PCI/P2PDMA: Attempt to set map_type if it has not been set Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:33 ` Christoph Hellwig 2022-06-29 6:33 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 03/21] PCI/P2PDMA: Expose pci_p2pdma_map_type() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:39 ` Christoph Hellwig 2022-06-29 6:39 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 04/21] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:39 ` Christoph Hellwig 2022-06-29 6:39 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 05/21] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:40 ` Christoph Hellwig 2022-06-29 6:40 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 06/21] dma-direct: support PCI P2PDMA pages in dma-direct map_sg Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:40 ` Christoph Hellwig 2022-06-29 6:40 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 07/21] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:41 ` Christoph Hellwig 2022-06-29 6:41 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 08/21] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 12:07 ` Robin Murphy 2022-06-29 12:07 ` Robin Murphy 2022-06-29 15:57 ` Logan Gunthorpe 2022-06-29 15:57 ` Logan Gunthorpe 2022-06-29 19:15 ` Robin Murphy 2022-06-29 19:15 ` Robin Murphy 2022-06-29 22:41 ` Logan Gunthorpe 2022-06-29 22:41 ` Logan Gunthorpe 2022-06-30 14:56 ` Robin Murphy 2022-06-30 14:56 ` Robin Murphy 2022-06-30 21:21 ` Logan Gunthorpe 2022-06-30 21:21 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 09/21] nvme-pci: check DMA ops when indicating support for PCI P2PDMA Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:41 ` Christoph Hellwig 2022-06-29 6:41 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 10/21] nvme-pci: convert to using dma_map_sgtable() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:42 ` Christoph Hellwig 2022-06-29 6:42 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 11/21] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:42 ` Christoph Hellwig 2022-06-29 6:42 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 12/21] RDMA/rw: drop pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:42 ` Christoph Hellwig 2022-06-29 6:42 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 13/21] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:43 ` Christoph Hellwig 2022-06-29 6:43 ` Christoph Hellwig 2022-06-15 16:12 ` Logan Gunthorpe [this message] 2022-06-15 16:12 ` [PATCH v7 14/21] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Logan Gunthorpe 2022-06-29 6:45 ` Christoph Hellwig 2022-06-29 6:45 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 15/21] iov_iter: introduce iov_iter_get_pages_[alloc_]flags() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:45 ` Christoph Hellwig 2022-06-29 6:45 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 16/21] block: add check when merging zone device pages Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:46 ` Christoph Hellwig 2022-06-29 6:46 ` Christoph Hellwig 2022-06-29 16:06 ` Logan Gunthorpe 2022-06-29 16:06 ` Logan Gunthorpe 2022-06-30 21:50 ` Logan Gunthorpe 2022-06-30 21:50 ` Logan Gunthorpe 2022-07-04 6:07 ` Christoph Hellwig 2022-07-04 6:07 ` Christoph Hellwig 2022-06-15 16:12 ` [PATCH v7 17/21] lib/scatterlist: " Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 18/21] block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 19/21] block: set FOLL_PCI_P2PDMA in bio_map_user_iov() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 20/21] PCI/P2PDMA: Introduce pci_mmap_p2pmem() Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:48 ` Christoph Hellwig 2022-06-29 6:48 ` Christoph Hellwig 2022-06-29 16:00 ` Logan Gunthorpe 2022-06-29 16:00 ` Logan Gunthorpe 2022-06-29 17:59 ` Jason Gunthorpe 2022-06-29 17:59 ` Jason Gunthorpe 2022-07-05 7:51 ` Christoph Hellwig 2022-07-05 7:51 ` Christoph Hellwig 2022-07-05 13:51 ` Jason Gunthorpe 2022-07-05 13:51 ` Jason Gunthorpe 2022-07-05 16:12 ` Christoph Hellwig 2022-07-05 16:12 ` Christoph Hellwig 2022-07-05 16:29 ` Jason Gunthorpe 2022-07-05 16:29 ` Jason Gunthorpe 2022-07-05 16:40 ` Christoph Hellwig 2022-07-05 16:40 ` Christoph Hellwig 2022-07-05 16:41 ` Logan Gunthorpe 2022-07-05 16:41 ` Logan Gunthorpe 2022-07-05 16:43 ` Christoph Hellwig 2022-07-05 16:43 ` Christoph Hellwig 2022-07-05 16:44 ` Logan Gunthorpe 2022-07-05 16:44 ` Logan Gunthorpe 2022-07-05 16:50 ` Christoph Hellwig 2022-07-05 16:50 ` Christoph Hellwig 2022-07-05 17:21 ` Greg Kroah-Hartman 2022-07-05 17:21 ` Greg Kroah-Hartman 2022-07-05 17:32 ` Logan Gunthorpe 2022-07-05 17:32 ` Logan Gunthorpe 2022-07-05 17:42 ` Greg Kroah-Hartman 2022-07-05 17:42 ` Greg Kroah-Hartman 2022-07-05 18:16 ` Logan Gunthorpe 2022-07-05 18:16 ` Logan Gunthorpe 2022-07-06 6:51 ` Christoph Hellwig 2022-07-06 6:51 ` Christoph Hellwig 2022-07-06 7:04 ` Greg Kroah-Hartman 2022-07-06 7:04 ` Greg Kroah-Hartman 2022-07-06 21:30 ` Logan Gunthorpe 2022-06-15 16:12 ` [PATCH v7 21/21] nvme-pci: allow mmaping the CMB in userspace Logan Gunthorpe 2022-06-15 16:12 ` Logan Gunthorpe 2022-06-29 6:49 ` [PATCH v7 00/21] Userspace P2PDMA with O_DIRECT NVMe devices Christoph Hellwig 2022-06-29 6:49 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220615161233.17527-15-logang@deltatee.com \ --to=logang@deltatee.com \ --cc=christian.koenig@amd.com \ --cc=ckulkarnilinux@gmail.com \ --cc=dan.j.williams@intel.com \ --cc=daniel.vetter@ffwll.ch \ --cc=dave.b.minturn@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=ddutile@redhat.com \ --cc=hch@lst.de \ --cc=helgaas@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=ira.weiny@intel.com \ --cc=jason@jlekstrand.net \ --cc=jgg@ziepe.ca \ --cc=jhubbard@nvidia.com \ --cc=jianxin.xiong@intel.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=martin.oliveira@eideticom.com \ --cc=rcampbell@nvidia.com \ --cc=robin.murphy@arm.com \ --cc=sbates@raithlin.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.