From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Stephen Bates" <sbates@raithlin.com>, "Christoph Hellwig" <hch@lst.de>, "Dan Williams" <dan.j.williams@intel.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Christian König" <christian.koenig@amd.com>, "John Hubbard" <jhubbard@nvidia.com>, "Don Dutile" <ddutile@redhat.com>, "Matthew Wilcox" <willy@infradead.org>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Minturn Dave B" <dave.b.minturn@intel.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Xiong Jianxin" <jianxin.xiong@intel.com>, "Bjorn Helgaas" <helgaas@kernel.org>, "Ira Weiny" <ira.weiny@intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Logan Gunthorpe" <logang@deltatee.com> Subject: [PATCH v4 15/23] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Date: Wed, 17 Nov 2021 14:54:02 -0700 [thread overview] Message-ID: <20211117215410.3695-16-logang@deltatee.com> (raw) In-Reply-To: <20211117215410.3695-1-logang@deltatee.com> Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to allow obtaining P2PDMA pages. If a caller does not set this flag and tries to map P2PDMA pages it will fail. This is implemented by checking failing if PCI p2pdma pages are found when FOLL_PCI_P2PDMA is set. This is only done if pte_devmap() is set. FOLL_PCI_P2PDMA cannot be set if FOLL_LONGTERM is set. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- include/linux/mm.h | 1 + mm/gup.c | 22 +++++++++++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a7e4a9e7d807..65cb27cebbab 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2971,6 +2971,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ #define FOLL_PIN 0x40000 /* pages must be released via unpin_user_page */ #define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ +#define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ /* * FOLL_PIN and FOLL_LONGTERM may be used in various combinations with each diff --git a/mm/gup.c b/mm/gup.c index 2c51e9748a6a..c31461c3d256 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -527,6 +527,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, page = pte_page(pte); else goto no_page; + + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) { + page = ERR_PTR(-EREMOTEIO); + goto out; + } } else if (unlikely(!page)) { if (flags & FOLL_DUMP) { /* Avoid special (like zero) pages in core dumps */ @@ -980,6 +986,9 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma)) return -EOPNOTSUPP; + if ((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA)) + return -EOPNOTSUPP; + if (vma_is_secretmem(vma)) return -EFAULT; @@ -2297,6 +2306,10 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); + if (unlikely(pte_devmap(pte) && !(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) + goto pte_unmap; + head = try_grab_compound_head(page, 1, flags); if (!head) goto pte_unmap; @@ -2374,6 +2387,12 @@ static int __gup_device_huge(unsigned long pfn, unsigned long addr, undo_dev_pagemap(nr, nr_start, flags, pages); break; } + + if (!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { + undo_dev_pagemap(nr, nr_start, flags, pages); + break; + } + SetPageReferenced(page); pages[*nr] = page; if (unlikely(!try_grab_page(page, flags))) { @@ -2842,7 +2861,8 @@ static int internal_get_user_pages_fast(unsigned long start, if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN | FOLL_GET | - FOLL_FAST_ONLY | FOLL_NOFAULT))) + FOLL_FAST_ONLY | FOLL_NOFAULT | + FOLL_PCI_P2PDMA))) return -EINVAL; if (gup_flags & FOLL_PIN) -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: "Minturn Dave B" <dave.b.minturn@intel.com>, "Martin Oliveira" <martin.oliveira@eideticom.com>, "Ira Weiny" <ira.weiny@intel.com>, "John Hubbard" <jhubbard@nvidia.com>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Robin Murphy" <robin.murphy@arm.com>, "Matthew Wilcox" <willy@infradead.org>, "Christian König" <christian.koenig@amd.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Logan Gunthorpe" <logang@deltatee.com>, "Chaitanya Kulkarni" <ckulkarnilinux@gmail.com>, "Jason Ekstrand" <jason@jlekstrand.net>, "Daniel Vetter" <daniel.vetter@ffwll.ch>, "Bjorn Helgaas" <helgaas@kernel.org>, "Dan Williams" <dan.j.williams@intel.com>, "Stephen Bates" <sbates@raithlin.com>, "Jakowski Andrzej" <andrzej.jakowski@intel.com>, "Christoph Hellwig" <hch@lst.de>, "Xiong Jianxin" <jianxin.xiong@intel.com> Subject: [PATCH v4 15/23] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Date: Wed, 17 Nov 2021 14:54:02 -0700 [thread overview] Message-ID: <20211117215410.3695-16-logang@deltatee.com> (raw) In-Reply-To: <20211117215410.3695-1-logang@deltatee.com> Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to allow obtaining P2PDMA pages. If a caller does not set this flag and tries to map P2PDMA pages it will fail. This is implemented by checking failing if PCI p2pdma pages are found when FOLL_PCI_P2PDMA is set. This is only done if pte_devmap() is set. FOLL_PCI_P2PDMA cannot be set if FOLL_LONGTERM is set. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- include/linux/mm.h | 1 + mm/gup.c | 22 +++++++++++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a7e4a9e7d807..65cb27cebbab 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2971,6 +2971,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ #define FOLL_PIN 0x40000 /* pages must be released via unpin_user_page */ #define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ +#define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ /* * FOLL_PIN and FOLL_LONGTERM may be used in various combinations with each diff --git a/mm/gup.c b/mm/gup.c index 2c51e9748a6a..c31461c3d256 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -527,6 +527,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, page = pte_page(pte); else goto no_page; + + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) { + page = ERR_PTR(-EREMOTEIO); + goto out; + } } else if (unlikely(!page)) { if (flags & FOLL_DUMP) { /* Avoid special (like zero) pages in core dumps */ @@ -980,6 +986,9 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma)) return -EOPNOTSUPP; + if ((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA)) + return -EOPNOTSUPP; + if (vma_is_secretmem(vma)) return -EFAULT; @@ -2297,6 +2306,10 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); + if (unlikely(pte_devmap(pte) && !(flags & FOLL_PCI_P2PDMA) && + is_pci_p2pdma_page(page))) + goto pte_unmap; + head = try_grab_compound_head(page, 1, flags); if (!head) goto pte_unmap; @@ -2374,6 +2387,12 @@ static int __gup_device_huge(unsigned long pfn, unsigned long addr, undo_dev_pagemap(nr, nr_start, flags, pages); break; } + + if (!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { + undo_dev_pagemap(nr, nr_start, flags, pages); + break; + } + SetPageReferenced(page); pages[*nr] = page; if (unlikely(!try_grab_page(page, flags))) { @@ -2842,7 +2861,8 @@ static int internal_get_user_pages_fast(unsigned long start, if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN | FOLL_GET | - FOLL_FAST_ONLY | FOLL_NOFAULT))) + FOLL_FAST_ONLY | FOLL_NOFAULT | + FOLL_PCI_P2PDMA))) return -EINVAL; if (gup_flags & FOLL_PIN) -- 2.30.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-11-17 21:54 UTC|newest] Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-11-17 21:53 [PATCH v4 00/23] Userspace P2PDMA with O_DIRECT NVMe devices Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 01/23] lib/scatterlist: cleanup macros into static inline functions Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 21:51 ` Chaitanya Kulkarni 2021-12-13 21:51 ` Chaitanya Kulkarni via iommu 2021-12-21 9:00 ` Christoph Hellwig 2021-12-21 9:00 ` Christoph Hellwig 2021-12-21 17:23 ` Logan Gunthorpe 2021-12-21 17:23 ` Logan Gunthorpe 2021-12-22 8:22 ` Christoph Hellwig 2021-12-22 8:22 ` Christoph Hellwig 2021-11-17 21:53 ` [PATCH v4 02/23] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 21:55 ` Chaitanya Kulkarni 2021-12-13 21:55 ` Chaitanya Kulkarni via iommu 2021-12-21 9:02 ` Christoph Hellwig 2021-12-21 9:02 ` Christoph Hellwig 2021-11-17 21:53 ` [PATCH v4 03/23] PCI/P2PDMA: Attempt to set map_type if it has not been set Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 22:00 ` Chaitanya Kulkarni 2021-12-13 22:00 ` Chaitanya Kulkarni via iommu 2021-11-17 21:53 ` [PATCH v4 04/23] PCI/P2PDMA: Expose pci_p2pdma_map_type() Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 22:05 ` Chaitanya Kulkarni 2021-12-13 22:05 ` Chaitanya Kulkarni via iommu 2021-11-17 21:53 ` [PATCH v4 05/23] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 06/23] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 07/23] dma-direct: support PCI P2PDMA pages in dma-direct map_sg Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 08/23] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 09/23] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 10/23] nvme-pci: check DMA ops when indicating support for PCI P2PDMA Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 22:10 ` Chaitanya Kulkarni 2021-12-13 22:10 ` Chaitanya Kulkarni via iommu 2021-11-17 21:53 ` [PATCH v4 11/23] nvme-pci: convert to using dma_map_sgtable() Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-12-13 22:21 ` Chaitanya Kulkarni 2021-12-13 22:21 ` Chaitanya Kulkarni via iommu 2021-12-13 22:28 ` Logan Gunthorpe 2021-12-13 22:28 ` Logan Gunthorpe 2021-11-17 21:53 ` [PATCH v4 12/23] RDMA/core: introduce ib_dma_pci_p2p_dma_supported() Logan Gunthorpe 2021-11-17 21:53 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 13/23] RDMA/rw: drop pci_p2pdma_[un]map_sg() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 14/23] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe [this message] 2021-11-17 21:54 ` [PATCH v4 15/23] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 16/23] iov_iter: introduce iov_iter_get_pages_[alloc_]flags() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-12-21 9:04 ` Christoph Hellwig 2021-12-21 9:04 ` Christoph Hellwig 2021-11-17 21:54 ` [PATCH v4 17/23] block: add check when merging zone device pages Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-12-21 9:05 ` Christoph Hellwig 2021-12-21 9:05 ` Christoph Hellwig 2021-11-17 21:54 ` [PATCH v4 18/23] lib/scatterlist: " Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 19/23] block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 20/23] block: set FOLL_PCI_P2PDMA in bio_map_user_iov() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 21/23] mm: use custom page_free for P2PDMA pages Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-12-21 9:06 ` Christoph Hellwig 2021-12-21 9:06 ` Christoph Hellwig 2021-12-21 17:27 ` Logan Gunthorpe 2021-12-21 17:27 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 22/23] PCI/P2PDMA: Introduce pci_mmap_p2pmem() Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-11-17 21:54 ` [PATCH v4 23/23] nvme-pci: allow mmaping the CMB in userspace Logan Gunthorpe 2021-11-17 21:54 ` Logan Gunthorpe 2021-12-21 9:07 ` Christoph Hellwig 2021-12-21 9:07 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211117215410.3695-16-logang@deltatee.com \ --to=logang@deltatee.com \ --cc=andrzej.jakowski@intel.com \ --cc=christian.koenig@amd.com \ --cc=ckulkarnilinux@gmail.com \ --cc=dan.j.williams@intel.com \ --cc=daniel.vetter@ffwll.ch \ --cc=dave.b.minturn@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=ddutile@redhat.com \ --cc=hch@lst.de \ --cc=helgaas@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=ira.weiny@intel.com \ --cc=jason@jlekstrand.net \ --cc=jgg@ziepe.ca \ --cc=jhubbard@nvidia.com \ --cc=jianxin.xiong@intel.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=martin.oliveira@eideticom.com \ --cc=robin.murphy@arm.com \ --cc=sbates@raithlin.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.