From: Ralph Campbell <rcampbell@nvidia.com> To: <linux-rdma@vger.kernel.org>, <linux-mm@kvack.org>, <nouveau@lists.freedesktop.org>, <linux-kselftest@vger.kernel.org>, <linux-kernel@vger.kernel.org> Cc: Jerome Glisse <jglisse@redhat.com>, John Hubbard <jhubbard@nvidia.com>, Christoph Hellwig <hch@lst.de>, Jason Gunthorpe <jgg@mellanox.com>, "Andrew Morton" <akpm@linux-foundation.org>, Shuah Khan <shuah@kernel.org>, "Ben Skeggs" <bskeggs@redhat.com>, Ralph Campbell <rcampbell@nvidia.com> Subject: [PATCH v3 2/5] mm/hmm: add hmm_mapping order Date: Wed, 1 Jul 2020 15:53:49 -0700 [thread overview] Message-ID: <20200701225352.9649-3-rcampbell@nvidia.com> (raw) In-Reply-To: <20200701225352.9649-1-rcampbell@nvidia.com> hmm_range_fault() returns an array of page frame numbers and flags for how the pages are mapped in the requested process' page tables. The PFN can be used to get the struct page with hmm_pfn_to_page() and the page size order can be determined with compound_order(page) but if the page is larger than order 0 (PAGE_SIZE), there is no indication that a compound page is mapped by the CPU using a larger page size. Without this information, the caller can't safely use a large device PTE to map the compound page because the CPU might be using smaller PTEs with different read/write permissions. Add a new function hmm_pfn_to_map_order() to return the mapping size order so that callers know the pages are being mapped with consistent permissions and a large device page table mapping can be used if one is available. Signed-off-by: Ralph Campbell <rcampbell@nvidia.com> --- include/linux/hmm.h | 24 ++++++++++++++++++++++-- mm/hmm.c | 14 +++++++++++--- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index f4a09ed223ac..e7a21a21f11f 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -37,16 +37,17 @@ * will fail. Must be combined with HMM_PFN_REQ_FAULT. */ enum hmm_pfn_flags { - /* Output flags */ + /* Output fields and flags */ HMM_PFN_VALID = 1UL << (BITS_PER_LONG - 1), HMM_PFN_WRITE = 1UL << (BITS_PER_LONG - 2), HMM_PFN_ERROR = 1UL << (BITS_PER_LONG - 3), + HMM_PFN_ORDER_SHIFT = (BITS_PER_LONG - 8), /* Input flags */ HMM_PFN_REQ_FAULT = HMM_PFN_VALID, HMM_PFN_REQ_WRITE = HMM_PFN_WRITE, - HMM_PFN_FLAGS = HMM_PFN_VALID | HMM_PFN_WRITE | HMM_PFN_ERROR, + HMM_PFN_FLAGS = 0xFFUL << HMM_PFN_ORDER_SHIFT, }; /* @@ -61,6 +62,25 @@ static inline struct page *hmm_pfn_to_page(unsigned long hmm_pfn) return pfn_to_page(hmm_pfn & ~HMM_PFN_FLAGS); } +/* + * hmm_pfn_to_map_order() - return the CPU mapping size order + * + * The hmm_pfn entry returned by hmm_range_fault() is for a PAGE_SIZE + * address range. hmm_pfn_to_map_order() lets the caller know that the + * underlying physical page order is at least as large as the return value and + * that the CPU has mapped that physical range with the same permissions so + * that a device MMU mapping of up to the size of the return value can be + * used without giving the device more access than the CPU process. + * + * This must be called under the caller 'user_lock' after a successful + * mmu_interval_read_begin(). The caller must have tested for HMM_PFN_VALID + * already. + */ +static inline unsigned int hmm_pfn_to_map_order(unsigned long hmm_pfn) +{ + return (hmm_pfn >> HMM_PFN_ORDER_SHIFT) & 0x1F; +} + /* * struct hmm_range - track invalidation lock on virtual address range * diff --git a/mm/hmm.c b/mm/hmm.c index e9a545751108..de04bbed47b3 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -170,7 +170,10 @@ static inline unsigned long pmd_to_hmm_pfn_flags(struct hmm_range *range, { if (pmd_protnone(pmd)) return 0; - return pmd_write(pmd) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : HMM_PFN_VALID; + return ((unsigned long)(PMD_SHIFT - PAGE_SHIFT) << + HMM_PFN_ORDER_SHIFT) | + pmd_write(pmd) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : + HMM_PFN_VALID; } #ifdef CONFIG_TRANSPARENT_HUGEPAGE @@ -389,7 +392,10 @@ static inline unsigned long pud_to_hmm_pfn_flags(struct hmm_range *range, { if (!pud_present(pud)) return 0; - return pud_write(pud) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : HMM_PFN_VALID; + return ((unsigned long)(PUD_SHIFT - PAGE_SHIFT) << + HMM_PFN_ORDER_SHIFT) | + pud_write(pud) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : + HMM_PFN_VALID; } static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, @@ -468,13 +474,15 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, unsigned long cpu_flags; spinlock_t *ptl; pte_t entry; + unsigned long horder = huge_page_order(hstate_vma(vma)); ptl = huge_pte_lock(hstate_vma(vma), walk->mm, pte); entry = huge_ptep_get(pte); i = (start - range->start) >> PAGE_SHIFT; pfn_req_flags = range->hmm_pfns[i]; - cpu_flags = pte_to_hmm_pfn_flags(range, entry); + cpu_flags = pte_to_hmm_pfn_flags(range, entry) | + (horder << HMM_PFN_ORDER_SHIFT); required_fault = hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags); if (required_fault) { -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: Ralph Campbell <rcampbell-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> To: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-kselftest-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: Ralph Campbell <rcampbell-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>, Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>, Ben Skeggs <bskeggs-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Shuah Khan <shuah-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> Subject: [PATCH v3 2/5] mm/hmm: add hmm_mapping order Date: Wed, 1 Jul 2020 15:53:49 -0700 [thread overview] Message-ID: <20200701225352.9649-3-rcampbell@nvidia.com> (raw) In-Reply-To: <20200701225352.9649-1-rcampbell-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> hmm_range_fault() returns an array of page frame numbers and flags for how the pages are mapped in the requested process' page tables. The PFN can be used to get the struct page with hmm_pfn_to_page() and the page size order can be determined with compound_order(page) but if the page is larger than order 0 (PAGE_SIZE), there is no indication that a compound page is mapped by the CPU using a larger page size. Without this information, the caller can't safely use a large device PTE to map the compound page because the CPU might be using smaller PTEs with different read/write permissions. Add a new function hmm_pfn_to_map_order() to return the mapping size order so that callers know the pages are being mapped with consistent permissions and a large device page table mapping can be used if one is available. Signed-off-by: Ralph Campbell <rcampbell-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> --- include/linux/hmm.h | 24 ++++++++++++++++++++++-- mm/hmm.c | 14 +++++++++++--- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index f4a09ed223ac..e7a21a21f11f 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -37,16 +37,17 @@ * will fail. Must be combined with HMM_PFN_REQ_FAULT. */ enum hmm_pfn_flags { - /* Output flags */ + /* Output fields and flags */ HMM_PFN_VALID = 1UL << (BITS_PER_LONG - 1), HMM_PFN_WRITE = 1UL << (BITS_PER_LONG - 2), HMM_PFN_ERROR = 1UL << (BITS_PER_LONG - 3), + HMM_PFN_ORDER_SHIFT = (BITS_PER_LONG - 8), /* Input flags */ HMM_PFN_REQ_FAULT = HMM_PFN_VALID, HMM_PFN_REQ_WRITE = HMM_PFN_WRITE, - HMM_PFN_FLAGS = HMM_PFN_VALID | HMM_PFN_WRITE | HMM_PFN_ERROR, + HMM_PFN_FLAGS = 0xFFUL << HMM_PFN_ORDER_SHIFT, }; /* @@ -61,6 +62,25 @@ static inline struct page *hmm_pfn_to_page(unsigned long hmm_pfn) return pfn_to_page(hmm_pfn & ~HMM_PFN_FLAGS); } +/* + * hmm_pfn_to_map_order() - return the CPU mapping size order + * + * The hmm_pfn entry returned by hmm_range_fault() is for a PAGE_SIZE + * address range. hmm_pfn_to_map_order() lets the caller know that the + * underlying physical page order is at least as large as the return value and + * that the CPU has mapped that physical range with the same permissions so + * that a device MMU mapping of up to the size of the return value can be + * used without giving the device more access than the CPU process. + * + * This must be called under the caller 'user_lock' after a successful + * mmu_interval_read_begin(). The caller must have tested for HMM_PFN_VALID + * already. + */ +static inline unsigned int hmm_pfn_to_map_order(unsigned long hmm_pfn) +{ + return (hmm_pfn >> HMM_PFN_ORDER_SHIFT) & 0x1F; +} + /* * struct hmm_range - track invalidation lock on virtual address range * diff --git a/mm/hmm.c b/mm/hmm.c index e9a545751108..de04bbed47b3 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -170,7 +170,10 @@ static inline unsigned long pmd_to_hmm_pfn_flags(struct hmm_range *range, { if (pmd_protnone(pmd)) return 0; - return pmd_write(pmd) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : HMM_PFN_VALID; + return ((unsigned long)(PMD_SHIFT - PAGE_SHIFT) << + HMM_PFN_ORDER_SHIFT) | + pmd_write(pmd) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : + HMM_PFN_VALID; } #ifdef CONFIG_TRANSPARENT_HUGEPAGE @@ -389,7 +392,10 @@ static inline unsigned long pud_to_hmm_pfn_flags(struct hmm_range *range, { if (!pud_present(pud)) return 0; - return pud_write(pud) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : HMM_PFN_VALID; + return ((unsigned long)(PUD_SHIFT - PAGE_SHIFT) << + HMM_PFN_ORDER_SHIFT) | + pud_write(pud) ? (HMM_PFN_VALID | HMM_PFN_WRITE) : + HMM_PFN_VALID; } static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, @@ -468,13 +474,15 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, unsigned long cpu_flags; spinlock_t *ptl; pte_t entry; + unsigned long horder = huge_page_order(hstate_vma(vma)); ptl = huge_pte_lock(hstate_vma(vma), walk->mm, pte); entry = huge_ptep_get(pte); i = (start - range->start) >> PAGE_SHIFT; pfn_req_flags = range->hmm_pfns[i]; - cpu_flags = pte_to_hmm_pfn_flags(range, entry); + cpu_flags = pte_to_hmm_pfn_flags(range, entry) | + (horder << HMM_PFN_ORDER_SHIFT); required_fault = hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags); if (required_fault) { -- 2.20.1
next prev parent reply other threads:[~2020-07-01 22:54 UTC|newest] Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-07-01 22:53 [PATCH v3 0/5] mm/hmm/nouveau: add PMD system memory mapping Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell 2020-07-01 22:53 ` [PATCH v3 1/5] nouveau/hmm: fault one page at a time Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell [this message] 2020-07-01 22:53 ` [PATCH v3 2/5] mm/hmm: add hmm_mapping order Ralph Campbell 2020-07-01 22:53 ` [PATCH v3 3/5] nouveau: fix mapping 2MB sysmem pages Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell 2020-07-08 3:19 ` [Nouveau] " Ben Skeggs 2020-07-08 3:19 ` Ben Skeggs 2020-07-08 3:19 ` Ben Skeggs 2020-07-01 22:53 ` [PATCH v3 4/5] nouveau/hmm: support mapping large " Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell 2020-07-01 22:53 ` [PATCH v3 5/5] hmm: add tests for hmm_pfn_to_map_order() Ralph Campbell 2020-07-01 22:53 ` Ralph Campbell 2020-07-10 19:27 ` [PATCH v3 0/5] mm/hmm/nouveau: add PMD system memory mapping Jason Gunthorpe 2020-07-10 19:27 ` Jason Gunthorpe 2020-07-10 20:13 ` Ralph Campbell 2020-07-10 20:13 ` Ralph Campbell
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200701225352.9649-3-rcampbell@nvidia.com \ --to=rcampbell@nvidia.com \ --cc=akpm@linux-foundation.org \ --cc=bskeggs@redhat.com \ --cc=hch@lst.de \ --cc=jgg@mellanox.com \ --cc=jglisse@redhat.com \ --cc=jhubbard@nvidia.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-rdma@vger.kernel.org \ --cc=nouveau@lists.freedesktop.org \ --cc=shuah@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.