From: Joao Martins <joao.m.martins@oracle.com>
To: linux-nvdimm@lists.01.org
Cc: Dan Williams <dan.j.williams@intel.com>,
Vishal Verma <vishal.l.verma@intel.com>,
Dave Jiang <dave.jiang@intel.com>,
Ira Weiny <ira.weiny@intel.com>,
Alex Williamson <alex.williamson@redhat.com>,
Cornelia Huck <cohuck@redhat.com>,
kvm@vger.kernel.org, Andrew Morton <akpm@linux-foundation.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
"H . Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, Liran Alon <liran.alon@oracle.com>,
Nikita Leshenko <nikita.leshchenko@oracle.com>,
Barret Rhoden <brho@google.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
Matthew Wilcox <willy@infradead.org>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: [PATCH RFC 04/10] mm: Handle pud entries in follow_pfn()
Date: Fri, 10 Jan 2020 19:03:07 +0000 [thread overview]
Message-ID: <20200110190313.17144-5-joao.m.martins@oracle.com> (raw)
In-Reply-To: <20200110190313.17144-1-joao.m.martins@oracle.com>
When follow_pfn hits a pud_huge() it won't return a valid PFN
for a PUD. Fix it by adding @pudp and thus allow callers to
get the pud pointer. If we encounter such a huge page, we
calculate the offset to the PUD accordingly.
This allows KVM to handle 1G hugepage pfns on VM_PFNMAP vmas.
Co-developed-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Signed-off-by: Nikita Leshenko <nikita.leshchenko@oracle.com>
Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
---
mm/memory.c | 58 ++++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 49 insertions(+), 9 deletions(-)
diff --git a/mm/memory.c b/mm/memory.c
index 109643219e1b..f46646630497 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4261,9 +4261,10 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
}
#endif /* __PAGETABLE_PMD_FOLDED */
-static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address,
+static int __follow_pte_pud(struct mm_struct *mm, unsigned long address,
struct mmu_notifier_range *range,
- pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp)
+ pte_t **ptepp, pmd_t **pmdpp, pud_t **pudpp,
+ spinlock_t **ptlp)
{
pgd_t *pgd;
p4d_t *p4d;
@@ -4280,6 +4281,28 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address,
goto out;
pud = pud_offset(p4d, address);
+ VM_BUG_ON(pud_trans_huge(*pud));
+
+ if (pud_huge(*pud)) {
+ if (!pudpp)
+ goto out;
+
+ if (range) {
+ mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0,
+ NULL, mm, address & PUD_MASK,
+ (address & PUD_MASK) + PUD_SIZE);
+ mmu_notifier_invalidate_range_start(range);
+ }
+ *ptlp = pud_lock(mm, pud);
+ if (pud_huge(*pud)) {
+ *pudpp = pud;
+ return 0;
+ }
+ spin_unlock(*ptlp);
+ if (range)
+ mmu_notifier_invalidate_range_end(range);
+ }
+
if (pud_none(*pud) || unlikely(pud_bad(*pud)))
goto out;
@@ -4335,8 +4358,8 @@ static inline int follow_pte(struct mm_struct *mm, unsigned long address,
/* (void) is needed to make gcc happy */
(void) __cond_lock(*ptlp,
- !(res = __follow_pte_pmd(mm, address, NULL,
- ptepp, NULL, ptlp)));
+ !(res = __follow_pte_pud(mm, address, NULL,
+ ptepp, NULL, NULL, ptlp)));
return res;
}
@@ -4348,12 +4371,26 @@ int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
/* (void) is needed to make gcc happy */
(void) __cond_lock(*ptlp,
- !(res = __follow_pte_pmd(mm, address, range,
- ptepp, pmdpp, ptlp)));
+ !(res = __follow_pte_pud(mm, address, range,
+ ptepp, pmdpp, NULL, ptlp)));
return res;
}
EXPORT_SYMBOL(follow_pte_pmd);
+static int follow_pte_pud(struct mm_struct *mm, unsigned long address,
+ struct mmu_notifier_range *range,
+ pte_t **ptepp, pmd_t **pmdpp, pud_t **pudpp,
+ spinlock_t **ptlp)
+{
+ int res;
+
+ /* (void) is needed to make gcc happy */
+ (void) __cond_lock(*ptlp,
+ !(res = __follow_pte_pud(mm, address, range,
+ ptepp, pmdpp, pudpp, ptlp)));
+ return res;
+}
+
/**
* follow_pfn - look up PFN at a user virtual address
* @vma: memory mapping
@@ -4368,6 +4405,7 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address,
unsigned long *pfn)
{
pmd_t *pmdpp = NULL;
+ pud_t *pudpp = NULL;
int ret = -EINVAL;
spinlock_t *ptl;
pte_t *ptep;
@@ -4375,11 +4413,13 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address,
if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
return ret;
- ret = follow_pte_pmd(vma->vm_mm, address, NULL,
- &ptep, &pmdpp, &ptl);
+ ret = follow_pte_pud(vma->vm_mm, address, NULL,
+ &ptep, &pmdpp, &pudpp, &ptl);
if (ret)
return ret;
- if (pmdpp)
+ if (pudpp)
+ *pfn = pud_pfn(*pudpp) + ((address & ~PUD_MASK) >> PAGE_SHIFT);
+ else if (pmdpp)
*pfn = pmd_pfn(*pmdpp) + ((address & ~PMD_MASK) >> PAGE_SHIFT);
else
*pfn = pte_pfn(*ptep);
--
2.17.1
next prev parent reply other threads:[~2020-01-10 19:07 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-10 19:03 [PATCH RFC 00/10] device-dax: Support devices without PFN metadata Joao Martins
2020-01-10 19:03 ` [PATCH RFC 01/10] mm: Add pmd support for _PAGE_SPECIAL Joao Martins
2020-02-03 21:34 ` Matthew Wilcox
2020-02-04 16:14 ` Joao Martins
2020-01-10 19:03 ` [PATCH RFC 02/10] mm: Handle pmd entries in follow_pfn() Joao Martins
2020-02-03 21:37 ` Matthew Wilcox
2020-02-04 16:17 ` Joao Martins
2020-01-10 19:03 ` [PATCH RFC 03/10] mm: Add pud support for _PAGE_SPECIAL Joao Martins
2020-01-10 19:03 ` Joao Martins [this message]
2020-01-10 19:03 ` [PATCH RFC 05/10] device-dax: Do not enforce MADV_DONTFORK on mmap() Joao Martins
2020-01-10 19:03 ` [PATCH RFC 06/10] device-dax: Introduce pfn_flags helper Joao Martins
2020-01-10 19:03 ` [PATCH RFC 07/10] device-dax: Add support for PFN_SPECIAL flags Joao Martins
2020-01-10 19:03 ` [PATCH RFC 08/10] dax/pmem: Add device-dax support for PFN_MODE_NONE Joao Martins
2020-01-10 19:03 ` [PATCH RFC 09/10] vfio/type1: Use follow_pfn for VM_FPNMAP VMAs Joao Martins
2020-02-07 21:08 ` Jason Gunthorpe
2020-02-11 16:23 ` Joao Martins
2020-02-11 16:50 ` Jason Gunthorpe
2020-01-10 19:03 ` [PATCH RFC 10/10] nvdimm/e820: add multiple namespaces support Joao Martins
2020-02-04 15:28 ` Barret Rhoden
2020-02-04 16:44 ` Dan Williams
2020-02-04 18:20 ` Barret Rhoden
2020-02-04 19:24 ` Joao Martins
2020-02-04 21:43 ` Dan Williams
2020-02-04 21:57 ` Barret Rhoden
2020-02-04 1:24 ` [PATCH RFC 00/10] device-dax: Support devices without PFN metadata Dan Williams
2020-02-04 19:07 ` Joao Martins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200110190313.17144-5-joao.m.martins@oracle.com \
--to=joao.m.martins@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=alex.williamson@redhat.com \
--cc=boris.ostrovsky@oracle.com \
--cc=bp@alien8.de \
--cc=brho@google.com \
--cc=cohuck@redhat.com \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=hpa@zytor.com \
--cc=ira.weiny@intel.com \
--cc=konrad.wilk@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=liran.alon@oracle.com \
--cc=mingo@redhat.com \
--cc=nikita.leshchenko@oracle.com \
--cc=tglx@linutronix.de \
--cc=vishal.l.verma@intel.com \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).