From: Peter Xu <peterx@redhat.com>
To: Mike Kravetz <mike.kravetz@oracle.com>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Mike Rapoport <rppt@linux.vnet.ibm.com>,
Andrea Arcangeli <aarcange@redhat.com>,
Axel Rasmussen <axelrasmussen@google.com>,
Matthew Wilcox <willy@infradead.org>,
"Kirill A . Shutemov" <kirill@shutemov.name>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH v2 4/4] hugetlb/userfaultfd: Unshare all pmds for hugetlbfs when register wp
Date: Thu, 18 Feb 2021 12:55:55 -0500 [thread overview]
Message-ID: <20210218175555.GC108961@xz-x1> (raw)
In-Reply-To: <f2122560-fbb9-c0fc-1b05-6d82ccf4525c@oracle.com>
On Wed, Feb 17, 2021 at 05:46:30PM -0800, Mike Kravetz wrote:
> On 2/17/21 12:46 PM, Peter Xu wrote:
> > Huge pmd sharing for hugetlbfs is racy with userfaultfd-wp because
> > userfaultfd-wp is always based on pgtable entries, so they cannot be shared.
> >
> > Walk the hugetlb range and unshare all such mappings if there is, right before
> > UFFDIO_REGISTER will succeed and return to userspace.
> >
> > This will pair with want_pmd_share() in hugetlb code so that huge pmd sharing
> > is completely disabled for userfaultfd-wp registered range.
> >
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > ---
> > fs/userfaultfd.c | 4 ++++
> > include/linux/hugetlb.h | 1 +
> > mm/hugetlb.c | 51 +++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 56 insertions(+)
> >
> > diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> > index 894cc28142e7..e259318fcae1 100644
> > --- a/fs/userfaultfd.c
> > +++ b/fs/userfaultfd.c
> > @@ -15,6 +15,7 @@
> > #include <linux/sched/signal.h>
> > #include <linux/sched/mm.h>
> > #include <linux/mm.h>
> > +#include <linux/mmu_notifier.h>
> > #include <linux/poll.h>
> > #include <linux/slab.h>
> > #include <linux/seq_file.h>
> > @@ -1448,6 +1449,9 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
> > vma->vm_flags = new_flags;
> > vma->vm_userfaultfd_ctx.ctx = ctx;
> >
> > + if (is_vm_hugetlb_page(vma) && uffd_disable_huge_pmd_share(vma))
> > + hugetlb_unshare_all_pmds(vma);
> > +
> > skip:
> > prev = vma;
> > start = vma->vm_end;
> > diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> > index 3b4104021dd3..97ecfd4c20b2 100644
> > --- a/include/linux/hugetlb.h
> > +++ b/include/linux/hugetlb.h
> > @@ -188,6 +188,7 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma,
> > unsigned long address, unsigned long end, pgprot_t newprot);
> >
> > bool is_hugetlb_entry_migration(pte_t pte);
> > +void hugetlb_unshare_all_pmds(struct vm_area_struct *vma);
> >
> > #else /* !CONFIG_HUGETLB_PAGE */
> >
> > diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> > index f53a0b852ed8..83c006ea3ff9 100644
> > --- a/mm/hugetlb.c
> > +++ b/mm/hugetlb.c
> > @@ -5723,4 +5723,55 @@ void __init hugetlb_cma_check(void)
> > pr_warn("hugetlb_cma: the option isn't supported by current arch\n");
> > }
> >
> > +/*
> > + * This function will unconditionally remove all the shared pmd pgtable entries
> > + * within the specific vma for a hugetlbfs memory range.
> > + */
>
> Thanks for updating this!
>
> > +void hugetlb_unshare_all_pmds(struct vm_area_struct *vma)
> > +{
> > + struct hstate *h = hstate_vma(vma);
> > + unsigned long sz = huge_page_size(h);
> > + struct mm_struct *mm = vma->vm_mm;
> > + struct mmu_notifier_range range;
> > + unsigned long address, start, end;
> > + spinlock_t *ptl;
> > + pte_t *ptep;
> > +
> > + if (!(vma->vm_flags & VM_MAYSHARE))
> > + return;
> > +
> > + start = ALIGN(vma->vm_start, PUD_SIZE);
> > + end = ALIGN_DOWN(vma->vm_end, PUD_SIZE);
> > +
> > + if (start >= end)
> > + return;
> > +
> > + /*
> > + * No need to call adjust_range_if_pmd_sharing_possible(), because
> > + * we're going to operate on the whole vma
>
> not necessary, but perhaps change to:
> * we're going to operate on ever PUD_SIZE aligned sized range
> * within the vma.
>
> > + * we're going to operate on the whole vma
> > + */
> > + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm,
> > + vma->vm_start, vma->vm_end);
>
> Should we use start, end here instead of vma->vm_start, vma->vm_end ?
>
> > + mmu_notifier_invalidate_range_start(&range);
> > + i_mmap_lock_write(vma->vm_file->f_mapping);
> > + for (address = start; address < end; address += PUD_SIZE) {
> > + unsigned long tmp = address;
> > +
> > + ptep = huge_pte_offset(mm, address, sz);
> > + if (!ptep)
> > + continue;
> > + ptl = huge_pte_lock(h, mm, ptep);
> > + /* We don't want 'address' to be changed */
> > + huge_pmd_unshare(mm, vma, &tmp, ptep);
> > + spin_unlock(ptl);
> > + }
> > + flush_hugetlb_tlb_range(vma, vma->vm_start, vma->vm_end);
>
> start, end ?
Right we can even shrink the notifier, I'll respin shortly. Thanks,
--
Peter Xu
next prev parent reply other threads:[~2021-02-18 17:56 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-17 20:44 [PATCH v2 0/4] hugetlb: Disable huge pmd unshare for uffd-wp Peter Xu
2021-02-17 20:44 ` [PATCH v2 1/4] hugetlb: Pass vma into huge_pte_alloc() and huge_pmd_share() Peter Xu
2021-02-17 20:46 ` [PATCH v2 2/4] hugetlb/userfaultfd: Forbid huge pmd sharing when uffd enabled Peter Xu
2021-02-17 20:46 ` [PATCH v2 3/4] mm/hugetlb: Move flush_hugetlb_tlb_range() into hugetlb.h Peter Xu
2021-02-17 20:46 ` [PATCH v2 4/4] hugetlb/userfaultfd: Unshare all pmds for hugetlbfs when register wp Peter Xu
2021-02-18 1:46 ` Mike Kravetz
2021-02-18 17:55 ` Peter Xu [this message]
2021-02-18 18:32 ` Axel Rasmussen
2021-02-18 20:32 ` Peter Xu
2021-02-18 20:34 ` Axel Rasmussen
2021-02-18 20:41 ` Peter Xu
2021-02-18 1:34 ` [PATCH v2 2/4] hugetlb/userfaultfd: Forbid huge pmd sharing when uffd enabled Mike Kravetz
2021-02-18 18:54 ` [PATCH v2 0/4] hugetlb: Disable huge pmd unshare for uffd-wp Axel Rasmussen
2021-02-18 20:33 ` Peter Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210218175555.GC108961@xz-x1 \
--to=peterx@redhat.com \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=axelrasmussen@google.com \
--cc=kirill@shutemov.name \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=rppt@linux.vnet.ibm.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).