From: Jason Gunthorpe <jgg@ziepe.ca>
To: Peter Xu <peterx@redhat.com>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Linus Torvalds <torvalds@linux-foundation.org>,
Michal Hocko <mhocko@suse.com>,
Kirill Shutemov <kirill@shutemov.name>,
Jann Horn <jannh@google.com>, Oleg Nesterov <oleg@redhat.com>,
Kirill Tkhai <ktkhai@virtuozzo.com>,
Hugh Dickins <hughd@google.com>,
Leon Romanovsky <leonro@nvidia.com>, Jan Kara <jack@suse.cz>,
John Hubbard <jhubbard@nvidia.com>,
Christoph Hellwig <hch@lst.de>,
Andrew Morton <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>
Subject: Re: [PATCH 5/5] mm/thp: Split huge pmds/puds if they're pinned when fork()
Date: Wed, 23 Sep 2020 14:17:56 -0300 [thread overview]
Message-ID: <20200923171756.GC9916@ziepe.ca> (raw)
In-Reply-To: <20200923152409.GC59978@xz-x1>
On Wed, Sep 23, 2020 at 11:24:09AM -0400, Peter Xu wrote:
> On Tue, Sep 22, 2020 at 09:05:05AM -0300, Jason Gunthorpe wrote:
> > On Mon, Sep 21, 2020 at 05:20:31PM -0400, Peter Xu wrote:
> > > Pinned pages shouldn't be write-protected when fork() happens, because follow
> > > up copy-on-write on these pages could cause the pinned pages to be replaced by
> > > random newly allocated pages.
> > >
> > > For huge PMDs, we split the huge pmd if pinning is detected. So that future
> > > handling will be done by the PTE level (with our latest changes, each of the
> > > small pages will be copied). We can achieve this by let copy_huge_pmd() return
> > > -EAGAIN for pinned pages, so that we'll fallthrough in copy_pmd_range() and
> > > finally land the next copy_pte_range() call.
> > >
> > > Huge PUDs will be even more special - so far it does not support anonymous
> > > pages. But it can actually be done the same as the huge PMDs even if the split
> > > huge PUDs means to erase the PUD entries. It'll guarantee the follow up fault
> > > ins will remap the same pages in either parent/child later.
> > >
> > > This might not be the most efficient way, but it should be easy and clean
> > > enough. It should be fine, since we're tackling with a very rare case just to
> > > make sure userspaces that pinned some thps will still work even without
> > > MADV_DONTFORK and after they fork()ed.
> > >
> > > Signed-off-by: Peter Xu <peterx@redhat.com>
> > > mm/huge_memory.c | 26 ++++++++++++++++++++++++++
> > > 1 file changed, 26 insertions(+)
> > >
> > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > > index 7ff29cc3d55c..c40aac0ad87e 100644
> > > +++ b/mm/huge_memory.c
> > > @@ -1074,6 +1074,23 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
> > >
> > > src_page = pmd_page(pmd);
> > > VM_BUG_ON_PAGE(!PageHead(src_page), src_page);
> > > +
> > > + /*
> > > + * If this page is a potentially pinned page, split and retry the fault
> > > + * with smaller page size. Normally this should not happen because the
> > > + * userspace should use MADV_DONTFORK upon pinned regions. This is a
> > > + * best effort that the pinned pages won't be replaced by another
> > > + * random page during the coming copy-on-write.
> > > + */
> > > + if (unlikely(READ_ONCE(src_mm->has_pinned) &&
> > > + page_maybe_dma_pinned(src_page))) {
> > > + pte_free(dst_mm, pgtable);
> > > + spin_unlock(src_ptl);
> > > + spin_unlock(dst_ptl);
> > > + __split_huge_pmd(vma, src_pmd, addr, false, NULL);
> > > + return -EAGAIN;
> > > + }
> >
> > Not sure why, but the PMD stuff here is not calling is_cow_mapping()
> > before doing the write protect. Seems like it might be an existing
> > bug?
>
> IMHO it's not a bug, because splitting a huge pmd should always be safe.
Sur splitting is safe, but testing has_pinned without checking COW is
not, for what Jann explained.
The 'maybe' in page_maybe_dma_pinned() means it can return true when
the correct answer is false. It can never return false when the
correct answer is true.
It is the same when has_pinned is involved, the combined expression
must never return false when true is correct. Which means it can only
be applied for COW cases.
Jason
next prev parent reply other threads:[~2020-09-23 17:18 UTC|newest]
Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-21 21:17 [PATCH 0/5] mm: Break COW for pinned pages during fork() Peter Xu
2020-09-21 21:17 ` [PATCH 1/5] mm: Introduce mm_struct.has_pinned Peter Xu
2020-09-21 21:43 ` Jann Horn
2020-09-21 22:30 ` Peter Xu
2020-09-21 22:47 ` Jann Horn
2020-09-22 11:54 ` Jason Gunthorpe
2020-09-22 14:28 ` Peter Xu
2020-09-22 15:56 ` Jason Gunthorpe
2020-09-22 16:25 ` Linus Torvalds
2020-09-21 23:53 ` John Hubbard
2020-09-22 0:01 ` John Hubbard
2020-09-22 15:17 ` Peter Xu
2020-09-22 16:10 ` Jason Gunthorpe
2020-09-22 17:54 ` Peter Xu
2020-09-22 19:11 ` Jason Gunthorpe
2020-09-23 0:27 ` Peter Xu
2020-09-23 13:10 ` Peter Xu
2020-09-23 14:20 ` Jan Kara
2020-09-23 17:12 ` Jason Gunthorpe
2020-09-24 7:44 ` Jan Kara
2020-09-24 14:02 ` Jason Gunthorpe
2020-09-24 14:45 ` Jan Kara
2020-09-23 17:07 ` Jason Gunthorpe
2020-09-24 14:35 ` Peter Xu
2020-09-24 16:51 ` Jason Gunthorpe
2020-09-24 17:55 ` Peter Xu
2020-09-24 18:15 ` Jason Gunthorpe
2020-09-24 18:34 ` Peter Xu
2020-09-24 18:39 ` Jason Gunthorpe
2020-09-24 21:30 ` Peter Xu
2020-09-25 19:56 ` Linus Torvalds
2020-09-25 21:06 ` Linus Torvalds
2020-09-26 0:41 ` Jason Gunthorpe
2020-09-26 1:15 ` Linus Torvalds
2020-09-26 22:28 ` Linus Torvalds
2020-09-27 6:23 ` Leon Romanovsky
2020-09-27 18:16 ` Linus Torvalds
2020-09-27 18:45 ` Linus Torvalds
2020-09-28 12:49 ` Jason Gunthorpe
2020-09-28 16:17 ` Linus Torvalds
2020-09-28 17:22 ` Peter Xu
2020-09-28 17:54 ` Linus Torvalds
2020-09-28 18:39 ` Jason Gunthorpe
2020-09-28 19:29 ` Linus Torvalds
2020-09-28 23:57 ` Jason Gunthorpe
2020-09-29 0:18 ` John Hubbard
2020-09-28 19:36 ` Linus Torvalds
2020-09-28 19:50 ` Linus Torvalds
2020-09-28 22:51 ` Jason Gunthorpe
2020-09-29 0:30 ` Peter Xu
2020-10-08 5:49 ` Leon Romanovsky
2020-09-28 17:13 ` Peter Xu
2020-09-25 21:13 ` Peter Xu
2020-09-25 22:08 ` Linus Torvalds
2020-09-22 18:02 ` John Hubbard
2020-09-22 18:15 ` Peter Xu
2020-09-22 19:11 ` John Hubbard
2020-09-27 0:41 ` [mm] 698ac7610f: will-it-scale.per_thread_ops 8.2% improvement kernel test robot
2020-09-21 21:17 ` [PATCH 2/5] mm/fork: Pass new vma pointer into copy_page_range() Peter Xu
2020-09-21 21:17 ` [PATCH 3/5] mm: Rework return value for copy_one_pte() Peter Xu
2020-09-22 7:11 ` John Hubbard
2020-09-22 15:29 ` Peter Xu
2020-09-22 10:08 ` Oleg Nesterov
2020-09-22 10:18 ` Oleg Nesterov
2020-09-22 15:36 ` Peter Xu
2020-09-22 15:48 ` Oleg Nesterov
2020-09-22 16:03 ` Peter Xu
2020-09-22 16:53 ` Oleg Nesterov
2020-09-22 18:13 ` Peter Xu
2020-09-22 18:23 ` Oleg Nesterov
2020-09-22 18:49 ` Peter Xu
2020-09-23 6:52 ` Oleg Nesterov
2020-09-23 17:16 ` Linus Torvalds
2020-09-23 21:24 ` Linus Torvalds
2020-09-21 21:20 ` [PATCH 4/5] mm: Do early cow for pinned pages during fork() for ptes Peter Xu
2020-09-21 21:55 ` Jann Horn
2020-09-21 22:18 ` John Hubbard
2020-09-21 22:27 ` Jann Horn
2020-09-22 0:08 ` John Hubbard
2020-09-21 22:27 ` Peter Xu
2020-09-22 11:48 ` Oleg Nesterov
2020-09-22 12:40 ` Oleg Nesterov
2020-09-22 15:58 ` Peter Xu
2020-09-22 16:52 ` Oleg Nesterov
2020-09-22 18:34 ` Peter Xu
2020-09-22 18:44 ` Oleg Nesterov
2020-09-23 1:03 ` Peter Xu
2020-09-23 20:25 ` Linus Torvalds
2020-09-24 15:08 ` Peter Xu
2020-09-24 11:48 ` Kirill Tkhai
2020-09-24 15:16 ` Peter Xu
2020-09-21 21:20 ` [PATCH 5/5] mm/thp: Split huge pmds/puds if they're pinned when fork() Peter Xu
2020-09-22 6:41 ` John Hubbard
2020-09-22 10:33 ` Jan Kara
2020-09-22 20:01 ` John Hubbard
2020-09-23 9:22 ` Jan Kara
2020-09-23 13:50 ` Peter Xu
2020-09-23 14:01 ` Jan Kara
2020-09-23 15:44 ` Peter Xu
2020-09-23 20:19 ` John Hubbard
2020-09-24 18:49 ` Peter Xu
2020-09-23 16:06 ` Peter Xu
2020-09-22 12:05 ` Jason Gunthorpe
2020-09-23 15:24 ` Peter Xu
2020-09-23 16:07 ` Yang Shi
2020-09-24 15:47 ` Peter Xu
2020-09-24 17:29 ` Yang Shi
2020-09-23 17:17 ` Jason Gunthorpe [this message]
2020-09-23 10:21 ` [PATCH 0/5] mm: Break COW for pinned pages during fork() Leon Romanovsky
2020-09-23 15:37 ` Peter Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200923171756.GC9916@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=hch@lst.de \
--cc=hughd@google.com \
--cc=jack@suse.cz \
--cc=jannh@google.com \
--cc=jhubbard@nvidia.com \
--cc=kirill@shutemov.name \
--cc=ktkhai@virtuozzo.com \
--cc=leonro@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=oleg@redhat.com \
--cc=peterx@redhat.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).