From: Andrea Arcangeli <aarcange@redhat.com>
To: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>
Cc: Michael Rapoport <RAPOPORT@il.ibm.com>,
"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
Mike Kravetz <mike.kravetz@oracle.com>,
Pavel Emelyanov <xemul@parallels.com>,
Hillf Danton <hillf.zj@alibaba-inc.com>
Subject: [PATCH 00/42] userfaultfd tmpfs/hugetlbfs/non-cooperative v2
Date: Fri, 16 Dec 2016 15:47:39 +0100 [thread overview]
Message-ID: <20161216144821.5183-1-aarcange@redhat.com> (raw)
Hello,
these userfaultfd features are finished and are ready for larger
exposure in -mm and upstream merging.
1) tmpfs non present userfault
2) hugetlbfs non present userfault
3) non cooperative userfault for fork/madvise/mremap
qemu development code is already exercising 2) and container postcopy
live migration needs 3).
1) is not currently used but there's a self test and we know some qemu
user for various reasons uses tmpfs as backing for KVM so it'll need
it too to use postcopy live migration with tmpfs memory.
All review feedback from the previous submit has been handled and the
fixes are included. There's no outstanding issue AFIK.
Upstream code just did a s/fe/vmf/ conversion in the page faults and
this has been converted as well incrementally.
In addition to the previous submits, this also wakes up stuck
userfaults during UFFDIO_UNREGISTER. The non cooperative testcase
actually reproduced this problem by getting stuck instead of quitting
clean in some rare case as it could call UFFDIO_UNREGISTER while some
userfault could be still in flight. The other option would have been
to keep leaving it up to userland to serialize itself and to patch the
testcase instead but the wakeup during unregister I think is
preferable.
David also asked the UFFD_FEATURE_MISSING_HUGETLBFS and
UFFD_FEATURE_MISSING_SHMEM feature flags to be added so QEMU can avoid
to probe if the hugetlbfs/shmem missing support is available by
calling UFFDIO_REGISTER. QEMU already checks HUGETLBFS_MAGIC with
fstatfs so if UFFD_FEATURE_MISSING_HUGETLBFS is also set, it knows
UFFDIO_REGISTER will succeed (or if it fails, it's for some other more
concerning reason). There's no reason to worry about adding too many
feature flags. There are 64 available and worst case we've to bump the
API if someday we're really going to run out of them.
The round-trip network latency of hugetlbfs userfaults during postcopy
live migration is still of the order of dozen milliseconds on 10GBit
if at 2MB hugepage granularity so it's working perfectly and it should
provide for higher bandwidth or lower CPU usage (which makes it
interesting to add an option in the future to support THP granularity
too for anonymous memory, UFFDIO_COPY would then have to create THP if
alignment/len allows for it). 1GB hugetlbfs granularity will require
big changes in hugetlbfs to work so it's deferred for later.
Andrea Arcangeli (17):
userfaultfd: document _IOR/_IOW
userfaultfd: correct comment about UFFD_FEATURE_PAGEFAULT_FLAG_WP
userfaultfd: convert BUG() to WARN_ON_ONCE()
userfaultfd: use vma_is_anonymous
userfaultfd: non-cooperative: report all available features to
userland
userfaultfd: non-cooperative: Add fork() event, build warning fix
userfaultfd: non-cooperative: optimize mremap_userfaultfd_complete()
userfaultfd: non-cooperative: avoid MADV_DONTNEED race condition
userfaultfd: non-cooperative: wake userfaults after UFFDIO_UNREGISTER
userfaultfd: hugetlbfs: gup: support VM_FAULT_RETRY
userfaultfd: hugetlbfs: UFFD_FEATURE_MISSING_HUGETLBFS
userfaultfd: shmem: add tlbflush.h header for microblaze
userfaultfd: shmem: lock the page before adding it to pagecache
userfaultfd: shmem: avoid leaking blocks and used blocks in
UFFDIO_COPY
userfaultfd: hugetlbfs: UFFD_FEATURE_MISSING_SHMEM
userfaultfd: selftest: test UFFDIO_ZEROPAGE on all memory types
mm: mprotect: use pmd_trans_unstable instead of taking the pmd_lock
Mike Kravetz (9):
userfaultfd: hugetlbfs: add copy_huge_page_from_user for hugetlb
userfaultfd support
userfaultfd: hugetlbfs: add hugetlb_mcopy_atomic_pte for userfaultfd
support
userfaultfd: hugetlbfs: add __mcopy_atomic_hugetlb for huge page
UFFDIO_COPY
userfaultfd: hugetlbfs: fix __mcopy_atomic_hugetlb retry/error
processing
userfaultfd: hugetlbfs: add userfaultfd hugetlb hook
userfaultfd: hugetlbfs: allow registration of ranges containing huge
pages
userfaultfd: hugetlbfs: add userfaultfd_hugetlb test
userfaultfd: hugetlbfs: userfaultfd_huge_must_wait for hugepmd ranges
userfaultfd: hugetlbfs: reserve count on error in
__mcopy_atomic_hugetlb
Mike Rapoport (11):
userfaultfd: non-cooperative: dup_userfaultfd: use mm_count instead of
mm_users
userfaultfd: introduce vma_can_userfault
userfaultfd: shmem: add shmem_mcopy_atomic_pte for userfaultfd support
userfaultfd: shmem: introduce vma_is_shmem
userfaultfd: shmem: use shmem_mcopy_atomic_pte for shared memory
userfaultfd: shmem: add userfaultfd hook for shared memory faults
userfaultfd: shmem: allow registration of shared memory ranges
userfaultfd: shmem: add userfaultfd_shmem test
userfaultfd: non-cooperative: selftest: introduce userfaultfd_open
userfaultfd: non-cooperative: selftest: add ufd parameter to copy_page
userfaultfd: non-cooperative: selftest: add test for FORK,
MADVDONTNEED and REMAP events
Pavel Emelyanov (5):
userfaultfd: non-cooperative: Split the find_userfault() routine
userfaultfd: non-cooperative: Add ability to report non-PF events from
uffd descriptor
userfaultfd: non-cooperative: Add fork() event
userfaultfd: non-cooperative: Add mremap() event
userfaultfd: non-cooperative: Add madvise() event for MADV_DONTNEED
request
fs/userfaultfd.c | 461 +++++++++++++++++++++++++++--
include/linux/hugetlb.h | 12 +-
include/linux/mm.h | 14 +
include/linux/shmem_fs.h | 11 +
include/linux/userfaultfd_k.h | 42 +++
include/uapi/asm-generic/ioctl.h | 10 +-
include/uapi/linux/userfaultfd.h | 67 ++++-
kernel/fork.c | 10 +-
mm/gup.c | 2 +-
mm/hugetlb.c | 162 ++++++++++-
mm/madvise.c | 2 +
mm/memory.c | 32 ++
mm/mprotect.c | 44 +--
mm/mremap.c | 17 +-
mm/shmem.c | 147 +++++++++-
mm/userfaultfd.c | 235 ++++++++++++++-
tools/testing/selftests/vm/Makefile | 8 +
tools/testing/selftests/vm/run_vmtests | 24 ++
tools/testing/selftests/vm/userfaultfd.c | 486 ++++++++++++++++++++++++++++---
19 files changed, 1641 insertions(+), 145 deletions(-)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2016-12-16 14:48 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-16 14:47 Andrea Arcangeli [this message]
2016-12-16 14:47 ` [PATCH 01/42] userfaultfd: document _IOR/_IOW Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 02/42] userfaultfd: correct comment about UFFD_FEATURE_PAGEFAULT_FLAG_WP Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 03/42] userfaultfd: convert BUG() to WARN_ON_ONCE() Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 04/42] userfaultfd: use vma_is_anonymous Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 05/42] userfaultfd: non-cooperative: Split the find_userfault() routine Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 06/42] userfaultfd: non-cooperative: Add ability to report non-PF events from uffd descriptor Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 07/42] userfaultfd: non-cooperative: report all available features to userland Andrea Arcangeli
2017-01-04 23:01 ` Andrew Morton
2017-01-05 12:56 ` Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 08/42] userfaultfd: non-cooperative: Add fork() event Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 09/42] userfaultfd: non-cooperative: Add fork() event, build warning fix Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 10/42] userfaultfd: non-cooperative: dup_userfaultfd: use mm_count instead of mm_users Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 11/42] userfaultfd: non-cooperative: Add mremap() event Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 12/42] userfaultfd: non-cooperative: optimize mremap_userfaultfd_complete() Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 13/42] userfaultfd: non-cooperative: Add madvise() event for MADV_DONTNEED request Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 14/42] userfaultfd: non-cooperative: avoid MADV_DONTNEED race condition Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 15/42] userfaultfd: non-cooperative: wake userfaults after UFFDIO_UNREGISTER Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 16/42] userfaultfd: hugetlbfs: add copy_huge_page_from_user for hugetlb userfaultfd support Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 17/42] userfaultfd: hugetlbfs: add hugetlb_mcopy_atomic_pte for " Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 18/42] userfaultfd: hugetlbfs: add __mcopy_atomic_hugetlb for huge page UFFDIO_COPY Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 19/42] userfaultfd: hugetlbfs: fix __mcopy_atomic_hugetlb retry/error processing Andrea Arcangeli
2016-12-16 14:47 ` [PATCH 20/42] userfaultfd: hugetlbfs: add userfaultfd hugetlb hook Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 21/42] userfaultfd: hugetlbfs: allow registration of ranges containing huge pages Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 22/42] userfaultfd: hugetlbfs: add userfaultfd_hugetlb test Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 23/42] userfaultfd: hugetlbfs: userfaultfd_huge_must_wait for hugepmd ranges Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 24/42] userfaultfd: hugetlbfs: gup: support VM_FAULT_RETRY Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 25/42] userfaultfd: hugetlbfs: reserve count on error in __mcopy_atomic_hugetlb Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 26/42] userfaultfd: hugetlbfs: UFFD_FEATURE_MISSING_HUGETLBFS Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 27/42] userfaultfd: introduce vma_can_userfault Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 28/42] userfaultfd: shmem: add shmem_mcopy_atomic_pte for userfaultfd support Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 29/42] userfaultfd: shmem: introduce vma_is_shmem Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 30/42] userfaultfd: shmem: add tlbflush.h header for microblaze Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 31/42] userfaultfd: shmem: use shmem_mcopy_atomic_pte for shared memory Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 32/42] userfaultfd: shmem: add userfaultfd hook for shared memory faults Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 33/42] userfaultfd: shmem: allow registration of shared memory ranges Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 34/42] userfaultfd: shmem: add userfaultfd_shmem test Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 35/42] userfaultfd: shmem: lock the page before adding it to pagecache Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 36/42] userfaultfd: shmem: avoid leaking blocks and used blocks in UFFDIO_COPY Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 37/42] userfaultfd: hugetlbfs: UFFD_FEATURE_MISSING_SHMEM Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 38/42] userfaultfd: non-cooperative: selftest: introduce userfaultfd_open Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 39/42] userfaultfd: non-cooperative: selftest: add ufd parameter to copy_page Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 40/42] userfaultfd: non-cooperative: selftest: add test for FORK, MADVDONTNEED and REMAP events Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 41/42] userfaultfd: selftest: test UFFDIO_ZEROPAGE on all memory types Andrea Arcangeli
2016-12-16 14:48 ` [PATCH 42/42] mm: mprotect: use pmd_trans_unstable instead of taking the pmd_lock Andrea Arcangeli
2017-01-06 19:02 ` [PATCH 00/42] userfaultfd tmpfs/hugetlbfs/non-cooperative v2 Dr. David Alan Gilbert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161216144821.5183-1-aarcange@redhat.com \
--to=aarcange@redhat.com \
--cc=RAPOPORT@il.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=dgilbert@redhat.com \
--cc=hillf.zj@alibaba-inc.com \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=xemul@parallels.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.