* [PATCH v2 01/31] Revert "userfaultfd: don't fail on unrecognized features"
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
@ 2023-04-12 16:38 ` Peter Xu
2023-04-12 16:41 ` [PATCH v2 02/31] selftests/mm: Update .gitignore with two missing tests Peter Xu
` (29 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:38 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Leonardo Bras Soares Passos, Andrea Arcangeli, Axel Rasmussen,
Nadav Amit, Andrew Morton, Mike Rapoport, Mike Kravetz,
David Hildenbrand, peterx, Dmitry Safonov, linux-stable
This is a proposal to revert commit 914eedcb9ba0ff53c33808.
I found this when writting a simple UFFDIO_API test to be the first unit
test in this set. Two things breaks with the commit:
- UFFDIO_API check was lost and missing. According to man page, the
kernel should reject ioctl(UFFDIO_API) if uffdio_api.api != 0xaa. This
check is needed if the api version will be extended in the future, or
user app won't be able to identify which is a new kernel.
- Feature flags checks were removed, which means UFFDIO_API with a
feature that does not exist will also succeed. According to the man
page, we should (and it makes sense) to reject ioctl(UFFDIO_API) if
unknown features passed in.
Link: https://lore.kernel.org/r/20220722201513.1624158-1-axelrasmussen@google.com
Fixes: 914eedcb9ba0 ("userfaultfd: don't fail on unrecognized features")
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Dmitry Safonov <0x7f454c46@gmail.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: linux-stable <stable@vger.kernel.org>
Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
fs/userfaultfd.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 8395605790f6..3b2a41c330e6 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1977,8 +1977,10 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx,
ret = -EFAULT;
if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api)))
goto out;
- /* Ignore unsupported features (userspace built against newer kernel) */
- features = uffdio_api.features & UFFD_API_FEATURES;
+ features = uffdio_api.features;
+ ret = -EINVAL;
+ if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES))
+ goto err_out;
ret = -EPERM;
if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE))
goto err_out;
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 02/31] selftests/mm: Update .gitignore with two missing tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
2023-04-12 16:38 ` [PATCH v2 01/31] Revert "userfaultfd: don't fail on unrecognized features" Peter Xu
@ 2023-04-12 16:41 ` Peter Xu
2023-04-12 16:41 ` [PATCH v2 03/31] selftests/mm: Dump a summary in run_vmtests.sh Peter Xu
` (28 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:41 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Axel Rasmussen, David Hildenbrand, Andrea Arcangeli,
Mike Rapoport, Nadav Amit, peterx, Mike Kravetz, Andrew Morton,
Leonardo Bras Soares Passos, Mike Rapoport
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/.gitignore | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 1f8c36a9fa10..347277f2adc3 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -36,3 +36,5 @@ split_huge_page_test
ksm_tests
local_config.h
local_config.mk
+ksm_functional_tests
+mdwe_test
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 03/31] selftests/mm: Dump a summary in run_vmtests.sh
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
2023-04-12 16:38 ` [PATCH v2 01/31] Revert "userfaultfd: don't fail on unrecognized features" Peter Xu
2023-04-12 16:41 ` [PATCH v2 02/31] selftests/mm: Update .gitignore with two missing tests Peter Xu
@ 2023-04-12 16:41 ` Peter Xu
2023-04-12 16:41 ` [PATCH v2 04/31] selftests/mm: Merge util.h into vm_util.h Peter Xu
` (27 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:41 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Axel Rasmussen, David Hildenbrand, Andrea Arcangeli,
Mike Rapoport, Nadav Amit, peterx, Mike Kravetz, Andrew Morton,
Leonardo Bras Soares Passos, Mike Rapoport
Dump a summary after running whatever test specified. Useful for human
runners to identify any kind of failures (besides exit code).
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/run_vmtests.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index c0f93b668c0c..ddf40f883747 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -5,6 +5,9 @@
# Kselftest framework requirement - SKIP code is 4.
ksft_skip=4
+count_pass=0
+count_fail=0
+count_skip=0
exitcode=0
usage() {
@@ -149,11 +152,14 @@ run_test() {
"$@"
local ret=$?
if [ $ret -eq 0 ]; then
+ count_pass=$(( count_pass + 1 ))
echo "[PASS]"
elif [ $ret -eq $ksft_skip ]; then
+ count_skip=$(( count_skip + 1 ))
echo "[SKIP]"
exitcode=$ksft_skip
else
+ count_fail=$(( count_fail + 1 ))
echo "[FAIL]"
exitcode=1
fi
@@ -279,4 +285,6 @@ CATEGORY="soft_dirty" run_test ./soft-dirty
# COW tests
CATEGORY="cow" run_test ./cow
+echo "SUMMARY: PASS=${count_pass} SKIP=${count_skip} FAIL=${count_fail}"
+
exit $exitcode
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 04/31] selftests/mm: Merge util.h into vm_util.h
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (2 preceding siblings ...)
2023-04-12 16:41 ` [PATCH v2 03/31] selftests/mm: Dump a summary in run_vmtests.sh Peter Xu
@ 2023-04-12 16:41 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper Peter Xu
` (26 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:41 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Axel Rasmussen, David Hildenbrand, Andrea Arcangeli,
Mike Rapoport, Nadav Amit, peterx, Mike Kravetz, Andrew Morton,
Leonardo Bras Soares Passos, Mike Rapoport
There're two util headers under mm/ kselftest. Merge one with another. It
turns out util.h is the easy one to move.
When merging, drop PAGE_SIZE / PAGE_SHIFT because they're unnecessary
wrappers to page_size() / page_shift(), meanwhile rename them to psize()
and pshift() so as to not conflict with some existing definitions in some
test files that includes vm_util.h.
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/Makefile | 4 ++
tools/testing/selftests/mm/gup_test.c | 5 +-
tools/testing/selftests/mm/ksm_tests.c | 2 +-
tools/testing/selftests/mm/mrelease_test.c | 11 ++-
tools/testing/selftests/mm/transhuge-stress.c | 12 ++--
tools/testing/selftests/mm/util.h | 69 -------------------
tools/testing/selftests/mm/vm_util.c | 31 +++++++++
tools/testing/selftests/mm/vm_util.h | 31 +++++++++
8 files changed, 80 insertions(+), 85 deletions(-)
delete mode 100644 tools/testing/selftests/mm/util.h
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 8235dddbbbc6..9d9822b4bf24 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -113,6 +113,10 @@ $(OUTPUT)/mkdirty: vm_util.c
$(OUTPUT)/soft-dirty: vm_util.c
$(OUTPUT)/split_huge_page_test: vm_util.c
$(OUTPUT)/userfaultfd: vm_util.c
+$(OUTPUT)/gup_test: vm_util.c
+$(OUTPUT)/mrelease_test: vm_util.c
+$(OUTPUT)/transhuge-stress: vm_util.c
+$(OUTPUT)/ksm_tests: vm_util.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
diff --git a/tools/testing/selftests/mm/gup_test.c b/tools/testing/selftests/mm/gup_test.c
index e43879291dac..ec2229136384 100644
--- a/tools/testing/selftests/mm/gup_test.c
+++ b/tools/testing/selftests/mm/gup_test.c
@@ -12,8 +12,7 @@
#include <assert.h>
#include <mm/gup_test.h>
#include "../kselftest.h"
-
-#include "util.h"
+#include "vm_util.h"
#define MB (1UL << 20)
@@ -251,7 +250,7 @@ int main(int argc, char **argv)
if (touch) {
gup.gup_flags |= FOLL_TOUCH;
} else {
- for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE)
+ for (; (unsigned long)p < gup.addr + size; p += psize())
p[0] = 0;
}
diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c
index f9eb4d67e0dd..74281593a124 100644
--- a/tools/testing/selftests/mm/ksm_tests.c
+++ b/tools/testing/selftests/mm/ksm_tests.c
@@ -12,7 +12,7 @@
#include "../kselftest.h"
#include <include/vdso/time64.h>
-#include "util.h"
+#include "vm_util.h"
#define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"
#define KSM_FP(s) (KSM_SYSFS_PATH s)
diff --git a/tools/testing/selftests/mm/mrelease_test.c b/tools/testing/selftests/mm/mrelease_test.c
index 6c62966ab5db..37b6d33b9e84 100644
--- a/tools/testing/selftests/mm/mrelease_test.c
+++ b/tools/testing/selftests/mm/mrelease_test.c
@@ -9,8 +9,7 @@
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
-
-#include "util.h"
+#include "vm_util.h"
#include "../kselftest.h"
@@ -32,7 +31,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
unsigned long i;
char *buf;
- buf = (char *)mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
+ buf = (char *)mmap(NULL, nr_pages * psize(), PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANON, 0, 0);
if (buf == MAP_FAILED) {
perror("mmap failed, halting the test");
@@ -40,7 +39,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
}
for (i = 0; i < nr_pages; i++)
- *((unsigned long *)(buf + (i * PAGE_SIZE))) = i;
+ *((unsigned long *)(buf + (i * psize()))) = i;
/* Signal the parent that the child is ready */
if (write(pipefd, "", 1) < 0) {
@@ -54,7 +53,7 @@ static int alloc_noexit(unsigned long nr_pages, int pipefd)
timeout--;
}
- munmap(buf, nr_pages * PAGE_SIZE);
+ munmap(buf, nr_pages * psize());
return (timeout > 0) ? KSFT_PASS : KSFT_FAIL;
}
@@ -87,7 +86,7 @@ static int child_main(int pipefd[], size_t size)
/* Allocate and fault-in memory and wait to be killed */
close(pipefd[0]);
- res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
+ res = alloc_noexit(MB(size) / psize(), pipefd[1]);
close(pipefd[1]);
return res;
}
diff --git a/tools/testing/selftests/mm/transhuge-stress.c b/tools/testing/selftests/mm/transhuge-stress.c
index e3f00adb1b82..ba9d37ad3a89 100644
--- a/tools/testing/selftests/mm/transhuge-stress.c
+++ b/tools/testing/selftests/mm/transhuge-stress.c
@@ -15,7 +15,7 @@
#include <fcntl.h>
#include <string.h>
#include <sys/mman.h>
-#include "util.h"
+#include "vm_util.h"
int backing_fd = -1;
int mmap_flags = MAP_ANONYMOUS | MAP_NORESERVE | MAP_PRIVATE;
@@ -34,10 +34,10 @@ int main(int argc, char **argv)
int pagemap_fd;
ram = sysconf(_SC_PHYS_PAGES);
- if (ram > SIZE_MAX / sysconf(_SC_PAGESIZE) / 4)
+ if (ram > SIZE_MAX / psize() / 4)
ram = SIZE_MAX / 4;
else
- ram *= sysconf(_SC_PAGESIZE);
+ ram *= psize();
len = ram;
while (++i < argc) {
@@ -58,7 +58,7 @@ int main(int argc, char **argv)
warnx("allocate %zd transhuge pages, using %zd MiB virtual memory"
" and %zd MiB of ram", len >> HPAGE_SHIFT, len >> 20,
- ram >> (20 + HPAGE_SHIFT - PAGE_SHIFT - 1));
+ ram >> (20 + HPAGE_SHIFT - pshift() - 1));
pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
if (pagemap_fd < 0)
@@ -92,7 +92,7 @@ int main(int argc, char **argv)
if (pfn < 0) {
nr_failed++;
} else {
- size_t idx = pfn >> (HPAGE_SHIFT - PAGE_SHIFT);
+ size_t idx = pfn >> (HPAGE_SHIFT - pshift());
nr_succeed++;
if (idx >= map_len) {
@@ -108,7 +108,7 @@ int main(int argc, char **argv)
}
/* split transhuge page, keep last page */
- if (madvise(p, HPAGE_SIZE - PAGE_SIZE, MADV_DONTNEED))
+ if (madvise(p, HPAGE_SIZE - psize(), MADV_DONTNEED))
err(2, "MADV_DONTNEED");
}
clock_gettime(CLOCK_MONOTONIC, &b);
diff --git a/tools/testing/selftests/mm/util.h b/tools/testing/selftests/mm/util.h
deleted file mode 100644
index b27d26199334..000000000000
--- a/tools/testing/selftests/mm/util.h
+++ /dev/null
@@ -1,69 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-
-#ifndef __KSELFTEST_VM_UTIL_H
-#define __KSELFTEST_VM_UTIL_H
-
-#include <stdint.h>
-#include <sys/mman.h>
-#include <err.h>
-#include <string.h> /* ffsl() */
-#include <unistd.h> /* _SC_PAGESIZE */
-
-static unsigned int __page_size;
-static unsigned int __page_shift;
-
-static inline unsigned int page_size(void)
-{
- if (!__page_size)
- __page_size = sysconf(_SC_PAGESIZE);
- return __page_size;
-}
-
-static inline unsigned int page_shift(void)
-{
- if (!__page_shift)
- __page_shift = (ffsl(page_size()) - 1);
- return __page_shift;
-}
-
-#define PAGE_SHIFT (page_shift())
-#define PAGE_SIZE (page_size())
-/*
- * On ppc64 this will only work with radix 2M hugepage size
- */
-#define HPAGE_SHIFT 21
-#define HPAGE_SIZE (1 << HPAGE_SHIFT)
-
-#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
-#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
-
-
-static inline int64_t allocate_transhuge(void *ptr, int pagemap_fd)
-{
- uint64_t ent[2];
-
- /* drop pmd */
- if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
- MAP_FIXED | MAP_ANONYMOUS |
- MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
- errx(2, "mmap transhuge");
-
- if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
- err(2, "MADV_HUGEPAGE");
-
- /* allocate transparent huge page */
- *(volatile void **)ptr = ptr;
-
- if (pread(pagemap_fd, ent, sizeof(ent),
- (uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))
- err(2, "read pagemap");
-
- if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
- PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
- !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
- return PAGEMAP_PFN(ent[0]);
-
- return -1;
-}
-
-#endif
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 8dc74dd022c2..85411ee7ff8b 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -8,6 +8,9 @@
#define SMAP_FILE_PATH "/proc/self/smaps"
#define MAX_LINE_LENGTH 500
+unsigned int __page_size;
+unsigned int __page_shift;
+
uint64_t pagemap_get_entry(int fd, char *start)
{
const unsigned long pfn = (unsigned long)start / getpagesize();
@@ -149,3 +152,31 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size)
{
return __check_huge(addr, "ShmemPmdMapped:", nr_hpages, hpage_size);
}
+
+int64_t allocate_transhuge(void *ptr, int pagemap_fd)
+{
+ uint64_t ent[2];
+
+ /* drop pmd */
+ if (mmap(ptr, HPAGE_SIZE, PROT_READ | PROT_WRITE,
+ MAP_FIXED | MAP_ANONYMOUS |
+ MAP_NORESERVE | MAP_PRIVATE, -1, 0) != ptr)
+ errx(2, "mmap transhuge");
+
+ if (madvise(ptr, HPAGE_SIZE, MADV_HUGEPAGE))
+ err(2, "MADV_HUGEPAGE");
+
+ /* allocate transparent huge page */
+ *(volatile void **)ptr = ptr;
+
+ if (pread(pagemap_fd, ent, sizeof(ent),
+ (uintptr_t)ptr >> (pshift() - 3)) != sizeof(ent))
+ err(2, "read pagemap");
+
+ if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
+ PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
+ !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - pshift())) - 1)))
+ return PAGEMAP_PFN(ent[0]);
+
+ return -1;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 1995ee911ef2..6edeb531afc6 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -1,6 +1,27 @@
/* SPDX-License-Identifier: GPL-2.0 */
#include <stdint.h>
#include <stdbool.h>
+#include <sys/mman.h>
+#include <err.h>
+#include <string.h> /* ffsl() */
+#include <unistd.h> /* _SC_PAGESIZE */
+
+extern unsigned int __page_size;
+extern unsigned int __page_shift;
+
+static inline unsigned int psize(void)
+{
+ if (!__page_size)
+ __page_size = sysconf(_SC_PAGESIZE);
+ return __page_size;
+}
+
+static inline unsigned int pshift(void)
+{
+ if (!__page_shift)
+ __page_shift = (ffsl(psize()) - 1);
+ return __page_shift;
+}
uint64_t pagemap_get_entry(int fd, char *start);
bool pagemap_is_softdirty(int fd, char *start);
@@ -13,3 +34,13 @@ uint64_t read_pmd_pagesize(void);
bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
+int64_t allocate_transhuge(void *ptr, int pagemap_fd);
+
+/*
+ * On ppc64 this will only work with radix 2M hugepage size
+ */
+#define HPAGE_SHIFT 21
+#define HPAGE_SIZE (1 << HPAGE_SHIFT)
+
+#define PAGEMAP_PRESENT(ent) (((ent) & (1ull << 63)) != 0)
+#define PAGEMAP_PFN(ent) ((ent) & ((1ull << 55) - 1))
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (3 preceding siblings ...)
2023-04-12 16:41 ` [PATCH v2 04/31] selftests/mm: Merge util.h into vm_util.h Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-15 15:44 ` Lorenzo Stoakes
2023-04-12 16:42 ` [PATCH v2 06/31] selftests/mm: Link vm_util.c always Peter Xu
` (25 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
specify programs that need to build. Logically all these binaries should
all fall into TEST_GEN_PROGS.
Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
all the tests easily later.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/Makefile | 63 +++++++++++++++--------------
1 file changed, 32 insertions(+), 31 deletions(-)
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 9d9822b4bf24..74988f5adc5f 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -31,35 +31,36 @@ MAKEFLAGS += --no-builtin-rules
CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
LDLIBS = -lrt -lpthread
-TEST_GEN_FILES = cow
-TEST_GEN_FILES += compaction_test
-TEST_GEN_FILES += gup_test
-TEST_GEN_FILES += hmm-tests
-TEST_GEN_FILES += hugetlb-madvise
-TEST_GEN_FILES += hugepage-mmap
-TEST_GEN_FILES += hugepage-mremap
-TEST_GEN_FILES += hugepage-shm
-TEST_GEN_FILES += hugepage-vmemmap
-TEST_GEN_FILES += khugepaged
+
+TEST_GEN_PROGS = cow
+TEST_GEN_PROGS += compaction_test
+TEST_GEN_PROGS += gup_test
+TEST_GEN_PROGS += hmm-tests
+TEST_GEN_PROGS += hugetlb-madvise
+TEST_GEN_PROGS += hugepage-mmap
+TEST_GEN_PROGS += hugepage-mremap
+TEST_GEN_PROGS += hugepage-shm
+TEST_GEN_PROGS += hugepage-vmemmap
+TEST_GEN_PROGS += khugepaged
TEST_GEN_PROGS = madv_populate
-TEST_GEN_FILES += map_fixed_noreplace
-TEST_GEN_FILES += map_hugetlb
-TEST_GEN_FILES += map_populate
-TEST_GEN_FILES += memfd_secret
-TEST_GEN_FILES += migration
+TEST_GEN_PROGS += map_fixed_noreplace
+TEST_GEN_PROGS += map_hugetlb
+TEST_GEN_PROGS += map_populate
+TEST_GEN_PROGS += memfd_secret
+TEST_GEN_PROGS += migration
TEST_GEN_PROGS += mkdirty
-TEST_GEN_FILES += mlock-random-test
-TEST_GEN_FILES += mlock2-tests
-TEST_GEN_FILES += mrelease_test
-TEST_GEN_FILES += mremap_dontunmap
-TEST_GEN_FILES += mremap_test
-TEST_GEN_FILES += on-fault-limit
-TEST_GEN_FILES += thuge-gen
-TEST_GEN_FILES += transhuge-stress
-TEST_GEN_FILES += userfaultfd
+TEST_GEN_PROGS += mlock-random-test
+TEST_GEN_PROGS += mlock2-tests
+TEST_GEN_PROGS += mrelease_test
+TEST_GEN_PROGS += mremap_dontunmap
+TEST_GEN_PROGS += mremap_test
+TEST_GEN_PROGS += on-fault-limit
+TEST_GEN_PROGS += thuge-gen
+TEST_GEN_PROGS += transhuge-stress
+TEST_GEN_PROGS += userfaultfd
TEST_GEN_PROGS += soft-dirty
TEST_GEN_PROGS += split_huge_page_test
-TEST_GEN_FILES += ksm_tests
+TEST_GEN_PROGS += ksm_tests
TEST_GEN_PROGS += ksm_functional_tests
TEST_GEN_PROGS += mdwe_test
@@ -77,24 +78,24 @@ CFLAGS += -no-pie
endif
ifeq ($(CAN_BUILD_I386),1)
-TEST_GEN_FILES += $(BINARIES_32)
+TEST_GEN_PROGS += $(BINARIES_32)
endif
ifeq ($(CAN_BUILD_X86_64),1)
-TEST_GEN_FILES += $(BINARIES_64)
+TEST_GEN_PROGS += $(BINARIES_64)
endif
else
ifneq (,$(findstring $(MACHINE),ppc64))
-TEST_GEN_FILES += protection_keys
+TEST_GEN_PROGS += protection_keys
endif
endif
ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64))
-TEST_GEN_FILES += va_128TBswitch
-TEST_GEN_FILES += virtual_address_range
-TEST_GEN_FILES += write_to_hugetlbfs
+TEST_GEN_PROGS += va_128TBswitch
+TEST_GEN_PROGS += virtual_address_range
+TEST_GEN_PROGS += write_to_hugetlbfs
endif
TEST_PROGS := run_vmtests.sh
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper
2023-04-12 16:42 ` [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper Peter Xu
@ 2023-04-15 15:44 ` Lorenzo Stoakes
0 siblings, 0 replies; 43+ messages in thread
From: Lorenzo Stoakes @ 2023-04-15 15:44 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Axel Rasmussen, Leonardo Bras Soares Passos, Mike Rapoport
On Wed, Apr 12, 2023 at 12:42:18PM -0400, Peter Xu wrote:
> TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
> specify programs that need to build. Logically all these binaries should
> all fall into TEST_GEN_PROGS.
>
> Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
> all the tests easily later.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/Makefile | 63 +++++++++++++++--------------
> 1 file changed, 32 insertions(+), 31 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 9d9822b4bf24..74988f5adc5f 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -31,35 +31,36 @@ MAKEFLAGS += --no-builtin-rules
>
> CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
> LDLIBS = -lrt -lpthread
> -TEST_GEN_FILES = cow
> -TEST_GEN_FILES += compaction_test
> -TEST_GEN_FILES += gup_test
> -TEST_GEN_FILES += hmm-tests
> -TEST_GEN_FILES += hugetlb-madvise
> -TEST_GEN_FILES += hugepage-mmap
> -TEST_GEN_FILES += hugepage-mremap
> -TEST_GEN_FILES += hugepage-shm
> -TEST_GEN_FILES += hugepage-vmemmap
> -TEST_GEN_FILES += khugepaged
> +
> +TEST_GEN_PROGS = cow
> +TEST_GEN_PROGS += compaction_test
> +TEST_GEN_PROGS += gup_test
> +TEST_GEN_PROGS += hmm-tests
> +TEST_GEN_PROGS += hugetlb-madvise
> +TEST_GEN_PROGS += hugepage-mmap
> +TEST_GEN_PROGS += hugepage-mremap
> +TEST_GEN_PROGS += hugepage-shm
> +TEST_GEN_PROGS += hugepage-vmemmap
> +TEST_GEN_PROGS += khugepaged
> TEST_GEN_PROGS = madv_populate
This now results in TEST_GEN_PROGS being set to "cow compaction_test
gup_test..." then here that is discarded and we start again from
madv_populate meaning none of the prior tests get built.
I notice this today when trying to build the gup_test :)
I think this should be a += not an = (which made sense before as there were
two different variables being assigned then).
> -TEST_GEN_FILES += map_fixed_noreplace
> -TEST_GEN_FILES += map_hugetlb
> -TEST_GEN_FILES += map_populate
> -TEST_GEN_FILES += memfd_secret
> -TEST_GEN_FILES += migration
> +TEST_GEN_PROGS += map_fixed_noreplace
> +TEST_GEN_PROGS += map_hugetlb
> +TEST_GEN_PROGS += map_populate
> +TEST_GEN_PROGS += memfd_secret
> +TEST_GEN_PROGS += migration
> TEST_GEN_PROGS += mkdirty
> -TEST_GEN_FILES += mlock-random-test
> -TEST_GEN_FILES += mlock2-tests
> -TEST_GEN_FILES += mrelease_test
> -TEST_GEN_FILES += mremap_dontunmap
> -TEST_GEN_FILES += mremap_test
> -TEST_GEN_FILES += on-fault-limit
> -TEST_GEN_FILES += thuge-gen
> -TEST_GEN_FILES += transhuge-stress
> -TEST_GEN_FILES += userfaultfd
> +TEST_GEN_PROGS += mlock-random-test
> +TEST_GEN_PROGS += mlock2-tests
> +TEST_GEN_PROGS += mrelease_test
> +TEST_GEN_PROGS += mremap_dontunmap
> +TEST_GEN_PROGS += mremap_test
> +TEST_GEN_PROGS += on-fault-limit
> +TEST_GEN_PROGS += thuge-gen
> +TEST_GEN_PROGS += transhuge-stress
> +TEST_GEN_PROGS += userfaultfd
> TEST_GEN_PROGS += soft-dirty
> TEST_GEN_PROGS += split_huge_page_test
> -TEST_GEN_FILES += ksm_tests
> +TEST_GEN_PROGS += ksm_tests
> TEST_GEN_PROGS += ksm_functional_tests
> TEST_GEN_PROGS += mdwe_test
>
> @@ -77,24 +78,24 @@ CFLAGS += -no-pie
> endif
>
> ifeq ($(CAN_BUILD_I386),1)
> -TEST_GEN_FILES += $(BINARIES_32)
> +TEST_GEN_PROGS += $(BINARIES_32)
> endif
>
> ifeq ($(CAN_BUILD_X86_64),1)
> -TEST_GEN_FILES += $(BINARIES_64)
> +TEST_GEN_PROGS += $(BINARIES_64)
> endif
> else
>
> ifneq (,$(findstring $(MACHINE),ppc64))
> -TEST_GEN_FILES += protection_keys
> +TEST_GEN_PROGS += protection_keys
> endif
>
> endif
>
> ifneq (,$(filter $(MACHINE),arm64 ia64 mips64 parisc64 ppc64 riscv64 s390x sh64 sparc64 x86_64))
> -TEST_GEN_FILES += va_128TBswitch
> -TEST_GEN_FILES += virtual_address_range
> -TEST_GEN_FILES += write_to_hugetlbfs
> +TEST_GEN_PROGS += va_128TBswitch
> +TEST_GEN_PROGS += virtual_address_range
> +TEST_GEN_PROGS += write_to_hugetlbfs
> endif
>
> TEST_PROGS := run_vmtests.sh
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 06/31] selftests/mm: Link vm_util.c always
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (4 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 05/31] selftests/mm: Use TEST_GEN_PROGS where proper Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 07/31] selftests/mm: Merge default_huge_page_size() into one Peter Xu
` (24 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
We do have plenty of files that want to link against vm_util.c. Just make
it simple by linking it always.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/Makefile | 13 +------------
1 file changed, 1 insertion(+), 12 deletions(-)
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 74988f5adc5f..5f7626550e5f 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -106,18 +106,7 @@ TEST_FILES += va_128TBswitch.sh
include ../lib.mk
-$(OUTPUT)/cow: vm_util.c
-$(OUTPUT)/khugepaged: vm_util.c
-$(OUTPUT)/ksm_functional_tests: vm_util.c
-$(OUTPUT)/madv_populate: vm_util.c
-$(OUTPUT)/mkdirty: vm_util.c
-$(OUTPUT)/soft-dirty: vm_util.c
-$(OUTPUT)/split_huge_page_test: vm_util.c
-$(OUTPUT)/userfaultfd: vm_util.c
-$(OUTPUT)/gup_test: vm_util.c
-$(OUTPUT)/mrelease_test: vm_util.c
-$(OUTPUT)/transhuge-stress: vm_util.c
-$(OUTPUT)/ksm_tests: vm_util.c
+$(TEST_GEN_PROGS): vm_util.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 07/31] selftests/mm: Merge default_huge_page_size() into one
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (5 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 06/31] selftests/mm: Link vm_util.c always Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 08/31] selftests/mm: Use PM_* macros in vm_utils.h Peter Xu
` (23 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
There're already 3 same definitions of the three functions. Move it into
vm_util.[ch].
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/hugetlb-madvise.c | 25 +-------------------
tools/testing/selftests/mm/thuge-gen.c | 19 +--------------
tools/testing/selftests/mm/userfaultfd.c | 24 -------------------
tools/testing/selftests/mm/vm_util.c | 21 ++++++++++++++++
tools/testing/selftests/mm/vm_util.h | 1 +
5 files changed, 24 insertions(+), 66 deletions(-)
diff --git a/tools/testing/selftests/mm/hugetlb-madvise.c b/tools/testing/selftests/mm/hugetlb-madvise.c
index 9a127a8fe176..28426e30d9bc 100644
--- a/tools/testing/selftests/mm/hugetlb-madvise.c
+++ b/tools/testing/selftests/mm/hugetlb-madvise.c
@@ -18,6 +18,7 @@
#include <unistd.h>
#include <sys/mman.h>
#include <fcntl.h>
+#include "vm_util.h"
#define MIN_FREE_PAGES 20
#define NR_HUGE_PAGES 10 /* common number of pages to map/allocate */
@@ -35,30 +36,6 @@
unsigned long huge_page_size;
unsigned long base_page_size;
-/*
- * default_huge_page_size copied from mlock2-tests.c
- */
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
-
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
-
- free(line);
- fclose(f);
- return hps;
-}
-
unsigned long get_free_hugepages(void)
{
unsigned long fhp = 0;
diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
index 361ef7192cc6..380ab5f0a534 100644
--- a/tools/testing/selftests/mm/thuge-gen.c
+++ b/tools/testing/selftests/mm/thuge-gen.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include <stdarg.h>
#include <string.h>
+#include "vm_util.h"
#define err(x) perror(x), exit(1)
@@ -74,24 +75,6 @@ void find_pagesizes(void)
globfree(&g);
}
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
- free(line);
- return hps;
-}
-
void show(unsigned long ps)
{
char buf[100];
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index a96d126cb40e..4cc80a0e8955 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1703,30 +1703,6 @@ static int userfaultfd_stress(void)
|| userfaultfd_events_test() || userfaultfd_minor_test();
}
-/*
- * Copied from mlock2-tests.c
- */
-unsigned long default_huge_page_size(void)
-{
- unsigned long hps = 0;
- char *line = NULL;
- size_t linelen = 0;
- FILE *f = fopen("/proc/meminfo", "r");
-
- if (!f)
- return 0;
- while (getline(&line, &linelen, f) > 0) {
- if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
- hps <<= 10;
- break;
- }
- }
-
- free(line);
- fclose(f);
- return hps;
-}
-
static void set_test_type(const char *type)
{
if (!strcmp(type, "anon")) {
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 85411ee7ff8b..7ffad87aa7e8 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -180,3 +180,24 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd)
return -1;
}
+
+unsigned long default_huge_page_size(void)
+{
+ unsigned long hps = 0;
+ char *line = NULL;
+ size_t linelen = 0;
+ FILE *f = fopen("/proc/meminfo", "r");
+
+ if (!f)
+ return 0;
+ while (getline(&line, &linelen, f) > 0) {
+ if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
+ hps <<= 10;
+ break;
+ }
+ }
+
+ free(line);
+ fclose(f);
+ return hps;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 6edeb531afc6..d7163fff8fb7 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -35,6 +35,7 @@ bool check_huge_anon(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_file(void *addr, int nr_hpages, uint64_t hpage_size);
bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
int64_t allocate_transhuge(void *ptr, int pagemap_fd);
+unsigned long default_huge_page_size(void);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 08/31] selftests/mm: Use PM_* macros in vm_utils.h
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (6 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 07/31] selftests/mm: Merge default_huge_page_size() into one Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 09/31] selftests/mm: Reuse pagemap_get_entry() in vm_util.h Peter Xu
` (22 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
We've got the macros in uffd-stress.c, move it over and use it in
vm_util.h.
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/userfaultfd.c | 8 --------
tools/testing/selftests/mm/vm_util.c | 16 ++++------------
tools/testing/selftests/mm/vm_util.h | 8 ++++++++
3 files changed, 12 insertions(+), 20 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 4cc80a0e8955..7e841f7e2884 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1389,14 +1389,6 @@ static int userfaultfd_minor_test(void)
return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
}
-#define BIT_ULL(nr) (1ULL << (nr))
-#define PM_SOFT_DIRTY BIT_ULL(55)
-#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
-#define PM_UFFD_WP BIT_ULL(57)
-#define PM_FILE BIT_ULL(61)
-#define PM_SWAP BIT_ULL(62)
-#define PM_PRESENT BIT_ULL(63)
-
static int pagemap_open(void)
{
int fd = open("/proc/self/pagemap", O_RDONLY);
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 7ffad87aa7e8..54d227d6f70a 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -25,25 +25,17 @@ uint64_t pagemap_get_entry(int fd, char *start)
bool pagemap_is_softdirty(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- // Check if dirty bit (55th bit) is set
- return entry & 0x0080000000000000ull;
+ return pagemap_get_entry(fd, start) & PM_SOFT_DIRTY;
}
bool pagemap_is_swapped(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- return entry & 0x4000000000000000ull;
+ return pagemap_get_entry(fd, start) & PM_SWAP;
}
bool pagemap_is_populated(int fd, char *start)
{
- uint64_t entry = pagemap_get_entry(fd, start);
-
- /* Present or swapped. */
- return entry & 0xc000000000000000ull;
+ return pagemap_get_entry(fd, start) & (PM_PRESENT | PM_SWAP);
}
unsigned long pagemap_get_pfn(int fd, char *start)
@@ -51,7 +43,7 @@ unsigned long pagemap_get_pfn(int fd, char *start)
uint64_t entry = pagemap_get_entry(fd, start);
/* If present (63th bit), PFN is at bit 0 -- 54. */
- if (entry & 0x8000000000000000ull)
+ if (entry & PM_PRESENT)
return entry & 0x007fffffffffffffull;
return -1ul;
}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index d7163fff8fb7..d9fadddb5c69 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -6,6 +6,14 @@
#include <string.h> /* ffsl() */
#include <unistd.h> /* _SC_PAGESIZE */
+#define BIT_ULL(nr) (1ULL << (nr))
+#define PM_SOFT_DIRTY BIT_ULL(55)
+#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
+#define PM_UFFD_WP BIT_ULL(57)
+#define PM_FILE BIT_ULL(61)
+#define PM_SWAP BIT_ULL(62)
+#define PM_PRESENT BIT_ULL(63)
+
extern unsigned int __page_size;
extern unsigned int __page_shift;
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 09/31] selftests/mm: Reuse pagemap_get_entry() in vm_util.h
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (7 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 08/31] selftests/mm: Use PM_* macros in vm_utils.h Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb Peter Xu
` (21 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
Meanwhile drop pagemap_read_vaddr().
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/userfaultfd.c | 31 +++++++-----------------
1 file changed, 9 insertions(+), 22 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 7e841f7e2884..795fbc4d84f8 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1399,19 +1399,6 @@ static int pagemap_open(void)
return fd;
}
-static uint64_t pagemap_read_vaddr(int fd, void *vaddr)
-{
- uint64_t value;
- int ret;
-
- ret = pread(fd, &value, sizeof(uint64_t),
- ((uint64_t)vaddr >> 12) * sizeof(uint64_t));
- if (ret != sizeof(uint64_t))
- err("pread() on pagemap failed");
-
- return value;
-}
-
/* This macro let __LINE__ works in err() */
#define pagemap_check_wp(value, wp) do { \
if (!!(value & PM_UFFD_WP) != wp) \
@@ -1427,7 +1414,7 @@ static int pagemap_test_fork(bool present)
if (!child) {
/* Open the pagemap fd of the child itself */
fd = pagemap_open();
- value = pagemap_read_vaddr(fd, area_dst);
+ value = pagemap_get_entry(fd, area_dst);
/*
* After fork() uffd-wp bit should be gone as long as we're
* without UFFD_FEATURE_EVENT_FORK
@@ -1446,24 +1433,24 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
/* Test applying pte marker to anon unpopulated */
wp_range(uffd, (uint64_t)area_dst, page_size, true);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Test unprotect on anon pte marker */
wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Test zap on anon marker */
wp_range(uffd, (uint64_t)area_dst, page_size, true);
if (madvise(area_dst, page_size, MADV_DONTNEED))
err("madvise(MADV_DONTNEED) failed");
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Test fault in after marker removed */
*area_dst = 1;
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Drop it to make pte none again */
if (madvise(area_dst, page_size, MADV_DONTNEED))
@@ -1522,7 +1509,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
/* Touch the page */
*area_dst = 1;
wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Make sure uffd-wp bit dropped when fork */
if (pagemap_test_fork(true))
@@ -1536,7 +1523,7 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
err("madvise(MADV_PAGEOUT) failed");
/* Uffd-wp should persist even swapped out */
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, true);
/* Make sure uffd-wp bit dropped when fork */
if (pagemap_test_fork(false))
@@ -1544,12 +1531,12 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
/* Unprotect; this tests swap pte modifications */
wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
/* Fault in the page from disk */
*area_dst = 2;
- value = pagemap_read_vaddr(pagemap_fd, area_dst);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
pagemap_check_wp(value, false);
close(pagemap_fd);
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (8 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 09/31] selftests/mm: Reuse pagemap_get_entry() in vm_util.h Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 17:07 ` Axel Rasmussen
2023-04-12 16:42 ` [PATCH v2 11/31] selftests/mm: Drop test_uffdio_zeropage_eexist Peter Xu
` (20 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx
Make the check as simple as "test_type == TEST_HUGETLB" because that's the
only mem that doesn't support ZEROPAGE.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/userfaultfd.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 795fbc4d84f8..d724f1c78847 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
{
struct uffdio_zeropage uffdio_zeropage;
int ret;
- bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
+ bool has_zeropage = !(test_type == TEST_HUGETLB);
__s64 res;
if (offset >= nr_pages * page_size)
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb
2023-04-12 16:42 ` [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb Peter Xu
@ 2023-04-12 17:07 ` Axel Rasmussen
0 siblings, 0 replies; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 17:07 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 9:42 AM Peter Xu <peterx@redhat.com> wrote:
>
> Make the check as simple as "test_type == TEST_HUGETLB" because that's the
> only mem that doesn't support ZEROPAGE.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
The end state we get to in patch 26 is what I was hoping for - we
check the ioctls reported by UFFDIO_REGISTER to decide if we rest this
or not. So then this intermediate state used to get rid of
get_expected_ioctls() is fine.
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
> ---
> tools/testing/selftests/mm/userfaultfd.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 795fbc4d84f8..d724f1c78847 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -1118,7 +1118,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
> {
> struct uffdio_zeropage uffdio_zeropage;
> int ret;
> - bool has_zeropage = get_expected_ioctls(0) & (1 << _UFFDIO_ZEROPAGE);
> + bool has_zeropage = !(test_type == TEST_HUGETLB);
> __s64 res;
>
> if (offset >= nr_pages * page_size)
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 11/31] selftests/mm: Drop test_uffdio_zeropage_eexist
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (9 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 10/31] selftests/mm: Test UFFDIO_ZEROPAGE only when !hugetlb Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 16:42 ` [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch] Peter Xu
` (19 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
The idea was trying to flip this var in the alarm handler from time to time
to test -EEXIST of UFFDIO_ZEROPAGE, but firstly it's only used in the
zeropage test so probably only used once, meanwhile we passed
"retry==false" so it'll never got tested anyway.
Drop both sides so we always test UFFDIO_ZEROPAGE retries if has_zeropage
is set (!hugetlb).
One more thing to do is doing UFFDIO_REGISTER for the alias buffer too,
because otherwise the test won't even pass! We were just lucky that this
test never really got ran at all.
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/userfaultfd.c | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index d724f1c78847..3487ec0bfcc8 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -88,7 +88,6 @@ static bool test_dev_userfaultfd;
/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
#define ALARM_INTERVAL_SECS 10
static volatile bool test_uffdio_copy_eexist = true;
-static volatile bool test_uffdio_zeropage_eexist = true;
/* Whether to test uffd write-protection */
static bool test_uffdio_wp = true;
/* Whether to test uffd minor faults */
@@ -1114,7 +1113,7 @@ static void retry_uffdio_zeropage(int ufd,
}
}
-static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
+static int __uffdio_zeropage(int ufd, unsigned long offset)
{
struct uffdio_zeropage uffdio_zeropage;
int ret;
@@ -1138,11 +1137,8 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
if (res != page_size) {
err("UFFDIO_ZEROPAGE unexpected size");
} else {
- if (test_uffdio_zeropage_eexist && retry) {
- test_uffdio_zeropage_eexist = false;
- retry_uffdio_zeropage(ufd, &uffdio_zeropage,
- offset);
- }
+ retry_uffdio_zeropage(ufd, &uffdio_zeropage,
+ offset);
return 1;
}
} else
@@ -1153,7 +1149,7 @@ static int __uffdio_zeropage(int ufd, unsigned long offset, bool retry)
static int uffdio_zeropage(int ufd, unsigned long offset)
{
- return __uffdio_zeropage(ufd, offset, false);
+ return __uffdio_zeropage(ufd, offset);
}
/* exercise UFFDIO_ZEROPAGE */
@@ -1177,6 +1173,13 @@ static int userfaultfd_zeropage_test(void)
assert_expected_ioctls_present(
uffdio_register.mode, uffdio_register.ioctls);
+ if (area_dst_alias) {
+ /* Needed this to test zeropage-retry on shared memory */
+ uffdio_register.range.start = (unsigned long) area_dst_alias;
+ if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ err("register failure");
+ }
+
if (uffdio_zeropage(uffd, 0))
if (my_bcmp(area_dst, zeropage, page_size))
err("zeropage is not zero");
@@ -1763,7 +1766,6 @@ static void sigalrm(int sig)
if (sig != SIGALRM)
abort();
test_uffdio_copy_eexist = true;
- test_uffdio_zeropage_eexist = true;
alarm(ALARM_INTERVAL_SECS);
}
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch]
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (10 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 11/31] selftests/mm: Drop test_uffdio_zeropage_eexist Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 17:59 ` Axel Rasmussen
2023-04-12 16:42 ` [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests Peter Xu
` (18 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
Move common utility functions into uffd-common.[ch] files from the original
userfaultfd.c. This prepares for a split of userfaultfd.c into two tests:
one to only cover the old but powerful stress test, the other one covers
all the functional tests.
This movement is kind of a brute-force effort for now, with light touch-ups
but nothing should really change. There's chances to optimize more, but
let's leave that for later.
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/Makefile | 2 +
tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 117 ++++
tools/testing/selftests/mm/userfaultfd.c | 694 +----------------------
4 files changed, 731 insertions(+), 693 deletions(-)
create mode 100644 tools/testing/selftests/mm/uffd-common.c
create mode 100644 tools/testing/selftests/mm/uffd-common.h
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 5f7626550e5f..36467c15ca00 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -108,6 +108,8 @@ include ../lib.mk
$(TEST_GEN_PROGS): vm_util.c
+$(OUTPUT)/userfaultfd: uffd-common.c
+
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64))
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
new file mode 100644
index 000000000000..c57757c2a36f
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -0,0 +1,611 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd tests util functions
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+
+#include "uffd-common.h"
+
+#define BASE_PMD_ADDR ((void *)(1UL << 30))
+
+volatile bool test_uffdio_copy_eexist = true;
+unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
+int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
+bool map_shared, test_collapse, test_dev_userfaultfd;
+bool test_uffdio_wp = true, test_uffdio_minor = false;
+unsigned long long *count_verify;
+uffd_test_ops_t *uffd_test_ops;
+
+static void anon_release_pages(char *rel_area)
+{
+ if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+}
+
+static void anon_allocate_area(void **alloc_area, bool is_src)
+{
+ *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+}
+
+static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+}
+
+static void hugetlb_release_pages(char *rel_area)
+{
+ if (!map_shared) {
+ if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+ } else {
+ if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
+ err("madvise(MADV_REMOVE) failed");
+ }
+}
+
+static void hugetlb_allocate_area(void **alloc_area, bool is_src)
+{
+ off_t size = nr_pages * page_size;
+ off_t offset = is_src ? 0 : size;
+ void *area_alias = NULL;
+ char **alloc_area_alias;
+
+ *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
+ (map_shared ? MAP_SHARED : MAP_PRIVATE) |
+ (is_src ? 0 : MAP_NORESERVE),
+ mem_fd, offset);
+ if (*alloc_area == MAP_FAILED)
+ err("mmap of hugetlbfs file failed");
+
+ if (map_shared) {
+ area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
+ MAP_SHARED, mem_fd, offset);
+ if (area_alias == MAP_FAILED)
+ err("mmap of hugetlb file alias failed");
+ }
+
+ if (is_src) {
+ alloc_area_alias = &area_src_alias;
+ } else {
+ alloc_area_alias = &area_dst_alias;
+ }
+ if (area_alias)
+ *alloc_area_alias = area_alias;
+}
+
+static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+ if (!map_shared)
+ return;
+
+ *start = (unsigned long) area_dst_alias + offset;
+}
+
+static void shmem_release_pages(char *rel_area)
+{
+ if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
+ err("madvise(MADV_REMOVE) failed");
+}
+
+static void shmem_allocate_area(void **alloc_area, bool is_src)
+{
+ void *area_alias = NULL;
+ size_t bytes = nr_pages * page_size;
+ unsigned long offset = is_src ? 0 : bytes;
+ char *p = NULL, *p_alias = NULL;
+
+ if (test_collapse) {
+ p = BASE_PMD_ADDR;
+ if (!is_src)
+ /* src map + alias + interleaved hpages */
+ p += 2 * (bytes + hpage_size);
+ p_alias = p;
+ p_alias += bytes;
+ p_alias += hpage_size; /* Prevent src/dst VMA merge */
+ }
+
+ *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
+ mem_fd, offset);
+ if (*alloc_area == MAP_FAILED)
+ err("mmap of memfd failed");
+ if (test_collapse && *alloc_area != p)
+ err("mmap of memfd failed at %p", p);
+
+ area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
+ mem_fd, offset);
+ if (area_alias == MAP_FAILED)
+ err("mmap of memfd alias failed");
+ if (test_collapse && area_alias != p_alias)
+ err("mmap of anonymous memory failed at %p", p_alias);
+
+ if (is_src)
+ area_src_alias = area_alias;
+ else
+ area_dst_alias = area_alias;
+}
+
+static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
+{
+ *start = (unsigned long)area_dst_alias + offset;
+}
+
+static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
+{
+ if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
+ err("Did not find expected %d number of hugepages",
+ expect_nr_hpages);
+}
+
+struct uffd_test_ops anon_uffd_test_ops = {
+ .allocate_area = anon_allocate_area,
+ .release_pages = anon_release_pages,
+ .alias_mapping = noop_alias_mapping,
+ .check_pmd_mapping = NULL,
+};
+
+struct uffd_test_ops shmem_uffd_test_ops = {
+ .allocate_area = shmem_allocate_area,
+ .release_pages = shmem_release_pages,
+ .alias_mapping = shmem_alias_mapping,
+ .check_pmd_mapping = shmem_check_pmd_mapping,
+};
+
+struct uffd_test_ops hugetlb_uffd_test_ops = {
+ .allocate_area = hugetlb_allocate_area,
+ .release_pages = hugetlb_release_pages,
+ .alias_mapping = hugetlb_alias_mapping,
+ .check_pmd_mapping = NULL,
+};
+
+void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
+{
+ int i;
+ unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
+
+ for (i = 0; i < n_cpus; i++) {
+ miss_total += stats[i].missing_faults;
+ wp_total += stats[i].wp_faults;
+ minor_total += stats[i].minor_faults;
+ }
+
+ printf("userfaults: ");
+ if (miss_total) {
+ printf("%llu missing (", miss_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].missing_faults);
+ printf("\b) ");
+ }
+ if (wp_total) {
+ printf("%llu wp (", wp_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].wp_faults);
+ printf("\b) ");
+ }
+ if (minor_total) {
+ printf("%llu minor (", minor_total);
+ for (i = 0; i < n_cpus; i++)
+ printf("%lu+", stats[i].minor_faults);
+ printf("\b)");
+ }
+ printf("\n");
+}
+
+static int __userfaultfd_open_dev(void)
+{
+ int fd, _uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
+
+ _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
+ if (_uffd < 0)
+ errexit(errno == ENOTTY ? KSFT_SKIP : 1,
+ "creating userfaultfd failed");
+ close(fd);
+ return _uffd;
+}
+
+void userfaultfd_open(uint64_t *features)
+{
+ struct uffdio_api uffdio_api;
+
+ if (test_dev_userfaultfd)
+ uffd = __userfaultfd_open_dev();
+ else {
+ uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
+ if (uffd < 0)
+ errexit(errno == ENOSYS ? KSFT_SKIP : 1,
+ "creating userfaultfd failed");
+ }
+ uffd_flags = fcntl(uffd, F_GETFD, NULL);
+
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = *features;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api))
+ err("UFFDIO_API failed.\nPlease make sure to "
+ "run with either root or ptrace capability.");
+ if (uffdio_api.api != UFFD_API)
+ err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
+
+ *features = uffdio_api.features;
+}
+
+static inline void munmap_area(void **area)
+{
+ if (*area)
+ if (munmap(*area, nr_pages * page_size))
+ err("munmap");
+
+ *area = NULL;
+}
+
+static void uffd_test_ctx_clear(void)
+{
+ size_t i;
+
+ if (pipefd) {
+ for (i = 0; i < nr_cpus * 2; ++i) {
+ if (close(pipefd[i]))
+ err("close pipefd");
+ }
+ free(pipefd);
+ pipefd = NULL;
+ }
+
+ if (count_verify) {
+ free(count_verify);
+ count_verify = NULL;
+ }
+
+ if (uffd != -1) {
+ if (close(uffd))
+ err("close uffd");
+ uffd = -1;
+ }
+
+ munmap_area((void **)&area_src);
+ munmap_area((void **)&area_src_alias);
+ munmap_area((void **)&area_dst);
+ munmap_area((void **)&area_dst_alias);
+ munmap_area((void **)&area_remap);
+}
+
+void uffd_test_ctx_init(uint64_t features)
+{
+ unsigned long nr, cpu;
+
+ uffd_test_ctx_clear();
+
+ uffd_test_ops->allocate_area((void **)&area_src, true);
+ uffd_test_ops->allocate_area((void **)&area_dst, false);
+
+ userfaultfd_open(&features);
+
+ count_verify = malloc(nr_pages * sizeof(unsigned long long));
+ if (!count_verify)
+ err("count_verify");
+
+ for (nr = 0; nr < nr_pages; nr++) {
+ *area_mutex(area_src, nr) =
+ (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
+ count_verify[nr] = *area_count(area_src, nr) = 1;
+ /*
+ * In the transition between 255 to 256, powerpc will
+ * read out of order in my_bcmp and see both bytes as
+ * zero, so leave a placeholder below always non-zero
+ * after the count, to avoid my_bcmp to trigger false
+ * positives.
+ */
+ *(area_count(area_src, nr) + 1) = 1;
+ }
+
+ /*
+ * After initialization of area_src, we must explicitly release pages
+ * for area_dst to make sure it's fully empty. Otherwise we could have
+ * some area_dst pages be errornously initialized with zero pages,
+ * hence we could hit memory corruption later in the test.
+ *
+ * One example is when THP is globally enabled, above allocate_area()
+ * calls could have the two areas merged into a single VMA (as they
+ * will have the same VMA flags so they're mergeable). When we
+ * initialize the area_src above, it's possible that some part of
+ * area_dst could have been faulted in via one huge THP that will be
+ * shared between area_src and area_dst. It could cause some of the
+ * area_dst won't be trapped by missing userfaults.
+ *
+ * This release_pages() will guarantee even if that happened, we'll
+ * proactively split the thp and drop any accidentally initialized
+ * pages within area_dst.
+ */
+ uffd_test_ops->release_pages(area_dst);
+
+ pipefd = malloc(sizeof(int) * nr_cpus * 2);
+ if (!pipefd)
+ err("pipefd");
+ for (cpu = 0; cpu < nr_cpus; cpu++)
+ if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
+ err("pipe");
+}
+
+uint64_t get_expected_ioctls(uint64_t mode)
+{
+ uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
+
+ if (test_type == TEST_HUGETLB)
+ ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
+
+ if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
+ ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
+
+ if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
+ ioctls &= ~(1 << _UFFDIO_CONTINUE);
+
+ return ioctls;
+}
+
+void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
+{
+ uint64_t expected = get_expected_ioctls(mode);
+ uint64_t actual = ioctls & expected;
+
+ if (actual != expected) {
+ err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
+ expected, actual);
+ }
+}
+
+void wp_range(int ufd, __u64 start, __u64 len, bool wp)
+{
+ struct uffdio_writeprotect prms;
+
+ /* Write protection page faults */
+ prms.range.start = start;
+ prms.range.len = len;
+ /* Undo write-protect, do wakeup after that */
+ prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
+
+ if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
+ err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
+}
+
+static void continue_range(int ufd, __u64 start, __u64 len)
+{
+ struct uffdio_continue req;
+ int ret;
+
+ req.range.start = start;
+ req.range.len = len;
+ req.mode = 0;
+ if (test_uffdio_wp)
+ req.mode |= UFFDIO_CONTINUE_MODE_WP;
+
+ if (ioctl(ufd, UFFDIO_CONTINUE, &req))
+ err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
+ (uint64_t)start);
+
+ /*
+ * Error handling within the kernel for continue is subtly different
+ * from copy or zeropage, so it may be a source of bugs. Trigger an
+ * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
+ */
+ req.mapped = 0;
+ ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
+ if (ret >= 0 || req.mapped != -EEXIST)
+ err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
+ ret, (int64_t) req.mapped);
+}
+
+int uffd_read_msg(int ufd, struct uffd_msg *msg)
+{
+ int ret = read(uffd, msg, sizeof(*msg));
+
+ if (ret != sizeof(*msg)) {
+ if (ret < 0) {
+ if (errno == EAGAIN || errno == EINTR)
+ return 1;
+ err("blocking read error");
+ } else {
+ err("short read");
+ }
+ }
+
+ return 0;
+}
+
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
+{
+ unsigned long offset;
+
+ if (msg->event != UFFD_EVENT_PAGEFAULT)
+ err("unexpected msg event %u", msg->event);
+
+ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
+ /* Write protect page faults */
+ wp_range(uffd, msg->arg.pagefault.address, page_size, false);
+ stats->wp_faults++;
+ } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
+ uint8_t *area;
+ int b;
+
+ /*
+ * Minor page faults
+ *
+ * To prove we can modify the original range for testing
+ * purposes, we're going to bit flip this range before
+ * continuing.
+ *
+ * Note that this requires all minor page fault tests operate on
+ * area_dst (non-UFFD-registered) and area_dst_alias
+ * (UFFD-registered).
+ */
+
+ area = (uint8_t *)(area_dst +
+ ((char *)msg->arg.pagefault.address -
+ area_dst_alias));
+ for (b = 0; b < page_size; ++b)
+ area[b] = ~area[b];
+ continue_range(uffd, msg->arg.pagefault.address, page_size);
+ stats->minor_faults++;
+ } else {
+ /*
+ * Missing page faults.
+ *
+ * Here we force a write check for each of the missing mode
+ * faults. It's guaranteed because the only threads that
+ * will trigger uffd faults are the locking threads, and
+ * their first instruction to touch the missing page will
+ * always be pthread_mutex_lock().
+ *
+ * Note that here we relied on an NPTL glibc impl detail to
+ * always read the lock type at the entry of the lock op
+ * (pthread_mutex_t.__data.__type, offset 0x10) before
+ * doing any locking operations to guarantee that. It's
+ * actually not good to rely on this impl detail because
+ * logically a pthread-compatible lib can implement the
+ * locks without types and we can fail when linking with
+ * them. However since we used to find bugs with this
+ * strict check we still keep it around. Hopefully this
+ * could be a good hint when it fails again. If one day
+ * it'll break on some other impl of glibc we'll revisit.
+ */
+ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
+ err("unexpected write fault");
+
+ offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
+ offset &= ~(page_size-1);
+
+ if (copy_page(uffd, offset))
+ stats->missing_faults++;
+ }
+}
+
+void *uffd_poll_thread(void *arg)
+{
+ struct uffd_stats *stats = (struct uffd_stats *)arg;
+ unsigned long cpu = stats->cpu;
+ struct pollfd pollfd[2];
+ struct uffd_msg msg;
+ struct uffdio_register uffd_reg;
+ int ret;
+ char tmp_chr;
+
+ pollfd[0].fd = uffd;
+ pollfd[0].events = POLLIN;
+ pollfd[1].fd = pipefd[cpu*2];
+ pollfd[1].events = POLLIN;
+
+ for (;;) {
+ ret = poll(pollfd, 2, -1);
+ if (ret <= 0) {
+ if (errno == EINTR || errno == EAGAIN)
+ continue;
+ err("poll error: %d", ret);
+ }
+ if (pollfd[1].revents) {
+ if (!(pollfd[1].revents & POLLIN))
+ err("pollfd[1].revents %d", pollfd[1].revents);
+ if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
+ err("read pipefd error");
+ break;
+ }
+ if (!(pollfd[0].revents & POLLIN))
+ err("pollfd[0].revents %d", pollfd[0].revents);
+ if (uffd_read_msg(uffd, &msg))
+ continue;
+ switch (msg.event) {
+ default:
+ err("unexpected msg event %u\n", msg.event);
+ break;
+ case UFFD_EVENT_PAGEFAULT:
+ uffd_handle_page_fault(&msg, stats);
+ break;
+ case UFFD_EVENT_FORK:
+ close(uffd);
+ uffd = msg.arg.fork.ufd;
+ pollfd[0].fd = uffd;
+ break;
+ case UFFD_EVENT_REMOVE:
+ uffd_reg.range.start = msg.arg.remove.start;
+ uffd_reg.range.len = msg.arg.remove.end -
+ msg.arg.remove.start;
+ if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
+ err("remove failure");
+ break;
+ case UFFD_EVENT_REMAP:
+ area_remap = area_dst; /* save for later unmap */
+ area_dst = (char *)(unsigned long)msg.arg.remap.to;
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
+ unsigned long offset)
+{
+ uffd_test_ops->alias_mapping(&uffdio_copy->dst,
+ uffdio_copy->len,
+ offset);
+ if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
+ /* real retval in ufdio_copy.copy */
+ if (uffdio_copy->copy != -EEXIST)
+ err("UFFDIO_COPY retry error: %"PRId64,
+ (int64_t)uffdio_copy->copy);
+ } else {
+ err("UFFDIO_COPY retry unexpected: %"PRId64,
+ (int64_t)uffdio_copy->copy);
+ }
+}
+
+static void wake_range(int ufd, unsigned long addr, unsigned long len)
+{
+ struct uffdio_range uffdio_wake;
+
+ uffdio_wake.start = addr;
+ uffdio_wake.len = len;
+
+ if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
+ fprintf(stderr, "error waking %lu\n",
+ addr), exit(1);
+}
+
+int __copy_page(int ufd, unsigned long offset, bool retry)
+{
+ struct uffdio_copy uffdio_copy;
+
+ if (offset >= nr_pages * page_size)
+ err("unexpected offset %lu\n", offset);
+ uffdio_copy.dst = (unsigned long) area_dst + offset;
+ uffdio_copy.src = (unsigned long) area_src + offset;
+ uffdio_copy.len = page_size;
+ if (test_uffdio_wp)
+ uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
+ else
+ uffdio_copy.mode = 0;
+ uffdio_copy.copy = 0;
+ if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
+ /* real retval in ufdio_copy.copy */
+ if (uffdio_copy.copy != -EEXIST)
+ err("UFFDIO_COPY error: %"PRId64,
+ (int64_t)uffdio_copy.copy);
+ wake_range(ufd, uffdio_copy.dst, page_size);
+ } else if (uffdio_copy.copy != page_size) {
+ err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
+ } else {
+ if (test_uffdio_copy_eexist && retry) {
+ test_uffdio_copy_eexist = false;
+ retry_copy_page(ufd, &uffdio_copy, offset);
+ }
+ return 1;
+ }
+ return 0;
+}
+
+int copy_page(int ufd, unsigned long offset)
+{
+ return __copy_page(ufd, offset, false);
+}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
new file mode 100644
index 000000000000..d9430cfdcb19
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd tests common header
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+#ifndef __UFFD_COMMON_H__
+#define __UFFD_COMMON_H__
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <errno.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <time.h>
+#include <signal.h>
+#include <poll.h>
+#include <string.h>
+#include <linux/mman.h>
+#include <sys/mman.h>
+#include <sys/syscall.h>
+#include <sys/ioctl.h>
+#include <sys/wait.h>
+#include <pthread.h>
+#include <linux/userfaultfd.h>
+#include <setjmp.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <stdint.h>
+#include <sys/random.h>
+
+#include "../kselftest.h"
+#include "vm_util.h"
+
+#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
+
+#define _err(fmt, ...) \
+ do { \
+ int ret = errno; \
+ fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
+ fprintf(stderr, " (errno=%d, @%s:%d)\n", \
+ ret, __FILE__, __LINE__); \
+ } while (0)
+
+#define errexit(exitcode, fmt, ...) \
+ do { \
+ _err(fmt, ##__VA_ARGS__); \
+ exit(exitcode); \
+ } while (0)
+
+#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
+
+/* pthread_mutex_t starts at page offset 0 */
+#define area_mutex(___area, ___nr) \
+ ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
+/*
+ * count is placed in the page after pthread_mutex_t naturally aligned
+ * to avoid non alignment faults on non-x86 archs.
+ */
+#define area_count(___area, ___nr) \
+ ((volatile unsigned long long *) ((unsigned long) \
+ ((___area) + (___nr)*page_size + \
+ sizeof(pthread_mutex_t) + \
+ sizeof(unsigned long long) - 1) & \
+ ~(unsigned long)(sizeof(unsigned long long) \
+ - 1)))
+
+/* Userfaultfd test statistics */
+struct uffd_stats {
+ int cpu;
+ unsigned long missing_faults;
+ unsigned long wp_faults;
+ unsigned long minor_faults;
+};
+
+struct uffd_test_ops {
+ void (*allocate_area)(void **alloc_area, bool is_src);
+ void (*release_pages)(char *rel_area);
+ void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
+ void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
+};
+typedef struct uffd_test_ops uffd_test_ops_t;
+
+extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
+extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
+extern bool map_shared, test_collapse, test_dev_userfaultfd;
+extern bool test_uffdio_wp, test_uffdio_minor;
+extern unsigned long long *count_verify;
+extern volatile bool test_uffdio_copy_eexist;
+
+extern uffd_test_ops_t anon_uffd_test_ops;
+extern uffd_test_ops_t shmem_uffd_test_ops;
+extern uffd_test_ops_t hugetlb_uffd_test_ops;
+extern uffd_test_ops_t *uffd_test_ops;
+
+void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
+void uffd_test_ctx_init(uint64_t features);
+void userfaultfd_open(uint64_t *features);
+uint64_t get_expected_ioctls(uint64_t mode);
+void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
+int uffd_read_msg(int ufd, struct uffd_msg *msg);
+void wp_range(int ufd, __u64 start, __u64 len, bool wp);
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
+int __copy_page(int ufd, unsigned long offset, bool retry);
+int copy_page(int ufd, unsigned long offset);
+void *uffd_poll_thread(void *arg);
+
+#define TEST_ANON 1
+#define TEST_HUGETLB 2
+#define TEST_SHMEM 3
+
+#endif
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
index 3487ec0bfcc8..c68a9aeefc41 100644
--- a/tools/testing/selftests/mm/userfaultfd.c
+++ b/tools/testing/selftests/mm/userfaultfd.c
@@ -34,96 +34,20 @@
* transfer (UFFDIO_COPY).
*/
-#define _GNU_SOURCE
-#include <stdio.h>
-#include <errno.h>
-#include <unistd.h>
-#include <stdlib.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <time.h>
-#include <signal.h>
-#include <poll.h>
-#include <string.h>
-#include <linux/mman.h>
-#include <sys/mman.h>
-#include <sys/syscall.h>
-#include <sys/ioctl.h>
-#include <sys/wait.h>
-#include <pthread.h>
-#include <linux/userfaultfd.h>
-#include <setjmp.h>
-#include <stdbool.h>
-#include <assert.h>
-#include <inttypes.h>
-#include <stdint.h>
-#include <sys/random.h>
-
-#include "../kselftest.h"
-#include "vm_util.h"
+#include "uffd-common.h"
#ifdef __NR_userfaultfd
-static unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
-
#define BOUNCE_RANDOM (1<<0)
#define BOUNCE_RACINGFAULTS (1<<1)
#define BOUNCE_VERIFY (1<<2)
#define BOUNCE_POLL (1<<3)
static int bounces;
-#define TEST_ANON 1
-#define TEST_HUGETLB 2
-#define TEST_SHMEM 3
-static int test_type;
-
-#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
-
-#define BASE_PMD_ADDR ((void *)(1UL << 30))
-
-/* test using /dev/userfaultfd, instead of userfaultfd(2) */
-static bool test_dev_userfaultfd;
-
/* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
#define ALARM_INTERVAL_SECS 10
-static volatile bool test_uffdio_copy_eexist = true;
-/* Whether to test uffd write-protection */
-static bool test_uffdio_wp = true;
-/* Whether to test uffd minor faults */
-static bool test_uffdio_minor = false;
-static bool map_shared;
-static int mem_fd;
-static unsigned long long *count_verify;
-static int uffd = -1;
-static int uffd_flags, finished, *pipefd;
-static char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
static char *zeropage;
pthread_attr_t attr;
-static bool test_collapse;
-
-/* Userfaultfd test statistics */
-struct uffd_stats {
- int cpu;
- unsigned long missing_faults;
- unsigned long wp_faults;
- unsigned long minor_faults;
-};
-
-/* pthread_mutex_t starts at page offset 0 */
-#define area_mutex(___area, ___nr) \
- ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
-/*
- * count is placed in the page after pthread_mutex_t naturally aligned
- * to avoid non alignment faults on non-x86 archs.
- */
-#define area_count(___area, ___nr) \
- ((volatile unsigned long long *) ((unsigned long) \
- ((___area) + (___nr)*page_size + \
- sizeof(pthread_mutex_t) + \
- sizeof(unsigned long long) - 1) & \
- ~(unsigned long)(sizeof(unsigned long long) \
- - 1)))
#define swap(a, b) \
do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
@@ -166,22 +90,6 @@ static void usage(void)
exit(1);
}
-#define _err(fmt, ...) \
- do { \
- int ret = errno; \
- fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
- fprintf(stderr, " (errno=%d, line=%d)\n", \
- ret, __LINE__); \
- } while (0)
-
-#define errexit(exitcode, fmt, ...) \
- do { \
- _err(fmt, ##__VA_ARGS__); \
- exit(exitcode); \
- } while (0)
-
-#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
-
static void uffd_stats_reset(struct uffd_stats *uffd_stats,
unsigned long n_cpus)
{
@@ -195,189 +103,6 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats,
}
}
-static void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
-{
- int i;
- unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
-
- for (i = 0; i < n_cpus; i++) {
- miss_total += stats[i].missing_faults;
- wp_total += stats[i].wp_faults;
- minor_total += stats[i].minor_faults;
- }
-
- printf("userfaults: ");
- if (miss_total) {
- printf("%llu missing (", miss_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].missing_faults);
- printf("\b) ");
- }
- if (wp_total) {
- printf("%llu wp (", wp_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].wp_faults);
- printf("\b) ");
- }
- if (minor_total) {
- printf("%llu minor (", minor_total);
- for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].minor_faults);
- printf("\b)");
- }
- printf("\n");
-}
-
-static void anon_release_pages(char *rel_area)
-{
- if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-}
-
-static void anon_allocate_area(void **alloc_area, bool is_src)
-{
- *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
-}
-
-static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
-}
-
-static void hugetlb_release_pages(char *rel_area)
-{
- if (!map_shared) {
- if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
- } else {
- if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
- err("madvise(MADV_REMOVE) failed");
- }
-}
-
-static void hugetlb_allocate_area(void **alloc_area, bool is_src)
-{
- off_t size = nr_pages * page_size;
- off_t offset = is_src ? 0 : size;
- void *area_alias = NULL;
- char **alloc_area_alias;
-
- *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
- (map_shared ? MAP_SHARED : MAP_PRIVATE) |
- (is_src ? 0 : MAP_NORESERVE),
- mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of hugetlbfs file failed");
-
- if (map_shared) {
- area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
- MAP_SHARED, mem_fd, offset);
- if (area_alias == MAP_FAILED)
- err("mmap of hugetlb file alias failed");
- }
-
- if (is_src) {
- alloc_area_alias = &area_src_alias;
- } else {
- alloc_area_alias = &area_dst_alias;
- }
- if (area_alias)
- *alloc_area_alias = area_alias;
-}
-
-static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
- if (!map_shared)
- return;
-
- *start = (unsigned long) area_dst_alias + offset;
-}
-
-static void shmem_release_pages(char *rel_area)
-{
- if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
- err("madvise(MADV_REMOVE) failed");
-}
-
-static void shmem_allocate_area(void **alloc_area, bool is_src)
-{
- void *area_alias = NULL;
- size_t bytes = nr_pages * page_size;
- unsigned long offset = is_src ? 0 : bytes;
- char *p = NULL, *p_alias = NULL;
-
- if (test_collapse) {
- p = BASE_PMD_ADDR;
- if (!is_src)
- /* src map + alias + interleaved hpages */
- p += 2 * (bytes + hpage_size);
- p_alias = p;
- p_alias += bytes;
- p_alias += hpage_size; /* Prevent src/dst VMA merge */
- }
-
- *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
- mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of memfd failed");
- if (test_collapse && *alloc_area != p)
- err("mmap of memfd failed at %p", p);
-
- area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
- mem_fd, offset);
- if (area_alias == MAP_FAILED)
- err("mmap of memfd alias failed");
- if (test_collapse && area_alias != p_alias)
- err("mmap of anonymous memory failed at %p", p_alias);
-
- if (is_src)
- area_src_alias = area_alias;
- else
- area_dst_alias = area_alias;
-}
-
-static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
-{
- *start = (unsigned long)area_dst_alias + offset;
-}
-
-static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
-{
- if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
- err("Did not find expected %d number of hugepages",
- expect_nr_hpages);
-}
-
-struct uffd_test_ops {
- void (*allocate_area)(void **alloc_area, bool is_src);
- void (*release_pages)(char *rel_area);
- void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
- void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
-};
-
-static struct uffd_test_ops anon_uffd_test_ops = {
- .allocate_area = anon_allocate_area,
- .release_pages = anon_release_pages,
- .alias_mapping = noop_alias_mapping,
- .check_pmd_mapping = NULL,
-};
-
-static struct uffd_test_ops shmem_uffd_test_ops = {
- .allocate_area = shmem_allocate_area,
- .release_pages = shmem_release_pages,
- .alias_mapping = shmem_alias_mapping,
- .check_pmd_mapping = shmem_check_pmd_mapping,
-};
-
-static struct uffd_test_ops hugetlb_uffd_test_ops = {
- .allocate_area = hugetlb_allocate_area,
- .release_pages = hugetlb_release_pages,
- .alias_mapping = hugetlb_alias_mapping,
- .check_pmd_mapping = NULL,
-};
-
-static struct uffd_test_ops *uffd_test_ops;
-
static inline uint64_t uffd_minor_feature(void)
{
if (test_type == TEST_HUGETLB && map_shared)
@@ -388,171 +113,6 @@ static inline uint64_t uffd_minor_feature(void)
return 0;
}
-static uint64_t get_expected_ioctls(uint64_t mode)
-{
- uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
-
- if (test_type == TEST_HUGETLB)
- ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
-
- if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
- ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
-
- if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
- ioctls &= ~(1 << _UFFDIO_CONTINUE);
-
- return ioctls;
-}
-
-static void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
-{
- uint64_t expected = get_expected_ioctls(mode);
- uint64_t actual = ioctls & expected;
-
- if (actual != expected) {
- err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
- expected, actual);
- }
-}
-
-static int __userfaultfd_open_dev(void)
-{
- int fd, _uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
-
- _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
- if (_uffd < 0)
- errexit(errno == ENOTTY ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- close(fd);
- return _uffd;
-}
-
-static void userfaultfd_open(uint64_t *features)
-{
- struct uffdio_api uffdio_api;
-
- if (test_dev_userfaultfd)
- uffd = __userfaultfd_open_dev();
- else {
- uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
- if (uffd < 0)
- errexit(errno == ENOSYS ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- }
- uffd_flags = fcntl(uffd, F_GETFD, NULL);
-
- uffdio_api.api = UFFD_API;
- uffdio_api.features = *features;
- if (ioctl(uffd, UFFDIO_API, &uffdio_api))
- err("UFFDIO_API failed.\nPlease make sure to "
- "run with either root or ptrace capability.");
- if (uffdio_api.api != UFFD_API)
- err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
-
- *features = uffdio_api.features;
-}
-
-static inline void munmap_area(void **area)
-{
- if (*area)
- if (munmap(*area, nr_pages * page_size))
- err("munmap");
-
- *area = NULL;
-}
-
-static void uffd_test_ctx_clear(void)
-{
- size_t i;
-
- if (pipefd) {
- for (i = 0; i < nr_cpus * 2; ++i) {
- if (close(pipefd[i]))
- err("close pipefd");
- }
- free(pipefd);
- pipefd = NULL;
- }
-
- if (count_verify) {
- free(count_verify);
- count_verify = NULL;
- }
-
- if (uffd != -1) {
- if (close(uffd))
- err("close uffd");
- uffd = -1;
- }
-
- munmap_area((void **)&area_src);
- munmap_area((void **)&area_src_alias);
- munmap_area((void **)&area_dst);
- munmap_area((void **)&area_dst_alias);
- munmap_area((void **)&area_remap);
-}
-
-static void uffd_test_ctx_init(uint64_t features)
-{
- unsigned long nr, cpu;
-
- uffd_test_ctx_clear();
-
- uffd_test_ops->allocate_area((void **)&area_src, true);
- uffd_test_ops->allocate_area((void **)&area_dst, false);
-
- userfaultfd_open(&features);
-
- count_verify = malloc(nr_pages * sizeof(unsigned long long));
- if (!count_verify)
- err("count_verify");
-
- for (nr = 0; nr < nr_pages; nr++) {
- *area_mutex(area_src, nr) =
- (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
- count_verify[nr] = *area_count(area_src, nr) = 1;
- /*
- * In the transition between 255 to 256, powerpc will
- * read out of order in my_bcmp and see both bytes as
- * zero, so leave a placeholder below always non-zero
- * after the count, to avoid my_bcmp to trigger false
- * positives.
- */
- *(area_count(area_src, nr) + 1) = 1;
- }
-
- /*
- * After initialization of area_src, we must explicitly release pages
- * for area_dst to make sure it's fully empty. Otherwise we could have
- * some area_dst pages be errornously initialized with zero pages,
- * hence we could hit memory corruption later in the test.
- *
- * One example is when THP is globally enabled, above allocate_area()
- * calls could have the two areas merged into a single VMA (as they
- * will have the same VMA flags so they're mergeable). When we
- * initialize the area_src above, it's possible that some part of
- * area_dst could have been faulted in via one huge THP that will be
- * shared between area_src and area_dst. It could cause some of the
- * area_dst won't be trapped by missing userfaults.
- *
- * This release_pages() will guarantee even if that happened, we'll
- * proactively split the thp and drop any accidentally initialized
- * pages within area_dst.
- */
- uffd_test_ops->release_pages(area_dst);
-
- pipefd = malloc(sizeof(int) * nr_cpus * 2);
- if (!pipefd)
- err("pipefd");
- for (cpu = 0; cpu < nr_cpus; cpu++)
- if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
- err("pipe");
-}
-
static int my_bcmp(char *str1, char *str2, size_t n)
{
unsigned long i;
@@ -562,47 +122,6 @@ static int my_bcmp(char *str1, char *str2, size_t n)
return 0;
}
-static void wp_range(int ufd, __u64 start, __u64 len, bool wp)
-{
- struct uffdio_writeprotect prms;
-
- /* Write protection page faults */
- prms.range.start = start;
- prms.range.len = len;
- /* Undo write-protect, do wakeup after that */
- prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
-
- if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
- err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
-}
-
-static void continue_range(int ufd, __u64 start, __u64 len)
-{
- struct uffdio_continue req;
- int ret;
-
- req.range.start = start;
- req.range.len = len;
- req.mode = 0;
- if (test_uffdio_wp)
- req.mode |= UFFDIO_CONTINUE_MODE_WP;
-
- if (ioctl(ufd, UFFDIO_CONTINUE, &req))
- err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
- (uint64_t)start);
-
- /*
- * Error handling within the kernel for continue is subtly different
- * from copy or zeropage, so it may be a source of bugs. Trigger an
- * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
- */
- req.mapped = 0;
- ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
- if (ret >= 0 || req.mapped != -EEXIST)
- err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
- ret, (int64_t) req.mapped);
-}
-
static void *locking_thread(void *arg)
{
unsigned long cpu = (unsigned long) arg;
@@ -635,222 +154,11 @@ static void *locking_thread(void *arg)
return NULL;
}
-static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
- unsigned long offset)
-{
- uffd_test_ops->alias_mapping(&uffdio_copy->dst,
- uffdio_copy->len,
- offset);
- if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
- /* real retval in ufdio_copy.copy */
- if (uffdio_copy->copy != -EEXIST)
- err("UFFDIO_COPY retry error: %"PRId64,
- (int64_t)uffdio_copy->copy);
- } else {
- err("UFFDIO_COPY retry unexpected: %"PRId64,
- (int64_t)uffdio_copy->copy);
- }
-}
-
-static void wake_range(int ufd, unsigned long addr, unsigned long len)
-{
- struct uffdio_range uffdio_wake;
-
- uffdio_wake.start = addr;
- uffdio_wake.len = len;
-
- if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
- fprintf(stderr, "error waking %lu\n",
- addr), exit(1);
-}
-
-static int __copy_page(int ufd, unsigned long offset, bool retry)
-{
- struct uffdio_copy uffdio_copy;
-
- if (offset >= nr_pages * page_size)
- err("unexpected offset %lu\n", offset);
- uffdio_copy.dst = (unsigned long) area_dst + offset;
- uffdio_copy.src = (unsigned long) area_src + offset;
- uffdio_copy.len = page_size;
- if (test_uffdio_wp)
- uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
- else
- uffdio_copy.mode = 0;
- uffdio_copy.copy = 0;
- if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
- /* real retval in ufdio_copy.copy */
- if (uffdio_copy.copy != -EEXIST)
- err("UFFDIO_COPY error: %"PRId64,
- (int64_t)uffdio_copy.copy);
- wake_range(ufd, uffdio_copy.dst, page_size);
- } else if (uffdio_copy.copy != page_size) {
- err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
- } else {
- if (test_uffdio_copy_eexist && retry) {
- test_uffdio_copy_eexist = false;
- retry_copy_page(ufd, &uffdio_copy, offset);
- }
- return 1;
- }
- return 0;
-}
-
static int copy_page_retry(int ufd, unsigned long offset)
{
return __copy_page(ufd, offset, true);
}
-static int copy_page(int ufd, unsigned long offset)
-{
- return __copy_page(ufd, offset, false);
-}
-
-static int uffd_read_msg(int ufd, struct uffd_msg *msg)
-{
- int ret = read(uffd, msg, sizeof(*msg));
-
- if (ret != sizeof(*msg)) {
- if (ret < 0) {
- if (errno == EAGAIN || errno == EINTR)
- return 1;
- err("blocking read error");
- } else {
- err("short read");
- }
- }
-
- return 0;
-}
-
-static void uffd_handle_page_fault(struct uffd_msg *msg,
- struct uffd_stats *stats)
-{
- unsigned long offset;
-
- if (msg->event != UFFD_EVENT_PAGEFAULT)
- err("unexpected msg event %u", msg->event);
-
- if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
- /* Write protect page faults */
- wp_range(uffd, msg->arg.pagefault.address, page_size, false);
- stats->wp_faults++;
- } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
- uint8_t *area;
- int b;
-
- /*
- * Minor page faults
- *
- * To prove we can modify the original range for testing
- * purposes, we're going to bit flip this range before
- * continuing.
- *
- * Note that this requires all minor page fault tests operate on
- * area_dst (non-UFFD-registered) and area_dst_alias
- * (UFFD-registered).
- */
-
- area = (uint8_t *)(area_dst +
- ((char *)msg->arg.pagefault.address -
- area_dst_alias));
- for (b = 0; b < page_size; ++b)
- area[b] = ~area[b];
- continue_range(uffd, msg->arg.pagefault.address, page_size);
- stats->minor_faults++;
- } else {
- /*
- * Missing page faults.
- *
- * Here we force a write check for each of the missing mode
- * faults. It's guaranteed because the only threads that
- * will trigger uffd faults are the locking threads, and
- * their first instruction to touch the missing page will
- * always be pthread_mutex_lock().
- *
- * Note that here we relied on an NPTL glibc impl detail to
- * always read the lock type at the entry of the lock op
- * (pthread_mutex_t.__data.__type, offset 0x10) before
- * doing any locking operations to guarantee that. It's
- * actually not good to rely on this impl detail because
- * logically a pthread-compatible lib can implement the
- * locks without types and we can fail when linking with
- * them. However since we used to find bugs with this
- * strict check we still keep it around. Hopefully this
- * could be a good hint when it fails again. If one day
- * it'll break on some other impl of glibc we'll revisit.
- */
- if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
- err("unexpected write fault");
-
- offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
- offset &= ~(page_size-1);
-
- if (copy_page(uffd, offset))
- stats->missing_faults++;
- }
-}
-
-static void *uffd_poll_thread(void *arg)
-{
- struct uffd_stats *stats = (struct uffd_stats *)arg;
- unsigned long cpu = stats->cpu;
- struct pollfd pollfd[2];
- struct uffd_msg msg;
- struct uffdio_register uffd_reg;
- int ret;
- char tmp_chr;
-
- pollfd[0].fd = uffd;
- pollfd[0].events = POLLIN;
- pollfd[1].fd = pipefd[cpu*2];
- pollfd[1].events = POLLIN;
-
- for (;;) {
- ret = poll(pollfd, 2, -1);
- if (ret <= 0) {
- if (errno == EINTR || errno == EAGAIN)
- continue;
- err("poll error: %d", ret);
- }
- if (pollfd[1].revents & POLLIN) {
- if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
- err("read pipefd error");
- break;
- }
- if (!(pollfd[0].revents & POLLIN))
- err("pollfd[0].revents %d", pollfd[0].revents);
- if (uffd_read_msg(uffd, &msg))
- continue;
- switch (msg.event) {
- default:
- err("unexpected msg event %u\n", msg.event);
- break;
- case UFFD_EVENT_PAGEFAULT:
- uffd_handle_page_fault(&msg, stats);
- break;
- case UFFD_EVENT_FORK:
- close(uffd);
- uffd = msg.arg.fork.ufd;
- pollfd[0].fd = uffd;
- break;
- case UFFD_EVENT_REMOVE:
- uffd_reg.range.start = msg.arg.remove.start;
- uffd_reg.range.len = msg.arg.remove.end -
- msg.arg.remove.start;
- if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
- err("remove failure");
- break;
- case UFFD_EVENT_REMAP:
- area_remap = area_dst; /* save for later unmap */
- area_dst = (char *)(unsigned long)msg.arg.remap.to;
- break;
- }
- }
-
- return NULL;
-}
-
pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
static void *uffd_read_thread(void *arg)
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch]
2023-04-12 16:42 ` [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch] Peter Xu
@ 2023-04-12 17:59 ` Axel Rasmussen
0 siblings, 0 replies; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 17:59 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos, Mike Rapoport
On Wed, Apr 12, 2023 at 9:42 AM Peter Xu <peterx@redhat.com> wrote:
>
> Move common utility functions into uffd-common.[ch] files from the original
> userfaultfd.c. This prepares for a split of userfaultfd.c into two tests:
> one to only cover the old but powerful stress test, the other one covers
> all the functional tests.
>
> This movement is kind of a brute-force effort for now, with light touch-ups
> but nothing should really change. There's chances to optimize more, but
> let's leave that for later.
>
> Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
> ---
> tools/testing/selftests/mm/Makefile | 2 +
> tools/testing/selftests/mm/uffd-common.c | 611 ++++++++++++++++++++
> tools/testing/selftests/mm/uffd-common.h | 117 ++++
> tools/testing/selftests/mm/userfaultfd.c | 694 +----------------------
> 4 files changed, 731 insertions(+), 693 deletions(-)
> create mode 100644 tools/testing/selftests/mm/uffd-common.c
> create mode 100644 tools/testing/selftests/mm/uffd-common.h
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 5f7626550e5f..36467c15ca00 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -108,6 +108,8 @@ include ../lib.mk
>
> $(TEST_GEN_PROGS): vm_util.c
>
> +$(OUTPUT)/userfaultfd: uffd-common.c
> +
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64))
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> new file mode 100644
> index 000000000000..c57757c2a36f
> --- /dev/null
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -0,0 +1,611 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Userfaultfd tests util functions
> + *
> + * Copyright (C) 2015-2023 Red Hat, Inc.
> + */
> +
> +#include "uffd-common.h"
> +
> +#define BASE_PMD_ADDR ((void *)(1UL << 30))
> +
> +volatile bool test_uffdio_copy_eexist = true;
> +unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> +char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> +int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
> +bool map_shared, test_collapse, test_dev_userfaultfd;
> +bool test_uffdio_wp = true, test_uffdio_minor = false;
> +unsigned long long *count_verify;
> +uffd_test_ops_t *uffd_test_ops;
> +
> +static void anon_release_pages(char *rel_area)
> +{
> + if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
> + err("madvise(MADV_DONTNEED) failed");
> +}
> +
> +static void anon_allocate_area(void **alloc_area, bool is_src)
> +{
> + *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> + MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
> +}
> +
> +static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> +{
> +}
> +
> +static void hugetlb_release_pages(char *rel_area)
> +{
> + if (!map_shared) {
> + if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
> + err("madvise(MADV_DONTNEED) failed");
> + } else {
> + if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
> + err("madvise(MADV_REMOVE) failed");
> + }
> +}
> +
> +static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> +{
> + off_t size = nr_pages * page_size;
> + off_t offset = is_src ? 0 : size;
> + void *area_alias = NULL;
> + char **alloc_area_alias;
> +
> + *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
> + (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> + (is_src ? 0 : MAP_NORESERVE),
> + mem_fd, offset);
> + if (*alloc_area == MAP_FAILED)
> + err("mmap of hugetlbfs file failed");
> +
> + if (map_shared) {
> + area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
> + MAP_SHARED, mem_fd, offset);
> + if (area_alias == MAP_FAILED)
> + err("mmap of hugetlb file alias failed");
> + }
> +
> + if (is_src) {
> + alloc_area_alias = &area_src_alias;
> + } else {
> + alloc_area_alias = &area_dst_alias;
> + }
> + if (area_alias)
> + *alloc_area_alias = area_alias;
> +}
> +
> +static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> +{
> + if (!map_shared)
> + return;
> +
> + *start = (unsigned long) area_dst_alias + offset;
> +}
> +
> +static void shmem_release_pages(char *rel_area)
> +{
> + if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
> + err("madvise(MADV_REMOVE) failed");
> +}
> +
> +static void shmem_allocate_area(void **alloc_area, bool is_src)
> +{
> + void *area_alias = NULL;
> + size_t bytes = nr_pages * page_size;
> + unsigned long offset = is_src ? 0 : bytes;
> + char *p = NULL, *p_alias = NULL;
> +
> + if (test_collapse) {
> + p = BASE_PMD_ADDR;
> + if (!is_src)
> + /* src map + alias + interleaved hpages */
> + p += 2 * (bytes + hpage_size);
> + p_alias = p;
> + p_alias += bytes;
> + p_alias += hpage_size; /* Prevent src/dst VMA merge */
> + }
> +
> + *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
> + mem_fd, offset);
> + if (*alloc_area == MAP_FAILED)
> + err("mmap of memfd failed");
> + if (test_collapse && *alloc_area != p)
> + err("mmap of memfd failed at %p", p);
> +
> + area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
> + mem_fd, offset);
> + if (area_alias == MAP_FAILED)
> + err("mmap of memfd alias failed");
> + if (test_collapse && area_alias != p_alias)
> + err("mmap of anonymous memory failed at %p", p_alias);
> +
> + if (is_src)
> + area_src_alias = area_alias;
> + else
> + area_dst_alias = area_alias;
> +}
> +
> +static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> +{
> + *start = (unsigned long)area_dst_alias + offset;
> +}
> +
> +static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
> +{
> + if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
> + err("Did not find expected %d number of hugepages",
> + expect_nr_hpages);
> +}
> +
> +struct uffd_test_ops anon_uffd_test_ops = {
> + .allocate_area = anon_allocate_area,
> + .release_pages = anon_release_pages,
> + .alias_mapping = noop_alias_mapping,
> + .check_pmd_mapping = NULL,
> +};
> +
> +struct uffd_test_ops shmem_uffd_test_ops = {
> + .allocate_area = shmem_allocate_area,
> + .release_pages = shmem_release_pages,
> + .alias_mapping = shmem_alias_mapping,
> + .check_pmd_mapping = shmem_check_pmd_mapping,
> +};
> +
> +struct uffd_test_ops hugetlb_uffd_test_ops = {
> + .allocate_area = hugetlb_allocate_area,
> + .release_pages = hugetlb_release_pages,
> + .alias_mapping = hugetlb_alias_mapping,
> + .check_pmd_mapping = NULL,
> +};
> +
> +void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
> +{
> + int i;
> + unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
> +
> + for (i = 0; i < n_cpus; i++) {
> + miss_total += stats[i].missing_faults;
> + wp_total += stats[i].wp_faults;
> + minor_total += stats[i].minor_faults;
> + }
> +
> + printf("userfaults: ");
> + if (miss_total) {
> + printf("%llu missing (", miss_total);
> + for (i = 0; i < n_cpus; i++)
> + printf("%lu+", stats[i].missing_faults);
> + printf("\b) ");
> + }
> + if (wp_total) {
> + printf("%llu wp (", wp_total);
> + for (i = 0; i < n_cpus; i++)
> + printf("%lu+", stats[i].wp_faults);
> + printf("\b) ");
> + }
> + if (minor_total) {
> + printf("%llu minor (", minor_total);
> + for (i = 0; i < n_cpus; i++)
> + printf("%lu+", stats[i].minor_faults);
> + printf("\b)");
> + }
> + printf("\n");
> +}
> +
> +static int __userfaultfd_open_dev(void)
> +{
> + int fd, _uffd;
> +
> + fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> + if (fd < 0)
> + errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
> +
> + _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
> + if (_uffd < 0)
> + errexit(errno == ENOTTY ? KSFT_SKIP : 1,
> + "creating userfaultfd failed");
> + close(fd);
> + return _uffd;
> +}
> +
> +void userfaultfd_open(uint64_t *features)
> +{
> + struct uffdio_api uffdio_api;
> +
> + if (test_dev_userfaultfd)
> + uffd = __userfaultfd_open_dev();
> + else {
> + uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
> + if (uffd < 0)
> + errexit(errno == ENOSYS ? KSFT_SKIP : 1,
> + "creating userfaultfd failed");
> + }
> + uffd_flags = fcntl(uffd, F_GETFD, NULL);
> +
> + uffdio_api.api = UFFD_API;
> + uffdio_api.features = *features;
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api))
> + err("UFFDIO_API failed.\nPlease make sure to "
> + "run with either root or ptrace capability.");
> + if (uffdio_api.api != UFFD_API)
> + err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
> +
> + *features = uffdio_api.features;
> +}
> +
> +static inline void munmap_area(void **area)
> +{
> + if (*area)
> + if (munmap(*area, nr_pages * page_size))
> + err("munmap");
> +
> + *area = NULL;
> +}
> +
> +static void uffd_test_ctx_clear(void)
> +{
> + size_t i;
> +
> + if (pipefd) {
> + for (i = 0; i < nr_cpus * 2; ++i) {
> + if (close(pipefd[i]))
> + err("close pipefd");
> + }
> + free(pipefd);
> + pipefd = NULL;
> + }
> +
> + if (count_verify) {
> + free(count_verify);
> + count_verify = NULL;
> + }
> +
> + if (uffd != -1) {
> + if (close(uffd))
> + err("close uffd");
> + uffd = -1;
> + }
> +
> + munmap_area((void **)&area_src);
> + munmap_area((void **)&area_src_alias);
> + munmap_area((void **)&area_dst);
> + munmap_area((void **)&area_dst_alias);
> + munmap_area((void **)&area_remap);
> +}
> +
> +void uffd_test_ctx_init(uint64_t features)
> +{
> + unsigned long nr, cpu;
> +
> + uffd_test_ctx_clear();
> +
> + uffd_test_ops->allocate_area((void **)&area_src, true);
> + uffd_test_ops->allocate_area((void **)&area_dst, false);
> +
> + userfaultfd_open(&features);
> +
> + count_verify = malloc(nr_pages * sizeof(unsigned long long));
> + if (!count_verify)
> + err("count_verify");
> +
> + for (nr = 0; nr < nr_pages; nr++) {
> + *area_mutex(area_src, nr) =
> + (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
> + count_verify[nr] = *area_count(area_src, nr) = 1;
> + /*
> + * In the transition between 255 to 256, powerpc will
> + * read out of order in my_bcmp and see both bytes as
> + * zero, so leave a placeholder below always non-zero
> + * after the count, to avoid my_bcmp to trigger false
> + * positives.
> + */
> + *(area_count(area_src, nr) + 1) = 1;
> + }
> +
> + /*
> + * After initialization of area_src, we must explicitly release pages
> + * for area_dst to make sure it's fully empty. Otherwise we could have
> + * some area_dst pages be errornously initialized with zero pages,
> + * hence we could hit memory corruption later in the test.
> + *
> + * One example is when THP is globally enabled, above allocate_area()
> + * calls could have the two areas merged into a single VMA (as they
> + * will have the same VMA flags so they're mergeable). When we
> + * initialize the area_src above, it's possible that some part of
> + * area_dst could have been faulted in via one huge THP that will be
> + * shared between area_src and area_dst. It could cause some of the
> + * area_dst won't be trapped by missing userfaults.
> + *
> + * This release_pages() will guarantee even if that happened, we'll
> + * proactively split the thp and drop any accidentally initialized
> + * pages within area_dst.
> + */
> + uffd_test_ops->release_pages(area_dst);
> +
> + pipefd = malloc(sizeof(int) * nr_cpus * 2);
> + if (!pipefd)
> + err("pipefd");
> + for (cpu = 0; cpu < nr_cpus; cpu++)
> + if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
> + err("pipe");
> +}
> +
> +uint64_t get_expected_ioctls(uint64_t mode)
> +{
> + uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
> +
> + if (test_type == TEST_HUGETLB)
> + ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
> +
> + if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
> + ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
> +
> + if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
> + ioctls &= ~(1 << _UFFDIO_CONTINUE);
> +
> + return ioctls;
> +}
> +
> +void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
> +{
> + uint64_t expected = get_expected_ioctls(mode);
> + uint64_t actual = ioctls & expected;
> +
> + if (actual != expected) {
> + err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
> + expected, actual);
> + }
> +}
> +
> +void wp_range(int ufd, __u64 start, __u64 len, bool wp)
> +{
> + struct uffdio_writeprotect prms;
> +
> + /* Write protection page faults */
> + prms.range.start = start;
> + prms.range.len = len;
> + /* Undo write-protect, do wakeup after that */
> + prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
> +
> + if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
> + err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
> +}
> +
> +static void continue_range(int ufd, __u64 start, __u64 len)
> +{
> + struct uffdio_continue req;
> + int ret;
> +
> + req.range.start = start;
> + req.range.len = len;
> + req.mode = 0;
> + if (test_uffdio_wp)
> + req.mode |= UFFDIO_CONTINUE_MODE_WP;
> +
> + if (ioctl(ufd, UFFDIO_CONTINUE, &req))
> + err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
> + (uint64_t)start);
> +
> + /*
> + * Error handling within the kernel for continue is subtly different
> + * from copy or zeropage, so it may be a source of bugs. Trigger an
> + * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
> + */
> + req.mapped = 0;
> + ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
> + if (ret >= 0 || req.mapped != -EEXIST)
> + err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
> + ret, (int64_t) req.mapped);
> +}
> +
> +int uffd_read_msg(int ufd, struct uffd_msg *msg)
> +{
> + int ret = read(uffd, msg, sizeof(*msg));
> +
> + if (ret != sizeof(*msg)) {
> + if (ret < 0) {
> + if (errno == EAGAIN || errno == EINTR)
> + return 1;
> + err("blocking read error");
> + } else {
> + err("short read");
> + }
> + }
> +
> + return 0;
> +}
> +
> +void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> +{
> + unsigned long offset;
> +
> + if (msg->event != UFFD_EVENT_PAGEFAULT)
> + err("unexpected msg event %u", msg->event);
> +
> + if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
> + /* Write protect page faults */
> + wp_range(uffd, msg->arg.pagefault.address, page_size, false);
> + stats->wp_faults++;
> + } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
> + uint8_t *area;
> + int b;
> +
> + /*
> + * Minor page faults
> + *
> + * To prove we can modify the original range for testing
> + * purposes, we're going to bit flip this range before
> + * continuing.
> + *
> + * Note that this requires all minor page fault tests operate on
> + * area_dst (non-UFFD-registered) and area_dst_alias
> + * (UFFD-registered).
> + */
> +
> + area = (uint8_t *)(area_dst +
> + ((char *)msg->arg.pagefault.address -
> + area_dst_alias));
> + for (b = 0; b < page_size; ++b)
> + area[b] = ~area[b];
> + continue_range(uffd, msg->arg.pagefault.address, page_size);
> + stats->minor_faults++;
> + } else {
> + /*
> + * Missing page faults.
> + *
> + * Here we force a write check for each of the missing mode
> + * faults. It's guaranteed because the only threads that
> + * will trigger uffd faults are the locking threads, and
> + * their first instruction to touch the missing page will
> + * always be pthread_mutex_lock().
> + *
> + * Note that here we relied on an NPTL glibc impl detail to
> + * always read the lock type at the entry of the lock op
> + * (pthread_mutex_t.__data.__type, offset 0x10) before
> + * doing any locking operations to guarantee that. It's
> + * actually not good to rely on this impl detail because
> + * logically a pthread-compatible lib can implement the
> + * locks without types and we can fail when linking with
> + * them. However since we used to find bugs with this
> + * strict check we still keep it around. Hopefully this
> + * could be a good hint when it fails again. If one day
> + * it'll break on some other impl of glibc we'll revisit.
> + */
> + if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
> + err("unexpected write fault");
> +
> + offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
> + offset &= ~(page_size-1);
> +
> + if (copy_page(uffd, offset))
> + stats->missing_faults++;
> + }
> +}
> +
> +void *uffd_poll_thread(void *arg)
> +{
> + struct uffd_stats *stats = (struct uffd_stats *)arg;
> + unsigned long cpu = stats->cpu;
> + struct pollfd pollfd[2];
> + struct uffd_msg msg;
> + struct uffdio_register uffd_reg;
> + int ret;
> + char tmp_chr;
> +
> + pollfd[0].fd = uffd;
> + pollfd[0].events = POLLIN;
> + pollfd[1].fd = pipefd[cpu*2];
> + pollfd[1].events = POLLIN;
> +
> + for (;;) {
> + ret = poll(pollfd, 2, -1);
> + if (ret <= 0) {
> + if (errno == EINTR || errno == EAGAIN)
> + continue;
> + err("poll error: %d", ret);
> + }
> + if (pollfd[1].revents) {
> + if (!(pollfd[1].revents & POLLIN))
> + err("pollfd[1].revents %d", pollfd[1].revents);
> + if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
> + err("read pipefd error");
> + break;
> + }
> + if (!(pollfd[0].revents & POLLIN))
> + err("pollfd[0].revents %d", pollfd[0].revents);
> + if (uffd_read_msg(uffd, &msg))
> + continue;
> + switch (msg.event) {
> + default:
> + err("unexpected msg event %u\n", msg.event);
> + break;
> + case UFFD_EVENT_PAGEFAULT:
> + uffd_handle_page_fault(&msg, stats);
> + break;
> + case UFFD_EVENT_FORK:
> + close(uffd);
> + uffd = msg.arg.fork.ufd;
> + pollfd[0].fd = uffd;
> + break;
> + case UFFD_EVENT_REMOVE:
> + uffd_reg.range.start = msg.arg.remove.start;
> + uffd_reg.range.len = msg.arg.remove.end -
> + msg.arg.remove.start;
> + if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
> + err("remove failure");
> + break;
> + case UFFD_EVENT_REMAP:
> + area_remap = area_dst; /* save for later unmap */
> + area_dst = (char *)(unsigned long)msg.arg.remap.to;
> + break;
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
> + unsigned long offset)
> +{
> + uffd_test_ops->alias_mapping(&uffdio_copy->dst,
> + uffdio_copy->len,
> + offset);
> + if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
> + /* real retval in ufdio_copy.copy */
> + if (uffdio_copy->copy != -EEXIST)
> + err("UFFDIO_COPY retry error: %"PRId64,
> + (int64_t)uffdio_copy->copy);
> + } else {
> + err("UFFDIO_COPY retry unexpected: %"PRId64,
> + (int64_t)uffdio_copy->copy);
> + }
> +}
> +
> +static void wake_range(int ufd, unsigned long addr, unsigned long len)
> +{
> + struct uffdio_range uffdio_wake;
> +
> + uffdio_wake.start = addr;
> + uffdio_wake.len = len;
> +
> + if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
> + fprintf(stderr, "error waking %lu\n",
> + addr), exit(1);
> +}
> +
> +int __copy_page(int ufd, unsigned long offset, bool retry)
> +{
> + struct uffdio_copy uffdio_copy;
> +
> + if (offset >= nr_pages * page_size)
> + err("unexpected offset %lu\n", offset);
> + uffdio_copy.dst = (unsigned long) area_dst + offset;
> + uffdio_copy.src = (unsigned long) area_src + offset;
> + uffdio_copy.len = page_size;
> + if (test_uffdio_wp)
> + uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
> + else
> + uffdio_copy.mode = 0;
> + uffdio_copy.copy = 0;
> + if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
> + /* real retval in ufdio_copy.copy */
> + if (uffdio_copy.copy != -EEXIST)
> + err("UFFDIO_COPY error: %"PRId64,
> + (int64_t)uffdio_copy.copy);
> + wake_range(ufd, uffdio_copy.dst, page_size);
> + } else if (uffdio_copy.copy != page_size) {
> + err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
> + } else {
> + if (test_uffdio_copy_eexist && retry) {
> + test_uffdio_copy_eexist = false;
> + retry_copy_page(ufd, &uffdio_copy, offset);
> + }
> + return 1;
> + }
> + return 0;
> +}
> +
> +int copy_page(int ufd, unsigned long offset)
> +{
> + return __copy_page(ufd, offset, false);
> +}
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> new file mode 100644
> index 000000000000..d9430cfdcb19
> --- /dev/null
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -0,0 +1,117 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Userfaultfd tests common header
> + *
> + * Copyright (C) 2015-2023 Red Hat, Inc.
> + */
> +#ifndef __UFFD_COMMON_H__
> +#define __UFFD_COMMON_H__
> +
> +#define _GNU_SOURCE
> +#include <stdio.h>
> +#include <errno.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <time.h>
> +#include <signal.h>
> +#include <poll.h>
> +#include <string.h>
> +#include <linux/mman.h>
> +#include <sys/mman.h>
> +#include <sys/syscall.h>
> +#include <sys/ioctl.h>
> +#include <sys/wait.h>
> +#include <pthread.h>
> +#include <linux/userfaultfd.h>
> +#include <setjmp.h>
> +#include <stdbool.h>
> +#include <assert.h>
> +#include <inttypes.h>
> +#include <stdint.h>
> +#include <sys/random.h>
> +
> +#include "../kselftest.h"
> +#include "vm_util.h"
> +
> +#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
> +
> +#define _err(fmt, ...) \
> + do { \
> + int ret = errno; \
> + fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
> + fprintf(stderr, " (errno=%d, @%s:%d)\n", \
> + ret, __FILE__, __LINE__); \
> + } while (0)
> +
> +#define errexit(exitcode, fmt, ...) \
> + do { \
> + _err(fmt, ##__VA_ARGS__); \
> + exit(exitcode); \
> + } while (0)
> +
> +#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
> +
> +/* pthread_mutex_t starts at page offset 0 */
> +#define area_mutex(___area, ___nr) \
> + ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
> +/*
> + * count is placed in the page after pthread_mutex_t naturally aligned
> + * to avoid non alignment faults on non-x86 archs.
> + */
> +#define area_count(___area, ___nr) \
> + ((volatile unsigned long long *) ((unsigned long) \
> + ((___area) + (___nr)*page_size + \
> + sizeof(pthread_mutex_t) + \
> + sizeof(unsigned long long) - 1) & \
> + ~(unsigned long)(sizeof(unsigned long long) \
> + - 1)))
> +
> +/* Userfaultfd test statistics */
> +struct uffd_stats {
> + int cpu;
> + unsigned long missing_faults;
> + unsigned long wp_faults;
> + unsigned long minor_faults;
> +};
> +
> +struct uffd_test_ops {
> + void (*allocate_area)(void **alloc_area, bool is_src);
> + void (*release_pages)(char *rel_area);
> + void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
> + void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
> +};
> +typedef struct uffd_test_ops uffd_test_ops_t;
> +
> +extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> +extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> +extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
> +extern bool map_shared, test_collapse, test_dev_userfaultfd;
> +extern bool test_uffdio_wp, test_uffdio_minor;
> +extern unsigned long long *count_verify;
> +extern volatile bool test_uffdio_copy_eexist;
> +
> +extern uffd_test_ops_t anon_uffd_test_ops;
> +extern uffd_test_ops_t shmem_uffd_test_ops;
> +extern uffd_test_ops_t hugetlb_uffd_test_ops;
> +extern uffd_test_ops_t *uffd_test_ops;
> +
> +void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
> +void uffd_test_ctx_init(uint64_t features);
> +void userfaultfd_open(uint64_t *features);
> +uint64_t get_expected_ioctls(uint64_t mode);
> +void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
> +int uffd_read_msg(int ufd, struct uffd_msg *msg);
> +void wp_range(int ufd, __u64 start, __u64 len, bool wp);
> +void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
> +int __copy_page(int ufd, unsigned long offset, bool retry);
> +int copy_page(int ufd, unsigned long offset);
> +void *uffd_poll_thread(void *arg);
> +
> +#define TEST_ANON 1
> +#define TEST_HUGETLB 2
> +#define TEST_SHMEM 3
> +
> +#endif
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/userfaultfd.c
> index 3487ec0bfcc8..c68a9aeefc41 100644
> --- a/tools/testing/selftests/mm/userfaultfd.c
> +++ b/tools/testing/selftests/mm/userfaultfd.c
> @@ -34,96 +34,20 @@
> * transfer (UFFDIO_COPY).
> */
>
> -#define _GNU_SOURCE
> -#include <stdio.h>
> -#include <errno.h>
> -#include <unistd.h>
> -#include <stdlib.h>
> -#include <sys/types.h>
> -#include <sys/stat.h>
> -#include <fcntl.h>
> -#include <time.h>
> -#include <signal.h>
> -#include <poll.h>
> -#include <string.h>
> -#include <linux/mman.h>
> -#include <sys/mman.h>
> -#include <sys/syscall.h>
> -#include <sys/ioctl.h>
> -#include <sys/wait.h>
> -#include <pthread.h>
> -#include <linux/userfaultfd.h>
> -#include <setjmp.h>
> -#include <stdbool.h>
> -#include <assert.h>
> -#include <inttypes.h>
> -#include <stdint.h>
> -#include <sys/random.h>
> -
> -#include "../kselftest.h"
> -#include "vm_util.h"
> +#include "uffd-common.h"
>
> #ifdef __NR_userfaultfd
>
> -static unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
> -
> #define BOUNCE_RANDOM (1<<0)
> #define BOUNCE_RACINGFAULTS (1<<1)
> #define BOUNCE_VERIFY (1<<2)
> #define BOUNCE_POLL (1<<3)
> static int bounces;
>
> -#define TEST_ANON 1
> -#define TEST_HUGETLB 2
> -#define TEST_SHMEM 3
> -static int test_type;
> -
> -#define UFFD_FLAGS (O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY)
> -
> -#define BASE_PMD_ADDR ((void *)(1UL << 30))
> -
> -/* test using /dev/userfaultfd, instead of userfaultfd(2) */
> -static bool test_dev_userfaultfd;
> -
> /* exercise the test_uffdio_*_eexist every ALARM_INTERVAL_SECS */
> #define ALARM_INTERVAL_SECS 10
> -static volatile bool test_uffdio_copy_eexist = true;
> -/* Whether to test uffd write-protection */
> -static bool test_uffdio_wp = true;
> -/* Whether to test uffd minor faults */
> -static bool test_uffdio_minor = false;
> -static bool map_shared;
> -static int mem_fd;
> -static unsigned long long *count_verify;
> -static int uffd = -1;
> -static int uffd_flags, finished, *pipefd;
> -static char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
> static char *zeropage;
> pthread_attr_t attr;
> -static bool test_collapse;
> -
> -/* Userfaultfd test statistics */
> -struct uffd_stats {
> - int cpu;
> - unsigned long missing_faults;
> - unsigned long wp_faults;
> - unsigned long minor_faults;
> -};
> -
> -/* pthread_mutex_t starts at page offset 0 */
> -#define area_mutex(___area, ___nr) \
> - ((pthread_mutex_t *) ((___area) + (___nr)*page_size))
> -/*
> - * count is placed in the page after pthread_mutex_t naturally aligned
> - * to avoid non alignment faults on non-x86 archs.
> - */
> -#define area_count(___area, ___nr) \
> - ((volatile unsigned long long *) ((unsigned long) \
> - ((___area) + (___nr)*page_size + \
> - sizeof(pthread_mutex_t) + \
> - sizeof(unsigned long long) - 1) & \
> - ~(unsigned long)(sizeof(unsigned long long) \
> - - 1)))
>
> #define swap(a, b) \
> do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
> @@ -166,22 +90,6 @@ static void usage(void)
> exit(1);
> }
>
> -#define _err(fmt, ...) \
> - do { \
> - int ret = errno; \
> - fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \
> - fprintf(stderr, " (errno=%d, line=%d)\n", \
> - ret, __LINE__); \
> - } while (0)
> -
> -#define errexit(exitcode, fmt, ...) \
> - do { \
> - _err(fmt, ##__VA_ARGS__); \
> - exit(exitcode); \
> - } while (0)
> -
> -#define err(fmt, ...) errexit(1, fmt, ##__VA_ARGS__)
> -
> static void uffd_stats_reset(struct uffd_stats *uffd_stats,
> unsigned long n_cpus)
> {
> @@ -195,189 +103,6 @@ static void uffd_stats_reset(struct uffd_stats *uffd_stats,
> }
> }
>
> -static void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
> -{
> - int i;
> - unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
> -
> - for (i = 0; i < n_cpus; i++) {
> - miss_total += stats[i].missing_faults;
> - wp_total += stats[i].wp_faults;
> - minor_total += stats[i].minor_faults;
> - }
> -
> - printf("userfaults: ");
> - if (miss_total) {
> - printf("%llu missing (", miss_total);
> - for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].missing_faults);
> - printf("\b) ");
> - }
> - if (wp_total) {
> - printf("%llu wp (", wp_total);
> - for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].wp_faults);
> - printf("\b) ");
> - }
> - if (minor_total) {
> - printf("%llu minor (", minor_total);
> - for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].minor_faults);
> - printf("\b)");
> - }
> - printf("\n");
> -}
> -
> -static void anon_release_pages(char *rel_area)
> -{
> - if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
> - err("madvise(MADV_DONTNEED) failed");
> -}
> -
> -static void anon_allocate_area(void **alloc_area, bool is_src)
> -{
> - *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> - MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
> -}
> -
> -static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> -{
> -}
> -
> -static void hugetlb_release_pages(char *rel_area)
> -{
> - if (!map_shared) {
> - if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
> - err("madvise(MADV_DONTNEED) failed");
> - } else {
> - if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
> - err("madvise(MADV_REMOVE) failed");
> - }
> -}
> -
> -static void hugetlb_allocate_area(void **alloc_area, bool is_src)
> -{
> - off_t size = nr_pages * page_size;
> - off_t offset = is_src ? 0 : size;
> - void *area_alias = NULL;
> - char **alloc_area_alias;
> -
> - *alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
> - (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> - (is_src ? 0 : MAP_NORESERVE),
> - mem_fd, offset);
> - if (*alloc_area == MAP_FAILED)
> - err("mmap of hugetlbfs file failed");
> -
> - if (map_shared) {
> - area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
> - MAP_SHARED, mem_fd, offset);
> - if (area_alias == MAP_FAILED)
> - err("mmap of hugetlb file alias failed");
> - }
> -
> - if (is_src) {
> - alloc_area_alias = &area_src_alias;
> - } else {
> - alloc_area_alias = &area_dst_alias;
> - }
> - if (area_alias)
> - *alloc_area_alias = area_alias;
> -}
> -
> -static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> -{
> - if (!map_shared)
> - return;
> -
> - *start = (unsigned long) area_dst_alias + offset;
> -}
> -
> -static void shmem_release_pages(char *rel_area)
> -{
> - if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE))
> - err("madvise(MADV_REMOVE) failed");
> -}
> -
> -static void shmem_allocate_area(void **alloc_area, bool is_src)
> -{
> - void *area_alias = NULL;
> - size_t bytes = nr_pages * page_size;
> - unsigned long offset = is_src ? 0 : bytes;
> - char *p = NULL, *p_alias = NULL;
> -
> - if (test_collapse) {
> - p = BASE_PMD_ADDR;
> - if (!is_src)
> - /* src map + alias + interleaved hpages */
> - p += 2 * (bytes + hpage_size);
> - p_alias = p;
> - p_alias += bytes;
> - p_alias += hpage_size; /* Prevent src/dst VMA merge */
> - }
> -
> - *alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
> - mem_fd, offset);
> - if (*alloc_area == MAP_FAILED)
> - err("mmap of memfd failed");
> - if (test_collapse && *alloc_area != p)
> - err("mmap of memfd failed at %p", p);
> -
> - area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
> - mem_fd, offset);
> - if (area_alias == MAP_FAILED)
> - err("mmap of memfd alias failed");
> - if (test_collapse && area_alias != p_alias)
> - err("mmap of anonymous memory failed at %p", p_alias);
> -
> - if (is_src)
> - area_src_alias = area_alias;
> - else
> - area_dst_alias = area_alias;
> -}
> -
> -static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
> -{
> - *start = (unsigned long)area_dst_alias + offset;
> -}
> -
> -static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
> -{
> - if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
> - err("Did not find expected %d number of hugepages",
> - expect_nr_hpages);
> -}
> -
> -struct uffd_test_ops {
> - void (*allocate_area)(void **alloc_area, bool is_src);
> - void (*release_pages)(char *rel_area);
> - void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
> - void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
> -};
> -
> -static struct uffd_test_ops anon_uffd_test_ops = {
> - .allocate_area = anon_allocate_area,
> - .release_pages = anon_release_pages,
> - .alias_mapping = noop_alias_mapping,
> - .check_pmd_mapping = NULL,
> -};
> -
> -static struct uffd_test_ops shmem_uffd_test_ops = {
> - .allocate_area = shmem_allocate_area,
> - .release_pages = shmem_release_pages,
> - .alias_mapping = shmem_alias_mapping,
> - .check_pmd_mapping = shmem_check_pmd_mapping,
> -};
> -
> -static struct uffd_test_ops hugetlb_uffd_test_ops = {
> - .allocate_area = hugetlb_allocate_area,
> - .release_pages = hugetlb_release_pages,
> - .alias_mapping = hugetlb_alias_mapping,
> - .check_pmd_mapping = NULL,
> -};
> -
> -static struct uffd_test_ops *uffd_test_ops;
> -
> static inline uint64_t uffd_minor_feature(void)
> {
> if (test_type == TEST_HUGETLB && map_shared)
> @@ -388,171 +113,6 @@ static inline uint64_t uffd_minor_feature(void)
> return 0;
> }
>
> -static uint64_t get_expected_ioctls(uint64_t mode)
> -{
> - uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
> -
> - if (test_type == TEST_HUGETLB)
> - ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
> - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
> - ioctls &= ~(1 << _UFFDIO_CONTINUE);
> -
> - return ioctls;
> -}
> -
> -static void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
> -{
> - uint64_t expected = get_expected_ioctls(mode);
> - uint64_t actual = ioctls & expected;
> -
> - if (actual != expected) {
> - err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
> - expected, actual);
> - }
> -}
> -
> -static int __userfaultfd_open_dev(void)
> -{
> - int fd, _uffd;
> -
> - fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> - if (fd < 0)
> - errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
> -
> - _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
> - if (_uffd < 0)
> - errexit(errno == ENOTTY ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - close(fd);
> - return _uffd;
> -}
> -
> -static void userfaultfd_open(uint64_t *features)
> -{
> - struct uffdio_api uffdio_api;
> -
> - if (test_dev_userfaultfd)
> - uffd = __userfaultfd_open_dev();
> - else {
> - uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
> - if (uffd < 0)
> - errexit(errno == ENOSYS ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - }
> - uffd_flags = fcntl(uffd, F_GETFD, NULL);
> -
> - uffdio_api.api = UFFD_API;
> - uffdio_api.features = *features;
> - if (ioctl(uffd, UFFDIO_API, &uffdio_api))
> - err("UFFDIO_API failed.\nPlease make sure to "
> - "run with either root or ptrace capability.");
> - if (uffdio_api.api != UFFD_API)
> - err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
> -
> - *features = uffdio_api.features;
> -}
> -
> -static inline void munmap_area(void **area)
> -{
> - if (*area)
> - if (munmap(*area, nr_pages * page_size))
> - err("munmap");
> -
> - *area = NULL;
> -}
> -
> -static void uffd_test_ctx_clear(void)
> -{
> - size_t i;
> -
> - if (pipefd) {
> - for (i = 0; i < nr_cpus * 2; ++i) {
> - if (close(pipefd[i]))
> - err("close pipefd");
> - }
> - free(pipefd);
> - pipefd = NULL;
> - }
> -
> - if (count_verify) {
> - free(count_verify);
> - count_verify = NULL;
> - }
> -
> - if (uffd != -1) {
> - if (close(uffd))
> - err("close uffd");
> - uffd = -1;
> - }
> -
> - munmap_area((void **)&area_src);
> - munmap_area((void **)&area_src_alias);
> - munmap_area((void **)&area_dst);
> - munmap_area((void **)&area_dst_alias);
> - munmap_area((void **)&area_remap);
> -}
> -
> -static void uffd_test_ctx_init(uint64_t features)
> -{
> - unsigned long nr, cpu;
> -
> - uffd_test_ctx_clear();
> -
> - uffd_test_ops->allocate_area((void **)&area_src, true);
> - uffd_test_ops->allocate_area((void **)&area_dst, false);
> -
> - userfaultfd_open(&features);
> -
> - count_verify = malloc(nr_pages * sizeof(unsigned long long));
> - if (!count_verify)
> - err("count_verify");
> -
> - for (nr = 0; nr < nr_pages; nr++) {
> - *area_mutex(area_src, nr) =
> - (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER;
> - count_verify[nr] = *area_count(area_src, nr) = 1;
> - /*
> - * In the transition between 255 to 256, powerpc will
> - * read out of order in my_bcmp and see both bytes as
> - * zero, so leave a placeholder below always non-zero
> - * after the count, to avoid my_bcmp to trigger false
> - * positives.
> - */
> - *(area_count(area_src, nr) + 1) = 1;
> - }
> -
> - /*
> - * After initialization of area_src, we must explicitly release pages
> - * for area_dst to make sure it's fully empty. Otherwise we could have
> - * some area_dst pages be errornously initialized with zero pages,
> - * hence we could hit memory corruption later in the test.
> - *
> - * One example is when THP is globally enabled, above allocate_area()
> - * calls could have the two areas merged into a single VMA (as they
> - * will have the same VMA flags so they're mergeable). When we
> - * initialize the area_src above, it's possible that some part of
> - * area_dst could have been faulted in via one huge THP that will be
> - * shared between area_src and area_dst. It could cause some of the
> - * area_dst won't be trapped by missing userfaults.
> - *
> - * This release_pages() will guarantee even if that happened, we'll
> - * proactively split the thp and drop any accidentally initialized
> - * pages within area_dst.
> - */
> - uffd_test_ops->release_pages(area_dst);
> -
> - pipefd = malloc(sizeof(int) * nr_cpus * 2);
> - if (!pipefd)
> - err("pipefd");
> - for (cpu = 0; cpu < nr_cpus; cpu++)
> - if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
> - err("pipe");
> -}
> -
> static int my_bcmp(char *str1, char *str2, size_t n)
> {
> unsigned long i;
> @@ -562,47 +122,6 @@ static int my_bcmp(char *str1, char *str2, size_t n)
> return 0;
> }
>
> -static void wp_range(int ufd, __u64 start, __u64 len, bool wp)
> -{
> - struct uffdio_writeprotect prms;
> -
> - /* Write protection page faults */
> - prms.range.start = start;
> - prms.range.len = len;
> - /* Undo write-protect, do wakeup after that */
> - prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0;
> -
> - if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms))
> - err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
> -}
> -
> -static void continue_range(int ufd, __u64 start, __u64 len)
> -{
> - struct uffdio_continue req;
> - int ret;
> -
> - req.range.start = start;
> - req.range.len = len;
> - req.mode = 0;
> - if (test_uffdio_wp)
> - req.mode |= UFFDIO_CONTINUE_MODE_WP;
> -
> - if (ioctl(ufd, UFFDIO_CONTINUE, &req))
> - err("UFFDIO_CONTINUE failed for address 0x%" PRIx64,
> - (uint64_t)start);
> -
> - /*
> - * Error handling within the kernel for continue is subtly different
> - * from copy or zeropage, so it may be a source of bugs. Trigger an
> - * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG.
> - */
> - req.mapped = 0;
> - ret = ioctl(ufd, UFFDIO_CONTINUE, &req);
> - if (ret >= 0 || req.mapped != -EEXIST)
> - err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64,
> - ret, (int64_t) req.mapped);
> -}
> -
> static void *locking_thread(void *arg)
> {
> unsigned long cpu = (unsigned long) arg;
> @@ -635,222 +154,11 @@ static void *locking_thread(void *arg)
> return NULL;
> }
>
> -static void retry_copy_page(int ufd, struct uffdio_copy *uffdio_copy,
> - unsigned long offset)
> -{
> - uffd_test_ops->alias_mapping(&uffdio_copy->dst,
> - uffdio_copy->len,
> - offset);
> - if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) {
> - /* real retval in ufdio_copy.copy */
> - if (uffdio_copy->copy != -EEXIST)
> - err("UFFDIO_COPY retry error: %"PRId64,
> - (int64_t)uffdio_copy->copy);
> - } else {
> - err("UFFDIO_COPY retry unexpected: %"PRId64,
> - (int64_t)uffdio_copy->copy);
> - }
> -}
> -
> -static void wake_range(int ufd, unsigned long addr, unsigned long len)
> -{
> - struct uffdio_range uffdio_wake;
> -
> - uffdio_wake.start = addr;
> - uffdio_wake.len = len;
> -
> - if (ioctl(ufd, UFFDIO_WAKE, &uffdio_wake))
> - fprintf(stderr, "error waking %lu\n",
> - addr), exit(1);
> -}
> -
> -static int __copy_page(int ufd, unsigned long offset, bool retry)
> -{
> - struct uffdio_copy uffdio_copy;
> -
> - if (offset >= nr_pages * page_size)
> - err("unexpected offset %lu\n", offset);
> - uffdio_copy.dst = (unsigned long) area_dst + offset;
> - uffdio_copy.src = (unsigned long) area_src + offset;
> - uffdio_copy.len = page_size;
> - if (test_uffdio_wp)
> - uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
> - else
> - uffdio_copy.mode = 0;
> - uffdio_copy.copy = 0;
> - if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) {
> - /* real retval in ufdio_copy.copy */
> - if (uffdio_copy.copy != -EEXIST)
> - err("UFFDIO_COPY error: %"PRId64,
> - (int64_t)uffdio_copy.copy);
> - wake_range(ufd, uffdio_copy.dst, page_size);
> - } else if (uffdio_copy.copy != page_size) {
> - err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy);
> - } else {
> - if (test_uffdio_copy_eexist && retry) {
> - test_uffdio_copy_eexist = false;
> - retry_copy_page(ufd, &uffdio_copy, offset);
> - }
> - return 1;
> - }
> - return 0;
> -}
> -
> static int copy_page_retry(int ufd, unsigned long offset)
> {
> return __copy_page(ufd, offset, true);
> }
>
> -static int copy_page(int ufd, unsigned long offset)
> -{
> - return __copy_page(ufd, offset, false);
> -}
> -
> -static int uffd_read_msg(int ufd, struct uffd_msg *msg)
> -{
> - int ret = read(uffd, msg, sizeof(*msg));
> -
> - if (ret != sizeof(*msg)) {
> - if (ret < 0) {
> - if (errno == EAGAIN || errno == EINTR)
> - return 1;
> - err("blocking read error");
> - } else {
> - err("short read");
> - }
> - }
> -
> - return 0;
> -}
> -
> -static void uffd_handle_page_fault(struct uffd_msg *msg,
> - struct uffd_stats *stats)
> -{
> - unsigned long offset;
> -
> - if (msg->event != UFFD_EVENT_PAGEFAULT)
> - err("unexpected msg event %u", msg->event);
> -
> - if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
> - /* Write protect page faults */
> - wp_range(uffd, msg->arg.pagefault.address, page_size, false);
> - stats->wp_faults++;
> - } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
> - uint8_t *area;
> - int b;
> -
> - /*
> - * Minor page faults
> - *
> - * To prove we can modify the original range for testing
> - * purposes, we're going to bit flip this range before
> - * continuing.
> - *
> - * Note that this requires all minor page fault tests operate on
> - * area_dst (non-UFFD-registered) and area_dst_alias
> - * (UFFD-registered).
> - */
> -
> - area = (uint8_t *)(area_dst +
> - ((char *)msg->arg.pagefault.address -
> - area_dst_alias));
> - for (b = 0; b < page_size; ++b)
> - area[b] = ~area[b];
> - continue_range(uffd, msg->arg.pagefault.address, page_size);
> - stats->minor_faults++;
> - } else {
> - /*
> - * Missing page faults.
> - *
> - * Here we force a write check for each of the missing mode
> - * faults. It's guaranteed because the only threads that
> - * will trigger uffd faults are the locking threads, and
> - * their first instruction to touch the missing page will
> - * always be pthread_mutex_lock().
> - *
> - * Note that here we relied on an NPTL glibc impl detail to
> - * always read the lock type at the entry of the lock op
> - * (pthread_mutex_t.__data.__type, offset 0x10) before
> - * doing any locking operations to guarantee that. It's
> - * actually not good to rely on this impl detail because
> - * logically a pthread-compatible lib can implement the
> - * locks without types and we can fail when linking with
> - * them. However since we used to find bugs with this
> - * strict check we still keep it around. Hopefully this
> - * could be a good hint when it fails again. If one day
> - * it'll break on some other impl of glibc we'll revisit.
> - */
> - if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE)
> - err("unexpected write fault");
> -
> - offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
> - offset &= ~(page_size-1);
> -
> - if (copy_page(uffd, offset))
> - stats->missing_faults++;
> - }
> -}
> -
> -static void *uffd_poll_thread(void *arg)
> -{
> - struct uffd_stats *stats = (struct uffd_stats *)arg;
> - unsigned long cpu = stats->cpu;
> - struct pollfd pollfd[2];
> - struct uffd_msg msg;
> - struct uffdio_register uffd_reg;
> - int ret;
> - char tmp_chr;
> -
> - pollfd[0].fd = uffd;
> - pollfd[0].events = POLLIN;
> - pollfd[1].fd = pipefd[cpu*2];
> - pollfd[1].events = POLLIN;
> -
> - for (;;) {
> - ret = poll(pollfd, 2, -1);
> - if (ret <= 0) {
> - if (errno == EINTR || errno == EAGAIN)
> - continue;
> - err("poll error: %d", ret);
> - }
> - if (pollfd[1].revents & POLLIN) {
> - if (read(pollfd[1].fd, &tmp_chr, 1) != 1)
> - err("read pipefd error");
> - break;
> - }
> - if (!(pollfd[0].revents & POLLIN))
> - err("pollfd[0].revents %d", pollfd[0].revents);
> - if (uffd_read_msg(uffd, &msg))
> - continue;
> - switch (msg.event) {
> - default:
> - err("unexpected msg event %u\n", msg.event);
> - break;
> - case UFFD_EVENT_PAGEFAULT:
> - uffd_handle_page_fault(&msg, stats);
> - break;
> - case UFFD_EVENT_FORK:
> - close(uffd);
> - uffd = msg.arg.fork.ufd;
> - pollfd[0].fd = uffd;
> - break;
> - case UFFD_EVENT_REMOVE:
> - uffd_reg.range.start = msg.arg.remove.start;
> - uffd_reg.range.len = msg.arg.remove.end -
> - msg.arg.remove.start;
> - if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range))
> - err("remove failure");
> - break;
> - case UFFD_EVENT_REMAP:
> - area_remap = area_dst; /* save for later unmap */
> - area_dst = (char *)(unsigned long)msg.arg.remap.to;
> - break;
> - }
> - }
> -
> - return NULL;
> -}
> -
> pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
>
> static void *uffd_read_thread(void *arg)
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (11 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 12/31] selftests/mm: Create uffd-common.[ch] Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 18:03 ` Axel Rasmussen
2023-04-12 16:42 ` [PATCH v2 14/31] selftests/mm: uffd_[un]register() Peter Xu
` (17 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
In many ways it's weird and unwanted to keep all the tests in the same
userfaultfd.c at least when still in the current way.
For example, it doesn't make much sense to run the stress test for each
method we can create an userfaultfd handle (either via syscall or /dev/
node). It's a waste of time running this twice for the whole stress as the
stress paths are the same, only the open path is different.
It's also just weird to need to manually specify different types of memory
to run all unit tests for the userfaultfd interface. We should be able to
just run a single program and that should go through all functional uffd
tests without running the stress test at all. The stress test was more for
torturing and finding race conditions. We don't want to wait for stress to
finish just to regress test a functional test.
When we start to pile up more things on top of the same file and same
functions, things start to go a bit chaos and the code is just harder to
maintain too with tons of global variables.
This patch creates a new test uffd-unit-tests to keep userfaultfd unit
tests in the future, currently empty.
Meanwhile rename the old userfaultfd.c test to uffd-stress.c.
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/.gitignore | 3 ++-
tools/testing/selftests/mm/Makefile | 8 +++---
tools/testing/selftests/mm/run_vmtests.sh | 10 ++++---
.../mm/{userfaultfd.c => uffd-stress.c} | 0
tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++
5 files changed, 40 insertions(+), 8 deletions(-)
rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%)
create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c
diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
index 347277f2adc3..8917455f4f51 100644
--- a/tools/testing/selftests/mm/.gitignore
+++ b/tools/testing/selftests/mm/.gitignore
@@ -21,7 +21,8 @@ protection_keys
protection_keys_32
protection_keys_64
madv_populate
-userfaultfd
+uffd-stress
+uffd-unit-tests
mlock-intersect-test
mlock-random-test
virtual_address_range
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 36467c15ca00..5a3434419403 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
# Avoid accidental wrong builds, due to built-in rules working just a little
# bit too well--but not quite as well as required for our situation here.
#
-# In other words, "make userfaultfd" is supposed to fail to build at all,
+# In other words, "make $SOME_TEST" is supposed to fail to build at all,
# because this Makefile only supports either "make" (all), or "make /full/path".
# However, the built-in rules, if not suppressed, will pick up CFLAGS and the
# initial LDLIBS (but not the target-specific LDLIBS, because those are only
@@ -57,7 +57,8 @@ TEST_GEN_PROGS += mremap_test
TEST_GEN_PROGS += on-fault-limit
TEST_GEN_PROGS += thuge-gen
TEST_GEN_PROGS += transhuge-stress
-TEST_GEN_PROGS += userfaultfd
+TEST_GEN_PROGS += uffd-stress
+TEST_GEN_PROGS += uffd-unit-tests
TEST_GEN_PROGS += soft-dirty
TEST_GEN_PROGS += split_huge_page_test
TEST_GEN_PROGS += ksm_tests
@@ -108,7 +109,8 @@ include ../lib.mk
$(TEST_GEN_PROGS): vm_util.c
-$(OUTPUT)/userfaultfd: uffd-common.c
+$(OUTPUT)/uffd-stress: uffd-common.c
+$(OUTPUT)/uffd-unit-tests: uffd-common.c
ifeq ($(MACHINE),x86_64)
BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index ddf40f883747..efe22dc569f0 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a
# Dump pages 0, 19, and 4096, using pin_user_pages:
CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
+CATEGORY="userfaultfd" run_test ./uffd-unit-tests
uffd_mods=("" ":dev")
+uffd_stress_bin=./uffd-stress
for mod in "${uffd_mods[@]}"; do
- CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
# Hugetlb tests require source and destination huge pages. Pass in half
# the size ($half_ufd_size_MB), which is used for *each*.
- CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
+ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
done
#cleanup
diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/uffd-stress.c
similarity index 100%
rename from tools/testing/selftests/mm/userfaultfd.c
rename to tools/testing/selftests/mm/uffd-stress.c
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
new file mode 100644
index 000000000000..6857388783be
--- /dev/null
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Userfaultfd unit tests.
+ *
+ * Copyright (C) 2015-2023 Red Hat, Inc.
+ */
+
+#include "uffd-common.h"
+
+#ifdef __NR_userfaultfd
+
+int main(int argc, char *argv[])
+{
+ return KSFT_PASS;
+}
+
+#else /* __NR_userfaultfd */
+
+#warning "missing __NR_userfaultfd definition"
+
+int main(void)
+{
+ printf("Skipping %s (missing __NR_userfaultfd)\n", __file__);
+ return KSFT_SKIP;
+}
+
+#endif /* __NR_userfaultfd */
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests
2023-04-12 16:42 ` [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests Peter Xu
@ 2023-04-12 18:03 ` Axel Rasmussen
0 siblings, 0 replies; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 18:03 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos, Mike Rapoport
On Wed, Apr 12, 2023 at 9:42 AM Peter Xu <peterx@redhat.com> wrote:
>
> In many ways it's weird and unwanted to keep all the tests in the same
> userfaultfd.c at least when still in the current way.
>
> For example, it doesn't make much sense to run the stress test for each
> method we can create an userfaultfd handle (either via syscall or /dev/
> node). It's a waste of time running this twice for the whole stress as the
> stress paths are the same, only the open path is different.
>
> It's also just weird to need to manually specify different types of memory
> to run all unit tests for the userfaultfd interface. We should be able to
> just run a single program and that should go through all functional uffd
> tests without running the stress test at all. The stress test was more for
> torturing and finding race conditions. We don't want to wait for stress to
> finish just to regress test a functional test.
>
> When we start to pile up more things on top of the same file and same
> functions, things start to go a bit chaos and the code is just harder to
> maintain too with tons of global variables.
>
> This patch creates a new test uffd-unit-tests to keep userfaultfd unit
> tests in the future, currently empty.
>
> Meanwhile rename the old userfaultfd.c test to uffd-stress.c.
>
> Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
> ---
> tools/testing/selftests/mm/.gitignore | 3 ++-
> tools/testing/selftests/mm/Makefile | 8 +++---
> tools/testing/selftests/mm/run_vmtests.sh | 10 ++++---
> .../mm/{userfaultfd.c => uffd-stress.c} | 0
> tools/testing/selftests/mm/uffd-unit-tests.c | 27 +++++++++++++++++++
> 5 files changed, 40 insertions(+), 8 deletions(-)
> rename tools/testing/selftests/mm/{userfaultfd.c => uffd-stress.c} (100%)
> create mode 100644 tools/testing/selftests/mm/uffd-unit-tests.c
>
> diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore
> index 347277f2adc3..8917455f4f51 100644
> --- a/tools/testing/selftests/mm/.gitignore
> +++ b/tools/testing/selftests/mm/.gitignore
> @@ -21,7 +21,8 @@ protection_keys
> protection_keys_32
> protection_keys_64
> madv_populate
> -userfaultfd
> +uffd-stress
> +uffd-unit-tests
> mlock-intersect-test
> mlock-random-test
> virtual_address_range
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 36467c15ca00..5a3434419403 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -20,7 +20,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
> # Avoid accidental wrong builds, due to built-in rules working just a little
> # bit too well--but not quite as well as required for our situation here.
> #
> -# In other words, "make userfaultfd" is supposed to fail to build at all,
> +# In other words, "make $SOME_TEST" is supposed to fail to build at all,
> # because this Makefile only supports either "make" (all), or "make /full/path".
> # However, the built-in rules, if not suppressed, will pick up CFLAGS and the
> # initial LDLIBS (but not the target-specific LDLIBS, because those are only
> @@ -57,7 +57,8 @@ TEST_GEN_PROGS += mremap_test
> TEST_GEN_PROGS += on-fault-limit
> TEST_GEN_PROGS += thuge-gen
> TEST_GEN_PROGS += transhuge-stress
> -TEST_GEN_PROGS += userfaultfd
> +TEST_GEN_PROGS += uffd-stress
> +TEST_GEN_PROGS += uffd-unit-tests
> TEST_GEN_PROGS += soft-dirty
> TEST_GEN_PROGS += split_huge_page_test
> TEST_GEN_PROGS += ksm_tests
> @@ -108,7 +109,8 @@ include ../lib.mk
>
> $(TEST_GEN_PROGS): vm_util.c
>
> -$(OUTPUT)/userfaultfd: uffd-common.c
> +$(OUTPUT)/uffd-stress: uffd-common.c
> +$(OUTPUT)/uffd-unit-tests: uffd-common.c
>
> ifeq ($(MACHINE),x86_64)
> BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32))
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index ddf40f883747..efe22dc569f0 100644
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -196,14 +196,16 @@ CATEGORY="gup_test" run_test ./gup_test -a
> # Dump pages 0, 19, and 4096, using pin_user_pages:
> CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
>
> +CATEGORY="userfaultfd" run_test ./uffd-unit-tests
> uffd_mods=("" ":dev")
> +uffd_stress_bin=./uffd-stress
> for mod in "${uffd_mods[@]}"; do
> - CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
> # Hugetlb tests require source and destination huge pages. Pass in half
> # the size ($half_ufd_size_MB), which is used for *each*.
> - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32
> - CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32
> - CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
> + CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
> done
>
> #cleanup
> diff --git a/tools/testing/selftests/mm/userfaultfd.c b/tools/testing/selftests/mm/uffd-stress.c
> similarity index 100%
> rename from tools/testing/selftests/mm/userfaultfd.c
> rename to tools/testing/selftests/mm/uffd-stress.c
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> new file mode 100644
> index 000000000000..6857388783be
> --- /dev/null
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -0,0 +1,27 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Userfaultfd unit tests.
> + *
> + * Copyright (C) 2015-2023 Red Hat, Inc.
> + */
> +
> +#include "uffd-common.h"
> +
> +#ifdef __NR_userfaultfd
> +
> +int main(int argc, char *argv[])
> +{
> + return KSFT_PASS;
> +}
> +
> +#else /* __NR_userfaultfd */
> +
> +#warning "missing __NR_userfaultfd definition"
> +
> +int main(void)
> +{
> + printf("Skipping %s (missing __NR_userfaultfd)\n", __file__);
> + return KSFT_SKIP;
> +}
> +
> +#endif /* __NR_userfaultfd */
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 14/31] selftests/mm: uffd_[un]register()
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (12 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 13/31] selftests/mm: Split uffd tests into uffd-stress and uffd-unit-tests Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 18:20 ` Axel Rasmussen
2023-04-12 16:42 ` [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}() Peter Xu
` (16 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx
Add two helpers to register/unregister to an uffd. Use them to drop
duplicate codes.
This patch also drops assert_expected_ioctls_present() and
get_expected_ioctls(). Reasons:
- It'll need a lot of effort to pass test_type==HUGETLB into it from the
upper, so it's the simplest way to get rid of another global var
- The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
any app can already detect kernel support on any ioctl via its
corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
it's probably destined no user app will even use it.
- It's not friendly to one future goal of uffd to run on old kernels, the
problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
which is a value that can change depending on where the test is compiled,
rather than reflecting what the kernel underneath has. It means it'll
report false negatives on old kernels so it's against our will.
So let's make our live easier.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/hugepage-mremap.c | 7 +-
.../selftests/mm/ksm_functional_tests.c | 6 +-
tools/testing/selftests/mm/uffd-common.c | 27 ------
tools/testing/selftests/mm/uffd-common.h | 2 -
tools/testing/selftests/mm/uffd-stress.c | 83 +++++--------------
tools/testing/selftests/mm/vm_util.c | 37 +++++++++
tools/testing/selftests/mm/vm_util.h | 4 +
7 files changed, 62 insertions(+), 104 deletions(-)
diff --git a/tools/testing/selftests/mm/hugepage-mremap.c b/tools/testing/selftests/mm/hugepage-mremap.c
index e53b5eaa8fce..2084692fe1c4 100644
--- a/tools/testing/selftests/mm/hugepage-mremap.c
+++ b/tools/testing/selftests/mm/hugepage-mremap.c
@@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len)
{
long uffd; /* userfaultfd file descriptor */
struct uffdio_api uffdio_api;
- struct uffdio_register uffdio_register;
/* Create and enable userfaultfd object. */
@@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
* handling by the userfaultfd object. In mode, we request to track
* missing pages (i.e., pages that have not yet been faulted in).
*/
-
- uffdio_register.range.start = (unsigned long)addr;
- uffdio_register.range.len = len;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
+ if (uffd_register(uffd, addr, len, true, false, false)) {
perror("ioctl-UFFDIO_REGISTER");
exit(1);
}
diff --git a/tools/testing/selftests/mm/ksm_functional_tests.c b/tools/testing/selftests/mm/ksm_functional_tests.c
index d8b5b4930412..d3f26050dfd7 100644
--- a/tools/testing/selftests/mm/ksm_functional_tests.c
+++ b/tools/testing/selftests/mm/ksm_functional_tests.c
@@ -178,7 +178,6 @@ static void test_unmerge_discarded(void)
static void test_unmerge_uffd_wp(void)
{
struct uffdio_writeprotect uffd_writeprotect;
- struct uffdio_register uffdio_register;
const unsigned int size = 2 * MiB;
struct uffdio_api uffdio_api;
char *map;
@@ -210,10 +209,7 @@ static void test_unmerge_uffd_wp(void)
}
/* Register UFFD-WP, no need for an actual handler. */
- uffdio_register.range.start = (unsigned long) map;
- uffdio_register.range.len = size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) < 0) {
+ if (uffd_register(uffd, map, size, false, true, false)) {
ksft_test_result_fail("UFFDIO_REGISTER_MODE_WP failed\n");
goto close_uffd;
}
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index c57757c2a36f..daa5b5781e7a 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)
err("pipe");
}
-uint64_t get_expected_ioctls(uint64_t mode)
-{
- uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
-
- if (test_type == TEST_HUGETLB)
- ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
-
- if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
- ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
-
- if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
- ioctls &= ~(1 << _UFFDIO_CONTINUE);
-
- return ioctls;
-}
-
-void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
-{
- uint64_t expected = get_expected_ioctls(mode);
- uint64_t actual = ioctls & expected;
-
- if (actual != expected) {
- err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
- expected, actual);
- }
-}
-
void wp_range(int ufd, __u64 start, __u64 len, bool wp)
{
struct uffdio_writeprotect prms;
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index d9430cfdcb19..11f770391bd9 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -101,8 +101,6 @@ extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
void uffd_test_ctx_init(uint64_t features);
void userfaultfd_open(uint64_t *features);
-uint64_t get_expected_ioctls(uint64_t mode);
-void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index c68a9aeefc41..e6d39a755082 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -463,28 +463,19 @@ static int uffdio_zeropage(int ufd, unsigned long offset)
/* exercise UFFDIO_ZEROPAGE */
static int userfaultfd_zeropage_test(void)
{
- struct uffdio_register uffdio_register;
-
printf("testing UFFDIO_ZEROPAGE: ");
fflush(stdout);
uffd_test_ctx_init(0);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (area_dst_alias) {
/* Needed this to test zeropage-retry on shared memory */
- uffdio_register.range.start = (unsigned long) area_dst_alias;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
}
@@ -498,7 +489,6 @@ static int userfaultfd_zeropage_test(void)
static int userfaultfd_events_test(void)
{
- struct uffdio_register uffdio_register;
pthread_t uffd_mon;
int err, features;
pid_t pid;
@@ -514,17 +504,10 @@ static int userfaultfd_events_test(void)
fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
err("uffd_poll_thread create");
@@ -550,7 +533,6 @@ static int userfaultfd_events_test(void)
static int userfaultfd_sig_test(void)
{
- struct uffdio_register uffdio_register;
unsigned long userfaults;
pthread_t uffd_mon;
int err, features;
@@ -566,17 +548,10 @@ static int userfaultfd_sig_test(void)
fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
if (faulting_process(1))
err("faulting process failed");
@@ -629,7 +604,6 @@ void check_memory_contents(char *p)
static int userfaultfd_minor_test(void)
{
unsigned long p;
- struct uffdio_register uffdio_register;
pthread_t uffd_mon;
char c;
struct uffd_stats stats = { 0 };
@@ -642,17 +616,10 @@ static int userfaultfd_minor_test(void)
uffd_test_ctx_init(uffd_minor_feature());
- uffdio_register.range.start = (unsigned long)area_dst_alias;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MINOR;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ false, test_uffdio_wp, true))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
-
/*
* After registering with UFFD, populate the non-UFFD-registered side of
* the shared mapping. This should *not* trigger any UFFD minor faults.
@@ -777,7 +744,6 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
static void userfaultfd_pagemap_test(unsigned int test_pgsize)
{
- struct uffdio_register uffdio_register;
int pagemap_fd;
uint64_t value;
@@ -805,10 +771,8 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
err("madvise(MADV_NOHUGEPAGE) failed");
}
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
err("register failed");
pagemap_fd = pagemap_open();
@@ -858,8 +822,8 @@ static int userfaultfd_stress(void)
{
void *area;
unsigned long nr;
- struct uffdio_register uffdio_register;
struct uffd_stats uffd_stats[nr_cpus];
+ uint64_t mem_size = nr_pages * page_size;
uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
@@ -894,20 +858,13 @@ static int userfaultfd_stress(void)
fcntl(uffd, F_SETFL, uffd_flags & ~O_NONBLOCK);
/* register */
- uffdio_register.range.start = (unsigned long) area_dst;
- uffdio_register.range.len = nr_pages * page_size;
- uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
- if (test_uffdio_wp)
- uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst, mem_size,
+ true, test_uffdio_wp, false))
err("register failure");
- assert_expected_ioctls_present(
- uffdio_register.mode, uffdio_register.ioctls);
if (area_dst_alias) {
- uffdio_register.range.start = (unsigned long)
- area_dst_alias;
- if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
+ if (uffd_register(uffd, area_dst_alias, mem_size,
+ true, test_uffdio_wp, false))
err("register failure alias");
}
@@ -949,12 +906,10 @@ static int userfaultfd_stress(void)
nr_pages * page_size, false);
/* unregister */
- if (ioctl(uffd, UFFDIO_UNREGISTER, &uffdio_register.range))
+ if (uffd_unregister(uffd, area_dst, mem_size))
err("unregister failure");
if (area_dst_alias) {
- uffdio_register.range.start = (unsigned long) area_dst;
- if (ioctl(uffd, UFFDIO_UNREGISTER,
- &uffdio_register.range))
+ if (uffd_unregister(uffd, area_dst_alias, mem_size))
err("unregister failure alias");
}
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 54d227d6f70a..bb633d050d71 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
#include <string.h>
#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <linux/userfaultfd.h>
#include "../kselftest.h"
#include "vm_util.h"
@@ -193,3 +195,38 @@ unsigned long default_huge_page_size(void)
fclose(f);
return hps;
}
+
+int uffd_register(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor)
+{
+ struct uffdio_register uffdio_register = { 0 };
+ uint64_t mode = 0;
+ int ret = 0;
+
+ if (miss)
+ mode |= UFFDIO_REGISTER_MODE_MISSING;
+ if (wp)
+ mode |= UFFDIO_REGISTER_MODE_WP;
+ if (minor)
+ mode |= UFFDIO_REGISTER_MODE_MINOR;
+
+ uffdio_register.range.start = (unsigned long)addr;
+ uffdio_register.range.len = len;
+ uffdio_register.mode = mode;
+
+ if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
+ ret = -errno;
+
+ return ret;
+}
+
+int uffd_unregister(int uffd, void *addr, uint64_t len)
+{
+ struct uffdio_range range = { .start = (uintptr_t)addr, .len = len };
+ int ret = 0;
+
+ if (ioctl(uffd, UFFDIO_UNREGISTER, &range) == -1)
+ ret = -errno;
+
+ return ret;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index d9fadddb5c69..3a9762022efd 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -45,6 +45,10 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
int64_t allocate_transhuge(void *ptr, int pagemap_fd);
unsigned long default_huge_page_size(void);
+int uffd_register(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor);
+int uffd_unregister(int uffd, void *addr, uint64_t len);
+
/*
* On ppc64 this will only work with radix 2M hugepage size
*/
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 14/31] selftests/mm: uffd_[un]register()
2023-04-12 16:42 ` [PATCH v2 14/31] selftests/mm: uffd_[un]register() Peter Xu
@ 2023-04-12 18:20 ` Axel Rasmussen
2023-04-12 19:33 ` Peter Xu
0 siblings, 1 reply; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 18:20 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 9:42 AM Peter Xu <peterx@redhat.com> wrote:
>
> Add two helpers to register/unregister to an uffd. Use them to drop
> duplicate codes.
>
> This patch also drops assert_expected_ioctls_present() and
> get_expected_ioctls(). Reasons:
>
> - It'll need a lot of effort to pass test_type==HUGETLB into it from the
> upper, so it's the simplest way to get rid of another global var
>
> - The ioctls returned in UFFDIO_REGISTER is hardly useful at all, because
> any app can already detect kernel support on any ioctl via its
> corresponding UFFD_FEATURE_*. The check here is for sanity mostly but
> it's probably destined no user app will even use it.
>
> - It's not friendly to one future goal of uffd to run on old kernels, the
> problem is get_expected_ioctls() compiles against UFFD_API_RANGE_IOCTLS,
> which is a value that can change depending on where the test is compiled,
> rather than reflecting what the kernel underneath has. It means it'll
> report false negatives on old kernels so it's against our will.
I do think there is some value in this test, but I see patch 31 adds a
similar test back so it's fine to remove it in the meantime to clean
up things like get_expected_ioctls().
>
> So let's make our live easier.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
> ---
> tools/testing/selftests/mm/hugepage-mremap.c | 7 +-
> .../selftests/mm/ksm_functional_tests.c | 6 +-
> tools/testing/selftests/mm/uffd-common.c | 27 ------
> tools/testing/selftests/mm/uffd-common.h | 2 -
> tools/testing/selftests/mm/uffd-stress.c | 83 +++++--------------
> tools/testing/selftests/mm/vm_util.c | 37 +++++++++
> tools/testing/selftests/mm/vm_util.h | 4 +
> 7 files changed, 62 insertions(+), 104 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/hugepage-mremap.c b/tools/testing/selftests/mm/hugepage-mremap.c
> index e53b5eaa8fce..2084692fe1c4 100644
> --- a/tools/testing/selftests/mm/hugepage-mremap.c
> +++ b/tools/testing/selftests/mm/hugepage-mremap.c
> @@ -60,7 +60,6 @@ static void register_region_with_uffd(char *addr, size_t len)
> {
> long uffd; /* userfaultfd file descriptor */
> struct uffdio_api uffdio_api;
> - struct uffdio_register uffdio_register;
>
> /* Create and enable userfaultfd object. */
>
> @@ -96,11 +95,7 @@ static void register_region_with_uffd(char *addr, size_t len)
> * handling by the userfaultfd object. In mode, we request to track
> * missing pages (i.e., pages that have not yet been faulted in).
> */
> -
> - uffdio_register.range.start = (unsigned long)addr;
> - uffdio_register.range.len = len;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
> + if (uffd_register(uffd, addr, len, true, false, false)) {
For what it's worth, I agree with Mike that the booleans here are hard
to read. It's not clear what "true, false, false" means without going
to look at the header file, whereas "uffd_register(uffd, addr, len,
MINOR)" would be immediately clear.
One solution I've seen outside the kernel is to comment the bools, like:
uffd_register(..., /*minor=*/true, /*wp=*/false, /*minor=*/false);
But, then I feel we lose most of the benefit we wanted from switching
to bools anyway (code length). :)
I do agree the macro names are unwieldy, and lots of tests already use
booleans so we'd have to convert from bool -> flag. If it were me, I
would resolve that by:
- Define much shorter aliases for these macros in uffd-common.h
- Consider refactoring callers to not use bools either.
Then again, I also agree with Mike that it's not a deal breaker, if
you like this way much more than that alternative we can leave it.
> perror("ioctl-UFFDIO_REGISTER");
> exit(1);
> }
> diff --git a/tools/testing/selftests/mm/ksm_functional_tests.c b/tools/testing/selftests/mm/ksm_functional_tests.c
> index d8b5b4930412..d3f26050dfd7 100644
> --- a/tools/testing/selftests/mm/ksm_functional_tests.c
> +++ b/tools/testing/selftests/mm/ksm_functional_tests.c
> @@ -178,7 +178,6 @@ static void test_unmerge_discarded(void)
> static void test_unmerge_uffd_wp(void)
> {
> struct uffdio_writeprotect uffd_writeprotect;
> - struct uffdio_register uffdio_register;
> const unsigned int size = 2 * MiB;
> struct uffdio_api uffdio_api;
> char *map;
> @@ -210,10 +209,7 @@ static void test_unmerge_uffd_wp(void)
> }
>
> /* Register UFFD-WP, no need for an actual handler. */
> - uffdio_register.range.start = (unsigned long) map;
> - uffdio_register.range.len = size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) < 0) {
> + if (uffd_register(uffd, map, size, false, true, false)) {
> ksft_test_result_fail("UFFDIO_REGISTER_MODE_WP failed\n");
> goto close_uffd;
> }
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index c57757c2a36f..daa5b5781e7a 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -330,33 +330,6 @@ void uffd_test_ctx_init(uint64_t features)
> err("pipe");
> }
>
> -uint64_t get_expected_ioctls(uint64_t mode)
> -{
> - uint64_t ioctls = UFFD_API_RANGE_IOCTLS;
> -
> - if (test_type == TEST_HUGETLB)
> - ioctls &= ~(1 << _UFFDIO_ZEROPAGE);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_WP) && test_uffdio_wp))
> - ioctls &= ~(1 << _UFFDIO_WRITEPROTECT);
> -
> - if (!((mode & UFFDIO_REGISTER_MODE_MINOR) && test_uffdio_minor))
> - ioctls &= ~(1 << _UFFDIO_CONTINUE);
> -
> - return ioctls;
> -}
> -
> -void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls)
> -{
> - uint64_t expected = get_expected_ioctls(mode);
> - uint64_t actual = ioctls & expected;
> -
> - if (actual != expected) {
> - err("missing ioctl(s): expected %"PRIx64" actual: %"PRIx64,
> - expected, actual);
> - }
> -}
> -
> void wp_range(int ufd, __u64 start, __u64 len, bool wp)
> {
> struct uffdio_writeprotect prms;
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index d9430cfdcb19..11f770391bd9 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -101,8 +101,6 @@ extern uffd_test_ops_t *uffd_test_ops;
> void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
> void uffd_test_ctx_init(uint64_t features);
> void userfaultfd_open(uint64_t *features);
> -uint64_t get_expected_ioctls(uint64_t mode);
> -void assert_expected_ioctls_present(uint64_t mode, uint64_t ioctls);
> int uffd_read_msg(int ufd, struct uffd_msg *msg);
> void wp_range(int ufd, __u64 start, __u64 len, bool wp);
> void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index c68a9aeefc41..e6d39a755082 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -463,28 +463,19 @@ static int uffdio_zeropage(int ufd, unsigned long offset)
> /* exercise UFFDIO_ZEROPAGE */
> static int userfaultfd_zeropage_test(void)
> {
> - struct uffdio_register uffdio_register;
> -
> printf("testing UFFDIO_ZEROPAGE: ");
> fflush(stdout);
>
> uffd_test_ctx_init(0);
>
> - uffdio_register.range.start = (unsigned long) area_dst;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (test_uffdio_wp)
> - uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + true, test_uffdio_wp, false))
> err("register failure");
>
> - assert_expected_ioctls_present(
> - uffdio_register.mode, uffdio_register.ioctls);
> -
> if (area_dst_alias) {
> /* Needed this to test zeropage-retry on shared memory */
> - uffdio_register.range.start = (unsigned long) area_dst_alias;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
> + true, test_uffdio_wp, false))
> err("register failure");
> }
>
> @@ -498,7 +489,6 @@ static int userfaultfd_zeropage_test(void)
>
> static int userfaultfd_events_test(void)
> {
> - struct uffdio_register uffdio_register;
> pthread_t uffd_mon;
> int err, features;
> pid_t pid;
> @@ -514,17 +504,10 @@ static int userfaultfd_events_test(void)
>
> fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
>
> - uffdio_register.range.start = (unsigned long) area_dst;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (test_uffdio_wp)
> - uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + true, test_uffdio_wp, false))
> err("register failure");
>
> - assert_expected_ioctls_present(
> - uffdio_register.mode, uffdio_register.ioctls);
> -
> if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> err("uffd_poll_thread create");
>
> @@ -550,7 +533,6 @@ static int userfaultfd_events_test(void)
>
> static int userfaultfd_sig_test(void)
> {
> - struct uffdio_register uffdio_register;
> unsigned long userfaults;
> pthread_t uffd_mon;
> int err, features;
> @@ -566,17 +548,10 @@ static int userfaultfd_sig_test(void)
>
> fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
>
> - uffdio_register.range.start = (unsigned long) area_dst;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (test_uffdio_wp)
> - uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + true, test_uffdio_wp, false))
> err("register failure");
>
> - assert_expected_ioctls_present(
> - uffdio_register.mode, uffdio_register.ioctls);
> -
> if (faulting_process(1))
> err("faulting process failed");
>
> @@ -629,7 +604,6 @@ void check_memory_contents(char *p)
> static int userfaultfd_minor_test(void)
> {
> unsigned long p;
> - struct uffdio_register uffdio_register;
> pthread_t uffd_mon;
> char c;
> struct uffd_stats stats = { 0 };
> @@ -642,17 +616,10 @@ static int userfaultfd_minor_test(void)
>
> uffd_test_ctx_init(uffd_minor_feature());
>
> - uffdio_register.range.start = (unsigned long)area_dst_alias;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MINOR;
> - if (test_uffdio_wp)
> - uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
> + false, test_uffdio_wp, true))
> err("register failure");
>
> - assert_expected_ioctls_present(
> - uffdio_register.mode, uffdio_register.ioctls);
> -
> /*
> * After registering with UFFD, populate the non-UFFD-registered side of
> * the shared mapping. This should *not* trigger any UFFD minor faults.
> @@ -777,7 +744,6 @@ static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
>
> static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> {
> - struct uffdio_register uffdio_register;
> int pagemap_fd;
> uint64_t value;
>
> @@ -805,10 +771,8 @@ static void userfaultfd_pagemap_test(unsigned int test_pgsize)
> err("madvise(MADV_NOHUGEPAGE) failed");
> }
>
> - uffdio_register.range.start = (unsigned long) area_dst;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst, nr_pages * page_size,
> + false, true, false))
> err("register failed");
>
> pagemap_fd = pagemap_open();
> @@ -858,8 +822,8 @@ static int userfaultfd_stress(void)
> {
> void *area;
> unsigned long nr;
> - struct uffdio_register uffdio_register;
> struct uffd_stats uffd_stats[nr_cpus];
> + uint64_t mem_size = nr_pages * page_size;
>
> uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
>
> @@ -894,20 +858,13 @@ static int userfaultfd_stress(void)
> fcntl(uffd, F_SETFL, uffd_flags & ~O_NONBLOCK);
>
> /* register */
> - uffdio_register.range.start = (unsigned long) area_dst;
> - uffdio_register.range.len = nr_pages * page_size;
> - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> - if (test_uffdio_wp)
> - uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst, mem_size,
> + true, test_uffdio_wp, false))
> err("register failure");
> - assert_expected_ioctls_present(
> - uffdio_register.mode, uffdio_register.ioctls);
>
> if (area_dst_alias) {
> - uffdio_register.range.start = (unsigned long)
> - area_dst_alias;
> - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register))
> + if (uffd_register(uffd, area_dst_alias, mem_size,
> + true, test_uffdio_wp, false))
> err("register failure alias");
> }
>
> @@ -949,12 +906,10 @@ static int userfaultfd_stress(void)
> nr_pages * page_size, false);
>
> /* unregister */
> - if (ioctl(uffd, UFFDIO_UNREGISTER, &uffdio_register.range))
> + if (uffd_unregister(uffd, area_dst, mem_size))
> err("unregister failure");
> if (area_dst_alias) {
> - uffdio_register.range.start = (unsigned long) area_dst;
> - if (ioctl(uffd, UFFDIO_UNREGISTER,
> - &uffdio_register.range))
> + if (uffd_unregister(uffd, area_dst_alias, mem_size))
> err("unregister failure alias");
> }
>
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index 54d227d6f70a..bb633d050d71 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -1,6 +1,8 @@
> // SPDX-License-Identifier: GPL-2.0
> #include <string.h>
> #include <fcntl.h>
> +#include <sys/ioctl.h>
> +#include <linux/userfaultfd.h>
> #include "../kselftest.h"
> #include "vm_util.h"
>
> @@ -193,3 +195,38 @@ unsigned long default_huge_page_size(void)
> fclose(f);
> return hps;
> }
> +
> +int uffd_register(int uffd, void *addr, uint64_t len,
> + bool miss, bool wp, bool minor)
> +{
> + struct uffdio_register uffdio_register = { 0 };
> + uint64_t mode = 0;
> + int ret = 0;
> +
> + if (miss)
> + mode |= UFFDIO_REGISTER_MODE_MISSING;
> + if (wp)
> + mode |= UFFDIO_REGISTER_MODE_WP;
> + if (minor)
> + mode |= UFFDIO_REGISTER_MODE_MINOR;
> +
> + uffdio_register.range.start = (unsigned long)addr;
> + uffdio_register.range.len = len;
> + uffdio_register.mode = mode;
> +
> + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
> + ret = -errno;
> +
> + return ret;
> +}
> +
> +int uffd_unregister(int uffd, void *addr, uint64_t len)
> +{
> + struct uffdio_range range = { .start = (uintptr_t)addr, .len = len };
> + int ret = 0;
> +
> + if (ioctl(uffd, UFFDIO_UNREGISTER, &range) == -1)
> + ret = -errno;
> +
> + return ret;
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index d9fadddb5c69..3a9762022efd 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -45,6 +45,10 @@ bool check_huge_shmem(void *addr, int nr_hpages, uint64_t hpage_size);
> int64_t allocate_transhuge(void *ptr, int pagemap_fd);
> unsigned long default_huge_page_size(void);
>
> +int uffd_register(int uffd, void *addr, uint64_t len,
> + bool miss, bool wp, bool minor);
> +int uffd_unregister(int uffd, void *addr, uint64_t len);
> +
> /*
> * On ppc64 this will only work with radix 2M hugepage size
> */
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v2 14/31] selftests/mm: uffd_[un]register()
2023-04-12 18:20 ` Axel Rasmussen
@ 2023-04-12 19:33 ` Peter Xu
0 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 19:33 UTC (permalink / raw)
To: Axel Rasmussen
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 11:20:35AM -0700, Axel Rasmussen wrote:
> > - uffdio_register.range.start = (unsigned long)addr;
> > - uffdio_register.range.len = len;
> > - uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
> > - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) {
> > + if (uffd_register(uffd, addr, len, true, false, false)) {
>
> For what it's worth, I agree with Mike that the booleans here are hard
> to read. It's not clear what "true, false, false" means without going
> to look at the header file, whereas "uffd_register(uffd, addr, len,
> MINOR)" would be immediately clear.
>
> One solution I've seen outside the kernel is to comment the bools, like:
>
> uffd_register(..., /*minor=*/true, /*wp=*/false, /*minor=*/false);
>
> But, then I feel we lose most of the benefit we wanted from switching
> to bools anyway (code length). :)
>
> I do agree the macro names are unwieldy, and lots of tests already use
> booleans so we'd have to convert from bool -> flag. If it were me, I
> would resolve that by:
>
> - Define much shorter aliases for these macros in uffd-common.h
> - Consider refactoring callers to not use bools either.
>
> Then again, I also agree with Mike that it's not a deal breaker, if
> you like this way much more than that alternative we can leave it.
No strong feeling here either. I kept it just to avoid code churns and
rebases.
I'll see whether I'll need a repost, if so I can go back to use modes.
Thanks,
--
Peter Xu
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}()
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (13 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 14/31] selftests/mm: uffd_[un]register() Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 18:25 ` Axel Rasmussen
2023-04-12 16:42 ` [PATCH v2 16/31] selftests/mm: UFFDIO_API test Peter Xu
` (15 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx, Mike Rapoport
Provide two helpers to open an uffd handle. Drop the error checks around
SKIPs because it's inside an errexit() anyway, which IMHO doesn't really
help much if the test will not continue.
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 28 +++++-------------------
tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++
tools/testing/selftests/mm/vm_util.h | 2 ++
3 files changed, 31 insertions(+), 23 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index daa5b5781e7a..09ea24c5f02c 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
printf("\n");
}
-static int __userfaultfd_open_dev(void)
-{
- int fd, _uffd;
-
- fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
- if (fd < 0)
- errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
-
- _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
- if (_uffd < 0)
- errexit(errno == ENOTTY ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- close(fd);
- return _uffd;
-}
-
void userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
if (test_dev_userfaultfd)
- uffd = __userfaultfd_open_dev();
- else {
- uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
- if (uffd < 0)
- errexit(errno == ENOSYS ? KSFT_SKIP : 1,
- "creating userfaultfd failed");
- }
+ uffd = uffd_open_dev(UFFD_FLAGS);
+ else
+ uffd = uffd_open_sys(UFFD_FLAGS);
+ if (uffd < 0)
+ err("uffd open failed (dev=%d)", test_dev_userfaultfd);
uffd_flags = fcntl(uffd, F_GETFD, NULL);
uffdio_api.api = UFFD_API;
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index bb633d050d71..5ee6c4688a7c 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -3,6 +3,8 @@
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/userfaultfd.h>
+#include <sys/syscall.h>
+#include <unistd.h>
#include "../kselftest.h"
#include "vm_util.h"
@@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
return ret;
}
+
+int uffd_open_dev(unsigned int flags)
+{
+ int fd, uffd;
+
+ fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+ if (fd < 0)
+ return fd;
+ uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+ close(fd);
+
+ return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+ return syscall(__NR_userfaultfd, flags);
+#else
+ return -1;
+#endif
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 3a9762022efd..481354141533 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -48,6 +48,8 @@ unsigned long default_huge_page_size(void);
int uffd_register(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor);
int uffd_unregister(int uffd, void *addr, uint64_t len);
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}()
2023-04-12 16:42 ` [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}() Peter Xu
@ 2023-04-12 18:25 ` Axel Rasmussen
0 siblings, 0 replies; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 18:25 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos, Mike Rapoport
On Wed, Apr 12, 2023 at 9:43 AM Peter Xu <peterx@redhat.com> wrote:
>
> Provide two helpers to open an uffd handle. Drop the error checks around
> SKIPs because it's inside an errexit() anyway, which IMHO doesn't really
> help much if the test will not continue.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
> ---
> tools/testing/selftests/mm/uffd-common.c | 28 +++++-------------------
> tools/testing/selftests/mm/vm_util.c | 24 ++++++++++++++++++++
> tools/testing/selftests/mm/vm_util.h | 2 ++
> 3 files changed, 31 insertions(+), 23 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index daa5b5781e7a..09ea24c5f02c 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -192,34 +192,16 @@ void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
> printf("\n");
> }
>
> -static int __userfaultfd_open_dev(void)
> -{
> - int fd, _uffd;
> -
> - fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> - if (fd < 0)
> - errexit(KSFT_SKIP, "opening /dev/userfaultfd failed");
> -
> - _uffd = ioctl(fd, USERFAULTFD_IOC_NEW, UFFD_FLAGS);
> - if (_uffd < 0)
> - errexit(errno == ENOTTY ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - close(fd);
> - return _uffd;
> -}
> -
> void userfaultfd_open(uint64_t *features)
> {
> struct uffdio_api uffdio_api;
>
> if (test_dev_userfaultfd)
> - uffd = __userfaultfd_open_dev();
> - else {
> - uffd = syscall(__NR_userfaultfd, UFFD_FLAGS);
> - if (uffd < 0)
> - errexit(errno == ENOSYS ? KSFT_SKIP : 1,
> - "creating userfaultfd failed");
> - }
> + uffd = uffd_open_dev(UFFD_FLAGS);
> + else
> + uffd = uffd_open_sys(UFFD_FLAGS);
> + if (uffd < 0)
> + err("uffd open failed (dev=%d)", test_dev_userfaultfd);
> uffd_flags = fcntl(uffd, F_GETFD, NULL);
>
> uffdio_api.api = UFFD_API;
> diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
> index bb633d050d71..5ee6c4688a7c 100644
> --- a/tools/testing/selftests/mm/vm_util.c
> +++ b/tools/testing/selftests/mm/vm_util.c
> @@ -3,6 +3,8 @@
> #include <fcntl.h>
> #include <sys/ioctl.h>
> #include <linux/userfaultfd.h>
> +#include <sys/syscall.h>
> +#include <unistd.h>
> #include "../kselftest.h"
> #include "vm_util.h"
>
> @@ -230,3 +232,25 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
>
> return ret;
> }
> +
> +int uffd_open_dev(unsigned int flags)
> +{
> + int fd, uffd;
> +
> + fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
> + if (fd < 0)
> + return fd;
> + uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
> + close(fd);
> +
> + return uffd;
> +}
> +
> +int uffd_open_sys(unsigned int flags)
> +{
> +#ifdef __NR_userfaultfd
> + return syscall(__NR_userfaultfd, flags);
> +#else
> + return -1;
> +#endif
> +}
> diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
> index 3a9762022efd..481354141533 100644
> --- a/tools/testing/selftests/mm/vm_util.h
> +++ b/tools/testing/selftests/mm/vm_util.h
> @@ -48,6 +48,8 @@ unsigned long default_huge_page_size(void);
> int uffd_register(int uffd, void *addr, uint64_t len,
> bool miss, bool wp, bool minor);
> int uffd_unregister(int uffd, void *addr, uint64_t len);
> +int uffd_open_dev(unsigned int flags);
> +int uffd_open_sys(unsigned int flags);
>
> /*
> * On ppc64 this will only work with radix 2M hugepage size
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 16/31] selftests/mm: UFFDIO_API test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (14 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 15/31] selftests/mm: uffd_open_{dev|sys}() Peter Xu
@ 2023-04-12 16:42 ` Peter Xu
2023-04-12 19:47 ` Axel Rasmussen
2023-04-20 10:54 ` Mike Rapoport
2023-04-12 16:43 ` [PATCH v2 17/31] selftests/mm: Drop global mem_fd in uffd tests Peter Xu
` (14 subsequent siblings)
30 siblings, 2 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:42 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Andrew Morton, Andrea Arcangeli, Mike Kravetz, Nadav Amit,
David Hildenbrand, Mike Rapoport, Axel Rasmussen,
Leonardo Bras Soares Passos, peterx
Add one simple test for UFFDIO_API. With that, I also added a bunch of
small but handy helpers along the way.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++-
1 file changed, 108 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 6857388783be..bb492c258486 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -9,9 +9,116 @@
#ifdef __NR_userfaultfd
+static void uffd_test_report(void)
+{
+ printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
+ ksft_get_pass_cnt(),
+ ksft_get_xskip_cnt(),
+ ksft_get_fail_cnt(),
+ ksft_test_num());
+}
+
+static void uffd_test_pass(void)
+{
+ printf("done\n");
+ ksft_inc_pass_cnt();
+}
+
+#define uffd_test_start(...) do { \
+ printf("Testing "); \
+ printf(__VA_ARGS__); \
+ printf("... "); \
+ fflush(stdout); \
+ } while (0)
+
+#define uffd_test_fail(...) do { \
+ printf("failed [reason: "); \
+ printf(__VA_ARGS__); \
+ printf("]\n"); \
+ ksft_inc_fail_cnt(); \
+ } while (0)
+
+#define uffd_test_skip(...) do { \
+ printf("skipped [reason: "); \
+ printf(__VA_ARGS__); \
+ printf("]\n"); \
+ ksft_inc_xskip_cnt(); \
+ } while (0)
+
+/*
+ * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
+ * return 1 even if some test failed as long as uffd supported, because in
+ * that case we still want to proceed with the rest uffd unit tests.
+ */
+static int test_uffd_api(bool use_dev)
+{
+ struct uffdio_api uffdio_api;
+ int uffd;
+
+ uffd_test_start("UFFDIO_API (with %s)",
+ use_dev ? "/dev/userfaultfd" : "syscall");
+
+ if (use_dev)
+ uffd = uffd_open_dev(UFFD_FLAGS);
+ else
+ uffd = uffd_open_sys(UFFD_FLAGS);
+ if (uffd < 0) {
+ uffd_test_skip("cannot open userfaultfd handle");
+ return 0;
+ }
+
+ /* Test wrong UFFD_API */
+ uffdio_api.api = 0xab;
+ uffdio_api.features = 0;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should fail with wrong api but didn't");
+ goto out;
+ }
+
+ /* Test wrong feature bit */
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = BIT_ULL(63);
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should fail with wrong feature but didn't");
+ goto out;
+ }
+
+ /* Test normal UFFDIO_API */
+ uffdio_api.api = UFFD_API;
+ uffdio_api.features = 0;
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api)) {
+ uffd_test_fail("UFFDIO_API should succeed but failed");
+ goto out;
+ }
+
+ /* Test double requests of UFFDIO_API with a random feature set */
+ uffdio_api.features = BIT_ULL(0);
+ if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
+ uffd_test_fail("UFFDIO_API should reject initialized uffd");
+ goto out;
+ }
+
+ uffd_test_pass();
+out:
+ close(uffd);
+ /* We have a valid uffd handle */
+ return 1;
+}
+
int main(int argc, char *argv[])
{
- return KSFT_PASS;
+ int has_uffd;
+
+ has_uffd = test_uffd_api(false);
+ has_uffd |= test_uffd_api(true);
+
+ if (!has_uffd) {
+ printf("Userfaultfd not supported or unprivileged, skip all tests\n");
+ exit(KSFT_SKIP);
+ }
+ uffd_test_report();
+
+ return ksft_get_fail_cnt() ? KSFT_FAIL : KSFT_PASS;
}
#else /* __NR_userfaultfd */
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
2023-04-12 16:42 ` [PATCH v2 16/31] selftests/mm: UFFDIO_API test Peter Xu
@ 2023-04-12 19:47 ` Axel Rasmussen
2023-04-12 20:08 ` Peter Xu
2023-04-20 10:54 ` Mike Rapoport
1 sibling, 1 reply; 43+ messages in thread
From: Axel Rasmussen @ 2023-04-12 19:47 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 9:43 AM Peter Xu <peterx@redhat.com> wrote:
>
> Add one simple test for UFFDIO_API. With that, I also added a bunch of
> small but handy helpers along the way.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++-
> 1 file changed, 108 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 6857388783be..bb492c258486 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -9,9 +9,116 @@
>
> #ifdef __NR_userfaultfd
>
> +static void uffd_test_report(void)
> +{
> + printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
> + ksft_get_pass_cnt(),
> + ksft_get_xskip_cnt(),
> + ksft_get_fail_cnt(),
> + ksft_test_num());
> +}
> +
> +static void uffd_test_pass(void)
> +{
> + printf("done\n");
> + ksft_inc_pass_cnt();
> +}
> +
> +#define uffd_test_start(...) do { \
> + printf("Testing "); \
> + printf(__VA_ARGS__); \
> + printf("... "); \
> + fflush(stdout); \
> + } while (0)
> +
> +#define uffd_test_fail(...) do { \
> + printf("failed [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + ksft_inc_fail_cnt(); \
> + } while (0)
> +
> +#define uffd_test_skip(...) do { \
> + printf("skipped [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + ksft_inc_xskip_cnt(); \
> + } while (0)
Some of this is duplicating what's in kselftest_harness.h - e.g.
assertions, printing test summary, keeping track of pass/fail/skip
counts, etc.
I wonder how you feel about using the standard harness? E.g. is the
plan to do this (more straightforward?) refactor first, and then
switch later? Or is there some reason it can't be used?
> +
> +/*
> + * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
> + * return 1 even if some test failed as long as uffd supported, because in
> + * that case we still want to proceed with the rest uffd unit tests.
> + */
> +static int test_uffd_api(bool use_dev)
> +{
> + struct uffdio_api uffdio_api;
> + int uffd;
> +
> + uffd_test_start("UFFDIO_API (with %s)",
> + use_dev ? "/dev/userfaultfd" : "syscall");
> +
> + if (use_dev)
> + uffd = uffd_open_dev(UFFD_FLAGS);
> + else
> + uffd = uffd_open_sys(UFFD_FLAGS);
> + if (uffd < 0) {
> + uffd_test_skip("cannot open userfaultfd handle");
> + return 0;
> + }
> +
> + /* Test wrong UFFD_API */
> + uffdio_api.api = 0xab;
> + uffdio_api.features = 0;
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should fail with wrong api but didn't");
> + goto out;
> + }
> +
> + /* Test wrong feature bit */
> + uffdio_api.api = UFFD_API;
> + uffdio_api.features = BIT_ULL(63);
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should fail with wrong feature but didn't");
> + goto out;
> + }
> +
> + /* Test normal UFFDIO_API */
> + uffdio_api.api = UFFD_API;
> + uffdio_api.features = 0;
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api)) {
> + uffd_test_fail("UFFDIO_API should succeed but failed");
> + goto out;
> + }
> +
> + /* Test double requests of UFFDIO_API with a random feature set */
> + uffdio_api.features = BIT_ULL(0);
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should reject initialized uffd");
> + goto out;
> + }
> +
> + uffd_test_pass();
> +out:
> + close(uffd);
> + /* We have a valid uffd handle */
> + return 1;
> +}
> +
> int main(int argc, char *argv[])
> {
> - return KSFT_PASS;
> + int has_uffd;
> +
> + has_uffd = test_uffd_api(false);
> + has_uffd |= test_uffd_api(true);
> +
> + if (!has_uffd) {
> + printf("Userfaultfd not supported or unprivileged, skip all tests\n");
> + exit(KSFT_SKIP);
> + }
> + uffd_test_report();
> +
> + return ksft_get_fail_cnt() ? KSFT_FAIL : KSFT_PASS;
> }
>
> #else /* __NR_userfaultfd */
> --
> 2.39.1
>
^ permalink raw reply [flat|nested] 43+ messages in thread* Re: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
2023-04-12 19:47 ` Axel Rasmussen
@ 2023-04-12 20:08 ` Peter Xu
0 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 20:08 UTC (permalink / raw)
To: Axel Rasmussen
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 12:47:39PM -0700, Axel Rasmussen wrote:
> > +#define uffd_test_fail(...) do { \
> > + printf("failed [reason: "); \
> > + printf(__VA_ARGS__); \
> > + printf("]\n"); \
> > + ksft_inc_fail_cnt(); \
> > + } while (0)
> > +
> > +#define uffd_test_skip(...) do { \
> > + printf("skipped [reason: "); \
> > + printf(__VA_ARGS__); \
> > + printf("]\n"); \
> > + ksft_inc_xskip_cnt(); \
> > + } while (0)
>
> Some of this is duplicating what's in kselftest_harness.h - e.g.
> assertions, printing test summary, keeping track of pass/fail/skip
> counts, etc.
>
> I wonder how you feel about using the standard harness? E.g. is the
> plan to do this (more straightforward?) refactor first, and then
> switch later? Or is there some reason it can't be used?
David asked similar question. I wanted to arrange the output mostly for:
(1) one test per line, (2) dump test name before anything (even failing).
The 2nd one is important for uffd tests since it uses err() which bails out
immediately on unexpected failures (comparing to uffd_test_fail).
To achieve that I found I can use ksft_inc_xskip_cnt() etc. so I kept using
the counters in the harness but leave the print format on my own. I assume
that's also why these were exported in kselftests.h so when tests want to
have customized output it can still use the counters.
Thanks,
--
Peter Xu
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH v2 16/31] selftests/mm: UFFDIO_API test
2023-04-12 16:42 ` [PATCH v2 16/31] selftests/mm: UFFDIO_API test Peter Xu
2023-04-12 19:47 ` Axel Rasmussen
@ 2023-04-20 10:54 ` Mike Rapoport
1 sibling, 0 replies; 43+ messages in thread
From: Mike Rapoport @ 2023-04-20 10:54 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, Andrew Morton, Andrea Arcangeli,
Mike Kravetz, Nadav Amit, David Hildenbrand, Mike Rapoport,
Axel Rasmussen, Leonardo Bras Soares Passos
On Wed, Apr 12, 2023 at 12:42:57PM -0400, Peter Xu wrote:
> Add one simple test for UFFDIO_API. With that, I also added a bunch of
> small but handy helpers along the way.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> ---
> tools/testing/selftests/mm/uffd-unit-tests.c | 109 ++++++++++++++++++-
> 1 file changed, 108 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
> index 6857388783be..bb492c258486 100644
> --- a/tools/testing/selftests/mm/uffd-unit-tests.c
> +++ b/tools/testing/selftests/mm/uffd-unit-tests.c
> @@ -9,9 +9,116 @@
>
> #ifdef __NR_userfaultfd
>
> +static void uffd_test_report(void)
> +{
> + printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
> + ksft_get_pass_cnt(),
> + ksft_get_xskip_cnt(),
> + ksft_get_fail_cnt(),
> + ksft_test_num());
> +}
> +
> +static void uffd_test_pass(void)
> +{
> + printf("done\n");
> + ksft_inc_pass_cnt();
> +}
> +
> +#define uffd_test_start(...) do { \
> + printf("Testing "); \
> + printf(__VA_ARGS__); \
> + printf("... "); \
> + fflush(stdout); \
> + } while (0)
> +
> +#define uffd_test_fail(...) do { \
> + printf("failed [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + ksft_inc_fail_cnt(); \
> + } while (0)
> +
> +#define uffd_test_skip(...) do { \
> + printf("skipped [reason: "); \
> + printf(__VA_ARGS__); \
> + printf("]\n"); \
> + ksft_inc_xskip_cnt(); \
> + } while (0)
> +
> +/*
> + * Returns 1 if specific userfaultfd supported, 0 otherwise. Note, we'll
> + * return 1 even if some test failed as long as uffd supported, because in
> + * that case we still want to proceed with the rest uffd unit tests.
> + */
> +static int test_uffd_api(bool use_dev)
> +{
> + struct uffdio_api uffdio_api;
> + int uffd;
> +
> + uffd_test_start("UFFDIO_API (with %s)",
> + use_dev ? "/dev/userfaultfd" : "syscall");
> +
> + if (use_dev)
> + uffd = uffd_open_dev(UFFD_FLAGS);
> + else
> + uffd = uffd_open_sys(UFFD_FLAGS);
> + if (uffd < 0) {
> + uffd_test_skip("cannot open userfaultfd handle");
> + return 0;
> + }
> +
> + /* Test wrong UFFD_API */
> + uffdio_api.api = 0xab;
> + uffdio_api.features = 0;
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should fail with wrong api but didn't");
> + goto out;
> + }
> +
> + /* Test wrong feature bit */
> + uffdio_api.api = UFFD_API;
> + uffdio_api.features = BIT_ULL(63);
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should fail with wrong feature but didn't");
> + goto out;
> + }
> +
> + /* Test normal UFFDIO_API */
> + uffdio_api.api = UFFD_API;
> + uffdio_api.features = 0;
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api)) {
> + uffd_test_fail("UFFDIO_API should succeed but failed");
> + goto out;
> + }
> +
> + /* Test double requests of UFFDIO_API with a random feature set */
> + uffdio_api.features = BIT_ULL(0);
> + if (ioctl(uffd, UFFDIO_API, &uffdio_api) == 0) {
> + uffd_test_fail("UFFDIO_API should reject initialized uffd");
> + goto out;
> + }
> +
> + uffd_test_pass();
> +out:
> + close(uffd);
> + /* We have a valid uffd handle */
> + return 1;
> +}
> +
> int main(int argc, char *argv[])
> {
> - return KSFT_PASS;
> + int has_uffd;
> +
> + has_uffd = test_uffd_api(false);
> + has_uffd |= test_uffd_api(true);
> +
> + if (!has_uffd) {
> + printf("Userfaultfd not supported or unprivileged, skip all tests\n");
> + exit(KSFT_SKIP);
> + }
> + uffd_test_report();
> +
> + return ksft_get_fail_cnt() ? KSFT_FAIL : KSFT_PASS;
> }
>
> #else /* __NR_userfaultfd */
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 17/31] selftests/mm: Drop global mem_fd in uffd tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (15 preceding siblings ...)
2023-04-12 16:42 ` [PATCH v2 16/31] selftests/mm: UFFDIO_API test Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 18/31] selftests/mm: Drop global hpage_size " Peter Xu
` (13 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz, Mike Rapoport
Drop it by creating the memfd dynamically in the tests.
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 28 +++++++++++++++++++++++-
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 15 -------------
3 files changed, 28 insertions(+), 17 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 09ea24c5f02c..b1617f5d4517 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -12,12 +12,32 @@
volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
-int mem_fd, uffd = -1, uffd_flags, finished, *pipefd, test_type;
+int uffd = -1, uffd_flags, finished, *pipefd, test_type;
bool map_shared, test_collapse, test_dev_userfaultfd;
bool test_uffdio_wp = true, test_uffdio_minor = false;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
+static int uffd_mem_fd_create(off_t mem_size, bool hugetlb)
+{
+ unsigned int memfd_flags = 0;
+ int mem_fd;
+
+ if (hugetlb)
+ memfd_flags = MFD_HUGETLB;
+ mem_fd = memfd_create("uffd-test", memfd_flags);
+ if (mem_fd < 0)
+ err("memfd_create");
+ if (ftruncate(mem_fd, mem_size))
+ err("ftruncate");
+ if (fallocate(mem_fd,
+ FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
+ mem_size))
+ err("fallocate");
+
+ return mem_fd;
+}
+
static void anon_release_pages(char *rel_area)
{
if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED))
@@ -51,6 +71,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
off_t offset = is_src ? 0 : size;
void *area_alias = NULL;
char **alloc_area_alias;
+ int mem_fd = uffd_mem_fd_create(size * 2, true);
*alloc_area = mmap(NULL, size, PROT_READ | PROT_WRITE,
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
@@ -73,6 +94,8 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
}
if (area_alias)
*alloc_area_alias = area_alias;
+
+ close(mem_fd);
}
static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -95,6 +118,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
size_t bytes = nr_pages * page_size;
unsigned long offset = is_src ? 0 : bytes;
char *p = NULL, *p_alias = NULL;
+ int mem_fd = uffd_mem_fd_create(bytes * 2, false);
if (test_collapse) {
p = BASE_PMD_ADDR;
@@ -124,6 +148,8 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
area_src_alias = area_alias;
else
area_dst_alias = area_alias;
+
+ close(mem_fd);
}
static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 11f770391bd9..0dfab7057295 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -87,7 +87,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
-extern int mem_fd, uffd, uffd_flags, finished, *pipefd, test_type;
+extern int uffd, uffd_flags, finished, *pipefd, test_type;
extern bool map_shared, test_collapse, test_dev_userfaultfd;
extern bool test_uffdio_wp, test_uffdio_minor;
extern unsigned long long *count_verify;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index e6d39a755082..4eca1a0276c2 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -1090,21 +1090,6 @@ int main(int argc, char **argv)
}
nr_pages = nr_pages_per_cpu * nr_cpus;
- if (test_type == TEST_SHMEM || test_type == TEST_HUGETLB) {
- unsigned int memfd_flags = 0;
-
- if (test_type == TEST_HUGETLB)
- memfd_flags = MFD_HUGETLB;
- mem_fd = memfd_create(argv[0], memfd_flags);
- if (mem_fd < 0)
- err("memfd_create");
- if (ftruncate(mem_fd, nr_pages * page_size * 2))
- err("ftruncate");
- if (fallocate(mem_fd,
- FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0,
- nr_pages * page_size * 2))
- err("fallocate");
- }
printf("nr_pages: %lu, nr_pages_per_cpu: %lu\n",
nr_pages, nr_pages_per_cpu);
return userfaultfd_stress();
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 18/31] selftests/mm: Drop global hpage_size in uffd tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (16 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 17/31] selftests/mm: Drop global mem_fd in uffd tests Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args Peter Xu
` (12 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz, Mike Rapoport
hpage_size was wrongly used. Sometimes it means hugetlb default size,
sometimes it was used as thp size.
Remove the global variable and use the right one at each place.
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 7 ++++---
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 6 +++---
3 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index b1617f5d4517..f02dfcf10714 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -10,7 +10,7 @@
#define BASE_PMD_ADDR ((void *)(1UL << 30))
volatile bool test_uffdio_copy_eexist = true;
-unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
bool map_shared, test_collapse, test_dev_userfaultfd;
@@ -115,7 +115,7 @@ static void shmem_release_pages(char *rel_area)
static void shmem_allocate_area(void **alloc_area, bool is_src)
{
void *area_alias = NULL;
- size_t bytes = nr_pages * page_size;
+ size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
unsigned long offset = is_src ? 0 : bytes;
char *p = NULL, *p_alias = NULL;
int mem_fd = uffd_mem_fd_create(bytes * 2, false);
@@ -159,7 +159,8 @@ static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
static void shmem_check_pmd_mapping(void *p, int expect_nr_hpages)
{
- if (!check_huge_shmem(area_dst_alias, expect_nr_hpages, hpage_size))
+ if (!check_huge_shmem(area_dst_alias, expect_nr_hpages,
+ read_pmd_pagesize()))
err("Did not find expected %d number of hugepages",
expect_nr_hpages);
}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 0dfab7057295..47565b2f2dee 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -85,7 +85,7 @@ struct uffd_test_ops {
};
typedef struct uffd_test_ops uffd_test_ops_t;
-extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size, hpage_size;
+extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
extern bool map_shared, test_collapse, test_dev_userfaultfd;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 4eca1a0276c2..54fc9b4ffa3c 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -655,7 +655,7 @@ static int userfaultfd_minor_test(void)
uffd_test_ops->check_pmd_mapping(area_dst,
nr_pages * page_size /
- hpage_size);
+ read_pmd_pagesize());
/*
* This won't cause uffd-fault - it purely just makes sure there
* was no corruption.
@@ -997,7 +997,7 @@ static void parse_test_type_arg(const char *raw_type)
err("Unsupported test: %s", raw_type);
if (test_type == TEST_HUGETLB)
- page_size = hpage_size;
+ page_size = default_huge_page_size();
else
page_size = sysconf(_SC_PAGE_SIZE);
@@ -1035,6 +1035,7 @@ static void sigalrm(int sig)
int main(int argc, char **argv)
{
size_t bytes;
+ size_t hpage_size = read_pmd_pagesize();
if (argc < 4)
usage();
@@ -1043,7 +1044,6 @@ int main(int argc, char **argv)
err("failed to arm SIGALRM");
alarm(ALARM_INTERVAL_SECS);
- hpage_size = default_huge_page_size();
parse_test_type_arg(argv[1]);
bytes = atol(argv[2]) * 1024 * 1024;
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (17 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 18/31] selftests/mm: Drop global hpage_size " Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-20 10:55 ` Mike Rapoport
2023-04-12 16:43 ` [PATCH v2 20/31] selftests/mm: Let uffd_handle_page_fault() take wp parameter Peter Xu
` (11 subsequent siblings)
30 siblings, 1 reply; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz, Mike Rapoport
Prepare for adding more fields into the struct.
Suggested-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 28 ++++++-------
tools/testing/selftests/mm/uffd-common.h | 6 +--
tools/testing/selftests/mm/uffd-stress.c | 51 ++++++++++++------------
3 files changed, 42 insertions(+), 43 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index f02dfcf10714..e746405aa8f3 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -186,34 +186,34 @@ struct uffd_test_ops hugetlb_uffd_test_ops = {
.check_pmd_mapping = NULL,
};
-void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
+void uffd_stats_report(struct uffd_args *args, int n_cpus)
{
int i;
unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
for (i = 0; i < n_cpus; i++) {
- miss_total += stats[i].missing_faults;
- wp_total += stats[i].wp_faults;
- minor_total += stats[i].minor_faults;
+ miss_total += args[i].missing_faults;
+ wp_total += args[i].wp_faults;
+ minor_total += args[i].minor_faults;
}
printf("userfaults: ");
if (miss_total) {
printf("%llu missing (", miss_total);
for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].missing_faults);
+ printf("%lu+", args[i].missing_faults);
printf("\b) ");
}
if (wp_total) {
printf("%llu wp (", wp_total);
for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].wp_faults);
+ printf("%lu+", args[i].wp_faults);
printf("\b) ");
}
if (minor_total) {
printf("%llu minor (", minor_total);
for (i = 0; i < n_cpus; i++)
- printf("%lu+", stats[i].minor_faults);
+ printf("%lu+", args[i].minor_faults);
printf("\b)");
}
printf("\n");
@@ -397,7 +397,7 @@ int uffd_read_msg(int ufd, struct uffd_msg *msg)
return 0;
}
-void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args)
{
unsigned long offset;
@@ -407,7 +407,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
/* Write protect page faults */
wp_range(uffd, msg->arg.pagefault.address, page_size, false);
- stats->wp_faults++;
+ args->wp_faults++;
} else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
uint8_t *area;
int b;
@@ -430,7 +430,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
for (b = 0; b < page_size; ++b)
area[b] = ~area[b];
continue_range(uffd, msg->arg.pagefault.address, page_size);
- stats->minor_faults++;
+ args->minor_faults++;
} else {
/*
* Missing page faults.
@@ -460,14 +460,14 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
offset &= ~(page_size-1);
if (copy_page(uffd, offset))
- stats->missing_faults++;
+ args->missing_faults++;
}
}
void *uffd_poll_thread(void *arg)
{
- struct uffd_stats *stats = (struct uffd_stats *)arg;
- unsigned long cpu = stats->cpu;
+ struct uffd_args *args = (struct uffd_args *)arg;
+ unsigned long cpu = args->cpu;
struct pollfd pollfd[2];
struct uffd_msg msg;
struct uffdio_register uffd_reg;
@@ -502,7 +502,7 @@ void *uffd_poll_thread(void *arg)
err("unexpected msg event %u\n", msg.event);
break;
case UFFD_EVENT_PAGEFAULT:
- uffd_handle_page_fault(&msg, stats);
+ uffd_handle_page_fault(&msg, args);
break;
case UFFD_EVENT_FORK:
close(uffd);
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 47565b2f2dee..f8d2ad178827 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -70,7 +70,7 @@
- 1)))
/* Userfaultfd test statistics */
-struct uffd_stats {
+struct uffd_args {
int cpu;
unsigned long missing_faults;
unsigned long wp_faults;
@@ -98,12 +98,12 @@ extern uffd_test_ops_t shmem_uffd_test_ops;
extern uffd_test_ops_t hugetlb_uffd_test_ops;
extern uffd_test_ops_t *uffd_test_ops;
-void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
+void uffd_stats_report(struct uffd_args *args, int n_cpus);
void uffd_test_ctx_init(uint64_t features);
void userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
-void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
+void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args);
int __copy_page(int ufd, unsigned long offset, bool retry);
int copy_page(int ufd, unsigned long offset);
void *uffd_poll_thread(void *arg);
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 54fc9b4ffa3c..ce7251ab93ef 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -90,16 +90,15 @@ static void usage(void)
exit(1);
}
-static void uffd_stats_reset(struct uffd_stats *uffd_stats,
- unsigned long n_cpus)
+static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
{
int i;
for (i = 0; i < n_cpus; i++) {
- uffd_stats[i].cpu = i;
- uffd_stats[i].missing_faults = 0;
- uffd_stats[i].wp_faults = 0;
- uffd_stats[i].minor_faults = 0;
+ args[i].cpu = i;
+ args[i].missing_faults = 0;
+ args[i].wp_faults = 0;
+ args[i].minor_faults = 0;
}
}
@@ -163,7 +162,7 @@ pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
static void *uffd_read_thread(void *arg)
{
- struct uffd_stats *stats = (struct uffd_stats *)arg;
+ struct uffd_args *args = (struct uffd_args *)arg;
struct uffd_msg msg;
pthread_mutex_unlock(&uffd_read_mutex);
@@ -172,7 +171,7 @@ static void *uffd_read_thread(void *arg)
for (;;) {
if (uffd_read_msg(uffd, &msg))
continue;
- uffd_handle_page_fault(&msg, stats);
+ uffd_handle_page_fault(&msg, args);
}
return NULL;
@@ -210,7 +209,7 @@ static void *background_thread(void *arg)
return NULL;
}
-static int stress(struct uffd_stats *uffd_stats)
+static int stress(struct uffd_args *args)
{
unsigned long cpu;
pthread_t locking_threads[nr_cpus];
@@ -225,12 +224,12 @@ static int stress(struct uffd_stats *uffd_stats)
if (bounces & BOUNCE_POLL) {
if (pthread_create(&uffd_threads[cpu], &attr,
uffd_poll_thread,
- (void *)&uffd_stats[cpu]))
+ (void *)&args[cpu]))
return 1;
} else {
if (pthread_create(&uffd_threads[cpu], &attr,
uffd_read_thread,
- (void *)&uffd_stats[cpu]))
+ (void *)&args[cpu]))
return 1;
pthread_mutex_lock(&uffd_read_mutex);
}
@@ -264,7 +263,7 @@ static int stress(struct uffd_stats *uffd_stats)
if (write(pipefd[cpu*2+1], &c, 1) != 1)
err("pipefd write error");
if (pthread_join(uffd_threads[cpu],
- (void *)&uffd_stats[cpu]))
+ (void *)&args[cpu]))
return 1;
} else {
if (pthread_cancel(uffd_threads[cpu]))
@@ -493,7 +492,7 @@ static int userfaultfd_events_test(void)
int err, features;
pid_t pid;
char c;
- struct uffd_stats stats = { 0 };
+ struct uffd_args args = { 0 };
printf("testing events (fork, remap, remove): ");
fflush(stdout);
@@ -508,7 +507,7 @@ static int userfaultfd_events_test(void)
true, test_uffdio_wp, false))
err("register failure");
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
+ if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
pid = fork();
@@ -526,9 +525,9 @@ static int userfaultfd_events_test(void)
if (pthread_join(uffd_mon, NULL))
return 1;
- uffd_stats_report(&stats, 1);
+ uffd_stats_report(&args, 1);
- return stats.missing_faults != nr_pages;
+ return args.missing_faults != nr_pages;
}
static int userfaultfd_sig_test(void)
@@ -538,7 +537,7 @@ static int userfaultfd_sig_test(void)
int err, features;
pid_t pid;
char c;
- struct uffd_stats stats = { 0 };
+ struct uffd_args args = { 0 };
printf("testing signal delivery: ");
fflush(stdout);
@@ -557,7 +556,7 @@ static int userfaultfd_sig_test(void)
uffd_test_ops->release_pages(area_dst);
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
+ if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
pid = fork();
@@ -606,7 +605,7 @@ static int userfaultfd_minor_test(void)
unsigned long p;
pthread_t uffd_mon;
char c;
- struct uffd_stats stats = { 0 };
+ struct uffd_args args = { 0 };
if (!test_uffdio_minor)
return 0;
@@ -629,7 +628,7 @@ static int userfaultfd_minor_test(void)
page_size);
}
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
+ if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
/*
@@ -645,7 +644,7 @@ static int userfaultfd_minor_test(void)
if (pthread_join(uffd_mon, NULL))
return 1;
- uffd_stats_report(&stats, 1);
+ uffd_stats_report(&args, 1);
if (test_collapse) {
printf("testing collapse of uffd memory into PMD-mapped THPs:");
@@ -664,7 +663,7 @@ static int userfaultfd_minor_test(void)
printf(" done.\n");
}
- return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
+ return args.missing_faults != 0 || args.minor_faults != nr_pages;
}
static int pagemap_open(void)
@@ -822,7 +821,7 @@ static int userfaultfd_stress(void)
{
void *area;
unsigned long nr;
- struct uffd_stats uffd_stats[nr_cpus];
+ struct uffd_args args[nr_cpus];
uint64_t mem_size = nr_pages * page_size;
uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
@@ -894,10 +893,10 @@ static int userfaultfd_stress(void)
*/
uffd_test_ops->release_pages(area_dst);
- uffd_stats_reset(uffd_stats, nr_cpus);
+ uffd_stats_reset(args, nr_cpus);
/* bounce pass */
- if (stress(uffd_stats))
+ if (stress(args))
return 1;
/* Clear all the write protections if there is any */
@@ -926,7 +925,7 @@ static int userfaultfd_stress(void)
swap(area_src_alias, area_dst_alias);
- uffd_stats_report(uffd_stats, nr_cpus);
+ uffd_stats_report(args, nr_cpus);
}
if (test_type == TEST_ANON) {
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* Re: [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args
2023-04-12 16:43 ` [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args Peter Xu
@ 2023-04-20 10:55 ` Mike Rapoport
0 siblings, 0 replies; 43+ messages in thread
From: Mike Rapoport @ 2023-04-20 10:55 UTC (permalink / raw)
To: Peter Xu
Cc: linux-kernel, linux-mm, David Hildenbrand, Andrew Morton,
Leonardo Bras Soares Passos, Andrea Arcangeli, Nadav Amit,
Mike Rapoport, Axel Rasmussen, Mike Kravetz, Mike Rapoport
On Wed, Apr 12, 2023 at 12:43:37PM -0400, Peter Xu wrote:
> Prepare for adding more fields into the struct.
>
> Suggested-by: Mike Rapoport (IBM) <rppt@kernel.org>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
> ---
> tools/testing/selftests/mm/uffd-common.c | 28 ++++++-------
> tools/testing/selftests/mm/uffd-common.h | 6 +--
> tools/testing/selftests/mm/uffd-stress.c | 51 ++++++++++++------------
> 3 files changed, 42 insertions(+), 43 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
> index f02dfcf10714..e746405aa8f3 100644
> --- a/tools/testing/selftests/mm/uffd-common.c
> +++ b/tools/testing/selftests/mm/uffd-common.c
> @@ -186,34 +186,34 @@ struct uffd_test_ops hugetlb_uffd_test_ops = {
> .check_pmd_mapping = NULL,
> };
>
> -void uffd_stats_report(struct uffd_stats *stats, int n_cpus)
> +void uffd_stats_report(struct uffd_args *args, int n_cpus)
> {
> int i;
> unsigned long long miss_total = 0, wp_total = 0, minor_total = 0;
>
> for (i = 0; i < n_cpus; i++) {
> - miss_total += stats[i].missing_faults;
> - wp_total += stats[i].wp_faults;
> - minor_total += stats[i].minor_faults;
> + miss_total += args[i].missing_faults;
> + wp_total += args[i].wp_faults;
> + minor_total += args[i].minor_faults;
> }
>
> printf("userfaults: ");
> if (miss_total) {
> printf("%llu missing (", miss_total);
> for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].missing_faults);
> + printf("%lu+", args[i].missing_faults);
> printf("\b) ");
> }
> if (wp_total) {
> printf("%llu wp (", wp_total);
> for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].wp_faults);
> + printf("%lu+", args[i].wp_faults);
> printf("\b) ");
> }
> if (minor_total) {
> printf("%llu minor (", minor_total);
> for (i = 0; i < n_cpus; i++)
> - printf("%lu+", stats[i].minor_faults);
> + printf("%lu+", args[i].minor_faults);
> printf("\b)");
> }
> printf("\n");
> @@ -397,7 +397,7 @@ int uffd_read_msg(int ufd, struct uffd_msg *msg)
> return 0;
> }
>
> -void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> +void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args)
> {
> unsigned long offset;
>
> @@ -407,7 +407,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) {
> /* Write protect page faults */
> wp_range(uffd, msg->arg.pagefault.address, page_size, false);
> - stats->wp_faults++;
> + args->wp_faults++;
> } else if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_MINOR) {
> uint8_t *area;
> int b;
> @@ -430,7 +430,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> for (b = 0; b < page_size; ++b)
> area[b] = ~area[b];
> continue_range(uffd, msg->arg.pagefault.address, page_size);
> - stats->minor_faults++;
> + args->minor_faults++;
> } else {
> /*
> * Missing page faults.
> @@ -460,14 +460,14 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats)
> offset &= ~(page_size-1);
>
> if (copy_page(uffd, offset))
> - stats->missing_faults++;
> + args->missing_faults++;
> }
> }
>
> void *uffd_poll_thread(void *arg)
> {
> - struct uffd_stats *stats = (struct uffd_stats *)arg;
> - unsigned long cpu = stats->cpu;
> + struct uffd_args *args = (struct uffd_args *)arg;
> + unsigned long cpu = args->cpu;
> struct pollfd pollfd[2];
> struct uffd_msg msg;
> struct uffdio_register uffd_reg;
> @@ -502,7 +502,7 @@ void *uffd_poll_thread(void *arg)
> err("unexpected msg event %u\n", msg.event);
> break;
> case UFFD_EVENT_PAGEFAULT:
> - uffd_handle_page_fault(&msg, stats);
> + uffd_handle_page_fault(&msg, args);
> break;
> case UFFD_EVENT_FORK:
> close(uffd);
> diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
> index 47565b2f2dee..f8d2ad178827 100644
> --- a/tools/testing/selftests/mm/uffd-common.h
> +++ b/tools/testing/selftests/mm/uffd-common.h
> @@ -70,7 +70,7 @@
> - 1)))
>
> /* Userfaultfd test statistics */
> -struct uffd_stats {
> +struct uffd_args {
> int cpu;
> unsigned long missing_faults;
> unsigned long wp_faults;
> @@ -98,12 +98,12 @@ extern uffd_test_ops_t shmem_uffd_test_ops;
> extern uffd_test_ops_t hugetlb_uffd_test_ops;
> extern uffd_test_ops_t *uffd_test_ops;
>
> -void uffd_stats_report(struct uffd_stats *stats, int n_cpus);
> +void uffd_stats_report(struct uffd_args *args, int n_cpus);
> void uffd_test_ctx_init(uint64_t features);
> void userfaultfd_open(uint64_t *features);
> int uffd_read_msg(int ufd, struct uffd_msg *msg);
> void wp_range(int ufd, __u64 start, __u64 len, bool wp);
> -void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_stats *stats);
> +void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args);
> int __copy_page(int ufd, unsigned long offset, bool retry);
> int copy_page(int ufd, unsigned long offset);
> void *uffd_poll_thread(void *arg);
> diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
> index 54fc9b4ffa3c..ce7251ab93ef 100644
> --- a/tools/testing/selftests/mm/uffd-stress.c
> +++ b/tools/testing/selftests/mm/uffd-stress.c
> @@ -90,16 +90,15 @@ static void usage(void)
> exit(1);
> }
>
> -static void uffd_stats_reset(struct uffd_stats *uffd_stats,
> - unsigned long n_cpus)
> +static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
> {
> int i;
>
> for (i = 0; i < n_cpus; i++) {
> - uffd_stats[i].cpu = i;
> - uffd_stats[i].missing_faults = 0;
> - uffd_stats[i].wp_faults = 0;
> - uffd_stats[i].minor_faults = 0;
> + args[i].cpu = i;
> + args[i].missing_faults = 0;
> + args[i].wp_faults = 0;
> + args[i].minor_faults = 0;
> }
> }
>
> @@ -163,7 +162,7 @@ pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
>
> static void *uffd_read_thread(void *arg)
> {
> - struct uffd_stats *stats = (struct uffd_stats *)arg;
> + struct uffd_args *args = (struct uffd_args *)arg;
> struct uffd_msg msg;
>
> pthread_mutex_unlock(&uffd_read_mutex);
> @@ -172,7 +171,7 @@ static void *uffd_read_thread(void *arg)
> for (;;) {
> if (uffd_read_msg(uffd, &msg))
> continue;
> - uffd_handle_page_fault(&msg, stats);
> + uffd_handle_page_fault(&msg, args);
> }
>
> return NULL;
> @@ -210,7 +209,7 @@ static void *background_thread(void *arg)
> return NULL;
> }
>
> -static int stress(struct uffd_stats *uffd_stats)
> +static int stress(struct uffd_args *args)
> {
> unsigned long cpu;
> pthread_t locking_threads[nr_cpus];
> @@ -225,12 +224,12 @@ static int stress(struct uffd_stats *uffd_stats)
> if (bounces & BOUNCE_POLL) {
> if (pthread_create(&uffd_threads[cpu], &attr,
> uffd_poll_thread,
> - (void *)&uffd_stats[cpu]))
> + (void *)&args[cpu]))
> return 1;
> } else {
> if (pthread_create(&uffd_threads[cpu], &attr,
> uffd_read_thread,
> - (void *)&uffd_stats[cpu]))
> + (void *)&args[cpu]))
> return 1;
> pthread_mutex_lock(&uffd_read_mutex);
> }
> @@ -264,7 +263,7 @@ static int stress(struct uffd_stats *uffd_stats)
> if (write(pipefd[cpu*2+1], &c, 1) != 1)
> err("pipefd write error");
> if (pthread_join(uffd_threads[cpu],
> - (void *)&uffd_stats[cpu]))
> + (void *)&args[cpu]))
> return 1;
> } else {
> if (pthread_cancel(uffd_threads[cpu]))
> @@ -493,7 +492,7 @@ static int userfaultfd_events_test(void)
> int err, features;
> pid_t pid;
> char c;
> - struct uffd_stats stats = { 0 };
> + struct uffd_args args = { 0 };
>
> printf("testing events (fork, remap, remove): ");
> fflush(stdout);
> @@ -508,7 +507,7 @@ static int userfaultfd_events_test(void)
> true, test_uffdio_wp, false))
> err("register failure");
>
> - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
> err("uffd_poll_thread create");
>
> pid = fork();
> @@ -526,9 +525,9 @@ static int userfaultfd_events_test(void)
> if (pthread_join(uffd_mon, NULL))
> return 1;
>
> - uffd_stats_report(&stats, 1);
> + uffd_stats_report(&args, 1);
>
> - return stats.missing_faults != nr_pages;
> + return args.missing_faults != nr_pages;
> }
>
> static int userfaultfd_sig_test(void)
> @@ -538,7 +537,7 @@ static int userfaultfd_sig_test(void)
> int err, features;
> pid_t pid;
> char c;
> - struct uffd_stats stats = { 0 };
> + struct uffd_args args = { 0 };
>
> printf("testing signal delivery: ");
> fflush(stdout);
> @@ -557,7 +556,7 @@ static int userfaultfd_sig_test(void)
>
> uffd_test_ops->release_pages(area_dst);
>
> - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
> err("uffd_poll_thread create");
>
> pid = fork();
> @@ -606,7 +605,7 @@ static int userfaultfd_minor_test(void)
> unsigned long p;
> pthread_t uffd_mon;
> char c;
> - struct uffd_stats stats = { 0 };
> + struct uffd_args args = { 0 };
>
> if (!test_uffdio_minor)
> return 0;
> @@ -629,7 +628,7 @@ static int userfaultfd_minor_test(void)
> page_size);
> }
>
> - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats))
> + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
> err("uffd_poll_thread create");
>
> /*
> @@ -645,7 +644,7 @@ static int userfaultfd_minor_test(void)
> if (pthread_join(uffd_mon, NULL))
> return 1;
>
> - uffd_stats_report(&stats, 1);
> + uffd_stats_report(&args, 1);
>
> if (test_collapse) {
> printf("testing collapse of uffd memory into PMD-mapped THPs:");
> @@ -664,7 +663,7 @@ static int userfaultfd_minor_test(void)
> printf(" done.\n");
> }
>
> - return stats.missing_faults != 0 || stats.minor_faults != nr_pages;
> + return args.missing_faults != 0 || args.minor_faults != nr_pages;
> }
>
> static int pagemap_open(void)
> @@ -822,7 +821,7 @@ static int userfaultfd_stress(void)
> {
> void *area;
> unsigned long nr;
> - struct uffd_stats uffd_stats[nr_cpus];
> + struct uffd_args args[nr_cpus];
> uint64_t mem_size = nr_pages * page_size;
>
> uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
> @@ -894,10 +893,10 @@ static int userfaultfd_stress(void)
> */
> uffd_test_ops->release_pages(area_dst);
>
> - uffd_stats_reset(uffd_stats, nr_cpus);
> + uffd_stats_reset(args, nr_cpus);
>
> /* bounce pass */
> - if (stress(uffd_stats))
> + if (stress(args))
> return 1;
>
> /* Clear all the write protections if there is any */
> @@ -926,7 +925,7 @@ static int userfaultfd_stress(void)
>
> swap(area_src_alias, area_dst_alias);
>
> - uffd_stats_report(uffd_stats, nr_cpus);
> + uffd_stats_report(args, nr_cpus);
> }
>
> if (test_type == TEST_ANON) {
> --
> 2.39.1
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH v2 20/31] selftests/mm: Let uffd_handle_page_fault() take wp parameter
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (18 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 19/31] selftests/mm: Rename uffd_stats to uffd_args Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 21/31] selftests/mm: Allow allocate_area() to fail properly Peter Xu
` (10 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz
Make the handler optionally apply WP bit when resolving page faults for
either missing or minor page faults. This moves towards removing global
test_uffdio_wp outside of the common code.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 17 +++++++++--------
tools/testing/selftests/mm/uffd-common.h | 6 ++++--
tools/testing/selftests/mm/uffd-stress.c | 16 ++++++++++------
3 files changed, 23 insertions(+), 16 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index e746405aa8f3..daa2a95408e6 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -353,7 +353,7 @@ void wp_range(int ufd, __u64 start, __u64 len, bool wp)
err("clear WP failed: address=0x%"PRIx64, (uint64_t)start);
}
-static void continue_range(int ufd, __u64 start, __u64 len)
+static void continue_range(int ufd, __u64 start, __u64 len, bool wp)
{
struct uffdio_continue req;
int ret;
@@ -361,7 +361,7 @@ static void continue_range(int ufd, __u64 start, __u64 len)
req.range.start = start;
req.range.len = len;
req.mode = 0;
- if (test_uffdio_wp)
+ if (wp)
req.mode |= UFFDIO_CONTINUE_MODE_WP;
if (ioctl(ufd, UFFDIO_CONTINUE, &req))
@@ -429,7 +429,8 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args)
area_dst_alias));
for (b = 0; b < page_size; ++b)
area[b] = ~area[b];
- continue_range(uffd, msg->arg.pagefault.address, page_size);
+ continue_range(uffd, msg->arg.pagefault.address, page_size,
+ args->apply_wp);
args->minor_faults++;
} else {
/*
@@ -459,7 +460,7 @@ void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args)
offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst;
offset &= ~(page_size-1);
- if (copy_page(uffd, offset))
+ if (copy_page(uffd, offset, args->apply_wp))
args->missing_faults++;
}
}
@@ -555,7 +556,7 @@ static void wake_range(int ufd, unsigned long addr, unsigned long len)
addr), exit(1);
}
-int __copy_page(int ufd, unsigned long offset, bool retry)
+int __copy_page(int ufd, unsigned long offset, bool retry, bool wp)
{
struct uffdio_copy uffdio_copy;
@@ -564,7 +565,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
uffdio_copy.dst = (unsigned long) area_dst + offset;
uffdio_copy.src = (unsigned long) area_src + offset;
uffdio_copy.len = page_size;
- if (test_uffdio_wp)
+ if (wp)
uffdio_copy.mode = UFFDIO_COPY_MODE_WP;
else
uffdio_copy.mode = 0;
@@ -587,7 +588,7 @@ int __copy_page(int ufd, unsigned long offset, bool retry)
return 0;
}
-int copy_page(int ufd, unsigned long offset)
+int copy_page(int ufd, unsigned long offset, bool wp)
{
- return __copy_page(ufd, offset, false);
+ return __copy_page(ufd, offset, false, wp);
}
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index f8d2ad178827..0ec07d025cfe 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -72,6 +72,8 @@
/* Userfaultfd test statistics */
struct uffd_args {
int cpu;
+ /* Whether apply wr-protects when installing pages */
+ bool apply_wp;
unsigned long missing_faults;
unsigned long wp_faults;
unsigned long minor_faults;
@@ -104,8 +106,8 @@ void userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args);
-int __copy_page(int ufd, unsigned long offset, bool retry);
-int copy_page(int ufd, unsigned long offset);
+int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
+int copy_page(int ufd, unsigned long offset, bool wp);
void *uffd_poll_thread(void *arg);
#define TEST_ANON 1
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index ce7251ab93ef..747d588c0d69 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -96,6 +96,7 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
for (i = 0; i < n_cpus; i++) {
args[i].cpu = i;
+ args[i].apply_wp = test_uffdio_wp;
args[i].missing_faults = 0;
args[i].wp_faults = 0;
args[i].minor_faults = 0;
@@ -155,7 +156,7 @@ static void *locking_thread(void *arg)
static int copy_page_retry(int ufd, unsigned long offset)
{
- return __copy_page(ufd, offset, true);
+ return __copy_page(ufd, offset, true, test_uffdio_wp);
}
pthread_mutex_t uffd_read_mutex = PTHREAD_MUTEX_INITIALIZER;
@@ -308,7 +309,7 @@ static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
* This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
* feature. Using monitor thread, verify no userfault events are generated.
*/
-static int faulting_process(int signal_test)
+static int faulting_process(int signal_test, bool wp)
{
unsigned long nr;
unsigned long long count;
@@ -343,7 +344,7 @@ static int faulting_process(int signal_test)
if (steps == 1) {
/* This is a MISSING request */
steps++;
- if (copy_page(uffd, offset))
+ if (copy_page(uffd, offset, wp))
signalled++;
} else {
/* This is a WP request */
@@ -507,6 +508,7 @@ static int userfaultfd_events_test(void)
true, test_uffdio_wp, false))
err("register failure");
+ args.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
@@ -515,7 +517,7 @@ static int userfaultfd_events_test(void)
err("fork");
if (!pid)
- exit(faulting_process(0));
+ exit(faulting_process(0, test_uffdio_wp));
waitpid(pid, &err, 0);
if (err)
@@ -551,11 +553,12 @@ static int userfaultfd_sig_test(void)
true, test_uffdio_wp, false))
err("register failure");
- if (faulting_process(1))
+ if (faulting_process(1, test_uffdio_wp))
err("faulting process failed");
uffd_test_ops->release_pages(area_dst);
+ args.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
@@ -564,7 +567,7 @@ static int userfaultfd_sig_test(void)
err("fork");
if (!pid)
- exit(faulting_process(2));
+ exit(faulting_process(2, test_uffdio_wp));
waitpid(pid, &err, 0);
if (err)
@@ -628,6 +631,7 @@ static int userfaultfd_minor_test(void)
page_size);
}
+ args.apply_wp = test_uffdio_wp;
if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
err("uffd_poll_thread create");
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 21/31] selftests/mm: Allow allocate_area() to fail properly
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (19 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 20/31] selftests/mm: Let uffd_handle_page_fault() take wp parameter Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 22/31] selftests/mm: Add framework for uffd-unit-test Peter Xu
` (9 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz
Mostly to detect hugetlb allocation errors and skip hugetlb tests when
pages are not allocated.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 47 +++++++++++++++++-------
tools/testing/selftests/mm/uffd-common.h | 4 +-
2 files changed, 36 insertions(+), 15 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index daa2a95408e6..bc6c5c38d6dd 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -44,10 +44,15 @@ static void anon_release_pages(char *rel_area)
err("madvise(MADV_DONTNEED) failed");
}
-static void anon_allocate_area(void **alloc_area, bool is_src)
+static int anon_allocate_area(void **alloc_area, bool is_src)
{
*alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (*alloc_area == MAP_FAILED) {
+ *alloc_area = NULL;
+ return -errno;
+ }
+ return 0;
}
static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -65,7 +70,7 @@ static void hugetlb_release_pages(char *rel_area)
}
}
-static void hugetlb_allocate_area(void **alloc_area, bool is_src)
+static int hugetlb_allocate_area(void **alloc_area, bool is_src)
{
off_t size = nr_pages * page_size;
off_t offset = is_src ? 0 : size;
@@ -77,14 +82,16 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
(is_src ? 0 : MAP_NORESERVE),
mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of hugetlbfs file failed");
+ if (*alloc_area == MAP_FAILED) {
+ *alloc_area = NULL;
+ return -errno;
+ }
if (map_shared) {
area_alias = mmap(NULL, size, PROT_READ | PROT_WRITE,
MAP_SHARED, mem_fd, offset);
if (area_alias == MAP_FAILED)
- err("mmap of hugetlb file alias failed");
+ return -errno;
}
if (is_src) {
@@ -96,6 +103,7 @@ static void hugetlb_allocate_area(void **alloc_area, bool is_src)
*alloc_area_alias = area_alias;
close(mem_fd);
+ return 0;
}
static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -112,7 +120,7 @@ static void shmem_release_pages(char *rel_area)
err("madvise(MADV_REMOVE) failed");
}
-static void shmem_allocate_area(void **alloc_area, bool is_src)
+static int shmem_allocate_area(void **alloc_area, bool is_src)
{
void *area_alias = NULL;
size_t bytes = nr_pages * page_size, hpage_size = read_pmd_pagesize();
@@ -132,15 +140,20 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
*alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
mem_fd, offset);
- if (*alloc_area == MAP_FAILED)
- err("mmap of memfd failed");
+ if (*alloc_area == MAP_FAILED) {
+ *alloc_area = NULL;
+ return -errno;
+ }
if (test_collapse && *alloc_area != p)
err("mmap of memfd failed at %p", p);
area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
mem_fd, offset);
- if (area_alias == MAP_FAILED)
- err("mmap of memfd alias failed");
+ if (area_alias == MAP_FAILED) {
+ munmap(*alloc_area, bytes);
+ *alloc_area = NULL;
+ return -errno;
+ }
if (test_collapse && area_alias != p_alias)
err("mmap of anonymous memory failed at %p", p_alias);
@@ -150,6 +163,7 @@ static void shmem_allocate_area(void **alloc_area, bool is_src)
area_dst_alias = area_alias;
close(mem_fd);
+ return 0;
}
static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset)
@@ -282,14 +296,19 @@ static void uffd_test_ctx_clear(void)
munmap_area((void **)&area_remap);
}
-void uffd_test_ctx_init(uint64_t features)
+int uffd_test_ctx_init(uint64_t features)
{
unsigned long nr, cpu;
+ int ret;
uffd_test_ctx_clear();
- uffd_test_ops->allocate_area((void **)&area_src, true);
- uffd_test_ops->allocate_area((void **)&area_dst, false);
+ ret = uffd_test_ops->allocate_area((void **)&area_src, true);
+ if (ret)
+ return ret;
+ ret = uffd_test_ops->allocate_area((void **)&area_dst, false);
+ if (ret)
+ return ret;
userfaultfd_open(&features);
@@ -337,6 +356,8 @@ void uffd_test_ctx_init(uint64_t features)
for (cpu = 0; cpu < nr_cpus; cpu++)
if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK))
err("pipe");
+
+ return 0;
}
void wp_range(int ufd, __u64 start, __u64 len, bool wp)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 0ec07d025cfe..9479a0649d7f 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -80,7 +80,7 @@ struct uffd_args {
};
struct uffd_test_ops {
- void (*allocate_area)(void **alloc_area, bool is_src);
+ int (*allocate_area)(void **alloc_area, bool is_src);
void (*release_pages)(char *rel_area);
void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset);
void (*check_pmd_mapping)(void *p, int expect_nr_hpages);
@@ -101,7 +101,7 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;
extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_args *args, int n_cpus);
-void uffd_test_ctx_init(uint64_t features);
+int uffd_test_ctx_init(uint64_t features);
void userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 22/31] selftests/mm: Add framework for uffd-unit-test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (20 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 21/31] selftests/mm: Allow allocate_area() to fail properly Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 23/31] selftests/mm: Move uffd pagemap test to unit test Peter Xu
` (8 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz
Add a framework to be prepared to move unit tests from uffd-stress.c into
uffd-unit-tests.c. The goal is to allow detection of uffd features for
each test, and also loop over specified types of memory that a test support.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 124 +++++++++++++++++++
tools/testing/selftests/mm/vm_util.c | 37 ++++++
tools/testing/selftests/mm/vm_util.h | 2 +
3 files changed, 163 insertions(+)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index bb492c258486..936b24a6f468 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -9,6 +9,66 @@
#ifdef __NR_userfaultfd
+/* The unit test doesn't need a large or random size, make it 32MB for now */
+#define UFFD_TEST_MEM_SIZE (32UL << 20)
+
+#define MEM_ANON BIT_ULL(0)
+#define MEM_SHMEM BIT_ULL(1)
+#define MEM_SHMEM_PRIVATE BIT_ULL(2)
+#define MEM_HUGETLB BIT_ULL(3)
+#define MEM_HUGETLB_PRIVATE BIT_ULL(4)
+
+struct mem_type {
+ const char *name;
+ unsigned int mem_flag;
+ uffd_test_ops_t *mem_ops;
+ bool shared;
+};
+typedef struct mem_type mem_type_t;
+
+mem_type_t mem_types[] = {
+ {
+ .name = "anon",
+ .mem_flag = MEM_ANON,
+ .mem_ops = &anon_uffd_test_ops,
+ .shared = false,
+ },
+ {
+ .name = "shmem",
+ .mem_flag = MEM_SHMEM,
+ .mem_ops = &shmem_uffd_test_ops,
+ .shared = true,
+ },
+ {
+ .name = "shmem-private",
+ .mem_flag = MEM_SHMEM_PRIVATE,
+ .mem_ops = &shmem_uffd_test_ops,
+ .shared = false,
+ },
+ {
+ .name = "hugetlb",
+ .mem_flag = MEM_HUGETLB,
+ .mem_ops = &hugetlb_uffd_test_ops,
+ .shared = true,
+ },
+ {
+ .name = "hugetlb-private",
+ .mem_flag = MEM_HUGETLB_PRIVATE,
+ .mem_ops = &hugetlb_uffd_test_ops,
+ .shared = false,
+ },
+};
+
+/* Returns: UFFD_TEST_* */
+typedef void (*uffd_test_fn)(void);
+
+typedef struct {
+ const char *name;
+ uffd_test_fn uffd_fn;
+ unsigned int mem_targets;
+ uint64_t uffd_feature_required;
+} uffd_test_case_t;
+
static void uffd_test_report(void)
{
printf("Userfaults unit tests: pass=%u, skip=%u, fail=%u (total=%u)\n",
@@ -105,9 +165,50 @@ static int test_uffd_api(bool use_dev)
return 1;
}
+/*
+ * This function initializes the global variables. TODO: remove global
+ * vars and then remove this.
+ */
+static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
+{
+ map_shared = mem_type->shared;
+ uffd_test_ops = mem_type->mem_ops;
+
+ if (mem_type->mem_flag & (MEM_HUGETLB_PRIVATE | MEM_HUGETLB))
+ page_size = default_huge_page_size();
+ else
+ page_size = psize();
+
+ nr_pages = UFFD_TEST_MEM_SIZE / page_size;
+ /* TODO: remove this global var.. it's so ugly */
+ nr_cpus = 1;
+
+ return uffd_test_ctx_init(test->uffd_feature_required);
+}
+
+static bool uffd_feature_supported(uffd_test_case_t *test)
+{
+ uint64_t features;
+
+ if (uffd_get_features(&features))
+ return false;
+
+ return (features & test->uffd_feature_required) ==
+ test->uffd_feature_required;
+}
+
+uffd_test_case_t uffd_tests[] = {
+};
+
int main(int argc, char *argv[])
{
+ int n_tests = sizeof(uffd_tests) / sizeof(uffd_test_case_t);
+ int n_mems = sizeof(mem_types) / sizeof(mem_type_t);
+ uffd_test_case_t *test;
+ mem_type_t *mem_type;
+ char test_name[128];
int has_uffd;
+ int i, j;
has_uffd = test_uffd_api(false);
has_uffd |= test_uffd_api(true);
@@ -116,6 +217,29 @@ int main(int argc, char *argv[])
printf("Userfaultfd not supported or unprivileged, skip all tests\n");
exit(KSFT_SKIP);
}
+
+ for (i = 0; i < n_tests; i++) {
+ test = &uffd_tests[i];
+ for (j = 0; j < n_mems; j++) {
+ mem_type = &mem_types[j];
+ if (!(test->mem_targets & mem_type->mem_flag))
+ continue;
+ snprintf(test_name, sizeof(test_name),
+ "%s on %s", test->name, mem_type->name);
+
+ uffd_test_start(test_name);
+ if (!uffd_feature_supported(test)) {
+ uffd_test_skip("feature missing");
+ continue;
+ }
+ if (uffd_setup_environment(test, mem_type)) {
+ uffd_test_skip("environment setup failed");
+ continue;
+ }
+ test->uffd_fn();
+ }
+ }
+
uffd_test_report();
return ksft_get_fail_cnt() ? KSFT_FAIL : KSFT_PASS;
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 5ee6c4688a7c..1bc0ceb01adb 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -254,3 +254,40 @@ int uffd_open_sys(unsigned int flags)
return -1;
#endif
}
+
+int uffd_open(unsigned int flags)
+{
+ int uffd = uffd_open_sys(flags);
+
+ if (uffd < 0)
+ uffd = uffd_open_dev(flags);
+
+ return uffd;
+}
+
+int uffd_get_features(uint64_t *features)
+{
+ struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+ /*
+ * This should by default work in most kernels; the feature list
+ * will be the same no matter what we pass in here.
+ */
+ int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+ if (fd < 0)
+ /* Maybe the kernel is older than user-only mode? */
+ fd = uffd_open(0);
+
+ if (fd < 0)
+ return fd;
+
+ if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+ close(fd);
+ return -errno;
+ }
+
+ *features = uffdio_api.features;
+ close(fd);
+
+ return 0;
+}
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 481354141533..634eb2f41145 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -50,6 +50,8 @@ int uffd_register(int uffd, void *addr, uint64_t len,
int uffd_unregister(int uffd, void *addr, uint64_t len);
int uffd_open_dev(unsigned int flags);
int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_get_features(uint64_t *features);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 23/31] selftests/mm: Move uffd pagemap test to unit test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (21 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 22/31] selftests/mm: Add framework for uffd-unit-test Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:43 ` [PATCH v2 24/31] selftests/mm: Move uffd minor " Peter Xu
` (7 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz, Mike Rapoport
Move it over and make it split into two tests, one for pagemap and one for
the new WP_UNPOPULATED (to be a separate one).
The thp pagemap test wasn't really working (with MADV_HUGEPAGE). Let's
just drop it (since it never really worked anyway..) and leave that for
later.
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-stress.c | 166 -------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 145 ++++++++++++++++
2 files changed, 145 insertions(+), 166 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 747d588c0d69..61d025d87bf2 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -670,157 +670,6 @@ static int userfaultfd_minor_test(void)
return args.missing_faults != 0 || args.minor_faults != nr_pages;
}
-static int pagemap_open(void)
-{
- int fd = open("/proc/self/pagemap", O_RDONLY);
-
- if (fd < 0)
- err("open pagemap");
-
- return fd;
-}
-
-/* This macro let __LINE__ works in err() */
-#define pagemap_check_wp(value, wp) do { \
- if (!!(value & PM_UFFD_WP) != wp) \
- err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
- } while (0)
-
-static int pagemap_test_fork(bool present)
-{
- pid_t child = fork();
- uint64_t value;
- int fd, result;
-
- if (!child) {
- /* Open the pagemap fd of the child itself */
- fd = pagemap_open();
- value = pagemap_get_entry(fd, area_dst);
- /*
- * After fork() uffd-wp bit should be gone as long as we're
- * without UFFD_FEATURE_EVENT_FORK
- */
- pagemap_check_wp(value, false);
- /* Succeed */
- exit(0);
- }
- waitpid(child, &result, 0);
- return result;
-}
-
-static void userfaultfd_wp_unpopulated_test(int pagemap_fd)
-{
- uint64_t value;
-
- /* Test applying pte marker to anon unpopulated */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
-
- /* Test unprotect on anon pte marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Test zap on anon marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Test fault in after marker removed */
- *area_dst = 1;
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
- /* Drop it to make pte none again */
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-
- /* Test read-zero-page upon pte marker */
- wp_range(uffd, (uint64_t)area_dst, page_size, true);
- *(volatile char *)area_dst;
- /* Drop it to make pte none again */
- if (madvise(area_dst, page_size, MADV_DONTNEED))
- err("madvise(MADV_DONTNEED) failed");
-}
-
-static void userfaultfd_pagemap_test(unsigned int test_pgsize)
-{
- int pagemap_fd;
- uint64_t value;
-
- /* Pagemap tests uffd-wp only */
- if (!test_uffdio_wp)
- return;
-
- /* Not enough memory to test this page size */
- if (test_pgsize > nr_pages * page_size)
- return;
-
- printf("testing uffd-wp with pagemap (pgsize=%u): ", test_pgsize);
- /* Flush so it doesn't flush twice in parent/child later */
- fflush(stdout);
-
- uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
-
- if (test_pgsize > page_size) {
- /* This is a thp test */
- if (madvise(area_dst, nr_pages * page_size, MADV_HUGEPAGE))
- err("madvise(MADV_HUGEPAGE) failed");
- } else if (test_pgsize == page_size) {
- /* This is normal page test; force no thp */
- if (madvise(area_dst, nr_pages * page_size, MADV_NOHUGEPAGE))
- err("madvise(MADV_NOHUGEPAGE) failed");
- }
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- false, true, false))
- err("register failed");
-
- pagemap_fd = pagemap_open();
-
- /* Smoke test WP_UNPOPULATED first when it's still empty */
- if (test_pgsize == page_size)
- userfaultfd_wp_unpopulated_test(pagemap_fd);
-
- /* Touch the page */
- *area_dst = 1;
- wp_range(uffd, (uint64_t)area_dst, test_pgsize, true);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
- /* Make sure uffd-wp bit dropped when fork */
- if (pagemap_test_fork(true))
- err("Detected stall uffd-wp bit in child");
-
- /* Exclusive required or PAGEOUT won't work */
- if (!(value & PM_MMAP_EXCLUSIVE))
- err("multiple mapping detected: 0x%"PRIx64, value);
-
- if (madvise(area_dst, test_pgsize, MADV_PAGEOUT))
- err("madvise(MADV_PAGEOUT) failed");
-
- /* Uffd-wp should persist even swapped out */
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, true);
- /* Make sure uffd-wp bit dropped when fork */
- if (pagemap_test_fork(false))
- err("Detected stall uffd-wp bit in child");
-
- /* Unprotect; this tests swap pte modifications */
- wp_range(uffd, (uint64_t)area_dst, page_size, false);
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- /* Fault in the page from disk */
- *area_dst = 2;
- value = pagemap_get_entry(pagemap_fd, area_dst);
- pagemap_check_wp(value, false);
-
- close(pagemap_fd);
- printf("done\n");
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -932,21 +781,6 @@ static int userfaultfd_stress(void)
uffd_stats_report(args, nr_cpus);
}
- if (test_type == TEST_ANON) {
- /*
- * shmem/hugetlb won't be able to run since they have different
- * behavior on fork() (file-backed memory normally drops ptes
- * directly when fork), meanwhile the pagemap test will verify
- * pgtable entry of fork()ed child.
- */
- userfaultfd_pagemap_test(page_size);
- /*
- * Hard-code for x86_64 for now for 2M THP, as x86_64 is
- * currently the only one that supports uffd-wp
- */
- userfaultfd_pagemap_test(page_size * 512);
- }
-
return userfaultfd_zeropage_test() || userfaultfd_sig_test()
|| userfaultfd_events_test() || userfaultfd_minor_test();
}
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 936b24a6f468..4690c95a9420 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -197,7 +197,152 @@ static bool uffd_feature_supported(uffd_test_case_t *test)
test->uffd_feature_required;
}
+static int pagemap_open(void)
+{
+ int fd = open("/proc/self/pagemap", O_RDONLY);
+
+ if (fd < 0)
+ err("open pagemap");
+
+ return fd;
+}
+
+/* This macro let __LINE__ works in err() */
+#define pagemap_check_wp(value, wp) do { \
+ if (!!(value & PM_UFFD_WP) != wp) \
+ err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \
+ } while (0)
+
+static int pagemap_test_fork(bool present)
+{
+ pid_t child = fork();
+ uint64_t value;
+ int fd, result;
+
+ if (!child) {
+ /* Open the pagemap fd of the child itself */
+ fd = pagemap_open();
+ value = pagemap_get_entry(fd, area_dst);
+ /*
+ * After fork() uffd-wp bit should be gone as long as we're
+ * without UFFD_FEATURE_EVENT_FORK
+ */
+ pagemap_check_wp(value, false);
+ /* Succeed */
+ exit(0);
+ }
+ waitpid(child, &result, 0);
+ return result;
+}
+
+static void uffd_wp_unpopulated_test(void)
+{
+ uint64_t value;
+ int pagemap_fd;
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
+ err("register failed");
+
+ pagemap_fd = pagemap_open();
+
+ /* Test applying pte marker to anon unpopulated */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+
+ /* Test unprotect on anon pte marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, false);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Test zap on anon marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Test fault in after marker removed */
+ *area_dst = 1;
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+ /* Drop it to make pte none again */
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+
+ /* Test read-zero-page upon pte marker */
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ *(volatile char *)area_dst;
+ /* Drop it to make pte none again */
+ if (madvise(area_dst, page_size, MADV_DONTNEED))
+ err("madvise(MADV_DONTNEED) failed");
+
+ uffd_test_pass();
+}
+
+static void uffd_pagemap_test(void)
+{
+ int pagemap_fd;
+ uint64_t value;
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ false, true, false))
+ err("register failed");
+
+ pagemap_fd = pagemap_open();
+
+ /* Touch the page */
+ *area_dst = 1;
+ wp_range(uffd, (uint64_t)area_dst, page_size, true);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+ /* Make sure uffd-wp bit dropped when fork */
+ if (pagemap_test_fork(true))
+ err("Detected stall uffd-wp bit in child");
+
+ /* Exclusive required or PAGEOUT won't work */
+ if (!(value & PM_MMAP_EXCLUSIVE))
+ err("multiple mapping detected: 0x%"PRIx64, value);
+
+ if (madvise(area_dst, page_size, MADV_PAGEOUT))
+ err("madvise(MADV_PAGEOUT) failed");
+
+ /* Uffd-wp should persist even swapped out */
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, true);
+ /* Make sure uffd-wp bit dropped when fork */
+ if (pagemap_test_fork(false))
+ err("Detected stall uffd-wp bit in child");
+
+ /* Unprotect; this tests swap pte modifications */
+ wp_range(uffd, (uint64_t)area_dst, page_size, false);
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ /* Fault in the page from disk */
+ *area_dst = 2;
+ value = pagemap_get_entry(pagemap_fd, area_dst);
+ pagemap_check_wp(value, false);
+
+ close(pagemap_fd);
+ uffd_test_pass();
+}
+
uffd_test_case_t uffd_tests[] = {
+ {
+ .name = "pagemap",
+ .uffd_fn = uffd_pagemap_test,
+ .mem_targets = MEM_ANON,
+ .uffd_feature_required = UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "wp-unpopulated",
+ .uffd_fn = uffd_wp_unpopulated_test,
+ .mem_targets = MEM_ANON,
+ .uffd_feature_required =
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_UNPOPULATED,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 24/31] selftests/mm: Move uffd minor test to unit test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (22 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 23/31] selftests/mm: Move uffd pagemap test to unit test Peter Xu
@ 2023-04-12 16:43 ` Peter Xu
2023-04-12 16:44 ` [PATCH v2 25/31] selftests/mm: Move uffd sig/events tests into uffd unit tests Peter Xu
` (6 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:43 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz, Zach O'Keefe
This moves the minor test to the new unit test.
Rewrite the content check with char* opeartions to avoid fiddling with
my_bcmp().
Drop global vars test_uffdio_minor and test_collapse, just assume test them
always in common code for now.
OTOH make this single test into five tests:
- minor test on [shmem, hugetlb] with wp=false
- minor test on [shmem, hugetlb] with wp=true
- minor test + collapse on shmem only
One thing to mention that we used to test COLLAPSE+WP but that doesn't
sound right at all. It's possible it's silently broken but unnoticed
because COLLAPSE is not part of the default test suite.
Make the MADV_COLLAPSE test fail-able (by skip it when failing), because
it's not guaranteed to success anyway.
Drop a bunch of useless code after the move, because the unit test always
use aligned num of pages and has nothing to do with n_cpus.
Cc: Zach O'Keefe <zokeefe@google.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 25 ++--
tools/testing/selftests/mm/uffd-common.h | 4 +-
tools/testing/selftests/mm/uffd-stress.c | 131 +------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 120 +++++++++++++++++
4 files changed, 135 insertions(+), 145 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index bc6c5c38d6dd..12ac84712a38 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -13,8 +13,8 @@ volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
-bool map_shared, test_collapse, test_dev_userfaultfd;
-bool test_uffdio_wp = true, test_uffdio_minor = false;
+bool map_shared, test_dev_userfaultfd;
+bool test_uffdio_wp = true;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
@@ -128,15 +128,14 @@ static int shmem_allocate_area(void **alloc_area, bool is_src)
char *p = NULL, *p_alias = NULL;
int mem_fd = uffd_mem_fd_create(bytes * 2, false);
- if (test_collapse) {
- p = BASE_PMD_ADDR;
- if (!is_src)
- /* src map + alias + interleaved hpages */
- p += 2 * (bytes + hpage_size);
- p_alias = p;
- p_alias += bytes;
- p_alias += hpage_size; /* Prevent src/dst VMA merge */
- }
+ /* TODO: clean this up. Use a static addr is ugly */
+ p = BASE_PMD_ADDR;
+ if (!is_src)
+ /* src map + alias + interleaved hpages */
+ p += 2 * (bytes + hpage_size);
+ p_alias = p;
+ p_alias += bytes;
+ p_alias += hpage_size; /* Prevent src/dst VMA merge */
*alloc_area = mmap(p, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
mem_fd, offset);
@@ -144,7 +143,7 @@ static int shmem_allocate_area(void **alloc_area, bool is_src)
*alloc_area = NULL;
return -errno;
}
- if (test_collapse && *alloc_area != p)
+ if (*alloc_area != p)
err("mmap of memfd failed at %p", p);
area_alias = mmap(p_alias, bytes, PROT_READ | PROT_WRITE, MAP_SHARED,
@@ -154,7 +153,7 @@ static int shmem_allocate_area(void **alloc_area, bool is_src)
*alloc_area = NULL;
return -errno;
}
- if (test_collapse && area_alias != p_alias)
+ if (area_alias != p_alias)
err("mmap of anonymous memory failed at %p", p_alias);
if (is_src)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 9479a0649d7f..4bd5915cf5b4 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -90,8 +90,8 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
-extern bool map_shared, test_collapse, test_dev_userfaultfd;
-extern bool test_uffdio_wp, test_uffdio_minor;
+extern bool map_shared, test_dev_userfaultfd;
+extern bool test_uffdio_wp;
extern unsigned long long *count_verify;
extern volatile bool test_uffdio_copy_eexist;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 61d025d87bf2..f9322bbaf825 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -52,8 +52,6 @@ pthread_attr_t attr;
#define swap(a, b) \
do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
-#define factor_of_2(x) ((x) ^ ((x) & ((x) - 1)))
-
const char *examples =
"# Run anonymous memory test on 100MiB region with 99999 bounces:\n"
"./userfaultfd anon 100 99999\n\n"
@@ -79,8 +77,6 @@ static void usage(void)
"Supported mods:\n");
fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
- fprintf(stderr, "\tcollapse - Test MADV_COLLAPSE of UFFDIO_REGISTER_MODE_MINOR\n"
- "memory\n");
fprintf(stderr, "\nExample test mod usage:\n");
fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
@@ -584,92 +580,6 @@ static int userfaultfd_sig_test(void)
return userfaults != 0;
}
-void check_memory_contents(char *p)
-{
- unsigned long i;
- uint8_t expected_byte;
- void *expected_page;
-
- if (posix_memalign(&expected_page, page_size, page_size))
- err("out of memory");
-
- for (i = 0; i < nr_pages; ++i) {
- expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
- memset(expected_page, expected_byte, page_size);
- if (my_bcmp(expected_page, p + (i * page_size), page_size))
- err("unexpected page contents after minor fault");
- }
-
- free(expected_page);
-}
-
-static int userfaultfd_minor_test(void)
-{
- unsigned long p;
- pthread_t uffd_mon;
- char c;
- struct uffd_args args = { 0 };
-
- if (!test_uffdio_minor)
- return 0;
-
- printf("testing minor faults: ");
- fflush(stdout);
-
- uffd_test_ctx_init(uffd_minor_feature());
-
- if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
- false, test_uffdio_wp, true))
- err("register failure");
-
- /*
- * After registering with UFFD, populate the non-UFFD-registered side of
- * the shared mapping. This should *not* trigger any UFFD minor faults.
- */
- for (p = 0; p < nr_pages; ++p) {
- memset(area_dst + (p * page_size), p % ((uint8_t)-1),
- page_size);
- }
-
- args.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
- err("uffd_poll_thread create");
-
- /*
- * Read each of the pages back using the UFFD-registered mapping. We
- * expect that the first time we touch a page, it will result in a minor
- * fault. uffd_poll_thread will resolve the fault by bit-flipping the
- * page's contents, and then issuing a CONTINUE ioctl.
- */
- check_memory_contents(area_dst_alias);
-
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, NULL))
- return 1;
-
- uffd_stats_report(&args, 1);
-
- if (test_collapse) {
- printf("testing collapse of uffd memory into PMD-mapped THPs:");
- if (madvise(area_dst_alias, nr_pages * page_size,
- MADV_COLLAPSE))
- err("madvise(MADV_COLLAPSE)");
-
- uffd_test_ops->check_pmd_mapping(area_dst,
- nr_pages * page_size /
- read_pmd_pagesize());
- /*
- * This won't cause uffd-fault - it purely just makes sure there
- * was no corruption.
- */
- check_memory_contents(area_dst_alias);
- printf(" done.\n");
- }
-
- return args.missing_faults != 0 || args.minor_faults != nr_pages;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -782,7 +692,7 @@ static int userfaultfd_stress(void)
}
return userfaultfd_zeropage_test() || userfaultfd_sig_test()
- || userfaultfd_events_test() || userfaultfd_minor_test();
+ || userfaultfd_events_test();
}
static void set_test_type(const char *type)
@@ -797,13 +707,10 @@ static void set_test_type(const char *type)
map_shared = true;
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
- /* Minor faults require shared hugetlb; only enable here. */
- test_uffdio_minor = true;
} else if (!strcmp(type, "shmem")) {
map_shared = true;
test_type = TEST_SHMEM;
uffd_test_ops = &shmem_uffd_test_ops;
- test_uffdio_minor = true;
}
}
@@ -821,8 +728,6 @@ static void parse_test_type_arg(const char *raw_type)
test_dev_userfaultfd = true;
else if (!strcmp(token, "syscall"))
test_dev_userfaultfd = false;
- else if (!strcmp(token, "collapse"))
- test_collapse = true;
else
err("unrecognized test mod '%s'", token);
}
@@ -830,9 +735,6 @@ static void parse_test_type_arg(const char *raw_type)
if (!test_type)
err("failed to parse test type argument: '%s'", raw_type);
- if (test_collapse && test_type != TEST_SHMEM)
- err("Unsupported test: %s", raw_type);
-
if (test_type == TEST_HUGETLB)
page_size = default_huge_page_size();
else
@@ -854,8 +756,6 @@ static void parse_test_type_arg(const char *raw_type)
test_uffdio_wp = test_uffdio_wp &&
(features & UFFD_FEATURE_PAGEFAULT_FLAG_WP);
- test_uffdio_minor = test_uffdio_minor &&
- (features & uffd_minor_feature());
close(uffd);
uffd = -1;
@@ -872,7 +772,6 @@ static void sigalrm(int sig)
int main(int argc, char **argv)
{
size_t bytes;
- size_t hpage_size = read_pmd_pagesize();
if (argc < 4)
usage();
@@ -884,36 +783,8 @@ int main(int argc, char **argv)
parse_test_type_arg(argv[1]);
bytes = atol(argv[2]) * 1024 * 1024;
- if (test_collapse && bytes & (hpage_size - 1))
- err("MiB must be multiple of %lu if :collapse mod set",
- hpage_size >> 20);
-
nr_cpus = sysconf(_SC_NPROCESSORS_ONLN);
- if (test_collapse) {
- /* nr_cpus must divide (bytes / page_size), otherwise,
- * area allocations of (nr_pages * paze_size) won't be a
- * multiple of hpage_size, even if bytes is a multiple of
- * hpage_size.
- *
- * This means that nr_cpus must divide (N * (2 << (H-P))
- * where:
- * bytes = hpage_size * N
- * hpage_size = 2 << H
- * page_size = 2 << P
- *
- * And we want to chose nr_cpus to be the largest value
- * satisfying this constraint, not larger than the number
- * of online CPUs. Unfortunately, prime factorization of
- * N and nr_cpus may be arbitrary, so have to search for it.
- * Instead, just use the highest power of 2 dividing both
- * nr_cpus and (bytes / page_size).
- */
- int x = factor_of_2(nr_cpus);
- int y = factor_of_2(bytes / page_size);
-
- nr_cpus = x < y ? x : y;
- }
nr_pages_per_cpu = bytes / page_size / nr_cpus;
if (!nr_pages_per_cpu) {
_err("invalid MiB");
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 4690c95a9420..cba04608bdb0 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -329,6 +329,103 @@ static void uffd_pagemap_test(void)
uffd_test_pass();
}
+static void check_memory_contents(char *p)
+{
+ unsigned long i, j;
+ uint8_t expected_byte;
+
+ for (i = 0; i < nr_pages; ++i) {
+ expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
+ for (j = 0; j < page_size; j++) {
+ uint8_t v = *(uint8_t *)(p + (i * page_size) + j);
+ if (v != expected_byte)
+ err("unexpected page contents");
+ }
+ }
+}
+
+static void uffd_minor_test_common(bool test_collapse, bool test_wp)
+{
+ unsigned long p;
+ pthread_t uffd_mon;
+ char c;
+ struct uffd_args args = { 0 };
+
+ /*
+ * NOTE: MADV_COLLAPSE is not yet compatible with WP, so testing
+ * both do not make much sense.
+ */
+ assert(!(test_collapse && test_wp));
+
+ if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
+ /* NOTE! MADV_COLLAPSE may not work with uffd-wp */
+ false, test_wp, true))
+ err("register failure");
+
+ /*
+ * After registering with UFFD, populate the non-UFFD-registered side of
+ * the shared mapping. This should *not* trigger any UFFD minor faults.
+ */
+ for (p = 0; p < nr_pages; ++p)
+ memset(area_dst + (p * page_size), p % ((uint8_t)-1),
+ page_size);
+
+ args.apply_wp = test_wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
+ err("uffd_poll_thread create");
+
+ /*
+ * Read each of the pages back using the UFFD-registered mapping. We
+ * expect that the first time we touch a page, it will result in a minor
+ * fault. uffd_poll_thread will resolve the fault by bit-flipping the
+ * page's contents, and then issuing a CONTINUE ioctl.
+ */
+ check_memory_contents(area_dst_alias);
+
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, NULL))
+ err("join() failed");
+
+ if (test_collapse) {
+ if (madvise(area_dst_alias, nr_pages * page_size,
+ MADV_COLLAPSE)) {
+ /* It's fine to fail for this one... */
+ uffd_test_skip("MADV_COLLAPSE failed");
+ return;
+ }
+
+ uffd_test_ops->check_pmd_mapping(area_dst,
+ nr_pages * page_size /
+ read_pmd_pagesize());
+ /*
+ * This won't cause uffd-fault - it purely just makes sure there
+ * was no corruption.
+ */
+ check_memory_contents(area_dst_alias);
+ }
+
+ if (args.missing_faults != 0 || args.minor_faults != nr_pages)
+ uffd_test_fail("stats check error");
+ else
+ uffd_test_pass();
+}
+
+void uffd_minor_test(void)
+{
+ uffd_minor_test_common(false, false);
+}
+
+void uffd_minor_wp_test(void)
+{
+ uffd_minor_test_common(false, true);
+}
+
+void uffd_minor_collapse_test(void)
+{
+ uffd_minor_test_common(true, false);
+}
+
uffd_test_case_t uffd_tests[] = {
{
.name = "pagemap",
@@ -343,6 +440,29 @@ uffd_test_case_t uffd_tests[] = {
.uffd_feature_required =
UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_UNPOPULATED,
},
+ {
+ .name = "minor",
+ .uffd_fn = uffd_minor_test,
+ .mem_targets = MEM_SHMEM | MEM_HUGETLB,
+ .uffd_feature_required =
+ UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM,
+ },
+ {
+ .name = "minor-wp",
+ .uffd_fn = uffd_minor_wp_test,
+ .mem_targets = MEM_SHMEM | MEM_HUGETLB,
+ .uffd_feature_required =
+ UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM |
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "minor-collapse",
+ .uffd_fn = uffd_minor_collapse_test,
+ /* MADV_COLLAPSE only works with shmem */
+ .mem_targets = MEM_SHMEM,
+ /* We can't test MADV_COLLAPSE, so try our luck */
+ .uffd_feature_required = UFFD_FEATURE_MINOR_SHMEM,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 25/31] selftests/mm: Move uffd sig/events tests into uffd unit tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (23 preceding siblings ...)
2023-04-12 16:43 ` [PATCH v2 24/31] selftests/mm: Move uffd minor " Peter Xu
@ 2023-04-12 16:44 ` Peter Xu
2023-04-12 16:44 ` [PATCH v2 26/31] selftests/mm: Move zeropage test " Peter Xu
` (5 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:44 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz
Move the two tests into the unit test, and convert it into 20 standalone
tests:
- events test on all 5 mem types, with wp on/off
- signal test on all 5 mem types, with wp on/off
Testing sigbus on anon... done
Testing sigbus on shmem... done
Testing sigbus on shmem-private... done
Testing sigbus on hugetlb... done
Testing sigbus on hugetlb-private... done
Testing sigbus-wp on anon... done
Testing sigbus-wp on shmem... done
Testing sigbus-wp on shmem-private... done
Testing sigbus-wp on hugetlb... done
Testing sigbus-wp on hugetlb-private... done
Testing events on anon... done
Testing events on shmem... done
Testing events on shmem-private... done
Testing events on hugetlb... done
Testing events on hugetlb-private... done
Testing events-wp on anon... done
Testing events-wp on shmem... done
Testing events-wp on shmem-private... done
Testing events-wp on hugetlb... done
Testing events-wp on hugetlb-private... done
It'll also remove a lot of global references along the way,
e.g. test_uffdio_wp will be replaced with the wp value passed over.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-stress.c | 227 +---------------
tools/testing/selftests/mm/uffd-unit-tests.c | 264 +++++++++++++++++++
2 files changed, 265 insertions(+), 226 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index f9322bbaf825..ce51180238d8 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -273,133 +273,6 @@ static int stress(struct uffd_args *args)
return 0;
}
-sigjmp_buf jbuf, *sigbuf;
-
-static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
-{
- if (sig == SIGBUS) {
- if (sigbuf)
- siglongjmp(*sigbuf, 1);
- abort();
- }
-}
-
-/*
- * For non-cooperative userfaultfd test we fork() a process that will
- * generate pagefaults, will mremap the area monitored by the
- * userfaultfd and at last this process will release the monitored
- * area.
- * For the anonymous and shared memory the area is divided into two
- * parts, the first part is accessed before mremap, and the second
- * part is accessed after mremap. Since hugetlbfs does not support
- * mremap, the entire monitored area is accessed in a single pass for
- * HUGETLB_TEST.
- * The release of the pages currently generates event for shmem and
- * anonymous memory (UFFD_EVENT_REMOVE), hence it is not checked
- * for hugetlb.
- * For signal test(UFFD_FEATURE_SIGBUS), signal_test = 1, we register
- * monitored area, generate pagefaults and test that signal is delivered.
- * Use UFFDIO_COPY to allocate missing page and retry. For signal_test = 2
- * test robustness use case - we release monitored area, fork a process
- * that will generate pagefaults and verify signal is generated.
- * This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
- * feature. Using monitor thread, verify no userfault events are generated.
- */
-static int faulting_process(int signal_test, bool wp)
-{
- unsigned long nr;
- unsigned long long count;
- unsigned long split_nr_pages;
- unsigned long lastnr;
- struct sigaction act;
- volatile unsigned long signalled = 0;
-
- split_nr_pages = (nr_pages + 1) / 2;
-
- if (signal_test) {
- sigbuf = &jbuf;
- memset(&act, 0, sizeof(act));
- act.sa_sigaction = sighndl;
- act.sa_flags = SA_SIGINFO;
- if (sigaction(SIGBUS, &act, 0))
- err("sigaction");
- lastnr = (unsigned long)-1;
- }
-
- for (nr = 0; nr < split_nr_pages; nr++) {
- volatile int steps = 1;
- unsigned long offset = nr * page_size;
-
- if (signal_test) {
- if (sigsetjmp(*sigbuf, 1) != 0) {
- if (steps == 1 && nr == lastnr)
- err("Signal repeated");
-
- lastnr = nr;
- if (signal_test == 1) {
- if (steps == 1) {
- /* This is a MISSING request */
- steps++;
- if (copy_page(uffd, offset, wp))
- signalled++;
- } else {
- /* This is a WP request */
- assert(steps == 2);
- wp_range(uffd,
- (__u64)area_dst +
- offset,
- page_size, false);
- }
- } else {
- signalled++;
- continue;
- }
- }
- }
-
- count = *area_count(area_dst, nr);
- if (count != count_verify[nr])
- err("nr %lu memory corruption %llu %llu\n",
- nr, count, count_verify[nr]);
- /*
- * Trigger write protection if there is by writing
- * the same value back.
- */
- *area_count(area_dst, nr) = count;
- }
-
- if (signal_test)
- return signalled != split_nr_pages;
-
- area_dst = mremap(area_dst, nr_pages * page_size, nr_pages * page_size,
- MREMAP_MAYMOVE | MREMAP_FIXED, area_src);
- if (area_dst == MAP_FAILED)
- err("mremap");
- /* Reset area_src since we just clobbered it */
- area_src = NULL;
-
- for (; nr < nr_pages; nr++) {
- count = *area_count(area_dst, nr);
- if (count != count_verify[nr]) {
- err("nr %lu memory corruption %llu %llu\n",
- nr, count, count_verify[nr]);
- }
- /*
- * Trigger write protection if there is by writing
- * the same value back.
- */
- *area_count(area_dst, nr) = count;
- }
-
- uffd_test_ops->release_pages(area_dst);
-
- for (nr = 0; nr < nr_pages; nr++)
- if (my_bcmp(area_dst + nr * page_size, zeropage, page_size))
- err("nr %lu is not zero", nr);
-
- return 0;
-}
-
static void retry_uffdio_zeropage(int ufd,
struct uffdio_zeropage *uffdio_zeropage,
unsigned long offset)
@@ -483,103 +356,6 @@ static int userfaultfd_zeropage_test(void)
return 0;
}
-static int userfaultfd_events_test(void)
-{
- pthread_t uffd_mon;
- int err, features;
- pid_t pid;
- char c;
- struct uffd_args args = { 0 };
-
- printf("testing events (fork, remap, remove): ");
- fflush(stdout);
-
- features = UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_EVENT_REMAP |
- UFFD_FEATURE_EVENT_REMOVE;
- uffd_test_ctx_init(features);
-
- fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- args.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
- err("uffd_poll_thread create");
-
- pid = fork();
- if (pid < 0)
- err("fork");
-
- if (!pid)
- exit(faulting_process(0, test_uffdio_wp));
-
- waitpid(pid, &err, 0);
- if (err)
- err("faulting process failed");
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, NULL))
- return 1;
-
- uffd_stats_report(&args, 1);
-
- return args.missing_faults != nr_pages;
-}
-
-static int userfaultfd_sig_test(void)
-{
- unsigned long userfaults;
- pthread_t uffd_mon;
- int err, features;
- pid_t pid;
- char c;
- struct uffd_args args = { 0 };
-
- printf("testing signal delivery: ");
- fflush(stdout);
-
- features = UFFD_FEATURE_EVENT_FORK|UFFD_FEATURE_SIGBUS;
- uffd_test_ctx_init(features);
-
- fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- if (faulting_process(1, test_uffdio_wp))
- err("faulting process failed");
-
- uffd_test_ops->release_pages(area_dst);
-
- args.apply_wp = test_uffdio_wp;
- if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &args))
- err("uffd_poll_thread create");
-
- pid = fork();
- if (pid < 0)
- err("fork");
-
- if (!pid)
- exit(faulting_process(2, test_uffdio_wp));
-
- waitpid(pid, &err, 0);
- if (err)
- err("faulting process failed");
- if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
- err("pipe write");
- if (pthread_join(uffd_mon, (void **)&userfaults))
- return 1;
-
- printf("done.\n");
- if (userfaults)
- err("Signal test failed, userfaults: %ld", userfaults);
-
- return userfaults != 0;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -691,8 +467,7 @@ static int userfaultfd_stress(void)
uffd_stats_report(args, nr_cpus);
}
- return userfaultfd_zeropage_test() || userfaultfd_sig_test()
- || userfaultfd_events_test();
+ return userfaultfd_zeropage_test();
}
static void set_test_type(const char *type)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index cba04608bdb0..94549696f4b2 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -18,6 +18,9 @@
#define MEM_HUGETLB BIT_ULL(3)
#define MEM_HUGETLB_PRIVATE BIT_ULL(4)
+#define MEM_ALL (MEM_ANON | MEM_SHMEM | MEM_SHMEM_PRIVATE | \
+ MEM_HUGETLB | MEM_HUGETLB_PRIVATE)
+
struct mem_type {
const char *name;
unsigned int mem_flag;
@@ -426,6 +429,237 @@ void uffd_minor_collapse_test(void)
uffd_minor_test_common(true, false);
}
+static sigjmp_buf jbuf, *sigbuf;
+
+static void sighndl(int sig, siginfo_t *siginfo, void *ptr)
+{
+ if (sig == SIGBUS) {
+ if (sigbuf)
+ siglongjmp(*sigbuf, 1);
+ abort();
+ }
+}
+
+/*
+ * For non-cooperative userfaultfd test we fork() a process that will
+ * generate pagefaults, will mremap the area monitored by the
+ * userfaultfd and at last this process will release the monitored
+ * area.
+ * For the anonymous and shared memory the area is divided into two
+ * parts, the first part is accessed before mremap, and the second
+ * part is accessed after mremap. Since hugetlbfs does not support
+ * mremap, the entire monitored area is accessed in a single pass for
+ * HUGETLB_TEST.
+ * The release of the pages currently generates event for shmem and
+ * anonymous memory (UFFD_EVENT_REMOVE), hence it is not checked
+ * for hugetlb.
+ * For signal test(UFFD_FEATURE_SIGBUS), signal_test = 1, we register
+ * monitored area, generate pagefaults and test that signal is delivered.
+ * Use UFFDIO_COPY to allocate missing page and retry. For signal_test = 2
+ * test robustness use case - we release monitored area, fork a process
+ * that will generate pagefaults and verify signal is generated.
+ * This also tests UFFD_FEATURE_EVENT_FORK event along with the signal
+ * feature. Using monitor thread, verify no userfault events are generated.
+ */
+static int faulting_process(int signal_test, bool wp)
+{
+ unsigned long nr, i;
+ unsigned long long count;
+ unsigned long split_nr_pages;
+ unsigned long lastnr;
+ struct sigaction act;
+ volatile unsigned long signalled = 0;
+
+ split_nr_pages = (nr_pages + 1) / 2;
+
+ if (signal_test) {
+ sigbuf = &jbuf;
+ memset(&act, 0, sizeof(act));
+ act.sa_sigaction = sighndl;
+ act.sa_flags = SA_SIGINFO;
+ if (sigaction(SIGBUS, &act, 0))
+ err("sigaction");
+ lastnr = (unsigned long)-1;
+ }
+
+ for (nr = 0; nr < split_nr_pages; nr++) {
+ volatile int steps = 1;
+ unsigned long offset = nr * page_size;
+
+ if (signal_test) {
+ if (sigsetjmp(*sigbuf, 1) != 0) {
+ if (steps == 1 && nr == lastnr)
+ err("Signal repeated");
+
+ lastnr = nr;
+ if (signal_test == 1) {
+ if (steps == 1) {
+ /* This is a MISSING request */
+ steps++;
+ if (copy_page(uffd, offset, wp))
+ signalled++;
+ } else {
+ /* This is a WP request */
+ assert(steps == 2);
+ wp_range(uffd,
+ (__u64)area_dst +
+ offset,
+ page_size, false);
+ }
+ } else {
+ signalled++;
+ continue;
+ }
+ }
+ }
+
+ count = *area_count(area_dst, nr);
+ if (count != count_verify[nr])
+ err("nr %lu memory corruption %llu %llu\n",
+ nr, count, count_verify[nr]);
+ /*
+ * Trigger write protection if there is by writing
+ * the same value back.
+ */
+ *area_count(area_dst, nr) = count;
+ }
+
+ if (signal_test)
+ return signalled != split_nr_pages;
+
+ area_dst = mremap(area_dst, nr_pages * page_size, nr_pages * page_size,
+ MREMAP_MAYMOVE | MREMAP_FIXED, area_src);
+ if (area_dst == MAP_FAILED)
+ err("mremap");
+ /* Reset area_src since we just clobbered it */
+ area_src = NULL;
+
+ for (; nr < nr_pages; nr++) {
+ count = *area_count(area_dst, nr);
+ if (count != count_verify[nr]) {
+ err("nr %lu memory corruption %llu %llu\n",
+ nr, count, count_verify[nr]);
+ }
+ /*
+ * Trigger write protection if there is by writing
+ * the same value back.
+ */
+ *area_count(area_dst, nr) = count;
+ }
+
+ uffd_test_ops->release_pages(area_dst);
+
+ for (nr = 0; nr < nr_pages; nr++)
+ for (i = 0; i < page_size; i++)
+ if (*(area_dst + nr * page_size + i) != 0)
+ err("page %lu offset %lu is not zero", nr, i);
+
+ return 0;
+}
+
+static void uffd_sigbus_test_common(bool wp)
+{
+ unsigned long userfaults;
+ pthread_t uffd_mon;
+ pid_t pid;
+ int err;
+ char c;
+ struct uffd_args args = { 0 };
+
+ fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
+
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, wp, false))
+ err("register failure");
+
+ if (faulting_process(1, wp))
+ err("faulting process failed");
+
+ uffd_test_ops->release_pages(area_dst);
+
+ args.apply_wp = wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
+ err("uffd_poll_thread create");
+
+ pid = fork();
+ if (pid < 0)
+ err("fork");
+
+ if (!pid)
+ exit(faulting_process(2, wp));
+
+ waitpid(pid, &err, 0);
+ if (err)
+ err("faulting process failed");
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, (void **)&userfaults))
+ err("pthread_join()");
+
+ if (userfaults)
+ uffd_test_fail("Signal test failed, userfaults: %ld", userfaults);
+ else
+ uffd_test_pass();
+}
+
+static void uffd_sigbus_test(void)
+{
+ uffd_sigbus_test_common(false);
+}
+
+static void uffd_sigbus_wp_test(void)
+{
+ uffd_sigbus_test_common(true);
+}
+
+static void uffd_events_test_common(bool wp)
+{
+ pthread_t uffd_mon;
+ pid_t pid;
+ int err;
+ char c;
+ struct uffd_args args = { 0 };
+
+ fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK);
+ if (uffd_register(uffd, area_dst, nr_pages * page_size,
+ true, wp, false))
+ err("register failure");
+
+ args.apply_wp = wp;
+ if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
+ err("uffd_poll_thread create");
+
+ pid = fork();
+ if (pid < 0)
+ err("fork");
+
+ if (!pid)
+ exit(faulting_process(0, wp));
+
+ waitpid(pid, &err, 0);
+ if (err)
+ err("faulting process failed");
+ if (write(pipefd[1], &c, sizeof(c)) != sizeof(c))
+ err("pipe write");
+ if (pthread_join(uffd_mon, NULL))
+ err("pthread_join()");
+
+ if (args.missing_faults != nr_pages)
+ uffd_test_fail("Fault counts wrong");
+ else
+ uffd_test_pass();
+}
+
+static void uffd_events_test(void)
+{
+ uffd_events_test_common(false);
+}
+
+static void uffd_events_wp_test(void)
+{
+ uffd_events_test_common(true);
+}
+
uffd_test_case_t uffd_tests[] = {
{
.name = "pagemap",
@@ -463,6 +697,36 @@ uffd_test_case_t uffd_tests[] = {
/* We can't test MADV_COLLAPSE, so try our luck */
.uffd_feature_required = UFFD_FEATURE_MINOR_SHMEM,
},
+ {
+ .name = "sigbus",
+ .uffd_fn = uffd_sigbus_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_SIGBUS |
+ UFFD_FEATURE_EVENT_FORK,
+ },
+ {
+ .name = "sigbus-wp",
+ .uffd_fn = uffd_sigbus_wp_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_SIGBUS |
+ UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ },
+ {
+ .name = "events",
+ .uffd_fn = uffd_events_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_EVENT_FORK |
+ UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE,
+ },
+ {
+ .name = "events-wp",
+ .uffd_fn = uffd_events_wp_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_EVENT_FORK |
+ UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE |
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP |
+ UFFD_FEATURE_WP_HUGETLBFS_SHMEM,
+ },
};
int main(int argc, char *argv[])
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 26/31] selftests/mm: Move zeropage test into uffd unit tests
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (24 preceding siblings ...)
2023-04-12 16:44 ` [PATCH v2 25/31] selftests/mm: Move uffd sig/events tests into uffd unit tests Peter Xu
@ 2023-04-12 16:44 ` Peter Xu
2023-04-12 16:45 ` [PATCH v2 27/31] selftests/mm: Workaround no way to detect uffd-minor + wp Peter Xu
` (4 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:44 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: David Hildenbrand, Andrew Morton, Leonardo Bras Soares Passos,
peterx, Andrea Arcangeli, Nadav Amit, Mike Rapoport,
Axel Rasmussen, Mike Kravetz
Simplifies it a bit along the way, e.g., drop the never used offset
field (which was always the 1st page so offset=0).
Introduce uffd_register_with_ioctls() out of uffd_register() to detect
uffdio_register.ioctls got returned. Check that automatically when testing
UFFDIO_ZEROPAGE on different types of memory (and kernel).
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-stress.c | 94 +-------------------
tools/testing/selftests/mm/uffd-unit-tests.c | 93 +++++++++++++++++++
tools/testing/selftests/mm/vm_util.c | 14 ++-
tools/testing/selftests/mm/vm_util.h | 2 +
4 files changed, 108 insertions(+), 95 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index ce51180238d8..d78f88850011 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -109,15 +109,6 @@ static inline uint64_t uffd_minor_feature(void)
return 0;
}
-static int my_bcmp(char *str1, char *str2, size_t n)
-{
- unsigned long i;
- for (i = 0; i < n; i++)
- if (str1[i] != str2[i])
- return 1;
- return 0;
-}
-
static void *locking_thread(void *arg)
{
unsigned long cpu = (unsigned long) arg;
@@ -273,89 +264,6 @@ static int stress(struct uffd_args *args)
return 0;
}
-static void retry_uffdio_zeropage(int ufd,
- struct uffdio_zeropage *uffdio_zeropage,
- unsigned long offset)
-{
- uffd_test_ops->alias_mapping(&uffdio_zeropage->range.start,
- uffdio_zeropage->range.len,
- offset);
- if (ioctl(ufd, UFFDIO_ZEROPAGE, uffdio_zeropage)) {
- if (uffdio_zeropage->zeropage != -EEXIST)
- err("UFFDIO_ZEROPAGE error: %"PRId64,
- (int64_t)uffdio_zeropage->zeropage);
- } else {
- err("UFFDIO_ZEROPAGE error: %"PRId64,
- (int64_t)uffdio_zeropage->zeropage);
- }
-}
-
-static int __uffdio_zeropage(int ufd, unsigned long offset)
-{
- struct uffdio_zeropage uffdio_zeropage;
- int ret;
- bool has_zeropage = !(test_type == TEST_HUGETLB);
- __s64 res;
-
- if (offset >= nr_pages * page_size)
- err("unexpected offset %lu", offset);
- uffdio_zeropage.range.start = (unsigned long) area_dst + offset;
- uffdio_zeropage.range.len = page_size;
- uffdio_zeropage.mode = 0;
- ret = ioctl(ufd, UFFDIO_ZEROPAGE, &uffdio_zeropage);
- res = uffdio_zeropage.zeropage;
- if (ret) {
- /* real retval in ufdio_zeropage.zeropage */
- if (has_zeropage)
- err("UFFDIO_ZEROPAGE error: %"PRId64, (int64_t)res);
- else if (res != -EINVAL)
- err("UFFDIO_ZEROPAGE not -EINVAL");
- } else if (has_zeropage) {
- if (res != page_size) {
- err("UFFDIO_ZEROPAGE unexpected size");
- } else {
- retry_uffdio_zeropage(ufd, &uffdio_zeropage,
- offset);
- return 1;
- }
- } else
- err("UFFDIO_ZEROPAGE succeeded");
-
- return 0;
-}
-
-static int uffdio_zeropage(int ufd, unsigned long offset)
-{
- return __uffdio_zeropage(ufd, offset);
-}
-
-/* exercise UFFDIO_ZEROPAGE */
-static int userfaultfd_zeropage_test(void)
-{
- printf("testing UFFDIO_ZEROPAGE: ");
- fflush(stdout);
-
- uffd_test_ctx_init(0);
-
- if (uffd_register(uffd, area_dst, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
-
- if (area_dst_alias) {
- /* Needed this to test zeropage-retry on shared memory */
- if (uffd_register(uffd, area_dst_alias, nr_pages * page_size,
- true, test_uffdio_wp, false))
- err("register failure");
- }
-
- if (uffdio_zeropage(uffd, 0))
- if (my_bcmp(area_dst, zeropage, page_size))
- err("zeropage is not zero");
-
- printf("done.\n");
- return 0;
-}
-
static int userfaultfd_stress(void)
{
void *area;
@@ -467,7 +375,7 @@ static int userfaultfd_stress(void)
uffd_stats_report(args, nr_cpus);
}
- return userfaultfd_zeropage_test();
+ return 0;
}
static void set_test_type(const char *type)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 94549696f4b2..160bd8ccda55 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -660,7 +660,100 @@ static void uffd_events_wp_test(void)
uffd_events_test_common(true);
}
+static void retry_uffdio_zeropage(int ufd,
+ struct uffdio_zeropage *uffdio_zeropage)
+{
+ uffd_test_ops->alias_mapping(&uffdio_zeropage->range.start,
+ uffdio_zeropage->range.len,
+ 0);
+ if (ioctl(ufd, UFFDIO_ZEROPAGE, uffdio_zeropage)) {
+ if (uffdio_zeropage->zeropage != -EEXIST)
+ err("UFFDIO_ZEROPAGE error: %"PRId64,
+ (int64_t)uffdio_zeropage->zeropage);
+ } else {
+ err("UFFDIO_ZEROPAGE error: %"PRId64,
+ (int64_t)uffdio_zeropage->zeropage);
+ }
+}
+
+static bool do_uffdio_zeropage(int ufd, bool has_zeropage)
+{
+ struct uffdio_zeropage uffdio_zeropage = { 0 };
+ int ret;
+ __s64 res;
+
+ uffdio_zeropage.range.start = (unsigned long) area_dst;
+ uffdio_zeropage.range.len = page_size;
+ uffdio_zeropage.mode = 0;
+ ret = ioctl(ufd, UFFDIO_ZEROPAGE, &uffdio_zeropage);
+ res = uffdio_zeropage.zeropage;
+ if (ret) {
+ /* real retval in ufdio_zeropage.zeropage */
+ if (has_zeropage)
+ err("UFFDIO_ZEROPAGE error: %"PRId64, (int64_t)res);
+ else if (res != -EINVAL)
+ err("UFFDIO_ZEROPAGE not -EINVAL");
+ } else if (has_zeropage) {
+ if (res != page_size)
+ err("UFFDIO_ZEROPAGE unexpected size");
+ else
+ retry_uffdio_zeropage(ufd, &uffdio_zeropage);
+ return true;
+ } else
+ err("UFFDIO_ZEROPAGE succeeded");
+
+ return false;
+}
+
+/*
+ * Registers a range with MISSING mode only for zeropage test. Return true
+ * if UFFDIO_ZEROPAGE supported, false otherwise. Can't use uffd_register()
+ * because we want to detect .ioctls along the way.
+ */
+static bool
+uffd_register_detect_zeropage(int uffd, void *addr, uint64_t len)
+{
+ uint64_t ioctls = 0;
+
+ if (uffd_register_with_ioctls(uffd, addr, len, true,
+ false, false, &ioctls))
+ err("zeropage register fail");
+
+ return ioctls & (1 << _UFFDIO_ZEROPAGE);
+}
+
+/* exercise UFFDIO_ZEROPAGE */
+static void uffd_zeropage_test(void)
+{
+ bool has_zeropage;
+ int i;
+
+ has_zeropage = uffd_register_detect_zeropage(uffd, area_dst, page_size);
+ if (area_dst_alias)
+ /* Ignore the retval; we already have it */
+ uffd_register_detect_zeropage(uffd, area_dst_alias, page_size);
+
+ if (do_uffdio_zeropage(uffd, has_zeropage))
+ for (i = 0; i < page_size; i++)
+ if (area_dst[i] != 0)
+ err("data non-zero at offset %d\n", i);
+
+ if (uffd_unregister(uffd, area_dst, page_size))
+ err("unregister");
+
+ if (area_dst_alias && uffd_unregister(uffd, area_dst_alias, page_size))
+ err("unregister");
+
+ uffd_test_pass();
+}
+
uffd_test_case_t uffd_tests[] = {
+ {
+ .name = "zeropage",
+ .uffd_fn = uffd_zeropage_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = 0,
+ },
{
.name = "pagemap",
.uffd_fn = uffd_pagemap_test,
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 1bc0ceb01adb..9b06a5034808 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -198,8 +198,9 @@ unsigned long default_huge_page_size(void)
return hps;
}
-int uffd_register(int uffd, void *addr, uint64_t len,
- bool miss, bool wp, bool minor)
+/* If `ioctls' non-NULL, the allowed ioctls will be returned into the var */
+int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor, uint64_t *ioctls)
{
struct uffdio_register uffdio_register = { 0 };
uint64_t mode = 0;
@@ -218,10 +219,19 @@ int uffd_register(int uffd, void *addr, uint64_t len,
if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
ret = -errno;
+ else if (ioctls)
+ *ioctls = uffdio_register.ioctls;
return ret;
}
+int uffd_register(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor)
+{
+ return uffd_register_with_ioctls(uffd, addr, len,
+ miss, wp, minor, NULL);
+}
+
int uffd_unregister(int uffd, void *addr, uint64_t len)
{
struct uffdio_range range = { .start = (uintptr_t)addr, .len = len };
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 634eb2f41145..b950bd16083a 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -52,6 +52,8 @@ int uffd_open_dev(unsigned int flags);
int uffd_open_sys(unsigned int flags);
int uffd_open(unsigned int flags);
int uffd_get_features(uint64_t *features);
+int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
+ bool miss, bool wp, bool minor, uint64_t *ioctls);
/*
* On ppc64 this will only work with radix 2M hugepage size
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 27/31] selftests/mm: Workaround no way to detect uffd-minor + wp
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (25 preceding siblings ...)
2023-04-12 16:44 ` [PATCH v2 26/31] selftests/mm: Move zeropage test " Peter Xu
@ 2023-04-12 16:45 ` Peter Xu
2023-04-12 16:45 ` [PATCH v2 28/31] selftests/mm: Allow uffd test to skip properly with no privilege Peter Xu
` (3 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:45 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Nadav Amit, Mike Rapoport, Leonardo Bras Soares Passos, peterx,
David Hildenbrand, Axel Rasmussen, Andrea Arcangeli,
Mike Kravetz, Andrew Morton
Userfaultfd minor+wp mode was very recently added. The test will fail on
the old kernels at ioctl(UFFDIO_CONTINUE) which is misterious.
Unfortunately there's no feature bit to detect for this support.
Add a hack to leverage WP_UNPOPULATED to detect whether that feature
existed, since WP_UNPOPULATED was merged right after minor+wp.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 160bd8ccda55..a33d6c928eeb 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -780,7 +780,13 @@ uffd_test_case_t uffd_tests[] = {
.mem_targets = MEM_SHMEM | MEM_HUGETLB,
.uffd_feature_required =
UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM |
- UFFD_FEATURE_PAGEFAULT_FLAG_WP,
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP |
+ /*
+ * HACK: here we leveraged WP_UNPOPULATED to detect whether
+ * minor mode supports wr-protect. There's no feature flag
+ * for it so this is the best we can test against.
+ */
+ UFFD_FEATURE_WP_UNPOPULATED,
},
{
.name = "minor-collapse",
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 28/31] selftests/mm: Allow uffd test to skip properly with no privilege
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (26 preceding siblings ...)
2023-04-12 16:45 ` [PATCH v2 27/31] selftests/mm: Workaround no way to detect uffd-minor + wp Peter Xu
@ 2023-04-12 16:45 ` Peter Xu
2023-04-12 16:45 ` [PATCH v2 29/31] selftests/mm: Drop sys/dev test in uffd-stress test Peter Xu
` (2 subsequent siblings)
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:45 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Nadav Amit, Mike Rapoport, Leonardo Bras Soares Passos, peterx,
David Hildenbrand, Axel Rasmussen, Andrea Arcangeli,
Mike Kravetz, Andrew Morton
Allow skip a unit test properly due to no privilege (e.g. sigbus and events
tests).
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-common.c | 27 ++++++++++++--------
tools/testing/selftests/mm/uffd-common.h | 4 +--
tools/testing/selftests/mm/uffd-stress.c | 6 +++--
tools/testing/selftests/mm/uffd-unit-tests.c | 10 +++++---
4 files changed, 29 insertions(+), 18 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 12ac84712a38..3e98e129f8bd 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -232,7 +232,7 @@ void uffd_stats_report(struct uffd_args *args, int n_cpus)
printf("\n");
}
-void userfaultfd_open(uint64_t *features)
+int userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
@@ -241,18 +241,19 @@ void userfaultfd_open(uint64_t *features)
else
uffd = uffd_open_sys(UFFD_FLAGS);
if (uffd < 0)
- err("uffd open failed (dev=%d)", test_dev_userfaultfd);
+ return -1;
uffd_flags = fcntl(uffd, F_GETFD, NULL);
uffdio_api.api = UFFD_API;
uffdio_api.features = *features;
if (ioctl(uffd, UFFDIO_API, &uffdio_api))
- err("UFFDIO_API failed.\nPlease make sure to "
- "run with either root or ptrace capability.");
+ /* Probably lack of CAP_PTRACE? */
+ return -1;
if (uffdio_api.api != UFFD_API)
err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api);
*features = uffdio_api.features;
+ return 0;
}
static inline void munmap_area(void **area)
@@ -295,7 +296,7 @@ static void uffd_test_ctx_clear(void)
munmap_area((void **)&area_remap);
}
-int uffd_test_ctx_init(uint64_t features)
+int uffd_test_ctx_init(uint64_t features, const char **errmsg)
{
unsigned long nr, cpu;
int ret;
@@ -303,13 +304,19 @@ int uffd_test_ctx_init(uint64_t features)
uffd_test_ctx_clear();
ret = uffd_test_ops->allocate_area((void **)&area_src, true);
- if (ret)
- return ret;
- ret = uffd_test_ops->allocate_area((void **)&area_dst, false);
- if (ret)
+ ret |= uffd_test_ops->allocate_area((void **)&area_dst, false);
+ if (ret) {
+ if (errmsg)
+ *errmsg = "memory allocation failed";
return ret;
+ }
- userfaultfd_open(&features);
+ ret = userfaultfd_open(&features);
+ if (ret) {
+ if (errmsg)
+ *errmsg = "possible lack of priviledge";
+ return ret;
+ }
count_verify = malloc(nr_pages * sizeof(unsigned long long));
if (!count_verify)
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 4bd5915cf5b4..32e590ce9442 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -101,8 +101,8 @@ extern uffd_test_ops_t hugetlb_uffd_test_ops;
extern uffd_test_ops_t *uffd_test_ops;
void uffd_stats_report(struct uffd_args *args, int n_cpus);
-int uffd_test_ctx_init(uint64_t features);
-void userfaultfd_open(uint64_t *features);
+int uffd_test_ctx_init(uint64_t features, const char **errmsg);
+int userfaultfd_open(uint64_t *features);
int uffd_read_msg(int ufd, struct uffd_msg *msg);
void wp_range(int ufd, __u64 start, __u64 len, bool wp);
void uffd_handle_page_fault(struct uffd_msg *msg, struct uffd_args *args);
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index d78f88850011..c0e804f05002 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -271,7 +271,8 @@ static int userfaultfd_stress(void)
struct uffd_args args[nr_cpus];
uint64_t mem_size = nr_pages * page_size;
- uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED);
+ if (uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED, NULL))
+ err("context init failed");
if (posix_memalign(&area, page_size, page_size))
err("out of memory");
@@ -435,7 +436,8 @@ static void parse_test_type_arg(const char *raw_type)
* feature.
*/
- userfaultfd_open(&features);
+ if (userfaultfd_open(&features))
+ err("Userfaultfd open failed");
test_uffdio_wp = test_uffdio_wp &&
(features & UFFD_FEATURE_PAGEFAULT_FLAG_WP);
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index a33d6c928eeb..b0acf558e8cb 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -172,7 +172,8 @@ static int test_uffd_api(bool use_dev)
* This function initializes the global variables. TODO: remove global
* vars and then remove this.
*/
-static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
+static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type,
+ const char **errmsg)
{
map_shared = mem_type->shared;
uffd_test_ops = mem_type->mem_ops;
@@ -186,7 +187,7 @@ static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type)
/* TODO: remove this global var.. it's so ugly */
nr_cpus = 1;
- return uffd_test_ctx_init(test->uffd_feature_required);
+ return uffd_test_ctx_init(test->uffd_feature_required, errmsg);
}
static bool uffd_feature_supported(uffd_test_case_t *test)
@@ -835,6 +836,7 @@ int main(int argc, char *argv[])
uffd_test_case_t *test;
mem_type_t *mem_type;
char test_name[128];
+ const char *errmsg;
int has_uffd;
int i, j;
@@ -860,8 +862,8 @@ int main(int argc, char *argv[])
uffd_test_skip("feature missing");
continue;
}
- if (uffd_setup_environment(test, mem_type)) {
- uffd_test_skip("environment setup failed");
+ if (uffd_setup_environment(test, mem_type, &errmsg)) {
+ uffd_test_skip(errmsg);
continue;
}
test->uffd_fn();
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 29/31] selftests/mm: Drop sys/dev test in uffd-stress test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (27 preceding siblings ...)
2023-04-12 16:45 ` [PATCH v2 28/31] selftests/mm: Allow uffd test to skip properly with no privilege Peter Xu
@ 2023-04-12 16:45 ` Peter Xu
2023-04-12 16:45 ` [PATCH v2 30/31] selftests/mm: Add shmem-private test to uffd-stress Peter Xu
2023-04-12 16:45 ` [PATCH v2 31/31] selftests/mm: Add uffdio register ioctls test Peter Xu
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:45 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Nadav Amit, Mike Rapoport, Leonardo Bras Soares Passos, peterx,
David Hildenbrand, Axel Rasmussen, Andrea Arcangeli,
Mike Kravetz, Andrew Morton
With the new uffd unit test covering the /dev/userfaultfd path and syscall
path of uffd initializations, we can safely drop the devnode test in the
old stress test.
One thing is to avoid duplication of running the stress test twice which is
an overkill to only test the /dev/ interface in run_vmtests.sh.
The other benefit is now all uffd tests (that uses userfaultfd_open) can
run automatically as long as any type of interface is enabled (either
syscall or dev), so it's more likely to succeed rather than fail due to
unprivilege.
With this patch lands, we can drop all the "mem_type:XXX" handlings too.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/run_vmtests.sh | 15 +++++--------
tools/testing/selftests/mm/uffd-common.c | 7 ++----
tools/testing/selftests/mm/uffd-common.h | 2 +-
tools/testing/selftests/mm/uffd-stress.c | 27 ++---------------------
4 files changed, 11 insertions(+), 40 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index efe22dc569f0..ecc16ea6fc40 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -197,16 +197,13 @@ CATEGORY="gup_test" run_test ./gup_test -a
CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000
CATEGORY="userfaultfd" run_test ./uffd-unit-tests
-uffd_mods=("" ":dev")
uffd_stress_bin=./uffd-stress
-for mod in "${uffd_mods[@]}"; do
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon${mod} 20 16
- # Hugetlb tests require source and destination huge pages. Pass in half
- # the size ($half_ufd_size_MB), which is used for *each*.
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared${mod} "$half_ufd_size_MB" 32
- CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem${mod} 20 16
-done
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
+# Hugetlb tests require source and destination huge pages. Pass in half
+# the size ($half_ufd_size_MB), which is used for *each*.
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
#cleanup
echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c
index 3e98e129f8bd..61c6250adf93 100644
--- a/tools/testing/selftests/mm/uffd-common.c
+++ b/tools/testing/selftests/mm/uffd-common.c
@@ -13,7 +13,7 @@ volatile bool test_uffdio_copy_eexist = true;
unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
int uffd = -1, uffd_flags, finished, *pipefd, test_type;
-bool map_shared, test_dev_userfaultfd;
+bool map_shared;
bool test_uffdio_wp = true;
unsigned long long *count_verify;
uffd_test_ops_t *uffd_test_ops;
@@ -236,10 +236,7 @@ int userfaultfd_open(uint64_t *features)
{
struct uffdio_api uffdio_api;
- if (test_dev_userfaultfd)
- uffd = uffd_open_dev(UFFD_FLAGS);
- else
- uffd = uffd_open_sys(UFFD_FLAGS);
+ uffd = uffd_open(UFFD_FLAGS);
if (uffd < 0)
return -1;
uffd_flags = fcntl(uffd, F_GETFD, NULL);
diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h
index 32e590ce9442..6068f2346b86 100644
--- a/tools/testing/selftests/mm/uffd-common.h
+++ b/tools/testing/selftests/mm/uffd-common.h
@@ -90,7 +90,7 @@ typedef struct uffd_test_ops uffd_test_ops_t;
extern unsigned long nr_cpus, nr_pages, nr_pages_per_cpu, page_size;
extern char *area_src, *area_src_alias, *area_dst, *area_dst_alias, *area_remap;
extern int uffd, uffd_flags, finished, *pipefd, test_type;
-extern bool map_shared, test_dev_userfaultfd;
+extern bool map_shared;
extern bool test_uffdio_wp;
extern unsigned long long *count_verify;
extern volatile bool test_uffdio_copy_eexist;
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index c0e804f05002..4e071a7d0ff5 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -55,8 +55,6 @@ pthread_attr_t attr;
const char *examples =
"# Run anonymous memory test on 100MiB region with 99999 bounces:\n"
"./userfaultfd anon 100 99999\n\n"
- "# Run the same anonymous memory test, but using /dev/userfaultfd:\n"
- "./userfaultfd anon:dev 100 99999\n\n"
"# Run share memory test on 1GiB region with 99 bounces:\n"
"./userfaultfd shmem 1000 99\n\n"
"# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
@@ -69,18 +67,9 @@ const char *examples =
static void usage(void)
{
- fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces> "
- "[hugetlbfs_file]\n\n");
+ fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n");
fprintf(stderr, "Supported <test type>: anon, hugetlb, "
"hugetlb_shared, shmem\n\n");
- fprintf(stderr, "'Test mods' can be joined to the test type string with a ':'. "
- "Supported mods:\n");
- fprintf(stderr, "\tsyscall - Use userfaultfd(2) (default)\n");
- fprintf(stderr, "\tdev - Use /dev/userfaultfd instead of userfaultfd(2)\n");
- fprintf(stderr, "\nExample test mod usage:\n");
- fprintf(stderr, "# Run anonymous memory test with /dev/userfaultfd:\n");
- fprintf(stderr, "./userfaultfd anon:dev 100 99999\n\n");
-
fprintf(stderr, "Examples:\n\n");
fprintf(stderr, "%s", examples);
exit(1);
@@ -400,21 +389,9 @@ static void set_test_type(const char *type)
static void parse_test_type_arg(const char *raw_type)
{
- char *buf = strdup(raw_type);
uint64_t features = UFFD_API_FEATURES;
- while (buf) {
- const char *token = strsep(&buf, ":");
-
- if (!test_type)
- set_test_type(token);
- else if (!strcmp(token, "dev"))
- test_dev_userfaultfd = true;
- else if (!strcmp(token, "syscall"))
- test_dev_userfaultfd = false;
- else
- err("unrecognized test mod '%s'", token);
- }
+ set_test_type(raw_type);
if (!test_type)
err("failed to parse test type argument: '%s'", raw_type);
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 30/31] selftests/mm: Add shmem-private test to uffd-stress
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (28 preceding siblings ...)
2023-04-12 16:45 ` [PATCH v2 29/31] selftests/mm: Drop sys/dev test in uffd-stress test Peter Xu
@ 2023-04-12 16:45 ` Peter Xu
2023-04-12 16:45 ` [PATCH v2 31/31] selftests/mm: Add uffdio register ioctls test Peter Xu
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:45 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Leonardo Bras Soares Passos, peterx, Mike Rapoport, Nadav Amit,
Andrea Arcangeli, Mike Kravetz, David Hildenbrand, Andrew Morton,
Axel Rasmussen
The userfaultfd stress test never tested private shmem, which I think was
overlooked long due. Add it so it matches with uffd unit test and it'll
cover all memory supported with the three memory types.
Meanwhile, rename the memory types a bit. Considering shared mem is the
major use case for both shmem / hugetlbfs, changing from:
anon, hugetlb, hugetlb_shared, shmem
To (with shmem-private added):
anon, hugetlb, hugetlb-private, shmem, shmem-private
Add the shmem-private to run_vmtests.sh too.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/run_vmtests.sh | 3 ++-
tools/testing/selftests/mm/uffd-stress.c | 11 +++++++----
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index ecc16ea6fc40..438eb49567b6 100644
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -202,8 +202,9 @@ CATEGORY="userfaultfd" run_test ${uffd_stress_bin} anon 20 16
# Hugetlb tests require source and destination huge pages. Pass in half
# the size ($half_ufd_size_MB), which is used for *each*.
CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb "$half_ufd_size_MB" 32
-CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb_shared "$half_ufd_size_MB" 32
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} hugetlb-private "$half_ufd_size_MB" 32
CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem 20 16
+CATEGORY="userfaultfd" run_test ${uffd_stress_bin} shmem-private 20 16
#cleanup
echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c
index 4e071a7d0ff5..f1ad9eef1c3a 100644
--- a/tools/testing/selftests/mm/uffd-stress.c
+++ b/tools/testing/selftests/mm/uffd-stress.c
@@ -59,8 +59,8 @@ const char *examples =
"./userfaultfd shmem 1000 99\n\n"
"# Run hugetlb memory test on 256MiB region with 50 bounces:\n"
"./userfaultfd hugetlb 256 50\n\n"
- "# Run the same hugetlb test but using shared file:\n"
- "./userfaultfd hugetlb_shared 256 50\n\n"
+ "# Run the same hugetlb test but using private file:\n"
+ "./userfaultfd hugetlb-private 256 50\n\n"
"# 10MiB-~6GiB 999 bounces anonymous test, "
"continue forever unless an error triggers\n"
"while ./userfaultfd anon $[RANDOM % 6000 + 10] 999; do true; done\n\n";
@@ -69,7 +69,7 @@ static void usage(void)
{
fprintf(stderr, "\nUsage: ./userfaultfd <test type> <MiB> <bounces>\n\n");
fprintf(stderr, "Supported <test type>: anon, hugetlb, "
- "hugetlb_shared, shmem\n\n");
+ "hugetlb-private, shmem, shmem-private\n\n");
fprintf(stderr, "Examples:\n\n");
fprintf(stderr, "%s", examples);
exit(1);
@@ -376,14 +376,17 @@ static void set_test_type(const char *type)
} else if (!strcmp(type, "hugetlb")) {
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
- } else if (!strcmp(type, "hugetlb_shared")) {
map_shared = true;
+ } else if (!strcmp(type, "hugetlb-private")) {
test_type = TEST_HUGETLB;
uffd_test_ops = &hugetlb_uffd_test_ops;
} else if (!strcmp(type, "shmem")) {
map_shared = true;
test_type = TEST_SHMEM;
uffd_test_ops = &shmem_uffd_test_ops;
+ } else if (!strcmp(type, "shmem-private")) {
+ test_type = TEST_SHMEM;
+ uffd_test_ops = &shmem_uffd_test_ops;
}
}
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread* [PATCH v2 31/31] selftests/mm: Add uffdio register ioctls test
2023-04-12 16:38 [PATCH v2 00/31] selftests/mm: Split / Refactor userfault test Peter Xu
` (29 preceding siblings ...)
2023-04-12 16:45 ` [PATCH v2 30/31] selftests/mm: Add shmem-private test to uffd-stress Peter Xu
@ 2023-04-12 16:45 ` Peter Xu
30 siblings, 0 replies; 43+ messages in thread
From: Peter Xu @ 2023-04-12 16:45 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Leonardo Bras Soares Passos, peterx, Mike Rapoport, Nadav Amit,
Andrea Arcangeli, Mike Kravetz, David Hildenbrand, Andrew Morton,
Axel Rasmussen
This new test tests against the returned ioctls from UFFDIO_REGISTER, where
put into uffdio_register.ioctls.
This also tests the expected failure cases of UFFDIO_REGISTER, aka:
- Register with empty mode should fail with -EINVAL
- Register minor without page cache (anon) should fail with -EINVAL
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/uffd-unit-tests.c | 112 ++++++++++++++++---
1 file changed, 97 insertions(+), 15 deletions(-)
diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index b0acf558e8cb..d871bf732e62 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -62,8 +62,14 @@ mem_type_t mem_types[] = {
},
};
+/* Arguments to be passed over to each uffd unit test */
+struct uffd_test_args {
+ mem_type_t *mem_type;
+};
+typedef struct uffd_test_args uffd_test_args_t;
+
/* Returns: UFFD_TEST_* */
-typedef void (*uffd_test_fn)(void);
+typedef void (*uffd_test_fn)(uffd_test_args_t *);
typedef struct {
const char *name;
@@ -172,8 +178,9 @@ static int test_uffd_api(bool use_dev)
* This function initializes the global variables. TODO: remove global
* vars and then remove this.
*/
-static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type,
- const char **errmsg)
+static int
+uffd_setup_environment(uffd_test_args_t *args, uffd_test_case_t *test,
+ mem_type_t *mem_type, const char **errmsg)
{
map_shared = mem_type->shared;
uffd_test_ops = mem_type->mem_ops;
@@ -187,6 +194,9 @@ static int uffd_setup_environment(uffd_test_case_t *test, mem_type_t *mem_type,
/* TODO: remove this global var.. it's so ugly */
nr_cpus = 1;
+ /* Initialize test arguments */
+ args->mem_type = mem_type;
+
return uffd_test_ctx_init(test->uffd_feature_required, errmsg);
}
@@ -239,7 +249,7 @@ static int pagemap_test_fork(bool present)
return result;
}
-static void uffd_wp_unpopulated_test(void)
+static void uffd_wp_unpopulated_test(uffd_test_args_t *args)
{
uint64_t value;
int pagemap_fd;
@@ -285,7 +295,7 @@ static void uffd_wp_unpopulated_test(void)
uffd_test_pass();
}
-static void uffd_pagemap_test(void)
+static void uffd_pagemap_test(uffd_test_args_t *args)
{
int pagemap_fd;
uint64_t value;
@@ -415,17 +425,17 @@ static void uffd_minor_test_common(bool test_collapse, bool test_wp)
uffd_test_pass();
}
-void uffd_minor_test(void)
+void uffd_minor_test(uffd_test_args_t *args)
{
uffd_minor_test_common(false, false);
}
-void uffd_minor_wp_test(void)
+void uffd_minor_wp_test(uffd_test_args_t *args)
{
uffd_minor_test_common(false, true);
}
-void uffd_minor_collapse_test(void)
+void uffd_minor_collapse_test(uffd_test_args_t *args)
{
uffd_minor_test_common(true, false);
}
@@ -603,12 +613,12 @@ static void uffd_sigbus_test_common(bool wp)
uffd_test_pass();
}
-static void uffd_sigbus_test(void)
+static void uffd_sigbus_test(uffd_test_args_t *args)
{
uffd_sigbus_test_common(false);
}
-static void uffd_sigbus_wp_test(void)
+static void uffd_sigbus_wp_test(uffd_test_args_t *args)
{
uffd_sigbus_test_common(true);
}
@@ -651,12 +661,12 @@ static void uffd_events_test_common(bool wp)
uffd_test_pass();
}
-static void uffd_events_test(void)
+static void uffd_events_test(uffd_test_args_t *args)
{
uffd_events_test_common(false);
}
-static void uffd_events_wp_test(void)
+static void uffd_events_wp_test(uffd_test_args_t *args)
{
uffd_events_test_common(true);
}
@@ -724,7 +734,7 @@ uffd_register_detect_zeropage(int uffd, void *addr, uint64_t len)
}
/* exercise UFFDIO_ZEROPAGE */
-static void uffd_zeropage_test(void)
+static void uffd_zeropage_test(uffd_test_args_t *args)
{
bool has_zeropage;
int i;
@@ -748,7 +758,77 @@ static void uffd_zeropage_test(void)
uffd_test_pass();
}
+/*
+ * Test the returned uffdio_register.ioctls with different register modes.
+ * Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test.
+ */
+static void
+do_register_ioctls_test(uffd_test_args_t *args, bool miss, bool wp, bool minor)
+{
+ uint64_t ioctls = 0, expected = BIT_ULL(_UFFDIO_WAKE);
+ mem_type_t *mem_type = args->mem_type;
+ int ret;
+
+ ret = uffd_register_with_ioctls(uffd, area_dst, page_size,
+ miss, wp, minor, &ioctls);
+
+ /*
+ * Handle special cases of UFFDIO_REGISTER here where it should
+ * just fail with -EINVAL first..
+ *
+ * Case 1: register MINOR on anon
+ * Case 2: register with no mode selected
+ */
+ if ((minor && (mem_type->mem_flag == MEM_ANON)) ||
+ (!miss && !wp && !minor)) {
+ if (ret != -EINVAL)
+ err("register (miss=%d, wp=%d, minor=%d) failed "
+ "with wrong errno=%d", miss, wp, minor, ret);
+ return;
+ }
+
+ /* UFFDIO_REGISTER should succeed, then check ioctls returned */
+ if (miss)
+ expected |= BIT_ULL(_UFFDIO_COPY);
+ if (wp)
+ expected |= BIT_ULL(_UFFDIO_WRITEPROTECT);
+ if (minor)
+ expected |= BIT_ULL(_UFFDIO_CONTINUE);
+
+ if ((ioctls & expected) != expected)
+ err("unexpected uffdio_register.ioctls "
+ "(miss=%d, wp=%d, minor=%d): expected=0x%"PRIx64", "
+ "returned=0x%"PRIx64, miss, wp, minor, expected, ioctls);
+
+ if (uffd_unregister(uffd, area_dst, page_size))
+ err("unregister");
+}
+
+static void uffd_register_ioctls_test(uffd_test_args_t *args)
+{
+ int miss, wp, minor;
+
+ for (miss = 0; miss <= 1; miss++)
+ for (wp = 0; wp <= 1; wp++)
+ for (minor = 0; minor <= 1; minor++)
+ do_register_ioctls_test(args, miss, wp, minor);
+
+ uffd_test_pass();
+}
+
uffd_test_case_t uffd_tests[] = {
+ {
+ /* Test returned uffdio_register.ioctls. */
+ .name = "register-ioctls",
+ .uffd_fn = uffd_register_ioctls_test,
+ .mem_targets = MEM_ALL,
+ .uffd_feature_required = UFFD_FEATURE_MISSING_HUGETLBFS |
+ UFFD_FEATURE_MISSING_SHMEM |
+ UFFD_FEATURE_PAGEFAULT_FLAG_WP |
+ UFFD_FEATURE_WP_HUGETLBFS_SHMEM |
+ UFFD_FEATURE_MINOR_HUGETLBFS |
+ UFFD_FEATURE_MINOR_SHMEM,
+ },
{
.name = "zeropage",
.uffd_fn = uffd_zeropage_test,
@@ -835,6 +915,7 @@ int main(int argc, char *argv[])
int n_mems = sizeof(mem_types) / sizeof(mem_type_t);
uffd_test_case_t *test;
mem_type_t *mem_type;
+ uffd_test_args_t args;
char test_name[128];
const char *errmsg;
int has_uffd;
@@ -862,11 +943,12 @@ int main(int argc, char *argv[])
uffd_test_skip("feature missing");
continue;
}
- if (uffd_setup_environment(test, mem_type, &errmsg)) {
+ if (uffd_setup_environment(&args, test, mem_type,
+ &errmsg)) {
uffd_test_skip(errmsg);
continue;
}
- test->uffd_fn();
+ test->uffd_fn(&args);
}
}
--
2.39.1
^ permalink raw reply related [flat|nested] 43+ messages in thread