* [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
@ 2022-01-04 2:17 Mike Kravetz
2022-01-04 22:35 ` Axel Rasmussen
0 siblings, 1 reply; 6+ messages in thread
From: Mike Kravetz @ 2022-01-04 2:17 UTC (permalink / raw)
To: linux-kernel, linux-mm
Cc: Axel Rasmussen, Peter Xu, Andrea Arcangeli, Mina Almasry,
Shuah Khan, Andrew Morton, Mike Kravetz
The message for commit f5c73297181c ("userfaultfd/selftests: fix hugetlb
area allocations") says there is no need to create a hugetlb file in the
non-shared testing case. However, the commit did not actually change
the code to prevent creation of the file.
While it is technically true that there is no need to create and use a
hugetlb file in the case of non-shared-testing, it is useful. This is
because 'hole punching' of a hugetlb file has the potentially incorrect
side effect of also removing pages from private mappings. The
userfaultfd test relies on this side effect for removing pages from the
destination buffer during rounds of stress testing.
Remove the incomplete code that was added to deal with no hugetlb file.
Just keep the code that prevents reserves from being created for the
destination area.
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
---
tools/testing/selftests/vm/userfaultfd.c | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
index 9354a5e0321c..0b543a9a42b2 100644
--- a/tools/testing/selftests/vm/userfaultfd.c
+++ b/tools/testing/selftests/vm/userfaultfd.c
@@ -87,7 +87,7 @@ static bool test_uffdio_minor = false;
static bool map_shared;
static int shm_fd;
-static int huge_fd = -1; /* only used for hugetlb_shared test */
+static int huge_fd;
static char *huge_fd_off0;
static unsigned long long *count_verify;
static int uffd = -1;
@@ -223,9 +223,6 @@ static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
static void hugetlb_release_pages(char *rel_area)
{
- if (huge_fd == -1)
- return;
-
if (fallocate(huge_fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
rel_area == huge_fd_off0 ? 0 : nr_pages * page_size,
nr_pages * page_size))
@@ -238,17 +235,17 @@ static void hugetlb_allocate_area(void **alloc_area)
char **alloc_area_alias;
*alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- map_shared ? MAP_SHARED :
- MAP_PRIVATE | MAP_HUGETLB |
+ (map_shared ? MAP_SHARED : MAP_PRIVATE) |
+ MAP_HUGETLB |
(*alloc_area == area_src ? 0 : MAP_NORESERVE),
- huge_fd,
- *alloc_area == area_src ? 0 : nr_pages * page_size);
+ huge_fd, *alloc_area == area_src ? 0 :
+ nr_pages * page_size);
if (*alloc_area == MAP_FAILED)
err("mmap of hugetlbfs file failed");
if (map_shared) {
area_alias = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
- MAP_SHARED,
+ MAP_SHARED | MAP_HUGETLB,
huge_fd, *alloc_area == area_src ? 0 :
nr_pages * page_size);
if (area_alias == MAP_FAILED)
--
2.33.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
2022-01-04 2:17 [PATCH] userfaultfd/selftests: clean up hugetlb allocation code Mike Kravetz
@ 2022-01-04 22:35 ` Axel Rasmussen
2022-01-05 0:24 ` Mike Kravetz
2022-01-05 23:56 ` Andrew Morton
0 siblings, 2 replies; 6+ messages in thread
From: Axel Rasmussen @ 2022-01-04 22:35 UTC (permalink / raw)
To: Mike Kravetz
Cc: LKML, Linux MM, Peter Xu, Andrea Arcangeli, Mina Almasry,
Shuah Khan, Andrew Morton
On Mon, Jan 3, 2022 at 6:17 PM Mike Kravetz <mike.kravetz@oracle.com> wrote:
>
> The message for commit f5c73297181c ("userfaultfd/selftests: fix hugetlb
> area allocations") says there is no need to create a hugetlb file in the
> non-shared testing case. However, the commit did not actually change
> the code to prevent creation of the file.
>
> While it is technically true that there is no need to create and use a
> hugetlb file in the case of non-shared-testing, it is useful. This is
> because 'hole punching' of a hugetlb file has the potentially incorrect
> side effect of also removing pages from private mappings. The
> userfaultfd test relies on this side effect for removing pages from the
> destination buffer during rounds of stress testing.
>
> Remove the incomplete code that was added to deal with no hugetlb file.
> Just keep the code that prevents reserves from being created for the
> destination area.
>
> Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
> ---
> tools/testing/selftests/vm/userfaultfd.c | 15 ++++++---------
> 1 file changed, 6 insertions(+), 9 deletions(-)
>
> diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c
> index 9354a5e0321c..0b543a9a42b2 100644
> --- a/tools/testing/selftests/vm/userfaultfd.c
> +++ b/tools/testing/selftests/vm/userfaultfd.c
> @@ -87,7 +87,7 @@ static bool test_uffdio_minor = false;
>
> static bool map_shared;
> static int shm_fd;
> -static int huge_fd = -1; /* only used for hugetlb_shared test */
> +static int huge_fd;
> static char *huge_fd_off0;
> static unsigned long long *count_verify;
> static int uffd = -1;
> @@ -223,9 +223,6 @@ static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset)
>
> static void hugetlb_release_pages(char *rel_area)
> {
> - if (huge_fd == -1)
> - return;
> -
> if (fallocate(huge_fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
> rel_area == huge_fd_off0 ? 0 : nr_pages * page_size,
> nr_pages * page_size))
> @@ -238,17 +235,17 @@ static void hugetlb_allocate_area(void **alloc_area)
> char **alloc_area_alias;
>
> *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> - map_shared ? MAP_SHARED :
> - MAP_PRIVATE | MAP_HUGETLB |
> + (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> + MAP_HUGETLB |
> (*alloc_area == area_src ? 0 : MAP_NORESERVE),
> - huge_fd,
> - *alloc_area == area_src ? 0 : nr_pages * page_size);
> + huge_fd, *alloc_area == area_src ? 0 :
> + nr_pages * page_size);
Sorry to nitpick, but I think it was slightly more readable when the
ternary was all on one line.
> if (*alloc_area == MAP_FAILED)
> err("mmap of hugetlbfs file failed");
>
> if (map_shared) {
> area_alias = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> - MAP_SHARED,
> + MAP_SHARED | MAP_HUGETLB,
> huge_fd, *alloc_area == area_src ? 0 :
> nr_pages * page_size);
> if (area_alias == MAP_FAILED)
> --
> 2.33.1
>
Looks functionally correct to me besides the one style gripe,
Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
2022-01-04 22:35 ` Axel Rasmussen
@ 2022-01-05 0:24 ` Mike Kravetz
2022-01-05 23:56 ` Andrew Morton
1 sibling, 0 replies; 6+ messages in thread
From: Mike Kravetz @ 2022-01-05 0:24 UTC (permalink / raw)
To: Axel Rasmussen
Cc: LKML, Linux MM, Peter Xu, Andrea Arcangeli, Mina Almasry,
Shuah Khan, Andrew Morton
On 1/4/22 14:35, Axel Rasmussen wrote:
> On Mon, Jan 3, 2022 at 6:17 PM Mike Kravetz <mike.kravetz@oracle.com> wrote:
>
> Looks functionally correct to me besides the one style gripe,
>
> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com>
Thanks Axel!
The userfaultfd test is not so easy to follow. While looking at this, I
noticed that we skip the mremap() testing for hugetlb. However, Mina recently
added hugetlb mremap support. Unfortunately, mremap hugetlb testing fails
if enabled. I am still trying to determine the root cause, but would like
to eliminate this special case if possible.
--
Mike Kravetz
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
2022-01-04 22:35 ` Axel Rasmussen
2022-01-05 0:24 ` Mike Kravetz
@ 2022-01-05 23:56 ` Andrew Morton
2022-01-06 17:43 ` Mike Kravetz
1 sibling, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2022-01-05 23:56 UTC (permalink / raw)
To: Axel Rasmussen
Cc: Mike Kravetz, LKML, Linux MM, Peter Xu, Andrea Arcangeli,
Mina Almasry, Shuah Khan
On Tue, 4 Jan 2022 14:35:34 -0800 Axel Rasmussen <axelrasmussen@google.com> wrote:
> On Mon, Jan 3, 2022 at 6:17 PM Mike Kravetz <mike.kravetz@oracle.com> wrote:
> >
> > The message for commit f5c73297181c ("userfaultfd/selftests: fix hugetlb
> > area allocations") says there is no need to create a hugetlb file in the
> > non-shared testing case. However, the commit did not actually change
> > the code to prevent creation of the file.
> >
> > While it is technically true that there is no need to create and use a
> > hugetlb file in the case of non-shared-testing, it is useful. This is
> > because 'hole punching' of a hugetlb file has the potentially incorrect
> > side effect of also removing pages from private mappings. The
> > userfaultfd test relies on this side effect for removing pages from the
> > destination buffer during rounds of stress testing.
> >
> > Remove the incomplete code that was added to deal with no hugetlb file.
> > Just keep the code that prevents reserves from being created for the
> > destination area.
> >
> > *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> > - map_shared ? MAP_SHARED :
> > - MAP_PRIVATE | MAP_HUGETLB |
> > + (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> > + MAP_HUGETLB |
> > (*alloc_area == area_src ? 0 : MAP_NORESERVE),
> > - huge_fd,
> > - *alloc_area == area_src ? 0 : nr_pages * page_size);
> > + huge_fd, *alloc_area == area_src ? 0 :
> > + nr_pages * page_size);
>
> Sorry to nitpick, but I think it was slightly more readable when the
> ternary was all on one line.
When you have that many arguments I think it's clearer to put one per
line, viz.
*alloc_area = mmap(NULL,
nr_pages * page_size,
PROT_READ | PROT_WRITE,
(map_shared ? MAP_SHARED : MAP_PRIVATE) |
MAP_HUGETLB |
(*alloc_area == area_src ? 0 : MAP_NORESERVE),
huge_fd,
*alloc_area == area_src ? 0 : nr_pages * page_size);
But whatever...
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
2022-01-05 23:56 ` Andrew Morton
@ 2022-01-06 17:43 ` Mike Kravetz
2022-01-06 18:25 ` Axel Rasmussen
0 siblings, 1 reply; 6+ messages in thread
From: Mike Kravetz @ 2022-01-06 17:43 UTC (permalink / raw)
To: Andrew Morton, Axel Rasmussen
Cc: LKML, Linux MM, Peter Xu, Andrea Arcangeli, Mina Almasry, Shuah Khan
On 1/5/22 15:56, Andrew Morton wrote:
> On Tue, 4 Jan 2022 14:35:34 -0800 Axel Rasmussen <axelrasmussen@google.com> wrote:
>
>> On Mon, Jan 3, 2022 at 6:17 PM Mike Kravetz <mike.kravetz@oracle.com> wrote:
>>>
>>> The message for commit f5c73297181c ("userfaultfd/selftests: fix hugetlb
>>> area allocations") says there is no need to create a hugetlb file in the
>>> non-shared testing case. However, the commit did not actually change
>>> the code to prevent creation of the file.
>>>
>>> While it is technically true that there is no need to create and use a
>>> hugetlb file in the case of non-shared-testing, it is useful. This is
>>> because 'hole punching' of a hugetlb file has the potentially incorrect
>>> side effect of also removing pages from private mappings. The
>>> userfaultfd test relies on this side effect for removing pages from the
>>> destination buffer during rounds of stress testing.
>>>
>>> Remove the incomplete code that was added to deal with no hugetlb file.
>>> Just keep the code that prevents reserves from being created for the
>>> destination area.
>>>
>>> *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
>>> - map_shared ? MAP_SHARED :
>>> - MAP_PRIVATE | MAP_HUGETLB |
>>> + (map_shared ? MAP_SHARED : MAP_PRIVATE) |
>>> + MAP_HUGETLB |
>>> (*alloc_area == area_src ? 0 : MAP_NORESERVE),
>>> - huge_fd,
>>> - *alloc_area == area_src ? 0 : nr_pages * page_size);
>>> + huge_fd, *alloc_area == area_src ? 0 :
>>> + nr_pages * page_size);
>>
>> Sorry to nitpick, but I think it was slightly more readable when the
>> ternary was all on one line.
>
> When you have that many arguments I think it's clearer to put one per
> line, viz.
>
> *alloc_area = mmap(NULL,
> nr_pages * page_size,
> PROT_READ | PROT_WRITE,
> (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> MAP_HUGETLB |
> (*alloc_area == area_src ? 0 : MAP_NORESERVE),
> huge_fd,
> *alloc_area == area_src ? 0 : nr_pages * page_size);
>
>
> But whatever...
I agree, and also agree with Axel's comment about keeping the ternary all on
one line. However, there are examples of breaking both these conventions throughout the file.
My intention here was just to clean up the mess I created with the previous
patch. As such, I would prefer to leave this patch as is. If someone really
wants this modified, I will. However, IMO if we make this one call easier
to read, we should use the same convention throughout the file. I can do that
as well, but would prefer to first try to enable using mremap with hugetlb
within the test.
--
Mike Kravetz
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] userfaultfd/selftests: clean up hugetlb allocation code
2022-01-06 17:43 ` Mike Kravetz
@ 2022-01-06 18:25 ` Axel Rasmussen
0 siblings, 0 replies; 6+ messages in thread
From: Axel Rasmussen @ 2022-01-06 18:25 UTC (permalink / raw)
To: Mike Kravetz
Cc: Andrew Morton, LKML, Linux MM, Peter Xu, Andrea Arcangeli,
Mina Almasry, Shuah Khan
On Thu, Jan 6, 2022 at 9:43 AM Mike Kravetz <mike.kravetz@oracle.com> wrote:
>
> On 1/5/22 15:56, Andrew Morton wrote:
> > On Tue, 4 Jan 2022 14:35:34 -0800 Axel Rasmussen <axelrasmussen@google.com> wrote:
> >
> >> On Mon, Jan 3, 2022 at 6:17 PM Mike Kravetz <mike.kravetz@oracle.com> wrote:
> >>>
> >>> The message for commit f5c73297181c ("userfaultfd/selftests: fix hugetlb
> >>> area allocations") says there is no need to create a hugetlb file in the
> >>> non-shared testing case. However, the commit did not actually change
> >>> the code to prevent creation of the file.
> >>>
> >>> While it is technically true that there is no need to create and use a
> >>> hugetlb file in the case of non-shared-testing, it is useful. This is
> >>> because 'hole punching' of a hugetlb file has the potentially incorrect
> >>> side effect of also removing pages from private mappings. The
> >>> userfaultfd test relies on this side effect for removing pages from the
> >>> destination buffer during rounds of stress testing.
> >>>
> >>> Remove the incomplete code that was added to deal with no hugetlb file.
> >>> Just keep the code that prevents reserves from being created for the
> >>> destination area.
> >>>
> >>> *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE,
> >>> - map_shared ? MAP_SHARED :
> >>> - MAP_PRIVATE | MAP_HUGETLB |
> >>> + (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> >>> + MAP_HUGETLB |
> >>> (*alloc_area == area_src ? 0 : MAP_NORESERVE),
> >>> - huge_fd,
> >>> - *alloc_area == area_src ? 0 : nr_pages * page_size);
> >>> + huge_fd, *alloc_area == area_src ? 0 :
> >>> + nr_pages * page_size);
> >>
> >> Sorry to nitpick, but I think it was slightly more readable when the
> >> ternary was all on one line.
> >
> > When you have that many arguments I think it's clearer to put one per
> > line, viz.
> >
> > *alloc_area = mmap(NULL,
> > nr_pages * page_size,
> > PROT_READ | PROT_WRITE,
> > (map_shared ? MAP_SHARED : MAP_PRIVATE) |
> > MAP_HUGETLB |
> > (*alloc_area == area_src ? 0 : MAP_NORESERVE),
> > huge_fd,
> > *alloc_area == area_src ? 0 : nr_pages * page_size);
> >
> >
> > But whatever...
> I agree, and also agree with Axel's comment about keeping the ternary all on
> one line. However, there are examples of breaking both these conventions throughout the file.
For what it's worth, I don't at all mind Andrew's way either, where
the two "outcomes" of the ternary are indented a bit.
Not a big deal though, whatever you'd prefer is fine. :)
>
> My intention here was just to clean up the mess I created with the previous
> patch. As such, I would prefer to leave this patch as is. If someone really
> wants this modified, I will. However, IMO if we make this one call easier
> to read, we should use the same convention throughout the file. I can do that
> as well, but would prefer to first try to enable using mremap with hugetlb
> within the test.
+1, sounds like a good plan.
> --
> Mike Kravetz
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-06 18:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-04 2:17 [PATCH] userfaultfd/selftests: clean up hugetlb allocation code Mike Kravetz
2022-01-04 22:35 ` Axel Rasmussen
2022-01-05 0:24 ` Mike Kravetz
2022-01-05 23:56 ` Andrew Morton
2022-01-06 17:43 ` Mike Kravetz
2022-01-06 18:25 ` Axel Rasmussen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.