[PATCH linux-next] selftests/vm: calculate variables in correct order

[PATCH linux-next] selftests/vm: calculate variables in correct order

[Joel Savitz]

commit b5ba705c2608 ("selftests/vm: enable running select groups of tests")
unintentionally reversed the ordering of some of the lines of
run_vmtests.sh that calculate values based on system configuration.
Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later
value is not read from /proc/meminfo until later, causing userfaultfd
tests to incorrectly fail since $half_ufd_size_MB will always be 0.

Switch these statements around into proper order to fix the invocation
of the userfaultfd tests that use $half_ufd_size_MB.

Suggested-by: Nico Pache <npache@redhat.com>
Signed-off-by: Joel Savitz <jsavitz@redhat.com>
---
 tools/testing/selftests/vm/run_vmtests.sh | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh
index fff00bb77086..ce52e4f5ff21 100755
--- a/tools/testing/selftests/vm/run_vmtests.sh
+++ b/tools/testing/selftests/vm/run_vmtests.sh
@@ -82,16 +82,6 @@ test_selected() {
 	fi
 }
 
-# Simple hugetlbfs tests have a hardcoded minimum requirement of
-# huge pages totaling 256MB (262144KB) in size.  The userfaultfd
-# hugetlb test requires a minimum of 2 * nr_cpus huge pages.  Take
-# both of these requirements into account and attempt to increase
-# number of huge pages available.
-nr_cpus=$(nproc)
-hpgsize_MB=$((hpgsize_KB / 1024))
-half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
-needmem_KB=$((half_ufd_size_MB * 2 * 1024))
-
 # get huge pagesize and freepages from /proc/meminfo
 while read -r name size unit; do
 	if [ "$name" = "HugePages_Free:" ]; then
@@ -102,6 +92,16 @@ while read -r name size unit; do
 	fi
 done < /proc/meminfo
 
+# Simple hugetlbfs tests have a hardcoded minimum requirement of
+# huge pages totaling 256MB (262144KB) in size.  The userfaultfd
+# hugetlb test requires a minimum of 2 * nr_cpus huge pages.  Take
+# both of these requirements into account and attempt to increase
+# number of huge pages available.
+nr_cpus=$(nproc)
+hpgsize_MB=$((hpgsize_KB / 1024))
+half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
+needmem_KB=$((half_ufd_size_MB * 2 * 1024))
+
 # set proper nr_hugepages
 if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
 	nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages)
-- 
2.31.1

Re: [PATCH linux-next] selftests/vm: calculate variables in correct order

[Andrew Morton]

On Fri, 28 Oct 2022 09:26:40 -0400 Joel Savitz <jsavitz@redhat.com> wrote:

> commit b5ba705c2608 ("selftests/vm: enable running select groups of tests")
> unintentionally reversed the ordering of some of the lines of
> run_vmtests.sh that calculate values based on system configuration.
> Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later
> value is not read from /proc/meminfo until later, causing userfaultfd
> tests to incorrectly fail since $half_ufd_size_MB will always be 0.
> 
> Switch these statements around into proper order to fix the invocation
> of the userfaultfd tests that use $half_ufd_size_MB.

Does this fix address the failure in
https://lkml.kernel.org/r/202211021026.61b267d1-yujie.liu@intel.com?

Thanks.

Re: [PATCH linux-next] selftests/vm: calculate variables in correct order

[Joel Savitz]

On Wed, Nov 16, 2022 at 8:09 PM Joel Savitz <jsavitz@redhat.com> wrote:
>
> On Tue, Nov 8, 2022 at 8:31 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>>
>> On Fri, 28 Oct 2022 09:26:40 -0400 Joel Savitz <jsavitz@redhat.com> wrote:
>>
>> > commit b5ba705c2608 ("selftests/vm: enable running select groups of tests")
>> > unintentionally reversed the ordering of some of the lines of
>> > run_vmtests.sh that calculate values based on system configuration.
>> > Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later
>> > value is not read from /proc/meminfo until later, causing userfaultfd
>> > tests to incorrectly fail since $half_ufd_size_MB will always be 0.
>> >
>> > Switch these statements around into proper order to fix the invocation
>> > of the userfaultfd tests that use $half_ufd_size_MB.
>>
>> Does this fix address the failure in
>> https://lkml.kernel.org/r/202211021026.61b267d1-yujie.liu@intel.com?
>>
>> Thanks.
>>
>
> I have tried to reproduce this failure on a couple of different systems before and after the application of this commit but I haven't had any success in doing so. I suspect that there was some sort of hugepage configuration issue on the test system but I'd have to look into it more to be sure.
>
> However, I noticed that on the mm-everything branch, the hugepage-mmap test fails:
>
> # ./run_vmtests.sh -t "hugetlb"
> running: ./hugepage-mmap
> -----------------------
> running ./hugepage-mmap
> -----------------------
> Open failed: No such file or directory
> [FAIL]
> ...
>
> It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt point for hugetlb in run_vmtests.sh")
> as the test still replies on the ./huge mountpoint removed in that commit. The test passes before that patchset is applied.
>
> Additionally, I just noticed an extraneous 'echo "running: $1"' line in run_test(), the effects of which are seen above, and I have just sent a patch to remove it.
>
> Joel

Resending this reply since it appears a bit of HTML slipped into the
last reply and it got rejected by the lists.

Re: [PATCH linux-next] selftests/vm: calculate variables in correct order

[Peter Xu]

[-- Attachment #1: Type: text/plain, Size: 726 bytes --]

On Wed, Nov 16, 2022 at 08:09:11PM -0400, Joel Savitz wrote:
> However, I noticed that on the mm-everything branch, the hugepage-mmap test
> fails:
> 
> # ./run_vmtests.sh -t "hugetlb"
> running: ./hugepage-mmap
> -----------------------
> running ./hugepage-mmap
> -----------------------
> Open failed: No such file or directory
> [FAIL]
> ...
> 
> It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt
> point for hugetlb in run_vmtests.sh")
> as the test still replies on the ./huge mountpoint removed in that commit.
> The test passes before that patchset is applied.

Oops, sorry I totally overlooked this hard-coded test case using the
mntpoint.

Fix is simple though, which is attached.

-- 
Peter Xu

[-- Attachment #2: 0001-selftests-vm-use-memfd-for-hugepage-mmap-test.patch --]
[-- Type: text/plain, Size: 1755 bytes --]

From 71da2480d4bac0fc598e4d1f05f71aba8b980bc4 Mon Sep 17 00:00:00 2001
From: Peter Xu <peterx@redhat.com>
Date: Thu, 17 Nov 2022 16:29:15 -0500
Subject: [PATCH] selftests/vm: use memfd for hugepage-mmap test
Content-type: text/plain

This test was overlooked with a hard-coded mntpoint path in test when we're
removing the hugetlb mntpoint in commit 0796c7b8be84.  Fix it up so the
test can keep running.

Reported-by: Joel Savitz <jsavitz@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tools/testing/selftests/vm/hugepage-mmap.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/vm/hugepage-mmap.c b/tools/testing/selftests/vm/hugepage-mmap.c
index 93f9e7b81331..955ef87f382c 100644
--- a/tools/testing/selftests/vm/hugepage-mmap.c
+++ b/tools/testing/selftests/vm/hugepage-mmap.c
@@ -16,14 +16,13 @@
  * range.
  * Other architectures, such as ppc64, i386 or x86_64 are not so constrained.
  */
-
+#define _GNU_SOURCE
 #include <stdlib.h>
 #include <stdio.h>
 #include <unistd.h>
 #include <sys/mman.h>
 #include <fcntl.h>
 
-#define FILE_NAME "huge/hugepagefile"
 #define LENGTH (256UL*1024*1024)
 #define PROTECTION (PROT_READ | PROT_WRITE)
 
@@ -67,16 +66,16 @@ int main(void)
 	void *addr;
 	int fd, ret;
 
-	fd = open(FILE_NAME, O_CREAT | O_RDWR, 0755);
+	fd = memfd_create("hugepage-mmap", MFD_HUGETLB);
 	if (fd < 0) {
-		perror("Open failed");
+		perror("memfd_create() failed");
 		exit(1);
 	}
 
 	addr = mmap(ADDR, LENGTH, PROTECTION, FLAGS, fd, 0);
 	if (addr == MAP_FAILED) {
 		perror("mmap");
-		unlink(FILE_NAME);
+		close(fd);
 		exit(1);
 	}
 
@@ -87,7 +86,6 @@ int main(void)
 
 	munmap(addr, LENGTH);
 	close(fd);
-	unlink(FILE_NAME);
 
 	return ret;
 }
-- 
2.37.3

Re: [PATCH linux-next] selftests/vm: calculate variables in correct order

[Mike Kravetz]

On 11/17/22 16:33, Peter Xu wrote:
> On Wed, Nov 16, 2022 at 08:09:11PM -0400, Joel Savitz wrote:
> > However, I noticed that on the mm-everything branch, the hugepage-mmap test
> > fails:
> > 
> > # ./run_vmtests.sh -t "hugetlb"
> > running: ./hugepage-mmap
> > -----------------------
> > running ./hugepage-mmap
> > -----------------------
> > Open failed: No such file or directory
> > [FAIL]
> > ...
> > 
> > It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt
> > point for hugetlb in run_vmtests.sh")
> > as the test still replies on the ./huge mountpoint removed in that commit.
> > The test passes before that patchset is applied.
> 
> Oops, sorry I totally overlooked this hard-coded test case using the
> mntpoint.
> 
> Fix is simple though, which is attached.
> 
> -- 
> Peter Xu

> From 71da2480d4bac0fc598e4d1f05f71aba8b980bc4 Mon Sep 17 00:00:00 2001
> From: Peter Xu <peterx@redhat.com>
> Date: Thu, 17 Nov 2022 16:29:15 -0500
> Subject: [PATCH] selftests/vm: use memfd for hugepage-mmap test
> Content-type: text/plain
> 
> This test was overlooked with a hard-coded mntpoint path in test when we're
> removing the hugetlb mntpoint in commit 0796c7b8be84.  Fix it up so the
> test can keep running.
> 
> Reported-by: Joel Savitz <jsavitz@redhat.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  tools/testing/selftests/vm/hugepage-mmap.c | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)

Thanks Peter!
That is also something I noticed and was on my todo list.

Acked-by: Mike Kravetz <mike.kravetz@oracle.com>
-- 
Mike Kravetz