[PATCH 0/1] selftests/vm: 8x compaction_test speedup

[PATCH 0/1] selftests/vm: 8x compaction_test speedup

[John Hubbard]

Hi,

Maybe this should really be an RFC, given that I don't fully understand
why the compaction_test.c program was mmap'ing 1 MB at a time. So
apologies in advance if I've mucked up something important, but if so,
maybe we can still find a way to get this fixed up to something better.

Anyway: there are 20+ tests in tools/testing/selftests/vm/. The entire
running time for these (via run_vmtest.sh) is about 56 seconds, of which
over half is due to just one test: compaction_test, which takes 27 sec!
(A runner-up is HMM, at 11 sec, so it's up for a look next.) The other
tests mostly take a few ms, and a few take 1.0 sec.

This drops the compaction_test run time from 27, to 3.3 sec. Enjoy. :)

thanks,
John Hubbard
NVIDIA

John Hubbard (1):
  selftests/vm: 8x compaction_test speedup

 tools/testing/selftests/vm/compaction_test.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

-- 
2.28.0

[PATCH 1/1] selftests/vm: 8x compaction_test speedup

[John Hubbard]

This patch reduces the running time for compaction_test from about 27
sec, to 3.3 sec, which is about an 8x speedup.

These numbers are for an Intel x86_64 system with 32 GB of DRAM.

The compaction_test.c program was spending most of its time doing
mmap(), 1 MB at a time, on about 25 GB of memory.

Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make
things go much faster, because other parts of the program are using the
remaining time.)

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
 tools/testing/selftests/vm/compaction_test.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/vm/compaction_test.c b/tools/testing/selftests/vm/compaction_test.c
index bcec71250873..9b420140ba2b 100644
--- a/tools/testing/selftests/vm/compaction_test.c
+++ b/tools/testing/selftests/vm/compaction_test.c
@@ -18,7 +18,8 @@
 
 #include "../kselftest.h"
 
-#define MAP_SIZE 1048576
+#define MAP_SIZE_MB	100
+#define MAP_SIZE	(MAP_SIZE_MB * 1024 * 1024)
 
 struct map_list {
 	void *map;
@@ -165,7 +166,7 @@ int main(int argc, char **argv)
 	void *map = NULL;
 	unsigned long mem_free = 0;
 	unsigned long hugepage_size = 0;
-	unsigned long mem_fragmentable = 0;
+	long mem_fragmentable_MB = 0;
 
 	if (prereq() != 0) {
 		printf("Either the sysctl compact_unevictable_allowed is not\n"
@@ -190,9 +191,9 @@ int main(int argc, char **argv)
 		return -1;
 	}
 
-	mem_fragmentable = mem_free * 0.8 / 1024;
+	mem_fragmentable_MB = mem_free * 0.8 / 1024;
 
-	while (mem_fragmentable > 0) {
+	while (mem_fragmentable_MB > 0) {
 		map = mmap(NULL, MAP_SIZE, PROT_READ | PROT_WRITE,
 			   MAP_ANONYMOUS | MAP_PRIVATE | MAP_LOCKED, -1, 0);
 		if (map == MAP_FAILED)
@@ -213,7 +214,7 @@ int main(int argc, char **argv)
 		for (i = 0; i < MAP_SIZE; i += page_size)
 			*(unsigned long *)(map + i) = (unsigned long)map + i;
 
-		mem_fragmentable--;
+		mem_fragmentable_MB -= MAP_SIZE_MB;
 	}
 
 	for (entry = list; entry != NULL; entry = entry->next) {
-- 
2.28.0

Re: [PATCH 1/1] selftests/vm: 8x compaction_test speedup

[Andrew Morton]

On Fri, 2 Oct 2020 01:06:21 -0700 John Hubbard <jhubbard@nvidia.com> wrote:

> This patch reduces the running time for compaction_test from about 27
> sec, to 3.3 sec, which is about an 8x speedup.
> 
> These numbers are for an Intel x86_64 system with 32 GB of DRAM.
> 
> The compaction_test.c program was spending most of its time doing
> mmap(), 1 MB at a time, on about 25 GB of memory.
> 
> Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make
> things go much faster, because other parts of the program are using the
> remaining time.)

Seems nice.  It's been 5 years, but hopefully Sri is still at Akamai?

> --- a/tools/testing/selftests/vm/compaction_test.c
> +++ b/tools/testing/selftests/vm/compaction_test.c
> @@ -18,7 +18,8 @@
>  
>  #include "../kselftest.h"
>  
> -#define MAP_SIZE 1048576
> +#define MAP_SIZE_MB	100
> +#define MAP_SIZE	(MAP_SIZE_MB * 1024 * 1024)
>  
>  struct map_list {
>  	void *map;
> @@ -165,7 +166,7 @@ int main(int argc, char **argv)
>  	void *map = NULL;
>  	unsigned long mem_free = 0;
>  	unsigned long hugepage_size = 0;
> -	unsigned long mem_fragmentable = 0;
> +	long mem_fragmentable_MB = 0;
>  
>  	if (prereq() != 0) {
>  		printf("Either the sysctl compact_unevictable_allowed is not\n"
> @@ -190,9 +191,9 @@ int main(int argc, char **argv)
>  		return -1;
>  	}
>  
> -	mem_fragmentable = mem_free * 0.8 / 1024;
> +	mem_fragmentable_MB = mem_free * 0.8 / 1024;
>  
> -	while (mem_fragmentable > 0) {
> +	while (mem_fragmentable_MB > 0) {
>  		map = mmap(NULL, MAP_SIZE, PROT_READ | PROT_WRITE,
>  			   MAP_ANONYMOUS | MAP_PRIVATE | MAP_LOCKED, -1, 0);
>  		if (map == MAP_FAILED)
> @@ -213,7 +214,7 @@ int main(int argc, char **argv)
>  		for (i = 0; i < MAP_SIZE; i += page_size)
>  			*(unsigned long *)(map + i) = (unsigned long)map + i;
>  
> -		mem_fragmentable--;
> +		mem_fragmentable_MB -= MAP_SIZE_MB;
>  	}
>  
>  	for (entry = list; entry != NULL; entry = entry->next) {
> -- 
> 2.28.0
> 

Re: [PATCH 1/1] selftests/vm: 8x compaction_test speedup

[Jayaramappa, Srilakshmi]

On 10/2/20, 6:05 PM, "Andrew Morton" <akpm@linux-foundation.org> wrote:

    On Fri, 2 Oct 2020 01:06:21 -0700 John Hubbard <jhubbard@nvidia.com> wrote:

    > This patch reduces the running time for compaction_test from about 27
    > sec, to 3.3 sec, which is about an 8x speedup.
    > 
    > These numbers are for an Intel x86_64 system with 32 GB of DRAM.
    > 
    > The compaction_test.c program was spending most of its time doing
    > mmap(), 1 MB at a time, on about 25 GB of memory.
    > 
    > Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make
    > things go much faster, because other parts of the program are using the
    > remaining time.)

    Seems nice.  It's been 5 years, but hopefully Sri is still at Akamai?

    Looks good to me.

    Thanks
    -Sri