[LTP] [PATCH 0/2 v4] Add mbind and set_mempolicy tests

[LTP] [PATCH 0/2 v4] Add mbind and set_mempolicy tests

[Cyril Hrubis]

Changes from v3 for the set_mempolicy patch follows comments from Jan,
the allocation stays in the library but was made generic enough to
support mbind tests.

The mbind tests are not 100% ready to be merged, but are included to
show that the API is flexible enough now.

Cyril Hrubis (2):
  syscalls: Add set_mempolicy numa tests.
  syscalls/mbind0{2,3,4}: Add basic mbind tests

 include/tst_numa.h                            | 112 ++++++++++
 lib/tst_numa.c                                | 208 ++++++++++++++++++
 runtest/numa                                  |   5 +
 runtest/syscalls                              |   3 +
 testcases/kernel/syscalls/mbind/.gitignore    |   3 +
 testcases/kernel/syscalls/mbind/mbind.h       |  26 +++
 testcases/kernel/syscalls/mbind/mbind02.c     |  99 +++++++++
 testcases/kernel/syscalls/mbind/mbind03.c     | 123 +++++++++++
 testcases/kernel/syscalls/mbind/mbind04.c     | 127 +++++++++++
 .../kernel/syscalls/set_mempolicy/.gitignore  |   4 +
 .../kernel/syscalls/set_mempolicy/Makefile    |   7 +
 .../syscalls/set_mempolicy/set_mempolicy.h    |  21 ++
 .../syscalls/set_mempolicy/set_mempolicy01.c  | 120 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy02.c  | 117 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy03.c  | 113 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy04.c  | 142 ++++++++++++
 16 files changed, 1230 insertions(+)
 create mode 100644 include/tst_numa.h
 create mode 100644 lib/tst_numa.c
 create mode 100644 testcases/kernel/syscalls/mbind/mbind.h
 create mode 100644 testcases/kernel/syscalls/mbind/mbind02.c
 create mode 100644 testcases/kernel/syscalls/mbind/mbind03.c
 create mode 100644 testcases/kernel/syscalls/mbind/mbind04.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/.gitignore
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/Makefile
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c

-- 
2.19.2

[LTP] [PATCH 1/2 v4] syscalls: Add set_mempolicy numa tests.

[Cyril Hrubis]

This is initial attempt to replace numa.sh tests that despite having
been fixed several times have still many shortcommings that wouldn't
easy to fix. It's not finished nor 100% replacement at this point but it
should be pretty good start.

The main selling points of these testcases are:

The memory allocated for the testing is tracked exactly. We are using
get_mempolicy() with MPOL_F_NODE | MPOL_F_ADDR that returns the node ID on
which specified address is allocated on to count pages allocated per node after
we set desired memory policy.

We also check for free memory on each numa memory mode and skip nodes
that don't have sufficient amount of memory for a particular test. The
tests checks usuall for twice as much memory per each node in order to
allow for allocations to be "misplaced".

The tests for file based shared interleaved mappings are no longer
mapping a single small file but rather than that we accumulate statistic
for larger amount of files over longer period of time and we also allow
for small offset (currently 10%). We should probably also increase the
number of samples we take as currently it's about 5MB in total on x86
although I haven't managed to make this test fail so far. This also
fixes the test on Btrfs where the synthetic test that expects the pages
to be distributed exactly equally fails.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
CC: Michal Hocko <mhocko@kernel.org>
CC: Vlastimil Babka <vbabka@suse.cz>
CC: Jan Stancek <jstancek@redhat.com>
---
 include/tst_numa.h                            | 112 ++++++++++
 lib/tst_numa.c                                | 208 ++++++++++++++++++
 runtest/numa                                  |   5 +
 .../kernel/syscalls/set_mempolicy/.gitignore  |   4 +
 .../kernel/syscalls/set_mempolicy/Makefile    |   7 +
 .../syscalls/set_mempolicy/set_mempolicy.h    |  21 ++
 .../syscalls/set_mempolicy/set_mempolicy01.c  | 120 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy02.c  | 117 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy03.c  | 113 ++++++++++
 .../syscalls/set_mempolicy/set_mempolicy04.c  | 142 ++++++++++++
 10 files changed, 849 insertions(+)
 create mode 100644 include/tst_numa.h
 create mode 100644 lib/tst_numa.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/.gitignore
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/Makefile
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
 create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c

diff --git a/include/tst_numa.h b/include/tst_numa.h
new file mode 100644
index 000000000..a4cd1be37
--- /dev/null
+++ b/include/tst_numa.h
@@ -0,0 +1,112 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#ifndef TST_NUMA_H__
+#define TST_NUMA_H__
+
+/**
+ * Numa nodemap.
+ */
+struct tst_nodemap {
+        /** Number of nodes in map */
+	unsigned int cnt;
+	/** Page allocation counters */
+	unsigned int *counters;
+	/** Array of numa ids */
+	unsigned int map[];
+};
+
+/**
+ * Clears numa counters. The counters are lazy-allocated on first call of this function.
+ *
+ * @nodes Numa nodemap.
+ */
+void tst_nodemap_reset_counters(struct tst_nodemap *nodes);
+
+/**
+ * Prints pages allocated per each node.
+ *
+ * @nodes Numa nodemap.
+ */
+void tst_nodemap_print_counters(struct tst_nodemap *nodes);
+
+/**
+ * Returns a name for a mempolicy/mbind mode.
+ *
+ * @mode Numa mempolicy mode.
+ */
+const char *tst_numa_mode_name(int mode);
+
+/**
+ * Maps pages into memory, if path is NULL the mapping is anonymous otherwise is backed by the file.
+ *
+ * @path Path to a file, if not NULL mapping is file based.
+ * @size Mapping size.
+ */
+void *tst_numa_map(const char *path, size_t size);
+
+/*
+ * Writes to memory in order to get the pages faulted.
+ *
+ * @ptr Start of the mapping.
+ * @size Size of the mapping.
+ */
+static inline void tst_numa_fault(void *ptr, size_t size)
+{
+	memset(ptr, 'a', size);
+}
+
+/*
+ * Frees the memory.
+ *
+ * @ptr Start of the mapping.
+ * @size Size of the mapping.
+ */
+static inline void tst_numa_unmap(void *ptr, size_t size)
+{
+	SAFE_MUNMAP(ptr, size);
+}
+
+/**
+ * Check on which numa node resides each page of the mapping starting at ptr
+ * and continuing pages long and increases nodemap counters accordingly.
+ *
+ * @nodes Nodemap with initialized counters.
+ * @ptr   Pointer to start of a mapping.
+ * @size  Size of the mapping.
+ */
+void tst_nodemap_count_pages(struct tst_nodemap *nodes, void *ptr, size_t size);
+
+/**
+ * Frees nodemap.
+ *
+ * @nodes Numa nodemap to be freed.
+ */
+void tst_nodemap_free(struct tst_nodemap *nodes);
+
+/**
+ * Bitflags for tst_get_nodemap() function.
+ */
+enum tst_numa_types {
+	TST_NUMA_ANY = 0x00,
+	TST_NUMA_MEM = 0x01,
+};
+
+/**
+ * Allocates and returns numa node map, which is an array of numa nodes which
+ * contain desired resources e.g. memory.
+ *
+ * @type       Bitflags of enum tst_numa_types specifying desired resources.
+ * @min_mem_kb Minimal free RAM on memory nodes, if given node has less than
+ *             requested amount of free+buffers memory it's not included in
+ *             the resulting list of nodes.
+ *
+ * @return On success returns allocated and initialized struct tst_nodemap which contains
+ *         array of numa node ids that contains desired resources.
+ */
+struct tst_nodemap *tst_get_nodemap(int type, size_t min_mem_kb);
+
+#endif /* TST_NUMA_H__ */
diff --git a/lib/tst_numa.c b/lib/tst_numa.c
new file mode 100644
index 000000000..9c33115fd
--- /dev/null
+++ b/lib/tst_numa.c
@@ -0,0 +1,208 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <ctype.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+#endif
+
+#define TST_NO_DEFAULT_MAIN
+#include "tst_test.h"
+#include "tst_numa.h"
+
+void tst_nodemap_print_counters(struct tst_nodemap *nodes)
+{
+	unsigned int i;
+
+	for (i = 0; i < nodes->cnt; i++) {
+		tst_res(TINFO, "Node %i allocated %u pages",
+		        nodes->map[i], nodes->counters[i]);
+	}
+}
+
+void tst_nodemap_reset_counters(struct tst_nodemap *nodes)
+{
+	size_t arr_size = sizeof(unsigned int) * nodes->cnt;
+
+	if (!nodes->counters)
+		nodes->counters = SAFE_MALLOC(arr_size);
+
+	memset(nodes->counters, 0, arr_size);
+}
+
+void tst_nodemap_free(struct tst_nodemap *nodes)
+{
+	free(nodes->counters);
+	free(nodes);
+}
+
+#ifdef HAVE_NUMA_H
+
+const char *tst_numa_mode_name(int mode)
+{
+	switch (mode) {
+	case MPOL_DEFAULT:
+		return "MPOL_DEFAULT";
+	case MPOL_BIND:
+		return "MPOL_BIND";
+	case MPOL_PREFERRED:
+		return "MPOL_PREFERRED";
+	case MPOL_INTERLEAVE:
+		return "MPOL_INTERLEAVE";
+	default:
+		return "???";
+	}
+}
+
+
+static void inc_counter(unsigned int node, struct tst_nodemap *nodes)
+{
+	unsigned int i;
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->map[i] == node) {
+			nodes->counters[i]++;
+			break;
+		}
+	}
+}
+
+void tst_nodemap_count_pages(struct tst_nodemap *nodes,
+                             void *ptr, size_t size)
+{
+	size_t page_size = getpagesize();
+	unsigned int i;
+	int node;
+	unsigned int pages = (size + page_size - 1)/page_size;
+
+	for (i = 0; i < pages; i++) {
+		get_mempolicy(&node, NULL, 0, ptr + i * page_size, MPOL_F_NODE | MPOL_F_ADDR);
+
+		if (node < 0 || (unsigned int)node >= nodes->cnt) {
+			tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node);
+			continue;
+		}
+
+		inc_counter(node, nodes);
+	}
+}
+
+void *tst_numa_map(const char *path, size_t size)
+{
+	char *ptr;
+	int fd = -1;
+	int flags = MAP_PRIVATE|MAP_ANONYMOUS;
+
+	if (path) {
+		fd = SAFE_OPEN(path, O_CREAT | O_EXCL | O_RDWR, 0666);
+		SAFE_FTRUNCATE(fd, size);
+		flags = MAP_SHARED;
+	}
+
+	ptr = SAFE_MMAP(NULL, size,
+	                PROT_READ|PROT_WRITE, flags, fd, 0);
+
+	if (path) {
+		SAFE_CLOSE(fd);
+		SAFE_UNLINK(path);
+	}
+
+	return ptr;
+}
+
+static int node_has_enough_memory(int node, size_t min_kb)
+{
+	char path[1024];
+	char buf[1024];
+	long mem_total = 0;
+	long mem_used = 0;
+
+	snprintf(path, sizeof(path), "/sys/devices/system/node/node%i/meminfo", node);
+
+	if (access(path, F_OK)) {
+		tst_res(TINFO, "File '%s' does not exist! NUMA not enabled?", path);
+		return 0;
+	}
+
+	FILE *fp = fopen(path, "r");
+	if (!fp)
+		tst_brk(TBROK | TERRNO, "Failed to open '%s'", path);
+
+	while (fgets(buf, sizeof(buf), fp)) {
+		long val;
+
+		if (sscanf(buf, "%*s %*i MemTotal: %li", &val) == 1)
+			mem_total = val;
+
+		if (sscanf(buf, "%*s %*i MemUsed: %li", &val) == 1)
+			mem_used = val;
+	}
+
+	fclose(fp);
+
+	if (!mem_total || !mem_used) {
+		tst_res(TWARN, "Failed to parse '%s'", path);
+		return 0;
+	}
+
+	if (mem_total - mem_used < (long)min_kb) {
+		tst_res(TINFO,
+		        "Not enough free RAM on node %i, have %likB needs %zukB",
+		        node, mem_total - mem_used, min_kb);
+		return 0;
+	}
+
+	return 1;
+}
+
+struct tst_nodemap *tst_get_nodemap(int type, size_t min_mem_kb)
+{
+	struct bitmask *membind;
+	struct tst_nodemap *nodes;
+	unsigned int i, cnt;
+
+	if (type & ~(TST_NUMA_MEM))
+		tst_brk(TBROK, "Invalid type %i\n", type);
+
+	membind = numa_get_membind();
+
+	cnt = 0;
+	for (i = 0; i < membind->size; i++) {
+		if (type & TST_NUMA_MEM && !numa_bitmask_isbitset(membind, i))
+			continue;
+
+		cnt++;
+	}
+
+	tst_res(TINFO, "Found %u NUMA memory nodes", cnt);
+
+	nodes = SAFE_MALLOC(sizeof(struct tst_nodemap)
+	                    + sizeof(unsigned int) * cnt);
+	nodes->cnt = cnt;
+	nodes->counters = NULL;
+
+	cnt = 0;
+	for (i = 0; i < membind->size; i++) {
+		if (type & TST_NUMA_MEM &&
+		    (!numa_bitmask_isbitset(membind, i) ||
+		     !node_has_enough_memory(i, min_mem_kb)))
+			continue;
+
+		nodes->map[cnt++] = i;
+	}
+
+	nodes->cnt = cnt;
+
+	numa_bitmask_free(membind);
+
+	return nodes;
+}
+
+#endif
diff --git a/runtest/numa b/runtest/numa
index 12aedbb4b..7885be90c 100644
--- a/runtest/numa
+++ b/runtest/numa
@@ -11,3 +11,8 @@ move_pages09 move_pages09
 move_pages10 move_pages10
 move_pages11 move_pages11
 move_pages12 move_pages12
+
+set_mempolicy01 set_mempolicy01
+set_mempolicy02 set_mempolicy02
+set_mempolicy03 set_mempolicy03
+set_mempolicy04 set_mempolicy04
diff --git a/testcases/kernel/syscalls/set_mempolicy/.gitignore b/testcases/kernel/syscalls/set_mempolicy/.gitignore
new file mode 100644
index 000000000..c5e35a405
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/.gitignore
@@ -0,0 +1,4 @@
+/set_mempolicy01
+/set_mempolicy02
+set_mempolicy03
+set_mempolicy04
diff --git a/testcases/kernel/syscalls/set_mempolicy/Makefile b/testcases/kernel/syscalls/set_mempolicy/Makefile
new file mode 100644
index 000000000..d273b432b
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/Makefile
@@ -0,0 +1,7 @@
+top_srcdir		?= ../../../..
+
+include $(top_srcdir)/include/mk/testcases.mk
+
+LDLIBS  += $(NUMA_LIBS)
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h
new file mode 100644
index 000000000..da6419e18
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h
@@ -0,0 +1,21 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#ifndef SET_MEMPOLICY_H__
+#define SET_MEMPOLICY_H__
+
+static inline void alloc_fault_count(struct tst_nodemap *nodes,
+                                     const char *file, size_t size)
+{
+	void *ptr;
+
+	ptr = tst_numa_map(file, size);
+	tst_numa_fault(ptr, size);
+	tst_nodemap_count_pages(nodes, ptr, size);
+	tst_numa_unmap(ptr, size);
+}
+
+#endif /* SET_MEMPOLICY_H__ */
diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c
new file mode 100644
index 000000000..d3c17bb9b
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c
@@ -0,0 +1,120 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing set_mempolicy() with MPOL_BIND and MPOL_PREFERRED.
+ *
+ * For each node with memory we set its bit in nodemask with set_mempolicy()
+ * and verify that memory has been allocated accordingly.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#ifdef HAVE_NUMA_H
+
+#include "set_mempolicy.h"
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+#define PAGES_ALLOCATED 16u
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * PAGES_ALLOCATED * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void verify_mempolicy(unsigned int node, int mode)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int i;
+
+	numa_bitmask_setbit(bm, node);
+
+	TEST(set_mempolicy(mode, bm->maskp, bm->size+1));
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "set_mempolicy(%s) node %u",
+		        tst_numa_mode_name(mode), node);
+		return;
+	}
+
+	tst_res(TPASS, "set_mempolicy(%s) node %u",
+	        tst_numa_mode_name(mode), node);
+
+	numa_free_nodemask(bm);
+
+	const char *prefix = "child: ";
+
+	if (SAFE_FORK()) {
+		prefix = "parent: ";
+		tst_reap_children();
+	}
+
+	tst_nodemap_reset_counters(nodes);
+	alloc_fault_count(nodes, NULL, PAGES_ALLOCATED * page_size);
+	tst_nodemap_print_counters(nodes);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->map[i] == node) {
+			if (nodes->counters[i] == PAGES_ALLOCATED) {
+				tst_res(TPASS, "%sNode %u allocated %u",
+				        prefix, node, PAGES_ALLOCATED);
+			} else {
+				tst_res(TFAIL, "%sNode %u allocated %u, expected %u",
+				        prefix, node, nodes->counters[i],
+				        PAGES_ALLOCATED);
+			}
+			continue;
+		}
+
+		if (nodes->counters[i]) {
+			tst_res(TFAIL, "%sNode %u allocated %u, expected 0",
+			        prefix, i, nodes->counters[i]);
+		}
+	}
+}
+
+static void verify_set_mempolicy(unsigned int n)
+{
+	unsigned int i;
+	int mode = n ? MPOL_PREFERRED : MPOL_BIND;
+
+	for (i = 0; i < nodes->cnt; i++)
+		verify_mempolicy(nodes->map[i], mode);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_set_mempolicy,
+	.tcnt = 2,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c
new file mode 100644
index 000000000..e4387d274
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c
@@ -0,0 +1,117 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing set_mempolicy() with MPOL_INTERLEAVE.
+ *
+ * The test tries different subsets of memory nodes, sets the mask with
+ * memopolicy, and checks that the memory was interleaved between the nodes
+ * accordingly.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#ifdef HAVE_NUMA_H
+
+#include "set_mempolicy.h"
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 16 * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void alloc_and_check(size_t size, unsigned int *exp_alloc)
+{
+	unsigned int i;
+	const char *prefix = "child: ";
+
+	if (SAFE_FORK()) {
+		prefix = "parent: ";
+		tst_reap_children();
+	}
+
+	tst_nodemap_reset_counters(nodes);
+	alloc_fault_count(nodes, NULL, size * page_size);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->counters[i] == exp_alloc[i]) {
+			tst_res(TPASS, "%sNode %u allocated %u",
+			        prefix, nodes->map[i], exp_alloc[i]);
+		} else {
+			tst_res(TFAIL, "%sNode %u allocated %u, expected %u",
+			        prefix, nodes->map[i], nodes->counters[i],
+			        exp_alloc[i]);
+		}
+	}
+}
+
+static void verify_set_mempolicy(unsigned int n)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int exp_alloc[nodes->cnt];
+	unsigned int alloc_per_node = n ? 8 : 2;
+	unsigned int alloc_on_nodes = n ? 2 : nodes->cnt;
+	unsigned int alloc_total = alloc_per_node * alloc_on_nodes;
+	unsigned int i;
+
+	memset(exp_alloc, 0, sizeof(exp_alloc));
+
+	for (i = 0; i < alloc_on_nodes; i++) {
+		exp_alloc[i] = alloc_per_node;
+		numa_bitmask_setbit(bm, nodes->map[i]);
+	}
+
+	TEST(set_mempolicy(MPOL_INTERLEAVE, bm->maskp, bm->size+1));
+
+	tst_res(TINFO, "Allocating on nodes 1-%u - %u pages",
+	        alloc_on_nodes, alloc_total);
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "set_mempolicy(MPOL_INTERLEAVE)");
+		return;
+	}
+
+	tst_res(TPASS, "set_mempolicy(MPOL_INTERLEAVE)");
+
+	numa_free_nodemask(bm);
+
+	alloc_and_check(alloc_total, exp_alloc);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_set_mempolicy,
+	.tcnt = 2,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
new file mode 100644
index 000000000..a1e7f1eb5
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
@@ -0,0 +1,113 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing set_mempolicy() with MPOL_BIND and MPOL_PREFERRED backed by a
+ * file.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numaif.h>
+# include <numa.h>
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#define MNTPOINT "mntpoint"
+#define PAGES_ALLOCATED 16u
+
+#ifdef HAVE_NUMA_H
+
+#include "set_mempolicy.h"
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * PAGES_ALLOCATED * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void verify_mempolicy(unsigned int node, int mode)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int i;
+
+	numa_bitmask_setbit(bm, node);
+
+	TEST(set_mempolicy(mode, bm->maskp, bm->size+1));
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "set_mempolicy(%s) node %u",
+		        tst_numa_mode_name(mode), node);
+		return;
+	}
+
+	tst_res(TPASS, "set_mempolicy(%s) node %u",
+	        tst_numa_mode_name(mode), node);
+
+	numa_free_nodemask(bm);
+
+	tst_nodemap_reset_counters(nodes);
+	alloc_fault_count(nodes, MNTPOINT "/numa-test-file", PAGES_ALLOCATED * page_size);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->map[i] == node) {
+			if (nodes->counters[i] == PAGES_ALLOCATED) {
+				tst_res(TPASS, "Node %u allocated %u",
+				        node, PAGES_ALLOCATED);
+			} else {
+				tst_res(TFAIL, "Node %u allocated %u, expected %u",
+				        node, nodes->counters[i], PAGES_ALLOCATED);
+			}
+			continue;
+		}
+
+		if (nodes->counters[i]) {
+			tst_res(TFAIL, "Node %u allocated %u, expected 0",
+			        node, nodes->counters[i]);
+		}
+	}
+}
+
+static void verify_set_mempolicy(unsigned int n)
+{
+	unsigned int i;
+	int mode = n ? MPOL_PREFERRED : MPOL_BIND;
+
+	for (i = 0; i < nodes->cnt; i++)
+		verify_mempolicy(nodes->map[i], mode);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_set_mempolicy,
+	.tcnt = 2,
+	.needs_root = 1,
+	.all_filesystems = 1,
+	.mntpoint = MNTPOINT,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c
new file mode 100644
index 000000000..60601e7c1
--- /dev/null
+++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c
@@ -0,0 +1,142 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing set_mempolicy() with MPOL_INTERLEAVE on mmaped buffers backed
+ * by files.
+ *
+ * Apparently it takes a larger sample for the allocations to be correctly
+ * interleaved. The reason for this is that buffers for file metadata are
+ * allocated in batches in order not to loose performance. Also the pages
+ * cannot be interleaved completely evenly unless the number of pages is
+ * divideable by the number of nodes, which will not happen even if we tried
+ * hard since we do not have controll over metadata blocks for instance. Hence
+ * we cannot really expect to allocate a single file and have the memory
+ * interleaved precisely but it works well if we allocate statistic for a more
+ * than a few files.
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#define MNTPOINT "mntpoint"
+#define FILES 10
+
+#ifdef HAVE_NUMA_H
+
+#include "set_mempolicy.h"
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 20 * FILES * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void alloc_and_check(void)
+{
+	unsigned int i, j;
+	char path[1024];
+	unsigned int total_pages = 0;
+	unsigned int sum_pages = 0;
+
+	tst_nodemap_reset_counters(nodes);
+
+	/*
+	 * The inner loop loops node->cnt times to ensure the sum could
+	 * be evenly distributed among the nodes.
+	 */
+	for (i = 1; i <= FILES; i++) {
+		for (j = 1; j <= nodes->cnt; j++) {
+			size_t size = 10 * i + j % 10;
+			snprintf(path, sizeof(path), MNTPOINT "/numa-test-file-%i-%i", i, j);
+			alloc_fault_count(nodes, path, size * page_size);
+			total_pages += size;
+		}
+	}
+
+	for (i = 0; i < nodes->cnt; i++) {
+		float treshold = 1.00 * total_pages / 60; /* five percents */
+		float min_pages = 1.00 * total_pages / 3 - treshold;
+		float max_pages = 1.00 * total_pages / 3 + treshold;
+
+		if (nodes->counters[i] > min_pages && nodes->counters[i] < max_pages) {
+			tst_res(TPASS, "Node %u allocated %u <%.2f,%.2f>",
+			        nodes->map[i], nodes->counters[i], min_pages, max_pages);
+		} else {
+			tst_res(TFAIL, "Node %u allocated %u, expected <%.2f,%.2f>",
+			        nodes->map[i], nodes->counters[i], min_pages, max_pages);
+		}
+
+		sum_pages += nodes->counters[i];
+	}
+
+	if (sum_pages != total_pages) {
+		tst_res(TFAIL, "Sum of nodes %u != allocated pages %u",
+		        sum_pages, total_pages);
+		return;
+	}
+
+	tst_res(TPASS, "Sum of nodes equals to allocated pages (%u)", total_pages);
+}
+
+static void verify_set_mempolicy(void)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int alloc_on_nodes = nodes->cnt;
+	unsigned int i;
+
+	for (i = 0; i < alloc_on_nodes; i++)
+		numa_bitmask_setbit(bm, nodes->map[i]);
+
+	TEST(set_mempolicy(MPOL_INTERLEAVE, bm->maskp, bm->size+1));
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "set_mempolicy(MPOL_INTERLEAVE)");
+		return;
+	}
+
+	tst_res(TPASS, "set_mempolicy(MPOL_INTERLEAVE)");
+
+	alloc_and_check();
+
+	numa_free_nodemask(bm);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test_all = verify_set_mempolicy,
+	.forks_child = 1,
+	.needs_root = 1,
+	.all_filesystems = 1,
+	.mntpoint = MNTPOINT,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
-- 
2.19.2

[LTP] [PATCH 2/2] syscalls/mbind0{2,3,4}: Add basic mbind tests

[Cyril Hrubis]

This test is similar to the set_mempolicy() tests, but with mbind we are
actually binding memory to nodes after it was mapped.

mbind02: We are trying to get EIO with MPOL_MF_STRICT and page that was already
         faulted on wrong node.

mbind03: We are moving pages with MPOL_MF_MOVE and MPOL_MF_MOVE_ALL and
         checking the result.

mbind04: We are applying memory policy before we fault the pages and check
         that the pages were faulted accordingly.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
CC: Michal Hocko <mhocko@kernel.org>
CC: Vlastimil Babka <vbabka@suse.cz>
CC: Jan Stancek <jstancek@redhat.com>
---
 runtest/syscalls                           |   3 +
 testcases/kernel/syscalls/mbind/.gitignore |   3 +
 testcases/kernel/syscalls/mbind/mbind.h    |  26 +++++
 testcases/kernel/syscalls/mbind/mbind02.c  |  99 ++++++++++++++++
 testcases/kernel/syscalls/mbind/mbind03.c  | 123 ++++++++++++++++++++
 testcases/kernel/syscalls/mbind/mbind04.c  | 127 +++++++++++++++++++++
 6 files changed, 381 insertions(+)
 create mode 100644 testcases/kernel/syscalls/mbind/mbind.h
 create mode 100644 testcases/kernel/syscalls/mbind/mbind02.c
 create mode 100644 testcases/kernel/syscalls/mbind/mbind03.c
 create mode 100644 testcases/kernel/syscalls/mbind/mbind04.c

diff --git a/runtest/syscalls b/runtest/syscalls
index 0af0f4475..b4e9d053c 100644
--- a/runtest/syscalls
+++ b/runtest/syscalls
@@ -619,6 +619,9 @@ lstat03_64 lstat03_64
 mallopt01 mallopt01
 
 mbind01 mbind01
+mbind02 mbind02
+mbind03 mbind03
+mbind04 mbind04
 
 memset01 memset01
 memcmp01 memcmp01
diff --git a/testcases/kernel/syscalls/mbind/.gitignore b/testcases/kernel/syscalls/mbind/.gitignore
index 64139f75c..b78d20264 100644
--- a/testcases/kernel/syscalls/mbind/.gitignore
+++ b/testcases/kernel/syscalls/mbind/.gitignore
@@ -1 +1,4 @@
 /mbind01
+/mbind02
+/mbind03
+/mbind04
diff --git a/testcases/kernel/syscalls/mbind/mbind.h b/testcases/kernel/syscalls/mbind/mbind.h
new file mode 100644
index 000000000..62e749bca
--- /dev/null
+++ b/testcases/kernel/syscalls/mbind/mbind.h
@@ -0,0 +1,26 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#ifndef MBIND_H__
+#define MBIND_H__
+
+static inline const char *mbind_flag_name(unsigned flag)
+{
+	switch (flag) {
+	case 0:
+		return "0";
+	case MPOL_MF_STRICT:
+		return "MPOL_MF_STRICT";
+	case MPOL_MF_MOVE:
+		return "MPOL_MF_MOVE";
+	case MPOL_MF_MOVE_ALL:
+		return "MPOL_MF_MOVE_ALL";
+	default:
+		return "???";
+	}
+}
+
+#endif /* MBIND_H__ */
diff --git a/testcases/kernel/syscalls/mbind/mbind02.c b/testcases/kernel/syscalls/mbind/mbind02.c
new file mode 100644
index 000000000..2ffed4cff
--- /dev/null
+++ b/testcases/kernel/syscalls/mbind/mbind02.c
@@ -0,0 +1,99 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing mbind() EIO error.
+ *
+ * We first fault a allocated page, then attempt to mbind it to a different node.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#ifdef HAVE_NUMA_H
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void verify_policy(int mode)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int i;
+	void *ptr;
+	unsigned long size = page_size;
+	int node = 0;
+
+	ptr = tst_numa_map(NULL, size);
+	tst_nodemap_reset_counters(nodes);
+	tst_numa_fault(ptr, size);
+	tst_nodemap_count_pages(nodes, ptr, size);
+	tst_nodemap_print_counters(nodes);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (!nodes->counters[i]) {
+			node = nodes->map[i];
+			tst_res(TINFO, "Attempting to bind to node %i", node);
+			numa_bitmask_setbit(bm, node);
+			break;
+		}
+	}
+
+	TEST(mbind(ptr, size, mode, bm->maskp, bm->size + 1, MPOL_MF_STRICT));
+
+	if (TST_RET) {
+		tst_res(TPASS | TTERRNO,
+		        "mbind(%s, MPOL_MF_STRICT) node %u",
+		        tst_numa_mode_name(mode), node);
+	} else {
+		tst_res(TFAIL, "mbind(%s, MPOL_MF_STRICT) node %u succeded",
+		        tst_numa_mode_name(mode), node);
+	}
+
+	tst_numa_unmap(ptr, size);
+	numa_free_nodemask(bm);
+}
+
+static void verify_mbind(unsigned int n)
+{
+	int mode = n ? MPOL_PREFERRED : MPOL_BIND;
+
+	verify_policy(mode);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_mbind,
+	.tcnt = 2,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
diff --git a/testcases/kernel/syscalls/mbind/mbind03.c b/testcases/kernel/syscalls/mbind/mbind03.c
new file mode 100644
index 000000000..79ab4a62d
--- /dev/null
+++ b/testcases/kernel/syscalls/mbind/mbind03.c
@@ -0,0 +1,123 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing mbind() EIO error.
+ *
+ * We first fault a allocated page, then attempt to mbind it to a different node.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+# include "mbind.h"
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#ifdef HAVE_NUMA_H
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void verify_policy(int mode, unsigned flag)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int i;
+	void *ptr;
+	unsigned long size = page_size;
+	unsigned int node = 0;
+
+	ptr = tst_numa_map(NULL, size);
+	tst_nodemap_reset_counters(nodes);
+	tst_numa_fault(ptr, size);
+	tst_nodemap_count_pages(nodes, ptr, size);
+	tst_nodemap_print_counters(nodes);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (!nodes->counters[i]) {
+			node = nodes->map[i];
+			tst_res(TINFO, "Attempting to move to node %i", node);
+			numa_bitmask_setbit(bm, node);
+			break;
+		}
+	}
+
+	TEST(mbind(ptr, size, mode, bm->maskp, bm->size + 1, flag));
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "mbind(%s, %s) node %u",
+		        tst_numa_mode_name(mode), mbind_flag_name(flag), node);
+		goto exit;
+	} else {
+		tst_res(TPASS, "mbind(%s, %s) node %u succeded",
+		        tst_numa_mode_name(mode), mbind_flag_name(flag), node);
+	}
+
+	tst_nodemap_reset_counters(nodes);
+	tst_nodemap_count_pages(nodes, ptr, size);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->map[i] == node) {
+			if (nodes->counters[i] == 1) {
+				tst_res(TPASS, "Node %u allocated %u", node, 1);
+			} else {
+				tst_res(TFAIL, "Node %u allocated %u, expected %u",
+				        node, nodes->counters[i], 0);
+			}
+			continue;
+		}
+
+		if (nodes->counters[i]) {
+			tst_res(TFAIL, "Node %u allocated %u, expected 0",
+			        i, nodes->counters[i]);
+		}
+	}
+
+exit:
+	tst_numa_unmap(ptr, size);
+	numa_free_nodemask(bm);
+}
+
+void verify_mbind(unsigned int n)
+{
+	int mode = n ? MPOL_PREFERRED : MPOL_BIND;
+
+	verify_policy(mode, MPOL_MF_MOVE);
+	verify_policy(mode, MPOL_MF_MOVE_ALL);
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_mbind,
+	.tcnt = 2,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
diff --git a/testcases/kernel/syscalls/mbind/mbind04.c b/testcases/kernel/syscalls/mbind/mbind04.c
new file mode 100644
index 000000000..f2097c878
--- /dev/null
+++ b/testcases/kernel/syscalls/mbind/mbind04.c
@@ -0,0 +1,127 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+/*
+ * We are testing mbind() with MPOL_BIND and MPOL_PREFERRED.
+ *
+ * For each node with memory we set its bit in nodemask with set_mempolicy()
+ * and verify that memory has been faulted accordingly.
+ */
+
+#include <errno.h>
+#include "config.h"
+#ifdef HAVE_NUMA_H
+# include <numa.h>
+# include <numaif.h>
+# include "mbind.h"
+#endif
+#include "tst_test.h"
+#include "tst_numa.h"
+
+#ifdef HAVE_NUMA_H
+
+static size_t page_size;
+static struct tst_nodemap *nodes;
+
+#define PAGES_ALLOCATED 16u
+
+static void setup(void)
+{
+	page_size = getpagesize();
+
+	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * PAGES_ALLOCATED * page_size / 1024);
+	if (nodes->cnt <= 1)
+		tst_brk(TCONF, "Test requires@least two NUMA memory nodes");
+}
+
+static void cleanup(void)
+{
+	tst_nodemap_free(nodes);
+}
+
+static void verify_policy(unsigned int node, int mode, unsigned flag)
+{
+	struct bitmask *bm = numa_allocate_nodemask();
+	unsigned int i;
+	void *ptr;
+	unsigned long size = PAGES_ALLOCATED * page_size;
+
+	numa_bitmask_setbit(bm, node);
+
+	ptr = tst_numa_map(NULL, size);
+
+	TEST(mbind(ptr, size, mode, bm->maskp, bm->size + 1, flag));
+
+	if (TST_RET) {
+		tst_res(TFAIL | TTERRNO,
+		        "mbind(%s, %s) node %u",
+		        tst_numa_mode_name(mode), mbind_flag_name(flag), node);
+		return;
+	}
+
+	tst_res(TPASS, "mbind(%s, %s) node %u",
+	        tst_numa_mode_name(mode), mbind_flag_name(flag), node);
+
+	numa_free_nodemask(bm);
+
+	const char *prefix = "child: ";
+
+	if (SAFE_FORK()) {
+		prefix = "parent: ";
+		tst_reap_children();
+	}
+
+	tst_nodemap_reset_counters(nodes);
+	tst_numa_fault(ptr, size);
+	tst_nodemap_count_pages(nodes, ptr, size);
+	tst_numa_unmap(ptr, size);
+	tst_nodemap_print_counters(nodes);
+
+	for (i = 0; i < nodes->cnt; i++) {
+		if (nodes->map[i] == node) {
+			if (nodes->counters[i] == PAGES_ALLOCATED) {
+				tst_res(TPASS, "%sNode %u allocated %u",
+				        prefix, node, PAGES_ALLOCATED);
+			} else {
+				tst_res(TFAIL, "%sNode %u allocated %u, expected %u",
+				        prefix, node, nodes->counters[i],
+				        PAGES_ALLOCATED);
+			}
+			continue;
+		}
+
+		if (nodes->counters[i]) {
+			tst_res(TFAIL, "%sNode %u allocated %u, expected 0",
+			        prefix, i, nodes->counters[i]);
+		}
+	}
+}
+
+static void verify_mbind(unsigned int n)
+{
+	unsigned int i;
+	int mode = n ? MPOL_PREFERRED : MPOL_BIND;
+
+	for (i = 0; i < nodes->cnt; i++) {
+		verify_policy(nodes->map[i], mode, 0);
+		verify_policy(nodes->map[i], mode, MPOL_MF_STRICT);
+	}
+}
+
+static struct tst_test test = {
+	.setup = setup,
+	.cleanup = cleanup,
+	.test = verify_mbind,
+	.tcnt = 2,
+	.forks_child = 1,
+	.needs_checkpoints = 1,
+};
+
+#else
+
+TST_TEST_TCONF(NUMA_ERROR_MSG);
+
+#endif /* HAVE_NUMA_H */
-- 
2.19.2

[LTP] [PATCH 1/2 v4] syscalls: Add set_mempolicy numa tests.

[Jan Stancek]

----- Original Message -----
> This is initial attempt to replace numa.sh tests that despite having
> been fixed several times have still many shortcommings that wouldn't
> easy to fix. It's not finished nor 100% replacement at this point but it
> should be pretty good start.
> 
> The main selling points of these testcases are:
> 
> The memory allocated for the testing is tracked exactly. We are using
> get_mempolicy() with MPOL_F_NODE | MPOL_F_ADDR that returns the node ID on
> which specified address is allocated on to count pages allocated per node
> after
> we set desired memory policy.

Hi,

> 
> We also check for free memory on each numa memory mode and skip nodes
> that don't have sufficient amount of memory for a particular test. The
> tests checks usuall for twice as much memory per each node in order to
> allow for allocations to be "misplaced".

I wonder if we should add some constant to it. I have foggy idea, that
kernel keeps some reserves.

> 
> The tests for file based shared interleaved mappings are no longer
> mapping a single small file but rather than that we accumulate statistic
> for larger amount of files over longer period of time and we also allow
> for small offset (currently 10%). We should probably also increase the
> number of samples we take as currently it's about 5MB in total on x86
> although I haven't managed to make this test fail so far. This also
> fixes the test on Btrfs where the synthetic test that expects the pages
> to be distributed exactly equally fails.

API looks good to me, ACK. Maybe you could split it into 2 commits, one for
new library API, and one for new tests.

Couple small comments below.

> +
> +void tst_nodemap_count_pages(struct tst_nodemap *nodes,
> +                             void *ptr, size_t size)
> +{
> +	size_t page_size = getpagesize();
> +	unsigned int i;
> +	int node;
> +	unsigned int pages = (size + page_size - 1)/page_size;
> +
> +	for (i = 0; i < pages; i++) {
> +		get_mempolicy(&node, NULL, 0, ptr + i * page_size, MPOL_F_NODE |
> MPOL_F_ADDR);

We should check return value from get_mempolicy here.

> +
> +		if (node < 0 || (unsigned int)node >= nodes->cnt) {
> +			tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node);
> +			continue;
> +		}
> +
> +		inc_counter(node, nodes);
> +	}
> +}
> +

<snip>

> +
> +static void verify_set_mempolicy(unsigned int n)
> +{
> +	struct bitmask *bm = numa_allocate_nodemask();
> +	unsigned int exp_alloc[nodes->cnt];
> +	unsigned int alloc_per_node = n ? 8 : 2;

Can you make this "8" a define, so that "16" above can be derived from it?

> +	unsigned int alloc_on_nodes = n ? 2 : nodes->cnt;
> +	unsigned int alloc_total = alloc_per_node * alloc_on_nodes;
> +	unsigned int i;
> +
> +	memset(exp_alloc, 0, sizeof(exp_alloc));
> +
> +	for (i = 0; i < alloc_on_nodes; i++) {
> +		exp_alloc[i] = alloc_per_node;
> +		numa_bitmask_setbit(bm, nodes->map[i]);
> +	}
> +
> +	TEST(set_mempolicy(MPOL_INTERLEAVE, bm->maskp, bm->size+1));
> +
> +	tst_res(TINFO, "Allocating on nodes 1-%u - %u pages",
> +	        alloc_on_nodes, alloc_total);
> +
> +	if (TST_RET) {
> +		tst_res(TFAIL | TTERRNO,
> +		        "set_mempolicy(MPOL_INTERLEAVE)");
> +		return;
> +	}
> +
> +	tst_res(TPASS, "set_mempolicy(MPOL_INTERLEAVE)");
> +
> +	numa_free_nodemask(bm);
> +
> +	alloc_and_check(alloc_total, exp_alloc);
> +}
> +
> +static struct tst_test test = {
> +	.setup = setup,
> +	.cleanup = cleanup,
> +	.test = verify_set_mempolicy,
> +	.tcnt = 2,
> +	.forks_child = 1,
> +	.needs_checkpoints = 1,
> +};
> +
> +#else
> +
> +TST_TEST_TCONF(NUMA_ERROR_MSG);
> +
> +#endif /* HAVE_NUMA_H */
> diff --git a/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
> b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
> new file mode 100644
> index 000000000..a1e7f1eb5
> --- /dev/null
> +++ b/testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c
> @@ -0,0 +1,113 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + *
> + * Copyright (c) 2018 Cyril Hrubis <chrubis@suse.cz>
> + */
> +
> +/*
> + * We are testing set_mempolicy() with MPOL_BIND and MPOL_PREFERRED backed
> by a
> + * file.

Maybe extend this comment to say it's across all filesystems.

Regards,
Jan

[LTP] [PATCH 1/2 v4] syscalls: Add set_mempolicy numa tests.

[Cyril Hrubis]

Hi!
> > We also check for free memory on each numa memory mode and skip nodes
> > that don't have sufficient amount of memory for a particular test. The
> > tests checks usuall for twice as much memory per each node in order to
> > allow for allocations to be "misplaced".
> 
> I wonder if we should add some constant to it. I have foggy idea, that
> kernel keeps some reserves.

Hmm, no idea about this, I guess that adding a few megabytes for a good
luck in the library wouldn't do any harm.

> > The tests for file based shared interleaved mappings are no longer
> > mapping a single small file but rather than that we accumulate statistic
> > for larger amount of files over longer period of time and we also allow
> > for small offset (currently 10%). We should probably also increase the
> > number of samples we take as currently it's about 5MB in total on x86
> > although I haven't managed to make this test fail so far. This also
> > fixes the test on Btrfs where the synthetic test that expects the pages
> > to be distributed exactly equally fails.
> 
> API looks good to me, ACK. Maybe you could split it into 2 commits, one for
> new library API, and one for new tests.

Sure, will do.

Ok to commit the library and set_mempolicy() tests with fixed nits you
pointed out below or should I spin v5?

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 1/2 v4] syscalls: Add set_mempolicy numa tests.

[Jan Stancek]

----- Original Message -----
> Hi!
> > > We also check for free memory on each numa memory mode and skip nodes
> > > that don't have sufficient amount of memory for a particular test. The
> > > tests checks usuall for twice as much memory per each node in order to
> > > allow for allocations to be "misplaced".
> > 
> > I wonder if we should add some constant to it. I have foggy idea, that
> > kernel keeps some reserves.
> 
> Hmm, no idea about this, I guess that adding a few megabytes for a good
> luck in the library wouldn't do any harm.
> 
> > > The tests for file based shared interleaved mappings are no longer
> > > mapping a single small file but rather than that we accumulate statistic
> > > for larger amount of files over longer period of time and we also allow
> > > for small offset (currently 10%). We should probably also increase the
> > > number of samples we take as currently it's about 5MB in total on x86
> > > although I haven't managed to make this test fail so far. This also
> > > fixes the test on Btrfs where the synthetic test that expects the pages
> > > to be distributed exactly equally fails.
> > 
> > API looks good to me, ACK. Maybe you could split it into 2 commits, one for
> > new library API, and one for new tests.
> 
> Sure, will do.
> 
> Ok to commit the library and set_mempolicy() tests with fixed nits you
> pointed out below or should I spin v5?

For me, OK to commit, this set has been on mailing list for couple months.

> 
> --
> Cyril Hrubis
> chrubis@suse.cz
> 

[LTP] [PATCH 2/2] syscalls/mbind0{2, 3, 4}: Add basic mbind tests

[Petr Vorel]

Hi Cyril,

> This test is similar to the set_mempolicy() tests, but with mbind we are
> actually binding memory to nodes after it was mapped.

> mbind02: We are trying to get EIO with MPOL_MF_STRICT and page that was already
>          faulted on wrong node.

> mbind03: We are moving pages with MPOL_MF_MOVE and MPOL_MF_MOVE_ALL and
>          checking the result.

> mbind04: We are applying memory policy before we fault the pages and check
>          that the pages were faulted accordingly.

Generally LGTM, just Makefile needs updating after adding libltpnuma:
LDFLAGS += -L$(top_builddir)/libs/libltpnuma
LDLIBS  += $(NUMA_LIBS) -lltpnuma
(mbind01 does not need that, I'd ignore that).

...
> +++ b/testcases/kernel/syscalls/mbind/mbind.h
...
> +static inline const char *mbind_flag_name(unsigned flag)
> +{
> +	switch (flag) {
> +	case 0:
> +		return "0";
> +	case MPOL_MF_STRICT:
> +		return "MPOL_MF_STRICT";
> +	case MPOL_MF_MOVE:
> +		return "MPOL_MF_MOVE";
> +	case MPOL_MF_MOVE_ALL:
> +		return "MPOL_MF_MOVE_ALL";
> +	default:
> +		return "???";
I guess "???" is important for us. Otherwise I'd use macro:
#define MBIND_FLAG_NAME(x) #x

..
> +++ b/testcases/kernel/syscalls/mbind/mbind02.c
...
> +static void setup(void)
> +{
> +	page_size = getpagesize();
> +
> +	nodes = tst_get_nodemap(TST_NUMA_MEM, 2 * page_size / 1024);
> +	if (nodes->cnt <= 1)
> +		tst_brk(TCONF, "Test requires at least two NUMA memory nodes");
> +}
> +
> +static void cleanup(void)
> +{
> +	tst_nodemap_free(nodes);
> +}
Maybe setup and cleanup can be put as static inline into mbind.h? As it's
repeated. PAGES_ALLOCATED would have to be defined for all.

...
> +++ b/testcases/kernel/syscalls/mbind/mbind03.c
> @@ -0,0 +1,123 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + *
> + * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
> + */
> +
> +/*
> + * We are testing mbind() EIO error.
> + *
> + * We first fault a allocated page, then attempt to mbind it to a different node.
Here is a copy paste error. But commit message is correct:
    mbind03: We are moving pages with MPOL_MF_MOVE and MPOL_MF_MOVE_ALL and
             checking the result.

Kind regards,
Petr

[LTP] [PATCH 2/2] syscalls/mbind0{2, 3, 4}: Add basic mbind tests

[Petr Vorel]

HI Cyril,

> +#include <errno.h>
> +#include "config.h"
> +#ifdef HAVE_NUMA_H
> +# include <numa.h>
> +# include <numaif.h>
> +#endif
> +#include "tst_test.h"
> +#include "tst_numa.h"
> +
> +#ifdef HAVE_NUMA_H
This needs to be tested against HAVE_NUMA_V2 to get proper libnuma detection in
cross-compilation [1] [2]:

/tmp/cclVNEz5.o: In function `verify_policy':
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:46: undefined reference to `numa_allocate_nodemask'
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:51: undefined reference to `numa_bitmask_setbit'
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:53: undefined reference to `tst_numa_map'
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:55: undefined reference to `mbind'
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:58: undefined reference to `tst_numa_mode_name'
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:64: undefined reference to `tst_numa_mode_name'
/tmp/cclVNEz5.o: In function `numa_free_nodemask':
/usr/include/numa.h:183: undefined reference to `numa_bitmask_free'
/tmp/cclVNEz5.o: In function `verify_policy':
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:78: undefined reference to `tst_nodemap_count_pages'
/tmp/cclVNEz5.o: In function `setup':
/usr/src/ltp/testcases/kernel/syscalls/mbind/mbind04.c:34: undefined reference to `tst_get_nodemap'
collect2: error: ld returned 1 exit status

Id' personally move numa includes after it.

Kind regards,
Petr

[1] https://api.travis-ci.org/v3/job/540754157/log.txt
[2] https://travis-ci.org/pevik/ltp/builds/540754154