[LTP] [PATCH v3 0/2] Implement syscall variants

[LTP] [PATCH v3 0/2] Implement syscall variants

[Cyril Hrubis]

This is version 3 of the patchset, the main change is that the
previously called multiplexing is now hooked up much sooner in the
library, before we fork and run test setup().

The advange of this is that tests that exit with tst_brk(TCONF, ...) do
not exit the whole testsuite this way and also different test variants
are better isolated as well.

As a side effect the API has been simplified as well.

Cyril Hrubis (2):
  tst_test: Add test multiplex function
  syscalls/select04: Test four syscall variants

 doc/test-writing-guidelines.txt               | 66 +++++++++++++++
 include/tst_test.h                            | 12 +++
 lib/newlib_tests/.gitignore                   |  1 +
 lib/newlib_tests/variant.c                    | 55 +++++++++++++
 lib/tst_test.c                                | 22 +++--
 testcases/kernel/syscalls/select/select04.c   |  7 +-
 testcases/kernel/syscalls/select/select_var.h | 82 +++++++++++++++++++
 7 files changed, 239 insertions(+), 6 deletions(-)
 create mode 100644 lib/newlib_tests/variant.c
 create mode 100644 testcases/kernel/syscalls/select/select_var.h

-- 
2.19.2

[LTP] [PATCH v3 1/2] tst_test: Add test multiplex function

[Cyril Hrubis]

The test multiplex function is intended for running the test function in
a loop for a different settings. The indended purpose is to run tests
for both libc wrapper and raw syscall as well as for different syscall
variants.

The commit itself adds a test_multiplex() function into the tst_test
structure, if set this function is called in a loop before each test
iteration and is responsible for changing the test variant into next
one. The iterations continue until the function returns zero.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Steve Muckle <smuckle@google.com>
CC: Mark Salyzyn <salyzyn@android.com>
CC: Steve Muckle <smuckle@google.com>
CC: Jan Stancek <jstancek@redhat.com>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 doc/test-writing-guidelines.txt | 66 +++++++++++++++++++++++++++++++++
 include/tst_test.h              | 12 ++++++
 lib/newlib_tests/.gitignore     |  1 +
 lib/newlib_tests/variant.c      | 55 +++++++++++++++++++++++++++
 lib/tst_test.c                  | 22 ++++++++---
 5 files changed, 151 insertions(+), 5 deletions(-)
 create mode 100644 lib/newlib_tests/variant.c

diff --git a/doc/test-writing-guidelines.txt b/doc/test-writing-guidelines.txt
index c36b2a424..0f6dc8f12 100644
--- a/doc/test-writing-guidelines.txt
+++ b/doc/test-writing-guidelines.txt
@@ -1603,6 +1603,72 @@ sturct tst_test test = {
 };
 -------------------------------------------------------------------------------
 
+2.2.29 Testing similar syscalls in one test
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In some cases kernel has several very similar syscalls that do either the same
+or very similar job. This is most noticeable on i386 where we commonly have
+two or three syscall versions. That is because i386 was first platform that
+Linux was developed on and because of that most mistakes in API happened there
+as well. However this is not limited to i386 at all, it's quite common that
+version two syscall has added missing flags parameters or so.
+
+In such cases it does not make much sense to copy&paste the test code over and
+over, rather than that the test library provides support for test variants.
+The idea behind test variants is simple, we run the test several times each
+time with different syscall variant.
+
+The implemenation consist of test_variant integer that, if set, denotes number
+of test variants. The test is then forked and executed test_variant times each
+time with different value in global tst_variant variable.
+
+[source,c]
+-------------------------------------------------------------------------------
+#include "tst_test.h"
+
+static int do_foo(void)
+{
+	switch (tst_variant) {
+	case 0:
+		return foo();
+	case 1:
+		return syscall(__NR_foo);
+	}
+
+	return -1;
+}
+
+static void run(void)
+{
+	...
+
+	TEST(do_foo);
+
+	...
+}
+
+static void setup(void)
+{
+	switch (tst_variant) {
+	case 0:
+		tst_res(TINFO, "Testing foo variant 1");
+	break;
+	case 1:
+		tst_res(TINFO, "Testing foo variant 2");
+	break;
+	}
+}
+
+sturct tst_test test = {
+	...
+	.setup = setup,
+	.test_all = run,
+	.test_variants = 2,
+	...
+};
+-------------------------------------------------------------------------------
+
+
 2.3 Writing a testcase in shell
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/include/tst_test.h b/include/tst_test.h
index da41f776d..cf2447fe3 100644
--- a/include/tst_test.h
+++ b/include/tst_test.h
@@ -114,6 +114,8 @@ int tst_parse_int(const char *str, int *val, int min, int max);
 int tst_parse_long(const char *str, long *val, long min, long max);
 int tst_parse_float(const char *str, float *val, float min, float max);
 
+extern unsigned int tst_variant;
+
 struct tst_test {
 	/* number of tests available in test() function */
 	unsigned int tcnt;
@@ -146,6 +148,16 @@ struct tst_test {
 	 */
 	int all_filesystems:1;
 
+	/*
+	 * If set non-zero denotes number of test variant, the test is executed
+	 * variants times each time with tst_variant set to different number.
+	 *
+	 * This allows us to run the same test for different settings. The
+	 * intended use is to test different syscall wrappers/variants but the
+	 * API is generic and does not limit the usage in any way.
+	 */
+	unsigned int test_variants;
+
 	/* Minimal device size in megabytes */
 	unsigned int dev_min_size;
 
diff --git a/lib/newlib_tests/.gitignore b/lib/newlib_tests/.gitignore
index 645ba349e..d92b89872 100644
--- a/lib/newlib_tests/.gitignore
+++ b/lib/newlib_tests/.gitignore
@@ -25,3 +25,4 @@ tst_expiration_timer
 test_exec
 test_exec_child
 test_kconfig
+variant
diff --git a/lib/newlib_tests/variant.c b/lib/newlib_tests/variant.c
new file mode 100644
index 000000000..b5816fff0
--- /dev/null
+++ b/lib/newlib_tests/variant.c
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#include "tst_test.h"
+
+static void do_test(void)
+{
+	switch (tst_variant) {
+	case 0:
+		/* This is skipped after first iteration */
+		tst_brk(TCONF, "Test skipped");
+	break;
+	case 1:
+		/* This test is correctly looped with -i opt */
+		tst_res(TPASS, "Test passed");
+	break;
+	case 2:
+		/* This exits the test immediatelly */
+		tst_brk(TBROK, "Test broken");
+	break;
+	}
+
+	tst_res(TINFO, "test() function exitting normaly");
+}
+
+static void setup(void)
+{
+	tst_res(TINFO, "Running test setup()");
+
+	switch (tst_variant) {
+	case 0:
+		tst_res(TINFO, "Starting tst_brk TCONF test");
+	break;
+	case 1:
+		tst_res(TINFO, "Starting tst_res TPASS test");
+	break;
+	case 2:
+		tst_res(TINFO, "Starting tst_res TBROK test");
+	break;
+	}
+}
+
+static void cleanup(void)
+{
+	tst_res(TINFO, "Running test cleanup()");
+}
+
+static struct tst_test test = {
+	.test_all = do_test,
+	.test_variants = 3,
+	.setup = setup,
+	.cleanup = cleanup,
+};
diff --git a/lib/tst_test.c b/lib/tst_test.c
index ba5eab011..2d88adbd7 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -1180,9 +1180,12 @@ static int run_tcases_per_fs(void)
 	return ret;
 }
 
+unsigned int tst_variant;
+
 void tst_run_tcases(int argc, char *argv[], struct tst_test *self)
 {
-	int ret;
+	int ret = 0;
+	unsigned int test_variants = 1;
 
 	lib_pid = getpid();
 	tst_test = self;
@@ -1194,11 +1197,20 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)
 	SAFE_SIGNAL(SIGALRM, alarm_handler);
 	SAFE_SIGNAL(SIGUSR1, heartbeat_handler);
 
-	if (tst_test->all_filesystems)
-		ret = run_tcases_per_fs();
-	else
-		ret = fork_testrun();
+	if (tst_test->test_variants)
+		test_variants = tst_test->test_variants;
+
+	for (tst_variant = 0; tst_variant < test_variants; tst_variant++) {
+		if (tst_test->all_filesystems)
+			ret |= run_tcases_per_fs();
+		else
+			ret |= fork_testrun();
+
+		if (ret & ~(TCONF))
+			goto exit;
+	}
 
+exit:
 	do_exit(ret);
 }
 
-- 
2.19.2

[LTP] [PATCH v3 2/2] syscalls/select04: Test four syscall variants

[Cyril Hrubis]

This commit makes use of the newly added test_multiplex() function to
switch between different syscall variants/wrappers at runtime.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Mark Salyzyn <salyzyn@android.com>
CC: Steve Muckle <smuckle@google.com>
CC: Jan Stancek <jstancek@redhat.com>
---
 testcases/kernel/syscalls/select/select04.c   |  7 +-
 testcases/kernel/syscalls/select/select_var.h | 82 +++++++++++++++++++
 2 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 testcases/kernel/syscalls/select/select_var.h

diff --git a/testcases/kernel/syscalls/select/select04.c b/testcases/kernel/syscalls/select/select04.c
index 86bdffcdf..64aa4ac9b 100644
--- a/testcases/kernel/syscalls/select/select04.c
+++ b/testcases/kernel/syscalls/select/select04.c
@@ -27,6 +27,8 @@
 
 #include "tst_timer_test.h"
 
+#include "select_var.h"
+
 static int fds[2];
 
 static int sample_fn(int clk_id, long long usec)
@@ -39,7 +41,7 @@ static int sample_fn(int clk_id, long long usec)
 	FD_SET(fds[0], &sfds);
 
 	tst_timer_start(clk_id);
-	TEST(select(1, &sfds, NULL, NULL, &timeout));
+	TEST(do_select(1, &sfds, NULL, NULL, &timeout));
 	tst_timer_stop();
 	tst_timer_sample();
 
@@ -53,6 +55,8 @@ static int sample_fn(int clk_id, long long usec)
 
 static void setup(void)
 {
+	select_info();
+
 	SAFE_PIPE(fds);
 }
 
@@ -69,5 +73,6 @@ static struct tst_test test = {
 	.scall = "select()",
 	.sample = sample_fn,
 	.setup = setup,
+	.test_variants = TEST_VARIANTS,
 	.cleanup = cleanup,
 };
diff --git a/testcases/kernel/syscalls/select/select_var.h b/testcases/kernel/syscalls/select/select_var.h
new file mode 100644
index 000000000..d0bc5550e
--- /dev/null
+++ b/testcases/kernel/syscalls/select/select_var.h
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2019 Cyril Hrubis <chrubis@suse.cz>
+ */
+
+#ifndef SELECT_MPX
+#define SELECT_MPX
+
+#include "lapi/syscalls.h"
+
+struct compat_sel_arg_struct {
+	long _n;
+	long _inp;
+	long _outp;
+	long _exp;
+	long _tvp;
+};
+
+static int do_select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout)
+{
+	switch (tst_variant) {
+	case 0:
+		return select(nfds, readfds, writefds, exceptfds, timeout);
+	break;
+	case 1: {
+#ifdef __LP64__
+		return tst_syscall(__NR_select, nfds, readfds, writefds, exceptfds, timeout);
+#else
+		struct compat_sel_arg_struct arg = {
+			._n = (long)nfds,
+			._inp = (long)readfds,
+			._outp = (long)writefds,
+			._exp = (long)exceptfds,
+			._tvp = (long)timeout,
+		};
+
+		return tst_syscall(__NR_select, &arg);
+#endif /* __LP64__ */
+	}
+	case 2: {
+		int ret;
+		struct timespec ts = {
+			.tv_sec = timeout->tv_sec,
+			.tv_nsec = timeout->tv_usec * 1000,
+		};
+		ret = tst_syscall(__NR_pselect6, nfds, readfds, writefds, exceptfds, &ts, NULL);
+		timeout->tv_sec = ts.tv_sec;
+		timeout->tv_usec = ts.tv_nsec / 1000;
+		return ret;
+	}
+	case 3:
+#ifdef __NR__newselect
+		return tst_syscall(__NR__newselect, nfds, readfds, writefds, exceptfds, timeout);
+#else
+		tst_brk(TCONF, "__NR__newselect not implemented");
+#endif
+	break;
+	}
+
+	return -1;
+}
+
+static void select_info(void)
+{
+	switch (tst_variant) {
+	case 0:
+		tst_res(TINFO, "Testing glibc select()");
+	break;
+	case 1:
+		tst_res(TINFO, "Testing SYS_select syscall");
+	break;
+	case 2:
+		tst_res(TINFO, "Testing SYS_pselect6 syscall");
+	break;
+	case 3:
+		tst_res(TINFO, "Testing SYS__newselect syscall");
+	break;
+	}
+}
+
+#define TEST_VARIANTS 4
+
+#endif /* SELECT_MPX */
-- 
2.19.2

[LTP] [PATCH v3 0/2] Implement syscall variants

[Jan Stancek]

----- Original Message -----
> This is version 3 of the patchset, the main change is that the
> previously called multiplexing is now hooked up much sooner in the
> library, before we fork and run test setup().
> 
> The advange of this is that tests that exit with tst_brk(TCONF, ...) do
> not exit the whole testsuite this way and also different test variants
> are better isolated as well.
> 
> As a side effect the API has been simplified as well.

Acked-by: Jan Stancek <jstancek@redhat.com>

> 
> Cyril Hrubis (2):
>   tst_test: Add test multiplex function
>   syscalls/select04: Test four syscall variants
> 
>  doc/test-writing-guidelines.txt               | 66 +++++++++++++++
>  include/tst_test.h                            | 12 +++
>  lib/newlib_tests/.gitignore                   |  1 +
>  lib/newlib_tests/variant.c                    | 55 +++++++++++++
>  lib/tst_test.c                                | 22 +++--
>  testcases/kernel/syscalls/select/select04.c   |  7 +-
>  testcases/kernel/syscalls/select/select_var.h | 82 +++++++++++++++++++
>  7 files changed, 239 insertions(+), 6 deletions(-)
>  create mode 100644 lib/newlib_tests/variant.c
>  create mode 100644 testcases/kernel/syscalls/select/select_var.h
> 
> --
> 2.19.2
> 
> 
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
> 

[LTP] [PATCH v3 1/2] tst_test: Add test multiplex function

[Petr Vorel]

Hi Cyril,

Nice! Whole patchset LGTM, one typo below.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

> +2.2.29 Testing similar syscalls in one test
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +In some cases kernel has several very similar syscalls that do either the same
> +or very similar job. This is most noticeable on i386 where we commonly have
> +two or three syscall versions. That is because i386 was first platform that
> +Linux was developed on and because of that most mistakes in API happened there
> +as well. However this is not limited to i386 at all, it's quite common that
> +version two syscall has added missing flags parameters or so.
> +
> +In such cases it does not make much sense to copy&paste the test code over and
> +over, rather than that the test library provides support for test variants.
> +The idea behind test variants is simple, we run the test several times each
> +time with different syscall variant.
> +
> +The implemenation consist of test_variant integer that, if set, denotes number
Typo => implementation


Kind regards,
Petr

[LTP] [PATCH v3 2/2] syscalls/select04: Test four syscall variants

[Petr Vorel]

Hi Cyril,

>  testcases/kernel/syscalls/select/select04.c   |  7 +-
...
> +static void select_info(void)
> +{
> +	switch (tst_variant) {
> +	case 0:
> +		tst_res(TINFO, "Testing glibc select()");
nit: I'd use 'libc' instead of 'glibc'.

> +	break;
> +	case 1:
> +		tst_res(TINFO, "Testing SYS_select syscall");
> +	break;
> +	case 2:
> +		tst_res(TINFO, "Testing SYS_pselect6 syscall");
> +	break;
> +	case 3:
> +		tst_res(TINFO, "Testing SYS__newselect syscall");
> +	break;

Kind regards,
Petr

[LTP] [PATCH v3 0/2] Implement syscall variants

[Cyril Hrubis]

Hi!
I've updated the patch descriptions so that they match the changes in v3
patchset and pushed.

-- 
Cyril Hrubis
chrubis@suse.cz