io-uring.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 1/5] test: save dmesg output for each test and test file
@ 2020-09-18 10:47 Lukas Czerner
  2020-09-18 10:47 ` [PATCH 2/5] test: make a distinction between successful and skipped test Lukas Czerner
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Lukas Czerner @ 2020-09-18 10:47 UTC (permalink / raw)
  To: io-uring

Currently the dmesg output for each test will overwritten for every
test file so in the end only the dmesg output of the last test run will
be stored.

Fix it by using the test file name as well as test name in the dmesg log
file.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 test/runtests.sh | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/test/runtests.sh b/test/runtests.sh
index fa240f2..5107a0a 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -28,13 +28,18 @@ fi
 _check_dmesg()
 {
 	local dmesg_marker="$1"
-	local seqres="$2.seqres"
+	if [ -n "$3" ]; then
+		local dmesg_log=$(echo "${2}_${3}.dmesg" | \
+				  sed 's/\(\/\|_\/\|\/_\)/_/g')
+	else
+		local dmesg_log="${2}.dmesg"
+	fi
 
 	if [ $DO_KMSG -eq 0 ]; then
 		return 0
 	fi
 
-	dmesg | bash -c "$DMESG_FILTER" | grep -A 9999 "$dmesg_marker" >"${seqres}.dmesg"
+	dmesg | bash -c "$DMESG_FILTER" | grep -A 9999 "$dmesg_marker" >"$dmesg_log"
 	grep -q -e "kernel BUG at" \
 	     -e "WARNING:" \
 	     -e "BUG:" \
@@ -45,12 +50,12 @@ _check_dmesg()
 	     -e "INFO: possible circular locking dependency detected" \
 	     -e "general protection fault:" \
 	     -e "blktests failure" \
-	     "${seqres}.dmesg"
+	     "$dmesg_log"
 	# shellcheck disable=SC2181
 	if [[ $? -eq 0 ]]; then
 		return 1
 	else
-		rm -f "${seqres}.dmesg"
+		rm -f "$dmesg_log"
 		return 0
 	fi
 }
@@ -94,7 +99,7 @@ run_test()
 		echo "Test $test_name failed with ret $status"
 		FAILED="$FAILED <$test_string>"
 		RET=1
-	elif ! _check_dmesg "$dmesg_marker" "$test_name"; then
+	elif ! _check_dmesg "$dmesg_marker" "$test_name" "$dev"; then
 		echo "Test $test_name failed dmesg check"
 		FAILED="$FAILED <$test_string>"
 		RET=1
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/5] test: make a distinction between successful and skipped test
  2020-09-18 10:47 [PATCH 1/5] test: save dmesg output for each test and test file Lukas Czerner
@ 2020-09-18 10:47 ` Lukas Czerner
  2020-09-18 10:47 ` [PATCH 3/5] test: store test output to a log file Lukas Czerner
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Lukas Czerner @ 2020-09-18 10:47 UTC (permalink / raw)
  To: io-uring

Currently when the test is skipped, it just returns zero and the test
will be assumed to have been successful so the only way to know
which tests were actually skipped is to search the output and inspect
the code.

Change the tests to return -1 in the case that the entire test
is skipped. Some of the tests are exercising multiple features some of
which may not be available and hence can be skipped - I am not
changing those for now. The rule for the test to be marked as skipped
is that nothing meaningful was actually tested with regards to liburing.

One exception are tests that don't run anything if test file is
provided. I am not sure if it's usefull to mark them as skipped so I am
leaving the original behavior for now.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 test/500f9fbadef8-test.c     |  2 +-
 test/accept-link.c           |  6 ++++--
 test/accept-reuse.c          |  2 +-
 test/accept.c                |  2 +-
 test/cq-overflow-peek.c      |  8 +++++---
 test/cq-size.c               |  3 +--
 test/d4ae271dfaae-test.c     |  2 +-
 test/d77a67ed5f27-test.c     |  2 +-
 test/eventfd-disable.c       |  2 +-
 test/eventfd-ring.c          |  2 +-
 test/eventfd.c               |  2 +-
 test/fadvise.c               |  2 +-
 test/fallocate.c             |  7 ++++---
 test/fc2a85cb02ef-test.c     |  2 +-
 test/madvise.c               |  2 +-
 test/open-close.c            | 25 ++++++++++++-------------
 test/openat2.c               | 19 ++++++++-----------
 test/personality.c           |  4 ++--
 test/poll-link.c             |  8 ++++++--
 test/poll-many.c             |  6 +++---
 test/probe.c                 |  2 +-
 test/register-restrictions.c |  2 +-
 test/runtests.sh             |  5 ++++-
 test/send_recvmsg.c          |  1 -
 test/shared-wq.c             |  5 +++--
 test/shutdown.c              |  6 ++++--
 test/splice.c                |  3 +++
 test/sq-poll-dup.c           |  5 +++++
 test/sq-poll-kthread.c       |  2 ++
 test/sq-poll-share.c         |  2 +-
 test/statx.c                 | 19 +++++++++----------
 test/timeout-overflow.c      |  2 +-
 32 files changed, 90 insertions(+), 72 deletions(-)

diff --git a/test/500f9fbadef8-test.c b/test/500f9fbadef8-test.c
index 9ebff43..d3cdb66 100644
--- a/test/500f9fbadef8-test.c
+++ b/test/500f9fbadef8-test.c
@@ -87,5 +87,5 @@ skipped:
 	fprintf(stderr, "Polling not supported in current dir, test skipped\n");
 	close(fd);
 	unlink(buf);
-	return 0;
+	return -1;
 }
diff --git a/test/accept-link.c b/test/accept-link.c
index 7e4df48..c30bb3b 100644
--- a/test/accept-link.c
+++ b/test/accept-link.c
@@ -105,7 +105,7 @@ void *recv_thread(void *arg)
 		printf("Can't find good port, skipped\n");
 		data->stop = 1;
 		signal_var(&recv_thread_ready);
-		goto out;
+		goto skip;
 	}
 
         assert(listen(s0, 128) != -1);
@@ -158,12 +158,14 @@ ok:
 
 	signal_var(&recv_thread_done);
 
-out:
 	close(s0);
 	return NULL;
 err:
 	close(s0);
 	return (void *) 1;
+skip:
+	close(s0);
+	return (void *) -1;
 }
 
 static int test_accept_timeout(int do_connect, unsigned long timeout)
diff --git a/test/accept-reuse.c b/test/accept-reuse.c
index 59a2f79..1f2249f 100644
--- a/test/accept-reuse.c
+++ b/test/accept-reuse.c
@@ -55,7 +55,7 @@ int main(int argc, char **argv)
 	}
 	if (!(params.features & IORING_FEAT_SUBMIT_STABLE)) {
 		fprintf(stdout, "FEAT_SUBMIT_STABLE not there, skipping\n");
-		return 0;
+		return -1;
 	}
 
 	memset(&hints, 0, sizeof(hints));
diff --git a/test/accept.c b/test/accept.c
index faf81d6..24834f9 100644
--- a/test/accept.c
+++ b/test/accept.c
@@ -373,7 +373,7 @@ int main(int argc, char *argv[])
 		return ret;
 	}
 	if (no_accept)
-		return 0;
+		return -1;
 
 	ret = test_accept_sqpoll();
 	if (ret) {
diff --git a/test/cq-overflow-peek.c b/test/cq-overflow-peek.c
index 72b6768..9fc1dea 100644
--- a/test/cq-overflow-peek.c
+++ b/test/cq-overflow-peek.c
@@ -35,7 +35,7 @@ static int test_cq_overflow(struct io_uring *ring)
 	flags = IO_URING_READ_ONCE(*ring->sq.kflags);
 	if (!(flags & IORING_SQ_CQ_OVERFLOW)) {
 		fprintf(stdout, "OVERFLOW not set on -EBUSY, skipping\n");
-		goto done;
+		goto skip;
 	}
 
 	while (issued) {
@@ -52,10 +52,11 @@ static int test_cq_overflow(struct io_uring *ring)
 		issued--;
 	}
 
-done:
 	return 0;
 err:
 	return 1;
+skip:
+	return -1;
 }
 
 int main(int argc, char *argv[])
@@ -74,7 +75,8 @@ int main(int argc, char *argv[])
 
 	ret = test_cq_overflow(&ring);
 	if (ret) {
-		fprintf(stderr, "test_cq_overflow failed\n");
+		if (ret != -1)
+			fprintf(stderr, "test_cq_overflow failed\n");
 		return 1;
 	}
 
diff --git a/test/cq-size.c b/test/cq-size.c
index b7dd5b4..184626a 100644
--- a/test/cq-size.c
+++ b/test/cq-size.c
@@ -28,7 +28,7 @@ int main(int argc, char *argv[])
 	if (ret) {
 		if (ret == -EINVAL) {
 			printf("Skipped, not supported on this kernel\n");
-			goto done;
+			return -1;
 		}
 		printf("ring setup failed\n");
 		return 1;
@@ -50,7 +50,6 @@ int main(int argc, char *argv[])
 		goto err;
 	}
 
-done:
 	return 0;
 err:
 	io_uring_queue_exit(&ring);
diff --git a/test/d4ae271dfaae-test.c b/test/d4ae271dfaae-test.c
index 6f263c6..b571b56 100644
--- a/test/d4ae271dfaae-test.c
+++ b/test/d4ae271dfaae-test.c
@@ -48,7 +48,7 @@ int main(int argc, char *argv[])
 
 	if (geteuid()) {
 		fprintf(stdout, "Test requires root, skipping\n");
-		return 0;
+		return -1;
 	}
 
 	memset(&p, 0, sizeof(p));
diff --git a/test/d77a67ed5f27-test.c b/test/d77a67ed5f27-test.c
index f3ef071..11e8d2b 100644
--- a/test/d77a67ed5f27-test.c
+++ b/test/d77a67ed5f27-test.c
@@ -32,7 +32,7 @@ int main(int argc, char *argv[])
 	if (ret) {
 		if (geteuid()) {
 			fprintf(stdout, "SQPOLL requires root, skipped\n");
-			return 0;
+			return -1;
 		}
 		fprintf(stderr, "ring create failed: %d\n", ret);
 		return 1;
diff --git a/test/eventfd-disable.c b/test/eventfd-disable.c
index f172fd7..283a004 100644
--- a/test/eventfd-disable.c
+++ b/test/eventfd-disable.c
@@ -56,7 +56,7 @@ int main(int argc, char *argv[])
 	ret = io_uring_cq_eventfd_toggle(&ring, false);
 	if (ret) {
 		fprintf(stdout, "Skipping, CQ flags not available!\n");
-		return 0;
+		return -1;
 	}
 
 	sqe = io_uring_get_sqe(&ring);
diff --git a/test/eventfd-ring.c b/test/eventfd-ring.c
index 67e102c..20f0b65 100644
--- a/test/eventfd-ring.c
+++ b/test/eventfd-ring.c
@@ -31,7 +31,7 @@ int main(int argc, char *argv[])
 	}
 	if (!(p.features & IORING_FEAT_CUR_PERSONALITY)) {
 		fprintf(stdout, "Skipping\n");
-		return 0;
+		return -1;
 	}
 	ret = io_uring_queue_init(8, &ring2, 0);
 	if (ret) {
diff --git a/test/eventfd.c b/test/eventfd.c
index 1a7e3f3..8db54f9 100644
--- a/test/eventfd.c
+++ b/test/eventfd.c
@@ -37,7 +37,7 @@ int main(int argc, char *argv[])
 	}
 	if (!(p.features & IORING_FEAT_CUR_PERSONALITY)) {
 		fprintf(stdout, "Skipping\n");
-		return 0;
+		return -1;
 	}
 
 	evfd = eventfd(0, EFD_CLOEXEC);
diff --git a/test/fadvise.c b/test/fadvise.c
index 0759446..7a522a6 100644
--- a/test/fadvise.c
+++ b/test/fadvise.c
@@ -92,7 +92,7 @@ static int do_fadvise(struct io_uring *ring, int fd, off_t offset, off_t len,
 	if (ret == -EINVAL || ret == -EBADF) {
 		fprintf(stdout, "Fadvise not supported, skipping\n");
 		unlink(".fadvise.tmp");
-		exit(0);
+		exit(-1);
 	} else if (ret) {
 		fprintf(stderr, "cqe->res=%d\n", cqe->res);
 	}
diff --git a/test/fallocate.c b/test/fallocate.c
index e662a6a..d6cb657 100644
--- a/test/fallocate.c
+++ b/test/fallocate.c
@@ -71,6 +71,7 @@ static int test_fallocate_rlimit(struct io_uring *ring)
 		goto err;
 	}
 	io_uring_cqe_seen(ring, cqe);
+
 out:
 	unlink(buf);
 	return 0;
@@ -151,9 +152,6 @@ static int test_fallocate_fsync(struct io_uring *ring)
 	char buf[32];
 	int fd, ret, i;
 
-	if (no_fallocate)
-		return 0;
-
 	sprintf(buf, "./XXXXXX");
 	fd = mkstemp(buf);
 	if (fd < 0) {
@@ -236,6 +234,9 @@ int main(int argc, char *argv[])
 		return ret;
 	}
 
+	if (no_fallocate)
+		return -1;
+
 	ret = test_fallocate_fsync(&ring);
 	if (ret) {
 		fprintf(stderr, "test_fallocate_fsync failed\n");
diff --git a/test/fc2a85cb02ef-test.c b/test/fc2a85cb02ef-test.c
index e922d17..e4bebed 100644
--- a/test/fc2a85cb02ef-test.c
+++ b/test/fc2a85cb02ef-test.c
@@ -92,7 +92,7 @@ int main(int argc, char *argv[])
   mmap((void *) 0x20000000ul, 0x1000000ul, 3ul, 0x32ul, -1, 0);
   if (setup_fault()) {
     printf("Test needs failslab/fail_futex/fail_page_alloc enabled, skipped\n");
-    return 0;
+    return -1;
   }
   intptr_t res = 0;
   *(uint32_t*)0x20000000 = 0;
diff --git a/test/madvise.c b/test/madvise.c
index e3af4f1..330cb5e 100644
--- a/test/madvise.c
+++ b/test/madvise.c
@@ -93,7 +93,7 @@ static int do_madvise(struct io_uring *ring, void *addr, off_t len, int advice)
 	if (ret == -EINVAL || ret == -EBADF) {
 		fprintf(stdout, "Madvise not supported, skipping\n");
 		unlink(".madvise.tmp");
-		exit(0);
+		exit(-1);
 	} else if (ret) {
 		fprintf(stderr, "cqe->res=%d\n", cqe->res);
 	}
diff --git a/test/open-close.c b/test/open-close.c
index cb74d91..fdfbf18 100644
--- a/test/open-close.c
+++ b/test/open-close.c
@@ -100,7 +100,7 @@ int main(int argc, char *argv[])
 {
 	struct io_uring ring;
 	const char *path, *path_rel;
-	int ret, do_unlink;
+	int ret, do_unlink, err = 0;
 
 	ret = io_uring_queue_init(8, &ring, 0);
 	if (ret) {
@@ -131,38 +131,37 @@ int main(int argc, char *argv[])
 	if (ret < 0) {
 		if (ret == -EINVAL) {
 			fprintf(stdout, "Open not supported, skipping\n");
-			goto done;
+			err = -1;
+			goto out;
 		}
 		fprintf(stderr, "test_openat absolute failed: %d\n", ret);
-		goto err;
+		err = 1;
+		goto out;
 	}
 
 	ret = test_openat(&ring, path_rel, AT_FDCWD);
 	if (ret < 0) {
 		fprintf(stderr, "test_openat relative failed: %d\n", ret);
-		goto err;
+		err = 1;
+		goto out;
 	}
 
 	ret = test_close(&ring, ret, 0);
 	if (ret) {
 		fprintf(stderr, "test_close normal failed\n");
-		goto err;
+		err = 1;
+		goto out;
 	}
 
 	ret = test_close(&ring, ring.ring_fd, 1);
 	if (ret != -EBADF) {
 		fprintf(stderr, "test_close ring_fd failed\n");
-		goto err;
+		err = 1;
 	}
 
-done:
-	unlink(path);
-	if (do_unlink)
-		unlink(path_rel);
-	return 0;
-err:
+out:
 	unlink(path);
 	if (do_unlink)
 		unlink(path_rel);
-	return 1;
+	return err;
 }
diff --git a/test/openat2.c b/test/openat2.c
index 197821a..de59c4b 100644
--- a/test/openat2.c
+++ b/test/openat2.c
@@ -69,7 +69,7 @@ int main(int argc, char *argv[])
 {
 	struct io_uring ring;
 	const char *path, *path_rel;
-	int ret, do_unlink;
+	int ret, do_unlink, err = 0;
 
 	ret = io_uring_queue_init(8, &ring, 0);
 	if (ret) {
@@ -100,26 +100,23 @@ int main(int argc, char *argv[])
 	if (ret < 0) {
 		if (ret == -EINVAL) {
 			fprintf(stdout, "openat2 not supported, skipping\n");
-			goto done;
+			err = -1;
+			goto out;
 		}
 		fprintf(stderr, "test_openat2 absolute failed: %d\n", ret);
-		goto err;
+		err = 1;
+		goto out;
 	}
 
 	ret = test_openat2(&ring, path_rel, AT_FDCWD);
 	if (ret < 0) {
 		fprintf(stderr, "test_openat2 relative failed: %d\n", ret);
-		goto err;
+		err = 1;
 	}
 
-done:
-	unlink(path);
-	if (do_unlink)
-		unlink(path_rel);
-	return 0;
-err:
+out:
 	unlink(path);
 	if (do_unlink)
 		unlink(path_rel);
-	return 1;
+	return err;
 }
diff --git a/test/personality.c b/test/personality.c
index 591ec83..be8fc08 100644
--- a/test/personality.c
+++ b/test/personality.c
@@ -171,7 +171,7 @@ int main(int argc, char *argv[])
 
 	if (geteuid()) {
 		fprintf(stderr, "Not root, skipping\n");
-		return 0;
+		return -1;
 	}
 
 	ret = io_uring_queue_init(8, &ring, 0);
@@ -186,7 +186,7 @@ int main(int argc, char *argv[])
 		return ret;
 	}
 	if (no_personality)
-		return 0;
+		return -1;
 
 	ret = test_invalid_personality(&ring);
 	if (ret) {
diff --git a/test/poll-link.c b/test/poll-link.c
index d0786d4..3bb0e3c 100644
--- a/test/poll-link.c
+++ b/test/poll-link.c
@@ -102,7 +102,7 @@ void *recv_thread(void *arg)
 		fprintf(stderr, "Can't find good port, skipped\n");
 		data->stop = 1;
 		signal_var(&recv_thread_ready);
-		goto out;
+		goto skip;
 	}
 
         assert(listen(s0, 128) != -1);
@@ -150,7 +150,6 @@ void *recv_thread(void *arg)
 		io_uring_cqe_seen(&ring, cqe);
 	}
 
-out:
 	signal_var(&recv_thread_done);
 	close(s0);
 	io_uring_queue_exit(&ring);
@@ -160,6 +159,11 @@ err:
 	close(s0);
 	io_uring_queue_exit(&ring);
 	return (void *) 1;
+skip:
+	signal_var(&recv_thread_done);
+	close(s0);
+	io_uring_queue_exit(&ring);
+	return (void *) -1;
 }
 
 static int test_poll_timeout(int do_connect, unsigned long timeout)
diff --git a/test/poll-many.c b/test/poll-many.c
index 723a353..85322b0 100644
--- a/test/poll-many.c
+++ b/test/poll-many.c
@@ -156,7 +156,7 @@ int main(int argc, char *argv[])
 		rlim.rlim_max = rlim.rlim_cur;
 		if (setrlimit(RLIMIT_NOFILE, &rlim) < 0) {
 			if (errno == EPERM)
-				goto err_nofail;
+				goto err_skip;
 			perror("setrlimit");
 			goto err_noring;
 		}
@@ -191,8 +191,8 @@ err:
 err_noring:
 	fprintf(stderr, "poll-many failed\n");
 	return 1;
-err_nofail:
+err_skip:
 	fprintf(stderr, "poll-many: not enough files available (and not root), "
 			"skipped\n");
-	return 0;
+	return -1;
 }
diff --git a/test/probe.c b/test/probe.c
index 1961176..bc49744 100644
--- a/test/probe.c
+++ b/test/probe.c
@@ -123,7 +123,7 @@ int main(int argc, char *argv[])
 		return ret;
 	}
 	if (no_probe)
-		return 0;
+		return -1;
 
 	ret = test_probe_helper(&ring);
 	if (ret) {
diff --git a/test/register-restrictions.c b/test/register-restrictions.c
index 4f64c41..d46f860 100644
--- a/test/register-restrictions.c
+++ b/test/register-restrictions.c
@@ -569,7 +569,7 @@ int main(int argc, char *argv[])
 	ret = test_restrictions_sqe_op();
 	if (ret == TEST_SKIPPED) {
 		printf("test_restrictions_sqe_op: skipped\n");
-		return 0;
+		return -1;
 	} else if (ret == TEST_FAILED) {
 		fprintf(stderr, "test_restrictions_sqe_op failed\n");
 		return ret;
diff --git a/test/runtests.sh b/test/runtests.sh
index 5107a0a..acefe33 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -95,7 +95,7 @@ run_test()
 	# Check test status
 	if [ "$status" -eq 124 ]; then
 		echo "Test $test_name timed out (may not be a failure)"
-	elif [ "$status" -ne 0 ]; then
+	elif [ "$status" -ne 0 ] && [ "$status" -ne 255 ]; then
 		echo "Test $test_name failed with ret $status"
 		FAILED="$FAILED <$test_string>"
 		RET=1
@@ -103,6 +103,9 @@ run_test()
 		echo "Test $test_name failed dmesg check"
 		FAILED="$FAILED <$test_string>"
 		RET=1
+	elif [ "$status" -eq 255 ]; then
+		echo "Test skipped"
+		SKIPPED="$SKIPPED <$test_string>"
 	elif [ -n "$dev" ]; then
 		sleep .1
 		ps aux | grep "\[io_wq_manager\]" > /dev/null
diff --git a/test/send_recvmsg.c b/test/send_recvmsg.c
index 50c8e94..3f64f38 100644
--- a/test/send_recvmsg.c
+++ b/test/send_recvmsg.c
@@ -177,7 +177,6 @@ static void *recv_fn(void *data)
 		if (ret == -EINVAL) {
 			fprintf(stdout, "PROVIDE_BUFFERS not supported, skip\n");
 			goto out;
-			goto err;
 		} else if (ret < 0) {
 			fprintf(stderr, "PROVIDER_BUFFERS %d\n", ret);
 			goto err;
diff --git a/test/shared-wq.c b/test/shared-wq.c
index c0571e6..795d7b8 100644
--- a/test/shared-wq.c
+++ b/test/shared-wq.c
@@ -42,7 +42,7 @@ static int test_attach(int ringfd)
 	ret = io_uring_queue_init_params(1, &ring2, &p);
 	if (ret == -EINVAL) {
 		fprintf(stdout, "Sharing not supported, skipping\n");
-		return 0;
+		return -1;
 	} else if (ret) {
 		fprintf(stderr, "Attach to id: %d\n", ret);
 		goto err;
@@ -76,7 +76,8 @@ int main(int argc, char *argv[])
 
 	ret = test_attach(ring.ring_fd);
 	if (ret) {
-		fprintf(stderr, "test_attach failed\n");
+		if (ret != -1)
+			fprintf(stderr, "test_attach failed\n");
 		return ret;
 	}
 
diff --git a/test/shutdown.c b/test/shutdown.c
index eb66ded..533a9aa 100644
--- a/test/shutdown.c
+++ b/test/shutdown.c
@@ -102,7 +102,7 @@ int main(int argc, char *argv[])
 		if (cqe->res) {
 			if (cqe->res == -EINVAL) {
 				fprintf(stdout, "Shutdown not supported, skipping\n");
-				goto done;
+				goto skip;
 			}
 			fprintf(stderr, "writev: %d\n", cqe->res);
 			goto err;
@@ -140,10 +140,12 @@ int main(int argc, char *argv[])
 		io_uring_cqe_seen(&m_io_uring, cqe);
 	}
 
-done:
 	io_uring_queue_exit(&m_io_uring);
 	return 0;
 err:
 	io_uring_queue_exit(&m_io_uring);
 	return 1;
+skip:
+	io_uring_queue_exit(&m_io_uring);
+	return -1;
 }
diff --git a/test/splice.c b/test/splice.c
index e67bb10..d2b5a6d 100644
--- a/test/splice.c
+++ b/test/splice.c
@@ -472,6 +472,9 @@ int main(int argc, char *argv[])
 	if (!has_tee)
 		fprintf(stdout, "skip, doesn't support tee()\n");
 
+	if (!has_splice && !has_tee)
+		return -1;
+
 	ret = test_splice(&ring, &ctx);
 	if (ret) {
 		fprintf(stderr, "basic splice tests failed\n");
diff --git a/test/sq-poll-dup.c b/test/sq-poll-dup.c
index 79e623a..a8b39ce 100644
--- a/test/sq-poll-dup.c
+++ b/test/sq-poll-dup.c
@@ -23,6 +23,7 @@
 
 static struct iovec *vecs;
 static struct io_uring rings[NR_RINGS];
+static int no_nonfixed;
 
 static int create_buffers(void)
 {
@@ -142,6 +143,7 @@ static int test(int fd, int do_dup_and_close, int close_ring)
 		/* no sharing for non-fixed either */
 		if (!(p.features & IORING_FEAT_SQPOLL_NONFIXED)) {
 			fprintf(stdout, "No SQPOLL sharing, skipping\n");
+			no_nonfixed = 1;
 			return 0;
 		}
 	}
@@ -215,6 +217,9 @@ int main(int argc, char *argv[])
 		goto err;
 	}
 
+	if (no_nonfixed)
+		return -1;
+
 	ret = test(fd, 0, 1);
 	if (ret) {
 		fprintf(stderr, "test 0 1 failed\n");
diff --git a/test/sq-poll-kthread.c b/test/sq-poll-kthread.c
index ed7d0bf..f3b6ad3 100644
--- a/test/sq-poll-kthread.c
+++ b/test/sq-poll-kthread.c
@@ -153,6 +153,7 @@ int main(int argc, char *argv[])
 	ret = test_sq_poll_kthread_stopped(true);
 	if (ret == TEST_SKIPPED) {
 		printf("test_sq_poll_kthread_stopped_exit: skipped\n");
+		return -1;
 	} else if (ret == TEST_FAILED) {
 		fprintf(stderr, "test_sq_poll_kthread_stopped_exit failed\n");
 		return ret;
@@ -161,6 +162,7 @@ int main(int argc, char *argv[])
 	ret = test_sq_poll_kthread_stopped(false);
 	if (ret == TEST_SKIPPED) {
 		printf("test_sq_poll_kthread_stopped_noexit: skipped\n");
+		return -1;
 	} else if (ret == TEST_FAILED) {
 		fprintf(stderr, "test_sq_poll_kthread_stopped_noexit failed\n");
 		return ret;
diff --git a/test/sq-poll-share.c b/test/sq-poll-share.c
index 0f25389..0e1b438 100644
--- a/test/sq-poll-share.c
+++ b/test/sq-poll-share.c
@@ -140,7 +140,7 @@ int main(int argc, char *argv[])
 		/* no sharing for non-fixed either */
 		if (!(p.features & IORING_FEAT_SQPOLL_NONFIXED)) {
 			fprintf(stdout, "No SQPOLL sharing, skipping\n");
-			return 0;
+			return -1;
 		}
 	}
 
diff --git a/test/statx.c b/test/statx.c
index c846a4a..a371c4f 100644
--- a/test/statx.c
+++ b/test/statx.c
@@ -149,7 +149,7 @@ int main(int argc, char *argv[])
 {
 	struct io_uring ring;
 	const char *fname;
-	int ret;
+	int ret, err = 0;
 
 	ret = io_uring_queue_init(8, &ring, 0);
 	if (ret) {
@@ -171,23 +171,22 @@ int main(int argc, char *argv[])
 	if (ret) {
 		if (ret == -EINVAL) {
 			fprintf(stdout, "statx not supported, skipping\n");
-			goto done;
+			err = -1;
+			goto out;
 		}
 		fprintf(stderr, "test_statx failed: %d\n", ret);
-		goto err;
+		err = 1;
+		goto out;
 	}
 
 	ret = test_statx_fd(&ring, fname);
 	if (ret) {
 		fprintf(stderr, "test_statx_fd failed: %d\n", ret);
-		goto err;
+		err = 1;
+		goto out;
 	}
-done:
-	if (fname != argv[1])
-		unlink(fname);
-	return 0;
-err:
+out:
 	if (fname != argv[1])
 		unlink(fname);
-	return 1;
+	return err;
 }
diff --git a/test/timeout-overflow.c b/test/timeout-overflow.c
index f952f80..baa674b 100644
--- a/test/timeout-overflow.c
+++ b/test/timeout-overflow.c
@@ -192,7 +192,7 @@ int main(int argc, char *argv[])
 	}
 
 	if (not_supported)
-		return 0;
+		return -1;
 
 	ret = test_timeout_overflow();
 	if (ret) {
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 3/5] test: store test output to a log file
  2020-09-18 10:47 [PATCH 1/5] test: save dmesg output for each test and test file Lukas Czerner
  2020-09-18 10:47 ` [PATCH 2/5] test: make a distinction between successful and skipped test Lukas Czerner
@ 2020-09-18 10:47 ` Lukas Czerner
  2020-09-18 10:47 ` [PATCH 4/5] test: make test output more readable Lukas Czerner
  2020-09-18 10:47 ` [PATCH 5/5] test: handle the case when timeout is forced to KILL Lukas Czerner
  3 siblings, 0 replies; 5+ messages in thread
From: Lukas Czerner @ 2020-09-18 10:47 UTC (permalink / raw)
  To: io-uring

Store test output to a log file for further inspection. Depending on
the test result the log file name for the test will be one of the
following ${test_name}.{log,timeout,failed,skipped}

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 .gitignore       |  4 ++++
 test/runtests.sh | 27 ++++++++++++++++++++++++---
 2 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/.gitignore b/.gitignore
index 8f7f369..9e7865b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -107,6 +107,10 @@
 /test/timeout-overflow
 /test/wakeup-hang
 /test/*.dmesg
+/test/*.log
+/test/*.failed
+/test/*.skipped
+/test/*.timeout
 
 config-host.h
 config-host.mak
diff --git a/test/runtests.sh b/test/runtests.sh
index acefe33..b61cb27 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -83,29 +83,42 @@ run_test()
 	# Do we have to exclude the test ?
 	echo $TEST_EXCLUDE | grep -w "$test_name" > /dev/null 2>&1
 	if [ $? -eq 0 ]; then
-		echo "Test skipped"
+		echo "Test skipped by user" | tee ${test_name}.skipped
 		SKIPPED="$SKIPPED <$test_string>"
 		return
 	fi
 
+	# Prepare log file name
+	if [ -n "$dev" ]; then
+		local logfile=$(echo "${test_name}_${dev}" | \
+			    sed 's/\(\/\|_\/\|\/_\)/_/g')
+	else
+		local logfile=${test_name}
+	fi
+
 	# Run the test
-	timeout --preserve-status -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
-	local status=$?
+	timeout --preserve-status -s INT -k $TIMEOUT $TIMEOUT \
+		./$test_name $dev 2>&1 | tee ${logfile}.log
+	local status=${PIPESTATUS[0]}
 
 	# Check test status
 	if [ "$status" -eq 124 ]; then
 		echo "Test $test_name timed out (may not be a failure)"
+		mv ${logfile}.log ${logfile}.timeout
 	elif [ "$status" -ne 0 ] && [ "$status" -ne 255 ]; then
 		echo "Test $test_name failed with ret $status"
 		FAILED="$FAILED <$test_string>"
 		RET=1
+		mv ${logfile}.log ${logfile}.failed
 	elif ! _check_dmesg "$dmesg_marker" "$test_name" "$dev"; then
 		echo "Test $test_name failed dmesg check"
 		FAILED="$FAILED <$test_string>"
 		RET=1
+		mv ${logfile}.log ${logfile}.failed
 	elif [ "$status" -eq 255 ]; then
 		echo "Test skipped"
 		SKIPPED="$SKIPPED <$test_string>"
+		mv ${logfile}.log ${logfile}.skipped
 	elif [ -n "$dev" ]; then
 		sleep .1
 		ps aux | grep "\[io_wq_manager\]" > /dev/null
@@ -113,8 +126,16 @@ run_test()
 			MAYBE_FAILED="$MAYBE_FAILED $test_string"
 		fi
 	fi
+
+	# Only leave behing log file with some content in it
+	if [ ! -s "${logfile}.log" ]; then
+		rm -f ${logfile}.log
+	fi
 }
 
+# Clean up all the logs from previous run
+rm -f *.{log,timeout,failed,skipped,dmesg}
+
 # Run all specified tests
 for tst in $TESTS; do
 	run_test $tst
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 4/5] test: make test output more readable
  2020-09-18 10:47 [PATCH 1/5] test: save dmesg output for each test and test file Lukas Czerner
  2020-09-18 10:47 ` [PATCH 2/5] test: make a distinction between successful and skipped test Lukas Czerner
  2020-09-18 10:47 ` [PATCH 3/5] test: store test output to a log file Lukas Czerner
@ 2020-09-18 10:47 ` Lukas Czerner
  2020-09-18 10:47 ` [PATCH 5/5] test: handle the case when timeout is forced to KILL Lukas Czerner
  3 siblings, 0 replies; 5+ messages in thread
From: Lukas Czerner @ 2020-09-18 10:47 UTC (permalink / raw)
  To: io-uring

Make the result of the test clear and aligned for better readability.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 test/runtests.sh | 101 +++++++++++++++++++++++++++++++++++------------
 1 file changed, 76 insertions(+), 25 deletions(-)

diff --git a/test/runtests.sh b/test/runtests.sh
index b61cb27..ed28ec8 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -9,6 +9,7 @@ TEST_FILES=""
 FAILED=""
 SKIPPED=""
 MAYBE_FAILED=""
+TESTNAME_WIDTH=40
 
 # Only use /dev/kmsg if running as root
 DO_KMSG="1"
@@ -60,6 +61,62 @@ _check_dmesg()
 	fi
 }
 
+test_result()
+{
+	local result=$1
+	local logfile=$2
+	local test_string=$3
+	local msg=$4
+
+	[ -n "$msg" ] && msg="($msg)"
+
+	local RES=""
+	local logfile_move=""
+	local logmsg=""
+
+	case $result in
+		pass)
+			RES="OK";;
+		skip)
+			RES="SKIP"
+			SKIPPED="$SKIPPED <$test_string>"
+			logfile_move="${logfile}.skipped"
+			log_msg="Test ${test_string} skipped"
+			;;
+		timeout)
+			RES="TIMEOUT"
+			logfile_move="${logfile}.timeout"
+			log_msg="Test $test_name timed out (may not be a failure)"
+			;;
+		fail)
+			RET=1
+			RES="FAIL"
+			FAILED="$FAILED <$test_string>"
+			logfile_move="${logfile}.failed"
+			log_msg="Test ${test_string} failed"
+			;;
+		*)
+			echo "Unexpected result"
+			exit 1
+			;;
+	esac
+
+	# Print the result of the test
+	printf "\t$RES $msg\n"
+
+	[ "$result" == "pass" ] && return
+
+	# Show the test log in case something went wrong
+	if [ -s "${logfile}.log" ]; then
+		cat "${logfile}.log" | sed 's/^\(.\)/    \1/'
+	fi
+
+	echo "$log_msg $msg" >> ${logfile}.log
+
+	# Rename the log
+	[ -n "${logfile_move}" ] && mv ${logfile}.log ${logfile_move}
+}
+
 run_test()
 {
 	local test_name="$1"
@@ -73,19 +130,12 @@ run_test()
 
 	# Log start of the test
 	if [ "$DO_KMSG" -eq 1 ]; then
-		local dmesg_marker="Running test $test_string:"
-		echo $dmesg_marker | tee /dev/kmsg
+		local dmesg_marker="Running test $test_string"
+		echo $dmesg_marker > /dev/kmsg
+		printf "%-${TESTNAME_WIDTH}s" "$test_string"
 	else
 		local dmesg_marker=""
-		echo Running test $test_name $dev
-	fi
-
-	# Do we have to exclude the test ?
-	echo $TEST_EXCLUDE | grep -w "$test_name" > /dev/null 2>&1
-	if [ $? -eq 0 ]; then
-		echo "Test skipped by user" | tee ${test_name}.skipped
-		SKIPPED="$SKIPPED <$test_string>"
-		return
+		printf "%-${TESTNAME_WIDTH}s" "$test_name $dev"
 	fi
 
 	# Prepare log file name
@@ -96,35 +146,36 @@ run_test()
 		local logfile=${test_name}
 	fi
 
+	# Do we have to exclude the test ?
+	echo $TEST_EXCLUDE | grep -w "$test_name" > /dev/null 2>&1
+	if [ $? -eq 0 ]; then
+		test_skipped "${logfile}" "$test_string" "by user"
+		return
+	fi
+
 	# Run the test
 	timeout --preserve-status -s INT -k $TIMEOUT $TIMEOUT \
-		./$test_name $dev 2>&1 | tee ${logfile}.log
+		./$test_name $dev > ${logfile}.log 2>&1
 	local status=${PIPESTATUS[0]}
 
 	# Check test status
 	if [ "$status" -eq 124 ]; then
-		echo "Test $test_name timed out (may not be a failure)"
-		mv ${logfile}.log ${logfile}.timeout
+		test_result timeout "${logfile}" "${test_string}"
 	elif [ "$status" -ne 0 ] && [ "$status" -ne 255 ]; then
-		echo "Test $test_name failed with ret $status"
-		FAILED="$FAILED <$test_string>"
-		RET=1
-		mv ${logfile}.log ${logfile}.failed
+		test_result fail "${logfile}" "${test_string}" "status = $status"
 	elif ! _check_dmesg "$dmesg_marker" "$test_name" "$dev"; then
-		echo "Test $test_name failed dmesg check"
-		FAILED="$FAILED <$test_string>"
-		RET=1
-		mv ${logfile}.log ${logfile}.failed
+		test_result fail "${logfile}" "${test_string}" "dmesg check"
 	elif [ "$status" -eq 255 ]; then
-		echo "Test skipped"
-		SKIPPED="$SKIPPED <$test_string>"
-		mv ${logfile}.log ${logfile}.skipped
+		test_result skip "${logfile}" "${test_string}"
 	elif [ -n "$dev" ]; then
 		sleep .1
 		ps aux | grep "\[io_wq_manager\]" > /dev/null
 		if [ $? -eq 0 ]; then
 			MAYBE_FAILED="$MAYBE_FAILED $test_string"
 		fi
+		test_result pass "${logfile}" "${test_string}"
+	else
+		test_result pass "${logfile}" "${test_string}"
 	fi
 
 	# Only leave behing log file with some content in it
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 5/5] test: handle the case when timeout is forced to KILL
  2020-09-18 10:47 [PATCH 1/5] test: save dmesg output for each test and test file Lukas Czerner
                   ` (2 preceding siblings ...)
  2020-09-18 10:47 ` [PATCH 4/5] test: make test output more readable Lukas Czerner
@ 2020-09-18 10:47 ` Lukas Czerner
  3 siblings, 0 replies; 5+ messages in thread
From: Lukas Czerner @ 2020-09-18 10:47 UTC (permalink / raw)
  To: io-uring

Timeout is used to INTerrupt the test if it is still running after
pre-defined time. If the process does not terminate KILL signal is sent
after a pre-defined time, but the return status is different in that
case (128+9).
Catch it and treat is as test failure with the message that the
process was killed.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 test/runtests.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/test/runtests.sh b/test/runtests.sh
index ed28ec8..b210013 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -161,6 +161,8 @@ run_test()
 	# Check test status
 	if [ "$status" -eq 124 ]; then
 		test_result timeout "${logfile}" "${test_string}"
+	elif [ "$status" -eq 137 ]; then
+		test_failed "${logfile}" "${test_string}" "process Killed"
 	elif [ "$status" -ne 0 ] && [ "$status" -ne 255 ]; then
 		test_result fail "${logfile}" "${test_string}" "status = $status"
 	elif ! _check_dmesg "$dmesg_marker" "$test_name" "$dev"; then
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-09-18 10:54 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-18 10:47 [PATCH 1/5] test: save dmesg output for each test and test file Lukas Czerner
2020-09-18 10:47 ` [PATCH 2/5] test: make a distinction between successful and skipped test Lukas Czerner
2020-09-18 10:47 ` [PATCH 3/5] test: store test output to a log file Lukas Czerner
2020-09-18 10:47 ` [PATCH 4/5] test: make test output more readable Lukas Czerner
2020-09-18 10:47 ` [PATCH 5/5] test: handle the case when timeout is forced to KILL Lukas Czerner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).