All of lore.kernel.org
 help / color / mirror / Atom feed
* [LTP] [PATCH v3 00/29] Introduce runtime and conver tests
@ 2022-05-12 12:37 Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
                   ` (31 more replies)
  0 siblings, 32 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

This patchset introduces a concept of runtime, which is described well
enough in the first patch of the series and converts all tests that were
previously using timeout API to runtime API.

The main change to the v1 of the patchset is that the runtime is now per
iteration, e.g. if we run the test for all_filesystem the runtime
applies to a single filesystem only. And at the same time the overall
test timeout is a sum of static timeout and runtime. The static part of
the timeout is meant to cover test teardown and any inaccuracy in the
test runtime accounting.

The patches that convert tests to runtime API fall into these classes:

- fzsync tests
- faster timeout tests
- I/O tests
- timer precision tests
- oom tests
- misc tests that run for some time (controlled by command line
  parameter)

fzsync tests
============

These tests previously defined their runtime based on scale factor that
used slice of the default timeout (300s) for the test runtime.

Now the runtime is explicitly encoded in the test, which I find much
more user friendly because it's clear how long will the test run.

faster timeout tests
====================

There were several tests that set low timeout becuase the test may lock
up. In these cases the timeout was simply removed because the default
value (30s) should be small enough.

I/O tests
=========

The aiodio tests were modified so that their runtime is capped to
30 minutes by default. That means that even on very slow machine they
finish in a reasonable time.

timer precision tests
=====================

These test simply set the test timeout to be 110% of the sum of the
sleep_time * samples. That is to accomodate for the context switches and
upkeeping between the calls in the timer library.


oom tests
=========

Here we just set the .max_iteration_runtime to TST_RUNTIME_UNLIMITED,
which has the very same effect as .timeout = -1 previously had.

misc test
=========

In this case the test code was simplified quite a lot since the runtime
is now passed in a unified way and parsed in the test library.

v3:
   - renamed max_iteration_runtime to just max_runtime
     and tst_set_runtime() to tst_set_max_runtime()
   - fixed newlib_tests and CI runtest.sh
   - merged the two patches for af_alg02
   - added and fixed documentation
   - fixed a few bugs in the tst_test.c
     (LTP_RUNTIME_MUL etc.)

Cyril Hrubis (29):
  Introduce a concept of max runtime
  mtest06/mmap1: Convert to runtime
  mtest06/mmap3: Convert to runtime
  mtest01/mtest01: Convert to runtime
  cve/cve-2015-3290: Convert to runtime
  crypto/af_alg02: Convert to runtime.
  crypto/pcrypt_aead01: Convert to runtime
  syscalls/clock_gettime01: Remove useless timeout
  syscalls/fanotify22: Remove useless timeout
  syscalls/gettimeofday02: Convert to runtime.
  syscalls/inotify06: Convert to runtime.
  syscalls/inotify01: Remove now useless timeout
  syscalls/perf_event_open03: Convert to runtime
  syscalls/readv01: Remove now useless timeout
  syscalls/tgkill03: Remove now unused timeout
  syscalls/setsockopt09: Remove now useless timeout
  syscalls/userfaultfd01: Remove now useless timeout
  syscalls/move_pages12: Convert to runtime
  syscalls/rt_sigqueueinfo01: Remove now useless timeout
  mem/mallocstress: Convert to runtime
  mem/{oom,min_free_kbytes}: Convert to runtime
  fuzzy_sync: Convert to runtime
  ltp-aiodio/dio_sparse,aiodio_sparse: Convert to runtime.
  ltp-aiodio/read_checkzero: Remove
  ltp-aiodio/dio_{truncate,append}: Convert to runtime
  ltp-aiodio/dio_read: Convert to runtime
  timer_test: Convert to runtime
  ltp-aiodio/aiodio_append: Convert to runtime
  tst_test: Remove timeout stubs

 doc/c-test-api.txt                            |  42 +++--
 doc/user-guide.txt                            |  30 +++-
 include/tst_fuzzy_sync.h                      |  19 +--
 include/tst_test.h                            |  31 +++-
 lib/newlib_tests/.gitignore                   |   5 +-
 lib/newlib_tests/runtest.sh                   |   2 +-
 lib/newlib_tests/test10.c                     |  22 ---
 lib/newlib_tests/test12.c                     |  21 ---
 lib/newlib_tests/test13.c                     |   1 -
 lib/newlib_tests/test18.c                     |  22 ---
 lib/newlib_tests/test_children_cleanup.c      |   1 -
 lib/newlib_tests/test_runtime01.c             |  30 ++++
 lib/newlib_tests/test_runtime02.c             |  28 ++++
 lib/newlib_tests/tst_fuzzy_sync01.c           |   1 +
 lib/newlib_tests/tst_fuzzy_sync02.c           |   1 +
 lib/newlib_tests/tst_fuzzy_sync03.c           |   1 +
 lib/tst_test.c                                | 143 +++++++++++++-----
 lib/tst_timer_test.c                          |  12 +-
 runtest/ltp-aiodio.part4                      |   2 -
 testcases/cve/cve-2015-3290.c                 |   8 +-
 testcases/kernel/crypto/af_alg02.c            |   4 +-
 testcases/kernel/crypto/af_alg07.c            |   1 +
 testcases/kernel/crypto/pcrypt_aead01.c       |   3 +-
 testcases/kernel/io/ltp-aiodio/.gitignore     |   1 -
 .../kernel/io/ltp-aiodio/aiodio_append.c      |  25 ++-
 .../kernel/io/ltp-aiodio/aiodio_sparse.c      |  26 +++-
 testcases/kernel/io/ltp-aiodio/common.h       |  12 +-
 testcases/kernel/io/ltp-aiodio/dio_append.c   |   4 +
 testcases/kernel/io/ltp-aiodio/dio_read.c     |   9 ++
 testcases/kernel/io/ltp-aiodio/dio_sparse.c   |   6 +-
 testcases/kernel/io/ltp-aiodio/dio_truncate.c |   6 +
 .../kernel/io/ltp-aiodio/read_checkzero.c     |  84 ----------
 testcases/kernel/mem/mtest01/mtest01.c        |  14 +-
 testcases/kernel/mem/mtest06/mmap1.c          |  24 +--
 testcases/kernel/mem/mtest06/mmap3.c          |  12 +-
 testcases/kernel/mem/mtest07/mallocstress.c   |  11 +-
 testcases/kernel/mem/oom/oom01.c              |   2 +-
 testcases/kernel/mem/oom/oom02.c              |   2 +-
 testcases/kernel/mem/oom/oom03.c              |   2 +-
 testcases/kernel/mem/oom/oom04.c              |   2 +-
 testcases/kernel/mem/oom/oom05.c              |   2 +-
 testcases/kernel/mem/thp/thp04.c              |   1 +
 .../kernel/mem/tunable/min_free_kbytes.c      |   2 +-
 testcases/kernel/pty/pty03.c                  |   2 +-
 testcases/kernel/pty/pty05.c                  |   1 +
 testcases/kernel/pty/pty06.c                  |   1 +
 testcases/kernel/pty/pty07.c                  |   1 +
 testcases/kernel/sound/snd_seq01.c            |   2 +-
 testcases/kernel/sound/snd_timer01.c          |   1 +
 testcases/kernel/syscalls/bind/bind06.c       |   2 +-
 .../syscalls/clock_gettime/clock_gettime01.c  |   1 -
 .../kernel/syscalls/fanotify/fanotify22.c     |   1 -
 .../syscalls/gettimeofday/gettimeofday02.c    |  15 +-
 testcases/kernel/syscalls/inotify/inotify06.c |   7 +-
 testcases/kernel/syscalls/inotify/inotify09.c |   1 +
 testcases/kernel/syscalls/inotify/inotify11.c |   1 -
 .../kernel/syscalls/ipc/shmctl/shmctl05.c     |   2 +-
 .../kernel/syscalls/move_pages/move_pages12.c |   4 +-
 .../perf_event_open/perf_event_open03.c       |  15 +-
 testcases/kernel/syscalls/readv/readv01.c     |   1 -
 .../rt_sigqueueinfo/rt_sigqueueinfo01.c       |   1 -
 testcases/kernel/syscalls/sendmsg/sendmsg03.c |   1 +
 .../kernel/syscalls/setsockopt/setsockopt06.c |   2 +-
 .../kernel/syscalls/setsockopt/setsockopt07.c |   1 +
 .../kernel/syscalls/setsockopt/setsockopt09.c |   1 -
 testcases/kernel/syscalls/tgkill/tgkill03.c   |   1 -
 .../syscalls/timerfd/timerfd_settime02.c      |   1 +
 .../syscalls/userfaultfd/userfaultfd01.c      |   1 -
 testcases/kernel/syscalls/writev/writev03.c   |   2 +-
 69 files changed, 419 insertions(+), 327 deletions(-)
 delete mode 100644 lib/newlib_tests/test10.c
 delete mode 100644 lib/newlib_tests/test12.c
 delete mode 100644 lib/newlib_tests/test18.c
 create mode 100644 lib/newlib_tests/test_runtime01.c
 create mode 100644 lib/newlib_tests/test_runtime02.c
 delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c

-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 01/29] Introduce a concept of max runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-14  4:21   ` Li Wang
  2022-05-24  9:25   ` [LTP] [Automated-testing] " Richard Palethorpe
  2022-05-12 12:37 ` [LTP] [PATCH v3 02/29] mtest06/mmap1: Convert to runtime Cyril Hrubis
                   ` (30 subsequent siblings)
  31 siblings, 2 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

This commit introduces a concept of max test runtime. In other words
test runtime is capped at a certain value in order to make testruns more
deterministic. Test is free to to finish before the runtime is used up,
for example when maximal number of iterations was reached, but test must
stop once the runtime has been used up.

Testcases that run for more than a second or two must check for
remaining runtime by regular calls to tst_remaining_runtime() and should
exit when zero is returned.

The test max runtime must be set either by the .max_iteration_runtime in
the tst_test structure or in the test setup by a call to
tst_set_runtime().

The test timeout is then computed as a sum of DEFAULT_TIMEOUT (currently
set to 30 seconds) and the test runtime. The DEFAULT_TIMEOUT is nothing
more than a safety margin for teardown of the test.

This commit also maps the -I parameter to the test max runtime if
available and introduces LTP_RUNTIME_MUL enviroment variable so that we
have an easy controll over the runtime cap.

Lastly but not least the function related to the timeout are turned into
no-op by this commit and removed after all test are converted to the
runtime API.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 doc/c-test-api.txt                       |  42 +++++--
 doc/user-guide.txt                       |  30 ++++-
 include/tst_test.h                       |  27 ++++
 lib/newlib_tests/.gitignore              |   5 +-
 lib/newlib_tests/runtest.sh              |   2 +-
 lib/newlib_tests/test10.c                |  22 ----
 lib/newlib_tests/test12.c                |  21 ----
 lib/newlib_tests/test13.c                |   1 -
 lib/newlib_tests/test18.c                |  22 ----
 lib/newlib_tests/test_children_cleanup.c |   1 -
 lib/newlib_tests/test_runtime01.c        |  30 +++++
 lib/newlib_tests/test_runtime02.c        |  28 +++++
 lib/tst_test.c                           | 150 +++++++++++++++++------
 13 files changed, 264 insertions(+), 117 deletions(-)
 delete mode 100644 lib/newlib_tests/test10.c
 delete mode 100644 lib/newlib_tests/test12.c
 delete mode 100644 lib/newlib_tests/test18.c
 create mode 100644 lib/newlib_tests/test_runtime01.c
 create mode 100644 lib/newlib_tests/test_runtime02.c

diff --git a/doc/c-test-api.txt b/doc/c-test-api.txt
index 9f104ecd7..711b445d9 100644
--- a/doc/c-test-api.txt
+++ b/doc/c-test-api.txt
@@ -93,14 +93,35 @@ in range of [0, '.tcnt' - 1].
 
 IMPORTANT: Only one of '.test' and '.test_all' can be set at a time.
 
-Each test has a default timeout set to 300s. The default timeout can be
-overridden by setting '.timeout' in the test structure or by calling
-'tst_set_timeout()' in the test 'setup()'. There are a few testcases whose run
-time may vary arbitrarily, for these timeout can be disabled by setting it to
--1.
+Each test has a limit on how long it can run and the limit composes of two
+parts max_runtime and timeout. The max_runtime is a limit for how long can the
+'.test_all' or a set of '.test' functions take and the timeout is static part
+that should cover the duration of test setup and cleanup plus some safety.
 
-Test can find out how much time (in seconds) is remaining to timeout,
-by calling 'tst_timeout_remaining()'.
+Any test that runs for more than a second or two has to make sure to:
+
+- set the runtime either by setting the '.max_runtime' in tst_test or by
+  calling 'tst_set_runtime()' in the test setup
+
+- monitor remaning runtime by regular calls to 'tst_remaining_runtime()' and
+  exit when runtime has been used up
+
+Test is free to exit before max_runtime has been used up for example when
+minimal number of iteration was finished.
+
+The limit is applied to a single call of the '.test_all' function that means
+that for example when '.test_variants' or '.all_filesystems' is set the whole
+test will be limited by 'variants * (max_runtime + timeout)' seconds and the
+test runtime will be likely close to 'variants * max_runtime' seconds.
+
+[source,c]
+-------------------------------------------------------------------------------
+/*
+ * Returns number of seconds or zero in case that runtime has been used up.
+ */
+
+int tst_remaining_runtime(void);
+-------------------------------------------------------------------------------
 
 LAPI headers
 ++++++++++++
@@ -377,12 +398,13 @@ WARNING: This function is not thread safe.
 
 [source,c]
 -------------------------------------------------------------------------------
-void tst_set_timeout(unsigned int timeout);
+void tst_set_max_runtime(int max_runtime);
 -------------------------------------------------------------------------------
 
-Allows for setting timeout per test iteration dynamically in the test setup(),
+Allows for setting max_runtime per test iteration dynamically in the test setup(),
 the timeout is specified in seconds. There are a few testcases whose runtime
-can vary arbitrarily, these can disable timeouts by setting it to -1.
+can vary arbitrarily, these can disable timeouts by setting it to
+TST_UNLIMITED_RUNTIME.
 
 [source,c]
 -------------------------------------------------------------------------------
diff --git a/doc/user-guide.txt b/doc/user-guide.txt
index f41cbc733..96c9800ed 100644
--- a/doc/user-guide.txt
+++ b/doc/user-guide.txt
@@ -22,9 +22,13 @@ For running LTP network tests see `testcases/network/README.md`.
 | 'LTP_SINGLE_FS_TYPE'  | Testing only - specifies filesystem instead all
                           supported (for tests with '.all_filesystems').
 | 'LTP_DEV_FS_TYPE'     | Filesystem used for testing (default: 'ext2').
-| 'LTP_TIMEOUT_MUL'     | Multiply timeout, must be number >= 1 (> 1 is useful for
+| 'LTP_TIMEOUT_MUL'     | Multiplies timeout, must be number >= 0.1 (> 1 is useful for
                           slow machines to avoid unexpected timeout).
                           Variable is also used in shell tests, but ceiled to int.
+| 'LTP_RUNTIME_MUL'     | Multiplies maximal test iteration runtime. Tests
+                          that run for more than a second or two are capped on
+			  runtime. You can scale the default runtime both up
+			  and down with this multiplier.
 | 'LTP_VIRT_OVERRIDE'   | Overrides virtual machine detection in the test
                           library. Setting it to empty string tell the library
                           that system is not a virtual machine. Other possible
@@ -36,3 +40,27 @@ For running LTP network tests see `testcases/network/README.md`.
                           and others, which imply it, shell: 'TST_NEEDS_TMPDIR=1').
 | 'TST_NO_CLEANUP'      | Disable running test cleanup (defined in 'TST_CLEANUP').
 |==============================================================================
+
+
+2. Test execution time and timeout
+----------------------------------
+
+The limit on how long a test can run does compose of two parts max_runtime and
+timeout. The limit does apply to a single test variant, that means for example
+that tests that run for all available filesystems will apply this limit for a
+single filesytem only.
+
+The max_runtime is a cap on how long the run() function can take, for most
+testcases this part is set to zero. For tests that do run for more than a
+second or two the max_runtime has to be defined and the run() function has to
+check actively how much runtime is left.
+
+Test runtime can be scaled up and down with 'LTP_RUNTIME_MUL' environment
+variable or set on a commandline by the '-I' parameter. Hoewever be vary that
+setting the runtime too low will cause long running tests to exit prematurely
+possibly before the have a chance actually test anyting.
+
+The timeout part is a limit for the test setup and cleanup and also safety
+margin for the runtime accounting. It's currently set to 30 seconds but may
+change later. If your target machine is too slow it can be scaled up with the
+'LTP_TIMEOUT_MUL' environment variable.
diff --git a/include/tst_test.h b/include/tst_test.h
index dbe303bc8..01be56cd8 100644
--- a/include/tst_test.h
+++ b/include/tst_test.h
@@ -134,6 +134,8 @@ extern unsigned int tst_variant;
 
 #define TST_NO_HUGEPAGES ((unsigned long)-1)
 
+#define TST_UNLIMITED_RUNTIME (-1)
+
 struct tst_test {
 	/* number of tests available in test() function */
 	unsigned int tcnt;
@@ -236,6 +238,18 @@ struct tst_test {
 
 	/* override default timeout per test run, disabled == -1 */
 	int timeout;
+	/*
+	 * Maximal test runtime in seconds.
+	 *
+	 * Any test that runs for more than a second or two should set this and
+	 * also use tst_remaining_runtime() to exit when runtime was used up.
+	 * Tests may finish sooner, for example if requested number of
+	 * iterations was reached before the runtime runs out.
+	 *
+	 * If test runtime cannot be know in advance it should be set to
+	 * TST_UNLIMITED_RUNTIME.
+	 */
+	int max_runtime;
 
 	void (*setup)(void);
 	void (*cleanup)(void);
@@ -323,6 +337,19 @@ unsigned int tst_timeout_remaining(void);
 unsigned int tst_multiply_timeout(unsigned int timeout);
 void tst_set_timeout(int timeout);
 
+/*
+ * Returns remaining test runtime. Test that runs for more than a few seconds
+ * should check if they should exit by calling this function regularly.
+ *
+ * The function returns remaining runtime in seconds. If runtime was used up
+ * zero is returned.
+ */
+unsigned int tst_remaining_runtime(void);
+
+/*
+ * Sets maximal test runtime in seconds.
+ */
+void tst_set_max_runtime(int max_runtime);
 
 /*
  * Returns path to the test temporary directory in a newly allocated buffer.
diff --git a/lib/newlib_tests/.gitignore b/lib/newlib_tests/.gitignore
index f4414f6a1..59b57d063 100644
--- a/lib/newlib_tests/.gitignore
+++ b/lib/newlib_tests/.gitignore
@@ -7,9 +7,7 @@ test06
 test07
 test08
 test09
-test10
 test11
-test12
 test13
 test14
 test15
@@ -22,7 +20,6 @@ tst_safe_fileops
 tst_res_hexd
 tst_strstatus
 tst_print_result
-test18
 test19
 test20
 test22
@@ -56,3 +53,5 @@ tst_needs_cmds05
 tst_needs_cmds06
 tst_needs_cmds07
 tst_needs_cmds08
+test_runtime01
+test_runtime02
diff --git a/lib/newlib_tests/runtest.sh b/lib/newlib_tests/runtest.sh
index 327460e7b..f136bcb88 100755
--- a/lib/newlib_tests/runtest.sh
+++ b/lib/newlib_tests/runtest.sh
@@ -1,7 +1,7 @@
 #!/bin/sh
 # Copyright (c) 2021 Petr Vorel <pvorel@suse.cz>
 
-LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test12 test15 test18
+LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test15 test_runtime01
 tst_needs_cmds01 tst_needs_cmds02 tst_needs_cmds03 tst_needs_cmds06
 tst_needs_cmds07 tst_bool_expr test_exec test_timer tst_res_hexd tst_strstatus
 tst_fuzzy_sync03 test_zero_hugepage.sh test_kconfig.sh
diff --git a/lib/newlib_tests/test10.c b/lib/newlib_tests/test10.c
deleted file mode 100644
index df61908ac..000000000
--- a/lib/newlib_tests/test10.c
+++ /dev/null
@@ -1,22 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Copyright (c) 2016 Linux Test Project
- */
-
-/*
- * Test for watchdog timeout.
- */
-
-#include "tst_test.h"
-
-
-static void do_test(void)
-{
-	sleep(2);
-	tst_res(TPASS, "Not reached");
-}
-
-static struct tst_test test = {
-	.test_all = do_test,
-	.timeout = 1,
-};
diff --git a/lib/newlib_tests/test12.c b/lib/newlib_tests/test12.c
deleted file mode 100644
index b4f0d6303..000000000
--- a/lib/newlib_tests/test12.c
+++ /dev/null
@@ -1,21 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Copyright (c) 2016 Linux Test Project
- */
-
-/*
- * Test for timeout override.
- */
-
-#include "tst_test.h"
-
-static void do_test(void)
-{
-	sleep(1);
-	tst_res(TPASS, "Passed!");
-}
-
-static struct tst_test test = {
-	.timeout = 2,
-	.test_all = do_test,
-};
diff --git a/lib/newlib_tests/test13.c b/lib/newlib_tests/test13.c
index c447dc3dc..83c48f734 100644
--- a/lib/newlib_tests/test13.c
+++ b/lib/newlib_tests/test13.c
@@ -20,7 +20,6 @@ static void do_test(void)
 }
 
 static struct tst_test test = {
-	.timeout = 1,
 	.forks_child = 1,
 	.test_all = do_test,
 };
diff --git a/lib/newlib_tests/test18.c b/lib/newlib_tests/test18.c
deleted file mode 100644
index 026435d7d..000000000
--- a/lib/newlib_tests/test18.c
+++ /dev/null
@@ -1,22 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Copyright (c) 2018, Linux Test Project
- */
-
-#include <stdlib.h>
-#include <unistd.h>
-#include "tst_test.h"
-
-static void run(void)
-{
-	do {
-		sleep(1);
-	} while (tst_timeout_remaining() >= 4);
-
-	tst_res(TPASS, "Timeout remaining: %d", tst_timeout_remaining());
-}
-
-static struct tst_test test = {
-	.test_all = run,
-	.timeout = 5
-};
diff --git a/lib/newlib_tests/test_children_cleanup.c b/lib/newlib_tests/test_children_cleanup.c
index 2b1ca5f9c..4a1313f6d 100644
--- a/lib/newlib_tests/test_children_cleanup.c
+++ b/lib/newlib_tests/test_children_cleanup.c
@@ -39,5 +39,4 @@ static void run(void)
 static struct tst_test test = {
 	.test_all = run,
 	.forks_child = 1,
-	.timeout = 10,
 };
diff --git a/lib/newlib_tests/test_runtime01.c b/lib/newlib_tests/test_runtime01.c
new file mode 100644
index 000000000..5e027546d
--- /dev/null
+++ b/lib/newlib_tests/test_runtime01.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2018, Linux Test Project
+ *
+ * Runs for 4 seconds for each test iteration.
+ */
+#include <stdlib.h>
+#include <unistd.h>
+#include "tst_test.h"
+
+static void run(void)
+{
+	int runtime;
+
+	tst_res(TINFO, "Running variant %i", tst_variant);
+
+	do {
+		runtime = tst_remaining_runtime();
+		tst_res(TINFO, "Remaining runtime %d", runtime);
+		sleep(1);
+	} while (runtime);
+
+	tst_res(TPASS, "Finished loop!");
+}
+
+static struct tst_test test = {
+	.test_all = run,
+	.max_runtime = 4,
+	.test_variants = 2,
+};
diff --git a/lib/newlib_tests/test_runtime02.c b/lib/newlib_tests/test_runtime02.c
new file mode 100644
index 000000000..6d89cb531
--- /dev/null
+++ b/lib/newlib_tests/test_runtime02.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021, Linux Test Project
+ */
+/*
+ * This test is set up so that the timeout is not long enough to guarantee
+ * enough runtime for two iterations, i.e. the timeout without offset and after
+ * scaling is too small and the tests ends up with TBROK.
+ *
+ * The default timeout in the test library is set to 30 seconds. The test
+ * runtime is set to 5 so the test should timeout after 35 seconds.
+ */
+
+#include <stdlib.h>
+#include <unistd.h>
+#include "tst_test.h"
+
+static void run(void)
+{
+	tst_res(TINFO, "Sleeping for 40 seconds");
+	sleep(40);
+	tst_res(TFAIL, "Still alive");
+}
+
+static struct tst_test test = {
+	.test_all = run,
+	.max_runtime = 5,
+};
diff --git a/lib/tst_test.c b/lib/tst_test.c
index 8e258594a..ef2e614bb 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -45,6 +45,8 @@ const char *TCID __attribute__((weak));
 #define GLIBC_GIT_URL "https://sourceware.org/git/?p=glibc.git;a=commit;h="
 #define CVE_DB_URL "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-"
 
+#define DEFAULT_TIMEOUT 30
+
 struct tst_test *tst_test;
 
 static const char *tid;
@@ -63,6 +65,7 @@ struct results {
 	int warnings;
 	int broken;
 	unsigned int timeout;
+	int max_runtime;
 };
 
 static struct results *results;
@@ -464,6 +467,40 @@ pid_t safe_clone(const char *file, const int lineno,
 	return pid;
 }
 
+static void parse_mul(float *mul, const char *env_name, float min, float max)
+{
+	char *str_mul;
+	int ret;
+
+	if (*mul > 0)
+		return;
+
+	str_mul = getenv(env_name);
+
+	if (!str_mul) {
+		*mul = 1;
+		return;
+	}
+
+	ret = tst_parse_float(str_mul, mul, min, max);
+	if (ret) {
+		tst_brk(TBROK, "Failed to parse %s: %s",
+			env_name, tst_strerrno(ret));
+	}
+}
+
+static int multiply_runtime(int max_runtime)
+{
+	static float runtime_mul = -1;
+
+	if (max_runtime <= 0)
+		return max_runtime;
+
+	parse_mul(&runtime_mul, "LTP_RUNTIME_MUL", 0.0099, 100);
+
+	return max_runtime * runtime_mul;
+}
+
 static struct option {
 	char *optstr;
 	char *help;
@@ -477,6 +514,7 @@ static struct option {
 static void print_help(void)
 {
 	unsigned int i;
+	int timeout, runtime;
 
 	/* see doc/user-guide.txt, which lists also shell API variables */
 	fprintf(stderr, "Environment Variables\n");
@@ -489,10 +527,32 @@ static void print_help(void)
 	fprintf(stderr, "LTP_DEV_FS_TYPE      Filesystem used for testing (default: %s)\n", DEFAULT_FS_TYPE);
 	fprintf(stderr, "LTP_SINGLE_FS_TYPE   Testing only - specifies filesystem instead all supported (for .all_filesystems)\n");
 	fprintf(stderr, "LTP_TIMEOUT_MUL      Timeout multiplier (must be a number >=1)\n");
+	fprintf(stderr, "LTP_RUNTIME_MUL      Runtime multiplier (must be a number >=1)\n");
 	fprintf(stderr, "LTP_VIRT_OVERRIDE    Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n");
 	fprintf(stderr, "TMPDIR               Base directory for template directory (for .needs_tmpdir, default: %s)\n", TEMPDIR);
 	fprintf(stderr, "\n");
 
+	fprintf(stderr, "Timeout and runtime\n");
+	fprintf(stderr, "-------------------\n");
+
+	if (tst_test->max_runtime) {
+		runtime = multiply_runtime(tst_test->max_runtime);
+
+		if (runtime == TST_UNLIMITED_RUNTIME) {
+			fprintf(stderr, "Test iteration runtime is not limited\n");
+		} else {
+			fprintf(stderr, "Test iteration runtime cap %ih %im %is\n",
+				runtime/3600, (runtime%3600)/60, runtime % 60);
+		}
+	}
+
+	timeout = tst_multiply_timeout(DEFAULT_TIMEOUT);
+
+	fprintf(stderr, "Test timeout (not including runtime) %ih %im %is\n",
+		timeout/3600, (timeout%3600)/60, timeout % 60);
+
+	fprintf(stderr, "\n");
+
 	fprintf(stderr, "Options\n");
 	fprintf(stderr, "-------\n");
 
@@ -620,7 +680,10 @@ static void parse_opts(int argc, char *argv[])
 			iterations = atoi(optarg);
 		break;
 		case 'I':
-			duration = atof(optarg);
+			if (tst_test->max_runtime > 0)
+				tst_test->max_runtime = atoi(optarg);
+			else
+				duration = atof(optarg);
 		break;
 		case 'C':
 #ifdef UCLINUX
@@ -1034,6 +1097,11 @@ static void do_setup(int argc, char *argv[])
 	if (!tst_test)
 		tst_brk(TBROK, "No tests to run");
 
+	if (tst_test->max_runtime < -1) {
+		tst_brk(TBROK, "Invalid runtime value %i",
+			results->max_runtime);
+	}
+
 	if (tst_test->tconf_msg)
 		tst_brk(TCONF, "%s", tst_test->tconf_msg);
 
@@ -1404,39 +1472,36 @@ static void sigint_handler(int sig LTP_ATTRIBUTE_UNUSED)
 }
 
 unsigned int tst_timeout_remaining(void)
+{
+	tst_brk(TBROK, "Stub called!");
+	return 0;
+}
+
+unsigned int tst_remaining_runtime(void)
 {
 	static struct timespec now;
-	unsigned int elapsed;
+	int elapsed;
+
+	if (results->max_runtime == TST_UNLIMITED_RUNTIME)
+		return UINT_MAX;
+
+	if (results->max_runtime == 0)
+		tst_brk(TBROK, "Runtime not set!");
 
 	if (tst_clock_gettime(CLOCK_MONOTONIC, &now))
 		tst_res(TWARN | TERRNO, "tst_clock_gettime() failed");
 
-	elapsed = (tst_timespec_diff_ms(now, tst_start_time) + 500) / 1000;
-	if (results->timeout > elapsed)
-		return results->timeout - elapsed;
+	elapsed = tst_timespec_diff_ms(now, tst_start_time) / 1000;
+	if (results->max_runtime > elapsed)
+		return results->max_runtime - elapsed;
 
 	return 0;
 }
 
+
 unsigned int tst_multiply_timeout(unsigned int timeout)
 {
-	char *mul;
-	int ret;
-
-	if (timeout_mul == -1) {
-		mul = getenv("LTP_TIMEOUT_MUL");
-		if (mul) {
-			if ((ret = tst_parse_float(mul, &timeout_mul, 1, 10000))) {
-				tst_brk(TBROK, "Failed to parse LTP_TIMEOUT_MUL: %s",
-						tst_strerrno(ret));
-			}
-		} else {
-			timeout_mul = 1;
-		}
-	}
-	if (timeout_mul < 1)
-		tst_brk(TBROK, "LTP_TIMEOUT_MUL must to be int >= 1! (%.2f)",
-				timeout_mul);
+	parse_mul(&timeout_mul, "LTP_TIMEOUT_MUL", 0.099, 10000);
 
 	if (timeout < 1)
 		tst_brk(TBROK, "timeout must to be >= 1! (%d)", timeout);
@@ -1446,37 +1511,47 @@ unsigned int tst_multiply_timeout(unsigned int timeout)
 
 void tst_set_timeout(int timeout)
 {
-	if (timeout == -1) {
+	tst_brk(TBROK, "Stub called!");
+}
+
+static void set_timeout(void)
+{
+	unsigned int timeout = DEFAULT_TIMEOUT;
+
+	if (results->max_runtime == TST_UNLIMITED_RUNTIME) {
 		tst_res(TINFO, "Timeout per run is disabled");
 		return;
 	}
 
-	if (timeout < 1)
-		tst_brk(TBROK, "timeout must to be >= 1! (%d)", timeout);
+	if (results->max_runtime < 0) {
+		tst_brk(TBROK, "max_runtime must to be >= -1! (%d)",
+			results->max_runtime);
+	}
 
-	results->timeout = tst_multiply_timeout(timeout);
+	results->timeout = tst_multiply_timeout(timeout) + results->max_runtime;
 
 	tst_res(TINFO, "Timeout per run is %uh %02um %02us",
 		results->timeout/3600, (results->timeout%3600)/60,
 		results->timeout % 60);
+}
 
-	if (getpid() == lib_pid)
-		alarm(results->timeout);
-	else
-		heartbeat();
+void tst_set_max_runtime(int max_runtime)
+{
+	results->max_runtime = multiply_runtime(max_runtime);
+	tst_res(TINFO, "Updating max runtime to %uh %02um %02us",
+		max_runtime/3600, (max_runtime%3600)/60, max_runtime % 60);
+	set_timeout();
+	heartbeat();
 }
 
 static int fork_testrun(void)
 {
 	int status;
 
-	if (tst_test->timeout)
-		tst_set_timeout(tst_test->timeout);
-	else
-		tst_set_timeout(300);
-
 	SAFE_SIGNAL(SIGINT, sigint_handler);
 
+	alarm(results->timeout);
+
 	test_pid = fork();
 	if (test_pid < 0)
 		tst_brk(TBROK | TERRNO, "fork()");
@@ -1568,6 +1643,11 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)
 	SAFE_SIGNAL(SIGALRM, alarm_handler);
 	SAFE_SIGNAL(SIGUSR1, heartbeat_handler);
 
+	if (tst_test->max_runtime)
+		results->max_runtime = multiply_runtime(tst_test->max_runtime);
+
+	set_timeout();
+
 	if (tst_test->test_variants)
 		test_variants = tst_test->test_variants;
 
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 02/29] mtest06/mmap1: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 03/29] mtest06/mmap3: " Cyril Hrubis
                   ` (29 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/mem/mtest06/mmap1.c | 24 ++++++------------------
 1 file changed, 6 insertions(+), 18 deletions(-)

diff --git a/testcases/kernel/mem/mtest06/mmap1.c b/testcases/kernel/mem/mtest06/mmap1.c
index 10c47c35c..6726f11dd 100644
--- a/testcases/kernel/mem/mtest06/mmap1.c
+++ b/testcases/kernel/mem/mtest06/mmap1.c
@@ -42,10 +42,8 @@
 
 static int file_size = 1024;
 static int num_iter = 5000;
-static float exec_time = 0.05; /* default is 3 min */
 
 static void *distant_area;
-static char *str_exec_time;
 static jmp_buf jmpbuf;
 static volatile unsigned char *map_address;
 static unsigned long page_sz;
@@ -206,17 +204,10 @@ static void setup(void)
 	SAFE_MUNMAP(distant_area, distant_mmap_size);
 	distant_area += distant_mmap_size / 2;
 
-	if (tst_parse_float(str_exec_time, &exec_time, 0, FLT_MAX)) {
-		tst_brk(TBROK, "Invalid number for exec_time '%s'",
-			str_exec_time);
-	}
-
 	sigptr.sa_sigaction = sig_handler;
 	sigemptyset(&sigptr.sa_mask);
 	sigptr.sa_flags = SA_SIGINFO | SA_NODEFER;
 	SAFE_SIGACTION(SIGSEGV, &sigptr, NULL);
-
-	tst_set_timeout((int)(exec_time * 3600));
 }
 
 static void run(void)
@@ -224,8 +215,8 @@ static void run(void)
 	pthread_t thid[2];
 	int start, last_update;
 
-	start = last_update = tst_timeout_remaining();
-	while (tst_timeout_remaining() > STOP_THRESHOLD) {
+	start = last_update = tst_remaining_runtime();
+	while (tst_remaining_runtime()) {
 		int fd = mkfile(file_size);
 
 		tst_atomic_store(0, &mapcnt);
@@ -240,11 +231,11 @@ static void run(void)
 
 		close(fd);
 
-		if (last_update - tst_timeout_remaining() >= PROGRESS_SEC) {
-			last_update = tst_timeout_remaining();
+		if (last_update - tst_remaining_runtime() >= PROGRESS_SEC) {
+			last_update = tst_remaining_runtime();
 			tst_res(TINFO, "[%03d] mapped: %lu, sigsegv hit: %lu, "
 				"threads spawned: %lu",
-				start - tst_timeout_remaining(),
+				start - last_update,
 				map_count, mapped_sigsegv_count,
 				threads_spawned);
 			tst_res(TINFO, "      repeated_reads: %ld, "
@@ -258,9 +249,6 @@ static void run(void)
 static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
-	.options = (struct tst_option[]) {
-		{"x:", &str_exec_time, "Exec time (hours)"},
-		{}
-	},
+	.max_runtime = 180,
 	.needs_tmpdir = 1,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 03/29] mtest06/mmap3: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 02/29] mtest06/mmap1: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 04/29] mtest01/mtest01: " Cyril Hrubis
                   ` (28 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/mem/mtest06/mmap3.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/testcases/kernel/mem/mtest06/mmap3.c b/testcases/kernel/mem/mtest06/mmap3.c
index 23609752f..6d47b01e7 100644
--- a/testcases/kernel/mem/mtest06/mmap3.c
+++ b/testcases/kernel/mem/mtest06/mmap3.c
@@ -23,11 +23,9 @@
 static char *str_loops;
 static char *str_threads;
 static char *map_private;
-static char *str_exec_time;
 
 static int loops = 1000;
 static int threads = 40;
-static float exec_time = 24;
 
 static volatile int sig_caught;
 static int threads_running;
@@ -109,7 +107,7 @@ static void test_mmap(void)
 	long i;
 	pthread_t thids[threads];
 
-	alarm(exec_time * 3600);
+	alarm(tst_remaining_runtime());
 
 	while (!sig_caught) {
 		for (i = 0; i < threads; i++) {
@@ -138,11 +136,6 @@ static void setup(void)
 	if (tst_parse_int(str_threads, &threads, 1, INT_MAX))
 		tst_brk(TBROK, "Invalid number of threads '%s'", str_threads);
 
-	if (tst_parse_float(str_exec_time, &exec_time, 0.0005, 9000))
-		tst_brk(TBROK, "Invalid execution time '%s'", str_exec_time);
-
-	tst_set_timeout(exec_time * 3600 + 300);
-
 	SAFE_SIGNAL(SIGALRM, sig_handler);
 	SAFE_SIGNAL(SIGBUS, sig_handler);
 	SAFE_SIGNAL(SIGSEGV, sig_handler);
@@ -155,7 +148,6 @@ static void setup(void)
 	tst_res(TINFO, "Number of loops %i", loops);
 	tst_res(TINFO, "Number of threads %i", threads);
 	tst_res(TINFO, "MAP_PRIVATE = %i", map_private ? 1 : 0);
-	tst_res(TINFO, "Execution time %fH", exec_time);
 }
 
 static void cleanup(void)
@@ -184,11 +176,11 @@ static struct tst_test test = {
 		{"l:", &str_loops, "Number of map-write-unmap loops"},
 		{"n:", &str_threads, "Number of worker threads"},
 		{"p", &map_private, "Turns on MAP_PRIVATE (default MAP_SHARED)"},
-		{"x:", &str_exec_time, "float Execution time in hours (default 24H)"},
 		{}
 	},
 	.needs_tmpdir = 1,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = test_mmap,
+	.max_runtime = 86400,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 04/29] mtest01/mtest01: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (2 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 03/29] mtest06/mmap3: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 05/29] cve/cve-2015-3290: " Cyril Hrubis
                   ` (27 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/mem/mtest01/mtest01.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/testcases/kernel/mem/mtest01/mtest01.c b/testcases/kernel/mem/mtest01/mtest01.c
index 20cb95191..fb991ce8b 100644
--- a/testcases/kernel/mem/mtest01/mtest01.c
+++ b/testcases/kernel/mem/mtest01/mtest01.c
@@ -41,8 +41,6 @@
 #define ALLOC_THRESHOLD		(6*FIVE_HUNDRED_MB)
 #endif
 
-#define STOP_THRESHOLD 15	/* seconds remaining before reaching timeout */
-
 static pid_t *pid_list;
 static sig_atomic_t children_done;
 static int max_pids;
@@ -137,6 +135,7 @@ static void child_loop_alloc(unsigned long long alloc_bytes)
 {
 	unsigned long bytecount = 0;
 	char *mem;
+	int runtime_rem;
 
 	tst_res(TINFO, "... child %d starting", getpid());
 
@@ -153,12 +152,15 @@ static void child_loop_alloc(unsigned long long alloc_bytes)
 		if (bytecount >= alloc_bytes)
 			break;
 	}
+
+	runtime_rem = tst_remaining_runtime();
+
 	if (dowrite)
 		tst_res(TINFO, "... [t=%d] %lu bytes allocated and used in child %d",
-				tst_timeout_remaining(), bytecount, getpid());
+				runtime_rem, bytecount, getpid());
 	else
 		tst_res(TINFO, "... [t=%d] %lu bytes allocated only in child %d",
-				tst_timeout_remaining(), bytecount, getpid());
+				runtime_rem, bytecount, getpid());
 
 	kill(getppid(), SIGRTMIN);
 	raise(SIGSTOP);
@@ -195,10 +197,9 @@ static void mem_test(void)
 
 	/* wait in the loop for all children finish allocating */
 	while (children_done < pid_cntr) {
-		if (tst_timeout_remaining() < STOP_THRESHOLD) {
+		if (!tst_remaining_runtime()) {
 			tst_res(TWARN,
 				"the remaininig time is not enough for testing");
-
 			break;
 		}
 
@@ -234,6 +235,7 @@ static struct tst_test test = {
 		{"v",  &verbose,     	"Verbose"},
 		{}
 	},
+	.max_runtime = 300,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = mem_test,
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 05/29] cve/cve-2015-3290: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (3 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 04/29] mtest01/mtest01: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 06/29] crypto/af_alg02: " Cyril Hrubis
                   ` (26 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/cve/cve-2015-3290.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/testcases/cve/cve-2015-3290.c b/testcases/cve/cve-2015-3290.c
index fd1abe136..f61d2809b 100644
--- a/testcases/cve/cve-2015-3290.c
+++ b/testcases/cve/cve-2015-3290.c
@@ -395,10 +395,6 @@ static void *child_thread(void *arg LTP_ATTRIBUTE_UNUSED)
 	return (void *)niter;
 }
 
-#define TIMEOUT		(180)
-#define TIME_TO_GIVEUP	(TIMEOUT - 5)
-#define TIMER_TYPE	CLOCK_MONOTONIC
-
 static void do_child(void)
 {
 	int i, ncpus;
@@ -415,7 +411,7 @@ static void do_child(void)
 	for (i = 0; i < ncpus; i++)
 		SAFE_PTHREAD_CREATE(&threads[i], NULL, child_thread, NULL);
 
-	sleep(TIME_TO_GIVEUP);
+	sleep(tst_remaining_runtime());
 	running = 0;
 
 	for (i = 0; i < ncpus; i++) {
@@ -468,7 +464,7 @@ static struct tst_test test = {
 	.needs_root = 1,
 	.needs_checkpoints = 1,
 	.setup = setup,
-	.timeout = TIMEOUT,
+	.max_runtime = 180,
 	.test_all = run,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "9b6e6a8334d5"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 06/29] crypto/af_alg02: Convert to runtime.
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (4 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 05/29] cve/cve-2015-3290: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 07/29] crypto/pcrypt_aead01: " Cyril Hrubis
                   ` (25 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/crypto/af_alg02.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/testcases/kernel/crypto/af_alg02.c b/testcases/kernel/crypto/af_alg02.c
index 9894ffacd..40d07ca90 100644
--- a/testcases/kernel/crypto/af_alg02.c
+++ b/testcases/kernel/crypto/af_alg02.c
@@ -65,7 +65,7 @@ static void run(void)
 	TST_CHECKPOINT_WAIT(0);
 
 	while (!completed) {
-		if (tst_timeout_remaining() <= 10) {
+		if (!tst_remaining_runtime()) {
 			pthread_cancel(thr);
 			tst_brk(TBROK,
 				"Timed out while reading from request socket.");
@@ -77,7 +77,7 @@ static void run(void)
 
 static struct tst_test test = {
 	.test_all = run,
-	.timeout = 20,
+	.max_runtime = 20,
 	.needs_checkpoints = 1,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "ecaaab564978"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 07/29] crypto/pcrypt_aead01: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (5 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 06/29] crypto/af_alg02: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 08/29] syscalls/clock_gettime01: Remove useless timeout Cyril Hrubis
                   ` (24 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/crypto/pcrypt_aead01.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/crypto/pcrypt_aead01.c b/testcases/kernel/crypto/pcrypt_aead01.c
index 0609af9f6..5eefee4cb 100644
--- a/testcases/kernel/crypto/pcrypt_aead01.c
+++ b/testcases/kernel/crypto/pcrypt_aead01.c
@@ -55,7 +55,7 @@ void run(void)
 		if (TST_RET)
 			tst_brk(TBROK | TRERRNO, "del_alg");
 
-		if (tst_timeout_remaining() < 10) {
+		if (!tst_remaining_runtime()) {
 			tst_res(TINFO, "Time limit reached, stopping at "
 				"%d iterations", i);
 			break;
@@ -75,6 +75,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.max_runtime = 300,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "d76c68109f37"},
 		{"CVE", "2017-5754"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 08/29] syscalls/clock_gettime01: Remove useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (6 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 07/29] crypto/pcrypt_aead01: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 09/29] syscalls/fanotify22: " Cyril Hrubis
                   ` (23 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/clock_gettime/clock_gettime01.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/clock_gettime/clock_gettime01.c b/testcases/kernel/syscalls/clock_gettime/clock_gettime01.c
index 11da39cf0..a67639b05 100644
--- a/testcases/kernel/syscalls/clock_gettime/clock_gettime01.c
+++ b/testcases/kernel/syscalls/clock_gettime/clock_gettime01.c
@@ -126,5 +126,4 @@ static struct tst_test test = {
 	.test_variants = ARRAY_SIZE(variants),
 	.setup = setup,
 	.needs_root = 1,
-	.timeout = 10,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 09/29] syscalls/fanotify22: Remove useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (7 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 08/29] syscalls/clock_gettime01: Remove useless timeout Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 10/29] syscalls/gettimeofday02: Convert to runtime Cyril Hrubis
                   ` (22 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

I guess that the timeout was there to exit faster in a case that the
test gets stuck. However that is no longer needed as the default timeout
was shortented significantly.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/fanotify/fanotify22.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/fanotify/fanotify22.c b/testcases/kernel/syscalls/fanotify/fanotify22.c
index f45c8512d..1105172bb 100644
--- a/testcases/kernel/syscalls/fanotify/fanotify22.c
+++ b/testcases/kernel/syscalls/fanotify/fanotify22.c
@@ -305,7 +305,6 @@ static struct tst_test test = {
 		{"linux-git", "124e7c61deb2"},
 		{}
 	},
-	.timeout = 10,
 	.needs_cmds = (const char *[]) {
 		"debugfs",
 		NULL
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 10/29] syscalls/gettimeofday02: Convert to runtime.
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (8 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 09/29] syscalls/fanotify22: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 11/29] syscalls/inotify06: " Cyril Hrubis
                   ` (21 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 .../kernel/syscalls/gettimeofday/gettimeofday02.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/testcases/kernel/syscalls/gettimeofday/gettimeofday02.c b/testcases/kernel/syscalls/gettimeofday/gettimeofday02.c
index 948d2d8ec..7c462cc29 100644
--- a/testcases/kernel/syscalls/gettimeofday/gettimeofday02.c
+++ b/testcases/kernel/syscalls/gettimeofday/gettimeofday02.c
@@ -25,8 +25,6 @@
 #include "lapi/syscalls.h"
 
 static volatile sig_atomic_t done;
-static char *str_rtime;
-static int rtime = 10;
 
 static void breakout(int sig)
 {
@@ -37,6 +35,7 @@ static void verify_gettimeofday(void)
 {
 	struct __kernel_old_timeval tv1, tv2;
 	unsigned long long cnt = 0;
+	int rtime = tst_remaining_runtime();
 
 	done = 0;
 
@@ -68,21 +67,11 @@ static void verify_gettimeofday(void)
 
 static void setup(void)
 {
-	if (str_rtime) {
-		rtime = atoi(str_rtime);
-		if (rtime <= 0)
-			tst_brk(TBROK, "Invalid runtime '%s'", str_rtime);
-		tst_set_timeout(rtime + 60);
-	}
-
 	SAFE_SIGNAL(SIGALRM, breakout);
 }
 
 static struct tst_test test = {
 	.setup = setup,
-	.options = (struct tst_option[]) {
-		{"T:", &str_rtime, "Test iteration runtime in seconds"},
-		{},
-	},
+	.max_runtime = 10,
 	.test_all = verify_gettimeofday,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 11/29] syscalls/inotify06: Convert to runtime.
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (9 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 10/29] syscalls/gettimeofday02: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:37 ` [LTP] [PATCH v3 12/29] syscalls/inotify01: Remove now useless timeout Cyril Hrubis
                   ` (20 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/inotify/inotify06.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/syscalls/inotify/inotify06.c b/testcases/kernel/syscalls/inotify/inotify06.c
index 68813769b..86d74b9c8 100644
--- a/testcases/kernel/syscalls/inotify/inotify06.c
+++ b/testcases/kernel/syscalls/inotify/inotify06.c
@@ -87,6 +87,11 @@ static void verify_inotify(void)
 			myinotify_add_watch(inotify_fd, names[i], IN_MODIFY);
 		}
 		SAFE_CLOSE(inotify_fd);
+
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Test out of runtime, exitting");
+			break;
+		}
 	}
 	/* We survived for given time - test succeeded */
 	tst_res(TPASS, "kernel survived inotify beating");
@@ -108,7 +113,7 @@ static void cleanup(void)
 }
 
 static struct tst_test test = {
-	.timeout = 600,
+	.max_runtime = 600,
 	.needs_root = 1,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 12/29] syscalls/inotify01: Remove now useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (10 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 11/29] syscalls/inotify06: " Cyril Hrubis
@ 2022-05-12 12:37 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 13/29] syscalls/perf_event_open03: Convert to runtime Cyril Hrubis
                   ` (19 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:37 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/inotify/inotify11.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/inotify/inotify11.c b/testcases/kernel/syscalls/inotify/inotify11.c
index 062b92409..e426f9e1c 100644
--- a/testcases/kernel/syscalls/inotify/inotify11.c
+++ b/testcases/kernel/syscalls/inotify/inotify11.c
@@ -117,7 +117,6 @@ static void cleanup(void)
 }
 
 static struct tst_test test = {
-	.timeout = 10,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
 	.cleanup = cleanup,
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 13/29] syscalls/perf_event_open03: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (11 preceding siblings ...)
  2022-05-12 12:37 ` [LTP] [PATCH v3 12/29] syscalls/inotify01: Remove now useless timeout Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 14/29] syscalls/readv01: Remove now useless timeout Cyril Hrubis
                   ` (18 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 .../syscalls/perf_event_open/perf_event_open03.c  | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/testcases/kernel/syscalls/perf_event_open/perf_event_open03.c b/testcases/kernel/syscalls/perf_event_open/perf_event_open03.c
index 6dffc8272..999c8aa22 100644
--- a/testcases/kernel/syscalls/perf_event_open/perf_event_open03.c
+++ b/testcases/kernel/syscalls/perf_event_open/perf_event_open03.c
@@ -25,7 +25,7 @@
 
 const int iterations = 12000000;
 static int fd = -1;
-static int timeout;
+static int runtime;
 
 static void setup(void)
 {
@@ -43,12 +43,12 @@ static void setup(void)
 	SAFE_FILE_SCANF(INTEL_PT_PATH, "%d", &ev.type);
 	fd = perf_event_open(&ev, getpid(), -1, -1, 0);
 
-	timeout = tst_timeout_remaining();
+	runtime = tst_remaining_runtime();
 }
 
 /*
  * Check how fast we can do the iterations after 5 seconds of runtime.
- * If the rate is too small to complete for current timeout then
+ * If the rate is too small to complete for current runtime then
  * stop the test.
  */
 static void check_progress(int i)
@@ -67,11 +67,11 @@ static void check_progress(int i)
 	if (elapsed_ms > 5000) {
 		iter_per_ms = (float) i / elapsed_ms;
 		tst_res(TINFO, "rate: %f iters/ms", iter_per_ms);
-		tst_res(TINFO, "needed rate for current test timeout: %f iters/ms",
-			(float) iterations / (timeout * 1000));
+		tst_res(TINFO, "needed rate for current test runtime: %f iters/ms",
+			(float) iterations / (runtime * 1000));
 
-		if (iter_per_ms * 1000 * (timeout - 1) < iterations)
-			tst_brk(TCONF, "System too slow to complete test in specified timeout");
+		if (iter_per_ms * 1000 * (runtime - 1) < iterations)
+			tst_brk(TCONF, "System too slow to complete test in specified runtime");
 	}
 }
 
@@ -108,6 +108,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.max_runtime = 300,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "7bdb157cdebb"},
 		{"CVE", "2020-25704"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 14/29] syscalls/readv01: Remove now useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (12 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 13/29] syscalls/perf_event_open03: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 15/29] syscalls/tgkill03: Remove now unused timeout Cyril Hrubis
                   ` (17 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/readv/readv01.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/readv/readv01.c b/testcases/kernel/syscalls/readv/readv01.c
index f12b3f09f..2bd3fd5eb 100644
--- a/testcases/kernel/syscalls/readv/readv01.c
+++ b/testcases/kernel/syscalls/readv/readv01.c
@@ -110,7 +110,6 @@ static struct tst_test test = {
 	.test = test_readv,
 	.tcnt = ARRAY_SIZE(testcase_list),
 	.needs_tmpdir = 1,
-	.timeout = 15,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "19f18459330f"},
 		{}
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 15/29] syscalls/tgkill03: Remove now unused timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (13 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 14/29] syscalls/readv01: Remove now useless timeout Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 16/29] syscalls/setsockopt09: Remove now useless timeout Cyril Hrubis
                   ` (16 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/tgkill/tgkill03.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/tgkill/tgkill03.c b/testcases/kernel/syscalls/tgkill/tgkill03.c
index 0002f3278..e46e95f73 100644
--- a/testcases/kernel/syscalls/tgkill/tgkill03.c
+++ b/testcases/kernel/syscalls/tgkill/tgkill03.c
@@ -117,5 +117,4 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test = run,
-	.timeout = 20,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 16/29] syscalls/setsockopt09: Remove now useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (14 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 15/29] syscalls/tgkill03: Remove now unused timeout Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 17/29] syscalls/userfaultfd01: " Cyril Hrubis
                   ` (15 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/setsockopt/setsockopt09.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/setsockopt/setsockopt09.c b/testcases/kernel/syscalls/setsockopt/setsockopt09.c
index 3047ec0ff..98f7fd00e 100644
--- a/testcases/kernel/syscalls/setsockopt/setsockopt09.c
+++ b/testcases/kernel/syscalls/setsockopt/setsockopt09.c
@@ -117,7 +117,6 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
-	.timeout = 5,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_USER_NS=y",
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 17/29] syscalls/userfaultfd01: Remove now useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (15 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 16/29] syscalls/setsockopt09: Remove now useless timeout Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 18/29] syscalls/move_pages12: Convert to runtime Cyril Hrubis
                   ` (14 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/userfaultfd/userfaultfd01.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/userfaultfd/userfaultfd01.c b/testcases/kernel/syscalls/userfaultfd/userfaultfd01.c
index 4e178b4f8..2dae2ec78 100644
--- a/testcases/kernel/syscalls/userfaultfd/userfaultfd01.c
+++ b/testcases/kernel/syscalls/userfaultfd/userfaultfd01.c
@@ -121,7 +121,6 @@ static void run(void)
 static struct tst_test test = {
 	.test_all = run,
 	.min_kver = "4.3",
-	.timeout = 20
 };
 
 #else
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 18/29] syscalls/move_pages12: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (16 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 17/29] syscalls/userfaultfd01: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 19/29] syscalls/rt_sigqueueinfo01: Remove now useless timeout Cyril Hrubis
                   ` (13 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/move_pages/move_pages12.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/testcases/kernel/syscalls/move_pages/move_pages12.c b/testcases/kernel/syscalls/move_pages/move_pages12.c
index 220130f4b..df55bbbe9 100644
--- a/testcases/kernel/syscalls/move_pages/move_pages12.c
+++ b/testcases/kernel/syscalls/move_pages/move_pages12.c
@@ -153,7 +153,6 @@ static void do_test(unsigned int n)
 	void *ptr;
 	pid_t cpid = -1;
 	int status;
-	unsigned int twenty_percent = (tst_timeout_remaining() / 5);
 
 	addr = SAFE_MMAP(NULL, tcases[n].tpages * hpsz, PROT_READ | PROT_WRITE,
 		MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
@@ -198,7 +197,7 @@ static void do_test(unsigned int n)
 
 		SAFE_MUNMAP(addr, tcases[n].tpages * hpsz);
 
-		if (tst_timeout_remaining() < twenty_percent)
+		if (!tst_remaining_runtime())
 			break;
 	}
 
@@ -341,6 +340,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(tcases),
+	.max_runtime = 240,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "e66f17ff7177"},
 		{"linux-git", "c9d398fa2378"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 19/29] syscalls/rt_sigqueueinfo01: Remove now useless timeout
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (17 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 18/29] syscalls/move_pages12: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 20/29] mem/mallocstress: Convert to runtime Cyril Hrubis
                   ` (12 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c b/testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c
index 5996e998b..c8b07c077 100644
--- a/testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c
+++ b/testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c
@@ -108,7 +108,6 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.needs_checkpoints = 1,
-	.timeout = 20,
 };
 
 #else
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 20/29] mem/mallocstress: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (18 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 19/29] syscalls/rt_sigqueueinfo01: Remove now useless timeout Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 21/29] mem/{oom, min_free_kbytes}: " Cyril Hrubis
                   ` (11 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/mem/mtest07/mallocstress.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/testcases/kernel/mem/mtest07/mallocstress.c b/testcases/kernel/mem/mtest07/mallocstress.c
index fa7494e7d..ae3a9a004 100644
--- a/testcases/kernel/mem/mtest07/mallocstress.c
+++ b/testcases/kernel/mem/mtest07/mallocstress.c
@@ -62,7 +62,7 @@ static void my_yield(void)
  *  0: success
  *  1: failure
  */
-int allocate_free(int scheme)
+int allocate_free(int scheme, int threadnum)
 {
 	int loop;
 	const int MAXPTRS = 50;	/* only 42 or so get used on 32 bit machine */
@@ -127,6 +127,11 @@ int allocate_free(int scheme)
 		}
 
 		my_yield();
+
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Thread [%d]: Test runtime is over, exitting", threadnum);
+			break;
+		}
 	}
 
 	/* Success! */
@@ -141,7 +146,7 @@ void *alloc_mem(void *threadnum)
 	TST_CHECKPOINT_WAIT(0);
 
 	/* thread N will use growth scheme N mod 4 */
-	err = allocate_free(((uintptr_t)threadnum) % 4);
+	err = allocate_free(((uintptr_t)threadnum) % 4, (uintptr_t)threadnum);
 	tst_res(TINFO,
 		"Thread [%d]: allocate_free() returned %d, %s.  Thread exiting.\n",
 		(int)(uintptr_t)threadnum, err,
@@ -189,7 +194,7 @@ static void cleanup(void)
 }
 
 static struct tst_test test = {
-	.timeout = 600,
+	.max_runtime = 600,
 	.needs_checkpoints = 1,
 	.setup = setup,
 	.cleanup = cleanup,
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 21/29] mem/{oom, min_free_kbytes}: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (19 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 20/29] mem/mallocstress: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 22/29] fuzzy_sync: " Cyril Hrubis
                   ` (10 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

This just sets the runtime to TST_UNLIMITED_RUNTIME which disables the
timeout.

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/mem/oom/oom01.c               | 2 +-
 testcases/kernel/mem/oom/oom02.c               | 2 +-
 testcases/kernel/mem/oom/oom03.c               | 2 +-
 testcases/kernel/mem/oom/oom04.c               | 2 +-
 testcases/kernel/mem/oom/oom05.c               | 2 +-
 testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +-
 6 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/testcases/kernel/mem/oom/oom01.c b/testcases/kernel/mem/oom/oom01.c
index 258bfd89e..9f7d76587 100644
--- a/testcases/kernel/mem/oom/oom01.c
+++ b/testcases/kernel/mem/oom/oom01.c
@@ -63,7 +63,7 @@ static void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_oom,
diff --git a/testcases/kernel/mem/oom/oom02.c b/testcases/kernel/mem/oom/oom02.c
index 2b9bcb1b2..b3719f723 100644
--- a/testcases/kernel/mem/oom/oom02.c
+++ b/testcases/kernel/mem/oom/oom02.c
@@ -70,7 +70,7 @@ static void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+        .max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_oom,
diff --git a/testcases/kernel/mem/oom/oom03.c b/testcases/kernel/mem/oom/oom03.c
index 48a0f20b5..0882c9bbe 100644
--- a/testcases/kernel/mem/oom/oom03.c
+++ b/testcases/kernel/mem/oom/oom03.c
@@ -95,7 +95,7 @@ static void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_oom,
diff --git a/testcases/kernel/mem/oom/oom04.c b/testcases/kernel/mem/oom/oom04.c
index 6b65383a3..ad39f7e77 100644
--- a/testcases/kernel/mem/oom/oom04.c
+++ b/testcases/kernel/mem/oom/oom04.c
@@ -91,7 +91,7 @@ static void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_oom,
diff --git a/testcases/kernel/mem/oom/oom05.c b/testcases/kernel/mem/oom/oom05.c
index 91c9bec83..e31146e7a 100644
--- a/testcases/kernel/mem/oom/oom05.c
+++ b/testcases/kernel/mem/oom/oom05.c
@@ -113,7 +113,7 @@ void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_oom,
diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c b/testcases/kernel/mem/tunable/min_free_kbytes.c
index 09741eea3..eab6c8b2e 100644
--- a/testcases/kernel/mem/tunable/min_free_kbytes.c
+++ b/testcases/kernel/mem/tunable/min_free_kbytes.c
@@ -231,7 +231,7 @@ static void cleanup(void)
 static struct tst_test test = {
 	.needs_root = 1,
 	.forks_child = 1,
-	.timeout = -1,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = min_free_kbytes_test,
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (20 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 21/29] mem/{oom, min_free_kbytes}: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-13  9:13   ` Li Wang
  2022-05-12 12:38 ` [LTP] [PATCH v3 23/29] ltp-aiodio/dio_sparse, aiodio_sparse: " Cyril Hrubis
                   ` (9 subsequent siblings)
  31 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: Richard Palethorpe, automated-testing

The runtime for fuzzy sync tests are computed as timeout * exec_time_p.
The default timeout was 300 and the default exec_time_p was 0.5.

CC: Richard Palethorpe <rpalethorpe@suse.com>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 include/tst_fuzzy_sync.h                      | 19 ++++---------------
 lib/newlib_tests/tst_fuzzy_sync01.c           |  1 +
 lib/newlib_tests/tst_fuzzy_sync02.c           |  1 +
 lib/newlib_tests/tst_fuzzy_sync03.c           |  1 +
 testcases/kernel/crypto/af_alg07.c            |  1 +
 testcases/kernel/mem/thp/thp04.c              |  1 +
 testcases/kernel/pty/pty03.c                  |  2 +-
 testcases/kernel/pty/pty05.c                  |  1 +
 testcases/kernel/pty/pty06.c                  |  1 +
 testcases/kernel/pty/pty07.c                  |  1 +
 testcases/kernel/sound/snd_seq01.c            |  2 +-
 testcases/kernel/sound/snd_timer01.c          |  1 +
 testcases/kernel/syscalls/bind/bind06.c       |  2 +-
 testcases/kernel/syscalls/inotify/inotify09.c |  1 +
 .../kernel/syscalls/ipc/shmctl/shmctl05.c     |  2 +-
 testcases/kernel/syscalls/sendmsg/sendmsg03.c |  1 +
 .../kernel/syscalls/setsockopt/setsockopt06.c |  2 +-
 .../kernel/syscalls/setsockopt/setsockopt07.c |  1 +
 .../syscalls/timerfd/timerfd_settime02.c      |  1 +
 testcases/kernel/syscalls/writev/writev03.c   |  2 +-
 20 files changed, 23 insertions(+), 21 deletions(-)

diff --git a/include/tst_fuzzy_sync.h b/include/tst_fuzzy_sync.h
index 4f09ed416..bef424002 100644
--- a/include/tst_fuzzy_sync.h
+++ b/include/tst_fuzzy_sync.h
@@ -160,15 +160,6 @@ struct tst_fzsync_pair {
 	int b_cntr;
 	/** Internal; Used by tst_fzsync_pair_exit() and fzsync_pair_wait() */
 	int exit;
-	/**
-	 * The maximum desired execution time as a proportion of the timeout
-	 *
-	 * A value x so that 0 < x < 1 which decides how long the test should
-	 * be run for (assuming the loop limit is not exceeded first).
-	 *
-	 * Defaults to 0.5 (~150 seconds with default timeout).
-	 */
-	float exec_time_p;
 	/** Internal; The test time remaining on tst_fzsync_pair_reset() */
 	float exec_time_start;
 	/**
@@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct tst_fzsync_pair *pair)
 	CHK(avg_alpha, 0, 1, 0.25);
 	CHK(min_samples, 20, INT_MAX, 1024);
 	CHK(max_dev_ratio, 0, 1, 0.1);
-	CHK(exec_time_p, 0, 1, 0.5);
 	CHK(exec_loops, 20, INT_MAX, 3000000);
 
 	if (tst_ncpus_available() <= 1)
@@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct tst_fzsync_pair *pair,
 	if (run_b)
 		SAFE_PTHREAD_CREATE(&pair->thread_b, 0, run_b, 0);
 
-	pair->exec_time_start = (float)tst_timeout_remaining();
+	pair->exec_time_start = (float)tst_remaining_runtime();
 }
 
 /**
@@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(struct tst_fzsync_pair *pair)
  */
 static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
 {
-	float rem_p = 1 - tst_timeout_remaining() / pair->exec_time_start;
+	float rem_p = 1 - tst_remaining_runtime() / pair->exec_time_start;
 
-	if ((pair->exec_time_p * SAMPLING_SLICE < rem_p)
-		&& (pair->sampling > 0)) {
+	if ((SAMPLING_SLICE < rem_p) && (pair->sampling > 0)) {
 		tst_res(TINFO, "Stopped sampling at %d (out of %d) samples, "
 			"sampling time reached 50%% of the total time limit",
 			pair->exec_loop, pair->min_samples);
@@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
 		tst_fzsync_pair_info(pair);
 	}
 
-	if (pair->exec_time_p < rem_p) {
+	if (rem_p >= 1) {
 		tst_res(TINFO,
 			"Exceeded execution time, requesting exit");
 		tst_atomic_store(1, &pair->exit);
diff --git a/lib/newlib_tests/tst_fuzzy_sync01.c b/lib/newlib_tests/tst_fuzzy_sync01.c
index 5f23a085b..d0748958c 100644
--- a/lib/newlib_tests/tst_fuzzy_sync01.c
+++ b/lib/newlib_tests/tst_fuzzy_sync01.c
@@ -227,4 +227,5 @@ static struct tst_test test = {
 	.test = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 150,
 };
diff --git a/lib/newlib_tests/tst_fuzzy_sync02.c b/lib/newlib_tests/tst_fuzzy_sync02.c
index c1c2a5327..afe4973b5 100644
--- a/lib/newlib_tests/tst_fuzzy_sync02.c
+++ b/lib/newlib_tests/tst_fuzzy_sync02.c
@@ -169,4 +169,5 @@ static struct tst_test test = {
 	.test = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 150,
 };
diff --git a/lib/newlib_tests/tst_fuzzy_sync03.c b/lib/newlib_tests/tst_fuzzy_sync03.c
index 0d74e1eae..47ce7675a 100644
--- a/lib/newlib_tests/tst_fuzzy_sync03.c
+++ b/lib/newlib_tests/tst_fuzzy_sync03.c
@@ -99,4 +99,5 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = run,
+	.max_runtime = 150,
 };
diff --git a/testcases/kernel/crypto/af_alg07.c b/testcases/kernel/crypto/af_alg07.c
index ef13ad764..9c2516632 100644
--- a/testcases/kernel/crypto/af_alg07.c
+++ b/testcases/kernel/crypto/af_alg07.c
@@ -125,6 +125,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.min_kver = "4.10.0",
 	.min_cpus = 2,
+	.max_runtime = 150,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "ff7b11aa481f"},
diff --git a/testcases/kernel/mem/thp/thp04.c b/testcases/kernel/mem/thp/thp04.c
index 985394dc3..c93da54e6 100644
--- a/testcases/kernel/mem/thp/thp04.c
+++ b/testcases/kernel/mem/thp/thp04.c
@@ -163,6 +163,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "a8f97366452e"},
 		{"linux-git", "8310d48b125d"},
diff --git a/testcases/kernel/pty/pty03.c b/testcases/kernel/pty/pty03.c
index 71bcb2eb6..7354fdb21 100644
--- a/testcases/kernel/pty/pty03.c
+++ b/testcases/kernel/pty/pty03.c
@@ -135,7 +135,6 @@ static void do_test(unsigned int n)
 static void setup(void)
 {
 	fzp.min_samples = 20;
-	fzp.exec_time_p = 0.1;
 
 	tst_fzsync_pair_init(&fzp);
 }
@@ -151,6 +150,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.max_runtime = 30,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "0ace17d568241"},
 		{"CVE", "2020-14416"},
diff --git a/testcases/kernel/pty/pty05.c b/testcases/kernel/pty/pty05.c
index afef051c8..35ecce922 100644
--- a/testcases/kernel/pty/pty05.c
+++ b/testcases/kernel/pty/pty05.c
@@ -97,6 +97,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "82f2341c94d27"},
 		{"CVE", "2017-2636"},
diff --git a/testcases/kernel/pty/pty06.c b/testcases/kernel/pty/pty06.c
index cc8eb1f85..7e0f4233c 100644
--- a/testcases/kernel/pty/pty06.c
+++ b/testcases/kernel/pty/pty06.c
@@ -97,6 +97,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_root = 1,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 	    { "linux-git", "ca4463bf8438"},
 	    {}
diff --git a/testcases/kernel/pty/pty07.c b/testcases/kernel/pty/pty07.c
index c63b71c89..68b3e470c 100644
--- a/testcases/kernel/pty/pty07.c
+++ b/testcases/kernel/pty/pty07.c
@@ -110,6 +110,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_root = 1,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{ "linux-git", "6cd1ed50efd8"},
 		{}
diff --git a/testcases/kernel/sound/snd_seq01.c b/testcases/kernel/sound/snd_seq01.c
index c56752230..31038b937 100644
--- a/testcases/kernel/sound/snd_seq01.c
+++ b/testcases/kernel/sound/snd_seq01.c
@@ -123,7 +123,7 @@ static struct tst_test test = {
 	.tcnt = ARRAY_SIZE(testfunc_list),
 	.setup = setup,
 	.cleanup = cleanup,
-	.timeout = 120,
+	.max_runtime = 60,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "d15d662e89fc"},
diff --git a/testcases/kernel/sound/snd_timer01.c b/testcases/kernel/sound/snd_timer01.c
index 51591c18e..3d8e715e3 100644
--- a/testcases/kernel/sound/snd_timer01.c
+++ b/testcases/kernel/sound/snd_timer01.c
@@ -139,6 +139,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "d11662f4f798"},
 		{"linux-git", "ba3021b2c79b"},
diff --git a/testcases/kernel/syscalls/bind/bind06.c b/testcases/kernel/syscalls/bind/bind06.c
index 1dfc534a7..618cfce46 100644
--- a/testcases/kernel/syscalls/bind/bind06.c
+++ b/testcases/kernel/syscalls/bind/bind06.c
@@ -102,7 +102,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
-	.timeout = 600,
+	.max_runtime = 300,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_USER_NS=y",
diff --git a/testcases/kernel/syscalls/inotify/inotify09.c b/testcases/kernel/syscalls/inotify/inotify09.c
index fdfc9c078..a18750794 100644
--- a/testcases/kernel/syscalls/inotify/inotify09.c
+++ b/testcases/kernel/syscalls/inotify/inotify09.c
@@ -94,6 +94,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = verify_inotify,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "d90a10e2444b"},
 		{}
diff --git a/testcases/kernel/syscalls/ipc/shmctl/shmctl05.c b/testcases/kernel/syscalls/ipc/shmctl/shmctl05.c
index 228174c9b..ca668aaf7 100644
--- a/testcases/kernel/syscalls/ipc/shmctl/shmctl05.c
+++ b/testcases/kernel/syscalls/ipc/shmctl/shmctl05.c
@@ -106,7 +106,7 @@ static void cleanup(void)
 }
 
 static struct tst_test test = {
-	.timeout = 20,
+	.max_runtime = 10,
 	.setup = setup,
 	.test_all = do_test,
 	.cleanup = cleanup,
diff --git a/testcases/kernel/syscalls/sendmsg/sendmsg03.c b/testcases/kernel/syscalls/sendmsg/sendmsg03.c
index c7d72f686..505a6dd24 100644
--- a/testcases/kernel/syscalls/sendmsg/sendmsg03.c
+++ b/testcases/kernel/syscalls/sendmsg/sendmsg03.c
@@ -105,6 +105,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "8f659a03a0ba"},
 		{"CVE", "2017-17712"},
diff --git a/testcases/kernel/syscalls/setsockopt/setsockopt06.c b/testcases/kernel/syscalls/setsockopt/setsockopt06.c
index c6fc2fba9..9c818646b 100644
--- a/testcases/kernel/syscalls/setsockopt/setsockopt06.c
+++ b/testcases/kernel/syscalls/setsockopt/setsockopt06.c
@@ -46,7 +46,6 @@ static void setup(void)
 	SAFE_FILE_PRINTF("/proc/self/gid_map", "0 %d 1", real_gid);
 
 	fzsync_pair.exec_loops = 100000;
-	fzsync_pair.exec_time_p = 0.9;
 	tst_fzsync_pair_init(&fzsync_pair);
 }
 
@@ -123,6 +122,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 270,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_USER_NS=y",
diff --git a/testcases/kernel/syscalls/setsockopt/setsockopt07.c b/testcases/kernel/syscalls/setsockopt/setsockopt07.c
index 147ae3899..616159a90 100644
--- a/testcases/kernel/syscalls/setsockopt/setsockopt07.c
+++ b/testcases/kernel/syscalls/setsockopt/setsockopt07.c
@@ -136,6 +136,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 150,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_USER_NS=y",
 		"CONFIG_NET_NS=y",
diff --git a/testcases/kernel/syscalls/timerfd/timerfd_settime02.c b/testcases/kernel/syscalls/timerfd/timerfd_settime02.c
index bd92ee964..84ce95538 100644
--- a/testcases/kernel/syscalls/timerfd/timerfd_settime02.c
+++ b/testcases/kernel/syscalls/timerfd/timerfd_settime02.c
@@ -112,6 +112,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.min_kver = "2.6.25",
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "1e38da300e1e"},
 		{"CVE", "2017-10661"},
diff --git a/testcases/kernel/syscalls/writev/writev03.c b/testcases/kernel/syscalls/writev/writev03.c
index d0c64743b..5b496203a 100644
--- a/testcases/kernel/syscalls/writev/writev03.c
+++ b/testcases/kernel/syscalls/writev/writev03.c
@@ -49,7 +49,6 @@ static void setup(void)
 	mapfd = SAFE_OPEN(MAPFILE, O_CREAT|O_RDWR|O_TRUNC, 0644);
 	SAFE_WRITE(1, mapfd, buf, BUF_SIZE);
 
-	fzsync_pair.exec_time_p = 0.25;
 	tst_fzsync_pair_init(&fzsync_pair);
 }
 
@@ -146,6 +145,7 @@ static struct tst_test test = {
 	.min_cpus = 2,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = 75,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "d4690f1e1cda"},
 		{}
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 23/29] ltp-aiodio/dio_sparse, aiodio_sparse: Convert to runtime.
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (21 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 22/29] fuzzy_sync: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 24/29] ltp-aiodio/read_checkzero: Remove Cyril Hrubis
                   ` (8 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Change the code so that the test runtime is capped at the maximal
runtime allowed for the test.

For this to work we also have to:

- check the run_child in the io_read() in the inner loop otherwise the
  test will timeout while waiting for the children to finish

- run the AIO writer in a separate process and kill it when we are out
  of runtime in the aiodio_sparse

CC: Andrea Cervesato <andrea.cervesato@suse.de>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 .../kernel/io/ltp-aiodio/aiodio_sparse.c      | 26 ++++++++++++++++---
 testcases/kernel/io/ltp-aiodio/common.h       |  6 ++++-
 testcases/kernel/io/ltp-aiodio/dio_sparse.c   |  6 ++++-
 3 files changed, 32 insertions(+), 6 deletions(-)

diff --git a/testcases/kernel/io/ltp-aiodio/aiodio_sparse.c b/testcases/kernel/io/ltp-aiodio/aiodio_sparse.c
index 2aa5662bb..d79933d9a 100644
--- a/testcases/kernel/io/ltp-aiodio/aiodio_sparse.c
+++ b/testcases/kernel/io/ltp-aiodio/aiodio_sparse.c
@@ -189,7 +189,7 @@ static void run(void)
 {
 	char *filename = "file.bin";
 	int status;
-	int i;
+	int i, pid;
 
 	*run_child = 1;
 
@@ -200,9 +200,27 @@ static void run(void)
 		}
 	}
 
-	tst_res(TINFO, "Parent create a sparse file");
+	pid = SAFE_FORK();
+	if (!pid) {
+		aiodio_sparse(filename, alignment, writesize, filesize, numaio);
+		return;
+	}
+
+	tst_res(TINFO, "Child %i creates a sparse file", pid);
+
+	for (;;) {
+		if (SAFE_WAITPID(pid, NULL, WNOHANG))
+			break;
 
-	aiodio_sparse(filename, alignment, writesize, filesize, numaio);
+		sleep(1);
+
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Test out of runtime, exitting");
+			kill(pid, SIGKILL);
+			SAFE_WAITPID(pid, NULL, 0);
+			break;
+		}
+	}
 
 	if (SAFE_WAITPID(-1, &status, WNOHANG))
 		tst_res(TFAIL, "Non zero bytes read");
@@ -229,7 +247,7 @@ static struct tst_test test = {
 		"tmpfs",
 		NULL
 	},
-	.timeout = 1800,
+	.max_runtime = 1800,
 };
 #else
 TST_TEST_TCONF("test requires libaio and its development packages");
diff --git a/testcases/kernel/io/ltp-aiodio/common.h b/testcases/kernel/io/ltp-aiodio/common.h
index c27c3a0ad..0a242b91e 100644
--- a/testcases/kernel/io/ltp-aiodio/common.h
+++ b/testcases/kernel/io/ltp-aiodio/common.h
@@ -63,7 +63,7 @@ static inline void io_read(const char *filename, int filesize, volatile int *run
 
 	tst_res(TINFO, "child %i reading file", getpid());
 
-	while (*run_child) {
+	for (;;) {
 		off_t offset = 0;
 		char *bufoff;
 
@@ -80,9 +80,13 @@ static inline void io_read(const char *filename, int filesize, volatile int *run
 				}
 				offset += r;
 			}
+
+			if (!*run_child)
+				goto exit;
 		}
 	}
 
+exit:
 	SAFE_CLOSE(fd);
 }
 
diff --git a/testcases/kernel/io/ltp-aiodio/dio_sparse.c b/testcases/kernel/io/ltp-aiodio/dio_sparse.c
index 0039daa8d..24cfc8f36 100644
--- a/testcases/kernel/io/ltp-aiodio/dio_sparse.c
+++ b/testcases/kernel/io/ltp-aiodio/dio_sparse.c
@@ -51,6 +51,10 @@ static void dio_sparse(int fd, int align, long long fs, int ws, long long off)
 	SAFE_LSEEK(fd, off, SEEK_SET);
 
 	for (i = off; i < fs;) {
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Test runtime is over, exitting");
+			return;
+		}
 		w = SAFE_WRITE(0, fd, bufptr, ws);
 		i += w;
 	}
@@ -135,5 +139,5 @@ static struct tst_test test = {
 		"tmpfs",
 		NULL
 	},
-	.timeout = 1800,
+	.max_runtime = 1800,
 };
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 24/29] ltp-aiodio/read_checkzero: Remove
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (22 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 23/29] ltp-aiodio/dio_sparse, aiodio_sparse: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 25/29] ltp-aiodio/dio_{truncate, append}: Convert to runtime Cyril Hrubis
                   ` (7 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Remove leftovers that are not used anymore.

CC: Andrea Cervesato <andrea.cervesato@suse.de>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 runtest/ltp-aiodio.part4                      |  2 -
 testcases/kernel/io/ltp-aiodio/.gitignore     |  1 -
 .../kernel/io/ltp-aiodio/read_checkzero.c     | 84 -------------------
 3 files changed, 87 deletions(-)
 delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c

diff --git a/runtest/ltp-aiodio.part4 b/runtest/ltp-aiodio.part4
index 54019d47b..d88c27a83 100644
--- a/runtest/ltp-aiodio.part4
+++ b/runtest/ltp-aiodio.part4
@@ -59,8 +59,6 @@ ADI009 dio_append
 DIT000 dio_truncate
 DIT001 dio_truncate
 DIT002 dio_truncate
-#Running read_checkzero
-#gread_checkzero
 #Running dio_read
 DOR000 dio_read -n 1 -i 100 -r 512k -w 512k -s 32M
 DOR001 dio_read -n 10 -i 30 -r 512k -w 512k -s 32M
diff --git a/testcases/kernel/io/ltp-aiodio/.gitignore b/testcases/kernel/io/ltp-aiodio/.gitignore
index f5f20d57e..09a49bfbf 100644
--- a/testcases/kernel/io/ltp-aiodio/.gitignore
+++ b/testcases/kernel/io/ltp-aiodio/.gitignore
@@ -7,4 +7,3 @@
 /dio_truncate
 /dio_read
 /dirty
-/read_checkzero
diff --git a/testcases/kernel/io/ltp-aiodio/read_checkzero.c b/testcases/kernel/io/ltp-aiodio/read_checkzero.c
deleted file mode 100644
index b48197a7b..000000000
--- a/testcases/kernel/io/ltp-aiodio/read_checkzero.c
+++ /dev/null
@@ -1,84 +0,0 @@
-/*
- * Copyright (c) 2004 Daniel McNeil <daniel@osdl.org>
- *               2004 Open Source Development Lab
- *   This program is free software;  you can redistribute it and/or modify
- *   it under the terms of the GNU General Public License as published by
- *   the Free Software Foundation; either version 2 of the License, or
- *   (at your option) any later version.
- *
- *   This program is distributed in the hope that it will be useful,
- *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
- *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
- *   the GNU General Public License for more details.
- *
- *   You should have received a copy of the GNU General Public License
- *   along with this program;  if not, write to the Free Software
- *   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Module: .c
- */
-
-/*
- * Change History:
- *
- * 2/2004  Marty Ridgeway (mridge@us.ibm.com) Changes to adapt to LTP
- *
- */
-#define _GNU_SOURCE
-
-#include <stdlib.h>
-#include <sys/types.h>
-#include <signal.h>
-#include <fcntl.h>
-#include <stdio.h>
-#include <unistd.h>
-
-#include "common_checkzero.h"
-
-int read_eof(char *filename)
-{
-	int fd;
-	int i;
-	int r;
-	char buf[4096];
-
-	if ((fd = open(filename, O_RDWR)) < 0) {
-		fprintf(stderr, "can't open file %s \n", filename);
-		exit(1);
-	}
-
-	for (i = 0; i < 100000; i++) {
-		off_t offset;
-		char *bufoff;
-
-		offset = lseek(fd, 4096, SEEK_END);
-		r = write(fd, "A", 1);
-
-		offset = lseek(fd, offset - 4096, SEEK_SET);
-
-		r = read(fd, buf, 4096);
-		if (r > 0) {
-			if ((bufoff = check_zero(buf, r))) {
-				fprintf(stderr, "non-zero read at offset %p\n",
-					offset + bufoff);
-				exit(1);
-			}
-		}
-	}
-	fprintf(stderr, "read_checkzero done\n");
-	return 0;
-}
-
-int main(int argc, char **argv)
-{
-	if (argc < 2) {
-		printf("You must pass a filename to the test \n");
-		exit(1);
-	}
-
-	char *filename = argv[1];
-
-	read_eof(filename);
-
-	return 0;
-}
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 25/29] ltp-aiodio/dio_{truncate, append}: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (23 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 24/29] ltp-aiodio/read_checkzero: Remove Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: " Cyril Hrubis
                   ` (6 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Change the code so that the test runtime is capped at the maximal
runtime allowed for the test, which is set to 30 minutes by default.

The io_append() in common.h has to check for runtime to make this work.

CC: Andrea Cervesato <andrea.cervesato@suse.de>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/io/ltp-aiodio/common.h       | 6 +++++-
 testcases/kernel/io/ltp-aiodio/dio_append.c   | 4 ++++
 testcases/kernel/io/ltp-aiodio/dio_truncate.c | 6 ++++++
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/io/ltp-aiodio/common.h b/testcases/kernel/io/ltp-aiodio/common.h
index 0a242b91e..d9cbd8611 100644
--- a/testcases/kernel/io/ltp-aiodio/common.h
+++ b/testcases/kernel/io/ltp-aiodio/common.h
@@ -44,9 +44,13 @@ static inline void io_append(const char *path, char pattern, int flags, size_t b
 
 	fd = SAFE_OPEN(path, flags, 0666);
 
-	for (i = 0; i < bcount; i++)
+	for (i = 0; i < bcount; i++) {
 		SAFE_WRITE(1, fd, bufptr, bs);
 
+		if (!tst_remaining_runtime())
+			break;
+	}
+
 	free(bufptr);
 	SAFE_CLOSE(fd);
 }
diff --git a/testcases/kernel/io/ltp-aiodio/dio_append.c b/testcases/kernel/io/ltp-aiodio/dio_append.c
index c099793f6..4ae8b5e33 100644
--- a/testcases/kernel/io/ltp-aiodio/dio_append.c
+++ b/testcases/kernel/io/ltp-aiodio/dio_append.c
@@ -71,6 +71,9 @@ static void run(void)
 
 	io_append(filename, 0, O_DIRECT | O_WRONLY | O_CREAT, writesize, appends);
 
+	if (!tst_remaining_runtime())
+		tst_res(TINFO, "Test out of runtime, exitting");
+
 	if (SAFE_WAITPID(-1, &status, WNOHANG))
 		tst_res(TFAIL, "Non zero bytes read");
 	else
@@ -87,6 +90,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
+	.max_runtime = 1800,
 	.options = (struct tst_option[]) {
 		{"n:", &str_numchildren, "Number of processes (default 16)"},
 		{"w:", &str_writesize, "Write size for each append (default 64K)"},
diff --git a/testcases/kernel/io/ltp-aiodio/dio_truncate.c b/testcases/kernel/io/ltp-aiodio/dio_truncate.c
index 1fbf83de0..b6f28ea71 100644
--- a/testcases/kernel/io/ltp-aiodio/dio_truncate.c
+++ b/testcases/kernel/io/ltp-aiodio/dio_truncate.c
@@ -142,6 +142,11 @@ static void run(void)
 			fail = 1;
 			break;
 		}
+
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Test out of runtime, exitting");
+			break;
+		}
 	}
 
 	if (fail)
@@ -158,6 +163,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
+	.max_runtime = 1800,
 	.options = (struct tst_option[]) {
 		{"n:", &str_numchildren, "Number of threads (default 16)"},
 		{"s:", &str_filesize, "Size of file (default 64K)"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (24 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 25/29] ltp-aiodio/dio_{truncate, append}: Convert to runtime Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-24  8:36   ` Petr Vorel
  2022-05-12 12:38 ` [LTP] [PATCH v3 27/29] timer_test: " Cyril Hrubis
                   ` (5 subsequent siblings)
  31 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

And cap the runtime on 30 minutes by default.

CC: Andrea Cervesato <andrea.cervesato@suse.de>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 testcases/kernel/io/ltp-aiodio/dio_read.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/testcases/kernel/io/ltp-aiodio/dio_read.c b/testcases/kernel/io/ltp-aiodio/dio_read.c
index f6ed59782..d28d474db 100644
--- a/testcases/kernel/io/ltp-aiodio/dio_read.c
+++ b/testcases/kernel/io/ltp-aiodio/dio_read.c
@@ -51,6 +51,9 @@ static void do_buffered_writes(int fd, char *bufptr, long long fsize, long long
 			tst_brk(TBROK, "pwrite: wrote %lld bytes out of %lld", w, wsize);
 
 		SAFE_FSYNC(fd);
+
+		if (!tst_remaining_runtime())
+			return;
 	}
 }
 
@@ -75,6 +78,11 @@ static int do_direct_reads(char *filename, char *bufptr, long long fsize, long l
 				goto exit;
 			}
 
+			if (!tst_remaining_runtime()) {
+				tst_res(TINFO, "Test out of runtime, exitting");
+				goto exit;
+			}
+
 			w = pread(fd, bufptr, rsize, offset);
 			if (w < 0)
 				tst_brk(TBROK, "pread: %s", tst_strerrno(-w));
@@ -170,6 +178,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
+	.max_runtime = 1800,
 	.options = (struct tst_option[]) {
 		{"n:", &str_numchildren, "Number of threads (default 8)"},
 		{"w:", &str_writesize, "Size of writing blocks (default 32M)"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 27/29] timer_test: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (25 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 28/29] ltp-aiodio/aiodio_append: " Cyril Hrubis
                   ` (4 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 lib/tst_timer_test.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/tst_timer_test.c b/lib/tst_timer_test.c
index 32fa55c7c..ef9b24d11 100644
--- a/lib/tst_timer_test.c
+++ b/lib/tst_timer_test.c
@@ -420,6 +420,9 @@ static struct tst_option options[] = {
 
 static void parse_timer_opts(void)
 {
+	size_t i;
+	long long runtime_us = 0;
+
 	if (str_sleep_time) {
 		if (tst_parse_int(str_sleep_time, &sleep_time, 0, INT_MAX)) {
 			tst_brk(TBROK,
@@ -441,14 +444,17 @@ static void parse_timer_opts(void)
 		if (!sample_cnt)
 			sample_cnt = 500;
 
-		long long timeout = sleep_time * sample_cnt / 1000000;
-
-		tst_set_timeout(timeout + timeout/10);
+		runtime_us = sleep_time * sample_cnt;
 
 		test->test_all = single_timer_test;
 		test->test = NULL;
 		test->tcnt = 0;
+	} else {
+		for (i = 0; i < ARRAY_SIZE(tcases); i++)
+			runtime_us += tcases[i].usec * tcases[i].samples;
 	}
+
+	tst_set_max_runtime((runtime_us + runtime_us/10)/1000000);
 }
 
 struct tst_test *tst_timer_test_setup(struct tst_test *timer_test)
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 28/29] ltp-aiodio/aiodio_append: Convert to runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (26 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 27/29] timer_test: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-12 12:38 ` [LTP] [PATCH v3 29/29] tst_test: Remove timeout stubs Cyril Hrubis
                   ` (3 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Cap the test runtime on 30 minutes by default.

For this to work we have to run the process that issues the aio requests
in a child process and kill it when runtime is over.

CC: Andrea Cervesato <andrea.cervesato@suse.de>
Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 .../kernel/io/ltp-aiodio/aiodio_append.c      | 25 ++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/testcases/kernel/io/ltp-aiodio/aiodio_append.c b/testcases/kernel/io/ltp-aiodio/aiodio_append.c
index 46cc74ee4..bf1665e0c 100644
--- a/testcases/kernel/io/ltp-aiodio/aiodio_append.c
+++ b/testcases/kernel/io/ltp-aiodio/aiodio_append.c
@@ -141,7 +141,7 @@ static void run(void)
 {
 	char *filename = "aiodio_append";
 	int status;
-	int i;
+	int i, pid;
 
 	*run_child = 1;
 
@@ -152,9 +152,27 @@ static void run(void)
 		}
 	}
 
-	tst_res(TINFO, "Parent append to file");
+	pid = SAFE_FORK();
+	if (!pid) {
+		aiodio_append(filename, appends, alignment, writesize, numaio);
+		return;
+	}
+
+	tst_res(TINFO, "Child %i appends to a file", pid);
+
+	for (;;) {
+		if (SAFE_WAITPID(pid, NULL, WNOHANG))
+			break;
 
-	aiodio_append(filename, appends, alignment, writesize, numaio);
+		sleep(1);
+
+		if (!tst_remaining_runtime()) {
+			tst_res(TINFO, "Test out of runtime, exitting");
+			kill(pid, SIGKILL);
+			SAFE_WAITPID(pid, NULL, 0);
+			break;
+		}
+	}
 
 	if (SAFE_WAITPID(-1, &status, WNOHANG))
 		tst_res(TFAIL, "Non zero bytes read");
@@ -172,6 +190,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
+	.max_runtime = 1800,
 	.options = (struct tst_option[]) {
 		{"n:", &str_numchildren, "Number of threads (default 16)"},
 		{"s:", &str_writesize, "Size of the file to write (default 64K)"},
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH v3 29/29] tst_test: Remove timeout stubs
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (27 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 28/29] ltp-aiodio/aiodio_append: " Cyril Hrubis
@ 2022-05-12 12:38 ` Cyril Hrubis
  2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
                   ` (2 subsequent siblings)
  31 siblings, 0 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-12 12:38 UTC (permalink / raw)
  To: ltp; +Cc: automated-testing

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
---
 include/tst_test.h |  4 ----
 lib/tst_test.c     | 11 -----------
 2 files changed, 15 deletions(-)

diff --git a/include/tst_test.h b/include/tst_test.h
index 01be56cd8..60316092d 100644
--- a/include/tst_test.h
+++ b/include/tst_test.h
@@ -236,8 +236,6 @@ struct tst_test {
 	unsigned int mnt_flags;
 	void *mnt_data;
 
-	/* override default timeout per test run, disabled == -1 */
-	int timeout;
 	/*
 	 * Maximal test runtime in seconds.
 	 *
@@ -333,9 +331,7 @@ const char *tst_strsig(int sig);
  */
 const char *tst_strstatus(int status);
 
-unsigned int tst_timeout_remaining(void);
 unsigned int tst_multiply_timeout(unsigned int timeout);
-void tst_set_timeout(int timeout);
 
 /*
  * Returns remaining test runtime. Test that runs for more than a few seconds
diff --git a/lib/tst_test.c b/lib/tst_test.c
index ef2e614bb..dad8aad92 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -1471,12 +1471,6 @@ static void sigint_handler(int sig LTP_ATTRIBUTE_UNUSED)
 	}
 }
 
-unsigned int tst_timeout_remaining(void)
-{
-	tst_brk(TBROK, "Stub called!");
-	return 0;
-}
-
 unsigned int tst_remaining_runtime(void)
 {
 	static struct timespec now;
@@ -1509,11 +1503,6 @@ unsigned int tst_multiply_timeout(unsigned int timeout)
 	return timeout * timeout_mul;
 }
 
-void tst_set_timeout(int timeout)
-{
-	tst_brk(TBROK, "Stub called!");
-}
-
 static void set_timeout(void)
 {
 	unsigned int timeout = DEFAULT_TIMEOUT;
-- 
2.35.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-12 12:38 ` [LTP] [PATCH v3 22/29] fuzzy_sync: " Cyril Hrubis
@ 2022-05-13  9:13   ` Li Wang
  2022-05-13 12:20     ` Cyril Hrubis
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-13  9:13 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: Richard Palethorpe, LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 4031 bytes --]

Cyril Hrubis <chrubis@suse.cz> wrote:


>
> diff --git a/include/tst_fuzzy_sync.h b/include/tst_fuzzy_sync.h
> index 4f09ed416..bef424002 100644
> --- a/include/tst_fuzzy_sync.h
> +++ b/include/tst_fuzzy_sync.h
> @@ -160,15 +160,6 @@ struct tst_fzsync_pair {
>         int b_cntr;
>         /** Internal; Used by tst_fzsync_pair_exit() and
> fzsync_pair_wait() */
>         int exit;
> -       /**
> -        * The maximum desired execution time as a proportion of the
> timeout
> -        *
> -        * A value x so that 0 < x < 1 which decides how long the test
> should
> -        * be run for (assuming the loop limit is not exceeded first).
> -        *
> -        * Defaults to 0.5 (~150 seconds with default timeout).
> -        */
> -       float exec_time_p;
>         /** Internal; The test time remaining on tst_fzsync_pair_reset() */
>         float exec_time_start;
>         /**
> @@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct
> tst_fzsync_pair *pair)
>         CHK(avg_alpha, 0, 1, 0.25);
>         CHK(min_samples, 20, INT_MAX, 1024);
>         CHK(max_dev_ratio, 0, 1, 0.1);
> -       CHK(exec_time_p, 0, 1, 0.5);
>         CHK(exec_loops, 20, INT_MAX, 3000000);
>
>         if (tst_ncpus_available() <= 1)
> @@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct
> tst_fzsync_pair *pair,
>         if (run_b)
>                 SAFE_PTHREAD_CREATE(&pair->thread_b, 0, run_b, 0);
>
> -       pair->exec_time_start = (float)tst_timeout_remaining();
> +       pair->exec_time_start = (float)tst_remaining_runtime();
>  }
>
>  /**
> @@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(struct
> tst_fzsync_pair *pair)
>   */
>  static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
>  {
> -       float rem_p = 1 - tst_timeout_remaining() / pair->exec_time_start;
> +       float rem_p = 1 - tst_remaining_runtime() / pair->exec_time_start;
>
> -       if ((pair->exec_time_p * SAMPLING_SLICE < rem_p)
> -               && (pair->sampling > 0)) {
> +       if ((SAMPLING_SLICE < rem_p) && (pair->sampling > 0)) {
>                 tst_res(TINFO, "Stopped sampling at %d (out of %d)
> samples, "
>                         "sampling time reached 50%% of the total time
> limit",
>                         pair->exec_loop, pair->min_samples);
> @@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct
> tst_fzsync_pair *pair)
>                 tst_fzsync_pair_info(pair);
>         }
>
> -       if (pair->exec_time_p < rem_p) {
> +       if (rem_p >= 1) {
>

I hit a new problem while testing new pty03, that seems here
will fall into an infinite loop and test timed out finally. The printf
shows rem_p will be overflow I haven't figured out why.

But with comparing with 0.9, it always gets passed on to the same system.

--- a/include/tst_fuzzy_sync.h
+++ b/include/tst_fuzzy_sync.h
@@ -644,7 +644,7 @@ static inline int tst_fzsync_run_a(struct
tst_fzsync_pair *pair)
                tst_fzsync_pair_info(pair);
        }

-       if (rem_p >= 1) {
+       if (rem_p >= 0.9) {
                tst_res(TINFO,
                        "Exceeded execution time, requesting exit");
                tst_atomic_store(1, &pair->exit);

----------------------
# ./pty03
...
../../../include/tst_fuzzy_sync.h:295: TINFO: start_a - start_b: { avg =
-90ns, avg_dev =     8ns, dev_ratio = 0.09 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_a - start_a  : { avg =
 2606ns, avg_dev =   127ns, dev_ratio = 0.05 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_b - start_b  : { avg =
 8785ns, avg_dev =   600ns, dev_ratio = 0.07 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_a - end_b    : { avg =
-6269ns, avg_dev =   566ns, dev_ratio = 0.09 }
../../../include/tst_fuzzy_sync.h:295: TINFO: spins            : { avg =
 3269  , avg_dev =   200  , dev_ratio = 0.06 }
Test timeouted, sending SIGKILL!
tst_test.c:1573: TINFO: If you are running on slow machine, try exporting
LTP_TIMEOUT_MUL > 1
tst_test.c:1575: TBROK: Test killed! (timeout?)


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 5713 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-13  9:13   ` Li Wang
@ 2022-05-13 12:20     ` Cyril Hrubis
  2022-05-13 13:08       ` Li Wang
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-13 12:20 UTC (permalink / raw)
  To: Li Wang; +Cc: Richard Palethorpe, LTP List, automated-testing

Hi!
> > diff --git a/include/tst_fuzzy_sync.h b/include/tst_fuzzy_sync.h
> > index 4f09ed416..bef424002 100644
> > --- a/include/tst_fuzzy_sync.h
> > +++ b/include/tst_fuzzy_sync.h
> > @@ -160,15 +160,6 @@ struct tst_fzsync_pair {
> >         int b_cntr;
> >         /** Internal; Used by tst_fzsync_pair_exit() and
> > fzsync_pair_wait() */
> >         int exit;
> > -       /**
> > -        * The maximum desired execution time as a proportion of the
> > timeout
> > -        *
> > -        * A value x so that 0 < x < 1 which decides how long the test
> > should
> > -        * be run for (assuming the loop limit is not exceeded first).
> > -        *
> > -        * Defaults to 0.5 (~150 seconds with default timeout).
> > -        */
> > -       float exec_time_p;
> >         /** Internal; The test time remaining on tst_fzsync_pair_reset() */
> >         float exec_time_start;
> >         /**
> > @@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct
> > tst_fzsync_pair *pair)
> >         CHK(avg_alpha, 0, 1, 0.25);
> >         CHK(min_samples, 20, INT_MAX, 1024);
> >         CHK(max_dev_ratio, 0, 1, 0.1);
> > -       CHK(exec_time_p, 0, 1, 0.5);
> >         CHK(exec_loops, 20, INT_MAX, 3000000);
> >
> >         if (tst_ncpus_available() <= 1)
> > @@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct
> > tst_fzsync_pair *pair,
> >         if (run_b)
> >                 SAFE_PTHREAD_CREATE(&pair->thread_b, 0, run_b, 0);
> >
> > -       pair->exec_time_start = (float)tst_timeout_remaining();
> > +       pair->exec_time_start = (float)tst_remaining_runtime();
> >  }
> >
> >  /**
> > @@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(struct
> > tst_fzsync_pair *pair)
> >   */
> >  static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
> >  {
> > -       float rem_p = 1 - tst_timeout_remaining() / pair->exec_time_start;
> > +       float rem_p = 1 - tst_remaining_runtime() / pair->exec_time_start;
> >
> > -       if ((pair->exec_time_p * SAMPLING_SLICE < rem_p)
> > -               && (pair->sampling > 0)) {
> > +       if ((SAMPLING_SLICE < rem_p) && (pair->sampling > 0)) {
> >                 tst_res(TINFO, "Stopped sampling at %d (out of %d)
> > samples, "
> >                         "sampling time reached 50%% of the total time
> > limit",
> >                         pair->exec_loop, pair->min_samples);
> > @@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct
> > tst_fzsync_pair *pair)
> >                 tst_fzsync_pair_info(pair);
> >         }
> >
> > -       if (pair->exec_time_p < rem_p) {
> > +       if (rem_p >= 1) {
> >
> 
> I hit a new problem while testing new pty03, that seems here
> will fall into an infinite loop and test timed out finally. The printf
> shows rem_p will be overflow I haven't figured out why.
> 
> But with comparing with 0.9, it always gets passed on to the same system.

That is strange, since we do:

	rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;


And the tst_remaining_runtime() should return 0 once the time is up so
the end result should be that rem_p will end up 1 sooner or later.

Anyways we can as well use the value from tst_remainig_runtime()
directly instead as:

diff --git a/include/tst_fuzzy_sync.h b/include/tst_fuzzy_sync.h
index bef424002..db5bec7a4 100644
--- a/include/tst_fuzzy_sync.h
+++ b/include/tst_fuzzy_sync.h
@@ -634,7 +634,8 @@ static inline void tst_fzsync_wait_b(struct tst_fzsync_pair *pair)
  */
 static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
 {
-       float rem_p = 1 - tst_remaining_runtime() / pair->exec_time_start;
+       int remaining_runtime = tst_remaining_runtime();
+       float rem_p = 1 - remaining_runtime / pair->exec_time_start;

        if ((SAMPLING_SLICE < rem_p) && (pair->sampling > 0)) {
                tst_res(TINFO, "Stopped sampling at %d (out of %d) samples, "
@@ -644,7 +645,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
                tst_fzsync_pair_info(pair);
        }

-       if (rem_p >= 1) {
+       if (!remaining_runtime) {
                tst_res(TINFO,
                        "Exceeded execution time, requesting exit");
                tst_atomic_store(1, &pair->exit);


Does that fix your problem?

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-13 12:20     ` Cyril Hrubis
@ 2022-05-13 13:08       ` Li Wang
  2022-05-13 14:22         ` Cyril Hrubis
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-13 13:08 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: Richard Palethorpe, LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 3306 bytes --]

Cyril Hrubis <chrubis@suse.cz> wrote:


> > > -       if (pair->exec_time_p < rem_p) {
> > > +       if (rem_p >= 1) {
> > >
> >
> > I hit a new problem while testing new pty03, that seems here
> > will fall into an infinite loop and test timed out finally. The printf
> > shows rem_p will be overflow I haven't figured out why.
> >
> > But with comparing with 0.9, it always gets passed on to the same system.
>
> That is strange, since we do:
>
>         rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
>

I guess the root cause is that 'pair->time_exec_start' has a possibility
to reach zero. in pty03 it has ".tcnt = 9" which made the
tst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will
be set only based on the original .max_runtime, with time elapsed the
remaining
time tends to be zero.


debug log
--------------
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 0.000000
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 0.000000
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 1.000000
../../../include/tst_fuzzy_sync.h:654: TINFO: Exceeded execution time,
requesting exit
pty03.c:132: TPASS: Did not crash with PPS TTY discipline
pty03.c:106: TINFO: Creating PTY with CAIF line discipline
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan
pty03.c:91: TCONF: You don't appear to have the CAIF TTY line discipline:
EINVAL (22)
pty03.c:106: TINFO: Creating PTY with GSM line discipline
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan
../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan



>
>
> And the tst_remaining_runtime() should return 0 once the time is up so
> the end result should be that rem_p will end up 1 sooner or later.
>

Yes, the pair->time_exec_start will be reset to 0 as well then
becomes the denominator. That makes rem_p get an overflowed
negative number, then the test falls into an infinite loop from this moment.



>
> Anyways we can as well use the value from tst_remainig_runtime()
> directly instead as:


> diff --git a/include/tst_fuzzy_sync.h b/include/tst_fuzzy_sync.h
> index bef424002..db5bec7a4 100644
> --- a/include/tst_fuzzy_sync.h
> +++ b/include/tst_fuzzy_sync.h
> @@ -634,7 +634,8 @@ static inline void tst_fzsync_wait_b(struct
> tst_fzsync_pair *pair)
>   */
>  static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)
>  {
> -       float rem_p = 1 - tst_remaining_runtime() / pair->exec_time_start;
> +       int remaining_runtime = tst_remaining_runtime();
> +       float rem_p = 1 - remaining_runtime / pair->exec_time_start;
>
>         if ((SAMPLING_SLICE < rem_p) && (pair->sampling > 0)) {
>                 tst_res(TINFO, "Stopped sampling at %d (out of %d)
> samples, "
> @@ -644,7 +645,7 @@ static inline int tst_fzsync_run_a(struct
> tst_fzsync_pair *pair)
>                 tst_fzsync_pair_info(pair);
>         }
>
> -       if (rem_p >= 1) {
> +       if (!remaining_runtime) {
>                 tst_res(TINFO,
>                         "Exceeded execution time, requesting exit");
>                 tst_atomic_store(1, &pair->exit);
>
>
> Does that fix your problem?
>

This amendment is more readable but does not solve the original problem.

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 5428 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-13 13:08       ` Li Wang
@ 2022-05-13 14:22         ` Cyril Hrubis
  2022-05-14  2:59           ` Li Wang
  2022-05-16  7:52           ` Richard Palethorpe
  0 siblings, 2 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-13 14:22 UTC (permalink / raw)
  To: Li Wang; +Cc: Richard Palethorpe, LTP List, automated-testing

Hi!
> > > I hit a new problem while testing new pty03, that seems here
> > > will fall into an infinite loop and test timed out finally. The printf
> > > shows rem_p will be overflow I haven't figured out why.
> > >
> > > But with comparing with 0.9, it always gets passed on to the same system.
> >
> > That is strange, since we do:
> >
> >         rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
> >
> 
> I guess the root cause is that 'pair->time_exec_start' has a possibility
> to reach zero. in pty03 it has ".tcnt = 9" which made the
> tst_fzsync_pair_reset()
> to be re-run many times, but in that function 'pair->time_exec_start' will
> be set only based on the original .max_runtime, with time elapsed the
> remaining time tends to be zero.

I guess that that the interaction of tcnt and runtime is not optimal
here. You are right that as long as we call tst_fzsync_pair_reset() on
each invocation of the run() function we may eventually get to state
where the runtime is exhausted, especially on slower hardware we end up
with division by zero and overflow.

The cleanest solution would be to rewrite the test to use .test_variants = 9
and setting the .max_runtime to a smaller value. That way we would have
precisely defined runtime for each iteration. What do you think?

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-13 14:22         ` Cyril Hrubis
@ 2022-05-14  2:59           ` Li Wang
  2022-05-16  7:52           ` Richard Palethorpe
  1 sibling, 0 replies; 61+ messages in thread
From: Li Wang @ 2022-05-14  2:59 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: Richard Palethorpe, LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 2264 bytes --]

On Fri, May 13, 2022 at 10:20 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > > > I hit a new problem while testing new pty03, that seems here
> > > > will fall into an infinite loop and test timed out finally. The
> printf
> > > > shows rem_p will be overflow I haven't figured out why.
> > > >
> > > > But with comparing with 0.9, it always gets passed on to the same
> system.
> > >
> > > That is strange, since we do:
> > >
> > >         rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
> > >
> >
> > I guess the root cause is that 'pair->time_exec_start' has a possibility
> > to reach zero. in pty03 it has ".tcnt = 9" which made the
> > tst_fzsync_pair_reset()
> > to be re-run many times, but in that function 'pair->time_exec_start'
> will
> > be set only based on the original .max_runtime, with time elapsed the
> > remaining time tends to be zero.
>
> I guess that that the interaction of tcnt and runtime is not optimal
> here. You are right that as long as we call tst_fzsync_pair_reset() on
> each invocation of the run() function we may eventually get to state
> where the runtime is exhausted, especially on slower hardware we end up
> with division by zero and overflow.
>
> The cleanest solution would be to rewrite the test to use .test_variants =
> 9
> and setting the .max_runtime to a smaller value. That way we would have
> precisely defined runtime for each iteration. What do you think?
>

Yes, that should work, but people have to painstakingly remember
that to avoid this interaction of tcnt and runtime in test writing.

Another simple choice I can think of is to use tst_set_max_runtime()
to reset both 'timeout' and 'runtime' at the beginning of each invocation
of the run().

And maybe we should invoke it in the tst_fzsync_pair_reset directly?
I'm not very sure about fixing this in fzsync-lib or out.

--- a/testcases/kernel/pty/pty03.c
+++ b/testcases/kernel/pty/pty03.c
@@ -105,6 +105,7 @@ static void do_test(unsigned int n)

        tst_res(TINFO, "Creating PTY with %s line discipline", ldisc->name);

+       tst_set_max_runtime(test.runtime);
        tst_fzsync_pair_reset(&fzp, hangup);
        while (tst_fzsync_run_a(&fzp)) {
                ptmx = SAFE_OPEN("/dev/ptmx", O_RDONLY);


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 3838 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 01/29] Introduce a concept of max runtime
  2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
@ 2022-05-14  4:21   ` Li Wang
  2022-05-24  9:25   ` [LTP] [Automated-testing] " Richard Palethorpe
  1 sibling, 0 replies; 61+ messages in thread
From: Li Wang @ 2022-05-14  4:21 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 3851 bytes --]

On Thu, May 12, 2022 at 8:36 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> This commit introduces a concept of max test runtime. In other words
> test runtime is capped at a certain value in order to make testruns more
> deterministic. Test is free to to finish before the runtime is used up,
> for example when maximal number of iterations was reached, but test must
> stop once the runtime has been used up.
>
> Testcases that run for more than a second or two must check for
> remaining runtime by regular calls to tst_remaining_runtime() and should
> exit when zero is returned.
>
> The test max runtime must be set either by the .max_iteration_runtime in
>

.max_runtime



> the tst_test structure or in the test setup by a call to
> tst_set_runtime().
>

tst_set_max_runtime()



>
> The test timeout is then computed as a sum of DEFAULT_TIMEOUT (currently
> set to 30 seconds) and the test runtime. The DEFAULT_TIMEOUT is nothing
> more than a safety margin for teardown of the test.
>
> This commit also maps the -I parameter to the test max runtime if
> available and introduces LTP_RUNTIME_MUL enviroment variable so that we
> have an easy controll over the runtime cap.
>
> Lastly but not least the function related to the timeout are turned into
> no-op by this commit and removed after all test are converted to the
> runtime API.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
> ---
>  doc/c-test-api.txt                       |  42 +++++--
>  doc/user-guide.txt                       |  30 ++++-
>  include/tst_test.h                       |  27 ++++
>  lib/newlib_tests/.gitignore              |   5 +-
>  lib/newlib_tests/runtest.sh              |   2 +-
>  lib/newlib_tests/test10.c                |  22 ----
>  lib/newlib_tests/test12.c                |  21 ----
>  lib/newlib_tests/test13.c                |   1 -
>  lib/newlib_tests/test18.c                |  22 ----
>  lib/newlib_tests/test_children_cleanup.c |   1 -
>  lib/newlib_tests/test_runtime01.c        |  30 +++++
>  lib/newlib_tests/test_runtime02.c        |  28 +++++
>  lib/tst_test.c                           | 150 +++++++++++++++++------
>  13 files changed, 264 insertions(+), 117 deletions(-)
>  delete mode 100644 lib/newlib_tests/test10.c
>  delete mode 100644 lib/newlib_tests/test12.c
>  delete mode 100644 lib/newlib_tests/test18.c
>  create mode 100644 lib/newlib_tests/test_runtime01.c
>  create mode 100644 lib/newlib_tests/test_runtime02.c
>
> diff --git a/doc/c-test-api.txt b/doc/c-test-api.txt
> index 9f104ecd7..711b445d9 100644
> --- a/doc/c-test-api.txt
> +++ b/doc/c-test-api.txt
> @@ -93,14 +93,35 @@ in range of [0, '.tcnt' - 1].
>
>  IMPORTANT: Only one of '.test' and '.test_all' can be set at a time.
>
> -Each test has a default timeout set to 300s. The default timeout can be
> -overridden by setting '.timeout' in the test structure or by calling
> -'tst_set_timeout()' in the test 'setup()'. There are a few testcases
> whose run
> -time may vary arbitrarily, for these timeout can be disabled by setting
> it to
> --1.
> +Each test has a limit on how long it can run and the limit composes of two
> +parts max_runtime and timeout. The max_runtime is a limit for how long
> can the
> +'.test_all' or a set of '.test' functions take and the timeout is static
> part
> +that should cover the duration of test setup and cleanup plus some safety.
>
> -Test can find out how much time (in seconds) is remaining to timeout,
> -by calling 'tst_timeout_remaining()'.
> +Any test that runs for more than a second or two has to make sure to:
> +
> +- set the runtime either by setting the '.max_runtime' in tst_test or by
> +  calling 'tst_set_runtime()' in the test setup
>

     ^ calling 'tst_set_max_runtime()'

Apart from the patch-22 tiny issue, the rest part looks good.

Reviewed-by: Li Wang <liwang@redhat.com>


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 5695 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-13 14:22         ` Cyril Hrubis
  2022-05-14  2:59           ` Li Wang
@ 2022-05-16  7:52           ` Richard Palethorpe
  2022-05-17 10:56             ` Cyril Hrubis
  1 sibling, 1 reply; 61+ messages in thread
From: Richard Palethorpe @ 2022-05-16  7:52 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing

Hello,

Cyril Hrubis <chrubis@suse.cz> writes:

> Hi!
>> > > I hit a new problem while testing new pty03, that seems here
>> > > will fall into an infinite loop and test timed out finally. The printf
>> > > shows rem_p will be overflow I haven't figured out why.
>> > >
>> > > But with comparing with 0.9, it always gets passed on to the same system.
>> >
>> > That is strange, since we do:
>> >
>> >         rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
>> >
>> 
>> I guess the root cause is that 'pair->time_exec_start' has a possibility
>> to reach zero. in pty03 it has ".tcnt = 9" which made the
>> tst_fzsync_pair_reset()
>> to be re-run many times, but in that function 'pair->time_exec_start' will
>> be set only based on the original .max_runtime, with time elapsed the
>> remaining time tends to be zero.
>
> I guess that that the interaction of tcnt and runtime is not optimal
> here. You are right that as long as we call tst_fzsync_pair_reset() on
> each invocation of the run() function we may eventually get to state
> where the runtime is exhausted, especially on slower hardware we end up
> with division by zero and overflow.
>
> The cleanest solution would be to rewrite the test to use .test_variants = 9
> and setting the .max_runtime to a smaller value. That way we would have
> precisely defined runtime for each iteration. What do you think?

Or each test case (defined by tcnt) could be given an equal share of the
runtime?

-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-16  7:52           ` Richard Palethorpe
@ 2022-05-17 10:56             ` Cyril Hrubis
  2022-05-17 11:07               ` [LTP] [Automated-testing] " Petr Vorel
  2022-05-17 12:28               ` [LTP] " Li Wang
  0 siblings, 2 replies; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-17 10:56 UTC (permalink / raw)
  To: Richard Palethorpe; +Cc: LTP List, automated-testing

Hi!
> >> I guess the root cause is that 'pair->time_exec_start' has a possibility
> >> to reach zero. in pty03 it has ".tcnt = 9" which made the
> >> tst_fzsync_pair_reset()
> >> to be re-run many times, but in that function 'pair->time_exec_start' will
> >> be set only based on the original .max_runtime, with time elapsed the
> >> remaining time tends to be zero.
> >
> > I guess that that the interaction of tcnt and runtime is not optimal
> > here. You are right that as long as we call tst_fzsync_pair_reset() on
> > each invocation of the run() function we may eventually get to state
> > where the runtime is exhausted, especially on slower hardware we end up
> > with division by zero and overflow.
> >
> > The cleanest solution would be to rewrite the test to use .test_variants = 9
> > and setting the .max_runtime to a smaller value. That way we would have
> > precisely defined runtime for each iteration. What do you think?
> 
> Or each test case (defined by tcnt) could be given an equal share of the
> runtime?

That would be solution as well. If everyone agrees on this I will go
with this version, it should be as easy as:

diff --git a/lib/tst_test.c b/lib/tst_test.c
index dad8aad92..69d7799c3 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -1349,6 +1349,8 @@ static void run_tests(void)

                if (results_equal(&saved_results, results))
                        tst_brk(TBROK, "Test %i haven't reported results!", i);
+
+               kill(getppid(), SIGUSR1);
        }
 }


With this we do reset the test timeout after each test iteration.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* Re: [LTP] [Automated-testing] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-17 10:56             ` Cyril Hrubis
@ 2022-05-17 11:07               ` Petr Vorel
  2022-05-17 12:28               ` [LTP] " Li Wang
  1 sibling, 0 replies; 61+ messages in thread
From: Petr Vorel @ 2022-05-17 11:07 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing

Hi all,

> Hi!
> > >> I guess the root cause is that 'pair->time_exec_start' has a possibility
> > >> to reach zero. in pty03 it has ".tcnt = 9" which made the
> > >> tst_fzsync_pair_reset()
> > >> to be re-run many times, but in that function 'pair->time_exec_start' will
> > >> be set only based on the original .max_runtime, with time elapsed the
> > >> remaining time tends to be zero.

> > > I guess that that the interaction of tcnt and runtime is not optimal
> > > here. You are right that as long as we call tst_fzsync_pair_reset() on
> > > each invocation of the run() function we may eventually get to state
> > > where the runtime is exhausted, especially on slower hardware we end up
> > > with division by zero and overflow.

> > > The cleanest solution would be to rewrite the test to use .test_variants = 9
> > > and setting the .max_runtime to a smaller value. That way we would have
> > > precisely defined runtime for each iteration. What do you think?

> > Or each test case (defined by tcnt) could be given an equal share of the
> > runtime?
LGTM

> That would be solution as well. If everyone agrees on this I will go
> with this version, it should be as easy as:

> diff --git a/lib/tst_test.c b/lib/tst_test.c
> index dad8aad92..69d7799c3 100644
> --- a/lib/tst_test.c
> +++ b/lib/tst_test.c
> @@ -1349,6 +1349,8 @@ static void run_tests(void)

>                 if (results_equal(&saved_results, results))
>                         tst_brk(TBROK, "Test %i haven't reported results!", i);
> +
> +               kill(getppid(), SIGUSR1);
>         }
>  }


> With this we do reset the test timeout after each test iteration.

+1.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-17 10:56             ` Cyril Hrubis
  2022-05-17 11:07               ` [LTP] [Automated-testing] " Petr Vorel
@ 2022-05-17 12:28               ` Li Wang
  2022-05-18  7:47                 ` Li Wang
  1 sibling, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-17 12:28 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 840 bytes --]

Cyril Hrubis <chrubis@suse.cz> wrote:


>
> > Or each test case (defined by tcnt) could be given an equal share of the
> > runtime?
>
> That would be solution as well. If everyone agrees on this I will go
> with this version, it should be as easy as:
>
> diff --git a/lib/tst_test.c b/lib/tst_test.c
> index dad8aad92..69d7799c3 100644
> --- a/lib/tst_test.c
> +++ b/lib/tst_test.c
> @@ -1349,6 +1349,8 @@ static void run_tests(void)
>
>                 if (results_equal(&saved_results, results))
>                         tst_brk(TBROK, "Test %i haven't reported
> results!", i);
> +
> +               kill(getppid(), SIGUSR1);
>

I have to say NO here.

This is only reset timeout for each test, the problem we're encountered
is the max_runtime exhausting, we have to reset 'results->max_runtime'
but not .timeout.

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 1702 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-17 12:28               ` [LTP] " Li Wang
@ 2022-05-18  7:47                 ` Li Wang
  2022-05-19  9:05                   ` Cyril Hrubis
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-18  7:47 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 1799 bytes --]

Li Wang <liwang@redhat.com> wrote:


> Cyril Hrubis <chrubis@suse.cz> wrote:
>
>
>>
>> > Or each test case (defined by tcnt) could be given an equal share of the
>> > runtime?
>>
>> That would be solution as well. If everyone agrees on this I will go
>> with this version, it should be as easy as:
>>
>> diff --git a/lib/tst_test.c b/lib/tst_test.c
>> index dad8aad92..69d7799c3 100644
>> --- a/lib/tst_test.c
>> +++ b/lib/tst_test.c
>> @@ -1349,6 +1349,8 @@ static void run_tests(void)
>>
>>                 if (results_equal(&saved_results, results))
>>                         tst_brk(TBROK, "Test %i haven't reported
>> results!", i);
>> +
>> +               kill(getppid(), SIGUSR1);
>>
>
> I have to say NO here.
>
> This is only reset timeout for each test, the problem we're encountered
> is the max_runtime exhausting, we have to reset 'results->max_runtime'
> but not .timeout.
>

Okay, my perspective is shortsighted as well.

This solution is correct in the direction but overlooks the significant
global value 'tst_start_time'. If that value reflush within tcnt loop we
don't need to reset max_runtime again, actually the real work in my
previous patch is to invoke heartbeat() which touches tst_start_time.

So I would suggest using heartbeat() instead of only sending SIGUSR1
to lib_pid. Or, do simply revision like:

--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -1349,6 +1349,10 @@ static void run_tests(void)

                if (results_equal(&saved_results, results))
                        tst_brk(TBROK, "Test %i haven't reported results!",
i);
+
+               if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
+                       tst_res(TWARN | TERRNO, "tst_clock_gettime()
failed");
+               kill(getppid(), SIGUSR1);
        }
 }


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 3684 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-18  7:47                 ` Li Wang
@ 2022-05-19  9:05                   ` Cyril Hrubis
  2022-05-19 10:02                     ` Li Wang
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-19  9:05 UTC (permalink / raw)
  To: Li Wang; +Cc: LTP List, automated-testing

Hi!
> Okay, my perspective is shortsighted as well.
> 
> This solution is correct in the direction but overlooks the significant
> global value 'tst_start_time'. If that value reflush within tcnt loop we
> don't need to reset max_runtime again, actually the real work in my
> previous patch is to invoke heartbeat() which touches tst_start_time.
> 
> So I would suggest using heartbeat() instead of only sending SIGUSR1
> to lib_pid. Or, do simply revision like:

Ah, right, I guess that we should just call heartbeat() before each
iteration of the test then.

So basically:

diff --git a/lib/tst_test.c b/lib/tst_test.c
index dad8aad92..f3090217b 100644
--- a/lib/tst_test.c
+++ b/lib/tst_test.c
@@ -1317,6 +1317,24 @@ static void do_cleanup(void)
 	cleanup_ipc();
 }
 
+static void heartbeat(void)
+{
+	if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
+		tst_res(TWARN | TERRNO, "tst_clock_gettime() failed");
+
+	if (getppid() == 1) {
+		tst_res(TFAIL, "Main test process might have exit!");
+		/*
+		 * We need kill the task group immediately since the
+		 * main process has exit.
+		 */
+		kill(0, SIGKILL);
+		exit(TBROK);
+	}
+
+	kill(getppid(), SIGUSR1);
+}
+
 static void run_tests(void)
 {
 	unsigned int i;
@@ -1324,6 +1342,7 @@ static void run_tests(void)
 
 	if (!tst_test->test) {
 		saved_results = *results;
+		heartbeat();
 		tst_test->test_all();
 
 		if (getpid() != main_pid) {
@@ -1339,6 +1358,7 @@ static void run_tests(void)
 
 	for (i = 0; i < tst_test->tcnt; i++) {
 		saved_results = *results;
+		heartbeat();
 		tst_test->test(i);
 
 		if (getpid() != main_pid) {
@@ -1349,6 +1369,8 @@ static void run_tests(void)
 
 		if (results_equal(&saved_results, results))
 			tst_brk(TBROK, "Test %i haven't reported results!", i);
+
+		kill(getppid(), SIGUSR1);
 	}
 }
 
@@ -1379,24 +1401,6 @@ static void add_paths(void)
 	free(new_path);
 }
 
-static void heartbeat(void)
-{
-	if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
-		tst_res(TWARN | TERRNO, "tst_clock_gettime() failed");
-
-	if (getppid() == 1) {
-		tst_res(TFAIL, "Main test process might have exit!");
-		/*
-		 * We need kill the task group immediately since the
-		 * main process has exit.
-		 */
-		kill(0, SIGKILL);
-		exit(TBROK);
-	}
-
-	kill(getppid(), SIGUSR1);
-}
-
 static void testrun(void)
 {
 	unsigned int i = 0;
@@ -1425,7 +1429,6 @@ static void testrun(void)
 			break;
 
 		run_tests();
-		heartbeat();
 	}
 
 	do_test_cleanup();


I guess that this should go in a separate patch on the top of the
"Introduce concept of max runtime".

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-19  9:05                   ` Cyril Hrubis
@ 2022-05-19 10:02                     ` Li Wang
  2022-05-19 10:27                       ` Cyril Hrubis
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-19 10:02 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 3510 bytes --]

On Thu, May 19, 2022 at 5:03 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > Okay, my perspective is shortsighted as well.
> >
> > This solution is correct in the direction but overlooks the significant
> > global value 'tst_start_time'. If that value reflush within tcnt loop we
> > don't need to reset max_runtime again, actually the real work in my
> > previous patch is to invoke heartbeat() which touches tst_start_time.
> >
> > So I would suggest using heartbeat() instead of only sending SIGUSR1
> > to lib_pid. Or, do simply revision like:
>
> Ah, right, I guess that we should just call heartbeat() before each
> iteration of the test then.
>
> So basically:
>
> diff --git a/lib/tst_test.c b/lib/tst_test.c
> index dad8aad92..f3090217b 100644
> --- a/lib/tst_test.c
> +++ b/lib/tst_test.c
> @@ -1317,6 +1317,24 @@ static void do_cleanup(void)
>         cleanup_ipc();
>  }
>
> +static void heartbeat(void)
> +{
> +       if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
> +               tst_res(TWARN | TERRNO, "tst_clock_gettime() failed");
> +
> +       if (getppid() == 1) {
> +               tst_res(TFAIL, "Main test process might have exit!");
> +               /*
> +                * We need kill the task group immediately since the
> +                * main process has exit.
> +                */
> +               kill(0, SIGKILL);
> +               exit(TBROK);
> +       }
> +
> +       kill(getppid(), SIGUSR1);
> +}
> +
>  static void run_tests(void)
>  {
>         unsigned int i;
> @@ -1324,6 +1342,7 @@ static void run_tests(void)
>
>         if (!tst_test->test) {
>                 saved_results = *results;
> +               heartbeat();
>                 tst_test->test_all();
>
>                 if (getpid() != main_pid) {
> @@ -1339,6 +1358,7 @@ static void run_tests(void)
>
>         for (i = 0; i < tst_test->tcnt; i++) {
>                 saved_results = *results;
> +               heartbeat();
>                 tst_test->test(i);
>
>                 if (getpid() != main_pid) {
> @@ -1349,6 +1369,8 @@ static void run_tests(void)
>
>                 if (results_equal(&saved_results, results))
>                         tst_brk(TBROK, "Test %i haven't reported
> results!", i);
> +
> +               kill(getppid(), SIGUSR1);
>

As we already invoke heartbeat() at the beginning of each tcnt loop,
why are we still sending SIGUSR1 here?

Otherwise this version looks good to me.



>         }
>  }
>
> @@ -1379,24 +1401,6 @@ static void add_paths(void)
>         free(new_path);
>  }
>
> -static void heartbeat(void)
> -{
> -       if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
> -               tst_res(TWARN | TERRNO, "tst_clock_gettime() failed");
> -
> -       if (getppid() == 1) {
> -               tst_res(TFAIL, "Main test process might have exit!");
> -               /*
> -                * We need kill the task group immediately since the
> -                * main process has exit.
> -                */
> -               kill(0, SIGKILL);
> -               exit(TBROK);
> -       }
> -
> -       kill(getppid(), SIGUSR1);
> -}
> -
>  static void testrun(void)
>  {
>         unsigned int i = 0;
> @@ -1425,7 +1429,6 @@ static void testrun(void)
>                         break;
>
>                 run_tests();
> -               heartbeat();
>         }
>
>         do_test_cleanup();
>
>
> I guess that this should go in a separate patch on the top of the
> "Introduce concept of max runtime".
>

Agree.


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 5307 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-19 10:02                     ` Li Wang
@ 2022-05-19 10:27                       ` Cyril Hrubis
  2022-05-19 10:29                         ` Li Wang
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-19 10:27 UTC (permalink / raw)
  To: Li Wang; +Cc: LTP List, automated-testing

Hi!
> > +
> > +               kill(getppid(), SIGUSR1);
> >
> 
> As we already invoke heartbeat() at the beginning of each tcnt loop,
> why are we still sending SIGUSR1 here?

My bad, I did forget to remove it.

> > I guess that this should go in a separate patch on the top of the
> > "Introduce concept of max runtime".
> >
> 
> Agree.

Should I send a V4 with this patch included?

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-19 10:27                       ` Cyril Hrubis
@ 2022-05-19 10:29                         ` Li Wang
  2022-05-19 12:01                           ` Cyril Hrubis
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-19 10:29 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing


[-- Attachment #1.1: Type: text/plain, Size: 237 bytes --]

Cyril Hrubis <chrubis@suse.cz> wrote:


>
> Should I send a V4 with this patch included?
>

No need. You can push with this new patch adding,
then I will kick start release testing for the whole LTP
from our side.


-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 873 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-19 10:29                         ` Li Wang
@ 2022-05-19 12:01                           ` Cyril Hrubis
  2022-05-24  9:24                             ` Richard Palethorpe
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-19 12:01 UTC (permalink / raw)
  To: Li Wang; +Cc: LTP List, automated-testing

Hi!
> No need. You can push with this new patch adding,
> then I will kick start release testing for the whole LTP
> from our side.

Great, please do so.

Andrea is going to send small fix to the aiodio tests that should be
merged as well, otherwise these may fail rarely.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (28 preceding siblings ...)
  2022-05-12 12:38 ` [LTP] [PATCH v3 29/29] tst_test: Remove timeout stubs Cyril Hrubis
@ 2022-05-20  8:36 ` Li Wang
  2022-05-20  8:36   ` [LTP] [PATCH 1/2] testcases: make use of runtime Li Wang
                     ` (2 more replies)
  2022-05-24  8:39 ` [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Petr Vorel
  2022-05-24  8:41 ` Petr Vorel
  31 siblings, 3 replies; 61+ messages in thread
From: Li Wang @ 2022-05-20  8:36 UTC (permalink / raw)
  To: ltp

I run a quick test for verifying those patches with the latest mainline kernel.
Apart from these two tiny issues, the result looks good.

  kernel: 5.18.0-rc7
  arch: x86_64, ppc64le, s390x, aarch64
  git-repo: https://github.com/wangli5665/ltp

Ps. I'm going to submit more tests against different RHEL products later.

Li Wang (2):
  testcases: make use of runtime
  mmap3: reset runtime to 10s

 runtest/mm                            | 2 +-
 testcases/cve/cve-2014-0196.c         | 1 +
 testcases/cve/cve-2016-7117.c         | 1 +
 testcases/cve/cve-2017-2671.c         | 1 +
 testcases/kernel/mem/mtest06/mmap3.c  | 2 +-
 testcases/network/can/cve/can_bcm01.c | 1 +
 testcases/network/packet/fanout01.c   | 1 +
 testcases/network/sockets/vsock01.c   | 1 +
 8 files changed, 8 insertions(+), 2 deletions(-)

-- 
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* [LTP] [PATCH 1/2] testcases: make use of runtime
  2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
@ 2022-05-20  8:36   ` Li Wang
  2022-05-20 11:03     ` Cyril Hrubis
  2022-05-20  8:36   ` [LTP] [PATCH 2/2] mmap3: reset runtime to 10s Li Wang
  2022-05-20 12:18   ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Cyril Hrubis
  2 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-20  8:36 UTC (permalink / raw)
  To: ltp

If test using the fzsync lib framework, it has to be set with .max_runtime.

Signed-off-by: Li Wang <liwang@redhat.com>
---
 testcases/cve/cve-2014-0196.c         | 1 +
 testcases/cve/cve-2016-7117.c         | 1 +
 testcases/cve/cve-2017-2671.c         | 1 +
 testcases/network/can/cve/can_bcm01.c | 1 +
 testcases/network/packet/fanout01.c   | 1 +
 testcases/network/sockets/vsock01.c   | 1 +
 6 files changed, 6 insertions(+)

diff --git a/testcases/cve/cve-2014-0196.c b/testcases/cve/cve-2014-0196.c
index 012cbb7cd..1937d566c 100644
--- a/testcases/cve/cve-2014-0196.c
+++ b/testcases/cve/cve-2014-0196.c
@@ -141,6 +141,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = run,
+	.max_runtime = 60,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "4291086b1f08"},
 		{"CVE", "2014-0196"},
diff --git a/testcases/cve/cve-2016-7117.c b/testcases/cve/cve-2016-7117.c
index dca002924..64bf0a85e 100644
--- a/testcases/cve/cve-2016-7117.c
+++ b/testcases/cve/cve-2016-7117.c
@@ -150,6 +150,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.min_kver = "2.6.33",
+	.max_runtime = 60,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "a2e2725541fa"},
 		{"CVE", "2016-7117"},
diff --git a/testcases/cve/cve-2017-2671.c b/testcases/cve/cve-2017-2671.c
index e72795d15..e17cd9765 100644
--- a/testcases/cve/cve-2017-2671.c
+++ b/testcases/cve/cve-2017-2671.c
@@ -109,6 +109,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.max_runtime = 20,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "43a6684519ab"},
 		{"CVE", "2017-2671"},
diff --git a/testcases/network/can/cve/can_bcm01.c b/testcases/network/can/cve/can_bcm01.c
index d4f1e4ec4..79a827cfa 100644
--- a/testcases/network/can/cve/can_bcm01.c
+++ b/testcases/network/can/cve/can_bcm01.c
@@ -142,6 +142,7 @@ static struct tst_test test = {
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.needs_root = 1,
 	.skip_in_compat = 1,
+	.max_runtime = 30,
 	.needs_drivers = (const char *const[]) {
 		"vcan",
 		"can-bcm",
diff --git a/testcases/network/packet/fanout01.c b/testcases/network/packet/fanout01.c
index 5067d83a8..0aad3321b 100644
--- a/testcases/network/packet/fanout01.c
+++ b/testcases/network/packet/fanout01.c
@@ -106,6 +106,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
+	.max_runtime = 180,
 	.tags = (const struct tst_tag[]) {
 		{"CVE", "2017-15649"},
 		{"linux-git", "4971613c1639"},
diff --git a/testcases/network/sockets/vsock01.c b/testcases/network/sockets/vsock01.c
index a168e4401..1e688df19 100644
--- a/testcases/network/sockets/vsock01.c
+++ b/testcases/network/sockets/vsock01.c
@@ -111,6 +111,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
+	.max_runtime = 60,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_VSOCKETS_LOOPBACK",
 		NULL
-- 
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* [LTP] [PATCH 2/2] mmap3: reset runtime to 10s
  2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
  2022-05-20  8:36   ` [LTP] [PATCH 1/2] testcases: make use of runtime Li Wang
@ 2022-05-20  8:36   ` Li Wang
  2022-05-20 11:05     ` Cyril Hrubis
  2022-05-20 12:18   ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Cyril Hrubis
  2 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-20  8:36 UTC (permalink / raw)
  To: ltp

As we have removed the "-x" support from this test, there is no way to
reduce running time by multiply small float number.

Let's set 10 seconds for the max_runtime (approximately equal to -x 0.002).

Signed-off-by: Li Wang <liwang@redhat.com>
---
 runtest/mm                           | 2 +-
 testcases/kernel/mem/mtest06/mmap3.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/runtest/mm b/runtest/mm
index 6537666a9..4ecb61f24 100644
--- a/runtest/mm
+++ b/runtest/mm
@@ -16,7 +16,7 @@ mtest01w mtest01 -p80 -w
 mtest05   mmstress
 mtest06   mmap1
 mtest06_2 mmap2 -x 0.002 -a -p
-mtest06_3 mmap3 -x 0.002 -p
+mtest06_3 mmap3 -p
 # Remains diabled till the infinite loop problem is solved
 #mtest-6_4 shmat1 -x 0.00005
 
diff --git a/testcases/kernel/mem/mtest06/mmap3.c b/testcases/kernel/mem/mtest06/mmap3.c
index 6d47b01e7..3064f5cf7 100644
--- a/testcases/kernel/mem/mtest06/mmap3.c
+++ b/testcases/kernel/mem/mtest06/mmap3.c
@@ -182,5 +182,5 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.test_all = test_mmap,
-	.max_runtime = 86400,
+	.max_runtime = 10,
 };
-- 
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply related	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 1/2] testcases: make use of runtime
  2022-05-20  8:36   ` [LTP] [PATCH 1/2] testcases: make use of runtime Li Wang
@ 2022-05-20 11:03     ` Cyril Hrubis
  2022-05-20 11:15       ` Li Wang
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-20 11:03 UTC (permalink / raw)
  To: Li Wang; +Cc: ltp

Hi!
This should be ammended to the "fuzzy_sync: Convert to runtime" patch.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 2/2] mmap3: reset runtime to 10s
  2022-05-20  8:36   ` [LTP] [PATCH 2/2] mmap3: reset runtime to 10s Li Wang
@ 2022-05-20 11:05     ` Cyril Hrubis
  2022-05-20 11:24       ` Li Wang
  0 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-20 11:05 UTC (permalink / raw)
  To: Li Wang; +Cc: ltp

Hi!
> As we have removed the "-x" support from this test, there is no way to
> reduce running time by multiply small float number.

Actually the -x has been replaced by -I so we may as well change the
runtest file to pass -I instead.

On the other hand I guess that the original runtime was way too long for
a default value.

> Let's set 10 seconds for the max_runtime (approximately equal to -x 0.002).

If we are touching the code we may as well pick slightly longer
interval, what about a minute so that we have chance to do some testing
even on slower targets?

> Signed-off-by: Li Wang <liwang@redhat.com>
> ---
>  runtest/mm                           | 2 +-
>  testcases/kernel/mem/mtest06/mmap3.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/runtest/mm b/runtest/mm
> index 6537666a9..4ecb61f24 100644
> --- a/runtest/mm
> +++ b/runtest/mm
> @@ -16,7 +16,7 @@ mtest01w mtest01 -p80 -w
>  mtest05   mmstress
>  mtest06   mmap1
>  mtest06_2 mmap2 -x 0.002 -a -p
> -mtest06_3 mmap3 -x 0.002 -p
> +mtest06_3 mmap3 -p
>  # Remains diabled till the infinite loop problem is solved
>  #mtest-6_4 shmat1 -x 0.00005
>  
> diff --git a/testcases/kernel/mem/mtest06/mmap3.c b/testcases/kernel/mem/mtest06/mmap3.c
> index 6d47b01e7..3064f5cf7 100644
> --- a/testcases/kernel/mem/mtest06/mmap3.c
> +++ b/testcases/kernel/mem/mtest06/mmap3.c
> @@ -182,5 +182,5 @@ static struct tst_test test = {
>  	.setup = setup,
>  	.cleanup = cleanup,
>  	.test_all = test_mmap,
> -	.max_runtime = 86400,
> +	.max_runtime = 10,
>  };
> -- 
> 2.31.1
> 
> 
> -- 
> Mailing list info: https://lists.linux.it/listinfo/ltp

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 1/2] testcases: make use of runtime
  2022-05-20 11:03     ` Cyril Hrubis
@ 2022-05-20 11:15       ` Li Wang
  0 siblings, 0 replies; 61+ messages in thread
From: Li Wang @ 2022-05-20 11:15 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List


[-- Attachment #1.1: Type: text/plain, Size: 159 bytes --]

> This should be ammended to the "fuzzy_sync: Convert to runtime" patch.
>

Yes, that would be better. Feel free to merge into that one.

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 623 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 2/2] mmap3: reset runtime to 10s
  2022-05-20 11:05     ` Cyril Hrubis
@ 2022-05-20 11:24       ` Li Wang
  0 siblings, 0 replies; 61+ messages in thread
From: Li Wang @ 2022-05-20 11:24 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List


[-- Attachment #1.1: Type: text/plain, Size: 828 bytes --]

On Fri, May 20, 2022 at 7:03 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > As we have removed the "-x" support from this test, there is no way to
> > reduce running time by multiply small float number.
>
> Actually the -x has been replaced by -I so we may as well change the
> runtest file to pass -I instead.
>

Good to know, thanks!

>
> On the other hand I guess that the original runtime was way too long for
> a default value.
>
> > Let's set 10 seconds for the max_runtime (approximately equal to -x
> 0.002).
>
> If we are touching the code we may as well pick slightly longer
> interval, what about a minute so that we have chance to do some testing
> even on slower targets?
>

Make sense!

Feel free to add your amend when patchset merging,
I'd not send V2 to bring additional review work.

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 1870 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
  2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
  2022-05-20  8:36   ` [LTP] [PATCH 1/2] testcases: make use of runtime Li Wang
  2022-05-20  8:36   ` [LTP] [PATCH 2/2] mmap3: reset runtime to 10s Li Wang
@ 2022-05-20 12:18   ` Cyril Hrubis
  2022-05-20 12:51     ` Li Wang
  2 siblings, 1 reply; 61+ messages in thread
From: Cyril Hrubis @ 2022-05-20 12:18 UTC (permalink / raw)
  To: Li Wang; +Cc: ltp

Hi!
> I run a quick test for verifying those patches with the latest mainline kernel.
> Apart from these two tiny issues, the result looks good.

Can we merge the patchset now?

There are three things that should be fixed over the latest version,
that are:

- the heartbeat() call after each test for .tcnt
- the missing runtime for fuzzy sync
- default to something more sane in mmap3 and fix the runtest file
  (could be ammended to the convert mmap3 patch as well)

Should I send v4 after all?

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
  2022-05-20 12:18   ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Cyril Hrubis
@ 2022-05-20 12:51     ` Li Wang
  2022-05-20 12:55       ` Petr Vorel
  0 siblings, 1 reply; 61+ messages in thread
From: Li Wang @ 2022-05-20 12:51 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List


[-- Attachment #1.1: Type: text/plain, Size: 720 bytes --]

Cyril Hrubis <chrubis@suse.cz> wrote:

Hi!
> > I run a quick test for verifying those patches with the latest mainline
> kernel.
> > Apart from these two tiny issues, the result looks good.
>
> Can we merge the patchset now?
>

I think yes, at least my test shows it is safe.



>
> There are three things that should be fixed over the latest version,
> that are:
>
> - the heartbeat() call after each test for .tcnt
> - the missing runtime for fuzzy sync
> - default to something more sane in mmap3 and fix the runtest file
>   (could be ammended to the convert mmap3 patch as well)
>
> Should I send v4 after all?
>

No need, unless someone explicitly explains they will do a review/test for
V4.

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 1545 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
  2022-05-20 12:51     ` Li Wang
@ 2022-05-20 12:55       ` Petr Vorel
  0 siblings, 0 replies; 61+ messages in thread
From: Petr Vorel @ 2022-05-20 12:55 UTC (permalink / raw)
  To: Li Wang; +Cc: LTP List

Hi Li, Cyril,

> > Should I send v4 after all?

> No need, unless someone explicitly explains they will do a review/test for
> V4.
I only briefly looked at the patchset, but LGTM.
+1 for merging.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: Convert to runtime
  2022-05-12 12:38 ` [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: " Cyril Hrubis
@ 2022-05-24  8:36   ` Petr Vorel
  0 siblings, 0 replies; 61+ messages in thread
From: Petr Vorel @ 2022-05-24  8:36 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: ltp, automated-testing

Hi Cyril,

...
> +++ b/testcases/kernel/io/ltp-aiodio/dio_read.c
> @@ -51,6 +51,9 @@ static void do_buffered_writes(int fd, char *bufptr, long long fsize, long long
>  			tst_brk(TBROK, "pwrite: wrote %lld bytes out of %lld", w, wsize);

>  		SAFE_FSYNC(fd);
> +
> +		if (!tst_remaining_runtime())
> +			return;
>  	}
>  }

> @@ -75,6 +78,11 @@ static int do_direct_reads(char *filename, char *bufptr, long long fsize, long l
>  				goto exit;
>  			}

> +			if (!tst_remaining_runtime()) {
> +				tst_res(TINFO, "Test out of runtime, exitting");
nit: s/exitting/exiting/

Kind regards,
Petr

> +				goto exit;

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 00/29] Introduce runtime and conver tests
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (29 preceding siblings ...)
  2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
@ 2022-05-24  8:39 ` Petr Vorel
  2022-05-24  8:41 ` Petr Vorel
  31 siblings, 0 replies; 61+ messages in thread
From: Petr Vorel @ 2022-05-24  8:39 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: ltp, automated-testing

Hi Cyril, all,

For whole patchset including that extra commit of unsent v4:
Acked-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 00/29] Introduce runtime and conver tests
  2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
                   ` (30 preceding siblings ...)
  2022-05-24  8:39 ` [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Petr Vorel
@ 2022-05-24  8:41 ` Petr Vorel
  31 siblings, 0 replies; 61+ messages in thread
From: Petr Vorel @ 2022-05-24  8:41 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: ltp, automated-testing

Hi,

very nit: I suppose it's "exiting", not exitting.
It's in several commits.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
  2022-05-19 12:01                           ` Cyril Hrubis
@ 2022-05-24  9:24                             ` Richard Palethorpe
  0 siblings, 0 replies; 61+ messages in thread
From: Richard Palethorpe @ 2022-05-24  9:24 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: LTP List, automated-testing

Hello,

Cyril Hrubis <chrubis@suse.cz> writes:

> Hi!
>> No need. You can push with this new patch adding,
>> then I will kick start release testing for the whole LTP
>> from our side.
>
> Great, please do so.
>
> Andrea is going to send small fix to the aiodio tests that should be
> merged as well, otherwise these may fail rarely.

With these changes please add:

Reviewed-by: Richard Palethorpe <rpalethorpe@suse.com>

-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

* Re: [LTP] [Automated-testing] [PATCH v3 01/29] Introduce a concept of max runtime
  2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
  2022-05-14  4:21   ` Li Wang
@ 2022-05-24  9:25   ` Richard Palethorpe
  1 sibling, 0 replies; 61+ messages in thread
From: Richard Palethorpe @ 2022-05-24  9:25 UTC (permalink / raw)
  To: Cyril Hrubis; +Cc: ltp, automated-testing

Hello,

"Cyril Hrubis" <chrubis@suse.cz> writes:

> This commit introduces a concept of max test runtime. In other words
> test runtime is capped at a certain value in order to make testruns more
> deterministic. Test is free to to finish before the runtime is used up,
> for example when maximal number of iterations was reached, but test must
> stop once the runtime has been used up.
>
> Testcases that run for more than a second or two must check for
> remaining runtime by regular calls to tst_remaining_runtime() and should
> exit when zero is returned.
>
> The test max runtime must be set either by the .max_iteration_runtime in
> the tst_test structure or in the test setup by a call to
> tst_set_runtime().
>
> The test timeout is then computed as a sum of DEFAULT_TIMEOUT (currently
> set to 30 seconds) and the test runtime. The DEFAULT_TIMEOUT is nothing
> more than a safety margin for teardown of the test.
>
> This commit also maps the -I parameter to the test max runtime if
> available and introduces LTP_RUNTIME_MUL enviroment variable so that we
> have an easy controll over the runtime cap.
>
> Lastly but not least the function related to the timeout are turned into
> no-op by this commit and removed after all test are converted to the
> runtime API.
>
> Signed-off-by: Cyril Hrubis <chrubis@suse.cz>

Reviewed-by: Richard Palethorpe <rpalethorpe@suse.com>

-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

^ permalink raw reply	[flat|nested] 61+ messages in thread

end of thread, other threads:[~2022-05-24  9:26 UTC | newest]

Thread overview: 61+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-12 12:37 [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 01/29] Introduce a concept of max runtime Cyril Hrubis
2022-05-14  4:21   ` Li Wang
2022-05-24  9:25   ` [LTP] [Automated-testing] " Richard Palethorpe
2022-05-12 12:37 ` [LTP] [PATCH v3 02/29] mtest06/mmap1: Convert to runtime Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 03/29] mtest06/mmap3: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 04/29] mtest01/mtest01: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 05/29] cve/cve-2015-3290: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 06/29] crypto/af_alg02: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 07/29] crypto/pcrypt_aead01: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 08/29] syscalls/clock_gettime01: Remove useless timeout Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 09/29] syscalls/fanotify22: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 10/29] syscalls/gettimeofday02: Convert to runtime Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 11/29] syscalls/inotify06: " Cyril Hrubis
2022-05-12 12:37 ` [LTP] [PATCH v3 12/29] syscalls/inotify01: Remove now useless timeout Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 13/29] syscalls/perf_event_open03: Convert to runtime Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 14/29] syscalls/readv01: Remove now useless timeout Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 15/29] syscalls/tgkill03: Remove now unused timeout Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 16/29] syscalls/setsockopt09: Remove now useless timeout Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 17/29] syscalls/userfaultfd01: " Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 18/29] syscalls/move_pages12: Convert to runtime Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 19/29] syscalls/rt_sigqueueinfo01: Remove now useless timeout Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 20/29] mem/mallocstress: Convert to runtime Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 21/29] mem/{oom, min_free_kbytes}: " Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 22/29] fuzzy_sync: " Cyril Hrubis
2022-05-13  9:13   ` Li Wang
2022-05-13 12:20     ` Cyril Hrubis
2022-05-13 13:08       ` Li Wang
2022-05-13 14:22         ` Cyril Hrubis
2022-05-14  2:59           ` Li Wang
2022-05-16  7:52           ` Richard Palethorpe
2022-05-17 10:56             ` Cyril Hrubis
2022-05-17 11:07               ` [LTP] [Automated-testing] " Petr Vorel
2022-05-17 12:28               ` [LTP] " Li Wang
2022-05-18  7:47                 ` Li Wang
2022-05-19  9:05                   ` Cyril Hrubis
2022-05-19 10:02                     ` Li Wang
2022-05-19 10:27                       ` Cyril Hrubis
2022-05-19 10:29                         ` Li Wang
2022-05-19 12:01                           ` Cyril Hrubis
2022-05-24  9:24                             ` Richard Palethorpe
2022-05-12 12:38 ` [LTP] [PATCH v3 23/29] ltp-aiodio/dio_sparse, aiodio_sparse: " Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 24/29] ltp-aiodio/read_checkzero: Remove Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 25/29] ltp-aiodio/dio_{truncate, append}: Convert to runtime Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: " Cyril Hrubis
2022-05-24  8:36   ` Petr Vorel
2022-05-12 12:38 ` [LTP] [PATCH v3 27/29] timer_test: " Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 28/29] ltp-aiodio/aiodio_append: " Cyril Hrubis
2022-05-12 12:38 ` [LTP] [PATCH v3 29/29] tst_test: Remove timeout stubs Cyril Hrubis
2022-05-20  8:36 ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Li Wang
2022-05-20  8:36   ` [LTP] [PATCH 1/2] testcases: make use of runtime Li Wang
2022-05-20 11:03     ` Cyril Hrubis
2022-05-20 11:15       ` Li Wang
2022-05-20  8:36   ` [LTP] [PATCH 2/2] mmap3: reset runtime to 10s Li Wang
2022-05-20 11:05     ` Cyril Hrubis
2022-05-20 11:24       ` Li Wang
2022-05-20 12:18   ` [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime Cyril Hrubis
2022-05-20 12:51     ` Li Wang
2022-05-20 12:55       ` Petr Vorel
2022-05-24  8:39 ` [LTP] [PATCH v3 00/29] Introduce runtime and conver tests Petr Vorel
2022-05-24  8:41 ` Petr Vorel

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.