[LTP] [PATCH 0/3] shell test timeout handling

[LTP] [PATCH 0/3] shell test timeout handling

[Petr Vorel]

Hi,

changes before previous attempt [1]
* _tst_kill_test() (Cyril)
* added test timeout03.sh
* update trap message (Joerg)

NOTE: we have no way to print summary from _tst_do_exit if cleanup get
stuck, but IMHO there is no simple way how to solve it (we'd have to use
mapped memory to increment counter also in shell API as Cyril suggested)

[1] https://patchwork.ozlabs.org/project/ltp/patch/20210301220222.22705-7-pvorel@suse.cz/

Petr Vorel (3):
  tst_test.sh: Run cleanup also after test timeout
  test: Adding timeout03.sh for testing _tst_kill_test()
  zram: Increase timeout according to used devices

 lib/newlib_tests/shell/timeout03.sh           | 47 +++++++++++++++++++
 .../kernel/device-drivers/zram/zram_lib.sh    |  2 +
 testcases/lib/tst_test.sh                     | 25 ++++++++--
 3 files changed, 71 insertions(+), 3 deletions(-)
 create mode 100755 lib/newlib_tests/shell/timeout03.sh

-- 
2.30.1

[LTP] [PATCH 1/3] tst_test.sh: Run cleanup also after test timeout

[Petr Vorel]

Also timeout requires to run a test cleanup (e.g. zram01.sh).
Thus send first SIGINT, but keep also SIGKILL for safety reasons
(after 5 sec to give some time to the cleanup function and
_tst_check_security_modules()).

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
 testcases/lib/tst_test.sh | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/testcases/lib/tst_test.sh b/testcases/lib/tst_test.sh
index 58056e28b..1aa940f61 100644
--- a/testcases/lib/tst_test.sh
+++ b/testcases/lib/tst_test.sh
@@ -21,7 +21,7 @@ export TST_LIB_LOADED=1
 . tst_security.sh
 
 # default trap function
-trap "tst_brk TBROK 'test interrupted'" INT
+trap "tst_brk TBROK 'test interrupted or timed out'" INT
 
 _tst_cleanup_timer()
 {
@@ -442,6 +442,26 @@ _tst_multiply_timeout()
 	return 0
 }
 
+_tst_kill_test()
+{
+	local i=10
+
+	tst_res TBROK "Test timeouted, sending SIGTERM! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1"
+	kill -INT -$pid
+	tst_sleep 100ms
+
+	while kill -0 $pid 2>&1 > /dev/null && [ $i -gt 0 ]; do
+		tst_res TINFO "Test is still running, waiting ${i}s"
+		sleep 1
+		i=$((i-1))
+	done
+
+	if kill -0 $pid 2>&1 > /dev/null; then
+		tst_res TBROK "Test still running, sending SIGKILL"
+		kill -KILL -$pid
+	fi
+}
+
 _tst_setup_timer()
 {
 	TST_TIMEOUT=${TST_TIMEOUT:-300}
@@ -465,8 +485,7 @@ _tst_setup_timer()
 	tst_res TINFO "timeout per run is ${h}h ${m}m ${s}s"
 
 	_tst_cleanup_timer
-
-	sleep $sec && tst_res TBROK "test killed, timeout! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1" && kill -9 -$pid &
+	sleep $sec && _tst_kill_test &
 
 	_tst_setup_timer_pid=$!
 }
-- 
2.30.1

[LTP] [PATCH 2/3] test: Adding timeout03.sh for testing _tst_kill_test()

[Petr Vorel]

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
new commit in this attempt

 lib/newlib_tests/shell/timeout03.sh | 47 +++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 lib/newlib_tests/shell/timeout03.sh

diff --git a/lib/newlib_tests/shell/timeout03.sh b/lib/newlib_tests/shell/timeout03.sh
new file mode 100755
index 000000000..89a4928b2
--- /dev/null
+++ b/lib/newlib_tests/shell/timeout03.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Petr Vorel <pvorel@suse.cz>
+
+# testing shell timeout handling in _tst_kill_test()
+# expected output:
+# timeout03 1 TINFO: timeout per run is 0h 0m 1s
+# timeout03 1 TINFO: testing killing test after TST_TIMEOUT
+# timeout03 1 TBROK: Test timeouted, sending SIGTERM! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
+# timeout03 1 TBROK: test interrupted or timed out
+# timeout03 1 TPASS: test run cleanup after timeout
+# timeout03 1 TINFO: Test is still running, waiting 10s
+# timeout03 1 TINFO: Test is still running, waiting 9s
+# timeout03 1 TINFO: Test is still running, waiting 8s
+# timeout03 1 TINFO: Test is still running, waiting 7s
+# timeout03 1 TINFO: Test is still running, waiting 6s
+# timeout03 1 TINFO: Test is still running, waiting 5s
+# timeout03 1 TINFO: Test is still running, waiting 4s
+# timeout03 1 TINFO: Test is still running, waiting 3s
+# timeout03 1 TINFO: Test is still running, waiting 2s
+# timeout03 1 TINFO: Test is still running, waiting 1s
+# timeout03 1 TBROK: Test still running, sending SIGKILL
+# Killed
+
+TST_TESTFUNC=do_test
+TST_CLEANUP=cleanup
+
+TST_TIMEOUT=1
+. tst_test.sh
+
+do_test()
+{
+	tst_res TINFO "testing killing test after TST_TIMEOUT"
+
+	tst_sleep 2
+	tst_res TFAIL "test: running after TST_TIMEOUT"
+}
+
+cleanup()
+{
+	tst_res TPASS "test run cleanup after timeout"
+
+	tst_sleep 15 # must be higher than wait time in _tst_kill_test
+	tst_res TFAIL "cleanup: running after TST_TIMEOUT"
+}
+
+tst_run
-- 
2.30.1

[LTP] [PATCH 3/3] zram: Increase timeout according to used devices

[Petr Vorel]

to avoid unexpected timeout, which occurred even on just 4 zram devices.

On my system where run with ext{2,3,4}, xfs, btrfs, vfat, exfat, ntfs
it run for 12 min, i.e. mean 90s. Multiply by security constant 5,
expecting 450 sec for each filesystem.

Reviewed-by: Li Wang <liwang@redhat.com>
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
The same as previous
https://patchwork.ozlabs.org/project/ltp/patch/20210301220222.22705-8-pvorel@suse.cz/

 testcases/kernel/device-drivers/zram/zram_lib.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/testcases/kernel/device-drivers/zram/zram_lib.sh b/testcases/kernel/device-drivers/zram/zram_lib.sh
index 65e431e86..fe9c915c3 100755
--- a/testcases/kernel/device-drivers/zram/zram_lib.sh
+++ b/testcases/kernel/device-drivers/zram/zram_lib.sh
@@ -47,6 +47,8 @@ zram_load()
 		tst_brk TBROK "dev_num must be > 0"
 	fi
 
+	tst_set_timeout $((dev_num*450))
+
 	tst_res TINFO "create '$dev_num' zram device(s)"
 
 	modprobe zram num_devices=$dev_num || \
-- 
2.30.1

[LTP] [PATCH 0/3] shell test timeout handling

[Li Wang]

On Sat, Mar 13, 2021 at 1:32 AM Petr Vorel <pvorel@suse.cz> wrote:

> Hi,
>
> changes before previous attempt [1]
> * _tst_kill_test() (Cyril)
> * added test timeout03.sh
> * update trap message (Joerg)
>
> NOTE: we have no way to print summary from _tst_do_exit if cleanup get
> stuck, but IMHO there is no simple way how to solve it (we'd have to use
> mapped memory to increment counter also in shell API as Cyril suggested)
>
> [1]
> https://patchwork.ozlabs.org/project/ltp/patch/20210301220222.22705-7-pvorel@suse.cz/
>
> Petr Vorel (3):
>   tst_test.sh: Run cleanup also after test timeout
>   test: Adding timeout03.sh for testing _tst_kill_test()
>   zram: Increase timeout according to used devices
>

Nice work!

For series:
Reviewed-by: Li Wang <liwang@redhat.com>


-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210315/d14a3ccf/attachment.htm>

[LTP] [PATCH 1/3] tst_test.sh: Run cleanup also after test timeout

[Cyril Hrubis]

Hi!
>  _tst_cleanup_timer()
>  {
> @@ -442,6 +442,26 @@ _tst_multiply_timeout()
>  	return 0
>  }
>  
> +_tst_kill_test()
> +{
> +	local i=10
> +
> +	tst_res TBROK "Test timeouted, sending SIGTERM! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1"
                                                   ^
						   SIGINT?
> +	kill -INT -$pid
> +	tst_sleep 100ms

Other than that it looks good,

Reviewed-by: Cyril Hrubis <chrubis@suse.cz>

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 2/3] test: Adding timeout03.sh for testing _tst_kill_test()

[Cyril Hrubis]

Hi!
Looks obviously fine, but we have to update the expected output if we
are going to change the SIGTERM to SIGINT in the library.

Apart from that Reviewed-by: Cyril Hrubis <chrubis@suse.cz>

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH 1/3] tst_test.sh: Run cleanup also after test timeout

[Petr Vorel]

> Hi!
> >  _tst_cleanup_timer()
> >  {
> > @@ -442,6 +442,26 @@ _tst_multiply_timeout()
> >  	return 0
> >  }

> > +_tst_kill_test()
> > +{
> > +	local i=10
> > +
> > +	tst_res TBROK "Test timeouted, sending SIGTERM! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1"
>                                                    ^
> 						   SIGINT?

Oops, thanks for noticing this.

Kind regards,
Petr

> > +	kill -INT -$pid
> > +	tst_sleep 100ms

> Other than that it looks good,

> Reviewed-by: Cyril Hrubis <chrubis@suse.cz>

[LTP] [PATCH 2/3] test: Adding timeout03.sh for testing _tst_kill_test()

[Petr Vorel]

Hi Cyril, Li,

> Hi!
> Looks obviously fine, but we have to update the expected output if we
> are going to change the SIGTERM to SIGINT in the library.

> Apart from that Reviewed-by: Cyril Hrubis <chrubis@suse.cz>

Both errors fixed and patchset merged.

Thanks a lot both for your review!

Kind regards,
Petr