* [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS @ 2018-05-22 19:34 Petr Vorel 2018-05-22 19:34 ` [LTP] [RFC PATCH v3 2/2] lib: Add tests Petr Vorel 2018-05-24 13:41 ` [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Cyril Hrubis 0 siblings, 2 replies; 23+ messages in thread From: Petr Vorel @ 2018-05-22 19:34 UTC (permalink / raw) To: ltp This is specific only for shell. Each run of tst_run gets passed sequence number of a test being run as '$1' and corresponding part of data from TST_TEST_DATA as '$2'. Also create internal functions tst_run_tests() and tst_run_test() to reduce duplicity. Signed-off-by: Petr Vorel <pvorel@suse.cz> --- Changes v2->v3: * Don't pass $TST_TEST_DATA as whole argument for $TST_CNT (Cyril) * Create tst_run_tests() for cleanup (Cyril). * Create tst_run_test() - more cleanup Maybe we should try to "hide" somehow these API functions (underscore at the front?). Kind regards, Petr --- doc/test-writing-guidelines.txt | 74 +++++++++++++++++++++++++++++++++++++---- testcases/lib/tst_test.sh | 61 +++++++++++++++++++++------------ 2 files changed, 106 insertions(+), 29 deletions(-) diff --git a/doc/test-writing-guidelines.txt b/doc/test-writing-guidelines.txt index de47443eb..531b828ff 100644 --- a/doc/test-writing-guidelines.txt +++ b/doc/test-writing-guidelines.txt @@ -1442,12 +1442,12 @@ TST_CNT=2 test1() { - tst_res TPASS "Test 1 passed" + tst_res TPASS "Test $1 passed" } test2() { - tst_res TPASS "Test 2 passed" + tst_res TPASS "Test $1 passed" } tst_run @@ -1455,7 +1455,8 @@ tst_run If '$TST_CNT' is set, the test library looks if there are functions named '$\{TST_TESTFUNC\}1', ..., '$\{TST_TESTFUNC\}$\{TST_CNT\}' and if these are -found they are executed one by one. +found they are executed one by one. The test number is passed to it in the '$1'. + [source,sh] ------------------------------------------------------------------------------- @@ -1471,8 +1472,8 @@ TST_CNT=2 do_test() { case $1 in - 1) tst_res TPASS "Test 1 passed";; - 2) tst_res TPASS "Test 2 passed";; + 1) tst_res TPASS "Test $1 passed";; + 2) tst_res TPASS "Test $1 passed";; esac } @@ -1483,6 +1484,65 @@ Otherwise, if '$TST_CNT' is set but there is no '$\{TST_TESTFUNC\}1', etc., the '$TST_TESTFUNC' is executed '$TST_CNT' times and the test number is passed to it in the '$1'. +[source,sh] +------------------------------------------------------------------------------- +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and +# $TST_TEST_DATA_IFS +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +TST_TEST_DATA_IFS=":" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo' +# test 2 TPASS: Test 2 passed with data 'bar' +# test 3 TPASS: Test 3 passed with data 'd dd' + +------------------------------------------------------------------------------- + +It's possible to pass data for function with '$TST_TEST_DATA'. Optional +'$TST_TEST_DATA_IFS' is used for splitting, default value is space. + +[source,sh] +------------------------------------------------------------------------------- +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=do_test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with data '$2'";; + 2) tst_res TPASS "Test $1 passed with data '$2'";; + esac +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' + +------------------------------------------------------------------------------- +When '$TST_TEST_DATA' is used with '$TST_CNT', it's passed as whole string in +'$2' ($1 is for the test number), '$TST_TEST_DATA_IFS' is ignored. Similar +would be when using these variables with separate functions. + 2.3.2 Library variables ^^^^^^^^^^^^^^^^^^^^^^^ @@ -1587,8 +1647,8 @@ these can be listed with passing help '-h' option to any test. The function that prints the usage is passed in '$TST_USAGE', the help for the options implemented in the library is appended when usage is printed. -Lastly the fucntion '$PARSE_ARGS' is called with the option name in '$1' and, -if option has argument, its value in '$2'. +Lastly the fucntion '$PARSE_ARGS' is called with the option name in the '$1' +and, if option has argument, its value in the '$2'. [source,sh] ------------------------------------------------------------------------------- diff --git a/testcases/lib/tst_test.sh b/testcases/lib/tst_test.sh index 464c4c41e..7680aa462 100644 --- a/testcases/lib/tst_test.sh +++ b/testcases/lib/tst_test.sh @@ -246,7 +246,7 @@ tst_rescmp() tst_run() { - local tst_i + local tst_i tst_data if [ -n "$TST_TEST_PATH" ]; then for tst_i in $(grep TST_ "$TST_TEST_PATH" | sed 's/.*TST_//; s/[="} \t\/:`].*//'); do @@ -255,7 +255,7 @@ tst_run() OPTS|USAGE|PARSE_ARGS|POS_ARGS);; NEEDS_ROOT|NEEDS_TMPDIR|NEEDS_DEVICE|DEVICE);; NEEDS_CMDS|NEEDS_MODULE|MODPATH|DATAROOT);; - IPV6);; + IPV6|TEST_DATA|TEST_DATA_IFS);; *) tst_res TWARN "Reserved variable TST_$tst_i used!";; esac done @@ -348,27 +348,17 @@ tst_run() #TODO check that test reports some results for each test function call while [ $TST_ITERATIONS -gt 0 ]; do - if [ -n "$TST_CNT" ]; then - if type test1 > /dev/null 2>&1; then - for tst_i in $(seq $TST_CNT); do - local res=$(tst_resstr) - $TST_TESTFUNC$tst_i - tst_rescmp "$res" - TST_COUNT=$((TST_COUNT+1)) - done - else - for tst_i in $(seq $TST_CNT); do - local res=$(tst_resstr) - $TST_TESTFUNC $tst_i - tst_rescmp "$res" - TST_COUNT=$((TST_COUNT+1)) - done - fi + if [ -n "$TST_TEST_DATA" ]; then + tst_i=1 + tst_check_cmds cut + while true; do + tst_data="$(echo "$TST_TEST_DATA" | cut -d"$TST_TEST_DATA_IFS" -f$tst_i)" + [ -z "$tst_data" ] && break + tst_run_tests "$tst_data" + tst_i=$((tst_i+1)) + done else - local res=$(tst_resstr) - $TST_TESTFUNC - tst_rescmp "$res" - TST_COUNT=$((TST_COUNT+1)) + tst_run_tests fi TST_ITERATIONS=$((TST_ITERATIONS-1)) done @@ -376,6 +366,31 @@ tst_run() tst_do_exit } +tst_run_tests() +{ + local tst_data="$1" + local tst_i + + for tst_i in $(seq ${TST_CNT:-1}); do + if type test1 > /dev/null 2>&1; then + tst_run_test "$TST_TESTFUNC$tst_i" $tst_i "$tst_data" + else + tst_run_test "$TST_TESTFUNC" $tst_i "$tst_data" + fi + done +} + +tst_run_test() +{ + local res=$(tst_resstr) + local tst_fnc="$1" + shift + + $tst_fnc "$@" + tst_rescmp "$res" + TST_COUNT=$((TST_COUNT+1)) +} + if [ -z "$TST_ID" ]; then filename=$(basename $0) TST_ID=${filename%%.*} @@ -400,6 +415,8 @@ if [ -z "$TST_NO_DEFAULT_RUN" ]; then tst_brk TBROK "TST_TESTFUNC is not defined" fi + TST_TEST_DATA_IFS="${TST_TEST_DATA_IFS:- }" + if [ -n "$TST_CNT" ]; then if ! tst_is_int "$TST_CNT"; then tst_brk TBROK "TST_CNT must be integer" -- 2.16.3 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 2/2] lib: Add tests 2018-05-22 19:34 [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Petr Vorel @ 2018-05-22 19:34 ` Petr Vorel 2018-05-24 13:46 ` Cyril Hrubis 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig 2018-05-24 13:41 ` [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Cyril Hrubis 1 sibling, 2 replies; 23+ messages in thread From: Petr Vorel @ 2018-05-22 19:34 UTC (permalink / raw) To: ltp Not-signed-off-by: Petr Vorel <pvorel@suse.cz> --- This is not intended to be merged. It's just a helper which you can use for testing the previous patch. (Side note: if we ever want to test tst_test.sh, it'd be possible to dump it into file and compare with saved output. It'd require to remove lines with 'stime=') --- lib/newlib_tests/Makefile | 1 + lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh | 22 ++++++++++++ lib/newlib_tests/test.TST_TEST.TST_CNT.sh | 20 +++++++++++ lib/newlib_tests/test.TST_TEST.getopts.sh | 40 +++++++++++++++++++++ lib/newlib_tests/test.TST_TEST.sh | 25 +++++++++++++ .../test.TST_TEST_DATA.TST_CNT.separate.sh | 26 ++++++++++++++ lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh | 24 +++++++++++++ lib/newlib_tests/test.TST_TEST_DATA.getopts.sh | 41 +++++++++++++++++++++ lib/newlib_tests/test.TST_TEST_DATA.sh | 18 ++++++++++ lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh | 42 ++++++++++++++++++++++ lib/newlib_tests/test.TST_TEST_DATA_IFS.sh | 21 +++++++++++ runtest/test | 13 +++++++ 12 files changed, 293 insertions(+) create mode 100755 lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/test.TST_TEST.TST_CNT.sh create mode 100755 lib/newlib_tests/test.TST_TEST.getopts.sh create mode 100755 lib/newlib_tests/test.TST_TEST.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA.getopts.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh create mode 100755 lib/newlib_tests/test.TST_TEST_DATA_IFS.sh create mode 100644 runtest/test diff --git a/lib/newlib_tests/Makefile b/lib/newlib_tests/Makefile index afa09373e..cc20920be 100644 --- a/lib/newlib_tests/Makefile +++ b/lib/newlib_tests/Makefile @@ -15,5 +15,6 @@ ifeq ($(ANDROID),1) FILTER_OUT_MAKE_TARGETS += test08 endif +INSTALL_TARGETS := *.sh include $(top_srcdir)/include/mk/generic_leaf_target.mk diff --git a/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh b/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh new file mode 100755 index 000000000..e26e2cb14 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh @@ -0,0 +1,22 @@ +#!/bin/sh +# +# Example test with tests in separate functions +# + +TST_TESTFUNC=test +TST_CNT=2 +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +test2() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +tst_run +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') diff --git a/lib/newlib_tests/test.TST_TEST.TST_CNT.sh b/lib/newlib_tests/test.TST_TEST.TST_CNT.sh new file mode 100755 index 000000000..d84457fea --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST.TST_CNT.sh @@ -0,0 +1,20 @@ +#!/bin/sh +# +# Example test with tests in a single function +# + +TST_TESTFUNC=do_test +TST_CNT=2 +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with no data ('$2')";; + 2) tst_res TPASS "Test $1 passed with no data ('$2')";; + esac +} + +tst_run +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') diff --git a/lib/newlib_tests/test.TST_TEST.getopts.sh b/lib/newlib_tests/test.TST_TEST.getopts.sh new file mode 100755 index 000000000..0ba848a0a --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST.getopts.sh @@ -0,0 +1,40 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run diff --git a/lib/newlib_tests/test.TST_TEST.sh b/lib/newlib_tests/test.TST_TEST.sh new file mode 100755 index 000000000..6ecae710d --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST.sh @@ -0,0 +1,25 @@ +#!/bin/sh +# +# This is a basic test for true shell buildin +# + +TST_TESTFUNC=do_test +. tst_test.sh + +do_test() +{ + true + ret=$? + + tst_res TINFO "Test $1 passed with no data ('$2')" + + if [ $ret -eq 0 ]; then + tst_res TPASS "true returned 0" + else + tst_res TFAIL "true returned $ret" + fi +} + +tst_run +# test 1 TINFO: Test 1 passed with no data ('') +# test 1 TPASS: true returned 0 diff --git a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh b/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh new file mode 100755 index 000000000..9a81ad27c --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh @@ -0,0 +1,26 @@ +#!/bin/sh +# +# Example test with tests in separate functions, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +test2() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' diff --git a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh b/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh new file mode 100755 index 000000000..798ad9621 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh @@ -0,0 +1,24 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=do_test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with data '$2'";; + 2) tst_res TPASS "Test $1 passed with data '$2'";; + esac +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' diff --git a/lib/newlib_tests/test.TST_TEST_DATA.getopts.sh b/lib/newlib_tests/test.TST_TEST_DATA.getopts.sh new file mode 100755 index 000000000..1cd150618 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA.getopts.sh @@ -0,0 +1,41 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run diff --git a/lib/newlib_tests/test.TST_TEST_DATA.sh b/lib/newlib_tests/test.TST_TEST_DATA.sh new file mode 100755 index 000000000..4cb51c669 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA.sh @@ -0,0 +1,18 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' diff --git a/lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh b/lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh new file mode 100755 index 000000000..59dbd9598 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh @@ -0,0 +1,42 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" +TST_TEST_DATA_IFS=":" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run diff --git a/lib/newlib_tests/test.TST_TEST_DATA_IFS.sh b/lib/newlib_tests/test.TST_TEST_DATA_IFS.sh new file mode 100755 index 000000000..52a2e6d47 --- /dev/null +++ b/lib/newlib_tests/test.TST_TEST_DATA_IFS.sh @@ -0,0 +1,21 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and +# $TST_TEST_DATA_IFS +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +TST_TEST_DATA_IFS=":" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo' +# test 2 TPASS: Test 2 passed with data 'bar' +# test 3 TPASS: Test 3 passed with data 'd dd' diff --git a/runtest/test b/runtest/test new file mode 100644 index 000000000..469e3b787 --- /dev/null +++ b/runtest/test @@ -0,0 +1,13 @@ +TST_TEST_DATA_IFS test.TST_TEST_DATA_IFS.sh +TST_TEST_DATA test.TST_TEST_DATA.sh +TST_TEST_DATA.TST_CNT.separate test.TST_TEST_DATA.TST_CNT.separate.sh +TST_TEST_DATA.TST_CNT test.TST_TEST_DATA.TST_CNT.sh +TST_TEST test.TST_TEST.sh +TST_TEST.TST_CNT.separate test.TST_TEST.TST_CNT.separate.sh +TST_TEST.TST_CNT test.TST_TEST.TST_CNT.sh +TST_TEST.getopts test.TST_TEST.getopts.sh +TST_TEST.getopts_a_f test.TST_TEST.getopts.sh -a -f foo1 +TST_TEST_DATA.getopts test.TST_TEST_DATA.getopts.sh +TST_TEST_DATA.getopts_a_f test.TST_TEST_DATA.getopts.sh -a -f foo2 +TST_TEST_DATA_IFS.getopts test.TST_TEST_DATA_IFS.getopts.sh +TST_TEST_DATA_IFS.getopts_a_f test.TST_TEST_DATA_IFS.getopts.sh -a -f foo3 -- 2.16.3 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 2/2] lib: Add tests 2018-05-22 19:34 ` [LTP] [RFC PATCH v3 2/2] lib: Add tests Petr Vorel @ 2018-05-24 13:46 ` Cyril Hrubis 2018-05-24 14:00 ` Petr Vorel 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig 1 sibling, 1 reply; 23+ messages in thread From: Cyril Hrubis @ 2018-05-24 13:46 UTC (permalink / raw) To: ltp Hi! > This is not intended to be merged. It's just a helper which you can use > for testing the previous patch. > > (Side note: if we ever want to test tst_test.sh, it'd be possible to > dump it into file and compare with saved output. It'd require to remove > lines with 'stime=') That sounds like a good idea actually, I'm all for unit testing the shell library. I'm not that sure that we would want to put it into the runtest file though, maybe we just need a runtest.sh script in the lib/ directory that would loop over all *.sh file in a certain directory and compared the output against corresponding file names without the .sh suffix. We can then do something as lib/tests/ with runtest.sh that would loop over subdirectories in lib/tests/ executing actual testcases. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 2/2] lib: Add tests 2018-05-24 13:46 ` Cyril Hrubis @ 2018-05-24 14:00 ` Petr Vorel 0 siblings, 0 replies; 23+ messages in thread From: Petr Vorel @ 2018-05-24 14:00 UTC (permalink / raw) To: ltp > Hi! > > This is not intended to be merged. It's just a helper which you can use > > for testing the previous patch. > > (Side note: if we ever want to test tst_test.sh, it'd be possible to > > dump it into file and compare with saved output. It'd require to remove > > lines with 'stime=') > That sounds like a good idea actually, I'm all for unit testing the > shell library. I was inspired by recent change in kernel's kconfig. Masahiro Yamada added tests, which compare output of various make calls with saved. It's in scripts/kconfig/tests/, using python's pytest, you can see it by running 'make testconfig'. > I'm not that sure that we would want to put it into the runtest file > though, maybe we just need a runtest.sh script in the lib/ directory > that would loop over all *.sh file in a certain directory and compared > the output against corresponding file names without the .sh suffix. > We can then do something as lib/tests/ with runtest.sh that would loop > over subdirectories in lib/tests/ executing actual testcases. Sounds reasonable. I added it into runtest file just to help myself to test. And publish it just in case anyone wanted to use it for testing the patch. Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [PATCH 1/2] Make shell lib tests standalone 2018-05-22 19:34 ` [LTP] [RFC PATCH v3 2/2] lib: Add tests Petr Vorel 2018-05-24 13:46 ` Cyril Hrubis @ 2018-08-28 11:18 ` Christian Lanig 2018-08-28 11:18 ` [LTP] [PATCH 2/2] Add wanted output to shell lib test case Christian Lanig ` (2 more replies) 1 sibling, 3 replies; 23+ messages in thread From: Christian Lanig @ 2018-08-28 11:18 UTC (permalink / raw) To: ltp --- Please do not merge! See following message for details. lib/newlib_tests/Makefile | 1 - .../{ => shell}/test.TST_TEST.TST_CNT.separate.sh | 2 +- .../{ => shell}/test.TST_TEST.TST_CNT.sh | 2 +- .../{ => shell}/test.TST_TEST.getopts.sh | 2 +- lib/newlib_tests/{ => shell}/test.TST_TEST.sh | 2 +- .../test.TST_TEST_DATA.TST_CNT.separate.sh | 2 +- .../{ => shell}/test.TST_TEST_DATA.TST_CNT.sh | 2 +- .../{ => shell}/test.TST_TEST_DATA.getopts.sh | 2 +- lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.sh | 2 +- .../{ => shell}/test.TST_TEST_DATA_IFS.getopts.sh | 2 +- .../{ => shell}/test.TST_TEST_DATA_IFS.sh | 2 +- lib/newlib_tests/test.shell_lib.sh | 94 ++++++++++++++++++++++ runtest/test | 13 --- testcases/lib/tst_test.sh | 2 +- 14 files changed, 105 insertions(+), 25 deletions(-) rename lib/newlib_tests/{ => shell}/test.TST_TEST.TST_CNT.separate.sh (95%) rename lib/newlib_tests/{ => shell}/test.TST_TEST.TST_CNT.sh (95%) rename lib/newlib_tests/{ => shell}/test.TST_TEST.getopts.sh (96%) rename lib/newlib_tests/{ => shell}/test.TST_TEST.sh (95%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.TST_CNT.separate.sh (96%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.TST_CNT.sh (96%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.getopts.sh (96%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.sh (95%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA_IFS.getopts.sh (97%) rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA_IFS.sh (96%) create mode 100755 lib/newlib_tests/test.shell_lib.sh delete mode 100644 runtest/test diff --git a/lib/newlib_tests/Makefile b/lib/newlib_tests/Makefile index ad087b440..2fc50160a 100644 --- a/lib/newlib_tests/Makefile +++ b/lib/newlib_tests/Makefile @@ -16,6 +16,5 @@ ifeq ($(ANDROID),1) FILTER_OUT_MAKE_TARGETS += test08 endif -INSTALL_TARGETS := *.sh include $(top_srcdir)/include/mk/generic_leaf_target.mk diff --git a/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh similarity index 95% rename from lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh rename to lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh index e26e2cb14..75ab680f1 100755 --- a/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh @@ -5,7 +5,7 @@ TST_TESTFUNC=test TST_CNT=2 -. tst_test.sh +. ./tst_test.sh test1() { diff --git a/lib/newlib_tests/test.TST_TEST.TST_CNT.sh b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh similarity index 95% rename from lib/newlib_tests/test.TST_TEST.TST_CNT.sh rename to lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh index d84457fea..31b15204f 100755 --- a/lib/newlib_tests/test.TST_TEST.TST_CNT.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh @@ -5,7 +5,7 @@ TST_TESTFUNC=do_test TST_CNT=2 -. tst_test.sh +. ./tst_test.sh do_test() { diff --git a/lib/newlib_tests/test.TST_TEST.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh similarity index 96% rename from lib/newlib_tests/test.TST_TEST.getopts.sh rename to lib/newlib_tests/shell/test.TST_TEST.getopts.sh index 0ba848a0a..090133756 100755 --- a/lib/newlib_tests/test.TST_TEST.getopts.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh @@ -8,7 +8,7 @@ TST_USAGE=usage TST_PARSE_ARGS=parse_args TST_TESTFUNC=do_test -. tst_test.sh +. ./tst_test.sh ALTERNATIVE=0 MODE="foo" diff --git a/lib/newlib_tests/test.TST_TEST.sh b/lib/newlib_tests/shell/test.TST_TEST.sh similarity index 95% rename from lib/newlib_tests/test.TST_TEST.sh rename to lib/newlib_tests/shell/test.TST_TEST.sh index 6ecae710d..42f14151b 100755 --- a/lib/newlib_tests/test.TST_TEST.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.sh @@ -4,7 +4,7 @@ # TST_TESTFUNC=do_test -. tst_test.sh +. ./tst_test.sh do_test() { diff --git a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh similarity index 96% rename from lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh index 9a81ad27c..6d4a90141 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.separate.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh @@ -6,7 +6,7 @@ TST_TESTFUNC=test TST_CNT=2 TST_TEST_DATA="foo:bar:d dd" -. tst_test.sh +. ./tst_test.sh test1() { diff --git a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh similarity index 96% rename from lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh index 798ad9621..8dd25acae 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA.TST_CNT.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh @@ -6,7 +6,7 @@ TST_TESTFUNC=do_test TST_CNT=2 TST_TEST_DATA="foo:bar:d dd" -. tst_test.sh +. ./tst_test.sh do_test() { diff --git a/lib/newlib_tests/test.TST_TEST_DATA.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh similarity index 96% rename from lib/newlib_tests/test.TST_TEST_DATA.getopts.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh index 1cd150618..66e534ae5 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA.getopts.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh @@ -9,7 +9,7 @@ TST_PARSE_ARGS=parse_args TST_TESTFUNC=do_test TST_TEST_DATA="foo0:bar:d dd" -. tst_test.sh +. ./tst_test.sh ALTERNATIVE=0 MODE="foo" diff --git a/lib/newlib_tests/test.TST_TEST_DATA.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.sh similarity index 95% rename from lib/newlib_tests/test.TST_TEST_DATA.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA.sh index 4cb51c669..568cec01e 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.sh @@ -5,7 +5,7 @@ TST_TESTFUNC=do_test TST_TEST_DATA="foo:bar:d dd" -. tst_test.sh +. ./tst_test.sh do_test() { diff --git a/lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh similarity index 97% rename from lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh index 59dbd9598..41e3f99de 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA_IFS.getopts.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh @@ -10,7 +10,7 @@ TST_TESTFUNC=do_test TST_TEST_DATA="foo0:bar:d dd" TST_TEST_DATA_IFS=":" -. tst_test.sh +. ./tst_test.sh ALTERNATIVE=0 MODE="foo" diff --git a/lib/newlib_tests/test.TST_TEST_DATA_IFS.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh similarity index 96% rename from lib/newlib_tests/test.TST_TEST_DATA_IFS.sh rename to lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh index 52a2e6d47..4e73f5030 100755 --- a/lib/newlib_tests/test.TST_TEST_DATA_IFS.sh +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh @@ -7,7 +7,7 @@ TST_TESTFUNC=do_test TST_TEST_DATA="foo:bar:d dd" TST_TEST_DATA_IFS=":" -. tst_test.sh +. ./tst_test.sh do_test() { diff --git a/lib/newlib_tests/test.shell_lib.sh b/lib/newlib_tests/test.shell_lib.sh new file mode 100755 index 000000000..599eec78a --- /dev/null +++ b/lib/newlib_tests/test.shell_lib.sh @@ -0,0 +1,94 @@ +# !/bin/sh +# +# This script iterates over all test cases for the new shell lib and verifies +# the output. +# Do NOT use newline symbols in the names of files containing test cases! +# + +setup() { + color_green="\033[1;32m" + color_red="\033[1;31m" + standard_color="\033[0m" + start_dir="$PWD" + cd "$(dirname "$0")""/../../" || exit 1; + sh_lib_dir=""$PWD"/testcases/lib/" + sh_lib_test_dir=""$PWD"/lib/newlib_tests/shell/" + tst_cases=$(ls "$sh_lib_test_dir" | \ + sed "s/"$TEST_NAME".sh/"$TEST_NAME"/g") + cd "$sh_lib_dir" || exit 1 +} + +check_requirements() { + case "$0" in + -*) + printf "Please execute this script. Sourcing "; + printf "(. <SCRIPT>) is not supported. \n"; + return 1;; + *) + true; + esac +} + +verify_output() { + local output_found=1 + local wanted_output= + local parsed_line= + while read line; + do + if [ -z "$wanted_output" ] && [ "$line" = "# output:" ] + then + output_found=0 + elif [ $output_found -eq 0 ] || [ -n "$wanted_output" ] + then + if printf "$line" | grep "# " > /dev/null; + then + if [ $output_found -eq 0 ] + then + parsed_line=$(printf "$line" | \ + sed "s/^.\{2\}//") + output_found=1 + else + parsed_line="\n"$(printf "$line" | \ + sed "s/^.\{2\}//") + fi + elif printf "$line" | grep "#" > /dev/null; + then + parsed_line="\n"$(printf "$line" | \ + sed "s/^.\{1\}//") + else true; + fi + wanted_output=""$wanted_output""$parsed_line"" + else true; + fi + + done < ""$sh_lib_test_dir""$1".sh" + wanted_output=$(printf "$wanted_output") + local actual_output=$(""$sh_lib_test_dir""$1".sh") + actual_output=$(printf "$actual_output") + if [ "$wanted_output" = "$actual_output" ] + then + return 0 + else + return 1 + fi +} + +run_tests() { + for tst_case in $tst_cases + do + printf "Running Test: \""$tst_case"\"...\n" + if verify_output "$tst_case"; + then + printf ""$color_green"TPASS: "$standard_color"" + printf "Test "$tst_case" was successful.\n\n" + else + printf ""$color_red"TFAIL:"$standard_color"" + printf "Test "$tst_case" was unsuccessful.\n\n" + fi + done +} + +check_requirements +setup +run_tests +exit 0 diff --git a/runtest/test b/runtest/test deleted file mode 100644 index 469e3b787..000000000 --- a/runtest/test +++ /dev/null @@ -1,13 +0,0 @@ -TST_TEST_DATA_IFS test.TST_TEST_DATA_IFS.sh -TST_TEST_DATA test.TST_TEST_DATA.sh -TST_TEST_DATA.TST_CNT.separate test.TST_TEST_DATA.TST_CNT.separate.sh -TST_TEST_DATA.TST_CNT test.TST_TEST_DATA.TST_CNT.sh -TST_TEST test.TST_TEST.sh -TST_TEST.TST_CNT.separate test.TST_TEST.TST_CNT.separate.sh -TST_TEST.TST_CNT test.TST_TEST.TST_CNT.sh -TST_TEST.getopts test.TST_TEST.getopts.sh -TST_TEST.getopts_a_f test.TST_TEST.getopts.sh -a -f foo1 -TST_TEST_DATA.getopts test.TST_TEST_DATA.getopts.sh -TST_TEST_DATA.getopts_a_f test.TST_TEST_DATA.getopts.sh -a -f foo2 -TST_TEST_DATA_IFS.getopts test.TST_TEST_DATA_IFS.getopts.sh -TST_TEST_DATA_IFS.getopts_a_f test.TST_TEST_DATA_IFS.getopts.sh -a -f foo3 diff --git a/testcases/lib/tst_test.sh b/testcases/lib/tst_test.sh index e553b496d..769ed66c2 100644 --- a/testcases/lib/tst_test.sh +++ b/testcases/lib/tst_test.sh @@ -33,7 +33,7 @@ export TST_ITERATIONS=1 export TST_TMPDIR_RHOST=0 export TST_LIB_LOADED=1 -. tst_ansi_color.sh +. ./tst_ansi_color.sh # default trap function trap "tst_brk TBROK 'test interrupted'" INT -- 2.16.4 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [PATCH 2/2] Add wanted output to shell lib test case 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig @ 2018-08-28 11:18 ` Christian Lanig 2018-08-29 17:24 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Petr Vorel 2018-08-31 11:46 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Cyril Hrubis 2 siblings, 0 replies; 23+ messages in thread From: Christian Lanig @ 2018-08-28 11:18 UTC (permalink / raw) To: ltp --- Please do not merge! I would like to get some feedback for this one and the previous patch. It is based on Petr Vorel's [RFC,v3,2/2] lib: Add tests from May 22nd 2018 with the message ID 20180522193430.20117-2-pvorel@suse.cz. These two patches transfer Petr's test cases in a separate folder and a shell script loops over all files as suggested in the replies to the initial message. The output is verified against a comment section in the test case file itself. This section is started with the line "# output:" and it is expected that it goes to the end of the document without interception. I have run the script with various Shell implementations to make sure it's portable. Limitations: The script is currently unable to test e.g. whether TFAIL is displayed with red color. So far I have only added the correct output line in a single test case file so just one test is currently passing. Before proceeding I would like to know whether this is a good way to go. lib/newlib_tests/shell/test.TST_TEST.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/newlib_tests/shell/test.TST_TEST.sh b/lib/newlib_tests/shell/test.TST_TEST.sh index 42f14151b..aa4e20377 100755 --- a/lib/newlib_tests/shell/test.TST_TEST.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.sh @@ -21,5 +21,12 @@ do_test() } tst_run +# output: # test 1 TINFO: Test 1 passed with no data ('') # test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 -- 2.16.4 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [PATCH 1/2] Make shell lib tests standalone 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig 2018-08-28 11:18 ` [LTP] [PATCH 2/2] Add wanted output to shell lib test case Christian Lanig @ 2018-08-29 17:24 ` Petr Vorel 2018-08-29 17:30 ` Petr Vorel 2018-08-31 15:24 ` [LTP] [RFC PATCH 0/1] Add automated tests for shell lib Christian Lanig 2018-08-31 11:46 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Cyril Hrubis 2 siblings, 2 replies; 23+ messages in thread From: Petr Vorel @ 2018-08-29 17:24 UTC (permalink / raw) To: ltp Hi Christian, this expects have applied my patch https://patchwork.ozlabs.org/patch/918549/ http://lists.linux.it/pipermail/ltp/2018-May/008191.html Christian, next time merge it with your changes (these 3 commits should be just one, with you as author.) Before I delve into the details, just comment the functionality. $ ./lib/newlib_tests/test.shell_lib.sh Running Test: "test.TST_TEST_DATA.getopts"... TFAIL:Test test.TST_TEST_DATA.getopts was unsuccessful. Running Test: "test.TST_TEST_DATA_IFS.getopts"... TFAIL:Test test.TST_TEST_DATA_IFS.getopts was unsuccessful. Running Test: "test.TST_TEST_DATA_IFS"... TFAIL:Test test.TST_TEST_DATA_IFS was unsuccessful. Running Test: "test.TST_TEST_DATA"... TFAIL:Test test.TST_TEST_DATA was unsuccessful. Running Test: "test.TST_TEST_DATA.TST_CNT.separate"... TFAIL:Test test.TST_TEST_DATA.TST_CNT.separate was unsuccessful. Running Test: "test.TST_TEST_DATA.TST_CNT"... TFAIL:Test test.TST_TEST_DATA.TST_CNT was unsuccessful. Running Test: "test.TST_TEST.getopts"... TFAIL:Test test.TST_TEST.getopts was unsuccessful. Running Test: "test.TST_TEST"... TPASS: Test test.TST_TEST was successful. Running Test: "test.TST_TEST.TST_CNT.separate"... TFAIL:Test test.TST_TEST.TST_CNT.separate was unsuccessful. Running Test: "test.TST_TEST.TST_CNT"... TFAIL:Test test.TST_TEST.TST_CNT was unsuccessful. --- These tests must be all green before merging them. I suppose the failures are because only 5 of 10 tests have "# output:" (parsed by verify_output()). About the output. See Cyril's comments about my patch for running C tests http://lists.linux.it/pipermail/ltp/2018-August/009119.html Inconsistent output: space after 'TPASS:', not after 'TFAIL:'. I wouldn't use ':'. Output doesn't tell, what's wrong. IMHO there should be diff of expected output vs. real output. This is done in xfstests. + It could be less verbose. Let's use: $ ./lib/newlib_tests/test.shell_lib.sh TFAIL test.TST_TEST_DATA.getopts [ diff output whats wrong ] TPASS test.TST_TEST Summary: passed 1 failed 1 The name itself would IMHO be better test_tst_test.sh or test.tst_test.sh (but we don't use often dot in name). ATM we have old and new legacy API, test.shell_lib.sh doesn't say which it tests. > --- > Please do not merge! See following message for details. You can use --rfc switch for git format-patch next time. ... > --- a/lib/newlib_tests/test.TST_TEST.TST_CNT.separate.sh > +++ b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh > @@ -5,7 +5,7 @@ > TST_TESTFUNC=test > TST_CNT=2 > -. tst_test.sh > +. ./tst_test.sh This is not good. Much better is to make sure, that testcases/lib/ directory (where tst_test.sh is) is in PATH variable. The reasons are that you don't expect current working directory be in the same directory as the test. See: $ cd lib/newlib_tests $ ./shell/test.TST_TEST.TST_CNT.separate.sh bash: ./shell/test.TST_TEST.TST_CNT.separate.sh: No such file or directory And the tests are supposed to be also examples in docs, where we prefer not forcing cd into /opt/ltp/testcases/bin. ... > diff --git a/lib/newlib_tests/test.shell_lib.sh b/lib/newlib_tests/test.shell_lib.sh > new file mode 100755 > index 000000000..599eec78a > --- /dev/null > +++ b/lib/newlib_tests/test.shell_lib.sh > @@ -0,0 +1,94 @@ > +# !/bin/sh ^ Please no space after. checkbashisms.pl [1] warns you about it: $ checkbashisms.pl -f test.shell_lib.sh script test.shell_lib.sh does not appear to have a #! interpreter line; you may get strange results > +# > +# This script iterates over all test cases for the new shell lib and verifies > +# the output. > +# Do NOT use newline symbols in the names of files containing test cases! > +# Maybe you want to add your copyright here. You should add license (GPL v2 + any other). I prefer SPDX-License-Identifier as it's short: # SPDX-License-Identifier: GPL-2.0-or-later > + > +setup() { > + color_green="\033[1;32m" > + color_red="\033[1;31m" > + standard_color="\033[0m" > + start_dir="$PWD" > + cd "$(dirname "$0")""/../../" || exit 1; > + sh_lib_dir=""$PWD"/testcases/lib/" > + sh_lib_test_dir=""$PWD"/lib/newlib_tests/shell/" Wrong quote usage. In this interpretation $PWD isn't quoted. It should be: sh_lib_test_dir="$PWD/lib/newlib_tests/shell/" You probably want to setup it as global variable (out of all functions) - in this case global isn't that bad. Also I'd prefer simpler name. Using dirname does not require to be in git root: TESTDIR="$(dirname $0)/shell/" > + tst_cases=$(ls "$sh_lib_test_dir" | \ > + sed "s/"$TEST_NAME".sh/"$TEST_NAME"/g") > + cd "$sh_lib_dir" || exit 1 > +} > + > +check_requirements() { > + case "$0" in > + -*) > + printf "Please execute this script. Sourcing "; > + printf "(. <SCRIPT>) is not supported. \n"; > + return 1;; > + *) > + true; > + esac > +} Is this really needed? I think it's obvious. Now some syntax details. Good example of shell script is the library itself (testcases/lib/tst_test.sh) > +verify_output() { New line before { } verify_output() { > + local output_found=1 > + local wanted_output= > + local parsed_line= > + while read line; > + do while read line; do Keep 'do', 'then', 'else' on the same line > + if [ -z "$wanted_output" ] && [ "$line" = "# output:" ] > + then if [ -z "$wanted_output" ] && [ "$line" = "# output:" ]; then > + output_found=0 > + elif [ $output_found -eq 0 ] || [ -n "$wanted_output" ] > + then elif [ $output_found -eq 0 ] || [ -n "$wanted_output" ], then > + if printf "$line" | grep "# " > /dev/null; > + then > + if [ $output_found -eq 0 ] > + then > + parsed_line=$(printf "$line" | \ > + sed "s/^.\{2\}//") > + output_found=1 > + else > + parsed_line="\n"$(printf "$line" | \ > + sed "s/^.\{2\}//") > + fi > + elif printf "$line" | grep "#" > /dev/null; > + then > + parsed_line="\n"$(printf "$line" | \ > + sed "s/^.\{1\}//") > + else true; What is this for? You don't compare $?, which would be set by true. And it's ugly. > + fi > + wanted_output=""$wanted_output""$parsed_line"" > + else true; > + fi > + > + done < ""$sh_lib_test_dir""$1".sh" The easiest way is to store the output in separate file. Your way parsing the output is interesting, but maybe too complicated for updating the pattern. Even if you use this way, I'd prefer to get the output with sed/awk. Maybe there are better ways, but this would work: $ sed -ne '/^# output:/,$ p;' lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh | sed '1d' # test 1 TPASS: Test 1 passed with data 'foo:bar:d' # test 2 TPASS: Test 2 passed with data 'foo:bar:d' # test 3 TPASS: Test 1 passed with data 'dd' # test 4 TPASS: Test 2 passed with data 'dd' As I wrote, IMHO echo expected and actual output into the file and then run 'diff -u' on it would show, what's wrong. > + wanted_output=$(printf "$wanted_output") You probably don't want this (adding to wanted_output variable what is already there :) ). > + local actual_output=$(""$sh_lib_test_dir""$1".sh") Again, wrong quotes usage. > + actual_output=$(printf "$actual_output") Same here. Maybe use less verbose variable name (detail). > + if [ "$wanted_output" = "$actual_output" ] > + then > + return 0 > + else > + return 1 > + fi You can use [ "$wanted_output" = "$actual_output" ] && return 0 return 1 > +} > +run_tests() { > + for tst_case in $tst_cases > + do > + printf "Running Test: \""$tst_case"\"...\n" > + if verify_output "$tst_case"; > + then > + printf ""$color_green"TPASS: "$standard_color"" Again, wrong quotes usage. > + printf "Test "$tst_case" was successful.\n\n" > + else > + printf ""$color_red"TFAIL:"$standard_color"" > + printf "Test "$tst_case" was unsuccessful.\n\n" > + fi Again use '; do/then/else'. > + done > +} > + > +check_requirements > +setup > +run_tests > +exit 0 That's wrong. I'd like exit code was 0 for everything pass or 1 for failure. Kind regards, Petr [1] https://salsa.debian.org/debian/devscripts/raw/master/scripts/checkbashisms.pl ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [PATCH 1/2] Make shell lib tests standalone 2018-08-29 17:24 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Petr Vorel @ 2018-08-29 17:30 ` Petr Vorel 2018-08-31 15:24 ` [LTP] [RFC PATCH 0/1] Add automated tests for shell lib Christian Lanig 1 sibling, 0 replies; 23+ messages in thread From: Petr Vorel @ 2018-08-29 17:30 UTC (permalink / raw) To: ltp Hi Christian, > $ sed -ne '/^# output:/,$ p;' lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh | sed '1d' > # test 1 TPASS: Test 1 passed with data 'foo:bar:d' > # test 2 TPASS: Test 2 passed with data 'foo:bar:d' > # test 3 TPASS: Test 1 passed with data 'dd' > # test 4 TPASS: Test 2 passed with data 'dd' Sorry, this is the correct command: $ sed -ne '/^# output:/,$ p;' lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh | sed '1d; s/^# //' test 1 TPASS: Test 1 passed with data 'foo:bar:d' test 2 TPASS: Test 2 passed with data 'foo:bar:d' test 3 TPASS: Test 1 passed with data 'dd' test 4 TPASS: Test 2 passed with data 'dd' But I bet other will be against keeping the output into the file. + It'd be nice to describe, how testing works - how to modify the output (in README.md or at least in commit message). Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH 0/1] Add automated tests for shell lib 2018-08-29 17:24 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Petr Vorel 2018-08-29 17:30 ` Petr Vorel @ 2018-08-31 15:24 ` Christian Lanig 2018-08-31 15:24 ` [LTP] [RFC PATCH 1/1] " Christian Lanig 1 sibling, 1 reply; 23+ messages in thread From: Christian Lanig @ 2018-08-31 15:24 UTC (permalink / raw) To: ltp The following patch adds the test cases by Petr Vorel. They can be executed by automatically. doc/write-tests-for-shell-lib.txt describes how everything works. The output tells whether each test has passed and when it failed, the diff is printed. I apologize for the badly formatted message I sent before and hope that this time I used the right format options. @Petr > + It'd be nice to describe, how testing works - how to modify the output > (in README.md or at least in commit message). I am not sure that the text in doc/write-tests-for-shell-lib.txt should really go in README.md. Probably just a link or something. But I can change it if necessary. In regard of the other comments, I have removed all double quotes where nothing should go wrong as long as the person executing the script doesn't have unsupported file or directory naming or pretty uncommon bugs in his or her shell. I have also removed unnecessary else statements (which I thought would be necessary to stay compatible but this appeared to be a wrong impression when I read the text about it more accurately a second time). Also a -h / --help option was added which should be explanatory enough for people who intuitively run the script with this option and to replace comments about functionality in the header of the script file. Another feature is to specify single test cases as arguments to only run these and no other. Both outputs, the desired and the actual one, are now stored in a temporary folder and the PATH environment variable is modified temporarily, which implied the necessity to add a cleanup function. Regards, Christian Christian Lanig (1): Add automated tests for shell lib doc/write-tests-for-shell-lib.txt | 61 ++++++++++++ .../shell/test.TST_TEST.TST_CNT.separate.sh | 29 ++++++ lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh | 27 ++++++ lib/newlib_tests/shell/test.TST_TEST.getopts.sh | 48 ++++++++++ lib/newlib_tests/shell/test.TST_TEST.sh | 32 +++++++ .../shell/test.TST_TEST_DATA.TST_CNT.separate.sh | 32 +++++++ .../shell/test.TST_TEST_DATA.TST_CNT.sh | 30 ++++++ .../shell/test.TST_TEST_DATA.getopts.sh | 50 ++++++++++ lib/newlib_tests/shell/test.TST_TEST_DATA.sh | 24 +++++ .../shell/test.TST_TEST_DATA_IFS.getopts.sh | 52 ++++++++++ lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh | 27 ++++++ lib/newlib_tests/test_sh_newlib.sh | 105 +++++++++++++++++++++ 12 files changed, 517 insertions(+) create mode 100644 doc/write-tests-for-shell-lib.txt create mode 100755 lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh create mode 100755 lib/newlib_tests/test_sh_newlib.sh -- 2.16.4 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH 1/1] Add automated tests for shell lib 2018-08-31 15:24 ` [LTP] [RFC PATCH 0/1] Add automated tests for shell lib Christian Lanig @ 2018-08-31 15:24 ` Christian Lanig 2018-10-03 9:51 ` Cyril Hrubis 2018-10-03 11:32 ` Petr Vorel 0 siblings, 2 replies; 23+ messages in thread From: Christian Lanig @ 2018-08-31 15:24 UTC (permalink / raw) To: ltp Signed-off-by: Christian Lanig <clanig@suse.com> --- doc/write-tests-for-shell-lib.txt | 61 ++++++++++++ .../shell/test.TST_TEST.TST_CNT.separate.sh | 29 ++++++ lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh | 27 ++++++ lib/newlib_tests/shell/test.TST_TEST.getopts.sh | 48 ++++++++++ lib/newlib_tests/shell/test.TST_TEST.sh | 32 +++++++ .../shell/test.TST_TEST_DATA.TST_CNT.separate.sh | 32 +++++++ .../shell/test.TST_TEST_DATA.TST_CNT.sh | 30 ++++++ .../shell/test.TST_TEST_DATA.getopts.sh | 50 ++++++++++ lib/newlib_tests/shell/test.TST_TEST_DATA.sh | 24 +++++ .../shell/test.TST_TEST_DATA_IFS.getopts.sh | 52 ++++++++++ lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh | 27 ++++++ lib/newlib_tests/test_sh_newlib.sh | 105 +++++++++++++++++++++ 12 files changed, 517 insertions(+) create mode 100644 doc/write-tests-for-shell-lib.txt create mode 100755 lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh create mode 100755 lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh create mode 100755 lib/newlib_tests/test_sh_newlib.sh diff --git a/doc/write-tests-for-shell-lib.txt b/doc/write-tests-for-shell-lib.txt new file mode 100644 index 000000000..6f3ff4854 --- /dev/null +++ b/doc/write-tests-for-shell-lib.txt @@ -0,0 +1,61 @@ +How to format tests in order to test the shell library +====================================================== + +It is of big importance not only to test the Linux kernel functionality but also +to make sure LTP is running correctly itself. For this reason it is useful to +test intrinsic functionality of LTP - to reasonable extend. + +1. Running tests for the shell library +-------------------------------------- +The test cases reside in the folder `lib/newlib_tests/shell`. A script executing +them one by one is in the folder `lib/newlib_tests`. You can execute this script +to test all cases or specify test cases to be run. The script is called +`test_sh_newlib.sh`. + +2. Writing tests for the shell library +-------------------------------------- +The tests are written like all other test cases using the shell library. +Additionally, at the end of the file the desired output is added. As an example: + +[source,shell] +------------------------------------------------------------------------------- +#!/bin/sh +# +# This is a basic test for true shell buildin +# + +TST_TESTFUNC=do_test +. tst_test.sh + +do_test() +{ + true + ret=$? + + tst_res TINFO "Test $1 passed with no data ('$2')" + + if [ $ret -eq 0 ]; then + tst_res TPASS "true returned 0" + else + tst_res TFAIL "true returned $ret" + fi +} + +tst_run +# output: +# test 1 TINFO: Test 1 passed with no data ('') +# test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 +------------------------------------------------------------------------------- + +The most noticeable thing is to add the line `# output:` to show the parser that +parsing should start in the following line. For the following lines the `# ` +will be stripped before the output is then compared with the actual output that +gets printed on the terminal when running the test. Also make sure that there +are no trailing lines at the end of the document. But make sure to add one +trailing space after the `#` when the line will be actually empty. diff --git a/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh new file mode 100755 index 000000000..5e8d4bfb5 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.separate.sh @@ -0,0 +1,29 @@ +#!/bin/sh +# +# Example test with tests in separate functions +# + +TST_TESTFUNC=test +TST_CNT=2 +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +test2() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh new file mode 100755 index 000000000..62fa652cf --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST.TST_CNT.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# +# Example test with tests in a single function +# + +TST_TESTFUNC=do_test +TST_CNT=2 +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with no data ('$2')";; + 2) tst_res TPASS "Test $1 passed with no data ('$2')";; + esac +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh new file mode 100755 index 000000000..65810b5e8 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh @@ -0,0 +1,48 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST.sh b/lib/newlib_tests/shell/test.TST_TEST.sh new file mode 100755 index 000000000..fa6aaa57f --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST.sh @@ -0,0 +1,32 @@ +#!/bin/sh +# +# This is a basic test for true shell buildin +# + +TST_TESTFUNC=do_test +. tst_test.sh + +do_test() +{ + true + ret=$? + + tst_res TINFO "Test $1 passed with no data ('$2')" + + if [ $ret -eq 0 ]; then + tst_res TPASS "true returned 0" + else + tst_res TFAIL "true returned $ret" + fi +} + +tst_run +# output: +# test 1 TINFO: Test 1 passed with no data ('') +# test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh new file mode 100755 index 000000000..0ae773519 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.separate.sh @@ -0,0 +1,32 @@ +#!/bin/sh +# +# Example test with tests in separate functions, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +test2() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' +# +# Summary: +# passed 4 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh new file mode 100755 index 000000000..71e6acac7 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.TST_CNT.sh @@ -0,0 +1,30 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=do_test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with data '$2'";; + 2) tst_res TPASS "Test $1 passed with data '$2'";; + esac +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' +# +# Summary: +# passed 4 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh new file mode 100755 index 000000000..e1f18ba3a --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.getopts.sh @@ -0,0 +1,50 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo0:bar:d': a: '0', f: 'foo' +# test 2 TPASS: Test 1 passed with data 'dd': a: '0', f: 'foo' +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA.sh new file mode 100755 index 000000000..d3ced1954 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA.sh @@ -0,0 +1,24 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 1 passed with data 'dd' +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh new file mode 100755 index 000000000..6075904a5 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.getopts.sh @@ -0,0 +1,52 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" +TST_TEST_DATA_IFS=":" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo0': a: '0', f: 'foo' +# test 2 TPASS: Test 1 passed with data 'bar': a: '0', f: 'foo' +# test 3 TPASS: Test 1 passed with data 'd dd': a: '0', f: 'foo' +# +# Summary: +# passed 3 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh new file mode 100755 index 000000000..9a1bb9cf7 --- /dev/null +++ b/lib/newlib_tests/shell/test.TST_TEST_DATA_IFS.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and +# $TST_TEST_DATA_IFS +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +TST_TEST_DATA_IFS=":" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TPASS: Test 1 passed with data 'foo' +# test 2 TPASS: Test 1 passed with data 'bar' +# test 3 TPASS: Test 1 passed with data 'd dd' +# +# Summary: +# passed 3 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/test_sh_newlib.sh b/lib/newlib_tests/test_sh_newlib.sh new file mode 100755 index 000000000..bf335dd6e --- /dev/null +++ b/lib/newlib_tests/test_sh_newlib.sh @@ -0,0 +1,105 @@ +#!/bin/sh +# +# SPDX-License-Identifier: GPL-2.0-or-later +# © 2018 SUSE LLC. +# +# Author: Christian Lanig <clanig@suse.com> + +setup() +{ + path_sav=$PATH + PATH=""$PATH":$(dirname $0)/../../testcases/lib/" + color_blue='\033[1;34m' + color_green='\033[1;32m' + color_red='\033[1;31m' + standard_color='\033[0m' + tmp_dir="/tmp/sh_lib_tst-$$/" + mkdir $tmp_dir || cleanup 1 + sh_lib_tst_dir="$(dirname $0)/shell/" + tst_files="$(ls $sh_lib_tst_dir)" + return 0 +} + +cleanup() +{ + [ -d "$tmp_dir" ] && rm -rf "$tmp_dir" + PATH="$PATH_sav" + exit $1 +} + +print_help() +{ + printf "#############################################################\n" + printf "# This script iterates over test cases for the new shell #\n" + printf "# library and verifies the output. #\n" + printf "#############################################################\n" + printf "\n" + printf "Usage:\n" + printf "\t$(basename $0)\n" + printf "\t$(basename $0) [test file 1] [test file 2] ...\n\n" + exit 0; +} + +parse_params() +{ + [ -n "$1" ] && tst_files= + while [ -n "$1" ]; do + case "$1" in + --help) print_help;; + -h) print_help;; + -*) + printf "Unknown positional parameter "$1".\n" + cleanup 1;; + *) tst_files="$tst_files $1";; + esac + shift + done + return 0 +} + +verify_output() +{ + [ -e "$sh_lib_tst_dir$tst" ] || { printf "$tst not found\n" && \ + cleanup 1 ;} + # read all lines after line: `# output:`, and strip `# ` from beginning + sed -n -e '/^# output:/,$ p;' "$sh_lib_tst_dir$tst" | sed '1d; s/^# //'\ + > "$tmp_dir$tst.wanted" || cleanup 1 + + actual_output=$($sh_lib_tst_dir$tst) + # remove control signs, add newline at EOF and store in temporary file + actual_output=$(printf "$actual_output\n" > $tmp_dir$tst".actual") || + cleanup 1 + + cmp $tmp_dir$tst".actual" $tmp_dir$tst".wanted" > /dev/null && return 0 + return 1 +} + +run_tests() +{ + pass_cnt=0 + fail_cnt=0 + printf "\n" + for tst in $tst_files; do + if verify_output; then + pass_cnt=$(expr $pass_cnt + 1) + printf ""$color_green"TPASS$standard_color $tst" + printf "\n" + else + fail_cnt=$(expr $fail_cnt + 1) + printf ""$color_red"TFAIL$standard_color $tst\n" + printf ""$color_blue"Diff:$standard_color\n" + diff -u "$tmp_dir$tst.actual" "$tmp_dir$tst.wanted" + printf "\n" + fi + done + printf "\nSummary:\n" + printf ""$color_red"Failed:$standard_color $fail_cnt\n" + printf ""$color_green"Passed:$standard_color $pass_cnt\n\n" + return $fail_cnt +} + +setup +parse_params "$@" +run_tests +res=$? +cleanup $res -- 2.16.4 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH 1/1] Add automated tests for shell lib 2018-08-31 15:24 ` [LTP] [RFC PATCH 1/1] " Christian Lanig @ 2018-10-03 9:51 ` Cyril Hrubis 2018-10-03 10:46 ` Petr Vorel 2018-10-03 11:32 ` Petr Vorel 1 sibling, 1 reply; 23+ messages in thread From: Cyril Hrubis @ 2018-10-03 9:51 UTC (permalink / raw) To: ltp Hi! > diff --git a/lib/newlib_tests/shell/test.TST_TEST.getopts.sh b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh > new file mode 100755 > index 000000000..65810b5e8 > --- /dev/null > +++ b/lib/newlib_tests/shell/test.TST_TEST.getopts.sh > @@ -0,0 +1,48 @@ > +#!/bin/sh > +# > +# Optional test command line parameters > +# > + > +TST_OPTS="af:" > +TST_USAGE=usage > +TST_PARSE_ARGS=parse_args > +TST_TESTFUNC=do_test > + > +. tst_test.sh > + > +ALTERNATIVE=0 > +MODE="foo" > + > +usage() > +{ > + cat << EOF > +usage: $0 [-a] [-f <foo|bar>] > + > +OPTIONS > +-a Enable support for alternative foo > +-f Specify foo or bar mode > +EOF > +} > + > +parse_args() > +{ > + case $1 in > + a) ALTERNATIVE=1;; > + f) MODE="$2";; > + esac > +} > + > +do_test() > +{ > + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" > +} > + > +tst_run > +# output: > +# test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' > +# > +# Summary: > +# passed 1 > +# failed 0 > +# skipped 0 > +# warnings 0 We do need a way how to pass parameters to the tests so that we can test this functionality properly (and also the default options that are implemented in the test library). Obvious way how to do this is to allow to have several output: blocks at the end of the test with a different parameters, something as: # output: # test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' # # Summary: # passed 1 # failed 0 # skipped 0 # warnings 0 # output: # params: -a # test 1 TPASS: Test 1 passed with data '': a: '1', f: 'foo' # # Summary: # passed 1 # failed 0 # skipped 0 # warnings 0 ... > diff --git a/lib/newlib_tests/test_sh_newlib.sh b/lib/newlib_tests/test_sh_newlib.sh > new file mode 100755 > index 000000000..bf335dd6e > --- /dev/null > +++ b/lib/newlib_tests/test_sh_newlib.sh > @@ -0,0 +1,105 @@ > +#!/bin/sh > +# > +# SPDX-License-Identifier: GPL-2.0-or-later > +# ?? 2018 SUSE LLC. > +# > +# Author: Christian Lanig <clanig@suse.com> > + > +setup() > +{ > + path_sav=$PATH There is no need to save and restore the PATH, the script will be executed in subshell and the changes to variables are never propagated to the parent shell. The only way how variables can be changed in the currently running shell is to source the script, in that case the current shell will execute the commands. > + PATH=""$PATH":$(dirname $0)/../../testcases/lib/" > + > + color_blue='\033[1;34m' > + color_green='\033[1;32m' > + color_red='\033[1;31m' > + standard_color='\033[0m' ^ This is more of 'reset text attributes' as it not only changes the color to default but also turns off bold, underscore, blink, etc. So it would be better named something as reset_attrs='\033[0m' > + tmp_dir="/tmp/sh_lib_tst-$$/" > + mkdir $tmp_dir || cleanup 1 > + sh_lib_tst_dir="$(dirname $0)/shell/" > + tst_files="$(ls $sh_lib_tst_dir)" > + return 0 > +} > + > +cleanup() > +{ > + [ -d "$tmp_dir" ] && rm -rf "$tmp_dir" > + PATH="$PATH_sav" > + exit $1 > +} > + > +print_help() > +{ > + printf "#############################################################\n" > + printf "# This script iterates over test cases for the new shell #\n" > + printf "# library and verifies the output. #\n" > + printf "#############################################################\n" > + printf "\n" > + printf "Usage:\n" > + printf "\t$(basename $0)\n" > + printf "\t$(basename $0) [test file 1] [test file 2] ...\n\n" > + exit 0; > +} > + > +parse_params() > +{ > + [ -n "$1" ] && tst_files= > + while [ -n "$1" ]; do > + case "$1" in > + --help) print_help;; > + -h) print_help;; > + -*) > + printf "Unknown positional parameter "$1".\n" > + cleanup 1;; > + *) tst_files="$tst_files $1";; > + esac > + shift > + done > + return 0 > +} > + > +verify_output() > +{ > + [ -e "$sh_lib_tst_dir$tst" ] || { printf "$tst not found\n" && \ > + cleanup 1 ;} > + # read all lines after line: `# output:`, and strip `# ` from beginning > + sed -n -e '/^# output:/,$ p;' "$sh_lib_tst_dir$tst" | sed '1d; s/^# //'\ > + > "$tmp_dir$tst.wanted" || cleanup 1 Usually doing more complex text manipulation is easier with awk In this case you can use gsub() function that will replace substring and print all lines that were replaced (i.e. print all comments without the hash and space): awk 'gsub("^# ", "", $0)' script.sh Then you can even get one of the output: blocks with: awk -vb=1 'gsub("^# ", "", $0) {if (/output:/) {p++}; if (p==b && !/output:/) {print $0}};' script.sh ^ This gives you content of comments starting at first # output: line up to the next # output: or end of line, which is exactly what we need if we want to support passing parameters to testcases. Then we can simply count how many # output: lines are in a file and then run this command in a loop extracting one example output after another. > + actual_output=$($sh_lib_tst_dir$tst) > + # remove control signs, add newline at EOF and store in temporary file > + actual_output=$(printf "$actual_output\n" > $tmp_dir$tst".actual") || > + cleanup 1 > + > + cmp $tmp_dir$tst".actual" $tmp_dir$tst".wanted" > /dev/null && return 0 > + return 1 > +} > + > +run_tests() > +{ > + pass_cnt=0 > + fail_cnt=0 > + printf "\n" > + for tst in $tst_files; do > + if verify_output; then > + pass_cnt=$(expr $pass_cnt + 1) You can use arithmetic expansion here, i.e. pass_cnt=$((pass_cnt+1)) > + printf ""$color_green"TPASS$standard_color $tst" > + printf "\n" > + else > + fail_cnt=$(expr $fail_cnt + 1) And here as well. > + printf ""$color_red"TFAIL$standard_color $tst\n" > + printf ""$color_blue"Diff:$standard_color\n" > + diff -u "$tmp_dir$tst.actual" "$tmp_dir$tst.wanted" > + printf "\n" > + fi > + done > + printf "\nSummary:\n" > + printf ""$color_red"Failed:$standard_color $fail_cnt\n" > + printf ""$color_green"Passed:$standard_color $pass_cnt\n\n" > + return $fail_cnt > +} > + > +setup > +parse_params "$@" > +run_tests > +res=$? > +cleanup $res > -- > 2.16.4 > -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH 1/1] Add automated tests for shell lib 2018-10-03 9:51 ` Cyril Hrubis @ 2018-10-03 10:46 ` Petr Vorel 0 siblings, 0 replies; 23+ messages in thread From: Petr Vorel @ 2018-10-03 10:46 UTC (permalink / raw) To: ltp Hi, > > +do_test() > > +{ > > + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" > > +} > > + > > +tst_run > > +# output: > > +# test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' > > +# > > +# Summary: > > +# passed 1 > > +# failed 0 > > +# skipped 0 > > +# warnings 0 > We do need a way how to pass parameters to the tests so that we can test > this functionality properly (and also the default options that are > implemented in the test library). > Obvious way how to do this is to allow to have several output: blocks at > the end of the test with a different parameters, something as: > # output: > # test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' > # Summary: > # passed 1 > # failed 0 > # skipped 0 > # warnings 0 > # output: > # params: -a > # test 1 TPASS: Test 1 passed with data '': a: '1', f: 'foo' > # Summary: > # passed 1 > # failed 0 > # skipped 0 > # warnings 0 > ... Good idea! ... > > +verify_output() > > +{ > > + [ -e "$sh_lib_tst_dir$tst" ] || { printf "$tst not found\n" && \ > > + cleanup 1 ;} > > + # read all lines after line: `# output:`, and strip `# ` from beginning > > + sed -n -e '/^# output:/,$ p;' "$sh_lib_tst_dir$tst" | sed '1d; s/^# //'\ > > + > "$tmp_dir$tst.wanted" || cleanup 1 > Usually doing more complex text manipulation is easier with awk > In this case you can use gsub() function that will replace substring and > print all lines that were replaced (i.e. print all comments without the > hash and space): > awk 'gsub("^# ", "", $0)' script.sh > Then you can even get one of the output: blocks with: > awk -vb=1 'gsub("^# ", "", $0) {if (/output:/) {p++}; if (p==b && !/output:/) {print $0}};' script.sh > ^ > This gives you content of comments starting at first # output: > line up to the next # output: or end of line, which is exactly > what we need if we want to support passing parameters to > testcases. Then we can simply count how many # output: lines are > in a file and then run this command in a loop extracting one > example output after another. I wonder whether "extracting one example output after another" is still good to do in awk/sed/shell (with simple readable code). Probably yes, but I wouldn't mind to have either python (pytest) or some perl code doing it. Awk is everywhere and definitely more capable than sed or even shell, but not everybody is fluent in it. Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH 1/1] Add automated tests for shell lib 2018-08-31 15:24 ` [LTP] [RFC PATCH 1/1] " Christian Lanig 2018-10-03 9:51 ` Cyril Hrubis @ 2018-10-03 11:32 ` Petr Vorel 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 0/1] " Christian Lanig 1 sibling, 1 reply; 23+ messages in thread From: Petr Vorel @ 2018-10-03 11:32 UTC (permalink / raw) To: ltp Hi Christian, some more comments (beside these mentioned by Cyril). Mostly just shell code style comments. > +test2() > +{ > + tst_res TPASS "Test $1 passed with no data ('$2')" > +} > + > +tst_run > +# output: > +# test 1 TPASS: Test 1 passed with no data ('') > +# test 2 TPASS: Test 2 passed with no data ('') > +# Note, I'd prefer not have trailing whitespace after '#' (actually anywhere). Not only it's unnecessary it's also problematic for 'git am' applying patches. In my case all the tests failed because of it. Also some people setup their editors / IDE to delete trailing whitespace. (Not pointing all whitespace here.) > +# Summary: > +# passed 2 > +# failed 0 > +# skipped 0 > +# warnings 0 > diff --git a/lib/newlib_tests/test_sh_newlib.sh b/lib/newlib_tests/test_sh_newlib.sh ... > +setup() > +{ > + path_sav=$PATH > + PATH=""$PATH":$(dirname $0)/../../testcases/lib/" > + color_blue='\033[1;34m' > + color_green='\033[1;32m' > + color_red='\033[1;31m' > + standard_color='\033[0m' Maybe this variables can be outside of setup (no benefit to have them in setup. > + tmp_dir="/tmp/sh_lib_tst-$$/" Although /tmp is everywhere, maybe using $TMPDIR would be good. See tst_test.sh code: if [ -z "$TMPDIR" ]; then export TMPDIR="/tmp" fi TST_TMPDIR=$(mktemp -d "$TMPDIR/LTP_$TST_ID.XXXXXXXXXX") chmod 777 "$TST_TMPDIR" > + mkdir $tmp_dir || cleanup 1 > + sh_lib_tst_dir="$(dirname $0)/shell/" Maybe just tstdir ? > + tst_files="$(ls $sh_lib_tst_dir)" > +print_help() > +{ > + printf "#############################################################\n" > + printf "# This script iterates over test cases for the new shell #\n" > + printf "# library and verifies the output. #\n" > + printf "#############################################################\n" > + printf "\n" > + printf "Usage:\n" > + printf "\t$(basename $0)\n" > + printf "\t$(basename $0) [test file 1] [test file 2] ...\n\n" > + exit 0; Maybe just: cat <<EOF Unit tests for new shell API Usage: $(basename $0) [ TEST1 ] [ TEST2 ] EOF > +} > + > +parse_params() > +{ > + [ -n "$1" ] && tst_files= > + while [ -n "$1" ]; do > + case "$1" in > + --help) print_help;; > + -h) print_help;; > + -*) > + printf "Unknown positional parameter "$1".\n" > + cleanup 1;; > + *) tst_files="$tst_files $1";; > + esac > + shift > + done > + return 0 Why return, when you don't use it elsewhere? ... > +verify_output() > +{ > + [ -e "$sh_lib_tst_dir$tst" ] || { printf "$tst not found\n" && \ > + cleanup 1 ;} This might be more readable (nitpicking): if [ -n "$sh_lib_tst_dir$tst" ]; then echo "$tst not found" cleanup 1 fi > + # read all lines after line: `# output:`, and strip `# ` from beginning > + sed -n -e '/^# output:/,$ p;' "$sh_lib_tst_dir$tst" | sed '1d; s/^# //'\ > + > "$tmp_dir$tst.wanted" || cleanup 1 > + > + actual_output=$($sh_lib_tst_dir$tst) > + # remove control signs, add newline at EOF and store in temporary file > + actual_output=$(printf "$actual_output\n" > $tmp_dir$tst".actual") || > + cleanup 1 Suppose this part will be rewritten. But why do you assign empty string? echo "$actual_output" > $tmp_dir$tst".actual || cleanup 1 ... > + > + cmp $tmp_dir$tst".actual" $tmp_dir$tst".wanted" > /dev/null && return 0 > + return 1 This is enough (cmp returns it for you): cmp -s "$tmp_dir$tst.actual" "$tmp_dir$tst.wanted" > +} ... > +run_tests() > +{ > + pass_cnt=0 > + fail_cnt=0 > + printf "\n" > + for tst in $tst_files; do > + if verify_output; then > + pass_cnt=$(expr $pass_cnt + 1) > + printf ""$color_green"TPASS$standard_color $tst" ^ "" is empty string, why? > + printf "\n" You can also use {} to separate variable name: echo "${color_green}TPASS$standard_color $tst" echo adds new line for you (for simple usage it's portable). It's safer to quote whole string (with variables). > + else > + fail_cnt=$(expr $fail_cnt + 1) > + printf ""$color_red"TFAIL$standard_color $tst\n" > + printf ""$color_blue"Diff:$standard_color\n" > + diff -u "$tmp_dir$tst.actual" "$tmp_dir$tst.wanted" > + printf "\n" > + fi > + done > + printf "\nSummary:\n" > + printf ""$color_red"Failed:$standard_color $fail_cnt\n" > + printf ""$color_green"Passed:$standard_color $pass_cnt\n\n" And how about adding helper functions print_red, print_green, print_blue? Maybe not worth for this little script. print_red() { printf "$red$1$reset" } print_red "TFAIL echo "$tst" > + return $fail_cnt > +} > + > +setup > +parse_params "$@" > +run_tests > +res=$? > +cleanup $res No need to have $res: run_tests cleanup $? Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 0/1] Add automated tests for shell lib 2018-10-03 11:32 ` Petr Vorel @ 2019-08-22 19:12 ` Christian Lanig 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 1/1] " Christian Lanig 2019-09-19 14:26 ` [LTP] [RFC PATCH v2 0/1] " Petr Vorel 0 siblings, 2 replies; 23+ messages in thread From: Christian Lanig @ 2019-08-22 19:12 UTC (permalink / raw) To: ltp Hi, I am sorry that my reply took so long. Thank you very much for your comments which helped me to improve the code. I have made an overhaul of the code based on your suggestions but an option to pass parameters to the tests is still not implemented. I'm sending you what I have so far because it would help me when you could provide me more information regarding this. I agree that it is difficult to find a way to process text the way needed here without disregarding the Linux Kernel Coding Style that asks for code which is transparent and easy to read when we only consider Shell commands. To address this issue I have drafted a Python script that might go in the tools folder and may perhaps be helpful in other cases as well. Since there are already Python files in the project it might be a valid option. Thank you very much for your patience. Regards, Christian Lanig Christian Lanig (1): Add automated tests for shell lib doc/write-tests-for-shell-lib.txt | 59 ++++++++++ lib/newlib_tests/shell/test_sh_newlib.sh | 102 ++++++++++++++++++ .../testcases/test.TST_TEST.TST_CNT.separate.sh | 30 ++++++ .../shell/testcases/test.TST_TEST.TST_CNT.sh | 28 +++++ .../shell/testcases/test.TST_TEST.getopts.sh | 49 +++++++++ lib/newlib_tests/shell/testcases/test.TST_TEST.sh | 33 ++++++ .../test.TST_TEST_DATA.TST_CNT.separate.sh | 33 ++++++ .../shell/testcases/test.TST_TEST_DATA.TST_CNT.sh | 31 ++++++ .../shell/testcases/test.TST_TEST_DATA.getopts.sh | 51 +++++++++ .../shell/testcases/test.TST_TEST_DATA.sh | 25 +++++ .../testcases/test.TST_TEST_DATA_IFS.getopts.sh | 53 +++++++++ .../shell/testcases/test.TST_TEST_DATA_IFS.sh | 28 +++++ tools/lookup_split_cut.py | 120 +++++++++++++++++++++ 13 files changed, 642 insertions(+) create mode 100644 doc/write-tests-for-shell-lib.txt create mode 100755 lib/newlib_tests/shell/test_sh_newlib.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.sh create mode 100755 tools/lookup_split_cut.py -- 2.16.4 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 1/1] Add automated tests for shell lib 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 0/1] " Christian Lanig @ 2019-08-22 19:12 ` Christian Lanig 2019-09-19 16:41 ` Petr Vorel 2019-09-20 14:21 ` Clemens Famulla-Conrad 2019-09-19 14:26 ` [LTP] [RFC PATCH v2 0/1] " Petr Vorel 1 sibling, 2 replies; 23+ messages in thread From: Christian Lanig @ 2019-08-22 19:12 UTC (permalink / raw) To: ltp Signed-off-by: Christian Lanig <clanig@suse.com> --- doc/write-tests-for-shell-lib.txt | 59 ++++++++++ lib/newlib_tests/shell/test_sh_newlib.sh | 102 ++++++++++++++++++ .../testcases/test.TST_TEST.TST_CNT.separate.sh | 30 ++++++ .../shell/testcases/test.TST_TEST.TST_CNT.sh | 28 +++++ .../shell/testcases/test.TST_TEST.getopts.sh | 49 +++++++++ lib/newlib_tests/shell/testcases/test.TST_TEST.sh | 33 ++++++ .../test.TST_TEST_DATA.TST_CNT.separate.sh | 33 ++++++ .../shell/testcases/test.TST_TEST_DATA.TST_CNT.sh | 31 ++++++ .../shell/testcases/test.TST_TEST_DATA.getopts.sh | 51 +++++++++ .../shell/testcases/test.TST_TEST_DATA.sh | 25 +++++ .../testcases/test.TST_TEST_DATA_IFS.getopts.sh | 53 +++++++++ .../shell/testcases/test.TST_TEST_DATA_IFS.sh | 28 +++++ tools/lookup_split_cut.py | 120 +++++++++++++++++++++ 13 files changed, 642 insertions(+) create mode 100644 doc/write-tests-for-shell-lib.txt create mode 100755 lib/newlib_tests/shell/test_sh_newlib.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.separate.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.getopts.sh create mode 100755 lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.sh create mode 100755 tools/lookup_split_cut.py diff --git a/doc/write-tests-for-shell-lib.txt b/doc/write-tests-for-shell-lib.txt new file mode 100644 index 000000000..7e4e6f566 --- /dev/null +++ b/doc/write-tests-for-shell-lib.txt @@ -0,0 +1,59 @@ +How to format tests in order to test the shell library +====================================================== + +It is important to test the Linux kernel functionality but also to make sure +that LTP is running correctly itself. For this reason it is useful to test +intrinsic functionality of LTP. + +1. Running tests for the shell library +-------------------------------------- +The test cases reside in the folder `lib/newlib_tests/shell`. A script executing +them one by one is located in the folder `lib/newlib_tests`. You can execute +this script to test all cases or specify test cases to be run. The script is +called `test_sh_newlib.sh`. + +2. Writing tests for the shell library +-------------------------------------- +The tests are written like all other test cases using the shell library. +Additionally, at the end of the file the desired output is added. As an example: + +[source,shell] +------------------------------------------------------------------------------- +#!/bin/sh +# +# This is a basic test for true shell buildin +# + +TST_TESTFUNC=do_test +. tst_test.sh + +do_test() +{ + true + ret=$? + + tst_res TINFO "Test $1 passed with no data ('$2')" + + if [ $ret -eq 0 ]; then + tst_res TPASS "true returned 0" + else + tst_res TFAIL "true returned $ret" + fi +} + +tst_run +# output: +# test 1 TINFO: Test 1 passed with no data ('') +# test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 +------------------------------------------------------------------------------- + +The most noticeable thing is to add the line `# output:` to show the parser that +parsing should start in the following line. For the following lines the `# ` +will be stripped before the output is then compared with the actual output that +gets printed on the terminal when running the test. diff --git a/lib/newlib_tests/shell/test_sh_newlib.sh b/lib/newlib_tests/shell/test_sh_newlib.sh new file mode 100755 index 000000000..4aa19555b --- /dev/null +++ b/lib/newlib_tests/shell/test_sh_newlib.sh @@ -0,0 +1,102 @@ +#!/bin/sh +# +# SPDX-License-Identifier: GPL-2.0-or-later +# (c) 2019 SUSE LLC +# +# Author: Christian Lanig <clanig@suse.com> + +PATH="${PATH}:$(dirname $(readlink -f $0))/../../../testcases/lib/" +if [ -z "$TMPDIR" ]; then + export TMPDIR="/tmp" +fi +color_blue='\033[1;34m' +color_green='\033[1;32m' +color_red='\033[1;31m' +reset_attr='\033[0m' +tmp="${TMPDIR}/sh_lib_tst-${$}/" +mkdir $tmp || cleanup 1 +parent_dir=$(dirname $(readlink -f $0))/ +tooldir=${parent_dir}/../../../tools/ +testdir=${parent_dir}testcases/ +tst_files=$(ls $testdir) + +cleanup() +{ + [ -d "$tmp" ] && rm -rf "$tmp" + exit $1 +} + +print_help() +{ + cat <<EOF + +???????????????????????????????????????????????????????????????????????????????? +? This Shell script iterates over test cases for the new Shell library and ? +? verifies the output. ? +???????????????????????????????????????????????????????????????????????????????? + + Usage: + $(basename $0) [TEST_FILE_1] [TEST_FILE_2] + +EOF + exit 0 +} + +parse_params() +{ + [ -n "$1" ] && tst_files= + while [ -n "$1" ]; do + case "$1" in + --help) print_help;; + -h) print_help;; + -*) + printf "Unknown positional parameter ${1}.\n" + cleanup 1;; + *) tst_files="$tst_files $1";; + esac + shift + done +} + +verify_output() +{ + if [ ! -e "${testdir}$tst" ]; then + printf "$tst not found\n" + cleanup 1 + fi + + ${tooldir}lookup_split_cut.py -f ${testdir}$tst -d $tmp \ + -s '# output:\n' -c '# {0,1}' || cleanup 1 + + "${testdir}$tst" > "${tmp}$tst.actual" || cleanup 1 + cmp -s "${tmp}$tst.actual" "${tmp}${tst}_out/out.1" && return 0 + return 1 +} + +run_tests() +{ + pass_cnt=0 + fail_cnt=0 + printf "\n" + for tst in $tst_files; do + if verify_output; then + pass_cnt=$(($pass_cnt + 1)) + printf "${color_green}TPASS$reset_attr ${tst}\n" + else + fail_cnt=$(($fail_cnt + 1)) + printf "${color_red}TFAIL$reset_attr ${tst}\n" + printf "${color_blue}Diff:${reset_attr}\n" + diff -u "${tmp}${tst}.actual" \ + "${tmp}${tst}_out/out.1" + printf "\n" + fi + done + printf "\nSummary:\n" + printf "${color_red}Failed:$reset_attr $fail_cnt\n" + printf "${color_green}Passed:$reset_attr $pass_cnt\n\n" + return $fail_cnt +} + +parse_params "$@" +run_tests +cleanup $? diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.separate.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.separate.sh new file mode 100755 index 000000000..333c4f5fa --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.separate.sh @@ -0,0 +1,30 @@ +#!/bin/sh +# +# Example test with tests in separate functions +# + +TST_TESTFUNC=test +TST_CNT=2 +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +test2() +{ + tst_res TPASS "Test $1 passed with no data ('$2')" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.sh new file mode 100755 index 000000000..73abfc8b3 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST.TST_CNT.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# +# Example test with tests in a single function +# + +TST_TESTFUNC=do_test +TST_CNT=2 +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with no data ('$2')";; + 2) tst_res TPASS "Test $1 passed with no data ('$2')";; + esac +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with no data ('') +# test 2 TPASS: Test 2 passed with no data ('') +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST.getopts.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST.getopts.sh new file mode 100755 index 000000000..35a700bb2 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST.getopts.sh @@ -0,0 +1,49 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data '': a: '0', f: 'foo' +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST.sh new file mode 100755 index 000000000..930900e1d --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST.sh @@ -0,0 +1,33 @@ +#!/bin/sh +# +# This is a basic test for true shell buildin +# + +TST_TESTFUNC=do_test +. tst_test.sh + +do_test() +{ + true + ret=$? + + tst_res TINFO "Test $1 passed with no data ('$2')" + + if [ $ret -eq 0 ]; then + tst_res TPASS "true returned 0" + else + tst_res TFAIL "true returned $ret" + fi +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TINFO: Test 1 passed with no data ('') +# test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.separate.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.separate.sh new file mode 100755 index 000000000..1faa01a57 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.separate.sh @@ -0,0 +1,33 @@ +#!/bin/sh +# +# Example test with tests in separate functions, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +test1() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +test2() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' +# +# Summary: +# passed 4 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.sh new file mode 100755 index 000000000..889fc09c3 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.TST_CNT.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT +# + +TST_TESTFUNC=do_test +TST_CNT=2 +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + case $1 in + 1) tst_res TPASS "Test $1 passed with data '$2'";; + 2) tst_res TPASS "Test $1 passed with data '$2'";; + esac +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' +# test 3 TPASS: Test 1 passed with data 'dd' +# test 4 TPASS: Test 2 passed with data 'dd' +# +# Summary: +# passed 4 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.getopts.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.getopts.sh new file mode 100755 index 000000000..ba880f891 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.getopts.sh @@ -0,0 +1,51 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo0:bar:d': a: '0', f: 'foo' +# test 2 TPASS: Test 1 passed with data 'dd': a: '0', f: 'foo' +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.sh new file mode 100755 index 000000000..083943833 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA.sh @@ -0,0 +1,25 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' +# test 2 TPASS: Test 1 passed with data 'dd' +# +# Summary: +# passed 2 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.getopts.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.getopts.sh new file mode 100755 index 000000000..eb83d2e34 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.getopts.sh @@ -0,0 +1,53 @@ +#!/bin/sh +# +# Optional test command line parameters +# + +TST_OPTS="af:" +TST_USAGE=usage +TST_PARSE_ARGS=parse_args +TST_TESTFUNC=do_test +TST_TEST_DATA="foo0:bar:d dd" +TST_TEST_DATA_IFS=":" + +. tst_test.sh + +ALTERNATIVE=0 +MODE="foo" + +usage() +{ + cat << EOF +usage: $0 [-a] [-f <foo|bar>] + +OPTIONS +-a Enable support for alternative foo +-f Specify foo or bar mode +EOF +} + +parse_args() +{ + case $1 in + a) ALTERNATIVE=1;; + f) MODE="$2";; + esac +} + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2': a: '$ALTERNATIVE', f: '$MODE'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo0': a: '0', f: 'foo' +# test 2 TPASS: Test 1 passed with data 'bar': a: '0', f: 'foo' +# test 3 TPASS: Test 1 passed with data 'd dd': a: '0', f: 'foo' +# +# Summary: +# passed 3 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.sh b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.sh new file mode 100755 index 000000000..6c6f904b7 --- /dev/null +++ b/lib/newlib_tests/shell/testcases/test.TST_TEST_DATA_IFS.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# +# Example test with tests in a single function, using $TST_TEST_DATA and +# $TST_TEST_DATA_IFS +# + +TST_TESTFUNC=do_test +TST_TEST_DATA="foo:bar:d dd" +TST_TEST_DATA_IFS=":" +. tst_test.sh + +do_test() +{ + tst_res TPASS "Test $1 passed with data '$2'" +} + +tst_run +# output: +# test 1 TINFO: timeout per run is 0h 5m 0s +# test 1 TPASS: Test 1 passed with data 'foo' +# test 2 TPASS: Test 1 passed with data 'bar' +# test 3 TPASS: Test 1 passed with data 'd dd' +# +# Summary: +# passed 3 +# failed 0 +# skipped 0 +# warnings 0 diff --git a/tools/lookup_split_cut.py b/tools/lookup_split_cut.py new file mode 100755 index 000000000..2b3388ada --- /dev/null +++ b/tools/lookup_split_cut.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python +# +# SPDX-License-Identifier: GPL-2.0-or-later +# (c) 2019 SUSE LLC +# +# Author: Christian Lanig <clanig@suse.com> + +import sys +import os +import re +from sys import argv +from os import makedirs, path + +src_file_path = None +dest = None +delim = None +pattern = None +perim = None + +argv.reverse() +basename = path.split(argv.pop())[1] + +def print_help(): + + help = """ + This script can look up a passage in a specified text pattern, split a + text file and cut a pattern. The operation chain is: + + lookup > split > cut + + The output files are written to the specified destination directory. + + Usage: + """ + help += "\n\t\t" + basename + " -f [PATH] -d [PATH] -l " \ + + "[PERIMETER] -s [DELIMITER] \n" \ + + "\t\t\t\t -c [PATTERN]\n\n" + help += """ + -h, --help + print this help + -f, --file + source file to be processed + -d, --destination + destination path + -l, --lookup + look for data in text passage + -s, --split + split file by delimiter + -c, --cut + cut pattern from file + """ + + print(help) + sys.exit(0) + +def get_next_arg(): + if argv: + return argv.pop() + else: + print("Missing parameter. Run with \"--help\" for information.") + sys.exit(1) + +while argv: + arg = argv.pop() + if arg in ["-h", "--help"]: + print_help() + elif arg in ["-f", "--file"]: + src_file_path = get_next_arg() + elif arg in ["-d", "--destination"]: + dest = get_next_arg() + elif arg in ["-l", "--lookup"]: + perim = get_next_arg() + elif arg in ["-s", "--split"]: + delim = get_next_arg() + elif arg in ["-c", "--cut"]: + pattern = get_next_arg() + else: + print("Illegal argument. Run with \"--help\" for information.") + sys.exit(1) + +if not src_file_path: + print("Input file has to be specified.") + sys.exit(1) + +if not delim and not pattern and not perim: + print("Missing parameters. Run with \"--help\" for information.") + sys.exit(1) + +src_file = open(src_file_path, "r") +src = src_file.read() +src_file.close() + +capture = 0 +if perim: + try: + capture = re.search(perim, src).group(1) + except: + pass + +if delim: + src_file_name = path.split(src_file_path)[1] + out_dir = dest + "/" + src_file_name + "_out" + + if not path.exists(out_dir): + makedirs(out_dir) + + src = re.split(delim, src) +else: + src = [src] + +if pattern: + for i in range(len(src)): + src[i] = re.sub(pattern, "", src[i]) +if delim or pattern: + for i in range(len(src)): + out_file = open(out_dir + "/out." + str(i), "w") + out_file.write(src[i]) + out_file.close() + +sys.exit(capture) -- 2.16.4 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 1/1] Add automated tests for shell lib 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 1/1] " Christian Lanig @ 2019-09-19 16:41 ` Petr Vorel 2019-09-30 18:27 ` Christian Lanig 2019-09-20 14:21 ` Clemens Famulla-Conrad 1 sibling, 1 reply; 23+ messages in thread From: Petr Vorel @ 2019-09-19 16:41 UTC (permalink / raw) To: ltp Hi Christian, thank you for working on it. TL;DR: looks good to me, I have some code style objections, (use echo instead of printf, use $foo instead of ${foo} when possible) + some minor suggestions below. > +++ b/doc/write-tests-for-shell-lib.txt > @@ -0,0 +1,59 @@ > +How to format tests in order to test the shell library > +====================================================== > + > +It is important to test the Linux kernel functionality but also to make sure > +that LTP is running correctly itself. For this reason it is useful to test > +intrinsic functionality of LTP. > + > +1. Running tests for the shell library > +-------------------------------------- > +The test cases reside in the folder `lib/newlib_tests/shell`. A script executing > +them one by one is located in the folder `lib/newlib_tests`. You can execute > +this script to test all cases or specify test cases to be run. The script is > +called `test_sh_newlib.sh`. > + > +2. Writing tests for the shell library > +-------------------------------------- > +The tests are written like all other test cases using the shell library. > +Additionally, at the end of the file the desired output is added. As an example: Nice. I wonder if it should be in doc/test-writing-guidelines.txt. But this section is already too big, so it's probably good that it's separate. Then we need to add page this to wiki (simple). BTW I plan to introduce make check, which will run this and other checks as well. > + > +[source,shell] > +------------------------------------------------------------------------------- > +#!/bin/sh > +# > +# This is a basic test for true shell buildin typo: builtin > +# nit: I'd remove this empty lines just with '#' > + > +TST_TESTFUNC=do_test > +. tst_test.sh > + > +do_test() > +{ > + true > + ret=$? > + > + tst_res TINFO "Test $1 passed with no data ('$2')" > + > + if [ $ret -eq 0 ]; then > + tst_res TPASS "true returned 0" > + else > + tst_res TFAIL "true returned $ret" > + fi > +} > + > +tst_run > +# output: > +# test 1 TINFO: Test 1 passed with no data ('') > +# test 1 TPASS: true returned 0 > +# > +# Summary: > +# passed 1 > +# failed 0 > +# skipped 0 > +# warnings 0 > +------------------------------------------------------------------------------- > + > +The most noticeable thing is to add the line `# output:` to show the parser that > +parsing should start in the following line. For the following lines the `# ` > +will be stripped before the output is then compared with the actual output that > +gets printed on the terminal when running the test. > diff --git a/lib/newlib_tests/shell/test_sh_newlib.sh b/lib/newlib_tests/shell/test_sh_newlib.sh > new file mode 100755 > index 000000000..4aa19555b > --- /dev/null > +++ b/lib/newlib_tests/shell/test_sh_newlib.sh > @@ -0,0 +1,102 @@ > +#!/bin/sh > +# > +# SPDX-License-Identifier: GPL-2.0-or-later > +# (c) 2019 SUSE LLC > +# > +# Author: Christian Lanig <clanig@suse.com> > + > +PATH="${PATH}:$(dirname $(readlink -f $0))/../../../testcases/lib/" > +if [ -z "$TMPDIR" ]; then > + export TMPDIR="/tmp" > +fi > +color_blue='\033[1;34m' > +color_green='\033[1;32m' > +color_red='\033[1;31m' > +reset_attr='\033[0m' I'd prefer not reimplementing tst_ansi_color.sh. IMHO it'd be better either use part of it (or tst_ansi_color.sh) or not use colors at all. > +tmp="${TMPDIR}/sh_lib_tst-${$}/" nit: it can be $$ (instead of ${$}) > +mkdir $tmp || cleanup 1 > +parent_dir=$(dirname $(readlink -f $0))/ > +tooldir=${parent_dir}/../../../tools/ > +testdir=${parent_dir}testcases/ > +tst_files=$(ls $testdir) > + > +cleanup() > +{ > + [ -d "$tmp" ] && rm -rf "$tmp" nit: 1 could be a default parameter. > + exit $1 > +} > + > +print_help() > +{ > + cat <<EOF > + > +???????????????????????????????????????????????????????????????????????????????? > +? This Shell script iterates over test cases for the new Shell library and ? > +? verifies the output. ? > +???????????????????????????????????????????????????????????????????????????????? nit: I'd prefer to use ASCII (-) than unicode (?). > + > + Usage: > + $(basename $0) [TEST_FILE_1] [TEST_FILE_2] > + nit: another space not needed. > +EOF > + exit 0 > +} > + > +parse_params() > +{ > + [ -n "$1" ] && tst_files= > + while [ -n "$1" ]; do We usually prefer to use getopts, which does not allow long opts. It's ok for this small usage, but for more complicated output it's better not reimplementing it. > + case "$1" in > + --help) print_help;; > + -h) print_help;; > + -*) > + printf "Unknown positional parameter ${1}.\n" maybe use simple this simpler form: echo "Unknown positional parameter $1" > + cleanup 1;; > + *) tst_files="$tst_files $1";; > + esac > + shift > + done > +} > + > +verify_output() > +{ > + if [ ! -e "${testdir}$tst" ]; then This can safely be "$testdir$tst" > + printf "$tst not found\n" Again, use echo > + cleanup 1 > + fi > + > + ${tooldir}lookup_split_cut.py -f ${testdir}$tst -d $tmp \ > + -s '# output:\n' -c '# {0,1}' || cleanup 1 > + > + "${testdir}$tst" > "${tmp}$tst.actual" || cleanup 1 Again, ${} is not necessary. > + cmp -s "${tmp}$tst.actual" "${tmp}${tst}_out/out.1" && return 0 + check for cmp. > + return 1 > +} > + > +run_tests() > +{ > + pass_cnt=0 > + fail_cnt=0 > + printf "\n" again, echo is better here. > + for tst in $tst_files; do > + if verify_output; then > + pass_cnt=$(($pass_cnt + 1)) > + printf "${color_green}TPASS$reset_attr ${tst}\n" > + else > + fail_cnt=$(($fail_cnt + 1)) > + printf "${color_red}TFAIL$reset_attr ${tst}\n" > + printf "${color_blue}Diff:${reset_attr}\n" > + diff -u "${tmp}${tst}.actual" \ We might want to check for diff being available before we use. > + "${tmp}${tst}_out/out.1" > + printf "\n" > + fi > + done > + printf "\nSummary:\n" > + printf "${color_red}Failed:$reset_attr $fail_cnt\n" > + printf "${color_green}Passed:$reset_attr $pass_cnt\n\n" > + return $fail_cnt > +} ... (more tests) > diff --git a/tools/lookup_split_cut.py b/tools/lookup_split_cut.py > new file mode 100755 > index 000000000..2b3388ada > --- /dev/null > +++ b/tools/lookup_split_cut.py > @@ -0,0 +1,120 @@ > +#!/usr/bin/env python I guess this should be python3. I'd be a bit careful to bring python as another dependency, (there was some awk solution for this, proposed by Cyril), but as python is everywhere, it shouldn't be a problem. (We definitely don't want python on SUT, these tests will be eventually rewritten into C or shell.) > +# > +# SPDX-License-Identifier: GPL-2.0-or-later > +# (c) 2019 SUSE LLC > +# > +# Author: Christian Lanig <clanig@suse.com> > + > +import sys > +import os > +import re > +from sys import argv > +from os import makedirs, path > + > +src_file_path = None > +dest = None > +delim = None > +pattern = None > +perim = None > + > +argv.reverse() > +basename = path.split(argv.pop())[1] > + > +def print_help(): > + > + help = """ > + This script can look up a passage in a specified text pattern, split a > + text file and cut a pattern. The operation chain is: > + > + lookup > split > cut > + > + The output files are written to the specified destination directory. > + > + Usage: > + """ > + help += "\n\t\t" + basename + " -f [PATH] -d [PATH] -l " \ > + + "[PERIMETER] -s [DELIMITER] \n" \ > + + "\t\t\t\t -c [PATTERN]\n\n" nit: you can use format() to use variables in multiline strings. > + help += """ > + -h, --help > + print this help > + -f, --file > + source file to be processed > + -d, --destination > + destination path > + -l, --lookup > + look for data in text passage > + -s, --split > + split file by delimiter > + -c, --cut > + cut pattern from file > + """ > + > + print(help) > + sys.exit(0) > + > +def get_next_arg(): > + if argv: > + return argv.pop() > + else: > + print("Missing parameter. Run with \"--help\" for information.") > + sys.exit(1) > + > +while argv: > + arg = argv.pop() > + if arg in ["-h", "--help"]: > + print_help() > + elif arg in ["-f", "--file"]: > + src_file_path = get_next_arg() > + elif arg in ["-d", "--destination"]: > + dest = get_next_arg() > + elif arg in ["-l", "--lookup"]: > + perim = get_next_arg() > + elif arg in ["-s", "--split"]: > + delim = get_next_arg() > + elif arg in ["-c", "--cut"]: > + pattern = get_next_arg() > + else: > + print("Illegal argument. Run with \"--help\" for information.") I'd print help here. > + sys.exit(1) > + > +if not src_file_path: > + print("Input file has to be specified.") > + sys.exit(1) > + > +if not delim and not pattern and not perim: > + print("Missing parameters. Run with \"--help\" for information.") > + sys.exit(1) > + > +src_file = open(src_file_path, "r") > +src = src_file.read() > +src_file.close() > + > +capture = 0 > +if perim: > + try: > + capture = re.search(perim, src).group(1) > + except: > + pass > + > +if delim: > + src_file_name = path.split(src_file_path)[1] > + out_dir = dest + "/" + src_file_name + "_out" > + > + if not path.exists(out_dir): > + makedirs(out_dir) > + > + src = re.split(delim, src) > +else: > + src = [src] > + > +if pattern: > + for i in range(len(src)): > + src[i] = re.sub(pattern, "", src[i]) > +if delim or pattern: > + for i in range(len(src)): > + out_file = open(out_dir + "/out." + str(i), "w") > + out_file.write(src[i]) > + out_file.close() > + > +sys.exit(capture) Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 1/1] Add automated tests for shell lib 2019-09-19 16:41 ` Petr Vorel @ 2019-09-30 18:27 ` Christian Lanig 0 siblings, 0 replies; 23+ messages in thread From: Christian Lanig @ 2019-09-30 18:27 UTC (permalink / raw) To: ltp Hi Petr and Clemens, thank you for the reasonable comments and suggestions. I just want to tell you that I will not be able to work on this until next week. I appreciate your patience very much. Regards, Christian -- Christian Lanig - SUSE Engineering Apprentice SUSE Linux GmbH, Maxfeldstra?e 5, D-90409 N?rnberg Tel: +49-911-74053-0; Fax: +49-911-7417755; https://www.suse.com/ GF: Felix Imend?rffer, Jane Smithard, Graham Norton, HRB 21284 (AG N?rnberg) ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 1/1] Add automated tests for shell lib 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 1/1] " Christian Lanig 2019-09-19 16:41 ` Petr Vorel @ 2019-09-20 14:21 ` Clemens Famulla-Conrad 1 sibling, 0 replies; 23+ messages in thread From: Clemens Famulla-Conrad @ 2019-09-20 14:21 UTC (permalink / raw) To: ltp Hi Christian, This tests are nice! Lets keep going with it. On Thu, 2019-08-22 at 21:12 +0200, Christian Lanig wrote: <snip> > diff --git a/lib/newlib_tests/shell/test_sh_newlib.sh > b/lib/newlib_tests/shell/test_sh_newlib.sh > new file mode 100755 > index 000000000..4aa19555b > --- /dev/null > +++ b/lib/newlib_tests/shell/test_sh_newlib.sh > @@ -0,0 +1,102 @@ > +#!/bin/sh > +# > +# SPDX-License-Identifier: GPL-2.0-or-later > +# (c) 2019 SUSE LLC > +# > +# Author: Christian Lanig <clanig@suse.com> > + > +PATH="${PATH}:$(dirname $(readlink -f $0))/../../../testcases/lib/" > +if [ -z "$TMPDIR" ]; then > + export TMPDIR="/tmp" > +fi > +color_blue='\033[1;34m' > +color_green='\033[1;32m' > +color_red='\033[1;31m' > +reset_attr='\033[0m' > +tmp="${TMPDIR}/sh_lib_tst-${$}/" > +mkdir $tmp || cleanup 1 > +parent_dir=$(dirname $(readlink -f $0))/ > +tooldir=${parent_dir}/../../../tools/ > +testdir=${parent_dir}testcases/ > +tst_files=$(ls $testdir) > + > +cleanup() You use cleanup as a default handler for error handing. For instance, if a test doesn't have a `# output:` section we silently quit with exitcode 1. I would like to be informed more about such errors. Maybe we could just printout $tst if `$! != 0`. > +{ > + [ -d "$tmp" ] && rm -rf "$tmp" > + exit $1 > +} > + > +print_help() > +{ > + cat <<EOF > + > +???????????????????????????????????????????????????????????????????? > ???????????? > +? This Shell script iterates over test cases for the new Shell > library and ? > +? verifies the > output. ? > +???????????????????????????????????????????????????????????????????? > ???????????? > + > + Usage: > + $(basename $0) [TEST_FILE_1] [TEST_FILE_2] > + > +EOF > + exit 0 > +} > + > +parse_params() > +{ > + [ -n "$1" ] && tst_files= > + while [ -n "$1" ]; do > + case "$1" in > + --help) print_help;; > + -h) print_help;; > + -*) > + printf "Unknown positional parameter > ${1}.\n" > + cleanup 1;; > + *) tst_files="$tst_files $1";; > + esac > + shift > + done > +} > + > +verify_output() > +{ > + if [ ! -e "${testdir}$tst" ]; then > + printf "$tst not found\n" > + cleanup 1 > + fi > + > + ${tooldir}lookup_split_cut.py -f ${testdir}$tst -d $tmp \ > + -s '# output:\n' -c '# {0,1}' || cleanup 1 just an idea, in perl ( I'm not sure if we have perl already as dependency). My feeling is, that lookup_split_cut.py is to much for that task. cat $testdir$tst | perl -e '$o = 0; while (<STDIN>) {print substr($_, 2) if $o; $o = 1 if /^# output:/; }' > $tmp${tst}expected_output > + > + "${testdir}$tst" > "${tmp}$tst.actual" || cleanup 1 We should keep going on failed test. We need this to test timeout functionally or error handling... > + cmp -s "${tmp}$tst.actual" "${tmp}${tst}_out/out.1" && > return 0 > + return 1 > +} > + > +run_tests() > +{ > + pass_cnt=0 > + fail_cnt=0 > + printf "\n" > + for tst in $tst_files; do > + if verify_output; then > + pass_cnt=$(($pass_cnt + 1)) > + printf "${color_green}TPASS$reset_attr > ${tst}\n" > + else > + fail_cnt=$(($fail_cnt + 1)) > + printf "${color_red}TFAIL$reset_attr > ${tst}\n" > + printf "${color_blue}Diff:${reset_attr}\n" > + diff -u "${tmp}${tst}.actual" \ > + "${tmp}${tst}_out/out.1" > + printf "\n" > + fi > + done > + printf "\nSummary:\n" > + printf "${color_red}Failed:$reset_attr $fail_cnt\n" > + printf "${color_green}Passed:$reset_attr $pass_cnt\n\n" > + return $fail_cnt > +} > + > +parse_params "$@" > +run_tests > +cleanup $? > <snip> Thanks Clemens ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v2 0/1] Add automated tests for shell lib 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 0/1] " Christian Lanig 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 1/1] " Christian Lanig @ 2019-09-19 14:26 ` Petr Vorel 1 sibling, 0 replies; 23+ messages in thread From: Petr Vorel @ 2019-09-19 14:26 UTC (permalink / raw) To: ltp Hi Christian, > Hi, > I am sorry that my reply took so long. Thank you very much for your comments > which helped me to improve the code. > I have made an overhaul of the code based on your suggestions but an option to > pass parameters to the tests is still not implemented. > I'm sending you what I have so far because it would help me when you could > provide me more information regarding this. > I agree that it is difficult to find a way to process text the way needed here > without disregarding the Linux Kernel Coding Style that asks for code which is > transparent and easy to read when we only consider Shell commands. To address > this issue I have drafted a Python script that might go in the tools folder and > may perhaps be helpful in other cases as well. Since there are already Python > files in the project it might be a valid option. > Thank you very much for your patience. No problem, thanks continue for it. I'm also not sure if we should put them into separate testcases/ directory. I'd put them into shell/ directory and rename test_sh_newlib.sh to run.sh or something simple. BTW (nit, no hard feeling :) as lib/newlib_tests/test.*.sh are based on my previous work, you could might mention it in the change log. You updated them to have timeout output, so it's your work, (if not, it'd be better to send them as mine separate Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [PATCH 1/2] Make shell lib tests standalone 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig 2018-08-28 11:18 ` [LTP] [PATCH 2/2] Add wanted output to shell lib test case Christian Lanig 2018-08-29 17:24 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Petr Vorel @ 2018-08-31 11:46 ` Cyril Hrubis 2 siblings, 0 replies; 23+ messages in thread From: Cyril Hrubis @ 2018-08-31 11:46 UTC (permalink / raw) To: ltp Hi! > rename lib/newlib_tests/{ => shell}/test.TST_TEST.TST_CNT.separate.sh (95%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST.TST_CNT.sh (95%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST.getopts.sh (96%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST.sh (95%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.TST_CNT.separate.sh (96%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.TST_CNT.sh (96%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.getopts.sh (96%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA.sh (95%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA_IFS.getopts.sh (97%) > rename lib/newlib_tests/{ => shell}/test.TST_TEST_DATA_IFS.sh (96%) Just small comment, I do wonder if top level lib/ directory is correct for the shell tests since the shell library is located at testcases/lib/ (for a historical reasons). Maybe we should just create a 'test' directory under 'testcases/lib/' instead. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS 2018-05-22 19:34 [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Petr Vorel 2018-05-22 19:34 ` [LTP] [RFC PATCH v3 2/2] lib: Add tests Petr Vorel @ 2018-05-24 13:41 ` Cyril Hrubis 2018-05-24 13:53 ` Petr Vorel 1 sibling, 1 reply; 23+ messages in thread From: Cyril Hrubis @ 2018-05-24 13:41 UTC (permalink / raw) To: ltp Hi! > Signed-off-by: Petr Vorel <pvorel@suse.cz> > --- > Changes v2->v3: > * Don't pass $TST_TEST_DATA as whole argument for $TST_CNT (Cyril) > * Create tst_run_tests() for cleanup (Cyril). > * Create tst_run_test() - more cleanup > > Maybe we should try to "hide" somehow these API functions (underscore at > the front?). Actually this is a good idea, we can follow the python underscore notation that uses it for private variables and functions. Hence the counters will become _tst_i, etc. But please do that in a separate patch. > --- > doc/test-writing-guidelines.txt | 74 +++++++++++++++++++++++++++++++++++++---- > testcases/lib/tst_test.sh | 61 +++++++++++++++++++++------------ > 2 files changed, 106 insertions(+), 29 deletions(-) > > diff --git a/doc/test-writing-guidelines.txt b/doc/test-writing-guidelines.txt > index de47443eb..531b828ff 100644 > --- a/doc/test-writing-guidelines.txt > +++ b/doc/test-writing-guidelines.txt > @@ -1442,12 +1442,12 @@ TST_CNT=2 > > test1() > { > - tst_res TPASS "Test 1 passed" > + tst_res TPASS "Test $1 passed" > } > > test2() > { > - tst_res TPASS "Test 2 passed" > + tst_res TPASS "Test $1 passed" > } > > tst_run > @@ -1455,7 +1455,8 @@ tst_run > > If '$TST_CNT' is set, the test library looks if there are functions named > '$\{TST_TESTFUNC\}1', ..., '$\{TST_TESTFUNC\}$\{TST_CNT\}' and if these are > -found they are executed one by one. > +found they are executed one by one. The test number is passed to it in the '$1'. > + > > [source,sh] > ------------------------------------------------------------------------------- > @@ -1471,8 +1472,8 @@ TST_CNT=2 > do_test() > { > case $1 in > - 1) tst_res TPASS "Test 1 passed";; > - 2) tst_res TPASS "Test 2 passed";; > + 1) tst_res TPASS "Test $1 passed";; > + 2) tst_res TPASS "Test $1 passed";; > esac > } > > @@ -1483,6 +1484,65 @@ Otherwise, if '$TST_CNT' is set but there is no '$\{TST_TESTFUNC\}1', etc., > the '$TST_TESTFUNC' is executed '$TST_CNT' times and the test number is passed > to it in the '$1'. > > +[source,sh] > +------------------------------------------------------------------------------- > +#!/bin/sh > +# > +# Example test with tests in a single function, using $TST_TEST_DATA and > +# $TST_TEST_DATA_IFS > +# > + > +TST_TESTFUNC=do_test > +TST_TEST_DATA="foo:bar:d dd" > +TST_TEST_DATA_IFS=":" > +. tst_test.sh > + > +do_test() > +{ > + tst_res TPASS "Test $1 passed with data '$2'" > +} > + > +tst_run > +# output: > +# test 1 TPASS: Test 1 passed with data 'foo' > +# test 2 TPASS: Test 2 passed with data 'bar' > +# test 3 TPASS: Test 3 passed with data 'd dd' > + > +------------------------------------------------------------------------------- > + > +It's possible to pass data for function with '$TST_TEST_DATA'. Optional > +'$TST_TEST_DATA_IFS' is used for splitting, default value is space. > + > +[source,sh] > +------------------------------------------------------------------------------- > +#!/bin/sh > +# > +# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT > +# > + > +TST_TESTFUNC=do_test > +TST_CNT=2 > +TST_TEST_DATA="foo:bar:d dd" > +. tst_test.sh > + > +do_test() > +{ > + case $1 in > + 1) tst_res TPASS "Test $1 passed with data '$2'";; > + 2) tst_res TPASS "Test $1 passed with data '$2'";; > + esac > +} > + > +tst_run > +# output: > +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' > +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' > + > +------------------------------------------------------------------------------- > +When '$TST_TEST_DATA' is used with '$TST_CNT', it's passed as whole string in > +'$2' ($1 is for the test number), '$TST_TEST_DATA_IFS' is ignored. Similar > +would be when using these variables with separate functions. Now that we support both TST_CNT and TST_TEST_DATA_IFS can we change this example to include the TST_TEST_DATA_IFS=":" as well? > 2.3.2 Library variables > ^^^^^^^^^^^^^^^^^^^^^^^ > > @@ -1587,8 +1647,8 @@ these can be listed with passing help '-h' option to any test. > The function that prints the usage is passed in '$TST_USAGE', the help for > the options implemented in the library is appended when usage is printed. > > -Lastly the fucntion '$PARSE_ARGS' is called with the option name in '$1' and, > -if option has argument, its value in '$2'. > +Lastly the fucntion '$PARSE_ARGS' is called with the option name in the '$1' > +and, if option has argument, its value in the '$2'. > > [source,sh] > ------------------------------------------------------------------------------- > diff --git a/testcases/lib/tst_test.sh b/testcases/lib/tst_test.sh > index 464c4c41e..7680aa462 100644 > --- a/testcases/lib/tst_test.sh > +++ b/testcases/lib/tst_test.sh > @@ -246,7 +246,7 @@ tst_rescmp() > > tst_run() > { > - local tst_i > + local tst_i tst_data > > if [ -n "$TST_TEST_PATH" ]; then > for tst_i in $(grep TST_ "$TST_TEST_PATH" | sed 's/.*TST_//; s/[="} \t\/:`].*//'); do > @@ -255,7 +255,7 @@ tst_run() > OPTS|USAGE|PARSE_ARGS|POS_ARGS);; > NEEDS_ROOT|NEEDS_TMPDIR|NEEDS_DEVICE|DEVICE);; > NEEDS_CMDS|NEEDS_MODULE|MODPATH|DATAROOT);; > - IPV6);; > + IPV6|TEST_DATA|TEST_DATA_IFS);; > *) tst_res TWARN "Reserved variable TST_$tst_i used!";; > esac > done > @@ -348,27 +348,17 @@ tst_run() > > #TODO check that test reports some results for each test function call > while [ $TST_ITERATIONS -gt 0 ]; do > - if [ -n "$TST_CNT" ]; then > - if type test1 > /dev/null 2>&1; then > - for tst_i in $(seq $TST_CNT); do > - local res=$(tst_resstr) > - $TST_TESTFUNC$tst_i > - tst_rescmp "$res" > - TST_COUNT=$((TST_COUNT+1)) > - done > - else > - for tst_i in $(seq $TST_CNT); do > - local res=$(tst_resstr) > - $TST_TESTFUNC $tst_i > - tst_rescmp "$res" > - TST_COUNT=$((TST_COUNT+1)) > - done > - fi > + if [ -n "$TST_TEST_DATA" ]; then > + tst_i=1 > + tst_check_cmds cut > + while true; do > + tst_data="$(echo "$TST_TEST_DATA" | cut -d"$TST_TEST_DATA_IFS" -f$tst_i)" > + [ -z "$tst_data" ] && break > + tst_run_tests "$tst_data" > + tst_i=$((tst_i+1)) > + done > else > - local res=$(tst_resstr) > - $TST_TESTFUNC > - tst_rescmp "$res" > - TST_COUNT=$((TST_COUNT+1)) > + tst_run_tests > fi > TST_ITERATIONS=$((TST_ITERATIONS-1)) > done > @@ -376,6 +366,31 @@ tst_run() > tst_do_exit > } > > +tst_run_tests() > +{ > + local tst_data="$1" > + local tst_i > + > + for tst_i in $(seq ${TST_CNT:-1}); do > + if type test1 > /dev/null 2>&1; then I know that this is not your fault but this should be probably: if type ${TST_TESTFUNC}1 > /dev/null 2>&1; ... > + tst_run_test "$TST_TESTFUNC$tst_i" $tst_i "$tst_data" > + else > + tst_run_test "$TST_TESTFUNC" $tst_i "$tst_data" > + fi > + done > +} > + > +tst_run_test() > +{ > + local res=$(tst_resstr) Can we please prefix the res here with tst_ as well since we are touching the code? > + local tst_fnc="$1" > + shift > + > + $tst_fnc "$@" > + tst_rescmp "$res" > + TST_COUNT=$((TST_COUNT+1)) > +} > + > if [ -z "$TST_ID" ]; then > filename=$(basename $0) > TST_ID=${filename%%.*} > @@ -400,6 +415,8 @@ if [ -z "$TST_NO_DEFAULT_RUN" ]; then > tst_brk TBROK "TST_TESTFUNC is not defined" > fi > > + TST_TEST_DATA_IFS="${TST_TEST_DATA_IFS:- }" > + > if [ -n "$TST_CNT" ]; then > if ! tst_is_int "$TST_CNT"; then > tst_brk TBROK "TST_CNT must be integer" > -- > 2.16.3 > -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS 2018-05-24 13:41 ` [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Cyril Hrubis @ 2018-05-24 13:53 ` Petr Vorel 2018-05-24 14:00 ` Cyril Hrubis 0 siblings, 1 reply; 23+ messages in thread From: Petr Vorel @ 2018-05-24 13:53 UTC (permalink / raw) To: ltp Hi Cyril, Thanks for your comments > > Signed-off-by: Petr Vorel <pvorel@suse.cz> > > --- > > Changes v2->v3: > > * Don't pass $TST_TEST_DATA as whole argument for $TST_CNT (Cyril) > > * Create tst_run_tests() for cleanup (Cyril). > > * Create tst_run_test() - more cleanup > > Maybe we should try to "hide" somehow these API functions (underscore at > > the front?). > Actually this is a good idea, we can follow the python underscore > notation that uses it for private variables and functions. Hence the > counters will become _tst_i, etc. > But please do that in a separate patch. I meant to add underscore before new functions i.e. _tst_run_tests() and _tst_run_test(). Variables: do you mean to change local variables in tst_run()? That would be really for separate patch. > > --- > > doc/test-writing-guidelines.txt | 74 +++++++++++++++++++++++++++++++++++++---- > > testcases/lib/tst_test.sh | 61 +++++++++++++++++++++------------ > > 2 files changed, 106 insertions(+), 29 deletions(-) ... > > +TST_TESTFUNC=do_test > > +TST_CNT=2 > > +TST_TEST_DATA="foo:bar:d dd" > > +. tst_test.sh > > + > > +do_test() > > +{ > > + case $1 in > > + 1) tst_res TPASS "Test $1 passed with data '$2'";; > > + 2) tst_res TPASS "Test $1 passed with data '$2'";; > > + esac > > +} > > + > > +tst_run > > +# output: > > +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' > > +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' > > + > > +------------------------------------------------------------------------------- > > +When '$TST_TEST_DATA' is used with '$TST_CNT', it's passed as whole string in > > +'$2' ($1 is for the test number), '$TST_TEST_DATA_IFS' is ignored. Similar > > +would be when using these variables with separate functions. > Now that we support both TST_CNT and TST_TEST_DATA_IFS can we change > this example to include the TST_TEST_DATA_IFS=":" as well? I wanted to demonstrate that TST_TEST_DATA_IFS has a default value ' '). But ok, I'll change it to define TST_TEST_DATA_IFS as well. ... > > + for tst_i in $(seq ${TST_CNT:-1}); do > > + if type test1 > /dev/null 2>&1; then > I know that this is not your fault but this should be probably: > if type ${TST_TESTFUNC}1 > /dev/null 2>&1; ... Thanks, I'll fix it, in a separate commit. > > +tst_run_test() > > +{ > > + local res=$(tst_resstr) > Can we please prefix the res here with tst_ as well since we are > touching the code? Sure. To be honest I wasn't sure whether there is some reason for $res being named without prefix or not. Kind regards, Petr ^ permalink raw reply [flat|nested] 23+ messages in thread
* [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS 2018-05-24 13:53 ` Petr Vorel @ 2018-05-24 14:00 ` Cyril Hrubis 0 siblings, 0 replies; 23+ messages in thread From: Cyril Hrubis @ 2018-05-24 14:00 UTC (permalink / raw) To: ltp Hi! > > Actually this is a good idea, we can follow the python underscore > > notation that uses it for private variables and functions. Hence the > > counters will become _tst_i, etc. > > > But please do that in a separate patch. > I meant to add underscore before new functions i.e. _tst_run_tests() and _tst_run_test(). > Variables: do you mean to change local variables in tst_run()? > That would be really for separate patch. As I written, change everything private that leaks to the test context with _tst_ so that it's clear what the test is supposed to set/call. > > > --- > > > doc/test-writing-guidelines.txt | 74 +++++++++++++++++++++++++++++++++++++---- > > > testcases/lib/tst_test.sh | 61 +++++++++++++++++++++------------ > > > 2 files changed, 106 insertions(+), 29 deletions(-) > > ... > > > +TST_TESTFUNC=do_test > > > +TST_CNT=2 > > > +TST_TEST_DATA="foo:bar:d dd" > > > +. tst_test.sh > > > + > > > +do_test() > > > +{ > > > + case $1 in > > > + 1) tst_res TPASS "Test $1 passed with data '$2'";; > > > + 2) tst_res TPASS "Test $1 passed with data '$2'";; > > > + esac > > > +} > > > + > > > +tst_run > > > +# output: > > > +# test 1 TPASS: Test 1 passed with data 'foo:bar:d' > > > +# test 2 TPASS: Test 2 passed with data 'foo:bar:d' > > > + > > > +------------------------------------------------------------------------------- > > > +When '$TST_TEST_DATA' is used with '$TST_CNT', it's passed as whole string in > > > +'$2' ($1 is for the test number), '$TST_TEST_DATA_IFS' is ignored. Similar > > > +would be when using these variables with separate functions. > > > Now that we support both TST_CNT and TST_TEST_DATA_IFS can we change > > this example to include the TST_TEST_DATA_IFS=":" as well? > I wanted to demonstrate that TST_TEST_DATA_IFS has a default value ' '). > But ok, I'll change it to define TST_TEST_DATA_IFS as well. No problem with that, but then we should get rid of the : from the example data and keep it only at "foo bar". > > Can we please prefix the res here with tst_ as well since we are > > touching the code? > Sure. To be honest I wasn't sure whether there is some reason for $res being named without > prefix or not. I guess that it haven't been caught in the review process when I first wrote the library... -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2019-09-30 18:27 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-05-22 19:34 [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Petr Vorel 2018-05-22 19:34 ` [LTP] [RFC PATCH v3 2/2] lib: Add tests Petr Vorel 2018-05-24 13:46 ` Cyril Hrubis 2018-05-24 14:00 ` Petr Vorel 2018-08-28 11:18 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Christian Lanig 2018-08-28 11:18 ` [LTP] [PATCH 2/2] Add wanted output to shell lib test case Christian Lanig 2018-08-29 17:24 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Petr Vorel 2018-08-29 17:30 ` Petr Vorel 2018-08-31 15:24 ` [LTP] [RFC PATCH 0/1] Add automated tests for shell lib Christian Lanig 2018-08-31 15:24 ` [LTP] [RFC PATCH 1/1] " Christian Lanig 2018-10-03 9:51 ` Cyril Hrubis 2018-10-03 10:46 ` Petr Vorel 2018-10-03 11:32 ` Petr Vorel 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 0/1] " Christian Lanig 2019-08-22 19:12 ` [LTP] [RFC PATCH v2 1/1] " Christian Lanig 2019-09-19 16:41 ` Petr Vorel 2019-09-30 18:27 ` Christian Lanig 2019-09-20 14:21 ` Clemens Famulla-Conrad 2019-09-19 14:26 ` [LTP] [RFC PATCH v2 0/1] " Petr Vorel 2018-08-31 11:46 ` [LTP] [PATCH 1/2] Make shell lib tests standalone Cyril Hrubis 2018-05-24 13:41 ` [LTP] [RFC PATCH v3 1/2] tst_test.sh: Add TST_TEST_DATA and TST_TEST_DATA_IFS Cyril Hrubis 2018-05-24 13:53 ` Petr Vorel 2018-05-24 14:00 ` Cyril Hrubis
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.