* [Fuego] [fuego-core 0/7] upstreaming work
@ 2021-07-18 11:10 venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
` (7 more replies)
0 siblings, 8 replies; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: binh1.tranhai, fuego, huong4.nguyenthi
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
Hi Tim,
Below are some patches that we want to upstream our work from long time,
the below patches fixes some issues in ftc and LTP functional test
Daniel Sangorrin (1):
ftc: docker ps will not work on the local board
Nguyen Dat Tho (2):
LTP: execute all tests when spec specifies "all"
LTP: automatically obtain the list of tests
Tran Hai Binh (1):
LTP: Add the new tests and skip non-installed tests
nguyen thi huong (2):
ftc: fix test process is not killed when fuego test times out
LTP: fix fuego test could not skip list of test cases
yoshida toshiko (1):
testplan_smoketest.json: ftc fails to add job for tesplan_smoketest
overlays/testplans/testplan_smoketest.json | 2 +-
scripts/ftc | 19 +++--
tests/Functional.LTP/fuego_test.sh | 91 +++++++---------------
tests/Functional.LTP/ltp_target_run.sh | 23 +++---
4 files changed, 56 insertions(+), 79 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 18:28 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla
` (6 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: fuego
From: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
When running fuego directly on the local board, instead of
using a host-target approach, the check for a fuego
container caused an error.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
scripts/ftc | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/scripts/ftc b/scripts/ftc
index 4430dd2..2a6a640 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -5153,11 +5153,15 @@ def get_running_fuego_container_name():
return cached_container_name
# return the first container with "fuego" in the image or container name
- dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
- for line in dps_lines:
- if "fuego" in line:
- cached_container_name = line.strip().split(" ")[-1]
- break
+ try:
+ with open(os.devnull, 'w') as devnull:
+ dps_lines = subprocess.check_output("sudo docker ps", shell=True, stderr=devnull).split('\n')
+ for line in dps_lines:
+ if "fuego" in line:
+ cached_container_name = line.strip().split(" ")[-1]
+ break
+ except subprocess.CalledProcessError:
+ return None
return cached_container_name
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 18:34 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla
` (5 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: nguyen thi huong, fuego
From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
- Currently, fuego kill main process and does not kill test process when test times out
In that case, test process still runs but fuego test result is not available
- Revise implementation to kill all test processes and printout the result
Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
scripts/ftc | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/scripts/ftc b/scripts/ftc
index 2a6a640..5c300d0 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout):
dprint("ftc_exec_command: command=%s" % command)
- p = subprocess.Popen(command.split(), stdout=log, stderr=log)
+ p = subprocess.Popen(command.split(), stdout=log, stderr=log, preexec_fn=os.setpgrp)
# specify timeout for command operation
signal.signal(signal.SIGALRM, alarm_handler)
@@ -3960,7 +3960,8 @@ def ftc_exec_command(command, timeout):
# p.kill()
# abort with prejudice...
- p.kill()
+ pgrp = os.getpgid(p.pid)
+ os.killpg(pgrp, signal.SIGALRM)
timed_out = True
finally:
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 18:49 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla
` (4 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: fuego
From: yoshida toshiko <toshiko.yoshida@toshiba.co.jp>
add missing comma after Benchmark.dbench4
e.g: ftc add-jobs -b local -p testplan_smoketest
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
overlays/testplans/testplan_smoketest.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/overlays/testplans/testplan_smoketest.json b/overlays/testplans/testplan_smoketest.json
index 8c856c1..4bbbe4f 100644
--- a/overlays/testplans/testplan_smoketest.json
+++ b/overlays/testplans/testplan_smoketest.json
@@ -12,7 +12,7 @@
"testName": "Benchmark.Dhrystone"
},
{
- "testName": "Benchmark.dbench4"
+ "testName": "Benchmark.dbench4",
"timeout": "40m"
},
{
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
` (2 preceding siblings ...)
2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 18:54 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla
` (3 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: Tran Hai Binh, fuego
From: Tran Hai Binh <binh1.tranhai@toshiba.co.jp>
- Add the new tests in 20200930 that are not in ALLTESTS:
s390x_tests, uevent, crypto
- Skip non-installed tests in LTP
Signed-off-by: Tran Hai Binh <binh1.tranhai@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 6 +++---
tests/Functional.LTP/ltp_target_run.sh | 23 ++++++++++++++---------
2 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index a4f551b..6a84d1c 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -18,9 +18,9 @@ cve io_floppy net.ipv6_lib net.tirpc_tests
dio ipc net.multicast network_commands timers
dma_thread_diotest kernel_misc net.nfs nptl tpm_tools
fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing
-filecaps ltp-aiodio.part2 net.rpc_tests pipes
-fs ltp-aiodio.part3 net.sctp power_management_tests
-fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive
+filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests
+fs ltp-aiodio.part3 net.sctp power_management_tests uevent
+fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto
fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty
smoketest"
diff --git a/tests/Functional.LTP/ltp_target_run.sh b/tests/Functional.LTP/ltp_target_run.sh
index 52a006a..ded81de 100755
--- a/tests/Functional.LTP/ltp_target_run.sh
+++ b/tests/Functional.LTP/ltp_target_run.sh
@@ -5,6 +5,7 @@
OUTPUT_DIR=${PWD}/result
TMP_DIR=${PWD}/tmp
+RUNTEST_DIR=${PWD}/runtest
[ -d ${TMP_DIR} ] && rm -rf ${TMP_DIR}
mkdir -p ${TMP_DIR}
@@ -18,15 +19,19 @@ echo "ltp_target_run: ${TESTS} | ${PTSTESTS} | ${RTTESTS}"
# FIXTHIS: add -t option for limiting the duration of each test group execution
for i in ${TESTS}; do
- echo "ltp_target_run: doing test $i"
- mkdir -p ${OUTPUT_DIR}/${i}
- ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \
- -l ${OUTPUT_DIR}/${i}/result.log \
- -o ${OUTPUT_DIR}/${i}/output.log \
- -d ${TMP_DIR} \
- -S ./skiplist.txt \
- -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
- rm -rf ${TMP_DIR}/*
+ if [ -f ${RUNTEST_DIR}/${i} ]; then
+ echo "ltp_target_run: doing test $i"
+ mkdir -p ${OUTPUT_DIR}/${i}
+ ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \
+ -l ${OUTPUT_DIR}/${i}/result.log \
+ -o ${OUTPUT_DIR}/${i}/output.log \
+ -d ${TMP_DIR} \
+ -S ./skiplist.txt \
+ -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
+ rm -rf ${TMP_DIR}/*
+ else
+ echo "Warning: ${i} is not available in the installed LTP"
+ fi
done
# gather posix results into pts.log
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
` (3 preceding siblings ...)
2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 18:55 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla
` (2 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: nguyen thi huong, fuego
From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
Currently It can skip one test case or one file that definied in 'skiplist'
Result of skipped test cases are marked as CONF ('stat=32')
However it could not skip a list of LTP test cases
Correct the syntax of FUEGO_LTP_SKIPLIST variable to parse all items of 'skiplist'
Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index 6a84d1c..e620b17 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -380,7 +380,7 @@ function test_deploy {
# to skipfiles (text files containing a list of LTP test case names)
# usually located under /fuego-rw/boards/
if [ -n "${FUNCTIONAL_LTP_SKIPLIST}" ]; then
- for item in "${FUNCTIONAL_LTP_SKIPLIST}"; do
+ for item in ${FUNCTIONAL_LTP_SKIPLIST}; do
if [ -f "$item" ]; then
cat "$item" >> ${LOGDIR}/skiplist.txt
else
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
` (4 preceding siblings ...)
2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 19:03 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla
2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: fuego, Nguyen Dat Tho
From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
1 file changed, 24 insertions(+), 18 deletions(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index e620b17..16d556f 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -188,7 +188,6 @@ function test_pre_check {
assert_define AR
assert_define RANLIB
#assert_define LDFLAGS
- assert_define FUNCTIONAL_LTP_TESTS
# FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
echo "Tests skipped by default in Fuego for now"
@@ -453,25 +452,32 @@ function test_run {
# on the type of the test (regular, posize, or realtime)
# Separate the test list by type, and pass the tests in different
# variables
- for a in $FUNCTIONAL_LTP_TESTS; do
- for b in $ALLTESTS; do
- if [ "$a" == "$b" ]; then
- TESTS+="$a "
- fi
- done
-
- for b in $ALLPTSTESTS; do
- if [ "$a" == "$b" ]; then
- PTSTESTS+="$a "
- fi
- done
- for b in $ALLRTTESTS; do
- if [ "$a" == "$b" ]; then
- RTTESTS+="$a "
- fi
+ if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
+ for a in $FUNCTIONAL_LTP_TESTS; do
+ for b in $ALLTESTS; do
+ if [ "$a" == "$b" ]; then
+ TESTS+="$a "
+ fi
+ done
+
+ for b in $ALLPTSTESTS; do
+ if [ "$a" == "$b" ]; then
+ PTSTESTS+="$a "
+ fi
+ done
+
+ for b in $ALLRTTESTS; do
+ if [ "$a" == "$b" ]; then
+ RTTESTS+="$a "
+ fi
+ done
done
- done
+ else
+ TESTS=$ALLTESTS
+ PTSTESTS=$ALLPTSTESTS
+ RTTESTS=$ALLRTTESTS
+ fi
# Let some of the tests fail, the information will be in the result xlsx file
report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh"
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
` (5 preceding siblings ...)
2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla
@ 2021-07-18 11:10 ` venkata.pyla
2021-07-22 20:18 ` Tim.Bird
2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird
7 siblings, 1 reply; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: fuego, Nguyen Dat Tho
From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
set ALLTESTS, ALLPTSTESTS and ALLRTTESTS automatically by
listing all possible runtests
Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 47 +++---------------------------
1 file changed, 4 insertions(+), 43 deletions(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index 16d556f..23dfc6c 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -4,49 +4,6 @@ tarball=ltp-full-20210524.tar.bz2
NEED_ROOT=1
-ALLTESTS="
-admin_tools fs_perms_simple ltp-aio-stress.part2 net_stress.interface
-can fs_readonly ltplite net_stress.ipsec_dccp sched
-cap_bounds fsx lvm.part1 net_stress.ipsec_icmp scsi_debug.part1
-commands hugetlb lvm.part2 net_stress.ipsec_sctp securebits
-connectors hyperthreading math net_stress.ipsec_tcp smack
-containers ima mm net_stress.ipsec_udp stress.part1
-controllers input modules net_stress.multicast stress.part2
-cpuhotplug io net.features net_stress.route stress.part3
-crashme io_cd net.ipv6 net.tcp_cmds syscalls
-cve io_floppy net.ipv6_lib net.tirpc_tests syscalls-ipc
-dio ipc net.multicast network_commands timers
-dma_thread_diotest kernel_misc net.nfs nptl tpm_tools
-fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing
-filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests
-fs ltp-aiodio.part3 net.sctp power_management_tests uevent
-fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto
-fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty
-smoketest"
-
-ALLPTSTESTS="AIO MEM MSG SEM SIG THR TMR TPS"
-
-# This list can be obtained by doing ./testscripts/test_realtime.sh -t list
-ALLRTTESTS="
-perf/latency
-func/measurement
-func/hrtimer-prio
-func/gtod_latency
-func/periodic_cpu_load
-func/pthread_kill_latency
-func/sched_football
-func/pi-tests
-func/thread_clock
-func/rt-migrate
-func/matrix_mult
-func/prio-preempt
-func/prio-wake
-func/pi_perf
-func/sched_latency
-func/async_handler
-func/sched_jitter
-"
-
# OK - the logic here is a bit complicated
# We support several different usage scenarios:
# 1: fuego builds, deploys and installs LTP
@@ -444,6 +401,10 @@ function test_run {
cmd "sed -i 's/^dio30 diotest6 -b 65536 -n 100 -i 100 -o 1024000/dio30 diotest6 -b 65536 -n 5 -i 100 -o 1024000/' $LTP_DESTDIR/runtest/dio"
cmd "sed -i 's/^msgctl11 msgctl11/msgctl11 msgctl11 -n 5/' $LTP_DESTDIR/runtest/syscalls"
+ ALLTESTS=$(ls $LTP_DESTDIR/runtest)
+ ALLPTSTESTS=$(grep 'usage: $(basename "$0")' $LTP_DESTDIR/bin/run-posix-option-group-test.sh | sed -e 's/^.*\[\(.*\)\]/\1/g' -e 's/|/ /g')
+ ALLRTTESTS=$(find $LTP_DESTDIR/testcases/realtime -type f -name run_auto.sh -printf '%h\n' | sed 's/^.*realtime\///g')
+
TESTS=""
PTSTESTS=""
RTTESTS=""
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 0/7] upstreaming work
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
` (6 preceding siblings ...)
2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla
@ 2021-07-20 20:58 ` Tim.Bird
7 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-20 20:58 UTC (permalink / raw)
To: venkata.pyla; +Cc: huong4.nguyenthi, binh1.tranhai, fuego
Hey Venkata,
Thanks very much for these patches. I looked at them quickly, and they look
pretty good. I'll do a more thorough review and try to get them applied
later this week. Some of these items are quite nice.
I'll let you know when they are applied.
Thanks!
-- Tim
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> Hi Tim,
>
> Below are some patches that we want to upstream our work from long time,
> the below patches fixes some issues in ftc and LTP functional test
>
> Daniel Sangorrin (1):
> ftc: docker ps will not work on the local board
>
> Nguyen Dat Tho (2):
> LTP: execute all tests when spec specifies "all"
> LTP: automatically obtain the list of tests
>
> Tran Hai Binh (1):
> LTP: Add the new tests and skip non-installed tests
>
> nguyen thi huong (2):
> ftc: fix test process is not killed when fuego test times out
> LTP: fix fuego test could not skip list of test cases
>
> yoshida toshiko (1):
> testplan_smoketest.json: ftc fails to add job for tesplan_smoketest
>
> overlays/testplans/testplan_smoketest.json | 2 +-
> scripts/ftc | 19 +++--
> tests/Functional.LTP/fuego_test.sh | 91 +++++++---------------
> tests/Functional.LTP/ltp_target_run.sh | 23 +++---
> 4 files changed, 56 insertions(+), 79 deletions(-)
>
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
@ 2021-07-22 18:28 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 18:28 UTC (permalink / raw)
To: venkata.pyla; +Cc: fuego
Looks good. Applied.
Thanks!
-- Tim
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
>
> When running fuego directly on the local board, instead of
> using a host-target approach, the check for a fuego
> container caused an error.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> scripts/ftc | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/ftc b/scripts/ftc
> index 4430dd2..2a6a640 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -5153,11 +5153,15 @@ def get_running_fuego_container_name():
> return cached_container_name
>
> # return the first container with "fuego" in the image or container name
> - dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
> - for line in dps_lines:
> - if "fuego" in line:
> - cached_container_name = line.strip().split(" ")[-1]
> - break
> + try:
> + with open(os.devnull, 'w') as devnull:
I was not familiar with this idiom for executing a subprocess with 2>/dev/null.
It's interesting...
> + dps_lines = subprocess.check_output("sudo docker ps", shell=True, stderr=devnull).split('\n')
> + for line in dps_lines:
> + if "fuego" in line:
> + cached_container_name = line.strip().split(" ")[-1]
> + break
> + except subprocess.CalledProcessError:
> + return None
>
> return cached_container_name
>
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out
2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla
@ 2021-07-22 18:34 ` Tim.Bird
2021-07-26 8:29 ` huong4.nguyenthi
0 siblings, 1 reply; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 18:34 UTC (permalink / raw)
To: venkata.pyla; +Cc: huong4.nguyenthi, fuego
OK - I have a question on this one.
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
>
> - Currently, fuego kill main process and does not kill test process when test times out
> In that case, test process still runs but fuego test result is not available
> - Revise implementation to kill all test processes and printout the result
>
> Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> scripts/ftc | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/ftc b/scripts/ftc
> index 2a6a640..5c300d0 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout):
>
> dprint("ftc_exec_command: command=%s" % command)
>
> - p = subprocess.Popen(command.split(), stdout=log, stderr=log)
> + p = subprocess.Popen(command.split(), stdout=log, stderr=log, preexec_fn=os.setpgrp)
>
> # specify timeout for command operation
> signal.signal(signal.SIGALRM, alarm_handler)
> @@ -3960,7 +3960,8 @@ def ftc_exec_command(command, timeout):
> # p.kill()
>
> # abort with prejudice...
> - p.kill()
> + pgrp = os.getpgid(p.pid)
> + os.killpg(pgrp, signal.SIGALRM)
Why are you sending a SIGALARM instead of a SIGKILL?
This is a weaker signal, that can be ignored or masked.
> timed_out = True
>
> finally:
> --
> 2.20.1
>
I've applied this patch, but I'd like to know if you have some reason for weakening the
signal sent to kill the process.
Thanks.
-- Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest
2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla
@ 2021-07-22 18:49 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 18:49 UTC (permalink / raw)
To: venkata.pyla; +Cc: fuego
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: yoshida toshiko <toshiko.yoshida@toshiba.co.jp>
>
> add missing comma after Benchmark.dbench4
>
> e.g: ftc add-jobs -b local -p testplan_smoketest
>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> overlays/testplans/testplan_smoketest.json | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/overlays/testplans/testplan_smoketest.json b/overlays/testplans/testplan_smoketest.json
> index 8c856c1..4bbbe4f 100644
> --- a/overlays/testplans/testplan_smoketest.json
> +++ b/overlays/testplans/testplan_smoketest.json
> @@ -12,7 +12,7 @@
> "testName": "Benchmark.Dhrystone"
> },
> {
> - "testName": "Benchmark.dbench4"
> + "testName": "Benchmark.dbench4",
> "timeout": "40m"
> },
> {
> --
> 2.20.1
>
FYI - the standalone testplans are now deprecated.
The new method of doing this is with a Functional.batch job, like so:
$ ftc add-jobs -b local -t batch_smoketest
The 'batch' jobs have a testplan embedded in them (in json format
as a HERE document in the fuego_test.sh script for the test.
However, standalone testplans are still supported for backwards compatibility,
and this *is* a bug, so I have applied this.
Thanks!
-- Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests
2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla
@ 2021-07-22 18:54 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 18:54 UTC (permalink / raw)
To: venkata.pyla; +Cc: binh1.tranhai, fuego
Kind of weird to add these, and then remove them in patch 7, but OK.
It makes the patch series application easier, but you guys could have
done a git rebase and eliminated this one.
Applied.
-- Tim
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: Tran Hai Binh <binh1.tranhai@toshiba.co.jp>
>
> - Add the new tests in 20200930 that are not in ALLTESTS:
> s390x_tests, uevent, crypto
> - Skip non-installed tests in LTP
>
> Signed-off-by: Tran Hai Binh <binh1.tranhai@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> tests/Functional.LTP/fuego_test.sh | 6 +++---
> tests/Functional.LTP/ltp_target_run.sh | 23 ++++++++++++++---------
> 2 files changed, 17 insertions(+), 12 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index a4f551b..6a84d1c 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -18,9 +18,9 @@ cve io_floppy net.ipv6_lib net.tirpc_tests
> dio ipc net.multicast network_commands timers
> dma_thread_diotest kernel_misc net.nfs nptl tpm_tools
> fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing
> -filecaps ltp-aiodio.part2 net.rpc_tests pipes
> -fs ltp-aiodio.part3 net.sctp power_management_tests
> -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive
> +filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests
> +fs ltp-aiodio.part3 net.sctp power_management_tests uevent
> +fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto
> fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty
> smoketest"
>
> diff --git a/tests/Functional.LTP/ltp_target_run.sh b/tests/Functional.LTP/ltp_target_run.sh
> index 52a006a..ded81de 100755
> --- a/tests/Functional.LTP/ltp_target_run.sh
> +++ b/tests/Functional.LTP/ltp_target_run.sh
> @@ -5,6 +5,7 @@
>
> OUTPUT_DIR=${PWD}/result
> TMP_DIR=${PWD}/tmp
> +RUNTEST_DIR=${PWD}/runtest
>
> [ -d ${TMP_DIR} ] && rm -rf ${TMP_DIR}
> mkdir -p ${TMP_DIR}
> @@ -18,15 +19,19 @@ echo "ltp_target_run: ${TESTS} | ${PTSTESTS} | ${RTTESTS}"
>
> # FIXTHIS: add -t option for limiting the duration of each test group execution
> for i in ${TESTS}; do
> - echo "ltp_target_run: doing test $i"
> - mkdir -p ${OUTPUT_DIR}/${i}
> - ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \
> - -l ${OUTPUT_DIR}/${i}/result.log \
> - -o ${OUTPUT_DIR}/${i}/output.log \
> - -d ${TMP_DIR} \
> - -S ./skiplist.txt \
> - -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
> - rm -rf ${TMP_DIR}/*
> + if [ -f ${RUNTEST_DIR}/${i} ]; then
> + echo "ltp_target_run: doing test $i"
> + mkdir -p ${OUTPUT_DIR}/${i}
> + ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \
> + -l ${OUTPUT_DIR}/${i}/result.log \
> + -o ${OUTPUT_DIR}/${i}/output.log \
> + -d ${TMP_DIR} \
> + -S ./skiplist.txt \
> + -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
> + rm -rf ${TMP_DIR}/*
> + else
> + echo "Warning: ${i} is not available in the installed LTP"
> + fi
> done
>
> # gather posix results into pts.log
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases
2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla
@ 2021-07-22 18:55 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 18:55 UTC (permalink / raw)
To: venkata.pyla; +Cc: huong4.nguyenthi, fuego
Nice catch!! Thanks.
Applied.
-- Tim
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Sent: Sunday, July 18, 2021 5:10 AM
> To: Bird, Tim <Tim.Bird@sony.com>
> Cc: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>; daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org; venkata pyla
> <venkata.pyla@toshiba-tsip.com>
> Subject: [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases
>
> From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
>
> Currently It can skip one test case or one file that definied in 'skiplist'
> Result of skipped test cases are marked as CONF ('stat=32')
> However it could not skip a list of LTP test cases
> Correct the syntax of FUEGO_LTP_SKIPLIST variable to parse all items of 'skiplist'
>
> Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> tests/Functional.LTP/fuego_test.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index 6a84d1c..e620b17 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -380,7 +380,7 @@ function test_deploy {
> # to skipfiles (text files containing a list of LTP test case names)
> # usually located under /fuego-rw/boards/
> if [ -n "${FUNCTIONAL_LTP_SKIPLIST}" ]; then
> - for item in "${FUNCTIONAL_LTP_SKIPLIST}"; do
> + for item in ${FUNCTIONAL_LTP_SKIPLIST}; do
> if [ -f "$item" ]; then
> cat "$item" >> ${LOGDIR}/skiplist.txt
> else
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla
@ 2021-07-22 19:03 ` Tim.Bird
2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat
0 siblings, 1 reply; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 19:03 UTC (permalink / raw)
To: venkata.pyla; +Cc: fuego, tho1.nguyendat
This one has problems.
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
>
I'm not sure that the summary line of this change matches what it does.
From reading the code, it appears that "tests" in the spec needs to be
missing or empty, in order to activate this. If "tests" is "all",
then it appears that none of the tests will match, and you'd just
end up with empty lists.
Also, I had to intuit the reasoning here, since there is no description
for this. I don't mind the concept of being able to easily run all
LTP tests, but it would be nice to use the string "all" to signal
this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
It should probably go into a comment in the code as well, and maybe
in the test.yaml file, so that users and future developers will see
it and understand the behaviour.
Also, you might want to add a spec to the spec.json file for "all", to
show it's usage.
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
>
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> 1 file changed, 24 insertions(+), 18 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index e620b17..16d556f 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -188,7 +188,6 @@ function test_pre_check {
> assert_define AR
> assert_define RANLIB
> #assert_define LDFLAGS
> - assert_define FUNCTIONAL_LTP_TESTS
>
> # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> echo "Tests skipped by default in Fuego for now"
> @@ -453,25 +452,32 @@ function test_run {
> # on the type of the test (regular, posize, or realtime)
> # Separate the test list by type, and pass the tests in different
> # variables
> - for a in $FUNCTIONAL_LTP_TESTS; do
> - for b in $ALLTESTS; do
> - if [ "$a" == "$b" ]; then
> - TESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLPTSTESTS; do
> - if [ "$a" == "$b" ]; then
> - PTSTESTS+="$a "
> - fi
> - done
>
> - for b in $ALLRTTESTS; do
> - if [ "$a" == "$b" ]; then
> - RTTESTS+="$a "
> - fi
> + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> + for a in $FUNCTIONAL_LTP_TESTS; do
> + for b in $ALLTESTS; do
> + if [ "$a" == "$b" ]; then
> + TESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLPTSTESTS; do
> + if [ "$a" == "$b" ]; then
> + PTSTESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLRTTESTS; do
> + if [ "$a" == "$b" ]; then
> + RTTESTS+="$a "
> + fi
> + done
> done
> - done
> + else
> + TESTS=$ALLTESTS
> + PTSTESTS=$ALLPTSTESTS
> + RTTESTS=$ALLRTTESTS
> + fi
>
> # Let some of the tests fail, the information will be in the result xlsx file
> report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> ./ltp_target_run.sh"
> --
> 2.20.1
>
I have NOT applied this one. Please change this to check for the string "all"
in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
Thanks,
-- Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests
2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla
@ 2021-07-22 20:18 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-22 20:18 UTC (permalink / raw)
To: venkata.pyla; +Cc: fuego, tho1.nguyendat
I love this concept, but the implementation had problems.
See below.
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
>
> set ALLTESTS, ALLPTSTESTS and ALLRTTESTS automatically by
> listing all possible runtests
>
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> tests/Functional.LTP/fuego_test.sh | 47 +++---------------------------
> 1 file changed, 4 insertions(+), 43 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index 16d556f..23dfc6c 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -4,49 +4,6 @@ tarball=ltp-full-20210524.tar.bz2
>
> NEED_ROOT=1
>
> -ALLTESTS="
> -admin_tools fs_perms_simple ltp-aio-stress.part2 net_stress.interface
> -can fs_readonly ltplite net_stress.ipsec_dccp sched
> -cap_bounds fsx lvm.part1 net_stress.ipsec_icmp scsi_debug.part1
> -commands hugetlb lvm.part2 net_stress.ipsec_sctp securebits
> -connectors hyperthreading math net_stress.ipsec_tcp smack
> -containers ima mm net_stress.ipsec_udp stress.part1
> -controllers input modules net_stress.multicast stress.part2
> -cpuhotplug io net.features net_stress.route stress.part3
> -crashme io_cd net.ipv6 net.tcp_cmds syscalls
> -cve io_floppy net.ipv6_lib net.tirpc_tests syscalls-ipc
> -dio ipc net.multicast network_commands timers
> -dma_thread_diotest kernel_misc net.nfs nptl tpm_tools
> -fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing
> -filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests
> -fs ltp-aiodio.part3 net.sctp power_management_tests uevent
> -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto
> -fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty
> -smoketest"
> -
> -ALLPTSTESTS="AIO MEM MSG SEM SIG THR TMR TPS"
> -
> -# This list can be obtained by doing ./testscripts/test_realtime.sh -t list
> -ALLRTTESTS="
> -perf/latency
> -func/measurement
> -func/hrtimer-prio
> -func/gtod_latency
> -func/periodic_cpu_load
> -func/pthread_kill_latency
> -func/sched_football
> -func/pi-tests
> -func/thread_clock
> -func/rt-migrate
> -func/matrix_mult
> -func/prio-preempt
> -func/prio-wake
> -func/pi_perf
> -func/sched_latency
> -func/async_handler
> -func/sched_jitter
> -"
> -
> # OK - the logic here is a bit complicated
> # We support several different usage scenarios:
> # 1: fuego builds, deploys and installs LTP
> @@ -444,6 +401,10 @@ function test_run {
> cmd "sed -i 's/^dio30 diotest6 -b 65536 -n 100 -i 100 -o 1024000/dio30 diotest6 -b 65536 -n 5 -i 100 -o 1024000/'
> $LTP_DESTDIR/runtest/dio"
> cmd "sed -i 's/^msgctl11 msgctl11/msgctl11 msgctl11 -n 5/' $LTP_DESTDIR/runtest/syscalls"
>
> + ALLTESTS=$(ls $LTP_DESTDIR/runtest)
> + ALLPTSTESTS=$(grep 'usage: $(basename "$0")' $LTP_DESTDIR/bin/run-posix-option-group-test.sh | sed -e 's/^.*\[\(.*\)\]/\1/g' -e
> 's/|/ /g')
> + ALLRTTESTS=$(find $LTP_DESTDIR/testcases/realtime -type f -name run_auto.sh -printf '%h\n' | sed 's/^.*realtime\///g')
> +
The above commands only work for a local board. LTP_DESTDIR is on the target.
These should use 'cmd ...' to execute the operation on the board, instead of running
the command locally.
I made this change, and modified the ALLPTSTESTS check slightly, to make it easier to read.
> TESTS=""
> PTSTESTS=""
> RTTESTS=""
> --
> 2.20.1
>
Thanks. This is applied.
I'm going to push these changes to the fuego-core master branch. Can you please
pull them and check that they work for you?
Thanks.
-- Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out
2021-07-22 18:34 ` Tim.Bird
@ 2021-07-26 8:29 ` huong4.nguyenthi
0 siblings, 0 replies; 21+ messages in thread
From: huong4.nguyenthi @ 2021-07-26 8:29 UTC (permalink / raw)
To: Tim.Bird; +Cc: fuego, TSDVGroup1skerlet
Dear Tim,
Thanks for your feedback.
The function handle timeout event so I think send SIGALARM is more reasonable.
Test driver or subprocesses will decide what to do with timeout signal.
For now, SIGALARM is trapped in fuego functions.sh
Signal handler will do post_tests and processing before exiting main process
When main process exit, all subprocesses that belong to group process will be exited.
Best Regards,
Huong
-----Original Message-----
From: Tim.Bird@sony.com <Tim.Bird@sony.com>
Sent: Friday, July 23, 2021 1:35 AM
To: pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
Cc: nguyen thi huong(TSDV Eng 1) <huong4.nguyenthi@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □SWC◯ACT) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org
Subject: RE: [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out
OK - I have a question on this one.
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
>
> From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
>
> - Currently, fuego kill main process and does not kill test process when test times out
> In that case, test process still runs but fuego test result is not
> available
> - Revise implementation to kill all test processes and printout the
> result
>
> Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> scripts/ftc | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/ftc b/scripts/ftc index 2a6a640..5c300d0 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout):
>
> dprint("ftc_exec_command: command=%s" % command)
>
> - p = subprocess.Popen(command.split(), stdout=log, stderr=log)
> + p = subprocess.Popen(command.split(), stdout=log, stderr=log,
> + preexec_fn=os.setpgrp)
>
> # specify timeout for command operation
> signal.signal(signal.SIGALRM, alarm_handler) @@ -3960,7 +3960,8
> @@ def ftc_exec_command(command, timeout):
> # p.kill()
>
> # abort with prejudice...
> - p.kill()
> + pgrp = os.getpgid(p.pid)
> + os.killpg(pgrp, signal.SIGALRM)
Why are you sending a SIGALARM instead of a SIGKILL?
This is a weaker signal, that can be ignored or masked.
> timed_out = True
>
> finally:
> --
> 2.20.1
>
I've applied this patch, but I'd like to know if you have some reason for weakening the signal sent to kill the process.
Thanks.
-- Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-22 19:03 ` Tim.Bird
@ 2021-07-27 3:11 ` tho1.nguyendat
2021-07-27 23:00 ` Tim.Bird
0 siblings, 1 reply; 21+ messages in thread
From: tho1.nguyendat @ 2021-07-27 3:11 UTC (permalink / raw)
To: tim.bird; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 7700 bytes --]
Dear Tim,
I would like to send my update.
From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
tests/Functional.LTP/spec.json | 6 +++++
2 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index 85adabf..689abfe 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -414,25 +414,31 @@ function test_run {
# on the type of the test (regular, posize, or realtime)
# Separate the test list by type, and pass the tests in different
# variables
- for a in $FUNCTIONAL_LTP_TESTS; do
- for b in $ALLTESTS; do
- if [ "$a" == "$b" ]; then
- TESTS+="$a "
- fi
- done
-
- for b in $ALLPTSTESTS; do
- if [ "$a" == "$b" ]; then
- PTSTESTS+="$a "
- fi
- done
-
- for b in $ALLRTTESTS; do
- if [ "$a" == "$b" ]; then
- RTTESTS+="$a "
- fi
+ if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
+ TESTS=$ALLTESTS
+ PTSTESTS=$ALLPTSTESTS
+ RTTESTS=$ALLRTTESTS
+ else
+ for a in $FUNCTIONAL_LTP_TESTS; do
+ for b in $ALLTESTS; do
+ if [ "$a" == "$b" ]; then
+ TESTS+="$a "
+ fi
+ done
+
+ for b in $ALLPTSTESTS; do
+ if [ "$a" == "$b" ]; then
+ PTSTESTS+="$a "
+ fi
+ done
+
+ for b in $ALLRTTESTS; do
+ if [ "$a" == "$b" ]; then
+ RTTESTS+="$a "
+ fi
+ done
done
- done
+ fi
# Let some of the tests fail, the information will be in the result xlsx file
report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh"
diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
index d30ba4c..53ee115 100644
--- a/tests/Functional.LTP/spec.json
+++ b/tests/Functional.LTP/spec.json
@@ -1,6 +1,12 @@
{
"testName": "Functional.LTP",
"specs": {
+ "all": {
+ "tests": "all",
+ "noautoskip": "true",
+ "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
+ "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
+ },
"default": {
"tests": "syscalls SEM",
"extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
--
2.20.1
________________________________
Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
Đã gửi: 23 Tháng Bảy 2021 2:03 SA
Đến: pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
Cc: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □SWC◯ACT) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>
Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
This one has problems.
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
>
I'm not sure that the summary line of this change matches what it does.
From reading the code, it appears that "tests" in the spec needs to be
missing or empty, in order to activate this. If "tests" is "all",
then it appears that none of the tests will match, and you'd just
end up with empty lists.
Also, I had to intuit the reasoning here, since there is no description
for this. I don't mind the concept of being able to easily run all
LTP tests, but it would be nice to use the string "all" to signal
this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
It should probably go into a comment in the code as well, and maybe
in the test.yaml file, so that users and future developers will see
it and understand the behaviour.
Also, you might want to add a spec to the spec.json file for "all", to
show it's usage.
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
>
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> 1 file changed, 24 insertions(+), 18 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index e620b17..16d556f 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -188,7 +188,6 @@ function test_pre_check {
> assert_define AR
> assert_define RANLIB
> #assert_define LDFLAGS
> - assert_define FUNCTIONAL_LTP_TESTS
>
> # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> echo "Tests skipped by default in Fuego for now"
> @@ -453,25 +452,32 @@ function test_run {
> # on the type of the test (regular, posize, or realtime)
> # Separate the test list by type, and pass the tests in different
> # variables
> - for a in $FUNCTIONAL_LTP_TESTS; do
> - for b in $ALLTESTS; do
> - if [ "$a" == "$b" ]; then
> - TESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLPTSTESTS; do
> - if [ "$a" == "$b" ]; then
> - PTSTESTS+="$a "
> - fi
> - done
>
> - for b in $ALLRTTESTS; do
> - if [ "$a" == "$b" ]; then
> - RTTESTS+="$a "
> - fi
> + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> + for a in $FUNCTIONAL_LTP_TESTS; do
> + for b in $ALLTESTS; do
> + if [ "$a" == "$b" ]; then
> + TESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLPTSTESTS; do
> + if [ "$a" == "$b" ]; then
> + PTSTESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLRTTESTS; do
> + if [ "$a" == "$b" ]; then
> + RTTESTS+="$a "
> + fi
> + done
> done
> - done
> + else
> + TESTS=$ALLTESTS
> + PTSTESTS=$ALLPTSTESTS
> + RTTESTS=$ALLRTTESTS
> + fi
>
> # Let some of the tests fail, the information will be in the result xlsx file
> report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> ./ltp_target_run.sh"
> --
> 2.20.1
>
I have NOT applied this one. Please change this to check for the string "all"
in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
Thanks,
-- Tim
[-- Attachment #2: Type: text/html, Size: 17408 bytes --]
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat
@ 2021-07-27 23:00 ` Tim.Bird
2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat
0 siblings, 1 reply; 21+ messages in thread
From: Tim.Bird @ 2021-07-27 23:00 UTC (permalink / raw)
To: tho1.nguyendat; +Cc: fuego
The patch looks good. Thanks for the changes.
But I can't figure out how to apply it.
The patch content is encoded in base64, and when I tried to manually
decode it at the command line, I got an error "invalid input".
Can you please re-send this patch as an attachment instead of inline
in the message body? Something is going wrong with the mail
handling of the text.
-- Tim
> -----Original Message-----
> From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp>
> Dear Tim,
>
> I would like to send my update.
>
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
>
>
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
>
> ---
> tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> tests/Functional.LTP/spec.json | 6 +++++
> 2 files changed, 30 insertions(+), 18 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index 85adabf..689abfe 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -414,25 +414,31 @@ function test_run {
> # on the type of the test (regular, posize, or realtime)
> # Separate the test list by type, and pass the tests in different
> # variables
> - for a in $FUNCTIONAL_LTP_TESTS; do
> - for b in $ALLTESTS; do
> - if [ "$a" == "$b" ]; then
> - TESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLPTSTESTS; do
> - if [ "$a" == "$b" ]; then
> - PTSTESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLRTTESTS; do
> - if [ "$a" == "$b" ]; then
> - RTTESTS+="$a "
> - fi
> + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
> + TESTS=$ALLTESTS
> + PTSTESTS=$ALLPTSTESTS
> + RTTESTS=$ALLRTTESTS
> + else
> + for a in $FUNCTIONAL_LTP_TESTS; do
> + for b in $ALLTESTS; do
> + if [ "$a" == "$b" ]; then
> + TESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLPTSTESTS; do
> + if [ "$a" == "$b" ]; then
> + PTSTESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLRTTESTS; do
> + if [ "$a" == "$b" ]; then
> + RTTESTS+="$a "
> + fi
> + done
> done
> - done
> + fi
>
> # Let some of the tests fail, the information will be in the result xlsx file
> report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> ./ltp_target_run.sh"
> diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
> index d30ba4c..53ee115 100644
> --- a/tests/Functional.LTP/spec.json
> +++ b/tests/Functional.LTP/spec.json
> @@ -1,6 +1,12 @@
> {
> "testName": "Functional.LTP",
> "specs": {
> + "all": {
> + "tests": "all",
> + "noautoskip": "true",
> + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
> + },
> "default": {
> "tests": "syscalls SEM",
> "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> --
> 2.20.1
>
> ________________________________
>
> Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Đã gửi: 23 Tháng Bảy 2021 2:03 SA
> Đến: pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
> Cc: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □SWC◯ACT)
> <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>
> Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
>
> This one has problems.
>
> > -----Original Message-----
> > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> >
> I'm not sure that the summary line of this change matches what it does.
>
> From reading the code, it appears that "tests" in the spec needs to be
> missing or empty, in order to activate this. If "tests" is "all",
> then it appears that none of the tests will match, and you'd just
> end up with empty lists.
>
> Also, I had to intuit the reasoning here, since there is no description
> for this. I don't mind the concept of being able to easily run all
> LTP tests, but it would be nice to use the string "all" to signal
> this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
>
> It should probably go into a comment in the code as well, and maybe
> in the test.yaml file, so that users and future developers will see
> it and understand the behaviour.
>
> Also, you might want to add a spec to the spec.json file for "all", to
> show it's usage.
>
> > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> >
> > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > ---
> > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> > 1 file changed, 24 insertions(+), 18 deletions(-)
> >
> > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> > index e620b17..16d556f 100755
> > --- a/tests/Functional.LTP/fuego_test.sh
> > +++ b/tests/Functional.LTP/fuego_test.sh
> > @@ -188,7 +188,6 @@ function test_pre_check {
> > assert_define AR
> > assert_define RANLIB
> > #assert_define LDFLAGS
> > - assert_define FUNCTIONAL_LTP_TESTS
> >
> > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> > echo "Tests skipped by default in Fuego for now"
> > @@ -453,25 +452,32 @@ function test_run {
> > # on the type of the test (regular, posize, or realtime)
> > # Separate the test list by type, and pass the tests in different
> > # variables
> > - for a in $FUNCTIONAL_LTP_TESTS; do
> > - for b in $ALLTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - TESTS+="$a "
> > - fi
> > - done
> > -
> > - for b in $ALLPTSTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - PTSTESTS+="$a "
> > - fi
> > - done
> >
> > - for b in $ALLRTTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - RTTESTS+="$a "
> > - fi
> > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> > + for a in $FUNCTIONAL_LTP_TESTS; do
> > + for b in $ALLTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + TESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLPTSTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + PTSTESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLRTTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + RTTESTS+="$a "
> > + fi
> > + done
> > done
> > - done
> > + else
> > + TESTS=$ALLTESTS
> > + PTSTESTS=$ALLPTSTESTS
> > + RTTESTS=$ALLRTTESTS
> > + fi
> >
> > # Let some of the tests fail, the information will be in the result xlsx file
> > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> > ./ltp_target_run.sh"
> > --
> > 2.20.1
> >
>
> I have NOT applied this one. Please change this to check for the string "all"
> in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
>
> Thanks,
> -- Tim
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] Trả lời: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-27 23:00 ` Tim.Bird
@ 2021-07-28 0:42 ` tho1.nguyendat
2021-07-28 22:10 ` Tim.Bird
0 siblings, 1 reply; 21+ messages in thread
From: tho1.nguyendat @ 2021-07-28 0:42 UTC (permalink / raw)
To: Tim.Bird; +Cc: fuego
[-- Attachment #1.1: Type: text/plain, Size: 9222 bytes --]
Hi Tim,
I attached the patch file.
Could you please try it?
Thanks,
________________________________
Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
Đã gửi: 28 Tháng Bảy 2021 6:00 SA
Đến: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>
Cc: sangorrin daniel(サンゴリン ダニエル □SWC◯ACT) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>; pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
Chủ đề: RE: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
The patch looks good. Thanks for the changes.
But I can't figure out how to apply it.
The patch content is encoded in base64, and when I tried to manually
decode it at the command line, I got an error "invalid input".
Can you please re-send this patch as an attachment instead of inline
in the message body? Something is going wrong with the mail
handling of the text.
-- Tim
> -----Original Message-----
> From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp>
> Dear Tim,
>
> I would like to send my update.
>
> From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
>
>
> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
>
> ---
> tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> tests/Functional.LTP/spec.json | 6 +++++
> 2 files changed, 30 insertions(+), 18 deletions(-)
>
> diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> index 85adabf..689abfe 100755
> --- a/tests/Functional.LTP/fuego_test.sh
> +++ b/tests/Functional.LTP/fuego_test.sh
> @@ -414,25 +414,31 @@ function test_run {
> # on the type of the test (regular, posize, or realtime)
> # Separate the test list by type, and pass the tests in different
> # variables
> - for a in $FUNCTIONAL_LTP_TESTS; do
> - for b in $ALLTESTS; do
> - if [ "$a" == "$b" ]; then
> - TESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLPTSTESTS; do
> - if [ "$a" == "$b" ]; then
> - PTSTESTS+="$a "
> - fi
> - done
> -
> - for b in $ALLRTTESTS; do
> - if [ "$a" == "$b" ]; then
> - RTTESTS+="$a "
> - fi
> + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
> + TESTS=$ALLTESTS
> + PTSTESTS=$ALLPTSTESTS
> + RTTESTS=$ALLRTTESTS
> + else
> + for a in $FUNCTIONAL_LTP_TESTS; do
> + for b in $ALLTESTS; do
> + if [ "$a" == "$b" ]; then
> + TESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLPTSTESTS; do
> + if [ "$a" == "$b" ]; then
> + PTSTESTS+="$a "
> + fi
> + done
> +
> + for b in $ALLRTTESTS; do
> + if [ "$a" == "$b" ]; then
> + RTTESTS+="$a "
> + fi
> + done
> done
> - done
> + fi
>
> # Let some of the tests fail, the information will be in the result xlsx file
> report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> ./ltp_target_run.sh"
> diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
> index d30ba4c..53ee115 100644
> --- a/tests/Functional.LTP/spec.json
> +++ b/tests/Functional.LTP/spec.json
> @@ -1,6 +1,12 @@
> {
> "testName": "Functional.LTP",
> "specs": {
> + "all": {
> + "tests": "all",
> + "noautoskip": "true",
> + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
> + },
> "default": {
> "tests": "syscalls SEM",
> "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> --
> 2.20.1
>
> ________________________________
>
> Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Đã gửi: 23 Tháng Bảy 2021 2:03 SA
> Đến: pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
> Cc: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □SWC◯ACT)
> <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>
> Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
>
> This one has problems.
>
> > -----Original Message-----
> > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> >
> I'm not sure that the summary line of this change matches what it does.
>
> From reading the code, it appears that "tests" in the spec needs to be
> missing or empty, in order to activate this. If "tests" is "all",
> then it appears that none of the tests will match, and you'd just
> end up with empty lists.
>
> Also, I had to intuit the reasoning here, since there is no description
> for this. I don't mind the concept of being able to easily run all
> LTP tests, but it would be nice to use the string "all" to signal
> this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
>
> It should probably go into a comment in the code as well, and maybe
> in the test.yaml file, so that users and future developers will see
> it and understand the behaviour.
>
> Also, you might want to add a spec to the spec.json file for "all", to
> show it's usage.
>
> > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> >
> > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > ---
> > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> > 1 file changed, 24 insertions(+), 18 deletions(-)
> >
> > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> > index e620b17..16d556f 100755
> > --- a/tests/Functional.LTP/fuego_test.sh
> > +++ b/tests/Functional.LTP/fuego_test.sh
> > @@ -188,7 +188,6 @@ function test_pre_check {
> > assert_define AR
> > assert_define RANLIB
> > #assert_define LDFLAGS
> > - assert_define FUNCTIONAL_LTP_TESTS
> >
> > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> > echo "Tests skipped by default in Fuego for now"
> > @@ -453,25 +452,32 @@ function test_run {
> > # on the type of the test (regular, posize, or realtime)
> > # Separate the test list by type, and pass the tests in different
> > # variables
> > - for a in $FUNCTIONAL_LTP_TESTS; do
> > - for b in $ALLTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - TESTS+="$a "
> > - fi
> > - done
> > -
> > - for b in $ALLPTSTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - PTSTESTS+="$a "
> > - fi
> > - done
> >
> > - for b in $ALLRTTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - RTTESTS+="$a "
> > - fi
> > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> > + for a in $FUNCTIONAL_LTP_TESTS; do
> > + for b in $ALLTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + TESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLPTSTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + PTSTESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLRTTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + RTTESTS+="$a "
> > + fi
> > + done
> > done
> > - done
> > + else
> > + TESTS=$ALLTESTS
> > + PTSTESTS=$ALLPTSTESTS
> > + RTTESTS=$ALLRTTESTS
> > + fi
> >
> > # Let some of the tests fail, the information will be in the result xlsx file
> > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> > ./ltp_target_run.sh"
> > --
> > 2.20.1
> >
>
> I have NOT applied this one. Please change this to check for the string "all"
> in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
>
> Thanks,
> -- Tim
>
[-- Attachment #1.2: Type: text/html, Size: 20093 bytes --]
[-- Attachment #2: 0001-LTP-execute-all-tests-when-spec-specifies-all.patch --]
[-- Type: application/octet-stream, Size: 3182 bytes --]
From b0077de9fa9ba8d1ad18b282ba7d3ec6ece6532f Mon Sep 17 00:00:00 2001
From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Date: Tue, 27 Jul 2021 08:39:42 +0700
Subject: [PATCH] LTP: execute all tests when spec specifies "all"
Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
tests/Functional.LTP/spec.json | 6 +++++
2 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
index 85adabf..689abfe 100755
--- a/tests/Functional.LTP/fuego_test.sh
+++ b/tests/Functional.LTP/fuego_test.sh
@@ -414,25 +414,31 @@ function test_run {
# on the type of the test (regular, posize, or realtime)
# Separate the test list by type, and pass the tests in different
# variables
- for a in $FUNCTIONAL_LTP_TESTS; do
- for b in $ALLTESTS; do
- if [ "$a" == "$b" ]; then
- TESTS+="$a "
- fi
- done
-
- for b in $ALLPTSTESTS; do
- if [ "$a" == "$b" ]; then
- PTSTESTS+="$a "
- fi
- done
-
- for b in $ALLRTTESTS; do
- if [ "$a" == "$b" ]; then
- RTTESTS+="$a "
- fi
+ if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
+ TESTS=$ALLTESTS
+ PTSTESTS=$ALLPTSTESTS
+ RTTESTS=$ALLRTTESTS
+ else
+ for a in $FUNCTIONAL_LTP_TESTS; do
+ for b in $ALLTESTS; do
+ if [ "$a" == "$b" ]; then
+ TESTS+="$a "
+ fi
+ done
+
+ for b in $ALLPTSTESTS; do
+ if [ "$a" == "$b" ]; then
+ PTSTESTS+="$a "
+ fi
+ done
+
+ for b in $ALLRTTESTS; do
+ if [ "$a" == "$b" ]; then
+ RTTESTS+="$a "
+ fi
+ done
done
- done
+ fi
# Let some of the tests fail, the information will be in the result xlsx file
report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh"
diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
index d30ba4c..53ee115 100644
--- a/tests/Functional.LTP/spec.json
+++ b/tests/Functional.LTP/spec.json
@@ -1,6 +1,12 @@
{
"testName": "Functional.LTP",
"specs": {
+ "all": {
+ "tests": "all",
+ "noautoskip": "true",
+ "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
+ "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
+ },
"default": {
"tests": "syscalls SEM",
"extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
--
2.20.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] Trả lời: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat
@ 2021-07-28 22:10 ` Tim.Bird
0 siblings, 0 replies; 21+ messages in thread
From: Tim.Bird @ 2021-07-28 22:10 UTC (permalink / raw)
To: tho1.nguyendat; +Cc: fuego
> -----Original Message-----
> From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp>
>
> Hi Tim,
>
> I attached the patch file.
> Could you please try it?
This worked. This patch is applied and pushed. Can you please pull the latest
master branch, and test this out in your lab?
Thanks,
-- Tim
> ________________________________
>
> Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Đã gửi: 28 Tháng Bảy 2021 6:00 SA
> Đến: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>
> Cc: sangorrin daniel(サンゴリン ダニエル □SWC◯ACT) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org
> <fuego@lists.linuxfoundation.org>; pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
> Chủ đề: RE: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
>
> The patch looks good. Thanks for the changes.
> But I can't figure out how to apply it.
>
> The patch content is encoded in base64, and when I tried to manually
> decode it at the command line, I got an error "invalid input".
>
> Can you please re-send this patch as an attachment instead of inline
> in the message body? Something is going wrong with the mail
> handling of the text.
> -- Tim
>
> > -----Original Message-----
> > From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp>
> > Dear Tim,
> >
> > I would like to send my update.
> >
> > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> >
> >
> > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >
> >
> > ---
> > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> > tests/Functional.LTP/spec.json | 6 +++++
> > 2 files changed, 30 insertions(+), 18 deletions(-)
> >
> > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> > index 85adabf..689abfe 100755
> > --- a/tests/Functional.LTP/fuego_test.sh
> > +++ b/tests/Functional.LTP/fuego_test.sh
> > @@ -414,25 +414,31 @@ function test_run {
> > # on the type of the test (regular, posize, or realtime)
> > # Separate the test list by type, and pass the tests in different
> > # variables
> > - for a in $FUNCTIONAL_LTP_TESTS; do
> > - for b in $ALLTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - TESTS+="$a "
> > - fi
> > - done
> > -
> > - for b in $ALLPTSTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - PTSTESTS+="$a "
> > - fi
> > - done
> > -
> > - for b in $ALLRTTESTS; do
> > - if [ "$a" == "$b" ]; then
> > - RTTESTS+="$a "
> > - fi
> > + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then
> > + TESTS=$ALLTESTS
> > + PTSTESTS=$ALLPTSTESTS
> > + RTTESTS=$ALLRTTESTS
> > + else
> > + for a in $FUNCTIONAL_LTP_TESTS; do
> > + for b in $ALLTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + TESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLPTSTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + PTSTESTS+="$a "
> > + fi
> > + done
> > +
> > + for b in $ALLRTTESTS; do
> > + if [ "$a" == "$b" ]; then
> > + RTTESTS+="$a "
> > + fi
> > + done
> > done
> > - done
> > + fi
> >
> > # Let some of the tests fail, the information will be in the result xlsx file
> > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> > ./ltp_target_run.sh"
> > diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
> > index d30ba4c..53ee115 100644
> > --- a/tests/Functional.LTP/spec.json
> > +++ b/tests/Functional.LTP/spec.json
> > @@ -1,6 +1,12 @@
> > {
> > "testName": "Functional.LTP",
> > "specs": {
> > + "all": {
> > + "tests": "all",
> > + "noautoskip": "true",
> > + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> > + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
> > + },
> > "default": {
> > "tests": "syscalls SEM",
> > "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
> > --
> > 2.20.1
> >
> > ________________________________
> >
> > Từ: Tim.Bird@sony.com <Tim.Bird@sony.com>
> > Đã gửi: 23 Tháng Bảy 2021 2:03 SA
> > Đến: pyla venkata(TSIP) <Venkata.Pyla@toshiba-tsip.com>
> > Cc: nguyen dat tho(TSDV Eng 1) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □SWC◯ACT)
> > <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>
> > Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> >
> > This one has problems.
> >
> > > -----Original Message-----
> > > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all"
> > >
> > I'm not sure that the summary line of this change matches what it does.
> >
> > From reading the code, it appears that "tests" in the spec needs to be
> > missing or empty, in order to activate this. If "tests" is "all",
> > then it appears that none of the tests will match, and you'd just
> > end up with empty lists.
> >
> > Also, I had to intuit the reasoning here, since there is no description
> > for this. I don't mind the concept of being able to easily run all
> > LTP tests, but it would be nice to use the string "all" to signal
> > this instead of just leaving FUNCTIONAL_LTP_TESTS empty.
> >
> > It should probably go into a comment in the code as well, and maybe
> > in the test.yaml file, so that users and future developers will see
> > it and understand the behaviour.
> >
> > Also, you might want to add a spec to the spec.json file for "all", to
> > show it's usage.
> >
> > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > >
> > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp>
> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > > ---
> > > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++-------------
> > > 1 file changed, 24 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh
> > > index e620b17..16d556f 100755
> > > --- a/tests/Functional.LTP/fuego_test.sh
> > > +++ b/tests/Functional.LTP/fuego_test.sh
> > > @@ -188,7 +188,6 @@ function test_pre_check {
> > > assert_define AR
> > > assert_define RANLIB
> > > #assert_define LDFLAGS
> > > - assert_define FUNCTIONAL_LTP_TESTS
> > >
> > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream
> > > echo "Tests skipped by default in Fuego for now"
> > > @@ -453,25 +452,32 @@ function test_run {
> > > # on the type of the test (regular, posize, or realtime)
> > > # Separate the test list by type, and pass the tests in different
> > > # variables
> > > - for a in $FUNCTIONAL_LTP_TESTS; do
> > > - for b in $ALLTESTS; do
> > > - if [ "$a" == "$b" ]; then
> > > - TESTS+="$a "
> > > - fi
> > > - done
> > > -
> > > - for b in $ALLPTSTESTS; do
> > > - if [ "$a" == "$b" ]; then
> > > - PTSTESTS+="$a "
> > > - fi
> > > - done
> > >
> > > - for b in $ALLRTTESTS; do
> > > - if [ "$a" == "$b" ]; then
> > > - RTTESTS+="$a "
> > > - fi
> > > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then
> > > + for a in $FUNCTIONAL_LTP_TESTS; do
> > > + for b in $ALLTESTS; do
> > > + if [ "$a" == "$b" ]; then
> > > + TESTS+="$a "
> > > + fi
> > > + done
> > > +
> > > + for b in $ALLPTSTESTS; do
> > > + if [ "$a" == "$b" ]; then
> > > + PTSTESTS+="$a "
> > > + fi
> > > + done
> > > +
> > > + for b in $ALLRTTESTS; do
> > > + if [ "$a" == "$b" ]; then
> > > + RTTESTS+="$a "
> > > + fi
> > > + done
> > > done
> > > - done
> > > + else
> > > + TESTS=$ALLTESTS
> > > + PTSTESTS=$ALLPTSTESTS
> > > + RTTESTS=$ALLRTTESTS
> > > + fi
> > >
> > > # Let some of the tests fail, the information will be in the result xlsx file
> > > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\";
> > > ./ltp_target_run.sh"
> > > --
> > > 2.20.1
> > >
> >
> > I have NOT applied this one. Please change this to check for the string "all"
> > in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this).
> >
> > Thanks,
> > -- Tim
> >
>
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-07-28 22:10 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
2021-07-22 18:28 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla
2021-07-22 18:34 ` Tim.Bird
2021-07-26 8:29 ` huong4.nguyenthi
2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla
2021-07-22 18:49 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla
2021-07-22 18:54 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla
2021-07-22 18:55 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla
2021-07-22 19:03 ` Tim.Bird
2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat
2021-07-27 23:00 ` Tim.Bird
2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat
2021-07-28 22:10 ` Tim.Bird
2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla
2021-07-22 20:18 ` Tim.Bird
2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.