* [Fuego] [fuego-core 0/7] upstreaming work
@ 2021-07-18 11:10 venkata.pyla
2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla
` (7 more replies)
0 siblings, 8 replies; 21+ messages in thread
From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw)
To: tim.bird; +Cc: binh1.tranhai, fuego, huong4.nguyenthi
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
Hi Tim,
Below are some patches that we want to upstream our work from long time,
the below patches fixes some issues in ftc and LTP functional test
Daniel Sangorrin (1):
ftc: docker ps will not work on the local board
Nguyen Dat Tho (2):
LTP: execute all tests when spec specifies "all"
LTP: automatically obtain the list of tests
Tran Hai Binh (1):
LTP: Add the new tests and skip non-installed tests
nguyen thi huong (2):
ftc: fix test process is not killed when fuego test times out
LTP: fix fuego test could not skip list of test cases
yoshida toshiko (1):
testplan_smoketest.json: ftc fails to add job for tesplan_smoketest
overlays/testplans/testplan_smoketest.json | 2 +-
scripts/ftc | 19 +++--
tests/Functional.LTP/fuego_test.sh | 91 +++++++---------------
tests/Functional.LTP/ltp_target_run.sh | 23 +++---
4 files changed, 56 insertions(+), 79 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 21+ messages in thread* [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 18:28 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla ` (6 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: fuego From: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> When running fuego directly on the local board, instead of using a host-target approach, the check for a fuego container caused an error. Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- scripts/ftc | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/scripts/ftc b/scripts/ftc index 4430dd2..2a6a640 100755 --- a/scripts/ftc +++ b/scripts/ftc @@ -5153,11 +5153,15 @@ def get_running_fuego_container_name(): return cached_container_name # return the first container with "fuego" in the image or container name - dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n') - for line in dps_lines: - if "fuego" in line: - cached_container_name = line.strip().split(" ")[-1] - break + try: + with open(os.devnull, 'w') as devnull: + dps_lines = subprocess.check_output("sudo docker ps", shell=True, stderr=devnull).split('\n') + for line in dps_lines: + if "fuego" in line: + cached_container_name = line.strip().split(" ")[-1] + break + except subprocess.CalledProcessError: + return None return cached_container_name -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board 2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla @ 2021-07-22 18:28 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-22 18:28 UTC (permalink / raw) To: venkata.pyla; +Cc: fuego Looks good. Applied. Thanks! -- Tim > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > > When running fuego directly on the local board, instead of > using a host-target approach, the check for a fuego > container caused an error. > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > scripts/ftc | 14 +++++++++----- > 1 file changed, 9 insertions(+), 5 deletions(-) > > diff --git a/scripts/ftc b/scripts/ftc > index 4430dd2..2a6a640 100755 > --- a/scripts/ftc > +++ b/scripts/ftc > @@ -5153,11 +5153,15 @@ def get_running_fuego_container_name(): > return cached_container_name > > # return the first container with "fuego" in the image or container name > - dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n') > - for line in dps_lines: > - if "fuego" in line: > - cached_container_name = line.strip().split(" ")[-1] > - break > + try: > + with open(os.devnull, 'w') as devnull: I was not familiar with this idiom for executing a subprocess with 2>/dev/null. It's interesting... > + dps_lines = subprocess.check_output("sudo docker ps", shell=True, stderr=devnull).split('\n') > + for line in dps_lines: > + if "fuego" in line: > + cached_container_name = line.strip().split(" ")[-1] > + break > + except subprocess.CalledProcessError: > + return None > > return cached_container_name > > -- > 2.20.1 > ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla 2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 18:34 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla ` (5 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: nguyen thi huong, fuego From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> - Currently, fuego kill main process and does not kill test process when test times out In that case, test process still runs but fuego test result is not available - Revise implementation to kill all test processes and printout the result Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- scripts/ftc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/scripts/ftc b/scripts/ftc index 2a6a640..5c300d0 100755 --- a/scripts/ftc +++ b/scripts/ftc @@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout): dprint("ftc_exec_command: command=%s" % command) - p = subprocess.Popen(command.split(), stdout=log, stderr=log) + p = subprocess.Popen(command.split(), stdout=log, stderr=log, preexec_fn=os.setpgrp) # specify timeout for command operation signal.signal(signal.SIGALRM, alarm_handler) @@ -3960,7 +3960,8 @@ def ftc_exec_command(command, timeout): # p.kill() # abort with prejudice... - p.kill() + pgrp = os.getpgid(p.pid) + os.killpg(pgrp, signal.SIGALRM) timed_out = True finally: -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out 2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla @ 2021-07-22 18:34 ` Tim.Bird 2021-07-26 8:29 ` huong4.nguyenthi 0 siblings, 1 reply; 21+ messages in thread From: Tim.Bird @ 2021-07-22 18:34 UTC (permalink / raw) To: venkata.pyla; +Cc: huong4.nguyenthi, fuego OK - I have a question on this one. > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > > - Currently, fuego kill main process and does not kill test process when test times out > In that case, test process still runs but fuego test result is not available > - Revise implementation to kill all test processes and printout the result > > Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > scripts/ftc | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/scripts/ftc b/scripts/ftc > index 2a6a640..5c300d0 100755 > --- a/scripts/ftc > +++ b/scripts/ftc > @@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout): > > dprint("ftc_exec_command: command=%s" % command) > > - p = subprocess.Popen(command.split(), stdout=log, stderr=log) > + p = subprocess.Popen(command.split(), stdout=log, stderr=log, preexec_fn=os.setpgrp) > > # specify timeout for command operation > signal.signal(signal.SIGALRM, alarm_handler) > @@ -3960,7 +3960,8 @@ def ftc_exec_command(command, timeout): > # p.kill() > > # abort with prejudice... > - p.kill() > + pgrp = os.getpgid(p.pid) > + os.killpg(pgrp, signal.SIGALRM) Why are you sending a SIGALARM instead of a SIGKILL? This is a weaker signal, that can be ignored or masked. > timed_out = True > > finally: > -- > 2.20.1 > I've applied this patch, but I'd like to know if you have some reason for weakening the signal sent to kill the process. Thanks. -- Tim ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out 2021-07-22 18:34 ` Tim.Bird @ 2021-07-26 8:29 ` huong4.nguyenthi 0 siblings, 0 replies; 21+ messages in thread From: huong4.nguyenthi @ 2021-07-26 8:29 UTC (permalink / raw) To: Tim.Bird; +Cc: fuego, TSDVGroup1skerlet Dear Tim, Thanks for your feedback. The function handle timeout event so I think send SIGALARM is more reasonable. Test driver or subprocesses will decide what to do with timeout signal. For now, SIGALARM is trapped in fuego functions.sh Signal handler will do post_tests and processing before exiting main process When main process exit, all subprocesses that belong to group process will be exited. Best Regards, Huong -----Original Message----- From: Tim.Bird@sony.com <Tim.Bird@sony.com> Sent: Friday, July 23, 2021 1:35 AM To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> Cc: nguyen thi huong(ＴＳＤＶ Ｅｎｇ １) <huong4.nguyenthi@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org Subject: RE: [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out OK - I have a question on this one. > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > > - Currently, fuego kill main process and does not kill test process when test times out > In that case, test process still runs but fuego test result is not > available > - Revise implementation to kill all test processes and printout the > result > > Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > scripts/ftc | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/scripts/ftc b/scripts/ftc index 2a6a640..5c300d0 100755 > --- a/scripts/ftc > +++ b/scripts/ftc > @@ -3914,7 +3914,7 @@ def ftc_exec_command(command, timeout): > > dprint("ftc_exec_command: command=%s" % command) > > - p = subprocess.Popen(command.split(), stdout=log, stderr=log) > + p = subprocess.Popen(command.split(), stdout=log, stderr=log, > + preexec_fn=os.setpgrp) > > # specify timeout for command operation > signal.signal(signal.SIGALRM, alarm_handler) @@ -3960,7 +3960,8 > @@ def ftc_exec_command(command, timeout): > # p.kill() > > # abort with prejudice... > - p.kill() > + pgrp = os.getpgid(p.pid) > + os.killpg(pgrp, signal.SIGALRM) Why are you sending a SIGALARM instead of a SIGKILL? This is a weaker signal, that can be ignored or masked. > timed_out = True > > finally: > -- > 2.20.1 > I've applied this patch, but I'd like to know if you have some reason for weakening the signal sent to kill the process. Thanks. -- Tim ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla 2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla 2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 18:49 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla ` (4 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: fuego From: yoshida toshiko <toshiko.yoshida@toshiba.co.jp> add missing comma after Benchmark.dbench4 e.g: ftc add-jobs -b local -p testplan_smoketest Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- overlays/testplans/testplan_smoketest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overlays/testplans/testplan_smoketest.json b/overlays/testplans/testplan_smoketest.json index 8c856c1..4bbbe4f 100644 --- a/overlays/testplans/testplan_smoketest.json +++ b/overlays/testplans/testplan_smoketest.json @@ -12,7 +12,7 @@ "testName": "Benchmark.Dhrystone" }, { - "testName": "Benchmark.dbench4" + "testName": "Benchmark.dbench4", "timeout": "40m" }, { -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest 2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla @ 2021-07-22 18:49 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-22 18:49 UTC (permalink / raw) To: venkata.pyla; +Cc: fuego > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: yoshida toshiko <toshiko.yoshida@toshiba.co.jp> > > add missing comma after Benchmark.dbench4 > > e.g: ftc add-jobs -b local -p testplan_smoketest > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > overlays/testplans/testplan_smoketest.json | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/overlays/testplans/testplan_smoketest.json b/overlays/testplans/testplan_smoketest.json > index 8c856c1..4bbbe4f 100644 > --- a/overlays/testplans/testplan_smoketest.json > +++ b/overlays/testplans/testplan_smoketest.json > @@ -12,7 +12,7 @@ > "testName": "Benchmark.Dhrystone" > }, > { > - "testName": "Benchmark.dbench4" > + "testName": "Benchmark.dbench4", > "timeout": "40m" > }, > { > -- > 2.20.1 > FYI - the standalone testplans are now deprecated. The new method of doing this is with a Functional.batch job, like so: $ ftc add-jobs -b local -t batch_smoketest The 'batch' jobs have a testplan embedded in them (in json format as a HERE document in the fuego_test.sh script for the test. However, standalone testplans are still supported for backwards compatibility, and this *is* a bug, so I have applied this. Thanks! -- Tim ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla ` (2 preceding siblings ...) 2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 18:54 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla ` (3 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: Tran Hai Binh, fuego From: Tran Hai Binh <binh1.tranhai@toshiba.co.jp> - Add the new tests in 20200930 that are not in ALLTESTS: s390x_tests, uevent, crypto - Skip non-installed tests in LTP Signed-off-by: Tran Hai Binh <binh1.tranhai@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 6 +++--- tests/Functional.LTP/ltp_target_run.sh | 23 ++++++++++++++--------- 2 files changed, 17 insertions(+), 12 deletions(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index a4f551b..6a84d1c 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -18,9 +18,9 @@ cve io_floppy net.ipv6_lib net.tirpc_tests dio ipc net.multicast network_commands timers dma_thread_diotest kernel_misc net.nfs nptl tpm_tools fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing -filecaps ltp-aiodio.part2 net.rpc_tests pipes -fs ltp-aiodio.part3 net.sctp power_management_tests -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive +filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests +fs ltp-aiodio.part3 net.sctp power_management_tests uevent +fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty smoketest" diff --git a/tests/Functional.LTP/ltp_target_run.sh b/tests/Functional.LTP/ltp_target_run.sh index 52a006a..ded81de 100755 --- a/tests/Functional.LTP/ltp_target_run.sh +++ b/tests/Functional.LTP/ltp_target_run.sh @@ -5,6 +5,7 @@ OUTPUT_DIR=${PWD}/result TMP_DIR=${PWD}/tmp +RUNTEST_DIR=${PWD}/runtest [ -d ${TMP_DIR} ] && rm -rf ${TMP_DIR} mkdir -p ${TMP_DIR} @@ -18,15 +19,19 @@ echo "ltp_target_run: ${TESTS} | ${PTSTESTS} | ${RTTESTS}" # FIXTHIS: add -t option for limiting the duration of each test group execution for i in ${TESTS}; do - echo "ltp_target_run: doing test $i" - mkdir -p ${OUTPUT_DIR}/${i} - ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \ - -l ${OUTPUT_DIR}/${i}/result.log \ - -o ${OUTPUT_DIR}/${i}/output.log \ - -d ${TMP_DIR} \ - -S ./skiplist.txt \ - -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1 - rm -rf ${TMP_DIR}/* + if [ -f ${RUNTEST_DIR}/${i} ]; then + echo "ltp_target_run: doing test $i" + mkdir -p ${OUTPUT_DIR}/${i} + ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \ + -l ${OUTPUT_DIR}/${i}/result.log \ + -o ${OUTPUT_DIR}/${i}/output.log \ + -d ${TMP_DIR} \ + -S ./skiplist.txt \ + -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1 + rm -rf ${TMP_DIR}/* + else + echo "Warning: ${i} is not available in the installed LTP" + fi done # gather posix results into pts.log -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests 2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla @ 2021-07-22 18:54 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-22 18:54 UTC (permalink / raw) To: venkata.pyla; +Cc: binh1.tranhai, fuego Kind of weird to add these, and then remove them in patch 7, but OK. It makes the patch series application easier, but you guys could have done a git rebase and eliminated this one. Applied. -- Tim > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: Tran Hai Binh <binh1.tranhai@toshiba.co.jp> > > - Add the new tests in 20200930 that are not in ALLTESTS: > s390x_tests, uevent, crypto > - Skip non-installed tests in LTP > > Signed-off-by: Tran Hai Binh <binh1.tranhai@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > tests/Functional.LTP/fuego_test.sh | 6 +++--- > tests/Functional.LTP/ltp_target_run.sh | 23 ++++++++++++++--------- > 2 files changed, 17 insertions(+), 12 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index a4f551b..6a84d1c 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -18,9 +18,9 @@ cve io_floppy net.ipv6_lib net.tirpc_tests > dio ipc net.multicast network_commands timers > dma_thread_diotest kernel_misc net.nfs nptl tpm_tools > fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing > -filecaps ltp-aiodio.part2 net.rpc_tests pipes > -fs ltp-aiodio.part3 net.sctp power_management_tests > -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive > +filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests > +fs ltp-aiodio.part3 net.sctp power_management_tests uevent > +fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto > fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty > smoketest" > > diff --git a/tests/Functional.LTP/ltp_target_run.sh b/tests/Functional.LTP/ltp_target_run.sh > index 52a006a..ded81de 100755 > --- a/tests/Functional.LTP/ltp_target_run.sh > +++ b/tests/Functional.LTP/ltp_target_run.sh > @@ -5,6 +5,7 @@ > > OUTPUT_DIR=${PWD}/result > TMP_DIR=${PWD}/tmp > +RUNTEST_DIR=${PWD}/runtest > > [ -d ${TMP_DIR} ] && rm -rf ${TMP_DIR} > mkdir -p ${TMP_DIR} > @@ -18,15 +19,19 @@ echo "ltp_target_run: ${TESTS} | ${PTSTESTS} | ${RTTESTS}" > > # FIXTHIS: add -t option for limiting the duration of each test group execution > for i in ${TESTS}; do > - echo "ltp_target_run: doing test $i" > - mkdir -p ${OUTPUT_DIR}/${i} > - ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \ > - -l ${OUTPUT_DIR}/${i}/result.log \ > - -o ${OUTPUT_DIR}/${i}/output.log \ > - -d ${TMP_DIR} \ > - -S ./skiplist.txt \ > - -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1 > - rm -rf ${TMP_DIR}/* > + if [ -f ${RUNTEST_DIR}/${i} ]; then > + echo "ltp_target_run: doing test $i" > + mkdir -p ${OUTPUT_DIR}/${i} > + ./runltp -C ${OUTPUT_DIR}/${i}/failed.log \ > + -l ${OUTPUT_DIR}/${i}/result.log \ > + -o ${OUTPUT_DIR}/${i}/output.log \ > + -d ${TMP_DIR} \ > + -S ./skiplist.txt \ > + -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1 > + rm -rf ${TMP_DIR}/* > + else > + echo "Warning: ${i} is not available in the installed LTP" > + fi > done > > # gather posix results into pts.log > -- > 2.20.1 > ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla ` (3 preceding siblings ...) 2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 18:55 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla ` (2 subsequent siblings) 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: nguyen thi huong, fuego From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> Currently It can skip one test case or one file that definied in 'skiplist' Result of skipped test cases are marked as CONF ('stat=32') However it could not skip a list of LTP test cases Correct the syntax of FUEGO_LTP_SKIPLIST variable to parse all items of 'skiplist' Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index 6a84d1c..e620b17 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -380,7 +380,7 @@ function test_deploy { # to skipfiles (text files containing a list of LTP test case names) # usually located under /fuego-rw/boards/ if [ -n "${FUNCTIONAL_LTP_SKIPLIST}" ]; then - for item in "${FUNCTIONAL_LTP_SKIPLIST}"; do + for item in ${FUNCTIONAL_LTP_SKIPLIST}; do if [ -f "$item" ]; then cat "$item" >> ${LOGDIR}/skiplist.txt else -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases 2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla @ 2021-07-22 18:55 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-22 18:55 UTC (permalink / raw) To: venkata.pyla; +Cc: huong4.nguyenthi, fuego Nice catch!! Thanks. Applied. -- Tim > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > Sent: Sunday, July 18, 2021 5:10 AM > To: Bird, Tim <Tim.Bird@sony.com> > Cc: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp>; daniel.sangorrin@toshiba.co.jp; fuego@lists.linuxfoundation.org; venkata pyla > <venkata.pyla@toshiba-tsip.com> > Subject: [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases > > From: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > > Currently It can skip one test case or one file that definied in 'skiplist' > Result of skipped test cases are marked as CONF ('stat=32') > However it could not skip a list of LTP test cases > Correct the syntax of FUEGO_LTP_SKIPLIST variable to parse all items of 'skiplist' > > Signed-off-by: nguyen thi huong <huong4.nguyenthi@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > tests/Functional.LTP/fuego_test.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index 6a84d1c..e620b17 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -380,7 +380,7 @@ function test_deploy { > # to skipfiles (text files containing a list of LTP test case names) > # usually located under /fuego-rw/boards/ > if [ -n "${FUNCTIONAL_LTP_SKIPLIST}" ]; then > - for item in "${FUNCTIONAL_LTP_SKIPLIST}"; do > + for item in ${FUNCTIONAL_LTP_SKIPLIST}; do > if [ -f "$item" ]; then > cat "$item" >> ${LOGDIR}/skiplist.txt > else > -- > 2.20.1 > ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla ` (4 preceding siblings ...) 2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 19:03 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla 2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: fuego, Nguyen Dat Tho From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index e620b17..16d556f 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -188,7 +188,6 @@ function test_pre_check { assert_define AR assert_define RANLIB #assert_define LDFLAGS - assert_define FUNCTIONAL_LTP_TESTS # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream echo "Tests skipped by default in Fuego for now" @@ -453,25 +452,32 @@ function test_run { # on the type of the test (regular, posize, or realtime) # Separate the test list by type, and pass the tests in different # variables - for a in $FUNCTIONAL_LTP_TESTS; do - for b in $ALLTESTS; do - if [ "$a" == "$b" ]; then - TESTS+="$a " - fi - done - - for b in $ALLPTSTESTS; do - if [ "$a" == "$b" ]; then - PTSTESTS+="$a " - fi - done - for b in $ALLRTTESTS; do - if [ "$a" == "$b" ]; then - RTTESTS+="$a " - fi + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then + for a in $FUNCTIONAL_LTP_TESTS; do + for b in $ALLTESTS; do + if [ "$a" == "$b" ]; then + TESTS+="$a " + fi + done + + for b in $ALLPTSTESTS; do + if [ "$a" == "$b" ]; then + PTSTESTS+="$a " + fi + done + + for b in $ALLRTTESTS; do + if [ "$a" == "$b" ]; then + RTTESTS+="$a " + fi + done done - done + else + TESTS=$ALLTESTS + PTSTESTS=$ALLPTSTESTS + RTTESTS=$ALLRTTESTS + fi # Let some of the tests fail, the information will be in the result xlsx file report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh" -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla @ 2021-07-22 19:03 ` Tim.Bird 2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat 0 siblings, 1 reply; 21+ messages in thread From: Tim.Bird @ 2021-07-22 19:03 UTC (permalink / raw) To: venkata.pyla; +Cc: fuego, tho1.nguyendat This one has problems. > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > I'm not sure that the summary line of this change matches what it does. From reading the code, it appears that "tests" in the spec needs to be missing or empty, in order to activate this. If "tests" is "all", then it appears that none of the tests will match, and you'd just end up with empty lists. Also, I had to intuit the reasoning here, since there is no description for this. I don't mind the concept of being able to easily run all LTP tests, but it would be nice to use the string "all" to signal this instead of just leaving FUNCTIONAL_LTP_TESTS empty. It should probably go into a comment in the code as well, and maybe in the test.yaml file, so that users and future developers will see it and understand the behaviour. Also, you might want to add a spec to the spec.json file for "all", to show it's usage. > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > 1 file changed, 24 insertions(+), 18 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index e620b17..16d556f 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -188,7 +188,6 @@ function test_pre_check { > assert_define AR > assert_define RANLIB > #assert_define LDFLAGS > - assert_define FUNCTIONAL_LTP_TESTS > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream > echo "Tests skipped by default in Fuego for now" > @@ -453,25 +452,32 @@ function test_run { > # on the type of the test (regular, posize, or realtime) > # Separate the test list by type, and pass the tests in different > # variables > - for a in $FUNCTIONAL_LTP_TESTS; do > - for b in $ALLTESTS; do > - if [ "$a" == "$b" ]; then > - TESTS+="$a " > - fi > - done > - > - for b in $ALLPTSTESTS; do > - if [ "$a" == "$b" ]; then > - PTSTESTS+="$a " > - fi > - done > > - for b in $ALLRTTESTS; do > - if [ "$a" == "$b" ]; then > - RTTESTS+="$a " > - fi > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then > + for a in $FUNCTIONAL_LTP_TESTS; do > + for b in $ALLTESTS; do > + if [ "$a" == "$b" ]; then > + TESTS+="$a " > + fi > + done > + > + for b in $ALLPTSTESTS; do > + if [ "$a" == "$b" ]; then > + PTSTESTS+="$a " > + fi > + done > + > + for b in $ALLRTTESTS; do > + if [ "$a" == "$b" ]; then > + RTTESTS+="$a " > + fi > + done > done > - done > + else > + TESTS=$ALLTESTS > + PTSTESTS=$ALLPTSTESTS > + RTTESTS=$ALLRTTESTS > + fi > > # Let some of the tests fail, the information will be in the result xlsx file > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > ./ltp_target_run.sh" > -- > 2.20.1 > I have NOT applied this one. Please change this to check for the string "all" in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this). Thanks, -- Tim ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-22 19:03 ` Tim.Bird @ 2021-07-27 3:11 ` tho1.nguyendat 2021-07-27 23:00 ` Tim.Bird 0 siblings, 1 reply; 21+ messages in thread From: tho1.nguyendat @ 2021-07-27 3:11 UTC (permalink / raw) To: tim.bird; +Cc: fuego [-- Attachment #1: Type: text/plain, Size: 7700 bytes --] Dear Tim, I would like to send my update. From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- tests/Functional.LTP/spec.json | 6 +++++ 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index 85adabf..689abfe 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -414,25 +414,31 @@ function test_run { # on the type of the test (regular, posize, or realtime) # Separate the test list by type, and pass the tests in different # variables - for a in $FUNCTIONAL_LTP_TESTS; do - for b in $ALLTESTS; do - if [ "$a" == "$b" ]; then - TESTS+="$a " - fi - done - - for b in $ALLPTSTESTS; do - if [ "$a" == "$b" ]; then - PTSTESTS+="$a " - fi - done - - for b in $ALLRTTESTS; do - if [ "$a" == "$b" ]; then - RTTESTS+="$a " - fi + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then + TESTS=$ALLTESTS + PTSTESTS=$ALLPTSTESTS + RTTESTS=$ALLRTTESTS + else + for a in $FUNCTIONAL_LTP_TESTS; do + for b in $ALLTESTS; do + if [ "$a" == "$b" ]; then + TESTS+="$a " + fi + done + + for b in $ALLPTSTESTS; do + if [ "$a" == "$b" ]; then + PTSTESTS+="$a " + fi + done + + for b in $ALLRTTESTS; do + if [ "$a" == "$b" ]; then + RTTESTS+="$a " + fi + done done - done + fi # Let some of the tests fail, the information will be in the result xlsx file report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh" diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json index d30ba4c..53ee115 100644 --- a/tests/Functional.LTP/spec.json +++ b/tests/Functional.LTP/spec.json @@ -1,6 +1,12 @@ { "testName": "Functional.LTP", "specs": { + "all": { + "tests": "all", + "noautoskip": "true", + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"} + }, "default": { "tests": "syscalls SEM", "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, -- 2.20.1 ________________________________ Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> Đã gửi: 23 Tháng Bảy 2021 2:03 SA Đến: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> Cc: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org> Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" This one has problems. > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > I'm not sure that the summary line of this change matches what it does. From reading the code, it appears that "tests" in the spec needs to be missing or empty, in order to activate this. If "tests" is "all", then it appears that none of the tests will match, and you'd just end up with empty lists. Also, I had to intuit the reasoning here, since there is no description for this. I don't mind the concept of being able to easily run all LTP tests, but it would be nice to use the string "all" to signal this instead of just leaving FUNCTIONAL_LTP_TESTS empty. It should probably go into a comment in the code as well, and maybe in the test.yaml file, so that users and future developers will see it and understand the behaviour. Also, you might want to add a spec to the spec.json file for "all", to show it's usage. > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > 1 file changed, 24 insertions(+), 18 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index e620b17..16d556f 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -188,7 +188,6 @@ function test_pre_check { > assert_define AR > assert_define RANLIB > #assert_define LDFLAGS > - assert_define FUNCTIONAL_LTP_TESTS > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream > echo "Tests skipped by default in Fuego for now" > @@ -453,25 +452,32 @@ function test_run { > # on the type of the test (regular, posize, or realtime) > # Separate the test list by type, and pass the tests in different > # variables > - for a in $FUNCTIONAL_LTP_TESTS; do > - for b in $ALLTESTS; do > - if [ "$a" == "$b" ]; then > - TESTS+="$a " > - fi > - done > - > - for b in $ALLPTSTESTS; do > - if [ "$a" == "$b" ]; then > - PTSTESTS+="$a " > - fi > - done > > - for b in $ALLRTTESTS; do > - if [ "$a" == "$b" ]; then > - RTTESTS+="$a " > - fi > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then > + for a in $FUNCTIONAL_LTP_TESTS; do > + for b in $ALLTESTS; do > + if [ "$a" == "$b" ]; then > + TESTS+="$a " > + fi > + done > + > + for b in $ALLPTSTESTS; do > + if [ "$a" == "$b" ]; then > + PTSTESTS+="$a " > + fi > + done > + > + for b in $ALLRTTESTS; do > + if [ "$a" == "$b" ]; then > + RTTESTS+="$a " > + fi > + done > done > - done > + else > + TESTS=$ALLTESTS > + PTSTESTS=$ALLPTSTESTS > + RTTESTS=$ALLRTTESTS > + fi > > # Let some of the tests fail, the information will be in the result xlsx file > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > ./ltp_target_run.sh" > -- > 2.20.1 > I have NOT applied this one. Please change this to check for the string "all" in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this). Thanks, -- Tim [-- Attachment #2: Type: text/html, Size: 17408 bytes --] ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat @ 2021-07-27 23:00 ` Tim.Bird 2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat 0 siblings, 1 reply; 21+ messages in thread From: Tim.Bird @ 2021-07-27 23:00 UTC (permalink / raw) To: tho1.nguyendat; +Cc: fuego The patch looks good. Thanks for the changes. But I can't figure out how to apply it. The patch content is encoded in base64, and when I tried to manually decode it at the command line, I got an error "invalid input". Can you please re-send this patch as an attachment instead of inline in the message body? Something is going wrong with the mail handling of the text. -- Tim > -----Original Message----- > From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp> > Dear Tim, > > I would like to send my update. > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > > --- > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > tests/Functional.LTP/spec.json | 6 +++++ > 2 files changed, 30 insertions(+), 18 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index 85adabf..689abfe 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -414,25 +414,31 @@ function test_run { > # on the type of the test (regular, posize, or realtime) > # Separate the test list by type, and pass the tests in different > # variables > - for a in $FUNCTIONAL_LTP_TESTS; do > - for b in $ALLTESTS; do > - if [ "$a" == "$b" ]; then > - TESTS+="$a " > - fi > - done > - > - for b in $ALLPTSTESTS; do > - if [ "$a" == "$b" ]; then > - PTSTESTS+="$a " > - fi > - done > - > - for b in $ALLRTTESTS; do > - if [ "$a" == "$b" ]; then > - RTTESTS+="$a " > - fi > + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then > + TESTS=$ALLTESTS > + PTSTESTS=$ALLPTSTESTS > + RTTESTS=$ALLRTTESTS > + else > + for a in $FUNCTIONAL_LTP_TESTS; do > + for b in $ALLTESTS; do > + if [ "$a" == "$b" ]; then > + TESTS+="$a " > + fi > + done > + > + for b in $ALLPTSTESTS; do > + if [ "$a" == "$b" ]; then > + PTSTESTS+="$a " > + fi > + done > + > + for b in $ALLRTTESTS; do > + if [ "$a" == "$b" ]; then > + RTTESTS+="$a " > + fi > + done > done > - done > + fi > > # Let some of the tests fail, the information will be in the result xlsx file > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > ./ltp_target_run.sh" > diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json > index d30ba4c..53ee115 100644 > --- a/tests/Functional.LTP/spec.json > +++ b/tests/Functional.LTP/spec.json > @@ -1,6 +1,12 @@ > { > "testName": "Functional.LTP", > "specs": { > + "all": { > + "tests": "all", > + "noautoskip": "true", > + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"} > + }, > "default": { > "tests": "syscalls SEM", > "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > -- > 2.20.1 > > ________________________________ > > Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> > Đã gửi: 23 Tháng Bảy 2021 2:03 SA > Đến: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> > Cc: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) > <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org> > Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > This one has problems. > > > -----Original Message----- > > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > > I'm not sure that the summary line of this change matches what it does. > > From reading the code, it appears that "tests" in the spec needs to be > missing or empty, in order to activate this. If "tests" is "all", > then it appears that none of the tests will match, and you'd just > end up with empty lists. > > Also, I had to intuit the reasoning here, since there is no description > for this. I don't mind the concept of being able to easily run all > LTP tests, but it would be nice to use the string "all" to signal > this instead of just leaving FUNCTIONAL_LTP_TESTS empty. > > It should probably go into a comment in the code as well, and maybe > in the test.yaml file, so that users and future developers will see > it and understand the behaviour. > > Also, you might want to add a spec to the spec.json file for "all", to > show it's usage. > > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > --- > > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > > 1 file changed, 24 insertions(+), 18 deletions(-) > > > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > > index e620b17..16d556f 100755 > > --- a/tests/Functional.LTP/fuego_test.sh > > +++ b/tests/Functional.LTP/fuego_test.sh > > @@ -188,7 +188,6 @@ function test_pre_check { > > assert_define AR > > assert_define RANLIB > > #assert_define LDFLAGS > > - assert_define FUNCTIONAL_LTP_TESTS > > > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream > > echo "Tests skipped by default in Fuego for now" > > @@ -453,25 +452,32 @@ function test_run { > > # on the type of the test (regular, posize, or realtime) > > # Separate the test list by type, and pass the tests in different > > # variables > > - for a in $FUNCTIONAL_LTP_TESTS; do > > - for b in $ALLTESTS; do > > - if [ "$a" == "$b" ]; then > > - TESTS+="$a " > > - fi > > - done > > - > > - for b in $ALLPTSTESTS; do > > - if [ "$a" == "$b" ]; then > > - PTSTESTS+="$a " > > - fi > > - done > > > > - for b in $ALLRTTESTS; do > > - if [ "$a" == "$b" ]; then > > - RTTESTS+="$a " > > - fi > > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then > > + for a in $FUNCTIONAL_LTP_TESTS; do > > + for b in $ALLTESTS; do > > + if [ "$a" == "$b" ]; then > > + TESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLPTSTESTS; do > > + if [ "$a" == "$b" ]; then > > + PTSTESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLRTTESTS; do > > + if [ "$a" == "$b" ]; then > > + RTTESTS+="$a " > > + fi > > + done > > done > > - done > > + else > > + TESTS=$ALLTESTS > > + PTSTESTS=$ALLPTSTESTS > > + RTTESTS=$ALLRTTESTS > > + fi > > > > # Let some of the tests fail, the information will be in the result xlsx file > > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > > ./ltp_target_run.sh" > > -- > > 2.20.1 > > > > I have NOT applied this one. Please change this to check for the string "all" > in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this). > > Thanks, > -- Tim > ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] Trả lời: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-27 23:00 ` Tim.Bird @ 2021-07-28 0:42 ` tho1.nguyendat 2021-07-28 22:10 ` Tim.Bird 0 siblings, 1 reply; 21+ messages in thread From: tho1.nguyendat @ 2021-07-28 0:42 UTC (permalink / raw) To: Tim.Bird; +Cc: fuego [-- Attachment #1.1: Type: text/plain, Size: 9222 bytes --] Hi Tim, I attached the patch file. Could you please try it? Thanks, ________________________________ Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> Đã gửi: 28 Tháng Bảy 2021 6:00 SA Đến: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp> Cc: sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org>; pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> Chủ đề: RE: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" The patch looks good. Thanks for the changes. But I can't figure out how to apply it. The patch content is encoded in base64, and when I tried to manually decode it at the command line, I got an error "invalid input". Can you please re-send this patch as an attachment instead of inline in the message body? Something is going wrong with the mail handling of the text. -- Tim > -----Original Message----- > From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp> > Dear Tim, > > I would like to send my update. > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > > --- > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > tests/Functional.LTP/spec.json | 6 +++++ > 2 files changed, 30 insertions(+), 18 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index 85adabf..689abfe 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -414,25 +414,31 @@ function test_run { > # on the type of the test (regular, posize, or realtime) > # Separate the test list by type, and pass the tests in different > # variables > - for a in $FUNCTIONAL_LTP_TESTS; do > - for b in $ALLTESTS; do > - if [ "$a" == "$b" ]; then > - TESTS+="$a " > - fi > - done > - > - for b in $ALLPTSTESTS; do > - if [ "$a" == "$b" ]; then > - PTSTESTS+="$a " > - fi > - done > - > - for b in $ALLRTTESTS; do > - if [ "$a" == "$b" ]; then > - RTTESTS+="$a " > - fi > + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then > + TESTS=$ALLTESTS > + PTSTESTS=$ALLPTSTESTS > + RTTESTS=$ALLRTTESTS > + else > + for a in $FUNCTIONAL_LTP_TESTS; do > + for b in $ALLTESTS; do > + if [ "$a" == "$b" ]; then > + TESTS+="$a " > + fi > + done > + > + for b in $ALLPTSTESTS; do > + if [ "$a" == "$b" ]; then > + PTSTESTS+="$a " > + fi > + done > + > + for b in $ALLRTTESTS; do > + if [ "$a" == "$b" ]; then > + RTTESTS+="$a " > + fi > + done > done > - done > + fi > > # Let some of the tests fail, the information will be in the result xlsx file > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > ./ltp_target_run.sh" > diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json > index d30ba4c..53ee115 100644 > --- a/tests/Functional.LTP/spec.json > +++ b/tests/Functional.LTP/spec.json > @@ -1,6 +1,12 @@ > { > "testName": "Functional.LTP", > "specs": { > + "all": { > + "tests": "all", > + "noautoskip": "true", > + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"} > + }, > "default": { > "tests": "syscalls SEM", > "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > -- > 2.20.1 > > ________________________________ > > Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> > Đã gửi: 23 Tháng Bảy 2021 2:03 SA > Đến: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> > Cc: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) > <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org> > Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > This one has problems. > > > -----Original Message----- > > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > > I'm not sure that the summary line of this change matches what it does. > > From reading the code, it appears that "tests" in the spec needs to be > missing or empty, in order to activate this. If "tests" is "all", > then it appears that none of the tests will match, and you'd just > end up with empty lists. > > Also, I had to intuit the reasoning here, since there is no description > for this. I don't mind the concept of being able to easily run all > LTP tests, but it would be nice to use the string "all" to signal > this instead of just leaving FUNCTIONAL_LTP_TESTS empty. > > It should probably go into a comment in the code as well, and maybe > in the test.yaml file, so that users and future developers will see > it and understand the behaviour. > > Also, you might want to add a spec to the spec.json file for "all", to > show it's usage. > > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > --- > > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > > 1 file changed, 24 insertions(+), 18 deletions(-) > > > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > > index e620b17..16d556f 100755 > > --- a/tests/Functional.LTP/fuego_test.sh > > +++ b/tests/Functional.LTP/fuego_test.sh > > @@ -188,7 +188,6 @@ function test_pre_check { > > assert_define AR > > assert_define RANLIB > > #assert_define LDFLAGS > > - assert_define FUNCTIONAL_LTP_TESTS > > > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream > > echo "Tests skipped by default in Fuego for now" > > @@ -453,25 +452,32 @@ function test_run { > > # on the type of the test (regular, posize, or realtime) > > # Separate the test list by type, and pass the tests in different > > # variables > > - for a in $FUNCTIONAL_LTP_TESTS; do > > - for b in $ALLTESTS; do > > - if [ "$a" == "$b" ]; then > > - TESTS+="$a " > > - fi > > - done > > - > > - for b in $ALLPTSTESTS; do > > - if [ "$a" == "$b" ]; then > > - PTSTESTS+="$a " > > - fi > > - done > > > > - for b in $ALLRTTESTS; do > > - if [ "$a" == "$b" ]; then > > - RTTESTS+="$a " > > - fi > > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then > > + for a in $FUNCTIONAL_LTP_TESTS; do > > + for b in $ALLTESTS; do > > + if [ "$a" == "$b" ]; then > > + TESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLPTSTESTS; do > > + if [ "$a" == "$b" ]; then > > + PTSTESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLRTTESTS; do > > + if [ "$a" == "$b" ]; then > > + RTTESTS+="$a " > > + fi > > + done > > done > > - done > > + else > > + TESTS=$ALLTESTS > > + PTSTESTS=$ALLPTSTESTS > > + RTTESTS=$ALLRTTESTS > > + fi > > > > # Let some of the tests fail, the information will be in the result xlsx file > > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > > ./ltp_target_run.sh" > > -- > > 2.20.1 > > > > I have NOT applied this one. Please change this to check for the string "all" > in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this). > > Thanks, > -- Tim > [-- Attachment #1.2: Type: text/html, Size: 20093 bytes --] [-- Attachment #2: 0001-LTP-execute-all-tests-when-spec-specifies-all.patch --] [-- Type: application/octet-stream, Size: 3182 bytes --] From b0077de9fa9ba8d1ad18b282ba7d3ec6ece6532f Mon Sep 17 00:00:00 2001 From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Date: Tue, 27 Jul 2021 08:39:42 +0700 Subject: [PATCH] LTP: execute all tests when spec specifies "all" Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- tests/Functional.LTP/spec.json | 6 +++++ 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index 85adabf..689abfe 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -414,25 +414,31 @@ function test_run { # on the type of the test (regular, posize, or realtime) # Separate the test list by type, and pass the tests in different # variables - for a in $FUNCTIONAL_LTP_TESTS; do - for b in $ALLTESTS; do - if [ "$a" == "$b" ]; then - TESTS+="$a " - fi - done - - for b in $ALLPTSTESTS; do - if [ "$a" == "$b" ]; then - PTSTESTS+="$a " - fi - done - - for b in $ALLRTTESTS; do - if [ "$a" == "$b" ]; then - RTTESTS+="$a " - fi + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then + TESTS=$ALLTESTS + PTSTESTS=$ALLPTSTESTS + RTTESTS=$ALLRTTESTS + else + for a in $FUNCTIONAL_LTP_TESTS; do + for b in $ALLTESTS; do + if [ "$a" == "$b" ]; then + TESTS+="$a " + fi + done + + for b in $ALLPTSTESTS; do + if [ "$a" == "$b" ]; then + PTSTESTS+="$a " + fi + done + + for b in $ALLRTTESTS; do + if [ "$a" == "$b" ]; then + RTTESTS+="$a " + fi + done done - done + fi # Let some of the tests fail, the information will be in the result xlsx file report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; ./ltp_target_run.sh" diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json index d30ba4c..53ee115 100644 --- a/tests/Functional.LTP/spec.json +++ b/tests/Functional.LTP/spec.json @@ -1,6 +1,12 @@ { "testName": "Functional.LTP", "specs": { + "all": { + "tests": "all", + "noautoskip": "true", + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"} + }, "default": { "tests": "syscalls SEM", "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] Trả lời: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" 2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat @ 2021-07-28 22:10 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-28 22:10 UTC (permalink / raw) To: tho1.nguyendat; +Cc: fuego > -----Original Message----- > From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp> > > Hi Tim, > > I attached the patch file. > Could you please try it? This worked. This patch is applied and pushed. Can you please pull the latest master branch, and test this out in your lab? Thanks, -- Tim > ________________________________ > > Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> > Đã gửi: 28 Tháng Bảy 2021 6:00 SA > Đến: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp> > Cc: sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org > <fuego@lists.linuxfoundation.org>; pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> > Chủ đề: RE: Trả lời: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > The patch looks good. Thanks for the changes. > But I can't figure out how to apply it. > > The patch content is encoded in base64, and when I tried to manually > decode it at the command line, I got an error "invalid input". > > Can you please re-send this patch as an attachment instead of inline > in the message body? Something is going wrong with the mail > handling of the text. > -- Tim > > > -----Original Message----- > > From: tho1.nguyendat@toshiba.co.jp <tho1.nguyendat@toshiba.co.jp> > > Dear Tim, > > > > I would like to send my update. > > > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > > > > > --- > > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > > tests/Functional.LTP/spec.json | 6 +++++ > > 2 files changed, 30 insertions(+), 18 deletions(-) > > > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > > index 85adabf..689abfe 100755 > > --- a/tests/Functional.LTP/fuego_test.sh > > +++ b/tests/Functional.LTP/fuego_test.sh > > @@ -414,25 +414,31 @@ function test_run { > > # on the type of the test (regular, posize, or realtime) > > # Separate the test list by type, and pass the tests in different > > # variables > > - for a in $FUNCTIONAL_LTP_TESTS; do > > - for b in $ALLTESTS; do > > - if [ "$a" == "$b" ]; then > > - TESTS+="$a " > > - fi > > - done > > - > > - for b in $ALLPTSTESTS; do > > - if [ "$a" == "$b" ]; then > > - PTSTESTS+="$a " > > - fi > > - done > > - > > - for b in $ALLRTTESTS; do > > - if [ "$a" == "$b" ]; then > > - RTTESTS+="$a " > > - fi > > + if [ "$FUNCTIONAL_LTP_TESTS" == "all" ]; then > > + TESTS=$ALLTESTS > > + PTSTESTS=$ALLPTSTESTS > > + RTTESTS=$ALLRTTESTS > > + else > > + for a in $FUNCTIONAL_LTP_TESTS; do > > + for b in $ALLTESTS; do > > + if [ "$a" == "$b" ]; then > > + TESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLPTSTESTS; do > > + if [ "$a" == "$b" ]; then > > + PTSTESTS+="$a " > > + fi > > + done > > + > > + for b in $ALLRTTESTS; do > > + if [ "$a" == "$b" ]; then > > + RTTESTS+="$a " > > + fi > > + done > > done > > - done > > + fi > > > > # Let some of the tests fail, the information will be in the result xlsx file > > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > > ./ltp_target_run.sh" > > diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json > > index d30ba4c..53ee115 100644 > > --- a/tests/Functional.LTP/spec.json > > +++ b/tests/Functional.LTP/spec.json > > @@ -1,6 +1,12 @@ > > { > > "testName": "Functional.LTP", > > "specs": { > > + "all": { > > + "tests": "all", > > + "noautoskip": "true", > > + "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > > + "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"} > > + }, > > "default": { > > "tests": "syscalls SEM", > > "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}, > > -- > > 2.20.1 > > > > ________________________________ > > > > Từ: Tim.Bird@sony.com <Tim.Bird@sony.com> > > Đã gửi: 23 Tháng Bảy 2021 2:03 SA > > Đến: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com> > > Cc: nguyen dat tho(ＴＳＤＶ Ｅｎｇ １) <tho1.nguyendat@toshiba.co.jp>; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) > > <daniel.sangorrin@toshiba.co.jp>; fuego@lists.linuxfoundation.org <fuego@lists.linuxfoundation.org> > > Chủ đề: RE: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > > > This one has problems. > > > > > -----Original Message----- > > > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > > Subject: [fuego-core 6/7] LTP: execute all tests when spec specifies "all" > > > > > I'm not sure that the summary line of this change matches what it does. > > > > From reading the code, it appears that "tests" in the spec needs to be > > missing or empty, in order to activate this. If "tests" is "all", > > then it appears that none of the tests will match, and you'd just > > end up with empty lists. > > > > Also, I had to intuit the reasoning here, since there is no description > > for this. I don't mind the concept of being able to easily run all > > LTP tests, but it would be nice to use the string "all" to signal > > this instead of just leaving FUNCTIONAL_LTP_TESTS empty. > > > > It should probably go into a comment in the code as well, and maybe > > in the test.yaml file, so that users and future developers will see > > it and understand the behaviour. > > > > Also, you might want to add a spec to the spec.json file for "all", to > > show it's usage. > > > > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > > > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > > > --- > > > tests/Functional.LTP/fuego_test.sh | 42 +++++++++++++++++------------- > > > 1 file changed, 24 insertions(+), 18 deletions(-) > > > > > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > > > index e620b17..16d556f 100755 > > > --- a/tests/Functional.LTP/fuego_test.sh > > > +++ b/tests/Functional.LTP/fuego_test.sh > > > @@ -188,7 +188,6 @@ function test_pre_check { > > > assert_define AR > > > assert_define RANLIB > > > #assert_define LDFLAGS > > > - assert_define FUNCTIONAL_LTP_TESTS > > > > > > # FIXTHIS: use regex for selecting tests to skip once merged on LTP upstream > > > echo "Tests skipped by default in Fuego for now" > > > @@ -453,25 +452,32 @@ function test_run { > > > # on the type of the test (regular, posize, or realtime) > > > # Separate the test list by type, and pass the tests in different > > > # variables > > > - for a in $FUNCTIONAL_LTP_TESTS; do > > > - for b in $ALLTESTS; do > > > - if [ "$a" == "$b" ]; then > > > - TESTS+="$a " > > > - fi > > > - done > > > - > > > - for b in $ALLPTSTESTS; do > > > - if [ "$a" == "$b" ]; then > > > - PTSTESTS+="$a " > > > - fi > > > - done > > > > > > - for b in $ALLRTTESTS; do > > > - if [ "$a" == "$b" ]; then > > > - RTTESTS+="$a " > > > - fi > > > + if [ -n "$FUNCTIONAL_LTP_TESTS" ]; then > > > + for a in $FUNCTIONAL_LTP_TESTS; do > > > + for b in $ALLTESTS; do > > > + if [ "$a" == "$b" ]; then > > > + TESTS+="$a " > > > + fi > > > + done > > > + > > > + for b in $ALLPTSTESTS; do > > > + if [ "$a" == "$b" ]; then > > > + PTSTESTS+="$a " > > > + fi > > > + done > > > + > > > + for b in $ALLRTTESTS; do > > > + if [ "$a" == "$b" ]; then > > > + RTTESTS+="$a " > > > + fi > > > + done > > > done > > > - done > > > + else > > > + TESTS=$ALLTESTS > > > + PTSTESTS=$ALLPTSTESTS > > > + RTTESTS=$ALLRTTESTS > > > + fi > > > > > > # Let some of the tests fail, the information will be in the result xlsx file > > > report "cd $LTP_DESTDIR; export TESTS=\"$TESTS\"; export PTSTESTS=\"$PTSTESTS\"; export RTTESTS=\"$RTTESTS\"; > > > ./ltp_target_run.sh" > > > -- > > > 2.20.1 > > > > > > > I have NOT applied this one. Please change this to check for the string "all" > > in FUNCTIONAL_LTP_TESTS (and please submit a sample spec to go with this). > > > > Thanks, > > -- Tim > > > ^ permalink raw reply [flat|nested] 21+ messages in thread
* [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla ` (5 preceding siblings ...) 2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla @ 2021-07-18 11:10 ` venkata.pyla 2021-07-22 20:18 ` Tim.Bird 2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird 7 siblings, 1 reply; 21+ messages in thread From: venkata.pyla @ 2021-07-18 11:10 UTC (permalink / raw) To: tim.bird; +Cc: fuego, Nguyen Dat Tho From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> set ALLTESTS, ALLPTSTESTS and ALLRTTESTS automatically by listing all possible runtests Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> --- tests/Functional.LTP/fuego_test.sh | 47 +++--------------------------- 1 file changed, 4 insertions(+), 43 deletions(-) diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh index 16d556f..23dfc6c 100755 --- a/tests/Functional.LTP/fuego_test.sh +++ b/tests/Functional.LTP/fuego_test.sh @@ -4,49 +4,6 @@ tarball=ltp-full-20210524.tar.bz2 NEED_ROOT=1 -ALLTESTS=" -admin_tools fs_perms_simple ltp-aio-stress.part2 net_stress.interface -can fs_readonly ltplite net_stress.ipsec_dccp sched -cap_bounds fsx lvm.part1 net_stress.ipsec_icmp scsi_debug.part1 -commands hugetlb lvm.part2 net_stress.ipsec_sctp securebits -connectors hyperthreading math net_stress.ipsec_tcp smack -containers ima mm net_stress.ipsec_udp stress.part1 -controllers input modules net_stress.multicast stress.part2 -cpuhotplug io net.features net_stress.route stress.part3 -crashme io_cd net.ipv6 net.tcp_cmds syscalls -cve io_floppy net.ipv6_lib net.tirpc_tests syscalls-ipc -dio ipc net.multicast network_commands timers -dma_thread_diotest kernel_misc net.nfs nptl tpm_tools -fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing -filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests -fs ltp-aiodio.part3 net.sctp power_management_tests uevent -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto -fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty -smoketest" - -ALLPTSTESTS="AIO MEM MSG SEM SIG THR TMR TPS" - -# This list can be obtained by doing ./testscripts/test_realtime.sh -t list -ALLRTTESTS=" -perf/latency -func/measurement -func/hrtimer-prio -func/gtod_latency -func/periodic_cpu_load -func/pthread_kill_latency -func/sched_football -func/pi-tests -func/thread_clock -func/rt-migrate -func/matrix_mult -func/prio-preempt -func/prio-wake -func/pi_perf -func/sched_latency -func/async_handler -func/sched_jitter -" - # OK - the logic here is a bit complicated # We support several different usage scenarios: # 1: fuego builds, deploys and installs LTP @@ -444,6 +401,10 @@ function test_run { cmd "sed -i 's/^dio30 diotest6 -b 65536 -n 100 -i 100 -o 1024000/dio30 diotest6 -b 65536 -n 5 -i 100 -o 1024000/' $LTP_DESTDIR/runtest/dio" cmd "sed -i 's/^msgctl11 msgctl11/msgctl11 msgctl11 -n 5/' $LTP_DESTDIR/runtest/syscalls" + ALLTESTS=$(ls $LTP_DESTDIR/runtest) + ALLPTSTESTS=$(grep 'usage: $(basename "$0")' $LTP_DESTDIR/bin/run-posix-option-group-test.sh | sed -e 's/^.*\[\(.*\)\]/\1/g' -e 's/|/ /g') + ALLRTTESTS=$(find $LTP_DESTDIR/testcases/realtime -type f -name run_auto.sh -printf '%h\n' | sed 's/^.*realtime\///g') + TESTS="" PTSTESTS="" RTTESTS="" -- 2.20.1 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests 2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla @ 2021-07-22 20:18 ` Tim.Bird 0 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-22 20:18 UTC (permalink / raw) To: venkata.pyla; +Cc: fuego, tho1.nguyendat I love this concept, but the implementation had problems. See below. > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > From: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > > set ALLTESTS, ALLPTSTESTS and ALLRTTESTS automatically by > listing all possible runtests > > Signed-off-by: Nguyen Dat Tho <tho1.nguyendat@toshiba.co.jp> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com> > --- > tests/Functional.LTP/fuego_test.sh | 47 +++--------------------------- > 1 file changed, 4 insertions(+), 43 deletions(-) > > diff --git a/tests/Functional.LTP/fuego_test.sh b/tests/Functional.LTP/fuego_test.sh > index 16d556f..23dfc6c 100755 > --- a/tests/Functional.LTP/fuego_test.sh > +++ b/tests/Functional.LTP/fuego_test.sh > @@ -4,49 +4,6 @@ tarball=ltp-full-20210524.tar.bz2 > > NEED_ROOT=1 > > -ALLTESTS=" > -admin_tools fs_perms_simple ltp-aio-stress.part2 net_stress.interface > -can fs_readonly ltplite net_stress.ipsec_dccp sched > -cap_bounds fsx lvm.part1 net_stress.ipsec_icmp scsi_debug.part1 > -commands hugetlb lvm.part2 net_stress.ipsec_sctp securebits > -connectors hyperthreading math net_stress.ipsec_tcp smack > -containers ima mm net_stress.ipsec_udp stress.part1 > -controllers input modules net_stress.multicast stress.part2 > -cpuhotplug io net.features net_stress.route stress.part3 > -crashme io_cd net.ipv6 net.tcp_cmds syscalls > -cve io_floppy net.ipv6_lib net.tirpc_tests syscalls-ipc > -dio ipc net.multicast network_commands timers > -dma_thread_diotest kernel_misc net.nfs nptl tpm_tools > -fcntl-locktests ltp-aiodio.part1 net.rpc numa tracing > -filecaps ltp-aiodio.part2 net.rpc_tests pipes s390x_tests > -fs ltp-aiodio.part3 net.sctp power_management_tests uevent > -fs_bind ltp-aiodio.part4 net_stress.appl power_management_tests_exclusive crypto > -fs_ext4 ltp-aio-stress.part1 net_stress.broken_ip pty > -smoketest" > - > -ALLPTSTESTS="AIO MEM MSG SEM SIG THR TMR TPS" > - > -# This list can be obtained by doing ./testscripts/test_realtime.sh -t list > -ALLRTTESTS=" > -perf/latency > -func/measurement > -func/hrtimer-prio > -func/gtod_latency > -func/periodic_cpu_load > -func/pthread_kill_latency > -func/sched_football > -func/pi-tests > -func/thread_clock > -func/rt-migrate > -func/matrix_mult > -func/prio-preempt > -func/prio-wake > -func/pi_perf > -func/sched_latency > -func/async_handler > -func/sched_jitter > -" > - > # OK - the logic here is a bit complicated > # We support several different usage scenarios: > # 1: fuego builds, deploys and installs LTP > @@ -444,6 +401,10 @@ function test_run { > cmd "sed -i 's/^dio30 diotest6 -b 65536 -n 100 -i 100 -o 1024000/dio30 diotest6 -b 65536 -n 5 -i 100 -o 1024000/' > $LTP_DESTDIR/runtest/dio" > cmd "sed -i 's/^msgctl11 msgctl11/msgctl11 msgctl11 -n 5/' $LTP_DESTDIR/runtest/syscalls" > > + ALLTESTS=$(ls $LTP_DESTDIR/runtest) > + ALLPTSTESTS=$(grep 'usage: $(basename "$0")' $LTP_DESTDIR/bin/run-posix-option-group-test.sh | sed -e 's/^.*\[\(.*\)\]/\1/g' -e > 's/|/ /g') > + ALLRTTESTS=$(find $LTP_DESTDIR/testcases/realtime -type f -name run_auto.sh -printf '%h\n' | sed 's/^.*realtime\///g') > + The above commands only work for a local board. LTP_DESTDIR is on the target. These should use 'cmd ...' to execute the operation on the board, instead of running the command locally. I made this change, and modified the ALLPTSTESTS check slightly, to make it easier to read. > TESTS="" > PTSTESTS="" > RTTESTS="" > -- > 2.20.1 > Thanks. This is applied. I'm going to push these changes to the fuego-core master branch. Can you please pull them and check that they work for you? Thanks. -- Tim ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Fuego] [fuego-core 0/7] upstreaming work 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla ` (6 preceding siblings ...) 2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla @ 2021-07-20 20:58 ` Tim.Bird 7 siblings, 0 replies; 21+ messages in thread From: Tim.Bird @ 2021-07-20 20:58 UTC (permalink / raw) To: venkata.pyla; +Cc: huong4.nguyenthi, binh1.tranhai, fuego Hey Venkata, Thanks very much for these patches. I looked at them quickly, and they look pretty good. I'll do a more thorough review and try to get them applied later this week. Some of these items are quite nice. I'll let you know when they are applied. Thanks! -- Tim > -----Original Message----- > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com> > > Hi Tim, > > Below are some patches that we want to upstream our work from long time, > the below patches fixes some issues in ftc and LTP functional test > > Daniel Sangorrin (1): > ftc: docker ps will not work on the local board > > Nguyen Dat Tho (2): > LTP: execute all tests when spec specifies "all" > LTP: automatically obtain the list of tests > > Tran Hai Binh (1): > LTP: Add the new tests and skip non-installed tests > > nguyen thi huong (2): > ftc: fix test process is not killed when fuego test times out > LTP: fix fuego test could not skip list of test cases > > yoshida toshiko (1): > testplan_smoketest.json: ftc fails to add job for tesplan_smoketest > > overlays/testplans/testplan_smoketest.json | 2 +- > scripts/ftc | 19 +++-- > tests/Functional.LTP/fuego_test.sh | 91 +++++++--------------- > tests/Functional.LTP/ltp_target_run.sh | 23 +++--- > 4 files changed, 56 insertions(+), 79 deletions(-) > > -- > 2.20.1 > ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-07-28 22:10 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-07-18 11:10 [Fuego] [fuego-core 0/7] upstreaming work venkata.pyla 2021-07-18 11:10 ` [Fuego] [fuego-core 1/7] ftc: docker ps will not work on the local board venkata.pyla 2021-07-22 18:28 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 2/7] ftc: fix test process is not killed when fuego test times out venkata.pyla 2021-07-22 18:34 ` Tim.Bird 2021-07-26 8:29 ` huong4.nguyenthi 2021-07-18 11:10 ` [Fuego] [fuego-core 3/7] testplan_smoketest.json: ftc fails to add job for tesplan_smoketest venkata.pyla 2021-07-22 18:49 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 4/7] LTP: Add the new tests and skip non-installed tests venkata.pyla 2021-07-22 18:54 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 5/7] LTP: fix fuego test could not skip list of test cases venkata.pyla 2021-07-22 18:55 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 6/7] LTP: execute all tests when spec specifies "all" venkata.pyla 2021-07-22 19:03 ` Tim.Bird 2021-07-27 3:11 ` [Fuego] Trả lời: " tho1.nguyendat 2021-07-27 23:00 ` Tim.Bird 2021-07-28 0:42 ` [Fuego] Trả lời: " tho1.nguyendat 2021-07-28 22:10 ` Tim.Bird 2021-07-18 11:10 ` [Fuego] [fuego-core 7/7] LTP: automatically obtain the list of tests venkata.pyla 2021-07-22 20:18 ` Tim.Bird 2021-07-20 20:58 ` [Fuego] [fuego-core 0/7] upstreaming work Tim.Bird
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.